Plan to refactor webserver and separate a gateway service for the web-api

[pcrespov]

Provide web-api a dedicated gateway service to steer the traffic with the front-end. Same as the api-server acts as a gateway for the public-api.

For the sake of discussion we will assume the webserver is split in an web-api gateway service (simcore_service_web_api) and a core service (simcore_service_core) which holds all the domains currently in the webserver (e.g. projects, users, etc). Other names or further subdivisions can be considered later.

Some ideas to consider.

1. convert app from aiohttp to fastapi
   • gateway could keep aiohttp since e.g. auth, sessions etc is also based on that
2. convert clients from aiohttp to httpx
   • convenient mostly for respx mocking
3. well-defined service concept by @matusdrobuliak66:
   • single service (i.e. multiple replicas) that act as both master and workers.
   • To achieve this, selected interfaces (i.e. functions) are marked and when executed they are scheduled in a queue and executed by any worker available.
   • This is partially implemented by dereference concept by @GitHK but I believe its possible to have a simplified interface e.g. based on decorators.

[sanderegg]

@pcrespov may I ask the goal of this change? what is the point of doing this?

[GitHK]

From my point of view, I would see more benefit if we would abstract away all the DB access in a single service which all other services interface with.
This means that services like: catalog, storage, director, director-v2, dynamic-schduler, api-server and webserver will directly interface with this service for DB calls.

[bisgaard-itis]

I can definitely see the point of this change and thanks for taking the initiative. I suggest to think carefully about the requirements (and would be happy to contribute more to that discussion/work). What I have in mind is that on the one hand we have a beautiful ui in the frontend which we want to be responsive. Achieving that requires certain endpoints in the backend to respond very quickly. On the other hand we have an increasing number of endpoints which do not execute quickly (the famous long running tasks). There might be an advantage in separating the fast and slow endpoints so that they are running on different instances. They could be built into the same service, but such that "slow endpoints" are not scheduled in the same event loop as the "fast endpoints". This would allow one to establish better guarantees that the fast endpoints will actually respond within a given amount of time (e.g. 100ms). For this to work we should probably define precisely how fast an endpoint has to respond for it to be "fast".

One way to do that would be to separate master and worker and make sure master is only handling very fast endpoints - everything else is scheduled for the workers.

Please note that this requirement is not something I am cooking up in my head. This is a real issue we have seen, e.g. when doing meta-modelling. In that case what we observed is that a single user can essentially block out everyone else by scheduling a bunch of long running tasks so that other user's frontend doesn't receive responses from the backend.

[matusdrobuliak66]

Cool! Thanks for taking the initiative. I definitely see the benefits of having a decoupled core service and API Gateway. This will give us the flexibility to address potential issues (just as you did with the web server purely for the API server).

However, I’d like to challenge whether this should be our next priority. Personally, I believe one of the biggest gaps we have is observability, the goal of which should be to ensure we are notified if there is a major issue with the platform. And second big topic is restarbility, which goal is to be able to redeploy our platform anytime.

Anyway all of them are important :) so I agree we should have for sure a "core service" and as Andrei mentioned, would be nice if all other services talk to it via RPC interface.