← Blog
Product5 min read

Scale your project's API across servers, with Metis running the servers and instances

Your SnoutData Cloud project's API can now run on more than one server. Scale it to two or three instances, and Metis runs the servers and instances behind it: it starts a server when one is needed, places your instances and keeps them running, moves them to fill servers, and drains and retires a server it no longer needs, without your project ever having fewer instances than you asked for. The routing your calls follow is one of its outputs.

What Scale gives you

Scale, on a project's Scaling page, gives your app's API (auth, the data API and the router) more instances: 1 by default, up to 2 on Pro and 3 on Business, in your project's region, and every instance takes its share of the calls.

It is about the API in front of your database, not the database: your database stays on its own server, kept as its durability setting says, and functions, realtime and storage keep running beside it. Scale is capacity, not failover. Keeping your project running when a server is lost is our job on every plan, whatever its instances.

How a call finds an instance

How a call finds an instance, inside the Metis engineMetis is drawn as a boundary around the router and the servers: it starts a server when one is needed, places instances and keeps them running, moves them, and drains and retires servers. It pushes routing to a router in the project's region. A call to the project's API address reaches that router, which chooses an instance for each call: instance 1 on the project's own server beside its database, or instance 2 or 3, each an API Stack Copy. Instances 2 and 3 use the database on the project's server. An agent on each server runs what Metis placed there, restarts what stops and reports what runs.https://<ref>.api.snoutdata.coma call from your app, a browser, a webhookMetis enginestarts a server when one is needed; places instances and keeps them running;moves them, drains and retires serverspushes routingA router in your regionchooses an instance for each call; a call one refuses goes to anotherYour project's serveryour databaseinstance 1API Stack Copyinstance 2API Stack Copyinstance 3instances 2 and 3 use your database, on your project's serverAn agent on each server runs what Metis placed there, restarts what stops, reports what runs

Every call to your project's API address reaches one of your region's routers. Each router holds a table of where every instance runs, and it chooses an instance for each call. When an instance refuses a call, the router sends it to another instance of the same project.

That routing comes from Metis: it is where Metis has placed your instances, and it only ever travels one way: Metis pushes it to the routers, and a router never asks Metis for it. So Metis is never in a call's path. A call goes from the router to an instance.

Where instances run, and what keeps them running

  • Three members, by majority. Three Metis members run in three separate locations, and a decision needs a majority of them to agree on a leader.
  • An agent on every server. It runs the instances Metis places on its server, starts again any that stop, and reports what runs. Your project's own server, where its database runs, runs one share of its API, instance 1, and reports it the same way.
  • A new server when one is needed. When no server has room for the next instance, one is prepared for it, which takes about ten minutes, and your project goes on working on the instances it has meanwhile. The Scaling page says so while it happens.
  • Never the database. Metis places the services in front of your database. The database is a layer below it, and Metis does not touch it.

Moving an instance without dropping one

We sometimes move an instance to another server, to fill servers rather than leave them half empty. A move makes the new instance before it takes the old one away: the new instance has to run and answer for 30 seconds, and only then is the old one taken off. So while an instance moves, your project never has fewer instances than you scaled it to.

A server emptied this way is retired automatically. Every move shows on your project's audit log, so you can see when one of your instances changed servers.

What we measured

Before calling it done we ran two drills on throwaway projects in us-west-2, and then scaled a project on the live service:

  • Through one router: 40 of 40 calls answered, spread across two servers (90 and 91 packets). The data API, auth, storage (64 MB up and down, byte for byte), a function, containers and a realtime websocket held for 20 seconds all answered through it.
  • An instance stopped while 40 calls ran: 40 of 40 answered, the router retrying on the other instance. A server made silent to the router under load: 20 of 20 answered, the slowest in 2.746 seconds.
  • The router's extra hop within the region, 50 calls each way: a median of 2 ms through the router and 2 ms straight to the server, and a p95 of 4 ms against 7 ms.
  • Packing: with two projects on two half-full servers, after an hour the emptier server was drained. Its instance moved with two instances running at once and never fewer than one, and the emptied server was retired at once.
  • On the live service: a project on a Business plan, scaled to 2 through the same call the dashboard makes, got its second instance on a newly prepared server. Both live routers routed it to both instances, and 40 of 40 calls from the internet answered 200.

Use it

There is nothing to switch on. In the dashboard, open your project's Scaling page, first under Stack, and choose a number of instances: the plan shows what will change before anything does, and Scale applies it. Going back down is always allowed. From a terminal:

snoutdata projects scale --instances 2

It prints the plan first and asks before it changes anything. The Servers and instances guide explains Metis, and Scale describes the page and the Instances card.