The three pillars of scalability

The three pillars of scalability are statelessness, idempotency, and coding to interfaces.

If you keep the above three in mind, your application can scale a long way with your users. Of course, I am not implying these are the only three things to keep in mind while designing scalable applications.

Statelessness:

If an application does not store persistent state locally, one can scale it by adding servers.

Let us take the example of an application that requires users to sign in. Once a user signs in, the application has to remember that this particular user has logged in. You have the option of storing the logged-in state of the users in the application servers’ memory. When a subsequent request comes, the application looks up in its memory and acts accordingly.

If you are following the above scheme, you are storing the persistent state locally—in servers’ memory. The upside of this approach is its simplicity. The downside is that you cannot elastically scale the application by dynamically adding and removing application servers based on the load.

To figure out whether your application is stateless or not, ask the question: If the next request landed on a different instance of the server, will my operation fail? If the answer is yes, the application is not stateless.

Idempotency:

An operation is said to be idempotent if it produces the same result when executed multiple times.

Example:

a, b = 1, 2

a + b is idempotent—irrespective of how many times you execute this, the result is always 3.

a++ is not—each time you execute this, you get a different result.

If your application is idempotent, you can retry failed requests.

Applications can fail momentarily, especially under load. When this happens, ideally, you should retry the failed request. But you can do this only if the application is idempotent. With idempotency, you do not have the unintended side effect of retrying a request.

You are trying to create a user. You hit the user creation API. For some reason, you do not get a response; this could be due to anything—a temporary network glitch, an application error, or something else. The bottom line is that you are not sure whether the user is created or not. If the application is not idempotent, you cannot retry the request. One might end up creating multiple users with the same identity. Not so, if the application is idempotent. One can retry with abandon.

Coding to interfaces:

Coding to interfaces lets us swap components.

You are using a cache in your application. Instead of using the cache provider’s API directly, you hide it behind an interface of your own. In the future, when you have a deluge of users, if you find the cache lacking, you can swap it with a performant cache without incurring tons of maintainability. You can do this only if you decouple your application from the specific cache provider’s API and abstract it out.

Conclusion:

It is tough to foresee scalability problems. Following the above generic principles will help you to develop adaptable applications that you can cheaply scale while buying time to create sophisticated scaling strategies specific to your needs.