Homer runs a restaurant called “Eat Out”. A typical set of resources such a business has are tables and chefs. So far restaurant has 10 tables available and 4 chefs what is enough to handle 100 guests per day. If Homer wants to increase this number, to let’s say 150 customers, he needs to add resources to his company. He calculated that another 6 tables and 2 chefs are required, if such amount of guests is about to be handled.
It Is A Good Habit to Consult
Before making any further steps, Homer decided to call a counselor to validate his way of thinking about resources. He took his cell phone and after quick search he found a perfect advisor. His name was Martin, and his expertise was quick. What he proposed was to hire only one chef, so costs will be smaller and some money saved. Led by a greed, Homer decided to ignore his calculations and did as advised. Unfortunately, it wasn’t good for the restaurant. Sure, after adding places to sit, number of guests quickly raised to anticipated, but quality of service dropped significantly. 5 chefs simply couldn’t manage cooking for so many guests, time between ordering and serving increased significantly.
When angry Homer came back to Martin with a bunch of complaints, he was introduced to another solution. The proposition was to increase amount of chefs to 7, but to leave the quantity of tables unchanged. In his opinion, so much work power should decrease time between ordering and serving, what will eventually lead to faster rotation of guests and finally handling 150 of them on daily basis. The sad thing is, that Martin also missed this time. Sure, time needed to receive a meal dropped, but guests count raised only to 102, and most of the time 2 chefs were idle. Eat Out is still not in balance.
But Is It Always?
After two fails, Homer decided to trust his own experience and add resources in the quantities planned at the very beginning. Finally, his restaurant has 16 tables and 6 chefs. With this configuration, Eat Out is able to handle 178 guests daily, time of waiting decreased even more and no chef is idle for more than 5 minutes. It worked better than he expected, now he is able to maintain quality of his service, even in the rush hours when number of clients is the biggest.
How To Always Act The Same
Would ask every piece of the process calculating business routines. When using your system, end-users always expect all operations to be finished in the blink of an eye. If system’s performance degrades with time, so patience of the people using it. And when users are unhappy, they usually search for faster and generally better solutions. This is not what you need, users equals money, so your aim is to have them as much as possible.
When comparing complex IT systems to the restaurant, one thing becomes obvious. Computer systems companies have, contain much more resources than cooks and tables, there are usually multiple dimensions of scaling. It is not always about slow database access or long time of running a program. Inability to efficiently manage or administer system data also makes users angry and frustrated. It doesn’t matter that operations run in no time when number of these increases to 30. It means 30 times more steps to achieve something, what can be completed in single one.
Another set of scaling problems may show up within the source code and system architecture. Coupled, non-modularized and rigid source, will likely be the reason of slow development iterations. Instead of being a well known and widely used solution, every module introduces different approaches, naming conventions and architecture. Team working on the project at its beginning, did not care about a future. Due to lack of communication, code reviews and standards, source code has as many architectures as programmers who participated in it. Eventually, it will blow in the business face, which won’t be able to deliver features, necessary to defeat a competition.
Another very important dimension of Scaling is connected with the infrastructure. When the system becomes international, it must be as responsive as its local counterpart, users in new regions must experience the same look and feel as current users. It usually comes in pair with “loading scalability”, which is ability of system to handle increased number of users or load in general, together with auto-scaling capability. It is a feature, when the system is capable to close or create new calculating resources according to current demand, or at least make it easy when it comes to manual adjustments.
When System Becomes Hungry
Thankfully, there are ways to scale computer systems. In its broad sense, they fall into two categories: vertical and horizontal.
Vertical Scaling (Scale Out)
Every computer system runs on some machine, it doesn’t matter if it’s virtual or physical - CPU and RAM are always limited. When your project grows, so processor and memory usage. If they reach their limits, the fastest and most logical action is to add more CPUs and RAM. Yes, it will help, but it’s only a band-aid, so don’t expect any long-term solution to scaling issues. To handle the load, you will add another processing units and gigabytes of memory, but hey, the new problems showed up. You cannot keep your infrastructure as a code, because CPU management and its optimizations must have been reflected in a source code. They have polluted the space of system logic with some advanced processes routing and managing, making it hard to move to different infrastructure. Also mind, that every machine has its limits when it comes to hardware capabilities, CPU and RAM are not an exception, so treat Vertical Scaling as a giving yourself some time before applying Horizontal one.
Horizontal Scaling (Scale Up)
Process managed by Horizontal Scaling implements only business rules, no knowledge about infrastructural details is implemented. Every particular process has an instance with the same version of the system. Thanks to this, there are a few worker processes which process requests simultaneously. It is important to remember, that no data should be set on machines hosting workers, because it won’t be available to the other nodes. Every node process can be closed at any time, and instantiated again without interrupting the system. This approach also make it possible to autoscale resources, what is not an option when it comes hardware capabilities - CPU and RAM are not usually auto-scaled, but rather whole machines themselves.
Summary
To sum it up, Horizontal Scaling is a better approach when it comes to system scaling. Clear division between infrastructure and the code, makes the system more stable and easier to manage. Managing machine nodes is fairly simpler than CPU based-one, which is hard-wired to the application logic.