When designing enterprise application architecture, talking to clients and doing interviews for my group, I sometimes tackle the “scale up vs. scale out” software architectural dilemma.
To set the stage lets define what scale means as well as what scale-up and scale-out mean.
The problem domain:
We have an application that serves about a 100 users (an online video store web site for example). Now, all is dandy and the application is running fast and smoothly. But then our application becomes more popular and more users want to use it at the same time. Now the application needs to serve thousands or even millions of users. The application is becoming sluggish and non-responsive because it runs out of computing power, memory or network bandwidth. There is an irony in this situation because this is what we hoped for (millions of users) and this is the time when we need the application to work at its best, while in reality this is when most applications tend to fail.
The scalability solution:
scalability is a desirable property of a system, a network, or a process, which indicates its ability to handle growing amounts of work in a graceful manner.
There are two kinds of scalability strategies:
Scale up (Scale vertically) means to run the application on a stronger computer.
Scale out (scale horizontally) means to run the application on many regular computers.
If we think of this in terms of a housing problem, these solutions make more sense.
Let’s say we have a family house with five people in it.
After a few years the family grows and is now composed of ten people and there is no more room. What do we do?
Scale up would mean to put everyone in one big building:
Scale out would mean to put these people in several small size houses:
Scale up – pros and cons:
Pros:
This is a straight forward solution that does not demand a change in the architecture of the software we write. You just run the same application on a stronger computer.
Cons:
On the other hand, the problem with scale up is that it is a costly and not an infinite solution. Big computers, like houses, cost a lot and there is a physical limitation to the computing power and memory you can have in a single computer.
Scale out – pros and cons:
Pros:
If planed right, this solution offers infinite scalability, when you need to support more users you just add more low cost computers to your server farms.
Cons:
On the other hand, this is not a straight forward solution. You need to design, architect, and develop your application to be ready to scale out (this is a topic for another blog I plan to write)
Conclusion:
For small scale application scaling up might be cheaper and faster to develop and implement. Having said that, most large scale applications, such as Google, Amazon, and Microsoft, use the scale out solutions in order to handle their scalability challenges.
Bottom line conclusion – if you plan for success do some pre-thinking and additional development to make your application scale out ready.
Thanks for your post, it help me understand exactly these terms with very simple examples.
Very good post, Amir. Clear and concise
Want to add that the drawback of scale up as expensive is manifested -exponentially- the more you scale but in the lower end is actually cheaper than scaling out
To clarify where scale up is a low hanging fruit compared to scale out, that comfort area is determined if, after diagnosing where contention is being produced, we find -let’s say- that contention is for CPU usage but not for memory, vice versa, or even both but we could solve the problem by just adding another CPU, a couple of gigas to the server as long as we have available slots in the current infrastructure
When that is the case, we could say that scale up is still more cost-effective. When we have no more room to grow in that infrastructure, migrating to a bigger one could start being more expensive -in the long term- than scaling out
Still there are two ways to scale out, one being keeping exact copies of the application on each server in the farm -so we could say that impact in the application should be minimal or probably null-. The other one being by partitioning the application in tiers. This last case will have two negative impacts, one at development time (as we’ll need to consider inter-process communications, time-outs, distributed transactions and possible compensations, etc) and execution time: a single client request may traverse more than one tier to be resolved, adding a performance impact derived from network latency, parameter marshalling, and so forth
Scaling apps is one of the topics I’m presenting next May in SATURN 2009 (a SEI conference, http://www.sei.cmu.edu/architecture/saturn/2009/program.html#may7 )
Mentioning this ad you can get important discounts! (just kidding, I’m sorry)
Thank you,
I totally agree that scale up starts as a low hanging fruit, especially when virtualization is driving the complexity of scale up even lower.
I do think that scale out, by keeping exact copies of the application on each server in the farm, might hide some development effort in the form of synchronization and consistency of data in the farm. I agree that this kind of “horizontal” scale out might be simpler to implement.
Good luck on your presentation.
Amir Shevat