There was a time when single machine would host a web server with a database and all you need after that was a perfect network connectivity to run your website. I won’t continue without saying – it’s still true! What changed afterwards is part of the history. No single business can think of using that infrastructure if they want to survive with today’s requirements. Small blogs and static sites are still OK but not when you are getting tremendous amount of traffic and have dynamic content with transactions happening every second. Do you still have ambiguity about what web-scalability is? Let’s put that in words then,
Web Scalability is – you want to serve huge amounts of traffic without giving customer a bad user experience. In essence, as you grow, you have to handle more requests and your data requirements would increase which can produce a depreciating quality of service. In the hindsight, the requests could mean many things starting from user triggered operations, backend system communication (microservices architecture for lean startups) or API requests. To handle all this and in a cost-effective way is what web scalability is all about.
To get to the next level, your infrastructure must compliment your software architecture.
Who needs a Scalable Infrastructure?
It’s not for you, who is running a fairly low-traffic website that takes a few hundred orders a month, its for those who want to create a fairly hard rock infrastructure that can handle millions of requests (eventually) in a small amount of time, or have to store big chunks of application data and then grow with it.
If your infrastructure comprised of a single machine with a web server running either a website, a SaaS application or a mobile app backend, you will not gain much performance out of it. As the number of user’ requests increase, your requirements will surpass the limitations of your infrastructure. Consider a scenario where the machine that is running your webserver, like Apache, Glassfish or Nginx will run out of resources to serve the users. Let’s say that every user that connects to your webservice starts 5 connections to your site and then each connection holds 500KB of ram, which makes 2.5MB per user. Now imagine 2000 active users who will need atleast 5GB of RAM. Same thing applies to CPU consumption where an active connection might be doing certain operations that will starve processes for CPU.
How it all begins till the very end!
Lets discuss the aspects of scalability requirements that you want to fulfill in order to bring your startup in a shape that can handle heavy traffic.
It will not be wrong to say that DNS is the first entry point for a client to your website which is used to resolve the domain name to IP addresses. Every domain name has an associated IP address behind it. Once user connects to that IP address, the next thing they do is data exchange to serve content to the user. DNS hosting service is better to be a good one as its network latency will impact the user experience in a most apparent way. You don’t want to visit a site again that loads in 15 seconds and still looks broken. You can optimize your hosting performance by choosing the best hosting providers that can serve users from any part of the world with lower latency. Amazon’s Route53 and Cloudfare are two good options.
Loadbalancer: On busy websites, it’s common to receive more than 100 client requests/second. If you reach such traffic, you better be prepared. Loadbalancer is one of the first line of defense to safeguard your web servers from the flood of client requests. Hide your webservers behind a loadbalancer where client requests will be distributed to web servers and will be served rather evenly.
The beauty behind loadbalancer is you can horizontally scale your webservers by adding new ones without changing any client facing endpoints. The loadbalancer will need a slight configuration change though to include new webservers. Cloud based hosting solutions are perfect example of grabbing the most benefit out of load balancer, as the infrastructure can scale up and down based on usage. Cloud hosting solutions are provided by several good companies like Amazon, Rackspace, Azure, or Open Stack that you can give a look
Web Server: Coming towards Web Server – that do not matter in the initial stages – but if you want to plan for the future or at least thinking of scaling a poor webserver into a better one than dont just think of replacing any of the old Apache server to Nginx or Node.js but think in terms of whether you want it to be stateless or stateful. When it comes to scalability, stateless is your friend. Spinning up multiple webservers, without thinking of state will make your life easier. Imagine a scenario, where user’s session data is stored on the webserver and that will make life difficult if you want to restart the server for some reason or do maintenance. You don’t want your user to get logged out in the middle of his session or show inconsistent information. The task of webserver remains the same and that is to serve web text, images, and files to the client and all that sometimes under one click. For that, the webserver communicates to bunch of infrastructure components like data store, queues, log server or even caches making it the vital piece as a whole. Given your requirements, you can choose any webserver but to scale this piece, you must keep it stateless and instead serve the data from shared data stores, object caches, or streaming backends.
Caching: To quickly serve the user requests caching comes into play. For dynamic websites, where content is subject to each user interaction, it’s difficult to optimize caching but when the content is mostly static like images, videos, static text, or files then it’s easier to make better use of caching. One way of caching is to use CDN, where the necessary files or images will be cached for next similar request in your closest delivery node. Other way is to store files like *.js, *.css and images in the browser cache. Depending on your data freshness requirements, you can set the cache expiry time to invalidate the cache and grab the fresh copy when old is expired.
We are not finished here, we will cover other aspects of Scalability issues in our future articles.