Overview of Load Balancing with Nginx

Load balancing is a way of distributing traffic to computing resources such as servers or other network components. The technique is primarily used to improve performance: by increasing concurrency and improving reliability. A load balancer is a software or hardware that is used to implement load balancing. Nginx is one of the best software to use in implementing load balancing.

Importance of load balancing

• Improves performance of the system/application. Load balancing distributes traffic across various nodes. This ensures no single node is overburdened with traffic, which in turn means users will have a better experience using the system/application.
• Increases availability. With load balancing, the same application is deployed on several nodes and accessed depending on availability. A node that is not operational for some reason, does not receive traffic. A load balancer keeps checking that all backend servers are live so that to pass traffic only to online nodes.
• Is more economical. Load balancing allows horizontal scaling. Horizontal scaling refers to using more servers to to cater for growth, by distributing traffic to those servers, instead of using few bigger servers – buying the latest, most powerful servers available to cater for the growing traffic your site or application receives, but this becomes expensive. Load balancing helps deliver an equally fast and reliable experience at a lower budget.

Necessary Directives for Nginx Load Balancing

As a load balancer, Nginx relies on specific directives, which are built in Nginx by defualt. For load balancing, the directives needed are upstream, server and proxy_pass. These directives can take parameters such as:

• weight
• max_fails
• fail_timeout
• down
• backup
• slow_start
• max_conns

To better explain these parameters, we will use the example below:

upstream app {
    server s1.application.com slow_start=30s;
    server s2.application.com weight=3;
    server s3.application.com max_fails=5;
    server s4.application.com fail_timeout=10s;
    server s5.application.com backup;
}

Assume the above was extracted from a load balancers’ config file:

• weight=3 means s2 will be selected 3 times as much as the other servers. The default weight is 1
• slow_start=30s specifies the time (30seconds) a server which was down is given to recover its operations before being overburdened with connections, which may cause it to go down again.
• max_fails=5 means there should be 5 timeout connections within the period specified by fail_timeout directive, before the load balancer marks a server as inoperative
• backup means the server will not be used until all other servers are inoperative

Note: If there is only one server in the backend server group, the parameters max_fail, fail_timeout and slow start will be ignored as this server will never be considered to be inoperative.

Methods of load balancing with Nginx

Nginx open source supports 4 methods of load balancing as follows:

• Round robin – here, requests are passed to all servers equally, but taking server weight into consideration. Its the default load balancing method in Nginx and it has no directive.

upstream app {
    server s1.application.com;
    server s2.application.com weight=3;
}

• Least connections – here, requests are sent to the server having least active connections, taking server weight into account. To specify this method, use least_conn directive as follows:

upstream app {
    least_conn;
    server s1.application.com;
    server s2.application.com weight=3;
}

• IP hash – here, the backend server that will receive a request will be determined by the IP address. This ensures requests from the same IP go to the same server, provided it is available, making it possible to maintain login sessions. To specify this method, useip_hash directive as follows:

upstream app {
    ip_hash;
    server s1.application.com;
    server s2.application.com weight=3;
}

• Hash – here, the server to receive a request is determined by a user-defined key such as source IP and port. To specify this method, use hash directive as follows:

upstream app {
    hash request_uri;
    server s1.application.com;
    server s2.application.com weight=3;
}

What solutions does load balancing bring in?

Load balancing solves several key issues historically experienced while deploying cloud applications. Load balancing brings in advantages for all levels of people interacting with a cloud application. These pros include:

• Flexibility – load balancing allows sysadmins to easily add or remove nodes from a group of servers to cater for increasing/decreasing user traffic as needed.
• Availability – with load balancing, the same application is deployed in more than one server, meaning, even if one of the backend servers experiences an outage, the others can continue serving users as the inoperative node is sorted.
• Performance – horizontal scaling provides a faster web experience compared to vertical scaling. Apps hosted in several servers are faster compared to one hosted in one big server
• Economical – having several small servers for your app is cheaper than having one big server.

For more information about load balancing you can refer to our article on a practical test of how load balancing is beneficial and another article on how to create a load balancer using Nginx Open Source