Optimus Template

Are you choosing the right Load Balancer?

After a careful analysis of your infrastructure support vis-à-vis your business needs, you have finally realized how critically important a load balancing solution is for your business to achieve its full potential of growth. It is indeed a dream scenario to face a business problem like this. However, when you start evaluating the different load balancing solutions available in the market, your search is most likely going to end up in a fix. The problem is – you would find a number of manufacturers in the market claiming to meet exactly the business requirement that you have. And now the onus is upon you to select the right load balancer. Here is a checklist which will be helpful to you in choosing the right load balancer depending on your needs. Of course it goes without saying that among those handpicked based on your specific requirement; the most economical one will get a seat in your datacenter.

1.      Virtual IP –NAT:

Your site has only one address. End user accesses your site with address (e.g. www.viaedge.com). Since traffic has to be distributed to clustered server behind load balancer, your load balancer should create virtual IP address for all clustered server. To do this, does your load balancer use Network Address Translation?

2.      Balancing based on server capacity:

Does this solution distribute load as per the server capacity? Capacity of each server may or may not be the same, and therefore the solution must take care that one particular server is not over loaded and start giving poor response.

3.      Balancing based upon user category:

Solution should able to distinguish between a normal user and a premium user. Most of the time your site might be experiencing heavy traffic .You have limited server behind your load balancer and you have to ensure that your premium user must get efficient service. In this scenario, you have dedicated a few servers for your premium user. Hence solution must have capability to filter out normal users and priority users.

4.      URL Parsing

Does your solution allow distribution based upon HTTP request content and client address?
Does it also filter out HTTP header based upon cookie, TTP URL, HTTP version, application type, and actual HTTP request such as GET and PUT?

5.      Session Persistence

a.       Does the load-balancing solution create session persistence?

b.      Does the solution keep a listing of bindings between a user and a server?

c.       Can the countdown timer for such bindings provide the ability for wide adjustments, as wide a range as one minute to two days?

d.      Does the solution have intelligent persistence features, including not only IP source address and TCP port, but also SSL session IDs and user cookies?

e.       How does the solution address cookie persistence when a session involves multiple servers?

6.       Performance

a.       How fast can the solution distribute load? Is there a limit to its speed? Is the limit of its speed that of the servers themselves or of some part of the solution (such as the speed of the switch router)?

b.      What is its throughput, as measured in megabits per second?

c.       If a switch router is the core of the solution, can it support all the major routing protocols, including those necessary for the Internet (RIPv1, RIPv2, OSPF, IPX RIP, IPX SAP, and BGPv4) as well as multicasting protocol such as DVMRP v3 and PIM-SM/DM?

d.      Does the solution actively monitor content availability on the servers? That is, does it have extended content and application verification?

e.       Does it redirect with requests for applications or content when the response fails?

f.        Can the solution load-balance all TCP/IP services, including Web, proxy, virtual private networking, and streaming media services?

7.      Health Check of Servers

Solution must ensure that all traffics are routed to healthy servers.

a.       Does the load balancer regularly check the status of a server, including its current response speed, load, and capacity?

b.      Does the solution provide easy viewing of the servers' status? (Is that available on a GUI, as part of the management tools?)

c.       How does the solution check server condition?

                                 i.            Does it ping the server? How often?

                               ii.            Or does it actively monitor server conditions? For example, does it use agents that are internal to the server? That monitor server heartbeats, reporting on server conditions often more accurately and fully?

8.      Management

a.       Does the load-balancing solution have management tools, including:

                                 i.            Administration?

                               ii.            Monitoring?

                              iii.            Trending analysis?

b.      Do these features allow for both managing current traffic as well as forecasting, such as when upgrading of the server farm may be necessary? (See the next section for further questions about these features.)

c.       Can the solution be administered securely and remotely, either based on secure shell (SSH) for command line or SSL if browser-based?

d.      Does the solution have access-control features? (Limiting access maximizes bandwidth in and out of the server farm because it allows only authorized traffic, relieving all loads from unauthorized frames. It also strengthens protection from denial-of-service attacks.)

e.       How easy is the solution to use?

f.        How is updating done? Must servers be offline to update?

9.      Protection:

a.       Does the solution provide true traffic-surge protection, the ability to scale service on the cluster to meet sudden spikes in traffic?

                                 i.            For measuring traffic spikes, does it actively verify both availability of the servers and content on those servers?

                               ii.            Or does it provide the less desirable form of protection --the failure of actual traffic -- before responding?

b.      Does it effectively eliminate single points of failure on the cluster? Or does it create a single point of failure, such as a single switch router on which the entire load-balancing solution depends?

c.       Is there redundancy built into the solution? Or must you, as with a switch router-based solution, purchase an additional router to create it?

d.      Does it have a protocol for hot-standby failover?

10.  Security

a.       Can the solution be integrated with network security, so that it can be scanned like all other components?

b.      Does it integrate with network security products, including the following?

                                 i.            Firewalls

                               ii.            Routers (or router-like network devices)

                              iii.            Transparent caches

                             iv.            Proxy servers

c.       How easily does it integrate with them? Does it require additional software to do so?

d.      Do the security features of the solution include:

                                 i.            Default/deny

                               ii.            Trace routing

                              iii.            IP address check and port mapping

                             iv.            NAT and SNAT

e.       Is the solution able to be configured to protect against SYN attacks? For example, can administrators set it to enter protection mode after a certain number of unanswered SYNs?

11.  Integration

a.       With what platforms is the solution compatible?

b.      Can the solution work in an environment with multiple platforms?

c.       How easily can it be upgraded?

12.   The final check

If many vendors have satisfied or scored equally in above check list then the following questions will definitely help you filter out many and zero down on one

a.       How expensive is the solution compared to other solutions?

b.      If it is an all-software model without dispatch, what costs to performance and efficiency result from all servers on the farm having to see all packets before they are filtered?

c.       In the event of a server failure on this same kind of solution, how long does it take for the remaining servers to notify each other and adjust to this failure? How does that amount of time compare to the response time for failover in the other kinds of solutions?

d.      Is the non-dispatch, all-software solution able to partition the cluster to serve different kinds of traffic?

e.       Does the non-dispatch, all-software solutions have the comprehensive ability to handle persistent connections? (It probably has only minimal abilities to manage stateful traffic.)

Evolution of Load Balancer

The Journey from DNS Round Robin to IP Clustering to Server Load Balancer to Application Delivery Controller

The concept of LOAD was coined during dot com boom era of the last millennium. During initial days of commercial internet, wannabe dot com millionaire had a big issue concerning cost as well as technology in their business plan. Mainframe was generally out of budget for most of the start ups, and all they could afford was a PC based off the shelf server. However, they were unable to handle the large amount of traffic, but even if they could, it posed a great risk to their profitability if the server went down, because in that case their business was going offline and they could even face the danger of going out of business. It was indeed a serious issue, but as history has proved time and again that necessity is the mother of invention, this gave birth to the concept called load balancing.

The concept of load balancing has witnessed an evolution in the last 10 years, no less remarkable than the evolution of Modern Man from the Early Ancestors. Starting from the DNS Round Robin in the 90s, it evolved into IP Clustering and Server Load Balancer, and then to Application Delivery Controller or ADC, which is the latest technology in use by organizations world-wide at present.

DNS Round Robin:

Domain Name System or DNS translated human-readable names (www.viaedge.com) into machine-recognized IP addresses. DNS also provided a way in which each request for name resolution could be answered with multiple IP addresses in different order .This solution was simple and provided the basic characteristics of what customer were looking for by distributing users sequentially across multiple physical machines using the name as the virtualization point. As the service needed to grow, all that the business owner needed to do was to add a new server, include its IP address in the DNS records, and voila, attain increased capacity!

However this solution could not address the issue of high availability. A few serious limitations of this solution were as follows: Clients were earlier used to cache the name resolution. This means that clients did not ask for IP of the server; instead they went back to the server they used earlier irrespective of the fact that server was presently working or not. So there was a need for some good mechanism to ensure that clients did not bypass the load balancing due to caching

DNS had no capability of knowing if the servers listed were working or not. There was a need for the system to have the capability of automatically detecting the malfunctioning server and remove them

DNS provided the poor load balancing along with uncontrolled distribution. It highlighted the striking difference between “load distribution” and “load balancing".

Persistence was one of the major issues which DNS Round Robin had no capability to solve.

IP Clustering:

Clustering solution evolved in response to various limitations of DNS technology. It had all the servers in a cluster listening to a “cluster IP” in addition to their own physical IP addresses. Scalability of this solution was readily apparent. All you had to do was build a new server, add it to the cluster, and thus the capacity of your application is enhanced.

The availability was dramatically increased with this technology. Because the clustered members were in constant communication with each other, and also because the application developers could use their extensive application knowledge to know when a server was running correctly, this virtually eliminated the chance that users would ever reach a server that was unable to service their request.Predictability was also enhanced by these solutions. Since the application designers knew when and why users needed to be returned to the same server instead of being load balanced, they were able to embed logic that helped to ensure that users would stay persistent as long as needed.

This solution too was not free from constraints and had some serious limitations:

·        It had potential limitations on true scalability.

·        It was reliant on the application vendors to develop and maintain.

Server Load Balancer or Network-Based Load balancing Hardware:

Further research into load balancing technology gave birth to network based load balancing which was the foundation stone for the application delivery controller, the latest technology in this domain. These load balancing appliances were not only application neutral but also resided outside of the application server. The load balancer presented virtual servers to the outside world. Each virtual server pointed to a cluster of services that resided on one or more physical hosts.

Server Load balancer (SLB) which is now known as simple early generation load balancer is basically a Layer 4 Balancing technology, which has the ability to direct traffic based on MAC/IP address and TCP port (e.g., L2-L4 information) and is now prerequisite for all load-balancing solutions. It could be treated as the building block of entire balancing ecosystem. Functionality such as health monitoring, session persistence and network integration are minimum requirement of layer 4 load balancing.

The advent of the network-based load balancer ushered in a whole new era in the architecture of applications. Now Load balancer could control each server in terms of connections. It could provide at least basic load balancing services to nearly every application in a uniform, consistent manner - finally creating a truly virtualized service entry points unique to the application servers serving it. Scalability with this solution was only limited by the throughput of the load balancing equipment and the networks attached to it. Network-based load balancing hardware enabled the business owners to provide the high-levels of availability to all their applications instead of merely the select few with built-in load balancing. The added intelligence to create controlled load distribution (as opposed to the uncontrolled distribution of dynamic DNS) allowed business owners to finally use load distribution in a positive way, sending more connections to the bigger servers and less to the smaller ones.

Application Delivery Controllers:

Application Delivery Controller (ADC) could be imagined as the Load balancer technology at its zenith, i.e. the maximum limit to which the load balancer could be stretched. Although we may give more stress or weight to security, performance and availability, but load balancing technology is critical to execute all these said attributes. The intent of ADC is to have a single device that incorporates not just a core set of load-balancing capabilities but a comprehensive set of application performance and security services as well

Be it SSL offload or centralized authentication or application fluent firewall, one thing remain is that load balancer is the aggregate point of virtualization across all applications.ADC is based on the concept of Layer 7 load balancing and is a combination of application aware layer 7 switching capability over and above layer 4 load balancing (popularly known as server load balancing). Layer 7 switching takes its name from the OSI model, indicating that the device switches requests based on layer 7 (application) data. Layer 7 switching is also known as “request switching”, “application switching”, and “content based routing”.

Unlike server load balancing, layer 7 switching does not require that all servers in the pool (farm/cluster) have the same content. In fact, layer 7 switching expects that servers will have different content, thus recognizing the need to more deeply inspect requests before determining where they should be directed. Layer 7 switches are capable of directing requests based on URI, host, HTTP headers, and anything in the application message. The salient features of Layer 7 Load Balancing are:

·        This allows the architect to design an application delivery network that is highly optimized to serve specific types of content but is also highly available.

·        This has additional features offered by application delivery controllers to be applied based on content type, which further improves performance by executing only those policies that are applicable to the content.

·        This also allows for increased efficiency of the application infrastructure.

·        This allows getting better efficiencies out of your servers by grouping them so that some handle transactions, while others just act as massive storage systems for serving up static pages, or are optimized for downloading streaming video. For instance, URL Parsing involves looking at the URL that appears just after the HTTP GET header, and sending the request to one of a group of servers based on that address. You could also use the extension (.asp, .gif, etc) to point traffic at servers that have been optimized for serving that type of traffic.

·        This allows you to make sure that some users are directed at higher powered servers, if they are premium customers, or are on your site to place an order rather than just browse. A cookie value (which can be more or less any string) might indicate that this person has used your site before, so that you can welcome him by name the next time he visits, or if it is a service he has previously subscribed to, you can send him to the servers providing the services they have paid for.

Scalability, high availability, and persistence are common attributes of ADC with load balancer. Performance enhancement is another obvious extension to the load balancing concept. These ADCs often include caching, compression, and even rate-shaping technology to further increase the overall performance and delivery of applications.

So what lies in future?

Candidly speaking, it is hard for anyone to make a guess as to where do we go from here, but surely a probabilistic guess could be made at this point of time. Traffic need gave birth to the very concept of Load Balancing that further grew and evolved up to the level of ADC. The changing user needs and the associated technology required to meet those challenges will surely take us to an environment where the technology is more capable to take care of the following:

·        Integration of Network Access Control.

·        Some innovative way of application caching/compression.

·        Application of business rules to the management and control of application delivery.

·        Reduction in number of devices.

Let us wait and watch for some new surprises with some revolutionary breakthrough in technology. But till the time any innovative ground-breaking technology is evolved, the best that we can do is to concentrate on creating robust ADCs for betterment of the existing ecosystem.

Mystery and magic behind Load Balancer

The load balancer presents virtual servers to the outside world. Each virtual server points to a cluster of services that reside on one or more physical hosts.

Hey, don’t loose your heart by lexicons used in above statement. Instead, let’s examine each terminology one by one to make our life simpler.

Host: The physical server which receives the traffic from load balancer. This is synonymous with the IP address of the physical address (for example 192.168.57.10). In scenario when LB is absent host is the server name such as www.viaedge.com and this resolves to the IP address 192.168.57.10. Different vendor terms it as node / server/Host.

Services: It includes the application TCP port of actual application running as well as the host (IP address of the physical address of the server). A host (192.168.57.10) may have more than one service available (HTTP, FTP, DNS, and so on). By defining each application uniquely 192.168.57.10:80, 192.168.57.10:21, and 192.168.57.10:53. Different vendor terms it as member / services. Some vendor terms it as node too (quite unfortunately!!)

Clusters: They are collections of similar services available on any number of hosts. For instance, all services that offer the company Human resources would be collected into a cluster called “company HR page” and all services that offer ecommerce services would be collected into a cluster called “eCommerce.” Some vendor terms it as pool/cluster/farm.

Virtual Server: Like the definition of services, the virtual server usually includes the application port was well as the IP address. Since most vendors use virtual server terminology is often used in LB discussion, we will continue to use that terminology in our discussion, although the term “virtual service” would be more in keeping with the IP:Port convention.

Now we are equipped with all lexicons of load balancing, now see the statement used in the beginning “The load balancer presents virtual servers to the outside world. Each virtual server points to a cluster of services that reside on one or more physical hosts.” I got the meaning but still didn’t understand the concept. Not an issue, let’s dissects it further to understand it better.

The five magic steps of load balancing

(Assuming the load balancer will typically sit in-line between the client and the hosts that provide the services the client wants to use)

1. The client attempts to connect with the service on the load balancer.

2. The load balancer accepts the connection, and after deciding which host should receive the connection, changes the destination IP (and possibly port) to match the service of the selected host (note that the source IP of the client is not touched).

3. The host accepts the connection and responds back to the original source, the client, via its default route, the load balancer.

4. The load balancer intercepts the return packet from the host and now changes the source IP (and possible port) to match the virtual server IP and port, and forwards the packet back to the client.

5. The client receives the return packet, believing that it came from the virtual server, and continues the process.

Let’s further discuss the couple of issues to finally crack the rest of the mystery of load balancing

First, as the client knows, it sends packets to the virtual server and the virtual server responds—simple.

Second, the network address translation (NAT) takes place. This is where the load balancer replaces the destination IP sent by the client (of the virtual server) with the destination IP of the host to which it has chosen to load balance the request. Step three is the second half of this process (the part that makes the NAT “bi-directional”). The source IP of the return packet from the host will be the IP of the host; if this address were not changed and the packet was simply forwarded to the client, the client would be receiving a packet from someone it didn’t request one from, and would simply drop it. Instead, the load balancer, remembering the connection, rewrites the packet so that the source IP is that of the virtual server, thus solving this problem.

Hopefully by now, you are comfortable with the basics of load balancing. In next blog we will discuss on issues such as “Load balancing decision”, To load balance or not to load balance”, Application deliver controller (ADC) “and many more.

Tags: Load Balancer