Server load balancing: A practical guide

A practical guide to server load balancing by a Network World reader. Details server load balancing technologies and applications for systems managers.

This piece was contributed by a reader. If you have expertise or an experience you would like to share, contact Network World Editor in Chief John Dix (jdix@nww.com).

A few years ago server load balancing was an expensive luxury for sites with deep pockets. It wasn’t uncommon to pony up $100,000 for a redundant pair of load balancers. But today load balancing is within reach of companies with far more modest means. Vendors old and new are offering full-featured load balancers for less than $5,000.

These sub-$5,000 devices are no blue light specials. They offer the features and performance small and midsize businesses require. In fact, many are as good or better than the highest end load balancers of five or six years ago. They come in redundant configurations, perform network address translation, health checking, and some even do cookie persistence, SSL acceleration/offloading and URL switching.

However, load balancing technology is foreign to many customers who, until recently, couldn’t afford it. In addition, it requires knowledge of networking, server administration and applications. Because of the novelty and divergent skills involved, shopping for a load balancer can be daunting. This story will help you make a more informed purchase decision.

To simplify the job of shopping for a load balancer, we’ll examine four primary concepts:

● Feature set: What features you might need and what features the load balancing vendors offer.

● Costs: This includes the price of the unit, plus support and maintenance contracts that can boost the price.

● Performance: Everyone wants to make sure they get a box that will support their current traffic, as well as traffic for the foreseeable future.

● Integration: Will a given product work with your network? Sometimes a feature here, and unsupported implementation there, will lead to a “gotcha” and derail a project.

Server load balancers can come with a dizzying array of features, from cookie persistence to on-the-fly HTTP header re-writes, so you should know what’s important to you beyond the functions that all load balancers provide: distributing traffic from a virtual IP and port and checking to see if a server is responding.

Cookie persistence

Server persistence is one of the most important functions load balancers perform. Persistence (also called sticky and server affinity) is what keeps an individual user glued to a single Web server, rather than being sent to a different server for every request.

For interactive Web sites this is typically a must. Interactive session information for an individual (such as a logon credentials or a shopping cart) are often kept only on that server during the session and not shared with the other servers. So if a user were sent to a different Web server they might find their shopping cart is empty or be forced to log on again.

The two basic types of persistence methods for load balancers are source IP address and cookies. With source IP address, the load balancer looks at the source address of the incoming request to keep track of individual users. With cookie persistence, the load balancer looks at an HTTP cookie to differentiate users.

For HTTP traffic, cookie persistence is the best way to go in most situations. It is simple to setup and it gets around a number of tricky issues involved with source IP persistence, such as office routers and mega-proxies where hundreds or even thousands of users can come from a single IP. For non-HTTP traffic, source IP is your only option.

I would recommend picking a load balancer with cookie persistence for two reasons: One, even if you don’t need it now, it’s an excellent option to have down the road. And two, you don’t really pay any extra for it. Units from vendors that do offer cookie persistence cost roughly the same as units that don’t offer it.

Some vendors argue that cookie persistence isn’t necessary, and that writing applications that require persistence is bad design. Oddly enough, these are the vendors that don’t offer it. I think that’s garbage. To be certain, there are advantages to writing a stateless application that ensures that information is shared across all servers, but rewriting an application to avoid this is a lot more difficult (and probably more expensive) than choosing a load balancer that supports a simple solution to the persistence issue.

Health checking

Health checking is the mechanism by which the load balancer will check to ensure a server that’s being load balanced is up and functioning, and is one area where load balancers vary widely.

Basic health checking includes ICMP ping, TCP port open and doing an HTTP HEAD or GET command and looking for an HTTP 200 response. Some load balancers offer a more interactive health checking mechanism, allowing the user to send a specific request and parse for a certain response.

For most users, standard HTTP GET or HEAD commands should be sufficient. If you need to do a specific check, inquire with the potential supplier about their health checking methods. They can recommend a configuration or script that might suite your needs.

SSL acceleration

Some load balancers now offer SSL offloading and even SSL acceleration. With offloading, the cryptography is typically done with the load balancer’s main processor. With acceleration, the cryptography is done on a separate SSL accelerator card (usually an installed PCI card), enabling even a modest box to handle thousands of SSL transactions per second.

One benefit of SSL offloading is that you can perform cookie-based persistence, even on an SSL connection. Without SSL offloading/acceleration, the load balancer can’t see the HTTP headers, which contain the cookie that is used for persistence, so you’re stuck with source IP persistence (SSL session ID used to be a viable option, but no longer thanks to a change in the way SSL session IDs are handled).

If you terminate the SSL session at the load balancer, the HTTP headers are then viewable by the load balancer. With SSL acceleration, you get the ability to do cookie persistence on Secure-HTTP traffic, as well as the performance increase that hardware acceleration gives you.

Offloading is typically less expensive because no additional hardware is required. However, because the CPU-intensive cryptographic functions are done on the general processor, the number of SSL connections a given device will be able to handle is limited, and a high connection rate would affect all functions of the load balancer. For this reason, SSL offloading is only appropriate for light SSL traffic. SSL acceleration is capable of handling thousands of SSL connections per second typically, and suited to medium to high traffic sites.

URL parsing

Another Layer 7 (protocol aware) feature (sometimes referred to as content switching, but not switching in a Layer 2 network sense) is URL parsing, which allows a site to send traffic to different servers depending on what the URL is. For instance, a group of image servers will serve up everything under http://exampledomain.com/images, while a set of map servers will handle anything under http://exampledomain.com/map.

In my experience, this is not as useful of a feature as say, cookie persistence. In most cases it’s easy to get around by using virtual hosts (maps.domain.com and images.domain.com), and isn’t used that often by most sites anyway because they were designed without that ability.

Still, it’s a nice option to have, as it gives you a certain amount of flexibility in case you need to make some adjustments down the road.

Redundancy

Redundancy is available in most load balancers, but make sure to ask. Even if you can’t afford to pay for a redundant configuration up front, make sure the unit supports it so you can add it later.

Generally, there are two ways to do redundancy: Active-standby and active-active. The latter is available typically only in higher-end units, but I don’t think active-active is necessary.

I never liked active-active for two reasons, the first of which is that it’s complicated to setup and troubleshoot. When you’ve got two active network connections in a switching infrastructure, it’s a more complicated setup. Maybe I’m just being lazy, but active-standby has always seemed the most attractive option to me.

The second reason I’m not a big fan of active-active is actually one of the touted benefits: you get higher utilization. When you’ve got two units of equal power, active-active can provide twice the capacity over active-standby, because both units are in use. I’m personally not comfortable with this because, if you’re running close to 100% on both units, if you do have a failure you’re oversubscribed by 100% on the remaining unit. Not a good situation.

Because of the added complexity and capacity issues, I prefer to use active-standby for load balancers.

Capacity

Certainly, you’re going to want a load balancer that is capable of serving up the traffic you’re pushing, with plenty of room to grow down the road. Beyond that however, the question of capacity can get a little tricky. They are network devices, but load balancers actually have more in common with Web servers when it comes to performance characteristics. Web servers typically are measured in connections per second, while routers and switches are typically measured in pure throughput (bits per second).

Throughput is a useful metric in many respects, but it’s not much of a factor when it comes to load balancers. Pushing throughput just isn’t all that tough a task for load balancers today, even the modestly powered ones.

The most important performance metric for load balancers is connections per second. The work involved in accepting and establishing a TCP session, potentially parsing the HTTP header, and forwarding the traffic to another Web server, is substantial when compared with the relatively easy task of throughput.

Even with modest hardware, a load balancer can easily push line speed on a 100Mbps Fast Ethernet interface if the connection rate is low (less than 5 connections per second). If the connection rate is high (thousands of connections per second), even a high-end box would start to sweat.

There is one area of performance where throughput is a factor in load balancers, and that’s in picking the speed of the network interface. Load balancers typically come in either Fast Ethernet (100Mbps) or Gigabit Ethernet (1,000Mbps), the latter typically coming at a bit of a premium.

The infrastructure in most small to midsize businesses don’t push close to 100Mbps, so in most cases Fast Ethernet is sufficient. And with Fast Ethernet, it’s important to keep in mind that even if the load balancer is 100Mbps, it will not choke an otherwise Gigabit network down to 100Mbps. Only the load balanced traffic will go through the load balancer. In most cases, you don’t plug the servers directly into the load balancer (there’s no real reason to do so), so if you plug Gigabit servers into a Gigabit switch, they’ll still be able to communicate with each other at Gigabit speeds.

If you’re currently pushing 20Mbps or traffic or less, you probably don’t need Gigabit Ethernet. Even if you’re pushing 50Mbps, you’ve still got 50% of capacity growth for your site. If, however, you’re pushing more than 50Mbps, and you have a pretty good idea that your site’s traffic will grow quickly, you may want to consider ponying up for Gigabit.

Cost

The cost of the unit is only part of the total cost equation. If you want to have redundant load balancers (as for any network device, redundancy is highly recommended), you’ll need to double the single unit price.

Also keep in mind that some vendors charge extra for the support contract, and that support contracts often come in various levels (such as 9-to-5 tech support vs. 24/7 support). This is an important factor in the cost equation.

Vendor engagement

One of the greatest fears of buying a load balancer is having spent all that money to find out it doesn’t do what you need. This is particularly true if you’re new to load balancing and not even sure a given product will work in your environment. There are often little gotchas — separately working devices don’t seem to get along, or other infrastructure quirks that can impose incompatibilities that are difficult to foresee if you’re not familiar with all the pieces.

Certainly not an unfounded anxiety, but fret not; there are some steps you can take to mitigate that possible outcome.

First, most load balancing vendors will go over what you’re looking to accomplish with a load balancer as a free part of the pre-sales process. They’ll walk you through the process, recommend a configuration, and even trade a few Visios back and forth. They may be able to point out solutions you hadn’t thought of and warn you about potential gotchas before they strike.

Many vendors will allow you to “try it before you buy it” and if you’re unsure that a given load balancer can serve your needs, you may want to take them up on their offer.

Questions to ask

Based on the topics covered, here’s a cheat sheet to use when talking with a potential load balancing vendor:

* How much does the unit cost?

* Does this include support/maintenance?

* Does this include hardware replacement if the unit goes dead?

* What sort of connection rates can the unit handle?

* Does this unit offer SSL?

* If so, is it hardware accelerated, or on the general processor?

* Does the unit handle cookie persistence?

* Is there an evaluation period?

* Here is my network, and what we intend to do with the load balancer. Will your product work in our environment, and do you have any ideas that might make the implementation even more solid?

With this information, along with the cheat sheet, shopping for load balancers should be straightforward. And by shopping for load balancers in this new market, you can save a bundle while still getting advanced features.

Bourke runs the load balancing mailing list, editor of http://lbdigest.com: LB Digest, and has authored numerous articles on load balancing, networking, and server-related subjects. He lives in Brooklyn, N.Y., and can be contacted at tony@vegan.net.