Real Or Fake Tech Headlines? Weekly Quiz

Black Friday is mere weeks away and major retailers have already begun stress-testing their ecommerce sites to make sure they can handle the load. Failure to accommodate the onslaught of post-Thanksgiving shoppers might result not only in embarrassing headlines, but also millions of dollars in lost revenue.

BlazeMeter is a self-service, performance-testing platform provider that works with a variety of major retailers, including Walmart, Gap, and Disney. Its advice to retailers: September stress tests are essential to a glitch-free Black Friday.  

"Sites will go into a code freeze at least a few weeks before Black Friday, but now we're actually in the heat of performance testing," said Ophir Prusak, BlazeMeter's VP of product strategy, in a phone interview with InformationWeek.

[Want more on how to make your site bulletproof? Read How To Build Battle-Tested Websites.]

Prusak listed the six most common performance-testing mistakes made by ecommerce sites, as well as ways to prevent them. If your business is prepping for a Black Friday shop-a-thon, take heed:

Mistake #1: Not testing your production environment.
A lot of companies will test a staging or development system that they believe accurately reflects the production environment. But this dress rehearsal can lead to problems later. "There are a lot of things you can't anticipate in advance," such as slight differences in network infrastructure, said Prusak.

Not testing the production environment is a gamble. "Sometimes it works and sometimes it doesn't, but you're definitely taking a risk," noted Prusak, who added that "smart" companies will test their production systems, often in the wee hours of the morning.

"I can't name names, but I will say that one of the largest clothing retailers tests (its) production environment to the point of bringing the site down." This usually happens around 2 a.m. US Pacific Time once every two months or so, Prusak said.  

Mistake #2: Not accurately estimating the anticipated load, or number of users.
It's often hard for ecommerce sites to estimate the size of Black Friday crowds. "You never really know," said Prusak "We've seen people underestimate the actual increased load compared to previous years. Or even worse, you're a new entity and you don't have enough historic information to go on."

The testing solution? Push your system to the limit.

Figure 1: Image: Wikipedia

"It's not enough to say, ‘My site can handle 500,000 users, and that's what I anticipate this year,'" Prusak said. "You should push your system to the breaking point and say, ‘Well, if [Black Friday traffic] is 600,000 or 800,000 or a million, how close am I going to be to the breaking point?'"

Mistake #3: Failure to identify bottlenecks.
A lot of companies will stress-test and assume they can handle the heavy load. But bottlenecks vary by individual systems, so it's critical to run end-to-end tests across your entire infrastructure.

Where do these bottlenecks usually occur?

"It depends so much on the actual architecture," said Prusak. "I can't say there is one, but I will say that very often it is the database. It really depends on

Next Page

what you're doing, but the database system, or whatever the data store is, is often the bottleneck. Or it's the web servers themselves."

Mistake #4: Not having a backup.
Things always go wrong, of course, so smart companies have a backup system in place.

"Make sure you have backup servers, backup locations," Prusak said. "And also do a dry run. Very often companies do have backups, but the people who need to do it … aren't necessarily experienced at it."

Mistake #5: Overlooking the end user experience.
Most performance testing involves hitting the backend systems as hard as you can, noted Prusak, but don't forget to test the user experience with real web browsers while the site is under stress.

The server response time could become unacceptably slow, for instance. "The user experience might degrade from 5 seconds to 20 seconds," and the shopper will head elsewhere before completing the purchase.

Mistake #6: Not factoring in third-party integration.
"We kind of assume that third-party sites or tags are going to work, but you shouldn't make that assumption," said Prusak.

One solution is to use a technology called asynchronous scripting. "It means the webpage isn't waiting for the third-party script to load in order for the page to load," Prusak said.

Failing to factor in third-party glitches can spell Black Friday disaster.

"There was a client who had a third-party chat system, which allows users to chat with somebody at your store," said Prusak. "That's very common. But if there was a problem with the chat system, the whole page wouldn't load correctly."

Prusak's final piece of advice: Make sure any infrastructure improvements are worth the cost.

"If money were no object, every site could handle a limitless number of users," he said. "At the end of the day, you need to do the math and say, 'If it costs me so-and-so much to handle three times as much [traffic] as I anticipate, is it worth it?'"

Interested in shuttling workloads between public and private cloud? Better make sure it's worth doing, because hybrid means rethinking how you manage compliance, identity, connectivity, and more. Get the new New Tactics Needed For Hybrid Cloud Security issue of InformationWeek Tech Digest today. (Free registration required.)