Rolling Review: Web App Scanners Still Have Trouble With Ajax

LONG, STRANGE TRIP
When we kicked off this Rolling Review, in the May 14 issue of Network Computing, we knew a few things: That browsers are insecure, and rich Internet applications make us nervous. That Ajax is a wildly popular development environment, but it's very challenging from a security standpoint. And that few developers are committed to secure coding.

What we didn't know was whether an automated Web application scanner could handle Ajax, given its complexity. Normally, this code is written in at least two languages, JavaScript for the client plus whatever is running on the Web server.

Then there were the outside forces: Two of the five products were acquired midreview, while one vendor sued another for patent infringement. And it's not just the business side of Web application scanning products that's active. The technology itself is in constant flux.

The beauty of the Rolling Review format is that vendors get continual feedback and are able to address issues on an ongoing basis--and every product tested released a significant upgrade during the course of this review. Most added functionality specifically aimed at Ajax applications. WebInspect, for example, deserves a Most Improved award--SPI Dynamics and then, after its acquisition, Hewlett-Packard, have done a great job addressing a variety of bugs and errors, adding new features, fixing checks, and dramatically improving performance on our sample applications. While the history of errors in the product is somewhat disconcerting, WebInspect has strengths as well. If the bugs can stay worked out, HP's acquisition may pay off.

We'd also like to update our take on the risk metric Cenzic calls a "HARM score" in its Hailstorm scanner. We worried that values could be skewed by particular vulnerabilities, and we still believe that quantifying risk in an arbitrary metric is difficult, and customers might be prone to misuse or misunderstand the score. However, Cenzic addressed the majority of our concerns, informing us of options that can tweak HARM reporting to make it much more flexible than we realized at the time the review was written. Additionally, the prices stated in the original article were incorrect, listing ARC as starting at $26,000; Hailstorm starts at $26,000, and ARC starts at $52,000.

For those interested in the many issues associated with assigning metrics to application security issues, we recommend Andrew Jaquith's Security Metrics: Replacing Fear, Uncertainty, And Doubt (Addison-Wesley, 2007). The section on application security in chapter 3 provides an overview of mechanisms used to score applications and the advantages and disadvantages of each.

Finally, N-Stalker was able to identify and fix the majority of the bugs identified in its review. While the product suffered from a variety of problems, N-Stalker's response and quick release of a new version to address the bugs does bode well. Current customers should look for version 6.0.0.54, which was released on Sept. 25.