Metadata, Analytics To Push Next Wave In Search

Searching and navigating through Google, MSN, Yahoo or other search engines to find specific content has never been easy, and experts agree it will only get worse before it gets better.

Analytics and metadata are likely to become tools to assist people wade through the barrage of text, games, photos, music, and videos now made available through the Web.

Within three to five years, specific categories and clustering engines will likely take into consideration the links users click on, said Suranga Chandratillake, founder and chief technology officer at video search site Blinkx Inc., speaking on a panel at the Digital Hollywood Fall in Santa Monica, Calif. on Tuesday.

Along the same thought process, Manish Chandra, founder and CEO at Santa Clara, Calif., search engine Kaboodle, said user queries must learn from previous searches.

With the enormous amount of content coming online, one of the major improvements that must happen for search relies on shifting toward returning query results based on the user's clicks, rather than just the keywords inserted into the search box, Chandra said.

The problems and challenges of a world where search engines know all about those who use them "is built on a big brother concept, and we start to draw the line," said Steve Marder, CEO at search engine Eurekster. "There are a few companies today that know a lot about all of us and could leverage the data to make the experience better, but there are danger signs," he said.

Marder said during the next few years Internet search uses will see companies integrate metadata, context and better indexes to structure and organized content. This in turn will bring advertising efficiencies to target customers better. Perhaps not one-on-one marketing, but 1,000 to one, he said.

How much stuff is available? Companies are reluctant to estimate, but Cambridge, Mass., Akamai Technologies Inc., which caches content for customers to hands off to consumers seeking content on the Web, delivers tens of billions of daily Internet interactions, at times equaling more than 300 Gigabits per second of content and applications served.

On an average day, the company estimates it delivers more than 2 million hits, or requests per second, for http content through its global network of servers, which is roughly equivalent to serving up content to the entire population of Chicago, simultaneously.

"We are building technology that won't need to know what page you visited, which makes any privacy issues much less threatening," said Kosmix Chief Technology Officer Srinivasan Seshadri, in an interview.

In Seshadri's perfect world, which focuses on personalization as a fundamental tool, categories will become fundamental, and algorithms and metadata will help queries focus in on specific content. "The search model breakdown when it doesn't know what the searcher did in the previous query," he said. "When you ask an engineer for something, he typically doesn't throw back 10 whitepapers and says 'find it yourself'."

In personalization, search engines can take the person's browse history into consideration. Typically, if a person searches on the word "Ram," it's unclear to most search engines whether the person seeks information on random access memory, the animal or the football team. But if algorithms and metadata are embedded to distinguish the query, the search engine will return fewer targeted results, Seshadri said.

Consumers will have one type of search engine to find all content, from text to videos, said Andrew Pacer, chief operating officer at About.com, a New York Times Co. site. As search become more efficient, "it will become more difficult for spammers and link farms to succeed because analytical tools will get better and search engines smarter in the way they filter them out," he said.