Data Mine This: Government Challenges Scientists

Government Innovators

The federal technology standards organization is challenging experts to develop advanced text retrieval techniques to create better ways to find information in large digital collections of data.

The National Institute of Standards and Technology has announced the eight tracks--or task topics for data-mining research--for its 21st annual Text Retrieval Conference (TREC), which highlights the latest in data mining research.

NIST is inviting participants to submit applications for the November conference so they can begin data mining document sets related to those tracks beginning in March.

[ Read how data mining is being used to snare terrorists. See Government Developing Data Mining Tools To Fight Terrorism. ]

This year's TREC tracks are as follows:

--Contextual Suggestion, which investigates how to search information that's dependent on context and user interests; --Crowdsourcing, which looks at crowd-based methods for search evaluation and/or developing hybrid systems that leverage both automation and crowd search; --Knowledge Base Acceleration, which aims to improve the efficiency of people who maintain knowledge bases by having the system itself monitor data streams and suggest modifications or extensions; --Legal, which aims to develop search capabilities to meet the needs of attorneys searching through digital document collections; --Medical Records, which promotes research on providing content-based access to the free-text fields of electronic medical records; --Microblog, which looks at search and evaluation methods for information that seeks behaviors in microblogging environments; --Session, which aims to provide ways to query IR systems by sequence and a series of user interactions rather than for single, one-shot queries; and --Web, which looks at Internet-specific retrieval tasks for collections of up to 1 billion Web pages.

Contextual Suggestion and Knowledge Base Acceleration are two new tracks for 2012; the others carried over from the 2011 TREC conference, according to NIST.

Once NIST releases the tracks and data sets to be mined, scientists from academia as well as public and private-sector organizations will focus on improving information retrieval by developing algorithms to find information from the data collections.

Because much of the data-mining research and development is done before the conference, typically TREC attendees use the meetings to discuss their findings.

NIST's work with text retrieval is the leading experimental effort in the field, and it's proven to have a positive effect on current methods for retrieving digital data, according to the organization.

NIST recently had an economic impact study (PDF) prepared that found improvements related to TREC are responsible for about one-third of the Web-search advances between 1999 and 2009. Moreover, these improvements might have saved up to 3 billion hours of time searching on the Web, according to NIST.

Data mining is of particular interest for the federal government, particularly among intelligence agencies that are seeking new ways to identify clues in data for counter-terrorism and other activities.

The National Counterterrorism Center and the Intelligence Advanced Research Projects Agency are among federal organizations that have projects in the works to develop new data-mining software and other technology to search for information in intelligence-related data sets.

NIST is accepting applications for TREC 2012 through Feb. 22.

InformationWeek is conducting our third annual State of Enterprise Storage survey on data management technologies and strategies. Upon completion, you will be eligible to enter a drawing to receive an Apple 32-GB iPod Touch. Take our Enterprise Storage Survey now. Survey ends Jan. 13.