Government Developing Data Mining Tools To Fight Terrorism

Slideshow: Next Generation Defense Technologies (click for larger image and for full slideshow)

Federal intelligence agencies are developing new software that can analyze the communications networks and travel activities of terrorists to help discover relationships between them.

The software being developed by the National Counterterrorism Center (NCTC), called DataSphere, is just one of several projects intelligence agencies developed in 2010 to aid in retrieval and analysis of intelligence information, according to the Office of the Director of National Intelligence's (ODNI's) 2010 Data Mining Report.

The report covers a range of programs currently in development by the ODNI that are aimed at helping the intelligence community more efficiently use data, although the ODNI itself admits that they don't all adhere strictly to the data-mining definition.

For example, rather than mining data, DataSphere uses network analysis tools on existing data about known and suspected terrorists and their associates. It detects patterns in the data that links individuals with events and actions--including identifying a set of individuals that fit the parameters designated in a threat-intelligence communication, according to the report.

Intelligence authorities can then use the information in their investigations of potential terrorist attacks in development.

NCTC is working with the proper authorities--including ODNI's civil liberties and privacy and general counsel offices--to ensure its development of DataSphere is aligned with privacy and civil liberties protections, according to the report.

The Intelligence Advanced Research Projects Activity (IARPA) is another agency exploring innovations in data mining to collect and analyze intelligence information, according to the report. Last year IARPA continued to advance development of two programs it created in 2009.

The Knowledge Discovery and Dissemination (KDD) program is aimed at quickly disseminating information from large, complex, and varied data sets so they can be integrated with other data sets already in use. The project also will create analysis tools that can work across the data sets once they're properly aligned, according to the report. IARPA awarded research contracts for the KDD program last September.

IARPA's Automated Low-level Analysis and Description of Diverse Intelligence, or ALADDIN, meanwhile, is a video-query program aimed at replacing a manual process already in use, according to the report. The program allows intelligence analysts to search large video data sets to quickly and reliably locate clips showing a particular type of event.