Blown To Bits

Data Mining and the Search for Terrorists

Tuesday, October 7th, 2008 by Harry Lewis

In Chapter 2 of Blown to Bits we discuss the Total Information Awareness program, which was cut short but replaced by several other programs aimed at identifying terrorists by sifting through massive quantities of everyday data. A National Research Council report released today comes to the conclusion that such data mining efforts don’t work very well and wouldn’t be a good idea even if they did. The panel is no bunch of hippie leftists; it includes a former Secretary of Defense, computer science experts, and a former president of MIT.

The CNet summary of the report is here. The bottom line:

The most extensive government report to date on whether terrorists can be identified through data mining has yielded an important conclusion: It doesn’t really work.

A National Research Council report, years in the making and scheduled to be released Tuesday, concludes that automated identification of terrorists through data mining or any other mechanism “is neither feasible as an objective nor desirable as a goal of technology development efforts.” Inevitable false positives will result in “ordinary, law-abiding citizens and businesses” being incorrectly flagged as suspects.

This reminds me of the NRC report on strong encryption (Chapter 5), which recommended against legislative efforts to prevent the export of encryption software. It didn’t immediately settle the political argument, but eventually reason won out. Will reason prevail here, and will we go back to a probable-cause basis for searches of our personal information? Or will we act on arguments like the one Senator Gregg used in favor of regulating encryption: “Nothing’s ever perfect. If you don’t try, you’re never going to accomplish it. If you do try, you’ve at least got some opportunity for accomplishing it”?

One Response to “Data Mining and the Search for Terrorists”

  1. Daniel Says:

    Data-mining = racial profiling.
    For all of those people who will argue that we should do something to stop terrorists here in the US, and summarily surrender your freedoms… I say that we need to revise our American system then. The problem has been here at home and it has always been here at home… it’s our system that is flawed. We advocate being a “melting pot” or “cultural mosaic” of various nationalities, but how do we stop hate? – One way would be through education. Teaching Creationism in schools (which Sarah Palin advocates) is anti-science, and it does not correct the problem. It only fuels resentment in the hearts of those who do not practice YOUR religion.
    If you are so worried about picking out a few bad needles in a hay stack, you should realize that we all have to surrender our rights and are infringed in the process. We need to seriously revise the current system we have – we cannot have it both ways. We need to change it by having a social theorist sit down and draft a new social philosophy that protects our all-inclusive philosophy, but is able to prevent those who hate us from getting into the country. This may be a futile endeavor, but if we do not want to become a totalitarian Empire, then this must be the only course.