Wednesday, December 1, 2010

Bad dogs and other psychics

Writing in the New York Review of Books about Jimmy Carter's recently published White House Diary, Gary Wills observes: 
Jimmy Carter is a better man than his worst enemy would portray him as. And his worst enemy, it turns out, is himself. At least, I cannot imagine a more damaging blow to his reputation than he delivers in White House Diary.
Perhaps the only thing more embarrassing in the book than Carter's egomaniacal obsession with recording for posterity every passing thought and every flare up of the presidential hemorrhoids, Wills points out, is the ex-president's excruciating credulity regarding the abilities of "psychics" to "help us with sensitive intelligence matters."

Carter writes in the book:
We've had several reports of this parapsychology working; one discovered the map coordinates of a site and accurately described a camouflaged missile site. . . . We had a session in the Situation Room concerning a parapsychology project where people can envision what exists at a particular latitude and longitude. . . . The proven results of these exchanges between our intelligence services and parapsychologists raise some of the most intriguing and unanswerable questions of my presidency. They defy logic, but the facts were undeniable.
Anyone who's seen a carnival fortune teller at work or who has a passing acquaintance with the psychic-debunking work of the magician James "The Amazing" Randi will recognize the method by which psychics perform their prodigal feats: sometimes it's as simple as knowing the answer ahead of time; usually it's by offering predictions so vague, and unbounded in advance by agreed-on definitions of what constitutes a correct result, that they can always be claimed as a success after the fact ("something special will happen today"); always it's by relying on the quirk of human psychology that is more impressed by a single correct coincidence than a 100 blank shots.

Exactly the same foibles explain the obsession of the security agencies with other pieces of pseudoscientific malarkey — notably the lie detector, aka polygraph. Study after study has shown that the polygraph, a piece of 19th century electrical-gizmo quackery, is no better than chance at detecting deception; still, agencies like CIA and NSA justify its use by citing its "successes," defined usually as instances in which an employee confronted with a "deceptive" polygraph result suddenly blurts out a confession.

The trouble of course with such after-the-fact justification is (a) a certain number of genuine culprits will be fingered just by chance by any methodology that produces a fair number of positive hits, even one that is purely random and (b) it totally ignores the false-positive rate, that is, the number of perfectly innocent people wrongly fingered. (The latter is not a trivial point; repeatedly it has happened that intelligence officers have lost their security clearances and jobs on the basis of this witch-doctory even though a single scrap of real evidence is never adduced before or after of any wrongdoing on their part. The polygraph methodology is even worse in this regard, in fact, in that cool pathological liars tend to breeze through while conscientious honest people tend to flunk it. One of the striking revelations in the internal NSA history I referred to the other day is that in every one of the serious spy scandals to ht the agency mentioned in the report, the employees who were betraying their nation to the Soviets passed a polygraph test.)

A new field ripe for pseudoscientific mumbo-jumbo in the security biz, I fear, is now detection dogs. The business is absolutely booming for drug-sniffing dogs, bomb-sniffing dogs, money-sniffing dogs, illegally imported meat-and-plant sniffing dogs. But again, this is a perfect opportunity for the same flim flam. There's no real penalty for false positives, and the evidence is beginning to suggest that detection dogs generate a lot of false positives. (Excuse me: I was corrected by a guy from Homeland Security at a recent dog conference where I was giving a talk when I asked him about false-positives; he chastised me that the correct term is "nonproductive responses." And the rate, whatever you call it, is conveniently classified.) Likewise, there's enough people carrying some sort of stuff to keep dogs looking good no matter how scientific or unscientific the whole business is — you finger enough people, some of them are going to be bad guys.

The guy from Homeland Security actually I thought was even more revealing, in his answer to my question about the false-positive rate, when he said that part of the purpose of the dogs was "deterrence." Exactly: if you can make people think it works, it doesn't matter whether it's voodoo or not.