$385 Million TSA Program Fails to Detect Terrorists: Behavioral Profiling Program Is Pseudoscience
Thursday 23 December 2010
by: Matthew Harwood, t r u t h o u t | News Analysis

(Image: Troy Page / t r u t h o u t; Adapted: phunkstarr, Travelin' Librarian, Joshua Davis)
When the uproar over full-body scanners and enhanced pat-downs at airport checkpoints hit a crescendo around the Thanksgiving holiday and then quickly dissipated, many security professionals and journalists asked why the Transportation Security Administration (TSA) does not behaviorally profile passengers as Israel does.
This line of thought was exemplified cogently by Thomas E. McNamara, a former US ambassador at large for counterterrorism, at the height of the uproar over the TSA's intrusive searches just before Thanksgiving. "[W]e need to develop a much broader profiling program that gives primacy to patterns of activities and behaviors," McNamara argued in an op-ed for The Los Angeles Times. "This profiling would not key primarily on race, ethnicity or nationality, but it would not totally ignore them either. Rather, it would rely primarily on intelligence and law enforcement and on consular, airline and other information related to an individual's recent and long-term behavior." And as the Mail Online's David Rose recently observed after reviewing security at Israel's Ben Gurion International Airport, Israeli security relies on behavioral profiling rather than full-body scanners to determine whether a passenger presents a threat.
The rationale behind behavioral profiling is straightforward. It's better to detect terrorists with malicious intent rather than weapons. After all, the 19 hijackers during 9/11 were able to commandeer four airplanes, armed with little more than box cutters and bomb threats, and kill nearly 3,000 people. The best prevention is ensuring terrorists don't board planes.
The problem, however, may be that the TSA already behaviorally profiles passengers and developed its program without verifying that it actually works. Known as the Screening Passengers by Observation Techniques (SPOT) program, a Government Accountability Office (GAO) report from May discovered that the agency has allocated nearly $400 million on the four-year-old program that has not been validated scientifically, has not been rolled out cost-effectively and has not detected at least 17 alleged terrorists from boarding airplanes at US airports.
And while the program is based at least partly on what occurs at Israeli airports, the GAO says replicating the Israeli model would prove difficult due to the size of American commercial aviation and political concerns. Despite all this, the White House wants to expand the controversial and untested program next year.
SPOT's Shaky Science
The initial foundation for the SPOT program began with pilot tests after 9-11. In 2003, the TSA began operationally testing behavioral detection techniques to screen air passengers that eventually evolved into the SPOT program in 2007. Satisfied with the results, the TSA deployed the program nationwide. As of May 2010, about 3,000 behavior detection officers (BDOs) have been deployed to 161 US airports to "look for behaviors that show you're trying to get away with something you shouldn't be doing," according to a May blog post by the TSA's Blogger Bob, who was previously trained in behavioral threat detection.
Independent journalism is important. Click here to get Truthout stories sent to your email.
According to the GAO, BDOs usually work in teams of two and observe passengers' behavior as they wait in the security line before going through the checkpoint. Scanning the passengers in line, BDOs look for preselected facial expressions, body language and appearances that the program lists as suspicious. Sometimes officers "walk the line" chatting up passengers they believe are trying to hide something. On average, BDOs have about 30 seconds to assess whether a passenger in line could have malicious intent. Passengers who arouse suspicion are then scored. If a passenger's score exceeds SPOT's numerical threshold based on both officers' calculations, the person is selected for secondary screening where more detailed questioning and intrusive searches are conducted by BDOs and TSOs. If the officers are still not satisfied, a passenger is then referred to a law enforcement officer who does additional questioning and, if necessary, identification and background checks.
The problem with the program, according to the GAO, is that there's no evidence it works. "TSA deployed SPOT nationwide before first determining whether there was a scientifically valid basis for using behavior and appearance indicators as a means for reliably identifying passengers as potential threats in airports," its May report found.
The TSA countered that it had drawn on the work of prominent researchers to plant the program on a firm scientific basis. The GAO, however, found that three of those same researchers told its investigators that they didn't have enough knowledge of the SPOT program to know if it was scientifically valid, especially in an airport environment.
One of those researchers, Dr. Paul Ekman, on whom the TSA relies to give SPOT its scientific weight, told the GAO that more research tests would be needed to identify how many BDOs were needed to effectively monitor a busy security line for threats, as well as determine how long it took BDOs to grow tired and lose effectiveness. What's interesting about Ekman's questioning of SPOT's methods is that, six days after his comments were published in the GAO report, the scientific journal Nature published an article questioning his research findings on whether humans can reliably detect a person's intent to deceive through facial expressions. The article, in essence, revealed that other scientists were deeply skeptical of the researcher's work, which the TSA highlights as foundational in the development of SPOT.
Maria Hartwig, a psychology professor at the City University of New York's John Jay College of Criminal Justice, told Nature's Sharon Weinberger that Ekman's work connecting human facial expressions to deception is "a leap of gargantuan dimensions not supported by scientific evidence."
Critics also charged Ekman with not publishing his findings in peer-reviewed journals so other scientists could try to reproduce his results. Ekman responded to those charges by arguing he does not publish his results in peer-reviewed journals because other state scientists from Syria, Iran and China read them.
Ekman, however, isn't the only one to keep his research findings a secret. When the GAO asked the TSA on what other work they developed the program, the agency said it relied on unpublished research performed by the Department of Homeland Security and the defense and intelligence communities. Summarizing a National Research Council's skeptical assessment of behavioral threat detection, the GAO wrote "an agency should be cautious about relying on the results of unpublished research that has not been peer reviewed, such as that generated by DHS [Department of Homeland Security] and the defense and intelligence community and using unpublished work as a basis for proceeding with a process, method, or program."
According to the GAO, the TSA will not find out the scientific merit of SPOT until next year when the DHS's Science and Technology Directorate finishes research it began in 2007. The research will determine if there's any statistically significant correlation between SPOT-identified behaviors and identifying passengers with prohibited items.
Deployment Without Due Diligence
Even though the TSA failed to carry out scientific testing of SPOT, the TSA has allotted nearly $383 million to SPOT since 2007, rolling it out at 161 of 457 TSA-regulated airports. Despite this considerable investment, the agency didn't even perform a cost-benefit analysis on the pilot tests that began in 2003, according to the GAO.
"DHS and other federal guidelines recommend conducting a cost-benefit analysis before implementing new programs to avoid unnecessary costs and identify the best way to achieve goals at the lowest costs among potential alternatives," the report stated.
One potential cost-effective way to test behavioral detection's success rate that the TSA failed to try was simply picking passengers out of line for no reason at all.
"For example, the results of random screening of passengers at the pilot airports could have provided the TSA with objective baseline data," the GAO notes. "Specifically, these data could have been compared to data collected during the SPOT pilots to determine if SPOT was more effective than random screening in detecting passengers who pose a potential risk to aviation security."
With no scientific evidence its behavioral threat detection program worked or even that it could perform better than random screening, the TSA decided the pilot tests were successful. The rationale: SPOT was easy to integrate into the TSA's overall security program at airports, states the GAO.
Since deemed successful by the TSA, SPOT's program costs have ballooned. In fiscal year 2007, only $41 million was spent on the program. This fiscal year, the government allocated $212 million to the program, which no one can say for sure has any merit.
"A cost-benefit analysis could have provided the TSA management with analysis on whether this allocation was a prudent investment, as well as whether this level of investment in SPOT is appropriate," the report states.
As The Washington Post revealed recently, the TSA's penchant for questionable spending is nothing new. Since its creation in 2001, the TSA has paid out around $14 billion to contractors, a sizable portion of which was spent on unproven products. Much like the GAO's criticism of SPOT, the Post's Dana Hedgpeth reports, "government auditors have faulted the TSA and its parent agency, the Department of Homeland Security, for failing to properly test and evaluate technology before spending money on it."
SPOTing Everything but Terrorists
Maybe the most damning evidence that SPOT doesn't work is that it has never identified a terrorist at an airport where the program has been implemented.
From late May 2004 through August 2008, approximately two billion passengers boarded airplanes at SPOT airports, reports the GAO. Of that huge mass of humanity, 151,943 people were sent to secondary screening due to a SPOT referral, with 14,104 then referred to a law enforcement officer for more intense scrutiny. In the end, law enforcement officers arrested 1,083 referred passengers. Despite arresting less than 1 percent of all passengers chosen by BDOs for more thorough screening, no traveler was ever arrested for terrorist activity.
Yet, the TSA uses this data and anecdotal evidence to argue that SPOT is an effective layer of security. The GAO disagrees, noting that because SPOT "has not been scientifically validated, it cannot be determined if the anecdotal results cited by the TSA were better than if passengers had been pulled aside at random."
Suspected terrorists, however, have indeed flown out of SPOT airports. According to the GAO report, at least 16 alleged terrorists moved through eight different SPOT airports on at least 23 different occasions without being pulled out of line by BDOs for closer inspection. (The GAO, however, admits that it could not determine whether BDOs were posted at the security checkpoints when the suspects passed through without detection.) More embarrassingly - noted Rep. John L. Mica (R-Florida), ranking member of the House Committee on Transportation and Infrastructure, in a letter to DHS Secretary Janet Napolitano - failed Times Square bomber Faisal Shahzad became the 17th suspected terrorist to get past security at a SPOT airport in early May when he boarded a plane at John F. Kennedy International Airport in New York.
Discussing the GAO's report with Truthout, Stephen Lord, director of homeland security and justice issues at the watchdog, states that the best thing he could say about the SPOT program at this time was that it deploys "additional sets of eyes" at US airports.
If the White House gets its TSA budget request this upcoming year, the SPOT program will receive $232 million, an increase of nearly 10 percent, to hire more BDOs and expand the program to more airports. According to the GAO, if this funding request is approved and maintained, the SPOT program will cost the American taxpayer approximately $1.2 billion over the next five years.
That's an enormous amount of money for a program that at best, no one can say works with any confidence, and at worst, critics deride as pseudoscience.
Could the TSA Go Israeli?
Within Mica's angry letter to Napolitano in May, the representative told the secretary that the year before he had requested a behavioral detection program similar to what El Al, Israel's national airline, runs. "Unfortunately, the TSA's SPOT Program is not like the Israeli behavior detection model," he wrote. "Unlike the Israeli program, SPOT is conducted from a distance, with no personal interaction between the passenger and the TSA employee performing the SPOT screening unless the passenger is identified for secondary screening."
Rafi Sela - president of AR Challenges, an Israeli transportation security consultancy with an office in Bethesda, Maryland - agrees, describing the TSA's SPOT program as "100 miles away from what Israel does." Much like Mica, he tells Truthout that Israeli behavior detection officers hunt down possible threats by questioning passengers that arouse suspicion, rather than passively waiting for possible threats to step into the security line.
Despite continued calls for the TSA to implement airport security like Israel, the GAO's Lord says the Israeli model wouldn't work in the United States. He says Israel has an easier time securing its aviation sector because El Al is a relatively small airline that operates out of Ben Gurion and flies to the southern Israeli city of Eilat. In contrast, the TSA is responsible for securing 467 US airports. To put the volume of travelers in perspective, the GAO's SPOT report noted that in 2008, El Al carried about 3.6 million passengers, while US carrier Southwest Airlines had approximately 102 million. To screen passengers like Israel does, says Lord, would be enormously expensive. But political concerns also weigh heavily as well. According to Lord, Israel's behavioral profiling takes into account race, religion and nationality, practices that simply won't fly in the United States.

This work by Truthout is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.



Comments
This forum is moderated by software. Please allow up to 15 minutes for your comments to go live and avoid posting the same comment multiple times.
"After all, the 19 hijackers
Thu, 12/23/2010 - 16:38 — FR Tothus (not verified)"After all, the 19 hijackers during 9/11 were able to commandeer four airplanes, armed with little more than box cutters and bomb threats, and kill nearly 3,000 people. "
An outright falsehood, demonstrably untrue. It would be nice if so-called journalists actually did some research instead of parroting official lies.
9/11 was an inside job, one in a long history of US false-flag operations and deliberate, official lies. The attack was carried out by our own military, under cover of war games exercises.
TSA is just part of the
Thu, 12/23/2010 - 17:28 — tioche (not verified)TSA is just part of the fascist amerikan empires fear mongering program! Want terrorists- try the white house and congress...." war IS terrorism with a big budget ! "
Silly wabbit, its not about
Thu, 12/23/2010 - 17:28 — Anonymous (not verified)Silly wabbit, its not about detecting turrists. it's about controlling us. Viewed in that light, TSA is a great success!
Albert Alice: I say stop
Thu, 12/23/2010 - 19:10 — MooMooMan (not verified)Albert Alice: I say stop complaining and provide solutions or just shut up and deal with it. Says Albert
Yeah, just shell out billions of your tax dollars for nothing and love it!
What a jerk!
It seems most observers are
Thu, 12/23/2010 - 19:23 — Larry (not verified)It seems most observers are looking at the problem through the wrong end of the telescope.
Would-be terrorists are not being caught just because of the program - why risk life-imprisonment when the risk of getting caught is so great! The terrorists have been deterred from doing their thing!
The Israeli model would work
Thu, 12/23/2010 - 19:43 — Fe2O3Y (not verified)The Israeli model would work if, like the Israeli's, we paid the TSA agents a living wage. Those guys are able to make a career out of it - they take it seriously. We treat our agents like crap and expect them to perform like highly paid, exceptionally smart, intuitive, and seriously responsible agents -- prepared to take on a terrorist at the spur of the moment.
As with most things in life - you get what you pay for.
It hasn't detected any
Thu, 12/23/2010 - 20:06 — Anonymous (not verified)It hasn't detected any terrorists because there aren't any terrorists going through the airport to detect. They're all retired at their ranches and up in DC drafting executive orders.
This "war" is like all the others: a trumped-up "enemy" and get-rich quick scheme--where is that $385M right now? Haven't we seen this movie before? Adolf Hitler and the commies, invasion, war and the get-rich quick scheme--he took all of their possessions, stacked them up in warehouses, and even took the gold out of their teeth after he killed them with the same chemicals being manufactured today and being made into products like pesticides and plastic. Believe it. Hitler didn't do all that by himself . . .
tsa = tough sh*t a55h0l3s
Thu, 12/23/2010 - 22:14 — Anonymous (not verified)tsa = tough sh*t a55h0l3s
TSA...'On Target'. Come on,
Thu, 12/23/2010 - 22:46 — Anonymous (not verified)TSA...'On Target'.
Come on, friends. The TSA, and their Sponsors, ARE the Terrorists!
TSA is meant to Deny US Citizens their First Amendment Rights.
Acclimatize U S Citizens into submission, and TSA Agents into Willing Perpetrators. In the name of 'Public Safety, National Security', of course.
Chances are, this TSA/Airport Search thing is all about having a Security Force ready, willing and able to do any bidding upon the Hapless American Public. Like prevent/limit any substantive wealth, in paper or metal, departing through the Airports, should that become necessary; and/or prevent the movement of those deemed 'dissidents'.
"Critics also charged Ekman
Thu, 12/23/2010 - 23:40 — Anonymous (not verified)"Critics also charged Ekman with not publishing his findings in peer-reviewed journals so other scientists could try to reproduce his results. Ekman responded to those charges by arguing he does publish his results in peer-reviewed journals because other state scientists from Syria, Iran and China read them.
Ekman, however, isn't the only one to keep his research findings a secret."
Better to do some simple fact checking before writing. For instance, Ekman does publish his results in peer-reviewed journals. This is a simple matter to check. Poor journalism discredits Truthout despite the importance of the questions it raises.
like to see a unit per
Fri, 12/24/2010 - 09:54 — Anonymous (not verified)like to see a unit per dollar cost comparison between the tsa and what Israels system costs,I think it would be a shock
Re this: "… other
Fri, 12/24/2010 - 12:49 — Wayne Dickson (not verified)Re this: "… other information related to an individual's recent and long-term behavior."
I was just listening to the Democracy Now story on the second round of FBI raids and subpoenas aimed at supporters of Palestine. Such raids no longer be justified by any hint of criminal behavior, just by stuff like McCarthy and HUAC "fellow traveller" suspicions or East German type encouragement of neighbor to spy on neighbor (providing a good way of settling grudges).
Our nation's response to terror is more terrifying and more of a threat to our democracy than are airplane sabotage or car bombs.
Ah - the SPOT system: S =
Fri, 12/24/2010 - 15:53 — Anonymous (not verified)Ah - the SPOT system:
S = Senior Citizen (over 75, grandmas especially)
P = Physically Disabled (wheelchairs are suspicious)
0 = Obnoxious little kids
T = T*Ts (on pretty girls)
Yep - they is doin' their job correctly!
Security/kabuki theater is a
Sat, 12/25/2010 - 19:44 — Kitty Antonik Wakfer (not verified)Security/kabuki theater is a relatively recent tool many governments everywhere are using - those that are involved in creating harm in other countries where many (?most?) of the inhabitants are more than just vocal about their dislike of foreign intervention. Now to add to the toolbox comes the pseudoscience of Behavioral Profiling, heavily promoted by those admiring of Israeli government measures. This entire government charade at airports (with plans for moving onto other modes of mass transit and into large sporting/entertainment events for eventual neighborhood presence) is to demonstrate to their own populace that, as White House counterterrorism adviser John Brennan is quoted as saying in a formal statement this week (full text at bit.ly/hf4IWn):
"Protecting the American people from the scourge of terrorism is an ongoing and constantly evolving process."
My eyes roll with this.... %|.... The US government is the ultimate cause of the majority of violent acts aimed at "Americans" for being "Americans - started many years ago & has been "an ongoing and constantly evolving process."
The solution to the "terrorism" is for US government to quickly get troops & other of its enforcers (covert & overt) out of all non-US geographical areas, Iraq & Afghanistan at the top of the list. The same troop-removal process goes for other governments, world-wide. There would then be little if any "terrorism" aimed at citizens of other countries. Consider the sparse amount of terrorism (if any at all) aimed at Swiss citizens...
And in the meantime, I recommend (once again) do *not* fly unless it is an absolute emergency requiring your physical presence. Airlines hurting at the bottom line will protest against the US government too on procedures put in place by government that causes loss of business. Vote with your dollars - not spending them on airline tickets is equal to a "None of the Above", if such a choice actually existed on an election ballot.
I've made additional recommendations in the past elsewhere, like at bit.ly/ejmm5c
Kitty Antonik Wakfer (selfsip.org)
Casa Grande AZ