Looking for faces in the Super Bowl crowd
Mar 1, 2001 12:00 PM
The Super Bowl presented an incredible technical challenge. The project included two other venues in addition to the Super Bowl: the NFL experience and Ybor City.
When an image resembled one of the database faces, the highest probability match was brought up on the monitor with the acquired image of the Super Bowl ticketholder.
Steve Rehfeldt's bosses were pleased he was not a Super Bowl fan. Rehfeldt, a computer engineer for Graphco, Newtown, Mass., had a coveted ticket to the annual football event in late January, but he spent most of the game with his back to the field. Sequestered in a control room high above the field and away from the fray, Rehfeldt assisted the Tampa Police and stadium officials as they monitored the crowds at the game using a sophisticated facial recognition technology.
The 2001 Super Bowl, held in Tampa, Fla., left little to chance in the realm of security. A collaborative effort between the Tampa Police department, officials at the Raymond James Stadium the site of the game and a team of CCTV, biometrics and engineering experts, led by Graphco, ensured that the only excitement anyone saw was on the field.
Contracted by stadium officials, Graphco assembled a group of suppliers to implement a system using Viisage's facial recognition technology to identify and isolate potential criminals and expedite their removal from the stadium.
The mainstream press jumped on the use of facial recognition technology at the Super Bowl, many invoking the notion that Big Brother is watching. The headlines blared the story: Secret cameras scanned crowds at Super Bowl for criminals, said the Los Angeles Times. One newsmagazine called the flap Snooper Bowl.
The ability to capture a person's image and consult police records to determine if he or she is a known criminal clearly struck a civil liberty nerve. The Graphco team had foreseen the possibility. Detective Bill Todd of the Tampa police, on duty the day of the big game, was also aware of the concerns such a technology raises over civil liberties. Before deploying the equipment, we sought legal advice. There is no law prohibiting the use of cameras in a public place, Todd says.
The winning team watched from the sidelines
Veltek, Shrewsbury, Mass., provided the hardware, including 32 cameras, and the fiber optics. One mile of fiber-optic cable was installed, running from the nearby Ybor City tourist area to the stadium. Raytheon, Dallas, provided technical support and the thermal imaging equipment for nighttime viewing.
Graphco, an engineering firm, tied it all together, providing the programmatic software. To paraphrase Rehfeldt: Graphco's engineering was wrapped around Viisage's biometric product in the same way a car is wrapped around an engine.
Viisage, Littleton, Mass., offers a facial recognition system that uses recognition algorithms based on the Principal Component Analysis (PCA) theory developed at the Massachusetts Institute of Technology's Media Lab. This technology a pattern recognition system uses a composite face called an eigenface. There are about 128 standard face combinations that encompass all human facial characteristics. An eigenface tries on each of 128 masks until it finds a match.
Rehfeldt explains, The program identifies a face by comparing its encoded eigenface characteristic templates with those in the database and then selecting faces whose templates provide the highest probability match.
The Graphco-led team, anchored by project leader Rehfeldt, worked in tandem with the Tampa police department and stadium officials. Detective Bill Todd was, in his own words, responsible for any crime that occurred at the Super Bowl. He worked with 350 other police officers deployed throughout the complex and 50 plainclothes detectives.
Super security effort required for the Super Bowl
Todd is excited about the biometric crimestopper aid: The facial recognition technology is an extremely fast, technologically advanced version of placing a cop on a corner, giving him a face book of criminals and saying, Pick the criminals out of the crowd and detain them. It's just very fast and accurate.
The Super Bowl presented an incredible technical challenge to Graphco's team. The project included two other venues in addition to the Super Bowl: the NFL experience and Ybor City. The NFL experience was a sports carnival where fans learned how they measured up to the pros in shoe size, speed and strength. Ybor City is Old Town Tampa, a tourist attraction with nightspots. Ybor City has had a CCTV installation for some time, and its cameras were used to monitor the crowds.
In addition, there was the Gasparilla Parade, a mock invasion held annually in Tampa Bay by the pirates of Tampa, which was attended this year by a record 750,000 people. The Bud Bowl, NFL parties and other assorted gala events were also held the week of the Super Bowl.
Says Rehfeldt: The scale of the event and the corresponding installation provided a compelling challenge for us and for everyone involved. It was an unprecedented opportunity to test this technology.
The combined camera and biometric installation was in place for about 10 days for the NFL Experience, which provided a test bed for the stadium staff, police and the Graphco team to finetune the placement of the cameras. It was Graphco's first foray using the facial recognition technology for such a large and high-profile installation. Graphco has worked primarily in the voice recognition arena. The facial recognition product has been tested but is not currently in commercial use.
Concerned first and foremost with public safety, the Tampa police used its judgment in viewing the images brought up on the monitor. Although the cameras permitted the police to view crimes captured by the cameras and apprehend suspects for pick-pocketing and other petty crimes, their real goal was to ensure crowd safety. The Tampa Police were involved in forming the database and determining by threat level who was added to the database.
The goal among law enforcement officials, stadium personnel and game organizers was to prevent any acts of domestic terrorism. The team had contacted organizers of national events such as the 2000 Democratic National Convention for guidance. The database of images reflects the concern for domestic terrorism and the collaborative efforts of various law enforcement agencies, including local and state police and the FBI.
Preventing domestic terrorism
The images captured by the cameras were run through the Viisage engine to compare with the images of known criminals contained in the database. The monitors were in a self-contained room at the stadium and manned by law enforcement agents. When an image resembled one of the database faces, the highest probability match was brought up on the monitor with the acquired image of the Super Bowl ticketholder. The police then examined the two images for similarity and made a determination of the resemblance of the two images and the threat level posed. The facial technology has high-speed capability and pinpoint accuracy, but the process is still anchored by human beings who make a final determination based on a variety of factors.
Says Detective Todd: The Super Bowl attracts opportunists criminals looking to prey on unsuspecting football fans. Our interest was more akin to spotting a guy we knew disrupted a national event and keeping him from entering the stadium.
If the lighting is adequate and the camera angle right, the facial recognition software needs only a few seconds to acquire a positive read of a person.
The threat threshold is adjustable. If the system is set to a 97% level, when a 97% probability match comes back, audible and visible alarms alert monitoring personnel.
After the cameras captured images of the crowd and compared these images with the database, the images were discarded. The cameras did not record the captured images.
According to Barry Hodge, vice president of sales and marketing at Graphco, the cameras were not covert. Prominent signs were displayed at the entrances, and turnstiles alerted patrons that surveillance cameras were in use. The signs might have even been a deterrent to crime, says Hodge. Cameras were placed throughout the stadium, including on the gates of the stadium, along the entranceway and in the stands.
And the winner issecurity
Detective Todd was pleased with the results yielded by the facial recognition technology, and he thinks it can increase the effectiveness of law enforcement agencies. I am very excited, and I think the department is excited about this new technology. It provides a multitude of benefits and has various application possibilities. It was a success at the Super Bowl.
Although some are wary of the capabilities of the camera and biometric system, Rehfeldt chooses to emphasize the positive effects of the facial recognition technology. I believe the technology will be a long-term contributor to civil liberties. In the past, everyone's privacy was violated in similar situations. Care definitely needs to be taken with the power of this technology, but it is non-intrusive and it focuses on the miscreants of society.
Super Bowl XXXV has been billed as the most incident-free game in the history of the football championship game. The score for game day on the facial recognition board was 19 matches in all, with three to four matches occurring at the Super Bowl itself.
It was the first time the Tampa PD deployed such technology and accordingly, they learned some tips for the next time. If you get a capture that is a match, you need to be able to locate the person. It's what we call a choke point. We see the person but he passes too quickly from view and is lost in the crowd, says Todd.
The stadium is considering incorporating the facial recognition technology into its standard security program.
Jeanne Bonner is associate editor of Access Control & Security Systems Integration.