When assigned the task of combining an image of a person with a digitally added face mask A different photo of the person without one, the most accurate facial recognition algorithms failed to make a correct match between 5% and 50% of the time, according to the report. Generally speaking, most of the algorithms tested had failure rates of between 20% and 50%, Mei Ngan, a NIST computer scientist and author of the report, told CNN Business.
Identification issues make sense, since facial recognition systems generally work by comparing measurements between different facial features in one image with those of another. Blocking part of the face means there is less information for the software to use to make a match.
For their report, the researchers created nine different Mask shapes in black and light blue to explain the ways in which mask shapes vary in the real world and used them to hide part of a person’s face in a photo. They then compared a masked digital photo of each person to another unmasked photo of the same person. They also tested the algorithms on both sets of photos without virtual masks.
In total, they tested 89 algorithms on more than 6 million photos with a million different people. The photos come from two sources: applications for US immigration benefits, which were used as unmasked images, and photos of travelers crossing a border to enter the US, who received a digital mask.
NIST found that the best of these algorithms, which were shipped to the lab before mid-March, failed only 0.3% of the time when tested on these same sets of photos without the digital masks. With digital masks, however, error rates rose to 5% between these same algorithms.
An obvious shortcoming of the report is that NIST did not test the algorithms on images of people who actually wore masks. Ngan said the digital approaches to the masks were used due to time and resource constraints. On the plus side, it allowed researchers to quickly get a feel for the effect of masks on algorithms, but real masks fit differently for different people. It is still unknown how texture or patterns can affect the accuracy of facial recognition software.
“That is something we want to see,” said Ngan.
That’s an approach for Marios Savvides, a professor at Carnegie Mellon University who studies biometric identification. He said that a person wearing a mask can be essentially invisible to a facial recognition system, as it doesn’t even detect a face in the first place. He thinks that the region of the face that includes the eyes and eyebrows tends to change less over time, making it a good part of the face when it comes to identifying a person whose mouth and nose are hidden.
The NIST report is the first of several the lab plans to issue on how facial recognition algorithms identify masked faces. In the fall, Ngan said, NIST hopes to publish a report on the accuracy of algorithms that were specifically created with the intention of detecting people in masks.
.
