US weapons scanning company Evolv Technology will be banned from making unsupported claims about its products in a proposed settlement with the US government.
The company had claimed its AI scanner, used in the entrances of thousands of US schools, hospitals and stadiums, could detect all weapons.
However BBC investigations showed these claims to be false.
Evolv said it has now reached an agreement with the Federal Trade Commission (FTC), but hasn’t admitted wrongdoing.
The FTC said the action should be a warning to other AI companies.
“The FTC has been clear that claims about technology – including artificial intelligence – need to be backed up”, said Samuel Levine, Director of the Bureau of Consumer Protection.
Evolv Technology’s mission is to replace metal detectors with AI weapons scanners.
It claims to do this with artificial intelligence, which can actively detect concealed weapons like bombs, knives and guns.
The FTC’s complaint alleges the company deceptively advertised its scanners would detect “all weapons”.
In 2022 the BBC outlined some of the impressive claims Evolv’s then CEO had made about the technology.
“Metallic composition, shape, fragmentation – we have tens of thousands of these signatures, for all the weapons that are out there,” chief executive Peter George said in 2021. “All the guns, all the bombs and all the large tactical knives.”
However a BBC report in 2022 showed testing that had found Evolv’s system could not reliably detect guns or bombs – after a freedom of information request from security-analysis company IPVM.
Another BBC report in 2023 revealed details of a stabbing in a New York school where Evolv’s weapons scanners were used.
“It truly, truly does not find knives” the then Superintendent of Utica Schools told the BBC.
And earlier this year the BBC reported that Evolv’s claims that the UK government had tested and approved its technology were also false.
Under the US consumer watchdog’s proposed settlement, Evolv will be banned from making unsupported claims about its products’ ability to detect weapons and give some school customers the option to cancel their contracts.
It will now go to a judge for approval.
A spokesperson for Evolv said that it had worked “collaboratively” with the agency.
“The FTC did not challenge the fundamental effectiveness of our technology and that the resolution does not include any monetary relief” said Mike Ellenbogen, Interim President and CEO of Evolv.
“To be clear, this inquiry was about past marketing language and not our system’s ability to add value to security operations,” he said.
There is concern amongst officials in the US and UK about companies overstating the ability of artificial intelligence to improve products. In some instances it’s unclear whether artificial intelligence is being used at all.
The FTC has recently launched “Operation AI Comply” that targets companies making deceptive claims about AI.