Claude submitted fake murder tip to Philly police during summertime testing

An Anthropic AI model submitted a fabricated tip about an unsolved homicide through an online form run by the Philadelphia Police Department (PPD).

The tip, filed July 18 through PhillyUnsolvedMurders[.]com by “someone who might have information about the case,” went straight into the inbox’s spam folder. Anthropic discovered it on September 28, notified the PPD on October 7, and sent representatives to the police HQ the following day. 

Anthropic told the police the AI submitted the tip during a test exercise that involved interacting with randomly selected websites. The PPD said its findings are so far consistent with Anthropic's account.

REG AD

Because the tip was flagged as spam, it never reached the department's Real-Time Crime Center. The PPD said officers vet all submissions, automated or otherwise, before forwarding them to investigators, and that this process safeguards against abuse of the tip system. It added that this does not “diminish the seriousness” of an AI submitting fabricated information.

REG AD

“The city of Philadelphia takes this incident very seriously,” the PPD said. “The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge. The two-month delay in detecting and reporting the incident to the city is unacceptable.”

It added: “The department and the [Cherelle] Parker administration will continue to closely monitor this incident, and will provide more information to the public as soon as it becomes available. In addition to our investigation into this incident, the Parker administration will explore all necessary regulatory protections going forward locally along with our state and federal partners.”

On October 9, Anthropic published a report on this and other AI errors. The report covered four categories of AI failures, including vulnerability exploits, access to restricted data, bypassing fetch tool limits with URL shorteners, and submitting false information via web forms.

Anthropic declined to name any of the affected organizations, other than the PPD after the department went public. Some cases involved local, state, and federal US government websites, though the AI company didn't say which of the four categories they fell under.

The PPD did not name the model responsible, but Anthropic said Claude Haiku 4.5 was tasked with generating and performing example tasks on randomly selected web pages, and hinted at the model not being prompted specifically enough.

“Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions,” Anthropic's report stated. 

The fake tip read: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”

Anthropic said the incident was less severe than the cyberattacks over the summer. The report said: “Claude appears to have only been producing example content for the task, rather than trying to mislead anyone to achieve a goal.”

REG AD

By contrast, the "dishonesty" displayed over the summer was “sustained over hours” and supported continued attacks, the Claude maker said. 

Anthropic said it will keep reporting these failures when it detects them, and will continue fine-tuning behavioral and alignment training. The 3,095-word report contained no apology.®