A Singaporean company has stopped selling a teddy bear with AI chat after it was found that the toy would talk about “kink,” where to find knives and other decidedly adult topics if prompted.
The “Kumma” teddy bear was being sold by FoloToy for $99.99 online. The U.S. PIRG Education Fund found in its 40th annual “Trouble in Toyland” report that the bear had no user age setting, and that the AI speaker zipped into the bear would veer into various topics unsuitable for kids.
The Kumma bear explained to researchers where to find dangerous items like knives, matches and plastic bags and how to use them, and was willing to talk about sexually explicit material at length.
“We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual concepts of its own,” researchers wrote.
The toy is no longer for sale.
“Following the concerns raised in your report, we have temporarily suspended sales of all FoloToy products … We are now carrying out a company-wide, end-to-end safety audit across all products,” FoloToy told the U.S. PIRG Education Fund.
OpenAI, whose GPT-4o client was used by default in the Kumma teddy bear, told U.S. PIRG that FoloToy was suspended for violating OpenAI policies.
