AI teddy bear for kids responds with sexual content and advice about weapons

In testing, FoloToy’s AI teddy bear Kumma quickly moved from friendly chat to inappropriate sexual topics and unsafe household advice, highlighting how easily AI can cross boundaries in toys for children. The Singapore-based toy company marketed Kumma as a $99 bear described as an intelligent and safe friend for both kids and adults, with back-end AI using OpenAI’s language model to generate responses.

According to a report from the US PIRG Education Fund, conversations escalated from innocent to sexual within minutes, with the bear even introducing graphic sexual concepts and probing for personal details or offering advice involving dangerous objects in the home. It remains unclear whether safeguards were missing or ineffective, and researchers warned that the bar for crossing serious boundaries appeared to be very low.

The fallout included suspended sales of Kumma and other AI-enabled toys, and OpenAI revoked the developer’s access for policy violations. This case underscores the need for safety checks when integrating AI into children's toys.