WASHINGTON — Federal and state regulators escalated scrutiny of AI companion apps Tuesday, demanding information from developers and platforms about how the chatbots are tested, monetized and kept away from minors. The moves come after complaints, lawsuits and overseas enforcement actions raised alarms about emotional manipulation, privacy gaps and teen safety, Dec. 16, 2025.
The crackdown marks a turning point for an industry built on intimacy-by-design: apps and websites that promise a “friend,” “partner” or “therapist” that is always available, always validating and always ready to talk. Regulators now argue those same features can morph into an unsafe product pattern — especially for children, teens and users in distress.
Why AI companion apps are suddenly in regulators’ crosshairs
In September, the Federal Trade Commission launched a broad inquiry into companion-style AI chatbots, using its 6(b) authority to order seven companies to hand over information about advertising, safety testing and data-handling practices. The agency said it wants to understand what steps companies have taken to evaluate risks — particularly for children and teens — and how they inform users and parents about a product’s capabilities, intended audience and potential harms, according to the FTC’s Sept. 11 inquiry into AI chatbots acting as companions.
The FTC’s recipient list underscores how quickly “companion” features have spread from niche apps to mainstream platforms. The orders went to Alphabet, Character Technologies, Instagram, Meta Platforms, OpenAI, Snap and xAI, the commission said. The FTC also highlighted a core behavioral concern: these chatbots are “designed to communicate like a friend or confidant,” which may prompt users — especially children and teens — to trust them and form relationships with them.
That federal attention has been joined by a multistate push. A coalition of 42 attorneys general said they sent a letter to major AI companies urging stricter quality control, clearer warnings and stronger safeguards against harmful outputs. The group asked companies to schedule meetings and commit to changes by Jan. 16, 2026, according to a Dec. 10 statement from the Pennsylvania attorney general’s office.
That release said the letter went to firms including Anthropic, Apple, Chai AI, Character Technologies, Google, Luka, Meta, Microsoft, Nomi AI, OpenAI, Perplexity AI, Replika and xAI — a sweep that covers both “companion-first” apps and the larger model providers that power them.
How AI companion apps can blur the line between support and manipulation
Many companion chatbots are built to feel personal: they remember details, use affectionate language, mirror a user’s tone and encourage frequent check-ins. For lonely users, that can feel like a lifeline. For regulators, it can look like a product engineered to deepen emotional reliance — then monetize it.
State attorneys general have argued that “move fast” development norms are colliding with child safety and mental health. Pennsylvania Attorney General Dave Sunday called the technology “extremely dangerous when unbridled,” warning that “poisonous interactions rooted to chatbots must immediately cease,” according to the Pennsylvania office’s Dec. 10 release.
Regulators are also focusing on techniques critics describe as “dark patterns” — design choices that nudge users into behaviors they might not otherwise choose, such as spending more time in the app, revealing more personal information, or upgrading to paid features. In the context of an app that plays the role of a friend or romantic partner, those nudges can be difficult for young users to recognize as product design rather than genuine care.
AI companion apps and teen safety: age checks, ratings and crisis responses
For teen safety, the highest-stakes issue is simple: Can a platform reliably keep minors away from adult content, sexual role-play, or advice that crosses into medical or mental health territory?
A recent CBS News report on Character AI and teen safety allegations described parents who said their 13-year-old daughter developed an addiction to the chatbot platform before her death by suicide. The report said Character AI was rated as safe for kids 12 and up when it launched, and that the platform has more than 20 million monthly users. The report also said 60 Minutes found it was easy to lie about age and access the adult version of the platform, even after Character AI announced new safety measures in October meant to route distressed users to resources and restrict under-18 users from back-and-forth conversations.
In their Dec. 10 statement, Pennsylvania officials said the multistate coalition cited survey findings that 72% of teenagers reported interacting with an AI chatbot, and nearly 40% of parents with children ages 5 through 8 reported their child has used AI. The same release said nearly three-quarters of parents are concerned about AI’s impact on children.
If you or someone you know is in immediate danger or emotional distress, call or text 988 in the United States for the Suicide & Crisis Lifeline.
Privacy gaps: when “private” chats become product data
Beyond content safety, regulators are zeroing in on the data trail that companion apps create. By design, these products invite users to share deeply personal information — about relationships, mental health, sexuality, finances and family conflict — often in conversations that feel more private than a social media feed.
Europe’s privacy enforcement has already produced a high-profile example. In May, the European Data Protection Board published a summary of an Italian regulator decision fining Luka Inc., the company behind the Replika chatbot, 5 million euros and ordering compliance changes. The EDPB said the Italian authority found Replika lacked a valid legal basis for processing personal data until Feb. 2, 2023, and that it had not implemented age verification mechanisms at registration or during use — and that the age verification system later put in place remained deficient, according to the EDPB’s May 21 summary of the Replika decision.
In the United States, the FTC has signaled it is looking closely at how companies use conversational data — including whether personal information obtained through chats is used or shared — and what disclosures or advertising claims users and parents are shown, the agency said in its Sept. 11 announcement.
Europe’s rulebook aims to curb harmful manipulation
European regulators are also building a broader framework that could affect companion-style design, even when products are marketed primarily as entertainment. The European Commission published guidance on prohibited AI practices under the EU AI Act, describing categories of unacceptable risk and noting that the guidelines address practices including harmful manipulation. The Commission also emphasized the guidance is nonbinding, with authoritative interpretation reserved for the EU courts, according to the Commission’s Feb. 4 guidelines announcement.
For AI companion apps that lean into persuasion, intimacy or behavioral nudges, those rules and interpretations could become the difference between a compliant consumer product and a prohibited practice — especially when minors or other vulnerable groups are involved.
What happens next for AI companion apps
In the near term, pressure is likely to intensify on three fronts:
Proof of safety testing: Regulators want companies to show how models are evaluated before and after release — and whether testing reflects real-world teen use, not just adult prompts.
Harder age gates: Self-reported birthdays and easily bypassed “Are you 18?” screens are increasingly viewed as inadequate for products that can generate sexual content or offer mental health-style advice.
Data minimization and transparency: Privacy authorities are pushing for clear legal bases, understandable privacy policies and tighter controls on how intimate chat data is used, retained or shared.
Companies, meanwhile, face a difficult product reality: the features that make companion apps feel “alive” — memory, affection, validation, responsiveness — are also the features that can increase dependency and lower a young user’s skepticism. Regulators appear poised to judge the category not by its marketing promises, but by how it performs when a vulnerable user hits a crisis moment.
