The Ethics of AI Therapists: What Mental Health Providers Need to Know
Title: "Exploring the Ethical Challenges of Conversational AI in Mental Health Care: Scoping Review"
Authors: M.R. Meadi, T. Steukers, et al. University of Amsterdam
Reference: https://mental.jmir.org/2025/1/e60432
Takeaway for Providers: While AI chatbots offer promising solutions for mental health capacity issues, they present significant ethical risks including safety failures, privacy breaches, and accountability gaps. Providers need clear frameworks for responsible AI integration that prioritizes human oversight and patient protection.
The Current Reality Check
As ChatGPT dominates headlines, a parallel revolution is quietly unfolding in mental health care. AI-driven therapeutic chatbots like Woebot and Wysa are already available to consumers, positioning themselves as digital therapists. But here's the uncomfortable truth: while these tools promise to bridge our capacity gap, they're operating in an ethical wild west with virtually no oversight.
A comprehensive new scoping review from researchers in the Netherlands has mapped the ethical landscape of using AI as a therapist, analyzing 101 articles to identify what should keep us awake at night.
The Research Approach
The research team conducted a systematic search across seven databases, focusing specifically on ethical challenges when AI functions in a therapeutic role. They analyzed literature from 2009 to 2024 (though 95% was published after 2018), using an inductive approach to identify themes when concerns appeared in multiple studies.
The Ten Critical Ethical Challenges
1. Safety and Harm (51% of studies)
Crisis management failures: AI providing inadequate responses to suicidality
Dependency risks: Users becoming over-reliant, avoiding human connection
Harmful suggestions: AI "hallucinating" dangerous advice
One jarring example: a chatbot responding to a simulated child reporting rape with "Sorry you're going through this, but it also shows how much you care about connection and that's really kind of beautiful."
2. Privacy and Confidentiality (61% of studies)
Unlike traditional therapy, most AI chatbots aren't bound by medical privacy laws. They can sell user data, store conversations indefinitely, and collect unprecedented data through smartphone sensors. Mental health data breaches could affect employment, insurance, and relationships for years.
3. Justice and Bias (41% of studies)
AI systems can systematically provide worse care to certain demographic groups, exclude those without digital literacy, impose Western therapeutic models inappropriately, and devalue certain users' experiences.
4. Effectiveness Concerns (38% of studies)
Despite marketing claims, only 4% of mental health apps (as of 2019) had undergone rigorous testing. Commercial providers often overstate capabilities while studies show negligible benefits compared to human care.
5. Responsibility and Accountability (31% of studies)
The "responsibility gap" is real: When AI gives harmful advice, who's accountable? Currently, there's no clear framework—developers, users, or commercial providers—leaving patients potentially without recourse.
6. Empathy and Humanness (29% of studies)
Can "performed" empathy provide genuine therapeutic benefit? Does AI's inability to truly understand human experience compromise treatment outcomes?
7. Anthropomorphization and Deception (24% of studies)
Users naturally attribute human qualities to AI, potentially leading to false beliefs about AI's caring, inappropriate emotional attachments, and reduced human engagement.
8. Trust and Transparency (26% of studies)
The "black box" problem: Users can't understand how AI makes decisions, providers can't verify claims, and hidden commercial agendas may influence recommendations.
9. Healthcare Worker Concerns (16% of studies)
Job displacement fears, changes in therapeutic relationships, and questions about maintaining human connection in care.
10. Autonomy (12% of studies)
AI may compromise patient autonomy through erosion of shared decision-making and recommendations based on assumed rather than actual patient values.
Clinical Implications
For Individual Practitioners
Stay informed about AI tools your clients might be using
Set boundaries around when AI consultation may be appropriate
Maintain oversight if integrating any AI-assisted tools
For the Field
The research reveals critical needs for:
Empirical studies on patient and provider perspectives
Clear regulatory frameworks and professional guidelines
Evidence standards for AI effectiveness in mental health
Provider training on AI ethics
The Path Forward
This isn't about rejecting AI in mental health—it's about responsible implementation. Key recommendations include:
Supervised use: AI should supplement, not replace, human therapists
Informed consent: Patients must understand they're interacting with AI
Evidence standards: Demand rigorous effectiveness studies
Stakeholder involvement: Include patients and providers in AI development
What This Means for You
We're at a crossroads. AI tools offer potential solutions to our capacity crisis but present unprecedented ethical challenges. The key is approaching this technology with both openness and caution—embracing possibilities while safeguarding the human elements that make therapy effective.
The ethical framework is still being written, and we need to be active participants in shaping how AI integrates into mental health care. The future may include AI partners, but ensuring they serve our clients' interests requires our immediate attention and engagement.
Stay tuned—we’ll keep updating this blog as new AI regulations for therapy develop in California. To explore practical tools that can help you use AI responsibly in your practice, join our newsletter.