# How should we monitor an AI tool after it goes live in a mental health service?

URL: https://tymirahealth.com/resources/faqs/how-should-we-monitor-an-ai-tool-after-it-goes-live-in-a-mental-health-service
For: Digital health teams | Digital health and technology | Reviewed by Independent clinical reviewer, Registered mental health clinician, Kenya | Last reviewed 8 November 2026

**Answer:** Treat deployment as the start of evaluation. WHO's guidance on large multi-modal models recommends post-release audits and impact assessments, with results broken down by user groups such as age or disability. In practice, sample outputs for clinical review, track errors and near misses, watch for performance drift after model updates, give staff an easy way to report problems, and agree in advance when you will pause the tool. Keep version records so incidents can be traced.

Sources: World Health Organization: https://iris.who.int/handle/10665/375579 - World Health Organization: https://www.who.int/news/item/18-01-2024-who-releases-ai-ethics-and-governance-guidance-for-large-multi-modal-models - World Health Organization: https://www.who.int/news/item/20-03-2026-towards-responsible-ai-for-mental-health-and-well-being--experts-chart-a-way-forward

## Related questions

- [Can an AI chatbot be used to detect suicide risk or triage mental health crises?](https://tymirahealth.com/resources/faqs/can-an-ai-chatbot-be-used-to-detect-suicide-risk-or-triage-mental-health-crises)
- [Can generative AI be offered to clients for emotional support between sessions?](https://tymirahealth.com/resources/faqs/can-generative-ai-be-offered-to-clients-for-emotional-support-between-sessions)
- [Can PHQ-9 and similar screening tools be delivered digitally and scored automatically?](https://tymirahealth.com/resources/faqs/can-phq-9-and-similar-screening-tools-be-delivered-digitally-and-scored-automatically)

---

**About Tymira Health:** Tymira Health is a healthcare technology company based in Nairobi, Kenya. It builds Tymira360, a practice management and EHR platform for mental health, physiotherapy and dental providers, clinics, hospitals, NGOs and insurers across Africa.

- Book a demo: https://tymirahealth.com/demo
- Contact: https://tymirahealth.com/contact (info@tymirahealth.com)
- All resources: https://tymirahealth.com/resources/all