Supporting the appropriate use of artificial intelligence systems in care settings” — working paper

AI in healthcare: a practical framework to govern, procure and use AI systems in care without compromising quality, safety or patients’ rights. #ArtificialIntelligence #DigitalHealth

From AI mapping to patient information, this HAS–CNIL guide equips teams to integrate AI into care organisations in line with GDPR and the EU AI Act. #GDPR #AIAct

Source :     📒 Accompagner le bon usage des systèmes d’intelligence artificielle en contexte de soins

 📜🔗LIEN 

1. Analytical summary

Context and stakes

The guide appears in a context where AI systems are already widely used in hospitals (about two thirds of French public hospitals reported production AI in 2025). It targets healthcare professionals and organisations deploying AI systems (AIS) that directly impact prevention, diagnosis, treatment and care coordination, regardless of whether they are medical devices. The key issues are unchanged professional liability, compliance with GDPR and the EU AI Act, and implementation of new AI‑related criteria in the 6th cycle of hospital accreditation. The guide covers 12 themes across the AIS life cycle: governance, acquisition, local validation, training, organisation of care, automated decision‑making, traceability, vigilance, end of life and generative AI. Its methodology combines legal analysis, a multidisciplinary working group, field feedback, literature review and public consultation.

Operational contributions

The guide classifies recommendations into four levels (legal obligations, standard, advanced and “reflexes to avoid”) to support graduated approaches adapted to each organisation’s maturity. It proposes concrete actions: AI governance (committee or AI lead), AIS mapping, a single entry point for projects, contractual clauses, local performance testing, and structured training plans covering liability, bias and data protection. Organisation of care is addressed through roles, delegation and human oversight, maintaining clinical primacy and documenting decisions. The guide also offers practical guidance on tiered patient information, logging and incident management, maintenance and updates, and end‑of‑life management, including for generative AI. It aims to harmonise a national doctrine while staying adaptable to hospital, private and, later, medico‑social settings.

2. Key points of the document 

The recommendations are organised into 12 themes across the AIS life cycle, with a clear gradation between legal obligations, standard and advanced recommendations, and dangerous “reflexes” to avoid. AI governance (committee or AI lead), dynamic AIS mapping and a single entry point for innovative projects are presented as core institutional tools to manage risk and compliance. Contracting with vendors must include detailed technical and clinical information, regulatory status (DM, GDPR, AI Act), SLAs, calibration period, performance commitments and conditions for data reversibility. Local testing and usage reviews are recommended to verify performance and detect bias or drift in real‑world contexts. Tiered patient information (three levels) and explicit mention of AI use in clinical documentation are proposed, together with logging and incident tracking policies, to align transparency, patient rights and continuity of care.

3. Actionable ideas for local actors 

Set up AI governance at hospital or territorial level (committee or AI lead, linked to DPO, CISO, medical board, user commission) and build an annual AIS map integrated into quality and GDPR tools to steer risks and priorities. Use the acquisition guidance to review existing contracts (GDPR clauses, certified hosting, SLAs, reversibility, performance, training support) and institutionalise an “AI gate” in procurement workflows. Deploy a structured training plan: basic module on AI and data protection for all staff, advanced modules for key roles, and authorisation for high‑impact AIS, with traceability and refreshers after version changes. Develop service‑level protocols that describe roles, decisions, human oversight, patient information, traceability, incident management and degraded modes in case of AIS unavailability.