2.3 The importance of developing confidence in Artificial Intelligence (AI) among healthcare workers

Ethical AI encompasses practices that aim to address the individual and societal harms AI might cause.¹² 

In health and care settings, the optimal care of patients and avoidance of harm are paramount, as are endeavours to minimise disparities in patient outcomes between demographic groups, geographic locations and healthcare organisations.

A major limitation in how AI technologies are developed and deployed currently is the potential for negative impact towards certain patient groups, including through biases built into AI models.^13,14

Prevention and mitigation of these biases are critical aspects of ethical AI. Minimising bias and maximising AI performance in real-world settings can be achieved to some extent through using representative data sets and robust evaluation and implementation.

Understanding and critically appraising AI-derived information – essentially, maintaining an appropriate level of confidence – can further assist in identifying potential failure cases of AI technologies, including in relation to bias, which can, in turn, contribute to an ethical AI approach.

Appropriate levels of confidence in AI-derived information can also ensure trust is sustained in patient-clinician relationships. Clinicians will need to be able to explain their reasoning around the use of AI to their patients to maintain informed decision making and patient empowerment.

These factors suggest that critically appraising AI technologies is key to the ethical adoption of AI in health and care settings. Education and training will be essential to improve related knowledge and skills to avoid healthcare workers having inappropriately low or high confidence in AI. The second report outlines suggested pathways for such education and training.