In this chapter

An overview of Chapter 5.

This chapter discusses the challenges of incorporating artificial intelligence (AI) technologies into clinical reasoning and decision making (CRDM), and determining the appropriate level of confidence a clinician can place in AI-derived information (the output provided by an AI model to a clinician) for a case-specific clinical decision. 

Many AI technologies used within health and care settings do not directly affect CRDM (for example technologies that support workflow optimisation and scheduling like appointment booking tools). This chapter is not relevant to these applications of AI, for which confidence is built through trustworthiness, based on the factors described in Chapters 3 and 4. 

In other parts of the report, the term AI ‘user’ encompasses clinical and non-clinical (for example, administrative) users of any AI product used in healthcare. In this chapter, a ‘user’ is specifically a clinician who uses AI technologies to assist with, enhance or perform CRDM that will directly affect patient care. This may include screening, health monitoring, diagnostics, prognostics, treatment stratification, design, optimisation, response monitoring or any other clinical aspect of a patients’ care pathway.

The chapter provides an overview of key aspects of CRDM, addresses the factors affecting confidence in AI-derived information at the point of CRDM, and discusses the challenges of enabling clinicians to know when they have appropriate confidence in AI-derived information.

As described in section 2.2, appropriate confidence in AI-derived information is also supported by confidence derived from the trustworthiness of AI technologies and their implementation, through the factors presented in Chapters 3 and 4.