6.0 Artificial Intelligence
Primary domain 6, artificial intelligence, is shown in this section.
AI refers to the ability of machines to mimic human intelligence or behavioural patterns. In practice this often refers to the automation of various activities that involve tasks like finding patterns in data, and making predictions.
Capability statement - I understand that Artificial Intelligence (AI) is an umbrella term used to define digital technologies capable of performing tasks commonly thought to require human intelligence. I am aware AI is common in modern technology and can list uses of AI outside healthcare (for example, voice recognition, recommender systems, self-driving cars, image and video processing)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I can provide examples of AI systems used in healthcare and understand their potential benefits and risks (for example, imaging diagnostics and decision support tools)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware that “machine learning” is a subset of AI and is an umbrella term used to refer to techniques that allow computers to learn from examples/data without being explicitly programmed with step-by-step instructions
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware that all AI applications in healthcare are defined as ‘narrow’ AI that are trained to perform a particular and specific task
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I can identify the contribution that AI could make to healthcare processes in my area of practice and how it has potential to benefit the organization, workforce and patient
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I can articulate the risks and limitations of AI relevant to my professional area and consider them in my use of AI
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I can explain intellectual property issues pertaining to AI models and how this impacts on AI algorithms co-developed between the NHS and commercial providers
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - I can define the sub-fields of AI and machine learning and their key applications (for example, computer vision, audio processing, knowledge representation, natural language processing, expert systems)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I can describe the main types of bias that could affect AI systems (for example, reporting, selection, group attribution, implicit)
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I can take steps to identify and mitigate bias in AI systems, such as designing models inclusively (human centred design approaches), training with representative data and testing for bias
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - I understand the importance of and promote transparency of AI models used within my area of practice. For example, identifying the type of model used, training data, methods and potential model limitations and weaknesses
Archetypes:
- Creator
- Embedder
Capability statement - I understand the benefits and limitations of AI explainability. I keep abreast of research and developments in this area and am aware of the potential impact on confidence in clinical decision making
Archetypes:
- Creator
- Embedder
Capability statement - I understand that Artificial Intelligence (AI) is an umbrella term used to define digital technologies capable of performing tasks commonly thought to require human intelligence. I am aware AI is common in modern technology and can list uses of AI outside healthcare (for example, voice recognition, recommender systems, self-driving cars, image and video processing)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I can provide examples of AI systems used in healthcare and understand their potential benefits and risks (for example, imaging diagnostics and decision support tools)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware that “machine learning” is a subset of AI and is an umbrella term used to refer to techniques that allow computers to learn from examples/data without being explicitly programmed with step-by-step instructions
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware that all AI applications in healthcare are defined as ‘narrow’ AI that are trained to perform a particular and specific task
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I can identify the contribution that AI could make to healthcare processes in my area of practice and how it has potential to benefit the organization, workforce and patient
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I can articulate the risks and limitations of AI relevant to my professional area and consider them in my use of AI
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I can explain intellectual property issues pertaining to AI models and how this impacts on AI algorithms co-developed between the NHS and commercial providers
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - I can define the sub-fields of AI and machine learning and their key applications (for example, computer vision, audio processing, knowledge representation, natural language processing, expert systems)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I can describe the main types of bias that could affect AI systems (for example, reporting, selection, group attribution, implicit)
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I can take steps to identify and mitigate bias in AI systems, such as designing models inclusively (human centred design approaches), training with representative data and testing for bias
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - I understand the importance of and promote transparency of AI models used within my area of practice. For example, identifying the type of model used, training data, methods and potential model limitations and weaknesses
Archetypes:
- Creator
- Embedder
Capability statement - I understand the benefits and limitations of AI explainability. I keep abreast of research and developments in this area and am aware of the potential impact on confidence in clinical decision making
Archetypes:
- Creator
- Embedder
6.1 Machine learning and natural language processing
Machine learning (ML) is a sub-set of AI. Machine learning and natural language processing offer models and techniques for making predictions and classifications with large amounts of data. They can do this without the need for the explicit programming of step by step instructions, learning instead from the data provided.
Capability statement - I understand that machine learning algorithms require large quantities of data to learn from, and must be trained and evaluated using independent sub-sets of the available data
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware of some of the common uses for Natural Language Processing (NLP) methods and text mining within and outside of healthcare (for example, chat bots, speech/virtual assistants, dictation of clinical notes, processing electronic patient records)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware of the use of virtual assistants (for example, Amazon Alexa, Google Assistant) in healthcare to improve accessibility for patients (for example, patients with disabilities) to access health information and can recommend their use to patients where appropriate
Archetypes:
- User
Capability statement - I understand the differences between Artificial Intelligence (AI) for prediction (prospective) and AI for explanation of existing data (retrospective). I am aware of the risk of conflation of these use cases
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
Capability statement - I am familiar with core concepts and methodologies used in the field of Machine Learning (ML) (for example, data science, statistics, mathematics, computer programming)
Archetypes:
- Creator
- Embedder
Capability statement - I am aware of different learning methods and their suitability for a clinical task based on the available dataset (for example, supervised, unsupervised, reinforcement learning)
Archetypes:
- Creator
- Embedder
Capability statement - I am familiar with model evaluation metrics (for example, sensitivity, precision, recall, F1 score, Area under Receiver Operator Characteristic Curve (AUROC), Precision Recall Curve (PRC), mean error, mean absolute error, calibration) presented in academic papers and can use these to evaluate machine learning algorithms
Archetypes:
- Creator
- Embedder
Capability statement - I am confident in critically appraising literature regarding performance and validation of an AI solution for my area of expertise, comparing it to alternatives and to the current standard of care
Archetypes:
- Creator
- Embedder
Capability statement - I can critically assess the model development process of a machine learning solution (for example, feature labelling/extraction, dimensionality reduction, normalisation, model selection and tuning hyper-parameters)
Archetypes:
- Creator
- Embedder
Capability statement - I understand that machine learning algorithms require large quantities of data to learn from, and must be trained and evaluated using independent sub-sets of the available data
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware of some of the common uses for Natural Language Processing (NLP) methods and text mining within and outside of healthcare (for example, chat bots, speech/virtual assistants, dictation of clinical notes, processing electronic patient records)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware of the use of virtual assistants (for example, Amazon Alexa, Google Assistant) in healthcare to improve accessibility for patients (for example, patients with disabilities) to access health information and can recommend their use to patients where appropriate
Archetypes:
- User
Capability statement - I understand the differences between Artificial Intelligence (AI) for prediction (prospective) and AI for explanation of existing data (retrospective). I am aware of the risk of conflation of these use cases
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
Capability statement - I am familiar with core concepts and methodologies used in the field of Machine Learning (ML) (for example, data science, statistics, mathematics, computer programming)
Archetypes:
- Creator
- Embedder
Capability statement - I am aware of different learning methods and their suitability for a clinical task based on the available dataset (for example, supervised, unsupervised, reinforcement learning)
Archetypes:
- Creator
- Embedder
Capability statement - I am familiar with model evaluation metrics (for example, sensitivity, precision, recall, F1 score, Area under Receiver Operator Characteristic Curve (AUROC), Precision Recall Curve (PRC), mean error, mean absolute error, calibration) presented in academic papers and can use these to evaluate machine learning algorithms
Archetypes:
- Creator
- Embedder
Capability statement - I am confident in critically appraising literature regarding performance and validation of an AI solution for my area of expertise, comparing it to alternatives and to the current standard of care
Archetypes:
- Creator
- Embedder
Capability statement - I can critically assess the model development process of a machine learning solution (for example, feature labelling/extraction, dimensionality reduction, normalisation, model selection and tuning hyper-parameters)
Archetypes:
- Creator
- Embedder
6.2 Using and implementing AI systems
AI systems often have to be tailored for specific situations and uses. In the healthcare environment it is less likely that processes will be entirely automated. This often involves humans working with AI systems (human in the loop) where AI systems automate some (usually routine or repetitive) tasks to free up time for specialists to focus on other areas. This sees a closer working relationship between human experts and AI systems.
Capability statement - I am able to use Artificial Intelligence (AI) systems confidently to assist me to improve task efficiency while maintaining quality and safety
Archetypes:
- User
Capability statement - I know how to respond if an AI system fails or is inaccessible and can initiate an alternative process to maintain effective service provision
Archetypes:
- Embedder
- User
Capability statement - I understand the importance of sharing learning following failures of an AI system, to improve systems and practice
Archetypes:
- Creator
- Embedder
- User
Capability statement - I am aware of the limitations of AI systems and how to respond when AI derived information contradicts my clinical/professional intuition. I retain a ‘critical eye’ and am aware of how AI may influence my decision making
Archetypes:
- User
Capability statement - I can justify the use of AI in specific clinical scenarios and know when it is and isn’t appropriate to implement an AI solution based on desired outcomes, potential risks and organisational goals
Archetypes:
- Driver
- Embedder
Capability statement - I actively maintain my clinical knowledge and skills to ensure that my clinical performance is not adversely affected by de-skilling resulting from using AI
Archetypes:
- User
Capability statement - I am aware of the various stages of implementing an AI system, including risk assessment, interoperability, workflow integration, validation and verification, user training, on-going monitoring and model iteration
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - I can set thresholds for monitoring patients using AI enabled decision support systems for chronic health conditions to generate alerts to initiate appropriate action (for example, call patients in for review, alter treatment)
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I am confident at selecting suitable AI methods for given use cases
Archetypes:
- Creator
- Embedder
Capability statement - I understand the potential benefits and limitations of data augmentation, including data simulation and synthesis techniques when there is little available data for training AI systems. I can design testing using real-world data to ensure robustness of AI models trained in this way
Archetypes:
- Creator
- Embedder
Capability statement - I can effectively cost AI-driven solutions taking into account factors such as initial set up, workforce, maintenance and other running costs (for example, cloud storage costs), balancing these against potential efficiency savings
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - I am able to use Artificial Intelligence (AI) systems confidently to assist me to improve task efficiency while maintaining quality and safety
Archetypes:
- User
Capability statement - I know how to respond if an AI system fails or is inaccessible and can initiate an alternative process to maintain effective service provision
Archetypes:
- Embedder
- User
Capability statement - I understand the importance of sharing learning following failures of an AI system, to improve systems and practice
Archetypes:
- Creator
- Embedder
- User
Capability statement - I am aware of the limitations of AI systems and how to respond when AI derived information contradicts my clinical/professional intuition. I retain a ‘critical eye’ and am aware of how AI may influence my decision making
Archetypes:
- User
Capability statement - I can justify the use of AI in specific clinical scenarios and know when it is and isn’t appropriate to implement an AI solution based on desired outcomes, potential risks and organisational goals
Archetypes:
- Driver
- Embedder
Capability statement - I actively maintain my clinical knowledge and skills to ensure that my clinical performance is not adversely affected by de-skilling resulting from using AI
Archetypes:
- User
Capability statement - I am aware of the various stages of implementing an AI system, including risk assessment, interoperability, workflow integration, validation and verification, user training, on-going monitoring and model iteration
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - I can set thresholds for monitoring patients using AI enabled decision support systems for chronic health conditions to generate alerts to initiate appropriate action (for example, call patients in for review, alter treatment)
Archetypes:
- Driver
- Creator
- Embedder
- User
Capability statement - I am confident at selecting suitable AI methods for given use cases
Archetypes:
- Creator
- Embedder
Capability statement - I understand the potential benefits and limitations of data augmentation, including data simulation and synthesis techniques when there is little available data for training AI systems. I can design testing using real-world data to ensure robustness of AI models trained in this way
Archetypes:
- Creator
- Embedder
Capability statement - I can effectively cost AI-driven solutions taking into account factors such as initial set up, workforce, maintenance and other running costs (for example, cloud storage costs), balancing these against potential efficiency savings
Archetypes:
- Driver
- Creator
- Embedder
6.3 Evaluating AI systems
Once AI systems have been implemented they need to be evaluated to ensure they are safe and fit for purpose. Algorithmic drift can occur prompting the need for continual readjustment. The system also has to be evaluated in the local context to ensure it is configured correctly and working optimally for the chosen goals.
Capability statement - I can explain the difference between internal validation, external validation, local evaluation and prospective clinical evaluation of Artificial Intelligence (AI) technology and their relevance to clinical performance. I am aware of recommended standards for validation of different types of AI technologies used in healthcare (such as the National Institute for Health and Care Excellent (NICE) evidence standards framework for the digital health technologies)
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - When evaluating an AI system for use in my professional workflow, I can compare its performance against the expected standards in my professional area of practice
Archetypes:
- Driver
- Embedder
- User
Capability statement - I am aware of the challenges of bias and generalisability for AI algorithms (the ability to perform well in a different demographic group or clinical context to that used for evaluation). I am able to evaluate the publicly available evidence supporting an AI tool and identify the need for further evaluation in my local setting
Archetypes:
- Driver
- Embedder
Capability statement - I am aware of the potential ways in which use of an AI solution may affect human decision makers (for example, human cognitive biases around the use of AI and AI derived information, such as automation, anchoring and confirmation biases)
Archetypes:
- Driver
- Embedder
Capability statement - When commissioning AI technologies, I can discuss the requirements for safety testing (for example, user acceptance testing, quality assurance testing), in addition to algorithm performance evaluation
Archetypes:
- Driver
- Embedder
Capability statement - To improve performance of AI systems—I am aware that ‘optimisation’ can be used to discover a sub-set of potential ‘best choices’ of model once some AI analysis has been carried out
Archetypes:
- Driver
- Embedder
Capability statement - I can discuss the clinical validation standards and approval phases for healthcare AI models and understand the importance of continual post deployment surveillance
Archetypes:
- Driver
- Embedder
Capability statement - I am aware of recommendations that for evaluating and reporting of AI interventions in clinical trials (for example, SPIRIT-AI[27], CONSORT-AI[28]) and can apply these where appropriate
Archetypes:
- Driver
- Embedder
Capability statement - I can communicate potential benefits (improved consistency, availability, speed, efficiency) and challenges (for example, model explainability, biases, model under/over fitting) of using AI systems in healthcare to various stakeholders
Archetypes:
- Driver
- Embedder
Capability statement - I am able to carry out post market surveillance and ongoing clinical monitoring to determine if the system is still meeting required needs and to identify model decay (for example, the tendency for AI model performance to drop over time as (for example, the tendency for AI model performance to drop over time as data and patient characteristics change, requiring models to be regularly updated)
Archetypes:
- Driver
- Embedder
Capability statement - I can explain the difference between internal validation, external validation, local evaluation and prospective clinical evaluation of Artificial Intelligence (AI) technology and their relevance to clinical performance. I am aware of recommended standards for validation of different types of AI technologies used in healthcare (such as the National Institute for Health and Care Excellent (NICE) evidence standards framework for the digital health technologies)
Archetypes:
- Driver
- Creator
- Embedder
Capability statement - When evaluating an AI system for use in my professional workflow, I can compare its performance against the expected standards in my professional area of practice
Archetypes:
- Driver
- Embedder
- User
Capability statement - I am aware of the challenges of bias and generalisability for AI algorithms (the ability to perform well in a different demographic group or clinical context to that used for evaluation). I am able to evaluate the publicly available evidence supporting an AI tool and identify the need for further evaluation in my local setting
Archetypes:
- Driver
- Embedder
Capability statement - I am aware of the potential ways in which use of an AI solution may affect human decision makers (for example, human cognitive biases around the use of AI and AI derived information, such as automation, anchoring and confirmation biases)
Archetypes:
- Driver
- Embedder
Capability statement - When commissioning AI technologies, I can discuss the requirements for safety testing (for example, user acceptance testing, quality assurance testing), in addition to algorithm performance evaluation
Archetypes:
- Driver
- Embedder
Capability statement - To improve performance of AI systems—I am aware that ‘optimisation’ can be used to discover a sub-set of potential ‘best choices’ of model once some AI analysis has been carried out
Archetypes:
- Driver
- Embedder
Capability statement - I can discuss the clinical validation standards and approval phases for healthcare AI models and understand the importance of continual post deployment surveillance
Archetypes:
- Driver
- Embedder
Capability statement - I am aware of recommendations that for evaluating and reporting of AI interventions in clinical trials (for example, SPIRIT-AI[27], CONSORT-AI[28]) and can apply these where appropriate
Archetypes:
- Driver
- Embedder
Capability statement - I can communicate potential benefits (improved consistency, availability, speed, efficiency) and challenges (for example, model explainability, biases, model under/over fitting) of using AI systems in healthcare to various stakeholders
Archetypes:
- Driver
- Embedder
Capability statement - I am able to carry out post market surveillance and ongoing clinical monitoring to determine if the system is still meeting required needs and to identify model decay (for example, the tendency for AI model performance to drop over time as (for example, the tendency for AI model performance to drop over time as data and patient characteristics change, requiring models to be regularly updated)
Archetypes:
- Driver
- Embedder
6.4 Robotics
Robotics builds on many of the AI technologies and represents the intersections of these areas. Robots are being used for routine cleaning and disinfecting procedures, logistical processes, high risk areas (for example, highly infectious individuals) and surgical tasks through to providing companions for the elderly. Robotic technology takes on various forms and applications and is set to increase in use in the near future as the technology improves and the cost reduces.
Capability statement - I am aware of the use of robotic technology for healthcare and can cite examples of the use of robots for health, medical and social care (for example, social companion robots, surgical robots, room disinfectant robots, nanotechnology)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware that robotics comprises an intersection of most areas in Artificial Intelligence (AI) (for example, cognitive modelling, computer vision, natural language processing, machine learning and affective computing)
Archetypes:
- Driver
- Creator
Capability statement - I am aware of the uses of telepresence robots to carry out basic procedures (for example, temperature monitoring) in highly infectious patients and for use with elderly to support independence or for remote care and assessment
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I know where and how to access training (local or manufacturer provided) on robotic technology that I am expected to use
Archetypes:
- Embedder
- User
Capability statement - In my domain, I can work confidently with robotic technology and recognise the limits of such technology and when to override or desist use (for example, safety reasons, malfunction)
Archetypes:
- User
Capability statement - I am responsible for demystifying the use of robotics and democratising its use and integration within appropriate pathways
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am confident in critically appraising literature regarding robotics for my area of expertise in terms of their applications, limitations, required resources and suitability
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - As someone who instructs others to use robot-assisted surgery - I am able to identify and agree clear training goals with those I instruct
Archetypes:
- Driver
- Creator
Capability statement - As someone involved in procurement of robotic technology/solutions, I can effectively cost a solution taking into account factors such as initial set up, maintenance and other running costs
Archetypes:
- Driver
Capability statement - I am aware of the use of robotic technology for healthcare and can cite examples of the use of robots for health, medical and social care (for example, social companion robots, surgical robots, room disinfectant robots, nanotechnology)
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am aware that robotics comprises an intersection of most areas in Artificial Intelligence (AI) (for example, cognitive modelling, computer vision, natural language processing, machine learning and affective computing)
Archetypes:
- Driver
- Creator
Capability statement - I am aware of the uses of telepresence robots to carry out basic procedures (for example, temperature monitoring) in highly infectious patients and for use with elderly to support independence or for remote care and assessment
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I know where and how to access training (local or manufacturer provided) on robotic technology that I am expected to use
Archetypes:
- Embedder
- User
Capability statement - In my domain, I can work confidently with robotic technology and recognise the limits of such technology and when to override or desist use (for example, safety reasons, malfunction)
Archetypes:
- User
Capability statement - I am responsible for demystifying the use of robotics and democratising its use and integration within appropriate pathways
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - I am confident in critically appraising literature regarding robotics for my area of expertise in terms of their applications, limitations, required resources and suitability
Archetypes:
- Shaper
- Driver
- Creator
- Embedder
- User
Capability statement - As someone who instructs others to use robot-assisted surgery - I am able to identify and agree clear training goals with those I instruct
Archetypes:
- Driver
- Creator
Capability statement - As someone involved in procurement of robotic technology/solutions, I can effectively cost a solution taking into account factors such as initial set up, maintenance and other running costs
Archetypes:
- Driver
Page last reviewed: 14 February 2023
Next review due: 20 February 2024