Machine Learning and Artificial Intelligence (AI) more broadly have great immediate and future potential for transforming almost all aspects of medicine. However, in many applications, even outside medicine, a lack of transparency in AI applications has become increasingly problematic. This is particularly pronounced where users need to interpret the output of AI systems. Explainable AI (XAI) provides a rationale that allows users to understand why a system has produced a given output. The output can then be interpreted within a given context. One area that is in great need of XAI is that of Clinical Decision Support Systems (CDSSs). These systems support medical practitioners in their clinic decision-making and in the absence of explainability may lead to issues of under or over-reliance. Providing explanations for how recommendations are arrived at will allow practitioners to make more nuanced, and in some cases, life-saving decisions. The need for XAI in CDSS, and the medical field in general, is amplified by the need for ethical and fair decision-making and the fact that AI trained with historical data can be a reinforcement agent of historical actions and biases that should be uncovered. We performed a systematic literature review of work to-date in the application of XAI in CDSS. Tabular data processing XAI-enabled systems are the most common, while XAI-enabled CDSS for text analysis are the least common in literature. There is more interest in developers for the provision of local explanations, while there was almost a balance between post-hoc and ante-hoc explanations, as well as between model-specific and model-agnostic techniques. Studies reported benefits of the use of XAI such as the fact that it could enhance decision confidence for clinicians, or generate the hypothesis about causality, which ultimately leads to increased trustworthiness and acceptability of the system and potential for its incorporation in the clinical workflow. However, we found an overall distinct lack of application of XAI in the context of CDSS and, in particular, a lack of user studies exploring the needs of clinicians. We propose some guidelines for the implementation of XAI in CDSS and explore some opportunities, challenges, and future research needs.

Breast cancer is the most frequently diagnosed cancer in women, with more than 2.1 million new diagnoses worldwide every year. Personalised treatment is critical to optimising outcomes for patients with breast cancer. A major advance in medical practice is the incorporation of Clinical Decision Support Systems (CDSSs) to assist and support healthcare staff in clinical decision-making, thus improving the quality of decisions and overall patient care whilst minimising costs. The usage and availability of CDSSs in breast cancer care in healthcare settings is increasing. However, there may be differences in how particular CDSSs are developed, the information they include, the decisions they recommend, and how they are used in practice. This systematic review examines various CDSSs to determine their availability, intended use, medical characteristics, and expected outputs concerning breast cancer therapeutic decisions, an area that is known to have varying degrees of subjectivity in clinical practice. Utilising the methodology of Kitchenham and Charter, a systematic search of the literature was performed in Springer, Science Direct, Google Scholar, PubMed, ACM, IEEE, and Scopus. An overview of CDSS which supports decision-making in breast cancer treatment is provided along with a critical appraisal of their benefits, limitations, and opportunities for improvement.

Although it is widely assumed that Artificial Intelligence (AI) will revolutionise healthcare in the near future, considerable progress must yet be made in order to gain the trust of healthcare professionals and patients. Improving AI transparency is a promising avenue for addressing such trust issues. However, transparency still lacks maturation and definitions. We seek to answer what challenges do experts and professionals in computing and healthcare identify concerning transparency of AI in healthcare? Here, we examine AI transparency in healthcare from five angles: interpretability, privacy, security, equity, and intellectual property. We respond to this question based on recent literature discussing the transparency of AI in healthcare and on an international online survey we sent to professionals working in computing and healthcare and potentially within AI. We collected responses from 40 professionals around the world. Overall, the survey results and current state of the art suggest key problems are a generalised lack of information available to the general public, a lack of understanding of transparency aspects covered in this work, and a lack of involvement of all stakeholders in the development of AI systems. We propose a set of recommendations, the implementation of which can enhance the transparency of AI in healthcare.

Breast cancer is the most common disease among women, with over $2.1$ million new diagnoses each year worldwide. About $30\%$ of patients initially presenting with early stage disease have a recurrence of cancer within 10 years. Predicting who will have a recurrence and who will not remains challenging, with consequent implications for associated treatment. Artificial intelligence strategies that can predict the risk of recurrence of breast cancer could help breast cancer clinicians avoid ineffective overtreatment. Despite its significance, most breast cancer recurrence datasets are insufficiently large, not publicly available, or imbalanced, making these studies more difficult. This systematic review investigates the role of artificial intelligence in the prediction of breast cancer recurrence. We summarise common techniques, features, training and testing methodologies, metrics, and discuss current challenges relating to implementation in clinical practice. We systematically reviewed works published between 1 January 2011 and 1 November 2021 using the methodology of Kitchenham and Charter. We leveraged Springer, Google Scholar, PubMed, and IEEE search engines. This review found three areas that require further work. First, there is no agreement on artificial intelligence methodologies, feature predictors, or assessment metrics. Second, issues such as sampling strategies, missing data, and class imbalance problems are rarely addressed or discussed. Third, representative datasets for breast cancer recurrence are scarce, which hinders model validation and deployment. We conclude that predicting breast cancer recurrence remains an open problem despite the use of artificial intelligence.