Ethics, Bias, and Transparency for People and Machines

Our project aims to develop and implement novel artificial intelligence (AI)/ machine learning (ML) technologies to reduce experimental unethical bias in analysis of imaging data of our studies in our parent, NIH-funded grant (AG064579-02) that identifies mechanisms of neurodegeneration in Alzheimer’s disease . Reducing experimental unethical bias in analyzing imaging data is important because, it improves the reliability and validity of research findings, which makes those findings more useful for translating those findings into the discovery of new disease mechanisms, biomarkers, and therapeutics that work in the clinic. Since the initiation of this grant, we have developed novel approaches to study mechanisms of neurodegeneration with a unique biosensor (Genetically encoded cell death indicator – GEDI) that acutely identifies living neurons at a stage at which they are irreversibly committed to die. Initially, imaging data from these studies involved human curation, which carries some degree of experimental bias that can cause ethical problems in interpretation of data. To reduce experimental bias of our data analysis, we have developed ML and deep neural networks (DNN) and use a subclass of DNN, convolutional neural networks (CNNs) which have mathematical properties particularly adept at Computer Vision. We have developed deep learning (DL) algorithms for detecting neuronal death by constructing a novel quantitative RM pipeline that automatically generates GEDI-curated data to train a CNN without human input. The resulting GEDI-CNN detects neuronal death from images of morphology alone, alleviating the need for any additional use of GEDI in subsequent experiments. We find that the GEDI-CNN learns to detect death in neurons by locating morphology linked to death, despite receiving no explicit supervision toward these features. Uniquely, it detects cell death as a change in nuclear readouts as well as other cellular features, which human curation can’t easily identify. The advances we made in unbiased AI image analysis will be applicable to a large range of ML based imaging studies of other investigators because it focuses on improving how the CNN algorithms are trained to analyze data without the need of humans but with super-human accuracy. In this supplemental application, we will further develop this novel ethical AI technology for studies on neurodegeneration of human 3D brain organoids using GEDI-CNN. We will refine the CNN algorithms to optimize and standardize its widespread use in ethical analysis of live imaging analysis and provide the technology to the scientific community for AI-based imaging research.

Research on artificial intelligence (AI)-driven methods for the prediction, detection, characterization and monitoring of cognitive impairment, including Alzheimer’s Disease Related Dementias and mild cognitive impairment, is burgeoning. However, there is a critical gap in research on the ethical challenges of research involving this technology—an empirical examination of patients’ and clinicians’ views on the research and clinical implementation of the technology. We will expand on existing frameworks on the ethics of such research and clinical use of AI technologies by incorporating patients and clinicians’ perspectives, with a special focus on the views of racial and ethnic minority groups, who are underrepresented in this field of study. We will conduct this work in concert with the parent project, which is developing AI methods to detect cognitive impairment among older adults in primary care using patient voice recordings and electronic medical record data. In the process of conducting this work, we have become aware of various ethical challenges in research and implementation of this technology.

The specific aims are: 1) to identify and characterize the perceptions and concerns of patients and clinicians about research on AI methods for automated assessment of cognitive function in outpatient clinical settings; 2) to integrate this knowledge into an existing ethics framework for AI healthcare applications to advance researchers’ ability to ethically conduct AI technology development and implementation in clinical settings.

We will first interview patients from diverse backgrounds and seek their preferences, including best approaches to informed consent and disclosure of results, to maximize their autonomy in future clinical implementation. Next, we will conduct focus groups with clinicians from different clinical settings to ascertain their expertise and views on the feasibility of potential solutions to mitigate ethical challenges that could hinder adoption. The work will not only inform the ethical conduct of research on AI-cognitive function assessment, but AI research in other areas of healthcare as well.

Our project proposes a novel idea to improve algorithmic fairness in machine learning models by focusing on optimizing overall probability distribution distances, rather than enhancing one specific fairness definition at a time. The advantage of our approach is that the notions of parities are derived metrics of conditional probability distributions inherited from the original, more general subgroup probability distributions. By directly optimizing the distributional distances, different parities metrics are intrinsically incorporated into the optimization problem and can be enhanced simultaneously. We measure algorithmic fairness using the cross-group AUC, which quantifies the probability that a randomly chosen case from the "positive" samples in one protected subgroup is ranked higher than a randomly chosen unit from the "negative" samples in another subgroup. However, optimizing the raw definition of our FairAUC is computationally intensive due to its pairwise and multi-group nature. To address this challenge, we have developed a new optimization strategy that replaces the indicator function with a surrogate loss function, allowing for a more computationally friendly optimization problem without requiring pairwise comparisons. The loss function designed from this metric is differentiable and can be minimized via back-propagation, fitting naturally into deep neural network models and making it compatible with existing models. Our innovative approach offers three major advantages: (1) It naturally considers and optimizes multiple derived fairness metrics, which existing pipelines fail to do. (2) It bypasses the conflicting problem between certain derived fairness notions by optimizing high-level distribution distance rather than derived conditional distributions (fairness metrics). (3) The joint optimization strategy minimizes accuracy loss compared to iterative approaches. By developing a novel optimization procedure to enhance model fairness, our project contributes to the creation of more equitable machine learning models in healthcare, supporting the NIH's mission of enhancing health and reducing the burden of illness for all.

AI models, especially deep neural network (DNN) models, are black boxes to most stakeholders including patients, providers, policymakers, and ethicists; understanding and explaining AI systems is the subject of ongoing research. DNN models are particularly difficult to explain due to the highly complex transformations carried out between the “deep” layers in the DNN. Yet, without understanding what an AI model does, it is impossible for the stakeholders to trust the model or to detect the potentially intended and unintended biases associated with the development or utilization of the model. In the budding body of literature on ethical data science, the topics of trust and bias rank high on the list. Building on our ongoing development of explainable AI approaches, we will design and test explainable AI-based methods to improve the trust of and detect biases in DNN models with Alzheimer's Disease and Alzheimer's Disease Related dementia (ADRD) as the clinical domain of focus. Part of this project is to involve various stakeholders, such as patients, providers, policymakers, and ethicists, in the process of enhancing trust and detecting biases. The potential benefits of explaining AI systems include understanding risk and beneficial factors, exposing underlying biases, and demystifying AI for system users.

Artificial intelligence (AI) has shown exceptional promise in transforming healthcare, shifting physician responsibilities, and enhancing patient-centered care to provide earlier and more accurate diagnoses, and optimizing workflow and administrative tasks. Yet, the application of medical AI into practice has also created a wave of ethical concerns that ought to be identified and addressed. Global and regional legal regulations, including recent guidelines issued by the WHO and European Union, provide guidance on how to stay abreast of an ever-changing world and placate growing concerns and worries about the moral impacts of medical AI on the provision of healthcare delivery. Researchers have yet to unpack the specific ethical challenges in medical AI research at the intersection of AI and biomedical research. These ethical concerns include: autonomy, beneficence, nonmaleficence, and justice (the four principles of bioethics) and AI related ethical challenges such as fairness, safety, transparency, privacy, responsibility, and trust. This project seeks to (1) develop a scale to measure medical AI researchers’ knowledge of, attitudes towards, and past experiences with ethical deliberation in AI research, and (2) develop and test a VR-based, interactive application for education on ethical decision-making medical AI in research. This research project holds the potential to catalyze ethical advancements in the intersection of AI and biomedical research by equipping medical AI researchers with a robust tool to assess their knowledge, attitudes, and experiences related to ethical deliberation. Furthermore, the development of an immersive VR-based educational application offers a pioneering platform to enhance ethical decision-making in medical AI research, ultimately ensuring the responsible and equitable integration of AI technologies into healthcare, leading to improved patient outcomes and ethical standards in the field.

The emergence of Big Data healthcare research has advanced medicine and public health but faces many ethical challenges. An important but under-researched ethical issue is the risk of potential biases prevalent in healthcare datasets (e.g., electronic health records [EHR] data) during the data curation and acquisition cycles. Our parent project aims to develop a machine-learning based predictive model of viral suppression among people with HIV (PWH) using EHR and other relevant data from multiple sources in South Carolina. One important ethical challenge is how to assess the potential biases in the curation, acquisition, and processing of EHR data. In this project we will develop, refine, and pilot test an ethical framework-guided metric tool for assessing bias in EHR datasets. Specifically, we plan to 1) conduct a literature review and concept analysis to develop an ethical framework for unbiased and inclusive Big Data research; 2) create and modify a metric tool to assess potential biases in EHR data-based studies via in-depth interviews of key stakeholders of the parent project; and 3) refine and evaluate the metric tool through a community charette workshop among interdisciplinary scholars (ethics experts and disciplinary experts) and key stakeholders (data curators, data management experts, and data repository administrators; healthcare workers; and PWH) and pilot test it. Already, frameworks for conceptual and ethical analysis have been put forward at different stages in relation to the emerging research, and these have been iteratively refined and integrated with preliminary summaries of results in a way that informs subsequent research.

These iterative processes will help clarify concepts of “bias” and “equity” associated with Big Data healthcare research so we can develop an ethical framework to guide data scientists using EHR data. The project will advance our understanding of bias and equity issues in Big Data healthcare research and develop an ethical framework and a metric tool for assessing bias in EHR-based Big Data studies. We hope this tool helps researchers conducting EHR-based Big Data studies to implement a more thorough assessment and exploration of bias and ensure the ethical development of Big Data healthcare research beyond the parent project.

Artificial intelligence (AI) has the potential to revolutionize the diagnosis and treatment of colorectal cancer. However, AI models are shown to be biased, especially toward Black populations. Models that use racial categories in their development are problematic, and models can learn bias from different sources of data in direct and indirect ways. As a result, biased data sets, which train systems to make decisions, will perpetuate bias in subsequent decisions. To address this problem, this project aims to investigate AI bias in colorectal cancer algorithms and develop meaningful biomarkers while minimizing harm and unintended consequences of AI to Black populations.

Race is not a biological category, but rather a social construct replicating health inequity's impacts. Biological differences attributed to race instead represent socioeconomic differences or results of systemic racism. This project aims to review the clinical literature on colorectal cancer and its related risk factors to understand the separation of biological and socioeconomic factors, how these influence each other, and how this affects AI models and Black populations respectively. We will perform a comprehensive review of risk and race correction factors for colorectal cancer from the lens of race-based medicine and undertake a detailed review of potential race correction factors that are at play for Black patients.

Machine learning (ML) based clinical prediction models (CPMs) have proliferated over the past few years, becoming an increasingly central component of healthcare. These tools show great promise in informing both providers and patients of impending health outcomes, ultimately allowing for greater personalization of patient care. While these offer great value for improved health outcomes, there are varied and significant ethical concerns regarding ML based CPMs. In our parent R01, we are developing approaches to understand ethical aspects of model performance, such as algorithmic bias. This ethical focus is vital, but it leaves unexplored questions of ethical usage of ML based CPMs in the clinical workflow, in planning with patients and caregivers, and in shared decision-making.

In this project, we examine and identify ethical concerns that developers of ML based CPMs need to consider, particularly when it comes to predicting mortality, when these CPMs are put into clinical use. To understand these issues, we use qualitative focus group data collection with various end-user stakeholders - providers, patients, and caregivers - on the ethical concerns regarding using a ML based CPM for mortality for patients undergoing hemodialysis. In data collection, we address issues in trust in CPMs and ML tools, understanding of risk and optimal risk communication; and optimal usage of CPMs to empower patients and promote shared decision-making. We additionally determine how, and to what degree, data scientists are considering or could help to address the issues raised by these end-users. With this, we will then develop practice-oriented guidelines that can be disseminated across ML communities that develop tools and clinical communities that use those tools.

Through this project, we will engage with the important ethical question of how best to communicate mortality risk to patients and providers and how to return this information to data scientists. In doing so, we seek to develop valuable insights that can inform other teams that are developing similar tools.

The use of artificial intelligence (AI) continues to accelerate in biomedical and behavioral research. While the technical advances thus far are significant, concerns have been introduced recently regarding the (unintended) consequences of such techniques. For example, problems related to dataset bias can manifest in multiple ways, including the use of non-representative populations; continued propagation of unrecognized system and process prejudices; and equitable access. Given the potential downstream harm, ethical, legal, and social issues (ELSI) must now be integrated alongside the use of data and AI in biomedical and behavioral research and care delivery. However, best practices for ELSI and ethical AI (ETAI) have yet to fully emerge and there is no standard way of documenting ELSI/ETAI considerations in the development and use of predictive models. Building on our platform, the Predictive Model Index and Exchange Repository (PREMIERE), this award is developing (meta)data standards to document and share information around ELSI/ETAI, directly linking such information as part of a shared predictive model. An interdisciplinary collaboration between UCLA and Penn State University (PSU) that brings together in AI/ML, biomedical informatics, law, ethics, communication, and healthcare, our efforts include: 1) an examination of the ethics of using synthetic datasets, namely through key informant interviews; 2) an exploration of the utility of the establishment of a computational check-list for AI/ML and ELSI, leveraging a broad community of relevant parties (ranging from AI developers/users to individuals with AI-related ethics, policy, or oversight roles); and 3) development and implementation of this checklist as part of PREMIERE. A series of meetings convening national experts is planned, increasing awareness around the use of synthetic datasets, ELSI-focused computational methods and their associated complexities. These activities will result in published recommendations around the use of synthetic datasets and approaches for documenting ELSI/ETAI in the context of predictive ML models.

Artificial Intelligence and Machine Learning (AI/ML) applications are rapidly expanding in fields such as radiation oncology; yet the grand scale of data acquisition and scope of applications strains patient expectations and ethical paradigms in medicine and public health. Current regulatory regimes struggle to keep pace with the rapid pace of development in AI/ML and local health systems vary widely in their capacity to adopt and conduct quality assurance and review for in-house or commercially available AI/ML solutions. In general, the rapid expansion of AI/ML would benefit from the ability to measure patient attitudes and experiences that would enable evidence-based best practices for addressing medical and public health ethical issues such as trust, equity, and assurance, and bioethical principles of autonomy, beneficence, and non-maleficence. In the Parent R01, we are examining public trust in AI/ML as it applies to clinical decision support use cases. The goal of the proposed Supplemental project is to expand these efforts to assess values, attitudes, concerns, and trust of patients to inform policy that better serves people and institutions. Specifically, we propose to develop validated measures of patient attitudes and beliefs about key biomedical and public health ethical principles and issues such as autonomy, beneficence, non-maleficence, trust, equity, and assurance, as they relate to the expected benefit of and comfort with the use of AI/ML in radiation oncology. These ethical issues are multi-dimensional, complex, interrelated, and reliant on context. Our validation procedures will thus include structural equation modeling, which will capture the underlying relationships between variables that measure complex topics and will inform the interpretation and use of the measures. To examine the question of how context is associated with ethical values, we will examine these issues in current radiation oncology use cases: quality assessment (e.g., verifying dosage), outcome predictive models (e.g., predicting fibrosis), treatment predictive models (e.g., therapies), and generation of synthetic images (e.g., using MRI data to generate CT images).

Quantitative imaging (QI), where a numerical/statistical feature is computed from a patient image, is emerging as an important tool for diagnosis and therapy planning. Artificial intelligence (AI)-based QI tools are showing significant promise in this area. However, the measured quantitative value from these tools may also suffer from uncertainty. For the ethical application of AI-based QI tools, this uncertainty should be quantified and then incorporated in the clinical decision-making process, an inference that also emerged from a survey conducted by us across patient advocates (Birch et al, Nature Medicine 2022). Towards addressing this goal, in this proposal, we first propose to develop novel methods to quantify uncertainty of AI-based QI tools. Next, to incorporate the uncertainty of the AI-based QI tool, we propose to develop a questionnaire that will elicit the patient’s risk-value profiles towards treatments. For example, if an AI-based QI tool outputs a quantitative value that indicates aggressive therapy, but with high uncertainty, some patients may be risk averse and prefer to assign high weight to the uncertainty value, while other patients may value the benefits of the treatment and thus assign less weight to that uncertainty. This questionnaire will elicit the patient’s risk- value profiles and help generate personalized recommendations for each patient. The methods will be developed in the context of the highly significant clinical question of guiding therapy response in patients with stage III non-small cell lung cancer (NSCLC). Answering this question will help address a critical, urgent, and unmet need for strategies to personalize the treatment of NSCLC, a disease with high morbidity and mortality rates. This supplement is directly responsive to NOT-OD-22-065 in terms of developing a framework for ethical clinical use of AI. Overall, this project is poised to strongly impact the ethical clinical application of QI for treatment of NSCLC, as well as other cancers, cardiac and neurodegenerative diseases where QI has a role.

Clinical deployment of healthcare AI is still in its early stages. Bridging the gap between the development of innovative AI technologies and their adoption in real-world healthcare settings will require processes for addressing ethical concerns in clinical trial design for AI, especially regarding bias. Leveraging the experience of investigators who designed and conducted randomized control trials for the first FDA-approved autonomous AI medical device for screening diabetic retinopathy, we conducted a qualitative interview study of experts to identify ethical challenges with conducting clinical trials for AI.

In this qualitative interview study that included AI developers, clinicians, ethicists, and AI researchers, we have explored: (1) how research investigators define the clinical and social value for conducting AI-RCTs (2) methodological choices that impact scientific validity and minimizes harm (3) how investigators ensure representative, fair subject selection, inclusion and exclusion criteria (4) measures to safeguard and ensure the benefits and safety of the trial outweighs ethical risks (5) the independent review process to validate study outcomes (6) ensuring AI-RCT participants understand the study risks, benefits, and data use terms, and (7) how patients/ research participants' data and right to withdraw from the study are respected.

Our study found important ethical tensions pertaining to these issues. In the next phase of our work, we will develop expert consensus and a roadmap for how to address ethical concerns with future AI clinical trials.

The goal of this K23 supplement application proposal is to create ethical, disease-specific statistical bias detection for prediction models. The proposal introduces the first two steps of such as system: (1) convene an ethics-driven focus group to identify important demographic factors for which to consider correcting, when applicable; and (2) create a novel bias detection metric, “Selection and Information Bias Exposure and Rank” or SIBER, which details the involvement and relative importance of each demographic factor (selected by the focus group) to the prediction output of existing models. In the first part of the workflow, a multidisciplinary group of ethicists, data scientists, clinicians, and community-based healthcare advocates are asked to attend three 2-hour sessions focused on improving health equity in the disease of interest (e.g., sepsis). At the end of the three sessions, the group is expected to have identified the demographic groupings needed for the algorithmic component. The data from the focus groups will be analyzed using qualitative analytic techniques. Among the results will be the list of demographic groups/labels that are at risk for experiencing healthcare bias. A utility function of each demographic variable will be created to weigh their relative importance in prediction output, but only among those who are deemed at high risk of bias. (The process for determining bias is beyond the scope of this proposal, but will be in future work.) The bias detection system (SIBER) will be exposed to two different sepsis prediction models, one of which being the model deliverable for aim 1 of my K23. The models will determine sepsis risk on unseen data. The proposal will test the ability of SIBER to identify and rank the different demographic factors contributing to wide prediction intervals.

Sepsis is a major health challenge worldwide. Patients admitted with sepsis to the Intensive Care Unit (ICU), have a greater risk for multi-organ dysfunction, increased length of stay (LOS), and death. In this supplement, we evaluate possible causes of uncertainty induced within data for patients who developed sepsis in the Intensive Care Units (ICUs) by utilizing conformal predictions and hypothesis tests. We apply a method which can be described as non-parametric bootstrapping to perform hypothesis tests against pre-trained models to better understand their residual performances in different circumstances. Utilizing these approaches, we aim to better characterize the extent to which a model may be confident of a prediction. In addition to this post-processing approach, we also utilize a series of pre-processing (utilization of machine derived data) and in-processing (adapting loss functions to accommodate equitable training). In current practice, algorithms are often developed to generate a risk score or a classification. In that circumstance, the clinician observer is opaque to the underlying process by which those model-estimated outputs were generated. Additionally, the clinician observer is also removed from the key design decisions and assumptions that were made as part of the model training phase, wherein subtle assumptions, such as how to handle missingness and imputations may be consequential in unforeseen ways. By contrast, in our approach the end-user has the ability to ‘peek’ into the black-box model through the aforementioned method. In such scenario, the clinician observer is empowered to recognize when a model estimated output is unreliable based on broad prediction intervals. Thus, we will derive a novel framework which we term EQUITI that can be used to characterize the degree to which the model is uncertain, due to the influence of bias in the data.

The use of AI/ML analytical tools to predict disease risk, onset and progression, and treatment outcomes is growing and holds promise for improving health outcomes for marginalized health disparities population. Yet, there is indication that people with disabilities—the largest health disparities group in the US—will not be able to reap the benefits of these scientific advancements. One key concern is that disability bias presented in the medical documentation of blind patients will be used in AI/ML models and as shown with other marginalized communities, affect care and reproduce, even worsen, existing health disparities. The worry is amplified for blind patients encountering additional gender and racial marginalization, for whom health disparities are compounded. Assessing whether blind/disability bias—as an independent and intersectional factor—is presented in electronical medical records (EHRs) is thus crucial for AI/ML models to develop equitable analytical tools to improve health outcomes for all. The study is led by an interdisciplinary research team and uses an intersectionality framework and disability community-engaged model to begin closing the gaps. We will: 1) Develop, validate, and disseminate reproducible phenotype definitions for diabetes-related blindness and create cohorts for analyses using the EHRs of diabetes patients (2016-22) from a large urban medical center serving highly diverse racial/ethnic populations; 2) Identify and evaluate a list of blind/disability-related negative patient descriptors in clinical documentation; and 3) Assess the use of disability biased language in EHRs of diabetes patients (blind, nonblind) and if negative descriptors in EHRs varied intersectionally (men/women, Black/White patients). This project has the potential to inform equitable AI/ML models in clinical care, improve health outcomes of an often invisible but large and growing health disparity population, and build a dialogue on disability ethics and equity of AI/ML among clinicians, data scientists, blind adults, and ELSI researchers.

The use of population descriptors to account for ancestry-specific differences in genomics research has been a subject of debate among geneticists, social scientists and ethicists. A major concern includes the potential for descriptors of race, ethnicity, and nationality to be misinterpreted as primarily biological or genetic in nature, when there is broad agreement that such categories are social, historical and political constructs that, in an increasingly mixed US population, correspond poorly with underlying patterns of genetic variation. The US population is highly diverse and recent mixing has resulted in many admixed individuals in the US who carry ancestries beyond their self-identified race. Nevertheless, genome-wide association studies (GWAS) have been biased towards analyzing individuals of European ancestry. The Center for Admixture Science and Technology (CAST) was created to develop novel AI/ML methods to study how the complex relationships between genetics, individual behavior, socioeconomic status, and environmental factors influence health outcomes in all populations, thereby making GWAS more informative for individuals of diverse backgrounds, and enhancing equity in the distribution of benefits from genetics research. The All of Us (AoU) Program includes genetic, health and socioeconomic information on >300,000 participants, of which >50% self-identify with a non-white group, and provides a unique opportunity to identify factors contributing to health disparities in admixed and diverse populations. We investigate the ethical concerns related to local ancestry inference and to a method to perform ancestry-agnostic GWAS, which we will apply to the AoU program. We use Multi-Resolution Haplotyping (MuRHap), a haplotyping method which does not require local ancestry knowledge, to genotype each AoU individual. We perform a GWAS on five blood traits using a combination of haplotypes and SNPs. We calculate polygenic risk scores (PRS) and determine if haplotypes provide a better approach to genetic association studies for admixed and diverse populations. We use qualitative social science approaches to understand the range of perspectives in the literature around the ethics of using race descriptors in GWAS and PRS estimation, and to assess prospects and potential strategies for using ancestry-agnostic approaches to address the ethical pitfalls of population descriptors and lack of diversity in GWAS.

The ethical harms of AI in healthcare (AI-HC), such as biased outcomes, have largely been discovered only after deployment. Previously we developed a qualitative methodology for identifying ethical challenges emerging in the design to deployment process for AI-HC. This approach has been used to guide F.D.A. considerations and relies on several key ideas: multiple stakeholders (e.g. developers, clinicians, patients) are impacted by AI-HC; the design to deployment process involves making a series of decisions; how a stakeholder makes these decisions, or would want these decisions to be made, reflects their underlying values; and, where stakeholder groups disagree—where values collide—are where ethical problems are most likely to emerge.

We use interviews to identify stakeholder values, decisions, and tensions. We develop recommendations for resolving areas of value collision through consensus prioritization, drawing on a panel of national experts in computer science, ethics, and clinical medicine. Then, we generate an ethics “report” making legible ethical considerations identified for the specific AI-HC, and initial approaches to addressing them, to all potential users.

Starting with the parent award (a tool for risk stratification of pulmonary embolism being deployed across multiple clinical sites), we will iteratively apply and refine our ethical evaluation approach. This ethical evaluation process will be part of the initial comprehensive review of clinical AI tools being deployed at our home institution, allowing us to both refine our ethical review process as well as garner a series of case studies of proposed AI deployments, associated identified ethical considerations, strategies to address identified concerns, and feedback on communication of identified ethical considerations to stakeholders.

Consequently, in this research we: (1) identify the ethical challenges encountered with specific AI-HC applications, starting with a tool for risk stratification of pulmonary embolism (parent award), and (2) define prioritized recommendations for resolving identified ethical challenges; prioritize approaches to communicating ethical constraints to users.

Through this research, we will chart a generalizable approach for identifying and addressing ethical challenges with AI-HC as well as provide guidance for how to communicate ethical concerns with future AI-HC.

Accurate diagnosis is an essential step toward ensuring patients receive the right medical care. In patients hospitalized with acute dyspnea (acute shortness of breath), determining the underlying diagnosis is critical because the optimal treatment for patients varies considerably depending on the cause (e.g., diuretics in heart failure versus antibiotics in pneumonia). Artificial Intelligence (AI) tools can analyze clinical and imaging data to help determine a patient’s underlying diagnosis but are susceptible to bias present in training data that may lead to poor generalization performance and systematic inaccuracies in specific patient groups. An important, but often overlooked, source of bias is testing bias. We define disparate censorship as a difference in the frequency of diagnostic testing across patient groups defined by protected attributes. Whether or not a patient is tested may depend on their underlying risk, access to care, and/or clinician judgment (i.e., practitioners decide whether to perform a diagnostic test). Disparities in access or bias in judgment affect the reliability of diagnostic labels across patient subpopulations resulting in disparate censorship which may lead to biased AI tools. This project aims to develop a new theoretical basis for the problem of disparate censorship, utilizing tools from causal analysis to increase our mechanistic understanding of how disparate censorship leads to model performance gaps among different subgroups. We will also conduct empirical analyses to study under which conditions disparate censorship gives rise to performance gaps and to what extent existing approaches can be used to address these gaps. We will also comprehensively evaluate the size and the impact of this problem in clinical settings relevant to the diagnostic evaluation of patients with acute pulmonary and cardiac diseases. Ultimately, this project will result in a better understanding of how differences in testing can lead to model bias and inform efforts to ensure safe and effective deployment of AI models to improve care for all patients.

With the prevalence of big data, algorithms are becoming ubiquitous in society, promising a more customized and equitable approach to various fields. One primary convergence is the application of algorithms for biology, medicine, and health care. Nonetheless, a recent study shows that a widely used algorithm, typical of this industry-wide approach and affecting millions of patients, exhibits significant racial bias. Based on this algorithm given risk score, Black patients tend to have more uncontrolled illnesses than White patients. This is a specific example of a broader issue known as algorithmic bias, wherein algorithms reinstate the cultural biases encoded in the data sets they are trained on. The issue of “algorithmic bias” also involves health disparities in African ancestry. In Aim 1 for this project, the current metrics of the parent RCMI award will be redefined with statistics on the probability distribution space in the spectra of gender, race, and socioeconomic status. Based on the redefined metrics of Aim 1, AI techniques will be advanced in Aim 2 toward precision medicine considering the broad spectra of population and ancestry to reduce the identified biases. Specifically, inference and learning algorithms will be developed on probability spaces with meta-learning via Bayesian optimization with regularization. The ethics involve making sure PIs have relevant IRB approval and CITI training. Another ethical consideration is ensuring data obtained, especially genetic data, although de-identified, does not risk identification due to genetic identifying markers. For dissemination, we will host public seminars and workshops with case studies covering the topics in the interaction among ethics, AI, and health disparities.

This study explores practical, ethical and technical benefits, challenges and incentives for implementing smart contracts: an emerging blockchain-based technology with potential to automatically implement patient data sharing preferences about how and by whom their digital personal health information (dPHI) may be used in the future. Smart contracts are proposed as a potential computational solution to the growing capacity of algorithms to reveal sensitive information from de-identified health data, filling gaps in legal protections (e.g. HIPAA) and consent models (e.g. broad consent) that leave patients vulnerable to unwanted used of their dPHI. This study focuses in particular on dPHI generated by computer perception technologies (e.g. face and voice recognition; geolocation; accelerometry; etc.) by devices and wearables outside of the clinic which have a growing capacity to automatically reveal – and increasingly to predict – potentially sensitive or private emotional and behavioral states. When triangulated (using algorithms) with other clinical data, these digital health data can improve personalized medicine and tailored care. However, when de-identified and shared with or sold to other entities (e.g. data brokers; industry), they may be triangulated across other online data (e.g. social media, purchase or browsing histories) to build profiles of individuals that are highly desirable and monetizable, with potential to be used for purposes not directly linked to patient benefit, potentially exposing patients to risks that are difficult to predict. Bringing together interdisciplinary expertise in bioethics, medical anthropology, decision science/behavioral economics and machine learning, this study draws on in-depth interviews (n=40) with key stakeholders, including patients, physicians, health information exchange experts, technical experts in smart contracts, blockchain and machine learning, as well as scholars in law, policy and ethics, to identify attitudes and technical understandings about potential benefits, challenges, and ethical considerations for using smart contracts to implement patient data sharing preferences for dPHI. Informed by these qualitative findings and using behavioral economics insights, the study’s primary deliverable is to model an optimal “choice architecture” that aligns diverse stakeholders’ goals and incentives to implement smart contracts in ways that promote transparency and patient engagement in data sharing decisions. This study’s findings will provide knowledge and support for collaborations that modernize patient protections within a growing dPHI ecosystem characterized by complex social, economic, technical and legal relations.

Artificial intelligence and machine learning (AI/ML) models are expanding in medicine. AI/ML models are widespread for a variety of data-intensive tasks. The rapid development of AI/ML models has rapidly expanded into new fields and challenges existing ethical frameworks. Bias (e.g., misclassification by race or sex), potential liability (e.g., disagreement between clinicians and AI/ML models), and opaque models (a “black box”) hamper the appropriate and effective use of AI/ML. In this supplemental award, we consider potential ethical challenges and bias of AI/ML we have developed in the field of stroke (cerebrovascular disease), particularly for intracerebral hemorrhage (bleeding into the brain).

We focus on two complications of intracerebral hemorrhage that are predictable with AI/ML: hematoma expansion and seizures. In patients with intracerebral hemorrhage, keeping a small bleed from becoming large (“hematoma expansion”, like an enlarging bruise) reduces disability and death. Treatments are only effective when given quickly (limiting the time to learn about the patient), in certain clinical scenarios (if hematoma expansion is unlikely, the treatment is unlikely to benefit the patient), and have the potential for serious adverse events. AI to predict hematoma expansion would be valuable to select patients for treatment. The second complication we consider are seizures. Seizures occur in about 10-25% of patients with intracerebral hemorrhage in the first week. Seizures lead to more brain damage, reduced consciousness, and worse quality of life at follow-up. Prophylactic anti-seizure medication used to be recommended, however, we found that the side effects (fever, worse cognition) outweighed its potential benefits. AI/ML to predict seizures could lead to more precisely targeted prophylaxis with anti-seizure medication, obviating side effects in many patients.

We are using two methods to discover and remediate potential bias. We are testing AI/ML algorithms for hematoma expansion and seizures to determine if there is differential performance by race, sex, ethnicity, or social determinants. If bias is found, we will adjust models as appropriate to remove bias. Separately, we are qualitatively interviewing clinicians, patients, and proxies (e.g., spouses) regarding their perceptions of AI/ML models to determine their concerns regarding bias, liability, and trust to guide future AI/ML model development and testing.

Providing high quality ethical guidance and resources within the capture and development of human health data for Artificial Intelligence/Machine Learning (Al/ML) applications requires integration and support within the full data generation lifecycle. Data generating projects - used for applications ranging from clinical recruitment, outcomes evaluation or predictive modelling all require navigation of multiple experts, tools, data sources and teams from project initiation and continuing through design, acquisition, provisioning, annotation, and model building to application and evaluation. Each of these stages has different combinations of machine-driven and human-driven methods and expertise, as well as data sources and practices that are dependent on local systems and availability. Stages often cross or overlap data ownership boundaries (e.g., source and destination of clinical or health data generation and capture), data management requirements and guidelines (e.g., availability of secure computing and data exchange, deidentification and protection) and between clinical, operations and research needs. In even relatively simple data generation projects, investigators possessing well-defined, and IRB approved data protocols are challenged with maintaining both the original semantic and syntactic intent from initiation through product, as each individual stage requires translation of the requirements and design into and then out of local frames of reference.

Recent clinical informatics and translational science work has focused on reproducibility within these scientific data generation lifecycles, alignment of FAIR (Findability, Accessibility, Interoperability, and Reusability) principles, and the impact of common data models and tools as contributors to reproducibility. Building on these views of process within the data-generating ecosystem, we believe that there remain core challenges in maintaining linkage and provenance of the ethical frameworks motivating investigators research questions, defining data protocols and framing the application of these projects.

Mobile health applications (mHealth) and digital health sensors, including tools powered by artificial intelligence and machine learning (AI/ML), have the potential to advance equitable and accessible healthcare and support a holistic, preventive model of healthcare. These technologies can support individuals to integrate their daily life activities with healthcare delivery and can help overcome healthcare access barriers such as cost and geography. However, if these technologies are developed without incorporating the values of those intended to benefit from them, they risk amplifying existing structural assumptions in healthcare, resulting in technologies that are unusable, unresponsive, or otherwise not appropriate for the sociocultural context in which people live. Researchers developing mHealth and AI/ML technologies need a robust evidence base about community values to inform technology development, including which health conditions are highest priority and tools to support embedding community values throughout research.

This project builds on longstanding relationships that the University of Washington Institute of Translational Health Sciences (ITHS) has developed with Hispanic and Latinx communities across Washington State. We will examine ethical questions about the potential benefits and burdens of mHealth and AI/ML from the perspective of Hispanic/Latinx-identifying community members and produce a prototype translational resource for mHealth and AI/ML research and application development. This supplement brings together 2 key goals for ITHS: (1) integrating community stakeholders at all stages of translational research, and (2) developing accessible methods, tools, and education for informatics and machine learning. To advance these goals, we aim to: (1) develop community-facing materials conveying key features of potential mHealth applications; (2) describe perspectives of Hispanic/Latinx-identifying community members in Washington State about the use of mHealth and AI/ML; and (3) produce a prototype translational resource to guide mHealth and AI/ML researchers in development of community-responsive technologies. This resource can be refined by future research teams, both within the Clinical and Translational Science Award (CTSA) consortium and beyond, and can be used to guide future work across patient populations and clinical scenarios.