DataScience@NIH Blog

A physician-epidemiologist, I arrived at NIH last year after more than 22 years as a professor of population medicine and nutrition at Harvard University. During those years I worked on several large cohort studies and clinical trials that gave me insight into the importance of data quality—and in some cases quantity—for answering meaningful questions about health across the life course.

I was recruited by NIH to direct the nationwide Environmental influences on Child Health Outcomes (ECHO) program, the mission of which is to enhance the health of children for generations to come. In this new initiative, we face many challenges to ensure data quality and quantity—but tackling challenges like those is one of the reasons I love my job.

ECHO’s observational component (ECHO also contains a clinical trials network) is knitting data from 83 existing, ongoing cohort studies of mothers and children into a single ECHO-wide cohort comprising ~50,000 kids. Leveraging the large number of multiple cohorts will allow us to answer important questions about how a broad range of early environmental factors — including geographic, physical, chemical, social, behavioral, and biological—affect child health. These cohort studies focus on four key pediatric outcomes that have a high public health impact: pre-, peri-, and postnatal outcomes; upper and lower airway conditions; obesity; and neurodevelopment. There’s an innovative fifth outcome, positive health, which reflects the positive attributes of healthy growth and development. 

In putting together the ECHO-wide cohort, we’re paying attention to the whole life cycle of data, from collection to curation, cleaning, harmonization, management, access, and long-term storage. Our overarching challenge is how to integrate existing and new data from cohorts that started at different times in the life course, in different decades, with different data collection approaches and from different data sources, so that the whole is greater than the sum of the parts. And, yes, that is a complicated process.

One of our first efforts is to have the cohorts collect new data in as standardized a fashion as possible. “As possible” is the operative phrase, given the cohorts’ different populations, foci, and cultures. Therefore, the ECHO-wide Data Collection Protocol contains not only essential and recommended data elements (for each of several life course periods), but also preferred and acceptable measures for each element. 

Using the expertise of our cohort investigators and Person-Reported Outcomes Core, ECHO is making every effort on the front end, through existing evidence and validation studies, to ensure that cohorts collect harmonizable data. On the back end, our Data Analysis Center is creating novel approaches, e.g., accounting for missing data for missing data, in order to harmonize in the analysis phase. 

Employing our guiding principles of teamwork, impact, responsibility, and value, and operating within NIH guidelines, the ECHO investigators also developed and ratified data sharing and biospecimen use policies. These policies will facilitate widespread use of the ECHO-wide cohort data and at the same time ensure security of access so that data analysts preserve privacy and confidentiality. 

Working in concert with these two policies is our publications policy, which specifies how investigators propose and approve ideas for analyses, including appropriate attention to equity, validity, and reproducibility, which accompany cloud-based tools that limit data to only those needed for the analysis. Striking the right balance between the benefits of broad usage, on the one hand, and disclosure, conflict of interest, and other risks on the other, is an ongoing concern, especially given that ECHO is forging new territory in many of these areas.  That’s why we’re committed to regular evaluations of the protocol, policies, and strategies for protection of human subjects, with course corrections as needed. 

Speaking of data analyses, we recognize that the main ECHO cohorts address etiologic questions rather than prediction. In other words, “to what extent—and how—does x cause y?,”  rather than, “can I predict y using values of a, b, c, …and x?”  That means that conceptual models, taking care to distinguish confounders, mediators, and moderators, are crucial. 

In addition, data analyses from the ECHO-wide cohort face another threat to validity that researchers rarely have to consider. Every cohort has a different source population and different inclusion/exclusion criteria. That means that selection factors differ across the cohorts, and one can’t be certain that data from one are easily combinable with others. Just another challenge for our crack Data Analysis Center.

In ECHO, we are just starting to amass the large amount of observational data that we anticipate will ultimately inform practices, programs, and policies to improve the health of children and adolescents. We haven’t yet tackled the vital issue of what happens to all of these data down the road.

Given that ECHO is slated for seven years, and we’ve just entered the second year, we have some time to figure that out. But not too much time, as planning ahead—for all phases of the life cycle of a datum—is the name of the game. 

I hope that you now appreciate some of the ways that ECHO aims to be a model of how to maximize the value of data from a large, multi-cohort consortium to the scientific and general community. I invite you to learn more about the ECHO program.

---
About the Author:

Matthew W. Gillman, M.D. joined the National Institutes of Health on July 5, 2016 as the inaugural director of the Environmental influences on Child Health Outcomes (ECHO) Program. As a nationwide research program, ECHO aims to conduct impactful observational and intervention studies to assess the effects of a broad array of early environmental influences on child health and development. Dr. Gillman joined NIH from Harvard Medical School where he was a professor of population medicine and a professor of nutrition at Harvard School of Public Health. His background is in the fields of epidemiology, pediatrics, and internal medicine. He has extensive experience with cohort studies, having served as an investigator on several large, high-profile studies such as Project Viva, the Growing Up Today Study, PROBIT, and the Framingham Heart Study.

America’s opioid crisis isn’t just about startling statistics. It also involves heart-breaking personal losses.

Here’s an example that involves both. After speaking with two parents, each of whom had lost a child to opioid overdose, NIH Staff Scientist Dr. Vojtech Huser was emotionally moved—and more committed than ever to his research.

Dr. Huser, who works at the Lister Hill Center, a research arm of the National Library of Medicine (NLM), saw how the work done by his team was having a direct and meaningful impact on the many people currently suffering because of the opioid epidemic. He met the parents at the recent US Department of Health and Human Services (HHS) Office of the Chief Technology Officer (CTO) hosted Opioid Symposium and Code-a-Thon.

The HHS Code-a-Thon included 50 teams of three to five people each. These teams worked for two solid days—their dedication demonstrated when many stayed overnight in HHS headquarters—to develop resources to aid in the opioid addiction epidemic.

Both NLM and the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) sent teams. When submitting their applications to compete in the Code-a-Thon, they needed to select a track on which to work: Opioid Treatment, Usage, or Misuse Prevention. Given the data available and the problems they hoped to address, both NIH teams selected Opioid Usage, but their solutions and approaches were very different.

The team from NLM—spearheaded by Dr. Huser with assistance from NLM staff members Liz Amos, Melanie Huston, Ravi Teja Bhupatiraju, and Eric Moyer—focused on developing an application to help patients (or parents of patients) make an informed decision when choosing a treatment facility or addiction rehabilitation center—a critical decision with costly implications, both personally and economically. Their idea was to leverage federal data about treatment centers from Substance Abuse and Mental Health Services Administration and extend it with ability to add comments from the patient or parent perspective.

Some members of the team worked on coding, while others worked on communicating and presenting the idea. The varying skills within the team made for dynamic discussions, new concepts for innovation, creative thinking, and ultimately, strong solutions.

The NIDDK team included Drs. Joseph McKenna (team lead), Patrick Fletcher, Jung-Min Han, Danh-Tai Hoang, and Kimberly Leblanc. Together, they used machine learning to design a predictive model of future opioid epidemic hot spots, i.e., US counties in which the number of opioid-related deaths is likely to increase in the near future. They used county-level risk factor data, such as employment rate, education level, and number of primary care physicians, and then inferred causal relationships between the risk factors. The team reasoned that county and federal policymakers should find the predictions and causal relationships helpful for deciding how to allocate resources and design interventions to address epidemic outbreaks.

While neither team took a top prize, all left the event with positive experiences and an eagerness to participate in future Code-a-Thons.

As Dr. McKenna explained, “The opioid epidemic is currently causing thousands of preventable deaths in America. My family has been directly impacted by abuse of prescription opioids. I was interested in applying my skills, data analysis, and modeling to help mediate the epidemic. I think the competitive and group aspects of the Code-a-Thon fostered creative work in a short time span.”

Participants agreed that working collaboratively in such a large and powerful way—with federal and private data—created the right climate for breakthroughs and innovation, while leveraging the tremendous knowledge and skill of the people who work in NIH’s 27 Institutes and Centers.

Liz Amos, librarian with the National Information Center on Health Services Research and Health Care Technology (NICHSR) and member of the NLM team, is already thinking about similar future events. “NLM hosts several excellent hackathons through the NCBI [National Center for Biotechnology Information] and in collaboration with universities and other institutions,” she said. “They are an incredible and valuable experience—I encourage anyone interested in data science to get involved!”

So whether you’re an NIH staff scientist, a post-doctoral fellow, a librarian, or a programmer, consider applying your background, expertise, and talents to the next Code-a-Thon or Hack-a-Thon and help create a healthier and more resilient country by solving one problem at a time.

---

About the Author:

Grace Middleton (contractor with ICF) serves as a Communication Specialist for Data Science at NIH. She oversees the development of the weekly blog, manages the Data Science social media accounts and the Data Science at NIH Updates weekly newsletter. Prior to joining NLM, Ms. Middleton was the Director of Communication for the Medical Reserve Corps Program in the Office of the Assistant Secretary of Preparedness and Response (ASPR). She developed, executed and oversaw the strategic communication plan for the program, which supported more than 900 community based units across the United States made up of over 190,000 volunteers. 

On December 6 and 7, the US Department of Health and Human Services (HHS) Office of the Chief Technology Officer (CTO) hosted an Opioid Symposium and Code-a-Thon to promote innovative ways to leverage technology and data to address the nationwide opioid epidemic.

The symposium brought together federal, state, and local stakeholders charged with addressing the opioid epidemic to share insights, promising practices, and innovative methods. It also included testimonials by a few of those personally affected by the disease, such as Bridget Farrell, whose daughter Delaney died of an opioid overdose. Her family’s personal struggle highlighted the importance of identifying and sharing treatment options at the earliest stage possible.

Opioids’ toll on individuals, families, and the nation at large is shocking. Drug overdose deaths and opioid-involved deaths continue to increase in the United States. According to the Centers for Disease Control and Prevention, 140 Americans die each day from drug overdoses, 91 specifically due to opioids. In 2015, 52,404 Americans died from opioid overdoses, and preliminary numbers indicate at least 64,000 died in 2016.

The majority of drug overdose deaths (more than six out of ten) involve a prescription or illicit opioid. Ending this epidemic is among the Department’s top priorities, and to combat this crisis HHS has implemented a five-part strategy that provides the overarching framework for its opioid-related activities.

At the symposium, US Surgeon General Jerome M. Adams, MD, MPH, whose brother has struggled with addiction, reinforced the notion that the opioid phenomenon spares no one. Consequently, Adams emphasized, it is imperative to use all available data to more effectively develop solutions and prevention strategies to fight this battle.

The two-day event showcased many technology innovators who are using artificial intelligence, predictive analytics, and data-sharing strategies to mine data sources. They shared data and technology practices with proven positive outcomes, so state and local leaders can take them back to their communities to further their reach and impact.

The second half of the opioid event, the Code-a-Thon, brought together 50 teams selected from over 300 applicants to develop innovative solutions addressing one of the following challenge tracks: Opioid Treatment, Opioid Usage, and Opioid Misuse Prevention.

Comprised of three to five members each, the teams included participants from across specialities, including computer programmers, public health advocates, and innovators. The teams worked 24 hours straight to create data-driven solutions that could have immediate and practical impact on the opioid crisis.

The teams received access to over 70 datasets related to opioids, including HHS sources (such the Medicare Part D Opioid Prescriber Summary and the CDC Opioid Prescriber Rates), data from state agencies (such as Indiana’s Deaths from Drug Poisoning dataset), and data from private sources (such as the University of Wisconsin’s County Health Rankings). Teams were also free to use their own data sources. (The CTO’s office hopes to make many of these datasets publicly available in the future).

Three finalists were selected in each track to move on to the finals. Solutions were judged on design, potential for impact, technical achievement, and innovation.

The following three teams received $10,000 each for their innovative solutions:

• The Visionist, Inc. (Columbia, Maryland)
  • Track: Prevention
  • Problem: How can you help federal, state, and local stakeholders predict and analyze the supply and movement of legal and illicit opioids?
  • Winning solution: Their program, Take Back America, assesses the unmet need in five states for takeback programs at pharmacies where unused or unneeded opioids can be returned, thereby taking a source of opioids out of circulation.
     
• Origami Innovations (Yale University)
  • Track: Treatment
  • Problem: How can you help federal, state, and local stakeholders improve access to effective treatment and recovery services?
  • Winning solution: Their model allows first responders and health authorities to track in real-time events such as the outbreak of fentanyl overdoses in a community, enabling area hospitals and local health departments to allocate resources where they are most needed.
     
• Opioid Prescriber Awareness Tool Team (OPAT)
  • Track: Usage
  • Problem: How can you help federal, state, and local stakeholders identify at-risk populations and their underlying risk characteristics of opioid misuse or abuse?
  • Winning solution: Their program provides physicians with a visual representation of their opioid-prescribing patterns compared with their peers. The tool also allows physicians to visualize the prescribing patterns of other physicians in their area to whom they might refer patients.

“HHS’ Code-a-Thon was a major step forward in the efforts to use data to address the opioid crisis,” Acting HHS Secretary Eric D. Hargan said. “The innovative ideas developed today could turn into tomorrow’s solutions, as we work to combat the scourge of opioid addiction sweeping the nation.”

US Chief Technology Officer Bruce D. Greenstein challenged participants to consider where and how their solutions could be adopted, implemented, and scaled in other settings.

The innovations developed through this Code-a-Thon are just the beginning of solutions that may be devised by NIH or its grantees to stem the opioid addiction. In fact, the NIH Office of Translational Initiatives and Program Innovations at the National Institute on Drug Abuse (NIDA) shared information on how NIH and NIDA would continue fighting the epidemic of opioid addiction by funding programs to develop innovative technologies. Among these are the $100,000 for Start a SUD Startup Challenge, the Small Business Innovative Research grant program, and the Small Business Technology Transfer grant program.

-----

Though they did not emerge victorious, two NIH teams—one from the National Library of Medicine and the other from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)—contributed their insights and expertise to the Code-a-Thon. Check back next week to read more about their solutions and their experiences participating in the Code-a-Thon.

---

About the Authors:

Elizabeth Kittrie is a Strategic Advisor for Data and Open Science at the National Library of Medicine (NLM) where she is involved in developing the NLM long-range strategic plan and leading data science and open science initiatives.  Prior to joining to the NLM, she served as a Senior Advisor to the Associate Director for Data Science at the National Institutes of Health, where she led open innovation efforts including the Open Science Prize, a collaborative partnership between the NIH, Wellcome Trust and Howard Hughes Medical Institute. Kittrie has also served as Senior Advisor to the Chief Technology Officer of the U.S. Department of Health and Human Services, where she led open government activities across HHS and coordinated the Department’s efforts to develop a common Public Access Policy.  Prior to joining HHS, Kittrie served as the first Associate Director for the Department of Biomedical Informatics at Arizona State University.

Grace Middleton is a contractor with ICF, serving as a Communication Specialist for Data Science at NIH. She oversees the development of the weekly blog, manages the Data Science social media accounts and the Data Science at NIH Updates weekly newsletter. Prior to joining NLM, Ms. Middleton was the Director of Communication for the Medical Reserve Corps Program in the Office of the Assistant Secretary of Preparedness and Response (ASPR). She developed, executed and oversaw the strategic communication plan for the program, which supported more than 900 community based units across the United States made up of over 190,000 volunteers. 

NIH’s All of Us Research Program and the National Library of Medicine (NLM) have teamed up to raise awareness about the program, a landmark effort to advance precision medicine. Through this collaboration, the National Network of Libraries of Medicine has received a $4.5 million award to support community engagement efforts by public libraries across the United States and to improve participant access.

“We want to reach participants where they are. For many people in the country, including those with limited internet access, one of those places is the local library,” said Eric Dishman, director of the All of Us Research Program. “We’re excited to work with the National Library of Medicine to make more people aware of All of Us and the opportunity to take part.”

This partnership is a three-year pilot program, with several objectives:

• To increase the capacity of public library staff to improve health literacy.
• To equip public libraries with information about the All of Us Research Program to share with their local communities.
• To assess the potential impact of libraries on participant enrollment and retention.
• To highlight public libraries as a technology resource that participants can use to engage with the program, particularly those in underserved communities affected by the digital divide.
• To establish an online platform for education and training about All of Us and precision medicine, with resources for members of the public, health professionals, librarians and researchers.
• To help identify best practices in messaging and outreach that lead to increased public interest and engagement in the program.

“Libraries serve as vital community hubs, and this collaboration presents a perfect opportunity to help the public understand how health research impacts all of us,” said Patricia Flatley Brennan, R.N., Ph.D., director of NLM. “Working with our vast network of public libraries, we hope to contribute to medical breakthroughs that may lead to more tailored disease prevention and treatment solutions for generations to come.”

The All of Us Research Program aims to build one of the largest, most diverse datasets of its kind for health research, with one million or more volunteers nationwide who will sign up to share their information over time. Researchers will be able to access participants’ de-identified information for a variety of studies to learn more about the biological, behavioral and environmental factors that influence health and disease. Their findings may lead to more individualized health care approaches in the future.

Amanda J. Wilson, head of NLM’s National Network Coordinating Office, and Dara Richardson-Heron, M.D., chief engagement officer of the All of Us Research Program, lead the new partnership.

The All of Us Research Program is currently in beta testing. To learn more and to sign up for updates, please visit www.joinallofus.org. 

About the National Library of Medicine (NLM): The world’s largest biomedical library, NLM maintains and makes available a vast print collection and produces electronic information resources on a wide range of topics that are searched billions of times each year by millions of people around the globe. It also supports and conducts research, development, and training in biomedical informatics and health information technology. Additional information is available at www.nlm.nih.gov. 

About the National Institutes of Health (NIH): NIH, the nation's medical research agency, includes 27 Institutes and Centers and is a component of the U.S. Department of Health and Human Services. NIH is the primary federal agency conducting and supporting basic, clinical, and translational medical research, and is investigating the causes, treatments, and cures for both common and rare diseases. For more information about NIH and its programs, visit www.nih.gov. 

Precision Medicine Initiative, All of Us, the All of Us logo, and “The Future of Health Begins with You” are service marks of the U.S. Department of Health and Human Services.

Data, data, everywhere, but not a bit to share…

While a “bit” of an exaggeration, this twist on a common poetic phrase reflects statements about the difficulties of sharing data at a recent scientific meeting. The desire to share the ever-expanding proliferation of research Digital Objects was one of the key drivers behind the meeting, “Managing Digital Research Objects in an Expanding Science Ecosystem,” held November 15, 2017. The National Library of Medicine (NLM) hosted the meeting at its Lister Hill Auditorium. The session was co-sponsored by Commerce, Energy, NASA, Defense Information (CENDI); The National Academies of Sciences, Engineering, and Medicine (NASEM); National Federation of Advanced Information Services (NFAIS); and Research Data Alliance (RDA). It brought together over 80 participants from academia, government agencies, non-profit organizations, and industry. (Check out the meeting agenda and conference presentations.)

So, what are digital objects? Peter Wittenburg, RDA Director Europe, Max Planck Computing and Data Facility, defined Digital Objects as “‘meaningful entities’ existing in the digital domain of bits” and “the ‘atoms’ of our digital domain, since it makes sense to associate relevant characteristics with them.” Examples include digital datasets, publications, software, metadata, collections, queries, etc.

Digital objects are everywhere. For example, streams of high-resolution data are being generated continually with digital sensors across the Internet of Things. Researchers are also generating huge volumes of primary and derived data, as well as models, and other objects that ultimately course through the ecosystem of science and scholarship. A common theme echoed throughout the meeting is that, while researchers spend 75-80% of their time managing data, sharing this digital data and other digital objects is often difficult. Various speakers suggested many reasons for this difficulty, including:

• lack of infrastructure, tools, and training necessary to deposit and curate digital objects;
• the difficulty and expense of producing and managing consistent metadata; and
• insufficient incentives for sharing data and other digital objects.

Fundamental to discussions of sharing data and other digital objects are the topics of metadata and persistent identifiers (PIDs), which received a great deal of attention at this meeting. Basically, metadata is information about the digital object; something that helps describe or locate data, and is not the object, itself. Think card catalog to books. PID is a long-lasting reference to a digital object that typically includes a unique identifier and way to locate the object over time even if the location changes. Think Social Security Number to a person.

Sharing research data is much easier with clearly established metadata and PIDs. Todd Carpenter, executive director of the National Information Standards Organization, stated, “Data is useless unless you can do something with it. That is why metadata is so important.” Patricia Cruse, executive director of DataCite, further made the analogy to our current research practices, “We are building roads from A to B, and we are starting over every time.” Thus, researchers, universities, non-profit organizations, industry, and funders need to work collectively to establish common metadata standards, PIDs, and digital infrastructures that other researchers can use in the future.

The meeting also included discussions of data markets, secure transaction of information, smart contracts, FAIR principles, semantic mapping, global digital object clouds, data repositories, data registries, data lakes, digital package object, and cyberinfrastructure (to name a few topics), each of which could be its own topic of conversation but a bit too unwieldy for this blog.

My key take-away from the meeting: Funders, non-profit organizations, professional societies, and industry are increasingly thinking about how to leverage digital objects (e.g., research data). This is the wave of the future and understanding concepts about data sharing will be important for future endeavors. Having a data-sharing expert on your research team, collaborating with others with data-sharing expertise, or hiring third-party service providers with that expertise would be advantageous as the science ecosystem moves increasingly toward preserving and sharing digital objects.

The NIH is funding an effort to develop and populate with data an NIH Data Commons. NIH has also established data-sharing policies for genomic research data and has issued its plan for increasing access to the full spectrum of biomedical research data produced with NIH funding. Other large-scale NIH initiatives, such as the All Of Us Research Program, are developing ways for patients and the general population to share digital clinical, health, and behavioral information with biomedical researchers. Furthermore, NLM/NIH has long established policies on public access to digital publications for NIH-funded research. To be sure, the creation of metadata and PIDs to facilitate the sharing of research data is becoming increasingly important for biomedical researchers to understand.

But, as many speakers noted, the digital world is a complex landscape, and we have a while to go before digital + data sharing are the default procedures for research.

Much like the desalination of salt water, it will take innovative technology, a clear value proposition/business model, and significant investment to establish and scale the sharing of biomedical research digital objects.

---

About the Author:
Dr. Audie Atienza is a contractor for the National Library of Medicine and senior fellow at ICF. Dr. Atienza previously served as a program director at the National Cancer Institute, senior advisor to the Chief Technology Officer of the Department of Health and Human Services, and senior advisor to the Associate Director for Data Science of the National Institutes of Health. He led several health technology initiatives for NIH/HHS including Apps Against Abuse developer challenge, U.S. Surgeon General’s Healthy App Challenge , and Open Science Prize. Dr. Atienza has published extensively on technology and health, real-time data capture, mobile health, and innovative research methods.