DataScience@NIH Blog

I recently joined NCI to help support strategic data sharing and informatics projects within the Center for Biomedical Informatics and Information Technology (CBIIT). Having worked on information management at another Institute for five years and the trans-NIH Big Data to Knowledge (BD2K) initiative since its inception, this is an exciting opportunity for me to continue to contribute to enhancing data science across the biomedical community.  I have seen first-hand that the ability to access, analyze, and share research data is an imperative if we are to accelerate progress in prevention and treatment of diseases like cancer. Recent announcements of the Precision Medicine Initiative (PMI) and the Beau Biden Cancer Moonshot (Moonshot) further emphasized the importance of these goals and created additional drivers towards meeting them.

The Moonshot was announced in 2016 with the objective of accelerating cancer research and breaking down impediments to progress in the development of new treatments. The initiative specifically called out the need to enhance data access and facilitate collaborations among all stakeholders, from basic researchers to clinicians to patients. A Blue Ribbon Panel (BRP), formed to make recommendations for how this ambitious goal could be achieved, recommended development of an infrastructure to support the components of a National Cancer Data Ecosystem. This knowledge network would “collect, share, and interconnect a broad array of large datasets so that researchers, clinicians, and patients will be able to both contribute and analyze data, facilitating discovery that will ultimately improve patient care and outcomes.” The Ecosystem will also support the goals of the PMI and the “All of Us” research initiative, which include responsible data sharing, access, and use, to develop individualized treatments for each patient and improve overall outcomes.  NCI will play an important role in supporting development of such an ecosystem, providing components that allow for access to and sharing of consistent and harmonized cancer research data.

To that end, the NCI has launched a series of initiatives to create an NCI Cancer Research Data Commons (NCRDC), which will become a key contribution to the ecosystem described by the BRP. The Cancer Genomics Cloud Pilots, which are transitioning into ongoing NCI Cloud Resources, enable access and analysis of large-scale cancer genomic data. The Cloud Resources have been the subject of several posts on this blog (listed below), and are one component of a NCRDC. In parallel, the Genomic Data Commons (GDC) was launched last year, providing secure access to harmonized genomic data from NCI-sponsored programs. NCI is leveraging investment in both these programs to create a Data Commons Framework, comprising the key functionality of the Cloud Resources and the secure data access of the GDC.

The Framework will provide components required to stand up and maintain a Data Commons “node” — one branch of a Commons that stores a set of related data — including:

• secure user authentication and authorization
• metadata validation tools
• an approach allowing development of consistent domain-specific data models
• an API and container environment for tools and pipelines
• access to elastic compute resources
• workspaces for storing data, tools and results and for collaboration among researchers.

Data will be mirrored in multiple commercial clouds, for redundancy, stability, and scalability. Additionally, the Cloud Resources will continue to provide environments to analyze these data using hosted or user-provided analysis tools and pipelines. The vision of the NCI Cancer Research Data Commons is one that contains multiple nodes, with researchers, tool developers, clinicians, and patients contributing and accessing tools and data.

Development of the NCI Cancer Research Data Commons presents several challenges that we’ll need to be mindful of if we are to be successful.

First, the scale of what we are trying to do is big. Cancer research is vast and complex, the variety of tools and approaches for analyzing the data are wide-ranging, and the amount of biomedical data being generated is staggering. Add to that the lack of consistency and standardization of the data, and it would be easy to bite off more than we can chew, or get lost in the details. Our approach is to focus on leveraging what we have already developed in the GDC and the Cloud Pilots, and incrementally add data, tools, and infrastructure to solidify those into a platform for moving forward.

Second, the variety of data types presents a similar issue — what kind of data should we make available first, and where should it be located? With that in mind, NCI is working on several new Commons nodes, modeled on the GDC and using the new Framework — specifically, Imaging, Proteomics, and Immuno-oncology. Each of these data types is at a different level of maturity in terms of data standards, which provides an excellent opportunity to apply the framework to new domains and test their effectiveness so we can iterate for future nodes.

All of us at NCI are cognizant of the many different activities in the cancer research community which are related to our concept of a Data Commons. The Framework represents one component of a larger data ecosystem and therefore needs to be flexible, open, and “plug and play” as much as possible, so that usability and interoperability emerge as key features of the infrastructure. NCI will be holding workshops with the biomedical community in the near future to engage in a dialog around planning for a Data Commons and to move toward collaborative solutions. If you have thoughts and opinions to share now, I welcome your feedback at the link below. One thing has become abundantly clear in the world of cancer research — the ability to share data readily across institutional and domain boundaries is an absolute necessity. I am certain that as we come together as a community, we will progress towards making that a reality. The Cancer Research Data Commons is one step in that direction.

Previous blog posts on the NCI Cloud Resources:

• Democratizing Access to CCG Data: Cancer Genomics Cloud Pilots, by Izumi Hinkson, Ph.D.
• Cancer Genomics Cloud Pilots DREAM Challenge — Leveraging the Wisdom of the Crowd, by Tony Kerlavage, Ph.D.
• Cancer Data and Computation in the Cloud: One Path to Affordable Genomics Research, by Gad Getz, Ph.D.
• Usable, Collaborative, Reproducible, and Extensible: Four Key Tenets of Cloud Computing, by Brandi Davis-Dusenberry, Ph.D. and Nick Groves-Kirkby
• Cancer Research in the Cloud: The New Normal?, by Ilya Shmulevich, Ph.D.
• FY 2017: NCI Looks to the Clouds, by Warren Kibbe, Ph.D.

---
By Allen Dearry, Ph.D., Program Director

Big data –data science – data wrangling – data munging – data commons – cloud instances – the lingo of data science is growing almost as fast as the data itself is multiplying.  Many of these terms reflect the size of the burgeoning data stores found in almost every scientific discipline, from anthropology to zoology.  This leaves the casual observer or the data novice (and even some data impresarios!) with the idea that data science is only for big data, and somehow that these two terms are synonymous. I’d like to start a dialog for conceptual clarity about data science, undergirded by a plea for methodological flexibility and philosophical plurality.

First, let’s talk about data science – it means many things to many people – in this blog in the past I have advanced the idea that, within the health care context, data science is a set of principles and practices that underlie the effective use of data relevant to biomedicine to glean insights and make new discoveries that will improve human health. While size matters, and it is often the enormity of the data sets that exceed the limits of human and computational resources, data science isn’t reserved only for large data sets. Indeed, the robustness and methodological flexibility afforded by many data science approaches may be of significant value when applied to a wide variety of data sets. To me, data science approaches step in when a data set lacks the characteristics that make it amenable to investigation using well-understood frequentists inference approaches.  

Data science stands apart from other analytical and philosophical approaches to inquiry with three main distinctions: 1) the methods are un-encumbered by the distributional assumptions; 2) analytical tools can be applied continuously, as to streams of data, or in a distributed manner, to data sets stored in different locations and 3) the volume of data often exceeds the storage capacity of computers commonly used in research. Data science reflects a promising way to make sense out of large data sets, to add interpretability to high volume, high velocity data.

Data science is more that large data files and a range of analytic and visualization tools. It’s a way of thinking about discovery and knowledge building that complements traditional empirical and experimental approaches with a new set of methods less tied to the semantics of the phenomena and more tied to the characteristics of the data itself. In data science approaches, scholarly rigor emerges in a different manner than the more familiar design-driven approaches to rigor.

Now let’s unpack that one – “less tied to the semantics of the phenomena and more tied to the characteristics of the data itself” – semantic based research methods use measurement approaches and analytical strategies governed by the nature of the phenomena of interest – so, for example, variable definitions tie the actual data generated in the measurement to some underlying physical or anatomical or psychological phenomena. Counts of heart rate within a certain range indicate normal function of the heart. The data (counts) are a measure (heart rate) that is a representation of some phenomena (cardiac function).  Data science investigations can but are not restricted to a semantic alignment between data and phenomena. That is, the representation constraints are relaxed, and theory becomes useful as an interpretation tool at the end of an investigation, not at the beginning.  This becomes exceedingly useful when one is exploring data captured during normal processes, such as traffic flow or sound waves. The linking of the data to the phenomena comes late in the process, and is not asserted early. 

Now for the second assertion: “scholarly rigor emerges in a different manner than the more familiar design-driven approaches to rigor.” In empirical, experimental design approaches to research, the rigor is built in up front – Carefully explicated theoretical frameworks guide the initial hypothesis, measurement arises from characteristics of the phenomena under study, randomization, sample size, sampling plan,  procedures are well though-out and applied in a principled manner, only those analytical strategies supported by the data and aligned with the design are employed, and interpretation returns to the original theoretical premise.  A priori planning and careful execution lead to rigor, trustability and interpretability.

Data science methods allow investigation of a data set long after the data have been generated or collected. The curation of the data at the point of use, still executed in a principled manner, affords a level of tractability that enhances trust and interpretation. The data may have been collected for one purpose, and then used for a very different purpose.  For example, weather observations collected for the purposes of predicting storms in turn become input into predicting satellite performance or degradation.  Traffic patterns that guide the sequences of control for stoplights become explanatory variables in geographically-based investigations of sudden cardiac death. In data science investigations, the rigor emerges from the principled manner of selecting, curating, and investigating data, not from the a priori design and conduct.

Indeed, data science investigations yield different insights than do experiments – both are needed for discovery! It’s wasteful to think that data science approaches should only be used when one faces the challenges of large data sets – these emerging approaches hold great promise for learning from many situations. 

Bad research methods are a problem. They lead to inaccurate findings and the spread of misinformation.

In my field—systematic reviews and meta-analyses—an increasing number of meta-research studies are finding that many, even most, studies have deep methodological flaws.

Nevertheless, bad or superseded practices flourish. People emulate methods they saw in other studies. It’s the easy way to go, but it’s not reliable.

That’s why methods research or meta-research—research on research—is critical to improving the quality of any science, including data science.

In data science, like other areas of science, methods evolve—often quickly. Systematic reviews help make sense of the often-conflicting studies and the heated methodological controversies that result. Some fields generate hundreds, even thousands, of research articles every year, but it’s not easy to find those studies, complicating a systematic review as well. Search for information about a specific data science method, and all the studies that use the method drown out the ones that studied it.  

At PubMed Health, we’re trying to help overcome that. We are developing resources to help researchers with the science of meta-analysis and other aspects of systematic reviews in health.

We start by targeting important grey literature to identify empirical studies, systematic reviews, and agencies’ best practices. They feed into a growing collection at PubMed Health, which in turn feeds into PubMed.

Next we collaborate with researchers at AHRQ’s Effective Health Care Scientific Resource Center (SRC) to regularly supply new content for the methodology research filter in PubMed. The resulting collection of meta-research and methods guidance are then searchable in PubMed using the filter sysrev_methods[sb].

Enter that phrase into PubMed’s search box like any other search term to limit the results to those related to systematic reviews. For example, click the following link to see the results of this search:

sysrev_methods[sb] AND “network meta-analysis”

As data science develops, the science of data science is going to accelerate. We’re going to need to work together to find effective ways to help data scientists keep up. Systematically collecting data meta-science is one way to start.

Much of the data people produce is text, and NIH is no exception. Both extramural and intramural scientists use text to describe their research goals and results. Besides proposals, technical reports, and scientific publications, the raw data collected in NIH-supported studies also contain fair amounts of text: notes, including clinical notes, metadata describing collected data, semi-structured observations, and the results of qualitative studies. In addition, patients produce a wealth of information about health issues as they discuss their reactions to treatments, experiences with clinicians, and other health-related matters on websites, in social media, through their interactions with NIH customer services, and other venues. The list of text-based documents related to health does not stop there—think patient information, drug labels, and practice guidelines, to name a few.

In the context of research, these documents—along with the rest of big data—share one significant problem: volume. There are too many of them. Too many documents are produced for one individual to find all those relevant to a particular clinical or research question. (PubMed alone contains more than 27 million citations and is growing daily.) Even if all relevant documents are found, one cannot read and analyze them all. (Alper, et al, determined one would need an estimated 627.5 hours per month to evaluate articles published in 341 epidemiology-related journals.) And finally, no one individual can acquire and maintain the knowledge needed to comprehend the entirety of the data.

How can natural language processing (NLP) help this overwhelming situation?

To understand that, we need to set aside the complexity of language understanding and focus on NLP’s data science aspects.

NLP can find relevant documents by applying to information retrieval the more traditional NLP task of answering questions.

NLP can summarize content, turning large numbers of documents into digestible chunks. For example, NLP can automate the discovery of research for inclusion in systematic reviews, replacing the laborious process of researchers screening thousands of citations to determine their relevance. NLP can also keep systematic reviews and health care guidelines up to date.

In addition, NLP can support clinical decision-making by integrating and synthesizing symptoms, physical findings, and both positive and negative elements within a patient’s history. 

Together these functions can yield a concise representation of the textual dataset under analysis. From a precise pool of relevant documents to meaningful, actionable summaries to a dashboard of germane findings, NLP has the capacity to speed discovery and support decisions.

In the past, many of the above tasks were accomplished using knowledge-based methods, which resulted in biomedical and clinical NLP being one of the most resource-rich NLP areas. Today, most of the solutions are statistical and machine-learning methods, with the increasing use of deep learning that both needs and helps with an abundance of textual data.

This development parallels the progression of biomedical NLP from art, in which carefully hand-crafted systems supported few, into the realm of data science, where machines, with humans occasionally in the loop, will solve real-world problems using datasets too large to be processed by any one individual.

---

About the Author:

Biomedical research is increasingly driven by the acquisition and analysis of large digital datasets. Sharing of such datasets has the potential to accelerate research, since in many cases it is far more expensive to collect than to analyze data. As the number, size, and public availability of biomedical datasets grow, so do the opportunities to advance biomedical knowledge. However, this also presents new challenges to the biomedical researcher, as data sets relevant to a problem of interest may be scattered across multiple repositories and be difficult to find. The heterogeneous nature of biomedical data, the lack of data discovery infrastructure, and fragmented data environments, data standards, and documentation present a barrier to data sharing. Tools to enable researchers to discover and re-use data to facilitate biomedical science are needed.

The biomedical and healthCAre Data Discovery Index Ecosystem (bioCADDIE) project, a part of the NIH Big Data to Knowledge (BD2K) initiative, seeks to provide a prototype platform for researchers to find biomedical data to enable reanalysis and the creation of derived products through data integration. One of the outcomes of bioCADDIE is the aggregation of metadata across a variety of biomedical data repositories into a prototype discovery system named DataMed , which provides researchers with a PubMed-like search engine. DataMed supports the findability and accessibility of data sets, characteristics - along with interoperability and reusability - of the four FAIR principles to facilitate knowledge discovery in today's big data-intensive science landscape. A platform-independent common model (DATS) describes the metadata elements and the structure for datasets, and powers DataMed’s ingestion and indexing pipeline, as well as its search functionality. Publicly launched in 2016 and published in Nature Genetics, DataMed’s latest release (v3.0) contains new features, improvements based on user feedback, and indexing of 74 major repositories hosting over 2.3 million datasets. Users don’t need to know in advance which repositories may have data of interest nor use a different query strategy for each repository portal. Once they find datasets of interest in DataMed, they can follow links to retrieve them from the various repositories.

DataMed, a completely open-source prototype search system, uses a modular architecture to support use cases from the general, biological and translational communities with a number of common needs for metadata searches. This makes DataMed broad in its search capabilities, allowing it to easily span a range of diverse domains and types of data. Thus, DataMed is designed to be a common data index infrastructure, connecting with existing biomedical data repositories (e.g. dbGaP, PDB), aggregators (e.g., bioProject, OmicsDI, dkNET), and other data sources. Both DataMed and the underlying DATS metadata model use schema.org (http://schema.org) annotation to expose the harvested metadata to general search engines such as Google, Microsoft, Yahoo and Yandex. Additionally, the latest version of DataMed incorporates a RESTful API to provide programmatic access to all of DataMed’s harvested metadata and additional user-interface features. Such sharing capabilities make DataMed easily accessible not only to the biomedical community but also to those outside of the biomedical sphere.

---

About the Authors:

Executive Committee members of bioCADDIE:

• Lucila Ohno-Machado, MD, MBA, PhD: Dr. Ohno-Machado is the Principal Investigator for bioCADDIE and Professor of Medicine and founding chief of the Division of Biomedical Informatics at UCSD. She is associate dean for informatics and technology and has experience leading multidisciplinary projects at the intersections of biomedicine and quantitative sciences. Dr. Ohno-Machado is also director of the Biomedical Research Informatics for Global Health training program.
• George Alter, PhD: Dr. Alter is Director of the Inter-University Consortium for Political and Social Research (ICPSR), Research Professor at the Population Studies Center, and Professor of History at the University of Michigan. His research grows out of interests in the history of the family, demography, and economic history, and recent projects have examined the effects of early life conditions on health in old age and new ways of describing fertility transitions. 
• Susanna-Assunta Sansone, PhD: Susanna-Assunta Sansone is an Associate Director and Principal Investigator at the Oxford e-Research Centre, part of the Engineering Science at the University of Oxford; and Consultant for Springer Nature Scientific Data. Her activities are around and in support of data curation, management and publication and their pivotal roles in enabling reproducible research and knowledge discovery.
• Hua Xu, PhD: Dr. Xu is the Robert H. Graham Professor at the School of Biomedical Informatics and Director of the Center for Computational Biomedicine in The University of Texas Health Science Center at Houston (UTHealth). Dr. Xu is an expert in biomedical text processing and data mining.
• Jeffrey Grethe, PhD: Dr. Grethe is currently a co-investigator for the Neuroscience Information Framework (NIF) and Principal Investigator for the NIDDK Information Network (dkNET) in the Center for Research in Biological Systems (CRBS) at the University of California, San Diego. Throughout his career, he has been involved in enabling collaborative research, data sharing and discovery through the application of advanced informatics approaches.
• Ian Fore, D.Phil: Dr. Fore is the NIH Scientific Officer on bioCADDIE. He serves as the Senior Biomedical Informatics Program Manager at the National Cancer Institute (NCI).

Management team of bioCADDIE:

• Elizabeth Bell, MPH: Ms. Bell is the general bioCADDIE project manager. She has experience with patient-centered electronic consent studies. Her background is in environmental health.
• Anupama E Gururaj, PhD: Dr. Gururaj serves as the Project Manager for the Core Technology Development Group of BioCADDIE as well as all BioCADDIE-related activities at The University of Texas Health Science Center at Houston (UTHealth). She has research experience in cancer research focusing on cancer signaling pathways as well as biomedical informatics, specifically in the area of data mining and natural language processing.