Driving Discovery Through Data

0 Crowdsourcing and Citizen Science: Investigating Data Quality and Utility
/ 07.13.17

Citizen Science is a collaborative approach to research involving the public, not just as subjects of or advisors to the research, but as direct collaborators and partners. It is similar to crowdsourcing, which is defined in two ways: 1. voluntary involvement or contributions solicited from unknown individuals (aka “the crowd”), be they experts or not; and 2. opening a line of scientific inquiry to a group of experts (typically achieved through prizes and challenges).

The NIH Citizen Science Working Group, consisting of 60 staff from across NIH, investigates and shares best practices related to citizen science and crowdsourcing, and engages with other agencies and groups promoting citizen science in other fields. The working group explores how to incorporate citizen science into biomedical research while maintaining NIH’s high level of scientific and ethical standards.

We participate in various open innovation interest groups across the federal government, such as the Federal Community of Practice on Crowdsourcing and Citizen Science (CCS), comprising almost 400 members across more than 40 agencies.  The CCS shares developments in citizen science and crowdsourcing research methodology and plans related workshops. Recently, the CCS collaborated with the Office of Science and Technology Policy to assemble a toolkit on federal citizen science and crowdsourcing and to develop a catalog of federal citizen science and crowdsourcing projects, which launched in the spring of 2016.  Many NIH Citizen Science Working Group members also belong to the newly established Citizen Science Association (CSA), which just held its 2nd biennial meeting in May.

Even the staunchest citizen science believers at CSA raise questions about data quality. We just don’t let those concerns stop us. Instead, we are working to find answers and uncover solutions.

  • How can researchers ensure data collected in a citizen science project are accurate?
  • What quality control methods should be in place?
  • How do you make sure the project is meaningful for the communities directly impacted?

Questions such as these were proffered throughout the CSA meeting.

Studies on these very issues consistently find “the crowd” is as accurate as, and sometimes more accurate than, individual experts, and yet people still have doubts. They prefer to “leave it to the experts,” believing one must have an advanced degree to truly understand something. And though it can help, formal education is not the only route to expertise.

Experience, time, effort, and immersion can all deepen knowledge. Additionally, humans are quite adept at making inferences, visual perception, and abstract thought, regardless of degree or field of occupation. Groups of people without formal training in a specific discipline can attain the same level of accuracy in image analysis or puzzle solving as one expert. This has been demonstrated over and over again. And when it comes to medicine, are we not all experts in our own health, even if we don’t know the ins and outs of every molecular process?

Perhaps the better question is not of data quality, but of data utility. How useful is the data collected? Can the data be used again in another way to help another project? How does the data benefit the community most impacted by its use? How does the data impact metadata, ethics, or logistics?

The issues surrounding citizen science-derived data (quality, control, utility, sharing, security, etc.) are just part of the open innovation approaches our working group is exploring. We’re interested in hearing from others about what they’re doing, what question or concerns they have, and what solutions have worked for them.

Our outreach efforts will begin right here at NIH.

We’re kicking things off with a (closed) symposium on July 14, 2017 . NIH employees, volunteers, and contractors will explore examples of biomedical citizen science and crowdsourcing in paired talks given by a project leader and a citizen scientist, network with one another to find and establish new collaborations, and hear from open innovation experts from across the federal government on what works (and what doesn’t), services they provide to the larger federal community, and project highlights.

We’re hoping to excite NIH researchers about using citizen science to engage with the public and get them thinking of how they can use it to accomplish their research goals and enliven their grant portfolio.

Seating is limited so register now. The event will be recorded but not broadcast live, with the recording made available via NIH VideoCast archival services. For more information on the NIH Citizen Science Working Group please email


About the Author:

Katrina I. Theisz, MS

Katrina is a Program Analyst in the Structural Biology and Molecular Applications Branch (SBMAB) of the Division of Cancer Biology (DCB), at the National Cancer Institute (NCI), part of the National Institutes of Health (NIH). She serves as the coordinator of the NIH Citizen Science Working Group, As a part of her citizen science related duties Ms. Theisz is an active member in the federal open innovation community, serving as a steering committee member of the Federal Community of Practice on Crowdsourcing and Citizen Science, and participating in the Open Data and Makers communities. She also manages the curation, development, and maintenance of, a virtual collaboration space for biomedical citizen science stakeholders. As a member of the DCB Communications and Planning group, she helps oversee the web and social media development of the division. Ms. Theisz received her Master of Science in Mental Health Counseling from Pace University, and her Bachelor of Science in Psychology from York College of Pennsylvania.


Add New Comment

Posting Calendar

November 2018

Sun Mon Tue Wed Thu Fri Sat
Back to Top