Houston expert: Navigating dark data within research and innovation

houston voices

Let's talk about dark data — what it means and how to navigate it. Graphic by Miguel Tovar/University of Houston

Is it necessary to share ALL your data? Is transparency a good thing or does it make researchers “vulnerable,” as author Nathan Schneider suggests in the Chronicle of Higher Education article, “Why Researchers Shouldn’t Share All Their Data.”

Dark Data Defined

Dark data is defined as the universe of information an organization collects, processes and stores – oftentimes for compliance reasons. Dark data never makes it to the official publication part of the project. According to the Gartner Glossary, “storing and securing data typically incurs more expense (and sometimes greater risk) than value.”

This topic is reminiscent of the file drawer effect, a phenomenon which reflects the influence of the results of a study on whether or not the study is published. Negative results can be just as important as hypotheses that are proven.

Publication bias and the need to only publish positive research that supports the PI’s hypothesis, it can be argued, is not good science. According to an article in the Indian Journal of Anaesthesia, authors Priscilla Joys Nagarajan, et al., wrote: “It is speculated that every significant result in the published world has 19 non-significant counterparts in file drawers.” That’s one definition of dark data.

Total Transparency

But what to do with all your excess information that did not make it to publication, most likely because of various constraints? Should everything, meaning every little tidbit, be readily available to the research community?

Schneider doesn’t think it should be. In his article, he writes that he hides some findings in a paper notebook or behind a password, and he keeps interviews and transcripts offline altogether to protect his sources.

Open-source

Open-source software communities tend to regard total transparency as inherently good. What are the advantages of total transparency? You may make connections between projects that you wouldn’t have otherwise. You can easily reproduce a peer’s experiment. You can even become more meticulous in your note-taking and experimental methods since you know it’s not private information. Similarly, journalists will recognize this thought pattern as the recent, popular call to engage in “open journalism.” Essentially, an author’s entire writing and editing process can be recorded, step by step.

TMI

This trend has led researchers to open-source programs like Jupyter and GitHub. Open-source programs detail every change that occurs along a project’s timeline. Is unorganized, excessive amounts of unpublishable data really what transparency means? Or does it confuse those looking for meaningful research that is meticulously curated?

The Big Idea

And what about the “vulnerability” claim? Sharing every edit and every new direction taken opens a scientist up to scoffers and harassment, even. Dark data in industry even involves publishing salaries, which can feel unfair to underrepresented, marginalized populations.

In Model View Culture, Ellen Marie Dash wrote: “Let’s give safety and consent the absolute highest priority, with openness and transparency prioritized explicitly below those. This means digging deep, properly articulating in detail what problems you are trying to solve with openness and transparency, and handling them individually or in smaller groups.”

------

This article originally appeared on the University of Houston's The Big Idea. Sarah Hill, the author of this piece, is the communications manager for the UH Division of Research.

Ad Placement 300x100
Ad Placement 300x600

CultureMap Emails are Awesome

Property Showcase

UH lands $4M NIH grant to study early signs of autoimmune disease

NIH funding

The University of Houston recently received a $4 million National Institutes of Health grant to support a 10-year longitudinal study to identify the earliest biological markers of autoimmune disease.

Led by Chandra Mohan, the Hugh Roy and Lillie Cranz Cullen Endowed Professor of Biomedical Engineering, the study aims to examine what causes Systemic Autoimmune Rheumatic Diseases (SARDs) and to identify targets for future treatments. The study will be carried out in collaboration with Dr. Karen Costenbader at Harvard Medical School, Boston.

SARDs include conditions like rheumatoid arthritis, systemic lupus erythematosus, Sjögren’s syndrome and systemic sclerosis—all are considered chronic diseases currently without a cure. Autoimmune diseases affect over 30 million people globally, according to UH.

SARDs occur when the body’s immune system attacks healthy, non-threatening tissues and organs. According to UH, in these diseases, the body often attacks nuclear antigens, creating anti-nuclear autoantibodies, which can be early detection signs for SARDs in more than 50 percent of patients, Mohan says.

Researchers will study blood samples and environmental exposure over the 10 years to better understand anti-nuclear autoantibodies.

“Collectively, these studies will help identify the genetic, environmental and cellular factors that are operative at the two steps of SARD development, namely the emergence of anti-nuclear autoantibodies and disease onset,” Mohan said in a news release. “ More importantly, these studies will highlight functional molecular pathways and mechanisms that may be operative at each step."

Mohan predicts that looking at SARDs’ shared characteristics, rather than each disease individually, could help identify more treatment methods.

“Individual SARDs have been examined in silos without an attempt to discern shared underlying features at the molecular level,” he added in the release. “Current understanding of the initial (and likely shared) origins of SARDs is only rudimentary but urgently needed to develop means for prevention and treatment.”

Earlier this year, UH also received an $11 million NIH grant to conduct a first-of-its-kind study of early language development in children ages 18 to 24 months. Read more here.

New Texas Stock Exchange officially begins trading in Dallas

Welcome to Y'all Street

Two-step aside, New York Stock Exchange and Nasdaq. The Dallas-based Texas Stock Exchange, nicknamed Y’all Street, just kicked off live trading with five stocks — and lots of Lone Star ambition.

“The Texas Stock Exchange aims to revitalize competition for [stock] issuers, establish the premier venue for listings, and create a world-class trading platform for all market participants,” the exchange says in a fact sheet.

The exchange — whose Texas-influenced nickname is a nod to New York City’s Wall Street — has collected at least $275 million in investments. The roughly 90 financial backers of TXSE include Bank of America, BlackRock, Charles Schwab, Citadel Securities, Dell Family Office, Fortress, Goldman Sachs, and JPMorgan Chase.

Representatives of TXSE couldn’t be reached for comment. On its website, the exchange calls itself “the most well-capitalized equities exchange to ever be approved” by the U.S. Securities and Exchange Commission (SEC).

Not to be outdone, NYSE has launched Dallas-based NYSE Texas and Nasdaq has expanded its presence in Dallas.

Y’all Street adds to Dallas-Fort Worth’s rising status as a major hub for financial services, with The Wall Street Journal naming North Texas the country’s second biggest financial hub after New York City.

“A homegrown national exchange means more jobs, more investment, and more growth opportunities for businesses and communities across the Lone Star State,” Gabriela von zur Muehlen, senior vice president and chief policy officer at the Texas Association of Business, told The Texas Tribune.

Bulent Temel, an associate professor of practice in economics at the University of Texas at San Antonio, told Texas Standard that TXSE “is going to boost the credibility of the Texas economy.”

Texas’ estimated gross domestic product (GDP), a yardstick for the size of an economy, climbed to a record-setting $2.9 trillion in 2025, making it the state with the second highest GDP after California. DFW’s estimated GDP in 2023 stood at $744.6 billion, eclipsing the GDP of many countries.

“The center of gravity for American capitalism is now headquartered in the Boom Belt,” Abbott proclaimed in April, referring to an 11-state region (including Texas) in the South and Southeast that’s seeing tremendous economic and population growth. “The Texas Stock Exchange is the natural extension of that capitalism. It ensures that capital markets will reflect the quadrant that is driving American growth.”

---

This article originally appeared on CultureMap.com.

Orion vehicle manager reflects on Artemis II, looks to 2028 moon mission

Q&A

Humanity is finally headed back to the moon after more than half a century. This year's launch of the Artemis II mission in the Orion spacecraft put four crew members in lunar orbit and tested the new ship developed by Lockheed Martin.

Everything went smoothly, safely returning astronauts home, but there is always room to improve. InnovationMap chatted via email with Orion vehicle manager Branelle Rodriguez, shortly after a talk at The Ion, for insight on how Orion might perform in the future as the next lunar landing approaches in early 2028.

InnovationMap: How satisfied are you with the way Orion operated on this past mission?

Branelle Rodriguez: Orion performed exceptionally well during Artemis II, successfully demonstrating critical spacecraft capabilities, including life support systems, displays and controls, and executing manual piloting operations. Artemis II brought humans back to the moon, achieving key exploration and scientific imagery, while validating systems essential for future Artemis missions.

IM: What is the most important thing you learned about improving Orion for the next mission?

BR: The Artemis II mission provided invaluable insights into crew operations and spacecraft performance in a deep-space environment. With every mission, NASA applies lessons learned to continuously improve Orion’s operations, validate design and ensure mission readiness. Artemis II offered our first opportunity to evaluate several new systems and gain a deeper understanding of what it is like for astronauts to live and work inside the spacecraft. The operational, technical and human factors data collected are being integrated across the program to refine future missions, reduce risk and enhance overall mission success.

IM: How has Orion helped the mission to explore space?

BR: Orion is one of NASA’s foundational elements for human deep space exploration—not only supporting the mission but serving as a core component of it. It is currently the only spacecraft capable of carrying crew on deep space missions and returning them safely to Earth from the high speeds required from the vicinity of the moon. No other spacecraft has the technology to endure the extremes that come with human deep-space travel, such as advanced environmental and life support, navigation, communications, radiation shielding, and the world’s largest ablative heat shield to protect the astronauts during reentry into Earth’s atmosphere. Orion has already taken astronauts to explore space farther than ever before—252,756 miles from Earth— and will carry crews to the moon on future missions to explore the lunar South Pole region. The astronauts’ observations, samples, and data collected on these future missions will expand our understanding of our solar system and home planet.

---

This conversation has been edited for brevity and clarity.

View All Listings