Let's talk about dark data — what it means and how to navigate it. Graphic by Miguel Tovar/University of Houston

Is it necessary to share ALL your data? Is transparency a good thing or does it make researchers “vulnerable,” as author Nathan Schneider suggests in the Chronicle of Higher Education article, “Why Researchers Shouldn’t Share All Their Data.”

Dark Data Defined

Dark data is defined as the universe of information an organization collects, processes and stores – oftentimes for compliance reasons. Dark data never makes it to the official publication part of the project. According to the Gartner Glossary, “storing and securing data typically incurs more expense (and sometimes greater risk) than value.”

This topic is reminiscent of the file drawer effect, a phenomenon which reflects the influence of the results of a study on whether or not the study is published. Negative results can be just as important as hypotheses that are proven.

Publication bias and the need to only publish positive research that supports the PI’s hypothesis, it can be argued, is not good science. According to an article in the Indian Journal of Anaesthesia, authors Priscilla Joys Nagarajan, et al., wrote: “It is speculated that every significant result in the published world has 19 non-significant counterparts in file drawers.” That’s one definition of dark data.

Total Transparency

But what to do with all your excess information that did not make it to publication, most likely because of various constraints? Should everything, meaning every little tidbit, be readily available to the research community?

Schneider doesn’t think it should be. In his article, he writes that he hides some findings in a paper notebook or behind a password, and he keeps interviews and transcripts offline altogether to protect his sources.

Open-source

Open-source software communities tend to regard total transparency as inherently good. What are the advantages of total transparency? You may make connections between projects that you wouldn’t have otherwise. You can easily reproduce a peer’s experiment. You can even become more meticulous in your note-taking and experimental methods since you know it’s not private information. Similarly, journalists will recognize this thought pattern as the recent, popular call to engage in “open journalism.” Essentially, an author’s entire writing and editing process can be recorded, step by step.

TMI

This trend has led researchers to open-source programs like Jupyter and GitHub. Open-source programs detail every change that occurs along a project’s timeline. Is unorganized, excessive amounts of unpublishable data really what transparency means? Or does it confuse those looking for meaningful research that is meticulously curated?

The Big Idea

And what about the “vulnerability” claim? Sharing every edit and every new direction taken opens a scientist up to scoffers and harassment, even. Dark data in industry even involves publishing salaries, which can feel unfair to underrepresented, marginalized populations.

In Model View Culture, Ellen Marie Dash wrote: “Let’s give safety and consent the absolute highest priority, with openness and transparency prioritized explicitly below those. This means digging deep, properly articulating in detail what problems you are trying to solve with openness and transparency, and handling them individually or in smaller groups.”

------

This article originally appeared on the University of Houston's The Big Idea. Sarah Hill, the author of this piece, is the communications manager for the UH Division of Research.

Ad Placement 300x100
Ad Placement 300x600

CultureMap Emails are Awesome

Houston hypersonic engine company lands investment from deep-tech firm

super speed

Hypersonic engine technology company Venus Aerospace announced an investment for an undisclosed amount from deep-tech investment firm America's Frontier Fund (AFF).

Venus plans to use the funds to advance its VDR2 engine has the potential to support a commercial flight from Los Angeles to Tokyo in under two hours, according to a statement.

"AFF's investment in Venus Aerospace underscores our focus on supporting American companies that are revolutionizing industries of the future," Jordan Blashek, Co-Founder and Managing Partner of AFF, said in a news release. "With recent hypersonic advancements from China and Russia, safeguarding American innovation and securing our industrial base has never been more urgent. Venus Aerospace is poised to redefine hypersonic flight and ensure America's continued leadership in aerospace innovation."

Venus plans to integrate components of its RDRE engine into the advanced VDR2 engine, increasing efficiency with air-augmentation technology capable of 2,000 pounds of thrust. This will allow the VDR2 engine to operate at wider ranges of speeds and altitudes. The company plans to ground-test the VDR2 in 2025.

Venus also plans to make history this year by conducting the first atmospheric flight of its RDRE engine, which has applications in the space and defense industries.

"AFF's commitment to pioneering technological advancements aligns seamlessly with our mission to enhance American competitiveness in hypersonics,” Sassie Duggleby, CEO and co-Founder of Venus Aerospace, added in the news release. “Their deep-tech expertise and proven track record in scaling innovative companies will be invaluable as we work to achieve our goals."

Venus Aerospace believes these innovations can “strengthen U.S. leadership in aerospace, defense, and energy innovation."

Venus also aims to use its engine technology to power the Stargazer M4, the world's first Mach 4 reusable commercial aircraft and a prime example of advanced propulsion systems.

The company launched in 2020 and has gained $78.3 million in total investments, according to Pitchbook data.

Houston manufacturer names location of its $193.7 million facility

coming soon

Houston-based manufacturer of high-temperature superconducting wires MetOx International Inc. will build a major production facility in Chatham County, North Carolina, which is expected to create 333 jobs, and invest $193.7 million in the state.

MetOx is a leader in High Temperature Superconducting technology (HTS), which is an advanced power delivery technology that is capable of transmitting extremely high power at low voltage with zero heat generation or energy loss. The technology is assisting in the energy sectors like power transmission, distribution, and grid expansion.

“Establishing our new large-scale manufacturing facility in Chatham County is a pivotal step toward securing a reliable, domestic supply of HTS wire for the development of critical infrastructure in the United States,” Bud Vos, CEO of MetOx, says in a news release. “This facility will not only deliver transformative energy technologies that strengthen our grid and reduce carbon emissions but also create high-paying manufacturing jobs in a community eager to lead in innovation. We are proud to partner with North Carolina to drive forward a resilient energy future built on cutting-edge science and strong local collaboration.”

The new facility is funded in part by an $80 million investment from the United States Department of Energy, which the company announced in October. In September, the company closed $25 million in a series B extension round.

In late 2024, MetOx also announced that it received an undisclosed investment from Hawaii-based Elemental Impact, which is a leading climate-focused investment platform. As a national implementation partner for the EPA's $27 billion Greenhouse Gas Reduction Fund, Elemental Impact has received $100 million to deploy later-stage commercialized technologies according to the company.

The funding is expected to advance the expansion of MetOx’s Houston production line and the deployment of its HTS wire, which can make transmission cables up to ten times more efficient than traditional copper cables and will be used at the North Carolina facility.

“Building domestic manufacturing capacity for critical grid technologies is essential for America’s energy future," Danya Hakeem, vice president of Portfolio at Elemental Impact, says in a news release. “MetOx’s expansion in Houston demonstrates how we can simultaneously advance grid modernization and create quality manufacturing jobs. Their technology represents exactly the kind of innovation needed to unlock the next wave of clean energy deployment.”

The project in North Carolina will be facilitated with a Job Development Investment Grant formally awarded to a new company being created by MetOx. In the 12-year term of the grant, economists in the Department of Commerce estimated the project will grow North Carolina’s economy by $987.8 million.

------

This article originally was published on our sister site, EnergyCapital.

Houston Nobel Prize nominee earns latest award for public health research

Prized Research

Houston vaccine scientist Dr. Peter Hotez can add one more prize to his shelf.

Hotez — dean of the National School of Tropical Medicine and professor of Pediatrics and Molecular Virology & Microbiology at Baylor College of Medicine, co-director of the Texas Children’s Center for Vaccine Development (CVD) and Texas Children’s Hospital Endowed Chair of Tropical Pediatrics — is no stranger to impressive laurels. In 2022, he was even nominated for a Nobel Peace Prize for his low-cost COVID vaccine.

His first big win of 2025 is this year’s Hill Prize, awarded by the Texas Academy of Medicine, Engineering, Science and Technology (TAMEST).

Hotez and his team were selected to receive $500,000 from Lyda Hill Philanthropies to help fund The Texas Virosphere Project. The endeavor was born to help create a predictive disease atlas relating to climate disasters. Because the climate crisis has ushered in changes to the distribution of diseases, including dengue, chikungunya, Zika, Chagas disease, typhus and tick-borne relapsing fever, it’s important to predict outbreaks before they become a menace.

Rice University researchers are collaborating with Hotez and his team on a project that combines climate science and metagenomics to access 3,000 insect genomes. The goal is to aid health departments in controlling disease and informing policy.

The Hill Prize, which is being awarded to six innovators for the first time, thanks to a $10 million commitment from the philanthropic organization, is intended to back ideas that are high-risk and high-reward. Each of the projects was chosen for its potential real-life impact on some of Texas's — and the world’s — most challenging situations. Hotez’s prize is the first Hill Prize to be given in the realm of public health. The additional winners are:

  • Hill Prize in Medicine: Kenneth M. Hargreaves, D.D.S., Ph.D., The University of Texas Health Science Center at San Antonio
  • Hill Prize in Engineering: Joan Frances Brennecke, Ph.D. (NAE), The University of Texas at Austin
  • Hill Prize in Biological Sciences: David J. Mangelsdorf, Ph.D. (NAM, NAS), UT Southwestern Medical Center
  • Hill Prize in Physical Sciences: James Chelikowsky, Ph.D., The University of Texas at Austin
  • Hill Prize in Technology: Robert De Lorenzo, M.D., EmergenceMed, LLC
Read about other Houston-area researchers recognized by TAMEST here.