Houston expert: Navigating dark data within research and innovation

houston voices

Let's talk about dark data — what it means and how to navigate it. Graphic by Miguel Tovar/University of Houston

Is it necessary to share ALL your data? Is transparency a good thing or does it make researchers “vulnerable,” as author Nathan Schneider suggests in the Chronicle of Higher Education article, “Why Researchers Shouldn’t Share All Their Data.”

Dark Data Defined

Dark data is defined as the universe of information an organization collects, processes and stores – oftentimes for compliance reasons. Dark data never makes it to the official publication part of the project. According to the Gartner Glossary, “storing and securing data typically incurs more expense (and sometimes greater risk) than value.”

This topic is reminiscent of the file drawer effect, a phenomenon which reflects the influence of the results of a study on whether or not the study is published. Negative results can be just as important as hypotheses that are proven.

Publication bias and the need to only publish positive research that supports the PI’s hypothesis, it can be argued, is not good science. According to an article in the Indian Journal of Anaesthesia, authors Priscilla Joys Nagarajan, et al., wrote: “It is speculated that every significant result in the published world has 19 non-significant counterparts in file drawers.” That’s one definition of dark data.

Total Transparency

But what to do with all your excess information that did not make it to publication, most likely because of various constraints? Should everything, meaning every little tidbit, be readily available to the research community?

Schneider doesn’t think it should be. In his article, he writes that he hides some findings in a paper notebook or behind a password, and he keeps interviews and transcripts offline altogether to protect his sources.

Open-source

Open-source software communities tend to regard total transparency as inherently good. What are the advantages of total transparency? You may make connections between projects that you wouldn’t have otherwise. You can easily reproduce a peer’s experiment. You can even become more meticulous in your note-taking and experimental methods since you know it’s not private information. Similarly, journalists will recognize this thought pattern as the recent, popular call to engage in “open journalism.” Essentially, an author’s entire writing and editing process can be recorded, step by step.

TMI

This trend has led researchers to open-source programs like Jupyter and GitHub. Open-source programs detail every change that occurs along a project’s timeline. Is unorganized, excessive amounts of unpublishable data really what transparency means? Or does it confuse those looking for meaningful research that is meticulously curated?

The Big Idea

And what about the “vulnerability” claim? Sharing every edit and every new direction taken opens a scientist up to scoffers and harassment, even. Dark data in industry even involves publishing salaries, which can feel unfair to underrepresented, marginalized populations.

In Model View Culture, Ellen Marie Dash wrote: “Let’s give safety and consent the absolute highest priority, with openness and transparency prioritized explicitly below those. This means digging deep, properly articulating in detail what problems you are trying to solve with openness and transparency, and handling them individually or in smaller groups.”

------

This article originally appeared on the University of Houston's The Big Idea. Sarah Hill, the author of this piece, is the communications manager for the UH Division of Research.

Ad Placement 300x100
Ad Placement 300x600

CultureMap Emails are Awesome

Property Showcase

Houston investor on why 2025 will be the year of exits

houston innovators podcast episode 270

Samantha Lewis will be the first to admit that the past few years have been tough on startups and venture capital investors alike. However, as she explains on the Houston Innovators Podcast, the new year is expected to look very different.

"We're super excited going into 2025," says Lewis, who is a partner at Houston-based VC firm Mercury. "For us, 2024 was a year of laying a lot of groundwork for what we believe is going to be a massive year of startup exits and liquidity for the venture ecosystem. We've been hard at work making sure our companies are prepared for that."

Mercury, in fact, has already gotten a taste, with three of its portfolio companies celebrating exits — all with Houston roots. Fintech platform Brassica was acquired by BitGo in February, and Apparatus, founded as Topl in Houston, was acquired early last year. The third deal has yet to be announced publicly.

And it's just getting started, Lewis says. She explains that all of the companies in Mercury's portfolio that are promising — albeit not break-out, to-be-billion-dollar companies — are going to have opportunities to sell in 2025 and 2026.

"What we've started to do — and I encourage everyone to do this if you're working on a startup — is just start to just engage with strategic buyers, investment bankers, and people you think might be a great fit to buy your company," Lewis says, "because we really think that the next few years will be the best liquidity years we've seen in a really long time. And if you're not ready for it, you're going to miss the boat."

In addition to sharing her advice to get "exit preparedness," Lewis explains some specific tech trends she's keeping an eye on in Mercury's "power theme," which she works on directly. This encompasses fintech, blockchain, web3 and more.

SpaceX loses mega rocket in latest thrilling Starship test flight

Testing

SpaceX launched its Starship rocket on its latest test flight Thursday, but the spacecraft was destroyed following a thrilling booster catch back at the pad.

Elon Musk’s company said Starship broke apart — what it called a “rapid unscheduled disassembly." The spacecraft's six engines appeared to shut down one by one during ascent, with contact lost just 8 1/2 minutes into the flight.

The spacecraft — a new and upgraded model making its debut — was supposed to soar across the Gulf of Mexico from Texas on a near loop around the world similar to previous test flights. SpaceX had packed it with 10 dummy satellites for practice at releasing them.

A minute before the loss, SpaceX used the launch tower's giant mechanical arms to catch the returning booster, a feat achieved only once before. The descending booster hovered over the launch pad before being gripped by the pair of arms dubbed chopsticks.

The thrill of the catch quickly turned into disappointment for not only the company, but the crowds gathered along the southern tip of Texas.

“It was great to see a booster come down, but we are obviously bummed out about [the] ship,” said SpaceX spokesman Dan Huot. “It’s a flight test. It’s an experimental vehicle," he stressed.

The last data received from the spacecraft indicated an altitude of 90 miles and a velocity of 13,245 mph.

Musk said a preliminary analysis suggests leaking fuel may have built up pressure in a cavity above the engine firewall. Fire suppression will be added to the area, with increased venting and double-checking for leaks, he said via X.

The 400-foot rocket had thundered away in late afternoon from Boca Chica Beach near the Mexican border. The late hour ensured a daylight entry halfway around the world in the Indian Ocean. But the shiny retro-looking spacecraft never got nearly that far.

SpaceX had made improvements to the spacecraft for the latest demo and added a fleet of satellite mockups. The test satellites were the same size as SpaceX’s Starlink internet satellites and, like the spacecraft, were meant to be destroyed upon entry.

Musk plans to launch actual Starlinks on Starships before moving on to other satellites and, eventually, crews.

It was the seventh test flight for the world’s biggest and most powerful rocket. NASA has reserved a pair of Starships to land astronauts on the moon later this decade. Musk’s goal is Mars.

Hours earlier in Florida, another billionaire’s rocket company — Jeff Bezos’ Blue Origin — launched the newest supersized rocket, New Glenn. The rocket reached orbit on its first flight, successfully placing an experimental satellite thousands of miles above Earth. But the first-stage booster was destroyed, missing its targeted landing on a floating platform in the Atlantic.

Houston private equity firm beats target on first investment fund

fresh funds

Houston-based private equity firm Sallyport has raised $160 million for its first investment fund, exceeding the target amount by $10 million.

The Sallyport Partners Fund focuses primarily on investments in founder- and family-owned businesses, corporate carve-outs and startups in various industries.

The firm’s chairman, Doug Foshee, seeded the fund. He and managing partners Kyle Bethancourt and Ryan Howard started the firm in 2023.

“Sallyport Partners Fund was created to utilize the proven processes our team has developed over time to generate value for like-minded investors on a larger and more impactful scale,” Foshee says in a news release.

Investors in the Sallyport fund include entrepreneurs, business executives and influential Texas families. Aside from Foshee, names of the fund’s investors weren’t disclosed.

“We are deeply committed to working hand-in-hand with management teams to drive transformative growth and generate long-term value,” says Bethancourt. “Our operational capabilities are forged from decades of firsthand experience leading, investing in, and building thriving businesses from the ground up. We have a unique appreciation for the management team’s perspective because we’ve been in their shoes.”

Those shoes have covered some pretty impressive ground:

  • Foshee is former chairman, president, and CEO of Houston-based El Paso Corp., which owned and operated a 44,000-mile natural gas pipeline network. In 2012, El Paso merged with Houston-based pipeline company Kinder Morgan in a multibillion-dollar deal.
  • Before Sallyport, Bethancourt was a vice president in the credit division of Blackstone, an investment powerhouse with more than $1 trillion in assets under management. Earlier, he worked at D.E. Shaw & Co., a New York City-based hedge fund with more than $65 billion in assets under management.
  • Before Sallyport, Howard worked at Platform Partners, a Houston-based private equity firm. Earlier, he worked for the natural resources arm of investment banking giant Goldman Sachs.
View All Listings