Academics have learned quickly that investigations based on data from online research agencies can have problems. Here are those problems and alternatives, according to Rice University researchers. Photo via Getty Images

Academics are learning quickly that investigations based on data from online research agencies have their drawbacks. Thousands of such studies are released every year – and if the data is compromised, so too are the studies themselves.

So it’s natural for researchers, and the managers who rely on their findings, to be concerned about potential problems with the samples they’re studying. Among them: participants who aren’t in the lab and researchers who can’t see who is taking their survey, what they are doing while answering questions or even if they are who they claim to be online. In the wake of a 2018 media piece about Amazon’s Mechanical Turks Service, “Bots on Amazon’s MTurk Are Ruining Psychology Studies,” one psychology professor even mused, “I wonder if this is the end of MTurk research?” (It wasn’t).

To tackle this problem, Rice Business professor Mikki Hebl joined colleagues Carlos Moreno and Christy Nittrouer of Rice University along with several other colleagues to highlight the value of other research methods. Four alternatives – field experiments, archival data, observations and big data – represent smart alternatives to overreliance on online surveys. These methods also have the advantage of challenging academics to venture outside of their laboratories and examine real people and real data in the real world.

Field experiments have been around for decades. But their value is hard to overestimate. Unlike online studies, field experiments enhance the role of context, especially in settings that are largely uncontrolled. It’s hard to fake a field experiment in order to create positive results since each one costs a considerable time and money.

And field experiments can yield real-life results with remarkable implications for society at large. Consider one experiment among 56 middle schools in New Jersey, which found that spreading anti-conflict norms was hugely successful in reducing the need for disciplinary action. Such studies have an impact well beyond what could be achieved with a simple online survey.

The best way to get started with a good field experiment, Hebl and her colleagues wrote, is for researchers to think about natural field settings to which they have access, either personally or by leveraging their networks. Then, researchers should think about starting with the variables critical for any given setting and which they would most like to manipulate to observe the outcome. When choosing variables, it’s helpful to start by thinking about what variable might have conditions leading to the greatest degree of behavior change if introduced into the setting.

Archival data is another excellent way to work around the limitations of online surveys, the researchers argue. These data get around some of the critical drawbacks of field research, including problems around how findings apply in a more general way. Archival data, especially in the form of state or national level data sets, provide information and insight into a large, diverse set of samples that are more representative of the general population than online studies.

Archival data can also help answer questions that are either longitudinal or multilevel in nature, which can be particularly tricky or even impossible to capture with data collected by any single research team. As people spend increasing amounts of time on social media, the internet also serves as a source of newer forms of archival data that can lend unique insights into individuals’ thoughts, attitudes, and behaviors over time.

With every passing year, technology becomes increasingly robust and adept at collecting massive amounts of data on an endless variety of human behavior. For the scientists who research social and personality psychology, the term “big data” refers not only to very large sets of data but also to the tools and techniques that are used to analyze it. The three defining properties of Big Data in this context include the speed of data processing and collection, the vast amount of data being analyzed and the sheer variety of data available.

By using big data, social scientists can generate research based on various conditions, as well as collect data in natural settings. Big data also offers the opportunity to consolidate information from huge and highly diverse stores of data. This technology has many applications, including psychological assessments and improving security in airports and other transportation hubs. In future research, Hebl and her team noted, researchers will likely leverage big data and its applications to detect our unconscious emotions.

Big data, archival information and field studies can all be used in conjunction with each other to maximize the fidelity of research. But researchers shouldn’t forget even more old-fashioned techniques, including the oldest: keen observation. With observation, there are often very few, if any, manipulations and the goal is simply to systematically record the way people behave.

Researchers – and the managers who make decisions based on their findings – should consider the advantages of old-style, often underused methodologies, Hebl and her colleagues argue. Moving beyond the college laboratory and digital data survey-collection platforms and into the real world offers some unparalleled advantages to science. For the managers whose stock prices may hinge on this science, it’s worth knowing – and understanding – how your all-important data was gathered.

------

This article originally ran on Rice Business Wisdom and is based on research from Mikki Hebl, the Martha and Henry Malcolm Lovett Professor of psychology at Rice University, and Carlos Moreno and Christy Nittrouer, who are graduate students at Rice University. Additional researchers include Ho Kwan Cheung, Eden B. King, and Hannah Markellis of George Mason University.

Ad Placement 300x100
Ad Placement 300x600

CultureMap Emails are Awesome

Intuitive Machines to acquire NASA-certified deep space navigation company

space deal

Houston-based space technology, infrastructure and services company Intuitive Machines has agreed to buy Tempe, Arizona-based aerospace company KinetX for an undisclosed amount.

The deal is expected to close by the end of this year, according to a release from the company.

KinetX specializes in deep space navigation, systems engineering, ground software and constellation mission design. It’s the only company certified by NASA for deep space navigation. KinetX’s navigation software has supported both of Intuitive Machines’ lunar missions.

Intuitive Machines says the acquisition marks its entry into the precision navigation and flight dynamics segment of deep space operations.

“We know our objective, becoming an indispensable infrastructure services layer for space exploration, and achieving it requires intelligent systems and exceptional talent,” Intuitive Machines CEO Steve Altemus said in the release. “Bringing KinetX in-house gives us both: flight-proven deep space navigation expertise and the proprietary software behind some of the most ambitious missions in the solar system.”

KinetX has supported deep space missions for more than 30 years, CEO Christopher Bryan said.

“Joining Intuitive Machines gives our team a broader operational canvas and shared commitment to precision, autonomy, and engineering excellence,” Bryan said in the release. “We’re excited to help shape the next generation of space infrastructure with a partner that understands the demands of real flight, and values the people and tools required to meet them.”

Intuitive Machines has been making headlines in recent weeks. The company announced July 30 that it had secured a $9.8 million Phase Two government contract for its orbital transfer vehicle. Also last month, the City of Houston agreed to add three acres of commercial space for Intuitive Machines at the Houston Spaceport at Ellington Airport. Read more here.

Japanese energy tech manufacturer moves U.S. headquarters to Houston

HQ HOU

TMEIC Corporation Americas has officially relocated its headquarters from Roanoke, Virginia, to Houston.

TMEIC Corporation Americas, a group company of Japan-based TMEIC Corporation Japan, recently inaugurated its new space in the Energy Corridor, according to a news release. The new HQ occupies the 10th floor at 1080 Eldridge Parkway, according to ConnectCRE. The company first announced the move last summer.

TMEIC Corporation Americas specializes in photovoltaic inverters and energy storage systems. It employs approximately 500 people in the Houston area, and has plans to grow its workforce in the city in the coming year as part of its overall U.S. expansion.

"We are thrilled to be part of the vibrant Greater Houston community and look forward to expanding our business in North America's energy hub," Manmeet S. Bhatia, president and CEO of TMEIC Corporation Americas, said in the release.

The TMEIC group will maintain its office in Roanoke, which will focus on advanced automation systems, large AC motors and variable frequency drive systems for the industrial sector, according to the release.

TMEIC Corporation Americas also began operations at its new 144,000-square-foot, state-of-the-art facility in Brookshire, which is dedicated to manufacturing utility-scale PV inverters, earlier this year. The company also broke ground on its 267,000-square-foot manufacturing facility—its third in the U.S. and 13th globally—this spring, also in Waller County. It's scheduled for completion in May 2026.

"With the global momentum toward decarbonization, electrification, and domestic manufacturing resurgence, we are well-positioned for continued growth," Bhatia added in the release. "Together, we will continue to drive industry and uphold our legacy as a global leader in energy and industrial solutions."

---

This article originally appeared on EnergyCapitalHTX.com.

2 Texas cities named on LinkedIn's inaugural 'Cities on the Rise'

jobs data

LinkedIn’s 2025 Cities on the Rise list includes two Texas cities in the top 25—and they aren’t Houston or Dallas.

The Austin metro area came in at No. 18 and the San Antonio metro at No. 23 on the inaugural list that measures U.S. metros where hiring is accelerating, job postings are increasing and talent migration is “reshaping local economies,” according to the company. The report was based on LinkedIn’s exclusive labor market data.

According to the report, Austin, at No. 18, is on the rise due to major corporations relocating to the area. The datacenter boom and investments from tech giants are also major draws to the city, according to LinkedIn. Technology, professional services and manufacturing were listed as the city’s top industries with Apple, Dell and the University of Texas as the top employers.

The average Austin metro income is $80,470, according to the report, with the average home listing at about $806,000.

While many write San Antonio off as a tourist attraction, LinkedIn believes the city is becoming a rising tech and manufacturing hub by drawing “Gen Z job seekers and out-of-state talent.”

USAA, U.S. Air Force and H-E-B are the area’s biggest employers with professional services, health care and government being the top hiring industries. With an average income of $59,480 and an average housing cost of $470,160, San Antonio is a more affordable option than the capital city.

The No. 1 spot went to Grand Rapids due to its growing technology scene. The top 10 metros on the list include:

  • No. 1 Grand Rapids, Michigan
  • No. 2 Boise, Idaho
  • No. 3 Harrisburg, Pennsylvania
  • No. 4 Albany, New York
  • No. 5 Milwaukee, Wisconsin
  • No. 6 Portland, Maine
  • No. 7 Myrtle Beach, South Carolina
  • No. 8 Hartford, Connecticut
  • No. 9 Nashville, Tennessee
  • No. 10 Omaha, Nebraska

See the full report here.