Academics have learned quickly that investigations based on data from online research agencies can have problems. Here are those problems and alternatives, according to Rice University researchers. Photo via Getty Images

Academics are learning quickly that investigations based on data from online research agencies have their drawbacks. Thousands of such studies are released every year – and if the data is compromised, so too are the studies themselves.

So it’s natural for researchers, and the managers who rely on their findings, to be concerned about potential problems with the samples they’re studying. Among them: participants who aren’t in the lab and researchers who can’t see who is taking their survey, what they are doing while answering questions or even if they are who they claim to be online. In the wake of a 2018 media piece about Amazon’s Mechanical Turks Service, “Bots on Amazon’s MTurk Are Ruining Psychology Studies,” one psychology professor even mused, “I wonder if this is the end of MTurk research?” (It wasn’t).

To tackle this problem, Rice Business professor Mikki Hebl joined colleagues Carlos Moreno and Christy Nittrouer of Rice University along with several other colleagues to highlight the value of other research methods. Four alternatives – field experiments, archival data, observations and big data – represent smart alternatives to overreliance on online surveys. These methods also have the advantage of challenging academics to venture outside of their laboratories and examine real people and real data in the real world.

Field experiments have been around for decades. But their value is hard to overestimate. Unlike online studies, field experiments enhance the role of context, especially in settings that are largely uncontrolled. It’s hard to fake a field experiment in order to create positive results since each one costs a considerable time and money.

And field experiments can yield real-life results with remarkable implications for society at large. Consider one experiment among 56 middle schools in New Jersey, which found that spreading anti-conflict norms was hugely successful in reducing the need for disciplinary action. Such studies have an impact well beyond what could be achieved with a simple online survey.

The best way to get started with a good field experiment, Hebl and her colleagues wrote, is for researchers to think about natural field settings to which they have access, either personally or by leveraging their networks. Then, researchers should think about starting with the variables critical for any given setting and which they would most like to manipulate to observe the outcome. When choosing variables, it’s helpful to start by thinking about what variable might have conditions leading to the greatest degree of behavior change if introduced into the setting.

Archival data is another excellent way to work around the limitations of online surveys, the researchers argue. These data get around some of the critical drawbacks of field research, including problems around how findings apply in a more general way. Archival data, especially in the form of state or national level data sets, provide information and insight into a large, diverse set of samples that are more representative of the general population than online studies.

Archival data can also help answer questions that are either longitudinal or multilevel in nature, which can be particularly tricky or even impossible to capture with data collected by any single research team. As people spend increasing amounts of time on social media, the internet also serves as a source of newer forms of archival data that can lend unique insights into individuals’ thoughts, attitudes, and behaviors over time.

With every passing year, technology becomes increasingly robust and adept at collecting massive amounts of data on an endless variety of human behavior. For the scientists who research social and personality psychology, the term “big data” refers not only to very large sets of data but also to the tools and techniques that are used to analyze it. The three defining properties of Big Data in this context include the speed of data processing and collection, the vast amount of data being analyzed and the sheer variety of data available.

By using big data, social scientists can generate research based on various conditions, as well as collect data in natural settings. Big data also offers the opportunity to consolidate information from huge and highly diverse stores of data. This technology has many applications, including psychological assessments and improving security in airports and other transportation hubs. In future research, Hebl and her team noted, researchers will likely leverage big data and its applications to detect our unconscious emotions.

Big data, archival information and field studies can all be used in conjunction with each other to maximize the fidelity of research. But researchers shouldn’t forget even more old-fashioned techniques, including the oldest: keen observation. With observation, there are often very few, if any, manipulations and the goal is simply to systematically record the way people behave.

Researchers – and the managers who make decisions based on their findings – should consider the advantages of old-style, often underused methodologies, Hebl and her colleagues argue. Moving beyond the college laboratory and digital data survey-collection platforms and into the real world offers some unparalleled advantages to science. For the managers whose stock prices may hinge on this science, it’s worth knowing – and understanding – how your all-important data was gathered.

------

This article originally ran on Rice Business Wisdom and is based on research from Mikki Hebl, the Martha and Henry Malcolm Lovett Professor of psychology at Rice University, and Carlos Moreno and Christy Nittrouer, who are graduate students at Rice University. Additional researchers include Ho Kwan Cheung, Eden B. King, and Hannah Markellis of George Mason University.

Ad Placement 300x100
Ad Placement 300x600

CultureMap Emails are Awesome

Houston femtech co. debuts new lactation and wellness pods

mom pod

Houston-based femtech company Work&, previously known as Work&Mother, has introduced new products in recent months aimed at supporting working mothers and the overall health of all employees.

The company's new Lactation Pod and Hybrid Pod serve as dual-use lactation and wellness spaces to meet employer demand, the company shared in a news release. The compact pods offer flexible design options that can serve permanent offices and nearly all commercial spaces.

They feature a fully compliant lactation station while also offering wellness functionalities that can support meditation, mental health, telehealth and prayer. In line with Work&'s other spaces, the pods utilize the Work& scheduling platform, which prioritizes lactation bookings to help employers comply with the PUMP Act.

“This isn’t about perks,” Jules Lairson, Work& co-founder and COO, said in the release. “It’s about meeting people where they are—with dignity and intentional design. That includes the mother returning to work, the employee managing anxiety, and everyone in between.”

According to the company, several Fortune 500 companies are already using the pods, and Work& has plans to grow the products' reach.

Earlier this year, Work& introduced its first employee wellness space at MetroNational’s Memorial City Plazas, representing Work&'s shift to offer an array of holistic health and wellness solutions for landlords and tenants.

The company, founded in 2017 by Lairson and CEO Abbey Donnell, was initially focused on outfitting commercial buildings with lactation accommodations for working parents. While Work& still offers these services through its Work&Mother branch, the addition of its Work&Wellbeing arm allowed the company to also address the broader wellness needs of all employees.

The company rebranded as Work& earlier this year.

Rice biotech studio secures investment from Modi Ventures, adds founder to board

fresh funding

RBL LLC, which supports commercialization for ventures formed at the Rice University Biotech Launch Pad, has secured an investment from Houston-based Modi Ventures.

Additionally, RBL announced that it has named Sahir Ali, founder and general partner of Modi Ventures, to its board of directors.

Modi Ventures invests in biotech companies that are working to advance diagnostics, engineered therapeutics and AI-driven drug discovery. The firm has $134 million under management after closing an oversubscribed round this summer.

RBL launched in 2024 and is based out of Houston’s Texas Medical Center Helix Park. William McKeon, president and CEO of the TMC, previously called the launch of RBL a “critical step forward” for Houston’s life sciences ecosystem.

“RBL is dedicated to building companies focused on pioneering and intelligent bioelectronic therapeutics,” Ali said in a LinkedIn post. “This partnership strengthens the Houston biotech ecosystem and accelerates the transition of groundbreaking lab discoveries into impactful therapies.”

Ali will join board members like managing partner Paul Wotton, Rice bioengineering professor Omid Veiseh, scientist and partner at KdT Ventures Rima Chakrabarti, Rice alum John Jaggers, CEO of Arbor Biotechnologies Devyn Smith, and veteran executive in the life sciences sector James Watson.

Ali has led transformative work and built companies across AI, cloud computing and precision medicine. Ali also serves on the board of directors of the Drug Information Association, which helps to collaborate in drug, device and diagnostics developments.

“This investment by Modi Ventures will be instrumental to RBL’s growth as it reinforces confidence in our venture creation model and accelerates our ability to develop successful biotech startups,” Wotton said in the announcement. "Sahir’s addition to the board will also amplify this collaboration with Modi. His strategic counsel and deep understanding of field-defining technologies will be invaluable as we continue to grow and deliver on our mission.”