Why firms should conduct randomized controlled trials

Maria Guadalupe, INSEAD Associate Professor of Economics and Political Science

Field experiments help companies to make better decisions, while findings aid academic work.

Making mistakes and learning from them is par for the course for companies. But years of bad decisions could lead to the downfall of conglomerates.

In the case of General Electric, a major engine of America’s economy for more than a century, the firm shook investor confidence when it announced last year that it would have to slash its stock dividend in half – only the second time it had to cut its dividend since the Great Depression.

By the end of last year, General Electric shed more than US$100 billion in market value since November 2016, undone by a long history of overpriced acquisitions, opaque accounting and its overly complex business model.

For firms to make the right decisions, it would be useful to become long-term laboratories that conduct experiments – such as randomized controlled trials (RCTs) – in collaboration with academic researchers, says Guido Friebel, an economist and professor of human resources at Goethe University Frankfurt.

As part of my work as the Academic Director of the Randomized Control Trials (RCT) Lab at INSEAD, which works with firms to design such experiments, I interviewed Friebel during his recent visit to the school.

Friebel says such partnerships are a “win-win” proposition for both sides, as they can learn from each other. By allowing researchers to test and explore new theories, firms can make better decisions based on the results, while researchers can use the findings for their research and in their teaching.

RCTs are often used in scientific and medical studies to test the efficacy of an experimental treatment or to uncover adverse effects. They minimise selection bias by randomly allocating participants into two groups – one with the new treatment, the other a placebo treatment – while keeping other variables constant.

Experiment in practice

Friebel tells of how he and his team of researchers helped a German retail chain improve its performance amid increased competition by running a randomized experiment.

In 2014, the researchers introduced a bonus for teams that beat sales targets in half of the chain’s 193 bakery shops with 1,300 workers. It was a team bonus because the firm did not measure individual performance and depended on workers to be flexible in performing a variety of tasks – such as operating the till, serving customers, handling the goods deliveries and operating the oven.

Shops that were offered the bonus saw sales and customer visits rise by about 3 percent on average compared to those that did not, while workers’ wages increased by 2.3 percent. For the retail chain, each bonus dollar generated an extra US$3.80 of sales and $2.10 of operational profit.

The firm consequently decided to distribute the bonus in all of its shops after the experiment.

In addition, the researchers helped the firm to tackle its high turnover of workers. Following their intervention in a randomized and controlled fashion, they significantly reduced staff turnover by giving greater personnel management responsibility to middle managers.

The researchers were surprised, however, when that did not lead to increased sales, Friebel says. The gains from store managers’ efforts in making workers happy and motivated were “unfortunately cancelled out” by the costs of managers spending less time dealing with customers and goods.

As Friebel’s experiment illustrates, firms can better generate knowledge and understand the impact of their policies and the trade-offs that arise by working with researchers.

Even so, the INSEAD Randomized Controlled Trials Lab has found that firms have reservations about using RCTs. As Friebel notes, however, many firms “experiment all the time” but they often fail to do so in a controlled way. This lack of structure prevents companies from finding out whether their “certain intuitions” about the use of instruments – such as a new compensation scheme – was correct after implementation.

Friebel adds that firms should not be afraid to experiment because it can help them to learn whether their preconceptions about what would work in various areas – such as marketing campaigns, human resource systems or production lines – were right.

In the digital age, Big Data (massive compilations of snippets of information) is increasingly popular as an analytical tool with companies – mostly major tech firms such as Google and Amazon – that can yield insights into consumers and workers.

Although many small firms lack access to Big Data, they have a wealth of in-house data which they are often unable to fully exploit. That is not surprising, Friebel says, as top managers lack the time to conduct proper analysis of data on sales, personnel and performance, or they may not have skilled data scientists on their teams. This is where collaboration with researchers can work in both parties’ benefit, he says.

Value and limitations

For firms to realise their goals from RCTs, researchers and executives have to understand the objectives of both parties and how each side works, according to Friebel. They have to foster “an atmosphere of trust”, and it is very important for firms to have a culture of performance and transparency rather than one of bureaucracy.

In Friebel’s experience, researchers often face hurdles in carrying out RCTs because some executives felt threatened by the experiments. So it is of utmost importance that the top management voice their full support and stress that the researchers have no self-interest other than generating knowledge from the trials, Friebel says.

One oft-forgotten aspect of using RCTs is the value of trying and failing. By experimenting, firms can fully understand a new policy’s causal effect and its consequences. Indeed, the willingness to find out when things don’t work is useful to firms before they roll out a policy across the organisation.

RCTs also prevent firms from wasting money by implementing policies that worked for other companies but are not right for them, Friebel says.

As the use of RCTs is dependent on the unique circumstances of the firms involved, Friebel cautions against easy generalisations of the results of experiments to firms with different contexts. For instance, there could be circumstances in which the introduction of team incentives may not necessarily boost both worker performance and satisfaction. This means that firms can learn about how their organisation truly works if they are willing to investigate and understand their organisational dynamics.

Maria Guadalupe is an Associate Professor of Economics and Political Science and the Academic Director of the Randomized Control Trials (RCT) Lab at INSEAD.

This article is republished courtesy of http://knowledge.insead.edu. Copyright INSEAD 2018.