Research history

Inert source text, including its original formatting. It is not executed as page markup.

A **randomized controlled trial** (RCT) assigns people, villages or schools to receive a program or not by chance, like a coin flip. Because chance decides, the groups are expected to be alike in everything except the program. The difference in their outcomes then estimates the program’s effect. A trial is a way of seeing the [counterfactual](/record/encyclopedia/counterfactual-impact/): what would have happened without the program.

## In development economics

In 2019 the Royal Swedish Academy of Sciences awarded the Prize in Economic Sciences in Memory of Alfred Nobel to Abhijit Banerjee and Esther Duflo of MIT and Michael Kremer of Harvard “for their experimental approach to alleviating global poverty.” The Academy said they split poverty into smaller, more precise questions, often best answered by carefully designed experiments among the people most affected. It credited one of their studies with leading to remedial tutoring programs that have reached more than five million children in India. J-PAL, the research center Banerjee and Duflo founded at MIT in 2003 with Sendhil Mullainathan, says its network has led more than 2,700 randomized evaluations.

## What trials can show

Trials can confirm that something works and show how much. A Cochrane review of five trials with about 200,000 participants found that insecticide-treated nets cut child deaths from all causes by about 17 percent, and rated that evidence high-certainty.

They can also show that something does harm. A Cochrane review of nine trials of “Scared Straight” programs found that they increased offending compared with doing nothing.

## The critique

Angus Deaton and Nancy Cartwright argue that the public, and sometimes researchers, put too much trust in RCTs compared with other methods. Randomization does not guarantee that the groups in any one trial are equal in everything but the treatment. At best a trial gives an unbiased estimate, which they say is of limited practical value on its own. Its result applies to the people in the trial, and carrying it to a new place or population needs a further argument about why the program worked. They see trials as useful inside a cumulative program of research that combines them with other methods.

## Where trials stop

Many of the questions Charitology cares most about cannot be randomized. Some national laws and cultural changes cannot feasibly be assigned at random, though policy rollout or components can sometimes be randomized. Charitology hypothesizes that limits on trial evidence can affect funding for structure and [source](/record/encyclopedia/source-depth/) work. Finding other ways to measure it is [Open Problem 3](/open-problems/measuring-the-source/). How often trial findings hold up when repeated is [Open Problem 10](/open-problems/the-replication-rate/).

{
  "adapterVersion": "research-encyclopedia-v1",
  "authorship": {
    "actorIds": [],
    "complete": false,
    "refs": [],
    "state": "unknown"
  },
  "hash": "40be57fbec79d5453394451a9b82f32704e1349fc5ef5608a83b5325ace5c7e1",
  "id": "version:concept-randomized-controlled-trials-v1",
  "previousId": null,
  "recordId": "concept:randomized-controlled-trials",
  "sources": [
    {
      "selector": {
        "kind": "entry"
      },
      "sourceId": "source:encyclopedia-randomized-controlled-trials-v1"
    }
  ],
  "subject": {
    "body": "\nA **randomized controlled trial** (RCT) assigns people, villages or schools to receive a program or not by chance, like a coin flip. Because chance decides, the groups are expected to be alike in everything except the program. The difference in their outcomes then estimates the program’s effect. A trial is a way of seeing the [counterfactual](/record/encyclopedia/counterfactual-impact/): what would have happened without the program.\n\n## In development economics\n\nIn 2019 the Royal Swedish Academy of Sciences awarded the Prize in Economic Sciences in Memory of Alfred Nobel to Abhijit Banerjee and Esther Duflo of MIT and Michael Kremer of Harvard “for their experimental approach to alleviating global poverty.” The Academy said they split poverty into smaller, more precise questions, often best answered by carefully designed experiments among the people most affected. It credited one of their studies with leading to remedial tutoring programs that have reached more than five million children in India. J-PAL, the research center Banerjee and Duflo founded at MIT in 2003 with Sendhil Mullainathan, says its network has led more than 2,700 randomized evaluations.\n\n## What trials can show\n\nTrials can confirm that something works and show how much. A Cochrane review of five trials with about 200,000 participants found that insecticide-treated nets cut child deaths from all causes by about 17 percent, and rated that evidence high-certainty.\n\nThey can also show that something does harm. A Cochrane review of nine trials of “Scared Straight” programs found that they increased offending compared with doing nothing.\n\n## The critique\n\nAngus Deaton and Nancy Cartwright argue that the public, and sometimes researchers, put too much trust in RCTs compared with other methods. Randomization does not guarantee that the groups in any one trial are equal in everything but the treatment. At best a trial gives an unbiased estimate, which they say is of limited practical value on its own. Its result applies to the people in the trial, and carrying it to a new place or population needs a further argument about why the program worked. They see trials as useful inside a cumulative program of research that combines them with other methods.\n\n## Where trials stop\n\nMany of the questions Charitology cares most about cannot be randomized. Some national laws and cultural changes cannot feasibly be assigned at random, though policy rollout or components can sometimes be randomized. Charitology hypothesizes that limits on trial evidence can affect funding for structure and [source](/record/encyclopedia/source-depth/) work. Finding other ways to measure it is [Open Problem 3](/open-problems/measuring-the-source/). How often trial findings hold up when repeated is [Open Problem 10](/open-problems/the-replication-rate/).\n",
    "collection": "encyclopedia",
    "frontmatter": {
      "aliases": [
        "RCTs",
        "randomized evaluations",
        "field experiments"
      ],
      "conditions": [
        {
          "collection": "conditions",
          "id": "measurement-gap"
        }
      ],
      "related": [
        "/record/encyclopedia/counterfactual-impact/",
        "/record/principles/04-evidence-and-its-grades/",
        "/programs/evidence-reviews/",
        "/programs/replication-program/",
        "/open-problems/the-replication-rate/"
      ],
      "sources": [
        {
          "note": null,
          "publisher": "The Royal Swedish Academy of Sciences",
          "title": "The Prize in Economic Sciences 2019",
          "url": "https://www.kva.se/en/news/ekonomipriset-2019-2/",
          "year": 2019
        },
        {
          "note": null,
          "publisher": "Abdul Latif Jameel Poverty Action Lab (J-PAL)",
          "title": "About us",
          "url": "https://www.povertyactionlab.org/about-us",
          "year": null
        },
        {
          "note": "Freely readable at https://pmc.ncbi.nlm.nih.gov/articles/PMC6019115/",
          "publisher": "Angus Deaton and Nancy Cartwright, Social Science & Medicine 210",
          "title": "Understanding and misunderstanding randomized controlled trials",
          "url": "https://doi.org/10.1016/j.socscimed.2017.12.005",
          "year": 2018
        },
        {
          "note": null,
          "publisher": "Joseph Pryce, Marty Richardson and Christian Lengeler, Cochrane Database of Systematic Reviews",
          "title": "Insecticide-treated nets for preventing malaria (Cochrane systematic review)",
          "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC6418392/",
          "year": 2018
        },
        {
          "note": null,
          "publisher": "Anthony Petrosino and colleagues, Cochrane Database of Systematic Reviews",
          "title": "“Scared Straight” and other juvenile awareness programs for preventing juvenile delinquency",
          "url": "https://www.cochrane.org/evidence/CD002796_scared-straight-and-other-juvenile-awareness-programs-preventing-juvenile-delinquency",
          "year": 2022
        }
      ],
      "summary": "Experiments that assign people or communities to a program or to a comparison group by chance, so that differences in outcomes can be credited to the program. They reshaped development economics, and their limits are still debated.",
      "title": "Randomized controlled trials"
    },
    "type": "entry"
  },
  "subjectHash": "d6fdc38227c624e96c38e676691488ab366ba65b4ebd5ac2705fa1dc846fdf0f",
  "version": 1
}