How to Evaluate Cancer Statistics & News

This page is for patients and families trying to make sense of cancer news and numbers. Cancer headlines are written to be clicked. The number in the headline has usually travelled a long way from the study. You do not need statistical training to catch most of the distortions. You need a handful of questions and the willingness to ask them before you get upset or hopeful.

Every number in this page is invented as an illustration. None of them describes any real cancer, drug or population.

This is the single most useful thing to understand, and it explains most alarming health headlines.

Suppose that among people who never do a particular thing, 2 in 1,000 develop a certain cancer over ten years. Among people who do it regularly, 3 in 1,000 do. That is an increase of 1 person per 1,000. This is the absolute difference. It is also a 50% increase, because 3 is half again as much as 2. This is the relative difference. Both are true.

"Doing X raises your cancer risk by 50%" is a headline. "Doing X raises risk from 2 in 1,000 to 3 in 1,000" is the same finding, and it is the version you can actually make a decision with.

The same trick works in the hopeful direction. A treatment that "cuts the risk of recurrence by a third" might take an invented risk from 30 in 100 to 20 in 100. That is a large and meaningful difference. Or it might take the risk from 3 in 100 to 2 in 100, which may not be worth serious side effects. The relative figure alone cannot tell you which.

Whenever you see a percentage change, ask: a percentage of what? If the article never gives you the starting number, it has not told you the finding.

If a study reports a median survival of, say, 18 months, that means half the people in that study were still alive at 18 months and half were not. It is a description of a group. The National Cancer Institute says statistics from large groups cannot predict exactly what will happen to you. It is not a countdown.

Two things follow. First, medians say nothing about the shape of the tail. In many cancers there is a long right-hand tail of people who live far longer than the median, and the median cannot show you that. Second, the number describes people diagnosed and treated years ago. The National Cancer Institute notes that it takes years to see the benefit of new treatments, so statistics may not reflect treatments used today.

The same caution applies to the survival figure most people encounter first. See what a five-year survival rate actually means. And be careful with vocabulary: "responded to treatment" and "in remission" are not the same as cured, as remission versus cure explains.

What most people care about is living longer and living better. Those take years to measure, so trials often measure something faster that is expected to stand in for them. Examples are tumor shrinkage, a blood marker, or the time until a scan shows growth. These are surrogate endpoints.

They are not worthless. They are how promising treatments reach people sooner. But a surrogate is a bet that the shortcut predicts the thing you care about, and that bet does not always pay off. A drug can delay progression on a scan without helping anyone live longer or feel better. When a headline says a treatment "worked," find out what was measured. If it was a surrogate, the honest reading is "promising, not yet proven."

Also check quality of life and side effects. These are frequently collected and rarely reported in the press.

Where a story started affects how much you can trust it. Here are the usual starting points for a cancer news story:

• A press release from a company or university. These are marketing documents. They select the most favorable framing available and are the direct source of a great many health stories.
• A preprint. Posted publicly before peer review. Useful and legitimate, but not yet checked by independent reviewers. Results can and do change.
• A conference abstract. A few hundred words and a slide. Often the full data never appear, and when they do they are sometimes less impressive.
• A peer-reviewed paper. The strongest of these, though peer review catches sloppiness, not everything.

• Who was studied — how many people, and were they like me in age, stage and prior treatment?
• Compared with what? A placebo, standard care, or nothing at all?
• Was it a randomized trial in people, an observational study, or research in cells or mice?
• Are the numbers absolute or relative, and what was the baseline risk?
• What was actually measured — survival, symptoms, or a surrogate?
• How long were people followed?
• Who funded it, and who wrote the release?
• Has it been peer-reviewed and replicated?
• What are the harms, and are they reported as prominently as the benefits?

If you are weighing a treatment described in the news, what clinical trials are explains how the evidence gets built in phases. Our screening overview covers why detecting more cancers is not automatically the same as saving more lives.

Most of the time, doing this leaves you with a smaller, more uncertain finding than the headline promised. That is not cynicism. It is what the underlying research usually says. The advantage of reading this way is that when something genuinely important does come along, you will be able to tell.

• National Cancer Institute — Cancer Statistics
• National Cancer Institute — Understanding Cancer Prognosis