tags: #publish links: [[Health]], [[Medicine]], [[Mathematics]], [[Psychology]] created: 2021-12-03 Fri --- # Medical screening, survival rate, overdiagnosis and lead-time bias (This is not medical advice: but it is vital information about statistics that everyone should know when interacting with medical screening and treatment. It doesn't mean you shouldn't get tested and treated! Just that you should understand what those numbers mean, ask questions and discuss with a doctor that understands these issues enough to answer them.) Here's the best article I know about this subject: https://nutritionfacts.org/2021/12/02/learn-more-than-97-of-doctors-about-lead-time-bias/ These are tricky concepts, so I'm going to go over them a few times with examples and consequences. ## Survival rate and mortality rate **Survival rate** is a date-bound and specific-sample-bound measurement. People who, *at date X*, *are known to have the disease*, and are still alive *N years after you start counting* at date Y. (The *after you started counting* bit is key.) **Mortality rate** is the number of people *in the whole population* who die from the disease (perhaps "before age Z", *whether or not you knew about them having the disease*. This is the "real" death rate. It is the only number that really matters when we talk about *saving lives*. The critical thing is that *survival rate is not mortality rate*. Survival rate says nothing about the number of people who die overall. It's just the number of a particular set of people you are tracking, and is relative to a particular date (detection date) when you've started tracking each individual. So it actually tells you *nothing* about *mortality rate*. ## Survival rate is distorted by overdiagnosis and lead-time bias If you do either of these things, the *survival rate* appears to change but *mortality rate* does not: * Detect and start tracking more cases which actually wouldn't have killed people anyway (**overdiagnosis**). You've diluted your set of people so the survival rate seems better. But you're just counting some extra folks who wouldn't have died. * Detect earlier, changing the date from which you start counting the "survived for X years" date window for each particular person (**lead-time bias**). You've moved the window earlier, so of course they're less likely to die within X years. But they would still die on the same date. In both cases, survival rate has changed *simply because you changed the set of people or the date from which you're counting*, regardless of whether you did anything successful at saving lives! This is pretty much the definition of misleading statistics. You *cannot* meaningfully compare treatments or screening regimes based on survival rates, except in cases where you can avoid the effect of lead-time bias or overdiagnosis. See the examples below. ## Mortality rate is an estimate too Note that mortality rate is necessarily an estimate! There are people out there who die from the disease undetected or misdiagnosed. This makes it even harder to compare screening regimes. You may get to know about new cases or correctly attribute more deaths to the disease. But this didn't actually change the real mortality rate - those people were sick anyway, you just got more information about it. ## Examples For example, if you want to experiment with whether a new treatment saves lives, you can design a study where you take a set of patients at the diagnosis date, and treat some of them with your new thing and some of them the existing way, and yes in this situation you can compare the survival rates of your two sets within your study population, because the start date is the same and you are not changing the testing regime to distort the population. (You'll still be at risk of over-treatment of people who would have survived anyway, though. Eventual death rate from the disease, or age at death, is a better measure than N-year survival.) Another example: if you want to experiment with whether a new screening test saves lives e.g. by allowing earlier treatment, you cannot use survival rates to check this, because you aren't comparing like with like. The N-year survival rate of a group detected with test A cannot be compared to the survival rate of a group detected with test B, because they will detect different fractions of the absolute population of sick patients (overdiagnosis), and they will detect them earlier or later (lead-time bias), so both the pool and the date at which you start counting the N years are different. You *must* instead compare whether there was an improvement to whole-population mortality rate, or age at death from the disease. If you want to experiment with whether a new test just detects cases earlier (ignoring treatment), then even this is difficult. Again, you cannot use survival rates alone to assess this. If the new test detects the same set of sick people earlier, then it will move the window earlier and give a higher N-year survival rate: great. However if it detects a bigger set of people who have the disease but would not naturally get sick or die from it (overdiagnosis), or some people who don't actually have the disease at all (false positives), these will also improve the N-year survival rate, but that is a misleading useless conclusion because those people don't benefit from treatment and you haven't saved any lives. ## Is earlier detection good? You cannot assume that earlier detection is good on its own. - It's only good if you're detecting more cases where treatment could prolong life (improve mortality rate, not N-year survival rate). - It's not good if it only causes stress, reduced quality of life through aggressive treatment, and expense. You need to look at the detection and treatment regimes in the context of overall mortality and quality of life improvements, instead, and also not just those in your distorted sample. ## Conclusion Population statistics is hard and counter-intuitive. Evidence of value of screening and treatment must be evaluated carefully through the lens of lead-time bias and overdiagnosis, to make sure is is actually beneficial at saving or prolonging lives. It may be, but this needs careful design to prove meaningfully. We must also consider whether the quality of life and cost consequences of false positives are acceptable.