Actively against Assistive Incompetence
Some time ago I vented about A.I. in a post named Assistive Incompetence.
All my posts are written from a humanist and disinterested perspective. I have no control whatsoever about what people do with what I write, good or bad. I have the power to withhold my thoughts, but I feel I have the duty to air them. And so I do. This was the most read post I ever wrote, by far, and after surprise came acceptance.
As people hinge their lives on the results of varied statistics such as weather forecasts and medical screening tests, I want to explain something super fundamental about these statistics.
This is something so fundamental that I keep wondering why it is not taught in secondary level schools. Therefore I will explain it as if I was going to explain it to a 16 year old.
It starts with a simple context and a simple question.
Imagine your mum just got the result of her breast screening test (mammography), and it is positive. What does it mean?
Indeed the question is simple. Indeed the answer is anything but simple.
There will be a crisis that will impact all the family members, the real friends, some work colleagues... The main protagonist will come under the daily scrutiny of the family, and will be assailed by questions from every conceivable angle in a consumerist society, and a few will be related to her mental and physical health. We are already in anything but simplicity. If you are better equiped than a teaspoon in terms of emotional range, then you see the depth and breadth of the ocean. Actually, that's not my point, but that needed to be put in perspective for what follows.
The question I want to work on is:
What are the actual chances that your mother has cancer?
In a class context, I would count hands in order to draw a distribution on the whiteboard. If you wish, write what you believe on a piece of paper and you'll look at it again at the end.
Then I would introduce each of the 3 following pieces of information one after another, while stirring the assembly, because there is enough material there to make young people ask intelligent questions and make intelligent inferences.
Clue #1:
As you may know, when a screening test comes out positive, additional investigations are performed to find a tumour, and then if a tumour is found a biopsy is performed to find out if the tumour is benign or malignant. It works the same for any screening test, based on blood tests or x-ray or else, for women and men. Colorectal and prostate cancer are included.
Clue #2:
In OCDE countries, ~10% of the women will develop breast cancer during the course of their lives.
If a person who undertakes a breast cancer screening test has cancer, then there’s a 80% chance that the test will come out as positive.
If a person who undertakes a breast cancer screening test doesn’t have cancer, then there’s a 9.6% that the test will come out positive.
Clue #3:
The question What are the chances that your mother has breast cancer given that her mammography turned out positive? is NOT THE SAME as What are the chances that the screening test turns out positive for a person who has a cancer?
It is super important to note and understand that:
- Question #2 is important for the people who build tests.
- Question #1 is important for the patient and the family of the patient.
Here we go!
Point #2 gave all the necessary and sufficient information to answer the question. Yet one needs an additional skill in order to use the information: conditional probabilities.
Worth to note at that point: the approach that follows is universal and mathematically robust.
Step #1
We build a table like so with the information of Hint #2:
| Population with disease (10%) | Population without disease (90%) | |
|---|---|---|
| Test result is positive | 80% | 9.6% |
| -------- | -------- | -------- |
| Test result is negative |
It reads as: for a person who has the disease, there is 80% chances that the test gives a positive result.
Step #2
We infer the results of the last two cells:
| Population with disease (10%) | Population without disease (90%) | |
|---|---|---|
| Test result is positive | 80% | 9.6% |
| -------- | -------- | -------- |
| Test result is negative | 20% | 90.4% |
Step #3
Definition: A probability is the sum of all the desired events divided by the sum of all the possible outcomes.
We need to prepare things a little bit:
Step #3.1
Therefore we need to calculate the probabilities of getting a positive/negative test associated with a person who has/not the disease.
| Population with disease (10%) | Population without disease (90%) | |
|---|---|---|
| Test result is positive | 0.1*0.8 = 0.08 | 0.9*0.096 = 0.0864 |
| -------- | -------- | -------- |
| Test result is negative | 0.1*0.2 = 0.02 | 0.9*0.904 = 0.814 |
It reads as:
The (almost) probability of getting a positive test in a healthy person is 8.64%
FYI, the names of these probabilities depends of whom you are speaking to:
| Population with disease (10%) | Population without disease (90%) | |
|---|---|---|
| Test result is positive | True positive a.k.a. Power | False positive a.k.a. Type I error |
| -------- | -------- | -------- |
| Test result is negative | False negative a.k.a. Type II error | True negative |
Step #3.2
Reminder: A probability is the sum of all the desired events divided by the sum of all the possible outcomes.
In order to get a proper probability, we need to calculate the sums of the possible outcomes:
- Outcome #1 = positive result = 0.08 + 0.0864 = 0.1664
- Outcome #2 = negative result = 0.02 + 0.814 = 0.834
Step #3.3
The answer to the question What are the chances to have cancer when the result of the test is positive? comes from dividing :
- the true positive associated with the test (positive result x patient with cancer): 0.08
- with the sum of the possible outcome positive result: 0.1664
The actual probability of having cancer when the test comes out as positive is: 0.08/0.1664 = 0.481 = 48.1%
That's right. The flip of a coin. Do not be disappointed. Read on.
Step #4
DO NOT JUDGE THIS NUMBER WITHOUT LOOKING AT THE OTHER NUMBER(S)!
You are more than a social security number, you are more than a personal tax number, you are more than an entry on the birth register. YOU can not be reduced to a single value, whatever the number of digits after the comma. Therefore DO NOT judge values without their context.
The actual probability of having cancer when the test comes out as negative is: 0.02/0.834 = 0.0239 = 2.4%
That's right. These values make only sense when one looks at both sides together. It is clearly better to do further investigations when this kind of test returns a positive result under the given conditions.
TAKE AWAYS
When you are given a chance value of rain or having cancer, take a deep breath and remember that 99% of the time this is the answer to a question you are not interested in.
When you do the maths, write down the questions as conditional propositions: the chances of X given Y is true/false, before making the calculations or interpreting the numbers. This is the difficult part of the exercise because one must translate semantic-based propositions in logical (mathematical) propositions, and as for any translation job one needs to be really proficient in both languages to make a bullet proof translation.
When you do the maths, ALWAYS compare the 2 numbers you get at the end.
When you do the maths, do not change the orientation of that table without extreme care. If you do change the orientation, make sure to reword your sentences accordingly. Here is a spreadsheet.
Today this applies to anything in the web sphere, in the cloud apps, in the weather forecasts, the activity goals of your watch, the ranges in blood tests, the medical screening tests....
Today this applies also to ANY AUTOMATION PROJECT IN YOUR WORKPLACE. That's right. The so-called data scientists usually stop at the test-level statistics to demonstrate that their ideas will bring savings or increase the revenues of the company. You are now equipped to deal with their half-truths, finish the statistical analysis, and send them back to their desk if the patient doesn't get convincing answers ;)