Print Story The Return of Jake
Diary
By tmoertel (Fri Jul 30, 2004 at 09:19:12 AM EST) (all tags)
Recall the story from my previous diary, in which Jake is fired from his construction job for testing positive for heroin use. As it turns out, Jake's family hired an attorney and sued the construction company. Their case was persuasive, and in an out-of-court settlement, the company agreed to re-hire Jake and pay him the equivalent of two years' salary as compensation for wrongful termination and harm to reputation.

What was the argument that Jake's attorney used to demonstrate that Jake was falsely accused of heroin use and wrongfully terminated?



Soon after taking the case, Jake's attorney Sheila (who was his cousin) verified the facts. She determined that the company wasn't overstating the reliability of the drug test: 98 percent of the time, it was correct.

She didn't think a jury would be willing to fault the company for taking the 98-percent side of the bet, especially given the stakes. Even if she argued that the company should have re-tested Jake to give him the benefit of the 2-percent doubt, she knew the company's lawyers could argue convincingly against her. They would no doubt claim that drug users have a variety of techniques to avoid detection, and, now on the alert, Jake could use them to evade follow-up tests.

Nevertheless, she met with a mathematics professor at a local college to review her case. The professor immediately found a fundamental flaw in the company's logic – a flaw that tipped the scales dramatically in Jake's favor.

The professor said that the 98-percent reliability of the positive test result didn't mean that Jake was 98-percent likely to be a heroin user. Rather, it provided incremental evidence to support the claim that Jake used heroin, but what the company needed to reasonably terminate Jake was overwhelming total evidence. He said that in order to determine the total evidence against Jake, the company must take into account prior knowledge about Jake's likelihood of using heroin.

Shelia retrieved the company's yellow fact sheet from her briefcase and asked the professor if it would help.

"Yes," he said, looking it over, "I would say this is exactly what you want." He chuckled, then added, "It says in the opening sentence that 'only 2 out of 1,000 construction workers use heroin.' If that's right, the company made a rather significant error in its interpretation of the statistical evidence."

He said that the company's own statistics show that out of 1,000 construction workers, a whopping 998 of them will not be heroin users. Nevertheless, if you test those 998 clean workers for heroin, 2 percent of them – 19.96 on average – will test positive anyway. Of the 2 actual heroin users, 1.96 (on average) will test positive, too. In total, then, 21.92 people will test positive, but only 1.96 will truly be heroin users. Thus, he concluded, if you divide 1.96 by 21.92, you'll arrive at the proportion of workers who tested positive that actually are heroin users. That proportion turns out to be 8.9 percent.

Sheila couldn't believe it: "Are you saying that Jake's chance of being innocent wasn't 2 percent but in reality greater than 90 percent?!"

"That's exactly what I'm saying."

In the professor's sworn written statement, he put it more formally:

What we are interested in is the probability that a worker actually uses heroin, given than he has tested positive for heroin use. If we let H denote the event that a worker uses heroin, and T denote the event that a worker tests positive for heroin, then we wish to find the probability of H given T, which is written as follows:
P(H|T)
What we already know is the following: First, the probability of a construction worker using heroin is 0.2 percent. Second, the probability of the test being correct is 98 percent. Thus the probability of testing positive for heroin, given that heroin is in fact used, is 98 percent. Similarly, we can compute the odds of not testing positive (which we will denote ~T) given that heroin is not used (~H). From these, we can compute the probabilities of all the other possibilities. More formally:
P(H) = 0.002
P(~H) = 0.998
P(T|H) = P(~T|~H) = 0.98
P(~T|H) = P(T|~H) = 0.02
Now, we can use Reverend Thomas Bayes's famous rule for posterior probabilities to compute our desired value:
P(H|T) = P(T|H) P(H) / [ P(T|H) P(H) + P(T|~H) P(~H) ]
Substituting our probabilities from above yields the answer we seek:
P(H|T) = 0.98 * 0.002 / [ 0.98 * 0.002 + 0.02 * 0.998 ] = 0.089
Thus, even though Jake tested positive, it was only 8.9 percent likely that he was a heroin user. The overwhelming preponderance of the evidence – over 91 percent – supported the opposite conclusion, that Jake did not use heroin.
With the professor's written testimony, Shelia was able to convince the company of its mistake (and the difficulty of defending the mistake in court). The settlement was reached soon thereafter.

Thus ends our two-part series on Jake, part of the Mathematics Can Be Fun And Even Dramatic program. The goal of the program is to replace television's popular criminal justice–courtroom drama with a new kind of show: the criminal justice–statistical-courtroom drama. Please join us in our noble quest! A donation in the amount of 1/sqrt(–π) USD would be most appreciated.

< I broke up with Jen this morning | BBC White season: 'Rivers of Blood' >
The Return of Jake | 29 comments (29 topical, 0 hidden) | Trackback
Yay! by LoppEar (3.00 / 0) #1 Fri Jul 30, 2004 at 09:25:10 AM EST
Jake is the man! Da Man!


Jake by duxup (6.00 / 2) #2 Fri Jul 30, 2004 at 09:29:50 AM EST
The following week Jake, wasted off his ass dropped from a 12 story building, landed on, and killed a coworker.

This concludes our Mathematics Can Kill Dramatically Program.
____

Oh, yeah: Take the poll. Thx. [nt] by tmoertel (3.00 / 0) #3 Fri Jul 30, 2004 at 09:30:29 AM EST

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

Are you really by zantispam (6.00 / 1) #4 Fri Jul 30, 2004 at 09:30:42 AM EST
really going to start posting more of these?  Please?

I might actually learn math if you do...

Yeah, you better run before LiaD stabs you with his shiny metal cock! -- theboz

Hey, if we can get actors, let's put 'em on film! by tmoertel (6.00 / 5) #7 Fri Jul 30, 2004 at 10:13:41 AM EST
I can see the trailer now:
In a time when men worked hard . . .
Fade into shot of large construction site, sweating men positioning rebar, grunting mightily.

In a town where men played hard . . .

Cut to night-shot of Ford F-150 racing down main street of small town, drunken construction workers shouting from its bed, throwing beer bottles into street.

The company fired the wrong man . . .

Cut: Angry Human Resource Lady says, "You failed your drug test, you damn druggie! Your fired!"

That was their first mistake . . .

Cut to: Concerned Corporate Drone: "Something's wrong here. Something's wrong, but I can't put my finger on it. I'm scared, man. Really scared."

Their second mistake was worse . . .

Cut to: Court room, professor angrily shouting from the stand, "But you didn't check your math, did you? You didn't check your math! You damn fools!"

And now it's time to pay. . .

Loud, explosive fade to white to the thunderous sound of a gavel striking, then slow fade to black
HuSi's jacob is Jake The Construction Worker.

in

The Statistical Rage of Maths: The Jake Carbinsko Story

On screen text: "Coming soon to a theater near you (with 0.023 probability)".

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
If it's on film we can call it MathNet by Greener (3.00 / 0) #10 Fri Jul 30, 2004 at 10:58:19 AM EST
Wait, wasn't that on Square One Television.

Videos

[ Parent ]
+1 Front Page by zantispam (3.00 / 0) #11 Fri Jul 30, 2004 at 11:13:51 AM EST
discussion of math on teh intarweb

Yeah, you better run before LiaD stabs you with his shiny metal cock! -- theboz
[ Parent ]
Simplistic by ucblockhead (5.00 / 2) #5 Fri Jul 30, 2004 at 09:56:11 AM EST
This is a good illustrative problem on why "false positives" are dangerous, but it is a bit simplistic in that it assumes things on a strictly mathematical basis. For one, it assumes that the 2% false positive rate is completely and utterly random. This is dangerous in that there may be some other factor that causes false positives. Suppose you test your workers and discover that 50% of them test positive for heroin. Obviously that's not an issue of "false positives", right?

Well, wrong, because some digging might find that the poppyseed bagel co delivered free bagels to the lunchroom the day before the test. The 2% "false positive" rate determined statistically might have to do with the average poppyseed consumption of the general populous.

The reason it is dangerous to think strictly in terms of mathematics is because if you do, you'd be tempted to say that the chance of two false positives in a row are miniscule, after all, 1/50 * 1/50 is 1/2500, right? Well, wrong, if that 1/50 has a cause that doesn't change between tests.

This is why the best way to combat false positives is to use a different test the second time around. The common way is to have a cheap test with a high false positive rate, and follow up with an expensive test with a low false positive rate. Because the tests are different, the likelihood of running into an alternate cause is lessened.
---
[ucblockhead is] useless and subhuman

Fear not strictness but sloppiness! by tmoertel (3.00 / 0) #23 Sat Jul 31, 2004 at 05:10:59 AM EST
Good grief, did you read the dialog? How cheesy could it be? If you're going to poke holes in my stories, that should have been your target – or maybe the character development – but the math? Come on, man, focus! But I digress.

Now, let's get to the thrust of your argument, and I understand where you're thrusting, but your target is off the mark. Nothing is wrong with a strictly mathematical approach – unless you take strictly to mean sloppy, which is what your examples seem to suggest. Let me show you what I mean:

The reason it is dangerous to think strictly in terms of mathematics is because if you do, you'd be tempted to say that the chance of two false positives in a row are miniscule, after all, 1/50 * 1/50 is 1/2500, right?
But that's not "thinking strictly"; that's thinking sloppily. Thinking strictly would require us to acknowledge that the multiplication performed above is legitimate only if the events described by the individual probabilities are independent. That's fundamental probability theory. If we perform an analysis that ignores the very theory it's built upon, that's not strict in any sense of the word.
For one [example of how the analysis is simplistic], it assumes that the 2% false positive rate is completely and utterly random.
(Here I interpret "utterly random" to mean "free of influence from external factors.") The analysis does not take this position as a blind assumption. Rather, this is the only position supported by the information we are given in the story. No information is provided to suggest the influence of external factors. And if we had hypothesized that such factors did exist, we couldn't have supported the hypothesis and would have been forced to retreat from it. (Or can you think of a way to incorporate your poppy seed–bagel hypothesis into the model?)

It seems that what you are arguing against is really sloppiness.

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
Well... by ucblockhead (6.00 / 1) #24 Sat Jul 31, 2004 at 05:29:34 AM EST
I'm not arguing against your example per se. I'm just commenting that the "obvious" solution, to retest to see if you get the same result, is not a good one. I think that is what Patriculus was getting at with his link.
---
[ucblockhead is] useless and subhuman
[ Parent ]
Okay by tmoertel (3.00 / 0) #25 Sat Jul 31, 2004 at 05:54:10 AM EST
But I must still fault you for not having the decency to make fun of my writing's dry, soulless dialog.

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
This is really bizarre... by JWhiton (3.00 / 0) #6 Fri Jul 30, 2004 at 10:11:08 AM EST
...when you read it and your name is Jake.



The important question is... by the (3.00 / 0) #8 Fri Jul 30, 2004 at 10:14:12 AM EST
...how can I arrange to fail a drug test without actually abusing the drug so I can sue the testing company?

--
The Definite Article
Normal way by ucblockhead (3.00 / 0) #9 Fri Jul 30, 2004 at 10:47:03 AM EST
Consume products that are biologically similar yet legal.
---
[ucblockhead is] useless and subhuman
[ Parent ]
easy by 606 (3.00 / 0) #26 Sat Jul 31, 2004 at 06:27:24 AM EST
As ucblockhead noted, eat products that are chemically similar. The most common one is to eat poppy-seed products to test positive for heroin. I saw this tried out on the TV show Mythbusters and they showed that eating six poppy-seed bagels would make you test positive for heroin for about a day and a half.

-----
imagine dancing banana here
[ Parent ]
Give John Allen Paulos a TV show! by sien (6.00 / 1) #12 Fri Jul 30, 2004 at 11:29:32 AM EST
John Allen Paulos has written books that elucidate similar problems. They are well worth the read.

Nobody knows anything - William Goldman.
If this is section 2 by Breaker (6.00 / 2) #13 Fri Jul 30, 2004 at 01:02:52 PM EST
How about writing section 3: How to move to a country where companies aren't given the rights to dictate what you did last weekend.


better yet by alprazolam (6.00 / 3) #15 Fri Jul 30, 2004 at 01:40:02 PM EST
make this country that way.

[ Parent ]
So, uh.. by Driusan (5.50 / 2) #16 Fri Jul 30, 2004 at 03:28:36 PM EST
Is Jake a heroin user?

--
Vive le Montréal libre.
So how does the company know... by CrocoStimpy (3.00 / 0) #17 Fri Jul 30, 2004 at 05:38:48 PM EST
... the true proportion of construction workers who use heroin?  Was this determined by, oh say, a drug test?

an interesting story... by Patriclus (6.00 / 2) #18 Fri Jul 30, 2004 at 06:43:11 PM EST
... and, as so many others have said, keep 'em coming

Still: an objection. 

Unfortunately, it's quite suddenly gotten late and my mind tired, here, so all I can say is: 

The plaintiff seems to be confusing case probability with class probability, to follow the terms after L. Mises. See HERE and forward.

Etc.

Aside by ucblockhead (3.00 / 0) #21 Sat Jul 31, 2004 at 03:17:50 AM EST
I found this quote from the link (thanks for that, BTW) amusing given the definition of "social engineering" I grew up with:

It is customary nowadays to speak of "social engineering." Like planning, this term is a synonym for dictatorship and totalitarian tyranny.


---
[ucblockhead is] useless and subhuman

[ Parent ]
Please explain. by tmoertel (3.00 / 0) #22 Sat Jul 31, 2004 at 04:05:14 AM EST
After reading the page you linked to, I cannot see how the plaintiff has made this error. Can you show me the error?

Thx.

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
case probability by Patriclus (3.00 / 0) #28 Sat Jul 31, 2004 at 07:34:02 AM EST
I suppose what I saw may have been pretty well summed up in an early comment to your diary, to the effect of, So: Was the plaintiff a heroin user?

The probability--.02%--that establishes the class of construction workers vis-a-vis heroin use explains away factors in causal agency that are about pure chance--or 'gambling' in Mises' terminology. 

But the event--of the plaintiff deciding (somehow, anyway) whether or not to take heroin--was a unique event.  It must be understood by the methods of historical understanding (after Mises, again).  The plaintiff may, in truth, be a perfect representative of the class in which he was placed, but, still, he is an acting human being who, thus, has his own preferences of ends and means which statistics can tell us nothing about.

Thus, my feeble attempt at an explanation.

The other thing I noticed but probably should not insist on communicating, regarding your story, is the fact that the conditions established in the agreement of employment--whether in contract or otherwise--was pretty well ignored.

TTFN.

[ Parent ]
Hmm. by tmoertel (3.00 / 0) #29 Sat Jul 31, 2004 at 09:38:15 AM EST
I still don't see the problem. I don't see where class and case are conflated. I don't see the gambler's fallacy making an appearance.

Please help me out. Can you be specific? In other words, can you show me where my calculations go astray or rely upon unfounded assumptions?

Getting to your comments:

But the event--of the plaintiff deciding (somehow, anyway) whether or not to take heroin--was a unique event. It must be understood by the methods of historical understanding (after Mises, again). The plaintiff may, in truth, be a perfect representative of the class in which he was placed, but, still, he is an acting human being who, thus, has his own preferences of ends and means which statistics can tell us nothing about.

Absolutely.

We don't know whether Jake actually uses heroin. If we did, we would know P(H) cold, and it would be either 0 or 100 percent. However, we do not know anything about the reality of Jake's drug situation beyond the information related in the story. That we have made an inference about the likelihood of Jake using drugs does not influence reality but merely describes the limited knowledge of reality that we have.

Some people are tempted to read magical powers into statistical inferences such that the inferences are tied to reality itself. That the reality may depend upon specifics we don't know about, and that these specifics have not been factored into our inferences suggests to these people that the inferences cannot be correct. But that's only because they made the mistake of thinking that "correct" means "matches reality." It's the gambler's fallacy in reverse.

The truth is that correctness in this context means making the best possible statements about reality based on our limited knowledge. That is what the professor's calculations attempt to do – make the best of the limited information.

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
give me a sleep break by Patriclus (3.00 / 0) #30 Sat Jul 31, 2004 at 09:32:01 PM EST
... and I'll aim and get back to you.  Truth be told, I hardly expected I'd sustain your interest enough to make it obligatory for me to wake up and learn what I meant by what I said --as, oftentimes, I make comments as something of a token of a profession of belief in the importance of being social.

TTFN. Thanks for the challenge.

[ Parent ]
Doh! by R343L (6.00 / 1) #19 Fri Jul 30, 2004 at 07:25:28 PM EST
I forgot that you were the super math guy...so I gave you the general political/social answer, not a more mathematical one. And I even know those kinds of things (math major), but most people find the math reasons tedious, preferring the emotional, touchy-feely, reasons. ;)

Rachael

"There will be time, there will be time / To prepare a face to meet the faces that you meet." -- Eliot

Write in by gpig (5.00 / 1) #20 Fri Jul 30, 2004 at 09:31:41 PM EST
Founded the 'Mathematics Education for Personnel Workers' campaign.

Then he lived the rest of his life according to what the dice told him.
---
(,   ,') -- eep

There already is/was a math TV cop show by 606 (3.00 / 0) #27 Sat Jul 31, 2004 at 07:03:26 AM EST
There already was a TV show about cops using math to bust people. It was a serialized segment called MATHNET on a show called Square One produced by PBS.

MATHNET was based on DRAGNET and had its two hero cops, George Frankly and Kate Monday (later replaced by Pat Tuesday) investigating crimes like the theft of the Maltese Pigeon or the Trojan Hamburger, using mathematics and logic to "cogitate and solve" the crimes.

Here, watch an episode. Crappy quality and not too much math in this one, but it's pretty clever.

(In researching this post I ran across that MATHNET page, and the episode guide really brought me back and connected up many strange phrases that had been floating in my head since my childhood. Funny how that goes.)

-----
imagine dancing banana here

The Return of Jake | 29 comments (29 topical, 0 hidden) | Trackback