Print Story Analysis of the cynical-bastard hypothesis
Diary
By tmoertel (Mon Aug 09, 2004 at 05:01:30 PM EST) (all tags)
Recall my previous diary entry, in which I presented a (contrived) probability puzzle about the readership of HuSi. In a poll, twenty-two votes were cast among five choices. Of the five, the fifth was chosen more frequently than the others – eight times. Can we conclude that this choice's popularity is unlikely to be caused by chance alone and therefore caused by something else (HuSi users being, at least in part, cynical bastards)?

Read on for the answer.



One way to answer our question is to turn it into a hypothesis and then test it using simple probability-based tests. So let's make our hypothesis, which we will denote Ha:
Ha: The votes cast for the fifth choice are not attributable to chance alone – something else must have caused them.
This is called the alternative hypothesis because it's the alternative to the status quo. In our case, as in most cases, the status quo is that nothing funny is going on and thus chance alone is the explanation for the observed behavior. The status quo is often called the null hypothesis and is denoted H0:
H0: The votes cast for the fifth choice are attributable to chance alone.
Notice that we have defined our hypotheses to be mutually exclusive. One – and only one – must be true, and the other must be false. This is a handy property because one of the pair may be easier to accept or reject than the other. Mutual exclusivity lets us pick it and work with it, even if we're more concerned about the other. In our case, for example, we're interested in Ha, but it's easier to work with H0 because much of probability theory describes events under the control of chance alone, and that's what H0 proposes. So let us focus on H0.

Now we can approach our problem in a more structured fashion:

  1. Assume that H0 is true.
  2. Ask, How unlikely is it to get eight out of twenty-two votes being cast for a single one of five choices?
  3. Determine this likelihood.
  4. Finally, if the likelihood is very small, conclude that our assumption about H0 is probably wrong and thus reject H0 (and correspondingly accept Ha); otherwise, accept H0.

Step 1 is easy. Say it with me: We hereby assume that the events in question were caused by chance alone.

Step 2 is likewise easy. Again, say it with me: How unlikely is it to get eight out of twenty-two votes being cast for a single one of five choices?

Step 3 is, well, complicated. Twenty-two votes are cast among five, equally likely choices. One choice gets a whopping eight votes. How unlikely is this event?

To make things easier, let's consider a simpler scenario: One vote is cast among five, equally likely choices. One particular choice, the last, gets the vote. How likely is this event?

Now this question is straightforward. There are five choices, equally likely. Therefore each choice gets the vote with a 1-in-5 likelihood. If our choice in question (the last) gets the vote, we'll call the vote a "success." If it doesn't get the vote, we'll call the vote a "failure." Using these terms, the probability of a success is 1/5, and the probability of failure is 4/5.

What we have just done is created a building block for the larger analysis. A building block of this kind is often called a trial, and such trials are summarized by the single parameter p, which is simply the probability of success. (It is also customary to let q denote the probability of failure, but because this value can always be determined from p using the equation q = 1 – p, it is typically ignored until used in calculations, where notational convenience becomes important.)

At this point, let's consider a single trial. (We'll work our way up to the full twenty-two trials later.) What is the probability of a single success from a single trial? Letting n denote the number of successes and P(X) denote the probability of event X, we're asking for P(n = 1). This probability is easy to determine: It's simply p. Likewise, the probability of zero successes from one trial (in other words, a single failure) is simply q. Summarizing:

For 1 trial (N = 1):

P(n = 0) = q
P(n = 1) = p

Now, let's complicate things a little by moving up to two trials. For this case, P(n = 0) is straightforward because there's only one way to get zero successes: We must fail both trials. Failing the first trial comes with probability q, and failing the second also comes with probability q. Failing both the first and second trials is therefore

< L'Amour. | BBC White season: 'Rivers of Blood' >
Analysis of the cynical-bastard hypothesis | 5 comments (5 topical, 0 hidden) | Trackback
You're a very silly man. by komet (6.00 / 1) #1 Mon Aug 09, 2004 at 06:53:52 PM EST
And I mean that in the nicest possible way.

--
<ni> komet: You are functionally illiterate as regards trashy erotica.
Post and pre analysis by TPD (3.00 / 0) #2 Mon Aug 09, 2004 at 08:40:56 PM EST
as is probably obvious by my postings in your previous diary, I was having slight difficulty getting my head round the the hypothesis ideas at at what point conclusions are drawn. At the risk of showing myself up more than already I'm posting this more to get things straight in my head rather than add much useful to the debate...

lets take as an example that the poll is the result of a random number generator which generates random numbers from 1 to 5...

given the results posted one could say with ~72% certainty that the results indicate a bias (given an original hypothesis of is this RNG fair), why only 72%? because had the spike been in one of the other categories we would have equally latched onto this, so 28 odd percent of the time a fair (1-5) random number generator would appear to be spiked with the sort of magnitude seen.... One could then conclude with something like a 28% accuracy that the RNG was biased towards 5.

however if someone had suggested before the poll was seen "could you run this RNG 22 times and confirm if it is biased towards 5", we could basically then confirm with a 94.4% certainty that the original assumption was correct.

This is not a troll, just me trying to get my barin back.

why sit, when you can sit and swivel with The Ab-SwivellerTM

this came across very much as an assertion by TPD (3.00 / 0) #3 Mon Aug 09, 2004 at 08:55:06 PM EST
however it's more a question, is what I am syaing correct?

why sit, when you can sit and swivel with The Ab-SwivellerTM
[ Parent ]
I not sure I follow, but I'll try. by tmoertel (3.00 / 0) #4 Tue Aug 10, 2004 at 06:01:31 AM EST
given the results posted one could say with ~72% certainty that the results indicate a bias (given an original hypothesis of is this RNG fair), why only 72%? because had the spike been in one of the other categories we would have equally latched onto this, so 28 odd percent of the time a fair (1-5) random number generator would appear to be spiked with the sort of magnitude seen....
Are your numbers empirical or derived via calculation?

I think you're saying that because we calculated the likelihood of getting eight or more votes for a single choice as 5.6 percent, you can compute the odds of getting eight or more votes in any of the five choices as 5 

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
Not really by TPD (3.00 / 0) #5 Tue Aug 10, 2004 at 10:21:04 AM EST
and it's no surprise you don't follow as I am really not certain I understand my point that well myself... and certainly not putting it across well...

the result of 28 (actually 27.6 I was rounding up) and 5.6 percent are generated by 2 different but similar programs one counting the chance of producing an output with at least any one "bucket" having at least 8 entries, the other looking at only a specific bucket ie "5"....

My point is given the output it is then very tempting to say that this has a 5.6% chance of being produced by chance, however this seems misleading if for all intensive purposes we would treat a spike in one of the other buckets as also also indicating bias (though we would have concluded in a different direction), so actually one must conclude that given we were looking across the board for inbalances, that actually there is a 27.6% chance that this is caused by chance.

I really suck at explaining, especially where I am almost certainly talking crap about something which I know pathetically little about... feel free to ignore it's just something that has been eating away at my sanity since your original diary, (as to whether there is a 5.6 or 27.6 chance that the cynical bastard vote is due to chance)

why sit, when you can sit and swivel with The Ab-SwivellerTM

[ Parent ]
Analysis of the cynical-bastard hypothesis | 5 comments (5 topical, 0 hidden) | Trackback