## Introduction

What is the interpretation of a P-value of a hypothesis test in inferential statistics? How is it commonly misunderstood? amzn.to/3rjDOoA (Probability and Statistics with Applications: A Problem Solving Text, by Asimow and Maxwell)

#PValue #Statistics #DataScience

Links and resources

===============================

🔴 Subscribe to Bill Kinney Math: www.youtube.com/user/billkinneymath

🔴 Subscribe to my Math Blog, Infinity is Really Big: infinityisreallybig.com/

🔴 Follow me on Twitter: twitter.com/billkinneymath

🔴 Follow me on Instagram: www.instagram.com/billkinneymath/

🔴 You can support me by buying "Infinite Powers, How Calculus Reveals the Secrets of the Universe", by Steven Strogatz, or anything else you want to buy, starting from this link: amzn.to/3eXEmuA.

🔴 Check out my artist son Tyler Kinney's website: www.tylertkinney.co/

AMAZON ASSOCIATE

As an Amazon Associate I earn from qualifying purchases.

## Content

P-Value is always the probability of observing a test statistic as extreme or more extreme than what you actually observe when the null is true, that's, a mouthful, but it's worth writing down probability of observing a test statistic.

And you should know this for the exam, maybe even a multiple choice question as extreme or more extreme as what you actually observed as what you actually observed under the assumption that the null hypothesis is true.

That's, always what the p-value is in any situation.

Of course, that means you got a no test statistic.

And then the logic of thinking about making a decision with p-values is this.

This is really key as well.

Maybe I'll, ask you a multiple choice question about what I'm about to say, the logic is this.

If the p-value is small really small, how small is really small it's.

It is a matter of opinion.

But again, our standards of five percent that most people sort of default to is that really the best thing to default to it, not necessarily it's, just sort of habit tradition.

If the p-value is really small, then you then you observe have observed something rare when the null is true.

Therefore we think the null is false don't ever ever ever ever ever ever ever ever ever think that the p-value is the probability that the null is true.

That's sort of the common misperception.

Oh, the p-value is the probability of the null is true.

It's, small, therefore we think the null is false.

No that's, not what it is nobody.

But nobody can figure out the probability that the null is true.

Nobody can do such a thing it's impossible.

Not even a well-defined question.

Probably, I mean, when you think about it practically, I mean, if you're thinking about two-tailed tests, the probability that the null is true, even though it can't really be found in in any ordinary sense, you might say zero, if it's a two-tailed test right the probability that mu equals 20 exactly there's, pretty much no chance, but that's, not the philosophy of hypothesis testing.

The philosophy is you give the null benefit of the doubt until proven otherwise so to speak, these are important issues and it's important to you know, help your future work colleagues to understand that.

And they do have practical ramifications in terms of misunderstanding statistics and stuff people use up p-values all the time.

But they really don't understand them it's, the probability of observing a test statistic like a z-statistic or a t as extreme or more extreme than what you actually observed greater than or equal to or less than or equal to under the assumption that the null is true.

So when it's small, you observe something rare, if the null is true, therefore, you think the null's false.

How is this different alpha is the probability of a type, one error the probability of incorrectly incorrectly rejecting the null.

If you assume the null is true it's, when the p-value is less than alpha when you get a p-value, and it happens to be less than alpha that you say, okay, I'm going to reject the null.

And therefore, my probability of the type 1 error is alpha I'll say that again, well, let's just go ahead and compute it for an example.

Suppose let's, go ahead.

And suppose we get an x bar that I know ahead of time is going to be in the rejection region in terms of x bar.

Let's, pretend we get x bar equal to 18? Yeah, 18.16.

Suppose, the observed value of x bar is 18.1.

The p-value of the test then is the probability of getting x bar less than or equal to 18.1 given that the null is true.

I mean, usually when we compute a p-value, we don't write it in terms of x-bar.

We usually pretty much just compute z first, assuming the null is true so subtract 20.

So we have negative 1.9, divided by 0.8.

The observed value of the z statistic is negative 2.375.

So usually I don't write that first line that I just wrote right here, but it's, not a problem to write that.

And it is a less than or equal to here because it's a left-tailed test less than or equal to is what it means to say as extreme or more extreme it's, always with regard to the direction of the alternative hypothesis with the left-tailed test.

We want less than or equal to.

But by what I've just computed here, that's the same as the probability that z is less than or equal to negative 2.375.

And by what I've done already that's going to be a little less than 0.01 negative, 2.375 I'm going to get something slightly less than 0.01 0.009.

So the p-value is .009 less than alpha slightly less than alpha to decide to reject the null.

You don't have to think about alpha.

You could just say, hey, I got a small p value.

I've observed something rarer when the null is true, therefore I reject the null.

You don't have to think about alpha, however, in the classical approach, you do decide alpha ahead of time.

You do decide the rejection region you could reject the null just looking at the z statistic and seeing oh it's in the rejection region.

And you don't have to figure out the p-value, the more new approach is to figure out the p-value to specify exactly what the probability of observing.

The test statistic is if the null is true as extreme or more extreme.

If you want to relate it to the classical approach, then you say, oh it's less than alpha.

So I reject this is equivalent to z being in the reduction region because it's less than alpha.

You are rejecting the null, therefore your type one probability of a type.

One error is alpha, though we get into subtleties there, too because technically speaking, you've already done your random sampling.

So again, probabilities are really best interpreted before the fact, if you decide what alpha is going to be .01 it's best thought of as a probability before you actually do your sampling, you've set things up.

You've got a sample size of 25 and a standard deviation of four with this hypothesis test I'm about to do the test.

What are the chances that I'm going to make a type 1 error that's? What the best interpretation of the .01 is, I happen to get a p-value smaller than that.

So I do reject saying that my probability of a type 1 error now is .01 isn't after the fact probability is not quite technically the best interpretation of probability.

But most people don't worry about it, but don't ever say, the prob p value is the probability that the null's true it's, not there's, no way.

Anybody can find that.

Maybe, instead of emphasizing that the nulls mu equals 20 exactly.

Maybe you started emphasizing that the nulls mu is 20 with maybe a little bit of cushion of margin of error or something, but you're, not going to get into that either.

We want to keep things simple.

I mention these things to you so that you just realize maybe you've had misperceptions about things and plenty of other people do too sometimes the technical technically bad way of thinking about it doesn't have any effects like, oh to think of this as a probability of a type type one error.

Now after the fact, it's not a big deal really, but it is an after the fact probability, which is technically not correct it's kind of like with confidence levels with confidence.

Intervals as well.

If this had been a two-tailed test mu, not equal to 20 in the alternative hypothesis, I'm, telling you you should be able to handle it as far as the p-value goes.

You just double it.

Just double the one-tailed p-value as far as the rejection region's, actually a bit trickier.

Right? I have to find two tails for a two-tailed test where the area of both of them add to 0.01.

So the area of both of them would be half of that .005.

So p-value approach just double the one-tailed p-value for these symmetric distributions, at least with the rejection region approach it's a bit trickier.

You gotta split your rejection region into two pieces.