Pvalue

class: left, top, inverse
background-image: url(https://images.unsplash.com/photo-1498604636225-6b87a314baa0?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=1569&q=80)
background-size: cover

# .right.Large[Statistical Significance and p-values]

#### .left[Gina Reynolds Photo Credit: Toa Heftiba

---
# Statistical independence

> In probability theory, two events are independent, statistically independent, or stochastically independent if the occurrence of one does not affect the probability of occurrence of the other.

> Similarly, two random variables are independent if the realization of one does not affect the probability distribution of the other.

---
## "A statistically significant relationship"

- When you have *evidence* of statistical *dependence*.

- When the probability of observing of observing such a strong relationship --- or stronger --- under *statistical independence* (the null) is quite small.

- How small is small enough?  It depends what `$\alpha$` is... .01, .05, .1?

https://www.youtube.com/watch?v=gSyGVDMcg-U

---
## Hypotheses

Hypothesis testing:

- A null hypothsis: `$Ho$` (Often statistical independence)
- Alternate hypothesis: `$Ha$` (Statistical dependence)

---
## Rejecting the null hypothesis

- Comparison will be made with expectation about null (often no relationship).  Such a strong relationship is unlikely to be observed by chance in a sample, if there is *not* dependence.

---

- If data deviates from expecations given null (if observed relationships very surprising given null)

- we reject the null (there is no relationship - statistical independence)

- and accept the alternative (there is a relationship - statistical dependence)

---
## P-Value

"The probability that, given the null, a relationship as strong or stronger than that observed, would be observed."

http://fivethirtyeight.com/features/not-even-scientists-can-easily-explain-p-values/

---
# Inception: Lady tasting tea insight

- "P-value"
- "null hypothesis"
- "statistical significance" (at significance level, alpha level)
- Test: "Fisher's exact"

---
# Players

Phycologist (algae studies) Muriel Bristol:  Prefered milk poured first.

Ronald Fisher - mathematically astute, but problematic

![via wikipedia](https://upload.wikimedia.org/wikipedia/commons/thumb/a/aa/Youngronaldfisher2.JPG/220px-Youngronaldfisher2.JPG)

---
# The experiment

- 8 cups
- randomly fill 4 with milk first, 4 with tea first
- ask Bristol to identify which four milk and which for tea first
- assess evidence that Bristol can tell what is poured first

---

How many does she need to get correct to provide good evidence that she can really tell the difference?

---
# Null hypothesis and p-value

Ho - There is no relationship between my variables.  Statistical independence.

p-value - "the probability that you would observe a relationship as strong as or stronger than the observed relationship, if no relationship actually exists (i.e. if the null is true)."

---

Fisher on the null hypothesis - "never proved or established, but is possibly disproved, in the course of experimentation."  *The Design of Experiments*

---
# Rejecting the null

If we reject the null hypothesis, `$H_0$`, then we accept the alternative hypothesis `$H_A$`, that there is a relationship between two variables.

---
# Lab

- What might we have observed under the null
- Compare to actual result 
- Do we reject the null (establishing statistical significance)?

---
# Problem solving techniques

- Analytic: Hypergeometric distribution
- Computational: Simulation (lab)

---
class: inverse, center, middle

## The set up

---

```r
*set.seed(90210)
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
set.seed(90210)  
# truth
*sample(c("tea", "tea", "tea", "tea",
*        "milk", "milk", "milk", "milk"))
```
]]
.column[.content[

```
[1] "tea"  "milk" "tea"  "milk" "tea"  "milk" "tea"  "milk"
```
]]
---
class: split-40
count: false

```r
set.seed(90210)  
# truth
sample(c("tea", "tea", "tea", "tea",  
         "milk", "milk", "milk", "milk"))

# lady's responses
*sample(c("tea", "tea", "tea", "tea",
*        "milk", "milk", "milk", "milk"))
```
]]
.column[.content[

```
[1] "tea"  "milk" "tea"  "milk" "tea"  "milk" "tea"  "milk"
```

```
[1] "milk" "milk" "tea"  "tea"  "tea"  "milk" "milk" "tea" 
```
]]

---
class: inverse, center, middle

# Single realization - how many correct?

---

```r
*set.seed(90210)
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
set.seed(90210)  
# truth
*sample(c("tea", "tea", "tea", "tea",
*        "milk", "milk", "milk", "milk"))
```
]]
.column[.content[

```
[1] "tea"  "milk" "tea"  "milk" "tea"  "milk" "tea"  "milk"
```
]]
---
class: split-40
count: false

```r
set.seed(90210)  
# truth
sample(c("tea", "tea", "tea", "tea",  
         "milk", "milk", "milk", "milk")) ->  
* truth
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
set.seed(90210)  
# truth
sample(c("tea", "tea", "tea", "tea",  
         "milk", "milk", "milk", "milk")) ->  
  truth

# lady's responses
*sample(c("tea", "tea", "tea", "tea",
*        "milk", "milk", "milk", "milk"))
```
]]
.column[.content[

```
[1] "milk" "milk" "tea"  "tea"  "tea"  "milk" "milk" "tea" 
```
]]
---
class: split-40
count: false

```r
set.seed(90210)  
# truth
sample(c("tea", "tea", "tea", "tea",  
         "milk", "milk", "milk", "milk")) ->  
  truth

# lady's responses
sample(c("tea", "tea", "tea", "tea",  
         "milk", "milk", "milk", "milk")) ->  
* guess
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
set.seed(90210)  
# truth
sample(c("tea", "tea", "tea", "tea",  
         "milk", "milk", "milk", "milk")) ->  
  truth

# lady's responses
sample(c("tea", "tea", "tea", "tea",  
         "milk", "milk", "milk", "milk")) ->  
  guess

*sum(truth == guess)
```
]]
.column[.content[

```
[1] 4
```
]]

---
class: inverse, center, middle

# intro to the 'for loop' (DRY)

# doing a task several times

# 'for i in 1:5 ...'

# do what's in the 'curly braces' repeatedly

# syntax...

---

```r
*set.seed(90210)
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
set.seed(90210)  
*for (i in 1:10){

* print(i)

* }
```
]]
.column[.content[

```
[1] 1
[1] 2
[1] 3
[1] 4
[1] 5
[1] 6
[1] 7
[1] 8
[1] 9
[1] 10
```
]]

---
class: inverse, center, middle

# Many realizations under guessing

---

```r
*set.seed(90210)
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
set.seed(90210) 
*for (i in 1:10){
# truth
*truth <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

# lady's responses
*guess <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

*print(paste("trial", i))
*print(truth == guess)
*print(sum(truth == guess))
*} 
```
]]
.column[.content[

```
[1] "trial 1"
[1] FALSE  TRUE  TRUE FALSE  TRUE  TRUE FALSE FALSE
[1] 4
[1] "trial 2"
[1] FALSE  TRUE  TRUE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1] "trial 3"
[1]  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  TRUE
[1] 4
[1] "trial 4"
[1]  TRUE FALSE  TRUE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
[1] "trial 5"
[1]  TRUE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE
[1] 2
[1] "trial 6"
[1]  TRUE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE
[1] 2
[1] "trial 7"
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
[1] "trial 8"
[1]  TRUE  TRUE FALSE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1] "trial 9"
[1]  TRUE FALSE  TRUE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 6
[1] "trial 10"
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
```
]]
---
class: split-40
count: false

```r
set.seed(90210) 
for (i in 1:10){ 
# truth
truth <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

# lady's responses
guess <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

print(paste("trial", i))  
print(truth == guess)  
print(sum(truth == guess))  
}  
```
]]
.column[.content[

---
class: inverse, center, middle

# Law of large numbers -> approximates hypergeometric expectations (analytic solution)

---

```r
*for (i in 1:100){
# truth
*truth <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

# lady's responses
*guess <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

*print(truth == guess)
*print(sum(truth == guess))
*} 
```
]]
.column[.content[

```
[1]  TRUE FALSE  TRUE FALSE  TRUE  TRUE FALSE FALSE
[1] 4
[1] FALSE  TRUE  TRUE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE FALSE  TRUE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE FALSE FALSE  TRUE  TRUE
[1] 4
[1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE FALSE FALSE
[1] 4
[1] FALSE  TRUE FALSE  TRUE  TRUE FALSE  TRUE FALSE
[1] 4
[1] FALSE FALSE  TRUE FALSE  TRUE FALSE  TRUE  TRUE
[1] 4
[1] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE  TRUE
[1] 2
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 6
[1] FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 4
[1] FALSE  TRUE  TRUE  TRUE FALSE FALSE  TRUE FALSE
[1] 4
[1]  TRUE FALSE  TRUE FALSE FALSE FALSE  TRUE  TRUE
[1] 4
[1] FALSE FALSE FALSE  TRUE FALSE  TRUE  TRUE  TRUE
[1] 4
[1] FALSE FALSE  TRUE FALSE FALSE  TRUE FALSE FALSE
[1] 2
[1]  TRUE FALSE FALSE  TRUE  TRUE FALSE FALSE  TRUE
[1] 4
[1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE FALSE FALSE  TRUE FALSE FALSE FALSE  TRUE
[1] 2
[1] FALSE  TRUE FALSE  TRUE  TRUE FALSE  TRUE FALSE
[1] 4
[1] FALSE  TRUE FALSE FALSE  TRUE  TRUE FALSE  TRUE
[1] 4
[1] FALSE  TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1]  TRUE  TRUE FALSE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1]  TRUE  TRUE FALSE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1]  TRUE  TRUE  TRUE  TRUE FALSE FALSE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE FALSE FALSE FALSE  TRUE  TRUE
[1] 4
[1] FALSE FALSE  TRUE FALSE FALSE FALSE FALSE  TRUE
[1] 2
[1]  TRUE  TRUE FALSE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 6
[1]  TRUE FALSE  TRUE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE FALSE FALSE FALSE  TRUE FALSE FALSE
[1] 2
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1]  TRUE  TRUE FALSE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1] FALSE  TRUE  TRUE FALSE  TRUE  TRUE FALSE FALSE
[1] 4
[1]  TRUE  TRUE FALSE  TRUE FALSE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
[1]  TRUE  TRUE  TRUE FALSE FALSE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1] FALSE  TRUE FALSE FALSE FALSE  TRUE  TRUE  TRUE
[1] 4
[1] FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1]  TRUE FALSE  TRUE FALSE  TRUE FALSE  TRUE FALSE
[1] 4
[1] FALSE  TRUE  TRUE FALSE FALSE  TRUE FALSE  TRUE
[1] 4
[1] FALSE FALSE  TRUE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
[1] FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1] FALSE  TRUE FALSE FALSE FALSE  TRUE  TRUE  TRUE
[1] 4
[1] FALSE FALSE FALSE  TRUE FALSE  TRUE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE FALSE FALSE FALSE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE
[1] 2
[1] FALSE  TRUE FALSE FALSE  TRUE FALSE  TRUE  TRUE
[1] 4
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
[1]  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE FALSE  TRUE FALSE  TRUE FALSE FALSE FALSE
[1] 2
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1] FALSE  TRUE FALSE FALSE  TRUE FALSE FALSE FALSE
[1] 2
[1] FALSE FALSE  TRUE  TRUE FALSE FALSE  TRUE  TRUE
[1] 4
[1] FALSE  TRUE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE FALSE  TRUE  TRUE FALSE FALSE  TRUE
[1] 4
[1]  TRUE FALSE FALSE  TRUE  TRUE  TRUE FALSE FALSE
[1] 4
[1] FALSE FALSE FALSE  TRUE FALSE FALSE FALSE  TRUE
[1] 2
[1] FALSE FALSE  TRUE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
[1]  TRUE  TRUE FALSE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
[1]  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE FALSE  TRUE FALSE FALSE  TRUE  TRUE  TRUE
[1] 4
[1] FALSE FALSE  TRUE FALSE  TRUE FALSE FALSE FALSE
[1] 2
[1]  TRUE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE
[1] 4
[1]  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE
[1] 2
[1]  TRUE  TRUE  TRUE FALSE  TRUE  TRUE FALSE  TRUE
[1] 6
[1] FALSE  TRUE FALSE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[1] 8
[1] FALSE  TRUE FALSE  TRUE  TRUE FALSE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE FALSE  TRUE FALSE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE FALSE  TRUE FALSE FALSE FALSE FALSE
[1] 2
[1]  TRUE  TRUE  TRUE FALSE FALSE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 4
[1] FALSE  TRUE FALSE FALSE FALSE FALSE FALSE  TRUE
[1] 2
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE  TRUE FALSE
[1] 4
[1] FALSE  TRUE  TRUE FALSE  TRUE FALSE FALSE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
[1] FALSE  TRUE FALSE FALSE  TRUE FALSE  TRUE  TRUE
[1] 4
[1] FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 4
[1] FALSE  TRUE FALSE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1] FALSE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE
[1] 2
[1] FALSE  TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1] FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 4
[1] FALSE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE
[1] 2
[1] FALSE FALSE  TRUE  TRUE  TRUE FALSE  TRUE FALSE
[1] 4
[1]  TRUE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1] FALSE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE FALSE  TRUE FALSE  TRUE FALSE FALSE FALSE
[1] 2
[1] FALSE FALSE FALSE FALSE  TRUE FALSE  TRUE FALSE
[1] 2
[1]  TRUE  TRUE FALSE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 6
[1] FALSE FALSE  TRUE  TRUE  TRUE FALSE FALSE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE
[1] 2
[1]  TRUE  TRUE FALSE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
```
]]
---
class: split-40
count: false

```r
for (i in 1:100){ 
# truth
truth <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

# lady's responses
guess <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

print(truth == guess)  
print(sum(truth == guess))  
}  
```
]]
.column[.content[

```
[1]  TRUE FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE FALSE FALSE FALSE  TRUE FALSE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1] FALSE  TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 6
[1]  TRUE  TRUE FALSE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 6
[1] FALSE FALSE  TRUE FALSE FALSE FALSE FALSE  TRUE
[1] 2
[1] FALSE  TRUE FALSE  TRUE FALSE FALSE  TRUE  TRUE
[1] 4
[1]  TRUE  TRUE  TRUE FALSE FALSE  TRUE FALSE FALSE
[1] 4
[1]  TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE
[1] 4
[1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[1] 8
[1]  TRUE  TRUE FALSE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE FALSE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE
[1] 2
[1] FALSE FALSE FALSE FALSE  TRUE  TRUE FALSE FALSE
[1] 2
[1]  TRUE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE
[1] 4
[1] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE  TRUE
[1] 2
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1] FALSE FALSE  TRUE  TRUE FALSE FALSE  TRUE  TRUE
[1] 4
[1] FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE FALSE  TRUE FALSE FALSE
[1] 4
[1]  TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE FALSE
[1] 6
[1]  TRUE FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1] FALSE FALSE  TRUE FALSE FALSE  TRUE  TRUE  TRUE
[1] 4
[1] FALSE  TRUE FALSE FALSE FALSE FALSE  TRUE FALSE
[1] 2
[1]  TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE FALSE
[1] 6
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
[1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[1] 8
[1] FALSE  TRUE  TRUE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE  TRUE FALSE FALSE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1]  TRUE  TRUE  TRUE  TRUE FALSE  TRUE FALSE  TRUE
[1] 6
[1]  TRUE  TRUE FALSE  TRUE  TRUE FALSE FALSE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE
[1] 2
[1]  TRUE FALSE  TRUE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1] FALSE FALSE  TRUE FALSE  TRUE FALSE FALSE FALSE
[1] 2
[1] FALSE  TRUE FALSE  TRUE FALSE FALSE  TRUE  TRUE
[1] 4
[1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[1] 8
[1] FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1] FALSE  TRUE FALSE FALSE  TRUE FALSE  TRUE  TRUE
[1] 4
[1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[1] 8
[1]  TRUE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1]  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE  TRUE
[1] 4
[1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[1] 0
[1]  TRUE  TRUE FALSE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE FALSE FALSE FALSE  TRUE
[1] 4
[1]  TRUE FALSE FALSE  TRUE FALSE FALSE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE FALSE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE  TRUE FALSE  TRUE  TRUE FALSE FALSE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE  TRUE  TRUE FALSE FALSE
[1] 4
[1]  TRUE FALSE FALSE  TRUE  TRUE FALSE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE
[1] 4
[1] FALSE FALSE FALSE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 4
[1]  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE FALSE FALSE FALSE FALSE  TRUE FALSE
[1] 2
[1] FALSE FALSE  TRUE FALSE FALSE FALSE  TRUE FALSE
[1] 2
[1] FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE
[1] 2
[1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE  TRUE  TRUE  TRUE FALSE FALSE FALSE
[1] 4
[1] FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 6
[1]  TRUE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE
[1] 2
[1] FALSE  TRUE  TRUE FALSE  TRUE FALSE FALSE  TRUE
[1] 4
[1] FALSE FALSE  TRUE FALSE  TRUE FALSE  TRUE  TRUE
[1] 4
[1]  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE FALSE  TRUE  TRUE  TRUE FALSE FALSE  TRUE
[1] 4
[1]  TRUE FALSE  TRUE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE FALSE FALSE
[1] 4
[1]  TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE FALSE FALSE  TRUE FALSE  TRUE FALSE
[1] 4
[1]  TRUE  TRUE  TRUE FALSE  TRUE FALSE FALSE FALSE
[1] 4
[1] FALSE FALSE FALSE FALSE  TRUE FALSE  TRUE FALSE
[1] 2
[1] FALSE FALSE  TRUE FALSE FALSE  TRUE FALSE FALSE
[1] 2
[1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[1] 6
[1] FALSE FALSE  TRUE FALSE  TRUE  TRUE  TRUE FALSE
[1] 4
[1]  TRUE FALSE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE FALSE FALSE  TRUE FALSE  TRUE
[1] 4
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
[1] FALSE  TRUE  TRUE FALSE  TRUE FALSE FALSE  TRUE
[1] 4
[1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE FALSE  TRUE FALSE  TRUE FALSE  TRUE
[1] 4
[1] FALSE FALSE FALSE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 4
[1]  TRUE  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE  TRUE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 6
[1] FALSE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE  TRUE
[1] 6
[1]  TRUE FALSE  TRUE FALSE  TRUE FALSE FALSE  TRUE
[1] 4
[1] FALSE FALSE  TRUE FALSE FALSE FALSE FALSE  TRUE
[1] 2
[1]  TRUE FALSE  TRUE FALSE FALSE  TRUE  TRUE FALSE
[1] 4
[1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE FALSE FALSE
[1] 4
[1] FALSE FALSE  TRUE FALSE FALSE  TRUE FALSE FALSE
[1] 2
[1] FALSE  TRUE FALSE  TRUE FALSE  TRUE  TRUE FALSE
[1] 4
```
]]

---
class: inverse, center, middle

# Summarizing many realizations

# Collecting the result, num_trials

---

```r
*num_trials <- 100
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
num_trials <- 100 
*rep(NA, num_trials)
```
]]
.column[.content[

```
  [1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
 [26] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
 [51] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
 [76] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
```
]]
---
class: split-40
count: false

```r
num_trials <- 100 
rep(NA, num_trials) -> 
* num_correct
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
num_trials <- 100 
rep(NA, num_trials) -> 
 num_correct

*for (i in 1:num_trials){
# truth
*truth <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

# lady's responses
*guess <- sample(c("tea", "tea", "tea", "tea", "milk", "milk", "milk", "milk"))

*num_correct[i] <- sum(truth == guess)
*} 
```
]]
.column[.content[

]]

---
class: inverse, center, middle

# Interpreting the result: Bristol gets 6 correct cups...

---

```r
*library(tidyverse)
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
library(tidyverse)  
*tibble(which_trial = 1:num_trials,
*      num_correct)
```
]]
.column[.content[

```
# A tibble: 100 x 2
 which_trial num_correct
 <int> <int>
 1 1 6
 2 2 6
 3 3 6
 4 4 6
 5 5 4
 6 6 2
 7 7 4
 8 8 2
 9 9 4
10 10 8
# … with 90 more rows
```
]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
*guessing_results
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

*guessing_results
```
]]
.column[.content[

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
* group_by(num_correct)
```
]]
.column[.content[

```
# A tibble: 100 x 2
# Groups: num_correct [5]
 which_trial num_correct
 <int> <int>
 1 1 6
 2 2 6
 3 3 6
 4 4 6
 5 5 4
 6 6 2
 7 7 4
 8 8 2
 9 9 4
10 10 8
# … with 90 more rows
```
]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
  group_by(num_correct) %>%  
* summarize(trials = n())
```
]]
.column[.content[

```
# A tibble: 5 x 2
 num_correct trials
 <int> <int>
1 0 1
2 2 15
3 4 52
4 6 29
5 8 3
```
]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
  group_by(num_correct) %>%  
  summarize(trials = n()) ->  
*guessing_summary
```
]]
.column[.content[

]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
  group_by(num_correct) %>%  
  summarize(trials = n()) ->  
guessing_summary

*ggplot(data = guessing_results)
```
]]
.column[.content[
<img src="pvalue_files/figure-html/part7_auto_8_output-1.png" width="432" />
]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
  group_by(num_correct) %>%  
  summarize(trials = n()) ->  
guessing_summary

ggplot(data = guessing_results) + 
* aes(x = as_factor(num_correct))
```
]]
.column[.content[
<img src="pvalue_files/figure-html/part7_auto_9_output-1.png" width="432" />
]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
  group_by(num_correct) %>%  
  summarize(trials = n()) ->  
guessing_summary

ggplot(data = guessing_results) + 
 aes(x = as_factor(num_correct)) + 
* geom_bar()
```
]]
.column[.content[
<img src="pvalue_files/figure-html/part7_auto_10_output-1.png" width="432" />
]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
  group_by(num_correct) %>%  
  summarize(trials = n()) ->  
guessing_summary

ggplot(data = guessing_results) +  
  aes(x = as_factor(num_correct)) +  
  geom_bar()

*sum(guessing_results$num_correct >= 6) / num_trials
```
]]
.column[.content[
<img src="pvalue_files/figure-html/part7_auto_11_output-1.png" width="432" />

```
[1] 0.32
```
]]
---
class: split-40
count: false

```r
library(tidyverse)  
tibble(which_trial = 1:num_trials,  
       num_correct) ->  
guessing_results

guessing_results %>%  
  group_by(num_correct) %>%  
  summarize(trials = n()) ->  
guessing_summary

ggplot(data = guessing_results) +  
  aes(x = as_factor(num_correct)) +  
  geom_bar()

sum(guessing_results$num_correct >= 6) / num_trials 
```
]]
.column[.content[
<img src="pvalue_files/figure-html/part7_auto_12_output-1.png" width="432" />

```
[1] 0.32
```
]]

---

# Game Time

https://evangelinereynolds.shinyapps.io/EyeBallIt/