POL90: Statistics

class: center, middle, inverse, title-slide

# POL90: Statistics
## Alternatives to <em>t</em>-Tests
### Prof Wasow <br/> Assistant Professor, Politics <br/> Pomona College
### 2022-02-21

---

# Announcements

.large[
* Assignments

+ PS05 due <mark>Friday</mark>
    
    + Report 1

]

.large[
* Statistical Sleuth
    + Important to read along

+ Skim Chapter 4
]

---
# Schedule

<table>
 <thead>
  <tr>
   <th style="text-align:right;"> Week </th>
   <th style="text-align:left;"> Date </th>
   <th style="text-align:left;"> Day </th>
   <th style="text-align:left;"> Title </th>
   <th style="text-align:right;"> Chapter </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:right;"> 4 </td>
   <td style="text-align:left;"> Feb 7 </td>
   <td style="text-align:left;"> Mon </td>
   <td style="text-align:left;"> Inference Using t-Distributions </td>
   <td style="text-align:right;"> 2 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 4 </td>
   <td style="text-align:left;"> Feb 9 </td>
   <td style="text-align:left;"> Wed </td>
   <td style="text-align:left;"> Confidence Intervals </td>
   <td style="text-align:right;"> 2 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 5 </td>
   <td style="text-align:left;"> Feb 14 </td>
   <td style="text-align:left;"> Mon </td>
   <td style="text-align:left;"> A Closer Look at Assumptions </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 5 </td>
   <td style="text-align:left;"> Feb 16 </td>
   <td style="text-align:left;"> Wed </td>
   <td style="text-align:left;"> A Closer Look at Assumptions </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;color: black !important;background-color: yellow !important;"> 6 </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Feb 21 </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Mon </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Alternatives to the t-Tools </td>
   <td style="text-align:right;color: black !important;background-color: yellow !important;"> 4 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 6 </td>
   <td style="text-align:left;"> Feb 23 </td>
   <td style="text-align:left;"> Wed </td>
   <td style="text-align:left;"> Alternatives to the t-Tools </td>
   <td style="text-align:right;"> 4 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 7 </td>
   <td style="text-align:left;"> Feb 28 </td>
   <td style="text-align:left;"> Mon </td>
   <td style="text-align:left;"> Comparison Among Several Samples </td>
   <td style="text-align:right;"> 5 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 7 </td>
   <td style="text-align:left;"> Mar 2 </td>
   <td style="text-align:left;"> Wed </td>
   <td style="text-align:left;"> Comparison Among Several Samples </td>
   <td style="text-align:right;"> 5 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> Mar 7 </td>
   <td style="text-align:left;"> Mon </td>
   <td style="text-align:left;"> Simple Linear Regression </td>
   <td style="text-align:right;"> 7 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> Mar 9 </td>
   <td style="text-align:left;"> Wed </td>
   <td style="text-align:left;"> Simple Linear Regression </td>
   <td style="text-align:right;"> 7 </td>
  </tr>
</tbody>
</table>

---
## Assignment schedule

<table>
 <thead>
  <tr>
   <th style="text-align:right;"> Week </th>
   <th style="text-align:left;"> Date </th>
   <th style="text-align:left;"> Day </th>
   <th style="text-align:left;"> Assignment </th>
   <th style="text-align:right;"> Percent </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:right;"> 5 </td>
   <td style="text-align:left;"> Feb 18 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> PS04 </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;color: black !important;background-color: yellow !important;"> 6 </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Feb 25 </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Fri </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> PS05 </td>
   <td style="text-align:right;color: black !important;background-color: yellow !important;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;color: black !important;background-color: yellow !important;"> 7 </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Mar 4 </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Fri </td>
   <td style="text-align:left;color: black !important;background-color: yellow !important;"> Report1 </td>
   <td style="text-align:right;color: black !important;background-color: yellow !important;"> 6 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> Mar 11 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> PS06 </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 9 </td>
   <td style="text-align:left;"> Mar 18 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> Spring break </td>
   <td style="text-align:right;"> NA </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 10 </td>
   <td style="text-align:left;"> Mar 25 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> PS07 </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 11 </td>
   <td style="text-align:left;"> Apr 1 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> PS08 </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 12 </td>
   <td style="text-align:left;"> Apr 8 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> Report2 </td>
   <td style="text-align:right;"> 8 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 13 </td>
   <td style="text-align:left;"> Apr 15 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> PS09 </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 14 </td>
   <td style="text-align:left;"> Apr 22 </td>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:left;"> PS10 </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
</tbody>
</table>

---
## Knitr code tips: figures and tables

- Avoid giant plots with chunk headers like `fig.height = 3` and `fig.width = 5`

- Separate table chunks, put `results = 'asis'` in chunk header

---
class: middle, center, inverse

# Cognitive load theory &
# Wilcoxon Rank-sum test

---
## Cognitive load theory

.large[  
  - Data: 28 ninth year students were randomly assigned to one of two groups. One group received the conventional instructional materials and the other received the modified instructional materials. After self-study, students were given a question. The number of seconds required to arrive at a solution was recorded.

- Research Question: Is there sufficient evidence to suggest that the solution times for the modified instructional materials group are generally shorter than for the conventional materials group?
]

---
## Cognitive load theory: Conventional
.center[
    ![](images/ss_display_4_2.png)
  ]
---
## Cognitive load theory: Modified
  .center[
    ![](images/ss_display_4_3.png)
    ]
    
---
## Load data: What does censored mean?

```r
cog <- Sleuth3::case0402 %>% clean_names()

head(cog)
```

```
  time treatment censored
1   68  Modified        0
2   70  Modified        0
3   73  Modified        0
4   75  Modified        0
5   77  Modified        0
6   80  Modified        0
```

```r
tail(cog)
```

```
   time    treatment censored
23  265 Conventional        0
24  300 Conventional        1
25  300 Conventional        1
26  300 Conventional        1
27  300 Conventional        1
28  300 Conventional        1
```

---
.pull-left[
## Stem-and-leaf plot
    ]

.pull-right[
<img src="images/cognitive_load3.png" width="70%" style="display: block; margin: auto;" />
  ]

---
## Visualization: Ridge plot

.left-code[

```r
suppressMessages(
  # extends ggplot 
* library(ggridges)
  )

ggplot(data = cog) +
  aes(x = time) +
  aes(y = treatment) +
  # new geom
* geom_density_ridges(
    stat  = "binline",
*   scale = .7)
```
]

.right-plot[
<img src="week06_01_files/figure-html/cognitive_density_code_plot-1.png" width="100%" style="display: block; margin: auto;" />
]

---
## Visualization: Scatter Plot

.left-code[

```r
cog <- Sleuth3::case0402 %>% 
  clean_names()

ggplot(data = cog) +
  aes(x     = treatment,
      y     = time, 
      color = treatment) +
* geom_jitter(alpha = .5,
             size    = 4,
             width   = .1,
             height  = 1) +
  stat_summary(
    fun.y    = mean, 
    fun.ymin = mean, 
    fun.ymax = mean, 
    geom     = "crossbar", 
    width    = 0.25) 
```
]

.right-plot[
<img src="week06_01_files/figure-html/cog_load_plot1-1.png" width="100%" style="display: block; margin: auto;" />
]

---
## Calculating Rank-sum Statistic (Think Olympic Teams)

.footnote[Source: *Statistical Sleuth*, Display 4.5]

---
## Rank-sum Test

.large[
- Resistant alternative to the two-sample `$t$`-test

- It is distribution-free (non-parametric): not based on distribution assumptions

- It performs nearly as well as the `$t$`-test when the two populations are normal and considerably better when there are extreme outliers.

- One way to address "censored" observations
]

---
## Rank-sum Test

.large[  
- It is particulary useful when:
  
  - Only ordinal data (ranked) exists

- The data does not approximate a normal distribution

- The data may appear skewed, variances different, possibly due to a small sample size

- Its drawbacks: Associated confidence intervals are not computed by most of the statistical computer packages and it  does  not easily extend to more complicated situations (for example, comparisons among several samples)
]

---
## Rank Sum Test

.large[
- The test statistic, `$T$`, is the sum of all the ranks in one group

- Center: `$\text{Mean }(T)$` = `$n_1\bar{R}$`
  
- `$\bar{R}$`: Combined sample average

- Spread: SD( `$T$` ) = `$s_R \sqrt{\frac{n_1n_2}{(n_1 + n_2)}}$`
  
  - `$s_R$`: Combined sample standard deviation
]

---
## Rank Sum Test

.large[  
- Shape: shape of the sampling distribution approximately normal

- Conversion to ranks avoids distributional anomalies

- Randomization distribution of `$T$` approximately normal

- Exceptions: small `$N$` or large number of ties
]

---
## What is Rank Transformation?

```r
cog <- cog %>% 
  mutate(rank = rank(time)) %>% 
  arrange(rank)

head(cog, 15)
```

```
   time    treatment censored rank
1    68     Modified        0  1.0
2    70     Modified        0  2.0
3    73     Modified        0  3.0
4    75     Modified        0  4.0
5    77     Modified        0  5.0
6    80     Modified        0  6.5
7    80     Modified        0  6.5
8   130 Conventional        0  8.0
9   132     Modified        0  9.0
10  139 Conventional        0 10.0
11  146 Conventional        0 11.0
12  148     Modified        0 12.0
13  150 Conventional        0 13.0
14  155     Modified        0 14.0
15  161 Conventional        0 15.0
```

---
## What Happens to Censored Observations?

```r
tail(cog, 15)
```

```
   time    treatment censored rank
14  155     Modified        0   14
15  161 Conventional        0   15
16  177 Conventional        0   16
17  183     Modified        0   17
18  197     Modified        0   18
19  206     Modified        0   19
20  210     Modified        0   20
21  228 Conventional        0   21
22  242 Conventional        0   22
23  265 Conventional        0   23
24  300 Conventional        1   26
25  300 Conventional        1   26
26  300 Conventional        1   26
27  300 Conventional        1   26
28  300 Conventional        1   26
```

---
## Look at One Randomization

```r
cog_shuffle <- cog %>% 
  mutate(treat_shuffle = sample(treatment)) %>% 
  arrange(rank)

head(cog_shuffle, 15)
```

```
   time    treatment censored rank treat_shuffle
1    68     Modified        0  1.0  Conventional
2    70     Modified        0  2.0  Conventional
3    73     Modified        0  3.0  Conventional
4    75     Modified        0  4.0  Conventional
5    77     Modified        0  5.0  Conventional
6    80     Modified        0  6.5      Modified
7    80     Modified        0  6.5  Conventional
8   130 Conventional        0  8.0  Conventional
9   132     Modified        0  9.0      Modified
10  139 Conventional        0 10.0      Modified
11  146 Conventional        0 11.0  Conventional
12  148     Modified        0 12.0      Modified
13  150 Conventional        0 13.0  Conventional
14  155     Modified        0 14.0  Conventional
15  161 Conventional        0 15.0      Modified
```

---
## Randomization Distribution with Wilcox Test
.left-code[

```r
T_stat <- NA

for (i in 1:5000) {
* cog$treatment2 <-
*   sample(cog$treatment)

T_stat[i] <- 
    cog %>% 
    filter(treatment2 == "Modified") %>%
    pull(rank) %>% 
    sum()
    }

hist(T_stat)
```
]

.right-plot[
<img src="week06_01_files/figure-html/cog_sim_plot1-1.png" width="100%" style="display: block; margin: auto;" />

]

---
## Randomization Distribution with Wilcox Test
.left-code[

```r
cog <- Sleuth3::case0402 %>% 
  clean_names() %>% 
  mutate(rank = rank(time))

T_stat <- NA

for (i in 1:5000) {
  cog$treatment2 <-       
    sample(cog$treatment)

T_stat[i] <- 
    cog %>% 
    filter(treatment2 == "Modified") %>%
    pull(rank) %>% 
    sum()
    }

hist(T_stat)

*abline(v = 137, col = "red")
```
]

.right-plot[
<img src="week06_01_files/figure-html/cog_sim_plot2-1.png" width="100%" style="display: block; margin: auto;" />

]

---
## Theoretical Distribution with Wilcox Test

.large[
* We can also compare to a Normal distribution
]

.footnote[Source: *Statistical Sleuth*, Display 4.6]

---
## Rank Sum Test

.vertical-center[
.large[
`\begin{eqnarray*}
Z\textrm{-statistic} & = &\frac{ \left( T - \textrm{Mean}(T)\right)}{\textrm{SD}(T)} 
\end{eqnarray*}`
]
]

---
## Cognitive load theory example

.large[  
  - Summary of Statistical Findings: There is convincing evidence that a student could solve the problem more quickly if taught with the modified method than if taught with the conventional method (one-sided p-value is .0013 from the rank-sum test).

- The modified instructional materials shortened solution times by an estimated 152 seconds (95% confidence interval for an additive treatment effect is from 58 to 159 seconds).
]

---
## Summary of Method 
  
- Rank transformation is just another kind of transformation

- Rank-sum and signed-rank tests do not assume normality

- Also useful with censored data

- Rank-sum less intuitive than a mean or median

- Think Olympic teams

---
## Main take-aways

.large[
- One overarching approach with hypothesis testing
  - many different test statistics
  - sometimes we need to transform our data

- We can transform data many ways
  - `log()`
  - dichotomize (convert to binary or positive / negative)
  - rank
  - many others

- Communicating clearly is essential
]

---
class: center, middle, inverse

# Chapter 5: Multiple Comparisons with ANOVA

---

## Caloric restriction and longevity
<br>

.large[
* Question: 
  - Does reducing caloric intake increase longevity?

* Data: 
  - Female mice were randomly assigned to six diet groups and their lifetimes were measured

* Why do we care?
]

---