index.qmd

---
title: "Introduction to Statistics"
author: "Nicola Rennie"
format:
  revealjs:
    theme: [default, custom.scss]
engine: knitr
filters:
  - pyodide
---

# Descriptive statistics {background-color="#1A936F"}

Descriptive statistics provide a summary that quantitatively describes a sample of data.

```{python}
#| label: setup
#| echo: false
#| eval: true
#| message: false
import pandas as pd
import numpy as np

np.random.seed(1234)
population_df = pd.DataFrame({
    'ID': np.arange(1, 201),
    'x': np.repeat(np.arange(1, 21), 10),
    'y': np.tile(np.arange(1, 11), 20),
    'Value': np.random.poisson(250, 200)
})
sample_size = 10
sample_ids = np.random.choice(population_df['ID'], size=sample_size, replace=False)
sample_df = population_df[population_df['ID'].isin(sample_ids)]
```

## Population

**Population** refers to the entire group of individuals that we want to draw conclusions about. 

```{r}
#| label: pop-people
#| eval: true
#| echo: false
#| fig-align: center
#| fig-height: 4.16
library(emojifont)
library(ggplot2)
library(reticulate)
population_df <- py$population_df
ggplot() +
  geom_text(data = population_df,
            mapping = aes(x = x,
                          y = y,
                          label = fontawesome('fa-user'),
                          colour = Value),
            family='fontawesome-webfont', size = 20) +
  scale_colour_gradient(low = "#baded3", high = "#12664d") +
  labs(title = "Population: 200 people") +
  theme_void() +
  theme(legend.position = "none", 
        legend.title = element_blank(),
        plot.margin = margin(10, 10, 10, 10), 
        plot.title = element_text(face = "bold",
                                  hjust = 0.5,
                                  size = 36,
                                  margin = margin(b = 10)))

```

## Sample

**Sample** refers to the (usually smaller) group of people for which we have collected data on.

```{r}
#| label: samp-people
#| eval: true
#| echo: false
#| fig-align: center
#| fig-height: 4.16
sample_df <- py$sample_df
sample_size <- py$sample_size
ggplot() +
  geom_text(data = population_df,
            mapping = aes(x = x,
                          y = y,
                          label = fontawesome('fa-user')),
            family='fontawesome-webfont', size = 20, colour = "grey") +
  geom_text(data = sample_df,
            mapping = aes(x = x,
                          y = y,
                          label = fontawesome('fa-user'),
                          colour = Value),
            family='fontawesome-webfont', size = 20) +
  scale_colour_gradient(low = "#baded3", high = "#12664d") +
  labs(title = glue::glue("Sample: {sample_size} people")) +
  theme_void() +
  theme(legend.position = "none", 
        legend.title = element_blank(),
        plot.margin = margin(10, 10, 10, 10), 
        plot.title = element_text(face = "bold",
                                  hjust = 0.5,
                                  size = 36,
                                  margin = margin(b = 10)))

```

## Generate sample data {.scrollable}

For the examples later, let's create a population of data in Python:

```{pyodide-python}
# Generate population data
import numpy as np
np.random.seed(1234)
population = np.random.poisson(250, 200)
print("Population generated!")
```

## Generate sample data {.scrollable}

... and draw a sample from it:

```{pyodide-python}
# Pick a sample
np.random.seed(1234)
sample_size = 10
sample_data = np.random.choice(population, size=sample_size, replace=False)
print("You've created a sample of data!")
```

<br>

::: {.fragment}

What do the values look like?

```{pyodide-python}
sample_data
```

:::

## Mean

The mean, often simply called the *average*, is defined as *the sum of all values divided by the number of values*. It's a measure of central tendency that tells us what's happening near the middle of the data.

::::{style='text-align: center;'}

$\bar{x} = \frac{1}{n} \sum_{i=i}^{n} x_{i}$

::::

<br>

::: {.fragment}

In Python, we use the `mean()` function from `numpy`:

```{pyodide-python}
# Calculate mean
np.mean(sample_data)
```

:::

## Median

The median of a dataset is the middle value when the data is arranged in ascending order, or the average of the two middle values if the dataset has an even number of observations.

<br>

::: {.fragment}

In Python, we use the `median()` function from `numpy`:

```{pyodide-python}
# Calculate median
np.median(sample_data)
```

:::

## Mode

The mode statistic represents the value that appears most frequently in a dataset.

<br>

::: {.fragment}

In Python, we use the `mode()` function from `statistics`:

```{pyodide-python}
# Calculate mode
import statistics
statistics.mode(sample_data)
```

:::

## Range

The range is the difference between the maximum and minimum values in a dataset.

<br>

::: {.fragment}

In Python, we can use the `max()` and `min()` function and subtract the values:

```{pyodide-python}
# Subtract max and min values
max(sample_data) - min(sample_data)
```

Or, we can use the `ptp()` function from `numpy`:

```{pyodide-python}
# Calculate range using numpy
np.ptp(sample_data)
```

:::

## Sample variance

The sample variance tells us about how spread out the data is. A lower variance indicates that values tend to be close to the mean, and a higher variance indicates that the values are spread out over a wider range.

::::{style='text-align: center;'}

$s^2 = \frac{\Sigma_{i= 1}^{N} (x_i - \bar{x})^2}{n-1}$

::::

<br>

::: {.fragment}

In Python, we use the `var()` function from `numpy`:

```{pyodide-python}
# Calculate variance
np.var(sample_data)
```

:::

## Sample standard deviation

The sample standard deviation is the square root of the variance. It also tells us about how spread out the data is. 

::::{style='text-align: center;'}

$s = \sqrt{\frac{\Sigma_{i= 1}^{N} (x_i - \bar{x})^2}{n-1}}$

::::

<br>

::: {.fragment}

In Python, we use the `std()` function from `numpy`:

```{pyodide-python}
# Calculate standard deviation
np.std(sample_data)
```

:::

## Descriptive statistics

Descriptive statistics provide a summary that quantitatively describes a sample of data.

* Mean: The sum of the values divided by the number of values.
* Median: The middle value of the data when it's sorted.
* Mode: The value that appears most frequently.
* Range: The difference between the maximum and minimum values.
* Variance: The average of the squared differences from the mean.
* Standard deviation: The square root of the variance.

## Exercise

In Python:

* Import `numpy` and `statistics`.
* Import `data` from `pydataset` using `from pydataset import data`
* Load the `housing` data set using `housing = data('Housing')`
* Calculate the mean, median, mode, range, variance, and standard deviation of house prices. 

> Remember: you can extract a column in Python using `dataset['column_name']`.

## Exercise solutions

```{python}
#| echo: true
import numpy as np
import statistics
from pydataset import data

# load data
housing = data('Housing')
```

## Exercise solutions

```{python}
#| echo: true
# summary statistics
np.mean(housing['price'])
np.median(housing['price'])
statistics.mode(housing['price'])
max(housing['price']) - min(housing['price'])
np.var(housing['price'])
np.std(housing['price'])
```

# Questions? {background-color="#1A936F"}