618_06_categorical_and_text_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SI 618: Data Manipulation and Analysis\n",
    "## 06 - Categorical Data & Text Processing \n",
    "### Pivoting, contingency tables, crosstabs, mosaic plots and chi-squared\n",
    "\n",
    "### Dr. Chris Teplovs, School of Information, University of Michigan\n",
    "<small><a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"><img alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /></a> This work is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview for today\n",
    "* Review HW1\n",
    "* Project proposal review \n",
    "* Categorical Data: contingency tables, crosstabs, mosaic plots, chi-squared\n",
    "* Text Processing: regular expressions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Q0: What did you find confusing from last class?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Categorical Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contingency tables, crosstabs, and chi-square"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's generate a data frame to play with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame({'color' : ['red', 'green', 'green', 'black'] * 6,\n",
    "                   'make' : ['ford', 'toyota', 'dodge'] * 8,\n",
    "                   'vehicleClass' : ['suv', 'suv', 'suv', 'car', 'car', 'truck'] * 4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>make</th>\n",
       "      <th>vehicleClass</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>red</td>\n",
       "      <td>ford</td>\n",
       "      <td>suv</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>green</td>\n",
       "      <td>toyota</td>\n",
       "      <td>suv</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>green</td>\n",
       "      <td>dodge</td>\n",
       "      <td>suv</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>black</td>\n",
       "      <td>ford</td>\n",
       "      <td>car</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>red</td>\n",
       "      <td>toyota</td>\n",
       "      <td>car</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color    make vehicleClass\n",
       "0    red    ford          suv\n",
       "1  green  toyota          suv\n",
       "2  green   dodge          suv\n",
       "3  black    ford          car\n",
       "4    red  toyota          car"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the most basic transformations we can do is a crosstab:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>vehicleClass</th>\n",
       "      <th>car</th>\n",
       "      <th>suv</th>\n",
       "      <th>truck</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>color</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>black</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>green</th>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>red</th>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "vehicleClass  car  suv  truck\n",
       "color                        \n",
       "black           2    2      2\n",
       "green           4    6      2\n",
       "red             2    4      0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ct = pd.crosstab(df.color,df.vehicleClass)\n",
    "ct"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how similar it is to pivoting.  In fact, go ahead and use ```pivot_table``` to do the same sort of transformation:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q1: Use ```pivot_table``` to create a DataFrame similar to the one from the ```crosstab``` above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"3\" halign=\"left\">make</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>vehicleClass</th>\n",
       "      <th>car</th>\n",
       "      <th>suv</th>\n",
       "      <th>truck</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>color</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>black</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>green</th>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>red</th>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             make           \n",
       "vehicleClass  car  suv truck\n",
       "color                       \n",
       "black         2.0  2.0   2.0\n",
       "green         4.0  6.0   2.0\n",
       "red           2.0  4.0   NaN"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.pivot_table(data = df, columns = 'vehicleClass', index = 'color', aggfunc = len)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, we would like to visualize our results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a16f80978>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWIAAAEKCAYAAAAo+19NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGodJREFUeJzt3XmYVdWZ7/HvDwQBwSkgDiiOwUQDOCdxuGobY7dTEoxDq0n0MRg1iWQw16u5rVGxHdJm0Nz2luIQgy3KjUlURG2HcDUOgIKiiCKiFoI0JkZBFKrqvX+cjZ6LcOqc4uxadXb9Pj77ofY5e+311k7lrbfWXnsdRQRmZpZOj9QBmJl1d07EZmaJORGbmSXmRGxmlpgTsZlZYk7EZmaJORGbmdWZpI0lTZT0oqTZkr5Q6fj1OiswM7Nu5FfA5Ig4RlJvoF+lg+UHOszM6kfShsBMYPuoMsF22Yp47NAT/RvCzKpy/mvjta7nWLlkXtU5p/egHU4HRpe91BQRTdnX2wP/BdwoaQQwHTg7Ipat7XweIzYzq1FENEXEnmVbU9nb6wG7A/8eEbsBy4BzK53PidjMDKCttfqtsmagOSKezPYnUkrMa9VlhybMzDpVa0tdThMRiyS9IWlYRMwB/gF4oVIbJ2IzMyCirZ6n+x4wPpsxMQ84pdLBTsRmZgBt9UvEETED2LPa452IzcwA6lsR18SJ2MwMqrkJlxsnYjMzcEVsZpZa1GnWREc4EZuZQV1v1tXKidjMDDw0YWaWnG/WmZkl5orYzCwx36wzM0vMN+vMzNKK8BixmVlaHiM2M0vMQxNmZom5IjYzS6x1ZbKunYjNzMBDE2ZmyXlowswsMVfEZmaJORGbmaUVvllnZpaYx4jNzBLz0ISZWWKuiM3MEnNFbGaWmCtiM7PEWrwwfMMasMWmHPWLM+g/aCOiLXjm1oeYeuN9qcMqHF/n/HX7a+yKuHFFaxsPXjKeRbPm03uDPpx69yW8+ugslry8IHVoheLrnL9uf43rOEYsaT7wHtAKtETEnpWOdyJeR0sXv8PSxe8AsGLZB7w9900GDN6k+/zwdhJf5/x1+2tc/4r4oIhYUs2BPerdczlJm67hte3y7DOljYYMZPAuQ1kw45XUoRSar3P+uuU1bmurfquzXBMxcJekDVftSPoscFfOfSbRq9/6jLp2DA9cdAsrli5PHU5h+Trnr9te42irfqvibMD9kqZLGt3ewXkn4kspJeP+kvYA7gBOWtvBkkZLmiZp2tSlc3MOrX56rNeTUdeOYdYfHmPO5GmpwyksX+f8detr3NJS9Vaeq7Jt9WS7b0TsDvwjcJakAyp1nesYcUTcI6kXcD8wAPhKRLxc4fgmoAlg7NATI8/Y6unwK77N23MX8NT196YOpdB8nfPXra9xVJ9yynPVWt5/M/t3saQ7gb2BKWs7PpdELOlqSqX5KhsC84DvSSIivp9HvykM2fPTDB+1P2/Nfp3TJl0KwMNXTuCVh2cmjqxYfJ3z1+2vcZ3GfiVtAPSIiPeyrw8FLqrUJq+KePW/aabn1E9yzdNeYuzQE1OHUXi+zvnr9te4fjfhBgN3SoJSjr01IiZXapBLIo6Im+Gj3wwfRERrtt8TWD+PPs3M1kmdpq9FxDxgRC1t8r5Z9yDQt2y/L/CfOfdpZla71tbqtzrL+4GOPhGxdNVORCyV1C/nPs3Mapdw9bW8K+JlknZftZNNYetGExPNrGEkfKAj74p4DHCHpDez/S2A43Lu08ysdkVd9CcipkraGRgGCHgxItJ9Qp+Z2VpEW7pHFzpj0Z9hwGeBPsBu2Tzi33ZCv2Zm1SvqJ3RIugA4kFIinkTpcb9HASdiM+tacpgNUa28b9YdA/wDsCgiTqE0t87ziM2s6ynwzbrlEdEmqSVbhW0xsH3OfZqZ1a6oQxPANEkbA9dResx5KfBUzn2amdWuhkV/6i3vWRNnZl9eK2kysGFEPJtnn2ZmHVK0irj8IY41vRcRT+fRr5lZhxVw+tq/lX1d/t0p2z84p37NzDom4ayJvFZfOwhAUl/gTGA/Sgn4/wL/nkefZmbrIoo2NFHmZuBd4NfZ/gmU5hAfm3O/Zma1KeDQxCrDIqJ8Xc6HJXWT5f7NrKEkXGsi7wc6npH0+VU7kvYBHsu5TzOz2rVF9Vud5TVr4jlKY8K9gG9Iej3bHwq8kEefZmbrpKVgN+uAI3I6r5lZPoq2DGZEvJbHec3MclPgm3VmZg2hyNPXzMwagytiM7PEnIjNzBIr2iPOZmaNpuifWWdm1vU5EZuZJeZZE2ZmibkiNjNLrM6JWFJPYBqwICIqPm3sRGxmBkRr3YcmzgZmAxu2d2CXTcTfPfit1CEUXr8rm1KHUHhX7PE/U4dg1apjRSxpCHA4MBb4YXvH570MpplZQ4i2qHqTNFrStLJt9Gqn+yXwE6CqMrvLVsRmZp2qhoo4IpqANf5JKekIYHFETJd0YDXncyI2M4Mqa9eq7AscJemfgD7AhpJ+FxEnra2BhybMzIBoaat6q3ieiP8REUMiYlvgeOChSkkYXBGbmZWke57DidjMDPJZayIiHgEeae84J2IzM3BFbGaWmldfMzNLzRWxmVla0ZKubydiMzMgXBGbmSXmRGxmlpYrYjOzxJyIzcwSi1Yl69uJ2MwMV8RmZslFmytiM7OkXBGbmSUW4YrYzCwpV8RmZom1edaEmVlavllnZpaYE7GZWWKRbjliJ2IzM3BFbGaWXMrpaz3aO0BST0k/6IxgzMxSaW1V1Vu9tZuII6IVOLruPZuZdSERqnqrt2qHJh6TdA0wAVj2ceDxdN0jMjNLoBHGiL+Y/XtR2WsBHFzfcMzM0ujysyYi4qC8AzEzS6nLV8SSNgIuAA7IXvozcFFE/D2vwMzMOlNrW7u3zHJT7dDEDcAs4Nhs/2TgRuBreQTVkNSD/hf8L9r+toT3f/XT1NEU0rvvLeWCy37J3HmvgcTF5/2Akbt+JnVYhTFgi0056hdn0H/QRkRb8MytDzH1xvtSh9VpuvzQBLBDRIwq2/+ZpBl5BNSoen/pq7QufB316Zc6lMK67JfXsu8+e/KLsT9l5cqVLP/gw9QhFUq0tvHgJeNZNGs+vTfow6l3X8Krj85iycsLUofWKdrqNBtCUh9gCrA+pRw7MSIuqNSm2lp8uaT9yjraF1je0UCLRpsMpNeIfVgxZVLqUApr6bJlTJ85i1FHfhmAXr16seGA/omjKpali99h0az5AKxY9gFvz32TAYM3SRtUJ6rj9LUPgYMjYgQwEjhM0ucrNai2Ij4DuDkbKxbwV+Bb1TSUtBUwtLyviJhSZb8Noe8JZ7L89utcDeeoecEiNtl4I3469irmzJ3HZ4ftxLljvkO/vn1Sh1ZIGw0ZyOBdhrJgxiupQ+k09RqaiIgAlma7vbKt4tmrqogjYkaW3YcDn4uI3SJiZnvtJF0OPAb8FDgn235c4fjRkqZJmnbTnMb4c2i9EfvQ9t47tL32cupQCq2ltZXZL83luK8ezsSbfkPfvn0Yd8vtqcMqpF791mfUtWN44KJbWLG0+/zh2xaqeivPVdk2uvxc2RPJM4DFwAMR8WSlvitWxJJ+uJbXAYiIq9r53r4CDIuIqgbzIqIJaAL4+ymHJBw6r17PnXal18gv0Gv43tCrN+rTj76jz2V502WpQyuUzTcbyOBBAxm+y84AHHrgflz/OyfieuuxXk9GXTuGWX94jDmTp6UOp1PVMmuiPFet5f1WYKSkjYE7Je0aEbPWdnx7QxMDqo5szeZRKssLe1flw4nj+HDiOAB6DhvB+od93Uk4BwM/tSmbbzaIV19rZruhQ3hi+gx22Hab1GEVzuFXfJu35y7gqevvTR1Kp8uj8ouIdyQ9AhxGaebZGlVMxBHxs3WM431ghqQHKUvGEfH9dTyvdUPn/eAM/vvPrmBly0q23nILLj7Pa1HV05A9P83wUfvz1uzXOW3SpQA8fOUEXnm43VHIQqjjrIlBwMosCfcFDgEur9Sm2gc6hgBXA/tS+sXxKHB2RDS30/RP2dYttM6ZyftzuscPbQo7f3oHbr/h16nDKKzmaS8xduiJqcNIpo6L+WxBaXJDT0r34W6PiLsrNah21sSNwK3A17P9k7LXvlSpUUTcnP1G2CYi5lTZl5lZp6vXhzhHxLPAbrW0qXZ0elBE3BgRLdl2EzCovUaSjgRmAJOz/ZGSuk2FbGaNI1DVW71Vm4iXSDopm5LRU9JJwNtVtLsQ2Bt4B0rT4IDtOhSpmVmOWkJVb/VWbSI+ldI6E4uAhcAxwClVtGtZw8JADTEtzcy6l5QVcbVjxBcD34yIvwFI2hT4OaUEXcksSf8M9JS0E/B94C8dDdbMLC/1GiPuiGor4uGrkjBARPyV6gajvwfsQmnq2q3A34ExtQZpZpa3RqiIe0jaZLWKuN22EfE+cL6kSyNiWXvHm5mlkrIirjYR/xvwF0kTKY3xHguMba+RpC8C1wP9gW0kjQBOj4gzOxivmVkuWnOodKtV7Ucl/VbSNEqfUSfgaxHxQhVNfwF8meyhjoiYKemAyk3MzDpfwk9KqroiJku81STf1du9sWqRoExrrecwM8tbW1eviNfBG9nwREjqTWnWxOyc+zQzq1nKebV5f1red4CzgK2AZkqr1Z+Vc59mZjVrq2Grt9wq4mzBi5MjovuuImJmDaNN6YYmcquIs4WRj87r/GZm9dRaw1ZveY8RPybpGmAC8NE84oh4Oud+zcxq0hCzJjroi9m/qxaYF6Ux8YNz7tfMrCZFnjVxN6XEu+o7DOBdSSOzldjMzLqElLMm8k7EewB7UnqgQ8DhwFTgdEl3RMQVOfdvZlaVIg9NfArYPSKWAki6AJgIHABMB5yIzaxLaIS1JjpqG2BF2f5KYGhELJdU2E92NrPG01rgivhW4AlJf8z2jwT+Q9IGdOBxaTOzvBS2Io6IiyVNAvajNEb8nYiYlr3tBz3MrMsobCIGiIjplMaDzcy6rBw+iq5quSdiM7NGUOiK2MysEaRcn9eJ2MyMYs8jNjNrCB6aMDNLLGUizntheDOzhhA1bJVI2lrSw5JmS3pe0tnt9e2K2MyMuo4RtwA/ioinJQ0Apkt6oNIHLjsRm5lRv1kTEbEQWJh9/Z6k2ZQ+Lq7xEvE1Dw1OHULhffec0alDKLwLFvqzcjvD+XU4R1sNC2FKGg2U/x+oKSKa1nDctsBuwJOVztdlE7GZWWeq5WZdlnQ/kXjLSeoP/B9gTES8W+lYJ2IzM+q7MLykXpSS8PiI+H17xzsRm5lRv+lrkgSMA2ZHxFXVtHEiNjMDWlS3mnhf4GTgOUmrPhLuvIiYtLYGTsRmZtRvaCIiHoXaPonUidjMDD/ibGaWXC3T1+rNidjMjPrOmqiVE7GZGR6aMDNLrtVDE2ZmabkiNjNLLFwRm5ml5YrYzCwxT18zM0vM09fMzBJrcUVsZpaWb9aZmSXmm3VmZom5IjYzS8wVsZlZYq3hitjMLCnPIzYzS8xjxGZmiXmM2MwsMQ9NmJkl5qEJM7PEPGvCzCwxD02YmSXmm3VmZol5jNjMLLGUQxM9kvVcEAO22JQTbzuf0x+8gtEPXM5ep3w5dUjFpR70v/Ba+p19SepICuvLhx7I87Om8OILj/KTc85KHU6nioiqt3pzRbyOorWNBy8Zz6JZ8+m9QR9OvfsSXn10FkteXpA6tMLp/aWv0rrwddSnX+pQCqlHjx78+ldjOeyfTqC5eSFPPD6Ju+6+n9mzX04dWqdorWNFLOkG4AhgcUTs2t7xrojX0dLF77Bo1nwAViz7gLfnvsmAwZukDaqAtMlAeo3YhxVTJqUOpbD23ms3XnllPq+++jorV67k9tv/yFFHdp+/8NqIqrcq3AQcVm3fTsR1tNGQgQzeZSgLZrySOpTC6XvCmSy//TpoS/nJYsW25Vab80bzmx/tNy9YyJZbbp4wos5Vz6GJiJgC/LXavnMZmpB0FxU+iy8ijsqj35R69VufUdeO4YGLbmHF0uWpwymU9UbsQ9t779D22sv0HDYidTiFJekTr+UxHtpVFXEe8c+zf78GbA78Lts/AZi/tkaSRgOjAY7edG/26r9jTuHVV4/1ejLq2jHM+sNjzJk8LXU4hdNzp13pNfIL9Bq+N/Tqjfr0o+/oc1nedFnq0AplQfNCth6y5Uf7Q7bagoUL30oYUeeqZfpaea7KNEVEU0f7ziURR8SfASRdHBEHlL11l6QpFdo1AU0AY4ee2DC/ig+/4tu8PXcBT11/b+pQCunDieP4cOI4AHoOG8H6h33dSTgHU6fNYMcdt2PbbbdmwYJFHHvs0Zz8je4zc6KWR5zLc1U95D1rYpCk7SNiHoCk7YBBOffZqYbs+WmGj9qft2a/zmmTLgXg4Ssn8MrDMxNHZlab1tZWzh7zUybdcys9e/Tgppsn8MILL6UOq9MUcWhilR8Aj0ial+1vC5yec5+dqnnaS4wdemLqMLqN1jkzeX+Of8nl5d7JD3Hv5IdSh5FEPROxpP8ADgQGSmoGLoiIcWs7PtdEHBGTJe0E7Jy99GJEfJhnn2ZmHVHPG5MRcUItx+eaiCX1A34IDI2Ib0vaSdKwiLg7z37NzGpV5EecbwRWAF/I9psBP59qZl1O1PBfveU9RrxDRBwn6QSAiFiuNU1WNDNLrDXSLYSZdyJeIakv2cMdknYAPEZsZl1OyodXckvEWeV7LTAZ2FrSeGBf4Ft59Wlm1lGFnL4WESHpbOBQ4POAgLMjYklefZqZdVSRF4Z/Atg+Iu7JuR8zs3XSVsShicxBwOmSXgOWUaqKIyKG59yvmVlNilwR/2PO5zczq4vCzpqIiNfyPL+ZWb0UeWjCzKwhFHlowsysIbgiNjNLzBWxmVlirdGarG8nYjMzCvqIs5lZIynkI85mZo3EFbGZWWKeNWFmlphnTZiZJVbYR5zNzBqFx4jNzBLzGLGZWWKuiM3MEvM8YjOzxFwRm5kl5lkTZmaJ+WadmVliKYcmeiTr2cysC4ka/muPpMMkzZE0V9K57R3vitjMjPpVxJJ6Ar8BvgQ0A1Ml/SkiXlhbGydiMzPqOka8NzA3IuYBSLoNOBpovER8/mvjlTqGWkkaHRFNqeMoska7xi03po6gdo12jeulZcWCqnOOpNHA6LKXmsqu2VbAG2XvNQP7VDqfx4jra3T7h9g68jXOn69xOyKiKSL2LNvKf3GtKaFXLLediM3M6qsZ2LpsfwjwZqUGTsRmZvU1FdhJ0naSegPHA3+q1KDLjhE3qG43rpaAr3H+fI3XQUS0SPoucB/QE7ghIp6v1EYpJzGbmZmHJszMknMiNjNLzInYrOAkbSzpzDqd60BJd9fjXPYxJ+JOIMk3RS2ljYFPJOLsUVzrApyIayTpG5KelTRT0i2SjpT0pKRnJP2npMHZcRdKapJ0P/DbxGE3DEkbSLonu76zJB0nab6kgdn7e0p6RFKP7PWNy9rOXXX97f9zGbCDpBmSpkp6WNKtwHOStpU0a9WBkn4s6cLs6x2zn+mZkp6WtEP5SSXtlf3cb9+p300BuVKrgaRdgPOBfSNiiaRNKT0x8/mICEmnAT8BfpQ12QPYLyKWp4m4IR0GvBkRhwNI2gi4fPWDIqJN0h+BrwI3StoHmB8Rb3VqtI3hXGDXiBgp6UDgnmz/VUnbVmg3HrgsIu6U1IdS4bY1gKQvAlcDR0fE63kG3x24Iq7NwcDEiFgCEBF/pfTUzH2SngPOAXYpO/5PTsI1ew44RNLlkvaPiL9XOHYCcFz29fHZvrXvqYh4tdIBkgYAW0XEnQAR8UFEvJ+9/RlKc42PdBKuDyfi2ohPPjN+NXBNRHwOOB3oU/bess4KrCgi4iVKf0k8B/yrpH8BWvj4Z7X8+j4O7ChpEPAV4PedGWsDK/+5LL+28PH1rbQAzkLgA2C3OsfVbTkR1+ZB4FhJnwLIhiY2AhZk738zVWBFIWlL4P2I+B3wc2B3YD6l5AwwatWxUXoa6U7gKmB2RLzdudE2jPeAAWt57y1gM0mfkrQ+cARARLwLNEv6CoCk9SX1y9q8AxwOXJoNddg68hhxDSLieUljgT9LagWeAS4E7pC0AHgC2C5hiEXwOeBKSW3ASuAMoC8wTtJ5wJOrHT+B0rP93+rMIBtJRLwt6bHsptxySsl31XsrJV1E6bq+CrxY1vRk4H9n768Evl7W7i1JRwL3Sjo1Ilb/38Vq4EeczcwS89CEmVliTsRmZok5EZuZJeZEbGaWmBOxmVliTsSWq9XXMljtvYskHVKhbVUrfUnaW9IUSXMkvSjpekn9JH1L0jXrEr9ZZ/A8YksmIv5lXc+RLfJzB3B8RDwuSZQe+ljbAwxmXY4rYqtZtg7EmWX7F0r6kaRzstW9npX0s7ImPSVdJ+l5SfdL6pu1u0nSMdnXe0n6S7bS11PZWgflfW4g6Ybs/M9IOjp76yzg5oh4HEpP20XExNUX/6mwSt5/y1Ylm5G9N0DSFlmFPSNbAW7/ul9EszJOxNYRt/HxYjsAxwL/BewE7A2MBPaQdED2/k7AbyJiF0qPx44qa4tKn3Q7ATg7IkYAh1B6Aqzc+cBDEbEXcBClp+82AHYFplcR86OUVsnbLYv/J9nrPwbOioiRwP5Zv/8M3Je9NgKYUcX5zTrMQxNWs4h4RtJm2boQg4C/AcOBQyk99g3Qn1ICfh14NSJWJbPpwLarnXIYsDAipmbnfxegNMrwkUOBoyT9ONvvA2xTQ9hDgAmStgB6U3qcF+Ax4CpJ44HfR0SzpKnADZJ6AX8oi90sF66IraMmAsdQqoxvo7Ra179GxMhs2zEixmXHfljWrpVPFgBrWtVudQJGlZ1/m4iYDTzPxwsCVbLGVfIi4jLgNErrWTwhaeeImAIcQGkxp1skfaOK85t1mBOxddRtlNYAPoZSUr4POFVSfwBJW0narMpzvQhsKWmvrO0AffLjpe4DvpfdjEPSqiUYrwG+mS0MT/beSZI2X639GlfJk7RDRDwXEZcD04CdJQ0FFkfEdcA4SivAmeXGQxPWIdlKdAOABRGxEFgo6TPA41muXAqcRKkCbu9cKyQdB1yd3chbTmmcuNzFwC+BZ7NkPB84IlsF7Hjg51nibwOm8Mm1iS9kzavkjZF0UBbnC8C9lH7BnCNpZfZ9uCK2XHn1NTOzxDw0YWaWmBOxmVliTsRmZok5EZuZJeZEbGaWmBOxmVliTsRmZon9P23qIuzLE5UeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.heatmap(ct,annot=True) # shift+tab, can see the definition of the function, put in the ("
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Titanic data\n",
    "\n",
    "One of the more popular datasets that we use for experimenting with crosstabs is the \n",
    "survivor data from the Titanic disaster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>passtype</th>\n",
       "      <th>age</th>\n",
       "      <th>status</th>\n",
       "      <th>sex</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>third</td>\n",
       "      <td>child</td>\n",
       "      <td>dead</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>third</td>\n",
       "      <td>child</td>\n",
       "      <td>dead</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>third</td>\n",
       "      <td>child</td>\n",
       "      <td>dead</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>third</td>\n",
       "      <td>child</td>\n",
       "      <td>dead</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>third</td>\n",
       "      <td>child</td>\n",
       "      <td>dead</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  passtype    age status   sex\n",
       "0    third  child   dead  male\n",
       "1    third  child   dead  male\n",
       "2    third  child   dead  male\n",
       "3    third  child   dead  male\n",
       "4    third  child   dead  male"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic = pd.read_csv('data/titanic.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a crosstab of the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>status</th>\n",
       "      <th>alive</th>\n",
       "      <th>dead</th>\n",
       "      <th>All</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>passtype</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>crew</th>\n",
       "      <td>212</td>\n",
       "      <td>673</td>\n",
       "      <td>885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>first</th>\n",
       "      <td>203</td>\n",
       "      <td>122</td>\n",
       "      <td>325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>second</th>\n",
       "      <td>118</td>\n",
       "      <td>167</td>\n",
       "      <td>285</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>third</th>\n",
       "      <td>178</td>\n",
       "      <td>528</td>\n",
       "      <td>706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>All</th>\n",
       "      <td>711</td>\n",
       "      <td>1490</td>\n",
       "      <td>2201</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "status    alive  dead   All\n",
       "passtype                   \n",
       "crew        212   673   885\n",
       "first       203   122   325\n",
       "second      118   167   285\n",
       "third       178   528   706\n",
       "All         711  1490  2201"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ct = pd.crosstab(titanic.passtype,titanic.status,margins=True)\n",
    "ct"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's use our knowledge of data manipulation with pandas to generate some percentages totals:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q2: Generate this:</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/samplect.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">len</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">age</th>\n",
       "      <th colspan=\"2\" halign=\"left\">sex</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>status</th>\n",
       "      <th>alive</th>\n",
       "      <th>dead</th>\n",
       "      <th>alive</th>\n",
       "      <th>dead</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>passtype</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>crew</th>\n",
       "      <td>212</td>\n",
       "      <td>673</td>\n",
       "      <td>212</td>\n",
       "      <td>673</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>first</th>\n",
       "      <td>203</td>\n",
       "      <td>122</td>\n",
       "      <td>203</td>\n",
       "      <td>122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>second</th>\n",
       "      <td>118</td>\n",
       "      <td>167</td>\n",
       "      <td>118</td>\n",
       "      <td>167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>third</th>\n",
       "      <td>178</td>\n",
       "      <td>528</td>\n",
       "      <td>178</td>\n",
       "      <td>528</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           len                \n",
       "           age        sex     \n",
       "status   alive dead alive dead\n",
       "passtype                      \n",
       "crew       212  673   212  673\n",
       "first      203  122   203  122\n",
       "second     118  167   118  167\n",
       "third      178  528   178  528"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.pivot_table(data = titanic, columns = 'status', index = 'passtype', aggfunc = [len, ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>status</th>\n",
       "      <th>alive</th>\n",
       "      <th>dead</th>\n",
       "      <th>All</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>passtype</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>crew</th>\n",
       "      <td>212 (23.95%)</td>\n",
       "      <td>673 (76.05%)</td>\n",
       "      <td>885 (100.0%)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>first</th>\n",
       "      <td>203 (62.46%)</td>\n",
       "      <td>122 (37.54%)</td>\n",
       "      <td>325 (100.0%)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>second</th>\n",
       "      <td>118 (41.4%)</td>\n",
       "      <td>167 (58.6%)</td>\n",
       "      <td>285 (100.0%)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>third</th>\n",
       "      <td>178 (25.21%)</td>\n",
       "      <td>528 (74.79%)</td>\n",
       "      <td>706 (100.0%)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>All</th>\n",
       "      <td>711 (32.3%)</td>\n",
       "      <td>1490 (67.7%)</td>\n",
       "      <td>2201 (100.0%)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "status           alive          dead            All\n",
       "passtype                                           \n",
       "crew      212 (23.95%)  673 (76.05%)   885 (100.0%)\n",
       "first     203 (62.46%)  122 (37.54%)   325 (100.0%)\n",
       "second     118 (41.4%)   167 (58.6%)   285 (100.0%)\n",
       "third     178 (25.21%)  528 (74.79%)   706 (100.0%)\n",
       "All        711 (32.3%)  1490 (67.7%)  2201 (100.0%)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table = pd.crosstab(titanic.passtype,titanic.status,margins=True)\n",
    "table.alive = np.round(table.alive / table.All * 100,decimals=2) \n",
    "table.dead = np.round(table.dead / table.All * 100,decimals=2)  \n",
    "table.All  = np.round(table.All / table.All * 100,decimals=2)   \n",
    "table.alive = ct.alive.astype('str') + \" (\" + table.alive.astype('str') + \"%)\"\n",
    "table.dead = ct.dead.astype('str') + \" (\" + table.dead.astype('str') + \"%)\"\n",
    "table.All = ct.All.astype('str') + \" (\" + table.All.astype('str') + \"%)\"\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q3: Is this what we would have expected?</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'ctExt' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-27-f41128aeb6ed>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# fill in the correct numbers on the next two lines (where np.NaN is right now)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mexpectedAlive\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mctExt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtotal\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0;36m.323\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mexpectedDead\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mctExt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtotal\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0;36m.677\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'ctExt' is not defined"
     ]
    }
   ],
   "source": [
    "# fill in the correct numbers on the next two lines (where np.NaN is right now)\n",
    "expectedAlive = ctExt.total * .323\n",
    "expectedDead = ctExt.total * .677"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now generate a similar matrix for the *expected* (as oppposed to observed) values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ctExpected = ct.copy()\n",
    "ctExpected.alive = expectedAlive\n",
    "ctExpected.dead = expectedDead\n",
    "ctExpected['total'] = ctExpected.sum(axis=1)\n",
    "ctExpected.loc['total'] = ctExpected.sum(axis=0)\n",
    "alivePercent = np.round(ctExpected.alive/ctExpected.total * 100,decimals=2)\n",
    "deadPercent = np.round(ctExpected.dead/ctExpected.total * 100,decimals=2)\n",
    "totalPercent = np.round(ctExpected.total/ctExpected.total * 100,decimals=2)\n",
    "detailExp = ctExpected.copy()\n",
    "detailExp.alive = ctExpected.alive.astype('str') + \" (\" + alivePercent.astype('str') + \"%)\"\n",
    "detailExp.dead = ctExpected.dead.astype('str') + \" (\" + deadPercent.astype('str') + \"%)\"\n",
    "detailExp.total = ctExpected.total.astype('str') + \" (\" + totalPercent.astype('str') + \"%)\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "detailExp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "detailCT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, there we have the expected and observed values, along with their proportions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to the heatmap shown above, we can use a mosaic plot to visualize \n",
    "contingency tables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xt81PWd7/HXZ2aSTCb3+4VLCCC3gAEFFRTUFnS1drvarlrbtWpvu55tH6enffTso9uz1t3Tbrfd03PW9ezWXrzUbi1V9Kx3q1UsiMhFgkDAK5JIEsiFkCE3kpnv+WNGSCARAiEz4fd+Ph558Mvv9/395vP7MZn3/L6/33zHnHOIiIj3+BJdgIiIJIYCQETEoxQAIiIepQAQEfEoBYCIiEcpAEREPEoBIEnLzCab2SEz85+h7e8ws8vOxLZHQ3zfp47CdpyZTR+NmuTsogCQpGFm75vZ8g9/d87VOecynXOR+PLVZval0Xo851yVc271aG1vtMX3/b1E1yFnLwWASAKYWSDRNYgoACQpmNmDwGTgiXjXx7fNbEq8+yJgZt8HlgJ3x5ffHV/vX8ys3sw6zGyzmS0dsM3vmdnvzOxXZhaOd/ksHLD8yBmHmfnN7Dtm9m687WYzmzREnUEz+7WZtZpZu5ltNLOSY7c34PF/HZ/+cF++aGZ1wItm9qyZ/fUx299qZtfFp52ZTTezi8ysaWBXmJlda2ZvxKcvMLNX4/U0mtndZpZ62v8pctZTAEhScM79BVAHfDLe9fGjY5b/LbAG+Ov48g9fODcC84F84DfAw2YWHLDqnwK/BXKBx4G7hynhvwGfBa4GsoHbgK4h2n0ByAEmAQXAXwLdI9jVS4HZwJXxej/74QIzmwNUAE8NXME5tx7oBD42YPZN8fUBIsA3gEJgMfBx4PYR1CQepQCQcc0592vnXKtzrt8597+ANGDmgCZrnXNPx68jPAhUD7OpLwHfdc696WK2Oudah2jXR+yFf7pzLuKc2+yc6xhByd9zznU657qBx4D5ZlYRX/Y54FHnXO8Q6z1EPCzMLItYUD0UPwabnXPr48fgfeAeYkEj8pEUADKumdk3zWynmR00s3Zi784LBzRpGjDdBQSH6X+fBLx7Eg/5IPAc8FszazCzH5lZyghKrv9wwjkXJvZu/8b4rBuB/xhmvd8A15lZGnAd8Lpzbg+Amc0wsyfj3UQdwA8YfAxEhqQAkGRyoqFpBy2P9/f/d+B6IM85lwscBOwUHrsemHbCAp3rc87d6ZybAywBrgFuji/uBEIDmpcOtYljfn8I+KyZLQbSgZeGedxaYA9wFYO7fwD+HdgFnOOcywa+w6kdA/EYBYAkk33AR933fuzyLKAfaAYCZvZ3xPrvT8UvgH8ws3Ms5lwzKzi2kZldbmbz4hdkO4h1CUXii2uAG80sJX6x+TMn8bhPE+v3/3tgpXMu+hFtfwN8HVgGPDxgfla8lkNmNgv4q5N4XBEFgCSVfwS+G7+b5VtDLP8X4DNmdsDM7iLWFfMM8Baxd8c9DOhiGaGfAL8Dfk/sxfSXxN6RH6sUeCTeZifwMvDr+LL/Qews4gBwJ4PfpQ8p3t//KLD8JNo/BFwGvOicaxkw/1vEzgrCwM+BlSd6XBEA0xfCiIh4k84AREQ8SgEgIuJRCgAREY9SAIiIeFRSD0hlZk43Myc/88XeR0Q/8g7G5Oez8bcfPvMRjepGDjmWw7kTv3wmdwAAd0yZkugy5ATu2L0bgK/dWZngSk7Pv94R24/KO69McCUnb/cdz1F55Z2JLkOSzPu//95JvYtRF5CIiEcpAEREPEoBICLiUQoAERGPUgCIiHiUAkBExKMUACIiHqUAEBHxKAWAiIhHKQBERDxKASAi4lEKABERj1IAiIh4lAJARMSjFAAiIh6lABAR8SgFgIiMW845nNM3op2qpP5GsPHkQF8frf39TE9P556GBr5aXj5o+Zr2duZmZJCXkjLsNh7ev5/leXkf2UbEi5xztO16msPhfWCGL5BGIC2bvq5WiqqvJ1y/ke7Wd8E5CmZ/AswI122gYM41NL72C7IrLiK9cAbNbzxCyXk3JXp3koYCYJS09/fzbnc309PTh1y+NDf3uHlR5/CZvvVY5ES6m98E81F2wW0ANG97lNTscgrmXMPh8D76OlspW3Qr/b1h2nY+RfH8G+nrbCEa6cOXGqKnvQ5fSjppuRMTvCfJRQEwSjaFw9T39tLQ20t3NMqq5maa+/pYnJ1NdWYmjzU3syQnh65IhHUdHRgwJyODTL+fFw4cIC8QoDs6fr6MXGQs9XW2EMydfHSGGWk55fFlzfQerKdx433xRbGebV9KOt3NbxEqmklP2256DuwhvWDamNeezBQAo2RhVhbZgQBX5ufzw7o6PlFQgAEP7ttHdWbmoLa90Si3lpZiZvy8oYGbS0pI9fm4e+/exBQvkuRSMgrpaXufjNKq2AwHYEeWBfMqKKz6VGxRNAJAWt5kDu5eS+G513G4o5Hu1nfJnbosAdUnLwXAGZAXCBD0xd6FDHWBqjwtDYt3/Tgg5PcDUJqaOmY1iown6UUz6W55h8YNv8TMj/mP/q2kZpWSEiqgceN9GEawYCq5U5cRzKvg4HtrSM0oIi1vMocP7cd8eskbSEdjlPjNONl7EeyY6e5IhBSfj32HD5+BykTGPzOjYM41wy7PqbyEnMpLBs1Lyy5n8uXfBiCz7Fwyy849ozWOR6d1G6iZvW9mhfHpdaNT0vhUnJpKQ28vK/fvH9F6l+fl8cC+fTzS3Ex2/ExARGQsjNoZgHNuyWhtazwK+nzcVlZ23Pwvx28Hvbao6Mi8ygF3Ck1PTx/2ziERkTPppM8AzOz/mdlmM9thZl8ZYvmh+L8rzezqAfPvN7NPm5nfzH5sZhvN7A0z++ro7IKIiJyKkXQB3eacOx9YCHzdzAqGafdb4AYAM0sFPg48DXwROOicWwQsAr5sZpXHrmxmXzGzTWa2SZ/vE5Gh9HUfYH/NSgBadz1DNNKX4IrGp5EEwNfNbCuwHpgEnDNMu2eAj5lZGnAV8EfnXDdwBXCzmdUArwEFQ23DOfcz59xC59zC8f4RqQN9fUeuCTzT2kqf7vMXGXUFs67C59en50/FSV0DMLPLgOXAYudcl5mtBoJDtXXO9cSXX0nsTOChDzcDfM0599xp1jwuXVUw3AmTiJxIpPcQzdtW4aIR/KkZ5M1YcWRZ48b7KFlwE83bVlEw55ME0rLoqN+ImZ9Q8UxadjyO6+/FH8yicO61Rz4oJid/BpADHIi/+M8CLjpB+98CtwJLgQ9f8J8D/srMUgDMbIaZZZxCzUnrUCTCA01N3NvYyMr9+wfdFnpfYyO90Si/2bePcH8/ABs7Ong9HKYzEuGhffu4v6mJVc3NRDW4lcggvpQgJed9nrILbiOQnkNP2+7j2mSUVNG1rxaArv27CJXM5uDutWRPvpDSRbeQmlVG176dY116UjvZu4CeBf7SzN4A3iTWDfRRfg/8CnjcOffhze2/AKYAr1vsU1DNwJ+NuOIkFvT5+HxJCX4znm1rY3dPz3FtqjIyqO3q4sLsbHZ1dfGZoiL+ePAgF2ZnMzU9nXUHD7Kzq4uqjLMqG0VOS6Svm9baJ4n29xDpDZMZOL4DIlQ8k/01K8koqcJ8Afwp6fFhIvbS/t7LuEg/meX6LMBAJxUAzrleYv35x5oyoE3mgOk+Yn38A7cRBb4T/zkrdUciPNnaSk80SjgSIXjMEBAAM0MhVu7fT1VGBgEz0v1+mvv62Nvby8vt7fQ7x7lDrCfiZZ2N20gvmEb25Atoe/NZ/MEcCO8b1MYXCOILBOmoW39kyIiUjEJCxbMJ5lUAR4eJkBh9EngUbevsZFp6OhdkZ/NsWxs5fj/7jmkT9PkI+nys7+g48i6/MCWF2aEQFcHYu5qIuoBEBgnmV9Ky/TG6W9/F508hJbNkyHahkjm01j7BpEu/CUBO5TJaax+n/Z2XAMibsYK0nAljVneyUwCMospgkMdaWni3u5sUn4+SYcb1nxMK8URrK9+cNAmAZTk5PN7aykvt7QCsyMtjQlramNUtkuzSssuYsOT2QfOyJiwAoGzRrUfmZZbNI7Ns3pHf/akhiuffODZFjkMKgFFUlpbG7RMGv7tYkJUFwK0DPiU8LzOTeQO6eUJ+PzcWF49NkTJudGysxwI+shboHaucGbofSkTEo3QGIDICPfXttD2zC0vxE6zII21SLgfXvAdRR9aFk8mcV0ZfaxetT9bioo60CdnkXzGTg6+8T9fO2BWh/Ktnk1aeTcNPXyVtYg69jR2EZhWTu3Qq/e3dNK/ahi8tgC8YIDhNnx+RM0cBIDIC3W+3kHvpNEIzi3BRR9O9Gyi9ZRH4jKYHNpFRVcqB598i74oZpJVl46KO/nAvXW/up/S2C+hv76b1iVpKv7CQaE8/OZdU4s8O0nDPq+QuncrBV94n99KppE8vpHnVG4neXTnLKQBERiBr0SQOrt1N544m0qcW0NfWRdODmwGIdh0m0nmY/o4e0sqyATCf0d/eTWppFuYzUvJDRHtjHwT0pQcI5MZGgrVAbCjwvrYuUstj66ZNyBnr3ROPUQCIjIAvGKDgqlm4/igNP1tPSmEGpTefj/l9uEgU8/sIZAc53BQmtTQLF3UEctM53BSOnQ20d+MLfvhnd/xoVyn5IQ43hkmfVkBvQwfByvyx3UHxFAWAyAiEN31A1859uL4ImfPLSS3OpOlXmzEDXyiV4uuryVsxg9YnduAcpE3MIX/FDEIzi2m6dwMQuwYwnJyLp9C8ahsH172PL6gBzuTMUgCIjEDO4gpyFlcMmpc+vXDQ7ykFodh1gYHrXTyFnIunDJpX/tWjQ2qVf/lCAAK56ZR98YJRrFhkeLoNVETEoxQAIiIepQAQEfEoBYDIKOs70M3+lTUAtD6zi2ifRqCU5KSLwCJnUMFVsxJdgsiwFAAipylyqJfmVdtwkSj+jFTyVsw4sqzxvo2U3LSA5lXbKPjkHAJZabFB3vxGaGYxLY/vwPX2488KUnjtXMw33r8JW8YTdQGJnCZfMIWSz59H2W0XEMhJp2d323FtMqpK6KqNjQXUtWs/odklHFy7m+wLJ1N6yyJSy7KOjBUkMlZ0BiBymiLdfbQ+WUu0p59IuJfM4PF/VqGZxexfWUNGVQkW8OFPT6GvuZPevQdpf/k9XH+EzHPLE1C9eJkCQOQ0dW5rJH1aAdkXTKbt2Tfx5wRhX3hQG18wNrpnx/o6MqpKAUgpzCA0u5hgRR4ALhI9pcff/dwdp7cDctYx+95JtVMAiJymYGU+LY9tp/vdVnwpflJKhv5O59CcElqfqGXSNy8FIGdZJa2P19L+0jsA5K2YcUoDwF1+052nXrx4mgJA5DSllWUz4fYlg+Z9+C1eZbceHRIic14ZmfOOfjOcP5RK8Y3zx6ZIkSEkdQCYz8cdu3cnugwRkbNSUgeAi0a58/LLE12GnMAdL72U6BJE5BToNlAREY9SAIiIeJQCQJKGc44nf7OPe36wh01/bD+pdbZvCp+4kYgMKamvAYi3dB2KYAZf/U7Fcctc1A05TMKOzWHmLswai/JEzjoKAEkaa55po7G+l1/8qI5Z1ZlccmU+K3/aQOmkNA73Rpk2O8SmNQcJpBjnzM0AoL21j0fva+TCy/OYMCWY4D0QGV8UAJI0lqzI45XfH2DeBVm8/1Y3AD09Uaovyia3IIXnH23m458qpKAk9cgZQe3rh7ju1rITbFlEhqIAkKQWDPrILYh9OfrCZblsXd9B3+Eo8xZlUTpJ7/hFTocuAktyG9Dtn5Xj57JrCli8PI9X/xC7SGwaPVnklOkMQMaNDavbaarvpe+wY/6SbAByC1J4euV+zluSrTMCkRFSAEjSyM5L4aobigGYWJkOwA1fPTpE8pIV+cetc8Wni8amOJGzkLqAREQ8SgEgIuJRCgAREY9SAEhS6jjQxzMr9wOw5plW+vtO7duyRGR4uggsSW/pVQWJLkHkrKQAkKTQdSjC71c1E4040jP8LFmRd2TZo/c1cs1NJfx+VTOXf7KAjKwA2zd24PMblTNDvPh4C4d7HRlZfpZfW4hviDGDROR46gKSpJAW9PGnny/hutvKyMwJ8MHunuPaTK/K4J3aLgDe29XF1NkhNq89SPWF2Vx7SylFZam8t7NrrEsXGbd0BiBJoac7wuonW+ntidIZjpAWPP6L1afODPH0yv2cU5WBP2AE0/0caO5j395eNrzcTqTfMfPcob+QXUSOpwCQpPDWtk4mT0tn3gXZrHm2jawcP637BrdJDfpIDfqoWd/B9KrYaKC5hSlMmx2ivCL2KeBIxI116SLjlrqAJClMrAyybVOYpx7aR9ehyLDtps8JsW1DB5UzQwAsXJZDzasdPHZ/E4/d30RL0+GxKllk3NMZgCSForI0brp9wqB5sxfEvuhl4HDPM+ZlMmPe0W6e9JCfq28sHpsiRc4yOgMQEfEoBYCIiEcpAEREPEoBICLiUQoAERGPUgCIiHiUAkBExKMUACIiHqUAEBHxKAWAiIhHKQBERDxKASAi4lEaDE7kGLvveC7RJYiMCQWAyDEuv/OmRJdw0l664zeJLkHGMXUBiYh4lAJARMSjFAAiIh6lABAR8SgFgIiIRykAREQ8SgEgIuJRCgAREY9SAIiIeJQCQETEoxQAIiIepQAQEfEoBYCIiEcpAEREPEoBICLiUQoAERGPUgCIiHiUAkBExKMUACIiHqUAEBHxKAWAiIhHKQBERDxKASAi4lEKABERj1IAiIh4lAJARMSjFAAiIh6lABAR8SgFgIiIRykAREQ8SgEgIuJRCgAREY9SAIiIeJQCQETEoxQAIiIepQAQEfEoBcAoOdDdzTttbQDcs2nTccvX7NnDge7uj9zGwzt2nLCNiBzlnMM5l+gyxq1Aogs4W7T39PBuWxvT8/OHXL60ouK4eVHn8Jmd6dJExj3nHG9vfprO9n2AEUhNIy09m65wK1WXXE/D2xtpa3wXcJyz8BOYGXvf2sCMRdfw+vO/YOLMiygon0HtK48w79KbEr07SUMBMEo2NTRQ39FBQzhMd38/q2prae7qYvHEiVSXlvLYzp0smTSJrr4+1tXXY2bMKSoiMzWVF957j7xgkO7+/kTvhkhSat37JmY+Fiy/DYCdrz5KVn45MxZdw6H2fXR1tLJg+a30dod5e9NTzF16I10dLUT6+0hJC3GwuY5AajrZhRMTvCfJRQEwShaWl5OdlsaV06fzw7Vr+cSMGRjw4BtvUF1aOqhtbyTCrfPnY2b8fPNmbq6uJtXv5+4NGxJTvEiS6+poIado8tEZZmTll8eXNdPRUs+WP9wXXxTr2Q6kpdPa8BYF5TNp37ebg/v3kFc6bcxrT2YKgDMgLxgkGIgd2qH6J8uzsrB4148DQikpAJRmZo5ZjSLjSSi7kPb971M8uSo2wwHxv6FQViE5xRXMuvBTAESjEQByCidTV7uW2Uuu49CBRtqa3qWialkiyk9aCoBR4vf5ONlLUXbMdHdfHyl+P/sOHToDlYmMfwUTZtLW+A6vP/9LfD4//kDqkWWZeaWEsgrY8sJ9mBl5pVOpqFpGbnEFdbVryMguIqdoMp0H9+Pz6yVvoBEfDTP7HnDIOffPp/PAZvY+sNA513I620kWxRkZvPDee6zcvn1E611eWckDW7eSGwySnZZ2hqoTGd/MjBmLrhl2+eQ5lzB5ziWD5mXll3Pxdd8GoGTKuZRMOfeM1jgeKQ5HSTAQ4LYFC46b/+Xzzwfg2tmzj8yrzMs7Mj09P3/YO4dERM6kkwoAM/tb4GagHmgGNpvZNOD/AkVAF/Bl59wuM/sk8F0gFWgFPuec22dmBcBD8fYbGNwTImeBNKYkugQRGYETfhDMzM4HbgQWANcBi+KLfgZ8zTl3PvAt4N/i89cCFznnFgC/Bb4dn38HsDY+/3FgwCX9QY/3FTPbZGab9PEOETlWZ0czO9c/dkrr7n17I43vbRnlisavkzkDWAo85pzrAjCzx4EgsAR42I5+kOnDDuyJwEozKyN2FrA7Pn8ZsQDBOfeUmR0Y6sGccz8jFi74zMZlBjR3drK2rm5Qt8/J2rh3LwGfjwVlZWegMhGRo072GsCxL8Q+oN05N3+Itv8K/MQ597iZXQZ87yO2IyJyQtFohNp1j9B/uIeM7CIAWhvepq52Dc5FmXDOhZRMmUfT7q007a6h/3A3E2cuprSymp7OdmrXrSKQkkYgNajPAgxwMgHwR+B+M/thvP0ngXuA3Wb25865hy12GnCuc24rkAPsja/7hWO28zngf5rZVUAeZ5FINMojtbX09PdTlJEBwNutraypqyPqHBdOmMC8khK2NjVR09REd3//kU8Jt/f0sKq2lrRAgGAgwLS8s+rQiJy2lg92EcoqYGr1chre2czB5j3s2fEy8z92C+bzUfOHByieXEXRpNmUVlYTjfSz5YV7Ka2spm7nK0yZeyn5ZdOpXbcq0buSVE4YAM65181sJVAD7AHWxBd9Dvh3M/sukEKsv38rsXf8D5vZXmA9UBlvfyfwkJm9DrwM1I3ifiTcrpYWCkIhlk+dyuaGBvYcPMjLe/Zwy/z5+Mx4oKaGquJiZhcVUV1aSn80yr1btlBdWsordXVcOmUK0/PzWVVbm+hdEUk63YfajnzyN7tgAi0f7KQ73MbW1Q8C0NfbxeHeTsKte6nf9SoAXeHW2LrhwevKUSfVBeSc+z7w/SEW/ckQbf8T+M8h5rcCVwyY9Y2TrHFcaOvupjwrC4AJ2dnsbGmhrbubB7duBaCrr4/Ow4fZGw7zan09AK1dXUOuKyKDpWfmEz7QSNGkOYTbGkhJCxHKLqT68pvx+fxEoxF8Pj/btq+OnRX4/bz2xF2xdbNi6+aXTiPc1kBuSeUJHs079DmAUZKfnk5jOMycoiIawmFCKSkUhkLcXF2N3+cjEo3i9/lYvW0bt8yfj9+Mu157bdC60/LzaQiHqczNTfDeiCSXwomz2F+3nZoXHyCUXQhmVFQtY+uLv8LMSEkLUXXJ9RRNmkPNi/eTmVdGIDUIwOTZF1O7bhX1O9cdmScxCoBRMquwkO379/NATQ2FoRAGLKuo4Fdbt2JmhFJSuL6qijlFRdxfU0NZZuaR8YIunjyZVbW1rKuvPzJPRI7y+fzMveSG4+bnl00f9HtF1bLjxvsJZuRy3oovntH6xiu92owSv8/HDXPnHjf/2E/5LquoYNkx3w2QGwzyxfPOO6P1yfizd+Pb+AJ+yhZMTXQpcpbSN4KJiHiUzgBERuBgfQvvPLMZX4qf3IpisicVUremFhd1TLjwHErmTaGrNcxbT27ERaNkTyhg2hULqHtlJy076wHjnKvPJ6s8n00/fZbsiQWEGw9QOGsCFUur6GnvpHbVOgJpKQSCqeRNKz1hTSKnSgEgMgJtbzdQcelcCmdOwEUdW+59gfm3fCx2L/oDf6C4ajLvPV/DtCvmk1WWj4s6esPdtL65lwW3raCnvZM3n9jA/C98jP6ew0y+ZA5p2SE23fMsFUurqHtlJ1MunUv+9DJqV61L9O7KWU4BIDIC5YvOoW5tLc076sibWkp3W5itD64GoK+rl8OdvfR2dJFVFrv2Yz6jp72TzNJczGek52cS6e0DIJCeSjA39qFBX8APQHdbmKzy2LrZEwrGeO/EaxQAIiMQCKZwzlXnE+2PsPlnzxEqzKb65svx+X1EI1F8fh9p2SEONR0gszQPF3UEczM41NSOizp62jsJBFPiWzt+QNz0/CzCjQfIn1ZKuKGN3MqSsd1B8RQFgMgINGx6h5adHxDp66d0fiUZxbls/dWLsXvRQ2lUXX8JU1fM560nNuAcZE8sYNqK+RTMnMCWe5/nw2sAw5l88WxqV62jft1OAsHUYduJjAYFgMgITFo8i0mLZw2alz998MitoYIs5t/y8UHzJl88m8kXDx4dduFXrzwyff6XYx+SD+ZmcN4XV4xmySLD0m2gIiIepQAQEfEoBYCIiEcpAERGSWdzBzsfW39K6+7d+DaNW94b5YpEPpoCQETEo3QXkMhpiEai1D6yjv6ew2QUxb7LofXthuOGh2jaupummt30dx9m4uKZlFZXatgHSTgFgMhpaNn1AaGCLKYur6Zh8zsc3NPMnpd3HDc8RNHsSZRWVxLtj7Dl3hcora7UsA+ScAoAkdPQ3XZo0NANLTs/GHJ4iPDeVupf3QVAV2s4vq6GfZDEUgCInIb0/EzCjQcomjOJcEMbKaG0IYeH2LZ6e+yswO/ntbueiK+rYR8ksRQAIqehcNZE9m+vo+aBFwkVZse+qnBZ1XHDQxTNmUTN/S+SWZZ3ZIgHDfsgiaYAEDkNPr+PuTdcctz8Y4eHqFhWRcWyqkHzNOyDJJpuAxUR8SgFgIiIRykAREQ8SgEgIuJRCgBJGs45HvvNNu76wRrW/3HPSa3zxqaGM1yVyNlLdwFJ0ug8dBgz+Pp3lh63zEUd5jv+KxTf2NzAuQvLx6I8kbOOAkCSxkvPvENDfQf/9qNXmFNdwmVXTufBn26ifFI2vb0RzpldyGtr6khJ8TFzbjEAba3drLxvC0sur2TSlNwE74HI+KIAkKSxbMVUXv79uyy4YALvvtUKQE9PP+ddNJG8ghBPP7qTP/nUTApLMo+cEWx/vZEbbl2Q4MpFxicFgCS1YDBAXkEIgIuWVfD6+g/oOxyhelE55ZNyElydyPimi8CS1GxAt39WThrLr5nBJcunsvYPu49vICIjojMAGTdeXf0+DfUd9B2OsHDJJADyC9J5fOV2Fi6ZpDMCkRFSAEjSyMlL509vmAvApMo8AD7/1YVHli9bMe24da7+9JyxKU7kLKQuIBERj1IAiIh4lAJARMSjFACSVFqbO3n2sZ2ntO7WjXvZvqVxlCsSOXspAEREPEp3AUnCRSNRnnyklt6efvKLMgDY/XYrr62pIxp1LLhwArPnlVC7tYkdNU30dPdz/uKJzKkupaO9h6dW1ZKWFiAtGKBiWl6C90Zk/FAASMK9s6uFvIIQS5dP5Y3NDezdc5BXX97DDbfMx+czfvdADTOrijlndhFzqkvp74/y23th1e61AAAGVUlEQVS3MKe6lI2v1LH40ilMmZ7PU6tqE70rIuOKAkAS7kBbNyXlWQCUTsjmnZ0ttLd188iDWwHo7uqju/MwjXvDbH61PrZOaxcA7QPWLZuQnYDqRcYvBYAkXF5+Ovsbw8yYU8S+hjDpoRTyC0N85uZq/H4fkUgUv9/Hq6u3cf0t8/H7jV/e9RoAufF1K6bl09QQZnKlRgQVOVkKAEm46bMK2bV9Pw8/UENeYQiz2MBvj/xqK2ZGeiiFT15fxTlzivjd/TUUl2WSFow9dRddPJmnVtWyaV39kXkicnL0FyMJ5/P7jgwBMdCU6fmDfr9oWQUXLasYNC87N8hnv3jeGa1P5Gyl20BFRDxKASAi4lEKABERj1IAiIh4lAJARMSjFAAiIh6lABAR8SgFgIiIRykAREQ8SgEgIuJRCgAREY9SAIiIeJQCQETEoxQAIiIepQAQEfEoBYCIiEcpAEREPEoBICLiUQoAERGPUgCIiHiUAkBExKMUACIiHqUAEBHxKAWAiIhHKQBERDxKASAi4lEKABERj1IAiIh4lAJARMSjFAAiIh6lABAR8ShzziW6hmGZmQOiia5jlBmQvAc9uY3nYzeeax9PdJxjfM45O1GjpA6As5GZbXLOLUx0HePReD5247n28UTHeWTUBSQi4lEKABERj1IAjL2fJbqAcWw8H7vxXPt4ouM8AroGICLiUToDEBHxKAWAiIhHKQBGgZnlmtnt8enLzOzJYdr9wszmnMT2ht3G2crMvm5mO83sgJn9zQjWm2JmN53J2hLNi88HGRsKgNGRC9x+okbOuS8552qPnW9m/jNS1fhyO3C1cy7POffDYxeaWWCY9aYAZ3UAyPA+4nkhJ0EBMDp+CEwzsxrgx0CmmT1iZrvM7D/MzADMbLWZLYxPHzKzvzez14DFZvYn8fZrgesSticJYGY/BaYCj5vZN8zs7vj8+83sJ2b2EvBPZnapmdXEf7aYWRaxY780Pu8bZ6i+DDN7ysy2mtl2M7vBzM43s5fNbLOZPWdmZfG2083shXjb181smsX8OL7uNjO7Id72svhzYqjnimefD8cys5vN7I34MX1wiOdFhpnda2Yb48+LT8XXe9rMzo1PbzGzv4tP/4OZfSmBu5Q8nHP6Oc0fYu9Ct8enLwMOAhOJBeyrwCXxZauBhfFpB1wfnw4C9cA5xD7K/jvgyUTv1xgfw/eBQuAW4O74vPuBJwF//PcngIvj05lAIH68z+ixAj4N/HzA7znAOqAo/vsNwL3x6deAawf8v4bi6z8P+IESoA4oG+65oufDoGNfBbwJFMZ/zx/iefED4PPx6VzgLSAD+BvgvwDZwEbguXibl4CZid63ZPjRGcCZscE594FzLgrUEAuIY0WAVfHpWcBu59zbLvYM/fXYlDkuPOyci8SnXwF+YmZfB3Kdc/1jVMM2YLmZ/ZOZLQUmAXOB5+Nnfd8FJsbPSCY45x4DcM71OOe6iL2oP+Scizjn9gEvA4vi2x7quaLnw1EfAx5xzrUAOOfa4vMHPi+uAP4m/n+xmliATgbWAMuIHf+niJ2Zh4Apzrk3x24Xkpf6z86M3gHTEYY+zj0DnsCgAayG0/nhhHPuh2b2FHA1sN7Mlo9FAc65t8zs/Pjj/iOxd/M7nHOLB7Yzs+xhNvFRg3IN91zR8yFmuMHdOo9p8+ljX9TNLBVYCLxH7P+sEPgysPnMlDr+6AxgdISBrNNYfxdQaWbT4r9/9vRLOvuY2TTn3Dbn3D8Bm4i9Uz7dY38yj1sOdDnnfg38M3AhUGRmi+PLU8ysyjnXAXxgZn8Wn58Wf8f5R+AGM/ObWRGxd6UbPuIh9Xw46g/A9WZWAGBm+UO0eQ742oDrJwsAnHOHiXWlXQ+sJ3ZG8K34v4LOAEaFc67VzF4xs+1AN7BvhOv3mNlXgKfMrAVYS6yLQQb7r2Z2ObF3yrXAM8SGC+83s63A/c65/30GHnce8GMziwJ9wF8B/cBdZpZD7O/o/wA7gL8A7jGzv4+3/XPgMWAxsJXYu9lvO+eazGzWUA+m58NRzrkdZvZ94GUziwBbhmj2D8SO/xvxEHgfuCa+bA3wcedcl5mtIXa9RQEQp6EgREQ8Sl1AIiIepQAQEfEoBYCIiEcpAEREPEoBICLiUQoAERGPUgCIiHjU/wehRxk+luX7zwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from statsmodels.graphics.mosaicplot import mosaic\n",
    "t = mosaic(titanic, ['passtype','status'],title='titanic survival')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHpNJREFUeJzt3XmUFdWh7/HvrxukBcWGbmSQQYWEKdp6g1GmxgHwajBGjWhMgjgnJtd3k8szWWbQmBdjjC/XeH0vYhRFjCNIEqc4JSCIA6BgVET0ChJBwhiVSeje949dNE3TDd10nz7ndP0+a53Vdap2nbPrUNSvdtWuKoUQMDOz9CnIdgXMzCw7HABmZinlADAzSykHgJlZSjkAzMxSygFgZpZSDgDLWZJ6SvpEUmGGPv8NScdn4rObQrLshzfB5wRJfZqiTtayOAAsZ0haKmnkjvchhPdDCAeEECqS6TMkXdxU3xdCGBhCmNFUn9fUkmX/72zXw1ouB4BZFkhqle06mDkALCdImgL0BB5JDn1cKenQ5PBFK0k/B4YDtyTTb0nm+42k5ZI+kjRf0vBqn3mNpAcl3S3p4+SQz6Bq06taHJIKJV0l6d2k7HxJPWqpZ5GkeyStlbRB0lxJnWt+XrXvvycZ3rEsF0l6H/iLpD9L+k6Nz18o6cxkOEjqI+k4SR9WPxQm6QxJryXDX5D0QlKflZJukbRfo/9RrMVzAFhOCCF8A3gfOC059HFDjek/BGYB30mm79hwzgWOAjoC9wIPSSqqNuuXgPuBYuBPwC11VOF7wFeBU4H2wIXAplrKnQ8cBPQASoBvApsbsKgjgP7AyUl9v7pjgqQBQC/gseozhBBeBDYCJ1YbfV4yP0AF8F2gFBgMnARc3oA6WUo5ACyvhRDuCSGsDSFsDyH8X6AN0LdakdkhhMeT8whTgLI6Pupi4EchhMUhWhhCWFtLuW3EDX+fEEJFCGF+COGjBlT5mhDCxhDCZmA6cJSkXsm0rwEPhxC21jLffSRhIelAYlDdl/wG80MILya/wVJgIjFozPbIAWB5TdJ/SFok6Z+SNhD3zkurFfmw2vAmoKiO4+89gHfr8ZVTgCeB+yWtkHSDpNYNqPLyHQMhhI+Je/vnJqPOBX5fx3z3AmdKagOcCbwSQlgGIOmzkh5NDhN9BFzHrr+BWa0cAJZL9nZr2l2mJ8f7vw+MBTqEEIqBfwLah+9eDvTeawVD2BZC+GkIYQAwBBgDjEsmbwTaVivepbaPqPH+PuCrkgYD+wN/reN73wSWAaew6+EfgN8CbwGfCSG0B65i334DSxkHgOWSVcCe+r3XnH4gsB1YDbSS9BPi8ft9cTvwM0mfUXSkpJKahSSdIOmI5ITsR8RDQhXJ5AXAuZJaJyebv1KP732ceNz/WuCBEELlHsreC1wBlAMPVRt/YFKXTyT1A75Vj+81cwBYTvkF8KOkN8uEWqb/BviKpPWSbiYeinkCeJu4d7yFaodYGujXwIPAU8SN6R3EPfKaugBTkzKLgJnAPcm0HxNbEeuBn7LrXnqtkuP9DwMj61H+PuB44C8hhDXVxk8gtgo+Bn4HPLC37zUDkB8IY2aWTm4BmJmllAPAzCylHABmZinlADAzS6mcviGVpOCEyhMFBVC5px6MeaKlLIelWiUQQtjrtSA5HQAF7OxgbTmuogLUAq49ainLYammmAF75R1sM7OUcgCYmaWUA8DMLKUcAGZmKeUAMDNLKQeAmVlKOQDMzFLKAWBmllIOADOzlHIAmJmllAPAzCylHABmZinlADAzSykHgJlZSjkAzMxSygFgZpZSDgAzy1shedm+cQA0kaXAU8nwoFqmXw+8t5fPODf5HDPbVQC+DQwHRgBfAr4FjAQ2ANcl48uBvwFvApcn8w4BHgQ2Aqc1a61znwOgiSxlZwDU5gfAYTXG+cmzZvXzJ+Lza2cBM4Fi4o7Ws8DfgcXJ+AeBnwADgLeAzUApMBt4ATiuuSue43L6mcD55LfAHGAesBb4GnEv5HvAN4DxwARgDXAjMXnPAroC3wd6J/OZ2e7eIu7971AAHJMMLyL+3zs+eV+Y/O0IPEbc6/8L8BwwKtMVzTMOgCbyLaAHcePegRgIBcQV7hs1yn5E3FsRcY/kWaAd0L+5KmuWZ/oDfwW+krwP7Dx80Y94+Of25P225O9Q4BfA74FXiS30HzZHZfOIDwFlwOFAe+AAaj9BNYi48QeoIO6ptAHKmqV2ZvnnNGALMAw4EVhXbdqRwGeIIXAC8KtkfDmwnBgQQ4H9iP/PbCe3AJpIa+LGHHZu3OtSPXULgfXEFsBrGaiXWUsgYqu6Lt9PXtV9HvhHMvy15GW7alQLQNJSSaXJ8JymqVJ+OgKYT2yiNqRb2rXAScQeQD0yUC8zs7oohH3vRStpKTAohLCmyWpUTaEUKvZezHJBCKC9tX3yQEtZDks1QWUIoXBv5erdApD0B0nzJb0h6dJapn+S/H1A0qnVxt8l6SxJhZJ+JWmupNckXVbf7zYzs6bXkENAF4YQPk88h3mFpJI6yt0PnAMgaT/iEY7HgYuAf4YQjiH24LpEUs2u8Ui6VNI8SfN8hZ+Z1WYpO3sE/Tuxv781XEMC4ApJC4EXiYerP1NHuSeAEyW1AU4BngshbAZGA+MkLQBeAkpq+4wQwm0hhEEhhEH53hBfildSs0y7Cdg/25XIU/UKAEnHE6+6HhxCKCN2qy2qrWwIYQswAziZ2BK4f8fHAP8WQjgqeR0WQtjTxbMtildSs323irgBKifuVFU/N3g88Anx9hArk3G3AncQL7z8MrHr6NdrzGf1bwEcBKwPIWyS1I+9X1F9P3AB8eK9J5NxTwLfktQaQNJnJbXbhzrnLK+kZpnRAfgz8WrensQre2s6G5iaDP8BOJN4D64rkvJHA9MzXtP8Ut8A+DPQStJrwM+Ih4H25CnidvCZEMKnybjbiXdHeEXS68BEWth1CF5JzTJjHXGnagTwKLCiljKnA38k7ogVEf8/vglcTdwBexD4sBnqmk/qtQEOIWwlHs+v6dBqZQ6oNryNeIy/+mdUAlclrxZpHfBN4oVdK4nNpppOJ270x7LrSvoS8ZqAzex+6wiztPs98STi5cT7a/Ui3vWzuvbE/3M3EXe0IF4FfAY77yO0DauuRe2BZ5tXUrPMOIm4Y/Qk8ar5I+sodzZwCTsPs/4weX918v4Gar9de1o16kKwTMu3C8EWEFfSw4kr6Whic3UqsQn6KPH+QPezcyU9gHgX0EuI9zWHPF1JW8oFVC1lOSzV6nshmAPAmkZL2XDm0HLcSjxMOD7L9bD80+RXApuZWcviADBrgBeALxB7o/yE2OtrOPGxg/clZd4hHrMeAfzvZNyNwOCk3Pxk3L8QnyNxLPG+9QDvE295fCqxN5lZJvkQkDWNHDp00ih7WY4fEzfYY4iP9BxGvOqxFfFe9H8hdlf8CbFLbyXxlsRjk3JLgUuBZ4jniv5KvKz+aGAh8B3itSKjibcvHoUPAVnD1fcQkHsBmTXAt4l76w8SL/pbQtxYQ7ygbzXxGbVHJ+MKiBv9smT4cOCfybQOxJ5isPMq8XeI97GH2NIwyyQHgFkDHAT8BviUuKHuDzxNfCDQtuRvD+LefBmxBXAosYdYJTEMipPPqq2d0Yd4n5WRxOdLn5SRpTCLHABmDTAReBjYSDw08znixroA6ERsGdxA7NYbiIeLfkm8AHAocaP/X3v4/CuB84jnDIr3UM6sKfgcgDWNlJwDMMsH7gZqZmZ75AAwM0spB4CZWUo5AMya2FL8JDjLD+4FZJZBN2W7AmZ74BaAWSP5SXCWrxwAZo3kJ8FZvvIhILNG8pPgLF85AMwaKetPguvcGVat2te5LcUcAGaNlO3HFd44YQIbN27chzmtxbrmmnoVcwCYNdJR7L7HPz75O6PauHOT1w4lxPsKNZY3/ravfBLYGq9z52zXwMz2QU63AILET6++eu8FLavatWvHhGxXwswazC0AazQfgjDLTw4AM7OUcgBYzgjAacCBwM/rOc9tmauOWYuX0+cALF0+JO6RfFzLtEpq31u5jfiQdTNrOAeA5Yz/BcwhPlrxfOJjEf8FGEJ8kPpZxJZBO+CcZJ7FxPvt/BQY0bzVNct7DgDLGTcAE4DvAI8m49YTb6nch3ix1Z3E5/DuaBHcwa597c2s/nwOwHJaB+LGH+DHwP8DxgEvZ61GZi2HWwCW06rvofQAfgt8QGwN/AXw49vN9p1bAJY3riHec/804OJkXF/iuYEXs1Qns3zmFoDljEPZec/845O/86pN/2Ut89yTwfqYtXRuAZiZpZQDwMwspRwAZmYp5QCwnLSU+IB1iNcBbM5eVcxaLJ8Etpx3U7YrYNZCuQVgOWEVMJLYzfMrQEW1accDnwBfYufjFG8lXgW8BvgycCLw9RrzmdmeOQAsJ3QA/gw8B/QkXuRV09ns7Cb6B+BM4HrgiqT80cD0jNfUrOXwISDLCeuAbxLv/bMSOKiWMqcTN/pjgSJiaLwJvARcSzxP8I3mqKxZC+EAsJzwe2A0cDnwPaAXuz9ovT0xGG4itgYA+gFnAMOT99syXlOzlsOHgCwnnES8z8/pxOcC1OVs4JakHMAPgf8kngM4EViYwTqatTRuAVhOOIrd9/jHJ39nVBt3bvLaoQR4OGO1MmvZ3AIwM0spB4CZWUo5AMzMUsoBYGaWUg4AM7OUcgCYmaWUA8DMLKUcAGZmKeUAMDNLKQeAmVlKOQDMzFLKAWBmllIOALOaOnfOdg3qL5/qajnHdwM1q+HGCRPYuHFjtqtRL+3atYM8qavlHrcAzGrIl40/5FddLfc4AMzMUsoBYGaWUg4AM7OUcgCYmaWUA8DMLKUcAGZmKeUAMDNLKQeAmVlKOQDMzFLKAWBmllIOADOzlHIAmJmllAPAzCylHABmZinlADAzSykHgJlZSjkAzMxSygFgZpZSDgAzs5RyAJiZpZQDwMwspRwAZmYp5QAwM0spB4CZWUo5AMzMUsoBYGaWUg4AM7OUcgCYmaWUA8DMLKUcAGZmKeUAMDNLKQeAmVlKOQDMzFLKAWBmllIOADOzlHIANJH169fzzjvvADBx4sTdps+aNYv169fv8TMeeuihvZYxs51CCIQQsl2NvNUq2xVoKTZs2MC7775Lnz59ap0+fPjw3cZVVlZSUOAMNtubEAKPP/44q1atQhJt2rShffv2rF27lrFjxzJ37lzeffddQgh88YtfRBIvv/wyY8aM4fbbb+e4447js5/9LFOnTuW8887L9uLkDAdAE5k3bx7Lly9nxYoVbN68mWnTprF69WoGDx5MWVkZ06dPZ8iQIWzatIk5c+YgiQEDBnDAAQfwzDPP0KFDBzZv3pztxTDLSYsXL6agoIALL7wQgIcffphu3boxZswYVq1axdq1a7ngggv4+OOPeeyxxzj33HNZs2YN27Zto23btrz//vvsv//+dO/ePctLklscAE1k0KBBtG/fnpNPPpnrr7++ai9kypQplJWV7VJ269atXHDBBUjid7/7HePGjWO//fbjlltuyVLtzXLbmjVr6NmzZ9V7SXTr1g2A1atXs3z5cu68806Aqlb1/vvvz9tvv03fvn157733WLZsGb17927+yucwB0AGdOjQgaKiIoBaj09269YNSVXT27ZtC0CXLl2ar5JmeaS0tJSlS5cycODAqnE7/g+VlpbSq1cvTj/9dAAqKioA6NmzJ7Nnz+bMM89k5cqVvPvuu5SXlzd/5XOYA6CJFBYW1vtk1I4Vd8fw5s2bad26NatWrcpU9czyWt++fXnnnXe44447KCwsZL/99qua1qVLF0pKSrjzzjuRxOGHH055eTm9evVi1qxZdOrUiZ49e/KPf/yDVq28yauuwb+GpGuAT0IINzbmiyUtBQaFENY05nNyxcEHH8wzzzzDAw880KD5TjjhBCZPnkxxcTHt27fPUO3M8pskxowZU+f0YcOGMWzYsF3GdevWjSuvvBKAI488kiOPPDKjdcxHjsMmUlRUVHWCqrpLLrkEgDPOOKNq3GGHHVY13KdPnzp7DpmZZVK9+iBK+qGkxZKeAfom43pL+rOk+ZJmSeqXjD9N0kuSXpX0jKTOyfgSSU8l4ycCqvsbLR9tPOCAbFehUfK9/mYNtdcWgKTPA+cCRyflXwHmA7cB3wwhLJF0LPD/gROB2cBxIYQg6WLgSuA/gKuB2SGEayV9Ebi0ju+7tK5plttunDAh21VotKuzXQHbq9WrVzN79uxdWtX1NXfuXFq1asXRRx+dgZrln/ocAhoOTA8hbAKQ9CegCBgCPFTthGab5G934AFJXYH9gPeS8eXAmQAhhMck1XrJawjhNmK4UFBQkJeX+HkFNbN8UN9zADU3xAXAhhDCUbWU/S/g1yGEP0k6HrhmD59jZrZXFRUVTJ06lS1bttCpUycAlixZwqxZs6isrOTYY4/liCOOYOHChSxYsIDNmzdXXYS5YcMGpk2bRps2bSgqKvK1ANXUJwCeA+6SdH1S/jRgIvCepLNDCA8pNgOODCEsBA4CPkjmPb/G53wN+D+STgE6NNVC5AKvoGaZ89Zbb1FSUsLIkSOZP38+y5YtY+bMmYwfP56CggImT57MwIED6d+/P2VlZWzfvp1JkyZRVlbG888/z4gRI+jTpw/Tpk3L9qLklL0GQAjhFUkPAAuAZcCsZNLXgN9K+hHQGrgfWEjc439I0gfAi8COLi8/Be6T9AowE3i/CZcj67yCmmXOunXrqq78PeSQQ1i0aBHr1q1jypQpAGzatImNGzfywQcf8MILLwCwdu3aWue1nep1CCiE8HPg57VM+tdayv4R+GMt49cCo6uN+m4965gXvIKaZU7Hjh1ZuXIlAwYMYMWKFbRt25bS0lLGjRtHYWEhFRUVFBYWMmPGDMaPH09hYSE333zzLvP27t2bFStW7NINO+18HUAT8Qpqljn9+vXj9ddfZ/LkyZSWliKJ8vJy7r77biTRtm1bxo4dy4ABA7jrrrvo2rVr1e1Yhg4dyrRp05gzZ07VOIscAE3EK6hZ5hQWFnLOOefsNr7mRZTl5eW73e+nuLiYiy66KKP1y1cOgCbiFdSamrsEW6b5aSRmZinlFoBZAyxfvpwnnniC1q1b06tXL3r06LFbV9+1a9fy6KOPUllZySGHHMLo0aN5/vnnWbRoEQCnnnoq3bp149Zbb6V79+6sXLmSfv36MXz4cHcJtmblADBrgCVLljBixAj69u1LZWUlkyZN2q2r79NPP83o0aPp2rUrlZWVfPzxxyxevJgLL7yQDRs28Mgjj3D++eezZcsWhg0bRvv27Zk4cSLDhw93l2BrVg4AswY45phjmD17Nm+88QaHH354rV19P/roI7p27QrEp1Nt2LCBLl26UFBQQMeOHdm6dSsQn1hVXFwMUHWfencJtubkADBrgKKiIk455RS2b9/ObbfdVmtX3/bt2/Phhx/SpUsXKisrKS4u5sMPP6SyspINGzbssaeXuwRbc3IAmDXAvHnzWLRoEdu2beOoo47i4IMP3q2r76hRo3jkkUcIIdC9e3dGjRpF3759mTRpEhDPAdTFXYKtOTkAzBpg8ODBDB48eJdxNbv6lpSUMH78+F3GDR06lKFDh+4y7rLLLqsa3vHgIHcJtubkbqBmZinlADAzSykHgJlZSjkAzJrI6tWrmT59+j7NO3fuXF599dUmrpHZnjkAzMxSyr2AzBrBT4KzfOYAMGsEPwnO8pkDwKwR/CQ4y2cOALNG8JPgLJ85AMwawU+Cs3zmADBrBD8JzvKZu4GamaWUA8DMLKUcAGZmKeUAMDNLKQeA5YwQAvfeey/XXXcdzz33XL3mmTdvXoZrZdZyuReQ5YxPPvkESVx11VW7TausrKSgYPf9lfnz5zNo0KDmqJ5Zi+MAsJzxxBNPsHz5cm644QbKyso4+eSTufXWW+nRowdbt26lf//+zJo1i9atW/O5z30OiFfV3nnnnZxwwgkceuih2V0AszzjALCcMWrUKJ566im+8IUv8PbbbwOwZcsWjjvuOEpKSnj44Yc5/fTT6dy5c1WL4JVXXuGCCy7Ics3N8pMDwHJaUVERJSUlQLyY6sUXX+TTTz/lmGOOoUePHlmunVl+80lgy2mSqoYPOuggxowZw8iRI3n22Wd3m25mDeMWgOWNGTNmsHz5cj799FOGDBkCQElJCQ888ABDhgxxi8CsgRwAljM6dOhQdV+dHXfGvOyyy6qmjxo1ard5zjrrrOapnFkL5ENAZmYp5QAwM0spB4CZWUo5ACynrF69munTp+/TvHPnzuXVV19t4hqZtVwOADOzlHIvIMu6iooKpk6dypYtW+jUqRMAS5YsYdasWVRWVnLsscdyxBFHsHDhQhYsWMDmzZsZPHgwZWVlbNiwgWnTptGmTRuKioro3bt3lpfGLH84ACzr3nrrLUpKShg5ciTz589n2bJlzJw5k/Hjx1NQUMDkyZMZOHAg/fv3p6ysjO3btzNp0iTKysp4/vnnGTFiBH369GHatGnZXhSzvOIAsKxbt24d3bp1A+CQQw5h0aJFrFu3jilTpgCwadMmNm7cyAcffMALL7wAxJvA1TavmdWfA8CyrmPHjqxcuZIBAwawYsUK2rZtS2lpKePGjaOwsJCKigoKCwuZMWMG48ePp7CwkJtvvnmXeXv37s2KFSuqLiAzs71zAFjW9evXj9dff53JkydTWlqKJMrLy7n77ruRRNu2bRk7diwDBgzgrrvuomvXrhQVFQEwdOhQpk2bxpw5c6rGmVn9OAAs6woLC6tuAVFdnz59dnlfXl5OeXn5LuOKi4u56KKLMlo/s5bK3UDNzFLKAWBmllIOADOzlHIAmJmllAPAzCylHABmZinlADAzSykHgJlZSjkAzMxSygFgZpZSDgAzs5RyAJiZpZQDwMwspRwAZmYp5QAwM0spB4CZWUo5AMzMUsoBYGaWUg4AM7OUcgCYmaWUA8DMLKUcAGZmKeUAMDNLKQeAmVlKOQDMzFLKAWBmllIOADOzlHIAmJmllAPAzCylHABmZinlADAzSymFELJdhzpJCkBltuvRxATk7o+e2/L5t8vnuucT/85RQQhBeyuU0wHQEkmaF0IYlO165KN8/u3yue75xL9zw/gQkJlZSjkAzMxSygHQ/G7LdgXyWD7/dvlc93zi37kBfA7AzCyl3AIwM0spB4CZWUo5AJqApGJJlyfDx0t6tI5yt0saUI/Pq/MzWipJV0haJGm9pB80YL5DJZ2XybplWxrXB2seDoCmUQxcvrdCIYSLQwhv1hwvqTAjtcovlwOnhhA6hBCurzlRUqs65jsUaNEBYHXbw3ph9eAAaBrXA70lLQB+BRwgaaqktyT9XpIAJM2QNCgZ/kTStZJeAgZL+tek/GzgzKwtSRZIuhU4HPiTpO9KuiUZf5ekX0v6K/BLSSMkLUher0o6kPjbD0/GfTdD9Wsn6TFJCyW9LukcSZ+XNFPSfElPSuqalO0j6Zmk7CuSeiv6VTLv3ySdk5Q9PlknaltXUrs+1CRpnKTXkt90Si3rRTtJkyTNTdaL05P5Hpd0ZDL8qqSfJMM/k3RxFhcpd4QQ/Grki7gX+noyfDzwT6A7MWBfAIYl02YAg5LhAIxNhouA5cBniJeyPwg8mu3laubfcClQCowHbknG3QU8ChQm7x8BhibDBwCtkt87o78VcBbwu2rvDwLmAJ2S9+cAk5Lhl4Azqv27tk3mfxooBDoD7wNd61pXvD7s8tsPBBYDpcn7jrWsF9cBX0+Gi4G3gXbAD4BvA+2BucCTSZm/An2zvWy58HILIDNeDiH8PYRQCSwgBkRNFcC0ZLgf8F4IYUmIa+g9zVPNvPBQCKEiGX4e+LWkK4DiEML2ZqrD34CRkn4paTjQA/gc8HTS6vsR0D1pkRwSQpgOEELYEkLYRNyo3xdCqAghrAJmAsckn13buuL1YacTgakhhDUAIYR1yfjq68Vo4AfJv8UMYoD2BGYB5cTf/zFiy7wtcGgIYXHzLULu8vGzzNhabbiC2n/nLdVWYPANrOqyccdACOF6SY8BpwIvShrZHBUIIbwt6fPJ9/6CuDf/RghhcPVyktrX8RF7uilXXeuK14eorpu7baxR5qyaG3VJ+wGDgP8m/puVApcA8zNT1fzjFkDT+Bg4sBHzvwUcJql38v6rja9SyyOpdwjhbyGEXwLziHvKjf3t6/O93YBNIYR7gBuBY4FOkgYn01tLGhhC+Aj4u6QvJ+PbJHuczwHnSCqU1Im4V/ryHr7S68NOzwJjJZUASOpYS5kngX+rdv7kaIAQwqfEQ2ljgReJLYIJyV/DLYAmEUJYK+l5Sa8Dm4FVDZx/i6RLgcckrQFmEw8x2K7+XdIJxD3lN4EniLcL3y5pIXBXCOE/M/C9RwC/klQJbAO+BWwHbpZ0EPH/0U3AG8A3gImSrk3Kng1MBwYDC4l7s1eGED6U1K+2L/P6sFMI4Q1JPwdmSqoAXq2l2M+Iv/9rSQgsBcYk02YBJ4UQNkmaRTzf4gBI+FYQZmYp5UNAZmYp5QAwM0spB4CZWUo5AMzMUsoBYGaWUg4AM7OUcgCYmaXU/wAflfmz68PB2QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# slightly easier to read\n",
    "props = lambda key: {'color': 'r' if 'alive' in key else 'gray'}\n",
    "t = mosaic(titanic, ['passtype','status'],title='titanic survival',properties=props)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can go beyond visual exploration and apply analytic tests to see if the \n",
    "observed values differ from the expected ones.  The chi-square test sums the squares of the differences\n",
    "between the observed and expected values, normalized for the expected values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's talk about $\\chi^2$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chi2 =  190.40110361683327\n",
      "p-val =  6.705773035354302e-37\n",
      "degree of freedom =  8\n",
      "Expected:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>285.885961</td>\n",
       "      <td>599.114039</td>\n",
       "      <td>885.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>104.986370</td>\n",
       "      <td>220.013630</td>\n",
       "      <td>325.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>92.064970</td>\n",
       "      <td>192.935030</td>\n",
       "      <td>285.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>228.062699</td>\n",
       "      <td>477.937301</td>\n",
       "      <td>706.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>711.000000</td>\n",
       "      <td>1490.000000</td>\n",
       "      <td>2201.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            0            1       2\n",
       "0  285.885961   599.114039   885.0\n",
       "1  104.986370   220.013630   325.0\n",
       "2   92.064970   192.935030   285.0\n",
       "3  228.062699   477.937301   706.0\n",
       "4  711.000000  1490.000000  2201.0"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.stats import chi2_contingency\n",
    "chi2, p, dof, ex = chi2_contingency(ct)\n",
    "print(\"chi2 = \", chi2)\n",
    "print(\"p-val = \", p)\n",
    "print(\"degree of freedom = \",dof)\n",
    "print(\"Expected:\")\n",
    "pd.DataFrame(ex)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's apply these ideas to another dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this component, we'll use the Comic Characters data set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>publisher</th>\n",
       "      <th>name</th>\n",
       "      <th>Identity</th>\n",
       "      <th>align</th>\n",
       "      <th>eye</th>\n",
       "      <th>hair</th>\n",
       "      <th>sex</th>\n",
       "      <th>alive</th>\n",
       "      <th>appearances</th>\n",
       "      <th>first_appearance</th>\n",
       "      <th>month</th>\n",
       "      <th>year</th>\n",
       "      <th>date</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Marvel</td>\n",
       "      <td>Spider-Man (Peter Parker)</td>\n",
       "      <td>Secret Identity</td>\n",
       "      <td>Good Characters</td>\n",
       "      <td>Hazel Eyes</td>\n",
       "      <td>Brown Hair</td>\n",
       "      <td>Male Characters</td>\n",
       "      <td>Living Characters</td>\n",
       "      <td>4043.0</td>\n",
       "      <td>1962, August</td>\n",
       "      <td>August</td>\n",
       "      <td>1962.0</td>\n",
       "      <td>8/1/1962</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   publisher                       name         Identity            align  \\\n",
       "id                                                                          \n",
       "1     Marvel  Spider-Man (Peter Parker)  Secret Identity  Good Characters   \n",
       "\n",
       "           eye        hair              sex              alive  appearances  \\\n",
       "id                                                                            \n",
       "1   Hazel Eyes  Brown Hair  Male Characters  Living Characters       4043.0   \n",
       "\n",
       "   first_appearance   month    year      date  \n",
       "id                                             \n",
       "1      1962, August  August  1962.0  8/1/1962  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comic_characters = pd.read_csv(\"data/comic_characters.csv\", index_col=\"id\")\n",
    "comic_characters.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example\n",
    "\n",
    "We'd like to know which publisher uses different 'identity' types for their characters? Have DC characters appeared more publicly? What is the average number of times Marvel Characters appeared known to authorities identity?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>publisher</th>\n",
       "      <th>DC</th>\n",
       "      <th>Marvel</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Identity</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Identity Unknown</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Known to Authorities Identity</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>74.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>No Dual Identity</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>24.402013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Public Identity</th>\n",
       "      <td>28.287166</td>\n",
       "      <td>23.407615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Secret Identity</th>\n",
       "      <td>32.655157</td>\n",
       "      <td>17.379794</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "publisher                             DC     Marvel\n",
       "Identity                                           \n",
       "Identity Unknown                3.000000   0.000000\n",
       "Known to Authorities Identity   0.000000  74.400000\n",
       "No Dual Identity                0.000000  24.402013\n",
       "Public Identity                28.287166  23.407615\n",
       "Secret Identity                32.655157  17.379794"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comic_characters.groupby(['Identity','publisher'])['appearances'].mean().unstack().fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Identity                       publisher\n",
       "Identity Unknown               DC            3.000000\n",
       "Known to Authorities Identity  Marvel       74.400000\n",
       "No Dual Identity               Marvel       24.402013\n",
       "Public Identity                DC           28.287166\n",
       "                               Marvel       23.407615\n",
       "Secret Identity                DC           32.655157\n",
       "                               Marvel       17.379794\n",
       "Name: appearances, dtype: float64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comic_characters.groupby(['Identity','publisher'])['appearances'].mean() # change the format by unstack()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can use .pivot_table(). For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>publisher</th>\n",
       "      <th>DC</th>\n",
       "      <th>Marvel</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Identity</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Identity Unknown</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Known to Authorities Identity</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>74.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>No Dual Identity</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>24.402013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Public Identity</th>\n",
       "      <td>28.287166</td>\n",
       "      <td>23.407615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Secret Identity</th>\n",
       "      <td>32.655157</td>\n",
       "      <td>17.379794</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "publisher                             DC     Marvel\n",
       "Identity                                           \n",
       "Identity Unknown                3.000000   0.000000\n",
       "Known to Authorities Identity   0.000000  74.400000\n",
       "No Dual Identity                0.000000  24.402013\n",
       "Public Identity                28.287166  23.407615\n",
       "Secret Identity                32.655157  17.379794"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "avg_appearance_per_identity = comic_characters.pivot_table(index='Identity', \n",
    "                                                          columns='publisher', \n",
    "                                                          values='appearances',\n",
    "                                                          aggfunc='mean')\n",
    "avg_appearance_per_identity.fillna(0).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For .pivot_table(), you need to specify these four arguments:\n",
    "1. index: the field that will become the index of the output table\n",
    "2. columns: the field that will become the columns of the output table\n",
    "3. values: the field to be aggregated/summarized\n",
    "4. aggfunc: the aggregation operator applied to values, if there are more than 1 entry corresponding to each (index, column) pair, such as \"mean\", \"count\", \"max\"\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's warm up with a few groupby and pivot_table exercises:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q4: What is the total number of appearances of characters for each publisher?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "publisher\n",
       "DC        154532.0\n",
       "Marvel    260270.0\n",
       "Name: appearances, dtype: float64"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comic_characters.groupby('publisher')['appearances'].sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q5: What is the total number of appearances of characters by each publisher in each year? Output a table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "publisher\n",
       "DC         2209.0\n",
       "Marvel    25046.0\n",
       "Name: 1963.0, dtype: float64"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# comic_characters.head()\n",
    "# comic_characters.groupby(['publisher', 'year'])['appearances'].sum().unstack().fillna(0)\n",
    "b = comic_characters.groupby(['year', 'publisher'])['appearances'].sum().unstack().fillna(0)\n",
    "b.loc[1963.0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q6: Construct a contingency table of sex and character alignment normalized by all values.\n",
    "\n",
    "Display the normalized values in percentage (%) format. Use brief sentences to explain your findings.  \n",
    "\n",
    "Hint: use \"normalize=all\" in your crosstab statement.  What does normalize do? (read the docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>align</th>\n",
       "      <th>Bad Characters</th>\n",
       "      <th>Good Characters</th>\n",
       "      <th>Neutral Characters</th>\n",
       "      <th>Reformed Criminals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sex</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Agender Characters</th>\n",
       "      <td>0.001045</td>\n",
       "      <td>0.000523</td>\n",
       "      <td>0.000679</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Female Characters</th>\n",
       "      <td>0.082197</td>\n",
       "      <td>0.130114</td>\n",
       "      <td>0.043685</td>\n",
       "      <td>0.000052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Genderfluid Characters</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000052</td>\n",
       "      <td>0.000052</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Genderless Characters</th>\n",
       "      <td>0.000575</td>\n",
       "      <td>0.000314</td>\n",
       "      <td>0.000157</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Male Characters</th>\n",
       "      <td>0.395099</td>\n",
       "      <td>0.251293</td>\n",
       "      <td>0.094006</td>\n",
       "      <td>0.000105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Transgender Characters</th>\n",
       "      <td>0.000052</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "align                   Bad Characters  Good Characters  Neutral Characters  \\\n",
       "sex                                                                           \n",
       "Agender Characters            0.001045         0.000523            0.000679   \n",
       "Female Characters             0.082197         0.130114            0.043685   \n",
       "Genderfluid Characters        0.000000         0.000052            0.000052   \n",
       "Genderless Characters         0.000575         0.000314            0.000157   \n",
       "Male Characters               0.395099         0.251293            0.094006   \n",
       "Transgender Characters        0.000052         0.000000            0.000000   \n",
       "\n",
       "align                   Reformed Criminals  \n",
       "sex                                         \n",
       "Agender Characters                0.000000  \n",
       "Female Characters                 0.000052  \n",
       "Genderfluid Characters            0.000000  \n",
       "Genderless Characters             0.000000  \n",
       "Male Characters                   0.000105  \n",
       "Transgender Characters            0.000000  "
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table = pd.crosstab(comic_characters.sex , comic_characters['align'], normalize = True) # align is a reserved function, conflicted\n",
    "table # margins=True, in crosstab : add them up to a All column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q7: Create a mosaic plot of character alignment and alive status."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgAAAAD8CAYAAAABtxyxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xl0XNWV6P/vrrlUqtI8y6OM5xHbGDMYGzBDQyBAGEIIYcjU6aQ7/ZKXl5f0r4HuTjq/1+m8TkKnOw2NHQIhBBwzz2BmDHjCMzaW8CTJkqXSXKrxvD/ulVySyrZkDDbU/qzFonTr3nvOLWn57nvO2bvEGINSSimlsovjRHdAKaWUUp88DQCUUkqpLKQBgFJKKZWFNABQSimlspAGAEoppVQW0gBAKaWUykIaACillFJZSAMApZRSKgtpAKCUUkplIdeJ7oD6+ImIkRPdCfWJEYcV16dM6gT35Ng45NPdf/Xp1/83mEphzGf3n08NALKAALeNHXuiu6E+IbfV1QHwnTvGneCeHJtf32b1f9wdF57gnqhsVXfbswCIyGc6CtUpAKWUUioLaQCglFJKZSENAJRSSqkspAGAUkoplYU0AFBKKaWykAYASimlVBbSAEAppZTKQhoAKKWUUllIAwCllFIqC2kAoJRSSmUhDQCUUkqpLKTfBaCOuz29vbzU1oYxBoAFoRBTA4FjOte7HR24RJgTDA7ZvrG7GwFynU4uLy7mqZYWzsjLo8zj+aiXkNGazk7mDeqHUkp9WmkAoI6rnmSSJ1tauKGsjKDLRdIY6qPR49rGrkiEHZEIXykvxyVCYyxG0g42RiplDA4Z3pd9rR1BADCS8yql1ImgAYA6rnZEIkzOySHosv60nCKM8vkAqI1EeDEcBmBeMMicYJDGWIwnW1owwES/n0X5+bQlEqxobsbrcOBzOKixj++zqbubM0IhXPYNtjztif/tjg7CiQQeEa4rLaU7lWJFczNJYwg4nVxdUkJ7IsHKgwfJdTqp9HoJOp1s6OoikkqxMBRiVm4u3ckkjxw8SCyVIuRyMTUnh5Z4nGUNDcwPhaj0eHiypYWEMVR4vVxUWMj6zk4+iESIGcOCUIi3OzqIpVIY4MtlZbgdOuOmlDp5aACgjquuZLL/5l8bifBKWxteh4Pry8p4MRzm+rIyvA4Hdzc0MD0Q4MVwmMuKiih2u7n3wAFmxOO82dHBOfn5TPD7WdHcfMQ2Bhvt83FZbi4rmps5EI9T7HZzQ1kZThGeaW2lrreXQpeLzmSSG+0RhFgqxazcXBLGcE9DA7Nyc3mtvZ1Tc3OZEgj0P82/2t7OzRUVAPypqYlLiooodLt5qqWF/fYoh1OEL5WW0hqP4xLhSxUVGGOQEY4GhONxfrl/P1+tqKDa62VnTw/7olGWFBQM+xwd4ThtLQlGT/APa/9t6zuJxwwzF4QGbN+1rZsNb3YgAm6vgyWfK2LL2k5KK72Mm5QzousaruXLl2MSKcSlQZNSHxcNANRxFXQ6aYnHARjv9zPe7+e39fUAGCDgdAJQZN+Eu5NJSuwn+EqPh3AiQWs8TqW9rcrrzdhGRyJBsds95L0K+7iQ00lvKkUkmeSJlhZ6Uyk6k0nKPR4KXS7KPJ7+EYTa3l7eam8H6O/7wXics/PyADIO5R+Mx3n04EEAYsYwzh6lqLT7W+h2M9bnY2VzMyGXiyX5+SOeEihxu3mjvZ1rS0tHdFyfjrYEe3ZFhgQAJmUQx/D6Em6Ose71di6/sRyP10FHOE4qeWzTLSNpd/ny5ZgznMMKAEZyXqXUIRoAqOPqFL+fN9rbmRcMEnK5SKXNzQvQnUzidThoSSQIOp0EnE6aYzGK3W7qYzHmBYMUut00xGLU+P3UR6P9N9c+0wMB3uroYIzPh1OEA7EYITuwSGeMYVN3NzV+P6eFQjzT2tq/MDH9dvFyWxs3lZfjBH61fz8AxW43e6NRJufk9I8ApB9T7HZzQWEh+S4XxhhSwMaurv59EsYwPxjEEQrx2MGD7IlGGTvoOo6m2O0mZQzNsdiA7Tt7enitvZ0UsCAYZEZuLiubm/sXQP7whz/koosuYvOaThr3Rmmqj3LJF8t4+O4GSis9+ANORk/ws/a1duKxFDVTcph7dn7GPuzc0sO0eUE8XutGHCo4FHTt3NzNpnc7ScRSfO6GMlwu4bH7DpBMGpxO4eJrSvH4HNx/5/4jtpuIp3jx0Ra6OxM4HML1F7zFhg0b6K0TAlPLCMyo4OBjWzDRBM6gj+IrptO7O0zHmx+CCIGppUTqwiTCPdbndsUM3AXDG/VQKpsdNQAQkSSwCXADCeB3wL8ZY1Ifc98+EhG5Hegyxvw8w3s3Aj/Aug8IcI8x5uci8jLwfWPMmk+oj4uBmDHmzU+ivU9CjtPJpUVFrGhuRkRwAKeHrCHl8woK+MOBA4B143I7HJxXUMBjaWsACtxuzszLY0VzM2+2t+PLMG9e4/fTGo+zrLERB9aIwGXFxRn7M87nY+XBg+yKRHA7HJRlGDWYmpPD8sZGKjye/vbOzsvjkYMHeau9nXyXiytKShjr8/GHAweYGwxyfkEBT9hrAAS4fFD7bYkEjx08iAG8Dkf/yMRInZGXx5sdHUzNsYbaU8bwSns7N5WX4wB+19jItMNkWEyfFyQ35OKsCwsB6OpIcNWt5fj8TuKxFFfcVI4xhofvbmDW6aGM5+jpSlJWPXQUBiC/yM1pi/N568Uw+2p7GTc5h0u+WIrL7eC91R3s3NLNtLnBo7a7ZW0XZVUeZi8swaQMCxcuZPbs2dSd4cThddH67PuEFozGP76I9jc/pGfbARw5HlLRBOU3z4eUoePdvVR8dQEigkkd2wiFUtlmOCMAEWPMbAARKQX+AOQBt32cHfu4iMjFwHeBC4wx9SLiA758HM7rNMYkR3jYYqALGHYAICIuY0xihO18okb7fP1z5en6pgTSlXs83Dpo33yXa8i2weaHQswPDbxpXVFS0v96aWFh/+tvVVUNOT59WH1Rfj6L8gc+AQecTr5UVjZg2wVp5wS4YdD76amKxW43txzlGoZjjM/HqrY2OpPWn1ZPKkVrPM7vGxv7f+5OJgesMTCHyYjIL3Tj81sjJQcbY7y9qo1UytARTtDTnTmeDwSddHdk/nMrqbCCmmDISW9vingsxarHW+hqTxDtTVEzNTCsdsMH40yZkwuQcSg/3txNdH87ba/UYhJJcmdW4sjx4K3Ms67bKYQWjKHl0S2I10XBuRMQrw5uKnU0I1phY4xpAr4OfFssThH5FxF5V0Q2isg3+vYVkR+IyCYReU9EfmZvqxGRZ0RkrYi8JiKT7e2fE5G3RWS9iLwgImX29nNEZIP933oRCdrb/2dam3ektfljEXlfRF4AJh3mMv431lN+vX1NvcaYu9Lev1pE3hGRHSJytn3esXZ/19n/nWFvXywiq0TkD1ijJIjII/b1bRGRr6f17SL72PdE5EURGQt8E/hb+/rOFpESEVlhX9u7InKmfeztIvJfIvIccK+ITLP7uMH+DE4Zye9RfbqcHgqxuqMDgByHg2K3mxvLy7m5ooJvVlYSdLnwORx0JKwb9bp16wBwOoUBsUDavXXt6+0suriQK75STm6eCw4TNEyYFmDL2k5iUStA6GxL0BGOD93RGHZ/ECEQdHLlLRVMPTV4KBA5SrsFxW4a91qLKPue3t1ud3+f3MUBCs47hYqb51P5tdMJzqsecF6TMgSmlVH8+ek4Ax56tjUN63NVKtuNOEw2xtSKiAMoBS4H2o0x80XEC7xh36QmA58HFhhjekSk79Hpv4BvGmN2isgC4DfAucDrwOnGGCMiX8Uanv8e8H3gr4wxb4hILtArIhcApwCnYf0T8JiILAK6geuAOfZ1rQPWZriE6YfZ3v+ZGGNOE5G/wBrlOB9oApYaY3rtm+0DwDx7/9OA6caYOvvnW4wxrSLiB94VkRVYgdZdwCJjTJ2IFNr7/Cdp0xR2IPF/jTGvi8ho4Flgin3eucBZxpiIiPwa+KUx5n4R8QBDJsDt4OPrg7erT59Jfj8v2OmTDhEW5edzb2MjIkKOw8E1paXMttcBvNvZSc0U60+mqNTDWy+EefrBJs77/MApipopOTzzcDNFpR7cnsMvoCsodnPqmXk8dt8BHAIen4PFlxZl3Le82svaV9t5/P4DBHKdBEJD12Vkanfa3FxeeOQgu7Z243QLd94Bl112GT/45R0EppWRt2gcLY9tpW3VB1aflk4ccM5UNEHTA+utVaYCJV+YObwP9mMUD0douGs17pJcTCxJ6Iwx5M44+ohQz86DRPe1UbBkwoDt3dsO0PHmbhBweF0UfW4qnWv34a3MI2dSyWHO9tF0rt9P7owKzcT4DJPDDRf27yDSZYzJHbStDesJ+9+BmUCP/VYe8A3gQmB7+pO1fQNvBt5PO5XXGDNFRGYA/wpUAB6gzhhzkYj8ELgCuB/4szFmn4j8HPgC0GafIxf4ZyAIFBpj/t5u7xdA/eA1ACLSCowzxrRnuNaXgR/bAUcZ8IYxZoKI5AF3ArOBJDDRGJNjz+HfZoxZknaO2+0+A4y1P4sS4DpjzJcGtXc7AwOAJqA+bZcSrGDqe4Axxtxh73c98GPgXvtz2Tn4WtI5RMxtY8ceaZejCsfj3NXQQKk9lz3G62VRfj7Ok6TYzV319XytsnLAtvZEgqdaWoikUqSAhXb+/nPh8DGvrD+ahmiUJFCdIXvhk3JbnRWLfueOcSesDx/Fr2+z+j/ujgtPcE+OXTwcIfzc+5ReO5tULEnjPe9Q+c2FRz0uUwAQa+7m4CObKL9xHg6vi3g4Ahi6NtSPOAAYScZEw7J3Kbt+Do5hTKd81jIx6m57FgARSRljhkaynxEjHgEQkfFYN8EmrCfw7xhjnh20z0VY8Xg6B9DWt55gkF8DvzDGPGbfVG8HMMb8TESeBP4CWC0i59tt/rMx5reD2vxuhjYz2YL1NP3SYd7vK1uX5NDn87fAAWCWfR29aft3p/VhMdaIwUJ75ONlwGf3eTh9c9jHRtI32vO7/e0YY/4gIm8DlwDPishXjTGHu57jZozPx7WlpRhjWNXWxuqODs60U+VORiuamzmvoIAxPh9JY9h3jBUJ+zMHhhHsNMZixIwZVgCg1QKzg4knEbd1D4nsaqH9tVpSsSQ5U8rIP3scyUic5oc2goAr5MMZGvi307OlkeC8Uf034vQMh+7NDXS+u4dULEnZDXMRl4MD963DJFOIUyi9ZjYOn4v9d76BpzKEM+DBP6F4SB9S8SQtj24h0RlFHEL+uROINXZy4P51monxGTaiAEBESoD/BO60h+ufBf5SRF4yxsRFZCKwH3gO+HsR+UPfFIA95F0nIlcbYx4S61/TmcaY97BGDvbbzXwlrb0aY8wmYJOILMR6Gn4W+EcRud8Y0yUiVUAceBVYbq83cAGfAwYECbZ/Bv6PiFxqjGm0py6+YYz51REuPQ/YZ4xJichXyDDknrZf2L7mycDp9va3gH8XkXHpUwBAJ5C+ku054NvAv9jXP9sYs2FwI3YQVmuM+ZX9eiaHD2iOO7GHoX/X2MiZeXkZ09Ja4nGeaGkhZQxVXi8X2JXy1nd1kQLOzc9nvJ0yuDMSIZpKcX5BATV+Py+Gw9T19uIAlhYUUO318nRrK03xOA6sFfd5LhevtrWxIxKhwuMZEl21JRIIVtACVoGeMT4f4XiczmSSB5uaaInHubCwkJrD9GNlczNuh4OWeJyrS0p42K4o6BThmtJSfA4H6zo7WdvZiVOExfn5rOnsJJJKsTMS4YayMl5ta2NXJIIBLikqoszj4T/r6xnl9RJNpZiSk8Nr7e24RZgeCAxZ2Kg+vXp3h2lY9i6Jlh7yz60BwDsqn/Kb5mOMoeHudwidPpqudfsJTC0jOK+a8ItDB/OSXVG81ZkDbXdRgPzFNYRf3ElvbQs5k0sp/eJsHG4nHat3072lkeDcahIdvZTfehpOv5tULDm0D2v34anKo2ThmP6neU95sH8EQDMxPpuGEwD4RWQDh9IAfw/8wn7vbqxh7nX2Db0Z+Lwx5hkRmQ2sEZEY8BTwI+BLwH+IyN/Z5/sj8B7WE/9DIrIfWA30jV1+V0SWYD2NbwWeNsZERWQK8Jb9RNYF3GCMWSciDwIbgN3Aa5kuxhjzlD28/4LdZwPcc5TP4DfAChG5GlhF2tP4IM8A3xSRjVhTHavtNpvtOfk/2+snmoClwOPAwyJyOfAd4K+xAoWNWL+bV7EWCg52LXCDiMSBRuAfjtL/484lQtKYw6alPR8Oc0FBARVeLylj6E4m2dzdzc3l5cSN4YGmJsb7/cwPBjkzL4/uZJKHmpup8fvZFYlwa0UFThFSxrAjEsHncHBTeTn10Sivt7ezKC/P2q+8nJZEgvvs9MI+R6oWGEkmubm8nLZEgufDYWoO0w+wihNdWmTNeX+xtBS3w8Hqjg62dHczOSeH9V1d3JLW13nBYH8p4AOxGC3xODdXVNCZSPBkayvXlZbSm0pxeihEkdvNn5ubWZKfz8qDB9nS3c2Wnp6TbnrlobvqufprA6dXutoTvPJUC9FIimQK5iwMUVrp4Y3nwlx87cczvRJt6ICkOezN8GTjG1NA6bWzMYkUjfeuITCjglhjB22rdmFShkS4h1R3jHhrD8G51sJGb1WedZ1pnEEviY7Mo1eeCivzxBnykeqNk4olaHl8G4n2CKneBIGpVqaKuzAHp99Kgc3Yh4Pd5M6xsmU0EyN7HPU3dKT5D7sWwI/s/wa/9zPgZ4O21QEXZdj3UeDRDNu/c5h2fwn8MsP2nwA/OVx/0/ZbBizLsH1x2uuDWMEN9hx7+sqi/21vfxl4Oe2YKHDxYdp8Gnh60LYdg84L1s198LG3D/r5n7FGMk6YhP0kfLi0tI5Eggp7GNwhQjiRoDkeZ3nafmDV9X/PLqDTZae6nWvn2DuAxfn5NMfjbO/pYXevNfMScrloSyQo83gQEYrd7iH1AvqqBWZS6vHgFOmvFni4fsChyn6xVIrHW1poTyToTaWYGggQTiQot8/Vd53pmuNx9kajLGtoGPC+z+GgyK5HsCg/n1fa2nCLcJ492vFpmF55bkUzp59XQOUYH6mkoXHfxz+9EmvsxMSSwwoATqY5aXE5rMeMZIr21+sovHgy7uIADXe9jTHWzTnW0IG3MkS0vmNglSogMK2c5pWbCEwrw+F1kWiLpCVtpKd/QuSDFpxBLyVXzaBj9W6SETtjI+3zzdiH4gDRvW14q/L6PztxyoBMjJwppfjGWKWoTTJF7562IZkYuTMraHu1lp5tTeTOHhg0phuwSDKRouiSKXgrM49+GWNoemA9qd4EpdfNwZkztJbH8RRr7uamm25i+fLlA7aLlRX2j1hTtXHgb4wxmwft81tjzDc4Cnua3G+MWTncftlrxtYYY54Y7jFHoyGaOiavtbUxKSdnQFqa0x4VcIoQcrlojMUo93hIGUOBXX73+tJSxN4P4M32dr5VVUVvKsU99o1yjNfLBL+fjV1drO3qotzjYVogwDl2rn7SGHqSSZricYwxtNo35XR5LhcC7O7tZYzPR8peAxAcVDHQHKEfcOif1w8iEYJOJ1eVlLC6o4NIMkmBfY1915yy/993zmK3mzE+X3+RoL5rTv/3Pc/p5Nz8fJ5qbeVFuyLhR51eOfullzj33HNZ90Y7u3dGiEVTLDy/gNE1fla/GGZvXS9OB5yxtICyai+vPt1Ka1MchwPOu7yY3DwXa15to25HhNIKz5AMwc62BAhUjrGmVxxOoXKMj45wnO7OJE8/2ERbS5yzLixkVI0/Yz9eWNmMy+2grSXOhVeX8NzDzf0VBH/ytx2EQiE61+2nc+0+xCnkL66hc81eUpEEkZ3NlN0wl7ZXa4nsagFjKLpkCp6yIPX/+RbeUfmkoglyppTR/lot4nYSmF5OaP6okf6ZfyR9UwAkU/gnluDwucmZUkbzwxvxlOYiHutvMffUKpof2kj31gO4gl6ceQMrRrqLA+SdOY4D962zsgB8LoounZqxTW91Hu2v1nLg/nU4c71D1hMAmfswt5qDj2yme+sBxO2k/MtzyZlUStNDGz+2TIy+EZLePW20v15H6TWzMu6X7IqBCBW3nHbUc35cgZ+IFGGNBC+1p46LgMpB+ziGc/MHMMY8c9w7eQw0AFDDtru3l+WNjRhjGO3zcXoodNi0tKUFBTze0oKxF8QtLSxkeiBgVe8TodTt5i+Kihjr87GssZEqjwev/RT/x6Ym4saQNIbLiospdbupi0T6Rw9mBgKcGgwy3ufj7oYGKrxecjJUDLyypISnWlp4KRzuzwIYHAD0ydSPdNVeL6+2t3P/gQPkOp2E7DLGc3JzuaehAbfDwaK8PKq9XlYePEh9NMqVJSUUud0sa2hARBjv8w0pOPRyWxt1vb20xONcYk81fNTplZ/85Cece+65zJgf5NQz84h0J3nmoWZG1/jZsyvCF26twOG05mk/3BHB53NwxU3lNNVHWft6O/MW5dn7ldPWkuDx+wZOr3R3JQkEM//T0RtJcuXN5XS0JXjz+TCjavwZ+wFQWunpTylMryD44IMP8vnPf56u9fupuGU+4nRgUobgvFFWSt2C0cQOdBJv6abi5vkkOqO0PrmN0utmk+qNEzp9DO6iHJr/vIniy6fhKQt+4nPS7gI/o3+wZMj24JwqgnOGFqYqv3HuEc8XmFrWP5zfJz1TID24yZRtUPmN0/tfH64PpVcPvAGHFowmtGD0ofevG7p+2z/OyvB2+t3DukFnkuqN9480JLtjQxYbtj69nejeNpr+9B4lV83g4MrNJDqjONxOiq+cQSqa4ODKTThzvXgrQ8SauxGnEG/pwVMSwJnrJVLXiqcsSNFfTM7YBsbQ/PBGUr0J3CUBmDJhcDcvwcq4agQwxrQALSJyE9aoby7wSxH5qTFmnv20PhEowlpY/ixwGdBtjLnSPi7XGHOniGzDSlufirUY/vci8mXgJqAAKzX8930dEZEJWFPxUWCHMeaY0701AFDDUuB284PRozO+N8HvZ8KgCn9Fbjc3lZcP2DYrN5dZuQMySvlchhK+Xx50HMDFRUNzzzNV8EuX53LxxUHV+uBQFUC3w9FfsTBTP9IrC4ZcLr5ZOXRI89RgkFPTKgACA6oYnpWXx1mDhvK/kXaepYWFhONxnguHmWl/Nh91eiVgn2fHpm62v9eFiFXSF2DBuQWseqIFhwNOW5xPuDnOru097N9tTa/khlx0tiUoKrOmVwqK3Xh8AwOiI1UHLCz14HAKuSEn0d7UYfsBUFppXcPgCoL1p9dTW1uLpzyIOK22Bz/VxZu7ie5tt56w0953+Ny4i6yyyfmLxtOxejepWJLg/FH4Rh3+b0V98np3h6m/azWJ1gjlX7HKqrS/XjdksWHB0olWSuU1s+je0ogzz0fJF2bStWE/ne/sITCrkmRnlPIb5yEuB80rN+MbW0jxZdNouOcd8iYUk7+4hvrfriYVT2ZsA6wFlQXnn0Ln2n2ZulvBwBTtdDFjzCUAIvLTtO1bjDE/EZHfY9WXuVBEVtqL5dOVA38JpIDnsW7uK+xAwItVJ+f3afufA9xvBw8fqUiDBgBKnWQ+6vTKjzZYiSPr32zn+m9VEe1NseIea1qjaoyXMRP8vL+xiy1ruygp93DKtADzz7GnV5KG3p4krU3W9Ep7a4JY78DplWCeCwTqd/daawBShgP7ogSCzgHTG31TIZn6AYempfsqCF5wVQnvre7AGMP48eOtOf9kqn8EQJwO+qpgu4sD+MYUUHz5NKutZGrgSQFnno+iS6eS6Ojl4J83UX7T/OPzC1LHRd8UQMfq3UT3t+MpD2ZcbJgu3hrBW2UF1J6qfCK1tdbrsuCAgkWeMisIdgZ9uPte53ow0UTGNlKxJB57DYK3Kg8GJGID1s3/cBVX3z3M9o32//cPej34O71rjTEdAHJoMcxSEfkf9uvBAcOfgB+LyL3AC1j1YI6JBgBKnQSO5/RKy/e+x69+9SuqxvpYsayR8ipP/7f5PfnHJpJxQzJpWHJZMUWlbvbVRVi53Bo9mDQzwNRTg4wa7+PhuxsoqfDiyxn6kHHBlSW88lQLq18KY1Iwe2GIQDDz9EqmfqQbXEEQoKSkhNw5lTTc8w4Ot5O8RePxVudxcOVmovUdlFw5A3dRDg3L3kUEfOOLyF80fsB5217eRXRvm12Jb+wx/276Fqx5Sq0biXdMAfmLxvePTpxo9Xe9TeXXFgzYlmjvpeWpbaQicUgZQgvH4KnM6y9O9HE41iyN4PxRNPz3O+TOrsy42DA9A8Jd6Ce6v53A1DJi+9twF1qjPYMXTg40cKFkpjZ6tjcRa+ggMLWMaH370Fs0PAm8IiK/sdcAFHJoDcDhvhjPHOb14N5mmp+6DViCNcy/a9B7CWPMDwFEZKuI3GeO8cv5NABQ6gQ73tMrt/3KKmmx5HNDpzUu//LQ6ZWzLx46vTJvUT7zFh1+yDw3z8UlXxw6vdKXAuhyO7jy5orD9uP8Kw5Nr+SGXFz7zUNPerffdjsAwVOrCZ5aPeC4ilsPzTPnnTWOvLMGVjtMn+suXDr4wenY9af0GUPbql10rN5N3pknb6XF5hUbKTjvFHxjCjDJFNF9QwqfDsvHlaWRvrJUnA7844vo2daUcbGhI+fQN2nmTC6lZ1uTFRh6nBRfOZNUdGTfjZapjZzJpXRvbqTxd2twFweGBAB2HZtvAQ/an0UM+JsRNTwyK7BSztcD4UHvXSYi3wa8wDPHevOHYZQCVp9+x6MUsPr00FLAx1d6WV/Ayuv/3Roqbj2Nnp0HaX+tFlKG4ILR5M6oIN7SQ8sTWzEpg7cqROEFk+hcv5+u9fshZcg/d4I1//xGHZGdB0lFExScPxF/TZFV0KeuFRxCwdKJeKvzaH16O/GmLnAIxZdPx5XnszIgdjTjqQgR3d9O5dcPBT6JtgjNKzdTcfP8IddxcMVGnEEv8ZYeCi+chL8mcz+aV27G4XYQb+mh5OqZND+8aUh1wcFZGuEXd5KKJHAX+j8VWRpHoqWAlVJKDSEuByaZwqQM7a/sstYWOITG360hMK2c8PM7KLhgIt6KECZuDb3qAAAgAElEQVRlSHbH6N7cSPnN8zHxFE0PrMc/vojg/NHknTmOZHeM5ofew19TRGRXCxW3nta/7iGy4yAOn5vym+YTre+g/fU68haNJ7KrhfJbTyPR0mOlB6ZJdkVxBTOXok5G4pTfPJ9EW4Tw8zvx12TuB4CnMtSfbji4umDO5NJPdZaGsmgAoJRSI2AS1sLEVI9Vxa/x99aXi6Z6YiS7YyQ6evFWWAvKxCEkwhHizV00Ll/Tvx9A96YGut6rBxGSXdY8d8G5E2h5Yhs4rKfqeHMXPdub6N1tjQK7Qj4SbRE8ZbmICO7iAA7fwH/GnUEfiY5eMvGU5iJOR3/lwMP1A8BbaQ3lZ6oumAj3aJbGZ4AGAEopNQJtr9WSM6kER44Hd3GA8hvnWk/BdsaCK+Qj1tiJp9x6snUV+PGUBSm9fo5VJ9/OWGh/80OqvnUGqd4EDfe8A1gLDP0Tiuna2GDV5y8PEphWRv451ncJmGSKZE+ceFMXxhgSrdZNOZ0rzwci9O4OW2sAUobovjacwYEFhvpmfzP1A+hfqpapuqCrIEezND4DNABQSqmj6N0dpnH5uxgDvtH5hE4fY31r3qLxNN67FhFw5HgovWYWBUsn0vL4FoyxKvMVLp1IYHo5jcveRRyCu9QqSOMbW0jjsnfxVOX1f9Nf0x83YOJJTDJF8WXTcJfmEqlrpXG59SQdmFlB8NRqfOOLaLj7HbwVQRwZSuOWXDmDlqe2EX7pg/4sgMEBQJ9M/UiXqbqgM+AZcZZGYEYFsQOd/deS6gsK4qmPlKVxvGTKpgAQkUeAQqzvr/lXYA3wc2PMFz6Oftjfo+Mxxrxz1J0/alu6CPCzTxcBZhddBKhORumLKfuyKRxe50mTTZEeAKQtAgRYZIx5TURcWN/wuo8RBgB9+f1mGDfc9CqBw9jX8VGyAHQEQCml1CdKxB49+d0a8s4cd1JkUwz+0os9e/YAYIx5zf5/AnhdRMYClSLyMFaRnv9hjHlBRP4nVlngPOCHxpjnRWQ5VlmhicA1IvIA4MH6MqGrjDEdInIL8HWs1MLbsaoCForIJcaYi0XkR8CFWJMyf2WM2SQi64A3gTwRWQH8GOtbah80xvzHcH8PGgAopZT6xJ3s2RQNaV8KlkERsAjrq+v/f6yKfP9ujPkXESkGHsIq6wvWN/j9JYCIXG6MiYjIXwPX2tMLtwJnG2Pidmnf/+DQ9wTMACYZY84RkXL7vSuwKhX8mzHmA7vU8M3GmM0jLQ2sAYBSSqlP3MmeTVGZ4bs/0mw2xiREZC+HygZ9SURuxKoMmF5x610AEQkAvxWR0UA+VrGf8cB6Y0wcwBiTGlR0aQpwhoi8bP/c92UaYWPMB/brfwT+1j7/b4DVR+p4Og0AlFJKfeJO9myKUaOswkQicra9BsCJtQZgP5lL+34fmIYVELye9n7fHP1FQL0x5gZ7BKAQqAVmi4jLDigcWNMDfcWHtgOvGGO+avfFPeicAHuNMX8pIlVYXxp07nB/BxoAZAFxOPoXhiml1InyacumwLrRfk9E/gnrpvwLrAAgk1XAa8DbQEeG91djfYnPk0ADsM8Y0ywiy4A3RKQb+CfgLeBeEZlnjPmyiOwUkVc49G2BPx103ttFZCHWVxL/fDi/hz6aBZAFHCLmtsWLT3Q31CfktlWrAM0CUOpYZUsp4JPj66yUUkop9YnSAEAppZTKQhoAKKWUUllIAwCllFIqC2kWgFKfUX2L6ZRSKhMNAJT6jPr+HUtOdBeOyc9vW3Wiu6BUVtApAKWUUioLaQCglFJKZSENAJRSSqkspAGAUkoplYU0AFBKKaWykAYASimlVBbSAEAppZTKQhoAKKWUUllIAwCllFIqC2kAoJRSSmUhLQWslDop1d327InuglKfaRoAKKVOSkvuuP5Ed0FlqVW3/eFEd+EToVMASimlVBbSAEAppZTKQhoAKKWUUllIAwCllFIqC2kAoJRSSmUhDQCUUkqpLKQBgFJKKZWFNABQSimlspAGAEoppVQW0gBAKaWUykIaACillFJZSAMApZRSKgtpAKCUUkplIQ0AlFJKqSykAYBSSimVhTQAUEoppbKQBgBKKaVUFtIAQCmllMpCGgAopZRSWUgDAKWUUioLaQCglFJKZSENAJRSSqkspAGAUkoplYU0AFBKKaWykAYASimlVBbSAEAppZTKQhoAKKWUUllIAwCllFIqC2kAoJRSSmUhDQCUUkqpLKQBgFJKKZWFNABQSimlspAGAEoppVQW0gBAKaWUykIaACillFJZSAMApZRSKgtpAKCUUkplIQ0AlFJKqSykAYBSSimVhTQAUEoppbKQBgBKKaVUFtIAQCmllMpCGgAopZRSWUgDAKWUUioLaQCglFJKZSENAJRSSqkspAGAUkoplYU0AFBKKaWykAYASimlVBbSAEAppZTKQhoAKKWUUllIAwCllFIqC2kAoJRSSmUhDQCUUkqpLKQBgFJKKZWFNABQSimlspAGAEoppVQW0gBAKaWUykKuE90B9dmzp72dl+rqMMYAsKC6mqklJcd0rnf378flcDCnomLI9o0HDiAi5Ho8XD5pEk/t3MkZo0ZRlpv7ka8hkzX19cyrrPxYzq2UUp80DQDUcdUTj/Pkjh3cMHMmQa+XZCpFfWfncW1jV2srO1pa+Mrs2bgcDhq7ukjawcZIpYzBITKsfdeOIAAYyXmVUupE0ABAHVc7WlqYXFxM0OsFwOlwMCovD4DacJgXa2sBmFdZyZyKChq7unhyxw4MMLGoiEVjxtDW28uKrVvxulz4XC5qCgoGtLGpqYkzRo3C5bBmsMrTnvjf3r+fcCSCx+nkuunT6Y7HWbF1K0ljCLjdXD1tGu29vazcvp1cj4fKYJCgx8OGxkYiiQQLq6uZVV5OdyzGI9u3E0smCXm9TC0poSUSYdn69cyvqqIyGOTJHTtIpFJUBINcNGEC6xsa+KC1lVgyyYLqat7et49YMokBvjxzJm6n8+P/BSil1DBpAKCOq65YrP/mXxsO88qHH+J1ubh+xgxerK3l+hkz8Lpc3L1uHdNLS3mxtpbLJk2iOCeHe997jxmlpby5dy/njB3LhMJCVmzdesQ2Bhudl8dlkyaxYutWDnR3U5yTww0zZ+J0OHjmgw+oC4cp9PvpjEa5cdYsXA4HsWSSWeXlJFIp7lm/nlnl5by2Zw+nVlQwpaSk/2n+1d27uXnOHAD+tGULl0ycSKHfz1M7d7K/owOwAp4vTZtGaySCy+HgSzNnYoxBRjgaEI5E+OXbb/PVU0+lOhRiZ0sL+zo6WDJu3LDP0R6OEG6JMHZC4bD237y+gXgsyZwF1QO279zWzJo39yIieLxOln5uEpvW1lNWGaRmUvGIrmu4li9fTiqRxOHSoEmpj4sGAOq4Cno8tEQiAIwvKGB8QQG/XbMGAAMEPB4Aivx+OmMxuuNxSgIBACqDQcK9vbRGIlQGgwBUhUIZ2+iIRinOyRnyXoU9GhDyeulNJIjE4zyxYwe9iQSdsRjlubkU+v2U5eb2jyDUhsO8tXcvAC09PQAc7Onh7NGjATIO5R/s6eHR7dsBiCWTjMvP778GgEK/n7H5+azcto2Q18uSceNGPCVQkpPDG3v2cO306SM6rk9HWy8f7modEgCYlEEcw+tLS3M377y+h6tvnIXH66I9HCGVTB1Tf0bS7vLly0mdUTKsAGAk51VKHaIBgDquTikq4o0NG5hXWUnI6yWVNjcvQHcshtfloiUSIejxEHC7abaf1Os7O5lXWUmh309DZyc1hYXUd3b231z7TC8t5a29exmTl4fT4eBAVxehDCMCxhg2NTVRU1jIaVVVPPPBB/0LE9NvFy9/+CE3zZ6NU4Rfvf02AMU5Oezt6GBycXH/CED6U3xxTg4X1NSQ7/NhjCFljLUo0X4/kUoxv6oKhwiPvf8+e9rbGTvoOo6mOCeHlDE0d3cP2L6zpYXX9uwhZQwLqqqYUVbGym3b+hdA/vCHP+Siiy5iw5p66vd2cKC+k89/cQZ/uHsd5ZVB/AE3YycU8s5re4jHkkyYUsyCs8dk7MOOLc3MmleJx2v9U5FX4O9/b/vmJja8W088luSqG2bicjlYcd9GkskUDqeDy66ZhtfnYtmd7xyx3Xg8ybOPvk93ZxSHQ7jqgrfYsGED1HkomTqK0hljef+xt0lG43iCfqZccTptu5vZ++Z2RISSqdWE65roDXcBMPmK0/EXfDwLQZX6LPnEAwAR6TLG5A7a9k2gxxhz7xGOuxv4hTFm6JjwyPtQDvwbMB+IAh8C3wUqge8bYy79qG2MoC8/Msb89JNq7+OW43Zz6cSJrNi6FRHBIcLp1daQ8nnjx/OHTZsAWFBVhdvp5Lzx43ns/ff71wAU+P2cOXo0K7Zu5c29e/G5hv6J1hQW0hqJsGzDBhwiBD0eLps0KWN/xuXns3L7dna1tuJ2OimzRxvSTS0pYfmGDVTk5va3d/bo0TyyfTtv7d1Lvs/HFVOmMDY/nz9s2sTcigrOHz+eJ+w1AAJcPnnygHO29fZa12UMXperf2RipM4YNYo39+7tz6JIGcMru3dz0+zZOET43YYNTCstzXjs7HmVBENeFl84AYCujihLbp2Dz+8mHktyzU2zMcbwwN3rOPX06ozn6O6KUVE9dBQGoLAoh4WLx/L6i7Xsrg0zYXIxl39xOm63k3Wr9/H+liZmzq08arub1jZQURVk7sKpmJRh4cKFzJ49m+QZJbi8bj54dj3VCyZSML6cvW9uo3nbPtw5XpLROLNvPg+TMux/9wNO/epSRASTOrYFoUplm5NiBMAY85/D2Oerx6MtsR7jVgK/M8ZcZ2+bDZQdh3O7jDGJER72I2BEAYCIOI0xyRG284kZnZfXP1eebnxBAePnzh2wrTw3l1tPPXXAtnyfb8i2weZXVTG/qmrAtiumTOl/vbSmpv/1t+bPH3J8+rD6ojFjWDRm4BNwwOPhSzNnDth2Qdo5AW4Y9H56qmJxTg63ZPgMRmpMfj6rPvyQzlgMsLIsWiMRfv/ee/0/d8diA0YnzGEyIvIL/fj8bgCaGrt4c1UdqZShLdxLpDue8ZhA0ENnRzTje6UVVlATDHmJ9iaIx5I8//j7dLRHifYmOGVqybDabT3Yw/Q55QAZh/J7mtvp3N/Ch69sIZVIUjZzLO4cL8HKQkQEcQrVCyby/qNv4/S6GXfuTFxe95E/WKXUyVEISERuF5Hvi8gUEXknbftYEdlov35ZRObZr7tE5Cci8p6IrBaRMnt7jf3zuyLyDyLSlaG5JUA8Pegwxmwwxrxm/5grIg+LyHYRud8OGBCRv7fPu1lE/itt+8si8lMReQX4GxH5nIi8LSLrReSFtL7lisgyEdkkIhtF5CoR+RngF5ENInK/vd8NIvKOve23IuJMu+Z/EJG3gYUi8jMR2Wqf6+fH8/ehTi6nV1ezet8+wBphKc7J4cZZs7h5zhy+OW8eQa8Xn8tFR9S6Ua9btw4Ah9NBeiyQvgThndf3cO7Fp3DNV2YTyvMeNmiYNK2UjWvriUWtuLajrZf2cGTIfsYY6j5oJRD0ct0tc5hxagV9jR+t3cLiHOr3Woso+57e3W53//E5xSHGnTeTOTefx9yvXUDlvAkDLsikUpRMG83kz5+OJ+Dj4LZ9w/5slcpmJ8UIQB9jzDYR8YjIeGNMLXAt8KcMuwaA1caYH4vI/wG+BvwT8Evgl8aYB+xphUymA2uP0I05wDSgHngDOBN4HbjTGPMPACLye+BS4HH7mHxjzDn2ewXA6cYYIyJfBX4AfA/4/4B2Y8yMvv2MMStE5NvGmNn2tin2NZ9pjImLyG+ALwH32te82Rjz9yJSCPw3MNluZ8jksoh8Hfj6Ea5TfUpMKiriBTt90iHCojFjuPe99xARctxurpk2jdnl5azcto136+upOe00AIpLA7z2Qi2PPbiZiz4/cIrilCnFPPHwVopLA7g9h19oV1icw2lnjmbFfRsREbw+F0svnZhx34rqEG+/ups/37+RQK6HYGjouoxM7c6YW8Ezj2xnx9ZmXG4H/3oHXHbZZdz2y59SOm00YxZN4/3H3uHDVdb00filswecMxFNsPmBV/uzLaZ+4YxhfrIfn0i4i3V3PUdOSR7JWIJRZ0ymbEbmdRbpWnbW07GvhXFLZgzY3rxtb/+aB6fXzaTPzad+7S6ClYUUT6o6zNk+mob1tZTNGKOZGJ9hcrjI/2NrMPMagNuBLmPMz0XkR0DKGPMzEVkHXGuM2SkiL2PNz68RkSjgs29+1wJLjTFfFZEWoMwYkxCREFCfoa2/BsYZY/42Q98WAz82xiy1f/4P4A1jzH0ichXWzTwHKAR+bffxZeA2Y8wr9jEzgH8FKgAPUGeMuUhE1gLXGWN2Hu7zEJFvY00JNNlv+4EHjDG3i0gC8BpjkiLiwgpi1gBPAk8YY2KH+8wdIua2xYsP9/ZxEY5EeG7XrgFD66/t3s300lIK/P4h++9saSGRSjHlGCsEpozh1d272dXaClhD7pdPnsyy9ev7Uw0/Dp+GaoC3rVoFwPfvWHKCe3Jsfn6b1f8ld1x/gnty7CLhLnY9t4Hp155FMpZg/T0vMO+bFx31uEwBQHdzB9sfWc2sG5fg8rqJ2IsdGzfUjTgAGEnGxPplLzLj+kXDmk75rGVirLrtDwCISMoY85mNgE6qEQDbg8BDIvJnwAy+Ydri5lDkkmRk17EF+MIR3k+f8EwCLhHxAb8B5hlj9toBiy9tv/Rl2r/GWqz4mB1Q3G5vF6xMuCMRrLUJ/zvDe7198/52gHMacB5wHfBt4NyjnPsTd/aYwz/xnFJU9JHOvb6hgc5olFvmzEFEqAuHj/lcWg1QfZyS8SQOt3UPad3VwJ7XtpKMJSieMooxZ08lHomx9aE3QMAbysEbGpje2rxlD5XzJvTfiNMzHJo276b+3Z0kYwlm3rAYh8vJxvteJpVM4XAK0645G5fPzTt3PkmwshB3wEvhhIohfUjGE7z/6NtEOyOIw8G4c2fS1Rhm0/2vaCbGZ9hJFwAYY3aJSBJryPzBER6+GrjKPu66w+zzEvBTEfmaMeYuABGZj/Vkfzh9N/uDIpKLFUA8fJh984D99uuvpG1/DutG/V27zQJjTBiIi4jbGBMHXgQeFZH/a4xpsof6g8aY3ekN2H3IMcY8JSKrgQ+O0PcTpi81bU19PbPKy/sL2uzt6KDA5+uvmHfnO+9QkZtLc09PfyW+hs5OHnv/fYJeL4I1Dz4urSLgpqYmLp80qX/xW/p7L9XV0dDVRWkgwKUTJ3Kgq4unP/jAqtqXm8slEydSFw7z5l6ruM3UkhK6YzF2trYSTSQ4f/x4agoLaenp4YkdO0gZQ1UoRIHP118NcMm4cbgdDp6vrSVlDJOKijhz9GhW1dXR1ttLdzzOWfbPQH8xJJU92nc3sX7Zi0RaOhl3rvW7zxtVwuybzsMYw7q7n6f69Ik0rNtFydRRVM6bQO2L7w05T6wrQqg6c8CcUxRi7OLp1L74HuHaRoonVzP9i2fjdLvYt/p9mrbspnLuBKIdPcy5dSluv4dkLDG0D2t3EawqYurCyf1P87nlBf0jAJqJ8dl0IgKAHBFJX6Xziwz7PAj8CzD8smeW7wL3icj3sIbG2wfvYE8bXAH8m4j8EOjlUBpgxrE0Y0ybiNwFbLL3ffcIfbgdawRjP1ZA0ncN/wT8u4hsxhpZuAP4M/BfwEYRWWeM+ZKI/B3wnIg4gDjwV8DuQW0EsQIFH9aowZDpjJPJ9NJStjQ1UR0KsaW5mTNGjeqvnAdWZb9LJk5EgN9v3Mis8nJeqqvjqqlTKfL7WbZhw5BzHqka4JSSEi4+5RT+e906ehMJCv1+vjJrFiLCHzdv7i/2E00muXn2bESEWDLJmaNH0x2L8dDWrdQUFvJ8bS0X1NRQEQz2P82va2joz3D43YYNXDttGn63mz9u3kyXvVI/z04brA2HqQwGuXDChAH1EEZqpNMrzzzzDJHI0IV6w2VShrde3c3uXdb0SmFxDhdePpkHl63niutn9NcEON42rqln5ryTe3plJPLGlDL92rNIJZK8d+8qSmeMoasxTN2qTZhUit5wF/HuKJHWLirnWhkmoaoiOhsGjmZ5gn6iHT0Z28itsAJfbyiHRG+MZCzB+4+/Q7S9h0RvjJKpowDwFwZx+60iXJn60HOwg/I54wHNxMgmn3gAYIw5auaBMebnwM8HbVuc9jo37fXDHHoa38+hBXjXYc2RZzp/PXBNhrd2Ai+n7ffttNd/B/xdhnMtHvTzo8CjGfbrYuCIQN/2/wX8r7SfHyTDyMega24ATsvQ/5PS6Lw8nq+tJZFKEY5EKA0EBgQABT5ff/5938xOdzzeX+mvr7peur5qgIUZboCDqwHGk0me3bWLeDJJuLe3P6WuMhjsH0HYdOAA79mFfPpu5B3RKBV225mG8g90d/PHzZsB6E0kaO/tHdDfMXl57G1vZ+W2bZQGApxpVxY8Ho40vXLRRdZc8xsbf3VM5960voHuzijX3WJNr+ypO/bplZHMDW9cO/wA4NM05+xwOTHGYJIp9ry+lVMunktOcYi1dz2HMQZ/YS6dDWGClYV01rcOTJsASqeNZtvK1ZRMG43L66a3rTtj1oYx0PpBA96gn6lXncG+1e8Tj9hLg9LOmakPOcUhOvYeJFRV1P/ZOpwyIBOjeEo1+WOsmhOpZIr2Pc1DMjHKZo5l96tbOLhtH+WzD//8lr5IMpVIMvGSeQQrM5esNsaw+YFXSfTGmX7d2bhzMgf+x0t3cwc33XQTy5cvH7BdRM4G/hErey4O/I0xZvOgfX5rjPnG0doQkYsAvzFm5XD7ZU89rzHGPDHcY47mpJsC+IjmAnfaKXptwC0nuD8KEBGqQyFe/vBDagqHV5c+4HbT0tPTXxVw0qA1A9NLS3ljzx4unTgREeHDtraMlfaMMbxbX89pVVVMLCrij5s3Z6wG+ObevXxr/nx6EwnuWb8esAKIxq4uynNzM1YDLM/N5Zpp0/C5XKSMQbC+DKlvj5QxnDN2LAD3vvce00pLyfelLx05dkeaXhmzfDldXdZc7LI736G0IpeW5h7mLaxm6qxyDjR08vxj7xMIehGBuadXM2rcoSmU7ZuauPDyQ9Mro9Pee+OlOg40dFFcGuD8SyfSfKCLVU9/QCKRoqwil/MumcjeujBr3twLIkycWkJPd4y6na1EownOPn88Y2sKCbf08MITO0ilDOVVIfIKfLS2RHhw2Xo+t/gVAoEAG5a/iEkZiiZVMfrMKdSt2kRvm/XEOvqsqdTZWQEur5sZ1y86Lp/r8dQ3BWCSSYomVuHyeSieMoqtD79BoDQfp50FUXFqDVsfeoPmrXvwBv148wYWq8opDjH6zClsvO9lRASXz83ES4fWtgAIVRex+9UtbLz/FTy5viHrCYDMfZhbw/ZH3qZ5614cbiezvryEoknVbHnojY8tE6NvhKR9TzN7Xt/KtGvOyrhfrKsXRJhzy/lHPefHFRiKSBHWOrClxphG++fKQfs4hnPzBzDGPHPcO3kMPlMBgJ3LP+tE9yNb7Wlv53f2cP2EQTf6aSUl3LN+Pd9ZsGBY51oybhwPb91KrseDx+nE6Rg4cDSnooLOWIx71q9HRCiya+9nMrGoiGc/+ID1DQ2HzXcfm5/Psg0bqAoG+zMIlo4fz+M7dmCMoToUYmlNDUV+Pw9u3swZo0Zx/vjxPLh5MwZwinDdoJr9+zs7ebG2FoNV3ChTueKPKtP0SrrurhjnXzIREXj49xuZOqucN16q4y+umkpBkZ8Hlw2dXunuihEIZu7rKVNKWHLxKTzw3+uI9ibIL/Rz9Ves6ZVH/7iZcIs9vRJNcu3N1vRKPJZk/pmj6emO8cRDWxlbU8irz9ey6IIayiqC/f9ob17XwLU3z+Gcc87hvPPOY9q1Z+P2e9j8x9eIdVlTGr68AFOuWEi4tpFgZSETLpxzUs43+wtyOfMHVw7ZXjFnPBX2UHu6WTceOWOjZOqo/uH8PumZAlXzT+l/nSnbYN43LjxqH6ZdfeaAn6sXTKR6waGUz+nXnT3kmIJxVv00t98zrBt0JoneeH+9ilh3dMhiww+eXkvH3oNs+dPrTLnqDLavXE20swen28WUKxeSiMbZvnI1nlwfwcpCups7cDgd9LR0EigJ4cn1/7/27jy+qvLO4/jnR9gCN2RjiywGkEXABipuWBUV7Fi3arXtjE7B2lZrp7bT0m1qW/vq5tRpa2vrdJWiXVDrUm07VRAQqqIsCRDAhU1QEMqesIUkz/zxPDc5yb03G0Ei5/t+vXhxcu655zz3Oc8953ee7bJ7w1sk+uUz/H2npz2Gc7D6T89TfaiKnn16MfrUlB+5ugx41Dn3FoBzbiew08ymAZcCCeDHZvZd59yE8LQ+AijEdyx/CrgS2O+cuya8L+Gc+6mZrQGWAaPxHcgfMLN/B6YB+cCPnHMPJBNiZqcAD4T9vuqca/Nw7xMqAJDjJz87my+ce27G1wfl5hIdihidNe/mCRPqlj8eZgrs17MnN0+YQK1zzCwrI7/Rk3MnMyYVFzMpPGEnRWcgvG7MmLq0nXJmaotJtOPgFWmmEi7s0YNp4xo+6Xxg9OgGf09t9Hr01/qK8/KandHwaKVrXonKze9Ot+7hax6usgf2H6Ggt38y7HdSavNKIqcrlfsOk1eQ2rySMvvfkRrmP7WO6iM17N19iMoKX+Xc/6T65pU1K7exevk2zHxwAVCx7zD9ivyx0z2xrVy5kqp1fn31oSoO7fWBRbKaOPfkvuzdvIM1j71Az755DD731JR9SMe29/XtLP3V0xzcVcG4qX4Q06Z/rE7pbDh0yjjWPV3GmA++h+2rNtEttwejr53IW2XrefOlV+lXMoTDFQcp+ciFdOqcxZrHFpFX3JeRV55J6X1zKDiliC5ypJ8AABaeSURBVOJJY1nyi6eoOVKT9hgAPQpzGDq5hC1L0/apLsLPDZNOlXPuMgAzi87quso5950wb0xn59x7zewxM2s8kUZ/4JNALTAbf3N/JAQC3fDz0DwQ2f4C4PcheDiqyfwUAEiH9Ma+fczdsIEjtbWMLCzM2OEv7pprXklXGdqzZxd27zxAXkE227dWcMrIhs0rI8f2ZfFzm5h8uW9e2bxxD4OK0zevlC3ewvgzBzB0RCF/nlVeF2RED7zk+c1Mu/UMDh2qZtZ9vnklp1c3tr9VSd/+ifpq20jzSklJCYcm5NK5e1dcbS2YsfPVLQ3anIsv8DUuy++fR98xg+mel/o7D9JxJZsA3lj0Cvve3Emif37azoZRB3dV0muAL685A3qze/02ABL98hpMWNSzny+vXXOy65cT3ak5XJX2GDVV1XXBZa8BhZDah3YLMDxlrZepU/iK8P+bjZbzG2233jm3D+qmqgeYYmafC8uNA4aHgK+a2f3AHPxEcW2iAEA6pJPz8tL+nkBctWfzysQLh/DXP62mZ6IrXbpm0Smr4UPEaeOLqKyoYtZ9vnklvzA7bQAAMGxEIfOfWkt5aebmlUHFecyaUUbRgJy6EQTnTxnK7Cd980rRwF6cP2UYBYXZPPFgOde+dxF33nknk6+7DJzDsjqlVD9XvLnLD5lzju55ibRt3W+n6MQ/Sa8vXE3fsYPTjoff+doWaqtr6HPqoJTXWsLVOl5fUM6udW8Bvp/AqKvOatXkPW2xZcna+qmY28lJZwyn9Dez6T9uaNrOhtERENkFCfa9uZM+owdR8eYOsgtCDVYL59twLn2Hxh0vv0HF1t1+v1t2QX7KT8P8FXjWzO4NfQAKqO8DkOn3sV2G5caJTffF+QZ+2vrDwLpGr1U7574MEKaD/51zrk2/0a0AQKSDa23zyrRp0wCY/s1HuOHm+uaVf/u4b17p3a8nN9w8AVfreGhmGbn5DZtXrJMxcVIxEycVN1j/oRvrA7LLr/PNK7n52Uw7JbV5JdqpcMoVqc0r+YU9+OC0hs0n7/uAb145++yzAeqqhZOibd55xX15901TUvbbkZx83uiMrxUOP7rhjltL13G44iDjPzoZM2P3hm1t3ldrOs5tWbquxQFAS/fbKasT+UP7sWPN5rSdDaO9/nuPGsiONW9Qet8csrrW9wFojXTH6D1qINvLN1E2cy49evdKeUZ3zu0ys1uBB8NDehXwmVYduHUeAeYBpUDjYThXhlljuwF/b+vNH47DVMDy9ns7pgKWjqO5qYDfeH0Pz83dwJEjtZwyspCzLyh+G1PXvHfiVMDpagDWPLaIQRNHsWXJWvqXDKHXwEI/1e/mHXTPT1BTVc3As0bw0k//SqLIV38PPGcU/UuGULF1F688sZhuOdlgMPDskXUd7gDKfvsMI686K6V2oXTGMyT651G5dTc9++Yy4vIzqNy2h7X/t5Ta6loSRfmMuGwCuzdsazCLX9X+w+x6bQvVh48wdHIJBcOKOLCzglf/shhXW0uvAYV0z0+wfnYZiaIChlx4Gp26ZLF+dtk7eqRGJpoKWEROSANPzmvwNC/HVt+xg9m+6nV6DSzkn6s2MWjiqex7c2fd61WVBxlx2SVgsOKBefQvGcKGuSsZ/YFzyC7MoWzGMyn7rKo85IODNPqcOojhl57Ost/MpvpQFdkFCUqmXoSZUT5rIQd2VgDUzeJnZtRUVTP43FOp2n+Y1Q//g4JhRayfXcawS8aRU1RQ9zS/ddl6xt94MQBlM+e+o0dqiAIAEZFjKndwH9bPLqO2uoaDuyvp2Te3QQDQPT9B5+6+zT5ZIXtk/yFfFQ1pJ8jxswMeJLsgtX9Bw9kBj1BzpJp1T5VSc6SaQ7v3U1VxsG6/yT5n21ZuZNvyjWBh3D1weN8Bcor8sdNV5e/ftofyWf5X1DVS453pqIYQiIhI08yMXgN7s3F+OQXDitJtkbKmS8/uHNhZgXOOiq27Ul7vO/ZkNj23uq7j5Z6N21O2AT9SY8vi1xhw5gjG3ziZnJMK6jtrRjrObX7+ZUqmXtRgMp5uvXpQ+ZZvfk4+wUf72iX653Hav57H+BsvZsLN760PVBqN1Dj16nPYve4tDu2J/maadASqARARaQd7N22nbOZcAApOaXij7zNmMKX3zeGsT1/Won0NufA0Vv/pebomupPVtUvKSI2i8UOpqjhA6X1zMDOyC3PIK+6bdl+FIwaw9qllbC1dl3GkRl5xP8pmzCFnQGHdCIKhU8bx6pMv4ZyfYXDYlHFkF+ZQ/uBCBk08laGTx1H+4D/abaRGRxpJUfn5ShKJY/Nrhmb2CefcL4/JzltJnQBjQJ0A46W5ToAd3TuxE2B78z/n2wlX6yibOZfR107M2OZ/okgXABxLW5aupWLLLkZcfkbdSIr8If0onfEMb5SvI5FItLgTYJgGuEW98c1siXNuQvNbtm6/baEaABGRDmbfGzvZMHc5tUdqKBw54IS/+WdyLEdSbF/5OiOvOquuH0T0tdtvv51ly5ZBaJ8xs9OAH+N/Gn6Zc+4/zGwSMB0/D8AjZtYXPy1wLvBl59zsMG3vL/D32peA9cBIM5uPH+u/H//Lt52BPzvn/idMIzwE6AvcaWbfDMna55y7sj3zVwGAiEgHk3dyH8bf2La59U9Eb/dIimuuuYa7774b83KBtcDF4ZdmHzWz5KyAvYALwvoezrm7zKw38DB+Wt+7gOnOudLk07yZ3ZT8FVkzewa4xjm3O0wTnJzyd5NzbqqZXYz/BcDpRzvtbzrqBCgiIh1a7uA+7Nu8o8FIiqjkSIrO3bqkjKQwsyZHUqQzvuEspHlAMfA3M3sWmED9LIBLXH07+vVmthA/iU/y9YHOuVKADFX5pwGPhRqBoUCyQ0NyeuFngb1mNhP4XOrbj44CABER6dDe7pEU1nBqYQNuBe5xzl0ALIkcMHpTn46fvvfayLrNZlYS9pm830Y73i0Hrgo1Au8Gljbabxfn3Lecc1OBS8xscJoP32ZqAhARkQ6ho46kAJ4EfmRmNwGZOgXOAxYCLwL7wrovAr8KP/LzIvAl4BUzewTfPPBl4NEQHFQB72+0zzPCLwxmARuAN1r04VtIowBiQKMA4kWjAESObiSFpgIWERF5h9JIiuYpABARkROORlI0T50ARUREYkgBgIiISAwpABAREYkhBQAiIiIxpABAREQkhhQAiIiIxJACABERkRhSACAiIhJDCgBERERiSAGAiIhIDCkAEBERiSEFACIiIjGkAEBERCSGFACIiIjEkAIAERGRGFIAICIiEkMKAERERGJIAYCIiEgMKQAQERGJIQUAIiIiMaQAQEREJIYUAIiIiMSQAgAREZEYUgAgIiISQwoAREREYkgBgIiISAwpABAREYkhBQAiIiIxpABAREQkhhQAiIiIxJACABERkRhSACAiIhJDCgBERERiSAGAiIhIDCkAEBERiSEFACIiIjGkAEBERCSGFACIiIjEkAIAERGRGFIAICIiEkMKAERERGJIAYCIiEgMKQAQERGJIQUAIiIiMaQAQEREJIYUAIiIiMSQAgAREZEYUgAgIiISQwoAREREYkgBgIiISAwpABAREYkhBQAiIiIxpABAREQkhhQAiIiIxJACABERkRhSACAiIhJDCgBERERiSAGAiIhIDCkAEBERiSEFACIiIjGkAEBERCSGzDl3vNMgx5iZOaD2eKfjODBABfzYUN52bDo/7aOTc86OdyKOlc7HOwFy7J3IBbgpZrbEOTfheKfjRKS87dh0ftqHmS053mk4ltQEICIiEkMKAERERGJIAYCcyH55vBNwAlPedmw6P+3jhM5HdQIUERGJIdUAiIiIxJACAGkTM+tnZn8ws/VmttTMXjCzq9tp3/PNLKUHs5l1MbM7zew1Mys3s5fM7NLwWmV7HLsVaZxmZie1w36cmf0g8vd0M7ujjfvKM7Nb2/jejWbWO836hJn9wszWmdkqM1tgZmeZWbGZlbflWG1lZp81sx5v5zFbwsxqzKzMzJab2TIzm9jK999hZtMzvPaRUNZXmdnq5HaZviPHiplNasPnSuZLuZk9aWZ5LXjPbWa2xsx+3/bUHp1M1xIz629ms8J3YbWZ/c3MRmTY9vlWHvMWM/tIG9Pb5u+iAgBpNTMz4HFggXNuqHPudODDwMBjfOhvAUXAWOfcWOAKIOdod2pmbRkOOw1oVQCQ4TiHgWvS3XzbIA9IGwCYWVYb9/lrYBcw3Dk3Bv+5jzqtbczzzwKtCgCO4nO3xkHn3DjnXAnwFeB77bHTENx+Frgk5P27gb3tsN+25MkkoFUBAPX5MhZfhj7VgvfcCrzPOXd9Sw7QxnLUauGa9xgw3zk3zDk3GvgvoF+j7bIAnHOtyivn3M+dc/e3V3pbc2D9079W/QMuBp5t4vXuwAxgJVAKXNjM+mxgFrACeBB4EZjQaJ89gJ1ArwzHrAS+AywHFgH9wvorwv5KgTmR9XfgO/g8DfwBKAYWAsvCv4mRfX8xpHk5cCdwbTjeK0BZSP/pwLPAUuApoCi8dz7w3fDa54HrgPKwrwVhP18BvhO2nw7cEZb7AI8Ai8O/cyNpnx5JX3lI/yzgYEjTXfiL9rzw+VaHbR8PaVwFfCKyj41A70Z5OgzYAGSlye9iYA3wq7Cvp4Hs8NrHQ3qXh/T3COt/C/wwpOkHwJnA8+HcPA+MDNtlAf8T8nwF8GngNqAqrJsXtrsEeCGcr4eBROSzfB34Bz4wvQ1YHfY16xh8Hyojy9cBj4flBPBMSN9K4KrIdl8N5WcO8Mfo+YxsswC4KMMx5wP/DbwEvAqcFzkvKeW4lWXhX8J7l4f0FwNvAW+GsnUeTZfN5PfqCDAmpHEzsAMfSAJ8IbxvBfDNsO7nkXP8n0BBSOMK/Hf6XRm+u9PCdk/iy+t/AJ/Dl6tFQEGkPP89fOaFwKiwfkgoR4vxDxmVafL7IvwDT7pzkS5vKyOvPQs8FM7TncD1IU9WAsMaf6fbcG6LgfKwnMzvspBvw5ssu+39ZdC/E/8f/oL6oyZe/zwwIyyPAjbhb/6Z1n8OuC+sfxdQTWoA8C6gtIljOuCKsPx94PawnE99Z9ePAT8Iy3eEC0HyptUD6B6WhwNLwvKl+JtT8iaWvJjMT6YR6BK26RP+/lDk88wH7o2kcyUwICzn4QOAXvibVi4NA4A/AO8Jy4OBNZG0pwsA6i4EYf0kYD8wJLIumf7s8L7C8PdGUgOAK4HHMuR3cThP48LfDwE3hOXCyHbfBj4dln8L/IUQUITP3TksTwYeCcufxN9cOjdKc10a8bUQC4Ce4e8vAV+PbPfFSBq2AN2SeX4Mvg81+Avuy/gn9NPD+s6EgDWkdy1+hr7TQznoEfJgLekDgF1AboZjzqe+LL8PmNNMOW5RWcDf2Dcnt4tscwcNy1xTZXNp2GclcA9wAz5AuzysvwR/Azd8LfRfgPPTnON7gG+E5YuAsgzf3WkhD3NC+vcCt4TXfgR8Niw/Q30AchYwNyw/AXwkLH+K9AFAxmtehryNBgB78DWX3fBBVDLg+Qxwd+P8bcO5LaY+ALgHuD4sd03mUaZ/mglQjpqZ/Qx4D1DlnDsjLN8D4Jx72cxeB0Y0sf584Cdh/QozW9GGZFThLyTgLw5TwvJA4EEzK8J/ITZE3vOEc+5gWO4C/NTMxuEv6Mm2vcn4oOVASN+uNMceCYwFZvuaQrKArZHXH4wsPwf81sweAh4N+9xnZvfjLzIHI9tOBkaHfQL0MrPWNnm85JyLfubbIn01BuEvJDtbuc+kDc65srC8FH8hAhhrZt/GBzgJfI1I0sPOuZqwnAvMNLPh+ACuS1g/Gfi5c64aMub52cBo4LmQP13xT3FJ0TxfAfzezB7HPym2t4POuXEAZnYOcL+ZjcXf4L5rZufjp+IegK8yPg8fWB0I73mijcd9NPwfzftM5RhaVhb64J90N0DGvIemy+YTzrmDZpaNr4G7Ff+9uz2svwQfBJSG7RPh2AsaHeM9wAdCOuaaWaGZ5UaPEdl2nnOuAqgws7342gDwgda7zCyBb8J4OJLmbuH/c5PHAR7AP323VuO8jVrsnNsKYGbr8DUXybRdmOE9rT23SS8AXzWzgcCjzrnXmkq0AgBpi1XUf2Fwzn0qtGEnp83MNPVwU1MSu2aOuRYYbGY54Yve2BEXwl78lyNZtu8Bfuice8LMJuEj7aT9keX/BLYBJfinkkORNDeXNgNWOefOyfB63XGcc7eY2VnAZfinxqS78dV6MyLrOgHnNLrQYWbVNOy/072JtNUdO3z+yWGfB8xsfjPvXQWUmFkn51y635I4HFmuwT/dgX/Sf79zbrmZTcM/BaWkB1/dOs85d7WZFeOffKDleT7bOfevGV6PHucyfJB5JfA1MxuTDC7am3PuhfBd6IN/euuDrxE4YmYbqc/v5j4f+Pw/HZib4fVk/kfLe6ZyDC0rCy39DYFMZTN6nIPOuWIzK8Hf0J4zsw+GY3zPOfeLZo6R7nqRTNv+RuujZbE28nctPm86AXuSgVoT+81kFb7pL5PG6WlN2pp6T0vPLQDOuT+Y2Yv4Mv+UmX3MOZep/KgToLTJXKC7mX0ysi7aOWsBvp2L0Et2ML69syXrx+Kr+xsIT0u/AX5iZl3DtkVmdkMzac3FV7sBTG1mu63hRvfv+Kd48NH6R5O9z82sIKyvoL4D4itAn/D0lxytMCbdQcxsmHPuRefc1/Ftop3C59uFr0a/KbL50/j2zOR7kxevjfgOYZjZu/FtmI3TlOkz7g4X/FH4p+iMnHPr8EHdN0MnKMxsuJld1dT7Qhq2mlkXwnltIj3JczMtsv5p4JZkB68Meb4IONfMTgnb9EjXI9vMOgGDnHPz8H05krUSx0TI1yx8rUousD3c/C8ETg6bLQCuNrPs8NR8RYbdfQ/4vpn1D/vuZma3NZOETOU43XbpysILwAVmNiQcM13eQ+ay2YCZDcXXwFyLr6UZh68R+mh4KsfMBphZ3zRvj14XJgE7nHP7Mn/0zML7NpjZdWF/FgIT8LVyHw7LmcrrXKCbmX088tnOMLML2pKeNmr23Ib8Xu+c+wm+aSPlWhqlAEBaLTxpvx9/odhgZi8BM/HtsAD3AllmthJfFTvNOXe4ifX/CyRC1f8X8Z1Y0rkd+Cew2vywl8fD3025A1/ttxB/w83kXmCqmS3CV63tD5/17/gv0hIzK8O30YN/yv15WJeFv8D9t5ktxz/ZZ+oFfJeZrQzpX0DDX2n8AQ172N8GTDCzFWa2GrglrH8EKAjH/iS+oxDOuZ34p6xyM7srzbH/DnQO+fwt/E20OR8D+gNrw3n7Fb5NvSlfw3e8nI1vF8/k+8D3zOw5Gl7Mfo3vH7Ii5Oe/hfW/BP7PzOY55/6JDxr+GD7PIny/ksaygN+FtJfi23H3NJP+1so2P9ytDF+up4Zmjt/jz98S/I3lZQDn3LKwXRn+XC5Mt1Pn3N+AnwFzzGwVvjq4uVrbtOU4jbRlIeTrJ4BHQ94nm1KexActZWZ2HpnLZmMfwvcvmIF/Yj3gnEt23nshnJc/kT5wvSN5DHznuaYC+Ja4HrgpfK5VQDKQ/QzwKTNbjL/JpgjXvKuBKRaGxIb0NfddaE8tObcfAspDWRwFNDmyQDMBioiIxJBqAERERGJIAYCIiEgMKQAQERGJIQUAIiIiMaQAQEREJIYUAIiIiMSQAgAREZEYUgAgIiISQ/8PKfY0dctx5poAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# props = lambda key: {'color': 'r' if 'alive' in key else 'gray'}\n",
    "t = mosaic(comic_characters, ['align','alive']) # ,title='character survival',properties=props"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q8: Conduct a $\\chi^2$ test of ```align``` and ```alive```. Please specify your (null and alternative) hypotheses and explain your findings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chi2 =  155.77673609805584\n",
      "p-val =  1.4943918171359309e-33\n",
      "degree of freedom =  3\n",
      "Expected:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2363.193534</td>\n",
       "      <td>1835.44478</td>\n",
       "      <td>681.624263</td>\n",
       "      <td>0.737423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7250.806466</td>\n",
       "      <td>5631.55522</td>\n",
       "      <td>2091.375737</td>\n",
       "      <td>2.262577</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             0           1            2         3\n",
       "0  2363.193534  1835.44478   681.624263  0.737423\n",
       "1  7250.806466  5631.55522  2091.375737  2.262577"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.stats import chi2_contingency\n",
    "ct = pd.crosstab(comic_characters.alive, comic_characters['align'])\n",
    "ct\n",
    "chi2, p, dof, ex = chi2_contingency(ct)\n",
    "print(\"chi2 = \", chi2)\n",
    "print(\"p-val = \", p)\n",
    "print(\"degree of freedom = \",dof)\n",
    "print(\"Expected:\")\n",
    "pd.DataFrame(ex)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BREAK!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Processing I: Basics and Regular Expressions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, a slideshow.... "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, let's load up some data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews = pd.read_csv('data/amazon_food_reviews.zip')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a really small sample, just so we can experiment with the various "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews_sample = reviews.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>User Id</th>\n",
       "      <th>ProfileName</th>\n",
       "      <th>HelpfulnessNumerator</th>\n",
       "      <th>HelpfulnessDenominator</th>\n",
       "      <th>Score</th>\n",
       "      <th>Time</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>B001E4KFG0</td>\n",
       "      <td>A3SGXH7AUHU8GW</td>\n",
       "      <td>delmartian</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1303862400</td>\n",
       "      <td>Good Quality Dog Food</td>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>B00813GRG4</td>\n",
       "      <td>A1D87F6ZCVE5NK</td>\n",
       "      <td>dll pa</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1346976000</td>\n",
       "      <td>Not as Advertised</td>\n",
       "      <td>Product arrived labeled as Jumbo Salted Peanut...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>B000LQOCH0</td>\n",
       "      <td>ABXLMWJIXXAIN</td>\n",
       "      <td>Natalia Corres \"Natalia Corres\"</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1219017600</td>\n",
       "      <td>\"Delight\" says it all</td>\n",
       "      <td>This is a confection that has been around a fe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>B000UA0QIQ</td>\n",
       "      <td>A395BORC6FGVXV</td>\n",
       "      <td>Karl</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1307923200</td>\n",
       "      <td>Cough Medicine</td>\n",
       "      <td>If you are looking for the secret ingredient i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A1UQRSCLF8GW1T</td>\n",
       "      <td>Michael D. Bigham \"M. Wassir\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350777600</td>\n",
       "      <td>Great taffy</td>\n",
       "      <td>Great taffy at a great price.  There was a wid...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>ADT0SRK1MGOEU</td>\n",
       "      <td>Twoapennything</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1342051200</td>\n",
       "      <td>Nice Taffy</td>\n",
       "      <td>I got a wild hair for taffy and ordered this f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A1SP2KVKFXXRU1</td>\n",
       "      <td>David C. Sullivan</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1340150400</td>\n",
       "      <td>Great!  Just as good as the expensive brands!</td>\n",
       "      <td>This saltwater taffy had great flavors and was...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A3JRGQVEQN31IQ</td>\n",
       "      <td>Pamela G. Williams</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1336003200</td>\n",
       "      <td>Wonderful, tasty taffy</td>\n",
       "      <td>This taffy is so good.  It is very soft and ch...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>B000E7L2R4</td>\n",
       "      <td>A1MZYO9TZK0BBI</td>\n",
       "      <td>R. James</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1322006400</td>\n",
       "      <td>Yay Barley</td>\n",
       "      <td>Right now I'm mostly just sprouting this so my...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>B00171APVA</td>\n",
       "      <td>A21BT40VZCCYT4</td>\n",
       "      <td>Carol A. Reed</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1351209600</td>\n",
       "      <td>Healthy Dog Food</td>\n",
       "      <td>This is a very healthy dog food. Good for thei...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id   ProductId        User Id                       ProfileName  \\\n",
       "0   1  B001E4KFG0  A3SGXH7AUHU8GW                       delmartian   \n",
       "1   2  B00813GRG4  A1D87F6ZCVE5NK                           dll pa   \n",
       "2   3  B000LQOCH0   ABXLMWJIXXAIN  Natalia Corres \"Natalia Corres\"   \n",
       "3   4  B000UA0QIQ  A395BORC6FGVXV                             Karl   \n",
       "4   5  B006K2ZZ7K  A1UQRSCLF8GW1T    Michael D. Bigham \"M. Wassir\"   \n",
       "5   6  B006K2ZZ7K   ADT0SRK1MGOEU                   Twoapennything   \n",
       "6   7  B006K2ZZ7K  A1SP2KVKFXXRU1                David C. Sullivan   \n",
       "7   8  B006K2ZZ7K  A3JRGQVEQN31IQ               Pamela G. Williams   \n",
       "8   9  B000E7L2R4  A1MZYO9TZK0BBI                         R. James   \n",
       "9  10  B00171APVA  A21BT40VZCCYT4                    Carol A. Reed   \n",
       "\n",
       "   HelpfulnessNumerator  HelpfulnessDenominator  Score        Time  \\\n",
       "0                     1                       1      5  1303862400   \n",
       "1                     0                       0      1  1346976000   \n",
       "2                     1                       1      4  1219017600   \n",
       "3                     3                       3      2  1307923200   \n",
       "4                     0                       0      5  1350777600   \n",
       "5                     0                       0      4  1342051200   \n",
       "6                     0                       0      5  1340150400   \n",
       "7                     0                       0      5  1336003200   \n",
       "8                     1                       1      5  1322006400   \n",
       "9                     0                       0      5  1351209600   \n",
       "\n",
       "                                         Summary  \\\n",
       "0                          Good Quality Dog Food   \n",
       "1                              Not as Advertised   \n",
       "2                          \"Delight\" says it all   \n",
       "3                                 Cough Medicine   \n",
       "4                                    Great taffy   \n",
       "5                                     Nice Taffy   \n",
       "6  Great!  Just as good as the expensive brands!   \n",
       "7                         Wonderful, tasty taffy   \n",
       "8                                     Yay Barley   \n",
       "9                               Healthy Dog Food   \n",
       "\n",
       "                                                Text  \n",
       "0  I have bought several of the Vitality canned d...  \n",
       "1  Product arrived labeled as Jumbo Salted Peanut...  \n",
       "2  This is a confection that has been around a fe...  \n",
       "3  If you are looking for the secret ingredient i...  \n",
       "4  Great taffy at a great price.  There was a wid...  \n",
       "5  I got a wild hair for taffy and ordered this f...  \n",
       "6  This saltwater taffy had great flavors and was...  \n",
       "7  This taffy is so good.  It is very soft and ch...  \n",
       "8  Right now I'm mostly just sprouting this so my...  \n",
       "9  This is a very healthy dog food. Good for thei...  "
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    I have bought several of the Vitality canned d...\n",
       "1    Product arrived labeled as Jumbo Salted Peanut...\n",
       "2    This is a confection that has been around a fe...\n",
       "3    If you are looking for the secret ingredient i...\n",
       "4    Great taffy at a great price.  There was a wid...\n",
       "5    I got a wild hair for taffy and ordered this f...\n",
       "6    This saltwater taffy had great flavors and was...\n",
       "7    This taffy is so good.  It is very soft and ch...\n",
       "8    Right now I'm mostly just sprouting this so my...\n",
       "9    This is a very healthy dog food. Good for thei...\n",
       "Name: Text, dtype: object"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample['Text']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's review some basic string functionality from Pandas that can be applied to any Series or Index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                         delmartian\n",
       "1                             dll pa\n",
       "2    natalia corres \"natalia corres\"\n",
       "3                               karl\n",
       "4      michael d. bigham \"m. wassir\"\n",
       "5                     twoapennything\n",
       "6                  david c. sullivan\n",
       "7                 pamela g. williams\n",
       "8                           r. james\n",
       "9                      carol a. reed\n",
       "Name: ProfileName, dtype: object"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.ProfileName.str.lower()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                         DELMARTIAN\n",
       "1                             DLL PA\n",
       "2    NATALIA CORRES \"NATALIA CORRES\"\n",
       "3                               KARL\n",
       "4      MICHAEL D. BIGHAM \"M. WASSIR\"\n",
       "5                     TWOAPENNYTHING\n",
       "6                  DAVID C. SULLIVAN\n",
       "7                 PAMELA G. WILLIAMS\n",
       "8                           R. JAMES\n",
       "9                      CAROL A. REED\n",
       "Name: ProfileName, dtype: object"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.ProfileName.str.upper()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    21\n",
       "1    17\n",
       "2    21\n",
       "3    14\n",
       "4    11\n",
       "5    10\n",
       "6    45\n",
       "7    22\n",
       "8    10\n",
       "9    16\n",
       "Name: Summary, dtype: int64"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.Summary.str.len()  # .max() find the max value "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember, the ```columns``` attribute of a DataFrame is an Index object, which means that we can use str operators on the column names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Id', 'ProductId', 'User Id ', 'ProfileName', 'HelpfulnessNumerator',\n",
       "       'HelpfulnessDenominator', 'Score', 'Time', 'Summary', 'Text'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['id', 'productid', 'user id ', 'profilename', 'helpfulnessnumerator',\n",
       "       'helpfulnessdenominator', 'score', 'time', 'summary', 'text'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.columns.str.lower() # columns represents index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the \"User Id\" column of the dataframe looks weird:  it has a space in the middle *and* at the end.  Columns that are named like that will invariable trip us up in downstream (i.e. later) analyses, so it's wise to correct them now.  Something like the following can help:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['id', 'productid', 'user_id', 'profilename', 'helpfulnessnumerator',\n",
       "       'helpfulnessdenominator', 'score', 'time', 'summary', 'text'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.columns.str.strip().str.lower().str.replace(' ','_')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can assign that back to the columns attribute to actually rename the columns:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews_sample.columns = reviews_sample.columns.str.strip().str.lower().str.replace(' ','_')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>productid</th>\n",
       "      <th>user_id</th>\n",
       "      <th>profilename</th>\n",
       "      <th>helpfulnessnumerator</th>\n",
       "      <th>helpfulnessdenominator</th>\n",
       "      <th>score</th>\n",
       "      <th>time</th>\n",
       "      <th>summary</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>B001E4KFG0</td>\n",
       "      <td>A3SGXH7AUHU8GW</td>\n",
       "      <td>delmartian</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1303862400</td>\n",
       "      <td>Good Quality Dog Food</td>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>B00813GRG4</td>\n",
       "      <td>A1D87F6ZCVE5NK</td>\n",
       "      <td>dll pa</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1346976000</td>\n",
       "      <td>Not as Advertised</td>\n",
       "      <td>Product arrived labeled as Jumbo Salted Peanut...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>B000LQOCH0</td>\n",
       "      <td>ABXLMWJIXXAIN</td>\n",
       "      <td>Natalia Corres \"Natalia Corres\"</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1219017600</td>\n",
       "      <td>\"Delight\" says it all</td>\n",
       "      <td>This is a confection that has been around a fe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>B000UA0QIQ</td>\n",
       "      <td>A395BORC6FGVXV</td>\n",
       "      <td>Karl</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1307923200</td>\n",
       "      <td>Cough Medicine</td>\n",
       "      <td>If you are looking for the secret ingredient i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A1UQRSCLF8GW1T</td>\n",
       "      <td>Michael D. Bigham \"M. Wassir\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350777600</td>\n",
       "      <td>Great taffy</td>\n",
       "      <td>Great taffy at a great price.  There was a wid...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>ADT0SRK1MGOEU</td>\n",
       "      <td>Twoapennything</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1342051200</td>\n",
       "      <td>Nice Taffy</td>\n",
       "      <td>I got a wild hair for taffy and ordered this f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A1SP2KVKFXXRU1</td>\n",
       "      <td>David C. Sullivan</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1340150400</td>\n",
       "      <td>Great!  Just as good as the expensive brands!</td>\n",
       "      <td>This saltwater taffy had great flavors and was...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A3JRGQVEQN31IQ</td>\n",
       "      <td>Pamela G. Williams</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1336003200</td>\n",
       "      <td>Wonderful, tasty taffy</td>\n",
       "      <td>This taffy is so good.  It is very soft and ch...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>B000E7L2R4</td>\n",
       "      <td>A1MZYO9TZK0BBI</td>\n",
       "      <td>R. James</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1322006400</td>\n",
       "      <td>Yay Barley</td>\n",
       "      <td>Right now I'm mostly just sprouting this so my...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>B00171APVA</td>\n",
       "      <td>A21BT40VZCCYT4</td>\n",
       "      <td>Carol A. Reed</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1351209600</td>\n",
       "      <td>Healthy Dog Food</td>\n",
       "      <td>This is a very healthy dog food. Good for thei...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id   productid         user_id                      profilename  \\\n",
       "0   1  B001E4KFG0  A3SGXH7AUHU8GW                       delmartian   \n",
       "1   2  B00813GRG4  A1D87F6ZCVE5NK                           dll pa   \n",
       "2   3  B000LQOCH0   ABXLMWJIXXAIN  Natalia Corres \"Natalia Corres\"   \n",
       "3   4  B000UA0QIQ  A395BORC6FGVXV                             Karl   \n",
       "4   5  B006K2ZZ7K  A1UQRSCLF8GW1T    Michael D. Bigham \"M. Wassir\"   \n",
       "5   6  B006K2ZZ7K   ADT0SRK1MGOEU                   Twoapennything   \n",
       "6   7  B006K2ZZ7K  A1SP2KVKFXXRU1                David C. Sullivan   \n",
       "7   8  B006K2ZZ7K  A3JRGQVEQN31IQ               Pamela G. Williams   \n",
       "8   9  B000E7L2R4  A1MZYO9TZK0BBI                         R. James   \n",
       "9  10  B00171APVA  A21BT40VZCCYT4                    Carol A. Reed   \n",
       "\n",
       "   helpfulnessnumerator  helpfulnessdenominator  score        time  \\\n",
       "0                     1                       1      5  1303862400   \n",
       "1                     0                       0      1  1346976000   \n",
       "2                     1                       1      4  1219017600   \n",
       "3                     3                       3      2  1307923200   \n",
       "4                     0                       0      5  1350777600   \n",
       "5                     0                       0      4  1342051200   \n",
       "6                     0                       0      5  1340150400   \n",
       "7                     0                       0      5  1336003200   \n",
       "8                     1                       1      5  1322006400   \n",
       "9                     0                       0      5  1351209600   \n",
       "\n",
       "                                         summary  \\\n",
       "0                          Good Quality Dog Food   \n",
       "1                              Not as Advertised   \n",
       "2                          \"Delight\" says it all   \n",
       "3                                 Cough Medicine   \n",
       "4                                    Great taffy   \n",
       "5                                     Nice Taffy   \n",
       "6  Great!  Just as good as the expensive brands!   \n",
       "7                         Wonderful, tasty taffy   \n",
       "8                                     Yay Barley   \n",
       "9                               Healthy Dog Food   \n",
       "\n",
       "                                                text  \n",
       "0  I have bought several of the Vitality canned d...  \n",
       "1  Product arrived labeled as Jumbo Salted Peanut...  \n",
       "2  This is a confection that has been around a fe...  \n",
       "3  If you are looking for the secret ingredient i...  \n",
       "4  Great taffy at a great price.  There was a wid...  \n",
       "5  I got a wild hair for taffy and ordered this f...  \n",
       "6  This saltwater taffy had great flavors and was...  \n",
       "7  This taffy is so good.  It is very soft and ch...  \n",
       "8  Right now I'm mostly just sprouting this so my...  \n",
       "9  This is a very healthy dog food. Good for thei...  "
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Splitting and Replacing Strings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, we want to split strings into lists.  We might want to do that with the \"summary\" column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [B, 1E4KFG0]\n",
       "1    [B, 813GRG4]\n",
       "2    [B, 0LQOCH0]\n",
       "3    [B, 0UA0QIQ]\n",
       "4    [B, 6K2ZZ7K]\n",
       "5    [B, 6K2ZZ7K]\n",
       "6    [B, 6K2ZZ7K]\n",
       "7    [B, 6K2ZZ7K]\n",
       "8    [B, 0E7L2R4]\n",
       "9    [B, 171APVA]\n",
       "Name: productid, dtype: object"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.productid.str.split('00')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1E4KFG0\n",
       "1    813GRG4\n",
       "2    0LQOCH0\n",
       "3    0UA0QIQ\n",
       "4    6K2ZZ7K\n",
       "5    6K2ZZ7K\n",
       "6    6K2ZZ7K\n",
       "7    6K2ZZ7K\n",
       "8    0E7L2R4\n",
       "9    171APVA\n",
       "Name: productid, dtype: object"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.productid.str.split('00').str.get(1) # access the element, str.get(1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Equivalently:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1E4KFG0\n",
       "1    813GRG4\n",
       "2    0LQOCH0\n",
       "3    0UA0QIQ\n",
       "4    6K2ZZ7K\n",
       "5    6K2ZZ7K\n",
       "6    6K2ZZ7K\n",
       "7    6K2ZZ7K\n",
       "8    0E7L2R4\n",
       "9    171APVA\n",
       "Name: productid, dtype: object"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.productid.str.split('00').str[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                            Good Quality Dog Food\n",
       "1                                Not as Advertised\n",
       "2                            \"Delight\" says it all\n",
       "3                                   Cough Medicine\n",
       "4                                      Great taffy\n",
       "5                                       Nice Taffy\n",
       "6    Great!  Just as good as the expensive brands!\n",
       "7                           Wonderful, tasty taffy\n",
       "8                                       Yay Barley\n",
       "9                                 Healthy Dog Food\n",
       "Name: summary, dtype: object"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Replace (regex time!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                         good quality health food\n",
       "1                                not as advertised\n",
       "2                            \"delight\" says it all\n",
       "3                                   cough medicine\n",
       "4                                      great taffy\n",
       "5                                       nice taffy\n",
       "6    great!  just as good as the expensive brands!\n",
       "7                           wonderful, tasty taffy\n",
       "8                                       yay barley\n",
       "9                              healthy health food\n",
       "Name: summary, dtype: object"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary.str.lower().str.replace('dog','health')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                         good quality health food\n",
       "1                                not as advertised\n",
       "2                            \"delight\" says it all\n",
       "3                                   cough medicine\n",
       "4                                     great health\n",
       "5                                      nice health\n",
       "6    great!  just as good as the expensive brands!\n",
       "7                          wonderful, tasty health\n",
       "8                                       yay barley\n",
       "9                              healthy health food\n",
       "Name: summary, dtype: object"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary.str.lower().str.replace('dog|taffy','health')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extracting Substrings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0\n",
       "0  Dog\n",
       "1  NaN\n",
       "2  NaN\n",
       "3  NaN\n",
       "4  NaN\n",
       "5  NaN\n",
       "6  NaN\n",
       "7  NaN\n",
       "8  NaN\n",
       "9  Dog"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary.str.extract(r'(Dog)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Taffy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0\n",
       "0    Dog\n",
       "1    NaN\n",
       "2    NaN\n",
       "3    NaN\n",
       "4    NaN\n",
       "5  Taffy\n",
       "6    NaN\n",
       "7    NaN\n",
       "8    NaN\n",
       "9    Dog"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary.str.extract(r'(Dog|Taffy)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>taffy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Taffy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>taffy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0\n",
       "0    Dog\n",
       "1    NaN\n",
       "2    NaN\n",
       "3    NaN\n",
       "4  taffy\n",
       "5  Taffy\n",
       "6    NaN\n",
       "7  taffy\n",
       "8    NaN\n",
       "9    Dog"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary.str.extract(r'(Dog|[Tt]affy)') # match any character in the []: T or t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Dog\n",
       "1      NaN\n",
       "2      NaN\n",
       "3      NaN\n",
       "4    taffy\n",
       "5    Taffy\n",
       "6      NaN\n",
       "7    taffy\n",
       "8      NaN\n",
       "9      Dog\n",
       "Name: summary, dtype: object"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# returns a Series\n",
    "reviews_sample.summary.str.extract(r'(Dog|[Tt]affy)', expand = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>match</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <th>0</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <th>0</th>\n",
       "      <td>taffy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <th>0</th>\n",
       "      <td>Taffy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <th>0</th>\n",
       "      <td>taffy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <th>0</th>\n",
       "      <td>Dog</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             0\n",
       "  match       \n",
       "0 0        Dog\n",
       "4 0      taffy\n",
       "5 0      Taffy\n",
       "7 0      taffy\n",
       "9 0        Dog"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary.str.extractall(r'(Dog|[Tt]affy)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>match</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <th>0</th>\n",
       "      <td>as</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">6</th>\n",
       "      <th>0</th>\n",
       "      <td>as</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>as</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <th>0</th>\n",
       "      <td>as</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          0\n",
       "  match    \n",
       "1 0      as\n",
       "6 0      as\n",
       "  1      as\n",
       "7 0      as"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_sample.summary.str.extractall(r'(as)')\n",
    "# Matches whatever regular expression is inside the parentheses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing for Strings that Match or Contain a Pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews_sample.text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pattern = r'[Gg]ood'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews_sample.text.str.contains(pattern)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews_sample.text.str.match(pattern)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pattern = r'.*[Gg]ood.*'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews_sample.text.str.match(pattern)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Helpful resources:\n",
    "- Pandas text documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/text.html\n",
    "- Regex Cheat Sheet: https://regexr.com/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q9: How many rows from the Amazon Food Reviews data set contain HTML tags in the ```text``` column?</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">10: Remove all HTML tags from the Amazon Food Reviews text column and save the results to a column called text_no_html."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q11: Replace the following words in the text column with the word 'POSITIVE_ADJ' (denoting positive adjectives) and save the results to a column called ```text_coded```. \n",
    "    \n",
    "In all cases, you should find words that are either all lowercase, all uppercase, or words that start with an uppercase letter with the remaining letters lowercase:\n",
    "    \n",
    "good, great, excellent, best, perfect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"magenta\">Q12: How many rows contain multiple positive adjectives?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add your code here"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}