04-party_votes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 04 - Scraping party votes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code in this notebook uses the data created in `03 - Cleaning data` to scape each individual senate webpage for confirmations of circuit judges and extract each senator's vote."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import re\n",
    "import numpy as np\n",
    "\n",
    "df = pd.read_csv(\"data/votes.csv\", na_values=['NaN'])\n",
    "\n",
    "df['confirmation_date'] = pd.to_datetime(df.confirmation_date)\n",
    "\n",
    "# Convert the votes and record vote numbers into integers instead of float to remove the decimals. \n",
    "df['yea_votes'] = df['yea_votes'].astype('Int64')\n",
    "df['nay_votes'] = df['nay_votes'].astype('Int64')\n",
    "df['record_vote_number'] = df['record_vote_number'].astype('Int64')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then convert the record_vote_number into a three digit string – this will be important later when I have to loop through URLs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>nomination_no</th>\n",
       "      <th>congress_no</th>\n",
       "      <th>circuit</th>\n",
       "      <th>confirmation_date</th>\n",
       "      <th>yea_votes</th>\n",
       "      <th>nay_votes</th>\n",
       "      <th>record_vote_number</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Roopali H. Desai</td>\n",
       "      <td>PN2262</td>\n",
       "      <td>117</td>\n",
       "      <td>Ninth</td>\n",
       "      <td>2022-08-04</td>\n",
       "      <td>67</td>\n",
       "      <td>29</td>\n",
       "      <td>284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Florence Y. Pan</td>\n",
       "      <td>PN2193</td>\n",
       "      <td>117</td>\n",
       "      <td>District of Columbia</td>\n",
       "      <td>2022-09-20</td>\n",
       "      <td>52</td>\n",
       "      <td>42</td>\n",
       "      <td>340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sarah A. L. Merriam</td>\n",
       "      <td>PN2141</td>\n",
       "      <td>117</td>\n",
       "      <td>Second</td>\n",
       "      <td>2022-09-15</td>\n",
       "      <td>53</td>\n",
       "      <td>44</td>\n",
       "      <td>337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Lara E. Montecalvo</td>\n",
       "      <td>PN2140</td>\n",
       "      <td>117</td>\n",
       "      <td>First</td>\n",
       "      <td>2022-09-14</td>\n",
       "      <td>52</td>\n",
       "      <td>47</td>\n",
       "      <td>335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Salvador Mendoza Jr</td>\n",
       "      <td>PN1966</td>\n",
       "      <td>117</td>\n",
       "      <td>Ninth</td>\n",
       "      <td>2022-09-12</td>\n",
       "      <td>46</td>\n",
       "      <td>40</td>\n",
       "      <td>331</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  name nomination_no  congress_no               circuit  \\\n",
       "0     Roopali H. Desai        PN2262          117                 Ninth   \n",
       "1      Florence Y. Pan        PN2193          117  District of Columbia   \n",
       "2  Sarah A. L. Merriam        PN2141          117                Second   \n",
       "3   Lara E. Montecalvo        PN2140          117                 First   \n",
       "4  Salvador Mendoza Jr        PN1966          117                 Ninth   \n",
       "\n",
       "  confirmation_date  yea_votes  nay_votes record_vote_number  \n",
       "0        2022-08-04         67         29                284  \n",
       "1        2022-09-20         52         42                340  \n",
       "2        2022-09-15         53         44                337  \n",
       "3        2022-09-14         52         47                335  \n",
       "4        2022-09-12         46         40                331  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['record_vote_number'] = df['record_vote_number'].apply(lambda x: '{0:0>3}'.format(x))\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['confirmation_year']=df['confirmation_date'].dt.to_period('Y')\n",
    "\n",
    "df.confirmation_year = df.confirmation_year.map(str)\n",
    "df.confirmation_year = df.confirmation_year.astype(int)\n",
    "\n",
    "df['session'] = df.confirmation_year.apply(lambda x: \"2\" if x % 2 == 0 else \"1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because the record vote numbers are repeated every year, some nominations share the number. Later on I will need a unique number for each nomination, so I create a vote_id by merging the congress #, the session # and and the record vote #."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['vote_id'] = df.congress_no.astype(str) + df.session.astype(str) + df.record_vote_number.astype(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"voteinfo.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scraping each senators vote for each confirmation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from bs4 import BeautifulSoup\n",
    "import requests\n",
    "from playwright.async_api import async_playwright\n",
    "\n",
    "playwright = await async_playwright().start()\n",
    "browser = await playwright.chromium.launch(headless = False)\n",
    "page = await browser.new_page()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The loop below loops through every webpage and scrapes the content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DO NOT RUN ALL THE TIME! MIGHT UPSET THE SERVER...\n",
    "\n",
    "votes = [] \n",
    "for index, row in df.iterrows():\n",
    "    record_vote_number = row['record_vote_number']\n",
    "    congress_no = row['congress_no']\n",
    "    session = row['session']\n",
    "    await page.goto(f\"https://www.senate.gov/legislative/LIS/roll_call_votes/vote{congress_no}{session}/vote_{congress_no}_{session}_00{record_vote_number}.htm\")\n",
    "\n",
    "    html = await page.content()\n",
    "\n",
    "    doc = BeautifulSoup(html)\n",
    "\n",
    "    nominations = {}\n",
    "    nominations[\"congress_no\"] = row['congress_no']\n",
    "    nominations[\"session\"] = row['session']\n",
    "    nominations[\"vote_number\"] = doc.select(\"#secondary_col2 > div:nth-child(7) > div:nth-child(2)\")\n",
    "\n",
    "    results = doc.select(\"#secondary_col2 > div:nth-child(14) > span \")\n",
    "\n",
    "    nomination = []\n",
    "    for row in results:\n",
    "        nomination.append(row.text)\n",
    "\n",
    "    nomination1 = [item.split('\\n') for item in nomination]\n",
    "\n",
    "    nomination1_flat = [item for l in nomination1 for item in l]\n",
    "\n",
    "    nomination2 = [item.split(',') for item in nomination1_flat]\n",
    "\n",
    "    nominations[\"votes\"] = nomination2\n",
    "\n",
    "    votes.append(nominations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This loop loops through the scrape and extracts the democratic, republican and independent votes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n",
      "What is going on here?\n"
     ]
    }
   ],
   "source": [
    "for vote in votes:\n",
    "    vote1 = {}\n",
    "\n",
    "    D_yea = 0\n",
    "    D_nay = 0\n",
    "    D_no_vote = 0\n",
    "    R_yea = 0\n",
    "    R_nay = 0\n",
    "    R_no_vote = 0\n",
    "    I_yea = 0\n",
    "    I_nay = 0\n",
    "    I_no_vote = 0\n",
    "    for senator in vote['votes']:\n",
    "        dem = \"(D\"\n",
    "        rep = \"(R\"\n",
    "        ind = \"(I\"\n",
    "        if dem in senator[0]:\n",
    "            if senator[1] == \" Yea\":\n",
    "                D_yea = D_yea + 1\n",
    "            elif senator[1] == \" Nay\":\n",
    "                D_nay = D_nay + 1\n",
    "            else:\n",
    "                D_no_vote = D_no_vote + 1\n",
    "        elif rep in senator[0]:\n",
    "            if senator[1] == \" Yea\":\n",
    "                R_yea = R_yea + 1\n",
    "            elif senator[1] == \" Nay\":\n",
    "                R_nay = R_nay + 1\n",
    "            else:\n",
    "                R_no_vote = R_no_vote + 1\n",
    "        elif ind in senator[0]:\n",
    "            if senator[1] == \" Yea\":\n",
    "                I_yea = I_yea + 1\n",
    "            elif senator[1] == \" Nay\":\n",
    "                I_nay = I_nay + 1\n",
    "            else:\n",
    "                I_no_vote = I_no_vote + 1        \n",
    "        else:\n",
    "            print(\"What is going on here?\") \n",
    "\n",
    "    vote['D_yea'] = D_yea\n",
    "    vote['D_nay'] = D_nay\n",
    "    vote['D_no_vote'] = D_no_vote\n",
    "    vote['R_yea'] = R_yea\n",
    "    vote['R_nay'] = R_nay\n",
    "    vote['R_no_vote'] = R_no_vote\n",
    "    vote['I_yea'] = I_yea\n",
    "    vote['I_nay'] = I_nay\n",
    "    vote['I_no_vote'] = I_no_vote"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(votes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['vote_number'] = df['vote_number'].astype(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['vote_number'] = df['vote_number'].str.replace(r'<[^<>]*>', '', regex=True)\n",
    "df['vote_number'] = df['vote_number'].str.extract('(\\d+)', expand=False).str.strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>congress_no</th>\n",
       "      <th>session</th>\n",
       "      <th>vote_number</th>\n",
       "      <th>votes</th>\n",
       "      <th>D_yea</th>\n",
       "      <th>D_nay</th>\n",
       "      <th>D_no_vote</th>\n",
       "      <th>R_yea</th>\n",
       "      <th>R_nay</th>\n",
       "      <th>R_no_vote</th>\n",
       "      <th>I_yea</th>\n",
       "      <th>I_nay</th>\n",
       "      <th>I_no_vote</th>\n",
       "      <th>vote_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>117</td>\n",
       "      <td>2</td>\n",
       "      <td>284</td>\n",
       "      <td>[[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  Na...</td>\n",
       "      <td>46</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>19</td>\n",
       "      <td>29</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1172284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>117</td>\n",
       "      <td>2</td>\n",
       "      <td>340</td>\n",
       "      <td>[[Baldwin (D-WI),  Not Voting], [Barrasso (R-W...</td>\n",
       "      <td>46</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>42</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1172340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>117</td>\n",
       "      <td>2</td>\n",
       "      <td>337</td>\n",
       "      <td>[[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  Na...</td>\n",
       "      <td>48</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>44</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1172337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>117</td>\n",
       "      <td>2</td>\n",
       "      <td>335</td>\n",
       "      <td>[[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  Na...</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1172335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>117</td>\n",
       "      <td>2</td>\n",
       "      <td>331</td>\n",
       "      <td>[[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  No...</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>40</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1172331</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   congress_no session vote_number  \\\n",
       "0          117       2         284   \n",
       "1          117       2         340   \n",
       "2          117       2         337   \n",
       "3          117       2         335   \n",
       "4          117       2         331   \n",
       "\n",
       "                                               votes  D_yea  D_nay  D_no_vote  \\\n",
       "0  [[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  Na...     46      0          2   \n",
       "1  [[Baldwin (D-WI),  Not Voting], [Barrasso (R-W...     46      0          2   \n",
       "2  [[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  Na...     48      0          0   \n",
       "3  [[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  Na...     47      0          1   \n",
       "4  [[Baldwin (D-WI),  Yea], [Barrasso (R-WY),  No...     41      0          7   \n",
       "\n",
       "   R_yea  R_nay  R_no_vote  I_yea  I_nay  I_no_vote  vote_id  \n",
       "0     19     29          2      2      0          0  1172284  \n",
       "1      4     42          4      2      0          0  1172340  \n",
       "2      3     44          3      2      0          0  1172337  \n",
       "3      3     47          0      2      0          0  1172335  \n",
       "4      3     40          7      2      0          0  1172331  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['vote_number'] = df['vote_number'].apply(lambda x: '{0:0>3}'.format(x))\n",
    "df['vote_id'] = df.congress_no.astype(str) + df.session.astype(str) + df.vote_number.astype(str)\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"data/party_votes.csv\", index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.11.0rc2 64-bit ('3.11.0rc2')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "fa07e5dff76123542c28abca75c0e47ea742948dbeeb360ccfbd2a5c7c277f8b"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}