Regression-Analysis-with-XGBoos.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b385a2a1",
   "metadata": {},
   "source": [
    "# Regression Analysis with XGBoost"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d1b6a86",
   "metadata": {},
   "source": [
    "Hi Guys, Welcome to [Tirendaz Academy](https://youtube.com/c/tirendazacademy) 😀 </br>\n",
    "In this notebook, I'm going to talk about regression analysis with XGBoost. </br>\n",
    "I'll cover the following topics:</br>\n",
    "- What is XGBoost?\n",
    "- Building a regression model with XGBoost\n",
    "- Building a linear regression model with Scikit-Learn\n",
    "\n",
    "Happy learning 🐱‍🏍 "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82404b55",
   "metadata": {},
   "source": [
    "# What is XGBoost"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3be34c1",
   "metadata": {},
   "source": [
    "XGBoost is short for Extreme Gradient Boosting.\n",
    "\n",
    "You can use the XGBoost package to implement gradient boosting.\n",
    "\n",
    "It provides a parallel tree boosting that solves many data science problems in a fast and accurate way. \n",
    "\n",
    "To install XGBoost, you can use the `pip install xgboost` command."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8a8207a",
   "metadata": {},
   "source": [
    "# Loading Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b80dc2b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>dteday</th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weekday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.344167</td>\n",
       "      <td>0.363625</td>\n",
       "      <td>0.805833</td>\n",
       "      <td>0.160446</td>\n",
       "      <td>331</td>\n",
       "      <td>654</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2011-01-02</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.363478</td>\n",
       "      <td>0.353739</td>\n",
       "      <td>0.696087</td>\n",
       "      <td>0.248539</td>\n",
       "      <td>131</td>\n",
       "      <td>670</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>2011-01-03</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.196364</td>\n",
       "      <td>0.189405</td>\n",
       "      <td>0.437273</td>\n",
       "      <td>0.248309</td>\n",
       "      <td>120</td>\n",
       "      <td>1229</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>2011-01-04</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>0.212122</td>\n",
       "      <td>0.590435</td>\n",
       "      <td>0.160296</td>\n",
       "      <td>108</td>\n",
       "      <td>1454</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>2011-01-05</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.226957</td>\n",
       "      <td>0.229270</td>\n",
       "      <td>0.436957</td>\n",
       "      <td>0.186900</td>\n",
       "      <td>82</td>\n",
       "      <td>1518</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant      dteday  season   yr  mnth  holiday  weekday  workingday  \\\n",
       "0        1  2011-01-01     1.0  0.0   1.0      0.0      6.0         0.0   \n",
       "1        2  2011-01-02     1.0  0.0   1.0      0.0      0.0         0.0   \n",
       "2        3  2011-01-03     1.0  0.0   1.0      0.0      1.0         1.0   \n",
       "3        4  2011-01-04     1.0  0.0   1.0      0.0      2.0         1.0   \n",
       "4        5  2011-01-05     1.0  0.0   1.0      0.0      3.0         1.0   \n",
       "\n",
       "   weathersit      temp     atemp       hum  windspeed  casual  registered  \\\n",
       "0           2  0.344167  0.363625  0.805833   0.160446     331         654   \n",
       "1           2  0.363478  0.353739  0.696087   0.248539     131         670   \n",
       "2           1  0.196364  0.189405  0.437273   0.248309     120        1229   \n",
       "3           1  0.200000  0.212122  0.590435   0.160296     108        1454   \n",
       "4           1  0.226957  0.229270  0.436957   0.186900      82        1518   \n",
       "\n",
       "    cnt  \n",
       "0   985  \n",
       "1   801  \n",
       "2  1349  \n",
       "3  1562  \n",
       "4  1600  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(\"bike_rentals.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66bdae46",
   "metadata": {},
   "source": [
    "# Understanding The Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "304848ac",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>instant</th>\n",
       "      <td>731.0</td>\n",
       "      <td>366.000000</td>\n",
       "      <td>211.165812</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>183.500000</td>\n",
       "      <td>366.000000</td>\n",
       "      <td>548.500000</td>\n",
       "      <td>731.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>season</th>\n",
       "      <td>731.0</td>\n",
       "      <td>2.496580</td>\n",
       "      <td>1.110807</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>yr</th>\n",
       "      <td>730.0</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500343</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth</th>\n",
       "      <td>730.0</td>\n",
       "      <td>6.512329</td>\n",
       "      <td>3.448303</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>9.750000</td>\n",
       "      <td>12.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>holiday</th>\n",
       "      <td>731.0</td>\n",
       "      <td>0.028728</td>\n",
       "      <td>0.167155</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday</th>\n",
       "      <td>731.0</td>\n",
       "      <td>2.997264</td>\n",
       "      <td>2.004787</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>workingday</th>\n",
       "      <td>731.0</td>\n",
       "      <td>0.682627</td>\n",
       "      <td>0.465773</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weathersit</th>\n",
       "      <td>731.0</td>\n",
       "      <td>1.395349</td>\n",
       "      <td>0.544894</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>temp</th>\n",
       "      <td>730.0</td>\n",
       "      <td>0.495587</td>\n",
       "      <td>0.183094</td>\n",
       "      <td>0.059130</td>\n",
       "      <td>0.336875</td>\n",
       "      <td>0.499166</td>\n",
       "      <td>0.655625</td>\n",
       "      <td>0.861667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>atemp</th>\n",
       "      <td>730.0</td>\n",
       "      <td>0.474512</td>\n",
       "      <td>0.163017</td>\n",
       "      <td>0.079070</td>\n",
       "      <td>0.337794</td>\n",
       "      <td>0.487364</td>\n",
       "      <td>0.608916</td>\n",
       "      <td>0.840896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>hum</th>\n",
       "      <td>728.0</td>\n",
       "      <td>0.627987</td>\n",
       "      <td>0.142331</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.521562</td>\n",
       "      <td>0.627083</td>\n",
       "      <td>0.730104</td>\n",
       "      <td>0.972500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>windspeed</th>\n",
       "      <td>726.0</td>\n",
       "      <td>0.190476</td>\n",
       "      <td>0.077725</td>\n",
       "      <td>0.022392</td>\n",
       "      <td>0.134494</td>\n",
       "      <td>0.180971</td>\n",
       "      <td>0.233218</td>\n",
       "      <td>0.507463</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>casual</th>\n",
       "      <td>731.0</td>\n",
       "      <td>848.176471</td>\n",
       "      <td>686.622488</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>315.500000</td>\n",
       "      <td>713.000000</td>\n",
       "      <td>1096.000000</td>\n",
       "      <td>3410.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>registered</th>\n",
       "      <td>731.0</td>\n",
       "      <td>3656.172367</td>\n",
       "      <td>1560.256377</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>2497.000000</td>\n",
       "      <td>3662.000000</td>\n",
       "      <td>4776.500000</td>\n",
       "      <td>6946.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cnt</th>\n",
       "      <td>731.0</td>\n",
       "      <td>4504.348837</td>\n",
       "      <td>1937.211452</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>3152.000000</td>\n",
       "      <td>4548.000000</td>\n",
       "      <td>5956.000000</td>\n",
       "      <td>8714.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            count         mean          std        min          25%  \\\n",
       "instant     731.0   366.000000   211.165812   1.000000   183.500000   \n",
       "season      731.0     2.496580     1.110807   1.000000     2.000000   \n",
       "yr          730.0     0.500000     0.500343   0.000000     0.000000   \n",
       "mnth        730.0     6.512329     3.448303   1.000000     4.000000   \n",
       "holiday     731.0     0.028728     0.167155   0.000000     0.000000   \n",
       "weekday     731.0     2.997264     2.004787   0.000000     1.000000   \n",
       "workingday  731.0     0.682627     0.465773   0.000000     0.000000   \n",
       "weathersit  731.0     1.395349     0.544894   1.000000     1.000000   \n",
       "temp        730.0     0.495587     0.183094   0.059130     0.336875   \n",
       "atemp       730.0     0.474512     0.163017   0.079070     0.337794   \n",
       "hum         728.0     0.627987     0.142331   0.000000     0.521562   \n",
       "windspeed   726.0     0.190476     0.077725   0.022392     0.134494   \n",
       "casual      731.0   848.176471   686.622488   2.000000   315.500000   \n",
       "registered  731.0  3656.172367  1560.256377  20.000000  2497.000000   \n",
       "cnt         731.0  4504.348837  1937.211452  22.000000  3152.000000   \n",
       "\n",
       "                    50%          75%          max  \n",
       "instant      366.000000   548.500000   731.000000  \n",
       "season         3.000000     3.000000     4.000000  \n",
       "yr             0.500000     1.000000     1.000000  \n",
       "mnth           7.000000     9.750000    12.000000  \n",
       "holiday        0.000000     0.000000     1.000000  \n",
       "weekday        3.000000     5.000000     6.000000  \n",
       "workingday     1.000000     1.000000     1.000000  \n",
       "weathersit     1.000000     2.000000     3.000000  \n",
       "temp           0.499166     0.655625     0.861667  \n",
       "atemp          0.487364     0.608916     0.840896  \n",
       "hum            0.627083     0.730104     0.972500  \n",
       "windspeed      0.180971     0.233218     0.507463  \n",
       "casual       713.000000  1096.000000  3410.000000  \n",
       "registered  3662.000000  4776.500000  6946.000000  \n",
       "cnt         4548.000000  5956.000000  8714.000000  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe().T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "93aecae4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 731 entries, 0 to 730\n",
      "Data columns (total 16 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   instant     731 non-null    int64  \n",
      " 1   dteday      731 non-null    object \n",
      " 2   season      731 non-null    float64\n",
      " 3   yr          730 non-null    float64\n",
      " 4   mnth        730 non-null    float64\n",
      " 5   holiday     731 non-null    float64\n",
      " 6   weekday     731 non-null    float64\n",
      " 7   workingday  731 non-null    float64\n",
      " 8   weathersit  731 non-null    int64  \n",
      " 9   temp        730 non-null    float64\n",
      " 10  atemp       730 non-null    float64\n",
      " 11  hum         728 non-null    float64\n",
      " 12  windspeed   726 non-null    float64\n",
      " 13  casual      731 non-null    int64  \n",
      " 14  registered  731 non-null    int64  \n",
      " 15  cnt         731 non-null    int64  \n",
      "dtypes: float64(10), int64(5), object(1)\n",
      "memory usage: 91.5+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05dbd9b9",
   "metadata": {},
   "source": [
    "# Data Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c297ac3",
   "metadata": {},
   "source": [
    "### Handling Missing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ebee4e6c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "instant       0\n",
       "dteday        0\n",
       "season        0\n",
       "yr            1\n",
       "mnth          1\n",
       "holiday       0\n",
       "weekday       0\n",
       "workingday    0\n",
       "weathersit    0\n",
       "temp          1\n",
       "atemp         1\n",
       "hum           3\n",
       "windspeed     5\n",
       "casual        0\n",
       "registered    0\n",
       "cnt           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5690c535",
   "metadata": {},
   "source": [
    "Let's fill missing data with the median of each column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "85db549c",
   "metadata": {},
   "outputs": [],
   "source": [
    "values = {\"yr\":df[\"yr\"].median(),\n",
    "         \"mnth\":df[\"mnth\"].median(),\n",
    "         \"temp\":df[\"temp\"].median(),\n",
    "         \"atemp\":df[\"atemp\"].median(),\n",
    "         \"hum\":df[\"hum\"].median(),\n",
    "         \"windspeed\":df[\"windspeed\"].median()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b238555a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.fillna(value = values, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12a20b0d",
   "metadata": {},
   "source": [
    "Let's take a look at missing data again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c30a1766",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "instant       0\n",
       "dteday        0\n",
       "season        0\n",
       "yr            0\n",
       "mnth          0\n",
       "holiday       0\n",
       "weekday       0\n",
       "workingday    0\n",
       "weathersit    0\n",
       "temp          0\n",
       "atemp         0\n",
       "hum           0\n",
       "windspeed     0\n",
       "casual        0\n",
       "registered    0\n",
       "cnt           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce029314",
   "metadata": {},
   "source": [
    "### Removing unnecessary columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "444430d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop([\"casual\",\"registered\",\"dteday\"], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "988f3a0f",
   "metadata": {},
   "source": [
    "### Creating the target and feature variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6570d220",
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df[\"cnt\"]\n",
    "X = df.drop(\"cnt\", axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9fea581",
   "metadata": {},
   "source": [
    "### Splitting the dataset into the training and test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "56635212",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8ee22393",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f928f546",
   "metadata": {},
   "source": [
    "# Building The Model with XGBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "25276b6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBRegressor\n",
    "from sklearn.model_selection import cross_val_score\n",
    "model = XGBRegressor()\n",
    "xg_scores = cross_val_score(model, X, y, scoring=\"neg_mean_squared_error\", cv=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cbe54d3",
   "metadata": {},
   "source": [
    "Let's take a look at the mean of the scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7371b227",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 716.58107786,  669.242233  ,  513.94478758,  699.10947944,\n",
       "        843.96424434, 1046.137147  , 1005.04935475,  866.77141114,\n",
       "        904.24420278, 1698.27662405])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "rmse = np.sqrt(-xg_scores)\n",
    "rmse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "cabd6c2f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "896.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(rmse).round()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f420388f",
   "metadata": {},
   "source": [
    "# Building a Linear Regression Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f380fbba",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "969.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "lr = LinearRegression()\n",
    "lr_scores = cross_val_score(lr, X, y, scoring=\"neg_mean_squared_error\", cv = 10)\n",
    "lr_rmse = np.sqrt(-lr_scores)\n",
    "lr_rmse.mean().round()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c4bc232",
   "metadata": {},
   "source": [
    "Don't forget to follow us on [YouTube](http://youtube.com/tirendazacademy) | [Medium](http://tirendazacademy.medium.com) | [Twitter](http://twitter.com/tirendazacademy) | [GitHub](http://github.com/tirendazacademy) | [Linkedin](https://www.linkedin.com/in/tirendaz-academy) | [Kaggle](https://www.kaggle.com/tirendazacademy) 😎"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}