W02_linear_polynomial_regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Polynomial regression \n",
    "\n",
    "In this assignment we investigate the change in average temperature per year in the Netherlands. We are looking for a model that can accurately predict the average temperature of a year. We use polynomial regression for this.\n",
    "\n",
    "As usual, we will perform the following steps:\n",
    "\n",
    "- Load the dataset\n",
    "- Prepare the data set\n",
    "- Perform regression\n",
    "- Evaluate the results\n",
    "- Plot and evaluate the learning curves\n",
    "\n",
    "For this assignment we use the [GlobalLandTemperaturesByCountry.csv](https://www.kaggle.com/siddharthnishtala/basic-polynomial-regression-india/data).\n",
    "\n",
    "Some code will be given. In this case, make sure that your own variable names are the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# all packages we need\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "from sklearn import linear_model\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Read and prepare the data\n",
    "Perform the following steps:\n",
    "\n",
    "#### 1. Read in:\n",
    "- Read the CSV file.\n",
    "\n",
    "#### 2a. Prepare - remove what is unnecessary\n",
    "\n",
    "- Select only the rows where ** Country ** is ** Netherlands **.\n",
    "- With method `dropna ()` remove the rows where ** NaN ** 's exist for the ** AverageTemperature **.\n",
    "- Remove the columns with no added value: ** AverageTemperatureUncertainty ** and ** Country **.\n",
    "\n",
    "\n",
    "#### 2b. Prep - create what is needed\n",
    "\n",
    "- Create a new column ** Year **, in which we store the year of the measurement. Use the method `df.index.year`\n",
    "\n",
    "#### 2c. Aggregate\n",
    "\n",
    "- We now have a data frame with multiple temperature data per year. We will aggregate these to get a table with year and average temperature.\n",
    "- Using the data frame made above, create a new data frame with average temperature per year. To do this, use the method `df.groupby (). Mean ()`\n",
    "- Remove the years for which less than 12 values are known (why do we do this?)\n",
    "\n",
    "#### 2d. Plot the data\n",
    "\n",
    "- Plot the data (year on the x-axis, average temperature on the y-axis) to inspect it. What do you see?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code to prepare the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. load the data\n",
    "df = pd.read_csv(\"data/GlobalLandTemperaturesByCountry.csv\", index_col=0, parse_dates=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2a. enter your code here for the next steps\n",
    "# Select only the rows where Country is Netherlands.\n",
    "# Using method dropna () remove the rows where NaNs occur for the AverageTemperature.\n",
    "# Remove the columns with no added value: AverageTemperatureUncertainty and Country."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2b. enter your code here for the next steps\n",
    "# create column Year, in which we store the year of the measurement. use df.index.year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2c. enter your code for the aggregation here\n",
    "# Create data from df data frame with average temperature per year; use df.groupby (). mean ()\n",
    "# Delete the years for which less than 12 values are known\n",
    "data = data[df[\"Year\"].value_counts()==12]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.head()\n",
    "# in the generated overview year 1753 must have a temperature of 9.440083"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2d. Plot the data\n",
    "data[\"Year\"] = data.index\n",
    "plt.scatter(data[\"Year\"], data[\"AverageTemperature\"])\n",
    "plt.ylabel('Average Temperature')\n",
    "plt.xlabel('Year')\n",
    "plt.title('Average temperature in the Netherlands')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Moving average every 10 years\n",
    "\n",
    "We are now going to take the moving average every 10 years. This reduces the variance in the data set. When applying the moving average, `NaN`s sometimes arise again, so we have to remove those again. The code for this is given below. Then we plot the year `x` against the mean temperature` y`.\n",
    "\n",
    "sources:\n",
    "- moving average: https://en.wikipedia.org/wiki/Moving_average\n",
    "- moving average in pandas: https://pandas.pydata.org/pandasdocs/stable/reference/api/pandas.DataFrame.rolling.html\n",
    "\n",
    "NOTE: 10 was chosen randomly as the rolling average window. Feel free to try other windows later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# code for moving average\n",
    "df = data[\"AverageTemperature\"].rolling(window=10).mean()\n",
    "df = df.dropna()\n",
    "\n",
    "y = df.values.reshape(-1,1)\n",
    "X = df.index.values\n",
    "X = X.reshape(-1,1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the data\n",
    "plt.plot(X,y)\n",
    "# add code here to enhance the plot "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 3. Perform polynomial regression\n",
    "\n",
    "By plotting the moving average, we now have a sense of how the regression line should run. This is clearly not linear but a polynomial. We will now try in steps to fit the line with polynomial regression. We do this with the following steps\n",
    "\n",
    "#### 3a. Prepare the data\n",
    "- Normalize the data.\n",
    "We do this with a `StandardScaler ()` that normalizes to a Z score. We use the code from the `sklearn.preprocessing`. We always normalize the X values.\n",
    "- Split the data into train and test set.\n",
    "We use the method `train_test_split` from the module` sklearn.model_selection`. Take a `test_size = 0.3` and` random_state = 1`\n",
    "\n",
    "#### 3b. Model\n",
    "- Now try the polyfit model with different polynomial degrees (eg grade 1,2,3,4,10,40).\n",
    "- Plot the model against the data\n",
    "\n",
    "#### 3c. Evaluate\n",
    "- Calculate the Mean Square Error (Or the Root Mean Square Error) of the train and test data\n",
    "- What do you think is the best model? Why?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Functions you need\n",
    "def model(X, y, degree):\n",
    "    \"\"\" function that models a polynomial\"\"\"\n",
    "    poly_features = PolynomialFeatures(degree=degree, include_bias=False)\n",
    "    X_poly = poly_features.fit_transform(X)\n",
    "    lin_reg = LinearRegression()\n",
    "    lin_reg.fit(X_poly, y)\n",
    "    print(f'\\nmodel outcome for {degree} degrees:')\n",
    "    print(f'intercept: {lin_reg.intercept_} coefficient {lin_reg.coef_}')\n",
    "    return [lin_reg, poly_features]\n",
    "\n",
    "\n",
    "def predict(X, model, poly_features, n=1000):\n",
    "    \"\"\" function that predicts y based on a model \"\"\"\n",
    "    X_pred=np.linspace(X.min(), X.max(), n).reshape(n, 1)\n",
    "    X_pred_poly = poly_features.transform(X_pred)\n",
    "    y_pred = model.predict(X_pred_poly)\n",
    "    return [X_pred, y_pred]\n",
    "\n",
    "\n",
    "# Plot the points and the regression curve\n",
    "def plot_polynomial(X,y,X_pred,y_pred):\n",
    "    \"\"\"function that plots data and prediction\"\"\"\n",
    "    plt.plot(X, y, \"b.\", label = 'Data')\n",
    "    plt.plot(X_pred, y_pred, \"r-\", linewidth=2, label=\"model trainset\")\n",
    "    plt.ylabel('Average Temperature')\n",
    "    plt.xlabel('Year')\n",
    "    plt.title('Average temperature in the Netherlands -  TRAIN')\n",
    "    plt.legend()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#3a. Normalize the data (no changes needed)\n",
    "scaler = StandardScaler()\n",
    "scaler = scaler.fit(X)\n",
    "X = scaler.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#3a. Split de data (you can change test size if you want)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)\n",
    "print(\"train and test are splitted in:\")\n",
    "print(X_train.shape, y_train.shape)\n",
    "print(X_test.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#3b. Model\n",
    "degree = 3 # change this\n",
    "# call the model function\n",
    "# call predict() \n",
    "#3b. call plot_polynomial() to plot\n",
    "\n",
    "#3c. calculate mean squared error from test set\n",
    "X_pred, y_pred = predict(X_test, lin_reg, poly_features, len(y_test))\n",
    "print('Mean squared error: {mean_squared_error(y_test, y_pred):.2f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next\n",
    "\n",
    "This is the end of this notebook. We looked at only 1 feature and different order (degrees) of the model. Some models perform better than others. We can see that by the (root) mean square error. Because we only had 1 feature, we could visualize it easily. This becomes more difficult with multiple features. Next we will look at a technique called ** learning curve **. That is a method where you can evaluate models with multiple features."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}