sckit_learn_fcc_predict_health_costs_with_regression(1).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "sckit_learn_fcc_predict_health_costs_with_regression.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M9TX15KOkPBV"
      },
      "source": [
        "*Note: You are currently reading this using Google Colaboratory which is a cloud-hosted version of Jupyter Notebook. This is a document containing both text cells for documentation and runnable code cells. If you are unfamiliar with Jupyter Notebook, watch this 3-minute introduction before starting this challenge: https://www.youtube.com/watch?v=inN8seMm7UI*\n",
        "\n",
        "---\n",
        "\n",
        "In this challenge, you will predict healthcare costs using a regression algorithm.\n",
        "\n",
        "You are given a dataset that contains information about different people including their healthcare costs. Use the data to predict healthcare costs based on new data.\n",
        "\n",
        "The first two cells of this notebook import libraries and the data.\n",
        "\n",
        "Make sure to convert categorical data to numbers. Use 80% of the data as the `train_dataset` and 20% of the data as the `test_dataset`.\n",
        "\n",
        "`pop` off the \"expenses\" column from these datasets to create new datasets called `train_labels` and `test_labels`. Use these labels when training your model.\n",
        "\n",
        "Create a model and train it with the `train_dataset`. Run the final cell in this notebook to check your model. The final cell will use the unseen `test_dataset` to check how well the model generalizes.\n",
        "\n",
        "To pass the challenge, `model.evaluate` must return a Mean Absolute Error of under 3500. This means it predicts health care costs correctly within $3500.\n",
        "\n",
        "The final cell will also predict expenses using the `test_dataset` and graph the results."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1rRo8oNqZ-Rj"
      },
      "source": [
        "# Import libraries. You may or may not use all of these.\n",
        "!pip install -q git+https://github.com/tensorflow/docs\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "try:\n",
        "  # %tensorflow_version only exists in Colab.\n",
        "  %tensorflow_version 2.x\n",
        "except Exception:\n",
        "  pass\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CiX2FI4gZtTt"
      },
      "source": [
        "# Import data\n",
        "!wget https://cdn.freecodecamp.org/project-data/health-costs/insurance.csv\n",
        "df = pd.read_csv('insurance.csv')\n",
        "df.tail()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "97AkXl-LcqXG"
      },
      "source": [
        "df.isnull().sum()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K5GPUsPjeyEr"
      },
      "source": [
        "df[\"region\"].value_counts()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LcopvQh3X-kX"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import pandas as pd\n",
        "\n",
        "# from sklearn.impute import SimpleImputer\n",
        "\n",
        "# imputer = SimpleImputer(missing_values=np.nan, strategy='mean')\n",
        "\n",
        "\n",
        "sex = df.iloc[:,1:2].values\n",
        "print(sex)\n",
        "\n",
        "from sklearn import preprocessing\n",
        "\n",
        "ohe = preprocessing.OneHotEncoder(drop=\"first\") #dropping dummies\n",
        "sex = ohe.fit_transform(sex).toarray()\n",
        "print(sex)\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aGf3FgUBlzeK"
      },
      "source": [
        "smoker = df.iloc[:,4:5]\n",
        "ohe = preprocessing.OneHotEncoder(drop=\"first\") #dropping dummies\n",
        "smoker = ohe.fit_transform(smoker).toarray()\n",
        "print(smoker)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8VRsD22senn1"
      },
      "source": [
        "le = preprocessing.LabelEncoder()\n",
        "\n",
        "region = le.fit_transform(df.iloc[:,5:6])\n",
        "\n",
        "print(region)\n",
        "ohe = preprocessing.OneHotEncoder()\n",
        "region = ohe.fit_transform(region.reshape(-1,1)).toarray()\n",
        "print(region)\n",
        "\n",
        "\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eEgju-SojucV"
      },
      "source": [
        "r1 =  pd.DataFrame(data=sex, index= range(1338), columns = [\"sex\"])\n",
        "print(r1)\n",
        "\n",
        "r2 =  pd.DataFrame(data=smoker, index= range(1338), columns = [\"smoker\"])\n",
        "print(r2)\n",
        "\n",
        "\n",
        "r3 =  pd.DataFrame(data=region, index= range(1338), columns = [\"southeast\",\"southwest\",\n",
        "                                                               \"northeast\",\"northwest\"])\n",
        "print(r3)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RSy7ZXv-mNcX"
      },
      "source": [
        "df_r = pd.concat([r1,r2],axis=1)\n",
        "\n",
        "df_r = pd.concat([df_r,r3],axis=1)\n",
        "\n",
        "df_r"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JOgfIbMWvDk-"
      },
      "source": [
        "df = df.drop([\"sex\",\"smoker\",\"region\"],axis=1)\n",
        "df_r= pd.concat([df_r,df],axis=1)\n",
        "\n",
        "df_r"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RQ3jV1m7cZZN"
      },
      "source": [
        "from sklearn.metrics import mean_absolute_error\n",
        "\n",
        "train_labels = df_r.iloc[:,1:9].values #independent \n",
        "test_labels = df_r.iloc[:,9:].values #dependent\n",
        "\n",
        "#Splitting\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "x_train, x_test,y_train,y_test = train_test_split(train_labels,test_labels,test_size=0.2, random_state=0)\n",
        "\n",
        "\n",
        "\n",
        "#Scaling\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "sc=StandardScaler()\n",
        "\n",
        "X_train = sc.fit_transform(x_train)\n",
        "X_test = sc.transform(x_test)\n",
        "\n",
        "\n",
        "from sklearn.linear_model import LinearRegression\n",
        "lr = LinearRegression()\n",
        "lr.fit(X_train,y_train)\n",
        "\n",
        "y_pred = lr.predict(X_test)\n",
        "print(y_pred)\n",
        "print(y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vEkmjnu03DMd"
      },
      "source": [
        "from sklearn.metrics import mean_absolute_error, mean_squared_error\n",
        "mae = mean_absolute_error(y_test, y_pred)\n",
        "# mean_squared_error(y_test, y_pred)\n",
        "plt.scatter(y_test, y_pred)\n",
        "plt.xlabel('True values (expenses)')\n",
        "plt.ylabel('Predictions (expenses)')\n",
        "\n",
        "\n",
        "print(mae)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UFBNk_O48FoX"
      },
      "source": [
        "from sklearn.model_selection import cross_val_predict\n",
        "predicted = cross_val_predict(lr, X= df_r, y= test_labels, cv=10)\n",
        "\n",
        "fig, ax = plt.subplots()\n",
        "ax.scatter(df[\"expenses\"], predicted)\n",
        "ax.plot([x_train.min(), x_train.max()], [x_train.min(), x_train.max()], 'k--', lw=4)\n",
        "ax.set_xlabel('Measured')\n",
        "ax.set_ylabel('Predicted')\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Uf1aoUWt8UIM"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}