pytorch_primer.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "pytorch_primer.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sweetpand/PyTorch_fun/blob/master/pytorch_primer.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Q1AhoIB-pkp",
        "colab_type": "text"
      },
      "source": [
        "# Introduction to Neural Networks\n",
        "\n",
        "This primer assumes familiarity with the concept of neural networks and is geared toward those who want to get started with Pytorch."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yJZkGE3oR_zQ",
        "colab_type": "text"
      },
      "source": [
        "# Getting Pytorch Going\n",
        "\n",
        "Let's get started by importing pytorch and some of the more important sub-modules."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4iSaSfH8bgSq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import torch.nn.functional as F\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy\n",
        "import random"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vJ3pDxmCqi1R",
        "colab_type": "text"
      },
      "source": [
        "You will likely want to make sure this notebook has access to a GPU. Run the following cell. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0As-zGSjqzJk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "torch.cuda.is_available()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0e6zcHTmq9y4",
        "colab_type": "text"
      },
      "source": [
        "If the result is ```False```, use the Edit >> Notebook Settings menu to turn on GPU access."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xzQHX38CcOC_",
        "colab_type": "text"
      },
      "source": [
        "# Generate a data set\n",
        "\n",
        "Before we start, we need a dataset. We could grab a dataset off the internet but we might not understand the nuances of the data very well. Instead, we will generate some data and then train on it.\n",
        "\n",
        "Let's pretend we are building a self-driving car. The car has proximity sensor and can act on that sensor information: it can accelerate or break, and it can turn the steering wheel left or right. Based on values from the proximity sensors, we want to infer how much energy to apply to the accelerator or breaks and how much to turn the steering wheel. \n",
        "\n",
        "Let's further assume that we have also put sensors on human driver's cars and we have been recording the data from the proximity sensors and also recording how much the human drivers press the accelerator or breaks and turn the wheel. This is called a *supervised learning* task. For every set of recorded sensor data, we know what a human has really done in response. Now all we need to do is train a neural network to produce the known-correct response when given a set of sensor data. If our model is good enough, we will trust it to make decisions on the car in situations that it has never seen before.\n",
        "\n",
        "What we are going to do here is synthetically generate the data set. Let's assume the car has 4 sensor: \n",
        "* proximity to car in front (0...1)\n",
        "* proximity to car in back (0...1)\n",
        "* proximity to car to the left (0...1)\n",
        "* proximity to car in on the right (0...1)\n",
        "\n",
        "We will randomly generate this data.\n",
        "\n",
        "We also need a supervisions signal: what the car should do under different circumstances. I have provided some simple rules for how the car should behave. This is a bit artificial--why do we need a neural network to drive the car if I know what the rules are and can write the code for those rules? In reality we wouldn't have these rules, but for learning pytorch we need a ground-truth of non-random supervision. So we will pretend that we acquired behavioral data for the car.\n",
        "\n",
        "There are 2 controls for the car:\n",
        "* accelerate: how much should we accelerate (-1...1)? Negative values means braking \n",
        "* turn: how much should we turn the steering wheel (-1...1)? Negative means left, positive means right.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_59yqkXObt_n",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def make_data(num_data):\n",
        "  data = []\n",
        "  for n in range(num_data):\n",
        "\n",
        "    # Generate a random state\n",
        "    # the distance to the nearest car in front/back/left/right is normalized from 0.0 (closest) to 1.0 (farthest)\n",
        "    carInFrontDist = random.random()\n",
        "    carInBackDist = random.random()\n",
        "    carLeftDist = random.random()\n",
        "    carRightDist = random.random()\n",
        "\n",
        "    # Response to the state. 1 =  brakes/accelerator/steer-left/steer-right is activated. 0=not activated\n",
        "    # Though binary, we will be using numbers\n",
        "    accel = 1.0\n",
        "    turn = 0.0\n",
        "\n",
        "    # Should I accelerate or brake?\n",
        "    if carInFrontDist < 0.50:\n",
        "      # Car is close, brake\n",
        "      # Unless there is another car close behind\n",
        "      if carInBackDist > 0.50:\n",
        "        # Okay to brake\n",
        "        accel = -carInFrontDist/0.50\n",
        "      else:\n",
        "        # Not okay to brake, but at least stop accelerating\n",
        "        accel = 0\n",
        "    else:\n",
        "      # Car in front is not close, continue to accelerate\n",
        "      accel = (carInFrontDist - 0.50)/0.50\n",
        "\n",
        "    # Should I turn left or right? (can't do both)\n",
        "    if carLeftDist < 0.5 or carRightDist < 0.5:\n",
        "      turn = (1.0 - (carLeftDist)) - (1.0 - carRightDist)\n",
        "\n",
        "    # Store the data\n",
        "    x = (carInFrontDist, carInBackDist, carLeftDist, carRightDist)\n",
        "    y = (accel, turn)\n",
        "    data.append((x, y))\n",
        "  return data"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Acgr6TPP_MZc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data = make_data(10000)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i2fMI5zzkK6B",
        "colab_type": "text"
      },
      "source": [
        "Let's take a look at the data. There is a lot, so let's just look at one data element.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MY7yZfRTdjHA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data[2]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H6yR1G17kTJv",
        "colab_type": "text"
      },
      "source": [
        "The data is an array. Each element is a tuple. The first element in the tuple is the input data (*x*), which contains the four sensor inputs. The second element in the tuple is the output data (*y*), the correct behavior of the car in response to the sensor inputs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VquL4SCW_ue0",
        "colab_type": "text"
      },
      "source": [
        "# Data Preparation\n",
        "\n",
        "We need to do two things. First, we need to convert our data into tensors, which are the array-like data structures that Pytorch uses. Fortunately, it is easy to create tensors from arrays and tuples.\n",
        "\n",
        "Second, we need to group chunks of data into batches. A batch is a chunk of data that can be run through the neural network in parallel. This greatly speeds up the training of the neural network and has some other advantages as well.\n",
        "\n",
        "The following function will get a chunk of data and convert it into tensors. We return a tensor for the input (*x*) and a tensor for the output (*y*). You will see that we are splitting out the inputs from the outputs. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N0HvIaRRlrgK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def get_batch(data, batch_size, index):\n",
        "  # Get a chunk of data (array of tuples)\n",
        "  start_index = index * batch_size\n",
        "  end_index = start_index + batch_size\n",
        "  batch = data[start_index:end_index]\n",
        "  # Turn the array into tensors\n",
        "  batch_inputs = [e[0] for e in batch]\n",
        "  batch_outputs = [e[1] for e in batch]\n",
        "  x = torch.tensor(batch_inputs)\n",
        "  y = torch.tensor(batch_outputs)\n",
        "  return x, y"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xvAiiH56AnPs",
        "colab_type": "text"
      },
      "source": [
        "Let's take a look at a batch:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y8YHvOtnoGD5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x, y = get_batch(train_data, 8, 0)\n",
        "\n",
        "print('x:')\n",
        "print(x)\n",
        "print('shape of x:', x.size())\n",
        "\n",
        "print('y:')\n",
        "print(y)\n",
        "print('shape of y:', y.size())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BZlGx9Jec7kY",
        "colab_type": "text"
      },
      "source": [
        "The first row of the *x* tensor should look like the first part of the training data tuple at index 0 above. Each row in the tensor is the input portion of a different line of data. It has been merged into what looks like a multi-dimensional array.\n",
        "\n",
        "The array has a particular *shape*. In particular it is an 8 x 4 array. The first dimension is for batching (there are 8 data points in the batch). The second dimension reveals that each data point has four values making it up.\n",
        "\n",
        "The first row of the *y* tensor matches the second part of the training data tuple at index 0 above. The *y* tensor is an 8 x 2 array because there are two controls for the car."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jhs9H3z8eS8s",
        "colab_type": "text"
      },
      "source": [
        "# The Neural Network\n",
        "\n",
        "Next we need to define the neural network. In pytorch we do this by creating a new class that sub-classes from ```nn.Module```. In this class we will define the different layers and indicate how the layers go together so that inputs flow through the layers to create outputs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YxRJRov_oIcx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class CarNet(nn.Module):\n",
        "\n",
        "    def __init__(self):\n",
        "        # Call the parent class constructor\n",
        "        super(CarNet, self).__init__()\n",
        "        # These are the layers\n",
        "        self.linear1 = nn.Linear(4, 16)\n",
        "        self.activation1 = nn.Tanh()\n",
        "        self.linear2 = nn.Linear(16, 8)\n",
        "        self.activation2 = nn.Tanh()\n",
        "        self.linear3 = nn.Linear(8, 2)\n",
        "        self.activation3 = nn.Tanh()\n",
        "\n",
        "    def forward(self, x):\n",
        "        h1 = self.activation1(self.linear1(x))\n",
        "        h2 = self.activation2(self.linear2(h1))\n",
        "        y_hat = self.activation3(self.linear3(h2))\n",
        "        return y_hat"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zzFOtjIueo8p",
        "colab_type": "text"
      },
      "source": [
        "Let's walk through this. We are going to create 2 layers of neurons with tanh activations. There are 4 inputs (4 sensor values) and 2 output values (accelerate and turn steering wheel). We use tanh activations instead of the traditional sigmoid because tanh can produce values between -1 and 1.\n",
        "\n",
        "In the constructor, I have set up the first layer as ```nn.Linear(4, 16)```. This creates a layer of 16 hidden neurons and fully connects it to 4 inputs. Actually, it doesn't know what these hidden neurons will be connected to, except that there will be 4 neurons. ```nn.Linear()``` is a way of growing or shrinking the width of a neural network.\n",
        "\n",
        "But wait, where are the weights? They are hidden inside the ```nn.Linear()``` object that we just instantiated.\n",
        "\n",
        "A linear layer doesn't have an activation function---it just grows or shrinks the width of our network. So we need to create another layer that has an activation function. ```activation1 = nn.Tahn()``` is the tanh activation function. It can take an arbitrary number of inputs and produces an identical number of outputs.\n",
        "\n",
        "For the next layer, we shrink the layer of 16 hidden nodes to a layer of 8 hidden nodes using another ```nn.Linear()``` and another ```nn.Tanh()```. \n",
        "\n",
        "Finally, we need to squeeze those eight activations into two outputs.  We use a third ```nn.Linear()``` to reduce from 8 to 2. The final activation makes sure our output values are between -1 and 1.\n",
        "\n",
        "Now all of our layers are instantiated in the constructor. However, we haven't specified exactlly how the outputs of one layer flows into another layer. We do that in the ```forward()``` function.\n",
        "\n",
        "The ```forward()``` function is the forward pass through the neural network. We are going to pass the inputs (*x*) into the forward function, and this function tells us what to do with those inputs. Specifically, we are going to send the inputs through ```linear1```. Recall that ```linear1``` is anticipating 4 inputs and will produce 16 outputs. The input (*x*) is an 8 x 4 tensor, meaning a batch of 8 inputs containing 4 values each.\n",
        "\n",
        "We send those outputs through ```activation1``` and store them in ```h1``` for convenience.\n",
        "\n",
        "Let's take a look at just this part of the forward function. The following simulated what is happing in the first layers of the neural network:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UEqricG05Z13",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "linear1 = nn.Linear(4, 16)\n",
        "activation1 = nn.Tanh()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kZa0i1hb6-pX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x, _ = get_batch(train_data, 8, 0)\n",
        "\n",
        "h1 = activation1(linear1(x))\n",
        "print(h1)\n",
        "print(\"shape of h1:\", h1.size())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8xqvtR5h6AX7",
        "colab_type": "text"
      },
      "source": [
        "You are looking at the output activations after running a batch of 8 data points (each consisting of 4 sensor values) through a linear layer and a tanh activation layer.\n",
        "\n",
        "The variable ```h1``` is a tensor of size 8 x 16. Our four inputs have become 16 values between -1 and 1. We still have 8 batches. These numbers are gibberish. What is happening is that the inputs are being multiplied by weights and adding additional bias values. This is what happens inside the ```nn.Linear()``` module. The network hasn't been trained so the input values are being multiplied with random weights in the linear expansion. Then those values are being run through tanh."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T3dhDuIL794t",
        "colab_type": "text"
      },
      "source": [
        "Let's now make a ```CarNet``` object:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LhWszFOBpzwN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net = CarNet()\n",
        "if torch.cuda.is_available():\n",
        "  net = net.to('cuda')\n",
        "print(net)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mkw_k1Ao8DdB",
        "colab_type": "text"
      },
      "source": [
        "The ```.to('cuda')``` call moves the network and all it's weights to the GPU's memory if the GPU is available.\n",
        "\n",
        "We can look at all the weights hidden deep inside the pytorch objects:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NArz4xdLgSZn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for i, param in enumerate(net.parameters()):\n",
        "  print('Parameter', i)\n",
        "  print(param)\n",
        "  print('shape:', param.size(), '\\n')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EwE2gvGih8Aa",
        "colab_type": "text"
      },
      "source": [
        "The first tensor has a 16 x 4 shape because it stores the weights inside ```linear1```, which is fully connecting the 4 input values to 16 hidden nodes. A fully-connected layer going from 4 inputs to 16 hidden nodes would have 64 weights, and that is what we see.\n",
        "\n",
        "The second tensor is 16 x 1. These are the bias weights inside ```linear1``` that are applied after the 4 inputs are multiplied by the weights in the prior tensor.\n",
        "\n",
        "The third tensor is 8 x 16 because these are the weights connecting the layer of 16 hidden notes to the layer of 8 hidden nodes. The fourth tensor is the 8 bias weights.\n",
        "\n",
        "The fifth and sixth tensors fully connect the eight hidden nodes to the two outputs values and apply bias weights.\n",
        "\n",
        "The activation functions don't have weights. They just apply their non-linear function to whatever inputs are passed in, regardless of size or shape."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JJ2l1gfD8Wt0",
        "colab_type": "text"
      },
      "source": [
        "# The Optimizer\n",
        "\n",
        "Now we have a neural network. Running the network's forward pass will construct the network. Then we will want to backpropagate a loss signal. There are a lot of different optimization functions. We have to pick one and instantiate it. \n",
        "\n",
        "We will use an optimization function called ```Adam``` and tell it about the parameters of our neural network. Since the CarNet class is just a wrapper, we must pass in parameters themselves---the weights.**bold text**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Mw7zwDRVqsoo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "optimizer = optim.Adam(net.parameters())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J-vYKyrpLU9E",
        "colab_type": "text"
      },
      "source": [
        "To optimize, we also need to be able to calculate the network's loss. In this case we will use Mean Square Error."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ls2oL2doEXdb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss_fn = nn.MSELoss()\n",
        "if torch.cuda.is_available():\n",
        "  loss_fn = loss_fn.to('cuda')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uSuCLAX_LlSR",
        "colab_type": "text"
      },
      "source": [
        "# Forward Pass\n",
        "\n",
        "To run the neural net, you call the network class object (CarNet) as if it were a function and pass in the inputs. This automatically calls the ```forward()``` function.\n",
        "\n",
        "Let's get a batch of data and move it to the GPU. Calling the neural network's forward function produces the predicted output, which we will save in ```y_hat``` to denote that it is a prediction and may not be accurate. (Indeed it won't be accurate because we haven't done any training yet.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2VDFngOosSu1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Get a batch\n",
        "x, y = get_batch(train_data, 8, 0)\n",
        "\n",
        "# Move data to the GPU\n",
        "if torch.cuda.is_available():\n",
        "  x = x.to('cuda')\n",
        "  y = y.to('cuda')\n",
        "  \n",
        "# Call the forward pass\n",
        "y_hat = net(x)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iQPzQKOuMXWu",
        "colab_type": "text"
      },
      "source": [
        "Let's take a look at what the output looks like:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0GmGS3-lMYaG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(y_hat)\n",
        "print('Shape of y_hat:', y_hat.size())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_9QIoFbsMy_Z",
        "colab_type": "text"
      },
      "source": [
        "The output tensor is the same shape as the *y* tensor. This is important because next we are going to compare *y_hat* to *y* to see how much we missed the correct answer by. That happens next."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3vyT77PAM6W4",
        "colab_type": "text"
      },
      "source": [
        "# Compute Loss\n",
        "\n",
        "Now we can compute the network's loss by passing *y_hat* and *y* into our loss function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BBjw8FrWsX0K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss = loss_fn(y_hat, y)\n",
        "print(loss)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kfa5KjzDNSvf",
        "colab_type": "text"
      },
      "source": [
        "# Backward Pass\n",
        "\n",
        "Now that we have the loss computed, we can backpropagate the loss through our network and the optimizer will adjust the weights for us.\n",
        "\n",
        "This is kind of weird, but to do this you call ```loss.backward()```. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZmdAJp1ItoRY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "optimizer.zero_grad()\n",
        "loss.backward()\n",
        "optimizer.step()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Td1r2Z76j7hi",
        "colab_type": "text"
      },
      "source": [
        "What is going on here? When we called ```forward()``` each element remembers how it was created. For example inside ```forward()```, *y_hat* remembered that it was created by ```activation3``` which was a tanh. The inputs to ```activation3``` remembered that it was created by ```linear2``` which was a ```Linear``` layer. Similarly, the *loss* variable remembered that it was created by the ```MSELoss``` function. You can see this remembering when you print tensors (the ```grad_fn``` part of each tensor).\n",
        "\n",
        "When you call ```backward()``` on the *loss* variable, you are telling it to pass it's value back up this chain (this chain is called a computation graph and is technically a directed acyclic graph). As the loss moves up the computation graph, it will encounter weights, such as those stored inside the ```Linear``` layers. The weights have gradients attached to them and the gradients are updated. This all happens because modules in Pytorch have ```backward()``` functions that know how to compute gradients and you never have to worry about it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZFH1RccYPlEi",
        "colab_type": "text"
      },
      "source": [
        "Let's look at the network's weights again. This time the weights are different---the optimizer adjusted them for us. We can also see the gradients that were computed (```.grad.data```) for each weight. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2pI_wR2APo8S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for i, param in enumerate(net.parameters()):\n",
        "  print('Parameter', i)\n",
        "  print(param)\n",
        "  print('gradients:')\n",
        "  print(param.grad.data, '\\n')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dNopTi8eQsAZ",
        "colab_type": "text"
      },
      "source": [
        "# Training Loop\n",
        "\n",
        "Putting this all together gives the following training loop. \n",
        "\n",
        "There are a few details that didn't come up before. \n",
        "* Before calling ```backward()```, we should zero out all the weights using ```optimizer.zero_grad()```. \n",
        "* After calling ```backward()```, we should call ```optimizer.step()``` which tells the optimizer that we completed another backward pass so that it can do any preparation for the next pass (like automatically adjusting learning rate).\n",
        "\n",
        "Run it and you should see the loss going down after every epoch."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6zFugTmAu6Ok",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "num_epochs = 100 # Number of epochs to run\n",
        "batch_size = 8   # Batch size\n",
        "num_batches = len(train_data) // batch_size # How many batches per epoch?\n",
        "loss_history = [] # Keep a record of losses over time for plotting\n",
        "\n",
        "# Make new network and optimizer\n",
        "net = CarNet()\n",
        "if torch.cuda.is_available():\n",
        "  net = net.to('cuda')\n",
        "optimizer = optim.Adam(net.parameters()) \n",
        "\n",
        "\n",
        "# Iterate a number of times\n",
        "for epoch in range(num_epochs):\n",
        "  epoch_loss = 0  # Keep track of how much loss we've acrued during this epoch\n",
        "  # An epoch is a complete run through all the batches\n",
        "  for i in range(num_batches):\n",
        "    # Zero out gradients\n",
        "    optimizer.zero_grad()\n",
        "    # Get a batch\n",
        "    x, y = get_batch(train_data, batch_size, i)\n",
        "    if torch.cuda.is_available():\n",
        "      x = x.to('cuda')\n",
        "      y = y.to('cuda')\n",
        "    # Forward pass\n",
        "    y_hat = net(x)\n",
        "    # Compute loss\n",
        "    loss = loss_fn(y_hat, y)\n",
        "    # Backward pass\n",
        "    loss.backward()\n",
        "    # Clean up\n",
        "    optimizer.step()\n",
        "    # Keep some stats\n",
        "    epoch_loss = epoch_loss + loss.item()\n",
        "  # Done with an epoch, print stats\n",
        "  print('epoch', epoch, 'epoch loss', epoch_loss/num_batches)\n",
        "  loss_history.append(epoch_loss/num_batches)   "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5a-CZyZqwYVL",
        "colab_type": "text"
      },
      "source": [
        "The exact loss values aren't so important. The key is that the numbers go down. If we plot the loss per epoch, we should see this more clearly."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l_61bqFu0ecO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# set up matplotlib\n",
        "is_ipython = 'inline' in matplotlib.get_backend()\n",
        "if is_ipython:\n",
        "    from IPython import display\n",
        "plt.ion()\n",
        "\n",
        "\n",
        "plt.figure(1)\n",
        "plt.clf()\n",
        "plt.xlabel('Epoch')\n",
        "plt.ylabel('Loss')\n",
        "plt.plot(numpy.array(loss_history))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YhegtERMS9_9",
        "colab_type": "text"
      },
      "source": [
        "# Evaluation\n",
        "\n",
        "How do we know the neural network is trained sufficiently? Loss much smaller than when we started. That is a good sign. But what we really need to do is test it on some data that the neural network has never seen before---we want to make sure it isn't memorizing the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ACZmqeQQ8JwU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "test_data = make_data(100)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h8_XZleETYHl",
        "colab_type": "text"
      },
      "source": [
        "We also need a testing procedure. We could measure how much difference there is between the network's predicted outputs and the true outputs. But maybe that isn't that meaningful. Does it matter if the neural network turns the steering wheel a little bit more than in the original data? The real test would be to drive the car. But we know this isn't a real car and we don't have a simulator anyway. \n",
        "\n",
        "So I'm going to make up a testing procedure. I am going to say that if the difference between the predicted acceleration and the true acceleration is greater than 10% then the car will crash. I'll do the same for the steering.\n",
        "\n",
        "The code below will compute the number of crashes:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ClT4W0KKCUoS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "total_crashes = 0 # How many crashes?\n",
        "\n",
        "# Prepare the network for evaluation. This turns off stuff inside the \n",
        "#    neural net modules that might result in randomness.\n",
        "net.eval()\n",
        "\n",
        "# Iterate through all the test data\n",
        "for i in range(len(test_data)):\n",
        "  # This time the batch is just a single datapoint\n",
        "  x, y = get_batch(test_data, 1, i)\n",
        "  if torch.cuda.is_available():\n",
        "    x = x.to('cuda')\n",
        "    y = y.to('cuda')\n",
        "  # Forward pass\n",
        "  y_hat = net(x)\n",
        "  # Compute the difference\n",
        "  diff = (y - y_hat).abs()\n",
        "  # Create a mask, 1 if difference is greater than 0.1\n",
        "  mask = diff > 0.1\n",
        "  crashes = mask.int().sum()\n",
        "  total_crashes = total_crashes + crashes.item()\n",
        "\n",
        "print(total_crashes)\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W5_jLhWEQcnk",
        "colab_type": "text"
      },
      "source": [
        "You'll notice in the above code that tensors come with a lot of built-in mathematical functions. You can subtract tensors, call functions like ```.abs()``` and ```.sum()```. You can even apply boolean operators to tensor elements.\n",
        "\n",
        "How did the car do? Did it crash too many times. You can go back an increase the number of training epochs. You should be able to bring the number of crashes down."
      ]
    }
  ]
}