19. 欠拟合与过拟合.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实验：欠拟合与过拟合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实验概要\n",
    "\n",
    "#### 过拟合和欠拟合\n",
    "\n",
    "模型训练中经常出现两类典型问题：\n",
    "\n",
    "- 一类是模型无法得到较低的训练误差，我们将这一现象称作 **欠拟合（underfitting）**\n",
    "- 另一类是模型的训练误差远小于它在测试数据集上的误差，我们称该现象为 **过拟合（overfitting）**\n",
    "\n",
    "在实践中，我们要尽可能同时应对欠拟合和过拟合。虽然有很多因素可能导致这两种拟合问题，在这里我们重点讨论两个因素：模型复杂度和训练数据集大小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 模型复杂度\n",
    "\n",
    "为了解释模型复杂度，我们以多项式函数拟合为例。给定一个由标量数据特征 $x$ 和对应的标量标签 $y$ 组成的训练数据集，多项式函数拟合的目标是找一个$K$阶多项式函数\n",
    "\n",
    "$$\\hat{y} = b + \\sum_{k=1}^K x^k w_k$$\n",
    "\n",
    "来拟合 $y$。\n",
    "\n",
    "在上式中，$w_k$ 是模型的权重参数（weight），$b$ 是偏置参数（bias）。与线性回归相同，多项式函数拟合也使用平方损失函数。特别地，一阶多项式函数拟合又叫线性函数拟合。因为高阶多项式函数模型参数更多，模型函数的选择空间更大，所以高阶多项式函数比低阶多项式函数的复杂度更高。因此，高阶多项式函数比低阶多项式函数更容易在相同的训练数据集上得到更低的训练误差。给定训练数据集，模型复杂度和误差之间的关系通常如图所示。给定训练数据集，如果模型的复杂度过低，很容易出现欠拟合；如果模型复杂度过高，很容易出现过拟合。应对欠拟合和过拟合的一个办法是针对数据集选择合适复杂度的模型。\n",
    "\n",
    "<img src=\"./img/overfit_19.jpg\" width=\"50%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练数据集大小\n",
    "\n",
    "影响欠拟合和过拟合的另一个重要因素是训练数据集的大小。一般来说，如果训练数据集中样本数过少，特别是比模型参数数量（按元素计）更少时，过拟合更容易发生。此外，泛化误差不会随训练数据集里样本数量增加而增大。因此，在计算资源允许的范围之内，我们通常希望训练数据集大一些，特别是在模型复杂度较高时，如层数较多的深度学习模型。\n",
    "\n",
    "为了理解模型复杂度和训练数据集大小对欠拟合和过拟合的影响，下面我们以多项式函数拟合为例来实验。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实验目标\n",
    "\n",
    "在本实验中，我们将生成一个人工数据集，通过 TensorFlow 生成神经网络模型，分别观察：正常、欠拟合与过拟合的训练曲线。从而使你对训练曲线的状态建立直观的感知，通过观察训练曲线，更准确的判断模型是否存在过拟合状态。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 导入库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 生成数据集\n",
    "\n",
    "在本实验中，我们将生成一个人工数据集。在训练数据集和测试数据集中，给定样本特征$x$，我们使用如下的三阶多项式函数来生成该样本的标签：\n",
    "\n",
    "$$y = 1.2x - 3.4x^2 + 5.6x^3 + 5 + \\epsilon,$$\n",
    "\n",
    "其中噪声项 $\\epsilon$ 服从均值为 0、标准差为 0.1 的正态分布。训练数据集和测试数据集的样本数都设为 100。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_train, n_test, true_w, true_b = 100, 100, [1.2, -3.4, 5.6], 5\n",
    "features = tf.random.normal(shape=(n_train + n_test, 1))\n",
    "poly_features = tf.concat([features, tf.pow(features, 2), \n",
    "                           tf.pow(features, 3)], axis=1)\n",
    "labels = (true_w[0] * poly_features[:, 0] + true_w[1] * poly_features[:, 1] + \n",
    "          true_w[2] * poly_features[:, 2] + true_b)\n",
    "labels += tf.random.normal(labels.shape, stddev=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看一看生成的数据集的前两个样本："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Tensor: shape=(2, 1), dtype=float32, numpy=\n",
       " array([[-1.7774429 ],\n",
       "        [-0.41630965]], dtype=float32)>,\n",
       " <tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
       " array([[-1.7774429 ,  3.1593034 , -5.6154814 ],\n",
       "        [-0.41630965,  0.17331372, -0.07215218]], dtype=float32)>,\n",
       " <tf.Tensor: shape=(2,), dtype=float32, numpy=array([-39.31537  ,   3.5542302], dtype=float32)>)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features[:2], poly_features[:2], labels[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设置训练曲线显示图形的参数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython import display\n",
    "def use_svg_display():\n",
    "    \"\"\"以 svg 格式在 jupyter notebook 中显示图形\"\"\"\n",
    "    display.set_matplotlib_formats('svg')\n",
    "\n",
    "def set_figsize(figsize=(3.5, 2.5)):\n",
    "    \"\"\"设置 Matplotlib 图形尺寸\"\"\"\n",
    "    use_svg_display()\n",
    "    plt.rcParams['figure.figsize'] = figsize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 定义、训练和测试模型\n",
    "\n",
    "我们先定义作图函数 `semilogy`，其中 `y` 轴使用了对数尺度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def semilogy(x_vals, y_vals, x_label, y_label, x2_vals=None, y2_vals=None,\n",
    "             legend=None, figsize=(3.5, 2.5)):\n",
    "    set_figsize(figsize)\n",
    "    plt.xlabel(x_label)\n",
    "    plt.ylabel(y_label)\n",
    "    plt.semilogy(x_vals, y_vals)\n",
    "    if x2_vals and y2_vals:\n",
    "        plt.semilogy(x2_vals, y2_vals, linestyle=':')\n",
    "        plt.legend(legend)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和线性回归一样，多项式函数拟合也使用平方损失函数。\n",
    "\n",
    "因为我们将尝试使用不同复杂度的模型来拟合生成的数据集，所以我们把模型定义部分放在 `fit_and_plot` 函数中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_epochs, loss = 100, tf.losses.MeanSquaredError()\n",
    "\n",
    "def fit_and_plot(train_features, test_features, train_labels, test_labels):\n",
    "    net = tf.keras.Sequential()\n",
    "    net.add(tf.keras.layers.Dense(1))\n",
    "    batch_size = min(10, train_labels.shape[0])\n",
    "    train_iter = tf.data.Dataset.from_tensor_slices(\n",
    "        (train_features, train_labels)).batch(batch_size)\n",
    "    test_iter = tf.data.Dataset.from_tensor_slices(\n",
    "        (test_features, test_labels)).batch(batch_size)\n",
    "    optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)\n",
    "    train_ls, test_ls = [], []\n",
    "    for _ in range(num_epochs):\n",
    "        for X, y in train_iter:\n",
    "            with tf.GradientTape() as tape:\n",
    "                l = loss(y, net(X))\n",
    "\n",
    "            grads = tape.gradient(l, net.trainable_variables)\n",
    "            optimizer.apply_gradients(zip(grads, net.trainable_variables))\n",
    "\n",
    "        train_ls.append(loss(train_labels, net(train_features)).numpy().mean())\n",
    "        test_ls.append(loss(test_labels, net(test_features)).numpy().mean())\n",
    "    print('final epoch: train loss', train_ls[-1], 'test loss', test_ls[-1])\n",
    "    semilogy(range(1, num_epochs + 1), train_ls, 'epochs', 'loss',\n",
    "             range(1, num_epochs + 1), test_ls, ['train', 'test'])\n",
    "    print('weight:', net.get_weights()[0],\n",
    "          '\\nbias:', net.get_weights()[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. 三阶多项式函数拟合（正常）\n",
    "\n",
    "我们先使用与数据生成函数同阶的三阶多项式函数拟合。\n",
    "\n",
    "实验表明，这个模型的训练误差和在测试数据集的误差都较低。训练出的模型参数也接近真实值：$w_1 = 1.2, w_2=-3.4, w_3=5.6, b = 5$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "final epoch: train loss 0.00947398 test loss 0.011410384\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
       "<svg height=\"184.15625pt\" version=\"1.1\" viewBox=\"0 0 257.378125 184.15625\" width=\"257.378125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 184.15625 \n",
       "L 257.378125 184.15625 \n",
       "L 257.378125 -0 \n",
       "L 0 -0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 51.378125 146.6 \n",
       "L 246.678125 146.6 \n",
       "L 246.678125 10.7 \n",
       "L 51.378125 10.7 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"me476a743d5\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.462009\" xlink:href=\"#me476a743d5\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "z\n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(55.280759 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"94.329778\" xlink:href=\"#me476a743d5\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 20 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(87.967278 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"130.197546\" xlink:href=\"#me476a743d5\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 40 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"DejaVuSans-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(123.835046 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"166.065315\" xlink:href=\"#me476a743d5\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 60 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "z\n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(159.702815 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"201.933084\" xlink:href=\"#me476a743d5\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 80 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "z\n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "z\n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "z\n",
       "\" id=\"DejaVuSans-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(195.570584 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"237.800852\" xlink:href=\"#me476a743d5\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 100 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(228.257102 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- epochs -->\n",
       "     <defs>\n",
       "      <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "z\n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"DejaVuSans-65\"/>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "z\n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-70\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "z\n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-6f\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-63\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-68\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "z\n",
       "\" id=\"DejaVuSans-73\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(131.195312 174.876562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"61.523438\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"125\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"186.181641\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"241.162109\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"304.541016\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"me6c353f8ab\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"51.378125\" xlink:href=\"#me6c353f8ab\" y=\"139.608292\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- $\\mathdefault{10^{-2}}$ -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.59375 35.5 \n",
       "L 73.1875 35.5 \n",
       "L 73.1875 27.203125 \n",
       "L 10.59375 27.203125 \n",
       "z\n",
       "\" id=\"DejaVuSans-2212\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 143.407511)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use transform=\"translate(186.855469 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"51.378125\" xlink:href=\"#me6c353f8ab\" y=\"104.903497\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- $\\mathdefault{10^{-1}}$ -->\n",
       "      <g transform=\"translate(20.878125 108.702716)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use transform=\"translate(186.855469 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"51.378125\" xlink:href=\"#me6c353f8ab\" y=\"70.198702\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- $\\mathdefault{10^{0}}$ -->\n",
       "      <g transform=\"translate(26.778125 73.997921)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"51.378125\" xlink:href=\"#me6c353f8ab\" y=\"35.493907\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- $\\mathdefault{10^{1}}$ -->\n",
       "      <g transform=\"translate(26.778125 39.293126)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -2 0 \n",
       "\" id=\"mf9da71ec23\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"144.984133\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"142.971534\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"141.196296\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"129.161108\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_9\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"123.049897\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_10\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"118.713923\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_11\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"115.350681\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_12\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"112.602712\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_13\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"110.279338\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_14\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"108.266739\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_15\">\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"106.491501\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_16\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"94.456313\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_17\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"88.345102\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_18\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"84.009129\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_19\">\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"80.645886\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_20\">\n",
       "     <g id=\"line2d_26\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"77.897918\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_21\">\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"75.574543\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_22\">\n",
       "     <g id=\"line2d_28\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"73.561944\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_23\">\n",
       "     <g id=\"line2d_29\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"71.786706\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_24\">\n",
       "     <g id=\"line2d_30\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"59.751518\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_25\">\n",
       "     <g id=\"line2d_31\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"53.640307\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_26\">\n",
       "     <g id=\"line2d_32\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"49.304334\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_27\">\n",
       "     <g id=\"line2d_33\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"45.941092\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_28\">\n",
       "     <g id=\"line2d_34\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"43.193123\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_29\">\n",
       "     <g id=\"line2d_35\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"40.869748\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_30\">\n",
       "     <g id=\"line2d_36\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"38.857149\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_31\">\n",
       "     <g id=\"line2d_37\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"37.081912\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_32\">\n",
       "     <g id=\"line2d_38\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"25.046723\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_33\">\n",
       "     <g id=\"line2d_39\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"18.935512\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_34\">\n",
       "     <g id=\"line2d_40\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"14.599539\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_35\">\n",
       "     <g id=\"line2d_41\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"51.378125\" xlink:href=\"#mf9da71ec23\" y=\"11.236297\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-6c\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798437 88.307812)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"27.783203\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"88.964844\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"141.064453\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_42\">\n",
       "    <path clip-path=\"url(#p6dcb88f616)\" d=\"M 60.255398 23.571677 \n",
       "L 62.048786 36.130539 \n",
       "L 63.842175 39.721462 \n",
       "L 65.635563 42.528872 \n",
       "L 67.428951 45.31561 \n",
       "L 69.22234 48.097966 \n",
       "L 71.015728 50.869609 \n",
       "L 72.809117 53.630001 \n",
       "L 74.602505 56.379865 \n",
       "L 76.395894 59.119601 \n",
       "L 78.189282 61.849078 \n",
       "L 79.98267 64.567731 \n",
       "L 81.776059 67.274726 \n",
       "L 83.569447 69.968974 \n",
       "L 85.362836 72.649191 \n",
       "L 87.156224 75.313859 \n",
       "L 88.949613 77.961349 \n",
       "L 90.743001 80.589723 \n",
       "L 92.536389 83.196925 \n",
       "L 94.329778 85.78059 \n",
       "L 96.123166 88.338109 \n",
       "L 97.916555 90.866624 \n",
       "L 99.709943 93.363003 \n",
       "L 101.503332 95.823832 \n",
       "L 103.29672 98.245338 \n",
       "L 105.090108 100.623547 \n",
       "L 106.883497 102.954135 \n",
       "L 108.676885 105.232661 \n",
       "L 110.470274 107.45434 \n",
       "L 112.263662 109.61432 \n",
       "L 114.057051 111.707807 \n",
       "L 115.850439 113.729832 \n",
       "L 117.643827 115.675835 \n",
       "L 119.437216 117.541308 \n",
       "L 121.230604 119.322319 \n",
       "L 123.023993 121.015579 \n",
       "L 124.817381 122.618239 \n",
       "L 126.61077 124.128495 \n",
       "L 128.404158 125.545165 \n",
       "L 130.197546 126.86825 \n",
       "L 131.990935 128.098465 \n",
       "L 133.784323 129.237493 \n",
       "L 135.577712 130.287763 \n",
       "L 137.3711 131.252534 \n",
       "L 139.164489 132.135704 \n",
       "L 140.957877 132.941459 \n",
       "L 142.751265 133.674566 \n",
       "L 144.544654 134.339784 \n",
       "L 146.338042 134.942031 \n",
       "L 148.131431 135.486285 \n",
       "L 149.924819 135.977427 \n",
       "L 151.718208 136.419994 \n",
       "L 153.511596 136.818372 \n",
       "L 155.304985 137.17671 \n",
       "L 157.098373 137.498957 \n",
       "L 158.891761 137.78859 \n",
       "L 160.68515 138.048879 \n",
       "L 162.478538 138.282875 \n",
       "L 164.271927 138.493208 \n",
       "L 166.065315 138.682305 \n",
       "L 167.858704 138.852431 \n",
       "L 169.652092 139.005379 \n",
       "L 171.44548 139.143203 \n",
       "L 173.238869 139.267437 \n",
       "L 175.032257 139.379308 \n",
       "L 176.825646 139.480206 \n",
       "L 178.619034 139.571318 \n",
       "L 180.412423 139.653601 \n",
       "L 182.205811 139.727902 \n",
       "L 183.999199 139.79511 \n",
       "L 185.792588 139.855912 \n",
       "L 187.585976 139.910971 \n",
       "L 189.379365 139.9608 \n",
       "L 191.172753 140.005915 \n",
       "L 192.966142 140.046956 \n",
       "L 194.75953 140.084153 \n",
       "L 196.552918 140.117899 \n",
       "L 198.346307 140.14855 \n",
       "L 200.139695 140.176455 \n",
       "L 201.933084 140.201814 \n",
       "L 203.726472 140.224822 \n",
       "L 205.519861 140.245819 \n",
       "L 207.313249 140.264937 \n",
       "L 209.106637 140.282292 \n",
       "L 210.900026 140.298237 \n",
       "L 212.693414 140.312756 \n",
       "L 214.486803 140.325941 \n",
       "L 216.280191 140.338006 \n",
       "L 218.07358 140.348979 \n",
       "L 219.866968 140.359039 \n",
       "L 221.660356 140.368117 \n",
       "L 223.453745 140.376623 \n",
       "L 225.247133 140.38428 \n",
       "L 227.040522 140.39133 \n",
       "L 228.83391 140.397788 \n",
       "L 230.627299 140.403714 \n",
       "L 232.420687 140.409086 \n",
       "L 234.214075 140.414083 \n",
       "L 236.007464 140.41859 \n",
       "L 237.800852 140.422727 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_43\">\n",
       "    <path clip-path=\"url(#p6dcb88f616)\" d=\"M 60.255398 16.877273 \n",
       "L 62.048786 31.911774 \n",
       "L 63.842175 32.987729 \n",
       "L 65.635563 34.544471 \n",
       "L 67.428951 36.694688 \n",
       "L 69.22234 39.072626 \n",
       "L 71.015728 41.541381 \n",
       "L 72.809117 44.051127 \n",
       "L 74.602505 46.580299 \n",
       "L 76.395894 49.117323 \n",
       "L 78.189282 51.654954 \n",
       "L 79.98267 54.188204 \n",
       "L 81.776059 56.713506 \n",
       "L 83.569447 59.22811 \n",
       "L 85.362836 61.729971 \n",
       "L 87.156224 64.217261 \n",
       "L 88.949613 66.688576 \n",
       "L 90.743001 69.142579 \n",
       "L 92.536389 71.578069 \n",
       "L 94.329778 73.993754 \n",
       "L 96.123166 76.388535 \n",
       "L 97.916555 78.761089 \n",
       "L 99.709943 81.110165 \n",
       "L 101.503332 83.434438 \n",
       "L 103.29672 85.732467 \n",
       "L 105.090108 88.002757 \n",
       "L 106.883497 90.243714 \n",
       "L 108.676885 92.453548 \n",
       "L 110.470274 94.630459 \n",
       "L 112.263662 96.772457 \n",
       "L 114.057051 98.877536 \n",
       "L 115.850439 100.943178 \n",
       "L 117.643827 102.967346 \n",
       "L 119.437216 104.947555 \n",
       "L 121.230604 106.880953 \n",
       "L 123.023993 108.76494 \n",
       "L 124.817381 110.597022 \n",
       "L 126.61077 112.374216 \n",
       "L 128.404158 114.093863 \n",
       "L 130.197546 115.753232 \n",
       "L 131.990935 117.349739 \n",
       "L 133.784323 118.881168 \n",
       "L 135.577712 120.345318 \n",
       "L 137.3711 121.740045 \n",
       "L 139.164489 123.064162 \n",
       "L 140.957877 124.316557 \n",
       "L 142.751265 125.496576 \n",
       "L 144.544654 126.604355 \n",
       "L 146.338042 127.640233 \n",
       "L 148.131431 128.604743 \n",
       "L 149.924819 129.499679 \n",
       "L 151.718208 130.326834 \n",
       "L 153.511596 131.088391 \n",
       "L 155.304985 131.7871 \n",
       "L 157.098373 132.425856 \n",
       "L 158.891761 133.007721 \n",
       "L 160.68515 133.536146 \n",
       "L 162.478538 134.014509 \n",
       "L 164.271927 134.446058 \n",
       "L 166.065315 134.834628 \n",
       "L 167.858704 135.183045 \n",
       "L 169.652092 135.494868 \n",
       "L 171.44548 135.773273 \n",
       "L 173.238869 136.021117 \n",
       "L 175.032257 136.241163 \n",
       "L 176.825646 136.436124 \n",
       "L 178.619034 136.608403 \n",
       "L 180.412423 136.76034 \n",
       "L 182.205811 136.893742 \n",
       "L 183.999199 137.010899 \n",
       "L 185.792588 137.113218 \n",
       "L 187.585976 137.202391 \n",
       "L 189.379365 137.279907 \n",
       "L 191.172753 137.346915 \n",
       "L 192.966142 137.404589 \n",
       "L 194.75953 137.454161 \n",
       "L 196.552918 137.496151 \n",
       "L 198.346307 137.531953 \n",
       "L 200.139695 137.562064 \n",
       "L 201.933084 137.587169 \n",
       "L 203.726472 137.607584 \n",
       "L 205.519861 137.624414 \n",
       "L 207.313249 137.637902 \n",
       "L 209.106637 137.648179 \n",
       "L 210.900026 137.655941 \n",
       "L 212.693414 137.661398 \n",
       "L 214.486803 137.664928 \n",
       "L 216.280191 137.666912 \n",
       "L 218.07358 137.667226 \n",
       "L 219.866968 137.666324 \n",
       "L 221.660356 137.664216 \n",
       "L 223.453745 137.661465 \n",
       "L 225.247133 137.657802 \n",
       "L 227.040522 137.653509 \n",
       "L 228.83391 137.64877 \n",
       "L 230.627299 137.643614 \n",
       "L 232.420687 137.638042 \n",
       "L 234.214075 137.632064 \n",
       "L 236.007464 137.626079 \n",
       "L 237.800852 137.619698 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 51.378125 146.6 \n",
       "L 51.378125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 246.678125 146.6 \n",
       "L 246.678125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 51.378125 146.6 \n",
       "L 246.678125 146.6 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 51.378125 10.7 \n",
       "L 246.678125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 184.403125 48.05625 \n",
       "L 239.678125 48.05625 \n",
       "Q 241.678125 48.05625 241.678125 46.05625 \n",
       "L 241.678125 17.7 \n",
       "Q 241.678125 15.7 239.678125 15.7 \n",
       "L 184.403125 15.7 \n",
       "Q 182.403125 15.7 182.403125 17.7 \n",
       "L 182.403125 46.05625 \n",
       "Q 182.403125 48.05625 184.403125 48.05625 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_44\">\n",
       "     <path d=\"M 186.403125 23.798437 \n",
       "L 206.403125 23.798437 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_45\"/>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train -->\n",
       "     <defs>\n",
       "      <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-74\"/>\n",
       "      <path d=\"M 41.109375 46.296875 \n",
       "Q 39.59375 47.171875 37.8125 47.578125 \n",
       "Q 36.03125 48 33.890625 48 \n",
       "Q 26.265625 48 22.1875 43.046875 \n",
       "Q 18.109375 38.09375 18.109375 28.8125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 20.953125 51.171875 25.484375 53.578125 \n",
       "Q 30.03125 56 36.53125 56 \n",
       "Q 37.453125 56 38.578125 55.875 \n",
       "Q 39.703125 55.765625 41.0625 55.515625 \n",
       "z\n",
       "\" id=\"DejaVuSans-72\"/>\n",
       "      <path d=\"M 34.28125 27.484375 \n",
       "Q 23.390625 27.484375 19.1875 25 \n",
       "Q 14.984375 22.515625 14.984375 16.5 \n",
       "Q 14.984375 11.71875 18.140625 8.90625 \n",
       "Q 21.296875 6.109375 26.703125 6.109375 \n",
       "Q 34.1875 6.109375 38.703125 11.40625 \n",
       "Q 43.21875 16.703125 43.21875 25.484375 \n",
       "L 43.21875 27.484375 \n",
       "z\n",
       "M 52.203125 31.203125 \n",
       "L 52.203125 0 \n",
       "L 43.21875 0 \n",
       "L 43.21875 8.296875 \n",
       "Q 40.140625 3.328125 35.546875 0.953125 \n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \n",
       "Q 6 8.015625 6 15.921875 \n",
       "Q 6 25.140625 12.171875 29.828125 \n",
       "Q 18.359375 34.515625 30.609375 34.515625 \n",
       "L 43.21875 34.515625 \n",
       "L 43.21875 35.40625 \n",
       "Q 43.21875 41.609375 39.140625 45 \n",
       "Q 35.0625 48.390625 27.6875 48.390625 \n",
       "Q 23 48.390625 18.546875 47.265625 \n",
       "Q 14.109375 46.140625 10.015625 43.890625 \n",
       "L 10.015625 52.203125 \n",
       "Q 14.9375 54.109375 19.578125 55.046875 \n",
       "Q 24.21875 56 28.609375 56 \n",
       "Q 40.484375 56 46.34375 49.84375 \n",
       "Q 52.203125 43.703125 52.203125 31.203125 \n",
       "z\n",
       "\" id=\"DejaVuSans-61\"/>\n",
       "      <path d=\"M 9.421875 54.6875 \n",
       "L 18.40625 54.6875 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 64.59375 \n",
       "L 9.421875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-6e\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(214.403125 27.298437)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"80.322266\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"141.601562\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"169.384766\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_46\">\n",
       "     <path d=\"M 186.403125 38.476562 \n",
       "L 206.403125 38.476562 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_47\"/>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test -->\n",
       "     <g transform=\"translate(214.403125 41.976562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"100.732422\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"152.832031\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p6dcb88f616\">\n",
       "   <rect height=\"135.9\" width=\"195.3\" x=\"51.378125\" y=\"10.7\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight: [[ 1.1958038]\n",
      " [-3.392395 ]\n",
      " [ 5.5950413]] \n",
      "bias: [4.9951715]\n"
     ]
    }
   ],
   "source": [
    "fit_and_plot(poly_features[:n_train, :], poly_features[n_train:, :],\n",
    "             labels[:n_train], labels[n_train:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. 线性函数拟合（欠拟合）\n",
    "\n",
    "我们再试试线性函数拟合。很明显，该模型的训练误差在迭代早期下降后便很难继续降低。在完成最后一次迭代周期后，训练误差依旧很高。线性模型在非线性模型（如三阶多项式函数）生成的数据集上容易欠拟合。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "final epoch: train loss 76.13087 test loss 196.67671\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
       "<svg height=\"184.15625pt\" version=\"1.1\" viewBox=\"0 0 268.478125 184.15625\" width=\"268.478125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 184.15625 \n",
       "L 268.478125 184.15625 \n",
       "L 268.478125 -0 \n",
       "L 0 -0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 62.478125 146.6 \n",
       "L 257.778125 146.6 \n",
       "L 257.778125 10.7 \n",
       "L 62.478125 10.7 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"mb1098c5391\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"69.562009\" xlink:href=\"#mb1098c5391\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "z\n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(66.380759 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"105.429778\" xlink:href=\"#mb1098c5391\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 20 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(99.067278 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"141.297546\" xlink:href=\"#mb1098c5391\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 40 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"DejaVuSans-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(134.935046 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"177.165315\" xlink:href=\"#mb1098c5391\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 60 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "z\n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(170.802815 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"213.033084\" xlink:href=\"#mb1098c5391\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 80 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "z\n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "z\n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "z\n",
       "\" id=\"DejaVuSans-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(206.670584 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"248.900852\" xlink:href=\"#mb1098c5391\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 100 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(239.357102 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- epochs -->\n",
       "     <defs>\n",
       "      <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "z\n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"DejaVuSans-65\"/>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "z\n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-70\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "z\n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-6f\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-63\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-68\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "z\n",
       "\" id=\"DejaVuSans-73\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(142.295313 174.876562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"61.523438\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"125\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"186.181641\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"241.162109\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"304.541016\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m0a870a9ade\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"62.478125\" xlink:href=\"#m0a870a9ade\" y=\"121.955083\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- $\\mathdefault{10^{2}}$ -->\n",
       "      <g transform=\"translate(37.878125 125.754302)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -2 0 \n",
       "\" id=\"m03db205e74\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"62.478125\" xlink:href=\"#m03db205e74\" y=\"146.095753\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"62.478125\" xlink:href=\"#m03db205e74\" y=\"137.058008\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"62.478125\" xlink:href=\"#m03db205e74\" y=\"129.086151\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"62.478125\" xlink:href=\"#m03db205e74\" y=\"75.041114\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- $\\mathdefault{2\\times10^{2}}$ -->\n",
       "      <defs>\n",
       "       <path d=\"M 70.125 53.71875 \n",
       "L 47.796875 31.296875 \n",
       "L 70.125 8.984375 \n",
       "L 64.3125 3.078125 \n",
       "L 41.890625 25.484375 \n",
       "L 19.484375 3.078125 \n",
       "L 13.71875 8.984375 \n",
       "L 35.984375 31.296875 \n",
       "L 13.71875 53.71875 \n",
       "L 19.484375 59.625 \n",
       "L 41.890625 37.203125 \n",
       "L 64.3125 59.625 \n",
       "z\n",
       "\" id=\"DejaVuSans-d7\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 78.840333)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use transform=\"translate(83.105469 0.765625)\" xlink:href=\"#DejaVuSans-d7\"/>\n",
       "       <use transform=\"translate(186.376953 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(250 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(314.580078 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"62.478125\" xlink:href=\"#m03db205e74\" y=\"47.598202\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- $\\mathdefault{3\\times10^{2}}$ -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "z\n",
       "\" id=\"DejaVuSans-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(20.878125 51.39742)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use transform=\"translate(83.105469 0.765625)\" xlink:href=\"#DejaVuSans-d7\"/>\n",
       "       <use transform=\"translate(186.376953 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(250 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(314.580078 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"62.478125\" xlink:href=\"#m03db205e74\" y=\"28.127145\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- $\\mathdefault{4\\times10^{2}}$ -->\n",
       "      <g transform=\"translate(20.878125 31.926364)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use transform=\"translate(83.105469 0.765625)\" xlink:href=\"#DejaVuSans-d7\"/>\n",
       "       <use transform=\"translate(186.376953 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(250 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(314.580078 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"62.478125\" xlink:href=\"#m03db205e74\" y=\"13.024221\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-6c\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798437 88.307812)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"27.783203\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"88.964844\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"141.064453\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_15\">\n",
       "    <path clip-path=\"url(#pc8c9f73c3c)\" d=\"M 71.355398 76.42675 \n",
       "L 73.148786 89.701681 \n",
       "L 74.942175 101.002289 \n",
       "L 76.735563 110.34433 \n",
       "L 78.528951 117.851188 \n",
       "L 80.32234 123.728419 \n",
       "L 82.115728 128.226258 \n",
       "L 83.909117 131.60337 \n",
       "L 85.702505 134.099869 \n",
       "L 87.495894 135.922654 \n",
       "L 89.289282 137.240496 \n",
       "L 91.08267 138.185831 \n",
       "L 92.876059 138.859588 \n",
       "L 94.669447 139.337153 \n",
       "L 96.462836 139.673947 \n",
       "L 98.256224 139.910304 \n",
       "L 100.049613 140.075325 \n",
       "L 101.843001 140.189877 \n",
       "L 103.636389 140.268858 \n",
       "L 105.429778 140.322877 \n",
       "L 107.223166 140.359445 \n",
       "L 109.016555 140.383847 \n",
       "L 110.809943 140.399856 \n",
       "L 112.603332 140.410097 \n",
       "L 114.39672 140.416405 \n",
       "L 116.190108 140.420055 \n",
       "L 117.983497 140.421947 \n",
       "L 119.776885 140.4227 \n",
       "L 121.570274 140.422727 \n",
       "L 123.363662 140.422327 \n",
       "L 125.157051 140.421676 \n",
       "L 126.950439 140.420909 \n",
       "L 128.743827 140.420109 \n",
       "L 130.537216 140.419315 \n",
       "L 132.330604 140.418569 \n",
       "L 134.123993 140.417877 \n",
       "L 135.917381 140.417253 \n",
       "L 137.71077 140.416704 \n",
       "L 139.504158 140.416215 \n",
       "L 141.297546 140.415781 \n",
       "L 143.090935 140.415415 \n",
       "L 144.884323 140.415096 \n",
       "L 146.677712 140.414818 \n",
       "L 148.4711 140.414574 \n",
       "L 150.264489 140.414377 \n",
       "L 152.057877 140.414201 \n",
       "L 153.851265 140.414045 \n",
       "L 155.644654 140.413923 \n",
       "L 157.438042 140.413821 \n",
       "L 159.231431 140.413733 \n",
       "L 161.024819 140.413658 \n",
       "L 162.818208 140.413583 \n",
       "L 164.611596 140.413536 \n",
       "L 166.404985 140.413482 \n",
       "L 168.198373 140.413448 \n",
       "L 169.991761 140.413414 \n",
       "L 171.78515 140.413387 \n",
       "L 173.578538 140.41336 \n",
       "L 175.371927 140.413332 \n",
       "L 177.165315 140.413319 \n",
       "L 178.958704 140.413312 \n",
       "L 180.752092 140.413292 \n",
       "L 182.54548 140.413285 \n",
       "L 184.338869 140.413271 \n",
       "L 186.132257 140.413265 \n",
       "L 187.925646 140.413265 \n",
       "L 189.719034 140.413258 \n",
       "L 191.512423 140.413251 \n",
       "L 193.305811 140.413251 \n",
       "L 195.099199 140.413251 \n",
       "L 196.892588 140.413238 \n",
       "L 198.685976 140.413238 \n",
       "L 200.479365 140.413238 \n",
       "L 202.272753 140.413231 \n",
       "L 204.066142 140.413238 \n",
       "L 205.85953 140.413231 \n",
       "L 207.652918 140.413231 \n",
       "L 209.446307 140.413231 \n",
       "L 211.239695 140.413231 \n",
       "L 213.033084 140.413231 \n",
       "L 214.826472 140.413224 \n",
       "L 216.619861 140.413231 \n",
       "L 218.413249 140.413231 \n",
       "L 220.206637 140.413231 \n",
       "L 222.000026 140.413231 \n",
       "L 223.793414 140.413231 \n",
       "L 225.586803 140.413231 \n",
       "L 227.380191 140.413231 \n",
       "L 229.17358 140.413231 \n",
       "L 230.966968 140.413231 \n",
       "L 232.760356 140.413224 \n",
       "L 234.553745 140.413224 \n",
       "L 236.347133 140.413231 \n",
       "L 238.140522 140.413224 \n",
       "L 239.93391 140.413224 \n",
       "L 241.727299 140.413224 \n",
       "L 243.520687 140.413224 \n",
       "L 245.314075 140.413224 \n",
       "L 247.107464 140.413224 \n",
       "L 248.900852 140.413224 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_16\">\n",
       "    <path clip-path=\"url(#pc8c9f73c3c)\" d=\"M 71.355398 16.877273 \n",
       "L 73.148786 26.383214 \n",
       "L 74.942175 34.572974 \n",
       "L 76.735563 41.545833 \n",
       "L 78.528951 47.423802 \n",
       "L 80.32234 52.339644 \n",
       "L 82.115728 56.426534 \n",
       "L 83.909117 59.810426 \n",
       "L 85.702505 62.605311 \n",
       "L 87.495894 64.91109 \n",
       "L 89.289282 66.813106 \n",
       "L 91.08267 68.383101 \n",
       "L 92.876059 69.680569 \n",
       "L 94.669447 70.75442 \n",
       "L 96.462836 71.644701 \n",
       "L 98.256224 72.384108 \n",
       "L 100.049613 72.999277 \n",
       "L 101.843001 73.511935 \n",
       "L 103.636389 73.939825 \n",
       "L 105.429778 74.297485 \n",
       "L 107.223166 74.596824 \n",
       "L 109.016555 74.847641 \n",
       "L 110.809943 75.058012 \n",
       "L 112.603332 75.234619 \n",
       "L 114.39672 75.382991 \n",
       "L 116.190108 75.507753 \n",
       "L 117.983497 75.612702 \n",
       "L 119.776885 75.701028 \n",
       "L 121.570274 75.775395 \n",
       "L 123.363662 75.838044 \n",
       "L 125.157051 75.890826 \n",
       "L 126.950439 75.935313 \n",
       "L 128.743827 75.972824 \n",
       "L 130.537216 76.004443 \n",
       "L 132.330604 76.031119 \n",
       "L 134.123993 76.053617 \n",
       "L 135.917381 76.072605 \n",
       "L 137.71077 76.088614 \n",
       "L 139.504158 76.102119 \n",
       "L 141.297546 76.113519 \n",
       "L 143.090935 76.123151 \n",
       "L 144.884323 76.131258 \n",
       "L 146.677712 76.138112 \n",
       "L 148.4711 76.143896 \n",
       "L 150.264489 76.148772 \n",
       "L 152.057877 76.152887 \n",
       "L 153.851265 76.156378 \n",
       "L 155.644654 76.159308 \n",
       "L 157.438042 76.16178 \n",
       "L 159.231431 76.16387 \n",
       "L 161.024819 76.165639 \n",
       "L 162.818208 76.167141 \n",
       "L 164.611596 76.16838 \n",
       "L 166.404985 76.169451 \n",
       "L 168.198373 76.170338 \n",
       "L 169.991761 76.171089 \n",
       "L 171.78515 76.17173 \n",
       "L 173.578538 76.172281 \n",
       "L 175.371927 76.172738 \n",
       "L 177.165315 76.173111 \n",
       "L 178.958704 76.173436 \n",
       "L 180.752092 76.173704 \n",
       "L 182.54548 76.17394 \n",
       "L 184.338869 76.174135 \n",
       "L 186.132257 76.174303 \n",
       "L 187.925646 76.174444 \n",
       "L 189.719034 76.17456 \n",
       "L 191.512423 76.174665 \n",
       "L 193.305811 76.174749 \n",
       "L 195.099199 76.174822 \n",
       "L 196.892588 76.17488 \n",
       "L 198.685976 76.174933 \n",
       "L 200.479365 76.174975 \n",
       "L 202.272753 76.175006 \n",
       "L 204.066142 76.175043 \n",
       "L 205.85953 76.175069 \n",
       "L 207.652918 76.175101 \n",
       "L 209.446307 76.175101 \n",
       "L 211.239695 76.175116 \n",
       "L 213.033084 76.175127 \n",
       "L 214.826472 76.175137 \n",
       "L 216.619861 76.175143 \n",
       "L 218.413249 76.175148 \n",
       "L 220.206637 76.175148 \n",
       "L 222.000026 76.175158 \n",
       "L 223.793414 76.175164 \n",
       "L 225.586803 76.175169 \n",
       "L 227.380191 76.175174 \n",
       "L 229.17358 76.175174 \n",
       "L 230.966968 76.175174 \n",
       "L 232.760356 76.175174 \n",
       "L 234.553745 76.17519 \n",
       "L 236.347133 76.175206 \n",
       "L 238.140522 76.175206 \n",
       "L 239.93391 76.175206 \n",
       "L 241.727299 76.175206 \n",
       "L 243.520687 76.175206 \n",
       "L 245.314075 76.175206 \n",
       "L 247.107464 76.175206 \n",
       "L 248.900852 76.175206 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 62.478125 146.6 \n",
       "L 62.478125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 257.778125 146.6 \n",
       "L 257.778125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 62.478125 146.6 \n",
       "L 257.778125 146.6 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 62.478125 10.7 \n",
       "L 257.778125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 195.503125 48.05625 \n",
       "L 250.778125 48.05625 \n",
       "Q 252.778125 48.05625 252.778125 46.05625 \n",
       "L 252.778125 17.7 \n",
       "Q 252.778125 15.7 250.778125 15.7 \n",
       "L 195.503125 15.7 \n",
       "Q 193.503125 15.7 193.503125 17.7 \n",
       "L 193.503125 46.05625 \n",
       "Q 193.503125 48.05625 195.503125 48.05625 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_17\">\n",
       "     <path d=\"M 197.503125 23.798437 \n",
       "L 217.503125 23.798437 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_18\"/>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train -->\n",
       "     <defs>\n",
       "      <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-74\"/>\n",
       "      <path d=\"M 41.109375 46.296875 \n",
       "Q 39.59375 47.171875 37.8125 47.578125 \n",
       "Q 36.03125 48 33.890625 48 \n",
       "Q 26.265625 48 22.1875 43.046875 \n",
       "Q 18.109375 38.09375 18.109375 28.8125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 20.953125 51.171875 25.484375 53.578125 \n",
       "Q 30.03125 56 36.53125 56 \n",
       "Q 37.453125 56 38.578125 55.875 \n",
       "Q 39.703125 55.765625 41.0625 55.515625 \n",
       "z\n",
       "\" id=\"DejaVuSans-72\"/>\n",
       "      <path d=\"M 34.28125 27.484375 \n",
       "Q 23.390625 27.484375 19.1875 25 \n",
       "Q 14.984375 22.515625 14.984375 16.5 \n",
       "Q 14.984375 11.71875 18.140625 8.90625 \n",
       "Q 21.296875 6.109375 26.703125 6.109375 \n",
       "Q 34.1875 6.109375 38.703125 11.40625 \n",
       "Q 43.21875 16.703125 43.21875 25.484375 \n",
       "L 43.21875 27.484375 \n",
       "z\n",
       "M 52.203125 31.203125 \n",
       "L 52.203125 0 \n",
       "L 43.21875 0 \n",
       "L 43.21875 8.296875 \n",
       "Q 40.140625 3.328125 35.546875 0.953125 \n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \n",
       "Q 6 8.015625 6 15.921875 \n",
       "Q 6 25.140625 12.171875 29.828125 \n",
       "Q 18.359375 34.515625 30.609375 34.515625 \n",
       "L 43.21875 34.515625 \n",
       "L 43.21875 35.40625 \n",
       "Q 43.21875 41.609375 39.140625 45 \n",
       "Q 35.0625 48.390625 27.6875 48.390625 \n",
       "Q 23 48.390625 18.546875 47.265625 \n",
       "Q 14.109375 46.140625 10.015625 43.890625 \n",
       "L 10.015625 52.203125 \n",
       "Q 14.9375 54.109375 19.578125 55.046875 \n",
       "Q 24.21875 56 28.609375 56 \n",
       "Q 40.484375 56 46.34375 49.84375 \n",
       "Q 52.203125 43.703125 52.203125 31.203125 \n",
       "z\n",
       "\" id=\"DejaVuSans-61\"/>\n",
       "      <path d=\"M 9.421875 54.6875 \n",
       "L 18.40625 54.6875 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 64.59375 \n",
       "L 9.421875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-6e\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(225.503125 27.298437)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"80.322266\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"141.601562\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"169.384766\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_19\">\n",
       "     <path d=\"M 197.503125 38.476562 \n",
       "L 217.503125 38.476562 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_20\"/>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test -->\n",
       "     <g transform=\"translate(225.503125 41.976562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"100.732422\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"152.832031\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pc8c9f73c3c\">\n",
       "   <rect height=\"135.9\" width=\"195.3\" x=\"62.478125\" y=\"10.7\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight: [[12.646394]] \n",
      "bias: [2.226844]\n"
     ]
    }
   ],
   "source": [
    "fit_and_plot(features[:n_train, :], features[n_train:, :], labels[:n_train],\n",
    "             labels[n_train:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. 训练样本不足（过拟合）\n",
    "\n",
    "事实上，即便使用与数据生成模型同阶的三阶多项式函数模型，如果训练样本不足，该模型依然容易过拟合。让我们只使用两个样本来训练模型。显然，训练样本过少了，甚至少于模型参数的数量。这使模型显得过于复杂，以至于容易被训练数据中的噪声影响。在迭代过程中，尽管训练误差较低，但是测试数据集上的误差却很高。这是典型的过拟合现象。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "final epoch: train loss 1.9505903 test loss 11.535108\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
       "<svg height=\"184.15625pt\" version=\"1.1\" viewBox=\"0 0 251.478125 184.15625\" width=\"251.478125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 184.15625 \n",
       "L 251.478125 184.15625 \n",
       "L 251.478125 -0 \n",
       "L 0 -0 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 45.478125 146.6 \n",
       "L 240.778125 146.6 \n",
       "L 240.778125 10.7 \n",
       "L 45.478125 10.7 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" id=\"m9561d05f29\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.562009\" xlink:href=\"#m9561d05f29\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "z\n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(49.380759 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"88.429778\" xlink:href=\"#m9561d05f29\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 20 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(82.067278 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"124.297546\" xlink:href=\"#m9561d05f29\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 40 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"DejaVuSans-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(117.935046 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"160.165315\" xlink:href=\"#m9561d05f29\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 60 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "z\n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(153.802815 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"196.033084\" xlink:href=\"#m9561d05f29\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 80 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "z\n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "z\n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "z\n",
       "\" id=\"DejaVuSans-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(189.670584 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"231.900852\" xlink:href=\"#m9561d05f29\" y=\"146.6\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 100 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(222.357102 161.198437)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- epochs -->\n",
       "     <defs>\n",
       "      <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "z\n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"DejaVuSans-65\"/>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "z\n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "z\n",
       "\" id=\"DejaVuSans-70\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "z\n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "z\n",
       "\" id=\"DejaVuSans-6f\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-63\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-68\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "z\n",
       "\" id=\"DejaVuSans-73\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(125.295312 174.876562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"61.523438\" xlink:href=\"#DejaVuSans-70\"/>\n",
       "      <use x=\"125\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"186.181641\" xlink:href=\"#DejaVuSans-63\"/>\n",
       "      <use x=\"241.162109\" xlink:href=\"#DejaVuSans-68\"/>\n",
       "      <use x=\"304.541016\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" id=\"m5368d1279a\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#m5368d1279a\" y=\"97.614359\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- $\\mathdefault{10^{1}}$ -->\n",
       "      <g transform=\"translate(20.878125 101.413578)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.684375)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(63.623047 0.684375)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(128.203125 38.965625)scale(0.7)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#m5368d1279a\" y=\"37.306778\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- $\\mathdefault{10^{2}}$ -->\n",
       "      <g transform=\"translate(20.878125 41.105997)scale(0.1 -0.1)\">\n",
       "       <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -2 0 \n",
       "\" id=\"m2ffbcf2fd5\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"139.767549\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"129.147911\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"121.613158\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"115.76875\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"110.99352\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_8\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"106.956121\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_9\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"103.458767\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_10\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"100.373882\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_11\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"79.459968\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_12\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"68.84033\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_13\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"61.305577\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_14\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"55.461169\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_15\">\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"50.685939\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_16\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"46.64854\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_17\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"43.151186\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_18\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"40.066301\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_19\">\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m2ffbcf2fd5\" y=\"19.152387\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"DejaVuSans-6c\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798437 88.307812)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-6c\"/>\n",
       "      <use x=\"27.783203\" xlink:href=\"#DejaVuSans-6f\"/>\n",
       "      <use x=\"88.964844\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"141.064453\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_26\">\n",
       "    <path clip-path=\"url(#pb68f7c202a)\" d=\"M 54.355398 18.966328 \n",
       "L 56.148786 46.714834 \n",
       "L 57.942175 67.988504 \n",
       "L 59.735563 80.283135 \n",
       "L 61.528951 85.698735 \n",
       "L 63.32234 87.902276 \n",
       "L 65.115728 88.969293 \n",
       "L 66.909117 89.676785 \n",
       "L 68.702505 90.274759 \n",
       "L 70.495894 90.839743 \n",
       "L 72.289282 91.394841 \n",
       "L 74.08267 91.946961 \n",
       "L 75.876059 92.498197 \n",
       "L 77.669447 93.049164 \n",
       "L 79.462836 93.600053 \n",
       "L 81.256224 94.15092 \n",
       "L 83.049613 94.701774 \n",
       "L 84.843001 95.252629 \n",
       "L 86.636389 95.803486 \n",
       "L 88.429778 96.354348 \n",
       "L 90.223166 96.905197 \n",
       "L 92.016555 97.456051 \n",
       "L 93.809943 98.006907 \n",
       "L 95.603332 98.557763 \n",
       "L 97.39672 99.108613 \n",
       "L 99.190108 99.659469 \n",
       "L 100.983497 100.210325 \n",
       "L 102.776885 100.761178 \n",
       "L 104.570274 101.312032 \n",
       "L 106.363662 101.862891 \n",
       "L 108.157051 102.413747 \n",
       "L 109.950439 102.964606 \n",
       "L 111.743827 103.515455 \n",
       "L 113.537216 104.066312 \n",
       "L 115.330604 104.617162 \n",
       "L 117.123993 105.16802 \n",
       "L 118.917381 105.718878 \n",
       "L 120.71077 106.269727 \n",
       "L 122.504158 106.820582 \n",
       "L 124.297546 107.371436 \n",
       "L 126.090935 107.922292 \n",
       "L 127.884323 108.473145 \n",
       "L 129.677712 109.024001 \n",
       "L 131.4711 109.574856 \n",
       "L 133.264489 110.12571 \n",
       "L 135.057877 110.676565 \n",
       "L 136.851265 111.227422 \n",
       "L 138.644654 111.778273 \n",
       "L 140.438042 112.329129 \n",
       "L 142.231431 112.87998 \n",
       "L 144.024819 113.430836 \n",
       "L 145.818208 113.981691 \n",
       "L 147.611596 114.532544 \n",
       "L 149.404985 115.083401 \n",
       "L 151.198373 115.63426 \n",
       "L 152.991761 116.185112 \n",
       "L 154.78515 116.735964 \n",
       "L 156.578538 117.286825 \n",
       "L 158.371927 117.837677 \n",
       "L 160.165315 118.388534 \n",
       "L 161.958704 118.939392 \n",
       "L 163.752092 119.490246 \n",
       "L 165.54548 120.041103 \n",
       "L 167.338869 120.591958 \n",
       "L 169.132257 121.142814 \n",
       "L 170.925646 121.693665 \n",
       "L 172.719034 122.244523 \n",
       "L 174.512423 122.795374 \n",
       "L 176.305811 123.346238 \n",
       "L 178.099199 123.897088 \n",
       "L 179.892588 124.447939 \n",
       "L 181.685976 124.998798 \n",
       "L 183.479365 125.549653 \n",
       "L 185.272753 126.100503 \n",
       "L 187.066142 126.651361 \n",
       "L 188.85953 127.202214 \n",
       "L 190.652918 127.753071 \n",
       "L 192.446307 128.303923 \n",
       "L 194.239695 128.854777 \n",
       "L 196.033084 129.405636 \n",
       "L 197.826472 129.95649 \n",
       "L 199.619861 130.507343 \n",
       "L 201.413249 131.058196 \n",
       "L 203.206637 131.609051 \n",
       "L 205.000026 132.159909 \n",
       "L 206.793414 132.710763 \n",
       "L 208.586803 133.26162 \n",
       "L 210.380191 133.812468 \n",
       "L 212.17358 134.363326 \n",
       "L 213.966968 134.914181 \n",
       "L 215.760356 135.465035 \n",
       "L 217.553745 136.015888 \n",
       "L 219.347133 136.566754 \n",
       "L 221.140522 137.117599 \n",
       "L 222.93391 137.668461 \n",
       "L 224.727299 138.219306 \n",
       "L 226.520687 138.770169 \n",
       "L 228.314075 139.321019 \n",
       "L 230.107464 139.87188 \n",
       "L 231.900852 140.422727 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_27\">\n",
       "    <path clip-path=\"url(#pb68f7c202a)\" d=\"M 54.355398 16.877273 \n",
       "L 56.148786 32.549062 \n",
       "L 57.942175 42.242216 \n",
       "L 59.735563 47.723526 \n",
       "L 61.528951 50.802935 \n",
       "L 63.32234 52.632009 \n",
       "L 65.115728 53.82041 \n",
       "L 66.909117 54.676088 \n",
       "L 68.702505 55.355774 \n",
       "L 70.495894 55.940918 \n",
       "L 72.289282 56.474654 \n",
       "L 74.08267 56.980089 \n",
       "L 75.876059 57.469713 \n",
       "L 77.669447 57.950304 \n",
       "L 79.462836 58.42554 \n",
       "L 81.256224 58.897439 \n",
       "L 83.049613 59.367069 \n",
       "L 84.843001 59.83505 \n",
       "L 86.636389 60.301665 \n",
       "L 88.429778 60.767102 \n",
       "L 90.223166 61.231435 \n",
       "L 92.016555 61.694727 \n",
       "L 93.809943 62.156976 \n",
       "L 95.603332 62.618208 \n",
       "L 97.39672 63.0784 \n",
       "L 99.190108 63.537554 \n",
       "L 100.983497 63.995671 \n",
       "L 102.776885 64.452734 \n",
       "L 104.570274 64.908731 \n",
       "L 106.363662 65.363655 \n",
       "L 108.157051 65.817505 \n",
       "L 109.950439 66.270252 \n",
       "L 111.743827 66.721885 \n",
       "L 113.537216 67.172405 \n",
       "L 115.330604 67.621798 \n",
       "L 117.123993 68.070064 \n",
       "L 118.917381 68.517174 \n",
       "L 120.71077 68.963123 \n",
       "L 122.504158 69.407903 \n",
       "L 124.297546 69.8515 \n",
       "L 126.090935 70.293909 \n",
       "L 127.884323 70.735107 \n",
       "L 129.677712 71.175092 \n",
       "L 131.4711 71.613841 \n",
       "L 133.264489 72.051361 \n",
       "L 135.057877 72.487617 \n",
       "L 136.851265 72.922624 \n",
       "L 138.644654 73.356351 \n",
       "L 140.438042 73.788803 \n",
       "L 142.231431 74.219948 \n",
       "L 144.024819 74.649793 \n",
       "L 145.818208 75.078306 \n",
       "L 147.611596 75.505497 \n",
       "L 149.404985 75.931352 \n",
       "L 151.198373 76.355843 \n",
       "L 152.991761 76.778974 \n",
       "L 154.78515 77.200732 \n",
       "L 156.578538 77.621111 \n",
       "L 158.371927 78.040066 \n",
       "L 160.165315 78.457623 \n",
       "L 161.958704 78.873747 \n",
       "L 163.752092 79.28845 \n",
       "L 165.54548 79.7017 \n",
       "L 167.338869 80.113499 \n",
       "L 169.132257 80.523805 \n",
       "L 170.925646 80.932645 \n",
       "L 172.719034 81.339999 \n",
       "L 174.512423 81.74585 \n",
       "L 176.305811 82.150189 \n",
       "L 178.099199 82.552996 \n",
       "L 179.892588 82.954262 \n",
       "L 181.685976 83.35398 \n",
       "L 183.479365 83.752135 \n",
       "L 185.272753 84.148721 \n",
       "L 187.066142 84.54373 \n",
       "L 188.85953 84.937149 \n",
       "L 190.652918 85.328954 \n",
       "L 192.446307 85.719145 \n",
       "L 194.239695 86.107716 \n",
       "L 196.033084 86.494643 \n",
       "L 197.826472 86.879938 \n",
       "L 199.619861 87.263571 \n",
       "L 201.413249 87.645524 \n",
       "L 203.206637 88.025808 \n",
       "L 205.000026 88.404404 \n",
       "L 206.793414 88.781289 \n",
       "L 208.586803 89.156482 \n",
       "L 210.380191 89.529948 \n",
       "L 212.17358 89.901681 \n",
       "L 213.966968 90.271684 \n",
       "L 215.760356 90.639937 \n",
       "L 217.553745 91.006433 \n",
       "L 219.347133 91.371163 \n",
       "L 221.140522 91.734109 \n",
       "L 222.93391 92.095271 \n",
       "L 224.727299 92.454629 \n",
       "L 226.520687 92.812205 \n",
       "L 228.314075 93.167963 \n",
       "L 230.107464 93.521889 \n",
       "L 231.900852 93.873983 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 45.478125 146.6 \n",
       "L 45.478125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 240.778125 146.6 \n",
       "L 240.778125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 45.478125 146.6 \n",
       "L 240.778125 146.6 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 45.478125 10.7 \n",
       "L 240.778125 10.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 178.503125 48.05625 \n",
       "L 233.778125 48.05625 \n",
       "Q 235.778125 48.05625 235.778125 46.05625 \n",
       "L 235.778125 17.7 \n",
       "Q 235.778125 15.7 233.778125 15.7 \n",
       "L 178.503125 15.7 \n",
       "Q 176.503125 15.7 176.503125 17.7 \n",
       "L 176.503125 46.05625 \n",
       "Q 176.503125 48.05625 178.503125 48.05625 \n",
       "z\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_28\">\n",
       "     <path d=\"M 180.503125 23.798437 \n",
       "L 200.503125 23.798437 \n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_29\"/>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- train -->\n",
       "     <defs>\n",
       "      <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"DejaVuSans-74\"/>\n",
       "      <path d=\"M 41.109375 46.296875 \n",
       "Q 39.59375 47.171875 37.8125 47.578125 \n",
       "Q 36.03125 48 33.890625 48 \n",
       "Q 26.265625 48 22.1875 43.046875 \n",
       "Q 18.109375 38.09375 18.109375 28.8125 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 20.953125 51.171875 25.484375 53.578125 \n",
       "Q 30.03125 56 36.53125 56 \n",
       "Q 37.453125 56 38.578125 55.875 \n",
       "Q 39.703125 55.765625 41.0625 55.515625 \n",
       "z\n",
       "\" id=\"DejaVuSans-72\"/>\n",
       "      <path d=\"M 34.28125 27.484375 \n",
       "Q 23.390625 27.484375 19.1875 25 \n",
       "Q 14.984375 22.515625 14.984375 16.5 \n",
       "Q 14.984375 11.71875 18.140625 8.90625 \n",
       "Q 21.296875 6.109375 26.703125 6.109375 \n",
       "Q 34.1875 6.109375 38.703125 11.40625 \n",
       "Q 43.21875 16.703125 43.21875 25.484375 \n",
       "L 43.21875 27.484375 \n",
       "z\n",
       "M 52.203125 31.203125 \n",
       "L 52.203125 0 \n",
       "L 43.21875 0 \n",
       "L 43.21875 8.296875 \n",
       "Q 40.140625 3.328125 35.546875 0.953125 \n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \n",
       "Q 6 8.015625 6 15.921875 \n",
       "Q 6 25.140625 12.171875 29.828125 \n",
       "Q 18.359375 34.515625 30.609375 34.515625 \n",
       "L 43.21875 34.515625 \n",
       "L 43.21875 35.40625 \n",
       "Q 43.21875 41.609375 39.140625 45 \n",
       "Q 35.0625 48.390625 27.6875 48.390625 \n",
       "Q 23 48.390625 18.546875 47.265625 \n",
       "Q 14.109375 46.140625 10.015625 43.890625 \n",
       "L 10.015625 52.203125 \n",
       "Q 14.9375 54.109375 19.578125 55.046875 \n",
       "Q 24.21875 56 28.609375 56 \n",
       "Q 40.484375 56 46.34375 49.84375 \n",
       "Q 52.203125 43.703125 52.203125 31.203125 \n",
       "z\n",
       "\" id=\"DejaVuSans-61\"/>\n",
       "      <path d=\"M 9.421875 54.6875 \n",
       "L 18.40625 54.6875 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 64.59375 \n",
       "L 9.421875 64.59375 \n",
       "z\n",
       "\" id=\"DejaVuSans-69\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "z\n",
       "\" id=\"DejaVuSans-6e\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(208.503125 27.298437)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use x=\"80.322266\" xlink:href=\"#DejaVuSans-61\"/>\n",
       "      <use x=\"141.601562\" xlink:href=\"#DejaVuSans-69\"/>\n",
       "      <use x=\"169.384766\" xlink:href=\"#DejaVuSans-6e\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_30\">\n",
       "     <path d=\"M 180.503125 38.476562 \n",
       "L 200.503125 38.476562 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_31\"/>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- test -->\n",
       "     <g transform=\"translate(208.503125 41.976562)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use x=\"100.732422\" xlink:href=\"#DejaVuSans-73\"/>\n",
       "      <use x=\"152.832031\" xlink:href=\"#DejaVuSans-74\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pb68f7c202a\">\n",
       "   <rect height=\"135.9\" width=\"195.3\" x=\"45.478125\" y=\"10.7\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight: [[-0.31323293]\n",
      " [-3.6265852 ]\n",
      " [ 5.4798284 ]] \n",
      "bias: [2.4761918]\n"
     ]
    }
   ],
   "source": [
    "fit_and_plot(poly_features[0:2, :], poly_features[n_train:, :], labels[0:2],\n",
    "             labels[n_train:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实验小结\n",
    "\n",
    "在本实验中，你在一个多项式函数生成的人工数据集中，分别采用三阶多项式函数拟合、线性函数拟合、以及降低训练样本的方式，展示了当模式处于正常、欠拟合与过拟合状态下的训练曲线形态。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}