RLOSF Python Coding Exercise.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Importing the necessary packages**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from vowpalwabbit import pyvw\n",
    "import random\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Defining the reward values**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "USER_LIKED_ARTICLE = -1.0\n",
    "USER_DISLIKED_ARTICLE = 0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Introducing varying noise in the reward:**\n",
    "Noise is the stochastic choice of USER_LIKED_ARTICLE with a small probability of getting the opposite reward instead of choosing it deterministically. Hence with probability of epsilon the opposite reward is chosen and probability of (1-epsilon) the expected reward.\\\n",
    "\n",
    "**Why does noise make sense in our context?**\n",
    "Noise makes sense in the this context because even if the user likes sports in the morning generally, sometimes he might dislike it, so this noise helps to model that behaviour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "#With probability of (1-eps) USER_LIKED_ARTICLE chosen\n",
    "def positivereward(p,eps):\n",
    "    if p<eps:\n",
    "        return USER_DISLIKED_ARTICLE\n",
    "    else:\n",
    "        return USER_LIKED_ARTICLE\n",
    "    \n",
    "#With probability of (1-eps) USER_DISLIKED_ARTICLE chosen    \n",
    "def negativereward(p,eps):\n",
    "    if p<eps:\n",
    "        return USER_LIKED_ARTICLE\n",
    "    else:\n",
    "        return USER_DISLIKED_ARTICLE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**To incorporate mutiple changes in the reward distribution over time, multiple cost functions are defined with changing preferences:**\n",
    "For example, Tom might be keen on watching politics in the morning if there is an election taking place however would be keen on watching sports in the morning if the Olympic Games are being held."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_cost_elections(context,action,eps): # Election being held hence politics preferred by both\n",
    "    if context['user'] == \"Tom\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'politics':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'music':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "    elif context['user'] == \"Anna\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'sports':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'politics':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "            \n",
    "def get_cost_olympics(context,action, eps): # Olympics being held hence sports preferred by both\n",
    "    if context['user'] == \"Tom\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'politics':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'sports':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "    elif context['user'] == \"Anna\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'sports':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'sports':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "        \n",
    "def get_cost_grammy(context,action, eps): # Grammy Awards being held hence music preferred by both\n",
    "    if context['user'] == \"Tom\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'music':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'sports':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "    elif context['user'] == \"Anna\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'music':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'food':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "        \n",
    "def get_cost_world_health_day(context,action, eps): # World Health Day hence health preferred by both\n",
    "    if context['user'] == \"Tom\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'music':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'health':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "    elif context['user'] == \"Anna\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'health':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'food':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "        \n",
    "def get_cost_budget(context,action, eps): # Budget being disclosed hence finance preferred by both\n",
    "    if context['user'] == \"Tom\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'camping':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'finance':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n",
    "    elif context['user'] == \"Anna\":\n",
    "        p=np.random.random()\n",
    "        if context['time_of_day'] == \"morning\" and action == 'finance':\n",
    "            return positivereward(p,eps)\n",
    "        elif context['time_of_day'] == \"afternoon\" and action == 'finance':\n",
    "            return positivereward(p,eps)\n",
    "        else:\n",
    "            return negativereward(p,eps)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Function to modify (context, action, cost, probability) to VW friendly format**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "def to_vw_example_format(context, actions, cb_label = None):\n",
    "    if cb_label is not None:\n",
    "        chosen_action, cost, prob = cb_label\n",
    "    example_string = \"\"\n",
    "    example_string += \"shared |User user={} time_of_day={}\\n\".format(context[\"user\"], context[\"time_of_day\"])\n",
    "    for action in actions:\n",
    "        if cb_label is not None and action == chosen_action:\n",
    "            example_string += \"0:{}:{} \".format(cost, prob)\n",
    "        example_string += \"|Action article={} \\n\".format(action)\n",
    "    return example_string[:-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Function to sample the pmf to get the action after calculating the pmf values for the list of actions**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sample_custom_pmf(pmf):\n",
    "    total = sum(pmf)\n",
    "    scale = 1 / total\n",
    "    pmf = [x * scale for x in pmf]\n",
    "    draw = random.random()\n",
    "    sum_prob = 0.0\n",
    "    for index, prob in enumerate(pmf):\n",
    "        sum_prob += prob\n",
    "        if(sum_prob > draw):\n",
    "            return index, prob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Function to return the action and probability of choosing it**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_action(vw, context, actions):\n",
    "    vw_text_example = to_vw_example_format(context,actions)\n",
    "    pmf = vw.predict(vw_text_example)\n",
    "    chosen_action_index, prob = sample_custom_pmf(pmf)\n",
    "    return actions[chosen_action_index], prob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Defining users, actions, times of day and functions to randomly make a user and time of day choice**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "users = ['Tom', 'Anna']\n",
    "times_of_day = ['morning', 'afternoon']\n",
    "actions = [\"politics\", \"sports\", \"music\", \"food\", \"finance\", \"health\", \"camping\"]\n",
    "def choose_user(users):\n",
    "    return random.choice(users)\n",
    "\n",
    "def choose_time_of_day(times_of_day):\n",
    "    return random.choice(times_of_day)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Function for plotting and incorporating the non-stationarity with a changing reward distribution over time**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_ctr(num_iterations, ctr):\n",
    "    plt.plot(range(1,num_iterations+1), ctr)\n",
    "    plt.xlabel('num_iterations', fontsize=14)\n",
    "    plt.ylabel('ctr', fontsize=14)\n",
    "    plt.ylim([0,1])\n",
    "\n",
    "def run_simulation_multiple_cost_functions(vw, num_iterations, users, times_of_day, actions, cost_functions, epsilon, do_learn = True):\n",
    "    cost_sum = 0.\n",
    "    ctr = []\n",
    "    start_counter = 1\n",
    "    end_counter = start_counter + num_iterations\n",
    "    for cost_function in cost_functions:\n",
    "        for i in range(start_counter, end_counter):\n",
    "            user = choose_user(users)\n",
    "            time_of_day = choose_time_of_day(times_of_day)\n",
    "            context = {'user': user, 'time_of_day': time_of_day}\n",
    "            action, prob = get_action(vw, context, actions)\n",
    "            cost = cost_function(context, action, epsilon)\n",
    "            cost_sum += cost\n",
    "            if do_learn:\n",
    "                vw_format = vw.parse(to_vw_example_format(context, actions, (action, cost, prob)),pyvw.vw.lContextualBandit)\n",
    "                vw.learn(vw_format)\n",
    "            ctr.append(-1*cost_sum/i)\n",
    "        start_counter = end_counter\n",
    "        end_counter = start_counter + num_iterations\n",
    "\n",
    "    return ctr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Defining the cost function and epsilon parameter for the noise. Further to increase the stochasticity the different preference based cost functions are chosen randomly**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_iterations_per_cost_func = 5000\n",
    "epsilon=0.15\n",
    "cost_functions = [get_cost_olympics, get_cost_world_health_day, get_cost_grammy, get_cost_budget,get_cost_elections]\n",
    "cost_functions= random.choices(cost_functions,k=4)\n",
    "total_iterations = num_iterations_per_cost_func * len(cost_functions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using the cb_explore_adf contextual bandit algorithm and exploring with the epsilon-greedy algorithm with epsilon value = 0.2 and plotting ctr vs num_iterations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhcdZ3v8fe3u7q7eu8sna0JJEAwBkWWiAzIMoPDquI2CjpXRb1cRvHBO+MdYbguI4zgjLggYIaLXNxx7oiKGgXngjIICAECJIFgSELSZOvsvVZ3dX3nj3M6qdNdnXQnVac6OZ/X89TTddb65nTnfOr8zjm/Y+6OiIjIkIpyFyAiIhOLgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCQitmAws7vMbIuZLRtlupnZLWa2ysyeM7OT46pNRET2ivOI4W7ggn1MvxCYF76uAL4VQ00iIjJMbMHg7g8D2/cxyyXAdz3wONBiZjPjqU5ERIakyl1AnjZgfd5wezhu4/AZzewKgqMK6uvrT5k/f34sBYqIHC6eeuqpre7eWmjaRAoGKzCuYH8d7n4HcAfAwoULfcmSJaWsS0TksGNmr4w2bSJdldQOzM4bPgLYUKZaREQSayIFw33AB8Ork04Ddrn7iGYkEREprdiakszsR8A5wFQzawc+D1QBuPsiYDFwEbAK6AEuj6s2ERHZK7ZgcPfL9jPdgU/EVI6IiIxiIjUliYjIBKBgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRMQaDGZ2gZmtNLNVZnZNgenNZvYLM3vWzJab2eVx1iciIjEGg5lVArcBFwILgMvMbMGw2T4BrHD3NwDnADebWXVcNYqISLxHDKcCq9x9tbv3A/cAlwybx4FGMzOgAdgOZGOsUUQk8eIMhjZgfd5wezgu363Aa4ENwPPA1e6eG74iM7vCzJaY2ZKOjo5S1SsikkhxBoMVGOfDhs8HlgKzgBOBW82sacRC7ne4+0J3X9ja2lr8SkVEEizOYGgHZucNH0FwZJDvcuBeD6wC1gDzY6pPRESINxieBOaZ2dzwhPKlwH3D5lkHnAtgZtOB1wCrY6xRRCTxUnF9kLtnzewq4H6gErjL3Zeb2ZXh9EXA9cDdZvY8QdPTZ9x9a1w1iohIjMEA4O6LgcXDxi3Ke78BOC/OmkREJEp3PouISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIlLlLkBEkiM7mOPVnb3Ba0cvG3f10Z3JsrNngG3d/by+rZnL3jSbaY3pcpdaFu7O7r4sW3b3YQZbu/rpHRikozNDZ1+Wjs4M27oyVJjR1Z/l/ONn8PY3zCp6HQoGETkgmewgO3sG6OjMsLUrQ0dnhg07++jo6mN7dz9bu/oxoLs/S382x7aufrZ1949YT3WqgubaKhrTKf7jhc3c8uCfOGveVM6c10rbpFom1VWTqjQm1VWTrgrmrauemLuuoW2yo6ef7kyWrswgXX1ZuvuzdPZl2dU7QN/AINu6+unKDLC9u5/uzCC9A4Ps6h2gO5Mlk82Nuv6qSmNKfQ05d2qrKzlpdktJ/h0Tc+uKSNls6exjw84+Ks3I5nJs2tVHJpvjpc2ddHRmWLutm5WbOtndly24fEtdFZPqqplSX40ZtDbUUJ2q4JSjJjGtMU1bSy1tk2ppa6llRnOamlQFZgbA6o4ufvzkehYv28hDKztGrXFaYw0N6RQ4NNZWMbW+mua6Klpqq6mvqSRdVUl/Nkcmm6M7kyVdVUFddYrGdIrm2ioqK4z6mhRN6SrCj8YAM6O5toqm2hR1VcHusTMzwObdGTIDg+zuy9KdybK1K0POYVtXhh09A6zZ2sW67T1s7RoZfMPVpCqYUl9NQzrFpLpqZjanSVdXMqmuitqqSloba5jeFBwxTamvoba6ktaGGppqg3orKmwcv80DY+5e8g8ppYULF/qSJUvKXYbIIWFX7wBrtnazbnsPm3f1saWzj4FBpyuT5ZVt3Wzc1Uf7jt6Cy6YqjMn11Rw1pY5jpzUwvSlNU7qKWS1pWuqqmd6UZlZLmppU5UHX6e50dGXYvCvDzt5+soPOjp5++gZybOvKsG57Dz0DgwDs7g2OWnb3DrCrd4CegUGGdmtVlUa6qpKBwRx9A6N/Ez9QNakKJtVVM2dqHXOm1DOjOc2Uhhom1wUB1VCTojFdRV11JU3p4Kgojh37WJjZU+6+sNA0HTGIHEZyOWdrd9CsEzTnBO9f3NTJyk2drNi4m/zvgjWpCqpTFdRXp5g9uZYTZ7fwwT87imNaGxjMOalKo7UhTW11BUdMqiNddfA7/bEwM6Y1pg/4XENPf5Z0GFBDO+LsYI7OvqBJJ+ceNN30ZyHcHg7k3Nne3U9P/yA9/UHwNNRUMq0xTW11sKNvqEkxpaGaygojnaqcMDv6YlIwiBwgd9/TBDIwGDS5vLqzlxc37ualLV2s397Dh0+fw7mvnV6Uz8vlnK1dGVZu7mR1R/Dtfv2OHnb3DvDqzl427OwlO+hkcyNbAaY11jBvegNXnzuP10xvZM7UemY119JUm9rzbzicFDoHkaqsYFJ9NZPqq8tQ0aFFwSAyTC7nmEFXJsu2rn629/SzZXcfL3d0s2Ljbl7YuJtNu/ro6R/EDBpqUvQNDDIwuHeHXFtVSW11JR/9zhLeNHcyrY011KQqaamr4qgpwTfvIyfXMau5lnRVBVMbanCgozPDls4+Nu3qY1fvAO07enlh4242d2ZYvaWLzszedv0Kg9mT62ipreK4aY2cdvQUmmurmN5Yw5SGGqY0VDOjKc2kOu0MZXwUDJJ427v7+clT7Tz8pw5Wd3SzaXcfNamKPU0J+WY2pzmmtYEzjpm6pxlmV+8ATbVVzGpOc+SUeuZOqeeISbUM5HJ863cv89NnXmXN1m6yOaerL0v/4Njbus1gzpR6pjfV8LYTZ3FsawPzZzRyzLQGptRXk6rUrUhSfLGefDazC4BvAJXAne5+U4F5zgG+DlQBW9397H2tUyefDz0dnRmmNlTH3oTh7nR0Znhm/U5e7uhi864+nl63kxUbdzOYc46b3sCx0xqY1pimb2CQ2ZPrmFxfTUttFW2Tapk7tZ7GdNVB1ZDLORt395EdzPFyRxcdnRl6+wfZ3jMA7rQ21tDaWMOM5lqaa6uCK1ZiateXZJkQJ5/NrBK4DfhLoB140szuc/cVefO0ALcDF7j7OjObFld9UlzuzoMvbuGVbT207+jl9y9twYHVHd175pk7tZ6/mD+Ny049ktbGGn63cgs1qQpOOnISzbVVRdkhDuac367YzG+WbeT+5ZvpHdh7FFCTquDkIyfxN2cfw9tPnMVx0xsP+vP2p6LCaGupBeCoKfUl/zyRAxFnU9KpwCp3Xw1gZvcAlwAr8uZ5P3Cvu68DcPctMdYnB6GjM8MP/7iOtdu6ybnz86UbRswzub6a17c1c/oxU1i6fidL1+/k24+s4duPrNnnuhvTKU45ahKVZrQ21uAO63f0UF+Toq66kle29bC1K8OMpjT1NSm2d/djBu7wpy2d9A3kaEqnOO/46by+rZn5M5o4YXYzDdUT59JBkYkkzmBoA9bnDbcDbxo2z3FAlZn9DmgEvuHu3x2+IjO7ArgC4MgjjyxJsbJ/27oyrNnazf/8t6V0dGYi14nXVVfynlOO4Mx5rcxsTjO9KU1rY01k+b6BQe5fvomXNnfyyrYeZrXU0lJXxR9WbWXOlHqWrt/Jtq5+jpvRSPuOXgZzzhNrtpNzZ1ZLLa9s62FgMMfsyXXMn9HEjp5+Nuzs3fM52UHn0jceyWlHT+EvF0ynUiEgMiZxBkOh/5XDT3CkgFOAc4Fa4DEze9zdX4os5H4HcAcE5xhKUKsU4O588ZcreGVbDw++GD2Ye9dJbXzkzXOZXF/NpLpqaqv33wyUrqrkkhPbRoz/+DnHFq1mERm/OIOhHZidN3wEMLy9oZ3ghHM30G1mDwNvAF5Cymbp+p18/PtPsWFXX2T8zOY0b5wzmQ+dfhSnHDW5TNWJSLHFGQxPAvPMbC7wKnApwTmFfD8HbjWzFFBN0NT0tRhrFIIjgxt+9cKItv+2llpaG2v46nvfwNGtDWWqTkRKLbZgcPesmV0F3E9wuepd7r7czK4Mpy9y9xfM7DfAc0CO4JLWZXHVmBSbdvVxzb3P8buVHRw3vYGb3n0CD724hW8+uGrUZb763jfwrpOPiLFKESkXdaJ3mMvlnFUdXcyb1oCZ8cVfrOCuP+z7KiCAc+dP46Z3nzDihLGIHB4mxH0Msteca34FwOovXcSXFr9ANud8/m0LDvqGryfXbuevFj02pnlveMfrOPu4Vq772TKeXLOd17U18fFzjuWc17Qeln3niMjYKRhitmbr3hu8jv6HxXve3/3oWn7wsTcxqyW4w3Y8Xu7o4tybfz/m+V+8/oI9N4999yOnjuuzROTwp2CISVcmy+s+f/8+5/nAnX/c8/6dJ7XR0Znhi5ccP+qJ3qXrd/KO2/4QGXfvx0/npNktZHNOlfrREZEDoGAYxZxrfsX8GY385lNnHfS6nlm3g3fe/mhk3IvXX8DbvvkIP7/qDFZs2M17hjUB/fSZVwH4i7wjgQ+fPofPvnUBlRXG6o6uEaGQfyRQVanmIBE5MDr5XED+t/u1N1087uXXhj1pzmhOjzhKWPaP59NQM3oed2WyLLzht2N+2tTX33ci7zhp5E1iIiL7opPP4/SdR9ce0HJDJ5VHk/+NfjQNNSlevP7CyLjVHV1cfveTvLKtJzL+xne9XqEgIkWnYCjgX+5fOa75cznnW79/edTpz3/hvIPqrvno1gZ+/7/+/ICXFxEZDwVDnr68h4gPyX9842iO//z9e7pzbkqn+O3fns2StTv4zfJNfPOyk0pVrohISSgY8sz/7G9GjFu3vWef/eY/sHxTpI//575wPgAXnzCTi0+YWfwiRURKbEzXM5pZlZltMrPjS11QuTz0YuFHP/zwiXUjxrk77s6ca37FFd97CoC3vHY6q790UUlrFBGJw5iCwd0HgAFGdpN9WBgYzHH53U9Gxv3kb04HYPOwHkVXbeli7rWLmXvt4sj4Oz+0UA99EZHDwnjugPomcG3Y8+lhpX1H74hxJ81uAeCx1dsi49/y1ZF3GD/1v99SmsJERMpgPDv5M4GzgVfNbBnQnT/R3d9ezMLidOd/rh4xbujb/+bdmT3j1uZ1Z3HxCTP5/NsWMK0xXfoCRURiNJ5g2Ar8pFSFlNMP/rj3PMILX7wAH6XF7LyvP7zn/W3vP7nkdYmIlMOYg8HdLy9lIRPBCUc0Rx5J+a6T27j36Vfp7R+ktrqSt7x2Gouf33RAd0OLiBwqxnyOwcweNLOWAuObzOzB4pZVHnd+MHp3eC4XHDk8276TXM5Z/PymcpQlIhKr8TQlnUPwuM3h0gTnHw5ZF58wk+fadzKtKXq+4PIz5vKzpRu49I7Hy1SZiEj89hsMZpbfmH6CmW3PG64Ezid4hvMhKzMwSGPNyC4rjppSN2Lcf/69uqYQkcPbWI4YlhDcv+DAAwWm9wKfLGZRcesbyEXOLQxpqRt5gNTWUhtHSSIiZTOWYJgLGLAaeCPB1UlD+oEt7j5YaMFDxSOrtjLavWnL/vH8PV1nP37tubqJTUQOe/s9+ezur7j7WuAm4I3h8NBrI/Dfzez6UhdaarlR7uluqEkxrbEGgBnNumdBRA5/4zn5/NfAXxUY/zRwLfDZolQ0AT1xne5sFpHkGE+XGNOAjgLjtwLTi1NO/Hb1DADw+rbmMlciIjIxjCcY1lH4stSzgPbilBO/Ox8JusNYtmFXmSsREZkYxtOU9K/A18ysGhi6oe1c4Ebgy8UuLC69/cF583Rq34/cFBFJivF0iXGzmU0FbmHvjW79wDfc/Z9LUVwclryyA4D6msOu01gRkQMyrr2hu19rZjcACwguYV3h7l0lqSwmz7bvBOCtetqaiAgwvnMMALh7t7s/6e5PHOqhAPDWE2YBcN7xh+z5cxGRohp3MBxu3jhnEgCvmd5Y5kpERCaGxAdDZiAHQHUq8ZtCRARIeDB0dGb4p8UvAFCjq5JERICEB8PV9zyz531VpfpAEhGBhAfDoy9v2/PeTMEgIgIJDwYRERlJwSAiIhGxBoOZXWBmK81slZlds4/53mhmg2b2njjrExGRGIPBzCqB24ALCe6cvszMFowy35eB++OqTURE9orziOFUYJW7r3b3fuAe4JIC830S+AmwJa7C7rvqjLg+SkRkwoszGNqA9XnD7eG4PcysDXgnsGhfKzKzK8xsiZkt6ego9IiI8Zkztf6g1yEicriIMxgKXQ86/IGaXwc+s79nSLv7He6+0N0Xtra2HnRhg4OjPNdTRCSB4uxruh2YnTd8BLBh2DwLgXvCewqmAheZWdbdf1bKwhrT6nJbRGRInHvEJ4F5ZjYXeBW4FHh//gzuPnfovZndDfyy1KEAMDDoqEcMEZFAbMHg7lkzu4rgaqNK4C53X25mV4bT93leoZTUHYaIyF6xtqG4+2Jg8bBxBQPB3T8cR00AqUrd5yciMiSxe8TOvoFylyAiMiElNhgy2Vy5SxARmZASGwwV6k1VRKSgxAaDiIgUpmAQEZGIxAaDu+52FhEpJLnBUO4CREQmqMQGQ05HDCIiBSU2GB56MbZevUVEDimJDYbe/n124CoikliJDQY1JImIFJbYYMgpGURECkpsMIiISGGJDQbdxyAiUlhig0FERApTMIiISISCQUREIhIbDDrFICJSWGKDYUZzGoD5MxrLXImIyMSS2GA4prUBgKvPnVfmSkREJpbEBsMQ05PcREQiEhsMrk4xREQKSmwwDNEBg4hIVOKDQUREohIbDLpcVUSksMQGwxC1JImIRCU+GEREJCrxwaDLVUVEohIfDCIiEpXYYNDJZxGRwhIbDEPUkCQiEpX4YBARkajEBoO6xBARKSyxwTBEFyWJiETFGgxmdoGZrTSzVWZ2TYHpHzCz58LXo2b2hjjrExGRGIPBzCqB24ALgQXAZWa2YNhsa4Cz3f0E4HrgjlLVo6uSREQKi/OI4VRglbuvdvd+4B7gkvwZ3P1Rd98RDj4OHFHqotSUJCISFWcwtAHr84bbw3Gj+Sjw60ITzOwKM1tiZks6OjoOqBgdMIiIFBZnMBT6bl5w/2xmf04QDJ8pNN3d73D3he6+sLW19SCL0iGDiEi+VIyf1Q7Mzhs+AtgwfCYzOwG4E7jQ3bfFVJuIiITiPGJ4EphnZnPNrBq4FLgvfwYzOxK4F/hv7v5SKYtxnX0WESkotiMGd8+a2VXA/UAlcJe7LzezK8Ppi4DPAVOA28NeT7PuvrCkhaklSUQkIs6mJNx9MbB42LhFee8/BnwszppERCQqsXc+qyFJRKSwxAbDELUkiYhEJTYYdO5ZRKSwxAbDED3aU0QkKvHBICIiUQkOBrUliYgUkuBgCKghSUQkKvHBICIiUYkNBl2VJCJSWGKDYYguShIRiUpsMOiAQUSksMQGwxA9j0FEJCrxwSAiIlGJDQadfBYRKSyxwTBEJ59FRKISHwwiIhKV2GDQoz1FRApLbDAMUUuSiEhUYoNBxwsiIoUlNhj20CGDiEiEgkFERCISGww69ywiUlhig2GIusQQEYlKfDCIiEhUYoPBdV2SiEhBiQ2GIeoSQ0QkKrnBoAMGEZGCkhsMIR0wiIhEJT4YREQkKrHBoJYkEZHCEhsMQ0xnn0VEIhIfDCIiEpXYYFCXGCIihSU2GIaoJUlEJCqxwaA7n0VECos1GMzsAjNbaWarzOyaAtPNzG4Jpz9nZieXvKZSf4CIyCEmtmAws0rgNuBCYAFwmZktGDbbhcC88HUF8K1S1ZMZyJVq1SIih7Q4jxhOBVa5+2p37wfuAS4ZNs8lwHc98DjQYmYzS1HMys2dAFRVJrY1TUSkoFSMn9UGrM8bbgfeNIZ52oCN+TOZ2RUERxQAXWa28gBrmnril9l6gMuW0lRQXeM0UWtTXeOjusbnYOo6arQJcQZDoeb84WeAxzIP7n4HcMdBF2S2xN0XHux6ik11jd9ErU11jY/qGp9S1RVnO0o7MDtv+AhgwwHMIyIiJRRnMDwJzDOzuWZWDVwK3DdsnvuAD4ZXJ50G7HL3jcNXJCIipRNbU5K7Z83sKuB+oBK4y92Xm9mV4fRFwGLgImAV0ANcXuKyDro5qkRU1/hN1NpU1/iorvEpSV3m6htCRETy6FpNERGJUDCIiEhEYoNhf91zFPmzZpvZQ2b2gpktN7Orw/FfMLNXzWxp+Loob5lrw9pWmtn5eeNPMbPnw2m3WBEeKGFma8N1LjWzJeG4yWb2WzP7U/hzUpy1mdlr8rbLUjPbbWafKsc2M7O7zGyLmS3LG1e07WNmNWb243D8H81szkHU9S9m9mLYpcxPzawlHD/HzHrzttuimOsq2u+tyHX9OK+mtWa2tAzba7T9Q/n+xtw9cS+Ck98vA0cD1cCzwIISft5M4OTwfSPwEkG3IF8APl1g/gVhTTXA3LDWynDaE8CfEdzz8WvgwiLUtxaYOmzcPwPXhO+vAb5cjtryfl+bCG7IiX2bAWcBJwPLSrF9gI8Di8L3lwI/Poi6zgNS4fsv59U1J3++YeuJo66i/d6KWdew6TcDnyvD9hpt/1C2v7GkHjGMpXuOonH3je7+dPi+E3iB4I7u0VwC3OPuGXdfQ3CV1qkWdA/S5O6PefAb/i7wjhKVfQnwnfD9d/I+pxy1nQu87O6v7KfektTl7g8D2wt8XrG2T/66/h04dyxHNYXqcvcH3D0bDj5OcC/QqOKqax/Kur2GhMu/F/jRvtZRorpG2z+U7W8sqcEwWtcbJRcewp0E/DEcdVV42H9X3qHiaPW1he+Hjz9YDjxgZk9Z0N0IwHQP7yEJf04rU20QfMPJ/w87EbZZMbfPnmXCnfouYEoRavwIwbfGIXPN7Bkz+72ZnZn32XHVVazfWym215nAZnf/U9642LfXsP1D2f7GkhoMY+p6o+gfatYA/AT4lLvvJug99hjgRIL+oG7eT32lqvsMdz+ZoHfbT5jZWfuYN9baLLgZ8u3A/wtHTZRtNpoDqaPoNZrZdUAW+EE4aiNwpLufBPwt8EMza4qxrmL+3krxO72M6JeP2LdXgf3DqLOO8jlFqy2pwRB71xtmVkXwS/+Bu98L4O6b3X3Q3XPA/yFo4tpXfe1EmwaKUre7bwh/bgF+GtaxOTw0HTp83lKO2gjC6ml33xzWOCG2GcXdPnuWMbMU0MzYm2JGMLMPAW8FPhA2KRA2O2wL3z9F0C59XFx1Ffn3VuztlQLeBfw4r95Yt1eh/QNl/BtLajCMpXuOognb8r4NvODuX80bn9+l+DuBoasl7gMuDa8kmEvwfIonwsPJTjM7LVznB4GfH2Rt9WbWOPSe4OTlsrCGD4WzfSjvc2KrLRT5JjcRtlne5xVr++Sv6z3Ag0M79PEyswuAzwBvd/eevPGtFjwTBTM7OqxrdYx1FfP3VrS6Qm8BXnT3Pc0wcW6v0fYPlPNvbF9npg/nF0HXGy8RfBO4rsSf9WaCw7bngKXh6yLge8Dz4fj7gJl5y1wX1raSvKtogIUE/6leBm4lvHv9IGo7muAKh2eB5UPbgqD98f8Dfwp/Ti5DbXXANqA5b1zs24wgmDYCAwTfvD5azO0DpAmaylYRXFVy9EHUtYqgLXno72zoSpR3h7/fZ4GngbfFXFfRfm/FrCscfzdw5bB549xeo+0fyvY3pi4xREQkIqlNSSIiMgoFg4iIRCgYREQkQsEgIiIRCgYREYlQMIjshwU9bbqZTaiHwU/UuuTQp2AQ2b/1BD1gDnXJfE64Q54aVwFm9jszu3VfdYkUS2zPfBY5VLn7IEG330VnZlXuPnAgy5ayLkk2HTHIhBJ+M77dzL5kZlsteLDKV8ysIpy+1sw+XWCZW/OG15rZ58zsbjPrNLP1ZvY+M2sxs3vMrMuCh5+cN8aa9jTZhL1fPhRO6gjH3x3OZ2b292b2sgUPeXnezP66wHouM7MHzawX+B9mNsXMfmRm7eFyy83s8rzl7gbOJujg0MPXnEJNSWZ2lgUPYukzs81m9rWw25cxbd9wnndZ0Atqr5ltt6B30elj2VZyeFAwyET0AYKeQU8HrgI+BbxvnOv4FMGt/ycD/0bQF/0PgcUEPXw+DHzfzNLjXO96gu4SAI4naMq5Ohy+gaD7h08QPEzlRuBfzeziYeu4Ebg9nOdnBN0VPE3Q8d3xwDfC5c4N578aeAz4v+HnzSTa7TIAZtZG0M32MwRdN3+UoK+pG4fNOur2NbMZBM8n+Q7wWoKH23xvDNtFDicH05eNXnoV+wX8Dnhs2LjfAneG79cy7Elg4TK35g2vBX6UN9xA0BfNLXnj5oTjFo6hpsi8wDnh8NS8eeqBXuDMYct+HVg8bD1/N4bPvGfo31zo3zhKXf9E0BdORd48HwYyQN0Yt+/J4TqPKvffgl7le+kcg0xEzw0b3sDeh5SMex3u3mVmPQSduA3ZHP4c73pHs4Dgm/9vzCy/A7IqgqDKtyR/IOzF8xqCb+1tBI9srCbYiY/Hawl2+rm8cY+E6zqWvdtkX9v3WeA/gGVm9kD4/t/dvWOctcghTMEgE9Hwk7HO3mbPHCMfOlI1xnUMDBuG4jWnDq3nbcC6/dTSPWz408DfETQZPQ90AV9i/KFljP7wlfzxo25fdx8Mz72cRtAF+0eBG83sbHd/dpz1yCFKwSCHmtydG6sAAAGQSURBVA6CNnYAwnME8wna1ePSH/6szBu3gqDJ5ih3f3Cc63sz8At3/x7s6Z//OGDnsM+sLLBsvhXAe82sIu+o4c3hsi+PtRh3d4JzGo+Z2RcJup9+H8HRhCSAgkEONQ8CHzGz+whC4joKHzGU0isE37IvNrNfAL3u3mlmXwG+Eu7YHyY4t3EakHP3O/axvpeA95nZm4GtwCeBuUTDbi3BA9/nEBxRFHr61u0EJ5JvN7NvEDxr4yaCcxM9BeYfwcxOI3hwzf0EzW0nETz5a8VYlpfDg65KkkPNjQTh8HPgAYI29KfjLMDdXwU+T3CydzPBA1EAPgt8gaBpaDnBSd13A2v2s8obCK6g+jVBoHSz91nNQ75C8M1/BUEgHjlKXRcS7MyXAncRPJzmH8bxz9sFnAH8kuABMTcD17v798exDjnE6UE9IiISoSMGERGJUDBI4pnZovBu6EKvReWuTyRuakqSxDOzaUDTKJN3u/uWOOsRKTcFg4iIRKgpSUREIhQMIiISoWAQEZEIBYOIiET8F9YLe2d2nyuXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --epsilon 0.2\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The above case without learning**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeN0lEQVR4nO3de5gcdZ3v8fenLzOTmVwhCbDhKhchLl4wIMcb7GEVgmeNl30UZMVl3YflWfHRZ/Uc8XDc5VlRdI+4ygIix4MorqCruILigh50eXYFJYFwCRgIGEi4JIFAbjOZnu7+nj+qZujq9CQzSU/NhP68nqef6bp09TfVnfp0/arqV4oIzMzMhhUmuwAzM5taHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZuQWDpGskrZf04CjTJekySask3S/puLxqMzOzl+S5x3AtcNpOpi8Gjkwf5wJfy6EmMzNrklswRMQdwMadzLIE+HYk7gJmSzogn+rMzGxYabILaLAAWNMwvDYd90zzjJLOJdmroK+v7/VHH310LgWamb1cLFu27LmImNdq2lQKBrUY17K/joi4GrgaYNGiRbF06dKJrMvM7GVH0hOjTZtKZyWtBQ5qGD4QeHqSajEz61hTKRhuAs5Oz046EdgUETs0I5mZ2cTKrSlJ0vXAycBcSWuBvwPKABFxFXALcDqwCugHzsmrNjMze0luwRARZ+5iegAfyakcMzMbxVRqSjIzsynAwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8vINRgknSZppaRVki5oMX2WpJsl3SdphaRz8qzPzMxyDAZJReAKYDGwEDhT0sKm2T4CPBQRrwFOBi6V1JVXjWZmlu8ewwnAqoh4PCIqwA3AkqZ5ApghScB0YCNQzbFGM7OOl2cwLADWNAyvTcc1uhw4BngaeAD4WETUmxck6VxJSyUt3bBhw0TVa2bWkfIMBrUYF03DpwLLgT8AXgtcLmnmDi+KuDoiFkXEonnz5rW/UjOzDpZnMKwFDmoYPpBkz6DROcCNkVgF/B44Oqf6zMyMfIPhbuBISYelB5TPAG5qmudJ4BQASfsBrwQez7FGM7OOV8rrjSKiKul84FagCFwTESsknZdOvwr4LHCtpAdImp4+FRHP5VWjmZnlGAwAEXELcEvTuKsanj8NvD3PmszMLMtXPpuZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDJyDQZJp0laKWmVpAtGmedkScslrZD073nWZ2ZmUMrrjSQVgSuAtwFrgbsl3RQRDzXMMxu4EjgtIp6UND+v+szMLJHnHsMJwKqIeDwiKsANwJKmeT4A3BgRTwJExPoc6zMzM/INhgXAmobhtem4RkcBcyT9StIySWe3WpCkcyUtlbR0w4YNE1SumVlnyjMY1GJcNA2XgNcD7wBOBT4j6agdXhRxdUQsiohF8+bNa3+lZmYdLLdjDCR7CAc1DB8IPN1inuciYhuwTdIdwGuAR/Ip0czM8txjuBs4UtJhkrqAM4Cbmub5MfAWSSVJvcAbgIdzrNHMrOPltscQEVVJ5wO3AkXgmohYIem8dPpVEfGwpH8D7gfqwDci4sG8ajQzM1BEczP/3mXRokWxdOnSyS7DzGyvImlZRCxqNc1XPpuZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVnGmIJBUlnSs5JeNdEFmZnZ5BpTMETEEDDEjt1km5nZy8x4mpL+Cfi0pDy76jYzs5yNZyP/FuAk4ClJDwLbGidGxDvbWZiZmU2O8QTDc8APJ6oQMzObGsYcDBFxzkQWYmZmU8OYjzFIul3S7BbjZ0q6vb1lmZnZZBnPweeTga4W43tIjj+YmdnLwC6bkiQd1zD4akkbG4aLwKnAU+0uzMzMJsdYjjEsJbl+IYDbWkwfAD7azqLMzGzyjCUYDgMEPA4cT3J20rAKsD4iahNQm5mZTYJdBkNEPAEg6QvA8RFxVeN0SedJWhARn5mgGs3MLEfjOfj8Z8C9LcbfA5zdnnLMzGyyjScY5gMbWox/DtivPeWYmdlkG08wPEnr01LfCqxtTzlmZjbZxtMlxteBf5TUBQxf0HYKcAnwxXYXZmZmk2M8XWJcKmkucBkvXehWAb4aEf8wEcWZmVn+xtWFdkR8WtLFwEKSU1gfioitE1KZmZlNinHfWyEitgF3T0AtZmY2Bfiez2ZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZRq7BIOk0SSslrZJ0wU7mO15STdKf5lmfmZnlGAySisAVwGKSK6fPlLRwlPm+CNyaV21mZvaSPPcYTgBWRcTjEVEBbgCWtJjvo8APgfU51mZmZqk8g2EBsKZheG06boSkBcC7gcxd4ppJOlfSUklLN2xodYsIMzPbXXkGg1qMi6bhrwCf2tU9pCPi6ohYFBGL5s2b17YCzcxsNzrR2wNrgYMahg8Enm6aZxFwgySAucDpkqoR8a/5lGhmZnkGw93AkZIOA54CzgA+0DhDRBw2/FzStcBPHApmZvnKLRgioirpfJKzjYrANRGxQtJ56fSdHlcwM7N85LnHQETcAtzSNK5lIETEn+dRk5mZZfnKZzMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMAArn93CA2s3TXYZZmZTQq5dYkxVp37lDgBWf+Edk1yJmdnk8x5Dg22D1ckuwcxs0nVsMLzYX+HQC37KD5etHRl3/W+f5BPfv4+Byk7vE2Rm9rLWsU1Ja18YAOCbv/79yLiLf/owAJsGKnzjQ8dPSl1mZpOtY/cYSsXkTqND1ea7i8IvHl6fdzlmZlNG5wZDIQ2Gen2HaQftMy3vcszMpoyODQZIgoEddxhYs3GAodqOgWFm1gk6NhiU5kJllAD4yi8eybEaM7Opo3ODIf07fBAakusYFv/h/gBc8cvHeO/Xfj0JlZmZTa6ODYbC8C5Dk8s/cNzI82VPvMBtK57NqyQzsymhY4OhORf++uTDASgWxGmv2n9k/LnXLeNdV/wnT704gJlZJ+jYYGjWUy6OPL/qg6/n8c+fPjK8fM2LvOkLt/PR6++lUvVBaTN7eevYC9xEdpeh0LQHUSiIr3/w9fzVdctGxt1839PcfN/TI8OPXLwYgL/5/nJ+cv8zAFx4+jGc/cZD6C4VMTPbG3VsMETTearHHDBzh3lOfdX+3Prxt450stfsqP/1sx3Gfe6Wh/ncLQ+3nP/HH3kTB+/Ty5y+rt2o2MwsH50bDE3XL5xyzH4t53vl/jNY/YV38OPlT/Hc1gqf/clDu/2eS674z8zw2xbux5ff9xoeeGoTbzx87m4vt50GKjW2DA7RVSwwu9cBZtaJOjcYGp7P6S3vcv4lr10AwD/d/igv9g9x72fexn//wf384uF1XHvO8Zz8yvnZ5UcgiW2DVbpKBf7+5oe47q4nMvP8/KF1HHvRbZlxB+/Ty6Xvew0HzOphwexp1OrJcorNbV2j2D5Uo6tYYMtglX9Zuoa+7hJPvzjAyme3cP5/PYJjF8xCEk+9OMA53/wt67cM8mL/0C6X++7XLeCV+8/g+EPncMT8Gcyatut1lpeN2yr0dhVHjhPV6kGtHpSLIiI50aC/UqMeQX+lRqVaZ/tQjRk9Zfab2Y0kBqs11mwcYNNAhVo9+fwA6pE0M/ZXamwaGKK7VKCnXKRUFNPKRXq7StQjmN1bZkZPmf5KlVo9GKoFtXqd57dWeGbTdiToLhWZ2VOit7tErV6nq1iku1ygr7vE9PRRUPKjpTDGz/vlbLBaY6iWfI4DlRpbB6vMnFamq1igu1RAo5xZ2G6Vap0XBypEQD2Ccvr+pUKBzduHRi6GLRcLFCTKRSGJrmKBnnJ+dbaTovmn815m0aJFsXTp0nG/7su3reSy21cBMG9GN3df+MftLm1UEcF3f/skF/7owd16/SXvOZYzTziYZzdtZ8kV/8G6zYNtq23+jG4O3qeXcrHAnY8/v9N5D923lyPmT+fAOb2sWr+VA2b1UJCo1Ops2DLIo+u3MFCpsXl7lb6uInP6ulj7wgCvmNvHoXP7mD2tzIH79NJdKrD2hX42DQwxOFSnq1RgqFbnyY39dJUKHDSnF4BSMZlveneJ7lKB7UN11m3ezqPrtwLQUy5QlNhWqY0cM5LE9O4SmwZah9+0cpFiQQwM1ajVJ/f/wnDN5WKBAPbt66JYSMKtq1Sgq1hgVm+Zffu66OsuMXtamf6hGkPVOqWi2DpYo1QQ3aUCc6d3M1Sr89SLA1RrQW9Xkek9JSKSfsIKEn3dJSKCbYM11m/ZDsD+M3vYXq2xeaDKhi2D9FeSruif21pBgmot6CoVmFYuIiW1SsnGs6tUGNl47j+zh3oEm7dX6Skn8w+fIr51sMq2Sm0kfCvVOpVqnU0DyUZWEpVqnYGh0Xs5lqC3XCRITj2vR/JvHKzWqdeTGocDvChRrQfTu0vJj4Z03qJEqVigVg82bBlkqFanmn4HCkrOUOwfrLFlD7rjLxdFb1fy+3tauUgQFCW6y8VknRSSHwLVWjBUr9PbVaReh77uIqVCcm5Q8tlW2bq9ipScKDMcQO987QI+eOIhu1WbpGURsajVtI7dY9iw9aWNaTnnX2eSOOsNh3DWG7IfaLVW56KbV/Cdu57c6es/feMDfPrGB3b5PgXBsQfO5qA505jRU2bzwBA/fSA5SH70/jOY0VPi6g8uGtMxj/Wbt3P779Zzx6MbGKjU+OXKDax+vp/Vz/fv9HVHzJ/OkfuV2bq9yj59XfR1lThgdg9PPL+Ne7dVeCHdWykXNdJ0Na1cpFYPjtpvOoPVOg8/s5mhWvIf9g9m97B+8yD9Q1XKhQIL5kxj8R/uT6EgXthWoR7Q112iXEwCikg2agtm91AqFugqFZjTW6ZYKLB1+xBPbhxgsFpj1rQyh8+bzj7puigVNbK3FhHM6CnTXSqkewN16gH9lSqbBpJmt439FTYPVJk1rUwh3WAWC6KnXOSI+dOpR7Bl+xADlWSDVyok9VWq9ZH/9Ju3D1GPYHCoTgAv9icbymI67+BQstfyyLotbB2ssnkg2RstF5MNSF93st62D9XZuG2QUqHAfrO66SkV6U9/cQ9vxCNgYKiGlKzv+TO6CZK92GldRWZNKzOnt4tZvV1EBIfPnw5pqAxvtOuRfGfraXBVqvVkjwd4ZlMSNHP6ygxUamzcNkREEAHTe0rM7CmNrNvuUrLnNLOnTFdRBCQhOK1MV6lAtR4je2oDlRqVWp2BSo3+SlJ/PYKCRH+lRldRFAsFKrUag0N1hmp1apH0jbZ1sEqpkITitkqVCEbW76H79o4ESRJuUKvXmVZOftDs29cFEqWCGKrVk2XX62nNyfofqtdH9hYjgkqtzpbtVfoHqwRJM21BohbB9qEa29MfI4W0RaBcLCSfCUl4VuvJ57S9GvR1lZg/oxshtleT11VrMWHbro4NhkaV2tTYayoVC1z8rmO5+F3H7jBt08AQM7pL3Pn485z1jd+MjJ87vZuL3rmQk46ax4yeXTfvXLGbtc2f2cMZJxzMGSccvMO0arorPfwfeHj3fyy70NsGq1SqdWZOK4+5ucx2rVqrU5B22iSVbJTYK5s6bGI5GIDntravKWaiDLfpv+mIuVPuFqSl4vAubzLcVRr7hqavu0Rf90RU1dmGP5OdcRDbaDr2Are9/NCKmdmE6dhgMDOz1hwMZmaW4WAwM7OMXINB0mmSVkpaJemCFtPPknR/+vi1pNfkWZ+ZmeUYDJKKJGdLLgYWAmdKWtg02++BkyLi1cBngasnqh4ffDYzay3PPYYTgFUR8XhEVIAbgCWNM0TEryPihXTwLuDAHOszMzPyDYYFwJqG4bXpuNF8GNix+1JA0rmSlkpaumHDhj0u7JL37HhBmZlZp8ozGFpdTdOyQUfSH5EEw6daTY+IqyNiUUQsmjdv3h4XdmaLq3nNzDpVnlc+rwUOahg+EHi6eSZJrwa+ASyOiJ334mZmZm2X5x7D3cCRkg6T1AWcAdzUOIOkg4EbgQ9GxCMTWUzzjXrMzCyRWzBERBU4H7gVeBj4fkSskHSepPPS2f4W2Be4UtJySePvT3uM/ii9f8J5Jx0+UW9hZrZXyrUTvYi4BbiladxVDc//EvjLPGoZ7uL5pKP2/BiFmdnLScde+eymJDOz1jo2GIa5K3ozs6yODwYzM8vq3GBwS5KZWUudGwwptySZmWV1fDCYmVlWxwaDW5LMzFrr2GAYJp+WZGaW0fHBYGZmWR0bDL5Rj5lZax0bDMPckmRmltWxweAuMczMWuvYYBjmHQYzs6yODwYzM8vq2GDwwWczs9Y6NhiG+eCzmVlWxweDmZlldWwwuCXJzKy1jg2Gl7gtycyskYPBzMwyOjYYwqclmZm11LHBMMxnJZmZZXV8MJiZWVbHBoMbkszMWuvYYBjmliQzs6yODwYzM8vq3GBwW5KZWUudGwwp3/PZzCyr44PBzMyyOjYYfAc3M7PWOjYYhrkhycwsq2ODwT1imJm11rHBMMzHns3Msjo+GMzMLKtjg8FNSWZmreUaDJJOk7RS0ipJF7SYLkmXpdPvl3TchNfkw89mZhm5BYOkInAFsBhYCJwpaWHTbIuBI9PHucDXJqqewWp9ohZtZrZXy3OP4QRgVUQ8HhEV4AZgSdM8S4BvR+IuYLakAyaimMc2bAWgXPIeg5lZo1KO77UAWNMwvBZ4wxjmWQA80ziTpHNJ9igAtkpauZs1zT3mizy3m6+dSHPBdY3TVK3NdY2P6xqfPanrkNEm5BkMrX6aNx8CHss8RMTVwNV7XJC0NCIW7ely2s11jd9Urc11jY/rGp+JqivPpqS1wEENwwcCT+/GPGZmNoHyDIa7gSMlHSapCzgDuKlpnpuAs9Ozk04ENkXEM80LMjOziZNbU1JEVCWdD9wKFIFrImKFpPPS6VcBtwCnA6uAfuCcCS5rj5ujJojrGr+pWpvrGh/XNT4TUpfCV3qZmVmDjr3y2czMWnMwmJlZRscGw66652jzex0k6ZeSHpa0QtLH0vEXSXpK0vL0cXrDaz6d1rZS0qkN418v6YF02mVqw71JJa1Ol7lc0tJ03D6Sfi7p0fTvnDxrk/TKhvWyXNJmSR+fjHUm6RpJ6yU92DCubetHUrek76XjfyPp0D2o639L+l3apcyPJM1Oxx8qaaBhvV2Vc11t+9zaXNf3GmpaLWn5JKyv0bYPk/cdi4iOe5Ac/H4MeAXQBdwHLJzA9zsAOC59PgN4hKRbkIuAT7aYf2FaUzdwWFprMZ32W+C/kFzz8TNgcRvqWw3MbRr3D8AF6fMLgC9ORm0Nn9ezJBfk5L7OgLcCxwEPTsT6Af4auCp9fgbwvT2o6+1AKX3+xYa6Dm2cr2k5edTVts+tnXU1Tb8U+NtJWF+jbR8m7TvWqXsMY+meo20i4pmIuCd9vgV4mOSK7tEsAW6IiMGI+D3JWVonKOkeZGZE3BnJJ/xt4F0TVPYS4Fvp8281vM9k1HYK8FhEPLGLeiekroi4A9jY4v3atX4al/UD4JSx7NW0qisibouIajp4F8m1QKPKq66dmNT1NSx9/fuA63e2jAmqa7Ttw6R9xzo1GEbremPCpbtwrwN+k446P93tv6ZhV3G0+hakz5vH76kAbpO0TEl3IwD7RXoNSfp3/iTVBskvnMb/sFNhnbVz/Yy8Jt2obwL2bUONf0Hyq3HYYZLulfTvkt7S8N551dWuz20i1tdbgHUR8WjDuNzXV9P2YdK+Y50aDGPqeqPtbypNB34IfDwiNpP0Hns48FqS/qAu3UV9E1X3myLiOJLebT8i6a07mTfX2pRcDPlO4F/SUVNlnY1md+poe42SLgSqwD+no54BDo6I1wF/A3xX0swc62rn5zYRn+mZZH985L6+WmwfRp11lPdpW22dGgy5d70hqUzyof9zRNwIEBHrIqIWEXXg/5A0ce2svrVkmwbaUndEPJ3+XQ/8KK1jXbprOrz7vH4yaiMJq3siYl1a45RYZ7R3/Yy8RlIJmMXYm2J2IOlDwH8DzkqbFEibHZ5Pny8jaZc+Kq+62vy5tXt9lYD3AN9rqDfX9dVq+8Akfsc6NRjG0j1H26Rtef8XeDgivtwwvrFL8XcDw2dL3ASckZ5JcBjJ/Sl+m+5ObpF0YrrMs4Ef72FtfZJmDD8nOXj5YFrDh9LZPtTwPrnVlsr8kpsK66zh/dq1fhqX9afA7cMb9PGSdBrwKeCdEdHfMH6eknuiIOkVaV2P51hXOz+3ttWV+mPgdxEx0gyT5/oabfvAZH7HdnZk+uX8IOl64xGSXwIXTvB7vZlkt+1+YHn6OB24DnggHX8TcEDDay5Ma1tJw1k0wCKS/1SPAZeTXr2+B7W9guQMh/uAFcPrgqT98f8Bj6Z/95mE2nqB54FZDeNyX2ckwfQMMETyy+vD7Vw/QA9JU9kqkrNKXrEHda0iaUse/p4Nn4ny3vTzvQ+4B/iTnOtq2+fWzrrS8dcC5zXNm+f6Gm37MGnfMXeJYWZmGZ3alGRmZqNwMJiZWYaDwczMMhwMZmaW4WAwM7MMB4PZLijpaTMkTambwU/Vumzv52Aw27U1JD1gDnfJfHK6QZ6bVwGSfiXp8p3VZdYuud3z2WxvFRE1km6/205SOSKGdue1E1mXdTbvMdiUkv4yvlLS5yU9p+TGKl+SVEinr5b0yRavubxheLWkv5V0raQtktZIer+k2ZJukLRVyc1P3j7GmkaabNLeL3+ZTtqQjr82nU+S/oekx5Tc5OUBSX/WYjlnSrpd0gDwV5L2lXS9pLXp61ZIOqfhddcCJ5F0cBjp49BWTUmS3qrkRizbJa2T9I9pty9jWr/pPO9R0gvqgKSNSnoX3W8s68peHhwMNhWdRdIz6BuB84GPA+8f5zI+TnLp/3HA90n6ov8ucAtJD593AN+R1DPO5a4h6S4B4FUkTTkfS4cvJun+4SMkN1O5BPi6pHc0LeMS4Mp0nn8l6a7gHpKO714FfDV93Snp/B8D7gS+mb7fAWS7XQZA0gKSbrbvJem6+cMkfU1d0jTrqOtX0v4k9yf5FnAMyc1trhvDerGXkz3py8YPP9r9AH4F3Nk07ufAN9Lnq2m6E1j6mssbhlcD1zcMTyfpi+ayhnGHpuMWjaGmzLzAyenw3IZ5+oAB4C1Nr/0KcEvTcj4xhve8Yfjf3OrfOEpdnyPpC6fQMM+fA4NA7xjX73HpMg+Z7O+CH5P38DEGm4rubxp+mpduUjLuZUTEVkn9JJ24DVuX/h3vckezkOSX/79JauyArEwSVI2WNg6kvXheQPKrfQHJLRu7SDbi43EMyUa/3jDuP9JlHcFL62Rn6/c+4BfAg5JuS5//ICI2jLMW24s5GGwqaj4YG7zU7Flnx5uOlMe4jKGmYWhfc+rwcv4EeHIXtWxrGv4k8AmSJqMHgK3A5xl/aInRb77SOH7U9RsRtfTYy4kkXbB/GLhE0kkRcd8467G9lIPB9jYbSNrYAUiPERxN0q6el0r6t9gw7iGSJptDIuL2cS7vzcDNEXEdjPTPfxTwYtN7Flu8ttFDwPskFRr2Gt6cvvaxsRYTEUFyTONOSX9P0v30+0n2JqwDOBhsb3M78BeSbiIJiQtpvccwkZ4g+ZX9Dkk3AwMRsUXSl4AvpRv2O0iObZwI1CPi6p0s7xHg/ZLeDDwHfBQ4jGzYrSa54fuhJHsUre6+dSXJgeQrJX2V5F4bXyA5NtHfYv4dSDqR5MY1t5I0t72O5M5fD43l9fby4LOSbG9zCUk4/Bi4jaQN/Z48C4iIp4C/IznYu47khigAnwEuImkaWkFyUPe9wO93sciLSc6g+hlJoGzjpXs1D/sSyS//h0gC8eBR6lpMsjFfDlxDcnOa/zmOf94m4E3AT0huEHMp8NmI+M44lmF7Od+ox8zMMrzHYGZmGQ4G63iSrkqvhm71uGqy6zPLm5uSrONJmg/MHGXy5ohYn2c9ZpPNwWBmZhluSjIzswwHg5mZZTgYzMwsw8FgZmYZ/x/LN2y4ElP8YgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --epsilon 0.2\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon, do_learn=False)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using the cb_explore_adf contextual bandit algorithm and exploring with the explore-first algorithm with tau value = 2 and plotting ctr vs num_iterations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXRdZ3nv8e+jebYkS55kO5IT24kdMtiyEyDjTZrEpCW30JYE2rQ0vWlWSRe0pSVcVoFbuE1ogRZuoG5K00BpY1oIEEIGAyG40ARHdhzHQ2wrHuVBgydJljUdPfePvSWfLUuOFB/tI/v8PmudpbOHs8/jLfn8zn733u9r7o6IiMigrHQXICIik4uCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJCK2YDCzR82sxcw2jbLczOzLZtZoZhvNbElctYmIyClxHjE8Btx2huUrgPnh417gH2KoSUREhoktGNx9DXDkDKvcAXzDAy8B5WY2M57qRERkUE66C0hSA+xLmm4K5x0cvqKZ3UtwVEFxcfHSiy++OJYCRUTOF+vWrWtz9+qRlk2mYLAR5o3YX4e7PwI8AlBfX+8NDQ0TWZeIyHnHzPaMtmwyXZXUBMxJmp4NHEhTLSIiGWsyBcOTwN3h1UlXA8fd/bRmJBERmVixNSWZ2ePADUCVmTUBnwJyAdx9JfA08C6gEegCPhhXbSIickpsweDud73Jcgc+FFM5IiIyisnUlCQiIpOAgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhIRazCY2W1mts3MGs3sgRGWTzGzH5jZq2a22cw+GGd9IiISYzCYWTbwFWAFsAi4y8wWDVvtQ8AWd78cuAH4gpnlxVWjiIjEe8SwHGh0953u3gusAu4Yto4DpWZmQAlwBOiPsUYRkYwXZzDUAPuSppvCeckeBi4BDgCvAR9294HhGzKze82swcwaWltbJ6peEZGMFGcw2AjzfNj0rcAGYBZwBfCwmZWd9iL3R9y93t3rq6urU1+piEgGizMYmoA5SdOzCY4Mkn0QeMIDjcAu4OKY6hMREeINhpeB+WZWF55QvhN4ctg6e4GbAMxsOrAQ2BljjSIiGS8nrjdy934zux94DsgGHnX3zWZ2X7h8JfAZ4DEze42g6elj7t4WV40iIhJjMAC4+9PA08PmrUx6fgC4Jc6aREQkSnc+i4hIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREImINBjO7zcy2mVmjmT0wyjo3mNkGM9tsZj+Lsz4REYGcuN7IzLKBrwC/AjQBL5vZk+6+JWmdcuCrwG3uvtfMpsVVn4iIBOI8YlgONLr7TnfvBVYBdwxb5/3AE+6+F8DdW2KsT0REiDcYaoB9SdNN4bxkC4AKM3vBzNaZ2d0jbcjM7jWzBjNraG1tnaByRUQyU5zBYCPM82HTOcBS4HbgVuAvzWzBaS9yf8Td6929vrq6OvWViohksNjOMRAcIcxJmp4NHBhhnTZ3PwGcMLM1wOXA9nhKFBGROI8YXgbmm1mdmeUBdwJPDlvn+8C1ZpZjZkXAVcDWGGsUEcl4sR0xuHu/md0PPAdkA4+6+2Yzuy9cvtLdt5rZs8BGYAD4mrtviqtGEREBcx/ezH9uqa+v94aGhnSXISJyTjGzde5eP9Iy3fksIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEWMKBjPLNbNDZrZ4ogsSEZH0GlMwuHsf0Mfp3WSLiMh5ZjxNSf8P+LiZxdlVt4iIxGw8H/LXAtcD+81sE3AieaG7vzuVhYmISHqMJxjagO9MVCEiIjI5jDkY3P2DE1mIiIhMDmM+x2Bmz5tZ+Qjzy8zs+dSWJSIi6TKek883AHkjzC8gOP8gIiLngTdtSjKzJUmTl5nZkaTpbOBWYH+qCxMRkfQYyzmGBoL7FxxYPcLyk8Afp7IoERFJn7EEQx1gwE5gGcHVSYN6gRZ3T0xAbSIikgZvGgzuvgfAzB4Clrn7yuTlZnafmdW4+19OUI0iIhKj8Zx8/m3glRHmrwfuTk05IiKSbuMJhmlA6wjz24DpqSlHRETSbTzBsJeRL0u9DmhKTTkiIpJu4+kS4x+BvzOzPGDwhrabgAeBz6W6MBERSY/xdInxBTOrAr7MqRvdeoEvufvfTERxIiISv3F1oe3uHzezzwKLCC5h3eLunRNSmYiIpMW4x1Zw9xPAyxNQi4iITAIa81lERCIUDCIiEqFgEBGRCI3fLCJp0dOfYO2uI2zYe4xNB47z8x1tVJfms6y2krdfOJWr501lVnlhusvMSAoGEYnFwIBz4PhJGnYf5amNB3jxjcOc6E1gBnMqiljxtpkcP9nH6i3N/Oe64J7ZmvJCbr5kGktrK1kyt5zZFUVp/ldkBgWDiEyIxICz9WA76/Yc5ReNbazfe4y2zh4g+MC/48oablw4jeW1lUwpyj3tdQ27j/DfbxzmWw37+PqLewCYV1XMNfOrWDK3gsvnlFM7tQgzS8u/73xm7p7uGs5KfX29NzQ0pLsMEQGOdfXyzKZDPLf5EGt3HaGrN+iRf9aUApbVVbKstpLFs8q4fHY5WVlj+0Dv7R9ge3MHa3cd4afbWli35+jQdqeX5XPFnHKuqpvKVfMqWTi9lJxsnTodCzNb5+71Iy6LMxjM7DbgSwQjv33N3R8aZb1lwEvA+9z922fapoJBJH3cnV80HuYnrzfzWtNxXtl3jMSAM7eyiHdeNJXldZVcOaeCC1L4zb4/McD25k5e2XeUX+48woZ9x9h7pAuAwtxs6msrWF5byeKaMi6bXU5VSX5K3vd8MymCwcyyge3ArxB0uvcycJe7bxlhvR8B3cCjCgaRyeVYVy9rdx3hv3a08ZOtzRw43k1BbhaLZpZx9byprLh0JpfWlMXaxNN0tIt1e47yyt5j/LyxjcaWUx0y1FUVc+Xccq6YU86y2krmTyvRUQVnDoY4zzEsBxrdfWdY1CrgDmDLsPX+GPgOwWhxIpJmPf0JNjYd59V9x/jRlmZe3n2EAQ++nV8zv4o/v20hKy6dSUFudtpqnF1RxOyKIu64ogaAju4+th7sYP3eozTsPsKa7a08sT4Ymj4vJ4uF00upr60YatqaU1E05qatTBBnMNQA+5Kmm4Crklcwsxrg14H/wRmCwczuBe4FmDt3bsoLFcl0Xb39PPXqQX62o5WfbWuls6cfgPnTSvijGy7imvlVXDm3nPyc9IXBmZQW5LK8rpLldZVw/YW4O01HT9Kw5whbD3bw6r5jPL52L//yi91AEHK1VcVcMaechdNLuKCqmIuqS6gpL8zIwIgzGEbau8Pbsf4e+Ji7J850GOrujwCPQNCUlLIKRTJYW2cPL+08zDObDvGTrc109w0wrTSfFZfO4KZLpnPFnHJmTClId5lviZkxp7KIOZVF/PqVwby+xACb9h9ne3MH2w51sqOlg6c2HuDx7v6h1xXnZbNgRinvvLCKq+dN5cq55RTnn/8Xc8b5L2wC5iRNzwYODFunHlgVhkIV8C4z63f378VTokhm6elP8OMtLfzwtQM8t7mZxIBTXpTLby6dwx1XzGLpBRXn7eWgudlZXDm3givnVgzNc3daO3rYfbiLxpZOth1q57X9x/mHn73Bwz9tJMtgXnUJC6aXUDu1mEtrprD0ggqml52bgTmaOIPhZWC+mdUB+4E7gfcnr+DudYPPzewx4CmFgkhqDQw4G/cf53uv7Oc765vo6O5nanEev/eOWn71spksnjWFvJzMPDlrZkwrK2BaWUHQDBXq6O7jlb3HaNhzlC0H2tlyoJ3Vm5vpHwgaLKpK8lg4o5QF00u5dNYULq2ZQl1V8Tm7H2MLBnfvN7P7gecILld91N03m9l94fKVcdUikmmOnuhlzY5WVm9u5vnXWzjZlyAvO4tbFk/nN5bO5tr51WRnYFv6WJUW5HLdgmquW1A9NK+7L8Hrhzpo2H2EbYc62Nbcwaq1+zjZtxuA7CxjTkUh86eXUldVTO3UYi6sLmbhjFLKi/JGeafJQTe4iZyn3J01O9p4Yn0Tz246RE//ANWl+dywoJpldZXcumhG5I5jOXuJAaexpZOtB9tpbOnkjdZOdrR0svdIF739A0Pr1ZQXcvGMUi6ZWcbiWWUsnjWFOZWFsTbbTZbLVUVkgrk7DXuO8u+/3Mt/7WijrbOH8qJc3rNkNu9dUsOVcyt0ZDCBsrOMhTNKWTijNDJ/sJ+oHS2dbDvUwab9x9l2qIMXtreSCJujyotymVtZxMLppUPNUpfMLKO6NP4b9BQMIueBts4eVq3dy3+ua2LP4S7KCnK4dkE1N108jdsvmzlpLyvNFFlZNnSvxY0Lpw3N7+5LsL25g9f2H2fzgXb2Hu7i+ddbhjoRBJhWms/FM8u4qLqEuqoi5k8PQqOyeOKaoxQMIueo9u4+frjxIN96eR8bm44x4LC8tpIP3XgRt79tZkZcVnmuK8jN5rLZ5Vw2uzwyv62zh+3NHUMnul8/1MHaXYfp7jvVHFVVks//uraOP7z+wpTXpb8ckXPMrrYTPLJmJ9/fsJ+u3gTzqou5/8aLePcVNVw0rSTd5UkKVJXkU1WSzzsurBqa5+4cau9m26EOdjQH913MnKDxKhQMIueA7r4Ez246xNdf3M0re4+Rn5PFuy+fxfuvmssVc8rP23sN5BQzY+aUQmZOKeSGpOaoiaBgEJmk3J1nNh3i6dcO0rD7KIfau5lXVcxHb1nAb9bPOe9uqpLJQ8EgMokMXu744httPPHKfjY2HaeqJI+31Uzhwfe8jesXVGdk3z0SLwVDCtU+8MOh5/Oqi1n9kevUva+ckbtz5EQv2w51sHpLMz949QCHT/QCwWhlD73nbfxW/RyFgcRKwfAW3PuNBlZvaT7jOjtbT3DRJ57h5kum8093L1UbsERsPdjOqrV7eWF7K3sOB4PM5GYbNyycxi2LprP0ggrqqor1dyNpoWAYp1v/bg3bmjvGvP6PtzZT9/Gn+ZffW8aNF0/sCaPx2t7cwQ83HuT1Q+3kZGUxpSiXk70J9h7pYlltJX2JAQpys3hbzRTaT/az/9hJnnilidzs4MTne5fMZlZ5IVmGPsDOoL27j437jtPa2c2WA+2s2d7GtuYO8nOyWF5XyV3L57JgeglL51bqTmSZFNQlxjAtHd1MLc4fujvU3bn+b18YGjrwTP7xd5Zy6+IZQHCn4198ZyPfTrpRBeCTv7qIv3oqGJto22dv4ydbW7hmfhVlBac+EDp7+inOy4582Lo7Xb0JFn/qudPe96WP38QHvvYSb7SeiMy/55o6unoTLJpVhgHPv97C86+3UF2aT2tHz9h2yBi9/6q5dPcmeHHnYS6eUUp2lpEYcGqrinmj9QQ3XzKNWxfPYFpp/tC/qy8xQM7QfgYLA6art5/e/gH2HO7iUHs3+450ceREL7PKCynJzyEx4PQPDNCXcApys+nuS7D/2EnmVBRRW1VEV0+CfUe7mF5WQHlRLgeOddPc3k1iwJlelk9fwqkqyaP9ZNC98rbmDkryc6gpLyQn2zhyopeKojymluSRFdZjZhTkZlNWkMO8qhKmFOXS1dtPbnYWA+4cPNZNV2+CvsQAZpBlxo6WDp5Yv5+Xdh6mLxH8P8vLzuKKueXcsijoo2iy95kj569JMbTnREllMBzu7GHpZ3/Mncvm8NB7LwNg8Sef5UQ48Hiy3Q/dPubt3vzFn0WGGpxM3j5vKsvqKmlp72ZqSR6zK4q4aFoJ7Sf7MIPOngRbDrQzpTCX0oIcsrOM2RWFrN9zjPV7jzLgzn/taBvXe1aV5NHW2TvissFAiVNedha9iYE3XzFJeVEux7r63nS92RWFrLh0BtfMr6amvIDZFUVpHelMZJD6ShqjfUdPArDq5X089N7L+MpPG0cMhZ9/7MZxbffHf3o9AP/881185qnhI5mO37LaCt5/1VzW7jrC42tPDYr3vQ+9kxllBXzxR9v48M0L+JNVG1i7+wi3LZ7Bs5sPUZKfw5q/uJHsLGNK4dibLN59+azT5l07vzoy3dbZQ38i6Mt/8IOvq7efLDM27DtGY0sn33xpDyX5ObR391GYl82sKYVUlebz6r5jXFhdQm62kZ1lVBbnM3NKAbMrgmu251UXU12ST2NrJ129CfJzskgMOFMKczl+so+8nCxqpxaz72gXB46dxDAumlbC/mNddPYkqCkvoLq0gNxs43BnL2bQ0tFDWUEO7jB3ahHu0NLeQ9/AAKUFORw50Uv7yX4SA05hXjaJAWcgPFG8/VAHu9pOUFNRSP+Ak5tl1FQUUpiXQ1FuNonwy1ZFUR5LL1DfRHLu0RFDqL27j8s+vXpoevdDt0euMtr0f26lJMVdDOw5fIJppQUU5gUfpD39QVfIaq8XkYmmI4YxSA4FgGc3HRp6vubPb0x5KABcMLU4Mq2OzkRkMtBF9qO475vrAPj9d9Yxd2pRmqsREYmPgoHgCqJBy2srI8s+euuCuMsREUkrBQPw0q7DQ8+/dNcVkWVFeWptE5HMok894D9ePnVlz8wphSyYXsKSuRVDl6yKiGQSBQOweNYUvrfhwND06j+5Po3ViIikl5qSgM+v3gbAD+6/Js2ViIikn4IB6OkP7nqdVa7+7UVEFAxJppbkp7sEEZG0UzCIiEiEgkFERCIUDCIiEqFgCC2rrUh3CSIik4KCAZhanMeC6aXpLkNEZFJQMAC9/QPk5WhXiIiAggGAnoSCQURkUMZ/Gro7vf0D5Gdn/K4QEQEUDEODtHf2nD6Ep4hIJsr4YHj9UDsAj/5iV5orERGZHDI+GIrDITt/bYQB70VEMlGswWBmt5nZNjNrNLMHRlj+ATPbGD7+28wun+iaEuHobTdfMm2i30pE5JwQWzCYWTbwFWAFsAi4y8wWDVttF3C9u18GfAZ4ZKLruvcbDQB84rubJvqtRETOCXEeMSwHGt19p7v3AquAO5JXcPf/dvej4eRLwOyJLuqSmWUA/MbSCX8rEZFzQpzBUAPsS5puCueN5h7gmZEWmNm9ZtZgZg2tra1nVdTtl80E4P1XzT2r7YiInC/iDAYbYZ6PuKLZjQTB8LGRlrv7I+5e7+711dXVZ1VUf3i5ak7WSOWJiGSeOMd8bgLmJE3PBg4MX8nMLgO+Bqxw98MTXVRfIhi9LVc3uImIAPEeMbwMzDezOjPLA+4EnkxewczmAk8Av+Pu2+MoavAGNwWDiEggtiMGd+83s/uB54Bs4FF332xm94XLVwKfBKYCXzUzgH53r5/IuvoHgiOGnGw1JYmIQLxNSbj708DTw+atTHr+B8AfxFnT0BFDlo4YRERAdz7Tn9ARg4hIsowPhlebjgGQrauSREQABQPPv94C6HJVEZFBGR8Md7+9lsLcbHJ0VZKICKBgoKcvQX5uxu8GEZEhsV6VNNmc6Onn6y/uSXcZIiKTSkZ/VV78qefSXYKIyKST0cEgIiKnUzCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBiAWxZNT3cJIiKThoIBeMeFU9NdgojIpKFgAJbXKRhERAZldJcY00rzmV1RyKJZZekuRURk0sjoI4bc7CzmVZekuwwRkUklo4Ph8IkeEgOe7jJERCaVjA6G7r4BvvvK/nSXISIyqWR0MIiIyOkUDCIiEqFgEBGRiIwPhruWz0l3CSIik0rGB0N1aUG6SxARmVQyNhjcg8tULc11iIhMNhkcDMFPUzKIiERkbjCEP03HDCIiEZkbDINNScoFEZGIzA2G8KdyQUQkKnODQecYRERGlLnBwGBTkpJBRCRZxgaDiIiMLGODwdXbtojIiDI2GAapJUlEJCrWYDCz28xsm5k1mtkDIyw3M/tyuHyjmS2ZqFqGTj7ruiQRkYjYgsHMsoGvACuARcBdZrZo2GorgPnh417gHyaqnu6+RFjXRL2DiMi5Kc4jhuVAo7vvdPdeYBVwx7B17gC+4YGXgHIzmzkRxbywvQWAvOyMb00TEYnIifG9aoB9SdNNwFVjWKcGOJi8kpndS3BEAdBpZtveYk1V93yOtnve4osnUBXQlu4iRjBZ64LJW5vqGh/VNT5nU9cFoy2IMxhGarQZfm3QWNbB3R8BHjnrgswa3L3+bLeTaqpr/CZrbaprfFTX+ExUXXG2ozQByaPizAYOvIV1RERkAsUZDC8D882szszygDuBJ4et8yRwd3h10tXAcXc/OHxDIiIycWJrSnL3fjO7H3gOyAYedffNZnZfuHwl8DTwLqAR6AI+OMFlnXVz1ARRXeM3WWtTXeOjusZnQuoy1y3AIiKSRNdqiohIhIJBREQiMjYY3qx7jhS/1xwz+6mZbTWzzWb24XD+p81sv5ltCB/vSnrNx8PatpnZrUnzl5rZa+GyL1sK+g03s93hNjeYWUM4r9LMfmRmO8KfFXHWZmYLk/bLBjNrN7OPpGOfmdmjZtZiZpuS5qVs/5hZvpl9K5z/SzOrPYu6/tbMXg+7lPmumZWH82vN7GTSflsZc10p+72luK5vJdW028w2pGF/jfb5kL6/MXfPuAfBye83gHlAHvAqsGgC328msCR8XgpsJ+gW5NPAR0dYf1FYUz5QF9aaHS5bC7yd4J6PZ4AVKahvN1A1bN7fAA+Ezx8APpeO2pJ+X4cIbsiJfZ8B1wFLgE0TsX+APwJWhs/vBL51FnXdAuSEzz+XVFdt8nrDthNHXSn7vaWyrmHLvwB8Mg37a7TPh7T9jWXqEcNYuudIGXc/6O7rw+cdwFaCO7pHcwewyt173H0XwVVayy3oHqTM3V/04Df8DeB/TlDZdwBfD59/Pel90lHbTcAb7r7nTeqdkLrcfQ1wZIT3S9X+Sd7Wt4GbxnJUM1Jd7r7a3fvDyZcI7gUaVVx1nUFa99eg8PW/BTx+pm1MUF2jfT6k7W8sU4NhtK43Jlx4CHcl8Mtw1v3hYf+jSYeKo9VXEz4fPv9sObDazNZZ0N0IwHQP7yEJf05LU20QfMNJ/g87GfZZKvfP0GvCD/XjwNQU1Pj7BN8aB9WZ2Stm9jMzuzbpveOqK1W/t4nYX9cCze6+I2le7Ptr2OdD2v7GMjUYxtT1Rsrf1KwE+A7wEXdvJ+g99kLgCoL+oL7wJvVNVN3vdPclBL3bfsjMrjvDurHWZsHNkO8G/jOcNVn22WjeSh0pr9HMPgH0A/8WzjoIzHX3K4E/Bf7dzMpirCuVv7eJ+J3eRfTLR+z7a4TPh1FXHeV9UlZbpgZD7F1vmFkuwS/939z9CQB3b3b3hLsPAP9E0MR1pvqaiDYNpKRudz8Q/mwBvhvW0Rwemg4ePrekozaCsFrv7s1hjZNin5Ha/TP0GjPLAaYw9qaY05jZ7wK/CnwgbFIgbHY4HD5fR9AuvSCuulL8e0v1/soB3gN8K6neWPfXSJ8PpPFvLFODYSzdc6RM2Jb3z8BWd/9i0vzkLsV/HRi8WuJJ4M7wSoI6gvEp1oaHkx1mdnW4zbuB759lbcVmVjr4nODk5aawht8NV/vdpPeJrbZQ5JvcZNhnSe+Xqv2TvK3fAJ4f/EAfLzO7DfgY8G5370qaX23BmCiY2bywrp0x1pXK31vK6grdDLzu7kPNMHHur9E+H0jn39iZzkyfzw+Crje2E3wT+MQEv9c1BIdtG4EN4eNdwL8Cr4XznwRmJr3mE2Ft20i6igaoJ/hP9QbwMOHd62dR2zyCKxxeBTYP7guC9sefADvCn5VpqK0IOAxMSZoX+z4jCKaDQB/BN697Url/gAKCprJGgqtK5p1FXY0EbcmDf2eDV6K8N/z9vgqsB34t5rpS9ntLZV3h/MeA+4atG+f+Gu3zIW1/Y+oSQ0REIjK1KUlEREahYBARkQgFg4iIRCgYREQkQsEgIiIRCgaRN2FBT5tuZpNqMPjJWpec+xQMIm9uH0EPmINdMt8QfiBXxVWAmb1gZg+fqS6RVIltzGeRc5W7Jwi6/U45M8t197638tqJrEsym44YZFIJvxl/1cz+2szaLBhY5fNmlhUu321mHx3hNQ8nTe82s0+a2WNm1mFm+8zsfWZWbmarzKzTgsFPbhljTUNNNmHvlz8NF7WG8x8L1zMz+wsze8OCQV5eM7PfHmE7d5nZ82Z2EvhDM5tqZo+bWVP4us1m9sGk1z0GXE/QwaGHj9qRmpLM7DoLBmLpNrNmM/u7sNuXMe3fcJ33WNAL6kkzO2JB76LTx7Kv5PygYJDJ6AMEPYO+A7gf+AjwvnFu4yMEt/4vAf6DoC/6fweeJujhcw3wTTMrGOd29xF0lwCwmKAp58Ph9GcJun/4EMFgKg8C/2hmtw/bxoPAV8N1vkfQXcF6go7vFgNfCl93U7j+h4EXgX8J328m0W6XATCzGoJutl8h6Lr5HoK+ph4ctuqo+9fMZhCMT/J14BKCwW3+dQz7Rc4nZ9OXjR56pPoBvAC8OGzej4Cvhc93M2wksPA1DydN7wYeT5ouIeiL5stJ82rDefVjqCmyLnBDOF2VtE4xcBK4dthr/x54eth2/mwM77lq8N880r9xlLr+L0FfOFlJ6/we0AMUjXH/Lgm3eUG6/xb0SN9D5xhkMto4bPoApwYpGfc23L3TzLoIOnEb1Bz+HO92R7OI4Jv/s2aW3AFZLkFQJWtIngh78XyA4Ft7DcGQjXkEH+LjcQnBh/5A0ryfh9u6iFP75Ez79zLaIsoAAAH+SURBVFXgx8AmM1sdPv+2u7eOsxY5hykYZDIafjLWOdXsOcDpg47kjnEbfcOmIXXNqYPb+TVg75vUcmLY9EeBPyNoMnoN6AT+mvGHljH64CvJ80fdv+6eCM+9XE3QBfs9wINmdr27vzrOeuQcpWCQc00rQRs7AOE5gosJ2tXj0hv+zE6at4WgyeYCd39+nNu7BviBu/8rDPXPvwA4Nuw9s0d4bbItwG+ZWVbSUcM14WvfGGsx7u4E5zReNLO/Iuh++n0ERxOSARQMcq55Hvh9M3uSICQ+wchHDBNpD8G37NvN7AfASXfvMLPPA58PP9jXEJzbuBoYcPdHzrC97cD7zOwaoA34Y6COaNjtJhjwvZbgiGKk0be+SnAi+atm9iWCsTYeIjg30TXC+qcxs6sJBq55jqC57UqCkb+2jOX1cn7QVUlyrnmQIBy+D6wmaENfH2cB7r4f+BTByd5mggFRAP4S+DRB09BmgpO67wV2vckmP0twBdUzBIFyglNjNQ/6PME3/y0EgTh3lLpWEHyYbwAeJRic5n+P4593HHgn8BTBADFfAD7j7t8cxzbkHKeBekREJEJHDCIiEqFgkIxnZivDu6FHeqxMd30icVNTkmQ8M5sGlI2yuN3dW+KsRyTdFAwiIhKhpiQREYlQMIiISISCQUREIhQMIiIS8f8BWDSWKZZZWYkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --first 2\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The above case without learning**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAdtUlEQVR4nO3de7hcdX3v8fdnLvuWKyEBYgISLKjRgzVGxKMoPljk0sqx9qmgVov6UFrwwVN9KtYqPtWKeOR4OYApx8NBrIr10oo2CvZQ5OkjKgG5hZsBgYQEkpD7vs7le/5Ya+/M2sxO9iaz1+wwn9fzzLNnrfWbNd+smazPrNtvKSIwMzMbVWh3AWZmNrM4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDJyCwZJ10jaLOm+CaZL0lckrZN0j6QVedVmZmZ75bnFcC1w2j6mnw4cmz7OA76aQ01mZjZObsEQEbcC2/bR5Czgukj8EpgvaXE+1ZmZ2ahSuwtosARY3zC8IR23aXxDSeeRbFUwa9asV73kJS/JpUAzs+eLO+64Y2tELGo2bSYFg5qMa9pfR0RcDVwNsHLlylizZs101mVm9rwj6fGJps2ks5I2AEc2DC8FNrapFjOzjjWTguEG4D3p2UknAjsj4lm7kczMbHrltitJ0reBk4GFkjYAlwBlgIhYBawGzgDWAQPAuXnVZmZme+UWDBFxzn6mB3BBTuWYmdkEZtKuJDMzmwEcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzs4xcg0HSaZIekrRO0sVNps+T9CNJd0taK+ncPOszM7Mcg0FSEbgSOB1YDpwjafm4ZhcA90fEK4CTgcsldeVVo5mZ5bvFcAKwLiIejYgR4HrgrHFtApgjScBsYBtQzbFGM7OOl2cwLAHWNwxvSMc1ugJ4KbARuBe4KCLq42ck6TxJaySt2bJly3TVa2bWkfIMBjUZF+OG3wLcBbwA+H3gCklzn/WiiKsjYmVErFy0aFHrKzUz62B5BsMG4MiG4aUkWwaNzgV+EIl1wO+Al+RUn5mZkW8w3A4cK2lZekD5bOCGcW2eAE4BkHQ48GLg0RxrNDPreKW83igiqpIuBG4EisA1EbFW0vnp9FXAp4FrJd1LsuvpoxGxNa8azcwsx2AAiIjVwOpx41Y1PN8InJpnTWZmluUrn83MLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhm5BoOk0yQ9JGmdpIsnaHOypLskrZX08zzrMzMzKOX1RpKKwJXAHwAbgNsl3RAR9ze0mQ9cBZwWEU9IOiyv+szMLJHnFsMJwLqIeDQiRoDrgbPGtXkn8IOIeAIgIjbnWJ+ZmZFvMCwB1jcMb0jHNToOOETSLZLukPSeZjOSdJ6kNZLWbNmyZZrKNTPrTHkGg5qMi3HDJeBVwJnAW4BPSDruWS+KuDoiVkbEykWLFrW+UjOzDpbbMQaSLYQjG4aXAhubtNkaEf1Av6RbgVcAD+dTopmZ5bnFcDtwrKRlkrqAs4EbxrX5IXCSpJKkPuA1wAM51mhm1vFy22KIiKqkC4EbgSJwTUSslXR+On1VRDwg6afAPUAd+FpE3JdXjWZmBooYv5v/4LJy5cpYs2ZNu8swMzuoSLojIlY2m+Yrn83MLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCxjUsEgqSzpKUkvm+6CzMysvSYVDBFRASo8u5tsMzN7npnKrqT/BXxMUp5ddZuZWc6mspI/CXgj8KSk+4D+xokR8dZWFmZmZu0xlWDYCnx/ugoxM7OZYdLBEBHnTmchZmY2M0z6GIOkmyXNbzJ+rqSbW1uWmZm1y1QOPp8MdDUZ30Ny/MHMzJ4H9rsrSdKKhsHjJW1rGC4CbwGebHVhZmbWHpM5xrCG5PqFAG5qMn0Q+GArizIzs/aZTDAsAwQ8Crya5OykUSPA5oioTUNtZmbWBvsNhoh4HEDS54BXR8SqxumSzpe0JCI+MU01mplZjqZy8PndwG+ajL8TeE9ryjEzs3abSjAcBmxpMn4rcHhryjEzs3abSjA8QfPTUt8AbGhNOWZm1m5T6RLjH4EvSuoCRi9oOwW4FLis1YWZmVl7TKVLjMslLQS+wt4L3UaAL0fE56ejODMzy9+UutCOiI9J+gywnOQU1vsjYs+0VGZmZm0x5XsrREQ/cPs01GJmZjOA7/lsZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWXkGgySTpP0kKR1ki7eR7tXS6pJ+pM86zMzsxyDQVIRuBI4neTK6XMkLZ+g3WXAjXnVZmZme+W5xXACsC4iHo2IEeB64Kwm7T4IfB/YnGNtZmaWyjMYlgDrG4Y3pOPGSFoCvA3I3CVuPEnnSVojac2WLc1uEWFmZs9VnsGgJuNi3PCXgI/u7x7SEXF1RKyMiJWLFi1qWYFmZvYcOtE7ABuAIxuGlwIbx7VZCVwvCWAhcIakakT8az4lmplZnsFwO3CspGXAk8DZwDsbG0TEstHnkq4FfuxQMDPLV27BEBFVSReSnG1UBK6JiLWSzk+n7/O4gpmZ5SPPLQYiYjWwety4poEQEX+eR01mZpblK5/NzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBiexyq1OhHR7jLM7CBTancBB7uRap1LbljLt3/9BN//y9fyqhcuOKD5be8f4U2X38KOgQoAS+b38uSOwX2+Zk53id3DVY5c0Mv6bc9uu3B2F31dJdZvH2A0J172grmc8V8Wc/rLj2Beb5kABCyY1YWkA/o3HAz6h6sMjNRYODv7740IRmp1Nu4YolQQT+8aoreryIJZXSya3U2p6N9SzycRQa0eFAtiqFJnYKSKJPq6ivSUi+0uL6NW3/sjr1qvM1ytM7urRKHQ+v+vOth/Ua5cuTLWrFkz7e9TqdW598mdrDjqkMz4oy/+t2e1ffeJR/H3b335lD+w63/9BBf/4N4DqrPVJDh0VhfLFs5CiGJB7BisIGB2T4mBkSrb+yvM6Slx4jGH8siWPVRrwUsXz+XpXUNU63XqAT3lIk/vGmJBXxdHzOthfl+ZOT1lSgVRqdWp1oPuUoFFc7qZ21Nmfl+Zeb1lDp/bQ7UeDFdqSKJWDx57pp/Z3SUWzu6mHsHGHYNs6x/hiW0DPLhpN5V6nR0DFQZGqmzrH6GrVGB+bxfdpQK7hips2jnEhu17A3RWV5FSsUCtHgxValTrzf9PdJUKzOstMzhSIyLo7Spy6KxuFs/vYcGsLpYe0kdBsHheDz3lIgMjNfq6ikTA4XN7WDyvh740ZKRkWe5PrR6MVOsEyd/dQ1W2D4xwSF8X8/vKDFZq1OvQUy6wZ7hK/3CNPcMVhip1KrU6xYLGailK1CMIoLtUoLtUoJCGYi2CSrXOtv4RCgUxOFJjsJK8ZnZPie5SgXrAnJ4SEXu/F33dJer1oB6BEP0jVUaqdWZ1lxip1tkzXKVcTL4zA8M1ersKDFfq7ByssH2gQrVWp1wqUK0lK7quYoHdw1UGRqrU6sGW3cOM1IJqrU53qUBPuchwtU49guFKndk9JSq1+lhtveUiAvYMJ5+9BOU0zHcNVtgzXGX7QIVaPcYeg5Va02U/p7uEBL1dRfq6kt/Qi2Z3011OvivD1WSLfM9wdWyZ1APq9WBub5nZ3SXqafCU0+9XuVRgYLhKd7lAsVBgcKTKjoEK/cNVyulnMlSpM1ytUZAYqdYZrNSo1OpUas/+Xn7g9cv4uz9cvt/vUTOS7oiIlU2nORgm56u3PMJlP32Q1yxbwHXvP4FSocBF1/+GH9+zadLz+MlFJ/HSxXMZqtS484ntvGLpfGZ1l7jutsf45A/XZtq+73XLmN9X5ucPb+GCN72Ik487jB2DFT7zb/czVKlx0SnH8eIj5nDX+h3M7Snx+DMD3PboMyw9pJc/O/GFmV/BN619ip2DFd72yiUAPLKln99t3cO3fr2eRzbvQYIN2wc5/eVH8JP7nnpW3YfN6Wbz7uEJ/11dpQIj1fqklwMk/4mm46vXWy6moVOiq1TgkL5kJbxrsMLgSI1Z3UWWHNLH0Yf2Ma+3zI6BZGVRrdep1ZMV5vy+Mi+Y38tItc7hc3uo1Ops3TPMk9sH2TlYSVYW5SQUt+4Z4aldg2zdPcJTu4amVGu5KOb0lDl0VhcAQbISGxipUpDYtHOQocrUluvBbPQ70VUq0NdVpFQosHB2EujFghip1Rmu1OlKA627VEiDp0BBsHuoylClRpB8DxbO6YaA4VqdgmB2d4nZ3SUWzOqiVBDFQvK6ub1lKrU6PeUifV3JVsKeoSrP9I8QkQTHYCUJoy27hxmpJoHbXSqgdL6FNHQLEoWCxkJIQKlYSEJayY+g3q4ilVpQrQe95QJze8rM7S1TrdUZqtTpKRfoLhWpRSTLopz8cOktFwmSAC4o+bF1/NJ5vOaYQ5/j8nYwHJBt/SOs+PTPJpz+z3/xWk5YtoBt/SOc+sVb2bpn4pXo/rzhuEVc974TnvPrp1NEsGuwyry+ctPpQ5Xa2C/LjTsGWZCu8GZ1lzLzkMRQpcaOgQq1CIoSXaUCg5Uam3YMMlyts3uowjP9I2P/Eef3lalH8gv6qAV9SMl7dJeKHD432QJZMr+XpYf0tm1XWLVWJ4BNO4YYrCQhNLpif3LHIE/vGmLXYIX+4Rq1CIarNXYOVMaWQ6mQLJeechEJDpvTw9yeEj3pyqq7VBzb4tjeP8LOwQo95WKy0qwmK5xZ3SXmdJfoKRfpKolaPdltNqenRD32Lv9KrZ6sRNP//sWiqNeDw+f2AIytJGv15BfxUKVGsSB2D1XHVuBb9ww3rJiTZd5bLtJdKtA/UqVUKDCnp0S1Xmd2d5m5vSUGR2p0lZKV4YJZXZRHV5oF0VUsUKnX6S7NrF04z1cOhikYqSa/SEbV68Exf7t6wvaL5/Vw28dOyYzbPVThP3+7lROPOZTPrn6AS976Ml5+yY37fN/vnv9aXn30gR2fMDObLAfDJP1i3Vbe+bVfAXD3J09lXl+Z0750Kw8+tXuszaOfPWMsKO6+5FTm9Tb/9dzMwEiVp3YOccyi2WPjnto5xBHzelpSv5nZZO0rGHxWUoPRUAD48Hfv4mvvffVYKHzv/Ney/AVzKRTEY5878znNv6+rlAkFwKFgZjOOgyG1Mz09dNS/P7A5c8bRSu/mMbMO4ZOyU3/5zTvaXYKZ2YyQazBIOk3SQ5LWSbq4yfR3SbonffxC0ivyqm3d5j1jz//mtBdnpt3/92/Jqwwzs7bLLRgkFYErgdOB5cA5ksZfmfE74I0RcTzwaeDqvOo7akEfABe86UX81cm/Nzb+jr9789jFLWZmnSDPNd4JwLqIeBRA0vXAWcD9ow0i4hcN7X8JLJ3uoj7/0we5+cHNYweZ//oPkq2FM49fTLVW59DZ3dNdgpnZjJJnMCwB1jcMbwBes4/27wd+0myCpPOA8wCOOuqoAyrqqlseyQyPdlNw5TtXHNB8zcwOVnkeY2h2OWrTiygkvYkkGD7abHpEXB0RKyNi5aJFi1pYopmZ5bnFsAE4smF4KbBxfCNJxwNfA06PiGems6DP/Pj+/TcyM+sweW4x3A4cK2mZpC7gbOCGxgaSjgJ+APxZRDw83QXd8vCWzPCZxy+e7rc0M5vxcttiiIiqpAuBG4EicE1ErJV0fjp9FfBJ4FDgqrQjtOpEl2y3QuMpqgCfPuvl0/VWZmYHjVzPw4yI1cDqceNWNTz/APCBPGtqdMgEvYaamXUSX/ncoBPuXGZmtj8OBjMzy3AwAC8+fA43f/iN7S7DzGxGcF8PwI3//Q3tLsHMbMbo2C2Gg/0GRWZm06Vjg+GZ/pF2l2BmNiN1bDD4/CMzs+Y6NhgKPjXVzKwpB4OZmWV0bDCYmVlzHRsM0bzHbzOzjte5weBcMDNrqnODod0FmJnNUJ0bDOkmw9tXTPttpc3MDiqdGwzp31ccOa+tdZiZzTSdGwxpMvikVTOzrM4NhtFtBl/PYGaW0bHBgLcYzMya6thgGD3G4A0GM7Oszg2GNBncNYaZWVbHBkM9TQbHgplZVscGg3clmZk117nBMLbF4GQwM2vUwcGQPnEumJlldGwwjHIumJlldWwwjF357IMMZmYZnRsM+KwkM7NmOjcY3COGmVlTnRsM6V8Hg5lZVucGg09XNTNrqnODIf3rLQYzs6zODQbf29PMrKmODYbRbQafrmpmltWxwVAf6121vXWYmc00HRsMe2/t6WQwM2uUazBIOk3SQ5LWSbq4yXRJ+ko6/R5JK6arlrEL3JwLZmYZuQWDpCJwJXA6sBw4R9Lycc1OB45NH+cBX52ueoYq9aSu6XoDM7ODVJ5bDCcA6yLi0YgYAa4HzhrX5izgukj8EpgvafF0FHPH49sB6Cp17N40M7OmSjm+1xJgfcPwBuA1k2izBNjU2EjSeSRbFAB7JD30HGta+ObL2PocXzudFoLrmqKZWpvrmhrXNTUHUtcLJ5qQZzA022sz/mqCybQhIq4Grj7ggqQ1EbHyQOfTaq5r6mZqba5ralzX1ExXXXnuR9kAHNkwvBTY+BzamJnZNMozGG4HjpW0TFIXcDZww7g2NwDvSc9OOhHYGRGbxs/IzMymT267kiKiKulC4EagCFwTEWslnZ9OXwWsBs4A1gEDwLnTXNYB746aJq5r6mZqba5ralzX1ExLXQp3GmRmZg18rqaZmWU4GMzMLKNjg2F/3XO0+L2OlPQfkh6QtFbSRen4T0l6UtJd6eOMhtd8LK3tIUlvaRj/Kkn3ptO+ohZ0DyvpsXSed0lak45bIOlnkn6b/j0kz9okvbhhudwlaZekD7VjmUm6RtJmSfc1jGvZ8pHULek76fhfSTr6AOr6H5IeTLuU+RdJ89PxR0sabFhuq3Kuq2WfW4vr+k5DTY9JuqsNy2ui9UP7vmMR0XEPkoPfjwDHAF3A3cDyaXy/xcCK9Pkc4GGSbkE+BXykSfvlaU3dwLK01mI67dfAa0mu+fgJcHoL6nsMWDhu3OeBi9PnFwOXtaO2hs/rKZILcnJfZsAbgBXAfdOxfIC/Alalz88GvnMAdZ0KlNLnlzXUdXRju3HzyaOuln1uraxr3PTLgU+2YXlNtH5o23esU7cYJtM9R8tExKaIuDN9vht4gOSK7omcBVwfEcMR8TuSs7ROUNI9yNyIuC2ST/g64L9NU9lnAV9Pn3+94X3aUdspwCMR8fh+6p2WuiLiVmBbk/dr1fJpnNf3gFMms1XTrK6IuCkiqungL0muBZpQXnXtQ1uX16j09X8KfHtf85imuiZaP7TtO9apwTBR1xvTLt2EeyXwq3TUhelm/zUNm4oT1bckfT5+/IEK4CZJdyjpbgTg8EivIUn/Htam2iD5hdP4H3YmLLNWLp+x16Qr9Z3AoS2o8X0kvxpHLZP0G0k/l3RSw3vnVVerPrfpWF4nAU9HxG8bxuW+vMatH9r2HevUYJhU1xstf1NpNvB94EMRsYuk99gXAb9P0h/U5fupb7rqfl1ErCDp3fYCSW/YR9tca1NyMeRbge+mo2bKMpvIc6mj5TVK+jhQBb6ZjtoEHBURrwT+GviWpLk51tXKz206PtNzyP74yH15NVk/TNh0gvdpWW2dGgy5d70hqUzyoX8zIn4AEBFPR0QtIurA/ybZxbWv+jaQ3TXQkrojYmP6dzPwL2kdT6ebpqObz5vbURtJWN0ZEU+nNc6IZUZrl8/YaySVgHlMflfMs0h6L/CHwLvSXQqkux2eSZ/fQbJf+ri86mrx59bq5VUC/hj4TkO9uS6vZusH2vgd69RgmEz3HC2T7sv7P8ADEfE/G8Y3din+NmD0bIkbgLPTMwmWkdyf4tfp5uRuSSem83wP8MMDrG2WpDmjz0kOXt6X1vDetNl7G94nt9pSmV9yM2GZNbxfq5ZP47z+BLh5dIU+VZJOAz4KvDUiBhrGL1JyTxQkHZPW9WiOdbXyc2tZXak3Aw9GxNhumDyX10TrB9r5HdvXkenn84Ok642HSX4JfHya3+v1JJtt9wB3pY8zgG8A96bjbwAWN7zm42ltD9FwFg2wkuQ/1SPAFaRXrx9AbceQnOFwN7B2dFmQ7H/8f8Bv078L2lBbH/AMMK9hXO7LjCSYNgEVkl9e72/l8gF6SHaVrSM5q+SYA6hrHcm+5NHv2eiZKG9PP9+7gTuBP8q5rpZ9bq2sKx1/LXD+uLZ5Lq+J1g9t+465SwwzM8vo1F1JZmY2AQeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmO2Hkp42Q9KMuhn8TK3LDn4OBrP9W0/SA+Zol8wnpyvkhXkVIOkWSVfsqy6zVsntns9mB6uIqJF0+91yksoRUXkur53OuqyzeYvBZpT0l/FVkj4raauSG6t8QVIhnf6YpI80ec0VDcOPSfqkpGsl7Za0XtI7JM2XdL2kPUpufnLqJGsa22WT9n75H+mkLen4a9N2kvQ3kh5RcpOXeyW9u8l8zpF0s6RB4C8kHSrp25I2pK9bK+nchtddC7yRpIPDSB9HN9uVJOkNSm7EMiTpaUlfTLt9mdTyTdv8sZJeUAclbVPSu+jhk1lW9vzgYLCZ6F0kPYP+V+BC4EPAO6Y4jw+RXPq/Avhnkr7ovwWsJunh81bgnyT1THG+60m6SwB4GcmunIvS4c+QdP9wAcnNVC4F/lHSmePmcSlwVdrmX0m6K7iTpOO7lwFfTl93Str+IuA24P+m77eYbLfLAEhaQtLN9m9Ium5+P0lfU5eOazrh8pV0BMn9Sb4OvJTk5jbfmMRyseeTA+nLxg8/Wv0AbgFuGzfuZ8DX0uePMe5OYOlrrmgYfgz4dsPwbJK+aL7SMO7odNzKSdSUaQucnA4vbGgzCxgEThr32i8Bq8fN58OTeM/rR//Nzf6NE9T1DyR94RQa2vw5MAz0TXL5rkjn+cJ2fxf8aN/DxxhsJrpn3PBG9t6kZMrziIg9kgZIOnEb9XT6d6rznchykl/+P5XU2AFZmSSoGq1pHEh78byY5Ff7EpJbNnaRrMSn4qUkK/16w7j/TOf1e+xdJvtavncD/w7cJ+mm9Pn3ImLLFGuxg5iDwWai8Qdjg727Pes8+6Yj5UnOozJuGFq3O3V0Pn8EPLGfWvrHDX8E+DDJLqN7gT3AZ5l6aImJb77SOH7C5RsRtfTYy4kkXbC/H7hU0hsj4u4p1mMHKQeDHWy2kOxjByA9RvASkv3qeRlJ/xYbxt1PssvmhRFx8xTn93rgRxHxDRjrn/84YMe49yw2eW2j+4E/lVRo2Gp4ffraRyZbTEQEyTGN2yT9PUn30+8g2ZqwDuBgsIPNzcD7JN1AEhIfp/kWw3R6nORX9pmSfgQMRsRuSV8AvpCu2G8lObZxIlCPiKv3Mb+HgXdIej2wFfggsIxs2D1GcsP3o0m2KJrdfesqkgPJV0n6Msm9Nj5HcmxioEn7Z5F0IsmNa24k2d32SpI7f90/mdfb84PPSrKDzaUk4fBD4CaSfeh35llARDwJXEJysPdpkhuiAHwC+BTJrqG1JAd13w78bj+z/AzJGVQ/IQmUfvbeq3nUF0h++d9PEohHTVDX6SQr87uAa0huTvO3U/jn7QReB/yY5AYxlwOfjoh/msI87CDnG/WYmVmGtxjMzCzDwWAdT9Kq9GroZo9V7a7PLG/elWQdT9JhwNwJJu+KiM151mPWbg4GMzPL8K4kMzPLcDCYmVmGg8HMzDIcDGZmlvH/AUxWg8EybDBqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --first 2\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon, do_learn=False)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using the cb_explore_adf contextual bandit algorithm and exploring with the bagging explorer algorithm with m = 5 and plotting ctr vs num_iterations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhc9X3v8fdXq23JmzYv8iLZxhs4xkYGg1nrJGCTYJrmFkhSEpo8hKfQJ2mbe0OaZrlNGpLe0DS5hFJuSmlIA7QkIYaYAAkQQsCAbbyDbXmXbMuSF3mTrWW+949zZObII2PZozOy5/N6nnk0Z5lzvj4zns+c3znnd8zdERER6ZST6QJERKRvUTCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhExBYMZvaQme02s9XdTDcz+4GZ1ZrZSjObGVdtIiLyrjj3GB4GrjvJ9HnAeeHjduBfYqhJRES6iC0Y3P1lYO9JZlkA/NgDi4EhZjYinupERKRTXqYLSFIJbE8argvH7ew6o5ndTrBXQVFR0UWTJ0+OpUARkXPF0qVLm9y9PNW0vhQMlmJcyv463P1B4EGAmpoaX7JkSW/WJSJyzjGzrd1N60tnJdUBo5OGRwE7MlSLiEjW6kvBsBC4NTw7aTbQ7O4nNCOJiEjviq0pycweBa4GysysDvgakA/g7g8Ai4D5QC1wBLgtrtpERORdsQWDu9/yHtMduDOmckREpBt9qSlJRET6AAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkItZgMLPrzGydmdWa2d0ppg82s6fMbIWZrTGz2+KsT0REYgwGM8sFfgjMA6YCt5jZ1C6z3QmsdffpwNXAvWZWEFeNIiIS7x7DxUCtu29y91bgMWBBl3kcGGhmBhQDe4H2GGsUEcl6cQZDJbA9abguHJfsPmAKsANYBXzO3RNdF2Rmt5vZEjNb0tjY2Fv1iohkpTiDwVKM8y7D1wLLgZHAhcB9ZjbohBe5P+juNe5eU15env5KRUSyWJzBUAeMThoeRbBnkOw24OceqAU2A5Njqk9ERIg3GN4EzjOz6vCA8s3Awi7zbAPmApjZMGASsCnGGkVEsl5eXCty93Yzuwt4FsgFHnL3NWZ2Rzj9AeAbwMNmtoqg6emL7t4UV40iIhJjMAC4+yJgUZdxDyQ93wF8MM6aREQkSlc+i4hIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREImINBjO7zszWmVmtmd3dzTxXm9lyM1tjZr+Lsz4REYG8uFZkZrnAD4EPAHXAm2a20N3XJs0zBLgfuM7dt5lZRVz1iYhIIM49houBWnff5O6twGPAgi7zfAz4ubtvA3D33THWJyIixBsMlcD2pOG6cFyyicBQM3vJzJaa2a2pFmRmt5vZEjNb0tjY2EvliohkpziDwVKM8y7DecBFwPXAtcBXzGziCS9yf9Dda9y9pry8PP2ViohksdiOMRDsIYxOGh4F7EgxT5O7HwYOm9nLwHRgfTwlikicjrZ1sKHhEGt2NLOyvpnzRw5i3gUjKCkqyHRpWS3OYHgTOM/MqoF64GaCYwrJfgncZ2Z5QAFwCfC9GGsUkV7U3pFgVX0zr27cw5Ite3lt0x6OtiUA6Jefw09fT/D1hWt4/5Rh3HppFZdUl5CTk6qxQXpTbMHg7u1mdhfwLJALPOTua8zsjnD6A+7+tpn9GlgJJIAfufvquGoUkfRq70iwom4/v9/QxLJt+1m2dR+HjrUDMK68iJtnjeGS6hImVBQzoaKY1fUH+OXyev57aR3PrN7FqKH9+cTssXxkZiUVA/tl+F+TPcy9azP/2aWmpsaXLFmS6TJEBDjS2s7SrftYvGkPq+oP8Na2fRw82o4ZTBo2kBljhjJnQimzx5VSVlzY7XJaWjv49ZqdPPbGdl7fvJccg2smVfDh6SOZO6WCgf3yY/xXnZvMbKm716ScpmAQkTOxfe8Rfr16F6/UNvHqxibaOpzcHKO6rIhZVUOZM6GMKyaUM3jA6X2Zb2w8xH8vqeMXb9XRcOAY/fNzuXHGSBZcWMmsqhJy1dR0WhQMIpJWdfuO8ORb9Ty3toGVdc1A0DQ0d3IFl40v45JxJQwoSG9LdSLhLNu2j/9asp2nVuykpa2DioGF3DRrNAsurGRCRXFa13euUzCIyBmr3X2IF95p4Ddrd/PGlr0ATB89hOvOH87100YwpnRAbLUcPtbOi+t287Oldby0vhF3mDpiEPOnDeeG6ZWx1nK2UjCIyGnZ3HSYZ1bv5Nerdx3fM5g0bCAfnj6iz3wBNxw4yq9W7uTJ5fWsqm/GHS6uLuFPZlYyb9oIBul4REoKBhE5ZQ0HjvLUih08sbSOd3YdBOCCykH88YxRXHv+MEYNzXwYdGdncws/X1bPz5bVsanxMAV5OVw9sZyPXjSKS8eX6qB1EgXDSRxr76AwLzeNFYmcXdydt3ce5JXaRp5d08DSrfsAuHD0EG6YPpJrLxhO5ZD+Ga6yZ9yd5dv3s3DFDp5asZOmQ8fIzzUuG1/Gh943gvnTRlBUGOdlXH2PgqEbL67bzW3//ia/vHMO00cPSXNlIn1XIuEsr9vPM6t28us1u9i+twWAycMHMn/aCOZPG86EioEZrjI92joSLN60h5fXNx7/txYV5HLlxHI+eP4wrplUwZAB2Xel9cmCIasj8+X1QQd8b27Zq2CQc17nnsEji7fw/NoGmg61kp9rzJlQxl3XTOCqiRUMH3zuXUSWn5vDFeeVc8V55fzt/Cks3bqPJ5bW8dK6Rp5ZvYvcHGP2uBJuvLCS6y4YruYmsjwYtu45AsD6hoMZrkSk97yz6wCPvLaV329oYtveI/TPz2XulArmTqngjyYNO+3rC85GZkZNVQk1VSXH95p+s7aBX63ayf98YiV/9+Rq5k6p4PppI7l6UnnWNjedUlOSmeUTdJk9193X9HpVPXAmTUlVd//q+PMt374+XSWJZNzO5hYee2M7T6/cwcbGw/TLz2HO+DKuOK+MG2dUZmXTycm4B9dIPPnWDp5ZvZOmQ630z8/l6klBc9P7pww75/Ykzrgpyd3bzKyNE7vJPifoykk5Fxxt6+DplTt59I1trNi+nw53ZleX8onZY7nxwkqGqsfSbpkZF40t4aKxJXztw1NZsnUfT63YwfNrG3hm9S4K83KYVVXCNZMr+KPJFVSXFWW65F51ygefzex/AdOA29y9vVer6oF07DGUFhWw9CsfSGdZIrFpPHiMH7+2hUcWb2X/kTbGlxdx7fnDueXiMYwu6bunlp4NEgnnre37eGrFTl7d2MT6hkMAVA7pzwemDmPulAouqS6lIC/OW9ukR7oOPl8BXAXUm9lq4HDyRHe/4fRLzKw9h1szXYJIj7g7r2/ey+NvBs1F7Qln7uRh/PmcKi4dX4qZ9oLTISfn3T0JCPqFenHdbn63rpFH39jGw69uYWC/PK6cWM7ccG/iXGim60kwNAE/661CMuGDU4fx3NqGTJcRu45E0MmZu9Pc0tbtBzmRcL7/2w389p0G1u44wMgh/emfn8uYkgHsPdLKpy+v5ppJFRTm5bD3SCv3Prue84YVU5ify9iSATQcOMrew60s3rSHlrYOJg8fxLBB/SgpyifHjIurSygrLqR/fi5v7zrA+PJiCnJzqN/fQv3+FoYN6kd7R4JDx9opLsxjf0sb9ftaGDG4H6XFBTQcOMaBljZmjBkaOZvG3WnrcHYfPMrQAQUca08wsF8e+bnRX3UHj7axqr6Zo2FtHQmnpa2DvByjMD+X3QeOYma0dyQYV15Ma3uCioGFGb0/QEtrB08ur+ehVzazYfchBvbL45aLx/Cpy6oYV66+gnrb6JIB3HppFbdeWkVLawd/qG3iubW7eGldI79auZPcHGP6qMHMqi7hsvFlzKoamvY+o+KQ1dcxfOXJ1TyyeCuQmYPPiYRz/0u13HHVeFraOk754FZHwnlm9U7+6vHllBcXsqP5KOPKijh4rJ3Gg8eA4OKk5dv397im940azJThg/jd+kZ2HTja49dnSllxAQW5OXS403jwGIkuH+v8XKNfXi65ucbQAQXsaj5KS1tHj9djBsUFeZQWF5Bw6J+fy6D+efTLz6W1PbjhzNH2BJt2H2JQ/3yuOC/oUO5Mz5Wv3X2Qnyzexi/eqqe5pY0LKgfxiUvGcuOMSvrl6wLNTEsknJX1zfxmbQN/2NjE6vpm2jqcgtwcZlUP5ZpJFcyZUMbk4QP7zN5cWi5wM7MXgI+4+/4u4wcBT7r7H51xpafhdIPh8LF2zv/aswDMnzac+z9+UbpLO6m9h1uZ+Y3nU04rzMvhWPglk2kzxwzh+veN5CMzKjl4tJ2XNzQyZcRA1jccYsmWoN/95pY2Flw4ktKiAi6bUMaG3Yd4Ymkdl08o5f1ThjFySH8qBhZypLWD329oJD83h3UNB3nyrXqqSosY1D+fvYdbGTW0P7uaj1I+sJDqsiLaOpzCvBz6F+RiQN2+FqaOHERhXg67Dx5j6IB8+uXnsqHhEGt3HqC1I0GuGaXFBRTk5TBiUD/2Hm7FzGhuaaO1I0Fbe4IDR9soLS6ktKiAaZWDKe6Xx+r6ZgrzcinMyyHhwX0Fhg/uR16OkWPG6h0HGNw/n8aDxzjS2s7WPUco7pdHa3uC/UdaOXSsg+LCXFo7nP75OVSXFdN06Bh/qG3iSGsHOQbjy4upqRpKzdgS5k0b/p6/JN2dNzbv5d9e2cxzaxvIyzHmTRvBJy4Zw8XVJX3mC0ZO1NLaweub9/CH2iZeWtfIht3vHpu4dHwpcyaUMmP0UMaWDsjY+5iuYEgAw919d5fxFUC9u2fkXK7TDYZfrdzJnT9dBkBejlH7rfmntf6/fnw5P3+rnmVf+cAJ96ldXd/Mh/7vK/zvG87nk5dVAdDanmDi3z1zWuvqzotfuJqnV+zg3ufX840bL2DG6CEs3rSHAy1tfGpONSVFBbg7Znb8b1eJ8Cf28rr97Go+ysRhxefMla+Z1JFwVtbt57dv72ZF3X6Wb9vPwWPt5OUYl44v5boLhlMztoTqsqLjBzCPtnWwcMUOfrJ4KyvrmhkyIJ8/mz2WT15WddKb20jftWN/C69saOKFd3bzWvhjCqB8YCFzxpcyZ0IZl00oi7XrkTMKBjObGT5dAnwQ2Js0ORe4FviMu1edeak9l45ggFNrStqxv4XW9gSjSwYcb6Ov/tKi49Mfv302k4YP5Mp/fJGpIwexeNPekywtUPsP88gL277dnW8/8w7/+vImvrHgfArzchlfUcyFo4fw7JpdtHUkmD9tBAbHXyNnF3fnD7V7+P2GRp5euZP6/UFXFEUFuVw1qZzSokKeWrnj+NlFn5pTzUdnjqJ/gZqLzhUdCWfdroMs376fxZv28OrGJpoOBSfAjC0dwNUTy7lkXCmXVJdQ2os/BM40GBK8e/1Cqn2eFuAv3f2hM6ryNJ1uMCxatZO/+M9TD4ZEwhn3t4tOOk9PbPzWfF0/keXcnY2Nh1i+vZklW/bywju72Xu4lesuGM7HLh6js4uyRCLhrGs4yKsbg6anVzc2cbQtaEqeUFHMrKqhTKscwvTRg5k4bOAJJ1GcrjM9XbWaIBA2AbMIzk7q1ArsdveeH8XrwxoPHqN84LtJ3d6RYMKXu2/+uWH6SBau2JFy2pZvX8/3nl/P93+7gfs+NoP5F4zI6Fkt0neYGRMqBjKhYiAfvWgU7k5rR0K9/WaZnBxjyohBTBkxiE9fXk1bR4KVdc28vnkPb2zey69W7uTRN7YD0C8/h+mjhnDJuFJmjB7CzLFDGdw//a347xkM7r4VwMy+Dcxy9weSp5vZHWZW6e5fSXt1GfBqbRMf+9Hr/OjWGt4/dRgAH/vR693Ov/me+ZgZf3bpWKZVDmb3gWMn3Lzkrz4wkb/6wMRerVvOfmamUBDyc3O4aOxQLho7lL+4Otiz3LLnCKvqm3lr2z7e3LKX+17YQMLhtjlVfO3D56e9hp6cYPsJ4H+kGL8M+BJwVgVD19/s7R0JjrUnjofAZ368hC3fvp6ORHBmSKeN35rPE0u3M3xwfyqH9D++qz+rKrgApi/c0UpEzh1mRnVZEdVlRdwwfSQQnFW5sq6ZsuLeuZiuJ8FQATSmGN8EDEtPOfHp2nTb2pE4fvpqp58s3srfPbn6+HDncYibZo3p9fpERLpTVJjHpeNLe235PTmKsY2gW4yurgTq0lNO5kz96rMnjEsOhS/NmxxnOSIiGdOTPYZ/Bb5nZgXAC+G4ucA9wHfSXVhf89mrxme6BBGRWJxyMLj7vWZWBvwA6GzYagW+7+7/2BvF9a7uzwz66oem8vdPrz0+/L5Rg+MoSESkT+hR707u/iUz+yYwleCbda27H+qVyjLo47PHsKJuP3deM4EhA/Ip19WmIpJFetztn7sfBt7shVr6hG/eeAGFebl8/+YZmS5FRCQjsrZfhe4uKO3NI/0iImeDrA2GVD575TjGneO37BMReS9n3x0k0iR5h+EjMyr57FXjmTRcvYmKiGRtMCT7p5suzHQJIiJ9hpqSREQkImuDQd0Zi4iklrXBICIiqcUaDGZ2nZmtM7NaM7v7JPPNMrMOM/tonPWJiEiMwWBmucAPgXkEV07fYmZTu5nvO8CJvdqls57eXLiIyFkszj2Gi4Fad9/k7q3AY8CCFPP9JfAzYHeMtYmISCjOYKgEticN14XjjjOzSuCPgchd4roys9vNbImZLWlsTHWLCBEROV1xBkOq1hvvMvzPwBff6x7S7v6gu9e4e015efnpFaO2JBGRlOK8wK0OGJ00PArY0WWeGuCx8FTSMmC+mbW7+5PxlCgiInEGw5vAeWZWDdQDNwMfS57B3as7n5vZw8DTCgURkXjFFgzu3m5mdxGcbZQLPOTua8zsjnD6SY8rpJuakkREUou1ryR3XwQs6jIuZSC4+6d6sxbTCasiIinpymcREYlQMIiISET2BoNakkREUsreYBARkZQUDCIiEpG1waCWJBGR1LI2GLr2xSEiIoGsDQYlg4hIatkbDCIiklLWBoNrl0FEJKXsDQblgohISgoGERGJyN5gyHQBIiJ9VPYGg3YZRERSytpgEBGR1LI2GIoKg1tRTKgoznAlIiJ9S9YGQ7/8XAC+PH9KhisREelbsjYYOuXkqNckEZFkWRwMOvgsIpJKFgdDQPsLIiJRWR8MIiISlbXBoMsYRERSy9pg6GRqSxIRicj6YBARkaisDQa1JImIpJa1wdDJdF6SiEhE1geDiIhEZW0w6KwkEZHUsjYYOumsJBGRqKwPBhERicraYNCNekREUsvaYOikliQRkaisDwYREYlSMIiISESswWBm15nZOjOrNbO7U0z/uJmtDB+vmtn03qpFRxhERFKLLRjMLBf4ITAPmArcYmZTu8y2GbjK3d8HfAN4sPcL6/U1iIicVeLcY7gYqHX3Te7eCjwGLEiewd1fdfd94eBiYFSM9YmICPEGQyWwPWm4LhzXnU8Dz6SaYGa3m9kSM1vS2Nh4WsXobFURkdTiDIZUjTYpv57N7BqCYPhiqunu/qC717h7TXl5+RkWpbYkEZFkeTGuqw4YnTQ8CtjRdSYzex/wI2Ceu++JqTYREQnFucfwJnCemVWbWQFwM7AweQYzGwP8HPgzd1/fm8W4zksSEUkptj0Gd283s7uAZ4Fc4CF3X2Nmd4TTHwC+CpQC91vQu127u9f0Zl3qRE9EJCrOpiTcfRGwqMu4B5Kefwb4TJw1iYhIVPZe+ayWJBGRlLI3GEJqSRIRicr6YBARkaisDQa1JImIpJa1wdDJdFqSiEhE1geDiIhEZW0wqK8kEZHUsjYYOqklSUQkKuuDQUREorI2GNRXkohIalkbDJ3UkiQiEpX1wSAiIlFZGww6K0lEJLWsDYZOOitJRCQq64NBRESisjYY1JIkIpJa1gbDu9SWJCKSTMEgIiIRCgYREYnI2mBwna8qIpJS1gZDJ52uKiISlfXBICIiUVkbDGpIEhFJLWuDoZNakkREorI+GEREJCp7g0FtSSIiKWVvMIRMpyWJiERkfTCIiEhU1gaDbu0pIpJa1gZDJzUkiYhEZX0wiIhIVNYGg7pKEhFJLWuDoZNOShIRicr6YBARkaisDQY1JYmIpBZrMJjZdWa2zsxqzezuFNPNzH4QTl9pZjN7vSadlyQiEhFbMJhZLvBDYB4wFbjFzKZ2mW0ecF74uB34l96q51h7orcWLSJyVotzj+FioNbdN7l7K/AYsKDLPAuAH3tgMTDEzEb0RjEbGw8BkJ+nPQYRkWR5Ma6rEtieNFwHXHIK81QCO5NnMrPbCfYoAA6Z2brTrKlsyndoOs3X9qYyUF091FdrU109o7p65kzqGtvdhDiDIdVP866HgE9lHtz9QeDBMy7IbIm715zpctJNdfVcX61NdfWM6uqZ3qorzqakOmB00vAoYMdpzCMiIr0ozmB4EzjPzKrNrAC4GVjYZZ6FwK3h2UmzgWZ339l1QSIi0ntia0py93Yzuwt4FsgFHnL3NWZ2Rzj9AWARMB+oBY4At/VyWWfcHNVLVFfP9dXaVFfPqK6e6ZW6zHWll4iIJMnaK59FRCQ1BYOIiERkbTC8V/ccaV7XaDN70czeNrM1Zva5cPzXzazezJaHj/lJr/lSWNs6M7s2afxFZrYqnPYDS8NNq81sS7jM5Wa2JBxXYmbPm9mG8O/QOGszs0lJ22W5mR0ws89nYpuZ2UNmttvMVieNS9v2MbNCM3s8HP+6mVWdQV3/x8zeCbuU+YWZDQnHV5lZS9J2eyDmutL2vqW5rseTatpiZsszsL26+37I3GfM3bPuQXDweyMwDigAVgBTe3F9I4CZ4fOBwHqCbkG+DnwhxfxTw5oKgeqw1txw2hvApQTXfDwDzEtDfVuAsi7j/hG4O3x+N/CdTNSW9H7tIrggJ/ZtBlwJzARW98b2Af4CeCB8fjPw+BnU9UEgL3z+naS6qpLn67KcOOpK2/uWzrq6TL8X+GoGtld33w8Z+4xl6x7DqXTPkTbuvtPdl4XPDwJvE1zR3Z0FwGPufszdNxOcpXWxBd2DDHL31zx4h38M3NhLZS8A/iN8/h9J68lEbXOBje6+9T3q7ZW63P1lYG+K9aVr+yQv6wlg7qns1aSqy92fc/f2cHAxwbVA3YqrrpPI6PbqFL7+T4FHT7aMXqqru++HjH3GsjUYuut6o9eFu3AzgNfDUXeFu/0PJe0qdldfZfi86/gz5cBzZrbUgu5GAIZ5eA1J+LciQ7VB8Asn+T9sX9hm6dw+x18Tfqk3A6VpqPHPCX41dqo2s7fM7HdmdkXSuuOqK13vW29sryuABnffkDQu9u3V5fshY5+xbA2GU+p6I+0rNSsGfgZ83t0PEPQeOx64kKA/qHvfo77eqnuOu88k6N32TjO78iTzxlqbBRdD3gD8dziqr2yz7pxOHWmv0cy+DLQD/xmO2gmMcfcZwF8DPzWzQTHWlc73rTfe01uI/viIfXul+H7odtZu1pO22rI1GGLvesPM8gne9P90958DuHuDu3e4ewL4fwRNXCerr45o00Ba6nb3HeHf3cAvwjoawl3Tzt3n3ZmojSCslrl7Q1hjn9hmpHf7HH+NmeUBgzn1ppgTmNkngQ8BHw+bFAibHfaEz5cStEtPjKuuNL9v6d5eecBHgMeT6o11e6X6fiCDn7FsDYZT6Z4jbcK2vH8D3nb3f0oan9yl+B8DnWdLLARuDs8kqCa4P8Ub4e7kQTObHS7zVuCXZ1hbkZkN7HxOcPBydVjDJ8PZPpm0nthqC0V+yfWFbZa0vnRtn+RlfRR4ofMLvafM7Drgi8AN7n4kaXy5BfdEwczGhXVtirGudL5vaasr9H7gHXc/3gwT5/bq7vuBTH7GTnZk+lx+EHS9sZ7gl8CXe3ldlxPstq0EloeP+cAjwKpw/EJgRNJrvhzWto6ks2iAGoL/VBuB+wivXj+D2sYRnOGwAljTuS0I2h9/C2wI/5ZkoLYBwB5gcNK42LcZQTDtBNoIfnl9Op3bB+hH0FRWS3BWybgzqKuWoC2583PWeSbKn4Tv7wpgGfDhmOtK2/uWzrrC8Q8Dd3SZN87t1d33Q8Y+Y+oSQ0REIrK1KUlERLqhYBARkQgFg4iIRCgYREQkQsEgIiIRCgaR92BBT5tuZn3qZvB9tS45+ykYRN7bdoIeMDu7ZL46/EIui6sAM3vJzO47WV0i6RLbPZ9Fzlbu3kHQ7XfamVm+u7edzmt7sy7JbtpjkD4l/GV8v5l9y8yaLLixynfNLCecvsXMvpDiNfclDW8xs6+a2cNmdtDMtpvZTWY2xMweM7NDFtz85IOnWNPxJpuw98sXw0mN4fiHw/nMzP6XmW204CYvq8zsEymWc4uZvWBmLcBnzazUzB41s7rwdWvM7Lak1z0MXEXQwaGHj6pUTUlmdqUFN2I5amYNZva9sNuXU9q+4TwfsaAX1BYz22tB76LDTmVbyblBwSB90ccJega9DLgL+DxwUw+X8XmCS/9nAv9F0Bf9T4FFBD18vgz8xMz69XC52wm6SwA4n6Ap53Ph8DcJun+4k+BmKvcA/2pm13dZxj3A/eE8TxJ0V7CMoOO784Hvh6+bG87/OeA14N/D9Y0g2u0yAGZWSdDN9lsEXTd/mqCvqXu6zNrt9jWz4QT3J/kPYArBzW0eOYXtIueSMzAD7h0AAALDSURBVOnLRg890v0AXgJe6zLueeBH4fMtdLkTWPia+5KGtwCPJg0XE/RF84OkcVXhuJpTqCkyL3B1OFyWNE8R0AJc0eW1/wws6rKcvzmFdT7W+W9O9W/spq5/IOgLJydpnk8Bx4ABp7h9Z4bLHJvpz4IemXvoGIP0RSu7DO/g3ZuU9HgZ7n7IzI4QdOLWqSH829PldmcqwS//X5tZcgdk+QRBlWxJ8kDYi+fdBL/aKwlu2VhA8CXeE1MIvvQTSeNeCZc1gXe3ycm27wrgN8BqM3sufP6Euzf2sBY5iykYpC/qejDWebfZM8GJNx3JP8VltHUZhvQ1p3Yu58PAtveo5XCX4S8Af0PQZLQKOAR8i56HltH9zVeSx3e7fd29Izz2MpugC/ZPA/eY2VXuvqKH9chZSsEgZ5tGgjZ2AMJjBJMJ2tXj0hr+zU0at5agyWasu7/Qw+VdDjzl7o/A8f75JwL7u6wzN8Vrk60F/tTMcpL2Gi4PX7vxVItxdyc4pvGamf09QffTNxHsTUgWUDDI2eYF4M/NbCFBSHyZ1HsMvWkrwa/s683sKaDF3Q+a2XeB74Zf7C8THNuYDSTc/cGTLG89cJOZXQ40AX8JVBMNuy0EN3yvItijSHX3rfsJDiTfb2bfJ7jXxrcJjk0cSTH/CcxsNsGNa54laG6bQXDnr7Wn8no5N+isJDnb3EMQDr8EniNoQ18WZwHuXg98jeBgbwPBDVEAvgJ8naBpaA3BQd0/ATa/xyK/SXAG1TMEgXKYd+/V3Om7BL/81xIE4phu6ppH8GW+HHiI4OY0f9uDf14zMAd4muAGMfcC33D3n/RgGXKW0416REQkQnsMIiISoWCQrGdmD4RXQ6d6PJDp+kTipqYkyXpmVgEM6mbyAXffHWc9IpmmYBARkQg1JYmISISCQUREIhQMIiISoWAQEZGI/w/U/E0rFHFLEgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --bag 5\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The above case without learning**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeiklEQVR4nO3deZRedZ3n8ffnWWpPkYSELaAEBTEqKgZbbEE82Aq4MC5Hoe1G0TkMp8XWGT0NtuMyrd24d+sgTTOKqN2CM6224KDoiEIrqIR9azBCICFIEhKyVFWq6nme7/xxb4XnVm4lVUnVrQr38zrnOXnu/uU+xf3c9XcVEZiZmY2pzHYBZmY2tzgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMgoLBkmXSVon6e4JhkvSlyWtlHSnpGOLqs3MzJ5S5BHD5cApuxh+KnBk+jkH+McCajIzs3EKC4aIuAHYuItRTge+GYlfA/MlHVxMdWZmNqY22wW0WQKsbutek/Z7bPyIks4hOaqgt7f3JUcfffSUF7Z5aJRHNg5y5AHz6Kr7UouZlcstt9yyISIW5w2bS8GgnH657XVExKXApQDLly+PFStWTHlh//fOx3jvt2/le//1RI46cN6Upzcz25dJeniiYXNpV3kNcFhb96HA2lmqxcystOZSMFwFnJXenfQyYHNE7HQayczMZlZhp5IkXQGcBCyStAb4OFAHiIhLgGuA04CVwCBwdlG1mZnZUwoLhog4czfDA3hvQeW0LbfoJZqZzW1z6VRSoZR3qdvMzMobDGZmls/BYGZmGQ4GMzPLKH0wRP4zdGZmpVXaYPC1ZzOzfKUNBjMzy+dgMDOzDAeDmZllOBjMzCyj9MHgJjHMzLJKGwxuEsPMLF9pg8HMzPI5GMzMLMPBYGZmGaUPBl98NjPLKnEw+OqzmVmeEgeDmZnlcTCYmVmGg8HMzDIcDGZmllH6YPCLeszMskobDG4Sw8wsX2mDwczM8jkYzMwsw8FgZmYZpQ8GN4lhZpZV2mDwtWczs3ylDQYzM8vnYDAzswwHg5mZZTgYzMwsw8FgZmYZpQ0GuU0MM7NcpQ0GMzPLV2gwSDpF0v2SVkq6IGf4fpKulnSHpHsknV1kfWZmVmAwSKoCXwFOBZYBZ0paNm609wL3RsQLgZOAL0jqKKpGMzMr9ojhpcDKiHgwIkaAK4HTx40TwDwlFwD6gI1AYyaLcpMYZmZZRQbDEmB1W/eatF+7i4DnAmuBu4D3R0Rr/IwknSNphaQV69ev36NifOnZzCxfkcGQty0ev7/+WuB24BDgRcBFkvp3miji0ohYHhHLFy9ePP2VmpmVWJHBsAY4rK37UJIjg3ZnA9+LxErgIeDoguozMzOKDYabgSMlLU0vKJ8BXDVunEeAkwEkHQg8B3iwwBrNzEqvVtSCIqIh6TzgWqAKXBYR90g6Nx1+CfBJ4HJJd5Gcejo/IjYUVaOZmRUYDAARcQ1wzbh+l7R9Xwu8ptCadrrMYWZWbqV98tktYpiZ5SttMJiZWT4Hg5mZZTgYzMwso/TB4CYxzMyyShsMvvhsZpavtMFgZmb5HAxmZpbhYDAzswwHg5mZZZQ+GHxTkplZVmmDQX5Vj5lZrtIGg5mZ5XMwmJlZhoPBzMwySh8M4TYxzMwyyhsMvvZsZparvMFgZma5HAxmZpbhYDAzswwHg5mZZZQ+GHxPkplZVmmDwTclmZnlK20wmJlZPgeDmZllOBjMzCyj9MHgFjHMzLJKGwySLz+bmeUpbTCYmVk+B4OZmWU4GMzMLMPB4GefzcwyShsMvvRsZpavtMFgZmb5Cg0GSadIul/SSkkXTDDOSZJul3SPpOuLrM/MzKBW1IIkVYGvAH8CrAFulnRVRNzbNs584GLglIh4RNIBRdVnZmaJIo8YXgqsjIgHI2IEuBI4fdw4fwp8LyIeAYiIdQXWZ2ZmFBsMS4DVbd1r0n7tjgIWSPqFpFsknZU3I0nnSFohacX69ev3qig3iWFmllVkMOTdCDR+s1wDXgK8Dngt8FFJR+00UcSlEbE8IpYvXrx4z4rxbUlmZrkKu8ZAcoRwWFv3ocDanHE2RMQAMCDpBuCFwAPFlGhmZkUeMdwMHClpqaQO4AzgqnHj/AA4QVJNUg/wR8B9BdZoZlZ6hR0xRERD0nnAtUAVuCwi7pF0bjr8koi4T9KPgTuBFvDViLi7qBrNzKzYU0lExDXANeP6XTKu+3PA5wqrqagFmZntI0r75LPcKIaZWa7SBoOZmeVzMJiZWYaDwczMMhwMZmaWUfpgcJMYZmZZpQ0GN4lhZpavtMFgZmb5HAxmZpYxqWCQVJf0B0nPm+mCzMxsdk0qGCJiFBjladiCRPjqs5lZxlROJf1P4MOSCm1faab42rOZWb6pbORPAF4JPCrpbmCgfWBEvHE6CzMzs9kxlWDYAHx3pgoxM7O5YdLBEBFnz2QhZmY2N0z6GoOk6yTNz+nfL+m66S3LzMxmy1QuPp8EdOT07yK5/rBP8j1JZmZZuz2VJOnYts5jJG1s664CrwUene7CZpxvSzIzyzWZawwrSHasA/hJzvAh4H3TWZSZmc2eyQTDUpL96weB40juThozAqyLiOYM1GZmZrNgt8EQEQ8DSPo0cFxEXNI+XNK5kpZExEdnqEYzMyvQVC4+/xlwW07/W4Gzpqec4rlFDDOzrKkEwwHA+pz+G4ADp6ec4shXn83Mck0lGB4h/7bUE4E101OOmZnNtqk0ifFPwN9L6gDGHmg7GbgQ+Mx0F2ZmZrNjKk1ifEHSIuDLPPWg2wjwpYj47EwUZ2ZmxZtSE9oR8WFJnwKWkdzCem9EbJuRyszMbFZM+d0KETEA3DwDtcyKcKMYZmYZpX3ns3xTkplZrtIGg5mZ5XMwmJlZhoPBzMwyHAy+9mxmllHaYPC1ZzOzfKUNBjMzy1doMEg6RdL9klZKumAX4x0nqSnprUXWZ2ZmBQaDpCrwFeBUkienz5S0bILxPgNcW1RtZmb2lCKPGF4KrIyIByNiBLgSOD1nvPcB3wXWFVibmZmligyGJcDqtu41ab8dJC0B3gRk3hI3nqRzJK2QtGL9+rxXREyeb0oyM8sqMhjybgQav13+B+D83b1DOiIujYjlEbF88eLFe1aM28QwM8s15Ub09sIa4LC27kOBtePGWQ5cmW60FwGnSWpExL8VU6KZmRUZDDcDR0paCjwKnAH8afsIEbF07Luky4EfOhTMzIpVWDBEREPSeSR3G1WByyLiHknnpsN3eV3BzMyKUeQRAxFxDXDNuH65gRAR7yqmpiKWYma27yjtk8++9mxmlq+0wWBmZvkcDGZmluFgMDOzDAeDmZlllD4Ywo1imJlllDYYfFOSmVm+0gaDmZnlczCYmVmGg8HMzDJKHwxuEsPMLKu0weAmMczM8pU2GMzMLF9pg2H91mEArn9g714Namb2dFPaYLhjzWYAvnnTqlmtw8xsriltMFTSawy++GxmllXaYFD67LNzwcwsq7TBMCbGHTJsHBjh4l+spNVyZJhZORX6as+5ZOx21VYk4SCJjQMjHPvJnwLw/EP248SjFs9ihWZms6O0RwztjzEs/fA1jDZbvPvym3f0O+uy3/Lk4MhO0zWaLb72y4d4YttwAVWamRWvtEcM459wu+2RJ7l99ZOZfi/6m5/u+P6ulx/Ox9+wjGd/5EcAfPKH93L/p06hs1ad+VrNzApU2mD4999ln1/Y3ZPQl9+4istvXJXp95z//mM6qhVGmi2+/xcv500X3whArSK+fvZxPOegeSzu60R+zNrM9iGlDYZ1W7KngrZuH93x/Z3HP5Nv3PTwhNN+/A3L+B9X3wvASLMFsCMUABqt4M+/9tudpnvowtN2GxIr123jZ/c9zmELe7j+/vX85qEnuPDNx3D8s/bf/X+Umdk00Pi7cvY1y5cvjxUrVkx5uld85jrWbBraqX9/V407P/HaHd1X37GW5x7cz6u/eP2Ofqs+/Tq2jzY5+qM/nvJyzz/laI47fAG/fvAJlh3Sz7svn3rtE/nIac/lP5+wFEk0mi0u+N5dXP/AepbM7+b21U9y2gsOoiLxyMZBjj5oHmcdfzjPO6TfRzRmJSTplohYnjusrMFwwmevY/XGnYPhb05/Hmcdf3juNH95xW285xVLeeFh83caNnZnU56V67by6i/eMOUaAQ7q7+IPW7bv0bSTtaCnDsCB/V1UJA7s72Tl+m0Mj7ZY2NvBsxb3sWHbMKe/aAmnveAg+jprVCS2DjdYv3WYbcMNnn1AHzeu3MA9a7cwMNzgrkc3s3lolG3DDSJgYW8HFcFwo8Xh+/ciQaUiWq3gwP4uNg2OsGHbMEcdOI++zhqHLeihs16ho1qhq6PKo5uG2Dbc4IhFvSxZ0M2aTUOs2TTEcKNJb0eNA/s7qVUqDI422a+7zvzuOj0dyfWfRisYGm2yeXCU4UaL7o4qI40WzVZw8H5ddNQqRMAB/Z30ddboqlcZbbZoRbBlqMHjW7Yz2mwxNNpkYW8HSxf10lGtIInhRpOOanIPx0S///bRJsONFr0dVQaGm2wdHmVoJOkXASPNJtVKhUV9HSzs7aCnY+oH8sONZrqs5Ai21Qo2D41SkdgwMMxIo8Vos0VHtUJvZ41qRQwMN6hURH9XjZFGUK+KWrVCo9liYKTJtu0NRppNWi2oVsVoo0VfV/LbN1vBY5u302i2qFcrtCIYabYYGG7QVa/SagVPDo0yONKkXhUjjdaOdVSRmNdVY15Xjf17O+nrqnHAvE4W9XWyoKdOrVrae2IK5WDIceJnf84jGwd36v/mFy/hi29/0XSUltFotnjjRb/i3se2ZPo/Y2EPP/vgK6lIfPfWNRx/xP4ctrBnt/NrtYI7H93Mr1Zu4JTnH8TqjYO86+s3Z8Y5+qB5/O2bXkBnrcJRB85jw7ZhVjy8iZOPPoCr71jLJ394LwMjzWn97xyvo1qhv7tOvSoW9HRw72NbWNTXSUXQUasw3GgxONygs15lfk+dh58YpDnLz5CMXTfalXo1CYHRZtDXWWNotMmCnjr7ddfpqldptoInBkYYbbZ4cjA5TSlN7kn77nqV3s4qXfUqEbBfd51KBQaHm/xhy3ZGGi06ahXmd9fprFcZGmnO+M7DnuqqV3bUK0QQNFvBaDN/RUjQ15kEY7Uims2gq6NKRclDqfN76szvqTOvq053vUq1IqoVURHUq8nfU7MVNFpBT71KtSoioNlqMa+rTle9QletSpCsV0jWd3dHlZ6OKr2dNbaPNhkcabJ9tElnrUq1QhpmdSqCrdsbNFotWgHNVtBZqzCvK9kRGW60GG4k0480WtQqorezRn93nc5asi4GRho0W8l6GE7HaQW0IhhtJjsL1YrYMjSKBLVKhe50J6daEb0dNerVZL5L5nezoLdjj34bB0OO911xG1ffsXan/v/+V6+a1Ib56WxwpEG1IjprVbZsH6XZDKpV8avfbeCyXz3EYBomj2wc5PXHHEy1IhrN4KD9unj9MYcwMNxgyYJuFvZ0UKlM7TTVSKPFSLPFxm0jrNu6nUpFbNve4OD9uljUlxzJrN44yEH7dfHsxX10dVTZsHWY9VuHqVZEf3edTQMjPL412UuuVqBaqdCTBk9XvcrW7Y0dRyOrNw7SjCACNg2OsGVolI0Do3TVK7QCFvV1sGR+Nx21CrVqhU0DI6x6YoCh0SbNZtDXVWPtk0Ms6Olg89AoW7aP7lg/C3o66KpXOai/i856hcHhBv3ddXo7a3TXq3TWKlQqoqNWodUK1m0dZsPWYTYNjjIw3GB7o0m1IjYNjDA40qS7o8oRi/roqicbwE2DIzs2uocu6KFeEfVahaqElCy/0WqxsLeTjlqF7npypLRtONkw9XZWdxwVdaYh3WglRw49HckefVc92SA10/7bhhsQyZ7/gf2ddNarjDZaVCTqNdFTrzHSbFER9HXV6KxVdzqajkg2iFu3N9iwbZiB4QZ/2LKdjQMjbNiW/AYRQZBskLePNmm2ku4nB0d5cnCEbcMNhkabtCJoNCM98mrRWatQrYhaRQyOJMMh2bhu2T66IzieLs458Qj++rTn7tG0uwqG0l58PmxBd37/kocCkDmV0d9V3/H91BcczKkvOHhGl91Rq9BRq9DXWeMZ++/8WxzXu5DjDl+Y6dffVeeIxX17tLznL9lvj6azyRt/ik0SXfXkiGjxvM5CaxkLpbE98lYkp/oGRhpsHx07FVahu17bcQTQbAWtSE7NRSSBV6+OHakkp8m2bE9OD9aryd59d71KR61CsxUMDDeSUBptUatW6O2sUq9WqAg6a8nRpZSEYHJklZz+7O+uE5EcXY2dKmw0Y8fRyLbhUZ65f++MrKfSBkOea/7yhNkuwcxm0FgoAezfV2wo7UscDMBn33oMqzcOsuyQ/tkuxcxs1jkYgLctP2y2SzAzmzN8X5iZmWWUNhgqfqjLzCxXiYNhtiswM5ubCg0GSadIul/SSkkX5Ax/h6Q708+Nkl44g8XM2KzNzPZlhQWDpCrwFeBUYBlwpqRl40Z7CHhlRBwDfBK4dKbqqToYzMxyFXnE8FJgZUQ8GBEjwJXA6e0jRMSNEbEp7fw1cOhMFeNcMDPLV2QwLAFWt3WvSftN5D3Aj/IGSDpH0gpJK9avX583ym695SUzljlmZvu0IoMhbx89t9ESSa8iCYbz84ZHxKURsTwili9evGfvZV4yP79JDDOzsivyAbc1QPuTZIcCO7ViJ+kY4KvAqRHxREG1mZlZqsgjhpuBIyUtldQBnAFc1T6CpGcA3wP+PCIeKLA2MzNLFXbEEBENSecB1wJV4LKIuEfSuenwS4CPAfsDF6ctMjYmahZ2Ovzy/FdR9QMNZmYZpX0fg5lZme3qfQylffLZzMzyORjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZhQaDpFMk3S9ppaQLcoZL0pfT4XdKOrbI+szMrMBgkFQFvgKcCiwDzpS0bNxopwJHpp9zgH8sqj4zM0sUecTwUmBlRDwYESPAlcDp48Y5HfhmJH4NzJd0cIE1mpmVXq3AZS0BVrd1rwH+aBLjLAEeax9J0jkkRxQA2yTdv4c1LQI27OG0M8l1Td1crc11TY3rmpq9qeuZEw0oMhiU0y/2YBwi4lLg0r0uSFoREcv3dj7TzXVN3VytzXVNjeuampmqq8hTSWuAw9q6DwXW7sE4ZmY2g4oMhpuBIyUtldQBnAFcNW6cq4Cz0ruTXgZsjojHxs/IzMxmTmGnkiKiIek84FqgClwWEfdIOjcdfglwDXAasBIYBM6e4bL2+nTUDHFdUzdXa3NdU+O6pmZG6lLETqfwzcysxPzks5mZZTgYzMwso7TBsLvmOaZ5WYdJ+rmk+yTdI+n9af9PSHpU0u3p57S2aT6c1na/pNe29X+JpLvSYV+WlHeL71TrW5XO83ZJK9J+CyX9VNLv0n8XFFmbpOe0rZfbJW2R9IHZWGeSLpO0TtLdbf2mbf1I6pT0nbT/byQdvhd1fU7Sf6RNynxf0vy0/+GShtrW2yUF1zVtv9s01/WdtppWSbp9FtbXRNuH2fsbi4jSfUgufv8eOALoAO4Als3g8g4Gjk2/zwMeIGkW5BPAh3LGX5bW1AksTWutpsN+CxxP8szHj4BTp6G+VcCicf0+C1yQfr8A+Mxs1Nb2e/2B5IGcwtcZcCJwLHD3TKwf4C+AS9LvZwDf2Yu6XgPU0u+faavr8Pbxxs2niLqm7XebzrrGDf8C8LFZWF8TbR9m7W+srEcMk2meY9pExGMRcWv6fStwH8kT3RM5HbgyIoYj4iGSu7ReqqR5kP6IuCmSX/ibwH+aobJPB76Rfv9G23Jmo7aTgd9HxMO7qXdG6oqIG4CNOcubrvXTPq9/BU6ezFFNXl0R8ZOIaKSdvyZ5FmhCRdW1C7O6vsak078NuGJX85ihuibaPsza31hZg2GipjdmXHoI92LgN2mv89LD/svaDhUnqm9J+n18/70VwE8k3aKkuRGAAyN9hiT994BZqg2SPZz2/2HnwjqbzvWzY5p0o74Z2H8aanw3yV7jmKWSbpN0vaQT2pZdVF3T9bvNxPo6AXg8In7X1q/w9TVu+zBrf2NlDYZJNb0x7QuV+oDvAh+IiC0krcc+C3gRSXtQX9hNfTNV9x9HxLEkrdu+V9KJuxi30NqUPAz5RuD/pL3myjqbyJ7UMe01SvoI0AD+Je31GPCMiHgx8N+Ab0vqL7Cu6fzdZuI3PZPszkfh6ytn+zDhqBMsZ9pqK2swFN70hqQ6yY/+LxHxPYCIeDwimhHRAv4XySmuXdW3huypgWmpOyLWpv+uA76f1vF4emg6dvi8bjZqIwmrWyPi8bTGObHOmN71s2MaSTVgPyZ/KmYnkt4JvB54R3pKgfS0wxPp91tIzksfVVRd0/y7Tff6qgFvBr7TVm+h6ytv+8As/o2VNRgm0zzHtEnP5X0NuC8ivtjWv71J8TcBY3dLXAWckd5JsJTk/RS/TQ8nt0p6WTrPs4Af7GVtvZLmjX0nuXh5d1rDO9PR3tm2nMJqS2X25ObCOmtb3nStn/Z5vRW4bmyDPlWSTgHOB94YEYNt/RcreScKko5I63qwwLqm83ebtrpSrwb+IyJ2nIYpcn1NtH1gNv/GdnVl+un8IWl64wGSPYGPzPCyXkFy2HYncHv6OQ34FnBX2v8q4OC2aT6S1nY/bXfRAMtJ/qf6PXAR6dPre1HbESR3ONwB3DO2LkjOP/4M+F3678JZqK0HeALYr61f4euMJJgeA0ZJ9rzeM53rB+giOVW2kuSukiP2oq6VJOeSx/7Oxu5EeUv6+94B3Aq8oeC6pu13m8660v6XA+eOG7fI9TXR9mHW/sbcJIaZmWWU9VSSmZlNwMFgZmYZDgYzM8twMJiZWYaDwczMMhwMZruhpKXNkDSnXgY/V+uyfZ+DwWz3VpO0gDnWJPNJ6QZ5UVEFSPqFpIt2VZfZdCnsnc9m+6qIaJI0+z3tJNUjYnRPpp3JuqzcfMRgc0q6Z3yxpL+TtEHJi1U+L6mSDl8l6UM501zU1r1K0sckXS5pq6TVkt4uab6kKyVtU/Lyk9dMsqYdp2zS1i9/ng5an/a/PB1Pkv5K0u+VvOTlLkl/ljOfMyVdJ2kI+C+S9pd0haQ16XT3SDq7bbrLgVeSNHAY6efwvFNJkk5U8iKW7ZIel/T3abMvk1q/6ThvVtIK6pCkjUpaFz1wMuvKnh4cDDYXvYOkZdCXA+cBHwDePsV5fIDk0f9jgf9N0hb9t4FrSFr4vAH4Z0ldU5zvapLmEgCeR3Iq5/1p96dImn94L8nLVC4E/knS68bN40Lg4nScfyNpruBWkobvngd8KZ3u5HT89wM3AV9Pl3cw2WaXAZC0hKSZ7dtImm5+D0lbUxeOG3XC9SvpIJL3k3wDeC7Jy22+NYn1Yk8ne9OWjT/+TPcH+AVw07h+PwW+mn5fxbg3gaXTXNTWvQq4oq27j6Qtmi+39Ts87bd8EjVlxgVOSrsXtY3TCwwBJ4yb9h+Aa8bN54OTWOaVY//Nef+NE9T1tyRt4VTaxnkXMAz0THL9HpvO85mz/bfgz+x9fI3B5qI7x3Wv5amXlEx5HhGxTdIgSSNuYx5P/53qfCeyjGTP/8eS2hsgq5MEVbsV7R1pK54XkOy1LyF5ZWMHyUZ8Kp5LstFvtfX7ZTqvZ/PUOtnV+r0D+H/A3ZJ+kn7/14hYP8VabB/mYLC5aPzF2OCp054tdn7pSH2S8xgd1w3Tdzp1bD5vAB7ZTS0D47o/BHyQ5JTRXcA24O+YemiJiV++0t5/wvUbEc302svLSJpgfw9woaRXRsQdU6zH9lEOBtvXrCc5xw5Aeo3gaJLz6kUZSf+ttvW7l+SUzTMj4ropzu8VwNUR8S3Y0T7/UcCT45ZZzZm23b3A2yRV2o4aXpFO+/vJFhMRQXJN4yZJf0PS/PTbSY4mrAQcDLavuQ54t6SrSELiI+QfMcykh0n2sl8n6WpgKCK2Svo88Pl0w34DybWNlwGtiLh0F/N7AHi7pFcAG4D3AUvJht0qkhe+H05yRJH39q2LSS4kXyzpSyTv2vg0ybWJwZzxdyLpZSQvrrmW5HTbi0ne/HXvZKa3pwfflWT7mgtJwuEHwE9IzqHfWmQBEfEo8HGSi72Pk7wQBeCjwCdITg3dQ3JR9y3AQ7uZ5adI7qD6EUmgDPDUu5rHfJ5kz/9ekkB8xgR1nUqyMb8duIzk5TR/PYX/vM3AHwM/JHlBzBeAT0bEP09hHraP84t6zMwsw0cMZmaW4WCw0pN0Sfo0dN7nktmuz6xoPpVkpSfpAKB/gsFbImJdkfWYzTYHg5mZZfhUkpmZZTgYzMwsw8FgZmYZDgYzM8v4/w8Hogb8dVLzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --bag 5\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon, do_learn=False)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using the cb_explore_adf contextual bandit algorithm and exploring with the online cover algorithm with m = 3 and plotting ctr vs num_iterations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgcdb3v8fd3epbMmsk2IZlkspgQiBAhDIsKsikEUDnqUUA8elFvDo+icMWroI8ejxuicBREjMjNRUWNXjeiIqAiB5FFEghkIyEr2ZOZZPZ95nv/qJrQNelJZpLumhn683qeftJV9evqL9VDf7p+VfUrc3dERER65Qx1ASIiMrwoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCJiCwYzW2xme81sVT/LzczuNLMNZvaimc2PqzYREXlVnHsM9wELDrP8EmB2+FgIfD+GmkREpI/YgsHdHwf2H6bJ5cCPPfA0UG5mk+KpTkREeuUOdQFJKoFtSdPbw3m7+jY0s4UEexUUFxefdsIJJ8RSoIjIa8Xy5ctr3H1CqmXDKRgsxbyU43W4+z3APQDV1dW+bNmyTNYlIvKaY2Zb+1s2nM5K2g5MTZqeAuwcolpERLLWcAqGpcAHw7OTzgLq3f2QbiQREcms2LqSzOznwHnAeDPbDvwHkAfg7ouAB4FLgQ1AC3BNXLWJiMirYgsGd7/qCMsd+HhM5YiISD+GU1eSiIgMAwoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIRKzBYGYLzGydmW0ws5tSLB9tZr83sxfMbLWZXRNnfSIiEmMwmFkC+B5wCTAXuMrM5vZp9nFgjbu/ATgPuN3M8uOqUURE4t1jOAPY4O6b3L0DWAJc3qeNA6VmZkAJsB/oirFGEZGsF2cwVALbkqa3h/OS3QWcCOwEVgLXu3tP3xWZ2UIzW2Zmy/bt25epekVEslKcwWAp5nmf6YuBFcBk4BTgLjMrO+RF7ve4e7W7V0+YMCH9lYqIZLE4g2E7MDVpegrBnkGya4DfeGADsBk4Iab6RESEeIPhWWC2mc0IDyhfCSzt0+YV4EIAM5sIzAE2xVijiEjWy43rjdy9y8yuAx4GEsBid19tZteGyxcBXwHuM7OVBF1Pn3X3mrhqFBGRGIMBwN0fBB7sM29R0vOdwEVx1iQiIlG68llERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhIRazCY2QIzW2dmG8zspn7anGdmK8xstZn9d5z1iYgI5Mb1RmaWAL4HvA3YDjxrZkvdfU1Sm3LgbmCBu79iZhVx1SciIoE49xjOADa4+yZ37wCWAJf3afN+4Dfu/gqAu++NsT4RESHeYKgEtiVNbw/nJTseGGNmj5nZcjP7YKoVmdlCM1tmZsv27duXoXJFRLJTnMFgKeZ5n+lc4DTgMuBi4AtmdvwhL3K/x92r3b16woQJ6a9URCSLxXaMgWAPYWrS9BRgZ4o2Ne7eDDSb2ePAG4D18ZQoIiJx7jE8C8w2sxlmlg9cCSzt0+YB4BwzyzWzIuBMYG2MNYqIZL3Y9hjcvcvMrgMeBhLAYndfbWbXhssXuftaM3sIeBHoAe5191Vx1SgiImDufbv5R5bq6mpftmzZUJchIjKimNlyd69OtUxXPouISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiIxA9a2dHGjuyMi6BzQkhpnlEQyZfaG7r85IJSIictCehjbW72mkpqmd1Tsa2Lq/hZqmdmqa2qlr7qSxvYvrzp/Fpy+ek/b3HlAwuHunmXVy6DDZIiJyDOpbOtlc28z63Y1sO9DCut2NrNnVwPYDrQfbFOTmMG1cERNKCzitagzlRfkcN3oUb5w5LiM1DWYQve8CN5vZNe7elZFqRERewxraOlm1vZ41uxpYsa2OVTvq2VLbcnB5jsG0ccWcMrWcD71xOvOmjGZscT7TxxeTl4iv538wwXAOcC6ww8xWAc3JC939neksTERkpHJ3Glq72LCvkRe21fPslv1s2tfMy3sb6Qn7XSrLCzmpsoz3Vk9ldkUJsypKqBpbRG6MAdCfwQRDDfDrTBUiIjIS1bd0snpXPc+/EuwB7KxrZev+FupaOg+2mTq2kNkVpSw46ThOmzaGEyaVUlE6agirPrwBB4O7X5PJQkREhruu7h5W7Wzg+VcO8Pj6fazZ1cCehvaDy6vGFjFtXBGXnDSJmeOLmTmhmDnHlTJlTNEQVj14Aw4GM3sUeLe71/WZXwb8zt0vSHdxmbSvsZ1P/vx57nr/qYwrKRjqckRek7p7nKb2Lprbu+js7qEwL0FXj+NAbo5R19JJU3snOWZ09TizK0ooL8qPtcaOrh521rXS1tVNd49TnJ9LbXMHa3c18NLuBrbtb6Wts5v61k427Wumo7sHgOnjinjzrPHMrijlhEmlzK8aw+jCvFhrz5TBdCWdB6T6xEYRHH8YUX781Bae2lTLT595hU9eOHuoy5ERaHNNM41tnTS1dXGgpZPa5nZqGtvZ29hOY3sX7s7Frz+Od8ybTE6ODXW5GdPW2U1dSyeNbZ28uL2e7QdaWbmjjpU76qlp6qC7Z+AnM+YYnFw5mvecNoWLX38cY4ryaWrvIpFjFOcnSOQYTe1dbK1tobm9i7rWTrbUNFPb3MHmmmb2NrbT1tGNGZSOymV8SQFF+bm0dXbT3NFFR1cPxQW5dHb30NTWxY66VmqbOg5+2fdVUpDLjPHFFOYnmDR6FOfOmcDJlaM5tWoMleWF6dqEw84Rg8HM5idNzjOz/UnTCeBiYEe6CxMZrrbtb+FbD69j6Qs7D1lmBqML8xhdmEdDaycPrtzN4ic285kFJ/Cm143DbGQEREv4JVqYn6CnJ/jyf2V/Cxv3NXGgpZOdda1sP9DCC9vq2d3Qdsjre39NTx5dSHlRHkX5ueQljLauHnJzDAO63SkdlUfpqFy6u51Ewnh+6wH+vHYvX3xgNV98YDVmkHyTydycYM+ir4LcHCrHFFJZXsjE0gJ63Gls62L9nkbaOnsoyM2hqCBBQW6C/c0t5JhRVpjLG183jorSUcyqKKEwL0EiB5rbuxldmBd2ARWOmM8snQayx7CM4PoFBx5JsbwV+EQ6ixIZjpZv3c/tj6znyY21FOTm8LHzXsfMCSWMKcpjYtkoKkoLKCvMY1ReAoCeHud3K3bwzYfWcfW9zzBzQjEfPGsa762eSnFBbLdbP1hLc0cXrZ3d5Jixr7Gdjq6eg1+gDW2d7Kpr48Ud9azcXsfW/S0c7q6/o/JymDS6kNNnjOX4ihLKi/MpKUgwd9JoqsYWUZifOKo6z59TwacumsP6PY38de1eWju7KS/Mo7vHae3sprWzmzFFeVSWF1FelEdxQS4zxhVTVpiblV/gmTKQv84ZgAGbgNMJzk7q1QHsdffuDNQWixF+y2vJsF31rTz60l7+8MIuntpUy7jifG582/G857QpTD5CV0JOjvHu+VO49ORJPLBiB0ue3caXfr+Gbz28LjhFcWIJ7jChtIBReQnaO7uZMqaIyjGFx9xX7e5sqmnmyY21rNxex6Mv7aWm6cjDJ1SWF3LipFIuP6WSssI8Wtq7SCSMgtygK2XOcaWMKcpnTFFeRr+Ij59YyvETSzO2fjm8IwaDu28FMLNvAKe7+6Lk5WZ2rZlVuvsXMlRjRui3xfDV3N7FltpmEjnGjPHFFOQe3a/Po7WvsZ01uxpYumInD6zYQVePUzW2iP998Rw+9KbplAzy1/6ovARXnF7FFadXsXzrfhY/sYWfPL31sH3vleWFzJxQTFlhHkbQ777gpOOoGlsUHLx1aO3oprY5PKbR1sW63Q28tLuRnXWtbK5p5kB4umRpQS5nzx7PqVXl5CdyMDPGleQzKjdBjztjivMpzs+loqyA8ToRQxjcwecPAO9NMf854GZgRAWDdhSGl8a2Tm5/ZD1/eHEXNU2vnv5XnJ+galwx58+ZQPX0MZw/pyIjv1T3NLSxdMVOfvv8DtbubsAd8nNzuPrMKt5bPZW5k8rScgD5tGljOW3aWLp7nFf2t5Awo6a5nfbOHgrycthxoJWttc28vLeJdbsb2ba/hca2Lv7w4i5u+dNL5Ofm0NGV+kApBIFy3OhRXDT3OOZNHc2ZM8Yxq6LkmOuW7DKYYKgA9qWYXwNMTE858VO3ZOZtrW3mn5v38/jLNZSNyqWjq4dNNc0s33qAiWUFdHU7tUmjRF5+ymTGFRewo66FmqYOlm89wNpdDQCMKcrj5CnljC7MY2JpAdXTxzCxbBR1rcEpjz09zuTyQmZXlBz8bM2Mzu4edte38dj6fWytaT7Yr765ppld9W3Utwa/rudXlfPx82Yxb8poTqkqz9hFSL17QwBV4149x31+1ZiU7TfXNPO3l/ays66VklG59PQEB27Li/KYUBqceTOrooSxxfGe6imvTYMJhlcITkvd1Gf+W4DtaasoZjrGkDlrdjbw5T+s5ulN+/tts6ehnVOrynn7vEmcf0IF5x4/IeUewd7GNu79+2Zqmtp5YVsdq3Z0sr+5g3uf2JxyvaUFuTR1BEN6jSnKp66l4+BQBLk5RllhHiUFuUwsK+DkyokcP7GUM2eOZd6U8mP/D8+AGeOLmXH2jKEuQ7LEYILhB8C3zSwfeDScdyFwC3BrugvLtGzdUTjQ3MGqnfWcM3sC/9hQw1/X7mXhW2bS2NbJ2779eKTtjPHF1Ld2MnN8Mbe99w1MD3/hHsnW2mY+uWQFL2wLroXMSxgffvMM5k8Lzv2eNq6IvEQOo/ISuPuAuoYqSkfxuUtPjMxrbOtk7a5gWOKC3Bzcg1/iG/c18eL2egAK8xK0dHZTWV7I1LGFnDK1nNdPHj2g/w6RbDWYITFuN7PxwJ28eqFbB3CHu38zE8VJ+rg7M25+MOWyxf9I/at7c00wTuL+5g7Ou+2xyLLTp4+htqmDuZPLqBpbxN2PbYwsz80x3lc9hSljirju/Fn99s8fy/GC0lF5nDFj7CHzzz+h4qjXKSKD22PA3W82s68Ccwl+dK9x96aMVJZh2dCD1NXdw/cf28jUsUXc8IsVKdv0nnZYUVrAO94wmfPnVDB3clmkzYptddz28Dqe2PDqmcrPbjkAwKaayCC7AFSUFvDLf3/jgPcwRGR4GfRVNu7eDDybgVqGxLf/sp7rLphFYgQOWdDZ3cMP/76Jbz60LjL/pMoyVu1oOKT9/KpyfnXtmwZ9ds0pU8u5/6NnRubVt3by02e2cslJk+jo6mFMUR4VZcN3tEgRGbh4L78cRpK/Gldsq+O0aanPBhmu2jq7OeELD6VclioUliw8i7PSeLen0YV5fOy8WWlbn4gMH1kbDMPVS7sbWPCdv/OBs6r46r+cTEtHFwt/vDzSjdPXQzecQ3lhPp/65Qqe3FjL3EllrNnVwFM3X8Ck0a/dgb5EJDMUDAz9tQz7Gts5/Wt/icy7/+lXuP/pVw77uvdVT+HW98w7eAD3Z//zrIzVKCLZQ8EwRLbWNpNjxoGWDt551z+O2H7hW2Zy7bmvoyg/cXCQNhGRTFAwDIHpN/0x5fy/f+Z8po4NroJ1d3Y3tNHW2XPwClkRkTgM/V2nh4H/1c+pnJnw0R8tO2Te+JJ8tnzjsoOhAMH5/ZNGFyoURCR22mMAtta2ZHT9qfYQ7nr/qbx93mR6evw1fXcvERl5Yt1jMLMFZrbOzDaY2U2HaXe6mXWb2b/GVduGvY0ZWe87vvvEIfPu/8iZvH3eZACFgogMO7EFg5klgO8BlxBcOX2Vmc3tp92twMNx1Qbw1v96nJZw0LXpN/2Ra3+ynOb2LurDMe2PxvSb/sjKHcGYPZXlhfzp+nP4643ncvbs8WmpWUQkE+LsSjoD2ODumwDMbAlwObCmT7tPAL8muFtcxry899CRPJraurDw0reHVu/mof/YDcCWb1w26PU/mXTdwR1XnsLlp1QeZaUiIvGKMxgqgW1J09uByDgLZlYJvAu4gMMEg5ktBBYCVFVVHVUxz2w+dCjo+tZOzvj6Xw+Z33uMYPMtlx4yEN0nLpjF4ic2s+o/L6a7x9lR18ozm/fzmV+9CMAP/u00Ln79cUdVo4jIUIgzGFJ1pvcdy+47wGfdvftwo266+z3APQDV1dVHNR5eT4obMfQddrqvVKOTfvfRDf0uAxQKIjLixBkM24GpSdNTgJ192lQDS8JQGA9camZd7v67dBcTxw16Xv7aJZl/ExGRNIszGJ4FZpvZDGAHcCXw/uQG7n7wFlVmdh/wh0yEAkDPYW7E/ra5E/nhB6uB1Kearv3yAgrzEzS3d1GYl6C2uYPL7vw7owvzOP+ECm686Hg6u528hC4TEZGRJ7ZgcPcuM7uO4GyjBLDY3Veb2bXh8kVx1QKpu5J6PbWx9uDzjV+/lM7uHu7468t8/7GNPHnTBRTmB0NSFBcEm29CaQH//PxbI+so0BUiIjJCmY/wmx5XV1f7smWHXk18JJ/77Up+9kwwSN0dV57C9Utevfr56jOr+Nq7Tk5bjSIiw42ZLXf36lTLsravY0JJwcHnb583mU9c8Oq9BT5w1rShKElEZFjI2mBI3k9K5Bg3XjTn4PSsipL4CxIRGSaytyc8RRfami9fTFNblw4ai0hWy95gSKEoP5eifG0SEcluWf/TeKjv3iYiMtxkbTD0XsagXBARicraYOh1uKE3RESyUdYGgx8yTJOIiEAWB8OJk8oAuOHC2UNciYjI8JK1wVASjlnxZt00R0QkImuDQR1JIiKpZW0w9NKhZxGRqKwPBhERiVIwiIhIhIJBREQisjcYdPRZRCSl7A2GkK58FhGJyvpgEBGRKAWDiIhEKBhERCRCwSAiIhFZGwwaXVVEJLWsDYZeOidJRCQq64NBRESiFAwiIhKhYBARkYisDQbXsWcRkZSyNhh6aUQMEZGorA8GERGJUjCIiEiEgkFERCKyNhh08FlEJLWsDYZepmufRUQisj4YREQkKtZgMLMFZrbOzDaY2U0pll9tZi+GjyfN7A1x1iciIjEGg5klgO8BlwBzgavMbG6fZpuBc919HvAV4J646hMRkUCcewxnABvcfZO7dwBLgMuTG7j7k+5+IJx8GpgSY30iIkK8wVAJbEua3h7O689HgD+lWmBmC81smZkt27dv31EVo5OSRERSizMYUp3+k/L72czOJwiGz6Za7u73uHu1u1dPmDDh2IrSSUkiIhG5Mb7XdmBq0vQUYGffRmY2D7gXuMTda2OqTUREQnHuMTwLzDazGWaWD1wJLE1uYGZVwG+Af3P39THWJiIiodj2GNy9y8yuAx4GEsBid19tZteGyxcBXwTGAXdb0MfT5e7VcdUoIiLxdiXh7g8CD/aZtyjp+UeBj8ZUSxxvIyIy4ujKZxERiVAwiIhIhIJBREQiFAwiIhKhYBARkYisDQadkyQiklrWBkMvDYkhIhKV9cEgIiJRCgYREYlQMIiISETWBoNGxBARSS1rg6GXpbxNhIhI9sr6YBARkSgFg4iIRCgYREQkQsEgIiIRWRwMOi1JRCSVLA6GgIbEEBGJyvpgEBGRKAWDiIhEKBhERCQia4NBQ2KIiKSWtcHQSwefRUSisj4YREQkSsEgIiIRCgYREYlQMIiISETWBoNOShIRSS1rg6GXbtQjIhKV9cEgIiJRCgYREYlQMIiISETWBoOGxBARSS1rg6GXhsQQEYnK+mAQEZGoWIPBzBaY2Toz22BmN6VYbmZ2Z7j8RTObH2d9IiISYzCYWQL4HnAJMBe4yszm9ml2CTA7fCwEvh9XfSIiEohzj+EMYIO7b3L3DmAJcHmfNpcDP/bA00C5mU2KsUYRkayXG+N7VQLbkqa3A2cOoE0lsCu5kZktJNijAGgys3VHWdP4ObdSc5SvzaTxoLoGabjWproGR3UNzrHUNa2/BXEGQ6rzf/qeNDqQNrj7PcA9x1yQ2TJ3rz7W9aSb6hq84Vqb6hoc1TU4maorzq6k7cDUpOkpwM6jaCMiIhkUZzA8C8w2sxlmlg9cCSzt02Yp8MHw7KSzgHp339V3RSIikjmxdSW5e5eZXQc8DCSAxe6+2syuDZcvAh4ELgU2AC3ANRku65i7ozJEdQ3ecK1NdQ2O6hqcjNRlrrEhREQkia58FhGRCAWDiIhEZG0wHGl4jjS/11Qz+5uZrTWz1WZ2fTj/S2a2w8xWhI9Lk15zc1jbOjO7OGn+aWa2Mlx2p9mxDwNoZlvCda4ws2XhvLFm9mczezn8d0yctZnZnKTtssLMGszshqHYZma22Mz2mtmqpHlp2z5mVmBmvwjnP2Nm04+hrm+Z2UvhkDK/NbPycP50M2tN2m6LYq4rbZ9bmuv6RVJNW8xsxRBsr/6+H4bub8zds+5BcPB7IzATyAdeAOZm8P0mAfPD56XAeoJhQb4EfDpF+7lhTQXAjLDWRLjsn8AbCa75+BNwSRrq2wKM7zPvm8BN4fObgFuHorakz2s3wQU5sW8z4C3AfGBVJrYP8DFgUfj8SuAXx1DXRUBu+PzWpLqmJ7frs5446krb55bOuvosvx344hBsr/6+H4bsbyxb9xgGMjxH2rj7Lnd/LnzeCKwluKK7P5cDS9y93d03E5yldYYFw4OUuftTHnzCPwb+JUNlXw78KHz+o6T3GYraLgQ2uvvWI9Sbkbrc/XFgf4r3S9f2SV7Xr4ALB7JXk6oud3/E3bvCyacJrgXqV1x1HcaQbq9e4evfB/z8cOvIUF39fT8M2d9YtgZDf0NvZFy4C3cq8Ew467pwt39x0q5if/VVhs/7zj9WDjxiZsstGG4EYKKH15CE/1YMUW0Q/MJJ/h92OGyzdG6fg68Jv9TrgXFpqPHDBL8ae80ws+fN7L/N7Jyk946rrnR9bpnYXucAe9z95aR5sW+vPt8PQ/Y3lq3BMKChN9L+pmYlwK+BG9y9gWD02NcBpxCMB3X7EerLVN1vdvf5BKPbftzM3nKYtrHWZsHFkO8E/l84a7hss/4cTR1pr9HMPg90AT8NZ+0Cqtz9VOBTwM/MrCzGutL5uWXiM72K6I+P2LdXiu+Hfpv28z5pqy1bgyH2oTfMLI/gQ/+pu/8GwN33uHu3u/cAPyTo4jpcfduJdg2kpW533xn+uxf4bVjHnnDXtHf3ee9Q1EYQVs+5+56wxmGxzUjv9jn4GjPLBUYz8K6YQ5jZh4C3A1eHXQqE3Q614fPlBP3Sx8dVV5o/t3Rvr1zg3cAvkuqNdXul+n5gCP/GsjUYBjI8R9qEfXn/B1jr7v+VND95SPF3Ab1nSywFrgzPJJhBcH+Kf4a7k41mdla4zg8CDxxjbcVmVtr7nODg5aqwhg+FzT6U9D6x1RaK/JIbDtss6f3StX2S1/WvwKO9X+iDZWYLgM8C73T3lqT5Eyy4JwpmNjOsa1OMdaXzc0tbXaG3Ai+5+8FumDi3V3/fDwzl39jhjky/lh8EQ2+sJ/gl8PkMv9fZBLttLwIrwselwE+AleH8pcCkpNd8PqxtHUln0QDVBP9TbQTuIrx6/Rhqm0lwhsMLwOrebUHQ//hX4OXw37FDUFsRUAuMTpoX+zYjCKZdQCfBL6+PpHP7AKMIuso2EJxVMvMY6tpA0Jfc+3fWeybKe8LP9wXgOeAdMdeVts8tnXWF8+8Dru3TNs7t1d/3w5D9jWlIDBERicjWriQREemHgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhEjsCCkTbdzIbVzeCHa10y8ikYRI5sG8EImL1DMp8XfiGPj6sAM3vMzO46XF0i6RLbPZ9FRip37yYY9jvtzCzP3TuP5rWZrEuym/YYZFgJfxnfbWZfN7MaC26scpuZ5YTLt5jZp1O85q6k6S1m9kUzu8/MGs1sm5ldYWblZrbEzJosuPnJRQOs6WCXTTj65d/CRfvC+feF7czMPmNmGy24yctKM/tAivVcZWaPmlkr8O9mNs7Mfm5m28PXrTaza5Jedx9wLsEAhx4+pqfqSjKzt1hwI5Y2M9tjZt8Oh30Z0PYN27zbglFQW81svwWji04cyLaS1wYFgwxHVxOMDPom4DrgBuCKQa7jBoJL/+cDvyQYi/5nwIMEI3w+DtxvZqMGud5tBMMlALyeoCvn+nD6qwTDP3yc4GYqtwA/MLPL+qzjFuDusM3vCIYreI5g4LvXA3eEr7swbH898BTwf8P3m0R02GUAzKySYJjt5wmGbv4IwVhTt/Rp2u/2NbPjCO5P8iPgRIKb2/xkANtFXkuOZSwbPfRI9wN4DHiqz7w/A/eGz7fQ505g4WvuSpreAvw8abqEYCyaO5PmTQ/nVQ+gpkhb4LxwenxSm2KgFTinz2u/AzzYZz03DuA9l/T+N6f6b+ynrq8RjIWTk9TmfwDtQNEAt+/8cJ3ThvpvQY+he+gYgwxHL/aZ3smrNykZ9DrcvcnMWggGceu1J/x3sOvtz1yCX/4PmVnyAGR5BEGVbFnyRDiK500Ev9orCW7ZmE/wJT4YJxJ86fckzXsiXNcsXt0mh9u+LwB/AVaZ2SPh81+5+75B1iIjmIJBhqO+B2OdV7s9ezj0piN5A1xHZ59pSF93au963gG8coRamvtMfxq4kaDLaCXQBHydwYeW0f/NV5Ln97t93b07PPZyFsEQ7B8BbjGzc939hUHWIyOUgkFGmn0EfewAhMcITiDoV49LR/hvIs/RHhIAAAF9SURBVGneGoIum2nu/ugg13c28Ht3/wkcHJ//eKCuz3smUrw22RrgfWaWk7TXcHb42o0DLcbdneCYxlNm9mWC4aevINibkCygYJCR5lHgw2a2lCAkPk/qPYZM2krwK/syM/s90OrujWZ2G3Bb+MX+OMGxjbOAHne/5zDrWw9cYWZnAzXAJ4AZRMNuC8EN36cT7FGkuvvW3QQHku82szsI7rXxDYJjEy0p2h/CzM4iuHHNwwTdbacS3PlrzUBeL68NOitJRppbCMLhAeARgj705+IswN13AP9BcLB3D8ENUQC+AHyJoGtoNcFB3fcAm4+wyq8SnEH1J4JAaebVezX3uo3gl/8agkCs6qeuSwi+zFcAiwluTvO5Qfzn1QNvBv5AcIOY24GvuPv9g1iHjHC6UY+IiERoj0FERCIUDJL1zGxReDV0qseioa5PJG7qSpKsZ2YVQFk/ixvcfW+c9YgMNQWDiIhEqCtJREQiFAwiIhKhYBARkQgFg4iIRPx/F0MvdPCEacgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --cover 3\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The above case without learning**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAd0UlEQVR4nO3de5SkdX3n8fenLt3T3XNnBmQGRgbl6hEBWyCJCK6JAl5mvZwImlWJOSxZ8eAaT0Q5GFddUaNrZAGRNYiaRDBeAhoQXAkhWTEyILcBweE+DDAzwFx6+lK37/7xPN3U01M90z3T/VQP9XmdU6fruda3n+quT/1+T9XvUURgZmY2qtDuAszMbHZxMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWXkFgySrpC0QdK9EyyXpIskrZV0t6Rj86rNzMxekGeL4UrglJ0sPxU4JL2dBXw9h5rMzGyc3IIhIm4BntvJKquA70TiV8BCSfvnU52ZmY0qtbuAJsuBJ5qm16Xznhq/oqSzSFoV9PX1vfrwww/PpUAzsxeL22+/fVNELG21bDYFg1rMazleR0RcDlwO0N/fH6tXr57JuszMXnQkPTbRstn0qaR1wIFN0wcA69tUi5lZx5pNwXAt8L7000knAFsiYoduJDMzm1m5dSVJ+h5wMrBE0jrgr4AyQERcBlwHnAasBQaBM/OqzczMXpBbMETEGbtYHsCHcirHzMwmMJu6kszMbBZwMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDJyDQZJp0h6QNJaSee1WL5A0k8k3SVpjaQz86zPzMxyDAZJReAS4FTgSOAMSUeOW+1DwH0R8SrgZOArkrryqtHMzPJtMRwHrI2IhyOiAlwFrBq3TgDzJAmYCzwH1HKs0cys4+UZDMuBJ5qm16Xzml0MHAGsB+4Bzo2IxvgdSTpL0mpJqzdu3DhT9ZqZdaQ8g0Et5sW46TcBdwLLgKOBiyXN32GjiMsjoj8i+pcuXTr9lZqZdbA8g2EdcGDT9AEkLYNmZwI/isRa4BHg8JzqMzMz8g2G24BDJK1MTyifDlw7bp3HgTcASNoPOAx4OMcazcw6XimvB4qImqRzgBuAInBFRKyRdHa6/DLgs8CVku4h6Xr6eERsyqtGMzPLMRgAIuI64Lpx8y5rur8eeGOeNZmZWZa/+WxmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7MMB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8vINRgknSLpAUlrJZ03wTonS7pT0hpJ/5pnfWZmBqW8HkhSEbgE+CNgHXCbpGsj4r6mdRYClwKnRMTjkvbNqz4zM0vk2WI4DlgbEQ9HRAW4Clg1bp33AD+KiMcBImJDjvWZmRn5BsNy4Imm6XXpvGaHAosk3Szpdknva7UjSWdJWi1p9caNG2eoXDOzzpRnMKjFvBg3XQJeDbwZeBNwgaRDd9go4vKI6I+I/qVLl05/pWZmHSy3cwwkLYQDm6YPANa3WGdTRGwHtku6BXgV8GA+JZqZWZ4thtuAQyStlNQFnA5cO26da4ATJZUk9QLHA/fnWKOZWcfLrcUQETVJ5wA3AEXgiohYI+nsdPllEXG/pJ8BdwMN4JsRcW9eNZqZGShifDf/3qW/vz9Wr17d7jLMzPYqkm6PiP5Wy/zNZzMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8uYVDBIKkt6WtIrZrogMzNrr0kFQ0RUgSo7DpNtZmYvMlPpSvrfwCck5TlUt5mZ5WwqL/InAicBT0q6F9jevDAi3jadhZmZWXtMJRg2AT+cqULMzGx2mHQwRMSZM1mImZnNDpM+xyDpJkkLW8yfL+mm6S3LzMzaZSonn08GulrMn0Ny/sHMzF4EdtmVJOnYpsmjJD3XNF0E3gQ8Od2FmZlZe0zmHMNqku8vBHBji+VDwIensygzM2ufyQTDSkDAw8BrSD6dNKoCbIiI+gzUZmZmbbDLYIiIxwAkfQF4TURc1rxc0tmSlkfEBTNUo5mZ5WgqJ5//BPhNi/l3AO+bnnLMzKzdphIM+wIbW8zfBOw3PeWYmVm7TSUYHqf1x1JfB6ybnnLMzKzdpjIkxjeAr0rqAka/0PYG4ELgi9NdmJmZtcdUhsT4iqQlwEW88EW3CvC1iPjSTBRnZmb5m9IQ2hHxCUmfA44k+QjrfRExMCOVmZlZW0z52goRsR24bQZqMTOzWcDXfDYzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMnINBkmnSHpA0lpJ5+1kvddIqkt6V571mZlZjsEgqQhcApxK8s3pMyQdOcF6XwRuyKs2MzN7QZ4thuOAtRHxcERUgKuAVS3W+zDwQ2BDjrWZmVkqz2BYDjzRNL0unTdG0nLg7UDmKnHjSTpL0mpJqzdubHWJCDMz2115BoNazItx038DfHxX15COiMsjoj8i+pcuXTptBZqZ2W4MorcH1gEHNk0fAKwft04/cJUkgCXAaZJqEfFP+ZRoZmZ5BsNtwCGSVgJPAqcD72leISJWjt6XdCXwU4eCmVm+cguGiKhJOofk00ZF4IqIWCPp7HT5Ts8rmJlZPvJsMRAR1wHXjZvXMhAi4gN51GRmZln+5rOZmWU4GMzMLMPBYGZmGQ4GMzPLcDCYmVmGg8HMzDIcDGZmluFgMDOzDAeDmZllOBjMzCzDwWBmZhkOBjMzy3AwmJlZhoPBzMwyHAxmZpbhYDAzswwHg5mZZTgYzMwsw8FgZmYZDgYzM8twMJiZWYaDwczMMhwMZmaW4WAwM7OMjg+GRiOIiHaXYWY2a3R0MGwZrHLwJ6/jG7c83O5SzMxmjVK7C2inq1c/DsAXrv8tS+Z2865XH9Dmimw6bB2u8vSWYbpLBeZ2lxipNdh3Xjel4ux6H9RoBI2IlnVtGaqydahKrRF0lwqUi8nvsnbDAMO1Ol3FAl2lAs9tryCgq1RgYW+Z4WqDgZEa24ZrdJUKLOgpUyqI7lKB7ZU6zw9WmD+nTLXeoLtUYFFvF9V6g0JBANTqQU+5yMK+Ml3FAvVGUC4WKBVEtdGgWg+eG6gwWK1RKoiRWoNyWn+13qAgUSyIUkE0AgqC3q4Sfd1F5naXeHZ7ha1DVYaqdWr1oFQU3aUi1XqDrUNVhmsNIoJSoUA9gmqtwYLeMuVigd6uIsWCWNTbRa3RoChRqTeoN4JqPTmWwNhjD1XqDIzUGK7WqUdAwEitTqlQoFJvMFKrUywUqNUbAIzUkn12lQp0lwpE+jwMV+tpfckxXdRbppA+BkBEUKk1KBbEcLVBpZbsLwiEaEQQQL2RHKtKrcFIrUGt3qBST47pUKVOqSgK6e9EkDxuIzkG1bTGegS1evJ4AyM13nP8Cj70+pdP+99mRwfD1qHa2P2P/eNdrDp62dgfeZ5uXPM0Z333dv7tL1/PgYt7J73d5sEKR3/m5wC8Ytl83n7Mch7aOMDC3i5OPnQpxx+8T8vtavUGjz03yPdXP8H2kRp3r9tCV7HA8QcvpijRf9BiDtlvLvsv6BnbZu2GbVRqwRH7z0PSTusaqdX5twc38fTWYTZsHeZ3GwbYMlTloCV9LFswh0o9GKrUeGrLMM8PVtg+UmfJ3G6WLZxDQWKkVqdcLFCtN6jUgjnlAttHalTrQbEg+rqL9HaV2DpUHftHefTZ7WzcNsK24RoDI7UdaioVxH7z51CpN1jUW2b7SJ2+7iLFQvKC01MuMm9OiRX79DJ/TplnByo8tWWI5wcrPDtQoVQssKCnxPKFyfMzXKuPBU9BIiJY9/wQW4erbNg2QqXWYG53ieWLeihIDFXqPLl5iGcHRigWxGClTi19ZZk3p8Si3i6KBbFx2wiNCAYr9Un/HeRBgk7vcS2IsTDYk+1Hg7pUTAK/XBQ9XUXqjaDeCLpKyWvQnFKRclGU0zcBEdBTTEJ39I3CyiV90/TbZWlv71/v7++P1atX79a2//3qO/nxb54cm/7B2b9H/0GLp7SP57dXeHZ7he0jNVZd8v8AOGifXr515nG8/ss3A3DiIUv45GlHUCqIf77nKU5/zQpOuPAXAKxc0scjm7bvsN/PrHoF7z3+pVTrDTZsHeFv//1hvn3rY7v1e06ngmBRbxcnHbaU5Qt72DQwwlClTrUeDFZqrN88zAPPbJvUfpbO62ZRbxfz55TZtH2ETdtGGBipUSok/yySmFMuMDBSo7tUZGFvmUYEWwarjNQazCkXKRcL9HQVWLagh3lzSiyd183+C3pYsbiX7ZUa29P9bRwY4cnnhygWxLbhGn3dRQYrdSKSgK3WG2weqvLEc4M0AooFsWJxL92lAgcu7qVSa7BlqMr6zUMA9HWXxt611dNXi2UL57C4r4t9+rrpTsNs3fNDREBvV5EFPWVW7NNLtd6gt6tEsSDKBfH8YJXnBysALOgp010qst/8bhb2lilIVOtBtZ481kH79DG/pzT2znRxXxeQhPHmwSo9acjN7ylTqzfYPJi0Oir1Bj3lIov7ymwdrlEuFBiuJi2IrlKBRiTvbosFsT1tcVTqybvgai15ZzunXKRUEPN7ysybU6KetmZqjSDSF7zknXFQaySth9GQ2z5SY2Ckzvw0BPu6i2n4x9i7+LlzSvR2FdPfOdm+XEyer2q9wWClTr0RPD+YBHUjfREdbaFAEmCNBhQKyQtrX3e6z4KIgDnlArV6sl1XKdnH6JvBrlIhbX00GKrWKUgs6CnTUy5SKoq+rhK1RrB5qEIECEAkdRaSY9hdLtBdKo79nUckb2aS2kSt3pg1LVdJt0dEf8tlnRwMB533zzvMe/QLb97ldsPVOo0IBoZrHPf5X+zWY0+Xly3t4/pzX8eN9z3Np65Zw0i1zisPWMCvHn5ul9v2v3QR5/ynlyOJVy5fwKaBET76/TvZPFhl3fNDY+sdtt885veU6O0q8eAz29g8mHQFtLKwt0xfV4m3HLU/Jx+2L8sWzuElC+bQVUze8QxUajQawYKecsuWR6MRSOyyVTKThip1CgUy/+BmLzY7C4aO7kraHe/8+i+5/bHnJ73+qqOXcc2d61su++Gf/z7v/PovgRcCadPACLc8uJGPfv+uHdb/6rtfxVuPWsb9T23jsJfMG2tyArzlqGW85ahlU/lVdrC4r4uffvjESa1bqzdYv3mYnq6kC6ZcLFDYxQu6BPPnlHe639G+7nbq6XIgWGfr2GAYPZkz3vrNQyxb2NNy2bMDIy1D4bbz/5DFfV1jTcbxvnb6MRPWMb6FsmRuN+849gDecezEJ8JfecCCCZflpVQssGKfyZ8PMbO9x+zo7JpFrvr143z31kfZMljdYdlJf33z2P0TD1kCwEf/6FCWzuueMBTMzPY2HdtiGO/PT34ZX7/5IS66aS0AF1yzBoBHLjwNSQxX62OfdhmdZ2b2YpRri0HSKZIekLRW0nktlr9X0t3p7ZeSXpVXbR8/5fCW8y9Pv/x2+AU/G5vnUDCzF7PcgkFSEbgEOBU4EjhD0pHjVnsEOCkijgI+C1w+U/U0fxjrvFNbhwLAhdf/NvPppW994DUzVZKZ2ayQZ4vhOGBtRDwcERXgKmBV8woR8cuIGD27+ysgl68in33SywD42Ude+ETOBW8Zn1mJ1x++bx4lmZm1TZ7nGJYDTzRNrwOO38n6HwSub7VA0lnAWQArVqzYrWKSL6lnHf6S+ZlPCS2Z28W5V90JQE+5yL3/40279VhmZnuTPIOhVcd8y2/XSXo9STC8ttXyiLictJupv79/xr6ht+ro5axY3Mv3fv04X3pXbqc7zMzaKs9gWAcc2DR9ALDDN78kHQV8Ezg1Ip6dqWIm+4XvY1Ys4pgVi2aqDDOzWSfPcwy3AYdIWimpCzgduLZ5BUkrgB8B/yUiHsyxNjMzS+XWYoiImqRzgBuAInBFRKyRdHa6/DLgU8A+wKXpR0JrE43lYWZmMyPXL7hFxHXAdePmXdZ0/8+AP8uzJjMzy+rYITH28kFlzcxmTMcGw28en/wIqWZmnaRjg2H9luF2l2BmNit1bDA8t32k3SWYmc1KHRsMoxfsNjOzrI4NhqXzuttdgpnZrNSxweChs83MWuvcYGh3AWZms1TnBoNbDGZmLXVuMLS7ADOzWapzg8HJYGbWkoPBzMwyOjcY3JlkZtZSxwbDor4uAA5c3NPmSszMZpeODYbFvUkwnH/aEW2uxMxsdunYYIj0ctOlQsceAjOzljr+VdEnoc3Msjo2GHyhHjOz1jo3GNKfbjGYmWV1bDCM8sdWzcyyOjYYwn1JZmYtdW4wjN5xg8HMLKNjg2GUc8HMLKtjg8E9SWZmrXVsMIzydRnMzLI6OBjcZDAza6Vjg2G0K8ntBTOzrI4NhlHuSTIzy+rYYHBHkplZa50bDGNdSW4ymJk169hgGOWuJDOzrI4NBg+JYWbWWucGQ/rTDQYzs6yODYYxTgYzs4yODQb3JJmZtZZrMEg6RdIDktZKOq/Fckm6KF1+t6RjZ7wmNxnMzDJyCwZJReAS4FTgSOAMSUeOW+1U4JD0dhbw9ZmqZ7hWn6ldm5nt1fJsMRwHrI2IhyOiAlwFrBq3zirgO5H4FbBQ0v4zUcz9T20FoKvUsb1pZmYtlXJ8rOXAE03T64DjJ7HOcuCp5pUknUXSogAYkPTAbta0pP+LbNrNbWfSEnBdUzRba3NdU+O6pmZP6nrpRAvyDIZWnfnjTwFPZh0i4nLg8j0uSFodEf17up/p5rqmbrbW5rqmxnVNzUzVlWc/yjrgwKbpA4D1u7GOmZnNoDyD4TbgEEkrJXUBpwPXjlvnWuB96aeTTgC2RMRT43dkZmYzJ7eupIioSToHuAEoAldExBpJZ6fLLwOuA04D1gKDwJkzXNYed0fNENc1dbO1Ntc1Na5ramakLnnMIDMza+bPapqZWYaDwczMMjo2GHY1PMc0P9aBkv5F0v2S1kg6N53/aUlPSrozvZ3WtM0n0toekPSmpvmvlnRPuuwiac+vKCHp0XSfd0panc5bLOnnkn6X/lyUZ22SDms6LndK2irpI+04ZpKukLRB0r1N86bt+EjqlnR1Ov8/JB20B3X9taTfpkPK/FjSwnT+QZKGmo7bZTnXNW3P2zTXdXVTTY9KurMNx2ui14f2/Y1FRMfdSE5+PwQcDHQBdwFHzuDj7Q8cm96fBzxIMizIp4GPtVj/yLSmbmBlWmsxXfZr4PdIvvNxPXDqNNT3KLBk3LwvAeel988DvtiO2pqer6dJvpCT+zEDXgccC9w7E8cH+G/AZen904Gr96CuNwKl9P4Xm+o6qHm9cfvJo65pe96ms65xy78CfKoNx2ui14e2/Y11aothMsNzTJuIeCoi7kjvbwPuJ/lG90RWAVdFxEhEPELyKa3jlAwPMj8ibo3kGf4O8J9nqOxVwLfT+99uepx21PYG4KGIeGwX9c5IXRFxC/Bci8ebruPTvK8fAG+YTKumVV0RcWNE1NLJX5F8F2hCedW1E209XqPS7f8Y+N7O9jFDdU30+tC2v7FODYaJht6YcWkT7hjgP9JZ56TN/iuamooT1bc8vT9+/p4K4EZJtysZbgRgv0i/Q5L+3LdNtUHyDqf5H3Y2HLPpPD5j26Qv6luAfaahxj8ledc4aqWk30j6V0knNj12XnVN1/M2E8frROCZiPhd07zcj9e414e2/Y11ajBMauiNaX9QaS7wQ+AjEbGVZPTYlwFHk4wH9ZVd1DdTdf9BRBxLMrrthyS9bifr5lqbki9Dvg34x3TWbDlmE9mdOqa9RknnAzXg79NZTwErIuIY4KPAP0ian2Nd0/m8zcRzegbZNx+5H68Wrw8TrjrB40xbbZ0aDLkPvSGpTPKk/31E/AggIp6JiHpENID/Q9LFtbP61pHtGpiWuiNiffpzA/DjtI5n0qbpaPN5QztqIwmrOyLimbTGWXHMmN7jM7aNpBKwgMl3xexA0vuBtwDvTbsUSLsdnk3v307SL31oXnVN8/M23cerBLwDuLqp3lyPV6vXB9r4N9apwTCZ4TmmTdqX97fA/RHxv5rmNw8p/nZg9NMS1wKnp58kWElyfYpfp83JbZJOSPf5PuCaPaytT9K80fskJy/vTWt4f7ra+5seJ7faUpl3crPhmDU93nQdn+Z9vQu4afQFfaoknQJ8HHhbRAw2zV+q5JooSDo4revhHOuazudt2upK/SHw24gY64bJ83hN9PpAO//GdnZm+sV8Ixl640GSdwLnz/BjvZak2XY3cGd6Ow34LnBPOv9aYP+mbc5Pa3uApk/RAP0k/1QPAReTfnt9D2o7mOQTDncBa0aPBUn/4y+A36U/F7ehtl7gWWBB07zcjxlJMD0FVEneeX1wOo8PMIekq2wtyadKDt6DutaS9CWP/p2NfhLlnenzexdwB/DWnOuatudtOutK518JnD1u3TyP10SvD237G/OQGGZmltGpXUlmZjYBB4OZmWU4GMzMLMPBYGZmGQ4GMzPLcDCY7YKSkTZD0qy6GPxsrcv2fg4Gs117gmQEzNEhmU9OX5CX5FWApJslXbyzusymS27XfDbbW0VEnWTY72knqRwR1d3Zdibrss7mFoPNKuk740slfV7SJiUXVvmypEK6/FFJH2uxzcVN049K+pSkKyVtk/SEpHdLWijpKkkDSi5+8sZJ1jTWZZOOfvkv6aKN6fwr0/Uk6S8lPaTkIi/3SPqTFvs5Q9JNkoaA/yppH0nfk7Qu3W6NpDObtrsSOIlkgMNIbwe16kqS9DolF2IZlvSMpK+mw75M6vim67xDySioQ5KeUzK66H6TOVb24uBgsNnovSQjg/4+cA7wEeDdU9zHR0i++n8s8H2Ssej/AbiOZITPW4C/kzRnivt9gmS4BIBXkHTlnJtOf45k+IcPkVxM5ULgG5LePG4fFwKXpuv8E8lwBXeQDHz3CuBr6XZvSNc/F7gV+Fb6ePuTHXYZAEnLSYbZ/g3J0M0fJBlr6sJxq054fCW9hOT6JN8GjiC5uM13J3Fc7MVkT8ay8c236b4BNwO3jpv3c+Cb6f1HGXclsHSbi5umHwW+1zQ9l2Qsmoua5h2UzuufRE2ZdYGT0+klTev0AUPAieO2/RvgunH7+YtJPOZVo79zq99xgrr+J8lYOIWmdT4AjAC9kzy+x6b7fGm7/xZ8a9/N5xhsNrp73PR6XrhIyZT3EREDkgZJBnEb9Uz6c6r7nciRJO/8fyapeQCyMklQNVvdPJGO4nkeybv25SSXbOwieRGfiiNIXvQbTfP+Pd3Xy3nhmOzs+N4F/F/gXkk3pvd/EBEbp1iL7cUcDDYbjT8ZG7zQ7dlgx4uOlCe5j+q4aZi+7tTR/bwVeHwXtWwfN/0x4C9IuozuAQaAzzP10BITX3ylef6Exzci6um5lxNIhmD/IHChpJMi4q4p1mN7KQeD7W02kvSxA5CeIzicpF89L5X0Z7Fp3n0kXTYvjYibpri/1wI/iYjvwtj4/IcCm8c9ZrHFts3uA/5YUqGp1fDadNuHJltMRATJOY1bJX2GZPjpd5O0JqwDOBhsb3MT8KeSriUJifNp3WKYSY+RvMt+s6SfAEMRsU3Sl4Evpy/st5Cc2zgBaETE5TvZ34PAuyW9FtgEfBhYSTbsHiW54PtBJC2KVlffupTkRPKlkr5Gcq2NL5Ccmxhssf4OJJ1AcuGaG0i6244hufLXfZPZ3l4c/Kkk29tcSBIO1wA3kvSh35FnARHxJPBXJCd7nyG5IArABcCnSbqG1pCc1H0n8Mgudvk5kk9QXU8SKNt54VrNo75M8s7/PpJAXDFBXaeSvJjfCVxBcnGaT07h19sC/AHwU5ILxHwF+GxE/N0U9mF7OV+ox8zMMtxiMDOzDAeDdTxJl6Xfhm51u6zd9ZnlzV1J1vEk7QvMn2Dx1ojYkGc9Zu3mYDAzswx3JZmZWYaDwczMMhwMZmaW4WAwM7OM/w+QPVVBeknpkQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf -q UA --quiet --cover 3\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon, do_learn=False)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using the cb_explore_adf contextual bandit algorithm and exploring with the softmax explorer algorithm with lambda = 10 and plotting ctr vs num_iterations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwcd3nn8c8z92hOSaPRMbot2ZKMkS0PPsD4wAEfBBRIgm0IEIfE8S5mYTck2MtCSCC2CZAEx3YUL3EcCLHZJYTIiYhNYo5gEFiWDx2WbJ2j0egYzX2fT/6oaqlr1CPNWNM1I9X3/Xr1a7qOrn5U3apv16+qfmXujoiISErOZBcgIiJTi4JBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkIrZgMLNHzeyomW0dZbqZ2QNmtsvMXjazNXHVJiIiJ8S5x/AYcOMppt8ELA8fdwB/FUNNIiIyQmzB4O4/BppPMcta4Ose2AhUmtnceKoTEZGUvMkuIE0NcCBtuD4cd2jkjGZ2B8FeBSUlJZeuWLEilgJFRM4Vzz///DF3n5Vp2lQKBsswLmN/He7+CPAIQG1trW/atCmbdYmInHPMbP9o06bSWUn1wIK04flAwyTVIiKSWFMpGNYDHwrPTroCaHP3k5qRREQku2JrSjKzx4FrgSozqwf+EMgHcPd1wAbgZmAX0A3cHldtIiJyQmzB4O63nWa6Ax+NqRwRERnFVGpKEhGRKUDBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiVAwiIhIhIJBREQiFAwiIhKhYBARkQgFg4iIRCgYREQkQsEgIiIRCgYREYlQMIiISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEqFgEBGRCAWDiIhEKBhERCRCwSAiIhEKBhERiYg1GMzsRjPbaWa7zOzuDNMrzOxJM3vJzLaZ2e1x1iciIjEGg5nlAg8BNwGrgNvMbNWI2T4KbHf31cC1wFfMrCCuGkVEJN49hsuAXe6+x937gSeAtSPmcaDMzAwoBZqBwRhrFBFJvDiDoQY4kDZcH45L9yCwEmgAtgAfd/fhkQsyszvMbJOZbWpsbMxWvSIiiRRnMFiGcT5i+AbgRWAecDHwoJmVn/Qi90fcvdbda2fNmjXxlYqIJFicwVAPLEgbnk+wZ5DuduA7HtgF7AVWxFSfiIgQbzA8Byw3syXhAeVbgfUj5qkDrgcws9nABcCeGGsUEUm8vLjeyN0Hzewu4CkgF3jU3beZ2Z3h9HXA54HHzGwLQdPTp9z9WFw1iohIjMEA4O4bgA0jxq1Le94AvCPOmkREJEpXPouISISCQUREIhQMIiISoWAQEZEIBYOIiEQoGEREJELBICIiEQoGERGJUDCIiEiEgkFERCIUDCIiEpHoYHh+fzPtvQOTXYaIyJSS2GDo6R/iV//qZ9zx9U2TXYqIyJSS2GDoHwruGLrtYPskVyIiMrUkNhiOy3TDURGRBEtuMIy827SIiAAJDgYPk0E7DCIiUYkNhhQzRYOISLrEBoOrKUlEJKPEBkOKdhhERKISHwwiIhKV2GBQS5KISGaJDYYUtSSJiEQlNhhcR59FRDJKbDCk6HRVEZGoxAZDan9Bew4iIlHJDYYwD7THICISldxg0HlJIiIZJTYYUrmgpiQRkajkBkNIsSAiEpXYYFAgiIhkltxg8OhfEREJJDYYUnSMQUQkKrHBoLOSREQyizUYzOxGM9tpZrvM7O5R5rnWzF40s21m9qNs1aIdBRGRzPLieiMzywUeAt4O1APPmdl6d9+eNk8l8DBwo7vXmVl1tutSPoiIRMW5x3AZsMvd97h7P/AEsHbEPO8HvuPudQDufjRbxfhJT0REBOINhhrgQNpwfTgu3fnAdDP7oZk9b2YfyrQgM7vDzDaZ2abGxsbXVYwOOouIZBZnMGTqlGjk1jkPuBR4J3AD8BkzO/+kF7k/4u617l47a9asMypK8SAiEhXbMQaCPYQFacPzgYYM8xxz9y6gy8x+DKwGXp3oYlxdYoiIZBTnHsNzwHIzW2JmBcCtwPoR8/wz8FYzyzOzacDlwCvZKKapqx+Arv6hbCxeROSsFdseg7sPmtldwFNALvCou28zszvD6evc/RUz+zfgZWAY+Jq7b81GPRv3NGVjsSIiZ704m5Jw9w3AhhHj1o0Y/hLwpezXku13EBE5OyX2ymcREclMwSAiIhGJDQb1lSQikllig0FERDJLbDDo4LOISGaJDYZ0Ow63T3YJIiJThoIBuPEv/nOySxARmTISGwxzyosmuwQRkSkpscGwal75ZJcgIjIljSkYzCzfzA6b2YXZLkhERCbXmILB3QeAAc7RXqrnVahZSUQkZTxNSX8J3GNmsfavFIeGtt7JLkFEZMoYz0b+rcA1wEEz2wp0pU9093dPZGHZpusYREQyG08wHAP+MVuFiIjICf2Dw+xv6qKpq5+6pm7qmrvpGRiivqWb/eHwh9+8mE/duGLC33vMweDut0/4u4uIAEc7etlS38aKueXUVBZPdjmx6Okf4kBLN40dfexr6mJPYxdHO/o43NbD4fZejrT10T80fHz+HIOi/FzmlBexpKqEN59XxRVLZ2altjEHg5k9A7zX3VtHjC8Hvuvub5vo4rJJneiJxKt3YIj9Td00tPawraGNQ2297DjcQUNrD4fC43xmULtoOmsvruGdF81leknBJFf9+gwODdPc3c/R9j52Hu5gz7FOdh7uoGdgiKPtfRxp76W9dzDymqL8HKrLiphXWcTFC6ZTc1ExK+aUMaOkgIUzprFwxjRyciyW+sfTlHQtkOlTKiI4/iAigruz91gXdc3d7G7s4rUjHWyua2HvsS4Ghk78ICsvymPFnHKuXDqTC+aU8YaaCp7f38L6lxr4P9/dyh89uY2LF1Ry5dKZXLeimjfOryQ3pg3jqbg7B5p72N/cRVvPAPuOBc09TZ397G/uZn9TF63dA5HX5Bgsqy6ltDCPJVUlXHneTKrLClkwYxrVZUXMn15MTWVxbBv+0zltMJjZmrTBN5pZc9pwLnADcHCiCxORqa+1u5/djZ1sPdjOloNt1DV3s+9Y0CSSUl6Ux5pF07luRTWr5pYzu7yIi2oqKCk8efPzlmVVfOxty9h+qJ31LzWwcXcTD/1wNw88s4s55UXcdNEc3nvJfN5QU47ZxG1Eh4edrv5BjrT3sb+piyPtfRzt6KWxo4/Gjj4Gh53u/kGau/qpb+mhe8S94lO/9hfMKObmi+Yyu6yIymn5VJcVsqy6lEUzSyjIO3uuJx7LHsMmgusXHHg6w/Qe4GMTWZSITD2NHX1sPdjGxj1NbG1oY39TNwdbe46f4VdVGjR5XLWsiksXT2d5dRlLZ5Uws6RgXBtxM+PCeRVcOK8CgKbOPn78WiMbthzmmxvr+Ntn9zF/ejFXLavi7atm85ZlVRTl555ymf2DwzR29nG0vZfth9p57UgnbT0DHGztoaG1hyPtvZG9maAOmD6tgFmlheTlGkX5ucfb9pdVl7K8upSyonwWV01jWsG5dRb/WP41SwAD9gBvIjg7KaUfOOruQ5leOJWlvswXzitnW0M7w8M+ZXbjRCbT0LBzoLmb+pYeth9q45VDHTy3r5n6lh4A8nONVXPLWbNwOr9+6QIunFfOynnlzKsomtBf8SkzSwt5zyXzec8l82np6uf7rxzh6W2H+dcth3jiuQMU5OWwcMY0Vs4tZ055IWZGd/9g0NzT1MWhtl76BocjyywtzKOiOJ+5FUVcumg6cyuKqZyWz+zyQuZPn0ZNZTEzSgpOGzjnqtMGg7vvBzCz+4E3ufu69OlmdqeZ1bj7Z7JUY1aldu/6h4Ypyknml0CSra17gJfqW9lc18ILda1sPdhGU1f/8emzygpZs7CSD1y+iIsXVLJ6QcWk/UKeXlLA+2oX8L7aBfQPDvPs7mP8dNcx9h7rZvP+Fho7+8ChpDCXORXFvKGmgutXzqaiOJ+q0kJmlxeydFYpi2dOy0qInSvG8+n+BvDrGcZvBu4BzspgyAv3Enr6hxL760CSo6d/iIOt3ew43MHm/a08X9fC1oNtDA07ZnDB7DKuPn8WVy6dyeyKIlbOLaO6bGp2GVOQl8N1F1Rz3QXVkfHuro3+GRpPMFQDjRnGHwNmT0w58XtuXwsAf/OTvXzyhgsmuRqRidXeO8Bze5vZXNfCT147xpaDbQyHzah5OcbFCyr5b9ecx2VLZrB6QSUVxfmTW/AEUCicufEEQx3Baal7Roy/GqifsIomyYM/2KVgkLNaT/8Qdc3dbD/UxksH2thc18L2hnYGh50cgzULp3PnNeexeGYJK+eWs3x2qfaSJaPxBMNfA39uZgXAM+G464H7gC9OdGEicmp1Td1s3NPECweCYwM7j3QcP6miKD+HFXPK+Z2rl3LVsipWL6ikNMPpoSKZjKdLjK+YWRXwACcudOsHvuruf5qN4kQk0NYzwN5jXexp7GTvsS6+v/0IOw53AFBSkMuaRdN5+6rZLKsuZVl1KSvmlE+Ji8Hk7DSunxDufo+ZfQFYRXAK63Z378xKZVmW+mX1+zdcwJee2jm5xYikGR529hzrYvuhdrbUt7KtoZ1N+1qO95uTY1C7aAb33LSC61ZUc96sUoWATKhx71u6exfwXBZqmRTnzy6b7BLkLNDS1c+xzj7ycnOoKM6nrCiPgaFhWrsHKMjLYfq0gjPaOB9o7uYHO4/yQl0rP3q1kebwdNGC3BzOn1PKB69cxBVLZzKvsohFM0vULCRZlfhvlwFXLJ0RuYRfzj0HW3vo7B3EcWaWFDKrrBAITm3s7h+ipbufA809tPX009Day6G2HgaGnMaOPl480MrB1p7I8nJzjKHhE1fKmsG8imLMgguyCsIrZd+9eh7vWj0v40Hezr5BfvJaI999oYGntx9m2KFyWj61i6Zz/crZXFRTwbJqHSCW+CU+GABmlBSwua6Vo+29VJdPzXO24zA4NMwrhzr4wc6jvK92Aa8cbqelq5/CvFymFeZySXg6Y+p0wPqWbo529HHx/MqMV40PDzvHOvsYGHbmlhfRPzTMC3WtlBfnceG8Ctp7B2ju7CfHjDkVRWw52Mbuxk6qSgtYNquM5u5+SgtzqS4vwoehrCiPox19DA4Pk5tjNHf10zswTEfvAPUtPQwNO209A7T3DPDq0bDLg5Ye+geHTurJsqwwj6qyQuqauyMb+BQzKAz3BC5ZWMn7L1/I/OnF9A0EXSu09wxQXJDLjJICegeGaGjtpbmrHyfowmFgaJj6lh5+/9sv8ycbXmHNwulctmQGZUV5HGnr5eWDbfzktWMMDjuV0/L5nauXckvtAhbPLNEV+DLpEhsM6d1uF+Xl0j84zGX3/gd777uZzr5ByorO/vO5R0pd+NPRO8AffPtlfran6aReIFP+7Puvjrqc0sI8OvtObGjzcozBYWf6tHyGPThQOplyDJZXl1FalMebz5tJcX4uNdOLmVtRREFeDvUtPdQ1ddPc3c+1F8yiojifmSUFLKkqpawoj7kVRcwqKzzj8+HdnWd3NfFPLxxkc10Lz+w4enxadVkhH7pyMdevrObSRdO1VyBTSmKDIcUMCtP+Uy65ZwMA7798Ife+56LJKmtUma7qdHce/uFuqkoLmFFSyO9+YxPpP4JnlhREujg4laWzSmjs6KOnf4gPXrmI2eVFfO0/93Kss49Vc8vZfqj9eCjMrSjiXavnsa2hjWd3NdEShswb51cwrSCXPY1dXDCnjNXzK/n53iZWzS0nPzeHtp4BXjjQyoLpxSyaWcK+pi76BoZZs6iSJVWl/OjVRiqK8yjMyyUvx9jd2En/kLNwRjFLq0oxC/rzSbXrlxTmMa+yiLLCfMqL8yguyKUwb/I3tGbGVcuruGp5FQCH2nqC5qLi/Iw9i4pMFfp2Ao//ou6kcf/w87rXFQwdvQP8Ym8z168c+8XgnX2D7G3s4tndx9hxqJ0Vc8u5/3s7gKAt+1M3XsDai2u4/N7/GHc9wKih8OErF3HLmxaycm7ZKX8d33nNea/rfV+vX7t0fqzvF5e5Fcm4M5mc/RQMr1OmX+59g0Nc9LkTPZM/cccVFOTl8MMdR2nq6udPRgSNux/fQ4l4seH406Fh594NO7h3w44x17Z6fgVVpYXcc/NK/vO1Rv7oye383tvP56PXLVP7tYicVmKDwcdwZ8/Fd/8rAL/1liU8+uxefmllNf/+yol24netnsdf3nYJP9hxlNsfO/kM3lsf2RgZ/ubPgz2T7X98AzlmrPjMv4363ktnlfDArZfwy3/5k8j43ffeTI4F9efkGEc7eplVOnp7+LLqUm5/y5LT/2NFRELmY9lCTtSbmd0IfJXgzm9fc/f7R5nvTcBG4BZ3//aplllbW+ubNm0ady0vHWhl7UPP8uhv1tLZN8T/ePwFIOhiuKaymBcPtJ5mCYHPvWsVn3tye2TcT+9+G2++/5lRXnGynV+4cUq0iYtIcpjZ8+5em2labHsMZpYLPAS8naDTvefMbL27b88w3xeBp+Kq7R2rZrP24nn8z186n8VVJXT0DkSahE4lPRR+7dL5fPnXVwOw7/53AkFTUHd/cJbTxj1NJ+1F/Oj3r1UoiMiUEmdT0mXALnffA2BmTwBrge0j5vsY8I8Ed4vLmvT9pKL8XL566yXHh8uK8nnyrqvYXNfCjsMdPP6LOj54xSK+sXE/f37Lat5zyXx6+odY+dkTTUGpIBgpN8eOn/p6xdKZ7Lv/nXT3D5JjplMURWRKijMYaoADacP1wOXpM5hZDfAe4G2cIhjM7A7gDoCFCxeeUVFG5rb5i+ZXcNH84J6z9703OGj8+V95w/HpxQW5PHv323jpQCtvXzW+21Gca/eHFZFzS5xbqExb4JEHOP4C+JS7D53q9El3fwR4BIJjDBNW4TjVVBZTU6lTEEXk3BJnMNQDC9KG5wMNI+apBZ4IQ6EKuNnMBt39u/GUKCIicQbDc8ByM1sCHARuBd6fPoO7Hz+v0sweA/4lW6EQ59lYIiJnk9iCwd0HzewugrONcoFH3X2bmd0ZTl8XVy0Rut5LRCQi1qOg7r4B2DBiXMZAcPffjKMmERGJypnsAiaLGpJERDJLbDCkqCVJRCQq8cEgIiJRCgYREYlIbDDobFURkcwSGwwpZ3r7RhGRc03ig0FERKIUDCIiEpHgYNBBBhGRTBIcDAEdYRARiUp8MIiISFRig0Gnq4qIZJbYYEjR2aoiIlGJDwYREYlSMIiISERig0GHGEREMktsMKSYTlgVEYlIfDCIiEiUgkFERCISGwy6jkFEJLPEBkOKrmMQEYlKfDCIiEhUYoPB1ZYkIpJRYoMhRS1JIiJRiQ8GERGJUjCIiEhEYoNBRxhERDJLbDAcp4MMIiIRCgYREYlIbDDobFURkcwSGwwp6l1VRCQq8cEgIiJRCgYREYlIbDC4TlgVEcko1mAwsxvNbKeZ7TKzuzNM/4CZvRw+fmpmq7NfU7bfQUTk7BJbMJhZLvAQcBOwCrjNzFaNmG0vcI27vxH4PPBIXPWJiEggzj2Gy4Bd7r7H3fuBJ4C16TO4+0/dvSUc3AjMj7E+EREh3mCoAQ6kDdeH40bzEeB7mSaY2R1mtsnMNjU2Nr6+anSIQUQkoziDIVNrfsbNs5ldRxAMn8o03d0fcfdad6+dNWvWhBclIpJkeTG+Vz2wIG14PtAwciYzeyPwNeAmd2+KqTYREQnFucfwHLDczJaYWQFwK7A+fQYzWwh8B/igu7+azWLUkiQikllsewzuPmhmdwFPAbnAo+6+zczuDKevAz4LzAQetuA80kF3r81mXabzVUVEIuJsSsLdNwAbRoxbl/b8t4HfjrMmERGJSuyVzyIikllig0HdbouIZJbYYEjRIQYRkajEB4OIiEQpGEREJCKxwaBut0VEMktsMKToEIOISFTig0FERKISGww6XVVEJLPEBkOKTlcVEYlKfDCIiEiUgkFERCISGww6xCAikllig+EEHWQQEUmnYBARkYjEBoPrfFURkYwSGwwpOl1VRCQq8cEgIiJRCgYREYlIbDDoCIOISGaJDYYUHWIQEYlKfDCIiEiUgkFERCKSGww6yCAiklFygyFkupBBRCQi8cEgIiJRiQ0GV1uSiEhGiQ2GFDUkiYhEJT4YREQkSsEgIiIRiQ0G9botIpJZYoMhRWeriohEJT4YREQkSsEgIiIRiQ0GHWMQEcks1mAwsxvNbKeZ7TKzuzNMNzN7IJz+spmtyXpNupJBRCQitmAws1zgIeAmYBVwm5mtGjHbTcDy8HEH8FfZqqdvcDhbixYROavFucdwGbDL3fe4ez/wBLB2xDxrga97YCNQaWZzs1HM7sZOAPLztMcgIpIuL8b3qgEOpA3XA5ePYZ4a4FD6TGZ2B8EeBUCnme18nTVVrfwix17na7OpClTXOE3V2lTX+Kiu8TmTuhaNNiHOYMj003zkIeCxzIO7PwI8csYFmW1y99ozXc5EU13jN1VrU13jo7rGJ1t1xdmUVA8sSBueDzS8jnlERCSL4gyG54DlZrbEzAqAW4H1I+ZZD3woPDvpCqDN3Q+NXJCIiGRPbE1J7j5oZncBTwG5wKPuvs3M7gynrwM2ADcDu4Bu4PYsl3XGzVFZorrGb6rWprrGR3WNT1bqMteVXiIikiaxVz6LiEhmCgYREYlIbDCcrnuOCX6vBWb2AzN7xcy2mdnHw/GfM7ODZvZi+Lg57TX3hLXtNLMb0sZfamZbwmkPmJ15x+Fmti9c5otmtikcN8PMvm9mr4V/p8dZm5ldkLZeXjSzdjP7xGSsMzN71MyOmtnWtHETtn7MrNDMvhWO/7mZLT6Dur5kZjvCLmX+ycwqw/GLzawnbb2ti7muCfvcJriub6XVtM/MXpyE9TXa9mHyvmPunrgHwcHv3cBSoAB4CViVxfebC6wJn5cBrxJ0C/I54JMZ5l8V1lQILAlrzQ2n/QK4kuCaj+8BN01AffuAqhHj/hS4O3x+N/DFyagt7fM6THBBTuzrDLgaWANszcb6Af47sC58fivwrTOo6x1AXvj8i2l1LU6fb8Ry4qhrwj63iaxrxPSvAJ+dhPU12vZh0r5jSd1jGEv3HBPG3Q+5++bweQfwCsEV3aNZCzzh7n3uvpfgLK3LLOgepNzdf+bBJ/x14FeyVPZa4O/C53+X9j6TUdv1wG5333+aerNSl7v/GGjO8H4TtX7Sl/Vt4Pqx7NVkqsvdn3b3wXBwI8G1QKOKq65TmNT1lRK+/n3A46daRpbqGm37MGnfsaQGw2hdb2RduAt3CfDzcNRd4W7/o2m7iqPVVxM+Hzn+TDnwtJk9b0F3IwCzPbyGJPxbPUm1QfALJ/0/7FRYZxO5fo6/JtyotwEzJ6DG3yL41ZiyxMxeMLMfmdlb0947rrom6nPLxvp6K3DE3V9LGxf7+hqxfZi071hSg2FMXW9M+JualQL/CHzC3dsJeo89D7iYoD+or5ymvmzV/RZ3X0PQu+1HzezqU8wba20WXAz5buD/h6OmyjobzeupY8JrNLNPA4PAN8NRh4CF7n4J8L+AfzCz8hjrmsjPLRuf6W1Ef3zEvr4ybB9GnXWU95mw2pIaDLF3vWFm+QQf+jfd/TsA7n7E3YfcfRj4vwRNXKeqr55o08CE1O3uDeHfo8A/hXUcCXdNU7vPRyejNoKw2uzuR8Iap8Q6Y2LXz/HXmFkeUMHYm2JOYmYfBn4Z+EDYpEDY7NAUPn+eoF36/LjqmuDPbaLXVx7wXuBbafXGur4ybR+YxO9YUoNhLN1zTJiwLe9vgFfc/c/Sxqd3Kf4eIHW2xHrg1vBMgiUE96f4Rbg72WFmV4TL/BDwz2dYW4mZlaWeExy83BrW8OFwtg+nvU9stYUiv+SmwjpLe7+JWj/py/o14JnUBn28zOxG4FPAu929O238LAvuiYKZLQ3r2hNjXRP5uU1YXaFfAna4+/FmmDjX12jbBybzO3aqI9Pn8oOg641XCX4JfDrL73UVwW7by8CL4eNm4BvAlnD8emBu2ms+Hda2k7SzaIBagv9Uu4EHCa9eP4PalhKc4fASsC21LgjaH/8DeC38O2MSapsGNAEVaeNiX2cEwXQIGCD45fWRiVw/QBFBU9kugrNKlp5BXbsI2pJT37PUmSi/Gn6+LwGbgXfFXNeEfW4TWVc4/jHgzhHzxrm+Rts+TNp3TF1iiIhIRFKbkkREZBQKBhERiVAwiIhIhIJBREQiFAwiIhKhYBA5DQt62nQzm1I3g5+qdcnZT8EgcnoHCHrATHXJfG24Qa6KqwAz+6GZPXiqukQmSmz3fBY5W7n7EEG33xPOzPLdfeD1vDabdUmyaY9BppTwl/HDZnavmR2z4MYqXzaznHD6PjP7ZIbXPJg2vM/MPmtmj5lZh5kdMLNbzKzSzJ4ws04Lbn7yjjHWdLzJJuz98gfhpMZw/GPhfGZmf2Bmuy24ycsWM/uNDMu5zcyeMbMe4HfNbKaZPW5m9eHrtpnZ7Wmvewy4hqCDQw8fizM1JZnZ1RbciKXXzI6Y2Z+H3b6Maf2G87zXgl5Qe8ys2YLeRWePZV3JuUHBIFPRBwh6Bn0zcBfwCeCWcS7jEwSX/q8B/h9BX/T/AGwg6OHzx8Dfm1nROJd7gKC7BIALCZpyPh4Of4Gg+4ePEtxM5T7gr83snSOWcR/wcDjPdwm6K9hM0PHdhcBXw9ddH87/ceBnwN+G7zeXaLfLAJhZDUE32y8QdN38EYK+pu4bMeuo69fM5hDcn+TvgJUEN7f5xhjWi5xLzqQvGz30mOgH8EPgZyPGfR/4Wvh8HyPuBBa+5sG04X3A42nDpQR90TyQNm5xOK52DDVF5gWuDYer0uYpAXqAt4547V8AG0Ys5/fG8J5PpP7Nmf6No9T1JwR94eSkzfObQB8wbYzrd024zEWT/V3QY/IeOsYgU9HLI4YbOHGTknEvw907zayboBO3lCPh3/EudzSrCH75/5uZpXdAlk8QVOk2pQ+EvXjeTfCrvYbglo0FBBvx8VhJsKrf9qwAAAINSURBVNEfThv3k3BZyzixTk61fl8C/h3YamZPh8+/7e6N46xFzmIKBpmKRh6MdU40ew5z8k1H8se4jIERwzBxzamp5bwLqDtNLV0jhj8J/B5Bk9EWoBO4l/GHljH6zVfSx4+6ft19KDz2cgVBF+wfAe4zs2vc/aVx1iNnKQWDnG0aCdrYAQiPEawgaFePS3/4Nzdt3HaCJptF7v7MOJd3FfCku38DjvfPfz7QOuI9czO8Nt124H1mlpO213BV+NrdYy3G3Z3gmMbPzOyPCbqfvoVgb0ISQMEgZ5tngN8ys/UEIfFpMu8xZNN+gl/Z7zSzJ4Eed+8wsy8DXw437D8mOLZxBTDs7o+cYnmvAreY2VXAMeBjwBKiYbeP4Ibviwn2KDLdfethggPJD5vZVwnutXE/wbGJ7gzzn8TMriC4cc1TBM1tlxDc+Wv7WF4v5wadlSRnm/sIwuGfgacJ2tA3x1mAux8E/pDgYO8RghuiAHwG+BxB09A2goO6vwrsPc0iv0BwBtX3CAKlixP3ak75MsEv/+0EgbhwlLpuItiYvwg8SnBzmv89jn9eG/AW4F8IbhDzFeDz7v7341iGnOV0ox4REYnQHoOIiEQoGCTxzGxdeDV0pse6ya5PJG5qSpLEM7NqoHyUye3ufjTOekQmm4JBREQi1JQkIiIRCgYREYlQMIiISISCQUREIv4Lv9KXZYH6tCkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vw = pyvw.vw(\"--cb_explore_adf --quiet --softmax --lambda 10\")\n",
    "ctr = run_simulation_multiple_cost_functions(vw, num_iterations_per_cost_func, users, times_of_day, actions, cost_functions, epsilon)\n",
    "plot_ctr(total_iterations, ctr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}