sprt.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:5fbf64b15aac770b9e355b16eecb59d9a79eecf9c42519120c159233ccd2b4d1"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<p class=\"title\">Wald's Sequential Probability Ratio Test</p>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Sequential test: draw data sequentially\n",
      "\n",
      "Sequence of data $X_1, X_2, \\ldots$. \n",
      "\n",
      "Two hypotheses, $H_0$ and $H_1$, each of which completely specifies the joint distribution of the data.\n",
      "\n",
      "Assume that the joint distributions under $H_0$ and $H_1$ are absolutely continuous with respect to each other, relative to some underlying measure.\n",
      "\n",
      "Let $f_{0m}$ be the likelihood of $H_0$ for data $(X_j)_{j=1}^m$ and let $f_{1m}$ be the likelihood of $H_1$ for data $(X_j)_{j=1}^m$.\n",
      "\n",
      "The likelihood ratio of $H_1$ to $H_0$ is $f_{1m}/f_{0m}$; this is (looselt speaking) the probability of observing $X_1, \\ldots, X_m$ if $H_1$ is true, divided by the probability of observing $X_1, \\ldots, X_m$ if $H_0$ is true.\n",
      "\n",
      "The probability of observing the data actually observed will tend to be higher for whichever\n",
      "hypothesis is in fact true, so this likelihood ratio will tend to be greater than $1$ if $H_1$ is true,\n",
      "and will tend to be less than $1$ if $H_0$ is true.\n",
      "The more observations we make, the more probable it is that the\n",
      "resulting likelihood ratio will be small if $H_0$ is true.\n",
      "Wald (1945) showed that if $H_0$ is true, then the probability is at most $\\alpha$\n",
      "that the likelihood ratio\n",
      "is ever greater than $1/\\alpha$, no matter how many observations are made.\n",
      "More generally, we have:\n",
      "<p class=\"gap01\"></p>\n",
      "\n",
      "<div class=\"theorem\"> Wald's SPRT\n",
      "\n",
      "<p>For any $\\alpha \\in (0, 1)$ and $\\beta \\in [0, 1)$, the following sequential \n",
      "    algorithm tests the hypothesis $H_0$ at level no larger than \n",
      "   $\\alpha$ and with power at least $1-\\beta$\n",
      "   against the alternative $H_1$.\n",
      "</p>\n",
      "\n",
      "<p>\n",
      "   Set $m=0$.\n",
      "</p> \n",
      "\n",
      "   <ol>\n",
      "     <li> Increment $m$\n",
      "     <li> If $\\frac{f_{1m}}{f_{0m}} \\ge \\frac{1-\\beta}{\\alpha}$, reject $H_0$ and stop.</li>\n",
      "     <li> If $\\frac{f_{1m}}{f_{0m}} \\le \\frac{\\beta}{1-\\alpha}$accept $H_0$ and stop.</li>\n",
      "     <li> If $\\frac{\\beta}{1-\\alpha} < \\frac{f_{1m}}{f_{0m}} < \\frac{1-\\beta}{\\alpha}$, go to step 1.  </li>\n",
      "   </ol>\n",
      "  \n",
      "</div>\n",
      "\n",
      "## SPRT miracle\n",
      "\n",
      "Don't need to know the distribution of the likelihood ratio $\\mbox{LR}=\\frac{f_{1m}}{f_{0m}}$ under the null hypothesis\n",
      "to find the critical values for the test.\n",
      "\n",
      "## Connection to Gambler's ruin\n",
      "\n",
      "For iid data, the likelihood ratio is a product of terms. On a log scale, it's a sum.  Each \"trial\" produces another term in the sum, positive or negative. But&mdash;unlike the classical Gambler's Ruin problem in which the game is fair&mdash;the terms are not equal in magnitude: the steps are not all the same size.\n",
      "\n",
      "\n",
      "\n",
      "## Likelihood ratio for two values of $p$ in iid Bernoulli trials\n",
      "\n",
      "Suppose $X_1, X_2, \\ldots,$ are iid $\\mbox{Bernoulli}(p)$ random variables. \n",
      "Start with $\\mbox{LR} = 1$ and $j = 0$.\n",
      "\n",
      " + Increment $j$\n",
      " + If $X_j = 1$, $\\mbox{LR} \\leftarrow \\mbox{LR} \\times p_1/p_0$. \n",
      " + If $X_j = 0$, $\\mbox{LR} \\leftarrow \\mbox{LR} \\times (1-p_1)/(1- p_0)$.\n",
      "\n",
      "What's $\\mbox{LR}$ at stage $m$?\n",
      "Let $T_m \\equiv \\sum_{j=1}^m X_j$.\n",
      "$$\n",
      "     \\frac{p_{1m}}{p_{0m}} \\equiv \n",
      "         \\frac{p_1^{T_m}(1-p_1)^{m-T_m}}{p_0^{T_m}(1-p_0)^{m-T_m}}.\n",
      "$$\n",
      "This is the ratio of binomial probability when $p = p_1$ to binomial probability when\n",
      "$p = p_0$ (the binomial coefficients in the numerator and denominator cancel).\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "<div class=\"callout\">\n",
      "\n",
      "<p class=\"subtitle\">Wald's SPRT for $p$ in iid Bernoulli trials</p>\n",
      "\n",
      "Conclude $p \\le p_0$ if \n",
      "$$\n",
      "     \\frac{p_{1m}}{p_{0m}} \\ge \\frac{1-\\beta}{\\alpha}.\n",
      "$$\n",
      "Conclude $p > p_0$ if\n",
      "$$\n",
      "     \\frac{p_{1m}}{p_{0m}} \\le \\frac{\\beta}{1-\\alpha}.\n",
      "$$\n",
      "Otherwise, draw again.\n",
      "\n",
      "</div>\n",
      "\n",
      "The SPRT approximately minimizes the \n",
      "expected sample size\n",
      "when $p \\ge p_0$ or $p < p_1$.\n",
      "For values in $(p_1, p_0)$, it can have larger sample sizes than fixed-sample-size tests.\n",
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Simulation of SPRT for Binomial\n",
      "\n",
      "Let's watch the SPRT in action for iid Bernoulli trials."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# This is the first cell with code: set up the Python environment\n",
      "%matplotlib inline\n",
      "import matplotlib.pyplot as plt\n",
      "import math\n",
      "import numpy as np\n",
      "import scipy as sp\n",
      "import scipy.stats\n",
      "from scipy.stats import binom\n",
      "import pandas as pd\n",
      "# For interactive widgets\n",
      "from IPython.html.widgets import interact, interactive, fixed\n",
      "from IPython.html import widgets\n",
      "from IPython.display import clear_output, display, HTML"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 19
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def plotBinomialSPRT(n, p, p0, p1, alpha, beta):\n",
      "    '''\n",
      "       Plots the progress of the SPRT for n iid Bernoulli trials with probabiity p\n",
      "       of success, for testing the hypothesis that p=p0 against the hypothesis p=p1\n",
      "       with significance level alpha and power 1-beta\n",
      "    '''\n",
      "    fig, ax = plt.subplots(nrows=2, ncols=1, sharex=True)\n",
      "    trials = sp.stats.binom.rvs(1, p, size=n+1)  # leave room to start at 1\n",
      "    terms = np.ones(n+1)\n",
      "    sfac = p1/p0\n",
      "    ffac = (1.0-p1)/(1.0-p0)\n",
      "    terms[trials == 1.0] = sfac\n",
      "    terms[trials == 0.0] = ffac\n",
      "    terms[0] = 1.0\n",
      "    logterms = np.log(terms)\n",
      "    #\n",
      "    ax[0].plot(range(n+1),np.cumprod(terms), color='b')\n",
      "    ax[0].axhline(y=(1-beta)/alpha, xmin=0, xmax=n, color='g', label=r'$(1-\\beta)/\\alpha$')\n",
      "    ax[0].axhline(y=beta/(1-alpha), xmin=0, xmax=n, color='r', label=r'$\\beta/(1-\\alpha)$')\n",
      "    ax[0].set_title('Simulation of Wald\\'s SPRT')\n",
      "    ax[0].set_ylabel('LR')\n",
      "    ax[0].legend(loc='best')\n",
      "    #\n",
      "    ax[1].plot(range(n+1),np.cumsum(logterms), color='b', linestyle='--')\n",
      "    ax[1].axhline(y=math.log((1-beta)/alpha), xmin=0, xmax=n, color='g', label=r'$log((1-\\beta)/\\alpha)$')\n",
      "    ax[1].axhline(y=math.log(beta/(1-alpha)), xmin=0, xmax=n, color='r', label=r'$log(\\beta/(1-\\alpha))$')\n",
      "    ax[1].set_ylabel('log(LR)')\n",
      "    ax[1].set_xlabel('trials')\n",
      "    ax[1].legend(loc='best')\n",
      "\n",
      "\n",
      "interact(plotBinomialSPRT, n=widgets.IntSliderWidget(min=5, max=300, step=5, value=100),\\\n",
      "         p=widgets.FloatSliderWidget(min=0.001, max=1, step=0.01, value=.45),\\\n",
      "         p0=widgets.FloatSliderWidget(min=0.001, max=1, step=0.01, value=.5),\\\n",
      "         p1=widgets.FloatSliderWidget(min=0.001, max=1, step=0.01, value=.6),\\\n",
      "         alpha=widgets.FloatSliderWidget(min=0.001, max=1, step=0.01, value=.05),\\\n",
      "         beta=widgets.FloatSliderWidget(min=0.001, max=1, step=0.01, value=.05)\n",
      "         )"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEZCAYAAACEkhK6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XdYFNf6B/Dv2omggqgYQEGKNBVUwIrITSwJtmBE0cRr\nSYwasSXGK96g3iSKBTFERU0wQUXF9oOQWDCKgJGioigWpAkiFqQLKML5/XHCysIuLAvLLvB+nmce\nmJkzZ87Mwrx7zpw5I2CMMRBCCCG1aKXoAhBCCGkaKGAQQgiRCgUMQgghUqGAQQghRCoUMAghhEiF\nAgYhhBCpUMAgMjt06BDGjh0rl7z//e9/47///a/M26upqSE1NbXhCiSFsrIyfPnll3j33Xfh7Ows\n133p6enhr7/+ErsuNDQUurq6ct0/aZkoYJAaXb9+HZ999hl0dXXRtWtXjBgxAlevXgUAzJw5E2fP\nnpXLfgUCAQQCgVRp7e3t8csvv4gsKygogJ6enhxKJtmVK1dw6dIlPHjwAEePHhVZl5mZiVatWuH5\n8+fCZd9//z1atWqFZ8+eiSwbP358rfuqy/nR09NDWlqalEfBMcawdetWjBo1Cp06dYK+vj6WL18u\nkuc777wDNTU1WFtb47///S9ev34NgH8eKioqUFNTg4WFBZYvX47nz5/jhx9+gJqaGtTU1KCiooI2\nbdoI5/v161en8hHFoIBBajRv3jyYm5sjPj4eGRkZcHd3R/v27Rtl39I+UyrthVPeIiIi0L9/f3Ts\n2LHaup49e8LQ0BCXLl0SLgsLC4OpqSnCwsJElo0aNapByyXL+QkKCoKfnx927tyJ/Px8hIaGYuDA\ngSJ5BgcHo6CgAD///DN+++03nDx5Urhu586dKCgoQFBQEP7++2/s2bMHa9asQUFBAQoKCuDj44Nh\nw4YJ52/dutVgx0vkhwIGkSghIQH37t3Dl19+iU6dOqFDhw54//33hd8Gf/31V4wcOVKYvlWrVjhw\n4AAsLS2hra2N7du348mTJxg7dix0dHTg7u6ON2/eiN22Yvvk5ORq5cjJyYGjoyO6d+8OIyMjfPvt\nt8Jv5W5ubggPD8eXX34JNTU1uLq6VsurqKgIu3fvRr9+/TBmzBj8/vvvwrx//fVXjBgxAhs2bIC2\ntjbGjRuHK1euSDwnGRkZWLt2LfT09DBnzhzExsYCANzd3eHu7o5jx45BTU0N+/fvr7atnZ2dMDiU\nlZUhNjYWS5cuFVkWGRkJOzs7JCcnw8HBAZqamujfvz88PDxQWFgotkylpaXYtWsX+vTpA1tbW9y9\ne1di+SMiIjB+/HhoaGhAR0cH27ZtE5suICAAU6dOhYWFBQCgd+/e+OSTT8SmHTBgAMaPH48//vij\n2ro+ffpgypQpIucc4F8GaJCJpocCBpHI2NgY2tramD59OoKDg5GXl1frNgcOHMDx48dx8OBBfP31\n15g5cybc3NwQFhaGAwcO1HgxloQxhnnz5iEtLQ1nzpxBdHQ0fvzxRwC8CWfkyJHCb7QVyyvz8PDA\nsWPHcOLECaxevRqurq4IDQ0Vro+JiQEA3L59G0OGDMGqVasklsXFxQV5eXmIiYnByJEjYW9vj5KS\nEqxfvx5r1qzB9OnTUVBQgDlz5lTbtnLAiI2NhampKRwcHESWlZaWwsbGBowxuLm5ITMzEwcPHkRA\nQACOHDkitkx79uyBr68vAgMDsXXrVnh5eYnUKlJSUtCrVy8AwPLly7FgwQJkZWUhPj4eDg4OYvMc\nO3YsfHx8sGXLFty+fVtsmooLfmxsLE6fPo0pU6ZUW5eQkICTJ09ixIgREs8paTooYJAaXbx4EUZG\nRli0aBF0dHSwbNkyZGdnS0y/cOFCGBoaYvTo0ejTpw8sLS1hZ2eHPn364L333pN4o7YmGhoamDJl\nCjp06AADAwN89dVXCAwMFElT07fVwMBArF69GsbGxnBwcMDMmTNx6tQp4fqOHTti7dq1UFdXx4IF\nCxAVFYWXL19WyycrKwsxMTHYtGkTunXrhrlz56Jfv344ffq0sAw1lcPOzg63b99GXl4ewsPDYWdn\nB0NDQzx//ly4bOjQoWjTpg0MDAzwr3/9C23btkX//v2xcOHCasdc4c8//8SiRYvQr18/jBw5Es7O\nzhLLUV5ejqSkJOTn56Nz586wsrISm27WrFnYtWsXzp07h8GDB8PGxgbBwcHC9YwxTJ48GRoaGli6\ndCkWLVqEyZMnC9e5urqic+fOMDExgaamJrZs2SLxvJCmgwIGqZGuri42btyItLQ0nD9/HiEhIfDy\n8pKYfsCAAcLfe/ToUW0+IyOjzmUoLy/HmjVrMHLkSHTp0gVOTk64c+eOyEVRUjt9QUEB4uLiMGjQ\nIOGyQYMGITw8XDhvbm6OVq34v0LPnj3x5s0bPH36tFpekZGR6NOnj8g9isGDByMiIkKq49DT04O2\ntjbCw8MRHh4ubJIbNmwYwsPDERYWBjs7OwBAYWEhli5dCmtra3Tu3BnLly9HXFyc2Hyjo6NhaWkp\nnJcUBADAz88PN2/eRJ8+ffDxxx/j5s2bYtO1atUKkydPRkhICJ4/fw5HR0dMnz5d2CwmEAgQGBiI\n7OxshIWFYfXq1cJzKBAI4O3tLQyCf//9N/7880+pzhFRbhQwiNRsbW0xY8YMkeYcWWlra4tclCvu\nBVRWEQSOHTuGP/74A/v370dWVhZOnDgh8m2+devWKC8vF7sfNTU19O/fX9izCwCuXr0qvDDXxZAh\nQ5CcnCxS+6hompKWnZ0dLl26hCtXrmDYsGEAgJEjR+LSpUu4fPmysFw7d+7E/fv3ERAQgNzcXGzf\nvl3iMdrY2Iicv+vXr0vcv7m5Ofz8/JCZmYl+/fph/vz5tZZZTU0N//nPf8AYQ3R0tFTHWfHZDB8+\nHJs2bcKKFSsklp80HRQwiET379+Hp6cnMjIyUFZWhuvXr+PQoUP49NNPpc6jci2g8u8jRoxAeno6\nzp07h/T0dGzevLnadhXpHz9+jC5dukBTUxMJCQnw8PAQSTto0CDExsZKbIaZNGkStmzZgoSEBISG\nhuLw4cPC5pO60NTUhLW1NdasWYNnz57h119/RXx8fJ2eRbGzs4Ofnx+0tbWhqqoKgJ8LPz8/5Ofn\nY+jQocJjVldXR/fu3RETE4OffvpJYp4ffPABfHx8cPv2bURERODYsWNi05WWluLQoUPCe1EV3WLF\n2bFjBy5duoTi4mK8ePECW7duRffu3WFvby/1sVaYN28eXr58KbFcpOmggEEkUlNTQ1RUFGxtbaGh\noYEVK1bAxcVFGDCqPgsgrlmo6vqKeRUVFezduxdfffUVxo4di+nTp0tMO3fuXGhra8PY2BiffPIJ\n5s6dK5J21qxZSExMRLdu3bBs2bJqZVi1ahUmT56Mjz76CN9//z08PT2FXVfFPc9QUzfUQ4cO4Z13\n3oG1tTVCQ0Px119/QUVFRWJeVY0aNQrPnz8XuQk8YMAAlJSUYNCgQejQoQMAfnO6uLgYvXv3xsqV\nK7Fo0SKJeX/++eeYPXs2JkyYgK+++grLli2TmPbgwYPQ19dHr169EBUVJbF58Z133sHKlSvRs2dP\nDB48GA8ePICfn5+w2ak2lffftm1bLF26VORLQV2eIyHKQ6CoFyilp6fj008/xbNnz9CtWzd8/vnn\ncHFxQUFBAWbNmoXY2FgMHDgQBw8eFH4TI4QQojgKq2G0bdsW27dvR3x8PI4fP461a9eioKAAu3fv\nRq9evfDgwQPo6OjAx8dHUUUkhBBSicIChpaWlrBnh6amJszNzRETE4Po6GjMmzcP7du3x9y5cxEV\nFaWoIhJCCKlEKe5hJCYmIj4+HjY2NoiJiYGJiQkAwMTEROpeGYQQQuRL4QGjoKAAzs7O2L59O1RV\nVWm4AEIIUVJtFLnz0tJSODk54ZNPPsGkSZMAANbW1rh79y6srKxw9+5dWFtbV9vO0NAQSUlJjV1c\nQghp0gwMDJCYmCjz9gqrYVSMD2RhYSHSFdLW1ha+vr4oLi6Gr68vhgwZUm3bpKQkYT/9lj65u7sr\nvAzKMtG5oHNB56Lmqb5ftBUWMC5fvoyDBw/iwoULsLKygpWVFc6cOYOFCxciLS0Nffv2RUZGBr74\n4gtFFZEQQkglCmuSGjFihMShAiQNskYIIURxFH7Tm9SPLEM1NFd0Lt6ic/EWnYuGo7AnvetDIBAA\n6xRdCkIIaWLWSf8mS3GabMBogsUmpMXT0NBATk6OoovR7Kmrq4t9b019r50UMAghjYb+dxuHpPNc\n3/NP9zAIIYRIhQIGIYQQqVDAIIQQIhUKGIQQQqRCAYMQQohUKGAQQogYKSkpcss7MzMTr169klv+\n8kIBgxBCqkhOTkZkZKTc8tfQ0MCmTZvklr+8UMAghJAq9uzZgxkzZgjnIyIiREbVrq/27dvDyMgI\nR48elZimoKAA9+/fb7B9NgQKGIQQUsnNmzeho6MjnPf09IS3tzfy8vJkzvPGjRvw9/fHiRMn4OHh\nAQBwcXFBWFiYxG0CAgKgqqoq8z7lgQIGIYRUEhwcDAcHB+H8ihUr8MEHH8icX1xcHHJzc+Hi4gIn\nJyecPXtWuK6oqAhpaWlit0tPT4e2trbM+5UHChiEEFJJTEwMzMzMRJbVZziNq1evCkfMvXbtGiws\nLITr9PX1cfPmzWrb3Lt3DyYmJjLvU14U+opWQgipSrBe0CD5MHfZLvJFRUV8ROxKqs5LKzMzE8bG\nxnj48CF27dqF69evw8/PT7jewMAAGRkZ1bYLDAzE8uXLhfM5OTnYt28fDA0NERERAVdXV2hqaqKw\nsBBaWloylU0WFDAIIUpF1gt9QykpKam2TFwNY/PmzSguLhabx+zZs6Gnp4eoqCg4OjqiTZs28PDw\ngLe3N/bv3481a9YA4L2l0tPTRbYtKytDaWkp2rVrBwAoLi7GxIkT4e/vD11dXXTs2BFubm5wcnKC\no6NjfQ+3ThQaMObOnYs//vgD3bt3x61btwAA69atw88//4xu3boBADZu3Ihx48YpspiEkBakrKwM\nxcXFUFFRES4TV8NYtWpVrXmVlJSgTZu3l9mEhAQMGDBAOJ+amgp1dXWRbc6fP48xY8YI58+dOwcj\nIyPo6uoCAIyMjBATE4OpU6cKg0pjUeg9jDlz5uDMmTMiywQCAVasWIHY2FjExsZSsCCENCoDAwMk\nJyeLLJP1HkZQUJDw96ysLFy5cgXOzs7CZUlJSTA0NBTZJjIyEjY2NsL5p0+fiqSpeJ/IlClTZCpT\nfSg0YIwcObJadAXqd4OJEELqw97eHklJScJ5Ly8v+Pj4ICQkBG5ubsjPz5cqn/j4eMybNw++vr4I\nCAjAzp07cerUKaipqQnTPHr0CMOHDxfO5+bmVrsmTp8+HS9evMCRI0fg7u6Ohw8fwtbWFocOHcLr\n16/rebR1xBQsJSWFWVhYCOfXrVvHevfuzWxtbdmmTZtYfn5+tW2UoNiEEBk0hf/dnJwctnjx4nrn\nc/To0RrXZ2VlsSVLlogs27t3L8vMzKz3viWd5/qef6W76b1w4UJ8++23yM/Px9dff409e/bgq6++\nqpZu3bp1wt/t7e3pRe+EkAbRpUsXaGlp4dq1axg0aJDM+bRqVXMDjoeHBxYvXiyy7PHjxw3a6yk0\nNBShoaENlp/CX9GampqKCRMmCG96V3bz5k0sWrQIly9fFllOr3kkpGlqKv+7jDFs374dK1askEv+\nKSkpiImJwbRp04TLkpOTcfPmzQa5NyGvV7QqXQ0jMzMTPXv2xJs3b+Dv71+vJywJIUQWFZ1v5EVf\nXx/6+voiy/r06YM+ffrIbZ8NQaEBY8aMGbh06RKysrKgq6uL9evXIzQ0FDdu3EC7du1gZ2eHhQsX\nKrKIhBBC/qHwJilZNJVqLSFEFP3vNg55NUnRWFKEEEKkQgGDEEKIVChgEEIIkQoFDEIIIVKhgEEI\nIUQqFDAIIYRIhQIGIYQQqVDAIISQBpCSkiL3fWRmZuLVq1dy348kFDAIIaSekpOTERkZKff9aGho\nYNOmTXLfjyQUMAghpI4KCgpw//594fyePXswY8YMkTQRERFYtmxZg+63ffv2MDIywtGjRxs0X2lR\nwCCEEDFu3LgBf39/nDhxAh4eHiLrAgICoKqqCoCPqq2joyOy3tPTE97e3sjLy2vwcrm4uCAsLKzB\n85UGBQxCCKkiLi4Oubm5cHFxgZOTE86ePSuyPj09Hdra2gCA4OBgODg4iKxfsWKFXEfaLioqQlpa\nmtzyl4QCBiGEVHH16lXhS9muXbsGCwsL4bp79+7BxMREOB8TEwMzM7NqechzkEV9fX3cvHlTbvlL\nonTvwyCEtHACQcPkI+MFOzMzE8bGxnj48CF27dqF69evw8/PT7g+MDAQy5cvF84XFRVBIKbM4pZJ\nIycnB/v27YOhoSEiIiLg6uoKTU1NFBYWCt/GZ2BggIyMDJnyrw8KGIQQ5aLg4c+joqLg6OiINm3a\nwMPDA97e3ti/fz/WrFmDsrIylJaWol27dsL0JSUlYvMRV8PYvHkziouLxaafPXs2evTogYkTJ8Lf\n3x+6urro2LEj3Nzc4OTkBEdHR2FaDQ0NpKen1/NI644CBiGEVFJSUoI2bd5eGhMSEjBgwAAAQEhI\nCMaMGSOSvqysDMXFxVBRURFZLq6GsWrVqhr3HRgYCCMjI+jq6gIAjIyMEBMTg6lTp4oEqdTUVKir\nq9ftwBpAve9hFBYWCn9njMHf3x/m5uZSbTt37lz06NED/fr1Ey4rKCjApEmT0KtXL0yePFkkf0II\nkbegoCDh71lZWbhy5QqcnZ0B8NqHjY2NSHoDAwMkJydXy0eWexhPnz6FoaGhcD4nJwcAqr3nOykp\nSSRdY5E5YJw/fx6DBw+GtrY2DAwMcPnyZQwZMgQBAQE4cOCAVHnMmTMHZ86cEVm2e/du9OrVCw8e\nPICOjg58fHxkLSIhhNRJfHw85s2bB19fXwQEBGDnzp04deoU1NTUkJubK/Zbvb29PZKSkkSWeXl5\nwcfHByEhIXBzc0N+fr5U+58+fTpevHiBI0eOwN3dHQ8fPoStrS0OHTqE169fC9M9evQIw4cPr9/B\nyoLJaOzYsSwyMpIxxtjvv//OWrVqxQ4cOFDnfFJSUpiFhYVw3snJicXGxjLGGLt27RqbOnVqtW3q\nUWxCiAIp+//u0aNHJa7bu3cvy8zMrLY8JyeHLV68WJ7FEpGVlcWWLFlSYxpJ57m+51/mGsbTp09h\na2sLAHB0dESfPn0wa9asegewmJgYYZc1ExMTREdH1ztPQgiRRqtWki+Jjx8/FvZSqqxLly7Q0tLC\ntWvX5Fk0IQ8PDyxevLhR9lWVzDe9nz9/Dk9PT2E7XXFxsXBeIBBgxYoVMuXLpGz3W7dunfB3e3t7\nYZ9pQgiR1dSpU8UuT05ORv/+/SVu5+bmhu3bt2PQoEHyKhoAPsDh4MGD0bdvX6nSh4aGIjQ0tMH2\nL2DSXqGrWLdunUgvgIpAUcHd3V2qfFJTUzFhwgTcunULAODk5IS1a9fCysoK165dw8aNG3H8+HHR\nQgsEcn0ohhAiH/S/2zgknef6nn+ZaxiVv+FXtX37dlmzha2tLXx9fbF582b4+vpiyJAhMudFCCGk\n4chlaBBPT0+p0s2YMQPDhg1DQkICdHV1sX//fixcuBBpaWno27cvMjIy8MUXX8ijiIQQQupI5iap\nmujq6sr1KUSq1hLSNNH/buOQV5MUDT5ICCFEKjLfw1BVVZU4uFZRUZHMBSKEEKKcZA4YNGQHIaSu\n1NXVZR7FlUhPXuNMyeUehrxROyghhNQd3cMghBDSKChgEEIIkQoFDEIIIVKhgEEIIUQqFDAIIYRI\nhQIGIYQQqVDAIIQQIhUKGIQQQqRCAYMQQohUKGAQQgiRCgUMQgghUqGAQQghRCoyj1Yrb3p6eujU\nqRNat26Ntm3bIjo6WtFFIoSQFk1pA4ZAIEBoaCg0NDQUXRRCCCFQ8iYpGsKcEEKUh9IGDIFAAAcH\nB0yePBlBQUGKLg4hhLR4SvsCpczMTPTs2RN3797FhAkTEBERAS0tLQD0AiVCCJFFfa+dSnsPo2fP\nngAAU1NTTJw4Eb///js+++wz4fp169YJf7e3t4e9vX0jl5AQQpRbaGgoQkNDGyw/paxhFBUVoays\nDGpqanj+/Dns7e1x5swZ6OrqAqAaBiGEyKJZ1jCePn2KKVOmAAC6du2KlStXCoMFIYQQxVDKGkZt\nqIZBCCF1V99rp9L2kiKEEKJcKGAQQgiRCgUMQgghUqGAQQghRCoUMAghhEiFAgYhhBCpUMAghBAi\nFQoYhBBCpEIBgxBCiFQoYMjo1i0gNVV++efnyy9vQgiRRYsMGGFhgJcXIMsT8owBP/0E2NsDI0YA\n9+83bNmKioD584Hu3YHTp6uvf/UKKCyUPf/UVOD994EdO2TPgxDSMrWogPH6NbB6NTB9OvDbb8Bn\nnwFv3rxdX1wMfPcdcOyY+O3z8wFnZ8DXF4iO5mkdHIC7d6Xbf0YGEBAAlJeLXx8fD1hb86Dw++/A\n7NnA//3f2/VXrgD9+/Pp3j3p9lnZ0aM8/xEjgF27gPXrpQ+ajMkWYAkhzQhrgmQp9u3bjFlaMjZx\nImPPnjFWUMDY++8zNnkyY8XFjF24wJihIWNOTozp6TG2di1jZWVvtw8OZszAgLEFC3j6Cn5+jPXs\nyfOX5M0bxn78kbGuXRkzNWXso4/4/iu8fs3Y9u2MaWoytn8/Y+XlfPnVq4z16MHYb78xtmoV//3Y\nMcZ+/ZX/Hh4u3bFnZTE2Zw5jRkY8T8YYe/KEsf79GVu+/O3+JLlzh7F//YsxHR3GLl+uvj41lbHs\nbOnKQghRnPpe8pt8wKjtYvf4MWOff84vxnv2iKYvKWFs2jTG9PUZ09VlLCiIL3/6lLFhw3jwuH6d\nsfHjGTM2ZuzPP8Xv49Ahxrp1Y+zAgerliYxkzNqaMTs7fuEtKWFs7lzG+vVjLDmZB6K+fRkbM4ax\nu3er5x0Xx1j37rwsT5++XX7uHN/nkSOSj72khLEtW/ixL14sGqQY4xf5IUMYmzFD/AU/P5+xr7/m\n2+/YwVhgIC/Ljz/y46w4t+rqjL37LmNnzkguCyFE8VpswHj2jLFvv+UXs/ffr36xzcxkbM0axjQ0\n+EXvxQvxeb15w5i/P2N5eaLLS0oYmz2bsU6dGNu6lbFXr2ou0/XrjJmb8wv78+eMhYYy9t57PBD9\n8otobaW8nDFvb8Y6dmTMxISxP/6oOfC9eiV+/Y0bPNhNm8ZYRsbb5S9fMrZvH183caL4QFShsJAH\nE21tHhAY4zUwd3cekGbP5rWRCklJvKZmZ8fP7Vdf8XN74QI/VldXxoqKaj5XhBDFaLEBo0sXxj77\njLFbtxjz9OTNPatWMXb+PGPOzox16cKbjx4+rN++Kl/oa1NczC+gHTvy5q1ffqk50KSl8eao+nj5\nkjE3N378mze/rRE4OvKgJa3QUN7kZmfHawyff87Y/fvi0xYVMbZzJ2+Kqiw7mwcvQ0PejFZaKvtx\n1VV2Nm8ue/NGPvlnZFAgJE1ffQNGk32B0pMnDD16vF2WmQl88w1w8yYwbx6/Ydy5s2LKl54O9OwJ\ntGnE9xneuwe4uQG9ewOLFwMGBnXPo6gIOH4cGDMG0NKSrRyMARcuABs28Jv8a9bwTgbvvFM9bWEh\ncPYsv7H/11/AyJHAF1/wHmgCQe37ys4GgoN5R4LwcODdd4HSUmDpUuDf/wbU1GQ7hgpJScCJE7wT\nRGIi0LYt78G2cCGgowPExfHOCVFRwLhxgIsLoK5ev30SIk/1fYGSUgaMsLAwLFiwAG/evIGrqyuW\nLFkisp7euPdWaGgo7O3tFV0MsS5dAjZt4r27PvwQmDEDUFHh3ZrDwoBr14AhQ4DJk4F//QsICQF2\n7wbKyoBPPgHGjwcsLYFW//Tle/oUuHoVCA3lQenBA2D0aGDaNGDiRODq1VC0b28PLy8egBwd+fIx\nYwBVVSAlhV/ck5OBYcOA4cOBdu143uXlfPmVKzz/ixd5AJ08Gfj4Y2DUKL5+507gwAGgQwd+LBMm\n8J5nv/8OnDnD9/nxx7xcnTq9PRevXwMPHwJ6ejzwyJsy/100NjoXbzXLgGFlZYUdO3agd+/eGDt2\nLCIiIqCpqSlcTwHjrXXr1mHdunWKLkaNnj7l39SPHuXdmO3s+DRsWPVaIGNARARPf/o0kJfHuxHH\nx/ML+KBBfFsHB8DG5u0FHxA9F48eAYGBfLpyBWjfnk+2toC+Pt/HvXu8i3FhIXDjBtClC7/4jx7N\nJ1NT8TWd/Hx+TIaGoutfvAAOHeLBIzISGDCA1/Ru3eL70tQEcnOBoUN5AGrVCrh9m085OXzfQ4fy\nycJCNOBUKC9/G0BrIsvfBWO8pp6YyGtX5eX83Jub8xpiWRnw+DEPfBoagJFR4wS/+moK/yONpb7X\nzkZsNJFOXl4eAMDOzg4AMGbMGERFReHDDz9UZLFIPfToASxaxKfaCAS8aWrkSP5wZXIyv6BaWPAL\nvTRNVQBvMlq8mE95eTwoaGuLpsnK4rWJLl0AKyuga1fp8u7USfzFvGtXwNWVT0VFPCg9fMiPu18/\nftHNzubNZ5cu8Qv/e+8By5bxwBkdzYPb4cP82Z4uXQATE6B1ax4AHz3i+fbuzS/Whob8ON99lzeB\ntm3L88/O5vtevpwHtqws3lTWqxef2rcHnj/n07NnvOnw0SMeDDp14vkaGvJj+uknHuw0NHg+mpo8\njxcveNNrnz6AsTFfpqvLy9O9O9CtG0+rrs73J+3nRpSb0gWMmJgYmJiYCOfNzMwQGRlJAaOF6tOH\nT/XRubP4+1mamsDUqfXLW5J33uFNYVVpaACTJvGpKiMjYOZM/nt5Ob+I373Lv9lXXIw7duRNa4mJ\nfMrI4LWvx4957a1rV76P4mKeftAgfpw5OUBaGs+vtJRf0Hv3BgYP5oG0IvB07Fi9XKWlPP8ePXhT\nXIWSEiAhgTcNpqfz/KOieCDKyuI/c3N5zaVTJ35PqWNHfm7eeedtra9dOz61bcunNm341Lr125+t\nWolOAgGv6uYoAAAgAElEQVSfKv9e9eHSiuVhYfwhW3HrKk8VyyunqUxS0JO0vLYv8tIG0YZOVx9K\nFzCkYWBgAAF9ZRFav369oougNOhcvPX338pzLl684JOiXLyoPOdCkQxk6Q1TidIFDGtra3z99dfC\n+fj4eIwbN04kTWJiYmMXixBCWjylG0uq8z9tB2FhYUhNTUVISAhsbW0VXCpCCCFKV8MAAC8vLyxY\nsAClpaVwdXUV6SFFCCFEMZSyWy0hhBDlo3RNUoQQQpQTBQxCCCFSoYBBCCFEKhQwCCGESIUCBiGE\nEKlQwCCEECIVChiEEEKkQgGDEEKIVChgEEIIkQoFDEIIIVKhgEEIIUQqShswysrKYGVlhQkTJii6\nKIQQQqDEAWPHjh0wMzOjFyURQoiSUMqA8ejRI/z555+YP39+vV5YTgghpOEoZcBYvnw5tmzZglat\nlLJ4hBDSIindC5SCg4PRvXt3WFlZITQ0VGwaQ0NDJCUlNW7BCCGkiTMwMKjXK66V7iv833//jaCg\nIOjr62PGjBm4cOECPv30U5E0SUlJYIzRxBjc3d0VXgZlmehc0Lmgc1HzVN8v2koXMH744Qekp6cj\nJSUFR44cgYODA/z8/BRdLEIIafGULmBURb2kCCFEOSjdPYzKRo0ahVGjRim6GErN3t5e0UVQGnQu\n3qJz8Radi4YjYIw1uX6rAoEATbDYhBCiUPW9dip9kxQhhBDlQAGDEEKIVJT6HkZNBOvpZjghhDQm\nuodBCCEtRLO8h5Geno7Ro0fD3Nwc9vb28Pf3V3SRCCGkxVPKGsaTJ0/w5MkTWFpaIisrCzY2Nrh5\n8ybU1NQAUA2DEEJk0SxrGFpaWrC0tAQAaGpqwtzcHFevXlVwqQghpGVTyoBRWWJiIuLj42FjY6Po\nohBCSIum1AGjoKAAzs7O2L59Ozp27Kjo4hBCSIumtN1qS0tL4eTkhE8++QSTJk2qtn7dunXC3+3t\n7enxf0IIqSI0NFTiayJkoZQ3vRljmD17NjQ1NeHp6VltPd30JoSQuqvvtVMpA0ZERATs7OzQv39/\n4Wi1GzduxLhx4wBQwCCEEFk0y4BRGwoYhBBSd82yWy0hhBDlQwGDEEKIVChgEEIIkQoFDEIIIVKh\ngCGD7Gxg9Wrgp5+ApnjvPS0NyMxUdCkIIU1NiwoYjAEPH0peX14OHD8OXLsmfv2bN8DOnYCpKQ8a\n+/cDLi4NFzRevABOnpS8/vFjYO5c4NNPgbw82ffz6hUweDBw6ZLseRBCWh6lDBhhYWEwNTWFkZER\nvL29GyTPGzcABwdgzhzxF/jLlwFbW2DTJmD8eMDHp3qa4GDg1Cng/Hlg716+zaxZgKCWdzmVlwOp\nqZLXl5YCO3bwQBQRUb18xcXAd98B/foBPXoAqqrARx/VesgSGRkBv/4KODsDW7fWHvAePAAmTQLM\nzPh5rCo5GXj+XPbyEEKaCKaELC0t2aVLl1hqairr27cve/78ucj6ysUuL685rydPGPvsM8a6d2ds\n1y7GSkurp/n1V8Z0dBg7eJCxsjLGEhIY2769erry8tr3V9WlS4xZWTHm7Cw+vz/+YKxvX8bGjGHs\n9m3xeezbx5iTE2PJyW+XZWXVvu9XrxhLSZG8/uFDxqyted55edXX5+QwtmIFY127MubhwdiBA/xc\nVnXqFGNaWoyFhdVeJkKI4tT3kq90ASM3N5dZWloK55csWcKCg4NF0gBgpaU8AGhqMvbRR+IveP7+\n/GK3ciW/+EmSl8dYYWFDHQGXnMzY1KmM9erFyyEu0OzezYPFH3/UPRDVpLycscBAxgwNGVu2rOa0\nJSWMLVjA2Pz51ddt3syXP3lS+z5Pn+ZB2dOzYY+FENJw6hswlO5J7/Pnz+OXX37B4cOHAQA+Pj7I\nyMjA//73P2EagUAACwsGTU3Aw4M3r7RpA/z4o2het24BHTrwJpjGFBAALFwILF8OrFwJqKiIT5eW\nBvTsCbRt23D7vnULWLECyMgAtm3jzWvSKCnh56oyxmpvbqssNRVwcgIMDYGffwb+ed8VIURJ1PdJ\nb6UdrbY2ZmbrYGIC/PknMG2aPYYPt6+Wpl+/xi8XwC+UcXGAtnbN6Xr1atj9fvUV4OcHfPstsGBB\n3QJR1WAB1C1YAICeHr+vs2QJv8ezcmXdtpdVaSmwaxe/x7R9O2Bh0Tj7JUTZNfvRavPy8mBvb4/Y\n2FgAwJIlSzBu3Dh8+OGHwjQ0lpR4d+4AWlqAhoaiS8Jv9LeSc5cKxvgXhpUrgd69gXHjACsroKFG\nun/9Gmjdmk/iFBUBW7bw2tQvvwBjxjTMfgmRl2Y3llTnzp0B8J5SqampCAkJga2trYJL1TSYmSlH\nsABqDxaZmbyLsLiuvdJ0GWaM9xRbuRLw9ATOnOFNgA0RLBgDfv+d11TOnq2+bts2HiD69gXu3gV8\nfXk3ZUKaO6VskvLy8sKCBQtQWloKV1dXaGpqKrpIpAEwxp8B2b6dX3TnzgX+eXW7UFkZMGwY7678\nzTeSA49AAKxaxS/UdWl6S0oCgoKAzz8HxL3E8dYtHngeP+b3xP4ZUV/kGHJyePPXkSPA8OHS75uQ\npk7pmqSkQU1STU9xMTBqFH9ew8qKN+UYGIhP++gR8PHH/JmT334D/ql0yiwtjefx/fe8NjBiBA9Y\nlff/8iW/B3TihGz3gGpz4wawZw/g5gbo6DRcvoTURbNrkiLNk4oKf0jwt9/40+ySggXAL6iXLvGf\ngwfzi7isf+O5ucCQIbyn3IsXvAbxf/9Xff8qKvym/b17wJdf1j9YvH7Ng1NmJvDZZ8DYsbxHXPfu\noumys4Hp0wETEyAmpn77JETeaq1hJCQk4Pr167h//z4EAgH69u0LKysrGBsbN1YZq6EaRstx6BCw\nbx8fskXWlsnMTN6MZGbWsGWrSVYWfzo+NhZYtAhYuxbo0qV6utevAW9v3qPO1ZU/0f/ZZ3XvoUaI\nNOT2xr1jx47Bx8cHrVu3homJCQwMDMAYQ3JyMu7evYuysjIsWrQIU6dOlXnnsqKAQZqC0lJ+A1/a\nQJeQwG/kW1vzbskN2SRWH5Wfx0lK4k1r33+vPOUj0pPbcxgpKSk4dOgQtLS0xK7PzMzEgQMHZN4x\nIc1d27Z1qxUZGwNRUcDBg/xB1MoY47Wsy5eBdevE11bqo7gYiI7m95kqu3iR90L76Sc+8KavL7Bh\nQ/XykZZBppveRUVFeOedd+RRHqlQDYO0NCdP8gu1pSUPGidOAP37174dY/w+SdeuktcfPcp7pI0e\nzUdNqKy0lPdG+/FH4N//5k1mPXvW92iIositSQoAysvLcefOHRgZGaF9+/YoKiqCl5cXdu/ejfT0\ndJl3WpOvv/4awcHBUFFRgZ2dHTZu3AiVKmNrUMAgLU1ZGf/ZujWvgSxfDnh5ATNnSt4mPp4PE6Oi\nwm/0VxUdzfMpKeF5jRwpOa+XL8V3QyZNi9x6SQUEBKB3795wcXFBjx49cPjwYfTv3x/Pnj1DdHS0\nzDuszZgxYxAfH4+rV6/i5cuX8Pf3l9u+CGkqKj9xPmsW8Ndf/CIuTlYWsHgxf4jxgw+AY8eqp/nx\nR2DyZGD+fN47q6ZgAdQeLLy9eVdl+h7XvEmsYYwePRoHDhyAjo4Obty4gYEDB+Kvv/7C6NGjG61w\nx48fR1BQEPz8/ESWUw2DEPF++40/T+LsDKxfL7kp6vFjPuZZQw0QmZoKTJ0K6Ovz+xw08KRyklsN\nIzc3Fzr/PGFkaWkJU1PTRg0WALBv3z5MmDChUfdJSFPWqxd/huWnnyQHCwB4992Gvajr6fGXf2lo\nADY2fMgU0vxIrGF069YNM2bMEEajo0ePYvr06WCMQSAQ4MeqY4nXwfvvv48nT55UW/7DDz8IA8SG\nDRsQFxeH48ePVy801TAIUVr79/Mb5QcP8gcWifKQW7faLVu2QFDp6aFBgwYJdyao51NFISEhNa7/\n9ddfcfbsWfz1118S06xbt074u729Pezt7aGhoYGcnJx6lY0onrq6OrKzsxVdDCKjOXN4b66aajik\ncSjF8OYrV67Etm3bGqwQlZ05cwYrV65EWFgYukr4i5MUJanm0TzQ50iIfMi1W60kurq6cutWa2Rk\nhNevX0Pjn3G6hw4dil27domkoYDRvNHnSIh8NLs37j148EDRRSCEyEF5OX8eZMoUGiurqZIYMCS1\nITPGUF5eLrcCEUKap9xc4Icf+HtEfvmFut42RRKbpPT09Gq8uZ2SkiK3QtWGmqSaN/ocm6+SEj4q\nb3g4H+7E1FTRJWpZFHIPQ9EoYDRv9Dk2f76+wOrVwK5d/IE/0jjkFjDu3bsHExOTGje+e/cuTBXw\nFUEgEEBcoQUAXWiaAUmfLyGkfup7jZQYMD799FM8evQIU6dOhampKfT09FBeXo7U1FTcu3cPx48f\nh66ubrVhOxpDS69hpKSkQF9fX+J8Q8vMzISGhgbat2/fKPm0lM+RkMYmt6FB/Pz8cPjwYRQUFGDv\n3r34+OOP4ezsjJ9//hmFhYU4cuSIQoJFS5ecnIzIyEiJ8/KgoaGBTZs2KU0+hBDFUMp7GNu2bcPX\nX3+NrKws4fMYlbXkGsY333wDDw8PifMAEBERgePHj8PLy6vB9uvv74/WrVvD2dlZ7PqCggI8fvwY\nffv2rVc+QMv4HIlkRUV8SHbqetvw5FbDqHDixAmcPHlSZIqNjZV5h7VJT09HSEgIevfuLbd9KNIv\nv/yCbt26ybTtzZs3hQNCipsHAE9PT3h7eyMvL0/mMt64cQP+/v44ceKEMBi5uLggLCxM4jYBAQFQ\nVVWtNe/a8iHEzQ2YMQMoLFR0SUhVtQaMgwcPYubMmfD29oa3tzdmzZqFFStWwNjYGKdPn27wAq1Y\nsQKbN29u8HyVxcCBA2Fvby/TtsHBwXBwcJA4D/Dz98EHH8hcvri4OOTm5sLFxQVOTk44e/ascF1R\nURHS0tLEbpeeng5tbW2p9lFTPoRs3AioqvJRb+/dU3RpSGW1Bozi4mLExsbi4sWLuHjxImJjY9G+\nfXucO3cOO3bsaNDCBAYGQkdHB/2lefdkE3XhwoVqF3lpxcTEwMzMTOJ8hfpUOa9evSoMaNeuXYOF\nhYVwnb6+Pm7evFltG2l61FUmKR9CAKBDB+Dnn4GVK/mLncQMWE0UpNahQR4/fizyzfHdd99FRkYG\n9PT08Pjx4zrvUNLQ5t9//z02btyIc+fOCZfVdOETN1qtNATrG6ZhlLnLdlEODQ3F1q1bAfCn6ffu\n3QtjY2NcvnwZixYtgoGBAcrKyrB582YYGxsjMzMTiYmJ8PLyQlFRkcjDlFXnK8g6mnBmZiaMjY3x\n8OFD7Nq1C9evXxfp2GBgYICMjIxq2wUGBmL58uXC+ZycHOzbtw+GhoaIiIiAq6srNDU1UVhYCC0t\nLYn5EFLZvHl81NupU4H27QF6NU7dNfRotbUGjPnz5+PDDz/ElClTAPCLw7x58/Dy5UuxN6RrI2lo\n89u3byMlJQUDBgwAADx69AiDBg1CdHQ0unfvXi195YBRF7Je6BtCWVkZEhMT0bdvXxQXF2PixIk4\nfPgwdHV10bZtW2zduhW7d+/G2rVrYWhoCCcnJxw6dEh4P6ekpEQkv6rzFcQF2s2bN6O4uFhs+tmz\nZ0NPTw9RUVFwdHREmzZt4OHhAW9vb+zfvx9r1qwBwHs5VR10sqysDKWlpWjXrh0ACI/L398furq6\n6NixI9zc3ODk5ARHR0eJ+RAizqBBwPXrNIyIrKp+mV6/fn298qs1YLi6umLo0KHCb/5bt24Vvhuj\nISOXhYUFnj59KpzX19fHtWvXZApKyiomJgbW1tYAgHPnzsHY2Bi6uroAgISEBGhqaqKsrAw+Pj7C\n2ltoaCiWLl0KgF+ci4uLoaKiIna+grgaxqpVq2otX0lJCdq0efsnkZCQIAzgAJCamgp1dXWRbc6f\nP48xY8YI58+dOwcjIyPhcRkZGSEmJgZTp04VBhVx+RAiCf2pKI9a72EAgLW1Nd577z289957GDx4\ncL1foCSNxthHYwsPD4e9vT1OnTqFp0+fwtDQEAD/Vn7ixAl88cUXwmYbFRUVvHz5EnFxccL7CAYG\nBkhOThbmV3W+gqz3MIKCgoS/Z2Vl4cqVKyLdX5OSkoRlrhAZGQkbGxvhfOXjAiB8oVVFDVVSPoQQ\n5VdrwAgNDYWRkRE2bNiADRs2wNjYGJcuXZJ7wZKTk5tV7QIArKyskJiYCA0NDUyfPh0vXrzA8ePH\n4enpiZ9//hna2tro3Lkzpk2bhiNHjuDbb78VuZlsb2+PpKQkifMA4OXlBR8fH4SEhMDNzQ35+flS\nlS0+Ph7z5s2Dr68vAgICsHPnTpw6dQpqldoCHj16hOHDhwvnc3Nzq9UUKo7ryJEjcHd3x8OHD2Fr\na4tDhw7h9evXYvMhpK6ysoAlS6jrbaNjtfjggw/YvXv3hPP3799n48ePr20zuZJUbCkOR+llZmay\n4uJixhhjGzZsYCdOnBCuy8nJYYsXL5Y4Xx9Hjx6tcX1WVhZbsmSJyLK9e/eyzMzMOu1HXD5VNYfP\nkchXURFjc+cyZmrK2N270m1TXs7Y1auMvXol37Ips/r+b9Vaw8jJyYGWlpZwvkePHsjNzZVbAGvp\nPDw8sHfvXuzbtw9t27bFRx99JFzXpUsXaGlp4dq1a2Ln66NVq5r/FDw8PLB48WKRZY8fPxb525CG\nuHwIqSsVFf5OjRUreNdbHx/gu+8kp795E/jXv4AxY/jPsrLGK2uzUltE8fHxYUOHDmXbtm1jW7du\nZcOHD2d79uypV5SqL0nFluJwmrzy8nK2bds2ifPykJycXK0GkpSUxE6ePFnvfMRpCZ8jaTgxMYwN\nG8bYpk2MlZWJrrt9m7EpUxjr3p2xXbsYe/2aschIxZRTGdT3f0uqsaTi4uIQHBwMgUAAR0dH9OvX\nT95xrEYteSyploA+R9JQEhKAwEDgs8+ALl0UXRrFoxcoSbGcNC30ORIiH3IbfFBVVRVqampip06d\nOsm8w9rs378fpqamMDc3xzfffCO3/RBCSIULF4BbtxRdCuUn8cG9QgX0V7t9+zb27t2LoKAgGBkZ\n4fnz541eBkJIy/P8OeDsDHh5ATNnKro0ykupmqS2bNkCdXV1zJ8/v8Z01CTVvNHnSBQhLg746CNg\n/Hhg2zbgn4EJmhW5vw+jMZ07dw63b9/G4MGDMX/+fNy5c0fRRSKEtBD9+wNXrwJpaYC9PUDjY1bX\n6AHj/fffR79+/apNQUFBKCkpQXZ2NsLDwzFp0iR8+eWXjV08QkgL1qULcOoUHxm36qg7jAH79wNT\npgAPHiimfIpW6+CDDU3SaLXA27GWVFRUMGHCBCxYsAAlJSXo0KFDtbSyDm9OCCE1adUK+M9/xK+L\njwcGDgSGDwf27gUmT649P8aAkBBeg6njc6711tDDmyvVPYyTJ0/iwoUL8Pb2RnR0NL766iuEh4dX\nS0f3MJo3+hyJsouKAj7+GHBx4U+Yt5Hw1TsmBli2DMjMBC5fBnr2bNxyVtWs7mFMmjQJb968gZmZ\nGTZt2gRPT09FF4kQQqqxtQWuXQOSkngwqCojA5g9G5g0ib8I6sEDxQeLhqBUNQxptfQaRkpKCvT1\n9RWeR20yMzOhoaGB9u3bi52XpKV8jqR5evSIvynws8+ANWvq9vKnV6+Atm15s5g8NKsaBqldcnIy\nIiMjFZ6HNDQ0NLBp0yaJ84Q0Rzo6wJ07wMaNNQeLsjJeQwGA1695V95Bg3jQUFYUMJqYPXv2YMaM\nGXXapqCgAPfv368xj4iICCxbtqxBylihffv2MDIywtGjR8XOE9JciXmrdDU3bgBDhgAbNgDm5sDF\ni8Dx43wkXmVFAaOR/fLLL+jWrZtM2968eRM6OjrVlt+4cQP+/v44ceIEPDw8qq0PCAiAqqqqxDw8\nPT3h7e2NvLw8mcpVExcXF4SFhUmcJ6SlGjQIOHeOPzD4009AcDBQ6X1pSokCRiMbOHCgzF2Ag4OD\n4eDgILIsLi4Oubm5cHFxgZOTE86ePVttu/T0dGhra0vMY8WKFfjggw9kKpM0ioqKkJaWJnGekJbK\nyorXKsaOlZymoAAoKmq8MtWEAkYju3DhQrULtrRiYmJgZmYmsuzq1avCAHTt2jXh+78r3Lt3T+Q1\nr+LyAGR/D7g09PX1cfPmTYnzhBDJfvsNGDr07f0ORWr0B/cUTiBomHxkvMCGhoZi69atAIDs7Gzs\n3bsXxsbGuHz5MhYtWgQDAwOUlZVh8+bNMDY2RmZmJhITE+Hl5YWioiIIKpU/MzMTxsbGePjwIXbt\n2oXr16/Dz89PZH+BgYFYvny5cL5qHhXELZNGTk4O9u3bB0NDQ0RERMDV1RWampooLCwUvo3PwMAA\nGZXGWag6TwiRbPFiftkaOpS/ZXDCBMWVRekCxp07d7Bq1So8evQIurq62Lx5M0xNTRtuBwrsrllW\nVobExET07dsXxcXFmDhxIg4fPgxdXV20bdsWW7duxe7du7F27VoYGhrCyckJhw4dQu/evQEAJSUl\nIvlFRUXB0dERbdq0gYeHB7y9vbF//36sWbNGuL/S0lK0qzSKWtU8KoirYWzevBnFxcVi08+ePRs9\nevTAxIkT4e/vD11dXXTs2BFubm5wcnKCo6OjMK2GhgbS09MlzhNCJBMIeNAYOBCYNo0/NLh+PdC6\ndeOXRekCxoYNG/Dpp59i2rRpOHz4MDZs2IDDhw8rulgNIiYmBtbW1gD4QIvGxsbQ1dUFACQkJEBT\nUxNlZWXw8fHB48ePAfAaydKlSwHwAFBcXAyVf7pRlJSUoE2lR0wTEhIwYMAA4fz58+cxZswYkTJU\nzaOCuBrGqlWrajyewMBAGBkZCY/ByMgIMTExmDp1qkiQSk1Nhbq6usR5Qkjthg7lDwtOn85vkv9z\nWWhUSncPo3Pnznjx4gXKy8vx4sWLZnVhqRgr69SpU3j69CkMDQ0BAMXFxThx4gS++OILYVOOiooK\nXr58ibi4OOF9CQMDAyRXGhEtKChI+HtWVhauXLkCZ2dn4bLIyEjY2NiIlKFqHhVkuYdR+RgA3jwF\nAFOmTBFJl5SUJJKu6jwhRDrdu/OeVV98oZj9K13A2LJlC3bs2AF1dXXs3LlTbDfRpsrKygqJiYnQ\n0NDA9OnT8eLFCxw/fhyenp74+eefoa2tjc6dO2PatGk4cuQIvv32W5Eb1vb29kj6585XfHw85s2b\nB19fXwQEBGDnzp04deoU1P55Uig3N1dssK2cRwUvLy/4+PggJCQEbm5uyM/Pl+p4Ko7hyJEjcHd3\nx8OHD2Fra4tDhw7h9evXwnSPHj3C8OHDJc4TQqTXpg1Qy2AJcqOQoUHef/99PHnypNry77//Hn5+\nfhg9ejQWLFiAnTt34vLlywgICBBJJxAI4O7uLpyvGK22OQwp8eTJE3Tp0gUdOnTA//73P5ibm+Oj\njz4CwIPA2rVr8dNPPyEgIADTpk2TmM++ffswYcIE4Y3nCpXzaAwvXrzA+vXr8eOPP4qdF6c5fI6E\nKIOqo9WuX7++Xv9bSjeWlJaWFlJSUqCiooLCwkIYGhpWCy7NeSyp5cuXQ19fHyoqKnjx4gVWr14t\nsv67777D+PHjkZKSgqlTp0rMZ/369SJBVVwegwYNatCyi7Nq1SrMmzcPffv2FTsvTnP4HAlpTC9f\n8sEOPTwAAwPJ6er9v8WUzPTp09mRI0cYY4wdPHiQzZo1q1oaScVWwsNpcOXl5Wzbtm01pklKSmIn\nT56sVx4NITk5mR09elTivCQt4XMkpCGVlzPm7c1Y9+6M/f675HT1/d9SuhpGfHw8vvvuO9y5cwcW\nFhb473//K9KODzTvGgahz5EQWV25wrve/vvfwLp11bve1vd/S+kChjQoYDRv9DkSIrunT3nX23bt\ngN9/5z8rUMCQYjlpWuhzJKR+3rwB/viDv8CpMgoYUiwnTQt9joTIB71AiRBCSKOggEEIIUQqSjeW\nVH2oq6vLPOoqUR7NaTgYQpoThdQwjh07BnNzc7Ru3RrXr18XWffjjz/CyMgIZmZmiIiIqFO+2dnZ\nYIzR1MSn7OzshvxzI4Q0EIUEjH79+uHUqVOws7MTWf7s2TPs2rULf/31F3bv3g1XV1dFFK9JqfzY\nf0tH5+ItOhdv0bloOAoJGCYmJjA2Nq62PCoqCuPGjUOvXr0watQoMMZQUFCggBI2HfTP8Badi7fo\nXLxF56LhKNVN7+joaJGXJfXt2xfR0dEKLBEhhJAKcrvpLWlE2h9++AETJLxjUNKzFYQQQpQAUyB7\ne3t27do14XxQUBBzdXUVzg8YMIDl5+dX287AwIABoIkmmmiiqQ6TgYFBva7ZCu9WyyrVKmxsbPD1\n118jLS0NycnJaNWqlfCFQJUlJiY2ZhEJIYRAQc9hnDp1Cq6ursjKysKHH34IKysrnD59Gj169MDC\nhQvh4OCAdu3aYc+ePYooHiGEEDGa5FhShBBCGp9S9ZKSRlhYGExNTWFkZARvb29FF6dRpaenY/To\n0TA3N4e9vT38/f0BAAUFBZg0aRJ69eqFyZMno7CwUMElbTxlZWWwsrISdqRoqefi5cuXmD17NoyN\njWFmZoaoqKgWey727duHYcOGYdCgQVi2bBmAlvN3MXfuXPTo0QP9+vUTLqvp2Ov6oHSTCxhLly7F\nnj17cP78eezcuRNZWVmKLlKjadu2LbZv3474+HgcP34ca9euRUFBAXbv3o1evXrhwYMH0NHRgY+P\nj6KL2mh27NgBMzMzYW+6lnou3N3d0atXL8TFxSEuLg4mJiYt8lxkZ2fjhx9+QEhICGJiYpCQkICz\nZ8+2mHMxZ84cnDlzRmSZpGOX5UHpJhUw8vLyAAB2dnbo3bs3xowZg6ioKAWXqvFoaWnB0tISAKCp\nqQlzc3PExMQgOjoa8+bNQ/v27TF37twWc04ePXqEP//8E/Pnzxd2nmip5+L8+fNYs2YNOnTogDZt\n2uTkgE4AAAUeSURBVKBz584t8lyoqKiAMYa8vDwUFxejqKgIXbp0aTHnYuTIkdXGYpN07LI8KN2k\nAkZMTIzI61rNzMwQGRmpwBIpTmJiIuLj42FjYyNyXkxMTFrMw47Lly/Hli1b0KrV2z/jlnguHj16\nhJKSEixcuBC2trbw8PBAcXFxizwXKioq2L17N/T09KClpYXhw4fD1ta2RZ6LCpKOPSoqqs4PSjep\ngEG4goICODs7Y/v27VBVVW2RLxsKDg5G9+7dYWVlJXL8LfFclJSUICEhAU5OTggNDUV8fDwCAgJa\n5Ll4/vw5Fi5ciDt37iA1NRVXrlxBcHBwizwXFepy7LU9KN2kAoa1tTXu3bsnnI+Pj8eQIUMUWKLG\nV1paCicnJ3zyySeY9M/7F62trXH37l0AwN27d2Ftba3IIjaKv//+G0FBQdDX18eMGTNw4cIFfPLJ\nJy3yXBgaGqJv376YMGECVFRUMGPGDJw5c6ZFnovo6GgMGTIEhoaG6Nq1Kz7++GOEh4e3yHNRQdKx\n29ra4s6dO8J09+7dq/W8NKmA0blzZwC8p1RqaipCQkJga2ur4FI1HsYY5s2bBwsLC2HvD4B/8L6+\nviguLoavr2+LCKI//PAD0tPTkZKSgiNHjsDBwQEHDhxokecCAIyMjBAVFYXy8nL88ccfeO+991rk\nuRg5ciSuXr2K7OxsvHr1CqdPn8aYMWNa5LmoIOnYbWxscPbsWaSlpSE0NFTig9Ii6vWcuAKEhoYy\nExMTZmBgwHbs2KHo4jSq8PBwJhAI2IABA5ilpSWztLRkp0+fZvn5+WzixIlMV1eXTZo0iRUUFCi6\nqI0qNDSUTZgwgTHGWuy5uH//PrO1tWUDBgxgK1euZIWFhS32XOzfv5/Z2dmxwYMHs7Vr17KysrIW\ncy6mT5/Oevbsydq1a8d0dHSYr69vjcfu5eXFDAwMmKmpKQsLC6s1f3pwjxBCiFSaVJMUIYQQxaGA\nQQghRCoUMAghhEiFAgYhhBCpUMAghBAiFQoYhBBCpEIBg5Ba5OXlYffu3RLXDx8+vNY8VFVVG7JI\nhCgEBQxCapGTk4Ndu3ZVW/7mzRsAwOXLl2vNo7YxeghpChT+Tm9ClN3q1auRlJQEKysrtG3bFh07\ndoSGhgbu3r2LO3fuQFVVFYWFhSgsLMTkyZORnZ0NLS0tfPPNNxg1apRIXi9fvsSsWbOQmpqKN2/e\nYPfu3RgxYoSCjoyQuqEnvQmpxcOHD+Ho6Ihbt27h0qVLcHBwwNWrV2FlZQUAUFNTQ0FBAcrKylBU\nVAQ1NTWkpaVh9uzZuHjxokia/fv3Izk5Gf/73//AGMPLly+puYo0GVTDIKQWrMrw6ZaWlsJgUVnr\n1q2xY8cO/Pnnn3j58iWSkpKQl5cnHDQTACwtLeHh4QGBQIA5c+ZAX1+/UY6BkIZA9zAIqaN3331X\n7PLQ0FCEh4fj7NmziI2NRYcOHYRviaxgZWWFqKgo9OzZExMnTkRwcHBjFJmQBkEBg5Ba9OjRA/n5\n+bWmy8jIgLa2NtTU1HDkyBFkZ2dXS5OWlgZVVVUsXLgQM2fORFxcnDyKTIhcUJMUIbVQUVGBs7Mz\nBg4ciDZt2kBLS0tkfUUPqMmTJ+PEiRMwNTXF6NGjYWZmVi3NxYsXsXXrVrRr1w56enrYt29f4x0I\nIfVEN70JIYRIhZqkCCGESIUCBiGEEKlQwCCEECIVChiEEEKkQgGDEEKIVChgEEIIkQoFDEIIIVKh\ngEEIIUQq/w9onDwpWD0D4AAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10dc3f590>"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Behavior\n",
      "For $p_0 < p_1$,\n",
      "+ when $p \\ge p_1$, the likelihood ratio is likely to cross the upper (green) line eventually and unlikely to cross the lower (red) line.\n",
      "+ when $p \\le p_0$, the likelihood ratio is likely to cross the lower (red) line eventually and unlikely to cross the upper (green) line.\n",
      "+ the SPRT approximately minimizes the expected number of trials before rejecting one or the other hypothesis, provided $p \\notin (p_0, p_1)$.\n",
      "\n",
      "For $p_1 < p_0$, the directions are reversed.\n",
      "\n",
      "## It works for $\\beta = 0$ too: $P$-values\n",
      "The inequalities hold when $\\beta = 0$ also. Then the likelihood ratio has chance at most $\\alpha$ of ever being greater than $1/\\alpha$, if in fact $p > p_0$.\n",
      "\n",
      "Hence, $1/\\mbox{LR}$ is the $P$-value of the hypothesis $p > p_0$. This can be used to construct one-sided confidence bounds for $p$ and other parameters. The next chapter does exactly that, to find a lower confidence bound for the mean of a nonnegative population."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## What's next?\n",
      "+ [Next: The Kaplan-Wald Confidence Bound for a Nonnegative Mean](kaplanWald.ipynb)\n",
      "+ [Lower confidence bounds for the mean of a nonnegative population: Markov's Inequality & methods based on the empirical distribution](markov.ipynb)\n",
      "+ [Index](index.ipynb)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%run talkTools.py"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<style>\n",
        "\n",
        "font-family: Helvetica\n",
        "\n",
        ".MathJax_Display {\n",
        "  padding: 10px;\n",
        "}\n",
        "\n",
        "div.callout {\n",
        "  color: #000000;\n",
        "  background-color: #DDDDDD;\n",
        "  margin: 20px 20px 20px 20px;\n",
        "  border-style: solid;\n",
        "  border-width: 2px;\n",
        "  padding: 10px 10px;\n",
        "}\n",
        ".rendered_html {\n",
        "  color: #2C5494;\n",
        "  font-family: Helvetica;\n",
        "  font-size: 140%;\n",
        "  line-height: 1.1;\n",
        "  margin: 0.5em 0;\n",
        "}\n",
        "\n",
        ".title {\n",
        "  color: #498AF3;\n",
        "  font-size: 250%;\n",
        "  font-weight:bold;\n",
        "  line-height: 1.2;\n",
        "  margin: 10px 50px 10px;\n",
        "}\n",
        "\n",
        ".subtitle {\n",
        "  color: #386BBC;\n",
        "  font-size: 180%;\n",
        "  font-weight:bold;\n",
        "  line-height: 1.2;\n",
        "  margin: 20px 50px 20px;\n",
        "}\n",
        "\n",
        ".slide-header, p.slide-header {\n",
        "  color: #498AF3;\n",
        "  font-size: 200%;\n",
        "  font-weight:bold;\n",
        "  margin: 0px 20px 10px;\n",
        "  page-break-before: always;\n",
        "  text-align: center;\n",
        "}\n",
        "\n",
        ".rendered_html .code {\n",
        "  background-color: #999999;\n",
        "}\n",
        "\n",
        ".rendered_html h1 {\n",
        "  color: #498AF3;\n",
        "  line-height: 1.2;\n",
        "  margin: 0.15em 0em 0.5em;\n",
        "  page-break-before: always;\n",
        "  text-align: center;\n",
        "}\n",
        "\n",
        "\n",
        ".rendered_html h2 {\n",
        "  color: #386BBC;\n",
        "  line-height: 1.2;\n",
        "  margin: 1.1em 0em 0.5em;\n",
        "}\n",
        "\n",
        ".rendered_html h3 {\n",
        "  font-size: 100%;\n",
        "  line-height: 1.2;\n",
        "  margin: 1.1em 0em 0.5em;\n",
        "}\n",
        "\n",
        ".rendered_html .definition, .proposition, .proof, .theorem {\n",
        "    padding-top: 20px;\n",
        "    color: #222299;\n",
        "    font-size: 120%;\n",
        "    font-style: italic;\n",
        "}\n",
        "\n",
        ".definition, .proposition, .theorem {\n",
        "  background-color: #EEEEEE;\n",
        "  border-style: solid;\n",
        "  border-width: 2px;\n",
        "  padding-left: 20px;\n",
        "  padding-right: 20px;\n",
        "}\n",
        "\n",
        ".rendered_html .definition::before{\n",
        "    content: \"Definition:\";\n",
        "    background-color: #DDDDDD;\n",
        "    color: #222299;\n",
        "    font-size: 110%;\n",
        "    font-weight: bold;\n",
        "    font-style: normal;\n",
        "}\n",
        "\n",
        ".rendered_html .proposition::before{\n",
        "    content: \"Proposition:\";\n",
        "    background-color: #DDDDDD;\n",
        "    color: #222299;\n",
        "    font-size: 110%;\n",
        "    font-weight: bold;\n",
        "    font-style: normal;\n",
        "}\n",
        "\n",
        ".rendered_html .proof::before{\n",
        "    content: \"Proof:\";\n",
        "    background-color: #DDDDDD;\n",
        "    color: #222299;\n",
        "    font-size: 110%;\n",
        "    font-weight: bold;\n",
        "    font-style: normal;\n",
        "}\n",
        "\n",
        ".rendered_html .theorem::before{\n",
        "    content: \"Theorem:\";\n",
        "    background-color: #DDDDDD;\n",
        "    color: #222299;\n",
        "    font-size: 110%;\n",
        "    font-weight: bold;\n",
        "    font-style: normal;\n",
        "}\n",
        "\n",
        "\n",
        ".rendered_html li {\n",
        "  line-height: 1.8;\n",
        "}\n",
        "\n",
        ".input_prompt, .CodeMirror-lines, .output_area {\n",
        "  font-family: Consolas;\n",
        "  font-size: 120%;\n",
        "}\n",
        "\n",
        ".gap-above {\n",
        "  padding-top: 200px;\n",
        "}\n",
        "\n",
        ".gap01 {\n",
        "  padding-top: 10px;\n",
        "}\n",
        "\n",
        ".gap05 {\n",
        "  padding-top: 50px;\n",
        "}\n",
        "\n",
        ".gap1 {\n",
        "  padding-top: 100px;\n",
        "}\n",
        "\n",
        ".gap2 {\n",
        "  padding-top: 200px;\n",
        "}\n",
        "\n",
        ".gap3 {\n",
        "  padding-top: 300px;\n",
        "}\n",
        "\n",
        ".emph {\n",
        "  color: #386BBC;\n",
        "}\n",
        "\n",
        ".warn {\n",
        "  color: red;\n",
        "}\n",
        "\n",
        ".center {\n",
        "  text-align: center;\n",
        "}\n",
        "\n",
        ".nb_link {\n",
        "    padding-bottom: 0.5em;\n",
        "}\n",
        "\n",
        "</style>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.HTML at 0x10dc2be50>"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 21
    }
   ],
   "metadata": {}
  }
 ]
}