ensemble.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pill 19 WIKI Side Quest: Ensemble"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Investigate why ensemble learning works. Focus on one of the models. For example in Bagging it is said that \"Bagging performance improvement is due to the reduction of the variace of the classifier while maintaining its bias\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### by Toni Miranda"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook mix notebooks posted by David, Huang, Markos, Indra and Gonzalo, revising different ensemble methods (I have also added Random Subspace Method).\n",
    "\n",
    "*(Gonzalo) Note: I think Iris data set with 2 features is not the best example to show the advantages of ensemble learning because of  ensemble methods (like Random Forest) improve the performance for large datasets and using a lot of features due to randomization and diversity. However, this way we can see the differences of models, representing the performance in 2D. Anyway, Iris is a toy example, so the result is also good.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why ensemble learning works?\n",
    "**(David)**\n",
    "\n",
    "Ensemble learning helps improve machine learning results by combining several models. This approach allows the production of better predictive performance compared to a single model. That is why ensemble methods placed first in many prestigious machine learning competitions, such as the Netflix Competition, KDD 2009, and Kaggle."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensemble methods are meta-algorithms that combine several machine learning techniques into one predictive model in order to decrease variance (bagging), bias (boosting), or improve predictions (stacking).\n",
    "\n",
    "Ensemble methods can be divided into:\n",
    "\n",
    "- **Bagging (or Bootstrapping aggregation):** Horitzontal aggregation. In bagging different models, obtained by training independent bootstrapped (resampling with replacement) elements, are combined using an aggregation technique (such as majority voting) in order to take the final decision.\n",
    "\n",
    "- **Boosting:**. Horitzontal aggregation. Each component in the aggregation depends on all the others. In Boosting algorithms each classifier is trained on data, taking into account the previous classifiers' success. After each training step, misclassified data increases its weights to emphasise the most difficult cases. In this way, subsequent learners will focus on them during their training. In this kind of techniques the two most well known approaches for linking members are the notions of residual (gradient boosting, XGBoost) or statistical resampling (adaptive boosting aka AdaBoost). \n",
    "\n",
    "- **Stacking:** Vertical aggregation. In this kind of techniques the output of one member of the ensemble is the input for the next member. We can use different model architectures and stack one on the other to improve the final performance.\n",
    "\n",
    "The basic motivation of parallel methods is to exploit independence between the base learners since the error can be reduced dramatically by averaging.\n",
    "Most ensemble methods use a single base learning algorithm to produce homogeneous base learners, i.e. learners of the same type, leading to homogeneous ensembles.\n",
    "\n",
    "There are also some methods that use heterogeneous learners, i.e. learners of different types, leading to heterogeneous ensembles. In order for ensemble methods to be more accurate than any of its individual members, the base learners have to be as accurate as possible and as diverse as possible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why ensemble learning?\n",
    "**(David)**\n",
    "\n",
    "To understand that why the generalization ability of an ensemble is usually much stronger than that of a single learner, there are three main reasons:\n",
    "- The first reason is that, the training data might not provide sufficient information for choosing a single best learner. For example, there may be many learners perform equally well on the training data set. Thus, combining these learners may be a better choice. \n",
    "- The second reason is that, the search processes of the learning algorithms might be imperfect.For example, even if there exists a unique best hypothesis, it might be difficult to achieve since running the algorithms result in sub-optimal hypotheses. Thus, ensembles can compensate for such imperfect search processes. \n",
    "- The third reason is that, the hypothesis space being searched might not contain the true target function, while ensembles can give some good approximation. For example, it is well-known that the classification boundaries of decision trees are linear segments parallel to coordinate axes. If the target classification boundary is a smooth diagonal line, using a single decision tree cannot lead to a good result yet a good approximation can be achieved by combining a set of decision trees."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bagging \n",
    "\n",
    "**(David)** \n",
    "\n",
    "Bootstrap aggregating, often abbreviated as bagging, is a involves having each model in the ensemble vote with equal weight. In order to promote model variance, bagging trains each model in the ensemble using a randomly drawn subset of the training set. As an example, the random forest algorithm combines random decision trees with bagging to achieve very high classification accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to reduce the variance of an estimate is to average together multiple estimates. For example, we can train M different trees on different subsets of the data (chosen randomly with replacement) and compute the ensemble:\n",
    "$$ f(x) = \\frac{1}{M} \\sum_{m=1}^{M}f_m(x)$$\n",
    "\n",
    "Bagging uses bootstrap sampling to obtain the data subsets for training the base learners. For aggregating the outputs of base learners, bagging uses voting for classification and averaging for regression.\n",
    "\n",
    "![alt text](algo_bagging.png)\n",
    "<p style=\"text-align: center;\">Figure 1. Bagging Algorithm .</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python code\n",
    "\n",
    "**(Huang)**\n",
    "\n",
    "We can study bagging in the context of classification on the Iris dataset. We can choose two base estimators: a decision tree and a k-NN classifier. Figure 1 shows the learned decision boundary of the base estimators as well as their bagging ensembles applied to the Iris dataset.\n",
    "\n",
    "The following code shows the accuracy of using a single estimator of Decision Tree and KNN, it also prints the accuracy of using Bagging classfier based on 10 estimators of Decision Tree and KNN with 0.8 subsampling of training data and 0.8 subsampling of features. Such length of estimators(10), subsamples size(0.8) and subsamples featrues(0.8) are followed by the previously David's work. Also, by default in the Sklearn tutorial page the lenght of estimator is 10 as well. More details, it can be found by [Sscikit-learn Bagging Classifier tutorial page](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy of using Desicion Tree is: 0.879901960784\n",
      "The accuracy of using KNN is: 0.953839869281\n",
      "The accuracy of using Bagging technique based on Desicion Tree is: 0.940767973856\n",
      "The accuracy of using Bagging technique based on KNN is: 0.960375816993\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import BaggingClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "from sklearn import datasets\n",
    "from sklearn import tree\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "import matplotlib.gridspec as gridspec\n",
    "import itertools\n",
    "\n",
    "# import some data to play with\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data[:, 1:3]  # we only take the first two features.\n",
    "y = iris.target\n",
    "\n",
    "#Single estimator Decision Tree \n",
    "clf_tree = tree.DecisionTreeClassifier(random_state=0)\n",
    "clf_tree.fit(X, y)\n",
    "scores_tree = cross_val_score(clf_tree, X, y)\n",
    "print(\"The accuracy of using Desicion Tree is:\",scores_tree.mean())\n",
    "\n",
    "#Single estimator KNeighbors(KNN)\n",
    "clf_KNN= KNeighborsClassifier()\n",
    "clf_KNN.fit(X, y)\n",
    "scores_KNN = cross_val_score(clf_KNN, X, y)\n",
    "print(\"The accuracy of using KNN is:\",scores_KNN.mean())\n",
    "\n",
    "# DecisionTreeClassifier 15 estimators\n",
    "clf_bagging_tree = BaggingClassifier(base_estimator=tree.DecisionTreeClassifier(random_state=0),n_estimators=15, random_state=0)\n",
    "clf_bagging_tree.fit(X,y)\n",
    "scores_bagging_tree = cross_val_score(clf_bagging_tree, X, y)\n",
    "print(\"The accuracy of using Bagging technique based on Desicion Tree is:\",scores_bagging_tree.mean())\n",
    "\n",
    "# KNeighborsClassifier 15 estimators\n",
    "clf_bagging_KNN = BaggingClassifier(KNeighborsClassifier(), n_estimators=15, random_state=0)\n",
    "clf_bagging_KNN.fit(X,y)\n",
    "scores_bagging_KNN = cross_val_score(clf_bagging_KNN, X, y)\n",
    "print(\"The accuracy of using Bagging technique based on KNN is:\",scores_bagging_KNN.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvQAAAIYCAYAAAACWnjvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XuclHXd//HXZ0+yCyygICBHS1vTUjQyOpGmhSdQbsko\nEu1XUdrhTuOOsnN31t2daWZl4AkxDymFgXdmEhKZkpKhaIoHVA4uR9ldWA67zH5/f8zsOrvMzHXt\nzFwz1zXzfj4e+2B3rpnr+szoXu/vfr/f63uZcw4REREREYmmimIXICIiIiIi2VODXkREREQkwtSg\nFxERERGJMDXoRUREREQiTA16EREREZEIU4NeRERERCTC1KCXUDOz+83sIh/P221mbypETSIiIiJh\noga95MzMXjGzvWa2y8yazOwRM/ucmeX8/5dz7kzn3K0+ntfPObcu1+MlS/yR0PnVkXiPnT/PyOex\nREQk/xL5dHrSz9PNbKeZfcDMnJn9scfzf2Nm3018f0riOb/q8ZyHzeziQtQv4pca9JIvk51z/YEx\nwP8Ac4CbiltSbhJ/JPRzzvUD1hN/j52P3d7z+WZWVfgqRUTEj8Ro7y+Bs4FXEw+/y8zek+FlrcCF\nZjY22OpEcqMGveSVc67ZObcY+ChwkZm9DcDMDjGzq8xsvZltMbNfm1lt5+vM7FwzW21mLWb2kpmd\nkXh8uZl9OvH9UWb2VzNrNrPtZvbbpNc7Mzsq8f0AM1tgZtvM7FUz+2bnaIGZXZzoXbkq0Uvzspmd\nmc17NbMfmNlvzexOM9sFfMLMKszsisR72G5md5nZoKTXvNfMViZGMlab2cRsji0iIv6Z2WeBnwKT\nnHOPJG36X+DKDC9tAuYD3wmuOpHcqUEvgXDOPQZsBN6feOh/gLcA44CjgBHAtwHM7GRgAfBfwEBg\nIvBKit3+N/BnYBAwErguzeGvAwYAbwI+AMwEPpm0/V3AWmAw8ZP5TWZmvX+XAEwF7kgc77fAZcR7\nfyYmatwN/BzAzEYBi4kHw6HA14Dfm9lhWR5bRES8XQJ8HzjNObeqx7ZfAW9JnpaTwpXA+WbWEFSB\nIrlSg16C9BpwaKKxPAu4zDn3unNuF/BDYHrieZ8CbnbOPeic63DObXLOPZdif+3Ep/Qc4Zzb55x7\nuOcTzKwysd+vO+d2OedeId4rc2HS0151zt3gnIsBtwLDgaFZvseHnXNLEnXvBT4HXJF4D/uA7wEf\nSYwQzAQWO+ceSDz/T8CTwBlZHltERLx9CFgJrEmxbS/xBvsP0r3YObcZ+DXxPwpEQkkNegnSCOB1\nYAhQB/wzMdWkCfhT4nGAUcBLPvb3VcCAx8zsGTP7fymeMxio5o35kSS+H5H08+bOb5xzexLf9vNx\n/FQ29Ph5NLAk6X12BsjhxP8Y+VjntsT2CcARWR5bRES8XUJ8hPjGNKOxNwJDzWxyhn38GJhkZicE\nUaBIrtSgl0CY2TuJN6IfBrYT7wU5zjk3MPE1IHGxKcQbxW/22qdzbrNz7jPOuSOAzwK/6pw3n2Q7\nb/TkdxoNbMrtHaUvq8fPG4EPJb3Pgc65Pokeng3ALT229XXO/SSg2kREBLYApxGfAvqrnhudc23E\nR1P/m3in0UGcczuAnyWeIxI6atBLXplZvZmdA9wF/MY5t8Y51wHcAFxjZocnnjfCzCYlXnYT8Ekz\nOy1xUekIMzsmxb4/YmYjEz/uJN6Y7kh+TmIazd3AlWbW38zGAJcDvwng7abya+CHZjY6UfPhZjYl\nse02YKqZfcjMKs2sj5mdambqoRcRCZBz7jXijfozzOyaFE+5DehD5imQVwPvAd6a/wpFcqMGveTL\nksRKLxuAbxA/8SVfiDoHeBFYaWYtwFKgAbouoP0kcA3QDPyV7j3snd4J/MPMdhO/uPQ/06w9/0Xi\nS42tIz5CcAdwc65v0KeriU8n+kvi83iEeN0k5vNPBb4FbCO+FOZX0O+hiEjgnHPrgQ8C04Af9dgW\nI75Qw6EZXt9CfCGFtM8RKRZzrueMARERERERiQr1DIqIiIiIRJga9CIiIiIiEaYGvYiIiIhIhKlB\nLyIiIiISYWrQi4iIiIhEWFUQO13RtEJL54iIpDFx4MSUN68pN8oKEZHM/OaFeuhFRERERCJMDXoR\nERERkQhTg15EREREJMLUoBcRERERibBALooVEQlcB1S3VVPZUYkRvmtMHY5YRYz2mnZ1nYiIFFMZ\n5IUa9CISSdVt1fSt6YtVG2YhPEE7h2t3tLa10t6nvdjliIiUrXLIC/UbiUgkVXZUhvbkDGBmWLVR\n2VFZ7FJERMpaOeSFGvQiEklGeE/OncwslMO7IiLlpBzyQg16EZEcPPzgw0w+cTJnnXAWN/70xmKX\nIyIiIRR0VqhBLyKSpVgsxpVfuZJf/f5X/OHxP3D/wvt56bmXil2WiIiESCGyQhfFikjJ+8zE6ezZ\n/vpBj9cNPpQbVtyV9X7XrFrD6DeNZtSRowA48/wzeei+h3jzMW/Oep8iIlI8QeRFIbJCDXoRKXl7\ntr/OysGDDnp8QoqTdm9sbdzKsBHDun4eOmIoT616Kqd9iohI8QSRF4XICk25ERERERGJMDXoRUSy\ndPjww9m8aXPXz1s2bWHo8KFFrEhERMKmEFmhBr2ISJbe9o638epLr7LxlY20t7Vz/+/u55SzTyl2\nWSIiEiKFyArNoRcRyVJVVRVXXHUFnzvvc8Q6Yky9cCpHvfWoYpclIiIhUoisUINeREpe3eBDU17Q\nVDf40Jz3PXHSRCZOmpjzfkREpPiCyougs0INehEpebksTSkiIuUjqnmhOfQiIiIiIhGmBr2IiIiI\nSIR5NujNrMHMVid9tZjZlwtRnIiIRIfyQkSkODzn0Dvn1gLjAMysEtgELAq4LhERiRjlhYhIcfR2\nys1pwEvOuVeDKEZEREqG8kJEpEB626CfDtyZaoOZzTKzVWa2avH8xblXJqGzculKZs+czYxTZjB7\n5mxWLl1Z7JJEiupbl3yLDxz5AaaePLXYpYRRyrxQVpQH5YVId0Hnhe8GvZnVAFOAe1Jtd87Nc86N\nd86Nn3LxlHzVJyGxculK5l4/l9oZtYybP47aGbXMvX6uTtJS1s6dcS7XL7q+2GWETqa8UFaUPuWF\nyMGCzove9NCfCTzhnNsSVDESXgsXLGTkrJEMPG4gFVUVDDxuICNnjWThgoXFLk3Et53bd3Lp+ZfT\ntKMpL/sb/77xDBg0IC/7KjHKizKmvJBSELW86E2D/mOkmW4jpa9xfSP1DfXdHqtvqGfz+s1Fqkik\n9xbOX8KaVfu45xZN9QiY8qKMKS+kFEQtL3w16M2sL/Ah4PfBliNhNXz0cFrWtnR7rGVtC8NGDytS\nRSK9s3P7Thbf8XcOG3odi+/4e956XaQ75YUoLyTqopgXvhr0zrlW59xhzrnmoAuScJo2cxob522k\n6ZkmOg500PRMExvnbWTazGnFLk3El4Xzl3DgwGQO6fMWDhyYHJlel6hRXojyQqIuinnhuQ69CMCE\n0ycA8bmRL69/mWGjh/HZSz7b9biXlUtXsnDBQhrXNzJ89HCmzZzm+7XlRp9V/nX2ttT1uwuAun4z\nWHzHdD7yySkMPGxgkasTKS3Ki8LRZ5V/Uc0LNejFtwmnT8jqRNG54sHIWSMZ1zCOlrUtzL1+btc+\n5Q36rILR2dtSVzUYgKqqwexJ9Lp8ZvbMrPf71U9+lcf/9jhNO5o4reE0Pn/F5/mPi/4jX2WLRJby\nInj6rIIR1bww51zedtZpRdOK/O9UImv2zNnUzqhl4HFv/GXb9EwTe2/fy1ULripiZeGjz8q/2j21\n1NbX+nrup87+Ehte3nHQ46OOPIyb/u/n+S6tm70te9lbt7fbYxMHTrRADxoRygrpSedA//RZ+VcO\neaEeeglc4/pGxjWM6/ZYfUM9L69/2dfry2lIMdfPSlIL+iQsIvmhvPBPeRGMqOZFb+8UK9Jruax4\nUG43KNHqECJSzpQX/ikvJJka9BK4XFY8KLcblGh1CBEpZ8oL/5QXkkxTbiRwuax4UG5DirmuDiEi\nEmXKC/+UF5JMDXopiGxXPOgcUky+6KfUhxSz/axEREqB8sI/5YV00pQbCbVyHFJcuXQls2fOZsYp\nM5g9c3bJzv8UEckn5YXyopyph15CrdyGFLWucLRs3riZK2ZdwY6tOzAzpn1yGp+49BPFLkukLCkv\nlBdhVYis0Dr0IiGidYX96826wkHZtnkb2zZv49hxx9K6q5WPvv+jXHvXtbz5mDd3PUfr0KenrBDJ\nnvLCv2LnhZ+sAK1DLyWgnNYOzqTcLuoqpL89+Ddum38br736GkeMOYILL76Q93/o/Tntc8iwIQwZ\nNgSAvv37cmTDkWx5bctBJ2kRyQ9lxRuUF8HJd14UIivUoJei07DhG8rxoq5C+NuDf+PqX17NqM+M\n4sRjTqT5uWau/uXVADk36jttenUTzz31HMePPz4v+xOR7pQV3SkvghF0XgSVFbooVoqu3NYOzqQc\nL+oqhNvm38aoz4xi0NsGUVFVwaC3DWLUZ0Zx2/zb8rL/Pbv3cNknLmPO/8yhX32/vOxTRLpTVnSn\nvAhGkHkRZFaoh16KTsOGbyi3i7oK5bVXX+PEY07s9tiAYwaw7tV1Oe+7vb2dyz5xGWdfcDann3t6\nzvsTkdSUFd0pL4IRVF4EnRVq0EvRadiwO60rnH9HjDmC5ueaGfS2QV2PNT/XzBFjjshpv845vvP5\n7/Cmhjdx0RcvyrVMEclAWXEw5UX+BZEXhcgKTbmRotOwoQTtwosvZMMNG9j59E46DnSw8+mdbLhh\nAxdefGFO+/3Xo/9iyZ1L+Mdf/8G090xj2numseKBFXmqWkSSKSukEILIi0JkhXropSAyrUzgNWwY\nxVUNolhzKeu8kOm2+bex7tV1HDHmCC7//OU5X+B00ntOYs2uNfkoUUQS0p0//UwxieK5N4o1l7Ig\n8qIQWaF16CVwySsT1DfU07K2hY3zNvqa65fLa4slijVHUbHXFfZL69Cnp6yQnpQX4a85isohL3xN\nuTGzgWa20MyeM7NnzezdWdQpZSqXlQmiuKpBFGsWyRflheRCeRH+miWc/E65uRb4k3NumpnVAHUB\n1iQlJpeVCXJd1cBrKDOIoU6txCBlTnkhWQtrXgQ1LUZ5Ifni2aA3swHAROBiAOdcG9AWbFlSSnJZ\nmSCX13rdhCSom5RoJYbCcDicc5iFd/aKcw5H+cwqUV5IrsKYF0BgN7RSXhRGOeSFnyk3RwLbgFvM\n7F9mdqOZ9c36iFJ2clmZYNrMabx47YusfXAt655dx9oH1/LitS/6eu3CBQupe1cda29ey4pPrGDt\nzWupe1dd11BmUEOdWomhMGIVMVx7/CQdRs45XLsjVhErdimFpLyQnIQxL4KcFqO8KIxyyAs/U26q\ngJOALzrn/mFm1wJfA76V/CQzmwXMAvjKNV9hysVTsi5KSkuuN7/o2NvB9oXbaWtuo2ZADdV7q329\n7uVnXqajuYPhs4bT9y19aX2+ldfmvUbFxvjfsUENdepmH4XRXtNOa1srlfsqMcLX6+KIn5zba9qL\nXUoheeaFskIyCWNe9OnfJ7BpMcqLwiiHvPBc5cbMhgErnXNjEz+/H/iac+7sdK/RygWSL7NnzqZ2\nRm234cimZ5rYe/terlpwVcbXnjf+PIZ9fRj1J9V3PdbyRAubf7SZe1fdm9O+RXJRqqvc9DYvlBWS\nT0HlxVHHHqWskKLxmxeePfTOuc1mtsHMGpxza4HTgH/nWqCETxjXwm1c30jdijoe//rjHGg9QFXf\nKkadOYq96/d6vraqsoqqQVXE9sao7FNJbF+MqkFVVFXG/7efNnNafB7kLA5aLkxEek95UT7KKS+U\nFRIFfle5+SJwe2LFgnXAJ4MrSYohqAtEc9XR1sHG5RsZ9Y1R9H1bX1qfbmXj1RsZ6AZ6vnZsw1ho\nglhNjP1t+6muqaZPU5/442ioUyQgyosSV255oayQKPDVoHfOrQbGB1yLFFHyRT9A/N9Z8ceLedLa\n1bqLEVeMoN8J/bAKo98J/RjxpRFs/uFmz9d29qp0u2HH7d17VZLvPigiuVNelL5yzAtlhYSd3x56\nKXFhXQu3bV8bA44fQMeBDmIuRoVVMOD4Aazft97ztRNOn8Da1WtZ9I1F7Nm1h7r+dUydPrXbSfnW\nq25l0V3dt180+6Ig35KISKSVY14oKyTs1KAXILxr4db1r6P1360HXahU19/7XjUrl65k+aPLefuV\nb+/qcVk+bzkNSxuYcPoEbr3qVu6+925GfX0U/d/en11rdnH3NXcD6EQtIpJGueXF2tVrlRUSen7W\noZcyENa1cKdOn8qGazbQ8kQLrt3R8kQLG67ZwNTpUz1f67V28KK7FjHqslHUn1SPVRv1J9Uz6rJR\nLLprUdBvS0QkssotL5QVEgXqoRfA3wWiuaxq4DVcmW7fnc9Z9KNFrNu1jrr+dVww/QJfr/UaFt6z\naw/9396/2/b+b+/Pul3rcn6/IiKlqph5kWm/XnmR6bWZ8sIrK3J9vyL5oAa9dMl00U8uqxp4TW3x\n2vdFsy9KO6yZ6bVew8J1/evYtWZXt+HZXWt2Ude/LrSrOIiIhEEx8qJhXIPnftPlhVdNmfJi9+7d\nabMi1/crki+aciO+5HLra6/hylz2nem1XsPCmYZng7zVt4hIKQsqL4LKCsg8jchrKo/yQsJAPfTi\nSy6rGngNV3rtO9Pwa+P6Ro4+4mg2vrKR9rZ2qmuqGXDEADav3+w5LJxpeHbGKTNCuYqDiEjYBZUX\nfvabLi8yZQVknkbUuS3dVJ6wrvoj5UUNevEll1UNMk1t8dq313Sdun51bHhsAwPfPZC6PnXE9sXY\n8OgGavvVAt5rB6cbng3rKg4iImEXVF547TdTXnhlBWTOi0xTP5UXEgaaciO+5LKqgddwZaZ9e03X\ncTHH9t9tZ+/Le3Exx96X97L9d9txMVe09ysiUs6Cyguv/WbKi6CyItf3K5Iv6qEXX3K59bXXygOZ\n9u01XWfv3r0c/t7DWfetdRxoPUBV3ypGnTmKvX/fm/P79boplYiIHCzIvMi030x5sbdfMFnR+X6V\nF1Js5lzuf532tKJpRf53KmXpvPHnMezrww66UcjmH23m3lX38ukzP80228aIz4+g71v60vp8K5t+\nuYkhbgg33n9j1sdNXrWg6zbg8zb6DiWRTCYOnGjFriEMlBWST5nyYvCQwYFkBSgvJFh+80JTbiRv\nVi5dyeyZs5lxygxmz5zNyqUrc96n13QdqzQGnz+Y2iNrsUqj9shaBp8/GKu0nGrSqgUiIsEIIisg\nc154ZUUudSkvJAw05UbyIqh1eL2GX/fs3sPRJx9N8+vN7G/bT3VNNaNOHsWLt72YU01atUBEJP+C\nXLM9U178+b4/p82KXOtSXkgYqEEveZHcQwHE/50VfzwfJ+lMqwvEXosx8riRXY81PdPEsNHDcqpJ\nqxaIiORfkFkBmVcuS5cVudalvJAw0JQbyYvG9Y3UN9R3e6y+ob5rjd9c3HrVrZw3/jw+3PBhzht/\nHrdedWvXtkyrC+RSk1YtEBHJvyCzAtLnhdc5XXkhUaceesmLoHoovNahz7SawsIFC7OuKZdVGkRE\nJLUge7O98gLSn9NzqUt5IWGgVW4kL4K6yt9rlZti1CSSK61yE6esKD9BnpeVF1KK/OaFeuglLyac\nPoG//O4v/PXLfyXWHqOyupIPnPKBrhNhuttxe9mzaw+x1hjPf/V52hrbqBlew5DJQ9iza0/Xc1Yu\nXcnCBQtpXN/I8NHDmTZzWtcd/7Q2sIhIeHhlBQSXF+myorMu5YVEmRr0khe3XnUrj6x5hLHfG0uf\nhjr2rd3DIz9/pGv+otcwaDrVldU03tHIiC+MoO4tdex5fg+bfrGJ6spqIPPKBADLH13O2698e1eP\ny/J5y2lY2qCTtIhIEWTKiotmX+Rr2kw6mfLCaxWblUtXKi8k0nxNuTGzV4BdQAw44Jwbn+n5GkYt\nP+eNP4+hXx9Kn7fWYVaDc23se3YPW360BSDrYdAZp8yg+mPVDHjvACpqK+jY20Hz35tpv7Od25ff\nzuyZs6mdUdtt3mPTM03svT1+9790265acFU+375Ir5TylJve5IWyovxkyop7V92b07SZTHkxfPTw\njHmQKUuUF1JMQUy5OdU5tz3LeiSPMg0b5iLbYU6ID3XWHdOXDioAAyqpO6Zv11Bnuttx+zH8xOE0\nNzYTi8WorKxk+InD2XDnBiDz+r8Op7WBRYpDeRESYcsLr6zYs2tPIHnhtVa81pKXqNOUm4gJ6qYc\nuQxzAtT2q6VlTQv9TxgMgFkVLU9tp6ZPLZjR/NQu6k9M6nF5ahc1tXU89njm/fYdOJwDjVX0HXoo\nu3bupW//Wg40GnUDh/HY4/HtW59sof6tSSsTPNtC3cD4ygTptnkdVyRIEz9U7AqkHIQxL9JlRW2/\nWgDq+texa82ubj30u9bsoq5/nWddw0cPp2pnFYMGH8rr2/Yy6NBaKrZb10o1mVax6bnKzZ49B+fF\nye9Mf+xMmZLpdfmSS6YVoj4Jnt8GvQP+bGYOmOucmxdgTZJBUDflWHTXIkZ9fVTXSbT+pHpGXTaK\nRT9a5KtB/+bRb+HfP3uGkZdV0/et9bQ+28Kmn73GiCHHcfJbz2bxNVdhl42i/3H17HqmhU3XbGTK\n+2fTsG9ixv1OP/V73PLry6meXEHtkQ3seGIt7Us6+OQZV9OwbyLTT/0et86bTfWn66g/ehAtL+xk\n643buejD8SHSdNu8jisiWVNehEQY8yJdVhw7+jgApk6fyt3X3M2oy974Y2HDNRu4YPoFnnVNmzmN\n6395PZWT+lB3ZAONj68l9sA+Lvn8JQDxP2ZmcdAqNgDHv3saf7puLiM+A/2Ormf3Cy1sndc9Lx57\nfEXG46fKlbV9Vvj6gyATP431XDLN632pwR8Nfhv073PObTKzw4EHzew551y3/wPMbBYwC+Ar13yF\nKRdPyXOpAsENC3YNczpob2unuqa6V8OcHbFD6NhYxSvfegEXc1ilUbGnDnf0IUy/4Nu89toLPPrN\n3xE7cIDKqirefdz5TL/g2577fcdJZ7LoDz/nuZ/9HdexDquo5piR7+UdJ53ZtR3g9zf8L/9a93+8\n+U0ncdGkq7oeB1h8yzW8vHUVQw8fy0Uf7r5NRPIuY14oKwonjHmRLis6Dj8EiPfwb3xpIyu+uYID\n7Qeoqq5i4ikTfXUsTTh9Anf/8nesufpZXMcrWEU1bz/urd3+eLnrprv455NP8JYTju5akvKxx+GY\n8RMY2fZ2Ft90Dc9tXZsyL7JpNCe/xqvh7Hc/+ZZp38l/kPihxn/x+GrQO+c2Jf7damaLgJOBFT2e\nMw+YB7rQKUjZ3vzC6xeypraO5qd20fftfenogPb2GK1rWtNOi+n5Szv9S+cw9/pNjJx1Qrfej1NO\nP4c/PP2/vLD/EU748Tu7ej5euOER/vD0/3LM+My9RKv/uoyXWh9l1OUnUzv2MPa+soOXbnmU3z72\nXT568neBeKN+3UvP8srTxrijzul2An7HSWeqAS9SQF55kZwVVz24wpX69LdiNnCCuolT57SYzryI\nHYjnhZ9pMemyYvol5wDxaUIvbn2Rk392ctf2F+e9yMqlKz1HFZYtWsbaxrWM/srJ1L3pMPas28Ha\nW1Yx76fLGDfxg1QMmMDQ4RtY+0hfhg4fR8WACd3yrd97+vLx93yz+0739frjSSuKI8O9rTmXP1pK\nSTF+7z0b9GbWF6hwzu1KfP9h4PuBVyYpTZs5LeOwYSaZfjEnv//LLL76KoZ9aTh9jx1Ey1M72fzz\nxpTTYtb2OfgXNt3Q7lO3LwRg0MQ6Xr19LbvW76L/6P4MmTiEpx5dyMxLMp+gf/5f86l/zxC2/9+z\ntG9tpfrwvtS/ZwgPLpzf1aBvadnOsmX3MWjQ9SxbdgmTJl1Eff1hXftoadnONdd8lssum9ftcRHJ\nr97mRRQbOL3ht3czqPDPJS8y6ZwWM+wL8bxoemonm3/R6GtajNc0oIULFlL3rjrW3pSUF+8a4mua\n0M3Xzqf/u4ew7b5EXgzpS793v5EXLS3bWfPnZxg88EbW/PkSPnHad7syoTPXWpubuOsn1/DNLy1U\nXmSh1H+n/fLzh02+f+/99NAPBRaZWefz73DO/Sm/ZYhfQd1ievoF3+bpZx7luW//Hdexvmtqi59p\nMZB5aHfvrr10NHcw9P8NZdiRR9D28j5eu/k1KjZWeO5364bN1NQewvBPD6f26MPZ+8JeGm9sZN/m\n/V3PeeCBBcRi51Bb28Du3efwwAO38pGPXN5t+/PP7zzocRHJO+VFEj+Nm1SN/nwFfVB5cdHsi3jy\n70+z5tvP4jo2JKa2HOtrWozXNKCXn3k567zY/Opm+qTIi/2b24HMWdH53+qexVez6VlTXkhOvH73\nM/2xn+3vv2eD3jm3Djghu91LEDrvgppPLS3b2bF1PyP6raOycjCx2HZ2bL2AlpYdvnop0g3t1g0c\nRlPTixxx4VAOOboWqOGQo40hFx7Oaz/Y4jmdp25QHcM+/caaxH1OOoTKT1ew8b83d9W9bNl91NbG\nV1iorZ3JsmUXdPXSe/Xei0j+KC96L9UIqFevfm8CP4i8aNrRxNaNxqihT1FVdRgHDuxg68bpNL/e\nzIBDB2R8rdc0oJjFGDozc16kev+PPQ59B9Qz7FNDGHBCfN99Tqij4lOVbP7BNs+sAO/RXpF8Sdfg\nT/X773dVNO8/eaUsdPZcVFbGlxKLN+rjPRh+TJs5jY3zNtL0TBMdBzpoeqaJjfM28r5J0zikoo7K\nAVV07AfnoGM/VA6o4pCKOhr2Tez6Gr71WG765ndofr25a79VlVVUDaoitjcGDmJ7Y1QNqqKyoqpb\n3QCvvXY+YN3qfuCBBbS3n8n27ZW0t5+Z8v20tGzne987n5aWHbl8hCIiOUs+J6b7euxxPL+CdN+t\nDxI7MJmqqnhjt6rqMGIHJrNk/p+BzHWly4ppM6cB4DqqqKivTJsXAA8tbeKzk7pnBQAVUD2omgP7\nDuCc48AY+b8gAAAgAElEQVS+A1QPqqaqosYzK8A7L5QVErRUv+9+BbIOfalf5BRFm6qfzrj9kTX3\nsq/jdfa13Nzj8UNTDjv2/G9cMWACp54OD1+/kGe3vMyhQ4dx6qTPcsz4CTy25H5an2/kkNghOLcH\ns2r2v9TO8GFHd9vHAw8sYP1zB/jVD/7MaR/7CACHDh+LbYX2yhj72vdTWV2Nbe3D0NFjAVi9ejmx\n2Gts2fIL9u4dyJYt76Rv30NZvfoIJk2aybJl99HWdh379sWoqTmLZcu+eFCvi6bkiEiU5DKcnyzb\nof1Vy58kduAhmrbf2e3xZUsO5S3v/0jaGtf2WZE2KzovUB18xEj2Pb+HQ2J9UuZFw76JXdNikrMC\n4MiR46jYupeOyv20HdhDVVUNNVv7M2rk4Rmz4iMfubyrdz5TXigrJMzMufwvSLNkCVrlJmTW9lmR\n1cm7MxRyudDlZ9fO4h/r/8jgCxuoHTOEva9uY/tta3nX6LP48n/Gl6huadnOnDkXUFl5PbHYJfz4\nx/dQX38Y/3zifm7982xGfHp011rym25c3205sXSvveeeq3nggRg7d07BbAzOvcqgQYuZNKmy62Sc\n7rUiQZo8GV+38i51yoriSbW4QbLe5EXyHw+5XhTplReZztleeZHptV55oayQYvGbF7pTrGR08jtz\nH3HZtnUzBzY4Nl39OI4YRiWV+/qzrc/mrud0DnXu3FlJv35ndvWA+FlnPt2FTqtXL6e19TlisRuo\nqKino6OF1lZYvfqYrga91wW1IiKlKJ9rj3vtrze88iJdVoD3fUkyne+98kJZIWGnOfQSuDlzbmbw\nYQ3041FcyxP041EGH9bAnDm3ACQNdZ7Fvn0x2trOYtmy+7rmKb7jpDMZd9RkKve8g3FHTe7WmH/j\nQqeZQOeFTvHXzplzMwMHjmTMmEcYO3YVY8Y8wsCBIw86bqrXioiUKz/z+LOd5+slU154ZQWkzwuv\n832mvFBWSBSoQS+B6+xRaW2toqpqDK2tVd0uOPLa3n3lge4n0UwX83pd6JvrhcAiIpJfmfLAKysg\nfV7kkgfKCokCNegjKoir7VcuXcnsmbOZccoMZs+czcqlK/Oy3/hQ5g3EYufQ3t5ALHYOra03sHr1\n8m7bDxw4i7a2MRw4cFa37Q88sID9+z/Mxo372b//w91OovELne6guXl811csdgerVy/v2rZz5zhe\nfXUoO3eO69rm9dpOWtVARKIuiPPYP5+4n+/85MN87r+O5js/+TD/fOL+vOw3U154ZQWkzwuv832m\nvPCTFaC8kOLSRbERdc89V3Pvvfdx3nnn+JrH53VR7MqlK5l7/VxGzhp50B0FKwZMyGlItfNioj17\nTqW5+SEGDDiVurqHui4qyrQdHHPmXMCOHf/Dvn396NNnN4cd9rVeXZDU288qX68VSUcXxcYpKwoj\n3+cxP4sVZMtPHnhlifJCSonfvFAPfQRlmoKSreTbcVdUVTDwuIGMnDWShQsW5rzv+DDpKeze/RCV\nldeze/dDtLef2mPKTertnb0t+/bVYjaWfftqD+qlzySXzyqIz1lEpJCCOI8t/ss1jPj0aAa+9bB4\nXrz1MEZ8ejSL/3JNzvv2ygOvLFFeSLlSgz6COufz1dQ0pJzHl82wX+P6Ruob6rs9Vt9Qz+b1m7s9\nls2+V69ezu7d84nFTsC5GLHYCezefUu3oc5021evXk5z8/XAVJx7DzCV5ubrfQ91en1WmeTyWhGR\nMAgiL7ZsfZn6owd1e6z+6EFs2fpKzvv2ygOvLPHKi0w1KS8kytSgjxg/V9sn3/wiWaa79/UdOJyt\nT7awZw9dX1ufbKFu4LBu+0i370zmzLmZQYPGMmbMNxg79ljGjPkGgwaN7VptJtP2Sy/9Cc5VU1Fx\nL/BbKiruxblqLr30p5515bIygVY1EJGoyyUvMhl6+JG0vLCz+7Fe2MnQw8d2eyzfeeGVJX7yIl1N\nyguJOjXoI8bravt0w35eS45NP/V7bJ23nb3PtFHd1pe9z7Sxdd52pp/6va7589kOKeayusDcuV+j\no2M6zlUnbvZRTUfHdObO/WrX/rNd1SCXmkVEwi7bvPAy5bTL2HTjepqe3UHHgQ6ant3BphvXM+W0\ny7qeE0ReeL0fr7zIdsW0XGoWKRTdWCpiOm9f3dx8R4/Hj8jp5hedFzItvuUaXt66iqGHjz3oAqds\n9+1Vc+f2HTvm09S0gYEDR1FVVcPq1Uewbt1TOLcS5+4E+gG7gT288MIhnnV5HTeXmkVEwq7U8gJI\nmxUf+cjlvPDCvzLmhdeNpZQXEmVa5aaEdF7hX1Nzd6KHYDttbRfk5RbVQe6707e/PZXVq9czbtxo\nvv/9Rb6OW4i6RPJNq9zEKSuKJ8p5kSorvI7buUKOskKiRqvclKEgh/387jvbdXg3bVrLmjVrgCtZ\ns2YNmza90O24sVg769adQSwW082hRERyVOy8yHdWJB83VV4oK6TUqUFfQvze/CLIfWdzERR0zn2c\nChxJR8fUrjmPncdtbBxPR8dmGhtP6vXNoUREpLti50W+syL5uKnyQlkhpU5TbiRvOoc7KyuvJxa7\nxPdQ5qZNa/nCF84mFrsNsxNx7l9UVl7IL35xPyNGHM2mTWv54hfPxuz3OPcfXHdd/HGRqNKUmzhl\nRXkKKis6n6O8kFKiKTdScPGbfpzJ9u2VtLef6Xs6TnKPC/ShZ89L58oFsVjdQSvciIhItHhlBaTO\nC6+seOM5ygspP2rQS150LgfW1nYW+/bFaGs7y/d6x88//0+c+w3wIZw7IfHvb3j++X+yadNann56\nDTCdjo4YMJ2nn+4+b1JERKLBT1ZA6rzIlBWA8kLKmhr0khedPS6trVVUVY2htbWqW89LpvV/zz//\nywwb9l8cddQajjrqSY46ag3Dhv0X55//ZV/r0IuISDR4ZQWkz4tMWQHe69CLlDLfDXozqzSzf5nZ\nfUEWJP5kWiEg29UD/Lw23fbVq5fT2noDsdhkOjreRyw2mdbWG7ouOMo0xJrpYqX4usI34dxZOPeu\nxL838cIL/+r1exOR4CkrwidMeeGVFZA+L7wubFVeSDnrzY2l/hN4FqgPqBbpheThyJ43rsi0LZf9\nZto+Z87NzJlzAYcf3n2N3zlzbkkaYr2Offti1NScxbJlX2TSpIuorz+MK69cnLaezOsKi0gIKStC\nJkx5kSkrgIx5kSkrAObOXaW8kLLlq4fezEYCZwM3BluO+JFp+kq2t9v289psb5vtZ4g1Ha0dLBId\nyorwCVteeJ3TlRci2fE75eZnwFeBjnRPMLNZZrbKzFb96U/z8lKcpNZ50qqpaTjoZJVpm5/9Zlp5\nINO+O4dCd+w4npdeGsSOHcd3W//Xa4g1Ha0dLBIpyoqQCVteZMqKzu3KC5He81yH3szOAc5yzl1q\nZqcAs51z52R6jdYWDk5Qt7bu3G9r63W0tFRRX3+Avn2/2PVav7fyTnVL7qBvAy4SNaW4Dr2yInzC\nnBepssKrZuWFlKN8rkP/XmCKmb0C3AV80Mx+k0NtkgOvqS3ZDjd6DXP62XfnkmHV1bd0WypMw6Ai\nZUFZETJhzYt0WeHntSKSmmeD3jn3defcSOfcWGA6sMw594nAK5OUMg0p+h1uzGblAT/7ji8Z9nEq\nKo6no+PjXUuFaRhUpPQpK8InrHmRLiv8vFZEUvOcctPtyRpGLQn33HM19957H+edd07XygO5DnO+\ncbvtv1FRMZyOjkace79uuy2SQilOuUmmrCgd+c4LZYVI7+Rzyk0X59xyrxO0hFu2Kw94eaPHZThA\n4kT9cd3QQ6QMKStKQxB5oawQCYbuFFtm4nMfT2H79itobz/V9w07vMRv3HET7e0jur5AN/QQEYmq\nIPJCWSESjN7cWEoirrO3pb39VPbt28khh8CyZff5umGHlzvvfCUvNYqISPEFlRfKCpFgqIe+jHT2\ntuze/RCVldeze/dD3Xpdii2XW5CLiEj+hDkvlBUiB1ODvoysXr2c3bvnE4udgHMxYrET2L37ltCs\nHpB8m3ARESmeMOeFskLkYGrQl5E5c25m0KCxjBnzDcaOPZYxY77BoEFjmTPnlmKXltMtyEVEJL/C\nmhfKCpHU1KAvokIPG+bjhh1B1ZzLLchFREqd8qJ7XcoKke7UoC+iQg8b5uOGHUHU3NnjUls7E4Da\n2pnqeRERSaK8UFZICXvssfRfPmmVmyLpPmx4CZMmXeTrBk65yHUlm6BqztQT1HkjExGRcqW8iFNW\nSKj1ovGdyuSTt+T0ejXoi6TzxFRb28Du3dE4IQVVc7wn6DWam+/o8fgRof9MRESCpryIU1ZIUflo\nsOfaKM+FGvRF8Maw4d1A57DhBQXpdclWkDXn2hMkIlKqlBdvUFZIQaRpuBezse6HGvRFEMVhwyjW\nLCISdVE890axZikzHr3tYW+8p6IGfREEPWzY0rKda675LJddNi9lb4jX9mLULCIiB1NeiOQgor3t\n2VCDvgiCHjZMXlkg1cnTa3sqGuoUESk85YWIDxl63Eux8Z6KGvQlxmtlgWKsliAiIuGjvJBIKqNe\n995Qg77EeK0sEMXVEkREJP+UFxI6IV9JJszUoC8hXisLRHG1BBERyT/lhRScz3Xa1WDPjhr0JcRr\nZQGtPCAiIqC8EJ9yvFlST2qsB0cN+hLitbKAVh4QERFQXkgPuqg08sw5l/edLllC/ncqIlIiJk/G\nil1DGCgrRIokRQNeDfeQmjzZV1549tCbWR9gBXBI4vkLnXPfya06Cats1hwWEQHlRblRXkSIGvAl\nz8+Um/3AB51zu82sGnjYzO53zq0MuDYpgmzWHBYRSVBelBHlRQhp6kzZ8mzQu/icnN2JH6sTXxom\nLUFac1hEcqG8KB/Ki5BJasir4V6eKvw8ycwqzWw1sBV40Dn3jxTPmWVmq8xs1Z/+NC/fdUoBdK5q\nUFPT0LWagYhIb3jlhbKiNCgvQuCxx7q+Jp+8petLylOvLoo1s4HAIuCLzrmn0z1PFzpFT0vLdubM\nuYCamrsTy5Ntp63tAn7843vU6yKSZ+VwUayfvFBWRJPyokg0D7485eui2GTOuSYzewg4A0jboJfo\n0ZrDIpJPyovSpbwoEDXgpRf8rHIzBGhPnJxrgQ8BPw68MikorTksIrlSXpQH5UUA1HiXHPnpoR8O\n3GpmlcTn3N/tnLsv2LKk0K68cnGxSxCR6FNelAHlRY7SrESjBrzkws8qN08BJxagFhERiTDlhUgS\nLSEpBdSrOfQiIiIikpCh0Q5quEvhqEEvIiIiko4a7RIBatCLiIiIdOrRgFeDXaJADXoREREpT1pd\nRkqEGvQiIiJSujRlRsqAGvQiIiJSGrQkpJQpNehFREQkGtTbLpKSGvQiIiISTprjLuKLGvQiIiIS\nDmrAi2RFDXoREREpHE2bEcm7QBr0k1kSxG5FRErE5GIXIBK8DA13NdpF8ks99CIiIpIfuimTSFGo\nQS8iIiLZ0Zx3kVBQg15ERET8UQNeJJTUoBcRESlXHheopqIGvEj4qEEvIiJSDnQXVZGSpQa9iIhI\nlPnsZVfDXaR0qUEvIiISJr2cBqOGuoioQS8iIhIkzVMXkYB5NujNbBSwABgKOGCec+7aoAsTEZFo\nKcu80HQXEQkBPz30B4CvOOeeMLP+wD/N7EHn3L8Drk1ERKKltPNCF5WKSEh5Nuidc41AY+L7XWb2\nLDACKI0TtIiI5EVJ5IVHj7sa7yISRr2aQ29mY4ETgX+k2DYLmAUw99JLmXXGGXkoT6Q4Pnj55bQ0\nNx/0eP2AASy7+uqSO65IvqXLi+SsuPTSuZxxxqyC1waot13yRnkhYeC7QW9m/YDfAV92zrX03O6c\nmwfMA2DJEpevAkWKoaW5mVUDBhz0+PgUJ89SOK5IPmXKi+SsWLKEwmWF7nAqAVFeSBj4atCbWTXx\nk/PtzrnfB1uSiIhEVWjyokcDXo13ESllfla5MeAm4FnnnMZwREQkpaLkhabOiIj46qF/L3AhsMbM\nViceu8I598fgyhIRkQgKLi8yXKyqxruIlDs/q9w8DFgBahERkQjLW16o111EpFd0p1gpW5lWCGhs\naWHEjh0Hv6i6OtCavI6rVQ2kpKjhLhHgdd5VXkgYqEEvZSvTCgHD6+uLsnqA13G1qoGUEjXcJQq8\nzrvKCwmDimIXICIiIiIi2VMPvYSa15Bhpu1A1sONL2/bxgnbth30+EafdWVr486djN+5M+1xRUQk\ntVzywOu1mWTKiyCnvSgvJJka9BJqXkOGXtuzHW6sBp60g6/tG+acr+Nmq7Kjg1Up5l2OaG/Pab8i\nIqUulzzI5ZyeKS+CnPaivJBkmnIjIiIiIhJh6qGXSHtp2zaOSDHUuRd485AhGV9bP2BAyl6S+gED\n2L9tGyPcwXel35/4d8vOnfw7xVCnn0v8Djv3XKpT7LvdjGrneHL//oO2xXzU7CXIoV+tpiAiYZYp\nK7x4nXf3Q9q8yCUroPTyQlkRHDXoJdIOAV5L8fgwH6/NdPIYNmUKmzLst6Ojg2NTDHV2+BjqrHaO\nzWmGZysrKjghxX4rE/vN5YQX5NCvVlMQkTALKis6950uL3LJCii9vFBWBEcNeim6TH+xr9u2jeMz\nXJyaidcFQ14XQWXquYl1dLAnTc9IFHsgoliziJQXr/NUpryo8dh3przwc35MlxeZsgKf+w6bKNZc\nDtSgl6LL9Bd7NXBXitecmvR9uhNpNbAgxeOTfBwXMgdAGzAxzeO59EDEKioYH4ulfDxI6jURkbDz\nOk955UWmTppMeeHn/JguLzJlBT73nY7yQpKpQS+hZsCxKYYbLWlOYW2a1w4dNIhjU5x0hvo86VRm\n2FYDrMqwCk62Rg4apBOliEgWvPIiXVZAcHkRVFaA8kK6U4O+hJTiMJgD/p3ixOfnVLhx2zaezHK6\njpdMF0EFqRT/G4tI4ZXiuSRTXhzcpO4uU16M9FhgIZNiZQWU5n9jSU8N+hJSisNgBhyb5nGInxRT\nXdS0n/iFSiek6BnBR8/IfmBEmsdJ7PvFFNuP9NxzfHWCVL0z7WaeqxLk8t84075TnfR7I5fVFESk\n8MotLzJlRacg8iKXrIDSywtlRXDUoJdI27l4cdptw6ZMyXq/dRUVbPK4YUem4dtMdvzhD1m+MjeZ\nemTGf+pTge1bRKTYMmVFrjLlRayjI+usgNLLC2VFcNSgl6LL9Bd727ZtHJ/iNW0pHkv1nBNS9Gx0\nvjbTcRtbWlLfbS/ppL0nzXGL1QORy/Cqek1EJOy8zlNtEEheeJ4fq6vT58X+/WmzonMfygvJBzXo\npei8egKyHTKsIf3tuL2O66WyooK6NOv/FqsHIpfhVfWaiEjYeZ2n3jRkSCB54XXcTffck3bbiPPO\nS5sVULxzr/Ki9AS7tpGIiIiIiARKPfQlpBSHwRpbWhixY8fBG1L0ePTUTuohVH/35/OQaYg1QEFe\n2Coi5UN50V1geVGkrADlRblRg76ElOIw2PD6+qyHBY/MYfjVS6Yh1iAFeWGriJQP5UV3QeVFsbIC\nlBflxnPKjZndbGZbzezpQhQkIiLRpLwQESkOPz3084FfkPquyCKhFcXVA3JRikPoEjnzUV5IBCkv\n3nhcosmzQe+cW2FmY4MvRSS/orh6QC5KcQhdokV5IVGlvJCo0yo3IiIiIiIRlreLYs1sFjALYO6l\nlzLrjDPytWspYxoWFCktygoJivJCylneGvTOuXnAPACWLDl47SeRLGhYUKS0KCskKMoLKWeaciMi\nIiIiEmGePfRmdidwCjDYzDYC33HO3RR0YSJRpWFfKVfKC5HeUV5IvphLcWe0nGkYVUQkvcmTrdgl\nhIKyQkQkM595oSk3IiIiIiIRpga9iIiIiEiEqUEvIiIiIhJhatCLiIiIiESYGvQiIiIiIhGmBr2I\niIiISISpQS8iIiIiEmFq0IuIiIiIRJga9CIiIiIiEaYGvYiIiIhIhKlBLyIiIiISYWrQi4iIiIhE\nmBr0IiIiIiIRpga9iIiIiEiEqUEvIiIiIhJhatCLiIiIiESYGvQiIiIiIhGmBr2IiIiISISpQS8i\nIiIiEmFq0IuIiIiIRJivBr2ZnWFma83sRTP7WtBFiYhINCkvREQKz7NBb2aVwC+BM4FjgY+Z2bFB\nFyYiItGivBARKQ4/PfQnAy8659Y559qAu4Bzgy1LREQiSHkhIlIEVT6eMwLYkPTzRuBdPZ9kZrOA\nWYkff+OcuzD38vLHzGY55+YVu46ewlhXGGsC1dUbYawJVFcZ8MyLsGcFhPf/hzDWFcaaIJx1hbEm\nUF29EcaaOuXtoljn3Dzn3Hjn3Hjgrfnabx7N8n5KUYSxrjDWBKqrN8JYE6iusheBrIDw/v8QxrrC\nWBOEs64w1gSqqzfCWBPgr0G/CRiV9PPIxGMiIiLJlBciIkXgp0H/OHC0mR1pZjXAdGBxsGWJiEgE\nKS9ERIrAcw69c+6AmX0BeACoBG52zj3j8bIwzi8KY00QzrrCWBOort4IY02gukpaFnkR1s9ddfkX\nxpognHWFsSZQXb0RxpoAMOdcsWsQEREREZEs6U6xIiIiIiIRpga9iIiIiEiEZd2gN7ObzWyrmT2d\nZruZ2c8Tt/9+ysxOyr7MvNV0ipk1m9nqxNe3g64pcdxRZvaQmf3bzJ4xs/9M8ZyCfl4+ayr452Vm\nfczsMTN7MlHX91I85xAz+23is/qHmY0NSV0Xm9m2pM/r00HXlThupZn9y8zuS7Gt4J+Vz7qK9Vm9\nYmZrEsdclWJ7wc9bpS6MWeGzrmKc/0KXFb2oS3nhv6ainP8Sxw5dXigr8sQ5l9UXMBE4CXg6zfaz\ngPsBAyYA/8j2WHms6RTgvqDrSHHc4cBJie/7A88Dxxbz8/JZU8E/r8T775f4vhr4BzChx3MuBX6d\n+H468NuQ1HUx8Isi/P91OXBHqv9WxfisfNZVrM/qFWBwhu0FP2+V+lcYs8JnXcU4/4UuK3pRl/LC\nf01FOf8ljh26vFBW5Ocr6x5659wK4PUMTzkXWODiVgIDzWx4tsfLU01F4ZxrdM49kfh+F/As8Tsq\nJivo5+WzpoJLvP/diR+rE189r9w+F7g18f1C4DQzsxDUVXBmNhI4G7gxzVMK/ln5rCusCn7eKnVh\nzAqfdRVcGLOiF3UVXBjzIqxZAeHMC2VF/gQ5hz7VLcCLfgIA3p0YCrvfzI4r9METQ1gnEv+rPVnR\nPq8MNUERPq/E8NtqYCvwoHMu7WflnDsANAOHhaAugPMTw28LzWxUiu359jPgq0BHmu1F+ax81AWF\n/6wgHqx/NrN/mlmqO/6F9bxVysL8mRctL8KYFaC8yFNNUJzzXxjzQlmRJ+V2UewTwBjn3AnAdcC9\nhTy4mfUDfgd82TnXUshjp+NRU1E+L+dczDk3jvhdJk82s7cV4rhefNS1BBjrnDseeJA3ejoCYWbn\nAFudc/8M8ji95bOugn5WSd7nnDsJOBP4vJlNLNBxJXqKlhdhzApQXuSxpoKf/8KYF8qK/AqyQR+6\nW4A751o6h8Kcc38Eqs1scCGObWbVxE+Etzvnfp/iKQX/vLxqKubnlThmE/AQcEaPTV2flZlVAQOA\nHcWuyzm3wzm3P/HjjcA7Ai7lvcAUM3sFuAv4oJn9psdzivFZedZVhM+q87ibEv9uBRYBJ/d4SujO\nW2UglJ95sc5/YcwKP3UpL/zXVKTzXxjzQlmRR0E26BcDMxNXAk8Amp1zjQEez5OZDeucD2ZmJxN/\n/4H/YieOeRPwrHPu6jRPK+jn5aemYnxeZjbEzAYmvq8FPgQ81+Npi4GLEt9PA5Y55wKdo+inrh7z\n56YQn2caGOfc151zI51zY4lfwLTMOfeJHk8r+Gflp65Cf1aJY/Y1s/6d3wMfBnqucBK681YZCOVn\nXqTzX+iywm9dygv/NRXj/BfGvFBW5FdVti80szuJX9U+2Mw2At8hfvEHzrlfA38kfhXwi8Ae4JO5\nFpuHmqYBl5jZAWAvMD3oxk3Ce4ELgTWJeXUAVwCjk2or9Oflp6ZifF7DgVvNrJJ4INztnLvPzL4P\nrHLOLSYeLLeZ2YvEL2qbHnBNfuv6kplNAQ4k6rq4AHUdJASflZ+6ivFZDQUWJdocVcAdzrk/mdnn\noHjnrVIXxqzwWVcxzn9hzAq/dSkv/NcUiqyAcOZFCD6rSGaFFaY9KyIiIiIiQSi3i2JFREREREqK\nGvQiIiIiIhGmBr2IiIiISISpQS8iIiIiEmFq0IuIiIiIRJga9CIiIiIiEaYGvYiIiIhIhKlBLyIi\nIiISYWrQi4iIiIhEmBr0IiIiIiIRpga9iIiIiEiEqUEvIiIiIhJhatBL5JnZM2Z2SrHrEBGR8FNm\nSClSg17ywsxeMbO9ZrbbzHaa2f+Z2ahCHNs5d5xzbnk+92lmVyTey24z22dmsaSfn8nnsUREyk2p\nZQaAmV1sZg8n/VxvZn83s9+ZWY2ZzTczZ2YnJz3nKDNzST8vT2TOqKTHTjezV/Jdr5QWNeglnyY7\n5/oBw4EtwHVFridrzrkfOuf6Jd7P54BHO392zh3X8/lmVlX4KkVEIq1kMqMnMxsE/AV4Ffioc64t\nsel14AceL28FvhVgeVKC1KCXvHPO7QMWAsd2PmZmZ5vZv8ysxcw2mNl3k19jZjPN7FUz22Fm30r0\n3pye2FZrZrcmenGeNbOvmtnGpNcmP/e7Zna3mS0ws12JodXxSc89KVHHLjO7x8x+a2ZeJ9eDmFlV\noqflUjN7EXgu8fixZrbUzF43s+fM7Pyk1/Qxs6sT73+Lmf3KzPr09tgiIqWk1DLDzIYADwFPA59w\nzh1I2nwrcLyZfSDDLn4OfMzM3pz5kxN5gxr0kndmVgd8FFiZ9HArMBMYCJwNXGJm5yWefyzwK2AG\n8Z6aAcCIpNd+BxgLvAn4EPAJjxKmAHcljrUY+EXiODXAImA+cChwJzA1qzfZ/VjvBN5uZv2AB4EF\nwODWE2EAACAASURBVOGJ9zPPzBoSz/0JcCRwPHB04j19I8fji4hEWollxqHAcuBR4P855zp6bN8D\n/BC4MsM+NgE3AN/zOJZIFzXoJZ/uNbMmoJn4SfQnnRucc8udc2uccx3OuaeInxg7eyimAUuccw8n\nhiW/Dbik/V4A/NA5t9M5t5F470UmDzvn/uiciwG3ASckHp8AVAE/d861O+d+DzyW0zt+o669wLnA\n8865Bc65A865fwL3AtPMrAL4DPDlxPNbgB8B03M8vohIVJViZowC3gLMd865NM+ZC4w2szMz7OdH\nwGQzO2iKp0gqatBLPp3nnBsI9AG+APzVzIYBmNm7zOwhM9tmZs3E56UPTrzuCGBD506cc3uAHUn7\n7ba9x/epbE76fg/QJzHH/QhgU4+TrNe+vCS/fgzwXjNr6vwi3us0HBgGHAI8mbTtPuI9+SIi5agU\nM+NJYDZwv5mdmOoJzrn9wH8nvlJyzm0jPlLwfY/jiQBq0EsAnHOxRE9GDHhf4uE7iA9ljnLODQB+\nDVhiWyMwsvP1ZlYLHJa0y27bifeAZKMRGGFmlvRYrqsq9DzR/8U5NzDpq59z7gvEL/hqAxqStg1I\nfBYiImWr1DLDOXct8D/Ag2b2tjRPu4X4FJ//yLCrnwCnAu/wV66UMzXoJe8s7lxgEPBs4uH+wOvO\nuX0WX7Lr40kvWUh8aPE9iTmL3+WNEzfA3cDXzWyQmY0g3pOTjUeJB8YXEhe1nguc7PGa3lgMHGdm\nHzez6sTXyWbWkBjKvRH4mZkNSXxGI83sw3k8vohI5JRiZjjn/he4FliadB1V8vYDxOf6z8mwjybg\np8BXs6hdyowa9JJPS8xsN9BC/IKfi5xznWu2Xwp838x2EZ/veHfnixLP+SLxi5Iagd3AVmB/4inf\nBzYCLwNLiZ/MO7f5lphr+R/Ap4Am4hdK3ZfNvtLsvxmYlNhvI/Fh3B8Rn2oD8BXiS5g9RnzO6J+J\nXxwrIlKOSjoznHP/Tbwj5y9pVqy5M1F/JtcS/6NCJCNLf82GSHEkVotpAo52zr2cYvslwHTnXKZl\nv/we6x/Ar51zt+S6LxERKTxlhoh66CUkzGyymdWZWV/gKmAN8Epi23Aze6+ZVSSGLr9CfCmxbI7z\nATMblhg+vYj4EpJ/ys+7EBGRQlBmiHSnu1tKWJxLfLkwA1YR703pHD6qIb7M15HEe2HuIr4GcTYa\niA/d9gXWAdOcc15DniIiEi7KDJEkmnIjIiIiIhJhmnIjIiIiIhJhgUy5WdG0Qt3+IiJpTBw40byf\nVfqUFSIimfnNC/XQi4iIiIhEmBr0IiIiIiIRpga9iIiIiEiEqUEvIiIiIhJhWodeRKKpA6rbqqns\nqMQI3zWmDkesIkZ7Tbu6TkREiqkM8kINehGJpOq2avrW9MWqDbMQnqCdw7U7Wttaae/TXuxyRETK\nVjnkhfqNRCSSKjsqQ3tyBjAzrNqo7KgsdikiImWtHPJCDXoRiSQjvCfnTmYWyuFdEZFyUg55oQa9\niIiIiEiEqUEvIpKDhx98mMknTuasE87ixp/eWOxyREQkhILOCjXoRUSyFIvFuPIrV/Kr3/+KPzz+\nB+5feD8vPfdSscsSEZEQKURWaJUbESl5n5k4nT3bXz/o8brBh3LDiruy3u+aVWsY/abRjDpyFABn\nnn8mD933EG8+5s1Z71NERIoniLwoRFaoQS8iJW/P9tdZOXjQQY9PSHHS7o2tjVsZNmJY189DRwzl\nqVVP5bRPEREpniDyohBZoSk3IiIiIiIRpga9iEiWDh9+OJs3be76ecumLQwdPrSIFYmISNgUIivU\noBcRydLb3vE2Xn3pVTa+spH2tnbu/939nHL2KcUuS0REQqQQWaE59CIiWaqqquKKq67gc+d9jlhH\njKkXTuWotx5V7LJERCRECpEVatCLSMmrG3xoygua6gYfmvO+J06ayMRJE3Pej4iIFF9QeRF0VqhB\nLyIlL5elKUVEpHxENS80h15EREREJMLUoBcRERERiTDPBr2ZNZjZ6qSvFjP7ciGKExGR6FBeiIgU\nh+cceufcWmAcgJlVApuARQHXJSIiEaO8EBEpjt5OuTkNeMk592oQxYiISMlQXoiIFEhvG/TTgTtT\nbTCzWWa2ysxWLZ6/OPfKJHRWLl3J7JmzmXHKDGbPnM3KpSuLXZJIUX3rkm/xgSM/wNSTpxa7lDBK\nmRfKivKgvBDpLui88N2gN7MaYApwT6rtzrl5zrnxzrnxUy6ekq/6JCRWLl3J3OvnUjujlnHzx1E7\no5a518/VSVrK2rkzzuX6RdcXu4zQyZQXyorSp7wQOVjQedGbHvozgSecc1uCKkbCa+GChYycNZKB\nxw2koqqCgccNZOSskSxcsLDYpYn4tnP7Ti49/3KadjTlZX/j3zeeAYMG5GVfJUZ5UcaUF1IKopYX\nvWnQf4w0022k9DWub6S+ob7bY/UN9Wxev7lIFYn03sL5S1izah/33KKpHgFTXpQx5YWUgqjlha8G\nvZn1BT4E/D7YciSsho8eTsvalm6PtaxtYdjoYUWqSKR3dm7fyeI7/s5hQ69j8R1/z1uvi3SnvBDl\nhURdFPPCV4PeOdf6/9u7//io6jvf469vfkkCJCAghF9Cuzb+/kFZlt1uXa1u1VZQd5Fl1wrtrYtX\nu7ar5VH6Y9due+vt9l4X2tquK9Uq2FpXabHg3dZqkaXbFREtioigokIgGKBJJiSBJDPf+8ckQyaZ\nOedkZs7MOTPv5+ORB2HOzDmfHPG8v/l+v+d7rLXjrLVtfhckwbRg8QIaVzXSurOVWG+M1p2tNK5q\nZMHiBYUuTcSTtQ9toLd3HqeM+AC9vfNC0+sSNsoLUV5I2IUxL1zXoRcBmHv5XCA+N/LtfW8zafok\nbr7l5sTrbrY8s4W1a9bStK+J+un1LFi8wPNnS43OVe7197bUjHoUgJpRN7D+kUVc/6n5jBk3psDV\niRQX5UX+6FzlXljzQg168Wzu5XMzulD0r3gwdelULmy4kMjuCPfde19in3KSzpU/+ntbairGA1BR\nMZ7Ovl6Xv122OOP9fuFTX+CF37xA69FWLmu4jM98+TP8xZK/yFXZIqGlvPCfzpU/wpoXxlqbs531\n29y6Ofc7ldBatngZ1TdUM+ack7/Ztu5spevHXdy95u4CVhY8OlfeVXdWU11b7em9n/74Z9n/9tEh\nr0+bOY4H/t93c11akq5IF101XUmvXTzmYuPrQUNCWSGD6Rronc6Vd6WQF+qhF9817WviwoYLk16r\nbajl7X1ve/p8KQ0pZnuuJDW/L8IikhvKC++UF/4Ia14M90mxIsOWzYoHpfaAEq0OISKlTHnhnfJC\nBlKDXnyXzYoHpfaAEq0OISKlTHnhnfJCBtKUG/FdNiselNqQYrarQ4iIhJnywjvlhQykBr3kRaYr\nHvQPKQ686afYhxQzPVciIsVAeeGd8kL6acqNBFopDilueWYLyxYv44ZLbmDZ4mVFO/9TRCSXlBfK\ni1KmHnoJtFIbUtS6wuFyqPEQX176ZY42H8UYw4JPLeATt36i0GWJlCTlhfIiqPKRFVqHXiRAtK6w\nd8NZV9gvhw8d5vChw5x94dl0tHfwVx/+K77z6Hd4/5nvT7xH69Cnp6wQyZzywrtC54WXrACtQy9F\noJTWDnZSajd15dNvnv4NDz/0MAffPcjk0ydz4ydv5MN//uGs9jlh0gQmTJoAwMjRI5nZMJP3Dr43\n5CItIrmhrDhJeeGfXOdFPrJCDXopOA0bnlSKN3Xlw2+e/g0rvr+CaX87jYvOvIi219tY8f0VAFk3\n6vsdePcAr7/yOufPPj8n+xORZMqKZMoLf/idF35lhW6KlYIrtbWDnZTiTV358PBDDzPtb6cx9tyx\nlFWUMfbcsUz722k8/NDDOdl/57FObv/E7Sz/5+WMqh2Vk32KSDJlRTLlhT/8zAs/s0I99FJwGjY8\nqdRu6sqXg+8e5KIzL0p6re7MOva+uzfrfff09HD7J27n4ws/zuXXXJ71/kQkNWVFMuWFP/zKC7+z\nQg16KTgNGybTusK5N/n0ybS93sbYc8cmXmt7vY3Jp0/Oar/WWr76ma/yvob3seS2JdmWKSIOlBVD\nKS9yz4+8yEdWaMqNFJyGDcVvN37yRvb/YD8tr7YQ643R8moL+3+wnxs/eWNW+/3dc79jw0828Px/\nPs+CP1nAgj9ZwOanNueoahEZSFkh+eBHXuQjK9RDL3nhtDKB27BhGFc1CGPNxaz/RqaHH3qYve/u\nZfLpk7njM3dkfYPTrD+ZxY72HbkoUUT6pLt+epliEsZrbxhrLmZ+5EU+skLr0IvvBq5MUNtQS2R3\nhMZVjZ7m+mXz2UIJY81hVOh1hb3SOvTpKStkMOVF8GsOo1LIC0899MaYMcD9wLmABf6Htfa5YdYp\nJWrgygRA/M+l8dfdLljZfBbcez786BnJtmaRMFNeSDaCmhd+9aIrLyRXvE65+Q7wS2vtAmNMFVDj\nY01SZLJZmSCbz7qtWezXmsZaiSE/LBZrLcYEt7PbWoul5DqhlReSsSDmBeDb+vfKi/wohbxwvSnW\nGFMHXAw80HfAbmtta8ZHlJLTvzLBQF5XJsjms25rFvu1pnE2NYt30bIotid+kQ4iay22xxItixa6\nlLxRXki2gpgXfq5/r7zIj1LICy899DOBw8CDxpgLgBeBz1lrOwa+yRizFFgK8PmVn2f+J+dnXJQU\nlwWLF8R7M5YyZI6gl89+Z8V3GH39aCqnVtLT2EP74+187o7PuX62aV8Tk49M5vnPP0/XgS6qp1Rz\n+vzTObTvUGK7Hz0j2fy84l1PVQ8d3R2UHy/HELxeF0v84txT1VPoUvLJNS+UFeIkiHlhsb71oisv\n8qMU8sJLg74CmAXcZq193hjzHeCLwD8mFWPtKmAV6EYnSZbtwy9iXTGOrD1Cd1s3VXVVVHZVevpc\ndXU1e368hymfmcLID4ykY08He76/hwnVEwD/1jTWwz7ypAx6RvTQQ0k1mIPONS+UFeIkiHkxZsIY\n39a/V17kSQnkhZcGfSPQaK19vu/va4lfoEU8y/ThF2vXrOUDX/xA0oW0dWerpxuGTLlh/HXjqZ5Z\njSk3VM+sZvxfjsesi/927mfPiB72ISVKeSFZC1pe+N2LrryQXHBt0FtrDxlj9htjGqy1u4HLgNf8\nL03yLYhr4Tbta6Jmcw0vfOkFejt6qRhZwbSrptG1r8v1s53HOjljzhm0/b6NE90nqKyqZNqcabz5\n8JuAekZEck15UTpKKS+UFRIGXle5uQ34cd+KBXuBT/lXkhSCXyu+ZCvWHaNxUyPTvjKNkeeOpOPV\nDhpXNDLGjnH9bP30eqIHo0w9Z2ritdadrUnDpOoZEck55UWRK8W8UFZI0LmucgNgrd1urZ1trT3f\nWnuttbbF78Ikv/y8iz8b7R3tTPnsFEZdMIqyyjJGXTCKKZ+dQntHu+tn9ZhwkfxTXhQ/5YVI8Hjt\noZciF9S1cLuPd1N3fh2x3hhRG6XMlFF3fh37ju9z/ezcy+eye/tu1n1lHZ3tndSMruG6Rdcl9bKs\nvns16x5N3r5k2RI/fyQRkVArxbxQVkjQqUEvgH8rvmSrZnQNHa91UDurNvFa5KUINaPdn1Wz5Zkt\nbHpuE+fddV7iRqZNqzbR8EwDcy+fy+q7V/PYE48x7UvTGH3eaNp3tPPYyscAdKEWEUmj1PJi9/bd\nygoJPE9TbqT4BXW48bpF17F/5X4iL0WwPZbISxH2r9zPdYuuc/2s27DwukfXMe32adTOqsVUGmpn\n1TLt9mmse3Sd3z+WiEholVpeKCskDNRDL4C3FV+yWdXAbbgy3b7737Pum+vY276XmtE1LFy00NNn\n3YaFO9s7GX3e6KTto88bzd72vVn/vCIixaqQeeG0X7e8cPqsU164ZUW2P69ILqhBLwlOd/Fns6qB\n29QWt30vWbYk7bCm02fdhoVrRtfQvqM9aXi2fUc7NaNrAruKg4hIEBQiLxoubHDdb7q8cKvJKS+O\nHTuWNiuy/XlFckVTbsSTbFY1cBuuzGbfTp91GxZ2Gp4N6ioOIiJB51de+JUV4DyNyG0qj/JCgkA9\n9OJJNqsauA1Xuu3bafi1aV8TZ0w+g8Z3Gunp7qGyqpK6yXUc2nfIdVjYaXj2hktuCOQqDiIiQedX\nXnjZb7q8cMoKcJ5G1L8t3VSeoK76I6VFDXrxJJtVDZymtrjt2226Ts2oGvZv3c+YPx5DzYgaosej\n7H9uP9WjqgH3h4GkG54N6ioOIiJB51deuO3XKS/csgKc88Jp6qfyQoJAU27Ek2xWNXAbrnTat9t0\nHRu1ND/STMtzLUTejNDyXAvNjzRjozbrn3fPP+/htzf/lmcXPctvb/4te/55T8FXcRARCTq/8sJt\nv0554VdW9P+8ygspNPXQiydeVjVIx23lAad9u03XaWluITYyxuGfHqa3rZeKugpiPTFaWrN/OGVZ\ndRnjF4yncmolPY09tD/u/rRBEZFS52deOO3XKS9arH9ZAcoLKTxjbfa/nQ62uXVz7ncqJena2dcy\n6UuThjwo5NA3D/HEtidct2dq2eJlVN9QnTSE2rqzla4fd3H3mrsz3q8IwMVjLjaFriEIlBWSS055\nAPiSFaC8EH95zQtNuZGc2fLMFpYtXsYNl9zAssXL2PLMlqz36TZdp6K8goqxFUS7omAh2hWlYmwF\nFeUVWdXUtK+J2obapNdqG2oTN1CJiEhm/MgKcM4Lt6zIpi7lhQSBptxITvi1Dq/b8OuMhhnQCtGq\nKCe6T1BZVcmI1hHMaJiRVU26yUlEJPf8XLPdKS92vLIjbVZkW5fyQoJAU24kJwo15DjwIlzbUEtk\nd4TGVY3cfMvNrF2zNuOanParB4VItjTlJk5ZUXqCmBVzL5+bVV3KC/GT17xQD73khJ/r8DqtQ+90\n89U937gn45qyualLRERS83vN9nR54XZNz6Yu5YUEgRr0khN+DTm6rUMP6dcOzrYmtzXsRURkePyc\nnuKWF07XdOWFhJ1uipWcyGbdYSdu69AXoiYREcmMn9dl5YWUMvXQS07MvXwuv/7pr/nPv/9Poj1R\nyivL+bNL/izRY+E0bcZJZ3sn0Y4oe76wh+6mbqrqq5gwbwKd7Z2J92x5Zgtr16ylaV8T9dPrWbB4\nQaK3ZPf23az7SvJx1YsiIlIYblkB/uVFuqzor0t5IWGmBr3kxOq7V/PfO/6bGV+bwYiGGo7v7uS/\nv/vfrL57NYDrtJl0KssraXqkiSl/N4WaD9TQuaeTA987QGV5JeC8MgHApuc2cd5d5yVuVNq0ahMN\nzzToIi0iUgBOWbFk2RJP0yzTccoLt1VstjyzRXkhoeZplRtjzDtAOxAFeq21s53er5ULSs+1s69l\n4pcmMuKsGoypwtpuju/q5L1vvgdk/kCPGy65gcq/rqTuQ3WUVZcR64rR9ts2en7Sw483/dhxZQJA\nD/uQQCrmVW6GkxfKitLjlBXZPizQKS/qp9c75oEeDiVB5ccqN5daa49kWI/kkNOwYTYyHeaE+FBn\nzZkjiVEGGKCcmjNHJoY60z2O24v6i+ppa2ojGo1SXl5O/UX17P/JfsB5ZQKL9XU1BRFJS3kREEHL\nC7es6Gzv9CUv3Fax8Xv1HRG/acpNyPj1UI5shjkBqkdVE9kRYfQF4wEwpoLIK0eoHlWNMYb2He1J\nPS7tO9qpGV0zZD9bX0j++8gx9fQ2VTBy4qm0t3QxcnQ1vU2GmjGT2PpCfHvzyxFqzxqwMsGuCDVj\n4isTpNs2+Dgi+XTxnxe6AikFQcwLp6wAqBld4zkvBqufXk9FSwVjx5/K7w93MfbUasqOmMRKNU6r\n2GS7yo1Tpsz5Q0+7EMmK1wa9BX5ljLHAfdbaVT7WJA7WrlnL1KVTExedMeeMgaXx17O5QK97dB3T\nvjQtcRFNrA7wzXWeGvTvn/4BXvv2TqbeXsnIs2rp2BXhwLcPcvb0c7jgQ+fy2MrHmHb7yYv//pX7\nWbhoYcp9NRy/OPH9oku/xoP/dgeV88qontnA0Zd207MhxqeuXEHD8YtZdOnXWL1qGZU31VB7xlgi\nb7TQfP8Rlnw0PkSabtvAY4hITikvAiKIeeGUFQDXLbpuWHkx0ILFC7j3+/dSfsUIamY20PTCbqJP\nHeeWz9wCEP9lZilDHv7U/9l02710AKXLlN0jNoeyA0m/hISP1wb9n1prDxhjTgOeNsa8bq3dPPAN\nxpilwFKAz6/8PPM/OT/HpQr4NyyYGOa00NPdQ2VV5bCGOWPRU4g1VvDOP76BjVpMuaGss4bYaaew\nZNkSGt9qZPM/bKa3p5eKygouvuRiT78ofHDWVaz7+Xd5/du/xcb2YsoqOXPqh/jgrKsS2wF+9oP/\nw+/2/j/e/75ZLLni7sTrAOsfXMnbzduYeNoMlnw0eZuI5JxjXigr8ieIeeGUFUBWeTH38rk89v2f\nsmPFLmzsHUxZJeedc1bSLy+PPvAoL778Eh+44Iykhz+lejjUpZffTFld/PVMO4HC2Hnk9kuIGvvB\n5KlBb6090PdnszFmHTAH2DzoPauAVQB3P73ZhvE30jBwmmKSzTmvqq6h7ZV2Rp43klgMenqidOzo\noKq6JuV+B/8PveizV3PfvQeYuvSCpN6NRbdczZZntvBm85vM+facxLY3V73Jlme2pOwl2j3i5D+t\n7f+5kbc6nmPaHXOonjGOrneO8taDz/HvW/+Jv5rzT0C8Ub/3rV2886rhwj+4OqnB/sFZV6kBL5JH\nbnlRallRyMaPXw9x6p8W058X0d54XniZFuOUFcCw82Kgjes2srtpN9M/P4ea942jc+9Rdj+4jY3r\nNvKR6z7C3Mvn8uYr+3lr60hm/+GFQ/bXv9zxwH+TYWyQZ8vpZw7TiEOp/eLhusqNMWYkUGatbe/7\n/mng69baX6b7zIYNaOUCn7z40i9Y/atlTLlpemIayYH792Xd8/zoY19n/Yt3M+mz9Yw8eywdr7Vw\n6LtNzP/gMhYtvDPpvbtHbB7yP4rbajNd53ZxeNth2ve1M3r6aCbMnkD1q9Wuqwd84pLF9F5USeeb\nnfQ0d1B52khq/qCG6JYe7v8/7wAQiRxh+fKFlJffSzR6C9/61uPU1o5L7CMSOcLKlTdz++2rkl4X\nKZR58yjKVW6GmxfFnhUDOyec+NXwGDiHfvA0klzMoZ/0dwPy4ntNLLx2oWtPuttqMssWL8t5XlT8\nrocfbVpD69FWbrrkf1NW/q/EorfymW9/hZG1dUn76Ghr5dH/u5J/+Oxa5UWIef1/r5C8/n+fy1Vu\nJgLrjDH973/EqTEv/upvtOd6GsmihXfy6s7neP3O32Jj+xJTWwY35tNxGtrtau8i1hZj4v+YyKSZ\nk+l++zgHf3iQskb3BxU37z9EVfUp1N9UT/UZp9H1RhdN9zdx/NCJxHueemoN0ejVVFc3cOzY1Tz1\n1Gquv/6OpO179rQMeV1Eck55MYCX3l2vPZ6ZNPpTTSPJtjEP8WkxL//2VXbcuQsb2983teVsT9Ni\n0mXFrjfi04De3vn2kLw48MBByg8458XWF+DQu4cYkSov3j3B1hfg1z95mmh0HnUjZnHsxELe+vkb\nQzLh8fUrOLDLKC9CLgwjK1tf8PZLh9dFFFwb9NbavcAF3nYn+eDHNJJI5AhHm08wZdReysvHE40e\n4WjzQiKRo556KZyGdt987U0mLp7IKWdUY0wVp5xhOG3xaYk16p3UjK1h0k0n1yQeMesUym8qo/F/\nHUrUvXHjk1RXx1dYqK5ezMaNC7niiiXU1o5LbB879l42brwl8bqI5J7yYvgybfR7beD3TyPJpdaj\nrTQ3GqZNfIWKinH09h6luXERbb9vo+7UOsfPDsyKzr4Hfkd2RTh1YnzaaNREOe3GeF7Aybw4+I33\nXKd/jqirYcqn66m7IJ5DIy6ooezT5Rz6xmHqm89mx692Ujfiq8DQrACUF5JXuf6lw72LVEpCfy93\neXl8KbF4oz7e2+3FgsULaFzVSOvOVmK9MVp3ttK4qpHz/3gBNlZBWW05sRMQi0HsBJTVlmNjFWx9\ngcTXs8+0cvMVX6Xt922J/VaUV1AxtoJoVxQsRLuiVIytoLysIqlugIMH/xIwSXU/9dQaenqu4siR\ncnp6rkr580QiR/ja1/6SSORoNqdQRMQXDccvTvoCkq6dqb789OTqp4n2zqOiIt7YragYR7R3Hhse\n+tWQ2gb//fw/XsBb9zRy6MV4VnTt7KZ51REWXfo1AGysgvK6eF5YG8+L8roKTimrSfzs9c1n8+iX\nvkNHpC3pOOVlFVSOraT3eC/WWnqP91I5tpKKsirXrAD3vFBWSJBpHXoBYPv2TUSjB2lre2TQ65NT\nDjsODoyyurlcejn8171r2fXe25w6cRKXXnEzZ86ey7QN59Cxp4lToiOwthNjKjnxVi/TJp2T9Bvq\n4+tXsO/1Xv71G7/isr++HoBT62dgmqGnPMrxnhOUV1ZimkcwcfqMpLrfe+97dHWN4b33/pCRI09l\n+/bJXHHFYjZufJLu7ns4fjxKVdXH2LjxtiG9LpqSIyJh4qVnz8twfqZz97dteplo77O0HvlJ0usb\nN5zKBz58faLGgSML/TU3nHsxU7vPY/0DK3m9eXfytNHjMG1SqrzooX7SGYn9PP7UCvbsaeGtn7/B\n+TfOTrw+c+qFlDV3ESs/QXdvJxUVVVQ1j2ba1NMcs+L66+9I9M475YWyQoJMDfoS4XaDyOJ/WZZ+\n4/Hkv6YLk4ZzL+aac78w5LMTas5g7+o3GH/jFKpPn0Dnu4c58vAh/mj6GYm39V9Mx4+5nx2/uoVP\nXPZP1NaOi68z//Cgm4Af3pdYZ/6uu9YnboodPz75ptjHH19BT89VdHRUUFFxOh0d71JVdVXSxVhD\nrCJSjNwa/dnM3f/2+m8k/T3dqjDpanCaNuqWF07X7PmX3c7qHw3Kix+dXDQiXVbAyd75dHmhYmgD\neAAAG9hJREFUrJCgU4O+hGTSG7P1hfiFP5u5XoebD9G733JgxQtYohjKKT8+msMjDiXe038xbWkp\nZ9SokxdRL+vMp7spdvv2TXR0vE40+gPKymqJxSJ0dMD27WcmGvRuN9SKiGQqyCtt5PKGXa/788It\nL9JlBbg/l8Tpeu+WF8oKCTo16MXRnD/Mfj7m8uU/ZPnyhXR03EMkUsHo2l5GjruN5csfBHAd6nRa\nZ97pptj+45522mOJG327uxcOOW66G2pFRLLRP+3EKz+Wr8zm+l2IlUKc8sLLtJh0eeF2vXfKC2WF\nhIEa9OI7t6HMbIY63W7mTbdtYI9Luu0iItny2ij284E9YVjCr59THgAZT6N0u95nkyUiQaAGfUj5\n8bCkLc9sYe2atTTta6J+ej0LFi/IyXJnJ4cyV2FtN8ZU0dFhEkOZ/dt7e/8Na49hzCg6OsqThjpP\nnPgov//9CU499aNDhknT3cwLEI0epKVlDZFIE7W19ZSVVSRugvJyI7AeSiUi+eBno9uP69iLL/2C\n9b9eyXvNbzPxtJnMv+z2nCyn7JQXgGNWAGnzwu163789VV4AnhaNUF5IIbk+KTYTxf70vyB4/PEV\nPPHEk1x77dWeeghSPd11IKcnCpbVzc0qbPpvROrsvJS2tmepq7uUmppnEzckOW0Hy/LlCzl69J85\nfnwUI0YcY9y4Lw55GqyT4Z6rXH1WJJ1ifVLscCkr8iPX1zG/nlgOznnRnwduWaK8kGLiNS+0Dn0I\nJQ8pPpmTNXHXrlnL1KVTGXPOGMoqyhhzzhimLp3K2jVrs953fAj1Eo4de5by8ns5duxZenouHbRW\nfOrt/b0tx49XY8wMjh+v5sSJj3peHz+bc+XHeRYRySc/rmPrf72SKTdNZ8xZ4+J5cdY4ptw0nfW/\nXpn1vt3ywC1LlBdSqtSgD6H+uX5VVQ0pH/6UycMvmvY1UdtQm/RabUMth/YdSnotk31v376JY8ce\nIhq9AGujRKMXcOzYg2zfvsl1+/btm2hruxe4Dmv/BLiOtrZ7E591q8vtXDnJ5rMiIkHgR1681/w2\ntWeMTXqt9oyxvNf8Ttb7dssDtyxxywunmpQXEmZq0IfMybvtFwP9d9sn9wYMfPjFQE5PFRw5pp7m\nlyN0dpL4an45Qs2YSUnTbdLt28ny5T9k7NgZnH76V5gx42xOP/0rjB07I7HajNP2W2/9v1hbSVnZ\nE8C/U1b2BNZWcuut/5J0jFR1eTlX2ZxnEZEgyyYvnEw8bSaRN1qSj/VGCxNPm5H0Wq7zwi1LvORF\nupqUFxJ2atCHjNud+OmG/QY/Onzw16JLv0bzqiN07eymsnvkkMdxO+0725r7t0ejPezdeyXRaDSx\n/b77vkgstghrKzHmdKytJBZbxH33nXyAVbq63I6bTc0iIkGXaV64mX/Z7Ry4fx+tu44S643Ruuso\nB+7fx/zLbk+8x4+8cMoKwDUvnGpSXkjYqUEfMvE78R+hrW124isafSQxpJjpsN8HZ13Fko/eTeeD\nUV6+aRudD0aH3OCU6b7dau7f3tQ0m1jsEE1NsxLb33jjd1j7ANZ+DGv/qO/PB3jjjd+51uV23Gxq\nFhEJumLLC6esAFzzwqkm5YWEnVa5KSL9d/hXVSU/GGM4d/gXYt8ABw7s5rbbPo4xP8Pav+Cee37B\nlClnuB7X77pE/KBVbuKUFYUT1rxIlxVux+1fIUdZIWGjVW5KkJ/Dfl73nclNUHByqDQarUkaInUb\nYtVQp4jI8BU6L3KdFQOPmyovlBVS7NSgLyJ+Dvt53XcmN0EdOLCbV1/dASwiFosCi3j11R0cOPCG\n6xCrhjpFRIav0HmR66wYeNxUeaGskGKnKTeSM/3DneXl9xKN3uJ5KPPOO6/jlVfOwdobgdOBdzHm\nYc4/fydf//o6xyFWkTDSlJs4ZUVp8isrwHlKjkgYacqN5F38oR9XceRIOT09V3mejuN2I5PTEKuI\niISLW1ZA6rzwskiC8kJKlRr0khP9y4F1d3+M48ejdHd/zPN6x/fdt43Jk89i5szn+YM/eJmZM59n\n8uSzuO++F12HWEVEJDy8ZAWkzgunrAD3KTkixUwNesmJ/h6Xjo4KKipOp6OjIqnnJdP1f72sQy8i\nIuHglhWQ+bNFlBdSyjw36I0x5caY3xljnvSzIPHGaYWATFcP8PLZdNu3b99ER8cPiEbnEYv9KdHo\nPDo6fpC03nG6IVanm5W8DLGKSHAoK4InSHnhlhWQPi/cbmxVXkgpqxjGez8H7AJqfapFhmHgcOT1\n19/heVs2+3Xavnz5D1m+fCGnnZa8xu/y5Q8OGGK9h+PHo1RVfYyNG2/jiiuWUFs7jrvuWp+2Hud1\nhUUkgJQVAROkvHDKCsAxL5yyAuJTcpQXUqo89dAbY6YCHwfu97cc8cJp+kqmj9v28tlMp814GWJN\nR2sHi4SHsiJ4gpYXbtd05YVIZrxOufk28AUglu4Nxpilxphtxphtv/zlqpwUJ6k5Pb4608dt93/W\naeUBL4/NPnr0fN56ayxHj56ftP6v2xBrOlo7WCRUlBUBE7S8cMqK/u3KC5Hhc12H3hhzNfAxa+2t\nxphLgGXW2qudPqO1hf3j16Ot+/fb0XEPkUgFtbW9jBx5W+KzXh/lfeed17F9+z4uvHB6Yl1gPx8D\nLhJGxbgOvbIieIKcF6mywq1m5YWUolyuQ/8hYL4x5h3gUeAjxpgfZVGbZMFtakumw41uw5xe9t2/\nZFhl5YNJS4VpGFSkJCgrAiaoeZEuK7x8VkRSc23QW2u/ZK2daq2dASwCNlprP+F7ZZKS05Ci1+HG\nTFYe8LLv+JJhf0NZ2fnEYn+TWCpMw6AixU9ZETxBzYt0WeHlsyKSmuuUm6Q3axi1KDz++AqeeOJJ\nrr326sTKA9kOc5583PZvKCurJxZrwtoP67HbIikU45SbgZQVxSPXeaGsEBmeXE65SbDWbnK7QEuw\nRSJHePrpn9HTU8nTT6/zvPKAm5M9LvUAfRfqv9EDPURKkLKiOPiRF8oKEX/oSbEl5qmn1nDs2AS6\nu3s4dmy85wd2uIk/uOMBenqmJL5AD/QQEQkrP/JCWSHij2FNufFKw6jBFIkcYdmyazlypBxjVmHt\nUsaPj3H33U9o9QCRPCr2KTdeKSuCS3khEgy+TLmRcOvvbYFrKCtrAK5J6nUptGweQS4iIrkT5LxQ\nVogMpQZ9Cdm27Sm6ul7G2j8iGn0Na/+Irq6X2bbtV4UuDUh+TLiIiBROkPNCWSEyVEWhC5D8mT37\nClparmDUqA8lXjt27O+YPbuARfVJfkz4LVxxxRIN64qIFEhQ80JZIZKaeugLKN/DhrlY39evmrN5\nBLmISLFTXsQpK0RSUw99AQ0cNuxf39dPd921Put9+FFzf49LdfVjAFRXL2bjxoXqeRER6aO8UFZI\nkdu6NfXr8+Z4+rh66AskedjwyVDc3ONXzXrUt4hIesqLOGWFhN7Wrem/gHlz3hvy5ZV66Auk/8JU\nXd3AsWNX563XJRt+1Rwf2j1IW9sjg16fHPhzIiLiN+VFnLJCQiFdT3uf4TTSh0MN+gII47ChnzXn\nYmhXRKQYKS9OUlZIoKRpuPvVYHejKTcFEMZhwzDWLCISdmG89oaxZhFXOZ4ik2vqoS8Av4cNI5Ej\nrFx5M7ffviplb4jb9kLULCIiQykvRPLAZZpMv0I22N2oQV8Afg8buq0skMnKAxrqFBHJP+WFSI4U\naG57vqhBX2TcHrqhh3KIiAgoL6QIFXmj3Yka9EXGbWWBMK6WICIiuae8kNBL0YAv5ka7EzXoi4jb\nygJhXC1BRERyT3khgeFx/noqpdp4T0UN+iLitLLA9dff4bpdRERKg/JCciaLBnk/NcyzpwZ9EXFb\nWUArD4iICCgvZJgcGu1qjAeDsdbmfKcbNpD7nYqIFIl58zCFriEIlBUiAROwhyUJMG+ep7xw7aE3\nxowANgOn9L1/rbX2q9lVJ0GVyZrDIiKgvCg1youQKYK11iU9L1NuTgAfsdYeM8ZUAv9ljPmFtXaL\nz7VJAWSy5rCISB/lRQlRXgSUpseUJNcGvY3PyTnW99fKvi8NkxYhrTksItlQXpQO5UWAaOlGAcq8\nvMkYU26M2Q40A09ba59P8Z6lxphtxphtv/zlqlzXKXnQv6pBVVVDYjUDEZHhcMsLZUVxUF7k0dat\nzl/EG/ADv6T0DOumWGPMGGAdcJu19tV079ONTuETiRxh+fKFVFU91rc82RG6uxfyrW89rl4XkRwr\nhZtiveSFsiKclBc+Um+7DJarm2IHsta2GmOeBa4E0jboJXy05rCI5JLyongpL3JIDXjJES+r3EwA\nevouztXAnwPf8r0yySutOSwi2VJelAblRRbUgBefeOmhrwdWG2PKic+5f8xa+6S/ZUm+3XXX+kKX\nICLhp7woAcqLYRrUiFcDXvzgZZWbV4CL8lCLiIiEmPJCZIABDXk14sVvw5pDLyIiIiJ9tOa7BIQa\n9CIiIiJO0jTc1WiXoFCDXkRERATU4y6hpQa9iIiIlCatOiNFQg16ERERKV4Ove6gBrwUBzXoRURE\nJNzUaJcSpwa9iIiIhINuThVJSQ16ERERCRbdnCoyLGrQi4iISDDoYUwiGVGDXkRERPJDc91FfOFL\ng34eG/zYrYhIkZhX6AJE/Kf57iJ5ox56ERERyZwa7iIFpwa9iIiIeKeHMYkEjhr0IiIipSrdnPY5\nc7TSjEiIqEEvIiJSjFxuQIXUDfMNWycmPquGu0g4qEEvIiISNh4a65BZg1yNeJHwUYNeREQkiLTE\no4h4pAa9iIhIIaW7yXROAWoRkVBSg15ERCRftMSjiPjAtUFvjJkGrAEmAhZYZa39jt+FiYhIuCgv\nBtAKMSKSR1566HuBz1trXzLGjAZeNMY8ba19zefaREQkXEorLzTHXUQCwrVBb61tApr6vm83xuwC\npgDFeYEWEZGMFH1e6IFKIhJQw5pDb4yZAVwEPJ9i21JgKcB9t97K0iuvzEF5IoXxkTvuINLWNuT1\n2ro6Nq5YUXTHFcm1dHkxMCtuvfU+rrxyad5rc6V57jIMygsJAs8NemPMKOCnwN9bayODt1trVwGr\nANiwweaqQJFCiLS1sa2ubsjrs1NcPIvhuCK55JQXA7NiwwYKlxWaLiM5oryQIPDUoDfGVBK/OP/Y\nWvszf0sSEZGwKnheeHzgEqjRLiLFw8sqNwZ4ANhlrdUYjoiIpFSQvBjUgFcjXURKkZce+g8BNwI7\njDHb+177srX2P/wrS0REQsi/vNC8dhGRtLyscvNfgMlDLSIiEmI5yQut3y4iMmx6UqyULKcVApoi\nEaYcPTr0Q5WVvtbkdlytaiBFRb3uEgJu113lhQSBGvRSspxWCKivrS3I6gFux9WqBlJM1HCXMHC7\n7iovJAjKCl2AiIiIiIhkTj30EmhuQ4ZO24GMhxvfPnyYCw4fHvJ6o8e6MtXY0sLslpa0xxURkdSy\nyQO3zzpxygs/p70oL2QgNegl0NyGDN22ZzrcWAm8bIbe2zfJWk/HzVR5LMa2FPMup/T0ZLVfEZFi\nl00eZHNNd8oLP6e9KC9kIDXoJdTeOnyYySl6RrqA90+YkPF+TwBT7NCHWJ7o+/O9lhZeS9Ez4mVG\n8LhrrqEyxb57jKHSWl47cWLItqiH/brxs6dIN1+JSJA5ZUW2nPIim6yA4ssLZYV/1KCXUDsFOJji\n9UkePltbV5eyl6S2ro5TDh9mf4rPTO77MxaLcXaKnpGYh56RSms55ND7PzPDmt342VOkm69EJMj8\nyor+fafLi2yyAoovL5QV/lGDXkqWU2/ApPnzC3LHeHlZGdUpLv7lfRd/9WCIiOSXl+uu8kIKTQ16\nKTinIbi9hw9zvsPNqU7cbhhyuwnKaSg2GovRmWaoM4xDimGsWURKi9t1yikvqlz27ZQXXq6P6fLC\nKSvwuO+gCWPNpUANeik4pyG4SuDRFJ+5dMD36S6klcCaFK9f4eG44BwA3cDFaV7PZkgxWlbG7OjQ\nGZDRMn/7fzQMKiJB53adcssLp04ap7zwcn1MlxdOWYHHfaejvJCB1KCXQDPA2SnmD5oBNwlVp/ns\nxLFjOTvFRWeix4tOucO2KmCbw7zGTE0dO1YXShGRDLjlRbqsAP/ywq+sAOWFJFODvogU4zCYBV5L\nceHrf+UEqW9qOgE0Hj7MyxlO1zkBTEnzeuK4DqvgOOkxJuVne4xxvYkpm//GTvtOtc/hyObmKxHJ\nv1LLi27SZwU458VUlxXT3PIi06yA4ssLZYV/1KAvIsU4DGaAs9O8DtCyfn3az06aP58LUvSM4KFn\npKasjAMO6/ueAhxKdUzXPcPRn//cw7tSy+a/sdMFfPanP51xTW77FpHgKbW8cMqKfn7kRTQWyzgr\noPjyQlnhn0LcmC0iIiIiIjmiHnopOKchuO7Dhzk/xWe6U7yW6j0XpOhd6f+s03GbIpHUT9sb0AvT\nmea4hRpS9Gt4VUQkCNyuU93gS164Xh8rK9PnxYkTabOifx/KC8kFNeil4NyG9jIdMqwi/eO43Y7r\nprysjJo06/8WakjRr+FVEZEgcLtOvW/CBF/ywu24Bx5/PO22KddemzYroHDXXuVF8dGUGxERERGR\nEFMPfREpxmGwpkiEKUePDt2QosdjsB5SD6F6e+C2C6chVh/5uVKNiJQO5UUy3/KiQFkByotSowZ9\nESnGYbD62tqMhwVnZjH86sZpiNVPfq5UIyKlQ3mRzK+8KFRWgPKi1LhOuTHG/NAY02yMeTUfBYmI\nSDgpL0RECsNLD/1DwPdI/VRkkcAK4+oB2SjGIXQJnYdQXkgIKS9Ovi7h5Nqgt9ZuNsbM8L8UkdwK\n4+oB2SjGIXQJF+WFhJXyQsJOq9yIiIiIiIRYzm6KNcYsBZYC3HfrrSy98spc7VpKmIYFRYqLskL8\noryQUpazBr21dhWwCoANG4au/SSSAQ0LihQXZYX4RXkhpUxTbkREREREQsy1h94Y8xPgEmC8MaYR\n+Kq19gG/CxMJKw37SqlSXogMj/JCcsXYFE9Gy5qGUUVE0ps3zxS6hEBQVoiIOPOYF5pyIyIiIiIS\nYmrQi4iIiIiEmBr0IiIiIiIhpga9iIiIiEiIqUEvIiIiIhJiatCLiIiIiISYGvQiIiIiIiGmBr2I\niIiISIipQS8iIiIiEmJq0IuIiIiIhJga9CIiIiIiIaYGvYiIiIhIiKlBLyIiIiISYmrQi4iIiIiE\nmBr0IiIiIiIhpga9iIiIiEiIqUEvIiIiIhJiatCLiIiIiISYGvQiIiIiIiHmqUFvjLnSGLPbGPOm\nMeaLfhclIiLhpLwQEck/1wa9MaYc+D5wFXA28NfGmLP9LkxERMJFeSEiUhheeujnAG9aa/daa7uB\nR4Fr/C1LRERCSHkhIlIAXhr0U4D9A/7e2PeaiIjIQMoLEZECqMjVjowxS4GlfX/9kbX2xlztOxeM\nMUuttasKXcdgQawriDWB6hqOINYEqkuCnxUQ3H8PQawriDVBMOsKYk2guoYjiDX189JDfwCYNuDv\nU/teS2KtXWWtnW2tnQ2claP6cmmp+1sKIoh1BbEmUF3DEcSaQHUVO9e8CEFWQHD/PQSxriDWBMGs\nK4g1geoajiDWBHhr0L8AnGGMmWmMqQIWAev9LUtEREJIeSEiUgCuU26stb3GmL8DngLKgR9aa3f6\nXpmIiISK8kJEpDA8zaG31v4H8B/D2G8Q5xcFsSYIZl1BrAlU13AEsSZQXUVvmHkR1POuurwLYk0Q\nzLqCWBOoruEIYk0AGGttoWsQEREREZEMeXpSrIiIiIiIBFPGDXpjzA+NMc3GmFfTbDfGmO/2Pf77\nFWPMrMzLzFlNlxhj2owx2/u+7vS7pr7jTjPGPGuMec0Ys9MY87kU78nr+fJYU97PlzFmhDFmqzHm\n5b66vpbiPacYY/6971w9b4yZEZC6PmmMOTzgfN3kd119xy03xvzOGPNkim15P1ce6yrUuXrHGLOj\n75jbUmzP+3Wr2AUxKzzWVYjrX+CyYhh1KS+811SQ61/fsQOXF8qKHLHWZvQFXAzMAl5Ns/1jwC8A\nA8wFns/0WDms6RLgSb/rSHHcemBW3/ejgT3A2YU8Xx5ryvv56vv5R/V9Xwk8D8wd9J5bgX/r+34R\n8O8BqeuTwPcK8O/rDuCRVP+tCnGuPNZVqHP1DjDeYXver1vF/hXErPBYVyGuf4HLimHUpbzwXlNB\nrn99xw5cXigrcvOVcQ+9tXYz8HuHt1wDrLFxW4Axxpj6TI+Xo5oKwlrbZK19qe/7dmAXQ5+emNfz\n5bGmvOv7+Y/1/bWy72vwjR7XAKv7vl8LXGaMMQGoK++MMVOBjwP3p3lL3s+Vx7qCKu/XrWIXxKzw\nWFfeBTErhlFX3gUxL4KaFRDMvFBW5I6fc+iD+gjwP+4bCvuFMeacfB+8bwjrIuK/tQ9UsPPlUBMU\n4Hz1Db9tB5qBp621ac+VtbYXaAPGBaAugL/sG35ba4yZlmJ7rn0b+AIQS7O9IOfKQ12Q/3MF8WD9\nlTHmRRN/YulgQb1uFbMgn/OC5UUQswKUFzmqCQpz/QtiXigrcqTUbop9CTjdWnsBcA/wRD4PbowZ\nBfwU+HtrbSSfx07HpaaCnC9rbdRaeyHxp0zOMcacm4/juvFQ1wZghrX2fOBpTvZ0+MIYczXQbK19\n0c/jDJfHuvJ6rgb4U2vtLOAq4DPGmIvzdFwJn4LlRRCzApQXOawp79e/IOaFsiK3/GzQuz4CPN+s\ntZH+oTAbXyu50hgzPh/HNsZUEr8Q/tha+7MUb8n7+XKrqZDnq++YrcCzwJWDNiXOlTGmAqgDjha6\nLmvtUWvtib6/3g980OdSPgTMN8a8AzwKfMQY86NB7ynEuXKtqwDnqv+4B/r+bAbWAXMGvSVw160S\nEMhzXqjrXxCzwktdygvvNRXo+hfEvFBW5JCfDfr1wOK+O4HnAm3W2iYfj+fKGDOpfz6YMWYO8Z/f\n9/+x+475ALDLWrsizdvyer681FSI82WMmWCMGdP3fTXw58Drg962HljS9/0CYKO11tc5il7qGjR/\nbj7xeaa+sdZ+yVo71Vo7g/gNTButtZ8Y9La8nysvdeX7XPUdc6QxZnT/98BHgcErnATuulUCAnnO\nC3T9C1xWeK1LeeG9pkJc/4KYF8qK3PL0pNhUjDE/IX5X+3hjTCPwVeI3f2Ct/TfiTwr8GPAm0Al8\nKttic1DTAuAWY0wv0AUs8rtx0+dDwI3Ajr55dQBfBqYPqC3f58tLTYU4X/XAamNMOfFAeMxa+6Qx\n5uvANmvteuLB8rAx5k3iN7Ut8rkmr3V91hgzH+jtq+uTeahriACcKy91FeJcTQTW9bU5KoBHrLW/\nNMb8TyjcdavYBTErPNZViOtfELPCa13KC+81BSIrIJh5EYBzFcqs0JNiRURERERCrNRuihURERER\nKSpq0IuIiIiIhJga9CIiIiIiIaYGvYiIiIhIiKlBLyIiIiISYmrQi4iIiIiEmBr0IiIiIiIhpga9\niIiIiEiI/X8tbByghoPufwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1406c3d9e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "gs = gridspec.GridSpec(2, 2)\n",
    "\n",
    "fig = plt.figure(figsize=(13,9))\n",
    "\n",
    "for clf, lab, grd in zip([clf_tree, clf_KNN, clf_bagging_tree, clf_bagging_KNN], \n",
    "                         ['Decision Tree', \n",
    "                          'KNN', \n",
    "                          'Bagging Tree',\n",
    "                          'Bagging KNN'],\n",
    "                          itertools.product([0, 1], repeat=2)):\n",
    "\n",
    "    clf.fit(X, y)\n",
    "    ax = plt.subplot(gs[grd[0], grd[1]])\n",
    "    fig = plot_decision_regions(X=X, y=y, clf=clf)\n",
    "    plt.title(lab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Forest\n",
    "\n",
    "**(David)**\n",
    "\n",
    "In random forests, each tree in the ensemble is built from a sample drawn with replacement (i.e. a bootstrap sample) from the training set. In addition, instead of using all the features, a random subset of features is selected, further randomizing the tree.\n",
    "\n",
    "As a result, the bias of the forest increases slightly, but due to the averaging of less correlated trees, its variance decreases, resulting in an overall better model.\n",
    "\n",
    "\n",
    "![alt text](algo_random_forest.png )\n",
    "<p style=\"text-align: center;\">Figure 3. Random Forest Algorithm.</p>\n",
    "\n",
    "\n",
    "![alt text](random_forest.png )\n",
    "<p style=\"text-align: center;\">Figure 4. Example of classification of one sample by the trees on a random forest with n trees.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In an extremely randomized trees algorithm randomness goes one step further: the splitting thresholds are randomized. Instead of looking for the most discriminative threshold, thresholds are drawn at random for each candidate feature and the best of these randomly-generated thresholds is picked as the splitting rule. This usually allows reduction of the variance of the model a bit more, at the expense of a slightly greater increase in bias."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy of using Random Forest is: 0.914215686275\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "# import some data to play with\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data[:, 1:3]  # we only take the first 3 features.\n",
    "y = iris.target\n",
    "\n",
    "rf = RandomForestClassifier(random_state=1)\n",
    "rf.fit(X,y)\n",
    "scores_rf = cross_val_score(rf, X, y)\n",
    "print(\"The accuracy of using Random Forest is:\",scores_rf.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f142528c0b8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEICAYAAAB/Dx7IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VPW57/HPk2QCCRCCgNzCzVbxVkGllp5WqrVeUEE5\npW5aFdtzLK329KLllF52be05nm572NjL3rVQb9BWu5UWRbutlYPW7m5RUfEuosgtBCEIScg9M8/5\nYyYhIcnMSmYmM5N8369XXiTr8ltPFvrwm+f3+61l7o6IiOSOvEwHICIiPaPELSKSY5S4RURyjBK3\niEiOUeIWEckxStwiIjlGiVtygpmdY2a7Mx2HSDZQ4pZeM7PtZlZvZofNbK+Z3WNmQzMdV7LMzM2s\nNvZ7HTazQ318ff0jJXEpcUuy5rr7UGAGcDrwnQzHkyrT3X1o7Ku0pyebWUE6ghIBJW5JEXffCzxG\nNIEDYGaXmNmLZlZtZrvM7Ift9k2J9WyvMbOdZlZpZt9rt78o1oM/aGavAx9ufz0zO8nMnjSzQ2b2\nmpnNa7fvHjP7pZk9Gusx/93MxprZT2PtvWlmp/fm9zSzL5rZ22b2vpmtM7Px7fa5mX3FzLYCW2Pb\nTjSzx2PHbzGzK9odf7GZvW5mNWZWbmZLzGwI8Cgwvl2Pf3ynQGRAU+KWlDCzMmAO8Ha7zbXAIqAU\nuAS4zswuP+rUjwPTgPOAm8zspNj2HwAfiH1dCFzT7loh4GHgL8CxwFeB35nZtHbtXgH8IzAKaASe\nBl6I/bwGWN6L3/GTwI9jbY8DdgC/P+qwy4GPACfHkvDjwL2xOBcCvzSzk2PH3gl8yd2HAacCG9y9\nluh93NOux7+np7FK/6bELcl60MxqgF3APqIJFwB3f9LdX3H3iLu/DNwHfOKo829293p3fwl4CZge\n234FcIu7v+/uu4CftztnFjAU+Cd3b3L3DcAjwGfbHbPW3Z939wZgLdDg7qvdPQz8G9GyTjwvxHrz\nh8ys9dpXAne5+wvu3ki0LPRRM5vS7rwfx2KuBy4Ftrv73e7e4u4vAn8APhM7tplogi9x94Pu/kKC\nmEQAJW5J3uWxHuM5wIlEe7QAmNlHzOwJM9tvZlXAl9vvj9nb7vs6ogkZYDzRfwxa7Wj3/Xhgl7tH\njto/od3P77X7vr6LnxMNop7h7qWxr6+1u25bHO5+GDhw1HXbxzwZ+Ei7fwAOEU3+Y2P7Pw1cDOww\ns7+a2UcTxCQCKHFLirj7X4F7gGXtNt8LrAMmuvtw4FeABWyyApjY7udJ7b7fA0w0s7yj9pf3MOye\n2kM0GQMQK4WMPOq67R+3uQv4a7t/AEpjpY/rANz9OXe/jGgZ5UHg/i7aEOlEiVtS6afA+WbWWu4Y\nBrzv7g1mdhbwuR60dT/wHTMbEauff7XdvmeI9s6/ZWYhMzsHmEvnenOq3Qd8wcxmmNkg4P8Az7j7\n9m6OfwQ4wcyujsUZMrMPxwZWC83sSjMb7u7NQDXQ+gniPWCkmQ1P8+8jOUqJW1LG3fcDq4GbYpuu\nB34Uq4HfxJEeZRA3Ey1LvEt0EPI37a7TRDRRzwEqgV8Ci9z9zWR/h3jcfT3wfaJ16gqiA6cL4xxf\nA1wQO2YP0bLQrcCg2CFXA9vNrJpoGenK2HlvEv1HYlusxKJZJdKB6UUKIiK5RT1uEZEco8QtIpJj\nlLhFRHKMEreISI5Jy4Nwnjr0lEY8RUR6YHbp7KBrHNTjFhHJNUrcIiI5RolbRCTHKHGLiOQYvaVD\nRPqPCISaQuRH8rHAzzPrO44TzgvTXNicVLdZiVtE+o1QU4ghhUOwkGGWhYnbHW92aptqaR7c3Ot2\nVCoRkX4jP5KftUkbwMywkJEfyU+qHSVuEek3jOxN2q3MLOkyjhK3iEiOUeIWEUmx/3j8P5h7+lwu\nnn4xd/zzHSlvX4lbRCSFwuEwt3zzFn75x1/y0HMP8eiaR3nnzXdSeg3NKhGRAemLsxdSV/l+p+3F\no47h10/1/i14r2x6hUnHTWLi1OgrU+d8eg5PPPIEHzjxA71u82hK3CIyINVVvs/GUSM6bZ/VRTLv\niX0V+xg7YWzbz2MmjOHlTS8n1ebRVCoREckxStwiIil07Lhj2Vu+t+3n98rfY8y4MSm9hhK3iEgK\nnXrmqex4Zwe7t++muamZR//wKOdcck5Kr6Eat4hIChUUFPDdZd/ly5d/mXAkzPyr5/PBkz6Y2muk\ntDURkRxRPOqYLgcii0cdk3Tbsy+czewLZyfdTneUuEVkQEpmyl+mqcYtIpJjEiZuM5tmZpvbfVWb\n2Tf6IjgREeksYanE3bcAMwDMLB8oB9amOS4REelGT0sl5wHvuPuOdAQjIiKJ9XRwciFwXzoCkdyx\ncf1G1qxeQ8XOCsZNGseCRQuY9alZmQ5LZMAI3OM2s0JgHvBAN/sXm9kmM9u07p51qYpPsszG9RtZ\ncfsKiq4sYsY9Myi6sogVt69g4/qNmQ5NJCt8/7rv84mpn2D+WfPTdo2elErmAC+4+3td7XT3le4+\n091nzvv8vNREJ1lnzeo1lC0uo/SUUvIK8ig9pZSyxWWsWb0m06GJZIXLrryM29fentZr9CRxfxaV\nSQa8ip0VlEwr6bCtZFoJe3fu7eYMkex2sPIg13/6Rg4dOJSS9mZ+fCbDRwxPSVvdCZS4zWwIcD7w\nx7RGI1lv3KRxVG+p7rCteks1YyeN7eYMkey25p6HeWVTAw/cnTsl3kCJ291r3X2ku1elOyDJbgsW\nLWD3yt0ceu0QkZYIh147xO6Vu1mwaEGmQxPpsYOVB1l3798ZOeYXrLv37ynrdaeblrxLj7TOHlmz\neg3v7nyXsZPG8qXrvhR4VolmpPSc7ln6rLnnYVpa5lI89ASqD83lgbvX8cUlizIdVkJK3NJjsz41\nq1eJo3VGStniMmZMm0H1lmpW3L6irU3pTPcsfVp728VDo88sKR56JevuXchnvjCP0pGlGY4uPj2r\nRPqMZqT0nO5Z+rT2tgsKRgFQUDCKlpa5Sde6v/WFb3HVeVexfet2zpt2Hn9clfqhQfW4pc9U7Kxg\nxrQZHbaVTCvh3Z3vBjp/IJYMkr1n0r2NT2wi3HKAg5X3HbV9ZFLlkp/c/ZNkQ0tIiVv6TOuMlNJT\njnwMDTojZaCWDJK5ZxLfnX/6eaZD6DWVSqTPJDMjZaCWDDSLR7qiHrf0mWRmpAzUkkGys3ikf1Li\nlj7V2xkpA7lk0Nt7Jv2XSiWSEwZyyWDj+o0sWbSEK8+5kiWLluiBXqIet+SGgVoyGKiDshKfuXvK\nG33q0FOpb1RkAFqyaAlFVxZ1KBEdeu0Q9b+rZ9nqZRmMLDsV1RVRVFKU0Rj27t7Ldxd/lwP7DmBm\nLPjCAq66/qoOx9RX11NfXN9h2+zS2Rb0GupxS1YZiHO14xmog7K5LL8gnyX/ZwknzziZ2ppa/uHs\nf+Cjn/woHzjxAym7hhK3ZA2VBTobyIOyfeFvj/+N39zzG/bs2MP4yeO5+vNXc/b5ZyfV5uixoxk9\ndjQAQ4YNYeq0qby3572UJm4NTkrWGKhzteMZyIOy6fa3x//G8n9dzuDPDeb0Vacz+HODWf6vy/nb\n439L2TXKd5Tz5stvctrM01LWJqjHLVlEZYHOBuqgbF/4zT2/YeIXJzLi1BEA0T+/GN2ebK8boO5w\nHTdcdQNL/2kpQ0uGJt1ee0rckjVUFuia5nGnx54dezj9xNM7bBt+4nC27diWdNvNzc3ccNUNXHLF\nJXzqsk8l3d7RlLilT8UbfFywaEG0pr042tOu3lLN7pW7+dJ1X0p4bjbL1bj7u/GTx1P1ZlVbjxug\n6s0qxk8en1S77s4PvvIDjpt2HNd89Zpkw+xS/g9/+MOUN7qjYUfqG5Wc1zr4OPILI5l87WQi4yM8\nseIJxo4cS9lxZZQdV8bYkWN5+bcv886qdxi8czCLrl3ErE/NSnhutsrVuHNVqDlEaFAo0LGlJaU8\n9qvHKJxYyKBjBnHo9UPs+vUuvrz4y0z+wORex/Di0y9y69Jbqa+r54G7HuD+O+9nbNlYJn/wSJst\njS20hFo6nDd58OSbg15D87ilzyQzJzlX5zPnaty5qqfzuNMxqySIPpnHbWalwB3AqYAD/83dn+5B\nnCJJDT6mYuAyXskiXeUMDbhmt7PPP7tPEnWqBa1x/wz4s7svMLNCoDiNMUk/lczgY7IDl/HmiANp\nmz+uAVdJh4TzuM1sODAbuBPA3ZvcPTdehSxZJZk5ycnOZ443Rzyd88c1D7tvOU46yr+p5O44ycUY\npMc9FdgP3G1m04Hnga+7e237g8xsMbAY4Ju3fZN5n5+XVGDS/yQzJ3nWp2axZfMW1n5vLXU1dRQP\nK2b+wvmBe8QVOysYXzmeZ775DPXl9RRNKGLyvMns3bkXx9NWztA87L4VzgvjzQ4hMAtcMu4z7o43\nO+G8cFLtJBycNLOZwEbgY+7+jJn9DKh29+93d44GJyXV2pc6jp4qGCQJXjvnWvbbfiZ8ZQJDThhC\n7Vu1lP9rOaN9NKWjSzWA2F9EINQUIj+Sj5GFiZto0m4ubO5U70j14ORuYLe7PxP7eQ3w7aAXEEmF\n9uUMIPrn4uj2IInb8o1R80dRNLUIyzeKphYx6tOjsLWWcP645JA8aB7cTDPNmY4krRImbnffa2a7\nzGyau28BzgNeT39okinZuGCkYmcFxU8V89x3nqOltoWCIQVMnDOR+p31iU8muvz4+LOOp+r9Khqb\nGgkVhph41kTe/s3bKmdIzgk6q+SrwO9iM0q2AV9IX0iSSdn6hL5IU4TdT+5m4vcmMuTUIdS+Wsvu\n5bsp9dLEJxOd3RHeE6bslCOLXg69dqhtdoeWlUsuCfR0QHff7O4z3f00d7/c3Q+mOzDJjGx9Ql9N\nbQ0TvjaBodOHkhfKY+j0oUz42gRqamsCna/ZHdKf6Fkl0kG2Lhhpamhi+GnDibRECHuYPMtj+GnD\n2dmwM9D5iWalrFq2irW/77jvmiXpec6ESLKUuKWDbF0wUjysmNrXayk5o6RtW/UL1RQPC7YWbOP6\njTz59JN86JYPtQ1APrnySaatn8aWzVu4/8H7mfidiQz70DBqXqnh/tvuB1DylqykxC0dBJlhkczg\nZaKebXdtz184n/tvu5+JNxxJrrtu28UVC68IFFe8WSlvv/42E78zse0fhZIzSph4w0TW/nhtW2zZ\nOGArA5cSt3SQaIZFMoOXq5atituzjdd2awJd++O1bKvZRvGwYq5YeEWHxBovrngloLqaOoZ9aFiH\nfcM+NIxtNduS/p1F0kFPB5QeSeZpd5fPvJyx3xnbqdyx98d7eXDTg2l9emC8/W+//nba4hIJqicL\ncPTOSemRip0VlEwr6bCtZFoJe3fuTXhudz3bupq6QG2vWraKy2dezgXTLuDymZezatmqDnHlj89n\n9/bdvPvWu+zevpv88flt58abVTJ/4Xx23baL6heq8Wan+oVqdt22i/kL5yf9O4ukg0ol0iPJDF4W\nDyum5pWaDj3bmldq2gYY47WdqMxSPLSYXc/uovSjpRQPLibcEGbX07soGhp9NnO8ElDrvu7KMNk6\nYCsDl3rc0iPJzIdO1LON1/ba369l4g3RAUQL2ZEBxN+vBcDDTuUfKql/tx4PO/Xv1lP5h0o8fKRq\nN+tTs1i2ehm/ffK3LFu9rEN9+pol1/Dgpgf5y5a/8OCmBzsMmGoOuGQb9bilR5JZHp5ogDFe24kG\nEOvr6zn2Y8ey7fvbOi6J/3uwJfGJfudknkwokmpK3NJjySwPv2bJNXHnRnfXdqIyS1FREfte3Mdx\n/+u4jk//KxrdqzjbizcHXMlbMkGlEskJicoslm+M+nQXT//LT/7Rntn6GAAZuNTjlpRLx2KVRGWW\neE//SzaubH0MgAxcStySUulcrBKvzJLo6X/JxKVZJZJtVCqRlEpnWSHePO5EMz+SiUuzSiTbqMct\nKZWuskKiedyJZrskE5detCDZRolbUipdZYW1v1+b8EFQ8Wa7JBuXXrQg2USJW1JqwaIF3HrTrTQX\nN9NS3UJBSQGhuhBLf7QU6P1zr+tq6gjXhnnrW2/RVNFE4bhCRs8d3bZcHuIPPi5YtICfLf8Zwz4z\njFBZiObdzdQ8UMPXb/x6em6ESBopcUtKbdm8hUYaOfYfxjBoYiGNu5rYd/d7bNm8JannXofyQ1Tc\nW8GE/zGB4hOKqXurjvJ/KSeUHwKCDT5G6iNUrqmkqaqJwuGFhOpDabwTIumjxC0p1VrSGHxSMWaF\nDJ7UxKBjCln74+jS9ETlju6UHltKaH6IwVMGQz4MnjKYUfNH0Xxf9G3eid4Cv2b1Gk749gmdnvAX\n9C3xItkkUOI2s+1ADRAGWtx9ZjqDksTS+WD/ZF7jVVdTR/GJQ4iQBxiQT/GJQ9pKGvGWrScy7vRx\nVFVUEQ6Hyc/PZ9zp49h13y4g8eCj5mJLf9KTHve57l6ZtkgksHTOlU40eyORoqFFVL9SzbDpowAw\nK6D65UoKBxeBGVUv11ByervnXr9cQ2FRMc8+F7/dIaXjaKkoYMiYY6g5WM+QYUW0VBjFpWN59rno\n/n0vVVNyUrvBxzeqA++P56wPJ/y1+9yzv3sr0yFIis3+yuzAx6pUkoMSlQWSEWT2RjwfmHQCr//0\nNcpuCDHkpBJq36im/Kd7mDD6FM466RLW3bYMu2Eiw04poea1aspv2828s5cwrSH+f7QLz72Zu391\nI6G5eRRNncaBF7bQ/HCEL1y0nGkNs1l47s2sWrmE0LXFlBw/guqtB9l3RyXXXLAs0P7ubBn8VM9u\nYB9aMmlMpkOQDAmauB34i5k5sMLdVx59gJktBhYDfPO2bzLv8/NSF6V0kM6P/W1P4XNobmomVBjq\nUTkjEh5EZHcB27+/FQ87lm/k1RXjxw9i4RU3sWfPVp7+xz8Qbmkhv6CAj57yaRZecVPCds88Yw5r\nH/o5b/7073hkG5YX4sSyj3HmGXPa9gP88dc/4cVtf+IDx53BNRcu67R/3d238e6+TYw5dgrXXHBk\nv0guCZq4P+7u5WZ2LPC4mb3p7h26IrFkvhL06rJ06+2c5EQlAYDComKqXq5hyIeGEIlAc3OY2ldq\nA5UzzvowLPzapay4vZyyxdM7vGz4nE9dykOv/oStjf/J9Fs/zNDjSzi8tZqtv/5PHnr1J5w4M/4n\nhc1/3cA7tU8z8cazKJoykvrtB3jn7qf5t2d/yIzZn2Raw2zOPGMO2955g+2vGjM+eGmnpHzmGXN6\nlaiD3LfuxCuz5GKJRrJDoMTt7uWxP/eZ2VrgLCB7P0P2c0HexN6dRCWJuWd/g3XLlzH2a+MYcvII\nql8+yN6fVyQsZ7SWFLor47z8u+jS8hGzi9nxuy3U7Kxh2KRhjJ49mpefXsOi6+In7p//z3so+S+j\nqfzTGzTvqyV07BBK/stonnr4HmbM/iQA1dWVbNjwCCNG3M6GDddx4YXXUFIyskM71dWV3Hbbl7jh\nhpWd9nUl0f2KJ0iZpbv2s7lEI5mXMHGb2RAgz91rYt9fAPwo7ZFJt9K5BPv0Refw7Bt/YsdNb+CR\nnVheiMmTTuL0RedAQ/xzn30Otm+t4EMTZ1B3ZF0MBRNL2LvzXepr6olURRjz38Ywdup4mt5tYM9d\ne8jbndep93l0b3Pfrr0UFg1i3LXjKDr+WOq31lNxRwVNuxqBaKL7f2sfoD5yHsXF4wnXXMpjj63i\nM5+5sUM7jz22mrfeOtjlPpFcEaTHPQZYa2atx9/r7n9Oa1SSUDqWYG8Z/BQnHHeIxipj0piXKSgY\nSUvLARqrFlJbXcWWkqe67SG2bi8bfRKNW1ooPelIb3b/1l0Ul47l8OG3GbNoDIOOL8KskEHHG8cu\nOpby//VehzYAnn3uqQ7Ju3hEMWOvPfIm9sFnDCL/2jz2/ngvZ30YDh04xM//9hwjRv6exkYoKlrE\nhg1XdOh1B+mRi+SChE8HdPdt7j499nWKu9/SF4FJZjyy6nHCLXMpKIgmtIKCkYRb5lL+0l8CnT/v\nvBsov2Mnh944EH2S3hsH2Leyko9fuICC/ALyS/KJNIJHINII+SX55OcVdEja1dWV3PmPP6Dq/aq2\nbQX5BRSMKCBcHwaHcH2YghEFFOQXdIgboLL8KsAIh6O97laPPbaa5uY5VFbm09w8p8O+1uvefPOn\nqa4+0Kt7J9JX0jIdMJnBHMmsTU++RLjlCQ5V3nfU9mM44ezPJDy/u9kbQ2cO4a9/KqPurcMMDg/G\nvQ6zEA3vNDFqfFmHNh57bDU732zhl//7L5z32eg1jxk3BdsHzflhGpobyQ+FsH2DOWbcFJ59DjY8\n/BJ1dU/w/v4VNNaP4r33PsyQIcewefN4PvOZG9t6201Nv6ChIUxh4cVs2PDVDr3udJRRkvl/IdG5\nDz+r6YD9ydy5wY8199RPAHn4YTSrJAdtGfxUwlkQyQzW/fRni3lm578z6uppFE0eTf2O/VT+Zgsf\nmXQx3/h6dIZpdXUlS5deQX7+7YTD13HrrQ9QUjKS5194lFV/WcKEaye1zcMuv2Nnhyl93Z0L8MAD\ny3nssTAHD87DbDLuOxgxYh0XXpjflti7O1ekL8ydS+D37OlFCtJn9u/bS8sup3z5c2z9+sOUL3+O\nll3O/n17247prpxx5hlzuOaCZVT/uoG/LvgT1b9u6DQP+7HHVhMOX0ph4bROZZLNm5+ktvbXhMNz\niUQ+Tjg8l9raX7N585MJzxXJNkrc0meWLr2LUSOnMZSn8eoXGMrTjBo5jaVL7wZoV864mIaGME1N\nF7NhwyNtNeczz5jDjA/OJb/uTGZ8cG6HpN16blHRIqB1cPLIuUuX3kVpaRmTJ/8nU6ZsYvLk/6S0\ntIylS+9OeK5ItlHilj7T2puurS2goGAytbUFHXrVifZ3nBXSMbG29pjz86PPSMnPH9Wh5xxvf6Jz\nRbKNEncOS9csiI3rN7Jk0RKuPOdKlixawsb1G1PS7pFyxaU0N0dLEu3LFa37W1oupqlpMi0tF3cq\nZzQ2XsDu3Y00Nl7QqRQSDt9LVdXMtq9w+N4ObYfD93Lw4Ax27BjDwYMz2vYnOreVZp1IttDgZA57\n4IHlPPjgI1x++aUpmQWxZfBTRKqOPHnw6FWZecNnJTU42ToAWFd3LlVVTzB8+LkUFz/RNhAYbz84\nS5dewYED/0RDw1AGDz7MyJHf7vEgYjL3LNX3W6Q9DU4OAPHKBu2P6WkPMejb0HvTdrQUcg6HDz9B\nfv7tHD78BM3N5x5VKul6f2tvu6GhCJhAQ0NRp153oriC3LPuJHOuSKopceeoILMg2s9LDmr71gpK\nppV02FYyrYTtW/d22NabtjdvfpLDh+8hHJ6Oe5hweDqHD9/doZzR3f7Nm5+kqup24HJgNnA5VVW3\ndypnxIsrmZkjmnUi2USJOwcFmQXRmx7itIbZHDMm+sKBujravva9VE3Z6BPbyiS97X0uXXoXI0ZM\nYfLk7zFlyslMnvw9RoyY0jarJN7+66//v0QiIeC3mD0D/JZIJMT11/9zoN85mZkjmnUi2UaJOwcF\nmQXR2x7iwnNvZt/KSupfayLUNIT615rYt7KSeefdkHTbQWd+hMPNbNt2EeFwuG3/ihXfJhKZD0wF\nBgNTiUTms2LFtwLFlczMEc06kWyjN+DkoOgsiD1UVd171PaOy7uLiqKvHOvqgUvdSfTCgWTaThR3\n6/6Kin8mEhlNRcUZlJaOY/Pm8bzzzmbcNwJ/xj0PiOBeyVtvhQLFlejaycQt0tc0q6QfeuCB5Tz+\nOAwdeiSpHD68nPPPJ+lEk862AcrLt/DVr16C2R9x/6/84hePMmHC8Qmvm+64RNJNs0oGuKDzktPZ\ndm/nPEdLIgsJh4uJRBa2lUJar3vgwGm8884IDhw4rct52un4nUWyjXrckha9mfPc2tuORB4kEgmR\nl9dMXt7lbb1ugJtums/mzTuZMWMSP/rR2nT+CiJ9Sj1uyajezjpp7W27h2JP8At16HWXl2/h1Vdf\nIRS6m1dffYXy8q3p/DVEspYSt6Rcb19YsHXri7jfifvFuH8k9uedbN36ItB9GUVkoFHilpRK9IQ/\n6H6RzIoVmxg//iSmTn2GD37wJaZOfYbx409ixYrn23rbsJBIJAwsVK9bBqzAidvM8s3sRTN7JJ0B\nSXCJBgCTeShSb9sO8oS/9esforn526xf/1DgJ/wlKqOIDCQ96XF/HXgjXYFIzyVadt6bZenJth3k\nhQU1NZ+gqWkyNTWfCPyEv0RlFJGBJNACHDMrAy4BbgE0KTYLJHpjeTJvNE+m7aVL72Lp0is49tj7\nYz3mSpqarmh7YcH69Q/R0HArBQWTaWi4mPXrl7adf8st6+LGtHTpFRQWdmw3+uRAkYEl6MrJnwLf\nAoZ1d4CZLQYWA1x//Qouumhx8tFJt1rLCkVF0zh8+NJOL7hNtD9R283Nczh4MJ+hQ+f0qO32y9Z3\n7LiIsrJVHZaH19R8ApiC2WBgSluvO1Fs8cooA3KBzbPPZjoCSbW5ZwU+NGHiNrNLgX3u/ryZndPd\nce6+ElgJmsedbomWdyezLD3R29CDLi3vatl6ONxMff2rwCOEw3m4R6iv38/zz5+qZee9MPes9zId\ngmRIkB73x4B5ZnYx0af7lJjZb939qvSGJt1J1PtMpnfaeXBxB4WFcwK3fcst69oW0oRCq3H/r/zo\nR/e1LVt///3zOy1LP/PMxL9zvDKKyECTcHDS3b/j7mXuPgVYCGxQ0s6soK/p6s2y9ESDi0Hajs4A\n+Rx5eacRiXyu07J1LUsXSU6PlrzHSiVL3P3SeMepVJIbulqWHm8QMMjg5pGHRP2NvLxxRCIVuJ/d\nYdm6pMCzz6pU0t/MnZueJe/u/mSipC25obtl6ck+e/pIb3scQCx5f07zrUVSSCsnB6jW9ztWVn63\nw3sfky1nROdV30lz84S2L9B8a5FU0osUBqDW3nZz87k0NBxk0CDYsOERLrzwmqQHAe+7b3tKYhSR\n7qnHPQAlett6piWzVF9kIFDiHoASvW0905JZqi8yEChxD0CJ3raeSb19lrfIQKLEnWGZKAuk4q3l\n6Yq7t28XaDuSAAAMI0lEQVSQFxlINDiZYe3LAn21dDsVy8fTEXcyS/Vzkp43Ir2kxJ1ByTzBLxnJ\nzhxJV9wD8UFSWkQjvaFSSQblalkgXXFrSbxIMOpxZ0iulgXSGbceJCUSjHrcGZKKAcJMyNW4RfoT\nJe4M6YuyQDreSalyhkjmqVSSIX1RFkg086M3M0NUzmhHs0IkQ5S4+6l0vpNSjtCsEMkElUr6qUQz\nP3J1RouIKHH3S0dmfiwCWmd+HFk+nmi/iGQ3lUr6oXS+k3LAUR1bspASdz+UaEm73pjeM6pjS7bp\n0Tsng9I7J6Xf0Lsdpa+k8p2TZjbYzJ41s5fM7DUzuzm56CQX6GUGItkryOBkI/BJd58OzAAuMrNZ\n6Q1LMk0vMxDJXgkTt0cdjv0Yin2pFNKP6WUGItkt0HRAM8s3s83APuBxd3+mi2MWm9kmM9v05z+v\nTHWc0oc0x1skuwWaVeLuYWCGmZUCa83sVHd/9ahjVgIrQYOTuSxXn1rYa5ruJzmoR9MB3f2QmT0B\nXAS8muh4yT0DcY63Zo1Irgkyq2R0rKeNmRUB5wNvpjswyQw9/U8k+wXpcY8DVplZPtFEf7+7P5Le\nsCRT9PQ/keyXMHG7+8vA6X0Qi4iIBKCHTImI5BglbhGRHKPELSKSY5S4RURyjB7rKtlPi2REOlDi\nlpygRTIiR6hUIiKSY9TjluygcohIYErcknl6y4xIj6Qncav3JCKSNmlJ3Oo9iYikjwYnRURyjBK3\niEiOUeIWEckxStwiIjlGiVtEJMcocYuI5BglbhGRHBPkZcETzewJM3vdzF4zs6/3RWAiItK1IAtw\nWoBvuvsLZjYMeN7MHnf319Mcm4iIdCFhj9vdK9z9hdj3NcAbwIR0ByYiIl3r0ZJ3M5tC9I3vz6Qj\nGJFs9Mkbb6S6qqrT9pLhw9mwfHm/vbZkr8CJ28yGAn8AvuHu1V3sXwwsBlhx/fUsvuiilAUpkknV\nVVVsGj680/aZXSTU/nRtyV6BEreZhYgm7d+5+x+7OsbdVwIrAXj4YU9VgCIi0lGQWSUG3Am84e76\nbCYikmFB5nF/DLga+KSZbY59XZzmuEREpBsJSyXu/h+A9UEsIiISgF5dJgNeopkbFdXVTDhwoPOJ\noVDaY4t3bc04GbiUuGXASzRzY1xJScZmdsS7tmacDFx6VomISI5Rj1tyQqKyQDL7E3l3/36m79/f\nafvugLElY/fBg8w8eLDLa5eNGJFU25K7lLglJyQqCyS7P54Q8JJ1Hp8f655024nkRyJs6qKWPqG5\nOem2JXcpcUu/8M7+/Yzvoldcn4K2G4EJ3nlNWWPsz/cOHuT1LnrF7wVoe+RllxHqou1mMw489BDh\nSITXGxs77Q8HaDuRdH1S0KBp+ilxS78wCNjTxfaxAc4tGT68y95xaxllELCri/PGx/6MRCKc3EWv\nOBKgVxxyZ2+c3jzA1G7OTRR3Iun6pKBB0/RT4pYBL0gvMFOj+Pl5eRR18Y9CfnOzeq8DmBK3ZI14\nH7G37d/PaXEGCBOJN8gX5KN9vJJLOBKhrptyRq6WDXI17oFCiVuyRryP2CHg912cc2677+Ml1xCw\nuovtFya4bqvCOG03AbO72Z5s2SCcl8fMcOeKdjgvvZ8BVO7IbkrckhMMOLmLWrC1qwUXxTl/zIgR\nnNxFIhoTMBHlx9lXCGxKUKfurbIRI5RApRMl7n6oP37MdeD1LhJh65ZGuh6IbC1g7N6/n5e6KbWU\njR4d99qNdP3Kp8Z2f3aVpDsXTzprNuvy3ObYPwTxBiCT/XuO13ZX7QaV7KCpJKbE3Q/1x4+5Bpzc\nzXaAg+vWJWxjehe9YgL0iovz8iiPM5d6ELC3i/OCzGg58NBDcffHS8Az//t/T+rvOVHbvZWrnYNc\noiXvIiI5Rj1uyRrxPmI37d/PaV2c0xSw7SZgehe966YE1wUgFOp6pWK7XnhdN9fNZNkgmVKKyh3Z\nTYlbskY6ywKFdL9sPVESK3/ggbj78/PyKM7CudbJlMxU7shuStySE5JZVi7S3yhx90P98WNuM7Co\nm+1Bz++qVJKSRzUFKKWkQ3/8e5ZglLj7of74MTfZ+cxTR49O20ybRKWUdOmPf88STMLEbWZ3AZcC\n+9z91PSHJNJ/9Mc59ZJ5QXrc9wD/QtcrhkVyQqbKCpmcU69SSv8V5C3vT5nZlPSHIpI+A7F3OxB/\n54EiZTVuM1sMLAZYcf31LL7oolQ1LaLeo0g7KUvc7r4SWAnAww8n/3QdkXbUexQ5QkveRURyjKYD\niqSRSjySDkGmA94HnAOMMrPdwA/c/c50BybSH6jEI+kQZFbJZ/siEBERCUY1bhGRHKPELSKSY5S4\nRURyjBK3iEiOUeIWEckxStwiIjlGiVtEJMcocYuI5BglbhGRHKPELSKSY5S4RURyjBK3iEiOUeIW\nEckxStwiIjlGiVtEJMcocYuI5BglbhGRHKPELSKSY5S4RURyTKDEbWYXmdkWM3vbzL6d7qBERKR7\nCRO3meUD/wrMAU4GPmtmJ6c7MBER6VqQHvdZwNvuvs3dm4DfA5elNywREelOQYBjJgC72v28G/jI\n0QeZ2WJgcezH37r71cmHl1pmttjdV2Y6jqMprp7J1rgge2NTXD2TrXG1StngpLuvdPeZ7j4TOClV\n7abY4sSHZITi6plsjQuyNzbF1TPZGhcQLHGXAxPb/VwW2yYiIhkQJHE/BxxvZlPNrBBYCKxLb1gi\nItKdhDVud28xs/8BPAbkA3e5+2sJTsvW2pDi6hnF1XPZGpvi6plsjQsAc/dMxyAiIj2glZMiIjlG\niVtEJMf0OnGb2V1mts/MXu1mv5nZz2PL5F82szN6H2ZK4zrHzKrMbHPs66Y+imuimT1hZq+b2Wtm\n9vUujunzexYwrj6/Z2Y22MyeNbOXYnHd3MUxg8zs32L36xkzm5IlcX3ezPa3u1/XpjuudtfON7MX\nzeyRLvb1+f3qQWwZuWdmtt3MXoldc1MX+zOSxxJy9159AbOBM4BXu9l/MfAoYMAs4JneXivFcZ0D\nPNIXsRx13XHAGbHvhwFvASdn+p4FjKvP71nsHgyNfR8CngFmHXXM9cCvYt8vBP4tS+L6PPAvff3f\nWOzaNwL3dvX3lYn71YPYMnLPgO3AqDj7M5LHEn31usft7k8B78c55DJgtUdtBErNbFxvr5fCuDLC\n3Svc/YXY9zXAG0RXpbbX5/csYFx9LnYPDsd+DMW+jh5JvwxYFft+DXCemVkWxJURZlYGXALc0c0h\nfX6/ehBbtspIHksknTXurpbKZzwhxHw09lH3UTM7pa8vHvuIejrR3lp7Gb1nceKCDNyz2EfrzcA+\n4HF37/Z+uXsLUAWMzIK4AD4d+2i9xswmdrE/HX4KfAuIdLM/I/crJlFskJl75sBfzOx5iz6242hZ\nmccG4uDkC8Bkd58O/AJ4sC8vbmZDgT8A33D36r68djwJ4srIPXP3sLvPILpa9ywzO7UvrptIgLge\nBqa4+2nA4xzp5aaNmV0K7HP359N9rZ4KGFuf37OYj7v7GUSffvoVM5vdR9dNSjoTd1YulXf36taP\nuu7+70DIzEb1xbXNLEQ0Of7O3f/YxSEZuWeJ4srkPYtd8xDwBHDRUbva7peZFQDDgQOZjsvdD7h7\nY+zHO4Az+yCcjwHzzGw70Sd4ftLMfnvUMZm6Xwljy9A9w93LY3/uA9YSfRpqe1mZx9KZuNcBi2Kj\nsrOAKnevSOP1AjGzsa11PTM7i+g9SPt/vLFr3gm84e7Luzmsz+9ZkLgycc/MbLSZlca+LwLOB948\n6rB1wDWx7xcAGzw2opTJuI6qgc4jOm6QVu7+HXcvc/cpRAceN7j7VUcd1uf3K2hsmbhnZjbEzIa1\nfg9cABw9Gy0r81iQx7p2yczuIzrbYJSZ7QZ+QHSgBnf/FfDvREdk3wbqgC8kG2yK4loAXGdmLUA9\nsLAv/uMl2uu4GnglVh8F+C4wqV1smbhnQeLKxD0bB6yy6Is88oD73f0RM/sRsMnd1xH9B+c3ZvY2\n0QHphWmOKWhcXzOzeUBLLK7P90FcXcqC+xU0tkzcszHA2lifpAC4193/bGZfhszmsUS05F1EJMcM\nxMFJEZGcpsQtIpJjlLhFRHKMEreISI5R4hYRyTFK3CIiOUaJW0Qkx/x/Ps0r5XXX3W0AAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f14442596d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plot_decision_regions(X=X, y=y, clf=rf)\n",
    "plt.title(\"Random Forest\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacking\n",
    "\n",
    "**(Markos)**\n",
    "\n",
    "Stacking is an ensemble learning technique that combines multiple classification or regression models via a meta-classifier or a meta-regressor. The base level models are trained based on a complete training set, then the meta-model is trained on the outputs of the base level model as features.\n",
    "\n",
    "The base level often consists of different learning algorithms and therefore stacking ensembles are often heterogeneous. The algorithm below summarizes stacking."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](algo_stacking.png )\n",
    "<p style=\"text-align: center;\">Figure 5. Stacking algorithm.</p>\n",
    "\n",
    "The following example consists of k-NN, Random Forest, and Naive Bayes base classifiers whose predictions are combined by Logistic Regression as a meta-classifier. We can see the blending of decision boundaries achieved by the stacking classifier. The figure also shows that stacking achieves higher accuracy than individual classifiers and based on learning curves, it shows no signs of overfitting.\n",
    "\n",
    "Finally it is important to mention that stacking is a commonly used technique for winning the Kaggle data science competition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3-fold cross validation:\n",
      "\n",
      "Accuracy: 0.91 (+/- 0.01) [KNN]\n",
      "Accuracy: 0.91 (+/- 0.06) [Random Forest]\n",
      "Accuracy: 0.92 (+/- 0.03) [Naive Bayes]\n",
      "Accuracy: 0.95 (+/- 0.03) [StackingClassifier]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import model_selection\n",
    "from sklearn import tree\n",
    "from sklearn import metrics\n",
    "from sklearn import ensemble\n",
    "from sklearn.ensemble import BaggingClassifier\n",
    "from sklearn.model_selection import cross_val_score\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import model_selection\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.naive_bayes import GaussianNB \n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from mlxtend.classifier import StackingClassifier\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data[:, 1:3], iris.target\n",
    "\n",
    "clf1 = KNeighborsClassifier(n_neighbors=1)\n",
    "clf2 = RandomForestClassifier(random_state=1)\n",
    "clf3 = GaussianNB()\n",
    "lr = LogisticRegression()\n",
    "sclf = StackingClassifier(classifiers=[clf1, clf2, clf3], \n",
    "                          meta_classifier=lr)\n",
    "\n",
    "print('3-fold cross validation:\\n')\n",
    "\n",
    "for clf, label in zip([clf1, clf2, clf3, sclf], \n",
    "                      ['KNN', \n",
    "                       'Random Forest', \n",
    "                       'Naive Bayes',\n",
    "                       'StackingClassifier']):\n",
    "\n",
    "    scores = model_selection.cross_val_score(clf, X, y, \n",
    "                                              cv=3, scoring='accuracy')\n",
    "    print(\"Accuracy: %0.2f (+/- %0.2f) [%s]\" \n",
    "          % (scores.mean(), scores.std(), label))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvQAAAIYCAYAAAACWnjvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmYVNWd//H3txekG2hAQHbBRG1j3INoNqLRRFFxGYlD\ngqKZMSSaZWLChEkyMTETJ5P5GY3ZDLhrXKIkGDQxLoOELBI0iqJRXHBh34RuaBq6qT6/P+pWU91U\n1b3dVbfq3qrP63n6sbvuUt8u9X5On3PuueacQ0RERERE4qmq1AWIiIiIiEjvqUEvIiIiIhJjatCL\niIiIiMSYGvQiIiIiIjGmBr2IiIiISIypQS8iIiIiEmNq0IuIiEismdlJZra61HWIlIoa9BJLZvam\nmZ2a9vM0M9tqZh8xM2dmv++2/y/N7Dve9yd5+/y82z5/NrNLilG/iEi5867TrWa2w8zWm9ltZta/\n1HXly8uPFu/32mFm24r8/vrjRfahBr3EnpldDPwMOBN4y3v5BDP7QI7DWoCLzGx8uNWJiFS0Kc65\n/sAxwLHA10tcT6Ec7Zzr730N6unBZlYTRlFSudSgl1gzs88CPwROc879NW3T/wJX5zh0G3Ab8O3w\nqhMREQDn3HrgEZINewDM7Ewze9bMms1sVWoU1ds23usJv9jM3jazzWb2zbTtdV6P/1Yz+wdwfPr7\nmdl7zGyRmW0zsxfN7Oy0bbeZ2c/N7GGvh/0vZjbCzH7kne9lMzu2N7+nmX3GzF4zs3fMbIGZjUrb\n5szs82b2KvCq99phZvaYt/8KM7sgbf8zzOwfZrbdzNaY2Swz6wc8DIxKGyEYtU8hUnHUoJc4uwz4\nLnCKc+7pbtt+DhyaPi0ng6uB882sMawCRUQEzGwMMBl4Le3lFmAGMIjkCOtlZnZut0M/BDQCpwBX\nmtl7vNe/Dbzb+zoNuDjtvWqBB4FHgQOALwJ3dbvWXwD8JzAU2A08CTzj/TwPuLYXv+NHge975x5J\ncsT43m67nQucABzuNc4fA+726pwG/NzMDvf2vRn4rHNuAHAEsNA510Lyc1ybNkKwtqe1SvlRg17i\n7GPAEmB5hm2tJBvs38t2sNdj9AuSfxSIiEjhPWBm24FVwEbSRkWdc4ucc8udcx3OueeBe4CPdDv+\nKudcq3PuOeA54Gjv9QuAq51z7zjnVgE/TjvmRKA/8D/OuTbn3ELgIeCTafvMd8793Tm3C5gP7HLO\n3eGcSwC/Ijk9KJdnvN7/bWaWeu/pwC3OuWecc7tJTi96f7epnd/3am4FzgLedM7d6pzb45x7Fvg1\n8Alv33aSDf8G59xW59wzPjVJBVODXuLsMuBQ4CYzswzbbwKGm9mUHOf4AXCamR2dYx8REemdc70e\n5pOAw0j2gANgZieY2RNmtsnMmoDPpW/3rE/7fifJhjrAKJJ/JKS8lfb9KGCVc66j2/bRaT9vSPu+\nNcPPfjfvHuecG+R9fSntfTvrcM7tALZ0e9/0mseRvN8r9YfBNpJ/FIzwtp8PnAG8ZWZ/NLP3+9Qk\nFUwNeomzDSSHYT9McopNF865NuAq4L+ATA1+nHNbgB95+4iISAicc38ked/SNWkv3w0sAMY65waS\nHDHNeK3OYB0wNu3nA9O+XwuMNbOqbtvX9LDsnlpLspEOgDelZki393Vp368C/pj2h8EgbwrNZQDO\nuaecc+eQnI7zAHBfhnOIAGrQS8x5cwdPAU43s+sy7HIn0Bc4PcdprgU+ALwnxz4iIpKfHwEfSxsR\nHQC845zbZWYTgU/14Fz3AV83s8He/Pwvpm37G8ne/K+ZWa2ZnQRMYd/57IV2D/BpMzvGzPYD/hv4\nm3PuzSz7P0TyXq+LvDprzex474bePmY23cwGOufagWYgNeKwARhiZgND/n0kRtSgl9hzzr0NfBSY\nSvKGpPRtCeBKYP8cxzeTXBUn6z4iIpIf59wm4A6S12SAy4HvenPsr2RvD3QQV5Gc3vIGyZtf70x7\nnzaSDfjJwGaSI7gznHMv5/s75OKcexz4Fsl58OtI3rA7Lcf+24GPe/usJTm96AfAft4uFwFvmlkz\nyelI073jXib5x8NKb6qOVrkRzDmN3IiIiIiIxJV66EVEREREYkwNehERERGRGFODXkREREQkxtSg\nFxERERGJMTXoRURERERirCaMky7etlhL54iIZDFp0KSgD88pa8oKEZHcguaFeuhFRERERGJMDXoR\nERERkRhTg15EREREJMbUoBcRERERibFQbooVEQldB9S21VLdUY0RvXtMHY5EVYL2Pu3qOhERKaUK\nyAs16EUklmrbaunXpx9Wa5hF8ALtHK7d0dLWQnvf9lKXIyJSsSohL9RvJCKxVN1RHdmLM4CZYbVG\ndUd1qUsREalolZAXatCLSCwZ0b04p5hZJId3RUQqSSXkhRr0IiJ5+PNjf2bKsVM44+gzuOmHN5W6\nHBERiaCws0INehGRXkokElz91av5+W9+zm+f+i0Pz3uY119+vdRliYhIhBQjK3RTrIiUvc9MmsbO\nze/s83r90P25cfG9vT7v8qeXc+C7DmTsQWMBmHz+ZJ546Anefdi7e31OEREpnTDyohhZoQa9iJS9\nnZvfYcnQwfu8fmKGi3ZPbFy3kRGjR3T+PHz0cJ5/+vm8zikiIqUTRl4UIys05UZEREREJMbUoBcR\n6aUDRh7A+jXrO3/esGYDw0cOL2FFIiISNcXICjXoRUR66Yj3HcFbr7/F6jdX097WzsO/fpiTzjyp\n1GWJiEiEFCMrNIdeRKSXampq+MY13+Bz536OREeC8y46j4Pfc3CpyxIRkQgpRlaoQS8iZa9+6P4Z\nb2iqH7p/3ueedNokJp02Ke/ziIhI6YWVF2FnhRr0IlL28lmaUkREKkdc80Jz6EVEREREYkwNehER\nERGRGPNt0JtZo5ktS/tqNrMvF6M4ERGJD+WFiEhp+M6hd86tAI4BMLNqYA0wP+S6REQkZpQXIiKl\n0dMpN6cArzvn3gqjGBERKRvKCxGRIulpg34acE+mDWY208yeNrOnF9y2IP/KJHKWPL6EWTNmMf2k\n6cyaMYsljy8pdUkiJfWty77FRw76COdNPK/UpURRxrxQVlQG5YVIV2HnReAGvZn1Ac4G7s+03Tk3\n1zk3wTk34exLzi5UfRIRSx5fwpwb5lA3vY5jbjuGuul1zLlhji7SUtHOmX4ON8y/odRlRE6uvFBW\nlD/lhci+ws6LnvTQTwaecc5tCKsYia55d8xjzMwxDHrvIKpqqhj03kGMmTmGeXfMK3VpIoFt3byV\ny8//Ctu2bCvI+SZ8aAIDBw8syLnKjPKigikvpBzELS960qD/JFmm20j5W/f2OhoaG7q81tDYwPq3\n15eoIpGem3fbgyx/ehf336qpHiFTXlQw5YWUg7jlRaAGvZn1Az4G/CbcciSqRh44kuYVzV1ea17R\nzIgDR5SoIpGe2bp5Kwvu/gtDhv+EBXf/pWC9LtKV8kKUFxJ3ccyLQA1651yLc26Ic64p7IIkmqbO\nmMrquavZ9uI2OvZ0sO3Fbayeu5qpM6aWujSRQObd9iB79kxhv76HsmfPlNj0usSN8kKUFxJ3ccwL\n33XoRQBOPPVEIDk38o2332DEgSP47GWf7Xzdz5LHlzDvjnmse3sdIw8cydQZUwMfW2n0WRVeqrel\nvv+9ANT3n86Cu6fxiU+fzaAhg0pcnUh5UV4Ujz6rwotrXqhBL4GdeOqJvbpQpFY8GDNzDMc0HkPz\nimbm3DCn85yylz6rcKR6W+prhgJQUzOUnV6vy2dmzej1eb/26a/x1J+eYtuWbZzSeAqf/8bn+aeL\n/6lQZYvElvIifPqswhHXvDDnXMFOlrJ42+LCn1Ria9aMWdRNr2PQe/f+ZbvtxW203tXKNXdcU8LK\nokefVXB1O+uoa6gLtO+/nvklVr2xZZ/Xxx40hJt/9+NCl9ZFa3MrrfWtXV6bNGiShfqmMaGskO50\nDQxOn1VwlZAX6qGX0K17ex3HNB7T5bWGxgbeePuNQMdX0pBivp+VZBb2RVhECkN5EZzyIhxxzYue\nPilWpMfyWfGg0h5QotUhRKSSKS+CU15IOjXoJXT5rHhQaQ8o0eoQIlLJlBfBKS8knabcSOjyWfGg\n0oYU810dQkQkzpQXwSkvJJ0a9FIUvV3xIDWkmH7TT7kPKfb2sxIRKQfKi+CUF5KiKTcSaZU4pLjk\n8SXMmjGL6SdNZ9aMWWU7/1NEpJCUF8qLSqYeeom0ShtS1LrC8bJ+9Xq+MfMbbNm4BTNj6qencuHl\nF5a6LJGKpLxQXkRVMbJC69CLRIjWFQ6uJ+sKh2XT+k1sWr+Jw485nJbtLfzzh/+Z6++9nncf9u7O\nfbQOfXbKCpHeU14EV+q8CJIVoHXopQxU0trBuVTaTV3F9KfH/sSdt93J2rfWMmrcKC665CI+/LEP\n53XOYSOGMWzEMAD6DejHQY0HsWHthn0u0iJSGMqKvZQX4Sl0XhQjK9Sgl5LTsOFelXhTVzH86bE/\nce3PrmXsZ8Zy7GHH0vRyE9f+7FqAvBv1KWveWsPLz7/MUROOKsj5RKQrZUVXyotwhJ0XYWWFboqV\nkqu0tYNzqcSbuorhztvuZOxnxjL4iMFU1VQx+IjBjP3MWO687c6CnH/njp1cceEVzP6f2fRv6F+Q\nc4pIV8qKrpQX4QgzL8LMCvXQS8lp2HCvSrupq1jWvrWWYw87tstrAw8byMq3VuZ97vb2dq648ArO\nvOBMTj3n1LzPJyKZKSu6Ul6EI6y8CDsr1KCXktOwYVdaV7jwRo0bRdPLTQw+YnDna00vNzFq3Ki8\nzuuc49uf/zbvanwXF3/x4nzLFJEclBX7Ul4UXhh5UYys0JQbKTkNG0rYLrrkIlbduIqtL2ylY08H\nW1/YyqobV3HRJRfldd5nn3yWB+95kL/98W9M/cBUpn5gKosfWVygqkUknbJCiiGMvChGVqiHXooi\n18oEfsOGcVzVII41l7PUjUx33nYnK99ayahxo/jK57+S9w1Ox33gOJZvX16IEkXEk+36GWSKSRyv\nvXGsuZyFkRfFyAqtQy+hS1+ZoKGxgeYVzayeuzrQXL98ji2VONYcR6VeVzgorUOfnbJCulNeRL/m\nOKqEvAg05cbMBpnZPDN72cxeMrP396JOqVD5rEwQx1UN4lizSKEoLyQfyovo1yzRFHTKzfXAH5xz\nU82sD1AfYk1SZvJZmSDfVQ38hjLDGOrUSgxS4ZQX0mtRzYuwpsUoL6RQfBv0ZjYQmARcAuCcawPa\nwi1Lykk+KxPkc6zfQ0jCekiJVmIoDofDOYdZdGevOOdwVM6sEuWF5CuKeQGE9kAr5UVxVEJeBJly\ncxCwCbjVzJ41s5vMrF+v31EqTj4rE0ydMZXXrn+NFY+tYOVLK1nx2Apeu/61QMfOu2Me9SfUs+KW\nFSy+cDErbllB/Qn1nUOZYQ11aiWG4khUJXDtyYt0FDnncO2ORFWi1KUUk/JC8hLFvAhzWozyojgq\nIS+CTLmpAY4Dvuic+5uZXQ/8B/Ct9J3MbCYwE+Cr132Vsy85u9dFSXnJ9+EXHa0dbJ63mbamNvoM\n7ENta22g49548Q06mjoYOXMk/Q7tR8srLaydu5aq1cm/Y8Ma6tTDPoqjvU87LW0tVO+qxoher4sj\neXFu79Ne6lKKyTcvlBWSSxTzou+AvqFNi1FeFEcl5IXvKjdmNgJY4pwb7/38YeA/nHNnZjtGKxdI\nocyaMYu66XVdhiO3vbiN1rtaueaOa3Iee+6Ecxnx9RE0HNfQ+VrzM82s//56Hnj6gbzOLZKPcl3l\npqd5oayQQgorLw4+/GBlhZRM0Lzw7aF3zq03s1Vm1uicWwGcAvwj3wIleqK4Fu66t9dRv7iep77+\nFHta9lDTr4axk8fS+nar77E11TXUDK4h0Zqgum81iV0JagbXUFOd/M9+6oypyXmQM9lnuTAR6Tnl\nReWopLxQVkgcBF3l5ovAXd6KBSuBT4dXkpRCWDeI5qujrYPVi1Yz9ptj6XdEP1peaGH1tasZ5Ab5\nHju+cTxsg0SfBLvbdlPbp5a+2/omX0dDnSIhUV6UuUrLC2WFxEGgBr1zbhkwIeRapITSb/oBkv+c\nmXy9lBet7S3bGf2N0fQ/uj9WZfQ/uj+jvzSa9f+93vfYVK9Klwd23NW1VyX96YMikj/lRfmrxLxQ\nVkjUBe2hlzIX1bVw23a1MfCogXTs6SDhElRZFQOPGsjbu972PfbEU09kxbIVzP/mfHZu30n9gHrO\nm3Zel4vy7dfczvx7u26/eNbFYf5KIiKxVol5oayQqFODXoDoroVbP6Celn+07HOjUv0A/2fVLHl8\nCYueXMSRVx/Z2eOyaO4iGh9v5MRTT+T2a27nvgfuY+zXxzLgyAFsX76d+667D0AXahGRLCotL1Ys\nW6GskMgLsg69VICoroV73rTzWHXdKpqfaca1O5qfaWbVdas4b9p5vsf6rR08/975jL1iLA3HNWC1\nRsNxDYy9Yizz750f9q8lIhJblZYXygqJA/XQCxDsBtF8VjXwG67Mdu7UPvO/P5+V21dSP6CeC6Zd\nEOhYv2Hhndt3MuDIAV22DzhyACu3r8z79xURKVelzItc5/XLi1zH5soLv6zI9/cVKQQ16KVTrpt+\n8lnVwG9qi9+5L551cdZhzVzH+g0L1w+oZ/vy7V2GZ7cv3079gPrIruIgIhIFpciLxmMafc+bLS/8\nasqVFzt27MiaFfn+viKFoik3Ekg+j772G67M59y5jvUbFs41PBvmo75FRMpZWHkRVlZA7mlEflN5\nlBcSBeqhl0DyWdXAb7jS79y5hl/Xvb2OQ0Ydwuo3V9Pe1k5tn1oGjhrI+rfX+w4L5xqenX7S9Eiu\n4iAiEnVh5UWQ82bLi1xZAbmnEaW2ZZvKE9VVf6SyqEEvgeSzqkGuqS1+5/abrlPfv55VS1cx6P2D\nqO9bT2JXglVPrqKufx3gv3ZwtuHZqK7iICISdWHlhd95c+WFX1ZA7rzINfVTeSFRoCk3Ekg+qxr4\nDVfmOrffdB2XcGz+9WZa32jFJRytb7Sy+debcQlXst9XRKSShZUXfufNlRdhZUW+v69IoaiHXgLJ\n59HXfisP5Dq333Sd1tZWDvjgAaz81kr2tOyhpl8NYyePpfUvrXn/vn4PpRIRkX2FmRe5zpsrL1r7\nh5MVqd9XeSGlZs7l/9dpd4u3LS78SaUinTvhXEZ8fcQ+DwpZ//31PPD0A1w6+VI22SZGf340/Q7t\nR8srLaz52RqGuWHc9PBNvX7f9FULOh8DPnd14FASyWXSoElW6hqiQFkhhZQrL4YOGxpKVoDyQsIV\nNC805UYKZsnjS5g1YxbTT5rOrBmzWPL4krzP6Tddx6qNoecPpe6gOqzaqDuojqHnD8WqLa+atGqB\niEg4wsgKyJ0XflmRT13KC4kCTbmRgghrHV6/4dedO3ZyyMRDaHqnid1tu6ntU8vYiWN57c7X8qpJ\nqxaIiBRemGu258qLRx96NGtW5FuX8kKiQA16KYj0Hgog+c+ZydcLcZHOtbpAYm2CMe8d0/nathe3\nMeLAEXnVpFULREQKL8ysgNwrl2XLinzrUl5IFGjKjRTEurfX0dDY0OW1hsaGzjV+83H7Nbdz7oRz\n+Xjjxzl3wrncfs3tndtyrS6QT01atUBEpPDCzArInhd+13TlhcSdeuilIMLqofBbhz7Xagrz7pjX\n65ryWaVBREQyC7M32y8vIPs1PZ+6lBcSBVrlRgoirLv8/Va5KUVNIvnSKjdJyorKE+Z1WXkh5Sho\nXqiHXgrixFNP5P9+/X/88ct/JNGeoLq2mo+c9JHOC2G2x3H72bl9J4mWBK987RXa1rXRZ2Qfhk0Z\nxs7tOzv3WfL4EubdMY91b69j5IEjmTpjaucT/7Q2sIhIdPhlBYSXF9myIlWX8kLiTA16KYjbr7md\nvy7/K+OvGk/fxnp2rdjJX3/81875i37DoNnUVtey7u51jP7CaOoPrWfnKztZ89M11FbXArlXJgBY\n9OQijrz6yM4el0VzF9H4eKMu0iIiJZArKy6edXGgaTPZ5MoLv1Vsljy+RHkhsRZoyo2ZvQlsBxLA\nHufchFz7axi18pw74VyGf304fd9Tj1kfnGtj10s72fD9DQC9HgadftJ0aj9Zy8APDqSqroqO1g6a\n/tJE+z3t3LXoLmbNmEXd9Lou8x63vbiN1ruST//Ltu2aO64p5K8v0iPlPOWmJ3mhrKg8ubLigacf\nyGvaTK68GHngyJx5kCtLlBdSSmFMuTnZObe5l/VIAeUaNsxHb4c5ITnUWX9YPzqoAgyopv6wfp1D\nndkexx3EyGNH0rSuiUQiQXV1NSOPHcmqe1YBudf/dTitDSxSGsqLiIhaXvhlxc7tO0PJC7+14rWW\nvMSdptzETFgP5chnmBOgrn8dzcubGXD0UADMamh+fjN1/eswM7Yv396lx2X78u3UD6jv/HnpU5nP\n22/QSPasq6Hf8P3ZvrWVfgPq2LPOqB80gqVP+a9MoLWBRaRSRTEvcmXF0qegT109Tc9vp+HYtB76\n57fTp64+a06k5MoLgI3PNdPwnrQ8eKm5M0v6DRqZc3suE4/Pvb3Ylj4FvPJKqcuQApn0+UmB9gva\noHfAo2bmgDnOubm9LUzyE9ZDOebfO5+xXx/b2ehuOK6BsVeMZf735wdq0L/7wEP5x49eZMwVtfR7\nTwMtLzWz5kdrOfzA93L0B4/gvuvuY+wVey/+q65bxQXTLgD2NuYbd+37H+20k6/i1l98hdopVdQd\n1MiWZ1bQ/mAHnz79WgCOev9U/vCTOYz+DPQ/pIEdrzaz5sbVnH7mZzn0EJLhNZN9Vi0ohUyhELUg\nECkA5UVERDEvsmXF6MHvpXHXJKZ8+MssuO4a7IqxDHhvA9tfbGbNdas5+8OzMmZEOr+8uH3uLGov\nrafhkME0v7qVjTdt5uKPX0PjrklMO/mqnNuzWdF3cQ8/veJoXH8UUyZuKHUZUkRBG/Qfcs6tMbMD\ngMfM7GXnXJf/is1sJjAT4KvXfZWzLzm7wKUKhDcs2DnM6aC9rZ3aPrU9GubsSOxHx+oa3vzWq7iE\nw6qNqp31dBywHxfPupjVr69m8X8uZk/7Hmpqa5h00qRAfyi877jJzP/tj3n5R3/BdazEqmo5bMwH\ned9xk2EXNB4xiTFtR/KbOf/LCysX8+53Hcelp/2U9x0xmRV9F3PyqfDnG+bx0oY32H/4CE4+rbBL\nkPn13HSXHgwr+i7u8fFB6I8EKbGceaGsKJ4o5kW2rHCH7AfAtAuuZO3aV3nyP39NYs8eqmtqeP97\nz2faBVf6njtnXnh+c+P/8uzK3/Hudx3Hxadd07kt9c8Ft17HGxufZvgB47n449d0OVYkygI16J1z\na7x/bjSz+cBEYHG3feYCc0E3OoWpNw+/CNJoTA1z9juyHx0d0N6eoGV5S8ZhzkwNxmlfOos5N6xh\nzMyju/SGT7vsLJY8voTXNr7GxB9N7Nz22tzXWPL4Ek489cTO8y19at+ejmV/XMjrLU8y9isTqRs/\nhNY3t/D6rU/yq6Xf4Z8nfgdIXohXvv4Sb75gHHPwWZ0X4MZdk2g8YhLnHPG1zvMVuhHt12MU1rHZ\nhPVHghTWpI+VuoLw+OWFsqJ4opAX3WXLipmXncWKvot5+eklvLr7rxz9g+M7R11fvfGv/PaF/+Ww\nCbk7Y/zyIltWpLzvuMm9asDnc83N1gET5JzqvJF0vg16M+sHVDnntnvffxz4buiVSUZTZ0zt0TSS\npU8FazhO+fCXWXDtNYz40kj6HT6Y5ue3sv7H6/YZ5sw2vJhraBeg/oR6Vty8gu1vb2fAgQMYdsKw\nfYZ9M12cfvzvt9HwgWFs/t1LtG9sofaAfjR8YBiPzbuts0Hf3LyZhQsfYvDgG1i48DJOO+1iGhqG\ndJ6juXkz1133Wa64Yi6NFL4RHSVh/JEgEpTyIlp6mhcpfteRoHnR3Yq+i3NmxTV3nMh9P5nH4En1\nvHVXWl5MGsbzT85jxmW5G/R+eeGXFdA1L7pv681nlYvfdJ04TvWR0gnSQz8cmG9mqf3vds79IdSq\nJKuePGK6J70G0y64khdefJKXr/wLruPtzqHKIMOckHtot3V7Kx1NHQz/l+GMOGgUbW/sYu0ta6la\nXeV73o2r1tOnbj9GXjqSukMOoPXVVtbdtI5d63d37vPII3eQSJxFXV0jO3acxSOP3M4nPvGVLttf\neWXrPq+LSMEpLyKkJ3nRE/nkhd80oDdefCO0vPDLitQ+yguJI98GvXNuJXB0EWqRgFJPQQ0iaO9B\nc/Nmtmzczej+K6muHkoisZktGy+guXlLoF6K9KHdnd5DXFMrBGzb9hqjLhrOfofUAX3Y7xBj2EUH\nsPZ7G7r80ZGph75+cD0jLt27JnHf4/aj+tIqVv/Xelb0XUxL0zYWLnyIurrkCgt1dTNYuPCCzp6X\nID0yIlIYyovo6UleBJG67q7btJUhA56nunoIicQW1m2axjNtD3JcnylZj23cNSnnajIACUswfEYy\nL8ySeXHAjANY818bfBcWyJYX67+/nmd2L+APT9xHTf297LLtWP0/sXDhv3TJBOWFxJn/n7wSO0uf\nCj7VJiXVc1FdnVxKLNmoT/ZgBHHU+6fy+k9Ws/7v26ht60fri21snLuZaSdfxX5V9VQPrKFjNzgH\nHbuhemAN+1XVJ+e5e3U+8fg2Pnvat2l6p6nzvDXVNdQMriHRmgAHidYENYNr6LtfDROPhyW/f4zW\njlMAWLv2fMC61P3II3fQ3j6ZzZuraW+fnPH3aW7ezFVXnU9z85bAn5eISKVa8/xj1NZMoaFhCP36\nQUPDEGprprDkd4/6Hjvt5KvYOHczrS+2dcmKD502FUhe86sbqpN50eHlRUM11VU1nXkxcuPh3Pv1\n62lpbupy7mx5UVNd01lzfT1sXX8hwD4Z55cXygqJslDWoddNeaXX03l9y5YtIpFYS1PT3d1eH5V1\n2DH93/NhE05kTNuRLLj5Op7rtkLAyEcOoeWVdeyX2A/ndmJWy+7X2xk54pAu9d6/4FrefnkPP//e\no5zyyU94hgHsAAAgAElEQVQw8XgY3zgetkGiT4Ldbbup7VOLbezL/iPHs/QpePXZ56je8yQbNtxM\na+sgNmw4nn799mfZslGcdtoMFi58iLa2n7BrV4I+fc5g4cIv7tProiFWkdJQVsTT04ueI7HnCbZt\nvqfL668+uz+cl/vYbKvJ9J/Qj6VPwcBhY9j5yg76Jvp25sWu19sYOmpM5zlS1+wlv3uUfg2f6Hx9\n/5HjsY3QXp1gV/tuqmv35sXCB59j584neGfTHHbvHEqiYxID6g/ozLhU73yuvAgjK/L5f8Dv2AeX\nDu/9ySUypmQf9OoilAa9bsyLn6uvXhB43/SLSJd/18eRcYWAYfWHsPL2Vxl60Wjqxg1j51ub2Hzn\nek44cG+DPnUxHTroJpY/ehkXnvIdlj71YnKd+Vvm8O4vjqFmbHLFgw2/XM2lH/spjbsmcd13F9Pc\nvJnZsy9g6NAbSCQu4wc/uJ+GhiHcf/+1tLdPpqWlhpqacbS0vEWfPpO7XIw1xCpSOsqKePrRgu9l\nfH3pU8Au/+MzribjHXdg3wn87fbfM/Sisd3y4iSg6zU7lRWpa/a0k6/i9jtnMfrSAzvXkl9z59ud\nnUuprKge0jUrYG/vfLa8CCMrQl0lbWKvTy0xpSfFVohC3hHf04vQpo3r2bPKsebap3AkMKqp3jWA\nTX3Xd+6Tuphu3VpN//57L6KNRyTf648/uZe1ry3bu858Whhku9Fp2bJFtLS8TCJxI1VVDXR0NNPS\nAsuWHdbZoA9yk5SIiBSHX15kywrY26GUba35XNd7v7xQVkjUaQ59BZl4fP5f0PM/DmbPvoWhQxrp\nz5O45mfoz5MMHdLI7Nm3AqQNdZ7Brl0J2trOYOHChzrnKZ5zxNd4/7supHrn+zjm4CldGvOpY+vq\nZgCpm2KTx86efQuDBo1h3Li/Mn7804wb91cGDRqzz/tmOlZERIovV174ZQUkG/XHHDxln7zwu97n\nygtlhcSBGvTSI+mN+qAN+32HMmu63HDkt73rUGfXi2ium3n9bvTN90ZgEREprFx54JcVkD0v8skD\nZYXEgRr0MRXG3fZLHl/CrBmzmH7SdGbNmMWSx5dk3C+9tz6I5FDmjSQSZ9He3kgicRYtLTeybNmi\nLtv37DmDtrZx7NlzRpftjzxyB7t3f5zVq3eze/fHu1xEkzfz3k1T04TOr0TibpYtW9S5bevWY3jr\nreFs3XpM5za/Y1O0qoGIxF0p86KncuWFX1ZA9rzwu97nyosgWQHKCyktc67wT95+8EH0OO+Q3X//\ntTzwwEOce+5Zgebxrei7OGcjfMnjS5hzwxzGzByzzxMF/R5a5TenPnUj0s6dJ9PU9AQDB55Mff0T\nnTck5doOjtmzL2DLlv9h167+9O27gyFD/qPLzUx+evpZFepYkWymTMFKXUMUKCuKo5DXsRV9F9PR\nlDkvTj71s5xzxNfyOn+QPPDLEuWFlJOgeaEe+hjKNQUll1xLXKU/jruqpopB7x3EmJljmHfHvH3O\nkTpP+hryuSSHSU9ix44nqK6+gR07nqC9/eRuU24yb0/1tuzaVYfZeHbtqtunlz6X3n5W+R4rIhIF\nYVzHsuXFnx+Z53+wD7888MsS5YVUKjXoYyg1n69Pn8aM8/gyDfulGt7pDfJ0695eR0NjQ5fXGhob\nWP/2+s5jlj5F5wM9Rm58b+B6ly1bxI4dt5FIHI1zCRKJo9mx49YuQ53Zti9btoimphuA83DuA8B5\nNDXdEHio0++zyiWfY0VEoqA3eeEnW168s2F9l9d6c26/PPDLEr+8yFWT8kLiTA36mAlyt336wy/S\npfeod2/UjzxwJM0rmjt/3rkTNj6393HcqWOznTuX2bNvYfDg8Ywb903Gjz+cceO+yeDB4ztXm8m1\n/fLL/x/O1VJV9QDwK6qqHsC5Wi6//Idd3iNTXfmsTKBVDUQk7vLJi1z6DRrJxuea2bmTzq+NzzWz\n//ARXfYrdF74ZUmQvMhWk/JC4k4N+pjxu9s+yLBfpt76qTOmsnruatb/fRs7mjtofmkbG+duZtrJ\nV3Xu39shxXxWF5gz5z/o6JiGc7WYjcO5Wjo6pjFnzt55mr1d1SCfmkVEoq4QedFd465JTDv5KjbO\n3Uzri23UtvWj9cW2zrxICSMv/H4fv7zo7Ypp+dQsUixq0MeM3932QYf9uvfWVw08kZNP/SztN+/H\ny/+ygvab9+t8ul5Kb4cUg64usGXLUbz++mC2bDmqc/urrz6Lczfj3Bk4d4L3z5t59dVnfesKujJB\nb2oWEYm6QuVFd+87bjIXf/wadt6a4LlLn2bnrYmi5EWurAB88yJXTcoLiTutclNGUnf49+lzn9dD\nsJm2tgt6dId/Kc6dcuWV57Fs2dscc8yBfPe78wO9bzHqEik0rXKTpKwonTjnRaas8Hvf1Ao5ygqJ\nG61yU4HCHPYLeu7ersO7Zs0Kli9fDlzN8uXLWbPm1S7vm0i0s3Ll6SQSCT0cSkQkT6XOi0JnRfr7\nZsoLZYWUOzXoy0iYw35Bz92bm6AgNffxPOAgOjrO65zzmHrfdesm0NGxnnXrjuvxw6FERKSrUudF\nobMi/X0z5YWyQsqdptxIwaSGO6urbyCRuCzwUOaaNSv4whfOJJG4E7Njce5Zqqsv4qc/fZjRow9h\nzZoVfPGLZ2L2G5z7J37yk+TrInGlKTdJyorKFFZWpPZRXkg50ZQbKbrkQz8ms3lzNe3tkwNPx0nv\ncYG+dO95Sa1ckEjU77PCjYiIxItfVkDmvPDLir37KC+k8qhBLwWRWg6sre0Mdu1K0NZ2RuD1jl95\n5e8490vgYzh3tPfPX/LKK39nzZoVvPDCcmAaHR0JYBovvNB13qSIiMRDkKyAzHmRKysA5YVUNDXo\npSBSPS4tLTXU1IyjpaWmS89LrvV/zz//y4wY8e8cfPByDj74OQ4+eDkjRvw755//5UDr0IuISDz4\nZQVkz4tcWQH+69CLlLPADXozqzazZ83soTALkmByrRDQ29UDghybbfuyZYtoabmRRGIKHR0fIpGY\nQkvLjV3WO842xJrrZqUg69CLSHQoK6InSnnhlxWQPS/8bmxVXkglq+nBvv8GvAQ0hFSL9ED6cOQn\nPvGVwNvyOW+u7bNn38Ls2RdwwAFd1/idPfvWtCHWn7BrV4I+fc5g4cIvctppF9PQMISrr16QtZ7c\n6wqLSAQpKyImSnmRKyuAnHmRKysA5sx5WnkhFStQg97MxgBnAlcDPfs/Xgqu63DkZZ0NY79t+ZzX\nb7vfGr9dh1jfok+fyYECJNd5exo+IhIuZUX0RC0v/K7p+07JUV70ytKlpa5ACmXKxEC7Be2h/xHw\nNWBAth3MbCYwE+Dyy+dw+ukzA55aeip10aqra2THjq4Xq1zbgpy3vX0yW7dW07//vhfQXOdODoWu\nZcuW29i2bRWDBo2lpqYPy5aNAqCl5WUSiRupqmqgo6OZlhZYtuww39pS521qurvb66Mq7wItEn3K\nioiJWl7kyorUduVFnpYuZcrEDaWuQorMdx16MzsLOMM5d7mZnQTMcs6dlesYrS0cnrAebZ06b0vL\nT2hurqGhYQ/9+n2x89igj/LO9EjusB8DLhI35bgOvbIieqKcF5mywq9m5UVAatCXlylTCrYO/QeB\ns83sTeBe4KNm9ss8SpM85BpSzOfR1n4rDwQ5d2rJsNraW7ssFaZHbotUBGVFxEQ1L7JlRZBjRSQz\n3wa9c+7rzrkxzrnxwDRgoXPuwtArk4xy3eUf9NHWvVl5IMi5k0uGfYqqqqPo6PhU51JheuS2SPlT\nVkRPVPMiW1YEOVZEMvOdctNlZw2jloX777+WBx54iHPPPatzXmG+w5x7H7f9J6qqRtLRsQ7nPqzH\nbotkUI5TbtIpK8pHofNCWVEEmnJTXgo45aaTc26R3wVaoi3bAzvyHebc2+MyEsC7UH9KD/QQqUDK\nivIQRl4oK0TCoSfFVpjk3MeT2Lz5G7S3nxz4gR1+kg/uuJn29tGdX6AHeoiIxFUYeaGsEAlHTx4s\nJTGX6m1pbz+ZXbu2st9+sHDhQ4Ee2OHnnnveLEiNIiJSemHlhbJCJBzqoa8gqd6WHTueoLr6Bnbs\neKJLr0up5fMIchERKZwo54WyQmRfatBXkGXLFrFjx20kEkfjXIJE4mh27Lg1MqsHpD8mXERESifK\neaGsENmXGvQVZPbsWxg8eDzjxn2T8eMPZ9y4bzJ48Hhmz7611KVlvflKRESKL6p5oawQyUwN+hIq\n9rBhIR7YEVbNqdr69GnUQ0RERLpRXnStS1kh0pUa9CVU7GHDQjywI4yaUz0udXUzAKirm6GeFxGR\nNMqLCsuKpUt7/yUVSavclEjXYcPLOO20iwM9wCkf+a5kE1bNuXqCUg8yERGpVMqLpErLCj0cSnpC\nPfQlEsdhw7Bq1qO+paypB03ypLxIUlaIZKce+hLYO2x4H5AaNrygKL0uvRVmzfn2BIkUTS8b4+pp\nk95SXuylrBDJTg36EojjsGEcaxYBCt4jrsa5FFMcr71xrFkk7tSgL4HksOFampru7vb6qIJc7Jqb\nN3PddZ/liivmZuwN8dteippFulAjXARQXohIMGrQl0DYw4bpKwtkunj6bc9EQ53SY3k2ytUIF1Fe\nxJrul5EiUoO+zPitLFCK1RKkQmQILzXKRaJLeRE+XQOlWLTKTZnxW1kgjqslSMTkWPt4ysQNXb5E\nJLqUFyLlQz30ZcRvZYE4rpYgJZZlyFiNdZF4U16IlBc16MuI38oCWnlActKUGZGKobwoEM2Tl4hQ\ng76M+K0soJUHpAs14EUqlvKicHTdlCgw51zBT/rggxT+pCLSe5o6Ey1TplipS4gCZYXE3tKluo5K\nuALmhW8PvZn1BRYD+3n7z3POfTu/6iSqerPmsESIGu5SQsqLyqK8EImOIFNudgMfdc7tMLNa4M9m\n9rBzbknItUkJ9GbNYSmBHPM21XiXElJeVBDlhUh0+DboXXJOzg7vx1rvS8OkZUhrDkeY5rtLDCgv\nKofyQiRaAq1Db2bVZrYM2Ag85pz7W4Z9ZprZ02b29B/+MLfQdUoRaM3hCNEa7xJTfnmhrCgPyguR\naAm0yo1zLgEcY2aDgPlmdoRz7oVu+8wF5oJudIojrTlcIpo6I2XGLy+UFfFXUXmhZSklJnq0bKVz\nbpuZPQGcDrzgt7/Eh9YcDpka7lJhlBflq9LyQtdoiYMgq9wMA9q9i3Md8DHgB6FXJkWlNYcLwKcn\nR6Eg5U55URmUFyLRE6SHfiRwu5lVk5xzf59z7qFwy5Jiu/rqBaUuIR7UaBfJRXlRAZQXItETZJWb\n54Fji1CLSHRoioxIjykvRERKo0dz6EXKXlpDXg13ERERiQM16KWydeuJVyNeRERE4kYNeqk86oUX\nERGRMqIGvVQG9cSLiIhImVKDXspThpta1YgXERGRcqQGvZQP9cKLiFQmPdFVKpwa9BJf6oUXERGP\nrv9SydSgl/jRTa0iIpKi3nkRNeglJtSIFxGpXHpKt0hOatBLNGk6jUj5U8+q9IAyQCQ7NeglOtQL\nL1JR9P+5iEhhqEEvpaWVaURERETyoga9FJem0oiIiIgUlBr0Ei414EVERERCVboGfbFuhpo4sTjv\nU4kC/jtUA15EREQkPOE06CPS0Htw6fDe/eGgPwIyU2+7iIiISOSE0qCPSiOvt3U8WIzRg6j90RCR\nP8JEREREpGc0hz6DYjRai/JHA+z9wyHA+6mxLiIiIhI/atCXSHH+aBiutd1FREREypxvg97MxgJ3\nAMMBB8x1zl0fdmGSPzXgRaSYlBciIqURpId+D/BV59wzZjYA+LuZPeac+0fItYmISLwoL0RESqDK\nbwfn3Drn3DPe99uBl4DRYRcmIiLxorwQESmNHs2hN7PxwLHA3zJsmwnMBJhz+eXMPP30ApQnUhof\n/cpXaG5q2uf1hoEDWXjttWX3viKFli0vlBVSbpQXEgWBG/Rm1h/4NfBl51xz9+3OubnAXAAefNAV\nqkCRUmhuauLpgQP3eX1ChotnObyvSCHlygtlhZQb5YVEge+UGwAzqyV5cb7LOfebcEsSEZG4Ul6I\niBSfb4PezAy4GXjJOacxHBERyUh5ISJSGkF66D8IXAR81MyWeV9nhFyXiIjEj/JCRKQEfOfQO+f+\nDFgRahERkRhTXoiIlIaeFCsVK9cKAeuamxm9Zcu+B9XWhlqT3/tqVQMRkeLyu+4qLyQK1KCXipVr\nhYCRDQ0lWT3A7321qoGISHH5XXeVFxIFgVa5ERERERGRaFIPvUSa35Bhru1Ar4cb39i0iaM3bdrn\n9dUB6+qt1Vu3MmHr1qzvKyIimeWTB37H5pIrL8Kc9qK8kHRq0Euk+Q0Z+m3v7XBjLfCc7Xtv3wjn\nAr1vb1V3dPB0hnmXo9vb8zqviEi5yycP8rmm58qLMKe9KC8knabciIiIiIjEmHroJdZe37SJURmG\nOluBdw8blvPYhoEDM/aSNAwcyO5Nmxjt9n0q/W7vnxu2buUfGYY6NwSoecg551Cb4dztZtQ6x3O7\nd++zLRGgZj9hDv1qNQURibJcWeHH77q7G7LmRT5ZAeWXF8qK8KhBL7G2H7A2w+sjAhyb6+Ix4uyz\nWZPjvB0dHRyeYaizI8BQZ61zrM8yPFtdVcXRGc5b7Z03nwtemEO/Wk1BRKIsrKxInTtbXuSTFVB+\neaGsCI8a9FJyuf5iX7lpE0fluDk1F78bhvxugsrVc5Po6GBnlp6ROPZAxLFmEaksftepXHnRx+fc\nufIiyPUxW17kygoCnjtq4lhzJVCDXkou11/stcC9GY45Oe37bBfSWuCODK+fFuB9IXcAtAGTsrye\nTw9EoqqKCYlExtfDpF4TEYk6v+uUX17k6qTJlRdBro/Z8iJXVhDw3NkoLySdGvQSaQYcnmG40dLm\nFNZlOXb44MEcnuGiMzzgRac6x7Y+wNM5VsHprTGDB+tCKSLSC355kS0rILy8CCsrQHkhXalBX0bK\ncRjMAf/IcOELcilcvWkTz/Vyuo6fXDdBhakc/x2LSPGV47UkV17s26TuKldejPFZYCGXUmUFlOe/\nY8lODfoyUo7DYAYcnuV1SF4UM93UtJvkjUpHZ+gZIUDPyG5gdJbX8c79WobtB/meObk6QabemXYz\n31UJ8vl3nOvcmS76PZHPagoiUnyVlhe5siIljLzIJyug/PJCWREeNegl1rYuWJB124izz+71eeur\nqljj88COXMO3uWz57W97eWR+cvXITPjXfw3t3CIipZYrK/KVKy8SHR29zgoov7xQVoRHDXopuVx/\nsbdt2sRRGY5py/Bapn2OztCzkTo21/uua27O/LS9tIv2zizvW6oeiHyGV9VrIiJR53edaoNQ8sL3\n+lhbmz0vdu/OmhWpcygvpBDUoJeS8+sJ6O2QYR+yP47b7339VFdVUZ9l/d9S9UDkM7yqXhMRiTq/\n69S7hg0LJS/83nfN/fdn3Tb63HOzZgWU7tqrvCg/4a5tJCIiIiIioVIPfRkpx2Gwdc3NjN6yZd8N\nGXo8umsn8xBqsOfz+cg1xBqiMG9sFZHKobzoKrS8KFFWgPKi0qhBX0bKcRhsZENDr4cFD8pj+NVP\nriHWMIV5Y6uIVA7lRVdh5UWpsgKUF5XGd8qNmd1iZhvN7IViFCQiIvGkvBARKY0gPfS3AT8l81OR\nRSIrjqsH5KMch9Aldm5DeSExpLzY+7rEk2+D3jm32MzGh1+KSGHFcfWAfJTjELrEi/JC4kp5IXGn\nVW5ERERERGKsYDfFmtlMYCbAnMsvZ+bppxfq1FLBNCwoUl6UFRIW5YVUsoI16J1zc4G5ADz44L5r\nP4n0goYFRcqLskLCoryQSqYpNyIiIiIiMebbQ29m9wAnAUPNbDXwbefczWEXJhJXGvaVSqW8EOkZ\n5YUUirkMT0bLm4ZRRUSymzLFSl1CJCgrRERyC5gXmnIjIiIiIhJjatCLiIiIiMSYGvQiIiIiIjGm\nBr2IiIiISIypQS8iIiIiEmNq0IuIiIiIxJga9CIiIiIiMaYGvYiIiIhIjKlBLyIiIiISY2rQi4iI\niIjEmBr0IiIiIiIxpga9iIiIiEiMqUEvIiIiIhJjatCLiIiIiMSYGvQiIiIiIjGmBr2IiIiISIyp\nQS8iIiIiEmNq0IuIiIiIxJga9CIiIiIiMaYGvYiIiIhIjAVq0JvZ6Wa2wsxeM7P/CLsoERGJJ+WF\niEjx+Tbozawa+BkwGTgc+KSZHR52YSIiEi/KCxGR0gjSQz8ReM05t9I51wbcC5wTblkiIhJDygsR\nkRKoCbDPaGBV2s+rgRO672RmM4GZ3o+/dM5dlH95hWNmM51zc0tdR3dRrCuKNYHq6oko1gSqqwL4\n5kXUswKi+99DFOuKYk0QzbqiWBOorp6IYk0pBbsp1jk31zk3wTk3AXhPoc5bQDP9dymJKNYVxZpA\ndfVEFGsC1VXxYpAVEN3/HqJYVxRrgmjWFcWaQHX1RBRrAoI16NcAY9N+HuO9JiIikk55ISJSAkEa\n9E8Bh5jZQWbWB5gGLAi3LBERiSHlhYhICfjOoXfO7TGzLwCPANXALc65F30Oi+L8oijWBNGsK4o1\ngerqiSjWBKqrrPUiL6L6uauu4KJYE0SzrijWBKqrJ6JYEwDmnCt1DSIiIiIi0kt6UqyIiIiISIyp\nQS8iIiIiEmO9btCb2S1mttHMXsiy3czsx97jv583s+N6X2bBajrJzJrMbJn3dWXYNXnvO9bMnjCz\nf5jZi2b2bxn2KernFbCmon9eZtbXzJaa2XNeXVdl2Gc/M/uV91n9zczGR6SuS8xsU9rndWnYdXnv\nW21mz5rZQxm2Ff2zClhXqT6rN81sufeeT2fYXvTrVrmLYlYErKsU17/IZUUP6lJeBK+pJNc/770j\nlxfKigJxzvXqC5gEHAe8kGX7GcDDgAEnAn/r7XsVsKaTgIfCriPD+44EjvO+HwC8Ahxeys8rYE1F\n/7y837+/930t8DfgxG77XA78wvt+GvCriNR1CfDTEvz39RXg7kz/rkrxWQWsq1Sf1ZvA0Bzbi37d\nKvevKGZFwLpKcf2LXFb0oC7lRfCaSnL98947cnmhrCjMV6976J1zi4F3cuxyDnCHS1oCDDKzkb19\nvwLVVBLOuXXOuWe877cDL5F8omK6on5eAWsqOu/33+H9WOt9db9z+xzgdu/7ecApZmYRqKvozGwM\ncCZwU5Zdiv5ZBawrqop+3Sp3UcyKgHUVXRSzogd1FV0U8yKqWQHRzAtlReGEOYc+0yPAS34BAN7v\nDYU9bGbvLfabe0NYx5L8qz1dyT6vHDVBCT4vb/htGbAReMw5l/Wzcs7tAZqAIRGoC+B8b/htnpmN\nzbC90H4EfA3oyLK9JJ9VgLqg+J8VJIP1UTP7u5lleuJfVK9b5SzKn3nJ8iKKWQHKiwLVBKW5/kUx\nL5QVBVJpN8U+A4xzzh0N/AR4oJhvbmb9gV8DX3bONRfzvbPxqakkn5dzLuGcO4bkUyYnmtkRxXhf\nPwHqehAY75w7CniMvT0doTCzs4CNzrm/h/k+PRWwrqJ+Vmk+5Jw7DpgMfN7MJhXpfSV+SpYXUcwK\nUF4UsKaiX/+imBfKisIKs0EfuUeAO+eaU0NhzrnfA7VmNrQY721mtSQvhHc5536TYZeif15+NZXy\n8/LecxvwBHB6t02dn5WZ1QADgS2lrss5t8U5t9v78SbgfSGX8kHgbDN7E7gX+KiZ/bLbPqX4rHzr\nKsFnlXrfNd4/NwLzgYnddoncdasCRPIzL9X1L4pZEaQu5UXwmkp0/YtiXigrCijMBv0CYIZ3J/CJ\nQJNzbl2I7+fLzEak5oOZ2USSv3/o/2N773kz8JJz7tosuxX18wpSUyk+LzMbZmaDvO/rgI8BL3fb\nbQFwsff9VGChcy7UOYpB6uo2f+5skvNMQ+Oc+7pzboxzbjzJG5gWOucu7LZb0T+rIHUV+7Py3rOf\nmQ1IfQ98HOi+wknkrlsVIJKfeYmuf5HLiqB1KS+C11SK618U80JZUVg1vT3QzO4heVf7UDNbDXyb\n5M0fOOd+Afye5F3ArwE7gU/nW2wBapoKXGZme4BWYFrYjRvPB4GLgOXevDqAbwAHptVW7M8rSE2l\n+LxGArebWTXJQLjPOfeQmX0XeNo5t4BksNxpZq+RvKltWsg1Ba3rS2Z2NrDHq+uSItS1jwh8VkHq\nKsVnNRyY77U5aoC7nXN/MLPPQemuW+UuilkRsK5SXP+imBVB61JeBK8pElkB0cyLCHxWscwKK057\nVkREREREwlBpN8WKiIiIiJQVNehFRERERGJMDXoRERERkRhTg15EREREJMbUoBcRERERiTE16EVE\nREREYkwNehERERGRGFODXkREREQkxtSgFxERERGJMTXoRURERERiTA16EREREZEYU4NeRERERCTG\n1KCXyDGzb5jZTaWuQ0RESs/MbjOz72XZ9gsz+1ap6yjQ+XeY2bu87+vM7EEzazKz+81supk9GtZ7\nS/ypQS8FZ2ZvmtlGM+uX9tqlZrYoyPHOuf92zl0aQl2LzGyXd9FsMrPFZnZkod9HRKRSmNmHzOyv\n3jX1HTP7i5kdb2aXmNmfw35/59znnHP/VYhzWdKXzOwFM2sxs9VeY7ooOeGc6++cW+n9OBUYDgxx\nzn3COXeXc+7jxahD4kkNeglLNfBvpS4igy845/oD+wOLgDtLW46ISDyZWQPwEPATktfU0cBVwO5S\n1pWH60nm1pdI/j6HAg8AZ5aglnHAK865PfmeyMyqC1CPRJwa9BKW/wfMMrNBmTaa2fVmtsrMms3s\n72b24bRt3zGzX3rfP2xmX+h27HNm9k/e94eZ2WNez9AKM7sgSHHOuQRwL3B42nknmtmTZrbNzNaZ\n2U/NrI+37Wdm9sNudSwwsyu870eZ2a/NbJOZvWFmX+p23qe933WDmV0bpEYRkYg7FMA5d49zLuGc\na3jJHS4AACAASURBVHXOPQq0A78A3u+NiG4DMLMzzexZ71q4ysy+k36ytN7+bd72S7q/oZkNMLMn\nzOzHXo965zQYMzvJ61X/qjdKvM7MPp127BBvGkuzmT1lZt9LjSKY2SHA54FPOucWOud2O+d2ej3j\n/5OhjsFm9pB3zd/qfT8mbfslZrbSzLZ7mTDde/1gM/ujN6Kx2cx+lXaM87ZfBVwJ/LP3+f1r9xGP\nXNnnfSY3mNnvzawFODn4v1KJKzXoJSxPk+wBn5Vl+1PAMSR7Qe4G7jezvhn2uwf4ZOoHMzucZM/F\n7yw5pecx7/gDgGnAz719cvIa6tOBJWkvJ4ArgKHA+4FTgMu9bbcDnzSzKu/4ocCpwN3eaw8Cz5Hs\noToF+LKZneYdez1wvXOuAXg3cJ9ffSIiMfAKkDCz281sspkNBnDOvQR8DnjSm0aS6thpAWYAg0j2\nel9mZucCmNk44GGSvf3DSObDsvQ3M7MhwP8Bf3HOfck55zLUNAIYSPJa/K/Az1J1AT/zahgBXOx9\npZwCrHbOLQ34u1cBt5LMowOBVuCnXp39gB8Dk51zA4APpP0u/wU8CgwGxni/bxfOuW8D/w38yvv8\nbu72OQTJvk8BVwMDgNCnPknpqUEvYboS+KKZDeu+wTn3S+fcFufcHufcD4H9gMYM55gPHONd7CHZ\nCP+Nc243cBbwpnPuVu88zwK/Bj6Ro6Yfe71F24EvkBweTtX0d+fcEu9cbwJzgI9425YCTSQv+pC8\ngC5yzm0AjgeGOee+65xr8+ZA3ujtA8neqoPNbKhzbodzLv2PCBGRWHLONQMfAhzJa94mb+RyeJb9\nFznnljvnOpxzz5PssPmIt/lTwONeb3+7lw/pDfpRwB+B+51z/5mjrHbgu945fg/sABotOe3kfODb\nXs/7P0h21KQMAdb14Hff4pz7tXeu7SQbzx9J26UDOMLM6pxz65xzL6bVNw4Y5Zzb5ZzrTWM7SPb9\n1jn3F++z3tWL95CYUYNeQuOce4Hk/Mr/6L7NzGaZ2UvesOM2kj0qQzOcYzvwO/Y2jj8J3OV9Pw44\nwRue3eadZzrJ3pdsvuT1FtWRvCjOM7OjvJoO9YZN15tZM8kekvSabgcu9L6/kL3z78cBo7rV8Q2S\nNzRBspfoUOBlb5j3rBz1iYjEhnPuJefcJc65McARJBveP8q0r5md4E2X2WRmTSR78VPX2LHA6zne\n6kyS1+1f+JS0pdu8851Af5K9/jXAqrRt6d9vAUb6nLuTmdWb2Rwze8vLi8XAIDOrds61AP9M8vdb\nZ2a/M7PDvEO/Bhiw1MxeNLN/CfqeaYJk36rMh0q5UoNewvZt4DMkhz8BsOR8+a8BFwCDvQZ2E8mL\nXCb3kJzu8n6gL/CE9/oq4I/OuUFpX/2dc5f5FeX1WvwJeA1IrRxwA/AycIg3PeYb3Wr6JXCOmR0N\nvIfkzVKpOt7oVscA59wZ3nu96pz7JMmh0R+Q/COiHyIiZcQ59zJwG8mGfabpMHcDC4CxzrmBJBvn\nqWvsKpJTErO5EfgD8PteXj83AXtITnNJGZv2/f8BY8xsQsDzfZXkqPIJXl5M8l43AOfcI865j5H8\nI+Flr36cc+udc59xzo0CPktyqszBPfxdgmRfps9fypga9BIq59xrwK9IrhqQMoDkhXUTUGNmVwIN\nOU7ze5I9Et8lOaeww3v9IeBQM7vIzGq9r+PN7D1BavP+QDgcSA2FDgCagR1eb0qXPwycc6tJzv2/\nE/i1c67V27QU2G5msy25dnC1mR1hZsd773OhmQ3z6t7mHdOBiEiMeTdmfjV1M6iZjSU5iroE2ECy\ngdwn7ZABwDvOuV1mNpHkNJuUu4BTzewCM6ux5A2sx3R7yy8AK4AHzayuJ7V6CyH8BviO17t+GMn5\n/KntrwI/B+6x5M21fcysr5lNM7N9Rpm936UV2GZm+5PsvEp9LsPN7BzvD4/dJKf9dHjbPmF7b57d\nSrLh3dM8yCv7pDypQS/F8F0gvUflEZI9La8AbwG7yDE86M2X/w3eTahpr28n2bs+DVgLrCfZA75f\njlp+6q0asINkw/w/nXMPe9tmkQyY7SR7U36V4fjbgSNJW+7SC4qzSN7E9QawGbiJ5DQigNOBF733\nvB6YlvbHgIhIXG0HTgD+5q2msgR4gWTv9UKSnSXrzWyzt//lwHfNbDvJe6w6Fwhwzr0NnOEd+w7J\nm0iPTn8z7ybYmcBq4LdZFlLI5Qskr8vrSV7D76HrEptfInlj689Idr68DpxHctGD7n5EcgrQZu/3\n/kPatirgKyRz6R2Sc+tTHUTHk/y8dpAcrfg3t3ft+UB6mX1S5izzTeIikomZTSI59WZclhUWREQk\nBszsB8AI59zFvjuLRJx66EUCMrNakg8duUmNeRGRePGmCB1lSRNJLlgwv9R1iRSCGvQiAXhzE7eR\nvMEp4woOIiISaQNITt9sITml8ofAb0takUiBaMqNiIiIiEiMqYdeRERERCTGasI46eJti9XtLyKS\nxaRBk7I9c6GiKCtERHILmhfqoRcRERERiTE16EVEREREYkwNehERERGRGFODXkREREQkxkK5KVZE\nJHQdUNtWS3VHNUb07jF1OBJVCdr7tKvrRESklCogL9SgF5FYqm2rpV+fflitYRbBC7RzuHZHS1sL\n7X3bS12OiEjFqoS8UL+RyP9v787jpKru/P+/Tm/QDTSNrM2OBttdRELIRkw0UVRQJugwo4L5xuBo\nxiQYviHLd5LJTPxN8v06EmMyBuIGJmqEBANmXIMMiREBDYoGAUVlsaGBdHc13U0vVef3R3U3Xd1V\ndW9X1a26VfV+Ph48gKq6tz5ck/M+fc6550pWKgwV+rZxBjDGYIoNhaHCTJciIpLX8iEv1KEXkaxk\n8G/j3MkY48vpXRGRfJIPeaEOvYiIiIhIFlOHXkQkCX967k/MvmA2l59/Off9532ZLkdERHzI66xQ\nh15EJEHBYJA7vn4H//Xb/+J3W3/HU2ue4p233sl0WSIi4iPpyArtciMiOe9LM+fTdPRvvV4vG3YK\nv9j0WMLn3bFtB+NPHc+4SeMAmPX5Wbzw5AucdsZpCZ9TREQyx4u8SEdWqEMvIjmv6ejf2DxsSK/X\nZ0RptPuiprqGUWNGdf195JiRvL7t9aTOKSIimeNFXqQjK7TkRkREREQki6lDLyKSoBGVIzh08FDX\n3w8fPMzIypEZrEhERPwmHVmhDr2ISILOufAc3n/nfQ68d4C21jae+s1TXHTFRZkuS0REfCQdWaE1\n9CIiCSoqKuLbd36bf7r6nwiGgsy9YS4fOvNDmS5LRER8JB1ZoQ69iOS8smGnRL2hqWzYKUmfe+al\nM5l56cykzyMiIpnnVV54nRXq0ItIzktma0oREckf2ZoXWkMvIiIiIpLF1KEXEREREclijh16Y0yV\nMWZ7t18BY8zX0lGciIhkD+WFiEhmOK6ht9buAqYAGGMKgYPAWo/rEhGRLKO8EBHJjL4uubkYeMda\n+74XxYiISM5QXoiIpElfO/TzgUejvWGMWWSM2WaM2bbuoXXJVya+s/n5zSxZsITrLrqOJQuWsPn5\nzZkuSSSj/uWWf+FTkz7F3OlzM12KH0XNC2VFflBeiETyOi9cd+iNMSXAHGB1tPettSustdOstdPm\n3DgnVfWJT2x+fjPL711O6XWlTHloCqXXlbL83uVqpCWvXXXdVdy79t5Ml+E78fJCWZH7lBcivXmd\nF30ZoZ8FvGqtPexVMeJfa1atYeyisVScXUFBUQEVZ1cwdtFY1qxak+nSRFyrPVrLrZ+/nbpjdSk5\n37RPTGPwkMEpOVeOUV7kMeWF5IJsy4u+dOj/gRjLbST3Ve+rpryqPOK18qpyDu07lKGKRPpuzUPr\n2bHtBKsf1FIPjykv8pjyQnJBtuWFqw69MWYA8Fngt96WI35VOb6SwK5AxGuBXQFGjR+VoYpE+qb2\naC3rHnmRoSPvYd0jL6Zs1EUiKS9EeSHZLhvzwlWH3lrbaK0daq2t97og8ad5C+ZxYMUB6t6sI9Qe\nou7NOg6sOMC8BfMyXZqIK2seWk97+2z69T+d9vbZWTPqkm2UF6K8kGyXjXnhuA+9CMCMS2YA4bWR\n7+57l1HjR3HzLTd3ve5k8/ObWbNqDdX7qqkcX8m8BfNcH5tvdK1Sr3O0pWzgYwCUDbyOdY/M55ov\nzKFiaEWGqxPJLcqL9NG1Sr1szQt16MW1GZfMSKih6NzxYOyisUypmkJgV4Dl9y7vOqecpGvljc7R\nlrKiYQAUFQ2jqWPU5UtLFiR83m984Rts/eNW6o7VcXHVxXz521/m7xb+XarKFslaygvv6Vp5I1vz\nwlhrU3ayTpvqNqX+pJK1lixYQul1pVScffIn27o362j+VTN3rrozg5X5j66Ve6VNpZSWl7r67Bev\n+Ar73z3W6/Vxk4Zy/+9/kurSIjQHmmkua454bWbFTOPpl2YJZYX0pDbQPV0r9/IhLzRCL56r3lfN\nlKopEa+VV5Xz7r53XR2fT1OKyV4ric7rRlhEUkN54Z7ywhvZmhd9fVKsSJ8ls+NBvj2gRLtDiEg+\nU164p7yQ7tShF88ls+NBvj2gRLtDiEg+U164p7yQ7rTkRjyXzI4H+TalmOzuECIi2Ux54Z7yQrpT\nh17SItEdDzqnFLvf9JPrU4qJXisRkVygvHBPeSGdtORGfC0fpxQ3P7+ZJQuWcN1F17FkwZKcXf8p\nIpJKygvlRT7TCL34Wr5NKWpf4exy6MAhvr3o2xyrOYYxhnlfmMf1t16f6bJE8pLyQnnhV+nICu1D\nL+Ij2lfYvb7sK+yVI4eOcOTQEc6achaNDY38/Sf/nrsfu5vTzjit6zPahz42ZYVI4pQX7mU6L9xk\nBWgfeskB+bR3cDz5dlNXOv3xuT/y8EMP88H7HzB6wmhuuPEGPvnZTyZ1zuGjhjN81HAABgwawKSq\nSRz+4HCvRlpEUkNZcZLywjupzot0ZIU69JJxmjY8KR9v6kqHPz73R+762V2M+9I4LjjjAurfqueu\nn90FkHSnvtPB9w/y1utvcd6081JyPhGJpKyIpLzwhtd54VVW6KZYybh82zs4nny8qSsdHn7oYcZ9\naRxDzhlCQVEBQ84ZwrgvjePhhx5Oyfmbjjex+PrFLP3hUgaWD0zJOUUkkrIikvLCG17mhZdZoRF6\nyThNG56Ubzd1pcsH73/ABWdcEPHa4DMGs/f9vUmfu62tjcXXL+aKa6/gkqsuSfp8IhKdsiKS8sIb\nXuWF11mhDr1knKYNI2lf4dQbPWE09W/VM+ScIV2v1b9Vz+gJo5M6r7WW7335e5xadSoLb1uYbJki\nEoeyojflRep5kRfpyAotuZGM07SheO2GG29g/y/2U/tGLaH2ELVv1LL/F/u54cYbkjrvX176C+sf\nXc/L//My8z42j3kfm8emZzalqGoR6U5ZIengRV6kIys0Qi9pEW9nAqdpw2zc1SAba85lnTcyPfzQ\nw+x9fy+jJ4zm9i/fnvQNTlM/NpUdDTtSUaKIdIjVfrpZYpKNbW821pzLvMiLdGSF9qEXz3XfmaC8\nqpzArgAHVhxwtdYvmWMzJRtrzkaZ3lfYLe1DH5uyQnpSXvi/5myUD3nhaoTeGFMB3AecA1jgf1lr\nX+pjnZKnuu9MAIR/XxR+3anBSuZYcB758GJkJNmaRbKZ8kKS4de88GoUXXkhqeJ2yc3dwNPW2nnG\nmBKgzMOaJMckszNBMsc67Vns1Z7G2okhPSwWay3G+Hew21qLJe8GoZUXkjA/5gXg2f73yov0yIe8\ncLwp1hgzGJgJ3N/xha3W2rqEv1HyTufOBN253ZkgmWOd9iz2ak/jZGoW94IFQWxbuJH2I2stts0S\nLAhmupS0UV5IsvyYF17uf6+8SI98yAs3I/STgCPAg8aY84FXgK9aaxu7f8gYswhYBPD1ZV9nzo1z\nEi5Kcsu8BfPCoxmL6LVG0M2xd991N4OuGUTx2GLaDrTRsLqBr97+Vcdjq/dVM/roaF7++ss0H2ym\ndEwpE+ZM4NC+Q13vezEyksy/V9xrK2mjsbWRwhOFGPw36mIJN85tJW2ZLiWdHPNCWSHx+DEvLNaz\nUXTlRXrkQ1646dAXAVOB26y1Lxtj7ga+CfxLRDHWrgBWgG50kkjJPvwi1Bzi6JqjtNa3UjK4hOLm\nYlfHlZaWsvtXuxnz5TEMOH0Ajbsb2f2z3QwvHQ54t6exHvaRJgXQ1r+NNvKqw+x3jnmhrJB4/JgX\nFcMrPNv/XnmRJnmQF2469AeAA9balzv+voZwAy3iWqIPv1izag2nf/P0iIa07s06VzcMmULDsLnD\nKJ1Uiik0lE4qZdjnh2HWhn8693JkRA/7kDylvJCk+S0vvB5FV15IKjh26K21h4wx+40xVdbaXcDF\nwF+9L03SzY974Vbvq6ZsUxlbv7WV9sZ2igYUMW7WOJr3NTse23S8icnTJ1P/t3paWlsoLilm3PRx\nvP3w24BGRkRSTXmRP/IpL5QVkg3c7nJzG/Crjh0L9gJf8K4kyQSvdnxJVqg1xIGNBxj3nXEMOGcA\njW80cuCuA1TYCsdjK8dXEvwgyNizx3a9VvdmXcQ0qUZGRFJOeZHj8jEvlBXid4673ABYa7dba6dZ\na8+z1l5tra31ujBJLy/v4k9GQ2MDY74yhoHnD6SguICB5w9kzFfG0NDY4HisHhMukn7Ki9ynvBDx\nH7cj9JLj/LoXbuuJVgafN5hQe4igDVJgChh83mD2ndjneOyMS2awa/su1n5nLU0NTZQNKmPu/LkR\noywr71zJ2sci31+4ZKGX/yQRkayWj3mhrBC/U4deAO92fElW2aAyGv/aSPnU8q7XAq8GKBvk/Kya\nzc9vZuNLGzn3jnO7bmTauGIjVc9XMeOSGay8cyWPP/E44741jkHnDqJhRwOPL3scQA21iEgM+ZYX\nu7bvUlaI77laciO5z6/TjXPnz2X/sv0EXg1g2yyBVwPsX7afufPnOh7rNC289rG1jFs8jvKp5Zhi\nQ/nUcsYtHsfax9Z6/c8SEcla+ZYXygrJBhqhF8Ddji/J7GrgNF0Z69ydn1n7H2vZ27CXskFlXDv/\nWlfHOk0LNzU0MejcQRHvDzp3EHsb9ib97xURyVWZzIt453XKi3jHxssLp6xI9t8rkgrq0EuXeHfx\nJ7OrgdPSFqdzL1yyMOa0ZrxjnaaFywaV0bCjIWJ6tmFHA2WDyny7i4OIiB9kIi+qplQ5njdWXjjV\nFC8vjh8/HjMrkv33iqSKltyIK8nsauA0XZnMueMd6zQtHG961q+7OIiI+J1XeeFVVkD8ZUROS3mU\nF+IHGqEXV5LZ1cBputLp3PGmX6v3VTN59GQOvHeAttY2ikuKGTx6MIf2HXKcFo43PXvdRdf5chcH\nERG/8yov3Jw3Vl7EywqIv4yo871YS3n8uuuP5Bd16MWVZHY1iLe0xencTst1ygaWsX/Lfio+WkFZ\n/zKCJ4Lsf2k/pQNLAeeHgcSanvXrLg4iIn7nVV44nTdeXjhlBcTPi3hLP5UX4gdaciOuJLOrgdN0\nZbxzOy3XsUFLzSM11L5US+DtALUv1VLzSA02aJP+9+7+4W5evPlFXpj/Ai/e/CK7f7g747s4iIj4\nnVd54XTeeHnhVVZ0/nuVF5JpGqEXV9zsahCL084D8c7ttFyntqaW0IAQR35zhPb6dooGFxFqC1Fb\nl/zDKQtKCxg2bxjFY4tpO9BGw2rnpw2KiOQ7L/Mi3nnj5UWt9S4rQHkhmWesTf6n05421W1K/Ukl\nL1097WpGfWtUrweFHPqPQzyx7QnH9xO1ZMESSq8rjZhCrXuzjuZfNXPnqjsTPq8IwMyKmSbTNfiB\nskJSKV4eAJ5kBSgvxFtu88KTEfotW5M7fvqHU1OHpJcX+/DOnT+Xx5c9zrjFJ9dE7l+2n2vnXwtA\nUWERRUOKCDYHKexfSPBEkKIhRRQVFiVVk25yEhHxhld7tsfLi/Wr18fNimTqUl6IH3jSoa86MTPh\nY3f135TQDwT6ISCzvNqH12n6dWLVRKiDYEmQltYWikuK6V/Xn4lVE5OqSTc5iYiknpd7tsfLix2v\n74iZFcnWpbwQP/Bkyc369aR1GnVX/00JHacfAlInU1OO3Rvh8qpyArsCHFhxgJtvuZk1q9YkXFO8\n8+pBIZIsLbkJ05Kb/OPHrJhxyYyk6lJeiJcyuuQm3RKdEdiyNfYPAurs942XU47x9qGPd/PVPT+4\nJ+GakrmpS0REovN6eUqsvHBq05OpS3khfpATHfpExftBIF5nP5p8/wHAqylHp33oIfbewcnW5LSH\nvYiI9I2Xy1Oc8iJem668kGyX1x36ePoy6t+Xdf+52vGft2BeeL3hInpNOSZj7WNrGfetcV07E3Tt\nK/wfa2M+5MPrmkREJDFetsvKC8ln6tCnQF86/30Z+c+mzv+MS2bwh9/8gf/52v8QbAtSWFzIpy76\nVNeIRbxlM/E0NTQRbAyy+xu7aa1upaSyhOGzh9PU0NT1mVg7E8y4ZAa7tu9i7Xciv1ejKCIimeGU\nFeBdXsTbxUZ5IdlOHfo0c9v5dxr191tnf+WdK/nzjj8z8fsT6V9VxoldTfz5J39m5Z0rARyXzcRS\nXFhM9SPVjPnnMZSdXkbT7iYO/vQgxYXFQPydCQA2vrSRc+84t2vEZeOKjVQ9X6VGWkQkA+JlxcIl\nC10ts4wlXl447WKz+fnNygvJaq52uTHGvAc0AEGg3Vo7Ld7n073LTb5xs6tPujv8V0+7mpHfGkn/\nM8swpgRrWzmxs4nD/3EYSPyBHtdddB3F/1DM4I8PpqC0gFBziPoX62l7tI1fbfxV3J0JAD3sQ3wp\nl3e56UteaJeb/BMvK5J9WGC8vKgcXxk3D/RwKPErL3a5+bS19miC9UgKHf9zI+v+sIzDNe8ycsQk\n5ly8mAunzor4jJulPT07/YlOc0J4qrPsjAGEKAAMUEjZGQO6pjpjPY7bjcoLKqmvricYDFJYWEjl\nBZXsf3Q/EH9nAovVwz5EMkN54RNePcQpmWUx8bKiqaHJk7xw2sVGD4eSbKclN1nmlVefYuWzSxhz\n03jOn/xhAntqWXnfEoCITr3T0p6eS3qee2QlL255nLHfHMegcwbR8EYDv3Y5zQlQOrCUwI4Ag84f\nBoAxRQReP0rpwFKMMTTsaIgYcWnY0UDZoDLH81aOr6Sotoghw07hb0eaGXJKKQVHTdfOA047E+hh\nHyKSr7x6iFMyy2LiZQVA2aAyx7yItRx1QEUl7dVFDBh5Cg21zQwYVEp7taGswjkPcunhUIk8nFP8\na+Zn3X3ObYfeAs8aYyyw3Fq7IsG6JEnr/rCMMTeNp+LMoQDh32+CdQ8u6zVKH0/PDv8dL8xj3P8Z\nz+Ap4casYsoQWAyrf7CWMz8VvYHuPsJ/2vjT+euP32Ts4mIGnFlO484AB3/8AWeNP5vzP35OzMdx\nO5m3YB73/uxeCi/tT9mkKqq37iL4zAlu+fItXe/H25lAuxaIpJ3ywifWrFrD2EVjuzqpFWdXwKLw\n68l06JPZTSZeVgDMnT83bl50dlajDVrN//T3efDnt1M8u4DSSVUce3UXbetDfOGyuzhQsoOn71nO\nmC/BwMnlHN8T4OAvDnDZFTezZSuc99F5PH3Pck67Lf15kcoOeGcuJ/p8Hslebjv0n7DWHjTGjACe\nM8a8Za2NWNNhjFkELAK49dblXHbZohSXKgCHa97l/MmRa2XKJw/h3ZptSZ23uamBQWefhsXS3tZK\nUXEJ5WcP5r2md6M2DD1H+Ovq+tG+r4j3/mUPNmgxhYaCpjJCI/qxcMlCDrxzgE3/ZxPtbe0UFRcx\n86KZrkb+Z1wyg8d/9ht23LUTG3oPU1DMuWefGbEzAcBj9z/GK6+9yunnT+71QA897EMkreLmRfes\n+PqyrzPnxjmZqjPnebWMpGtZjIW21jaKS4pdL4sJBfsROhA9K4Ck8uLCqbNY+7uf8NaPX8SG9mIK\nijlj7Me5cOosLmQWY1vP5bfL/y9v7N3EaadO5aZLf8qF58yCE1B1Tjjn/nTvGnYefpdTRo7i05em\nLi+cOu2p6ID3ZQttyT2uOvTW2oMdv9cYY9YC04FNPT6zAlgBuinWSyNHTCKwp7ZrhB4gsKeWkSMm\nJnXe0rJBNLwZYMC5AwhZSzDYTuObjZSWDYr6+Z6Nz/VXf4uVzy5hxKJhEaMfU2ddyap7N/Pm/rc5\n74fTGXF+eOTj7RVvs/n5zY6N5Ya1G9hVvYvxX59O2alDadp7jF0PbmPD2g18Zu5ngHCn/u3X9/PO\nlgFM+/CUiHPqYR8i6eWUF92zQjfFeiuRZSRuOoQlpWXUv94QzosQtLUFadzRSElpWcTx0TZnmP+V\nK1l+70HGLjo/YiR8/i1XAuFlQm/XvM30H0/ver97XnSeM9p9Ytv/ZwPvNL7EuNunUzpxKM3vHeOd\nB1/i11v+lb+f/q9cOHUWe9/ZyXtvGKZ86Mpes9pXnfMNrjrnG11/T3UH2etRc43K5zfHDr0xZgBQ\nYK1t6Pjz54B/87wyiWrOxYvDa+ZvCo/MB/bUcvC+fSz8XHJ34V/+yX9m3V13MuorlQw4awj1r9dy\n6CfVzPnkElfHdy0Fqgr/oFFaNZjiL5bx6oPPA1D+kSHsXbWLNw42UDZmEKdcMJwHfrqGgsEnO9vR\nGv8H7n6IQR8dzpEnd9JW00jxiAEM+uhwHrj7oa4Ofd2xOp5+dCtDR/wXTz96K7Nv/ByDTxncdY66\nY3X8f7cs4zs/vz3idRFJLeWFvyT6sCSnjuHsT34tIi8C3fKi89hYu7E5LQNas2oNZR8pY9f9u2jY\n18Cg8YMY/pHhvZYJRcuLn/zvhyj/2HCO/v5kXpR/bDjPrXmIv5/+rwQCR9mw4UmGDLmXDRtu4dJL\nF1JePjTiHIHAUZYtu5nFi1dQhTrIkj3cjNCPBNYaYzo//4i19mlPq5KYOkcU1j24jHdrtjFyP82J\nWwAAIABJREFUxEQWfu7OPq2fj2b+td/ljTdf4q3vvogN7euaqpx/7XddHR9vKdCJxuNg2xjxxRH0\nO3U0LXtbqLn/ELxXEtH4RxsJOfT+IfqX9qPypkpKJ4+geU8z1fdV07q/peszT658jmD7bMoGTqal\nbjbrH3qW62+/JuL9na+093pdRFJOeeEjnR3gVC87vGDBRWzZ+Xve/+7OrryYMP5MLlhwEZyIf2ys\nZUA794SXAb375ruE6kOM/F8jGTVpNK3vnuCDBz6g4EBBV0bE2pa5Zv8hSqLkxYlD4bx45plVBINX\nUlpaxfHjV/LMMyu55prbI87xzDOr2L27Nup7In7m2KG31u4Fzk9DLeLShVNnJd2B7ykQOMqxmhbG\nDNxLYeEwgsGjHKu5lkDgWK8RjGjiLQV6d992Ri0cQf/T+wMl9D/dMGLhCA794EjXZ2ONCA0YXM7I\nLw6jfEr45quSKf0wXyzggx8cYstWOP3U8Oh86cDHACgdeB1PPzq/a5TeafReRFJHeeE/qV52uKv/\nJk4/tY6WesP4ka9TVDSU9vZjtNTPpzFQDyXxj+++DKip44HfgZ0BThkZXgYUNEFGLhhJv8mlGFNC\nv8mGEQtGcPDfD3flxJatm6J26suGlDHqppN72Pef2o/Cmwo48O+HeLVlHU+/8DgDS58EoLR0ARs2\nXBsxSu9mBF/ErwoyXYD4Q+fIRWFheCuxcKc+PILhxpyLF3Pwvn3U7TxGqD1E3c5jHLxvH3MuXkxR\nQQkFgwsItYC1EGqBgsEFFBVEtvyBwFG+//3PEwgc63qtqKCEkiElhJotJlRAqNlSMqSEfgXhLczu\nveM5Ghtn09QE7+++nqYmaGwMj9JDeHS+reUKag4OoK31iq7Xu6s7Vsc3rv0e9X+rT+jaiYjkk85Z\n0aKicGe3qGgowfbZbP597/a1p/M+Oo937jnAoVfqCLWHaH6zlZoVR/nEpfPC5yosorC8MJwXoXBe\nFJYXUlhwcvyxsb6Omy/t3WYXFRZRNKSIYHMQLASbgxQNKaJ/vyIOvv4cweBsTpjjHPjgKk6Y470y\n7plnVtHWNoujRwtpa5vVK/+iZZSIX2gfegFg+/aNBIMfUF//SI/XR7uadoy3FKjymck07q6mX7Af\n1jZhTDEt77RROWpyxDmiTXWOG3sWBTXNhApbaG1voqiohJKaQYwbO4KqEzNZte1OCttfov7Q/bQ0\nV1B/6DOUDCxjw/pTGHPeZ3niwa00N/6c5kaLKZrNEw/+U69Rei3JEckM7ciRnbZtfI1g+wvUHX00\n4vU9fzkF5kY/pvO/9RnTZjC29VzW3b+Mt2p2dWXFwGkD2LIVBg8fS9Pu4/QP9u/KixPvtDJs9Niu\nc72z7m0O7jT81w+e5eJ/CLfZ0z8ME6smQh0ES4K0tLZQXFKMqenPKZUT2bD+NULBF6g9vJyW5mHU\nHp7JoLIRXRnXOTrf2noPJ04EKSm5nA0bbosYpddyHPEzdegFgDvuWJf0OWItBRpeNpm9K/cw7IYx\nlE4YTtP7Rzj68CE+Mv5khz7WVOecixez8pfhB2l13QT8y5M3Ad9xxzoCgaMsXXotw4bdSzB4Cz/6\n0WrKy4eyevVdmNbP03K8lKKiCbQcf5/ikisiQqCxvo4nHtSSHJFM0K4c/hTrhtZOP173g6ivb9kK\nu7ptgNf9B7aI/9ZT6Z0VHWvvx/efxssr/5thN4zrkRcXAZFZsePZW7j+4n+lvHwoW7ZuCu8l/8By\nxnxpLBUdu60d/uUBbvrsT7lw6qyurCgcGpkVcHJ0vrGxiKKiCTQ2vk9JyayuzruW44jfqUMvnjtS\nc4j2/ZaDd23FEsRQSOGJQRzpf6jrM52NaW1tIQMHnmxEOxv93/7i//KXvb/ntFOnsvDSyJuAY93o\ntH37Rhob3yIY/AUFBeWEQgFaj8PBbWdQNTccLqvX3UVh+7W0BSfT1Di714iPiEg+6ezMJ9L+dT8m\n3gOg4nHKi1hZUXViJlXnzIy+13xHXsS7KTZaXjQ2wvbtZ3DNNbe7uqFWJJO0hl48t3TpAwwbWsVA\nXsIGXmUgLzFsaBVLlz4I0G2q83JOnAjS2no5GzY82bVO8cKps5jyodkUNl3IlA/NjujMdx5bWroA\n6LzRKXzs0qUPUFExlgkT/szEiduYMOHPVFSM7fW9paUL6G8HMbj/zex49k0qa8JPLNyytfcvEZFc\nl4rBjM5zOI329xQvL5yyAmLnRbys6PzeWHnhdKyIH6hDL57rPZVZFHHDkdP7kVOdkY1ovJt5nW70\njfd+1YmZUX9F6+Sroy8i0lsinfp4eeCUFRA7L5LJg2Q3jRBJBy25yVLdH36RqnV8r7z6FOv+sIzD\nNe8ycsQk5ly8OCXbY56cylyBta0YU0Jjo+mayux8v73951h7HGMG0thYGDHV2dLyOf72txZOOeVz\nvaZJY93MCxAMfkBt7SoCgWrKyyspKCjqugnKzY3APa9ztOnjeE8T1LIdEck0L/Ji8/ObWbNqDdX7\nqqkcX8m8BfOibo85/cMda+s7OvVOS3Di5QUQNyuAmHnh1N53vh8tLwBXm0Z4cZ1F3DLWpv7J2+vX\no8d5e2z16rt44oknufrqK1Oyju+VV59i5bM9bj7teAJtsp36zhuRmpo+TX39Cwwe/GnKyl7ouiEp\n3vtgWbr0Wo4d+yEnTgykf//jDB36zYibmZwkc62SOTbWqJQ6+TKzYqbJdA1+oKxIj760Y27W0G9+\nfjPL713O2EVjez2BNlqnPubNsVG4yQOnLMnGvBCJZfZsXOWFltxkoXhLUBK17g/LGHPTeCrOHEpB\nUQEVZw5lzE3jWfeHZUmfOzxNehHHj79AYeG9HD/+Am1tn+6x5Cb6+52jLSdOlGLMRE6cKKWl5XOu\npzqTuVbJXme3y3ZERLzS13bs5MObYn9mzao1jF00loqzK8J5cXYFYxeNZc2qNV2f6d7GdW//nDjl\ngVOWZGteiCRLHfos1Lmer6SkKuo6vkQefnG45l3KJw+JeK188hAO17yX9Lm3b9/I8eMPEQyej7VB\ngsHzOX78QbZv3+j4/vbtG6mvvxeYi7UfA+ZSX39v17FOdTldq3iSOTYeN+vyRURSIZG86N42RVO9\nr5ryqvKI18qrynlvz6GINqyy5iwe+9bdKcsLN1nilBfxMsyPeSHiljr0WcbN3fbdH37h1sgRkwjs\nqY38rj21jBwxMeK1RM69dOkDDBkykQkTvsPEiWcxYcJ3GDJkYtduM/Hev/XW/4e1xRQUPAH8moKC\nJ7C2mFtv/U/HupLZmSCduxr07OCDbrwVkeQlmxfdBxy6qxxfSWBXoOvvTU1Q81qAU0aOimjLUp0X\nTlniJi9i1ZQteSESizr0WcbpbvtEp/3mXLyYg/fto27nMULtIep2HuPgffuYc/Hirs8kem63uwsE\ng23s3XsZwWCw6/3ly79JKDQfa4sxZgLWFhMKzWf58m841pXMzgSZ3NVAS3VEJBVSkRfRluDMWzCP\nd+45wKFX6jgeCNH8Zis1K44y/9Pf7/qMF3kRLysAx7xIdMe0ZK+zSDpol5ss43SnfqIPv+i88XXd\ng8t4t2Zb1+O43TzAKdmaO9+vrv5PQqHhVFdPpaKiku3bR7N37+tYuxlrHwUGAseBJvbs6edYl5td\nbBKtORN6rj/dsrX3Tbe64VZEOqUqL6pOzIzYzatg8AwuuwJevf953qrZlba8AGJmxTXX3M6ePX+J\nmxdOD5bKpbyQ/KNdbnJI5x3+JSWPd4wQHKW19do+3eGfiXMDHDy4i9tuuwJjfou1f8c99zzFmDGT\nHb/X67qySbRdddTB9yftchOmrMicbM2LWFnh9L2dO+QoKyTbaJebPOTltJ/bcydy0yycnCoNBssi\npkidplg11XmS27X4Wq4jIpnOi1RnRffvjZYXygrJdVpyk0O8nPZze+7uNxy5/c6DB3fxxhs7gH8n\nFApSUDCfN964moMH98RdjpPssppcF2uLuGgPwtJIvkh+yXRepDorxoyZHDcvwN3DoUSylZbcSMp0\nTncWFt5LMHiL66nM7353Lq+/fjbW3gBMAN7HmIc577w3+bd/Wxt3ilWSp6U66aclN2HKivzkVVZA\n/CU5ItlIS24k7cIP/ZjF0aOFtLXNcr0cJ3wj0/1YeznWfqTj9/vZs+cvQPwpVkme01IdEZFUcsoK\niJ4XTlkBygvJX+rQS0p0bgfW2no5J04EaW293PV+x8uXb2P06DOZNOllPvSh15g06WVGjz6T5ctf\n6TbFOp9QKAjM5403dnDw4J70/gPziNM6fBGRRLnJCoieF/GyAlBeSF5Th15SonPEpbGxiKKiCTQ2\nFkWMvCS6/6+bfejFO25utBURccspKyDxZ4soLySfue7QG2MKjTF/McY86WVB4k68HQIS3T3AzbGx\n3t++fSONjb8gGJxNKPQJgsHZNDb+ouuR2/GmWMM3Mj1Cff20rl/B4CNs377R1RSrpI86+OJEWeE/\nfsoLp6yA2HkRLyvA3ZIckVzVl11uvgrsBMo9qkX6IN4OAYnsHuD22FjvL136AEuXXsuIEZF7/C5d\n+mC3KdZ7OHEiSEnJ5WzYcBuXXrqQ8vKh3HHHupj1xN9XWDKt5046PXfQ0c21eUlZ4TN+yot4WQHE\nzYt4WQHhJTnKC8lXrkbojTFjgSuA+7wtR9yIt3wl0cdtuzk20WUzbqZYY9HewdlFo/f5TVnhP37L\nC6c2XXkhkhi3S25+DHwDCMX6gDFmkTFmmzFm29NPr0hJcRJdZ6NVUlLVq7GK956b88bbeSDeuTun\nQo8dO4933hnCsWPndU2FuplijcVpilX8S8tz8pKywmf8lhfxsqLzfeWFSN857kNvjLkSuNxae6sx\n5iJgibX2ynjHaG9h73j1aOvO8zY23kMgUER5eTsDBtzWdazbR3l/97tz2b59H1OmjO/aF9jLx4BL\n9srn/e9zcR96ZYX/+DkvomWFU83KC8lHqdyH/uPAHGPMe8BjwGeMMb9MojZJgtPSlkSnG52mOd2c\nu3PLsOLiByO2CtM0qETTcwS/6sRMjd5nN2WFz/g1L2JlhZtjRSQ6xw69tfZb1tqx1tqJwHxgg7X2\nes8rk6jiTSm6nW5MZOcBN+cObxn2jxQUnEco9I9dW4VpGlTc0sOtspeywn/8mhexssLNsSISneOS\nm4gPaxo1J6xefRdPPPEkV199ZdfOA8lOc5583PYfKSioJBSqxtpP6rHbkrRcXJqTi0tuulNW5I5U\n54WyQqRvUrnkpou1dqNTAy3+Fggc5bnnfktbWzHPPbfW9c4DTk6OuFQCdDTU/6gHekjS4t1cK/6k\nrMgNXuSFskLEG33Zh15ywDPPrOL48eG0ttZz/Piwrv2Bw9OcH1Bf/0jE57dvH+1qb+Lwgzu20NZ2\nf4/Xi1NZvkjE3vdbtube6L2IX3iRF8oKEW/0acmNW5pG9adA4ChLllzN0aOFGLMCaxcxbFiIO+98\nQrsHSE7ovjzHzx37XF9y45aywr+UFyL+4MmSG8lunaMtcBUFBVXAVV2jLn6QzCPIRQDtmCOSIn7O\nC2WFSG/q0OeRbdueobn5Naz9CMHgX7H2IzQ3v8a2bc9mujQg8jHhIsnSjjkiifNzXigrRHrTGvo8\nMm3apdTWXsrAgR/veu348X9m2rQMFtUh8jHht3DppQs1rSsp0X3N/a7+m7o69X5ekiOSaX7NC2WF\nSHQaoc+gdE8bpmJ/X69qTuYR5CJuxVqSo5F78TvlRZiyQiQ6jdBnUPdpQzc7ySTrjjvWJX0OL2ru\nHHEpLX0cgNLSBWzYcK1GXsRT3UfuIXLHHI3ei98oL5QVIvFohD5DIqcNn8yKm3u8qlmP+hY/0Jp7\n8SvlRZiyQiQ2degzJBunDb2qWY/6Fj/RQ6zEb5QXYcoKkdi0D30GJPPY7EzJxppFUimVe9xrH/ow\nZYWzbGx7s7FmEb9yuw+91tBnQLxpw3SsjUxENtYskkqdI/bdd8oBrbcXb2Vj25uNNYuwZUufD5k9\n/XDc99dvGXnyL9On9/n8faEOfQYk89hsNwKBoyxbdjOLF6+IOhri9H4mahbJFtoGU9JJeSESRwKd\n8HicOujJnG9991o96NxryU0OWr36Lp544kmuvvrKqI2n0/si0nd9WZKjJTdhyorMU15IxiXZKU91\nJ9xrEaP24Ni515KbPOX00A09lEPEGz2X5GjEXvxOeSFpF6Pznm2d8mREjtqPDF+TFIzYa5ebHOO0\ns0A27pYgkk2iPbhKxI+UF+K5LVsifs2efjjqr3zV9W/vfp0SpBH6HOL00A09lEMkfXQTrfiZ8kJS\nSiPvCYs6Yg99HrXXCH0OcXrohh7KIZJ+0fa1F8k05YUkpcfIO6CR9xSIuHZ9HLHXCH0OcdpZQDsP\niGRW9x1yRDJJeSGuaOQ9Y/p6jbXLjYhImrndtSDXKStEfCZKB16d9wybPTs1u9wYY/oDm4B+HZ9f\nY639XnLViV8lsuewiAgoL/KN8iKLxVnKoQ58dnKz5KYF+Iy19rgxphj4kzHmKWvtZo9rkwx45plV\n7N5dqyf6iUgilBd5RHmRBdRxzxuOHXobXpNzvOOvxR2/NE2ag7TnsIgkQ3mRP5QXPqUlM3nL1S43\nxphCY8x2oAZ4zlr7cpTPLDLGbDPGbHv66RWprlPSQHsOi0iynPJCWZEblBc+0HOnmRi7zUh+6NNN\nscaYCmAtcJu19o1Yn9ONTtknEDjK0qXXUlLyeMf2ZEdpbb2WH/1otUZdRFIsH26KdZMXyorspLzI\nAI28569U3RTbnbW2zhjzAnAZELNDL9kn3p7DWhspIn2lvMhdyguPaatISYCbXW6GA20djXMp8Fng\nR55XJmmlPYdFJFnKi/ygvEgB3awqKeZmhL4SWGmMKSS85v5xa+2T3pYl6XbHHesyXYKIZD/lRR5Q\nXjhw8XRPddol1dzscvM6cEEaahERkSymvJC8pPXt4gN9WkMvIiIikve6deLVeRc/UIdeRCTVnKbc\nZ09PTx0ikhoahRefU4deRMQtF2tjQUEvkjM0Ei9ZQh16EZFODh12BbpI/tH/7yUbqEMvIrnN5ah6\nJ4W3iIhkG3XoRSS7qIMuIsnqYzsi4nfq0ItI5iQYquqki0if9Ghr1IZIrlGHXkRSR6PnIuIXuqFV\n8og69CLijnZ4ERG/00i85Cl16EXykZa6iEgu0P7wIoA69CLZT51zEcknGoUX6UUdehE/SHLHBQWa\niOQsjcKLOFKHXiQdXHTYFVAiIt3oplYR19ShF3GSgv2KFUYiInFoFF4kKerQS+5Th1wku6XrIUDT\np6fne/KRZilFPKUOvfhTigNcQSHiQz7aCnX9lpHp+cEhV39ocLh2aoNFvKUOvSTPoxBUAIjkNj/9\nf1w/NLigTruIb6lDL0kFzOzphyFHB5xERFIpXR3e9Ym26dOnx8wDtfUi/qYOfbqlay1oH2lkRUQk\nNyTSnnefPVAeiGQfxw69MWYcsAoYCVhghbX27rgH+bTT6gdqKEUkVyWUF+ILyiaR7OZmhL4d+Lq1\n9lVjzCDgFWPMc9bav8Y6QA2DiEhe6nNeiIhI8gqcPmCtrbbWvtrx5wZgJzDG68JERCS7KC9ERDKj\nT2vojTETgQuAl6O8twhYBLD81ltZdNllKShPJDM+c/vtBOrre71ePngwG+66K+e+VyTVYuWFskJy\njfJC/MB1h94YMxD4DfA1a22g5/vW2hXACgDWr7epKlAkEwL19WwbPLjX69OiNJ658L0iqRQvL5QV\nkmuUF+IHjktuAIwxxYQb519Za3/rbUkiIpKtlBciIunn2KE3xhjgfmCntVZzOCIiEpXyQkQkM9yM\n0H8cuAH4jDFme8evyz2uS0REso/yQkQkAxzX0Ftr/wSYNNQiIiJZTHkhIpIZelKs5K14OwRUBwKM\nOXas90HFxZ7W5PS92tVARCS9nNpd5YX4gTr0krfi7RBQWV6ekd0DnL5XuxqIiKSXU7urvBA/cLXL\njYiIiIiI+JNG6MXXnKYM470PJDzd+O6RI5x/5Eiv1w+4rCtRB2prmVZbG/N7RUQkumTywOnYeOLl\nhZfLXpQX0p069OJrTlOGTu8nOt1YDLxmet/bN8paV9+bqMJQiG1R1l2OaWtL6rwiIrkumTxIpk2P\nlxdeLntRXkh36tBLVnvnyBFGRxkZaQZOGz484fO2AGNs74dYtnT8fri2lr9GGRk57OLcQ6+6iuIo\n524zhmJr+WtLS6/3gi7O68TLkSLdfCUifhYvK5IVLy+SyQrIvbxQVnhHHXrJav2AD6K8PsrFseWD\nB0cdJSkfPJh+R46wP8oxozt+D4VCnBVlZCTkYmSk2FoOxRn9n5RgzU68HCnSzVci4mdeZUXnuWPl\nRTJZAbmXF8oK76hDL3kr3mjAqDlzMnLHeGFBAaVRGv/CjsZfIxgiIunlpt1VXkimqUMvGRdvCm7v\nkSOcF+fm1Hicbhhyugkq3lRsMBSiKcZUZzZOKWZjzSKSX5zaqXh5UeJw7nh54aZ9jJUX8bICl+f2\nm2ysOR+oQy8ZF28Krhh4LMoxn+7251gNaTGwKsrrl7r4XogfAK3AzBivJzOlGCwoYFqw9wrIYIG3\n4z+aBhURv3Nqp5zyIt4gTby8cNM+xsqLeFmBy3PHoryQ7tShF18zwFlR1g+abjcJlcY4duSQIZwV\npdEZ6bLRKYzzXgmwLc66xkSNHTJEDaWISAKc8iJWVoB3eeFVVoDyQiKpQ59DcnEazAJ/jdLwdb7S\nQvSbmlqAA0eO8FqCy3VagDExXu/63ji74MTTZkzUY9uMcbyJKZn/xvHOHe2cfZHMzVcikn75lhet\nxM4KiJ8XYx12THPKi0SzAnIvL5QV3lGHPofk4jSYAc6K8TpA7bp1MY8dNWcO50cZGcHFyEhZQQEH\n4+zv2w84FO07Hc8Mx373Oxefii6Z/8bxGvBpX/xiwjU5nVtE/Cff8iJeVnTyIi+CoVDCWQG5lxfK\nCu9k4sZsERERERFJEY3QS8bFm4JrPXKE86Ic0xrltWifOT/K6ErnsfG+tzoQiP60vW6jME0xvjdT\nU4peTa+KiPiBUzvVCp7khWP7WFwcOy9aWmJmRec5lBeSCurQS8Y5Te0lOmVYQuzHcTt9r5PCggLK\nYuz/m6kpRa+mV0VE/MCpnTp1+HBP8sLpew+uXh3zvTFXXx0zKyBzba/yIvdoyY2IiIiISBbTCH0O\nycVpsOpAgDHHjvV+I8qIR09tRJ9CdffAbQfxplg95OVONSKSP5QXkTzLiwxlBSgv8o069DkkF6fB\nKsvLE54WnJTE9KuTeFOsXvJypxoRyR/Ki0he5UWmsgKUF/nGccmNMeYBY0yNMeaNdBQkIiLZSXkh\nIpIZbkboHwJ+SvSnIov4VjbuHpCMXJxCl6zzEMoLyULKi5OvS3Zy7NBbazcZYyZ6X4pIamXj7gHJ\nyMUpdMkuygvJVsoLyXba5UZEREREJIul7KZYY8wiYBHA8ltvZdFll6Xq1JLHNC0okluUFeIV5YXk\ns5R16K21K4AVAKxf33vvJ5EEaFpQJLcoK8QrygvJZ1pyIyIiIiKSxRxH6I0xjwIXAcOMMQeA71lr\n7/e6MJFspWlfyVfKC5G+UV5Iqhgb5cloSdM0qohIbLNnm0yX4AvKChGR+FzmhZbciIiIiIhkMXXo\nRURERESymDr0IiIiIiJZTB16EREREZEspg69iIiIiEgWU4deRERERCSLqUMvIiIiIpLF1KEXERER\nEcli6tCLiIiIiGQxdehFRERERLKYOvQiIiIiIllMHXoRERERkSymDr2IiIiISBZTh15EREREJIup\nQy8iIiIiksXUoRcRERERyWLq0IuIiIiIZDF16EVEREREspg69CIiIiIiWcxVh94Yc5kxZpcx5m1j\nzDe9LkpERLKT8kJEJP0cO/TGmELgZ8As4CzgH4wxZ3ldmIiIZBflhYhIZrgZoZ8OvG2t3WutbQUe\nA67ytiwREclCygsRkQxw06EfA+zv9vcDHa+JiIh0p7wQEcmAolSdyBizCFjU8ddfWmtvSNW5U8EY\ns8hauyLTdfTkx7r8WBOorr7wY02gusT/WQH+/d+DH+vyY03gz7r8WBOorr7wY02d3IzQHwTGdfv7\n2I7XIlhrV1hrp1lrpwFnpqi+VFrk/JGM8GNdfqwJVFdf+LEmUF25zjEvsiArwL//e/BjXX6sCfxZ\nlx9rAtXVF36sCXDXod8KTDbGTDLGlADzgXXeliUiIllIeSEikgGOS26ste3GmH8GngEKgQestW96\nXpmIiGQV5YWISGa4WkNvrf1v4L/7cF4/ri/yY03gz7r8WBOorr7wY02gunJeH/PCr9dddbnnx5rA\nn3X5sSZQXX3hx5oAMNbaTNcgIiIiIiIJcvWkWBERERER8aeEO/TGmAeMMTXGmDdivG+MMT/pePz3\n68aYqYmXmbKaLjLG1Btjtnf8+q7XNXV87zhjzAvGmL8aY940xnw1ymfSer1c1pT262WM6W+M2WKM\nea2jru9H+Uw/Y8yvO67Vy8aYiT6p60ZjzJFu1+smr+vq+N5CY8xfjDFPRnkv7dfKZV2ZulbvGWN2\ndHzntijvp73dynV+zAqXdWWi/fNdVvShLuWF+5oy0v51fLfv8kJZkSLW2oR+ATOBqcAbMd6/HHgK\nMMAM4OVEvyuFNV0EPOl1HVG+txKY2vHnQcBu4KxMXi+XNaX9enX8+wd2/LkYeBmY0eMztwI/7/jz\nfODXPqnrRuCnGfjf1+3AI9H+W2XiWrmsK1PX6j1gWJz3095u5fovP2aFy7oy0f75Liv6UJfywn1N\nGWn/Or7bd3mhrEjNr4RH6K21m4C/xfnIVcAqG7YZqDDGVCb6fSmqKSOstdXW2lc7/twA7KT30xPT\ner1c1pR2Hf/+4x1/Le741fNGj6uAlR1/XgNcbIwxPqgr7YwxY4ErgPtifCTt18plXX6AWp3FAAAD\nbUlEQVSV9nYr1/kxK1zWlXZ+zIo+1JV2fswLv2YF+DMvlBWp4+Uaer8+AvyjHVNhTxljzk73l3dM\nYV1A+Kf27jJ2veLUBBm4Xh3Tb9uBGuA5a23Ma2WtbQfqgaE+qAvg8x3Tb2uMMeOivJ9qPwa+AYRi\nvJ+Ra+WiLkj/tYJwsD5rjHnFhJ9Y2pNf261c5udrnrG88GNWgPIiRTVBZto/P+aFsiJF8u2m2FeB\nCdba84F7gCfS+eXGmIHAb4CvWWsD6fzuWBxqysj1stYGrbVTCD9lcrox5px0fK8TF3WtByZaa88D\nnuPkSIcnjDFXAjXW2le8/J6+cllXWq9VN5+w1k4FZgFfNsbMTNP3SvbJWF74MStAeZHCmtLe/vkx\nL5QVqeVlh97xEeDpZq0NdE6F2fBeycXGmGHp+G5jTDHhhvBX1trfRvlI2q+XU02ZvF4d31kHvABc\n1uOtrmtljCkCBgPHMl2XtfaYtbal46/3ARd6XMrHgTnGmPeAx4DPGGN+2eMzmbhWjnVl4Fp1fu/B\njt9rgLXA9B4f8V27lQd8ec0z1f75MSvc1KW8cF9Thto/P+aFsiKFvOzQrwMWdNwJPAOot9ZWe/h9\njowxozrXgxljphP+93v+f+yO77wf2GmtvSvGx9J6vdzUlInrZYwZboyp6PhzKfBZ4K0eH1sHLOz4\n8zxgg7XW0zWKburqsX5uDuF1pp6x1n7LWjvWWjuR8A1MG6y11/f4WNqvlZu60n2tOr5zgDFmUOef\ngc8BPXc48V27lQd8ec0z1P75Livc1qW8cF9TJto/P+aFsiK1XD0pNhpjzKOE72ofZow5AHyP8M0f\nWGt/TvhJgZcDbwNNwBeSLTYFNc0DbjHGtAPNwHyvOzcdPg7cAOzoWFcH8G1gfLfa0n293NSUietV\nCaw0xhQSDoTHrbVPGmP+DdhmrV1HOFgeNsa8Tfimtvke1+S2rq8YY+YA7R113ZiGunrxwbVyU1cm\nrtVIYG1Hn6MIeMRa+7Qx5p8gc+1WrvNjVrisKxPtnx+zwm1dygv3NfkiK8CfeeGDa5WVWaEnxYqI\niIiIZLF8uylWRERERCSnqEMvIiIiIpLF1KEXEREREcli6tCLiIiIiGQxdehFRERERLKYOvQiIiIi\nIllMHXoRERERkSymDr2IiIiISBb7/wFhisPs5nSS+AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1444256048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "gs = gridspec.GridSpec(2, 2)\n",
    "\n",
    "fig = plt.figure(figsize=(13,9))\n",
    "\n",
    "for clf, lab, grd in zip([clf1, clf2, clf3, sclf], \n",
    "                         ['KNN', \n",
    "                          'Random Forest', \n",
    "                          'Naive Bayes',\n",
    "                          'StackingClassifier'],\n",
    "                          itertools.product([0, 1], repeat=2)):\n",
    "\n",
    "    clf.fit(X, y)\n",
    "    ax = plt.subplot(gs[grd[0], grd[1]])\n",
    "    fig = plot_decision_regions(X=X, y=y, clf=clf)\n",
    "    plt.title(lab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boosting\n",
    "\n",
    "### AdaBoost \n",
    "\n",
    "**(Indra)**\n",
    "\n",
    "Adaboosting is an ensemble method that creates a strong classifier from a number of week classifiers. A week classifier performs poorly, but better than guessing.\n",
    "\n",
    "The most suited and therefore most common algorithm used with AdaBoost are decision trees with one level. Because these trees are so short and only contain one decision for classification, they are often called decision stumps. But Adaboost can be applied to any classification algorithm.\n",
    "\n",
    "Adaboost trains several weak classifiers on different random subsets of training data. After each training Adaboost assigns a weight to each traing data point, so that data points with higher weights will have a higher probability of being included in the next training subset. From each training, the examples that were misclassified, are assigned higher weight, so they will have a higher probability of being corrected in the next training. (Before the first training all data points are assigned equal probability.)\n",
    "\n",
    "![alt text](algo_boosting.png)\n",
    "<p style=\"text-align: center;\">Figure 6. Adaboost algorithm.</p>\n",
    "\n",
    "After all the classidiers are trained, they are each given weights based on their accuracies. Classifiers with 50% accuracies are assigned weight zero (because 50% accuracy is the same as guessing), classifiers with higher than 50% accuracies are assigned weights larger than zero, and classifiers with less than 50% accuracies are assigned negative weights (because they predict the opposite of correct). \n",
    "\n",
    "$w_0(x_i)=1/N$ - weights before training the first model\n",
    "\n",
    "Create classifier $h_j(x), i=j,..,K$ from the training subset.\n",
    "\n",
    "Then compute the output weight for that classifier: $\\alpha_j=\\frac{1}{2}\\ln\\big(\\frac{1-\\epsilon_{j}}{ \\epsilon_{j}}\\big)$\n",
    "\n",
    "After computing the classfier weight, we then compute weights for each training data point. \n",
    "\n",
    "$\n",
    "w_j(x_i)=\\frac{w_{j-1}(x_i) e^{- y_i \\alpha_j h_j(x_i)}}{Z_j}\n",
    "$\n",
    "\n",
    "$w_j$ is a vector of weights for each point in the training data set. \n",
    "\n",
    "$Z_j=\\sum_i w_j(x_i)$ is the sum of all weights, used to normalize the weights so that they sum up to $1$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\alpha_j=\\frac{1}{2}\\ln\\big(\\frac{1-\\epsilon_{j}}{ \\epsilon_{j}}\\big)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn import model_selection\n",
    "from sklearn import tree\n",
    "from sklearn import metrics\n",
    "from sklearn import ensemble\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "iris = datasets.load_iris()\n",
    "X, y = iris.data[:, 1:3], iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.94730392156862742"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ab = AdaBoostClassifier(tree.DecisionTreeClassifier(random_state=0),algorithm=\"SAMME\",n_estimators=200)\n",
    "ab.fit(X, y)\n",
    "scores_ab = cross_val_score(clf, X, y)\n",
    "scores_ab.mean()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy of using AdaBoost technique is: 0.947303921569\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f1425541828>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEICAYAAAB/Dx7IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VPW57/HPkwskAQIIyF3AavFWQaVsepGq9a6gbJGy\nqwXd22J17160HGm7T231bHe3PRR73Ra8gtcqFUWtomz0aFsRUVHxgheUSwxXIQkhkGTmOX/MJCRk\nMrOSmcnMJN/36zWvJLPW+q0nC15PfvP8fr+1zN0REZHckZfpAEREpG2UuEVEcowSt4hIjlHiFhHJ\nMUrcIiI5RolbRCTHKHFLzjCzy8zsr5mOQyTTlLglK5jZ82a2y8y6p6i9n5tZnZntib7eNbOLUtF2\nnHPqD4t0CCVuyTgzGwmcDDgwOYVN/8nde7p7T+AHwL1mNjCF7YtkhBK3ZIMZwErgbmBmw5tm1s/M\nlppZpZmtAj7X9CAz+42ZbYpuf9XMTm7tBO6+DKhq2oaZfdvMPjSzz6LnGdJk25fN7BUzq4h+/XKT\nbZeZ2XozqzKzj83sEjM7Gvgj8KVoD3930ldFpBVK3JINZgD3RV9nNekV/wHYBwwG/jn6auoVYCxw\nCHA/8LCZFR3cuEWcB3QD3om+dxrwC2BatP0NwIPRbYcATwK/BfoB84Ano39IekTfP8fdewFfBta4\n+7vAd4CXor38PklfFZFWKHFLRpnZV4ERwEPu/irwEfBNM8sHLgKud/dqd18LLGx6rLvf6+473b3e\n3X8FdAdGN9llWrTnuwdYCvynuzf0hC8B7nT319x9P/BjIr3lkcB5wAfufk+07QeA94BJ0WPDwHFm\nVuzu5e7+doovi0hcStySaTOBZ9x9R/Tn+6PvDQAKgE1N9t3Q9EAzmx0ddKyIJujeQP8muzzk7n3c\nvQeREskMM7syum1I0/bcfQ+wExh68LYm5x7q7tXAN4j0rsvN7EkzO6qdv7tIuyhxS8aYWTGRUsXX\nzGyLmW0BrgHGAAOBemB4k0MOa3LsycB10eP7RksTFYDFOpe7fwI8xYFe86dEevoN7fUgUhYpO3hb\nk3OXRdta5u5nECmxvAfc1nCa4L+9SPspcUsmXQiEgGOI1KrHAkcDLxKpez8C/NzMSszsGJoMXAK9\niCT27UCBmV0PlLZ2IjMbBpwNNJQ1HgAuN7Ox0SmI/wm8HE3wfwE+b2bfNLMCM/tGNMYnzGygmV0Q\nTfT7iZRhwtE2twLDzKxbUldFJAElbsmkmcBd7r7R3bc0vIDfE6lB/xvQE9hCZMbJXU2OXQY8DbxP\npIyxj+ZlFYBvNMzjJjKQ+TfgBgB3Xw78FPgzUE6klDI9um0ncD7wQyLlk+uA86PlnDzgWiK98s+A\nrwFXRc+3gsgfhi1m1lD6EUk504MURERyi3rcIiI5RolbRCTHKHGLiOQYJW4RkRxTkI5GX9j9gkY8\nRUTaYGKfiTHXIMSiHreISI5R4hYRyTFK3CIiOUaJW0Qkx6RlcFJEJCPCUFhbSH44H4t9v7GMcpxQ\nXoi6bnVJdZuVuEWk0yisLaRHtx5YoWGWhYnbHa9zqmurqSuqa3c7KpWISKeRH87P2qQNYGZYoZEf\nzk+qHSVuEek0jOxN2g3MLOkyjhK3iEiOUeIWEUmxvz77VyadMIlzx5zL7b+6PeXtK3GLiKRQKBTi\nph/exH8/8t889spjPLX4KT5676OUnkOzSkSkS/r2xOns3fFZi/dL+h/CbS882O5231r9FocdfhjD\nR0Uel3rORefw3BPP8bmjPtfuNg+mxC0iXdLeHZ+xsn/fFu9PiJHM22Jb+TYGDR3U+PPAoQN5c/Wb\nSbV5MJVKRERyjBK3iEgKHTr4ULaUbWn8eWvZVgYOHpjScyhxi4ik0HEnHceGjzaw+ZPN1NXW8dSf\nn+KU805J6TlU4xYRSaGCggJ+MvcnfOfC7xAKh5jyrSkccfQRqT1HSlsTEckRJf0PiTkQWdL/kKTb\nnnjWRCaeNTHpdlqjxC0iXVIyU/4yTTVuEZEckzBxm9loM1vT5FVpZj/oiOBERKSlhKUSd18HjAUw\ns3ygDFiS5rhERKQVbS2VfB34yN03pCMYERFJrK2Dk9OBB9IRiOSOlctXsnjRYso3ljP4sMFMnTGV\nCadPyHRYIl1G4B63mXUDJgMPt7J9lpmtNrPVS+9emqr4JMusXL6S+bfOp/iSYsbePZbiS4qZf+t8\nVi5fmenQRLLCT6/6KV8b9TWmjJ+StnO0pVRyDvCau2+NtdHdF7j7OHcfN/myyamJTrLO4kWLGTZr\nGH2O7UNeQR59ju3DsFnDWLxocaZDE8kKF1xyAbcuuTWt52hL4v4nVCbp8so3llM6urTZe6WjS9my\ncUsrR4hkt107dnH1Rdeye+fulLQ37qvj6N23d0raak2gxG1mPYAzgEfSGo1kvcGHDaZyXWWz9yrX\nVTLosEGtHCGS3Rbf/Thvrd7Hw3flTok3UOJ292p37+fuFekOSLLb1BlT2bxgM7vf3k24Pszut3ez\necFmps6YmunQRNps145dLL3/b/Qb+DuW3v+3lPW6001L3qVNGmaPLF60mI83fsygwwZx5VVXBp5V\nohkpbadrlj6L736c+vpJlPT8PJW7J/HwXUv59uwZmQ4rISVuabMJp09oV+JomJEybNYwxo4eS+W6\nSubfOr+xTWlJ1yx9GnrbJT0j9ywp6XkJS++fzsWXT6ZPvz4Zji4+3atEOoxmpLSdrln6NPS2Cwr6\nA1BQ0J/6+klJ17qvu/w6Lv36pXzywSd8ffTXeWRh6ocG1eOWDlO+sZyxo8c2e690dCkfb/w40PFd\nsWSQ7DWT1q18bjWh+p3s2vHAQe/3S6pc8su7fplsaAkpcUuHaZiR0ufYAx9Dg85I6aolg2SumcR3\nx5O/zXQI7aZSiXSYZGakdNWSgWbxSCzqcUuHSWZGSlctGSQ7i0c6JyVu6VDtnZHSlUsG7b1m0nmp\nVCI5oSuXDFYuX8nsGbO55JRLmD1jtm7oJepxS27oqiWDrjooK/GZu6e80Rd2v5D6RkW6oNkzZlN8\nSXGzEtHut3dTc18NcxfNzWBk2al4bzHFpcUZjWHL5i38ZNZP2LltJ2bG1MuncunVlzbbp6ayhpqS\nmmbvTewz0YKeQz1uySpdca52PF11UDaX5RfkM/s/Z3PM2GOorqrmGyd/gy+d9iU+d9TnUnYOJW7J\nGioLtNSVB2U7wovPvsg9d9/Dpxs+ZciIIXzrsm9x8hknJ9XmgEEDGDBoAAA9evVg1OhRbP10a0oT\ntwYnJWt01bna8XTlQdl0e/HZF5n3h3kUfbOIExaeQNE3i5j3h3m8+OyLKTtH2YYy3nvzPY4fd3zK\n2gT1uCWLqCzQUlcdlO0I99x9D8O/PZy+x/UFiHz9duT9ZHvdAHv37OWaS69hzn/NoWdpz6Tba0qJ\nW7KGygKxaR53eny64VNOOOqEZu/1Pqo36zesT7rturo6rrn0Gs6bdh6nX3B60u0dTIlbOlS8wcep\nM6ZGatqzIj3tynWVbF6wmSuvujLhsdksV+Pu7IaMGELFexWNPW6AivcqGDJiSFLtujs/+9efcfjo\nw5n53ZnJhhlT/s9//vOUN7ph34bUNyo5r2Hwsd/l/RhxxQjCQ8I8N/85BvUbxLDDhzHs8GEM6jeI\nN+99k48WfkTRxiJmXDGDCadPSHhstsrVuHNVYV0hhd0LA+3bp7QPy/64jG7Du9H9kO7sfmc3m27b\nxHdmfYcRnxvR7hhef+l1bp5zMzV7a3j4zod56I6HGDRsECOOONBm/f566gvrmx03omjEDUHPoXnc\n0mGSmZOcq/OZczXuXNXWedzpmFUSRIfM4zazPsDtwHGAA//s7i+1IU6RpAYfUzFwGa9kka5yhgZc\ns9vJZ5zcIYk61YLWuH8DPO3uU82sG1CSxpikk0pm8DHZgct4c8SBtM0f14CrpEPCedxm1huYCNwB\n4O617p4bj0KWrJLMnORk5zPHmyOezvnjmofdsRwnHeXfVHJ3nORiDNLjHgVsB+4yszHAq8D33b26\n6U5mNguYBfDDW37I5MsmJxWYdD7JzEmecPoE1q1Zx5J/X8Leqr2U9CphyvQpgXvE5RvLGbJjCC//\n8GVqymooHlrMiMkj2LJxC46nrZyhedgdK5QXwuscCsEscMm4w7g7XueE8kJJtZNwcNLMxgErga+4\n+8tm9hug0t1/2toxGpyUVGta6jh4qmCQJHjFOVew3bYz9F+H0uPzPah+v5qyP5QxwAfQZ0AfDSB2\nFmEorC0kP5yPkYWJm0jSrutW16LekerByc3AZnd/OfrzYuBHQU8gkgpNyxlA5OusyPtBErflG/2n\n9Kd4VDGWbxSPKqb/Rf2xJZZw/rjkkDyoK6qjjrpMR5JWCRO3u28xs01mNtrd1wFfB95Jf2iSKdm4\nYKR8YzklL5Twyo9fob66noIeBQw/Zzg1G2sSH0xk+fGR44+k4rMK9tfup7BbIcPHD+fDez5UOUNy\nTtBZJd8F7ovOKFkPXJ6+kCSTsvUOfeHaMJuf38zwfx9Oj+N6UL22ms3zNtPH+yQ+mMjsjtCnIYYd\ne2DRy+63dzfO7tCycsklge4O6O5r3H2cux/v7he6+650ByaZka136KuqrmLo94bSc0xP8grz6Dmm\nJ0O/N5Sq6qpAx2t2h3QmuleJNJOtC0Zq99XS+/jehOvDhDxEnuXR+/jebNy3MdDxiWalLJy7kCUP\nNt82c3Z67jMhkiwlbmkmWxeMlPQqofqdakpPLG18r/K1Skp6BVsLtnL5Sp5/6Xm+cNMXGgcgn1/w\nPKOXj2bdmnU89OhDDP/xcHp9oRdVb1Xx0C0PASh5S1ZS4pZmgsywSGbwMlHPtrW2p0yfwkO3PMTw\naw4k1023bGLa9GmB4oo3K+XDdz5k+I+HN/5RKD2xlOHXDGfJL5Y0xpaNA7bSdSlxSzOJZlgkM3i5\ncO7CuD3beG03JNAlv1jC+qr1lPQqYdr0ac0Sa7y44pWA9lbtpdcXejXb1usLvVhftT7p31kkHXR3\nQGmTZO52d+G4Cxn040Etyh1bfrGFR1c/mta7B8bb/uE7H6YtLpGg2rIAR8+clDYp31hO6ejSZu+V\nji5ly8YtCY9trWe7t2pvoLYXzl3IheMu5MzRZ3LhuAtZOHdhs7jyh+Sz+ZPNfPz+x2z+ZDP5Q/Ib\nj403q2TK9ClsumUTla9V4nVO5WuVbLplE1OmT0n6dxZJB5VKpE2SGbws6VVC1VtVzXq2VW9VNQ4w\nxms7UZmlpGcJm1Ztos+X+lBSVEJoX4hNL22iuGfk3szxSkAN21orw2TrgK10XepxS5skMx86Uc82\nXttLHlzC8GsiA4hWaAcGEB9cAoCHnB1/3kHNxzV4yKn5uIYdf96Bhw5U7SacPoG5i+Zy7/P3MnfR\n3Gb16ZmzZ/Lo6kd5Zt0zPLr60WYDppoDLtlGPW5pk2SWhycaYIzXdqIBxJqaGg79yqGs/+n65kvi\n/xZsSXyi3zmZOxOKpJoSt7RZMsvDZ86eGXdudGttJyqzFBcXs+31bRz+fw5vfve/4gHtirOpeHPA\nlbwlE1QqkZyQqMxi+Ub/i2Lc/S8/+Vt7ZuttAKTrUo9bUi4di1USlVni3f0v2biy9TYA0nUpcUtK\npXOxSrwyS6K7/yUTl2aVSLZRqURSKp1lhXjzuBPN/EgmLs0qkWyjHrekVLrKConmcSea7ZJMXHrQ\ngmQbJW5JqXSVFZY8uCThjaDizXZJNi49aEGySVoS96pX0tFq1zb+i8H2S3Ttg7bTXlNnTOXm62+m\nrqSO+sp6CkoLKNxbyJwb5wDtv+/13qq9hKpDvH/d+9SW19JtcDcGTBrQuFwe4g8+Tp0xld/M+w29\nLu5F4bBC6jbXUfVwFd+/9vvpuRAiaZSWxD1638R0NNtlrSt6oU37t3b929pOWzT8wfifJ9exz/cz\n8BsD6T68G/s31bL1zq38z5PrkrrvdWF+IeX3lzP034ZS8vkS9r6/l7Lfl1GYXwgEG3wM14TZsXgH\ntRW1dOvdjcKawjRdDZH0UqlEUmb0vonc9NxUDvvfIyg6pgjoRtFhtXTvW8RL/7GEggISljta0+fQ\nPhROKaRoZBHkQ9HIIvpP6U/dA5GneSd6CvziRYv5/I8+3+IOf0GfEi+STQIlbjP7BKgCQkC9u49L\nZ1CSWDpv7J/MY7xq9lZRctQIwhhGHk4+JUeVsH/vXmqNuMvWExl8wmAqyisIhULk5+cz+ITBbHpg\nE5B48FFzsaUzaUuP+1R33xFkx3R+JJf4ZYG83hOSuv6JZm8kUlTck8q1lfQ6vh8ARgGVa3fSvbgY\nzKh4s4rSE5rc9/rNKroVlySszffoM5j68gJ6DDyEql019OhVTH25UdJnEKteiWzf9kYlpUc3GXx8\ntzLwdpFMm3hG8H3TUipJ9wBYVxevLDB3UXK97nizN47+WuuJu6xwLQADBhxO2a/fZugPCuhxTCnV\n71RS9usyhg44lvFHn8fSW+Zi1wyn17GlVL1dSdktm5l88uyE4yLTT72Bu/54LYWT8igeNZqdr62j\n7vEwl589j9H7JjL91BtYuGA2hVeUUHpkXyo/2MW223cw88y5gbaL5JKgiduBZ8zMgfnuvuDgHcxs\nFjAL4Ie3/JDJl01OXZTSTDo/9jfehc+hrraOwm6FzcoZ8ZLc+C/C/SXdCb9XwIbrP8BDjuUbeXtL\n8CO7M33a9Xz66Qe89L//TKi+nvyCAr507EVMn3Z9wrhOOvEcljz2W9779d/w8Hosr5Cjhn2Fk048\np3E7wCO3/ZLX1z/J5w4/kZlnzW2xfeldt/DxttUMPHQkM888sF0klwRN3F919zIzOxR41szec/dm\nn8ejyXwBwNxnX3B9/Eytpp9i4s1JTva6dysuoeLNKnp8oQfhMNTVhah+q5puxSUtknasksz0753P\n/FvLGDZrTLOHDZ9y+vk8tvaXfLD/74y5+Yv0PLKUPR9U8sFtf+extb/kqHHxPyms+X8r+Kj6JYZf\nO57ikf2o+WQnH931En9a9XPGTjyN0fsmctKJ57D+o3f5ZK0x9ojzWyTlk048R4laOoVAidvdy6Jf\nt5nZEmA80GohVR89U+vgBJnoSezJXP9JJ/+ApfPmMuh7g+lxTF8q39zFlt+WM/nk2TH3P7gs1loZ\n5837IkvL+04sYcN966jaWEWvw3oxYOIA3nxpMTOuip+4f/u/7qb0ywPY8eS71G2rpvDQHpR+eQAv\nPH43YyeeBkBl5Q5WrHiCvn1vZcWKqzjrrJmUlvZr1k5l5Q5uueVKrrlmQYttIrkiYeI2sx5AnrtX\nRb8/E7gx7ZFJM0170nm9J3Dq6fDXWxfz7taPOWTgIE4960ryeic/q2T6tOtZ+/ZLvHf93/DwxsaS\nRGvljIa4GhJ4vDJOTVUN4YowA/95IINGDaH24318euen5G1OfMucbZu20K24O4OvGEzxkYdS80EN\n5beXU7tpf+M+y5YtIhQ6n+Li0ezZcz7Lli3k4ouvbdbOsmWLeP/9XTG3ieSKID3ugcASM2vY/353\nfzqtUUkzsXrQo4+byAXHXdf8zX3Jn6uycgc7t+1naM/15Of3JxTawc5t06is3Nmih9o0rlWvvMD4\nL8Yv43z4zocMnDGQ7kcWY9aN7kcah844lK2/2JowrpK+JQy64sCT2ItO7E7+FXls+cWWxrhXrHiC\n4uLIDJji4hmsWDGtWa87SI9cJBck7Oq4+3p3HxN9HevuN3VEYJIZDb3W/Pz+ANHkHem9BhHvTnoF\n+QXkl+YT3g8ehvB+yC/NpyC/ef9h987dXDftZ1R8VtH4XkF+AQV9CwjVhMAhVBOioG9B47ENcQN8\n+ulFgLWIe9myRdTVncOOHfnU1Z3T4neqrNzBDTdcRGXlzjZfN5GOpNu6SjNr1jxPKHQ/FRXjGl+h\n0P2sWfN8oOMnnD6BK6+6kpr7anjjsjeoua+GK6+KlHF6DxjG3vdrqdtq7Nu4l7qtxt73a+k9YBir\nXqHxdetNz/LWy/X893880/jeIYNHYtuKqNsepnr9Xuq2h7FtRRwyeGSzuLduHU9Nzcds3frFZnE3\n9LZra89l374QtbXnsmLFE82SdNMyikg205J3abSu6AVm/Cr2ICQQuBQT6056q16Bw4rG8fLCv9D/\nW8MpHjGAvRu2s+OeLfzDYac0ll0qK3fw1jNv07/P7bz1zFVc+vWfU1raLzIP+57ZDL3isMZ52GX3\nbGych33TTROprNzBnDnT6N//VkKhq7j55ocbSyENve3q6gIKCkZQXb2Bbt3Oaax1q4wiuUSJW5pp\nbfHUqlciiT2ZGSvbt22hfpNTNu8VnBBGPvn7erG9aEvjPg0JdteufHr2PJBYE83Tbji2tcHJNWue\np7r6PUKh28jLKyUcrqS6GtasOYqLL7420MCmSLZQqUQCCbIatmm54+AXwJw5d9K/32h68hJe+Ro9\neYn+/UYzZ85dQOJyxkknnsPYIyaRv/ckxh4xqVnSPjA4OQNoGJw8cOycOXfSp88wRoz4OyNHrmbE\niL/Tp88w5sy5K+GxItlGiVtSYvS+iQlfLcsVBc0GCRNtb17OaFmfjjeoGm97sgOyIh1NiTuHpWsW\nxMrlK5k9YzaXnHIJs2fMZuXylSlpN1KuuI1Q6Hzq6kYTCp1PdfVtjQOIDdvr68+ltnYE9fXnNtu+\nbNki9u8/k82b97N//5nNEmuiQdWG7bt2jWXDhoHs2jW2cXvQAVnNOpFsYe6e8kYff5zUNyotPPzw\nPB599AkuvPD8lNRj1xW9QLjiwJ0HD16Vmdd7QlI17obBw717T6Wi4jl69z6VkpLnGgcR420HZ86c\naezc+V/s29eToqI99Ov3o2YDkEEkc81Sfb1Fmpo0CQu6r3rcOSpe2aDpPm3tIQZ9Gnp72o6UQk5h\nz57nyM+/lT17nqOu7tSDSiWxtzf0tvftKwaGsm9fcYted6K4glyz1iRzrEiqKXHnqIa6bLduo1ut\nx7ZnXnL5xnJKR5c2e690dClbNm5p9l572l6z5nn27LmbUGgM7iFCoTHs2XNXs3JGa9vXrHmeiopb\ngQuBicCFVFTc2qKcES+uINesNckcK5JqStw5KMgsiPb2EBuWrDc730FPQ29v23Pm3EnfviMZMeLf\nGTnyGEaM+Hf69h3ZOKsk3varr/6/hMOFwL2YvQzcSzhcyNVX/ypQXMnMHNGsE8k2Stw5KMgsiPb2\nEOMtWU+27aAzP0KhOtavP5tQKNS4ff78HxEOTwFGAUXAKMLhKcyff12L9mPFlczMEc06kWyjxJ2D\nEs2CSKaH2NqS9YaVkMm0HXTmR3n5OMLhLZSXn9i4/f33X8X9XuAM3MdEv97L+++/GiiuZJbyJ3sb\nAJFU06ySTujhh+fx7LPQs+eBmQ979szjjDOIOxtiXdELcRfarHoF3rxndbvaDqqsbB3f/e55mD2C\n+z/yu989xdChRyb8ndr7O4tkC80q6eLS2UNM95znSElkOqFQCeHw9MZSSMN5d+48no8+6svOncfH\n7K2rVyxdgXrc0ijI0+GDzuNuz5znht52OPwo4XAheXl15OVd2NjrBrj++imsWbORsWMP48YblwRq\nVyQXtKXHrZtMSaNUPXKuvXfaa+htuxdiNgL3DY297htvXEJZ2TrWrn2LwsJHWLv2Hykr+6AxoYt0\nJSqVSMq194EFH3zwOu534H4u7v8Q/XoHH3zwOtB6GUWkq1HilpRK5oEF8+evZsiQoxk16mWOOOIN\nRo16mSFDjmb+/Fcbe9swnXA4BExn7dq3KCv7oGN/QZEsEDhxm1m+mb1uZk+kMyAJLtEAYDI3RWpv\n20Hu8Ld8+WPU1f2I5csfC3yHv5ZllEL1uqXLakuP+/vAu+kKRNou0bLzZB7F1d62D9wBcBLh8FcJ\nhSa1uMNfVdXXqK0dQVXV1wLf4S9RGUWkKwk0OGlmw4DzgJsATYrNAokGAJN5FFcybc+Zcydz5kzj\n0EMfanxKfG3ttMYHFixf/hj79t1MQcEI9u07l+XL5zQef9NNS+PGNGfONLp1a95u5M6BIl1L0B73\nr4HrgHBrO5jZLDNbbWarn356QUqCk9YlWnae7A2V4g0uBllaHmvZekNvG0ZiVgSMbNHrTvT7atm5\nSIDEbWbnA9vc/dV4+7n7Ancf5+7jzj57VsoClJYSLe9OxQ2VWhtcDLq0PNay9VdffZaamrtwP5dQ\naBzu51JTcxevvvpswri0wEbkgCClkq8Ak83sXCJ39yk1s3vd/dL0hiatidf7bPrg29a2J2o73tPQ\nE7V9001LGxfSFBYuwv0fufHGBxqXrX/22RktlqWfdFLi3zleGUWkq0nY43b3H7v7MHcfCUwHVihp\nZ1bQmzW1Z1l6osHFIG1HZoB8k7y84wmHv9li2bp6zSLJadOSdzM7BZjt7ufH209L3nNDrGXp8QYB\ngwxuHrhJ1Ivk5Q0mHC7H/eRmy9ZFpKW03WTK3Z9PlLQlN7T20IFkBwEP9LYHA0ST9zc131okhbRy\nsotqeL7jjh0/afbcx2TLGZF51XdQVze08QWaby2SSrrJVBfU0NuuqzuVfft20b07rFjxBGedNTPp\nQcAHHvgkJTGKSOvU4+6CEj1tPdOSWaov0hUocXdBiZ62nmnJLNUX6QqUuLugRE9bz6T2PkFepCtR\n4s6wTJQFUrF8PF1xJ7NUX6SrUOLOsEyUBVKxECYdcSezVF+kK9GskgxK5g5+yUh25ki64k5mqb5I\nV6IedwblalkgXXFrSbxIMOpxZ8iBssBDQENZYFqH9brbK51x60ZSIsGox50huXp/6VyNW6QzUeLO\nkI4oC6TjmZQqZ4hknkolGdIRZYGmMz9iDe4l2h6LyhkimacedyeVaCGLFrqI5C4l7k4qnc+kFJHM\nUuLuhNL5TEoRyTwl7k4o0cwPzQwRyW0anOyEIjM/PqWi4v6D3h/CxRdfm3C7iGS3Nj1zMig9c1JE\npG1S+sxJMysys1Vm9oaZvW1mNyQXnuQCPcxAJHsFqXHvB05z9zHAWOBsM5uQ3rAk0/QwA5HslTBx\ne8Se6I/QN+38AAALAklEQVSF0ZdKIZ2Y5niLZLdAs0rMLN/M1gDbgGfd/eUY+8wys9Vmtvrppxek\nOk7pQJrjLZLdAiVudw+5+1hgGDDezI6Lsc8Cdx/n7uPOPntWquOUDqI53iLZr03zuN19N/AccHZ6\nwpFM0xxvkewXZFbJADPrE/2+GDgDeC/dgUlm6O5/ItkvyAKcwcBCM8snkugfcvcn0huWZIru/ieS\n/RImbnd/EzihA2IREZEAtORdOs6qVZmOQCR7TRofeFclbukYq1YxafzWTEch0ino7oAiIjlGiVtE\nJMcocYuI5BglbhGRHKPELSKSY5S4RURyjBK3iEiOUeIWEckxStwiIjlGiVtEJMcocYuI5BglbhGR\nHKPELSKSY5S4RURyjBK3iEiOSc/9uHXDfBGRtEmYuM1sOLAIGAg4sMDdfxPvGN0wX0QkfYL0uOuB\nH7r7a2bWC3jVzJ5193fSHJuIiMSQsMbt7uXu/lr0+yrgXWBougMTEZHY2lTjNrORRJ74/nI6ghHJ\nRqddey2VFRUt3i/t3ZsV8+Z12nNL9gqcuM2sJ/Bn4AfuXhlj+yxgFsD8q69m1tlnpyxIkUyqrKhg\nde/eLd4fFyOhdqZzS/YKlLjNrJBI0r7P3R+JtY+7LwAWAPD4456qAEVEpLmENW4zM+AO4F1312cz\nEZEMC7IA5yvAt4DTzGxN9HVumuMSEZFWJCyVuPtfAeuAWEREJID0rJwUySGJZm6UV1YydOfOlgcW\nFqY9tnjn1oyTrkuJW7q8RDM3BpeWZmxmR7xza8ZJ16WbTImI5Bj1uCUnJCoLJLM9kY+3b2fM9u0t\n3t8cMLZkbN61i3G7dsU897C+fZNqW3KXErfkhERlgWS3x1MIvGEtx+cHuSfddiL54TCrY9TSh9bV\nJd225C4lbukUPtq+nSExesU1KWh7PzDUW64p2x/9unXXLt6J0SsOco/MfhdcQGGMtuvM2PnYY4TC\nYd7Zv7/F9lCAthNJ1ycFDZqmnxK3dArdgU9jvD8owLGlvXvH7B03lFG6A5tiHDck+jUcDnNMjF5x\nOECvuNCdLXF68wCjWjk2UdyJpOuTggZN00+JW7q8IL3ATI3i5+flURzjj0J+XZ16r12YErdkjXgf\nsddv387xcQYIE4k3yBfko328kksoHGZvK+WMXC0b5GrcXYUSt2SNeB+xC4EHYxxzapPv4yXXQiKP\ncTrYWQnO26BbnLZrgYmtvJ9s2SCUl8e4UMuKdigvvZ8BVO7IbkrckhMMOCZGLdia1IKL4xw/sG9f\njomRiAYGTET5cbZ1A1YnqFO317C+fZVApQUl7k6oM37MdeCdGImw4Z39xB6IbChgbN6+nTdaKbUM\nGzAg7rn3E/uRT/ubfI2VpFsWT1qqM4t5bF30D0G8Achk/53jtR2r3aCSHTSVxJS4O6HO+DHXgGNa\neR9g19KlCdsYE6NXTIBecUleHmVx5lJ3B7bEOC7IjJadjz0Wd3u8BDzuX/4lqX/nRG23V652DnKJ\nlryLiOQY9bgla8T7iF27fTvHxzimNmDbtcCYGL3r2gTnBaCwMPZKxSa98L2tnDeTZYNkSikqd2Q3\nJW7JGuksC3Sj9WXriZJY2cMPx92en5dHSRbOtU6mZKZyR3ZT4packMyycpHORom7E+qMH3PrgBmt\nvB/0+FilkpTcqilAKSUdOuO/swSjxN0JdcaPucnOZx41YEDaZtokKqWkS2f8d5ZgEiZuM7sTOB/Y\n5u7HpT8kkc6jM86pl8wL0uO+G/g9sVcMi+SETJUVMjmnXqWUzivIU95fMLOR6Q9FJH26Yu+2K/7O\nXUXKatxmNguYBTD/6quZdfbZqWpaRL1HkSZSlrjdfQGwAIDHH0/+7joiTaj3KHKAlryLiOQYTQcU\nSSOVeCQdgkwHfAA4BehvZpuBn7n7HekOTKQzUIlH0iHIrJJ/6ohAREQkGNW4RURyjBK3iEiOUeIW\nEckxStwiIjlGiVtEJMcocYuI5BglbhGRHKPELSKSY5S4RURyjBK3iEiOUeIWEckxStwiIjlGiVtE\nJMcocYuI5BglbhGRHKPELSKSY5S4RURyjBK3iEiOUeIWEckxgRK3mZ1tZuvM7EMz+1G6gxIRkdYl\nTNxmlg/8ATgHOAb4JzM7Jt2BiYhIbEF63OOBD919vbvXAg8CF6Q3LBERaU1BgH2GApua/LwZ+IeD\ndzKzWcCs6I/3uvu3kg8vtcxslrsvyHQcB1NcbZOtcUH2xqa42iZb42qQssFJd1/g7uPcfRxwdKra\nTbFZiXfJCMXVNtkaF2RvbIqrbbI1LiBY4i4Dhjf5eVj0PRERyYAgifsV4EgzG2Vm3YDpwNL0hiUi\nIq1JWON293oz+zdgGZAP3Onubyc4LFtrQ4qrbRRX22VrbIqrbbI1LgDM3TMdg4iItIFWToqI5Bgl\nbhGRHNPuxG1md5rZNjNb28p2M7PfRpfJv2lmJ7Y/zJTGdYqZVZjZmujr+g6Ka7iZPWdm75jZ22b2\n/Rj7dPg1CxhXh18zMysys1Vm9kY0rhti7NPdzP4UvV4vm9nILInrMjPb3uR6XZHuuJqcO9/MXjez\nJ2Js6/Dr1YbYMnLNzOwTM3sres7VMbZnJI8l5O7tegETgROBta1sPxd4CjBgAvBye8+V4rhOAZ7o\niFgOOu9g4MTo972A94FjMn3NAsbV4dcseg16Rr8vBF4GJhy0z9XAH6PfTwf+lCVxXQb8vqP/j0XP\nfS1wf6x/r0xcrzbElpFrBnwC9I+zPSN5LNGr3T1ud38B+CzOLhcAizxiJdDHzAa393wpjCsj3L3c\n3V+Lfl8FvEtkVWpTHX7NAsbV4aLXYE/0x8Lo6+CR9AuAhdHvFwNfNzPLgrgywsyGAecBt7eyS4df\nrzbElq0ykscSSWeNO9ZS+YwnhKgvRT/qPmVmx3b0yaMfUU8g0ltrKqPXLE5ckIFrFv1ovQbYBjzr\n7q1eL3evByqAflkQF8BF0Y/Wi81seIzt6fBr4Dog3Mr2jFyvqESxQWaumQPPmNmrFrltx8GyMo91\nxcHJ14AR7j4G+B3waEee3Mx6An8GfuDulR157ngSxJWRa+buIXcfS2S17ngzO64jzptIgLgeB0a6\n+/HAsxzo5aaNmZ0PbHP3V9N9rrYKGFuHX7Oor7r7iUTufvqvZjaxg86blHQm7qxcKu/ulQ0fdd39\nL0ChmfXviHObWSGR5Hifuz8SY5eMXLNEcWXymkXPuRt4Djj7oE2N18vMCoDewM5Mx+XuO919f/TH\n24GTOiCcrwCTzewTInfwPM3M7j1on0xdr4SxZeia4e5l0a/bgCVE7obaVFbmsXQm7qXAjOio7ASg\nwt3L03i+QMxsUENdz8zGE7kGaf/PGz3nHcC77j6vld06/JoFiSsT18zMBphZn+j3xcAZwHsH7bYU\nmBn9fiqwwqMjSpmM66Aa6GQi4wZp5e4/dvdh7j6SyMDjCne/9KDdOvx6BY0tE9fMzHqYWa+G74Ez\ngYNno2VlHgtyW9eYzOwBIrMN+pvZZuBnRAZqcPc/An8hMiL7IbAXuDzZYFMU11TgKjOrB2qA6R3x\nn5dIr+NbwFvR+ijAT4DDmsSWiWsWJK5MXLPBwEKLPMgjD3jI3Z8wsxuB1e6+lMgfnHvM7EMiA9LT\n0xxT0Li+Z2aTgfpoXJd1QFwxZcH1ChpbJq7ZQGBJtE9SANzv7k+b2Xcgs3ksES15FxHJMV1xcFJE\nJKcpcYuI5BglbhGRHKPELSKSY5S4RURyjBK3iEiOUeIWEckx/x/sbx6za+aFuQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1444325588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"The accuracy of using AdaBoost technique is:\",scores_ab.mean())\n",
    "fig = plot_decision_regions(X=X, y=y, clf=ab)\n",
    "plt.title(\"AdaBoost\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Boosting\n",
    "\n",
    "**(Gonzalo)**\n",
    "\n",
    "Gradient Tree Boosting or Gradient Boosted Regression Trees (GBRT) is a generalization of boosting to arbitrary differentiable loss functions. GBRT is an accurate and effective off-the-shelf procedure that can be used for both regression and classification problems.\n",
    "\n",
    "In Gradient Boosting each model is weighted by the residual of the model trained before. \n",
    "\n",
    "*Note: Extreme gradient boosting technique, the famous classifier winner of the majority of competitions) is just a simple random projection followed by a non-linearity and a straight forward least squares optimization.*\n",
    "\n",
    "![alt text](algo_gradient_boosting.png)\n",
    "<p style=\"text-align: center;\">Figure 7. Gradient Boosting algorithm.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy of using Gradient Boosting technique is: 0.90114379085\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f13fe90d4a8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEICAYAAAB/Dx7IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VPWd//HXJ8kEEiAEAbnfvBRvVVSWYi9U6x1FZUXL\negHdtVTtdq3KSi+/3uzarq3Fuq1rwStq1VUURauiLLpaKyoqijcUkUswXIUkkECSmc/vj5mEhExm\nJpmZzEzyfj4eeSSZc873fObA45PvfL7f7znm7oiISO7Iy3QAIiLSNkrcIiI5RolbRCTHKHGLiOQY\nJW4RkRyjxC0ikmOUuCXjzGyNmZ0U+fnHZnZHpmPqSGb2ZzP7aabjkNyhxC0xmdlUM3vNzHaZ2ebI\nz1eamaXjfO7+a3e/LNl2zGykmbmZFcTY5xdmVmdmOyNfH5rZucmeO05cl5jZ35q+5u6Xu/uv0nle\n6VyUuKVVZnYtcAvwO2AgMAC4HPgaUNjKMfkdFmBq/I+793T3nsAPgPvNbECmgxKJRYlbojKz3sD1\nwJXuPt/dqzzsbXe/0N33RPa7x8xuM7OnzWwXcIKZnWFmb5tZpZmtN7Nf7NP2xWa21sy2mdlP9tn2\nCzO7v8nv483s72a2w8zeMbPjm2x70cx+ZWavmFmVmT1nZv0im1+KfN8R6U0fF+89u/sioAo4sMk5\nvmNmq8zsCzNbaGaDm2z7qpm9YWYVke9fbbLtEjNbHYnrMzO70MwOBf4MHBeJaUeTa/gfkZ+PN7My\nM7s28gmn3MwubdJuXzN7MnJt3zCz/9i3By+dnxK3tOY4oBvwRAL7XgDcAPQC/gbsAqYBpcAZwBVm\ndg6AmR0G3AZcDAwG+gJDozVqZkOAvwL/AewHzAQeNbP++5z7UmB/wp8CZkZenxD5XhrpUb8a6w1Y\n2BmRNj6IvPYt4DfA+cAgYC3wUGTbfpHY/ivyHmYDf40k1h6R1093917AV4Hl7v4h4U8sr0ZiKm0l\nnIFAb2AI8C/ArWbWJ7LtVsLXdyAwPfIlXYwSt7SmH7DV3esbXmjS860xswlN9n3C3V9x95C773b3\nF919ReT3d4EHgW9G9p0CPOXuL0V67T8FQq3EcBHwtLs/HWnreWAZMLHJPne7+8fuXgM8DIxp4/s8\nP9Lz3QksBH7t7jsi2y4E7nL3tyKx/ohwb3kk4T9In7j7fe5e7+4PAh8BkyLHhoAjzKzI3cvd/f02\nxFQHXO/ude7+dCS20ZEy1LnAz9292t0/AOa18f1KJ6DELa3ZBvRrOrjn7l+N9BK30fz/zvqmB5rZ\nV8zsBTPbYmYVhHuZDSWMwU33d/ddkfaiGQGcF/ljsSOSYL9OuPfbYGOTn6uBnm15k8DD7l7q7j0I\nl0immdl3m8S6tkmsOyOxDtl3W8RaYEjkPX2b8PsuN7O/mtkhbYhpW9M/mOx9X/2BAppf72bXXroG\nJW5pzavAHuDsBPbd9xaTDxDuvQ5z996E67oNs1DKgWENO5pZMeFSQzTrgfsiibXhq4e7/2c7Yop/\ngPsa4Bn29po/J/zHoyHWHpFYN+y7LWJ4ZBvuvsjdTyb8R+Yj4Pb2xtXEFqCe5qWlYa3sK52YErdE\nFSkX/BL4bzObYma9zCzPzMYAPeIc3gv4wt13m9k4wnXoBvOBM83s62ZWSHgAtLX/h/cDk8zsVDPL\nN7PukcG7qDXxfWwhXK44IIF9AYi0exrQUNZ4ELjUzMaYWTfg18BrkQT/NPAlM7vAzArM7NvAYcBT\nZjbAzM6OJPo9hEsdDeWgTcDQyHtvE3cPAo8BvzCz4kgvflpb25Hcp8QtrXL33wLXANcRTjibgDnA\nLODvMQ69ErjezKqAnxGuPTe0+T7wPcK98nJgO1DWyvnXE+7x/5hwIl4P/DsJ/L9192rCA6avRMos\n41vZ9duRGR47gTeAVwj/wcLdFxOuwT8aifVAYGpk2zbgTOBawuWT64Az3X1rJL5rCPfKvyBc378i\ncr4lhP8wbDSzrfHeRxT/SnjgciNwH+E/Lnva0Y7kMNODFERyl5ndCAx0d80u6ULU4xbJIWZ2iJkd\nGZm+OI7wdMEFmY5LOlary4FFJCv1IlweGUy4dPV7EptrL52ISiUiIjlGpRIRkRyTllLJSzteUjde\nRKQNJpROSPiOm+pxi4jkGCVuEZEco8QtIpJjlLhFRHKM5nGLSOcRgkBtgPxQPkZanq6XFMcJ5gWp\nK6xLqtusxC0inUagNkCPwh5YwLD0PBY1Ke6O1zm7andR172u3e2oVCIinUZ+KD9rkzaAmWEBIz+U\n3KNZlbhFpNMwsjdpNzCzpMs4StwiIjlGiVtEJMX+9vzfmHT0JCYeNZE7fn9HyttX4hYRSaFgMMgN\n197Afz/23zzxxhM8M/8ZPv3o05SeQ7NKRKRL+s6EqVRv/aLF68X99uP2lx5qd7srlq1g+AHDGTYq\n/DjQ0889nReeeoEDDzmw3W3uS4lbRLqk6q1fsLRfnxavj4+SzNtic/lmBg4Z2Pj7gCEDeHfZu0m1\nuS+VSkREcowSt4hICu0/aH82btjY+PumDZsYMGhASs+hxC0ikkJHHHsEaz9dS9maMupq63jm0Wc4\n/ozjU3oO1bhFRFKooKCAH9/0Yy4/53KCoSCTL57MQYcelNpzpLQ1EZEcUdxvv6gDkcX99ku67Qmn\nTmDCqROSbqc1Stwi0iUlM+Uv01TjFhHJMXETt5mNNrPlTb4qzewHHRGciIi0FLdU4u4rgTEAZpYP\nbAAWpDkuERFpRVtLJScCn7r72nQEIyIi8bV1cHIq8GA6ApHcsXTxUubfO5/ydeUMGj6IKdOmMP6k\n8ZkOS6TLSLjHbWaFwFnAI61sn2Fmy8xs2cJ7FqYqPskySxcvZc5tcyi6sIgx94yh6MIi5tw2h6WL\nl2Y6NJGs8NMrfso3R32TyeMmp+0cbSmVnA685e6bom1097nuPtbdx551yVmpiU6yzvx75zN0xlBK\nDy8lryCP0sNLGTpjKPPvnZ/p0ESywtkXns1tC25L6znakrj/CZVJurzydeWUjC5p9lrJ6BI2rtvY\nyhEi2W371u1cee417Ni2IyXtjf36WHr36Z2StlqTUOI2sx7AycBjaY1Gst6g4YOoXFnZ7LXKlZUM\nHD6wlSNEstv8e55kxbLdPHJ37pR4E0rc7r7L3fu6e0W6A5LsNmXaFMrmlrHj/R2E6kPseH8HZXPL\nmDJtSqZDE2mz7Vu3s/CBV+g74I8sfOCVlPW6001L3qVNGmaPzL93Pp+t+4yBwwfy3Su+m/CsEs1I\naTtds/SZf8+T1NdPorjnl6jcMYlH7l7Id2ZOy3RYcSlxS5uNP2l8uxJHw4yUoTOGMmb0GCpXVjLn\ntjmNbUpLumbp09DbLu4ZvmdJcc8LWfjAVM679CxK+5ZmOLrYdK8S6TCakdJ2umbp09DbLijoB0BB\nQT/q6yclXeu+7tLruOjEi1jzyRpOHH0ij81L/dCgetzSYcrXlTNm9Jhmr5WMLuGzdZ8ldHxXLBkk\ne82kdUtfWEawfhvbtz64z+t9kyqX/Pbu3yYbWlxK3NJhGmaklB6+92NoojNSumrJIJlrJrHd+df/\nynQI7aZSiXSYZGakdNWSgWbxSDTqcUuHSWZGSlctGSQ7i0c6JyVu6VDtnZHSlUsG7b1m0nmpVCI5\noSuXDJYuXsrMaTO58PgLmTltpm7oJepxS27oqiWDrjooK7GZu6e80Zd2vJT6RkW6oJnTZlJ0YVGz\nEtGO93dQ85cabrr3pgxGlp2KqosoKinKaAwbyzby4xk/ZtvmbZgZUy6dwkVXXtRsn5rKGmqKa5q9\nNqF0giV6DvW4Jat0xbnasXTVQdlcll+Qz8xfz+SwMYexq2oX3/7GtznuW8dx4CEHpuwcStySNVQW\naKkrD8p2hJeff5n77rmPz9d+zuARg7n4kov5xsnfSKrN/gP7039gfwB69OrBqNGj2PT5ppQmbg1O\nStboqnO1Y+nKg7Lp9vLzLzP71tl0v6A7R887mu4XdGf2rbN5+fmXU3aODWs38NG7H3Hk2CNT1iao\nxy1ZRGWBlrrqoGxHuO+e+xj2nWH0OaIPQPj7d8KvJ9vrBqjeWc3VF13NrP+cRc+Snkm315QSt2QN\nlQWi0zzu9Ph87eccfcjRzV7rfUhvVq9dnXTbdXV1XH3R1Zxx/hmcdPZJSbe3LyVu6VCxBh+nTJsS\nrmnPCPe0K1dWUja3jO9e8d24x2azXI27sxs8YjAVH1U09rgBKj6qYPCIwUm16+78/Hs/54DRBzD9\n+9OTDTOq/F/84hcpb3Tt7rWpb1RyXsPgY99L+zLishGEBod4Yc4LDOw7kKEHDGXoAUMZ2Hcg797/\nLp/O+5Tu67oz7bJpjD9pfNxjs1Wuxp2rAnUBAt0CCe1bWlLKoj8vonBYId3268aOD3aw/vb1XD7j\nckYcOKLdMbz96tvcOOtGaqpreOSuR3j4zocZOHQgIw7a22b9nnrqA/XNjhvRfcQvEz2H5nFLh0lm\nTnKuzmfO1bhzVVvncadjVkkiOmQet5mVAncARwAO/LO7v9qGOEWSGnxMxcBlrJJFusoZGnDNbt84\n+RsdkqhTLdEa9y3As+4+xcwKgeI0xiSdVDKDj8kOXMaaIw6kbf64BlwlHeLO4zaz3sAE4E4Ad691\n99x4FLJklWTmJCc7nznWHPF0zh/XPOyO5TjpKP+mkrvjJBdjIj3uUcAW4G4zOwp4E7jK3Xc13cnM\nZgAzAK69+VrOuuSspAKTzieZOcnjTxrPyuUrWfCTBVRXVVPcq5jJUycn3CMuX1fO4K2Dee3a16jZ\nUEPRkCJGnDWCjes24njayhmah92xgnlBvM4hAGYJl4w7jLvjdU4wL5hUO3EHJ81sLLAU+Jq7v2Zm\ntwCV7v7T1o7R4KSkWtNSx75TBRNJgpedfhlbbAtDvjeEHl/qwa6Pd7Hh1g309/6U9i/VAGJnEYJA\nbYD8UD5GFiZuwkm7rrCuRb0j1YOTZUCZu78W+X0+8MNETyCSCk3LGUD4+4zw64kkbss3+k3uR9Go\nIizfKBpVRL9z+2ELLO78cckheVDXvY466jIdSVrFTdzuvtHM1pvZaHdfCZwIfJD+0CRTsnHBSPm6\ncopfKuaNH71B/a56CnoUMOz0YdSsq4l/MOHlxwePO5iKLyrYU7uHQGGAYeOGseq+VSpnSM5JdFbJ\n94G/RGaUrAYuTV9IkknZeoe+UG2IshfLGPaTYfQ4oge73ttF2ewySr00/sGEZ3cEPw8y9PC9i152\nvL+jcXaHlpVLLkno7oDuvtzdx7r7ke5+jrtvT3dgkhnZeoe+ql1VDPm3IfQ8qid5gTx6HtWTIf82\nhKpdVQkdr9kd0pnoXiXSTLYuGKndXUvvI3sTqg8R9CB5lkfvI3uzbve6hI6PNytl3k3zWPBQ823T\nZ6bnPhMiyVLilmaydcFIca9idn2wi5JjShpfq3yrkuJeia0FW7p4KS+++iJfvuHLjQOQL859kdGL\nR7Ny+Uoefvxhhv1oGL2+3IuqFVU8fPPDAErekpWUuKWZRGZYJDN4Ga9n21rbk6dO5uGbH2bY1XuT\n6/qb13P+1PMTiivWrJRVH6xi2I+GNf5RKDmmhGFXD2PBbxY0xpaNA7bSdSlxSzPxZlgkM3g576Z5\nMXu2sdpuSKALfrOA1VWrKe5VzPlTz2+WWGPFFasEVF1VTa8v92q2rdeXe7G6anXS71kkHXR3QGmT\nZO52d87Ycxj4o4Etyh0bf7ORx5c9nta7B8bavuqDVWmLSyRRbVmAo2dOSpuUryunZHRJs9dKRpew\ncd3GuMe21rOtrqpOqO15N83jnLHncMroUzhn7DnMu2les7jyB+dTtqaMzz7+jLI1ZeQPzm88Ntas\nkslTJ7P+5vVUvlWJ1zmVb1Wy/ub1TJ46Oen3LJIOKpVImyQzeFncq5iqFVXNerZVK6oaBxhjtR2v\nzFLcs5j1r6+n9LhSirsXE9wdZP2r6ynqGb43c6wSUMO21sow2TpgK12XetzSJsnMh47Xs43V9oKH\nFjDs6vAAogVs7wDiQwsA8KCz9dGt1HxWgwedms9q2ProVjy4t2o3/qTx3HTvTdz/4v3cdO9NzerT\n02dO5/Flj/Pcyud4fNnjzQZMNQdcso163NImySwPjzfAGKvteAOINTU17P+1/Vn909XNl8S/ktiS\n+HjvOZk7E4qkmhK3tFkyy8Onz5wec250a23HK7MUFRWx+e3NHPCrA5rf/a+of7vibCrWHHAlb8kE\nlUokJ8Qrs1i+0e/cKHf/y0/+1p7ZehsA6brU45aUS8dilXhlllh3/0s2rmy9DYB0XUrcklLpXKwS\nq8wS7+5/ycSlWSWSbVQqkZRKZ1kh1jzueDM/kolLs0ok26jHLSmVrrJCvHnc8Wa7JBOXHrQg2UaJ\nW1IqXWWFBQ8tiHsjqFizXZKNSw9akGyixC0pNWXaFG782Y3UFddRX1lPQUkBgeoAs66fBbT/vtfV\nVdUEdwX5+LqPqS2vpXBQIf0n9W9cLg+xBx+nTJvCLbNvodd5vQgMDVBXVkfVI1Vcdc1V6bkQImmk\nxC0ptXL5Svawh/2/PYBuwwrZs76WzXdvYuXylUnd9zqQH6D8gXKG/OsQir9UTPXH1Wz40wYC+QEg\nscHHUE2IrfO3UltRS2HvQgI1gTReCZH0UeKWlGooaXQ/tBizQroPr6XbfoUs+E14aXq8ckdrSvcv\nJTA5QPeR3SEfuo/sTr/J/ah7MPw073hPgZ9/73y+9MMvtbjDX6JPiRfJJgklbjNbA1QBQaDe3cem\nMyiJL5039k/mMV7VVdUUH9KDEHmAAfkUH9KjsaQRa9l6PIOOHkRFeQXBYJD8/HwGHT2I9Q+uB+IP\nPmoutnQmbelxn+DuW9MWiSQsnXOlY83eOPSb8ZN3YfciKldU0vPIfoRv9V5A5YqtFHYvoiBgUZet\nFxYV8/obsdsdNHwQBdsL6NNvP77YUkOf/YrI22oMHD6Q19+AHqWD2PxOJSWHNhl8/LCS4tLEtotk\n2oSTE99XpZIcFK8skIxYszcO/eZ0Ru+eEPP4UX3H88kfXmboDwL0OKyEXR9UsuEPnzOk/+FMOO2I\nqI8f++oJ58dsd2X3l5gybQq33Xob+ad2p3jUaMrfWElw0W6u+N4VAEw94ZfMmzuTwGXFlBzch8pP\ntrP5jq1MP+UmRu+eEHe7SC5JNHE78JyZOTDH3efuu4OZzQBmAFx787WcdclZqYtSmknnx/7Gu/A5\n1NXWESgMtKmc4fUBQmu7seZnn+BBx/KNvKpe+CHdmD5zOmWflvHS/3uJ+rp6CgIFTDh+AidfMB12\nx253/EnjefjWR1kx+0M8tAbLC/Dlww9l/Enjef0NOPaY0wF47Pbf8vbqv3LgAccw/dSbGl9v+L7w\n7pv5bPMyBuw/kumn7N0ukksSTdxfd/cNZrY/8LyZfeTuLzXdIZLM54IeXZZu6VyC3XAXvh5f7kEo\nBMH6ILtW7Er4aer/eO4VzHvuU4ZcNq6xZ7vhjnUcf+aZLF28lFWbVzHuD+Ma77K3au4qhi1byugj\nYvd6lyxYwsrylQy/dhzFB/SlevU2Vt69jCULltBz6LeAcHJe/emHrHnPGHPQmS2S8rHHnK5ELZ1C\nQkve3X1D5PtmYAEwLp1BSWzpXILdcBe+HW9V4PUF7Hirotld+OJZ+L83M+Sy4ZQe2je8tPzQvgy5\nbDh/WzSf+ffOp/grxay8cyVLpi5h5Z0rKf5KMX9bFH/Z+V233EOv4/qz5akPWfX/FrHlqQ/pdVx/\n7rrlnsZ9Kiu3smTJU/TpcxtLljxFZeW2Fu1UVm7ll788N+o2kVwRt8dtZj2APHevivx8CnB92iOT\nVqVzCfb0mdN555X3WPGzD/HQ+khJ4jCmz5ye0CDeps2fcdTB/9DstZKD+/DxpvfZtr6GUEWIAf88\ngIGjBlP72W4+v+tzgp/F7z9sXr+RwqJuDLpsEEUH70/NJzWU31FO7fo9jfssWnQvweCZFBWNZufO\nM1m0aB7nnXdNs3YWLbqXjz/eHnWbSK5IpFQyAFhgZg37P+Duz6Y1KokrXUuwd2zbweYyY9iAdyko\n6Et9/TY2l02l4osKoHfc4wfsP4rKT7ZTemjfxtcqP9nOfgMGsrlsFQOmDaDbwUWYFdLtYGP/afuz\n4Veb4rZb3KeYgZftfRJ792O6kX9ZHht/E35gb0Nvu6goPAOmqGgaS5acz6mnTqekpG+zfcI98iua\nbRPJJXG7Ou6+2t2Pinwd7u43dERgkhlPzXueYP0kCgrCCa2goC/B+kk8ec9zCR1/1olXs+GOdez4\ncFu4jPPhNjbcsY6vnzqFgvwC8kvyCe0BD0FoD+SX5JOf17z/EK2cUZBfQEGfAoI1QXAI1gQp6FNA\nQX742IbeNsDnn58LGMFguNfdYNGie6mrO52tW/Opqzu92bbWziuSjdIyHVDzYnPXkiffobr6Baqr\nH9zn9f340jfOi3t8a7M3eo7twf/9dSjVH++ke7A77tWYBdj9aS39Bg9t1ka0csZ+g0Zim6EuP8ju\nuj3kBwLY5u7sN2gkAMuXv0gw+DmbNv2JmppSNm36B3r02I/lywdz3nnXNPa2a2v/yO7dQQoLJ7Jk\nyfeb9bpVRpFcYe6pnwDy5JNoVom08IdbZvDauqfpd/Foikb0p2btFrbet5KvDJ/ID64KzzCtrNzK\nrFnnk59/G8HgFdx44yOUlPTlzbeeYd5zMxly2fBms1WaTulr7ViARx6ZzaJFQbZvPwuzEbivpU+f\nhZx6an5jYm/tWJGOMGkSCT9nTw9SkA6zZfNG6tc7G2a/wSdXPcmG2W9Qv97Zsnlj4z6tlTOOPeZ0\npp9yE5W37+b/pvyVytt3t5iH3VAuKSwc3aJMsnz5i+zadTvB4CRCoa8TDE5i167bWb78xbjHimQb\nJW7pMLNm3UW/vqPpyat45Vv05FX69R3NrFl3AzQpZ0xk9+4gtbUTm03rO/aY0xlz0CTyq49lzEGT\nmiXtvYOT04CGwcm9x86adRelpUMZMeLvjBy5jBEj/k5p6VBmzbo77rEi2UaJWzpMQ296164CCgpG\nsGtXQbNedbztseZpN/SY8/P7AZCf369ZzznW9njHimQbJe4clq5ZEG++9Qw//90pXP7vB/Pz353C\nm289k5J295YrzqSuLlySaFquaNheXz+R2toR1NdPbFHO2LPnFMrK9rBnzyktSiHB4ANUVIxt/AoG\nH2jWdjD4ANu3j2Ht2gFs3z6mcXu8Yxto1olkCw1O5rBHHpnN448/xTnnnJmyWRCJDAK2V8MAYHX1\nCVRUvEDv3idQXPxC40BgrO3gzJp1Ptu2/Se7d/eke/ed9O37wzYPIiZzzdJxvUUaaHCyC0jX8u7W\nlqwv/N+bk247XAo5np07XyA//zZ27nyBuroT9imVRN/e0NvevbsIGMLu3UUtet3x4krkmrUmmWNF\nUk2JO0clMgui6bzkRG3a/BklB/dp9lrJwX3YtHlN0m0vX/4iO3feQzB4FO5BgsGj2Lnz7mbljNa2\nL1/+IhUVtwHnABOAc6iouK1FOSNWXMnMHNGsE8kmStw5KJFZEO3tITYsWW92vk+2M2D/kUm3PWvW\nXfTpM5IRI37CyJGHMWLET+jTZ2TjrJJY26+88neEQgHgfsxeA+4nFApw5ZW/TyiuZGaOaNaJZBsl\n7hyUyCyI9vYQW1uyftaJVyfddqIzP4LBOlavPo1gMNi4fc6cHxIKTQZGAd2BUYRCk5kz57qE4kpm\n5ohmnUi20RNwclDD8u6Kigf2eb358u5YN1xqTbwHDiTTdry4G7aXl/+eUKg/5eXHUFo6iOXLB/Pp\np8txXwo8i3seEMJ9Kx9/HEgornjnTiZukY6mWSWd0COPzOb556Fnz71JZefO2Zx8MkknmnS2DbBh\nw0q+//0zMHsM93/kj398hiFDDo573nTHJZJumlXSxSU6Lzmdbbd3znO4JDKVYLCYUGhqYymk4bzb\nth3Jp5/2Ydu2I6PO007HexbJNupxS1q0Z85zQ287FHqcUChAXl4deXnnNPa6AX72s8ksX76OMWOG\nc/31C9L5FkQ6lHrcklHtnXXS0Nt2D0Tu4Bdo1uvesGEl7723gkDgbt57bwUbNnySzrchkrWUuCXl\n2vvAgk8+eRv3O3GfiPtXIt/v5JNP3gZaL6OIdDVK3JJS8e7wB60vkpkzZxmDBx/KqFGvcdBB7zBq\n1GsMHnwoc+a82djbhqmEQkFgqnrd0mUlnLjNLN/M3jazp9IZkCQu3gBgMjdFam/bidzhb/HiJ6ir\n+yGLFz+R8B3+4pVRRLqStvS4rwI+TFcg0nbxlp23Z1l6sm0n8sCCqqpvUls7gqqqbyZ8h794ZRSR\nriShBThmNhQ4A7gB0KTYLBDvieXJPNE8mbZnzbqLWbPOZ//9H470mLdSW3t+4wMLFi9+gt27b6Sg\nYAS7d09k8eJZjcffcMPCmDHNmnU+hYXN2w3fOVCka0m0x/0H4Dog1NoOZjbDzJaZ2bJnn52bkuCk\ndfGWnSd7Q6VYg4uJLC2Ptmy9obcNIzHrDoxs0euO93617FwkgcRtZmcCm939zVj7uftcdx/r7mNP\nO21GygKUluLd9CgVN1RqbXAxXtsN5Y7y8rGEQhspLz+msdzx5pvPU1NzN+4TCQbH4j6Rmpq7efPN\n5+PGpQU2InslUir5GnCWmU0kfHefEjO7390vSm9o0ppYvc/zzrsm7vZ4bTcfXFxLYeHpCbd9ww0L\nGxfSBAL34v6PXH/9g43L1r/44uQWy9KPPTb+e45VRhHpauL2uN39R+4+1N1HAlOBJUramZXoY7ra\nsyw93uBiIm2HZ4BcQF7ekYRCF7RYtq5es0hy2rTk3cyOB2a6+5mx9tOS99wQbVl6rEHARAY3994k\n6mXy8gYRCpXj/o1my9ZFpKW0LXl39xfjJW3JDa0tS092EHBvb3sQQCR5X6D51iIppJWTXVTD8x23\nbv1xs+c+JlvOCM+rvpO6uiGNX6D51iKppAcpdEENve26uhPYvXs73brBkiVPceqp05MeBHzwwTUp\niVFEWqdmar73AAAMTElEQVQedxcU72nrmZbMUn2RrkCJuwuK97T1TEtmqb5IV6DE3QXFe9p6JrX3\nXt4iXYkSd4ZloiyQiuXj6Yo7maX6Il2FEneGZaIskIqFMOmIO5ml+iJdiWaVZFAyd/BLRrIzR9IV\ndzJL9UW6EvW4MyhXywLpiltL4kUSox53huwtCzwMNJQFzu+wXnd7pTNu3UhKJDHqcWdIrt5fOlfj\nFulMlLgzpCPKAul4JqXKGSKZp1JJhnREWaDpzI9og3vxtkejcoZI5qnH3UnFW8iihS4iuUuJu5NK\n5zMpRSSzlLg7oXQ+k1JEMk+JuxOKN/NDM0NEcpsGJzuh8MyPz6moeGCf1wdz3nnXxN0uItmtTc+c\nTJSeOSki0jYpfeakmXU3s9fN7B0ze9/MfplceJIL9DADkeyVSI17D/Atdz8KGAOcZmbj0xuWZJoe\nZiCSveImbg/bGfk1EPlSKaQT0xxvkeyW0KwSM8s3s+XAZuB5d38tyj4zzGyZmS179tm5qY5TOpDm\neItkt4QSt7sH3X0MMBQYZ2ZHRNlnrruPdfexp502I9VxSgfRHG+R7NemedzuvgN4ATgtPeFIpmmO\nt0j2S2RWSX8zK438XAScDHyU7sAkM3T3P5Hsl8gCnEHAPDPLJ5zoH3b3p9IblmSK7v4nkv3iJm53\nfxc4ugNiERGRBOheJSIiOUaJW0Qkxyhxi4jkGCVuEZEco8QtIpJjlLhFRHKMEreISI5R4hYRyTFK\n3CIiOSY9z5x8/fW0NCsi0mlNGpfwrmlJ3JPGbUpHsyIigkolIiI5R4lbRCTHKHGLiOQYJW4RkRyj\nxC0ikmOUuEVEcowSt4hIjknkYcHDzOwFM/vAzN43s6s6IjAREYkukQU49cC17v6WmfUC3jSz5939\ngzTHJiIiUcTtcbt7ubu/Ffm5CvgQGJLuwEREJLo2LXk3s5GEn/j+WjqCEclG37rmGiorKlq8XtK7\nN0tmz+6055bslXDiNrOewKPAD9y9Msr2GcAMgDlXXsmM005LWZAimVRZUcGy3r1bvD42SkLtTOeW\n7JVQ4jazAOGk/Rd3fyzaPu4+F5gLwJNPeqoCFBGR5hKZVWLAncCH7q7PZiIiGZbIPO6vARcD3zKz\n5ZGviWmOS0REWhG3VOLufwOsA2IREZEEpOcJOCI5JN7MjfLKSoZs29bywEAg7bHFOrdmnHRdStzS\n5cWbuTGopCRjMztinVszTrou3atERCTHqMctOSFeWSCZ7fF8tmULR23Z0uL1sgRjS0bZ9u2M3b49\n6rmH9umTVNuSu5S4JSfEKwskuz2WAPCOtRyfH+iedNvx5IdCLItSSx9SV5d025K7lLilU/h0yxYG\nR+kV16Sg7T3AEG+5pmxP5Pum7dv5IEqveFMCbfc9+2wCUdquM2PbE08QDIX4YM+eFtuDCbQdT7o+\nKWjQNP2UuKVT6AZ8HuX1gQkcW9K7d9TecUMZpRuwPspxgyPfQ6EQh0XpFYcS6BUH3NkYozcPMKqV\nY+PFHU+6Pilo0DT9lLily0ukF5ipUfz8vDyKovxRyK+rU++1C1PilqwR6yP26i1bODLGAGE8sQb5\nEvloH6vkEgyFqG6lnJGrZYNcjburUOKWrBHrI3YAeCjKMSc0+TlWcg0A90Z5/dQ4521QGKPtWmBC\nK68nWzYI5uUxNtiyoh3MS+9nAJU7spsSt+QEAw6LUgu2JrXgohjHD+jTh8OiJKIBCSai/BjbCoFl\ncerU7TW0Tx8lUGlBibsT6owfcx34IEoibHhlD9EHIhsKGGVbtvBOK6WWof37xzz3HqI/8mlPk+/R\nknTL4klLdWZRj62L/CGINQCZ7L9zrLajtZuoZAdNJT4l7k6oM37MNeCwVl4H2L5wYdw2jorSKyaB\nXnFxXh4bYsyl7gZsjHJcIjNatj3xRMztsRLw2H/5l6T+neO13V652jnIJVryLiKSY9TjlqwR6yN2\n7ZYtHBnlmNoE264FjorSu66Nc14AAoHoKxWb9MKrWzlvJssGyZRSVO7IbkrckjXSWRYopPVl6/GS\n2IZHHom5PT8vj+IsnGudTMlM5Y7spsQtOSGZZeUinY0SdyfUGT/m1gHTWnk90eOjlUpScqumBEop\n6dAZ/50lMUrcnVBn/Jib7HzmUf37p22mTbxSSrp0xn9nSUzcxG1mdwFnApvd/Yj0hyTSeXTGOfWS\neYn0uO8B/kT0FcMiOSFTZYVMzqlXKaXzSuQp7y+Z2cj0hyKSPl2xd9sV33NXkbIat5nNAGYAzLny\nSmacdlqqmhZR71GkiZQlbnefC8wF4Mknk7+7jkgT6j2K7KUl7yIiOUbTAUXSSCUeSYdEpgM+CBwP\n9DOzMuDn7n5nugMT6QxU4pF0SGRWyT91RCAiIpIY1bhFRHKMEreISI5R4hYRyTFK3CIiOUaJW0Qk\nxyhxi4jkGCVuEZEco8QtIpJjlLhFRHKMEreISI5R4hYRyTFK3CIiOUaJW0Qkxyhxi4jkGCVuEZEc\no8QtIpJjlLhFRHKMEreISI5R4hYRyTEJJW4zO83MVprZKjP7YbqDEhGR1sVN3GaWD9wKnA4cBvyT\nmR2W7sBERCS6RHrc44BV7r7a3WuBh4Cz0xuWiIi0piCBfYYA65v8XgZ8Zd+dzGwGMCPy6/3ufnHy\n4aWWmc1w97mZjmNfiqttsjUuyN7YFFfbZGtcDVI2OOnuc919rLuPBQ5NVbspNiP+LhmhuNomW+OC\n7I1NcbVNtsYFJJa4NwDDmvw+NPKaiIhkQCKJ+w3gYDMbZWaFwFRgYXrDEhGR1sStcbt7vZn9K7AI\nyAfucvf34xyWrbUhxdU2iqvtsjU2xdU22RoXAObumY5BRETaQCsnRURyjBK3iEiOaXfiNrO7zGyz\nmb3XynYzs/+KLJN/18yOaX+YKY3reDOrMLPlka+fdVBcw8zsBTP7wMzeN7OrouzT4dcswbg6/JqZ\nWXcze93M3onE9cso+3Qzs/+JXK/XzGxklsR1iZltaXK9Lkt3XE3OnW9mb5vZU1G2dfj1akNsGblm\nZrbGzFZEzrksyvaM5LG43L1dX8AE4BjgvVa2TwSeAQwYD7zW3nOlOK7jgac6IpZ9zjsIOCbycy/g\nY+CwTF+zBOPq8GsWuQY9Iz8HgNeA8fvscyXw58jPU4H/yZK4LgH+1NH/xyLnvgZ4INq/VyauVxti\ny8g1A9YA/WJsz0gei/fV7h63u78EfBFjl7OBez1sKVBqZoPae74UxpUR7l7u7m9Ffq4CPiS8KrWp\nDr9mCcbV4SLXYGfk10Dka9+R9LOBeZGf5wMnmpllQVwZYWZDgTOAO1rZpcOvVxtiy1YZyWPxpLPG\nHW2pfMYTQsRxkY+6z5jZ4R198shH1KMJ99aayug1ixEXZOCaRT5aLwc2A8+7e6vXy93rgQqgbxbE\nBXBu5KP1fDMbFmV7OvwBuA4ItbI9I9crIl5skJlr5sBzZvamhW/bsa+szGNdcXDyLWCEux8F/BF4\nvCNPbmY9gUeBH7h7ZUeeO5Y4cWXkmrl70N3HEF6tO87MjuiI88aTQFxPAiPd/Ujgefb2ctPGzM4E\nNrv7m+k+V1slGFuHX7OIr7v7MYTvfvo9M5vQQedNSjoTd1YulXf3yoaPuu7+NBAws34dcW4zCxBO\njn9x98ei7JKRaxYvrkxes8g5dwAvAKfts6nxeplZAdAb2JbpuNx9m7vvifx6B3BsB4TzNeAsM1tD\n+A6e3zKz+/fZJ1PXK25sGbpmuPuGyPfNwALCd0NtKivzWDoT90JgWmRUdjxQ4e7laTxfQsxsYENd\nz8zGEb4Gaf/PGznnncCH7j67ld06/JolElcmrpmZ9Tez0sjPRcDJwEf77LYQmB75eQqwxCMjSpmM\na58a6FmExw3Syt1/5O5D3X0k4YHHJe5+0T67dfj1SjS2TFwzM+thZr0afgZOAfadjZaVeSyR27pG\nZWYPEp5t0M/MyoCfEx6owd3/DDxNeER2FVANXJpssCmKawpwhZnVAzXA1I74z0u413ExsCJSHwX4\nMTC8SWyZuGaJxJWJazYImGfhB3nkAQ+7+1Nmdj2wzN0XEv6Dc5+ZrSI8ID01zTElGte/mdlZQH0k\nrks6IK6osuB6JRpbJq7ZAGBBpE9SADzg7s+a2eWQ2TwWj5a8i4jkmK44OCkiktOUuEVEcowSt4hI\njlHiFhHJMUrcIiI5RolbRCTHKHGLiOSY/w/sjXumYkeiNgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f13fe94afd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data[:, 1:3], iris.target\n",
    "\n",
    "gb = GradientBoostingClassifier(n_estimators=100, learning_rate=1.0, max_depth=1, random_state=0)\n",
    "gb.fit(X,y)\n",
    "scores_gb = cross_val_score(gb, X, y)\n",
    "print(\"The accuracy of using Gradient Boosting technique is:\",scores_gb.mean())\n",
    "fig = plot_decision_regions(X=X, y=y, clf=gb)\n",
    "plt.title(\"Gradient Boosting\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Subspace Method (RSM)\n",
    "\n",
    "**(Toni)**\n",
    "\n",
    "The Random Subspace Method (RSM) [1] randomly selects an arbitrary number of subspace from the original feature space, and build a classifier on each subspace. This randomization should create classifiers that are complementary. The combination can be carried out by simple fixed rules. Experimental evidences showed that RSM works well with feature spaces with large feature sets and redundant features. It avoids the curse of dimensionality. The concepts of RSM can be related to the theory of stochastic discrimination of Kleinberg [2].\n",
    "\n",
    "![alt text](algo_RSM.png )\n",
    "<p style=\"text-align: center;\">Figure 8. Random Subspace Method algorithm.</p>\n",
    "\n",
    "The algorithm projects each feature vector into a less dimensional subspace, by selecting m random components. RSM is similar to Bagging but instead of sampling objects, it performs a kind of feature sampling withouth replacement since it would be useless to include one feature more than once. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Subspace Method (RSM)\n",
    "\n",
    "**(Toni)**\n",
    "\n",
    "The Random Subspace Method (RSM) [1] randomly selects an arbitrary number of subspace from the original feature space, and build a classifier on each subspace. This randomization should create classifiers that are complementary. The combination can be carried out by simple fixed rules. Experimental evidences showed that RSM works well with feature spaces with large feature sets and redundant features. It avoids the curse of dimensionality. The concepts of RSM can be related to the theory of stochastic discrimination of Kleinberg [2].\n",
    "\n",
    "![alt text](algo_RSM.png )\n",
    "<p style=\"text-align: center;\">Figure 8. Random Subspace Method algorithm.</p>\n",
    "\n",
    "The algorithm projects each feature vector into a less dimensional subspace, by selecting m random components. RSM is similar to Bagging but instead of sampling objects, it performs a kind of feature sampling withouth replacement since it would be useless to include one feature more than once. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### References:\n",
    "\n",
    "[1] T. K. Ho, “The random subspace method for constructing decision forests,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 8, pp. 832–844, 1998.\n",
    "\n",
    "[2] E. M. Kleinberg, “On the algorithmic implementation of stochastic discrimination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 5, pp. 473–490, 2000."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bayesian Voting\n",
    "\n",
    "**(Giuseppe)**\n",
    "\n",
    "In a Bayesian probabilistic setting, each hypothesis *h* defines a conditional probability distribution: $ h(x) = P (f(x) = y | x, h)$. Given a new data point x and a training sample S, the problem of predicting the value of $f(x)$ can be viewed as the problem of computing $P (f(x)=y | S, x) $.\n",
    "We can rewrite this as weighted sum over all hypotheses in H: $ P (f(x) = y | S,x) = \\sum _{h \\in H} {h(x) P(h|S)}$ . \n",
    "\n",
    "*\"We can view this as an ensemble method in which the ensemble consists of all of\n",
    "the hypotheses in H each weighted by its posterior probability $ P(h|S) $ .\" *[1]\n",
    "\n",
    "\n",
    "Thanks to this definition, we can implement a bayesian voting as an ensemble technique by using distinct classifiers outputs as hypoteses, then choosing the hypotesys that reaches the highest probability.\n",
    "I'm going to show an example of it using sklearn VotingClassifier module, that works exactly as my idea, on the iris dataset to continue what already done by my classmates.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** References **\n",
    "\n",
    "[1] Dietterich, Thomas G. \"Ensemble methods in machine learning.\" Multiple classifier systems 1857 (2000): 1-15."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy of using NB is: 0.899509803922\n",
      "The accuracy of using LR is: 0.907679738562\n",
      "The accuracy of using NB is: 0.920751633987\n",
      "The accuracy of using simple voting is: 0.940767973856\n",
      "The accuracy of using soft voting is: 0.912990196078\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.ensemble import VotingClassifier\n",
    "from sklearn import datasets\n",
    "from sklearn import tree\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "# import some data to play with\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data[:, 1:3]  # we only take the first two features.\n",
    "y = iris.target\n",
    "\n",
    "# Decision Tree \n",
    "clf_tree = tree.DecisionTreeClassifier(random_state=1)\n",
    "clf_tree.fit(X, y)\n",
    "probabDT = clf_tree.predict_proba(X)\n",
    "scoresDT = cross_val_score(clf_tree, X, y)\n",
    "print(\"The accuracy of using NB is:\",scoresDT.mean())\n",
    "\n",
    "\n",
    "# Logistic Regression \n",
    "clfLr = LogisticRegression(random_state=1)\n",
    "clfLr.fit(X, y)\n",
    "probabLR = clfLr.predict_proba(X)\n",
    "scoresLr = cross_val_score(clfLr, X, y)\n",
    "print(\"The accuracy of using LR is:\",scoresLr.mean())\n",
    "\n",
    "\n",
    "# Naive Bayes \n",
    "clfNb = GaussianNB()\n",
    "clfNb.fit(X, y)\n",
    "probabNb = clfNb.predict_proba(X)\n",
    "scoresNb = cross_val_score(clfNb, X, y)\n",
    "print(\"The accuracy of using NB is:\",scoresNb.mean())\n",
    "\n",
    "#Hard Voting Classifier\n",
    "eclf1 = VotingClassifier(estimators=[('dt', clf_tree), ('lr', clfLr), ('nb',clfNb)], voting='hard')\n",
    "eclf1.fit(X, y)\n",
    "scores_Vot1 = cross_val_score(eclf1, X, y)\n",
    "print(\"The accuracy of using simple voting is:\",scores_Vot1.mean())\n",
    "\n",
    "#Soft Voting Classifier\n",
    "\n",
    "eclf2 = VotingClassifier(estimators=[('dt', clf_tree), ('lr', clfLr), ('nb',clfNb)], voting='soft')\n",
    "eclf2.fit(X, y)\n",
    "scores_Vot2 = cross_val_score(eclf2, X, y,scoring='accuracy')\n",
    "print(\"The accuracy of using soft voting is:\",scores_Vot2.mean())\n",
    "    \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy of using my own bayesian voting and 10-Fold CV is: 0.893333333333\n"
     ]
    }
   ],
   "source": [
    "from sklearn.exceptions import NotFittedError\n",
    "from sklearn.utils import check_array\n",
    "from sklearn.base import BaseEstimator\n",
    "\n",
    "class bayesVoting(BaseEstimator):\n",
    "    def __init__(self, estimators):\n",
    "        self.estimators = estimators\n",
    "        self.isFitted = False\n",
    "        \n",
    "    def fit(self, X, y):\n",
    "        estimators = []\n",
    "        for estimator in self.estimators:\n",
    "            estimator.fit(X, y)\n",
    "            estimators.append(estimator)\n",
    "        self.isFitted = True\n",
    "        self.X = X\n",
    "        self.y = y\n",
    "        self.estimators = estimators\n",
    "    \n",
    "    def fit_transform(self, X, y):\n",
    "        for estimator in self.estimators:\n",
    "            estimator.fit_transform(X, y)\n",
    "        self.isFitted = True\n",
    "        self.X = X\n",
    "        self.y = y\n",
    "    \n",
    "    def _getProbabilities(self, X):\n",
    "        def _getProbsFromBinary(X, estimator):\n",
    "            outputClasses = estimator.predict(X)\n",
    "            allClasses = set(self.y)\n",
    "            probabs = np.zeros(shape=(len(X),len(allClasses)))\n",
    "            \n",
    "            for i in range(len(X)):\n",
    "                outputClass = outputClasses[i]\n",
    "                probabs[i][outputClass] = 1\n",
    "            return outputClasses\n",
    "            \n",
    "        sumProbs = []\n",
    "        for estimator in self.estimators:\n",
    "            try:\n",
    "                estProbs = estimator.predict_proba(X)\n",
    "            except AttributeError:\n",
    "                estProbs = _getProbsFromBinary(X=X, estimator=estimator)\n",
    "            sumProbs = sum(sumProbs,[estProbs])\n",
    "        return sumProbs[0]\n",
    "    \n",
    "    def predict(self, X):\n",
    "        X = check_array(X, accept_sparse='csr')\n",
    "        if not self.isFitted:\n",
    "            raise NotFittedError\n",
    "            return\n",
    "        estimatedProbs = self._getProbabilities(X=X)\n",
    "        predictedClasses = np.zeros(len(estimatedProbs))\n",
    "        for i in range (len(estimatedProbs)):\n",
    "            predictedClasses[i] = list(estimatedProbs[i]).index(np.max(estimatedProbs[i]))\n",
    "        return predictedClasses\n",
    "        \n",
    "    def predict_proba(self, X):\n",
    "        if not self.isFitted:\n",
    "            raise NotFittedError\n",
    "            return\n",
    "        estimatedProbs = self._getProbabilities(X)\n",
    "        return np.array(estimatedProbs)/len(self.estimators)\n",
    "\n",
    "bv = bayesVoting(estimators=[clf_tree, clfLr, clfNb])\n",
    "bv.fit(X, y)\n",
    "#accuracy_score(y_true=y,y_pred=bv.predict(X))\n",
    "scores_Vot3 = cross_val_score(bv, X, y,scoring='accuracy',cv=10)\n",
    "print(\"The accuracy of using my own bayesian voting and 10-Fold CV is:\",scores_Vot3.mean())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}