{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lecture 24: Second-order optimization\n", "\n", "[EE227C course page](https://ee227c.github.io/) \n", "[Download ipynb file](https://ee227c.github.io/code/lecture24.ipynb)\n", "\n", "Overview:\n", "\n", "* [Newton iteration](#newton)\n", "* [Logistic regression](#logistic)\n", "* [Newton with approximate least squares](#lsq)\n", "* [BFGS](#bfgs)\n", "* [Hessian-vector products](#hvprod)\n", "* [A more realistic example](#adult)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "\n", "import numpy as np\n", "import matplotlib\n", "import matplotlib.pyplot as plt\n", "import sklearn.linear_model as linear_model\n", "\n", "np.random.seed(1337)\n", "from timeit import timeit\n", "\n", "from optimizers import gradient_descent, gss\n", "from plotters import error_plot, convergence_plot, kwargs, setup_layout\n", "\n", "setup_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Newton iteration\n", "\n", "Below is a naive implementation of Newton's iteration that we analyzed in the last lecture." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def newton_method(init, steps, grad, hessian, num_to_keep=None):\n", " \"\"\"Newton iteration with full Hessian inverse.\n", " \n", " Parameters\n", " ----------\n", " initial : array\n", " starting point\n", " steps : list of floats\n", " step size schedule for the algorithm\n", " grad : function\n", " mapping arrays to arrays of same shape\n", " hessian : function\n", " mapping 1d arrays to 2d arrays\n", " num_to_keep : integer, optional\n", " number of points to keep\n", " \n", " Returns\n", " -------\n", " List of points computed by Newton iteration. Length of the\n", " list is determined by `num_to_keep`.\n", " \"\"\"\n", " xs = [init]\n", " for step in steps:\n", " Hinv = np.linalg.pinv(hessian(xs[-1]))\n", " xs.append(xs[-1] - step * Hinv.dot(grad(xs[-1])))\n", " if num_to_keep:\n", " xs = xs[-num_to_keep:]\n", " return xs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As a sanity check, Newton's method should solve a quadratic in one step. The same is true for least squares." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def least_squares(A, b, x):\n", " \"\"\"Least squares objective.\"\"\"\n", " return (0.5/len(A)) * np.linalg.norm(A.dot(x)-b)**2\n", "\n", "def least_squares_gradient(A, b, x):\n", " \"\"\"Gradient of least squares objective at x.\"\"\"\n", " return A.T.dot(A.dot(x)-b)/len(A)\n", "\n", "def least_squares_hessian(A, x):\n", " \"\"\"Hessian of least squares objective.\"\"\"\n", " return A.T.dot(A)/len(A)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "m, n = 1000, 100\n", "A = np.random.normal(0, 1, (m, n))\n", "x_opt = np.random.normal(0, 1, n)\n", "noise = np.random.normal(0, 0.1, m)\n", "b = A.dot(x_opt)\n", "objective = lambda x: least_squares(A, b, x)\n", "gradient = lambda x: least_squares_gradient(A, b, x)\n", "hessian = lambda x: least_squares_hessian(A, x)\n", "x0 = np.random.normal(0, 1, n)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "xs = newton_method(x0, [1], gradient, hessian)\n", "errors = [objective(x) for x in xs]" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjYAAAICCAYAAADcXYeOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XecVOXZ//HPNbMN2KX3Loj0vsASo9HYklhjjHQEgcUk\nPiaPwfyMMT4x5lGjMUaNeaSICEiLsWGJPSYqS1mKCgICAtKb9O17//6Y2dlZ3F22n92Z7/v1Oq+Z\nueY+c65Vypf73HOOOecQERERiQQ+rxsQERERqSoKNiIiIhIxFGxEREQkYijYiIiISMRQsKlhZjbI\nzF4ws+1mlmFmB83sAzO7yuveRERE6roYrxuIQl2AeGA2sAdoAPwIWGpmP3HOPeVlcyIiInWZ6eve\n3jMzP5AONHDOdfO6HxERkbpKp6JqAedcHrALaOx1LyIiInWZTkV5xMwSgQQCYeZa4HvAEk+bEhER\nqeOiesbGzJLM7Bozu8/M3jCzQ2bmgluPMn5GazN7zMy2mlmmme03s6VmdslZdn0KOAh8ATwEvAj8\nrHI/kYiISHSL6jU2ZnYdgUBRnJ7OuY1n2b8f8B7QLFg6DiQSCIwOuMs592AJ+/YG2gBtgVFAJvAT\n59y+8v4cIiIiEhDVMzZBB4DXgXuB1LLuZGb1gFcIhJo1QB/nXCOgCfAIYMD9ZnZ5cfs759Y7595x\nzs0FfkAgEL1iZlaZH0ZERCSaRfuMjT+4cLfgdWfgy+DLUmdszOwXwKPASaCHc273Ge+/CFwHrHbO\nDS5DL6nA9OBnbSptbPPmzV3nzp3P9pEiIiIRIT09/ZBzrkVZxkb14uHwUFMBY4KPC84MNUEPEwg2\ng8ys+9nCClAv+NjobAfu3Lkzq1atKnunIiIidZiZ7SjrWJ2KqgAzSwIKZmHeLGFYGnAs+Dy0kNjM\nWhbzeXHATUAGsKHqOhUREYkuUT1jUwk9CayhAVhf3ADnXL6ZbQKGAr3C3lpkZlnAx8BeAouHxwLd\ngF86505WW9ciIiIRTsGmYtqEPd9TyriC98LHzwPGA7cCTQl8kyodmOace6UqmxQREYk2CjYV0yDs\neUYp404HHxMLCs65Z4BnynvA4OLiVICOHTuWd3cREZGooDU2dYRzboZzLtk5l9yiRZkWhouIiEQd\nBZuKORX2vF6Jo6B+8FHrZkRERGqAgk3FhK+raVvKuIL39lZjLyIiIhKkYFMxGwncMgGgd3EDzMwH\ndA++1Fe4RUREaoCCTQU4504ABVfIu6yEYcMovNjeu9XelIiIiCjYVMKC4OMYM2tTzPvTgo/pZbjq\nsIiIiFSBqA82Zta8YCNwA8sCjcPfC55aCjcd2AEkAa+aWa/g5yWZ2UPA9cFxd1X3zyAiIiIBuo4N\nHCyhvuyM1+cA2wteOOcyzOxaAqeZBgHrzew4gWvW+AiswbnLOfdWlXcsIiIixYr6GZvKcM6tA/oA\njwPbgHjgMPAacJlz7kEP2xMREYk6UT9j45yzs48qdf99wM+Dm4iIiHhIMzYiIiISMRRsotyOw6d4\n/N0vcM6dfbCIiEgtF/WnoqLZrq9PM3rmcnYfzeDQySx+d3VvfL5KnZkTERHxlGZsolRmTh5jZgVC\nDcDcZTv4zUufkp+vmRsREam7FGyiVEKsn1u+0xULm6BZuOIr7nj+E/IUbkREpI5SsIlio4Z25OEb\n+hN+9ukfq3dx+5K15Oble9eYiIhIBSnYRLkbBrfn0RED8Ielm5fX7uG2RWvIUbgREZE6RsFGuHZA\nO/46aiAxYeHm9U/38ZP5q8nKzfOwMxERkfJRsBEAvt+3Df83djCx/sJw887n+5k6L53MHIUbERGp\nGxRsJOSyXq2YMT6ZuJjCXxb/2nSQyc+uIiNb4UZERGo/BRsp4uLuLZl90xASYgt/aXy45RAT56zg\nVFauh52JiIicnYKNfMO3uzVnzsSh1I/zh2pp245w0+wVnMjM8bAzERGR0inYSLFSujRj7s1DSYwv\nvDj1qh1fM/bpFRzLULgREZHaScFGSpTcuSnzJg0lKaEw3Kz76ihjZqXx9alsDzsTEREpnoKNlGpg\nxyYsnJJC4/qxodpnu48zamYah05mediZiIjINynYyFn1adeIhVNSaNYgLlTbuO8Eo2akceB4poed\niYiIFKVgI2XSs01DFqWm0CIpPlT74sBJRs5IY98xhRsREakdFGykzLq1SmJxagqtGyaEatsOneLG\n6cvY9fVpDzsTEREJULCRcunSIpHFU1No17heqLbzyGlGTE9j52GFGxER8ZaCjZRbp2YNWDw1hQ5N\nC8PN7qMZjJixjC8PnfKwMxERiXYKNlIh7ZvUZ8nU4ZzTvEGotvdYJjdOX8aWAyc87ExERKKZgo1U\nWJtG9VicmkLXFoXh5uCJLEZMT2PjvuMediYiItFKwUYqpWXDBBalDqd7q6RQ7fCpbEbNSOOz3cc8\n7ExERKKRgo1UWoukeBamptCrTcNQ7evTOYyemca6r4562JmIiEQbBRupEk0bxLFwSgr92zcK1Y5n\n5jJ21nLSdxzxsDMREYkmCjZSZRrVj2Xe5GEM7tQkVDuRlcu4p1ewfNthDzsTEZFooWAjVaphQizP\n3jyUoec0DdVOZ+dx0zMr+GjLIQ87ExGRaKBgI1UuMT6GOROH8K2uzUK1zJx8bp6zkg82H/SwMxER\niXQKNlIt6sfFMHvCEL5zXotQLSs3nynPruKdDfs97ExERCKZgo1Um4RYPzPGD+bSni1Dtey8fG6Z\nn84/P9vrYWciIhKpFGxqmJklmtm9Zva6mR00M2dmd3rdV3WJj/HztzGD+V7v1qFabr7jZwvWsHTd\nHg87ExGRSKRgU/OaA/cAfYE1HvdSI+JifDwxeiBX928bquXlO36+aA0vrN7lYWciIhJpYrxuIArt\nBdo55/aYWWfgS2/bqRmxfh9/GTGAWJ/xwprdAOQ7+OXf15Gb57hxSAePOxQRkUigGZsa5pzLcs5F\n5TkYv894+Mf9GZFcGGKcg1/94xPmpe3wsDMREYkUCjZSo/w+44Hr+zI2pWOR+m9f+ozZH0bF5JWI\niFSjiAg2ZpZkZteY2X1m9oaZHQouynVm1qOMn9HazB4zs61mlmlm+81sqZldUt39Rxufz7jv2j5M\nPL9zkfrvX93A9A+2etOUiIhEhEhZY3MJ8GJFdzazfsB7QMEV5Y4TWOR7FXClmd3lnHvwjH0MiC/j\nIbKdc/kV7S8SmRn3XNWLuBgf0z/YFqo/8MZGsnPz+a9LunnYnYiI1FURMWMTdAB4HbgXSC3rTmZW\nD3iFQKhZA/RxzjUCmgCPAAbcb2aXn7Hr+UBGGbcLK/xTRTAz487v9eC2755bpP7I25v581ubcM55\n1JmIiNRVkTJjs9Q591LBi+C3jcpqKtAJOAlc7ZzbDeCcOw5MM7OuwHXAA8BbYfttBiaW8Rgby9FP\nVDEzbr+8O7F+H4+8vTlUf/y9LWTl5XPn93oQmBwTERE5u4gINs65vErsPib4uKAg1JzhYQLBZpCZ\ndXfObQoe8wAwpxLHlTD/dUk3YmN8PPhGYQac/sE2cnIdv72qp8KNiIiUSSSdiio3M0sCBgdfvlnC\nsDTgWPC5FhJXo1u+05XfXtWrSG32R19yz8vryc/XaSkRETm7iJixqYSeBNbQAKwvboBzLt/MNgFD\ngV7FjSkvM7sVaBzcAC42s4L/F084544Vv2fkm/Ttc4iL8fHblz4L1eal7SAnL5/7f9gXn08zNyIi\nUrJoDzZtwp6XdtG8gvfalDKmPKYRWNdT4PLgBjCfwhmiqDQupRNxfuPOFz6lYP3wopVfkZ2Xz8M3\n9MevcCMiIiWI9mDTIOx5RinjTgcfE6vioM65zuXdx8xSCX7bq2PHjmcZXfeNGNKRWL+PaX9fR8FZ\nqBdW7yYnz/HnG/sT64/qs6giIlIC/e1QRzjnZjjnkp1zyS1atPC6nRpx/aD2/GXkwCIzNEvX7eG2\nhWvIztVlgURE5JuiPdicCnter5Rx9YOPJ6uxFynGNf3b8tdRA4kJCzdvfLaPnz6XTlZuZb4MJyIi\nkSjag034upq2pYwreG9vNfYiJfh+3zY8NXYwcWGnn975/ACpc9PJzFG4ERGRQtEebDYCBd8j7l3c\nADPzAd2DLzfURFPyTZf2asWM8YOJjyn8JfvB5oNMenYlp7NzPexMRERqk6gONs65E8Cq4MvLShg2\nDGgUfP5utTclJbqoe0tmTxhCQmzhL9uPthxmwjMrOZmlcCMiIlEebIIWBB/HmFlxX+eeFnxML7jq\nsHjn/HOb8+zEodSP84dqK748wvinl3M8M8fDzkREpDaImGBjZs0LNgI3sCzQOPy94KmlcNOBHUAS\n8KqZ9Qp+XpKZPQRcHxx3V3X/DFI2w7o0Y96koSTFF16tYPXOo4ybtZxjpxVuRESiWcQEG+Bg2LY6\nrL7sjPeKXATGOZcBXAscBgYB683sGHAUuIPAGpxfO+fCb4ApHhvcqSnzJw+jYUJhuFm36xijZ6Vx\n5FS2h52JiIiXIinYVJhzbh3QB3gc2AbEEwg6rwGXOece9LA9KUH/Do1ZMCWFJvVjQ7X1e44zemYa\nh05mediZiIh4xZzTzQXrmuTkZLdq1aqzD4wSG/cdZ8zM5RwOm6k5t2UiCyYPo2XDBA87ExGRqmBm\n6c655LKM1YyN1Hk9WjdkUWoKLZLiQ7UtB04yYkYae4+VdqcMERGJNAo2EhG6tUpicWoKrcNmaL48\ndIoR09PY9fXpUvYUEZFIomAjEaNLi0SWTB1Ou8aFd8fYeeQ0I6ansePwqVL2FBGRSKFgIxGlY7P6\nLJ6aQsem9UO13UczGDE9ja0HdasvEZFIp2AjEad9k/osmTqcLs0bhGr7jmcyckYaX+w/4WFnIiJS\n3RRsJCK1bpTAotQUurVMDNUOnshi5Iw0Pt973MPORESkOinYSMRq2TCBhakp9GidFKodPpXNqJlp\nfLb7mIediYhIdVGwkYjWPDGehVNS6NOuYah29HQOo2emsfarox52JiIi1UHBRiJekwZxPDc5hf4d\nGodqxzNzGTtrOau2H/GwMxERqWoKNhIVGtWLZf6koQzuVHh/1JNZuYyfvYK0bYc97ExERKqSgo1E\njaSEWObePJRh5zQN1U5n5zHhmRV8+MUhDzsTEZGqomAjUaVBfAxzJg7l2+c2D9Uyc/K5+dmVvL/p\ngIediYhIVVCwkahTL87PrJuSuah7i1AtOzefqXPTeXvDfg87ExGRylKwkaiUEOtn+rjBXNqzVaiW\nnZfPT+an88anez3sTEREKkPBRqJWfIyfv40ZxPf7tA7VcvMdty5cw8trd3vYmYiIVJSCjUS1uBgf\nT4wayDX924ZqefmO/168lufTd3nYmYiIVISCjUS9GL+PR0cM4EeD2odq+Q7ueH4di1bs9LAzEREp\nLwUbEcDvMx6+oR8jh3QI1ZyDO1/4lHnLtnvWl4iIlI+CjUiQz2fc/8O+jEvpVKT+25fX8/SHX3rU\nlYiIlIeCjUgYn8/4/bW9mfTtc4rU73t1A//3r60edSUiImWlYCNyBjPj7it78pOLuhap//GfG3n8\n3S886kpERMpCwUakGGbGr67ozs8v6Vak/ue3N/OnNzfhnPOoMxERKY2CjUgJzIz/vuw8pl1+XpH6\nX9/fwoNvbFS4ERGphRRsRM7i1u92464f9ChSm/7vbfz+1Q0KNyIitYyCjUgZpF7Ylf+5uleR2jMf\nbefulz4jP1/hRkSktlCwESmjieefw//+sE+R2nPLd3LnC5+Qp3AjIlIrKNiIlMOYYZ146Ef9MCus\nLVm1i2l/X0duXr53jYmICKBgI1JuNw7pwJ9v7I8vLNy8uGY3v1i8lhyFGxERTynYiFTADwe257GR\nA/GHpZtXP9nLrQtWk52rcCMi4hUFG5EKurp/W54cPYhYf2G4eXP9fn4yP53MnDwPOxMRiV4KNjXM\nzC4yM1fCluJ1f1I+3+vTmqfGDibOX/hb6d2NB5gyd5XCjYiIBxRsvPMkMO6MbYunHUmFXNKzFTNv\nSiY+pvC303++OMTNc1ZyOjvXw85ERKKPgo13PnTOzT9jO+R1U1Ix3zmvBc9MGEK9WH+o9vHWw0yY\nvZKTWQo3IiI1RcHGQ2aWaGYxXvchVeNb5zbn2ZuH0iCuMNys2H6E8U8v53hmjoediYhEDwUb78wE\nTgCZZvYvMxvqdUNSeUPPacrcScNIii/Mq6t3HmXsrOUcPZ3tYWciItEhIoKNmSWZ2TVmdp+ZvWFm\nh8IW5PY4+yeAmbU2s8fMbKuZZZrZfjNbamaXVHG72cA/gJ8D1wJ3A32Af5vZkCo+lnhgcKcmPDdl\nGA0TCsPNJ7uOMXrmco6cUrgREalOFgk38TOz64AXS3i7p3Nu41n27we8BzQLlo4DiQSCnwPucs49\neMY+BsSXscVs51yJFzcxs3OBT4CPnXOXnu3DkpOT3apVq8p4aPHK+j3HGDtrOV+fLjwN1b1VEvMn\nD6NFUll/6YiIiJmlO+eSyzI2ImZsgg4ArwP3Aqll3cnM6gGvEAg1a4A+zrlGQBPgEcCA+83s8jN2\nPR/IKON2YWk9OOe2AC8DF5pZbFl7l9qtd9tGLEodTvPEuFBt0/4TjJyxjP3HMz3sTEQkckXKwtWl\nzrmXCl6YWedy7DsV6AScBK52zu0GcM4dB6aZWVfgOuAB4K2w/TYDE8t4jFJnjIK+AmKBJOBIGT9X\narnurZNYlDqc0TPTOHAiC4CtB08xYvoyFkxJoW3jeh53KCISWSIi2DjnKnMltDHBxwUFoeYMDxMI\nNoPMrLtzblPwmAeAOZU47pm6EFh/c7wKP1NqgXNbJrJ4aiDc7D0WmKnZfvg0I2YsY8HkFDo0re9x\nhyIikSOSTkWVm5klAYODL98sYVgacCz4vNILic2sRTG1/sA1wDvOOV30JAKd07wBS6YOp13YDM1X\nRzIYMX0ZOw6f8rAzEZHIEtXBBuhJYA0NwPriBgQX/W4KvuxVBcdcbGavmdlvzGyKmT0KfExgLc6v\nquDzpZbq0LQ+S24ZTqdmhTM0e45lcuP0ZWw9eNLDzkREIke0B5s2Yc/3lDKu4L02pYwpq5eA5sDt\nwN+AkQS+0ZXsnCs2XEnkaNe4HotTh9OlRYNQbf/xLEZMT2Pz/hMediYiEhmiPdg0CHueUcq408HH\nxMoe0Dn3uHNumHOumXMu1jnXxjk31jn3RWU/W+qG1o0SWJSaQreWhb+cDp3MYuSMNDbs0RIrEZHK\niPZgU2eYWaqZrTKzVQcPHvS6HamklkmBcNOzTcNQ7cipbEbNTOPTXcdK2VNEREoT7cEmfNVmad+7\nLVgU4dlCCOfcDOdcsnMuuUWLb6w/ljqoWWI8C6cMo2+7RqHasYwcRs9KY83Orz3sTESk7or2YBO+\nrqZtKeMK3ttbjb1IFGpcP475k4cxoEPjUO1EZi7jnl7Byu26nJGISHlFe7DZSOCWCQC9ixtgZj6g\ne/DlhppoSqJLo3qxzJs0lCGdm4RqJ7NyuWn2CpZtPexhZyIidU9UBxvn3Amg4KZLl5UwbBhQcK7g\n3WpvSqJSUkIscyYOZXiXZqHa6ew8Js5ZwX++0JoqEZGyiupgE7Qg+DjGzIr7Ove04GN6wVWHRapD\ng/gYZk8YwgXdmodqmTn5THp2Fe9vPOBhZyIidUfEBBsza16wEbiBZYHG4e8FTy2Fmw7sIHCPplfN\nrFfw85LM7CHg+uC4u6r7ZxCpF+dn5vhkLu5euEA8Ozef1HmreGv9Pg87ExGpGyIm2AAHw7bVYfVl\nZ7zXMXwn51wGcC1wGBgErDezY8BR4A4Ca3B+7ZwLvwGmSLVJiPXz1LjBXNarVaiWk+f46XOree0T\nrV8XESlNJAWbCnPOrQP6AI8D24B4AkHnNeAy59yDHrYnUSg+xs/fxgziyr6FZ0dz8x3/tXA1L68t\n7l6tIiICEXJ3bwDnnJ19VKn77wN+HtxEPBfr9/HYyAHE+o2X1gauTJDv4BeL15Kdm8+Pkzt43KGI\nSO2jGRuRWizG7+ORGwdww+D2oZpzcMfzn7Bg+U4POxMRqZ0UbERqOb/PeOhH/Rg1tMjyMO568VPm\nLtvuSU8iIrWVgo1IHeDzGff/sA83De9UpH7Py+uZ9Z9tHnUlIlL7KNiI1BFmxu+u6c2UC84pUv/D\na5/z5PtbPOpKRKR2UbARqUPMjLt+0JOfXtS1SP3hNzfxl3c245wrYU8RkeigYCNSx5gZd1zRnV9c\n2q1I/S/vfMGf3tqkcCMiUU3BRqQOMjN+cel53HFF9yL1J9/fyv2vf65wIyJRS8FGpA772cXncveV\nPYvUZv7nS+5dukHhRkSikoKNSB03+YIu3HtN7yK1OR9v564XPyM/X+FGRKKLgo1IBLjpW525/4d9\ni9QWrtjJr/7xCXkKNyISRRRsRCLE6GEdeeiGfljYzUWeT9/FL5esJTcv37vGRERqkIKNSAS5MbkD\nj944AF9YuHlp7R5+vmgtOQo3IhIFFGxEIsx1A9vxxKhBxISlm9c+3cvPnltNVm6eh52JiFQ/BRuR\nCHRlvzY8OWYQsf7CcPPWhv38ZP5qMnMUbkQkcinYiESoK3q3Zvq4wcTFFP42f2/jAabMXUVGtsKN\niEQmBRuRCPbdHq2YNT6Z+LBw858vDnHznJWczs71sDMRkeqhYCMS4S48rwXPTBxCvVh/qLZs22Fu\nmr2CE5k5HnYmIlL1FGxEosC3ujZn7qShJMbHhGort3/NuKdXcCxD4UZEIoeCjUiUGNK5KXMnDSUp\noTDcrP3qKGNnLefo6WwPOxMRqToKNiJRZFDHJiyYnEKjerGh2qe7jzFq5nIOn8zysDMRkaqhYCMS\nZfq2b8TCKSk0bRAXqn2+9zijZqZx4ESmh52JiFSego1IFOrVtiGLUlNonhgfqm3ef5KRM9LYd0zh\nRkTqLgUbkSh1XqskFk9NoVXDwnCz7eApRsxYxu6jGR52JiJScQo2IlGsa4tEFqcOp22jhFBtx+HT\njJi+jK+OnPawMxGRilGwEYlynZs3YPHU4bRvUi9U2/V1BiOmL2P7oVMediYiUn4KNiJCh6b1WTJ1\nOJ2b1Q/V9hzL5Mbpy9hy4KSHnYmIlI+CjYgA0LZxPRZPHU7XFg1CtQMnshg5Yxmb9p3wsDMRkbJT\nsBGRkFYNE1iUOpzzWiWGaodOZjNyxjLW7znmYWciImWjYCMiRbRIimfhlBR6tmkYqn19OofRM5fz\nya6jHnYmInJ2CjYi8g3NEuNZOGUY/do3CtWOZeQwZuZyVu/82sPORERKp2AjIsVqXD+O+ZOHMbBj\n41DtRFYu42YtZ8WXRzzsTESkZAo2HjCzgWb2mpkdM7NTZvaBmV3gdV8iZ2qYEMu8ScMY0rlJqHYq\nO4+bZq/g462HPOxMRKR4CjY1zMwGAB8CvYD/Be4GmgLvmNn5XvYmUpzE+BievXkow7s0C9UycvKY\n+MxK/r35oIediYh8k4JNzfsDkAukOOcecs49CqQAB4C/eNqZSAnqx8Uwe8IQLujWPFTLys1n8rOr\neG/jfg87ExEpSsGm5l0AvOecC/1t4Jw7BbwMJJvZuZ51JlKKenF+Zo5P5rs9WoZq2Xn5TJ2Xzj8/\n2+dhZyIihRRsal48UNxNeApqyTXYi0i5JMT6eWrsYK7o3SpUy8lz/GzBal79ZI+HnYmIBEREsDGz\nJDO7xszuM7M3zOyQmbng1qOMn9HazB4zs61mlmlm+81sqZldUsXtbgKGm1nMGfULg4/tqvh4IlUq\nLsbHX0cP4sp+bUK1vHzHbQvX8OKaXR52JiICZ/7lWlddArxY0Z3NrB/wHlCwOvI40By4CrjSzO5y\nzj14xj5GYPalLLKdc/nB538FZgBzzewBIAe4FRgUfL9eMfuL1Cqxfh+PjRhAnN/Hi2t2A5Dv4PYl\n68jJc9yY3MHjDkUkWkXEjE3QAeB14F4gtaw7mVk94BUCoWYN0Mc51whoAjwCGHC/mV1+xq7nAxll\n3ApmY3DOzQz2eD3wCfA5cBnwm+AQ3ZRH6oQYv48//bg/Px7cPlRzDn71/Cc8t3yHh52JSDSLlBmb\npc65lwpemFnncuw7FegEnASuds7tBnDOHQemmVlX4DrgAeCtsP02AxPLeIyN4S+cc78zsz8DfYBM\nYC0wJexzReoEv8/444/6ERfj47nlO0P137z4GTm5+Uw4/xwPuxORaBQRwcY5l1eJ3ccEHxcUhJoz\nPEwg2Awys+7OuU3BYx4A5lT0oMHg9HHB6+CM0Gngo4p+pogXfD7jD9f1IdbvY87H20P13y3dQHZe\nPqkXdvWuORGJOpF0KqrczCwJGBx8+WYJw9KAgtsaV/VC4oI+LiQQnmYGA49InWJm/M/VvUi9sEuR\n+v2vb+TJ97d41JWIRKOImLGphJ4E1tAArC9ugHMu38w2AUMJXC24UoIh5ncEgtQhYAAwGUgncBVi\nkTrJzPj193sQ5/fx17Aw8/Cbm8jOzecXl3YjsOZeRKT6RHuwaRP2vLSLcBS816aUMWW1G8gGfgk0\nAnYCfwIecM4Vd30bkTrDzJh2RXfiYnz8+e3C5WKPvfsF2Xn5/OqK7go3IlKtoj3YNAh7nlHKuILA\nkVjZAzrntgLfK+9+ZpZK8NteHTt2rGwbItXqtku6Eev38cd/Fq6b/79/bSU7N5+7r+ypcCMi1Saq\n19jUJc65Gc65ZOdccosWLbxuR+SsfnJRV+6+smeR2tMffsn/vLKe/HznUVciEumiPdicCnte2oXx\n6gcfT1ZjLyIRZ/IFXfj9tb2L1OYu28FvXvpU4UZEqkW0B5vwdTVtSxlX8N7eauxFJCKNH96ZB67v\nS/jZp4UrvuKO5z8hT+FGRKpYtAebjUDBn6y9ixtgZj6ge/DlhppoSiTSjBrakYdv6I8vLNz8Y/Uu\nbl+ylty8/JJ3FBEpp6gONs65E8Cq4MvLShg2jMC3lwDerfamRCLUDYPb8+iIAfjD0s3La/dw26I1\n5CjciEgViepgE7Qg+DjGzIr7Ove04GN6wVWHRaRirh3QjidGDSQmLNy8/uk+fvrcarJyK3MBcRGR\ngIgJNmZs6aV7AAAgAElEQVTWvGAjcAPLAo3D3wueWgo3HdgBJAGvmlmv4OclmdlDBG5WCXBXdf8M\nItHgB33b8Lcxg4j1F4abtzfs55Z56WTmKNyISOVETLABDoZtq8Pqy854r8hFYJxzGcC1wGFgELDe\nzI4BR4E7CKzB+bVzLvwGmCJSCZf3bs2M8cnExRT+EfT+poNMfnYVGdkKNyJScZEUbCrMObeOwJ22\nHwe2AfEEgs5rwGXOuQc9bE8kIl3cvSWzbxpCQmzhH0MfbjnExDkrOJWV62FnIlKXmXP6umVdk5yc\n7FatWnX2gSJ1QNq2w9w8ZyWnw2Zqkjs14ZmJQ0hKiPWwMxGpLcws3TmXXJaxmrEREU+ldGnG3JuH\nkhhfeIeXVTu+ZtzTKziWkeNhZyJSFynYiIjnkjs3Zd6koSQlFIabtV8dZcysNL4+le1hZyJS1yjY\niEitMLBjExZOSaFx/cLTT5/tPs6omWkcOpnlYWciUpco2IhIrdGnXSMWTkmhWYO4UG3jvhOMmpHG\ngeOZHnYmInWFgo2I1Co92zRkUWoKzRPjQ7UvDpxk5Iw09h1TuBGR0inYiEit061VEounptCqYWG4\n2XboFCNmLGP30QwPOxOR2k7BRkRqpa4tElkydTjtGtcL1XYcPs2NTy1j5+HTHnYmIrWZgo2I1Fqd\nmjVg8dQUOjQtDDe7j2YwYsYyvjx0ysPORKS2UrARkVqtfZP6LJk6nHOaNwjV9h7LZMT0ZWw5cMLD\nzkSkNlKwEZFar02jeixOTaFri8Jwc+BEFiNnpLFpn8KNiBRSsBGROqFlwwQWpQ6ne6ukUO3QyWxG\nzljGZ7uPediZiNQmCjYiUme0SIpnYWoKvdo0DNW+Pp3D6JlprPvqqIediUhtUeFgY2a3Bbe2VdmQ\niEhpmjaIY+GUFPq3bxSqHc/MZeys5aTvOOJhZyJSG1RmxuZR4E/AoSrqRUSkTBrVj2Xe5GEM6tg4\nVDuRlcv4p1ewfNthDzsTEa9VJtgcAk4453SHOhGpcQ0TYpk7aRhDz2kaqp3KzmPCMyv5aIv+vSUS\nrSoTbFYDjcysRVU1IyJSHonxMcyZOIRvdW0WqmXk5HHznJV8sPmgh52JiFcqE2weD+7/2yrqRUSk\n3OrHxTB7whC+c17hv7GycvOZ8uwq3tmw38PORMQLFQ42zrk3gGnALWY2z8z6V11bIiJllxDrZ8b4\nwVzas2Wolp2Xzy3z0/nnZ3s97ExEapo55yq2o9m24NPWQMGd6jKAw0BeCbs551zXCh1QQpKTk92q\nVau8bkOk1snOzee2hWv45/p9oZrfZ/xlxACu7q8vcIrUVWaW7pxLLsvYmEocp3MxtfrBrSQVS1Ei\nImUQF+PjidEDuX3JOpau2wNAXr7j54vWkJOXz/WD2nvcoYhUt8oEm4urrAsRkSoS6/fxlxEDiPUZ\nL6zZDUC+g1/+fR25eY4bh3TwuEMRqU4VDjbOuQ+qshERkari9xkP/7g/sX4fi1d9BYBz8Kt/fEJW\nXj7jUjp53KGIVBfdUkFEIpLfZzxwfV/GpnQsUv/tS58x+8MvPepKRKpbZU5FFWFmBnQHCr5zeRDY\n5Cq6OllEpJJ8PuO+a/sQ6/fxzEfbQ/Xfv7qBnLx8pn5H32UQiTSVnrExs3PNbA5wDFgP/Cu4rQeO\nmdkzZnZuZY8jIlIRZsY9V/Vi6ne6FKk/8MZGnnj3C4+6EpHqUqlgY2bXAGuAcUAiYGdsicB4YI2Z\nXVW5VkVEKsbMuPN7Pbjtu0X/jfXI25v581ub0MSySOSozN29uwKLgAbANmAq0A2oF9y6AbcAW4Nj\nlgT3ERGpcWbG7Zd35/bLzitSf/y9Lfzxnwo3IpGiMjM2vwISgPeBfs65mc65rc65rOC21Tk3A+gP\nfEDgIn53VL5lEZGKu+2Sbtz5/R5Fak99sJX7Xv1c4UYkAlQm2FxG4IJ7U51zGSUNCr43lcCpqcsr\ncTwRkSpxy3e68turehWpzf7oS+55eT35+Qo3InVZZYJNG+CYc27L2QY65zYDR4P7iIh4btK3z+G+\n6/oUqc1L28FdL36qcCNSh1Um2JwG6ptZ7NkGmlkcgXU2Jc7s1GVmlmhm95rZ62Z20Mycmd1ZwthB\nZvaCmW03s4zg+A+0uFqk5o1L6cQff9QXs8LaopVfMe35deQp3IjUSZUJNp8CscBNZRh7U3DsJ5U4\nXm3WHLgH6EvgW2Kl6UJgvdFs4L+APxA4TbfUzG6pziZF5JtGDOnIIz/ujy8s3Lyweje/WLyW3Lx8\n7xoTkQqpzAX65gEXAo8Hrs3H02dejM/MEoBU4I8E1uM8W4nj1WZ7gXbOuT1m1hko8bKmzrnngefD\na2b2VyAd+CXwVPW1KSLFuX5Qe2L8Pv578drQTM3SdXvIzcvnsZEDiYvRRdpF6orK/G6dDbxN4JtR\n04FdZrbIzB4xsyfNbCmwE3iUwAzF28CcSvZbKwW/BbanEvvnAbuAxlXXlYiUxzX92/LXUQOJCZu6\neeOzffz0uXSycvM87ExEyqPCwSY4O3MdMIPAbEwb4EbgFwSuX3MlgVM0jsAsxA91e4VCwXU5zYNX\nbv4l8D0C4U9EPPL9vm14auxg4vyFfzS+8/kBUuemk5mjcCNSF1RqftU5l+GcuwXoCtwOzAfeCm7z\ng7UuzrmflvaV8OKYWZKZXWNm95nZG2Z2KLgo15lZj7N/AphZazN7zMy2mlmmme03s6Vmdkn5ftJq\n8RSB+2l9ATwEvAj8zNOORIRLe7VixvjBxIedfvpg80EmPbuS09m5HnYmImVRJTfBdM7tAP5SFZ8V\n5hICf9lXiJn1A94DmgVLxwnMIF0FXGlmdznnHjxjHyNw2qwssp1zlVlZ+ACBU3NtgVEE/l+U9dgi\nUo0u6t6S2ROGMOnZlWTmBH6bf7TlMBOeWcnsCUNIjK+y+weLSBWrzC0VVptZupl1OfvoCjsAvA7c\nS2ARcpmYWT3gFQKhZg3QxznXCGgCPELgW0j3m9mZFww8n8BX0suyXVjhnwpwzq13zr3jnJsL/IDA\nfbVeCYYrEfHY+ec2Z87EodSP84dqK748wk2zV3A8M8fDzkSkNJX5Z0cvArMW26qqmTMsdc69VPAi\n+G2jspoKdAJOAlc753YDOOeOA9OC96y6jsCsyVth+20GJpbxGBvL0U+pnHPOzP5OYBH2ecCmqvps\nEam4lC7NmDdpKBNmr+REVuA0VPqOrxk3azlzbx5Go/pnvYyXiNSwygSb3UDLqmrkTMFvClXUmODj\ngoJQc4aHCQSbQWbW3Tm3KXjMA3j3za16wcdGHh1fRIoxuFNT5k8exrinl3M8MxBu1u06xuhZacyb\nNIymDeI87lBEwlVm8fCbBK48PKyqmqkKZpYEDA6+fLOEYWnAseDzGl1IbGbfCIPBKzPfROAU14aa\n7EdEzq5/h8YsmJJC47AZmvV7jjN6ZhqHTmZ52JmInKkyweYPwGHgKTNrXkX9VIWeBNbQAKwvbkBw\n0W/B6Z5exY0pLzO71czuBm4Nli42s7uDW/gszKLgt7x+a2aTzewe4DNgIHC3c+5kVfQjIlWrT7tG\nLEpNoVnYDM3GfScYOSONA8czPexMRMJV5lTUucBvCCzG3WRmc4FlBL7CXOJpJOfcvytxzLIIv9Fm\naRfNK3ivqm7MOY3Aup4Cl1N4N/P5FM4QzQPGEwhATQl8WysdmOace6WKehGRatCjdUMWpaYwetZy\nDp4IzNRsOXCSETPSWDBlGG0a1TvLJ4hIdbOKXjPPzPIJXHwPAjMkZfkg55yrUJg641YFPZ1zxS7e\nNbPRwHPBl7HOuWIvPGFmzwGjgbecc1dUpKeaZGapBL8Z1rFjx8E7duzwuCOR6LXt4ElGz1zOvrCZ\nmo5N67NgyjDaN6nvYWcikcnM0p1zyWUZW5lTUTvDth1nvC5p+6oSx4tqzrkZzrlk51xyixYtvG5H\nJKp1aZHIkqnDade4cIZm55HTjJiexo7DpzzsTEQqc0uFzs65c8q7VWXzJQj/U6W0eeGCf1ZpTYuI\nlFvHZvVZPDWFjk0LZ2h2H81gxPQ0th3UHysiXonEW9aGr6tpW8q4gvf2VmMvIhLB2jcJhJsuzRuE\navuOZzJiRhpf7D/hYWci0asyVx7+2swOV/OVhytiI4XrfXoXN8DMfED34Et9vVpEKqxNo3osSk2h\nW8vEUO3giSxGzkjj873HPexMJDpVZsYmDvBX45WHK8Q5dwJYFXx5WQnDhlF4Ibx3q70pEYloLRsm\nsDA1hR6tk0K1w6eyGTUzjc92HytlTxGpapVdPFxbL7m5IPg4xsyK+zr3tOBjesFVh0VEKqN5YjwL\np6TQu23DUO3o6RxGz0xj7VdHPexMJLpUJti8AsSbWUmzIpVmZs0LNgI3sCzQOPy94KmlcNMJfFMr\nCXjVzHoFPy/JzB4Crg+Ou6u6eheR6NOkQRwLJqfQv0PjUO14Zi5jZy0nfccRDzsTiR6VuY5NI2A1\n4Ae+75z7vCobCx6jrM2d45zbfsa+/QmcZmoWLB0ncAdtH4E1OHc55x6solZrVHJyslu1atXZB4qI\nJ05k5jDhmZWk7/g6VKsf52f2hCGkdGlWyp4iUpzyXMemMsFmPIGbYN5D4JTUG5TtysNzy3GMCgeb\n4P6tgV8DVwHtCISbFcCjzrk6u7ZGwUak9juVlcvNc1ay/MvCmZqEWB+zxg/h291q011oRGq/mgo2\nBVceLrgvU5k+yDnnr9ABJUTBRqRuyMjOY8rcVXy45VCoFhfjY/q4wVzc/Rv3wxWREtRUsPkXZQwz\n4ZxzF1fogBKiYCNSd2Tm5HHL/HT+telgqBbn9/HkmEFc1quVh52J1B01EmzEOwo2InVLVm4eP3tu\nDe98vj9Ui/EZT4wayPf7VtV9eEUiV03dK0pERMogPsbP38YM4vt9WodqufmOWxeu4eW1uz3sTCTy\nlDnYmNltZjaphPcSzaxhce+FjXnUzJ4ub4MiIpEgLsbHE6MGck3/wju95OU7/nvxWp5P3+VhZyKR\npTwzNn8Bfl/Ce18AZ7tIw0hgQjmOJyISUWL8Ph4dMYDrB7UL1fId3PH8Ohat2OlhZyKRo7ynoqyC\n74mICOD3GX+6oT8jh3QI1ZyDO1/4lHnLtnvWl0ik0BobEZEa5vMZ9/+wL+NSOhWp//bl9Tz94Zce\ndSUSGRRsREQ84PMZv7+2N5O+fU6R+n2vbuD//rXVo65E6j4FGxERj5gZd1/Zk59c1LVI/Y//3Mjj\n737hUVcidZuCjYiIh8yMX13Rndsu6Vak/ue3N/PIW5vQtcZEykfBRkTEY2bG7Zedx7TLzytSf+K9\nLTz4xkaFG5FyULAREaklbv1uN+76QY8iten/3sbvX92gcCNSRjHlHN/UzN4rrg5QwntFxoiISMlS\nL+xKrN/HvUs3hGrPfLSd7Nx87ru2Dz6frqwhUpryBps44KJS3i/tPajATTNFRKLNxPPPIdbv4+6X\nPgvVnlu+k5y8fB64vh9+hRuREpUn2DxbbV2IiEgRY1M6Eef38f9e+ISCs1BLVu0iN8/x0A39iPFr\nJYFIccocbJxzE6uzERERKerGIR2IjTF+uWQd+cFw88Ka3WTn5fPoiAHEKtyIfIN+V4iI1GI/HNie\nx0YOLHL66dVP9nLrgtVk5+Z72JlI7aRgIyJSy13dvy1Pjh5ErL8w3Ly5fj8/mZ9OZk6eh52J1D4K\nNiIidcD3+rTmqbGDiQs7/fTuxgOkzlO4EQmnYCMiUkdc0rMVM29KJj6m8I/uf28+yM1zVnI6O9fD\nzkRqDwUbEZE65DvnteCZCUOoF+sP1T7eepgJs1dyMkvhRkTBRkSkjvnWuc159uahNIgrDDcrth9h\n/NPLOZ6Z42FnIt5TsBERqYOGntOUuZOGkRRfeNWO1TuPMnbWco6dVriR6KVgIyJSRw3u1IT5k4fR\nMKEw3Hyy6xijZqZx5FS2h52JeEfBRkSkDuvfoTELU1NoUj82VNuw9zijZqRx8ESWh52JeEPBRkSk\njuvdthGLUofTPDEuVNu0/wQjZyxj//FMDzsTqXkKNiIiEaB76yQWpQ6nZVJ8qLb14ClGTF/GnqMZ\nHnYmUrMUbEREIsS5LRNZPHU4bRolhGrbD59mxIxlfHXktIedidQcBRsRkQhyTvMGLJk6nHaN64Vq\nXx3JYOSMNHYcPuVhZyI1Q8FGRCTCdGhanyW3DKdTs/qh2u6jGdw4fRlbD570sDOR6qdgUwXMLNHM\n7jWz183soJk5M7uzhLFzgu+XtJ1f0/2LSORp17gei1OH06VFg1Bt//EsRkxPY/P+Ex52JlK9FGyq\nRnPgHqAvsOYsY6cD44rZDgBHgJXV16aIRJPWjRJYlJpCt5aJodqhk1mMnJHGhj3HPexMpPoo2FSN\nvUA751wHILW0gc65Zc65+eEbkA60BP7unNNVtUSkyrRMCoSbHq2TQrUjp7IZNTONT3cd87Azkeqh\nYFMFnHNZzrk9lfiIscHH+VXRj4hIuGaJ8SyckkKfdg1DtWMZOYyelcaanV972JlI1VOw8ZiZGTAa\n+BL4yON2RCRCNWkQx3OTUxjQoXGodiIzl3FPr2Dl9iMediZStWptsDGzJDO7xszuM7M3zOxQ2ALb\nHmX8jNZm9piZbTWzTDPbb2ZLzeyS6u6/HM4HOgPPOeecx72ISARrVC+WeZOGMqRzk1DtZFYuN81e\nwbKthz3sTKTq1NpgA1wCvAzcDXwPaFaenc2sH/AZcBvQBcgisMj3KuDt4r61ZAEJZdyq6r+dTkOJ\nSI1JSohlzsShpHRpGqqdzs5j4pwV/OeLgx52JlI1anOwgcA3hV4H7uUsi3LDmVk94BUCYWgN0Mc5\n1whoAjwCGHC/mV1+xq7nAxll3C6s8E9V2Gcc8GNglXNuU2U/T0SkLBrEx/DMhKFc0K15qJaZk8+k\nZ1fx/sYDHnYmUnkxZx/imaXOuZcKXphZ53LsOxXoBJwErnbO7QZwzh0HpplZV+A64AHgrbD9NgMT\ny3iMjeXopyQ/AJoCv6+CzxIRKbN6cX5mjk/mJ/PTeX9TYKYmOzef1HmreHL0IC7v3drjDkUqptYG\nG+dcXiV2HxN8XFAQas7wMIFgM8jMuhfMljjnDgBzKnHc8hoD5AILa/CYIiIAJMT6eWrcYG5dsIa3\nN+wHICfP8dPnVvPYyIFc2a+Nxx2KlF9tPxVVbmaWBAwOvnyzhGFpQMEFHDxZSGxmjQiu9wkGKhGR\nGhcf4+dvYwZxZd/CEJOb7/ivhat5eW1x/y4Uqd1q7YxNJfQksIYGYH1xA5xz+Wa2CRgK9KqKg5rZ\nrUDj4AZwsZkV/Pd9wjl35pWwbgAS0KJhEfFYrN/HYyMHEOM3Xl4buCRXvoNfLF5LTp7jhsHtPe5Q\npOwiMdiEz52WdtG8gveqaq51GoF1PQUuD24QCC9nBpsxBNYAvYSIiMdi/D7+fOMAYv0+nk/fBYBz\ncMfz68jJy2fU0I4edyhSNpEYbBqEPc8oZdzp4GNiKWPKzDnXuZzjv1ue8WaWSvCbYR076g8YEal6\nfp/x0I/6Eev3sXDFTiAQbn79wqfk5OUzfnhnbxsUKYOIW2MTqZxzM5xzyc655BYtWnjdjohEKJ/P\nuP+HfbhpeKci9XteXs+s/2zzqCuRsovEYHMq7Hm9UsbVDz6erMZeRETqHDPjd9f0ZvK3zylS/8Nr\nn/O3f23xqCuRsonEYBO+rqZtKeMK3ttbjb2IiNRJZsZvruzJTy/qWqT+0D838dg7X6A7wEhtFYnB\nZiNQ8Duud3EDgrdD6B58uaEmmhIRqWvMjDuu6M4vLu1WpP7oO5v501ubFG6kVoq4YOOcOwGsCr68\nrIRhw4BGwefvVntTIiJ1lJnxi0vP444ruhepP/n+Vu5//XOFG6l1Ii7YBC0IPo4xs+K+zj0t+Jiu\nezSJiJzdzy4+l7uv7FmkNvM/X3Lv0g0KN1Kr1OpgY2bNCzYCN7As0Dj8vWLutD0d2AEkAa+aWa/g\n5yWZ2UPA9cFxd1X3zyAiEikmX9CFe68peoZ/zsfb+c1Ln5Gfr3AjtUNtv47NwRLqy854fQ6wveCF\ncy7DzK4lcJppELDezI4TuGaNj8AanLucc28hIiJldtO3OhPr93HXi5+GaguW7yQnN58Hf9QPv89K\n2Vuk+tXqGZvKcM6tA/oAjwPbgHjgMPAacJlz7kEP2xMRqbNGD+vIQzf0w8IyzN/Td/HLJWvJzcv3\nrjERavmMjXOuUtHfObcP+HlwExGRKnJjcgfi/D5uX7KWgrNQL63dQ06e4y8jA7dmEPGCfuWJiEiF\nXDewHU+MGlTk9NNrn+7lZ8+tJjtXMzfiDQUbERGpsCv7teFvYwYR6y8MN29t2M8t89PJzMnzsDOJ\nVgo2IiJSKVf0bs30cYOJiyn8K+W9jQeYMncVGdkKN1KzFGxERKTSvtujFbPGJxMfFm7+88Uhbp6z\nktPZuR52JtFGwUZERKrEhee14JmJQ6gX6w/Vlm07zE2zV3AiM8fDziSaKNiIiEiV+VbX5sydNJQG\ncYXhZuX2rxk/ewXHMhRupPop2IiISJUa0rkp8yYPIymh8Ioia3YeZeys5Rw9ne1hZxINFGxERKTK\nDerYhAWTU2hULzZU+3T3MUbNXM7hk1kediaRTsFGRESqRd/2jVg4JYWmDeJCtc/3HmfUzDQOnMj0\nsDOJZAo2IiJSbXq1bcii1BSaJ8aHapv3n2TkjDT2H1e4kaqnYCMiItXqvFZJLEpNoWVSYbjZdvAU\nI6YvY8/RDA87k0ikYCMiItXu3JaJLJk6nLaNEkK17YdPc+P0ZXx15LSHnUmkUbAREZEa0bl5AxZP\nHU77JvVCtV1fZzBi+jK2HzrlYWcSSRRsRESkxnRoWp8lU4fTuVn9UG3PsUxunL6MLQdOetiZRAoF\nGxERqVFtG9dj8dThdGnRIFQ7cCKLkTOWsWnfCQ87k0igYCMiIjWuVcMEFqcO57xWiaHaoZPZjJqZ\nxoY9xz3sTOo6BRsREfFEi6R4Fk5JoWebhqHakVOBcPPJrqMediZ1mYKNiIh4plliPAunDKNf+0ah\n2rGMHMbMXM7qnV972JnUVQo2IiLiqcb145g/eRgDOzYO1U5k5TJu1nJWbj/iYWdSFynYiIiI5xom\nxDJv0jCGdG4Sqp3KzmP80yv4eOshDzuTukbBRkREaoXE+BievXkow7s0C9UycvKY+MxK/r35oIed\nSV2iYCMiIrVG/bgYZk8YwgXdmodqWbn5TH52Fe9t3O9hZ1JXKNiIiEitUi/Oz8zxyXy3R8tQLTsv\nn6nz0vnnZ/s87EzqAgUbERGpdRJi/Tw1djBX9G4VquXkOX62YDWvfrLHw86ktlOwERGRWikuxsdf\nRw/iyn5tQrW8fMdtC9fw0prdHnYmtZmCjYiI1Fqxfh+PjRjADwe2C9XyHfz3krUsWfWVh51JbaVg\nIyIitVqM38efftyfHw9uH6o5B796/hOeW77Dw86kNlKwERGRWs/vM/74o36MHtaxSP03L37GnI++\n9KgrqY0UbEREpE7w+Yz/va4PE77VuUj9d0s3MPPf27xpSmodBRsREakzzIz/uboXqRd2KVL/39c/\n58n3t3jUldQmCjZVwMyGmNlfzWy9mZ0ys51mtsTMzitmbKKZ3Wtmr5vZQTNzZnanF32LiNRFZsav\nv9+DWy8+t0j94Tc38ejbm3HOedSZ1AYKNlXj/wE/At4Ffg7MAC4EVptZ3zPGNgfuAfoCa2qySRGR\nSGFmTLuiO7dfVvTfj4+9+wUPvblJ4SaKxXjdQIT4MzDaOZddUDCzxcCnwF3AqLCxe4F2zrk9ZtYZ\n0Ko3EZEKuu2SbsT4jYf+uSlU+79/bSUnN5/fXNkTM/OwO/GCZmyqgHPu4/BQE6x9AawHep1Rz3LO\n6bKZIiJV5KcXncvdV/YsUpv14Zf87pX15Odr5ibaKNhUEwv8M6EVcMjrXkREIt3kC7rw+2t7F6k9\nu2wHv3npU4WbKFNrg42ZJZnZNWZ2n5m9YWaHggttnZn1KONntDazx8xsq5llmtl+M1tqZpdUd//A\nGKAdsKgGjiUiEvXGD+/MA9f3Jfzs08IVX3HH85+Qp3ATNWrzGptLgBcrurOZ9QPeA5oFS8cJLNy9\nCrjSzO5yzj14xj4GxJfxENnOufwSjt0DeBJIA2ZXoH0REamAUUM7Euv38avn11GQZf6xehe5+fk8\n8uP+xPhr7b/npYrU9v/DB4DXgXuB1LLuZGb1gFcIhJo1QB/nXCOgCfAIYMD9Znb5GbueD2SUcbuw\nhGO3Bl4DjgE/cs7llbVvERGpvBsGt+fREQPw+wqnbl5eu4fbFq0hJ6/Yf49KBKnNMzZLnXMvFbwI\nfoOorKYCnYCTwNXOud0AzrnjwDQz6wpcBzwAvBW232ZgYhmPsfHMgpk1At4AGgMXaJGwiIg3rh3Q\njli/j9sWriE3OHXz+qf7yMlbzV9HDyQ+xu9xh1Jdam2wqeRMx5jg44KCUHOGhwkEm0Fm1t05tyl4\nzAPAnIoc0MwSgKXAecClzrkNFfkcERGpGj/o24YYn/GzBavJyQuEm7c37OeWeen839jBJMQq3ESi\n2n4qqtzMLAkYHHz5ZgnD0gicKoLAWp7KHtMPLAaGAz92zi2r7GeKiEjlXd67NTPGJxMXU/jX3fub\nDjL52VVkZGulQCSKuGAD9CSwhgYC15H5huCi34KrOfUqbkw5PQJcQ+A0VFMzGxu+nTnYzG41s7uB\nW4Oli83s7uDWqAr6ERGRoIu7t2T2TUNIiC38K+/DLYeYOGcFp7JyPexMqkOtPRVVCW3Cnpe2xqXg\nvTaljCmrAcHHq4Pbmeaf8XoagTVABS4PbgVjjyEiIlXm292a88yEoUx6diWngzM1aduOcNPsFTwz\ncQhJCbEedyhVJRJnbBqEPc8oZdzp4GNiZQ/onLvIOWclbcWM71zK+O2V7UdERL5peNdm/P/27jxO\niii/LSMAAB35SURBVPLe9/jnNzMM68CwyhI2F0DEILINxqsYxWNugjGaCCgahDB6b5KjJ/FkX86J\n90RjYoyaDcQNjUAWo8ctiyYxN8ogjEoiBlxBRFkVGHaY+Z0/qnqmGadnema6p7prvu/Xq17V9dRT\nT//qoaf7R9VTVYvnTqJbx7r/06/a8B6X3fEsu/YfjjAyyaQ4JjaxZGblZrbKzFZt27Yt6nBERPLS\nhGG9uHfeJEo61SU3L2zcyaWLKnhv76FGtpR8EcfEZm/S686N1OsSzvdkMZaMcfeF7j7B3Sf07ds3\n6nBERPLWuCE9WTK/jNIudaefXty0m1m3V7B9z8EII5NMiGNikzyuZmAj9RLr3sliLCIikoPGDOrB\nkvll9OpaXFu2dnMVsxZWsLXqQISRSWvFMbFZCyQeCnJSQxXMrAAYGS7qfjMiIu3QiQO6s7S8jD7d\n6p6k88rWPcxcUMHmXUpu8lXsEht3rwJWhYvTUlSbDCQuq34y60GJiEhOGnFMCcuuLOOY7nXJzevb\n9zJj4XI27Wzs+hPJVbFLbEL3h/NLzayhy7mvDeeVibsOi4hI+3Rc32788sopDCqtG5a5Ycc+Lv75\nct7csa+RLSUX5XRiY2Z9EhPBAywTSpPXhaeWki0ANgAlwCNmNjpsr8TMbgQuDOt9Ldv7ICIiuW9o\n764sLS9jcK+65GbTzv3MWLicN7bvbWRLyTU5ndgA25Km55LKl9dbNyR5I3ffD3wc2AGcCqwxs13A\nTuDfCcbgfNXdkx+AKSIi7djgXl1YVj6F4X3qbof2zq4DzFiwnFe3VkUYmTRHric2Lebuq4ExwK3A\n60BHgkTnUWCau98QYXgiIpKDBpZ2Zll5Gcf1rUtutlYdZObCCtZtVnKTD8zdm64lOWXChAm+atWq\npiuKiEiLbKs6yOxFK1i3pS6Z6dmlA/fOm8yYQXqkX1szs0p3n5BO3dgesREREWmpviUdWVJexugB\n3WvL3tt3mEtur2D1xp0RRiZNUWIjIiLSgF5di1kyv4yxH6g7QrP7wBFmL1pB5Yb3IoxMGqPERkRE\nJIUeXTpw72cmc+qQ0tqyqoNHuPyOFax4fUeEkUkqSmxEREQa0b1TBxbPm8yk4b1qy/YeqmbOXSt5\n+tXtEUYmDVFiIyIi0oRuHYu4+4qJnHZc79qy/YermXv3Sp56eVuEkUl9SmxERETS0KW4iDvnTOSM\nEX1ryw4eqWH+Pat48p9bIoxMkimxERERSVOnDoUsvGw8Z4/qV1t2qLqGq+6r5Hcvbo4wMklQYiMi\nItIMnToU8rPZ4znvpP61ZYernc/e/xwPr347wsgElNiIiIg0W3FRAbddMo7pYwfWllXXOFcvfZ4H\nnnsrwshEiY2IiEgLdCgs4EczTuHCcYNqy2ocvvir1fxy5cYII2vflNiIiIi0UGGB8f1PjWXGhMG1\nZe7wpd/8nfsqNkQYWfulxEZERKQVCguM6y88mdllQ44q/8aDL3LX029EFFX7pcRGRESklQoKjOs+\nPoYrPjTsqPL/fPglFjz1WjRBtVNKbERERDLAzPjWx0Zz5ZnHHlV+/eNrue3JVyKKqv1RYiMiIpIh\nZsZXzhvF5z98/FHlN/3xZX74h3W4e0SRtR9KbERERDLIzPjiuSP5wrQRR5Xf+qdX+d7vlNxkmxIb\nERGRLPjXs0/gKx8ZdVTZz596jese+aeSmyxSYiMiIpIlV515HN/82Oijyu58+g2+9dAaamqU3GSD\nEhsREZEsmnf6cK67YMxRZfdWbOBrv/2HkpssUGIjIiKSZZeVDeV7F52MWV3Z0pUbufbXq6lWcpNR\nSmxERETawIyJQ/jBJ8dSkJTcPPDcJv5t2Qscqa6JLrCYUWIjIiLSRi4a/wF+NHMchUnZzX+vfpvP\nL3meQ0eU3GSCEhsREZE2dP7Ygfx41jiKkpKbx1/czP/9RSUHj1RHGFk8KLERERFpYx85eQA/nz2e\n4sK6n+En/rmV8sWVHDis5KY1lNiIiIhE4JzRx7Dw8vEUF9X9FD/18jbm3bOS/YeU3LSUEhsREZGI\nTB3Zj7vmTKRTh7qf46df3cGcu55l78EjEUaWv5TYiIiIROhDx/fh7ism0aW4sLZsxRvvcvmdz7L7\nwOEII8tPSmxEREQiVnZsb+6dN4mSjkW1ZZUb3uOyRSvYtU/JTXMosREREckB44f24t7PTKZ7p7rk\nZvVbu7hkUQXv7T0UYWT5RYmNiIhIjjhlcCn3zy+jtEuH2rI1b+9m1u0VbN9zMMLI8ocSGxERkRwy\nZlAPlpaX0btrcW3Z2s1VzFxYwdbdByKMLD8osckAM5toZj82szVmttfM3jSzX5rZiAbqTjUzTzGV\nRRG/iIjkllH9u7O0vIy+JR1ry17duocZCyt4Z9f+CCPLfUpsMuPLwEXAk8DVwELgDOA5Mzs5xTY/\nAS6rN72a/VBFRCQfnHBMCcvKy+jfvVNt2Rvb9zJjQQVvvbcvwshym7nrqaKtZWanAavc/VBS2QnA\nP4DfuvuspPKpwJ+BWe6+tCXvN2HCBF+1alXrghYRkbzw5o59zLq9gk07647UDCrtzJL5ZQzp3SXC\nyNqOmVW6+4R06uqITQa4+zPJSU1Y9gqwBhidajsz62ZmRanWi4iIDOndhWVXljGkV10Ss2nnfi5e\nsJzXt+2JMLLcpMQmS8zMgGOA7Smq3A5UAQfM7C9mNqnNghMRkbzygZ5BcnNsn661ZZt3H2DGwgpe\n2VIVYWS5J2cTGzMrMbPzzew6M3vczLYnDbIdlWYb/c3sFjN7zcwOmNkWM3vYzM7OdvzApcAgoP7p\npkPAbwjG4nwc+AYwBvirmU1sg7hERCQPDejRmaXlZZzQr1tt2baqg8xcWME/39kdYWS5JWfH2JjZ\nBcBvU6w+0d3XNrH9B4E/Ab3Dot1AN4JkzoGvufsN9bYxoCPpOeTuNSneexSwAngJON3dG32amZkd\nD/wdeMbdz2nqjTXGRkSk/dq+5yCzF61g7ea6IzWlXTpw37zJjBnUI8LIsidOY2y2Ao8B/wmUp7uR\nmXUG/psgqXkeGOPuPYCewE2AAd81s3PrbfohYH+a0xkp3rs/8CiwC7ioqaQGwN1fBR4CzjCzDk3V\nFxGR9qtPt44smV/GSQO715bt3HeYS26v4IWNOyOMLDfk8sDVh939wcSCmQ1rxrZXAkOBPcB0d98E\n4O67gWvN7DjgAuB64A9J270MXJHme7zviJGZ9QAeB0qB/+Xubzcj5o1AB6AEeLcZ24mISDvTs2sx\n93+mjMvvepbVYTKz+8ARZi9awT1zJzJ+aK+II4xOziY26RzpaMSl4fz+RFJTz/cJEptTzWyku68L\n33MrcHdL3tDMOgEPAyOAc9z9pWY2cSzB+BudKBURkSb16NKB++ZNYs5dK6nc8B4Aew4e4bI7nuXO\nORMpO7Z3Ey3EU66fimo2MysBxoeLv09RrYLgVBFAqwcSm1khsAyYAnzK3Zc3UrdvA2VjgfOBJ9z9\nSGvjERGR9qGkUwcWz53E5OF1R2j2Hapmzl3P8rdXUl2UG2+xS2yAEwnG0EBwH5n3CQf9rgsXU95n\nphluIkhMHgd6mdns5Kle3WVm9qiZfd3M5pvZzcAzBON2vpSBWEREpB3p2rGIu6+YxOnH96ktO3C4\nhrn3rOQv67ZGGFk04pjYDEh63dgYl8S6AY3USdcp4Xw6cG8DU7IHgT7AF4CfAjMJrv6a4O4NJmIi\nIiKN6VxcyKJPT+DMEXUnBQ4dqaF8cSVPvLQlwsjaXhwTm65Jrxt7UljiQRvdGqmTFnef6u6WaqpX\n91Z3n+zuvd29g7sPcPfZ4Z2KUzKzcjNbZWartm3b1tqQRUQkZjp1KGTh5eM558RjassOVddw1X2V\nPP6PdyKMrG3FMbGJJXdf6O4T3H1C377vG6YjIiJCx6JCfnrpqXxkTP/asiM1zueWPM9DLzR0LU38\nxDGx2Zv0unMj9RIP3dCDNkREJDaKiwq4bdY4zh87sLasusb5t2Uv8JvKtyKMrG3EMbFJHlczMGWt\nunXt5/iciIi0C0WFBdw84xQuPHVQbVmNw7W/Xs2ylW9GGFn2xTGxWUvwyASAkxqqYGYFwMhwsbn3\nmxEREcl5hQXGDz45lpkTB9eWucOXf/MP7l2+PrK4si12iY27VwGJBylNS1FtMpB4oMaTWQ9KREQk\nAgUFxnc/cTKXlQ09qvybD63hjr+9EVFU2RW7xCZ0fzi/1Mwaupz72nBembjrsIiISBwVFBjf+fhJ\nzDt9+FHl1z3yEj/7y2sRRZU9OZ3YmFmfxETwAMuE0uR14amlZAuADQTPXXrEzEaH7ZWY2Y3AhWG9\nr2V7H0RERKJmZnzjoydy1ZnHHVX+vd+t5dYnG73bSN7J2WdFhVLdsKX+IwuGA+sTC+6+38w+TnCa\n6VRgjZntJrhnTQHBGJyvufsfEBERaQfMjC+fN5LiooKjkpkf/vFlDlfX8IVpIzCzRlrIDzl9xKY1\n3H01MAa4FXgd6AjsAB4Fprn7DRGGJyIi0ubMjC9MG8G15444qvy2P73KDY+vxd1TbJk/cvqITf27\n9rZg+83A1eEkIiIiwOc+fALFRQV897G1tWUL/vo6h6pr+NbHRuf1kZvYHrERERGR1MrPOI5vTz/6\nOdB3Pb2ebz70IjU1+XvkRomNiIhIO3XFh4bz/y4Yc1TZfRVv8tUH/kF1niY3SmxERETasdllQ7nx\nog+SfPZp2aqN/PuvVnOkuia6wFpIiY2IiEg7d/HEwfzw4rEUJCU3Dzy/iWuWvcDhPEtulNiIiIgI\nnxj3AW6ZOY7CpOzmkb+/w+fuf45DR/InuVFiIyIiIgBMHzuQn1xyKh0K65Kb36/Zwv+5r5KDR6oj\njCx9SmxERESk1nlj+vPz2eMpLqxLEZ5cu5X5iys5cDj3kxslNiIiInKUs088hts/PYGORXVpwl9f\n3sbcu1ey79CRCCNrmhIbEREReZ8zR/TlrjkT6dyhsLbsmdd2MOfOlew5mLvJjRIbERERadBpx/fh\nnrmT6Fpcl9w8u/5dLr9jBbsPHI4wstSU2IiIiEhKk4b3YvG8yZR0rHsK03Nv7uSyRSvYtS/3khsl\nNiIiItKo8UN7ct9nJtO9U11ys/qtXcy6vYJ39x6KMLL3U2IjIiIiTRo7uJQl5WX07NKhtuyld3Yz\na2EF26oORhjZ0ZTYiIiISFpOGtiDpeVT6NOtuLZs3ZYqZi5czpbdByKMrI4SGxEREUnbyP4lLC2f\nQr+SjrVlr23by4wFy3l75/4IIwsosREREZFmOb5fN5ZdOYUBPTrVlq3fsY8ZC5ez8d19EUamxEZE\nRERaYHifrvzyyikMKu1cW7bx3f3MXFjBhh17I4tLiY2IiIi0yOBeXfjlVVMY2rtLbdmmnfu5eMFy\nXtu2J5KYlNiIiIhIiw0q7cyy8ikc26drbdmW3QeZsaCCl7dUtXk8SmxERESkVfr36MTSK8s4oV+3\n2rLqmhpq3Ns8FiU2IiIi0mr9SjqxtLyMUf1L6N6piHvnTWZU/+5tHkdR01VEREREmta7W0eWzC/j\n7V37OWlgj0hiUGIjIiIiGdOzazE9uxY3XTFLdCpKREREYkOJjYiIiMSGEhsRERGJDSU2IiIiEhtK\nbERERCQ2lNiIiIhIbCixERERkdhQYiMiIiKxocRGREREYkOJjYiIiMSGEhsRERGJDSU2IiIiEhtK\nbERERCQ2lNiIiIhIbCixERERkdhQYiMiIiKxYe4edQzSTGa2DdiQwSb7ANsz2F57pD7MDPVj66kP\nW0992HqZ7sOh7t43nYpKbAQzW+XuE6KOI5+pDzND/dh66sPWUx+2XpR9qFNRIiIiEhtKbERERCQ2\nlNgIwMKoA4gB9WFmqB9bT33YeurD1ousDzXGRkRERGJDR2xEREQkNpTYiIiISGwosYkRM+tvZreY\n2WtmdsDMtpjZw2Z2di62m4syva9mNsTMrgnbeNPMDppZlZmtNrMbzGxApvcham3xeTGzbma20cw8\nnOZkqu1ckM0+NLORZnabma0zs71mtsvM/mlmd5rZmZmIPxdkow/NrMDMrjCzJ8xsm5kdNrOdZrbC\nzL5uZiWZ3IcomVmJmZ1vZteZ2eNmtj3p721UBtrP3veEu2uKwQR8kOBmSB5Ou4Dq8HUN8JVcajcX\np0zvKzA43M7rtXkkafld4Kyo9z1X+7CR9/lRvX6dE/W+50MfAv8KHExquwrYn7S8KOr9z9U+BLoA\nT9b73O2s9ze+Hjg26v3PUB9eUG9fk6dRufbvc1T7UXeeptZPQOfwD8qB54CTwvLuwA+SPizn5kK7\nuThlY1+BYeE2jwCfBHqG5cXAR4DXk/6o+0fdB7nYhyne51SC5LAi6YtxTtT7n+t9CFwZbl8N3AAM\nSVrXH7gMmBt1H+RqHwLXJ//wAj3C8mJgJvBeuP5PUfdBhvrxAmAL8CjwH8B8MpDYtMX3ROSdp6n1\nE3ANdf/7GtTA+t+G6ytzod1cnLKxr0APYGwj60dR97/lb0fdB7nYhw20UQCsDBObcTFMbLL1tzwM\n2BtuOz/q/czTPtwQbndHivVzkj6PPaPuhwz0Y2EDn6FMJDZZ/57QGJt4uDSc3+/umxpY//1wfqqZ\njcyBdnNRxvfV3Xe5++pG1q8lOOoAMD7tSHNXW3xePg9MAH7m7s+3sI1clq0+vJrgVMoKd7+9NQHm\ngWz14THhPNXnrjLpdZdmtJuT3L06S01n/XtCiU2eCwerJX4Uf5+iWgXB6Q6AtAZmZavdXBTxvu4I\n54UZbLPNtUUfmtkg4DqCw+PfaO72uS7LfXhJOF/SgtDyRpb7cH04H5difeJ9t6T4wW732uq7VolN\n/jsRsPD1moYquHsNsC5cHB1xu7kokn01syLgQ+Hii5loM0Jt0Ye3ASXAte6+q6nKeSgrfWhmxwH9\nwsXnzawsvPpkh5ntN7O1ZvZ9M+vXWDt5Ipufw8SRrivM7Ctm1gPAzIrNbAZwM8EplGubHXX70Sbf\ntUps8l/y5cJvN1IvsS7dy4uz1W4uimpfP0swaLMGuCdDbUYlq31oZtOBTwB/cff7mhlbvshWH56Q\n9Hoq8DfgY0AHgh/ikQQ/xi+Y2Ulptpmrsvk5/BHwE4If5uuBnWa2k2Cc3FJgLXB+jD+fmdAm37VK\nbPJf16TX+xupty+cd4u43VzU5vtqZh8k+HIE+LG7v9TaNiOWtT40s67Aj4HDBMlgXGWrD0uTXn8b\neBkoc/fuYRv/G9hK8CPym/BIYr7K2ucwHHNyDfBFgsHrEFwgkPgdLQH6ptteO9Um37VKbETaWHhT\nvgcJLnusBL4cbUQ57zvAEODmGCSAUUj+nnfgE+6+AoLD/u7+ODA3XD8SuLCN48sLZtYfeBq4CfgF\nMJbgh/cE4KvAscCdZnZ9ykakTSixyX97k153bqReYpT+nojbzUVttq9m1gv4AzAceAX4qLsfaGl7\nOSQrfWhmpxBc0bORIMGJs2x9DpPr/c7d19Wv4O6PEhzJgTy+EIDs/i0vBiYRXO49x93/7u573f1V\nd7+B4D5BAF+KwSm9bGmT71olNvkv+TzlwEbqJda9E3G7uahN9jUcbPh7YAzwJnCOu29pSVs5KFt9\neAvBFWNfByx8lELtlFSvY1iWz5fZtsXf8vuSmgbWDU6z3VyUlT40s9HAtHDx5obquPu9BFc5FgDT\n02m3HWqT71olNvlvLcHhZYAG/5dgZgUEh5gB0j2Un612c1HW9zUcJ/IYwT1YNhMkNW82P9Scla0+\nHBrOFxPc0Kv+lPDzcFmfw/d7iWCAerq86So5K1t9eGLS6zcaqfd6OB+WZrvtTZv8riixyXPuXgWs\nChenpag2mWCQGwTPOoms3VyU7X01s87Aw8BpBP+jO8fdX2lBqDmrPX1esiWLf8v7gOXhYmM3PEus\nW59Ou7koi5/D5MRwSCP1Eol4VSN12q22+p5QYhMP94fzS1M8LTpxX4XKhs6vR9BuLsrKvppZMfAA\ncBbBA/POdfcG798QAxnvQ3cf5u6WakqqekVYNqwV8eeCbP3NLQ7n5zV0N1cz+ygwIlx8rBnt5qJs\n9GHyHcTnN1QhvCVB4l5AK9Jstz3K/u9Ktp83oSn7E0c/VKwSGB2WlwA3Uvd8j3PrbTcsad2cTLWb\nj1M2+pBgbMivw3W7CS6xjXxf86kP03jPuD0rKlt/y0UEN0TzcD4pLC8AziM4PeoER3Ys6n7I0T78\nPXUPEb0e6BeWdyN4TtSOcP0bQHHU/ZChvuyTNCU/m62s3rqCZvRj1n9XIu84TZmZCC49bNZj4NP5\nQWlJu/k6ZboPgTOS1u0PfzxSTSuj3v9c7MM03i9WiU02+5DgcuQ3k+rtpu7BmImE530PJczHKRt9\nSHCfn5eS6iT6MHl5MzAu6v3PYD96mtOwZn4Ws/q7olNRMeHBwxbHALcSDGDrSPA/iEeBaR5cjpgz\n7eaiLOxr8t9XJ4KH6KWaYnFjr/b0ecmWLP4tvw6cDPwXwQ90EcEPyXME92GZ5DF5xlE2+tDd3yF4\nztE1wF+BdwkuS95N0IfXASd7PB/OmlHZ/p6wMHsSERERyXs6YiMiIiKxocRGREREYkOJjYiIiMSG\nEhsRERGJDSU2IiIiEhtKbERERCQ2lNiIiIhIbCixERERkdhQYiMiIiKxURR1ACIimWJmRcBsYCbB\n82h6EzwPaTPBrdv/P/And382aZtTgAuA9e5+d1vHLCKZpUcqiEgsmFlf4DFgQlLxAeAg0B2wsGyX\nu5cmbTcHuAt4yt2ntkmwIpI1OhUlInFxH0FSUwV8CRjg7p3DJKYHMA34KbAzuhBFJNt0KkpE8p6Z\njQLODRfnuvuvk9e7exXwBPCEmX2xreMTkbajIzYiEgcnJ71+pLGK7n4g8drMnOA0FMCZZub1pqn1\ntzez081sqZm9ZWYHzWyHmT1hZrPMzBqoPzVsa324PN3M/mxm75nZHjNbbmaXtGCfRaQBOmIjInEz\nCHgtzbpbgM4EY3AOA+/WW38oecHMvkdwmithN9ATODuczjezS929pqE3M7NrgJsBB3aF710GlJnZ\nae7+uTTjFpEUdMRGROKgMun1T8KBxE1y9/7A1eHiM+7ev970TKKumV1NkNRsAcqBUnfvAXQluApr\nczj/coq36wvcCCwmGP/TE+gD3BSu/6yO3Ii0nq6KEpFYMLN7gMvDxUMEl3ZXACsJkpZtKbabQxNX\nRZlZKbCR4Ch3mbuvbqDOFOBpgsHJ/d39UFg+FfhzWO2PwL94vS9eM7sb+DTwKjCi/noRSZ+O2IhI\nXMwHfkiQ1BQTnBr6OvAgsNXMnjWzSxsaB5OGi4BuwBMNJTUA7r4ceIPg1NT4FO1cnyJp+a9wfjzB\n/XdEpIWU2IhILLj7IXf/IjAYuApYArxCMJ4FYCLBJeHLzKy5332nhfMPm9nmVFP43iTNkx0mOKLT\nUOyvAO+Ei6c2MzYRSaLBwyISK+6+FVgQTpjZMcB04FsECcenCBKMW5rR7IBw3iWcmtJQne2J01Mp\nbArfJ63xQSLSMB2xEZFYc/ct7r6I4EjIlrB4bjObSXxX3uLulsZ0d6biF5HmUWIjIu2Cu28HHgoX\nRzRz80RCNKQVIfQxs+JG1g8M5w0OchaR9CixEZH2ZG84Tz4llLjnTGODipeH86lm1rmF790BmNLQ\nCjM7nrrE5rkWti8iKLERkRgws+FmdlwTdboQPMUb4IWkVbvDeSmp/YogKepJMFansffp2cjqr6a4\nKuur4fwVd3+hgfUikiYlNiISBycB68zsATO72MwSg30xs65mNp3gvjbDw+LkgcNrwvloM5vcUOPu\nvoO65OMrZna7mdWezjKzLmZ2lpktBJ5pqA1gH8El6HeYWb9wu9LwbsaJMT//keb+ikgKukGfiOQ9\nM/sX4Hf1ivcTnHLqkVRWDXzL3b9bb/ungDPCxXcJnhAOMNPdK5LqfQP4DnWnrfaG71GaVLbe3Ycn\nbTOV4AZ9G4AfUfdIhZ1hbIn/YP5Ej1QQaT0lNiISC+ERlOnA6cAYgmdGFRMkKa8DfwUWufuaBrbt\nTZCwfCRpO4Cz3P0v9eqeDHwOOAv4AFBIMOD3ReBJYIm7v5VUfyphYuPuw8KjR18AxhGMu/k78GN3\n/0WrO0FElNiIiGRT/cQm2mhE4k9jbERERCQ2lNiIiIhIbCixERERkdhQYiMiIiKxocHDIiIiEhs6\nYiMiIiKxocRGREREYkOJjYiIiMSGEhsRERGJDSU2IiIiEhtKbERERCQ2/gcuJfvThPLWKwAAAABJ\nRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "error_plot(errors)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Logistic regression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's move on to a less trivial example that we'll stick to for the rest of the lecture: logistic regression. A logistic model makes predictions based on a sigmoid function applied to a linear function of the data." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def sigmoid(x):\n", " \"\"\"Sigmoid function.\"\"\"\n", " return 1./(1.+np.exp(-x))" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEKCAYAAADuEgmxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4XNWd//H3V73Lai5ykXtv2AYMoYROQofAJjgkEAKE\nbAjJQgg4Cywk9ABpG37AhhACZjcQIEAgQIAQQrFxxb1gLBfZsmxZvUvn98eMRpItyTNqVzP6vJ5n\nnjvn3qs7X/BIn5l77znHnHOIiIgEI8rrAkREJHwoNEREJGgKDRERCZpCQ0REgqbQEBGRoCk0REQk\naAoNEREJmkJDRESCptAQEZGgxXhdQE/Lzs52o0eP9roMEZGwsmzZsn3OuZzD7Rd0aJhZKnAScCQw\nz7/M8m+e4pzb0JVCWx1/KHALcDYwHCgFlgC/cM69HexxRo8ezdKlS7tTiojIgGNm+cHsF8o3jVOA\nF7tWTufMbCbwDi0hVAZk4wuQs8xsoXPu3t54bRERCV6o1zT2Aq8BdwBX90QBZpYIvIwvMFYA051z\n6UAG8CBgwN1mdnpPvJ6IiHRdKN80XnHOvdTcMLPRPVTDNUAeUAGc45zbBeCcKwNuNLNxwPnAPcCb\nPfSaIiLSBUF/03DONfZSDQv8y0XNgXGQB/zLOWY2qZdqEBGRIHh6y63/4vpcf/ONDnb7GN9FcfBd\nVxEREY943U9jCr5rFgBr29vBOdcEbPQ3p/ZFUSIi0j6v+2kMa/W8oJP9mrcN62QfEZGIVNfQREVt\nA+U19ZTXNFBR20BlrW9ZVddIpX9ZVdfIiIxEvj4/r9dq8To0kls9r+5kvyr/MqW9jWZ2Nf67uUaN\nGtUzlYmI9DDnHGU1DRSV11JcWcf+ilr2V9ZRUlXHgap6DlTVUVpVT2l1PSXV9ZRV+0Kiuj74S8rz\nx2ZGdGj0COfcY8BjAPPmzdOk5yLS5xoam9hdWsOukmp2l1azu7SGPaU1FJbVUFhWS1G571HX2NSr\ndVTV9dY9Sz5eh0Zlq+eJQHkH+yX5lxW9W46ISMeq6hr4fF8lW4sqyd9fyfbiKvL3V7HzgC8omjz+\nyBoXHUWU2eF37AavQ6P1dYxcWi54HyzXv9zdu+WIiEBNfSObCstZv7uMTYUVbN5bwZbCcgpKa3rt\nNRNjo8lIimVQUhzpibGBR1piDKkJsaQlxJCSEEtKfAypCTEkx8eQEh9NUpzveVJcNLHRvX9vk9eh\nsQFw+O6gmkY7oWFmUUBz/4x1fVeaiAwEVXUNrN5ZyupdpazZ5Vt+vq+yR741pMTHMCQtniFpCQxO\njSfH/8hOiScrJZ6s5DiyUuLISIojITa6+y/YBzwNDedcuZktxTf44WnAC+3sdjSQ7n8e9MCFIiLt\n2V1azZLPi/lkWzHL80vYWFhOYxcTIjM5jlGZSYzMTGJkRiLDMxIZPiiR3EGJDEtPIDUhtoer957X\n3zQAFuELjQVmdqdz7uBTUDf6l8uccx2dvhIRaVdReS0ffraPD7fs58Ot+9hR3NmNmoeKjTbGZCcz\nYXAq4wanMC4nmTHZyYzOTiYtAkPhcEIKDTPLbtXMaPV80EHbiv2d8prHqPrcv/4K59yTBx32UeAH\n+MafetXMLnPOrfP3Fr8VuNC/38JQahWRgamxybFyRwn/2LiXdzfuZc2usqB/Nj0xlhnD05k2PI2p\nw9KYPDSNsTnJfXKtIFyE+k2jqIP1Hx3UHgNsC+aAzrlqMzsP36mnOcBaMyvD1ycjCt81j4XOOQ1W\nKCLtqmto4oPP9vHm2j28ta6QfRV1h/2Z2GhjWm46c0ZlcMSoQcweOYgRGYlYL999FO76w+kpnHOr\nzGw6bSdh2o9vEqaHQ5mESUQGhsYmx8db9/PyygJeX7ObspqGTvePjTbmjMrgmHFZHDUmkyNGZpAY\nFx4Xn/uTkELDORdyBDvnttEyvlRn++0Brvc/RETatbWogueW7eSF5TspLKvtdN+xOcmcNGkwJ0zM\n4cjRGSTF9YvPyWFN/wdFpN+ra2jib2v38PRH+SzZVtzhfmZwZF4mp08bwmlTh5CXldzhvtI1Cg0R\n6bf2ltfwx4/yeXbJ9g6vU5jBkaMzOXdWLmdOH0p2SnwfVzmwKDREpN/ZXFjO4+9v5aUVBR2O1TQ2\nJ5mL547k/CNyGZae2McVDlwKDRHpN9YWlPKbd7bw+po97W6Pj4ni3Fm5fPWoUcwZNUh3OnlAoSEi\nntu4p5yfv7mRt9YVtrt9ZGYi3zxmNF+ZO4JBSXF9XJ20ptAQEc/sPFDFQ29t4sUVu3DtjOQxe+Qg\nrjlhLKdPG0p0lL5V9AcKDRHpc5W1Dfz3u1v4n/c/b/eaxbHjsrju5AnMH5upU1D9jEJDRPqMc44X\nV+zi3tc3sLf80D4W88dmcsPpkzhydKYH1UkwFBoi0ie27K1g4YurWfL5of0spuWmcdOZkzlhQra+\nWfRzCg0R6VW1DY088o/P+O27nx1yKmpwajw3f2ky588eTpSuWYQFhYaI9Jo1u0q58blVbNjTdibn\nuOgorjx+DP9+0nhS4vVnKJzoX0tEelxDYxO//cdn/OrtzTQcNMHR0WMyueuCGYwfnOJRddIdCg0R\n6VE7iqu4/n9XsHx7SZv16Ymx/OSsKVw8d4SuW4QxhYaI9JjXV+/mx3/+9JBhyk+alMO9F81kSFqC\nR5VJT1FoiEi31TU0cddf1/GHj/LbrE+Oi+bWs6fyb0eO1LeLCKHQEJFu2VtWw3efWc7S/ANt1k8f\nnsavvzaHMdkanjySKDREpMuW5R/g2qeXHdJR78rjxnDTmZOIj9HMeJFGoSEiXfLSil3c9Pynbfpe\npMTH8POLZ3Hm9KEeVia9SaEhIiFxzvHw3zfzq7c3t1k/LieZRy+bp1tpI5xCQ0SCVtfQxI+eX8Vf\nVha0WX/qlCH84quz1VFvANC/sIgEpbK2ge88vYz3N+9rs/7qE8by4zMna+jyAUKhISKHtb+ilm89\n+QmrdpYG1kVHGT89bzqXHj3Kw8qkryk0RKRTe0pruPR/PmZrUWVgXWJsNI98fQ5fnDTYw8rECwoN\nEenQzgNVXPr4YrYXVwXWZSTF8sTlR3LEqAwPKxOvKDREpF3b91fxtcc/ZldJdWBdbnoCT115tO6Q\nGsAUGiJyiO37q7jk0Y/YU1YTWDcyM5Fnr5rPiIwkDysTryk0RKSNXSXVfO3xj9sExpjsZBZddTTD\n0hM9rEz6A4WGiAQUltWw4KBTUuNyknn2qvkM1gi1AkR5XYCI9A/FlXUs+J/FbNvfctF7dFaSAkPa\nUGiICJW1DVzx5Cds2VsRWDciI5FFCgw5iEJDZICra2ji2meWs2pHy0x7Q9MSWPTt+eQO0jUMaUuh\nITKANTU5bnp+Ff/cVBRYNygplj9eeRSjsnSXlBxKoSEygD3w5kZeajX4YEJsFL/75pFMGJLqYVXS\nnyk0RAao/12ynUf+8VmgHR1lPLJgLnPz1NNbOqbQEBmA3t9cxE9eWtNm3b0XzuCkyRpLSjqn0BAZ\nYDYXlvPdp5fT2OQC6647eTwXzxvpYVUSLhQaIgNISVUd335qKeW1DYF1583O5T9Om+hhVRJOFBoi\nA0RDYxPXPbuC/Fad9+blZXDfRTMx0wRKEhyFhsgAcc/rG9rMujd8UCL/77K5JMRGe1iVhBuFhsgA\n8MLynfzuX58H2gmxUTz2jblkp8R7WJWEI4WGSIRbv7uMhS+ubrPu5xfPYlpuukcVSThTaIhEsNLq\neq59ehk19U2Bdd/94jjOnpnrYVUSzhQaIhGqqclx43Or2oxae9z4bG44fZKHVUm4U2iIRKjH3t/K\nW+sKA+3c9AR++dXZREfpTinpOoWGSARall/MA29sDLRjo43/XjCHLF34lm5SaIhEmJKqOr7/7Mo2\nPb5vPXsqR4zSmFLSfQoNkQjinONHz3/aZrrWL88YymXz8zysSiKJQkMkgvzhw21trmOMyEjkngvV\n41t6jkJDJEJs2FPG3a9vCLRjoozfXDqH9MRYD6uSSKPQEIkANfWNXP/sSuoaWvpj/PjMycweOcjD\nqiQSKTREIsC9r29gY2F5oH38hGyuPG6MhxVJpFJoiIS5dzfu5ckPtwXaGUmx/PziWUSpP4b0AoWG\nSBgrrqzjR8992mbdfRfNZEhagkcVSaRTaIiEKecc//nSavZV1AbWfe2oUZw+baiHVUmkU2iIhKmX\nVxXw2uo9gXZeVhK3nj3Fw4pkIFBoiIShPaU13PrSmkA7yuChS2aRFBfjYVUyECg0RMKMc46b/vwp\nZTUt83x/58RxzM3L9LAqGSgUGiJh5k9Ld/DPTUWB9pRhafzg1IkeViQDiUJDJIwUlFTzs1fXB9qx\n0cZDl8wiLka/ytI39E4TCRPOOW55YTXltS2npa47eQJThqV5WJUMNAoNkTDx3LKdvNfqtNS03DSu\n/eI4DyuSgUihIRIGCstq+Omr6wLtmCjj5xfPIjZav8LSt/SOE+nnfJ341lDe6m6p7508XqelxBMK\nDZF+7rXVe9rMkTF5aCrf/eJ4DyuSgUyhIdKPHais4/aX23biu/8rM3W3lHhG7zyRfuynf13Hvoq6\nQPvbx49l5gjNkSHeUWiI9FP/2ryPF5bvCrTzspL4oTrxiccUGiL9UHVdIwtfXN1m3T0XzCAxLtqj\nikR8FBoi/dCv3tnM9uKqQPuSeSM4dny2hxWJ+Cg0RPqZ9bvLeOyfWwPtrOQ4Fn5ZQ55L/6DQEOlH\nGpscN7+wmsYmF1h32zlTGZQU52FVIi0UGiL9yDOL81m1oyTQPnFiDufOyvWwIpG2FBoi/URhWQ33\n/21joJ0QG8XPzp+OmXlYlUhbCg2RfuLOV9ZR0WoE2x+cOpGRmUkeViRyKIWGSD/w7sa9/HX17kB7\n8tBUrjxujIcVibRPoSHiseq6xjbzfQPcdcEMjWAr/ZLelSIe+/U7m9l5oDrQvvToUczNy/CwIpGO\nKTREPLRlbzmPv9/SJyM7JY4fnzHZw4pEOqfQEPFI8zwZ9Y0tfTJ+ctYU0pNiPaxKpHMKDRGPvLhi\nFx9vLQ60jxmbxfmzh3tYkcjhKTREPFBaVc9df10faMdGGz9VnwwJAyGHhpkNNbNfmtlnZlZjZoVm\n9oqZndKVAszscjNzh3lUdOXYIv3VA29uYH9lyzwZ15wwjvGDUzysSCQ4MaHsbGYzgXeALP+qMiAb\nOBs4y8wWOufu7WIt9UBxB9squ3hMkX5n1Y4Snlm8PdAekZHIv5+k6VslPAT9TcPMEoGX8QXGCmC6\ncy4dyAAeBAy428xO72ItHzrnhnbwGNfFY4r0K41NvovfruXaN3ecO03zZEjYCOX01DVAHlABnOOc\nWwvgnCtzzt0IvIQvOO7p8SpFIsSixfms3lUaaJ8+dQinTBniYUUioQklNBb4l4ucc7va2f6AfznH\nzCZ1ryyRyFNUXsv9b7QMSJgYG81t50z1sCKR0AUVGmaWCsz1N9/oYLePgeaPUF26KC4Sye55bT3l\nNS0DEn7/lAmMyNCAhBJegv2mMQXfqSeAte3t4JxrApo/RnXl49M0M1trZtVmVm5ma8zsYTPTqG0S\n9j7eup8XVrR8QR+Xk6wBCSUsBRsaw1o9L+hkv+ZtwzrZpyPZ+MKpCkgApgE/ANaa2aVdOJ5Iv1Df\n2HTIgIQ/PX86cTHqJiXhJ9h3bXKr59Ud7uX7gw8Qyg3nBcDtwHQgwTmX5f/5s4B1QCLwBzM7oaMD\nmNnVZrbUzJYWFRWF8NIive+Jf33O5r0tXY3On53LseOyPaxIpOs8/6jjnHvTOXenc26tc67Ov67W\nOfcacCywBV9/kg77fzjnHnPOzXPOzcvJyembwkWCUFBSzS/f3hxop8bHsPCsKR5WJNI9wYZG6851\niZ3s13xVr0d6cDvnSoG7/c35ZqaPZxJWfvrqOqrqGgPtG06fyODUBA8rEumeYEOj9XWMzma5b962\nu5N9QrXYvzRAVw4lbLy7cS+vr9kTaE8dlsbX5+d5WJFI9wUbGhuA5j6s09rbwcyigOb+Geu6WZdI\nWKupb+T2v7TcaGgGd10wnRjNxidhLqh3sHOuHFjqb57WwW5HA+n+5293s66Dj9tsWw8eV6TXPPKP\nz9heXBVof/XIURwxSrPxSfgL5WPPIv9ygZm1d0vtjf7lMufcxna2H8IOMw60maUBN/ubS5xzujVK\n+r1t+yp55L3PAu3M5DhuOkODJEhkCCU0HgXygVTgVTObCr7e4mZ2P3Chf7+FrX/IzEa3GuL88oOO\nmWdmH5rZN81seKufiTOzM4EPgIlAE3BLKP9hIl5wznHrX9ZQ19AUWHfzlyaTkRznYVUiPSfoodGd\nc9Vmdh6+U09z8HW6K8PXpyIK3zWPhc65N0Os4Rj/AzOrxnenVjrQPOdlFfAd59w7IR5XpM/9dfVu\n3t+8L9Cel5fBV+aM8LAikZ4V0nwazrlVZjYd36f+s4HhwH5gCfCwcy7UaxmFwPeB44BZQA4wCF9w\nbMYXUI845/JDPK5InyuvqefOV1ruAYmOMn52wXSiojQbn0SOkEIDwDm3B7je/whm/220jFt18LZq\n4Nf+h0hYe/DNTewtrw20rzxuDJOHpnlYkUjP0/1/Ij1gza5SnvpoW6Cdm57A9adM8Kwekd6i0BDp\npsYmx8IXV9PUaja+286ZRnJ8yF/kRfo9hYZINz39cT6f7myZje/kyYM5Y5pm45PIpNAQ6YbCshoe\naDUbX0JsFHecO43DdEESCVsKDZFuuOOVtVTUtszG98NTJzIyU7PxSeRSaIh00bsb9vLa6pYBCScP\nTeVbmo1PIpxCQ6QLKmsb+M9Ws/H5BiScQawGJJQIp3e4SBc8/NYmdpW0TGL5taNGMTdPAxJK5FNo\niIRo9c5Snvjg80B7cGo8Pz5zsocVifQdhYZICBoam7j5hU/b9Mm449xppCfGdvxDIhFEoSESgt9/\nsI21BWWB9qlThnDm9KEeViTStxQaIkHK31/Jg2+19MlIjovmzvPUJ0MGFoWGSBCcc9z859XU1LfM\nk/GjMyaROyjRw6pE+p5CQyQI//fJDj7auj/QnpuXwWXHjPauIBGPKDREDqOwrIa7XlsfaMdFR3Hf\nRTOI1jwZMgApNEQ64ZzjJy+uobymZaiQ604ez/jBqR5WJeIdhYZIJ/6ysoC/ry8MtCcPTeWaE8d5\nWJGItxQaIh3YW17D7S+vDbSjo4z7vzKTuBj92sjApXe/SDuaT0uVVtcH1n3nxLHMHDHIw6pEvKfQ\nEGnHy6sKeGtdy2mpiUNS+L6mbxVRaIgcrLCshtv+0va01M8vnkV8TLSHVYn0DwoNkVacc9z0/Kdt\nTktdc4JOS4k0U2iItLJoyXbe21QUaE8emsr1p+q0lEgzhYaI37Z9lfzs1ZZOfLHRxkOXzNZpKZFW\nFBoi+IY8v+G5VVTXNwbW/fC0iUzNTfOwKpH+R6EhAvzm3S0syz8QaM/Ny+CaE9SJT+RgCg0Z8Jbl\nF/OrtzcH2klx0Tx48SyNLSXSDoWGDGjlNfVc/78r28zE91/nTmN0drJ3RYn0YwoNGbCcc9z2l7Xs\nPFAdWPflGUO5eO4ID6sS6d8UGjJgPb9sJy+u2BVoD0tP4O4LZmgmPpFOKDRkQNqyt7xNr28zeOiS\n2QxKivOwKpH+T6EhA05NfSP//syKNrfXXnfyBI4Zl+VhVSLhQaEhA84dr6xjY2F5oH30mEyu12CE\nIkFRaMiA8udlO3l2yfZAOzM5jl9+9QjdXisSJIWGDBjrCspY+OLqNusevGQWQ9MTPKpIJPwoNGRA\nKK2u59pnllHb0BRY972TxnPSpMEeViUSfhQaEvGamhw3/Gkl+furAuuOn5DND0+b6GFVIuFJoSER\n7xd/38Tf1+8NtHPTE3QdQ6SLFBoS0f766W5+9c6WQDsuOorffn0umcnqjyHSFQoNiVjrCsq48blV\nbdb97ILpzB6pWfhEukqhIRGpqLyWq55a2qYD3xVfGM0l80Z6WJVI+FNoSMSprmvk208tZVdJy0CE\nXxifxU++PMXDqkQig0JDIkpTk+M//rSSVTtKAuvyspL4zdfmEBOtt7tId+m3SCLKfW9s4PU1ewLt\n9MRYnrj8SDJ04VukRyg0JGI8+cHnPPre1kA7Ntp49LK5jMtJ8bAqkcii0JCI8PKqAu54dV2bdfdd\nNJP5YzVyrUhPUmhI2Ht/cxE3/GklrtWUrf9x2kQunKMZ+ER6mkJDwtry7Qe45o/LqG9sSYxvHJPH\ndSeP97Aqkcil0JCwtXpnKd98YglVdS19Mc6aOYzbz5mmKVtFeolCQ8LS+t1lXPbEYsprGgLrjhuf\nzUOXzNKYUiK9SKEhYWdTYTmX/W4xJVX1gXVHjs7gsW/MJT4m2sPKRCJfjNcFiIRiXUEZX//dYoor\n6wLrZo0cxBOXH0lSnN7OIr1Nv2USNj7dWcJlv1tCaXXLN4ypw9J46oqjSE2I9bAykYFDoSFh4ZNt\nxXzryU/aXMOYMTydp751FOlJCgyRvqLQkH7vrXWFfG/R8jZTtR4xahBPXnEU6YkKDJG+pNCQfu1P\nS3dwywuraWxq6Ydx1OhMnrjiSFLi9fYV6Wv6rZN+yTnHr97ewsN/39Rm/SmTB/ObS+eQGKe7pES8\noNCQfqemvpGb//wpL60saLP+ojkjuPeiGcRqiHMRzyg0pF/ZV1HLd/64jKX5B9qsv+aEsdz8pcnq\n6S3iMYWG9Bsrd5Rw7dPL2F1aE1gXHWXced40Fhyd52FlItJMoSH9wv99sp1bX1pLXWPLHVKpCTH8\ndsEcjp+Q42FlItKaQkM8VVnbwO0vr+X5ZTvbrB+dlcTj35jHhCGpHlUmIu1RaIhn1hWU8b1nl7O1\nqLLN+lMmD+ahf5utPhgi/ZBCQ/pcU5Pj9x9u476/baCuVYc9M7j+lAl8/+QJRGmkWpF+SaEhfWpH\ncRU3PreKxZ8Xt1mfnRLPL/5tNsdNyPaoMhEJhkJD+kRjk2PR4nzufX0Dla0mTQI4YWIOD148i5zU\neI+qE5FgKTSk163fXcYtL6xm5Y6SNuvjY6L40RmT+NYXxuh0lEiYUGhIrymrqefXb2/m9x9so6HV\n2FHgmwPjwYtnMX5wikfViUhXKDSkxzU1OZ5ftpP739jAvoq6NtsSYqO4/pSJXHX8GGI0HIhI2FFo\nSI9xzvHepiLu/9tG1u0uO2T78ROyuev8GYzKSvKgOhHpCQoN6RHL8ot54I2NfLy1+JBtw9ITuOXL\nUzhn5jCNHSUS5hQa0i2Lt+7n1+9s4V9b9h2yLT4mimtOHMe1J47TUOYiEUKhISFranK8vWEvj/9z\nK0u2HfrNIsrgwjkj+OFpExk+KNGDCkWktyg0JGgVtQ28uHwnv/9gG1v3Vba7z6lThnDTmZOYqDGj\nRCKSQkMOa/3uMp5ZnM+Ly3cd0jGv2RnThnDdyROYPjy9j6sTkb6k0JB2Hais4+VVBTy3bAdrdh16\nJxRATJRx7qxcrjphLFOGpfVxhSLiBYWGBFTWNvDWukJeXlXAPzcVHdIhr1l6YixfPWoklx87mmHp\numYhMpAoNAa44so6/r6+kDfX7uGfm/e1GXX2YDNHpHPZ/DzOmZVLQqzuhhIZiBQaA0xjk2NtQSnv\nbSzinY17WbmjBNf+FwoAMpJiOW/2cC6eN4JpubpeITLQKTQiXFOTY/PeCpZsK+bDLfv48LP9lFbX\nd/ozCbFRnDJlCOfMzOWkyTnEx+hbhYj4KDQiTGVtA5/uLGXFjgMszy9haX4xJVWdhwRAUlw0J00a\nzOnThnDKlCGkxOutISKHCvkvg5kNBW4BzgaGA6XAEuAXzrm3u1pIbx03kh2orGPDnnLW7y5jza5S\nVu8q5bOiCjq4fn2I4YMSOXnyYE6anMOx47J1nUJEDiuk0DCzmcA7QJZ/VRmQje8P/VlmttA5d2+o\nRfTWcSOBc459FXVsLapg675KNhdWsHlvOZsLK9hTVhPSsVITYpg/NovjxmfzhfFZjMtJ0VhQIhKS\noEPDzBKBl/H9YV8BXOacW2tmacBtwA3A3Wa23Dn3ptfHDSeVtQ3sLq1m54GWx/biSvL3V7F9fxXl\ntQ1dOu7g1HjmjMrgqDGZHDUmkynD0ojWZEci0g2hfNO4BsgDKoBznHO7AJxzZcCNZjYOOB+4Bwjl\nj3tvHddTzjnKahrYX1HL/so69pXXUlRRS1F5LYVlNRSW+ZYFJdWU1XQtFFpLS4hh+vB0ZgxPZ8aI\ndI4YlUFueoK+SYhIjwolNBb4l4ua/7Af5AF8f9znmNkk59xGj4/bbfWNTVTWNlDR/KhpoLymgbKa\nespqGiirrqesup7S6npKquopqa7jQGU9B6rqOFBVR31jkBcXQhAbbYzNTmH8kBQmDUllyrA0Jg9N\nZURGogJCRHpdUKFhZqnAXH/zjQ52+xjfxet04BTgsH/ce+u4odpdWs21Ty+nqq6BytpG37KusdOO\nbr0pOsrIHZRAXmYyIzOTGJudzJjsZMbkJDMqM4lYzXgnIh4J9pvGFKD5Y+za9nZwzjWZ2UbgKGCq\nx8cNiWGs3FHSG4c+9LUMspLjGZIWz+DUeIYNSiQ3PYFh6YkMz0hkREYiQ9MSNBWqiPRLwYbGsFbP\nCzrZr3nbsE726YvjhiQpvuu3msZFR5GeFMugxFjSE2PJSI4jIymWjKQ4slLiyEyOJysljpyUeHJS\n48lMjtM3BREJW8GGRnKr59Wd7FflX6b05XHN7GrgaoBRo0YF+dItkmKjOX5CNslxMSTFR/uWcdEk\nx8eQHB9DSnw0KfGxpCTEkJoQQ1pCDGkJsaQlxhIfE6VrCSIyYEREt1/n3GPAYwDz5s0L+epzTHQU\nf7zy6B6vS0Qk0gR7nqT1NG2djYWd5F9WeHxcERHpBcGGRuvrDbmd7Ne8bbfHxxURkV4QbGhsAJpP\n+0xrbwcziwIm+ZvrPD6uiIj0gqBCwzlXDiz1N0/rYLej8fWlAAhqgMHeOq6IiPSOUO79XORfLjCz\n9m59vdG/XBZir+3eOq6IiPSwUELjUSAfSAVeNbOp4OvVbWb3Axf691vY+ofMbLSZOf/j8p46roiI\n9L2gb7kiwMoRAAAEsklEQVR1zlWb2Xn4ThHNAdaaWRm+vhNR+K5NLAx1JNreOq6IiPQ8c51NEN3e\nDxw6WVIZvsmSHm5vsiQzGw187m9e4Zx7sieO20l9Rfi+uXRVNrCvGz8v0hm9v6Q3def9leecyznc\nTiGHRqQzs6XOuXle1yGRSe8v6U198f7SIEgiIhI0hYaIiARNoXGox7wuQCKa3l/Sm3r9/aVrGiIi\nEjR90xARkaApNEREJGgKjU6Y2Rdb9Wbv7JHtda3Sf5nZUDP7pZl9ZmY1ZlZoZq+Y2Sle1ybhy8wu\nD+JvU49PJxERkzD1gSag6DDbRQ5hZjOBd4As/6oyfB2wzgbOMrOFzrl7vapPIkI9UNzBtsoO1neZ\nQiM4O5xzo70uQsKLmSUCL+MLjBXAZc65tWaWBtwG3ADcbWbLNUyOdMOHzrkv9tWL6fSUSO+5BsjD\nN+PkOc65tQDOuTLn3I3AS4AB93hXokhoFBoivWeBf7nIOberne0P+JdzzGxSO9tF+h2FhkgvMLNU\nYK6/+UYHu30MlPqf66K4hAWFRnByzGy5mVX6H5vM7DEzm+F1YdJvTcF36glgbXs7OOeagOaJxab2\nRVESkaaZ2VozqzazcjNbY2YPm9mY3ngxhUZwkoAjgFp8Nw9MAK4CVpjZjZ39oAxYrWehLOhkv+Zt\n7c1aKRKMbHwfUqqABGAa8AN8cxNd2tMvptDoXAm+887zgETnXCa+ADkR+BCIBh7ojX8YCXvJrZ5X\nd7JflX+Z0ou1SGQqAG4HpgMJzrksfO+js4B1QCLwBzM7oSdfNOLGnjKz2/DdztgV9znnfhLk68Th\nu//+C8BOfBOYqL+GAOD/IPGMvxnrnGvoYL9ngEuBN51zZ/RVfRLZzCwdWAqMBz5yzh3bU8eOxG8a\nUfi+AXT1ERTnXB1wq785At/pK5FmrTtVJXayX5J/2eM9d2Xgcs6VAnf7m/N7ctSKiAsN59x/Oees\ni4+bQ3y5xa2ej+3J/w4Je62vY+R2sl/ztt29WIsMTM1/nwzosYviERcaIv3EBqD53O+09nYwsyig\nuX/Gur4oSqS7FBrdc3Sr5597VoX0O865cnznlAFO62C3o4F0//O3e70oGWha/33a1lMHVWh0wsys\nk22xwJ3+5m5geZ8UJeFkkX+5wMzau6W2+XbtZc65je1sF2lXZ3+b/NvTgObT7Uucc50NuBoShUbn\nVpvZd81sbPM/kplFm9lx+D4ZHuff7xbdOSXteBTIB1KBV81sKvh6i5vZ/cCF/v0WelSfhK88M/vQ\nzL5pZsObV5pZnJmdCXwATMQ3AvctPfnCEXfLbU8ys9b/c2qBciANiPOvawD+0zl3X1/XJuHBzGbh\n+4DRemj0FHwf2BygodElZGY2mranxKvx3bGXDsT611UB33HO/bFHX1uh0TEzuxpfP4y5wGAgA98/\nzjbgPeAR55wuYEqnzGwovk97ZwPD8QXHEuBh55yuZUjI/MPufxvf2Y5ZQA6+D7SVwGZ8H1Qecc7l\n9/hrKzRERCRYuqYhIiJBU2iIiEjQFBoiIhI0hYaIiARNoSEiIkFTaIiISNAUGiIiEjSFhoiIBE2h\nISIiQVNoiIhI0BQaIiIStP8PTZREMMbrEt8AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "xs = np.linspace(-5, 5, 100)\n", "plt.plot(xs, sigmoid(xs), **kwargs);" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def logistic_model(weights, data):\n", " \"\"\"Logistic model.\"\"\"\n", " return sigmoid(data.dot(weights))\n", "\n", "def log_likelihood(weights, data, labels):\n", " \"\"\"Normalized negative log likelihood.\"\"\"\n", " scores = np.dot(data, weights)\n", " return -np.mean(labels*scores - np.log(1 + np.exp(scores)))\n", "\n", "def log_likelihood_gradient(weights, data, labels):\n", " \"\"\"Gradient of negative log likelihood.\"\"\"\n", " predictions = logistic_model(weights, data) \n", " return -data.T.dot(labels - predictions)/len(data)\n", "\n", "def log_likelihood_hessian(weights, data, labels):\n", " \"\"\"Hessian of negative log likelihood.\"\"\"\n", " predictions = logistic_model(weights, data)\n", " diag = np.diag(predictions * (1 - predictions))\n", " return data.T.dot(diag.dot(data))/len(data)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true }, "outputs": [], "source": [ "m, n = 1000, 100\n", "data = np.random.normal(0, 1, (m, n))\n", "wopt = np.random.normal(0, 1, n)\n", "labels = (data.dot(wopt) > 0).astype(np.float)\n", "gradient = lambda w: log_likelihood_gradient(w, data, labels)\n", "hessian = lambda w: log_likelihood_hessian(w, data, labels)\n", "init = np.zeros(n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's get a baseline solution out of sklearn's logistic model." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.052474218544034071" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cls = linear_model.LogisticRegression()\n", "cls.fit(data, labels)\n", "weights = np.reshape(cls.coef_, cls.coef_.shape[1])\n", "baseline = log_likelihood(weights, data, labels)\n", "baseline" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ws_gd = gradient_descent(init, np.ones(1000), gradient)\n", "ws_newton = newton_method(init, np.ones(10), gradient, hessian)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAH/CAYAAACmZYytAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8VPW9//H3N5N9m5CEEAiEXUBARRFFRXHXKlbb291a\n9Fexdam9XW7tvfa2V3tvvba91e4ubalal9YNUautIIgKKiDILsoOCSH7RrbJ9/fHTJKZkJVJcmbO\nvJ6PxzySs8185pvtne/5nvM11loBAABEmzinCwAAADgehBgAABCVCDEAACAqEWIAAEBUIsQAAICo\nRIgBAABRiRADAACiEiEGAABEJUIMAACISvFOFwC/3NxcO27cOKfLAABgyKxbt67UWjv8eI8nxESI\ncePGae3atU6XAQDAkDHG7A3neE4nOcwYs8AY82BVVZXTpQAAEFUIMQ6z1i611i7yer1OlwIAQFQh\nxAAAgKhEiAEAAFGJEAMAAKISIQYAAEQlQgwAAIhKhBgAABCVCDEAACAqEWIAAEBUYtoBAIhQ1lrV\n1NSourpa9fX18vl8TpcEhPB4PEpNTVVmZqYyMjJkjBnS1yfEAEAEstaqpKREdXV1ys7OVn5+vjwe\nz5D/kQC6Y62Vz+dTbW2tSktLdfToUeXl5Q3p9yghBgAiUE1Njerq6jR27Fh5PB6nywGOYYxRfHy8\nsrKylJGRob1796qmpkaZmZlDVgNjYgAgAlVXVys7O5sAg6jg8XiUnZ2t6urqIX1dQgwARKD6+nql\np6c7XQbQZ+np6aqvrx/S1+R0ksu0+Fp1tNmn1lbJZ60ykuOV4CGrAtHG5/PRC4Oo4vF4hnzwOSHG\nYcaYBZIWTJo0aUCe742dR3TD4rXty8/fcrZOGZM1IM8NYGgxiBfRxInvV/5Fd5i1dqm1dpHX6x2Q\n54vr9E3ka7UD8rwAAEQaQozLxMeFfkkJMQAAtyLEuEynDEOIAQC4FiHGZTr3xLRaQgwAdGfx4sUy\nxmj+/Pn9Ou5HP/qRjDFauHDhoNSFviHEuEznC5Fa6IkBALgUIcZlOg/sbSXEAABcihDjMp1PJ9ET\nAwBwK0KMyzCwFwAQKwgxLsPAXgCxrqmpSffff7/OOussZWVlKSEhQSNGjNDJJ5+sW265RatXr+7z\ncz366KNKSEhQXFycfve73/WrjjfffFOf//znNXr0aCUlJSknJ0cXXXSRnnjiCdlufjevX79ed9xx\nh8455xwVFha2Hzd//nw9/PDD3d4RN3igcWtrq379619rzpw5ysrKkjFGGzZskCQtXLhQxhj96Ec/\nks/n03333aeTTz5Zqampys7O1pVXXqm1a9d2+RqRiDv2ugwDewHEspaWFl1yySVauXKlJP9dZL1e\nr8rKylRSUqIPPvhAZWVlmjt3bq/P9Zvf/Ea33XabPB6PHnnkEV177bV9ruN73/ue7r333vblzMxM\nVVRUaNmyZVq2bJleeOEF/eUvf1Fcp388L7nkEpWVlUmSUlNTlZqaqvLycq1cuVIrV67Uc889pyVL\nlig+vus/39ZafepTn9KSJUvk8XiUkZHR5X4tLS264oor9OqrryohIUFJSUmqqKjQSy+9pGXLlmn5\n8uV9aiOn0RPjMp7OPTGEGAAx5PHHH9fKlSuVmpqqRx99VPX19aqoqFBjY6P27t2rX//61zr55JN7\nfZ7//u//1q233qrExEQ9/fTT/Qow999/v+69916NGDFCDz74oCorK1VVVaW6ujo9+eSTys/P15NP\nPqn//d//PebYSy65RE888YSKiopUV1eniooK1dbW6tFHH1V+fr5efvll/eIXv+j2tZ999lm98sor\n+u1vf6vq6mpVVFTo8OHDmjBhQsh+v/nNb/Tee+/pqaeeUm1trWpqarRx40bNmDFDDQ0Nuv322/v8\nfp1ET4zLeDpdnURPDOBCf79DKt7kdBUDI3+mdPk9A/Z0a9askSRdd911IcHD4/GosLBQt9xyS6/P\n8d3vflc/+9nPlJaWpiVLlujCCy/s8+tXVlbqzjvvVHJysl599dWQwJSSkqLPfe5zKiws1Nlnn62f\n/vSn+va3v63ExMT2fR5//PFjnjMtLU3XXnutxo4dq3PPPVe//e1v9d3vfrfL16+trdUDDzygRYsW\nta/Ly8vrss5Vq1bpnHPOaV930kknafHixZo9e7bee+897du3T4WFhX1+704gxLhM54G99MQALlS8\nSdr7ptNVRKTMzExJUlFRUb+PbW1t1U033aSHH35YWVlZevnll/t9SuWZZ55RbW2trrzyym57fObO\nnavx48dr165dWrduXZ9fY968ecrKytKePXt06NAhjRo16ph9cnJydMMNN/TpuYIDTJvTTjtNo0eP\n1oEDB7R58+aIDzGcTnIZLrEGEMsuv/xySdKSJUt01VVX6dlnn20fY9KT5uZmfeELX9DDDz+svLw8\nrVix4rjGhLz99tuSpOXLlys/P7/bx/79+yWp/WOwv/3tb7r66qtVWFiolJQUGWPaH5WVlZKkQ4cO\ndfn6s2fP7na8TLDTTz+9220FBQWSpIqKil6fx2n0xLjMMZdYc3US4D75M52uYOAM8Hs577zzdNdd\nd+muu+7S0qVLtXTpUknS1KlTdcUVV+imm27S5MmTjzmuLXxI0tNPP92ncTNdaesBqq+vV319fa/7\nB+/T0tKiz372s3ruuefa1yUlJSk3N1cej0eSdOTIEbW2tqqurq7L5xs+fHif6uxuwK8kJScnS/IH\nu0hHiHGZYy6xpicGcJ8BHEPiRj/4wQ907bXX6qmnntKKFSu0evVqbd++Xdu3b9f999+vP/zhD7ru\nuutCjpk5c6Z8Pp+2bt2qm2++Wa+//rpyc3P7/dqtra2SpNtvv1333Xdfv4596KGH9Nxzzyk1NVX3\n3HOPrrnmGo0ePTpknzFjxujAgQPdXqLdFnZiBaeTXIaBvQAgjR8/XnfccYdeeeUVlZeX6/XXX9e5\n556rlpYW3XzzzSopKQnZPzs7W8uWLdOUKVO0efNmXXzxxcd1OmXEiBGSpH379vX72L/97W+S/CHs\ntttuOybA+Hw+lZaW9vt53YwQ4zIM7AWAUB6PR/Pnz9eLL76ohIQE1dXVdXlDt/z8fC1btkwTJ07U\nhg0bdMkll6iqqqpfr9U2jmbFihU6evRov449cOCAJGnWrFldbn/rrbfU0NDQr+d0O0KMyzCwF0As\na2pq6nZbYmJi++mWxsbGLvcpKCjQ8uXLNW7cOK1du1aXX365amtr+/z6n/nMZ5SWlqaKigrddddd\nPe7buafH6/VKkjZtOvby+ZaWFt155519riNWEGIcZoxZYIx5sL9pvzvH9MQwsBdADLnuuut0/fXX\n69VXX1VNTU37+j179ugrX/mKGhoalJKSonnz5nX7HIWFhVq+fLlGjx6t1atX64orrujTIF3Jf4nz\nT37yE0nSPffcoxtvvFEffvhh+/b6+nq9/vrrWrRokc4666yQYy+++GJJ0t13360lS5a0TzGwfft2\nLViwQO+++67S0tL61hAxghDjMGvtUmvtorYEHq7OPTFMAAkgljQ0NGjx4sW67LLL5PV6NWzYMKWl\npWn8+PF66qmn5PF49MADD/Q6aHf8+PFavny5Ro4cqTfeeENXXXVVn0/l3Hbbbbr77rtljNHDDz+s\nKVOmKD09XdnZ2UpPT9cFF1yghx566Jjn+853vqOJEyequrpaV199tVJSUuT1ejVt2jT985//1O9/\n//vjGmzsZoQYl4kLHdfL6SQAMeWee+7Rvffeq8suu0wTJkxQU1OTfD6fJk6cqOuvv17r16/Xl7/8\n5T491+TJk7V8+XKNGDFCy5Yt0zXXXNPtaajO7rzzTm3cuFGLFi3S5MmT2y+LHjVqlC699FLde++9\nWrVqVcgx2dnZWrNmjb7+9a+3D+pNSUnR1VdfrZUrV2rhwoX9aotYYLq7TAtDa/bs2XagZg6d8P2X\n1JZdbj1/kr5z6ZQBeV4AQ2fbtm2aNm2a02UA/dLf71tjzDpr7ezjfT16Ylwo+JQSPTEAALcixLhQ\n8LAYBvYCANyKEONCwT0xDOwFALgVIcaFggf3EmIAAG5FiHEhT1CKIcQAANyKEONCHgb2AgBiACHG\nhTzBA3sJMQAAlyLEuFDIwF6uTgIAuBQhxoWCx8S0+FodrARAOLgZKaKJE9+vhBgXSvB0hJhmTicB\nUcnj8bRPAAhEA5/P1z5L+FAhxLhQQtCgmOYWemKAaJSamqra2lqnywD6rLa2VqmpqUP6moQYFwoJ\nMZxOAqJSZmamysvL6Y1BVPD5fCovL1dmZuaQvm78kL4ahkTw6SQusQaiU0ZGho4ePaq9e/cqOztb\n6enp8ng8Msb0fjAwBKy18vl8qq2tVXl5udLS0pSRkTGkNRBiXCi4J6aJ00lAVDLGKC8vTzU1Naqu\nrlZJSQm9Mog4Ho9Hqampys3NVUZGxpCHbEKMC3E6CXAHY4wyMzOHvIseiBaMiXGhkKuTfJxOAgC4\nEyHGheiJAQDEAkKMCyXEE2IAAO5HiHGhxJCeGE4nAQDciRDjMGPMAmPMg1VVVQP2nPFxwWNi6IkB\nALgTIcZh1tql1tpFXq93wJ4z9HQSPTEAAHcixLhQIgN7AQAxgBDjQqGXWBNiAADuRIhxoXh6YgAA\nMYAQ40IJna5OspZxMQAA9yHEuFCiJ3TuCiaBBAC4ESHGhYJ7YiROKQEA3IkQ40LHhJgWemIAAO5D\niHGhhE6nk5pb6YkBALgPIcaFOJ0EAIgFhBgX4nQSACAWEGJcKHjaAUlqoicGAOBChBgXSojrfIk1\nIQYA4D6EGBfidBIAIBYQYlyI00kAgFhAiHGhYy6xJsQAAFyIEONCnU8ntfg4nQQAcB9CjAtxnxgA\nQCwgxLhQ59NJjIkBALgRIcaFEumJAQDEAEKMCzEmBgAQCwgxLhTP6SQAQAwgxLgQp5MAALGAEONC\nx96xlxADAHAfQowLccdeAEAsIMS4UHKnENPQTIgBALgPIcaF4j1xIfeKOdrsc7AaAAAGByHGYcaY\nBcaYB6uqqgb0eZPjPe2fNxBiAAAuRIhxmLV2qbV2kdfrHdDnTU4MDjGcTgIAuA8hxqWSEzq+tPTE\nAADciBDjUpxOAgC4HSHGpVKCTicxsBcA4EaEGJeiJwYA4HaEGJdKChkTw8BeAID7EGJcKiWBnhgA\ngLsRYlwqmRADAHA5QoxLhfbEcDoJAOA+hBiXCr5PDFcnAQDciBDjUpxOAgC4HSHGpYJDTGNLq1pb\nrYPVAAAw8AgxLhUcYiR/kAEAwE0IMS4VPCZG4pQSAMB9CDEuldKpJ4bBvQAAtyHEuFTn00n0xAAA\n3IYQ41KdQww9MQAAtyHEuNSxY2IY2AsAcBdCjEt1HhPD6SQAgNsQYlwqLSk+ZLm2scWhSgAAGByE\nGJfqHGLqCDEAAJchxLhUWlLo6SRCDADAbQgxLpV+zOkkxsQAANyFEONSKQkexZmOZXpiAABuQ4hx\nKWOM0hI7emMY2AsAcBtCjIsFD+6tbyLEAADchRDjYsGDe+sYEwMAcBlCjIsFD+7ldBIAwG0IMS4W\nfDqJgb0AALchxLhYKgN7AQAuRohxsfTgMTEM7AUAuAwhxsVCTycxsBcA4C6EGBdjYC8AwM0IMS4W\n3BPT1NKqZl+rg9UAADCwCDEudsz8SQ30xgAA3IMQ42KZKQkhy9UNzQ5VAgDAwCPEOMwYs8AY82BV\nVdWAP3dmcmhPTPVRemIAAO5BiHGYtXaptXaR1+sd8Of2duqJqTpKTwwAwD0IMS7G6SQAgJsRYlys\nc4ihJwYA4CaEGBfrfDqpmhADAHARQoyLpSV65Ikz7cv0xAAA3IQQ42LGmJArlBgTAwBwE0KMywWP\ni6niEmsAgIsQYlwueFwMY2IAAG5CiHG5zOTgnhhCDADAPQgxLkdPDADArQgxLudN7QgxlYQYAICL\nEGJcLjs1sf3zivom+Vqtg9UAADBwCDEul53WEWKslSrrmxysBgCAgUOIcbmc9MSQ5fI6QgwAwB0I\nMS4X3BMjSWWEGACASxBiXK5ziKEnBgDgFoQYl8tJSwpZJsQAANyCEONyw9JCZ7ImxAAA3IIQ43JJ\n8R6lJ3VMAkmIAQC4BSEmBgSPi2FgLwDALQgxMSA4xJTXNTpYCQAAA4cQEwNygntiaumJAQC4AyEm\nBgT3xFRwx14AgEsQYmJAdnrw6aQmWcv8SQCA6EeIiQHBk0A2+6xqGlscrAYAgIFBiIkBuemhN7w7\nUsPgXgBA9CPExIARmckhy4erGxyqBACAgUOIiQEjMkN7Ykqq6YkBAEQ/QkwMyKMnBgDgQoSYGJCZ\nHK+UBE/78mF6YgAALkCIiQHGmJBTSodr6IkBAEQ/QkyMCD6ldLiKEAMAiH6EmBgRfIUSPTEAADcg\nxMSIERlBp5OqG7lrLwAg6hFiYkRwT0xTS6uqjjY7WA0AAOEjxMSIvE73iuEKJQBAtCPExIjOd+0t\n5l4xAIAoR4iJESO9oSGmqPKoQ5UAADAwCDExYqQ3RcZ0LB+oIMQAAKIbISZGJMbHKS/oCqWD9MQA\nAKIcISaGjB6W2v75QXpiAABRjhATQwqyUto/P1BR72AlAACEjxATQwqGdYSY4uoGNftaHawGAIDw\nEGJiyOigENNqpWLmUAIARDFCTAwJPp0kcYUSACC6EWJiSPDAXokrlAAA0Y0QE0OO7YlhcC8AIHoR\nYmJISqJHw4PuFbO3jBADAIhehJgYMz4nrf3z3aV1DlYCAEB4CDExZnwuIQYA4A6EGIcZYxYYYx6s\nqqoaktcbFxRiqo42q6KuaUheFwCAgUaIcZi1dqm1dpHX6x2S1wvuiZGkXfTGAACiFCEmxnQOMXsI\nMQCAKEWIiTFjc1JlTMcy42IAANGKEBNjkhM8GuXtuF8MIQYAEK0IMTGIK5QAAG5AiIlB43I7ph/Y\nU1Yna62D1QAAcHwIMTFofG56++f1TT4drm50sBoAAI4PISYGTRgeeoXSzpIahyoBAOD4EWJi0NT8\njJDlHcWEGABA9CHExKD8zGRlJMe3LxNiAADRiBATg4wxIb0xOw4TYgAA0YcQE6OmBIWYDw/XqLWV\nK5QAANGFEBOjpozoCDENza3aV17vYDUAAPQfISZGTcnPDFnmlBIAINocd4gxxnwj8Bg1kAVhaAT3\nxEgM7gUARJ9wemJ+IelnkkoHqBYMIW9qgvIzk9uXCTEAgGgTTogplVRjrW0aqGIwtIIH924tqnaw\nEgAA+i+cELNektcYM3ygisHQmlngbf98d2mdqhuaHawGAID+CSfE/DJw/A8GqBYMsZmjvSHLmw9W\nOVQJAAD9d9whxlr7d0nfkfQ1Y8yjxpiTB64sDIXgnhhJ2nSAEAMAiB7xve/SNWPMrsCnLZK+KOmL\nxpijksok+bo5zFprJx7va2JgjfQmKzc9UaW1/mFNH9ATAwCIIscdYiSN62JdauDRHW4LG0GMMZpZ\n4NXrO45IoicGABBdwgkx5w9YFXBMcIjZV16vyvomZaUmOlwVAAC9O+4QY61dOZCFwBkzR2eFLG86\nWKV5k7ngDAAQ+Zh2IMad1OkKpQ37Kh2qBACA/gnndFIIY4yRNEVS27/xRyTtsNYyDiaCjchMVkFW\nig5WHpUkrd1b4XBFAAD0Tdg9McaYScaYxZKqJG2RtCLw2CKpyhjzJ2PMpHBfB4PntLHD2j9fv69C\nra3kTgBA5AsrxBhjrpL0vqQvS0qXZDo90iVdJ+l9Y8yV4ZWKwRIcYmoaWrSzpNbBagAA6JtwZrGe\nKOlJSWmSdkm6SdJkSSmBx2RJX5P0cWCfvwaOQYQJDjGStHZvuUOVAADQd+H0xPybpGRJr0s6yVr7\nkLX2Y2ttY+DxsbX2QUknS1opKUnSd8MvGQNtan6G0hI97cvr9jAuBgAQ+cIJMRfLf/O6m6y1R7vb\nKbDtJvlPL10SxuthkMR74jSrsKM3hsG9AIBoEE6IGSmpylr7UW87Wms/lFQZOAYR6NSgU0r7yutV\nVNVtLgUAICKEE2LqJaUaYxJ629EYkyj/uBj+MkaouRNyQpbf/qjMoUoAAOibcELMJkkJkr7Sh32/\nEtj3gzBeD4NoVmGWkuI7vh3e+rjUwWoAAOhdOCHmUfnHufzSGPPVwM3uQhhjko0x35D0S/nHz/w5\njNfDIEpO8Oj0cdnty29/VCbuUwgAiGThhJg/Svqn/FcoPSDpgDHmSWPMz40xvzHGLJW0T9Iv5L8y\n6Z+SFodZLwbRWZM6TikVVzdod2mdg9UAANCzcCaAtMaYq+UPKV+Vf9DuZ+XvcZH8vTSS1Cp/yPk2\nUxBEtrMn5kra0b781sdlmjA83bmCAADoQVhzJwUun/6aMeYnkq6RdKpC505aL+lZa+2+sKrEkJhR\n4FVGcrxqGlokSW9/VKovnznW4aoAAOjagEwAaa3dK+m+gXguOMcTZ3TmhBz9c+thSdLqXWXytVp5\n4o4Z7gQAgOPCmXZgvTFmnTFmwkAWBGedMym3/fPK+mZtPFDpYDUAAHQvnIG9J0qabK3dNVDFwHnz\npwwPWV6+rcShSgAA6Fk4IeagOgbvwiXG5qRpUl7HYN5l2wkxAIDIFE6IeVX+O/aeMVDFIDJcMDWv\n/fNtRdU6VMmNlgEAkSecEPNjSWWSfm+Mye1tZ0SP4BAjScvpjQEARKBwrk6aJOk/JP1c0g5jzCOS\nVst/abWvu4OstW+E8ZoYAqeNHabM5HhVBy61fn17ia7lUmsAQIQJJ8SsUOiN7b4RePTEhvmaGAIJ\nnjidNyVPSzcekiS9+VGp6ptalJrIlw4AEDnC+au0Tx0hBi5z4dSOENPY0qoVO47oEzNHOlwVAAAd\nwpl2YNwA1oEIc8G0PCV64tTka5UkvbypiBADAIgo4QzshYtlJido3uSO8drLt5eoobnboU4AAAy5\ncO7YW2GMKeOOve51eVDPS32TTyt2cJUSACByhNMTkyjJwx173eviaSOU4Om4n+HLm4odrAYAgFDh\nhJh98gcZuJQ3NUFnB82ltGzbYU4pAQAiRjgh5gVJScaYiweqGESe4MG8dU0+/SMwwzUAAE4LJ8T8\nj6Q9kh4yxkwbmHIQaS6bka+k+I5vk2fXH3CwGgAAOoRzn5hPSvqdpP+U9L4x5u/q2x17HwnjNTHE\nMpMTdMn0/PZ7xrzx4RGV1DQoLyPZ4coAALEunBCzWP6b3bWN/Lwq8OgNISbKfOrUgvYQ02qlFzYc\n0lfncVEaAMBZ4YSYN8Qde2PCvEm5Gp6RpCM1jZKkp9cdIMQAABwXzh175w9gHYhg8Z44XX3KKD20\narckaXtxjbYeqtaJozIdrgwAEMu4Yy/65FOnjg5Z/uva/Q5VAgCAX59DjDHmG8aY/9fNtnRjTI//\nlhtjfmGM+UN/C0RkmDYyU9ODel6eWX9A9U0tDlYEAIh1/emJuU/SXd1s2ympvJfjPy9pYT9eDxHm\nS2eMbf+8pqGlfbAvAABO6O/pJHOc2+ACnzxllNKTOoZRPbpmr6xlbDcAwBmMiUGfpSXF61OnFrQv\nbz5YrY0HqhysCAAQywgx6JdrzxwbsvzYmr0OVQIAiHWEGPTLCSMyNGd8dvvyCxsPqbS20cGKAACx\nihCDfrtubkdvTFNLqx5dTW8MAGDoEWLQb5dNz1dBVkr78qNr9qqhudvpsgAAGBSEGPRbvCdON5wz\nvn25vK5JzzC7NQBgiPV32oFsY8zyrtZLUjfbQvaBO3zu9DG677UPVdPgv+HdH1bt1hdOL1RcHFfa\nAwCGRn9DTKKk+T1s72mbxISRrpGeFK8vnlGoB1bukiTtKq3TP7Ye1mUz8h2uDAAQK/oTYv48aFUg\nKl1/1nj98c3davb5s+mvlu/UpdNHyBh6YwAAg6/PIcZae/1gFoLok+9N1qdPHa0n3/NPBrnlULVe\n21aii08c4XBlAIBYwMBehOWW8ycpPmgczP3LPmQqAgDAkCDEICxjslP16VNHty9vPlitZdtKHKwI\nABArCDEDwBiTboz5L2PMy8aYI8YYa4y5w+m6hsot50+SJ6Q3Zie9MQCAQUeIGRi5kv5T0kxJ7ztc\ny5ArzEnVp2Z1TAy56WCVXt1y2MGKAACxgBAzMIokFVhrx0ha5HQxTrj1gtDemHtf2a5mX6uDFQEA\n3I4QMwCstY3W2kNO1+GksTlp+uKcwvblXaV1eipw1RIAAIOBEIMB840LJyst0dO+fN9rO1XX2OJg\nRQAAN4vYEGOMyTDGXGWMudsY83djTGlgwKw1xkzt43PkG2PuN8Z8bIxpMMYcNsYsNcZcONj1x6Lh\nGUladO7E9uXS2kY9tGqXgxUBANwsYkOMpAslLZF0p6TLJOX052BjzEmSNkv6hqQJkhrlH4B7paR/\ndnX1kPFL7uMjktvOMV+dN17DM5Lalx98Y5eKqxocrAgA4FaR/oe4RNLLkv5L/Rgwa4xJkfSC/MHn\nfUkzrLVeScMk/VySkfQ/xphLOh16tqSjfXyce9zvysXSkuL1rxed0L5c3+TT/7y8zcGKAABu1d8J\nIIfSUmvt820Lxphx/Tj2JkljJdVKWmCtPShJ1tpqSd8xxkyUdLWkn0j6R9BxH0rq6/QK2/tRT0z5\n7OzRemT1Hm0vrpEkvbDxkL54RqHOnNCvzjQAAHoUsSHGWusL4/AvBT4+3hZgOvmp/CHmVGPMFGvt\njsBrlkhaHMbrQlK8J053fXKGPvvA6vZ1P3phi1687RzFeyK98w8AEC1c9xfFGJMh6bTA4qvd7LZG\nUlXgcwb5DoI547P1yVNGtS9vL67RY2v2OlgRAMBtIrYnJgzT5B/zIklbutrBWttqjNkhaY6kEwfi\nRY0xt0rKCjwk6XxjTFv7/spaW9X1ke7175+Ypte2HlZdk79T7ef/+FCXzRipfG+yw5UBANzAdT0x\nkkYGfd7TDejato3sYZ/++I6kuyV9O7B8SWD5bvkHFB/DGLPIGLPWGLP2yJEjA1RG5BiRmaxvXDi5\nfbmmsUV3Pr+ZeZUAAAPCjSEmLejzoz3sVx/4mD4QL2qtHWetNd089nRzzIPW2tnW2tnDhw8fiDIi\nzg3njNfTGqDXAAAgAElEQVSJIzPbl1/bdlgvbSpysCIAgFu4McQggiR44vS/nz5JQdMq6UcvbFFF\nXZNzRQEAXMGNIaYu6POUHvZLDXysHcRaIGnmaK9unDehfbm0tkk/fol7xwAAwuPGEBM8DmZUt3t1\nbOPcxhD45kUnaGxOavvyM+sP6J9bDztYEQAg2rkxxGyX1DZydHpXOwSmDJgSWNw6FEXFupREj35y\nzcyQdXc884GO1DQ6VBEAINq5LsRYa2skrQ0sXtzNbmdI8gY+XzboRUGSdNakXH1l7tj25bK6Jt3x\nzAdcrQQAOC6uCzEBjwc+fskY09Ul1N8JfFzXdrdeDI07Lp+micM7LiBbtr1Ej7+7z8GKAADRKqJD\njDEmt+2h0HutZAVv62JG6Qck7ZWUIelFY8yJgefLMMbcK+lTgf3+fbDfA0KlJHp0/+dnKT7ocqUf\nv7hNHx9hfDUAoH8iOsRIOhL0WB+0fnWnbYXBB1lrj0r6pKQySadK2mKMqZJUKem78o+Z+b61Nnjy\nRwyRGQVe/evFHTNdH2326Za/rNfRpnCmywIAxJpIDzHHzVq7UdIMSb+UtEtSkvyh5iVJF1tr73Gw\nvJj3tfMmas647Pbl7cU1+uELmx2sCAAQbSI6xPRwB9y+3hG32Fp7u7V2orU22VqbZ6290lrLYF6H\neeKMfvmFWcpOS2xf99e1B/T0ugMOVgUAiCYRHWLgbvneZN33uVNkgu7me+fzm7SjuMa5ogAAUYMQ\nA0ede8Jw3Xr+pPblhuZWff0v61Td0OxgVQCAaECIcZgxZoEx5sGqqiqnS3HMNy86QXMn5LQv7zpS\np28+uUG+Vu4fAwDoHiHGYdbapdbaRV6vt/edXcoTZ3T/F07RiMyk9nXLt5fop69yCx8AQPcIMYgI\neRnJeuDLs5UY3/Et+fuVH2vJhoMOVgUAiGSEGESMU8Zk6X8/HTq/0r89/YE27q90qCIAQCQjxCCi\nXDNrtBadO6F9ubGlVf/vz2u1v7zewaoAAJGIEIOI873Lpuq8E4a3L5fWNuorf3pXFXVNDlYFAIg0\nhBhEHE+c0a++OEtT8zPa1+06UqcbH1mrhmamJgAA+BFiEJEykxO0+Po5GulNbl+3dm+FvvXXDWrl\n0msAgAgxiGD53mQtvn6OMpLj29e9vKlY/7V0i6wlyABArCPEIKJNyc/QA18+TQmejrkJ/rx6r372\nD+4hAwCxjhCDiHfWxFz97DMnh6z7zesf67crPnKoIgBAJCDEICp88pQC3f3J6SHr7n1lhx5ZvceR\negAAziPEIGp8ee443XH51JB1/7lki/66dr9DFQEAnESIQVT52nkTddsFk0LWfe+ZD/Tku/scqggA\n4BRCDKLOty4+QdefPa592Vrpjmc36bE1e50rCgAw5AgxDjPGLDDGPFhVVeV0KVHDGKP/vPJEfWXu\n2JD1dz6/WX9+e48zRQEAhhwhxmHW2qXW2kVer9fpUqKKMUY/umq6bjh7fMj6H76wRQ+v2uVQVQCA\noUSIQdQyxugHV04LmTBSkn780jb93z8/5IZ4AOByhBhENWOMvn/5VN08f2LI+l8u26kfLNksH1MU\nAIBrEWIQ9Ywx+u6lU/SvF50Qsv6xNfv0jSfeV2MLk0YCgBsRYuAKxhjdftFk3fXJ6TIdMxTopU1F\numHxe6ptbHGuOADAoCDEwFWumztOv/z8rJC5lt76qEyff3C1Dlc3OFgZAGCgEWLgOgtOHqU/Ljxd\nqYme9nWbD1br6t+8pa2Hqh2sDAAwkAgxcKV5k4friRvPVHZaYvu6oqoGfeb3b2v59sMOVgYAGCiE\nGLjWyWOy9OzXz9KE3LT2dXVNPn31z2u1+K3dDlYGABgIhBi42rjcND1389maOyGnfV2rlX60dKvu\nfH6TmlpaHawOABAOQgxcz5uaoD/fMEefOW10yPrH1uzTlx5eo5IaBvwCQDQixCAmJMbH6d5/OUn/\ndtmUkPXv7anQgl+9qff3VThUGQDgeBFiEDOMMbp5/iQ98OXTlBZ05dLh6kZ97oE1euq9fQ5WBwDo\nL0IMYs6l0/O15NazQwb8Nvla9b1nNunfn9ukhmbu8AsA0YAQg5g0KS9Dz996ti6alhey/vF39ulT\nv31bu0vrHKoMANBXhBjErMzkBD345dn65kWTQ9ZvLarWlb9cpSUbDjpUGQCgLwgxDjPGLDDGPFhV\nVeV0KTEpLs7omxedoD98Zba8KQnt6+uafLr9yQ36/rMfcHoJACIUIcZh1tql1tpFXq/X6VJi2oXT\nRujl2+fp1MKskPVPvLtfV//mLX1UUuNQZQCA7hBigICCrBQ9ddNc3XTehJD124trdMUv39Sf394j\na61D1QEAOiPEAEESPHH6/uXT9KeFp2tYasfppcaWVv3whS267o/vMhs2AEQIQgzQhfOn5unl2+fp\njPHZIetX7SzVpfe9oZc3FTlUGQCgDSEG6MZIb4oev/FM/fsnpirR0/GjUlnfrJv/sl7femqDqhua\nHawQAGIbIQbogSfOaNG5E7Xk1rM1ZURGyLZn3z+oS3/xhl7fXuJQdQAQ2wgxQB9MG5mpJbeerRvn\njZcxHeuLqhp0/eL39K2nNqiirsm5AgEgBhFigD5KTvDoP644UX/56hka5U0O2fbs+wd18S9W6u+M\nlQGAIUOIAfrprIm5evVfz9WXzigMWV9a26Sv/2W9vv7YOpXUcAUTAAw2QgxwHDKSE/Tf18zUEzee\nqbE5qSHb/r65WBf9fKUeW7NXra3cVwYABgshBgjD3Ik5euX2c/XVc0LHylQ3tOjO5zfrmt+9rc0H\nmVICAAYDIQYIU0qiR3deeaKe+fpZmpyXHrJt4/5KXfXrN/VfS7eohsuxAWBAEWKAAXJq4TC99I15\n+u6lU5QU3/Gj1WqlP721Rxf930q9+MEhpi4AgAFCiAEGUGJ8nG45f5Je+9Z5On/K8JBth6sbdevj\n7+uLD72jbUXVDlUIAO5BiAEGwZjsVP1x4en6/bWnamSny7FX7yrTFb9cpTuf36Ry7i0DAMeNEAMM\nEmOMLpsxUq996zzdOG+8PHEdI39brfTYmn2a/9PX9cc3d6vZ1+pgpQAQnQgxwCBLS4rXf1xxol65\nfZ7mTc4N2Vbd0KK7Xtyqy+57Qyt2MH0BAPQHIQYYIpNHZOiRG+bo4etma1yne8t8fKROC//0nq59\n+B0uyQaAPjJcKeEsY8wCSQsmTZp0486dO50uB0OkscWnxW/t0a+Wf6TaxpZjtl99yih9+5IpGpOd\n2sXRAOAOxph11trZx308ISYyzJ49265du9bpMjDESmoa9NNXdujp9QfU+Ucx0ROn6+aO1S3nT9Kw\ntERnCgSAQUSIcQlCTGzbeqha97yyXW98eOSYbRnJ8bp5/iQtPGucUhI9DlQHAIODEOMShBhI0lsf\nleonf9+mzQePvY/M8Iwk3TJ/or5wRqGS4gkzAKIfIcYlCDFo09pqtfSDQ/rZP3Zof/nRY7aP9Cbr\ntgsm6zOzRyvBw9h8ANGLEOMShBh01tji01/W7NOvlu9URf2x8y6NyU7RNy6YrGtmFSieMAMgChFi\nXIIQg+7UNDRr8Vt79OCqXappOPZKpgm5abr9osm68qRRITfUA4BIR4hxCUIMelNV36yH39ylP765\nW3VNvmO2j8tJ1c3zJ+nqWQVKjKdnBkDkI8S4BCEGfVVe16QH3vhYj7y9V0ebjw0zo7zJ+tr8ifrs\n7DFKTmAAMIDIRYhxCUIM+utITaN+t+JjPfbOXjW1HDv3Um56khadO15fOmOs0pLiHagQAHpGiHEJ\nQgyOV0lNg/6warceW7O3y9NMWakJuv6s8Vp41jh5UxMcqBAAukaIcQlCDMJVUdekP729R4vf2q3q\nLgYApyZ69NnZY/T/zhnPdAYAIgIhxiUIMRgoNQ3NemzNPj28apfK6pqO2R5npMtnjtRN507QSaOz\nHKgQAPwIMS5BiMFAO9rk05Pv7dMDK3epuLqhy33OGJ+tRedO0PlT8hTH5dkAhhghxiUIMRgsTS2t\nWrrxkB5atUvbi2u63Gfi8DTdOG+Crp5VwBVNAIZMuCGGSxbc5u93SMWbnK4CESRR0qclfcprVZXY\nrKLKBlU1dLoDcJWkF6XNLxvlZSZrRGaSkjyEGQCd5M+ULr/H6SraEWLcpniTtPdNp6tABDKSsgIP\n9XQvvOrAAwAiHLf1BAAAUYmeGLfJn+l0BYhCLa2tOlLbqMNVDWro4sZ5khQfZ5SXETjVFM+pJiAm\nRdjfGEKM20TQuUpEj3hJIyWNaLVaufOIFr+1Rys/PHLsjg1SXKk0f0qevjinUOdPzWPSSQCO4eqk\nCMHVSYg0u47U6pHVe/X0ugOqbTz25nmSNNKbrM+dPkafP71Q+d7kIa4QQLTjEmuXIMQgUtU0NOvZ\n9Qf157f3aFdpXZf7xBnpwmkj9MUzCnXu5OH0zgDoE0KMSxBiEOlaW63e/rhMj7+7V//YclgtrV3/\n7ijIStEX5ozRZ2ePUV4mvTMAukeIiXLGmAWSFkyaNOnGnTt3Ol0O0CclNQ3629oDeuLdfTpQcbTL\nfTxxRvNPGK7PzB6jC6bmKTGeiyEBhCLEuAQ9MYhGvlarVTuP6C/v7NPy7SXyddM7k52WqGtmFegz\ns0dran7mEFcJIFIRYlyCEINoV1zVoKfe268n39unoqqu52qSpJkFXn129mhddXKBvKkJQ1ghgEhD\niHEJQgzcwtdqtfLDEv1t7QG9tu2wmn1d/45JjI/TpdPz9ZnTRuvsSbkMBgZiECHGJQgxcKPyuiYt\n2XBQf117QNuKup/LYERmkj55SoE+ecoonTgyU8YQaIBYQIhxCUIM3G7zwSo9ve6Ant9wUJX1zd3u\nd8KIdF09q0CfPKVABVkpQ1ghgKFGiHEJQgxiRWOLT69tLdHf1u3XGx8eUTdjgSVJZ4zP1jWzCnT5\nzJHypjB+BnAbQoxLEGIQi4qrGvTCxoN67v1DPZ5uSvTE6YKpebp6VoHOnzqcuZsAlyDEuAQhBrFu\ne3G1nn//kJZsONjj1U0ZyfG65MR8XXnySJ0zKVcJHu4/A0QrQoxLEGIAv9ZWq3d2l+v59w/q5U1F\nqulm3iZJykpN0KWBQDN3Qo7iCTRAVCHEuAQhBjhWQ7NPy7eX6Pn3D+r1HSXdXq4t+W+od9mMfF15\n0kidMT6HS7aBKECIcQlCDNCzyvomvbypWC9tOqTVH5f1OCB4eEaSPjEjX1ecNEqzxw5THIEGiEiE\nGJcgxAB9d6SmUa9sKdaLGw/p3T3l6unX2IjMJF06PV+XTc/XnPHZnHICIgghxiUIMcDxOVzdoJc3\nFenFD4q0bm9Fj/tmpSbo4mkjdNmMfJ09KVfJCVzlBDiJEOMShBggfIcqj+rlTUVa+kGRNu6v7HHf\ntESPzp+ap8tm5Gv+lDylJ8UPUZUA2hBiXIIQAwys/eX1enlTkV7ZUqz39/UcaBLj43Tu5FxdMj1f\nF08boWFpiUNUJRDbCDEuQYgBBk9xVYP+sbVYr2wu1ju7y+XrYVSwJ87o9HHDdNG0Ebpo2giNy00b\nwkqB2EKIcQlCDDA0yuua9Nq2w3p1c7FW7SxVk6+1x/0nDk/TRSeO0MXTRmhW4TAu3QYGECHGJQgx\nwNCraWjWih1H9MqWYr2+vUT1Tb4e989OS9T8KcN18bQRmnfCcMbRAGEixLgEIQZwVkOzT6t2luqf\nW4u1fHuJSmubetw/0ROnMyfm6KJpebpw2ghm3AaOAyHGJQgxQORobbXaeKBSr207rGXbSrS9uKbX\nY6aNzNT8KcM1/4ThOnXsMOZ0AvqAEOMShBggcu0vr9eybYf12rYSvbO7rMfpDyT/JJXzJudq/pQ8\nzT9huPIyk4eoUiC6EGJcghADRIfqhma98eERLdtWotd3lKiyvrnXY04cmanzpw7X/Cl5mjUmi7sG\nAwGEGJcgxADRp8XXqnV7K7Rse4lW7CjRh4drez0mMzle807wn3Y6b8pw5WXQS4PYRYhxCUIMEP0O\nVh7Vih0lWrHjiN76qLTXq50kafqoTM2bPFznTs7VaeOGKSmeqRAQOwgxLkGIAdylscWntXsqtGJH\niV7fcUQflfTeS5OcEKc543N07uRcnTM5V1NGZMgY7ksD9yLEuAQhBnC3/eX1WvnhEa3YUaK3PirT\n0ebee2mGZyRp3iR/oDlnUi4DhOE6hBiXIMQAsaOxxaf3dlfo9R0lenNnqXYc7v0Sbkmamp+hcwKh\n5ozxOUpJ5NQTohshJsoZYxZIWjBp0qQbd+7c6XQ5ABxwuLpBb+4s1aqdR/TmR2UqrW3s9ZhET5xm\njxumsybmaO7EXJ002su9aRB1CDEuQU8MAEmy1mp7cY1W7TyiVTtL9e7ucjW29Dy/kySlJXo0Z3y2\n5k7M0VkTczVtZCbzPCHiEWJcghADoCsNzf4Bwm2hZmtRdZ+O86Yk6MwJ2Zo7IUdnTcrV5Lx0Bgkj\n4hBiXIIQA6AvSmsb9dZHpVq1s1Rvf1SqQ1UNfTouNz1Rcyfm+kPNxByNzUkl1MBxhBiXIMQA6C9r\nrfaV1+vtj8u0+uMyvf1x38bTSNIob7LOnJijMyfk6Izx2SrMJtRg6BFiXIIQAyBc1lp9VFKr1bvK\n9PZHZVq9q0xVR3ufFkGSRmQmac74HM0Zn60zx2drEqefMAQIMS5BiAEw0FpbrbYWVWvNLn8vzTu7\nylTXh7sIS1J2WqJOHzdMc8b7e2oYKIzBQIhxCUIMgMHW7GvVpoNVWh04/bR2b7kamnu/8kmSMpLi\nNTsQauaMz9bMAq8S47mkG+EhxLgEIQbAUGtq8Yead3eX693dZVq7p0I1jS19OjYlwaNTx2Zpzrgc\nzR43TKeMyVJaUvwgVwy3IcS4BCEGgNN8rVbbiqr1TiDUvLu7XBX1fRtT44kzmjYyQ7PHZuu0scM0\ne9wwjfSmDHLFiHaEGJcgxACINK2tVh8fqdWa3eV6d3e53tlVppKavl39JEkFWSntgea0scM0NZ9x\nNQhFiHEJQgyASNd2Sfc7u8v1zq5yvbO7TAcqjvb5+LREj2YVDmsPNrMKhymdU1AxjRDjEoQYANGo\nuKpBa/eWa+2eCq3fV6Eth6rla+3b35U4I03Nz2zvqTlt7DAVZKVwaXcMIcS4BCEGgBvUN7Vow/5K\nrdtTobV7/cGmpqFvg4UlaXhGkk4Zk6VZhVmaNWaYThrtZcCwixFiXIIQA8CNWlutPiyp0do9FVq3\nt0Jr95Zrf3nfT0HFGWlKfqZmFWbplDFZOrUwSxNy0xXH2BpXIMS4BCEGQKwoqW4IBBr/Y8vBKrX0\n8RSUJGUkxwd6a4b5w83oLA1LSxzEijFYCDEuQYgBEKuONvn0wYFKvb+/Uhv2VWr9vop+XQUlSeNz\n0zSr7TRU4TBNyc9Qgoeb8UU6QoxLEGIAwM9aq6KqBr2/r1Lv76vQ+/srtelglZpa+nZ3YUlKio/T\niaMydfLoLJ002quTRmdpQm4ap6EiDCHGJQgxANC9ppZWbSuqbg81G/ZXam9Zfb+eIyMpXjMKvDpp\njLc93HA1lLMIMS5BiAGA/imrbdSG/ZX+Hpv9Fdq4v0q1fZw2oU1OWmJ7T83JY/wfc9OTBqlidEaI\ncQlCDACEx9dq9VFJrTbsr9DGA1X64EClthfV9GvQsOS/03BbsDlptFczR3uVmZwwSFXHNkKMSxBi\nAGDgNTT7tK2oWh8cqNLGA5X64ECVPj5Sq/7+6ZuQm6bpBV7NLMjUjFFeTR/llTeVYBMuQoxLEGIA\nYGjUNDRr88FqfRAINRsPVPZr+oQ2hdmpmlGQqemjvJpR4NWMUZnK4VRUvxBiXIIQAwDOKatt1AcH\nq/TBfv9pqI0HqlRa27/LvCVplDdZ0wu8mjHKq5mj/b02eZnJg1CxOxBiXIIQAwCRo+0y77ZAs/mg\n/1FR39zv5xqekaQZozI1s8DrDzgFXo3yJnNVlAgxrkGIAYDIZq3VoaoGbT5YpS0Hq7T5ULU2HazS\nkX7emE+SstMSNX1Upk4clakTR/of43PTFB9jN+gjxLgEIQYAolNJdYM2H6rS5oP+ULPlYJUOVTX0\n+3mS4uM0NT9D00Z2hJupIzOV7uIJMAkxLkGIAQD3KKtt1JZD1YFw4w84+8r7d3O+NmNzUtt7a9oC\nzkiXnI4ixLgEIQYA3K2qvllbiqq0JdBjs/lQlXaX1vX7cm9JykpN0LT8oNNRozI1cXi6EuOj63QU\nIcYlCDEAEHuONvm043CNth6q1taiKm09VK3txTWqb/L1+7kSPEaT8zpOR03Lz9CU/IyIvuybEOMS\nhBgAgCS1tlrtLa/X1kPV2lZUra1F1dp6qFrF1f0fZyNJuelJmjYyQ1NGZGjqyExNzc/QpLx0JSd4\nBrjy/iPEuAQhBgDQk/K6Jn+oOeQPNtuKqrWzpFa+fk6rIEmeOKNxOan+UDPC32MzbWSmCrJShnSm\nb0KMSxBiAAD91dDs00clte3BZmtRtbYdqlZNPyfCbJOW6NGU/AxNyc/s6L3Jzxy0KRbCDTHuvW4L\nAACXS07w+Kc8KPC2r7PW6mDlUe0ortH2tkdRtXaV1vXaa1PX5NP6fZVav68yZP1Ib7Km5PsDzbVn\nFmr0sNRBeT/9RYgBAMBFjDEaPSxVo4el6sJpI9rXN7b4e212FNdoR3GNthXXaEdxtQ5X936zvqKq\nBhVVNWjFjiP69KkFg1l+vxBiAACIAUnxHk0PzMAdrKKuKdBjU93ee7OjuEZHm4+9QirRE6dxuWlD\nVXKvCDEAAMSwYWmJmjsxR3Mn5rSva2212l9Rr21FNYFg4w84aUnxSoigqREIMQ4zxiyQtGDSpElO\nlwIAgCQpLs5obE6axuak6bIZ+e3rj+dKqMEUOXEqRllrl1prF3m93t53BgDAQZ4hvPy6LwgxAAAg\nKhFiAABAVCLEAACAqESIAQAAUYkQAwAAohIhBgAARCVCDAAAiEqEGAAAEJUIMQAAICoRYgAAQFQi\nxAAAgKhEiAEAAFGJEAMAAKISIQYAAEQlQgwAAIhKhBgAABCVjLXW6RogyRhzRNLeAXzKXEmlA/h8\nsYp2DB9tGD7aMHy0YfgGow3HWmuHH+/BhBiXMsastdbOdrqOaEc7ho82DB9tGD7aMHyR2IacTgIA\nAFGJEAMAAKISIca9HnS6AJegHcNHG4aPNgwfbRi+iGtDxsQAAICoRE8MAACISoQYAAAQlQgxAAAg\nKhFiXMQYk2+Mud8Y87ExpsEYc9gYs9QYc6HTtQ0VY0yhMeabgfe9zxjTaIypMcZsNMbcY4wZ2cvx\nicaYfzPGbDDG1BpjKo0xq40xi4wxpg+v/xljzHJjTJkxpt4Ys80Y82NjTMbAvcuhZYxJN8bsN8bY\nwGNhD/vSfp0YY6YYY35ljNlhjKkzxlQF3tcfjTHndXMM7SjJGBNnjLneGPOaMeaIMaY50BbvGGP+\no6f3EwttaIzJMMZcZYy52xjzd2NMadDP6dQ+HO9YGxljZhtjnjTGHAr8vdpnjHnYGDOpr+9fkmSt\n5eGCh6ST5L+Tog08qiT5Ap+3SrrD6RqHoA3GBN6r7dQOLUHL5ZLO7+b4TElrg/atk9QYtLxUUnwP\nr/9g0L7NkmqClj+WNMrpNjrOdr2vU5supP363Hbf6NQGNZKOBi0/TDt2+z5SJS3r9L1X2elnfI+k\nCbHahpKu7tQ+wY+pvRzrWBtJ+krgmLa/T5VBx9ZKuqDPbeD0F4FH+A9JKYEfZitpvaTpgfWZkn4W\n9I1yidO1DnI7jAu8zxcl/YukYYH1iZIul7RLHcEmv4vjnwpsL5N0pSQjyRP4gWv7w/Pf3bz21wPb\nfZK+IykpsP6soK/Nm0630XG06anyh8A1Qb9kFnazL+0X+p5uCnpP90gqDNqWL+nLkm6gHbttv58E\n/e66Q5I3sD5R0uclVQS2L4/VNpQ/xByW9JKkH0m6MejntLcQ40gbyf8Pd1Ngn8ckDQ+sHyvpH4H1\nFW3re20Dp78IPMJ/SPqmOv7LK+hi+3OB7eucrnWQ28Er6eQetk8N+uH8Yadts4J++K/q4tjbA9vq\nJeV12pYU+EViJf1fF8fOUsd/jwucbqd+tGecpPfkDzHB7bOwm/dI+3XUPE7+/2ytpBv7cRzt2FHv\n3kCtf+hm+8KgthoWi20oydPF912vIcbJNpL0fGDbe13Uny5pX2D7z/vUBk5/EXiE/wh8M1hJD3Sz\n/aygb9gpTtfrcFu9HmiHFzqt/2lg/fZujktRR5fn1zptu0Id/zEeEyID+7T9h/Gk023Qj7Zq+0X2\nq8ByTyGG9gut9xeBetf08zjasaPWhkCtt3azfWbQ92RB0PqYbUP1PcQ40kaSstTRC/OFbo7998D2\nQwrcy66nBwN7o1xgANVpgcVXu9ltjfynUCQpZgb5dqMs8NHTaf35gY//6Ooga+1RSasCixd0c+xm\na+3Bbl637WvT+diIZIwpkHS3/P9x3dmHQ2i/UF8MfHyin8fRjh32BD7O6mZ72++9w53eL23YO6fa\n6BxJCT29dtCxIyVN62afdoSY6DdN/nOZkrSlqx2sta2SdgQWTxyKoiKRMSZe0tmBxc1B6438p5qk\nbtowYGvgY+c2bFvuy7HDjTG5vVfruF9JypD0HWttVU870n6hjDETJeUFFt83xpxp/FfLlRljjhpj\nthtjfmqMyet0HO0Y6qHAx+uNMXcYY7xS+xU1n1NHb9d32g6gDXvncBu1HVtsrS1T17YGfd7r3ytC\nTPQLvmT4UA/7tW3r8RJjl7tF/gGVrZL+HLQ+U1Ja4PPjacORnbb3dGxXx0cUY8wCSddIWmGtfawP\nh9B+oSYHfT5f0pvyD5xMUOCUrvx/eDcYY6YH7Us7hrpP0m/k/yftJ5IqjTGV8o9re1LSdvnHcwR/\nj9KGvXOyjXo9NtALVNnNax+DEBP90oI+P9rDfvWBj+mDWEvEMsacJP8vQkn6tbU2OO2H24Ztx/fl\n2CYOYkoAAAobSURBVK6OjxjGmDRJv5b/8sdb+ngY7RcqK+jzH0r6UNKZ1tpM+Wv/hKQS+X9BPxPo\nIZRoxxDWWp/8Fy18W/7B5ZJ/8H7b360MScM7HUYb9s7JNurLsT299jEIMXA947/B3fPyD1ZbJ+l7\nzlYU0e6SVCjpF52CHvou+PeqlXSNtfYdyX9q11r7d0k3BLZPkfSpIa4vKhhj8iW9Jennkv4i6WT5\n/6hNlvR9SRMk/dEY85NunwSuR4iJfnVBn6f0sF9q4GPtINYScYwx2fIPIBsvaaekK6y1DZ12C7cN\n247vy7FdHR8RjDGnyH9F0n75w0xf0X6hgut7xVq7o/MO1tqX5O+hkToG29OOoR6RNEf+S6wXWms/\nsNbWWWs/stbeI/99eCTp34JOy9GGvXOyjfpybE+vfQxCTPQLPrc4qof92rYVDWItESUwEPBVSTPk\nv/fARdbaw13sWq2OH67jacNDnbb3dGxXx0eK++W/aus/5B//lx78CNovKbCu7RcN7Rcq+GfymADT\nxbYxgY+0Y4Ax5kRJFwcWf9HVPtbaR+W/2jBO0oLAatqwd062Ua/HGmNS1HFKttf2JcREv+3yd1lL\n0vSudjDGxMnfbS2Fjvx2rcDYjpclzZZULH+A2dfVvtZ/c4JtgcUu2zCgbaR85zZsW+7LsUestaU9\n7OeksYGPj8h/48TOjza/DyxvlWi/LmyVf/B4X/lvwkM7Bgu+tHZ3D/vtCnwcJ9GGfeFwG7Udm2+M\nyenl2K5e+xiEmChnra2Rf/4LqeM/l87OkH9AnOSfi8TVAkl+qfw3+SuTP8Ds7OWw1wMfu2xDY0yy\npHmBxc5t2HbsdNP9BJOXdHOsW9B+AdbaekmrA4tTeti1bdueoHW0o19wCCzsYb+24B0csmnD3jnV\nRm/Kf9GAJF3Uy7GH1BG2uuf0HQZ5hP9Qx7QD1ZJGdrH9mcD2tU7XOgRtkSjp7+qYf+PUPh4XfBvu\nK7vYfpv6dhvun3Vx7MnqmIzzmOeOlkdQ+yyk/Xptq0WBelvUxV2y1XHXUyv/OC3aMbTWCUHt0OXt\n5+U/hdS2z7/Qhv26Y69jbaSOaQfekRTXaVuaOqabOOa5u3wvTjc6j/AfCp0Acp2kEwPrMyTdG/TN\n6vYJID2SnlZHoDuzn8e3TYhWKukTQc95XeCH2apvE6J9Wx0Tos1Vx8STETFpXBjt222Iof2OeT/x\n8t8MzAY+zgmsj5N0mfynOK38PTam07G0o7/mV4Pey0/a/pjKf4XSQvl7Wa38p5sSY7UNJeUGPYLD\nyZmdtnUODI60kfwhp23qgUck5QbWFwZ9zZkAMtYegW+M0qBv4Cp1pOFWSXc4XeMQtMG5Qe//aOAP\nRXeP97o4vqup6RuClvszNX2T+jE1fTQ8gt7Lwm62036h72eCOiazawvWdUHLW9T1hK20o/99jJR/\nTIRVaBsGLxdLmhXLbdipPXp6jIuUNpJ/puxmdfx9qgw6tlbSBX1+/05/AXgM3EP+u9HeH/gGapD/\nhlovSrrQ6dqG6P3P78cP9J5uniNR/vvIbAj8MFXJ/9/yIvVhMjJJn5W0XFK5/EFqm6QfS8pwun0G\noH17DDG0X5fvxxuof4v8/93Wyt9beoekNNqx1/eRIv9l/yvl73lpCbTFOvlvA9Dtf+ux0ob9+J03\nLpLaSP6LLp6S/wqkRvkD/x8kTerP+zeBJwMAAIgqXJ0EAACiEiEGAABEJUIMAACISoQYAAAQlQgx\nAAAgKhFiAABAVCLEAACAqESIAQAAUYkQAwAAolK80wUAwP9v725DpKriOI5/f1SCGrmLWtoTSSJR\nCT0QrSI1FiW92AiisIwSoQgSjIRKComgoqAHIYPKyKKwKKIgImjLLFIxCot8EfZgZOSaWmqWrdS/\nF+dMexlmdmd2dZ2bvw9cztx7z7n3zL4Y/nvO/94zGElHAzcAc0nrhI0nrfWyjbTg3MfABxGxodDm\nHOAq0hITK0e6z2Z26HnZATNra5ImAu+Q1lqp2k9ab+U4QPnY7ojoKLSbDzwPrImIyoh01sxGlKeT\nzKzdvUQKYPYCdwKTI2J0DljGAZcBT5FWwjWzI4ink8ysbUk6A7g87y6IiNeL5yNiL9AD9EhaPNL9\nM7PDyyMxZtbOphc+vz1QxYjYX/0sKUhTSQAXS4qarVLbXtIsSa9I2irpL0k7JfVIuk6S6tSv5Gtt\nyfvdklZL+lXS75LWSbp+CN/ZzJrkkRgzK4uTgG+brNsLjCblzBwAdtWc7yvuSHqYNFVVtQfoBC7N\n25WS5kXEP/VuJul24HEggN353l1Al6SZEbGwyX6bWQs8EmNm7eyzwuflOcl3UBExCViUd9dGxKSa\nbW21rqRFpACmF7gF6IiIccBY0tNQ23J5V4PbTQQeAV4k5et0AhOAR/P52zwiY3Zo+OkkM2trkl4A\nbsy7faTHqdcDn5IClF8atJvPIE8nSeoAfiSNSndFxBd16swAPiElDk+KiL58vAKsztXeA+ZEzQ+q\npJXATcA3wLTa82Y2PB6JMbN2dzPwGCmAGUWa3rkHeBPYLmmDpHn18laacDVwLNBTL4ABiIh1wPek\n6aXzG1znoQYBygO5nEp6v42ZHUQOYsysrUVEX0QsBk4BbgVWAZtJ+ScAF5Aew35VUqu/aTNzeYmk\nbY22fG8KZdEB0khNvb5vBn7Ou+e12DczG4QTe82sFCJiO/B03pB0AtANLCUFF9eQgollLVx2ci7H\n5G0w9ersqE4xNfBTvk9T+Txm1jyPxJhZKUVEb0SsII1w9ObDC1q8TPU3cFlEqIlt5cHqv5kNn4MY\nMyu1iNgBvJV3p7XYvBr8nDqMLkyQNGqA8yfmsm4CspkNnYMYM/s/2JfL4rRO9Z0uAyX8rstlRdLo\nId77GGBGvROSptIfxHw+xOubWQMOYsysbUmaIun0QeqMIa1WDbCxcGpPLjto7DVSANRJyq0Z6D6d\nA5xe0uDpqCW53BwRG+ucN7NhcBBjZu3sLOBrSW9IulZSNREXSWMldZPeGzMlHy4m9W7K5ZmSLqx3\n8YjYSX+gcbekZyX9NyUlaYyk2ZKeAdbWuwbwB+mx7+ckHZ/bdeS3AFdzdO5r8vuaWQv8sjsza1uS\n5gDv1hz+kzRtNK5w7G9gaUQ8WNN+DXBR3t1FWgkbYG5ErC/Uuxe4n/6pp335Hh2FY1siYkqhTYX0\nsrsfgCfoX3bgt9y36j+Jy73sgNmh4SDGzNpaHhnpBmYBZ5PWUBpFCki+Az4CVkTEpjptx5OCkysK\n7QBmR8SHNXWnAwuB2cDJwFGkZNyvgPeBVRGxtVC/Qg5iIuK0PCp0B3AuKU/mS+DJiHh52H8EM6vL\nQYyZ2RDUBjGHtzdmRybnxJiZmVkpOYgxMzOzUnIQY2ZmZqXkIMbMzMxKyYm9ZmZmVkoeiTEzM7NS\nchBjZmZmpeQgxszMzErJQYyZmZmVkoMYMzMzK6V/AT4Zz8BjTWb+AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "error_plot([log_likelihood(w, data, labels) for w in ws_gd])\n", "plt.plot(range(len(ws_gd)), [baseline]*len(ws_gd),\n", " label='sklearn', **kwargs)\n", "_ = plt.legend()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAisAAAIGCAYAAACVnM/AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VGXaBvD7yaR3kkBCC71XSQhFwIIgLEVExQahKEUR\nsO66ru6n4iqyawEbhCooRRSEIIIKinSS0EGa9B5I78nk/f7IZJLB9JnkTLl/1zVXcp5zzpwHxHBz\n5n3fI0opEBEREVkrJ60bICIiIioLwwoRERFZNYYVIiIismoMK0RERGTVGFYsRETcRGSGiFwWkUwR\n2Ssi92vdFxERka1jWLGcxQBeArAcwDQAuQB+EJG7tGyKiIjI1gmnLptPRCIA7AHwqlLqfUPNHcAR\nAAlKqQgt+yMiIrJlvLNiGQ8DyAcQVVhQSmUBWACgq4g01qYtIiIi28ewYhl3APhTKZV4W31vsf1E\nRERUBXYXVkTER0SGish0EflRRG6KiDK8WlfwPUJEZJaI/CkiWSJyXUSiRaRvKafUBXC1hHphrV5V\nfi1EREQEOGvdQDXoC2BNVU8WkY4AtgAINJRSAAQBGAxgkIi8ppSacdtpHgCyS3i7rGL7iYiIqArs\n7s6KwQ0AGwC8BWBCRU8SEQ8A61AQVPYDaK+U8gNQC8AHAATAuyLS/7ZTMwG4lfCW7sX2ExERURXY\n452VaKXU94UblRzcOhFAIwBpAIYopS4DgFIqBcDLItIMwDAA7wH4qdh5Vw3n3a6u4euVSvRARERE\nxdjdnRWllN6M0580fF1WGFRu81/D1y4i0qpY/QCAZiJS67bjuxXbT0RERFVgd2GlqkTEB0CYYXNT\nKYftBpBs+L74YNtvUfB7afzISUTcAIwFEKeUOmvZbomIiByHPX4MVFVtUDAmBQCOlnSAUipfRE4A\niADQtlh9j4isAvCOiAQBOAUgEkATAP2qtWsiIiI7x7BSpG6x78saY1K4r+5t9UgAbwMYCSAABavX\nDlFK/VqRiwcFBanGjRtXrFMiIiI7EBcXd1MpVbu84xhWingV+76s2TsZhq/exYuGFWv/bnhViIhM\ngOGjo9DQUMTGxlb0VCIiIpsnIucrchzHrGhIKRWllApXSoXXrl1usCQiInJIDCtF0ot9X9Yibp6G\nr2nV2AsREREZMKwUKT5Opazl8Qv3lbS8PhEREVkYw0qR4wCU4ft2JR0gIk4ACtdXOVYTTRERETk6\nhhUDpVQqgMIRrqVNN+4GwM/w/WZLXFdEhohIVHJycvkHExEROSCGFVPLDF+fFJHbpyYDwMuGr3FK\nqROWuKBSKlopNcHPz6/8g4mIiByQXYYVEQkqfKHgIYSF/IvvM3ysU9xcAOcB+ABYLyJtDe/nIyIz\nAQw3HPdadf8aiIiIqIC9rrMSX0p9123bTQCcK9xQSmWKyAMo+IinC4CjIpKCgjVVnFAwpuU1pdRP\nICIiohphl3dWzKGUOgigPYDZAM4AcANwC8APAPoppWZo2B4REZHDscs7K0opKf+oMs+/BmCa4UVE\nREQassuwYktEZAiAIc2bN9e6FSKyMkoppKamIiUlBRkZGdDr9Vq3RGRCp9PB09MTvr6+8PHxgYhZ\n9wpKJUqp8o+iahceHq74bCAiKqSUwo0bN5Ceno6AgAB4e3tDp9NV218GRJWllIJer0daWhoSEhLg\n5eWFOnXqVOrPqIjEKaXCyzuOd1aIiKxQamoq0tPT0ahRI+h0Oq3bIfoLEYGzszP8/f3h4+OD8+fP\nIzU1Fb6+vha/FgfYEhFZoZSUFAQEBDCokE3Q6XQICAhASkpKtbw/wwoRkRXKyMiAt7e31m0QVZi3\ntzcyMjKq5b0ZVuyQPl/hg59O4FpyltatEFEV6fV63lUhm6LT6aptEDjDip3JytVj8tf78MmW0xi9\ncC+SM3K1bomIqoiDacmWVOefV4YVjVn6QYb/XnsEG49eAwCcuJ6Kp5fEICuX0x2JiMh2MaxozNIP\nMpxybwvU9nEzbsecS8Rzy/YjT59vkfcnIiKqaQwrdqZhgCeWjIuAj1vRrPRf/riOf605Aq6pQ0RE\ntohhxQ61qeuL+aPD4epc9J93ZexFfPDTSQ27IiKyPosXL4aI4O67767UeW+++SZEBGPGjKmWvsgU\nw4qd6tY0EJ88fgecio13+vTX01i846x2TREREVUBw4odu79dCP7zYAeT2lvrj2HdwSsadURERFR5\nDCt27vGIULzUr6VxWyngpW8OYNupeA27IiIiqjiGFQfw3L3NEdmjkXE7V68waWkcDl1K0rArIiKi\nimFY0Zil11kp5Rr4vyHtMKhjXWMtPUePsYticPZmerVdl4hICzk5OZg1axZ69uwJf39/uLi4IDg4\nGJ06dcLkyZOxa9euCr/X0qVL4eLiAicnJ3zxxReV6mP79u147LHH0KBBA7i5uSEwMBD33Xcfli9f\nXurszH379uHVV19Fr169EBoaajzv7rvvxvz580tdIbb4gN/8/Hx8+umniIiIgL+/P0QEBw4cAACM\nGTMGIoI333wTer0eH3/8MTp16gRPT08EBARg8ODBiI2NrdSvsybwqcsaU0pFA4gODw8fX53X0TkJ\nPhzRCUkZOdhx+hYA4FZ6DkYt2IPVz/REHV/36rw8EVGNyMvLQ//+/bF161YABf9Y8/Pzw61bt3Dj\nxg0cOnQIt27dQo8ePcp9r88++wxTpkyBTqfDkiVLMHLkyAr38Y9//AMzZ840bvv6+iIxMRGbN2/G\n5s2bsW7dOnz99ddwcjK9Z9C/f3/culXwM9rT0xOenp5ISEjA1q1bsXXrVqxZswZr166Fs3PJf30r\npTB8+HCsXbsWOp0OPj4+JR6Xl5eHQYMGYdOmTXBxcYGbmxsSExPxww8/YPPmzdiyZUuFfo9qCu+s\nOBA3Zx3mjAxD+/pFj+++lJiJyIV7kZzJZfmJyPYtW7YMW7duhaenJ5YuXYqMjAwkJiYiOzsb58+f\nx6effopOnTqV+z7/+c9/8Nxzz8HV1RXffvttpYLKrFmzMHPmTAQHByMqKgpJSUlITk5Geno6VqxY\ngZCQEKxYsQLvv//+X87t378/li9fjqtXryI9PR2JiYlIS0vD0qVLERISgg0bNuCjjz4q9dqrV6/G\nxo0b8fnnnyMlJQWJiYm4fv06mjZtanLcZ599hpiYGKxcuRJpaWlITU3FwYMH0b59e2RlZWHatGkV\n/vXWBN5ZcTA+7i5YNCYCD8/ZifO3Cp6OefxaKsYvicWScRFwd+GD04hswo+vAtcOa92FZYR0AAbO\nsMhb7d69GwAQGRlpEjB0Oh1CQ0MxefLkct/jlVdewf/+9z94eXlh7dq16Nu3b4Wvn5SUhNdffx3u\n7u7YtGmTSTDy8PDAo48+itDQUNx5553473//i5deegmurq7GY5YtW/aX9/Ty8sLIkSPRqFEj9OnT\nB59//jleeeWVEq+flpaGuXPnYsKECcZanTp1Suxz27Zt6NWrl7HWsWNHLF68GOHh4YiJicGFCxcQ\nGhpa4V97dWJYcUC1fdywdFw3DP9iJ26mZQMA9p5NwLQV+/H5k2HQOfHhaURW79ph4Px2rbuwOr6+\nBXeOr169Wulz8/PzMXHiRMyfPx/+/v7YsGFDpT8K+e6775CWlobBgweXegenR48eaNKkCc6cOYO4\nuLgKX6N3797w9/fHuXPncOXKFdSrV+8vxwQGBmLcuHEVeq/iQaVQWFgYGjRogEuXLuHIkSNWE1b4\nMZCDCg30xJfjusK72LL8m45ex+vfc1l+IrJdAwcOBACsXbsWQ4cOxerVq41jQMqSm5uLxx9/HPPn\nz0edOnXw22+/VWnMxs6dOwEAW7ZsQUhISKmvixcvAoDxa3GrVq3CsGHDEBoaCg8PD4iI8ZWUVDCL\n88qVktfLCg8PL3U8S3Fdu3YtdV/9+vUBAImJieW+T03hnRUH1q6eH6IiwzBmYQxyDA86XL73Amp7\nu+LF/q007o6IyhTSofxjbIUFfy133XUX3n77bbz99tuIjo5GdHQ0AKB169YYNGgQJk6ciBYtWvzl\nvMKQAQDffvtthca1lKTwjk5GRgYyMjLKPb74MXl5eRgxYgTWrFljrLm5uSEoKAg6XcFH9PHx8cjP\nz0d6eskzOWvXrl2hPksbeAsA7u4FEy5yc61nLCPDioPr2SwIsx7rjGeX7UPhDZXZW04jyMcNkT0a\na9obEZXBQmM87NEbb7yBkSNHYuXKlfjtt9+wa9cuHD9+HMePH8esWbOwYMECREZGmpzToUMH6PV6\nHDt2DM8++yx+/fVXBAUFVfra+fkF//CbNm0aPv7440qdO2/ePKxZswaenp6YMWMGHnzwQTRo0MDk\nmIYNG+LSpUul3gEvDDX2hh8DEQZ2qIvpD7Q3qf3fuqNYf4jL8hORbWrSpAleffVVbNy4EQkJCfj1\n11/Rp08f5OXl4dlnn8WNGzdMjg8ICMDmzZvRqlUrHDlyBP369avSxyDBwcEAgAsXLlT63FWrVgEo\nCFtTpkz5S1DR6/W4efNmpd/XHjCsEABgZPdGeP6+olujSgEvrDyAHacd838MIrIfOp0Od999N9av\nXw8XFxekp6eXuPBZSEgINm/ejGbNmuHAgQPo378/KrtgZ+E4l99++w2ZmZmVOvfSpUsAgDvuuKPE\n/Tt27EBWVlal3tNeMKxorCZWsK2oaX1b4MluRSO/c/UKE5bE4shl7XsjIqqInJycUve5uroaPybJ\nzs4u8Zj69etjy5YtaNy4MWJjYzFw4ECkpaVV+PqPPPIIvLy8kJiYiLfffrvMY2+/c+Pn5wcAOHz4\nr1PS8/Ly8Prrr1e4D3vDsKIxpVS0UmpC4R9SLYkI3n6gPQa2DzHW0nP0GLNoL85xWX4isgGRkZEY\nO3YsNm3ahNTUVGP93LlzGD16NLKysuDh4YHevXuX+h6hoaHYsmULGjRogF27dmHQoEEVGiwLFEwd\nfu+99wAAM2bMwPjx43Hy5Enj/oyMDPz666+YMGECevbsaXJuv379AADTp0/H2rVrjUvrHz9+HEOG\nDMHevXvh5eVVsd8IO8OwQiZ0ToKPHu2M7k0DjLWbaTmIXLgXN1Id8/YjEdmOrKwsLF68GAMGDICf\nnx9q1aoFLy8vNGnSBCtXroROp8PcuXPLHTzbpEkTbNmyBXXr1sXvv/+OoUOHVvgjmClTpmD69OkQ\nEcyfPx+tWrWCt7c3AgIC4O3tjXvvvRfz5s37y/u9/PLLaNasGVJSUjBs2DB4eHjAz88Pbdq0wc8/\n/4w5c+ZUadCvPWBYob9wd9EhKjIcbesWLct/ISEDoxfGICXLeqayERHdbsaMGZg5cyYGDBiApk2b\nIicnB3q9Hs2aNcPYsWOxb98+jBo1qkLv1aJFC2zZsgXBwcHYvHkzHnzwwVI/Prrd66+/joMHD2LC\nhAlo0aKFcbpxvXr1cP/992PmzJnYtm2byTkBAQHYvXs3nnnmGePgWg8PDwwbNgxbt27FmDFjKvV7\nYU+EC4BZh/DwcGVtT7q8kZqFh7/YhQsJRbc/uzcNwOKxXJafqLr98ccfaNOmjdZtEFVKZf/cikic\nUiq8vON4Z4VKVcfHHUvGRSDIu+i5FbvPJOCFlQegz2fIJSKimsGwQmVqHOSFxWMjTJbl//HINfx7\nLZflJyKimsGwQuVqX98PUaPC4Kor+uPy9Z4LmLX5lIZdERGRo2BYoQrp2TwIHz3aGVLsgcwf/3IK\nX+0+r11TRETkEBhWqMIGdayLt4e2M6m9sfYINhyu/KPYiYiIKophhSplVI/GmHpvc+O2UsDzKw5g\n559clp+IiKoHw4rGrGm5/Yp6oV9LPB5RtCx/jj4fE5bEcVl+IiKqFgwrGrOm5fYrSkTwzrD2uL9d\nsLGWlp2HMYticP4Wl+UnIiLLYlihKtE5CWY9dgcimhRflj8bkQv3Ij61Yis8ElHZuDwA2ZLq/PPK\nsEJV5u6iw7zIcLQO8THWzt/KwJhFe5HKZfmJzKLT6YwPsiOyBXq93vhUa0tjWCGz+Hm4YMm4CDSo\n5WGsHb2SgolL45Cdxx+0RFXl6emJtLQ0rdsgqrC0tDR4enpWy3szrJDZ6vi6Y+lT3RDoVbQs/84/\nb+HFlQe5LD9RFfn6+iIhIYF3V8gm6PV6JCQkwNfXt/yDq4BhhSyiSZAXFo3tCi/XoluAPxy+irei\nj/Jzd6Iq8PHxgZeXF86fP4+kpCTk5eXx/yWyKkop5OXlISkpCefPn4eXlxd8fHzKP7EKnMs/hKhi\nOjbwx5xRYRi3OAa5+oIfqkt2nUeQtxum9m2hcXdEtkVEUKdOHaSmpiIlJQU3btzgXRayOjqdDp6e\nnggKCoKPjw+k+DLnFsSwQhbVu0VtfDCiM6at2I/CfwR++PNJBHm74YluoWWfTEQmRAS+vr7Vdmud\nyFbwYyCyuKGd6uH/Brc1qb3+/WFsPMJl+YmIqPIYVqhajLmzCSbf08y4na+AqSsOYPeZWxp2RURE\ntohhharNy/1b4dHwhsbtnLx8jP8yFseupGjYFRER2RqGFao2IoL/PNge/doWLcufmp2H0Yv24mJC\nhoadERGRLWFYoWrlrHPCJ4/fga6Naxlr8anZGLVgD26mcVl+IiIqH8MKVTt3Fx3mR3ZFq+Ci+ffn\nbmVg7KIYpGXnadgZERHZAoYVjYnIEBGJSk5O1rqVauXn6YIlT0Wgvn/RsvyHLydjxJxdOH2DS4oT\nEVHpGFY0ppSKVkpN8PPz07qVahfs644lT0WglqeLsXbsagqGfLIdK2MucHVOIiIqEcMK1ahmtb2x\neGwE/DyKAktmrh7/+O4wnlu+H8mZfFozERGZYlihGtepoT9+nNYbEY0DTOo/HLqKv83ahrjzCRp1\nRkRE1ohhhTRRz98Dyyd0xwv3tYRTsUdJXE7KxIi5u/HJ5lN8YjMREQFgWCEN6ZwE0+5rgZUTe5gM\nvNXnK3zw80k8MW83riZnatghERFZA4YV0lzXxgHYMLU3/tYhxKS+52wCBs7ahk1Hr2nUGRERWQOG\nFbIKfp4u+OyJLpgxvAPcXYr+WCZl5GLi0ji8/v1hZOXqNeyQiIi0wrBCVkNE8FhEKNZP6YU2dX1N\n9n21+wKGfrodJ66latQdERFphWGFrE7zOj5Y82xPjOnZ2KR+8noahn66HUt3n+eaLEREDoRhhayS\nu4sObw5thwWjwxHg5WqsZ+fl443vj2Di0jgkpudo2CEREdUUhhWyan3bBOPHab1xZ/NAk/pPx65j\n4Kxt2H3mlkadERFRTWFYIasX7OuOpeO64R8DWsO52KIs11Ky8Pi83fjgpxPI0+dr2CEREVUnhhWy\nCU5OgmfuboZvn+mJ0ABPY10p4JMtpzFi7i5cTMjQsEMiIqouDCtkUzo39McPU3thWOd6JvV9F5Lw\nt9nbEH3wikadERFRdWFYIZvj4+6Cjx+7Ax+O6AQvV52xnpqVhynL9+Pv3x5ERk6ehh0SEZElMayQ\nzRrepQHWT+2NDvX9TOrfxF7C4E+248jlZI06IyIiS2JYIZvWJMgL3z3TExP7NDWpn4lPx/DPd2LB\n9rNck4WIyMYxrJDNc3V2wj//1gZLxkUgyNvNWM/R52P6+mMYtzgGN9OyNeyQiIjMwbBCdqNPy9rY\n+Hxv3N2qtkn91xPxGPDxNmw7Fa9RZ0REZA6GFbIrQd5uWDi6K94Y3BYuuqI1WW6mZWPUgr14b8Mf\nyMnjmixERLaEYYXsjpOT4KleTbDm2TvRNMjLZN/c38/g4Tk7ce5mukbdERFRZTGsaExEhohIVHIy\nZ65YWvv6foie0gsjwhuY1A9dSsag2duwet8ljTojIqLKEM6UsA7h4eEqNjZW6zbsVvTBK3ht9WGk\nZpuuv/LgHfXx9gPt4OPuolFnRESOS0TilFLh5R3HOyvkEIZ0qocN03qjS6i/SX3N/ssYNHs7DlxM\n0qgzIiIqD8MKOYyGAZ74ZmIPTLm3OaRo7C0uJGTg4S924ovf/kR+Pu80EhFZG4YVcijOOie81L8V\nlj3dHSG+7sZ6Xr7C+xuPI3LhXtxIydKwQyIiuh3DCjmkHs0C8eO03ujXNtikvv30TQyYtQ1bjl/X\nqDMiIrodwwo5rFperogaFYbpD7SDm3PR/woJ6TkYtzgWb0UfRXaeXsMOiYgIYFghByciGNWjMdY9\n1wstg71N9i3acQ7DPtuJk9dTNeqOiIgAhhUiAECrEB+se64XRnYPNan/cTUFgz/ZjvnbznDwLRGR\nRhhWiAzcXXR4Z1gHzB0VBj+PonVXcvLy8c4Pf+DJ+XtwOSlTww6JiBwTwwrRbe5vF4KNz/fGnc0D\nTeq7ztzCgI9+x+p9l8DFFImIag7DClEJ6vp5YOm4bvi/IW1NBt+mZufhxW8OYvKyfUhMz9GwQyIi\nx8GwQlQKJyfB2Dub4IepvdC+vq/Jvg2Hr6H/x7/j1xM3NOqOiMhxMKwQlaN5HR+sfuZOTLm3OZyK\nrXwbn5qNsYti8K81h5GRk1f6GxARkVkYVogqwNW5YOXbVZN6onGgp8m+r/dcwKDZ27H/QqJG3RER\n2TeGFaJKCGtUCz9M7Y0nuplOcT57Mx0Pz9mFD386gVx9vkbdERHZJ4YVokrycnPGuw92wMIx4Qjy\ndjPW9fkKs7ecxvDPd+L0jTQNOyQisi8MK0RVdG/rYPz0Qh8MaBdiUj98ORmDZm/D4h1nuZAcEZEF\nMKwQmSHAyxVfjOyCDx7pBG83Z2M9Oy8fb0Yfw+hFe3EtmU9xJiIyB8MKkZlEBA+FNcDG53ujW5MA\nk33bTt1E/4+2Yt3BKxp1R0Rk+xhWiCykQS1PLB/fHf/6Wxu46or+10rJysPU5fsxZfl+JGfkatgh\nEZFtYlghsiAnJ8H4Pk2xbsqdaFPXdCG56INXcP/Hv2PbqXiNuiMisk0MK0TVoHWIL76f3BOT7moG\nKbaQ3LWULIxasBdvrjuKzBy9dg0SEdkQhhULEBFvEXlLRDaISLyIKBF5Veu+SFtuzjq8OrA1Vk7o\ngQa1PEz2Ld55DoM+2YZDl5I06o6IyHYwrFhGEIB/A+gAYL/GvZCViWgSgI3P98Gj4Q1N6mfi0zH8\n852Y9csp5HEhOSKiUjGsWMZVAPWVUg0BTNC6GbI+3m7OeP/hjogaFYZAL1djPS9f4aNfTuLhObtw\nJp4LyRERlYRhxQKUUtlKKc5NpXL1bxeCTS/0wX1tgk3qBy4mYdDs7Vi6+zyU4kJyRETFMawQ1bAg\nbzfMiwzD+w91gJerzljPzNXjje+PYOziGNxI4UJyRESFrDasiIiPiAwVkeki8qOI3DQMXFUi0rqC\n7xEiIrNE5E8RyRKR6yISLSJ9q7t/orKICB7tGoofp/VBeKNaJvt+OxGP/h//jg2Hr2rUHRGRdbHa\nsAKgL4C1AF4HMABAYGVOFpGOAI4AmAqgKYBsFAyEHQzg55Jm60gB9wq+rPn3jmxEaKAnVk7sgb8P\naAUXXdEc56SMXDz79T68uPIAUrK4kBwROTZr/wv3BoANAN5CJQauiogHgHUoCDj7AbRXSvkBqAXg\nAwAC4F0R6X/bqXcCyKzgq0+Vf1VExeicBM/e3RzfT74TLYO9Tfat3n8ZAz76HTv/vKlRd0RE2nMu\n/xDNRCulvi/cEJHGlTh3IoBGANIADFFKXQYApVQKgJdFpBmAYQDeA/BTsfNOAhhbwWscr0Q/ROVq\nV88P657rhf9tOoEFO86icJztleQsPDFvD57q1QSv3N8K7i66st+IiMjOWG1YUUqZs7znk4avywqD\nym3+i4Kw0kVEWimlThiueQPAYjOuS2QWdxcdXh/cFn3bBOPlVQdxOSnTuG/B9rPYdioeH47ojPb1\n/TTskoioZln7x0CVJiI+AMIMm5tKOWw3gGTD9xxsS1anR7NA/Ph8bwzvUt+kfvJ6Gh78fAc++/U0\n9Pmc4kxEjsFq76yYoQ0KxqQAwNGSDlBK5YvICQARANpa4qIi8hwAf8MLAO4RkcLf30+UUskln0lU\nMl93F3w4ojP6tQnGa2sOI9HwxOZcvcJ/N53AluM38OGITmgU6KVxp0RE1cvu7qwAqFvs+7IWaivc\nV7eMYyrjZQDTAbxk2O5v2J6OgoG9RFUysENdbHq+D+5pVdukHnc+EQNnbcOKvRe4kBwR2TV7DCvF\n/5mZWepRQIbhq3cZx1SYUqqxUkpKeZ0r6RwRmSAisSISGx8fb4k2yE7V8XXHwjFd8Z8H28Oj2ADb\njBw9Xl19GOOXxOFmWraGHRIRVR97DCs2QykVpZQKV0qF165du/wTyKGJCJ7s1ggbpvXGHaH+Jvt+\n+eM6Bnz8O7Ycv65Rd0RE1ccew0p6se89yjjO0/CVT48jm9IkyAurJvbAi/1awtmpaCG5m2k5GLc4\nFv9acxgZOXkadkhEZFn2GFaKj1OpV8Zxhfu4pjnZHGedE6b2bYHvnumJpkGmA2y/3nMBg2Zvx4GL\nSRp1R0RkWfYYVo4DKBxt2K6kAwxL5bcybB6riaaIqkOnhv5YP7UXRnYPNamfvZmOh77YiVm/nEKe\nPl+j7oiILMPuwopSKhVArGGzXymHdQNQuKrW5mpvqgwiMkREopKTObOZqsbT1RnvDOuARWO6Isjb\nzVjX5yt89MtJPDJ3F87dTC/jHYiIrJvdhRWDZYavT4pISVOTXzZ8jStcvVYrSqlopdQEPz+uSErm\nuad1HWx6vjf6tQ02qe+/kIS/zeYUZyKyXVYdVkQkqPAF07VK/IvvK+EJyHMBnAfgA2C9iLQ1vJ+P\niMwEMNxw3GvV/WsgqkmB3m6IGhWG9x/qAE9XTnEmIvsg1vwvLRGpaHNNbl/LREQ6oeAjnkBDKQUF\na6o4oWBMy2tKqRkWatVs4eHhKjY2tvwDiSro/K10vLDyAPZdMB1oG+TtipkPd8S9rYNLOZOIqGaI\nSJxSKry846z6zoo5lFIHAbQHMBvAGQBuAG4B+AFAP2sKKkTVoVGgF76Z2AMv9WsJXQlTnF/jFGci\nshFWfWfFkfDOClWngxeT8MLKAzhz20DbJkFe+OjRzujc0L+UM4mIqo/D31khoiKc4kxEtoxhRWOc\nukw1hVMU8JwsAAAgAElEQVScichWMaxojFOXqaZxijMR2RqGFSIHxCnORGRLGFaIHJSI4NGuofhx\nWm904VOciciKMawQObjiU5xLeoozpzgTkdYYVogIzjonTOnbAquf/etTnJfxKc5EpDGGFSIy6tiA\nU5yJyPowrGiMU5fJ2nCKMxFZG4YVjXHqMlmrwinO/UuZ4rycU5yJqIYwrBBRqQK93TB3VBhmPtQR\nXrdNcf4npzgTUQ1hWCGiMokIRnRtiA2c4kxEGmFYIaIK4RRnItIKwwoRVZjJFOfanOJMRDWDYYWI\nKq1jA3/8MKU3RnVvZFLnFGciqg4MK0RUJR6uOkwf1p5TnImo2jGsaIzrrJCt4xRnIqpuwh8i1iE8\nPFzFxsZq3QZRlSmlsCr2Et6KPor0HL3JviGd6mHG8A7wcnPWqDsiskYiEqeUCi/vON5ZISKLKJzi\n/OO0PghrVMtkX/TBK3jgsx04fSNVo+6IyJYxrBCRRYUGemLlhO54uX9L6IpNcT59Iw1DP92B9Yeu\naNgdEdkihhUisjhnnROeu7cFvn66m8ng24wcPZ5bth9vRR9FTh5nCxFRxfADZHv046vAtcNad0GE\n7gB2huTj1PVUpGYXWzAuBjh5yBktg33gquO/mYhsRkgHYOCMGr8sw4o9unYYOL9d6y6IAACuANoB\nf72PmwvgUo23Q0Q2iP+kISIiIqvGOyv2KKSD1h0QlSozV4+T11ORmWs6vbmWpyua1faCsxP/DUVk\ntTT6+4VhRWMiMgTAkObNm1vuTTX4PJGoojwAtMjJw7/WHMGa/ZeLduQAjXSe+PzJLmhXz0+z/ojI\n+vCfMBpTSkUrpSb4+fGHMzkOT1dnfDiiE6YPaw8XXdH05vO3MjD8851YFXtRw+6IyNowrBCRJkQE\no7o3wqpJPVHPz91Yz87LxyvfHsI/Vx9C1m0fFRGRY2JYISJNdW7oj/VTe6N3iyCT+vK9F/HwnJ24\nmJChUWdEZC0YVohIcwFerlg8NgJT+7YwqR+5nILBn2zHr8dvaNQZEVkDhhUisgo6J8GL/Vpi0diu\n8Pd0MdaTM3MxdnEMPvzpBPT5fPAqkSNiWCEiq3JPqzqIfq4XOtQ3HXQ+e8tpjFm0FwnpORp1RkRa\nYVghIqvTMMATqyb1wBPdQk3q207dxODZ27D/QqJGnRGRFhhWiMgqubvo8O6DHfC/RzrBzbnoR9WV\n5CyMmLsLS3adg1L8WIjIETCsEJFVezisAdY8eycaBXoaa7l6hX+vPYrnVx5ARk5eGWcTkT1gWCEi\nq9e2ni/WPdcL/doGm9TXHriCYZ/twJ/xaRp1RkQ1gWGFiGyCn4cLokaF4dWBreFUtOgtTl5Pw9BP\ntmPD4avaNUdE1YphRWMiMkREopKTk7VuhcjqiQgm3dUMXz/dHUHebsZ6eo4ez369D++sP4Zcfb6G\nHRJRdWBY0RifDURUeT2aBeKHqb3QtXEtk/r87WfxxLzduJ6SpVFnRFQdGFaIyCYF+7pj2fjueLpX\nE5N6zLlEDJq9HbvP3NKoMyKyNIYVIrJZLjonvD64LT5/sgu8XHXG+s20bDw5fw/mbv2T05uJ7ADD\nChHZvL91qIt1U3qhRR1vY02fr/Dej8cxcWkcUrJyNeyOiMzFsEJEdqFZbW98P/lODO1Uz6T+07Hr\nGPrJdvxxNUWjzojIXAwrRGQ3vNycMeuxznj7gXZw0RXNbz53KwMPfr4D38Vd0rA7IqoqhhUisisi\ngsgejbFyYg/U9XM31rNy8/HSqoN4bc1hZOXqNeyQiCqLYYWI7FKX0FpYP6UXercIMqkv23MBI+bu\nwsWEDI06I6LKYlghIrsV6O2GxWMjMPXe5ib1Q5eSMeTT7fjtxA2NOiOiymBYISK7pnMSvNi/FRaO\nCYefh4uxnpSRi7GLY/DRzyehz+f0ZiJrxrBCRA7h3tbBWD+lF9rX9zXWlAJmbT6Fp76MQXo2n95M\nZK0YVojIYTQM8MS3k3ri8YiGJvXfTsTjyfl7kJSRo1FnRFQWhhUicijuLjq8N7wjZj7cEW7ORT8C\nD1xMwoi5u3Atmc8VIrI2DCtE5JBGhDfEigndTcaxnLyehofn7MS5m+kadkZEt2NY0ZiIDBGRqOTk\nZK1bIXI4d4TWwqpJPRDs62asXUrMxMNzduHYFa54S2QtGFY0ppSKVkpN8PPz07oVIofUMtgH307q\nicaBnsbazbRsPBq1CzHnEjTsjIgKMawQkcNrGOCJVZN6ok3doplCqVl5GLVgD349zrVYiLTGsEJE\nBKC2jxtWTOiO8Ea1jLWs3HyMXxKLtQcua9gZETGsEBEZ+Hm4YOlT3XBPq9rGWl6+wvMrD2DJrnOa\n9UXk6BhWiIiK8XDVISoyHA90rmesKQX8e+1RzPrlFJTiardENY1hhYjoNi46J3w0ojMiezQyqX/0\ny0m8FX0M+Vyen6hGMawQEZXAyUnw1tB2mNq3hUl98c5zeHnVQeTq8zXqjMjxVDmsiMhUw6te+UcT\nEdkeEcGL/Vri34PbmtRX77+MZ76KQ1auXqPOiByLOXdWPgLwPwA3LdQLEZFVGterCT4c0Qk6JzHW\nfvnjBiIX7kVKVq6GnRE5BnPCyk0AqUopPvmLiOze8C4NMHdkGFyLPU9o79kEPB61GzfTsjXsjMj+\nmRNW9gHwE5Ha5R5JRGQH7msbjCXjIuDj5mysHb2SghFzduFSYoaGnRHZN3PCymzD+W9YqBciIqvX\nvWkglk/ojkAvV2PtzM10PDJnF07fSNWwMyL7VeWwopT6EcDLACaJyFIR6WS5toiIrFf7+n5YNakH\n6vt7GGtXk7PwyJxdOHgxScPOiOyTVHWBIxE5Y/g2BEDhI0szAdwCUNoQeaWUalalC9q58PBwFRsb\nq3UbRFQJV5MzMXL+HvwZn26sebnqMC8yHD2bB2nYGZFtEJE4pVR4eceZ8zFQY8PLHYAYXp4AGhbb\nV9KLiMgu1PXzwKpJPdGxQdFT09Nz9BizKAYbj1zTsDMi+2LOnZW7qnKeUmprlS5o53hnhch2pWXn\nYcKSWOz885ax5iTAjOEdMaJrQw07I7JuFb2z4lzeAaVh6CAiKuDt5oyFY7pi2or92HT0OgAgXwF/\n/+4QkjNzMb5PU407JLJtXG6fiMgC3F10+OyJLhgR3sCk/p8Nf2DmxuN8ACKRGap8Z+V2IiIAWgEo\nXHclHsAJxf9DichBOOuc8P5DHeHv6Yqo388Y65//9ieSMnMx/YH2JqvgElHFmH1nRUSai8hiAMkA\njgL4zfA6CiBZRBaJSHNzr0NEZAtEBP8c2Bp/H9DKpL5szwVMXbEfOXl8ACJRZZkVVkRkKID9AEYB\n8EbRrKDClzeASAD7RWSwea3aJxEZIiJRycnJWrdCRBYiInj27uZ498EOkGI3Un44dBVPfRmDjJw8\n7ZojskHmPHW5GYAVALwAnAEwEUALAB6GVwsAkwD8aTjmG8M5VIxSKlopNcHPz6/8g4nIpjzRLRSf\nPH4HXHRFiWXbqZt4cv4eJGXwsWpEFWXOnZW/o2CNlV8BdFRKzVNK/amUyja8/lRKRQHoBGArChaO\ne8X8lomIbMfgjvUwf3RXeLjojLX9F5Lw6NzduJ6SpWFnRLbDnLDSD4ACMFEplVnaQYZ9E1HwsVB/\nM65HRGST7mpZG1893Q1+Hi7G2onrqXh4zk6cv5VexplEBJgXVuoCSFZKnS7vQKXUSQBJhnOIiBxO\nWKNa+GZiD9TxcTPWLiZk4uE5u/DH1RQNOyOyfuaElQwAniLiUt6BIuKKgnErpd6BISKyd61CfPDt\npJ5oFOhprMWnZmPE3F2IPZegYWdE1s2csHIYgAuA0RU4drTh2ENmXI+IyOaFBnpi1aQeaB3iY6yl\nZuVh5II9+PXEDQ07I7Je5oSVpSgYhzJbRJ42LApnQkTcRWQqgNkoGN/ypRnXIyKyC3V83LFyQg+E\nN6plrGXl5mP8l7FYe+Cyhp0RWSdzHmQoADaiaKDtNQDbAFxGwSyhUADdAASiINT8BGAgV7QtGR9k\nSOR4MnP0eObrOPx2It5YEwHefqA9RnVvpGFnRDWjog8yrPKdFUPoGAYgCgVhpS6AEQCeR8H6KoMA\nBBn2zQHwIIMKEVERD1cdokaFY0inesaaUsAb3x/BJ5tP8XlCRAZmPRvIMC15koi8B+BBAF1g+myg\nfQBWK6UumNUlEZGdcnV2wsePdoafhzO+2l30o/KDn08iMSMXrw9qAyc+T4gcnEUeZKiUOg/gY0u8\nFxGRo9E5CaY/0B61PF3xyZai1SAW7jiL5MxcvP9QBzjrzH6UG5HNMme5/X0iEiciTS3ZEBGRIxIR\nvNS/Fd4Y3Nak/t2+S5j01T5k5eo16oxIe+ZE9bYAWiilzpR7JBERVchTvZrgg0c6QVfso59f/riO\nSV/FIVfPJzaTYzInrFxGwSwfIiKyoIfCGmDOyDC4Ohf9iP7tRDz+/u0h5Odz0C05HnPCyiYUrGDb\nzVLNEBFRgX5tg/Hl2Ai4uxT9mF6z/zLe+eEPzhIih2NOWHkHwC0Ac0QkyEL9EBGRQY9mgfj8yS4m\nHwkt3HEWX2z9U8OuiGqeObOBmgP4F4APAJwQkSUAdqFgynKpI8GUUr+bcU0iIodyb+tgzHyoI15a\nddBYm7nxBAI8XfFYRKiGnRHVHHPCym8oWPANKBi7MtXwKosy85pERA7nobAGSMzIwTs//GGsvbbm\nMGp5ueL+diEadkZUM8wJDhdQFFaIiKgaPd27KW6m5WCO4SOgfAVMWb4fS8ZFoHvTQI27I6peVQ4r\nSqnGFuyDiIjK8Y8BrZCQno1vYi8BAHLyCh5+uGJid7Sr56dxd0TVh0siEhHZCBHBuw92wH1tgo21\n1Ow8jF4Ygwu3MjTsjKh6mbOCbaKI3OIKtkRENcdZ54RPn7gDEY0DjLWbadkYtXAPbqRmadgZUfUx\n586KKwAdV7AlIqpZ7i46zBsdjtYhPsba+VsZGLMwBilZuRp2RlQ9zAkrF1AQWByeiHQVkU9F5KiI\npIvIBRH5RkRaat0bEdknPw8XLBkXgYYBHsbasaspGP9lLJ8jRHbHnLCyDoCbiPSzVDM27B8AHgKw\nGcA0AFEA+gDYJyIdtGyMiOxXHV93LB3XDUHeRf9u3HM2AVOX70cenyNEdsScsPIugHMA5olIG8u0\nY7M+BNBIKTVVKTVfKfUOgN4omG31mratEZE9axzkhcVjI+DtVjS586dj1/H690e4LD/ZDXPWWXkA\nwBcA/g1gv4j8iIqtYLvEjGtaJaXUzhJqp0TkKAqeTk1EVG3a1/fDvMhwjF64FzmGOyorYi4i0NsV\nr9zfWuPuiMwnVU3eIpKPgkXhCh9aUaE3UkrpqnRBGyMiAuAigBNKqb7lHR8eHq5iY2OrvzEislsb\nj1zFs1/vQ/EHM78xuC2e6tVEu6aIyiAicUqp8PKOM+fOyu+oxhVsRcQHwD0AugIIN3wtXKaxjVLq\neAXeIwTAPwEMBlAfQDKAvQA+Vkptro6+i3nScM23qvk6REQAgAHt6+KdYR3w2prDxtr09ccQ6OWK\nYXfU17AzIvOYs4Lt3RbsoyR9Aayp6ski0hHAFhQFnBQAQSgILoNE5DWl1IzbzhEAbhW8RI5SqsQR\nbCLSGsBnAHYDWFiF9omIquSJbqFISM/G/346aay9vOog/DxdcE+rOhp2RlR11r6C7Q0AG1Bwd2JC\nRU8SEQ8UzFYKBLAfQHullB+AWih4SrQAeFdE+t926p0AMiv46lPKtUMA/ICCuzgPKaU4h5CIatTk\ne5pjTM/Gxu28fIVnvopD3PlE7ZoiMkOF76yIyFQA6UqpBSXs8wbgpJRKKeP8jwD4KqWequAlo5VS\n3xc7v3FFewUwEUAjAGkAhiilLgOAob+XRaQZgGEA3gPwU7HzTgIYW8Fr/OVjKBHxA/AjAH8AvZVS\nVyrRMxGRRYgI/j24LRLSc7DuYMGPoazcfIxbHINVk3qgZbBPOe9AZF0qPMDWMKD2qlLqLx98ishV\nALWVUqWGH8Mxdao6wNYQVs4aNsscsyIiMSgY5xKllJpYwv6eAHYYNlsrpU5Upafb3tMdBcEnDMB9\nSqldlTmfA2yJyNJy8vLx9JJY/H4y3lgL8XXHd8/2RH1/jzLOJKoZFR1gW9mPgaSK+2qMYWBumGFz\nUymH7UbBxzRAwdgYc6+pA7ASQA8Aj1Q2qBARVQdXZyfMGdkFnRv6G2vXUrIwasEeJKTnaNgZUeVY\n+5iVqmiDouB0tKQDDANjC++mWGIdlA8ADEXBR0ABIjKy+MsC709EVCWers5YNKYrmtfxNtbOxKdj\n7KK9SM/O07Azooqzx7BSt9j3ZY0ZKdxXt4xjKqqz4esQAEtLeJVIRCaISKyIxMbHx5d2GBGRWWp5\nuWLJuAjU9XM31g5eSsakr+KQk8dl+cn62WNY8Sr2fWYZx2UYvnqXcUyFKKXuVkpJaa8yzotSSoUr\npcJr165tbhtERKWq5++BpU9FwN/TxVjbduomXlp1EPn5XJafrJs9hhUiIipB8zo+WDSmKzxciuY5\nRB+8greij/I5QmTV7DGspBf7vqzh7p6Gr2nV2AsRkVW5I7QW5owKg7NT0U3fL3edx6dbTmvYFVHZ\n7DGsFB+nUq+M4wr3Xa3GXoiIrM5dLWvjgxGdTGof/HwSX+85r1FHRGWr7HL7ASKypaQ6AJSyz+SY\nGnAcRQ9YbIeiWT9GIuIEoJVh81gN9UVEZDUe6FwfCek5eCu66Efg698fQS1PV/ytgyXmHRBZTmXD\niiuAu8vYX9Y+oBoffGi8gFKpIhKLggcf9gOwuoTDugHwM3xf3Q80LJOIDAEwpHnz5lq2QUQOaOyd\nTZCQnoNPDB8BKQU8v+IA/D1c0LN5kMbdERWpTFj5stq6sLxlKAgrT4rI20qp2z/qednwNc4Sq9ea\nQykVDSA6PDx8vJZ9EJFjerFfS9xMy8HyvRcAADn6fIxfEosVE3qgQwO/cs4mqhkVDitKqYo+M8di\nRKR4tK9V7Hv/2/Yl3PYE5LkAnkfB84HWi8gopdQxw+q2bwAYbjjuterom4jIVogI3hnWHonpOdh4\n9BoAID1HjzGL9uLbZ3qiSZBXOe9AVP2sfYBtfLHXvmL1XbftCy1+klIqE8ADAG4B6ALgqIgkA0gC\n8AoKPo76p1Kq+EMMiYgcks5J8PFjndG9adHQwlvpORi1YA+up2Rp2BlRAWsPK1WmlDoIoD2A2QDO\nAHBDQXj5AUA/pdQMDdsjIrIq7i46zIsMR7t6vsbapcRMRC7Yi+SMXA07I7LysFLWqrC3vc6Vcv41\npdQ0pVQzpZS7UqqOUmqwUkrTQbVERNbIx90Fi8dGoFGgp7F24noqnvoyBpk5eg07I0dn1WHFEYjI\nEBGJSk5OLv9gIqJqVtvHDUvHdUNtHzdjLfZ8Ip5btg+5ej5HiLTBsKIxpVS0UmqCnx9H3RORdQgN\n9MSScRHwcS+ag7H5+A38c/VhLstPmmBYISKiv2hT1xcLRneFm3PRXxPfxl3CjI3HNeyKHBXDChER\nlSiiSQA+faILdMWeIzR36xlE/f6nhl2RI2JYISKiUvVrG4z3hncwqb274Ti+jbukUUfkiBhWiIio\nTCPCG+LVga1Nav/47hA2/3Fdo47I0TCsEBFRuSb2aYrxvZsYt/X5Cs9+vQ8x5xI07IocBcOKxjh1\nmYhsgYjgnwPbYHiX+sZadl4+nlocg+PXUjTsjBwBw4rGOHWZiGyFk5Pg/Yc64t7WdYy1lKw8RC7Y\ni4sJGRp2RvaOYYWIiCrMReeEz57ogrBGRc+WvZGajciFe5GQnqNhZ2TPGFaIiKhSPFx1WDi6K1oF\n+xhrZ2+m45mv4pCTx1VuyfIYVoiIqNL8PF3w5bgI1Pf3MNb2nE3A/607wlVuyeIYVoiIqEpC/Nyx\ncExXeLnqjLXley9i8c5z2jVFdolhhYiIqqxViA9mP34HpGiRW0xffwxbT8Zr1xTZHYYVIiIyS982\nwXh1QNGicfkKeG7ZPpy+kaZhV2RPGFY0xnVWiMgeTOjT1GQNltSsPDz9ZQySMjhDiMzHsKIxrrNC\nRPZARPDugx3QJdTfWDt3KwOTl+1Drp4zhMg8DCtERGQR7i46zB0Vjnp+7sbajtO38Hb0MQ27InvA\nsEJERBZT28cN80aHw8OlaIbQ0t3nsXTXOc16ItvHsEJERBbVrp4fPnq0s0ntzehj2HH6pkYdka1j\nWCEiIosb0D4EL/dvadwufErz2ZvpGnZFtophhYiIqsXke5pjaKd6xu3kzFw89WUMkjNzNeyKbBHD\nChERVQsRwcyHO6JTg6LZjmfi0zFl+X7kcYYQVQLDChERVRt3Fx3mRYYjxLdohtDvJ+Pxnw1/aNgV\n2RqGFY1xUTgisnd1fN0xLzIc7i5Ff+Us2nEOy/de0LArsiUMKxrjonBE5Ag6NPDDB4+YzhB64/sj\n2H3mlkYdkS1hWCEiohoxqGNdTOvbwridl6/wzFdxuHArQ8OuyBYwrBARUY2Z1rcFBnWoa9xOzMjF\n00tikJrFGUJUOoYVIiKqMU5Ogv890gnt6/saayevp2HaigPQ5ysNOyNrxrBCREQ1ysO1YIZQbR83\nY23L8RuYufG4hl2RNWNYISKiGlfXzwNRo8Lg6lz019Dc389gVexFDbsia8WwQkREmrgjtBb++3BH\nk9q/1hxB7LkEjToia8WwQkREmnmgc31MvqeZcTtHn4+JS+NwKZEzhKgIwwoREWnqpX6t0L9tsHH7\nVnoOnv4yFunZeRp2RdaEYYWIiDTl5CT46NHOaFO3aIbQ8WupeGHlAeRzhhCBYYWIiKyAl5sz5kWG\nIcjb1Vj76dh1fPDzCQ27ImvBsKIxPhuIiKhAg1qemDsqDK66or+aPvv1T3y//7KGXZE1YFjRGJ8N\nRERUJKxRAN4d3sGk9vfvDmH/hUSNOiJrwLBCRERW5eGwBpjYp6lxOycvHxOWxuFKUqaGXZGWGFaI\niMjq/H1Aa/RtXce4HZ+ajfFLYpGRwxlCjohhhYiIrI7OSfDxY53RMtjbWDt6JQUvrzrIGUIOiGGF\niIisko+7C+ZHdkUtTxdjbcPha5i1+ZSGXZEWGFaIiMhqhQZ6Ys7IMDg7ibE2a/MprD90RcOuqKYx\nrBARkVXr1jQQ7wxrb1J76ZuDOHQpSaOOqKYxrBARkdV7LCIU4+5sYtzOzsvH+CWxuJ6SpWFXVFMY\nVoiIyCa89rfW6NOytnH7eko2JiyJRVauXsOuqCYwrBARkU1w1jnh0yfuQLPaXsbawUvJeOXbQ1CK\nM4TsGcMKERHZDF93F8wf3RV+HkUzhKIPXsFnv57WsCuqbgwrRERkU5oEeeGLJ7tAV2yG0P9+OomN\nR65q2BVVJ4YVIiKyOT2bB+HNoe1Mai+sPIijV/hQWHvEsEJERDZpVPdGiOzRyLidmavH+C9jcSOV\nM4TsDcOKxkRkiIhEJSfzXwNERJX1xuC2uLN5oHH7SnIWJi6N4wwhO8OwojGlVLRSaoKfn5/WrRAR\n2RwXnRM+fyIMTYKKZgjtv5CE11Yf5gwhO8KwQkRENs3P0wXzIsPh4+5srK3efxlzfz+jYVdkSQwr\nRERk85rX8cZnT3RBsQlCeH/jcfx87Lp2TZHFMKwQEZFd6NOyNt4Y3Na4rRTw/Ir9OH4tRcOuyBIY\nVoiIyG6M6dkYj0eEGrfTc/R4+stY3ErL1rArMhfDChER2Q0RwVtD26FbkwBj7VJiJiZ9FYecvHwN\nOyNzMKwQEZFdcXV2whcjwxAa4GmsxZxLxBvfH+EMIRvFsEJERHYnwMsV80eHw9utaIbQytiL+GrP\nBQ27oqpiWCEiIrvUMtgHsx/vDCk2Q+itdUcRcy5Bu6aoShhWiIjIbt3bOhgv929l3M7LV3jmq324\nlswl+W0JwwoREdm1Z+9uhoHtQ4zbN9OyMemrOGTncUl+W8GwQkREdk1E8N9HOqFFHW9j7cDFJPzf\n2qMccGsjGFaIiMjuebs5I+q2JflXxFzEsr0ccGsLGFaIiMghNAnywqzHTAfcvrnuKOLOc8CttWNY\nISIih3Fv62C8eF9L43auXmHSV/twPYUDbq0ZwwoRETmUyfc0x/3tgo3b8anZeIYDbq0awwoRETkU\nJyfBByM6o3mxAbf7LiThzXXHNOyKysKwQkREDsfbzRlRo8LgU2yF2+V7L2AZV7i1SgwrRETkkJrW\n9sbHj3U2qf3fuiOIO5+oUUdUGoYVIiJyWH3bBOOF2wbcPvNVHG5wwK1VYVjRmIgMEZGo5ORkrVsh\nInJIU+5tjn5tiwbc3kjNxjNf70NOXr6GXVFxDCsaU0pFK6Um+Pn5ad0KEZFDcnISfDiiE5rV9jLW\n4s4n4q3ooxp2RcUxrBARkcPzcXdBVGQ4vIsNuP16zwWs4Aq3VoFhhYiICECz2t746FHTAbf/XnsU\n+y9wwK3WGFaIiIgM+rUNxrS+LYzbOfp8TPoqDjdSOeBWSwwrRERExUzr2wL3talj3L6eko3JHHCr\nKYYVIiKiYpycBB8+2hlNg4oG3MacS8T09VzhVisMK0RERLfxdXdBVGSYyYDbpbvP45uYixp25bgY\nVoiIiErQvI4PPhjRyaT2+vdHcOBikkYdOS6GFSIiolLc3y4EU+9tbtzO0edj0tI4xKdma9iV42FY\nISIiKsPz97VE39ZFA26vpWRh8tf7kKvngNuawrBCRERUhsIBt02KDbjdey4B73DAbY1hWCEiIiqH\nn4cLokaFwctVZ6x9ues8VsVywG1NYFghIiKqgBbBfx1w+6/vj+DQJQ64rW4MK0RERBU0oH1dPHdP\nsQG3efmYuDQON9M44LY6MawQERFVwgv9WuLuVrWN21eTOeC2ujGsEBERVYLOSTDrsTvQONDTWNtz\nNgH/+eEPDbuybwwrREREleTn4YK5o8LhWWzA7eKd5/Bd3CUNu7JfDCtERERV0CrEBx88Yjrg9rU1\nh84T8zsAABoNSURBVHH4UrJGHdkvhhUiIqIqGtihLp69u5lxOzsvH5O+isMtDri1KIYVIiIiM7zU\nvxXualk04PZyUiYmL9uHPA64tRiGFSIiIjPonASzH7sDoQFFA253n0nAuxuOa9iVfWFYISIiMpOf\npwuiIsPg4VI04HbhjrNYs58Dbi2BYYWIiMgCWof44r+PdDSpvfrdYRy5zAG35mJYsQAR6SIiq0Xk\nnIhkiki8iGwVkcFa90ZERDVncMd6mHSX6YDbiUvjkJCeo2FXto9hxTKaAnADsBDAFADvABAA0SIy\nScvGiIioZr1yfyv0bhFk3L6clInnOODWLKKU0roHuyQiOgBxALyUUi3KOz48PFzFxsZWf2NERFTt\nkjJyMOTT7biYkGmsje/dBP8a1FbDrqyPiMQppcLLO453VqqJUkoP4BIAf617ISKimuXv6YqoUeEm\nA27nbTuLtQcua9iV7WJYsSAR8RaRIBFpLiIvARgA4Get+yIioprXpq4v3n/YdMDtP747hKNXOOC2\nsqw2rIiIj4gMFZHpIvKjiNwUEWV4ta7ge4SIyCwR+VNEskTkuohEi0jfamp7DoB4AKcAzASwBsDk\naroWERFZuaGd6mFin6bG7azcggG3iRxwWylWG1YA9AWwFsDrKLhDEViZk0WkI4AjAKaiYABsNoAg\nAIMB/Cwir5ZwjoiIewVfJf3evQegH4DRAH4C4Pz/7d15nFx1me/xzzedfesOJJAQCJAgARKCSRAQ\nkcUZFsEADuIAES84Q+AK1+QOCCIO19FBVIZhVQiKIpvRUVQWHZ0I4sLikBAYExJASNiTkJCFpLNA\nnvvHOdVVaaor3enlnKr+vl+vep36nfM7VU9Ouuo8dc5vIWl4a2Zm3dTnjx3LYXsVG9y+8lYjF/zQ\nDW7bIs/JCsAy4JfAvwDTWruTpH7AvSQJzpPA+IioB4YAV5P01PmapGOa7fohoLGVj8Obv29EzI+I\n2RFxO3A8MBC4V5JaG7uZmdWWnnU9uOH0iew6pF/Tuj89v4Krfr0ow6iqS8+sA6jgvoj4eaEgaY82\n7HsusDvwNjAlIl4FiIg1wEWSxgAnk1wJ+U3Jfs8CZ7fyPSqOoxwRIek/gJnA3oD/Ks3MuqkhA3oz\n88zJnHLTI2zYnFxRmfn7Fxg3sp4TD9gl4+jyL7fJStqbZntNTZd3FxKVZq4iSVYmSRobEYvS91wG\n3NaO922ukEbXd+BrmplZFRq3Sz3fOGUC02fNa1p38U+eYq9hA9lvl8EZRpZ/eb8N1GaSBgGT0+Kv\nW6j2GFBojt3uxraSdiqzrjdJ25VGYEF738PMzKrfSe8fyTkf3rOpvGHzFs698wk3uN2GmktWgH1J\n2qQAzC9XISK2ULwt0xEj9MxKeyz9s6R/lHQ5SePeicCXIuLtDngPMzOrAZcctw+Hjin2GXl5ZSOf\nm/Uk727xIK0tqcVkZUTJ89cq1CtsG1GhTmvdAfQFLgBuAqYDi4GTIuLfO+D1zcysRvSs68GNZ0xi\nZEOxwe0fnnvTDW4rqMVkZUDJ88YWa8H6dDmwvW8YEd+PiKMiYueI6BURO0bEMRFxb6X9JE2T9ISk\nJ5YvX97eMMzMrErskDa47dOzeBq++eG/cv/TlX5jd1+1mKxUjYi4JSIOjIgDhw0blnU4ZmbWhcaP\nTBrclrr4J0/z7NK1GUWUX7WYrKwred6vxVrQP126PYmZmWXi5Ikj+cyHig1u1296l/PumMOaDZsz\njCp/ajFZKb2GVqnzemHb650Yi5mZWUWXHr8PB+25Q1P5hTfXceGPn2KLG9w2qcVkZSFQ+B8eV65C\nOlT+2LTobsVmZpaZXnU9+NYZk9h5cHF2lv9asJSbHv5rhlHlS80lKxGxFngiLR7dQrWDKQ7U9ttO\nD6oCSVMk3bJ6tWfhNDPrroYN6sO3p06mV11xdpZ/+80ifv+sO19ADSYrqbvT5VRJ5bomX5Qu5xRG\nr81KRNwXEdPq6z3IrZlZdzZ59yFcPqV4QyACPjfrSV5eub7CXt1DrpMVSUMLD5JJCAsaSreVmQF5\nJrAEGATcL2m/9PUGSfom8HdpvS929r/BzMystT518ChOmbRrU3nV+s2cd+ccNmxuzww01S/XyQqw\nvOQxt2T9o822jSrdKSIagZOAFcAkYL6k1cAq4PMkbVoujYjSSQzNzMwyJYkrPj6ecSVzBc1/bQ2X\n/ewvRHTfBrd5T1a2W0Q8BYwHrgdeAPqQJC8PAEdHxNczDM/MzKysvr3quPlTk2no36tp3U/nvsKd\nj7+UYVTZynWyEhFq5WNxC/u/ERHTI2JMRPSNiJ0i4mMRkWmjWjMzs0p226E/1582ERXb2/KV++Yz\nZ8lb2QWVoVwnK92BewOZmVk5h+89jIuOGdtU3vxu8Nm75rB87cYMo8qGk5WMuTeQmZm15H8fMYaj\n99u5qbx0zUbOv3sum9/dkmFUXc/JipmZWU716CGu/uQBjB5anKP3zy+u5Ou/WphhVF3PyYqZmVmO\nDe7bi5vPnEz/3nVN627944vc+1T3maHZyYqZmVnO7b3zIK76xAFbrbvkJ0+z8I01GUXUtZysmJmZ\nVYETJoxg2uGjm8qNm5MZmlc31v4MzU5WzMzMqsTFx47lg6N3bCovXrGeC388r+ZnaHaykjF3XTYz\ns9bqWdeDG86YyIj6vk3rZj+zjBsfej7DqDqfk5WMueuymZm1xdCBffj21En0riuewq+Z/SwPLVqW\nYVSdy8mKmZlZlZk4aghfPnHrGZpnzJrHSytqc4ZmJytmZmZV6PSDduOTBxZnaF7duJlz75xD46ba\nm6HZyYqZmVkVksRXThrPhF2LzQieeX0Nl/3sf2puhmYnK2ZmZlWqb686vj11EkNKZmi+58lXueOx\nJRlG1fGcrJiZmVWxXYf054bTJ9FjqxmaF/DE4pXZBdXBnKyYmZlVucPeN5TPH7tPU/mdLcFn75rL\nsrUbMoyq4zhZyZjHWTEzs45w3hGjOW7c8KbysrUbOf+u2pih2clKxjzOipmZdQRJXHXqBMYMK87Q\n/N+L3+KKB57JMKqO4WTFzMysRgzq24uZZ05mQMkMzbc9sphfzHs1w6jaz8mKmZlZDdlrp0H826nN\nZmj+6dM883r1ztDsZMXMzKzGfHT/EZx3xJim8obNWzj3jjmsXl+dMzQ7WTEzM6tBFx2zNx/aqzhD\n80sr1zPjR09W5QzNTlbMzMxqUM+6Hlx/2kR2KZmh+aFFy7n+wecyjGr7OFkxMzOrUTsO7MPNZ06m\nd8/i6f7a2c/x4MKlGUbVdk5WzMzMatiEXRv46knjtlo3Y9Y8lqxYl1FEbedkxczMrMb9/QdGcfpB\nuzWV12x4h3PvmMP6Te9kGFXrOVnJmEewNTOzrvDlE8dxwG4NTeWFb6zl0nuqY4ZmJysZ8wi2ZmbW\nFfr0rOOmqZPYcUDvpnW/mPcatz2yOLugWsnJipmZWTexS0M/bjh94lYzNF/xwDP8+cV8z9DsZMXM\nzKwbOXSvoXzho++doXnpmvzO0OxkxczMrJs558OjOWH/EU3lN9/eyGfvmsumd/I5Q7OTFTMzs25G\nEt/4xAT22mlg07o5S97iigcWZBhVy5ysmJmZdUMD+/Rk5pmTGdinZ9O6Hzy6hHvmvpJhVOU5WTEz\nM+umxgwbyNWf3HqG5kvv+R/mv5av4TScrJiZmXVjx44bzvlHFWdo3vjOFs67cw6r1m/KMKqtOVkx\nMzPr5v7p6LF8+H1Dm8ovr2xk+qx5vJuTGZqdrJiZmXVzdT3E9adNZGRDv6Z1Dz+7nOtmP5thVEVO\nVjLm4fbNzCwPhgzozcwzJ9OnZIbm6x98ntkLsp+h2clKxjzcvpmZ5cX4kfX868njt1r3f380jxff\nzHaGZicrZmZm1uTUA3dj6sGjmsprN77DeRnP0OxkxczMzLZy+ZT9mDiqOEPzoqVrueSn2c3Q7GTF\nzMzMtpLM0DyZoQOLMzTf99Rr3PrHFzOJx8mKmZmZvcfw+r7ceMYk6kqmaL7yVwt57IUVXR6LkxUz\nMzMr65DRO3JpyQzN726JTHoHOVkxMzOzFv3DYXvysQkj6FUn/vXk8Vx2wr5dHkPPbVcxMzOz7koS\n3/zEBKYdPpoJuzZse4dO4CsrZmZmVlH/3j0zS1TAyYqZmZnlnJMVMzMzyzUnK2ZmZpZrTlbMzMws\n15ysmJmZWa45WTEzM7Ncc7KSMUlTJN2yevXqrEMxMzPLJScrGYuI+yJiWn19fdahmJmZ5ZKTFTMz\nM8s1JytmZmaWa05WzMzMLNecrJiZmVmuOVkxMzOzXHOyYmZmZrnmZMXMzMxyzcmKmZmZ5ZoiIusY\nDJC0HFjSwS87FHizg1+zu/ExbD8fw/bzMWw/H8P264xjuHtEDNtWJScrNUzSExFxYNZxVDMfw/bz\nMWw/H8P28zFsvyyPoW8DmZmZWa45WTEzM7Ncc7JS227JOoAa4GPYfj6G7edj2H4+hu2X2TF0mxUz\nMzPLNV9ZMTMzs1xzsmJmZma55mTFzMzMcs3JSo2RNFzSdZL+KmmDpKWS7pP0N1nHlneSRkmakR6v\nlyRtlLRW0lOSvi5pRNYxVhtJAyW9LCnSx1lZx1RNJI2VdIOkRZLWSVot6RlJ35N0RNbx5ZmkHpLO\nljRb0nJJmyWtkvS4pMskDco6xqxJGiTpRElflfQrSW+WfFb3acX+vSVdLGmepLfT4/uopGmS1KGx\nuoFt7ZA0AXgQ2DFdtQYYSJKUBvDFiPh6RuHlmqTdSEYQLv2ArQEGAHVp+S3glIh4qIvDq1qSrgWm\nl6w6OyJuyyicqiLpc8BVQO901dtAT6BvWr41Iv4xi9jyTlJ/4D7gIyWrVwODKX7GlwAfiYgXuji8\n3JB0MvCzFjbvGxELK+w7mOR8MzldtZ7k77Pw93o/8PGIeKcjYvWVlRohqR9wL0mi8iQwPiLqgSHA\n1SQf0K9JOia7KHOtkJA8AJwK7JAev/7A8cCLJMfy55KGZxNidZE0CbgAeDzrWKqNpHOB60i+/L9B\nMiT5oIjoB4wAPg08kmGIeffPJIlKAJcCDRHRQJLonQ6sAnYHvptZhPmxDPgl8C/AtDbs9x2SRGUl\nMIXkh3F/4CxgA/Cx9DU7hK+s1AhJM4BrSH597RMRrzbb/jPgZGBuREwu8xLdmqR6YI+IeKqF7fuQ\nJIF9gS9HRId9CGuRpB4kScpE4APA3HSTr6xsg6Q9gPkkX/zTIuI7mQZUhSQtAUYB34uIfyiz/Szg\n+2lxh4h4qwvDyw1JdRHxbkl5D5IfZlDhyoqkiRQ/0ydFxL3Ntk8HrgUaSb5Xl7U3Vl9ZqR1T0+Xd\nzROV1FXpcpKksV0UU9WIiNUtJSrp9oXAY2nRyd62/R/gQOCmiHgy62CqzHSSROVxJyrbbed02dLf\n3pyS5/07OZbcKk1U2uiMdLmoeaKSuoXktls/4O+28z224mSlBqQNxQon0F+3UO0xkj8eADe23T4r\n0mVdxVrdnKSRwFeBpcCXMg6nGhVOBD/MNIrqtjhdTmxhe+H7cmkLP+6ssqPS5W/KbYyIRuAPafEj\n5eq0lZOV2rAvxUZj88tViIgtwKK0uF9XBFVLJPUEPpQW/5JlLFXgBmAQcFFErN5WZSuSNAbYKS0+\nKemQtHfaCkmNkhZKukrSTpVexyhckTpb0hfS27yF3it/T3LLPICLsgqwWqW9fAo9hcqeb1IL0mWH\nnG+crNSG0i61r1WoV9jmLrhtdz4wHNgC/CDjWHJL0hTg48DvIuLOrOOpQu8reX4k8EeShoq9SE6u\nY0lOsPMkjevy6KrHtcC3SH7EXQmskrSKpA3FLGAhcKL/RrfLYJJektCF5xsnK7VhQMnzxgr11qfL\ngZ0YS81Ju4RfmRZvjIgFlep3V5IGADcCm0mSO2u7hpLn/w94FjgkIgaTfG6PJ+m9MQL4aXrFz5pJ\n22LMAC4ECl1n6yme8wYBwzIIrRZkcr5xsmJWQToQ3M9JGorNAS7JNqJc+wpJD4xrnNBtt9Lv5CAZ\np+JxSG7lRsSvgM+k28fSQY0Xa006vMCfSIZtuAs4gOSk+T6Srsyjge9JurLFF7FccbJSG9aVPO9X\noV6h1fvbnRhLzZC0A0kDsj2B54ATImJDtlHlk6T3k/RieZkkabHtU/rZ/M+IWNS8QkQ8QHLFBdxY\nviW3AweRDJx3VkQ8HRHrIuL5dGDMc9N6F/t2Wptlcr5xslIbSu8b7lKhXmHb650YS01IG+T9GhgP\nvAT8bUQszTaqXLuOpJfUZSRt8AaWPkrq9UnXddvuottQ+ll+T6JSZttunRhLVZK0H3B0WrymXJ2I\nuIOkd18PkgHNrPXWUExYuux842SlNiwkuWQMUPZXQjpIV2F8FV+iryBte/FLknFC3iBJVF7KNqrc\n2z1d3g6sLfMouDkt+2+wvAUkjbhby6N6vte+Jc9fbLEWFIbZ36PzQqk9kYwk+0xarHRVqtALqEM+\n605WakBErAWeSItHt1DtYJIGZgC/7fSgqlQ6bcF9wKEkv7z+NiKeyzYq6y4iYj3waFqsNHhjYdvi\nTg2oOpUme6Mq1Csk2Gsr1LHyCvOjlT3fSOoLfDgtdsj5xslK7bg7XU5tYXbgwngCc8rdB7dkDAbg\nHpIBj1YBx0REpXEELBURe0SEWnqUVD07XbdHVrFWgdvT5XHlRpuWdAKwd1r8ZZdFVT1KR6I+p1yF\ntIt9Yawaz13VdoUBC/eR9LEy288h+XHcSMsTJbaJk5XaMZNkFtFBwP3pfdvCFODfpNhr4IsZxZdr\nkupIEr7jSH5pfTQi5lbey6xTfI/k0nkdcI+kgyC5lSvpOODWtN5jOFl5j3QW5cLIqjMkXVkYRC9t\nL3UWcFu6fTHJBLDdlqShhQfJZK0FDaXb0qYEAKRTaPw4Ld4m6fj0teokfZpk8k1Iega2e14g8ESG\nNUXSASSX3HZMV60h6a7Xg+Te9hfTlvDWjKTDgYfT4gaKUxOU83JEfKDzo6odkgpfNJ7IsBUkjQZ+\nR7EB7VqS5KXQMHkByZU/DxVfRnp1+bds3X5lLcmPuYKlJD9KuvXcVSWfzW3ZMyIWl+w3GHiQ4tQF\n60n+Rvuk5ftJut6/QwfwlZUakk7ENx64nqTxWB+SdhcPAEc7Uamo9LPQl2QitJYeHkzKOlV6dWB/\n4AqSxKQnyQ+OuSTjhBzkRKVlEfE6yUl0BvB7YCVJoreG5Bh+Fdi/uycq7RERa0ja9n2B5NZbABtJ\nrvidSzJCcIckKuArK2ZmZpZzvrJiZmZmueZkxczMzHLNyYqZmZnlmpMVMzMzyzUnK2ZmZpZrTlbM\nzMws15ysmJmZWa45WTEzM7Ncc7JiZmZmudYz6wDMzLZFUk/gU8BpwAEk81+tA94gmVriD8CDEfHn\nkn3eD5wMLPZ8RGbVzcPtm1muSRpGMrvwgSWrN5DMQzIYULpudUQ0lOx3FvB94OGIOLJLgjWzTuHb\nQGaWd3eSJCprgYuBERHRL01M6oGjgW8Dq7IL0cw6k28DmVluSdoHOCYtfiYiflK6PSLWArOB2ZIu\n7Or4zKxr+MqKmeXZ/iXP769UMSI2FJ5LCpJbQABHSIpmjyOb7y/pMEmzJL0iaaOkFZJmSzpdksrU\nPzJ9rcVpeYqkhyS9JeltSY9KOmM7/s1m1oyvrJhZtRgJ/LWVdZcC/UjatGwGVjbbvqm0IOkbJLeY\nCtYAQ4C/SR8nSpoaEVvKvZmkGcA1QACr0/c+BDhE0qERcUEr4zazMnxlxczybE7J82+ljW23KSKG\nA9PT4iMRMbzZ45FCXUnTSRKVpcA0oCEi6oEBJL2P3kiXl7TwdsOAbwK3k7SnGQIMBa5Ot5/vKyxm\n7ePeQGaWa5J+AHw6LW4i6ab8GPDfJInI8hb2O4tt9AaS1AC8THKV+ZCIeKpMnQ8CfyJpwDs8Ijal\n648EHkqr/RdwbDT7QpV0G/C/gOeBvZtvN7PW8ZUVM8u7c4B/J0lUepPclrkM+DmwTNKfJU0t166k\nFU4BBgKzyyUqABHxKPAiyW2hyS28zpUtJCJXpMu9SMaHMbPt4GTFzHItIjZFxIXAbsB5wA+B50ja\nhwB8gKR7848ktfU77dB0+RFJb7T0SN+bkmWpzSRXXsrF/hzwelqc1MbYzCzlBrZmVhUiYhkwM30g\naWdgCnA5SRJxKknScF0bXnZEuuyfPralXJ03C7eGWvBq+j6tam9jZu/lKytmVpUiYmlEfJfkisXS\ndPVn2vgyhe/A6yJCrXjc1lHxm1nrOVkxs6oWEW8Cv0iLe7dx90KSM6odIQyV1LvC9l3SZdmGwGa2\nbU5WzKwWrEuXpbdjCmOiVGp4+2i6PFJSv+18717AB8ttkLQXxWRl7na+vlm352TFzHJL0p6Sxmyj\nTn+S2ZUB5pVsWpMuG2jZf5AkOkNI2r5Uep8hFTZf2kJvpEvT5XMRMa/MdjNrBScrZpZn44BFku6R\n9ElJhQaxSBogaQrJuCt7pqtLG9fOT5f7STq43ItHxAqKCcUXJH1HUtOtJEn9JR0l6RbgkXKvAawn\n6U59q6Sd0v0a0lFxC21ovtzKf6+ZleFB4cwstyQdC/xns9WNJLd76kvWvQtcHhFfa7b/w8DhaXEl\nyczNAKdFxGMl9b4EfIXiLaN16Xs0lKxbHBF7luxzJMmgcEuAaykOt78qja3wY/BbHm7frH2crJhZ\nrqVXOqYAhwHjSeYI6k2SeLwA/B74bkTML7PvjiRJyEdL9gM4KiJ+16zu/sAFwFHArkAdSaPYvwC/\nBX4YEa+U1D+SNFmJiD3Sqzz/BEwkacfyNHBjRNzV7oNg1s05WTEz2w7Nk5VsozGrbW6zYmZmZrnm\nZMXMzMxyzcmKmZmZ5ZqTFTMzM8s1N7A1MzOzXPOVFTMzM8s1JytmZmaWa05WzMzMLNecrJiZmVmu\nOVkxMzOzXPv/4lIOSId1am4AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "error_plot([log_likelihood(w, data, labels) for w in ws_newton])\n", "plt.plot(range(len(ws_newton)), [baseline]*len(ws_newton),\n", " label='sklearn', **kwargs)\n", "plt.legend();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Speed-up via linear equations\n", "\n", "We'll now see how to speed up Newton's iteration with a simple trick. Instead of computing a full pseudo-inverse, we'll solve set of a linear equations approximately. Recall that the system $Hx=g$ has the solution $x=H^{-1}g$." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def newton_method_lsq(init, steps, grad, hessian, least_squares, \n", " num_to_keep=None):\n", " \"\"\"Newton iteration with least squares solver.\n", " \n", " Parameters\n", " ----------\n", " initial : array\n", " starting point\n", " steps : list of floats\n", " step size schedule for the algorithm\n", " grad : function\n", " mapping arrays to arrays of same shape\n", " hessian : function\n", " mapping 1d arrays to 2d arrays\n", " least_squares : function\n", " solver for linear equations\n", " num_to_keep : integer, optional\n", " number of points to keep\n", " \n", " Returns\n", " -------\n", " List of points computed by Newton iteration. Length of the\n", " list is determined by `num_to_keep`.\n", " \"\"\"\n", " xs = [init]\n", " for step in steps:\n", " # Use least squares solver for Hessian-gradient product\n", " direction = least_squares(hessian(xs[-1]), grad(xs[-1]))\n", " xs.append(xs[-1] - step * direction)\n", " if num_to_keep:\n", " xs = xs[-num_to_keep:]\n", " return xs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we use a full least squares solve, this is just equivalent to the pseudo-inverse we had before. This why Newton's method for logistic regression is sometimes called **iteratively reweighted least squares**." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAisAAAIGCAYAAACVnM/AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VGXaBvD7yaR3kkBCC71XSQhFwIIgLEVExQahKEUR\nsO66ru6n4iqyawEbhCooRRSEIIIKinSS0EGa9B5I78nk/f7IZJLB9JnkTLl/1zVXcp5zzpwHxHBz\n5n3fI0opEBEREVkrJ60bICIiIioLwwoRERFZNYYVIiIismoMK0RERGTVGFYsRETcRGSGiFwWkUwR\n2Ssi92vdFxERka1jWLGcxQBeArAcwDQAuQB+EJG7tGyKiIjI1gmnLptPRCIA7AHwqlLqfUPNHcAR\nAAlKqQgt+yMiIrJlvLNiGQ8DyAcQVVhQSmUBWACgq4g01qYtIiIi28ewYhl3APhTKZV4W31vsf1E\nRERUBXYXVkTER0SGish0EflRRG6KiDK8WlfwPUJEZJaI/CkiWSJyXUSiRaRvKafUBXC1hHphrV5V\nfi1EREQEOGvdQDXoC2BNVU8WkY4AtgAINJRSAAQBGAxgkIi8ppSacdtpHgCyS3i7rGL7iYiIqArs\n7s6KwQ0AGwC8BWBCRU8SEQ8A61AQVPYDaK+U8gNQC8AHAATAuyLS/7ZTMwG4lfCW7sX2ExERURXY\n452VaKXU94UblRzcOhFAIwBpAIYopS4DgFIqBcDLItIMwDAA7wH4qdh5Vw3n3a6u4euVSvRARERE\nxdjdnRWllN6M0580fF1WGFRu81/D1y4i0qpY/QCAZiJS67bjuxXbT0RERFVgd2GlqkTEB0CYYXNT\nKYftBpBs+L74YNtvUfB7afzISUTcAIwFEKeUOmvZbomIiByHPX4MVFVtUDAmBQCOlnSAUipfRE4A\niADQtlh9j4isAvCOiAQBOAUgEkATAP2qtWsiIiI7x7BSpG6x78saY1K4r+5t9UgAbwMYCSAABavX\nDlFK/VqRiwcFBanGjRtXrFMiIiI7EBcXd1MpVbu84xhWingV+76s2TsZhq/exYuGFWv/bnhViIhM\ngOGjo9DQUMTGxlb0VCIiIpsnIucrchzHrGhIKRWllApXSoXXrl1usCQiInJIDCtF0ot9X9Yibp6G\nr2nV2AsREREZMKwUKT5Opazl8Qv3lbS8PhEREVkYw0qR4wCU4ft2JR0gIk4ACtdXOVYTTRERETk6\nhhUDpVQqgMIRrqVNN+4GwM/w/WZLXFdEhohIVHJycvkHExEROSCGFVPLDF+fFJHbpyYDwMuGr3FK\nqROWuKBSKlopNcHPz6/8g4mIiByQXYYVEQkqfKHgIYSF/IvvM3ysU9xcAOcB+ABYLyJtDe/nIyIz\nAQw3HPdadf8aiIiIqIC9rrMSX0p9123bTQCcK9xQSmWKyAMo+IinC4CjIpKCgjVVnFAwpuU1pdRP\nICIiohphl3dWzKGUOgigPYDZAM4AcANwC8APAPoppWZo2B4REZHDscs7K0opKf+oMs+/BmCa4UVE\nREQassuwYktEZAiAIc2bN9e6FSKyMkoppKamIiUlBRkZGdDr9Vq3RGRCp9PB09MTvr6+8PHxgYhZ\n9wpKJUqp8o+iahceHq74bCAiKqSUwo0bN5Ceno6AgAB4e3tDp9NV218GRJWllIJer0daWhoSEhLg\n5eWFOnXqVOrPqIjEKaXCyzuOd1aIiKxQamoq0tPT0ahRI+h0Oq3bIfoLEYGzszP8/f3h4+OD8+fP\nIzU1Fb6+vha/FgfYEhFZoZSUFAQEBDCokE3Q6XQICAhASkpKtbw/wwoRkRXKyMiAt7e31m0QVZi3\ntzcyMjKq5b0ZVuyQPl/hg59O4FpyltatEFEV6fV63lUhm6LT6aptEDjDip3JytVj8tf78MmW0xi9\ncC+SM3K1bomIqoiDacmWVOefV4YVjVn6QYb/XnsEG49eAwCcuJ6Kp5fEICuX0x2JiMh2MaxozNIP\nMpxybwvU9nEzbsecS8Rzy/YjT59vkfcnIiKqaQwrdqZhgCeWjIuAj1vRrPRf/riOf605Aq6pQ0RE\ntohhxQ61qeuL+aPD4epc9J93ZexFfPDTSQ27IiKyPosXL4aI4O67767UeW+++SZEBGPGjKmWvsgU\nw4qd6tY0EJ88fgecio13+vTX01i846x2TREREVUBw4odu79dCP7zYAeT2lvrj2HdwSsadURERFR5\nDCt27vGIULzUr6VxWyngpW8OYNupeA27IiIiqjiGFQfw3L3NEdmjkXE7V68waWkcDl1K0rArIiKi\nimFY0Zil11kp5Rr4vyHtMKhjXWMtPUePsYticPZmerVdl4hICzk5OZg1axZ69uwJf39/uLi4IDg4\nGJ06dcLkyZOxa9euCr/X0qVL4eLiAicnJ3zxxReV6mP79u147LHH0KBBA7i5uSEwMBD33Xcfli9f\nXurszH379uHVV19Fr169EBoaajzv7rvvxvz580tdIbb4gN/8/Hx8+umniIiIgL+/P0QEBw4cAACM\nGTMGIoI333wTer0eH3/8MTp16gRPT08EBARg8ODBiI2NrdSvsybwqcsaU0pFA4gODw8fX53X0TkJ\nPhzRCUkZOdhx+hYA4FZ6DkYt2IPVz/REHV/36rw8EVGNyMvLQ//+/bF161YABf9Y8/Pzw61bt3Dj\nxg0cOnQIt27dQo8ePcp9r88++wxTpkyBTqfDkiVLMHLkyAr38Y9//AMzZ840bvv6+iIxMRGbN2/G\n5s2bsW7dOnz99ddwcjK9Z9C/f3/culXwM9rT0xOenp5ISEjA1q1bsXXrVqxZswZr166Fs3PJf30r\npTB8+HCsXbsWOp0OPj4+JR6Xl5eHQYMGYdOmTXBxcYGbmxsSExPxww8/YPPmzdiyZUuFfo9qCu+s\nOBA3Zx3mjAxD+/pFj+++lJiJyIV7kZzJZfmJyPYtW7YMW7duhaenJ5YuXYqMjAwkJiYiOzsb58+f\nx6effopOnTqV+z7/+c9/8Nxzz8HV1RXffvttpYLKrFmzMHPmTAQHByMqKgpJSUlITk5Geno6VqxY\ngZCQEKxYsQLvv//+X87t378/li9fjqtXryI9PR2JiYlIS0vD0qVLERISgg0bNuCjjz4q9dqrV6/G\nxo0b8fnnnyMlJQWJiYm4fv06mjZtanLcZ599hpiYGKxcuRJpaWlITU3FwYMH0b59e2RlZWHatGkV\n/vXWBN5ZcTA+7i5YNCYCD8/ZifO3Cp6OefxaKsYvicWScRFwd+GD04hswo+vAtcOa92FZYR0AAbO\nsMhb7d69GwAQGRlpEjB0Oh1CQ0MxefLkct/jlVdewf/+9z94eXlh7dq16Nu3b4Wvn5SUhNdffx3u\n7u7YtGmTSTDy8PDAo48+itDQUNx5553473//i5deegmurq7GY5YtW/aX9/Ty8sLIkSPRqFEj9OnT\nB59//jleeeWVEq+flpaGuXPnYsKECcZanTp1Suxz27Zt6NWrl7HWsWNHLF68GOHh4YiJicGFCxcQ\nGhpa4V97dWJYcUC1fdywdFw3DP9iJ26mZQMA9p5NwLQV+/H5k2HQOfHhaURW79ph4Px2rbuwOr6+\nBXeOr169Wulz8/PzMXHiRMyfPx/+/v7YsGFDpT8K+e6775CWlobBgweXegenR48eaNKkCc6cOYO4\nuLgKX6N3797w9/fHuXPncOXKFdSrV+8vxwQGBmLcuHEVeq/iQaVQWFgYGjRogEuXLuHIkSNWE1b4\nMZCDCg30xJfjusK72LL8m45ex+vfc1l+IrJdAwcOBACsXbsWQ4cOxerVq41jQMqSm5uLxx9/HPPn\nz0edOnXw22+/VWnMxs6dOwEAW7ZsQUhISKmvixcvAoDxa3GrVq3CsGHDEBoaCg8PD4iI8ZWUVDCL\n88qVktfLCg8PL3U8S3Fdu3YtdV/9+vUBAImJieW+T03hnRUH1q6eH6IiwzBmYQxyDA86XL73Amp7\nu+LF/q007o6IyhTSofxjbIUFfy133XUX3n77bbz99tuIjo5GdHQ0AKB169YYNGgQJk6ciBYtWvzl\nvMKQAQDffvtthca1lKTwjk5GRgYyMjLKPb74MXl5eRgxYgTWrFljrLm5uSEoKAg6XcFH9PHx8cjP\nz0d6eskzOWvXrl2hPksbeAsA7u4FEy5yc61nLCPDioPr2SwIsx7rjGeX7UPhDZXZW04jyMcNkT0a\na9obEZXBQmM87NEbb7yBkSNHYuXKlfjtt9+wa9cuHD9+HMePH8esWbOwYMECREZGmpzToUMH6PV6\nHDt2DM8++yx+/fVXBAUFVfra+fkF//CbNm0aPv7440qdO2/ePKxZswaenp6YMWMGHnzwQTRo0MDk\nmIYNG+LSpUul3gEvDDX2hh8DEQZ2qIvpD7Q3qf3fuqNYf4jL8hORbWrSpAleffVVbNy4EQkJCfj1\n11/Rp08f5OXl4dlnn8WNGzdMjg8ICMDmzZvRqlUrHDlyBP369avSxyDBwcEAgAsXLlT63FWrVgEo\nCFtTpkz5S1DR6/W4efNmpd/XHjCsEABgZPdGeP6+olujSgEvrDyAHacd838MIrIfOp0Od999N9av\nXw8XFxekp6eXuPBZSEgINm/ejGbNmuHAgQPo378/KrtgZ+E4l99++w2ZmZmVOvfSpUsAgDvuuKPE\n/Tt27EBWVlal3tNeMKxorCZWsK2oaX1b4MluRSO/c/UKE5bE4shl7XsjIqqInJycUve5uroaPybJ\nzs4u8Zj69etjy5YtaNy4MWJjYzFw4ECkpaVV+PqPPPIIvLy8kJiYiLfffrvMY2+/c+Pn5wcAOHz4\nr1PS8/Ly8Prrr1e4D3vDsKIxpVS0UmpC4R9SLYkI3n6gPQa2DzHW0nP0GLNoL85xWX4isgGRkZEY\nO3YsNm3ahNTUVGP93LlzGD16NLKysuDh4YHevXuX+h6hoaHYsmULGjRogF27dmHQoEEVGiwLFEwd\nfu+99wAAM2bMwPjx43Hy5Enj/oyMDPz666+YMGECevbsaXJuv379AADTp0/H2rVrjUvrHz9+HEOG\nDMHevXvh5eVVsd8IO8OwQiZ0ToKPHu2M7k0DjLWbaTmIXLgXN1Id8/YjEdmOrKwsLF68GAMGDICf\nnx9q1aoFLy8vNGnSBCtXroROp8PcuXPLHTzbpEkTbNmyBXXr1sXvv/+OoUOHVvgjmClTpmD69OkQ\nEcyfPx+tWrWCt7c3AgIC4O3tjXvvvRfz5s37y/u9/PLLaNasGVJSUjBs2DB4eHjAz88Pbdq0wc8/\n/4w5c+ZUadCvPWBYob9wd9EhKjIcbesWLct/ISEDoxfGICXLeqayERHdbsaMGZg5cyYGDBiApk2b\nIicnB3q9Hs2aNcPYsWOxb98+jBo1qkLv1aJFC2zZsgXBwcHYvHkzHnzwwVI/Prrd66+/joMHD2LC\nhAlo0aKFcbpxvXr1cP/992PmzJnYtm2byTkBAQHYvXs3nnnmGePgWg8PDwwbNgxbt27FmDFjKvV7\nYU+EC4BZh/DwcGVtT7q8kZqFh7/YhQsJRbc/uzcNwOKxXJafqLr98ccfaNOmjdZtEFVKZf/cikic\nUiq8vON4Z4VKVcfHHUvGRSDIu+i5FbvPJOCFlQegz2fIJSKimsGwQmVqHOSFxWMjTJbl//HINfx7\nLZflJyKimsGwQuVqX98PUaPC4Kor+uPy9Z4LmLX5lIZdERGRo2BYoQrp2TwIHz3aGVLsgcwf/3IK\nX+0+r11TRETkEBhWqMIGdayLt4e2M6m9sfYINhyu/KPYiYiIKophhSplVI/GmHpvc+O2UsDzKw5g\n559clp+IiKoHw4rGrGm5/Yp6oV9LPB5RtCx/jj4fE5bEcVl+IiKqFgwrGrOm5fYrSkTwzrD2uL9d\nsLGWlp2HMYticP4Wl+UnIiLLYlihKtE5CWY9dgcimhRflj8bkQv3Ij61Yis8ElHZuDwA2ZLq/PPK\nsEJV5u6iw7zIcLQO8THWzt/KwJhFe5HKZfmJzKLT6YwPsiOyBXq93vhUa0tjWCGz+Hm4YMm4CDSo\n5WGsHb2SgolL45Cdxx+0RFXl6emJtLQ0rdsgqrC0tDR4enpWy3szrJDZ6vi6Y+lT3RDoVbQs/84/\nb+HFlQe5LD9RFfn6+iIhIYF3V8gm6PV6JCQkwNfXt/yDq4BhhSyiSZAXFo3tCi/XoluAPxy+irei\nj/Jzd6Iq8PHxgZeXF86fP4+kpCTk5eXx/yWyKkop5OXlISkpCefPn4eXlxd8fHzKP7EKnMs/hKhi\nOjbwx5xRYRi3OAa5+oIfqkt2nUeQtxum9m2hcXdEtkVEUKdOHaSmpiIlJQU3btzgXRayOjqdDp6e\nnggKCoKPjw+k+DLnFsSwQhbVu0VtfDCiM6at2I/CfwR++PNJBHm74YluoWWfTEQmRAS+vr7Vdmud\nyFbwYyCyuKGd6uH/Brc1qb3+/WFsPMJl+YmIqPIYVqhajLmzCSbf08y4na+AqSsOYPeZWxp2RURE\ntohhharNy/1b4dHwhsbtnLx8jP8yFseupGjYFRER2RqGFao2IoL/PNge/doWLcufmp2H0Yv24mJC\nhoadERGRLWFYoWrlrHPCJ4/fga6Naxlr8anZGLVgD26mcVl+IiIqH8MKVTt3Fx3mR3ZFq+Ci+ffn\nbmVg7KIYpGXnadgZERHZAoYVjYnIEBGJSk5O1rqVauXn6YIlT0Wgvn/RsvyHLydjxJxdOH2DS4oT\nEVHpGFY0ppSKVkpN8PPz07qVahfs644lT0WglqeLsXbsagqGfLIdK2MucHVOIiIqEcMK1ahmtb2x\neGwE/DyKAktmrh7/+O4wnlu+H8mZfFozERGZYlihGtepoT9+nNYbEY0DTOo/HLqKv83ahrjzCRp1\nRkRE1ohhhTRRz98Dyyd0xwv3tYRTsUdJXE7KxIi5u/HJ5lN8YjMREQFgWCEN6ZwE0+5rgZUTe5gM\nvNXnK3zw80k8MW83riZnatghERFZA4YV0lzXxgHYMLU3/tYhxKS+52wCBs7ahk1Hr2nUGRERWQOG\nFbIKfp4u+OyJLpgxvAPcXYr+WCZl5GLi0ji8/v1hZOXqNeyQiIi0wrBCVkNE8FhEKNZP6YU2dX1N\n9n21+wKGfrodJ66latQdERFphWGFrE7zOj5Y82xPjOnZ2KR+8noahn66HUt3n+eaLEREDoRhhayS\nu4sObw5thwWjwxHg5WqsZ+fl443vj2Di0jgkpudo2CEREdUUhhWyan3bBOPHab1xZ/NAk/pPx65j\n4Kxt2H3mlkadERFRTWFYIasX7OuOpeO64R8DWsO52KIs11Ky8Pi83fjgpxPI0+dr2CEREVUnhhWy\nCU5OgmfuboZvn+mJ0ABPY10p4JMtpzFi7i5cTMjQsEMiIqouDCtkUzo39McPU3thWOd6JvV9F5Lw\nt9nbEH3wikadERFRdWFYIZvj4+6Cjx+7Ax+O6AQvV52xnpqVhynL9+Pv3x5ERk6ehh0SEZElMayQ\nzRrepQHWT+2NDvX9TOrfxF7C4E+248jlZI06IyIiS2JYIZvWJMgL3z3TExP7NDWpn4lPx/DPd2LB\n9rNck4WIyMYxrJDNc3V2wj//1gZLxkUgyNvNWM/R52P6+mMYtzgGN9OyNeyQiIjMwbBCdqNPy9rY\n+Hxv3N2qtkn91xPxGPDxNmw7Fa9RZ0REZA6GFbIrQd5uWDi6K94Y3BYuuqI1WW6mZWPUgr14b8Mf\nyMnjmixERLaEYYXsjpOT4KleTbDm2TvRNMjLZN/c38/g4Tk7ce5mukbdERFRZTGsaExEhohIVHIy\nZ65YWvv6foie0gsjwhuY1A9dSsag2duwet8ljTojIqLKEM6UsA7h4eEqNjZW6zbsVvTBK3ht9WGk\nZpuuv/LgHfXx9gPt4OPuolFnRESOS0TilFLh5R3HOyvkEIZ0qocN03qjS6i/SX3N/ssYNHs7DlxM\n0qgzIiIqD8MKOYyGAZ74ZmIPTLm3OaRo7C0uJGTg4S924ovf/kR+Pu80EhFZG4YVcijOOie81L8V\nlj3dHSG+7sZ6Xr7C+xuPI3LhXtxIydKwQyIiuh3DCjmkHs0C8eO03ujXNtikvv30TQyYtQ1bjl/X\nqDMiIrodwwo5rFperogaFYbpD7SDm3PR/woJ6TkYtzgWb0UfRXaeXsMOiYgIYFghByciGNWjMdY9\n1wstg71N9i3acQ7DPtuJk9dTNeqOiIgAhhUiAECrEB+se64XRnYPNan/cTUFgz/ZjvnbznDwLRGR\nRhhWiAzcXXR4Z1gHzB0VBj+PonVXcvLy8c4Pf+DJ+XtwOSlTww6JiBwTwwrRbe5vF4KNz/fGnc0D\nTeq7ztzCgI9+x+p9l8DFFImIag7DClEJ6vp5YOm4bvi/IW1NBt+mZufhxW8OYvKyfUhMz9GwQyIi\nx8GwQlQKJyfB2Dub4IepvdC+vq/Jvg2Hr6H/x7/j1xM3NOqOiMhxMKwQlaN5HR+sfuZOTLm3OZyK\nrXwbn5qNsYti8K81h5GRk1f6GxARkVkYVogqwNW5YOXbVZN6onGgp8m+r/dcwKDZ27H/QqJG3RER\n2TeGFaJKCGtUCz9M7Y0nuplOcT57Mx0Pz9mFD386gVx9vkbdERHZJ4YVokrycnPGuw92wMIx4Qjy\ndjPW9fkKs7ecxvDPd+L0jTQNOyQisi8MK0RVdG/rYPz0Qh8MaBdiUj98ORmDZm/D4h1nuZAcEZEF\nMKwQmSHAyxVfjOyCDx7pBG83Z2M9Oy8fb0Yfw+hFe3EtmU9xJiIyB8MKkZlEBA+FNcDG53ujW5MA\nk33bTt1E/4+2Yt3BKxp1R0Rk+xhWiCykQS1PLB/fHf/6Wxu46or+10rJysPU5fsxZfl+JGfkatgh\nEZFtYlghsiAnJ8H4Pk2xbsqdaFPXdCG56INXcP/Hv2PbqXiNuiMisk0MK0TVoHWIL76f3BOT7moG\nKbaQ3LWULIxasBdvrjuKzBy9dg0SEdkQhhULEBFvEXlLRDaISLyIKBF5Veu+SFtuzjq8OrA1Vk7o\ngQa1PEz2Ld55DoM+2YZDl5I06o6IyHYwrFhGEIB/A+gAYL/GvZCViWgSgI3P98Gj4Q1N6mfi0zH8\n852Y9csp5HEhOSKiUjGsWMZVAPWVUg0BTNC6GbI+3m7OeP/hjogaFYZAL1djPS9f4aNfTuLhObtw\nJp4LyRERlYRhxQKUUtlKKc5NpXL1bxeCTS/0wX1tgk3qBy4mYdDs7Vi6+zyU4kJyRETFMawQ1bAg\nbzfMiwzD+w91gJerzljPzNXjje+PYOziGNxI4UJyRESFrDasiIiPiAwVkeki8qOI3DQMXFUi0rqC\n7xEiIrNE5E8RyRKR6yISLSJ9q7t/orKICB7tGoofp/VBeKNaJvt+OxGP/h//jg2Hr2rUHRGRdbHa\nsAKgL4C1AF4HMABAYGVOFpGOAI4AmAqgKYBsFAyEHQzg55Jm60gB9wq+rPn3jmxEaKAnVk7sgb8P\naAUXXdEc56SMXDz79T68uPIAUrK4kBwROTZr/wv3BoANAN5CJQauiogHgHUoCDj7AbRXSvkBqAXg\nAwAC4F0R6X/bqXcCyKzgq0+Vf1VExeicBM/e3RzfT74TLYO9Tfat3n8ZAz76HTv/vKlRd0RE2nMu\n/xDNRCulvi/cEJHGlTh3IoBGANIADFFKXQYApVQKgJdFpBmAYQDeA/BTsfNOAhhbwWscr0Q/ROVq\nV88P657rhf9tOoEFO86icJztleQsPDFvD57q1QSv3N8K7i66st+IiMjOWG1YUUqZs7znk4avywqD\nym3+i4Kw0kVEWimlThiueQPAYjOuS2QWdxcdXh/cFn3bBOPlVQdxOSnTuG/B9rPYdioeH47ojPb1\n/TTskoioZln7x0CVJiI+AMIMm5tKOWw3gGTD9xxsS1anR7NA/Ph8bwzvUt+kfvJ6Gh78fAc++/U0\n9Pmc4kxEjsFq76yYoQ0KxqQAwNGSDlBK5YvICQARANpa4qIi8hwAf8MLAO4RkcLf30+UUskln0lU\nMl93F3w4ojP6tQnGa2sOI9HwxOZcvcJ/N53AluM38OGITmgU6KVxp0RE1cvu7qwAqFvs+7IWaivc\nV7eMYyrjZQDTAbxk2O5v2J6OgoG9RFUysENdbHq+D+5pVdukHnc+EQNnbcOKvRe4kBwR2TV7DCvF\n/5mZWepRQIbhq3cZx1SYUqqxUkpKeZ0r6RwRmSAisSISGx8fb4k2yE7V8XXHwjFd8Z8H28Oj2ADb\njBw9Xl19GOOXxOFmWraGHRIRVR97DCs2QykVpZQKV0qF165du/wTyKGJCJ7s1ggbpvXGHaH+Jvt+\n+eM6Bnz8O7Ycv65Rd0RE1ccew0p6se89yjjO0/CVT48jm9IkyAurJvbAi/1awtmpaCG5m2k5GLc4\nFv9acxgZOXkadkhEZFn2GFaKj1OpV8Zxhfu4pjnZHGedE6b2bYHvnumJpkGmA2y/3nMBg2Zvx4GL\nSRp1R0RkWfYYVo4DKBxt2K6kAwxL5bcybB6riaaIqkOnhv5YP7UXRnYPNamfvZmOh77YiVm/nEKe\nPl+j7oiILMPuwopSKhVArGGzXymHdQNQuKrW5mpvqgwiMkREopKTObOZqsbT1RnvDOuARWO6Isjb\nzVjX5yt89MtJPDJ3F87dTC/jHYiIrJvdhRWDZYavT4pISVOTXzZ8jStcvVYrSqlopdQEPz+uSErm\nuad1HWx6vjf6tQ02qe+/kIS/zeYUZyKyXVYdVkQkqPAF07VK/IvvK+EJyHMBnAfgA2C9iLQ1vJ+P\niMwEMNxw3GvV/WsgqkmB3m6IGhWG9x/qAE9XTnEmIvsg1vwvLRGpaHNNbl/LREQ6oeAjnkBDKQUF\na6o4oWBMy2tKqRkWatVs4eHhKjY2tvwDiSro/K10vLDyAPZdMB1oG+TtipkPd8S9rYNLOZOIqGaI\nSJxSKry846z6zoo5lFIHAbQHMBvAGQBuAG4B+AFAP2sKKkTVoVGgF76Z2AMv9WsJXQlTnF/jFGci\nshFWfWfFkfDOClWngxeT8MLKAzhz20DbJkFe+OjRzujc0L+UM4mIqo/D31khoiKc4kxEtoxhRWOc\nukw1hVMU8JwsAAAgAElEQVScichWMaxojFOXqaZxijMR2RqGFSIHxCnORGRLGFaIHJSI4NGuofhx\nWm904VOciciKMawQObjiU5xLeoozpzgTkdYYVogIzjonTOnbAquf/etTnJfxKc5EpDGGFSIy6tiA\nU5yJyPowrGiMU5fJ2nCKMxFZG4YVjXHqMlmrwinO/UuZ4rycU5yJqIYwrBBRqQK93TB3VBhmPtQR\nXrdNcf4npzgTUQ1hWCGiMokIRnRtiA2c4kxEGmFYIaIK4RRnItIKwwoRVZjJFOfanOJMRDWDYYWI\nKq1jA3/8MKU3RnVvZFLnFGciqg4MK0RUJR6uOkwf1p5TnImo2jGsaIzrrJCt4xRnIqpuwh8i1iE8\nPFzFxsZq3QZRlSmlsCr2Et6KPor0HL3JviGd6mHG8A7wcnPWqDsiskYiEqeUCi/vON5ZISKLKJzi\n/OO0PghrVMtkX/TBK3jgsx04fSNVo+6IyJYxrBCRRYUGemLlhO54uX9L6IpNcT59Iw1DP92B9Yeu\naNgdEdkihhUisjhnnROeu7cFvn66m8ng24wcPZ5bth9vRR9FTh5nCxFRxfADZHv046vAtcNad0GE\n7gB2huTj1PVUpGYXWzAuBjh5yBktg33gquO/mYhsRkgHYOCMGr8sw4o9unYYOL9d6y6IAACuANoB\nf72PmwvgUo23Q0Q2iP+kISIiIqvGOyv2KKSD1h0QlSozV4+T11ORmWs6vbmWpyua1faCsxP/DUVk\ntTT6+4VhRWMiMgTAkObNm1vuTTX4PJGoojwAtMjJw7/WHMGa/ZeLduQAjXSe+PzJLmhXz0+z/ojI\n+vCfMBpTSkUrpSb4+fGHMzkOT1dnfDiiE6YPaw8XXdH05vO3MjD8851YFXtRw+6IyNowrBCRJkQE\no7o3wqpJPVHPz91Yz87LxyvfHsI/Vx9C1m0fFRGRY2JYISJNdW7oj/VTe6N3iyCT+vK9F/HwnJ24\nmJChUWdEZC0YVohIcwFerlg8NgJT+7YwqR+5nILBn2zHr8dvaNQZEVkDhhUisgo6J8GL/Vpi0diu\n8Pd0MdaTM3MxdnEMPvzpBPT5fPAqkSNiWCEiq3JPqzqIfq4XOtQ3HXQ+e8tpjFm0FwnpORp1RkRa\nYVghIqvTMMATqyb1wBPdQk3q207dxODZ27D/QqJGnRGRFhhWiMgqubvo8O6DHfC/RzrBzbnoR9WV\n5CyMmLsLS3adg1L8WIjIETCsEJFVezisAdY8eycaBXoaa7l6hX+vPYrnVx5ARk5eGWcTkT1gWCEi\nq9e2ni/WPdcL/doGm9TXHriCYZ/twJ/xaRp1RkQ1gWGFiGyCn4cLokaF4dWBreFUtOgtTl5Pw9BP\ntmPD4avaNUdE1YphRWMiMkREopKTk7VuhcjqiQgm3dUMXz/dHUHebsZ6eo4ez369D++sP4Zcfb6G\nHRJRdWBY0RifDURUeT2aBeKHqb3QtXEtk/r87WfxxLzduJ6SpVFnRFQdGFaIyCYF+7pj2fjueLpX\nE5N6zLlEDJq9HbvP3NKoMyKyNIYVIrJZLjonvD64LT5/sgu8XHXG+s20bDw5fw/mbv2T05uJ7ADD\nChHZvL91qIt1U3qhRR1vY02fr/Dej8cxcWkcUrJyNeyOiMzFsEJEdqFZbW98P/lODO1Uz6T+07Hr\nGPrJdvxxNUWjzojIXAwrRGQ3vNycMeuxznj7gXZw0RXNbz53KwMPfr4D38Vd0rA7IqoqhhUisisi\ngsgejbFyYg/U9XM31rNy8/HSqoN4bc1hZOXqNeyQiCqLYYWI7FKX0FpYP6UXercIMqkv23MBI+bu\nwsWEDI06I6LKYlghIrsV6O2GxWMjMPXe5ib1Q5eSMeTT7fjtxA2NOiOiymBYISK7pnMSvNi/FRaO\nCYefh4uxnpSRi7GLY/DRzyehz+f0ZiJrxrBCRA7h3tbBWD+lF9rX9zXWlAJmbT6Fp76MQXo2n95M\nZK0YVojIYTQM8MS3k3ri8YiGJvXfTsTjyfl7kJSRo1FnRFQWhhUicijuLjq8N7wjZj7cEW7ORT8C\nD1xMwoi5u3Atmc8VIrI2DCtE5JBGhDfEigndTcaxnLyehofn7MS5m+kadkZEt2NY0ZiIDBGRqOTk\nZK1bIXI4d4TWwqpJPRDs62asXUrMxMNzduHYFa54S2QtGFY0ppSKVkpN8PPz07oVIofUMtgH307q\nicaBnsbazbRsPBq1CzHnEjTsjIgKMawQkcNrGOCJVZN6ok3doplCqVl5GLVgD349zrVYiLTGsEJE\nBKC2jxtWTOiO8Ea1jLWs3HyMXxKLtQcua9gZETGsEBEZ+Hm4YOlT3XBPq9rGWl6+wvMrD2DJrnOa\n9UXk6BhWiIiK8XDVISoyHA90rmesKQX8e+1RzPrlFJTiardENY1hhYjoNi46J3w0ojMiezQyqX/0\ny0m8FX0M+Vyen6hGMawQEZXAyUnw1tB2mNq3hUl98c5zeHnVQeTq8zXqjMjxVDmsiMhUw6te+UcT\nEdkeEcGL/Vri34PbmtRX77+MZ76KQ1auXqPOiByLOXdWPgLwPwA3LdQLEZFVGterCT4c0Qk6JzHW\nfvnjBiIX7kVKVq6GnRE5BnPCyk0AqUopPvmLiOze8C4NMHdkGFyLPU9o79kEPB61GzfTsjXsjMj+\nmRNW9gHwE5Ha5R5JRGQH7msbjCXjIuDj5mysHb2SghFzduFSYoaGnRHZN3PCymzD+W9YqBciIqvX\nvWkglk/ojkAvV2PtzM10PDJnF07fSNWwMyL7VeWwopT6EcDLACaJyFIR6WS5toiIrFf7+n5YNakH\n6vt7GGtXk7PwyJxdOHgxScPOiOyTVHWBIxE5Y/g2BEDhI0szAdwCUNoQeaWUalalC9q58PBwFRsb\nq3UbRFQJV5MzMXL+HvwZn26sebnqMC8yHD2bB2nYGZFtEJE4pVR4eceZ8zFQY8PLHYAYXp4AGhbb\nV9KLiMgu1PXzwKpJPdGxQdFT09Nz9BizKAYbj1zTsDMi+2LOnZW7qnKeUmprlS5o53hnhch2pWXn\nYcKSWOz885ax5iTAjOEdMaJrQw07I7JuFb2z4lzeAaVh6CAiKuDt5oyFY7pi2or92HT0OgAgXwF/\n/+4QkjNzMb5PU407JLJtXG6fiMgC3F10+OyJLhgR3sCk/p8Nf2DmxuN8ACKRGap8Z+V2IiIAWgEo\nXHclHsAJxf9DichBOOuc8P5DHeHv6Yqo388Y65//9ieSMnMx/YH2JqvgElHFmH1nRUSai8hiAMkA\njgL4zfA6CiBZRBaJSHNzr0NEZAtEBP8c2Bp/H9DKpL5szwVMXbEfOXl8ACJRZZkVVkRkKID9AEYB\n8EbRrKDClzeASAD7RWSwea3aJxEZIiJRycnJWrdCRBYiInj27uZ498EOkGI3Un44dBVPfRmDjJw8\n7ZojskHmPHW5GYAVALwAnAEwEUALAB6GVwsAkwD8aTjmG8M5VIxSKlopNcHPz6/8g4nIpjzRLRSf\nPH4HXHRFiWXbqZt4cv4eJGXwsWpEFWXOnZW/o2CNlV8BdFRKzVNK/amUyja8/lRKRQHoBGArChaO\ne8X8lomIbMfgjvUwf3RXeLjojLX9F5Lw6NzduJ6SpWFnRLbDnLDSD4ACMFEplVnaQYZ9E1HwsVB/\nM65HRGST7mpZG1893Q1+Hi7G2onrqXh4zk6cv5VexplEBJgXVuoCSFZKnS7vQKXUSQBJhnOIiBxO\nWKNa+GZiD9TxcTPWLiZk4uE5u/DH1RQNOyOyfuaElQwAniLiUt6BIuKKgnErpd6BISKyd61CfPDt\npJ5oFOhprMWnZmPE3F2IPZegYWdE1s2csHIYgAuA0RU4drTh2ENmXI+IyOaFBnpi1aQeaB3iY6yl\nZuVh5II9+PXEDQ07I7Je5oSVpSgYhzJbRJ42LApnQkTcRWQqgNkoGN/ypRnXIyKyC3V83LFyQg+E\nN6plrGXl5mP8l7FYe+Cyhp0RWSdzHmQoADaiaKDtNQDbAFxGwSyhUADdAASiINT8BGAgV7QtGR9k\nSOR4MnP0eObrOPx2It5YEwHefqA9RnVvpGFnRDWjog8yrPKdFUPoGAYgCgVhpS6AEQCeR8H6KoMA\nBBn2zQHwIIMKEVERD1cdokaFY0inesaaUsAb3x/BJ5tP8XlCRAZmPRvIMC15koi8B+BBAF1g+myg\nfQBWK6UumNUlEZGdcnV2wsePdoafhzO+2l30o/KDn08iMSMXrw9qAyc+T4gcnEUeZKiUOg/gY0u8\nFxGRo9E5CaY/0B61PF3xyZai1SAW7jiL5MxcvP9QBzjrzH6UG5HNMme5/X0iEiciTS3ZEBGRIxIR\nvNS/Fd4Y3Nak/t2+S5j01T5k5eo16oxIe+ZE9bYAWiilzpR7JBERVchTvZrgg0c6QVfso59f/riO\nSV/FIVfPJzaTYzInrFxGwSwfIiKyoIfCGmDOyDC4Ohf9iP7tRDz+/u0h5Odz0C05HnPCyiYUrGDb\nzVLNEBFRgX5tg/Hl2Ai4uxT9mF6z/zLe+eEPzhIih2NOWHkHwC0Ac0QkyEL9EBGRQY9mgfj8yS4m\nHwkt3HEWX2z9U8OuiGqeObOBmgP4F4APAJwQkSUAdqFgynKpI8GUUr+bcU0iIodyb+tgzHyoI15a\nddBYm7nxBAI8XfFYRKiGnRHVHHPCym8oWPANKBi7MtXwKosy85pERA7nobAGSMzIwTs//GGsvbbm\nMGp5ueL+diEadkZUM8wJDhdQFFaIiKgaPd27KW6m5WCO4SOgfAVMWb4fS8ZFoHvTQI27I6peVQ4r\nSqnGFuyDiIjK8Y8BrZCQno1vYi8BAHLyCh5+uGJid7Sr56dxd0TVh0siEhHZCBHBuw92wH1tgo21\n1Ow8jF4Ygwu3MjTsjKh6mbOCbaKI3OIKtkRENcdZ54RPn7gDEY0DjLWbadkYtXAPbqRmadgZUfUx\n586KKwAdV7AlIqpZ7i46zBsdjtYhPsba+VsZGLMwBilZuRp2RlQ9zAkrF1AQWByeiHQVkU9F5KiI\npIvIBRH5RkRaat0bEdknPw8XLBkXgYYBHsbasaspGP9lLJ8jRHbHnLCyDoCbiPSzVDM27B8AHgKw\nGcA0AFEA+gDYJyIdtGyMiOxXHV93LB3XDUHeRf9u3HM2AVOX70cenyNEdsScsPIugHMA5olIG8u0\nY7M+BNBIKTVVKTVfKfUOgN4omG31mratEZE9axzkhcVjI+DtVjS586dj1/H690e4LD/ZDXPWWXkA\nwBcA/g1gv4j8iIqtYLvEjGtaJaXUzhJqp0TkKAqeTk1EVG3a1/fDvMhwjF64FzmGOyorYi4i0NsV\nr9zfWuPuiMwnVU3eIpKPgkXhCh9aUaE3UkrpqnRBGyMiAuAigBNKqb7lHR8eHq5iY2OrvzEislsb\nj1zFs1/vQ/EHM78xuC2e6tVEu6aIyiAicUqp8PKOM+fOyu+oxhVsRcQHwD0AugIIN3wtXKaxjVLq\neAXeIwTAPwEMBlAfQDKAvQA+Vkptro6+i3nScM23qvk6REQAgAHt6+KdYR3w2prDxtr09ccQ6OWK\nYXfU17AzIvOYs4Lt3RbsoyR9Aayp6ski0hHAFhQFnBQAQSgILoNE5DWl1IzbzhEAbhW8RI5SqsQR\nbCLSGsBnAHYDWFiF9omIquSJbqFISM/G/346aay9vOog/DxdcE+rOhp2RlR11r6C7Q0AG1Bwd2JC\nRU8SEQ8UzFYKBLAfQHullB+AWih4SrQAeFdE+t926p0AMiv46lPKtUMA/ICCuzgPKaU4h5CIatTk\ne5pjTM/Gxu28fIVnvopD3PlE7ZoiMkOF76yIyFQA6UqpBSXs8wbgpJRKKeP8jwD4KqWequAlo5VS\n3xc7v3FFewUwEUAjAGkAhiilLgOAob+XRaQZgGEA3gPwU7HzTgIYW8Fr/OVjKBHxA/AjAH8AvZVS\nVyrRMxGRRYgI/j24LRLSc7DuYMGPoazcfIxbHINVk3qgZbBPOe9AZF0qPMDWMKD2qlLqLx98ishV\nALWVUqWGH8Mxdao6wNYQVs4aNsscsyIiMSgY5xKllJpYwv6eAHYYNlsrpU5Upafb3tMdBcEnDMB9\nSqldlTmfA2yJyNJy8vLx9JJY/H4y3lgL8XXHd8/2RH1/jzLOJKoZFR1gW9mPgaSK+2qMYWBumGFz\nUymH7UbBxzRAwdgYc6+pA7ASQA8Aj1Q2qBARVQdXZyfMGdkFnRv6G2vXUrIwasEeJKTnaNgZUeVY\n+5iVqmiDouB0tKQDDANjC++mWGIdlA8ADEXBR0ABIjKy+MsC709EVCWers5YNKYrmtfxNtbOxKdj\n7KK9SM/O07Azooqzx7BSt9j3ZY0ZKdxXt4xjKqqz4esQAEtLeJVIRCaISKyIxMbHx5d2GBGRWWp5\nuWLJuAjU9XM31g5eSsakr+KQk8dl+cn62WNY8Sr2fWYZx2UYvnqXcUyFKKXuVkpJaa8yzotSSoUr\npcJr165tbhtERKWq5++BpU9FwN/TxVjbduomXlp1EPn5XJafrJs9hhUiIipB8zo+WDSmKzxciuY5\nRB+8greij/I5QmTV7DGspBf7vqzh7p6Gr2nV2AsRkVW5I7QW5owKg7NT0U3fL3edx6dbTmvYFVHZ\n7DGsFB+nUq+M4wr3Xa3GXoiIrM5dLWvjgxGdTGof/HwSX+85r1FHRGWr7HL7ASKypaQ6AJSyz+SY\nGnAcRQ9YbIeiWT9GIuIEoJVh81gN9UVEZDUe6FwfCek5eCu66Efg698fQS1PV/ytgyXmHRBZTmXD\niiuAu8vYX9Y+oBoffGi8gFKpIhKLggcf9gOwuoTDugHwM3xf3Q80LJOIDAEwpHnz5lq2QUQOaOyd\nTZCQnoNPDB8BKQU8v+IA/D1c0LN5kMbdERWpTFj5stq6sLxlKAgrT4rI20qp2z/qednwNc4Sq9ea\nQykVDSA6PDx8vJZ9EJFjerFfS9xMy8HyvRcAADn6fIxfEosVE3qgQwO/cs4mqhkVDitKqYo+M8di\nRKR4tK9V7Hv/2/Yl3PYE5LkAnkfB84HWi8gopdQxw+q2bwAYbjjuterom4jIVogI3hnWHonpOdh4\n9BoAID1HjzGL9uLbZ3qiSZBXOe9AVP2sfYBtfLHXvmL1XbftCy1+klIqE8ADAG4B6ALgqIgkA0gC\n8AoKPo76p1Kq+EMMiYgcks5J8PFjndG9adHQwlvpORi1YA+up2Rp2BlRAWsPK1WmlDoIoD2A2QDO\nAHBDQXj5AUA/pdQMDdsjIrIq7i46zIsMR7t6vsbapcRMRC7Yi+SMXA07I7LysFLWqrC3vc6Vcv41\npdQ0pVQzpZS7UqqOUmqwUkrTQbVERNbIx90Fi8dGoFGgp7F24noqnvoyBpk5eg07I0dn1WHFEYjI\nEBGJSk5OLv9gIqJqVtvHDUvHdUNtHzdjLfZ8Ip5btg+5ej5HiLTBsKIxpVS0UmqCnx9H3RORdQgN\n9MSScRHwcS+ag7H5+A38c/VhLstPmmBYISKiv2hT1xcLRneFm3PRXxPfxl3CjI3HNeyKHBXDChER\nlSiiSQA+faILdMWeIzR36xlE/f6nhl2RI2JYISKiUvVrG4z3hncwqb274Ti+jbukUUfkiBhWiIio\nTCPCG+LVga1Nav/47hA2/3Fdo47I0TCsEBFRuSb2aYrxvZsYt/X5Cs9+vQ8x5xI07IocBcOKxjh1\nmYhsgYjgnwPbYHiX+sZadl4+nlocg+PXUjTsjBwBw4rGOHWZiGyFk5Pg/Yc64t7WdYy1lKw8RC7Y\ni4sJGRp2RvaOYYWIiCrMReeEz57ogrBGRc+WvZGajciFe5GQnqNhZ2TPGFaIiKhSPFx1WDi6K1oF\n+xhrZ2+m45mv4pCTx1VuyfIYVoiIqNL8PF3w5bgI1Pf3MNb2nE3A/607wlVuyeIYVoiIqEpC/Nyx\ncExXeLnqjLXley9i8c5z2jVFdolhhYiIqqxViA9mP34HpGiRW0xffwxbT8Zr1xTZHYYVIiIyS982\nwXh1QNGicfkKeG7ZPpy+kaZhV2RPGFY0xnVWiMgeTOjT1GQNltSsPDz9ZQySMjhDiMzHsKIxrrNC\nRPZARPDugx3QJdTfWDt3KwOTl+1Drp4zhMg8DCtERGQR7i46zB0Vjnp+7sbajtO38Hb0MQ27InvA\nsEJERBZT28cN80aHw8OlaIbQ0t3nsXTXOc16ItvHsEJERBbVrp4fPnq0s0ntzehj2HH6pkYdka1j\nWCEiIosb0D4EL/dvadwufErz2ZvpGnZFtophhYiIqsXke5pjaKd6xu3kzFw89WUMkjNzNeyKbBHD\nChERVQsRwcyHO6JTg6LZjmfi0zFl+X7kcYYQVQLDChERVRt3Fx3mRYYjxLdohtDvJ+Pxnw1/aNgV\n2RqGFY1xUTgisnd1fN0xLzIc7i5Ff+Us2nEOy/de0LArsiUMKxrjonBE5Ag6NPDDB4+YzhB64/sj\n2H3mlkYdkS1hWCEiohoxqGNdTOvbwridl6/wzFdxuHArQ8OuyBYwrBARUY2Z1rcFBnWoa9xOzMjF\n00tikJrFGUJUOoYVIiKqMU5Ogv890gnt6/saayevp2HaigPQ5ysNOyNrxrBCREQ1ysO1YIZQbR83\nY23L8RuYufG4hl2RNWNYISKiGlfXzwNRo8Lg6lz019Dc389gVexFDbsia8WwQkREmrgjtBb++3BH\nk9q/1hxB7LkEjToia8WwQkREmnmgc31MvqeZcTtHn4+JS+NwKZEzhKgIwwoREWnqpX6t0L9tsHH7\nVnoOnv4yFunZeRp2RdaEYYWIiDTl5CT46NHOaFO3aIbQ8WupeGHlAeRzhhCBYYWIiKyAl5sz5kWG\nIcjb1Vj76dh1fPDzCQ27ImvBsKIxPhuIiKhAg1qemDsqDK66or+aPvv1T3y//7KGXZE1YFjRGJ8N\nRERUJKxRAN4d3sGk9vfvDmH/hUSNOiJrwLBCRERW5eGwBpjYp6lxOycvHxOWxuFKUqaGXZGWGFaI\niMjq/H1Aa/RtXce4HZ+ajfFLYpGRwxlCjohhhYiIrI7OSfDxY53RMtjbWDt6JQUvrzrIGUIOiGGF\niIisko+7C+ZHdkUtTxdjbcPha5i1+ZSGXZEWGFaIiMhqhQZ6Ys7IMDg7ibE2a/MprD90RcOuqKYx\nrBARkVXr1jQQ7wxrb1J76ZuDOHQpSaOOqKYxrBARkdV7LCIU4+5sYtzOzsvH+CWxuJ6SpWFXVFMY\nVoiIyCa89rfW6NOytnH7eko2JiyJRVauXsOuqCYwrBARkU1w1jnh0yfuQLPaXsbawUvJeOXbQ1CK\nM4TsGcMKERHZDF93F8wf3RV+HkUzhKIPXsFnv57WsCuqbgwrRERkU5oEeeGLJ7tAV2yG0P9+OomN\nR65q2BVVJ4YVIiKyOT2bB+HNoe1Mai+sPIijV/hQWHvEsEJERDZpVPdGiOzRyLidmavH+C9jcSOV\nM4TsDcOKxkRkiIhEJSfzXwNERJX1xuC2uLN5oHH7SnIWJi6N4wwhO8OwojGlVLRSaoKfn5/WrRAR\n2RwXnRM+fyIMTYKKZgjtv5CE11Yf5gwhO8KwQkRENs3P0wXzIsPh4+5srK3efxlzfz+jYVdkSQwr\nRERk85rX8cZnT3RBsQlCeH/jcfx87Lp2TZHFMKwQEZFd6NOyNt4Y3Na4rRTw/Ir9OH4tRcOuyBIY\nVoiIyG6M6dkYj0eEGrfTc/R4+stY3ErL1rArMhfDChER2Q0RwVtD26FbkwBj7VJiJiZ9FYecvHwN\nOyNzMKwQEZFdcXV2whcjwxAa4GmsxZxLxBvfH+EMIRvFsEJERHYnwMsV80eHw9utaIbQytiL+GrP\nBQ27oqpiWCEiIrvUMtgHsx/vDCk2Q+itdUcRcy5Bu6aoShhWiIjIbt3bOhgv929l3M7LV3jmq324\nlswl+W0JwwoREdm1Z+9uhoHtQ4zbN9OyMemrOGTncUl+W8GwQkREdk1E8N9HOqFFHW9j7cDFJPzf\n2qMccGsjGFaIiMjuebs5I+q2JflXxFzEsr0ccGsLGFaIiMghNAnywqzHTAfcvrnuKOLOc8CttWNY\nISIih3Fv62C8eF9L43auXmHSV/twPYUDbq0ZwwoRETmUyfc0x/3tgo3b8anZeIYDbq0awwoRETkU\nJyfBByM6o3mxAbf7LiThzXXHNOyKysKwQkREDsfbzRlRo8LgU2yF2+V7L2AZV7i1SgwrRETkkJrW\n9sbHj3U2qf3fuiOIO5+oUUdUGoYVIiJyWH3bBOOF2wbcPvNVHG5wwK1VYVjRmIgMEZGo5ORkrVsh\nInJIU+5tjn5tiwbc3kjNxjNf70NOXr6GXVFxDCsaU0pFK6Um+Pn5ad0KEZFDcnISfDiiE5rV9jLW\n4s4n4q3ooxp2RcUxrBARkcPzcXdBVGQ4vIsNuP16zwWs4Aq3VoFhhYiICECz2t746FHTAbf/XnsU\n+y9wwK3WGFaIiIgM+rUNxrS+LYzbOfp8TPoqDjdSOeBWSwwrRERExUzr2wL3talj3L6eko3JHHCr\nKYYVIiKiYpycBB8+2hlNg4oG3MacS8T09VzhVisMK0RERLfxdXdBVGSYyYDbpbvP45uYixp25bgY\nVoiIiErQvI4PPhjRyaT2+vdHcOBikkYdOS6GFSIiolLc3y4EU+9tbtzO0edj0tI4xKdma9iV42FY\nISIiKsPz97VE39ZFA26vpWRh8tf7kKvngNuawrBCRERUhsIBt02KDbjdey4B73DAbY1hWCEiIiqH\nn4cLokaFwctVZ6x9ues8VsVywG1NYFghIiKqgBbBfx1w+6/vj+DQJQ64rW4MK0RERBU0oH1dPHdP\nsQG3efmYuDQON9M44LY6MawQERFVwgv9WuLuVrWN21eTOeC2ujGsEBERVYLOSTDrsTvQONDTWNtz\nNgH/+eEPDbuybwwrREREleTn4YK5o8LhWWzA7eKd5/Bd3CUNu7JfDCtERERV0CrEBx88Yjrg9rU1\nh84T8zsAABoNSURBVHH4UrJGHdkvhhUiIqIqGtihLp69u5lxOzsvH5O+isMtDri1KIYVIiIiM7zU\nvxXualk04PZyUiYmL9uHPA64tRiGFSIiIjPonASzH7sDoQFFA253n0nAuxuOa9iVfWFYISIiMpOf\npwuiIsPg4VI04HbhjrNYs58Dbi2BYYWIiMgCWof44r+PdDSpvfrdYRy5zAG35mJYsQAR6SIiq0Xk\nnIhkiki8iGwVkcFa90ZERDVncMd6mHSX6YDbiUvjkJCeo2FXto9hxTKaAnADsBDAFADvABAA0SIy\nScvGiIioZr1yfyv0bhFk3L6clInnOODWLKKU0roHuyQiOgBxALyUUi3KOz48PFzFxsZWf2NERFTt\nkjJyMOTT7biYkGmsje/dBP8a1FbDrqyPiMQppcLLO453VqqJUkoP4BIAf617ISKimuXv6YqoUeEm\nA27nbTuLtQcua9iV7WJYsSAR8RaRIBFpLiIvARgA4Get+yIioprXpq4v3n/YdMDtP747hKNXOOC2\nsqw2rIiIj4gMFZHpIvKjiNwUEWV4ta7ge4SIyCwR+VNEskTkuohEi0jfamp7DoB4AKcAzASwBsDk\naroWERFZuaGd6mFin6bG7azcggG3iRxwWylWG1YA9AWwFsDrKLhDEViZk0WkI4AjAKaiYABsNoAg\nAIMB/Cwir5ZwjoiIewVfJf3evQegH4DRAH4C4Pz/7d15nFx1me/xzzedfesOJJAQCJAgARKCSRAQ\nkcUZFsEADuIAES84Q+AK1+QOCCIO19FBVIZhVQiKIpvRUVQWHZ0I4sLikBAYExJASNiTkJCFpLNA\nnvvHOdVVaaor3enlnKr+vl+vep36nfM7VU9Ouuo8dc5vIWl4a2Zm3dTnjx3LYXsVG9y+8lYjF/zQ\nDW7bIs/JCsAy4JfAvwDTWruTpH7AvSQJzpPA+IioB4YAV5P01PmapGOa7fohoLGVj8Obv29EzI+I\n2RFxO3A8MBC4V5JaG7uZmdWWnnU9uOH0iew6pF/Tuj89v4Krfr0ow6iqS8+sA6jgvoj4eaEgaY82\n7HsusDvwNjAlIl4FiIg1wEWSxgAnk1wJ+U3Jfs8CZ7fyPSqOoxwRIek/gJnA3oD/Ks3MuqkhA3oz\n88zJnHLTI2zYnFxRmfn7Fxg3sp4TD9gl4+jyL7fJStqbZntNTZd3FxKVZq4iSVYmSRobEYvS91wG\n3NaO922ukEbXd+BrmplZFRq3Sz3fOGUC02fNa1p38U+eYq9hA9lvl8EZRpZ/eb8N1GaSBgGT0+Kv\nW6j2GFBojt3uxraSdiqzrjdJ25VGYEF738PMzKrfSe8fyTkf3rOpvGHzFs698wk3uN2GmktWgH1J\n2qQAzC9XISK2ULwt0xEj9MxKeyz9s6R/lHQ5SePeicCXIuLtDngPMzOrAZcctw+Hjin2GXl5ZSOf\nm/Uk727xIK0tqcVkZUTJ89cq1CtsG1GhTmvdAfQFLgBuAqYDi4GTIuLfO+D1zcysRvSs68GNZ0xi\nZEOxwe0fnnvTDW4rqMVkZUDJ88YWa8H6dDmwvW8YEd+PiKMiYueI6BURO0bEMRFxb6X9JE2T9ISk\nJ5YvX97eMMzMrErskDa47dOzeBq++eG/cv/TlX5jd1+1mKxUjYi4JSIOjIgDhw0blnU4ZmbWhcaP\nTBrclrr4J0/z7NK1GUWUX7WYrKwred6vxVrQP126PYmZmWXi5Ikj+cyHig1u1296l/PumMOaDZsz\njCp/ajFZKb2GVqnzemHb650Yi5mZWUWXHr8PB+25Q1P5hTfXceGPn2KLG9w2qcVkZSFQ+B8eV65C\nOlT+2LTobsVmZpaZXnU9+NYZk9h5cHF2lv9asJSbHv5rhlHlS80lKxGxFngiLR7dQrWDKQ7U9ttO\nD6oCSVMk3bJ6tWfhNDPrroYN6sO3p06mV11xdpZ/+80ifv+sO19ADSYrqbvT5VRJ5bomX5Qu5xRG\nr81KRNwXEdPq6z3IrZlZdzZ59yFcPqV4QyACPjfrSV5eub7CXt1DrpMVSUMLD5JJCAsaSreVmQF5\nJrAEGATcL2m/9PUGSfom8HdpvS929r/BzMystT518ChOmbRrU3nV+s2cd+ccNmxuzww01S/XyQqw\nvOQxt2T9o822jSrdKSIagZOAFcAkYL6k1cAq4PMkbVoujYjSSQzNzMwyJYkrPj6ecSVzBc1/bQ2X\n/ewvRHTfBrd5T1a2W0Q8BYwHrgdeAPqQJC8PAEdHxNczDM/MzKysvr3quPlTk2no36tp3U/nvsKd\nj7+UYVTZynWyEhFq5WNxC/u/ERHTI2JMRPSNiJ0i4mMRkWmjWjMzs0p226E/1582ERXb2/KV++Yz\nZ8lb2QWVoVwnK92BewOZmVk5h+89jIuOGdtU3vxu8Nm75rB87cYMo8qGk5WMuTeQmZm15H8fMYaj\n99u5qbx0zUbOv3sum9/dkmFUXc/JipmZWU716CGu/uQBjB5anKP3zy+u5Ou/WphhVF3PyYqZmVmO\nDe7bi5vPnEz/3nVN627944vc+1T3maHZyYqZmVnO7b3zIK76xAFbrbvkJ0+z8I01GUXUtZysmJmZ\nVYETJoxg2uGjm8qNm5MZmlc31v4MzU5WzMzMqsTFx47lg6N3bCovXrGeC388r+ZnaHaykjF3XTYz\ns9bqWdeDG86YyIj6vk3rZj+zjBsfej7DqDqfk5WMueuymZm1xdCBffj21En0riuewq+Z/SwPLVqW\nYVSdy8mKmZlZlZk4aghfPnHrGZpnzJrHSytqc4ZmJytmZmZV6PSDduOTBxZnaF7duJlz75xD46ba\nm6HZyYqZmVkVksRXThrPhF2LzQieeX0Nl/3sf2puhmYnK2ZmZlWqb686vj11EkNKZmi+58lXueOx\nJRlG1fGcrJiZmVWxXYf054bTJ9FjqxmaF/DE4pXZBdXBnKyYmZlVucPeN5TPH7tPU/mdLcFn75rL\nsrUbMoyq4zhZyZjHWTEzs45w3hGjOW7c8KbysrUbOf+u2pih2clKxjzOipmZdQRJXHXqBMYMK87Q\n/N+L3+KKB57JMKqO4WTFzMysRgzq24uZZ05mQMkMzbc9sphfzHs1w6jaz8mKmZlZDdlrp0H826nN\nZmj+6dM883r1ztDsZMXMzKzGfHT/EZx3xJim8obNWzj3jjmsXl+dMzQ7WTEzM6tBFx2zNx/aqzhD\n80sr1zPjR09W5QzNTlbMzMxqUM+6Hlx/2kR2KZmh+aFFy7n+wecyjGr7OFkxMzOrUTsO7MPNZ06m\nd8/i6f7a2c/x4MKlGUbVdk5WzMzMatiEXRv46knjtlo3Y9Y8lqxYl1FEbedkxczMrMb9/QdGcfpB\nuzWV12x4h3PvmMP6Te9kGFXrOVnJmEewNTOzrvDlE8dxwG4NTeWFb6zl0nuqY4ZmJysZ8wi2ZmbW\nFfr0rOOmqZPYcUDvpnW/mPcatz2yOLugWsnJipmZWTexS0M/bjh94lYzNF/xwDP8+cV8z9DsZMXM\nzKwbOXSvoXzho++doXnpmvzO0OxkxczMrJs558OjOWH/EU3lN9/eyGfvmsumd/I5Q7OTFTMzs25G\nEt/4xAT22mlg07o5S97iigcWZBhVy5ysmJmZdUMD+/Rk5pmTGdinZ9O6Hzy6hHvmvpJhVOU5WTEz\nM+umxgwbyNWf3HqG5kvv+R/mv5av4TScrJiZmXVjx44bzvlHFWdo3vjOFs67cw6r1m/KMKqtOVkx\nMzPr5v7p6LF8+H1Dm8ovr2xk+qx5vJuTGZqdrJiZmXVzdT3E9adNZGRDv6Z1Dz+7nOtmP5thVEVO\nVjLm4fbNzCwPhgzozcwzJ9OnZIbm6x98ntkLsp+h2clKxjzcvpmZ5cX4kfX868njt1r3f380jxff\nzHaGZicrZmZm1uTUA3dj6sGjmsprN77DeRnP0OxkxczMzLZy+ZT9mDiqOEPzoqVrueSn2c3Q7GTF\nzMzMtpLM0DyZoQOLMzTf99Rr3PrHFzOJx8mKmZmZvcfw+r7ceMYk6kqmaL7yVwt57IUVXR6LkxUz\nMzMr65DRO3JpyQzN726JTHoHOVkxMzOzFv3DYXvysQkj6FUn/vXk8Vx2wr5dHkPPbVcxMzOz7koS\n3/zEBKYdPpoJuzZse4dO4CsrZmZmVlH/3j0zS1TAyYqZmZnlnJMVMzMzyzUnK2ZmZpZrTlbMzMws\n15ysmJmZWa45WTEzM7Ncc7KSMUlTJN2yevXqrEMxMzPLJScrGYuI+yJiWn19fdahmJmZ5ZKTFTMz\nM8s1JytmZmaWa05WzMzMLNecrJiZmVmuOVkxMzOzXHOyYmZmZrnmZMXMzMxyzcmKmZmZ5ZoiIusY\nDJC0HFjSwS87FHizg1+zu/ExbD8fw/bzMWw/H8P264xjuHtEDNtWJScrNUzSExFxYNZxVDMfw/bz\nMWw/H8P28zFsvyyPoW8DmZmZWa45WTEzM7Ncc7JS227JOoAa4GPYfj6G7edj2H4+hu2X2TF0mxUz\nMzPLNV9ZMTMzs1xzsmJmZma55mTFzMzMcs3JSo2RNFzSdZL+KmmDpKWS7pP0N1nHlneSRkmakR6v\nlyRtlLRW0lOSvi5pRNYxVhtJAyW9LCnSx1lZx1RNJI2VdIOkRZLWSVot6RlJ35N0RNbx5ZmkHpLO\nljRb0nJJmyWtkvS4pMskDco6xqxJGiTpRElflfQrSW+WfFb3acX+vSVdLGmepLfT4/uopGmS1KGx\nuoFt7ZA0AXgQ2DFdtQYYSJKUBvDFiPh6RuHlmqTdSEYQLv2ArQEGAHVp+S3glIh4qIvDq1qSrgWm\nl6w6OyJuyyicqiLpc8BVQO901dtAT6BvWr41Iv4xi9jyTlJ/4D7gIyWrVwODKX7GlwAfiYgXuji8\n3JB0MvCzFjbvGxELK+w7mOR8MzldtZ7k77Pw93o/8PGIeKcjYvWVlRohqR9wL0mi8iQwPiLqgSHA\n1SQf0K9JOia7KHOtkJA8AJwK7JAev/7A8cCLJMfy55KGZxNidZE0CbgAeDzrWKqNpHOB60i+/L9B\nMiT5oIjoB4wAPg08kmGIeffPJIlKAJcCDRHRQJLonQ6sAnYHvptZhPmxDPgl8C/AtDbs9x2SRGUl\nMIXkh3F/4CxgA/Cx9DU7hK+s1AhJM4BrSH597RMRrzbb/jPgZGBuREwu8xLdmqR6YI+IeKqF7fuQ\nJIF9gS9HRId9CGuRpB4kScpE4APA3HSTr6xsg6Q9gPkkX/zTIuI7mQZUhSQtAUYB34uIfyiz/Szg\n+2lxh4h4qwvDyw1JdRHxbkl5D5IfZlDhyoqkiRQ/0ydFxL3Ntk8HrgUaSb5Xl7U3Vl9ZqR1T0+Xd\nzROV1FXpcpKksV0UU9WIiNUtJSrp9oXAY2nRyd62/R/gQOCmiHgy62CqzHSSROVxJyrbbed02dLf\n3pyS5/07OZbcKk1U2uiMdLmoeaKSuoXktls/4O+28z224mSlBqQNxQon0F+3UO0xkj8eADe23T4r\n0mVdxVrdnKSRwFeBpcCXMg6nGhVOBD/MNIrqtjhdTmxhe+H7cmkLP+6ssqPS5W/KbYyIRuAPafEj\n5eq0lZOV2rAvxUZj88tViIgtwKK0uF9XBFVLJPUEPpQW/5JlLFXgBmAQcFFErN5WZSuSNAbYKS0+\nKemQtHfaCkmNkhZKukrSTpVexyhckTpb0hfS27yF3it/T3LLPICLsgqwWqW9fAo9hcqeb1IL0mWH\nnG+crNSG0i61r1WoV9jmLrhtdz4wHNgC/CDjWHJL0hTg48DvIuLOrOOpQu8reX4k8EeShoq9SE6u\nY0lOsPMkjevy6KrHtcC3SH7EXQmskrSKpA3FLGAhcKL/RrfLYJJektCF5xsnK7VhQMnzxgr11qfL\ngZ0YS81Ju4RfmRZvjIgFlep3V5IGADcCm0mSO2u7hpLn/w94FjgkIgaTfG6PJ+m9MQL4aXrFz5pJ\n22LMAC4ECl1n6yme8wYBwzIIrRZkcr5xsmJWQToQ3M9JGorNAS7JNqJc+wpJD4xrnNBtt9Lv5CAZ\np+JxSG7lRsSvgM+k28fSQY0Xa006vMCfSIZtuAs4gOSk+T6Srsyjge9JurLFF7FccbJSG9aVPO9X\noV6h1fvbnRhLzZC0A0kDsj2B54ATImJDtlHlk6T3k/RieZkkabHtU/rZ/M+IWNS8QkQ8QHLFBdxY\nviW3AweRDJx3VkQ8HRHrIuL5dGDMc9N6F/t2Wptlcr5xslIbSu8b7lKhXmHb650YS01IG+T9GhgP\nvAT8bUQszTaqXLuOpJfUZSRt8AaWPkrq9UnXddvuottQ+ll+T6JSZttunRhLVZK0H3B0WrymXJ2I\nuIOkd18PkgHNrPXWUExYuux842SlNiwkuWQMUPZXQjpIV2F8FV+iryBte/FLknFC3iBJVF7KNqrc\n2z1d3g6sLfMouDkt+2+wvAUkjbhby6N6vte+Jc9fbLEWFIbZ36PzQqk9kYwk+0xarHRVqtALqEM+\n605WakBErAWeSItHt1DtYJIGZgC/7fSgqlQ6bcF9wKEkv7z+NiKeyzYq6y4iYj3waFqsNHhjYdvi\nTg2oOpUme6Mq1Csk2Gsr1LHyCvOjlT3fSOoLfDgtdsj5xslK7bg7XU5tYXbgwngCc8rdB7dkDAbg\nHpIBj1YBx0REpXEELBURe0SEWnqUVD07XbdHVrFWgdvT5XHlRpuWdAKwd1r8ZZdFVT1KR6I+p1yF\ntIt9Yawaz13VdoUBC/eR9LEy288h+XHcSMsTJbaJk5XaMZNkFtFBwP3pfdvCFODfpNhr4IsZxZdr\nkupIEr7jSH5pfTQi5lbey6xTfI/k0nkdcI+kgyC5lSvpOODWtN5jOFl5j3QW5cLIqjMkXVkYRC9t\nL3UWcFu6fTHJBLDdlqShhQfJZK0FDaXb0qYEAKRTaPw4Ld4m6fj0teokfZpk8k1Iega2e14g8ESG\nNUXSASSX3HZMV60h6a7Xg+Te9hfTlvDWjKTDgYfT4gaKUxOU83JEfKDzo6odkgpfNJ7IsBUkjQZ+\nR7EB7VqS5KXQMHkByZU/DxVfRnp1+bds3X5lLcmPuYKlJD9KuvXcVSWfzW3ZMyIWl+w3GHiQ4tQF\n60n+Rvuk5ftJut6/QwfwlZUakk7ENx64nqTxWB+SdhcPAEc7Uamo9LPQl2QitJYeHkzKOlV6dWB/\n4AqSxKQnyQ+OuSTjhBzkRKVlEfE6yUl0BvB7YCVJoreG5Bh+Fdi/uycq7RERa0ja9n2B5NZbABtJ\nrvidSzJCcIckKuArK2ZmZpZzvrJiZmZmueZkxczMzHLNyYqZmZnlmpMVMzMzyzUnK2ZmZpZrTlbM\nzMws15ysmJmZWa45WTEzM7Ncc7JiZmZmudYz6wDMzLZFUk/gU8BpwAEk81+tA94gmVriD8CDEfHn\nkn3eD5wMLPZ8RGbVzcPtm1muSRpGMrvwgSWrN5DMQzIYULpudUQ0lOx3FvB94OGIOLJLgjWzTuHb\nQGaWd3eSJCprgYuBERHRL01M6oGjgW8Dq7IL0cw6k28DmVluSdoHOCYtfiYiflK6PSLWArOB2ZIu\n7Or4zKxr+MqKmeXZ/iXP769UMSI2FJ5LCpJbQABHSIpmjyOb7y/pMEmzJL0iaaOkFZJmSzpdksrU\nPzJ9rcVpeYqkhyS9JeltSY9KOmM7/s1m1oyvrJhZtRgJ/LWVdZcC/UjatGwGVjbbvqm0IOkbJLeY\nCtYAQ4C/SR8nSpoaEVvKvZmkGcA1QACr0/c+BDhE0qERcUEr4zazMnxlxczybE7J82+ljW23KSKG\nA9PT4iMRMbzZ45FCXUnTSRKVpcA0oCEi6oEBJL2P3kiXl7TwdsOAbwK3k7SnGQIMBa5Ot5/vKyxm\n7ePeQGaWa5J+AHw6LW4i6ab8GPDfJInI8hb2O4tt9AaS1AC8THKV+ZCIeKpMnQ8CfyJpwDs8Ijal\n648EHkqr/RdwbDT7QpV0G/C/gOeBvZtvN7PW8ZUVM8u7c4B/J0lUepPclrkM+DmwTNKfJU0t166k\nFU4BBgKzyyUqABHxKPAiyW2hyS28zpUtJCJXpMu9SMaHMbPt4GTFzHItIjZFxIXAbsB5wA+B50ja\nhwB8gKR7848ktfU77dB0+RFJb7T0SN+bkmWpzSRXXsrF/hzwelqc1MbYzCzlBrZmVhUiYhkwM30g\naWdgCnA5SRJxKknScF0bXnZEuuyfPralXJ03C7eGWvBq+j6tam9jZu/lKytmVpUiYmlEfJfkisXS\ndPVn2vgyhe/A6yJCrXjc1lHxm1nrOVkxs6oWEW8Cv0iLe7dx90KSM6odIQyV1LvC9l3SZdmGwGa2\nbU5WzKwWrEuXpbdjCmOiVGp4+2i6PFJSv+18717AB8ttkLQXxWRl7na+vlm352TFzHJL0p6Sxmyj\nTn+S2ZUB5pVsWpMuG2jZf5AkOkNI2r5Uep8hFTZf2kJvpEvT5XMRMa/MdjNrBScrZpZn44BFku6R\n9ElJhQaxSBogaQrJuCt7pqtLG9fOT5f7STq43ItHxAqKCcUXJH1HUtOtJEn9JR0l6RbgkXKvAawn\n6U59q6Sd0v0a0lFxC21ovtzKf6+ZleFB4cwstyQdC/xns9WNJLd76kvWvQtcHhFfa7b/w8DhaXEl\nyczNAKdFxGMl9b4EfIXiLaN16Xs0lKxbHBF7luxzJMmgcEuAaykOt78qja3wY/BbHm7frH2crJhZ\nrqVXOqYAhwHjSeYI6k2SeLwA/B74bkTML7PvjiRJyEdL9gM4KiJ+16zu/sAFwFHArkAdSaPYvwC/\nBX4YEa+U1D+SNFmJiD3Sqzz/BEwkacfyNHBjRNzV7oNg1s05WTEz2w7Nk5VsozGrbW6zYmZmZrnm\nZMXMzMxyzcmKmZmZ5ZqTFTMzM8s1N7A1MzOzXPOVFTMzM8s1JytmZmaWa05WzMzMLNecrJiZmVmu\nOVkxMzOzXPv/4lIOSId1am4AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "lsq = lambda A, b: np.linalg.lstsq(A, b)[0]\n", "ws_newton_lsq = newton_method_lsq(init, np.ones(10), gradient, hessian, lsq)\n", "error_plot([log_likelihood(w, data, labels) for w in ws_newton_lsq])\n", "plt.plot(range(len(ws_newton_lsq)), [baseline]*len(ws_newton_lsq),\n", " label='sklearn', **kwargs)\n", "_ = plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's plug in an approximate solver to see what happens. All the tools for linear equations we've seen in previous lectures (gradient descent, conjugate gradient etc.) could be used here. We'll just stick to gradient descent to keep it simple." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAisAAAH/CAYAAACW6Z2MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8XXWd//H3J/ueNE3atOm+F1oKtGwVSlmFgSK4ICpi\n0aEuiDojOs5vcEZhVGRcwAGVRQfBBQRZFUFsoYCUpYUW2lK670uSptn35Pv7454kN/uee8/J6/l4\nnMe9Z73ffHubvvv9fs/3mHNOAAAA0Som0gUAAADoDmEFAABENcIKAACIaoQVAAAQ1QgrAAAgqhFW\nAABAVCOsAACAqEZYAQAAUY2wAgAAolpcpAsw0pnZMknL0tPTr5s1a1akiwMAwLBZt25dkXMut6fj\njOn2o8OiRYvc2rVrI10MAACGjZmtc84t6uk4uoEAAEBUI6wAAICoRlgBAABRjbACAACiGmEFAABE\nNcIKAACIaoQVAAAQ1QgrAAAgqhFWAABAVCOsAACAqMazgQAgSjnnVF5errKyMlVVVamxsTHSRQLa\niI2NVUpKijIyMpSeni4zG5LPIawAQBRyzqmgoECVlZXKzs5WXl6eYmNjh+wfA6CvnHNqbGxURUWF\nioqKVF1drTFjxgzJd5SwAgBRqLy8XJWVlZo8ebJiY2MjXRygAzNTXFycsrKylJ6erj179qi8vFwZ\nGRmD/lmMWQGAKFRWVqbs7GyCCnwhNjZW2dnZKisrG5LrE1YAIApVVVUpLS0t0sUAei0tLU1VVVVD\ncm26gQKmobFJVfWNck2Sk1NqYpziY8mkgN80NjbSqgJfiY2NHbJB4ISVCDOzZZKWzZgxY1Cu9/L2\nIl37f2+2rD/2pcU6edKoQbk2gOHFYFr4yVB+X/kvd4Q55552zq3IzMwclOvFtPuyODcolwUAIGII\nKwET0y7YOtIKAMDnCCsB075lpYmsAgDwOcJKwLTvMmyiZQUAunT//ffLzLR06dI+nfed73xHZqbl\ny5cPSbnQFmElYEztW1YIKwAAfyOsBEzHMSuRKQcAAIOFsBIwMTHcDQQACBbCSsC0b1mhGwgA4HeE\nlYBpPykPYQXASFNXV6c77rhDixcvVlZWluLj4zV27FgtWLBA119/vdasWdPraz344IOKj49XTEyM\nfvGLX/SpHK+88oquuuoqTZgwQYmJiRo9erTOP/98/eEPf+hyWom33npL3/rWt3TmmWdq0qRJLect\nXbpU9913X5czxIYP+G1qatKdd96pU089VVlZWTIzrV+/XpK0fPlymZm+853vqLGxUbfffrsWLFig\nlJQUZWdn69JLL9XatWv79HMOB2awDRgmhQMwkjU0NOjCCy/U6tWrJYX+A5eZmamjR4+qoKBA77zz\njo4ePaozzjijx2vddddduuGGGxQbG6sHHnhAV199da/L8W//9m+67bbbWtYzMjJ07NgxrVy5UitX\nrtRTTz2l3/3ud4qJadtmcOGFF+ro0aOSpJSUFKWkpKi4uFirV6/W6tWr9fjjj+vJJ59UXFzn/3w7\n5/ThD39YTz75pGJjY5Went7pcQ0NDbrkkkv03HPPKT4+XomJiTp27Jj+8pe/aOXKlVq1alWv6mi4\n0LISMO0nO6ZlBcBI8vvf/16rV69WSkqKHnzwQVVVVenYsWOqra3Vnj17dOedd2rBggU9Xud73/ue\nvvzlLyshIUGPPvpon4LKHXfcodtuu01jx47VPffco5KSEpWWlqqyslIPPfSQ8vLy9NBDD+mHP/xh\nh3MvvPBC/eEPf9ChQ4dUWVmpY8eOqaKiQg8++KDy8vL0zDPP6Kc//WmXn/3YY4/p2Wef1c9//nOV\nlZXp2LFjOnLkiKZNm9bmuLvuuktvvvmmHn74YVVUVKi8vFwbNmzQvHnzVFNTo69+9au9/nmHAy0r\nAUPLCjAC/PVb0uF3I12KwZE3X7r41kG73GuvvSZJuuaaa9oEjNjYWE2aNEnXX399j9f4xje+oR/9\n6EdKTU3Vk08+qfPOO6/Xn19SUqKbbrpJSUlJeu6559oEo+TkZH384x/XpEmT9IEPfED/8z//o69/\n/etKSEhoOeb3v/99h2umpqbq6quv1uTJk7VkyRL9/Oc/1ze+8Y1OP7+iokJ33323VqxY0bJtzJgx\nnZbz5Zdf1plnntmy7YQTTtD999+vRYsW6c0339TevXs1adKkXv/sQ4mwEjBMCgeMAIfflfa8EulS\nRKWMjAxJ0qFDh/p8blNTkz7/+c/rvvvuU1ZWlp555pk+d4X86U9/UkVFhS699NIuW3DOOOMMTZ06\nVTt37tS6det6/RlnnXWWsrKytHv3bh08eFDjx4/vcMzo0aP12c9+tlfXCg8qzRYuXKgJEyZo//79\n2rhxY9SEFbqBAobp9gGMZBdffLEk6cknn9Rll12mxx57rGUMSHfq6+v1iU98Qvfdd5/GjBmjF198\nsV9jNl599VVJ0qpVq5SXl9flsm/fPklqeQ33yCOP6PLLL9ekSZOUnJwsM2tZSkpKJEkHDx7s9PMX\nLVrU5XiWcKecckqX+/Lz8yVJx44d6/E6w4WWlYBpN1aLBxkCQZQ3P9IlGDyD/LOcffbZuvnmm3Xz\nzTfr6aef1tNPPy1JmjNnji655BJ9/vOf18yZMzuc1xwyJOnRRx/t1biWzjS36FRVVamqqqrH48OP\naWho0JVXXqnHH3+8ZVtiYqJycnIUGxsrSSosLFRTU5MqKys7vV5ubm6vytnVwFtJSkpKkhQKcNGC\nsBIwHafbj1BBAAydQRzjEUTf/va3dfXVV+vhhx/Wiy++qDVr1mjLli3asmWL7rjjDv3qV7/SNddc\n0+ac+fPnq7GxUZs3b9aXvvQlvfDCC8rJyenzZzc1NUmSvvrVr+r222/v07n33nuvHn/8caWkpOjW\nW2/VFVdcoQkTJrQ5ZuLEidq/f3+X/xFtDjVBQzdQwDApHABIU6dO1be+9S09++yzKi4u1gsvvKAl\nS5aooaFBX/rSl1RQUNDm+OzsbK1cuVKzZ8/Wxo0bdcEFF/SrG2Ts2LGSpL179/b53EceeURSKGzd\ncMMNHYJKY2OjioqK+nzdICCsBEz7SeGIKgBGutjYWC1dulR//vOfFR8fr8rKyk4nPsvLy9PKlSs1\nffp0rV+/XhdeeKFKS0v79FnN41xefPFFVVdX9+nc/fv3S5JOOumkTvf/4x//UE1NTZ+uGRSElYDp\n+CBD4gqAkaOurq7LfQkJCS3dJLW1tZ0ek5+fr1WrVmnKlClau3atLr74YlVUVPT68z/2sY8pNTVV\nx44d080339ztse1bbjIzMyVJ777b8bb0hoYG3XTTTb0uR9AQVgKm491AhBUAI8c111yja6+9Vs89\n95zKy8tbtu/evVuf+cxnVFNTo+TkZJ111lldXmPSpElatWqVJkyYoDVr1uiSSy7p1WBZKXTr8A9+\n8ANJ0q233qrrrrtOW7dubdlfVVWlF154QStWrNDixYvbnHvBBRdIkm655RY9+eSTLVPrb9myRcuW\nLdMbb7yh1NTU3lVEwBBWIszMlpnZPX1tauxKh7DSNCiXBQBfqKmp0f3336+LLrpImZmZGjVqlFJT\nUzV16lQ9/PDDio2N1d13393j4NmpU6dq1apVGjdunF566SVddtllve6CueGGG3TLLbfIzHTfffdp\n9uzZSktLU3Z2ttLS0nTuuefq3nvv7XC9G2+8UdOnT1dZWZkuv/xyJScnKzMzU3PnztXzzz+vX/7y\nl/0a9BsEhJUIc8497Zxb0dz8N1BMCgdgJLv11lt122236aKLLtK0adNUV1enxsZGTZ8+Xddee63e\neustffrTn+7VtWbOnKlVq1Zp7NixWrlypa644oouu4/au+mmm7RhwwatWLFCM2fObLndePz48frg\nBz+o2267TS+//HKbc7Kzs/Xaa6/pi1/8Ysvg2uTkZF1++eVavXq1li9f3qe6CBJjTEN0WLRokRuM\nJ13uP1alM3/4Qsv6bR89QVcumjjg6wIYXu+9957mzp0b6WIAfdLX762ZrXPOLerpOFpWAqbjs4EI\nowAAfyOsBAzT7QMAgoawEjBMCgcACBrCStB0CCuRKQYAAIOFsBIwjFkBAAQNYSVgOoaVCBUEAIBB\nQlgJGMasAACChrASMO0fZMiYFQCA3xFWAoYHGQLBwd9f+MlQfl8JKwHTsWWFX3aAH8XGxrY8yA7w\ng8bGxpanWg82wkrAdGxZiUw5AAxMSkqKKioqIl0MoNcqKiqUkpIyJNcmrAQMM9gCwZCRkaHi4mJa\nV+ALjY2NKi4uVkZGxpBcP25IroqI4anLQDCkp6erurpae/bsUXZ2ttLS0hQbG9uhqxeIFOecGhsb\nVVFRoeLiYqWmpio9PX1IPouwEjBMCgcEg5lpzJgxKi8vV1lZmQoKCmhlQdSJjY1VSkqKcnJylJ6e\nPmRhmrASMO2/JnQDAf5lZsrIyBiypnXALxizEjAdx6yQVgAA/kZYCZj2LXBkFQCA3xFWAsbM2gQW\nxqwAAPyOsBJA4V1BjFkBAPgdYSWAwieGY8wKAMDvCCsBZKJlBQAQHISVAGozZkWkFQCAvxFWIszM\nlpnZPaWlpYN2zfAxK/QCAQD8jrASYc65p51zKzIzMwftmm3GrNAPBADwOcJKAHE3EAAgSAgrQcTd\nQACAACGsBFDbMSuEFQCAvxFWAiimzd1AAAD4G2ElgNqOWSGuAAD8jbASQMYAWwBAgBBWAiiGBxkC\nAAKEsBJA1maelciVAwCAwUBYCaA2dwMxxBYA4HOElQBiUjgAQJAQVgLImBQOABAghJUA4kGGAIAg\nIawEEC0rAIAgIawEEGNWAABBQlgJIGOeFQBAgBBWAogxKwCAICGsBFAMY1YAAAFCWAkgHmQIAAgS\nwkrAMcAWAOB3hJUAYswKACBICCsBFBP2p8rdQAAAvyOsBBBjVgAAQUJYCSBjUjgAQIAQVgIo7M5l\nWlYAAL5HWAmg8HlWAADwO8JKADFmBQAQJISVAGoTVpoiWBAAAAYBYSWAjOn2AQABQliJMDNbZmb3\nlJaWDto1mRQOABAkhJUIc8497ZxbkZmZOWjXpGUFABAkhJUAatOyEsFyAAAwGAgrAUTLCgAgSAgr\nARTDDLYAgAAhrARQ+KRwPMgQAOB3hJUAMiaFAwAECGElgNq2rESuHAAADAbCSgDx1GUAQJAQVgKI\nMSsAgCAhrAQQDzIEAAQJYSWA2s6zErlyAAAwGAgrAcTdQACAICGsBFB4NxDz7QMA/I6wEkAxTLcP\nAAgQwkoAMd0+ACBICCsBxIMMAQBBQlgJIFNrWiGrAAD8jrASQEwKBwAIEsJKADFmBQAQJISVAIoJ\n+1NlzAoAwO8IKwHEgwwBAEFCWAmgsCErjFkBAPgeYSWAeJAhACBICCsBFMNs+wCAACGsBFCbMSsM\nWgEA+BxhJYDCu4HoBQIA+B1hJYB4kCEAIEgIKwHU9tlAkSsHAACDgbASQG26gRhiCwDwOcJKADEp\nHAAgSAgrAcSDDAEAQUJYCSAeZAgACBLCSgAZdwMBAAKEsBJAxjwrAIAAIawEUPiYFYlxKwAAfyOs\nBFD4mBWJcSsAAH8jrARQ+5YVxq0AAPyMsBJA1qFlhbACAPAvwkoAWYcxK5EpBwAAg4GwEmFmtszM\n7iktLR20a8a2SyuNDFoBAPgYYSXCnHNPO+dWZGZmDto142Lb/rE2NBJWAAD+RVgJoPjYti0r9U1N\nESoJAAADR1gJoLgYWlYAAMFBWAmguPYtK420rAAA/IuwEkBxMQywBQAEB2ElgDoMsGXMCgDAxwgr\nARQf074biJYVAIB/EVYCiFuXAQBBQlgJoA4DbOkGAgD4GGElgOK5dRkAECCElQCKbTdmhQG2AAA/\nI6wEUPsZbGlZAQD4GWElgLh1GQAQJISVAGo/KRy3LgMA/IywEkDx3LoMAAgQwkoAtb91mW4gAICf\nEVYCiFuXAQBBQlgJoFhaVgAAAUJYCSCeDQQACBLCSgB1fDYQLSsAAP8irARQxwG2tKwAAPyLsBJA\nHQbYElYAAD5GWAmgDi0rdAMBAHyMsBJAzGALAAgSwkoAmVmbJy9z6zIAwM8IKwEV3rrCpHAAAD8j\nrARU+POB6AYCAPgZYSWgwgfZNtINBADwMcJKQMWF3b5cz63LAAAfI6wEVNsxK7SsAAD8i7ASUOHd\nQAywBQD4GWEloNoMsKUbCADgY4SVgKIbCAAQFISVgAp/8jLPBgIA+BlhJaDiY2lZAQAEA2EloNp0\nA9GyAgDwMcJKQLWZZ4WWFQCAjxFWAopblwEAQUFYCag4bl0GAAQEYSWg4mN4NhAAIBgIKwFFNxAA\nICgIKwHVphuIAbYAAB8jrARUYlzrH21NPWEFAOBfhJWASkuMa3lfWdcQwZIAADAwhJWASkloDStV\ntY0RLAkAAANDWAmotMTYlvd1jU2qa6ArCADgT4SVgApvWZGkKrqCAAA+RVgJqPAxK5JUUUtYAQD4\nE2EloFLCuoEkqZJxKwAAnyKsBFRqu5YV7ggCAPgVYSWgUtuNWamkGwgA4FOElYBKSaAbCAAQDISV\ngGo/wJaWFQCAXxFWAqr9AFtuXQYA+BVhJcLMbJmZ3VNaWjqo1+146zLdQAAAfyKsRJhz7mnn3IrM\nzMxBvW5yfKzMWtdpWQEA+BVhJaDMrM0dQUwKBwDwq7ieD+mcmX3Fe/uoc+7gIJUHgyglIbYlpPAw\nQwCAXw2kZeWnkn4kqWiQyoJBFj5uhZYVAIBf9btlRaGQEuecqxuswmBwZSTHt7wvqeaPCQDgTwNp\nWXlLUqaZ5Q5WYTC4RqW0hpVjlfURLAkAAP03kLDyM+/8bw9SWTDIRqUktLwvrSasAAD8qd9hxTn3\nV0k3SvqCmT1oZgsGr1gYDFlhYeVYFd1AAAB/GsjdQDu9tw2SPinpk2ZWLemopK5uPXHOuen9/Uz0\nTVZYN1BVXaNqGxqVGBfbzRkAAESfgQywndLJthRv6YobwOehj8LHrEhSSVW9xmYQVgAA/jKQsHLO\noJUCQyK8G0gKdQWNzUiKUGkAAOiffocV59zqwSwIBt+o9mGFO4IAAD7EdPsBltWuG6iUuVYAAD40\nkG6gNszMJM2W1DzvSqGk951zjFOJkFGp7buBaFkBAPjPgMOKmc2QdJOkD0tKbbe70sz+JOl7zrnt\nA/0s9E37AbbFlbSsAAD8Z0DdQGZ2maS3JX1aUpoka7ekSbpG0ttmdunAioq+SkmIU0pC690/heW1\nESwNAAD90++wYmbTJT2kUGvKTkmflzRTUrK3zJT0BUk7vGP+6J2DYTQmPbHlPWEFAOBHA2lZ+aak\nJEkvSDrBOXevc26Hc67WW3Y45+6RtEDSakmJkr4x8CKjL3IJKwAAnxtIWLlAoUnePu+cq+7qIG/f\n5xXqFrpwAJ+HfhiT3jqvSkF5TQRLAgBA/wwkrIyTVNqbgbPOua2SSrxzMIzCW1YKaFkBAPjQQMJK\nlaQUM4vv6UAzS1Bo3EqXLTAYGuFhpaquUZW1DREsDQAAfTeQsPKupHhJn+nFsZ/xjn1nAJ+Hfggf\nYCvRugIA8J+BhJUHFRqH8jMz+2dvUrg2zCzJzL4i6WcKjW/5zQA+D/0wpt2zgArKGLcCAPCXgUwK\n92tJVyo00PZuSd81s5clHVDoLqFJkk6TNFqhUPM3SfcPpLDou7EZbVtWDhNWAAA+M5AHGTozu1zS\nTyX9s0KDZ69UqAVFCgUUSWpSKMx8nan3h19+VnKb9f3HGDYEAPCXAU23792W/AUz+4GkKySdrLbP\nBnpL0mPOub0DKiX6LT0pXhlJcSqrCQ2sPVBCWAEA+MugPMjQObdH0u2DcS0MvvFZySo7XC5JOkDL\nCgDAZwYy3f5bZrbOzKYNZoEw+CaMau0KOkjLCgDAZwbSsnKcpDrn3M7BKgyGRvi4lQMl1XLOqZOb\ntwAAiEoDuXX5gFoH0SKK5Ye1rFTVNepYVX0ESwMAQN8MJKw8p9AMtqcNVmEwNCZlp7RZ31VUGaGS\nAADQdwMJK/8t6aikX5pZziCVB0NgWm5am/WdhRURKgkAAH03kDErMyT9h6QfS3rfzB6QtEahW5Yb\nuzrJOffSAD4T/TB5dIpiTGryZrnZScsKAMBHBhJWXlTbCeC+4i3dcQP8TPRDYlysJoxK0d7iKkm0\nrAAA/GUgwWGvWsMKoty03NSwsELLCgDAPwYy3f6UQSwHhtjUnFS9+H6hJGnP0So1NjnFxnAzFwAg\n+g1kgC18JHyQbV1jEzPZAgB8YyAz2B4zs6PMYOsP03NS26zvKGLcCgDAHwbSspIgKZYZbP2h4+3L\njFsBAPjDQMLKXoUCC3xgbEaiUhNiW9a5IwgA4BcDCStPSUo0swsGqzAYOmamqbmtXUHbjhBWAAD+\nMJCw8n1JuyXda2ZzB6c4GEpz8zJa3m8+VKamJu48BwBEv4HMs/IhSb+Q9J+S3jazv6p3M9g+MIDP\nxADMy8/UI+v2S5Iqahu0p7hKU9sNvAUAINoMJKzcr9CkcM2TdVzmLT0hrETIvPyMNusbD5QSVgAA\nUW8gYeUlMYOtr8wdlyEzyXl/ahsPlmrZgvGRLRQAAD0YyAy2SwexHBgGKQlxmp6bpu0FocG1mw6U\nRbhEAAD0jBlsR5h541u7gjYeLJVzNI4BAKJbr8OKmX3FzD7Xxb40M8vobF/YMT81s1/1tYAYXPPy\nM1vel1TV60AJ0+4DAKJbX1pWbpd0cxf7tkkq7uH8qyQt78PnYQgcPz6zzfo7+0sjVBIAAHqnr91A\n3T2ml0f4+sC8/AyFP2x57e5jkSsMAAC9wJiVESY9KV5zx7X22L25u6cGMQAAIouwMgKdMiW75f2m\ng6WqqG2IYGkAAOgeYWUEOnVqa1hpctJbe+gKAgBEL8LKCBTesiJJa+kKAgBEMcLKCJSbnthmmv03\nCCsAgChGWBmhTpkyquX923tLVFPf5bMnAQCIqL5Ot59tZqs62y5JXexrcwyiw+LpOfrj2tATmGsb\nmvT6rmKdPSs3wqUCAKCjvoaVBElLu9nf3T6JBx9GjTNn5rRZf2lrIWEFABCV+hJWfjNkpcCwy0lL\n1Pz8TL17IDSD7eqthfp2hMsEAEBneh1WnHPXDmVBMPyWzMppCSvbCyp0oKRa+VnJES4VAABtMcB2\nBFsys223z+r3CyNUEgAAukZYGcFOnjxK6YmtjWurthyJYGkAAOgcYWUEi4+N0dmzW1tXXtpWxNT7\nAICoQ1gZ4S6al9fyvq6hSS9sKYhgaQAA6IiwMsItnT1GCXGtX4NnNx2OYGkAAOiIsDLCpSXG6awZ\nrXOuvLClgNlsAQBRhbACfTCsK6iqrpGuIABAVCGsQB88Lk8Jsa1fhcffPhDB0gAA0BZhBcpMidc5\nc1rvCnrh/QKVVNVFsEQAALQirECSdMVJE1re1zc6/eXdQxEsDQAArQgrkCSdMydXGUmtE8Q9um5/\nBEsDAEArwgokSYlxsbp0wfiW9bf3lmjzwbIIlggAgBDCClp88tRJbdZ/+/qeCJUEAIBWhBW0mJef\nqQUTs1rWn3j7gMpr6iNYIgAACCto5+rTWltXquoa9QS3MQMAIoywgjaWLRivzOT4lvXfvrZXzrkI\nlggAMNIRVtBGUnysPrqw9Tbm94+U67WdxREsEQBgpCOsoINPnTZJZq3rv1y9I3KFAQCMeIQVdDAt\nN00XHje2ZX311kJtOlgawRIBAEYywgo69YWzp7dZ/+XqnREqCQBgpCOsDAIzSzOz75rZM2ZWaGbO\nzL4V6XINxEmTRun0adkt639556D2HK2MYIkAACMVYWVw5Ej6T0nzJb0d4bIMmi8undHyvslJv3iR\nsSsAgOFHWBkchyTlO+cmSloR6cIMliUzc3T8+IyW9UfW7dfOwooIlggAMBIRVgaBc67WOXcw0uUY\nbGamr50/q2W9scnpJ89vjWCJAAAjEWEF3Tp/7hidNKl1Cv4/v3OIO4MAAMMqasOKmaWb2WVmdouZ\n/dXMiryBq87M5vTyGnlmdoeZ7TCzGjM7YmZPm9l5Q13+oDAzfeODs9ts+9Fz70eoNACAkSgu0gXo\nxnmSHu/vyWZ2gqRVkkZ7m8oUGgh7qaRLzOz/OedubXeOSUrs5UfUOeea+ls+P1k8PUdnzczRy9uK\nJEkvvF+ol7cV6qyZuREuGQBgJIjalhVPgaRnJH1XfRi4ambJkp5SKKi8LWmecy5T0ihJP5Zkkr5v\nZhe2O/UDkqp7uSzp90/lQ9/8YNvGrO8+vVn1jSMiqwEAIiyaW1aeds490bxiZlP6cO7nJU2WVCFp\nmXPugCQ558ok3Whm0yVdLukHkv4Wdt5WSdf28jO29KE8vjd/QqY+unCCHl23X5K0vaBCD6zZo8+d\nOTXCJQMABF3UhhXnXOMATv+U9/r75qDSzv8oFFZONrPZzrn3vc8skHT/AD430L550Ww9u/GwKmob\nJEm3/32rPnTieOWk9bbnDACAvov2bqA+M7N0SQu91ee6OOw1Sc23tDDYtpfGpCfpK+e1ThRXXtOg\n7//lvQiWCAAwEgQurEiaq9CYFEna1NkB3sDY5ltajhuMDzWzL5vZTZK+7G06x8xu8pbMwfiMaLB8\n8VRNy0ltWX/s7QN6aWthBEsEAAi6IIaVcWHvu5uorXnfuG6O6YsbJd0i6eve+oXe+i0KDewNhIS4\nGP33FfPabPt/j7+rqrqGCJUIABB0QQwrqWHvq7s5rsp7TRuMD3XOTXHOWRfL7s7OMbMVZrbWzNYW\nFvqndWLx9BxddcrElvX9x6r1k78xsy0AYGgEMaz4hnPuHufcIufcotxcf81Z8u8Xz1VueuvA2l/9\nY5de23k0giUCAARVEMNKZdj75G6OS/FeeTJfP2SmxOvmy45vWXdO+vofN6i0uj6CpQIABFEQw0r4\nOJXx3RzXvO/QEJYl0C6eP05XnJTfsn6gpFr/9eTGCJYIABBEQQwrWyQ57/3xnR1gZjGSmh94s3k4\nChVU3/3Q8crPam3AemL9QT25vrOpbQAA6J/AhRXnXLmktd7qBV0cdpqk5tuJVw55oQIsIyleP/34\niTJr3XafUs1iAAAgAElEQVTTExu192hV1ycBANAHgQsrnt97r58ys85uTb7Re13XPHst+u/Uqdn6\n4tnTW9bLaxr0hd+uU039QCYhBgAgJKrDipnlNC9qO1dJVvg+r1sn3N2S9khKl/RnMzvOu166md0m\n6cPecf9vqH+GkeJr58/Sggmtc99tPlSmm57YKOdcN2cBANCzqA4rkgrDlrfCtq9pt29S+EnOuWpJ\nH5J0VNLJkjaZWamkEknfUGhMy78758IfYogBSIiL0c+vXqhRKfEt2x5dt19/eGNfBEsFAAiCaA8r\n/eac2yBpnqSfSdopKVGh8PIXSRc4526NYPECKT8rWXdcdVKb8SvfeWqT3tp7LHKFAgD4XlSHlW5m\nhO3VDLHOucPOua8656Y755Kcc2Occ5c65xhUO0SWzMrVv54/q2W9rrFJKx5Yq33FDLgFAPRPVIcV\n+NP158zQ+XPHtKwXVdTpn3+zVuU1TBgHAOg7wgoGXUyM6farTtKcvPSWbe8fKdeXf/+2GhqbIlgy\nAIAfEVYwJNIS4/Sr5acoJ631+UGrtxbq209u4g4hAECfEFYizMyWmdk9paWlkS7KoMvPStZ9n1mk\nxLjWr9kf3tirH/OEZgBAHxBWIsw597RzbkVmZmbPB/vQiROzdHu7GW7vfGG7fv3KrsgVCgDgK4QV\nDLmL54/TLR+a12bbzX/erMfe2h+hEgEA/ISwgmFx9emTdeOFs9psu/GRDTz0EADQI8IKhs3158zQ\ntR+Y0rLe5KR/eXg9gQUA0C3CCoaNmenblxynq06Z2LKNwAIA6AlhBcMqJsb0/SvmE1gAAL1GWMGw\naw4sH1/UNrB87eH1evC1PREsGQAgGhFWEBExMaYffHi+rlw0oWWbc9K3n9ion63cxsRxAIAWhBVE\nTEyM6dYPn6BPnTapzfafPL9VN/95s5qaCCwAAMIKIiwmxvTfl8/TDefOaLP9//6xW//6x/WqbWiM\nUMkAANGCsIKIMzN9/cLZ+valx7XZ/sT6g/r0fW/oWGVdhEoGAIgGhBVEjc+dOVU//tgCxca0zs3/\nxu5iffgXr2pXUWUESwYAiCTCCqLKRxZO0K+Xn6K0xLiWbbuKKnXFz/+h13cejWDJAACRQlhB1Dl7\nVq4e/eIZys9KbtlWUlWvT933uh5Ys5s7hQBghCGsICrNycvQ49cv1gkTWp9G3dDk9J9PbtKNj7yj\nmnoG3gLASEFYiTAzW2Zm95SWlka6KFFnTHqSHl5xhv5pfl6b7X96a78+9ss12n+sKkIlAwAMJ8JK\nhDnnnnbOrcjMzOz54BEoOSFWd33yZH3zotkKG3erdw+U6rI7/6FXthVFrnAAgGFBWEHUMzN9aekM\n3X/tqcpMjm/ZXlxZp0//+nX96Ln31dDYFMESAgCGEmEFvrFkVq6e/vKZmjsuo2Wbc9KdL2zXVfe8\npoMl1REsHQBgqBBW4CuTRqfosS8u1odPzm+zfe2eY7r4jpf1t02HI1QyAMBQIazAd5ITYvWTK0/U\njz+2QCkJsS3bS6vrteLBdfqPx99VZW1DBEsIABhMhBX41kcWTtCfbzhTx4V1C0nS717fq3/62ct6\nc3dxhEoGABhMhBX42rTcND32pcVavnhKm+17jlbpyrvX6AfPvMecLADgc4QV+F5SfKy+c9nx+r/l\np2hMemLLdueku1/aqcvufEUbDzCPDQD4FWEFgXHOnDF67mtLtGzB+Dbbtx6p0OV3/UM/fHYLrSwA\n4EOEFQTKqNQE/e8nTtKdnzxJWSmtc7I0NDn94sUd+uDtL+kf25lIDgD8hLCCQLr0hPH629eW6Nw5\nY9ps33O0Sp+673Xd+MgGHausi1DpAAB9QVhBYI3JSNKvPrNIt3/8RGWnJrTZ9+i6/TrvJ6v1+Nv7\neYozAEQ5wgoCzcx0+Un5WvmvZ+sjJ09os6+4sk7/8vAGXXn3Gm06yABcAIhWhBWMCKNSE/TjKxfo\nt587TZOyU9rse3P3MS3731d00xPvqqSKriEAiDaEFYwoZ87M0XNfW6IvLp2u+NjWxzg3Oem3r+3V\n0h+9qN++tkeNTXQNAUC0IKxgxElOiNW/XTRHz35tic6eldtmX0lVvW56YqMuu/MVrdlxNEIlBACE\nI6xgxJqem6b7rz1F916zqEPX0KaDZfrEva/pc/e/qW1HyiNUQgCARFjBCGdmuuC4sfrbvyzR1y+Y\npaT4tn8lVm4p0Advf0n//ti7KiiriVApAWBkM27bjCwzWyZp2YwZM67btm1bpIsz4h0oqdatf92i\npzcc7LAvJSFW1501TSuWTFNqYlwESgcAwWJm65xzi3o8jrASHRYtWuTWrl0b6WLAs2Ffib7/zHt6\nfVfHJzePTk3QF5dO19WnT1ZSfGwESgcAwUBY8RnCSvRxzmnlewW69dkt2l5Q0WH/2IxEXX/ODH38\nlIlKjCO0AEBfEVZ8hrASvRoam/THtfv1k+e3qqiitsP+/Kxk3XDuDH1k4QTFxzIMDAB6i7DiM4SV\n6FdZ26DfrNmtu1fvVGl1fYf9k0en6KvnzdRlC8YrjtACAD0irPgMYcU/ymrq9etXdulXL+9SeW1D\nh/2TslP0hbOn6yML8+keAoBuEFZ8hrDiPyVVdbrnpZ26/9Xdqqpr7LA/LyNJ1y2Zpk+cOlEpCdw9\nBADtEVZ8hrDiX0UVtbp79Q49sGaPahuaOuzPTk3Q586cqk+fMVkZSfERKCEARCfCis8QVvyvqKJW\nv3pllx5cs0cVnXQPpSfG6ZOnTdLyD0zRuMzkCJQQAKILYcVnCCvBUVpVr9+s2a1f/2OXSqo6DsSN\nizFdesI4/fNZ0zQvP3P4CwgAUYKw4jOEleCprG3QH97Yq3te2qmC8o63PEvSGdNG67olU7V01hjF\nxFinxwBAUBFWfIawElw19Y167K0Duu/lndpZVNnpMTPGpOlzZ07VFSflMysugBGDsOIzhJXga2py\nWrWlQPe+vLPTafwlKSslXh9fNFFXnz5ZE9s9CRoAgoaw4jOElZHlnf0luu/lXfrLu4fU2NTx76CZ\ndO7sMbpm8RSdNSOHLiIAgURY8RnCysh0oKRa//fKLj305r5O7yCSpKk5qbr69Mn66MIJykzm1mcA\nwUFY8RnCyshWXlOvx98+oN+8uls7Cjsf15IcH6vLT8rXp06bxF1EAAKBsOIzhBVIoSc9v7rjqB5Y\ns1vPbz6iTnqIJEnz8jN01SmT9KETxyudieYA+BRhxWcIK2jvQEm1fvfaHj305j4VV9Z1ekxyfKwu\nPWGcrjp1kk6elCUzxrYA8A/Cis8QVtCVmvpGPfPuIT2wZo/W7yvp8rjZY9P18VMm6sMn5ysrJWEY\nSwgA/UNY8RnCCnrjvUNleuiNvXr87QMqq+l8QG5CXIwuOG6sPnryBJ01M0dxsTHDXEoA6B3Cis8Q\nVtAXza0tD72xT2/s7nzOFknKSUvU5SeO10cWTtDccRnDWEIA6BlhxSfMbJmkZTNmzLhu27ZtkS4O\nfGh7QYX+uHafHl23v8uxLZJ03LgMfWThBH3oxPHKSUscxhICQOcIKz5DywoGqq6hSc9vPqI/vbVf\nq7cWdjrZnBR6kOLS2bn68MkTdO6cMUzvDyBiCCs+Q1jBYCoor9FT6w/q0XX7teVweZfHpSXG6cLj\nxmrZieN15owcxTO+BcAwIqz4DGEFQ2XzwTL96a39enL9ARVVdN1NlJ2aoIvn5elDJ+Zr0eRRTPEP\nYMgRVnyGsIKhVt/YpJe2FupPb+3X3zcXqK6xqctjx2UmadmC8bpswXgdPz6D+VsADAnCis8QVjCc\nSqvr9dymw3pq/UG9uqOoy5lyJWlaTqouOWGcLp43TnPHpRNcAAwaworPEFYQKQXlNXrmnUN6asNB\nvbW360nnJGnK6BRdNG+c/ml+nubnZxJcAAwIYcVnCCuIBvuKq/T0Owf11PqD3Q7MlaT8rGRdPC9P\nF88fp5MmZjHGBUCfEVZ8hrCCaLPtSLme2nBQf3nnkHYWdf4k6GZ5GUm6aF6eLp6Xp0VTshVLcAHQ\nC4QVnyGsIFo557T1SIWeefeQnt14WO8f6b7FZXRqgs6bO0bnzx2rs2bmKjmBeVwAdI6w4jOEFfjF\njsIKPbvxsJ5595A2HSzr9tik+BidOSNXFxw3RufOGavcdGbOBdCKsOIzhBX40d6jVfrrxkN6ZuNh\nbejmidCSZCadNDFLFxyXpwuOG6vpuakM0AVGOMKKzxBW4HcHSqr13MbDen7zEb2xu7jL6f6bTc1J\n1QXHjdV5c8Zo4eRRPB0aGIEIKz5DWEGQlFbV64X3C/T85iN68f0CVdY1dnt8RlKczpqVq3Nmj9HZ\ns3LpLgJGCMKKzxBWEFS1DY16bWexnt98WH/fXKDDZTU9nnPChEwtnT1G58zO1QkTsri7CAgoworP\nEFYwEjjntPFAmZ7ffFh/23ykx7lcpNAzi86elauls3O1ZGauRqUmDENJAQyH3oaVuOEoDIbRX78l\nHX430qUAOmWS5nvLv2ZKtamNKqmqV0l1nUqrG9TU2X+e6iVtCi3vK/Sk6KyUeGUmxystMU4mWl2A\nYZM3X7r41mH/WMJK0Bx+V9rzSqRLAfRKoqSx3iLzlp7USyr1FgAjAsPvAQBAVKNlJWjy5ke6BMCg\na3ROZdX1KqmqU0l1vWobmnp1XnpinDKT45WZQpcRMCgi9G8MYSVoItCXCAy1WEmjvMU5p91Hq/Ty\ntkK9vK1Ia3YcVUVtQ+cn1knyxvCmJ8XpjGmjtXj6aJ0xPUezxqYxKR3gE9wNFCW4Gwjon/rGJq3f\nV6KXtxXp5W2F2rCvRD3MRydJyklL0GleeFk8PUdTRqcQXoBhxq3LPkNYAQZHaXW91uwo8sJLkfYW\nV/XqvLyMJC2ePlqnTw8FmAmjUoa4pAAIKz5DWAGGxp6jlXp5W5Fe2Vak13YdVUlVfa/Om5idrMXT\ncnTG9NE6Y/pojc1IGuKSAiMPYcUnzGyZpGUzZsy4btu2bZEuDhBoTU1O7x0u05odR7Vmx1G9vqu4\n6/Eu7UzNSdWpU7J1ytRsnTY1WxNGJdNtBAwQYcVnaFkBhl9DY5M2HizTqztCA3Xf3F2smvre3Wk0\nLjNJp07N1qleeJmey4BdoK8IKz5DWAEir7ahURv2lWrNjqN6dUeR3t5borrG3oWX7NQEnTJllE6d\nOlqnTc3W3HEZPNMI6AFhxWcIK0D0qalv1Nrdx/TGrlCX0dv7SlTXhzleFk4ZpVOmhFpf5udnKik+\ndohLDPgLYcVnCCtA9KttaNQ7+0v1xq5ivb6rWOt2F6uyrrFX58bHmublZ2rhpFFaODm0jGHQLkY4\nworPEFYA/2lobNJ7h8r1+q6jemNXsd7cXaxjvbzbSArdcbRw0igtnJKthZNGaXZeOl1HGFEIKz5D\nWAH8r6nJaXthhd7YVey1vhzVkbLaXp+flhinEydmtbS8nDgpSxlJ8UNYYiCyCCs+Q1gBgsc5p33F\n1Vq3t1jr9hzTuj0l2nK4TL39tWsmzR6broWTR+nkSaHwMnV0qmJofUFAEFZ8hrACjAzlNfVav6/E\nCy/H9Pbekl7P9SJJGUlxWjAxSydNzNKJk7J04sRRyk5NGMISA0OHsOIzhBVgZGpsctp6pLwlvKzb\nc6zXjwhoNik7RSdOzAotk7J0/PgMJcZx5xGiH2HFZwgrAJoVlNforbDwsvFAWa/ne5FCdx4dNy6j\nJbycOHEUD2pEVCKs+AxhBUBXahsatflgmdbvK9H6fSV6e29Jn1tfslLitWBClhZMzNIJ+Zk6YUIm\nt04j4ggrPkNYAdAXRytqtWF/idbvK9X6fSXasK9EpdW9v21aksZmJGp+fqbm52fphAmZmj8hUzlp\niUNUYqAjworPEFYADIRzTruKKltaX9bvK9Hmg2VqaOrb7/jxmUmaPyFTJ0zI8oJMpkYxgBdDhLDi\nM4QVAIOtpr5Rm8K6j9bvO6Z9xdV9vs7E7OQ2LTDz8jOVmcz8Lxg4worPEFYADIdjlXXaeLBU7+wv\n1bv7S/XugVIdKOl7gJkyOkXz8jN1/PhMHT8+Q8ePz9BoupDQR4QVnyGsAIiUoopavXugVBv3l+qd\nA6EQc7isps/XyctIagkux3khZsKoZO5CQpcIKz5DWAEQTQrKavTuAa8Fxnstquj9owOaZSbH67hx\noQBzfH6Gjh+fqWk5qYqLjRmCUsNvCCs+Q1gBEM2cczpcVtPSdbRhf6k2HyxVUUVdn6+VGBejOc0B\nZnwowMzJS1dSPBPZjTSEFZ8hrADwG+ecCsprtelgqTYdKNOmg2XadKi0X4N4Y2NM03NTNXdchuaO\ny9CcvHTNHZehMemJdCMFGGHFZwgrAIKitLpemw+WadPBUu+1TNsLK9TYx9uoJSk7NUFz8tI1Jy9D\nc8al67hxGZoxJo1WmIAgrPgMYQVAkNXUN+r9w+Wh1peDpdp0sExbDpeppr73jxFoFhtjmpqTGtYC\nE2qFyctIohXGZwgrPkNYATDSNDQ2aVdRZUuA2XyoTO8dKldxZd/HwUihwbzN3Udzx4VaY2aNTVdy\nAq0w0Yqw4jOEFQAIjYMprKjVe4fKteVQmbYcLtd7h8q0vaCiz7PxSlKMSVNyUjV7bLpmecvsvDRN\nGc0dSdGAsOIzhBUA6FpdQ5N2FFZoy+FQ68t7XpApLO/77dSSlBAbo2m5qV548ULM2HRNGJWsmBi6\nkoYLYcVnCCsA0HdFFbXacqi8TYjZXlChusa+j4WRpOT4WM0cm9YSXmaOTdPsvHTGwwwRworPEFYA\nYHDUe2Nh3vPGwGw5XKZtRyr69ViBZulJcV54SdfssWmalRcKMzxiYGAIKz5DWAGAoVVeU69tBRXa\nerhc7x8p19Yj5dp6pKLfXUmSNDo1QTPHpmnGmDTNHJOuGWNC75kfpncIKz5DWAGAyCiurPOCi7cc\nrtD7R8pVWl3f72umJ8WFgktuWkuYmZHLmJj2CCs+Q1gBgOjRPDvv1iPlev9wKMS8f6RC246Uq6qu\nsd/XTYqP0bSc5paYtJaWmMmjU5UQN/LuTuptWIkbjsIAAOAnZqaxGUkam5Gks2bmtmxvanI6UFLt\nhZdybT1cru2FFdpeUNGrCe5q6pu0+VCZNh8qa7M9LsY0eXRKm66kGWPSND03jXliRMtKxJnZMknL\nZsyYcd22bdsiXRwAQD80h5jtBRUty7aCcm0vqFBZTUO/r2sm5Wcla1pumqblpGr6mDRNz0nVtNw0\njc3w/7gYuoF8hm4gAAie5knuth+p0PbCCm074oWZwoEN7JWk1IRYTc1N1bScUAvMtNzU0JLjn9YY\nuoEAAIgwM9OY9CSNSU/S4hk5bfaVVtVre2Go9WWbF2a2F1Ro/7He3WJdWdeojQfKtPFAWYd94zOT\nNC03TdNzQ60w07zXcRlJvhzgS8tKlKBlBQAgSVV1DdpZWNnSjbSjoFI7iyq0u6iq35PdNUuOj9XU\nnNSW8DLda4mZlpuq1MThb7+gZQUAAB9KSYjTvPxMzcvPbLO9sclp/7Eq7Sys1I7CCu0sqtTOwgrt\nLKxUQS+7lKrrGzsd4CtJYzMSNS0nzetaStWU0amampuqiaNSIn6nEi0rUYKWFQBAf5XX1GtnYagF\nZmdhZUug2VVUqdqGgbXGxMaYJo5K1tScVE3NSdPnzpqq/KzkQSk3LSsAAIwQ6UnxWjAxSwsmZrXZ\n3tTkdLC0WjsKW1thmgPNodKaXl27sclp99Eq7T5apRfeL9Q1Z0weih+hW4QVAAACKibGNGFUiiaM\nStHZs3Lb7KusbdCuokrtLKrUjoJQK8zuo6FWmYrazm+3josxTRg1OK0qfUFYAQBgBEpN7HxsjHNO\nRRV12lVUqV1FobExuwortauoUrExprjY4R+/QlgBAAAtzEy56YnKTU/UqVOz2+yL1DjXkfcgAgAA\n0C+RmjGXsAIAAKIaYQUAAEQ1wgoAAIhqhBUAABDVCCsAACCqEVYAAEBUI6wAAICoRlgBAABRjbAC\nAACiGmEFAABENcIKAACIaoQVAAAQ1QgrAAAgqhFWAABAVCOsAACAqEZYAQAAUc2cc5EuAySZWaGk\nPYN4yRxJRYN4vZGKehw46nDgqMOBow4HbijqcLJzLrengwgrAWVma51ziyJdDr+jHgeOOhw46nDg\nqMOBi2Qd0g0EAACiGmEFAABENcJKcN0T6QIEBPU4cNThwFGHA0cdDlzE6pAxKwAAIKrRsgIAAKIa\nYQUAAEQ1wgoAAIhqhJUAMbM8M7vDzHaYWY2ZHTGzp83svEiXbTiYWbqZXWZmt5jZX82syMyct8zp\nxfkJZvZNM1tvZhVmVmJma8xshZlZL87/mJmtMrOjZlZlZu+Z2X+bWfrg/ITDw8wmmdnXvO/OXjOr\nNbNyM9tgZrea2bgezh/x9Whmi7zv4bNmtt3MSr16PGBmT5rZ5T2cP+LrsD0zSzOzfWF/p5d3cyz1\nJ8nMlofVV1dLRTfnR089OudYArBIOkGhmQWdt5RKavTeN0n6VqTLOAx1cHnYz99+mdPDuRmS1oYd\nXympNmz9aUlx3Zx/T9ix9ZLKw9Z3SBof6frpZR1O9L4v4XVXKqkhbL1Y0jnUY7f1+Mt2dVguqbrd\ntkclxVOHva7T29vV33K+gz3W2XKv3HWSDnex7PBDPUa8MlkGvkhKlrTb+xK8Jel4b3uGpB+pNbBc\nGOmyDnE9XC7piKS/SPqOpOvC/nL0FFYe9o47KulSSSYpVtJnwv6R+V4X537R298o6UZJid72xWF/\nLq9Eun56WYdTvO/KnyV9VNIob3uCpIsl7VRrgMmjHrusx89I+pqkkyWlhW2fKOm2sO/lt6nDXtXn\nyQoF5tfC6m55F8dSf60/z3KvzC/249yoqseIVybLwBfvl2Lz/97yO9n/uLd/XaTLOsT1ENtufYp6\nEVYknRR23GWd7P+qt69K0ph2+xIVCkhO0k+6uHZzS8WySNdRL+owU9KCbvbPCftF9V/UY7/r+UHv\nZ9nRbjt12LHcMZLeVCishNfP8i5+RuqvtczL1Y+wEo31GPHKZBn44v1FdpLu7mL/4rAv3uxIl3cY\n62VK2M/dXVj5H++YLV3sT5ZU4h3zhXb7LlFry1WHoOgd8zfvmIciXSeDVK8veD/PU9Rjv+vwK97P\nUk0d9lhXzf8w/q+33l1Yof7alre/YSXq6pEBtj7nDVRa6K0+18VhrynUbC9JI2KwbR+d473+rbOd\nzrlqSS97q+d2ce5G59yBLq7f/OfS/ly/Ouq9xrbbTj323mLvdVe77dRhGDPLl3SLQv9Tv6kXp1B/\ngyPq6pGw4n9zFepLlKRNnR3gnGuS9L63etxwFMovvBHtzXcKdVp/ns3ea/v6a17vzbm5ZpbTtxJG\nFzOLk/QBb3Vj2HbqsQfe3SwnmNldkj7ubb4zbD912NH/SkqXdKNzrrS7A6m/bh1vZpvMrNq7s2+j\nmf3UzKa2PzBa65Gw4n/ht5Ee7Oa45n3d3nY6AmVISvXe96f+xrXb3925nZ3vN9dLylOoifc3Ydup\nx06Y2YTmW0QVGlO2QdKXJNUoNLj252GHU4dhzGyZpCsU6sL4bS9Oof66lqPQf2yrJCVJOl6hsY6b\nzOyT7Y6NynokrPhfatj76m6Oq/Je04awLH400PprPr8353Z2vm+Y2QmSfuCt3umc2xy2m3rsXKNC\nXRhHFLp9VAoNFP2BpLvaHUsdeswsVaFWp3qFAnJvUH8dHZT0X5LmSUpyzo1WqNyXKNS6kSzpN2a2\nJOycqKxHwgqAHnkTwT2h0C+3dZL+LbIl8gfn3CHnXJ5zLk+hupst6QFJ35W03syOj2gBo9fNkiZJ\n+mm7UIw+cM79zTl3s3Nuk3OuzttW65x7RqFxU9slxUm6NZLl7A3Civ9Vhr1P7ua4FO+1y9kKR6iB\n1l/z+b05t7Pzo56ZZSs00G6qpG2SLnHO1bQ7jHrsgXOuyTm31Tn3OUk/Uegf4wfNrPn3MHUoycxO\nVOgOoH0KhZbeov76wBsD9H1v9fSwsSNRWY+EFf8L7/sb381xzfsODWFZ/KhMrX+5+lN/B9vt7+7c\nzs6PamaWqdDI/XmS9ko63zl3pJNDqce++V/v9SRvkajDZncodKfZfyg03jMtfAk7LtHb1vwPH/XX\nd697r6bQf0akKK1Hwor/bVHofnUpNGiqA+9/brO9VZpUw7jQTf/veavdNck3j3BvX3/N6705t9A5\nV9S3EkaON27gGUmLFJqW+3zn3N7OjqUe+yz8ls7pEnUYZrL3+oBCg5LbL81+6a1vlqi/wRKt9UhY\n8TnnXLlCz2+QpAu6OOw0hWYmlaSVQ14o/3nBe+20/swsSdJZ3mr7+ms+93jr+gF/F3ZxbtQys2SF\nnv2xWKF5Vc53zm3r4TTqsffCbxkNbwanDgeG+uub08Le7w57H331GOkZ9lgGvqh1uv0ySeM62f8n\nb//aSJd1mOtlivo+3f6lney/Qb2bWvpHnZy7QK0PlOxw7WhcFHoO0F+9Mh+TdHIvz6MeQ2WNlWQ9\nHHOv97PUy3v+EnXY6/rtbgZb6q+1vD19BzMUmn/LSXo92usx4hXKMvBFbR9kuE7Scd72dLV9aFqg\nH2To/cw5YUv4X7jT2+2LaXde80O7iiT9k7ctVtI13l9Ip949tOvran1o1xlqffCfLx5+5v3Mj6o1\n/J7ex/NHfD0qFJLXSfqspAlh22MknSjpd2Hfy86enTLi67CH+u0yrFB/Hb6Hryr04MH8sO0Jki6S\n9G7Yz3putNdjxCuUZXAWhdJqUdhf5FK1ptcmSd+KdBmHqR5cL5cp7c7r7HHoNWHrfXkcep18+lh5\nSUvCyl2trh8rf1jSm52cP+LrUW1b9JrrsbBdPThJ/9dZXVCHPdZvT2GF+uv8e1jlfQ/r2tXNp/1Q\njxGvUJbBWxSaWfQO74tQI6lA0p8lnRfpsg1jHfQrrHjnJig0f8h6hcYRlEpaI2mFemhS9c6/UtIq\nSexjrzoAAAVySURBVMXeP1DvSfpvSemRrpc+1N/SPtTh7i6uMaLr0fv5r5R0t6S3FQp29d4v602S\n7pP0gV5cY8TWYQ8/W7dhhfpr+RmSFequeVihGzGOet/DEoUefnurpMl++R6ad0EAAICoxN1AAAAg\nqhFWAABAVCOsAACAqEZYAQAAUY2wAgAAohphBQAARDXCCgAAiGqEFQAAENUIKwAAIKrFRboAANAT\nM4uTdLWkqxR6DtZohZ5VclihB6O9LGmVc+6NsHNOlHS5Qo8FuH+4ywxg8DDdPoCoZma5kp6RtChs\nc42kWoUetmbetlLnXFbYecsVeljgaufc0mEpLIAhQTcQgGj3W4WCSrmkb0oa55xL9oJJpqQLJP1c\noQe0AQgguoEARC0zmyPpQm/1s865R8P3O+fKJf1d0t/N7OvDXT4Aw4OWFQDRbH7Y+z93d6Bzrqb5\nvZk5hbqAJOlsM3PtlqXtzzezM83sITPbb2a1ZnbUzP5uZp8wM+vk+KXetXZ768vM7AUzO2ZmFWa2\nxsw+2Y+fGUA7tKwA8It8STt6eewRSckKjWmpl1Tcbn9d+IqZ/VChLqZmZZJGSTrPWy4zs08555o6\n+zAz+5qkn0pykkq9zz5d0ulmttg59+VelhtAJ2hZARDN1oW9v8sbbNsj51yepK96q6865/LaLa82\nH2tmX1UoqByRtEJSlnMuU1KqQncfHfZe/62Lj8uVdJukBxQaTzNKUo6kH3v7r6eFBRgY7gYCENXM\n7DeSrvFW6xS6Tfk1SW8qFEQKuzhvuXq4G8jMsiTtU6iV+XTn3IZOjjlD0j8UGsCb55yr87YvlfSC\nd9jzkj7o2v1CNbP7JX1G0nZJs9rvB9A7tKwAiHbXSfqJQkElQaFumf+Q9ISkAjN7w8w+1dm4kl74\niKQ0SX/vLKhIknNujaRdCnULLeziOj/oIoh8z3udodD8MAD6gbACIKo55+qcc1+XNFHSFyT9QdI2\nhcaHSNIpCt3e/LCZ9fV32mLv9VwzO9zV4n22wl7D1SvU8tJZ2bdJOuStntzHsgHwMMAWgC845wok\n3e0tMrOxkpZJ+k+FQsTHFAoNd/ThsuO81xRv6UlnxxQ1dw114YD3Ob0abwOgI1pWAPiSc+6Ic+4+\nhVosjnibP9vHyzT/DrzDOWe9WO4frPID6D3CCgBfc84VSXrSW53Vx9ObQ86kARQhx8wSutk/3nvt\ndCAwgJ4RVgAEQaX3Gt4d0zwnSncDb9d4r0vNLLmfnx0v6YzOdpjZDLWGlbf6eX1gxCOsAIhaZjbV\nzKb3cEyKQk9XlqT1YbvKvNcsde0RhYLOKIXGvnT3OaO62f3vXdyN9O/e6zbn3PpO9gPoBcIKgGh2\nvKT3zewxM7vSzJoHxMrMUs1smULzrkz1NocPrt3kvR5nZqd1dnHn3FG1Bopvmdm9ZtbSlWRmKWZ2\njpndI+nVzq4hqUqh26l/ZWZjvPOyvFlxm8fQfKeXPy+ATjApHICoZWYflPRsu83VCnX3ZIZta5T0\nn86577c7f7WkJd5qsUJPbpakq5xzr4Udd5Okm9XaZVTpfUZW2LbdzrmpYecsVWhSuD2SblfrdPsl\nXtma/zN4F9PtAwNDWAEQ1byWjmWSzpQ0T6FnBCUoFDx2SnpJ0n3OuU2dnDtaoRBycdh5knSOc+7F\ndsfOl/RlSedImiApVqFBsRslrZT0B+fc/2/Xjm0gBGIoCroUWqADaqIWWqAhaqAPXwABIrwL+OJm\n4g1Wjp4s75f3U52x0t3DueWZq2qs445lq6qlu9efhwB/TqwAfOEeK8/+Bt7NzQoAEE2sAADRxAoA\nEE2sAADRHNgCANFsVgCAaGIFAIgmVgCAaGIFAIgmVgCAaB9fPQV5BwGidgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def lsq_gd(A, b):\n", " \"\"\"Approximate least squares with gradient descent\"\"\"\n", " gradient = lambda x: least_squares_gradient(A, b, x)\n", " return gradient_descent(b, 1000*np.ones(10), gradient)[-1]\n", "\n", "ws_newton_lsq_gd = newton_method_lsq(init, np.ones(500), gradient, hessian, lsq_gd)\n", "error_plot([log_likelihood(w, data, labels) for w in ws_newton_lsq_gd])\n", "plt.plot(range(len(ws_newton_lsq_gd)), [baseline]*len(ws_newton_lsq_gd),\n", " label='sklearn', **kwargs)\n", "_ = plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It looks like we didn't gain much compared with plain gradient descent. The number of iterations to reach the baseline got smaller, but each iteration is now costlier. Nonetheless, this idea gives rise to set of interesting heuristics." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## BFGS" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def bfgs(init, grad, hessian, line_search, num_steps, num_to_keep=None):\n", " \"\"\"BFGS algorithm (toy implementation).\n", " \n", " Parameters\n", " ----------\n", " initial : array\n", " starting point\n", " grad : function\n", " mapping arrays to arrays of same shape\n", " hessian : function\n", " mapping 1d arrays to 2d arrays\n", " line_search : function\n", " method to determine step size\n", " num_steps : number\n", " number of iterations\n", " num_to_keep : integer, optional\n", " number of points to keep\n", " \n", " Returns\n", " -------\n", " List of points computed by Newton iteration. Length of the\n", " list is determined by `num_to_keep`.\n", " \"\"\"\n", " xs = [init]\n", " I = np.eye(len(init))\n", " Hs = [hessian(init)]\n", " Hinvs = [I]\n", " \n", " for _ in range(num_steps):\n", " # update point\n", " direction = Hinvs[-1].dot(grad(xs[-1]))\n", " step = line_search(xs[-1], direction)\n", " xs.append(xs[-1] + step * direction)\n", " \n", " s = xs[-1] - xs[-2]\n", " y = grad(xs[-1]) - grad(xs[-2])\n", " ys = np.dot(y, s)\n", " ssT = np.outer(s, s)\n", " \n", " # udpate H\n", " H = Hs[-1] + np.outer(y, y)/ys\n", " H -= Hs[-1].dot(ssT.dot(Hs[-1]))/np.dot(s, Hs[-1].dot(s))\n", " Hs.append(H)\n", " \n", " # update Hinv\n", " left = I - np.outer(s, y)/ys\n", " right = I - np.outer(y, s)/ys\n", " Hinvs.append(left.dot(Hinvs[-1]).dot(right) + ssT/ys)\n", " \n", " if num_to_keep:\n", " xs = xs[-num_to_keep:]\n", " \n", " return xs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's first try this out with a constant step size." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAisAAAH/CAYAAACW6Z2MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd0VNXaBvBnZ9I7SSChhRo6hBJ6EQuISmgWQLpCkKb3\n2gvXz3JVxK70ohRpgiBFFL2AKJ2EIi1UCRBaQnpCyszs74+EkwwkpE2ypzy/tWaF8845My8uLjz3\nnF2ElBJERERElspBdQNERERE98KwQkRERBaNYYWIiIgsGsMKERERWTSGFSIiIrJoDCtERERk0RhW\niIiIyKIxrBAREZFFY1ghIiIii+aougHKFRAQIOvWrau6DSIiokoTFRUVL6WsWtx5DCsWom7duoiM\njFTdBhERUaURQsSU5Dw+BiIiIiKLxrBCREREFo1hhYiIiCwaw4piQohwIcS85ORk1a0QERFZJIYV\nxaSUG6WUET4+PqpbISIiskgMK0RERGTRGFaIiIjIojGsEBERkUVjWCEiIiKLxrBCREREFo3L7Ssm\nhAgHEN6wYUPVrRCRhZFSIjU1FSkpKcjIyIDBYFDdEpEJnU4Hd3d3eHt7w8vLC0KICvkeIaWskA+m\n0gkLC5PcG4iIbpNS4saNG0hPT4efnx88PT2h0+kq7B8DotKSUsJgMCAtLQ0JCQnw8PBAtWrVSvVn\nVAgRJaUMK+483lkhIrJAqampSE9PR506daDT6VS3Q3QXIQQcHR3h6+sLLy8vxMTEIDU1Fd7e3mb/\nLo5ZISKyQCkpKfDz82NQIaug0+ng5+eHlJSUCvl8hhUiIguUkZEBT09P1W0QlZinpycyMjIq5LMZ\nVmzQn6fjcPRyMjgeich6GQwG3lUhq6LT6SpsEDjHrNgYKSXe2Xgc5+PSUdffHX1b1UB4aA00DvJS\n3RoRlRIH05I1qcg/r7yzopi5d10+cTUF5+PSAQAXbmZgxvazePjLP9H7ix1YdeAi77YQEZHVYVhR\nzNy7Lu85d7PQ+unraXjtx6OYs+O8Wb6HiIiosjCs2Jix3evj93/3wPMPhqB+VY+73v/412j8euyq\ngs6IiIjKhmHFBoUEeuHFXo2w9cX78PPz3TC8U7DJ+/9adRhHL5vnsRMRkTVbtGgRhBDo2bNnqa57\n5513IITA6NGjK6QvMsWwYsOEEGhewwfv92+BEZ3qaPXMHCOeXXwAV5NvKeyOiIioZBhW7IAQAv8X\n3gzdQwK02o3ULDy7KBLpWXqFnRERERWPYcVOOOocMHNYW4RUy19k6sTVFDz3fRSy9UaFnREREd0b\nw4od8XZ1wsJR7eHn4azV/joTj1fWHIHRyCnNRERkmRhW7EywvzsWjgqDm1P+ypjrD1/Bf38+yTVY\niMgmZGdn46uvvkKXLl3g6+sLJycnBAYGIjQ0FJMmTcKePXtK/FlLly6Fk5MTHBwcMHv27FL1sXPn\nTgwZMgS1atWCi4sL/P398dBDD2HFihVF/n178OBBvP766+jWrRuCg4O163r27IkFCxYUuUJswQG/\nRqMRM2bMQIcOHeDr6wshBA4fPgwAGD16NIQQeOedd2AwGPDll18iNDQU7u7u8PPzQ9++fREZGVmq\n32dl4Aq2igkhwgGEN2zYsNK+s01wFcwa3hbjFkdCn3dH5dtd/6Cqlwsm9GxQaX0QEZmbXq9H7969\nsWPHDgC5Y/Z8fHxw8+ZN3LhxA3///Tdu3ryJzp07F/tZM2fOxJQpU6DT6bBkyRIMHz68xH289tpr\nmD59unbs7e2NxMREbN26FVu3bsWGDRuwbNkyODiY3jPo3bs3bt7MXS/L3d0d7u7uSEhIwI4dO7Bj\nxw6sW7cO69evh6Nj4f98SykxaNAgrF+/HjqdDl5eha9ertfr8dhjj2HLli1wcnKCi4sLEhMT8fPP\nP2Pr1q3Ytm1bif4bVRbeWVHM3IvCldT9jath+hOtTGof/xqN5fsuVmofRETmtHz5cuzYsQPu7u5Y\nunQpMjIykJiYiKysLMTExGDGjBkIDQ0t9nM++OADTJ48Gc7OzlizZk2pgspXX32F6dOnIzAwEPPm\nzUNSUhKSk5ORnp6OlStXIigoCCtXrsTHH39817W9e/fGihUrcPXqVaSnpyMxMRFpaWlYunQpgoKC\nsHnzZnzxxRdFfvfatWvx66+/YtasWUhJSUFiYiKuX7+O+vXrm5w3c+ZMHDhwAKtWrUJaWhpSU1Nx\n5MgRtGjRApmZmXjhhRdK/PutDLyzYscGta2Fm2nZ+GDzSa321k9H4erkgEFtaynsjIiK9cvrwLWj\nqrswj6CWwCPTzPJRe/fuBQCMHDnSJGDodDoEBwdj0qRJxX7GK6+8gk8//RQeHh5Yv349HnzwwRJ/\nf1JSEqZOnQpXV1ds2bLFJBi5ublh8ODBCA4ORteuXfHJJ5/gpZdegrNz/jjC5cuX3/WZHh4eGD58\nOOrUqYMePXpg1qxZeOWVVwr9/rS0NMydOxcRERFarVq1aoX2+ddff6Fbt25arVWrVli0aBHCwsJw\n4MABXLx4EcHBwXddqwLDip0b16M+4tOzMDdvGX4pgZdXH4GLow6PtaquuDsiKtK1o0DMTtVdWBxv\nb28AwNWrpV+p22g0Yvz48ViwYAF8fX2xefPmUj8K+fHHH5GWloa+ffsWeQenc+fOqFevHs6fP4+o\nqKgSf0f37t3h6+uLCxcu4MqVK6hRo8Zd5/j7++OZZ54p0WcVDCq3tWvXDrVq1cLly5dx7Ngxiwkr\nfAxEeL1PE5NF44wSeGHlIfzvxHWFXRERld4jjzwCAFi/fj369euHtWvXamNA7iUnJwdDhw7FggUL\nUK1aNfzxxx9lGrOxe/duAMC2bdsQFBRU5OvSpUsAoP0saPXq1RgwYACCg4Ph5uYGIYT2SkpKAgBc\nuXKl0O8PCwsrcjxLQe3bty/yvZo1awIAEhMTi/2cysI7KwQhBN7t1xyZOQasjroMANAbJSYuO4h5\nI9uhZ+O7byESkWJBLVV3YD5m/L3cd999eO+99/Dee+9h48aN2LhxIwCgSZMmeOyxxzB+/HiEhITc\ndd3tkAEAa9asKdG4lsLcvqOTkZGBjIyMYs8veI5er8dTTz2FdevWaTUXFxcEBARAp8udwRkXFwej\n0Yj09PRCP69q1aol6rOogbcA4OrqCiA3wFkKhhUCADg4CEx7vBUy9UZsPJKb2LMNRkQsjcLcEe1w\nPwMLkWUx0xgPW/Sf//wHw4cPx6pVq/DHH39gz549iI6ORnR0NL766issXLgQI0eONLmmZcuWMBgM\nOHHiBCZOnIjt27cjICCgiG8omtGYu8jmCy+8gC+//LJU186fPx/r1q2Du7s7pk2bhoEDB6JWLdPx\ng7Vr18bly5eLnPp8O9TYGj4GIo3OQeDzp0LxcPNArZatN2L8kihsj76hsDMiotKpV68eXn/9dfz6\n669ISEjA9u3b0aNHD+j1ekycOBE3bpj+nebn54etW7eicePGOHbsGHr16lWmxyCBgbl/f168WPqZ\nlatXrwaQG7amTJlyV1AxGAyIj48v9efaAoYVMuGkc8A3Q9uid7MCgcVgxPilUdgWzTEsRGR9dDod\nevbsiU2bNsHJyQnp6emFLnwWFBSErVu3okGDBjh8+DB69+6N5OTS7VB/e5zLH3/8gVu3SrdZ7OXL\nuY/h27RpU+j7u3btQmZmZqk+01YwrNBdnB1z9xEyucOSF1h+O35NYWdERPeWnZ1d5HvOzs7aY5Ks\nrKxCz6lZsya2bduGunXrIjIyEo888gjS0tJK/P1PPvkkPDw8kJiYiPfee++e59555+b2eltHj949\nJV2v12Pq1Kkl7sPWMKwoJoQIF0LMK216r2hOOgfMeLotHmkRpNVyDBITlh3UxrQQEVmakSNHYsyY\nMdiyZQtSU1O1+oULFzBq1ChkZmbCzc0N3bt3L/IzgoODsW3bNtSqVQt79uzBY489VqLBskDu1OGP\nPvoIADBt2jSMGzcOp0+f1t7PyMjA9u3bERERgS5duphc26tXLwDA+++/j/Xr12tL60dHRyM8PBz7\n9++Hh4dHyf5D2BiGFcVUrWBbEk46B3w9tA0ebZkfWAxGiRdWHsIPkXdPtyMiUi0zMxOLFi1Cnz59\n4OPjgypVqsDDwwP16tXDqlWroNPpMHfu3GIHz9arVw/btm1D9erV8eeff6Jfv34lfgQzZcoUvP/+\n+xBCYMGCBWjcuDE8PT3h5+cHT09PPPDAA5g/f/5dn/fyyy+jQYMGSElJwYABA+Dm5gYfHx80bdoU\nv//+O+bMmVOmQb+2gGGF7slJ54Cvh7TBgNb5iw8ZJfDqmr+xZM8FZX0RERVm2rRpmD59Ovr06YP6\n9esjOzsbBoMBDRo0wJgxY3Dw4EGMGDGiRJ8VEhKCbdu2ITAwEFu3bsXAgQOLfHx0p6lTp+LIkSOI\niIhASEiINt24Ro0aePjhhzF9+nT89ddfJtf4+flh7969mDBhgja41s3NDQMGDMCOHTswevToUv23\nsCWCO+1ahrCwMGmJO13eZjBKTP3pKFbsN72j8mqfxpjYs/I2YSSyFydPnkTTpk1Vt0FUKqX9cyuE\niJJShhV3Hu+sUInoHAQ+HNgSY7rWNalP//UUPvrlZJFz/omIiMqLYYVKTAiBt/s2w6T7G5jU5+44\njzfWHoXByMBCRETmx7BCpSKEwCsPN8EbjzQxqa88cAlTVhxElt6gqDMiIrJVDCtUJuPva4Bpg1rC\nQeTXNh+9hmcXRSItS6+uMSIisjkMK1RmQzoEY8bTbeGky08sO8/GY8i8PYhPK9mIeSIiouIwrFC5\nPNqyOr4d3R7uzvmbZx2LTcETs3fj4s2SLaJERER0LwwrVG7dQ6pixbhO8PNw1moXbmZg0OzdOBZr\nWSvzEhGR9WFYIbMIre2LNc91Rq0qblotPi0LQ+btxV9n4hR2RkRE1o5hRTFL3RuoLOpX9cSPE7qg\nSZCXVkvL0mPMdwew9uBlhZ0RWSeuX0TWpCL/vDKsKGbJewOVRaC3K1aN74xO9f20mt4o8eIPRzDr\nj7P8y5eohHQ6nbaRHZE1MBgM2q7W5sawQmbn4+aExc90QN9W1U3q0389hbfXH4feYFTUGZH1cHd3\nR1pamuo2iEosLS0N7u7uFfLZDCtUIVwcdfh6SBuM617PpL50bwzGL41CRjbXYiG6F29vbyQkJPDu\nClkFg8GAhIQEeHt7V8jnM6xQhXFwEHjrsWb4T99mEAUWj9safQOD5+7FjdSSbbdOZI+8vLzg4eGB\nmJgYJCUlQa/X8zEqWRQpJfR6PZKSkhATEwMPDw94eXkVf2EZcNdlC2Hpuy6X1+ajV/GvVYeRrc9/\nBFTT1w2Ln2mPhtUq5g83kbWTUiI1NRUpKSnIyMjgXRayODqdDu7u7vD29oaXlxdEwf9nWgIl3XWZ\nYcVC2HpYAYComASMXRyJxIwcrebt6og5I9qhS4MAhZ0REZEKJQ0rfAxElaZdHT+sndgVdfzzB2Cl\nZOoxcuF+rIni1GYiIiocwwpVqnoBHlg7oQvaBPtqNb1R4uXVR/D5b6f4TJ6IiO7CsEKVzt/TBSvG\ndcJjLU2nNn+97Sz+teowMnP4XJ6IiPIxrJASrk46fDO0DZ67r4FJff3hKxi+YB9uctdmIiLKw7BC\nyjg4CLz+SBN8NKgldA75I8gjYxIxcNZunL3BBbGIiIhhhSzA0A7BWDSmPbxcHLXaxYQMDJq1C7vO\nxivsjIiILAHDClmE7iFVsXZiF5Ndm1My9Rj17X6s2H9RYWdERKQawwpZjJBAL/w0qetdM4XeWHsU\n/910AgYjZwoREdkjhhWyKAF5M4XCQ2uY1Bfs/AcRSyKRlsU9hYiI7A3DimJCiHAhxLzk5GTVrVgM\nVycdvh7SGi88GGJS3xp9A0/M3o3LiRmKOiMiIhUYVhSTUm6UUkb4+PiobsWiCCHw716N8NWQ1nB2\nzP9jGn0tFQNm7kJUTKLC7oiIqDIxrJBF69+6JlaM64QAT2etFp+WjaHz9+KnQ7EKOyMiosrCsEIW\nr12dKvhpUlc0CcrfnTlbb8S/Vh3Gp1tOwciBt0RENo1hhaxCrSruWDOhCx5qWs2kPmP7WUxYFoV0\nDrwlIrJZDCtkNTxdHDF3RBjG96hvUt9y/DqemLMHsUm3FHVGREQViWGFrIrOQeCNR5vikydawUmX\nv0T/yasp6D9jJwfeEhHZIIYVskpPhtXGinGd4O9xx8DbeXuxJuqyws6IiMjcGFbIaoXV9bt74K3B\niJdXH8GHm09yxVsiIhvBsEJWrbZf7sDbXs0CTerz/jyPsYsPIDUzR1FnRERkLgwrZPU8XRwxd3g7\nTOzZwKS+/VQcBs7ajQvx6Yo6IyIic2BYIZvg4CDwap8md614e/ZGGvrP3IVdZ+MVdkdEROXBsEI2\npX/rmlg9vjOqebloteRbORj57X4s2vUPpOQ4FiIia8OwQjYntLYvNkzuhtBa+fstGYwS72w8gTfW\nHkW23qiwOyIiKi2GFbJJQT6uWDW+Mwa0rmFSX3ngEp6evxdxqVmKOiMiotJiWCGb5eqkwxeDW+P1\nR5pA5K8fh8iYRPSbsRPHYpPVNUdERCXGsEI2TQiB5+5rgIWjwuDl4qjVryZn4ok5u7HhyBWF3RER\nUUkwrJBdeKBJINZN6oK6/u5aLTPHiOdXHMLHv0ZzATkiIgvGsEJ2o2E1L6yf1A3dQwJM6rP/OIex\niw8ghQvIERFZJIYVsis+7k74bnR7jOtez6S+/VQcBszchXNxaYo6IyKiojCskN1x1Dngrcea4fOn\nQk0WkDsfl44BM3ZhW/R1hd0REdGdGFbIbg1qWwurx3dGkLerVkvN0uPZxZGYuf0sF5AjIrIQDCtk\n10Jr+2LDlK5oV6eKVpMS+GTLKUxefggZ2XqF3REREcCwopwQIlwIMS85mWt+qFLNyxXLx3XEkPa1\nTeo/H72KQbN241JChqLOiIgIYFhRTkq5UUoZ4ePjU/zJVGFcHHX4aFBLvN+/ORwd8leQi76WivAZ\nO7kRIhGRQgwrRHmEEBjRuS6Wje0Ifw9nrZ6UkYMRC/dhwV/nOY6FiEgBhhWiO3Ss748NU7qhRU1v\nrWaUwH9/PokXfziCzByDwu6IiOwPwwpRIWr6umHNc10wsE1Nk/q6Q7F4Ys5uxCbdUtQZEZH9YVgh\nKoKrkw6fPxWKqY81RYFhLDgWm4Lwb3Ziz7mb6pojIrIjDCtE9yCEwNju9bHkmY7wdXfS6gnp2Ri+\ncB++3fkPx7EQEVUwhhWiEugWEoCNk7uhafX8cSwGo8R7m07gJY5jISKqUAwrRCVU288dayd0Qb/Q\nGib1tYdi8fhsrsdCRFRRGFaISsHNWYevhrTGW4+ajmM5fiUF/bgeCxFRhWBYISolIQTG9cgdx1Kl\nwDiWxLz1WOb9eY7jWIiIzIhhhaiMuoUEYMPkbmhW3XQ9lg83R2PKCu4rRERkLgwrROVQ288dP064\nez2WTX/n7isUczNdUWdERLaDYYWonNycc9djebtvM+ju3Ffom53YHn1DYXdERNaPYYXIDIQQeKZb\nPXz/rOm+QimZejyz+AC+3noGRiPHsRARlQXDCpEZdW7gj41TuiG0Vv4u2lICn/9+GhFLI5GSmaOw\nOyIi68SwQmRmNXzdsGp8ZwwOq21S/9/JG+g/YxdOX09V1BkRkXViWCGqAK5OOnz8RCt8OLAlnHT5\n41j+iU/HgJm7sOnvKwq7IyKyLgwrRBXo6Y7B+GF8ZwR5u2q1jGwDJi8/hA9+PgG9waiwOyIi68Cw\nQlTB2gRXwcYp3dCxnp9Jff5f/2D4wn2IT8tS1BkRkXVgWCGqBFW9XLBsbEeM7VbPpL73fALCv9mJ\nQxcTFXVGRGT5GFaIKomjzgFT+zbDN0PbwN1Zp9WvJmdi8Ny9WLYvhsv0ExEVgmGFqJKFh9bAT5O6\nol6Ah1bLNhjx1rpjeHXN38jMMSjsjojI8jCsECnQKNAL6yd3Ra9mgSb11VGX8cSc3biUkKGoMyIi\ny8OwQqSIt6sT5g5vh1cebgyRP7sZx2JTED5jJ/44xWX6iYgAhhUipRwcBCbd3xCLx3SAr7uTVk/K\nyMGYRVymn4gIYFgxCyGEpxDiXSHEZiFEnBBCCiFeV90XWY8ejapi4+RuaFnz7mX6n118AMkZXKaf\niOwXw4p5BAB4G0BLAIcU90JWqrafO1Y/1xlPhdUyqW8/FYfwGTtx/Eqyos6IiNRiWDGPqwBqSilr\nA4hQ3QxZL1cnHaY/EYppg1rCWZf/P8+LCRkYNGs31kRdVtgdEZEaDCtmIKXMklJysxcymyEdgrH6\nuc6o6eum1bL0Rry8+gjeXHcUWXpObyYi+8GwQmShQmv7YuOUbugeEmBSX77vIp6aswexSbcUdUZE\nVLksNqwIIbyEEP2EEO8LIX4RQsTnDVyVQogmJfyMICHEV0KIc0KITCHEdSHERiHEgxXdP5E5+Hk4\nY9GYDpjyQEOT+pHLyej79V/483Scos6IiCqPxYYVAA8CWA9gKoA+APxLc7EQohWAYwCeB1AfQBZy\nB8L2BfB7YbN1RC7XEr4s+b8d2RCdg8BLvRtj4agweLs6avXEjByM+m4/vuH0ZiKycZb+D+4NAJsB\nvItSDFwVQrgB2IDcgHMIQAsppQ+AKgA+AyAAfCiE6H3HpV0B3Crhq0eZf1dEZfBg00BsmtIdTat7\nazUpgc9+P42xSyI5vZmIbJZj8acos1FK+dPtAyFE3VJcOx5AHQBpAMKllLEAIKVMAfCyEKIBgAEA\nPgLwW4HrTgMYU8LviC5FP0RmEezvjnUTu+Ctdcfw48H8mUHbom+g74y/MHtYO7QosFYLEZEtsNiw\nIqUsz3SHYXk/l98OKnf4BLlhpa0QorGU8lTed94AsKgc30tU4VyddPj0yVZoV6cK3tlwHNkGIwDg\nUsItPD57N94f0AJPhdVW3CURkflY+mOgUhNCeAFol3e4pYjT9gK4vcIWB9uS1RFC4OmOhU9vfnXN\n33hjLXdvJiLbYXNhBUBT5I5JAYDjhZ0gpTQCOJV32MwcXyqEmCyEmApgcl7pfiHE1LwX78tThQit\n7YtNU7qhR6OqJvUV+y9x92Yishm2GFaqF/j1vRZqu/1e9XucUxovA3gfwEt5x73zjt9H7sBeogpR\nxcMZ341uj+cfDDGpH4tNQd9vdmJ7NHdvJiLrZothxaPAr++1atbt/8vpaY4vlVLWlVKKIl4XCrtG\nCBEhhIgUQkTGxXG9DCo7nYPAi70a4bvR7eHjlr97c/KtHDyz+AA+//00DJzeTERWyhbDitWQUs6T\nUoZJKcOqVq1a/AVExbi/STVsmtINLWqaTm/+eusZjP5uPxLSsxV2R0RUNrYYVtIL/NqtyLMA97yf\naRXYC1Glq+3njjXPdcGQ9qYzgv46E4/wb3biyKUkRZ0REZWNLYaVguNUatzjvNvvXa3AXoiUcHXS\nYdrjrTD98VZwccz/n3ls0i08OWcPlu2LgZR8LERE1sEWw0o0gNt/Czcv7IS8pfIb5x2eqIymiFR4\nqn1t/DihC4L93LVatsGIt9Ydw0urj+BWNqc3E5Hls7mwIqVMBRCZd9iriNM6Arg9nXhrhTdFpFCL\nmj7YOLkbHmpazaS+9mAsBs7ahX/i04u4kojIMthcWMmzPO/nMCFEYVOTX877GXV79VpVhBDhQoh5\nycnJxZ9MVEY+7k6YNyIMrzzcGA4ivx59LRX9vtmJ345fU9ccEVExLDqsCCECbr9gulaJb8H3CtkB\neS6AGABeADYJIZrlfZ6XEGI6gEF5571Z0b+H4kgpN0opI3x8uG4cVSwHB4FJ9zfEkmc6ws/DWaun\nZukRsTQK036Jhj5v6X4iIksiLHmQnRCipM3Vu3MtEyFEKHIf8fjnlVKQu6aKA3LHtLwppZxmplbL\nLSwsTEZGRhZ/IpEZXE2+hYnLDuLQRdOZQZ3r++ProW1Q1ctFUWdEZE+EEFFSyrDizrPoOyvlIaU8\nAqAFgK8BnAfgAuAmgJ8B9LKkoEJU2ar7uGFVRGeM6lzHpL7n/E30/eYvRMUkKOqMiOhuFn1nxZ7w\nzgqpsv5wLF7/8ShuFdj40NFBYOpjTTGqS10IIe5xNRFR2dn9nRUiKpn+rWti/eSuqB+Qv1OF3ijx\nzsYTeGHlYaRn6RV2R0TEsKIcZwORJWgU6IX1k7uiT/Mgk/qGI1cwYOYunIvjQs9EpA7DimKcDUSW\nwsvVCbOHt8WbjzaBrsD85jM30tB/xi78cpSLPRORGgwrRKQRQiCiRwMsG9sRAZ7505vTsvSYsOwg\nPtp8ktObiajSMawQ0V061ffHz893R7s6VUzqc/88j+EL9yEuNUtRZ0RkjxhWiKhQgd6uWDGuE0Z3\nqWtS33s+IW96c6KaxojI7jCsEFGRnB0d8E6/5vh6aBu4O+u0+vWULAyZtweLd1/g7s1EVOEYVoio\nWP1Ca+CnSabTm3MMEv+34Tj+veowMrI5vZmIKg7DChGVyO3pzQ83DzSp/3T4CgbN2o0L3L2ZiCoI\nw4piXGeFrImXqxPmDG+HNx5pctfuzeEzduJ/J66ra46IbBbDimJcZ4WsjRAC4+9rgO+f7Qj/grs3\nZ+oxdkkkPvvtFAxGjmMhIvNhWCGiMunSMAAbp3RDaG1fk/o3287imUUHkJSRragzIrI1DCtEVGY1\nfN3ww/hOGNYx2KS+43QcwmfsxLFYPt4kovJjWCGicnFx1OGDgS3xyROt4OKY/1fKpYRbeHz2bvwY\ndVlhd0RkCxhWiMgsngyrjR8ndEGtKm5aLUtvxEurj+A/Px1Dtp7L9BNR2TCsEJHZtKjpg42Tu6F7\nSIBJfeneGAydvxfXUzIVdUZE1oxhhYjMqoqHMxaN6YDJ9zc0qUfFJKLvNztx4EKCos6IyFoxrCjG\ndVbIFukcBF5+uDHmjWgHTxdHrR6XmoWh8/ZymX4iKhWGFcW4zgrZst7Ng7B+clc0rOap1fTG3GX6\nX1p9BJk5BoXdEZG1YFghogrVoKonfprUFY+2DDKprz0Yiyfm7MblxAxFnRGRtWBYIaIK5+niiJlP\nt8VrfUyqdFZrAAAgAElEQVSX6T8Wm4J+M3Zh99l4dc0RkcVjWCGiSiGEwISeDbBoTAf4ujtp9YT0\nbAxfuA8L/jrPcSxEVCiGFSKqVD0aVcXGyd3QtLq3VjNK4L8/n8S/Vx3GrWyOYyEiU47Fn0JW55fX\ngWtHVXdBVKTaADZ5SZzPTMPN9AJ7CJ0AzpzVoVGgF1wddcr6I6IiBLUEHplW6V/LsGKLrh0FYnaq\n7oLonnQAQgCE3Hl/Vw8gtvL7ISLLxcdAREREZNF4Z8UWBbVU3QFRqWXpDTh1PRUZd4xZCfB0Qf0A\nDzgIUcSVRFRpFP37wrCimBAiHEB4w4YNiz23xBQ8TyQqLxcAIdkGvL72b6w/fCX/jQSgpZsP5o5o\nhxq+bkVeT0S2i4+BFOMKtkT53Jx1+HJwa7z1aFOT9ViOxiaj3wzuK0RkrxhWiMiiCCEwrkd9LH6m\nA3zc8tdjiU/LxtPz92L5vosKuyMiFRhWiMgidQ/JXY+lcaCXVssxSLy57ijeWncU2Xqjwu6IqDIx\nrBCRxQr2d8faiV3wSAvTfYWW7buI4Qv34WZalqLOiKgyMawQkUXzyNtX6KVejUzq+/9JQL8Zu3Di\nSoqizoiosjCsEJHFc3AQmPJgCOaNaAcP5/yVbWOTbuHx2bux+ehVhd0RUUVjWCEiq9G7eRDWTeqK\nYD93rXYrx4CJyw7ii99Pw2jkRohEtohhhYisSqNAL6yf1BVdG/qb1L/aegaTlh9ERrZeUWdEVFEY\nVojI6lTxcMbiMR0wuktdk/ovx67h8dl7cDkxQ01jRFQhGFaIyCo56hzwTr/mmDaoJZx0+SvInbya\ngv4zdiGSC8gR2QyGFSKyakM6BGPZ2E7w93DWajfTszF0/l6sjryksDMiMheGFSKyeh3q+WH95K5o\nWt1bq+UYJF5Z8zc++PkEDBx4S2TVGFYUE0KECyHmJScnq26FyKrVquKONc91xsPNA03q8//6B2MX\nH0BqZo6izoiovBhWFONGhkTm4+HiiNnD2mHKA6a7mG8/FYfHZ+/GpQQOvCWyRgwrRGRTHBwEXurd\nGF8NaQ0Xx/y/4k5fT0P/mbu4czORFWJYISKb1L91TayM6ISqXi5aLSE9d+fmNVGXFXZGRKXFsEJE\nNqtNcBWsn9QVze4YePvy6iP4+NdornhLZCUYVojIptXwdcOaCZ3Rp7npzs2z/ziHicu44i2RNWBY\nISKb5+7siFnD2mJizwYm9V+PX8PguXtxPSVTUWdEVBIMK0RkFxwcBF7t0wSfPRlqsuLt0dhk9J+x\nC8evcPkAIkvFsEJEduXxdrWwbGwnVHF30mrXUjLx5Jw92HryusLOiKgoDCtEZHc61PPDT5O6okFV\nD62WkW3AuCWR+G7XPwo7I6LCMKwQkV2q4++BtRO6oksDf61mlMC7G0/g7fXHoDcYFXZHRAUxrBCR\n3fJxd8LiZzpgcFhtk/qSPTGIWBqF9CzOFCKyBAwrRGTXnHQOmPZ4S7zWp4lJfVv0DTw5Zw+uJXOm\nEJFqZQ4rQojn8141zNkQEVFlE0JgQs8GmDWsrckS/SeupmDAzF04cSVFYXdEVJ47K18A+BRAvJl6\nISJS6tGW1bEiohP8PZy1Wu5Mod3449QNhZ0R2bfyhJV4AKlSymxzNUNEpFrb4CpYN9F0plB6tgHP\nLo7Eyv0XFXZGZL/KE1YOAvARQlQ1VzP2SAgRLoSYl5zMBamILEWwvzvWTuiKTvX9tJrBKPH62qP4\ndMspSMk9hYgqU3nCytd51//HTL3YJSnlRillhI+Pj+pWiKiA2zOFBrQ2HZY3Y/tZvPjDEWTrObWZ\nqLKUOaxIKX8B8DKA54QQS4UQoeZri4hIPRdHHb4Y3BpTHmhoUl93KBajv9uPlMwcRZ0R2RdR1tuZ\nQojzeb8MAuCS9+tbAG4CMBRxmZRSNijiPbsWFhYmIyMjVbdBREVYdeAi3lx3DAZj/t+ZTYK88N2Y\n9qju46awMyLrJYSIklKGFXdeeR4D1c17uQIQeS93ALULvFfYi4jI6gxuH4yFo8Lg4azTatHXUjFw\n5m5EX+PUZqKKVJ47K/eV5Top5Y4yfaGN450VIutwLDYZYxYdQFxqllbzcnHE3JHt0KVBgMLOiKxP\nSe+slDmskHkxrBBZj0sJGRj13X6cj0vXas46B3z2VCjCQ7lOJlFJVcZjICIiu1Tbzx1rJ3RB+7pV\ntFq2wYgpKw5h4U7u2kxkbmYLKyJXEyFE97xXEyGEMNfnExFZEl93Zyx9tiP6NA8yqb+/6QQ+3HwS\nRiPvWhOZS7nDihCioRBiEYBkAMcB/JH3Og4gWQjxnRCiYZEfQERkpVyddJg5rC1GdKpjUp/353m8\ntPoIcgxci4XIHMoVVoQQ/QAcAjACgCfyZwXdfnkCGAngkBCib/laJSKyPDoHgff6N8crDzc2qa87\nFIuxiyORka1X1BmR7SjPrssNAKwE4AHgPIDxAEIAuOW9QgA8B+Bc3jk/5F1DRGRThBCYdH9DfPJE\nK+gc8p9+7zgdh6Hz9yEhnVuoEZVHee6svIrcNVa2A2glpZwvpTwnpczKe52TUs4DEApgB3IXjnul\n/C0TEVmmJ8NqY/7IdnB1yv+r9cilJDwxZzcuJ2Yo7IzIupUnrPQCIAGMl1LeKuqkvPfGI/exUO9y\nfB8RkcV7oEkglo3tBB83J612Pi4dT8zegzPXUxV2RmS9yhNWqgNIllKeLe5EKeVpAEl51xAR2bR2\ndapgzXOdUd3HVatdS8nEk3P34NDFRIWdEVmn8oSVDADuQgin4k4UQjgjd9xKkXdgiIhsSUigF36c\n0AUNq3lqtaSMHAxbsA9/no5T2BmR9SlPWDkKwAnAqBKcOyrv3L/L8X1ERFalhq8bVo/vjNDavlot\nI9uAZxcfwKa/ryjsjMi6lCesLEXuOJSvhRBjC1sATgjhKoR4HsDXyB3fsrgc30dEZHWqeDhj+diO\n6B6Sv29QjkFiyopDWLH/osLOiKxHeTYyFAB+Rf5A22sA/gIQi9xZQsEAOgLwR26o+Q3AI5KbERWK\newMR2bZsvREv/nAYm/6+alJ//ZEmeO4+rupA9qmkewM5lvULpJRSCDEAwBcAxiJ38OxTyA0uQG5A\nAQAjgLkAXmJQISJ75ezogK+GtIG3mxOW78u/ozLtl2gkZeTgtT6NwR1KiApX5rACaNOSnxNCfARg\nIIC2AKrmvR0H4CCAtVJK3uskIruncxD4YEAL+Lg5YfYf57T6nB3nkJqZg/f7t4CDAwML0Z3KFVZu\nk1LGAPjSHJ9FRGTLhBB4rU8TeLs64eNfo7X6sn0XkZFtwCdPtIKjzmx7zBLZhPIst39QCBElhKhv\nzobsjRAiXAgxLzk5WXUrRFSJJvRsgA8HtkTBJz/rDsVi4rKDyNIb1DVGZIHKE9+bAQiRUp43VzP2\nSEq5UUoZ4ePjo7oVIqpkT3cMxpeDW5vsJ/TbievcAJHoDuUJK7HIH0RLRERl0L91TcwZ3g7Ojvl/\nHf91Jh6jvz2A1MwchZ0RWY7yhJUtyF3BtqO5miEiske9mgXiu9Ht4eak02r7LyRg+ML9SM5gYCEq\nT1j5L4CbAOYIIQKKO5mIiIrWtWEAvh/bEV6u+fMejlxKwtD5e3EzLUthZ0TqlWdRuB4AGgP4DEAO\ngCUA9iB3ynKRo8OklH+W6QttHBeFIyIAOBabjBEL9yGxwB2VhtU8sXxsR1Tzdr3HlUTWp6SLwpUn\nrBhhugBcST5ISinNMl3a1jCsENFtp66lYtiCfYgvcEelXoAHlo/riOo+bgo7IzKvkoaV8jwGuljg\nFXPHcVGvS+X4PiIiu9A4yAurxndCUIE7Kf/Ep2Pw3L24nJihsDMiNcp8Z4XMi3dWiOhOF29mYOj8\nvYhNuqXVavq6YcW4Tgj2d1fYGZF5VMadFSIiqkDB/u5YNb4Tgv3yg0ls0i0MnrcH/8SnK+yMqHKV\nZwXbRCHETa5gS0RUcWpVyQ0s9QI8tNrV5EwMnrsH5+LSFHZGVHnKc2fFGYCOK9gSEVWs6j5uWBXR\nCQ2q5geWG6lZGDpvL87eYGAh21feAbbO5mqEiIiKVs3bFavGd0bjQC+tdiM1C0Pn78XZG6kKOyOq\neOUJKxsAuAghepmrGSIiKlqApwuWj+uIJkH5gSUuNQtD5u3DmesMLGS7yhNWPgRwAcB8IURT87RD\nRET34u/pgmVjTQNLfFoWhs7fxzssZLPKsyjcSADVALyN3MdBv6BkK9guKdMX2jhOXSai0khIz8aw\nBftw8mqKVgvwdMHKiE5oWM1TYWdEJVeZK9je3nm5RB8kpdQVf5b9YVghotJKTM/G03cElqpeuYGl\nQVUGFrJ8JQ0r5Vn6/k+UMKAQEZH5VfFwxrKxHfH0/L2Ivpb7CCgub5bQyohOqM/AQjaCK9haCN5Z\nIaKySkjPNgksABDk7YpV4zuhjr/HPa4kUosr2BIR2Qm/vDssjQLz76RcS8nE0/P3cS8hsgklDitC\niOeFEM8W8Z6nEMK7mOu/EEIsLG2DRERUPH9PFywfZzq4NjbpFobO34urybfucSWR5SvNnZUvAbxX\nxHtnACQUc/0QAKNL8X1ERFQKAZ4uWD62I+oXWJr/UsItPD1/H26kZCrsjKh8SvsYSJTxPSIiqgTV\nvF2xfJzp5of/xKdj6Py9iE/LUtgZUdlxzAoRkY0J8nHF8nEdUdPXTaudi0vHiIX7kZSRrbAzorJh\nWCEiskG1qrhjxbhOqO7jqtVOXk3BqG/3IzUzR2FnRKXHsEJEZKOC/d2xbGxHBHi6aLUjl5PxzKID\nyMjWK+yMqHQYVsxACNFeCDFDCHFcCJEuhLgohPhBCNFIdW9EZN/qV/XEsrEdUcXdSasduJCIiCVR\nyMwpcmcUIovCsGIerwF4HMBWAC8AmAegB4CDQoiWKhsjImoc5IWlz3aEl2v+ouU7z8ZjyopDyDEY\nFXZGVDIMK+bxOYA6UsrnpZQLpJT/BdAdudsZvKm2NSIioEVNHywa0wHuzvnbs/1+4jpeXn0EBiNX\nMifLVtq9gfyEENsKqwNAEe+ZnGOLpJS7C6mdEUIcB9BMQUtERHdpV6cKFowKw+jvDiBbn3tHZf3h\nK3B3dsSHA1tACK5AQZaptGHFGUDPe7x/r/cAO9r4UOT+rz4QwCnVvRAR3dalQQBmD2uL8UujoM+7\no7Ji/0V4uujw5qNNGVjIIpUmrCyusC4KIYTwAnA/gPYAwvJ++ue93VRKGV2CzwgC8AaAvgBqAkgG\nsB/Al1LKrRXRdwHD8r7z3Qr+HiKiUnmwaSC+GNwaz688hNt72c7/6x/4uDlh8gMhapsjKkSJw4qU\nckxFNlKIBwGsK+vFQohWALYhP+CkAAhAbnB5TAjxppRy2h3XCAAuKJlsKWWhI9OEEE0AzASwF8C3\nZWifiKhChYfWQEa2Hq/9eFSrffrbafi4OWFE57rqGiMqhKUPsL0BYDNy705ElPQiIYQbgA3IDSqH\nALSQUvoAqALgM+RuDfChEKL3HZd2BXCrhK8eRXx3EICfkXsX53EpJecGEpFFGtw+GFMfa2pSe3vD\ncaw/HKuoI6LClXbMSmXaKKX86faBEKJuKa4dD6AOgDQA4VLKWACQUqYAeFkI0QDAAAAfAfitwHWn\nAZT0DtJdj6GEED4AfgHgC6C7lPJKKXomIqp0Y7vXR1JGDmZsPwsAkBJ48Ycj8HRxxINNAxV3R5TL\nYsNKOe9IDMv7ufx2ULnDJ8gNK22FEI2llKfyvvMGgEVl+UIhhCuAjQAaAXhISnmiLJ9DRFTZXurd\nCMm3crB0bwwAwGCUmLjsIJY+2xEd6tnsRE6yIpb+GKjU8gbmtss73FLEaXuR+5gGyB0bU97v1AFY\nBaAzgCellHvK+5lERJVFCIF3+zVH/9Y1tFqW3ohnFx/AiSspCjsjymVzYQVAU+SOSQGA44WdkDcw\n9vaUYnOsg/IZgH7IfQTkJ4QYXvBlhs8nIqpQDg4Cnz4ZigeaVNNqqZl6jPpuP2JupivsjMg2w0r1\nAr++15iR2+9Vv8c5JdU672c4gKWFvIiILJ6TzgEzn26L9nWraLW41CyMWLgfN1IyFXZG9s4Ww4pH\ngV/fusd5GXk/Pcv7hVLKnlJKUdSrqOuEEBFCiEghRGRcXFx52yAiKjc3Zx0WjGqPJkFeWu1iQgZG\nfrsfKZk5Cjsje2aLYcVqSCnnSSnDpJRhVatWVd0OEREAwMfNCUue6YDafm5aLfpaKsYtjuROzaSE\nLYaVgg9X3Yo8C3DP+5lWgb0QEVmlat6uWPpMRwR45q+Tue+fBPxr5WFufEiVzhbDSsFxKjWKPCv/\nvasV2AsRkdWqG+CBRWPaw9Mlf5WLX49fw3/WH4OUDCxUeWwxrEQjf8PE5oWdIIRwANA475DroRAR\nFaFFTR/MG9EOzrr8fy6W77uIL/93RmFXZG9sLqxIKVMBROYd9iritI4AfPJ+XdEbGhIRWbUuDQPw\nxeDWKLgh81dbz2D5vovqmiK7YnNhJc/yvJ/DhBCFTU1+Oe9n1O3Va1URQoQLIeYlJycXfzIRkSKP\ntaqO9/qZ3qye+tNR/H7iuqKOyJ5YdFgRQgTcfiF3E8LbfAu+l/dYp6C5AGIAeAHYJIRolvd5XkKI\n6QAG5Z33ZkX/HoojpdwopYzw8fEp/mQiIoVGdK6LKQ801I6NEpiy4iCiYhIVdkX2wKLDCoC4Aq+D\nBep77ngvuOBFUspbAPoDuAmgLYDjQohkAEkAXkHumJY3pJQFNzEkIqJivNirEZ5sV0s7zszJXZb/\nXBwnVlLFsfSwUmZSyiMAWgD4GsB5AC7IDS8/A+glpZymsD0iIqskhMCHg1qiZ+P8taGSMnIwkqvc\nUgWy6LByr1Vh73hdKOL6a1LKF6SUDaSUrlLKalLKvlJKDqolIiojJ50DZg1ri9Ba+Y+vY5NuYcyi\nA0jL0ivsjGyVRYcVIiKyTO7Ojvh2dHvU9XfXasevpGDisoPIMRgVdka2iGGFiIjKxN/TBYuf6QB/\nD2et9ufpOLyx9igXjSOzYlhRjFOXicia1fH3wLej28PNSafV1kRdxhdcNI7MiGFFMU5dJiJrF1rb\nFzOebgOHAovGfb31DFYd4KJxZB4MK0REVG4PNg3E+wNamNTeXHcMO07HKeqIbAnDChERmcWwjnUw\n+f78ReMMRomJ30fh+BU+5qbyYVghIiKzeal3Iwxonb/hfXq2Ac8sOoArSbcUdkXWjmGFiIjMRgiB\nj59ohU71/bTa9ZQsjPnuAFIycxR2RtaMYYWIiMzKxVGHucPD0LCap1Y7dT0VE7/nGixUNgwrinHq\nMhHZIh93Jywa0x5VvVy02s6z8Zi67hjXYKFSY1hRjFOXichW1arijm9Hma7BsiryEmb9cU5hV2SN\nGFaIiKjCtKzlg2+Gmq7B8smWU1h/OFZdU2R1GFaIiKhCPdQsEG/3bWZSe2X13zhwIUFRR2RtGFaI\niKjCje5aD2O61tWOsw1GRCyJxIX4dHVNkdVgWCEiokox9bFm6NUsUDtOzMjBM4sOICkjW2FXZA0Y\nVoiIqFLoHAS+GtIaLWvmTyg4H5+O576PQraeU5qpaAwrRERUadydHbFgVBiq+7hqtb3nE/DG2qOc\n0kxFYlghIqJKFejtim9Ht4eHc/6U5h8PXsbM7WcVdkWWjGFFMS4KR0T2qGl1b8x4uq3JlOZPfzuN\nTX9fUdcUWSyGFcW4KBwR2av7m1TDu/2am9Re+uEIDl1MVNQRWSqGFSIiUmZE57oY3aWudpylN2Lc\nkihcTsxQ1xRZHIYVIiJS6j99m+H+xlW14/i0LIxdHIlU7tJMeRhWiIhIKZ2DwDdPt0WTIC+tFn0t\nFc+vOASDkTOEiGGFiIgsgKeLIxaObo8Az/xdmrefisMHP59U2BVZCoYVIiKyCDV93bBgVBhcHPP/\nafp21z9Yti9GYVdkCRhWiIjIYrSu7YvPngo1qb29/jh2nolX1BFZAoYVIiKyKH1b1cCLvRppxwaj\nxIRlUTh7I01hV6QSwwoREVmcKQ80RP/WNbTj1Ew9xi7mpof2imFFMa5gS0R0NyEEPn68FdoG+2q1\nCzczMOH7g8gxcNNDe8OwohhXsCUiKpyrkw5zR4Shpq+bVttz/ibeXn+cmx7aGYYVIiKyWFW9XLBg\nVBjcC2x6uGL/RSzafUFdU1TpGFaIiMiiNa3uja+GtIEosOnh+5tOYPupG+qaokrFsEJERBavV7NA\nvN6niXZslMDzyw/h7I1UhV1RZWFYISIiqxDRoz6eaFdLO07N0uPZxZFITOcMIVvHsEJERFZBCIEP\nBrZAWJ0qWi3mZgYmLIviDCEbx7BCRERWw8VRhzkj2pnMENp7PgH/t4EzhGwZwwoREVmVAM+7Zwgt\n33cRizlDyGYxrBARkdUpdIbQzyfx15k4dU1RhWFYISIiq9SrWSBeebixdmwwSkxadhDn47iHkK1h\nWCEiIqs14b4GGNimpnackqnH2CWRSL6Vo7ArMjeGFcW4NxARUdkJIfDRoJYIrZ2/h9D5uHRMWXEI\nes4QshkMK4pxbyAiovJxddJh/oh2CPJ21Wp/no7DR79EK+yKzIlhhYiIrF41b1fMHxkGV6f8f9YW\n7vwHqyMvKeyKzIVhhYiIbELLWj745IlQk9pb644hKiZBUUdkLgwrRERkM8JDa2DKAw2142yDEeOX\nRiE26ZbCrqi8GFaIiMim/PuhRujdLFA7jk/LRsSSSGRk6xV2ReXBsEJERDbFwUHgi8Gt0STIS6sd\nv5KCV1b/zSX5rRTDChER2RwPF0fMHxkGPw9nrfbz0auYse2swq6orBhWiIjIJtX2c8esYW3h6JC/\nJv9nv5/GluPXFHZFZcGwQkRENqtTfX+827+5Se3FVYcRfS1FUUdUFgwrRERk04Z1rIPhnYK14/Rs\nA8YtiURCerbCrqg0GFaIiMjm/V94c3Sq76cdX0q4hYnLopDDJfmtAsMKERHZPCedA2YNa4daVdy0\n2t7zCXh/0wmFXVFJMawQEZFd8PNwxoJRYXB31mm1JXtisGL/RYVdUUkwrBARkd1oEuSNz59qbVJ7\ne/0xHLjAJfktGcMKERHZlT4tgvCvh0K04xyDxHNckt+iMawoJoQIF0LMS05OVt0KEZHdeP6BEDzS\nIkg7vpmeuyT/rWyDwq6oKAwrikkpN0opI3x8fFS3QkRkNxwcBD59MvSuJflf/ZFL8lsihhUiIrJL\nt5fkr+LupNU2HrmCOTvOK+yKCsOwQkREdqu2nztmDmsLXYEl+advicb26BsKu6I7MawQEZFd69Ig\nAG/3baYdSwk8v+IQzt5IU9gVFcSwQkREdm9k5zoY0r62dpyapUfEkkgk38pR2BXdxrBCRER2TwiB\nd/s3R7s6VbTa+fh0/GvlIRiMHHCrGsMKERERABdHHWYPb4sgb1ettv1UHD797ZTCrghgWCEiItJU\n83LFvJHt4OKY/8/j7D/OYcORKwq7IoYVIiKiAlrV8sW0x1ua1F5dcwTHYrl4pyoMK0RERHcY2KYW\nxnWvpx1n5hgxfmkU4tOyFHZlvxhWiIiICvFanyboHhKgHccm3cLEZQeRYzAq7Mo+MawQEREVwlHn\ngBlD26KOv7tW2/9PAt7fdEJhV/aJYYWIiKgIPu5OmD8yDB7OOq22ZE8MVu6/qLAr+8OwQkREdA+N\nAr3wxeDWJrX/rD+GqJgERR3ZH4YVIiKiYvRuHoR/PRSiHecYJMYvPYhryZkKu7IfDCtEREQl8PwD\nIejdLFA7jk/LwvilkcjMMSjsyj4wrBAREZWAg4PA54Nbo1Ggp1Y7cjkZb607Bim5JH9FYlghIiIq\nIU8XR8wfGQZvV0et9uPBy/hu1wV1TdkBhhUiIqJSqOPvgRlPt4WDyK99sPkkdp2NV9eUjWNYISIi\nKqUejari9UeaaMcGo8Sk5QdxKSFDYVe2i2GFiIioDMZ1r48BrWtox0kZORi3JBIZ2XqFXdkmhhUi\nIqIyEEJg2uOt0KKmt1aLvpaKV1b/zQG3ZsawopgQIlwIMS85mbt5EhFZG1cnHeaOCIO/h7NW+/no\nVcz645zCrmwPw4piUsqNUsoIHx8f1a0QEVEZ1PR1w6xhbeFYYMTtp7+dwvboGwq7si0MK0REROXU\nsb4//i+8mXYsJfD8ykM4F5emsCvbwbBCRERkBsM71cHgsNracWqmHhFLIpGamaOwK9vAsEJERGQG\nQgi8N6A52gb7arVzcen496rDMBo54LY8GFaIiIjMxMVRhznD26Gal4tW+9/JG/hy6xmFXVk/hhUi\nIiIzqubtijkj2sFZl/9P7Ndbz+DXY9cUdmXdGFaIiIjMrG1wFfx3QAuT2ks/HMbp66mKOrJuDCtE\nREQV4Kn2tTGqcx3tOD3bgHFLIpGcwQG3pcWwQkREVEGm9m2GjvX8tOOYmxmYsvIQDBxwWyoMK0RE\nRBXESeeAmcPaooaPq1b783Qcpm+JVtiV9WFYISIiqkABni6YNzIMLo75/+TO3XEeG45cUdiVdWFY\nISIiqmAtavrg48dbmdReXXMEx69wX7iSYFghIiKqBAPa1MS47vW048wcIyKWRCEhPVthV9aBYYWI\niKiSvNanCbqHBGjHsUm3MGnZQegNRoVdWT6GFSIiokriqHPAN0PbINjPXavtOX8TH27mgNt7YVgh\nIiKqRL7uzpg3sh3cnHRa7dtd/+DHqMsKu7JsDCtERESVrEmQNz57KtSk9sa6o/j7cpKijiwbwwoR\nEZECj7asjkn3N9COs/VGjF8ahbjULIVdWSaGFSIiIkVe7NUY9zeuqh1fTc7EpGUHka3ngNuCGFaI\niIgU0TkIfDmkDeoFeGi1/RcS8P6mEwq7sjwMK0RERAr5uDlh/sh28HRx1GpL98Zg5f6LCruyLAwr\nREREijWs5oXP7xhw+/b644iKSVTUkWVhWCEiIrIAvZsH4V8PhWjH2QYjJnwfhespmQq7sgwMK0RE\nRBtp3XgAABgZSURBVBbi+QdC0KtZoHZ8IzULz30fhSy9QWFX6jGsEBERWQgHB4HPnwpFw2qeWu3Q\nxSS8/dNxSCkVdqYWw4oZCCHaCiHWCiEuCCFuCSHihBA7hBB9VfdGRETWxcvVCfNGtIOXa/6A21WR\nl/D9PvsdcMuwYh71AbgA+BbAFAD/BSAAbBRCPKeyMSIisj71q3ri6yFtIER+7d0Nx7H/nwR1TSkk\n7Pm2UkUSQugARAHwkFKGFHd+WFiYjIyMrPjGiIjIaszcfhafbDmlHQd4OmPD5G6o4eumsCvzEUJE\nSSnDijuPd1YqiJTSAOAyAF/VvRARkXWa2LMBHmtZXTuOT8vG+KVRyMyxrwG3DCtmJITwFEIECCEa\nCiFeAtAHwO+q+yIiIuskhMD0J1qhSZCXVjsam4w31x21qwG3FhtWhBBeQoh+Qoj3hRC/CCHihRAy\n79WkhJ8RJIT4SghxTgiRKYS4LoTYKIR4sILangMgDsAZANMBrAMwqYK+i4iI7ICHiyPmjQiDj5uT\nVlt7MBbf7bqgrqlKZrFhBcCDANYDmIrcOxT+pblYCNEKwDEAzyN3AGwWgAAAfQH8LoR4vZBrhBDC\ntYSvwv7bfQSgF4BRAH4D4IjcgbdERERlFuzvjhlPt4FDgQG3H2w+id1n49U1VYksOawAwA0AmwG8\nCyCipBcJIdwAbEBuwDkEoIWU0gdAFQCfIXemzodCiN53XNoVwK0Svnrc+b1SyuNSyv9JKZcAeBSA\nJ4ANQhQcz01ERFR63UOq4o1HmmrHBqPEpOUHcSkhQ2FXlcOx+FOU2Sil/On2gRCibimuHQ+gDoA0\nAOFSylgAkFKmAP/f3p2HSVGdexz/vjMwyA4KCC6ISwQFZb1xSTQQl8QYjNmNRIM+kSRXc/Wq12jM\nTbyJicZc4xaTSDYjblnMZjSaoMYsogkgeEXBheCGIKAMuyC8949zerpounsaeoaq6f59nqee6lN1\nTvfpl2b67apTp7jQzPYHTiYcCfljot0zwBkVvsb8cjvd3c3sF8BNwIHAgnL1RUREWvPpo/Zl3uJm\nfjNnMQBvrNvElGmz+NXnjqRrU2PKvWs/mU1W4tU0O2pSXN+eS1QKfIuQrIwxs6HuviC+5mvAzVW8\nbqHctWW92/A5RUSkTpkZV3zoUJ59bQ3zFq8C4OlXV3HRXU9w/SmjqNUD+Vk/DbTdzKwnMDYW7y9R\n7VGgOT6uerCtmQ0osq2JMHZlPfBUta8hIiIC0LWpkZtOG8uu3Ztatt09dzFT/7IwxV61r5pLVoCD\nCGNSAOYVq+DuW8ifljm4DV7zznjF0n+b2afN7MuEwb2jgS+5+5o2eA0REREA9urbjRtPHUNjYsTt\nN++bz8PPLEuxV+2nFpOVQYnHi8vUy+0bVKZOpaYBuwDnAN8DzgUWAR9w92+3wfOLiIhs5Yj9d+O/\nT8wPuN3i8PnbZ7No+doUe9U+ajFZ6Z54vL5Mvdzw6R5l6lTE3X/i7hPcfXd37+zuu7n78e7+u3Lt\nzGyKmc00s5nLltVmNiwiIu3nU0cO4SNj92opr9rwFlOmzWTtm2+l2Ku2V4vJSofh7lPdfZy7j+vf\nv3/a3RERkQ7GzLj85BGM3Ct/HcczS9dwwc/n1tQMt7WYrCSPf5W701O3uNZ4EhER6bB26dzI908b\nS78e+TlI75u3hBsfei7FXrWtWkxWkuNU9ihTL7fv1Xbsi4iISLsb1Lsr3//kGDo35gfcXv2nZ3jg\n6aUp9qrt1GKyMh/IHfsaXqxCnCp/aCzqsmIREenwxg3ZlctOyn/tucN5d87h+WUd/wRCzSUr7r4a\nmBmLx5Wodhj5idoeaPdOiYiI7ASTDtuHUw8b3FJe/eZbnHXLTFZt2JRir6pXc8lKdHtcTzKzYpcm\nXxjXs3Kz16bFzCaa2dTm5ubWK4uIiLTisonDGbdP35bywmVr+c8757BlS8cdcJvpZMXM+uUWwk0I\nc/ok9xW5A/JNwAtAT+D3ZnZwfL6eZnYV8KFY74vt/R5a4+53u/uU3r01I7+IiFSvqVMD3/3kGHbv\nlR9w+8D817h2+jMp9qo6mU5WgGWJZXZi+4yCfYOTjdx9PfABYAUwBphnZs3ASuC/CGNaLnH35E0M\nRUREasKAnrtw02njaGrMf81f/+Bz3Pdkx7ymJOvJyg5z97nACOB6YCHQhZC83AMc5+5Xptg9ERGR\ndjVq7z5c/sERW207/+dzWbBkdUo92nGZTlbc3SpcFpVov8Tdz3X3/d19F3cf4O7vd3cNqhURkZr3\nsXF7M/nIIS3ldRs3M2XaTFau25hep3ZAppMVERERqc6lJx7EYfvu2lJ+YcU6Pn/H42zuQANulayk\nTFcDiYhIe+rc2MB3J41hzz75Sd3/+uxyrrp/foq92j5KVlKmq4FERKS97dajCzedNpYunfJf+zc9\nvJDfznklxV5VTsmKiIhIHRixZ2+u+sihW237wl1PMG9x9o/sK1kRERGpEx8YtSdTjt6vpbxh0xam\n3DKL19dme8CtkhUREZE6ctF7hnLU2/q1lF9ZuZ6zb5vNps1bUuxVeUpWRERE6kinxgZu+MRoBu/a\nrWXbjIUr+Ma9T6fYq/KUrIiIiNSZPt2amHr6WLo1NbZs+8nfF/HLWS+n2KvSlKyIiIjUoWEDe3H1\nR0dute2Lv/4/5r60MqUelaZkJWWaZ0VERNJywiGDOGfCAS3ljW9t4TPTZvHa6g0p9mpbSlZSpnlW\nREQkTecfdyDHDBvQUl6yagP/futsNr6VnQG3SlZERETqWEODcc0po9ivf/eWbTNfeIP/uXteir3a\nmpIVERGROtdrl85MPW0cPbt0atl222MvcvtjL6bYqzwlKyIiIsIBA3pw7SmjMMtv+8rvnmTmotfT\n61SkZEVEREQAOOag3Tn/2ANbyps2O5+9dTavNq9PsVdKVkRERCTh7AkH8N7hA1vKy9e8yWenzWLD\nps2p9UnJioiIiLRoaDCu/thIhu7es2Xb3JebufTXT+Lu6fQplVeVFppnRUREsqZ7l05MPX0svbt2\nbtl21+yXufmRRan0R8lKyjTPioiIZNE+u3Xnhk+MpiEx4Pbye57mkeeX7/S+KFkRERGRoo4+sD8X\nnzCspbx5i/PwgmU7vR9KVkRERKSks47aj5NG7kGnBuPyk0dwyfsO2ul96NR6FREREalXZsY3P3wo\nZ7xjCKMH902lDzqyIiIiImV1bWpMLVEBJSsiIiKScUpWREREJNOUrIiIiEimKVkRERGRTFOykjLN\nYCsiIlKekpWUaQZbERGR8pSsiIiISKYpWREREZFMU7IiIiIimaZkRURERDJNyYqIiIhkmpIVERER\nyTQlKyIiIpJpSlZEREQk05SsiIiISKYpWREREZFMM3dPuw8CmNky4IU2ftp+wPI2fs56oxhWTzGs\nnmJYPcWweu0Rw33cvX9rlZSs1DAzm+nu49LuR0emGFZPMayeYlg9xbB6acZQp4FEREQk05SsiIiI\nSKYpWaltU9PuQA1QDKunGFZPMayeYli91GKoMSsiIiKSaTqyIiIiIpmmZEVEREQyTcmKiIiIZJqS\nlRpjZgPN7Doze97MNpjZUjO728yOSbtvaTOzwWZ2XozHi2b2ppmtNrO5ZnalmQ1qpX2TmV1kZnPM\nbI2ZrTSzGWY2xcxsZ72PrDGzHmb2kpl5XCaXqasYJpjZUDO7wcwWmNlaM2s2s6fN7Mdm9q4SbRTD\nyMwazOwMM5tuZsvMbFOMx2NmdqmZ9SzTtubjaGY9zewkM/uamf3BzJYn/p8Oq6B9VTEys4+a2YNm\ntsLM1sXP9uXl/l1KcnctNbIAhxJmF/S4NAOb4+MtwMVp9zHF2OwdY+AF8XkrUX4dmFCifS9gZqLu\nWuDNRPluoFPa7zOl2F5bENfJimFFcfuPgve/GlifKP9QMSwbv27AAwWfvZUF/88XAfvVaxyBkwvi\nk1yGtdK2qhgRrhzK1d0UP9+58vPAHtv1XtIOppa2WYCu8T+mA7OB4XF7L+B/EwnL8Wn3NaX4DInv\n//fAR4C+cXsTcAKwkHwCM7BI+5/F/SuA9wMGNAKfSnzBfD3t95lCXMcQEr5HE3+IJpeoqxjmY/GZ\n+H43A1cCgxP7BgKnAWcqhmVjeEXi79rFQO+4vQk4BXgj7n+wXuNISFaWAvcAlwFnUXmyssMxAj6X\n+HxfCHSJ249MfE/9bbveS9rB1NI2C3Ae+V9nexbZ/+u4f1bafU0pPr2BkWX2D0v8B/xKwb7Rif/g\nJxVpe27ctw4YkPZ73YkxbQD+GZOVZIwmF6mrGObf6xDCr1QHztqOdorh1u/3hfh+f1Ri/+REvPrW\nYxyBxoLykEqSlWpiBHQhJEgOfLvEc+eOfk2s9L1ozErtmBTXt7v7K0X2fyuux5jZ0J3Up8xw92Z3\nn1tm/3zC0QGAsQW7T43rBe7+uyLNpxKOyHQFPlRtXzuQzwPjgO+5++Ot1FUM884lnMJ4zN1/sB3t\nFMOt7R7XpT57sxKPuyUe100c3X3zDjatJkbHAgMIycjVRfr0ODA9FicV7i9FyUoNiIOVcl+w95eo\n9ijhwwVQ94NtS1gR140F2yfE9R+LNXL39cBfY/Hd7dCvzDGzPYGvEX5BfamCJophXu6L4I7tbKcY\nbm1RXI8usT/3N3FpwQ84xbF11cQo1/bJEj+cIf89VXF8lazUhoMI5xMB5hWr4O5bgAWxePDO6FRH\nYmadgHfE4pOJ7UY4RQQlYhs9Fdf1EtsbgJ7Ahe7eXK6iYphnZvsTfnUCPG5mh8er01aY2Xozm29m\n3zKzAQXtFMNt5Y5KnWFmF5tZb2i5guXjwDWEX/cX5hoojq1rgxjlypW07W9m/Srpl5KV2pC85HZx\nmXq5fWUv0a1TZxMGNm4BfprY3gvoHh8rtoCZTQQ+CPzZ3W+toIlimPe2xOPxwN8Igxc7E75YhxK+\nXOeY2fBEXcVwW9cCNxJ+qF0BrDSzlYSxZ3cC8wnjLZKfUcWxddXGaFDB/nJti7UvSslKbeieeLy+\nTL11cd2jHfvS4ZjZoYQ/dgDfcfenErsV2wQz6w58h3Ap4tkVNlMM8/okHn8FeAY43N17Ed73+4DX\nCH/A74pH/EAx3EYcj3EecAFhkDeEgfS577WeQP+CZopj66qNUa59JW2LtS9KyYrUtTgR3G8IA8Vm\nAV9It0eZ91VgMHBNQVInlUn+zXXgg+7+GIRTte7+B+DMuH8oHXyAZ3sys4HA3wmDOG8DRhK++N4G\nXALsB/zYzK4o+STSYShZqQ1rE4+7lqmXGxG/ph370mGY2a6EAWT7As8CJ7r7hoJqim1kZqMIV7K8\nREhaKqUY5iXf233uvqCwgrvfQzjiAvnB8Irhtm4B3k64dHmyuz/h7mvd/Tl3v5Iwlw3ARYlTaopj\n66qNUa59JW2LtS9KyUptSJ7/26NMvdy+V9uxLx1CHIx3PzACeBE41t2XFqm6ivx/vnqP7XWEK6Uu\nJYzD65FcEvW6xG25P0iKYV7y/+o2iUqRfXvHtWKYYGYHA8fF4jXF6rj7NMIVfg3AxLhZcWxdtTFa\nXLC/XNti7YtSslIb5hMOKQMML1bBzBoIh5UhPxK7LsVxF/cS5ghZQkhUXixW18MsRk/HYtHYRrkR\n8LUc233i+hbC5IOFS873Y/kpUAwLPEUYxF0pB8WwiIMSj/9Vpt7CuB4CimMl2iBGuXIlbZe5+/JK\n+qVkpQa4+2rCPRwg/2uj0GGEwWcQ7qdRl8ysK+GeFkcSfnUd6+7PttLsobguGlsz2wU4KhbrNrat\nUAwBd18HzIjFcpMz5vYtSmxTDPOSCd/gMvVyCXYymVYcW1dNjHJth1vpm8MeX6JtSUpWasftcT2p\nxAckN9fArGLnyeuBmTUBvyJMWrSScJ+kcnMB5OQm7xpmZu8vsv8sQiK4nnBbg5rk7kPc3Uotiapn\nxG1DEtsUw7xb4vq9xWaTNrMTgQNj8d7ELsUwLzkb9VnFKsRL7HPz1TyW2KU4tq6aGD1AuKKtgXCl\n1lbMbCRhllsIA6Mrk9Y9C7S07cLWNzKcBRwct/cEriJ/n4d6vZFhI/DLGINVhMtFt6d97qZey4H3\nJZ7zdMJleDVx47MqY1zpjQzrOoZAJ8KEWR7Xb4/bG4D3Ek5NOuEIjCmGJeN4P/mb5V1BvEcN4Yqg\nyYQjp044TdRUr3EE+iWW5D1/Di/Y19BWMWLrGxleQP5GhkeQv2msbmRYrwvh0r3liQ9jc/ywOPHO\npGn3McXYHJ2Iy/r4hVBq+WeR9sVul74hUa6JW8pXGePWkhXFMB+L/QgDu3PvPTeoMVeeR/EbkiqG\n+VgMIoyP8II4JstLgNH1HMeCeJRbhrRljAj3D8rV3Ug4FZcrPw/ssT3vQ6eBaoiHG/WNAK4nZK9d\nCL8u7gGO83A5X71KftZ3IdwErdRSOJEU7r6KMM7lYsIhaAfeJNxz6TOEmTLfKmwneYphnrsvBA4B\nvk74wu1EiMdswhwhb/ci91VRDPPc/VXC/X/OA/4CvE64JHYVIY5fAw7xIjfZVBxbV22M3H0K8HHC\nGJY1hM/4fMJnfpS7l5vhdhsWMyARERGRTNKRFREREck0JSsiIiKSaUpWREREJNOUrIiIiEimKVkR\nERGRTFOyIiIiIpmmZEVEREQyTcmKiIiIZJqSFREREcm0Tml3QESkNWbWCfgkcArhHli7Ee5VsoRw\na4m/Ag+6+z8SbUYBJwOL3P3mnd1nEWk7mm5fRDLNzPoD9wLjEps3EO5T0guwuK3Z3fsk2k0GfgI8\n7O7jd0pnRaRd6DSQiGTdrYREZTVwETDI3bvGxKQ3cBzwXWBlel0Ukfak00AikllmNgw4PhbPdPdf\nJve7+2pgOjDdzC7Y2f0TkZ1DR1ZEJMsOSTz+fbmK7r4h99jMnHAKCOBdZuYFy/jC9mb2TjO708xe\nNrM3zWyFmU03s0+YmRWpPz4+16JYnmhmD5nZG2a2xsxmmNmpO/CeRaSAjqyISEexJ/B8hXWXAl0J\nY1o2Aa8X7N+YLJjZNwmnmHJWAX2BY+JykplNcvctxV7MzM4DrgEcaI6vfThwuJkd6e7nVNhvESlC\nR1ZEJMtmJR7fGAfbtsrdBwLnxuIj7j6wYHkkV9fMziUkKkuBKUAfd+8NdCdcfbQkrr9Q4uX6A1cB\ntxDG0/QF+gFXx/1n6wiLSHV0NZCIZJqZ/RQ4PRY3Ei5TfhT4JyERWVai3WRauRrIzPoALxGOMh/u\n7nOL1DkC+DthAO9Ad98Yt48HHorV/gS8xwv+oJrZzcCngOeAAwv3i0hldGRFRLLuLODbhESliXBa\n5lLgN8BrZvYPM5tUbFxJBT4M9ACmF0tUANx9BvAvwmmhsSWe54oSicjX4/oAwvwwIrIDlKyISKa5\n+0Z3vwDYG/gscAfwLGF8CMC/ES5v/pmZbe/ftCPj+t1mtqTUEl+bxDppE+HIS7G+Pwu8GotjtrNv\nIhJpgK2IdAju/hpwU1wws92BicCXCUnERwlJw3Xb8bSD4rpbXFpTrM7y3KmhEl6Jr1PReBsR2ZaO\nrIhIh+TuS939h4QjFkvj5jO382lyfwOvc3erYLm5rfovIpVTsiIiHZq7Lwd+G4sHbmfzXJIzuIou\n9DOzpjL794jrogOBRaR1SlZEpBasjevk6ZjcnCjlBt7OiOvxZtZ1B1+7M3BEsR1mdgD5ZGX2Dj6/\nSN1TsiIimWVm+5rZ/q3U6Ua4uzLAnMSuVXHdh9J+QUh0+hLGvpR7nb5ldl9S4mqkS+L6WXefU2S/\niFRAyYqIZNlwYIGZ/crMPmZmuQGxmFl3M5tImHdl37g5Obh2XlwfbGaHFXtyd19BPqG42Mx+YGYt\np5LMrJuZTTCzqcAjxZ4DWEe4nPpHZjYgtusTZ8XNjaG5rML3KyJFaFI4EcksM3sPcF/B5vWE0z29\nE9s2A192928UtH8YODoWXyfcuRngFHd/NFHvS8BXyZ8yWhtfo09i2yJ33zfRZjxhUrgXgGvJT7e/\nMvYt92PwRk23L1IdJSsikmnxSMdE4J3ACMI9gpoIicdC4C/AD919XpG2uxGSkBMS7QAmuPufC+oe\nApwDTAD2AhoJg2KfBB4A7nD3lxP1xxOTFXcfEo/ynA+MJoxjeQL4jrvfVnUQROqckhURkR1QmKyk\n2xuR2qYxKyIiIpJpSlZEREQk05SsiIiISKYpWREREZFM0wBbERERyTQdWREREZFMU7IiIiIimaZk\nRURERDJNyYqIiIhkmpIVERERybT/BxWs+8Ieln/wAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def line_search(x, d):\n", " \"\"\"Constant step size.\"\"\"\n", " return -0.1\n", "\n", "ws_bfgs = bfgs(init, gradient, hessian, line_search, 100)\n", "error_plot([log_likelihood(w, data, labels) for w in ws_bfgs])\n", "plt.plot(range(len(ws_bfgs)), [baseline]*len(ws_bfgs),\n", " label='sklearn', **kwargs)\n", "_ = plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now find a clever step size using the golden section search algorithm we discussed in a previous lecture." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAisAAAH/CAYAAACW6Z2MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8VFX+//H3yaR3AqEIhCZVivQisLo2FFDsqHQ1YAH9\n7erq7rpNd7+irruKqICAIEpZsCBgWRcU6b0JIkjvLSG9Tu7vj4RJRikJKXfK6/l4zAPOZ+6d+aAg\nb+8951xjWZYAAAA8VYDdDQAAAFwMYQUAAHg0wgoAAPBohBUAAODRCCsAAMCjEVYAAIBHI6wAAACP\nRlgBAAAejbACAAA8WqDdDaBQjRo1rIYNG9rdBgAAVWbDhg2nLcuKv9RxhBWbGWP6S+p/5ZVXav36\n9Xa3AwBAlTHGHCjNcdwGspllWQssy0qMiYmxuxUAADwSYQUAAHg0wgoAAPBohBUAAODRCCsAAMCj\nEVYAAIBHI6wAAACPRlgBAAAejbACAAA8GmEFAAB4NLbbBwAPZVmW0tLSlJqaqszMTDmdTrtbAtw4\nHA6Fh4crOjpaUVFRMsZUyvcQVgDAA1mWpZMnTyojI0NxcXGqXbu2HA5Hpf1lAJSVZVlyOp1KT0/X\n6dOnlZWVpZo1a1bK71HCCgB4oLS0NGVkZKhBgwZyOBx2twP8gjFGgYGBio2NVVRUlA4cOKC0tDRF\nR0dX+HcxZwUAPFBqaqri4uIIKvAKDodDcXFxSk1NrZTPJ6wAgAfKzMxUZGSk3W0ApRYZGanMzMxK\n+WzCig86lJSpt775Sc4Cy+5WAFwmp9PJVRV4FYfDUWmTwAkrNjPG9DfGTEpJSamQz8vJd+qxDzfq\n1a9+1LD31upMek6FfC6AqsdkWniTyvz9SlixmWVZCyzLSoyJiamQz3tx4Q5tO1IYfJbtPq1+by7X\nhgPJFfLZAADYgbDiY9rWi1VIYPG/1mMp2bpv4ipNXb5PlsVtIQCA9yGs+Jh7O9XXJ49do4bVw121\n/AJLLyzcoSdmblJadp6N3QEAUHaEFR/U6opofTa6p/pcVdutvmjbMd0+foV2Hq+cpWUA4G2mTZsm\nY4yuvfbaMp3317/+VcYYDRs2rFL6gjvCio+KDg3SO4M66Pm+LeUIKJ70tPd0hga8tUIfbzxsY3cA\nAJQeYcWHGWP0cK/Gmp3YTbWiQ1z17LwC/eY/W/SHT7YpO49njQAAPBthxQ90bhinhaN7qUeT6m71\nmWsO6u4JK3UoqXI28QEAoCIQVvxEfFSIZjzUVaN/faVb/fsjqeo7bpn+t+OETZ0BAHBxhBU/4ggw\n+u1NzfXesM6KCQty1VOz8/Xw++v18pc7le8ssLFDACi/3NxcvfHGG+rRo4diY2MVFBSkWrVqqV27\ndnr88ce1atWqUn/WjBkzFBQUpICAAL3zzjtl6mP58uUaOHCg6tWrp5CQEFWvXl033HCDZs2adcGt\nJDZu3KjnnntOPXv2VEJCguu8a6+9VpMnT77gDrElJ/wWFBRo/Pjx6tKli2JjY2WM0ebNmyVJw4YN\nkzFGf/3rX+V0OvX666+rXbt2Cg8PV1xcnPr166f169eX6ddZFXjqsh+6rkVNLRzdU4/P3Kith4t3\nzn3n2z3adDBZ4+5vr5pRoTZ2CACXJz8/XzfddJOWLl0qqXDuXkxMjM6cOaOTJ09q69atOnPmjLp3\n737Jz3rrrbc0evRoORwOvf/++xo0aFCp+3j22Wf1yiuvuMbR0dFKTk7W4sWLtXjxYn322Wf68MMP\nFRDgfs3gpptu0pkzZyRJ4eHhCg8PV1JSkpYuXaqlS5fqk08+0fz58xUYeP6/vi3L0p133qn58+fL\n4XAoKirqvMfl5+erb9+++uqrrxQUFKSQkBAlJydr0aJFWrx4sZYsWVKqf0ZVhSsrfqp+XLjmjuqu\nwd0auNVX701S33HLtWbvGZs6A4DLN3PmTC1dulTh4eGaMWOGMjMzlZycrJycHB04cEDjx49Xu3bt\nLvk5//jHP/TEE08oODhY8+bNK1NQeeONN/TKK6+oVq1amjRpks6ePauUlBRlZGRo9uzZql27tmbP\nnq2XX375F+fedNNNmjVrlo4dO6aMjAwlJycrPT1dM2bMUO3atfX555/r3//+9wW/++OPP9aXX36p\nt99+W6mpqUpOTtaJEyfUuHFjt+PeeustrVu3TnPmzFF6errS0tK0ZcsWtW7dWtnZ2XryySdL/eut\nClxZ8WMhgQ69OKC1OjWspuc+2qasopVBp9Jy9MDkNXrm5uYa2bsxzycBPM0Xz0nHt9ndRcWo3Ua6\nZWyFfdzq1aslSUOGDHELGA6HQwkJCXr88ccv+RnPPPOM/vnPfyoiIkLz58/X9ddfX+rvP3v2rJ5/\n/nmFhobqq6++cgtGYWFhuu+++5SQkKBrrrlGr776qn77298qODjYdczMmTN/8ZkREREaNGiQGjRo\noN69e+vtt9/WM888c97vT09P18SJE5WYmOiq1axZ87x9Llu2TD179nTV2rZtq2nTpqlTp05at26d\nDh48qISEhFL/2isTYQW6/eq6alUnWqM+2KA9pzIkSc4CS2O/2Kn1+5P12r3t3Oa4ALDZ8W3SgeV2\nd+GRoqOjJUnHjh0r87kFBQUaOXKkJk+erNjYWH3++edlvhXy0UcfKT09Xf369bvgFZzu3burUaNG\n2rt3rzZs2FDq7+jVq5diY2O1f/9+HT16VFdcccUvjqlevbpGjBhRqs8qGVTO6dixo+rVq6fDhw/r\n+++/95iwwm0gSJKa1orSZ0/0VP927r/5//fDCfV/c7m+P1IxT4UGgMp0yy23SJLmz5+v2267TR9/\n/LFrDsjF5OXl6f7779fkyZNVs2ZNffvtt5c1Z2PlypWSpCVLlqh27doXfB06dEiSXD+WNHfuXA0Y\nMEAJCQkKCwuTMcb1Onv2rCTp6NGj5/3+Tp06XXA+S0mdO3e+4Ht169aVJCUne85DcLmyApeIkECN\nG3i1OjesphcX7lCes3C2+sGkTN35zkq9cNtVuq9zfW4LAXar3cbuDipOBf9afvWrX+mFF17QCy+8\noAULFmjBggWSpBYtWqhv374aOXKkmjZt+ovzzoUMSZo3b16p5rWcz7krOpmZmcrMvPQeViWPyc/P\n17333qtPPvnEVQsJCVGNGjXkcDgkSadOnVJBQYEyMjLO+3nx8fGl6vNCE28lKTS0cIFFXp7nPEuO\nsAI3xhgN6d5QbevF6vEPN+rI2SxJUm5+gZ77eJvW7U/W3we0Vliww+ZOAT9WgXM8fNGf/vQnDRo0\nSHPmzNG3336rVatWaefOndq5c6feeOMNTZkyRUOGDHE7p02bNnI6ndqxY4cee+wxffPNN6pRo0aZ\nv7ugoHD7hyeffFKvv/56mc5999139cknnyg8PFxjx47VHXfcoXr16rkdU79+fR0+fPiCS5/PhRpf\nw20gnNfV9WO1cHRP/aqZe0r/aONh3fH2Cu09lW5TZwBwaY0aNdJzzz2nL7/8UklJSfrmm2/Uu3dv\n5efn67HHHtPJkyfdjo+Li9PixYvVvHlzff/997rxxhsv6zZIrVq1JEkHDx4s87lz586VVBi2Ro8e\n/Yug4nQ6dfr06TJ/ri8grOCCqkUE671hnfWbG5up5J2fncfTdNv4Ffp8W9knsAFAVXM4HLr22mu1\ncOFCBQUFKSMj47wbn9WuXVuLFy9WkyZNtHnzZt10001KSSnbfL1z81y+/fZbZWVllencw4cLHzDb\nvn37876/YsUKZWdnl+kzfQVhBRcVEGA05vqmmjGiq+IiipfXpefk67EPNxbNbWHXWwCeITc394Lv\nBQcHu26T5OTknPeYunXrasmSJWrYsKHWr1+vW265Renppb+SfM899ygiIkLJycl64YUXLnrsz6/c\nxMTESJK2bfvlsvT8/Hw9//zzpe7D1xBWbGaM6W+MmVTW9F7VejatoUVjeqpjg2pu9SnL92ngpNU6\nllK2/4MAgMowZMgQDR8+XF999ZXS0tJc9f3792vo0KHKzs5WWFiYevXqdcHPSEhI0JIlS1SvXj2t\nWrVKffv2LdVkWalw6fBLL70kSRo7dqweeeQR7dq1y/V+ZmamvvnmGyUmJqpHjx5u5954442SpBdf\nfFHz5893ba2/c+dO9e/fX2vXrlVERETp/kH4GMKKzSzLWmBZVuK5RO3J6sSEaXZiNz3Us5FbfcOB\nZPUdt1zLd/vnvVQAniM7O1vTpk1Tnz59FBMTo2rVqikiIkKNGjXSnDlz5HA4NHHixEtOnm3UqJGW\nLFmiOnXq6LvvvtNtt91W6lswo0eP1osvvihjjCZPnqzmzZsrMjJScXFxioyM1K9//Wu9++67v/i8\np59+Wk2aNFFqaqoGDBigsLAwxcTEqGXLlvr66681YcKEy5r06wsIKyiTIEeA/tSvld55sIMiQ4oX\nkyVl5Grw1DUat3i3CgrOP0sdACrb2LFj9corr6hPnz5q3LixcnNz5XQ61aRJEw0fPlwbN27U4MGD\nS/VZTZs21ZIlS1SrVi0tXrxYd9xxxwVvH/3c888/ry1btigxMVFNmzZ1LTe+4oordPPNN+uVV17R\nsmXL3M6Ji4vT6tWr9eijj7om14aFhWnAgAFaunSphg0bVqZ/Fr7EXGj5E6pWp06dLE980uXF7D2V\nrsc+3Kidx9Pc6r9qFq/X77ta1UrMcQFQNj/88INatmxpdxtAmZT1960xZoNlWZ0udRxXVnDZGsdH\n6pPHrtFdHdyX1y3ddUp9xy3TpoOes/shAMB7EVZQLmHBDv3znrYae2cbBQcW/3Y6mpKteyeu0vSV\n+y+4eREAAKVBWEG5GWM0sEuCPn60hxLiwl31PKelv3y2XWNmb1ZGTr6NHQIAvBlhBRWmdd0YLRjd\nUze2quVWX7DlqG4bv1y7T6Rd4EwAAC6MsIIKFRMWpEmDO+q5W1rIEVC87e2eUxm6bfwKzd98xMbu\nAADeiLCCCmeM0ahfNdGHD3dVfFSIq56V59STszfr+U+3KSffaWOHAABvQlhBpenWuLoWjempbo3j\n3OofrD6oeyas0qGk0u0ICQDwb4QVVKqaUaH64KGuevTaJm71rYdT1O/N5drI8mYAwCUQVlDpAh0B\nerZPC00e0knRocW73qZk5enJ2ZuUncctIeB8WPYPb1KZv18JK6gyN7SqpUVjeql13WhX7VBSlqau\n2GdjV4BncjgcrgfZAd7A6XS6nmpd0QgrqFL148L1wUNdFRse5Kq9teQnnUwt3QPCAH8RHh6u9PR0\nu9sASi09PV3h4eGXPvAyEFZQ5WLDg/X/bmjmGmfkOvXqVz/a2BHgeaKjo5WUlMTVFXgFp9OppKQk\nRUdHX/rgy0BYgS0e7JqgpjUjXeN5Gw9r2+EUGzsCPEtUVJQiIiJ04MABnT17Vvn5+cxhgUexLEv5\n+fk6e/asDhw4oIiICEVFRVXKdwVe+hCg4gU6AvSnfq00ZOpaSZJlSX9bsF1zR3WXMeYSZwO+zxij\nmjVrKi0tTampqTp58iRXWeBxHA6HwsPDVaNGDUVFRVXaf78JK7BN72bxur5FTS3eeVKStP5AshZu\nPab+7a6wuTPAMxhjFB0dXWmX1gFvwW0g2OqPfVsqyFGcxMd+sZOlzAAAN4QV2KpxfKSGdm/oGh85\nm6VJ3+21ryEAgMchrMB2o69vqriIYNf4nW/36HgKS5kBAIUIK7BdTFiQfntT8VLmrDynXv5yp40d\nAQA8CWEFHmFg5wS1qF285O2TTUd4bhAAQBJhxXbGmP7GmEkpKf69x4gjwOjP/Vu51V5YsEMFBewr\nAQD+jrBiM8uyFliWlRgTE2N3K7br0aSGbr6qlmu8+dBZfbr5iI0dAQA8AWEFHuWPt7ZSsKP4t+XL\nX+5URk6+jR0BAOxGWIFHSagerod6NXKNT6TmaMLSPTZ2BACwG2EFHufx665UfFSIazzpu706nJxp\nY0cAADsRVuBxIkMC9czNzV3jnPwCvfQFS5kBwF8RVuCR7u5QT23qFk86XrT1mNbuS7KxIwCAXQgr\n8EgB51vKvHA7S5kBwA8RVuCxOjeMU7+2dVzj74+kat6GwzZ2BACwA2EFHu33t7ZUSGDxb9NXvvpR\nadl5NnYEAKhqhBV4tLqxYRrZu7FrfDo9R299w1JmAPAnhBV4vFHXNlHt6FDXeOryfTpwJsPGjgAA\nVYmwAo8XHhyoZ28pXsqc6yzQ/33+g40dAQCqEmEFXuH2dnV1df1Y1/ir7Se08qfTNnYEAKgqhBV4\nhYAAo7/8YinzDuU7C2zqCABQVQgr8BrtE6rpjvZ1XeOdx9M0e90hGzsCAFQFwgq8yrN9WigsyOEa\n/+vrXUrJYikzAPgywgq8Su2YUD16bRPXOCkjV+MW77axIwBAZSOswOsk9m6surFhrvH0lfu151S6\njR0BACoTYQVeJzTIod/f2sI1zi+w9I9FLGUGAF9FWIFX6tumjro0jHONl+w8qaW7TtnYEQCgshBW\n4JWMKXwqszHFtRcX7lAeS5kBwOcQVuC1WteN0T0d67nGP51M14erD9jYEQCgMhBW4NWevrm5IkMC\nXeN//2+3kjNybewIAFDRCCvwajWjQvX4dVe6xilZeXr9f7ts7AgAUNEIK/B6I3o2VEJcuGv8wZqD\n2nUizcaOAAAVibACrxcS6NAfbm3pGjsLLL24cIcsy7KxKwBARSGswCfcfFUtdW9c3TVetvu0Fv9w\n0saOAAAVhbACn3BuKXNAiaXM//j8B+Xms5QZALwdYcVmxpj+xphJKSkpdrfi9VrWidbALgmu8b7T\nGZq+cr99DQEAKgRhxWaWZS2wLCsxJibG7lZ8wm9vbKao0OKlzOMW79bp9BwbOwIAlBdhBT6lemSI\nnry+qWuclpOv1/7LUmYA8GaEFficId0bqnGNCNd4zrqD2nE01caOAADlQViBzwkODNDz/YqXMhdY\n0gsLt7OUGQC8FGEFPum65jXVu1m8a7x6b5K+2n7cxo4AAJeLsAKfZIzRn/q2lKPEWuZ/fP6DsvOc\nNnYFALgchBX4rKa1ojS4WwPX+FBSlqau2GdjRwCAy0FYgU976oamig0Pco3fWvKTTqZm29gRAKCs\nCCvwabHhwfp/NzRzjTNynXr1qx9t7AgAUFaEFfi8B7smqGnNSNd47obD2nr4rI0dAQDKgrACnxfo\nCNCf+rVyq72wgKcyA4C3IKzAL/RuFq/rW9R0jdcfSNaCrcds7AgAUFqEFfiNP/ZtqSBH8VLmsZ//\noKxcljIDgKcjrMBvNI6P1NDuDV3joynZmvTdXvsaAgCUCmEFfmX09U0VFxHsGk9YukfHUrJs7AgA\ncCmEFfiVmLAg/fam4qXMWXlOvfzFThs7AgBcCmEFfmdg5wS1qB3lGn+6+ag2Hky2sSMAwMUQVuB3\nHAFGf+7vvpT5bwt2qKCApcwA4IkIK/BLPZrUUJ+rarvGWw6d1aebj9jYEQDgQggr8Ft/uLWlgh3F\nfwRe/nKnMnLybewIAHA+hBX4rYTq4XqoVyPX+ERqjiYs3WNjRwCA8yGswK89ft2Vio8KcY0nfrdX\nh5IybewIAPBzhBX4tciQQD1zc3PXODe/QGNZygwAHoWwAr93d4d6alM3xjVetO2Y1uw9Y2NHAICS\nCCvwewHnWcr8wsIdcrKUGQA8AmEFkNS5YZz6ta3jGm8/mqp5Gw7Z2BEA4BzCClDk97e2VEhg8R+J\nV7/6UWnZeTZ2BACQCCuAS93YMI3s3dg1Pp2eq/Hf/GRjRwAAibACuBl1bRPVjg51jd9bvl/r9ifZ\n2BEAgLAClBAeHKhnbymxlNlZoMFT1mjZ7lM2dgUA/o2wAvzM7e3qqnezeNc4O69AD01br/9uP25j\nVwDgvwgrwM8EBBhNHNRRvyoRWHKdBXr0w42az8MOAaDKEVaA8wgLdmjSkI5uT2Z2Flh6as5mzVp7\n0MbOAMD/EFaACwgJdGj8A+11Z/u6rpplSb//eJsmL9trY2cA4F8IK8BFBDoC9M972unBrglu9b8v\n+kHjFu+WZbHLLQBUNsIKcAkBAUZ/H9DabQ8WSfrX17s09oudBBYAqGSEFZsZY/obYyalpKTY3Qou\nwhij525pod/c2MytPvG7vXr+0+9VwHOEAKDSEFZsZlnWAsuyEmNiYi59MGxljNGY65vq+b4t3eof\nrjmop+duUb6zwKbOAMC3EVaAMnq4V2O9dGcbGVNc+3jTET0xc5Ny8p32NQYAPoqwAlyG+7sk6PX7\nrpYjoDixfLn9uBLf36CsXAILAFQkwgpwmW6/uq7eebCDgh3Ff4yW7jqloe+t5WnNAFCBCCtAOdx0\nVW1NGdZJYUEOV23tviQNmrxGZzNzbewMAHwHYQUop15N4/X+Q10UFRLoqm05nKKBk1brVFqOjZ0B\ngG8grAAVoHPDOM18pJtiw4NctZ3H03TvxFU6ejbLxs4AwPsRVoAK0qZejOYkdld8VIirtu90hu6Z\nsEr7T2fY2BkAeDfCClCBmteO0tyR3VU3NsxVO3I2S/dOXKVdJ9Js7AwAvBdhBahgDWtE6D+juqtR\njQhX7WRaju6buErbDrNTMQCUFWEFqAR1Y8M0Z2Q3Na8V5aolZ+bpgXdXa/3+JBs7AwDvQ1gBKknN\nqFDNTuymtvWKH6WQlpOvwVPWavnu0zZ2BgDehbACVKJqEcH68OGu6tIwzlXLynNqxLR1+nrHCRs7\nAwDvQVgBKllUaJCmj+ii3s3iXbVcZ4FGfbBB8zcfsbEzAPAOhBWgCoQFO/TukI66+aparpqzwNJT\nczZr9tqDNnYGAJ6PsAJUkZBAh956oIPuaF/XVbMs6bmPt2nK8n02dgYAno2wAlShQEeAXrunnR7o\nmuBWf3HhDr25eLcsy7KpMwDwXIQVoIoFBBj9Y0BrPdKrkVv9ta93aeyXOwksAPAzhBXABsYY/eHW\nlvp/NzRzq09culd/nr9dBQUEFgA4h7AC2MQYoydvaKrn+7Z0q89YfUBPz9uifGeBTZ0BgGchrAA2\ne7hXY/3fHW1kTHHt441HNHrWJuXmE1gAgLACeIAHuibo3/deLUdAcWL54vvjSpyxXtl5Ths7AwD7\nEVYADzGgfV299UAHBTuK/1h+++MpDZ26Vuk5+TZ2BgD2IqwAHqRP69qaPLSTQoOK/2iu2ZekByev\n0dnMXBs7AwD7EFYAD9O7WbzeH9FVkSGBrtqWQ2c1cNJqnUrLsbEzALAHYQXwQF0axenDh7sqNjzI\nVdt5PE33TVylo2ezbOwMAKoeYQXwUO3qx2pOYnfViAxx1faeztA9E1bpwJkMGzsDgKpFWAE8WPPa\nUZo7qruuiAl11Y6czdI9E1Zp94k0GzsDgKpDWAE8XKMaEZr7aA81rB7uqp1My9F9k1br+yMpNnYG\nAFWDsAJ4gbqxYfrPyO5qVivSVUvKyNX9k1Zrw4EkGzsDgMpHWAG8RM3oUM1J7K629WJctbScfA17\nb51+OJZqY2cAULkIK4AXqRYRrA8f7qrODau5amnZ+Ro6da0OJWXa2BkAVB7CCuBlokKDNH1EF3Vq\nUBxYTqblaOjUtTqTzj4sAHwPYQXwQuHBgZoytLPbHJa9pzM0Yto6ZbA1PwAfQ1gBvFRMeJDeH9FV\ndWPDXLUth1M06oMNPK0ZgE8hrABerHZMqKaP6OK20+2y3af1zLwtKiiwbOwMACoOYQXwclfWjNR7\nwzorLMjhqs3ffFR/X/SDLIvAAsD7EVYAH9A+oZreHtRBjgDjqk1dsU8Tv9trY1cAUDEIK4CPuK55\nTb16d1u32tgvdmru+kM2dQQAFYOwAviQOzvU0x9vbelWe+7jbVqy84RNHQFA+RFWAB/zSO/GSuzd\n2DV2Flh67MONbMsPwGsRVgAf9FyfFrqzfV3XODuvQCOmredJzQC8EmEF8EEBAUYv391W1zaPd9VS\nsvI0ZOpaHT2bZWNnAFB2hBXARwU5AvT2gx10df1YV+1YSraGTF2rs5m5NnYGAGVDWAF8WHhwoN4b\n1llN4iNctZ9OpmvEtHXKynXa2BkAlB5hBfBx1SKC9f5DXVU7OtRV23jwrB6fuVF5TrblB+D5CCuA\nH6gbG6bpI7ooOjTQVVuy86Se+2gbu9wC8HiEFcBPNK8dpSnDOisksPiP/UcbD+vlL3+0sSsAuDTC\nCuBHOjeM0/gH3Lfln7B0jyYvY1t+AJ6LsAL4mRtb1dJLd7Rxq/190Q/6dNMRmzoCgIsjrAB+6N7O\n9fXMzc3dak/P3aLvdp2yqSMAuDDCis2MMf2NMZNSUlLsbgV+5rFrm2hYj4aucX6BpVEfbNDmQ2ft\nawoAzoOwYjPLshZYlpUYExNjdyvwM8YY/blfK/VrW8dVy8x1asS0ddpzKt3GzgDAHWEF8GMBAUav\n3dtOPa+s4aolZeRqyJS1OpGabWNnAFCMsAL4uZBAhyYM7qg2dYuv7h05m6WhU9cqJSvPxs4AoBBh\nBYAiQwL13vDOalg93FXbeTxNj0xfr+w8tuUHYC/CCgBJUo3IEM14qKvio0JctbX7kzRm1iblsy0/\nABsRVgC41I8L17ThnRUVUrwt/393nNCf5n/PtvwAbENYAeDmqitiNGlIJwU7iv/zMGvtIf376102\ndgXAnxFWAPxC9ybV9cbAq2WKd+XXuCU/6f1V++1qCYAfI6wAOK9b2tTRi7e3dqv95bPtWrT1mE0d\nAfBXhBUAFzSoWwM9dUNT19iypP83Z7NW/nTaxq4A+BvCCoCLevL6pnqwa4JrnOssUOKMDfr+CI+I\nAFA1LjusGGPGFL2uqMiGAHgWY4xeuL21+lxV21VLz8nXsPfW6sCZDBs7A+AvynNl5d+S/imJ68GA\nj3MEGL0+8Gp1axznqp1Oz9XgKWt1Ki3Hxs4A+IPyhJXTktIsy8qtqGYAeK7QIIcmDemklnWiXbWD\nSZka9t5apWWzLT+AylOesLJRUowxJr6imgHg2aJDgzR9eGfVjwtz1bYfTdXIGRuUk8+2/AAqR3nC\nyrii8/9UQb0A8AI1o0P1/oiuqh4R7Kqt3HNGv5mzRc4CdrkFUPEuO6xYlvWFpKcljTLGzDDGtKu4\ntgB4skYK4fA8AAAgAElEQVQ1IjRteBdFBDtctUXbjulvC7azLT+AChd46UPOzxizt+in+ZIekPSA\nMSZL0hlJF7oebFmW1eRyvxOA52hTL0YTB3fS8GlrlecsDCjvrzqg+MgQjb6+6SXOBoDSK89toIZF\nr1BJpugVLql+iffO9wLgI3o2raF/3eu+Lf9rX+/SrLUH7WsKgM+57Csrkq6rsC4AeK3+7a7QmfQc\n/XXBDlftj59sU1xEsG4usTcLAFyuyw4rlmUtrchGAHivYdc00qn0HL31zR5JUoEljZ61STNGdFHX\nxtVt7g6At2O7fQAV4umbmuveTvVc49z8Aj38/nr9cCzVxq4A+IIKCyumUAtjTK+iVwtjSt7JBuDL\njDH6vzva6IaWNV21tOx8DZ3KtvwAyqfcYcUYc6UxZpqkFEnbJX1b9NouKcUY854x5sryfg8Azxfo\nCNCb93dQpwbVXLWTaTkaOGk1gQXAZStXWDHG3CZpk6TBkiJVvCro3CtS0hBJm4wx/crXKgBvEBbs\n0JShndW8VpSrdiwlWwMnrdb+0wQWAGVXnqcuN5E0W1KEpL2SRkpqKims6NVU0ihJe4qO+U/ROQB8\nXEx4kGY83EVN4iNctXOBZR+BBUAZlefKyu9UuMfKN5LaWpb1rmVZeyzLyil67bEsa5KkdpKWSgqR\n9Ez5WwbgDWpGhWpWYjddWTPSVTuemq2Bk1YRWACUSXnCyo2SLEkjLcvKutBBRe+NVOFtoZvK8X0A\nvEzNqFDNeqSbmpYILCdSc3TfxFXaeyrdxs4AeJPyhJU6klIsy/rpUgdalrVL0tmicwD4kfioEM38\nWWA5N+l2D4EFQCmUJ6xkSgo3xgRd6kBjTLAK561c8AoMAN8VHxWiWYnd1KyWe2C5f9Jq/XSSwALg\n4soTVrZJCpI0tBTHDi06dms5vg+AF6sRWXiFpeQqoZNpObr/XQILgIsrT1iZocJ5KOOMMQ+fbwM4\nY0yoMWaMpHEqnN8yvRzfB8DLFQaWrmpRuziwnCq6JfTTyTQbOwPgyYxlWZd3YmE4+VLFE22PS1om\n6YgKVwklSOoqqboKQ81/Jd1iXe4X+rhOnTpZ69evt7sNoEqcSc/Rg5PXaOfx4oBSIzJEsx7pqqYl\nrrwA8G3GmA2WZXW61HGXfWWlKHQMkDRJhWGljqR7JT2lwv1V+kqqUfTeBEl3EFQASFL1oltCJa+w\nnE4vvCW06wRXWAC4u+wrK24fYkwDSXdI6iApvqh8StJGSR9blnWw3F/i47iyAn+UnJGrByavcXvY\nYfWI4KLJuFxhAXxdaa+sVEhYQfkRVuCvkjNy9eDkNdrxs8Ay85Fual6bwAL4skq/DWSM2WiM2WCM\naXy5nwEA1SKCNfORrrrqimhX7UxGru5/d7V2Hk+9yJkA/EV5VgO1ktTUsqy9FdUMAP8UGx6sDx/u\nqtZ1iwNLUkauHnjX/RYRAP9UnrByRIWrfACg3GLDg/XBQ13Vpm6Mq1YYWFZrx1ECC+DPyhNWvlLh\nDrZdK6oZAP7tXGBpW684sCRn5unByQQWwJ+VJ6z8XdIZSROMMTUqqB8Afi4mPEgzzhNYHpi8WtuP\nptjYGQC7lGdTuN6Smkt6TVKepPclrVLhkmXnhc6zLOu7y/pCH8dqIMBdSlaehkxZoy2HiwNKTFhQ\n0dyWmIucCcBbVPrSZWNMgQo3fJMK566U5oMsy7ICL+sLfRxhBfil1Ow8DZ6yVlsOnXXVCCyA76j0\npcuSDpZ4HfjZ+EKvQ+X4PgB+Jjo0SDMe6qKr68e6ailZeXpw8hptO8wtIcBfsCmch+DKCnBhqdl5\nGjp1rTYdLL7CEh0aqA8e7qq29WIvciYAT1YVV1YAoEpEhwbp/RFd1CGhOJikZudr0OQ12nr47EXO\nBOALyrODbbIx5gw72AKoClGhQZo+oos6NqjmqqVm5+vByWvc5rQA8D3lubISLMnBDrYAqsq5wNKp\nRGBJK7rCspnAAvis8k6wDa6oRgCgNCJDAjVtRBd1blgisOTka/DkNdp0MNnGzgBUlvKElc8khRhj\nbqyoZgCgNCJDAvXe8C7q0jDOVUvLydfgKWu1kcAC+JzyhJX/k7Rf0rvGmJYV0w4AlE5hYOmsLo2K\nA0t6Tr6GTFmrDQcILIAvKc+mcEMk1ZT0ZxXeDvpCpdvB9v3L+kIfx9Jl4PJk5ORr+LR1WrsvyVWL\nDAnU9BGd1bFB3EXOBGC3qtzB9tyTl0v1QZZlOS7rC30cYQW4fJm5+RoxbZ1W7y0OLBHBjsLJuA0J\nLICnKm1YKc/W99+plAEFACpTeHCgpg7rrIemrdeqvWckSRm5Tg2durZoMi6BBfBm7GDrIbiyApRf\nVq5TI6atcwUWSQoPdmja8C5uc1sAeAZ2sAXgd8KCHZo6rLN6NKnuqmXmOjXsvbVaUyLAAPAupQ4r\nxpgxxpiHLvBepDEm+hLn/9sYM6WsDQJAWYQFOzRlaGf1vLKGq5aZ69TwaesILICXKsuVldclvXCB\n93ZLSrrAe+cMlDSsDN8HAJclLNihyUM7/SKwDHtvnVYTWACvU9bbQOYy3wOAKhUaVBhYejUtDixZ\neU4Nf2+dVu0hsADehDkrAHxWaJBD7w7ppN7N4l21rDynhk9bq5U/nbaxMwBlQVgB4NNCgxyaNLij\nflUisGTnFWjE9HVaQWABvAJhBYDPCw1yaOLgjrq2+c8CyzQCC+ANCCsA/EJokEMTBnXUdSUCS05+\nYWBZvpvAAngywgoAvxEa5NCEwR316xY1XbWc/AI9NH2dvv3xpI2dAbgYwgoAvxIS6NA7gzro+p8F\nlsT3N2jxDyds7AzAhZR6u/2iBxfmSlp5nrevUeFzhpZe5COukRTIgwzPj+32gaqVk+/U4x9u0v9K\nBJQgh9H4Bzro5qtq29gZ4D8q/KnLRWGlvCzCyvkRVoCql5tfoDGzNunL7cddtcAAo3H3t9etberY\n2BngHyrjqcvTy9EPAHic4MAAvflAez01Z7MWbT0mScovsDR61iblF1i6rd0VNncIQCpDWLEsa3hl\nNgIAdghyBOiN+65WYIDR/M1HJUnOAktPzd4kZ0GB7mhfz+YOATDBFoDfC3QE6F/3Xq27OhQHkwJL\n+s1/tug/6w/Z2BkAibACAJIkR4DRq3e31cDO9V01y5J+N2+rZq45aGNnAAgrAFAkIMDo/+5oo0Hd\nEtzqf/hkm95ftd+WngAQVgDATUCA0Yu3t9awHg3d6n+ev11Tlu+zpynAzxFWAOBnjDH6S/9WeqRX\nI7f6iwt3aOLSPTZ1BfgvwgoAnIcxRn+4taUevbaJW/2lL3bqrW9+sqkrwD8RVgDgAowx+t3NzTXm\n11e61V/96ke9/r9dKu2mmgDKh7BSAYwxkcaYvxljPjfGnDLGWMaY5+zuC0D5GWP0m5ua6zc3NnOr\nv/6/3XrtvwQWoCoQVipGDUl/ltRG0iabewFQCcZc31S/69PcrTb+m5809oudBBagkpVlu31c2DFJ\ndS3LOmqMaSiJJQOAD3rs2isVFBCgf3z+g6s28bu9ynNa+lO/ljLG2Ngd4Lu4slIBLMvKsSzrqN19\nAKh8j/RurL/0b+VWm7pin/7y2XYVFHCFBagMhBUAKKPh1zTSiwNau9XeX3VAf/z0ewILUAk8NqwY\nY6KMMbcZY140xnxhjDldNHHVMsa0KOVn1DbGvGGM2WOMyTbGnDDGLDDGXF/Z/QPwbYO7NdDYO9uo\n5J2fWWsP6tmPtspJYAEqlCfPWble0ieXe7Ixpq2kJZKqF5VSVTgRtp+kvsaYP1iWNfZn5xhJIaX8\nilzLsgoutz8A3m9glwQFOgL0zLwtOjfHdu6Gw3IWWHr1nnZyBDCHBagIHntlpchJSZ9L+pukxNKe\nZIwJk/SZCoPKJkmtLcuKkVRN0muSjKT/M8bc9LNTr5GUVcpX78v+VQHwGXd3rKd/33u1SuaSjzcd\n0VNzNivfyf/PABXBk6+sLLAs69Nzg6JVNqU1UlIDSemS+luWdUSSLMtKlfS0MaaJpAGSXpL03xLn\n7ZI0vJTfsbMM/QDwYQPa11Wgw+jJ2Ztdt4AWbDkqZ0GB3hjYXkEOT///QsCzeWxYsSzLWY7THyz6\ncea5oPIzr6owrHQwxjS3LOvHou88KWlaOb4XgJ/q1/YKOYzR6FmblF8UWD7fdlz5zo0a/0AHBQcS\nWIDL5XN/eowxUZI6Fg2/usBhqyWlFP2cybYAKsQtberonUEdFeQovif03x0n9OgHG5STX57//wL8\nm8+FFUktVTgnRZK2n++AoomxPxYNW53vmLIyxjxhjHle0hNFpeuMMc8XvWIq4jsAeL4bW9XSpMGd\n3K6kLN55Uonvb1B2HoEFuBy+GFbqlPj5xTZqO/denYscUxZPS3pR0m+LxjcVjV9U4cTeXzDGJBpj\n1htj1p86daqC2gBgt+ta1NTkIZ0UUiKwLN11Sg9PX6+sXAILUFa+GFYiSvw86yLHZRb9GFkRX2pZ\nVkPLsswFXvsvcM4ky7I6WZbVKT4+viLaAOAhejeL13vDOissyOGqLf/ptIZPW6uMnHwbOwO8jy+G\nFQDwCD2urKFpwzsrPLg4sKzem6Rh761VOoEFKDVfDCsZJX4edpHjwot+TK/EXgD4ua6Nq+v9EV0U\nGVK8+HLd/mQNnrJGqdl5NnYGeA9fDCsl56lccZHjzr13rBJ7AQB1ahinGQ91UVRocWDZdPCsBk9e\no5RMAgtwKb4YVnZKOvdgjqvOd4AxJkBS86LhjqpoCoB/a59QTR8+3FUxYUGu2pbDKXpg8molZ+Ta\n2Bng+XwurFiWlSZpfdHwxgsc1lXSueXEiyu9KQCQ1LZerGY+0lXVwosDy/ajqbr/3dU6k55jY2eA\nZ/O5sFJkZtGPDxpjzrc0+emiHzec270WAKrCVVfEaFZiN1WPCHbVdh5P0/3vrtapNAILcD4eHVaM\nMTXOveS+V0lsyfeKbuuUNFHSAUlRkhYaY1oVfV6UMeYVSXcWHfeHyv41AMDPtagdrdmJ3VQjsvgh\n77tOpGvgpFU6mZptY2eAZ/LosCLpVInXxhL1VT97L6HkSZZlZUm6XdIZSR0kbTfGpEg6K+kZFc5p\n+b1lWSUfYggAVaZprSjNGdlNtaKLA8ueUxm6b9JqHUu52BZRgP/x9LBy2SzL2iKptaRxkvZKClFh\neFkk6UbLssba2B4AqEl8pOYkdledmFBXbd/pDN03cbUOJ2de5EzAvxjLsi59FCpdp06drPXr11/6\nQAA+51BSpgZOWq0jZ4uvqNSNDdPsxG6qHxd+kTMB72aM2WBZVqdLHeezV1YAwFvUjwvXnJHdlFAi\nmBw5m6X7Jq7S/tMZFzkT8A+EFQDwAPWqFQaWRjWKH292NCVb901apT2n2Ggb/o2wYjNjTH9jzKSU\nlBS7WwFgszoxhbd+msQXB5YTqTkaOGm1dp1Is7EzwF6EFZtZlrXAsqzEmJiYSx8MwOfVig7V7MTu\nalqz+IHwp9JydPc7K7V+f5KNnQH2IawAgIeJjwrR7MRualE7ylVLzc7Xg5PX6OsdJ2zsDLAHYQUA\nPFD1yBDNeqSb2ifEumo5+QUaOWO95qw7aGNnQNUjrACAh6oWEawPH+6qX7eo6aoVWNKzH23T+CW7\nxdYT8BeEFQDwYOHBgZo4uKPu6lDPrf7P/+7SXz/bLmcBgQW+j7ACAB4uyBGgf97TVo9e28StPn3V\nAY2ZtUk5+U6bOgOqBmEFALyAMUbP9mmhP/Vr5VZftO2Yhr+3TmnZeTZ1BlQ+wgoAeJGHejbSGwOv\nVpDDuGor95zRfRNX62QaT2yGbyKsAICXuf3qupo6rLMigh2u2o5jqbrrnZVszw+fRFgBAC/Uq2m8\nZid2V43IYFftUFKW7npnpbYdZkds+BbCCgB4qTb1YjRvVA+3ByCeycjVwEmrtGz3KRs7AyoWYQUA\nvFjDGhGa92h3taoT7apl5Do1Yto6fbblqI2dARWHsAIAXq5mVKjmjOymHk2qu2p5TktjZm3S1OX7\nbOwMqBiEFQDwAVGhQXpveGf1bVvHrf7Cwh16+cud7HYLr0ZYsZkxpr8xZlJKChPiAJRPSKBDbw5s\nr6HdG7jV3/l2j56Zt1V5zgKbOgPKh7BiM8uyFliWlRgTE2N3KwB8QECA0V9vu0rP3NzcrT5vw2GN\nnLFBWbnsdgvvQ1gBAB9jjNHj112pl+9qo4DiveO0ZOdJPTB5tZIzcu1rDrgMhBUA8FH3dU7QxMGd\nFBJY/J/6TQfP6u4JK3XkbJaNnQFlQ1gBAB92Y6ta+vDhrooJC3LV9pzK0F1vr9SPx9Ns7AwoPcIK\nAPi4Tg3jNHdUd9WJCXXVjqdm654JK7Vuf5KNnQGlQ1gBAD/QrFaUPnq0h66sGemqpWbna9DkNfp6\nxwkbOwMujbACAH7iitgwzRvVXR0SYl21nPwCjZyxXrPXHrSxM+DiCCsA4Ediw4P14cPddH2Lmq5a\ngSU99/E2vbl4N5vHwSMRVgDAz4QFOzRxcEfd07GeW/21r3fpL59tl7OAwALPQlgBAD8U6AjQK3e3\n1WPXNnGrv7/qgEbP2qicfDaPg+cgrACAnzLG6Hd9Wugv/VvJlNg87vNtxzVs6jqlZufZ1xxQAmEF\nAPzc8GsaadzA9gpyFCeWVXvPaODE1TqZlm1jZ0AhwgoAQP3bXaH3hnVRRLDDVdtxLFV3vbNS+05n\n2NgZQFgBABTp2bSG5ozsrhqRwa7aoaQs3f3OSm09fNbGzuDvCCsAAJfWdWP00aM9lBAX7qqdycjV\nwEmrtWz3KRs7gz8jrAAA3DSoHqGPHu2hq66IdtUyc50aMW2d5m8+YmNn8FeEFZsZY/obYyalpKTY\n3QoAuMRHhWh2Yjddc2V1Vy3PaenJ2Zs1Zfk+GzuDPyKs2MyyrAWWZSXGxMTY3QoAuIkKDdLUYZ3V\nt20dt/qLC3do7Bc72e0WVYawAgC4oJBAh94c2F7DejR0q09YukdPz92qPGeBPY3BrxBWAAAXFRBg\n9Jf+rfTMzc3d6h9tPKzE99crMzffps7gLwgrAIBLMsbo8euu1Ct3tZUjoHjzuG9+PKUH3l2j5Ixc\nG7uDryOsAABK7d7O9TVxUEeFBBb/9bH50FndNWGl9rN5HCoJYQUAUCY3tKqlmY90VUxYkKu291SG\nbn9rhVbuOW1jZ/BVhBUAQJl1bBCneaO6q05MqKuWkpWnIVPWauaagzZ2Bl9EWAEAXJamtaL06ePX\nqF294q0X8gss/eGTbfrbgu3KZ6UQKghhBQBw2WpFh2rOyO7q97O9WN5bsV8PTV+v1Ow8mzqDLyGs\nAADKJTTIoTfvb6+nbmjqVl+665TufHulDpxh4i3Kh7ACACg3Y4yeuqGZxj/Q3m2l0E8n0zXgrRVa\nvfeMjd3B2xFWAAAVpl/bK/Sfkd1VMyrEVUvOzNOgyWs0Zx0Tb3F5CCsAgArVrn6sPnuip1rXLX5q\nc36BpWc/2qa/L9whZwHPFELZEFYAABWudkyo5o7soVvb1HarT16+Tw9PX6c0Jt6iDAgrAIBKERbs\n0Pj7O2jM9e4Tb7/58ZTuemelDiVl2tQZvA1hBQBQaQICjH5zYzONu7+9gktMvN11Il23v7VCa/cl\n2dgdvAVhBQBQ6W5rVzjxNr7ExNukjFw9OHm15q4/ZGNn8AaEFQBAlbi6fqw+e+IaXXVF8cTbPKel\nZ+Zt1Uuf/8DEW1wQYQUAUGXqxIRp7qju6nOV+8Tbid/t1cgZ65Wek29TZ/BkhBWbGWP6G2MmpaSk\n2N0KAFSJ8OBAvf1gBz1x3ZVu9f/9cFJ3M/EW50FYsZllWQssy0qMiYm59MEA4CMCAoyevrm5Xr/v\nareJtzuPp2nAWyu0fj8Tb1GMsAIAsM2A9nU1O7GbakQWT7w9k5GrB95do482HLaxM3gSwgoAwFYd\nEqpp/hPXqGWd4om3uc4C/XbuFo39YqcKmHjr9wgrAADb1Y0N07xR3XVjq1pu9QlL92jkBxuUwcRb\nv0ZYAQB4hIiQQE0c1FGPXtvErf71jhO6e8IqHTmbZVNnsBthBQDgMQICjJ7t00L/uredgh3Ff0X9\ncCxVt49frg0Hkm3sDnYhrAAAPM6dHeppVmJXVY8IdtVOp+fq/ndX69NNR2zsDHYgrAAAPFLHBnGa\n/8Q1alE7ylXLzS/QU3M269WvmHjrTwgrAACPVa9auOY92kM3tKzpVn/rmz169MMNysxl4q0/IKwA\nADxaZEigJg7upJG/auxW/2r7Cd39ziodZeKtzyOsAAA8niPA6Pe3tNSrd7dVkMO46juOpeq28Su0\n6SATb30ZYQUA4DXu6VRfMx/ppji3ibc5um/Sas3fzMRbX0VYAQB4lc4N4zT/8WvUrFakq5abX6An\nZ2/Wv/77IxNvfRBhBQDgderHheujR3vo1y3cJ96OW/KTHp+5kYm3PibQ7gZQwb54Tjq+ze4uAKDS\nRUmaIksH4zN1LCW7+I1d0p5XHGpeO9ptYzlUgNptpFvGVvnXElZ8zfFt0oHldncBAFXCSGogqcHP\nM0m+JB7a7DOInAAAwKNxZcXX1G5jdwcAYJvsfKd+PJ6mrDynW71ubJjqVQuTkbnAmSgVm/6OIaz4\nGhvuJQKApwiV1Cg7T6NnbtLSXaeK3zgp3RxfS6/e007RoUG29YfLw20gAIBPiQ4N0pShnTTimkZu\n9a+2n1D/N5dr+9EUmzrD5SKsAAB8TqAjQH/u30ov3dlGgQHFt34OnMnUnW+v1H/WHbKxO5QVYQUA\n4LPu75KgOSO7q05MqKuWk1+g3320Vc/M3aKsXOdFzoanIKwAAHxaxwbVtGhML/VqWsOtPnfDYd3x\n9grtO51hU2coLcKKzYwx/Y0xk1JSuIcKAJUlLiJY04Z30VM3NJUpsSBo5/E03fbmcn35/TH7msMl\nEVZsZlnWAsuyEmNiYuxuBQB8miPA6Kkbmmn68C6qFl68IigtJ1+jPtiovy/coTxngY0d4kIIKwAA\nv9K7WbwWjeml9gmxbvXJy/fp/kmrdbzk1v3wCIQVAIDfuSI2THMSu2v4NQ3d6usPJKvvuGVavvu0\nPY3hvAgrAAC/FBwYoL/0v0pvPdBBEcEOV/1MRq4GT12jcYt3q6DAsrFDnENYAQD4tb5t6+iz0T3V\nvFaUq2ZZ0r++3qXh09YpOSPXxu4gEVYAAFCT+Eh9+vg1urNDXbf60l2n1HfcMm06mGxTZ5AIKwAA\nSJLCgh167Z52GntnGwUHFv/1eDQlW/dOXKXpK/fLsrgtZAfCCgAARYwxGtglQR8/2kMJceGuep7T\n0l8+264xszcrPSffxg79E2EFAICfaV03RgtG99SNrWq51RdsOarbxy/XrhNpNnXmnwgrAACcR0xY\nkCYN7qjf39JCjhIPQ9xzKkO3j1+hTzcdsbE7/0JYAQDgAowxGvmrJpr5cFfFR4W46ll5Tj01Z7P+\n+Mk2ZefxMMTKRlgBAOASujaurkVjeqpb4zi3+odrDuqeCat0KCnTps78A2EFAIBSqBkVqg8e6qrH\nrm3iVt92JEX93lyuxT+csKkz30dYAQCglAIdAfpdnxaaMrSTokMDXfWUrDw9NH29Xvlyp/J5GGKF\nI6wAAFBG17espUVjeqlN3Ri3+tvf7tGgKWt0Mo2HIVYkwgoAAJehfly45o7qrkHdEtzqq/cmqd+4\n5Vqz94xNnfkewgoAAJcpNMihvw9oo9fvu1phQcUPQzyZlqMHJq/RxKV72PW2AhBWAAAopwHt6+qz\nJ65Rk/gIV81ZYOmlL3YqccYGpWTl2did9yOsAABQAZrWitJnT/RU/3ZXuNW/3nFC/d5cpu+PpNjU\nmfcjrAAAUEEiQgI1buDVeuH2qxTkKN719lBSlu58Z6VmrT3IbaHLQFgBAKACGWM0pHtDzR3VQ3Vj\nw1z13PwC/f7jbfrt3C3KymXX27IgrAAAUAmurh+rhaN76trm8W71jzce0YC3VmjvqXSbOvM+hBUA\nACpJtYhgTR3aWb+9sZlM8V0h/XgiTbeNX6FFW4/Z15wXIawAAFCJAgKMRl/fVDNGdFX1iGBXPT0n\nX4/P3Ki/Ldiu3Hx2vb0YwgoAAFWgZ9MaWjSmlzo1qOZWf2/Fft03aZWOns2yqTPPR1gBAKCK1I4J\n1azEbnqkVyO3+qaDZ9V33DJ9+f1xmzrzbIQVAACqUJAjQH/s20oTBnVQVEjxwxCTM/M06oMNenbe\nVqXn5NvYoechrNjMGNPfGDMpJYXNggDAn/RpXUcLRvdUyzrRbvU56w/p1jeWacOBZJs68zyEFZtZ\nlrXAsqzEmJiYSx8MAPApDWtE6JPHemj4NQ3d6geTMnXPhJX6139/VJ6TybeEFQAAbBQa5NBf+l+l\nGQ91Ua3oEFe9wJLGLflJd7+z0u/3ZCGsAADgAXo1jddXT/XWrW1qu9W3HE5R33HL9eGaA367VT9h\nBQAADxEbHqy3Huig1+5pp8gSk2+z8pz64yff6+Hp63UqLcfGDu1BWAEAwIMYY3RXx3r64sle6tzQ\nfU+WxTtPqs/r3+l/O07Y1J09CCsAAHig+nHhmp3YXb/r01yBAcV79Z/JyNXD76/X7z/epsxc/1ji\nTFgBAMBDOQKMHrv2Sn36+DVqEh/h9t6stQfVd9xybT501qbuqg5hBQAAD9e6bowWju6lod0buNX3\nnc7QXe+s1Bv/2618H17iTFgBAMALhAU79LfbW2v6iC6Kjype4uwssPTv/+3SPRNXaf/pDBs7rDyE\nFQAAvMivmhUuce5zlfsS500Hz+rWccs0e+1Bn1viTFgBAMDLxEUE651BHfTK3W0VEexw1TNznXru\n421KnLFBZ9J9Z4kzYQUAAC9kjNG9nerriyd7q2MD9yXOX+84oZtfX6Zvdp60qbuKRVgBAMCLJVQP\n15zEbnr6pmZuS5xPp+do+LR1+tOn3ysr12ljh+VHWAEAwMsFOgL0xK+b6uPHeqhxDfclzjNWH1Df\nN1yxmwQAABCSSURBVJdp62HvXeJMWAEAwEe0rRerhWN6alC3BLf63lMZuvPtlRq/ZLecBd43+Zaw\nAgCADwkPDtTfB7TR1GGdVCMy2FXPL7D0z//u0n0TV+lQUqaNHZYdYQUAAB/06xa19NVTvXVjq1pu\n9fUHktXn9e80d/0hr1niTFgBAMBHVY8M0aTBHTX2zjYKL7HEOSPXqWfmbdWjH2xUckaujR2WDmEF\nAAAfZozRwC4J+nxML11dP9btvS+3H9fNr3+npbtO2dRd6RBWAADwAw1rRGjeqO566oamcpRY4nwy\nLUdDp67VXz/bruw8z1ziTFgBAMBPBDoC9NQNzTRvVHc1rB7u9t60lfvV783l+v5Iik3dXRhhBQAA\nP9M+oZoWjeml+7u4L3H+6WS67nh7hd75do9HLXEmrAAA4IciQgL10p1t9O6QTqoeUbzEOc9p6eUv\nd+r+Sas9ZokzYQUAAD92Y6ta+vKp3vp1i5pu9bX7k3TrG8v08cbDti9xJqwAAODn4qNCNGVoJ/19\nQGuFBhVHg7ScfP3mP1v0xKxNOptp3xJnwgoAAJAxRoO6NdCiMb3Utl6M23uLth5Tn9eXafnu07b0\nRlgBAAAuTeIj9dGjPTTm11eqxApnHU/N1qApazR95f4q74mwAgAA3AQ5AvSbm5pr7qjuSogrXuIc\nFuRQr//f3r1HzVWVdxz//kgMJOROgokKSVAgIBSBgJF6CSiISCjWYlW6aKAldC1pk1VcyK3WytIo\nLIVUqMrNSFXsDalc1DbctJJgJYBLMDQqAbQQrrmQC6A8/WPv0/f4MjPv/T1n5v191pp1Zp+zz8zz\nnvXOnGf22XufvacNezxOVszMzKyhQ2dN5ZYlb+MD814HwAXH78de08cPexyjh/0dzczMrG2M33k0\nF/3RQZw0bw/mzZpSSQxOVszMzKxHh82eWtl7+zKQmZmZ1ZqTFTMzM6s1JytmZmZWa05WzMzMrNac\nrJiZmVmtOVkxMzOzWnOyUjFJCyVdsWnTpqpDMTMzqyUnKxWLiBsjYvGkSZN6rmxmZjYCOVkxMzOz\nWnOyYmZmZrXmZMXMzMxqzcmKmZmZ1ZqTFTMzM6s1JytmZmZWa05WzMzMrNacrJiZmVmtKSKqjsEA\nSU8BjwziS04Dnh7E1xupfBwHzsdw4HwMB87HcOCG4hjOiojpPVVystKhJP04IuZVHUe783EcOB/D\ngfMxHDgfw4Gr8hj6MpCZmZnVmpMVMzMzqzUnK53riqoD6BA+jgPnYzhwPoYD52M4cJUdQ/dZMTMz\ns1pzy4qZmZnVmpMVMzMzqzUnK2ZmZlZrTlY6iKQZkpZL+oWkHZI2SLpR0jurjq0dSNpT0tJ8zB6V\n9IKkLZLul/QZSTOrjrHdSBov6TFJkR+Lqo6pnUjaV9IXJD0kaaukTZJ+JukaSe+oOr46k7STpFMl\nrZT0lKSXJG2UdLek8yVNqDrGqkmaIOkESRdK+o6kp0uf1bm92H+MpLMl3Sfp+Xx8V0laLEmDGqs7\n2HYGSb8H3AbslldtBsaTEtIAzouIz1QUXu1J2oM0g3D5A7YZ2BUYlcvPAe+PiNuHOby2JelSYElp\n1akRsaKicNqKpL8CLgbG5FXPA6OBXXL56oj48ypiqztJ44AbgaNKqzcBE+n6jD8CHBURvxzm8GpD\n0onAt5ps3i8i1rbYdyLpnHNoXrWN9P9Z/L/eBLwvIn4zGLG6ZaUDSBoLfJuUqNwLHBARk4ApwOdI\nH85PSzqmuihrr0hIbgZOAqbmYzgOOA54mHQ8b5A0o5oQ24ukQ4AzgburjqXdSDoDWE768v8saUry\nCRExFpgJnALcVWGIdfc3pEQlgHOByRExmZTofQjYCMwCrqoswvp4ErgF+DtgcR/2u5KUqDwLLCT9\nOB4HLAJ2AMfn1xwUblnpAJKWApeQfnnNjYhfd9v+LeBEYE1EHNrgJUY8SZOA2RFxf5Ptc0mJ4C7A\nJyJi0D6EnUjSTqQk5WDgMGBN3uSWlR5Img08QPriXxwRV1YaUBuS9AiwJ3BNRPxZg+2LgK/k4tSI\neG4Yw6sNSaMi4rel8mzSDzNo0bIi6WC6PtN/EBHf7rZ9CXApsJ30vfrkQGN1y0pnODkvv9E9Ucku\nzstDJO07TDG1lYjY1CxRydvXAqtz0Qlfz/4SmAd8MSLurTqYNrOElKjc7USl316dl83+9+4pPR83\nxLHUVjlR6aMP5+VD3ROV7ArSZbexwB/28z1+h5OVNpc7iRUnz+81qbaa9I8D4M62/fdMXo5qWWuE\nk/Ra4EJgA3BBxeG0o+JEcF2lUbS39Xl5cJPtxXfmhiY/8Ky1I/PyPxptjIjtwA9y8ahGdfrKyUr7\n24+uDmMPNKoQES8DD+Xi/sMRVKeRNBr4/Vz8aZWxtIEvABOAj0bEpp4qWxdJrwd2z8V7Jc3Po9Oe\nkbRd0lpJF0vavdXrGEWL1KmSzsmXeYvRK39MumwewEerCrBd5VE+xUihhuec7MG8HJRzjpOV9lce\nTvu/LeoV2zz8tn8+AswAXga+WnEstSVpIfA+4I6I+FrV8bShvUvPFwD/Reqo+CrSyXVf0gn2Pklv\nHPbo2selwOWkH3LLgI2SNpL6UHwTWAuc4P/RfplIGiUJw3jOcbLS/nYtPd/eot62vBw/hLF0pDws\nfFkuXhYRD7aqP1JJ2hW4DHiJlNxZ300uPf9b4H+A+RExkfTZPY40emMm8G+5xc+6yX0xlgJnAcXQ\n2Ul0nfMmANMrCK0TVHLOcbJi1kKeCO4GUkexe4CPVRtRrX2SNALjEid0/Vb+Tg7SPBV3Q7qcGxHf\nAU7L2/dlkDovdpo8vcAPSVM3fB04iHTS3Js0lHkv4BpJy5q+iNWKk5X2t7X0fGyLekWP9+eHMJaO\nImkqqQPZHGAd8N6I2FFtVPUk6U2kUSyPkZIW65/y5/O7EfFQ9woRcTOpxQXcYb6Za4HDSRPnLYqI\nn0TE1oj4eZ4c84xc72xfTuuzSs45TlbaX/ma4Wta1Cu2PT6EsXSM3CHve8ABwKPAuyJiQ7VR1dpy\n0iip80l98MaXH6V6O+d1I3a4aA/Kn+dXJCoNtu0xhLG0JUn7A0fn4iWN6kTEP5JG9+1EmtDMem8z\nXQnLsJ1znKy0v7Wk5mKAhr8Q8gRdxfwqbp7vQe57cQtpnpAnSInKo9VGVXuz8vJaYEuDR+FLuez/\nw8YeJHXi7i3P6vlK+5WeP9y0FhTT7M8eulA6T6SZZH+Wi61apYpRQIPyWXey0uYiYgvw41w8ukm1\nN5M6lwHcOuRBtbF864IbgSNIv7zeFRHrqo3KRoqI2AasysVWEzgW29YPaUDtqZzs7dmiXpFgb2lR\nxxor7o/W8JwjaRfgbbk4KOccJyud4Rt5eXKTOwMXcwnc0+gauCWSxgDXkyY82ggcExGt5hGwLCJm\nR4SaPUpVT83rZlcVaxu4Ni+PbTTjtKT3Avvk4i3DFlX7KM9EfXqjCnmIfTFXje9d1XfFhIVzJR3f\nYPvppB/I22l+o8Q+cbLSGb5MuoPoBOCmfM22uP33RXSNGDivovhqT9IoUtJ3LOmX1nsiYk3rvcyG\nxDWkpvNRwPWSDod0OVfSscDVud5qnKy8Qr6LcjGz6lJJy4pJ9HJ/qUXAirx9PekmsCOWpGnFg3Sz\n1sLk8rbcnQCAfAuNf87FFZKOy681StIppJtvQhoZOOD7AoFvZNgxJB1Eam7bLa/aTBqqtxPpuvZ5\nuRe8NSDp7cCdubiDrtsTNPJYRBw29FF1DknFF41vZNgLkvYC7qCrA+0WUvJSdEx+kNTy56niG8gt\nzLfyu/1XtpB+0BU2kH6UjOh7V5U+mz2ZExHrS/tNBG6j69YF20j/ozvn8k2kofe/YRC4ZaVD5Jvw\nHQD8Panj2M6kPhc3A0c7UelR+bOwC+lGaM0enkzKhlRuHTgQ+BQpMRlN+tGxhjRPyOFOVJqLiMdJ\nJ9GlwPeBZ0mJ3mbSMbwQOHCkJyoDERGbSX37ziFdegvgBVKL3xmkGYIHJVEBt6yYmZlZzbllxczM\nzGrNyYqZmZnVmpMVMzMzqzUnK2ZmZlZrTlbMzMys1pysmJmZWa05WTEzM7Nac7JiZmZmteZkxczM\nzGptdNUBmJn1RNJo4E+ADwIHke6BtRV4gnR7iR8At0XEj0r7vAk4EVjv+xGZtTdPt29mtSZpOunu\nwvNKq3eQ7kMyEVBetykiJpf2WwR8BbgzIhYMS7BmNiR8GcjM6u5rpERlC3A2MDMixubEZBJwNPAP\nwMbqQjSzoeTLQGZWW5LmAsfk4mkR8a/l7RGxBVgJrJR01nDHZ2bDwy0rZlZnB5ae39SqYkTsKJ5L\nCtIlIIB3SIpujwXd95f0VknflPQrSS9IekbSSkkfkqQG9Rfk11qfywsl3S7pOUnPS1ol6cP9+JvN\nrBu3rJhZu3gt8Ite1t0AjCX1aXkJeLbb9hfLBUmfJV1iKmwGpgDvzI8TJJ0cES83ejNJS4FLgAA2\n5feeD8yXdEREnNnLuM2sAbesmFmd3VN6fnnubNujiJgBLMnFuyJiRrfHXUVdSUtIicoGYDEwOSIm\nAbuSRh89kZcfa/J204GLgGtJ/WmmANOAz+XtH3ELi9nAeDSQmdWapK8Cp+Tii6RhyquB/yYlIk81\n2W8RPYwGkjQZeIzUyjw/Iu5vUOctwA9JHXhnRMSLef0C4PZc7T+Bd0e3L1RJK4A/BX4O7NN9u5n1\njltWzKzuTgc+T0pUxpAuy5wP3AA8KelHkk5u1K+kF94PjAdWNkpUACJiFfAw6bLQoU1eZ1mTRORT\nefkG0vwwZtYPTlbMrNYi4sWIOAvYA/gL4DpgHal/CMBhpOHN/ySpr99pR+TlUZKeaPbI701pWfYS\nqeWlUezrgMdz8ZA+xmZmmTvYmllbiIgngS/nB5JeDSwEPk5KIk4iJQ3L+/CyM/NyXH70pFGdp4tL\nQ038Or9Pr/rbmNkruWXFzNpSRGyIiKtILRYb8urT+vgyxXfg8ohQLx4rBit+M+s9Jytm1tYi4mng\n33Nxnz7uXiQ5ew4ghGmSxrTY/pq8bNgR2Mx65mTFzDrB1rwsX44p5kRp1fF2VV4ukDS2n+/9KuAt\njTZIegNdycqafr6+2YjnZMXMakvSHEmv76HOONLdlQHuK23anJeTae5fSInOFFLfl1bvM6XF5nOb\njEY6Ny/XRcR9DbabWS84WTGzOnsj8JCk6yV9QFLRIRZJu0paSJp3ZU5eXe5c+0Be7i/pzY1ePCKe\noSuhOEfSlZL+/1KSpHGSjpR0BXBXo9cAtpGGU18tafe83+Q8K27Rh+YTvfx7zawBTwpnZrUl6d3A\nd7ut3k663DOptO63wMcj4tPd9r8TeHsuPku6czPAByNidaneBcAn6bpktDW/x+TSuvURMae0zwLS\npHCPAJfSNd3+xhxb8WPwck+3bzYwTlbMrNZyS8dC4K3AAaR7BI0hJR6/BL4PXBURDzTYdzdSEvKe\n0n4AR0bEHd3qHgicCRwJvA4YReoU+1PgVuC6iPhVqf4CcrISEbNzK89fAweT+rH8BLgsIr4+4INg\nNsI5WTEz64fuyUq10Zh1NvdZMTMzs1pzsmJmZma15mTFzMzMas3JipmZmdWaO9iamZlZrbllxczM\nzGrNyYqZmZnVmpMVMzMzqzUnK2ZmZlZrTlbMzMys1v4PdS8xhOQ4+wEAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def line_search(x, d):\n", " return gss(lambda step: log_likelihood(x+step*d, data, labels), -1, 1)\n", "\n", "ws_bfgs = bfgs(init, gradient, hessian, line_search, 10)\n", "error_plot([log_likelihood(w, data, labels) for w in ws_bfgs])\n", "plt.plot(range(len(ws_bfgs)), [baseline]*len(ws_bfgs),\n", " label='sklearn', **kwargs)\n", "_ = plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that BFGS with line search gets pretty close to Newton's method in iteration count, but each iteration is now much cheaper (in terms of the data dimension)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Hessian-vector products" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Recall that we can implement the Hessian inverse (approximately or exactly) by just using Hessian vector products. This follows from the Krylov methods for solving linear equations.\n", "\n", "This raises the question how we can, in general, compute Hessian-vector products of possibly complicated functions for which we have no exact formula for the Hessian.\n", "\n", "Luckily, this problem is also solved by automatic differentiation. Let's import `autograd` first and define some function. For simplicity we'll reuse the log-likelihood we saw before." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import autograd.numpy as np\n", "from autograd import grad\n", "\n", "np.random.seed(1337)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def log_likelihood(weights, data, labels):\n", " \"\"\"Normalized negative log likelihood.\"\"\"\n", " scores = np.dot(data, weights)\n", " return -np.mean(labels*scores - np.log(1 + np.exp(scores)))" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": true }, "outputs": [], "source": [ "m, n = 1000, 100\n", "data = np.random.normal(0, 1, (m, n))\n", "wopt = np.random.normal(0, 1, n)\n", "labels = (data.dot(wopt) > 0).astype(np.float)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here's now how to get Hessian vector products with just two invocations of automatic differentiation." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": true }, "outputs": [], "source": [ "objective = lambda w: log_likelihood(w, data, labels)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": true }, "outputs": [], "source": [ "vector = np.random.normal(0, 1, n)\n", "hvprod = grad(lambda x: np.dot(vector, grad(objective)(x)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's verify that `hvprod` correctly implements the product of the Hessian at a given point with `vector`." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def log_likelihood_hessian(weights, data, labels):\n", " \"\"\"Hessian of negative log likelihood.\"\"\"\n", " predictions = logistic_model(weights, data)\n", " diag = np.diag(predictions * (1 - predictions))\n", " return data.T.dot(diag.dot(data))/len(data)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "H = log_likelihood_hessian(wopt, data, labels)\n", "np.allclose(hvprod(wopt), H.dot(vector))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Adult data set\n", "\n", "Let's see how this all plays out on a more realistic data set, the [UCI adult data set](https://archive.ics.uci.edu/ml/datasets/adult). We'll first preprocess the data in a standard way." ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import pandas as pd\n", "import sklearn as skl\n", "import sklearn.preprocessing as preprocessing\n", "import sklearn.linear_model as linear_model\n", "import sklearn.metrics as metrics\n", "import sklearn.tree as tree" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": true }, "outputs": [], "source": [ "features = [\"Age\", \"Workclass\", \"fnlwgt\", \"Education\", \"Education-Num\", \"Martial Status\",\n", " \"Occupation\", \"Relationship\", \"Race\", \"Sex\", \"Capital Gain\", \"Capital Loss\",\n", " \"Hours per week\", \"Country\", \"Target\"] \n", "\n", "# Source: https://archive.ics.uci.edu/ml/datasets/adult\n", "original_train = pd.read_csv(\"adult.data.txt\", names=features, sep=r'\\s*,\\s*', engine='python', na_values=\"?\")\n", "original_test = pd.read_csv(\"adult.test.txt\", names=features, sep=r'\\s*,\\s*', engine='python', na_values=\"?\", skiprows=1)\n", "\n", "num_train = len(original_train)\n", "original = pd.concat([original_train, original_test])\n", "labels = original['Target']\n", "labels = labels.replace('<=50K', 0).replace('>50K', 1)\n", "labels = labels.replace('<=50K.', 0).replace('>50K.', 1)\n", "\n", "# Redundant column\n", "del original[\"Education\"]\n", "del original[\"Target\"]\n", "\n", "def data_transform(df):\n", " binary_data = pd.get_dummies(df)\n", " feature_cols = binary_data[binary_data.columns[:-2]]\n", " scaler = preprocessing.StandardScaler()\n", " data = pd.DataFrame(scaler.fit_transform(feature_cols), columns=feature_cols.columns)\n", " return data\n", "\n", "data = data_transform(original)\n", "train_data = data[:num_train]\n", "train_labels = labels[:num_train]\n", "test_data = data[num_train:]\n", "test_labels = labels[num_train:]" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(32561, 16281)" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(train_data), len(test_data)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
AgeWorkclassfnlwgtEducation-NumMartial StatusOccupationRelationshipRaceSexCapital GainCapital LossHours per weekCountry
039State-gov7751613Never-marriedAdm-clericalNot-in-familyWhiteMale2174040United-States
150Self-emp-not-inc8331113Married-civ-spouseExec-managerialHusbandWhiteMale0013United-States
238Private2156469DivorcedHandlers-cleanersNot-in-familyWhiteMale0040United-States
353Private2347217Married-civ-spouseHandlers-cleanersHusbandBlackMale0040United-States
428Private33840913Married-civ-spouseProf-specialtyWifeBlackFemale0040Cuba
537Private28458214Married-civ-spouseExec-managerialWifeWhiteFemale0040United-States
649Private1601875Married-spouse-absentOther-serviceNot-in-familyBlackFemale0016Jamaica
752Self-emp-not-inc2096429Married-civ-spouseExec-managerialHusbandWhiteMale0045United-States
831Private4578114Never-marriedProf-specialtyNot-in-familyWhiteFemale14084050United-States
942Private15944913Married-civ-spouseExec-managerialHusbandWhiteMale5178040United-States
1037Private28046410Married-civ-spouseExec-managerialHusbandBlackMale0080United-States
1130State-gov14129713Married-civ-spouseProf-specialtyHusbandAsian-Pac-IslanderMale0040India
1223Private12227213Never-marriedAdm-clericalOwn-childWhiteFemale0030United-States
1332Private20501912Never-marriedSalesNot-in-familyBlackMale0050United-States
1440Private12177211Married-civ-spouseCraft-repairHusbandAsian-Pac-IslanderMale0040NaN
1534Private2454874Married-civ-spouseTransport-movingHusbandAmer-Indian-EskimoMale0045Mexico
1625Self-emp-not-inc1767569Never-marriedFarming-fishingOwn-childWhiteMale0035United-States
1732Private1868249Never-marriedMachine-op-inspctUnmarriedWhiteMale0040United-States
1838Private288877Married-civ-spouseSalesHusbandWhiteMale0050United-States
1943Self-emp-not-inc29217514DivorcedExec-managerialUnmarriedWhiteFemale0045United-States
2040Private19352416Married-civ-spouseProf-specialtyHusbandWhiteMale0060United-States
2154Private3021469SeparatedOther-serviceUnmarriedBlackFemale0020United-States
2235Federal-gov768455Married-civ-spouseFarming-fishingHusbandBlackMale0040United-States
2343Private1170377Married-civ-spouseTransport-movingHusbandWhiteMale0204240United-States
2459Private1090159DivorcedTech-supportUnmarriedWhiteFemale0040United-States
2556Local-gov21685113Married-civ-spouseTech-supportHusbandWhiteMale0040United-States
2619Private1682949Never-marriedCraft-repairOwn-childWhiteMale0040United-States
2754NaN18021110Married-civ-spouseNaNHusbandAsian-Pac-IslanderMale0060South
2839Private3672609DivorcedExec-managerialNot-in-familyWhiteMale0080United-States
2949Private1933669Married-civ-spouseCraft-repairHusbandWhiteMale0040United-States
..........................................
1625181NaN2671111Married-civ-spouseNaNHusbandWhiteMale2936020United-States
1625260Private11790911Married-civ-spouseProf-specialtyHusbandWhiteMale7688040United-States
1625339Private22964713Never-marriedTech-supportNot-in-familyWhiteFemale0166940United-States
1625438Private14934714Married-civ-spouseProf-specialtyHusbandWhiteMale0050United-States
1625543Local-gov2315714Married-civ-spouseExec-managerialHusbandWhiteMale0190250United-States
1625623Private939779Never-marriedMachine-op-inspctOwn-childWhiteMale0040United-States
1625773Self-emp-inc15969110DivorcedExec-managerialNot-in-familyWhiteFemale0040United-States
1625835Private17696710Married-civ-spouseProtective-servHusbandWhiteMale0040United-States
1625966Private3444369WidowedSalesOther-relativeWhiteFemale008United-States
1626027Private43034010Never-marriedSalesNot-in-familyWhiteFemale0045United-States
1626140Private20216815Married-civ-spouseProf-specialtyHusbandWhiteMale15024055United-States
1626251Private827209Married-civ-spouseCraft-repairHusbandWhiteMale0040United-States
1626322Private26962310Never-marriedCraft-repairOwn-childWhiteMale0040United-States
1626464Self-emp-not-inc1364059WidowedFarming-fishingNot-in-familyWhiteMale0032United-States
1626550Local-gov13934714Married-civ-spouseProf-specialtyWifeWhiteFemale0040NaN
1626655Private2246559SeparatedPriv-house-servNot-in-familyWhiteFemale0032United-States
1626738Private24754711Never-marriedAdm-clericalUnmarriedBlackFemale0040United-States
1626858Private29271012DivorcedProf-specialtyNot-in-familyWhiteMale0036United-States
1626932Private1734499Married-civ-spouseHandlers-cleanersHusbandWhiteMale0040United-States
1627048Private2855709Married-civ-spouseAdm-clericalHusbandWhiteMale0040United-States
1627161Private896869Married-civ-spouseSalesHusbandWhiteMale0048United-States
1627231Private4401299Married-civ-spouseCraft-repairHusbandWhiteMale0040United-States
1627325Private3509779Never-marriedOther-serviceOwn-childWhiteFemale0040United-States
1627448Local-gov34923014DivorcedOther-serviceNot-in-familyWhiteMale0040United-States
1627533Private24521113Never-marriedProf-specialtyOwn-childWhiteMale0040United-States
1627639Private21541913DivorcedProf-specialtyNot-in-familyWhiteFemale0036United-States
1627764NaN3214039WidowedNaNOther-relativeBlackMale0040United-States
1627838Private37498313Married-civ-spouseProf-specialtyHusbandWhiteMale0050United-States
1627944Private8389113DivorcedAdm-clericalOwn-childAsian-Pac-IslanderMale5455040United-States
1628035Self-emp-inc18214813Married-civ-spouseExec-managerialHusbandWhiteMale0060United-States
\n", "

48842 rows × 13 columns

\n", "
" ], "text/plain": [ " Age Workclass fnlwgt Education-Num Martial Status \\\n", "0 39 State-gov 77516 13 Never-married \n", "1 50 Self-emp-not-inc 83311 13 Married-civ-spouse \n", "2 38 Private 215646 9 Divorced \n", "3 53 Private 234721 7 Married-civ-spouse \n", "4 28 Private 338409 13 Married-civ-spouse \n", "5 37 Private 284582 14 Married-civ-spouse \n", "6 49 Private 160187 5 Married-spouse-absent \n", "7 52 Self-emp-not-inc 209642 9 Married-civ-spouse \n", "8 31 Private 45781 14 Never-married \n", "9 42 Private 159449 13 Married-civ-spouse \n", "10 37 Private 280464 10 Married-civ-spouse \n", "11 30 State-gov 141297 13 Married-civ-spouse \n", "12 23 Private 122272 13 Never-married \n", "13 32 Private 205019 12 Never-married \n", "14 40 Private 121772 11 Married-civ-spouse \n", "15 34 Private 245487 4 Married-civ-spouse \n", "16 25 Self-emp-not-inc 176756 9 Never-married \n", "17 32 Private 186824 9 Never-married \n", "18 38 Private 28887 7 Married-civ-spouse \n", "19 43 Self-emp-not-inc 292175 14 Divorced \n", "20 40 Private 193524 16 Married-civ-spouse \n", "21 54 Private 302146 9 Separated \n", "22 35 Federal-gov 76845 5 Married-civ-spouse \n", "23 43 Private 117037 7 Married-civ-spouse \n", "24 59 Private 109015 9 Divorced \n", "25 56 Local-gov 216851 13 Married-civ-spouse \n", "26 19 Private 168294 9 Never-married \n", "27 54 NaN 180211 10 Married-civ-spouse \n", "28 39 Private 367260 9 Divorced \n", "29 49 Private 193366 9 Married-civ-spouse \n", "... ... ... ... ... ... \n", "16251 81 NaN 26711 11 Married-civ-spouse \n", "16252 60 Private 117909 11 Married-civ-spouse \n", "16253 39 Private 229647 13 Never-married \n", "16254 38 Private 149347 14 Married-civ-spouse \n", "16255 43 Local-gov 23157 14 Married-civ-spouse \n", "16256 23 Private 93977 9 Never-married \n", "16257 73 Self-emp-inc 159691 10 Divorced \n", "16258 35 Private 176967 10 Married-civ-spouse \n", "16259 66 Private 344436 9 Widowed \n", "16260 27 Private 430340 10 Never-married \n", "16261 40 Private 202168 15 Married-civ-spouse \n", "16262 51 Private 82720 9 Married-civ-spouse \n", "16263 22 Private 269623 10 Never-married \n", "16264 64 Self-emp-not-inc 136405 9 Widowed \n", "16265 50 Local-gov 139347 14 Married-civ-spouse \n", "16266 55 Private 224655 9 Separated \n", "16267 38 Private 247547 11 Never-married \n", "16268 58 Private 292710 12 Divorced \n", "16269 32 Private 173449 9 Married-civ-spouse \n", "16270 48 Private 285570 9 Married-civ-spouse \n", "16271 61 Private 89686 9 Married-civ-spouse \n", "16272 31 Private 440129 9 Married-civ-spouse \n", "16273 25 Private 350977 9 Never-married \n", "16274 48 Local-gov 349230 14 Divorced \n", "16275 33 Private 245211 13 Never-married \n", "16276 39 Private 215419 13 Divorced \n", "16277 64 NaN 321403 9 Widowed \n", "16278 38 Private 374983 13 Married-civ-spouse \n", "16279 44 Private 83891 13 Divorced \n", "16280 35 Self-emp-inc 182148 13 Married-civ-spouse \n", "\n", " Occupation Relationship Race Sex \\\n", "0 Adm-clerical Not-in-family White Male \n", "1 Exec-managerial Husband White Male \n", "2 Handlers-cleaners Not-in-family White Male \n", "3 Handlers-cleaners Husband Black Male \n", "4 Prof-specialty Wife Black Female \n", "5 Exec-managerial Wife White Female \n", "6 Other-service Not-in-family Black Female \n", "7 Exec-managerial Husband White Male \n", "8 Prof-specialty Not-in-family White Female \n", "9 Exec-managerial Husband White Male \n", "10 Exec-managerial Husband Black Male \n", "11 Prof-specialty Husband Asian-Pac-Islander Male \n", "12 Adm-clerical Own-child White Female \n", "13 Sales Not-in-family Black Male \n", "14 Craft-repair Husband Asian-Pac-Islander Male \n", "15 Transport-moving Husband Amer-Indian-Eskimo Male \n", "16 Farming-fishing Own-child White Male \n", "17 Machine-op-inspct Unmarried White Male \n", "18 Sales Husband White Male \n", "19 Exec-managerial Unmarried White Female \n", "20 Prof-specialty Husband White Male \n", "21 Other-service Unmarried Black Female \n", "22 Farming-fishing Husband Black Male \n", "23 Transport-moving Husband White Male \n", "24 Tech-support Unmarried White Female \n", "25 Tech-support Husband White Male \n", "26 Craft-repair Own-child White Male \n", "27 NaN Husband Asian-Pac-Islander Male \n", "28 Exec-managerial Not-in-family White Male \n", "29 Craft-repair Husband White Male \n", "... ... ... ... ... \n", "16251 NaN Husband White Male \n", "16252 Prof-specialty Husband White Male \n", "16253 Tech-support Not-in-family White Female \n", "16254 Prof-specialty Husband White Male \n", "16255 Exec-managerial Husband White Male \n", "16256 Machine-op-inspct Own-child White Male \n", "16257 Exec-managerial Not-in-family White Female \n", "16258 Protective-serv Husband White Male \n", "16259 Sales Other-relative White Female \n", "16260 Sales Not-in-family White Female \n", "16261 Prof-specialty Husband White Male \n", "16262 Craft-repair Husband White Male \n", "16263 Craft-repair Own-child White Male \n", "16264 Farming-fishing Not-in-family White Male \n", "16265 Prof-specialty Wife White Female \n", "16266 Priv-house-serv Not-in-family White Female \n", "16267 Adm-clerical Unmarried Black Female \n", "16268 Prof-specialty Not-in-family White Male \n", "16269 Handlers-cleaners Husband White Male \n", "16270 Adm-clerical Husband White Male \n", "16271 Sales Husband White Male \n", "16272 Craft-repair Husband White Male \n", "16273 Other-service Own-child White Female \n", "16274 Other-service Not-in-family White Male \n", "16275 Prof-specialty Own-child White Male \n", "16276 Prof-specialty Not-in-family White Female \n", "16277 NaN Other-relative Black Male \n", "16278 Prof-specialty Husband White Male \n", "16279 Adm-clerical Own-child Asian-Pac-Islander Male \n", "16280 Exec-managerial Husband White Male \n", "\n", " Capital Gain Capital Loss Hours per week Country \n", "0 2174 0 40 United-States \n", "1 0 0 13 United-States \n", "2 0 0 40 United-States \n", "3 0 0 40 United-States \n", "4 0 0 40 Cuba \n", "5 0 0 40 United-States \n", "6 0 0 16 Jamaica \n", "7 0 0 45 United-States \n", "8 14084 0 50 United-States \n", "9 5178 0 40 United-States \n", "10 0 0 80 United-States \n", "11 0 0 40 India \n", "12 0 0 30 United-States \n", "13 0 0 50 United-States \n", "14 0 0 40 NaN \n", "15 0 0 45 Mexico \n", "16 0 0 35 United-States \n", "17 0 0 40 United-States \n", "18 0 0 50 United-States \n", "19 0 0 45 United-States \n", "20 0 0 60 United-States \n", "21 0 0 20 United-States \n", "22 0 0 40 United-States \n", "23 0 2042 40 United-States \n", "24 0 0 40 United-States \n", "25 0 0 40 United-States \n", "26 0 0 40 United-States \n", "27 0 0 60 South \n", "28 0 0 80 United-States \n", "29 0 0 40 United-States \n", "... ... ... ... ... \n", "16251 2936 0 20 United-States \n", "16252 7688 0 40 United-States \n", "16253 0 1669 40 United-States \n", "16254 0 0 50 United-States \n", "16255 0 1902 50 United-States \n", "16256 0 0 40 United-States \n", "16257 0 0 40 United-States \n", "16258 0 0 40 United-States \n", "16259 0 0 8 United-States \n", "16260 0 0 45 United-States \n", "16261 15024 0 55 United-States \n", "16262 0 0 40 United-States \n", "16263 0 0 40 United-States \n", "16264 0 0 32 United-States \n", "16265 0 0 40 NaN \n", "16266 0 0 32 United-States \n", "16267 0 0 40 United-States \n", "16268 0 0 36 United-States \n", "16269 0 0 40 United-States \n", "16270 0 0 40 United-States \n", "16271 0 0 48 United-States \n", "16272 0 0 40 United-States \n", "16273 0 0 40 United-States \n", "16274 0 0 40 United-States \n", "16275 0 0 40 United-States \n", "16276 0 0 36 United-States \n", "16277 0 0 40 United-States \n", "16278 0 0 50 United-States \n", "16279 5455 0 40 United-States \n", "16280 0 0 60 United-States \n", "\n", "[48842 rows x 13 columns]" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "original" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": true }, "outputs": [], "source": [ "X = train_data.values\n", "y = train_labels.values" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAH/CAYAAACVXpOkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8nGW5//Hvlcm+NGmbtumWtpTSFugGZd8FaqGAgEdw\nAQSPgguI5yd4UPEcBUUEPYICioKiKFJREAotoC0UZFG6snWBFrq36Za92e/fH89kMkmzTDKTPLN8\n3q/XvJJ75nmeuQhdvn3ue67bnHMCAABA9NL8LgAAACBZEKwAAABihGAFAAAQIwQrAACAGCFYAQAA\nxAjBCgAAIEYIVgAAADFCsAIAAIgRglWKMrN8M/uemS00s91m5szsJr/rAgAgkaX7XQB8UyzpfyRt\nlbRS0tkRn1hc7MaPH99PZQEAEH+WL1++xzk3rKfjCFapa4ek0c657WY2XtIHkZ44fvx4LVu2rL/q\nAgAg7pjZpkiOI1ilKOdcvaTtftcBAEAyYY0VAABAjMRtsAoupo70cVoc1FtgZheY2a1mtsjM9oTV\nN6UX1ykxs7vNbIOZ1ZnZLjNbYGZn9mf9AAAgevE8Fbirh9cHScqR1CDp7f4vp0dnSnoimguY2XRJ\nSyQNDT5VKW+R+XmS5pnZt5xzt3c4xyRlRfgWDc65lmhqBAAAXYvbO1bOuZLuHpLWBw992jm3189a\nw5RJWijpe5Ku7s2JZpYj6Sl5oWqlpCOdc4WSBkv6iSSTdJuZzelw6kmSDkT4OLVP/1UAACAi8XzH\nqktmNlPSjODwdxGeky7pF5LudM6t7+HYEkl3Svqic64mwrIWOOf+FnaN8RGe1+oaSeMkVUs63zm3\nTZKcc5WSbjCziZIulPRDSc+Hnbde0lURvsfaXtYEAAB6ISGDlaTPBr+23iGKxNclfV7SXDM71TnX\naXsBMyuW9A9JR8ibivtKJBd3zjVHWEdXPhP8+khrqOrgTnnB6igzm+ycWxd83zJJD0X53gAAIAbi\ndiqwK8E7T58ODh9xzjVFeOo9kl6RNEbSYjMb08m1C+XdDTpC0jp5U3r9zswKJB0dHD7XxWGvS6oI\nfs9CdgAA4lAi3rE6R9Lw4PcRTQNKknOuxszmyVscfpS8cHWqc26X5G3xIulZSbPkNcs8M3g3aCBM\nlbeGSpLe6ewA51yLma2TdKykw2PxpmZ2raSi4EOSzggGV0n6uXOuovMzAQBAZxIxWF0Z/LraObeq\nNyc65yqCi7+XyrsrtTjYqqFW0tOSjpe0TV6o6mw6rr+MDPu+u6adra+N7OaY3rhB3rquVnOCD0n6\ng9rukAEAgAgkVLAysyHyWg9IvbhbFc45t9fMzpL0srxw9byk3ZJOk7dm68yu1l/1o7yw7w90c1xt\n8Gt+LN7UOTe+N8eb2fmSzj/00ENj8fYAkohzTlVVVaqsrFRtba2am6NddgrEViAQUG5urgYNGqSC\nggJ53YpiL6GClaRPScqU1CTpj329iHNuZ7Dh5svypgUlab+ks1sXheNgzrkFkhbMnj37C37XAiB+\nOOdUVlammpoaDRkyRCUlJQoEAv32FxfQW845NTc3q7q6Wnv27NGBAwc0fPjwfvk1mmiL11s/Dbgo\nBuuftsnrF9VqraT3orxmX4W3dMjp5rjc4NfqfqwFAHqlqqpKNTU1GjdunIqKipSenk6oQlwxM6Wn\np6uoqEjjxo1TTU2Nqqqq+uW9EiZYmdlUSccEh32aBgy7lkl6UNLH5IWaMkknSHrczDKjuXYfha+r\nGtXNca2v7ejHWgCgVyorKzVkyBAFAgG/SwF6FAgENGTIEFVWVvbL9RMmWKlt0fo+SQuivNa98u5+\nHZB0vqSzgtedK2l+2CfjBspaSS74/RGdHWBmaZImB4fvDkRRABCJ2tpa5efHZOknMCDy8/NVW1vb\n84F9kBDByswCki4LDv/knGuI4lo/lvQleXsMXuyce8E595a8T8NVymvC+XAwyAwI51yVpGXB4dld\nHHacpMLg94v7vagBtKe6XkvW7pJzrueDAcSd5uZm7lYhoQQCgX77gEVCBCt5d5Rap8H6PA1oZrfI\n68DeJOlS59yzra8555ZLOlfe1OAnJT1gA7tI4JHg18+YWWftFG4Ifl2eLAvsX1hbplPuWKLZ3/+H\nPvfQMm3e1z//egDQ/1hThUTSn79eEyVYtS5af9c590ZfLmBm10j6jqQWSZeH7+vXyjn3iqQLJNXJ\n23/vtl6+R3HrQ97mya2Kwl/r4m7Y/ZI2SSqQ9LSZHR68ZoGZ3SHp4uBx3+pNTfFsUE66tuxr6y6x\ncnO5j9UAABC9uA9WZjZI3vScFN2i9UWStkj6vHPu0a4Ocs4tkfRxeb2t5vfyPXaHPVaEPf9ah9dK\nO3nfA/IW0++V1wLiHTOrkFQu6UZ5a7C+6Zx7vuO5ieqIUYXKCLT9q2Hl5v0+VgMAQPTiPlhJukRe\nC4IWed3A+8Q5t1nSEc6530Zw7EJJE3vb2T1azrnVko6U9DNJGyVlyQtaz8jrsXX7QNbT37IzAjp8\n5KDQeNUW7lgBABJb3Acr59wDzjlzzgWcc91t9xLJtSJuWtGbY8POsQgfH3ZzjZ3OueudcxOdc9nO\nueHOufOcc0m1YL3VzLFFoe/f3VGpuka6NQNAZx566CGZmU4//fRenffd735XZqYrr7yyX+pCe3Ef\nrJDcZpW2LUVrbHZ6Z3v/9BUBAGAgEKzgq/A7VhLrrAAAiY1gBV+NG5qrwbkZofFK1lkBABIYwQq+\nMrN204GraLkAAEhgBCv4Lnw6cFv5AZVV1flYDQAMnIaGBt1999068cQTVVRUpIyMDI0YMUIzZszQ\nV77yFb322msRX+vhhx9WRkaG0tLS9Itf/KJXdfzzn//UJz/5SY0ZM0ZZWVkaOnSozjrrLP3pT3/q\ncleMFStW6KabbtLJJ5+s0tLS0Hmnn366HnjggS47m4cvpm9padE999yjY489VkVFRTIzrVrlfSD/\nyiuvlJnpu9/9rpqbm3XXXXdpxowZys3N1ZAhQ3Teeedp2bJlnb6HnwZ6TzzgILNK26+zWrW5XHOO\nKPGpGgAYGE1NTZozZ46WLl0qybuDX1hYqL1796qsrExvvvmm9u7dqxNOOKHHa91777267rrrFAgE\n9Pvf/16XXXZZj+e0+u///m/dcccdofGgQYO0f/9+LV68WIsXL9ZTTz2lP/7xj0pLa38vZs6cOdq7\nd68kKTc3V7m5udq3b5+WLl2qpUuX6oknntCTTz6p9PTOo4ZzThdffLGefPJJBQIBFRQUdHpcU1OT\n5s2bp+eee04ZGRnKysrS/v379cwzz2jx4sVasmRJRD+jgcIdK/hu+pgOC9hZZwUgBTzyyCNaunSp\ncnNz9fDDD6u2tlb79+9XfX29Nm3apHvuuUczZszo8To/+MEPdO211yozM1N/+ctfehWq7r77bt1x\nxx0aMWKEfvWrX6m8vFwVFRWqqanRo48+qpKSEj366KP60Y9+dNC5c+bM0Z/+9Cft2LFDNTU12r9/\nv6qrq/Xwww+rpKRECxcu1E9/+tMu3/vxxx/Xs88+q/vuu0+VlZXav3+/du3apUMOOaTdcffee6/e\neOMNzZ8/X9XV1aqqqtLq1at15JFHqq6uTtdff33E/70DgTtW8F1hToYOHZ6v98uqJfHJQCDpLLpJ\n2vmW31XERsk06ZzY9Gp+/fXXJUlXXHFFuzAUCARUWlqqr3zlKz1e48Ybb9SPf/xj5eXl6cknn9SZ\nZ54Z8fuXl5fr5ptvVnZ2tp577rl2IS4nJ0eXXnqpSktLddJJJ+nOO+/U17/+dWVmZoaOeeSRRw66\nZl5eni677DKNGzdOp556qu677z7deOONnb5/dXW17r//fl199dWh54YPH95pnS+//LJOPvnk0HPT\np0/XQw89pNmzZ+uNN97Q5s2bVVp60KYmviBYIS7MHFsUClZvbq1Qc4tTII1NXYGksPMtadM//a4i\n7gwa5O08sWPHjl6f29LSomuuuUYPPPCAioqKtHDhwl5Ph/31r39VdXW1zjvvvC7vjJ1wwgmaMGGC\nNm7cqOXLl0f8HqeccoqKior04Ycfavv27Ro1atRBxwwdOlSf+9znIrpWeKhqdfTRR2vMmDHaunWr\n3n777bgJVkwFIi6Er7OqbWjW+l29bnwPAAnlnHPOkSQ9+eSTuuCCC/T444+H1ix1p7GxUZ/61Kf0\nwAMPaPjw4XrxxRf7tMbo1VdflSQtWbJEJSUlXT62bNkiSaGv4R577DFdeOGFKi0tVU5Ojsws9Cgv\n95Z1bN/e+aYps2fP7nL9Vbhjjjmmy9dGjx4tSdq/P35mOrhjhbgwa+zgduNVW8o1NWwfQQAJrGSa\n3xXETgz/W0477TTdcsstuuWWW7RgwQItWLBAkjRlyhTNmzdP11xzjSZNmnTQea2BSJL+8pe/RLQO\nqzOtd8pqa2tVW1vb4/HhxzQ1NemSSy7RE088EXouKytLxcXFCgQCkqTdu3erpaVFNTU1nV5v2LBh\nEdXZ1aJ2ScrOzpbkhc14QbBCXDhsRL5yMgI6ENwrcOXm/frUsfFxWxdAlGK0JikZfec739Fll12m\n+fPn68UXX9Rrr72mtWvXau3atbr77rv14IMP6oorrmh3zrRp09Tc3Kx3331XX/7yl/XCCy+ouLi4\n1+/d0tIiSbr++ut111139ercX//613riiSeUm5ur22+/XRdddJHGjBnT7pixY8dq69atXbZraA1g\nyYapQMSF9ECapo8pDI1X0igUQIqYMGGCbrrpJj377LPat2+fXnjhBZ166qlqamrSl7/8ZZWVlbU7\nfsiQIVq8eLEmT56st99+W2effXafpsJGjBghSdq8eXOvz33sscckecHwuuuuOyhUNTc3a8+ePb2+\nbjIgWCFuzAxbZ/X+7mpV1sXPrV0AGAiBQECnn366nn76aWVkZKimpqbTJpglJSVavHixJk6cqFWr\nVmnOnDmqqKjo1Xu1rst68cUXdeDAgV6du3XrVknSrFmzOn39lVdeUV1dajZ7JlghboSvs3JOenNL\n7/6QAIBE0tDQ0OVrmZmZoamy+vr6To8ZPXq0lixZovHjx2vZsmU655xzVF1dHfH7f+ITn1BeXp72\n79+vW265pdtjO94RKyz0ZhjeeuvgNhpNTU26+eabI64j2RCsEDcO6sC+JX4+5QEAsXbFFVfoqquu\n0nPPPaeqqrZPQn/44Yf67Gc/q7q6OuXk5OiUU07p8hqlpaVasmSJxowZo9dee03z5s2LaCG65LU7\n+OEPfyhJuv322/WFL3xB69evD71eW1urF154QVdffbVOPPHEdueeffbZkqRbb71VTz75ZGj7mrVr\n1+r888/Xv//9b+Xl5UX2g0gyBCvEjRGDsjWqMDs0Zp0VgGRWV1enhx56SHPnzlVhYaEGDx6svLw8\nTZgwQfPnz1cgEND999/f48L0CRMmaMmSJRo5cqReeuklXXDBBRFPw1133XW69dZbZWZ64IEHNHny\nZOXn52vIkCHKz8/XRz7yEf36178+6Ho33HCDJk6cqMrKSl144YXKyclRYWGhpk6dqr///e/65S9/\n2acF9cmAYIW4Mqu0bTpw5ZbyLj9NAgCJ7vbbb9cdd9yhuXPn6pBDDlFDQ4Oam5s1ceJEXXXVVVqx\nYoUuv/zyiK41adIkLVmyRCNGjNDixYt10UUXdTmF2NHNN9+s1atX6+qrr9akSZNCLRJGjRqlj370\no7rjjjv08ssvtztnyJAhev311/WlL30ptHA9JydHF154oZYuXaorr7yyVz+LZGL8xYXemj17tuuv\nHcV//dJG/WDhmtD4pRvPUOnQ3H55LwCxsWbNGk2dOtXvMoBe6e2vWzNb7pyb3dNx3LFCXOm4zmol\n66wAAAmEYIW4cuToQqWH7RHIOisAQCIhWCGuZGcE2m1ls3ILwQoAkDgIVog74dOBa7ZXqr6p2cdq\nAACIHMEKcWfm2LZg1dDcone2V/pYDQAAkSNYIe6Et1yQWGcFAEgcBCvEnfFDc1WUmxEar2KdFQAg\nQRCsEHfMrN104MrNtFwAACQGghXiUviGzFv3H9Duqsg6CAPwB82mkUj689crwQpxaeZBGzIzHQjE\nq0AgENqEF0gEzc3NCgQC/XJtghXi0swxHTqwMx0IxK3c3FxVV1f7XQYQserqauXm9s92aQQrxKXC\n3AxNHJYXGnPHCohfgwYN0r59+7hrhYTQ3Nysffv2adCgQT0f3AcEK8StmWHrrFZvKVdzC2s4gHhU\nUFCgvLw8bdq0SeXl5WpqamLNFeKKc05NTU0qLy/Xpk2blJeXp4KCgn55r/R+uSoQA7NKi/TXFVsl\nSTUNzXq/rFqTS/rnNwKAvjMzDR8+XFVVVaqsrFRZWRl3rxB3AoGAcnNzVVxcrIKCAplZzyf1AcEK\ncSu85YLkrbMiWAHxycw0aNCgfpteARIFU4GIW1NKCpSd0fZLlA7sAIB4R7BC3EoPpGn66La7Vixg\nBwDEO4IV4tqssH5W68uqVFXX6GM1AAB0j2CFuBYerJyT3tpa4WM1AAB0j2CFuBbeckGSVjIdCACI\nYwQrxLWSwmyNLMwOjenADgCIZwQrxL3wtgurtpTTeBAAELcIVoh74eus9lQ3aOv+Az5WAwBA1whW\niHuzSllnBQBIDAQrxL0jRxUqkNa29QDrrAAA8YpghbiXkxnQ1JFtW9nQgR0AEK8IVkgI4QvY391e\nqfomNngFAMQfghUSwqywflYNzS16d3ulj9UAANA5ghUSQvgnAyX2DQQAxCeCFRLChOI8FeZkhMas\nswIAxCOCFRKCmbVbZ7VyC58MBADEH4IVEkZ4sNqy74D2VNf7WA0AAAcjWCFhHLTOiulAAECcIVgh\nYYTfsZJYwA4AiD8EKySMotxMHVKcFxqzzgoAEG8IVkgoM8OmA1dvqVBzi/OxGgAA2iNYIaHMCpsO\nrK5v0obd1T5WAwBAewQrJJRZpYPbjdmQGQAQTwhWSCiTSwqUndH2y5YF7ACAeEKwQkLJCKRp2ujC\n0JgO7ACAeEKwQsIJnw5cv6tK1fVNPlYDAEAbghUSTng/qxYnvbmVu1YAgPhAsELC6diBnelAAEC8\nIFgh4YwszNGIQVmhMQvYAQDxgmCFhDRrbNs6q5Wby+UcjUIBAP4jWCEhhU8H7qmu17byAz5WAwCA\nh2CFhNRxQ2bWWQEA4gHBCglp2phCBdIsNCZYAQDiAcEKCSk3M12TRxSExqu2sLUNAMB/BCskrPB1\nVm9vr1RDU4uP1QAAQLBCAgvvwN7Q1KI1Oyp9rAYAAIIVEtjBC9iZDgQA+ItghYR1SHGeBmWnh8Yr\naRQKAPAZwQoJKy3NNCPsrhUd2AEAfiNYIaGFr7PatLdWe6vrfawGAJDqCFZIaB03ZF69lbtWAAD/\nEKyQ0GaOoQM7ACB+EKyQ0AbnZWpCcV5oTLACAPiJYIWEF952YfWWcrW0OB+rAQCkMoIVEl74Oquq\n+iZt2F3tYzUAgFRGsELCmzV2cLsx/awAAH4hWCHhTRlZoKz0tl/KrLMCAPiFYIWElxFI07TRhaEx\nW9sAAPxCsEJSCF/Avn5XlWrqm3ysBgCQqghWSArhHdhbnPTm1gofqwEApCqCFZJCxw7s7BsIAPAD\nwQpJYWRhtoYXZIXGrLMCAPiBYIWkYGbt7lqt3FIu52gUCgAYWAQrJI2ZYf2sdlfVa3tFnY/VAABS\nEcEKSaPjOiumAwEAA41ghaQxbXSh0qxtvIpGoQCAAUawQtLIy0rX5JJBoTFb2wAABhrBCkklfDrw\nrW0Vamhq8bEaAECqIVghqYR3YG9oatHanZU+VgMASDUEKySVow5awM50IABg4BCskFQOKc5XQXZ6\naEwHdgDAQCJYIamkpVm76UBaLgAABhLBCklnVliw+nBvrfbXNPhYDQAglRCsUpSZ5ZvZ98xsoZnt\nNjNnZjf5XVcszGRDZgCATwhWqatY0v9ImiZppc+1xFT41jYS04EAgIFDsEpdOySNds6NlXS138XE\n0pC8TI0bmhsa0ygUADBQCFYpyjlX75zb7ncd/SV8ndWqLeVqaXE+VgMASBUEKySlWaVt04FVdU3a\nuKfGx2oAAKkiIYKVmU02s5+b2TozqzGzCjNbY2a/MbPT/K5PksyswMwuMLNbzWyRme0JLgh3Zjal\nF9cpMbO7zWyDmdWZ2S4zW2BmZ/Zn/ckmvOWCxDorAMDASO/5EH+Z2Vcl3SkpM/hUdfD7KcFHi6Sl\n/lTXzpmSnojmAmY2XdISSUODT1XKW2R+nqR5ZvYt59ztHc4xSVkRvkWDcy4lNs+bOnKQMtPTQnsF\nrtxSrk/MHutzVQCAZBfXd6zM7BpJd8sLgD+SNM45V+Ccy5E0UtIVkl71scSOyiQtlPQ99XJBuJnl\nSHpKXqhaKelI51yhpMGSfiLJJN1mZnM6nHqSpAMRPk7t039VAspMT9ORowaFxqvY2gYAMADi9o6V\nmY2X9H/B4Redc78Of905t1PSw724XrqkX0i60zm3vodjS+TdJfuicy7SxTkLnHN/C7vG+EhrC7pG\n0jh5d+TOd85tkyTnXKWkG8xsoqQLJf1Q0vNh562XdFWE77G2lzUltFmlg7UiGKjW7qxUbUOTcjPj\n9pc8ACAJxPPfMtdLypX0r46hqo++Lunzkuaa2anOuQ86O8jMiiX9Q9IR8qbivhLJxZ1zzVHW95ng\n10daQ1UHd8oLVkeZ2WTn3Lrg+5ZJeijK905Ks8IahbY46a2tFTrukKHdnAEAQHTieSrw08Gvf4rR\n9e6R9IqkMZIWm9mYjgeYWaG8u0FHSFonb0qv35lZgaSjg8PnujjsdUkVwe9ZyB6Bgxaw088KANDP\n4vKOVXDaa3hwuNLMjpf0bUknyruLtUnSAnnTemWRXNM5V2Nm8+QtDj9KXrg61Tm3K/ie+ZKelTRL\n0geSzoz02jEwVd4aKkl6p7MDnHMtZrZO0rGSDo/Fm5rZtZKKgg9JOiM4ZSpJP3fOVXR+ZmIYXZSj\nYQVZ2l1VL4lPBgIA+l9cBitJk8K+P13e1isBSVWSnKTJwcdnzOxs51ynYaQj51xFcPH3Unl3pRYH\n2zXUSnpa0vGStskLVZ1Nx/WXkWHfd9e0s/W1kd0c0xs3yFvX1WpO8CFJf1DbHbKEZGaaObZIf393\nlyRp5eZyOefkfZASAIDYi9epwPA5nP+Vt0D7eOfcIEn5ks6V9wm8kZL+GnaXpUfOub2SzpL0vrxw\n9by8NgmnBa95Zlfrr/pRXtj3B7o5rjb4NT8Wb+qcG++csy4eH3Y83szON7NfVVQkTt4KX2dVVlWv\nHRV1PlYDAEh28Rqswutyki5yzv1L8qbEnHOLJH0u+PpkSRf35uLBTxSeKWmzvGnBj0raL+ns1kXh\nOJhzboFz7urCwkK/S4nYrA4bMq9inRUAoB/Fa7CqDvv+2c7CjnPuGXl3sqS+LebeJq9fVKu1kt7r\nw3ViIbylQ043x7XuLFzdzTEIM31ModLCZv5YZwUA6E/xGqzC1xl1dwep9bVetdQOdit/UNLH5IWa\nMkknSHrczDK7O7efhP/3jurmuNbXdvRjLUklLytdh40oCI1X0igUANCP4jVYvStvq5pIuV5e/15J\nn5W3nul8eWuu9kmaK2l+b9Zsxchatf03HNHZAWaWJm/aU/J+PohQ+Dqrt7ZVqLE5JXb1AQD4IC6D\nlXOuVtJrweHkbg5tfe3DSK9tZj+W9CVJDZIuds694Jx7S96n4SrlNeF8OBhkBoRzrkrSsuDw7C4O\nO05S6+Kmxf1eVBIJX2dV39SitTuqfKwGAJDM4jJYBf0++HWumR0UroI9qQ4LDhdGckEzu0VeB/Ym\nSZc6555tfc05t1zepw1rJH1S0gM2sJ/LfyT49TNm1lk7hRuCX5ezwL53Zpa2bxS6agvrrAAA/SOe\ng9Vv5E15BeStfTpW8qbEzGyuvDVSkteRvMdgFdzQ+TvyphgvD9/Xr5Vz7hVJF0iqk7f/3m29KdjM\nilsf8jZPblUU/loXd8Pul9f4tEDS02Z2ePCaBWZ2h9o++fit3tQE6dBh+SrIapvdZZ0VAKC/xGuD\nUDnnmszsfEkvyus0/i8zq5IXtFo/HfeupP9wzkWyxmqRpC2S/tc592g377vEzD4ub/+9+b0se3cX\nz7/WYTxBHaYvnXMHzOxj8qb5jpL0jplVyutZlSZvDda3nHPPC72SlmaaMbZI/3x/jyS2tgEA9J94\nvmMl59xGSdMk/UBeiEqXFzBWSPqmpGMj7ZDunNss6Qjn3G8jOHahpInOuVV9rb0vnHOrJR0p6WeS\nNkrKkrRX0jPyemzdPpD1JJPwfQM/2FOj/TUNPlYDAEhWcXvHqlVwv7qbg49orxXxquXeHBt2TtRr\nsoLNS68PPhAjszqus9parjMmD+/iaAAA+iau71gBsRJ+x0qSVrHOCgDQDwhWSAlD87NUOiQ3NGad\nFQCgPxCskDLCpwNXbd6vlpbe9pUFAKB7BCukjPDpwMq6Jn2wt6abowEA6D2CFVLGrNLB7cb0swIA\nxBrBCilj6sgCZQbafsnTgR0AEGsEK6SMrPSAjhg9KDTmjhUAINYIVkgp4Rsyr91ZpQMNzT5WAwBI\nNgQrpJTwDZmbW5ze2lbhYzUAgGRDsEJKmdWhUejKzayzAgDEDsEKKWXM4BwV52eGxqtoFAoAiCGC\nFVKKmWlm2DorFrADAGKJYIWUE96BfWdlnXZUHPCxGgBAMiFYIeV0XGfFhswAgFghWCHlTB9bJLO2\nMRsyAwBihWCFlJOfla7DhheExtyxAgDECsEKKSl8ndWb28rV2NziYzUAgGRBsEJKCg9WdY0tWrez\nysdqAADJgmCFlBTeckFinRUAIDYIVkhJhw7PV35WemhMB3YAQCwQrJCSAmmm6WMKQ2M6sAMAYoFg\nhZQVvs5q4+4aVdQ2+lgNACAZEKyQsjqus1q1lbtWAIDoEKyQsmZ26MDOOisAQLQIVkhZwwqyNHZI\nTmjMhswAgGgRrJDSwqcDV20pl3POx2oAAImOYIWUFr4hc8WBRn2wp8bHagAAiY5ghZQ2s7T9Oiva\nLgAAokEgbUdRAAAgAElEQVSwQko7YtQgZQbafhuwzgoAEA2CFVJaVnpAh48aFBqv3MInAwEAfUew\nQsoLb7uwdkeVDjQ0+1gNACCREayQ8sI7sDe1OL29vcLHagAAiYxghZQ3q2MHdtZZAQD6iGCFlDd2\nSI6G5mWGxqyzAgD0FcEKKc/M2k0H8slAAEBfEawAtV/AvqOiTjsr6nysBgCQqAhWgKRZpR3WWTEd\nCADog3S/C0CKW3STtPMtv6vQcS0tejSzLUyNfC5beiPPx4oAAFEpmSadc/uAvy3BCv7a+Za06Z9+\nV6F0SceH37+tCj4AAOgFpgIBAABihDtW8FfJNL8rCNlVVacP9tSExtNGFyovk98iAJCQfPr7hb81\n4C8f5r+7sn9npT5518uh8fdnHKnLjh/nY0UAgETDVCAQNGl4gfIyA6Hxqi30swIA9A7BCggKpJmm\njwlvFErLBQBA7xCsgDDhHdg37K5RxYFGH6sBACQaghUQJrwDuyStZjoQANALBCsgzMzS9sGKfQMB\nAL1BsALCDC/I1uiinNCYrW0AAL3R52BlZl8NPkbFsiDAb+HrrFZtKZdzzsdqAACJJJo7Vj+V9GNJ\ne2JUCxAXwtdZ7a9t1Ka9tT5WAwBIJNEEqz2SqpxzDbEqBogHs0oHtxuvZDoQABChaILVCkmFZjYs\nVsUA8eCIUYOUEbDQmAXsAIBIRROsfhY8/zsxqgWIC9kZAR0+clBoTAd2AECk+hysnHOLJN0g6Ytm\n9rCZzYhdWYC/wqcD391eqbrGZh+rAQAkij5vwmxmG4PfNkn6tKRPm9kBSXsldfW3kHPOTezrewID\nJXwBe1OL0zvbK3T0uCE+VgQASAR9DlaSxnfyXG7w0RU+t46EMKuTRqEEKwBAT6IJVmfErAogzpQO\nydWQvEztq/E+9MoCdgBAJPocrJxzS2NZCBBPzEwzxxZpydoySSxgBwBEhi1tgC7MCltnta38gMoq\n63ysBgCQCGIWrMwzxcxOCT6mmJn1fCYQnw7akJm7VgCAHkQdrMzsUDN7SFKFpHckvRh8vCOpwsx+\na2aHRvs+wECbMbZI4f80YJ0VAKAnUQUrM7tA0kpJl0vKl2QdHvmSrpC00szOi65UYGANys7QocPy\nQ+MVm9naBgDQvT4HKzObKOlRSXmSNkq6RtIkSTnBxyRJX5S0IXjMn4PnAAkjvJ/Vsg/3ae3OSh+r\nAQDEu2juWH1DUrakFyRNd8792jm3wTlXH3xscM79StIMSUslZUm6MfqSgYFz3oxRoe9bnPSDZ9bI\nOdqxAQA6F02wOltew89rnHMHujoo+No18qYG50TxfsCAO3VSsU4+tDg0fvm9PXpx3W4fKwIAxLNo\ngtVISRXOufd7OtA5t15SefAcIGGYmb49b6rSwhaxf/+Zd9XY3OJfUQCAuBVNsKqVlGtmGT0daGaZ\n8tZZdXlnC4hXU0cO0qXHjA2NN+yu0Z/+vdnHigAA8SqaYPWWpAxJn43g2M8Gj30zivcDfPNfZx+m\nvMxAaPzTv69XxYFGHysCAMSjaILVw/LWTf3MzD7fWTNQM8s2s69K+pm89Vi/i+L9AN8ML8jWl89o\na8e2v7ZR977Q4yw4ACDFRBOsfiPp7/I+GXi/pK1m9qiZ/cTM7jWzBZI2S/qpvE8E/l3SQ1HWC/jm\nP0+eoNFFOaHxb1/5QJv21vhYEQAg3vQ5WDnvM+cXSvqVvLtRIyVdIulr8vpXzZNUHHztl5IucnxO\nHQksOyOgb8ydHBo3NjvdvmitjxUBAOJNVJ3XnXMHnHNflDRR0v+T9AdJzwcffwg+d4hz7svdtWQA\nEsUFM0a1axq66O2d+vcH+3ysCAAQT4ybSOit2bNnu2XLlvldhm+Wb9qvj//i1dB42uhCPfmVk5SW\nxp7jAJCszGy5c252T8dFs6XNCjNbbmaH9PUaQCI6etxgnTe9rSXbW9sq9LdV23ysCAAQL6KZCjxc\n0iTn3MZYFQMkiv+eO0WZ6W2/fe54dp0ONDT7WBEAIB5EE6y2yWu3AKScsUNy9Z8nTwiNd1bW6Vcv\n8W8MAEh10QSr5+R1Xj8uVsUAieTLp0/U0LzM0PiXSzdoV2WdjxUBAPwWTbD6vqS9kn5pZsU9HQwk\nm4LsDP2/OYeFxgcam/Xj59b5WBEAwG/pUZx7qKRvS/qJpHVm9ntJr0naLanLxSbOuZeieE8grlw6\ne6x+/+omrdtVJUn6y4qt+uyJ43Xk6EKfKwMA+KHP7RbMrEVe80/JW2sVyYWccy6aMIc4kOrtFjp6\naf1uXfGbf4fGxx8yRH/6wvHqZJcnAECCirTdQjQhZ7MiC1NAUjv1sGE6ffIwvbhutyTp9Y379Pd3\nd2nOESU+VwYAGGh9DlbOufExrANIaN8+d6pefm+Pmlu8f2vctnCNTp88vF1LBgBA8uNPfSAGJo0o\n0KePLQ2NP9xbq4df3+RjRQAAP0TTeX2/me2l8zrg+dpZk1SQ1XYT+GeL31N5bYOPFQEABlo0d6wy\nJQXovA54huZn6dqPHBoaVxxo1F3/eM/HigAAAy2aYLVZXrgCEHTlSeM1dkhOaPyH1zdpw+5qHysC\nAAykaILVU5KyzOzsWBUDJLqs9IC+ec7U0LipxemHC9f6WBEAYCBFE6xuk/ShpF+b2dQejgVSxjlH\nluiY8YND43+s2aVX39/jY0UAgIESTR+rj0n6haT/kbTSzBYpss7rv4/iPYG4Z2a6ed7h+ti9r4Se\nu/WZNXr6upMVSKNpKAAks2iC1UPyGoS2/k1xQfDRE4IVkt6MsUW6aNZoPbFymyRpzY5K/XX5Vl1y\nzFifKwMA9KdogtVLovM60KUbPzpZi97eobrGFknSnc+v07nTRyo/i12dACBZRdN5/fQY1gEknVFF\nObr6lEP0syXvS5J2V9Xr/qUb9PU5k32uDADQX+i8DvSja06bqGEFWaHxr17aqO3lB3ysCADQnyIO\nVmb2VTP7zy5eyzezQT2c/1Mze7C3BQKJLC8rXTeG3aGqb2rRHc/SfgEAklVv7ljdJemWLl57T9K+\nHs7/pKQre/F+QFL4+NFjdPjItn93/G3Vdq3aUu5jRQCA/tLbqcDuPivO58iBTgTSTDfPa9/q7ftP\nvyvn+OwHACQb1lgBA+DEQ4t11tQRofGyTfu18K2dPlYEAOgPBCtggHzr3ClKD2sQevuza1TX2GUv\nXQBAAiJYAQPkkGH5uvyEcaHxln0H9LtXP/SvIABAzBGsgAF0/ZmTVJiTERrfs+R97amu97EiAEAs\nEayAAVSUm6nrz5wUGlfVN+muf6z3sSIAQCwRrIABdtnx4zShOC80fuRfm7V+V5WPFQEAYqW3W9oM\nMbMlnT0vSV281u4YINVlpqfpm+dM0dUPL5cktTjpB8+s0e8+d6zPlQEAotXbYJUp6fRuXu/uNYlN\nmwFJ0tmHj9AJhwzVaxv3SpKWrt+tF9eV6fTJw32uDAAQjd4Eq9/1WxVAijEzfXveVJ1/zz/V2if0\ntoVrdPKhxUoPMEMPAIkq4mDlnLuqPwsBUs2Rowv1H0eN0WPLt0qS1u+q1qNvbNFlx4/r4UwAQLzi\nn8aAj2746GTlZgZC45/+fb0q6xp9rAgAEA2CFeCjEYOy9cXTJobGe2sadN8LG3ysCAAQDYIV4LMv\nnHKIRhZmh8a/+ecH2rKv1seKAAB9RbACfJaTGdA35k4OjRuaW3T7s2t9rAgA0FcEKyAOfGzGaE0f\nUxgaP/PmDi3ftM/HigAAfUGwAuJAWprp5nmHt3vulqfXqKWF1m8AkEgIVkCcOHbCEJ07rSQ0Xr2l\nXAve3O5jRQCA3iJYAXHkv+dOUWZYg9AfLVqrusZmHysCAPQGwQqII+OG5unKk8aHxtsr6vTAyxv9\nKwgA0CsEKyDOfOWMQzUkLzM0vu/FDSqrqvOxIgBApAhWQJwpzMnQf501KTSubWjW/z2/3seKAACR\nIlgBcehTx5bq0OH5ofH8ZVv07vZKHysCAESCYAXEofRAmr49b2po7Jz0/WfelXO0XwCAeEawSlFm\nlm9m3zOzhWa228ycmd3kd11oc/phw3TKpOLQ+NUNe7VkbZmPFQEAekKwSl3Fkv5H0jRJK32uBZ0w\n85qGplnbcz9YuEaNzS3+FQUA6BbBKnXtkDTaOTdW0tV+F4POTS4p0CePLQ2NN+6u0R9f3+RjRQCA\n7hCsUpRzrt45R1vvBPBfZx2m/Kz00Piuxe+porbRx4oAAF0hWAFxblhBlr58xsTQuLy2UT9b8p6P\nFQEAuhK3wcrMrgwuqO7uUe13na3MrMDMLjCzW81skZntCatzSi+uU2Jmd5vZBjOrM7NdZrbAzM7s\nz/oR3z530gSNLsoJjX//2of6YE+NfwUBADoVt8EqTKOkXd084sWZkp6UdLOkuZKG9vYCZjZd0tuS\nvirpEEn18haZnyfp7519as882RE+EuH/NzqRnRHQTee05fPGZqfbF63xsSIAQGcS4S/aV51zJV08\nJvZ8+oAqk7RQ0vfUywXhZpYj6Sl5gWylpCOdc4WSBkv6iSSTdJuZzelw6kmSDkT4OLVP/1WIC+dN\nH6mjSotC4+fe2aXXNuz1sSIAQEfpPR+SHMwsXdIvJN3pnOt2fxAzK5F0p6QvOucinW9Z4Jz7W9g1\nxveyxGskjZNULel859w2SXLOVUq6wcwmSrpQ0g8lPR923npJV0X4Hmt7WRPiiJnp5vMO18X3vRp6\n7vvPvKsF156stPCeDAAA36RMsJL0dUmflzTXzE51zn3Q2UFmVizpH5KOkFQp6SuRXNw51xxlfZ8J\nfn2kNVR1cKe8YHWUmU12zq0Lvm+ZpIeifG8kiKNKB+uCGaP01GrvA53vbK/U4yu36T+OHuNzZQAA\nKTGmAmPlHkmvSBojabGZHfQ3kZkVyrsbdISkdfKm9PqdmRVIOjo4fK6Lw16XVBH8noXsKewbcycr\nK73tt+6dz61VbUOTjxUBAFolQrA6wszeMbMDZlZlZm+b2U/NbEJvLhKc0psnaYWkCfLC1YjW180s\nX9KzkmZJ+kDSmcG7QQNhqrw1VJL0TmcHOOda5IU9STo8Fm9qZtea2c2Srg0+dYaZ3Rx8FMbiPRB7\nYwbn6vOntP3y31VZr/uXbvSxIgBAq0QIVsXygketpGx5d5O+JukdM/t0by7knKuQNEdeeDlMXrga\nGlw4/rSk4yVtkxeqOpuO6y8jw77vrmln62sjuzmmN26QdKu8aVLJ+9ncGnwMjtF7oB986fRDVZyf\nGRrf/9IG7ayo87EiAIAU38Fqu6T/lXSkpGzn3FBJ+fLuOr0rKUfS78ysV590c87tlXSWpPflhbTn\nJT0h6TR5n+o7s6v1V/0oL+z7A90cVxv8mh+LN3XOjXfOWRePDzseb2bnm9mvKioqOrkaBlJ+Vrq+\nPmdyaFzX2KI7nuOzCQDgt7gNVs65551ztzjn3nHONQSfq3fOLZR0orxglC7p9j5ce6e8dUqbJR0l\n6aOS9ks6u3VROA7mnFvgnLu6sJBZwnhwyeyxmlJSEBo/vmKb3t5G6AUAP8VtsOpOcErvtuDw+OAn\n+Xprm7x+Ua3WSvJrn5Dwlg45XR4l5Qa/xk3HefgnkGa6eV775Xa/feVDf4oBAEhK0GAV9K/gV5O3\nGD1iZmaSHpT0MXmhpkzSCZIeN7PM7s7tJ+HrqkZ1c1zrazv6sRYkkJMnFev4Q4aExgvf2qGqOjZo\nBgC/JHKwisa9kj4rbz3T+fLWXO2TtxXN/GAz0YG0VpILfn9EZwcEt6NpXVTz7kAUhcTwyWNKQ98f\naGzW02+SuwHAL4kcrI4L+/7DSE8ysx9L+pKkBkkXO+decM69Je8TcZXymnA+PJD76jnnqiQtCw7P\n7uKw4yS1Lm5a3O9FIWHMPbJEBdlt/xb487ItPlYDAKktLoNVcKquu9cHSWrdkPjfzrndEV73Fnmt\nBZokXeqce7b1Nefccknnypsa/KSkB3qqI8YeCX79jJl11k7hhuDX5SywR7jsjIA+NrNtBnnl5nK9\nt6vKx4oAIHXFZbCSNM7MXjWzz5rZ6NYnzSzTzObK66B+mKQWSd+M5IJmdo2k7wTPuTx8X79WzrlX\nJF0gqU7e/nu3dTymh/cobn2ofR+oovDXurgbdr+kTZIKJD1tZocHr1lgZndIujh43Ld6UxNSwyWz\nx7Ybc9cKAPwRr8FK8haTPyRpq5nVmtlueZ+GWySvt1WtpCudc0sivN4iSVskfd4592hXBwWv93FJ\nuyXN72XNu8MeK8Kef63Da6UdT3TOHZC3mH6vvBYQ75hZhaRySTfKW4P1Tefc8x3PBaaNLjyo9UJj\nc4uPFQFAaorXYLVL0lcl/VneNi4HJBXJC1PLJP1I0uHOuYcjvaBzbrOkI5xzv43g2IWSJjrnVvWh\n9j5zzq2WFxp/JmmjpCx5QesZeT22et2zC6nBzNrdtdpb06AlawdqRyYAQCtzzvV8FBBm9uzZbtmy\nZT0fiAG1r6ZBx932DzU2e7+nz5wyXA9eeYzPVQFAcjCz5c652T0dF693rAD00pC8TJ19eGhfcb2w\nrky7Ktk/EAAGEsEKSCLh04EtzltrBQAYOAQrIImcMmmYSgZlh8aPLdsipvsBYOAQrIAkEkgz/cfR\nY0LjjXtqtGzTfh8rAoDUQrACkswnZo9pN57/Bj2tAGCgEKyAJDNuaF67jZmfeXOHquubfKwIAFIH\nwQpIQpce07aI/UBjs555c7uP1QBA6iBYAUlo7hEjVZDVtjEz04EAMDAIVkASyskM6PywjZlXbC7X\n+2VszAwA/Y1gBSSpSztszPzYsq0+VQIAqYNgBSSp6WMKNXlE28bMf12xlY2ZAaCfEayAJGVm7Vov\n7Klu0AtszAwA/YpgBSSxi2aNVkbAQuM/Mx0IAP2KYAUksaH5WTpravuNmcvYmBkA+g3BCkhy4Rsz\nN7c4Pb6SjZkBoL8QrIAkd+ph7Tdm/jMbMwNAvyFYAUkukGb6+NGjQ+ONu2u0nI2ZAaBfEKyAFPCJ\no9v3tPrzMjqxA0B/IFgBKWB8cZ6Om9C2MfPTb+5QDRszA0DMEayAFBG+iL22oVnPvLnDx2oAIDkR\nrIAUcc60EuWHbczMdCAAxB7BCkgRuZnpOn9G28bMyzbt14bd1T5WBADJh2AFpJBLwra4kbhrBQCx\nRrACUsjMsUWaNDw/NP7r8m1szAwAMUSwAlKImenSY9oWse+prteL63b7WBEAJBeCFZBiLpw1Wulp\n4RszMx0IALFCsAJSTHF+ls6cOjw0XrK2TGVVbMwMALFAsAJSUPh0YHOL0xMr2JgZAGKBYAWkoFMn\nDdPwgqzQmI2ZASA2CFZACkoPpOnjR7e1Xtiwu0YrNpf7WBEAJAeCFZCiwre4kaQ/v8EidgCIFsEK\nSFETivN07PjwjZm3szEzAESJYAWksEvCFrHXNDRr4VtszAwA0SBYASns3GklyssMhMb0tAKA6BCs\ngBTWcWPmNz7cr41szAwAfUawAlJc+HSgJD22fKtPlQBA4iNYASlu1tgiHdpuY+atamJjZgDoE4IV\nkOLMTJfMbutpVVZVr6Xr2ZgZAPqCYAVAF80aw8bMABADBCsAGlaQpY9MaduYefGaMu2uqvexIgBI\nTAQrAJLad2JvanH620o2ZgaA3iJYAZAknT55mIaFbcw8n42ZAaDXCFYAJAU3Zj6qbRH7+2XVWrmF\njZkBoDcIVgBCPhH26UBJeoxF7ADQKwQrACETh+XrmPGDQ+MFq3eotoGNmQEgUgQrAO18ImwRe3V9\nkxa+tdPHagAgsRCsALQzb9pINmYGgD4iWAFoJy8rXedNb9uY+d8f7NMHe2p8rAgAEgfBCsBBLjmG\nRewA0BcEKwAHOap0sA4Zlhca/3UFGzMDQCQIVgAOYma6NGwR+67Ker30HhszA0BPCFYAOnXRUaMV\nCN+Y+Y2tPlYDAImBYAWgU8MLsnXG5LaNmf+xZpf2VrMxMwB0h2AFoEuXHtN+Y+Yn2JgZALpFsALQ\npdMnD1NxftjGzG+wMTMAdIdgBaBLGYE0ffyo0aHxe2XVWr21wseKACC+EawAdCt8ixvJu2sFAOgc\nwQpAtw4dnq+jx4VvzLxdBxqafawIAOIXwQpAjy6Z3daJ3duYeYeP1QBA/CJYAejRvOmjlMvGzADQ\nI4IVgB7lZ6Vr3rSRofG/PtinD9mYGQAOQrACEJHwnlaS9Nhy7loBQEcEKwAROXrcYB1S3LYx81+W\nb1VzCz2tACAcwQpARMysXesFNmYGgIMRrABE7OMHbczMdCAAhCNYAYjY8EHZOmPysNCYjZkBoD2C\nFYBeCZ8ObGx2+tuq7T5WAwDxhWAFoFc+MmW4ivMzQ+M/szEzAIQQrAD0SkYgTRcf1daJfd2uKr3J\nxswAIIlgBaAPwre4kejEDgCtCFYAeu3Q4QU6qrQoNH5qFRszA4BEsALQR5eELWKvqm/Ss++wMTMA\nEKwA9Mm86SOVkxG2MfMbW32sBgDiA8EKQJ8UZGdo3vS2jZlf27hXm/ayMTOA1EawAtBn4dOBkrd/\nIACkMoIVgD47ZvxgTWBjZgAIIVgB6DNvY+a21gs7Kur0MhszA0hhBCsAUfn4UWMUti+zHlvGdCCA\n1EWwAhCVEYOydfrk4aHx8+/u1L6aBh8rAgD/EKwARO2Sjhszr9zmYzUA4B+CFYCofWTKcA3NC9uY\neRkbMwNITQQrAFHLTE/TRbNGh8Zrd1bprW1szAwg9RCsAMTEJce072nFxswAUhHBCkBMHDaiQDPH\ntm3M/OSq7aprZGNmAKmFYAUgZtptzFzXpGff3uljNQAw8AhWAGLm/BkjlZ3R9scK04EAUg3BCkDM\nFGRn6NxpbRszv7phr7bsq/WxIgAYWAQrADF1aYeNmR/jrhWAFEKwAhBTx04YovFDc0NjNmYGkEoI\nVgBiytuYue2u1faKOr3y/h4fKwKAgUOwAhBzHTdmns90IIAUQbACEHMlhdk67bBhofHf39ml/WzM\nDCAFEKwA9IvwnlYNzS362yo2ZgaQ/AhWAPrFmVNHaEjYxszz32BjZgDJj2AFoF90tjHzO9srfawI\nAPofwQpAv7mkQ0+r+W+wiB1AciNYAeg3k0sKNCNsY+b5b2zR7179kClBAEmLYAWgX33ymPaL2P/3\nqXf0hd8v0z4+JQggCRGsAPSrTxw9RmdNHdHuuX+sKdM5d7+kV2kcCiDJEKwA9Kv0QJp+dfnR+ta5\nU5QRaOsauquyXp958F+687m1amxu8bFCAIgdghWAfpeWZrr61In665dObLePoHPSvS9s0CX3v6Yt\n+2p9rBAAYoNgBWDATB9TpKe/eoouPmp0u+dXbi7XuXe/rKdWb/epMgCIDYIVgAGVn5Wu/7tkpu66\ndKbys9JDz1fVN+mrf1qpGx9brZr6Jh8rBIC+I1gB8MWFs0brma+erBljCts9/9jyrTr/5//U29sq\nfKoMAPqOYAXAN+OG5umxL56oL542sd3zG/fU6KL7XtEDL2+k5xWAhEKwAuCrzPQ03XTOFP3hP4/T\nsIKs0PONzU7ff2aNrnroDe2prvexQgCIHMEKQFw4eVKxnr3+FJ0xeVi7519ct1tz73pZL7+326fK\nACByBCsAcWNofpZ+c+Ux+p/zDldmoO2Ppz3V9br8wX/rh4vWqKGJnlcA4hfBCkBcMTN97uQJevzL\nJ+qQYXntXrt/6UZ94pevatPeGp+qA4DuEawAxKUjRxfq6etO1qWzx7Z7fvXWCp1798t6YuVWnyoD\ngK4RrADErdzMdP3oP6br55+apYKwnlc1Dc36r/mr9f/mr1I1Pa8AxBGCFYC4d/6MUVp4/Sk6qrSo\n3fOPr9ymeT97WW9uLfepMgBoj2AFICGMHZKrP19zgq77yKGytr2ctWlvrS6+71Xdv3SDWlroeQXA\nXwQrAAkjPZCmr8+ZrD9+/jiNGNTW86qpxemHi9bqs7/9t8qq6nysEECqI1gBSDgnTizWoutP1VlT\nR7R7/uX39uicu17WC+vKfKoMQKojWAFISEPyMvXrK47WrR87QpnpbX+U7a1p0FW/fUO3Pv2u6pua\nfawQQCoiWAFIWGamy08Yr6euPUmThue3e+3Bf36gi+97VRt3V/tUHYBURLACkPCmlAzSU9eerE8f\nV9ru+Xe2V+q8n/9Tjy3bwmbOAAYEwQpAUsjJDOi2i6bpl5cdpUHZbT2vahuadeNf3tRXH12lyrpG\nHysEkAoIVgCSytwjR2rR107VseOHtHt+wertOvful7Vi836fKgOQCghWAJLO6KIcPfKF4/S1syYp\nLazn1db9B/SJX76me194X830vALQDwhWAJJSeiBNXzvrMM2/5gSNKswOPd/c4nTnc+t0+YP/0q5K\nel4BiC2CFYCkdsz4IVp0/ak658iSds+/umGv5t71khav2eVTZQCSEcEKQNIrzM3QfZ85SrddNE3Z\nGW1/7O2vbdR//m6ZvvvUO6prpOcVgOgRrACkBDPTp48r1YJrT9aUkoJ2rz306oe68N5X9Pa2ClXU\nNqqusZn2DAD6xPjDA701e/Zst2zZMr/LAPqsrrFZP1y4Rr97bVO3x2Wlpyk7I6DsjDRlpXtfszMC\nyk4PKKvjcxlpoeez0wNt52UEwq4TUHbw+47HZWcElBlIU1r4ansAccPMljvnZvd0XHpPBwBAssnO\nCOh7HztSJx1arG/89U2V13be36q+qUX1TS2qODBwtWWmp4XCV3a7UOYFOUuA3GWJUCRSwg8uPFJj\nh+QO6HsSrACkrDlHlGj6mCJ9bf5Kvb5xn9/lSJIamlrU0NSiyromv0sBEl5Nw8D/PiJYAUhpJYXZ\neuTzx+tfH+zTxj3VqmtsUV1js+obm1XX1OJ9bWxRXVOz6oLf1zc1tx3XekxTS/D1ZtEiC0hdBCsA\nKS8tzXTCxKE6YeLQqK/lnFNjs+sQvg4OZOHBrK6xORjcvGPqG9tCWvh59U3NivfMxrJdxJPs9MCA\nvwRah+8AAA5lSURBVCfBCgBiyMyUmW7KTE9TQXbPxwNILrRbAAAAiBGCFQAAQIwQrAAAAGKEYAUA\nABAjBCsAAIAYIVgBAADECMEKAAAgRghWAAAAMUKwAgAAiBGCFQAAQIwQrAAAAGKEYAUAABAjBCsA\nAIAYIVgBAADECMEKAAAgRghWAAAAMUKwAgAAiBFzzvldAxKMme2WtCnGly2WtCfG10w1/Ayjx88w\nevwMo8fPMHr98TMc55wb1tNBBCvEBTNb5pyb7XcdiYyfYfT4GUaPn2H0+BlGz8+fIVOBAAAAMUKw\nAgAAiBGCFeLFr/wuIAnwM4weP8Po8TOMHj/D6Pn2M2SNFQAAQIxwxwoAACBGCFYAAAAxQrACAACI\nEYIVfGNmJWZ2t5ltMLM6M9tlZgvM7Ey/a4t3ZlZqZl8L/rw2m1m9mVWZ2Wozu93MRvpdY6Ixs3wz\n22JmLvi40u+aEomZTTazn5vZOjOrMbMKM1tjZr8xs9P8ri+emVmamV1lZv8ws91m1mhm5Wb2LzP7\ntpkV+F2j38yswMwuMLNbzWyRme0J+706JYLzM83sG2a2ysyqgz/f18zsajOzmNbK4nX4wcymS1oi\naWjwqUpJ+fLCvpP0Lefc7T6VF9fMbKy8zvfhfxhUSsqTFAiO90v6uHPuhQEuL2GZ2V2Srg976irn\n3EM+lZNQzOyrku6UlBl8qlpSuqTs4PhB59z/b+/Og+Yo6jCOfx/ClUCSN1wGFEy4z+ISiIga0CBX\nFLU8EAsjClgFFimxFEQRoSSFlgIKKqcRDzwBFfACBQ8IKgEswGAUgogQFMxBDrl+/tE9vsMyu++R\neXfffd/nUzU129M9s/2+tbvzm56e7vd3om7DnaRxwI+Bg0qblwIT6P2OPwQcFBEPtLl6w4akI4Fr\nmmTvFBELWuw7gXS+2TtvWkn6fBaf1+uAN0fEs3XU1S1W1naSxgI/IgVVdwK7RsREYBLwOdKPyTmS\nDu5cLYe1Ini6HngbsFH+/40DDgMeJP0vr5U0uTNV7C6S9gJOAm7vdF26jaQTgAtIJ6pzSdN+jI+I\nscDmwDHArR2s4nD3CVJQFcBpQE9E9JCC0qOAJcDLgcs6VsPh43HgBuBTwPED2O9SUlD1JDCTdBE/\nDpgFrAaOyMeshVusrO0kzQbOI13V7hgRjzTkXwMcCcyPiL0rDjGqSZoITImIu5vk70gKWNcHzoyI\n2n4wRiJJa5ECqj2BfYD5OcstVn2QNAW4l3SSOj4iLu1ohbqQpIeArYArIuJ9FfmzgK/m5EYR8Z82\nVm/YkDQmIp4rpaeQLiKhRYuVpD3p/U6/KSJ+1JB/MnA+sIr0u/r4mtbVLVbWCUfn9bcag6rss3m9\nl6Qd2lSnrhERS5sFVTl/ATAvJx2Y9u2DwCuAL0fEnZ2uTJc5mRRU3e6gatBektfNPnt3lF6PG+K6\nDFvloGqA3pXX9zcGVdklpFuvY4G3DPI9XsCBlbVV7oRZnOx/1qTYPNIHHcAd2Qfnibwe07LUKCfp\npcDZwGLg4x2uTjcqTlpXdbQW3W1RXu/ZJL/4vVzc5ELUWjswr39elRkRq4Df5ORBVWUGyoGVtdtO\n9HbIvLeqQEQ8D9yfkzu3o1IjiaS1gVfl5D2drEsX+CIwHvhwRCztq7D1krQNsFlO3ilpWn5K9QlJ\nqyQtkPRZSZu1Oo5RtPS9V9Kp+VZ/8RTbO0jdJgL4cKcq2K3y037FE4OV55vsvryu5XzjwMrarTwM\nwD9blCvyPGzAwJ0ITAaeB77W4boMW5JmAm8Gbo6Ib3S6Pl1ou9Lr6cBvSZ2A1yEFAjuQgoG7JO3S\n9tp1j/OBi0gXnHOAJZKWkPr8fBtYALzRn9FBmUB6WhraeL5xYGXttkHp9aoW5Vbm9YZDWJcRJw9j\nMScnL4yI+1qVH60kbQBcCDxDCkRt4HpKrz8J/AWYFhETSN/bw0hPcW0O/CC3pFqD3HdoNnAKUDzu\nP5He8/N4YNMOVG0k6Mj5xoGV2QiRBwW9ltQJ8w7go52t0bB2FulJrPMcfA5a+fwRpHGAbod0Oz8i\nfgIcm/N3oKaOwSNNHhLld6ShZr4J7E46wW9HGn5ha+AKSXOaHsSGFQdW1m4rSq/HtihXPP3y1BDW\nZcSQtBGpc+ZUYCFweESs7mythidJe5CeZnuYFGDZ4JS/mz+NiPsbC0TE9aSWLPCDKM1cCexLGkR1\nVkT8KSJWRMRf8yDJJ+RyH/Et1QHryPnGgZW1W/k+9xYtyhV5jw5hXUaE3Nn1Z8CuwN+B10fE4s7W\nali7gPS05Omk/q0blpdSufXytlH7iHsfyt/lFwVVFXlbDmFdupKknYEZOXleVZmI+DrpKd+1SINb\nWv8toze4atv5xoGVtdsC0m0DgMqrrzxgYzF+lW/TtJD7Ct1AGofpMVJQ9ffO1mrYe3leXwksr1gK\nX8lpfwar3Ud6QKK/PBr1i+1Uev1g01JQTGUzZeiqMvJEGgH9zznZqrWveBqwlu+6Aytrq4hYDvwx\nJ2c0KbYfqfMmwE1DXqkulacG+jGwP+mK9vURsbCztbLRIiJWArflZKuBfIu8RUNaoe5UDky3alGu\nuBhY3qKMVSvmS60830haH3h1TtZyvnFgZZ3wrbw+One4blSM13JHVb8NS2PcAFeTBr9bAhwcEa3G\nabEsIqZEhJotpaLvzdumdKquXeDKvD6kapYESYcD2+fkDW2rVfcoz6BwXFWBPCxIMRaY57IcuGLw\n2h0lHVGRfxzpQn4VzSd5HhAHVtYJF5Nmax8PXJf7GSBpvKTP0Pv00Mc6VL9hTdIYUnB6COkK9tCI\nmN96L7MhcQXp9skY4GpJ+0K6nS/pEODyXG4eDqxeJCIeoHdE8NmS5hQDqub+fbOAuTl/EWny+lFL\n0ibFQppovtBTzsvdSQDI01R9NyfnSjosH2uMpGNIE4dDekJ4jecJBE/CbB0iaXdSs+vGedMy0iPG\na5H6YnwsPxFjDSS9BrglJ1fTO/1PlYcjYp+hr9XIIan4UfQkzP0gaWvgZno7py8nBVpFp//7SC2q\nno6lQm61v4kX9rdaTrrwLCwmXUCN6rksS9/NvkyNiEWl/SYAv6R3eqCVpM/oejl9HWm4kGepgVus\nrCPyJMK7Al8gdcxcj9RP6HpghoOqlsrf2/VJk7g2WzywoA2p3OqyG/BpUhC1NuniaD5pHKZ9HVQ1\nFxGPkk74s4FfA0+SgtJlpP/h2cBuoz2oWhMRsYzUF/VU0u3XAP5Lakk9gTSyfS1BFbjFyszMzKw2\nbrEyMzMzq4kDKzMzM7OaOLAyMzMzq4kDKzMzM7OaOLAyMzMzq4kDKzMzM7OaOLAyMzMzq4kDKzMz\nM7OaOLAyMzMzq8nana6AmdlIImlt4N3AO4HdSfNhrgAeI03f9BvglxHx+9I+ewBHAos8P6FZd/OU\nNmZmNZG0KXAD8IrS5tWkeckmAMrblkZET2m/WcBXgVsiYnpbKmtmQ8K3As3M6vMNUlC1HPgIsHlE\njM1B1ERgBvAlYEnnqmhmQ8m3As3MaiBpR+DgnDw2Ir5fzo+I5cCNwI2STml3/cysPdxiZWZWj91K\nr69rVTAiVhevJQXpNiDAayVFwzK9cX9JB0j6tqR/SPqvpCck3SjpKEmqKD89H2tRTs+U9CtJ/5H0\nlKTbJL1rEH+zmTVwi5WZWf1eCvytn2UXA2NJfbCeAZ5syH+6nJB0Luk2Y2EZMAl4XV7eKOnoiHi+\n6s0kzQbOAwJYmt97GjBN0v4RcVI/621mFdxiZWZWjztKry/KHdn7FBGTgZNz8taImNyw3FqUlXQy\nKahaDBwP9ETERGAD0lOIj+X1R5u83abAZ4ArSf2/JgGbAJ/L+Se65cpszfipQDOzmkj6GnBMTj5N\nGlphHvAHUtD0ryb7zaKPpwIl9QAPk+40TIuIuyvKvBL4Halz/OSIeDpvnw78Khf7BfCGaPjxlzQX\neA/wV2D7xnwz6x+3WJmZ1ec44POkoGpd0q2504Frgccl/V7S0VX9oPrhrcCGwI1VQRVARNwGPEi6\nNbh3k+PMaRI0fTqvtyWNv2Vmg+DAysysJhHxdEScAmwJfAC4ClhI6s8EsA9pSIbvSBro7+/+eX2Q\npMeaLfm9Ka3LniG1aFXVfSHwaE7uNcC6mVnmzutmZjWLiMeBi/OCpJcAM4EzSAHP20gBzgUDOOzm\neT0uL32pKvPv4vZgE4/k9+lX/zAzezG3WJmZDbGIWBwRl5FaghbnzccO8DDF7/UFEaF+LHPrqr+Z\n9Z8DKzOzNomIfwM/zMntB7h7EZBttQZV2ETSui3yt8jryk72ZtY3B1ZmZu21Iq/Lt+SKMadadWq/\nLa+nSxo7yPdeB3hlVYakbekNrOYP8vhmo54DKzOzGkiaKmmbPsqMA47MybtKWcvyuofmvkcKyiaR\n+mq1ep9JLbJPa/JU4ml5vTAi7qrIN7N+cGBlZlaPXYD7JV0t6e2Sis7mSNpA0kzSuFZT8+Zyx/V7\n83pnSftVHTwinqA3+DlV0qWS/n87UdI4SQdKugS4teoYwErSEBCXS9os79eTR3Mv+nyd2c+/18wq\neIBQM7MaSHoD8NOGzatIt/wmlrY9B5wREec07H8L8JqcfBJYnl+/MyLmlcp9HDiL3tuGK/J79JS2\nLYqIqaV9ppMGCH0IOJ/eKW2W5LoVF9kXeUobszXjwMrMrCa5BWkmcACwK2nOwHVJQdIDwK+ByyLi\n3op9NyYFTIeW9gM4MCJubii7G3AScCDwMmAMqcP5PcBNwFUR8Y9S+enkwCoipuTWsw8Be5L6Xf0J\nuDAivrnG/wSzUc6BlZnZCNcYWHW2NmYjm/tYmZmZmdXEgZWZmZlZTRxYmZmZmdXEgZWZmZlZTdx5\n3czMzKwmbrEyMzMzq4kDKzMzM7OaOLAyMzMzq4kDKzMzM7OaOLAyMzMzq8n/AOCyUeqcq8QGAAAA\nAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "objective = lambda w: likelihood(w, X, y)\n", "gradient = lambda w: log_likelihood_gradient(w, X, y)\n", "hessian = lambda w: log_likelihood_hessian(w, X, y)\n", "init = np.zeros(X.shape[1])\n", "\n", "cls = linear_model.LogisticRegression()\n", "cls.fit(X, y)\n", "weights = np.reshape(cls.coef_, cls.coef_.shape[1])\n", "baseline = log_likelihood(weights, X, y)\n", "\n", "ws_newton = newton_method(init, np.ones(10), gradient, hessian)\n", "error_plot([log_likelihood(w, X, y) for w in ws_newton])\n", "plt.plot(range(len(ws_newton)), [baseline]*len(ws_newton),\n", " label='sklearn', **kwargs)\n", "_ = plt.legend()" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAH/CAYAAACVXpOkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8VPW9//HXN5N9JwkkQNhBlkCQigu4F0GsYtXWVlsX\nbK1a19tqe21/9t5W721d2qu02ta61LrbWq07YEFxARREWWURZCcEQvaQbeb7+2Mmw2RlskzOzOT9\nfDzmEc4y53zwUnjf8/2c79dYaxERERGR7otxugARERGRaKFgJSIiItJDFKxEREREeoiClYiIiEgP\nUbASERER6SEKVn2UMSbVGPMrY8ybxpgDxhhrjLnd6bpEREQimYJV35UD/BcwCfjU4VpERESiQqzT\nBYhj9gGDrbV7jTHDgS+dLUdERCTyKVj1UdbaOmCv03WIiIhEEwUr6bScnBw7fPhwp8sQERHpNZ98\n8slBa23/o50XtsHKGNOZtXbOsNYuCVkxQTDGpAFnAscDU30/s32Hx1trNwZ5nTzgZ8B5wGCgHPgY\neMBau6in6+6K4cOHs3LlSqfLEBER6TXGmB3BnBe2wQrYf5Tj6UASUA+sC305RzUDeLk7FzDGFAKL\nORLIKvA2mZ8HnGuM+bm19u4W3zFAQpC3qLfWerpTo4iIiLQvbN8KtNbmdfQBNvtOfd1aW+JkrQGK\ngTeBXwHXdOaLxpgk4FW8oepTYKK1NgPoB/wOMMCvjTGzWnz1ZOBwkJ/TuvS7EhERkaCE8xOrdhlj\njgUm+zb/FuR3YoE/AfdZazcf5dw84D7gOmttdZBlvWat/VfANYYH+b0m1wLDgCpgjrV2D4C1tgK4\nzRgzCrgA+A2wMOB7m4GrgrxHUMORIiIi0jURGayAK30/m54QBeNW4GpgtjHmNGttm9MLGGNygH8D\nBXiH4m4I5uLWWneQdbTnu76fzzaFqhbuwxusvmKMGWut3eS7bzHwRDfvLSIiIj0gbIcC2+N78vQd\n3+az1trGIL/6IPAhkA8sMsbkt3HtDLxPgwqATXiH9ELO1/h+nG9zQTunLcfbyA7efi4REREJM5H4\nxOocYIDv10ENAwJYa6uNMefibQ7/Ct5wdZq1dj94l3gB5gNT8E6WOcP3NKg3jMfbQwWwvq0TrLUe\nY8wm4ARgQk/c1BhzI5Dp+wCc6QuuAH+w1pa3/U0RERFpSyQGq7m+n6uttZ915ovW2nJf8/cSvE+l\nFhljTgdqgNeBk4A9eENVW8NxoTIw4NcdTdrZdGxgB+d0xm14+7qazPJ9AJ7myBMyERERCUJEBStj\nTBbeqQegE0+rAllrS4wxZwHv4w1XC4EDwOl4e7ZmtNd/FUIpAb8+3MF5Nb6fqT1xU2vt8M6cb4yZ\nA8wZPXp0T9xeREQk6kRaj9WlQDzQCDzT1YtYa4vw9intxDsseDZQCsxsagqX1qy1r1lrr8nIyHC6\nFBERkbAUacGq6W3At3qg/2kP3vmimmwEtnTzml0VOKVDUgfnJft+VoWwFhEREemiiBkKNMaMx7tM\nDHRxGDDgWgZ4DPg63lBTDUwDXjLGfN1aW9+d63dBYF/VILxvJLZlkO/nvtCWIyLSOdZaKisrqaio\noKamBre7uzPQiPQsl8tFcnIy6enppKWl4Y0CPS9ighVHmtYPAa9181oP4X36dRiYAxwE3gVmAy8Y\nYy7uxDQOPWEjYPG+Gdg01UMzxpgYYKxvc0PvlSYi0jFrLcXFxVRXV5OVlUVeXh4ulytk/3CJdJa1\nFrfbTVVVFQcPHuTw4cMMGDAgJH9GI2Io0BjjAi7zbT7XnSdKxpjfAj/Eu8bgRdbad6y1a/G+DVeB\ndxLOp3xBpldYayuBplWNZ7Zz2olAU3NTWCzGLCICUFlZSXV1NcOGDSMzM5PY2FiFKgkrxhhiY2PJ\nzMxk2LBhVFdXU1lZGZJ7RUSwAs7iyDBYl4cBjTF34p2BvRH4trV2ftMxa+0nwNfwDgteAjxqevdv\nhmd9P79rjGlrOoXbfD8/UYO9iISTiooKsrKycLlcTpciclQul4usrCwqKipCcv1ICVZNTesbrLUr\nunIBY8y1wC8AD3B54Lp+Tay1HwLnA7V419/7dSfvkdP0wbt4cpPMwGPtPA17GNgBpAGvG2Mm+K6Z\nZoy5F7jId97PO1OTiEio1dTUkJraI7PAiPSK1NRUampqjn5iF4R9sDLGpOMdnoPuNa2/BewCrrbW\nPt/eSdbaxcA38M5t9UIn73Eg4LMqYP+yFseGtnHfw3ib6UvwTgGx3hhTDpQBP8Hbg/Uza+3Clt+N\ndCVVdbyzqRiPxzpdioh0gdvt1tMqiSgulytkL1hEQvP6t/BOQeDBOxt4l1hrdxpjCnz9TEc7901j\nzKhgzu1J1trVxpiJwM/wToQ6GG/Q+hi431obVb1V72ws5o5/rWNPmXdO1MW3ns7I/vr/ekUikXqq\nJJKE8s9r2Acra+2jwKM9dK2gg1JXQpW1ttv/l/JNXnqL7xPV0pPi/KEKYM3ucgUrERGJaGE/FCjR\nq2BQOrExR7Lo6t1lDlYjIiLSfQpW4pjEOBfH5Kb5t9fs1prPIiIS2RSsxFGThxxZd3D93nIa3R4H\nqxERCV9PPPEExhjOOOOMTn3vl7/8JcYY5s6dG5K6pDkFK3FUYX6m/9e1DR4279cyiCIiErkUrMRR\nhfkZzbbXqM9KREQimIKVOOqY3DQSYo/8MVytPisREYlgClbiqDhXDBMHH3lqpSdWIiISyRSsxHGB\nw4GbiiqpbQjNbLgiIuGmvr6eefPmMX36dDIzM4mLiyM3N5fJkydzww03sGzZsqCv9dRTTxEXF0dM\nTAx/+tOfOlXHBx98wCWXXEJ+fj4JCQlkZ2dz1lln8dxzz2Ft26tirFq1ittvv51TTjmFoUOH+r93\nxhln8Oijj7Y7s3lgM73H4+HBBx/khBNOIDMzE2MMn332GQBz587FGMMvf/lL3G43DzzwAJMnTyY5\nOZmsrCzOO+88Vq5c2anfZ28I+wlCJfpNDmhgb/RYNuyr4CtD+3XwDRGRyNfY2MisWbNYsmQJ4J0N\nPCMjg5KSEoqLi1mzZg0lJSVMmzbtqNd66KGHuOmmm3C5XDz55JNcdtllQdfxn//5n9x7773+7fT0\ndEpLS1m0aBGLFi3i1Vdf5ZlnniEmpvmzmFmzZlFSUgJAcnIyycnJHDp0iCVLlrBkyRJefvllXnnl\nFWJj244a1louuugiXnnlFVwuF2lpaW2e19jYyLnnnsuCBQuIi4sjISGB0tJS3njjDRYtWsTixYuD\n+m/UW/TEShzXqoF9l4YDRST6PfvssyxZsoTk5GSeeuopampqKC0tpa6ujh07dvDggw8yefLko17n\nf//3f7nxxhuJj4/nxRdf7FSomjdvHvfeey+5ubn85S9/oaysjPLycqqrq3n++efJy8vj+eef5557\n7mn13VmzZvHcc8+xb98+qqurKS0tpaqqiqeeeoq8vDzefPNN7r///nbv/dJLLzF//nz++Mc/UlFR\nQWlpKfv372fkyJHNznvooYdYsWIFL7zwAlVVVVRWVrJ69WomTpxIbW0tt9wSXguV6ImVOG54dgpp\nibFU1jYCmihUJOq8dTsUrXW6ip6RNwnOubtHLrV8+XIArrjiimZhyOVyMXToUG644YajXuMnP/kJ\nv/3tb0lJSeGVV15hxowZQd+/rKyMO+64g8TERBYsWNAsxCUlJfHtb3+boUOHcvLJJ3Pfffdx6623\nEh8f7z/n2WefbXXNlJQULrvsMoYNG8Zpp53GH//4R37yk5+0ef+qqioefvhhrrnmGv++AQMGtFnn\n+++/zymnnOLfV1hYyBNPPMHUqVNZsWIFO3fuZOjQoUH/3kNJwUocFxNjKMzP4MMvvI+UtbSNSJQp\nWgs7PnC6irCTnp4OwL59+zr9XY/Hw7XXXsujjz5KZmYmb775ZqeHw/75z39SVVXFeeed1+6TsWnT\npjFixAi2bdvGJ598EvQ9Tj31VDIzM9m+fTt79+5l0KBBrc7Jzs7me9/7XlDXCgxVTY477jjy8/PZ\nvXs369atC5tgpaFACQuBE4VuO1hNZW2Dg9WIiITeOeecA8Arr7zC+eefz0svveTvWepIQ0MDl156\nKY8++igDBgzg3Xff7VKP0dKlSwFYvHgxeXl57X527doF4P8Z6B//+AcXXHABQ4cOJSkpCWOM/1NW\n5v1/kvfu3dvm/adOndpu/1Wg448/vt1jgwcPBqC0tPSo1+ktemIlYWFyQJ+VtbB2TznTR+U4WJGI\n9Ji8SU5X0HN68Pdy+umnc+edd3LnnXfy2muv8dprrwEwbtw4zj33XK699lrGjBnT6ntNgQjgxRdf\nDKoPqy1NT8pqamqoqak56vmB5zQ2NvKtb32Ll19+2b8vISGBnJwcXC4XAAcOHMDj8VBdXd3m9fr3\n7x9Une01tQMkJiYC3rAZLhSsJCwEPrECWL1LwUokavRQT1I0+sUvfsFll13GCy+8wLvvvsuyZcvY\nuHEjGzduZN68eTz22GNcccUVzb4zadIk3G43GzZs4Prrr+edd94hJ6fzf196PN61WW+55RYeeOCB\nTn33kUce4eWXXyY5OZm7776bCy+8kPz8/GbnDBkyhN27d7c7XUNTAIs2GgqUsDAwI5Gc1AT/tiYK\nFZG+YsSIEdx+++3Mnz+fQ4cO8c4773DaaafR2NjI9ddfT3FxcbPzs7KyWLRoEWPHjmXdunXMnDmz\nS0Nhubm5AOzcubPT3/3HP/4BeIPhTTfd1CpUud1uDh482OnrRgMFKwkLxphmw4F6M1BE+iKXy8UZ\nZ5zB66+/TlxcHNXV1W1OgpmXl8eiRYsYNWoUn332GbNmzaK8vHN/bzb1Zb377rscPny4U9/dvXs3\nAFOmTGnz+IcffkhtbW2nrhktFKwkbAQOB+4pO8zBqjoHqxERCa36+vp2j8XHx/uHyurq2v67cPDg\nwSxevJjhw4ezcuVKzjnnHKqqqoK+/8UXX0xKSgqlpaXceeedHZ7b8olYRob3/xFeu7b1NBqNjY3c\ncccdQdcRbRSsJGwUDmkxUaiGA0Ukil1xxRVcddVVLFiwgMrKSv/+7du3c+WVV1JbW0tSUhKnnnpq\nu9cYOnQoixcvJj8/n2XLlnHuuecG1YgO3ukOfvOb3wBw991384Mf/IDNmzf7j9fU1PDOO+9wzTXX\nMH369GbfnTlzJgB33XUXr7zyin/5mo0bNzJnzhw+/vhjUlJSgvsPEWUUrCRsTG6jgV1EJFrV1tby\nxBNPMHv2bDIyMujXrx8pKSmMGDGCF154AZfLxcMPP3zUxvQRI0awePFiBg4cyHvvvcf5558f9DDc\nTTfdxF133YUxhkcffZSxY8eSmppKVlYWqampfPWrX+WRRx5pdb3bbruNUaNGUVFRwQUXXEBSUhIZ\nGRmMHz+et99+mz//+c9daqiPBgpWEjayUuLJ75fk39YTKxGJZnfffTf33nsvs2fPZuTIkdTX1+N2\nuxk1ahRXXXUVq1at4vLLLw/qWmPGjGHx4sXk5uayaNEiLrzwwnaHEFu64447WL16Nddccw1jxozx\nT5EwaNAgzj77bO69917ef//9Zt/Jyspi+fLl/PCHP/Q3riclJXHBBRewZMkS5s6d26n/FtHEtPca\npEh7pk6dakO1ovgNz6zijbXeuVWyU+JZecdZGGNCci8R6Rmff/4548ePd7oMkU7p7J9bY8wn1tqp\nRztPT6wkrAQuyFxSXc+ess69qSIiIuIkBSsJKy0nCtW0CyIiEkkUrCSsTMrPIHDkTwsyi4hIJFGw\nkrCSmhDLqP6p/u01ejNQREQiiIKVhJ3APqt1e8rxePSChYiIRAYFKwk7gfNZVdY1su1g2yuji4iI\nhBsFKwk7k4e0bGBXn5WIiEQGBSsJO+MHphHnOtLBrjcDRUQkUihYSdhJiHUxLi/dv603A0XCnyab\nlkgSyj+vClYSlgIb2DfsraDB7XGwGhHpiMvl8i/CKxIJ3G43LpcrJNdWsJKwFNjAXtfoYVNRZQdn\ni4iTkpOTqaqqcroMkaBVVVWRnJwckmsrWElYKhyS0WxbfVYi4Ss9PZ1Dhw7pqZVEBLfbzaFDh0hP\nTz/6yV2gYCVhaXT/VJLijjym1ZuBIuErLS2NlJQUduzYQVlZGY2Njeq5krBiraWxsZGysjJ27NhB\nSkoKaWlpIblXbEiuKtJNsa4YJg5OZ8X2UgBW64mVSNgyxjBgwAAqKyupqKiguLhYT68k7LhcLpKT\nk8nJySEtLQ0TuH5aD1KwkrBVmJ/pD1ab91dyuN5NUnxomg1FpHuMMaSnp4dseEUkUmgoUMJW4JuB\nbo9l/V49tRIRkfCmYCVhK/DNQNBwoIiIhD8FKwlbw7KTyUiK82+rgV1ERMKdgpWELWNMs+FATbkg\nIiLhTsFKwlpgsPryYDXlhxscrEZERKRjClYS1gpb9Fmt1VMrEREJYwpWEtZaN7Crz0pERMKXgpWE\ntbyMRAakJfi31cAuIiLhTMFKwl7gcKAa2EVEJJwpWEnYmxzQwL6vvJbiyloHqxEREWmfgpWEvcIh\nzfus1uzSUysREQlPClYS9goHZzTbVp+ViIiEKwUrCXv9UuIZmpXs39bSNiIiEq4UrCQiTB4S2MBe\nhrXWwWpERETapmAlESGwgb20poHdpYcdrEZERKRtClYSEVrOwK6JQkVEJBwpWElEmDg4nRhzZFvz\nWYmISDhSsJKIkBwfy5gBaf7t1bv0xEpERMKPgpVEjMKAPqt1e8pxe9TALiIi4UXBSiJG4ESh1fVu\nth2ocrAaERGR1hSsJGIEvhkIms9KRETCj4KVRIxxeenEu478kdUM7CIiEm4UrCRixMfGMH5gQAO7\nnliJiEiYUbCSiBI4n9Xneyuob/Q4WI2IiEhzClYSUQLfDKx3e9hYVOFgNSIiIs0pWElECVwzEDQc\nKCIi4UXBSiLKqP6pJMe7/NtrNFGoiIiEEQUriSiuGMPEwUeGA7W0jYiIhBMFK4k4gfNZbSmupKa+\n0cFqREREjlCwkogT+Gagx8K6PWpgFxGR8KBgJRFncn7zBnZNFCoiIuFCwUoizpCsJPolx/m39Wag\niIiECwUriTjGGCYFPLXSEysREQkXClYSkQIb2HeU1FBWU+9gNSIiIl4KVhKRClv1WWk4UEREnKdg\nJREp8IkVaDhQRETCg4KVRKQB6YkMzEj0b6uBXUREwoGClUSswAWZ9cRKRETCgYKVRKzAPqv9FXXs\nr6h1sBoREREFK4lgLScKXa0FmUVExGEKVhKxJrVqYFeflYiIOEvBSiJWRlIcI3JS/Nur1WclIiIO\nU7CSiBbYwL52TznWWgerERGRvk7BSiJaYAN7WU0DOw/VOFiNiIj0dQpWEtFaThSq+axERMRJClYS\n0QoGZeCKMf7tNXozUEREHKRgJREtKd7FmAGp/m01sIuIiJMUrCTiBc5ntW5PBY1uj4PViIhIX6Zg\nJRGvcMiRPqvDDW6+OFDlYDUiItKXKVhJxGs5A/uaXWpgFxERZyhYScQbm5dGfOyRP8rqsxIREaco\nWEnEi3PFMGFgun9bS9uIiIhTFKwkKgTOZ7WxqIK6RreD1YiISF+lYCVRIXAG9ga35fN9lQ5WIyIi\nfZWClUSFyUOaz8C+Rn1WIiLiAAUriQojc1JJTYj1b6/Wm4EiIuIABSuJCjExhomDAxvY9cRKRER6\nn4KVRI3JQ470WX1xoIqqukYHqxERkb5IwUqiRuBEodbCuj0aDhQRkd6lYCVRozBfDewiIuIsBSuJ\nGoMzk8hOifdvr9ZEoSIi0ssUrCRqGGOaPbXSEysREeltClYSVQInCt116DCHqusdrEZERPoaBSuJ\nKpooVEREnKRgJVEl8IkVaEFmERHpXQpWElVyUhMYnJnk39YTKxER6U0KVhJ1AhvYV+8ux1rrYDUi\nItKXKFhJ1AkcDjxQWUdRRa2D1YiISF+iYCVRZ3KLiUJX79JwoIiI9A4FK4k6E1sGKzWwi4hIL1Gw\nkqiTnhjHyP4p/m01sIuISG9RsJKoFLgg85rd5Xg8amAXEZHQU7CSqBT4ZmBlbSPbS6odrEZERPoK\nBSuJSpooVEREnKBgJVGpYFA6sTHGv71afVYiItILFKwkKiXGuTgmN82/rSdWIiLSGxSsJGoFLsi8\nfm85jW6Pg9WIiEhfoGAlUSuwz6q2wcPm/VUOViMiIn2BgpVErcIWE4VqPisREQk1BSuJWsfkppEY\nd+SPuGZgFxGRUFOwkqgV54qhYNCRp1Z6YiUiIqGmYCVRLXA4cFNRJbUNbgerERGRaKdgJVEtcGmb\nRo9lw74KB6sREZFop2AlUa1VA/suDQeKiEjoKFhJVBuenUJaYqx/WxOFiohIKClYSVSLiTHNnlpp\naRsREQklBSuJeoEThW47WE1lbYOD1YiISDRTsJKoNzngiZW1sHaPhgNFRCQ0FKwk6gU+sQL1WYmI\nSOgoWEnUG5iRSE5qgn9bE4WKiEioKFhJ1DPGNBsOXL1LT6xERCQ0FKykTwgcDtxTdpiDVXUOViMi\nItFKwUr6hMIhLSYK1XCgiIiEgIKV9AmTWzSwazhQRERCQcFK+oSslHjy+yX5t/XESkREQkHBSvqM\nwKdWa3aXY611sBoREYlGClbSZwQubVNSXc+essMOViMiItFIwUr6DE0UKiIioaZgJX3GpPwMjDmy\nrQWZRUSkpylYSZ+RmhDLqP6p/u01ejNQRER6mIKV9CmBfVbr9pTj8aiBXUREeo6ClfQpxw450mdV\nWdfItoPVDlYjIiLRRsFK+pTWDezqsxIRkZ6jYCV9yviBacS5jnSw681AERHpSQpW0qckxLoYl5fu\n39abgSIi0pMUrKTPCWxg37C3gga3x8FqREQkmihYSZ8TuLRNXaOHTUWVDlYjIiLRRMFK+pzCIRnN\nttVnJSIiPUXBSvqc0f1TSYpz+bf1ZqCIiPQUBas+yhiTaoz5lTHmTWPMAWOMNcbc7nRdvSHWFcPE\nwYEN7HpiJSIiPUPBqu/KAf4LmAR86nAtvS5wPqvN+ys5XO92sBoREYkWClZ91z5gsLV2CHCN08X0\ntsA3A90ey4Z9emolIiLdp2DVR1lr66y1e52uwymTW8zAvloLMouISA9QsJI+aVh2MhlJcf5tNbCL\niEhPiIhgZYwZa4z5gzFmkzGm2hhTboz53BjzuDHmdKfrAzDGpBljzjfG3GWMecsYc9DXEG6NMeM6\ncZ08Y8w8Y8xWY0ytMWa/MeY1Y8yMUNbf1xhjmg0HqoFdRER6QqzTBRyNMeZm4D4g3reryvfrcb6P\nB1jiTHXNzABe7s4FjDGFwGIg27erAm+T+XnAucaYn1tr727xHQMkBHmLemutphn3KczP4P0tBwH4\n8mA15Ycbmj3FEhER6aywfmJljLkWmIc3AN4DDLPWpllrk4CBwBXAUgdLbKkYeBP4FZ1sCDfGJAGv\n4g1VnwITrbUZQD/gd4ABfm2MmdXiqycDh4P8nNal31WUKmzRZ7VWT61ERKSbwvaJlTFmOPB/vs3r\nrLWPBB631hYBT3XierHAn4D7rLWbj3JuHt6nZNdZa6uDvMVr1tp/BVxjeLC1+VwLDMP7RG6OtXYP\ngLW2ArjNGDMKuAD4DbAw4HubgauCvMfGTtYU1Vo1sO8u45QxOQ5VIyIi0SBsgxVwC5AMfNQyVHXR\nrcDVwGxjzGnW2i/bOskYkwP8GyjAOxR3QzAXt9Z2dyKk7/p+PtsUqlq4D2+w+ooxZqy1dpPvvsXA\nE928d5+Ul5HIgLQEiivrADWwi4hI94XzUOB3fD+f66HrPQh8COQDi4wx+S1PMMZk4H0aVABswjuk\nF3LGmDTgON/mgnZOWw40jVWpkb2HBA4Has1AERHprrAMVr5hrwG+zU+NMSf53owrMcYcNsZsNMbc\nZ4wZ0NF1AvmG9M4FVgEj8Iar3IB7pgLzgSnAl8AM39Og3jAebw8VwPq2TvA1nW/ybU7oiZsaY240\nxtwB3OjbdaYx5g7fJ6Oj70aLyQFvBu4rr6W4stbBakREJNKF61DgmIBfn4F36RUXUAlYYKzv811j\nzExrbZthpCVrbbmv+XsJ3qdSi3zTNdQArwMnAXvwhqq2huNCZWDArzuatLPp2MAOzumM2/D2dTWZ\n5fsAPM2RJ2RRq3BI8z6rNbvKOWtCokPViIhIpAvLJ1ZA4L92/423Qfska206kAp8De8beAOBf/oa\n04NirS0BzgK+wBuuFuKdJuF03zVntNd/FUIpAb8+3MF5Nb6fqT1xU2vtcGutaeezveX5xpg5xpi/\nlJdHT94qHNz8wZz6rEREpDvCNVgF1mWBC621H4F3SMxa+xbwPd/xscBFnbm4743CGcBO4CvA2UAp\nMLOpKVxas9a+Zq29JiMjekYJ+6XEMyw72b+tiUJFRKQ7wjVYVQX8en5bYcda+wbeJ1nQtWbuPXjn\ni2qyEdjShev0hMApHZI6OK8pAVR1cI50UvMG9jKstQ5WIyIikSxcg1Vgn1FHT5Cajg3pzMV9s5U/\nBnwdb6gpBqYBLxlj4jv6bogE/n4HdXBe07F9IaylzwlsYC+taWB3aUejsSIiIu0L12C1Ae9SNcHq\n7COGh4Ar8fYzzcHbc3UImA280JmerR6ykSO/h4K2TjDGxOAd9gTvfx/pIS1nYF+tPisREemisAxW\n1toaYJlvc2wHpzYd2x7stY0xvwV+CNQDF1lr37HWrsX7NlwF3kk4n/IFmV5hra0EVvo2Z7Zz2olA\n06OVRSEvqg+ZODidGHNkW/NZiYhIV4VlsPJ50vdztjGmVbgyxpwLHOPbfDOYCxpj7sQ7A3sj8G1r\n7fymY9baT/C+bVgNXAI86hsy7C3P+n5+1xjT1nQKt/l+fqIG+56VHB/LmAFp/u3Vu/TESkREuiac\ng9XjeIe8XHh7n04A75CYMWY23h4p8M5IftRg5VvQ+Rd4hxgvD1zXr4m19kPgfKAW7/p7v+5MwcaY\nnKYP3sWTm2QGHmvnadjDwA4gDXjdGDPBd800Y8y9HHnz8eedqUmCUxjQZ7VuTzlujxrYRUSk88I2\nWFlrG/HAHEyVAAAgAElEQVT2P+3CO9P4R8aYCryThL4F5OINXt+0wb3G9ZbvWldba5/v4L6LgW8A\nB4AXOln2gYDPqoD9y1ocG9rGfQ/jbaYvwTsFxHpjTDlQBvwEbw/Wz6y1C1t+V7ovcKLQ6no32w7o\nxUsREem8sA1WANbabcAk4H/xhqhYvAFjFfAz4IRgZ0i31u4ECqy1fw3i3DeBUdbaz7pae1dYa1cD\nE4HfA9uABLxB6w28c2zd3Zv19CWBbwaC5rMSEZGuCdclbfysteXAHb5Pd69VGYpzA77T7Z4s3+Sl\nt/g+0kvG5aUT74qh3u19GXXN7jK+eVyrdbpFREQ6FNZPrER6S3xsDOMHBjSw64mViIh0gYKViE/g\nfFaf762gvrEzU6mJiIh0I1gZY272fTqaKVwkYgS+GVjv9rCpqNOjwSIi0sd154nV/cBvgYM9VIuI\noyYP0QzsIiLSPd0JVgeBSmttfU8VI+KkUf1TSY53+bc1UaiIiHRWd4LVKiDDGNO/p4oRcZIrxjBx\n8JHhQC1tIyIindWdYPV73/d/0UO1iDgucD6rLcWV1NQ3OliNiIhEmi4HK2vtW3jXr7vOGPOUMWZy\nz5Ul4ozANwM9FtbtqXCwGhERiTRdniDUGLPN98tG4DvAd4wxh/HOFO5u52vWWjuqq/cUCbXJ+c0b\n2NfsLuOEEVkOVSMiIpGmOzOvD29jX7Lv0x6tbCthbUhWEv2S4yitaQA0UaiIiHROd4LVmT1WhUiY\nMMYwKT+T9zYfALxPrERERILV5WBlrV3Sk4WIhIvJ+Rn+YLWjpIaymnoyk+MdrkpERCKBlrQRaaF1\nn5WGA0VEJDg9FqyM1zhjzKm+zzhjjOmp64v0lsIhGc22NRwoIiLB6nawMsaMNsY8AZQD64F3fZ/1\nQLkx5q/GmNHdvY9IbxmQlsjAjET/thrYRUQkWN0KVsaY84FPgcuBVMC0+KQCVwCfGmPO616pIr0n\ncEFmPbESEZFgdTlYGWNGAc8DKcA24FpgDJDk+4wBrgO2+s75u+87ImEvcKLQ/RV17K+odbAaERGJ\nFN15YvVTIBF4Byi01j5ird1qra3zfbZaa/8CTAaWAAnAT7pfskjotWxg14LMIiISjO4Eq5l4J/y8\n1lp7uL2TfMeuxTs0OKsb9xPpNZPyWzawq89KRESOrjvBaiBQbq394mgnWms3A2W+74iEvYykOEbk\npPi3V6vPSkREgtCdYFUDJBtj4o52ojEmHm+fVbtPtkTCTWAD+9o95VirFZlERKRj3QlWa4E44Mog\nzr3Sd+6abtxPpFcFNrCX1TSw81CNg9WIiEgk6E6wegpv39TvjTFXtzUZqDEm0RhzM/B7vP1Yf+vG\n/UR61eQWfVaaz0pERI6mO4swPw58C28T+8PAr4wx7wN78L4tOBQ4EcjGG8AWAk90p1iR3lQwKANX\njMHt8Q4BrtlVxvmTBzlclYiIhLPuLMJsjTEXAPcDV+NtTP8W3idT4A1TAB68wetWqyYViSBJ8S7G\nDEhlY1ElACu2H8Jai1ZqEhGR9nRr5nVr7WFr7XXAKODHwNN4n0wt9P36x8BIa+31HU3JIBKujh1y\npM9q9e5y7nr9czWxi4hIu7ozFOhnrd0BPNAT1xIJJ5ecMJS/r9yFbzSQxz/8krTEWH408xhnCxMR\nkbDUnSVtVhljPjHGjOzJgkTCybFDMrnnG4XN9s1btIVH3tvmUEUiIhLOujMUOAEYY63VvzAS1S6e\nOoRfzpnQbN//vvk5z32806GKREQkXHUnWO3hSIO6SFSbe/IIbpvVfPjv5y+v5dXVex2qSEREwlF3\ngtUCvDOvn9hTxYiEsxvOHM21px8Z+bYWfvzCZ/x7w34HqxIRkXDSnWD1P0AJ8GdjTE4P1SMStowx\n3D57HN89cah/X6PHcv2zq1j6xUEHKxMRkXDRnbcCRwP/D/gdsMkY8ySwDDgAuNv7krX2vW7cU8RR\nxhju+vpEaurdvPzpHgDqGz1c/eRKnr76RL4ytJ/DFYqIiJNMV+fkMcZ4aD4ZaDAXstbaHpniQZwz\ndepUu3LlSqfLcFSj28MPn1nF2wHDgOmJsbxw7TTGD0x3sDIREQkFY8wn1tqpRzuvO0OBOwM+O1ps\nt/fZ1Y37iYSNWFcMf7h0CqeMPjIKXlHbyOWPfcS2A1UOViYiIk7q8hMr6bv0xOqImvpGLn/sYz7Z\nUerfNygjkb9fN438fskOViYiIj2pN55YifR5yfGxPD73eCYEDP/tLa/lskc/oriy1sHKRETECd2Z\neb3UGFOimdelr8tIiuPJ75/AyP4p/n3bS2q44rGPKaupd7AyERHpbd15YhUPuDTzugjkpCbwzNUn\nMjgzyb9vY1ElV/51BVV1jQ5WJiIivam7zevxPVWISKQbmJHEsz84kQFpCf59q3eVcfXfVlDb0O4M\nJCIiEkW6E6xeBRKMMTN7qhiRSDcsO4Wnrz6RzOQ4/77l2w5x/TOrqG/0OFiZiIj0hu4Eq18D24FH\njDHje6Yckch3TG4aT37vBFITjkzZtnhjMT/++2e4PXoLV0QkmnVnss6vA38C/gv41BjzFsHNvP5k\nN+4pEhEK8zN57MqpXPH4x9T5nlS9vmYfKfGx3P2NSRij9ctFRKJRT8y83vQvRFAXsta6unRDCRua\nxyp4724q5gdPrqTBfeR/Ht87eQS/OG+8wpWISAQJdh6r7jyxeo8gw5RIX3XG2AHMu2QKNz67iqZR\nwMc//JK0xFh+NPMYZ4sTEZEe1+VgZa09owfrEIlaX5s0kHu+UchPXlzj3zdv0RbSEmO5+lRNAyci\nEk0087pIL7h46hB+OWdCs33/88bnPPfxTocqEhGRUAg6WBljbjbGfL+dY6nGmPS2jgWcc78x5rHO\nFigSLeaePILbZjUf/vv5y2t5dfVehyoSEZGe1pknVg8Ad7ZzbAtw6CjfvwSY24n7iUSdG84czbWn\nHxn+sxZ+/MJn/HvDfgerEhGRntLZocCOXmPSK04iR2GM4fbZ4/juiUP9+xo9luufXcXSLw46WJmI\niPQE9ViJ9DJjDHd9fSIXThns31ff6OHqJ1eyamepg5WJiEh3KViJOCAmxnDfNwuZOSHXv6+m3s3c\nxz/m830VDlYmIiLdoWAl4pBYVwx/uHQKp4zO8e+rqG3k8sc+YtuBKgcrExGRrlKwEnFQYpyLv1xx\nHMcN6+ffd7Cqnsse/Yg9ZYcdrExERLpCwUrEYcnxsTw+93gmDDwyY8ne8lq++8hyiitrHaxMREQ6\nS8FKJAxkJMXx5PdPYGT/FP++7SU1XPHYx5TV1DtYmYiIdEZnl7TJMsYsbms/QDvHmp0jIm3LSU3g\nmatP5Jt/WuYfBtxYVMmVf13BM1efSGpCd5b2FBGR3tDZv6njgTM6ON7RMdCizdLSW7dD0Vqnqwgb\nA4FF2W7W11XQ4PZ4d+6Hnb+LZVxeOjFG08WJiAQlbxKcc3ev37YzwepvIatC+q6itbDjA6erCCuJ\nwHHQfKC+HtCygiIiYS/oYGWtvSqUhYiIiIhEOjVtiLPyJjldQVirqG3g86IKbMAgev/UBEb2T8Fo\nFSkRkfY59O+LgpU4y4Hx70iSDhzeVMwPnlxJg9uXrg7B9yeM4I5zx2PUcyUiElY03YJImDtj7ADm\nXTKFmIAM9dgHXzJv0RbnihIRkTYpWIlEgK9NGsg93yhstu+Bf2/h0fe3OVSRiIi0RcFKJEJcPHUI\nv5wzodm+/3njc577WK8LioiECwUrkQgy9+QR3DbrmGb7fv7yWl5dvdehikREJJCClUiEueHM0Vx7\n+kj/trXw4xc+Y9Hn+x2sSkREQMFKJOIYY7h99ji+e+JQ/75Gj+WHz6zi+Y93Yq0WOBARcYqClUgE\nMsZw19cncuGUwf599Y0ebn9pLTc//xmVtQ0OVici0ncpWIlEqJgYw33fLGR2QV6z/a+t3st5f/iA\nNbvLHKpMRKTvUrASiWCxrhge+u5X+PHMY5rNc7WjpIZv/Gkpj33wpYYGRUR6kYKVSIRzxRhunjGG\n56+ZRl56on9/g9ty1+sbuPpvKymtrnewQhGRvkPBSiRKnDAii7duOZUZ4wY0279oYzHnzHufj7aV\nOFSZiEjfoWAlEkX6pcTz6JVT+cV5E4hzHRkbLKqo5dJHljPv31twezQ0KCISKgpWIlHGGMP3TxnB\nP384nWHZyf79Hgv3/3szlz36Efsrah2sUEQkeilYiUSpwvxMXr/pFM6fPKjZ/mXbSvjavPd5d1Ox\nQ5WJiEQvBSuRKJaWGMe8S47lnm9MIjHuyP/cS6rrmfvXFfzmzc9pcHscrFBEJLooWIlEOWMM3z5+\nKK/eeArH5KY2O/bwe9u4+M/L2HWoxqHqRESii4KVSB9xTG4ar9xwCpeeMLTZ/s92lfG137/Pm2v3\nOVSZiEj0ULAS6UOS4l385qJJ/OHSKaQlxPr3V9Y2cv0zq/h/L6+ltsHtYIUiIpFNwUqkD5ozeRBv\n3HwqhfkZzfY/89FOLnjoQ74ornSoMhGRyKZgJdJHDc1O5sXrpnP1KSOa7d9YVMmcP3zI31fu0nI4\nIiKdpGAl0ofFx8Zwx3kTeHzuVPolx/n3H25w89MX1/CjFz6jqq7RwQpFRCKLgpWI8NVxubx1y2mc\nOCKr2f5/fbaXOX/4gHV7yh2qTEQksihYiQgAeRmJPPuDk7hlxhhijqyGw5cHq7noj0t54sMvNTQo\nInIUClYi4ueKMfxo5jE8c/VJ5KYn+PfXuz388rUNXPPUJ5TV1DtYoYhIeFOwEpFWpo3K5s2bT+XM\nsf2b7X97w36+Nu99Vmw/5FBlIiLhTcFKRNqUnZrAY1cez//72nhiA8YG95bXcslflvPg4i24PRoa\nFBEJpGAlIu2KiTH84LSRvPjD6QzJSvLvd3ssv124mSse/4jiyloHKxQRCS8KViJyVMcOyeSNm0/l\n3EkDm+3/8IsSvjbvfd7bfMChykREwouClYgEJT0xjge/M4VfXziJhNgjf3UcrKrnisc/5p75G2lw\nexysUETEeQpWIhI0YwzfOXEor9x4MqMHpDY79qd3t/Lth5exu7TGoepERJynYCUinTYuL51XbzyZ\nb08d0mz/qp1lfG3e+8xft8+hykREnKVgJSJdkhwfyz3fLGTeJceSEu/y76+obeS6p1fxX6+so7bB\n7WCFIiK9T8FKRLrl68cO5o2bT2Xi4PRm+59ctoML/7iUrQeqHKpMRKT3KViJSLcNz0nhnz+czlUn\nD2+2//N9Fcz5wwf885PdzhQmItLLFKxEpEckxLr47zkFPHLFVDKT4/z7a+rd3PqP1fz4759RXtPg\nYIUiIqFntKiqdNbUqVPtypUrnS5DwtjessPc8vynrNhe2mx/nMswfVQOsyfmcdb4XPqnJbRzBRGR\n8GKM+cRaO/Wo5ylYSWcpWEkwGt0e5i3awoPvfEFbf80YA8cPy2JWQS5nF+QxJCu594sUEQmSgpWE\njIKVdMaHXxzkx3//jP0VdR2eN3FwOmdPyGP2xDxGD0jFGNPh+SIivUnBSkJGwUo6q7bBzbubipm/\nrohFG4uprG3s8PyROSmcPTGP2QV5FOZnKGSJiOMUrCRkFKykO+obPSzbVsL8dUW8vaGIg1X1HZ4/\nMCORswvyOLsgj+OH9yPWpXduRKT3KVhJyChYSU9xeyyrdpYyf10RC9YXsbv0cIfn90uOY+YEb0/W\nyaNzSIxzdXi+iEhPUbCSkFGwklCw1rJ+bwUL1ntD1ub9HU8smhLv4sxxAzi7II8zxw0gNSG2lyoV\nkb5IwUpCRsFKesPWA1W+kLWf1bvKOjw3PjaGU0bnMLsgj7Mm5JKVEt9LVYpIX6FgJSGjYCW9bV/5\nYRau38/8dUV89GUJng7+2ooxcMKILGYX5DGrII9BmUm9V6iIRC0FKwkZBStx0qHqev79+X4WrCvi\n/S0HqXd7Ojx/cn6G/w3Dkf1Te6lKEYk2ClYSMgpWEi6q6hr90zi8s7GY6np3h+ePGZDK7IneNwwL\nBqVrGgcRCZqClYSMgpWEo9oGN0u3HvRN47Cf0qOsSzg4M8kfso4b1g9XjEKWiLRPwUpCRsFKwl2j\n28OK7aX+Nwz3ldd2eH5OajwzJ+Qyc0IuJ4zI1huGItKKgpWEjIKVRBJrLWt2lzN/fREL1hWx7WB1\nh+e7YgyF+RlMH5XNtJE5HDesH0nxmi9LpK9TsJKQUbCSSGWt5Yti7zQO89cXsW5PxVG/E++K4dih\nmUwbmc20UdlMGZpJQqyClkhfo2AlIaNgJdFi16EaFm7wvmG4YschgvnrMCE2hqnD+/mCVg6F+RnE\naZkdkainYCUho2Al0ehQdT3Lt5WwdOtBlm0tYeuBjocMmyTHuzh+eJZ36HBUNgWDMtQILxKFFKwk\nZBSspC8orqhl2bYSlm0tYdm2EnaU1AT1vbTEWE4c4Q1Z00ZmMy4vjRgFLZGIp2AlIaNgJX3RnrLD\n3pC1tYRlWw+y9yhvGjbplxzHSb7+rOmjshnVP1XzZ4lEIAUrCRkFK+nrrLXsPFTjf5q1dGsJByrr\ngvpu/7QEThqZ7XvrMJth2ckKWiIRQMFKQkbBSqQ5ay1bD1T7hg4PsnzbIQ5V1wf13YEZif5hw2mj\nssnvlxziakWkKxSsJGQUrEQ65vFYNu2v9D/RWr6thMraxqC+OzQr2R+ypo3KJjc9McTVikgwFKwk\nZBSsRDrH7bFs2FvBsm0HWbq1hBVfHjrquoZNRvZP8Qetk0Zmk5OaEOJqRaQtClYSMgpWIt3T4Paw\ndk+5vxl+5Y5D1DZ4gvpuYX4GsybkcnZBHqMHqBFepLcoWEnIKFiJ9Ky6Rjerd5X759D6dGcZ9e6j\nB60ROSnMKshl1oQ8pgzJ1LQOIiGkYCUho2AlElq1DW4+2VHq79FavauMRk/Hf1f3T0tgpu9J1rSR\n2cTHajZ4kZ6kYCUho2Al0ruq6xpZsf0Q7246wNsb9rOn7HCH56clxHLmuAHMKsjljLEDSE2I7aVK\nRaKXgpWEjIKViHOstazfW8HC9UUs3LCfjUWVHZ4f74rh5NHZnF2Qx4zxufRPU/O7SFcoWEnIKFiJ\nhI/tB6t5e8N+Fm4oYuWO0g4XkjYGpg7rx6wJecwqyGVYdkrvFSoS4RSsJGQUrETC04HKOhZ9vp8F\n64v48IuSozbAj8tLY9aEXGYV5FEwKF1vGIp0QMFKQkbBSiT8VdU18u6mYhau3887G4uprOt4gtLB\nmUn+NwyPH96PWJea30UCKVhJyChYiUSW+kYPy7aVsGB9EW9v2H/UdQ37JccxY7z3DcNTx+SQGOfq\npUpFwpeClYSMgpVI5PJ4LJ/uKmPhhiIWrt/PlwerOzw/Kc7F6cf0Z1ZBLl8dN4DM5PheqlQkvChY\nScgoWIlEB2stXxRXsXCDty9rze7yDs93xRhOGpnlb34fmJHUS5WKOE/BSkJGwUokOu0tO+x/w3D5\ntkO4jzIpaWF+BmcX5DFrQq6W15Gop2AlIaNgJRL9ymrqWbzR2/y+ZPMBDjd0vGj0yJwUZhbkMrsg\nj2OHZCpkSdRRsJKQUbAS6VsO17v54IuDLFhfxKLP91Na09Dh+ePy0rhy+nAuOHYwSfFqfJfooGAl\nIaNgJdJ3Nbo9rNhe6m9+72h5nYykOC45fgiXnTSMIVnJvVilSM9TsJKQUbASEQh+eZ0YAzPG5zJ3\n+nCmj8rWMKFEJAUrCRkFKxFpy9YDVTy9fAcvrtzd7oSkYwakcsX04Vw0ZTApWhxaIoiClYSMgpWI\ndKSqrpGXV+3miaXb2Xqg7Xmy0hJjufi4IVwxbRjDc7RmoYQ/BSsJGQUrEQmGtZYPvjjI35ZuZ9HG\n4jYXiDYGzhw7gCunD+fU0TnExGiYUMKTgpWEjIKViHTWzpIanlq+nRdW7KKitu1hwpE5KVwxbRjf\nOC6ftMS4Xq5QpGMKVhIyClYi0lU19Y3869O9PLH0Szbvr2rznNSEWL7xlcFcMX04o/qn9nKFIm1T\nsJKQUbASke6y1rJsWwl/W7qdtzfsp71J3k87pj9zpw/jjGMGaJhQHKVgJSGjYCUiPWl3aQ1PL9/J\n8yt2UtbO5KPDspO5/KRhXDx1CBlJGiaU3qdgJSGjYCUioVDb4ObVz/byxNLtbNhX0eY5yfEuLpwy\nmLnThzMmN62XK5S+TMFKQkbBSkRCyVrLyh2lPPHhduavL2p3Mejpo7KZO304M8bn4tIwoYSYgpWE\njIKViPSWfeWHeWb5Tp77eCcl1fVtnpPfL4nLTxrGt48fQmZyfC9XKH2FgpWEjIKViPS22gY3b6zZ\nx9+WbWfN7vI2z0mMi+GCYwdz5fThjB+Y3rsFStRTsJKQUbASEadYa/l0Vxl/W7qdN9bso7GdYcIT\nRmQxd/pwZk3IJdYV08tVSjRSsJKQUbASkXBQXFHLMx/t5NmPd3Kgsq7NcwZmJHLZScO45PghZKcm\n9HKFEk0UrCRkFKxEJJzUN3p4a90+nli6nU93lrV5TnxsDOdPHsTc6cOZODijlyuUaKBgJSGjYCUi\n4WrN7jKeWLqd11fvo97tafOcE0dk8dPZYzluWFYvVyeRTMFKQkbBSkTC3cGqOp77aCdPf7SD/RVt\nDxOeNT6Xn84eyzGaD0uCoGAlIaNgJSKRosHtYcH6Iv62dDsrtpe2Om4MXDQlnx/NHEN+v2QHKpRI\noWAlIaNgJSKRaN2ech5c/AXz1xe1OhbviuHyacO44czRZKVoLixpTcFKQkbBSkQi2ac7S7ln/kaW\nbzvU6lhaQizXnDaS750ygpSEWAeqk3ClYCUho2AlIpHOWsuSzQe4Z/4mPm9jXcKc1ARunjGaS44f\nSnys5sESBSsJIQUrEYkWHo/ltTV7+d3Czew8VNPq+NCsZG6ddQxzCgcRo/UI+zQFKwkZBSsRiTb1\njR6eX7GT3y/awsGq1msSFgxK56ezx3HamByMUcDqixSsJGQUrEQkWlXXNfLYB1/yl/e2UVXX2Or4\nSSOzuP2c8Rw7JNOB6sRJClYSMgpWIhLtSqrqeOidrTy9fEebE42eMzGPW2eNZfSAVAeqEycoWEnI\nKFiJSF+xu7SG+9/ewkuf7qblP5euGMPFx+XzH2cdQ15GojMFSq9RsJKQUbASkb5mU1El9y3YyL8/\nL251LCE2hrknD+f600eTkRznQHXSGxSsJGQUrESkr1qx/RD3vLWRlTtaz+KenhjLdWeM4qrpI0iK\ndzlQnYSSgpWEjIKViPRl1loWbyzm3vmb2LS/stXx3PQEbplxDN+amk+sS3NgRQsFKwkZBSsREXB7\nLP/6dA//9/Zm9pQdbnV8ZE4Kt509lnMm5mmKhiigYCUho2AlInJEXaObp5fv5MHFWyitaWh1vDA/\ng/+cPY6TR+c4UJ30FAUrCRkFKxGR1iprG3jkvW08+sGX1NS7Wx0/dUwO/zl7HBMHZzhQnXSXgpWE\njIKViEj7DlTW8eDiLTz78U4a3K3/jT2vcCC3zRrL8JwUB6qTrlKwkpBRsBIRObqdJTX87u1NvPLZ\n3lbHYmMMl5wwhJtnjGFAmubAigQKVhIyClYiIsFbv7ece+dvYsnmA62OJcW5+P4pI7jm9JGkJ2oO\nrHCmYCUho2AlItJ5y7aWcM/8jXy2q6zVsczkOG44YzSXTxtGYpzmwApHClYSMgpWIiJdY61lwfr9\n3LdgI1sPVLc6Pigjkf+YeQzf+Eo+rhhN0RBOFKwkZBSsRES6p9Ht4Z+rdnP/21soqqhtdbxgUDr/\nc8FEpgzt50B10hYFKwkZBSsRkZ5R2+DmyWXbeeidrZQfbj4HljFw6QlD+enZY8lMjnemQPELNlhp\nrn0RERGHJMa5uOa0Ubz30zO5/oxRJMYd+WfZWnj2o53M+N0SXvxkN3oQEhkUrERERByWkRTHT2eP\nY/GtZzC7IK/ZsZLqem77x2q+/fByNhW1XptQwouClYiISJgYlJnEny8/jr/OPZ4hWUnNjn28/RDn\n/v59fvPm51TXNTpUoRyNglUfZYxJNcb8yhjzpjHmgDHGGmNud7ouERGBM8cN4O0fnc7NXx1NvOvI\nP9WNHsvD721j5v8tYf66Ig0PhiEFq74rB/gvYBLwqcO1iIhIC4lxLn48ayxv/cepnDw6u9mxveW1\nXPf0J3zviRXsLKlxqEJpi4JV37UPGGytHQJc43QxIiLStlH9U3n6+yfy+0un0D8todmxdzYdYOb9\nS/jDoi3UNbZe+Fl6n4JVH2WtrbPWtl7ASkREwo4xhvMnD2LRraczd/pwAucOrWv08Lu3N3POA+/z\nwZaDzhUpgIKViIhIxEhPjOOX5xfw6o2ncOyQzGbHth2s5rLHPuKm5z6luI1JR6V3hG2wMsbM9TVU\nd/SpcrrOJsaYNGPM+caYu4wxbxljDgbUOa4T18kzxswzxmw1xtQaY/YbY14zxswIZf0iIhI5Jg7O\n4KUfTufXF04iI6n54s2vrd7LV3+3hL9++CWNbo9DFfZdsU4XEIQG4FA7x1ovtOScGcDL3bmAMaYQ\nWAw0dSlW4G0yPw841xjzc2vt3S2+Y4Dmg+7tq7fW6n9lIiJRICbG8J0Th3J2QS6/eWsjL36y23+s\nqq6RX722gRc/2a2lcXpZ2D6xCrDUWpvXzmeU08W1UAy8CfyKTjaEG2OSgFfxhqpPgYnW2gygH/A7\nwAC/NsbMavHVk4HDQX5O69LvSkREwlZ2agK/vXgyf792GmNz05odW7+3gov+tJSfvbSWspp6hyrs\nWyLhiVWPMMbEAn8C7rPWbj7KuXnAfcB11tpgn4q9Zq39V8A1hneyxGuBYUAVMMdauwfAWlsB3GaM\nGQVcAPwGWBjwvc3AVUHeY2MnaxIRkQhxwogsXr/5FP764Zc88O8t1NR73xK0Fp77eCcL1hfxs3PG\n8c3j8vEOdkgo9JlgBdwKXA3MNsacZq39sq2TjDE5wL+BArxDcTcEc3FrbXffc/2u7+ezTaGqhfvw\nBjfIIjYAABgVSURBVKuvGGPGWms3+e5bDDzRzXuLiEgUiHPFcM1pozivcBB3vraB+euL/McOVdfz\nkxfX8PeVu/ifCyYxNi+tgytJV0XCUGBPeRD4EMgHFhlj8lueYIzJwPs0qADYhHdIL+SMMWnAcb7N\nBe2cthwo9/1ajewiItKuwKVxhmYlNzu2YnspX/v9+/xaS+OERCQEqwJjzHpjzGFjTKUxZp0x5n5j\nzIjOXMQ3pHcusAoYgTdc5TYdN8akAvOBKcCXwAzf06DeMB5vDxXA+rZO8DWdb/JtTuiJmxpjbjTG\n3AHc6Nt1pjHmDt8noyfuISIizjlz3AAW/ui0VkvjuD2Wv7y3jbP+bwnz1+3T0jg9KBKCVQ7e4FED\nJOJ9mvQfwHpjzHc6cyFrbTkwC294OQZvuMr2NY6/DpwE7MEbqtoajguVgQG/7mjSzqZjAzs4pzNu\nA+7CO0wK3v82d/k+eoVERCQKNC2NM/8/TuWU0TnNju0rr+W6p1dpaZweFM7Bai/w38BEINFamw2k\n4n3qtAFIAv5mjOnUm27W2hLgLOALvCFtId5pEk7H+1bfjPb6r0IoJeDXhzs4r+lPfWpP3NRaO9xa\na9r5bG95vjFmjjHmL+Xl5W1cTUREwtnI/qk89f0TePA7UxjQztI4v9fSON0WtsHKWrvQWnuntXa9\ntbbet6/OWvsmMB1vMIoF7u7oOu1cuwhvn9JO4CvA2UApMLOpKVxas9a+Zq29JiNDo4QiIpHIGMN5\nhd6lcb538ohWS+P839ubma2lcbolbINVR3xDer/2bZ7ke5Ovs/bgnS+qyUZgS3dr66LAKR2SOjiv\nqQMxbGacFxGRyJOWGMd/zZnAazedwpShzZfG+dK3NM6Nz65iv5bG6bSIDFY+H/l+GrzN6EHzzVb+\nGPB1vKGmGJgGvGSMie/JIoMU2Fc1qIPzmo7tC2EtIiLSRxQMyuCf103nNxe1Xhrn9TX7mPG7JTz+\ngZbG6YxIDlbd8RBwJd5+pjl4e64OAbOBF3yTifamjUDTKxkFbZ1gjIkBxvo2N/RGUSIiEv1iYgyX\nnjCUxbeezremNp+JqKru/7d33+FyVeUex7+/JKQnBEIgyCUk1CAdQhElFKUFQQQboBhU2oNKAAti\nvXAFhashSLlUERALFtRIEZBexISilCCgAUQINSSkkPbeP9YazpzDnJazz5Rzfp/nmWfP3mvtPe/Z\nz5mZd9Zee61lnDr9MQ44924eePb1GkXYWBo5sdqx7Pnsju4k6X+BY4ElwEERcWtE/J10R9w80iCc\nV+ZEpioiYj4wI6/u2Uq1HYFS56Zbuj0oMzPrVUYOHcCZH9mKa45559Q4j70wj4M9NU6H1GVipXbG\n2pc0HDg5r94fES938LinkoYWWAZ8PCJuKJVFxExgEunS4CeAS9qLo2BX5+VhkioNp/ClvJzpDvZm\nZtZdth+bpsb5+qRNGdy/79vbS1Pj7PGD27lmxnMe+6oVdZlYAetJukfSpyWtU9ooqb+kfUgjqG8M\nrAC+1pEDSjoa+Gbe51Pl8/qVRMTdwAHAYtL8e6e3rNPOa6xRetB8HKgR5WWttIZdCDwDDAOmS3p3\nPuYwSWcCB+V6p3QmJjMzs85apW8fjpy4PrectCuTthjdrKw0Nc6nLr2f517z2FctqR4zzjyBcflY\nUotILUmrAqXedQtJkyRf2cFjjgHuAr4dET9up+4k0vx7e0XEQ52Iu6Mnc1wr40RtRbrMNzJvmkca\ns6oPqQ/WKRHR6eElijZhwoSYMWNG+xXNzKxHuPWJl/j27x7l2RaJ1JD+fTl53/EctuN69OnTsyd2\nljQzIia0W69OE6tBpAmT3wdsBYwChpOSqydJyccFEfFMJ487LPdnKrRu2T5dSqzyMUaTWuE+CKxD\nSq7uB6ZGRF30rXJiZWbW+yxeupzzb3uaC257iqXLm3/d7bT+6nz/4C1Zb+SQVvZufA2dWFl9c2Jl\nZtZ7/WPOfL58zcM8/O/ms3AMWqUvX957EybvPLZHtl51NLGq1z5WZmZmVoc2XmsYvz52Z07edzz9\n+zWlEYuWLufU6Y/xsQvv5emXe+841k6szMzMrFP69e3DMbtuwHVf3IVtW4zcPuOZ15k07U4uvP1p\nlq/ofVfFnFiZmZnZStlwzaFcc8zOfGO/TRm4SlNK8dayFZxx/SwOuuAenpzTqe7KDc+JlZmZma20\nvn3E53ZZnxuOn8gO41ZvVvbwc3PZ75y7OO/Wp1jaS6bFcWJlZmZmXTZ2jSH8/MidOPVDmzUbWHTJ\n8hWcdeMTfPj8u3n8hXk1jLA6nFiZmZlZIfr0EYe/Zyw3TpnIezcc2azskefnsf+P7mLqTf9gybKe\n23rlxMrMzMwKte7qg7nqszty+oe3YOiAfm9vX7YimHbLkxxw7l38vcVwDT2FEyszMzMrnCQO3XEM\nfzphIrtuPKpZ2awX53Pg+Xdz5g2zeGvZ8hpF2D2cWJmZmVm3edeIQVx+xPac9ZEtGT6wqfVq+Yrg\n/NueZr9z7uLBZ1+vYYTFcmJlZmZm3UoSH52wLjeduCsf2HTNZmVPvfQmB19wD6df9ziLlzZ+65UT\nKzMzM6uKtYYP5OLDJzDtE1szYvAqb29fEXDRHf9k32l38tfZr9Uwwq5zYmVmZmZVI4kPbb0ON52w\nK/tuPrpZ2b9eWcDHLryX7/z+URYuWVajCLvGiZWZmZlV3ahhA7jgk9tx3qHbMnJI/7e3R8Dl98xm\nn7Pv5N6nX61hhCvHiZWZmZnVzH5brs2fTpjIAVu9q9n2Z19byCEX38c3rv07b77VOK1XTqzMzMys\npkYOHcA5h2zDRZ/ajlHDBjQru+q+Z9l76h3c8Y+XaxRd5zixMjMzs7qw12ajuemEiRy07TrNtj8/\ndxGHX3Y/X/3V35i3eGmNousYJ1ZmZmZWN0YM7s8PP7Y1l02ewOjhA5uV/WLGc+z1wzu4ddZLNYqu\nfU6szMzMrO7sMX4t/nTiRD4+Yd1m21+ct5gjLv8rJ/7yIeYuXFKj6FrnxMrMzMzq0vCBq/D9j2zJ\nlZ/dgXVGDGpW9psHnmfPqXdw46Mv1ii6ypxYmZmZWV3bZaNR3HjCRD6505hm21+e/xZHXzmTL/zs\nQV5bUB+tV06szMzMrO4NHdCP/zlwC64+ckfGrD64WdkfHv4Pe/7wdv74txdqFF0TJ1ZmZmbWMHbe\nYA1umLILR7x3LFLT9lcXLOG4qx/g2Ktm8vL8t2oWnxMrMzMzayiD+/fj2/tvxi+Pfg/j1hjSrOz6\nR15kz6m3c+2DzxMRVY/NiZWZmZk1pO3Hrs71x+/CURPXp09Z69XchUuZ8ouHOG3641WPyYmVmZmZ\nNayBq/TllEmb8utjd2ajNYe+vV1K0+VUmxMrMzMza3jbjFmN6V98H8ftvgF9+4jPvncc2623WtXj\n6Ff1VzQzMzPrBgP69eXLe49n383XZoNRQ9vfoRs4sTIzM7MeZfN1Vq3Za/tSoJmZmVlBnFiZmZmZ\nFcSJlZmZmVlBnFiZmZmZFcSJlZmZmVlBnFiZmZmZFcSJlZmZmVlBnFiZmZmZFcSJlZmZmVlBnFiZ\nmZmZFcSJlZmZmVlBnFiZmZmZFcSJlZmZmVlBnFiZmZmZFcSJlZmZmVlBnFiZmZmZFUQRUesYrMFI\nehl4puDDrgG8UvAxexufw67zOew6n8Ou8znsuu44h+tFxKj2KjmxsrogaUZETKh1HI3M57DrfA67\nzuew63wOu66W59CXAs3MzMwK4sTKzMzMrCBOrKxeXFTrAHoAn8Ou8znsOp/DrvM57LqanUP3sTIz\nMzMriFuszMzMzArixMrMzMysIE6szMzMzArixMpqRtJoSdMkPS1psaQ5kv4g6f21jq3eSRojaUo+\nX89KekvSfEkPS/qepLVrHWOjkTRU0nOSIj8m1zqmRiJpE0k/kvSEpAWS3pD0uKTLJO1a6/jqmaQ+\nko6QdLOklyUtlTRX0l8kfV3SsFrHWGuShkk6QNJpkq6X9ErZe3V8B/bvL+krkh6S9GY+v/dKOkqS\nCo3VndetFiRtCfwZGJk3zQOGkpL9AE6JiO/VKLy6Jmld0sj35R8G84AhQN+8/jpwcETcWuXwGpak\ns4HjyzYdERGX1yichiLpi8BZQP+86U2gHzAwr18aEZ+rRWz1TtJg4A/AHmWb3wCG0/QefwbYIyL+\nWeXw6oakA4HftlK8aUTMamPf4aTvm+3ypoWk/8/S/+t04MMRsayIWN1iZVUnaRDwe1JS9SCweUSs\nCqwG/ID0YXK6pL1qF2VdKyVPfwQ+Cqyez99gYBLwL9K5vFbS6NqE2FgkbQt8HvhLrWNpNJKOBqaR\nvqi+T5r2Y1hEDALWBg4H7qlhiPXum6SkKoCvASMiYgQpKT0EmAusB1xSswjrx0vAdcB/A0d1Yr+L\nSUnVa8D+pB/xg4HJwGLgg/mYhXCLlVWdpCnAVNKv2vER8XyL8t8CBwIPRMR2FQ7Rq0laFRgbEQ+3\nUj6elLAOBL4TEYV9YPREkvqQEqptgO2BB3KRW6zaIWks8CjpS+qoiLi4pgE1IEnPAGOAyyLisxXK\nJwM/zqurR8TrVQyvbkjqGxHLy9bHkn5EQhstVpK2oek9/aGI+H2L8uOBs4FFpM/Vl7oaq1usrBYO\ny8urWyZV2Vl5ua2kTaoUU8OIiDdaS6py+SzgvrzqxLR9XwAmABdExIO1DqbBHE9Kqv7ipGqlrZWX\nrf3vzSx7PribY6lb5UlVJx2al0+0TKqyi0iXXgcBB63kazTjxMqqKnfCLH3Z39hKtftI/+gA7si+\ncl7Ny75t1urlJK0DnAbMAb5R43AaUelL62c1jaKxzc7LbVopL31ezmnlh6i1bfe8/FOlwohYBNyZ\nV/eoVKeznFhZtW1KU4fMRytViIgVwBN59d3VCKonkdQPeG9efaSWsTSAHwHDgC9FxBvtVbYmkjYA\n1syrD0raKd+l+qqkRZJmSTpL0pptHccotfQdIenkfKm/dBfbx0ndJgL4Uq0CbFT5br/SHYMVv2+y\nx/KykO8bJ1ZWbeXDAPynjXqlMg8b0HnHAaOBFcBPahxL3ZK0P/Bh4LaIuKrW8TSgjcqe7wbcReoE\nvAopEdiElAw8JGmzqkfXOM4GziP94DwDmCtpLqnPz8+BWcAB/h9dKcNJd0tDFb9vnFhZtQ0pe76o\njXoL83JoN8bS4+RhLM7Iq+dGxGNt1e+tJA0BzgWWkhJR67wRZc+/DfwD2CkihpPet5NId3GtDfw6\nt6RaC7nv0BTgJKB0u/+qNH0/DwNG1SC0nqAm3zdOrMx6iDwo6LWkTpgzga/WNqK6dirpTqypTj5X\nWvn3R5DGAfoLpMv5EXE98JlcvgkFdQzuafKQKHeThpr5KbAV6Qt+I9LwC+sDl0k6o9WDWF1xYmXV\ntqDs+aA26pXufnmzG2PpMSStTuqcOQ54EtgvIhbXNqr6JGlr0t1sz5ESLFs55e/NGyLiiZYVIuKP\npJYs8I0orbkC2IE0iOrkiPhbRCyIiKfyIMlH53pf8SXVTqvJ940TK6u28uvc72qjXqnshW6MpUfI\nnV1vBDYHngU+EBFzahtVXZtGulvy66T+rUPLH2X1BuRtvfYW93aUv5ffkVRVKFu3G2NpSJLeDeyZ\nV6dWqhMRV5Lu8u1DGtzSOm4eTclV1b5vnFhZtc0iXTYAqPjrKw/YWBq/ypdp2pD7Cl1HGofpRVJS\n9Wxto6p76+XlFcD8Co+S/8vr/h+s7DHSDRId5dGo32nTsuf/arUWlKayGdt9ofQ8kUZAfzyvttXa\nV7obsJD3uhMrq6qImA/MyKt7tlJtR1LnTYBbuj2oBpWnBvoDsDPpF+0HIuLJ2kZlvUVELATuzatt\nDeRbKpvdrQE1pvLEdEwb9Uo/Bua3UccqK82XWvH7RtJAYJe8Wsj3jRMrq4Wr8/Kw3OG6pdJ4LTMr\n9duwNMYN8BvS4Hdzgb0ioq1xWiyLiLERodYeZVWPyNvG1irWBnBFXu5TaZYESfsBG+fV66oWVeMo\nn0HhyEoV8rAgpbHAPJdl55UGrx0v6YMVyo8k/ZBfROuTPHeKEyurhQtJs7UPA6bnfgZIGibpTJru\nHjqlRvHVNUl9ScnpPqRfsPtGxANt72XWLS4jXT7pC/xG0g6QLudL2ge4NNe7DydW7xAR/6RpRPAp\nks4oDaia+/dNBi7P5bNJk9f3WpLWKD1IE82XjCgvy91JAMjTVP0yr14uaVI+Vl9Jh5MmDod0h3CX\n5wkET8JsNSJpK1Kz68i8aR7pFuM+pL4Yp+Q7YqwFSROB2/PqYpqm/6nkuYjYvvuj6jkklT4UPQlz\nB0haH7iNps7p80mJVqnT/2OkFlVPx1JBbrW/heb9reaTfniWzCH9gOrVc1mWvTfbMy4iZpftNxz4\nM03TAy0k/Y8OyOvTScOFLKMAbrGymsiTCG8OnEPqmDmA1E/oj8CeTqraVP6+HUiaxLW1hwcWtG6V\nW122AL5LSqL6kX4cPUAah2kHJ1Wti4gXSF/4U4A7gNdISek80jk8DdiitydVXRER80h9UU8mXX4N\n4C1SS+rRpJHtC0mqwC1WZmZmZoVxi5WZmZlZQZxYmZmZmRXEiZWZmZlZQZxYmZmZmRXEiZWZmZlZ\nQZxYmZmZmRXEiZWZmZlZQZxYmZmZmRXEiZWZmZlZQfrVOgAzs55EUj/gk8AngK1I82EuAF4kTd90\nJ/DniLi/bJ+tgQOB2Z6f0KyxeUobM7OCSBoFXAdMKNu8mDQv2XBAedsbETGibL/JwI+B2yNit6oE\na2bdwpcCzcyKcxUpqZoPfAVYOyIG5SRqVWBP4Hxgbu1CNLPu5EuBZmYFkDQe2CuvfiYiflVeHhHz\ngZuBmyWdVO34zKw63GJlZlaMLcqeT2+rYkQsLj2XFKTLgAC7SooWj91a7i/pfZJ+Lunfkt6S9Kqk\nmyUdIkkV6u+WjzU7r+8v6VZJr0t6U9K9kg5dib/ZzFpwi5WZWfHWAZ7uYN05wCBSH6ylwGstypeU\nr0j6PukyY8k8YDXg/flxgKTDImJFpReTNAWYCgTwRn7tnYCdJO0cEZ/vYNxmVoFbrMzMijGz7Pl5\nuSN7uyJiNHB8Xr0nIka3eNxTqivpeFJSNQc4ChgREasCQ0h3Ib6Yl19t5eVGAWcCV5D6f60GrAH8\nIJcf55Yrs67xXYFmZgWR9BPg8Ly6hDS0wn3AX0lJ08ut7DeZdu4KlDQCeI50pWGniHi4Qp33AHeT\nOsePjogleftuwK252k3A3tHiw1/S5cCngaeAjVuWm1nHuMXKzKw4RwI/JCVV/UmX5r4OXAu8JOl+\nSYdV6gfVAQcDQ4GbKyVVABFxL/Av0qXB7Vo5zhmtJE3fzcsNSeNvmdlKcGJlZlaQiFgSEScB6wLH\nAD8DniT1ZwLYnjQkwy8kdfbzd+e83EPSi6098mtTtiy3lNSiVSn2J4EX8uq2nYzNzDJ3XjczK1hE\nvARcmB9IWgvYH/gWKeH5KCnBmdaJw66dl4Pzoz2V6rxSujzYiufz63Sof5iZvZNbrMzMullEzImI\nS0gtQXPy5s908jClz+tpEaEOPC4vKn4z6zgnVmZmVRIRrwC/y6sbd3L3UkI2pgshrCGpfxvl78rL\nip3szax9TqzMzKprQV6WX5IrjTnVVqf2e/NyN0mDVvK1VwHeU6lA0oY0JVYPrOTxzXo9J1ZmZgWQ\nNE7SBu3UGQwcmFcfKiual5cjaN01pKRsNVJfrbZeZ7U2ir/Wyl2JX8vLJyPioQrlZtYBTqzMzIqx\nGfCEpN9I+pikUmdzJA2RtD9pXKtxeXN5x/VH8/LdknasdPCIeJWm5OdkSRdLevtyoqTBknaXdBFw\nT6VjAAtJQ0BcKmnNvN+IPJp7qc/Xdzr495pZBR4g1MysAJL2Bm5osXkR6ZLfqmXblgPfiojTW+x/\nOzAxr74GzM/PPxER95XV+wZwKk2XDRfk1xhRtm12RIwr22c30gChzwBn0zSlzdwcW+lH9nme0sas\na5xYmZkVJLcg7Q+8D9icNGdgf1KS9E/gDuCSiHi0wr4jSQnTvmX7AeweEbe1qLsF8Hlgd+C/gL6k\nDuePALcAP4uIf5fV342cWEXE2Nx6diKwDanf1d+AcyPip10+CWa9nBMrM7MermViVdtozHo297Ey\nMzMzK4gTKzMzM7OCOLEyMzMzK4gTKzMzM7OCuPO6mZmZWUHcYmVmZmZWECdWZmZmZgVxYmVmZmZW\nECdWZmZmZgVxYmVmZmZWkP8HizDGqTR9rKMAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def line_search(x, d):\n", " return gss(lambda step: log_likelihood(x+step*d, X, y), -1, 1, tol=0.1)\n", "\n", "ws_bfgs = bfgs(init, gradient, hessian, line_search, 10)\n", "error_plot([log_likelihood(w, X, y) for w in ws_bfgs])\n", "plt.plot(range(len(ws_bfgs)), [baseline]*len(ws_bfgs),\n", " label='sklearn', **kwargs)\n", "_ = plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Timing comparison" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1.6104111671447754" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "timeit(lambda: linear_model.LogisticRegression().fit(X, y), number=1)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "6.190078973770142" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "timeit(lambda: bfgs(init, gradient, hessian, line_search, 10), number=1)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1.7400920391082764" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "timeit(lambda: gradient_descent(init, np.ones(200), gradient), number=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Accuracy on the test set" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": true }, "outputs": [], "source": [ "Xtest = test_data.values\n", "ytest = test_labels.values" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def accuracy(weights):\n", " predictions = logistic_model(weights, Xtest)\n", " return np.mean((predictions > 0.5).astype(np.float) == ytest)" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.82636201707511825" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ws_gd = gradient_descent(init, np.ones(500), gradient)\n", "accuracy(ws_gd[-1])" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.83139856274184631" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ws_bfgs = bfgs(init, gradient, hessian, line_search, 20)\n", "accuracy(ws_bfgs[-1])" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8308457711442786" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cls = linear_model.LogisticRegression(C=1, fit_intercept=False)\n", "cls.fit(X, y)\n", "cls.score(Xtest, ytest)\n", "weights = np.reshape(cls.coef_, cls.coef_.shape[1])\n", "accuracy(weights)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that our toy implementation of BFGS is actually fairly competitive. So is gradient descent which reaches good test performance with an optimized constant step size." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## That's all. Thanks." ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.13" } }, "nbformat": 4, "nbformat_minor": 1 }