machinelearning_notebook/references_tips/logistic_regression_demo/4 - Logistic Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Logistic Regression\n",
    "==================="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we generate some data to train to train with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "X, y = make_blobs(centers=2)   # generate dataset consisting of two Gaussian clusters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a look at how large the data is and what the labels look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X.shape: (100, 2)\n",
      "y:  [1 0 0 1 1 0 1 0 0 0 1 0 1 1 1 0 0 1 0 0 0 0 1 1 1 1 1 1 1 1 1 0 0 1 1 0 0\n",
      " 0 1 0 0 1 0 1 0 1 1 0 0 1 1 0 1 1 1 1 0 1 0 1 1 0 1 1 0 0 0 1 0 1 1 0 1 0\n",
      " 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 0 0 0 1 1 0 1 0 1 0 0]\n"
     ]
    }
   ],
   "source": [
    "print \"X.shape:\", X.shape \n",
    "print \"y: \", y "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the data is two-dimensional, we can easily visualize it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0xa65ce6c>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD9CAYAAABZVQdHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVPX+x/HXDMuwCoi5omCogFdUVIQ0E3fc17LytohZ\nLmVmLpXdK7baLW8u7d2s3Cot7Wo3d0MSc8k1Tc0NMQUlN0B2OL8/xkh+aimLg+P7+XjM48E5c+b7\n/Zwjvufwne+cYzIMw0BEROyS2dYFiIhI+VHIi4jYMYW8iIgdU8iLiNgxhbyIiB1TyIuI2LFShXxM\nTAzVqlUjNDS0aF16ejq9e/emTp069OnTh4yMjFIXKSIiJVOqkB88eDDLly8vtu7dd9+lTp06HDhw\nAD8/P957771SFSgiIiVXqpBv06YNPj4+xdZt3ryZIUOGYLFYiImJYdOmTaUqUERESs6xrBvcsmUL\nwcHBAAQHB7N58+bLtjGZTGXdrYjILeF6L1JQ5h+8XmsBhmHoYRhMmjTJ5jVUlIeOhY6FjsWfP0qi\nzEM+PDycvXv3ArB3717Cw8PLugsREblGZR7yERERzJo1i6ysLGbNmkVkZGRZdyEiIteoVCF/3333\n0apVK3755Rdq167Nxx9/zPDhw0lKSiIoKIjjx48zbNiwsqrVLkVFRdm6hApDx+IPOhZ/0LEoHZNR\n0oGe0nRqMpV4fElE5FZVkuzUN15FROyYQl5ExI4p5EVE7JhCXkTEjinkRUTsmEL+GhmGQVpammYF\nichNRSF/DX788UfqVq9ODV9fqnt7s2bNGluXJCJyTTRP/i9kZ2cTWLMm086e5W7gO+Aed3f2HD5M\n1apVbV2eiNxCNE++HCQmJuKWl8fdF5fbAcGOjuzZs8eWZYmIXBOF/F+oWrUqp/LyOHpx+TRwIDeX\nmjVr2rIsEZFropD/C5UrV+blKVO4w82N+zw8aO7uztDHHycoKMjWpYmI/CWNyV+jHTt2sHv3burV\nq6cra4qITZQkOxXy5ej8+fO8+PzzHNi9m8YtWzJx8mRcXFxsXZaI3KQU8hVIXl4ebZo1o+GBA/TI\nyWGeqyvZERF8s3atbn8oIiVSkuws83u8itXWrVu5cPQoH+XkYAJ6ZmVRe9MmEhMTqVu3rq3LE5Fb\nhD54LSeGYRQ7uCasB9ve/4IRkYpFIV9OmjdvjmPNmjzu7Mwy4CEXF/7WtOlVz+KPHTtG/+hoGgcE\nMKhPH1JTU29swSJilxTy5cTZ2ZlVGzbA3//Omy1bUuORR/h61aorjsdnZmbSsVUrGq9ezadHj1L1\n22/p1rYt+fn5TJ44kepeXlR2d6dlw4Y8/sgj7N+/3wZ7JCI3I33weoMYhsFXX33Fnt27aRAUxMCB\nAzGbre+xCQkJPNmtGz+mpVm3BQLc3bl76FA+nzmTqgUFnAAmASeBdzw9Sdi6lfr165eqpj179hA7\ndixnUlPp1Ls34557DgcHh1K1KSLlRx+8VmCjhw1j3bx59MjM5E03N1YtWcJ/5s3j0KFDpKSkcC4/\nn3ys/yBZQEZ+PrPefpsHCgpYDswFOl5sKzs9nY/ef58pb7xR4nqSkpJof8cdPJuRQUPD4MW9e/nt\n1CnemDmz1PsqIhWHzuRvgGPHjhHWoAGHsrPxAjKBBq6uBISEkLhvH44mE7kmE6H5+fTKzmahmxsF\njRrhsG0bcfn5NAHeB37/CtYrwKkRIwhr2ZKDBw7QuEkTBgwYcF1TM2fMmMGu8eP5T04OAMeBRq6u\nnM3MLNN9F5GyozP5crR27VqWfPklHl5ejHjiieu6ds25c+eo4uSEV3Y2AG6AT2EhTrt3k5ibixl4\nxGLhcJMm7AoK4u6WLfGpXJn5Q4dCfj4PAo8B/wZOAa8CrXfvZsMnn5CfmckMYIKvL1+uXEmzZs2u\nqSYHBwdyLnlTyAEczPqIRsTuGDZgo25L7LP5842abm7Gv8B4wsHB8PP1NU6cOHHNr8/Ozjbq1axp\nTDWbjWQw3jWZDG8nJ+NTMIyLjxVgtG/evOg1p0+fNmpXqWK8YDYbK8AIBqMmGFXAqOHtbdQwm40o\nMEaAcRCMT8C4zcPjmutKSUkxalWubEx0cDDmgtHE3d2Y/Pzz131sROTGKUl26tTtGrz0zDN8lpnJ\nOGBGQQHdz5/n41mzrvn1FouF5fHxLG3WjEbu7swNDeXuQYNYabFQiPWD1q+dnQkKDWXhwoXc7uPD\n36pXx6dyZXZ27sw/g4Iw+/tTqXZtvJydeeLcOaoXFrIZmAEEAg8BkYbB+vXri/U9Z/ZsurZqRd+O\nHYmPjy9aX61aNRK2bePMgw+ypGtXHv/3v/nHCy+U/mCJSIWi4ZprkJmVxaW3B6man0/mhQvX1UZg\nYCDfbdlStJyRkUH3n34ieP9+HAH32rV54Z57uKd7d94xDKoCo375hS1nz3LoxAkcHR0Jb9CAN3Jz\naQq8DRQAqUB1oBBIMZnw8PAo6uOjDz9kyujRvJ6ZyVmg/w8/8L/vvqNly5YA+Pv78851vFmJyM1H\nIX8N7hk0iGEffMCbmZkkAe+5uvJtv36Xbbdz504+nzcPB0dHHo6JoV69eldt08PDg7c/+YTxI0eS\ndvYsfQcNYvZHHzHBMHjg4jafAN1TUxnQtSvBjRtjALmAF7AS6w1MIrCO129wccE1KIiOHTsW9fGf\nN9/k/cxM2l9cTs3MZM6HHxaFvIjYP4X8NXjp9deJdXLiwS++wMPDg9lTp9KiRYti22zYsIHenTox\nPDOTbLOZVm+9xbpNmwgJCblim0lJSXRo3Zrx6ekEGQbPPfsshwyDgEu2ycL6D9R79Wp+iI/nMHA/\n8C/gNyDPxYXnpkzhRFISnf39efTRR3Fycip6vclkIu+S9vIAs+bBi9xSNIWyjPRq146+cXEMvrj8\nislE4qBBfDBnzhW3nzZtGj9PmMAHubkAJALNgXxgHNYhmGeBl7CeqRtYz9rdsc6wMQNHTCZOnDtH\npUqVrtjHZ/PnM2HoUF7KzOQM8LK7O6sTEmjSpEmZ7LOI3Fi6x6sNZaSlUeOS5ZqGwYWL32C9EpPJ\nRP4ly79/ESoeeBlr0J8DHv59e6wBfxZ4HKiB9Vu0n3zyyRXbNwyDiMhInn7lFZZGR7O9f39WxMcr\n4EVuMRquKSP9H3yQCfv2USUzk2zgBTc3Xn/ggatu36ZNG57Ly8MfaAg8D2QDiy8+/zywERgEPAms\nB7YAB7Ge5ccANYEDBw7w4qRJ7Nuxg4bNmjH2mWdwcnIi5r77WLZkCd6Ojjj6+vLJwoWsXbuWVatW\nMWDAAAIDA8vpSIhIRaKQLyMjRo0iMzOTB995BwcHByZOnEj/AQOuuv3q1avp6eDAl/n5ZAFPAJ2w\nfpjqeXGb2VjDfiRw2s0Nn6wsql38U80ZsAAJK1dSNymJ3tnZfLVqFf3XraPvoEEc+OYbjmRn4wpM\nzsqiyx13cI+DA06GwR0vvcTqhAQaN25cbsdDRCqGW25M/sSJE8ROmEDy0aO07tSJsc8+i6PjjX+v\ne/GFF0h74QW+LChgJfD7pcZeBl5zdsacm8sUrIH/tIMDz7/5Ju9OnUqPpCTuMQzmAJ95euKcnc2R\nvDwcsX6werubG5369SN47lzGX2zzF+AuIOXi8gxgTfv2/HfNmhu2vyJSehqT/wvnzp2jTfPm+H72\nGUO+/56VU6YwcsgQm9TSr39/PrFYsAC7L64zgN0WCx26dsXN3Z1nsJ7h13Z05MWJE4nu35+3TCbu\nAv4D+GRk4JyXx+/zZRwBi9lMncBAvnVzI/vi+oVA7Uv6rg9sTkgg9+KHviJiv26pM/kFCxbw6ZAh\n/C8jA4B0oIqDA+mZmTg7O9/wehISEnjy4vXh73FwIMVi4ZdKlcg8d465WVl4AY8C9wKHsIb1cGAr\nsAHrcI0X0BXoD3zh7MyP9esTv3UrDw4YwA9r1+Lr6EiKYeCUns5ywAnrOP8JZ2e+Tki4bCqoiFRc\nukDZXzAMg4p0C+3WrVvz4969/PLLL6xcuRJPT0+++/ZbWi5YQLuL20wFngEOA2uBxljP+LsDwcBn\nWGfh9ALuHjCAZTNnYrFY+HzJEvbu3UtGRgYWi4V2LVrQJT+fQmAw8CnY5I1NRG6scgv5gIAAKlWq\nhIODA05OTmzevLm8urpmXbp04Vl3dyZmZRFRUMAMNzce7NvX5mHXoEEDGjRoAMC2jRtJNpuhsBCw\njqO7Yw3ygIvbmy7+HID1G7AuDg60CA/nw3nzito0mUw0bNgQsL651ahZkxpJSdwPfA1k5OXx22+/\nlfu+iYhtlduYvMlkIi4uju3bt1eIgAfw9vbm+61bOXXvvbzfujXtx4/n3avMMy+t7du30ykykiZ1\n6zJ62DCysrKu6XWjxo7lPVdXRgOTgSFAAuACDHN0JAlYhnXoxhvrtekXGAaGYfDrr79esU2TyURh\nQQGhQBzW69KPMQz+9/XXpdtJEanwyvWD14r4rdZatWrx4dy5/G/9ep6bNKlcZtYkJSXR5a67uG/T\nJj5OTOTop5/y6N//fk2vDQwMZNKUKax0cuIC1mvUnABMzs4Udu1KU2dn7gW8LBaGAyOA3YWFtPnx\nR7pHRVFQUHDFdj09POiAdVrmc0CKoyMeXl5lsLciUpGV23CNyWSiffv21K1bl5iYGHr16lXs+djY\n2KKfo6KiiIqKKq9SbrgVK1bQtbCQmIvLs7OzuW3JEj4tLCy6r+uf6dy5My84OvJgXh6NgDlA5cqV\nCW3WjLWrV9PQ05PdOTn4u7oy9eJfCJMKCvhPcjLHjh0jICDgsjZj//1vBt99N8MyM0lxcmKZlxeb\nRowos30WkbIXFxdHXFxcqdoot9k1ycnJ1KhRg71799KzZ0/Wr19P9erVrZ3a4bVrLjVnzhy+GD6c\nby5ejvgoEGqxcPjXX5nx5pv8lpxMxx496HfxSpaZmZns378fX19f6tSpA1ivOzNsyBCcAQ8vL/41\ncyajH36YbZmZVMM6u6YDcBrrnaZOA3WdnTly4gS+vr5XrGvz5s0s+fpr3NzdiRkypOjfQ0RuDiXJ\nzhsyhXLMmDGEhIQwdOhQa6d2HvLp6elEhIbSKjmZprm5vO3mxsCnnuKzTz+l7cmTNMrL4y03N4ZN\nmkTnbt3o1q4dXrm5JOfmEvPoo7w2bRomk4nc3FzOnDlD1apVWbx4MXNiYvj6kuvheJnNhFgsdMnK\nYpG7O92HDmXKm2/acM9FpDxVmJDPzMykoKAAT09PUlNTiYqKYvny5dSuXbvEhd5szp49y7SpUzn1\n66+069aN9PR0vhk1isUXb5R9AIh0d8e/Zk0eP3CAGKwXH2vl7s6bX35JdHR0sfb27t1LVPPmbMjK\nIhD4BnjU25uXpk4l8cgRwpo1o0+fPtd1M28RublUmHnyJ0+epG/fvgD4+vry9NNPFwX8rcLHx4fJ\nL71UtPz2229T5eK0SIAqQFZuLnsOH2bg768BOuflsWfPnstCPiQkhJf+/W+ajR5NNWdn0s1mvvrf\n/2jVqlX574yI3LRuqW+82tLhw4eJaNyYqRcuEApMcnWlSu/ebNuyhTGHDvEgcB4Id3am2/DhxMbG\n4u3tfVk7Z8+eJTk5mYCAANzc3K7aX35+Pg4ODn95Zv/DDz9w+PBhGjduTGhoaOl2UkTKVYmy87pv\n/V0GbNStzW3atMloHx5uhPr7G08NH25kZWUZO3bsMGpVrmw09fAwKoHRxMHB6OPqagTWqGGcPHny\nuvs4ffq0Ed2mjeFoNhuVXFyM995556rbjn/ySaOuu7sx0NPTqO7mZnzw7rul2T0RKWclyU6dyVcA\nGRkZRLdpQ+cdO/jnxXWjHB1xefxx/nWdH6T2j46m2tq1TM/L4wjQ0c2NOd9+S9u2bYttt2vXLrrd\ncQe7MzPxxnptnDCLheOpqXh6el6paRGxMV2F8ibl4eFBXlYWHS5Z1zg/n9Tjx6+7rfiEBP6Zl4cT\n0AD4e3Y28fHxl213/PhxQpyc+H1AKBDwdnTUpQ5E7IxCvoJo360bL7m6chZIAqa7u9O+R4/rbqea\nry/bLv5cCGx3cbnifPgmTZqwIz+fhIvL8wFcXfHz8ytR/SJSMWm4poLIzc1lZEwM8xYswGwyUd/P\nDx9vb7oOGMDTEyZc0zdlAdasWcO9vXrRDThsNmNq0IBVGzZgsVgu23b58uX8/e67yc7JoaqvL4uW\nLaNp06ZlvGciUlYqzDz5v+xUIX9Vhw8fJrJJE/6RkUEQEOvmRtthw3h16tRrbuPgwYOsW7cOb29v\nevbs+adX2SwsLCQ9PZ1KlSppjr1IBaeQtwNTp07l0HPP8c7FuzYdASI9PTl5yTddReTWpA9e7YCD\ngwM5l5xR5wAO1zhUIyLy/yk9KpiBAwey3M2Nf5rNzAUGuLkxauxYW5clIjcpDddUQImJibwWG8uZ\nU6foOmAADw0erPFyEdGYvIiIPdOYvIiIFKOQFxGxYwp5ERE7ppAXEbFjCnkRETumkBcRsWMKeRER\nO6aQFxGxYwp5ERE7ppAXEbFjCnkRETumkBcRsWMKeRERO6aQFxGxYwp5ERE7ppAXEbFjCnkRETum\nkBcRsWMKeRERO6aQFxGxYwp5ERE7ppAXEbFjCnkRETumkBcRsWMKeRERO1ZuIR8fH09ISAj169dn\n5syZ5dWNiIj8CZNhGEZ5NBwWFsb06dPx9/enS5curF+/nipVqlg7NZkop25FROxWSbKzXM7kz58/\nD8Bdd92Fv78/nTt3ZtOmTeXRlYiI/AnH8mh0y5YtBAcHFy03bNiQjRs30r1796J1sbGxRT9HRUUR\nFRVVHqWIiNy04uLiiIuLK1Ub5RLy1+LSkBcRkcv9/xPgyZMnX3cb5TJcEx4ezr59+4qW9+zZQ2Rk\nZHl0JSIif6JcQt7LywuwzrBJTExk1apVRERElEdXIiLyJ8ptuGbatGk89thj5OXlMWrUqKKZNSIi\ncuOU2xTKP+1UUyhFRK5bhZlCKSIiFYNCXkTEjinkRUTsmEJeRMSOKeRFROyYQl5ExI4p5EVE7JhC\nXkTEjinkRUTsmEJeRMSOKeRFROyYQl5ExI4p5EVE7JhCXkTEjinkRUTsmEJeRMSOKeRFROyYQl5E\nxI4p5EVE7JhCXkTEjinkRUTsmEJeRMSOKeRFROyYQl5ExI4p5EVE7JhCXkTEjinkRUTsmEJeRMSO\nKeRFROyYQl5ExI4p5EVE7JhCXkTEjinkRUTsmEJeRMSOKeRFROyYQl5ExI6VecjHxsbi5+dHWFgY\nYWFhLF++vKy7EBGRa+RY1g2aTCbGjBnDmDFjyrppERG5TuUyXGMYRnk0KyIi16nMz+QBZs6cycKF\nC+nbty8jRozA09Pzsm1iY2OLfo6KiiIqKqo8ShERuWnFxcURFxdXqjZMRglOuzt16kRKSspl619+\n+WUiIyO57bbbSEtLY9y4cTRo0ICxY8cW79Rk0tm+iMh1Kkl2lijkr9XOnTsZMWIECQkJxTtVyIuI\nXLeSZGeZj8knJycDkJ+fz/z58+nWrVtZdyEiIteozEN+woQJNG7cmMjISPLy8hg+fHhZdyEiIteo\nXIdrrtqphmtERK5bhRiuERGRikMhLyJixxTyIiJ2TCEvImLHFPIiInZMIS8iYscU8iIidkwhLyJi\nxxTyIiJ2TCEvImLHFPIiInZMIS9iJwzDKLquSXp6Oo898QhN7mhIv/t7c+zYMRtXJ7aikBe5yRmG\nQexLk/D0ccfVw4WYYQ/To380a87PpfLreznY4FtatYsgPT3d1qWKDegqlCI3uU9nf8r4N0bSfOkF\nHD1gaz9XTu/IpfvpAswXb/C5OgxyD7nQp3c/xoweQ/Pmzcus/7y8PL7++mtSU1Np06YNoaGhZda2\nFKerUIrcQk6fPk1cXByffTmH2qMv4O4PFl8IGJtFfl4hhTnW7QwDcs9AgWs2a8/MJ6pnK154JbZM\nasjLy6NDt7Y8NT2GGTufpnWHCL5a9FWZtC1lQ2fyIjehDRs20L1vNJUamEnddwG32wuI2mhgMsHP\nkyD5Ay+c6+RQe3g2yUshZTl02Q9ufpB9EuIaufLTlp8JCAgoVR2ff/45495+hIh1FzCZ4fRG+Km/\nD6nHz5TNjkoxJclOx3KqRUTK0T0P9Kfhh+nU7AV5abCqESS0tVBYWMCZXfn4tS7g5JYC9k9wxRKc\nhXuANeABXKqB9+3OnDhxotQhf+rUKTxC8zFdHBPwbgLnUtMwDAOTyVSqtqVsaLhG5CZTUFDAicST\n1OhuXXaqBLW7uNK6ag/O74SO26HFsgza78yjIL+QrF0uZB6D5G+s259cDelHCggODr5i+2vWrKHX\nwG70vKcrK1eu/NNa2rRpw4mvzJzdCgU5sO95R1q1i1TAVyAKeZGbjIODA/UaBnB0tnU5KxlOrTLR\ns2dPvOu44hFoXe9aCyrXd+HZsc9jNhzYMhgWe8CGXjB65NNUrlz5srbXrFlDv0G9ONFlGSldl3PP\nQ31ZsWLFVWsJCwvjg5kfs61bJZZ4mPHdFc7COYvKY7elhDQmL3IT2r17N517tCfPMYsLqXlMfO55\nnhw5mtq31yR0XjrVOsFvCbCtjztdukSzv/lX1B8NeecgdR0UzmjGxrVbL2u35z1dSem6nIDB1uWj\ns6Hy4g4sX7z6L2sqLCzEbNZ5Y3nSmLzILaJRo0Yc2X+MI0eOUKVKFapUqQLA4gVL6DewN9vJpSC7\nkKEPDyHl7AlMZjCZwNkHzI5QcJWgMAwDLhlpMTmAYRReU00K+IpJIS9SQS1dupT3Z7+Nk6MTT4+c\nwJ133lnseYvFctm4elRUFMuWrKRTt/b49Tf45sR/yNjkyoVvXXH0zMLRA34Z78Zbr469Yp+jho7h\n7oe+x+SQickM+8e5Mv8/V95Wbg4arhGpgBYtWkTMqAdo8GomBVnw8wQn7oi4g1YRd+JX048V677F\ny92Hgf3uIyIiotj4+l1dWpE18AcCYqzLux53otm5fpzPOkNObg7DHnqcuwfcfdW+ly9fzrT338Aw\nChk19Gm6d+9+2Tb5+fk4Ouoc8UYrSXYq5EUqoNadWmKM3EKtPtblA9PhxNdQcN6BvHSDoOcLSdsN\nh98DU74D3bp1Y/zTz9CqVStCmtenxjsH8Y2wvvbQu9Bo2/3M/nBeqevasWMH/e/vzZH9x6hVtzpf\nzltMREREqduVa6NvvIrYCes88z+WTQ7gFgAXfi3gjiWF5J6xzo93rweeYQUcaLKUrv07snjxYrp2\n7M6hWFdyfoOMg3BsuhvdO/UqdU1ZWVl06dmRKhOT6JdnUPuNZLr27sy5c+dK3baUH4W8SAX05KNj\n+fkJN459AYkfw8+TwdEdCnLhh/5wcoX1i0eOHpBxAM7tgqB/ZfHM5LFMefFfRPn1Y0VdMytDISMl\nn/j16zh48CD5+flX7XPuvLnUbxxAnQY1eD52IoWFxT9wPXjwIHjmUmcQmMxQqw+4B5j4+eefy/tw\nSCko5EUqoIH3DOQ/02bjOb8thyd54OgO6b9A0FjrdWhaL4HAEXDXautsGXd/2P0cpJ9Px9nZmdp+\n/tzW3ELXQ9B2cy6fLHmX4Kb1qVOvFnv27Lmsv2XLlvHEM49R662jBH+Vwqzl03j5tZeKbXPbbbeR\nkZxL9knrcu5ZOH8kl6pVq5Z4PxcvXsyIJ4fz4ssvkpaWVuJ25OoU8iIVVP9+/Vn13zji/reBgjQH\nWi+F6l3BUgVMTtZtzM7gWAn8HwJXPwhvFgnANyu+pt4LWbjWBM8GEPIPcK0NRqNT9BnY47K+Fv73\ncwLGZ3LbXeAVCkFTM/licfEx/OrVqzPu6fFsiHTnp6GuJLR0Z+jgx6hXr16J9m/K66/y6DN/J8H/\nPT7d9xIt2zTjwoULJWpLrk4fj4tUcD/++CNmJwOzE1RqZF330zNQeyAc+wIcXMEzGEyZTgx7ZDgA\nVXxv48xeuO0u6/bp+8CzPmCGI/uTyMnJwWKxFPXh6e5FznEzYB2iyToOnh6el9US+/wLdGjbiT17\n9hB0fxDt2rUr0T4ZhsELL0wm6qcc3APAMHL5setJ/vvf/3L//feXqE25Ms2uEanAlixZwt0P9cUw\nFVKrPwQMhl/nmUhZ6IyjkyN5DlkEPlnIhR0WPH6px6b4rVgsFrZv306rqJZUH5BPQQ6c/h68wqzj\n+mmrPPj0w7l06NABDw8PABITE2l+R1N8B2Tg6FPAoelm6gbU5bEhIxn9xOgyvxZNQUEBLm4Wepwt\nwNHNum7XQ26MazOdRx55pEz7sieaQiliRwzDoPrtvljuOIv/w/BzLGTsB6MAfGp6kHvaRMwDj5B8\n+ldur1OfCWOfwdPzj7PvnTt30qNPN1LPpWA4FeLoaCbnXCHeDZxx9IDCox68/M8pmM1mqlevzqDB\n91FgyebCqTxui4JqHeHUPDdGD3qeZ8Y9W+b71/feXuxiFYETszm3A3552oOdW3bj7+9f5n3ZC4W8\niB05c+YM1fyq0CvNKLrD07p21qmTHbfCiaVwcIQvKUmpxc60V69ezbr4deRk53DnnXdy8uRJUlNT\nWfndco43+J6mb8ORj2D3RCjIBK9AR84fyOf2kXByOWCCqu2t8/Kr94TC1X4c3lP294i9cOECo8aO\nZE3caqpVq8Zbr79HeHh4mfdjTxTyInYkPT0d32o+dEspwKmS9Q5PKxtBnfsg5HnrNkvcHElNOU2l\nSpUAmP7WNCZPnUi1v2dyOgHO74LCXPCq5kZuWiF/ezsbszPseBIiPgcnL9gaA+714bfvgULrzUUc\nLNZx+eUNIOD2Ohz86ajtDoQU0QXKROyIp6cnDw9+iCWd5+P3SDanVpnI/hXqPGj9T56yDCr5VMLT\n05M1a9awaMmXfPDBB9z5XSFVIq1vCuvuAtc6kJ+RiYebmV0TrB/GBo2n6Buxof+CHaOtd4yqHG4N\neACXmmByhJFDRtvoCEhZUMiLVGDvzfyQJu83IyEhnujgehjBhcxoNg2vAAsXjhUycdyzdO0ZTdz3\nawkcm0+dh2HTPdBhM7hUt35L1qM+/LoA6vyjkOQ1cOxz6zTM32UehfxzJpydzZzdWsDxr63DNQdn\ngLenN0+gjIPCAAAMR0lEQVSOetJWuy9loMTDNQsXLiQ2NpZ9+/axZcsWmjVrVvTcjBkzmDlzJk5O\nTtYzi/939TwN14j8ufz8fBYsWMDx48eJjIykTZs2Rc8lJSWRnJzMuvXrmPLWZGoNyyTtJzi3Hdpt\nhJ1PgqUqVI6ErUOhRg8ozIa8s2acfAsJmQjxHa3rnbzh0NvQOLQJ5y+cweepYxyYChcSwa0O9Ai/\nh/mzv7DdgZBibuhwTWhoKIsXL+axxx4rtv7UqVO88847rFmzhiNHjjBq1Ci2bdtW0m5EbjkFBQV0\n7xfNntMbqRSRyyuDnHjhmVd5YsQoAOrUqUOdOnWI7tmJlusyqRRifV1Cb/h1oXWY5eBUE4ffMzBh\nInkxODtb8Pb2xq1hKpUaFtBhC2y530T1jL+xZf1swsLCCL+rKY6ex+h88Quxu55wxL9SXRsdBSkr\nJQ75q90fctOmTURHRxf9IhqGQXp6erGpXSJydWvWrGFX0iZa/XgBsyMEPJHHuEbjGP7oiGKX9826\nkINLjT9e51INzmyE01+7sXF9PDVr1iQzM5PCwkKcnJywWCxE3tWCLT+lgwksxz1Y/v1KatSwNvLG\ni9PpOaAbZzdlc3aziQv7zPzWP5WUlBSqV69+ow+DlJEyH5PfvHkzISEhRctBQUFs3ryZDh06FNsu\nNja26OeoqCiioqLKuhSRm9Lp06fxqG8qmjbpFgCFRgH97+/N6hVrcbY406tbHyLvbMlPj/xI/Rez\nSdsDx+absLg5k5uWy90P9GPOh/Np3bp1sbZ3b9vH6tWriYv7jvl75hAYVJduPbvyyftzaNu2LQlr\nNzL66Sc4lfoDwVNyWf/TbFq0+paftv6Mj4/PjT8Yt7i4uDji4uJK1cafhnynTp1ISUm5bP0rr7xC\nz549r/iaK40XXenbcpeGvIj8oVWrVpwaVUjKcvC9Aw6+4YCnrzt7nNbgNyyHXxdkE2+ezflDFmr6\nBLC393kq+/pSq8Z5PB9IJvDJAk59l0RUlzY4O1ro2j2aj9+bjaenJ56enlSpUoVPv/yIFksycQuA\n7Y8v49EnhjD/4y9o1KgRGzdtIuqnXNzqAOSzLSmNxYsXExMTY+Mjc+v5/yfAkydPvu42/jTkV61a\ndd0NRkREsHr1Hzf93bdvn77gIHId/P39+XrBUgYPe4Afj6fS4o5m/Jqzm4Cnc4jvANG/gOU2yDuf\nw3dBSfz4/U68vLy4PaQOLf5RgMlkvQzw4TYGte/NZseqZcQMf4iFcxcBsGr1KmrGZOHT3Npf0JQc\nVrZaWdR/fl4Bjh5/1OPgaZCbm3sjD4GUoTK5CuWlZ+8tW7ZkxYoVJCUlERcXh9ls1ni8yHVq164d\nift/JSsjh+9X/UDlKj6c22qdFmm5zbqNkxdU8ncmNTUVT09P8rMLyTpufa4w1zo10r0uhEzNYcWy\nP0Lct7Iv2fv+uDhZ+j6o7OsNWP/qHvTg/Wy/z5XU7+HwOyZSlzle8RaAcpMwSmjRokWGn5+f4eLi\nYlSrVs2Ijo4uem7atGlGYGCgERISYsTHx1/22lJ0K3JLWrp0qeFRxcVwrozR/COMfjkYEV9gVK7u\nZZw7d84wDMOY8vqrRuUANyNotIPh3RSj1gCM/gUYbeMxagVWL2orLS3NCGpczwjo6WYEj3I2Kt3m\nZixbtqzo+dzcXGP8xLFGkzsaGp17tTd27dp1w/dXrqwk2anLGojcJHbs2MGcOXOYv2g2J5NO41+/\nFgvnLqZFixZF26xdu5aEhATen/UOjqHncWuQx/E5Tnzy3jz69u1btF1GRgafffYZaWlpdO7cmdDQ\nUFvsklwnXbtG5BZhGMafXv73woULzJ07l3PnztGxY0eaN29+A6uT8qKQFxGxYyXJTt3+T0TEjink\nReQyCxYuoGVUGOFtmzJn7hxblyOloKtQikgxS5Ys4bExg/nbu5mYHODJEcNwcnLi3oH32ro0KQGN\nyYtIMT3ujuZUzxX4P2hd/vVLcP3kTtZ+871tCxONyYtI6VmcLeSl/7Gcnw4WZxfbFSSlouEaESlm\n/Kjn6NRjNQVZ1uGaIy+78spn421dlpSQzuRFpJiIiAjWLltH6O57OTnDl5ysPHr06sbzsRM1zHoT\nUsiLyGVatGhBVs4FvLul0TMtn86J+XywYBqLFi2ydWlynRTyInJFP/zwA3WfysPkYL0hSbUHMtmw\nKcHWZcl1UsiLyBXVql2L0+utPxuFkL7BFX+/AJvWJNdPUyhF5Iq2b99Oh65RVG4JmScK8fcI4bvl\n8bi4aKaNrejaNSJSpk6ePMn69evx8PCgffv2ODk52bqkW5pCXkTEjunLUCIiUoxCXkTEjinkRUTs\nmEJeRMSOKeRFROyYQl5ExI4p5EVE7JhCXkTEjinkRUTsmEJeRMSOKeRFROyYQl5ExI4p5EVE7JhC\nXkTEjinkRUTsmEJeRMSOKeRFROyYQl5ExI4p5EVE7JhCXkTEjinkbSwuLs7WJVQYOhZ/0LH4g45F\n6ZQ45BcuXMjf/vY3HBwc2LZtW9H6xMREXF1dCQsLIywsjBEjRpRJofZKv8B/0LH4g47FH3QsSsex\npC8MDQ1l8eLFPPbYY5c9V69ePbZv316qwkREpPRKHPLBwcFlWYeIiJQHo5SioqKMrVu3Fi0fOXLE\ncHd3N5o0aWI8+uijxo4dOy57DaCHHnrooUcJHtfrT8/kO3XqREpKymXrX3nlFXr27HnF19SsWZNj\nx47h4+PDsmXLeOCBB9i1a1exbaw5LyIi5e1PQ37VqlXX3aCzszPOzs4AdO3alYkTJ3Lw4EHq1atX\nsgpFRKTEymQK5aVn5r/99hsFBQUAbNu2jaysLAW8iIiNlDjkFy9eTO3atdm4cSPdu3ena9euAKxb\nt44mTZrQtGlTXnnlFd5///0yK1ZERK5TKT5zvW4LFiwwGjZsaJjN5mIf1hqGYUyfPt2oV6+eERIS\nYnz//fc3sqwKYdKkSUatWrWMpk2bGk2bNjWWLVtm65JuqHXr1hnBwcFGvXr1jBkzZti6HJvz9/c3\nQkNDjaZNmxrh4eG2LueGGTx4sFG1alWjUaNGRevS0tKMXr16GbVr1zZ69+5tpKen27DCG+dKx6Ik\nOXFDQ37v3r3G/v37L5uRc/LkSSMoKMg4evSoERcXZ4SFhd3IsiqE2NhYY+rUqbYuw2aaNm1qrFu3\nzkhMTDSCgoKM1NRUW5dkUwEBAcbp06dtXcYNFx8fb2zbtq1YsL322mvG448/bmRnZxsjR440Xn/9\ndRtWeONc6ViUJCdu6GUNgoODadCgwWXrN23aRHR0NHXq1KFt27YYhkF6evqNLK1CMG7RWUfnz58H\n4K677sLf35/OnTuzadMmG1dle7fi70ObNm3w8fEptm7z5s0MGTIEi8VCTEzMLfO7caVjAdf/e1Eh\nrl2zefNmQkJCipaDgoLYvHmzDSuyjZkzZxIZGclrr712S73JbdmypdiX6xo2bMjGjRttWJHtmUwm\n2rdvT58+fViyZImty7GpS38/goODb8lsuNT15kSZh3ynTp0IDQ297LF06dKrvuZK70wmk6msS7O5\nqx2bJUuWMHz4cI4cOcKKFSs4dOiQPrC+xSUkJLBz505effVVxowZc8Xvq9wqbsW/aK6mJDlR4ssa\nXE1J5tZHRESwevXqouV9+/YRHh5elmVVCNdybLy8vBg5ciQjRoxg7NixN6Aq2wsPD2fcuHFFy3v2\n7CE6OtqGFdlejRo1AAgJCaFXr14sXbqUoUOH2rgq2wgPD2fv3r2EhYWxd+9eu8yGa1W1alXg+nLC\nZsM1l747t2zZkhUrVpCUlERcXBxmsxlPT09blWYTycnJAOTn5zN//ny6detm44puHC8vLwDi4+NJ\nTExk1apVRERE2Lgq28nMzCz6Mzw1NZUVK1bc0m96ERERzJo1i6ysLGbNmkVkZKStS7KZEuVEmX0U\nfA0WLVpk+Pn5GS4uLka1atWM6OjoouemTZtmBAYGGiEhIUZ8fPyNLKtCeOCBB4zQ0FCjefPmxlNP\nPXXLzayIi4szgoODjcDAQGP69Om2LsemDh8+bDRp0sRo0qSJ0b59e+Ojjz6ydUk3zL333mvUqFHD\ncHZ2Nvz8/IxZs2bdslMofz8WTk5Ohp+fn/HRRx+VKCdMhqEBLxERe1UhZteIiEj5UMiLiNgxhbyI\niB1TyIuI2DGFvIiIHVPIi4jYsf8DukTFtk9nJv0AAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.prism()\n",
    "plt.scatter(X[:, 0], X[:, 1], c=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import logistic regression from scikit-learn and generate a classification object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split the dataset into a training set and a test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = X[:50]\n",
    "y_train = y[:50]\n",
    "X_test = X[50:]\n",
    "y_test = y[50:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get an idea of how hard the task is, let us visualize the data again, this time only labeling the training points.\n",
    "\n",
    "The test points are plottet as white triangles.\n",
    "Can we see which class the test points belong to?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0xabbe70c>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD9CAYAAABZVQdHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4TNf/B/D3JJmssgoikU0iC9IkiMSeEntja5XSTSgV\nS3XR+lb7FZRSfK1Vqqhd7RW10xFSkhBEUlEhEVqRWCJ7JjNzfn9Eo/lZKskkE9f79TzzPO6dued8\n7hVvJ2fO3JEJIQSIiEiS9HRdABERVR+GPBGRhDHkiYgkjCFPRCRhDHkiIgljyBMRSViVQj4sLAwN\nGjSAj49P2b7c3Fz07dsXTk5O6NevH/Ly8qpcJBERVU6VQn7YsGHYv39/uX3fffcdnJyccPnyZTRq\n1AjLli2rUoFERFR5VQr5Dh06wNrauty+2NhYDB8+HEZGRggLC0NMTEyVCiQiosoz0HaDcXFx8PLy\nAgB4eXkhNjb2kdfIZDJtd0tE9EKo6E0KtP7G67MWIITgQwhMmTJF5zXUlgevBa8Fr8XTH5Wh9ZAP\nCAjAxYsXAQAXL15EQECAtrsgIqJnpPWQDwwMxKpVq1BYWIhVq1YhKChI210QEdEzqlLIv/HGG2jb\nti3++OMPODo6YvXq1Rg9ejTS09Ph6emJP//8E++//762apWk4OBgXZdQa/BaPMRr8RCvRdXIRGUn\neqrSqUxW6fklIqIXVWWyk594JSKSMIY8EZGEMeSJiCSMIU9EJGEMeSIiCWPIPyMhBHJycrgqiIie\nKwz5Z3D69Gm42tmhYd26sLOywpEjR3RdEhHRM+E6+X9RVFQEN3t7LLh3DwMB/ArgdTMzJF29ivr1\n6+u6PCJ6gXCdfDVIS0uDaUkJBj7YfhmAl4EBkpKSdFkWEdEzYcj/i/r16yOzpATXHmzfAXBZqYS9\nvb0uyyIieiYM+X9hY2ODGbNmoY2pKd6oUwctzczw3tix8PT01HVpRET/inPyz+jcuXNITEyEu7s7\n76xJRDpRmexkyFezvLw8fPnll5gzZw4MDLT+RVxE9ALhG6+10LfffosFCxZgw4YNui6FiF5AHMlX\no7y8PLi5uSEiIgLz5s1DcnIyR/NEVGkcydcy3377LTp37ozRo0fD0dGRo3kiqnEM+WqSl5eHOXPm\noGPHjjh27BhCQkIwffp0qFSqx77++vXreLVHD7zk4oKh/fohKyurhismIinidE01SUtLQ1hYGNRq\nddk+Q0ND/PTTT7CxsSn32oKCAvh7emLIzZvoo1ZjrVyOE+7uOJmQgBlTpuC7JUsgk8nQLjgYn//3\nv2jRokVNnw4R1QJcXVOLCSGwfft2JCUmwsPTE4MGDYKeXukvUtHR0figVy+czskpfS0AFzMzDHzv\nPWxevBh+ajU8ACyVy2HXsCFS09Igk8m0UteFCxewadMmzJw5UyvtEVH14Zx8LTbh/ffx1bvvomTa\nNMx/7z2MGDoUGo0Gly9fRkZGBrJVKvw9kVMIIE+lwqpvv8WrajWGAthpYICBAwdCrVbjg/HjtVbX\n5MmTMXv2bJw5c0ZrbRJR7cGRfA24fv06/D08cKWoCJYACgB4mJjAxdsbacnJMJDJoJTJ4KNSoU9R\nEbaamkLdvDn04+OhUKlwG4CnsTHik5Nx/PhxTBg3Dll370KlUmH9+vV49913KzWyP3PmDPr27YsJ\nEyYgKioKu3fv1vapE5EWcSRfzbZs2YKUlJQKH5ednQ1buRyWD7ZNAVhrNJAnJiKtoABX8/PRo6QE\nxS+9hIS33sLA2bMx+oMPUMfQEADwPwMDtG7bFnfv3oWnpyfUQuDXX3/F+vXrERYWhsDAQFy8eLHC\ndU2dOhWTJk3C2LFjER8fz9E8kQRxJP+MMjMz4erqim7dumHnzp0VOra4uBjNGzfG6IwMDNFosEsm\nw38MDLCwpARvP3jNQQCzW7bEkdOnAQB3796Fn6cn3rt7F7GGhoh7EPgaPT0YGBvj9UGDEBkZCVdX\nV0RHR6Nfv37YtGnTM9d05swZtGvXDuvWrYOxsTE2bdqE3NxcREZGVujciKjmcCRfjebOnYvBgwcj\nJiYGZ8+erdCxRkZG2B8VhcgWLdDczAzrfXwwcOhQHDQyggalb7TuMjSEp48Prl+/jvatWsHX0xN+\nfn5I6N4dWc7OqGttjWZNmuCDDz9ETm4utm7dCicnJ6SlpWHr1q04dOgQfv/990f6Xrd2LXq2bYv+\nISGIiooq25+fn4+uXbti7dq1+P7775GbmwsHB4cqXiUiqm04kn8GmZmZ8PLyQkJCArZv3w6FQlHh\n0fz/l5eXh97Bwbh56RIMAJg5OuLnw4cR3Lo1iu7fx+3iYryp0eCMuztiEhPLPik7YcIEqNVqrF+/\nHm+//TbOnz8PhUKBGTNmIDExsdxofuWKFZg1YQLmFBTgHoBPTU3xy6+/onXr1lWqnYh0gyP5ajJ3\n7lx06dIFSqUSISEhUCgUTxzNKxQKxMfH/2ubderUwdGYGMxYtQodhgzBb+fP49q1a1DdvQs7R0eM\nDQ+H3MAA927cwKeffoqsrCz89ddf+PHHHzFw4EBoNBqsXLkSjo6OWLhwIYQQ2LJlC5KTk8v6+GH+\nfCwvKEA/AMMATCwowLoVK7R0VYjoecAbqTyDlJQUnDt3DiEhIQAAa2trnD9/Hv7+/uVep1Qq8c47\n78DW1hanT5/+1xUv+vr6WLt2Lfbu3Yv27dtjyZIluGdggCVz5yIgIABe33+PEqUS8+fPh76+Pvbu\n3o2XfHyQnZ2N1atXY9iwYbh37x6uXr0KABg3bhyMjIzK2pfJZCj5R38lAPT09bVyTYjoOSF0QEfd\nVrvly5eLrl27Cl9fX7F79+5/fX1cXJxwcHAQM2bMEI6OjsLY2FhYW1uLbdu2ie3bt4tOnToJCwsL\nMW3aNGFhYSHsTUyEn4WF8HzwsLG0FD/++OMT29+4YYNwNDUVawAxHxC2Zmbi3Llz2jxlIqpBlclO\nzslriVKpRJMmTbB582ZkZGTgq6+++tfRfLdu3RAaGorhw4fDwcEBr732Gvbs2QNlYSE8hECKnh40\nenq4desWxo4di52rVqFFSQn6ANgJIBrA1wsXYvwTPhylUqmwefNm7Nm0CUYmJvjg8895SwSi5xjn\n5HXoxx9/hL29PRwdHdGyZUvcv38fe/bseeLr9+/fj2MKBRwdHREVFYXu3bsjMTER9vb2aJOTg7E5\nOTAwNMTw4cNx4cIF9O7dG9kAfgAwGkAkAGMAqampKC4uxhdffIGSkpJyfURERGDOnDnYGBmJGQsW\n4OjRo5g9ezauXLlSjVeCiGoTjuS1JDw8/JFPjI4YMQIRERGPff3IkSOh2LIF9QEkonR+XqPRoH37\n9ohTKGCh0cC0SROo1WrcuHEDdnZ2yLhxA/vz8hCI0mWXjQD8b/Nm3L17F+Hh4Vi5ciXCwsIAAHfu\n3IGHhwfq1auHcePG4avJk9G3sBByIfCTkREOR0fjpZdeqsYrQkTaVqns1OJ00TPTUbdCCCH+/PNP\n8d6bb4pXOnQQX0+bJkpKSnRSx7SpU8Un+vriMiA8zMzEihUrRNs2bUSXLl2EqZGRcLCzE/v37xcH\nDhwQQUFBYvz48aKps7P4VCYTpwHxASAcrKzE9OnThZ2dnZg7d65wdXUVSqVSCCHE5MmTxYgRI0Rk\nZKRoaGcnpshkQgBCAGIhIPp07qyT8yaiyqtMdr5QIX/v3j3R2M5OTNLXFzsB8bKpqRj59ts6qSUx\nMVHYmpqKLwDhVL++UCqV4sSJE8LZ2VnMmjVLNGzYUFhbWQk7a2vhYmMjrM3NxbJly4S5gYGwAYQ5\nIFrq64vO+vqiU8eOQgghunTpIlauXClu374tbGxsxNWrV4VGoxHNmjYVEx4EvADEXkDYGRmJ4uJi\nnZw7EVVOZbLzhVpCefDgQXjl5eHrB/d471JQANsNG7B4xQoYPrhtQE1p1qwZdh08iEGvvQZ7R0d8\n/vnn0NPTQ3Z2NmxsbKBSqVBUVIT4oiK4ANgMYMz48SjQaBBgaIjTKhW2q9Vob2qK/wwahNOnTyM0\nNBRfffUVMjMzoVQqy6Zu7ufkYLWxMd4rKoIcwBQA+kIgISEBrVq1qtHzJqKa9ULNyf/0009YN2IE\n9uTlAQByAdjq6yO3oKDGQ/5vW7ZsQWpqarl9OTk5iIuLg2/z5shfuhRLi4txA4CXkRFef/11/LRt\nG+wdHBBy7RrOyuUokctxBYC7uzvkcjl++OEHZGZmlrV3/fp1jAsLg4VaDQ1KPxi1xtAQv8TFcV6e\n6DlSq740xMXFBRYWFtDX14dcLkdsbOzDTnUU8tnZ2Wjh5YU3bt9GoFqNRaamcO3fHyvWr6/xWp6k\nuLgY7u7u2LFjB1xcXODp7IyzhYWYpqeHn4yNkXbtGiIiIlBYWIjtGzfidFERvjUwwIVWrXD45MnH\ntimEQHMXFzRMT8cQALsAHJfJsP3wYXTu3LlGz4+IKq9WLaGUyWRlH///Z8DrkpWVFY6fOYPMwYOx\nvF07dP70U3z344/V1l9CQgKGDRtWob+UVatWQQiB06dPY9u2bWjctCmaGxlhi6Eh3h02DHp6eggP\nD8e2bdvQq39/+MjlWFP63gpu3Ljx2DZlMhk0ajV8ACgABAH4SAj8smuXNk6TiGqxahvJu7q64vTp\n06hbt+6jnUpwCeXj9O3bF/v27cOePXvQrVu3Zzrml19+wZIlS5AaHY2XVSqoATRTKvG5oSEM5HIU\nFxeXfW2gtZUV7mZmIkmtxlp9fex0cUH8pUvQf8ytC1p7eeG/ly7hlQfbYwwMYDtpEqZOn66lsyWi\n6larpmsaN24Mc3NzuLq6IiwsDH369HnYqUyGKVOmlG0HBwcjODi4OsrQmfj4eISGhmLGjBn4/vvv\nER0d/czf3vTHH3+gvZ8fjhYWojmAdQCm2Nkh6epVfPPNN5gxYwY+/vhj7F28GN/l56PNg+McTU1x\nIikJLi4uj7S5d+9eDBs4EO8XFCBDLsc+S0vEJCSgYcOG2jplItIyhUIBhUJRtj116tTas07+r7/+\nEkII8fvvvws3Nzdx8+bNsueqsdtao0+fPmLRokVCpVIJb29vceDAASFE6Tr9FStWPPL6zMxModFo\nyrY3btggLIyNha2xsXBp0ECcP39eqNVq4evrKwIDA8XAgQOFu5mZKH6wLPI2IMwNDcXt27efWFNM\nTIyY/J//iBlffVXu74OIng+Vyc4aSdsPP/xQfP/99w87lXjInzlzRhgYGIjvvvtOrF69Wrzxxhui\nTZs2QgghRo4cKfT09ERycnLZ6wsKCkTDhg3F0qVLy7VTXFwsbt68KdRqtRBCiB07dogWLVqIW7du\nCRsbG9EzOFh0MDMT/wVEczMz8dmECTV3kkRU4yqTndUyXVNQUAC1Wg1zc3NkZWUhODgY+/fvh6Oj\nIwDpz8mfP38e8+fPL7evbt26GDduHFq2bIl33nkHWVlZWLduHQBg4cKFWLVqFe7du4fLly+Xu13w\n3zQaDVq0aIHp06cjNDQUn332GXJzc9G6dWtcvXIF/i1aoF+/fpX6Qm8iej7Umjn51NRU9O/fH0Bp\nuA0dOrTsgzmA9EP+SUaOHIl69erhs88+g5ubG06cOAEnJye4ubnhl19+wRdffIFXXnkFo0ePfuTY\n48ePo2PHjvD19YVMJkNeXh7S09ORnZ0NExMTHZwNEdW0WhPy/9rpCxjyaWlpaNasGY4cOYK6deti\n/vz5yM3NRatWrcq+TjA2Nhb9+/eHQqFAkyZNyh2vVqtx4cKFctfN1NQUnp6ej+1PpVJBX1//X0f2\nGo0GMTExaNOmzVNfR0S6x5Cvxfbu3YsPPvig3D57e3tcu3YNRkZGsLOzAwBER0fDx8enwl8W/re7\nd+9iaL9+OBwdDVNDQ3zzv/9h1GN+M/jbtm3bMHDgQJw7dw6+vr6V6pOIagZD/jmUkJCAe/fuAQAO\nHDiAyMhI3Lt3Dzt37kRAQECF23u1Rw80OHoUC0tKkAogxNQU6/buRadOnR55rUajga+vL5ycnGBs\nbIzt27dX9XSIqBpVJjtfqBuU1UZ/3ztGpVJhxIgRWLFiBS5evIipU6c+9UtHniQqOhoXSkogB+AB\n4M2iIkRFRT025Hfs2AETExNs3boVbm5uOH/+PEfzRBLDb4aqJTZu3AgHBwcEBwcjLCwMCQkJiIuL\nq3A7DerWRfyDP2sAnDU2LpsK+ieNRoOpU6ciIiICpqam+PTTTzFt2rSqnQQR1Tqcrqkl+vbti8jI\nyLJbFqjVarRv3x7Hjx+vUDtHjhzB4D590AvAVT09yDw8cOi33x5ZlnnmzBm0atUKtra2kMlkUKlU\nuH//Pu7cuQMrKyttnRYRaRHn5J9jGo0GGo2mbLt37944cuQIzpw5U+EplJSUFBw7dgxWVlYIDQ19\n4m2Ub9++Xa5PuVwOa2vryp0AEVU7hrxEnDx5EoMHD0Z4eDhiY2P5higRAWDIS0aPHj0wYMAAvPnm\nm3Bzc8P+/fv5higR1a77yVPlnDp1ClFRUbCxscHhw4fRvn17TJ06VddlEdFziksoa5nCwkKEhIRg\n7dq1ZfucnZ11WBERPc84XUNE9JzgdA0REZXDkCcikjCGPBGRhDHkiYgkjCFPRCRhDHkiIgljyBMR\nSRhDnohIwhjyREQSxpAnIpIwhjwRkYQx5ImIJIwhT0QkYQx5IiIJY8gTEUkYQ56ISMIY8kREEsaQ\nJyKSMIY8EZGEMeSJiCSMIU9EJGEMeSIiCWPIExFJGEOeiEjCGPJERBJWbSEfFRUFb29vNGnSBIsX\nL66uboiI6ClkQghRHQ37+/tj4cKFcHZ2Rvfu3XHixAnY2tqWdiqToZq6JSKSrMpkZ7WM5O/fvw8A\n6NixI5ydndGtWzfExMRUR1dERPQUBtXRaFxcHLy8vMq2mzZtilOnTqF3795l+yIiIsr+HBwcjODg\n4OoohYjouaVQKKBQKKrURrWE/LP4Z8gTEdGj/v8AeOrUqRVuo1qmawICApCcnFy2nZSUhKCgoOro\nioiInqJaQt7S0hJA6QqbtLQ0HDp0CIGBgdXRFRERPUW1TdcsWLAAo0aNQklJCcaPH1+2soaIiGpO\ntS2hfGqnXEJJRFRhtWYJJRER1Q4MeSIiCWPIExFJGEOeiEjCGPJERBLGkCcikjCGPBGRhDHkiYgk\njCFPRCRhDHkiIgljyBMRSRhDnohIwhjyREQSxpAnIpIwhjwRkYQx5ImIJIwhT0QkYQx5IiIJY8gT\nEUkYQ56ISMIY8kREEsaQJyKSMIY8EZGEMeSJiCSMIU9EJGEMeSIiCWPIExFJGEOeiEjCGPJERBLG\nkCcikjCGPBGRhDHkiYgkjCFPRCRhDHkiIgljyBMRSRhDnohIwrQe8hEREWjUqBH8/f3h7++P/fv3\na7sLIiJ6RgbablAmk+Gjjz7CRx99pO2miYiogqplukYIUR3NEhFRBWl9JA8AixcvxtatW9G/f3+E\nh4fD3Nz8kddERESU/Tk4OBjBwcHVUQoR0XNLoVBAoVBUqQ2ZqMSwu2vXrsjIyHhk/4wZMxAUFIR6\n9eohJycHEydOhIeHBz755JPyncpkHO0TEVVQZbKzUiH/rM6fP4/w8HBER0eX75QhT0RUYZXJTq3P\nyd+8eRMAoFKpsHHjRvTq1UvbXRAR0TPSesh/9tlneOmllxAUFISSkhKMHj1a210QEdEzqtbpmid2\nyukaIqIKqxXTNUREVHsw5ImIJIwhT0QkYQx5IiIJY8gTEUkYQ56ISMIY8kREEsaQJyKSMIY8EZGE\nMeSJiCSMIU9EJGEMeSKJEEKUu6+JSqXC22+//djvfqAXB0Oe6DknhEDEV1Ngbm0GkzrGGD56GEpK\nSrBhwwZs3LgRc+bM0XWJpEO8CyXRc27N2jX4dO4YtIzMh0Ed4NwbphjSegy2bd6BadOmYdSoURAG\nahgZG2LI629h0YJFkMlkWuu/pKQEu3btQlZWFjp06AAfHx+ttU3lVSY7q+U7XomoZqSlpWHPwV1w\nnJAPM+fSfY2/KMCm4Rvh2sgdQ4YMQdTxY/hV/AjLjvexZOgSmNcxx8wZM7XSf0lJCbr06oSrhRdQ\np5kKkyJkWL1sHV4d8KpW2qeq43QN0XMqOzsbfn5+uHvrPvITH47Xcs4D9zJz0KJFC2zduhXeXk2R\ntlaDmyvM0L1Hd6xbt1Zrv0lv374dqUUJCIrKg8/yIrTaXYj3x72nlbZJOziSJ3pOLViwAF5eXvgj\n+TI0KTaIv54H/ToCf+0C8nNyER8fjx27tkNtdw/12uhDfcUUu5J3oVmzZjh8+DC6du1a5RoyMzNR\nx0cF2YPhopUvkJ2VAyGEVqeEqPIY8kTPoezsbCxZsgSnTp3CuHHj0LVrV1hYWECpVGLnjZ1QqVQQ\nQmD71h3o3KMTDC0NMXPyDBgbG+Orr75CREQEQkJCnhjEP//8M9RqNQYMGPDUOjp06IAvZ+jBYThg\n0RxI/sIAbV8OZMDXJkIHdNQtkWRMmTJFvPvuu0IIIU6ePCmcnJxEcXGxiI2NFQ4ODiIvL0+4u7uL\nX3/9VZw4cUIAEObm5sLCwkLUq1dPABCXLl16bNtFRUWiUaNGwt7eXhQUFPxrLZt/2iys61sIfQM9\n0T6kjbh165ZWz5Ueqkx2cnUN0XPI1tYWderUgZWVFQDgwoUL2LJlC1avXo2ePXtizJgxWLt2LVat\nWgWFQoHly5dj5cqVcHZ2hqurKyZOnIh69eo9tu2lS5di79690NfXR5cuXTB+/Phnqkmj0UBPj2/z\nVafKZCdDnug5dPXqVeTk5JTbZ2VlBVdXVzg6OkIul6OkpATXr19HUlISBgwYgCVLlsDZ2Rlt27ZF\nSkoKLC0tH2m3uLgY7u7u2LFjB/T19REaGoqUlBSYmJjU1KnRU3AJJZGEREZGYvnabyE3kOPjMZ+h\nffv2Zc81btz4scdcu3YNJSUlUCqVCAkJQZcuXXDu3DkUFxdDX18ff/75Jzw8PLBo0SJ8+eWXjxy/\ncuVKWFpaIi8vD0DpbwwrVqx45tE81T4MeaJaaMeOHQgb/xY8vi6AuhDoGnoAbQLboF1QB0ya+B+s\nWLEC7du3h76+PpydnWFjYwMAcHJyAgCsWLECHh4eSE5ORmJiIpydnTF16lQAgFwuh0qlemy/RUVF\nqFevHqZNmwYAsLGxQXFx8SOvU6lUMDBgfDwPOF1DVAu169oaYkwcHPqVbl9eCPy1CzC3M4LpH43x\n+7lLMDM3hXFD4P61IvTs2ROffjwJbdu2hVKphIeHBzZs2ICzZ8/i4MGD2L17t1bqOnfuHF4d0hep\nl67DwdUO2zbsRGBgoFbapn9XmezkuyREtZAQAv9chSjTB0xdAP8Nxbjx5w18/PHHKFGVICc7Dw5v\nqnDZNxI9Xw3Bzp07sWbNGnh4eKBdu3YYMWIE4uPjcebMmSrXVFhYiO6hIbCdnI4BJQKOc2+iZ99u\nyM7OrnLbVH0Y8kS10AcjP8Hv40xx/ScgbTVwcRrg8i5wfrwMqgIBFxcXFOUXQy6TQ+QbwHU44Lu2\nEJOmfoKff/4Zhw4dgkwmg7W1NTIyMrB161bk5+f/a78bN27E559//tjnUlJSAHMlnIYCMj3AoR9g\n5iLD77//ruWzJ21iyBPVQoNeH4QfFqyF+cZOSI0wh20rA2QeBVK+FfD08MTXX3+NiIgIiFxDXP9J\ngyuzjGDSCMjPy8eePXsghMDly5dhbGwMExMTHD9+HG5ubk8N+uLiYkyaNAlLlixBamrqI8/Xq1cP\neTeVKLpVuq28B9xPVaJ+/fpVOteVK1ciKyurSm3QU2hlhX4F6ahboudSdna2eHfkW8LK2lLMmjVL\nGBsbi6CgIKHRaIS/v78YM2aMMLMyEQ3bGYsPPhlXdtw777wjpkyZIgYMGCBMTU2Fo6OjmDNnzhP7\nWbp0qejVq5eYPHmyGDFixGNfM2X6l8LGxUx4jjARdd3NxEefTajSuV24cEHIZDIxYULV2nlRVCY7\n+cYrUS138OBBTJz8EfQ1hjh18hTs7OzwzTffoH379lAoFJg/fz66deuGkzEncTL6JORyOVJSUhAU\nFISUlBRkZGQgKCgIrVu3RkJCAq5cuQIzM7NyfRQXF6NJkybYtm0b3Nzc4OHhgdOnT8PV1fWReo4f\nP46kpCR4enri5ZdfrtK5DRo0CPb29lizZg2SkpLQsGHDKrUndVwnTyQxhw8fxqtv9oGeSo4pX0zF\nkSNHYG5ujk8++QR6enowMjKCsbExbG1tkZiQiFu3bqFRo0aYNWsWHBwcMG/ePMhkMpibmyMmJgaB\ngYEYO3Ys+vXrhy5duqBOnToAgNWrV0OtViMmJgYxMTFo7NYYHV5uh48/mogJ4yaUuxdNhw4d0KFD\nhyqfW2JiIhQKBa5cuQIA+OabbzB//vwqt0vlcSRPVIv1GdQL5+T7cGenKUxs5NAzFlDekUFADY1K\noF3b9rC2tgZQ+u9q+vTpcHNzw65duzBlyhSkpKTAzMwM+vr6yM3NhampKXLzc2DpL6C5Vgcz/jsL\nenp6+OuvvzB3/hzomwkU3iuBqTNg1kwF5R9GmDD0C0ya+B+tn9vfo/jx48fj5s2b6N69O/744w+O\n5p+CtzUgkpi+r/dC1MkoLP1mBUaOHQ63KYXI2Ad02Af8FQmkhNdFRnrWI3d93Lx5M4QQSE1NRUpK\nCoyMjPBbbDQy612Ax0SgfhfgeIgM+eeMYN9bH6k7CuAaLnBrPwAZUL9z6bp8u1BAc7gRriZd1/q5\nBQQE4Pbt2+X2LV++HN26ddN6X1LB6RoiiWnexB/pl//C4MGDcfpsHL77cilabi79BKp9KHB60H3k\n5ubCwsKi7JiEhAQMGTIE5vWMoVe/GPnpAhb1TaDM0aDZf0oDXl0IFJw3hrpQwC0iH7avAucmANAA\n3S8B+kaAx8fAfg/ApXH1LMKLi4urlnapPC6hJKql1Go1tm3bhrZt22HTpk1wcnCGMl8Fowalz2fs\nAyysLWBubg4A2LJlC65cuYIR743AsLBh0KgFOh7TwOMzAcOmBbDprETCZ6UBf2WJDEGt2mP8Bx/g\nykxjWLfI9jJiAAAMu0lEQVQEim6VfuBK36i0fWN7QGYAjBk+QTcXgLSC0zVEtZRSqUR4eDgKCgrK\n9l26fAnJKYmwdjNB/nUNIrfvRVpaGvz8/NC6dWt07NgRMTExSE9PR/iH7+Oc/RbYvapC7FDA+0vg\n7DigJAcw1jdB9IloODs7w6WJI+p2UyJjv0CJSo3W60qna1IWAbeWWeFW+h3eQriWqNHpmq1btyIi\nIgLJycmIi4tDixYtyp5btGgRFi9eDLlcju+//77c3fOI6N+pVCps27YNnp6eCAoKKreaJT09HTdv\n3oSnpyfu3LmD4OBg+Pv744033sDPP/+M/v37Q61W48Pwj9ExZAfyM1Ww9AGurdZDg64a2PjLkLnA\nAkePHgUANHHxwt24O7BvpIH1h9eRNBmITQNMnYBXOndjwD/nKj2ST05Ohp6eHkaNGoV58+aVhXxm\nZiY6duyIgwcPIjU1FR9++CHi4+PLd8qRPNETqdVq9OrfHUl3TsEiUImb2+SYNulrjAt/9Ha/YWFh\nMDExwerVq3Hs2DFERUVh+vTpKCkpgVqthkZoIAxKYGxmDCsrK5i9koX6PdW4vV+Ou7/poX5RE3QJ\nDoG/vz8W//A/GIw7j0YDS9tOGGeAARYf4+sZs2r4CtCT1OhI3svL67H7Y2Ji0KNHDzg5OcHJyQlC\nCOTm5pbNGxLR0x05cgQJ6TFoezofegaAy7gSTGw+EaNHhpe7ve+VK1ewe/dupKSkwNjYGIsWLcKk\nSZMwffp0/Pbbb6hfvz4KCgqg0Wggl8thZGSEoI6tkH4hF5ABhll1cOD4wbIli87Ozgh9rRfuxRTh\nXqwM+cl6uP1qFjIyMmBnZ6ery0FVpPXVNbGxsfD29i7b9vT0RGxsLLp06VLudREREWV/Dg4ORnBw\nsLZLIXou3blzB3WayKD34F+nqQugEWq8OqQvDh84CjMLU7zzRhhuZ93G66+/jtzcXAwdOhRt27ZF\nZORuqNUahPTsjF1bd6Ndu3bl2k6MT8bhw4ehUPyKjUnr4Obpil6hPfHj8nXo1KkToo+ewoSPxyEz\n6yS8Zilx4sJatGq7FxfO/F62Hp9qjkKhgEKhqFIbTw35rl27IiMj45H9M2fORGho6GOPedyvEo/7\n5vZ/hjwRPdS2bVtkjtcgYz9Qtw2QMlcf5nXNkCQ/gi5Xi3H9pyLMHTMXjRs3hlKpRGRkJABAbiiH\nZa88+H+nQeavQHD3DjA0MELP3j2wetlamJubw9zcHLa2tlizbSVa7S6AqQtwduw+jBw3HBtX/4Tm\nzZvjVEwMgi8oYeoEACrEp+dg586dCAsL0+VleSH9/wHw31/8UhFPDflDhw5VuMHAwEAcPny4bDs5\nORkBAQEVbofoReXs7IxdWyIx7P23cPrPLLRq0wI3ihPh9U0xDG2A25vrYPDgV/Dbb78hPT0dMpkM\nmZmZaOzthIANGshkpbcBvtpBwHFwEc4d2oew0e9g6/odAIBDhw/BPqwQ1i1L+/OcVYyDbQ+W9a8q\nUcOgzsN69M0FlEplTV4C0iKtvG3+z9F769atceDAAaSnp0OhUEBPT4/z8UQV9PLLLyPt0g0U5hXj\n+KGTsLG1Rm4ykPUroHfLEuvWrYO5uTl+/vlnAIC5uTlURRoU/ll6vEYJFFwDzFwB73nFOLDvYYjX\ntamLomSjsu3cZMCmrhWA0t+6h749BGffMEHWceDqUhmy9hmgd+/eNXfypF2VveXljh07RKNGjYSx\nsbFo0KCB6NGjR9lzCxYsEG5ubsLb21tERUU9cmwVuiV6IUVGRoo6tsairqu5WLdunRBCiF27dgk/\nPz+h0WiEEELMmvO1sHExFZ4T9IWVH4TDaxCvqiE6RUE4uNmVtZWTkyM8X3IXLqGmwmu8obCoZyr2\n7dtX9rxSqRSfTv5E+LZpKrr16SwSEhJq9mTpiSqTnfwwFNFz4uDBg+jZsydkMlnZvyEhBC5evAgP\nDw8AwNGjRxEdHY3lq5bCwOc+TD1K8Oc6OX5ctgH9+/cvaysvLw+bNm1CTk4OunXrBh8fH12dFlUA\nb1BGJHEqlarcvx2ZTFZuWeXf8vPzsX79emRnZyMkJAQtW7asyTKpmjDkiYgkrDLZyc8rExFJGEOe\niB5r3bp1+OGHH3RdBlURp2uI6BH5+flwc3ODWq1GSkoKLC0tdV0SgdM1RKQly5YtQ4cOHdCzZ08s\nXrxY1+VQFXAkT0Tl/D2KP3ToEIyMjNCuXTuO5msJjuSJqMqWLVsGa2trJCUlIT4+Hg0bNuRo/jnG\n73glonJsbGzg6+uLXbt2QQiBO3fuIDExUddlUSVxuoaInujAgQMYPnw41Go1rly5AlNTU12X9ELj\ndA0RaY0QAhEREZg3bx7atm2L5cuX67okqgRO1xDRYx08eBA5OTl47bXX4OXlhR49emDUqFEczT9n\nGPJE9FizZs2CgYFB2ZeFFBQUYM2aNRg9erSOK6OK4Jw8ET3W0aNHcf369XL72rdvDzc3Nx1VRLxB\nGRGRhPGNVyIiKochT0QkYQx5IiIJY8gTEUkYQ56ISMIY8kREEsaQJyKSMIY8EZGEMeSJiCSMIU9E\nJGEMeSIiCWPIExFJGEOeiEjCGPJERBLGkCcikjCGPBGRhDHkiYgkjCFPRCRhDHkiIgljyBMRSRhD\nXscUCoWuS6g1eC0e4rV4iNeiaiod8lu3bkWzZs2gr6+P+Pj4sv1paWkwMTGBv78//P39ER4erpVC\npYo/wA/xWjzEa/EQr0XVGFT2QB8fH+zcuROjRo165Dl3d3ecPXu2SoUREVHVVTrkvby8tFkHERFV\nB1FFwcHB4syZM2XbqampwszMTPj6+oqRI0eKc+fOPXIMAD744IMPPirxqKinjuS7du2KjIyMR/bP\nnDkToaGhjz3G3t4e169fh7W1Nfbt24e33noLCQkJ5V5TmvNERFTdnhryhw4dqnCDhoaGMDQ0BAD0\n7NkTkydPRkpKCtzd3StXIRERVZpWllD+c2R++/ZtqNVqAEB8fDwKCwsZ8EREOlLpkN+5cyccHR1x\n6tQp9O7dGz179gQAHDt2DL6+vvDz88PMmTOxfPlyrRVLREQVVIX3XCtsy5YtomnTpkJPT6/cm7VC\nCLFw4ULh7u4uvL29xfHjx2uyrFphypQpwsHBQfj5+Qk/Pz+xb98+XZdUo44dOya8vLyEu7u7WLRo\nka7L0TlnZ2fh4+Mj/Pz8REBAgK7LqTHDhg0T9evXF82bNy/bl5OTI/r06SMcHR1F3759RW5urg4r\nrDmPuxaVyYkaDfmLFy+KS5cuPbIi59atW8LT01Ncu3ZNKBQK4e/vX5Nl1QoRERFi3rx5ui5DZ/z8\n/MSxY8dEWlqa8PT0FFlZWbouSadcXFzEnTt3dF1GjYuKihLx8fHlgm327Nli7NixoqioSIwZM0bM\nmTNHhxXWnMddi8rkRI3e1sDLywseHh6P7I+JiUGPHj3g5OSETp06QQiB3NzcmiytVhAv6Kqj+/fv\nAwA6duwIZ2dndOvWDTExMTquSvdexJ+HDh06wNrauty+2NhYDB8+HEZGRggLC3thfjYedy2Aiv9c\n1Ip718TGxsLb27ts29PTE7GxsTqsSDcWL16MoKAgzJ49+4X6Ty4uLq7ch+uaNm2KU6dO6bAi3ZPJ\nZOjcuTP69euH3bt367ocnfrnz4eXl9cLmQ3/VNGc0HrId+3aFT4+Po88IiMjn3jM4/5nkslk2i5N\n5550bXbv3o3Ro0cjNTUVBw4cwJUrV/iG9QsuOjoa58+fx9dff42PPvrosZ9XeVG8iL/RPEllcqLS\ntzV4ksqsrQ8MDMThw4fLtpOTkxEQEKDNsmqFZ7k2lpaWGDNmDMLDw/HJJ5/UQFW6FxAQgIkTJ5Zt\nJyUloUePHjqsSPcaNmwIAPD29kafPn0QGRmJ9957T8dV6UZAQAAuXrwIf39/XLx4UZLZ8Kzq168P\noGI5obPpmn/+79y6dWscOHAA6enpUCgU0NPTg7m5ua5K04mbN28CAFQqFTZu3IhevXrpuKKaY2lp\nCQCIiopCWloaDh06hMDAQB1XpTsFBQVlv4ZnZWXhwIEDL/R/eoGBgVi1ahUKCwuxatUqBAUF6bok\nnalUTmjtreBnsGPHDtGoUSNhbGwsGjRoIHr06FH23IIFC4Sbm5vw9vYWUVFRNVlWrfDWW28JHx8f\n0bJlS/Hhhx++cCsrFAqF8PLyEm5ubmLhwoW6Lkenrl69Knx9fYWvr6/o3LmzWLlypa5LqjGDBw8W\nDRs2FIaGhqJRo0Zi1apVL+wSyr+vhVwuF40aNRIrV66sVE7IhOCEFxGRVNWK1TVERFQ9GPJERBLG\nkCcikjCGPBGRhDHkiYgkjCFPRCRh/wec8kSBPkpGngAAAABJRU5ErkJggg==\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.prism()\n",
    "plt.scatter(X_train[:, 0], X_train[:, 1], c=y_train)\n",
    "plt.scatter(X_test[:, 0], X_test[:, 1], c='white', marker='^')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That doesn't look too hard.\n",
    "\n",
    "Now let's fit the logistic regression model to the training data, to see if we can do it automatically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, penalty='l2', random_state=None, tol=0.0001)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logreg.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we have a look at how logistic regression did on the training set.\n",
    "\n",
    "We do this by now setting colors using the predicted class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADtCAYAAAAcNaZ2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH89JREFUeJzt3Wd8VNXaxuH/JJM2SYQECISiEJCOCBh6NyDFQ1Egdg4I\nHkSxHAUEfVXUY0GKimBBRLCCiqBYwAJSE0JvgnQBaQktvc16P2xEFCGUzJ6U+/pEMjN7P5sfudlZ\ne631OIwxBhERsYWPtwsQESlOFLoiIjZS6IqI2EihKyJiI4WuiIiNnOd6weFw2FmHiEiRcb5JYecM\n3bw+KCIiZ8vrhlXDCyIiNlLoiojYSKErImIjha6IiI0UuiIiNlLoiojYSKErImIjha6IiI0UuiIi\nNipSoZubm0tycrK3yxAROaciE7pT3n6bksHBlA0PJ7p2bfbu3evtkkREzuI4V7seh8NRaPZeSEhI\noHvbtixIS6M68JyvL/Pr1WPxmjXeLk1Eipm8srNI3OnGx8fT3e2mBuAAhuXmsnz9+kLzn4aIFB9F\nInTLly/PSqeTrFNfxwGRYWHanlJECpwiEbo9evTgqlatuC4khNjQUHq5XEz+4ANvlyUicpYiMaYL\n4Ha7+fHHHzly5AhNmjShatWq3i5JRIqhvLKzyISuiEhBUCwepImIFBYKXRERGyl0RURspNC9CAsX\nLqRNgwY0qFqVJx97jJycHG+XJCKFjEL3Aq1bt47eXbty39q1vLVzJ4smTGDkI49w/PhxenfpQpjL\nxdXlyzN37lxvlyoiBZhmL5whNTWVadOmkZSYSPvrr6dFixanXxv19NOkP/ssL7rdAGwDrg8PJ6pa\nNRITEqhpDNHAC76+vPXhh8TGxnrnIkTEqzR74QKlp6fTJjqa+Y8+SvqoUfTp2JEP3n+fjIwMNmzY\nQGZWFkm+vqffnwTg40PCihUMNIbOwGtA49xc7r7rLtauXeulKxGRgkx3uqdMnTqVmUOG8E1qKg5g\nNdAlNBSXy0VAWhqHs7JwAHdkZ1PF7Wacy0W1+vXpvHw5j546xmxgCNAUiLj7bia+846XrkZEvEV3\nuhfo+PHjVM3O5o/dGqoCx1JSeOTwYX5JTmZHZiZlfX3Z1707W/v1483PPqNK5cr4n3GMAMANXAfk\n6iGbiPwDp7cLKChiYmJ43teXHkBtYERAAIHZ2dx6agy3JNA1M5Ow6GhGjBgBQEhICL1mz6ZUejoh\nwP1AZ2BsUBCz77nHOxciIgWa7nRPqVevHtM//5wHKlXi2pAQcjp1olq1asw+9XoK8H1gIDVq1GDh\nwoW0vvZahvTtS8yNN/Jxq1aMiooiLCqKQ61b88ncuTRv3vyCzpuZmckTw4YREx1Nv9hY9u3b57Fr\nFBHv05jueWzYsIEu7dpRLieH/VlZdO/Th0EPP0xM8+ZMSkujCjDM5aJh//6MmTDhks5xa/fupH7/\nPfelp7PE15ePypRhzdatXHHFFfl7MSJiC214c5mSk5PZuHEj4eHh1KhRg2efeYbkUaMYfWrYYQfQ\nJiyMfUePApCSksLPP/+MMYa2bdsSEhJyzmOnpKQQERbG0ZwcAk99r2NoKPe9/z7du3f38JWJiCfk\nlZ0a081DaGgozZo1O/11YFAQu51OyLK2TE8EggICADh06BBtoqMplZiIOzeXwSEhxG/cSGRk5D8e\n2+FwYIAzH7llAz4+GvWRgiU7G/z8vF1F0aCf7ot01113MT80lOsdDmoANzocdLrpJgCeHj6clnv3\n0jo9nfpZWdQ/epSOzZqd83+94OBgbuvdm+4uFzOBB/z8OBQeTvv27e27IJHz2LkT7r8f6tQBTcjJ\nHwrdPCxZsoR7//1vHvjPf9i0aRNly5blwaFD2e3ryxvAu8Yw6733mDdvHpvWruUbIA2oC6wHftuz\nhy1btpzz+G9Nn84Njz/OjJgYHP37s2jlSoKDg+25OJFzWLUKbrkFGjeGK66ARYvAqd+L84XGdM9j\n3rx53NWzJ4+lp5PqcPCqy8VPy5cz5K67eGztWjqdet+bQFyvXuzYt4+KcXF8fOr7a4HWwPdxcTRp\n0iTP88XFxfHDDz8QFhZG3759zzseLJLfjIH582H0aPj1V3j4YRg4EEJDvV1Z4aIx3csw9qmnmJCe\nTh8AY/BNS2Pi2LEEBAZy4oz3HXc4CAgKokPnzhyKizv9/XDA7XBQt27dPM81c8YMHuzfn74ZGawO\nCGDyq6+ydM0a3fWKx2Vnw4wZ8PLL4HbDsGEQGwv+/nl/Vi5egRxeWLp0KbFdu3JTTAyzZ8/O+wMe\nkpmRQYkzvi5pDFnp6TwyahQPBAUxHngceN7fn7rR0XTr1o0ZQUF8CKwAbnU4qN+wIVOmTOG33347\n77mGDxnCrLQ0XnS7+Tw9nfK7djFt2jTPXZwUeykp8MorULUqvPsuvPgirF8Pd96pwPUocw7necmj\n4uLiTOmgIPMWmA/BVHK5zIxPPvFKLW9NmmRqulzmRzBfgokMCjLz5s0zxhizePFic9tNN5nwwEDT\nITjYtAwNNfWvvtqMHDHChDkcprTDYYLB3OJ0mgEBASYiNNRs3LjxnOcqERhoDlm/4RkD5gEwzaKj\n7bpUKUYOHjRm5EhjSpUypndvY1as8HZFRUte2VngQnfgHXeYcWeEz5dg2jVs6JVa3G63eWPiRNOs\ndm3T6pprzKxZs/7yep+uXc3zPj7GgHGD6e/vb0J8fc1aMIPAjDzjOl5xOEyvTp3Oea6m9eqZ3mD2\ng1kApgyYkIAAT1+iFCNbtxpzzz3GhIUZM3iwMdu3e7uioimv7CxwwwvGmNObzgA4Tn3PGxwOB4MG\nD2bZpk0sWreOnj17/uX1PTt20PLUIgkH0CoriyscDuoDR7H2cPhDLWNIOnz4nOfq068f63x8uBb4\nDzAKCDhjK0mRSxUXBzffDC1bQrlysHUrTJxoDSuI/Qpc6N59//0873LxDvAJcJ/LxX+GDvV2WWdJ\nTU3lwOHDjAGygBPA26cesK0BOgIvYq1Y2wc843LRsUePcx7v9ttv56DDwW3AUGA84Ha7SUpK8vCV\nSFHkdsPcudC6Ndx6K7RrB7t2wahRUKaMt6sr3gpc6DZt2pTPv/uO+Z068Um7doyfPp1bbr3V22Wd\n5ZWxY7kuJQUfIAyIAJLCwpj6wQfEuFxMCA1lt9NJAz8/qjsc/JKby97du8nMzPzH4+Xk5OB0OvEF\nlgJPAw38/FixYoVdlyRFQGYmTJ0K9erBk0/CvffCtm3WAgdNhCkgLnVcorgbeMcdZtKp8doTYJaA\nqV+lijHGmMTERJOQkGDmzJljyrtcZjGYnWA6BwWZhwYN+sfjHTt2zAT7+ZnEU8fMBlM7JMQsWrTI\nzsuSQur4cWNGjzamfHljOnQw5vvvjXG7vV1V8ZRXdha4O93CIrpVK6a5XJwAXMC7/v5En9rOsVSp\nUlx33XUs/fln7k1LoyVQBRiXns5XX3zxj8crWbIk9w0eTPvgYP4HdHG5uKpRowveIlKKp99/h+HD\nISoK1q61hhTmz4eYGHA48v682E+LIy7R3QMGsD4hgfLTpuHncBDdqBGfTZz4l/eUCA9np5+fNfsc\n2AWUOM+WjS+OH0+j5s1ZtWIFPatWZcCAAfjqYZr8g82bYcwYmD3bmle7ahVUruztquRCaBnwZUpO\nTiYrK4vw8HAcf7u1SExMpMk119Di6FEqZGczJSCA9z77jC5dunipWinMjIGlS61luvHxMGSINWZb\nqpS3K5MzaT9dL0tKSmLatGmkJCfT9cYbadSokbdLkkLG7YYvv7TC9vBhePRR6NsXgoK8XZn8E4Wu\nSCGVkQHvv28NI5QoYe2J0LMnaMSpYNOGNwVYdnY2ftoZWv7m2DF480147TVo2BDeftuab6sHY0WD\nZi94werVq6lRsSKBAQFUjYwkPj7e2yVJAbB3L/z3v9ZKsV9+sWYhfP01tGmjwC1KFLo2S0tL418x\nMYzav59sYxhz8CA9briBEydO5P1hKZL+2Nmrfn3w8YF162D6dGuBgxQ9Cl2bbdu2jbDcXG7B+svv\nCVQCNm/e7N3CxFbGwIIF0Lkz3HAD1K5ttcYZMwYqVfJ2deJJGtO1WUREBL9nZXEYa+nwUWB3VhZl\ny5b1cmVih9xcmDXLmomQnAxDh1pzbU/1NpViQKFrs8jISP47dChNxo2jvTH87HAwcNAgoqKivF2a\neFBaGrz3Howda+309fjj0K2bNZwgxYumjHnJ0qVL2bRpEzVq1KBNmzbeLkc8JCnJ2kZx4kRo1sy6\ns23RwttViSdpnq6IF+zaBePHwwcfWHNrH30UatXydlVih7yyU7/ciOSj1aut/Wujo62tFDdtgilT\nFLjyJ4WuyGX6o3V5hw7WOO1111kzEV54ASIjvV2dFDR6kCZyibKzYeZMq3V5To61TPeWW9RJV85P\noStykVJSrCGDceOgShV4/nlrvq1WjcmFUOiKXKBDh2DCBHjrLWjbFj79FBo39nZVUthoTFckD9u2\nwaBBULMmJCbC8uUKXLl0Cl2Rc4iPh169oHlziIiwWpe/+SZUq+btyqQw0/CCyBncbvj2W2uZ7p49\n1q5f770HISHerkyKCoWuCJCVBR99ZG0443RaK8f69AFtdyz5TaErxdrJk9Ym4a++ao3Zjh+vTrri\nWQpdKZYOHLCCdvJk6NgR5syxujSIeJoepEmxsmULDBgAdepYO3+tXAkff6zAFfvoTleKhaVLrZVj\ny5fDfffBr79C6dLerkqKI4WuFFluN3z1lTUT4eBBeOQR62GZy+XtyqQ4U+hKkZOZaW2p+PLL1lSv\n4cPhppvUulwKBoWuFBnHj//Zurx+fXjjDWu5rmYiSEGiB2lS6O3daw0dREXB5s3w3XfWAod27RS4\nUvAodKXQ2rAB+va17mqNgbVrrdbl11zj7cpEzk2hK4WKMbBwIXTpYs2vrVEDduywtlm88kpvVyeS\nN43pSqGQmwtffGHNRDhxwuo5NmsWBAZ6uzKRi6PQlQItPf3P1uURETBiBHTvrtblUngpdKVASkqC\nSZPg9dehaVOYNk2ty6Vo0P2CFCi7d8MDD8DVV1t/XrjQ2hdBgStFhUJXCoQ1a+C226BRI2vF2MaN\nal0uRZNCV7zGGPjhB2sWwr/+BQ0awK5d8OKLUL68t6sT8QyN6YrtcnKsHmOjR1ubhw8dat3lqnW5\nFAcKXbFNaqo1ZDB+vDWn9rnnrNblmokgxYlCVzzu8GFrFsIbb0Dr1tb+tU2bersqEe/QPYZ4zPbt\nMHiwtWrs0CFrT9vPP1fgSvGm0JV8l5AAvXtDs2YQHm51a3jrLahe3duViXifhhckXxhj7e41ejTs\n3AkPPwxTp6p1ucjfKXTlsmRlwSefWBuG+/jAsGFqXS5yPgpduSTJyVYn3fHjrTHbsWOhQwftXyuS\nF4WuXJQDB6zODJMnQ0wMzJ5trSITkQujB2lyQbZuhYEDrdblKSmwYoU1rKDAFbk4utOV81q2zHo4\ntmyZWpeL5AeFrpzF7Ya5c62w/f13tS4XyU8KXTktMxM+/NCaiRAcbM1EuOkmcOpfiUi+0Y+TcOKE\ntXjh1Vehbl2YOFGddEU8RaFbjO3bZwXtu+9aG898843VWVdEPEezF4qhTZugXz+rVXlODqxeDR98\noMAVsYPudIsJY2DxYuvh2KpVMGSItSFNeLi3KxMpXhS6RVxurtVjbPRoOHrUal3+2WdqXS7iLQrd\nIio9HaZPt5bnhofD8OHQrRv4+nq7MpHiTaFbxBw9am0WPmECREdbnRpattRMBJGCQg/Siog9e+Ch\nh6BaNdixA376Cb76Clq1UuCKFCQK3UJu3Tq44w5o2NBq7LhhgzUFrHZtb1cmIv9EoVsIGQM//gg3\n3ABdulhTv3butB6WVajg7epE5Hw0pluI5ORYMw9eftl6UPboo/DllxAQ4O3KRORCOYwx5h9fcDg4\nx0tis9RUq/XN2LFQsaK1J0LXrmpdLlIQ5ZWdutMtwI4csfZBmDTJmoHw0UdWs0cRKbx0r1QA7dxp\n7V1bo4a1teLixTBrlgJXpChQ6BYgq1bBLbdA48ZQogRs3gxvv22Fr4gUDRpe8DJjYP58a+bBtm1W\n6/LJkyE01NuViYgnKHS9JDsbZsywZiIYYz0ci41V63KRok6ha7OUFHjnHRg3Dq6+Gl56yZpvq1Vj\nIsWDQtcmBw9a+yG8/bbVlWHWLLjuOm9XJSJ204M0D9u6Fe65x1qWe/w4xMXBzJkKXJHiSne6HhIX\nZz0cW7IEBg+2wrdMGW9XJSLeptDNR243fP219XBs716rdfn771uddUVEQKGbL/5oXT5mjNWRYdgw\n6NVLrctF5GyKhcvw99blEyZA+/aaiSAi56bQvQT791tBO2UKdOoEc+dCgwberkpECgPNXrgIf7Qu\nr1cPsrKs1uUffqjAFZELpzvdPBhjzUAYPRoSEtS6XEQuj0L3HM5sXZ6YCEOHWvNrg4K8XZmIFGYK\n3b/JyLBal48ZAyVLWq3Le/RQ63IRyR8K3VPObF3eqJG101fr1pqJICL5q9g/SPvtN2s7xWrVrK0V\nf/jBWuDQpo0CV0TyX7EN3fXr4c47rZkHTqf19XvvWfNtRUQ8pViFrjHw00/W3NpOnaBOHdixw1q2\nW7Git6sTkeKgWIzp5uTA559bMxHS0qzW5XPmqHW5iNivSLdgT0v7s3V5ZKQ1E+HGG9W6XEQ8p1i2\nYE9MtFqXT5wIzZtbO321aOHtqkREitiY7q5d1oqx6tVh3z5YtAhmz1bgikjBUSRC94/W5dHREBJi\n7ZEweTLUrOntykRE/qrQhq4xMG8exMRYK8aio6073RdesMZvRUQKokI3ppudbe2B8PLL1v4IQ4da\nd7n+/t6uTEQkb4UmdFNSrP1rx42DqCjrjrZTJ60aE5HCpcCH7qFD8Prr8Oab0LYtfPopNG7s7apE\nRC5NgR3T3bkTBg2yHoYlJcHy5QpcESn8Cuyd7i+/QESE1bo8IsLb1YiI5I8ivSJNRMRueWVngR1e\nEBEpihS6IiI2UuiKiNhIoSsiYiOFroiIjRS6IiI2UuiKiNhIoSsiYiOFroiIjRS6IiI2UuiKiNhI\noStiI+1nIgpdERskJyfTrXdXAoL8CYu4gslTJnu7JPESha6IDQbc14/NQT/SNTGH6J+SGfb0Qyxc\nuBCAAwcO0OaGFriuCKRyzUqnv5+fjhw5wpQpU5gyZQqHDx/O9+PLhdPWjiI2KF2+JE3iTuC60vp6\n85MObnY8wTOjnqFh8/pkttlMtWE5JC2HDX2DWZewkcqVK+fLuffs2UPjlo0IbZ4ODjixOID4xSuJ\niorKl+PLX2lrR5ECoFSZcE5stP5sDKRtDKBM6TIkJyezae1maj2fg38YRHaBiHY+LFu2LN/O/cSz\nIynT/xgNZqTR4JM0Iu89wchRw/Pt+HJxFLoiNpg49m3W93WxYVAgKzsHE7ynMv379ycoKAiHw4e0\n3db73DmQss0QHh6eb+c+cHg/V9R3n/46tL6bg0d+z7fjy8VR6IrYICYmhvhFK7mvzmieuW0SCUtW\nExwcjNPpZPTo0Sxv42LTI07i2wVTt0I0HTp0yLdzd2rXlT1jXGQchswjsGe0i07tbsy348vF0Ziu\nSAGwaNEi4uLiqFChArGxsTid+de+0O128/CwB3nrjbcxxjDgnrt5bezr+Pr65ts55E95ZadCV6SY\n+OPn2eFweLmSoi2v7Cyw3YBFJH8pbAsGjemKXKLc3FxefPkFOnRvS7//3MX+/fu9XZIUAgpdkUt0\n7wP38Prc50jt+zMrSn1MrWuvpsctN/LOu++c/vVyz549rFu3joyMDFtrM8aQk5Nj6znlwmhMV+QS\n5OTkEBQcSNfDufiVsL63qCMEV4G0FS763Xg/J5JP8P6H0/AP9SH7JPS9rT/Dhg6jUqVKHq1t+vvT\nuf+hwaSdTKdpm8bM+mgOERERHj2n/EkP0kQ84HToHsnF7wrre8t6QMXeUKY9zK/qJKyaP9WeS2PV\nAIgaBFlH4fiskqxctibfVpv9XUJCAjHd2tBkfjqhNeGXx5yU3dyUBd8u9sj55GxakSbiAU6nkzv+\nfTurero4MBc2PgHH10C5zuAMBneOm/Au6WwbBw0mQp1noMHrUObOk7zy+niP1bVkyRLK986lRD3w\n8YMao3JY9nO8x84nF0+hK3KJJk+cwsDrh5E7vhG7Jzmp0h9ObIK1twbRvHVTjs93kZ0MgRX+/Exg\nRTcnU457rKayZcuSss4Pc2oB2rHVEB5R0mPnk4un0BW5RE6nkydHPsXyH1eyeul6Kq2N4fjw2vSu\nfy/ff/0TnaJvImOXk7WD4fh6OLIIdo920afHrR6rqXfv3lTxv4a4liFsuNvFml4uJr/+rsfOJxdP\nY7oil8ntdjP+tXF8Nf8LIkqV5bn/e5Hq1asDsGXLFkaPe4nvF35HYGAATw1/jjtuv8Oj9eTk5DBn\nzhySkpJo1aoVtWrV8uj55K/0IE3Ew4Y/PpT3f5hElcfTSNniYO+4UL6Y8SXh4eHUqlXropf07t69\nm/j4eCIiImjbtu1fFjVs376dzZs3k7AqgfmLviYoMIgnhz5L+/bt8/uy5BIpdEU87IpSIbRcnUrw\nVdbXCXc6OPyVk+AyAZQrUZEF3y2mdOnSF3Ss+fPn0+v2noQ1zeXkFkPNCvW5p++9BAQEcPR4EiOf\nHo6zlJvUI5lUuBlCrrY2sJn/1U80adLEg1cpF0qhK+JhJUqH0GJlKsGVra/jb4fwaKj2IGx6yI8G\nJ3vy0dQZZ30uOTmZVatWkZ2dja+vL5GRkbTv3Jrq7yYS0R5yM+GH+hAY5iQ4PIB9P6fS9HOIj7Wm\npQVFwt6ZUOFm6BQwiEmvvmHvhcs/0t4LIh523+AhTO31GpVHpJG8GQ7Ng2teBocDysVms/6htWd9\nZseOHbSOaY6JSOX4vlQcxhen8SPleAZNm1vv8Q2AUq2gZP0cqt2fg8+98MuzENkdGk+z3lO6NWx6\nAvy6+dl4xXI5NHtB5DL9b9TzjOz3AiHT2+L3dV3KXBtAQITVIeLQ537UrXXNWZ8ZcP+/KXN/Ii3i\nU+m8G0KvyeWqhzMILA1rBlmfTdkOB+ZCWLT1mfCmkL4PSp5xuNBakHnEwb0DB9tyrXL5NLwgko8y\nMzPp0vMG1v6yEmeQg3D/cvw8f+lZy3CvrB5JrTkHueLUxIJfx0LaXshOdPD7NwaTA+4scFWBmATI\nSYO4jgEc+zUT/1LQ8lsILAvxt0Gzkjcy+9OvvHC18k80vCDiIUlJScycOZPMzEy6detGVFQUAQEB\nfD/3JzZv3kxWVhZ169bF39//rM9eW78B26d+T+2XcshJgX2fQsVY2PqRockMCL8OHH6woBl8Ge7A\n6evLgHv68d6eqVQfkcniDlYQu8o4ue35O71w9XKpdKcrcgkOHjxIo2bXEtT0JM4Sbg7O8uPHbxfS\nqFGjC/r8oUOHiOnalt8O7Cb1eAYBwU7cGT4EuvypNz2Fsh3BnQ1xrYIZ8+BkYmNjycjIIKx0STru\nzCawnNVPbWmDED5+7UvatWvn2QuWC6Y7XREPGD3uJUK7J1HvFWv7xNDoTB554kEWfrvkgj5ftmxZ\n1sZvZM+ePTidTlJTUyldujTr16/nplu6EdHGl+RfDY2qtaBPnz74+PjgcrkY+fhIJrQeQ+meqRz4\n3I9gAolPiKNly5b4+elhWmGgB2kil+Bw0kGCa/25X21ILdi/bz/XNq1LUEgAdRrVYP369Wd9bsGC\nBVSuURFXaCDtu7QmKCiIK6+8klq1alGmTBmuv/561q/czNM936BJ9TYsWrCI0pFhvDD6eYwxPPX4\n00wd+zHpsytQ8hqo8EQib/z0LDff1kO/mRYSCl2RS/CvG3rw23gXyVsh4yBsfyKQw4cO43vXZm7Y\nn0XwA78S06UdJ0+ePP2Z3bt306PPv6j0yn467M3kcP3lVK5VkdLlS/K/F587HZpXXXUVv2zfzOqj\nC2izKZ0mS5IZ/97/+PCjDwEoV64c6Y4TRH+aTeV/Q6M56Sz8+Sf27t3rjb8KuUgKXZFLENsnlqED\nnyS+dSgLawXRtGxnAkv5EjXY4FcCruoL/uVz2Lhx4+nPLFu2jIh2PpTrDP4loc5LhpxMN9fOPcGr\nH7zA1GlTT7/36/lzqPpUGkHlIbQ6XPlIGl/Nnw1AdnY2Tpfj9E+vjx/4BviQlZVl69+BXBqFrsgl\nGvbIcI4dOknysTReGTOB1ENZZB2zXstJgZR92YSHh59+f3h4OKnbDe5ToxKpu8DhAyXrQ5URacz+\n9rPT7y0VXprkLX+eK22Lk7KlygHQsGFDgrNK8csIJ4lLYeNgf6peeTVRUVEev2a5fApdkXxQoUIF\n7hk4iOUtgtn0qJPlLUPo1SOWmjVrnn5Phw4dqFM+mvh2wax9wMHClqdWrvlC6q8+lA77cy7vy8+M\nZ8f/hbBhUABrbg/k2MwwHnt0JACBgYEs/mE5dfZ3I+mRmjRx9+H7uQvw8dGPc2GgKWMi+cQYwzff\nfMPGjRupUaMG3bt3P6vteU5ODp988gmrVq1i8tS3qHBrLibTwdHvgs5q47Nr1y7mzJmDv78/vXv3\npkyZMjZfkVwKbXgjUkDt2rWLWbNm4evrS2xsLJGRkd4uSfKBQldExEZqTCkiUoAodEVEbKTQFRGx\nkUJXRMRGCl0RERspdEVEbKTQFRGxkUJXRMRGCl0RERspdEVEbKTQFRGxkUJXRMRGCl0RERspdEVE\nbKTQFRGxkUJXRMRGCl0RERspdEVEbKTQFRGxkUJXRMRGCl0RERspdEVEbKTQFRGxkUJXRMRGCl0R\nERspdEVEbKTQFRGxkUJXRMRGCl0RERspdEVEbKTQFRGxkUJXRMRGCl0RERspdEVEbKTQFRGxkUJX\nRMRGCl0RERs5z/eiw+Gwqw4RkWLhnKFrjLGzDhGRYkHDCyIiNlLoiojYSKErImIjha6IiI0UuiIi\nNlLoiojY6P8B89y1HqfLjHwAAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from utility import plot_decision_boundary\n",
    "y_pred_train = logreg.predict(X_train)\n",
    "plt.scatter(X_train[:, 0], X_train[:, 1], c=y_pred_train)\n",
    "plot_decision_boundary(logreg, X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That looks pretty good. It is not always that easy to inspect the result visually,\n",
    "so we can also use the logistic regression object to calculate a core for us:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set: 1.0\n"
     ]
    }
   ],
   "source": [
    "print \"Accuracy on training set:\", logreg.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.0 means an accuracy of 100%, just as it looked like.\n",
    "\n",
    "Now let us have a look at how the algorithm generalizes to the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test set: 1.0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADtCAYAAAAcNaZ2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd0VFXXx/HvTCaZzCQhhZaECBgIHQEjvYQqAoqIojy8\n+tgVQQFFilgooigtCoIUFUFFFEUQEBRRBKSXgBh6kyKE0NInycx+/7iI8FBCSWZS9mct1jJT7uw7\ny/y4nHvO2SYREZRSSrmF2dMFKKVUUaKhq5RSbqShq5RSbqShq5RSbqShq5RSbmS50hMmk8mddSil\nVKFxtUlhVwzdnN6olFLqUjldsOrwglJKuZGGrlJKuZGGrlJKuZGGrlJKuZGGrlJKuZGGrlJKuZGG\nrlJKuZGGrlJKuZGGrlJKuVGhCl2n00lycrKny1BKqSsqNKH78ZQpBPn5UTokhLrVqnHo0CFPl6SU\nUpcwXaldj8lkKjB7L6xfv557mzfn17Q0KgHDvbz4qWZNVmze7OnSlFJFTE7ZWSiudNeuXcu9LheV\nARPQ3+lk9datBeYvDaVU0VEoQjc8PJwNFguZ535eA4QFB+v2lEqpfKdQhG6nTp0o17Qpd/j781BA\nAA/Y7Uz9/HNPl6WUUpcoFGO6AC6Xi6VLl3LixAnq169PhQoVPF2SUqoIyik7C03o3oy4uDh+++UX\ner/0kqdLUUoVcBq616BNw4b8tm4dO/fs4dZbb/V0OUqpAqxIzF64GStXrmTP1q28ZDbz9uuve7oc\npVQhV+SvdNs0bEjXNWu4D4jy9WVDfLxe7Sqlbphe6V7FypUrWbthA+WAP4CYrKyrXu0uW7aMmDp1\nqFOhAm8MHEh2drbbalVKFQ5X7QZc2CUmJlKnenXevOBvpZLmy/89tGXLFrp06MCEtDTKA/3Hj2dQ\nejqDhg7l6W7d+HnZMkoEBVG/RQsmTJxIYGCge05CKVWgFPnhhQulpqYyffp0TiYm0rJVKxo3bnz+\nuaFDhpD+5pu843IBsBtoFRJCZMWKJK5fTxUR6gLDgbs6d2b2t9/mWl2fT5+Ot48PD/3nP7l2TKVU\n3tDZC9coPT2dptHRRBw4QDWHg+m+vrw7aRIPdOnC7t27+fLLLzkxejRTs7IAY9XbgyVKcDIxkbcB\nf+ANoDqwDFi6fDlNmza96bpSUlKoEB6OxcuLPUePYrPZbvqYSqm8o6F7jaZNm8bXL7zAD6mpmIBN\nQPuAAOx2O9a0NBIyMzEBD2dlcavLxVi7nYq1atFu9WpePneMucALQAPgSO3arNq8mcOHD7N161ba\nt29/Q3WNHDGCTcOHk2Ey0XL4cHr16ZMbp6uUyiNF6kaaiBA7diwZGRnX/d4zZ85QISuLf3ZrqACc\nTkmhb0IC25OT2etwUNrLi8P33svOxx9n0jffcGv58vhccAwr4ALuAP784w/OnDnDgBde4MFOnWjV\ntClZ566Sr1VKSgpjRozg9bQ0Bqem8u7QoaSnp1/3uSml8o9CFbrz58/npb59+Xjq1Ot+b+vWrfnK\ny4ufgaNAL6sVX5OJ/5z7GysI6OBwEF23LhM/+YR27drx6HPP8ZbNxhfAPOAZoB3wlsmEj9XKnDlz\nWPLjj5RxOlm1ciUzpk+/rpomjh9PaEYG+4DDQGBqKlMmTbruc1NK5R+FZnhBRIiuXJn2u3fzaXAw\ne44exdfX97qOsWjRIvo++yyJp0/TplUrdmzfTs9du3gCSAGa+vnx+owZhISE0P3cTa3opk05e+wY\nR48cIRsoFxHB7c2bM3zYMKqVL0+bv/5ihsvFDKBHiRLsPnoUb2/v85/pcDh48/XXWfPrr9wSGcmb\nY8YQEREBwLjRo1ny/fcX1dj+gQd4rlevm/imlFJ5KcfslCu4ylP50rx586S2v7+4QO7285MPxo27\n6WNu3bpVIooXlzsCAyXMZpPujz4qcXFxUtxmk69B1oO0sNul7/PPn3+P0+mUGuXLy5sgdpDOIG+C\nCEhTb2/5aOrUiz6ja8eOco/NJotBXvPyksjQUDl79uxN166U8oycsrNQhK7L5ZI6UVESC7IX5CuQ\nMsHBkp6eftnXT5s2TU6ePHlNx05KSpLBr79+PsSHDR0qfc+FqIDsAQkPDJQ3hw4Vl8slM2bMkIq+\nvtITxBfEBDIM5D2Q/4KUL1VKsrKyREQkOTlZbBaLpF9wvDYBATJ37tzc+WKUUm6XU3YWisURp06d\nIj0ri3ElSzLu3GM2Hx8OHDhAlSpVLnrttm3bePzxx+nfpw/vxsbmeGyTycTE2Fh8rVZSzp5l8cKF\nRF7wfCLgys7m9cGDqRAVxQuPP04lp5P1GNPIBDj48MP4+fkRBHT197/o2AJcuK4tCzBfYYGGUp6S\nlQUXjIqpm1BoxnSv1YMdOhC+eDGfWa3sOHiQkiVLXvX17779NnFvvcUZk4kVmZmYsrMxiXAHcAQj\ndJNNJoaIMCkwkDZnz1IKOH3u+bW+vvy2aRNVq1a97PGf7NaNA/Pm8WxaGiu9vfk5PJz1f/6Jn59f\n7p64Ujdg3z4YOxZ++gni48FSKC7T8laRmjKWk23btrH81195y+Wiq8vF6LffvurrU1JSeGf4cEhL\nIzA1Fa+sLGoDfgEB7DWZ+BD4FCgmQh0gIymJRUAaUAPYCjhymL727rhxtB44kK9at8b0xBMs37BB\nA1d53MaN0LUr1KsHxYrB8uUauLmlSIXusAED+K/DwWngYYeDSR9+yIkTJ674+t4vvIArPZ16GCvN\nsoGKImSmpDBZhJbAPRhLfycDNhFigPeB5zGmkQmQlJTErl27mPjBBxcdX0S4u0ULThw7xrdLlvCf\nxx5j0qRJTJgwgZSUlDz4BpS6MhH48Udo1Qo6dTICd/9+ePttCA31dHWFyI0OBhdENaOipExQ0L9/\nQkJkyZIlV3x9jYgI+eqCm1zDQUqClAaZdcHjb4EEmc1SOihIelzw+EEQP5NJUlJSpHPbtuJlMsme\nPXvOH3/RokVSwWaTYJtNJk+aJKF2uwwwm+U+m01qRUVJSkqKO74WVcRlZop89pnIbbeJ1KghMmOG\niMPh6aoKrpyyM1+G7sqVK+XB9u3lvlat5LvvvvNYHc1q1ZLFF4ToRJAgkLvbt5dAs1nGgrwC4gdi\nNZvlo48+kuI2m3wOshakkckkjaKjpX///lLSapV+Fos83rWriBgzLupXry6zQHpbrVLCZpNV5z7H\nBdLOYpEJEyZ47NxV4ZecLBIbK3LLLSItWoj88IOIy+Xpqgq+Ahe6a9askRI2m0wG+QLkFrtdvpo1\nyyO1TJ44UarY7bIU5HuQMJBHzGb57wMPSOcOHSTEx0f8TCZpYjbL7WazVAwLk4EDBkiQySQlTCbx\nA+lqscijJpMEgawCKe7rK3v27JFFixZJNT8/yQY5CmIFOX5BwPcCaVi3rkfOWxVux46JDBokUry4\nSJcuIuvWebqiwqXAhe7TDz8sYy8In+9BWtx+u0dqcblc8t6YMRJsMklZkE5mszzr5SVmk0kWLFgg\nARcsfHCBPOHjI35ms8SBdAcZdMF5xIK0Aenm5SWPd+0qdzZrJrdYrdI8MFCaBwaKP0gXkCMgv54b\nxvC3Wj1y3qpw2rlT5JlnRIKDRXr0ELlgpEvlopyyM9/djxSR85vOAJjOPeYJJpOJ7j17ImYzDofj\n/OPvmM18OGYMoUDMP68FmmZmsgCoBZwCmlxwrGrASB8fwsuWpUJKCuOnTuXIkSPnn589ezZLJ0+m\ntstFMDAUeN3LK29PUBUJa9bAqFGwYgU89xzs3Ak5zJRUeelG0zqvrF69Wkra7TIV5EuQsna7fDlz\npkdquZLNmzdLaV9fKQvSEcQBcgaknre3WEE2gXwEUuPcirVDII3tdhkxbNgVj3n8+HEp5uUlvUGm\ngkSBBPv6SmJiohvPTBUWTqfI/PkiTZuKlC8vMn68iN6XdY+csjPfXek2aNCAbxcvZvzbb5PpcBDb\nsyed778/zz5v2kcfkZqczPMvvnjN73lz4EACMjKojnGFG4wxnczmdNITaA2EAvuBmhjTxuxOJ7UO\nHMDhcGC1Wi85ZnZ2NhaLBS+nk9+BIcDH3t6sW7eOdu3a3eRZqqLC4YCZM2H0aLBaoV8/6NJF59jm\nKzea1oVBSkqKlC5WTELs9mvei0FEZPSIEVIrMlImnhuvPQuyEiTYZBJ/i0Ws526M+YAEg6wA2QfS\nzmaTPt27X/aYp0+fFj9vb0k8d8wskGr+/rJ8+fLcOl1ViJ05IzJypEh4uEibNiJLluhMBE/JKTuL\n1OKI/zVx/HhisrO5z+XivVGjrvl9fQcOpOeAAUy32zkL2IFPfHy4v1s3dv31F+3uuouy4eF0uv9+\nXsIY270VGJuezvzvvrvsMYOCgujZowct/fx4C2hvt1MuOppGjRrd/ImqQuvoURgwACIjIS4OFiww\nluy2bg0mU87vV+5X5PZe+EdqaioVwsP5OSkJO1DPbmfXoUOEhISwfPlyRISYmJiL3pOQkECpUqUA\ncLlc9H72WT6ZPh1vk4m60dF8s2gRSUlJ1KpUiXCzmdqdOuE7ezYfnesYsQh4LSqKjbt2XbYmEWH2\n7NlsXLeO8hUq8NRTT120965S/4iPN4YQ5s6FRx6BF1+E8uU9XZWCIrSf7vUaOWKEVLJYZBrINJDb\nLBZ5feBAycrKkqjwcKkQGnp+C0YRkfXr14uX2Szx8fEXHScpKUkSExPFde7fcj2eeEL6+/jI9yDV\ny5eXyNBQecRqlYFms5S02WThwoVuPU9VeLhcIitWiNxzj0ipUiJvvimi91nzn5yys8he6c787DN+\nmjfvosea3XknFl9fPu7ZExPw+PjxPPrYYwDc06IFf/32GzXuvZcvrjBEcOjQIWpXqsSOjAxKAHf4\n+9N7wgQSExNJSU6mw913Ex0dnbcnpgodlwu+/x5GjoSEBHj5ZXj0UdDG0PmTdgO+DtnZ2VQtW5Yp\nf/+NCXgqNJQdhw4RFxdHp2bNiEtPp5rNxvJNmy7Zpxdg8BtvMGbECKLsdgD+djioHh3N0t9/d/OZ\nqMIgIwM++8wYRggMhP794b77QKdv5285ZadOJLnAzJkz8T19mjIYU8H8zpzhi88/55vp03nl3NXr\ni5mZvPDkk8xfuvSSHmwv9e1Lp/vuu+ix0qVLX/HzsrKyrmnM9syZMxw5coTq1avfwFmpgub0aZg0\nCcaNg9tvhylToFkzvTFWWBTp2Qv/a9e2bWQEBtKhdGnaly5NWmAgq5cvZ8GyZXwWEEBMYCCzfX35\nedUqRr7zziXvDwwMpE6dOhf9CQ8Pv+R1mzZtonJEBL5WKxXCwli7du1V6xrYpw93xcRctCpOFT6H\nDsFLL0GFCrB9uzELYeFCiInRwC1MdHghB06nkzVr1pCdbTTVeXvwYFzLl7MtIIB9x45hu86BtbS0\nNKIiIhhz+jQPYuy52yMwkB0HDxIYGHjJ6w8ePMjtVapQ3WTiP6NH81yPHrlwVio/2brVWKa7cCE8\n8QT07g233OLpqtSN0s4RN8nLy4vGjRsTExNDWFgYm9au5RsRGjmdTP7ww+s+3u7duwl2OumK8eXf\nB9wCxMfHX/b1IwYP5lmnk9Hp6Yx44w292i0kRODXX6FdO2jbFqpVM1rjjB6tgVvYaeheh+Gvvkrv\nrCwCgTdSUxk5bBjp6enXdYxSpUpxNDOThHM/nwIOZGZeduz34MGDzP7qK17KyqIeUDMjg08+/vhm\nT0N5kNMJs2cbXRmeew4eeAAOHIBXXoGgIE9Xp9xBQ/caOZ1O5v7wA4NdLiwmE9EmEwlnz/LOiBHX\ndZywsDBe6teP+n5+PGm3U8/Pj6d79CAyMvKS18768kuSMjOpZrNRymbjN4eD6VOm5NYpKTdKS4OJ\nE6FSJXjvPXj1VWOBw5NPGnskqKJDx3Svg9PpPP+dnD17lsrlymH39WX3kSOX3cTman7//Xf+/PNP\nKleufMnKt39kZ2dz6tSpix7z8/PTxpUFyMmTMGGC8adhQ2MDmsaNPV2Vyks6TzePDB8yhF0jR3LS\nbObukSP1Bpe6yP79EBsLn39uzK19+WWoWtXTVSl30NDNA2fPnqVimTL8nprKGeCB4sVv6GpXFT6b\nNhkzEZYsgaefhl69ICzM01Upd9LZC3lgfGwsZTMz2QEcA3yTkvj4o488XZbyEBFjTm2bNtCxI9xx\nhzETYcQIDVx1KV2RdgP8bDbKNGzIPzFbBfDRtZlFTlYWfP21cWWbnW0s0+3aFXx8PF2Zys90eEGp\n65SSAh9/DGPHwq23GmHbrp2uGlMG3XtBqVxy/DiMHw+TJ0Pz5v/Ot1XqeuiYrlI52L0buneHKlUg\nMRFWr9bAVTdOQ1epK1i71lgx1qgRlCpltC6fNAkqVvR0Zaog0+EFpS7gcsGiRcaG4QcPGrt+ffop\n+Pt7ujJVWGjoKgVkZv7butxiMVaOPfggaIs6lds0dFWRlpRkbBL+/vvGmG1srHbSVXlLQ1cVSX//\nbQTt1Klw550wb57RpUGpvKY30lSRsmMHPPUUVK9u7Py1YQN8+aUGrnIfvdJVRcLvvxsrx1avhp49\nYdcuKFHC01WpokhDVxVaLhfMn2/MRDh2DPr2NW6WnWvWrJRHaOiqQsfhMLZUHDXKmOo1YAB07qyt\ny1X+oKGrCo0zZ/5tXV6rFnz4obFcV2ciqPxEb6SpAu/QIWPoIDLSaIGzeLGxwKFFCw1clf9o6KoC\n648/4NFHjataEYiLgxkz4LbbPF2ZUlemoasKFBFYtgzatzfm11auDHv3Gtssli3r6eqUypmO6aoC\nwemE774zZiKcPWv0HJszB3x9PV2ZUtdHQ1fla+npxoYzY8YYO3298grcey+Y9d9oqoDS0FX50smT\nMHEifPABNGgA06dr63JVOOj1gspXDhwwOuhGRRn/vWyZsS+CBq4qLDR0Vb6weTN06wbR0caKsW3b\njD5kVat6ujKlcpeGrvIYEfj5Z2MWwj33QJ06sH8/vPMOhId7ujql8oaO6Sq3y842eoyNHGlsHt6v\nn3GVq63LVVGgoavcJjXVGDKIjTXm1A4fbrQu15kIqijR0FV5LiHBmIXw4YfQrJmxf22DBp6uSinP\n0GsMlWf27IEePYxVY8ePG3vafvutBq4q2jR0Va5bvx66dIGGDSEkxOjWMHkyVKrk6cqU8jwdXlC5\nQsTY3WvkSNi3D158EaZN09blSv0vDV11UzIzYdYsY8Nwsxn699fW5UpdjYauuiHJyUYn3dhYY8x2\nzBho00b3r1UqJxq66rr8/bfRmWHqVGjdGubONVaRKaWujd5IU9dk5054+mmjdXlKCqxbZwwraOAq\ndX30Sldd1apVxs2xVau0dblSuUFDV13C5YIFC4ywPXpUW5crlZs0dNV5Dgd88YUxE8HPz5iJ0Lkz\nWPT/EqVyjf46Kc6eNRYvvP8+1KgBEyZoJ12l8oqGbhF2+LARtJ98Ymw888MPRmddpVTe0dkLRdCf\nf8LjjxutyrOzYdMm+PxzDVyl3EGvdIsIEVixwrg5tnEjvPCCsSFNSIinK1OqaNHQLeScTqPH2MiR\ncOqU0br8m2+0dblSnqKhW0ilp8OMGcby3JAQGDAAOnYELy9PV6ZU0aahW8icOmVsFj5+PNSta3Rq\naNJEZyIolV/ojbRC4uBB6NMHKlaEvXvhl19g/nxo2lQDV6n8REO3gNuyBR5+GG6/3Wjs+McfxhSw\natU8XZlS6nI0dAsgEVi6FNq2hfbtjalf+/YZN8vKlPF0dUqpq9Ex3QIkO9uYeTBqlHGj7OWX4fvv\nwWr1dGVKqWtlEhG57BMmE1d4SrlZaqrR+mbMGIiIMPZE6NBBW5crlR/llJ16pZuPnThh7IMwcaIx\nA2HmTKPZo1Kq4NJrpXxo3z5j79rKlY2tFVesgDlzNHCVKgw0dPORjRuha1eoVw8CAyE+HqZMMcJX\nKVU46PCCh4nATz8ZMw927zZal0+dCgEBnq5MKZUXNHQ9JCsLvvrKmIkgYtwce+ghbV2uVGGnoetm\nKSnw0UcwdixERcG77xrzbXXVmFJFg4aumxw7ZuyHMGWK0ZVhzhy44w5PV6WUcje9kZbHdu6EZ54x\nluWeOQNr1sDXX2vgKlVU6ZVuHlmzxrg5tnIl9OhhhG/Jkp6uSinlaRq6ucjlgoULjZtjhw4Zrcs/\n+8zorKuUUqChmyv+aV0+erTRkaF/f3jgAW1drpS6lMbCTfjf1uXjx0PLljoTQSl1ZRq6N+DIESNo\nP/4Y7roLFiyAOnU8XZVSqiDQ2QvX4Z/W5TVrQmam0br8iy80cJVS106vdHMgYsxAGDkS1q/X1uVK\nqZujoXsFF7YuT0yEfv2M+bU2m6crU0oVZBq6/yMjw2hdPno0BAUZrcs7ddLW5Uqp3KGhe86Frcuj\no42dvpo105kISqncVeRvpP31l7GdYsWKxtaKP/9sLHCIidHAVUrlviIbulu3wiOPGDMPLBbj508/\nNebbKqVUXilSoSsCv/xizK296y6oXh327jWW7UZEeLo6pVRRUCTGdLOz4dtvjZkIaWlG6/J587R1\nuVLK/Qp1C/a0tH9bl4eFGTMR7r5bW5crpfJOkWzBnphotC6fMAEaNTJ2+mrc2NNVKaVUIRvT3b/f\nWDFWqRIcPgzLl8PcuRq4Sqn8o1CE7j+ty+vWBX9/Y4+EqVOhShVPV6aUUhcrsKErAj/+CK1bGyvG\n6tY1rnRHjDDGb5VSKj8qcGO6WVnGHgijRhn7I/TrZ1zl+vh4ujKllMpZgQndlBRj/9qxYyEy0rii\nvesuXTWmlCpY8n3oHj8OH3wAkyZB8+YwezbUq+fpqpRS6sbk2zHdffuge3fjZtjJk7B6tQauUqrg\ny7dXutu3Q6lSRuvyUqU8XY1SSuWOQr0iTSml3C2n7My3wwtKKVUYaegqpZQbaegqpZQbaegqpZQb\naegqpZQbaegqpZQbaegqpZQbaegqpZQbaegqpZQbaegqpZQbaegqpZQbaegq5Ub/uyZ/z549PPXs\nk7rPSRGioauUGyQnJ9OxSwesNh+CSxVj6sdTARg0dADTPp7G4sWLPVyhcpd8u7WjUoXJUz0fJ962\nlA6J2aQeSKZ/uz74+viyaPEP1HhH6PrE/WSluigZVoLX+w3hqaeeytXPP3HiBN9//z0A99xzD6V0\nv1SP0a0dlcpDIsL+/fup1+R26q85i72s8Xj8GyZkXmUsXXZz5AcnJRpDlUFwcjWs6QIbV8VRq1at\nXKnh4MGD1GsSTUCjdDDB2RVW1q7YQGRkZK4cX11Mt3ZUyoOWLl1KVFQUAUEBnN1mPCYCqX/4sOOP\nHXgFOzmzGWqOBJ9gCGsPpVrA4KGDc62G194cRMknTlPnqzTqzEoj7LmzDBo6INeOr66Phq5SeURE\nGDjkZYLrCmXDy7P1UTt/dPdlQzs/UjfY8LKaKfFjDCaXibQDxntc2ZB2GH76eTGnT5/OlTr+TjhC\nsVqu8z8H1HJx7MTRXDm2un4aukrlkaVLl3LwxB4aLxI2bdnArOmz6Vl9JIM6jyUzI4uS9Sw0qdec\nsWNjWR1jY3Mv+C0GbBEQcT+MeW/0VY8/7K1h7Nq1K8c67mrRgYOj7WQkgOMEHBxp564Wd+fWaarr\nJVdwlaeUUjlwuVwS3biW1PsceUCQ24Z7SZeHO4uIyPARw6ViV7u03YkUK+EnZ86ckUGDBgkmxGxD\nfAJN4lvMR8JuKX3F42/evFlMJqRjlw451uJ0OqVX3+fFavcRH5u39OjdXbKzs3PtXNXFcspOvZGm\nVB6Ii4ujTp06hNYIwORlIivdxak9qezbt59adWtQf1kKxarBlv/a+L9K/Xl90Bt06daZbaUWcPw7\nHz6b+BUxMTEUK1bsssdv37kth6N/5sA4K6uWrqNGjRo51vTP77PJZMrVc1UXyyk7NXSVygMul4tt\n27bhcv07lmq1Wlm/fj2PP/EYweX8AEg/66CYLYiVy1ZTu34NWu7J4Mhs8PumEb/9+Ptljx0XF0fz\n9o1ouTedfR+YKbf+LuZ9vdAt56VypqGrVB5xOp2MGjuSpSt/JCK0LMPfGEGZMmWu+p7s7GwOHjwI\nwK5du2jfvj3Dhg9j5954lid9TeTzLlyZsO5+C7/8+BuNGjW65BjtO7dle8ASyj4mOFNhXRcvNq2P\nu6arXZX3cspOXRyh1A16rtcz/LBtFmV7p7Fug4Wqtb+hZauW3H1nJx7u9jBDhg7mqSefJjU1lcqV\nK+Pr64vFYqFChQoA9B3Uh3LdTIybOJa2be8iNLEmaUOMY98WbSIxMfGyn2u3+VHyr1qkDzN+rtPA\nzPHjxy8KXRHB6XRiseiveH6jV7pK3YDs7Gxsfr50SHDiHWg8tvxO8LsV0tbZqRXeiJ+X/ozV7o0t\nyJusJHi02xP079efW265hW3bttGoVT1a7U1ny8N+9Go5nD69+uRKbTM+m8HzfXqQlpROg5h6zJk5\nT1eguZEOLyiVB86H7gkn3ufuda3qBBFdoGRL+KEc+EXCbSNh41MQ2R0yT8GZOUFsWLWZPgNe4ED0\nD0T1d3F6M2y5O4i/9hzFZrPdVF3r16+ndccY6v+UTkAV2D7QQun4Bvy6aEUunLW6FroiTak8YLFY\nePix/2PjfXb+XgDbXoMzmyG0HVj8AIGwjrB7LNSZANWHQZ0PoOQjSYwdN5olPyxlywAX35jglwaQ\nePQMq1atwuFw5PjZzzz/FAsWLLjscytXriS8i5PAmmD2hspDs1n129pcPnt1MzR0lbpBUyd8zNOt\n+uOMjebARAu3PgFn/4RV9xoLHP6eB1nJ4HvBvTXfCBcpacmkJqchIkyZOgUfmxlrkBdD3xrMfV3v\nuepnbtmyhenTptN7QE+cTuclz5cuXZqULd7IuUkTpzdBSKmgmzrPjIwM3nv/Pf2Xb2650Qm+Sql/\nxcfHS7O2jcQnxCTFmyDegSYJaYh4ByOBtZDWW5CY35CgCLssWrRIREQcDoeElS8lzZYi1pKIlw0J\nKO0rmzZtuuLndOjcVmqPMUt4fX+ZNWvWJc9nZmZK09YNpUxDf6n8hF2KlbTL/Pnzb+rc3hv3ngCy\nYMGCmzpAzjXdAAAJx0lEQVROUZFTdmroKnWTnE6njI4dJWFRJSQ42iQNv0XMvkjLdUijhYh/ZcRa\nGrEWN0mfF/ucf9+UqVOkbCs/eUCQOhMR33Dklq5I23tbX/Zz4uLipFioTTqlIk0WIZHVyl52ZVlW\nVpZ88803MnnyZImPj7+pc0tPT5cS4cFS5RWkxh1VxOVy3dTxioKcslNvpCl1kwa82o9PfxxPxTcc\nJG2Hne+AAN4mC5nZ2fgEgbU0BNcG64aKxG/ahYgQUSEMU60EitUEyYSdo4zXeTutvPrSEOrXr0/z\n5s3PryC754F2rDq8mGK3QfphOL0WoiIrMe7dD2nZsmWenNv749/n/SWvcvvcVFbW9ufjEbPo0KFD\nnnxWYaGzF5TKY8WK+9NkUyp+5Yyf1/0fHJkH1iAzPk47DRo2xGr1ASDQP5hpk6djMpmIjY3l9cGv\nkW1xYPJxYTJDxjHwskOppmbS91qoUqYWzzz6HFarlW/nzmbhkvn4FHeRcUaIuB/8o4wNbH6a/wv1\n69fP1fPKyMggIjKMqHFnCI6GI3PBNbMKW9fF61Liq9DFEUrlMUGMS9vzP0ON4VCxt4s/+zgITgpm\n5rSvLnqPw+HA39+f8e9/wO7du0lISCAgIIBpM6dSe1YapVq6cDoy+bnWevpN2kRACV8O/5ZKg29h\n7UPGtDSzBXa+C2XuT2P6zE9zPXT37duHzWbjQD8LB849Zremc/r0aUJCQnL1s4oSDV2lblK58mVZ\nde8Oqr4BSX/C8R/htlFgMkHoQ1ls7RN3yXveefcd3nx3CMWr+pL8dwYm8cIi3qQkZxBybuWvlxWK\nN4WgWk4qPp+K+TnY/iaE3Qv1phuvKdEM/nwNvDt65/p5VatWjUN7dd/d3KZTxpS6CXv37mX7lp0E\nVIVdo2DfVChWFayljA4Rx7/1pkbV2wBjae7Y2LEkJSXx7rjhVBsCzTZk0O4ABNzmpNyLGfiWgM3d\njfem7IG/F0BwXeOzQhoYY7lBt/37+QFVwXHCxHNP93D7uasbo1e6St2EzMxM7n3wHgQXRIKznJNN\nGzeyLCoVi81EiE8oYxa+x/jx44mIiKDvS31Zt24tLpOTsHP7iJu9oXQbSDsEJWNMHP1emBcIrkyw\n3wqB1SEjAQ7EWnEkONj9HpRuC76lYWtfaN+mA1WqVPHsF6GumYauUjfo5MmTLFu2jGb1WtCxY8fz\njR5dLhfx8fFkZmZSo0YNPvv8M3r16kWZ8qFUeRXmvD+bwNuFA9Og5ruQnQKHZ0PEQ7BzplD/Kwi5\nA0zexmq1ecHG57Vp24yVB5dT6RUHK9pAdhrYS1ro9vYjHvwW1PXS2QtK3YBjx44R3bA2tgZJWAJd\nHJvjzdJFy4iOjr7odVlZWZSvHEFAlwQOfGSiQ4KwojWcWeMF3i5M3oIzHaz+FlwZZnztPtSckULp\nO8GVBb/V9+aO0OZUqlKJzh0foG37O7lzXxa+oUY/td/r+PPluO9p0aKFh74J9b909oJSeWDk2HcJ\nuPckNd/LBiCgroO+r/Vm2aKVF71uxmczsESmUeMdOLZY2DfF2Pxm355ADuw8xLFjx7BYLKSmplKi\nRAm2bt1K564dKRXjRfIuoX7Fxnw/eyFeXl4ADHp1EOObjabEfan8/a03fviydv0amjRpgrd37t9M\nU7lPb6QpdQMSTh7Dr2r2+Z/9q8KRw0eo3aAGNn8rVetEMWXKFN4YPojQJ1NIPwwVesCf/UxsfsbM\nyYRTNL+rCTabjbJly1K1alVKlixJq1at2LohniH3fUj9SjEs/3U5JcKCGTHybUSEwa8OYdqYL0mf\nW4ag26DMa4l8+Mub3N+tk/7LtIDQ0FXqBtzTthN/xdpJ3mksaNjzmi8JxxPw+m88bY9k4t97Dz1e\nfJbsTCf7+wezvlEwB4YUQ4B6s1zcfRySG8RRvmoEJcKDeOud4edDs1y5cmzfE8+mU78S82c69Vcm\nE/vpW3wx8wsAQkNDSTedpe7sLMo/BtHz0ln22y8cOnTIc1+IumYaukrdgIcefIh+T7/B2mYBLKtq\no0HpdvgW9yKyh+AdCOUfMzY0j2nagoRDp0g4dIrxYz6kfHt/QtuBTxBUf1fIdrioveAs738+gmnT\np50//sKf5lFhcBq2cAioBGX7pjH/p7mAMU5ssZvO//aavcHLaiYzM9MD34S6Xhq6St2g/n0HcPp4\nEsmn03hv9HhSj2eSedp4LjsFMk/Cwh/mc/z4cQBCQkJI3SO4zo1KpO4HkxmCasGtr6Qxd9E3549d\nPKQEyTv+/ay0HRZKFw8F4Pbbb8cvszjbX7GQ+Dts6+FDhbJR52dPqPxNQ1epXFCmTBmeebo7qxrZ\niesNvzaG8Pug7H9hxKi3AGjTpg3Vw+uytoUfcb1MLGtybuWaF6TuMlMi+N+WOqOGxbL3dX/+6G5l\n8//5cvrrYAa+PAgAX19fVvy8mupHOnKybxXqux5kyYJfMZv117kg0CljSuUSEWHAgAGMGjUKs9kE\nJmNlWZmyYRzafwQw2vzMmjWLjRs3MnXaZMr8x4k4TJxabGPDqs2UL1/+/PH279/PvHnz8PHxoUuX\nLpQsWdJDZ6auh+4yppQbiQjZ2dkXPWY2m89P+brQ/v37mTNnDl5eXjz00EOEhYW5q0yVhzR0lVLK\njbQxpVJK5SMaukoVYC+8+DybNm3ydBnqOujwglIF1Jo1a2jctBFNWzVi2eKVOb9BuYUOLyhVSL0y\ntB+1xsLW7ZtZvXq1p8tR10hDV6kCaM2aNWyJ38StzwqRr6bzytB+ni5JXSMNXaUKoFeG9sOvXhpH\n5oDZV1j56yq92i0gdGtHpQqgGlVrUuxoAMwFR4YDL6/l7Nq1i4YNG3q6NJUDvZGmVAHXp19vPvrq\nA5rXbcOCbxd7upwiTxdHKFWIJSQkUKFKeRqvTWd1MxvLF6+mVq1ani6rSNPZC0oVYm+PeouI/3MR\nEAWR/R28OmyAp0tSOdArXaUKqKSkJEqFlqRUMy/spc1kpwl7v0ljx44dVK5c2dPlFVnaI02pQspu\nt/PpJ9NxOBznHzN3NFOmTBkPVqVyole6SimVi3RMVyml8hENXaWUciMNXaWUciMNXaWUciMNXaWU\nciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMN\nXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWUciMNXaWU\nciPL1Z40mUzuqkMppYqEK4auiLizDqWUKhJ0eEEppdxIQ1cppdxIQ1cppdxIQ1cppdxIQ1cppdxI\nQ1cppdzo/wEzbZF7AN/ugwAAAABJRU5ErkJggg==\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred_test = logreg.predict(X_test)\n",
    "plt.scatter(X_test[:, 0], X_test[:, 1], c=y_pred_test, marker='^')\n",
    "plt.scatter(X_train[:, 0], X_train[:, 1], c=y_pred_train)\n",
    "plot_decision_boundary(logreg, X)\n",
    "print \"Accuracy on test set:\", logreg.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also take a look at the coefficients, but they probably don't tell you much."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.42842754, -0.62076837]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logreg.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MNIST DIGITS\n",
    "-------------\n",
    "Let's go back to the MNIST example of before.\n",
    "This dataset is a bit more exciting since it is much higher dimensional and not as easy to separate.\n",
    "First we load the data again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_mldata\n",
    "mnist = fetch_mldata(\"MNIST original\")\n",
    "X_digits, y_digits = mnist.data, mnist.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember what the dataset looks like:\n",
    "\n",
    "- 70000 examples, 784 dimensional\n",
    "- ten classes, 0-9\n",
    "- each 784 vector is a 28 * 28 digit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_digits.shape: (70000, 784)\n",
      "Unique entries of y_digits: [ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9.]\n"
     ]
    }
   ],
   "source": [
    "print \"X_digits.shape:\", X_digits.shape\n",
    "print \"Unique entries of y_digits:\", np.unique(y_digits)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0xac30b2c>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPwAAAD5CAYAAAADZljUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAELJJREFUeJzt3X9MVfX/B/DnrY/YJkgqIjTwR828Buq92AVqEy+2NYYF\nOmtow5qXloKlhtofzdklXWYtzVFTtrwtc1CzZaUWLldHjaWXJTDHblMrJzlIqE2ukwy38/2jbzcJ\neB/uvZx7r7yej80N7ut0zmvHnp577vt9z9ui67oOIhLhjmg3QESRw8ATCcLAEwnCwBMJwsATCcLA\nEwlieuBPnDiBmTNnYvr06aiurjb7cEGbOnUqZs+eDbvdjuzs7Gi3A5fLhUmTJmHWrFmB1/x+P4qL\nizF58mQsWrQI165di6n+3G430tLSYLfbYbfbUV9fH7X+2trakJ+fj4yMDDidTtTW1gKInXM4WH8R\nO4e6yWw2m378+HH94sWL+owZM/TOzk6zDxmUqVOn6r///nu02wg4ceKEfubMGT0zMzPw2vbt2/Xn\nn39e//PPP/XVq1frb775Zkz153a79bfeeitqPd2qvb1db2pq0nVd1zs7O/Vp06bp3d3dMXMOB+sv\nUufQ1Cv81atXAQB5eXmYMmUKHn30UZw+fdrMQ4ZEj6G5R/PmzcO4ceP6vOb1elFWVobRo0fD5XJF\n9RwO1B8QO+cwJSUFNpsNAJCUlISMjAw0NjbGzDkcrD8gMufQ1MA3NjbCarUGfn/ggQdw6tQpMw8Z\nNIvFggULFmDRokX44osvot3OgG49j1arFV6vN8od9VddXY3c3Fxs374dfr8/2u0AAC5cuIDW1lZk\nZ2fH5Dn8p7+cnBwAkTmH4j+0a2hoQEtLC7Zt24bKykp0dHREu6V+YuXqOZjy8nL88ssvOHr0KH76\n6SfU1NREuyX4/X6UlJRg586diI+Pj7lzeGt/Y8aMidg5NDXwDocDP/74Y+D31tZW5ObmmnnIoKWm\npgIAZs6ciaKiIhw6dCjKHfXncDjg8/kAAD6fDw6HI8od9ZWcnAyLxYLExESsXr0aBw8ejGo/vb29\nWLJkCZYvX47i4mIAsXUOB+ovUufQ1MAnJiYC+PuT+osXL+Lrr78OvH2JBdevXw+8ders7MTRo0dR\nUFAQ5a76y8nJgcfjQU9PDzweT8z9o9ne3g4AuHnzJmpra1FYWBi1XnRdR1lZGTIzM7Fu3brA67Fy\nDgfrL2Ln0OxPBTVN061Wq37ffffpu3btMvtwQfn555/1OXPm6HPmzNEXLFig7927N9ot6UuXLtVT\nU1P1uLg4PS0tTfd4PHp3d7deVFSkp6en68XFxbrf7496f6NGjdLT0tL0vXv36suXL9dnzZqlz507\nV3/xxRejOupx8uRJ3WKx6HPmzNFtNptus9n0r776KmbO4UD9ffnllxE7hxZdj7GbGyIyjfgP7Ygk\nYeCJBPmfGTu1WCxm7JaIhmiwO/WQr/BGc+R1XQ/8eeWVV/r8Hmt/2B/7G0n9qYQc+LVr16KmpgbH\njh3Du+++i66urlB3RUQRElLgb5c58kTUV0j38IPNkV+4cGHgNbfbHfj57rvvDr3DCHA6ndFuQYn9\nhWek96dpGjRNG9K2IY3DHzt2DHv37kVdXR0AYM+ePbh8+TK2bNny904tFsN7CSIyhyp/Ib2lvx3m\nyBNRfyEFPtbnyBPRwEIeh3/77bexcuVK9Pb2Ys2aNUhKShrOvojIBKbMpec9PFH0DPs9PBHdnhh4\nIkEYeCJBGHgiQRh4IkEYeCJBGHgiQRh4IkEYeCJBGHgiQRh4IkEYeCJBGHgiQRh4IkEYeCJBGHgi\nQRh4IkEYeCJBGHgiQRh4IkEYeCJBGHgiQUxZH55k+OGHH5T1d955R1n/4IMPlPVnnnlGWX/hhReU\n9aysLGVdIl7hiQRh4IkEYeCJBGHgiQRh4IkEYeCJBGHgiQThctE0oObmZsNt8vPzlfXu7u7hamdA\niYmJyvoff/xh6vFjlSp/IU+8mTp1KsaOHYs777wTo0aNgtfrDblBIoqMkANvsVigaRrGjx8/nP0Q\nkYnCmlqretvudrsDPzudTjidznAORUSD0DQNmqYNaduQ7+HvvfdeJCQkYNq0aXC5XCgqKvp3p7yH\nv+3xHv72Zco9fENDA1JTU+Hz+fD4448jOzsbKSkpITdJROYLeVguNTUVADBz5kwUFRXh0KFDw9YU\nEZkjpMBfv34dfr8fANDZ2YmjR4+ioKBgWBsjouEX0lv63377DYsXLwYATJgwAevXr0d6evqwNkbm\nMhpGXbJkieE+rl69qqxbLBZlfezYscp6XFycst7V1aWsf//998r63Llzwzr+7SikwE+bNm1IH+oQ\nUWzh1FoiQRh4IkEYeCJBGHgiQRh4IkEYeCJB+H3429T169eV9TNnzijrpaWlynpbW5thD0Z/x0bj\n8Ebj4C+99JKyXlJSoqwb9bd161Zl/eWXX1bWY5Uqf7zCEwnCwBMJwsATCcLAEwnCwBMJwsATCcLA\nEwnC9eFvUytXrlTWa2trI9RJ6IzWl7927ZqyPn/+fGXd6MGOZ8+eVdZHIl7hiQRh4IkEYeCJBGHg\niQRh4IkEYeCJBGHgiQThOHyMMhqjPnz4sLIe7vMIhrL452OPPaasb9iwQVm/5557lHW73a6sjxs3\nTln/9ttvlXWJz2zgFZ5IEAaeSBAGnkgQBp5IEAaeSBAGnkgQBp5IEOVz6V0uF44cOYLk5OTAd4f9\nfj9KS0vR1NSErKws7N+/H/Hx8X13yufSGzJabjs/P19Z7+7uDuv4hYWFynpdXZ3hPsL9vvmzzz6r\nrE+cONGwB5U77lBfz8aMGaOsHz9+XFnPysoKuqdICPm59CtWrEB9fX2f13bv3o3Jkyfj/PnzSEtL\nw549e4avUyIylTLw8+bN6zebyev1oqysDKNHj4bL5cLp06dNbZCIhk/QU2sbGxthtVoBAFarFV6v\nd8Dt3G534Gen0zmkqZpEFDxN0wxvr/4RdOCHem9+a+CJyDz/vaBWVVUNum3Qn9I7HA74fD4AgM/n\ng8PhCL5DIoqKoAOfk5MDj8eDnp4eeDwe5ObmmtEXEZlAGfhly5bh4Ycfxrlz55Ceno73338f5eXl\nuHTpEmbMmIHLly9j1apVkeqViMLE9eFNcu7cOWXd6DOOjz76SFk3GqNOTU1V1jdt2qSsP/HEE8r6\n7cBoHN5o/Xqj9edj9dn/XB+eiAAw8ESiMPBEgjDwRIIw8ESCMPBEgjDwRILwufQhunHjhrJu9Ez2\nI0eOKOtjx45V1vft26esP/jgg8p6T0+Psk5AW1tbtFsYdrzCEwnCwBMJwsATCcLAEwnCwBMJwsAT\nCcLAEwnCcfgQnTlzRlk3Gmc38vnnnyvr8+fPD2v/JBOv8ESCMPBEgjDwRIIw8ESCMPBEgjDwRIIw\n8ESCcBw+RJWVlcq60XP5jRbX5Dh7+MJdG2Ekrq3AKzyRIAw8kSAMPJEgDDyRIAw8kSAMPJEgDDyR\nIMpxeJfLhSNHjiA5ORlnz54F8Pe65u+9915gffJt27ahoKDA/E4j7PDhw8p6c3Ozsm609nhRUVHQ\nPVFwjP4OjOo2m20424kJyiv8ihUrUF9f3+c1i8WCyspKNDU1oampaUSGnWikUgZ+3rx5GDduXL/X\nR+IMJCIJQppaW11djQMHDmDx4sWoqKhAQkJCv23cbnfgZ6fTaTiVlIhCo2kaNE0b0rZBB768vByb\nN29Gd3c3Nm7ciJqamgHXUbs18ERknv9eUKuqqgbdNuhP6ZOTk2GxWJCYmIjVq1fj4MGDITVJRJEX\ndODb29sBADdv3kRtbS0KCwuHvSkiMofyLf2yZctw/PhxdHV1IT09HVVVVdA0Dc3NzYiLi0NeXh7K\ny8sj1SsRhUkZ+Lq6un6vuVwu05qJJUbrp//111/KenJysrJeUlISdE/S3LhxQ1kP93OiRx55RFl/\n/fXXw9p/LOJMOyJBGHgiQRh4IkEYeCJBGHgiQRh4IkEYeCJB+Fx6k9x1113KempqaoQ6iV1G4+xb\nt25V1t944w1lPT09XVlfv369sh4fH6+s3454hScShIEnEoSBJxKEgScShIEnEoSBJxKEgScShOPw\nJuFz542f3W80jv7xxx8r68XFxcr6p59+qqxLxCs8kSAMPJEgDDyRIAw8kSAMPJEgDDyRIAw8kSAc\nhx+E0Qq5RvXPPvtMWd+1a1fQPcWaHTt2KOtbtmxR1q9evaqsl5aWKuv79u1T1qk/XuGJBGHgiQRh\n4IkEYeCJBGHgiQRh4IkEYeCJBFGOw7e1teHpp5/GlStXMHHiRDz33HN46qmn4Pf7UVpaiqamJmRl\nZWH//v0j7hneFoslrHpHR4eyvmbNGmXd5XIp6xMmTFDWT506pax/+OGHynpLS4uyDvz9/4fKlClT\nlPWCggJlvaKiwrAHCo7yCj9q1Cjs3LkTra2t+OSTT7Bp0yb4/X7s3r0bkydPxvnz55GWloY9e/ZE\nql8iCoMy8CkpKbDZbACApKQkZGRkoLGxEV6vF2VlZRg9ejRcLhdOnz4dkWaJKDxDnlp74cIFtLa2\nIjs7GytWrIDVagUAWK1WeL3eftu73e7Az06nE06nM+xmiag/TdOgadqQth1S4P1+P0pKSrBz507E\nx8cbziMH+gaeiMzz3wtqVVXVoNsafkrf29uLJUuWYPny5YGHBjocDvh8PgCAz+eDw+EIs2UiigRl\n4HVdR1lZGTIzM7Fu3brA6zk5OfB4POjp6YHH40Fubq7pjRJR+JSBb2howP79+/HNN9/AbrfDbrej\nvr4e5eXluHTpEmbMmIHLly9j1apVkeqXiMJg0YdyQx7sTi2WId3nx7IDBw4o60uXLjX1+JMmTVLW\nExMTlfVz584NZzsDeuihh5T1BQsWKOuvvvrqcLZD/0+VP860IxKEgScShIEnEoSBJxKEgScShIEn\nEoSBJxKE4/CD+PXXX5X1J598Ulkf6AtFwTA6f0bfxzeSlJSkrA9lnsFIeLb+SMRxeCICwMATicLA\nEwnCwBMJwsATCcLAEwnCwBMJwnH4ELW3tyvrNTU1yrrR2unhjsOvXbtWWS8vL1fWp0+frqxT7OI4\nPBEBYOCJRGHgiQRh4IkEYeCJBGHgiQRh4IkE4Tg80QjDcXgiAsDAE4nCwBMJwsATCcLAEwnCwBMJ\nwsATCaIMfFtbG/Lz85GRkQGn04na2loAgNvtRlpaGux2O+x2O+rr6yPSLBGFRznxpqOjAx0dHbDZ\nbOjq6kJ2djZaWlqwY8cOJCQkoLKycuCdcuINUdSo8vc/1X+YkpKClJQUAH+vVJKRkYHGxkYAxk9k\nIaLYowz8rS5cuIDW1lbk5OTg5MmTqK6uxoEDB7B48WJUVFQgISGhz/Zutzvws9PphNPpHK6eiegW\nmqZB07QhbTukufR+vx9OpxObN29GcXExrly5gokTJ6K7uxsbN27E/fffjw0bNvy7U76lJ4oaVf4M\nA9/b24uFCxeisLAQ69at61dvaWlBRUUFGhoahnRAIjJXyF+e0XUdZWVlyMzM7BP2f57YevPmTdTW\n1qKwsHAY2yUisyiv8N999x3y8vIwe/bswGORX3vtNdTV1aG5uRlxcXHIy8vDpk2bMH78+H93yis8\nUdSE9ZZ+uA9IRObi9+GJCAADTyQKA08kCANPJAgDTyQIA08kCANPJAgDTyQIA08kCANPJEhEAj/U\n7+pGC/sLD/sLTyT7Y+DB/sLF/sIz4gJPRLGBgScSxLSvxxJR9IT01NrhPhgRRRff0hMJwsATCcLA\nEwnCwBMJwsATCcLAEwnyf5Vj4wB6AR6qAAAAAElFTkSuQmCC\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(y_digits[0])\n",
    "plt.rc(\"image\", cmap=\"binary\")\n",
    "plt.matshow(X_digits[0].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could try to learn all ten classes with logistic regression, but let us keep with two classes for the moment.\n",
    "We will use two classes that are quite hard to distinguish: seven and nine.\n",
    "\n",
    "To create a dataset only consisting of the classes zero and one, we need a new numpy trick!\n",
    "\n",
    "We can not only slice our data using ranges, like ``X[5:10]``, we can also select elements using conditions, like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "zeros.shape:  (6903, 784)\n",
      "ones.shape:  (7877, 784)\n"
     ]
    }
   ],
   "source": [
    "zeros = X_digits[y_digits==0]  # select all the rows of X where y is zero (i.e. the zeros)\n",
    "ones = X_digits[y_digits==1]   # select all the rows of X where y is one (i.e. the ones)\n",
    "print \"zeros.shape: \", zeros.shape\n",
    "print \"ones.shape: \", ones.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets have a quick look to see that we did it right."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0xaea1fac>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPwAAAD5CAYAAAADZljUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD2RJREFUeJzt3X9MVfUfx/HX/fJV/gjyR/xs4K+WYKCCjR9zSTfbGsMJ\nmn+YTk1gM4FKM/3P2XW1zDWzYk3dFGtzuLLNyiyZrh1/sCVsgStGSyvSGSq0iltaQzvfP76TVOQA\n93LuvfB5Pja363kfz3nvbC8/53PPj+uxbdsWACP8J9wNAAgdAg8YhMADBiHwgEEIPGAQAg8YxPXA\nnzhxQtOmTdODDz6o6upqt3c3aJMmTdKMGTOUnZ2t3NzccLejsrIyJSYmavr06T3L/H6/SkpKNGHC\nBC1YsEB//PFHRPXn8/mUkpKi7OxsZWdn68iRI2Hr78KFC3rssceUkZEhr9er2tpaSZFzDPvqL2TH\n0HZZVlaWffz4cbutrc1OS0uzOzo63N7loEyaNMn+5Zdfwt1GjxMnTthfffWVnZmZ2bNs69at9rPP\nPmv/9ddfdlVVlf36669HVH8+n8/etm1b2Hq6VXt7u93U1GTbtm13dHTYkydPtru6uiLmGPbVX6iO\noasj/O+//y5JKigo0MSJE/XEE0/o9OnTbu4yIHYE3Xs0Z84cjRs37rZlDQ0NKi8vV3R0tMrKysJ6\nDO/WnxQ5xzApKUlZWVmSpLi4OGVkZKixsTFijmFf/UmhOYauBr6xsVHp6ek9f3/ooYf05ZdfurnL\nQfN4PJo7d64WLFigTz75JNzt3NWtxzE9PV0NDQ1h7qi36upq5efna+vWrfL7/eFuR5J07tw5tbS0\nKDc3NyKP4c3+8vLyJIXmGBr/pV19fb3OnDmjLVu2aN26dbp06VK4W+olUkbPvlRUVOjHH39UXV2d\nvv/+e+3atSvcLcnv92vx4sXavn27YmJiIu4Y3trfPffcE7Jj6Grgc3Jy9O233/b8vaWlRfn5+W7u\nctCSk5MlSdOmTVNxcbEOHToU5o56y8nJUWtrqySptbVVOTk5Ye7odgkJCfJ4PBozZoyqqqp08ODB\nsPbT3d2tRYsWafny5SopKZEUWcfwbv2F6hi6GvgxY8ZI+v839W1tbTp69GjP6UskuHr1as+pU0dH\nh+rq6lRYWBjmrnrLy8tTTU2Nrl27ppqamoj7T7O9vV2SdP36ddXW1qqoqChsvdi2rfLycmVmZmrt\n2rU9yyPlGPbVX8iOodvfClqWZaenp9sPPPCA/dZbb7m9u0H54Ycf7JkzZ9ozZ860586da+/Zsyfc\nLdlPPfWUnZycbI8ePdpOSUmxa2pq7K6uLru4uNhOTU21S0pKbL/fH/b+Ro0aZaekpNh79uyxly9f\nbk+fPt1++OGH7RdeeCGsVz1Onjxpezwee+bMmXZWVpadlZVlf/755xFzDO/W32effRayY+ix7Qib\n3ABwjfFf2gEmIfCAQf7rxkY9Ho8bmwUwQH3N1AMe4fu7R9627Z4/L7300m1/j7Q/9Ed/I6k/JwEH\nfs2aNdq1a5eOHTumd955R52dnYFuCkCIBBT44XKPPIDbBTSH7+se+Xnz5vUs8/l8PZ/Hjh0beIch\n4PV6w92CI/oLzkjvz7IsWZY1oHUDug5/7Ngx7dmzR/v375ck7dy5UxcvXtTLL7/8/416PP3OJQC4\nwyl/AZ3SD4d75AH0FlDgI/0eeQB3F/B1+DfffFPPPPOMuru79fzzzysuLm4o+wLgAlfupWcOD4TP\nkM/hAQxPBB4wCIEHDELgAYMQeMAgBB4wCIEHDELgAYMQeMAgBB4wCIEHDELgAYMQeMAgBB4wCIEH\nDELgAYMQeMAgBB4wCIEHDELgAYMQeMAgBB4wCIEHDELgAYMQeMAgBB4wCIEHDELgAYMQeMAgBB4w\nSMC/Dw+47ZVXXnGsb9q0ybHe30+WW5blWH/00Ucd68NRwIGfNGmS7r33XkVFRWnUqFFqaGgYyr4A\nuCDgwHs8HlmWpfHjxw9lPwBcFNQpvdMpk8/n6/ns9Xrl9XqD2RWAPliW1e/05CaP3d9Epw9TpkxR\nbGysJk+erLKyMhUXF/+7UY+n3/kT0B/m8IFxyl/AI3x9fb2Sk5PV2tqq+fPnKzc3V0lJSQE3CcB9\nAV+WS05OliRNmzZNxcXFOnTo0JA1BcAdAQX+6tWr8vv9kqSOjg7V1dWpsLBwSBsDMPQCOqW/fPmy\nFi5cKEm677779OKLLyo1NXVIG8PI9+677zrWX3vtNcd6VFSUY/3GjRuOdY/H41gfiQIK/OTJk9Xc\n3DzUvQBwGbfWAgYh8IBBCDxgEAIPGITAAwYh8IBBeB4eYfPTTz851v/+++8QdWIORnjAIAQeMAiB\nBwxC4AGDEHjAIAQeMAiBBwzCdXi45tixY471t99+O6jtp6enO9Y//fRTx3piYmJQ+x+OGOEBgxB4\nwCAEHjAIgQcMQuABgxB4wCAEHjAI1+ERsFOnTjnWV65c6Vjv6uoKav8bNmxwrE+cODGo7Y9EjPCA\nQQg8YBACDxiEwAMGIfCAQQg8YBACDxjE8Tp8WVmZDh8+rISEBH399deSJL/fr2XLlqmpqUmzZs3S\nvn37FBMTE5JmEVnee+89x/rPP/8c1Pa9Xq9jfcWKFUFt30SOI3xpaamOHDly27IdO3ZowoQJOnv2\nrFJSUrRz505XGwQwdBwDP2fOHI0bN+62ZQ0NDSovL1d0dLTKysp0+vRpVxsEMHQGfWttY2Njz6uF\n0tPT1dDQcNf1fD5fz2ev19vv6RmAwFiWJcuyBrTuoANv2/aA1rs18ADcc+eAunnz5j7XHfS39Dk5\nOWptbZUktba2KicnZ/AdAgiLQQc+Ly9PNTU1unbtmmpqapSfn+9GXwBc4Bj4JUuWaPbs2fruu++U\nmpqqvXv3qqKiQufPn1daWpouXryo1atXh6pXAEHy2AOdlA9mox7PgOf6iEydnZ39rpOQkOBYj4qK\ncqyPHTvWsf7+++871ufOnetYN5VT/rjTDjAIgQcMQuABgxB4wCAEHjAIgQcMQuABg/BeekO1tbU5\n1p988knXe3juuecc61xnH3qM8IBBCDxgEAIPGITAAwYh8IBBCDxgEAIPGITr8Ia68/Xjd7r5OwTB\nePzxxx3ra9asCXofGBxGeMAgBB4wCIEHDELgAYMQeMAgBB4wCIEHDMJ76Ueojz76yLG+cuVKx/qf\nf/7Z7z5mz57tWP/ggw8c64mJif3uA4PHe+kBSCLwgFEIPGAQAg8YhMADBiHwgEEIPGAQx+fhy8rK\ndPjwYSUkJPQ8H+3z+bR7927Fx8dLkrZs2aLCwkL3O8VtIuG98lOmTHGsc5098jiO8KWlpb1elODx\neLRu3To1NTWpqamJsAPDiGPg58yZo3HjxvVazl10wPAU0CuuqqurdeDAAS1cuFCVlZWKjY3ttY7P\n5+v57PV65fV6A+0RgAPLsmRZ1oDW7fde+ra2Ns2fP79nDn/lyhXFx8erq6tLGzZs0NSpU7V+/frb\nN8q99K7rbw7f3/x6KDz99NOO9b1797reA3ob0nvpExIS5PF4NGbMGFVVVengwYNBNwggNAYd+Pb2\ndknS9evXVVtbq6KioiFvCoA7HOfwS5Ys0fHjx9XZ2anU1FRt3rxZlmWpublZo0ePVkFBgSoqKkLV\nK4Ag8Tz8MNXff7S7d+92vYdvvvnGsZ6WluZ6D+iN5+EBSCLwgFEIPGAQAg8YhMADBiHwgEEIPGAQ\nfh8+QjU3NzvW6+rqXN1/cXFxv+twnX34YYQHDELgAYMQeMAgBB4wCIEHDELgAYMQeMAgPA8foRIS\nEhzrv/76a1Dbz8vLc6zf+Xryu4mJiQmqB7iD5+EBSCLwgFEIPGAQAg8YhMADBiHwgEEIPGAQnoeP\nUJ2dnY71qKiooLZfVVXlWOca+8jECA8YhMADBiHwgEEIPGAQAg8YhMADBiHwgEEcr8NfuHBBK1as\n0JUrVxQfH69Vq1Zp6dKl8vv9WrZsmZqamjRr1izt27eP67aDVFpa6ljv730CN27cCGr/s2fPDurf\nY3hyHOFHjRql7du3q6WlRR9++KE2btwov9+vHTt2aMKECTp79qxSUlK0c+fOUPULIAiOgU9KSlJW\nVpYkKS4uThkZGWpsbFRDQ4PKy8sVHR2tsrIynT59OiTNAgjOgG+tPXfunFpaWpSbm6vS0lKlp6dL\nktLT09XQ0NBrfZ/P1/PZ6/XK6/UG3SyA3izLkmVZA1p3QIH3+/1avHixtm/frpiYmAG9r+7WwANw\nz50D6ubNm/tct99v6bu7u7Vo0SItX75cJSUlkqScnBy1trZKklpbW5WTkxNkywBCwTHwtm2rvLxc\nmZmZWrt2bc/yvLw81dTU6Nq1a6qpqVF+fr7rjQIInmPg6+vrtW/fPn3xxRfKzs5Wdna2jhw5ooqK\nCp0/f15paWm6ePGiVq9eHap+AQTBcQ7/yCOP6J9//rlr7eOPP3aloZGiv993P3r0qGPd4/E41qOj\nox3rlZWVjvXExETHOkYm7rQDDELgAYMQeMAgBB4wCIEHDELgAYMQeMAgvJfeJb/99ptj/fLly0Ft\n//7773esb9u2LajtY2RihAcMQuABgxB4wCAEHjAIgQcMQuABgxB4wCAEHjAIgQcMQuABgxB4wCAE\nHjAIgQcMQuABgxB4wCA8D++Smz+22Zf+fp/95MmTQ9kOIIkRHjAKgQcMQuABgxB4wCAEHjAIgQcM\nQuABg3hs27b7Kl64cEErVqzQlStXFB8fr1WrVmnp0qXy+XzavXu34uPjJUlbtmxRYWHhvxv1eOSw\nWQAucsqfY+AvXbqkS5cuKSsrS52dncrNzdWZM2f0xhtvKDY2VuvWrRv0DgG4yyl/jnfaJSUlKSkp\nSZIUFxenjIwMNTY2ShKBBoahAd9ae+7cObW0tCgvL08nT55UdXW1Dhw4oIULF6qyslKxsbG3re/z\n+Xo+e71eeb3eoeoZwC0sy5JlWQNa1/GU/ia/3y+v16tNmzappKSkZ07f1dWlDRs2aOrUqVq/fv2/\nG+WUHgibgOfwktTd3a158+apqKhIa9eu7VU/c+aMKisrVV9fP6AdAnCXU/4cL8vZtq3y8nJlZmbe\nFvb29nZJ0vXr11VbW6uioqIhbBeAWxxH+FOnTqmgoEAzZsyQx+ORJL366qvav3+/mpubNXr0aBUU\nFGjjxo0aP378vxtlhAfCJqhT+qHeIQB3BXxKD2BkIfCAQQg8YBACDxiEwAMGIfCAQQg8YBACDxiE\nwAMGIfCAQUIS+IE+qxsu9Bcc+gtOKPsj8KK/YNFfcEZc4AFEBgIPGMS1x2MBhE9Ab60d6p0BCC9O\n6QGDEHjAIAQeMAiBBwxC4AGDEHjAIP8DN0RXkNApfJkAAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.matshow(ones[0].reshape(28, 28))  # change the 0 to another number to see some more zeros. Or try looking at some ones."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we generate some new data set.\n",
    "\n",
    "For the training labels y, we write as many 0's as the ``zeros`` array is long, the asame for ``ones``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_new.shape:  (14780, 784)\n",
      "y_new.shape:  (14780,)\n",
      "y_new:  [0 0 0 ..., 1 1 1]\n"
     ]
    }
   ],
   "source": [
    "X_new = np.vstack([zeros, ones])  # this \"stacks\" sevens and nines vertically\n",
    "print \"X_new.shape: \", X_new.shape\n",
    "y_new = np.hstack([np.repeat(0, zeros.shape[0]), np.repeat(1, ones.shape[0])])\n",
    "print \"y_new.shape: \", y_new.shape\n",
    "print \"y_new: \", y_new"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we shuffle them around and create a training and test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.utils import shuffle\n",
    "X_new, y_new = shuffle(X_new, y_new)\n",
    "X_mnist_train = X_new[:5000]\n",
    "y_mnist_train = y_new[:5000]\n",
    "X_mnist_test = X_new[5000:]\n",
    "y_mnist_test = y_new[5000:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us learn a logistic regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, penalty='l2', random_state=None, tol=0.0001)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logreg.fit(X_mnist_train, y_mnist_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and visualize the coefficients.\n",
    "\n",
    "We can see that the middle is dark, corresponding to  high positive values for where\n",
    "a 1 would be. Around it is a lighter circle, corresponding to the position of a 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.colorbar.Colorbar instance at 0xb325564c>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARQAAADtCAYAAAB+moocAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X9UVHX+P/DnoKCmpk5CoPxyhOYHmiDMjJsErZWiHEFX\nd11aqSO4TWhhGdZ62q0+52w/d1t/sKacY1S7hHu29XCiWmWxDg7awmCiFgIpQSD5A7BiNsnA3t8/\n+M49DNz3ZZi5MzjM63HOnAPvue973/cyvObe+7rv91vBGGMghBAZ+I12AwghYwcFFEKIbCigEEJk\nQwGFECIbCiiEENlQQCFEJkqlEgqFwqGXUqkc7ea6hYLSxoTIQ6FQjGj5sfivN360G0DIWOJoUBmL\nwQSggEKIrHw9oNA9FEJk5Og9FB6z2QytVovo6Gjk5+eLLrN9+3aoVCrEx8ejoaHBobpvvvkmtFot\nYmJi8PTTTwvlu3fvRnR0NHQ6HY4dO+bi3gNghBBZAGD+/v4OvXj/erGxsezo0aOspaWFqdVq1tHR\nYfd+dXU1W7x4Mevq6mLFxcUsNTV12LqfffYZW7RoEfviiy8YY4xduXKFMcbY5cuXmVqtZl999RWr\nqKhgcXFxLh8DOkMhREaunKF89913AICkpCRERERg6dKlqK6utlumuroaa9euhVKpREZGBurr64et\ne+jQIWRnZyM6OhoAEBgYKKwrJSUF4eHhSE5OBmMMVqvVpf2ngEKIjHgBhDGGGzduCC8xNTU10Gg0\nwu86nQ5VVVV2y1gsFuh0OuH3wMBANDU1SdYtKyvD559/joSEBGzcuBFnz54V1qXVaoU6arUaFovF\npf2ngEKIjHgBZdy4cfD39xdezmKMDbmhyzvjsZVfv34dV69eRWVlJdLT0/Hoo48K6+LVcRYFFEJk\n5Molj16vt7vJWldXh0WLFtktYzQahTMMAOjo6IBKpUJCQsKQukajEQCwaNEirFu3DpMmTcLKlSvR\n0NCAH374Yci6GhoaoNfrXdp/CiiEyMiVgDJt2jQA/dmalpYWlJeXC0HBxmg04uDBg+jq6kJxcbFw\nyTJ9+nRu3Z/97Gc4dOgQGGOorq7G3LlzMXHiRBgMBpSVlaG1tRUVFRXw8/PD1KlTXTsALt/WHcbR\no0eZRqNhUVFRbPfu3e7e3BARERFs/vz5LDY2lun1erdua8OGDSwoKIjNmzdPKOvu7mZpaWksLCyM\npaenM6vV6tHtP/fcc2z27NksNjaWxcbGskOHDrlt+62treyee+5hOp2OJScns3feeYcx5rljwNu+\np44BADZ58mSHXrx/vYqKCqbRaNjcuXPZrl27GGOM7du3j+3bt09Y5umnn2aRkZFs4cKF7OzZs5J1\nGWOsr6+PmUwmptFo2KpVq5jFYhHe27lzJ5s7dy7TarXMbDa7fgxcXsMwhkuDuVtkZCTr6uryyLbM\nZjM7efKk3T/0K6+8wh599FH2ww8/sM2bN7M//elPHt3+888/z1577TW3bXOgixcvstraWsYYYx0d\nHWzOnDmsu7vbY8eAt31PHQMAbOrUqQ69PPBdPircesnjSBrME5iHnkq8++67MWPGDLsyi8WC7Oxs\nTJgwAVlZWW7df7HtA57b/+DgYMTGxgIAZs6ciZiYGNTU1HjsGPC2D3juGLj6YJu3c2tAcSQN5m4K\nhQJLlizBqlWrUFpa6tFtA/bHQKPRuJyWc0Z+fj4WLVqEV155xeXnDBx1/vx51NXVwWAwjMoxsG3f\ndh/BU8eAAsoYd/z4cZw+fRovvfQStm7dikuXLnl0+576ZuTJyclBc3MzysrK0NTUhIKCArdv02q1\nYt26ddixYwemTJni8WMwcPuTJ0/26DGggOJGjqTB3C0kJAQAoNVqkZaWhvfff9+j29fr9cLTjPX1\n9S6n5UYqKCgICoUC06ZNw+bNm1FSUuLW7fX29mLNmjXIzMxEeno6AM8eA7Hte/IYUEBxI0fSYO50\n7do14fS2o6MDZWVlSElJ8dj2gf40X2FhIXp6elBYWOjxgHrx4kUAQF9fH4qLi7FixQq3bYsxhuzs\nbMybNw+PP/64UO6pY8DbviePga8HFLffaualsjzhyy+/ZAsWLGALFixgS5YsYW+88YZbt/frX/+a\nhYSEsICAABYaGsoKCws9mja2bd/f35+FhoayN954g2VmZrL58+ez+Ph49sQTT7g141VZWckUCgVb\nsGCBXYrWU8dAbPv//ve/PXYMALDbbrvNoZcH/vVGBY3YRohMFAqF0PFuOB0dHaN+f80daIAlQmQ0\npi9nHEABhRAZUUBxktlshslkQl9fH3Jzc/HYY48J7/n6QSVjy0guTXz9s+/0PZS4uDjs2rULERER\nWLZsGY4dO4aZM2f2r1ShwP79+4VlS0tLkZaWJk+LnTCa2/flfR8L29+4caPDAUWhUAiPKQzn4sWL\nY/IeilNp45vlkXpCbja+njZ26pKH90h9amqqUDbwMffOzk4XmkiI5zQ2NqKxsdHp+mM5WDjCbTdl\nB55muvIHkoNarfbJbdP2R759tVptV2ek/b/8/MZ8bxZJTu39SB+p97YP1VjZNm3f89v39UsepwLK\naD9ST8jNytcDitOXPDt37oTJZEJvby9yc3OFDA8hvmwsBwtHOB1QkpOThR6khJB+FFAIIbKhgEII\nkY2vZ3kooBAiIzpDIYTIhgIKuWlI9e3o7e0VLedNa+nrH+zR4uvH3bcv+AiRmavPoZjNZmi1WkRH\nRyM/P190me3bt0OlUiE+Pt7uAdPh6r722mvw8/PD1atXAQAtLS2YNGkS4uLiEBcXh02bNrm493SG\nQoisXD1D2bJlCwoKCoRe/BkZGXbPeFksFlRWVuLEiRMoKytDXl4ePvjgg2HrtrW1oby8HBEREXbb\ni4qKQm1trUttHojOUAiRkStnKI704q+ursbatWuhVCqRkZEhPAs2XN2tW7fi1Vdfdccu26GAQoiM\n/Pz8RF89PT3o7OwUXmIcmRjPYrFAp9MJvwcGBqKpqUmy7nvvvYfQ0FDceeedQ7bZ3NyM2NhYmEwm\nnD592qV9B+iShxBZ8c4+pkyZgilTpgi/X7lyxan1s/75yB3apkKhQE9PD1588UWUl5fbrQMAZs2a\nhba2NsyYMQOHDh1CZmYmzpw541S7bCigjIKffvppROUAcP36ddFy26nuYNeuXeOu6/vvvx/RNgBw\nR3MPCgri1hk/XvzjJfUP4O1c2Qe9Xo9t27YJv9fV1Q2ZR8poNOLs2bNYtmwZgP7R81UqFZRKpWjd\npqYmtLS0YMGCBQCACxcuID4+HhaLBUFBQQgICAAALF++HM888wzOnz+PqKgop/eBLnkIkZEr91Ac\n6cVvNBpx8OBBdHV1obi4GFqtFgAwffp00brz5s3D5cuX0dzcjObmZoSGhuLkyZMICgpCZ2cnbty4\nAQA4efIkenp6XAomAJ2hECIrV8+yxHrx2+ZiNplMMBgMSExMREJCApRKJYqKiiTrSrXPbDbj2Wef\nxfjx4xEVFSXLnM9umehr8CDVxJ4zlzw9PT2i5T/88INoOV3yyGOkg1THxcU5tGxtbe2YHKSazlAI\nkdHNGBQ9iQIKITKi3saEENnQGQoR8K5ppT4ktrvkg/HuHwDAjBkzRMt5KWAAsFqtI9p+c3Mzd12D\nH7+2ue2220a8fV45wD+evDbzOjoCsHuGYyBb2vNmQQGFECIbCiiEENlQQCGEyIYCCiFENhRQCCGy\nobSxkyIjI3Hrrbdi3Lhx8Pf3h8VikbNdo4L37SL1Ifn2229Fy0NDQ7l1eJOijRs3jltn6tSpouW8\np16Dg4O56woJCREtl3pSl9c7lrd9gP/kLS/L880333DX1dXVJVpu68Mihvd0rzvRGYqTFAoFKioq\noFQq5WwPIV6NAooLxmJfBEJcQQHFSQqFAkuWLMGcOXOQlZWFtLQ0u/dLS0uFn9VqNdRqtfOtJMRD\nGhsb0djY6HR9CihOOn78OEJCQlBfX4+VK1fCYDDYXbcPDjCEeIPBX34Dvxgd4esBxelb0rYbe1qt\nFmlpaXj//fdlaxQh3srVaTS8nVNnKNeuXcONGzcwdepUdHR0oKysDE888YTcbXMbXvaBl+X48ccf\nuevifTgmTZrErdPX1ydazhvbBOBnRlpbW0XLpbJMvO1ERkZy6/CyWZ999hm3Tnt7u2i52GDJABAd\nHc1dF+9vINX/ZzRQ2tgJly9fxurVqwH0dyh78sknERYWJmvDCPFGY/nswxFOBZQ5c+bg1KlTcreF\nEK9HAYUQIhsKKIQQ2VBAIYTIhgIKIUQ2lOXxclevXhUt56VmgZFP43DLLbdw1zV79mzR8smTJ3Pr\n8KbE6O3t5dbhDRt5++23i5bzUrYAoFKpRMs7Ojq4deLj40XLeelsgD9sI2+oyfnz53PXxUt18zpN\nAvyUulQnRFf5+hmKb4dTQmTm6oNtZrMZWq0W0dHRyM/PF11m+/btUKlUiI+PR0NDw7B1//CHP2DB\nggWIjY1FZmamXc/t3bt3Izo6GjqdDseOHXN5/ymgECIjVwPKli1bUFBQgCNHjmDPnj3o7Oy0e99i\nsaCyshInTpxAXl4e8vLyhq371FNP4fTp0zh16hSio6Oxa9cuAP1DUrz++uv46KOPsHfvXuTm5rq8\n/xRQCJGRKwHFNutBUlISIiIisHTpUlRXV9stU11djbVr10KpVCIjIwP19fXD1rVdFvb19eH777/H\nxIkThXWlpKQgPDwcycnJYIxJzmLgCAoohMiIF0C6urqEnsy83sw1NTXQaDTC7zqdDlVVVXbLWCwW\n6HQ64ffAwEA0NTUNW/eZZ55BcHAwjh07hm3btgnrsk22DvR3jHR1oDQKKITIiBdQAgMDodVqhZez\nGGNDxiFy5EbwCy+8gNbWVhgMBjz11FPCusTa7wqvz/LwDoBU9sF2yufoum699VbuunhDLfIyOQCG\nXBfbSA1neOHChRFtRyrLxdsfqYnPL168OOLt8DJdvKEZpbJMvGMm9bcZjY6DrqSN9Xq9cPYAAHV1\ndUhJSbFbxmg04uzZs1i2bBmA/mOmUqmgVCqHrXvLLbcgKysLv/3tb4V1HTlyRHi/oaEBer3e6fYD\ndIZCiKxcuYcybdo0AP3ZmpaWFpSXl8NoNNotYzQacfDgQXR1daG4uFg427F9GYnVPXfuHID+4H/g\nwAH84he/AAAYDAaUlZWhtbUVFRUV8PPzk0zDO8Lrz1AIuZm4esmwc+dOmEwm9Pb2Ijc3FzNnzkRB\nQQEAwGQywWAwIDExEQkJCVAqlSgqKpKsC/SnmRsbGzFp0iTcc889whnK7bffjpycHCxZsgQBAQHC\ndlyhYG4YGFahUGD//v1yr1YU7yElqbFFRnrJwxulHuCP4SF1ydPW1iZaLnXJw7vk4G2HN34JACQk\nJIiWS42UzzvOUnMo8y5hoqKiRMulHgaU85KHty4xGzdudHjsZIVCgV/96lcOLfvPf/5zTI7JTGco\nhMjI15+UpYBCiIwooBBCZEMBxcvx0pZSKUNeh7pr166Jlkt9SHjX41LXx7wOjVJPKfJm6OPd35Ea\nU5Z3P0ZqHFypWQ15eM9b8P5mUmljXkc/XgdEQHqMXHeh3saEENnQGQohRDYUUAghsqGAQgiRDQUU\nQohsKKBIyMrKwocffoigoCBhhjir1Yr169ejtrYWCxcuRFFRkeSd9tEi1SeB93SlrS/FYFJDM/Ke\nVOU9DQvwMxNSH0ZeZ8dLly6Jlg8clWswXjZLqnMgL8sTEBDArcPLtI0fL/6xk8qM8YbAlPrb8GaI\ndCdfDyiSOa4NGzbg8OHDdmV79+5FeHg4zp07h9DQUOzbt8+tDSTEm/j5+Tn0Gqsk9+zuu+8e8s1g\nsViQnZ2NCRMmICsra8iIUoT4MposfYQGjgyl0Wi4IzyVlpYKP6vVaqjVaiebSIjnSI2o5oixHCwc\nMeKA4mgPybS0tBE3hpDRNvjLb+AXoyN8PaCM+GJOr9cLA+PW19e7PMITIWMJXfKMkNFoRGFhIV59\n9VUUFhZi0aJF7miXw3g3uHhZAYCfAeL9oaXGNnEmy8Kb0EoqM8W7tORlOaQyHLyJy86fP8+tM3fu\nXNHyWbNmcevwsmm8icak+jLxLkOkJu2iLI/nSZ6hZGRk4K677sIXX3yBsLAwvPnmm8jJyUFrayvU\najXa29vxyCOPeKqthNz06AxFwoEDB0TL33vvPbc0hhBvN5ZTwo6gJ2UJkdFYPvtwBAUUQmREAYUQ\nIhsKKIQQ2VBA8XK8VKtUCpY3jcbly5dFy6Ue5uOti5caBfjTSPCGZpRqw8cffyxaLjVkY0REhGj5\nbbfdxq2zfPly0fKvv/6aW0epVIqW89K53d3d3HXxhqf83//+x63z448/ct9zFwoohBDZ+HqWx7f3\nnhCZufocitlshlarRXR0NPLz80WX2b59O1QqFeLj49HQ0DBs3XfffRcxMTEYN24cTp48KZS3tLRg\n0qRJiIuLQ1xcHDZt2uTy/tMZCiEycvWSZ8uWLSgoKEBERASWLVuGjIwMu5kNLBYLKisrceLECZSV\nlSEvLw8ffPCBZN358+ejpKQEJpNpyPaioqJQW1vrUpsHojMUQmTkyhnKd999BwBISkpCREQEli5d\nOmR4kOrqaqxduxZKpRIZGRlCvzqpuhqNBnfccYe7dtkOBRRCZMQLIG1tbfjkk0+El5iBQ4MAgE6n\nQ1VVld0yFosFOp1O+D0wMBBNTU0O1RXT3NyM2NhYmEwmnD59eqS7O4TXX/LwhlOcMGECt85PP/0k\nWs775uBNTAXwMxYGg4FbJywsTLT8yy+/5NbhfTh4E4BJZTh42+d1WgT4E4e1t7dz6/AyQLyJ7KWG\nEuVlraSyWbwbpLy/vxx4n6GIiAi77Nrx48edWj9jbEjGz9nLrFmzZqGtrQ0zZszAoUOHkJmZiTNn\nzji1Lhs6QyFERq5c8uj1erubrHV1dUN68xuNRpw9e1b4vaOjAyqVCgkJCcPWHSwgIEDolb98+XKM\nHz9esse5IyigECIjV8aUtQ2Sbjab0dLSgvLychiNRrtljEYjDh48iK6uLhQXFwvTvU6fPn3YuoD9\n80ydnZ3C4OcnT55ET08P9xkpR3n9JQ8hNxNXszw7d+6EyWRCb28vcnNzMXPmTBQUFAAATCYTDAYD\nEhMTkZCQAKVSiaKiIsm6AFBSUoLc3Fx0dnYiNTUVcXFxOHToEI4ePYrnnnsO48ePR1RUlLAdV1BA\nIURGrgaU5ORkIXNjMzjd+/LLL+Pll192qC4ArF69GqtXrx5SvmbNGqxZs8al9g5GAYUQGdGj94QQ\n2VBA8XK8meuk/rC8VOvkyZNFy6VmAeR1WpMaa5XXZqlxaAfe2R+Ity9z5szhrovXoS46Oppbhzer\nYmRkJLcOr7Mjb/u84w/wZ07klQM3V9rYV3h9QCHkZkIBhRAiG1/vbUwBhRAZ0RkKIUQ2FFAIIbKh\ngCIhKysLH374IYKCgvDZZ58BAJ5//nns378fgYGBAICXXnoJKSkpbm2k1BCMvMyA1J183vp4GRup\nrt+8zmlSHd14WSOp3p6ff/65aPl9990nWs7rgAeA28fD9hi3mObmZtFy3iyEAD8zxBtqU+pv5u/v\nL1ou9Q/s6DzccvL1gCJ5B2nDhg04fPiwXZlCocDWrVtRW1uL2tpatwcTQryJr88cKBlQ7r77btE5\ngkcj8hPiDXw9oDh1DyU/Px/vvvsuVq9ejU2bNomOMF9aWir8rFaroVarnW8lIR7S2NjInZjdEZQ2\nHqGcnBw8++yz6O7uxrZt21BQUIC8vLwhy6WlpcnSQEI8afCX38AvRkeM5bMPR4w4nAYFBUGhUGDa\ntGnYvHkzSkpK3NEuQrwSXfKM0MWLFxESEoK+vj4UFxdjxYoV7miXHak/AO8UU6oOLzPBW5fUBFi9\nvb2i5bz+OgDwn//8R7R84Ihbg/EyM7zhHC9dusRdF6/Pzrfffsut40y/mKtXr4qWjx8v/rGTGs6R\nN6Sl1P08yvJ4nmRAycjIwNGjR9HZ2YmwsDD83//9HyoqKnDq1CkEBAQgKSkJOTk5nmorITc9CigS\nDhw4MKQsKyvLbY0hxNtRQCGEyIYCCiFENpQ2JoTIhs5QCCGyoYDi5XhpS15HP4CftuQNpyj1IRHr\nmgAATU1N3Dq894KDg7l1Bk6YPRAvbSyVtr5y5YpoudQMid98841ouVQnRN4QkBMnThQtl5rtkPe3\nkWozpY09z+sDCiE3E18PKL59B4kQmbn6pKzZbIZWq0V0dDTy8/NFl9m+fTtUKhXi4+PtHobk1bVa\nrUhPT0d4eDhWrVplN0j47t27ER0dDZ1Oh2PHjrm8/xRQCJGRK1ORAsCWLVtQUFCAI0eOYM+ePejs\n7LR732KxoLKyEidOnEBeXp5dP7rBdW2zKOzduxfh4eE4d+4cQkNDsW/fPgD9l76vv/46PvroI+zd\nuxe5ubmu77/LayCECFw5Q/nuu+8AAElJSYiIiMDSpUtRXV1tt0x1dTXWrl0LpVKJjIwMYaZAsbpV\nVVUA+oNQdnY2JkyYgKysLGGd1dXVSElJQXh4OJKTk8EYg9VqdWn/KaAQIiNeAPniiy/w/vvvCy8x\nNTU10Gg0wu86nU4ICjYWiwU6nU74PTAwEE1NTZJ1B76n0WhgsVgA9AeUgaP0qdVq4T1nef1NWd5d\n/gkTJox4XdeuXRMt/+qrr7h1eBkjqTE1eMND8jrTAcDtt98uWs7LjEidVtuG7xyMt/8AP2PDy74A\n/A6K06dPFy2XajNvQi9e50yAnwF0541T3ro1Go3dPzwvqAyHMTYke8Xbpq18JNkuV48NnaEQIiNX\nLnn0er3dTda6urohvcyNRqPdLJIdHR1QqVRISEgYUtdoNArrtV0a1dfXQ6/Xi66roaFBeM9ZFFAI\nkZErAcU2qLfZbEZLSwvKy8uFoGBjNBpx8OBBdHV1obi4WLhksZ31idU1Go0oLCxET08PCgsLhSBl\nMBhQVlaG1tZWVFRUwM/PT3T0xZHw+kseQm4mrl4y7Ny5EyaTCb29vcjNzcXMmTNRUFAAADCZTDAY\nDEhMTERCQgKUSiWKiook6wL9oyyuX78earUaCxcuxCuvvAKg/zI6JycHS5YsQUBAgLAdVyiYGx4n\nVCgU2L9/v9yrFcUb/Gj27NkjXhfvmp/3ZCfg3D0U3r2S1tZWbh3ek7Ij3QYA3HvvvaLlKpWKW6el\npUW0XM57KEqlkrsu3j0U3sTrUnVG8k+/ceNGh+9BKBQKvP322w4t+9BDD43Jwd7pDIUQGfn6k7Je\nH1B4mQHeBGAA/4zj66+/Fi3v6OjgrsuW/x+M18cG4Gd5pDJTvG/v2tpa0XKp4Rx539yDH6IaiHec\npfry8IZ07OnpGfG6eHWk+vI4MzyoqyigEEJkQwGFECIbCiiEENlQQCGEyIYCCiFENjSmLCFENnSG\nIqGtrQ0PPvggrly5gsDAQDz88MN44IEHYLVasX79etTW1mLhwoUoKiripkLdjdehrb29nVuH9y3C\nS3N2d3dz18VLT4eGhnLr8FK6Uh9G3oNyZ86cES2/6667uOviDQEp9ZAYLz3LS0ED/M55vDq81DAA\nXL9+fcTbl3og0V18PaBInp/5+/tjx44dqKurw7/+9S/8/ve/h9Vq5Q7YQoiv8/W5jSUDSnBwMGJj\nYwH0P/odExODmpoa7oAthPg6Xw8oDt9DOX/+POrq6mAwGLBhwwbRAVsGKi0tFX5Wq9VQq9UyNJcQ\n92psbJTshzWcsRwsHOFQQLFarVi3bh127NiBKVOmONSpKS0tzeXGEeJpg7/8Bn4xOsLXA8qwOa7e\n3l6sWbMGmZmZSE9PB8AfsIUQX+fqINXeTvIMhTGG7OxszJs3D48//rhQbhuw5dVXX7UbsGU08LI8\nUh3deIPI8IYTlPrWmTVrlmh5UFAQt05ISIhoudTkYLyJy3jZnKioKO66nNlP3lCTUtm0L7/8UrTc\nNpDQYFJt5u2/1NkyL2sk1XHUVXSGIuH48eMoKirCxx9/jLi4OMTFxeHw4cPIyclBa2sr1Go12tvb\n8cgjj3iqvYTc1OimrITExETuswTvvfeeWxpEiDcby8HCEfSkLCEyooBCCJENBRRCiGwooBBCZDOW\nU8KO8PqAwpvVT6pD35133ilaHhwcLFouNbL7iRMnRMv9/f25dXgj0t9xxx3cOrxvvhkzZoiW88bH\nBfgpWKkxXXlz3kp1zouMjBQt57VZ6pjx0sMBAQHcOqOBzlAIIbKhgEIIkY2vBxTfvuAjRGbuerDN\narUiPT0d4eHhWLVqFXfsGrPZDK1Wi+joaOTn5w9b/+rVq/j5z3+OqVOn4rHHHrNb1z333AONRiM8\n1Cr19LkNBRRCZOSugOLoGERbtmxBQUEBjhw5gj179qCrq0uy/sSJE/HHP/4Rf/7zn0X3pbi4GLW1\ntaitrXVo9koKKITIiNcZ8MyZM/jb3/4mvEbKkTGIbJPOJSUlISIiAkuXLkVVVZVk/VtuuQWLFy/m\nTjI30ulSvf4eCq+jmVTGgDec4OXLl0XLpTJGvI52vE6LAD/LJDW3Ma8NvLmdpWY75B0bqVn4eJ0d\npeYj5uENtSnFW+5N8Nppu2yweeutt0a03pqammHHIBq4DADodDpUVVUhNTV12Pq8dj/00EMICQnB\n+vXr8dBDDw3bTq8PKITcTFwJfPfff7/oBPMvvPCC0xOr29rjTP133nkHs2bNwldffYVf/vKXiImJ\nQUJCgmQdCiiEyMiVgFJeXs597+2330Z9fT3i4uK4YxDp9Xps27ZN+L2urg4pKSnCe8PVH8w2NEdE\nRATWr1+PkpKSYQMK3UMhREbuuilrG4Oop6eHOwaR7fLfbDajpaUF5eXlMBqNDtUffAZz48YNIavT\n3d2NkpISrFixYth2UkAhREbuCii8MYi+/vprpKamCsvt3LkTJpMJ9913HzZt2iRkZqTGMIqMjMST\nTz6Jt956C2FhYWhoaMD169eRkpKCBQsWIDU1FcuXL8fixYuHbSdd8hAiI3fdPJ46daroGESzZs3C\nhx9+KPxqwRt2AAAF2UlEQVSenJwsDM/qSH0AaGlpES3ndSuR4vUBhZfl4Q2zCEj3PxFz6623ct/j\n5eZ5mSQAOHXqlGh5c3Mztw5vqMX58+eLlks9hDR79mzRct5gWgC/09v48V7/EZIVdQ4khMjGW9Lb\n7kIBhRAZUUAhhMiGAgohRDYUUAghsqGAQgiRDQUUCW1tbXjwwQdx5coVBAYG4uGHH8YDDzyA559/\nHvv370dgYCAA4KWXXhIe8b1ZSHXO4+HNjieVZualWqXSxrxOeGfOnOHW4c12xxsXQ2r7vOEceUND\nAs71BfFFlDaW4O/vjx07diA2NhadnZ0wGAxYuXIlFAoFtm7diq1bt3qqnYR4BTpDkRAcHCwM3Dxz\n5kzExMSgpqYGAH1jESKGAoqDzp8/j7q6OhiNRlRWViI/Px/vvvsuVq9ejU2bNg2ZgLy0tFT4Wa1W\nQ61Wy9dqQtyksbERjY2NTtengOIAq9WKdevWYceOHZg8eTJycnLw7LPPoru7G9u2bUNBQQHy8vLs\n6qSlpbmlwYS40+Avv4FfjI7w9YAy7B2k3t5erFmzBpmZmUhPTwfQP3qXQqHAtGnTsHnzZpSUlLi9\noYR4A3f1NvYWkmcojDFkZ2dj3rx5ePzxx4XyixcvIiQkBH19fSguLnZonARv4KlJo3hDMN57770j\nXldPT49o+cSJE0e8LuK6sRwsHCEZUI4fP46ioiLceeedwniYL774Ig4cOIBTp04hICAASUlJyMnJ\n8UhjCbnZUdpYQmJiouhzFsuXL3dbgwjxZnSGQgiRDQUUQohsKKAQQmRDAYUQIhsKKMSr8Tr0SXX0\nI+7j6wHFIzkuVx5l9vbt+/K+++L2eXMbD36NVRRQxvC2afue376vPyk7dkMlIaPAXQHFarUiPT0d\n4eHhWLVqFXccHLPZDK1Wi+joaOTn5w9b32KxIC4uDrGxsbj33ntx+PBhoU59fT0WLlwIlUqFZ555\nxqF2UkAhREbuCih79+5FeHg4zp07h9DQUOzbt090uS1btqCgoABHjhzBnj170NXVJVl//vz5+PTT\nT3Hq1CkUFBQgNzdXGJrkySefxNNPP42amhocPXrUsYm/mBsAoBe9xsxrJJ/7CxcuOPQayXoZY2zN\nmjWstraWMcbYp59+ytauXTtkmW+//ZbFxsYKvz/22GPsgw8+cLj+yZMnmUajEX5XqVTCz6+99hr7\n61//Omw73ZLlYTT4EvFRvLOPTz75BP/973+dXm9NTQ00Gg0AQKPRwGKxSC4DADqdDlVVVUhNTZWs\nb7FY8Jvf/AYXLlxAZWUlgP7xj4KCguzW9c4772Dz5s2S7aS0MSEy4gWUxYsX2002/pe//GXIMvff\nfz8uXbo0pPyFF15w+kva1h6p+gaDAefOncOxY8ewcuVKtLe3D1ne0e1TQCFERq6khMvLy7nvvf32\n26ivr0dcXBzq6+uh1+uHLKPX67Ft2zbh97q6OmHweL1eP2z9xMREzJ49G+fOnYNarcbly5eF986e\nPYtFixYNuw90U5YQGbnrpqzRaERhYSF6enpQWFgo+s89bdo0AP2ZnpaWFpSXl8NoNErWb2lpEWZh\nOHPmDK5fvy6MWKfRaPCPf/wDnZ2dKCkpEdYlafjbQYQQRwBgV65cceg10n+97u5ulpaWxsLCwlh6\nejqzWq2MMcba29vZihUrhOUqKiqYRqNhc+fOZbt27Rq2/t///ncWExPDYmNj2bp161hlZaVQp66u\njsXFxbHIyEj2u9/9zqF2Kv7/gSCEuEihUKCjo8OhZQMDA8dk8oLuoRAio7H8FKwjKKAQIiMKKIQQ\n2Yzljn+OoIBCiIzoDIUQIhsKKIQQ2VBAIYTIhgIKIUQ2FFAIIbKhgEIIkQ2ljQkhsqEzFEKIbCig\nEEJkQwGFECIbCiiEENlQQCGEyIYCCiFENuPGjXNouRkzZri5JaODAgohMhmLI7CNlG8/hUMIkRUF\nFEKIbCigEEJkQwGFECIbCiiEENlQQCGEyOb/AX+QUIg/zZM4AAAAAElFTkSuQmCC\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.matshow(logreg.coef_.reshape(28, 28))\n",
    "plt.colorbar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let as look at the accuracy on training and test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy training set: 1.0\n",
      "Accuracy test set: 0.998466257669\n"
     ]
    }
   ],
   "source": [
    "print \"Accuracy training set:\", logreg.score(X_mnist_train, y_mnist_train)\n",
    "print \"Accuracy test set:\", logreg.score(X_mnist_test, y_mnist_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Playing around with this notebook\n",
    "=================================\n",
    "1. What would be the accuracy of a completely random predictor on the first dataset?\n",
    "\n",
    "2. Can you still pick out the class yourself if you set the ``cluster_std`` to ``10`` in ``make_blobs``? Can logistic regression do it?\n",
    "\n",
    "3. Try to separate some other digit classes from MNIST. Which are hard, which are not?\n",
    "\n",
    "4. Visualize the classes 0 and 1 using PCA down to two dimensions. Use a scatter plot as above. Would you expect they can be separated with a linear classifier?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}