machinelearning_notebook/6_pytorch/1_NN/2-logistic-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic 回归模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上一节课我们学习了简单的线性回归模型，这一次课中，我们会学习第二个模型，Logistic 回归模型。\n",
    "\n",
    "Logistic 回归是一种广义的回归模型，其与多元线性回归有着很多相似之处，模型的形式基本相同，虽然也被称为回归，但是其更多的情况使用在分类问题上，同时又以二分类更为常用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型形式\n",
    "Logistic 回归的模型形式和线性回归一样，都是 y = wx + b，其中 x 可以是一个多维的特征，唯一不同的地方在于 Logistic 回归会对 y 作用一个 logistic 函数，将其变为一种概率的结果。 Logistic 函数作为 Logistic 回归的核心，我们下面讲一讲 Logistic 函数，也被称为 Sigmoid 函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid 函数\n",
    "Sigmoid 函数非常简单，其公式如下\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$\n",
    "\n",
    "Sigmoid 函数的图像如下\n",
    "\n",
    "![](https://ws2.sinaimg.cn/large/006tKfTcly1fmd3dde091g30du060mx0.gif)\n",
    "\n",
    "可以看到 Sigmoid 函数的范围是在 0 ~ 1 之间，所以任何一个值经过了 Sigmoid 函数的作用，都会变成 0 ~ 1 之间的一个值，这个值可以形象地理解为一个概率，比如对于二分类问题，这个值越小就表示属于第一类，这个值越大就表示属于第二类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外一个 Logistic 回归的前提是确保你的数据具有非常良好的线性可分性，也就是说，你的数据集能够在一定的维度上被分为两个部分，比如\n",
    "\n",
    "![](https://ws1.sinaimg.cn/large/006tKfTcly1fmd3gwdueoj30aw0aewex.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，上面红色的点和蓝色的点能够几乎被一个绿色的平面分割开来"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 回归问题 vs 分类问题\n",
    "Logistic 回归处理的是一个分类问题，而上一个模型是回归模型，那么回归问题和分类问题的区别在哪里呢？\n",
    "\n",
    "从上面的图可以看出，分类问题希望把数据集分到某一类，比如一个 3 分类问题，那么对于任何一个数据点，我们都希望找到其到底属于哪一类，最终的结果只有三种情况，{0, 1, 2}，所以这是一个离散的问题。\n",
    "\n",
    "而回归问题是一个连续的问题，比如曲线的拟合，我们可以拟合任意的函数结果，这个结果是一个连续的值。\n",
    "\n",
    "分类问题和回归问题是机器学习和深度学习的第一步，拿到任何一个问题，我们都需要先确定其到底是分类还是回归，然后再进行算法设计"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 损失函数\n",
    "前一节对于回归问题，我们有一个 loss 去衡量误差，那么对于分类问题，我们如何去衡量这个误差，并设计 loss 函数呢？\n",
    "\n",
    "Logistic 回归使用了 Sigmoid 函数将结果变到 0 ~ 1 之间，对于任意输入一个数据，经过 Sigmoid 之后的结果我们记为 $\\hat{y}$，表示这个数据点属于第二类的概率，那么其属于第一类的概率就是 $1-\\hat{y}$。如果这个数据点属于第二类，我们希望 $\\hat{y}$ 越大越好，也就是越靠近 1 越好，如果这个数据属于第一类，那么我们希望 $1-\\hat{y}$ 越大越好，也就是 $\\hat{y}$ 越小越好，越靠近 0 越好，所以我们可以这样设计我们的 loss 函数\n",
    "\n",
    "$$\n",
    "loss = -(y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "其中 y 表示真实的 label，只能取 {0, 1} 这两个值，因为 $\\hat{y}$ 表示经过 Logistic 回归预测之后的结果，是一个 0 ~ 1 之间的小数。如果 y 是 0，表示该数据属于第一类，我们希望 $\\hat{y}$ 越小越好，上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = - (log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "在训练模型的时候我们希望最小化 loss 函数，根据 log 函数的单调性，也就是最小化 $\\hat{y}$，与我们的要求是一致的。\n",
    "\n",
    "而如果 y 是 1，表示该数据属于第二类，我们希望 $\\hat{y}$ 越大越好，同时上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = -(log(\\hat{y}))\n",
    "$$\n",
    "\n",
    "我们希望最小化 loss 函数也就是最大化 $\\hat{y}$，这也与我们的要求一致。\n",
    "\n",
    "所以通过上面的论述，说明了这么构建 loss 函数是合理的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们通过例子来具体学习 Logistic 回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7f089c0fccb0>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设定随机种子\n",
    "torch.manual_seed(2017)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们从 data.txt 读入数据，感兴趣的同学可以打开 data.txt 文件进行查看\n",
    "\n",
    "读入数据点之后我们根据不同的 label 将数据点分为了红色和蓝色，并且画图展示出来了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fb05e2923c8>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAH7RJREFUeJzt3X2MXNWZ5/HvY2Nj9Y4TwO6RIrdd3Vk5GwxBm7jNZCJlE4VlcZisvUlGI6BhgyYTh2xIRtpMFCIIioxayaxWMwpadiXnZWLcHRDLHyuPhsWMBlC0Uci6Ea82gRgP2O2MlE6b7IYELy959o9bDdXlerlddV/Ouff3kUquun1cdereW8899znnnmvujoiIVMuqsisgIiLZU3AXEakgBXcRkQpScBcRqSAFdxGRClJwFxGpIAV3EZEKUnAXEakgBXcRkQo6p6wP3rhxo4+Pj5f18SIiUXr00Ud/6e6j/cqVFtzHx8eZm5sr6+NFRKJkZi+mKae0jIhIBSm4i4hUkIK7iEgFKbiLiFSQgruISAX1De5m9j0z+4WZPd3l72Zmt5vZMTN70szel301MzQ7C+PjsGpV8u/sbNk1EhHJXJqW+/eBnT3+/lFga/OxB/hvw1crJ7OzsGcPvPgiuCf/7tmjAC8ildM3uLv7D4HTPYrsBu70xCPAeWb2jqwqmKmbb4bf/nb5st/+NlkuIlIhWeTcNwEnW17PN5edxcz2mNmcmc0tLCxk8NErdOLEypaLiESq0A5Vd9/n7pPuPjk62vfq2ext2bKy5RlRmr+etN2lTFkE91PA5pbXY81l4ZmehpGR5ctGRpLlOVGav5603aVsWQT3g8C/b46aeT/wf9z9nzJ43+xNTcG+fdBogFny7759yfKcKM1fnjJbztruUrY0QyHvAn4M/AszmzezT5vZDWZ2Q7PIfcBx4BjwbeA/5FbbLExNwQsvwO9+l/ybY2AHpfmL1BrMN26EP/3T8lrOMWx3pY0qzt1LeWzfvt3roNFwT8LL8kejUXbNqmVmxn1kpPO6LmO9D7rdZ2aSMmbJvzMz+dSv0/oaGcnv8yQ7wJyniLG6QjVnJaT5a6lTGqSTolrOg2z3IvP0ShtVX3WDeyDnnCWk+WspbdDOeWDUmwbZ7kUG3BjSRjIcS1r5xZucnPTcbtax1ARq/aWMjCiqVtj4eNLS7SX0XWDVqqTF3s4s6SLKUrf11WgkXVESLjN71N0n+5WrZstd55y10ykNsmYNbNgQzxlTkZdhKF1YfdUM7jrnrJ1OaZC/+Rv45S8LGxg1tCIDrtKF1VfNtIzOOSVSs7PJCeaJE0mLfXpaAVeWq3daRuecEqmCL8OQCqtmcNc5p4jUXDWDO6gJJEEKZISu1MA5ZVdApC7aR+guXaQEantI9qrbchcJjEboSpEU3EUKEtMIXaWP4qfgLlKQku4Vs2Kai74aFNxFChLLCF2lj6pBwV2kILGM0O2WJuo3d4+ERcFdpEAxjNDtliYyU2omJgruIpHLuvNzejoJ5O3clZqJiYK7SMTy6Pycmuo89TCEObJHOlNwF4nQUmv92mvz6fxsNDovD21kj3SXKrib2U4ze9bMjpnZTR3+3jCzfzCzJ83sYTMby76qEgONj85fa2u9m2Fb2LGM7JHu+gZ3M1sN3AF8FNgGXG1m29qK/WfgTne/BNgLfCPrikr4ND66GGnuFztsCzuWkT3SXZqW+6XAMXc/7u6vAncDu9vKbAMebD5/qMPfpWBltKA1ProY/VrlWbWwYxjZ00+dzyTTBPdNwMmW1/PNZa2eAD7RfP5xYL2ZbRi+ejKIslrQMV1eH7NerXK1sN9S9zPJrDpU/wL4kJk9BnwIOAW80V7IzPaY2ZyZzS0sLGT00dKurBZ0LJfXx65bPnxmJt4Wdh7qfiaZJrifAja3vB5rLnuTu//c3T/h7u8Fbm4u+1X7G7n7PnefdPfJ0dHRIaotvZTVglYnXDF65cPrnIZo/+7dOpxrcybp7j0fJHO+HwcmgLUkKZiL2spsBFY1n08De/u97/bt210SMzPujYa7WfLvzMxw79douCcnossfjcbwde0n6+8i6c3MuI+MLN/mIyP12AadvrtZeb+DPAFz3ie+evJVUxSCK4HngOeBm5vL9gK7ms//GPhZs8x3gHP7vaeCeyKPH2Sdf+R1VuZBvWzdvnt7gM/zd1BUwybT4J7HQ8E9kdcPUi3o+unWUjUru2b56/bdl35Lef8OimxQpQ3ulpQt3uTkpM/NzZXy2SFZtarzpd5myRA06W52NukcO3Ei6bSdnq53Z2K3PHOjkXS0VlnZ373IzzezR919sl85TT9QMo0wGUzdh7m1WupIfPHFsyf8qkuHdtmd+SEOA1ZwL1nZO2WsQhzmVsZIlfapCNzfCvB1GvNe9hW1QTbS0uRu8ngo5/4W5cdXLrT8clmd2HXuRA2Jcu4tlHOXYZSdY21XZH1a+xq6/XzVZ1O8ovqAlHOXSgstnVVUzrW9r6Eb9dkUL7S5eBTcJUpl51jbFZVzTTMjpPpsBBTcJWIhtZSKOpPodSYQwkFOwnFO2RUQqYKlYJp3znXLlrD6GiRcarmLZKSIM4nQ+hokXAruIhEJra9BwqW0jEhkpqYUzKU/tdxFRCpIwV1EpIIU3EVEKkjBXWqtzrelk2pTh6rU1tKl/EtXfC5NGwzqsJT4qeUutRXitMEiWVFwl9oK8QYLIllRcJfaCvIGCyIZSRXczWynmT1rZsfM7KYOf99iZg+Z2WNm9qSZXZl9VUWypUv5pcr6BnczWw3cAXwU2AZcbWbb2ordAtzj7u8FrgL+a9YVrSwN1yiNLuWXKkvTcr8UOObux939VeBuYHdbGQfe1nz+duDn2VWxwnSX59KFMG2wju+ShzTBfRNwsuX1fHNZq68D15rZPHAf8IVMald1Gq5Rezq+S16y6lC9Gvi+u48BVwIHzOys9zazPWY2Z2ZzCwsLGX30EMpuMmm4RunK3gV0fJe8pAnup4DNLa/HmstafRq4B8DdfwysAza2v5G773P3SXefHB0dHazGWQmhyaThGqUKYRfQ8V3ykia4Hwa2mtmEma0l6TA92FbmBHAZgJldSBLcA2ia9xBCk0nDNUoVwi6g43s4yj6Ly1rf4O7urwM3AoeAZ0hGxRwxs71mtqtZ7EvAZ8zsCeAu4Hr3XvdmD0AITSYN1yhVCLuAju9hCOEsLnPuXspj+/btXqpGwz3ZjssfjUa59ZLChLILzMwkn2mW/DszU+znDyrWencSyr6QBjDnKWJsfa9QzbPJVLXzu4oKpdUcwnDMlapaSzeEs7jMpTkC5PEoveXunk/TY2bGfWRk+eF/ZCSaZk2IrbE86xTi941BTC3dNGL6PqRsudc7uOchpr2kTYjHpRDrVDWDHODMOu/mZnnXNh8x7WcK7r3k2VyLeK8P8bgUYp2qZNCgVsXtEstZXNrgbknZ4k1OTvrc3FzxH9x+hwZIEq1ZjVIZH08SkO0ajSShGrBVq5KfaDuzJB9chhDrVCWD7q55/4ykOzN71N0n+5WrX4dq3oObQ+mlG0CIY65DrFOVDNqRqFG84atfcM+7WzzivT7E41KIdaqSbgfJVav6D/aKcZRPraTJ3eTxKC3nXsVkYYZCzDuGWKeq6JRzb3+E2rFYVyjn3oWShSLLzM4mWckTJ5LW+htvnF0mgi6j2lDOvZs0aRNdhBQNbarhtaZXunVSR30xT03Vr+Xej1r20ei0qdauhfXr4fTpJJ88Pa3NthIRD/aqDbXcBxXCVIGSSqdN9eqrsLhYjUviy6AO7HwVeaap4N6ukpNMVFOaTaLj8spEPNgreEXPx6O0TDudl0aj26ZqpwueJARZhRalZQaV9Xmpevxy02lTdaILniQERScFFNzbZXleWrV5UQPTvqk2bIA1a5aXUb5YQlH01dZKy+RJKZ7CtY7Z1mgZCUlWA/GUlgmBOmcLp0viz6bMYBiK7qxWcM+TZr0qXdUC20q/T9Uyg7Fvz0IbH2nmKAB2As8Cx4CbOvz9r4HHm4/ngF/1e8/K3qyjVUx3AEghtjleKrb6B/o+VZpKqWrbc1BkdbMOYDXwPPBOYC3wBLCtR/kvAN/r9761CO7u8UXELmL8YVUpsLkP9n0ivnfMWaq2PQeVNrinSctcChxz9+Pu/ipwN7C7R/mrgbtWdPpQZRVJAsd44W7VujwG+T7dMoDu8aU1Qt6evdJFZaWSzklRZhNwsuX1PPAHnQqaWQOYAB4cvmoSkpB/WN1s2dJ5sFKsXR6DfJ/p6bNHaCxZyr9DHG2OULdn+yiY1vUK3f+W9zrPukP1KuBed+8waSiY2R4zmzOzuYWFhYw/WvIUY99w1eZJGeT7tI7Q6CT0s69WoW7PXme1pZ7x9svbAH8IHGp5/VXgq13KPgZ8IE0+qDY594qIMefuXpkujzcN832qkH8PcXv2Wq95rHOyulmHmZ1DMgLmMuAUcBi4xt2PtJV7N3A/MOH93pSaXMRUMbpAKG66pi4fvdYrZL/OM7uIyd1fB24EDgHPAPe4+xEz22tmu1qKXgXcnSawS5wq0jdcW6GmNWLXa72Wus7TNO/zeCgtI1K8ENMaVdBrvWa9ztE9VCtIeZFgaFNIWdKmZdIMhZQQ9BpvpahSKG0KiYHmlolFjFcRVZQ2RRhin2cmbwrusYjxKqKK0qYoX14TolXpgKHgnoUi9ogYryKqKG2K8uVx9lS1GTQV3IdV1B4R+Ti2KrWIIt8UlZDH2VPl0m1phtTk8ajMUMgip6qLdBxbrFe39hLppqiMPH52sVzBi4ZCFmTVqmQfaGeWXO0jujJSMpfVLetaxbKf6jZ7RVECti91QErW8rhlXdXSbQruw6raHpEDHf8kD1lPh1H0PU7zpuA+rKrtEb0M2Cuq45/EokrzJ+kK1SxMTcW9F6QxxGWZS3/W5foixVGHqqQTS2+TSMWpQ1WypV5RkagouEs66hUViYqCu6SjXlGRqCi4Szp1GhXURZWmUJDilLXfKLhLekvjxA4cSF5fd11tolzVJpWSYpS539Q7uKsptnI1jXKVm1RKClHmflPf4F7TIHWWlR7gahrlNFhIBlHmfpMquJvZTjN71syOmdlNXcr8iZkdNbMjZvaDbKuZg5oGqWUGOcDltLeGfhKlwUIyiFL3m37TRgKrgeeBdwJrgSeAbW1ltgKPAec3X/9+v/ctfcrfWOb3zNMg86bmMNdqDFMCx1BHCU8e+w0pp/xN03K/FDjm7sfd/VXgbmB3W5nPAHe4+0vNA8Yvhj3o5E5NscFa4TkMiYzhJEqDhWQQZe43aYL7JuBky+v55rJW7wLeZWY/MrNHzGxnVhXMjcZtD3aAy2FvjSWfXaVJpaQ4Ze03WXWonkOSmvkwcDXwbTM7r72Qme0xszkzm1tYWMjoowekptjgB7iM91adRIlkL01wPwVsbnk91lzWah446O6vufs/As+RBPtl3H2fu0+6++To6Oigdc5O3ZtigRzgdBIlkr00wf0wsNXMJsxsLXAVcLCtzP8gabVjZhtJ0jTHM6yn5CWAA1wgxxiRSukb3N39deBG4BDwDHCPux8xs71mtqtZ7BCwaGZHgYeAL7v7Yl6VluoJ4BgjBQl92GtVaD53ESlMHje2rhvN515lavpI4LrtojEMe60K3WYvNkPc7k6kCL120ViGvVaB0jKx0e3uJHC9dlHQ7jsspWWqSk0fCVyvXVTDXouj4B4bXfEjgeu1i2rYa3EU3GOjpo8Ert8uqmGvxVBwj42aPhI47aJhUIeqiEhE1KEqIlJjCu4iIhWk4C4iUkEK7iIZ0+wQEgIFd8lPDaPcIPccF8mDgntd5R14Q4lyBR9gNDGWhEJDIeuoiHlXQ5gDp4T5ZVetSo5l7cySi3ZEhpV2KKSCex0VEXhDiHIlHGBCOKZJtWmcu3RXxORjIcyBU8Ika5odQkKh4F5HRQTeEKJcCQcYXXovoVBwr6MiAm/ZUW52Fl5++ezlBRxgNDGWhEDBvY56Bd4sR5eUFeWWOlIX2+7RvmGDmtFSG6mCu5ntNLNnzeyYmd3U4e/Xm9mCmT3efPxZ9lWlluOmc9Mp8IYyfHFYncYjAvze7ymwS230HS1jZquB54DLgXngMHC1ux9tKXM9MOnuN6b94BWPltFt0/NXlaEeIYzUEclJlqNlLgWOuftxd38VuBvYPWwFV0xXh+Sv2yiSTgE/ZCGM1BEpWZrgvgk42fJ6vrms3SfN7Ekzu9fMNnd6IzPbY2ZzZja3sLCwsprq3qH56xb8zOJKzYQwUqemlDkNR1Ydqn8LjLv7JcDfA/s7FXL3fe4+6e6To6OjK/sEtcbyNz2dBPJ27nGdIZU9UqemqtJlUxVpcu5/CHzd3a9ovv4qgLt/o0v51cBpd397r/dVzj1QnYL70nLlq6WHqnTZhC7LnPthYKuZTZjZWuAq4GDbh72j5eUu4JmVVDYVtcaK0Wh0Xq4zpCgVmSZR5jQsfYO7u78O3AgcIgna97j7ETPba2a7msW+aGZHzOwJ4IvA9bnUNs9x00oWJpSvroyi0yTKnAbG3Ut5bN++3YMxM+M+MuKe/AaSx8hIsryOZmbcGw13s+Tfz31u+eu6rpdu2tdXIOun0Vi+Sy89Go18Pk8/o2IAc54ixiq4uxf/K4iJfrG9lbh++h1TzDrv1mbl1UmGlza4a8pf0EUvvaiXrLeS1k+a8QXadNWkKX9XQsnCzmZnu1/ApF6yREm9iGmu6VP3yeCq0AWn4A76FXSy1DTspu4HviUlNQzSHFPqMMAsjyBcmfH6aXI3eTyCyrm7K1nYrls/hHLuy5WUc1c3UX6rPvR1izpUZSjdeuNAgb1dCQ2DvI8pMbR18grCZXREr0Ta4K4OVelMvXHBm51NcuwnTiRZoOnpbFIusVwMntc4iNB3/ep2qGaRZKtCb0ne1A8RvLyu6YtlAta8ujsqs+unad7n8RgoLZPFuajGbacXw7m5ZC7LtESeu1CeP+WQd30qmXNfaZKt0xYKvbdEpGRZ/USKaEe1/sQ3bEgeIQbkLFUzuK+kSdFtz+rWSRhKb4nEIeSm3ZCyCspFtqPqdEJezeC+kr2lW9nVq9Vyr5IygmwNIkmq1dqnUJGjTup0Ql7N4L6SH1WvoXwV/2HWRllBtk6RpJsU677I1RT68MUsVTO4u6dvqfXasyp8Sl0rZQXZOkWSblKs+yKPvXU63lY3uKdVg1Pn2isryIYQScpuoKRc90VVs04/dwV39/J/AJKvsoJs2ZGk7M93D+MA16YuP3cFd6m+ooNcKOPuQgisIRxgaiptcI/vClWRJUVOe9g+VeDiIrzyChw4kP0tH/sJ4WalU1PwqU/B6tXJ69Wrk9chzU9Qc5pbRiSNkCYcCaEusUxAU0Fp55ZJFdzNbCfwLWA18B13/2aXcp8E7gV2uHvPyN0puL/22mvMz89z5syZvnWKwbp16xgbG2PNmjVlV0WGFdLdukIIrCEcYGoqbXA/J8UbrQbuAC4H5oHDZnbQ3Y+2lVsP/Dnwk8GqDPPz86xfv57x8XHMbNC3CYK7s7i4yPz8PBMTE2VXR4a1ZUvnYFbGTUuWAngeU0KmFUJqSHpKk3O/FDjm7sfd/VXgbmB3h3K3AX8JDNzsPnPmDBs2bIg+sAOYGRs2bKjMWUjmYpuZM7SpAvOaEjIt3ZoyeGmC+ybgZMvr+eayN5nZ+4DN7v53w1aoCoF9SZW+S6ZivI9ZHe5ZtxKhHexWKrbGxQCGHi1jZquAvwK+lKLsHjObM7O5hYWFYT9aYhXLhOHtym4thyTmg12MjYsBpAnup4DNLa/HmsuWrAcuBh42sxeA9wMHzeyshL+773P3SXefHB0dHbzWS0o8+u7fv5+tW7eydetW9u/fX9jnVoLytdUQ68FukMZFjC39fgPhSTpdjwMTwFrgCeCiHuUfBib7vW+ni5iOHj2afiR/iRdRLC4u+sTEhC8uLvrp06d9YmLCT58+3bHsir5TXYRwEY7U10qnrQjsgi2yuojJ3V8HbgQOAc8A97j7ETPba2a78jjgpJLDqf3hw4e55JJLOHPmDL/5zW+46KKLePrpp88qd+jQIS6//HIuuOACzj//fC6//HLuv//+gT+3dmLP10rcVtoZHGkase9QSAB3vw+4r23ZrV3Kfnj4aqWQw6n9jh072LVrF7fccguvvPIK1157LRdffPFZ5U6dOsXmzW9lqsbGxjh16tRZ5aSLEIbySX1NT3e+TqBb4yLSNGKq4B6knMYd33rrrezYsYN169Zx++23D/Ve0sPUlIK5lGOljYuQrnFYgXjnlsnp1H5xcZGXX36ZX//6113HqG/atImTJ98aHTo/P8+mTZs6lhWRAK2kMzjSNGK8wT2noVif/exnue2225iamuIrX/lKxzJXXHEFDzzwAC+99BIvvfQSDzzwAFdcccVQnysigYp02Ge8aRnI/NT+zjvvZM2aNVxzzTW88cYbfOADH+DBBx/kIx/5yLJyF1xwAV/72tfYsWMHkKRyLrjggszqISKBiTCNGNSskM888wwXXnhhKfXJSxW/00BmZ9WBKpKBzCYOExla+yyGS1cEggK8SE4U3Ht46qmnuO6665YtO/fcc/nJTwae+LKeeo0TVnAXyYWCew/vec97ePzxx8uuRvwiHScsErN4R8tIPDQ9rEjhFNwlf5GOExaJmYK75C/SccIiMVPOXYoR4ThhkZhF3XIvc4rlnTt3ct555/Gxj32suA8VEUkp2uBe9s1UvvzlL3PgwIFiPkxEZIWiDe55TLGcdj53gMsuu4z169cP/mEiIjmKNueex9DptPO5i4iELtrgntcUy5rPXUSqINq0TF5Dp9PM5y4iErpog3teQ6fTzOcuEoQyh4tJ8KJNy0D2Q6fTzucO8MEPfpCf/vSnvPzyy4yNjfHd735XN+yQ4mimTekj1XzuZrYT+BawGviOu3+z7e83AJ8H3gBeBva4+9Fe76n53EWGMD7eudOp0UhuGyeVlXY+975pGTNbDdwBfBTYBlxtZtvaiv3A3d/j7v8S+E/AXw1QZxFJSzNtSh9p0jKXAsfc/TiAmd0N7AbebJm7+/9tKf/PgHJu75QxzecuwcpruJhURprgvgk42fJ6HviD9kJm9nngPwJrgbOT1BHSfO4SrOnp5Tl30Eybskxmo2Xc/Q53/+fAV4BbOpUxsz1mNmdmcwsLC93eJ6sqla5K30UCo5k2pY80wf0UsLnl9VhzWTd3A/+u0x/cfZ+7T7r75Ojo6Fl/X7duHYuLi5UIiu7O4uIi69atK7sqUlVTU0nn6e9+l/yrwC4t0qRlDgNbzWyCJKhfBVzTWsDMtrr7z5ov/wj4GQMYGxtjfn6ebq362Kxbt46xsbGyqyEiNdQ3uLv762Z2I3CIZCjk99z9iJntBebc/SBwo5n9a+A14CXgU4NUZs2aNUxMTAzyX0VEpEWqi5jc/T7gvrZlt7Y8//OM6yUiIkOIdvoBERHpTsFdRKSCUk0/kMsHmy0AHa7CSGUj8MsMq5OnmOoKcdU3prqC6punmOoKw9W34e5nDzdsU1pwH4aZzaWZWyEEMdUV4qpvTHUF1TdPMdUViqmv0jIiIhWk4C4iUkGxBvd9ZVdgBWKqK8RV35jqCqpvnmKqKxRQ3yhz7iIi0lusLXcREekh6OBuZjvN7FkzO2ZmN3X4+w1m9pSZPW5m/6vDTUQK06+uLeU+aWZuZqX27KdYt9eb2UJz3T5uZn9WRj2bdem7bs3sT8zsqJkdMbMfFF3Htrr0W7d/3bJenzOzX5VRz2Zd+tV1i5k9ZGaPmdmTZnZlGfVsqU+/+jbM7B+adX3YzEqb3MnMvmdmvzCzp7v83czs9uZ3edLM3pdpBdw9yAfJPDbPA+8kmSP+CWBbW5m3tTzfBdwfal2b5dYDPwQeASYDX7fXA/8lkv1gK/AYcH7z9e+HXN+28l8gma8pyLqS5IY/13y+DXgh5HUL/HfgU83nHwEOlFjffwW8D3i6y9+vBP4nYMD7gZ9k+fkht9zfvAOUu79KMpXw7tYCHs4doPrWtek24C+BM0VWroO09Q1Bmrp+BrjD3V8CcPdfFFzHVitdt1cDdxVSs7OlqasDb2s+fzvw8wLr1y5NfbcBDzafP9Th74Vx9x8Cp3sU2Q3c6YlHgPPM7B1ZfX7Iwb3THaA2tRcys8+b2fMk9279YkF1a9e3rs1Trs3u/ndFVqyLVOsW+GTzdPFeM9vc4e9FSFPXdwHvMrMfmdkjzRu6lyXtusXMGsAEbwWjoqWp69eBa81snmTywC8UU7WO0tT3CeATzecfB9ab2YYC6jaI1PvKIEIO7ql4ijtAlc3MVpHcNPxLZddlBf4WGHf3S4C/B/aXXJ9eziFJzXyYpCX8bTM7r9QapXMVcK+7v1F2RXq4Gvi+u4+RpBEONPfnUP0F8CEzewz4EMk9KEJev7kJeSNldgeoAvSr63rgYuBhM3uBJL92sMRO1b7r1t0X3f3/NV9+B9heUN3apdkP5oGD7v6au/8j8BxJsC/DSvbbqygvJQPp6vpp4B4Ad/8xsI5kXpQypNlvf+7un3D39wI3N5eV1mHdx0pj3MqU1dmQojPiHOA4yWnrUufJRW1ltrY8/7ckNw8Jsq5t5R+m3A7VNOv2HS3PPw48EnBddwL7m883kpzqbgi1vs1y7wZeoHmtSah1Jenwu775/EKSnHspdU5Z343AqubzaWBvWeu3WYdxuneo/hHLO1T/d6afXeYXT7FiriRphT0P3NxcthfY1Xz+LeAI8DhJ50nXgFp2XdvKlhrcU67bbzTX7RPNdfvugOtqJGmvo8BTwFUhr9vm668D3yyzninX7TbgR8394HHg3wRe3z8muc3ncyRnnOeWWNe7gH8iuUPdPMlZ0A3ADc2/G3BH87s8lXVM0BWqIiIVFHLOXUREBqTgLiJSQQruIiIVpOAuIlJBCu4iIhWk4C4iUkEK7iIiFaTgLiJSQf8fr3/Tekz1AzcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 从 data.txt 中读入点\n",
    "with open('./data.txt', 'r') as f:\n",
    "    data_list = [i.split('\\n')[0].split(',') for i in f.readlines()]\n",
    "    data = [(float(i[0]), float(i[1]), float(i[2])) for i in data_list]\n",
    "\n",
    "# 标准化\n",
    "x0_max = max([i[0] for i in data])\n",
    "x1_max = max([i[1] for i in data])\n",
    "data = [(i[0]/x0_max, i[1]/x1_max, i[2]) for i in data]\n",
    "\n",
    "x0 = list(filter(lambda x: x[-1] == 0.0, data)) # 选择第一类的点\n",
    "x1 = list(filter(lambda x: x[-1] == 1.0, data)) # 选择第二类的点\n",
    "\n",
    "plot_x0 = [i[0] for i in x0]\n",
    "plot_y0 = [i[1] for i in x0]\n",
    "plot_x1 = [i[0] for i in x1]\n",
    "plot_y1 = [i[1] for i in x1]\n",
    "\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们将数据转换成 NumPy 的类型，接着转换到 Tensor 为之后的训练做准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "np_data = np.array(data, dtype='float32') # 转换成 numpy array\n",
    "x_data = torch.from_numpy(np_data[:, 0:2]) # 转换成 Tensor, 大小是 [100, 2]\n",
    "y_data = torch.from_numpy(np_data[:, -1]).unsqueeze(1) # 转换成 Tensor，大小是 [100, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们来实现以下 Sigmoid 的函数，Sigmoid 函数的公式为\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义 sigmoid 函数\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画出 Sigmoid 函数，可以看到值越大，经过 Sigmoid 函数之后越靠近 1，值越小，越靠近 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb05e21e0b8>]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAHB9JREFUeJzt3XmYVNWd//H3V1YXIiDIjuAIRiZjXFqNOv7UURSIgsYNonFDSYg4cVxGHR00ap4kkp+JTjSKW9xZ4q9bRHhwHxNXlggqiDauoCwqImKgafj+/jjVWjbVdHV3VZ2qW5/X89ynqu493fXt28WnL+fee465OyIikizbxC5ARERyT+EuIpJACncRkQRSuIuIJJDCXUQkgRTuIiIJpHAXEUkghbuISAIp3EVEEqh1rDfu0qWL9+vXL9bbi4iUpLlz537i7l0baxct3Pv168ecOXNivb2ISEkys/ezaaduGRGRBFK4i4gkkMJdRCSBFO4iIgmkcBcRSaBGw93M7jKzlWb2egPbzcxuMrNqM1tgZvvkvkwREWmKbI7c/wwM2cr2ocCA1DIG+FPLyxIRkZZo9Dp3d3/OzPptpckI4F4P8/W9ZGYdzayHu3+coxpFJKncYcOGzMv69VBTA7W1sGlT5mVr2zZtgs2bw3vULXXv2dR1Tf26+j9jfcceC/vtl9t9WU8ubmLqBXyY9nppat0W4W5mYwhH9/Tt2zcHby0i0dTWwqefwiefwKpV4bHu+erVsHbtlsuXX4bHr74K4b1xY+yfojDMvv26Z8+SCPesuftEYCJARUWFZuYWKWbusHQpLFoEb70FH3wQlg8/DI8ffRSOjDPZYQfo0OGbZYcdoHfvb15vtx20axeW9u2/eZ6+tG8PbdtC69bQqtWWS0Pr05dttgnBmr5A89Y15euKQC7CfRnQJ+1179Q6ESkVGzbAggUweza88gq8/jq8+SasW/dNm3btoE8f6NsXjjgiPO/eHbp2hS5dvnncaacQyhJVLsJ9GjDOzCYBBwBr1N8uUuRqauDFF+GJJ+Cpp2DevLAOYOed4fvfh9GjYY89wrL77tCtW1EdmcrWNRruZvYQcBjQxcyWAlcBbQDc/VZgBjAMqAa+As7KV7Ei0gJr18Kjj8LUqSHU160LXRf77Qe/+AXsv39Y+vRRiCdANlfLjGpkuwPn5awiEcmdzZvh8cdh4kSYMSN0v/TqBWeeCYMHw2GHwY47xq5S8iDakL8ikkdffAG33hqWd98N/eE//SmcfDIceGA40SiJpnAXSZLPPoObboIbb4TPP4dDD4Vf/xqOOy6cEJWyoXAXSYKaGrj5ZvjlL2HNmhDmV14J++4buzKJROEuUuqefBLGjYPFi+Hoo+H662HPPWNXJZGp402kVK1dG/rRBw8OJ06nT4eZMxXsAujIXaQ0vfwyjBwJ778Pl1wC11wT7ugUSdGRu0gpcQ9XwBxySHj9t7+FbhgFu9SjcBcpFRs3wrnnwtix4fb/uXPhoINiVyVFSuEuUgrWrYMRI+DOO+GKK0L/eufOsauSIqY+d5Fi99lnMHQozJkDt90GY8bErkhKgMJdpJitWQNHHRVGaXz44XD9ukgWFO4ixWrt2nDEvmABVFbCD38YuyIpIQp3kWK0cSMcf3wYW33qVAW7NJnCXaTYuIcrYp56Cu65J4S8SBPpahmRYvO734WrYq68Ek4/PXY1UqIU7iLFZNYsuPRSOOWUMAiYSDMp3EWKxbJlcNpp8L3vwV13acx1aRF9ekSKQW0tjBoF//hHOIG63XaxK5ISpxOqIsXg2mvhr3+F++8Pk1GLtJCO3EVimzcPfvUrOOMMOPXU2NVIQijcRWKqqQmTVXfrBr//fexqJEHULSMS03XXwWuvhYHAOnWKXY0kiI7cRWJZuDBMXv2Tn+gOVMk5hbtIDO5w/vnQoQPccEPsaiSB1C0jEsPUqfD003DLLdClS+xqJIF05C5SaF9+CRddBHvtpbHZJW905C5SaNdfD0uXwqRJ0KpV7GokoXTkLlJIK1aEPvaTT4aDD45djSSYwl2kkK67DtavD3ekiuSRwl2kUN55J8yBOno0DBwYuxpJuKzC3cyGmNliM6s2s8sybO9rZs+Y2d/NbIGZDct9qSIlbvz40Md+1VWxK5Ey0Gi4m1kr4GZgKDAIGGVmg+o1uxKY4u57AyOBW3JdqEhJW7wYHnwwXNves2fsaqQMZHPkvj9Q7e7vuHsNMAkYUa+NA99JPd8R+Ch3JYokwG9+A+3ahUsgRQogm0shewEfpr1eChxQr83VwONmdj6wPXBkTqoTSYL33w9D+Y4dGwYIEymAXJ1QHQX82d17A8OA+8xsi+9tZmPMbI6ZzVm1alWO3lqkyE2YAGZwySWxK5Eykk24LwP6pL3unVqXbjQwBcDdXwTaA1vcU+3uE929wt0runbt2ryKRUrJ8uVwxx1hous+fRpvL5Ij2YT7bGCAmfU3s7aEE6bT6rX5ADgCwMz2IIS7Ds1FbrwRNm6Ey7a4yEwkrxoNd3evBcYBs4BFhKti3jCza8xseKrZRcC5ZjYfeAg40909X0WLlISvvgrXtR9/POy2W+xqpMxkNbaMu88AZtRbNz7t+UJA91KLpLvvPli9Gi64IHYlUoZ0h6pIPriHLpl99tEYMhKFRoUUyYcnnoBFi+Dee8OVMiIFpiN3kXz4wx+ge/cw+qNIBAp3kVyrroaZM8NNS+3axa5GypTCXSTXbr89DBB27rmxK5EypnAXyaWaGrj7bjj2WOjRI3Y1UsYU7iK59MgjsGqV5kaV6BTuIrk0cSL07QtHHRW7EilzCneRXFmyBJ58Es45RxNfS3QKd5FcueMO2GYbOPvs2JWIKNxFcqK2NpxIPeYY6NUrdjUiCneRnHj8cVixAs46K3YlIoDCXSQ37r0XdtoJhmlueCkOCneRllqzBqqqYORIaNs2djUigMJdpOX+8hfYsAF+8pPYlYh8TeEu0lL33gsDB8L++8euRORrCneRlnjvPXjuuTBHqob2lSKicBdpifvvD4+nnhq3DpF6FO4izeUeumQOPRT69Ytdjci3KNxFmmvePHj7bR21S1FSuIs01+TJ0Lo1nHBC7EpEtqBwF2kOd5gyBQYPhs6dY1cjsgWFu0hzvPwyvP8+nHJK7EpEMlK4izTHlCnhbtQRI2JXIpKRwl2kqTZvDuF+9NHQsWPsakQyUriLNNULL8CyZeqSkaKmcBdpqsmToX17GD48diUiDVK4izTFpk1hoLBhw6BDh9jViDRI4S7SFH/9Kyxfri4ZKXoKd5GmmDoVtt0WfvjD2JWIbJXCXSRbmzfDI4/AkCGw/faxqxHZqqzC3cyGmNliM6s2s8saaHOymS00szfM7MHclilSBObODVfJHH987EpEGtW6sQZm1gq4GRgMLAVmm9k0d1+Y1mYAcDlwsLuvNrOd81WwSDRVVdCqlbpkpCRkc+S+P1Dt7u+4ew0wCah/W965wM3uvhrA3VfmtkyRIlBVFYb31VgyUgKyCfdewIdpr5em1qUbCAw0s+fN7CUzG5LpG5nZGDObY2ZzVq1a1byKRWJ46y1YuBCOOy52JSJZydUJ1dbAAOAwYBRwu5ltcV+2u0909wp3r+jatWuO3lqkAKqqwqPGkpESkU24LwP6pL3unVqXbikwzd03uvu7wFuEsBdJhqoq2Gcf6Ns3diUiWckm3GcDA8ysv5m1BUYC0+q1qSIctWNmXQjdNO/ksE6ReD7+GF56SVfJSElpNNzdvRYYB8wCFgFT3P0NM7vGzOoG15gFfGpmC4FngEvc/dN8FS1SUI8+GibnUH+7lBBz9yhvXFFR4XPmzIny3iJNMmxYOKH69ttgFrsaKXNmNtfdKxprpztURbbmiy/gqafCUbuCXUqIwl1ka2bOhJoadclIyVG4i2xNVRXsvDMceGDsSkSaROEu0pANG+Cxx8KkHK1axa5GpEkU7iINefZZWLtWXTJSkhTuIg2pqgpD+x5xROxKRJpM4S6SSd3Y7UOHhvlSRUqMwl0kk1deCXemqktGSpTCXSSTqipo3Vpjt0vJUriLZFJVBYcfDh23GNxUpCQo3EXqe/NNWLxYXTJS0hTuIvVVVobH4cO33k6kiCncReqrqoL99oPevWNXItJsCneRdMuWhStlNHa7lDiFu0i6aal5aNTfLiVO4S6SrqoKBg6E7343diUiLaJwF6nz+efw9NMau10SQeEuUmfGDKitVZeMJILCXaROVRV07w4HHBC7EpEWU7iLAKxfH2ZdGjECttE/Cyl9+hSLQOhr//JLdclIYijcRSDcldqhQxhPRiQBFO4imzaF69uHDYN27WJXI5ITCneRF1+ElSt1V6okisJdpLIS2rYNsy6JJITCXcqbewj3I4+E73wndjUiOaNwl/L22mvw7ru6SkYSR+Eu5a2yMgw1oLHbJWEU7lLeKivh4IOhW7fYlYjklMJdyte778L8+bpKRhIpq3A3syFmttjMqs3ssq20O8HM3MwqcleiSJ5UVYVH9bdLAjUa7mbWCrgZGAoMAkaZ2aAM7ToAvwBeznWRInlRWQl77gm77hq7EpGcy+bIfX+g2t3fcfcaYBIwIkO7a4HfAutzWJ9IfqxcCX/7m7pkJLGyCfdewIdpr5em1n3NzPYB+rj7YzmsTSR/Hn00XOOuLhlJqBafUDWzbYAbgIuyaDvGzOaY2ZxVq1a19K1Fmq+yEvr1g+9/P3YlInmRTbgvA/qkve6dWlenA/A94Fkzew/4ATAt00lVd5/o7hXuXtG1a9fmVy3SEmvXwpNPhi4ZTacnCZVNuM8GBphZfzNrC4wEptVtdPc17t7F3fu5ez/gJWC4u8/JS8UiLTV9OmzYAD/6UexKRPKm0XB391pgHDALWARMcfc3zOwaM9NtfVJ6pkyBnj3hoINiVyKSN62zaeTuM4AZ9daNb6DtYS0vSyRP1q4N0+n99KeaTk8STZ9uKS91XTInnRS7EpG8UrhLeVGXjJQJhbuUj7oumRNPVJeMJJ4+4VI+Hn1UXTJSNhTuUj6mTlWXjJQNhbuUB3XJSJnRp1zKg7pkpMwo3KU8TJ6sLhkpKwp3Sb5PPw1dMqNGqUtGyoY+6ZJ8U6fCxo1w6qmxKxEpGIW7JN8DD8CgQbDXXrErESkYhbsk23vvhRmXTjtNw/tKWVG4S7I9+GB4/PGP49YhUmAKd0kud7j/fjjkENhll9jViBSUwl2S69VXYdEinUiVsqRwl+S6/35o00Y3LklZUrhLMm3cGML9mGOgc+fY1YgUnMJdkmn6dFi5EkaPjl2JSBQKd0mmO+8Mww0cfXTsSkSiULhL8ixbFoYbOPNMaJ3VNMEiiaNwl+S55x7YvBnOOit2JSLRKNwlWTZvhrvugkMPhd12i12NSDQKd0mW556DJUt0IlXKnsJdkuXWW6FjRzjhhNiViESlcJfk+OgjePhhOPts2G672NWIRKVwl+SYOBE2bYKxY2NXIhKdwl2SoaYGbrsNhg7ViVQRFO6SFJWVsHw5jBsXuxKRoqBwl2T44x/hn/5Jd6SKpCjcpfTNnh1mWzrvPE2ALZKifwlS+iZMgB13hHPOiV2JSNHIKtzNbIiZLTazajO7LMP2C81soZktMLOnzEzT3khhVFeHyx/HjoUOHWJXI1I0Gg13M2sF3AwMBQYBo8xsUL1mfwcq3H1P4C/A9bkuVCSjG24Ig4P9+7/HrkSkqGRz5L4/UO3u77h7DTAJGJHewN2fcfevUi9fAnrntkyRDFatgrvvhtNPhx49YlcjUlSyCfdewIdpr5em1jVkNDAz0wYzG2Nmc8xszqpVq7KvUiSTP/wBNmyAiy6KXYlI0cnpCVUzOw2oACZk2u7uE929wt0runbtmsu3lnLzySdw001w8snw3e/Grkak6GQzk8EyoE/a696pdd9iZkcCVwCHuvuG3JQn0oDf/Q7WrYOrropdiUhRyubIfTYwwMz6m1lbYCQwLb2Bme0N3AYMd/eVuS9TJM3KlfA//wM//jHssUfsakSKUqPh7u61wDhgFrAImOLub5jZNWY2PNVsArADMNXMXjWzaQ18O5GWmzAB1q+H8eNjVyJStLKaYNLdZwAz6q0bn/b8yBzXJZLZBx+EoQZOOw0GDoxdjUjR0h2qUlr+67/C43XXxa1DpMgp3KV0vPIKPPBAuPSxT5/G24uUMYW7lAZ3uPBC6NYNLr00djUiRS+rPneR6CZNguefh9tv1xgyIlnQkbsUv9Wr4T/+Ayoq4KyzYlcjUhJ05C7F7/LLwzgyM2dCq1axqxEpCTpyl+L2wgthbtQLLoC9945djUjJULhL8Vq3Ds48E/r2hV/+MnY1IiVF3TJSvC6+OEzG8dRTsMMOsasRKSk6cpfi9NhjcOut4fLHww+PXY1IyVG4S/FZujRcFfMv/wK/+lXsakRKksJdisuGDXDSSfCPf8DkydCuXeyKREqS+tyluFx4Ibz0EkydquF8RVpAR+5SPG65JSwXXwwnnhi7GpGSpnCX4vDII3D++XDssfDrX8euRqTkKdwlvuefh1GjYN994aGHoLV6C0VaSuEucb3wAgwdCr17w/TpsP32sSsSSQSFu8Tz4oswZAh07w7PPAM77xy7IpHEULhLHNOnw5FHfhPsvXrFrkgkURTuUnh/+hOMGBEudXzuOQW7SB4o3KVw1q+HsWPh5z+HYcPgf/83HLmLSM4p3KUwliyBgw4K48VccglUVurkqUge6Zozya/Nm0OgX3optGkDjz4KxxwTuyqRxNORu+TPwoVw2GFw3nlw4IHw978r2EUKROEuubdiBfzsZ2FUx9deg7vvhlmzYJddYlcmUjbULSO589FH8Pvfh6thNmyAcePgv/8bunSJXZlI2VG4S8u4w9y5oV/9vvugthZOOSVMizdgQOzqRMqWwl2aZ8WKMCzvHXfA/Pmw7bZwzjlw0UWw666xqxMpewp3yY47vPVWuLO0sjKMCeMeBvu65ZYw8FfHjrGrFJEUhbtkVlsLixeH8V+eeQaefTb0qQPstRdcfTUcf3w4aSoiRSercDezIcCNQCvgDnf/Tb3t7YB7gX2BT4FT3P293JYqebF5cwjtt98OR+bz58O8ebBgQZjqDqBbtzBJ9eGHw+DB0L9/3JpFpFGNhruZtQJuBgYDS4HZZjbN3RemNRsNrHb33cxsJPBb4JR8FCxNsHEjfP45LF8eArz+smQJVFd/E+IA3/kO7L13uJRx771hv/1g993BLN7PISJNls2R+/5Atbu/A2Bmk4ARQHq4jwCuTj3/C/BHMzN39xzWWrrcQzdHbW0I3Lrn9V/X31ZTA199FcI3/bH+ui+/DCG+evW3l3XrMtfTuTP06BFOfA4eDLvtFq5sGTAA+vSBbXT7g0ipyybcewEfpr1eChzQUBt3rzWzNcBOwCe5KPJb7roLJkwIgRne8NtLtusK9fW1taHrIx+23TYs228fTmZ26hQCu1Onb5aOHcPgXD17hqVHD2jfPj/1iEjRKOgJVTMbA4wB6Nu3b/O+SZcu4SReXTeB2ZZLpvXZrst129atw9KmTebnW3vdpg1st10I8PTH7baDdu10hC0iDcom3JcBfdJe906ty9RmqZm1BnYknFj9FnefCEwEqKioaF6XzfDhYRERkQZlc+g3GxhgZv3NrC0wEphWr8004IzU8xOBp9XfLiIST6NH7qk+9HHALMKlkHe5+xtmdg0wx92nAXcC95lZNfAZ4Q+AiIhEklWfu7vPAGbUWzc+7fl64KTcliYiIs2lM3IiIgmkcBcRSSCFu4hIAincRUQSSOEuIpJAFutydDNbBbzfzC/vQj6GNmg51dU0qqvpirU21dU0LalrF3fv2lijaOHeEmY2x90rYtdRn+pqGtXVdMVam+pqmkLUpW4ZEZEEUriLiCRQqYb7xNgFNEB1NY3qarpirU11NU3e6yrJPncREdm6Uj1yFxGRrSjacDezk8zsDTPbbGYV9bZdbmbVZrbYzI5u4Ov7m9nLqXaTU8MV57rGyWb2amp5z8xebaDde2b2WqrdnFzXkeH9rjazZWm1DWug3ZDUPqw2s8sKUNcEM3vTzBaYWaWZdWygXUH2V2M/v5m1S/2Oq1OfpX75qiXtPfuY2TNmtjD1+f9FhjaHmdmatN/v+EzfKw+1bfX3YsFNqf21wMz2KUBNu6fth1fN7Aszu6Bem4LtLzO7y8xWmtnraes6m9kTZvZ26rFTA197RqrN22Z2RqY2TeLuRbkAewC7A88CFWnrBwHzgXZAf2AJ0CrD108BRqae3wqMzXO9/xcY38C294AuBdx3VwMXN9KmVWrf7Qq0Te3TQXmu6yigder5b4Hfxtpf2fz8wM+BW1PPRwKTC/C76wHsk3reAXgrQ12HAdML9XnK9vcCDANmAgb8AHi5wPW1ApYTrgOPsr+A/wPsA7yetu564LLU88syfe6BzsA7qcdOqeedWlJL0R65u/sid1+cYdMIYJK7b3D3d4FqwiTeXzMzA/6NMFk3wD3AcfmqNfV+JwMP5es98uDric/dvQaom/g8b9z9cXevTb18iTCrVyzZ/PwjCJ8dCJ+lI1K/67xx94/dfV7q+VpgEWGO4lIwArjXg5eAjmbWo4DvfwSwxN2be3Nki7n7c4Q5LdKlf44ayqKjgSfc/TN3Xw08AQxpSS1FG+5bkWnC7vof/p2Az9OCJFObXDoEWOHubzew3YHHzWxuah7ZQhiX+q/xXQ38NzCb/ZhPZxOO8jIpxP7K5uf/1sTvQN3E7wWR6gbaG3g5w+YDzWy+mc00s38uUEmN/V5if6ZG0vABVoz9Vaebu3+cer4c6JahTc73XUEnyK7PzJ4EumfYdIW7P1LoejLJssZRbP2o/V/dfZmZ7Qw8YWZvpv7C56Uu4E/AtYR/jNcSuozObsn75aKuuv1lZlcAtcADDXybnO+vUmNmOwAPAxe4+xf1Ns8jdD18mTqfUgUMKEBZRft7SZ1TGw5cnmFzrP21BXd3MyvIJYpRw93dj2zGl2UzYfenhP8Stk4dcWVqk5MaLUwI/iNg3618j2Wpx5VmVknoEmjRP4ps952Z3Q5Mz7Apm/2Y87rM7EzgGOAIT3U2ZvgeOd9fGeRs4vdcM7M2hGB/wN3/X/3t6WHv7jPM7BYz6+LueR1DJYvfS14+U1kaCsxz9xX1N8TaX2lWmFkPd/841U21MkObZYRzA3V6E843NlspdstMA0amrmToT/gL/Ep6g1RoPEOYrBvC5N35+p/AkcCb7r4000Yz297MOtQ9J5xUfD1T21yp1895fAPvl83E57muawjwn8Bwd/+qgTaF2l9FOfF7qk//TmCRu9/QQJvudX3/ZrY/4d9xXv/oZPl7mQacnrpq5gfAmrTuiHxr8H/PMfZXPemfo4ayaBZwlJl1SnWjHpVa13yFOIPcnIUQSkuBDcAKYFbatisIVzosBoamrZ8B9Ew935UQ+tXAVKBdnur8M/Czeut6AjPS6pifWt4gdE/ke9/dB7wGLEh9sHrUryv1ehjhaowlBaqrmtCv+GpqubV+XYXcX5l+fuAawh8fgPapz0516rO0awH20b8SutMWpO2nYcDP6j5nwLjUvplPODF9UAHqyvh7qVeXATen9udrpF3llufatieE9Y5p66LsL8IfmI+Bjan8Gk04T/MU8DbwJNA51bYCuCPta89OfdaqgbNaWovuUBURSaBS7JYREZFGKNxFRBJI4S4ikkAKdxGRBFK4i4gkkMJdRCSBFO4iIgmkcBcRSaD/DydAb7nqWwBcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出 sigmoid 的图像\n",
    "\n",
    "plot_x = np.arange(-10, 10.01, 0.01)\n",
    "plot_y = sigmoid(plot_x)\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_data = Variable(x_data)\n",
    "y_data = Variable(y_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 PyTorch 当中，不需要我们自己写 Sigmoid 的函数，PyTorch 已经用底层的 C++ 语言为我们写好了一些常用的函数，不仅方便我们使用，同时速度上比我们自己实现的更快，稳定性更好\n",
    "\n",
    "通过导入 `torch.nn.functional` 来使用，下面就是使用方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义 logistic 回归模型\n",
    "w = Variable(torch.randn(2, 1), requires_grad=True) \n",
    "b = Variable(torch.zeros(1), requires_grad=True)\n",
    "\n",
    "def logistic_regression(x):\n",
    "    return torch.sigmoid(torch.mm(x, w) + b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在更新之前，我们可以画出分类的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fb05e19dbe0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3X90VPWd8PH3JyEQIr8TyFACCVZQAiQzEGh3bavVVamlULfYR8QWj1ZWbPvY9Xnc6sH2dNty1u169mnd2m6t2+Mv+mv32fMsj4+tp6169miX1ljCb38gIgYJhASFmEB+fZ8/7mSYJDOZO8n9PZ/XOXOYmVzmfufOvZ/7vd/v936+YoxBKaVUtBT5XQCllFLO0+CulFIRpMFdKaUiSIO7UkpFkAZ3pZSKIA3uSikVQRrclVIqgjS4K6VUBGlwV0qpCBrn14orKipMTU2NX6tXSqlQevnll08aY2bmWs634F5TU0NjY6Nfq1dKqVASkbfsLKfNMkopFUEa3JVSKoI0uCulVARpcFdKqQjS4K6UUhGUM7iLyE9E5ISI7M3ydxGRB0XkoIjsFpFlzhczwrZtg5oaKCqy/t22ze8SKaUiwE7N/VFg1Qh//wSwIPnYBPxw7MUqENu2waZN8NZbYIz176ZNGuCVUmOWM7gbY/4TaB9hkbXA48ayA5gmIrOdKmCkbdkCnZ2D3+vstN5XSqkxcKLNfQ7wdtrr5uR7w4jIJhFpFJHG1tZWB1YdckeO5Pf+KGnLT3jpb6dGy9MOVWPMw8aYBmNMw8yZOe+ejb558/J7fxS05Se89LdTY+FEcD8KzE17XZV8T+WydSuUlQ1+r6zMet8h2vIzNkNrznfc4V1NOmi/nV5FhIwxJucDqAH2ZvnbJ4FfAQJ8GPijnc9cvny5UcaYJ580prraGBHr3yefdPTjRYyx6n2DHyKOriaSnnzSmLKyzNtv4FFW5vhPlpLvb+fmrpRpW7j53VV2QKOxE7dzLgA/A44BPVjt6bcCtwO3J/8uwEPAG8AeoMHOigsyuLscyDOprs4cIKqrXV916GXbdl5ty3x+O7eDr+5HweFYcHfrUXDB3aeqj9a4Ri9bzdmrq6B8fju3g69eAQaH3eCud6h6xacG1A0b4OGHoboaRKx/H37Yel+NzG6/toP934Pk89u5PfDKg75/5TAN7l7xaNhjJhs2wOHD0N9v/auB3Z5M/d1DOdz/PYzd387t4OtB379ymAZ3r2jVJ3Qy1Zw3bw7mVZDbwVevAMNHrCYc7zU0NJiCmolpYNByetNMWZkeIcox27ZZrXxHjlh1hq1bddeKIhF52RjTkGs5rbl7Ras+yiHZxptr85tK59scqgVpwwY94tSYDL0AHLhrFXTXUoNpzV2pEAnaXaugd64GldbclQoRHwddZaRXEsGlNXelQiRog66CeCWhLBrclQoRJ4c8OtGc8tZb+b2vvKPBXakQcWrQlVPphIuL83tfeUfHuStVQAbGwmerWVdXW8Mo7RLJ/jefQkvk6Th3FVk6OmN00mvr2eTbMVtdnd/7yjsa3FWo6OxEo5ep83OofDtmw5ZzppAqBhrclWO8OHB0dMbo5aqVjyYoh+nG60KrGGibu3KEV6lziooyt+WKWLfdq+xqakZua496Lpps3z/ffga/aZu7ssWp2rZXNeqgjfMOk2xNKJs3W88/97noNVWk79/ZTmx+3QDmOjszerjxKLiZmALIyVmavJqpR2eWGpuhMz1u3hzd7WlnDlynpwr0YiZNdJo9lYuTU7N5Ocem2weQD1Pd+ibKc6PamQPXyROZVxUPDe4qJydr21GpUUfle9jx5JPZg14U5kYdaQ5cN07cXp0o7QZ3bXMvYE62X4dp1MRI/B6N49VQvYEO8Gyi0IeR7TtUV7uT8z5oSd205l7ACqmWapdXfQeZuP17pDc3FRd701ThJ6/376DV3DW4F7hCal+2w882aDfXbbdzEaK1D3i5fwetzV3HuSuVxs+pbt0cwz/SGPd0YRvzHTRezGOr49yVGgU/+w7cHMNvp903yGkDwiJI89hqcFdqCL8OUDfztGQ7QRQXh7sDXGWnwV2pgHDzqiHbieOxx4JRy1TO0zlUlQqQDRvcCbIDn+l2e7AKDg3uShUIt04cKpi0WUYppSJIg7tSDiqkySBUsGmzjFIOGTpGfmAyCNDmEOU9rbkr5RC/89Iolc5WcBeRVSLyqogcFJF7Mvx9nog8JyI7RWS3iFzrfFGVCrbAJY5SBS1ncBeRYuAh4BNALbBeRGqHLHYf8EtjTAK4AfiB0wVVLtFGYsfoLFEqSOzU3FcCB40xh4wx3cDPgbVDljHAlOTzqcA7zhVRuabQZgx2mZt3mI5Ez88qEzvBfQ7wdtrr5uR76b4B3CQizcDTwJcdKV2h8foo1UZix02ceP55ebn7t/Tr+Vll41SH6nrgUWNMFXAt8ISIDPtsEdkkIo0i0tja2urQqiPCj6NUG4kdM/DztbWdf6+ry/316vlZZWMnuB8F5qa9rkq+l+5W4JcAxpj/AkqBiqEfZIx52BjTYIxpmDlz5uhKHFV+HKXaSOwYv4Ksnp/dEYWmLjvB/SVggYjMF5HxWB2m24cscwS4EkBEFmEFd62a58OPo9SvRuII8ivIhuH8HLZAGZmmLjszemA1tbwGvAFsSb73TWBN8nkt8CKwC2gCrs71mToT0xBOT8NjdwoanYrJEX7N4BT0qRKDXr5M/JyNyw50mr2QcfIoCNER5ee5xcl1+7nJg3x+DnqgzMTPeXTt0OAeJgNHJ5yfuXgsR2lIjii/A6LT6w5ykHVKvt8x6IEyk6AfPhrcw8KNKBOSIyqqk1FH1Wh21TBu56Bf+GpwDws39v6QHFF+noNCcv4LlGy7VXFx9pp80ANlNkG+CrMb3DVxmN/cGGYRklEwfo70CMMok6DJtkv29WUfVeLnhONjEaSJrkdLg7vf3IgyITmi/DwHheT8Fyh2dslMY/ujECjDSIO733JFmdEOEg7BEeXnOSjXutM3e0WF9QjLOG23ZNpVM9EbqALCTtuNGw9tc0+TrYEvrA2WIZdps+tPYEnfVQcGdgW8aydQnGjLx2abu1jLeq+hocE0Njb6su7QqKmxGjKHqq62auPKFdk2ezr9CYbPPAVWzT6ALYCB4NT2EpGXjTENuZbTZpkg08QhvrCzefUnCE3XTmB4nX9Ig3uQOdXZGrbkHj6zs3l1VI0lBF07geF1XU2De5A5MaQjMlmQvJOr4zDqo2q0LuAOz4ff2mmYd+OhHao2jbUHJiQ3NAVN+mYvL7ceAz/B5s3BvcFlrLQP3z1ObVv0DlVljAntrZhBvUMwrMHP7vYMa10gqPvLUF6OltHgHnUhPFqDHEBDuDnz2p7Z6gID3zEIv8FQQd5f3KDBXVlCuOcHOYCG8UIon+2Zbdkg7zpB2V9GqpU7eWWhwV2dF5Zr1qQgB9CgBJJ85LM9c93AFcTvGoT9ZaQ6lNP1Kw3uKrSCHEBDeCGU9/ZMn17A76BpRxD2l5HK4HT57AZ3HQqpAifISb3CeONOvttzYOx6dXXmvwdtjH8Q9peRxrD7di+inTOAGw+tuauRhKwladS8+p6jWU+YrlL83l+CWHPX4K5y8/vIiagwBE/96e3RNncN7uEThggUUkFoK44iv05IQRsto1kh1cg0M6VrioqscD6UiJWrReXPyUyV27ZZSb2OHLH6GbZuDUbfimaFLERuJAUJSWbKMOZD0an+nOdU5sVIpGSyU71346HNMg5zq/kkBG0HYW05Cmu5g8ypMe9B3u3RoZAFxq1k0UEYZ5aD13mynRLGYZVB59TVUEguWEekwT0q3NobQxCBwnwgaj50ZzlVF4lCk5kG96hwc2/0IwLl0YgehQNROcOpukgILlhzs9N248ZD29wdFqUG3Dy/S5S+ugqOoI7xR4dCFqCgjt3K1yiGX0blqyuVi92hkBrcVfDoAHClstJx7iq8CqARPYzj8tXo+fF7a3BXwZOpN0sErr3Wn/I4LBI3yCjb/Pq9NbgHiVbnLBs2wMaNVkAfYAw89lgktklYx+Wr0fHr97YV3EVklYi8KiIHReSeLMt8VkT2i8g+Efmps8UsAIVSnbN7Anv66eHt7hGJgGEel6/y59fvnTO4i0gx8BDwCaAWWC8itUOWWQDcC1xqjFkMfMWFskZbIVTn8jmBOXhEBO2CqAC6FFQav35vOzX3lcBBY8whY0w38HNg7ZBlbgMeMsacAjDGnHC2mAWgEKpz+ZzAHDoignhBFIkbZJRtfv3edoL7HODttNfNyffSLQQWisiLIrJDRFY5VcCCUQjVuXxOYA4dEUG8IApBRgflIL9+b6c6VMcBC4DLgfXAj0Vk2tCFRGSTiDSKSGNra6tDq46IQqjO5XMCc+iICOoFkeaUKSx+/N52gvtRYG7a66rke+mage3GmB5jzJvAa1jBfhBjzMPGmAZjTMPMmTNHW+ZoKoTq3Ghnah7DEVEIF0RKZWInuL8ELBCR+SIyHrgB2D5kmf+DVWtHRCqwmmkOOVjOwhD16pwPJ7BCuCCKmqB1gIdVzuBujOkFvgQ8AxwAfmmM2Sci3xSRNcnFngHaRGQ/8BxwtzGmza1CK584cdR5fAIrhAuiKAliB3hYaW4ZZY+Tk1MqReZkb1u26JS9uWjiMOUsnShbOShbXWHoyKYBmjPuPE0cppwV1GEnKpSyDVEtLs68vHaA50+Du7JHh50oB2WrE/T1aQe4UzS4K3t02IlyULY6wUCHt3aAj50Gd2WPDjtRDhqprhD1EcFeGed3AVSIbNigR5pyxMBupFMjukdr7koFUCHcyKM1dHdpcFfhFdEIqDfyKCdocFfecDoQ+xUBPTihBDGTpQofvYlJuc+Nu1v9uKnKo7t0i4qGT0IFeiOPsugdqio43AjEfkRAj04oejOwGoneoaqCw427W/24qSpTxAXH79LVWwqUEzS4K/dlC7gzZoy+/drrCLhtm3VVkInDJxS9pUA5QYO7UyI6csMRmQLx+PFw+vToO0S9joBbtmRvBnLhhKLDBNVYaZu7EzQdbm5D87t2dEBbhpT/5eVw8qT35cslWxs/ZH9fKRdom7uXdOxabkOrou3tmZdrawvmVc9IyVCUCiAN7k7QdLj5G6mdOognRe3ldIy2YHpDg7sTNB1u/kYKikE8KRZQL6ebwVfvvvWQMcaXx/Lly01kPPmkMWVlxlj7q/UoK7PeH8tnVlcbI2L9O5bPCqry8sHbbOBRXe13yQqWG7tyuupq/cnHCmg0NmKs1tyd4HStrlCqN9/73vCmjpISq7O10K7ZPWyrGGlVbncfaQumh+ycAdx4RKrm7rRCqt6kX6GUlxszfrx71cagcru6nMeqRDLveiLOrL+Qdm23oDX3ECuk6s3AKJonnoB334Xu7sF/L4RRRx6Otsq1Kre7j8LYLx3WDmAN7kFUaB20A81QfX2Z/x7Fk1o6D0/muVbldvD1ql/aqYAc6hZSO9V7Nx7aLDMCDy/TAyHbtXqhXLN72FZhZ1X59uUHre/fycMniM1I2GyW0eAeFEOPkM2bg3XEuClbQ2/UT2oDAtTm7vfnOcHJgOx2H8RoaHD3w2irMEE8QryU7WgsLi6cbeBh9dfJVQWxZutkQA7i99Pg7jW7ATrTkRXEPchLhX5yCzEnAqnT5zUn6wpB3DU1uHvNbmNmpj0lW5OEn9d+Xgtaw61XQv69x1ovcev+v2yH1Wg+O2g/kQZ3r9mpwoxUpSjkmnsQeXFEB7FamCdbX2GEbenWReuTT0b3sNLg7jU7e2mujsMQH+SR4lXQjUhz3IjnwRzb0s0OyyB2hjpBg7vX7ASEkQ7moF37FTKvgq4f0cfr/SzHtnRzU0fk3DmMBnc/5DpwInAZXhDcDrojdaK7GX382P9ybEs3ixTVw02De1BpDT343KzyjdTb53b08aMqa2Odbh4SUTzc7AZ3nWZPqaHcnDaxpsa6hz2T6mrrPn+3csRnmypQxJohyw133AE//OHw9zdvhh/8wJ11Rpyj0+yJyCoReVVEDorIPSMs9xkRMSKSc8VKBZabCVCyJXcRcX8mbD9yFj39dH7vK8fkDO4iUgw8BHwCqAXWi0hthuUmA3cCf3C6kErlbayZo4bO+epU0PUzKZwfKRkLKcNpwNipua8EDhpjDhljuoGfA2szLPct4O+Bsw6WT6n8BTmVn585b/2YKjAsGU7Dmtd3JLka5YF1wCNprz8HfH/IMsuA/518/jzQkOWzNgGNQOO8efPc73lQhSnoY+Ci2MuXTRiGrORbRp9/P5waLZMruGPV/p8HakyO4J7+KNjRMsp9Ub17JayCfjLLpzIQgJOV3eCec7SMiPwZ8A1jzDXJ1/cma/x/l3w9FXgD6Ej+lxjQDqwxxmQdDqOjZZRrso1Iqa622s+VSpfPKKIA7FtOjpZ5CVggIvNFZDxwA7B94I/GmPeMMRXGmBpjTA2wgxyBXSlXhXEuN+WffPoFQtRBnDO4G2N6gS8BzwAHgF8aY/aJyDdFZI3bBVQqb350HKrwyqcyEJYOYtCbmJRSim3brFnCjxyxAnW2m8ncvMHNJkdvYlIqFKI4nE15w+59DSG6KhzndwGUcsTQGtXA2HYI5IGnQmzDhlDsU1pzV9GwZcvgS2WwXm/Z4k95lPKZBncVDSEaxaCUFzS4q2gI0SgGpbygwV1Fg45tV2oQDe4qGkI0ikEpL+hoGRUdIRnFoApDd183+07so6mliXlT53HlhVd6un4N7kopNUanuk6x6/gumlqaUo/9rfvp6e8B4PP1n9fgrpRSQWWM4e3Tb9PU0sTOYztpOm4F8sPvHk4tE5sUIx6Lc+2Ca4nH4sRjcS6acZHnZdXgrpRSGfT09XDg5IFBtfGmliZOnT0FgCAsLF/Ih+Z8iE3LNpGYnSAeixObFPO55BYN7koVGrt5VArI6XOn2X18t1Ubb2mi6XgTe0/spbuvG4DScaXUVdZxfe31JGYnqK+sp66yjgvGX+BzybPT4K5UISnwNA3GGN458w47W3YOqo2/ceqN1DIVZRXEY3Hu/NCdxGNxErEEC8oXMK4oXOFSs0IqVUgCMNmEV3r7e3mt7bVUAB8I6Cc7T6aWuWjGRVa7eKXVNp6YnWD2pNmIiI8lH5ndrJDhOhUppcYmomkaOro72HN8z6Aa+Z4TezjbexaA8cXjWTJrCWsvXpvq5KyrrGPKhCk+l9w9GtyVKiTz5mWuuYcoTUNLR8uw2vjrba9jsFohppdOJzE7wR0Nd6Q6OS8uv5iS4hKfS+4tDe5KFZKtWzNPNhHANA19/X0cbD84LJAff/94apmaaTUkYgk2LN1AfWU9idkJ5k6ZG+hmFa9ocFeqkAx0mgZstExnTyd7T+wd1Mm5+/hu3u95H4CSohIWz1rMqotWkYglSMxOUFdZx7TSab6WO8i0Q1Up5anW91uH1cZfbXuVftMPwNQJU1Pt4gO18dqZtYwvHu9zyYNBO1SVUr7qN/0cOnVo2E1AR88cTS1TNaWKRCzB9bXXpwJ6zbQabVZxgNbclVK29PT00NzczNmzZ4f9zRhDT38P3X3dgx7p8aWkuITxxeNTj5KiEoqLir38CqFSWlpKVVUVJSWDO4K15q6UclRzczOTJ0+mal4VXb1ddPZ00tVj/Xu29ywGgyBMlImUl5RTVlLGxHETrX9LJlIkmmHcLmMMbW1tNDc3M3/+/FF9hgZ3pVRGxhjeeu+tVJKsj0/+ONMnTufw8cOpZUqKSigrKWNa6TQmlliBfELxBG1WGSMRoby8nNbW1lF/hgZ3pRTdfd0caD0w7Lb89869B0CRFPGxVR9j8oTJqdp4WUlZwY0d99JYT5Aa3JUqMO+dfW9Q7vGdLTvZd2JfKvd4WUkZdZV13Lj0xlQn55JZS3jr4FtcOP1Cn0tv3+HDh/n973/PjTfeCEBTUxPvvPMO1157LQDbt29n//793HPPPWNe180338zq1atZt24dX/jCF7jrrruora0d8+eOhQZ3pSLKGEPz6eZBQw6bWpp48903U8vMumAWiViCqz98NYnZCRKxBBfNuCgSHZ2HDx/mpz/96aDg3tjYmArua9asYc2aNY6v95FHHnH8M0dDg7tSEdDb38srJ18ZNuywrasNsHKPXzTjIlbMWcFty25LJckKSu5xux5//HEeeOABRIS6ujqeeOKJQbVmgEmTJtHR0cE999zDgQMHiMfjrF+/noceeoiuri5eeOEF7r33Xrq6umhsbOT73/8+N998M1OmTKGxsZGWlha+853vsG7dOvr7+/nSl77Es88+y9y5cykpKeGWW25JrSuTyy+/nAceeICGhgYmTZrEnXfeyVNPPcXEiRP5j//4DyorK2ltbeX222/nSDKnz3e/+10uvfRSR7eVBnelQubMuTPsObEnlXt8Z8tO9p7Yy7m+c4CVe3zprKVcd8l1qdwqS2ctZfKEyY6V4Su//gpNLU2OfR5APBbnu6u+m/Xv+/bt49vf/ja///3vqaiooL29fcTPu//++3nggQd46qmnAKisrEwFc4BHH3100PLHjh3jhRde4JVXXmHNmjWsW7eOf//3f+fw4cPs37+fEydOsGjRIm655Rbb3+n999/nwx/+MFu3buVv/uZv+PGPf8x9993HnXfeyV//9V/zkY98hCNHjnDNNddw4MAB259rhwZ3pQLKGMOxjmPDauMH2w+mkmSVTywnMTvBl1d+OVUbX1i+MHS5x+149tlnuf7666moqABgxowZjn7+pz/9aYqKiqitreX4cSt/zQsvvMD1119PUVERsViMj3/843l95vjx41m9ejUAy5cv5ze/+Q0Av/3tb9m/f39qudOnT9PR0cGkSZMc+jYa3JUKhL7+Pl5vf31QbbyppYnWzvND4S6cfiGJWILP138+1dE5Z/IcX4YdjlTD9tq4cePo77dSF/T399Pd3T2qz5kwYULquVM3d5aUlKR+n+LiYnp7ewGrnDt27KC0tNSR9WSiwV0pj3X2dLL7+O5hSbK6ersAK/f44pmL+eTCT1pJsmJWkqyppVN9Lrm/rrjiCq677jruuusuysvLaW9vZ8aMGdTU1PDyyy/z2c9+lu3bt9PTY436mTx5MmfOnEn9/6Gv7bj00kt57LHH2LhxI62trTz//POpDtqxuPrqq/mnf/on7r77bsDq7I3H42P+3HQa3JVy0Yn3T6RuAmo6bgXy19peSyXJmlY6jUQswe0Nt6cSZS2auUiTZGWwePFitmzZwmWXXUZxcTGJRIJHH32U2267jbVr11JfX8+qVau44AJrXtO6ujqKi4upr6/n5ptvZuPGjdx///3E43HuvfdeW+v8zGc+w+9+9ztqa2uZO3cuy5YtY+rUsZ9kH3zwQb74xS9SV1dHb28vH/vYx/jnf/7nMX9uOs0to5QD+k3/oNzjA49jHcdSy1RPrU7NyTnQrDJv6rzQ3M154MABFi1a5HcxPDfQFt7W1sbKlSt58cUXicW8GWWUaZs7mltGRFYB3wOKgUeMMfcP+ftdwBeAXqAVuMUYk2G6F6XCr6uni32t+wbVyHe17ErlHh9XNI7ambVc9cGriFdanZz1lfVMnzjd55Kr0Vi9ejXvvvsu3d3dfO1rX/MssI9VzuAuIsXAQ8BVQDPwkohsN8bsT1tsJ9BgjOkUkc3Ad4D/5kaBlfJSW2fbsNzjr5x8hT7TB8Dk8ZOpj9VzS+KWVG188czFTBg3Iccnq7B4/vnn/S7CqNipua8EDhpjDgGIyM+BtUAquBtjnktbfgdwk5OFVMpt/aafw+8eHhbIm083p5apmlJFPBYfNH68ZlqNZjtUgWQnuM8B3k573Qx8aITlbwV+NZZCKeWmc73n2N+6f1AQ33V8F6fPnQagWIq5pOISLqu+LFUbj8fiVJRV+FxypexzdLSMiNwENACXZfn7JmATwLwQzbauwutU16lUkqyBQL6/dT+9/dZ44wtKLqCuso6blt6Uuglo8czFTCyZ6HPJlRobO8H9KDA37XVV8r1BROQvgC3AZcaYc5k+yBjzMPAwWKNl8i6tUlkYYzjy3pHzI1WOW52db713vl9/9qTZxGNxPrngk6m5OS+acZE2q6hIshPcXwIWiMh8rKB+AzBoFL+IJIAfAauMMSccL6VSaXr6ejhw8sCwYYenzp4CrCRZC8sX8mdz/4zNDZtTo1UqJ1X6XHKlvJMzuBtjekXkS8AzWEMhf2KM2Sci3wQajTHbgX8AJgH/mhyze8QY43wuTVVwTp87za6WIbnHW/fR3WfdYj5x3ETqKuu4vvb6QUmyLhh/gc8lV2zbBlu2wJEjMG8ebN0KGzZ4tvrHHnuMb3/72wDcd999bNy40bN1B4GtNndjzNPA00Pe+3ra879wuFyqwBhjeOfMO4Pyju9s2cmhU4dSy8wsm0lidoKvXPgV6mP1JGIJFpQviGSSrNDbtg02bYLOTuv1W29Zr8GTAN/e3s7f/u3f0tjYiIiwfPly1qxZw/TphXOvgR4VynO9/b28evLVQe3jTS1NnOw8mVpmwYwFLJu9jFsTt6Zuy//A5A+E5m7Ogrdly/nAPqCz03p/DMH9pZde4tZbb+WPf/wjfX19rFy5kl/84hcsWbJk0HLPPPMMV111VSpz5FVXXcWvf/1r1q9fP+p1h40Gd+Wqju6OYUmy9pzYw9neswBMKJ7AkllL+PTFn07Vxusq6xzNPa58kJyEwvb7Nq1YsYI1a9Zw33330dXVxU033TQssAMcPXqUuXPPjwOpqqri6NFh40AiTYO7ckxLR8uwJFmvt72eyj0+vXQ6idkJ7mi4I9XJeUnFJTrJchTNm2c1xWR6f4y+/vWvs2LFCkpLS3nwwQfH/HlRpcFd5a2vv29QkqyBdvLj7x9PLTN/2nzqY/VsWLohlSyrakqVNqsUiq1bB7e5A5SVWe+PUVtbGx0dHfT09HD27NlUFsh0c+bMGZQ2oLm5mcsvv3zM6w4TzQqpRtTZ08neE3sH1ch3H99NZ4910JYUlbB41uJUAK+vrKc+Vs+00mk+l1w5Le+skC6NllmzZg033HADb775JseOHUtNm5euvb2d5cuX86c//Qlec3JoAAAJIklEQVSAZcuW8fLLLzs+e5PbXM8KqQpD6/utw2rjr7a9mso9PnXCVOpj9akJluOxOLUzazX3uMpswwbHR8Y8/vjjlJSUcOONN9LX18ef//mf8+yzz3LFFVcMWm7GjBl87WtfY8WKFYDVlBO2wD5WWnMvQP2mn0OnDg0L5O+ceSe1zLyp81KjVAbyj9dMq9FmlQJWqPnc/aQ1d5XV2d6z7Duxb9Cww10tuzjTbU03VizFLJq5iCvnXzkoSdaMiYVVy1EqajS4R0h7V/uw2viB1gOp3OOTxk8iHovz+frPp2rji2ctpnSce5P0KuWmPXv28LnPfW7QexMmTOAPf/iDTyUKDg3uIWSMGZR7fCBJ1tunz2dm/sDkDxCPxVl78dpUkqwLp1+oSbJUpCxdupSmpia/ixFIGtwDrruvmwOtBwbdlt/U0sR7594DoEiKuLj8Yj5a/VHilVaTSn2snlkXzPK55EopP2lwD5B3z757PklW8iagfSf20dPfA0BZSRn1lfXcuPTGVG18yawllJWU+VxypVTQaHD3gTGG5tPNw5JkHX73cGqZygsqScxOsOqDq1K18QUzFlBcVOxfwZVSoaHB3WU9fT282vaqdQNQWo28vasdsHKPLyhfwIoPrGDTsk2ptLWxSeGYYV0pFUwa3B105tyZQUmydrbsZO+JvZzrsyamKh1XSl1lHesWrUslyVpauZRJ4yf5XHKlnOdzOndWrVrFjh07+MhHPsJTTz3l3YoDQoP7KBhjONZxbFht/GD7wdQy5RPLicfifHnll1O18YXlCzX3uCoIPqdzB+Duu++ms7OTH/3oR96sMGA00uTQ19/Ha22vDRs/3trZmlrmg9M/SDwWZ2P9xtT4cc09rgqZS+ncbedzB7jyyisHJQ8rNBrc03T2dKaaVQaSZO05voeu3i4AxhePZ8msJXxq4adSnZz1lfVMLZ3qc8mVChaX0rnbzueuCji4H+84PmwmoNfaXkslyZpeOp14LM7tDbenauOae1wpe1xM56753G2KfHDvN/2p3OPpk0i0dLSklqmZVkM8FueGxTekOjrnTZ2nzSpKjZKL6dxt5XNXEQvuXT1dqdzj6Umy3u95H4BxReOonVnLNR+8JpUgq76ynukTC2fSXKW8MNCu7sZomb/6q7/iW9/6Fm+++SZf/epXM+ZzVyEO7ic7Tw66Hb+ppYlXTr6SSpI1ZcIU4rE4tyRuSTWr1M6sZcK4CT6XXKnC4EI6d9v53AE++tGP8sorr9DR0UFVVRX/8i//wjXXXONsgQIsdPncH/nTI3zj+W9w9Mz5yW7nTJ5DYnYiFcTjsTjzp83XZhWlHKT53L1XUPncY5NiXF5z+flp3WL1VJRV+F0spZQKlNAF99ULV7N64Wq/i6GUCgDN555d6IK7UkoN0Hzu2enMDUop2/zqoytEY93WGtyVUraUlpbS1tamAd4Dxhja2tooLR39FJjaLKOUsqWqqorm5mZaW1tzL6zGrLS0lKqqqlH/fw3uSilbSkpKmD9/vt/FUDZps4xSSkWQBnellIogDe5KKRVBvqUfEJFWIENSUFsqgJMOFscpWq78aLnyF9SyabnyM5ZyVRtjZuZayLfgPhYi0mgnt4LXtFz50XLlL6hl03Llx4tyabOMUkpFkAZ3pZSKoLAG94f9LkAWWq78aLnyF9Syabny43q5QtnmrpRSamRhrbkrpZQaQaCDu4isEpFXReSgiNyT4e93ich+EdktIr8TkeqAlOt2EdkjIk0i8oKI1AahXGnLfUZEjIh4MorAxva6WURak9urSUS+EIRyJZf5bHIf2yciPw1CuUTkf6Vtq9dE5N2AlGueiDwnIjuTx+S1ASlXdTI+7BaR50Vk9Alb8ivXT0TkhIjszfJ3EZEHk+XeLSLLHC2AMSaQD6AYeAO4EBgP7AJqhyzzcaAs+Xwz8IuAlGtK2vM1wK+DUK7kcpOB/wR2AA1BKBdwM/D9AO5fC4CdwPTk61lBKNeQ5b8M/CQI5cJqR96cfF4LHA5Iuf4V2Jh8fgXwhEf72MeAZcDeLH+/FvgVIMCHgT84uf4g19xXAgeNMYeMMd3Az4G16QsYY54zxnQmX+4AvDgj2ynX6bSXFwBedGzkLFfSt4C/B856UKZ8yuU1O+W6DXjIGHMKwBhzIiDlSrce+FlAymWAKcnnU4F3AlKuWuDZ5PPnMvzdFcaY/wTaR1hkLfC4sewAponIbKfWH+TgPgd4O+11c/K9bG7FOgu6zVa5ROSLIvIG8B3gvwehXMnLvrnGmP/nQXlslyvpM8lL038TkbkBKddCYKGIvCgiO0RkVUDKBVjNDcB8zgcuv8v1DeAmEWkGnsa6qghCuXYBf5l8fh0wWUTKPShbLvnGuLwEObjbJiI3AQ3AP/hdlgHGmIeMMR8Evgrc53d5RKQI+Efgf/hdlgz+L1BjjKkDfgM85nN5BozDapq5HKuG/GMRmeZriQa7Afg3Y0yf3wVJWg88aoypwmpyeCK53/ntfwKXichO4DLgKBCUbeaaIGz4bI4C6TW4quR7g4jIXwBbgDXGmHNBKVeanwOfdrVEllzlmgwsAZ4XkcNYbXzbPehUzbm9jDFtab/dI8Byl8tkq1xYNantxpgeY8ybwGtYwd7vcg24AW+aZMBeuW4FfglgjPkvoBQrh4qv5TLGvGOM+UtjTAIrVmCM8aQTOod8Y0l+vOhYGGVnxDjgENZl50BHyeIhyySwOlMWBKxcC9KefwpoDEK5hiz/PN50qNrZXrPTnl8H7AhIuVYBjyWfV2BdQpf7Xa7kcpcAh0neqxKQ7fUr4Obk80VYbe6uls9muSqAouTzrcA3vdhmyfXVkL1D9ZMM7lD9o6Pr9upLjnLDXItVW3oD2JJ875tYtXSA3wLHgabkY3tAyvU9YF+yTM+NFGS9LNeQZT0J7ja3198lt9eu5Pa6JCDlEqymrP3AHuCGIJQr+fobwP1elCeP7VULvJj8HZuAqwNSrnXA68llHgEmeFSunwHHgB6sq8BbgduB29P2r4eS5d7j9PGod6gqpVQEBbnNXSml1ChpcFdKqQjS4K6UUhGkwV0ppSJIg7tSSkWQBnellIogDe5KKRVBGtyVUiqC/j/R+irSJx+ggQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出参数更新之前的结果 (FIXME: the plot is wrong)\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0.numpy() * plot_x - b0.numpy()) / w1.numpy()\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到分类效果基本是混乱的，我们来计算一下 loss，公式如下\n",
    "\n",
    "$$\n",
    "loss = -(y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}))\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 计算loss\n",
    "def binary_loss(y_pred, y):\n",
    "    logits = (y * y_pred.clamp(1e-12).log() + (1 - y) * (1 - y_pred).clamp(1e-12).log()).mean()\n",
    "    return -logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意到其中使用 `.clamp`，这是[文档](http://pytorch.org/docs/0.3.0/torch.html?highlight=clamp#torch.clamp)的内容，查看一下，并且思考一下这里是否一定要使用这个函数，如果不使用会出现什么样的结果\n",
    "\n",
    "**提示：查看一个 log 函数的图像**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.8986, grad_fn=<NegBackward>)\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_regression(x_data)\n",
    "loss = binary_loss(y_pred, y_data)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到 loss 之后，我们还是使用梯度下降法更新参数，这里可以使用自动求导来直接得到参数的导数，感兴趣的同学可以去手动推导一下导数的公式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7402, grad_fn=<NegBackward>)\n",
      "tensor(0.7348, grad_fn=<NegBackward>)\n",
      "tensor(0.7299, grad_fn=<NegBackward>)\n",
      "tensor(0.7254, grad_fn=<NegBackward>)\n",
      "tensor(0.7212, grad_fn=<NegBackward>)\n",
      "tensor(0.7175, grad_fn=<NegBackward>)\n",
      "tensor(0.7140, grad_fn=<NegBackward>)\n",
      "tensor(0.7108, grad_fn=<NegBackward>)\n",
      "tensor(0.7079, grad_fn=<NegBackward>)\n",
      "tensor(0.7052, grad_fn=<NegBackward>)\n",
      "tensor(0.7028, grad_fn=<NegBackward>)\n",
      "tensor(0.7005, grad_fn=<NegBackward>)\n",
      "tensor(0.6984, grad_fn=<NegBackward>)\n",
      "tensor(0.6965, grad_fn=<NegBackward>)\n",
      "tensor(0.6947, grad_fn=<NegBackward>)\n",
      "tensor(0.6931, grad_fn=<NegBackward>)\n",
      "tensor(0.6916, grad_fn=<NegBackward>)\n",
      "tensor(0.6901, grad_fn=<NegBackward>)\n",
      "tensor(0.6888, grad_fn=<NegBackward>)\n",
      "tensor(0.6876, grad_fn=<NegBackward>)\n",
      "tensor(0.6865, grad_fn=<NegBackward>)\n",
      "tensor(0.6855, grad_fn=<NegBackward>)\n",
      "tensor(0.6845, grad_fn=<NegBackward>)\n",
      "tensor(0.6836, grad_fn=<NegBackward>)\n",
      "tensor(0.6827, grad_fn=<NegBackward>)\n",
      "tensor(0.6819, grad_fn=<NegBackward>)\n",
      "tensor(0.6811, grad_fn=<NegBackward>)\n",
      "tensor(0.6804, grad_fn=<NegBackward>)\n",
      "tensor(0.6797, grad_fn=<NegBackward>)\n",
      "tensor(0.6791, grad_fn=<NegBackward>)\n",
      "tensor(0.6785, grad_fn=<NegBackward>)\n",
      "tensor(0.6779, grad_fn=<NegBackward>)\n",
      "tensor(0.6773, grad_fn=<NegBackward>)\n",
      "tensor(0.6768, grad_fn=<NegBackward>)\n",
      "tensor(0.6763, grad_fn=<NegBackward>)\n",
      "tensor(0.6758, grad_fn=<NegBackward>)\n",
      "tensor(0.6753, grad_fn=<NegBackward>)\n",
      "tensor(0.6749, grad_fn=<NegBackward>)\n",
      "tensor(0.6745, grad_fn=<NegBackward>)\n",
      "tensor(0.6740, grad_fn=<NegBackward>)\n",
      "tensor(0.6736, grad_fn=<NegBackward>)\n",
      "tensor(0.6732, grad_fn=<NegBackward>)\n",
      "tensor(0.6728, grad_fn=<NegBackward>)\n",
      "tensor(0.6725, grad_fn=<NegBackward>)\n",
      "tensor(0.6721, grad_fn=<NegBackward>)\n",
      "tensor(0.6718, grad_fn=<NegBackward>)\n",
      "tensor(0.6714, grad_fn=<NegBackward>)\n",
      "tensor(0.6711, grad_fn=<NegBackward>)\n",
      "tensor(0.6707, grad_fn=<NegBackward>)\n",
      "tensor(0.6704, grad_fn=<NegBackward>)\n",
      "tensor(0.6701, grad_fn=<NegBackward>)\n",
      "tensor(0.6698, grad_fn=<NegBackward>)\n",
      "tensor(0.6694, grad_fn=<NegBackward>)\n",
      "tensor(0.6691, grad_fn=<NegBackward>)\n",
      "tensor(0.6688, grad_fn=<NegBackward>)\n",
      "tensor(0.6685, grad_fn=<NegBackward>)\n",
      "tensor(0.6682, grad_fn=<NegBackward>)\n",
      "tensor(0.6679, grad_fn=<NegBackward>)\n",
      "tensor(0.6676, grad_fn=<NegBackward>)\n",
      "tensor(0.6673, grad_fn=<NegBackward>)\n",
      "tensor(0.6671, grad_fn=<NegBackward>)\n",
      "tensor(0.6668, grad_fn=<NegBackward>)\n",
      "tensor(0.6665, grad_fn=<NegBackward>)\n",
      "tensor(0.6662, grad_fn=<NegBackward>)\n",
      "tensor(0.6659, grad_fn=<NegBackward>)\n",
      "tensor(0.6656, grad_fn=<NegBackward>)\n",
      "tensor(0.6654, grad_fn=<NegBackward>)\n",
      "tensor(0.6651, grad_fn=<NegBackward>)\n",
      "tensor(0.6648, grad_fn=<NegBackward>)\n",
      "tensor(0.6645, grad_fn=<NegBackward>)\n",
      "tensor(0.6643, grad_fn=<NegBackward>)\n",
      "tensor(0.6640, grad_fn=<NegBackward>)\n",
      "tensor(0.6637, grad_fn=<NegBackward>)\n",
      "tensor(0.6634, grad_fn=<NegBackward>)\n",
      "tensor(0.6632, grad_fn=<NegBackward>)\n",
      "tensor(0.6629, grad_fn=<NegBackward>)\n",
      "tensor(0.6626, grad_fn=<NegBackward>)\n",
      "tensor(0.6624, grad_fn=<NegBackward>)\n",
      "tensor(0.6621, grad_fn=<NegBackward>)\n",
      "tensor(0.6618, grad_fn=<NegBackward>)\n",
      "tensor(0.6616, grad_fn=<NegBackward>)\n",
      "tensor(0.6613, grad_fn=<NegBackward>)\n",
      "tensor(0.6610, grad_fn=<NegBackward>)\n",
      "tensor(0.6608, grad_fn=<NegBackward>)\n",
      "tensor(0.6605, grad_fn=<NegBackward>)\n",
      "tensor(0.6603, grad_fn=<NegBackward>)\n",
      "tensor(0.6600, grad_fn=<NegBackward>)\n",
      "tensor(0.6597, grad_fn=<NegBackward>)\n",
      "tensor(0.6595, grad_fn=<NegBackward>)\n",
      "tensor(0.6592, grad_fn=<NegBackward>)\n",
      "tensor(0.6589, grad_fn=<NegBackward>)\n",
      "tensor(0.6587, grad_fn=<NegBackward>)\n",
      "tensor(0.6584, grad_fn=<NegBackward>)\n",
      "tensor(0.6582, grad_fn=<NegBackward>)\n",
      "tensor(0.6579, grad_fn=<NegBackward>)\n",
      "tensor(0.6576, grad_fn=<NegBackward>)\n",
      "tensor(0.6574, grad_fn=<NegBackward>)\n",
      "tensor(0.6571, grad_fn=<NegBackward>)\n",
      "tensor(0.6569, grad_fn=<NegBackward>)\n",
      "tensor(0.6566, grad_fn=<NegBackward>)\n"
     ]
    }
   ],
   "source": [
    "# 自动求导并更新参数\n",
    "for i in range(10):\n",
    "    w.grad.data.zero_()\n",
    "    b.grad.data.zero_()\n",
    "    \n",
    "    # calc grad\n",
    "    loss.backward()\n",
    "    w.data = w.data - 0.1 * w.grad.data\n",
    "    b.data = b.data - 0.1 * b.grad.data\n",
    "\n",
    "    # 算出一次更新之后的loss\n",
    "    y_pred = logistic_regression(x_data)\n",
    "    loss = binary_loss(y_pred, y_data)\n",
    "    print(loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fb0598c0cc0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xt8VPWd//HXNyEh3BQIyD0JICqogNy8glwTSv2hVv2pgD+pVapb225/24fVRW23LRV39/fT1bZbqXW9JFq3aFvWtSYBQQFFDRYEES0JF0EEDF64SiDf/WMmx0lIJjOZM3POzHk/H495MDM5nPOZM2e+n/O9nO8x1lpERCR4srwOQEREvKEEICISUEoAIiIBpQQgIhJQSgAiIgGlBCAiElBKACIiAaUEICISUEoAIiIB1c7rAKLp0aOHLSoq8joMEZG0sXbt2k+stT1jWdbXCaCoqIiqqiqvwxARSRvGmO2xLqsmIBGRgFICEBEJKCUAEZGAUgIQEQkoJQARkYByJQEYYx4zxuw1xmxs4e/GGPOQMWaLMeYdY8woN7YbGGVlUFQEWVmhf8vKvI5IRDKAWzWAx4HpUf7+NWBI+DEP+HeXtpv5yspg3jzYvh2sDf07b57rSUA5Jj3pe5NEuJIArLWvAvujLHI58KQNWQN0Ncb0cWPbGW/+fDh8uPF7hw+H3ndJinJMxooshHv0CD1SUSDH+70lO1koGaUha60rD6AI2NjC314ALol4vQwY08Ky84AqoKqgoMAGTmmptYWF1hoT+jf02z75YYxrm2xpM4WFrm0iY5WWWtuxY8tfU8eOoWWSIZ7vrbk43Ywt2euX2AFVNtZyO9YFW12RSwkg8jF69Ogk7SKfau5XZEzSS+eWNuFijslY0XJ0shNpS99bwzYbziEazimSGZtOIvwjngSQqlFAu4ABEa/7h9+TSM0191gLxjR+r2NHWLDAtc0WFMT3vnxlxw53lmmLlr4fY05uFtrewuQAbsXW0noa3lfzkD+lKgEsAf5PeDTQBcDn1trdKdp2+mjpV2QtFBaGftmFhbBoEcye7dpmFywI5ZRILueYjBVLkkxWIm3uezMmdLhEOnwYsrPjjy2eQjvaSURZGXzzm42T0je/qSTgC7FWFaI9gGeA3UAdsBP4FnArcGv47wb4FVANbCCG5h8bxCYgD+vRTbse0rntNpWfxcs+gIbtx9Jl1BBLLLGVllqbnx/fZ4nWB9DcuiD0vh+l+28BL/oAkvEIXAIIQE9asn9cXuzCyM+Unx96eFV4RDuHiGXft5bQItfV3P9tbv3R1uW1pjHfdlv6/wSVANKZD08/3AopFYVzKipRPvyKHInu41g6teNdp18TgBdjLlJx7CgBiGvcLLRTUTgne0STHypprRUiiRQysRT+8X5vfm0CijXZuXX8pOrYUQIQ17hZaKdiuGmmD3dMZiFSWhp9aGlbv7fSUmtzcxv/39xc72tO8XxWN77fVB078SQATQYnUbU2vC8eqRhumuwRTW7uj7ZI5oXh8+eHiqRYxfq9zZ4Njz3WeCDbY4+5OpCtTaINo43k1vHj9bHTHCUAicrNQjsVw01nzw6Nkk3WqNlE90ei4+GTWYjEs454v7fZs2HbNqivD/3rdeEPLR+Pt96anOPHl9fbxFpV8OKhJiDvld620nY0h1xrcvBzB2osEmmCcaP5JpnNCLG2iafj99YSr4cMqw9ACSA+HhyxpVxvC9lqDSdsodluS29bmbxtpoG2fgVuFN5tKURijdfraxqCQKOAlADazs1TiFiORK97PDNMtLPqeMRTiMR7yHhxTUM61gr9HLMSQCYqLbU2O9udAjnWUsEHs8R5/UNzc/stfX3Z2W5FezIvcngyE5Qf+D1mJYBM01rdPN4COdZSweMagNc/NLe371YNIB6pzuHRLq5qLhmkYyXT7zErAWSa1nrnsrPjO0WNtVTwuAT2+ocWz7QKt93mzxa1VG8zlvmIIveNDyqZcUtmzG7UOJUAMk08V6zEUkBH+5U2Peo8bIPxunCItttbmy+nua/Bq3mKkrHNlg6LWA7VyOTjdZJvi2TF7NZ3pQSQaeK5Zj2WIzFNhnskOrFZsrbfUlu+X/NpPKOA2jpaqKHgj2XfRCbwVCZFv89p5VZiUQLINC0dcbH8wqKts7WagMda+tipmrEx3t3u83waVTyFWrznI60dWqlIim4X2m2NuaX/V1qa2M85khJAJmruyHHjlMGtdpYk/YqT9bGTsf14Cr1Etp8M8ezTWJp5GmoCTZf1Khn6oakpnhMa1QCUAFrnxlVBLU3VmOyrkxLQ1pyVzCaA1h5t6bdI5W6NZ5/GkgAb/p/Xw3gbREtaicQUz+drS5Oi+gAkukQHXefknDxVo5/mJ3Bpc8luAmgYBeRmDSCVuzWebcWSAH3QgthItO8mkWsp4zmm4hnLkUhyUgKQ5rX0K8jPT+w0LcXDdUpLre2YW9f4h5db16azL7cLKjcTTSp3a7xxRzaF+aWZJ5rWklYqEnS8NYC2HptKANK8ZJUoqW5gLS21pTlzv5qfiK22lFmh0/AWpLowTbTZo9TFC7/j2aabHZt+42ZHq7XxH1PR+gDcvF+CEkCmSvSX5vcBzJHri/Y5W/ocxrS4zdY+up8KsWhnq348u04nbv4E2toU2fQ4Ky0NtcRGriMnRwlACSCSG4VsMnsVUznIOlpjagu/vmirTXEfdquiNRWo8E+Mm9+1W+ty+7xMCSATuXWUJPtUNxW1lGg9elHq8i2F5ochgpG8vgLaL6KNmU/kEHPzJ+DGutz+vpUAMlE6lApunBLF8jlLS1terg2ltt92rd8Skhe8vggwlbysAeiWkOnCl/eTa8KNG9bG8jlnzw7dt8+lm7f6bdem4taZftfSobRoUfLuiewVT7/vWDOFFw/VACL4raG6OW6cSsfzOWOpf8ewjOu71oV2AZc+WtqKd8y8nyrCbeHmd4magDKU33/xidZlIweXN4yBTHaHcpNNJ7xrU5So0+F8IBGpGjOfiVKeAIDpwPvAFuDOZv4+F9gHrAs/bo5lvRmVAPxeeLshkVKpuf+bm5vYPQk9uD4hVSVUpvcTBKkPwG0pTQBANlANDAJygfXAsCbLzAV+Ge+6MyYBZPrpWqS2JrpYJ5iJcrHXSby+lDaJ2/Rbx7XbSksbT1OVn+/eKKBMl+oEcCFQHvH6LuCuJssEOwEk43Qt034F8TT6xlor8MNkOm5uM+I7L8z+MKkfLdrhlYqRxEE5X0qGVCeAq4FHI17f0LSwDyeA3cA7wGJgQJT1zQOqgKqCgoKk7qiUcft0LRN/IbHUAJp7tDZhjdfTabq1zSafpZTrbUcOJuWjeX3RXCrytt+uBXCTHxNAPtA+/PzbwMuxrDvjawD5+bGvI/Ioy8SesNaaUNp6dp2qX2eyL99tZv2lXG8Lsz90/aNFK4Ddmv4gmmQ3b/nxamA3+a4JqMny2cDnsaw7YxJAaenJsz1B8xN+tDRZSCyFY7o3AEd+9pbuU+CXz930e0pW72TkyKgUffZoBbBbE6A12i1N9mVh/oGouT7RXRptd8a7fj92xqc6AbQDaoCBEZ3AZzdZpk/E8yuBNbGsO2MSgLWx3XilpV9LrIVhOtcAmnPbbfHfZTwVog1RcWs+pIaSpbXPn4TP3lIB3JYaQKvLN7MvS3PmnjTdt5t5tbVdGs/6/dgZ78Uw0BnAB+HRQPPD7/0UmBl+fh/wbjg5LAfOimW9GZUAYjlS2toO7savwq+iDa306nMn87QvnqawJN0IuTRn7sn9C+H7LcTb5NHqYd/CviztdEurZ+ptFcvPLNb1B74GkMxHRiWAWI6UeC9/zM72T89TMrVUKEaODWzLOtt6tp7M075YTwKS9Z2Ht1/K9Y3vt5D/XWeReHZdq4d9K/dqTMaujiXHNrv+Zj54swkxty60vzz6bSoB+FEsp07ROov91tOUam4P20hkfybztM/rJi+XS9xWd3Urp/nJ2tWtda2ctP4oH6TRoZl/wJbmzPX0t6oE4FetFWKtjb/z01izdBT3rz7KepKVkFurASS7MElCiRv10C0tbfmzGpP0UTYxrz/W/eKDNiElgHSmgj452lzvj7K+ZHxPzcXZcFaeiuPBi3GNrQyQSMWFZ62uP9aakQ96hZUARJpys+cv2bw+CUj19v04mL4p1QCUACSNuTn2T9znddJrTaxJygfJLJ4EoBvCSDBEu7tLYWHoTiOzZ6cuHr8rK4OiIsjKCv1bVpbc7c2eDdu2QX196F+/fRezZ4eOkcLC0I2IWjpmYl3OJ0woYfjTmDFjbFVVlddhSCYoK4N58xrfTqpjR1//OD2jfZXWjDFrrbVjYllWNQAJhjQ7M/OUG7f2lLSgGoCINJaVFWq9bsqYUBON+JpqACLSdi31l0TrR5G0pAQgIo0tWBBq84/UsWPofckoSgAi0pj6SwKjndcBiIgPzZ6tAj8AVAMQEQkoJQARkYBSAhARCSglABGRgFICEBEJKCUAEZGAUgIQEQkoJQARkYBSAhARCSglABGRgFICEBEJKCUAEZGAUgIQEQkoVxKAMWa6MeZ9Y8wWY8ydzfy9vTHm2fDf3zDGFLmxXRERabuEE4AxJhv4FfA1YBhwvTFmWJPFvgV8aq09HXgAuD/R7YqISGLcqAGMA7ZYa2ustceA3wOXN1nmcuCJ8PPFwBRjjHFh2yIi0kZuJIB+wIcRr3eG32t2GWvtceBzIL+5lRlj5hljqowxVfv27XMhPBERaY7vOoGttYustWOstWN69uzpdTgiIhnLjQSwCxgQ8bp/+L1mlzHGtANOBWpd2LaIiLSRGwngLWCIMWagMSYXuA5Y0mSZJcCN4edXAy9ba60L2xYRkTZK+Kbw1trjxpjbgXIgG3jMWvuuMeanQJW1dgnwO+ApY8wWYD+hJCEiIh5KOAEAWGtfBF5s8t69Ec+PAte4sS0REXGH7zqBRUQkNZQAREQCSglARCSglABERAJKCUBEJKCUAEREAkoJQEQkoJQAREQCSglARCSglABERAJKCUBEJKCUAEREAkoJQEQkoJQAREQCSglARCSglABERAJKCUBEJKCUAEREAkoJQEQkoJQAREQCSglARCSglABERAJKCUBEJKCUAEREAkoJQEQkoJQAREQCKqEEYIzpboypNMb8LfxvtxaWO2GMWRd+LElkmyIi4o5EawB3AsustUOAZeHXzTlirR0ZfsxMcJsiIuKCRBPA5cAT4edPAFckuD4REUmRRBNAL2vt7vDzj4FeLSyXZ4ypMsasMcYoSYiI+EC71hYwxiwFejfzp/mRL6y11hhjW1hNobV2lzFmEPCyMWaDtba6he3NA+YBFBQUtBaeiIi0UasJwFo7taW/GWP2GGP6WGt3G2P6AHtbWMeu8L81xpgVwHlAswnAWrsIWAQwZsyYlhKKiIgkKNEmoCXAjeHnNwJ/brqAMaabMaZ9+HkP4GJgU4LbFRGRBCWaABYC04wxfwOmhl9jjBljjHk0vMxQoMoYsx5YDiy01ioBiIh4rNUmoGistbXAlGberwJuDj9/DTg3ke2IiIj7dCWwiEhAKQGIiASUEoCISEBlZAIoe6eMt3e/Tb2t9zoUERHfSqgT2I+O1B3hpiU3cezEMXp27Mm0wdMoHlRM8eBi+nTp43V4IiK+kXEJoENOB7Z9fxtLa5ZSUVNBRXUFT294GoBzTzuX4sGhZDC+YDwdcjp4HK2IiHeMtf692HbMmDG2qqoqoXXU23rWf7yeyppKKqorWLljJcdOHCOvXR4TCidQPKiYktNLOLvn2RhjXIpcRMQbxpi11toxMS2b6QmgqcN1h3ll2ytUVFdQUVPBpn2ha9L6dukbqh0MKmbqoKn07NTT1e2KiKSCEkAcPvz8QyprKimvLmdpzVL2H9kPwKg+o5zawUUDLiI3OzepcYiIuEEJoI1O1J9g7e61VFaHEsLrO1/neP1xOuV0YmLRREoGl1A8uJgz8s9Qc5GI+JISgEsOfHmA5duWU1FdQXl1OVv2bwGg4NQCp3YweeBkunfo7lmMIiKRlACSpObTGqd2sGzrMr748guyTBZj+451Rhed3+98crJzvA5VRAJKCSAFjtcf542db1BRXUFlTSVv7HqDelvPKe1PYfLAyU4NYVC3QV6HKiIBogTggU+PfMrLW1+mvLqc8upydny+A4DB3QZTPLiYksElTBo4iVPan+JxpCKSyZQAPGat5W/7/0b5lnIqaipYvnU5h+oOkW2yuXDAhU5n8ug+o8nOyvY6XBHJIEoAPnPsxDFe//B1yqvLqaiuYO3utQB079CdqYOmOlNVDDh1gMeRiki6UwLwuX2H9jWaquKjAx8BcFaPs5zawaWFl9Ipt5PHkYpIulECSCPWWjbt2+TUDl7d/ipHjh8hNzuXSwoucWoHI3qPIMtk5OStIuIiJYA0dvT4UVbtWEX5llBn8oa9GwA4rdNpTBs0jeLBxUwbNE0zm4pIs5QAMsjuA7tZWrOU8upyKmsq2XtoLwDDew13agfjC8eT1y7P40hFxA+UADJUva3nnT3vOKOLVu1Y5cxsemnhpc7FaJrZVCS4lAAC4tCxQ7yy/RXKt4RqB+998h6gmU1FgkwJIKAiZzatrK7k06OfYjChmU3DF6NdOOBCzWwqksGUAMSZ2bSiOjTU9LUPX+OEPUGnnE5MGjjJ6T/QzKYimUUJQE7yxZdfsGLbCmd0UfWn1QAUnlro1A4mD5xMtw7dPI5URBKhBCCtqvm0xqkdRM5sOq7fOKd2cH7/82mXlXG3jRbJaClLAMaYa4CfAEOBcdbaZktrY8x04N+AbOBRa+3CWNavBJAadSfqeHPXm85tMt/c9aYzs+mUgVOc0UWa2VTE/1KZAIYC9cAjwA+bSwDGmGzgA2AasBN4C7jeWruptfUrAXhj/5H9oZlNw8NNG2Y2Pb376U7tQDObivhTPAkgofq9tfa98AajLTYO2GKtrQkv+3vgcqDVBCDe6N6hO1cPu5qrh12NtZYPaj9w7or2xPon+HXVr2mX1Y4L+381s+moPqM0s6lImklFA28/4MOI1zuB81OwXXGBMYYze5zJmT3O5Lvnf5cvj3/J6ztfdxLC3cvv5u7ldzszmzYkhP6n9Pc6dBFpRasJwBizFOjdzJ/mW2v/7HZAxph5wDyAgoICt1cvCWrfrj0TiyYysWgiv5jyi0Yzm5ZvKec/3/1PAIb2GOokgwmFEzSzqYgPuTIKyBizgpb7AC4EfmKtLQm/vgvAWntfa+tVH0B6sdayce9G52K0V7e/ytHjR8nNzmV8wXinM3l4r+Ga2VQkSVI+DLSVBNCOUCfwFGAXoU7gWdbad1tbrxJAejtSd4RVO1Y5zUXNzWxaPLiY3p2bq2CKSFukchTQlcDDQE/gM2CdtbbEGNOX0HDPGeHlZgAPEhoG+pi1dkEs61cCyCy7D+ymsqbSuf5g3+F9QGhm04bmoksKLtHMpiIJ0IVg4nv1tp71H693agerdqyirr7Omdm0ISEM6zlMU1WIxEEJQNJO5Mym5dXlvF/7PgD9uvRzmoqmDppKj449PI5UxN+UACTt7fh8h9NUtLRmaaOZTRtqB5rZVORkSgCSUU7Un6DqoypnqorXP3ydE/YEnXM7M6loklNDGNJ9iJqLJPCUACSjfX70c5ZvW+7cCKdhZtOirkXOVBWa2VSCSglAAqV6f7Vz7cGymmUcOHaALJPF+f3Od2oH4/qN08ymEghKABJYdSfqeGPXG07t4M1db2KxnNr+VKYMmuLUEAZ2G+h1qCJJoQQgErb/yH6W1Sxzhpt++EVoWqoh3Yc4tYNJRZPo0r6Lx5GKuCOjE0BdXR07d+7k6NGjHkUVHHl5efTv35+cnByvQ3GFtZb3a993ksGKbSs4XHeYdlntuGjARRQPKqbk9BJG9RmlqSokbWV0Ati6dStdunQhPz9fIz6SyFpLbW0tBw4cYODAzGwu+fL4l7z24WtOQvjrx38FIL9DfqOZTfud0s/jSEVil9EJ4L333uOss85S4Z8C1lo2b97M0KFDvQ4lJfYe2hua2TR8/cHug7sBOLvn2U5z0YTCCXTM6ehxpCItS9kNYbyiwj81grafT+t0GrPOncWsc2dhreXdfe86d0X796p/54E1DzSa2bRkcAnn9jpXzUWStnTkJtm2bdt4+umnndfr1q3jxRdfdF4vWbKEhQtjukVyq+bOncvixYsBuPnmm9m0STddaytjDOecdg7/cNE/UD6nnP137Kd8TjnfHfdd9h7ay4+W/oiRj4yk7//ryw1/vIGn1j/Fxwc/9jpskbikZQ0gnTQkgFmzZgGhBFBVVcWMGTMAmDlzJjNnznR9u48++qjr6wyyDjkdnGYggI8OfERFdQWVNZW8tOUlSt8pBWBErxFO7eDigos1s6n4mmoAbfDkk08yfPhwRowYwQ033AA0PvsG6Ny5MwB33nknK1euZOTIkdx///3ce++9PPvss4wcOZJnn32Wxx9/nNtvv91Zx/e+9z0uuugiBg0a5Kyvvr6ev/u7v+Oss85i2rRpzJgxo9G2mjNx4kQa+k86d+7M/PnzGTFiBBdccAF79uwBYN++fVx11VWMHTuWsWPHsnr1and3VAbr26Uvc0fOpewbZez54R7WzlvLLyb/gu4duvPgmgeZ+tRUut/fna+VfY0H1zzIpn2b8HN/mwRTWtcA/v6lv2fdx+tcXefI3iN5cPqDLf793Xff5ec//zmvvfYaPXr0YP/+/VHXt3DhQv71X/+VF154AYBevXpRVVXFL3/5SwAef/zxRsvv3r2bVatWsXnzZmbOnMnVV1/N888/z7Zt29i0aRN79+5l6NCh3HTTTTF/pkOHDnHBBRewYMEC7rjjDn77299y99138/3vf58f/OAHXHLJJezYsYOSkhLee++9mNcrIVkmi1F9RjGqzyjuGn8XB48d5JVtr1BeHboY7QflPwC+mtm0ZHAJUwZN0cym4rm0TgBeePnll7nmmmvo0SP04+3evbur67/iiivIyspi2LBhzpn6qlWruOaaa8jKyqJ3795MmjQprnXm5uZy2WWXATB69GgqKysBWLp0aaN+gi+++IKDBw86tRdpm865nfn6GV/n62d8HYDtn213pqr40+Y/8R/r/gODYXTf0c5Q0wv6X6CZTSXl0joBRDtTT7V27dpRX18PhJpsjh071qb1tG/f3nnuVpNBTk6OM6InOzub48ePA6E416xZQ16e2qmTqbBrITePupmbR93szGxaXl1ORXUFC1ctZMHKBXTO7czkgZOdqSpO73564EZhSeqpDyBOkydP5g9/+AO1tbUAThNQUVERa9euBUIje+rq6gDo0qULBw4ccP5/09exuPjii3nuueeor69nz549rFixwoVPAsXFxTz88MPO63Xr3G1Ok5NlZ2Vzfv/zuffSe1l10ypq76jlj9f+kTnnzmHDng3c/pfbOeOXZzDooUF8+7++zfPvPc9nRz/zOmzJUGldA/DC2Wefzfz587n00kvJzs7mvPPO4/HHH+eWW27h8ssvZ8SIEUyfPp1OnToBMHz4cLKzsxkxYgRz587lxhtvZOHChYwcOZK77rorpm1eddVVLFu2jGHDhjFgwABGjRrFqaeemvBneeihh/jOd77D8OHDOX78OBMmTOA3v/lNwuuV2J2adypXnHUFV5x1BRCa2bThyuRnNj7DorcXkW1CSaOhdjC231jNbCquSMsrgYNyZWqkhrb52tpaxo0bx+rVq+ndu3fStxvU/e0HdSfqWLNzjXMjnLd2vYXF0jWvK1MGTnGGpRZ1LfI6VPGRjL8SOIguu+wyPvvsM44dO8Y999yTksJfvJWTncP4wvGMLxzPzyb/jNrDtSzb+tXMps+99xwAZ+Sf4dQOJhZN1MymEjPVACQq7W9/stay+ZPNTu2gYWbTnKyc0Mym4dqBZjYNnoyfDE4FUupof6eHL49/yeoPVzsT2UXObDpt8DSKBxUzbfA0+p/S3+NIJdnUBCQSMO3btWfywMlMHjiZhVMXsufgHiprKqmsqaSiuoLfb/w9oJlNpTElAJEM1KtzL+YMn8Oc4XOw1rJx70bn2oNfv/VrHljzAO2z2zO+cLzTfzC813BdexAwagKSqLS/M8+RuiOs3LHSmep6496NAPTq1MupHUwbNI1enXt5HKm0RcqagIwx1wA/AYYC46y1zd7A1xizDTgAnACOxxqciLivpZlNK6or+MuWv/DUO08BoXmxGmoHmtk0MyU6PGAj8A3g1RiWnWStHZnywr+sDIqKICsr9G9ZWco2/cQTTzBkyBCGDBnCE088kbLtisSjYWbTp696mj0/3EPVLVX8YvIv6JrXlQfWPODMbDqjbIZmNs0wCdUArLXvgY/vHFVWBvPmweHDodfbt4deA8yendRN79+/n3/6p3+iqqoKYwyjR49m5syZdOvWLanbFUlElslidN/RjO47mrvG38WBLw/wyvZXnGsPGmY27X9Kf4oHFVNyeglTBk4hv2O+x5FLW6RqgLAFKowxa40x81K0TZg//6vCv8Hhw6H32+itt95i+PDhHD16lEOHDnH22WezcePGk5YrLy9n2rRpdO/enW7dujFt2jReeumlNm9XxAtd2nfhsjMu46GvPcT7t7/P1u9vZdFlizi/3/k8v/l5rl18LT3/pSfjfjuOe16+h5XbV1J3os7rsCVGrdYAjDFLgeYuO51vrf1zjNu5xFq7yxhzGlBpjNlsrW222SicIOYBFBQUxLj6FuzYEd/7MRg7diwzZ87k7rvv5siRI8yZM4dzzjnnpOV27drFgAEDnNf9+/dn165dbd6uiB8UdS3iltG3cMvoWzhef5y3dr3lTHV936r7+PnKn9MltwuTBk7SzKZpoNUEYK2dmuhGrLW7wv/uNcb8ERhHC/0G1tpFwCIIjQJKaMMFBaFmn+beT8C9997L2LFjycvL46GHHkpoXSLpql1WOy4ccCEXDriQey+9l8+Ofsbyrcud5qIl7y8BYGDXgU6n8+SBk+ma19XjyKVB0q8DMMZ0ArKstQfCz4uBnyZ7uwAsWNC4DwCgY8fQ+wmora3l4MF3YCRnAAAKR0lEQVSD1NXVcfToUWfmz0j9+vVrNG3zzp07mThxYkLbFfGzrnlduXLolVw59EqstWzZv8WpHZRtKOORtY9oZlOfSeg6AGPMlcDDQE/gM2CdtbbEGNMXeNRaO8MYMwj4Y/i/tAOettbGVAK7ch1AWVmozX/HjtCZ/4IFCXcAz5w5k+uuu46tW7eye/du5/aOkfbv38/o0aN5++23ARg1ahRr1651/Q5iyabrAMQNdSfqeH3n61RWhxJC1UdVJ81sWjK4hMKuhV6HmvZSdh2AtfaPfFW4R77/ETAj/LwGGJHIdhIye7arI36efPJJcnJymDVrFidOnOCiiy7i5ZdfZvLkyY2W6969O/fccw9jx44FQs1G6Vb4i7glJzuHCYUTmFA4wZnZdGnNUqeG0HRm05LTS5hYNJHOubo9aTLpSmCJSvtbks1ay3ufvOfUDlZsW8GR40cazWxaMriE8/qcp5lNY6DZQMU12t+Sag0zmzZMVbHu49CtSnt07MG0QdOcqSr6ndLP40j9SbOBptCGDRu44YYbGr3Xvn173njjDY8iEklvkTOb3s/97Dm4h6U1S53J7J7Z+AwA55x2jtOZPKFwAh1yOngcefpRDUCi0v4WP7HW8s6ed5wb4azcvpIvT3xJ++z2TCic4Aw3Pfe0cwN77YGagMQ12t/iZ4frDvPq9ledaw827dsEQO/OvUPJIHwjnNM6neZxpKmjJiARCYSOOR2Zfvp0pp8+HYCdX+yksrqSipoK/vuD/+bJ9U8CcF7v85zawcUDLqZ9u/Zehu0bqgFIVNrfkq7qbT1v737bmep69YerOV5/nI45HZlYNNEZbnpm/pkZ1VykGoCIBF6WyWJM3zGM6TuGfxz/jxz48gDLty13aggv/u1FKIcBpwxwagdTB02le4fgXK+T8YNqPbwdANOnT6dr165cdtllqduoiDSrS/suzDxzJg/PeJj3b3+fmu/V8MhljzCu3zgWb1rMtYuvpcc/9+D8R88PzMymGd0E1PR2ABCaCmjRoqTfDgCAZcuWcfjwYR555BFeeOGF5G8wCdQEJEHQMLNpw+iiNTvXUG/r6ZLbhckDJzsXow3uPtjrUFsVTxNQRtcAknA7gJjvBwAwZcoUunTp0vaNiUhKNMxs+uOJP2b1TaupvaOWxdcsZta5s1i/Zz3fefE7nP7w6Qx+aDC3vXAbf9r8Jz4/+rnXYScso/sAknA7gJjvByAi6atrXleuGnYVVw27ypnZtLy6nMqaSko3lPKbtb8h22RzQf8LKBlcQvHgYsb0HUN2VrbXoccloxNAkm4HoPsBiASIMYYh+UMYkj+E28fdzrETx1izc40zVcWPV/yYe1fcS7e8bkwZNMVJCAWnJljQpEBGJ4Ak3Q4gpvsBiEhmys3OdWY2XTBlAZ8c/oRlNcuci9EWb1oMwJn5Zzp9B5cWXerLmU0zOgE0dPS6fDsAvv3tb/Ozn/2MrVu38qMf/ajZ+wGISDD06NiDa8+5lmvPudaZ2bR8S6i56NG3H+XhNx8mJyuHSwoucYabjuw90hczm2b0KKBkePLJJ/nzn//Mc88959wP4L777jvpfgAA48ePZ/PmzRw8eJD8/Hx+97vfUVJS4kHUbef1/hZJZ0ePH2X1jtXORHbr96wHoGfHnkwbPM2ZqqJvl76ubVNzAYlrtL9F3PPxwY8bzWy699BeIDSz6ZLrljCw28CEt6ErgUVEfKh3597MGT6HOcPnUG/rnZlNX9n+iif3N1ACSJDuByAibZFlshjZeyQje4/kjovv8CQGJYAEnXvuuaxbt87rMERE4uZ9N3Qb+LnfIpNoP4tktrRLAHl5edTW1qpwSjJrLbW1teTl5XkdiogkSdo1AfXv35+dO3eyb98+r0PJeHl5efTv39/rMEQkSdIuAeTk5DBwYOJDpUREgi7tmoBERMQdSgAiIgGlBCAiElC+ngrCGLMPaGZC55j0AD5xMRy3KK74KK74KK74ZGJchdbanrEs6OsEkAhjTFWs82GkkuKKj+KKj+KKT9DjUhOQiEhAKQGIiARUJieARV4H0ALFFR/FFR/FFZ9Ax5WxfQAiIhJdJtcAREQkirROAMaY6caY940xW4wxdzbz9/9rjNlkjHnHGLPMGFPoo9huNcZsMMasM8asMsYM80NcEctdZYyxxpiUjJCIYX/NNcbsC++vdcaYm/0QV3iZ/x0+zt41xjzth7iMMQ9E7KsPjDGf+SSuAmPMcmPMX8O/yxk+iaswXEa8Y4xZYYxJ+iRYxpjHjDF7jTEbW/i7McY8FI75HWPMKNeDsNam5QPIBqqBQUAusB4Y1mSZSUDH8PPbgGd9FNspEc9nAi/5Ia7wcl2AV4E1wBg/xAXMBX7pw2NsCPBXoFv49Wl+iKvJ8t8FHvNDXITatm8LPx8GbPNJXH8Abgw/nww8lYK4JgCjgI0t/H0G8BfAABcAb7gdQzrXAMYBW6y1NdbaY8DvgcsjF7DWLrfWHg6/XAOkamrLWGL7IuJlJyAVnTGtxhX2M+B+4GgKYoonrlSLJa5bgF9Zaz8FsNbu9Ulcka4HnvFJXBY4Jfz8VOAjn8Q1DHg5/Hx5M393nbX2VWB/lEUuB560IWuArsaYPm7GkM4JoB/wYcTrneH3WvItQtk0FWKKzRjzHWNMNfDPwPf8EFe4mjnAWvvfKYgn5rjCrgpXhRcbYwb4JK4zgDOMMauNMWuMMdN9EhcQatoABvJV4eZ1XD8B5hhjdgIvEqqd+CGu9cA3ws+vBLoYY/JTEFs08ZZxcUvnBBAzY8wcYAzwL17HEsla+ytr7WDgR8DdXsdjjMkC/j/wD17H0oz/AoqstcOBSuAJj+Np0I5QM9BEQmfavzXGdPU0osauAxZba094HUjY9cDj1tr+hJo4ngofd177IXCpMeavwKXALsAv+yxp/LDj22oXEHkW2D/8XiPGmKnAfGCmtfZLP8UW4ffAFUmNKKS1uLoA5wArjDHbCLU7LklBR3Cr+8taWxvx/T0KjE5yTDHFReisbIm1ts5auxX4gFBC8DquBteRmuYfiC2ubwH/CWCtfR3IIzTvjadxWWs/stZ+w1p7HqHyAmttSjrOo4i3HIlfsjs6ktiB0g6oIVS9bejYObvJMucR6vwZ4sPYhkQ8/19AlR/iarL8ClLTCRzL/uoT8fxKYI1P4poOPBF+3oNQlT3f67jCy50FbCN8vY9P9tdfgLnh50MJ9QEkNb4Y4+oBZIWfLwB+mqJ9VkTLncBfp3En8Juubz8VHzKJO28GoTOuamB++L2fEjrbB1gK7AHWhR9LfBTbvwHvhuNaHq0gTmVcTZZNSQKIcX/dF95f68P76yyfxGUINZttAjYA1/khrvDrnwALUxFPHPtrGLA6/D2uA4p9EtfVwN/CyzwKtE9BTM8Au4E6QjXJbwG3ArdGHFu/Cse8IRm/RV0JLCISUOncByAiIglQAhARCSglABGRgFICEBEJKCUAEZGAUgIQEQkoJQARkYBSAhARCaj/AcX4apXmhBqIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出参数更新之前的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0.numpy() * plot_x - b0.numpy()) / w1.numpy()\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的参数更新方式其实是繁琐的重复操作，如果我们的参数很多，比如有 100 个，那么我们需要写 100 行来更新参数，为了方便，我们可以写成一个函数来更新，其实 PyTorch 已经为我们封装了一个函数来做这件事，这就是 PyTorch 中的优化器 `torch.optim`\n",
    "\n",
    "使用 `torch.optim` 需要另外一个数据类型，就是 `nn.Parameter`，这个本质上和 Variable 是一样的，只不过 `nn.Parameter` 默认是要求梯度的，而 Variable 默认是不求梯度的\n",
    "\n",
    "使用 `torch.optim.SGD` 可以使用梯度下降法来更新参数，PyTorch 中的优化器有更多的优化算法，在本章后面的课程我们会更加详细的介绍\n",
    "\n",
    "将参数 w 和 b 放到 `torch.optim.SGD` 中之后，说明一下学习率的大小，就可以使用 `optimizer.step()` 来更新参数了，比如下面我们将参数传入优化器，学习率设置为 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用 torch.optim 更新参数\n",
    "from torch import nn\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_regression(x):\n",
    "    return torch.sigmoid(torch.mm(x, w) + b)\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], lr=1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/bushuhui/.virtualenv/dl/lib/python3.5/site-packages/ipykernel_launcher.py:15: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number\n",
      "  from ipykernel import kernelapp as app\n",
      "/home/bushuhui/.virtualenv/dl/lib/python3.5/site-packages/ipykernel_launcher.py:17: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 200, Loss: 0.39010, Acc: 0.00000\n",
      "epoch: 400, Loss: 0.32184, Acc: 0.00000\n",
      "epoch: 600, Loss: 0.28917, Acc: 0.00000\n",
      "epoch: 800, Loss: 0.26983, Acc: 0.00000\n",
      "epoch: 1000, Loss: 0.25700, Acc: 0.00000\n",
      "\n",
      "During Time: 0.248 s\n"
     ]
    }
   ],
   "source": [
    "# 进行 1000 次更新\n",
    "import time\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "    # 前向传播\n",
    "    y_pred = logistic_regression(x_data)\n",
    "    loss = binary_loss(y_pred, y_data) # 计算 loss\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad() # 使用优化器将梯度归 0\n",
    "    loss.backward()\n",
    "    optimizer.step() # 使用优化器来更新参数\n",
    "    # 计算正确率\n",
    "    mask = y_pred.ge(0.5).float()\n",
    "    acc = (mask == y_data).sum().data[0] / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.data[0], acc))\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到使用优化器之后更新参数非常简单，只需要在自动求导之前使用**`optimizer.zero_grad()`** 来归 0 梯度，然后使用 **`optimizer.step()`**来更新参数就可以了，非常简便\n",
    "\n",
    "同时经过了 1000 次更新，loss 也降得比较低了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们画出更新之后的结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f083e823400>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3XlcFdX7B/DPAVncFzAtUSCzEhRQyTVzSUvL8KtpZViaKYqmv+qbZdqCW5nZYinuu5hrX9Pccsl9SdwVy9TcNQhMJUWW+/z+uGCIXJgLM3Nm5j7v1+u+uMvcmecOM8+cOXPmHEFEYIwxZi1usgNgjDGmPk7ujDFmQZzcGWPMgji5M8aYBXFyZ4wxC+LkzhhjFsTJnTHGLIiTO2OMWRAnd8YYs6ASshbs6+tLAQEBshbPGGOmtG/fvr+IqHJh00lL7gEBAYiPj5e1eMYYMyUhxFkl03G1DGOMWRAnd8YYsyBO7owxZkGc3BljzII4uTPGmAVxcmeMMQvi5M4YYxZkuuT+61+/4oNNHyAtM012KIwxZlimS+4rf1uJ0dtGo/6U+th9YbfscBhjzJBMl9wHNxuMNZFrkJqeimYzm+Gdn97BzYybssNijDFDMV1yB4B2D7XD0f5H0ad+H3yx6wuETg7F1rNbZYfFGGOGYcrkDgDlvMphcofJ2PjqRmTZstBidgsMXD0QqempskNjjDHpTJvcc7QObI3D0YcxqOEgTNw7EXUn1cXG0xtlh8UYY1KZPrkDQBnPMhjffjy2vrYVnu6eaDOvDfqu7Itraddkh8YYY1JYIrnneLzG4zjY9yAGNx2M6Qemo86kOljz+xrZYTHGmO4sldwBoKRHSYxtOxa7Xt+Fcl7l8MyCZ9BjeQ+k3EqRHRpjjOnGcsk9R8NqDbE/aj+GNR+GuMNxCI4NxvJfl8sOizHGdGHZ5A4AXiW8MKr1KOztsxdVSldBp0Wd0G1ZNyT9kyQ7NMYY05Slk3uOevfXw94+ezGy1UgsS1iGoNggLDq6CEQkOzTGGNOESyR3APBw98AHT3yA/X33I7BCIF5a9hKeX/w8rqRekR0aY4yprtDkLoSYKYRIFEIcdfC5EEJ8I4Q4KYQ4LISor36Y6qlzXx3sfH0nPmvzGVb/vhpBE4Mw99BceaX4uDggIABwc7P/jYuTEwdjzFKUlNxnA2hXwOftAdTKfkQBmFT8sLRVwq0E3m32Lg71O4TalWujx/Ie6PBdB1y4fkHfQOLigKgo4OxZgMj+NyqKEzxjrNgKTe5EtBVAQe0IOwKYS3a7AVQQQtyvVoBaesT3EWztuRXj243H5jObERwbjGn7pulXih82DLiZp9Ozmzft7zPGWDGoUedeDcD5XK8vZL93DyFElBAiXggRn5RkjBYr7m7uGNRoEA73O4z699dH1I9RaDuvLf64+of2Cz93zrn3mcvhWjtWVLpeUCWiqUQUTkThlStX1nPRhapZqSY2vroRk56dhD0X96DupLqY8MsE2Mim3UJr1HDu/SLiBGFOXGvHikON5H4RQPVcr/2y3zMdN+GGfuH9cKz/MTxe43EMXDMQLWe3xO/Jv2uzwNGjgVKl7n6vVCn7+yrhBKEuPQ+UXGvHioWICn0ACABw1MFnzwJYA0AAaAzgFyXzbNCgARmZzWajWQdmUflPy5P3KG8at2McZWZlqr+g+fOJ/P2JhLD/nT9f1dn7+xPZ0/rdD39/VRdjWbn/PT4+RJ6ed6/HUqVU/5fdIUT+/zshtFleYTTeVJlCAOJJSd4udALgOwCXAWTAXp/+OoB+APplfy4ATARwCsARAOFKFmz05J7j4vWL9NyC5wgxoEbTGtGxxGOyQ3KK0RKEmcyfb0/e+a0/PQ6Uzh6YtUy++a0LLQ9szDHVkrtWD7MkdyJ7KT7ucBxV+qwSeY70pE+2fkIZWRnOz0hC0YdL7kXnaN3pdaB0JqFqnXx5OzIOTu4auHLjCnVZ3IUQA6o/pT4dunJI+ZclFX24xFV0js569ExwSssDWidfPgM0Dk7uGlpybAnd9/l9VGJECfr454/pdubtwr8ksejDdaVFo6TkbpQDpdbJl0vuxqE0ubtM3zJq6hLUBQn9E/Bi8IsYvmU4wqeGY9+lfQV/SWKb9shI4MwZwGaz/42M1HyRlpBfYyYPD8DHBxAC8PcHpk41xvrUulWtDg27mMo4uReRTykfzO88HyteWoHkW8loNL0Rhm4cirTMtPy/oFObdqaeyEh78vb3/zeZz5oF/PWX8Q6UWiff/NaFUQ5szAElxXstHmaulsnr6q2r1Gt5L0IM6NEJj9LOczvvnYgrv5nGuPrNNYCrZfRTwbsCZnScgXXd1+Fmxk00m9kMb697Gzczct2BwkUfphJHN1Jx9RvLTdgPBPoLDw+n+Ph4KcvW0o3bN/DehvcwKX4SalasiRkRM9AioIXssJhF5NxxnPvO1VKluJzgSoQQ+4govLDpuOSusrJeZRH7bCx+7vEzCISWc1piwKoBuHH7huzQmAUYsUsC7rvImDi5a6RlQEsc7ncYbzZ6E5PiJ6HupLpYf2q97LCYyRmtI1Huu8i4OLlrqLRnaXzV7its77Ud3iW88dT8p9BnRR9cS7smOzRmUkZrdGXEMwlmx8ldB02rN8WBvgfwXrP3MPPgTATHBmPViVWyw2ImZLT25kY7k2D/4uSuk5IeJTGmzRjsfn03KpasiA7fdcAr/3sFKbcKGuSKsbup2ehKjbrySpWce5/ph1vLSHA78zY+2fYJPtn+CXxK+iD22Vh0rt1ZdljMhajV6sbXF0hOvvd9Hx/7zV5MfdxaxsC8SnhheKvh2NtnLx4o+wCeX/w8Xlz6IhL/SZQdGrO4nNJ69+7q1JWnODjxdPQ+0w8nd4nCqoZhT+89GN16NJb/uhzBscFYeHQhZJ1NmQU3vSua3C1bHHG2rtxoF3jZvzi5S+bh7oGhzYdif9R+PFjxQXRb1g2dFnXCpRuXZIdmSNz0rujya9mSl7NJ2WgXeAvjUgUDJX0UaPGwUt8yasnMyqRxO8aR9yhvqjCmAs06MItsNpvssBTTo28T7nq26Arrn76oXR2ZpU8bq3TvBIV9y/AFVQM6kXwCvVf0xrZz29DuoXaY0mEKapQ39nmuXrfFu7nZd8u8hLD3qcIcCwhwXCXj728vbVu5CwNHv9/f394Xj1nwBVUTe9jnYWzuuRnftv8W285uQ53YOpgSP8XQdfF63czCdbxF56gKJTra/vyVV6xXVZG7GsbRgc2ybfKVFO+1eHC1jDKnU05T6zmtCTGg1nNa06mUU6rOX61Tar2GYbPKqbUsef/f0dHWXZ+yBzjXCniYPeuw2Ww0NX4qlf2kLJUaXYrG7x5PWbasYs9XzUSpZ124Wep4zcDK1zBkDJOox7bJyd2Czv19jtrPb0+IATWb0Yx+++u3Ys1PzR3bSiVqVzp4WHng64IuIGvxv9VrH+DkblE2m41mH5hNFcZUIO9R3jR2+1jKzMos0rzU3rGtkBStdJAqzPz5RO7urldy1+q36bU8Tu4Wd+n6Jer4XUdCDKjhtIZ09M+jTs/DyqfkRSV7neh1gCyoPtoqBzO9D9R6nQVxcncBNpuNFh5ZSL5jfclzpCeN2jKK0jPTFX/flUqpSsmsptD6/5H7wOGoxO7ubq3/v55nk1xy5+SuusTURHpxyYuEGFC9yfXowOUDir9rhaoUNcksuWu5bKUtR6xQ1y4L17lzctfM9wnfU5XPq1CJESXow00fUlpGmuyQTEfm2YyWZw1KWo64epWcGozUWoZvYrKQTrU7IWFAAl6u+zJGbh2JBlMbYO/FvbLDMhU1+0t3lpY3aCm5UcfIfcKYRWSk/W5Xm83+V+Ydv5zcLaZSyUqY8585WPXyKvyd9jcaz2iM99a/h1sZt2SHZhqydlAtO+FydIBwd9f/IMb0wcndop6p9QyO9T+GXmG9MHbnWIRNCcOOcztkh8UKoOVZg6MDx5w5xihlMvVxcrew8t7lMS1iGta/sh63M2+j+azmeHPtm/gn/R/ZoTEHtDprkFndxOTgXiFdRGp6KoZsGIKJeyfiwYoPYvpz09EqsJXssBhjTuJeIdldyniWwYRnJmBzj80QEGg9tzWif4zG9dvXZYfGGNMAJ3cX0yKgBQ5HH8bbjd/GlH1TUCe2DtadXCc7LMaYyji5u6BSHqXwxdNfYEevHSjtWRrt4tqh1w+9cPXWVdmhmZ5LDePGDI2TuwtrUr0JDvQ9gPcffx9zD81FcGwwVv62UnZYpsXjuzIj4eTu4rxLeOOTJz/Bnt574FvKFxELIxD5fSSSbybLDs109BqNijElFCV3IUQ7IcRvQoiTQogh+XxeQwjxsxDigBDisBDiGfVDZVpq8EADxEfFI6ZFDBYfW4yg2CAsTVgqOyxTcXQXqGWHcWOGVmhyF0K4A5gIoD2AIADdhBBBeSb7AMBiIqoH4CUAsWoHyjSSq5LYs+bD+PjiQ9gXtQ9+5fzQdUlXdF3SFX+m/ik7SlPg8V2ZkSgpuTcEcJKIThNROoCFADrmmYYAlMt+Xh7AJfVCZJpxUEkcsuEI9vTeg0+f/BQrfluB4NhgLDiyALLuiTALLbsPKAhfxGX5UZLcqwE4n+v1hez3cosB0F0IcQHAagADVYnO1ei9lxZQSVzCrQSGPD4EB/seRC2fWoj8PhIdF3bExesXtY3J5EqW/Pe5j4/2d4HyRVzmiFoXVLsBmE1EfgCeATBPCHHPvIUQUUKIeCFEfFJSkkqLtggZe6mCSuLalWtj+2vb8eVTX2LD6Q0Ijg3GrAOzuBSfR86/LznXdehbOvTVxhdxmSNKkvtFANVzvfbLfi+31wEsBgAi2gXAG4Bv3hkR0VQiCiei8MqVKxctYquSsZcqrCR2d3PHW03ewuHowwitGopeK3qhXVw7nP37rHaxmYysJMsXcbVhhaouJcl9L4BaQohAIYQn7BdMV+SZ5hyAJwFACFEb9uTORXNnyNhLnawkfqjSQ/i5x8+Y+MxE7Di3A3Um1cHk+MmwkU27GE1CVpLli7jqs0xVl5IRPWCvajkB4BSAYdnvjQAQkf08CMAOAIcAHATwVGHz5JGY8pA1vlsRh4754+of1HZuW0IMqOXslnQy+aSmYRqdzH+f0cfBNdtQjrIHSS8MeJg9k1F7L9Vhj7LZbDR933Qq92k5KjW6FH2962vKzMpUfTlaUXMVyUyyRk6eZjj45CVzkHQlOLmbkVp7qc571Plr5+mZuGcIMaCmM5rS8aTjToUqIzFpsYqMnGRlMXopOD9Gj5mTu5monRUkbJ02m43mHZpHFcdUJK+RXjRm2xjKyMoo8DsyS3VG34GNytlN1eil4PwY/WyDk7tZaLElSdyjLt+4TJ0XdSbEgMKnhtORP484nFZmgjVj0pGtKJuqWQ+iRj4L4+RuFlps/ZL3KJvNRouPLqbKYyuTxwgPGrF5BKVnpt8zncwEa9akI5Ojdebu7jgJGr0UbEac3M1CiwxnkD0qMTWRui3tRogBhU4KpX2X9t31ucwEa5BVZCqONtXC1qGRS8FmxMndLLTKcAbao5YfX05Vx1Ul9+HuNHTDUErLSLsToswEa6BVZAqONlU++9EXJ3ezKCzDWSQDpdxMoR7/60GIAQVNDKLd53cTkXF/nlHjkim/TZWvW+iPk7uZOMoksou2Glh9YjX5felHbsPd6J1179DN9JuyQ7pHfqs9p0rC1RN97k3V3Z1L7jJwcrcCi171u5Z2jfqu7EuIAdX6phZtO7tNdkh3Kaz6weTHV9VYsOyhOTXOCJUmdx5mz8gs2itUOa9ymNxhMja+uhGZtkw8MesJDFozCKnpqbJDA1D46uVeF+0iI+1dGvv7A0LY/2rdxbGZ6d1nDSd3I1OrVyiDdnHXOrA1DkcfxhsN38C3v3yLkEkh2PTHJtlhKVq9Jj++qiYyEjhzBrDZ7H85sTumd8+hnNyNTI2hfQzexV0ZzzL4pv032NpzK0q4lcCTc59E35V9cf32dWkx5bfa8+JeF5mz9D4R5+RuZGqc95pkNIfm/s1xsN9BvNPkHUw/MB3BscFYe3KtlFhyr3bAvupz02PoPJkMeqJnerp3z6ykYl6LB19Q1YkJ77PffX43BU0MIsSAei7vSSk3U6TGk/ciWHS0dZtJ8kVS7ai1bsGtZRgRmbbFzey56VTuvhQCssitwjl6a+xe2SERkXmTn9JWGibdXExzX4KerWU4uVudCbNRvjfLeKRS40HfUGJqotTYzJj8nNkETHiiZ8ZNvFg4ubN/maVYk81hO/PyZ6jy2Mq06OgistlsUmIzY/Jz5oBUUBt/o246RjngFrSbqbkLcnJnpuU4gdoofGo4IQbUaWEnunzjsu6xGSWROMOZA1JhXQwYsURshANuQWcPap9ZcHJnplVQAs3IyqDPtn9GXiO9qOKYijT34FxdS/FmrAJw9oCUU8osqARvJEY44BYUg9rxcXJnpqUkgf6a9Cs1ndGUEAN6Nu5ZOn/tvK7xmaiWq8gHJCOUiJUwwgG3oHWl9nrk5M5MTUkCzczKpK93fU0lR5Wkcp+Wo+n7pkurize6ohyQjFAiVkr2AZdL7pzczUn2nlOIk8knqeXsloQYUJu5beiPq3/IDkkxI69aI5SIzYLr3Dm5m49J9vAsWxZN2juJynxShsp8UoYm/jKRsmxZssMqkBlWrZEPPkZjtNYywj6t/sLDwyk+Pl7KspkTAgLs/dHk5e9v7ynKYM5dO4c+K/vgp1M/4Qn/JzAjYgYeqvSQ7LDyZbJVaxpxcfbeNc6ds9/aP3q0tTo0E0LsI6LwwqbjvmVYwUzW7XCN8jWwNnItZkbMxKErhxAyKQRf7voSWbYs2aHdw2Sr1hTU7CfP7H3scHK3Ei22Rt17Oyqa3D89MFDAM+E1JAxIQJsH2+C/P/0Xj896HMeTjssO8y4mWbWmolY/eQbvTFUZJXU3Wjy4zl1lWlXgmqBiuKAQbTYbxR2Oo0qfVSKvkV706bZPKSMrQ3bIRGSKVWs6ajU7NHJLIfAFVRej5dZo8KtqSn76lRtXqMviLoQYUIMpDejQlUOywr2LwVet6ai1Gxi5jb/S5M4XVK3Czc2+/eUlhH2YHAtz5qcvTViK/qv642raVQxrPgxDmw+Fp7unPoEyzeVUp+SumilVyvlhEIx8sZsvqLoaq1XgOnH9wJmf3iWoCxIGJOCF4BcwfMtwhE8Nx75L+1QJmcmn1riuagyCJp2S4r0WD66WUZmVKnCd/C1F/ek//PoD3T/ufnIf7k7vb3ifbmXc0uDHMLMyapUZuM7dBRl1a3RWESpOi/rTr966Sq8tf40QA3p0wqO089xOFX4AY9pRmty5zp0Zj4TrB+tOrkOflX1w4foFvNX4LYxsPRKlPAoZJZsxCbjOnZmXhOsHTz/0NI72P4p+4f3w5e4vETIpBFvObNFseYxpjZM7Mx5JV7PKeZVD7LOx2PTqJhAILee0xIBVA5Canqr6ssx+9yNzjoz/Nyd3Zjw5TR58fP59r2RJ3RbfKrAVDvc7jP9r9H+YFD8JdWLrYMPpDarN3xJ3PzLFZP2/ObkbCRfn7nbr1r/Pk5N1zYClPUvj63ZfY9tr2+BVwgtt57VFnxV9cC3tWrHnrdYt8swcZP2/ObkbhasU55QewAySAZvVaIaDfQ/ivWbvYebBmQiODcaqE6uKNU/uMMy1yPp/K0ruQoh2QojfhBAnhRBDHEzzghAiQQhxTAixQN0wXYBBkpmmnDmAqbhHFPeEqKRHSYxpMwa7X9+NiiUrosN3HfDq/15Fyq0Up2MBrHe/GSuYtP93YW0lAbgDOAXgQQCeAA4BCMozTS0ABwBUzH59X2Hz5XbueRi5Mwu1ONN+XaVOQtS+tystI40+2vQRlRhRgqp8XoW+T/je6XlY6X4zVjjDjsQEoAmAdblevw/g/TzTjAXQW8kCcx6c3PMwcjd0anHmAKbSHqHVaj1w+QDVm1yPEAN6YckLlJia6NT3rXK/GVNGxkhMSpJ7FwDTc71+BcCEPNMsz07wOwDsBtDOwbyiAMQDiK9Ro0bRf50VuUJxztlMq8IeoeUJUXpmOo3aMoo8R3qS71hf+u7IdzxAN9Oc0uSu1gXVEtlVMy0BdAMwTQhRIe9ERDSViMKJKLxy5coqLdoi1OrxyMicbb8eGWnvgs9ms/8twrrQsr7Tw90Dw54Yhv1R+/FgxQfRbVk3dFrUCZdvXC7+zBkrJiXJ/SKA6rle+2W/l9sFACuIKIOI/gBwAvZkz5yhQjIzNAkHMD3uhwq+Lxg7e+3E520/x7pT6xAUG4Q5B+fknK0yJoWS5L4XQC0hRKAQwhPASwBW5JlmOeyldgghfAE8DOC0inEyq9D5AKbX8cTdzR3vNH0Hh/odQp376qDnDz3x7IJncf7aeXUX5AL4dg91FJrciSgTwBsA1gE4DmAxER0TQowQQkRkT7YOQLIQIgHAzwAGE1GyVkEzSUy61+l5PHnY52Fs6bkF37T7BlvObkFwbDCm7pvKpfh85Lc5ucrtHnrgXiGZMmoNceNCTl89jT4r+2DTH5vQOrA1pj83HYEVA2WHZQiONqeSJe03I+dlhBGQjEJpr5Cc3JkyRh53zMCICNP2T8M7P72DLMrCmCfHYEDDAXATrn1zuKPNyREXGC1SMe7yl6mL75kvEiEEohpE4Vj/Y3jC/wkMWjsILWa3wInkE7JDk8rZzYbv3nUeJ3emDN8zXyzVy1fH6pdXY3bH2TiaeBShk0Mxbuc4ZNmyZIcmhaPNxsfHAmOXGgQnd6aMJUYMlksIgR5hPZDQPwFP13wag9cPRtOZTZGQlCA7NN052pzGj7f+7R66UXKnkxYP7n7AhPieedXYbDb67sh35POZD3mO9KTRW0dTema67LB0xZtT0YDHUGXM+BL/ScTANQOx+Nhi1KtaD7M6zkJo1VDZYTED4wuqjJnAfaXvw6Iui7DshWW4dOMSwqeF4+OfP0Z6Vrrs0JjJcXJnzAA61+6MY/2P4aU6L2HE1hGo2ftD3O9322z3izED4eTOzEvGHbMaLtOnlA/mdZqH/5bZh4txH+PKRS++S5MVGSd3pg+1k6KM+9R1WubSb+uDMu5uSmK1QbmY9viCKtOeFl0XyLhjVqdlurnZjx33EITUtJso7VlatWUx8+ELqsw4tBgfVsYdszot0+F9YeXOImRyCDaf2azq8pg1cXJn2tMiKcq4Y7ZSJV2W6egGn2ExNyEg0GpOK/Rf1R83bt9QdbnMWji5M+1pkYj1vmM2Lg64fv3e9z09VV+moz7oR70ZhMPRh/F247cxOX4y6kyqg59O/aTqspmFKLnTSYuH5e5Q5dvtHHM0Pmx0dPHWmZ7r3NH4rz4+2i2zADvP7aRHJzxKiAH1Wt6Lrt66KiUOpj+oNUC2Vg9LJXdXGNy6uPIm4uhoc60zLUfaLqJbGbdoyPoh5D7cnR744gFa+dtKabEw/ShN7txaRg3c17nzHK0zHx/gr790D6dQBv4fx1+KR68feuFI4hFE1o3E+Hbj4VPKR2pMTDvcWkZP3Ne58xytm+RkY96tY+BeMcMfCEd8VDxiWsRg0bFFCIoNwrKEZbLDcsikozWaDid3NXBf584raN0Y8W4dvUbaLiJPd0983PJj7IvaB79yfuiypAu6LumKxH8SZYd2Fx4jVT+c3NVg4FKdYRW0box6xqPnSNtFFFIlBHt678GnT36KFb+tQNDEICw4sgDOVL9qWbLW4pYH5oCSinktHpa6oEqkfssNV2h94+OT/0VKd3dr/26dJCQmUOPpjQkxoOcWPEcXrl0o9Dtatw0w4HVp0wG3ljExV2l9k9/vzPuw4u/Oj0YH88ysTPpi5xfkPcqbyn9anmbun0nz59scLspRi09/f1XC0Xz+roCTu5m50h6QO6m5u7vO785Nh4P5ib9O0BOzniB07kZunrccLkrrkrWrlFu0xMndzFzx3HX+fMeldyv/biLdDuZZtiyqWPVagYvSIxSz1TgaLV6lyZ0vqBqRq7W+yWlC4YhVf3cOnZrSugk3/P1nuQIXpUfbAD2uS6t1UdjUrXuUHAG0eHDJvQCudu7qqLho9d+dQ8dqOEeLqlj1GmVmZRKR8yVVo5Vs1dx9jFhDCq6WMTmj7TFaclQNBVj7d+fQ8WCe36LcPG8ROnejpjOa0q9Jvxo1dMXUTMhGrCFVmty5WkZNxTkXzPtdwPBtqlXjqNrF39/avzuHjjdI5beouTO9MPej9jiedByhk0MxdsdYZNoyFc3PiO3W1azlMnUNqZIjgBYPy5Xci1OEMWLxR0+u/vsN4tL1S/Sfhf8hxIAem/oYHfnzSKHfUaNkq/ZJqpoldyNumuBqGZ0p3aLy25KNWLGnN1eqhjIwm81GC48sJN+xvuQxwoNGbhlJ6ZnpDqcv7qarRfJ0dPuEj0/R5mu0TZOTu96UFGEcbcmO6put3gSQGS9zZEtMTaQXl7xIiAGFTQ6j/Zf25zudouRcwG/Uqlwzf37+N0DLLnWrgZO73pRspY6mcdWbd4xMj6RrxHP+PP53/H9UdVxVch/uTsM2DqO0jLR7pilwVRXyG7W8YGnVE2JO7npTsqMW1CrE4Du5S9Er6Zok+yTfTKYe/+tBiAEFTQyiPRf2KP9yIb9Ry1VgxJYuauDkLkNhpb2CtmSDnp67JL2SrozsU4ztbNWJVeT3pR+5DXejwT8NppvpNwv/UiG/UcvjqEmOnU7j5G5EJjgNZ6RPBysF3bilVfZRYfv7+9bf1GdFH0IM6OFvH6btZ7cX/AUFGVarco1VdzdO7kbFJXTj07LIV1hPmFpmHxV/1/pT6yng6wASMYIGrR5EqbdT859w/nwiT8+7l+fpqdt2b8XdjZM7Y0Ulo64gd/WcVlQ+I7lx+wYNXD2QEAMK/DqQNp7eeO9E8+cTeXjcvTwPD2tkWUmUJndFd6gKIdoJIX4TQpwUQgwpYLrnhRAkhCh08FbGDEvLO0Yd3SYphPZ3Iqt8u2UZzzL4pv032NpzK9zd3PHk3CfR78d+uH77+r8TDRsGZGTc/cWMDB56SQeFJnchhDuAiQDaAwgC0E0IEZTPdGUB/B+APWqCXukgAAATfUlEQVQHyZjTitstoFZdF8q8n12jLh+b+zfHoX6H8N8m/8W0/dNQJ7YO1p5ca/+QB4+XpoSCaRoCOElEpwFACLEQQEcACXmmGwngMwCDixpMRkYGLly4gLS0tKLOgjnB29sbfn5+8PDwkB2KunL6ac3p9CSnn1ZAfl81o0ffHRug33i7Ob992DB7cq1Rw75cFdZJKY9SGPfUOHQJ6oJeP/RC+7j26BnWE9P9qsH9/IV7v2C0zlni4jRZL1IVVm8DoAuA6blevwJgQp5p6gNYlv18M4DwwuabX5376dOnKSkpiWw2mwY1VSw3m81GSUlJdPr0admhqM/obeCseJUvl7SMNBq6YSi5D3en6G7lKcPbS7+LxkVhsmY10KtXSCGEG4AvAfxXwbRRQoh4IUR8UlLSPZ+npaXBx8cHQojihsUKIYSAj4+PNc+SjF4VoMdoFRJ5lfDC6CdH45c+v2DHE/549ZnbSPItBdK4x8sic7ZrS7VGAtGYkuR+EUD1XK/9st/LURZAHQCbhRBnADQGsCK/i6pENJWIwokovHLlyvkujBO7fiy7rk3dT6t11L+/Pvb22Yvag0ag2qAMVPnMF0tWfQ56+WXZod3NmcKAiYZmUpLc9wKoJYQIFEJ4AngJwIqcD4noGhH5ElEAEQUA2A0ggojiNYnYQM6cOYMFCxbceX3w4EGsXr36zusVK1ZgzJgxqiyrZ8+eWLp0KQCgd+/eSEjIe8mD3aHHWHFMEU93T3zY4kPsi9oH/wr+eGHpC+iypAuupF6RHdq/nCkMGLEDewcKTe5ElAngDQDrABwHsJiIjgkhRgghIrQO0MgKS+4REREYMsRhy9Eimz59OoKC7mmwxHLoOPgFU6ZulbrY9foujHlyDFadWIXg2GDMPzw/55qdXM4UBoxe5Zebkop5LR75XVBNSEgo9sWG4pozZw7VrVuXQkJCqHv37kRE1KNHD1qyZMmdaUqXLk1ERI0aNaJy5cpRaGgojRkzhqpXr06+vr4UGhpKCxcupFmzZtGAAQPuzGPgwIHUpEkTCgwMvDO/rKwsio6OpkceeYTatGlD7du3v2tZOXLH0KJFC9q7d++dWIYOHUohISHUqFEjunLlChERJSYmUufOnSk8PJzCw8Np+/b8bxM3wjpnruV40nFqMr0JIQbUYUEHunDtguyQlF/kNsDFeii8oKqkKaQUb659EwevHFR1nmFVw/B1u68dfn7s2DGMGjUKO3fuhK+vL1JSUgqc35gxYzBu3Dj8+OOPAIAqVaogPj4eEyZMAADMnj37rukvX76M7du349dff0VERAS6dOmC77//HmfOnEFCQgISExNRu3Zt9OrVS/Fv+ueff9C4cWOMHj0a7777LqZNm4YPPvgA//d//4e33noLjz/+OM6dO4enn34ax48fVzxfxrTyqO+j2PbaNnz7y7cYunEogmKD8OVTX6JXvV7yrgNFRio7s5PZlNVJPIZqLps2bULXrl3h6+sLAKhUqZKq8//Pf/4DNzc3BAUF4c8//wQAbN++HV27doWbmxuqVq2KVq1aOTVPT09PdOjQAQDQoEEDnDlzBgCwYcMGvPHGGwgLC0NERASuX7+O1NRUVX+P4ZikFQMD3N3c8WbjN3Ek+gjqVa2H3it74+n5T+Ps32dlh1YwE1X5GbbkXlAJW28lSpSAzWYDANhsNqSnpxdpPl5eXneek0p1jR4eHndKO+7u7sjMtA9sbLPZsHv3bnh7e6uyHMMz8o1LzKGalWpiU49NmBI/Be9ueBd1JtXBZ20+Q7/wfnATBi17Ki3lS2bQtSdH69atsWTJEiQnJwPAnWqZgIAA7Nu3D4C9BUxGdl8ZZcuWxY0bN+58P+9rJZo1a4Zly5bBZrPhzz//xObNm1X4JcBTTz2Fb7/99s7rgwfVreIyHBO1YmB3cxNuiH4sGkejj6KJXxMMWD0Aree0xsmUk7JDMzVO7rkEBwdj2LBhaNGiBUJDQ/H2228DAPr06YMtW7YgNDQUu3btQunSpQEAISEhcHd3R2hoKL766iu0atUKCQkJCAsLw6JFixQt8/nnn4efnx+CgoLQvXt31K9fH+XLly/2b/nmm28QHx+PkJAQBAUFYfLkycWep6GZqRUDy5d/BX+s674OMyJm4OCVgwiZFIKvdn2FLFuW7NBMSahVPeCs8PBwio+/uyn88ePHUbt2bSnxyJSamooyZcogOTkZDRs2xI4dO1C1alVdlm2ZdR4QYK+Kycvf334XKDOVi9cvou+PfbHq91Vo4tcEMzvOxKO+j8oOyxCEEPuIqNCed7nkbgAdOnRAWFgYmjdvjg8//FC3xG4pfOOSpVQrVw0ru63E/E7z8VvybwibHIYx28cg05YpOzTTMOwFVVeiVj27S9Owx0MmhxACkSGRaPNgGwxYPQDvb3wfSxOWYmbHmQipEiI7PMPjkjuzDot3yOWqqpSpgqUvLMXiLotx7to5hE8Nx/DNw5GeVbRWa66CkztjzBS6BndFwoAEdA3uipgtMXhs2mPYf3m/7LAMi5M7Y8w0fEv5Iq5zHH546Qck/ZOEhtMaYujGoUjLtGDX1cXEyZ0xZjoRj0TgWP9jeDX0VXy6/VPUn1Ifuy/slh2WoXByZ4yZUsWSFTGz40ysjVyL1PRUNJvZDO/89A5uZtws/MsuwNzJXWJfInPmzEGtWrVQq1YtzJkzR7flMlZsFuuD5+mHnsbR/kcRVT8KX+z6AqGTQ7H17FbZYcmnpOtILR7F7vJX4riHycnJFBgYSMnJyZSSkkKBgYGUkpKi+XK1wF3+uhiTjRfqrI2nN1Lg14GEGNCAVQPoxu0bskNSHfQaQ1UaDfoS2bt3L0JCQpCWloZ//vkHwcHBOHr06D3TrVu3Dm3btkWlSpVQsWJFtG3bFmvXri3ychnTjcX74Gkd2BpHoo9gUMNBiN0bi7qT6mLD6Q2yw5LCvMldg75EHnvsMUREROCDDz7Au+++i+7du6NOnTr3THfx4kVUr/7vsLJ+fn64ePHiPdMxZjgu0AdPac/SGN9+PLa9tg2e7p5oO68tolZG4VraNdmh6cq8yV2jQZA/+ugjrF+/HvHx8Xj33XeLNS/GDMeFBg9vVqMZDvY9iMFNB2PGgRmoM6kOVv++uvAvWoR5k7tGfYkkJycjNTUVN27cQFpa/m1nq1WrhvPnz995feHCBVSrVq1Yy2VMFy7WB09Jj5IY23Ysdr2+C+W8yuHZBc+ix/IeSLlV8ChrlqCkYl6LhypjqCod99AJzz33HMXFxdGoUaPujH+aV3JyMgUEBFBKSgqlpKRQQEAAJScnF3vZMvAFVRekwX5jBmkZafTBxg/Ifbg7Vfm8Cn2f8L3skIoECi+omju5q2zOnDnUuXNnIiLKzMykhg0b0saNG/OddsaMGVSzZk2qWbMmzZw5U88wVSV7nTOmt/2X9lPY5DBCDOiFJS9QYmqi7JCcojS5c3/uLo7XOXNFGVkZGLtjLIZvGY7y3uXxbftv8WLwi/IG6HYC9+fOGGMOeLh7YNgTw3Cg7wEEVghEt2Xd0HlxZ1y+cVl2aKrh5F6AI0eOICws7K5Ho0aNZIfFGFNJ8H3B2Pn6ToxtMxZrfl+D4NhgzD00V7UB7GXiwToKULduXesPLM2YiyvhVgKDmw1GxCMReH3F6+ixvAcWHl2IKR2moHr56oXPwKC45M4YYwAe8X0EW1/bivHtxmPL2S0Ijg3GtH3TTFuK5+TOGGPZ3IQbBjUahCPRRxD+QDiifoxC23lt8cfVP2SH5jRO7owxlseDFR/Ehlc3YPKzk/HLxV9Qd1JdTPhlAmxkkx2aYpzcGWMsH27CDX3D++Jo/6N4vMbjGLhmIFrMboHfk3+XHZoipk7uMrulbteuHSpUqIAOHTrot1DGmO5qlK+BNZFrMKvjLBxNPIqQySEYt3McsmxZskMrkGmTe1wcEBUFnD1r75T67Fn7a70S/ODBgzFv3jx9FsYYk0oIgZ5hPXGs/zE8VfMpDF4/GE1nNkVCUoLs0BwybXLXoltqpf25A8CTTz6JsmXLFn1hjDHTeaDsA1j+4nIs6LwAp1JOod6Uevhk2yfIyMqQHdo9TJvcteiWWml/7owx1yWEQLe63ZAwIAEdH+mIYZuGodH0Rjh05ZDs0O5i2uSuVbfU3J87Y0yJ+0rfh8VdF2Np16W4eOMiwqeF4+OfP0Z6Vrrs0ACYOLlr1S21kv7cGWMsx/NBzyOhfwK61emGEVtHoMHUBoi/FF/4FzVm2uQeGQlMnQr4+wNC2P9OnWp/vzj69u2LkSNHIjIyEu+99546wTLGLM2nlA/mdpqLld1WIuVWChpNb4QhG4YgLVNeAdG0yR2wJ/IzZwCbzf63uIl97ty58PDwwMsvv4whQ4Zg79692LRpU77TNm/eHF27dsXGjRvh5+eHdevWFW/hjDHT6/BwBxzrfwyvhb2Gz3Z8hrDJYdh5fqeUWBT15y6EaAdgPAB3ANOJaEyez98G0BtAJoAkAL2I6GxB8+T+3I2B1zlj2lh/aj16r+yN89fO46MWHyGmZYwq81WtP3chhDuAiQDaAwgC0E0IEZRnsgMAwokoBMBSAGOdD5kxxqyjbc22OBp9FNHh0ahZsabuy1fS5W9DACeJ6DQACCEWAugI4E7rfSL6Odf0uwF0VzNIWY4cOYJXXnnlrve8vLywZ88eSRExxsykrFdZTHx2opRlK0nu1QCcz/X6AoCCRqx4HcCa4gRlFNyfO2PMrFQdrEMI0R1AOIAWDj6PAhAFADUcNEgnIlOMY2gFZu2nmjFWOCWtZS4CyD0ciV/2e3cRQrQBMAxABBHdzm9GRDSViMKJKLxy5cr3fO7t7Y3k5GROOjogIiQnJ8Pb21t2KIwxDSgpue8FUEsIEQh7Un8JwMu5JxBC1AMwBUA7IkosajB+fn64cOECkpKSijoL5gRvb2/4+fnJDoMxpoFCkzsRZQoh3gCwDvamkDOJ6JgQYgSAeCJaAeBzAGUALMmuUjlHRBHOBuPh4YHAwEBnv8YYYywPRXXuRLQawOo8732U63kbleNijDFWDKa+Q5Uxxlj+OLkzxpgFKep+QJMFC5EEoMAuCgrgC+AvFcNRC8flHI7LeUaNjeNyTnHi8ieie5sb5iEtuReHECJeSd8KeuO4nMNxOc+osXFcztEjLq6WYYwxC+LkzhhjFmTW5D5VdgAOcFzO4bicZ9TYOC7naB6XKevcGWOMFcysJXfGGGMFMHRyF0K0E0L8JoQ4KYQYks/nbwshEoQQh4UQG4UQ/gaJq58Q4ogQ4qAQYns+g5tIiSvXdM8LIUgIoUsrAgXrq6cQIil7fR0UQvQ2QlzZ07yQvY0dE0IsMEJcQoivcq2rE0KIvw0SVw0hxM9CiAPZ++QzBonLPzs/HBZCbBZC6NKhkhBiphAiUQhx1MHnQgjxTXbch4UQ9VUNgIgM+YC9H5tTAB4E4AngEICgPNO0AlAq+3k0gEUGiatcrucRANYaIa7s6coC2Ar7oCrhRogLQE8AEwy4fdWCfZSxitmv7zNCXHmmHwh7f0/S44K9Hjk6+3kQgDMGiWsJgB7Zz1sDmKfTNvYEgPoAjjr4/BnYx74QABoD2KPm8o1ccr8zAhQRpQPIGQHqDiL6mYhuZr/cDXt3xEaI63qul6UB6HFho9C4so0E8BkAvYZlVxqX3pTE1QfARCK6CgBUjB5PVY4rt24AvjNIXASgXPbz8gAuGSSuIAA5I93/nM/nmiCirQBSCpikI4C5ZLcbQAUhxP1qLd/IyT2/EaCqFTC9XiNAKYpLCDFACHEK9vFkBxkhruzTvupEtEqHeBTHle357FPTpUKI6vl8LiOuhwE8LITYIYTYnT1QvBHiAmCvbgAQiH8Tl+y4YgB0F0JcgL2jwYEGiesQgM7ZzzsBKCuE8NEhtsI4m+OcYuTkrliuEaA+lx1LDiKaSEQ1AbwH4APZ8Qgh3AB8CeC/smPJx0oAAWQfYH09gDmS48lRAvaqmZawl5CnCSEqSI3obi8BWEpEWbIDydYNwGwi8oO9ymFe9nYn2zsAWgghDsA+StxFAEZZZ5oxwop3RLURoGTElctCAP/RNCK7wuIqC6AOgM1CiDOw1/Gt0OGiaqHri4iSc/3vpgNooHFMiuKCvSS1gogyiOgPACdgT/ay48rxEvSpkgGUxfU6gMUAQES7AHjD3oeK1LiI6BIRdSaierDnChCRLhehC+FsLnGOHhcWingxogSA07CfduZcKAnOM0092C+m1DJYXLVyPX8O9kFNpMeVZ/rN0OeCqpL1dX+u550A7DZIXO0AzMl+7gv7KbSP7Liyp3sUwBlk36tikPW1BkDP7Oe1Ya9z1zQ+hXH5AnDLfj4awAg91ln28gLg+ILqs7j7guovqi5brx9ZxBXzDOylpVMAhmW/NwL2UjoAbADwJ4CD2Y8VBolrPIBj2TH9XFCS1TOuPNPqktwVrq9Ps9fXoez19ahB4hKwV2UlADgC4CUjxJX9OgbAGD3icWJ9BQHYkf1/PAjgKYPE1QXA79nTTAfgpVNc3wG4DCAD9rPA1wH0A9Av1/Y1MTvuI2rvj3yHKmOMWZCR69wZY4wVESd3xhizIE7ujDFmQZzcGWPMgji5M8aYBXFyZ4wxC+LkzhhjFsTJnTHGLOj/AaaF1CTFqYFCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出更新之后的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0.numpy() * plot_x - b0.numpy()) / w1.numpy()\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到更新之后模型已经能够基本将这两类点分开了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面我们使用了自己写的 loss，其实 PyTorch 已经为我们写好了一些常见的 loss，比如线性回归里面的 loss 是 `nn.MSE()`，而 Logistic 回归的二分类 loss 在 PyTorch 中是 `nn.BCEWithLogitsLoss()`，关于更多的 loss，可以查看[文档](http://pytorch.org/docs/0.3.0/nn.html#loss-functions)\n",
    "\n",
    "PyTorch 为我们实现的 loss 函数有两个好处，第一是方便我们使用，不需要重复造轮子，第二就是其实现是在底层 C++ 语言上的，所以速度上和稳定性上都要比我们自己实现的要好\n",
    "\n",
    "另外，PyTorch 出于稳定性考虑，将模型的 Sigmoid 操作和最后的 loss 都合在了 `nn.BCEWithLogitsLoss()`，所以我们使用 PyTorch 自带的 loss 就不需要再加上 Sigmoid 操作了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 使用自带的loss\n",
    "criterion = nn.BCEWithLogitsLoss() # 将 sigmoid 和 loss 写在一层，有更快的速度、更好的稳定性\n",
    "\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_reg(x):\n",
    "    return torch.mm(x, w) + b\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], 1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.6363\n",
      "[torch.FloatTensor of size 1]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_reg(x_data)\n",
    "loss = criterion(y_pred, y_data)\n",
    "print(loss.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 200, Loss: 0.39538, Acc: 0.88000\n",
      "epoch: 400, Loss: 0.32407, Acc: 0.87000\n",
      "epoch: 600, Loss: 0.29039, Acc: 0.87000\n",
      "epoch: 800, Loss: 0.27061, Acc: 0.87000\n",
      "epoch: 1000, Loss: 0.25753, Acc: 0.88000\n",
      "\n",
      "During Time: 0.527 s\n"
     ]
    }
   ],
   "source": [
    "# 同样进行 1000 次更新\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "    # 前向传播\n",
    "    y_pred = logistic_reg(x_data)\n",
    "    loss = criterion(y_pred, y_data)\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    # 计算正确率\n",
    "    mask = y_pred.ge(0.5).float()\n",
    "    acc = (mask == y_data).sum().data[0] / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.data[0], acc))\n",
    "\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，使用了 PyTorch 自带的 loss 之后，速度有了一定的上升，虽然看上去速度的提升并不多，但是这只是一个小网络，对于大网络，使用自带的 loss 不管对于稳定性还是速度而言，都有质的飞跃，同时也避免了重复造轮子的困扰"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一节课我们会介绍 PyTorch 中构建模型的模块 `Sequential` 和 `Module`，使用这个可以帮助我们更方便地构建模型"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}