machinelearning_notebook/6_pytorch/1_NN/2-logistic-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic 回归模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上一节课我们学习了简单的线性回归模型，这一次课中，我们会学习第二个模型，Logistic 回归模型。\n",
    "\n",
    "Logistic 回归是一种广义的回归模型，其与多元线性回归有着很多相似之处，模型的形式基本相同，虽然也被称为回归，但是其更多的情况使用在分类问题上，同时又以二分类更为常用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型形式\n",
    "Logistic 回归的模型形式和线性回归一样，都是 y = wx + b，其中 x 可以是一个多维的特征，唯一不同的地方在于 Logistic 回归会对 y 作用一个 logistic 函数，将其变为一种概率的结果。 Logistic 函数作为 Logistic 回归的核心，我们下面讲一讲 Logistic 函数，也被称为 Sigmoid 函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid 函数\n",
    "Sigmoid 函数非常简单，其公式如下\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$\n",
    "\n",
    "Sigmoid 函数的图像如下\n",
    "\n",
    "![](https://ws2.sinaimg.cn/large/006tKfTcly1fmd3dde091g30du060mx0.gif)\n",
    "\n",
    "可以看到 Sigmoid 函数的范围是在 0 ~ 1 之间，所以任何一个值经过了 Sigmoid 函数的作用，都会变成 0 ~ 1 之间的一个值，这个值可以形象地理解为一个概率，比如对于二分类问题，这个值越小就表示属于第一类，这个值越大就表示属于第二类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外一个 Logistic 回归的前提是确保你的数据具有非常良好的线性可分性，也就是说，你的数据集能够在一定的维度上被分为两个部分，比如\n",
    "\n",
    "![](https://ws1.sinaimg.cn/large/006tKfTcly1fmd3gwdueoj30aw0aewex.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，上面红色的点和蓝色的点能够几乎被一个绿色的平面分割开来"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 回归问题 vs 分类问题\n",
    "Logistic 回归处理的是一个分类问题，而上一个模型是回归模型，那么回归问题和分类问题的区别在哪里呢？\n",
    "\n",
    "从上面的图可以看出，分类问题希望把数据集分到某一类，比如一个 3 分类问题，那么对于任何一个数据点，我们都希望找到其到底属于哪一类，最终的结果只有三种情况，{0, 1, 2}，所以这是一个离散的问题。\n",
    "\n",
    "而回归问题是一个连续的问题，比如曲线的拟合，我们可以拟合任意的函数结果，这个结果是一个连续的值。\n",
    "\n",
    "分类问题和回归问题是机器学习和深度学习的第一步，拿到任何一个问题，我们都需要先确定其到底是分类还是回归，然后再进行算法设计"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 损失函数\n",
    "前一节对于回归问题，我们有一个 loss 去衡量误差，那么对于分类问题，我们如何去衡量这个误差，并设计 loss 函数呢？\n",
    "\n",
    "Logistic 回归使用了 Sigmoid 函数将结果变到 0 ~ 1 之间，对于任意输入一个数据，经过 Sigmoid 之后的结果我们记为 $\\hat{y}$，表示这个数据点属于第二类的概率，那么其属于第一类的概率就是 $1-\\hat{y}$。如果这个数据点属于第二类，我们希望 $\\hat{y}$ 越大越好，也就是越靠近 1 越好，如果这个数据属于第一类，那么我们希望 $1-\\hat{y}$ 越大越好，也就是 $\\hat{y}$ 越小越好，越靠近 0 越好，所以我们可以这样设计我们的 loss 函数\n",
    "\n",
    "$$\n",
    "loss = -(y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "其中 y 表示真实的 label，只能取 {0, 1} 这两个值，因为 $\\hat{y}$ 表示经过 Logistic 回归预测之后的结果，是一个 0 ~ 1 之间的小数。如果 y 是 0，表示该数据属于第一类，我们希望 $\\hat{y}$ 越小越好，上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = - (log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "在训练模型的时候我们希望最小化 loss 函数，根据 log 函数的单调性，也就是最小化 $\\hat{y}$，与我们的要求是一致的。\n",
    "\n",
    "而如果 y 是 1，表示该数据属于第二类，我们希望 $\\hat{y}$ 越大越好，同时上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = -(log(\\hat{y}))\n",
    "$$\n",
    "\n",
    "我们希望最小化 loss 函数也就是最大化 $\\hat{y}$，这也与我们的要求一致。\n",
    "\n",
    "所以通过上面的论述，说明了这么构建 loss 函数是合理的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们通过例子来具体学习 Logistic 回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7fd3e0d2dd20>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设定随机种子\n",
    "torch.manual_seed(2017)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们从 data.txt 读入数据，感兴趣的同学可以打开 data.txt 文件进行查看\n",
    "\n",
    "读入数据点之后我们根据不同的 label 将数据点分为了红色和蓝色，并且画图展示出来了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fd36236a860>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAfp0lEQVR4nO3dfYxd9X3n8fd3jI01jRPAnkqRxx5PKrPBEKTiMU3+yCZqltahWbN5qmwMG9SkDmlIKjVNAEEsZDRKW61atQu7khPRgMcFsfmjclUaC5GgqGlgPYRHgyDGGDMOUibjdjckeHnod/84d+DOnftw7r3n4fc75/OSrjz33uN7fvecc7/nd76/h2PujoiIxG+k7AKIiEg2FNBFRCpCAV1EpCIU0EVEKkIBXUSkIs4qa8Xr1q3zTZs2lbV6EZEoPfLIIz9397F275UW0Ddt2sTs7GxZqxcRiZKZvdjpPaVcREQqQgFdRKQiFNBFRCpCAV1EpCIU0EVEKqJnQDezO8zsZ2b2VIf3zcz+xsyOmdkTZnZJ9sXM0MGDsGkTjIwk/x48WHaJREQykaaG/m1ge5f3Pwpsbjz2AP9z+GLl5OBB2LMHXnwR3JN/9+xRUBeRSugZ0N39B8DpLotcAdzliYeAc8zs3VkVMFM33QS/+tXS1371q+R1EZHIZZFDXw+81PR8rvHaMma2x8xmzWx2fn4+g1X36eTJ/l4XEYlIoY2i7r7f3afcfWpsrO3I1Xxt3Njf6xlR2r6etN+laFkE9FPAhqbn443XwjM9DaOjS18bHU1ez4nS9vWk/S5lyCKgHwL+a6O3y/uB/+PuL2fwudnbvRv274eJCTBL/t2/P3k9J0rbl6fMGrL2u5QhTbfFu4EfAf/BzObM7LNmdq2ZXdtY5D7gOHAM+CbwR7mVNgu7d8OJE/Dv/578m2MwB6Xti9QcwNetgz/4g/JqyDHsd6WEKsjdS3ls3brV62Biwj0JKUsfExNll6xaZmbcR0fbb+sytvug+31mJlnGLPl3Ziaf8rXbXqOj+a1PsgPMeoe4qpGiOSshbV9L7VIc7RRVQx5kvxeZd1dKqJqqG9ADuZ4sIW1fS2kDdc4dmt4yyH4vMsjGkBKS/llSgy/e1NSU53aDi8WqTvOvY3RUkbTCNm1KarTdhH4IjIwkNfNWZkmTT5Y6ba+JiaRpScJlZo+4+1S796pZQ9f1ZO20S3GsXAlr18ZzZVTkMAmlAqupmgFd15O10y7F8bd/Cz//eWEdmoZWZJBVKrCaqply0fWkROrgweRC8uTJpGY+Pa0gK0vVL+Wi60mJVMHDJKRiqhnQdT0pIjVUzYAOqupIkALpTSsVdVbZBRCpi9betIsDh0D1DclGdWvoIoFRb1rJmwK6SEFi6k2r1FCcFNBFClLS/VX6prnc46WALlKQWHrTKjUULwV0kYLE0pu2Uwqo11w5Uj4FdJECxdCbtlMKyExpl9ApoItELusGzOnpJHi3clfaJXQK6CIRy6MBc/fu9tP4Qpg9cuRtCugiEVqslV91VT4NmBMT7V8PrUeOLKWALplS/+X8NdfKOxm2Jh1LjxxZKlVAN7PtZvasmR0zsxvavD9hZg+Y2RNm9qCZjWdfVAmd+i8XI839U4etScfSI0eW6hnQzWwFcDvwUWALsMvMtrQs9t+Au9z9YmAf8I2sCyr9KaOmrP7LxehV+86qJh1Dj5xe6nbFmKaGfilwzN2Pu/trwD3AFS3LbAG+1/j7+23elwKVVVOOaWh7zLrVvlWTflsdrxjTBPT1wEtNz+carzV7HPhE4++PA2vMbG3rB5nZHjObNbPZ+fn5QcorKZRVU45laHvsOuW3Z2birUnnoY5XjFk1iv4p8CEzexT4EHAKeLN1IXff7+5T7j41NjaW0aqlVVk1ZTWkFaNbfrtuKYZmrd+9U6Nxpa8Y3b3rA/gAcLjp+Y3AjV2Wfwcw1+tzt27d6pKYmXGfmHA3S/6dmRnu8yYm3JOLzKWPiYnhy9pL1t9F0puZcR8dXbrPR0frsQ/afXez8n4HeQJmvVP87fTGWwskN8E4DkwCq0jSKxe2LLMOGGn8PQ3s6/W5CuiJPH6Edf5h11mZJ/KydfrurUE9z99BUZWZoQJ68v+5HHgOeB64qfHaPmBH4+9PAT9pLPMt4Oxen6mAnsjrR6iacv10qpGalV2y/HX67ou/pbx/B0VWoroFdEveL97U1JTPzs6Wsu6QjIy0H2ZtlnQXk84OHkwauE6eTBpep6fr3SDYKW88MZE0llZZ2d+9yPWb2SPuPtXuPY0ULZl6hgymjl3SOllsDHzxxeWTatWlUbrsBvlQuuwqoJes7AMxViF2SSujh0nrNADubwf1OvVJL3tkazAVs065mLwfyqG/Tfnu/oWWLy6rIbrODaEhUQ5dOXQZQtk501ZFlqe57aDTz1dtMMUrqk1HOXSpnNBSVUXlUFvbDjpRG0zxQpj7RgFdolR2zrRVUTnUNDMtqg2mvhTQJVoh1IgWFXXF0K3GH8KJTcp1VtkFEKmCxQCadw5148aw2g4kLKqhi2SkiCuG0NoOJCwK6CIRCa3tQMKilItIZHbvVgCX9lRDFxGpCAV0EZGKUEAXEakIBXSptTrfsk2qR42iUluLw+gXR14uTsELanSUOKmGLrUV4hS8IsNQQJfaCuWmBCJZUUCX2grmpgQiGUkV0M1su5k9a2bHzOyGNu9vNLPvm9mjZvaEmV2efVFFsqVh9FI1PQO6ma0Abgc+CmwBdpnZlpbFbgbudfffBHYC/yPrglaWulmURsPopWrS1NAvBY65+3F3fw24B7iiZRkH3tn4+13AT7MrYoXpTselC2EKXp3TJStpAvp64KWm53ON15rdAlxlZnPAfcCXMild1ambRe3pnC5ZyqpRdBfwbXcfBy4HDpjZss82sz1mNmtms/Pz8xmteghlV43UzaJ0ZR8COqdLltIE9FPAhqbn443Xmn0WuBfA3X8ErAbWtX6Qu+939yl3nxobGxusxFkJoWqkbhalCuEQ0DldspQmoB8BNpvZpJmtImn0PNSyzEngIwBmdgFJQA+gCt5FCFUjdbMoVQiHgM7p4Sj7ai0LPQO6u78BXAccBp4h6c1y1Mz2mdmOxmJfAf7QzB4H7gauce92T/IAhFA1UjeLUoVwCOicHoYQrtYy4e6lPLZu3eqlmphwT/bd0sfERLnlksKEcgjMzCTrNEv+nZkpdv2DirXc7YRyLKQBzHqHuFrfkaJ5Vo2qcO1WA6HUjkPoOtmvytRoG0K4WstEp0if96P0Grp7PlWMmRn30dGlp/nR0WiqLyHWuvIsU4jfNwYx1WjTiOn70KWGXu+AnoeYjowWIZ6LQixT1QxyUjNrf5ib5V3afMR0nCmgt8qzWhbxkR7iuSjEMlXJoIGsivsllqu1bgHdkveLNzU15bOzs8WvuPWuBpAkTrPqXbJpU5JQbDUxkSRIAzYykvwsW5kl+d0yhFimKhn0cM37ZySdmdkj7j7V7r36NYrm3fk4lJa2AYTYJzrEMlXJoI2B6nEbpvoF9LybsyM+0kM8F4VYpirpdGIcGendSSvG3jmV1ykXk/ejtBx6FZN/GQoxjxhimaqiXQ699RFq42BdoRx6EyX/RJY4eDDJOJ48mdTK33xz+TIRNAHVhnLozdKkRDQwKBraVcNrTp10amiOboBNTdWvht6LavDRaLerVq2CNWvg9OkkPzw9rd3Wj4g7adWGauj9CGEKPkml3a567TVYWKjGcPQyqBE6X3lfUSqgt6rMpA7Vl2aX6Fzcn4g7aQWviPlvlHJppWvOaHTaVa00CElCkFVoUcqlH1lfc6rVLjftdlU7GoQkISji4l8BvVWW15xVm2M0MK27au1aWLly6TLK/0ooihj1rJRLnpS+KVxzn2r1cpGQZNWBTimXsqiBtXAajr6csn5hKKLBWQE9T5pZqnRVC2b9fp+qZf1i35+5Vzg6zQmQ96OyN7hoFtOs+SnENqdKxTb/QN+nSlMXVW1/Dophb3ABbAeeBY4BN7R5/6+AxxqP54B/6/WZtQjo7vFFwQ5i/DFVKZi5D/Z9Ir7fyjJV25+DGiqgAyuA54H3AKuAx4EtXZb/EnBHr8+tTUCviBh/TFUKZu6DfZ9O+21x34V8Qm4V8v7sVm/Luk7XLaCnyaFfChxz9+Pu/hpwD3BFl+V3AXf3k/aR8MXYvlu1JoxBvk+3vvqx5dND3Z/d2ikKb8PoFOkXH8CngG81Pb8auK3DshPAy8CKDu/vAWaB2Y0bNw53mpJCxVhDjzFN1M2g32exhtitph6DUPdnt99GHr8bhky59BPQrwf+e6/PdKVcohPqj6mXijRhvGWY7xNyyiKtEPdnt+2axzbvFtB7Diwysw8At7j77zae39io2X+jzbKPAl9093/pdWVQi4FFFaNBO3HTOLd8dNuukP02H3Zg0RFgs5lNmtkqYCdwqM1K3gucC/xosGJK6DRoJ26aGjcf3bZr0du8Z0B39zeA64DDwDPAve5+1Mz2mdmOpkV3Avd4ryq/iJRCU+Pmo9t2LXqbay6XmCjnEQztCilLt5TLWUUXRgbUOrPPYv8nUCQpmHaFhEpzucRCt8YLhnZFGGKf1yUPCuixiHFkT0VpV5QvrwE7sZ8kFNCzUMRREOowuRrSrihfHldJVZiZUgF9WEUdBZH3OYu95tMs8l1RCXlcJVUildZpxFHej8qMFC1yTHyIw+RSiHWUaTeR7orKyONnF8tIWoYZKZqXynRbHBlJ9nsr3Wr+LRqhKFnL6nZuzWI5TnULujwpodqTGhEla3kM2KlCKk0BfVhVOApypnOe5CHrqSiqMJJWAX1YVTgK0hqwZVPnPIlF7PMVaaRoFhYnbaiyIYZHLr6tofIi+VKjqKQTS4uRSMWpUVSGp5ZNkeApoEs6atkUCZ4CuqSjlk2R4CmgSzp16s3TQZWmL5DiFHncKKBLeot9ug4cSJ5ffXVtIlsVJm6S4hV93NQ7oKvK1b+aRrZKTNwkhSv6uKlvQK9pYFqm35NaTSObOvnIIIo+buob0GsamJYY5KSW0xEa+sWSOvnIIIo+blIFdDPbbmbPmtkxM7uhwzK/b2ZPm9lRM/u7bIuZA1W5Bjup5XCExnCxpE4+MojCj5tO8+ouPoAVwPPAe4BVwOPAlpZlNgOPAuc2nv96r88tfT70IucxD9UgE0DnMLl5LLtCc6DLILI+bugyH3qagP4B4HDT8xuBG1uW+Qvgc70+q/lRekCv4l0X+jVoJM34CI3lxgIiIegW0NOkXNYDLzU9n2u81ux84Hwz+6GZPWRm29t9kJntMbNZM5udn59PseocqV/14NeDGU9Jp/y0SDayahQ9iyTt8mFgF/BNMzundSF33+/uU+4+NTY2ltGqhxD7XJnDCuSkpvy0SDbSTJ97CtjQ9Hy88VqzOeBhd38deMHMniMJ8EcyKaXkJ4CpfzW9rkg20tTQjwCbzWzSzFYBO4FDLcv8PUntHDNbR5KCOZ5hOaXi6n6xVCehd1GNWc8auru/YWbXAYdJerzc4e5HzWwfSXL+UOO93zGzp4E3ga+6+0KeBReR+AxxnxRJIVUO3d3vc/fz3f033H268dreRjCn0fj6J+6+xd3f5+735Fno2lMVRwLX6RDVeL586RZ0sVEVRwLX7RDVeL586RZ0sdGt4CRw3Q5R0OE7LN2CrkpUxZHAdTtE1UU1XwrosdEoHAlct0M0kKEPlaWAHhtVcSRwvQ5RdVHNjwJ6bFTFkcDpEC2PGkVFRCKiRlERkRpQQBcRqQgFdBGRilBAF8mYZmaQsiigS35qGNliuD+qVJcCel3lHWxDiWwFn1Q0+ZSUSd0W66h19iRIRn5k2Vk4hDlnivieLUZGkvNXK7NkII3IsLp1W1RAr6Migm0Ika2Ek0oI5zGpNvVDl6WKmOArhDlnSpjITDMzSJkU0OuoiGAbQmQr4aSiYe9SJgX0Oioi2JYd2Q4ehFdeWf56AScVTT4lZVFAr6NuwTbLXiFlRbbFxtCFltvarl2r6rJUWqqAbmbbzexZMztmZje0ef8aM5s3s8caj89lX1Rq2a85N+2CbShdDYfVru8gwDveoWAuldazl4uZrQCeAy4D5oAjwC53f7ppmWuAKXe/Lu2K++7lUkIXtNqpSheNEHrYiORk2F4ulwLH3P24u78G3ANckWUBU9GIjfx16v3RLsiHLIQeNiIlSBPQ1wMvNT2fa7zW6pNm9oSZfcfMNrT7IDPbY2azZjY7Pz/fX0l1L838dQp4ZnGlXULoYVNTyoqWK6tG0X8ANrn7xcD9wJ3tFnL3/e4+5e5TY2Nj/a1Bta78TU8nwbuVe1xXQmX3sKmpqjTBxCxNDv0DwC3u/ruN5zcCuPs3Oiy/Ajjt7u/q9rnKoQeqXUBffF35Z+miKk0woRs2h34E2Gxmk2a2CtgJHGpZwbubnu4Anhm0sB2p1lWMiYn2r+tKKEpFpkCUFS1fz4Du7m8A1wGHSQL1ve5+1Mz2mdmOxmJfNrOjZvY48GXgmlxKm2e/ZiX/Eso/V0bRKRBlRQPg7qU8tm7d6sGYmXEfHXVPjvvkMTqavF5HMzPuExPuZsm/X/jC0ud13S6dtG6vQLbPxMTSQ3rxMTGRz/r0MyoGMOsd4qoCunvxR35M9CvtrsTt0+s8Ytb+sDYrr0wyvG4BXdPnggaidKOWru5K2j5p+gho11WTps/tRcm/9g4e7DyoSC1diZJaAtOMs1NzyOBibVJTQAcd+e0sVgE7qfvJblFJlYE055E6dAzLI/BG3Z++Uy4m70dQOXR3Jf9adWpXUA59qZJy6Gr2yW/Th75tUaOo9K1TixoomLcqoTKQ93kkhvpNXoG3jMbkfnQL6GoUlfbUoha8gweTnPnJk0mGZ3o6m3RKLIOy8+rLEPqhX61G0SySZrG2eBRJ7QrBy2ucXSwTm+bVfBH1od+p6p73Y6CUSxbXmepXnV4M192SuSxTDnkeQnn+lEM+9KlMDr3fpFm7vRJ6i4dIybL6iRRRd2r+ia9dmzxCDMJZqk5A76fq0Olo6tTQF0qLh8Qh5CrckLIKxEXWnep04V2dgN7PEdJp2RUrVEOvkjICaw2iR6rN2mOhInuL1OnCuzoBvZ8fUrdudxX/MdZGWYG1TtGjkxTbvsjNFHpXwyxVJ6C7p6+RdTuaKny5XCtlBdY6RY9OUmz7Is+3dTrHViugp1WDy+LaKyuwhhA9yq6UpNz2RRWzTj/3egZ09/IPeslXWYG17OhR9vrdwziptajLz72+AV2qrejAFkofuRCCaQgnlZrqFtDjGykqsqjI6QRbp+BbWIBXX4UDB7K/HWIvIdy8c/du+MxnYMWK5PmKFcnzkOYGqCHN5SKSRkgTfIRQllgmfKmgbnO5BBXQX3/9debm5jhz5kwpZcra6tWrGR8fZ+XKlWUXRYYV0l2tQgimIZxUaqpbQD8r5QdsB/4aWAF8y93/rMNynwS+A2xz976r33Nzc6xZs4ZNmzZhZv3+96C4OwsLC8zNzTE5OVl2cWRYGze2D2Bl3OhjMWjnMdViWiGkfWSZnjl0M1sB3A58FNgC7DKzLW2WWwP8MfDwoIU5c+YMa9eujT6YA5gZa9eurczVRuZim/EytCn48ppqMS3dtjFIaRpFLwWOuftxd38NuAe4os1ytwJ/DgwVwaoQzBdV6btkKsZ7fNXhfm79CO0E16/YKhQppQno64GXmp7PNV57i5ldAmxw93/s9kFmtsfMZs1sdn5+vu/CSkXEMuF2q7JrxSGJ+QQXY4UipaG7LZrZCPCXwFd6Levu+919yt2nxsbGhl11Zc+ylaf8azXEeoIbpEIRSaxJE9BPARuano83Xlu0BrgIeNDMTgDvBw6ZWdtW2MyUfJa988472bx5M5s3b+bOO+8sZJ2VofyrlKnfCkVMNfpOI44WHyQ9YY4Dk8Aq4HHgwi7LPwhM9frcdiNFn3766fTDpUocLbewsOCTk5O+sLDgp0+f9snJST99+nTbZfv6TnWhUYZSpn5jRwgjc5swzEhRd38DuA44DDwD3OvuR81sn5ntyOMkk0oOl+1Hjhzh4osv5syZM/zyl7/kwgsv5Kmnnlq23OHDh7nssss477zzOPfcc7nsssv47ne/O/B6ayfm/KvEr98G3YhShKn6obv7fcB9La/t7bDsh4cvVgo59Avetm0bO3bs4Oabb+bVV1/lqquu4qKLLlq23KlTp9iw4e0s1Pj4OKdOnVq2nHSxe7cCuJSj3378IY1B6CHeuVxy6ja1d+9e7r//fmZnZ/na17421GeJSKD6adCNqItmvAE9p8v2hYUFXnnlFX7xi190HBS0fv16Xnrp7Z6cc3NzrF+/vu2yIhK5iFKEQc3l8swzz3DBBReUUp5FO3bsYOfOnbzwwgu8/PLL3HbbbcuWOX36NFu3buXHP/4xAJdccgmPPPII55133rJlQ/hOIlId3eZyibeGnoO77rqLlStXcuWVV3LDDTdw5MgRvve97y1b7rzzzuPrX/8627ZtY9u2bezdu7dtMJcmkfTjFYmZaug5q+J36lsIswOKVIRq6FKuWIf6i0QmVbfFunryySe5+uqrl7x29tln8/DDA08oWU8R9eMViZkCehfve9/7eOyxx8ouRvwi6scrEjOlXCR/EfXjFYmZArrkL6J+vCIxU8pFiqGh/iK5i7qGrq7NIiJvizaglz1F8fbt2znnnHP42Mc+VswKRUR6iDagl921+atf/SoHDhwoZmUiIilEG9Dz6Nqcdj50gI985COsWbNm8JWJiGQs2kbRPLo2p50PXUQkRNHW0PPq2qz50EUkVtEG9Ly6NqeZD10kCOrmJS2iTblAPl2bP//5z3PrrbfywgsvcP3117edD12kdK0zWC528wL196+xVDV0M9tuZs+a2TEzu6HN+9ea2ZNm9piZ/bOZbcm+qPlLOx86wAc/+EE+/elP88ADDzA+Ps7hw4cLLq3UWtndvCRIPedDN7MVwHPAZcAccATY5e5PNy3zTnf/v42/dwB/5O7bu32u5kMXGcLISDIAo5VZcp9Mqaxh50O/FDjm7sfd/TXgHuCK5gUWg3nDrwHl3DVDpC46defSDJa1liaHvh54qen5HPBbrQuZ2ReBPwFWAb+dSelKpvnQJVjT0+3vAqUZLGsts0ZRd78duN3MrgRuBj7TuoyZ7QH2AGzsUJNwd8wsq2INZdj50Mu6vZ/UwGLD5003JaPpNm5MgrkaRGstTcrlFLCh6fl447VO7gH+S7s33H2/u0+5+9TY2Niy91evXs3CwkIlAqG7s7CwwOrVq8suilTV7t1w4kSSMz9xQsFcUtXQjwCbzWySJJDvBK5sXsDMNrv7TxpPfw/4CQMYHx9nbm6O+fn5Qf57cFavXs34+HjZxRCRmugZ0N39DTO7DjgMrADucPejZrYPmHX3Q8B1ZvafgNeBf6VNuiWNlStXMjk5Och/FRGpvVQ5dHe/D7iv5bW9TX//ccblEhGRPkU79F9ERJZSQBcRqYieI0VzW7HZPNBmAtxU1gE/z7A4eYqprBBXeWMqK6i8eYqprDBceSfcfXk3QUoM6MMws9lOQ19DE1NZIa7yxlRWUHnzFFNZIb/yKuUiIlIRCugiIhURa0DfX3YB+hBTWSGu8sZUVlB58xRTWSGn8kaZQxcRkeViraGLiEgLBXQRkYoIOqDHdOu7XmVtWu6TZuZmVmoXqxTb9hozm29s28fM7HNllLNRlp7b1sx+38yeNrOjZvZ3RZexpSy9tu1fNW3X58zs38ooZ6Msvcq60cy+b2aPmtkTZnZ5GeVsKk+v8k6Y2QONsj5oZqXNjmdmd5jZz8zsqQ7vm5n9TeO7PGFmlwy9UncP8kEyEdjzwHtIbprxOLClZZl3Nv29A/huqGVtLLcG+AHwEDAV+La9BrgtkuNgM/AocG7j+a+HXN6W5b9EMuFdkGUlabz7QuPvLcCJkLct8L+AzzT+/m3gQInl/Y/AJcBTHd6/HPgnwID3Aw8Pu86Qa+gx3fquZ1kbbgX+HDhTZOHaSFveEKQp6x8Ct7v7vwK4+88KLmOzfrftLuDuQkq2XJqyOvDOxt/vAn5aYPlapSnvFmDxzu7fb/N+Ydz9B8DpLotcAdzliYeAc8zs3cOsM+SA3u7Wd+tbFzKzL5rZ88BfAF8uqGytepa1cTm1wd3/sciCdZBq2wKfbFwKfsfMNrR5vwhpyno+cL6Z/dDMHjKzrjcoz1nabYuZTQCTvB2AipamrLcAV5nZHMmMq18qpmhtpSnv48AnGn9/HFhjZmsLKNsgUh8raYUc0FNx99vd/TeA60lufRccMxsB/hL4Stll6cM/AJvc/WLgfuDOksvTzVkkaZcPk9R4v2lm55RaonR2At9x9zfLLkgXu4Bvu/s4SYrgQON4DtWfAh8ys0eBD5HclCfk7ZupkHdMZre+K0Cvsq4BLgIeNLMTJPmyQyU2jPbctu6+4O7/r/H0W8DWgsrWKs1xMAcccvfX3f0F4DmSAF+Gfo7bnZSXboF0Zf0scC+Au/8IWE0ysVQZ0hy3P3X3T7j7bwI3NV4rrdG5h35jXG9lNRikaFA4CzhOckm62AByYcsym5v+/s8kd1AKsqwtyz9IuY2iabbtu5v+/jjwUMBl3Q7c2fh7Hcll7NpQy9tY7r3ACRqD+0ItK0mj3TWNvy8gyaGXUuaU5V0HjDT+ngb2lbV9G2XYROdG0d9jaaPo/x56fWV+2RQb43KS2tbzwE2N1/YBOxp//zVwFHiMpAGkYxAtu6wty5Ya0FNu2280tu3jjW373oDLaiQpraeBJ4GdIW/bxvNbgD8rs5wpt+0W4IeN4+Ax4HcCL++nSO5p/BzJleXZJZb1buBlkltzzpFc7VwLXNt434DbG9/lySxigob+i4hURMg5dBER6YMCuohIRSigi4hUhAK6iEhFKKCLiFSEArqISEUooIuIVMT/B1eAsHqTN8dBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 从 data.txt 中读入点\n",
    "with open('./data.txt', 'r') as f:\n",
    "    data_list = [i.split('\\n')[0].split(',') for i in f.readlines()]\n",
    "    data = [(float(i[0]), float(i[1]), float(i[2])) for i in data_list]\n",
    "# 标准化\n",
    "x0_max = max([i[0] for i in data])\n",
    "x1_max = max([i[1] for i in data])\n",
    "data = [(i[0]/x0_max, i[1]/x1_max, i[2]) for i in data]\n",
    "\n",
    "x0 = list(filter(lambda x: x[-1] == 0.0, data)) # 选择第一类的点\n",
    "x1 = list(filter(lambda x: x[-1] == 1.0, data)) # 选择第二类的点\n",
    "\n",
    "plot_x0 = [i[0] for i in x0]\n",
    "plot_y0 = [i[1] for i in x0]\n",
    "plot_x1 = [i[0] for i in x1]\n",
    "plot_y1 = [i[1] for i in x1]\n",
    "\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们将数据转换成 NumPy 的类型，接着转换到 Tensor 为之后的训练做准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "np_data = np.array(data, dtype='float32') # 转换成 numpy array\n",
    "x_data = torch.from_numpy(np_data[:, 0:2]) # 转换成 Tensor, 大小是 [100, 2]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们来实现以下 Sigmoid 的函数，Sigmoid 函数的公式为\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义 sigmoid 函数\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画出 Sigmoid 函数，可以看到值越大，经过 Sigmoid 函数之后越靠近 1，值越小，越靠近 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fd3622d7588>]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAcAElEQVR4nO3deZhU5Zn38e8tqygKCLIjOIKRZIxLS6KOr/oqCkRB4wbRuKFMiDhxXEYdHTVqriSS10QnGkTFuLPE6RYQLlTUMVFRGiKoINK4NsqiIiIGmpb7/eOp1rKppqubqnpq+X2u61xVdc7Tfe4+ffj14TnLY+6OiIgUvp1iFyAiIpmhQBcRKRIKdBGRIqFAFxEpEgp0EZEi0TLWijt37ux9+/aNtXoRkYK0YMGCj929S6pl0QK9b9++VFZWxlq9iEhBMrP3GlqmLhcRkSKhQBcRKRIKdBGRIqFAFxEpEgp0EZEi0Wigm9kkM1tjZq83sNzM7HYzqzKzxWZ2UObLFBGRxqRzhP5nYMh2lg8F+iemMcCfdrwsERFpqkavQ3f3582s73aajAAe8PAc3nlm1sHMurv7RxmqUUSKlTts3px62rQJamqgtha++ir1tL1lX30FW7eGddRNdets6rymfl39n7G+E0+EQw7J7LYkMzcW9QQ+SPpcnZi3TaCb2RjCUTx9+vTJwKpFJJraWvjkE/j4Y1i7NrzWvV+3DjZs2Hb64ovw+uWXIbC3bIn9U+SG2bc/9+iRt4GeNnefCEwEKCsr08gaIvnMHaqrYelSeOsteP/9MH3wQXj98MNwBJzKrrtC+/bfTLvuCr16ffO5XTto0yZMbdt+8z55atsWWreGli2hRYttp4bmJ0877RTCNHmC5s1rytdFkolAXwn0TvrcKzFPRArF5s2weDHMnw+vvAKvvw5vvgkbN37Tpk0b6N0b+vSBY44J77t1gy5doHPnb1732CMEseRcJgJ9OjDOzCYDPwDWq/9cJM/V1MBLL8FTT8HcubBwYZgHsOee8P3vw+jRsN9+Ydp3X+jaNfoRqGxfo4FuZo8CRwGdzawauB5oBeDuE4BZwDCgCvgSOC9bxYrIDtiwAWbMgGnTQpBv3Bi6JQ45BH7xCxg0KEy9eyu4C1Q6V7mMamS5AxdlrCIRyZytW+HJJ2HiRJg1K3St9OwJ554LgwfDUUfB7rvHrlIyJNrjc0Ukiz7/HCZMCNM774T+7X/9Vzj9dDj00HCyUIqOAl2kmHz6Kdx+O9x2G3z2GRx5JPz613DSSeGkphQ1BbpIMaipgTvugF/+EtavDwF+7bVw8MGxK5McUqCLFLqnn4Zx42DZMjj+eLjlFth//9hVSQTqSBMpVBs2hH7xwYPDyc+ZM2H2bIV5CdMRukghevllGDkS3nsPrrgCbrwx3FkpJU1H6CKFxD1cuXLEEeHz3/4WulgU5oICXaRwbNkCF14IY8eGW+8XLIDDDotdleQRBbpIIdi4EUaMgHvvhWuuCf3lnTrFrkryjPrQRfLdp5/C0KFQWQl33QVjxsSuSPKUAl0kn61fD8cdF55++Nhj4fpykQYo0EXy1YYN4ch88WIoL4cf/Sh2RZLnFOgi+WjLFjj55PBs8mnTFOaSFgW6SL5xD1eyzJ0L998fgl0kDbrKRSTf/O534WqWa6+Fs8+OXY0UEAW6SD6ZMweuvBLOOCM8aEukCRToIvli5Uo46yz43vdg0iQ9s1yaTHuMSD6orYVRo+Af/wgnQdu1i12RFCCdFBXJBzfdBH/9Kzz0UBiQWaQZdIQuEtvChfCrX8E558CZZ8auRgqYAl0kppqaMGBz167w+9/HrkYKnLpcRGK6+WZ47bXwsK2OHWNXIwVOR+gisSxZEgZw/ulPdSeoZIQCXSQGd7j4YmjfHm69NXY1UiTU5SISw7Rp8MwzcOed0Llz7GqkSOgIXSTXvvgCLrsMDjhAzzaXjNIRukiu3XILVFfD5MnQokXsaqSI6AhdJJdWrw595qefDocfHrsaKTIKdJFcuvlm2LQp3BkqkmEKdJFcefvtMCbo6NEwYEDsaqQIKdBFcuW660Kf+fXXx65EilRagW5mQ8xsmZlVmdlVKZb3MbNnzezvZrbYzIZlvlSRArZsGTzySLj2vEeP2NVIkWo00M2sBXAHMBQYCIwys4H1ml0LTHX3A4GRwJ2ZLlSkoP3mN9CmTbhcUSRL0jlCHwRUufvb7l4DTAZG1GvjwG6J97sDH2auRJEC99574bG4F14YHsIlkiXpBHpP4IOkz9WJecluAM4ys2pgFnBxqm9kZmPMrNLMKteuXduMckUK0PjxYAZXXBG7EilymTopOgr4s7v3AoYBD5rZNt/b3Se6e5m7l3Xp0iVDqxbJY6tWwT33hMGee/eOXY0UuXQCfSWQvCf2SsxLNhqYCuDuLwFtAT2gQuS222DLFrhqm2sJRDIunUCfD/Q3s35m1ppw0nN6vTbvA8cAmNl+hEBXn4qUti+/DNedn3wy7LNP7GqkBDQa6O5eC4wD5gBLCVezvGFmN5rZ8ESzy4ALzWwR8Chwrrt7tooWKQgPPgjr1sEll8SuREqExcrdsrIyr6ysjLJukaxzh+9+F3beGSorw0lRkQwwswXuXpZqmZ62KJINTz0FS5fCAw8ozCVndOu/SDb84Q/QrVt4qqJIjijQRTKtqgpmz4axY8PdoSI5okAXybS77w4P4brwwtiVSIlRoItkUk0N3HcfnHgidO8euxopMQp0kUx6/HFYu1ZjhUoUCnSRTJo4Efr0geOOi12JlCAFukimrFgBTz8NF1ygwZ8lCgW6SKbccw/stBOcf37sSqREKdBFMqG2NpwMPeEE6Fn/6dIiuaFAF8mEJ5+E1avhvPNiVyIlTIEukgkPPAB77AHDNJyuxKNAF9lR69dDRQWMHAmtW8euRkqYAl1kR/3lL7B5M/z0p7ErkRKnQBfZUQ88AAMGwKBBsSuREqdAF9kR774Lzz8fxgzVY3IlMgW6yI546KHweuaZcesQQYEu0nzuobvlyCOhb9/Y1Ygo0EWabeFCWL5cR+eSNxToIs01ZQq0bAmnnBK7EhFAgS7SPO4wdSoMHgydOsWuRgRQoIs0z8svw3vvwRlnxK5E5GsKdJHmmDo13BU6YkTsSkS+pkAXaaqtW0OgH388dOgQuxqRrynQRZrqxRdh5Up1t0jeUaCLNNWUKdC2LQwfHrsSkW9RoIs0xVdfhYdxDRsG7dvHrkbkWxToIk3x17/CqlXqbpG8pEAXaYpp02DnneFHP4pdicg2FOgi6dq6FR5/HIYMgV12iV2NyDbSCnQzG2Jmy8ysysyuaqDN6Wa2xMzeMLNHMlumSB5YsCBc3XLyybErEUmpZWMNzKwFcAcwGKgG5pvZdHdfktSmP3A1cLi7rzOzPbNVsEg0FRXQooW6WyRvpXOEPgiocve33b0GmAzUvz3uQuAOd18H4O5rMlumSB6oqAiPytWzWyRPpRPoPYEPkj5XJ+YlGwAMMLMXzGyemQ1J9Y3MbIyZVZpZ5dq1a5tXsUgMb70FS5bASSfFrkSkQZk6KdoS6A8cBYwC7jazbe6JdveJ7l7m7mVdunTJ0KpFcqCiIrzq2S2Sx9IJ9JVA76TPvRLzklUD0919i7u/A7xFCHiR4lBRAQcdBH36xK5EpEHpBPp8oL+Z9TOz1sBIYHq9NhWEo3PMrDOhC+btDNYpEs9HH8G8ebq6RfJeo4Hu7rXAOGAOsBSY6u5vmNmNZlb3MIs5wCdmtgR4FrjC3T/JVtEiOTVjRhjQQv3nkufM3aOsuKyszCsrK6OsW6RJhg0LJ0WXLwez2NVIiTOzBe5elmqZ7hQV2Z7PP4e5c8PRucJc8pwCXWR7Zs+Gmhp1t0hBUKCLbE9FBey5Jxx6aOxKRBqlQBdpyObN8MQTYSCLFi1iVyPSKAW6SEOeew42bFB3ixQMBbpIQyoqwmNyjzkmdiUiaVGgi6RS9+zzoUPD+KEiBUCBLpLKK6+EO0TV3SIFRIEukkpFBbRsqWefS0FRoIukUlEBRx8NHbZ5aKhI3lKgi9T35puwbJm6W6TgKNBF6isvD6/Dh2+/nUieUaCL1FdRAYccAr16xa5EpEkU6CLJVq4MV7jo2edSgBToIsmmJ8ZuUf+5FCAFukiyigoYMAC+853YlYg0mQJdpM5nn8Ezz+jZ51KwFOgidWbNgtpadbdIwVKgi9SpqIBu3eAHP4hdiUizKNBFADZtCqMTjRgBO+mfhRQm7bkiEPrOv/hC3S1S0BToIhDuDm3fPjy/RaRAKdBFvvoqXH8+bBi0aRO7GpFmU6CLvPQSrFmju0Ol4CnQRcrLoXXrMDqRSAFToEtpcw+BfuyxsNtusasR2SEKdCltr70G77yjq1ukKCjQpbSVl4fb/PXscykCCnQpbeXlcPjh0LVr7EpEdpgCXUrXO+/AokW6ukWKhgJdSldFRXhV/7kUibQC3cyGmNkyM6sys6u20+4UM3MzK8tciSJZUl4O++8Pe+8duxKRjGg00M2sBXAHMBQYCIwys4Ep2rUHfgG8nOkiRTJuzRr429/U3SJFJZ0j9EFAlbu/7e41wGRgRIp2NwG/BTZlsD6R7JgxI1yDru4WKSLpBHpP4IOkz9WJeV8zs4OA3u7+xPa+kZmNMbNKM6tcu3Ztk4sVyZjycujbF77//diViGTMDp8UNbOdgFuByxpr6+4T3b3M3cu6dOmyo6sWaZ4NG+Dpp0N3i4aakyKSTqCvBHonfe6VmFenPfA94Dkzexf4ITBdJ0Ylb82cCZs3w49/HLsSkYxKJ9DnA/3NrJ+ZtQZGAtPrFrr7enfv7O593b0vMA8Y7u6VWalYZEdNnQo9esBhh8WuRCSjGg10d68FxgFzgKXAVHd/w8xuNDPdLy2FZcOGMNTcqadqqDkpOi3TaeTus4BZ9eZd10Dbo3a8LJEsqetuOe202JWIZJwOUaS0qLtFipgCXUqHulukyGmvltIxY4a6W6SoKdCldEybpu4WKWoKdCkN6m6REqA9W0qDulukBCjQpTRMmaLuFil6CnQpfp98ErpbRo1Sd4sUNe3dUvymTYMtW+DMM2NXIpJVCnQpfg8/DAMHwgEHxK5EJKsU6FLc3n03jEx01ll6VK4UPQW6FLdHHgmvP/lJ3DpEckCBLsXLHR56CI44AvbaK3Y1IlmnQJfi9eqrsHSpToZKyVCgS/F66CFo1Uo3E0nJUKBLcdqyJQT6CSdAp06xqxHJCQW6FKeZM2HNGhg9OnYlIjmjQJfidO+94Vb/44+PXYlIzijQpfisXBlu9T/3XGiZ1iiLIkVBgS7F5/77YetWOO+82JWI5JQCXYrL1q0waRIceSTss0/sakRySoEuxeX552HFCp0MlZKkQJfiMmECdOgAp5wSuxKRnFOgS/H48EN47DE4/3xo1y52NSI5p0CX4jFxInz1FYwdG7sSkSgU6FIcamrgrrtg6FCdDJWSpUCX4lBeDqtWwbhxsSsRiUaBLsXhj3+Ef/on3RkqJU2BLoVv/vwwKtFFF2kQaClp2vul8I0fD7vvDhdcELsSkajSCnQzG2Jmy8ysysyuSrH8UjNbYmaLzWyumWl4GMmNqqpwqeLYsdC+fexqRKJqNNDNrAVwBzAUGAiMMrOB9Zr9HShz9/2BvwC3ZLpQkZRuvTU8gOvf/i12JSLRpXOEPgiocve33b0GmAyMSG7g7s+6+5eJj/OAXpktUySFtWvhvvvg7LOhe/fY1YhEl06g9wQ+SPpcnZjXkNHA7FQLzGyMmVWaWeXatWvTr1IklT/8ATZvhssui12JSF7I6ElRMzsLKAPGp1ru7hPdvczdy7p06ZLJVUup+fhjuP12OP10+M53YlcjkhfSefr/SqB30udeiXnfYmbHAtcAR7r75syUJ9KA3/0ONm6E66+PXYlI3kjnCH0+0N/M+plZa2AkMD25gZkdCNwFDHf3NZkvUyTJmjXw3/8NP/kJ7Ldf7GpE8kajge7utcA4YA6wFJjq7m+Y2Y1mNjzRbDywKzDNzF41s+kNfDuRHTd+PGzaBNddF7sSkbyS1oCL7j4LmFVv3nVJ74/NcF0iqb3/frjN/6yzYMCA2NWI5BXdKSqF5T//M7zefHPcOkTykAJdCscrr8DDD4fLFHv3bry9SIlRoEthcIdLL4WuXeHKK2NXI5KX0upDF4lu8mR44QW4+249s0WkATpCl/y3bh38+79DWRmcd17sakTylo7QJf9dfXV4bsvs2dCiRexqRPKWjtAlv734Yhgr9JJL4MADY1cjktcU6JK/Nm6Ec8+FPn3gl7+MXY1I3lOXi+Svyy8PA1jMnQu77hq7GpG8pyN0yU9PPAETJoRLFY8+OnY1IgVBgS75p7o6XM3yz/8Mv/pV7GpECoYCXfLL5s1w2mnwj3/AlCnQpk3sikQKhvrQJb9ceinMmwfTpunRuCJNpCN0yR933hmmyy+HU0+NXY1IwVGgS354/HG4+GI48UT49a9jVyNSkBToEt8LL8CoUXDwwfDoo9BSPYEizaFAl7hefBGGDoVevWDmTNhll9gViRQsBbrE89JLMGQIdOsGzz4Le+4ZuyKRgqZAlzhmzoRjj/0mzHv2jF2RSMFToEvu/elPMGJEuCzx+ecV5iIZokCX3Nm0CcaOhZ//HIYNg//933CELiIZoUCX3FixAg47LDyf5YoroLxcJ0BFMkzXh0l2bd0aQvzKK6FVK5gxA044IXZVIkVJR+iSPUuWwFFHwUUXwaGHwt//rjAXySIFumTe6tXws5+FpyW+9hrcdx/MmQN77RW7MpGipi4XyZwPP4Tf/z5cxbJ5M4wbB//1X9C5c+zKREqCAl12jDssWBD6yR98EGpr4YwzwpBx/fvHrk6kpCjQpXlWrw6PuL3nHli0CHbeGS64AC67DPbeO3Z1IiVJgS7pcYe33gp3eJaXh2ewuIcHat15Z3i4VocOsasUKWkKdEmtthaWLQvPW3n2WXjuudBHDnDAAXDDDXDyyeHEp4jkBQV6qdu6NQT18uXhCHzRIli4EBYvDsPAAXTtGgZqPvpoGDwY+vWLW7OIpJRWoJvZEOA2oAVwj7v/pt7yNsADwMHAJ8AZ7v5uZkuVJtuyBT77DFatCqFdf1qxAqqqvglugN12gwMPDJcdHnggHHII7LsvmMX7OUQkLY0Gupm1AO4ABgPVwHwzm+7uS5KajQbWufs+ZjYS+C1wRjYKLkjuoQujtjaEbN37+p/rL6upgS+/DIGb/Fp/3hdfhOBet+7b08aNqevp1Am6dw8nLwcPhn32CVek9O8PvXvDTro9QaQQpXOEPgiocve3AcxsMjACSA70EcANifd/Af5oZubunsFag0mTYPz4EJIQXpOndOfl6utra0O3RjbsvHOYdtklnJDs2DGEdMeO30wdOoQHYPXoEabu3aFt2+zUIyJRpRPoPYEPkj5XAz9oqI2715rZemAP4OPkRmY2BhgD0KdPn+ZV3LlzOBFX1wVgtu2Uan668zLdtmXLMLVqlfr99j63agXt2oXQTn5t1w7atNGRtIh8S05Pirr7RGAiQFlZWfOO3ocPD5OIiHxLOod4K4HeSZ97JealbGNmLYHdCSdHRUQkR9IJ9PlAfzPrZ2atgZHA9HptpgPnJN6fCjyTlf5zERFpUKNdLok+8XHAHMJli5Pc/Q0zuxGodPfpwL3Ag2ZWBXxKCH0REcmhtPrQ3X0WMKvevOuS3m8CTstsaSIi0hS6TEJEpEgo0EVEioQCXUSkSCjQRUSKhMW6utDM1gLvNfPLO1PvLtQ8obqaRnU1Xb7WprqaZkfq2svdu6RaEC3Qd4SZVbp7Wew66lNdTaO6mi5fa1NdTZOtutTlIiJSJBToIiJFolADfWLsAhqguppGdTVdvtamupomK3UVZB+6iIhsq1CP0EVEpB4FuohIkcjbQDez08zsDTPbamZl9ZZdbWZVZrbMzI5v4Ov7mdnLiXZTEo/+zXSNU8zs1cT0rpm92kC7d83stUS7ykzXkWJ9N5jZyqTahjXQbkhiG1aZ2VU5qGu8mb1pZovNrNzMOjTQLifbq7Gf38zaJH7HVYl9qW+2aklaZ28ze9bMliT2/1+kaHOUma1P+v1el+p7ZaG27f5eLLg9sb0Wm9lBOahp36Tt8KqZfW5ml9Rrk7PtZWaTzGyNmb2eNK+TmT1lZssTrx0b+NpzEm2Wm9k5qdo0yt3zcgL2A/YFngPKkuYPBBYBbYB+wAqgRYqvnwqMTLyfAIzNcr3/D7iugWXvAp1zuO1uAC5vpE2LxLbbG2id2KYDs1zXcUDLxPvfAr+Ntb3S+fmBnwMTEu9HAlNy8LvrDhyUeN8eeCtFXUcBM3O1P6X7ewGGAbMBA34IvJzj+loAqwg33kTZXsD/AQ4CXk+adwtwVeL9Van2e6AT8HbitWPifcemrj9vj9Ddfam7L0uxaAQw2d03u/s7QBVhIOuvmZkB/5cwYDXA/cBJ2ao1sb7TgUeztY4s+Hrwb3evAeoG/84ad3/S3WsTH+cRRr+KJZ2ffwRh34GwLx2T+F1njbt/5O4LE+83AEsJY/YWghHAAx7MAzqYWfccrv8YYIW7N/cO9B3m7s8TxoRIlrwfNZRFxwNPufun7r4OeAoY0tT1522gb0eqQavr7/B7AJ8lhUeqNpl0BLDa3Zc3sNyBJ81sQWKg7FwYl/hv76QG/ouXznbMpvMJR3Op5GJ7pfPzf2vwc6Bu8POcSHTxHAi8nGLxoWa2yMxmm9l3c1RSY7+X2PvUSBo+qIqxvep0dfePEu9XAV1TtMnItsvpINH1mdnTQLcUi65x98dzXU8qadY4iu0fnf+Lu680sz2Bp8zszcRf8qzUBfwJuInwD/AmQnfQ+TuyvkzUVbe9zOwaoBZ4uIFvk/HtVWjMbFfgMeASd/+83uKFhG6FLxLnRyqA/jkoK29/L4lzZMOBq1MsjrW9tuHubmZZu1Y8aqC7+7HN+LJ0Bq3+hPDfvZaJI6tUbTJSo4VBsX8MHLyd77Ey8brGzMoJ/93foX8I6W47M7sbmJliUTrbMeN1mdm5wAnAMZ7oPEzxPTK+vVJoyuDn1ZbDwc/NrBUhzB929/+pvzw54N19lpndaWad3T2rD6FK4/eSlX0qTUOBhe6+uv6CWNsryWoz6+7uHyW6oNakaLOS0Ndfpxfh/GGTFGKXy3RgZOIKhH6Ev7SvJDdIBMWzhAGrIQxgna0j/mOBN929OtVCM9vFzNrXvSecGHw9VdtMqddveXID60tn8O9M1zUE+A9guLt/2UCbXG2vvBz8PNFHfy+w1N1vbaBNt7q+fDMbRPh3nNU/NGn+XqYDZyeudvkhsD6pqyHbGvxfcoztVU/yftRQFs0BjjOzjoku0uMS85omF2d+mzMRgqga2AysBuYkLbuGcIXCMmBo0vxZQI/E+70JQV8FTAPaZKnOPwM/qzevBzArqY5FiekNQtdDtrfdg8BrwOLEztS9fl2Jz8MIV1GsyFFdVYR+wlcT04T6deVye6X6+YEbCX9wANom9p2qxL60dw620b8QusoWJ22nYcDP6vYzYFxi2ywinFw+LAd1pfy91KvLgDsS2/M1kq5Oy3JtuxACevekeVG2F+GPykfAlkR+jSacd5kLLAeeBjol2pYB9yR97fmJfa0KOK8569et/yIiRaIQu1xERCQFBbqISJFQoIuIFAkFuohIkVCgi4gUCQW6iEiRUKCLiBSJ/w+7XlyOwpsRRAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出 sigmoid 的图像\n",
    "\n",
    "plot_x = np.arange(-10, 10.01, 0.01)\n",
    "plot_y = sigmoid(plot_x)\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_data = Variable(x_data)\n",
    "y_data = Variable(y_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 PyTorch 当中，不需要我们自己写 Sigmoid 的函数，PyTorch 已经用底层的 C++ 语言为我们写好了一些常用的函数，不仅方便我们使用，同时速度上比我们自己实现的更快，稳定性更好\n",
    "\n",
    "通过导入 `torch.nn.functional` 来使用，下面就是使用方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义 logistic 回归模型\n",
    "w = Variable(torch.randn(2, 1), requires_grad=True) \n",
    "b = Variable(torch.zeros(1), requires_grad=True)\n",
    "\n",
    "def logistic_regression(x):\n",
    "    return torch.sigmoid(torch.mm(x, w) + b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在更新之前，我们可以画出分类的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fd362245390>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXgT1frA8e9pKdQKl6XgBrTFK152EIsCbqCyiIoo6hUB9wtFEK8KCpZNBEXxiiIgIAJyQdzlh+woIpso5Yogm4Dsi0DZKUvbvL8/EjTUpkmaZSbp+3mePE1mJnPeTibvnDlncsaICEoppaJXjNUBKKWUCi1N9EopFeU00SulVJTTRK+UUlFOE71SSkW5YlYHkJ/y5ctLSkqK1WEopVTEWLly5UERqZDfPFsm+pSUFDIyMqwOQymlIoYxZrunedp0o5RSUU4TvVJKRTlN9EopFeVs2Uafn+zsbHbt2sXp06etDiXqxcfHU6lSJeLi4qwORSkVBF4TvTGmMjAJuBgQYKyIvJ1nGQO8DbQCsoBHROR/rnkPA31ciw4SkQ8KE+iuXbsoVaoUKSkpOItToSAiZGZmsmvXLqpUqWJ1OEqpIPCl6SYHeE5EagANga7GmBp5lrkNqOp6dALeBTDGlAP6A9cC1wD9jTFlCxPo6dOnSUxM1CQfYsYYEhMT9cxJqSjiNdGLyN5ztXMROQ6sByrmWewuYJI4LQfKGGMuBVoA80XkkIgcBuYDLQsbrCb58NDtrFR08asz1hiTAlwF/JBnVkVgp9vrXa5pnqbnt+5OxpgMY0zGgQMH/AlLKaUizqbMTfSY1wOHOEJels+J3hhTEvgc+LeIHAt2ICIyVkRSRSS1QoV8f9wVUbZt28aHH374x+tVq1Yxa9asP15Pnz6dIUOGBKWsRx55hM8++wyAJ554gnXr1gVlvUqp4DuTc4aB3w2k9ru1ee9/77Hx4MaQl+lTojfGxOFM8lNE5It8FtkNVHZ7Xck1zdP0qOct0bdu3ZpevXoFvdxx48ZRo0beLhSllB0s3LaQuqPr0n9hf9pUa8OGrhuoXqF6yMv1muhdV9S8D6wXkTc9LDYdeMg4NQSOisheYC7Q3BhT1tUJ29w1LSJNmjSJOnXqULduXTp27AicX5sGKFmyJAC9evVi8eLF1KtXj9dee41+/frx8ccfU69ePT7++GMmTpxIt27d/lhH9+7dady4MZdffvkf63M4HDz55JNUq1aNZs2a0apVq/PKyk+TJk3+GD6iZMmSpKenU7duXRo2bMjvv/8OwIEDB2jbti0NGjSgQYMGLF26NLgbSil1ngMnD/DwtIdp+kFTsh3ZzGk/h4/u/YhLS10alvJ9uY7+OqAjsMYYs8o17UUgCUBERgOzcF5auRnn5ZWPuuYdMsa8DKxwvW+giBwKNOh/z/k3q/at8r6gH+pdUo+3Wr7lcf7atWsZNGgQy5Yto3z58hw6VPC/MWTIEN544w1mzJgBwMUXX0xGRgYjRowAYOLEiectv3fvXpYsWcKGDRto3bo19957L1988QXbtm1j3bp17N+/n+rVq/PYY4/5/D+dPHmShg0bMnjwYJ5//nnee+89+vTpw9NPP80zzzzD9ddfz44dO2jRogXr16/3eb1KKd84xMGEnybw/NfPc+zMMV68/kX63NiHC+IuCGscXhO9iCwBCrwMQ5w3nu3qYd54YHyhorORBQsWcN9991G+fHkAypUrF9T1t2nThpiYGGrUqPFHzXvJkiXcd999xMTEcMkll9C0aVO/1lm8eHHuuOMOAK6++mrmz58PwNdff31eO/6xY8c4ceLEH2cjSqnArTuwjrQZaSzesZjrk65n9O2jqXlRTUtiiZhfxrorqOYdbsWKFcPhcPaaOxwOzp49W6j1lChR4o/nwbphe1xc3B+XSsbGxpKTkwM441y+fDnx8fFBKUcp9aes7CwGLxrM0GVDKVWiFOPuHMejVz1KjLFuxBkd68ZHN998M59++imZmZkAfzTdpKSksHLlSsB5JU12djYApUqV4vjx43+8P+9rX1x33XV8/vnnOBwOfv/9dxYuXBiE/wSaN2/OO++888frVauC2wymVFE1Z/Mcao2qxStLXqFd7XZs6LqBx+s/bmmSB030PqtZsybp6encdNNN1K1bl2effRaAf/3rX3z33XfUrVuX77//ngsvvBCAOnXqEBsbS926dRk2bBhNmzZl3bp1f3TG+qJt27ZUqlSJGjVq0KFDB+rXr0/p0qUD/l+GDx9ORkYGderUoUaNGowePTrgdSpVlO09vpd/fvZPbptyG3GxcSx4aAEftPmAChfa41JxE6xmgmBKTU2VvDceWb9+PdWrh/4yJLs513aemZnJNddcw9KlS7nkkktCXm5R3d5K+SPXkcuYlWPo/U1vzuSc4cUbXuSF616gRLES3t8cZMaYlSKSmt+8iGyjL0ruuOMOjhw5wtmzZ+nbt29YkrxSyrtV+1bReUZnftz9I7dUuYV3b3+XqolVrQ4rX5robS5Y7fJKqeA4cfYE/b/tz9s/vE1iQiKT757Mg7UftPUYUZrolVLKR9M3TqfbrG7sPLaTTvU7MeTWIZS9oFAD8oaVJnqllPJi59GdPDX7Kf5v4/9R66JaTG07leuSrrM6LJ9poldKKQ9yHDm888M79P22Lw5xMOSWITzb6FniYiPr7mua6JVSKh8/7v6RzjM6s2rfKlpVbcWI20ZQpWxk3nVNE71SSrk5evoo6QvSGbViFJeWupRP7/uUttXb2rqz1Zvo/cHUlCmQkgIxMc6/U6aEregPPviAqlWrUrVqVT74oFC3yFVKhZmI8MnaT6g+sjqjVoyi2zXdWN91PffWuDeikzxEa41+yhTo1Amyspyvt293vgZo3z6kRR86dIiXXnqJjIwMjDFcffXVtG7dmrJl7d8zr1RRtfXwVp6c9SRzNs+h/qX1md5uOqmX5fvbo4gUnTX69PQ/k/w5WVnO6YW0YsUK6tSpw+nTpzl58iQ1a9bkl19++ctyc+fOpVmzZpQrV46yZcvSrFkz5syZU+hylVKhk52bzZAlQ6g5qiZLdizhrRZv8cMTP0RVkodordHv2OHfdB80aNCA1q1b06dPH06dOkWHDh2oVavWX5bbvXs3lSv/eVOtSpUqsXt3kbipllIRZemOpXSe0Zm1B9Zyd7W7GX7bcCr9rZLVYYVEdCb6pCRnc01+0wPQr18/GjRoQHx8PMOHDw9oXUopaxw6dYgX5r/AuJ/GkVQ6iekPTOfOf9xpdVghFZ1NN4MHQ0LC+dMSEpzTA5CZmcmJEyc4fvw4p0+fzneZihUrsnPnzj9e79q1i4oVKwZUrlIqcCLCf3/+L9VGVGPCqgn0aNSDtU+ujfokD9Ga6Nu3h7FjITkZjHH+HTs24I7Yzp078/LLL9O+fXteeOGFfJdp0aIF8+bN4/Dhwxw+fJh58+bRokWLgMpVSgXm18xfufW/t/LQtIe4vOzlrOy0kqHNh1KyeNG4q1p0Nt2AM6kH8QqbSZMmERcXx4MPPkhubi6NGzdmwYIF3HzzzectV65cOfr27UuDBg0AZ3NPsG87qJTyzemc0wxZMoRXl7zKBcUuYFSrUXRO7Wz5jUDCTcejV/nS7a0i3YKtC+gyswu/Zv5Ku1rteLPFm1xSMnqH+Q5oPHpjzHjgDmC/iPzlMhNjTE/gXNW5GFAdqCAih4wx24DjQC6Q4ykIpZQKlv0n9/PcvOeYvHoyl5e9nLkd5tL8782tDstSvjTdTARGAJPymykiQ4GhAMaYO4FnROSQ2yJNReRggHHazpo1a+jYseN500qUKMEPP/xgUURKFW0OcTD+p/E8P/95Tpw9QfoN6aTfkM4FcRdYHZrlvCZ6EVlkjEnxcX3tgKmBBBQpateurTfVVsom1u5fS+cZnVm6cyk3Jt/I6NtHU72CNj2eE7QeCWNMAtAS+NxtsgDzjDErjTGdglWWUkoBZGVn0fvr3tQbU48NBzcwvvV4Fj68UJN8HsG86uZOYGmeZpvrRWS3MeYiYL4xZoOILMrvza4DQSeApAB/2KSUin6zN82m66yubD2ylUfqPcLQZkMpn1De6rBsKZjXGD1AnmYbEdnt+rsf+BK4xtObRWSsiKSKSGqFChWCGJZSKprsOb6H+z+9n1YftqJEsRIsfHghE+6aoEm+AEGp0RtjSgM3AR3cpl0IxIjIcdfz5sDAYJSnlCp6ch25vJvxLukL0jmTc4aXm75Mz8Y9KVGshNWh2Z7XGr0xZirwPfAPY8wuY8zjxpg0Y0ya22J3A/NE5KTbtIuBJcaYn4EfgZkiErZhHC0cjp6WLVtSpkwZ7rjjjvAVqlQU+2nvTzR6vxFPzX6Kayteyy9P/kKfG/tokveRL1fdtPNhmYk4L8N0n/YbULewgQXCwuHoAejZsydZWVmMGTMm9IUpFcWOnzlOv2/7MfzH4VRIqMCH93zIA7UeiPgbgYRbVP4OOATD0fs8Hj3ALbfcQqlSpQpfmFKKaRumUWNUDd764S061e/Ehm4baFe7nSb5QojKsW5CMBy9z+PRK6UCs+PoDp6a/RTTN06n9kW1+eTeT2hUuZHVYUW0qEz0IRqOXsejVyqEchw5vL38bfov7I9DHLx262s80/AZ4mLjrA4t4kVl002IhqP3aTx6pZT/ftz9Iw3ea0CP+T1oktKEdV3X8fx1z2uSD5KoTPQhGo7ep/HolVK+O3r6KF1ndqXhuIbsP7mfz+77jK/afUVKmRSrQ4sqUdl0A0Efjt7n8egBbrjhBjZs2MCJEyeoVKkS77//vt58RCk3IsInaz/h33P/zf6T++l+bXdebvoypUroRQyhELWJPtgeeughHnroIQBiY2MLHKVy8eLF4QpLqYiz5dAWnpz1JPO2zOPqS69mRrsZXH3Z1VaHFdU00SulwuJs7lmGLh3KoMWDiIuJ4+2Wb9O1QVdiY2KtDi3qaaIvJB2PXinfLd6+mLSZaaw7sI621dvydsu3qfi3ilaHVWREVKIXEdv8WCKax6O34+0lVWTKzMrk+fnPM37VeJJLJ/NVu6+440odGiTcIibRx8fHk5mZSWJiom2SfTQSETIzM4mPj7c6FBXBRIRJP0+ix/weHD51mOcbP0+/m/pxYfELrQ6tSIqYRF+pUiV27drFgQMHrA4l6sXHx1OpUiWrw1ARauPBjaTNTGPhtoU0qtSIMXeMofbFta0Oq0iLmEQfFxdHlSpVrA5DKeXB6ZzTvLr4VYYsHUJCXAKjbx/Nv67+FzEmKn+uE1EiJtErpezrm9++ocvMLmw6tIkHaz/Im83f5OKSF1sdlnLRRK+UKrT9J/fz3LznmLx6MleUu4J5HebR7O/NrA5L5aGJXinlN4c4GPe/cfT6uhcns0/S78Z+9L6hN/HFtBPfjjTRK6X8sub3NaTNTGPZzmXclHwTo+8YTbXy1awOSxVAE71SyidZ2VkM/G4g//n+P5QuUZqJd03koboP6eXOEUATvVLKq1mbZtF1Vle2HdnGo/UeZWizoSQmJFodlvKRJnqllEd7ju/h6TlP89m6z6hevjrfPfIdNybfaHVYyk+a6JVSf5HryOXdjHd58ZsXyXZkM6jpIHpe15PiscWtDk0VgtdfMhhjxhtj9htj8r0TtjGmiTHmqDFmlevRz21eS2PMRmPMZmNMr2AGrpQKjf/t/R8N32/IU7OfolHlRvzS5RfSb0zXJB/BfKnRTwRGAJMKWGaxiJw3UpExJhYYCTQDdgErjDHTRWRdIWNVSoXQ8TPH6fttX9758R0qJFRgatup/LPmP7WzNQp4TfQissgYk1KIdV8DbBaR3wCMMR8BdwGa6JWyERFh2oZpPDX7KfYc30Pnqzvz6q2vUia+jNWhqSAJ1iAUjYwxPxtjZhtjarqmVQR2ui2zyzUtX8aYTsaYDGNMhg5cplR4bD+yndYfteaeT+4hMSGRZY8v49073tUkH2WC0Rn7PyBZRE4YY1oB04Cq/q5ERMYCYwFSU1N1QHSlQig7N5u3f3ib/gv7AzC02VCevvZp4mLjLI5MhULAiV5Ejrk9n2WMGWWMKQ/sBiq7LVrJNU0pZaHlu5bTeUZnVv++mjuvvJN3bnuH5DLJVoelQijgRG+MuQT4XUTEGHMNzuagTOAIUNUYUwVngn8AeDDQ8pRShXPk9BFe/OZFRmeM5rJSl/HF/V/Qplob7WwtArwmemPMVKAJUN4YswvoD8QBiMho4F6gizEmBzgFPCDOe9HlGGO6AXOBWGC8iKwNyX+hlPJIRPjol494Zu4zHMg6wNPXPs3ApgMpVaKU1aGpMPHlqpt2XuaPwHn5ZX7zZgGzCheaUipQWw5t4clZTzJvyzxSL0tlVvtZ1L+0vtVhqTDTX8YqFYXO5p5l6NKhDFo8iLiYOIa3HM6TDZ4kNibW6tCUBTTRKxVlFm1fRNqMNNYfXM99Ne5jWIthVPybxyubVRGgiV6pKHEw6yDPz3+eCasmkFImhZkPzqRV1VZWh6VsQBO9UhFORJi4aiI95/fk6JmjvHDdC/S7qR8JcQlWh6ZsQhO9UhFsw8ENpM1I47vt39G4cmNG3z6a2hfXtjosZTOa6JWKQKeyT/HK4ld4belrXFj8QsbcMYYn6j9BjAnWqCYqmmiiVyrCfP3b13SZ2YXNhzbToU4H3mj2BheXvNjqsJSNaaJXKkL8fuJ3np33LB+u+ZAryl3B/I7zufXyW60OS0UATfRK2ZxDHLy38j16fdOLrOws+t3Yj9439Ca+WLzVoakIoYleKRtb/ftq0mak8f2u72mS0oTRt4/mH+X/YXVYKsJoolfKhk6ePclL373Em9+/SdkLyvJBmw/oWKejDkCmCkUTvVI2M+PXGXSb1Y3tR7fz+FWP89qtr5GYkGh1WCqCaaJXyiZ2H9tN9znd+WL9F9SoUINFjyzihuQbrA5LRQFN9EpZLNeRy8gVI+mzoA/ZjmxeufkVnmv8HMVji1sdmooSmuiVstDKPSvpPKMzK/eupMXfWzDq9lFcXvZyq8NSUUYTvVIWOHbmGH0X9GXEihFcdOFFfNT2I+6veb92tqqQ0ESvVBiJCF+s/4Luc7qz9/heuqR2YfAtgykTX8bq0FQU00SvVJhsO7KNbrO6MXPTTOpdUo8v7v+Caytda3VYqgjQRK9UiGXnZjNs+TBe+u4lDIb/NP8P3a/tTrEY/fqp8NA9TakQWrZzGWkz0lizfw13/eMuht82nKTSSVaHpYoYr2OaGmPGG2P2G2N+8TC/vTFmtTFmjTFmmTGmrtu8ba7pq4wxGcEMXCk7O3zqMJ2/6sx146/j8OnDTPvnNKY9ME2TvLKELzX6icAIYJKH+VuBm0TksDHmNmAs4N7w2FREDgYUpVIRQkSY+stUnpn7DAezDvJMw2d4qclLlCpRyurQVBHmNdGLyCJjTEoB85e5vVwOVAo8LKUiz+ZDm+kyswtf//Y1DS5rwJz2c7jq0qusDkupoLfRPw7MdnstwDxjjABjRGSspzcaYzoBnQCSkvT0VkWOMzlneH3p6wxePJgSxUow4rYRpKWmERsTa3VoSgFBTPTGmKY4E/31bpOvF5HdxpiLgPnGmA0isii/97sOAmMBUlNTJVhxKRVKC7ctJG1GGhszN3J/zfsZ1mIYl5W6zOqwlDpPUG4waYypA4wD7hKRzHPTRWS36+9+4EvgmmCUp5TVDmYd5JFpj9D0g6aczT3L7Paz+fjejzXJK1sKuEZvjEkCvgA6isivbtMvBGJE5LjreXNgYKDlKWUlEWHCqgn0nN+TY2eO0fv63vS5sQ8JcQlWh6aUR14TvTFmKtAEKG+M2QX0B+IARGQ00A9IBEa5xunIEZFU4GLgS9e0YsCHIjInBP+DUmGx7sA60maksXjHYq5Pup7Rt4+m5kU1rQ5LKa98ueqmnZf5TwBP5DP9N6DuX9+hVGQ5lX2KwYsH8/rS1ylZvCTj7hzHo1c9SowJSsunUiGnv4xVqgDztszjyZlPsuXwFjrW6cgbzd/gogsvsjospfyiiV6pfOw7sY9n5j7DR798xJWJV/LNQ99wc5WbrQ5LqULRRK+UG4c4GJMxht7f9OZUzin639SfXtf3Ir5YvNWhKVVomuiVcvl538+kzUxj+a7l3FzlZt69/V2uTLzS6rCUCpgmelXknTh7ggELB/DW8rcod0E5JrWZRIc6HfRuTypqaKJXRdpXG7+i2+xu7Di6g3/V/xdDbh1CuQvKWR2WUkGliV4VSTuP7qT7nO5M2zCNWhfVYsmjS7gu6Tqrw1IqJDTRqyIlx5HDOz+8Q7+F/ch15PLqLa/yXKPniIuNszo0pUJGE70qMlbsXkHnGZ35ad9P3HbFbYxsNZIqZatYHZZSIaeJXkW9o6eP0mdBH0auGMklJS/h43s/5r4a92lnqyoyNNGrqCUifL7+c7rP7s6+E/vo2qArg24eROn40laHplRYaaJXUWnr4a10ndWV2ZtnU++Sekx7YBrXVNRRslXRpIleRZXs3Gz+8/1/GPjdQGJMDMNaDKPbNd0oFqO7uiq6dO9XUWPpjqWkzUzjl/2/0KZaG4a3HE7l0pWtDkspy2miVxHv0KlD9Pq6F+/97z0q/60y//fA/9H6H62tDksp29BEryKWiDBlzRSenfssh04d4rlGzzGgyQBKFi9pdWhK2YomehWRNmVuosvMLnyz9RuurXgt8zvOp+4lep8bpfKjiV5FlDM5Z3ht6Wu8svgV4ovFM6rVKDpd3YnYmFirQ1PKtjTRq4ixcNtC0maksTFzIw/UeoBhLYZxSclLrA5LKdvTRK9s78DJA/SY34NJP0/i8rKXM6f9HFpc0cLqsJSKGJrolW05xMHEVRPpOb8nx88cJ/2GdNJvSOeCuAusDk2piOLTbeyNMeONMfuNMb94mG+MMcONMZuNMauNMfXd5j1sjNnkejwcrMCLhClTICUFYmKcf6dMsTqisFm7fy1NJjbh8emPU7NCTValrWLQzYM0yStVCD4lemAi0LKA+bcBVV2PTsC7AMaYckB/4FrgGqC/MaZsYYMtUqZMgU6dYPt2EHH+7dQp6MnebseSrOws0r9Jp96Yeqw9sJb3W7/PwkcWUqNCDWsDsxm7fW7K5kTEpweQAvziYd4YoJ3b643ApUA7YIyn5Tw9rr76ainykpNFnCn+/EdyctCKmDxZJCHh/NUnJDinW2H2ptlS5a0qwgDk4S8flv0n9lsTSCFMnuz8aIxx/g3lNrTb5xbO/115BmSIp/ztacZfFiw40c8Arnd7/Q2QCvQA+rhN7wv08LCOTkAGkJGUlBSO7WJvxuSf6I0JWhFhOJb4ZM+xPfLPT/8pDED+8c4/5Nut34Y3gEI6l+DOfSzhSrz+fm6hTMR2O+gUZQUlel+bbkJORMaKSKqIpFaoUMHqcMIv77l4OQ/3LU1KClqRO3b4Nz3Ych25jFoximojqzFtwzQGNhnIz2k/0ySlSXgCCIB7yxo4U5y7rCxITw9N2f58bqFuAUxPd/6v7kL5v6vCCVai3w24jx5VyTXN03TlLr9v4/HjEJfn9nYJCTB4cNCK9XTMCOKxxKNV+1bReHxjus7qyjUVr2FNlzX0vakvJYqVCH3hQZBfgssrVAdMfz63UCdiqysLyjfBSvTTgYdcV980BI6KyF5gLtDcGFPW1Qnb3DVNucvv23j2LPztb5CcDMY4/44dC+3bB63YwYOdxw53QT6W/MWJsyfoMa8HqWNT2XZkG1PumcK8DvOomlg1dIWGgC+JLFQHTH8+t1AnYisrC8oPntp03B/AVGAvkA3sAh4H0oA013wDjAS2AGuAVLf3PgZsdj0e9aW8ItcZG4b2eE/C2ZE2bf00qfxmZWEA0ml6JzmUdSh0hYWYp3bycLVT5/e55Tct1P0w2kZvHwSjMzacjyKX6O3SKxoiO47skDYftREGILVG1ZKlO5ZaHVLA8ktw547XVlx54inhdukS+kRcUGVBr8gJH030dhel1aLs3Gx5c9mbcuHgC+WCQRfIa0tek7M5Z60OK2jslMQKqitYFefkySJxcefHExcX8bu1bWmijwR2yhpB8MOuH6Te6HrCAOT2KbfL1sNbrQ4pqgWz9S9Yu2JiYv4xJSYWbn2qYAUlettcXlnktW8P27aBw+H8G8RO13A6evoo3WZ1o+G4huw/uZ/P7vuMr9p9RUqZlJCXXZR/LRqMTtEpU6B8eejQITiXY2Zm+jddhY4mehUUIsInaz+h+sjqvJvxLk9d8xTru66nbY22GGNCnoTDNGKEbQV6BdW57ZdfEi4q18VHdUXBU1XfykeRbLqJYFsObZGWk1sKA5D6Y+rLit0rzpsfji6IKO/P9kkgTS7eriIqTCeznZtu8m6rcHRahxraRq8Kw1viOJNzRl5Z9IrED4qXkq+UlLe+f0uyc7P/sp5wJGELr1CNCp62XyCXjU6eLFK8+PnvL17c+uRZ0BVTobwMNdRdcJrold+81cKXbF8iNUfWFAYg93x8j+w8utPjusKRhMNxMImy/vLz+FKjL8w2teM28+d/DcY+Gq6L6jTRK795+jJUqpwjT/zfE8IAJGlYkkzfML3Q6wp2Eg7ll8kOV8CGMml26eJ7rT7Sz5J8/T+DtY+Gq1lRE73ym+cvQ67EvhQrPeb2kBNnTvi0rnAlyVAmQqv7AEK5DfNbd6iTn5U8fZahGoE0XM2KmuiV3zx9GYqX2yOr9q7ye312PIX3h9V9AKE80PjTlBFpHZT5KehXxKHYR7VGr4netpxfBsf5ST7+rPx3cq7VoVnC6hp9KA80vjZlROIB2pNovFFMQYler6NX+bq08QJKtX0OSm8D46BS5RzGj4ujQ/uiuctYMdKnu1COEultHQkJMHlyRP+O7y/C+fvE9u2dA8+GcCBar4rmt1Z5tP/kfjp+2ZFbJt1CydTpzF35K+KIYeeOYlHzJS+MQL+sgf4YJ5QHmvzWbYzzrxVJKRpZ/sN3T1V9Kx/adBN+uY5ceW/le1J2SFmJGxgnfb7pI1lns6wOKyoE69Q91LcEjOQ+FFVw041xzreX1NRUycjIsDqMImPt/rV0ntGZpTuXcmPyjYy+fTTVK1S3OvUbDtgAABTuSURBVKyokZLy5y0H3SUnO2t3SgWDMWaliKTmN0+bboqwrOwsen/dm3pj6rHh4AbGtx7PwocXOpN8VA/8EV5W3W5PP0J1jib6SBHkb+3sTbOpNaoWQ5YOoUOdDmzotoFHr3oUY4yOEBZkVtxuTz9C5U4TfSQI4rd2z/E93P/p/bT6sBUlipXg24e/ZcJdEyifUP7PhUJ9R+kiplUr/6YHQyR8hJF4xhGJMQPaGRsRgnARd05ujrzzwzvy2P3xsq004jCIIykp/143q38dJNZ3DgazfCuuwbfBR1ggOwwp4S+7x4z+YCqCTZ6c/zfWj2/tyj0rpcHYBtLuHuRU8Rjve6rFvw6y+gsV7PKtSLpWfIT+HByt/gFaYdg95oATPdAS2AhsBnrlM38YsMr1+BU44jYv123edF/K00Tv4m0QEi972LHTx+Tfs/8tMS/FyMVDL5YTl5b3bT0WZ1qrv1DBLt+qpBvOj9Dfm6Xb/YwjP3aPOaBED8QCW4DLgeLAz0CNApZ/Chjv9vqEtzLyPjTRuxQ0CImXb+2X67+USm9WEgYgnb/qLIeyDvm3p1rYdmL1F6qg8guzWaw6bobzI/Q2Xk7e/9fqg3lh2D3mQBN9I2Cu2+veQO8Cll8GNHN7rYm+sAoahMTDCEzbj2yX1lNbCwOQ2qNqy7Idy/5cn933VBerw/RUfmLiXxN28eLO6d6SqdV9DqHmy3g57p+f1c1zhRHqEUQD3T8CTfT3AuPcXncERnhYNhnYC8S6TcsBMoDlQJsCyunkWi4jKSmpEJsqCvmRcRwJCTKjfzu5cPCFkjA4QV5f8rqczTl7/voi5NtldZieyvd0azybb86w8GUEzLxnZJF48AtFzMHa38OZ6F8A3skzraLr7+XANuDv3srUGr2Lnxlna2nk9im3y7bD2wpe57k9NTHRt+qoBaxOAvmV788oj0WNL2PaF8Xt4otgncGGrekG+AloXMC6JgL3eitTE70bPzKOwxhxOBy+rzcCavd24u/t9iJlU/pzUC1o2XPzztXe7bJrWV1p8CZYfVKBJvpiwG9AFbfO2Jr5LFfNVWM3btPKAiVcz8sDmwrqyD330ERfMEdSUuBVAKsbwgvJyi+tv3diioTjpj/He3+urLFLcrVLfaag7WGLGr3z/bRyXTa5BUh3TRsItHZbZgAwJM/7GgNrXAeHNcDjvpSnid6zzZmb5dW0WnIiLsCsYvWlLYVghy9t3pavuDjvNXs78yfJ+HtljR3YoT5T0H47eXL+LbFhb6O34qGJ/q/O5JyRwYsGS/ygeCn1SimZPaCjs2Zf2CqTHb4BfrJjyO7NFRF23BQR/473/l5ZYwd2qM/4cxXXuelhv+rGiocm+vMt2rZIaoysIQxA2n7cVnYd3RX4SiNhkPQ87PCl9STYB6FwbdZg1ujt8lm4s0PlwNdO/EBj00QfoQ6ePCiPTXtMGIAkD0uWGRtnBLeAQLNJmNtS7PCl9SSYmyKcmzXQNno7fhbuPMVc2Fqz+3oDHe4h2AdLTfQRxuFwyMSfJkr518tL7Eux0nNeTzlx5kRgKw1FFTHMmbewCTBcteNglRPuA1phrro5l5DCdIwPSDDbwc+tz5/90N/fZWiNvgjYcGCDNJnYRBiANBrXSFbvWx34SkNVRbSgLcXfZGqHDlx/2bmJyp1drqzxRTAPnoVZV37bKtj7pib6CHAq+5T0XdBXir9cXMoMKSNjMsZIriM3OCsPVRXRqtG6/Mgudm7u8SQSY7a7YB48g7muYB4sNdHb3Pwt8+WK4VcIA5D2n7eXfcf3BbeAUFURw11dLkR5kVI7dheJZyF2Z3WNPhw00dvUvuP7pP3n7YUByBXDr5D5W+aHpqBQ7pnhPH8vxP9h1y+lN5HULBIJIrWz3B+a6G0m15Ero1eMljJDykjcwDjpu6CvnMo+5fkNEXZ1TMgUdJ2aB5H2r2uCD51gbttgrUubbqLU6n2rpdG4RsIA5KYJN8n6A+sLfkMEXu9eaN5i9FQ9PzdQfCFXaxeRdlBSgdHO2ChM9CfOnJDn5z0vxQYWk8TXEmXCTxN8G4AsUtse/OXLXj95sudafSG3h50OAkXlo1ZOwf68C0r0xjnfXlJTUyUjI8PqMIJm1qZZPDnzSbYf3c5j9R7j9Wavk5iQ6NubY2Kcn39exoDDEdxArZSSAtu3/3V6cjJs2/bna2Pyf38htseUKdCpE2Rl/TktIQHGjoX27f1aVVAUlY9aOQX78zbGrBSR1HzL8n91yle7j+3m3k/u5fYPbychLoHvHvmO9+963/ckD5CU5N/0SLVjh2/Tk5PzX64Q2yM9/fwkD87X6el+ryooispHXVhTpjjrAzExzr9TplgdUWDC+Xlrog+BXEcuw38YTvWR1Zm5aSaDmg5iVdoqbky+0f+VDR7srGa6S0hwTreTQL+Fvu71Qdwevh5bwiVSPupQy29XOnf2tX27sxa8fbvzdSQn+7B+3p7adKx8RHIb/co9KyV1bKowAGn+3+ayOXNz4Cu1U0NyfoLRq+TvoCtB2B52bBO3+0cdauEaLsAu9KqbCHPs9DF5evbTEvNSjFw89GKZumaq73d7inTByphhznJ6lYv9hGsAsGhUUKLXztgAiQhfbviS7rO7s+f4HtJS03jlllcoE1/G6tDCJ4J7EadMcbbJ79jhbCUaPNiajljl5GlX8iRvX31RVlBnbLFwBxNNth/ZTrfZ3Zjx6wzqXlyXz+//nGsrXWt1WOGXlJT/FTMR0IvYvr0mdjvxtCslJsKpU3+9Qqqo9V8UlnbGFkJ2bjZvLHuDGqNqsGDrAt5o9gYZnTKKZpIHe/YiWnGJRpjKjLarT9x52pXeftt52WtysvNEMTnZustgI5KnNh0rH3Zuo/9+5/dS5906wgDkzg/vlO1Htlsdkj0E0r4e7LZ5Kxrfw1RmUehXKOod0oWFttEH7sjpI/T+ujdjVo7hslKX8c5t79CmWhuMpx/wKN+E4ldLvv74KpjCVKYV/5qKDAH/YMoY09IYs9EYs9kY0yuf+Y8YYw4YY1a5Hk+4zXvYGLPJ9Xi48P+GNUSEqWumUm1ENcb+byxPX/s067uu5+7qd/uW5KP5PDsYPP1q6eGHC7/Nwn2B/JQp+WffEJRpt2v/VYTwVNU/9wBigS3A5UBx4GegRp5lHgFG5PPecsBvrr9lXc/LeivTLk03mzI3SbNJzYQBSOrYVFm5Z6V/KygK59mB8uXOycaIdOni+zrDeYG8txupBrlMO177r+yBAppufKnRXwNsFpHfROQs8BFwl4/HkRbAfBE5JCKHgflASx/fa5kzOWcYvGgwtUbVYvmu5Yy4bQTLH19O/Uvr+7ciu/3G3o58uTJHBEaP9r1mH87O4fw+4xCWacd+72DSE+AQ8XQEOPcA7gXGub3uSJ7aO84a/V5gNfAZUNk1vQfQx225vkAPb2VaWaNfuHWhVBtRTRiA3PfJfbL72O7CrywUtzeKtp4qbzXiwlZbw7WdCjojCVGZVu0CoS5XT4ADQyC/jPUx0ScCJVzPOwMLxM9ED3QCMoCMpKSk8GwZNwdOHpBHpz0qDEBS3kqRWb/OCnylwT7PjtZvgnsGiY313oxjpwNcONpSwpjZPRUVjl1Pm6UCE2iibwTMdXvdG+hdwPKxwFHX83bAGLd5Y4B23soMZ43e4XDIhJ8mSOJriVJsYDHpNb+XnDx7MjgrD/a3oyh8Ewoac96OB7hQZ8AwHtwLKiocu1447u9rx7tMBUugib4Yzk7UKvzZGVszzzKXuj2/G1juel4O2IqzI7as63k5b2WGK9GvP7BebppwkzAAafx+Y1m9b3XwCwnm3hCJd7oujC5dfEv2djnAhfIbH8aDe0FFFWbX83ezhPpfjfb7xgaU6J3vpxXwK86rb9Jd0wYCrV3PXwXWug4C3wLV3N77GLDZ9XjUl/JCneizzmZJ3wV9JW5gnJQdUlbGZoyVXEdu6AoM1o+JPDVr2CXhBdPkyb4140S7MB7cCyrK3yRcmEQY6uQZzAOJHU+uA0704X6EMtHP2zxP/v7234UBSIcvOsjvJ34PWVki4vvem9/BwJeOSqurEaHkrVYfjQe4cwpqL7GgRu9vEi5sInT/GiQmOh/BOlEqaHfyd/12PLnWRC8i+47vkwc/f1AYgFQdXlW+3vJ10MvIly97vL+DcMfG2qdhMJQKSnRWHeDC0TDr7QBvQRv9ufm+/us+JcICVhiK2r23IZD9Wb/W6G2W6HMduTJ6xWgp/WppKf5ycen/bX85lX0qaOv3ypc9Xgfhzp+nhJeYaF2SD3XHq7d9IZQH98mTZXLiU5LMVjHkSnLi8dBdN+BlW4Yikfpyguzr+qOyjT7cj2Al+p/3/SyNxjUSBiBNJzaVDQc2BGW9fvFlj/Wl49Eu1YZws9OlDaGsxvmShUJ5gA9y5vK6Ok/bMjFRRELXNOLtWOrP+u20a4oUwUR/4swJ6Tmvp8S+FCsVXq8gk1ZNsu5uT758gQra6e1WbSjKQtkw68tZXSgP8CE4iBWYCL380CzUTSN2bHoJVJFK9F9t/EqShyULA5An/u8JyczKLPS6gsbbob+gg4Hdqg2RLpDtGcrs4O2sLtQH+HD3LnrpaLbVzxMi5DtYJBL9kVNH5J6P7xEGIDVH1pTF2xf7vQ5LRcjOFNECzR6hzD7errAJ9f4Q7iru5Mme/1/XwSUcQy54Xb8/n7nF3+EikehzcnPkxgk3yiuLXpEzOWf8fr8qAoKRzEL1Zba6d8+K8j1dVWan9hNf9xmrPz8pIoleREL7oycV+ex48bM7q8/qwl2+DZKjV77uMzZo9C8o0esdplT0mzLFOZywp5uD6O2ZrHPus9mxwzlk9eDB9roRrK+39IqJcab2vIwBhyNU0eUpKsA7TCkVsc7dqtBTko+mwdwjUfv2zoTpcDj/2inJg+83APB0XwVf7rcQBproVXQr6MYgycmB3ZtWRb/27Z37SHKys3buaZ+x+R1htOlGRTcbnFKrIsLiZqiCmm6KhS0KpayQlJR/s41NTqlVFGnf3rZnh9p0o6KbzU+plQoHTfQquvnaxqqc9O7cUUmbblT0s/Epta2cu0LpXOf19u3O16DbL8JpjV4p5ZTfFUpZWc7pKqJpoldKOe3Y4d90FTE00SulnGz+ox9VeJrolVJOeoVS1NJEr5Ry0iuUopZPid4Y09IYs9EYs9kY0yuf+c8aY9YZY1YbY74xxiS7zcs1xqxyPaYHM3ilVJDZfewZVSheL680xsQCI4FmwC5ghTFmuoisc1vsJyBVRLKMMV2A14F/uuadEpF6QY5bKaWUj3yp0V8DbBaR30TkLPARcJf7AiLyrYicuy5rOVApuGEqpZQqLF8SfUVgp9vrXa5pnjwOzHZ7HW+MyTDGLDfGtPH0JmNMJ9dyGQcOHPAhLKWUUr4I6i9jjTEdgFTgJrfJySKy2xhzObDAGLNGRLbkfa+IjAXGgnP0ymDGpZRSRZkvNfrdQGW315Vc085jjLkVSAdai8iZc9NFZLfr72/AQuCqAOJVSinlJ6/j0RtjigG/ArfgTPArgAdFZK3bMlcBnwEtRWST2/SyQJaInDHGlAe+B+7K05GbX5kHAA+3BPKqPHCwkO8NJY3LPxqXfzQu/0RjXMkiUiG/GV6bbkQkxxjTDZgLxALjRWStMWYgzpvRTgeGAiWBT40xADtEpDVQHRhjjHHgPHsY4i3Ju8rMN1hfGGMyPA2+byWNyz8al380Lv8Utbh8aqMXkVnArDzT+rk9v9XD+5YBtQMJUCmlVGD0l7FKKRXlojHRj7U6AA80Lv9oXP7RuPxTpOKy5c3BlVJKBU801uiVUkq50USvlFJRLiITfSCjadogtjRjzBrXaJ5LjDE17BCX23JtjTFijAnLpWc+bK9HjDEH3EZAfcIOcbmWud+1n601xnxoh7iMMcPcttWvxpgjNokryRjzrTHmJ9f3spVN4kp25YjVxpiFxpiQj9NljBlvjNlvjPnFw3xjjBnuinm1MaZ+wIWKSEQ9cF7LvwW4HCgO/AzUyLNMUyDB9bwL8LGNYvub2/PWwBw7xOVarhSwCOfAdKl2iAt4BBhhw32sKs5RW8u6Xl9kh7jyLP8Uzt+9WB4Xzk7GLq7nNYBtNonrU+Bh1/Obgf+GIa4bgfrALx7mt8I5XpgBGgI/BFpmJNbo7Tyapi+xHXN7eSEQjt5wr3G5vAy8BpwOQ0z+xBVuvsT1L2CkiBwGEJH9NonLXTtgqk3iEuBvruelgT02iasGsMD1/Nt85gediCwCDhWwyF3AJHFaDpQxxlwaSJmRmOgDHU0zlHyKzRjT1RizBee4/d3tEJfr9LCyiMwMQzw+x+XS1nUK+5kxpnI+862I60rgSmPMUtfIrC1tEhfgbJIAqvBnErM6rgFAB2PMLpw/vnzKJnH9DNzjen43UMoYkxiG2Arib47zKhITvc/cRtMcanUs7kRkpIj8HXgB6GN1PMaYGOBN4DmrY8nHV0CKiNQB5gMfWBzPOcVwNt80wVlzfs8YU8bSiM73APCZiORaHYhLO2CiiFTC2TTxX9d+Z7UewE3GmJ9wjrq7G7DLNgsaO2xofwU0mqYdYnPzEeBxjP4g8hZXKaAWsNAYsw1nu+D0MHTIet1eIpLp9vmNA64OcUw+xYWzljVdRLJFZCvOgf+q2iCucx4gPM024FtcjwOfAIjI90A8zgG8LI1LRPaIyD0ichXOfIGIhKUDuwD+5hHvQt3xEIKOjGLAbzhPS891sNTMs8xVODthqtowtqpuz+/EOTCc5XHlWX4h4emM9WV7Xer2/G5guU3iagl84HpeHuepdqLVcbmWqwZsw/WDSJtsr9nAI67n1XG20Yc0Ph/jKg/EuJ4PBgaGaZul4Lkz9nbO74z9MeDywvFPhWAjtcJZg9oCpLumDcRZewf4GvgdWOV6TLdRbG8Da11xfVtQwg1nXHmWDUui93F7veraXj+7tlc1m8RlcDZ3rQPWAA/YIS7X6wE4R4oNyz7v4/aqASx1fY6rgOY2ieteYJNrmXFAiTDENBXYC2TjPDN8HEgD0tz2rZGumNcE47uoQyAopVSUi8Q2eqWUUn7QRK+UUlFOE71SSkU5TfRKKRXlNNErpVSU00SvlFJRThO9UkpFuf8HxJJmryxonWYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出参数更新之前的结果 (FIXME: the plot is wrong)\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0.numpy() * plot_x - b0.numpy()) / w1.numpy()\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到分类效果基本是混乱的，我们来计算一下 loss，公式如下\n",
    "\n",
    "$$\n",
    "loss = -\\{ y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}) \\}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 计算loss, 使用clamp的目的是防止数据过小而对结果产生较大影响。\n",
    "def binary_loss(y_pred, y):\n",
    "    logits = (y * y_pred.clamp(1e-12).log() + \\\n",
    "              (1 - y) * (1 - y_pred).clamp(1e-12).log()).mean()\n",
    "    return -logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意到其中使用 `.clamp`，这是[文档](http://pytorch.org/docs/0.3.0/torch.html?highlight=clamp#torch.clamp)的内容，查看一下，并且思考一下这里是否一定要使用这个函数，如果不使用会出现什么样的结果\n",
    "\n",
    "**提示：查看一个 log 函数的图像**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6412, grad_fn=<NegBackward>)\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_regression(x_data)\n",
    "loss = binary_loss(y_pred, y_data)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到 loss 之后，我们还是使用梯度下降法更新参数，这里可以使用自动求导来直接得到参数的导数，感兴趣的同学可以去手动推导一下导数的公式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'NoneType' object has no attribute 'data'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-37-8e380a74b61d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# 自动求导并更新参数\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m     \u001b[0mw\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrad\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzero_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m     \u001b[0mb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrad\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzero_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAttributeError\u001b[0m: 'NoneType' object has no attribute 'data'"
     ]
    }
   ],
   "source": [
    "# 自动求导并更新参数\n",
    "for i in range(10):\n",
    "    w.grad.data.zero_()\n",
    "    b.grad.data.zero_()\n",
    "    \n",
    "    # calc grad\n",
    "    loss.backward()\n",
    "    w.data = w.data - 0.1 * w.grad.data\n",
    "    b.data = b.data - 0.1 * b.grad.data\n",
    "\n",
    "    # 算出一次更新之后的loss\n",
    "    y_pred = logistic_regression(x_data)\n",
    "    loss = binary_loss(y_pred, y_data)\n",
    "    print(loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fd3621cf9b0>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXgT1frA8e9pKdQKl6XgBrTFK152EIsCbqCyiIoo6hUB9wtFEK8KCpZNBEXxiiIgIAJyQdzlh+woIpso5Yogm4Dsi0DZKUvbvL8/EjTUpkmaZSbp+3mePE1mJnPeTibvnDlncsaICEoppaJXjNUBKKWUCi1N9EopFeU00SulVJTTRK+UUlFOE71SSkW5YlYHkJ/y5ctLSkqK1WEopVTEWLly5UERqZDfPFsm+pSUFDIyMqwOQymlIoYxZrunedp0o5RSUU4TvVJKRTlN9EopFeVs2Uafn+zsbHbt2sXp06etDiXqxcfHU6lSJeLi4qwORSkVBF4TvTGmMjAJuBgQYKyIvJ1nGQO8DbQCsoBHROR/rnkPA31ciw4SkQ8KE+iuXbsoVaoUKSkpOItToSAiZGZmsmvXLqpUqWJ1OEqpIPCl6SYHeE5EagANga7GmBp5lrkNqOp6dALeBTDGlAP6A9cC1wD9jTFlCxPo6dOnSUxM1CQfYsYYEhMT9cxJqSjiNdGLyN5ztXMROQ6sByrmWewuYJI4LQfKGGMuBVoA80XkkIgcBuYDLQsbrCb58NDtrFR08asz1hiTAlwF/JBnVkVgp9vrXa5pnqbnt+5OxpgMY0zGgQMH/AlLKaUizqbMTfSY1wOHOEJels+J3hhTEvgc+LeIHAt2ICIyVkRSRSS1QoV8f9wVUbZt28aHH374x+tVq1Yxa9asP15Pnz6dIUOGBKWsRx55hM8++wyAJ554gnXr1gVlvUqp4DuTc4aB3w2k9ru1ee9/77Hx4MaQl+lTojfGxOFM8lNE5It8FtkNVHZ7Xck1zdP0qOct0bdu3ZpevXoFvdxx48ZRo0beLhSllB0s3LaQuqPr0n9hf9pUa8OGrhuoXqF6yMv1muhdV9S8D6wXkTc9LDYdeMg4NQSOisheYC7Q3BhT1tUJ29w1LSJNmjSJOnXqULduXTp27AicX5sGKFmyJAC9evVi8eLF1KtXj9dee41+/frx8ccfU69ePT7++GMmTpxIt27d/lhH9+7dady4MZdffvkf63M4HDz55JNUq1aNZs2a0apVq/PKyk+TJk3+GD6iZMmSpKenU7duXRo2bMjvv/8OwIEDB2jbti0NGjSgQYMGLF26NLgbSil1ngMnD/DwtIdp+kFTsh3ZzGk/h4/u/YhLS10alvJ9uY7+OqAjsMYYs8o17UUgCUBERgOzcF5auRnn5ZWPuuYdMsa8DKxwvW+giBwKNOh/z/k3q/at8r6gH+pdUo+3Wr7lcf7atWsZNGgQy5Yto3z58hw6VPC/MWTIEN544w1mzJgBwMUXX0xGRgYjRowAYOLEiectv3fvXpYsWcKGDRto3bo19957L1988QXbtm1j3bp17N+/n+rVq/PYY4/5/D+dPHmShg0bMnjwYJ5//nnee+89+vTpw9NPP80zzzzD9ddfz44dO2jRogXr16/3eb1KKd84xMGEnybw/NfPc+zMMV68/kX63NiHC+IuCGscXhO9iCwBCrwMQ5w3nu3qYd54YHyhorORBQsWcN9991G+fHkAypUrF9T1t2nThpiYGGrUqPFHzXvJkiXcd999xMTEcMkll9C0aVO/1lm8eHHuuOMOAK6++mrmz58PwNdff31eO/6xY8c4ceLEH2cjSqnArTuwjrQZaSzesZjrk65n9O2jqXlRTUtiiZhfxrorqOYdbsWKFcPhcPaaOxwOzp49W6j1lChR4o/nwbphe1xc3B+XSsbGxpKTkwM441y+fDnx8fFBKUcp9aes7CwGLxrM0GVDKVWiFOPuHMejVz1KjLFuxBkd68ZHN998M59++imZmZkAfzTdpKSksHLlSsB5JU12djYApUqV4vjx43+8P+9rX1x33XV8/vnnOBwOfv/9dxYuXBiE/wSaN2/OO++888frVauC2wymVFE1Z/Mcao2qxStLXqFd7XZs6LqBx+s/bmmSB030PqtZsybp6encdNNN1K1bl2effRaAf/3rX3z33XfUrVuX77//ngsvvBCAOnXqEBsbS926dRk2bBhNmzZl3bp1f3TG+qJt27ZUqlSJGjVq0KFDB+rXr0/p0qUD/l+GDx9ORkYGderUoUaNGowePTrgdSpVlO09vpd/fvZPbptyG3GxcSx4aAEftPmAChfa41JxE6xmgmBKTU2VvDceWb9+PdWrh/4yJLs513aemZnJNddcw9KlS7nkkktCXm5R3d5K+SPXkcuYlWPo/U1vzuSc4cUbXuSF616gRLES3t8cZMaYlSKSmt+8iGyjL0ruuOMOjhw5wtmzZ+nbt29YkrxSyrtV+1bReUZnftz9I7dUuYV3b3+XqolVrQ4rX5robS5Y7fJKqeA4cfYE/b/tz9s/vE1iQiKT757Mg7UftPUYUZrolVLKR9M3TqfbrG7sPLaTTvU7MeTWIZS9oFAD8oaVJnqllPJi59GdPDX7Kf5v4/9R66JaTG07leuSrrM6LJ9poldKKQ9yHDm888M79P22Lw5xMOSWITzb6FniYiPr7mua6JVSKh8/7v6RzjM6s2rfKlpVbcWI20ZQpWxk3nVNE71SSrk5evoo6QvSGbViFJeWupRP7/uUttXb2rqz1Zvo/cHUlCmQkgIxMc6/U6aEregPPviAqlWrUrVqVT74oFC3yFVKhZmI8MnaT6g+sjqjVoyi2zXdWN91PffWuDeikzxEa41+yhTo1Amyspyvt293vgZo3z6kRR86dIiXXnqJjIwMjDFcffXVtG7dmrJl7d8zr1RRtfXwVp6c9SRzNs+h/qX1md5uOqmX5fvbo4gUnTX69PQ/k/w5WVnO6YW0YsUK6tSpw+nTpzl58iQ1a9bkl19++ctyc+fOpVmzZpQrV46yZcvSrFkz5syZU+hylVKhk52bzZAlQ6g5qiZLdizhrRZv8cMTP0RVkodordHv2OHfdB80aNCA1q1b06dPH06dOkWHDh2oVavWX5bbvXs3lSv/eVOtSpUqsXt3kbipllIRZemOpXSe0Zm1B9Zyd7W7GX7bcCr9rZLVYYVEdCb6pCRnc01+0wPQr18/GjRoQHx8PMOHDw9oXUopaxw6dYgX5r/AuJ/GkVQ6iekPTOfOf9xpdVghFZ1NN4MHQ0LC+dMSEpzTA5CZmcmJEyc4fvw4p0+fzneZihUrsnPnzj9e79q1i4oVKwZUrlIqcCLCf3/+L9VGVGPCqgn0aNSDtU+ujfokD9Ga6Nu3h7FjITkZjHH+HTs24I7Yzp078/LLL9O+fXteeOGFfJdp0aIF8+bN4/Dhwxw+fJh58+bRokWLgMpVSgXm18xfufW/t/LQtIe4vOzlrOy0kqHNh1KyeNG4q1p0Nt2AM6kH8QqbSZMmERcXx4MPPkhubi6NGzdmwYIF3HzzzectV65cOfr27UuDBg0AZ3NPsG87qJTyzemc0wxZMoRXl7zKBcUuYFSrUXRO7Wz5jUDCTcejV/nS7a0i3YKtC+gyswu/Zv5Ku1rteLPFm1xSMnqH+Q5oPHpjzHjgDmC/iPzlMhNjTE/gXNW5GFAdqCAih4wx24DjQC6Q4ykIpZQKlv0n9/PcvOeYvHoyl5e9nLkd5tL8782tDstSvjTdTARGAJPymykiQ4GhAMaYO4FnROSQ2yJNReRggHHazpo1a+jYseN500qUKMEPP/xgUURKFW0OcTD+p/E8P/95Tpw9QfoN6aTfkM4FcRdYHZrlvCZ6EVlkjEnxcX3tgKmBBBQpateurTfVVsom1u5fS+cZnVm6cyk3Jt/I6NtHU72CNj2eE7QeCWNMAtAS+NxtsgDzjDErjTGdglWWUkoBZGVn0fvr3tQbU48NBzcwvvV4Fj68UJN8HsG86uZOYGmeZpvrRWS3MeYiYL4xZoOILMrvza4DQSeApAB/2KSUin6zN82m66yubD2ylUfqPcLQZkMpn1De6rBsKZjXGD1AnmYbEdnt+rsf+BK4xtObRWSsiKSKSGqFChWCGJZSKprsOb6H+z+9n1YftqJEsRIsfHghE+6aoEm+AEGp0RtjSgM3AR3cpl0IxIjIcdfz5sDAYJSnlCp6ch25vJvxLukL0jmTc4aXm75Mz8Y9KVGshNWh2Z7XGr0xZirwPfAPY8wuY8zjxpg0Y0ya22J3A/NE5KTbtIuBJcaYn4EfgZkiErZhHC0cjp6WLVtSpkwZ7rjjjvAVqlQU+2nvTzR6vxFPzX6Kayteyy9P/kKfG/tokveRL1fdtPNhmYk4L8N0n/YbULewgQXCwuHoAejZsydZWVmMGTMm9IUpFcWOnzlOv2/7MfzH4VRIqMCH93zIA7UeiPgbgYRbVP4OOATD0fs8Hj3ALbfcQqlSpQpfmFKKaRumUWNUDd764S061e/Ehm4baFe7nSb5QojKsW5CMBy9z+PRK6UCs+PoDp6a/RTTN06n9kW1+eTeT2hUuZHVYUW0qEz0IRqOXsejVyqEchw5vL38bfov7I9DHLx262s80/AZ4mLjrA4t4kVl002IhqP3aTx6pZT/ftz9Iw3ea0CP+T1oktKEdV3X8fx1z2uSD5KoTPQhGo7ep/HolVK+O3r6KF1ndqXhuIbsP7mfz+77jK/afUVKmRSrQ4sqUdl0A0Efjt7n8egBbrjhBjZs2MCJEyeoVKkS77//vt58RCk3IsInaz/h33P/zf6T++l+bXdebvoypUroRQyhELWJPtgeeughHnroIQBiY2MLHKVy8eLF4QpLqYiz5dAWnpz1JPO2zOPqS69mRrsZXH3Z1VaHFdU00SulwuJs7lmGLh3KoMWDiIuJ4+2Wb9O1QVdiY2KtDi3qaaIvJB2PXinfLd6+mLSZaaw7sI621dvydsu3qfi3ilaHVWREVKIXEdv8WCKax6O34+0lVWTKzMrk+fnPM37VeJJLJ/NVu6+440odGiTcIibRx8fHk5mZSWJiom2SfTQSETIzM4mPj7c6FBXBRIRJP0+ix/weHD51mOcbP0+/m/pxYfELrQ6tSIqYRF+pUiV27drFgQMHrA4l6sXHx1OpUiWrw1ARauPBjaTNTGPhtoU0qtSIMXeMofbFta0Oq0iLmEQfFxdHlSpVrA5DKeXB6ZzTvLr4VYYsHUJCXAKjbx/Nv67+FzEmKn+uE1EiJtErpezrm9++ocvMLmw6tIkHaz/Im83f5OKSF1sdlnLRRK+UKrT9J/fz3LznmLx6MleUu4J5HebR7O/NrA5L5aGJXinlN4c4GPe/cfT6uhcns0/S78Z+9L6hN/HFtBPfjjTRK6X8sub3NaTNTGPZzmXclHwTo+8YTbXy1awOSxVAE71SyidZ2VkM/G4g//n+P5QuUZqJd03koboP6eXOEUATvVLKq1mbZtF1Vle2HdnGo/UeZWizoSQmJFodlvKRJnqllEd7ju/h6TlP89m6z6hevjrfPfIdNybfaHVYyk+a6JVSf5HryOXdjHd58ZsXyXZkM6jpIHpe15PiscWtDk0VgtdfMhhjxhtj9htj8r0TtjGmiTHmqDFmlevRz21eS2PMRmPMZmNMr2AGrpQKjf/t/R8N32/IU7OfolHlRvzS5RfSb0zXJB/BfKnRTwRGAJMKWGaxiJw3UpExJhYYCTQDdgErjDHTRWRdIWNVSoXQ8TPH6fttX9758R0qJFRgatup/LPmP7WzNQp4TfQissgYk1KIdV8DbBaR3wCMMR8BdwGa6JWyERFh2oZpPDX7KfYc30Pnqzvz6q2vUia+jNWhqSAJ1iAUjYwxPxtjZhtjarqmVQR2ui2zyzUtX8aYTsaYDGNMhg5cplR4bD+yndYfteaeT+4hMSGRZY8v49073tUkH2WC0Rn7PyBZRE4YY1oB04Cq/q5ERMYCYwFSU1N1QHSlQig7N5u3f3ib/gv7AzC02VCevvZp4mLjLI5MhULAiV5Ejrk9n2WMGWWMKQ/sBiq7LVrJNU0pZaHlu5bTeUZnVv++mjuvvJN3bnuH5DLJVoelQijgRG+MuQT4XUTEGHMNzuagTOAIUNUYUwVngn8AeDDQ8pRShXPk9BFe/OZFRmeM5rJSl/HF/V/Qplob7WwtArwmemPMVKAJUN4YswvoD8QBiMho4F6gizEmBzgFPCDOe9HlGGO6AXOBWGC8iKwNyX+hlPJIRPjol494Zu4zHMg6wNPXPs3ApgMpVaKU1aGpMPHlqpt2XuaPwHn5ZX7zZgGzCheaUipQWw5t4clZTzJvyzxSL0tlVvtZ1L+0vtVhqTDTX8YqFYXO5p5l6NKhDFo8iLiYOIa3HM6TDZ4kNibW6tCUBTTRKxVlFm1fRNqMNNYfXM99Ne5jWIthVPybxyubVRGgiV6pKHEw6yDPz3+eCasmkFImhZkPzqRV1VZWh6VsQBO9UhFORJi4aiI95/fk6JmjvHDdC/S7qR8JcQlWh6ZsQhO9UhFsw8ENpM1I47vt39G4cmNG3z6a2hfXtjosZTOa6JWKQKeyT/HK4ld4belrXFj8QsbcMYYn6j9BjAnWqCYqmmiiVyrCfP3b13SZ2YXNhzbToU4H3mj2BheXvNjqsJSNaaJXKkL8fuJ3np33LB+u+ZAryl3B/I7zufXyW60OS0UATfRK2ZxDHLy38j16fdOLrOws+t3Yj9439Ca+WLzVoakIoYleKRtb/ftq0mak8f2u72mS0oTRt4/mH+X/YXVYKsJoolfKhk6ePclL373Em9+/SdkLyvJBmw/oWKejDkCmCkUTvVI2M+PXGXSb1Y3tR7fz+FWP89qtr5GYkGh1WCqCaaJXyiZ2H9tN9znd+WL9F9SoUINFjyzihuQbrA5LRQFN9EpZLNeRy8gVI+mzoA/ZjmxeufkVnmv8HMVji1sdmooSmuiVstDKPSvpPKMzK/eupMXfWzDq9lFcXvZyq8NSUUYTvVIWOHbmGH0X9GXEihFcdOFFfNT2I+6veb92tqqQ0ESvVBiJCF+s/4Luc7qz9/heuqR2YfAtgykTX8bq0FQU00SvVJhsO7KNbrO6MXPTTOpdUo8v7v+Caytda3VYqgjQRK9UiGXnZjNs+TBe+u4lDIb/NP8P3a/tTrEY/fqp8NA9TakQWrZzGWkz0lizfw13/eMuht82nKTSSVaHpYoYr2OaGmPGG2P2G2N+8TC/vTFmtTFmjTFmmTGmrtu8ba7pq4wxGcEMXCk7O3zqMJ2/6sx146/j8OnDTPvnNKY9ME2TvLKELzX6icAIYJKH+VuBm0TksDHmNmAs4N7w2FREDgYUpVIRQkSY+stUnpn7DAezDvJMw2d4qclLlCpRyurQVBHmNdGLyCJjTEoB85e5vVwOVAo8LKUiz+ZDm+kyswtf//Y1DS5rwJz2c7jq0qusDkupoLfRPw7MdnstwDxjjABjRGSspzcaYzoBnQCSkvT0VkWOMzlneH3p6wxePJgSxUow4rYRpKWmERsTa3VoSgFBTPTGmKY4E/31bpOvF5HdxpiLgPnGmA0isii/97sOAmMBUlNTJVhxKRVKC7ctJG1GGhszN3J/zfsZ1mIYl5W6zOqwlDpPUG4waYypA4wD7hKRzHPTRWS36+9+4EvgmmCUp5TVDmYd5JFpj9D0g6aczT3L7Paz+fjejzXJK1sKuEZvjEkCvgA6isivbtMvBGJE5LjreXNgYKDlKWUlEWHCqgn0nN+TY2eO0fv63vS5sQ8JcQlWh6aUR14TvTFmKtAEKG+M2QX0B+IARGQ00A9IBEa5xunIEZFU4GLgS9e0YsCHIjInBP+DUmGx7sA60maksXjHYq5Pup7Rt4+m5kU1rQ5LKa98ueqmnZf5TwBP5DP9N6DuX9+hVGQ5lX2KwYsH8/rS1ylZvCTj7hzHo1c9SowJSsunUiGnv4xVqgDztszjyZlPsuXwFjrW6cgbzd/gogsvsjospfyiiV6pfOw7sY9n5j7DR798xJWJV/LNQ99wc5WbrQ5LqULRRK+UG4c4GJMxht7f9OZUzin639SfXtf3Ir5YvNWhKVVomuiVcvl538+kzUxj+a7l3FzlZt69/V2uTLzS6rCUCpgmelXknTh7ggELB/DW8rcod0E5JrWZRIc6HfRuTypqaKJXRdpXG7+i2+xu7Di6g3/V/xdDbh1CuQvKWR2WUkGliV4VSTuP7qT7nO5M2zCNWhfVYsmjS7gu6Tqrw1IqJDTRqyIlx5HDOz+8Q7+F/ch15PLqLa/yXKPniIuNszo0pUJGE70qMlbsXkHnGZ35ad9P3HbFbYxsNZIqZatYHZZSIaeJXkW9o6eP0mdBH0auGMklJS/h43s/5r4a92lnqyoyNNGrqCUifL7+c7rP7s6+E/vo2qArg24eROn40laHplRYaaJXUWnr4a10ndWV2ZtnU++Sekx7YBrXVNRRslXRpIleRZXs3Gz+8/1/GPjdQGJMDMNaDKPbNd0oFqO7uiq6dO9XUWPpjqWkzUzjl/2/0KZaG4a3HE7l0pWtDkspy2miVxHv0KlD9Pq6F+/97z0q/60y//fA/9H6H62tDksp29BEryKWiDBlzRSenfssh04d4rlGzzGgyQBKFi9pdWhK2YomehWRNmVuosvMLnyz9RuurXgt8zvOp+4lep8bpfKjiV5FlDM5Z3ht6Wu8svgV4ovFM6rVKDpd3YnYmFirQ1PKtjTRq4ixcNtC0maksTFzIw/UeoBhLYZxSclLrA5LKdvTRK9s78DJA/SY34NJP0/i8rKXM6f9HFpc0cLqsJSKGJrolW05xMHEVRPpOb8nx88cJ/2GdNJvSOeCuAusDk2piOLTbeyNMeONMfuNMb94mG+MMcONMZuNMauNMfXd5j1sjNnkejwcrMCLhClTICUFYmKcf6dMsTqisFm7fy1NJjbh8emPU7NCTValrWLQzYM0yStVCD4lemAi0LKA+bcBVV2PTsC7AMaYckB/4FrgGqC/MaZsYYMtUqZMgU6dYPt2EHH+7dQp6MnebseSrOws0r9Jp96Yeqw9sJb3W7/PwkcWUqNCDWsDsxm7fW7K5kTEpweQAvziYd4YoJ3b643ApUA7YIyn5Tw9rr76ainykpNFnCn+/EdyctCKmDxZJCHh/NUnJDinW2H2ptlS5a0qwgDk4S8flv0n9lsTSCFMnuz8aIxx/g3lNrTb5xbO/115BmSIp/ztacZfFiw40c8Arnd7/Q2QCvQA+rhN7wv08LCOTkAGkJGUlBSO7WJvxuSf6I0JWhFhOJb4ZM+xPfLPT/8pDED+8c4/5Nut34Y3gEI6l+DOfSzhSrz+fm6hTMR2O+gUZQUlel+bbkJORMaKSKqIpFaoUMHqcMIv77l4OQ/3LU1KClqRO3b4Nz3Ych25jFoximojqzFtwzQGNhnIz2k/0ySlSXgCCIB7yxo4U5y7rCxITw9N2f58bqFuAUxPd/6v7kL5v6vCCVai3w24jx5VyTXN03TlLr9v4/HjEJfn9nYJCTB4cNCK9XTMCOKxxKNV+1bReHxjus7qyjUVr2FNlzX0vakvJYqVCH3hQZBfgssrVAdMfz63UCdiqysLyjfBSvTTgYdcV980BI6KyF5gLtDcGFPW1Qnb3DVNucvv23j2LPztb5CcDMY4/44dC+3bB63YwYOdxw53QT6W/MWJsyfoMa8HqWNT2XZkG1PumcK8DvOomlg1dIWGgC+JLFQHTH8+t1AnYisrC8oPntp03B/AVGAvkA3sAh4H0oA013wDjAS2AGuAVLf3PgZsdj0e9aW8ItcZG4b2eE/C2ZE2bf00qfxmZWEA0ml6JzmUdSh0hYWYp3bycLVT5/e55Tct1P0w2kZvHwSjMzacjyKX6O3SKxoiO47skDYftREGILVG1ZKlO5ZaHVLA8ktw547XVlx54inhdukS+kRcUGVBr8gJH030dhel1aLs3Gx5c9mbcuHgC+WCQRfIa0tek7M5Z60OK2jslMQKqitYFefkySJxcefHExcX8bu1bWmijwR2yhpB8MOuH6Te6HrCAOT2KbfL1sNbrQ4pqgWz9S9Yu2JiYv4xJSYWbn2qYAUlettcXlnktW8P27aBw+H8G8RO13A6evoo3WZ1o+G4huw/uZ/P7vuMr9p9RUqZlJCXXZR/LRqMTtEpU6B8eejQITiXY2Zm+jddhY4mehUUIsInaz+h+sjqvJvxLk9d8xTru66nbY22GGNCnoTDNGKEbQV6BdW57ZdfEi4q18VHdUXBU1XfykeRbLqJYFsObZGWk1sKA5D6Y+rLit0rzpsfji6IKO/P9kkgTS7eriIqTCeznZtu8m6rcHRahxraRq8Kw1viOJNzRl5Z9IrED4qXkq+UlLe+f0uyc7P/sp5wJGELr1CNCp62XyCXjU6eLFK8+PnvL17c+uRZ0BVTobwMNdRdcJrold+81cKXbF8iNUfWFAYg93x8j+w8utPjusKRhMNxMImy/vLz+FKjL8w2teM28+d/DcY+Gq6L6jTRK795+jJUqpwjT/zfE8IAJGlYkkzfML3Q6wp2Eg7ll8kOV8CGMml26eJ7rT7Sz5J8/T+DtY+Gq1lRE73ym+cvQ67EvhQrPeb2kBNnTvi0rnAlyVAmQqv7AEK5DfNbd6iTn5U8fZahGoE0XM2KmuiV3zx9GYqX2yOr9q7ye312PIX3h9V9AKE80PjTlBFpHZT5KehXxKHYR7VGr4netpxfBsf5ST7+rPx3cq7VoVnC6hp9KA80vjZlROIB2pNovFFMQYler6NX+bq08QJKtX0OSm8D46BS5RzGj4ujQ/uiuctYMdKnu1COEultHQkJMHlyRP+O7y/C+fvE9u2dA8+GcCBar4rmt1Z5tP/kfjp+2ZFbJt1CydTpzF35K+KIYeeOYlHzJS+MQL+sgf4YJ5QHmvzWbYzzrxVJKRpZ/sN3T1V9Kx/adBN+uY5ceW/le1J2SFmJGxgnfb7pI1lns6wOKyoE69Q91LcEjOQ+FFVw041xzreX1NRUycjIsDqMImPt/rV0ntGZpTuXcmPyjYy+fTTVK1S3OvUbDtgAABTuSURBVKyokZLy5y0H3SUnO2t3SgWDMWaliKTmN0+bboqwrOwsen/dm3pj6rHh4AbGtx7PwocXOpN8VA/8EV5W3W5PP0J1jib6SBHkb+3sTbOpNaoWQ5YOoUOdDmzotoFHr3oUY4yOEBZkVtxuTz9C5U4TfSQI4rd2z/E93P/p/bT6sBUlipXg24e/ZcJdEyifUP7PhUJ9R+kiplUr/6YHQyR8hJF4xhGJMQPaGRsRgnARd05ujrzzwzvy2P3xsq004jCIIykp/143q38dJNZ3DgazfCuuwbfBR1ggOwwp4S+7x4z+YCqCTZ6c/zfWj2/tyj0rpcHYBtLuHuRU8Rjve6rFvw6y+gsV7PKtSLpWfIT+HByt/gFaYdg95oATPdAS2AhsBnrlM38YsMr1+BU44jYv123edF/K00Tv4m0QEi972LHTx+Tfs/8tMS/FyMVDL5YTl5b3bT0WZ1qrv1DBLt+qpBvOj9Dfm6Xb/YwjP3aPOaBED8QCW4DLgeLAz0CNApZ/Chjv9vqEtzLyPjTRuxQ0CImXb+2X67+USm9WEgYgnb/qLIeyDvm3p1rYdmL1F6qg8guzWaw6bobzI/Q2Xk7e/9fqg3lh2D3mQBN9I2Cu2+veQO8Cll8GNHN7rYm+sAoahMTDCEzbj2yX1lNbCwOQ2qNqy7Idy/5cn933VBerw/RUfmLiXxN28eLO6d6SqdV9DqHmy3g57p+f1c1zhRHqEUQD3T8CTfT3AuPcXncERnhYNhnYC8S6TcsBMoDlQJsCyunkWi4jKSmpEJsqCvmRcRwJCTKjfzu5cPCFkjA4QV5f8rqczTl7/voi5NtldZieyvd0azybb86w8GUEzLxnZJF48AtFzMHa38OZ6F8A3skzraLr7+XANuDv3srUGr2Lnxlna2nk9im3y7bD2wpe57k9NTHRt+qoBaxOAvmV788oj0WNL2PaF8Xt4otgncGGrekG+AloXMC6JgL3eitTE70bPzKOwxhxOBy+rzcCavd24u/t9iJlU/pzUC1o2XPzztXe7bJrWV1p8CZYfVKBJvpiwG9AFbfO2Jr5LFfNVWM3btPKAiVcz8sDmwrqyD330ERfMEdSUuBVAKsbwgvJyi+tv3diioTjpj/He3+urLFLcrVLfaag7WGLGr3z/bRyXTa5BUh3TRsItHZbZgAwJM/7GgNrXAeHNcDjvpSnid6zzZmb5dW0WnIiLsCsYvWlLYVghy9t3pavuDjvNXs78yfJ+HtljR3YoT5T0H47eXL+LbFhb6O34qGJ/q/O5JyRwYsGS/ygeCn1SimZPaCjs2Zf2CqTHb4BfrJjyO7NFRF23BQR/473/l5ZYwd2qM/4cxXXuelhv+rGiocm+vMt2rZIaoysIQxA2n7cVnYd3RX4SiNhkPQ87PCl9STYB6FwbdZg1ujt8lm4s0PlwNdO/EBj00QfoQ6ePCiPTXtMGIAkD0uWGRtnBLeAQLNJmNtS7PCl9SSYmyKcmzXQNno7fhbuPMVc2Fqz+3oDHe4h2AdLTfQRxuFwyMSfJkr518tL7Eux0nNeTzlx5kRgKw1FFTHMmbewCTBcteNglRPuA1phrro5l5DCdIwPSDDbwc+tz5/90N/fZWiNvgjYcGCDNJnYRBiANBrXSFbvWx34SkNVRbSgLcXfZGqHDlx/2bmJyp1drqzxRTAPnoVZV37bKtj7pib6CHAq+5T0XdBXir9cXMoMKSNjMsZIriM3OCsPVRXRqtG6/Mgudm7u8SQSY7a7YB48g7muYB4sNdHb3Pwt8+WK4VcIA5D2n7eXfcf3BbeAUFURw11dLkR5kVI7dheJZyF2Z3WNPhw00dvUvuP7pP3n7YUByBXDr5D5W+aHpqBQ7pnhPH8vxP9h1y+lN5HULBIJIrWz3B+a6G0m15Ero1eMljJDykjcwDjpu6CvnMo+5fkNEXZ1TMgUdJ2aB5H2r2uCD51gbttgrUubbqLU6n2rpdG4RsIA5KYJN8n6A+sLfkMEXu9eaN5i9FQ9PzdQfCFXaxeRdlBSgdHO2ChM9CfOnJDn5z0vxQYWk8TXEmXCTxN8G4AsUtse/OXLXj95sudafSG3h50OAkXlo1ZOwf68C0r0xjnfXlJTUyUjI8PqMIJm1qZZPDnzSbYf3c5j9R7j9Wavk5iQ6NubY2Kcn39exoDDEdxArZSSAtu3/3V6cjJs2/bna2Pyf38htseUKdCpE2Rl/TktIQHGjoX27f1aVVAUlY9aOQX78zbGrBSR1HzL8n91yle7j+3m3k/u5fYPbychLoHvHvmO9+963/ckD5CU5N/0SLVjh2/Tk5PzX64Q2yM9/fwkD87X6el+ryooispHXVhTpjjrAzExzr9TplgdUWDC+Xlrog+BXEcuw38YTvWR1Zm5aSaDmg5iVdoqbky+0f+VDR7srGa6S0hwTreTQL+Fvu71Qdwevh5bwiVSPupQy29XOnf2tX27sxa8fbvzdSQn+7B+3p7adKx8RHIb/co9KyV1bKowAGn+3+ayOXNz4Cu1U0NyfoLRq+TvoCtB2B52bBO3+0cdauEaLsAu9KqbCHPs9DF5evbTEvNSjFw89GKZumaq73d7inTByphhznJ6lYv9hGsAsGhUUKLXztgAiQhfbviS7rO7s+f4HtJS03jlllcoE1/G6tDCJ4J7EadMcbbJ79jhbCUaPNiajljl5GlX8iRvX31RVlBnbLFwBxNNth/ZTrfZ3Zjx6wzqXlyXz+//nGsrXWt1WOGXlJT/FTMR0IvYvr0mdjvxtCslJsKpU3+9Qqqo9V8UlnbGFkJ2bjZvLHuDGqNqsGDrAt5o9gYZnTKKZpIHe/YiWnGJRpjKjLarT9x52pXeftt52WtysvNEMTnZustgI5KnNh0rH3Zuo/9+5/dS5906wgDkzg/vlO1Htlsdkj0E0r4e7LZ5Kxrfw1RmUehXKOod0oWFttEH7sjpI/T+ujdjVo7hslKX8c5t79CmWhuMpx/wKN+E4ldLvv74KpjCVKYV/5qKDAH/YMoY09IYs9EYs9kY0yuf+Y8YYw4YY1a5Hk+4zXvYGLPJ9Xi48P+GNUSEqWumUm1ENcb+byxPX/s067uu5+7qd/uW5KP5PDsYPP1q6eGHC7/Nwn2B/JQp+WffEJRpt2v/VYTwVNU/9wBigS3A5UBx4GegRp5lHgFG5PPecsBvrr9lXc/LeivTLk03mzI3SbNJzYQBSOrYVFm5Z6V/KygK59mB8uXOycaIdOni+zrDeYG8txupBrlMO177r+yBAppufKnRXwNsFpHfROQs8BFwl4/HkRbAfBE5JCKHgflASx/fa5kzOWcYvGgwtUbVYvmu5Yy4bQTLH19O/Uvr+7ciu/3G3o58uTJHBEaP9r1mH87O4fw+4xCWacd+72DSE+AQ8XQEOPcA7gXGub3uSJ7aO84a/V5gNfAZUNk1vQfQx225vkAPb2VaWaNfuHWhVBtRTRiA3PfJfbL72O7CrywUtzeKtp4qbzXiwlZbw7WdCjojCVGZVu0CoS5XT4ADQyC/jPUx0ScCJVzPOwMLxM9ED3QCMoCMpKSk8GwZNwdOHpBHpz0qDEBS3kqRWb/OCnylwT7PjtZvgnsGiY313oxjpwNcONpSwpjZPRUVjl1Pm6UCE2iibwTMdXvdG+hdwPKxwFHX83bAGLd5Y4B23soMZ43e4XDIhJ8mSOJriVJsYDHpNb+XnDx7MjgrD/a3oyh8Ewoac96OB7hQZ8AwHtwLKiocu1447u9rx7tMBUugib4Yzk7UKvzZGVszzzKXuj2/G1juel4O2IqzI7as63k5b2WGK9GvP7BebppwkzAAafx+Y1m9b3XwCwnm3hCJd7oujC5dfEv2djnAhfIbH8aDe0FFFWbX83ezhPpfjfb7xgaU6J3vpxXwK86rb9Jd0wYCrV3PXwXWug4C3wLV3N77GLDZ9XjUl/JCneizzmZJ3wV9JW5gnJQdUlbGZoyVXEdu6AoM1o+JPDVr2CXhBdPkyb4140S7MB7cCyrK3yRcmEQY6uQZzAOJHU+uA0704X6EMtHP2zxP/v7234UBSIcvOsjvJ34PWVki4vvem9/BwJeOSqurEaHkrVYfjQe4cwpqL7GgRu9vEi5sInT/GiQmOh/BOlEqaHfyd/12PLnWRC8i+47vkwc/f1AYgFQdXlW+3vJ10MvIly97vL+DcMfG2qdhMJQKSnRWHeDC0TDr7QBvQRv9ufm+/us+JcICVhiK2r23IZD9Wb/W6G2W6HMduTJ6xWgp/WppKf5ycen/bX85lX0qaOv3ypc9Xgfhzp+nhJeYaF2SD3XHq7d9IZQH98mTZXLiU5LMVjHkSnLi8dBdN+BlW4Yikfpyguzr+qOyjT7cj2Al+p/3/SyNxjUSBiBNJzaVDQc2BGW9fvFlj/Wl49Eu1YZws9OlDaGsxvmShUJ5gA9y5vK6Ok/bMjFRRELXNOLtWOrP+u20a4oUwUR/4swJ6Tmvp8S+FCsVXq8gk1ZNsu5uT758gQra6e1WbSjKQtkw68tZXSgP8CE4iBWYCL380CzUTSN2bHoJVJFK9F9t/EqShyULA5An/u8JyczKLPS6gsbbob+gg4Hdqg2RLpDtGcrs4O2sLtQH+HD3LnrpaLbVzxMi5DtYJBL9kVNH5J6P7xEGIDVH1pTF2xf7vQ5LRcjOFNECzR6hzD7errAJ9f4Q7iru5Mme/1/XwSUcQy54Xb8/n7nF3+EikehzcnPkxgk3yiuLXpEzOWf8fr8qAoKRzEL1Zba6d8+K8j1dVWan9hNf9xmrPz8pIoleREL7oycV+ex48bM7q8/qwl2+DZKjV77uMzZo9C8o0esdplT0mzLFOZywp5uD6O2ZrHPus9mxwzlk9eDB9roRrK+39IqJcab2vIwBhyNU0eUpKsA7TCkVsc7dqtBTko+mwdwjUfv2zoTpcDj/2inJg+83APB0XwVf7rcQBproVXQr6MYgycmB3ZtWRb/27Z37SHKys3buaZ+x+R1htOlGRTcbnFKrIsLiZqiCmm6KhS0KpayQlJR/s41NTqlVFGnf3rZnh9p0o6KbzU+plQoHTfQquvnaxqqc9O7cUUmbblT0s/Epta2cu0LpXOf19u3O16DbL8JpjV4p5ZTfFUpZWc7pKqJpoldKOe3Y4d90FTE00SulnGz+ox9VeJrolVJOeoVS1NJEr5Ry0iuUopZPid4Y09IYs9EYs9kY0yuf+c8aY9YZY1YbY74xxiS7zcs1xqxyPaYHM3ilVJDZfewZVSheL680xsQCI4FmwC5ghTFmuoisc1vsJyBVRLKMMV2A14F/uuadEpF6QY5bKaWUj3yp0V8DbBaR30TkLPARcJf7AiLyrYicuy5rOVApuGEqpZQqLF8SfUVgp9vrXa5pnjwOzHZ7HW+MyTDGLDfGtPH0JmNMJ9dyGQcOHPAhLKWUUr4I6i9jjTEdgFTgJrfJySKy2xhzObDAGLNGRLbkfa+IjAXGgnP0ymDGpZRSRZkvNfrdQGW315Vc085jjLkVSAdai8iZc9NFZLfr72/AQuCqAOJVSinlJ6/j0RtjigG/ArfgTPArgAdFZK3bMlcBnwEtRWST2/SyQJaInDHGlAe+B+7K05GbX5kHAA+3BPKqPHCwkO8NJY3LPxqXfzQu/0RjXMkiUiG/GV6bbkQkxxjTDZgLxALjRWStMWYgzpvRTgeGAiWBT40xADtEpDVQHRhjjHHgPHsY4i3Ju8rMN1hfGGMyPA2+byWNyz8al380Lv8Utbh8aqMXkVnArDzT+rk9v9XD+5YBtQMJUCmlVGD0l7FKKRXlojHRj7U6AA80Lv9oXP7RuPxTpOKy5c3BlVJKBU801uiVUkq50USvlFJRLiITfSCjadogtjRjzBrXaJ5LjDE17BCX23JtjTFijAnLpWc+bK9HjDEH3EZAfcIOcbmWud+1n601xnxoh7iMMcPcttWvxpgjNokryRjzrTHmJ9f3spVN4kp25YjVxpiFxpiQj9NljBlvjNlvjPnFw3xjjBnuinm1MaZ+wIWKSEQ9cF7LvwW4HCgO/AzUyLNMUyDB9bwL8LGNYvub2/PWwBw7xOVarhSwCOfAdKl2iAt4BBhhw32sKs5RW8u6Xl9kh7jyLP8Uzt+9WB4Xzk7GLq7nNYBtNonrU+Bh1/Obgf+GIa4bgfrALx7mt8I5XpgBGgI/BFpmJNbo7Tyapi+xHXN7eSEQjt5wr3G5vAy8BpwOQ0z+xBVuvsT1L2CkiBwGEJH9NonLXTtgqk3iEuBvruelgT02iasGsMD1/Nt85gediCwCDhWwyF3AJHFaDpQxxlwaSJmRmOgDHU0zlHyKzRjT1RizBee4/d3tEJfr9LCyiMwMQzw+x+XS1nUK+5kxpnI+862I60rgSmPMUtfIrC1tEhfgbJIAqvBnErM6rgFAB2PMLpw/vnzKJnH9DNzjen43UMoYkxiG2Arib47zKhITvc/cRtMcanUs7kRkpIj8HXgB6GN1PMaYGOBN4DmrY8nHV0CKiNQB5gMfWBzPOcVwNt80wVlzfs8YU8bSiM73APCZiORaHYhLO2CiiFTC2TTxX9d+Z7UewE3GmJ9wjrq7G7DLNgsaO2xofwU0mqYdYnPzEeBxjP4g8hZXKaAWsNAYsw1nu+D0MHTIet1eIpLp9vmNA64OcUw+xYWzljVdRLJFZCvOgf+q2iCucx4gPM024FtcjwOfAIjI90A8zgG8LI1LRPaIyD0ichXOfIGIhKUDuwD+5hHvQt3xEIKOjGLAbzhPS891sNTMs8xVODthqtowtqpuz+/EOTCc5XHlWX4h4emM9WV7Xer2/G5guU3iagl84HpeHuepdqLVcbmWqwZsw/WDSJtsr9nAI67n1XG20Yc0Ph/jKg/EuJ4PBgaGaZul4Lkz9nbO74z9MeDywvFPhWAjtcJZg9oCpLumDcRZewf4GvgdWOV6TLdRbG8Da11xfVtQwg1nXHmWDUui93F7veraXj+7tlc1m8RlcDZ3rQPWAA/YIS7X6wE4R4oNyz7v4/aqASx1fY6rgOY2ieteYJNrmXFAiTDENBXYC2TjPDN8HEgD0tz2rZGumNcE47uoQyAopVSUi8Q2eqWUUn7QRK+UUlFOE71SSkU5TfRKKRXlNNErpVSU00SvlFJRThO9UkpFuf8HxJJmryxonWYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出参数更新之前的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0.numpy() * plot_x - b0.numpy()) / w1.numpy()\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的参数更新方式其实是繁琐的重复操作，如果我们的参数很多，比如有 100 个，那么我们需要写 100 行来更新参数，为了方便，我们可以写成一个函数来更新，其实 PyTorch 已经为我们封装了一个函数来做这件事，这就是 PyTorch 中的优化器 `torch.optim`\n",
    "\n",
    "使用 `torch.optim` 需要另外一个数据类型，就是 `nn.Parameter`，这个本质上和 Variable 是一样的，只不过 `nn.Parameter` 默认是要求梯度的，而 Variable 默认是不求梯度的\n",
    "\n",
    "使用 `torch.optim.SGD` 可以使用梯度下降法来更新参数，PyTorch 中的优化器有更多的优化算法，在本章后面的课程我们会更加详细的介绍\n",
    "\n",
    "将参数 w 和 b 放到 `torch.optim.SGD` 中之后，说明一下学习率的大小，就可以使用 `optimizer.step()` 来更新参数了，比如下面我们将参数传入优化器，学习率设置为 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用 torch.optim 更新参数\n",
    "from torch import nn\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_regression(x):\n",
    "    return torch.sigmoid(torch.mm(x, w) + b)\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], lr=1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 200, Loss: 0.39672, Acc: 0.92000\n",
      "epoch: 400, Loss: 0.32435, Acc: 0.92000\n",
      "epoch: 600, Loss: 0.29053, Acc: 0.91000\n",
      "epoch: 800, Loss: 0.27069, Acc: 0.91000\n",
      "epoch: 1000, Loss: 0.25759, Acc: 0.90000\n",
      "\n",
      "During Time: 0.591 s\n"
     ]
    }
   ],
   "source": [
    "# 进行 1000 次更新\n",
    "import time\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "    # 前向传播\n",
    "    y_pred = logistic_regression(x_data)\n",
    "    loss = binary_loss(y_pred, y_data) # 计算 loss\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad() # 使用优化器将梯度归 0\n",
    "    loss.backward()\n",
    "    optimizer.step() # 使用优化器来更新参数\n",
    "    # 计算正确率\n",
    "    mask = y_pred.ge(0.5).float()\n",
    "    acc = (mask == y_data).sum().item() / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.item(), acc))\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到使用优化器之后更新参数非常简单，只需要在自动求导之前使用**`optimizer.zero_grad()`** 来归 0 梯度，然后使用 **`optimizer.step()`**来更新参数就可以了，非常简便\n",
    "\n",
    "同时经过了 1000 次更新，loss 也降得比较低了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们画出更新之后的结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fd3621aee48>"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxV8//A8ddnppn2dYowmuKbHzNpHUSSIqJkKySUVFosX74isowK6YuvpU0brSi+CBGVUiqaatoma0IpjSKlbWbu+/fHnXynae7MvTNnvfN+Ph73MXc59573PXPu+3zO53wWIyIopZTyvxi3A1BKKWUNTehKKRUlNKErpVSU0ISulFJRQhO6UkpFiXJurbh27dpSv359t1avlFK+tGrVqt9EpE5hr7mW0OvXr096erpbq1dKKV8yxvwY6jWtclFKqSihCV0ppaKEJnSllIoSmtCVUipKaEJXSqkooQldKaWihCZ0pZSKEr5L6F/99hUPL3yYQzmH3A5FKaU8xXcJ/b2v3+OJJU/Q7OVmfLH1C7fDUUopz/BdQh/UahAfdv+QfYf3cd7k87jv4/s4kH3A7bCUUsp1vkvoAB3+0YENAzbQt3lfnl3+LE3GNWHJj0vcDksppVzly4QOUK18NcZ2GsvCWxaSE8jhglcv4M65d7Lv8D63Q1NKKVf4NqEf0bZBW9b3X89dZ9/F6JWjOXPsmSzYvMDtsJRSynG+T+gAleMr88JlL/DZrZ8RHxvPxdMu5vb3bmfPwT1uh6aUUo6JioR+xPn1zifj9gwGnTeIiWsm0mhsIz789kO3w1JKKUdEVUIHqBhXkZHtR7L8tuVUK1+Ny2deTo93erD7wG63Q1NKKVtFXUI/4uyTzmZ139UMaT2EGetmkDImhXe+esftsJRSyjZRm9ABypcrz/B2w1nZZyXHVz6eq9+4mm5vdSPrryy3Q1NKKctFdUI/otkJzVjZZyXD2g7jrcy3SB6TzBsb3kBE3A5NKaUsUyYSOkBcbBwPX/Awq29fTYMaDbjhrRu4dta17Ni3w+3QlFLKEsUmdGPMZGPMTmPMhhCvG2PMi8aY74wx64wxza0P0zqNjmvEstuWMfLikcz9di7Jo5OZunaqe6X1GTOgfn2IiQn+nTHDnTiUUr4XTgn9VaBDEa9fBjTMu/UFxpY+LHuViynHoFaDWNtvLcl1kunxTg86vdaJrX9udTaQGTOgb1/48UcQCf7t21eTulKqRIpN6CLyGVBUm78rgakStAKoYYw5waoA7fR/tf+PxT0X80KHF1i0ZREpY1KYsGqCc6X1IUNg//6jn9u/P/i8UkpFyIo69JOAn/M93pr33DGMMX2NMenGmPSsLG+0NImNieWuc+5iXb91ND+hOX3f70v7ae3Z8scW+1f+00+RPa/KHK2RU5Fw9KKoiIwXkVQRSa1Tp46Tqy7WqbVOZcEtCxjbcSxfbPuCRmMaMfrL0QQkYN9K69WL7PkS0qTgT1ojpyJlRULfBpyc73Fi3nO+E2Ni6Jfaj40DNtKqXivu+PAOLnz1Qr7d9a09K3ziCahU6ejnKlUKPm8RTQrWcvLgqDVyKmIiUuwNqA9sCPFaR+BDwAAtgS/D+cwWLVqIlwUCAZm0epJUf6q6VBxeUZ5d9qzk5OZYv6Lp00WSkkSMCf6dPt3Sj09KEgmm8qNvSUmWriZq5f/3JCSIxMcfvR0rVbL8X/Y3Ywr/3xljz/qKY/OuqsIEpEuoXB3qhb8XgNeA7UA2wfrx24B+QL+81w0wGvgeWA+kFveZ4oOEfsS2P7fJFTOvENKQcyacI5k7M90OKSJeSwp+Mn16MGEXtv2cODhGejC2M+EWti3sPJip0EqV0O26+SWhiwRL6zPXzZSEpxMkfli8PPnZk5Kdmx35B7lQxNESesmF2nZOHRwjSaJ2J1zdj7xDE7pFduzdIV1ndRXSkOYvN5e1O9aG/2aXijhasiq5UGc3Tia1cMsAdidcPdPzDk3oFntz45ty3L+Pk3JDy8ljnz4mh3IOFf8mF4s4WvdZMuGU0L1ycLQ74WoJ3TuKSuhlZiwXK12bfC2ZAzK5PuV6Hl/8OKnjU1n1y6qi3+Rim/Pu3WHLFggEgn+7d7d9lVGhsEZIcXGQkADGQFISjB/vje1pdwtYBxpkKQtoQi+hhEoJTL9mOnNumMOuA7s4Z+I5PLTgIQ7mHCz8DQ61OVfW6d49mLCTkv6XwF95BX77zXsHR7sTbmHbwisHM5VPqKK73Tc/V7kU9PuB36XXO72ENOT0UafLsp+WHbuQVmYrm2nVWtmAVrnYq0aFGky6chLzbprH/uz9tJrcinvn3cv+7Hy9QrSIoywSqnOTVq0pE0z4zktNTZX09HRX1m2nvYf28sD8BxibPpZTa57KpM6TaFO/jdthqShxpOdv/h6klSpp2aAsMcasEpHUwl7TErrFqpavypiOY/i0x6cIwoVTLmTgBwPZe2iv26GpKODF4QB0rCDv0IRukwvrX8i6fuv45zn/ZGz6WM4ceyaffP+J22Epn/PaAJ06VpC3aEK3UeX4yvynw39Y2mspFcpV4JLpl9BnTh/2HNzjdmjKp7zWWMqLZwxlmSZ0B5x38nmsuX0ND7R6gMkZk0kZk8L737zvdljKh7zWHtxrZwxlnSZ0h1SMq8iIi0ew4rYV1KxYkyteu4Kb376Z3QeKmgxKqaNZ2VjKirrvWrUie17ZS1u5uOBQziGeXPIkTy59koSKCYzpOIZrzrjG7bBUGWJVa5natWHXrmOfT0gIdsBS1tNWLh5Tvlx5Hm/7OOl90jmx6olcO+tarn/zenb+tdPt0FSUO1Iqv+kma+q+d4c4wQz1vLKXJnQXNanbhC96f8ET7Z7gna/eIWVMCq9veB23zpr8QpvJlUz+FimhRFr37bWLtGWdJnSXxcXG8VDrh1hz+xpOrXkq3d7qxtVvXM32vdvdDs2TtJlcyRXWIqWgSBOx1y7SFifqCwOhxgSw+xZNY7lYJSc3R575/BmpMLyC1BhRQ15Z84oEAgG3wwqbE2OJ6DCuJVfc+O4lHVrIL2PIRMtwShQxloteFPWgb3d9y21zbmPJT0vo8I8OvNzpZepV9/Y5rFNd0mNigj/FgowJjmGiQqtfP3R1S1JSsFQdzcMHhPr+SUnBsW/8Qi+K+kzDhIYs6rmIly57iSU/LqHRmEa8nP6yp+vWnepgonW2JReqeqR//+D9m2+OvmqI/FUsoQ5mUdVmPlTR3e6bVrmEZ/PuzdJuSjshDWk3pZ18v/t7Sz/fqtNlp6Yoi5bTZrcU/H/37x+929PtSb7tgk5B52+BQEDGp4+Xqk9WlUpPVJIXV7wouYHcUn+ulcnRybptv9TZ+kE0X5NwYwpBJ/ZNTehR4qc/fpLLpl8mpCGtJrWSr3/7ulSfZ+WPOZpKzmXpgBHNkz8XdRHYjv+tU78BTehRJBAIyKtrXpUaI2pIheEVZOTSkZKTm1Oiz7L6xxwNiTCaDkzFmT5dJDa27JXQ7fpuTq1PE3oU+uXPX+TK164U0pCzJ5wtG37dEPFnRPPpdkm5vU2cOigWVb8cLQcwpw/OTp3taEKPUoFAQF5f/7rUHllb4ofFy/DFw+VwzuGw31+WSqPhcrMKwu7/R/6DRaiSeWxsdP3/nTxr1BK6ssTOfTvl+tnXC2lIs3HNZM32NWG/NxqqSazkZgndznWH2+IjGurO3aJ16MpS/838rxz/7+Ol3NBy8sjCR+Rg9kG3Q/IdN89a7Dw7CKfFR1mvbrOC261ctGNRFLn6jKvJHJjJjWfeyLDPhtFifAtWblvpdli+YuV445Gys9NUOJ1nvDwGi1907x7sdRoIBP863fNWE3qUqVWxFlOumsIHN37AHwf/oOWkljzwyQMcyD7gdmi+4daP0s6BrkIdFGJjnT9wKftoQo9Slze8nI0DNtKraS9GLhtJ05eb8vlPn7sdliqCnWcHoQ4WU6a4V5pU1tOEHsWqV6jOhM4T+OTmTziUc4jWr7Tmnx/9k78O/+V2aCoEu84O3KxKUs7R0RbLiH2H9zF4/mBGrxzNKTVPYeIVE2nboK3bYSmlIqSjLSqqxFdh1OWjWNRjEQZDu6nt6P9+f/489KfboSmlLKIJvYxpU78N6/qv496W9/LyqpdpNKYR876b53ZYSikLaEIvgyrFVeLZS59l2W3LqBJfhQ4zOtDr3V78fuB3t0Pzvaif4kx5mib0MqxlYktW376aB89/kKlrp5IyJoX3vn7P7bB8S+c7VW7ThF7GVShXgScvepIven9B7Uq16fx6Z7r/tzu/7f/N7dB8x6lZm5QKRRO6AqDFiS1I75vOY20eY9bGWaSMSeHNzDfdDstXQvXGjKopzpSnhZXQjTEdjDFfG2O+M8YMLuT1esaYT40xa4wx64wxl1sfqrJFvkrf+FNPI21bQ1b1XcXJ1U6m6+yudJnVhV/3/ep2lL6g850qtxWb0I0xscBo4DIgGehmjEkusNjDwCwRaQbcAIyxOlBlgxCVvo3nr2dF7xU8ddFTvPfNeySPSWb6uum41WfBL+zsul8UvRCrjginhH428J2IbBaRw8DrwJUFlhGgWt796sAv1oVYhjj9yyyi0rdcTDkGnz+YjNszOC3hNG5++2Y6v96ZbX9uszcmn6tY8X/3ExLs742pF2LVUUINw3jkBnQBJuZ7fDMwqsAyJwDrga3A70CLEJ/VF0gH0uvVq2f9uJJ+5sa4rWGO15qTmyPPLXtOKg6vKNWfqi6TVk+SQCBgX1w+5Nawu27PsKSchwPD53YDXhWRROByYJox5pjPFpHxIpIqIql16tSxaNVRwo0mEmFW+sbGxHLPufewrv86mtRtwm1zbqPDjA78+MeP9sXmM261cNELsfbwazVWOAl9G3ByvseJec/ldxswC0BElgMVgNpWBFhmuPHLjLDS9x+1/sGnPT5l9OWj+fynz2k0thHj0scRkIB9MfqEW4lVL8Raz8/VWOEk9JVAQ2NMA2NMPMGLnnMKLPMTcBGAMeYMggk9y8pAo54bv8wSDMEXY2IYcNYANgzYwLmJ59L/g/5cNPUivt/9vX1x+oBbidWtC7GR8Ftp19f9CULVxeS/EaxG+Qb4HhiS99xQoHPe/WTgc2AtkAFcUtxn6hR0BVhdCevAXFiBQEAmrpoo1Z6qJpWeqCTPL39ecnJzvBamI+t2c+o6L88L68eJyN2cKDwc6JyiPmHVL9PhX9HPe36Wy2dcLqQh5006TzZlbfJimLav28uJ1S1+vGjr9Zg1oXud1ZnAhT0yEAjItLXTpOaImlJ+WHkZsWSEZOdmey1MT6zbzyLdVb1e2i2M188qNKF7mR17j4u/ou17t8s1b1wjpCGp41Nl/a/rvRimLxON20qyq/r1wOnlsy1N6F5mxx7v8q8oEAjIrA2zpM7IOhI3NE6GLhoqh3MOeypMvyYaN4XaZrGxoROf10u7fqQJ3cvsKCp65FeU9VeWdHuzm5CGNBnbRFb/stozYXpkE/lKqF21uG3o5dKuH2lC9zK7iooe+hW9s+kdqftMXYl9PFaGLBgiB7MPeiJMD20iXwi1q+pZjrM0oXtZcUXFKMk6u/fvlh5v9xDSkOTRybLi5xVuh1SkKNnslipsV9XrEM7ThO51obJHFNYLzP1mriQ+lygxj8fIffPuk/2H97sd0jEK2+xHqhvKenLPv6vGxmoJ3Q2a0P0qSq/c/XHgD+k7p6+QhjR8saEs+XGJ2yEdpbiqBZ8fUy0TheUN21lx5ldUQtcZi7wsSkdeql6hOi9f8TLzb55PdiCbC165gLs+vIt9h/e5HRpQ/Ob1TTdwm5Vg5IgyzYkxYjShe5lVA4R4dDCNi065iPX913PH2Xfw0pcvcebYM1mweYHbYYW1eX1+TLVM9+6wZQsEAsG/msxDc2KMGE3oXmbFyEseHzquSnwVXrzsRT7r+RlxMXFcPO1ibn/vdv489KdrMRW22QvS0QxVpJw44daE7mVWnNP6ZOi41kmtWdtvLfedex8T10wkZUwKH377oSux5N/sENz0+XltNEOrefSEzvccGZEzVOW63Te9KOoQH/ZxX/HzCkkenSykIT3e7iG79+92NZ6CF7L694/eJo16odM+Vm1btJVLGebTljKvTj0s1Y7bLZArMTV+kntGrnQ7JBHxb8ILt3WFT3cX3/QbsLuViyb0aOfDDFRoB5a4fdLyrhcl668sV2PzY8KLZBfw4QmdH3fxUtGEXtb5pfiSJ2Q78OpbpM7IOvLGhjdcm6TajwkvkoNQUW3wvbrreOUgW9TPzMqfoCZ05Suhk2ZAUsenCmnINW9cI9v3bnc8Nq8kj0hEchAqrnu/F0u+XjjIFnWWYPUZhCZ05StFJc3s3Gx5eunTUn5Yeak5oqZMzZjqaGndj6f3kR6EjpQmiyqpe4kXDrJFxWB1fJrQla+EkzS/yvpKzpt0npCGdJzRUX7e87Oj8fmoBqvEByEvlHzD4YWDbFHbyurtqAld+U44STMnN0eeX/68VBxeUao9VU0mrproWt2615XkIOSFkm+43D7IagldE7p/uP1rKcZ3u76TC1+9UEhDLp56sfzw+w9uhxQ2L29aL5R8/ULr0DWh+4NPftW5gVwZu3KsVHmyilR5soqM/nK05AZy3Q6rSH7YtF4+4HiNF1q5mODrzktNTZX09HRX1q0iUL9+cPyXgpKSgqMxecxPe36iz3t9+Pj7j7kg6QImdZ7EP2r9w+2wCuWzTesbM2YER7b46adgt/onnoiuQcOMMatEJLWw13QsF1U0nw3hW696PT7q/hGTO09m7Y61NB7bmOeWP0duINft0I7hs03rC1aORefHMW00oUcTO/ZAR0YUKr38X71BA0N85q1kDszk4lMu5l8f/4vzXzmfTVmb3A7zKD7ZtL5i1Vh0Hh+kNLRQdTF237QO3WJ2Vcj6oKK3qBADgYDMWDdDaj1dS8oPKy9PLXlKsnOz3Q5ZRHyxaX3HqiaCXm7hg14ULQPs3AM9fmUsnK++Y+8O6TKri5CGtHi5hazdsdatcI/i8U3rO1b9DLzcBr+ohK4XRaNFTExwnyvImOB0MlEskq/+ZuabDJw7kN0HdjOk9RAeav0Q8bHxzgSqbHekqiR/tUulSpFPI+DlC9Z6UbQsiLYK2QiuB0Ty1bskd2HjgI1cl3Idjy9+nNTxqaz6ZZUlISv3WTXPqRWThbkiVNHd7ptWuVgsmipkI/wuJf3q7371rpzwzAkS+3isPDj/QTmQfcCGL6P8yqvVYWgdehnh1T0wUiWoCC3pV//9wO9y6zu3CmnI6aNOl2U/LbPgCyhln6ISutahK+9x4XrAvO/m0ee9Pmz9cyv3tLyHYe2GUSmumJmilXKB1qErf3HhesCl/7iUDQM20C+1H8+teI7GYxuzeMti29anlB00oSvvcemKVLXy1RjTcQwLb1mIIFw45UIGfjCQfYf3Wb4uP/ZCVCXn1P9bE7ryniNNFRIS/vdcxYqOrb5tg7as67eOu8+5m7HpY2k0phHzN8+37PN92wtRlYiT/29N6F6ixbajHTjwv/u7djma9SrHV+b5Ds+z5NYllC9XnvbT2tNnTh/2HNxT6s+2qnu68gdH/9+hrpbafdNWLgVEU7PDooTbHMVDfa/3H94v9398v8Q8HiMnPXuSvP/1+6X6PC/3QlTWc3LGorBK6MaYDsaYr40x3xljBodY5jpjTKYxZqMxZqalR52yoCwU2yI597RwKMLSnvhUjKvI0+2fZsVtK6hRoQadXuvELW/fwu4DuyOOBaKvD5gqmqP/71CZ/sgNiAW+B04B4oG1QHKBZRoCa4CaeY+PK+5ztYReQFkotkVS6raohG71ic/B7IPy6MJHpdzQcnL8v4+X/2b+N+LPKCsnYyrIUzMWAecC8/I9fhB4sMAyI4HexX1W/psm9AI8VMVgm0gOWhb9CuzarGu2r5Fm45oJach1s6+TX/f9GtH7o6UPmAqPUzMWhZPQuwAT8z2+GRhVYJl38pL658AKoEOIz+oLpAPp9erVK/k3ikZlodgWaXa14Fdg54nP4ZzDMnzxcIkfFi+1R9aW19a/ppNUK9sVldCtauVSLq/a5UKgGzDBGFOj4EIiMl5EUkUktU6dOhatOkpYNaqQl0Xavrx79+DQdoFA8G8JtoWd9ZdxsXEMuWAIq/uu5pSap9DtrW5c/cbVbN+7vfQfrlQJhJPQtwEn53ucmPdcfluBOSKSLSI/AN8QTPAqEhYkME9z4aDlRB+llONSWNZrGf9u/2/mfT+P5DHJTMmYcuSsVCnHhJPQVwINjTENjDHxwA3AnALLvEOwdI4xpjZwGrDZwjhVtHD4oOXUMSQ2Jpb7zruPtf3W0ui4RvR8tycdZ3bk5z0/W7uiMkC7Y5RcsQldRHKAO4B5wCZglohsNMYMNcZ0zltsHrDLGJMJfAoMEpFddgWtXOLTX5qTx5DTEk5jcc/FvNjhRRb/uJiUMSmMXzVeS+uFKGx30l60paOjLarwWDUVTBmy+ffN9HmvDwt/WEi7Bu2YeMVEGtRs4HZYnhBqd6pYMdgpuCAvzBTkFUWNtqgJXYXHy3NyeZiIMGH1BO77+D5yJZcRF41g4NkDiTFle9SNULtTKGVgJsWw6fC5qvQs7LlZlhhj6NuiLxsHbOSCpAu466O7aPNqG77Z9Y3bobkq0t1Ge9GGRxO6Co/2Vy+Vk6ufzNwb5/Lqla+yYecGmoxrwjPLniE3kOt2aK4ItdskJPh0Lk+P0ISuwuPbWXO9wxhDj6Y9yByQyaWnXsqgTwZx3uTzyMzKdDs0x4XanV54Ifq7Y9gqVI8ju2/a9d+HtL+6ZQKBgLy2/jVJeDpB4ofFy/DFw+VwzmG3w3KU7k4lg84pqpQ37fxrJ3d+eCezNs6iWd1mvHLlKzSp28TtsJSH6UVRpTzquMrH8UaXN3jrurf4Ze8vpE5I5bFPH+Nw7mG3Q1M+pAldKQ+45oxr2DhgI90adWPoZ0M5tfcjnJB4yG99uJTLNKEr/3Kj56qN60yolMDUq6fyryqr2TbjMXZsK6+9JVVENKErZ1idCN3oI+7QOt98qRmSfXQTkGibvErZQy+KKvvZMWyAGz1XHVpnTEzweHEMI+w7uJ/K8ZUtW5fyH70oqtxlx3ypbvRcdWidIftqVfuRxuMas2jLIkvXp6KHJnRlPzsSoRs9V2vVcmSdoTrdDEnbj8HQdkpbBnwwgL2H9lq6XuV/mtCV/exIvk73XJ0xA/7889jn4+MtX2eoMdyH/zOZdf3XcU/LexiXPo5GYxvx8fcfW7pu5XOhehzZfYu6nqLa7S20UPOl9u9fum3m5DYPNR9qQoJ96yzCsp+WyemjThfSkF7v9JLfD/zuShzKeZRmkmi7blGV0MvCBM+lVTD59u/vr21m52zTJXQg+4AM/mSwxD4eKyc+e6LM+WqOa7Eo5xSV0LWVixV0rPDIhdpmCQnw22+Oh1MsD/+P039Jp9e7vVi/cz3dz+zOCx1eIKFSgqsxKftoKxe76VjhkQu1bXbt8mYPGg+PNpl6YirpfdNJa5PGGxvfIHlMMm9lvuV2WCH5dCZDX9CEbgUdKzxyRW0bL/agcWq26RKKj43nsQsfY1XfVSRWS6TL7C50nd2VnX/tdDu0o+icofbShG4FD5fePKuobePVMxsnZ5suocbHN+aL3l/w1EVPMefrOSSPTmbm+plEUrVqZwnaji4JKp9Qlet236LqoqiI9S0uykKrmYSEwi80xsZG9/d2SObOTGk5saWQhlwx8wrZumdrse+x+/q+B68t+w7aysVnykqrmcK+Z8FbNH7vwth0AM/JzZFnlz0rFYZXkOpPVZfJqyfL9OmBkKsK1TozKcmScGz//LJAE7rflKW9Pn8ii40tO987PwcO4N/89o1c8MoFwjXdJCb+QMhV2V2CLitlFTtpQvebsnheOn166FJ6NH9vEccO4LmBXKlZd0+Rq3IiFL/VJnot3qISul4U9aKy1mrmSNOHUKL1ex/hULPXGBPDH79WK3JVTlzfd+LaslUXdn3XKidUprf7piX0IpS189JQxcJo/95HOFjFFmpVNevukZzcHBGJvETqtRKslT8fL9Z+olUuPuS1X4mdQlUxQXR/7yMcPIAXtqqY+APCNd3kvEnnyVdZX3k19LBZmYS9WPtZVELXKhcrleY8r+B7wfNtni0TqkolKSm6v/cRDnZaKmxVUyeXZ+qjl7EpaxNNxjVh5OcjyQnkhPV5XmxXbmUNlu9qP0NlertvUVdCL01RxYvFHCeV9e/vEb/8+Ytc9fpVQhpy1vizZP2v64t9jxUlWKtPRq0soXtx10SrXBwQ7l5U2N7rxYo6p5WlKiYPCwQC8vr616X2yNoSNzROhi0eJodzDodcvrS7rh0JM1T3hoSEkn2u13ZNTehOCKeoEmrvDVV/HO3N9ZT3skWenft2yvWzrxfSkKbjmsrqX1YXulxYCbmI72hXWWb69MI7IrtduraCJnQnhLNnhlqmrHao8TInEq0Xz+cLeHvT21L3mboS+3isDFkwRA5mHzxmmSI3VTHf0c6LjtF64qsJ3Qnh/DiLas3h8R92meJUovVJxtm1f5f0eLuHkIYkj06WL7Z+Ef6bi/mOdm4CL7ZQsYImdKcUV6orau/16Kl3meRUonUj45RiP/vgmw8k8blEiXk8RgZ9PEj2H95f/JuK+Y52Hjt9cryMmCZ0r/DBKbYSZwY0KaozlV0Zx4L9748Df0ifOX2ENOS0l06TpT8uLfoNYWRVu8oy0fpz04TuJVoS9z47i3bFjTBpZ8ax8Ht98v0nUv/5+mLSjNw19y7Zd2hf4QtOny4SH3/0+uLjHdvvo/HnpgldqUi4UQ+Qv+rNLhafeew9tFfunHunkIY0eL6BLNi84NiFpk8XiYs7en1xcdGRWV1SVEIPq6eoMaaDMeZrY8x3xpjBRSx3rTFGjDGFTmCqlC/Y2XMzVHdFY+zvEWxxt8cq8VV48bIX+aznZ8TGxHLR1Ivo934//jz05/8WGjIEsrOPfmN2tk5RZJNiE7oxJhYYDVwGJAPdjDHJhSxXFbgb+MLqIJWKWGmH27NrSEA3+5LbNJRi6yreiC8AABM9SURBVKTWrO23ln+d+y8mrJ5AozGN+Oi7j4Iv6gTqjioXxjJnA9+JyGYAY8zrwJVAZoHlhgFPA4NKGkx2djZbt27l4MGDJf0IFYEKFSqQmJhIXFyc26FY68iYp0cGGTky5im4PzbME08cHRs4N//ske8+ZEgwodarF1yvBdukUlwlnrnkGbokd6HXu724bMZl9Gzak4mJJxH789Zj3+C1wVBmzLBluzguVF3MkRvQBZiY7/HNwKgCyzQH3sq7vwhILe5zC6tD37x5s2RlZUkgELCn8kn9LRAISFZWlmzevNntUKzn9fZq0XilLp+D2QflofkPSezjsdK/W3XJrlDeuQu/JeGz5jDYOdqiMSYGeA74VxjL9jXGpBtj0rOyso55/eDBgyQkJGCMKW1YqhjGGBISEqLzbMjrp/lOzPDgovLlyvPERU/wZZ8v+fyCJG65/BBZtSshNo8kWWKRDhlp1ewZNggnoW8DTs73ODHvuSOqAo2ARcaYLUBLYE5hF0ZFZLyIpIpIap06dQpdmSZz50TttvbdmKfRqfkJzVnZZyVn3DWUk+7K5vinazP7g38jN97odmhHi6QA4PEpjMJJ6CuBhsaYBsaYeOAGYM6RF0Vkj4jUFpH6IlIfWAF0FpF0WyJWqjhOzKOmwhIfG88jbR5hVd9VJNVI4ro3r6PL7C7s2LfD7dD+J5ICgBcHgM+n2IQuIjnAHcA8YBMwS0Q2GmOGGmM62x2gl23ZsoWZM2f+/TgjI4O5c+f+/XjOnDmMGDHCknX17NmTN998E4DevXuTmVnwmrT6m4MTRqjwnHn8mSy/bTkjLhrBB998QMqYFKavm37kGpy7IikAeL06L1Tlut23wi6KZmZmWnHNwDGffvqpdOzY8e/Hr7zyigwcONCWdfXo0UNmz55t+ef6bZsr/9uUtUnOnXiukIZ0mtlJtu7Z6nZI4V+o9sAFd4q4KBpOs0VX/POjf5KxI8PSz2xatynPd3i+yGWmTp3KM888gzGGxo0bM23aNHr27EmnTp3o0qULAFWqVGHfvn0MHjyYTZs20bRpU7p168bo0aM5cOAAS5cu5cEHH+TAgQOkp6czatQoevbsSbVq1UhPT2fHjh2MHDmSLl26EAgEuOOOO1i4cCEnn3wycXFx9OrV6+91FebCCy/kmWeeITU1lSpVqnD33Xfz/vvvU7FiRd59912OP/54srKy6NevHz/llRyef/55WrVqZd3GVKqETq99OktuXcJLX77EQwseInlMMs9d8hy9mvVy77pO9+7hncG52ew0DDqnaD4bN25k+PDhLFy4kLVr1/LCCy8UufyIESNo3bo1GRkZPPDAAwwdOpTrr7+ejIwMrr/++mOW3759O0uXLuX9999n8OBgh9v//ve/bNmyhczMTKZNm8by5csjivmvv/6iZcuWrF27lgsuuIAJEyYAcPfdd3PPPfewcuVK3nrrLXr37h3R5/qSh1sfqKPFxsTyz5b/ZH3/9TSr24ze7/Xm0umX8uMfP7odWtE8Xp3n2RJ6cSVpOyxcuJCuXbtSu3ZtAGrVqmXp51911VXExMSQnJzMr7/+CsDSpUvp2rUrMTEx1K1bl7Zt20b0mfHx8XTq1AmAFi1a8MknnwAwf/78o+rZ//zzT/bt20eVKlUs+jYe4+XORCqkU2udysIeC3k5/WXun38/jcY24umLn6Zfaj9ijEfLm+GW5l3g0S3mLeXKlSMQCAAQCAQ4fPhwiT6nfPnyf98Xiy4GxcXF/X2aGhsbS05OcLb2QCDAihUryMjIICMjg23btkVvMgfPtz5QocWYGPqf1Z8N/TdwbuK5DJw7kHZT2vHd7u/cDs13NKHn065dO2bPns2uXbsA2L17NwD169dn1apVQLDlSnbeYENVq1Zl7969f7+/4ONwtGrVirfeeotAIMCvv/7KokWLLPgmcMkll/DSSy/9/Tgjw9rrEZ7j9dYHqlhJNZKYd9M8JnWeRMaODBqPbcx/lv+H3ECu26H5hib0fFJSUhgyZAht2rShSZMm3HvvvQD06dOHxYsX06RJE5YvX07lypUBaNy4MbGxsTRp0oT//Oc/tG3blszMTJo2bcobb7wR1jqvvfZaEhMTSU5O5qabbqJ58+ZUr1691N/lxRdfJD09ncaNG5OcnMy4ceNK/Zmepp2JooIxhl7NerFxwEbaNWjHvR/fS+tXWvPVb1+5HZo/hGr+YvctGpotWmXv3r0iIvLbb7/JKaecItu3b3ds3VGzzX02HocqXiAQkOlrp0utp2tJ+WHl5aklT0l2brbbYbkOO8dyUaXXqVMnmjZtSuvWrXnkkUeoW7eu2yH5j8dbH6jIGWPo3rg7mQMy6XRaJx5c8CAtJ7Zk3a/r3A7Ns4y41FMrNTVV0tOPHh1g06ZNnHHGGa7EU1bpNld+MXvjbAbOHcgfB/9gSOshPNj6QeJj490Oy3HGmFUiUugkQlpCV0r5QteUrmQOzKRrSlfSFqdx1oSzWL19tdtheYomdKWUb9SuVJsZ18zg3RveJeuvLM6ecDYPLXiIgzlROAx0CWhCV0r5Tuf/68zGARu5pcktPLX0KZq/3JwVW1e4HZbrNKErpXypZsWaTL5yMh91/4h9h/fRanIr7vv4PvZn7y/+zVHK3wldx+5QKnJR9ru59B+XsmHABvo278uzy5+lybgmfPbjZ26H5Qr/JnSXZw6ZMmUKDRs2pGHDhkyZMsWRdSpVah6fcaekqpWvxthOY1lwywJyA7m0ebUNd8y9g32H97kdmqP822yxfv3gzlhQUlJwnkYb7d69m9TUVNLT0zHG0KJFC1atWkXNmjVtXa8dtNliGePi78Ypfx3+i4cWPMRLX75EUo0kJlwxgYtPudjtsCwTnc0WbRi7Y+XKlTRu3JiDBw/y119/kZKSwoYNG45Zbt68ebRv355atWpRs2ZN2rdvz0cffVTi9SrlmDIw5k3l+Mq8cNkLLLl1CfGx8bSf1p6+7/Vlz8E9bodmO/8mdBvG7jjrrLPo3LkzDz/8MPfffz833XQTjRo1Oma5bdu2cfLJ/5s3OzExkW3bth2znFKeU4bGvGlVrxUZt2cw6LxBTFoziUZjGzH327nFv9HH/JvQbZoI+NFHH+WTTz4hPT2d+++/v1SfpZTnlLEJtCvGVWRk+5Esv2051cpXo+PMjvR4pwe7D+x2OzRb+Deh2zR2x65du9i3bx979+7l4MHCOyucdNJJ/Pzzz38/3rp1KyeddFKp1quUI8romDdnn3Q2q/uu5uHWDzNj3QySRyfz9qa33Q7Lcv69KGqTzp07c8MNN/DDDz+wfft2Ro0adcwyu3fvpkWLFqxeHex23Lx5c1atWmX5DEdO8MI2V8pJa7avodecXmTsyOC6lOsYddko6lSu43ZYYYvOi6I2mDp1KnFxcdx4440MHjyYlStXsnDhwmOWq1WrFo888ghnnXUWZ511Fo8++qgvk7lSZVGzE5rxZe8vGd52OG9vepvkMcm8vuF1y2YRc5OW0Ms43eaqLNu4cyO3vnsrK39ZyVWnX8WYy8dwQtUT3A6rSFpCV0qpQqQcl8Ky25Yx8uKRfPjth6SMSWHq2qm+La1rQi/C+vXradq06VG3c845x+2wlFIWKhdTjkGtBrG231qS6yTT450edJzZkZ/3/Fz8mz2mnNsBeNmZZ54Z/ZMrK6UA+L/a/8dnt37GqC9H8eCCB0kZk8KzlzxL7+a9Mca4HV5YtISulFJ5YkwMd51zF+v7ryf1xFT6vt+X9tPa88PvP7gdWlg0oSulVAGn1DyF+bfMZ1zHcXy57UvOHHsmo74cRUACbodWJE3oSilViBgTw+2pt7NhwAbOr3c+d354J21ebcO3u751O7SQfJ3Qo2xYZ6WUB9WrXo8Pu3/IK1e+woadG2g8rjHPLHuG3ECu26Edw7cJ3e1hnTt06ECNGjXo1KmTMytUSrnGGEPPpj3ZOGAjl5x6CYM+GcR5k88jMyvT7dCO4tuEPmQI7C8w09T+/cHnnTBo0CCmTZvmzMqUUp5wYtUTeef6d5h5zUy+3/09zV5uxpNLniQ7N9vt0AAfJ3Q7hnUOdzx0gIsuuoiqVauWfGVKKV8yxtDtzG5kDszkyv+7kiELh3DOxHNYu2Ot26H5N6HbMaxzuOOhK6XUcZWPY1bXWbzZ9U227d1G6oRUHvv0MQ7nHnYtJt8mdLuGddbx0JVSkbg2+VoyB2TSrVE3hn42lBbjW5D+S3rxb7SBbxO6XcM6hzMeulJK5ZdQKYGpV0/lvW7vsfvAbs6ZeA6D5w/mYI6zOcS3CR2CyXvLFggEgn+tGKP/9ttvZ9iwYXTv3p0HHnig9B+olCozOp3WiY0DNnJr01t5+vOnaTquKct+XubY+n2d0K0W7njoAK1bt6Zr164sWLCAxMRE5s2b53C0SikvqlGhBhM7T+Tjmz7mQM4Bzp98PmmL0hxZd1jjoRtjOgAvALHARBEZUeD1e4HeQA6QBfQSkR+L+kwdD90bdJsrZZ+9h/YyeP5gWia25OYmN1vymUWNh17saIvGmFhgNNAe2AqsNMbMEZH8LerXAKkist8Y0x8YCVxf+tCVUsq/qpavyuiOox1bXzjD554NfCcimwGMMa8DVwJ/J3QR+TTf8iuAm6wM0i3r16/n5puPPqqWL1+eL774wqWIlFIqtHAS+klA/pHetwJFzfJwG/BhYS8YY/oCfQHqhWgwLiKeGXs42sdD9+usLEqpwll6UdQYcxOQCvy7sNdFZLyIpIpIap06x86yXaFCBXbt2qWJxgEiwq5du6hQoYLboSilLBJOCX0bcHK+x4l5zx3FGHMxMARoIyKHShJMYmIiW7duJSsrqyRvVxGqUKECiYmJboehlLJIOAl9JdDQGNOAYCK/Abgx/wLGmGbAy0AHEdlZ0mDi4uJo0KBBSd+ulFJlWrFVLiKSA9wBzAM2AbNEZKMxZqgxpnPeYv8GqgCzjTEZxpg5tkWslFKqUGFNEi0ic4G5BZ57NN/9iy2OSymlVIS0p6hSSkWJsHqK2rJiY7KAInuTFqE28JuF4VhF44qMxhU5r8amcUWmNHElicixzQRxMaGXhjEmPVTXVzdpXJHRuCLn1dg0rsjYFZdWuSilVJTQhK6UUlHCrwl9vNsBhKBxRUbjipxXY9O4ImNLXL6sQ1dKKXUsv5bQlVJKFaAJXSmlooSnE7oxpoMx5mtjzHfGmMGFvH6vMSbTGLPOGLPAGJPkkbj6GWPW5w2DsNQYk+yFuPItd60xRowxjjTnCmN79TTGZOVtrwxjTG8vxJW3zHV5+9hGY8xML8RljPlPvm31jTHmD4/EVc8Y86kxZk3eb/Jyj8SVlJcf1hljFhljHBmRzhgz2Riz0xizIcTrxhjzYl7c64wxzUu9UhHx5I3gdHffA6cA8cBaILnAMm2BSnn3+wNveCSuavnudwY+8kJcectVBT4jOBFJqhfiAnoCozy4fzUkOBtXzbzHx3khrgLL3wlM9kJcBC/09c+7nwxs8Uhcs4EeeffbAdMc2scuAJoDG0K8fjnBuSMM0BL4orTr9HIJ/e+ZkkTkMHBkpqS/icinIrI/7+EKgkP7eiGuP/M9rAw4ceW52LjyDAOeBg46EFMkcTktnLj6AKNF5HcAKcVIohbHlV834DWPxCVAtbz71YFfPBJXMnBktvdPC3ndFiLyGbC7iEWuBKZK0AqghjHmhNKs08sJvbCZkk4qYvmQMyVZLKy4jDEDjTHfE5xf9S4vxJV3SneyiHzgQDxhx5Xn2rzTzjeNMScX8robcZ0GnGaM+dwYsyJvsnQvxAUEqxKABvwvWbkdVxpwkzFmK8HB/O70SFxrgWvy7l8NVDXGJDgQW3EizXHF8nJCD1txMyW5QURGi8ipwAPAw27HY4yJAZ4D/uV2LIV4D6gvIo2BT4ApLsdzRDmC1S4XEiwJTzDG1HA1oqPdALwpIrluB5KnG/CqiCQSrE6Ylrffue0+oI0xZg3QhuC8Dl7ZZpbywsYOJdKZkjpLCWdKsiOufF4HrrI1oqDi4qoKNAIWGWO2EKyzm+PAhdFit5eI7Mr3v5sItLA5prDiIlhimiMi2SLyA/ANwQTvdlxH3IAz1S0QXly3AbMARGQ5UIHgIFSuxiUiv4jINSLSjGCuQEQcuZBcjEhzSfGcuDhQwgsK5YDNBE8pj1zsSCmwTDOCF0QaeiyuhvnuXwGkeyGuAssvwpmLouFsrxPy3b8aWOGRuDoAU/Lu1yZ4epzgdlx5y50ObCGvc6BHtteHQM+8+2cQrEO3Nb4w46oNxOTdfwIY6sQ2y1tffUJfFO3I0RdFvyz1+pz6YiXcGJcTLBV9DwzJe24owdI4wHzgVyAj7zbHI3G9AGzMi+nTohKrk3EVWNaRhB7m9noqb3utzdtep3skLkOwmioTWA/c4IW48h6nASOciCeC7ZUMfJ73f8wALvFIXF2Ab/OWmQiUdyiu14DtQDbBs73bgH5Av3z71+i8uNdb8XvUrv9KKRUlvFyHrpRSKgKa0JVSKkpoQldKqSihCV0ppaKEJnSllIoSmtCVUipKaEJXSqko8f/NqGKsPWvd/gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出更新之后的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0.numpy() * plot_x - b0.numpy()) / w1.numpy()\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到更新之后模型已经能够基本将这两类点分开了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面我们使用了自己写的 loss，其实 PyTorch 已经为我们写好了一些常见的 loss，比如线性回归里面的 loss 是 `nn.MSE()`，而 Logistic 回归的二分类 loss 在 PyTorch 中是 `nn.BCEWithLogitsLoss()`，关于更多的 loss，可以查看[文档](http://pytorch.org/docs/0.3.0/nn.html#loss-functions)\n",
    "\n",
    "PyTorch 为我们实现的 loss 函数有两个好处，第一是方便我们使用，不需要重复造轮子，第二就是其实现是在底层 C++ 语言上的，所以速度上和稳定性上都要比我们自己实现的要好\n",
    "\n",
    "另外，PyTorch 出于稳定性考虑，将模型的 Sigmoid 操作和最后的 loss 都合在了 `nn.BCEWithLogitsLoss()`，所以我们使用 PyTorch 自带的 loss 就不需要再加上 Sigmoid 操作了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用自带的loss\n",
    "criterion = nn.BCEWithLogitsLoss() # 将 sigmoid 和 loss 写在一层，有更快的速度、更好的稳定性\n",
    "\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_reg(x):\n",
    "    return torch.mm(x, w) + b\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], 1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6363)\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_reg(x_data)\n",
    "loss = criterion(y_pred, y_data)\n",
    "print(loss.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 200, Loss: 0.39427, Acc: 0.88000\n",
      "epoch: 400, Loss: 0.32362, Acc: 0.87000\n",
      "epoch: 600, Loss: 0.29014, Acc: 0.87000\n",
      "epoch: 800, Loss: 0.27045, Acc: 0.87000\n",
      "epoch: 1000, Loss: 0.25743, Acc: 0.88000\n",
      "\n",
      "During Time: 0.372 s\n"
     ]
    }
   ],
   "source": [
    "# 同样进行 1000 次更新\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "    # 前向传播\n",
    "    y_pred = logistic_reg(x_data)\n",
    "    loss = criterion(y_pred, y_data)\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    # 计算正确率 0.5以上的判断为正确\n",
    "    mask = y_pred.ge(0.5).float()        \n",
    "    acc = (mask == y_data).sum().item() / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.item(), acc))\n",
    "\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，使用了 PyTorch 自带的 loss 之后，速度有了一定的上升，虽然看上去速度的提升并不多，但是这只是一个小网络，对于大网络，使用自带的 loss 不管对于稳定性还是速度而言，都有质的飞跃，同时也避免了重复造轮子的困扰"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一节课我们会介绍 PyTorch 中构建模型的模块 `Sequential` 和 `Module`，使用这个可以帮助我们更方便地构建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}