{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic 回归模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上一节课我们学习了简单的线性回归模型，这一次课中，我们会学习第二个模型，Logistic 回归模型。\n",
    "\n",
    "Logistic 回归是一种广义的回归模型，其与多元线性回归有着很多相似之处，模型的形式基本相同，虽然也被称为回归，但是其更多的情况使用在分类问题上，同时又以二分类更为常用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型形式\n",
    "Logistic 回归的模型形式和线性回归一样，都是 y = wx + b，其中 x 可以是一个多维的特征，唯一不同的地方在于 Logistic 回归会对 y 作用一个 logistic 函数，将其变为一种概率的结果。 Logistic 函数作为 Logistic 回归的核心，我们下面讲一讲 Logistic 函数，也被称为 Sigmoid 函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid 函数\n",
    "Sigmoid 函数非常简单，其公式如下\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$\n",
    "\n",
    "Sigmoid 函数的图像如下\n",
    "\n",
    "![](https://ws2.sinaimg.cn/large/006tKfTcly1fmd3dde091g30du060mx0.gif)\n",
    "\n",
    "可以看到 Sigmoid 函数的范围是在 0 ~ 1 之间，所以任何一个值经过了 Sigmoid 函数的作用，都会变成 0 ~ 1 之间的一个值，这个值可以形象地理解为一个概率，比如对于二分类问题，这个值越小就表示属于第一类，这个值越大就表示属于第二类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外一个 Logistic 回归的前提是确保你的数据具有非常良好的线性可分性，也就是说，你的数据集能够在一定的维度上被分为两个部分，比如\n",
    "\n",
    "![](https://ws1.sinaimg.cn/large/006tKfTcly1fmd3gwdueoj30aw0aewex.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，上面红色的点和蓝色的点能够几乎被一个绿色的平面分割开来"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 回归问题 vs 分类问题\n",
    "Logistic 回归处理的是一个分类问题，而上一个模型是回归模型，那么回归问题和分类问题的区别在哪里呢？\n",
    "\n",
    "从上面的图可以看出，分类问题希望把数据集分到某一类，比如一个 3 分类问题，那么对于任何一个数据点，我们都希望找到其到底属于哪一类，最终的结果只有三种情况，{0, 1, 2}，所以这是一个离散的问题。\n",
    "\n",
    "而回归问题是一个连续的问题，比如曲线的拟合，我们可以拟合任意的函数结果，这个结果是一个连续的值。\n",
    "\n",
    "分类问题和回归问题是机器学习和深度学习的第一步，拿到任何一个问题，我们都需要先确定其到底是分类还是回归，然后再进行算法设计"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 损失函数\n",
    "前一节对于回归问题，我们有一个 loss 去衡量误差，那么对于分类问题，我们如何去衡量这个误差，并设计 loss 函数呢？\n",
    "\n",
    "Logistic 回归使用了 Sigmoid 函数将结果变到 0 ~ 1 之间，对于任意输入一个数据，经过 Sigmoid 之后的结果我们记为 $\\hat{y}$，表示这个数据点属于第二类的概率，那么其属于第一类的概率就是 $1-\\hat{y}$。如果这个数据点属于第二类，我们希望 $\\hat{y}$ 越大越好，也就是越靠近 1 越好，如果这个数据属于第一类，那么我们希望 $1-\\hat{y}$ 越大越好，也就是 $\\hat{y}$ 越小越好，越靠近 0 越好，所以我们可以这样设计我们的 loss 函数\n",
    "\n",
    "$$\n",
    "loss = -(y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "其中 y 表示真实的 label，只能取 {0, 1} 这两个值，因为 $\\hat{y}$ 表示经过 Logistic 回归预测之后的结果，是一个 0 ~ 1 之间的小数。如果 y 是 0，表示该数据属于第一类，我们希望 $\\hat{y}$ 越小越好，上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = - (log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "在训练模型的时候我们希望最小化 loss 函数，根据 log 函数的单调性，也就是最小化 $\\hat{y}$，与我们的要求是一致的。\n",
    "\n",
    "而如果 y 是 1，表示该数据属于第二类，我们希望 $\\hat{y}$ 越大越好，同时上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = -(log(\\hat{y}))\n",
    "$$\n",
    "\n",
    "我们希望最小化 loss 函数也就是最大化 $\\hat{y}$，这也与我们的要求一致。\n",
    "\n",
    "所以通过上面的论述，说明了这么构建 loss 函数是合理的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们通过例子来具体学习 Logistic 回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x108f3c5f0>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设定随机种子\n",
    "torch.manual_seed(2017)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们从 data.txt 读入数据，感兴趣的同学可以打开 data.txt 文件进行查看\n",
    "\n",
    "读入数据点之后我们根据不同的 label 将数据点分为了红色和蓝色，并且画图展示出来了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x108137c50>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH8JJREFUeJzt3X+MXeV95/H317/iTuPG2J6krcfMDKrT8CMkisdui5oV\nCkJ2frp0kRaYhC6p6ji7RNk/FkGbQFdC1lKtVmpo0lTeJA0woyC0Sbp0l2CkoDbaqg0eWgM2hsg4\nwR53JYYxSmJ+yLH57h/njrlz5/4498758TznfF7S1XjuHN/7veec+z3P+T7PeY65OyIiUi0ryg5A\nRESyp+QuIlJBSu4iIhWk5C4iUkFK7iIiFaTkLiJSQUruIiIVpOQuIlJBSu4iIhW0qqw33rRpk4+N\njZX19iIiUXryySdfdvfhXsuVltzHxsaYmZkp6+1FRKJkZi+mWU5lGRGRClJyFxGpICV3EZEKUnIX\nEakgJXcRkQrqmdzN7Btm9pKZHe7wdzOze83smJk9bWYfyD7MDE1Pw9gYrFiR/JyeLjsiEZHMpWm5\nfxPY1eXvHwa2Nh57gK8uP6ycTE/Dnj3w4ovgnvzcs0cJXkQqp2dyd/cfAKe7LLIbuN8T/wSsN7Nf\nyyrATH3hC/Daa4ufe+215HkRkQrJoua+GTjZ9Pts47klzGyPmc2Y2czc3FwGb92nEyf6e15EJFKF\ndqi6+353n3D3ieHhnlfPZu/ii/t7PiMq89eTtruUKYvkfgrY0vT7SOO58OzbB0NDi58bGkqez4nK\n/PWk7S5lyyK5Pwzc3Bg189vAT939/2XwutmbnIT9+2F0FMySn/v3J8/nRGX+8pTZctZ2l7KlGQr5\nLeAfgd80s1kz+0Mz22tmexuLPAIcB44B/wP4D7lFm4XJSfjJT+DNN5OfOSZ2UJm/SM3JfNMm+PSn\ny2s5x7DdVTaqOHcv5bFt2zavg9FR9yS9LH6MjpYdWbVMTbkPDbVf12Ws90G3+9RUsoxZ8nNqKp/4\n2q2voaH83k+yA8x4ihyrK1RzVkKZv5balUHaKarlPMh2L7JOr7JR9VU3uQdyzllCmb+W0ibtnAdG\nXTDIdi8y4cZQNpLlsaSVX7yJiQnP7WYdC02g5m/K0JCyaoWNjSUt3W5C3wVWrEha7K3Mki6iLHVa\nX6OjSVeUhMvMnnT3iV7LVbPlrnPO2mlXBlm9GjZujOeMqcjLMFQurL5qJnedc9ZOuzLIX/81vPxy\nYQOjlq3IhKtyYfVVsyyjc06J1PR0coJ54kTSYt+3TwlXFqt3WUbnnBKpgi/DkAqrZnLXOaeI1Fw1\nkzuoCSTBCmSUrlTcqrIDEKmT1lG6Cxcqgdofkq3qttxFAqRRulIUJXeRAsU0Slflo7gpuYsUqKT7\nxfRN89HHT8ldpECxjNJV+Sh+Su4iBYpllG5M5SNpT6NlRAo2ORleMm+1YQPMz7d/XuKglrtIBajz\nU1qp5S4SuTzGzp8+3d/zEh613EUi1NxS/4M/yL7zM5ZRPdJZquRuZrvM7HkzO2Zmd7T5+0Vm9l0z\ne9rMnjCzK7IPVWKg8kD+Wocpnj/ffrnldH7GMqpHOuuZ3M1sJfAV4MPAZcCNZnZZy2J/Ahxy9yuB\nm4EvZR2ohE9jo4uR9n6xy2llxzKqRzpL03LfARxz9+PufhZ4ENjdssxlwOMA7v4cMGZm78o0UulL\nGS1ojY0uRpoWeRat7CrMvVfnM8k0yX0zcLLp99nGc82eAn4fwMx2AKPASBYBSv/KakFrbHQxOrXI\nV65UK7tZ3c8ks+pQvQdYb2aHgM8B/wIsqQSa2R4zmzGzmbm5uYzeWlqV1YJWJ1wxOtXD77sv7lZ2\n1up+JpkmuZ8CtjT9PtJ47gJ3/5m73+Lu7yepuQ8Dx1tfyN33u/uEu08MDw8vI2zppqwWtDrhitGt\nHl7nMkTrZ293p02o0Zmku3d9kIyFPw6MA2tISjCXtyyzHljT+PcfAff3et1t27a5JKam3EdH3c2S\nn1NTy3u90VH35ER08WN0dPmx9pL1Z5H0pqbch4YWb/OhoXpsg3af3ay870GegBnvkV89+agpFoKP\nAD8CXgC+0HhuL7C38e/fafz9eeA7wEW9XlPJPZHHF7LOX/I6K/OgXrZOn701wef5PSiqYZNpcs/j\noeSeyOsLqRZ0/XRqqZqVHVn+On32he9S3t+DIhtUaZO7JcsWb2JiwmdmZkp575CsWJHsCq3Mks4x\n6Wx6OukcO3Ei6bTdt6/eHYmd6syjo0kna5WV/dmLfH8ze9LdJ3otp+kHSqYRJoOp+zC3ZgsdiS++\nmDQKmtWlQ7vszvwQhwEruZes7J0yViEOcytjpErzQQ6SA91Cgq/TePeyr6gNspGWpnaTx0M197eo\nPt6/0OrLZXVi17kTNSSquTdRzV2Wo+waa6si42nua+j09VWfTfGK6gNSzV0qLbRyVlE119a+hk7U\nZ1O80ObiUXKXKJVdY21VVM01zYyQ6rMRUHKXiIXUUirqTKLbmUAIBzkJh26zJ5KBhWSad8314ovD\n6muQcKnlLpKRIs4kQutrkHApuYtEJLS+BgmXyjIikZmcVDKX3tRyFxGpICV3EZEKUnIXEakgJXep\ntTrflk6qTR2qUlsLl/IvXPG5MG0wqMNS4qeWu9RWiNMGi2RFyV1qK8QbLIhkRcldaivIGyyIZCRV\ncjezXWb2vJkdM7M72vz9HWb2t2b2lJkdMbNbsg9VJFu6lF+qrGdyN7OVwFeADwOXATea2WUti/1H\n4Fl3fx9wNfDfzWxNxrFWk4ZrlEaX8kuVpWm57wCOuftxdz8LPAjsblnGgXVmZsDbgdPAuUwjrSLd\n5bl0IUwbrOO75CFNct8MnGz6fbbxXLMvA5cC/wo8A3ze3XWTr140XKP2dHyXvGTVoboTOAT8OvB+\n4Mtm9iutC5nZHjObMbOZubm5jN56GcpuMmm4RunK3gV0fJe8pEnup4AtTb+PNJ5rdgvwncbNuY8B\nPwbe0/pC7r7f3SfcfWJ4eHjQmLMRQpNJwzVKFcIuoOO75CVNcj8IbDWz8UYn6Q3Awy3LnACuATCz\ndwG/CRzPMtDMhdBk0nCNUoWwC+j4Ho6yz+Ky1jO5u/s54FbgAHAUeMjdj5jZXjPb21jsbuAqM3sG\n+D5wu7u/nFfQmQihyaThGqUKYRfQ8T0MIZzFZc7dS3ls27bNSzU66p5sx8WP0dFy45LChLILTE0l\n72mW/JyaKvb9BxVr3O2Esi+kAcx4ihxb3ytU82wyVe38rqJCaTWHMByzX1Vr6YZwFpe5NEeAPB6l\nt9zd82l6TE25Dw0tPvwPDUXTrAmxNZZnTCF+3hjE1NJNI6bPQ8qWe72Tex5i2ktahHhcCjGmqhnk\nAGfWfjc3yzvafMS0nym5d5Nncy3ivT7E41KIMVXJoEmtitsllrO4tMndkmWLNzEx4TMzM8W/cesd\nGiAptGY1SmVsLClAthodTQqqAVuxIvmKtjJL6sFlCDGmKhl0d837aySdmdmT7j7Ra7n6dajmPbg5\nlF66AYQ45jrEmKpk0I5EjeINX/2Se97d4hHv9SEel0KMqUo6HSRXrOg92CvGUT61kqZ2k8ejtJp7\nFYuFGQqx7hhiTFXRrube+gi1Y7GuUM29AxULRRaZnk6qkidOJK318+eXLhNBl1FtqObeSZqyiS5C\nioY21fI1l1c6dVJHfTFPTdWv5d6LWvbRaLep1qyBdevg9OmknrxvnzZbPyIe7FUbarkPKoSpAiWV\ndpvq7FmYn6/GJfFlUAd2voo801Ryb1XJSSaqKc0m0XG5PxEP9gpe0fPxqCzTSuel0ei0qVrpgicJ\nQVapRWWZQWV9Xqoev9y021Tt6IInCUHRRQEl91ZZnpdWbV7UwLRuqo0bYfXqxcuoXiyhKPpqa5Vl\n8qQST+Gax2xrtIyEJKuBeCrLhECds4XTJfFLqTIYhqI7q5Xc86RZr0pXtcTW7+epWmUw9u1ZaOMj\nzRwFwC7geeAYcEebv98GHGo8DgPngQ3dXrOyN+toFtMdAFKIbY6Xiq3+gT5PlaZSqtr2HBRZ3awD\nWAm8AFwCrAGeAi7rsvzHgcd7vW4tkrt7fBmxgxi/WFVKbO6DfZ6I7x2zRNW256DSJvc0ZZkdwDF3\nP+7uZ4EHgd1dlr8R+FY/Zw+VVpEicIwX7laty2OQz9OpAugeX1kj5O3Zq1xURjkpTXLfDJxs+n22\n8dwSZjZEUsL59vJDk5CE/MXqpGpdHoN8nm7XAsRWfw91e/bq1yir3yPrDtWPA//g7qfb/dHM9pjZ\njJnNzM3NZfzWkqdQv1jdVG2elEE+T/MIjXZCP/tqFur27HVWW9pZb6+6DfA7wIGm3/8Y+OMOy34X\nuClNPag2NfeKiLHm7l6ZLo8LlvN5qlB/D3F79lqvWa93srpZh5mtAn4EXAOcAg42EviRluXeAfwY\n2OLur/Y6qNTiIqaK0QVCcdM1dfnotV6zXu+ZXcTk7ueAW4EDwFHgIXc/YmZ7zWxv06LXAY+lSewS\np4r0DddWqGWN2PVar6Wt9zTN+zweKsuIFC/EskYV9FqvWa53dA/VClJdJBjaFFKWtGWZVUUEIxlo\nnXVoYTwVKKsUTJtCYqC5ZWIR41VEFaVNEYbY55nJm5J7LGK8iqiitCnKl9eFQVU6YCi5Z6GIPSLG\nq4gqSpuifHmcPVVtBk0l9+Uqao+IfBxblVpEkW+KSsjj7Kly5bY0Q2ryeFRmKGSRU9VFOo4t1qtb\nu4l0U1RGHl+7WK7gRUMhC7JiRbIPtDJLrvYRXRkpmcvqlnXNYtlPdZu9oqgA25M6ICVredyyrmrl\nNiX35araHpEDHf8kD1lPh1H0PU7zpuS+XFXbI3oZoGdUxz+JRZXmT9IVqlmYnIx7L0hrwEszF/6k\ny/VFiqMOVUkvlh4nkQpTh6pkTz2jItFQcpf01DMqEg0ld0lPPaMi0VByl/TqNjKoRZWmUJDilLXf\naLSM9Kd16MvCxBsVT/Caw10GUeZ+U++Wu5pi/ava1HkpVW5SKSlEmftNfZN7TZNUW/0c5Gqa5TRQ\nSAZR5n6TKrmb2S4ze97MjpnZHR2WudrMDpnZETP7+2zDzEFNk9QS/R7kctpbQz+J0kAhGUSp+02v\naSOBlcALwCXAGuAp4LKWZdYDzwIXN35/Z6/XLX3K31jm98xbv3On5jDXagxTAscQo4Qnj/2GlFP+\npmm57wCOuftxdz8LPAjsblnmJuA77n6iccB4abkHndypKZbotyWew3DIGE6iaj5QSAZU5n6TJrlv\nBk42/T7beK7Zu4GLzOzvzOxJM7s5qwBzozHbiX4PcjnsrbHUs6s0qZQUp6z9JqsO1VXANuCjwE7g\nTjN7d+tCZrbHzGbMbGZubi6jtx6QmmKJQQ5yGe+tOokSyV6a5H4K2NL0+0jjuWazwAF3f9XdXwZ+\nALyv9YXcfb+7T7j7xPDw8KAxZ0dNsSAOcjqJEslemuR+ENhqZuNmtga4AXi4ZZn/Bfyuma0ysyHg\nt4Cj2YYquSn5IBfA8UWkcnomd3c/B9wKHCBJ2A+5+xEz22tmexvLHAUeBZ4GngC+5u6H8wtbqkYn\nUfUR+rDXqtB87iJSmDxubF03ms+9ytT0kcB12kVjGPZaFZo4LDaawUoC120XjWXYaxWoLBMb3epO\nAtdtFwXtvsulskxVqekjgeu2i2rYa3GU3GOjK34kcN12UQ17LY6Se2zU9JHA9dpFNey1GErusVHT\nRwKnXTQM6lAVEYmIOlRFRGpMyV1EpIKU3EVEKkjJXSQHmiFCyqbkLvmpaYbr957jInlQcq+rvBNv\nKBmuhAOMJseSEGgoZB0VMe9qCHPglDS/7IoVyfGslVly4Y7IcqQdCqnkXkdFJN4QMlxJB5gQjmtS\nXRrnLp0VMflYCHPglDTJmmaIkBAouddREYk3hAxX0gFGl99LCJTc66iIxFt2hpuehjNnlj5f0AFG\nk2NJ2ZTc66hb4s1ydElZGW6hI3V+fvHzGzeqCS21kSq5m9kuM3vezI6Z2R1t/n61mf3UzA41Hndl\nHyq1HTedi3aJN5Thi8vVbiwiwNvfrsQutdFztIyZrQR+BFwLzAIHgRvd/dmmZa4G/rO7fyztG/c9\nWka3Tc9fVYZ5hDBSRyQnWY6W2QEcc/fj7n4WeBDYvdwA+6YrQ/LXaRRJu4QfshBG6oiULE1y3wyc\nbPp9tvFcq6vM7Gkz+56ZXd7uhcxsj5nNmNnM3Nxcf5Hq3qH565T8zOIqzYQwUqfGVD0NQ1Ydqv8M\nXOzuVwJ/AfxNu4Xcfb+7T7j7xPDwcH/voNZY/vbtSxJ5K/e4zpDKHqlTY1XptqmCNMn9FLCl6feR\nxnMXuPvP3P1M49+PAKvNbFNmUYJaY0WYnGxfq4b4zpA0FrEUqp6GI01yPwhsNbNxM1sD3AA83LyA\nmf2qWdLkM7MdjdedX/JKy6HWWDFGR9s/rzOkKBVdIlH1NBw9k7u7nwNuBQ4AR4GH3P2Ime01s72N\nxa4HDpvZU8C9wA2ex6Q1ebbGVChM6AypMsookah6GhB3L+Wxbds2D8bUlPvQkHvyHUgeQ0PJ83U0\nNeU+Oupulvz87GcX/17X9dJJ6/oKZP2Mji7epRceo6P5vae+SvkDZjxFjlVydy/nWxALfVu7K3n9\ndDuumLXfrc3Ki0mWL21y15S/oIteuqnKhU15KXH99LquT5uumjTlbz9UKGxverrzBUzqIUuU2IPY\na2SKuk8GV4UuOCV30LegnYVmYSd1P/AtKLFh0Ou4UocBZnkk4cqM1U9Tu8njEVTN3V2Fwlad+iFU\nc1+sxJp73buK8lr1oa9X1KEqy9KpNw6U2FuV1DDI87gSQ1snryRcVkd0WmmTuzpUpT31xkVhejqp\nsZ84kVSC9u1bftkllglY8xoHEfquX90O1ayKbFXoMcmT+iGikMd1fbFMIZBXd0dldv00zfs8HgOV\nZbI6D9XY7XRiODeXzGVdlshrN6prWYpK1tz7LbJ12kKh95iIlCjLr0fe7ajmr/jGjckjxIScpWom\n936aFN32qtB7TCR8ITftlinLhFxUO6pOJ+PVTO797CndllXLvTrKSLI1yCSpVmuKhYpqR9XpK13N\n5N7Pl6rbXlWDL2ctlLUd65RJOkm57otaVXU6Ga9mcndP31LrtVdV+LS6NspKsnXKJJ2kXPdFHX/r\ndLytbnJPS63z6isryYaSScpsoPSx7osIs05fdyV3d7XOq66sJBtCJik7hlAOcE3q8nVXcpfqKzrB\nhTTuruzkWvbBpcbSJvdVpV09JbJcC5djZn39fTut1+TPzyeXLT7wQDnX5Jd9s9KFz/z5zyfrAuCX\nfqmY95ZU4pt+QKRZnvfVbRbaNfmh3IPg9dff+vf8fKRz41ZTqonDzGwX8CVgJfA1d7+nw3LbgX8k\nuUH2/+z2mu0mDvvFL37B7Owsb7zxRsrww7Z27VpGRkZYvXp12aHIcoV2t64QZvcKfYatiko7cVjP\nsoyZrQS+AlwLzAIHzexhd3+2zXJ/Bjw2WMgwOzvLunXrGBsbw8wGfZkguDvz8/PMzs4yPj5edjiy\nXBdf3D6RlXXTkiJLUp2UXRqSrtKUZXYAx9z9uLufBR4EdrdZ7nPAt4GXBg3mjTfeYOPGjdEndgAz\nY+PGjZU5C8lcbLNyhjhVYFElqU5CKQ1JW2mS+2bgZNPvs43nLjCzzcB1wFeXG1AVEvuCKn2WTMV4\nH7M63LOuXyEe8NKKrXExgKw6VP8cuN3duxYfzWyPmc2Y2czc3FxGby3RCa1zMq2yW8qhifWAF2Pj\nYgBpkvspYEvT7yON55pNAA+a2U+A64G/NLPfa30hd9/v7hPuPjE8PDxgyE1KPPred999bN26la1b\nt3LfffcV9r6VoFptdcR4wBukcRFjS7/XQHiSTtfjwDiwBngKuLzL8t8Eru/1uu0uYnr22WfTj+Qv\n8SKK+fl5Hx8f9/n5eT99+rSPj4/76dOn2y7b12eqi7IvwJF663faisAu2CLlRUw9W+7ufg64FTgA\nHAUecvcjZrbXzPbmccBJJYdT+4MHD3LllVfyxhtv8Oqrr3L55Zdz+PDhJcsdOHCAa6+9lg0bNnDR\nRRdx7bXX8uijjw78vrUTc61W4tdvR3CkZcRUV6i6+yPAIy3P/VWHZf/98sNKIYdT++3bt/OJT3yC\nL37xi7z++ut88pOf5Iorrliy3KlTp9iy5a1K1cjICKdOtVaqpKMQhvFJfe3b1/4agU6Ni0jLiPFO\nP5DTuOO77rqL7du3s3btWu69995lvZZ0MTmpZC7l6LdxEdo1DinFO/1ATqf28/PznDlzhp///Ocd\nx6hv3ryZkyffGh06OzvL5s2b2y4rIgHqpyM40jJivMk9p2FYn/nMZ7j77ruZnJzk9ttvb7vMzp07\neeyxx3jllVd45ZVXeOyxx9i5c+ey3ldEAhXpkM94yzKQ+an9/fffz+rVq7nppps4f/48V111FY8/\n/jgf+tCHFi23YcMG7rzzTrZv3w4kpZwNGzZkFoeIBCbCMmKqicPy0G7isKNHj3LppZeWEk9eqviZ\nBjI9rQ5UkQxkNnGYyLK1zmC4cEUgKMGL5ETJvYtnnnmGT33qU4uee9vb3sYPf/jDkiKKVLdxwkru\nIrlQcu/ive99L4cOHSo7jPhFOk5YJGbxjpaReGhqWJHCKblL/iIdJywSMyV3yV+k44RFYqaauxQj\nwnHCIjGLuuVe5hTLu3btYv369XzsYx8r7k1FRFKKNrmXfTOV2267jQceeKCYNxMR6VO0yT2PKZbT\nzucOcM0117Bu3brB30xEJEfR1tzzGDqddj53EZHQRZvc85piWfO5i0gVRFuWyWvodJr53EVEQhdt\ncs9r6HSa+dxFglHmkDEJWrRlGch+6HTa+dwBPvjBD/Lcc89x5swZRkZG+PrXv64bdkixNNumdJFq\nPncz2wV8CVgJfM3d72n5+27gbuBN4Bzwn9z9/3Z7Tc3nLrJMY2PtO55GR5Nbx0klZTafu5mtBL4C\nXAvMAgfN7GF3f7Zpse8DD7u7m9mVwEPAewYLXURS0Wyb0kWasswO4Ji7HwcwsweB3cCF5O7uZ5qW\n/2WgnNs7ZUzzuUvQ8hoyJpWQJrlvBk42/T4L/FbrQmZ2HfBfgXcCH80kupJpPncJ2r59i2vuoNk2\n5YLMRsu4+3fd/T3A75HU35cwsz1mNmNmM3Nzc51eJ6uQSlelzyIB0myb0kWa5H4K2NL0+0jjubbc\n/QfAJWa2qc3f9rv7hLtPDA8PL/m/a9euZX5+vhJJ0d2Zn59n7dq1ZYciVTY5mXSevvlm8lOJXRrS\nlGUOAlvNbJwkqd8A3NS8gJn9BvBCo0P1A8DbgPl+gxkZGWF2dpZOrfrYrF27lpGRkbLDEJEa6pnc\n3f2cmd0KHCAZCvkNdz9iZnsbf/8r4N8CN5vZL4DXgX/nAzS/V69ezfj4eL//TUREWqQa556HduPc\nRUSku7Tj3KOdfkBERDpTchcRqaDSyjJmNge0uQIjlU3AyxmGk6eYYgXFm6eYYoW44o0pVlhevKPu\nvnS4YYvSkvtymNlMmppTCGKKFRRvnmKKFeKKN6ZYoZh4VZYREakgJXcRkQqKNbnvLzuAPsQUKyje\nPMUUK8QVb0yxQgHxRllzFxGR7mJtuYuISBdBJ3cz22Vmz5vZMTO7o83fd5vZ02Z2qDHb5O+WEWcj\nlq6xNi233czOmdn1RcbXJo5e6/ZqM/tpY90eMrO7yoizEUvPdduI95CZHTGzvy86xpZYeq3b25rW\n62EzO29mGwKN9R1m9rdm9lRj3d5SRpxN8fSK9yIz+24jLzxhZleUEWcjlm+Y2UtmdrjD383M7m18\nlqcb83Jlx92DfJDMY/MCcAmwBngKuKxlmbfzVmnpSuC5UGNtWu5x4BHg+sDX7dXA/45kP1hPcvOY\nixu/vzPkeFuW/zjweKixAn8C/Fnj38PAaWBNwPH+N+BPG/9+D/D9EveFfwN8ADjc4e8fAb4HGPDb\nwA+zfP+QW+4X7gDl7meBhTtAXeDuZ7yxlij3DlA9Y234HPBt4KUig2sjbbwhSBPrTcB33P0EgLuX\nuX77Xbc3At8qJLKl0sTqwDozM5LG1GmS+ySXIU28l5E0oHD354AxM3tXsWEmPJn+/HSXRXYD93vi\nn4D1ZvZrWb1/yMm93R2gNrcuZGbXmdlzwP8BPl1QbK16xmpmm4HrgK8WGFcnqdYtcFXjdPF7ZnZ5\nMaEtkSbWdwMXmdnfmdmTZnZzYdEtlXbdYmZDwC6SA34Z0sT6ZeBS4F+BZ4DPu/ubxYS3RJp4nwJ+\nH8DMdgCjJPegCFHqfWUQISf3VDzFHaAC8efA7SV+Mfr1zyRljiuBvwD+puR4ulkFbCO5veNO4E4z\ne3e5IaXyceAf3L1b665sO4FDwK8D7we+bGa/Um5IXd1D0gI+RHKm/C/A+XJDKkeam3WUpe87QJnZ\nJWa2yd2LnmMiTawTwIPJ2S2bgI+Y2Tl3LyNp9ozX3X/W9O9HzOwvA163s8C8u78KvGpmPwDeB/yo\nmBAX6We/vYHySjKQLtZbgHsa5c9jZvZjklr2E8WEuEja/fYWSDosgR8Dx4sKsE995bi+ldXZkKIz\nYhXJRhnnrc6Ty1uW+Q3e6lD9QGPFWIixtiz/TcrtUE2zbn+1ad3uAE6Eum5Jygbfbyw7BBwGrgh1\n3TaWewdJPfaXA98Pvgr8l8a/39X4jm0KON71NDp8gT8iqWmXsn4bMYzRuUP1oyzuUH0iy/cOtuXu\nBd4BqqBYg5Ey3uuBz5rZOZJ1e0Oo69bdj5rZo8DTwJvA19y97fCzEOJtLHod8JgnZxulSBnr3cA3\nzewZkiR0uxd/9tZPvJcC95mZA0eAPywjVgAz+xbJqLNNZjYL/CmwGi7E+gjJiJljwGs0zjgye/8S\nvq8iIpKz6DtURURkKSV3EZEKUnIXEakgJXcRkQpSchcRqSAldxGRClJyFxGpICV3EZEK+v+rzMQP\ngeA7hAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10b072908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 从 data.txt 中读入点\n",
    "with open('./data.txt', 'r') as f:\n",
    "    data_list = [i.split('\\n')[0].split(',') for i in f.readlines()]\n",
    "    data = [(float(i[0]), float(i[1]), float(i[2])) for i in data_list]\n",
    "\n",
    "# 标准化\n",
    "x0_max = max([i[0] for i in data])\n",
    "x1_max = max([i[1] for i in data])\n",
    "data = [(i[0]/x0_max, i[1]/x1_max, i[2]) for i in data]\n",
    "\n",
    "x0 = list(filter(lambda x: x[-1] == 0.0, data)) # 选择第一类的点\n",
    "x1 = list(filter(lambda x: x[-1] == 1.0, data)) # 选择第二类的点\n",
    "\n",
    "plot_x0 = [i[0] for i in x0]\n",
    "plot_y0 = [i[1] for i in x0]\n",
    "plot_x1 = [i[0] for i in x1]\n",
    "plot_y1 = [i[1] for i in x1]\n",
    "\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们将数据转换成 NumPy 的类型，接着转换到 Tensor 为之后的训练做准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "np_data = np.array(data, dtype='float32') # 转换成 numpy array\n",
    "x_data = torch.from_numpy(np_data[:, 0:2]) # 转换成 Tensor, 大小是 [100, 2]\n",
    "y_data = torch.from_numpy(np_data[:, -1]).unsqueeze(1) # 转换成 Tensor，大小是 [100, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们来实现以下 Sigmoid 的函数，Sigmoid 函数的公式为\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 定义 sigmoid 函数\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画出 Sigmoid 函数，可以看到值越大，经过 Sigmoid 函数之后越靠近 1，值越小，越靠近 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10be61908>]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG1NJREFUeJzt3XuUVOWZ7/HvY4MIiqKCCgIHskQPGG/YgUyGKF6iiFGi\nRgSNjugadAxZicmZI4ljLitj5qjxTHREEZEI6ghG0i0aECXKMbMUsImioqgtZqABAUUBQS7d/Zw/\n3moty+ru6u5dtat2/T5r7VW1936b/fSu7h9vv/tm7o6IiCTLPnEXICIi0VO4i4gkkMJdRCSBFO4i\nIgmkcBcRSSCFu4hIAincRUQSSOEuIpJACncRkQTqFNeGe/bs6QMGDIhr8yIiJWn58uUfuHuv1trF\nFu4DBgygpqYmrs2LiJQkM/vvXNppWEZEJIEU7iIiCaRwFxFJIIW7iEgCKdxFRBKo1XA3sxlmtsnM\nXm9mvZnZnWZWa2avmtnQ6MsUEZG2yKXn/gAwqoX15wCDUtNE4J6OlyUiIh3R6nnu7v68mQ1oockY\nYJaH5/UtMbMeZtbb3TdEVKOIJJU77NkDu3d/cdq16/P39fXQ0JDblNm2sTFso2lq2mZz81Ety/we\nM40YAWedFe2+zBDFRUxHAmvT5utSy74U7mY2kdC7p3///hFsWkRi09gIW7bA5s3wwQdhanq/ZQts\n2waffALbt3952rkzBPiePXF/F4Vh9sX5G24oiXDPmbtPA6YBVFZW6sncIsXMHTZuhDffhLfegjVr\nPp/WroW6utBTzqZbN+je/YtT794waFB4v//+0KUL7LdfeM027bcf7LsvdO4MFRWtT506fXnZPvuE\nYE2foOX5KJYVgSjCfR3QL22+b2qZiJSK+np44w1YtixMr70Gq1bBxx9/3qZTJ+jbF/r3D8MK/fqF\nwO7ZE3r1Cq9N0377xfe9CBBNuM8DJpnZbGA4sFXj7SJFrqEBli+HZ56BRYtCoO/cGdb16AEnngjj\nx8PgwWE65hjo0yf0hqUktBruZvYIMBLoaWZ1wC+AzgDuPhWYD4wGaoGdwIR8FSsiHbBrFzz1FPzh\nDzB//ue98pNOgquvhuHDYdgwOOqoohpekPbJ5WyZ8a2sd+D7kVUkItFxhxdegHvvherqcDDz0EPh\nggvg7LPh9NPDkIokTmy3/BWRPNq1C2bOhClTwvh59+4wdmyYTjstHKSURFO4iyTJzp2hl37bbbBh\nAwwdCvfdB+PGwQEHxF2dFJDCXSQJGhvhoYdg8uQQ6iNHhvnTTtP4eZlSuIuUuuXL4brrwhkvw4bB\nnDnwzW/GXZXETHeFFClVe/bATTeFs1zWroVZs+DFFxXsAqjnLlKa3n47HBxdsQKuvBL+/d/D+eki\nKQp3kVJTXQ1XXBEu0Z83D847L+6KpAhpWEakVLjDL34RzlE/5pgw1q5gl2ao5y5SCurr4ZprYMaM\nMAxzzz26f4u0SD13kWK3axdceGEI9ptuCq8KdmmFeu4ixWzPHrjoIliwIFxtet11cVckJULhLlKs\n9u4NV5bOnx+uOp04Me6KpIRoWEakGLnDP/4jVFXBnXcq2KXNFO4ixeg3vwk3/vrVr+AHP4i7GilB\nCneRYjNnDvzLv8Dll4cDqCLtoHAXKSZNV5yOGBHu5qibfkk7KdxFisX27eGWAoccAnPnhitQRdpJ\nZ8uIFAP3cJFSbS089xwcdljcFUmJU7iLFIMHHoBHHoGbb4ZTTom7GkkADcuIxG3tWvjRj8IDNiZP\njrsaSQiFu0icms5nb2gItxXYR7+SEg0Ny4jE6fe/h4UL4a67YODAuKuRBFE3QSQumzbBj38Mp54K\n//RPcVcjCaNwF4nLT38KO3eG+8ZoOEYipp8okTgsWRLG2K+/Pjx4QyRiCneRQmtogEmToE+fcJsB\nkTzQAVWRQps5Mzwi7+GHoXv3uKuRhFLPXaSQPv00PAd12DAYPz7uaiTB1HMXKaS774a6Opg1SzcF\nk7xSz12kULZuDfdpP+ssOO20uKuRhMsp3M1slJm9ZWa1Zval66PN7CAze8LMVpjZSjObEH2pIiXu\ntttgyxb4t3+LuxIpA62Gu5lVAFOAc4AhwHgzG5LR7PvAG+5+AjASuN3M9o24VpHStWUL3HEHXHwx\nDB0adzVSBnLpuQ8Dat19tbvvAWYDYzLaONDdzAw4ANgC1EdaqUgp+4//gE8+0amPUjC5hPuRwNq0\n+brUsnR3AYOB9cBrwA/dvTGSCkVK3fbtodd+3nlw/PFxVyNlIqoDqmcDrwB9gBOBu8zswMxGZjbR\nzGrMrGbz5s0RbVqkyN17L3z0Edx4Y9yVSBnJJdzXAf3S5vumlqWbAPzRg1rgPeB/Zv5D7j7N3Svd\nvbJXr17trVmkdOzaBbffDmecAcOHx12NlJFcwv0lYJCZDUwdJB0HzMtoswY4A8DMDgeOAVZHWahI\nSZo5E95/X712KbhWL2Jy93ozmwQsBCqAGe6+0syuTa2fCvwaeMDMXgMMuMHdP8hj3SLFr7ERfvc7\nOPnk8JQlkQLK6QpVd58PzM9YNjXt/XrgrGhLEylxzzwDq1bBgw/qalQpOF2hKpIvd9wBRxwBY8fG\nXYmUIYW7SD6sWgULFsB118G+up5PCk/hLpIPd94JXbrANdfEXYmUKYW7SNS2bg1nyVx6KRx2WNzV\nSJlSuItE7T//MzwbVQ+9lhgp3EWi5B6uSD3xRKisjLsaKWMKd5Eo1dTAihUwcaJOf5RYKdxFojRt\nGnTrFsbbRWKkcBeJyrZt8MgjMG4cHHRQ3NVImVO4i0TlkUdgx44wJCMSM4W7SFTuuy/cr33YsLgr\nEVG4i0Ri5UpYvhwmTNCBVCkKCneRKDz4IFRU6ECqFA2Fu0hHNTTAQw/BqFG6IlWKhsJdpKMWL4Z1\n6+Dyy+OuROQzCneRjpo1Cw48EM4/P+5KRD6jcBfpiB07YO7ccM/2rl3jrkbkMwp3kY6oqgoBryEZ\nKTIKd5GOmDULBgyAESPirkTkCxTuIu21cSP8+c/h9Md99KskxUU/kSLtNXcuNDaGe8mIFBmFu0h7\nzZkDgwfDV78adyUiX6JwF2mP9evhL3+BSy7R7QakKCncRdrjscfCU5fGjo27EpGsFO4i7TFnDhx3\nXBiWESlCCneRtlq7Fl54IQzJiBQphbtIWz36aHhVuEsRU7iLtNWjj8LQoXDUUXFXItIshbtIW7z3\nHixbpl67FD2Fu0hbPPZYeL344njrEGmFwl2kLaqr4aSTYODAuCsRaVFO4W5mo8zsLTOrNbPJzbQZ\naWavmNlKM/t/0ZYpUgTefx9efBEuuCDuSkRa1am1BmZWAUwBvgXUAS+Z2Tx3fyOtTQ/gbmCUu68x\nMz1rTJLniSfChUvf+U7clYi0Kpee+zCg1t1Xu/seYDYwJqPNpcAf3X0NgLtvirZMkSJQXQ1f+Yru\nJSMlIZdwPxJYmzZfl1qW7mjgYDNbbGbLzeyKbP+QmU00sxozq9m8eXP7KhaJw/btsGhR6LXrXjJS\nAqI6oNoJOBk4FzgbuMnMjs5s5O7T3L3S3St79eoV0aZFCmDBAtizR0MyUjJaHXMH1gH90ub7ppal\nqwM+dPcdwA4zex44AXg7kipF4lZdDb16wTe+EXclIjnJpef+EjDIzAaa2b7AOGBeRpvHgRFm1snM\nugHDgTejLVUkJnv2wJ/+BOefDxUVcVcjkpNWe+7uXm9mk4CFQAUww91Xmtm1qfVT3f1NM3sKeBVo\nBKa7++v5LFykYBYvhm3bNCQjJSWXYRncfT4wP2PZ1Iz524DboitNpEhUV8P++8MZZ8RdiUjOdIWq\nSEsaG0O4jxoFXbvGXY1IzhTuIi156SXYsEFDMlJyFO4iLamuhk6d4Nxz465EpE0U7iItqa6GkSPh\n4IPjrkSkTRTuIs1ZtSpMGpKREqRwF2nO44+H1/PPj7cOkXZQuIs0p6oKKiuhX7/W24oUGYW7SDbr\n18PSpRqSkZKlcBfJZl7qDht6MIeUKIW7SDbV1TBoEAweHHclIu2icBfJtHUrPPus7t0uJU3hLpJp\n/nzYu1fj7VLSFO4imaqr4fDD4etfj7sSkXZTuIuk27079NzHjIF99OshpUs/vSLpnn0WPvlEQzJS\n8hTuIumqq+GAA+D00+OuRKRDFO4iTRobwy0HRo+GLl3irkakQxTuIk2WLIGNGzUkI4mgcBdpUl0N\nnTuHnrtIiVO4iwC4hxuFnX46HHRQ3NWIdJjCXQTgzTehtlZDMpIYCncRCL120L3bJTEU7iIQxtuH\nD4c+feKuRCQSCneRtWuhpka395VEUbiLVFeHV4W7JIjCXaSqCoYMgaOPjrsSkcgo3KW8ffghPP+8\nzpKRxFG4S3l78kloaNCQjCSOwl3KW1UV9OsHJ58cdyUikVK4S/nasQMWLtTj9CSRcgp3MxtlZm+Z\nWa2ZTW6h3dfMrN7MvhtdiSJ58vTTsGuXxtslkVoNdzOrAKYA5wBDgPFmNqSZdrcAT0ddpEheVFXB\nIYfAKafEXYlI5HLpuQ8Dat19tbvvAWYDY7K0+wEwF9gUYX0i+bF3LzzxBJx3HnTqFHc1IpHLJdyP\nBNamzdelln3GzI4ELgDuia40kTx6/nn4+GMNyUhiRXVA9XfADe7e2FIjM5toZjVmVrN58+aINi3S\nDlVV0LUrnHVW3JWI5EUuf4+uA/qlzfdNLUtXCcy2cMZBT2C0mdW7e3V6I3efBkwDqKys9PYWLdIh\njY3hlgOjRkG3bnFXI5IXuYT7S8AgMxtICPVxwKXpDdx9YNN7M3sAeDIz2EWKxtKlsG4dXHhh3JWI\n5E2r4e7u9WY2CVgIVAAz3H2lmV2bWj81zzWKROvRR2HffcPBVJGEyuk0AXefD8zPWJY11N39yo6X\nJZInjY3w2GNhSEaP05ME0xWqUl6WLoW6Orj44rgrEckrhbuUFw3JSJlQuEv50JCMlBGFu5SPJUs0\nJCNlQ+Eu5eMPf9CQjJQNhbuUBw3JSJlRuEt50JCMlBmFu5SHOXM0JCNlReEuyVdfD7Nnh2DXkIyU\nCYW7JN+iRbBpE1x2WdyViBSMwl2S7+GHoUcPGD067kpECkbhLsm2Y0e4d/vYsdClS9zViBSMwl2S\n7fHHQ8BrSEbKjMJdku2hh6B/fxgxIu5KRApK4S7JtWkTPP00XHop7KMfdSkv+omX5JozBxoa4Hvf\ni7sSkYJTuEty/f73cNJJcOyxcVciUnAKd0mml18O09VXx12JSCwU7pJM998fTn289NLW24okkMJd\nkufTT8OFSxddBAcfHHc1IrFQuEvyVFXBxx/DVVfFXYlIbBTukjz33w8DB8Jpp8VdiUhsFO6SLO+9\nB88+CxMm6Nx2KWv66ZdkufdeqKgI4S5SxhTukhyffgrTp8OYMdC3b9zViMRK4S7JMWcOfPghTJoU\ndyUisVO4SzK4w113wZAhMHJk3NWIxK5T3AWIRGLZMli+HO6+G8zirkYkduq5SzLcdRd0766bhImk\nKNyl9K1bF8bbJ0wIAS8iCndJgDvugMZGuP76uCsRKRo5hbuZjTKzt8ys1swmZ1l/mZm9amavmdkL\nZnZC9KWKZLF1K0ydChdfDAMGxF2NSNFoNdzNrAKYApwDDAHGm9mQjGbvAae6+3HAr4FpURcqktW9\n98L27fDP/xx3JSJFJZee+zCg1t1Xu/seYDYwJr2Bu7/g7h+lZpcAuoJE8m/37jAkc+aZMHRo3NWI\nFJVcwv1IYG3afF1qWXOuBhZkW2FmE82sxsxqNm/enHuVItnMmgXr16vXLpJFpAdUzew0QrjfkG29\nu09z90p3r+zVq1eUm5Zys2cP/Ou/wvDh8K1vxV2NSNHJ5SKmdUC/tPm+qWVfYGbHA9OBc9z9w2jK\nE2nGjBmwZg3cd58uWhLJIpee+0vAIDMbaGb7AuOAeekNzKw/8Efgcnd/O/oyRdLs3g033wx///fq\ntYs0o9Weu7vXm9kkYCFQAcxw95Vmdm1q/VTg58ChwN0WelH17l6Zv7KlrE2fDnV18MAD6rWLNMPc\nPZYNV1ZWek1NTSzblhK2fTsMGgTHHAOLFyvcpeyY2fJcOs+6cZiUlltugY0b4YknFOwiLdDtB6R0\nrFkDt98Ol10GX/ta3NWIFDWFu5SOn/0svP7mN/HWIVICFO5SGl54AR5+GH7yE+jfP+5qRIqewl2K\n3969cM010K8fTP7SfetEJAsdUJXid/vt8PrrMG8eHHBA3NWIlAT13KW4rV4Nv/oVXHghnHde3NWI\nlAyFuxSvhga48kro3Dnc/VFEcqZhGSlev/0t/OUv4UrUvrqLtEhbqOcuxenll+Gmm+Cii+CKK+Ku\nRqTkKNyl+GzbBuPHQ8+e4UlLuhJVpM00LCPFxR2uugpqa2HRIjj00LgrEilJCncpLr/9LcydG15H\njoy7GpGSpWEZKR5PPBEuUvrud+HHP467GpGSpnCX4rBsGVxySXjQte7TLtJhCneJ39tvw7e/DUcc\nAU8+CfvvH3dFIiVP4S7xevvtz8fWn3oKDj881nJEkkLhLvFpCvb6enjuOTj66LgrEkkMhbvEY9ky\nGDHi82A/9ti4KxJJFIW7FF51deixd+8ebi+gYBeJnMJdCqeh4fM7PB5/PLz4YnjQtYhEThcxSWFs\n2hSefbpoEVx+OUydCt26xV2VSGKp5y755Q6PPgpf/Sr813/B9Okwc6aCXSTPFO6SP2vWhLs6XnJJ\neO7psmVw9dW6QEmkABTuEr1t2+BnPwvj6QsWwK23wpIlcNxxcVcmUjY05i7R+egjmDIlPDXpgw/g\ne9+Dm28OvXYRKSiFu3TcqlUwbRrcdx988gmcey788pdQWRl3ZSJlS+Eu7fPRR/D443D//eFAaadO\nMHYs3HBDOM1RRGKlcJfcrVkDf/oTVFWFq0rr68MtA269NTwKT/eFESkaCnfJrrER3n0Xli4NQb54\nMaxeHdYdfTT85CdwwQUwbJjOfhEpQjmFu5mNAu4AKoDp7v5/MtZbav1oYCdwpbv/NeJaJR/cYePG\n8Fi7d96BFSvCw6lffhm2bw9tDj4YTj0VfvhDOPNMGDxYgS5S5FoNdzOrAKYA3wLqgJfMbJ67v5HW\n7BxgUGoaDtyTepU4NTTAxx+H8F6/HjZsCK/r18O6daEnXlv7eYgDdO0KJ54YriIdOhROPjmMoe+j\ns2ZFSkkuPfdhQK27rwYws9nAGCA93McAs9zdgSVm1sPMerv7hsgrLkXuYXw6fdq7t/n5pvd798Kn\nn8LOnWFqep/5umNHCPGPPvrilB7a6Q48EPr0gQEDwp0ZBw0K01FHwcCBUFFR0N0jItHLJdyPBNam\nzdfx5V55tjZHAtGH+1NPwfXXh/fuX5yyLWtueaG+vqEhTPnQpUu4jL9bN+jRIwyf9O8PJ5wQ3jct\nO/zwEOZ9+kDv3nrSkUgZKOgBVTObCEwE6N/eC1sOOihc6dg05mv2xSnbsra0jfrrO3X64tS5c/b3\nzc136xaGSjJfu3bVUImINCuXcF8H9Eub75ta1tY2uPs0YBpAZWWlt6nSJn/3d2ESEZFm5dL1ewkY\nZGYDzWxfYBwwL6PNPOAKC74ObNV4u4hIfFrtubt7vZlNAhYSToWc4e4rzeza1PqpwHzCaZC1hFMh\nJ+SvZBERaU1OY+7uPp8Q4OnLpqa9d+D70ZYmIiLtpSNyIiIJpHAXEUkghbuISAIp3EVEEkjhLiKS\nQObevmuJOrxhs83Af7fzy3sCH0RYTlSKtS4o3tpUV9uorrZJYl3/w917tdYotnDvCDOrcfeie4Zb\nsdYFxVub6mob1dU25VyXhmVERBJI4S4ikkClGu7T4i6gGcVaFxRvbaqrbVRX25RtXSU55i4iIi0r\n1Z67iIi0oGjD3cwuNrOVZtZoZpUZ635qZrVm9paZnd3M1x9iZs+Y2Tup14PzUOMcM3slNf3NzF5p\npt3fzOy1VLuaqOvIsr1fmtm6tNpGN9NuVGof1prZ5ALUdZuZrTKzV82sysx6NNOuIPurte8/dQvr\nO1PrXzWzofmqJW2b/czsOTN7I/Xz/8MsbUaa2da0z/fn+a4rbdstfjYx7bNj0vbFK2a2zcx+lNGm\nIPvMzGaY2SYzez1tWU5ZFPnvo7sX5QQMBo4BFgOVacuHACuALsBA4F2gIsvX3wpMTr2fDNyS53pv\nB37ezLq/AT0LuO9+CfyvVtpUpPbdV4B9U/t0SJ7rOgvolHp/S3OfSSH2Vy7fP+E21gsAA74OLC3A\nZ9cbGJp63x14O0tdI4EnC/Xz1JbPJo59luVzfZ9wLnjB9xlwCjAUeD1tWatZlI/fx6Ltubv7m+7+\nVpZVY4DZ7r7b3d8j3EN+WDPtZqbezwS+k59KQ28FGAs8kq9t5MFnDz539z1A04PP88bdn3b3+tTs\nEsITu+KSy/f/2YPf3X0J0MPMeuezKHff4O5/Tb3fDrxJeB5xqSj4PstwBvCuu7f3AskOcffngS0Z\ni3PJosh/H4s23FvQ3MO4Mx3unz8N6n3g8DzW9E1go7u/08x6BxaZ2fLUc2QL4QepP4tnNPNnYK77\nMV+uIvTwsinE/srl+491H5nZAOAkYGmW1d9Ifb4LzOzYQtVE659N3D9X42i+kxXXPssliyLfbwV9\nQHYmM1sEHJFl1Y3u/nhU23F3N7N2nRaUY43jabnXPsLd15nZYcAzZrYq9T98u7VUF3AP8GvCL+Kv\nCUNGV3Vke1HU1bS/zOxGoB54uJl/JvL9VWrM7ABgLvAjd9+WsfqvQH93/yR1PKUaGFSg0or2s7Hw\nGNDzgZ9mWR3nPvtMR7KorWINd3c/sx1fltPDuIGNZtbb3Tek/izclI8azawTcCFwcgv/xrrU6yYz\nqyL8CdahX4hc952Z3Qc8mWVVrvsx0rrM7Erg28AZnhpszPJvRL6/sojswe9RM7POhGB/2N3/mLk+\nPezdfb6Z3W1mPd097/dQyeGziWWfpZwD/NXdN2auiHOfkVsWRb7fSnFYZh4wzsy6mNlAwv++y5pp\n9w+p9/8ARPaXQIYzgVXuXpdtpZntb2bdm94TDiq+nq1tVDLGOC9oZnu5PPg86rpGAf8bON/ddzbT\nplD7qygf/J46fnM/8Ka7/99m2hyRaoeZDSP8Hn+Yz7pS28rlsyn4PkvT7F/Qce2zlFyyKPrfx3wf\nPW7vRAilOmA3sBFYmLbuRsKR5beAc9KWTyd1Zg1wKPBn4B1gEXBInup8ALg2Y1kfYH7q/VcIR75X\nACsJwxP53ncPAq8Br6Z+QHpn1pWaH004G+PdAtVVSxhXfCU1TY1zf2X7/oFrmz5PwhkfU1LrXyPt\nrK081jSCMJz2atp+Gp1R16TUvllBODD9jXzX1dJnE/c+S213f0JYH5S2rOD7jPCfywZgbyq/rm4u\ni/L9+6grVEVEEqgUh2VERKQVCncRkQRSuIuIJJDCXUQkgRTuIiIJpHAXEUkghbuISAIp3EVEEuj/\nAzXYfAgnMBMxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bc7a080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出 sigmoid 的图像\n",
    "\n",
    "plot_x = np.arange(-10, 10.01, 0.01)\n",
    "plot_y = sigmoid(plot_x)\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_data = Variable(x_data)\n",
    "y_data = Variable(y_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 PyTorch 当中，不需要我们自己写 Sigmoid 的函数，PyTorch 已经用底层的 C++ 语言为我们写好了一些常用的函数，不仅方便我们使用，同时速度上比我们自己实现的更快，稳定性更好\n",
    "\n",
    "通过导入 `torch.nn.functional` 来使用，下面就是使用方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义 logistic 回归模型\n",
    "w = Variable(torch.randn(2, 1), requires_grad=True) \n",
    "b = Variable(torch.zeros(1), requires_grad=True)\n",
    "\n",
    "def logistic_regression(x):\n",
    "    return F.sigmoid(torch.mm(x, w) + b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在更新之前，我们可以画出分类的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x10bf66c18>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FFXWwOHfTQhgAFkCooJJYGRkh8GAghuoCIKiDuiI\nAccVwiKOOwyKqIAojgsCAiIgA6KOCx+yo4gsihI0IktQlB0RCDthyXK+P7rRBrNUp6u7qrvP+zz9\nJF1dXXW6llO37q26ZUQEpZRS0SPG6QCUUkqFliZ+pZSKMpr4lVIqymjiV0qpKKOJXymloowmfqWU\nijKa+JVSKspo4ldKqSijiV8ppaJMKacDKEjVqlUlOTnZ6TCUUipsrFq1aq+IVLMyrisTf3JyMunp\n6U6HoZRSYcMYs8XquFrVo5RSUUYTv1JKRRlN/EopFWWKreM3xlwATAGqAwKMF5HXzhjHAK8BHYBs\n4C4R+db7WXvvZ7HABBEZXpJAc3Jy2L59O8ePHy/J15UfypYtS82aNYmLi3M6FKVUEFhp3M0FHhGR\nb40xFYBVxpiFIrLOZ5zrgTre1yXAG8AlxphYYDTQFtgOrDTGzDzju5Zs376dChUqkJycjOc4o4JB\nRMjKymL79u3UqlXL6XCUUkFQbFWPiPx6qvQuIoeB9UCNM0a7CZgiHiuASsaY84AWwEYR+UVETgLv\nesf12/Hjx0lISNCkH2TGGBISEvTMSqkI5lcdvzEmGfgb8PUZH9UAtvm83+4dVtjwEtGkHxq6nJWK\nbJYTvzGmPPAh8C8ROWR3IMaYHsaYdGNM+p49e+yevFJKucpPWT/x6IJHyZf8kM/bUuI3xsThSfrT\nROSjAkbZAVzg876md1hhw/9ERMaLSIqIpFSrZunmM1fbvHkz77zzzu/vMzIymDNnzu/vZ86cyfDh\nJWrn/pO77rqLDz74AID77ruPdev8bkJRSoXIidwTPPvFszR6oxFvfvsmG/ZuCHkMxSZ+7xU7bwHr\nReTlQkabCdxpPC4FDorIr8BKoI4xppYxpjRwu3fciFdc4u/UqRP9+/e3fb4TJkygfv36tk9XKRW4\nxZsX02RsE55e/DQ3172ZzD6Z1KtWL+RxWCnxXwZ0B642xmR4Xx2MMWnGmDTvOHOAX4CNwJtAbwAR\nyQX6AvPxNAq/LyJr7f4RoTJlyhQaN25MkyZN6N69O3B6aRugfPnyAPTv35+lS5fStGlTXnjhBQYN\nGsR7771H06ZNee+995g8eTJ9+/b9fRr9+vWjVatW1K5d+/fp5efn07t3b+rWrUvbtm3p0KHDafMq\nSOvWrX/v7qJ8+fIMHDiQJk2acOmll/Lbb78BsGfPHjp37kzz5s1p3rw5y5cvt3dBKaVOs+foHv45\n45+0ebsNOfk5zEudx7td3uW8Cuc5Ek+xl3OKyDKgyNY+ERGgTyGfzcFzYLDNv+b9i4xdGXZOkqbn\nNuXV9q8W+vnatWsZMmQIX375JVWrVmXfvn1FTm/48OG89NJLzJo1C4Dq1auTnp7OqFGjAJg8efJp\n4//6668sW7aMzMxMOnXqRJcuXfjoo4/YvHkz69atY/fu3dSrV4977rnH8m86evQol156KUOHDuXx\nxx/nzTff5Mknn+TBBx/koYce4vLLL2fr1q20a9eO9evXW56uUsqafMln0neTePzTxzl04hD/vvzf\nPHnlk5wVd5ajcbmykzY3WrRoEbfeeitVq1YFoEqVKrZO/+abbyYmJob69ev/XjJftmwZt956KzEx\nMZx77rm0adPGr2mWLl2aG264AYCLL76YhQsXAvDpp5+e1g5w6NAhjhw58vvZilIqcOv2rCNtVhpL\nty7l8sTLGdtxLA3OaeB0WECYJv6iSuahVqpUKfLzPa3y+fn5nDx5skTTKVOmzO//e06gAhcXF/f7\npZmxsbHk5uYCnjhXrFhB2bJlbZmPUuoP2TnZDF0ylBFfjqBCmQpMuHECd//tbmKMe3rIcU8kLnf1\n1Vfzv//9j6ysLIDfq3qSk5NZtWoV4LlSJycnB4AKFSpw+PDh379/5nsrLrvsMj788EPy8/P57bff\nWLx4sQ2/BK677jpef/31399nZNhbbaZUtJq3cR4NxzRk2LJhdG3Ulcw+mdzb7F5XJX3QxG9ZgwYN\nGDhwIFdddRVNmjTh4YcfBuD+++/niy++oEmTJnz11VeUK1cOgMaNGxMbG0uTJk145ZVXaNOmDevW\nrfu9cdeKzp07U7NmTerXr0+3bt1o1qwZFStWDPi3jBw5kvT0dBo3bkz9+vUZO3ZswNNUKpr9evhX\n/vHBP7h+2vXExcax6M5FvH3z21Qr585L041d1Qp2SklJkTMfxLJ+/Xrq1Qv9ZU9OO1X3npWVRYsW\nLVi+fDnnnntu0OcbrctbKX/k5ecxbtU4Bnw2gBO5J/j3Ff/micueoEypMsV/2WbGmFUikmJl3LCs\n448mN9xwAwcOHODkyZM89dRTIUn6SqniZezKoOesnnyz4xuuqXUNb3R8gzoJdZwOyxJN/C5nV72+\nUsoeR04e4enPn+a1r18jIT6BqbdM5Y5Gd4RVH1ea+JVSyqKZG2bSd05fth3aRo9mPRh+7XAqn1XZ\n6bD8polfKaWKse3gNh6Y+wD/t+H/aHhOQ6Z3ns5liZc5HVaJaeJXSqlC5Obn8vrXr/PU50+RL/kM\nv2Y4D7d8mLjY8H46nSZ+pZQqwDc7vqHnrJ5k7MqgQ50OjLp+FLUqR8ZT6TTxK6WUj4PHDzJw0UDG\nrBzDeRXO43+3/o/O9TqHVeNtcSL3Bq5p0yA5GWJiPH+nTQvZrN9++23q1KlDnTp1ePvtt0M2X6VU\nyYkI7699n3qj6zFm5Rj6tujL+j7r6VK/S0QlfYjUEv+0adCjB2Rne95v2eJ5D5CaGtRZ79u3j2ee\neYb09HSMMVx88cV06tSJypXDr+VfqWixaf8mes/pzbyN82h2XjNmdp1JyvmW7oUKS5FZ4h848I+k\nf0p2tmd4Ca1cuZLGjRtz/Phxjh49SoMGDVizZs2fxps/fz5t27alSpUqVK5cmbZt2zJv3rwSz1cp\nFTw5eTkMXzacBmMasGzrMl5t9ypf3/d1RCd9iNQS/9at/g23oHnz5nTq1Iknn3ySY8eO0a1bNxo2\nbPin8Xbs2MEFF/zxtMmaNWuyY0eBT5tUSjlo+dbl9JzVk7V71nJL3VsYef1Iap5d0+mwQiIyE39i\noqd6p6DhARg0aBDNmzenbNmyjBw5MqBpKaWcse/YPp5Y+AQTvptAYsVEZt4+kxsvutHpsEIqMqt6\nhg6F+PjTh8XHe4YHICsriyNHjnD48GGOHz9e4Dg1atRg27Ztv7/fvn07NWrUCGi+SqnAiQj//f6/\n1B1Vl0kZk3i05aOs7b026pI+RGriT02F8eMhKQmM8fwdPz7ght2ePXvy3HPPkZqayhNPPFHgOO3a\ntWPBggXs37+f/fv3s2DBAtq1axfQfJVSgfkx60eu/e+13DnjTmpXrs2qHqsYcd0IypeOzqfORWZV\nD3iSvI1X8EyZMoW4uDjuuOMO8vLyaNWqFYsWLeLqq68+bbwqVarw1FNP0bx5c8BTPWT3YxqVUtYc\nzz3O8GXDeX7Z85xV6izGdBhDz5SernswSqhpf/yqQLq8VbhbtGkRvWb34sesH+nasCsvt3uZc8tH\nbrfmtvbHb4yZCNwA7BaRP13GYox5DDhVtC4F1AOqicg+Y8xm4DCQB+RaDUoppUpq99HdPLLgEaau\nnkrtyrWZ320+1/3lOqfDchUrVT2TgVHAlII+FJERwAgAY8yNwEMiss9nlDYisjfAOF3nhx9+oHv3\n7qcNK1OmDF9//bVDESkV3fIln4nfTeTxhY9z5OQRBl4xkIFXDOSsuLOcDs11ik38IrLEGJNscXpd\ngemBBBQuGjVqpA8pV8ol1u5eS89ZPVm+bTlXJl3J2I5jqVdNqyoLY1sLhzEmHmgPfOgzWIBPjTGr\njDE97JqXUkoBZOdkM+DTATQd15TMvZlM7DSRxf9crEm/GHZe1XMjsPyMap7LRWSHMeYcYKExJlNE\nlhT0Ze+BoQdAYoA3WimlIt/cn+bSZ04fNh3YxF1N72JE2xFUja/qdFhhwc5rmm7njGoeEdnh/bsb\n+BhoUdiXRWS8iKSISEq1atVsDEspFUl2Ht7Jbf+7jQ7vdKBMqTIs/udiJt00SZO+H2wp8RtjKgJX\nAd18hpUDYkTksPf/64Bn7ZifUir65OXn8Ub6GwxcNJATuSd4rs1zPNbqMcqUKuN0aGGn2BK/MWY6\n8BVwkTFmuzHmXmNMmjEmzWe0W4AFInLUZ1h1YJkx5nvgG2C2iISsm0oHu+Onffv2VKpUiRtuuCF0\nM1Uqgn3363e0fKslD8x9gEtqXMKa3mt48sonNemXkJWrerpaGGcynss+fYf9AjQpaWCBcLA7fgAe\ne+wxsrOzGTduXPBnplQEO3ziMIM+H8TIb0ZSLb4a7/z9HW5veHvEPRgl1CLyvuUgdMdvuT9+gGuu\nuYYKFSqUfGZKKWZkzqD+mPq8+vWr9GjWg8y+mXRt1FWTvg0isq+eIHTHb7k/fqVUYLYe3MoDcx9g\n5oaZNDqnEe93eZ+WF7R0OqyIEpGJP0jd8Wt//EoFUW5+Lq+teI2nFz9NvuTzwrUv8NClDxEXG+d0\naBEnIqt6gtQdv6X++JVS/vtmxzc0f7M5jy58lNbJrVnXZx2PX/a4Jv0gicjEH6Tu+C31x6+Usu7g\n8YP0md2HSydcyu6ju/ng1g/4pOsnJFdKdjq0iBaRVT1ge3f8lvvjB7jiiivIzMzkyJEj1KxZk7fe\neksfxqKUDxHh/bXv86/5/2L30d30u6Qfz7V5jgpl9KKIUIjYxG+3O++8kzvvvBOA2NjYInvhXLp0\naajCUirs/LzvZ3rP6c2Cnxdw8XkXM6vrLC4+/2Knw4oqmviVUiFxMu8kI5aPYMjSIcTFxPFa+9fo\n07wPsTGxTocWdTTxl5D2x6+UdUu3LCVtdhrr9qyjc73OvNb+NWqcXcPpsKJWWCV+EXHNzRuR3B+/\nGx/HqcJTVnYWjy98nIkZE0mqmMQnXT/hhr9qVyZOC5vEX7ZsWbKyskhISHBN8o9EIkJWVhZly5Z1\nOhQVxkSE/67+L48seIT9x/bzWKvHePqqpylXupzToSnCKPHXrFmT7du3s2fPHqdDiXhly5alZs2a\nToehwtSGvRvoNbsXn2/+nJY1WzLuhnE0qt7I6bCUj7BJ/HFxcdSqVcvpMJRShTiee5znlz7P8OXD\niY+LZ2zHsdx/8f3EmIi8XSishU3iV0q512e/fEav2b34ad9P3NHoDl6+7mWql6/udFiqEJr4lVIl\ntvvobh6e/zDTfpjGhVUuZEG3BbT9S1unw1LF0MSvlPJbvuQz4dsJ9P+0P0dzjjLoykEMuGIAZUvp\nRQHhQBO/UsovP/z2A2mz0/hy25dclXQVY28YS92qdZ0OS/lBE79SypLsnGye/eJZ/vPVf6hYpiKT\nb5rMnU3u1Murw5AmfqVUseb8NIc+c/qw+cBm7m56NyPajiAhPsHpsFQJaeJXShVq5+GdPDjvQT5Y\n9wH1qtbji7u+4MqkK50OSwWo2AtsjTETjTG7jTEFPmDWGNPaGHPQGJPhfQ3y+ay9MWaDMWajMaa/\nnYErpYInLz+PUd+Mou6ousz6cRZD2gwhIy1Dk36EsFLinwyMAqYUMc5SETmtAw5jTCwwGmgLbAdW\nGmNmisi6EsaqlAqBb3/9lp6zepK+M53r/nIdYzqM4S9V/uJ0WMpGxZb4RWQJsK8E024BbBSRX0Tk\nJPAucFMJpqOUCoHDJw7zr3n/ovmbzdl2cBvTO09nXuo8TfoRyK46/lbGmNXADuBREVkL1AC2+Yyz\nHbjEpvkppWwiIszInMEDcx9g5+Gd9Ly4J89f+zyVylZyOjQVJHYk/m+BRBE5YozpAMwA6vg7EWNM\nD6AHQGJiog1hKaWKs+XAFvrO7cusH2fRuHpjPrjtAy6teanTYakgC7j3JBE5JCJHvP/PAeKMMVXx\nlP4v8Bm1pndYYdMZLyIpIpJSrVq1QMNSShUhJy+Hl758ifpj6rNo0yJGtB1B+v3pmvSjRMAlfmPM\nucBvIiLGmBZ4DiZZwAGgjjGmFp6EfztwR6DzU0oFZsX2FfSc1ZPVv63mxr/eyOvXv05SpSSnw1Ih\nVGziN8ZMB1oDVY0x24GngTgAERkLdAF6GWNygWPA7eJ5hFOuMaYvMB+IBSZ66/6VUg44cPwA//7s\n34xNH8v5Fc7no9s+4ua6N+udt1HIuPExeykpKZKenu50GEpFBBHh3TXv8tD8h9iTvYd+LfrxbJtn\nqVCmgtOhKRsZY1aJSIqVcfXOXaUi2M/7fqb3nN4s+HkBKeenMCd1Ds3Oa+Z0WMphmviVikAn804y\nYvkIhiwdQlxMHCPbj6R3897ExsQ6HZpyAU38SkWYJVuWkDYrjfV713Nr/Vt5pd0r1Di7htNhKRfR\nxK9UhNibvZfHFz7OpIxJJFdKZvYds+lQp4PTYSkX0sSvVJgTESZnTOaxhY9x8MRBnrjsCQZdNYj4\nuHinQ1MupYlfqTCWuTeTtFlpfLHlC1pd0IqxHcfSqHojp8NSLqeJX6kwdCznGMOWDuOF5S9QrnQ5\nxt0wjvua3UeMCfhmfBUFNPErFWY+/eVTes3uxcZ9G+nWuBsvtX2J6uWrOx2WCiOa+JUKE78d+Y2H\nFzzMOz+8w4VVLmRh94VcW/tap8NSYUgTv1Iuly/5vLnqTfp/1p/snGwGXTmIAVcMoGypsk6HpsKU\nJn6lXGz1b6tJm5XGV9u/onVya8Z2HMtFVS9yOiwV5jTxK+VCR08e5ZkvnuHlr16m8lmVefvmt+ne\nuLt2qKZsoYlfKZeZ9eMs+s7py5aDW7j3b/fywrUvkBCf4HRYKoJo4lfKJXYc2kG/ef34aP1H1K9W\nnyV3LeGKpCucDktFIE38SjksLz+P0StH8+SiJ8nJz2HY1cN4pNUjlI4t7XRoKkJp4lfKQat2rqLn\nrJ6s+nUV7f7SjjEdx1C7cm2nw1IRThO/Ug44dOIQTy16ilErR3FOuXN4t/O73NbgNm28VSGhiV+p\nEBIRPlr/Ef3m9ePXw7/SK6UXQ68ZSqWylZwOTUURTfxKhcjmA5vpO6cvs3+aTdNzm/LRbR9xSc1L\nnA5LRSFN/EoFWU5eDq+seIVnvngGg+E/1/2Hfpf0o1SM7n7KGbrlKRVEX277krRZafyw+wduuugm\nRl4/ksSKiU6HpaJcsX24GmMmGmN2G2PWFPJ5qjFmtTHmB2PMl8aYJj6fbfYOzzDGpNsZuFJutv/Y\nfnp+0pPLJl7G/uP7mfGPGcy4fYYmfeUKVkr8k4FRwJRCPt8EXCUi+40x1wPjAd+KyzYisjegKJUK\nEyLC9DXTeWj+Q+zN3stDlz7EM62foUKZCk6HptTvik38IrLEGJNcxOdf+rxdAdQMPCylws/GfRvp\nNbsXn/7yKc3Pb8681Hn87by/OR2WUn9idx3/vcBcn/cCfGqMyQPGicj4wr5ojOkB9ABITNTTYRU+\nTuSe4MXlLzJ06VDKlCrDqOtHkZaSRmxMrNOhKVUg2xK/MaYNnsR/uc/gy0VkhzHmHGChMSZTRJYU\n9H3vQWE8QEpKitgVl1LBtHjzYtJmpbEhawO3NbiNV9q9wvkVznc6LKWKZMsDOo0xjYEJwE0iknVq\nuIjs8P7dDXwMtLBjfko5bW/2Xu6acRdt3m7DybyTzE2dy3td3tOkr8JCwCV+Y0wi8BHQXUR+9Ble\nDogRkcPe/68Dng10fko5SUSYlDGJxxY+xqEThxhw+QCevPJJ4uPinQ5NKcuKTfzGmOlAa6CqMWY7\n8DQQByAiY4FBQAIwxtvPSK6IpADVgY+9w0oB74jIvCD8BqVCYt2edaTNSmPp1qVcnng5YzuOpcE5\nDZwOSym/Wbmqp2sxn98H3FfA8F+AJn/+hlLh5VjOMYYuHcqLy1+kfOnyTLhxAnf/7W5ijC01pUqF\nnN65q1QRFvy8gN6ze/Pz/p/p3rg7L133EueUO8fpsJQKiCZ+pQqw68guHpr/EO+ueZe/JvyVz+78\njKtrXe10WErZQhO/Uj7yJZ9x6eMY8NkAjuUe4+mrnqb/5f0pW6qs06EpZRtN/Ep5fb/re9Jmp7Fi\n+wqurnU1b3R8g78m/NXpsJSynSZ+FfWOnDzC4MWDeXXFq1Q5qwpTbp5Ct8bd9GlYKmJp4ldR7ZMN\nn9B3bl+2HtzK/c3uZ/i1w6lyVhWnw1IqqDTxq6i07eA2+s3rx4zMGTQ8pyHL7l7GZYmXOR2WUiGh\niV9Fldz8XF7/+nUGLR5EXn4ez1/zPI+0fIS42DinQ1MqZDTxq6ixcsdKes7qyXe7vuP6C69ndIfR\n1Kpcy+mwlAo5Tfwq4h08fpAnFz3J6JWjObf8ubzX5T1urX+rNt6qqKWJX0UsEeHD9R/Sb24/dh3Z\nRZ/mfRhy9RAqlq3odGhKOUoTv4pIm/Zvos+cPszdOJem5zZlxu0zaFFDewVXCjTxqwiTk5fDf776\nD89+8SwxJoZX2r1C3xZ9KRWjm7pSp+jeoCLG8q3LSZudxprda7i57s2MbD+SCype4HRYSrmOJn4V\n9vYd20f/T/vz5rdvcsHZF/B/t/8fnS7q5HRYSrmWJn4VtkSEaT9M4+H5D7Pv2D4eafkIg1sPpnzp\n8k6HppSraeJXYemnrJ/oNbsXn236jEtqXMLC7gtpcq4+90cpKzTxq7ByIvcELyx/gWFLh1G2VFnG\ndBhDj4t7EBsT63RoSoUNTfwqbCzevJi0WWlsyNrA7Q1v55V2r3Bu+XOdDkupsKOJX7nenqN7eHTh\no0z5fgq1K9dmXuo82l3YzumwlApbxT4t2hgz0Riz2xizppDPjTFmpDFmozFmtTGmmc9n7Y0xG7yf\n9bczcBX58iWfid9NpO7oukz/YToDrxjIml5rNOkrFaBiEz8wGWhfxOfXA3W8rx7AGwDGmFhgtPfz\n+kBXY0z9QIKNOtOmQXIyxMR4/k6b5nREIbN291paT27NvTPvpUG1BmSkZTDk6iGcFXeW06EpFfaK\nTfwisgTYV8QoNwFTxGMFUMkYcx7QAtgoIr+IyEngXe+4yopp06BHD9iyBUQ8f3v0sD35u+3Ykp2T\nzcDPBtJ0XFPW7lnLW53eYvFdi6lfTcsMvty23lR4sVLiL04NYJvP++3eYYUNV1YMHAjZ2acPy872\nDLdJiI4tls3bOI+GYxoybNkwUhulktknk3v+dg8xxo7NNLhCmYjdtt70IBSGRKTYF5AMrCnks1nA\n5T7vPwNSgC7ABJ/h3YFRRcyjB5AOpCcmJkrUM0bEs1+f/jLGtlkkJRU8i6Qk22Zhyc5DO+Uf//uH\nMBi56PWL5PNNn4c2gBKaOvWPZXjm6oqP93weDP6ut1NxGuP5a2dcU6d6fmuofrsqHJAuFvK5eFZT\nwIl/HNDV5/0G4DygJTDfZ/gAYICV+V188cVBXUCudObemZAQ9KwcgmNLkXLzcmX0N6Pl7OfPljLP\nlZFnFz8rx3OOh2bmASoo4YXqAOrPegt2YnZL4UGFPvF3BOYCBrgU+MY7vBTwC1ALKA18DzSwMr+o\nS/wF7Z2lS4vExQW1KOXkTvvdr99JizdbCIORa6dcKz/u/TH4M7VRYcsuFAdQf9ZbsNex04UH9Qd/\nEr+VyzmnA18BFxljthtj7jXGpBlj0ryjzPEm+I3Am0BvbxVSLtAXmA+sB94XkbXFzS8qFVSff/Ik\nnH02JCWBMZ6/48dDaqptsx06FOLjTx8WH+8ZHixHTh7h0QWPkjI+hc0HNjPt79NY0G0BdRLqBG+m\nQbB1a/HjJCYGZ97+rLfC4rQSvxWF/cZg/XZlE6tHiFC+oq7E72CxKZj1v2easX6GXPDyBcJgpMfM\nHrIve1/wZhZkxZX4g13PXdB6K2hYsEv8WsfvHthd1RPqV9Ql/givKN16YKvc/O7NwmCk4ZiGsnzr\ncqdDClhBCe/U8TvYB1Cr8cTHi/TqFfzEXFThIZQFi2iniT/cRGixKScvR17+8mUpN7ScnDXkLHlh\n2QtyMvek02HZxk1Jraiyg1NxTp3652aquLiw36xdSxN/OHJTFrHB19u/lqZjmwqDkY7TOsqm/Zuc\nDimi2VlbaNemWNiFaQkJJZueKpo/id/9d8ZEi9RU2LwZ8vM9f21sxA2lg8cP0ndOXy6dcCm7j+7m\ng1s/4JOun5BcKTno847mG4nsaGSdNg2qVoVu3ey5OSwry7/hKnQ08StbiAjvr32feqPr8Ub6GzzQ\n4gHW91lP5/qdMcYEPSm77W7WUAv0Cq1Ty6+gpGzzDeOuFVUFB6unBqF8RWVVTxj7ed/P0n5qe2Ew\n0mxcM1m5Y+Vpn4eiCSPC28ctCaSKxsp9Cf5O081VPWcuq1A0ggcbWsev7FBcIjmRe0KGLRkmZYeU\nlfLDysurX70qOXk5f5pOKJKy3kgUmMKWXyCXqU6d6rkP0ff7pUs7n0yLuiIrmJe9BrsJTxO/Clhx\npfRlW5ZJg9ENhMHI39/7u2w7uK3QaYUiKYfi4BJh7e+nsVLiL8kydeMy8+e32rGNhuqiPU38KmCF\n7Rw1L8iV+/7vPmEwkvhKoszMnFniadmdlIO5c7nhittgJtFevayX+sP9LMrq77RrGw1VNaQmfhWw\nwneOPIl9JlYenf+oHDlxxNK0QpU0g5kYnW5DCOYytNLhnBO/OVgKW5fB6mE1VNWQmvhVwArbOUpX\n2SkZv2b4PT03nvL7w+k2hGAeePyp+gi3Bs+CFHWXczC2US3xa+IPG56dI//0pF/2pPx3ap7ToTnC\n6RJ/MA88Vqs+wvGAXZhQFkTcWMev1/GrAp3XahEVOj8CFTeDyafmBblMnBBHt9To3GSc6MnUVzB7\nwSxuGvHxMHVqWN9X+CehvF8yNdXTsW4QO9r1W3TuxapQu4/upvvH3blmyjWUT5nJ/FU/IvkxbNta\nKmJ2+pIIdOcN9OagYB54Cpq2MZ6/bkhSkcB1N+ZbPTUI5UurekIvLz9P3lz1plQeXlnino2TJz97\nUrJPZjtAloqyAAAVAklEQVQdVkSw61Q/2I9QDOc2GOVfVY/xjO8uKSkpkp6e7nQYUWPt7rX0nNWT\n5duWc2XSlYztOJZ61eo5HVbESE72dCFxpqQkT+lPKTsYY1aJSIqVcbWqJ4pl52Qz4NMBNB3XlMy9\nmUzsNJHF/1zsSfpR1XFJcAX7KViF0VWoCqOJP1zYvBfP/WkuDcc0ZPjy4XRr3I3Mvpnc/be7McZo\nj2c2c+LxhLoKVVE08YcDG/finYd3ctv/bqPDOx0oU6oMn//zcybdNImq8VX/GKmgZwBHSxeNQdCh\ng3/D7RAOqzAcz0jCMeYCWW0MCOVLG3fPYMNF5Ll5ufL616/LPbeVlc0VkXyD5CcmFtyK5/TdSuJ8\nY6Od83fiHgAXrMIiuaELDH+5PWbsvoELaA9sADYC/Qv4/DEgw/taA+QBVbyfbQZ+8H5mKTBN/D6m\nTi14D/ZjL161c5U0H99cuv4dOVY6pvgt1+G7lZzeweyevxNJ2IlV6M/B0ukb4krC7THbmviBWOBn\noDZQGvgeqF/E+DcCi3zebwaqWg1INPH/obhOVIrZ4g4dPyT/mvsviXkmRqqPqC5HzqtqbToOZ16n\ndzC75+9UEg7lKvT34fNuPyMpiNtjtjvxtwTm+7wfAAwoYvx3gPt93mviL6miOlEpZi/+eP3HUvPl\nmsJgpOcnPWVf9j7/tlwH61qc3sGKmn9JFotTx9FQrsLi+vs58/c6fXAvCbfHbHfi7wJM8HnfHRhV\nyLjxwL5T1TzeYZu81TyrgB5WgtLE71VUJyqF9Ci15cAW6TS9kzAYaTSmkXy59cs/puf2LdfL6TAL\nm39Cwp8TeOnSnuHFJVen2yyCzUp/P77rz+nqvJIIdg+pgW4fTib+fwCfnDGshvfvOd5qoisL+W4P\nIB1IT0xM9P9XRyI/MlB+fLzMerqrlBtaTuKHxsuLy16Uk7knT59emOxtTodZ2PwLe5SgyxdnSFjp\n4fPMM7ZwPBgGI2a7tnfHqnqAj4E7ipjWYODR4uapJX4vPzPQpopIx2kdZfP+zUVP89SWm5Bgrbjq\nAKeTQkHz96cXy2hjpU//aFwuVth1hmt34i8F/ALU8mncbVDAeBW91TzlfIaVAyr4/P8l0L64eWri\n9+FHBso3RvLz861PNwxK/27i7+MJw2VR+nOQLWrcU5+dKt27ZdNyuhBRHLvatIJxOWcH4Efv1T0D\nvcPSgDSfce4C3j3je7W9B4rvgbWnvlvcSxN/0fITEwMvIjhdkV5CTu7E/j6pKhyOo/4c//25csct\nydYt5ZuilocrS/xOvDTxF25j1kZ5Pq2hHIkLMMs4felMCbhhJz6zpiwurujk7/LjqF9Jx98rd9zA\nDeWborbbqVMLrrl1vI7fiZcm/j87kXtChi4ZKmWHlJUKwyrI3MHdPSX/khap3LBH+MmNIftWb4TZ\ncVRE/Dv++3vljhu4oXzjz1Vip4Y7flWPEy9N/KdbsnmJ1B9dXxiMdH6vs2w/uD3wiYZDJ/FncMNO\nXBi7D0qhWqx2lvjdsi58uaGwYPWigEBj08QfIfYe3Sv3zLhHGIwkvZIkszbMsncGgWaXENe9uGEn\nLoydiyKUizXQOn43rgtfhcVc0lK173QD7Z7C7oOnJv4wl5+fL5O/myxVX6wqsc/EymMLHpMjJ44E\nNtFgFCFDnIlLmhBDVXq2az6hPsCV5KqeUwkqRMf8gNhZj35qev5sh/7eF6Il/iiUuSdTWk9uLQxG\nWk5oKat3rQ58osEqQjpQ9+JvcnVDg7C/3Fyl5cstV+5YYefBtCTTKmhZ2b1tauIPQ8dyjslTi56S\n0s+VlkrDK8m49HGSl59nz8SDVYR0qvcxP7KNm6uHChOOMbudnQdTO6dl58FTE3+YWfjzQrlw5IXC\nYCT1w1TZdXiXvTMIVhEy1MXpEswvXErPvsLxLMXtnC7xh4Im/jCx6/AuSf0wVRiMXDjyQln488Lg\nzCiYW2ooz/dL8DvcupMWJ5yqUcJBuDa++0MTv8vl5efJ2JVjpdLwShL3bJw8tegpOZZzrPAvhNnV\nN0FT1HVxhQi3n64JP3jsXLZ2TUureqIk8a/etVpaTmgpDEaumnSVrN+zvugvhOH19iVWXIyFFd9P\ndZRfwsm6RbgdpFRgtHE3ChL/kRNH5PEFj0upZ0tJwgsJMum7SdY6VAvXugp/WdkLpk4tvNRfwuXh\npoNCtKxq5WH3+vYn8RvP+O6SkpIi6enpTodhmzk/zaH37N5sObiFe5rew4ttXyQhPsHal2NiPNvD\nmYyB/Hx7A3VScjJs2fLn4UlJsHnzH++NKfj7JVge06ZBjx6Qnf3HsPh4GD8eUlP9mpQtomVVKw+7\n17cxZpWIpFiat/+TV1btOLSDLu93oeM7HYmPi+eLu77grZvesp70ARIT/RserrZutTY8Kang8Uqw\nPAYOPD3pg+f9wIF+T8oW0bKqS2raNE/5ICbG83faNKcjCoyT61sTfxDk5ecx8uuR1Btdj9k/zWZI\nmyFkpGVwZdKV/k9s6FBPMdRXfLxnuJsEulda3QtsXB5WjzWhEi6rOtgK2pROnZ1t2eIpJW/Z4nkf\nzsnf0fVttU4olK9wruNftXOVpIxPEQYj1/33OtmYtTHwibqpIrogdrRS+dtpjA3Lw4116m5f1cEW\nqu4N3EKv6gnzxH/o+CF5cO6DEvNMjFQfUV2m/zDd+tOwwp1dGTTEWU+vonGfUHVoFon8SfzauBsg\nEeHjzI/pN7cfOw/vJC0ljWHXDKNS2UpOhxY6YdwqOW2ap05/61ZPrdLQoc407CqPwjalwpzZ9h/N\n/GncLRXsYCLZlgNb6Du3L7N+nEWT6k348LYPuaTmJU6HFXqJiQVfkRMGrZKpqZro3aSwTSkhAY4d\n+/MVWNHW/mEXbdwtgZy8HF768iXqj6nPok2LeKntS6T3SI/OpA/ubJV04hKQEM0z0q5u8VXYpvTa\na57LbJOSPCeSSUnOXXYbEazWCYXy5eY6/q+2fSWN32gsDEZufOdG2XJgi9MhuUMg9fN21+07UXkf\nonlGQ7tEtDdwlxRax2+/A8cPMODTAYxbNY7zK5zP69e/zs11b8YUdkORsiYYd1FZvRnMTiGapxM/\nTYUH22/gMsa0N8ZsMMZsNMb0L+Dz1saYg8aYDO9rkNXvup2IMP2H6dQdVZfx347nwUseZH2f9dxS\n7xZrST+Sz8vtUNhdVP/8Z8mXWagv0J82reBsHIR5uu3eAxWmijslAGKBn4HaQGnge6D+GeO0BmaV\n5LsFvdxS1fNT1k/SdkpbYTCSMj5FVu1c5d8EouG8PFBWnkRtjEivXtanGcoL9It7EK3N83TjvQfK\nHfCjqsdKib8FsFFEfhGRk8C7wE0WjyuBfNcxJ3JPMHTJUBqOaciK7SsYdf0oVty7gmbnNfNvQm7r\nE8CNrFz5IwJjx1ov+YeysbmgdRzEebqxHd1OeoIcIsUdGYAuwASf992BUWeM0xrYB6wG5gINrH63\noJeTJf7FmxZL3VF1hcHIre/fKjsO7Sj5xILx+KdIa/kqrsRc0mJtqJZTUWcsQZqnU5tAsOerJ8iB\nwc47dy0m/rOB8t7/OwA/Wf2uz2c9gHQgPTExMfhL6Qx7ju6Ru2fcLQxGkl9Nljk/zgl8onafl0fq\nnuGbUWJji6/2cdMBLxR1LyHM9IXNKhSbnlZjBcbuxN8SmO/zfgAwoJjvbAaqluS7EuISf35+vkz6\nbpIkvJAgpZ4tJf0X9pejJ4/aM3G795Zo2DOK6nPfjQe8YGfEEB7si5pVKDa9UDwf2Y1P4bKL3Ym/\nFPALUIs/GmgbnDHOufD7paEtgK2AsfLdgl6hSvzr96yXqyZdJQxGWr3VSlbvWm3/TOzcOsLxyeEl\n0auXteTvlgNeMDNACA/2Rc2qJJuev4sl2D810p+7a2vi90yPDsCPeK7QGegdlgakef/vC6z1JvYV\nQKuivlvcK9iJP/tktjy16CmJezZOKg+vLOPTx0tefl7wZmjXzU2FVYO4JQHaaepUa9U+kS6EB/ui\nZuVvUi5JYgx2MrXzwOLGk2/bE3+oX8FM/As2LpC/vPYXYTDS7aNu8tuR34I2LxGxvjUXdHCw0vDp\ndDEjmIor9UfiAe+UoupXHCjx+5uUS5oYfXeDhATPy64TqaI2J3+n78aTb038Bdh1eJfc8eEdwmCk\nzsg68unPn9o+jwJZ2QP87YQ8NtY9FYvBVFTic+qAF4qK3eIO+A7U8Z/63OpPt5QYi5hgMEr/xXX5\n7M/0tcTv8sSfl58nY1eOlYrPV5TSz5WWpz9/Wo7lHLNt+sWysgdoJ+QFKywBJiQ4l/SD3ZBb3LYQ\nzIP91KkyNeEBSWKTGPIkKeFw8K5DKGZZBiOxWjmBtjr9qKjjD/XLrsT//a7vpeWElsJgpM3kNpK5\nJ9OW6frFyhZspSHTLcWKUHPTpRPBLOZZyUrBPODbnMmKnVxhyzIhQUSCV5VS3LHVn+m7adMU0cQv\nR04ckccWPCaxz8RKtReryZSMKc49DcvKDlXUTuC2YkU0C2bFrpWzvmAe8INwUCsyMRZz41uwq1Lc\nWFUTqKhO/J9s+ESSXkkSBiP3/d99kpWdVeJp2aa4okFRBwe3FSvCXSDLM5jZorizvmAf8EPdWllM\nw7Wrbo8Ik30wKhP/gWMH5O/v/V0YjDQY3UCWblnq9zQcFSYbV1gLNJsEMxsVdwVPsLeHUBeBp04t\n/Pd6Dzah6CKi2On7s84d3oejMvHn5uXKlZOulGFLhsmJ3BN+f19FATuSW7B2bqdbC52Yf2FXrbmp\nvsXqNuP0+pMoTfwiEtybsFT4c+PF176cPusL9fxdkCyLZXWbcUGjgT+JX5/ApSLftGme7pMLe1iK\nPr7KOafWzdatni66hw5114N0rT7yLCbGk+rPZAzk5wcrujNmZfMTuJQKW6ce7VhY0o+kzuzDUWqq\nJ4Hm53v+uinpg/UHIBT2XAkrz5twgCZ+FdmKelBKUlJgz/ZVkS811bONJCV5Su+FbTNh9oQcrepR\nkc0Fp+AqSjhcbeVPVU+pYAejlKMSEwuu5nHpKbgKY6mpYXP2qFU9KrKF2Sm4UqGgiV9FNqt1tMpD\nn3YeFbSqR0W+MDoFd9SpK6BONYZv2eJ5D7r8IoyW+JVSHgVdAZWd7RmuIoomfqWUx9at/g1XYUsT\nv1LKI8xuQlIlp4lfKeWhV0BFDUuJ3xjT3hizwRiz0RjTv4DPU40xq40xPxhjvjTGNPH5bLN3eIYx\nRu/KUsqt9AqoqFHsVT3GmFhgNNAW2A6sNMbMFJF1PqNtAq4Skf3GmOuB8cAlPp+3EZG9NsatlAoG\nvQIqKlgp8bcANorILyJyEngXuMl3BBH5UkT2e9+uAGraG6ZSSim7WEn8NYBtPu+3e4cV5l5grs97\nAT41xqwyxvTwP0SllFJ2svUGLmNMGzyJ/3KfwZeLyA5jzDnAQmNMpogsKeC7PYAeAIl6FYFSSgWN\nlRL/DuACn/c1vcNOY4xpDEwAbhKRrFPDRWSH9+9u4GM8VUd/IiLjRSRFRFKqVatm/RcopZTyi5XE\nvxKoY4ypZYwpDdwOzPQdwRiTCHwEdBeRH32GlzPGVDj1P3AdsMau4JVSSvnPUn/8xpgOwKtALDBR\nRIYaY9IARGSsMWYC0Bk41f9troikGGNq4ynlg6da6R0RKfaiYGPMHp9p+asq4MYriDQu/2hc/tG4\n/BOJcSWJiKXqElc+iCUQxph0qw8jCCWNyz8al380Lv9Ee1x6565SSkUZTfxKKRVlIjHxj3c6gEJo\nXP7RuPyjcfknquOKuDp+pZRSRYvEEr9SSqkihGXiD6S3UIfjuskbV4YxJt0Yc3lB03EiNp/xmhtj\nco0xXdwQlzGmtTHmoHeZZRhjBrkhLp/YMowxa40xX7ghLmPMYz7Lao0xJs8YU8UFcVU0xnxijPne\nu7zuDnZMFuOqbIz52LtffmOMaRiCmCYaY3YbYwq8p8l4jPTGvNoY08z2IEQkrF547iX4GagNlAa+\nB+qfMU4roLL3/+uBr10SV3n+qF5rDGS6ZZn5jLcImAN0cUNcQGtglgu3sUrAOiDR+/4cN8R1xvg3\nAovcEBfwb+AF7//VgH1AaRfENQJ42vt/XeCzECyvK4FmwJpCPu+Ap78zA1wajPwVjiV+t/YWaiWu\nI+Jds0A5PB3YhUKxsXk9AHwI7HZZXKFmJa47gI9EZCv83iWJG+Ly1RWY7pK4BKhgjDF4CkD7gFwX\nxFUfT2EHEckEko0x1YMZlHj6KttXxCg3AVPEYwVQyRhznp0xhGPiD7S30GCxFJcx5hZjTCYwG7gn\nBHFZis0YUwO4BXgjRDFZisurlfeUd64xpoFL4vorUNkYs9jb8+ydLokLAGNMPNAez4HcDXGNAuoB\nO4EfgAdFJN8FcX0P/B3AGNMCSML5buX9zXF+C8fEb5lPb6FPOB3LKSLysYjUBW4GnnM6Hh+vAk+E\nYGf017d4qlMaA68DMxyO55RSwMVAR6Ad8JQx5q/OhnSaG4HlIlJUyTKU2gEZwPlAU2CUMeZsZ0MC\nYDieEnUGnjPe74A8Z0MKPlu7ZQ4Rf3sLvV58egt1Oq5TRGSJMaa2MaaqBP/pZFZiSwHe9ZyJUxXo\nYIzJFZFgJtpi4xKRQz7/zzHGjAnBMrOyvLYDWSJyFDhqjFkCNAF+JHj82cZuJzTVPGAtrruB4d6q\nzo3GmE146tS/cTIu7/Z1N3gaVfE8TfCXIMZkhV+5pESC3ZARhIaRUnhWTC3+aLBpcMY4icBGoJXL\n4rqQPxp3m3lXpnFDbGeMP5nQNO5aWWbn+iyzFsDWYC8zi3HVAz7zjhuPp9fZhk7H5R2vIp465HLB\nXod+LK83gMHe/6t7t/2qLoirEt5GZuB+PHXroVhmyRTeuNuR0xt3v7F7/mFX4heRXGNMX2A+f/QW\nuta3t1BgEJAAjPGWYHMlyB0fWYyrM3CnMSYHOAb8Q7xr2gWxhZzFuLoAvYwxuXiW2e3BXmZW4hKR\n9caYecBqIB+YICJB7XLcj/V4C7BAPGcjQWcxrueAycaYH/AktCckyGe6FuOqB7xtjBFgLZ6q4aAy\nxkzHc7VaVWPMduBpIM4npjl4ruzZCGTjPSOxNYYQ5B2llFIuEtGNu0oppf5ME79SSkUZTfxKKRVl\nNPErpVSU0cSvlFJRRhO/UkpFGU38SikVZTTxK6VUlPl/bj4VeDuUF70AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bc77358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出参数更新之前的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0 * plot_x - b0) / w1\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到分类效果基本是混乱的，我们来计算一下 loss，公式如下\n",
    "\n",
    "$$\n",
    "loss = -(y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}))\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 计算loss\n",
    "def binary_loss(y_pred, y):\n",
    "    logits = (y * y_pred.clamp(1e-12).log() + (1 - y) * (1 - y_pred).clamp(1e-12).log()).mean()\n",
    "    return -logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意到其中使用 `.clamp`，这是[文档](http://pytorch.org/docs/0.3.0/torch.html?highlight=clamp#torch.clamp)的内容，查看一下，并且思考一下这里是否一定要使用这个函数，如果不使用会出现什么样的结果\n",
    "\n",
    "**提示：查看一个 log 函数的图像**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable containing:\n",
      " 0.6412\n",
      "[torch.FloatTensor of size 1]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_regression(x_data)\n",
    "loss = binary_loss(y_pred, y_data)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到 loss 之后，我们还是使用梯度下降法更新参数，这里可以使用自动求导来直接得到参数的导数，感兴趣的同学可以去手动推导一下导数的公式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable containing:\n",
      " 0.6407\n",
      "[torch.FloatTensor of size 1]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 自动求导并更新参数\n",
    "loss.backward()\n",
    "w.data = w.data - 0.1 * w.grad.data\n",
    "b.data = b.data - 0.1 * b.grad.data\n",
    "\n",
    "# 算出一次更新之后的loss\n",
    "y_pred = logistic_regression(x_data)\n",
    "loss = binary_loss(y_pred, y_data)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的参数更新方式其实是繁琐的重复操作，如果我们的参数很多，比如有 100 个，那么我们需要写 100 行来更新参数，为了方便，我们可以写成一个函数来更新，其实 PyTorch 已经为我们封装了一个函数来做这件事，这就是 PyTorch 中的优化器 `torch.optim`\n",
    "\n",
    "使用 `torch.optim` 需要另外一个数据类型，就是 `nn.Parameter`，这个本质上和 Variable 是一样的，只不过 `nn.Parameter` 默认是要求梯度的，而 Variable 默认是不求梯度的\n",
    "\n",
    "使用 `torch.optim.SGD` 可以使用梯度下降法来更新参数，PyTorch 中的优化器有更多的优化算法，在本章后面的课程我们会更加详细的介绍\n",
    "\n",
    "将参数 w 和 b 放到 `torch.optim.SGD` 中之后，说明一下学习率的大小，就可以使用 `optimizer.step()` 来更新参数了，比如下面我们将参数传入优化器，学习率设置为 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用 torch.optim 更新参数\n",
    "from torch import nn\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_regression(x):\n",
    "    return F.sigmoid(torch.mm(x, w) + b)\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], lr=1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 200, Loss: 0.39730, Acc: 0.92000\n",
      "epoch: 400, Loss: 0.32458, Acc: 0.92000\n",
      "epoch: 600, Loss: 0.29065, Acc: 0.91000\n",
      "epoch: 800, Loss: 0.27077, Acc: 0.91000\n",
      "epoch: 1000, Loss: 0.25765, Acc: 0.90000\n",
      "\n",
      "During Time: 0.595 s\n"
     ]
    }
   ],
   "source": [
    "# 进行 1000 次更新\n",
    "import time\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "    # 前向传播\n",
    "    y_pred = logistic_regression(x_data)\n",
    "    loss = binary_loss(y_pred, y_data) # 计算 loss\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad() # 使用优化器将梯度归 0\n",
    "    loss.backward()\n",
    "    optimizer.step() # 使用优化器来更新参数\n",
    "    # 计算正确率\n",
    "    mask = y_pred.ge(0.5).float()\n",
    "    acc = (mask == y_data).sum().data[0] / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.data[0], acc))\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到使用优化器之后更新参数非常简单，只需要在自动求导之前使用**`optimizer.zero_grad()`** 来归 0 梯度，然后使用 **`optimizer.step()`**来更新参数就可以了，非常简便\n",
    "\n",
    "同时经过了 1000 次更新，loss 也降得比较低了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们画出更新之后的结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x10c08ec50>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVNX7B/DPw6Zi5oKaKQpk2k9wocQ1c6lMU9OvW19N\nLXNBxcr6pmmZ3y+Wli2WmluuoWKlVu5LLqmpqaCJClbuW5oE5Y4C8/z+GDREBu7A3ed5v17zgpm5\nM+eZO3eee+65555DzAwhhBD24mV0AEIIIdQnyV0IIWxIkrsQQtiQJHchhLAhSe5CCGFDktyFEMKG\nJLkLIYQNSXIXQggbkuQuhBA25GNUwWXLluXg4GCjihdCCEvas2fPn8xcLr/lDEvuwcHBiI+PN6p4\nIYSwJCI6qWQ5aZYRQggbkuQuhBA2JMldCCFsSJK7EELYkCR3IYSwIUnuQghhQ5LchRDChiyX3JOv\nJuO1ta/hYtpFo0MRQgjTslxy33h8IybtnoSa02pizeE1RocjhBCmZLnk3q1mN/zU9yeULFISbRa2\nwQtLX0Dq9VSjwxJCCFOxXHIHgPqV6mNP5B6MajoKCw8sRNjUMCz9ZanRYQkhhGlYMrkDQBGfInin\nxTuI6x+HCvdUQMevO6L7N92RfDXZ6NCEEMJwlk3ut4RXCMfufrsxpsUYfJP0DUKnhuLrg1+DmY0O\nTQghDGP55A4Avt6+GNl0JH4e8DNCSoWg2zfd0HlRZ5y/ct7o0IQQwhC2SO63hJUPw46+O/Dhkx9i\n9eHVCJ0SinkJ86QWL4TwOLZK7gDg4+WDYY8OQ8LABISWC8ULS19Auy/b4cylM0aHJoQQurFdcr/l\nobIPYUvvLZjQagI2n9iMsKlhmLlnptTihRAewbbJHQC8vbwxpOEQ7B+4H4/c/wgiV0ai5fyWOPH3\nCaNDE0IITdk6ud9StUxVbHx+I6a3nY5dZ3eh5tSamLJ7ChzsMDo0IYTQhEckdwDwIi8MiBiAxKhE\nNKnSBC+teQktYlrgSOoRo0MTQgjV5ZvciWgOEV0gooMunicimkRER4hoPxE9on6Y6qlSsgrW9FiD\nuR3mIuF8AmpNq4XxO8Yj05FpXFCxsUBwMODl5fwbG2tcLEIIW1BSc/8CQOs8nn8aQLWsWySAaYUP\nS1tEhN7hvZE0OAktH2iJoeuH4tE5jyIpOUn/YGJjgchI4ORJgNn5NzJSErwQolDyTe7MvBVAXiNz\ndQAwj512AihFRPerFaCWKpaoiGXdlmFhp4U4knoED3/+MN778T2kZ6brF8TIkcC1a3c+du2a83Eh\nhCggNdrcKwE4ne3+mazH7kJEkUQUT0TxycnmGAOGiNC9VnckRiWi/UPtMXLTSDSc3RAJ5xP0CeDU\nKfceFx5FWuxEQel6QpWZZzBzBDNHlCtXTs+i83XfPfdhcdfFWNJ1Cc5cOoOImRGI3hyNm5k3tS24\nShX3Hi8gSRLWIy12ojDUSO5nAVTOdj8w6zFL6hzaGUlRSfh32L8xestoRMyIQPzv8doVOHYs4O9/\n52P+/s7HVSJJQj167iSlxU4UCjPnewMQDOCgi+faAlgDgAA0BLBbyXvWrVuXzW75L8u54viK7D3a\nm0esH8HX069rU9CCBcxBQcxEzr8LFqj69kFBzM60fuctKEjVYmwp+1cTEMDs53fnOvT3V/3ruo0o\n9++NSJvy8qPxZioUAhDPSvJ2vgsAXwI4ByAdzvb0vgAGAhiY9TwBmALgKIADACKUFGyF5M7M/Nf1\nv7jvsr6MaPBDnz3E209tNzokt5ktSVjFggXO5J3butNjJ+nuTlnL5JvbutByxyZcUy25a3WzSnK/\nZd2RdVzl0ypM0cSvrX2Nr9686v6bGFT1kZp7wbhab3rtJN1JqFonX9mGzEOSuwYupV3iqJVRjGhw\n1YlVefPxzcpfbGDVR2pdBePqiEfPBKe0PqB18pWjP/OQ5K6hH47/wA9MfIARDY5aGcWX0i7l/yKD\nqz7SXuo+JTV3s+wktU6+UnM3D6XJ3WPGllFT8+Dm2D9wP15t8CqmxU9DrWm1sP7o+rxfZHB/9h49\ngBMnAIfD+bdHD12KtbTcOjL5+gIBAQAREBQEzJhhjnWpdY9aHTp1CZVJci+g4n7F8WnrT7GtzzYU\n9SmKpxY8hf7L++Ni2sXcX6BTf3ahnh49nMk7KOifZD53LvDnn+bbSWqdfHNbF2bZsQkXlFTvtbhZ\nuVkmp+vp13nE+hHsNdqLK42vxCt/XXn3QtLwLTQmTW+eAdIso5+iPkXx/pPvY1e/XShdrDTafdkO\nvb7rhZRrKf8sJFUfoRJXF1JJ05vIjpw7Av1FRERwfLyGV34a5GbmTYzdOhbvbXsPAcUCMLXtVHSq\n0cnosIRN3LraOPuVq/7+Uk/wJES0h5kj8ltOau4q8/P2w+gWoxHfPx4VS1RE50Wd8eziZ3Hh6gWj\nQxM2YMYhCWTcInOS5K6ROhXqYFe/XRj7+Fgs+3UZQqeE4ssDX8KoIyVhD2YbRFTGLTIvSe4a8vX2\nxVuPvYWfB/yMB8s8iOe+fQ4dv+6I3y//bnRowqLM1unKjEcSwkmSuw5Cy4Vie5/t+Ljlx1h3dB3C\npobhi31fSC1euM1s/c1dHTGcPKlvHOJuktx14u3ljdcbv479A/ejVvlaeHHZi2izsA1OXZRJOYRy\nana6UqOt3NURA5E0zRhNessYwMEOTI2bihEbRsCLvPBRy48QWTcSRGR0aMJDqNXrJjYW6NXL2d6e\nU1CQs0umUJfS3jKS3A10/K/j6LeiHzYd34THQx7HzGdm4oHSDxgdlrCx2Fhne7irZpOCJGRXdRIi\nZ597oS7pCmkBIaVDsKHXBsxoNwNxZ+NQa1otfLbrMzhYfhF5ka53BZO9Z4srBel1ExSU++Mysoax\nJLkbjIjQv25/JEYlollQM7yy9hU0ndsUv6X8ZnRopiRd7wout54tORUkIZvtJG9ePKpioGSMAi1u\ndhpbRi0Oh4Nj9sVwqXGluOiYovzhtg85PTPd6LAU02NsExl6tuDyG5++MEMdWWFcG7sM7wSFY8tI\nm7sJnbt8DlGro7D0l6WoV7Ee5naYi7DyYUaHlSe9Lov38sr95J207+YvODjvtvaxY+09hIGrz2+1\nE7/S5m5h95e4H98++y2+6vwVjv99HA9//jDGbB2D9Mx0o0NzSa+LWcx2EY+VuGo+GTTI+X+vXvZr\nqsjeDONqx2bU1b2aU1K91+ImzTLKXLhygZ9d/CwjGhw+PZx/Pvezqu+v1uG0XtOw2eXQ2ig5v+9B\ng+y7Po2e4FwrkGn27OXbpG/5vo/uY593fPjtjW9zWnpaod9TzUSpZ1u4Fdp3rcLO5zCMmCZRj21T\nkrsNpVxL4ee/e54RDQ6bEsa7zuwq1Pup+cO2U43ak3Yedp74Oq8TyFp8t3r9BiS529iq31ZxpfGV\n2Gu0Fw/7fhhfu3mtQO+j9g/bDknRTjup/CxYwOzt7Xk1d60+m17lSXK3ub+v/839l/dnRIOrf1ad\nt53c5vZ72PmQvKCMXid67SDzao+2y85M7x21XkdBktw9xPqj6zl4QjBTNPGQNUP4yo0ril/rSbVU\npYxsptD6+8i+43BVY/f2ttf3r+fRpNTcJbmr7vKNy/zSqpcY0eAHJj7Am45tUvxaOzSlqMnImruW\nZSvtOWKHtnajSJu7JHfNbDmxhR+c9CAjGjxwxUC+mHbR6JAsx8ijGS2PGpT0HPH0Jjk1mKm3jFzE\nZCNNg5oiYWACXm/0OmbsnYGaU2ti3ZF1RodlKWqOl+4uLS/QUnKhjlnHg7GSHj2cV7s6HM6/Rl7x\nK8ndZvx9/fHxUx9je5/tuMfvHrSObY0+y/rgr+t/GR2aZRj1A9VyAC5XOwhvb/13YkIfktxtqmFg\nQ+wdsBdvNXkL8xLmIWxqGJb/utzosEQetDxqcLXjiIkxRy1TqE8GDvMAe37fgz7L+2D/H/vxXK3n\nMLH1RJT1L2t0WEJntybqOHXKWZO3+0BhdiUzMYk73My8iXHbxmHM1jEoXaw0prSZgi6hXYwOSwjh\nJhkVUtzBz9sP/232X+yJ3IPK91ZG18Vd0WVRF/xx5Q+jQxNCaECSu4epdV8t7Oy3E+8/8T5W/LYC\noVNDEbs/FkYdwQkhtCHJ3QP5ePlgRJMR2DdgH6oHVEfP73qiw1cdcPbSWaNDszyPmsZNmJokdw9W\no1wNbHtxGz556hNsOLYBYVPDMOfnOVKLLyCZ31WYiSR3D+ft5Y3XGr2G/YP2I7xCOPou74vWsa1x\n8m8X09YIl/SajUoIJRQldyJqTUS/EtERIhqRy/MliWgFESUQUSIRvah+qEJLD5Z5EJte2IQpbaZg\n+6ntqDmtJqbFTYODZWJSpVxdBWrbadyEqeWb3InIG8AUAE8DCAXQnYhCcyw2GEASM9cB0BzAeCLy\nUzlWoYVsjcReIQ8g6reSOBh1EI0CGyFqdRSemPcEjqYeNTpKS5D5XYWZKKm51wdwhJmPMfNNAF8B\n6JBjGQZQgogIwD0AUgFkqBqpUJ+LRuLgVduxruc6zHpmFvae24va02tj4s6JyHRkGh2xqWk5fEBe\n5CSuyI2S5F4JwOls989kPZbdZAA1APwO4ACAIcxyPO82vX+leTQSExH6PtIXiVGJaBHcAq+uexVN\nv2iKX/78RduYLK5YsX/+DwjQfrwWOYkrXFHrhGorAPsAVAQQDmAyEd2bcyEiiiSieCKKT05OVqlo\nmzDiV6qgkTjw3kCs6L4C8zvOx6HkQwifHo4Ptn2ADIccmGV36+tLSfnnsevXtS9XTuIKV5Qk97MA\nKme7H5j1WHYvAvg2a7jhIwCOA/i/nG/EzDOYOYKZI8qVK1fQmO3JiF+pwkZiIkLP2j2RNDgJbau3\nxYiNI9BodiMcvHBQu9gsxqgkKydxtWGHpi4lyT0OQDUiCsk6SdoNQM7hBU8BeAIAiOg+AA8BOKZm\noLZnxK/UzUbiCvdUwJKuS7CoyyKc/PskHvn8Eby75V2kZ6ZrF6NFGJVk5SSu+mzT1KVkRg8AbQD8\nBuAogJFZjw0EMDDr/4oAvoezvf0ggJ75vafMxJSDUfO7FXDqmOSrydx9SXdGNLjOtDq89/e9moZp\ndkZ+fWafB9dqUzkaPUl6fiDT7FmM2r9SnX5RSw8t5QofV2Dv0d48cuNITktPM2OYmpdvZJI1eh3m\nxQo7n5yMnCRdCUnuVmTRTJN6LZV7L+3NiAaHTgnlnad3mjFMzcs3c5I1itlrwbkxe8yS3K1Gzcxg\n0Na55vAarvxJZfYa7cVD1w3lazevmTFM05RvVe5sqmavBefG6EpHfiS5W4naW5OBv6iLaRd5wIoB\njGhwtUnVeOuJrWYM0xTlW5G7m6pVd6BmPgqT5G4lav8CTPCL2nhsI4dMCGGKJn559ct8+cZl04Vp\ndPlW5Gqd3VpvOZOg2WvBViTJ3UrUrkKa5Bd1+cZlfmX1K0zRxCETQnjjsY2mCtPo8q3I1aaa1/oz\ncy3YiiS5W4kWVUgT/aJ+PPkjV5tUjRENjlweyRfTLpomTKPLt5q8au5y5KMPSe5WklcV0ibZ59rN\nazx03VD2Gu3FgZ8E8prDa4wOKU82We2qy21TlXMW+lKa3GWyDjPo0cM5wlRQEEDk/DtjhvM5W1wq\nBxTzLYaPnvoIO/rswL1F7sXTsU+j99Le+Ov6X0aHdpfcrlDs1cv51Vj1UnS1ZN9UXZGrY01CyR5A\ni5vU3BWw6Rm/tPQ0HrlxJHuP9uYKH1fgpYeWGh3SHfJrepB2eSc5Z+E+NY4IIc0yNmDzvnp7f9/L\ndabVYUSDuy3pxheuXDA6JGbO/6ShDfavqpHmK+XU2hkqTe7SLGNmao0KZdIh7h6+/2Hs7r8b7zR/\nB98kfYPQqaH4+uDXzlqHgZSsXhl10alHD+DECcDhcP7Vcux6q9N75FBJ7mamxtQ+Jh/izs/bD6Oa\njcLeAXsRXCoY3b7phs6LOuP8lfOGxZTbas9J2pWFu/QeOVSSu5m5OtHqTvXIIrM51CxfEz/1/Qkf\nPPkBVh9ejdApoZifMN+QWnzOk4ZEdz6vx9R5RjPpwZ6l6T48s5K2Gy1u0uauEwu22/+S/As3nt2Y\nEQ1uG9uWT188bWg8OduVBw2ydzuznCjVht5t7pLc7c6iPW7mzc/k0hUuMpDJVOok9x2zkR0Oh9Fh\nWTbxuXPi04qbjFVO7EpvGaEeC2ajXC+U8b3CYQPH8vG/jhsam1UTnzubgNUO9iy4iReKJHfxD6tU\na7K4SqBU6iQXH1ucJ++azJmOTENis1riY3Z/h+RqeW9vc246Ztnh5vUzU/MnKMldWJbrBOrgp+Y/\nxYgGN53blA+nHNY9NrMkEne4u0PKa4gBM9aIzbDDzW8EETWPLCS5C8vKK4E6HA6es3cOl3y/JBcb\nU4zH7xjPGZkZusVmxSaAguyQFixw1tStsCMzww43rxjUjk+Su7AsJQn07KWz/MzCZxjR4IazGnLS\nhSRd47NQK1eBd0hmqBErYYYdbl7rSu31KMldWJqSBOpwODh2fyyX+aAM+73rx+9tfY/TM9P1DtUS\nCrJDMkONWCmjd7hSc5fkbl1G/3rycP7yee6yqAsjGlz387qccD7B6JAUM/FqNUWN2CqkzV2SuzVZ\n5Fe+OHExl/+oPPu+48vRP0TzjYwbRoeUJyusVjPvfMzGbL1lyLms/iIiIjg+Pt6QsoWbgoOdY9Lk\nFBTkHC3KRFKupWDI2iGIPRCLWuVrYW6Huahbsa7RYeXKQqvVUmJjnaNrnDrlvLR/7Fh7DWhGRHuY\nOSK/5WRsGZE/vUc8KoQA/wAs6LQAK7qvQMr1FDSY1QBvbngTaRlpRod2FwutVstQc5w8q4+vI8nd\nTrTaGnUf8ch9OT/6xbh2SIxKRO/w3hi3fRwe/vxh/HT6J6PDvIMFVqvlqDVOnskHU1VGSduNFjdp\nc1eZlg24Jm8czi+8dUfWcZVPqzBFE7+29jW+evOqsQFnMflqtSS1uh2auacQ5ISqh9F6azTxmTUl\nH/1S2iUetHIQIxpcdWJV3nx8s1Hh3sHEq9WS1PoZmLmPv9LkLidU7cLLy7n95UTknCbHxtz56D8c\n/wH9VvTDsb+OYXC9wXj/ifdRokgJfQIVmrvVnJK9acbf3/1pEMx8sltOqHoauzXgunH+wJ2P3iKk\nBfYP3I8hDYZgatxU1JpWC+uPrlclZGE8Nea3AdSZBM1wSqr3WtykWUZldmrAdfOzFPSjbzu5jat/\nVp0RDe63rB//ff1vDT6MsCqzNplB2tw9kFm3RncVoOG0oB/92s1r/Mb3b7DXaC+uNL4Sr/x1pQof\nQAjtKE3u0uYuzMeA8wdxZ+Pw4rIXkZiciF61e2FC6wkoU6yMJmUJURjS5i6sy4DzB/Uq1cOeyD0Y\n1XQUvjz4JUKnhOLbQ99qVp4QWpPkLszHoLNZRXyK4J0W7yCufxzuL3E/Oi/qjGcXP4sLVy+oXpbV\nr34U7jHi+5bkLsznVpeHgIB/HitWTLfiwyuEY3e/3RjTYgyW/boMoVNC8eWBL6FWE6Ytrn4Uihn1\nfUtyNxOpzt3p+vV//k9J0TUD+nr7YmTTkdgbuRdVy1TFc98+h45fd8S5y+cK/d5qXSIvrMGo71uS\nu1l4SnVO6Q7MJBkwrHwYtvfZjo9afoR1R9chdGooYvbFFKoWLwOGeRajvm9FyZ2IWhPRr0R0hIhG\nuFimORHtI6JEItqibpgewCTJTFPu7MBU/EUU9oDIx8sHQxsPRcLABNQsXxO9l/VGm4VtcPriabdj\nAex3vZnIm2Hfd359JQF4AzgK4AEAfgASAITmWKYUgCQAVbLul8/vfaWfew5mHsxCLe70X1dpkBC1\nr+3KdGTypJ2T2H+sP5d4rwR/Hv85OxwOQ2MS5mbamZgANAKwLtv9NwG8mWOZKABjlBR46ybJPQcz\nD0OnFnd2YCr9IrRarUdTj/LjMY8zosGPxzzOx1KPufV6u1xvJpQxYiYmJcm9C4BZ2e73AjA5xzIT\nAEwBsBnAHgDPu3ivSADxAOKrVKlS8E9nR55QnXM306rwi9DygMjhcPDn8Z9zifdKsP9Yf560cxJn\nOjIL/8ZC5EFpclfrhKoPgLoA2gJoBWAUEVXPuRAzz2DmCGaOKFeunEpF24RaIx6Zmbv913v0cA7B\n53A4/xZgXWjZ3klEiKwbicSoRDQLaoZX1r6CZl80w28pvxX+zYUoJCXJ/SyAytnuB2Y9lt0ZOJtu\nrjLznwC2AqijTogeRIVkZmoG7MD0uB6qcsnKWPXcKsT8KwYHLxxEnel18PGOj5HpyFSvECHcpCS5\nxwGoRkQhROQHoBuA5TmWWQagCRH5EJE/gAYADqkbqrAFnXdgeu1PiAjP13keSVFJaFW1FYatH4bG\ncxoj8UKiugV5ALncQx35JndmzgDwEoB1cCbsRcycSEQDiWhg1jKHAKwFsB/Abjjb6A9qF7YwhEV/\ndXruT+4vcT+++/d3+LLzlziaehSPzHgEY7eORXpmunaFWlRum5OnXO6hBxkVUiij1hQ3HuTC1Qt4\nZc0r+Drxa4RXCMfcDnMRXiHc6LBMwdXmVKyY82LknMwwA5JZKB0VUpK7UMbM846Z3HeHvsOgVYOQ\ncj0FbzZ5EyMfG4kiPkWMDstQrjYnVzxgtkjFZMhfoS65Zr7AOtboiKTBSeheszve3fou6s6oi7iz\ncUaHZSh3Nxu5etd9ktyFMnLNfKGUKVYG8zrOw8ruK/F32t9oOLshhq8fjuvp1/N/sQ252mwCAmww\nd6lJSHIXythixmDjta3eFolRiej7cF98uONDhH8ejh2ndxgdlu5cbU4TJ9r/cg+9SHIXynjCRVY6\nKVm0JGY8MwPre63HjYwbaDKnCV5d+yqu3rxqdGi6yWtzsvvlHnqRE6pCGOjKzSsYsWEEpsRNwQOl\nH8Ds9rPRPLi50WEJE5MTqkJYwD1+92Bym8nY0nsLCIQWMS0QtSoKl29cNjo0YXGS3IUwgaZBTbF/\n0H681vA1TI+fjprTamL4JwlWvGZMmIQkd2Ftel81q2F5/r7++KTVJ9jeZzvS93XFhyMelCs1RYFJ\nchf60CIp6n2tuk7lNarcCD4/fAikF7/jcbtNzCW0JSdUhfa0GrpA76tmdSzPy8u5/8iJiOFwkKpl\nCWuRE6rCPLSaH1bvq2Z1LM/VRT5U8gyWJC1RvTxhP5Lchfa0Sop6XzVbpoxu5eV2kU/RYg5U6TQd\nXRd3RdfFXfHHlT9UL1fYhyR3oT2tkrCeV83GxgKXLt39uJ+fJuXldpHPrJleODxzNN5/4n0s/3U5\nwqaGYeGBhTCqaVWYnJK5+LS42W6CbJnx2DVX88MOGlT4dabXenc1/2tAgDbl5SPpQhI3nNWQEQ1+\nZuEzfPbSWUPiEPqDWhNka3WzVXL3hMmtCytnEh40yFrrTMuZtgsoIzODx+8Yz0XHFOWS75fkOXvn\nsMPhMCweoQ+lyV16y6hBxjp3n6t1FhAA/Pmn7uHky8Tf8eGUw+i3oh+2ntyKp6o+hZnPzESVkjJa\np11Jbxk9yVjn7nO1blJSzHmljolHxawWUA0/vPADJj89GdtPbUfY1DBMj58OB5t3dguLzthoKZLc\n1SBjnbsvr3Vjxit1TD4qphd5YXD9wTgYdRANAxti0KpBeHLekzj21zGjQ7uLzJOqD0nuajBxrc60\n8lo3Zj3iscBYtMGlgvF9z+8x65lZ2HNuD2pNq4WJOyci05Gp+D20rlVrddmDyEFJw7wWN1udUGVW\nv9eGJ/S+CQjI/SSlt7e9P7dOTl88zW1i2zCiwY1nN+ZDyYfyfY0efQNMeG7aUiC9ZSzMU3rf5PY5\nc97s+Llzo9HO3OFw8PyE+Vx6XGku8m4RHvfjOJ43P8NlUa56fAYFqRKObmXYmSR3K/OkrT97UvP2\n9pzPnZ0OO/Nzl89xx686Mjp1Zy+/ay6L0qNW7Sl1F61IcrcyTzxuXbDAde3dzp+bWbeducPh4LL3\nX8mzKL3qFVZqdTRbrEqTu5xQNSNP631zq/uEK3b93Lfo1JWWiJByvniuz90qSq++AVqfm1brpLCl\ne/Yo2QNocZOaex487bjVVXXR7p/7Fh2b4VwVdW/5VE5LT2Nm92uqZqvZqvnzMWMLKaRZxuLM9ovR\nkqtmKMDen/sWHXfmuRXlXSSN0ak7h04J5Z2nd5o1dMXUTMhmbCFVmtylWUZNhTkWzPlawPR9qlXj\nqtklKMjen/sWHS+Qyq2omNlFsHpcL1y6cQmN5zTGsO+H4Xr6dUXvZ8Y+62q2clm6hVTJHkCLm+1q\n7oWpwpix+qMnT//8JvH39b85cnkkIxpc/bPq/OPJH/N9jRo1W7UPUtWsuZtx04Q0y+hM6RaV25Zs\nxoY9vXlSM5TJbTi6gYMnBDNFE7+8+mW+cuOKy2ULu+lqkTxdXT4REGDuUaWVkuSuNyVVGFdbsqv2\nZrt3ARTmyxxZLt+4zC+vfpkRDQ6ZEMIbj23MdTlFyTmPz6hVvWbBgtwvgDa61q0GSe56U7KVulrG\nUy/eMTM9kq4Zj/lz2HpiK1ebVI0RDY5cHskX0y7etUyeqyqfz6jlCUu7HhBLctebkh9qXr1CTP4j\n9yh6JV2LZJ+rN6/y0HVD2Wu0Fwd+EshrDq9R/uJ8PqOWq8CMPV3UIMndCPnV9vLakk16eO6R9Eq6\nRmSfQmxnO0/v5BqTazCiwb2X9ubUa6n5vyifz6jlftQi+063SXI3IwschgvWPunmdRJdy+yjwvaX\nlp7Gb214i71He/P9H9/Py35ZlvcLFGRYreo1dv25SXI3K6mhm5+WVb78RsLUMvuo+Ln2/L6Ha0+r\nzYgGd1/SnZOvJue+4IIFzH5+d5bn56fbdm/Hn5skdyEKyoi2guzNc1pR+YjkRsYNfmfzO+z7ji+X\n+7AcLzq46O4JuhcsYPb1vbM8X197ZFmDKE3uiq5QJaLWRPQrER0hohF5LFePiDKIqEuhr64Swiha\nXjHq6jJJIu2vRFb5cks/bz+MajYKeyL3IKhUEJ5d8iy6LO6C81fO/7PQyJFAevqdL0xPl2mXdJBv\ncicibwDoccvyAAATqElEQVRTADwNIBRAdyIKdbHcBwC+VztIIdxW2GEBtRq20Mjr2TUa8rHWfbXw\nU9+fMO6JcVj12yqETgnF/IT5zqYBmTzeMD4KlqkP4AgzHwMAIvoKQAcASTmWexnANwDqFTSY9PR0\nnDlzBmlpaQV9C+GGokWLIjAwEL6+vkaHoq5b47TeGvTk1jitgPFj1Ywde2dsgH7z7d767CNHOpNr\nlSrOclVYJz5ePhjeZDg6/F8H9FnWB88vfR6Lkhbhu8CK8Dl99u4XmG1wlthYTdaLofJrtwHQBcCs\nbPd7AZicY5lKALbAeSTwBYAu+b1vbm3ux44d4+Tk5Lvb7YTqHA4HJycn87Fjx4wORX1m7wNnx7N8\n2WRkZvCnP33KxcYU4z7PFuWbRXOcUDVblxWLdauBzqNCTgAwnJkdeS1ERJFEFE9E8cnJyXc9n5aW\nhoCAABCRSmEJV4gIAQEB9jxKMntTgNYzVRjM28sbrzZ8FQcGHcCxNg3xQpub+COgKFjjES8LzN2h\nLdWaCURjSpplzgKonO1+YNZj2UUA+CorKZcF0IaIMph5afaFmHkGgBkAEBERwbkVJoldP7Zd11Wq\nOJticntc6KZqmarY+PxGzAibgQcjhgHwwQdPvoGBEd3NNda4O5UBMzf55aBkHccBqEZEIUTkB6Ab\ngOXZF2DmEGYOZuZgAEsAROVM7HZ04sQJLFy48Pb9ffv2YfXq1bfvL1++HOPGjVOlrN69e2PJkiUA\ngH79+iEpKecpD3GbXnPFiXx5kRcGRgzEwUEH0bhyYwxePRgtYlrgSOoRo0P7hzsnuc04gL0L+SZ3\nZs4A8BKAdQAOAVjEzIlENJCIBmodoJnll9zbt2+PESNc9hwtsFmzZiE09K4OS+IWHSe/EMoElQrC\n2h5rMbv9bCScT0DtabXx6U+fItORaXRo7lUGzN7kl52ShnktbrmdUE1KSlLhdEPhxMTEcK1atbh2\n7drcs2dPZmZ+4YUXePHixbeXKV68ODMzN2jQgO+9916uU6cOjxs3jitXrsxly5blOnXq8FdffcVz\n587lwYMH336Pl19+mRs1asQhISG33y8zM5MHDRrEDz30ED/55JP89NNP31HWLdljaNasGcfFxd2O\n5a233uLatWtzgwYN+Pz588zMfOHCBe7UqRNHRERwREQEb9u2LdfPa4Z1LjzLmYtnuN3CdoxocKNZ\njfhQ8iGjQ1J+ktsEJ+uh8ISqkjZ3Q7y69lXsO79P1fcMrxCOCa0nuHw+MTERY8aMwY4dO1C2bFmk\npqbm+X7jxo3Dxx9/jJUrVwIA7rvvPsTHx2Py5MkAgC+++OKO5c+dO4dt27bhl19+Qfv27dGlSxd8\n++23OHHiBJKSknDhwgXUqFEDffr0UfyZrl69ioYNG2Ls2LF44403MHPmTLz99tsYMmQIXnvtNTRp\n0gSnTp1Cq1atcOjQIcXvK4RWKt1bCcu7LcfCAwvxytpXED49HNHNozG08VD4eBmUknr0UHZkZ2RX\nVjeZ6ryG0TZt2oSuXbuibNmyAIAyZcqo+v7/+te/4OXlhdDQUPzxxx8AgG3btqFr167w8vJChQoV\n0KJFC7fe08/PD+3atQMA1K1bFydOnAAAbNiwAS+99BLCw8PRvn17XLp0CVeuXFH185iORXoxCOfJ\n/B61eyApKgntqrfDmxvfRMNZDXHgjwNGh5Y3CzX5mbbmnlcNW28+Pj5wOJy9PB0OB27evFmg9ylS\npMjt/51HV4Xn6+t7u9eLt7c3MjIyADjj3LlzJ4oWLapKOaZnoV4M4h/33XMfljy7BEuSliBqVRTq\nzqiLkY+NxJuPvQk/bz+jw8ud0lq+waTmns3jjz+OxYsXIyUlBQBuN8sEBwdjz549AJw9YNKzxsoo\nUaIELl++fPv1Oe8r8eijj+Kbb76Bw+HAH3/8gc2bN6vwSYCnnnoKn3322e37+/ap28RlOhbqxSDu\n1iW0C5IGJ6FrWFdEb4lGvZn1sOf3PUaHZWmS3LMJCwvDyJEj0axZM9SpUwf/+c9/AAD9+/fHli1b\nUKdOHfz0008oXrw4AKB27drw9vZGnTp18Omnn6JFixZISkpCeHg4vv76a0Vldu7cGYGBgQgNDUXP\nnj3xyCOPoGTJkoX+LJMmTUJ8fDxq166N0NBQTJ8+vdDvaWpW6sUgclXWvyxiO8ViWbdlSL6ajAaz\nGuCtjW8hLcOGF9rpgNRqHnBXREQEx8fH3/HYoUOHUKNGDUPiMdKVK1dwzz33ICUlBfXr18f27dtR\noUIFXcq2zToPDs79wqWgIOdVoMJS/rr+F17//nXM3TcXNcrWwJwOc9AwsKHRYZkCEe1h5oj8lpOa\nuwm0a9cO4eHheOyxxzBq1CjdErutyIVLtlK6WGnM6TAHa3usxZWbV9B4dmO8vu51XEu/lv+LBQAT\nn1D1JGq1s3s0DUc8FMZp9WArHIw6iBEbRuCTnZ9g2a/LMLv9bDQLbmZ0aKYnNXdhHzYfkMtT3Vvk\nXkxtOxWbnt8EBqN5THO8tPolXLlp8669hSTJXQhhCS1CWmD/wP0Y0mAIpsZNRc2pNbHh2AajwzIt\nSe5CCMso7lccE1pPwI8v/ogiPkXQcn5L9F/eHxfTLhodmulIchdCWM6jVR7FvgH78EbjNzBn3xyE\nTQ3D6sOr83+hB5HkLoSwpGK+xfBByw+ws+9OlCpaCm0XtsULS19A6vW8x4TyFNZO7gaOJRITE4Nq\n1aqhWrVqiImJ0a1cIQrNZmPw1KtUD3si92BU01FYeGAhQqeE4rtD3xkdlvGUDB2pxa3QQ/4aOO9h\nSkoKh4SEcEpKCqempnJISAinpqZqXq4WZMhfD2Ox+ULd9fO5nzl8ejgjGvzs4mf5wpULRoekOug8\nh6r+NBhLJC4uDrVr10ZaWhquXr2KsLAwHDx48K7l1q1bh5YtW6JMmTIoXbo0WrZsibVr1xa4XCF0\nY/MxeMIrhGN3v90Y02IMvjv0HUKnhuKrg1+pNlCflVg3uWswlki9evXQvn17vP3223jjjTfQs2dP\n1KxZ867lzp49i8qV/5lWNjAwEGfP5pxWVggT8oAxeHy9fTGy6Uj8POBnhJQKQfdvuqPTok44d/mc\n0aHpyrrJ3Z15D93w3//+F+vXr0d8fDzeeOONQr2XEKaj0e/GjMLKh2FH3x348MkPsebwGoRNDUPM\nvhiPqcVbN7lrNJZISkoKrly5gsuXLyMtLffR6CpVqoTTp0/fvn/mzBlUqlSpUOUKoQsPG4PHx8sH\nwx4dhoSBCQgtF4rey3qj7cK2OH3xdP4vtjolDfNa3FSZQ1XpvIdueOaZZzg2NpbHjBlze/7TnFJS\nUjg4OJhTU1M5NTWVg4ODOSUlpdBlG0FOqHogDX43VpDpyOSJOyey/1h/LvFeCf48/nN2OBxGh+U2\nKDyhau3krrKYmBju1KkTMzNnZGRw/fr1eePGjbkuO3v2bK5atSpXrVqV58yZo2eYqjJ6nQuht6Op\nR7nFFy0Y0eDHYx7nY6nHjA7JLUqTu4zn7uFknQtPxMyYuXcmhn4/FJmciXFPjMPg+oPhReZvqZbx\n3IUQwgUiQmTdSByMOojHqjyGV9a+guZfNMfhlMNGh6YaSe55OHDgAMLDw++4NWjQwOiwhBAqqVKy\nCtb0WIO5HebiwIUDqD29NsbvGI9MR6bRoRWaTNaRh1q1atl/YmkhPBwRoXd4bzxV9SkMWjUIQ9cP\nxaKkRZjbYS5Cy4UaHV6BSc1dCCEAVCxREUv/vRQLOy3E0dSjePjzh/Hej+8hPTPd6NAKRJK7EEJk\nISJ0r9UdSYOT0OGhDhi5aSQazGqAhPMJRofmNknuQgiRQ/ni5bGo6yIs6boEZy+fRcTMCPzvh//h\nZuZNo0NTTJK7EEK40Dm0M5KiktCtZje8s/Ud1J1RF3Fn44wOSxFLJ3cjh6Vu3bo1SpUqhXbt2ulX\nqBBCdwH+AZjfcT5WdF+B1OupaDi7IYavH47r6deNDi1Plk3usbFAZCRw8qRzUOqTJ5339Urww4YN\nw/z58/UpTAhhuHbV2yExKhF9wvvgwx0f4uHPH8aO0zuMDsslyyZ3LYalVjqeOwA88cQTKFGiRMEL\nE0JYTqmipTCz/Ux83/N7pGWkocmcJnh17au4evOq0aHdxbL93LUYljr7eO7Xr193OZ67EMKztaza\nEgcGHcCbG9/ExF0TseK3FZjdfjaaBzc3OrTbLFtz12pYahnPXQihRIkiJTC5zWRsfmEzCIQWMS0Q\ntSoKl29cNjo0ABZO7loNS61kPHchhLilWXAz7B+0H/9p+B9Mj5+OmtNq4vuj3xsdlnWTe48ewIwZ\nQFAQQOT8O2OG8/HCGDBgAN5991306NEDw4cPVydYIYSt+fv6Y3yr8djeZzv8ff3RakEr9F3WF3+n\n/W1YTJZN7oAzkZ84ATgczr+FTezz5s2Dr68vnnvuOYwYMQJxcXHYtGlTrss+9thj6Nq1KzZu3IjA\nwECsW7eucIULISyvUeVG+HnAzxjx6AjEJMQgbGoYVv620pBYFI3nTkStAUwE4A1gFjOPy/F8DwDD\nARCAywAGMXOe1+vKeO7mIOtcCG3E/x6PPsv64MCFA+gT3gez2s8CERX6fZWO555vbxki8gYwBUBL\nAGcAxBHRcmZOyrbYcQDNmPkvInoawAwAMjauEMJjRVSMQHxkPN778T0U8S6iSmJ3h5KukPUBHGHm\nYwBARF8B6ADgdnJn5uw9+XcCCFQzSKMcOHAAvXr1uuOxIkWKYNeuXQZFJISwEj9vP0Q3jzakbCXJ\nvRKA7FOFn0HetfK+ANYUJiizkPHchRBWpepFTETUAs7k3sTF85EAIgGgiosO6cys++GLpzJq/lwh\nhPaU9JY5C6BytvuBWY/dgYhqA5gFoAMzp+T2Rsw8g5kjmDmiXLlydz1ftGhRpKSkSNLRATMjJSUF\nRYsWNToUIYQGlNTc4wBUI6IQOJN6NwDPZV+AiKoA+BZAL2b+raDBBAYG4syZM0hOTi7oWwg3FC1a\nFIGBtjg9IoTIId/kzswZRPQSgHVwdoWcw8yJRDQw6/npAP4LIADA1KwmlQwlXXVy8vX1RUhIiLsv\nE0IIkYOifu5ayK2fuxBCiLwp7edu6StUhRBC5E6SuxBC2JBhzTJElAzgZAFfXhbAnyqGoxazxgWY\nNzaJyz0Sl3vsGFcQM9/d3TAHw5J7YRBRfEFO2GrNrHEB5o1N4nKPxOUeT45LmmWEEMKGJLkLIYQN\nWTW5zzA6ABfMGhdg3tgkLvdIXO7x2Lgs2eYuhBAib1atuQshhMiDqZM7EbUmol+J6AgRjcjl+R5E\ntJ+IDhDRDiKqY5K4OmTFtY+I4oko11Ey9Y4r23L1iCiDiLqYIS4iak5EF7PW1z4i+q8Z4soW2z4i\nSiSiLWaIi4iGZVtXB4kok4jKmCCukkS0gogSstbXi1rHpDCu0kT0XdZvcjcR1dQprjlEdIGIDrp4\nnohoUlbc+4noEVUDYGZT3uAcx+YogAcA+AFIABCaY5nGAEpn/f80gF0miese/NPkVRvAL2aIK9ty\nmwCsBtDFDHEBaA5gpQm3r1JwTkpTJet+eTPElWP5ZwBsMkNcAN4C8EHW/+UApALwM0FcHwH4X9b/\n/wdgo07bWFMAjwA46OL5NnDOfUEAGqqdv8xcc789AxQz3wRwawao25h5BzP/lXVXrxmglMR1hbO+\nPQDFAehxYiPfuLK8DOAbABd0iMmduPSmJK7nAHzLzKcAgJn1WGfurq/uAL40SVwMoAQ5Rw+8B87k\nnmGCuELhrNCAmX8BEExE92kcF5h5K5zrwJUOAOax004ApYjofrXKN3Nyz20GqEp5LK/XDFCK4iKi\njkT0C4BVAPqYIS4iqgSgI4BpOsSjOK4sjbMOTdcQUZhJ4qoOoDQRbSaiPUT0vEniAgAQkT+A1nDu\nrM0Q12QANQD8DuAAgCHM7DBBXAkAOgEAEdUHEARzTAXqbo5zi5mTu2LZZoAabnQstzDzd8z8fwD+\nBeBdo+PJMgHAcB1+cO7aC2fTR20AnwFYanA8t/gAqAugLYBWAEYRUXVjQ7rDMwC2M3NetUM9tQKw\nD0BFAOEAJhPRvcaGBAAYB2eteB+cR64/A8g0NiTtqTrNnsrcnQHqaXYxA5QRcd3CzFuJ6AEiKsvM\nWo5xoSSuCABfZY25XxZAGyLKYGYtk2m+cTHzpWz/ryaiqSZZX2cApDDzVQBXiWgrgDoACjwhjUpx\n3dIN+jTJAMriehHAuKwmySNEdBzONu7dRsaVtX29CDhPYgI4DuCYhjEp5VYucZseJxYKeDLCB84v\nIAT/nCgJy7FMFQBHADQ2WVwP4p8Tqo9kfWFkdFw5lv8C+pxQVbK+KmRbX/UBnDLD+oKziWFj1rL+\nAA4CqGl0XFnLlYSzPbe41t+hG+trGoDorP/vy9ruy5ogrlLIOrELoD+c7dyar7Os8oLh+oRqW9x5\nQnW3mmWbtubOOs4ApUFcnQE8T0TpAK4D+DdnfZsGx6U7hXF1ATCIiDLgXF/dzLC+mPkQEa0FsB+A\nA8AsZs61W5uecWUt2hHA9+w8qtCcwrjeBfAFER2AM2ENZ22PvpTGVQNADBExgEQ4m3A1R0RfwtkT\nrCwRnQHwPwC+2eJaDWePmSMAriHr6EK18jX+DQkhhDCALU6oCiGEuJMkdyGEsCFJ7kIIYUOS3IUQ\nwoYkuQshhA1JchdCCBuS5C6EEDYkyV0IIWzo/wGYPq9L7FHL4QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bf2e588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出更新之后的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0 * plot_x - b0) / w1\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到更新之后模型已经能够基本将这两类点分开了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面我们使用了自己写的 loss，其实 PyTorch 已经为我们写好了一些常见的 loss，比如线性回归里面的 loss 是 `nn.MSE()`，而 Logistic 回归的二分类 loss 在 PyTorch 中是 `nn.BCEWithLogitsLoss()`，关于更多的 loss，可以查看[文档](http://pytorch.org/docs/0.3.0/nn.html#loss-functions)\n",
    "\n",
    "PyTorch 为我们实现的 loss 函数有两个好处，第一是方便我们使用，不需要重复造轮子，第二就是其实现是在底层 C++ 语言上的，所以速度上和稳定性上都要比我们自己实现的要好\n",
    "\n",
    "另外，PyTorch 出于稳定性考虑，将模型的 Sigmoid 操作和最后的 loss 都合在了 `nn.BCEWithLogitsLoss()`，所以我们使用 PyTorch 自带的 loss 就不需要再加上 Sigmoid 操作了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 使用自带的loss\n",
    "criterion = nn.BCEWithLogitsLoss() # 将 sigmoid 和 loss 写在一层，有更快的速度、更好的稳定性\n",
    "\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_reg(x):\n",
    "    return torch.mm(x, w) + b\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], 1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.6363\n",
      "[torch.FloatTensor of size 1]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_reg(x_data)\n",
    "loss = criterion(y_pred, y_data)\n",
    "print(loss.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 200, Loss: 0.39538, Acc: 0.88000\n",
      "epoch: 400, Loss: 0.32407, Acc: 0.87000\n",
      "epoch: 600, Loss: 0.29039, Acc: 0.87000\n",
      "epoch: 800, Loss: 0.27061, Acc: 0.87000\n",
      "epoch: 1000, Loss: 0.25753, Acc: 0.88000\n",
      "\n",
      "During Time: 0.527 s\n"
     ]
    }
   ],
   "source": [
    "# 同样进行 1000 次更新\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "    # 前向传播\n",
    "    y_pred = logistic_reg(x_data)\n",
    "    loss = criterion(y_pred, y_data)\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    # 计算正确率\n",
    "    mask = y_pred.ge(0.5).float()\n",
    "    acc = (mask == y_data).sum().data[0] / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.data[0], acc))\n",
    "\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，使用了 PyTorch 自带的 loss 之后，速度有了一定的上升，虽然看上去速度的提升并不多，但是这只是一个小网络，对于大网络，使用自带的 loss 不管对于稳定性还是速度而言，都有质的飞跃，同时也避免了重复造轮子的困扰"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一节课我们会介绍 PyTorch 中构建模型的模块 `Sequential` 和 `Module`，使用这个可以帮助我们更方便地构建模型"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}