{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 深层神经网络\n",
    "前面一章我们简要介绍了神经网络的一些基本知识，同时也是示范了如何用神经网络构建一个复杂的非线性二分类器，更多的情况神经网络适合使用在更加复杂的情况，比如图像分类的问题，下面我们用深度学习的入门级数据集 MNIST 手写体分类来说明一下更深层神经网络的优良表现。\n",
    "\n",
    "## MNIST 数据集\n",
    "mnist 数据集是一个非常出名的数据集，基本上很多网络都将其作为一个测试的标准，其来自美国国家标准与技术研究所, National Institute of Standards and Technology (NIST)。 训练集 (training set) 由来自 250 个不同人手写的数字构成, 其中 50% 是高中学生, 50% 来自人口普查局 (the Census Bureau) 的工作人员，一共有 60000 张图片。 测试集(test set) 也是同样比例的手写数字数据，一共有 10000 张图片。\n",
    "\n",
    "每张图片大小是 28 x 28 的灰度图，如下\n",
    "\n",
    "![](https://ws3.sinaimg.cn/large/006tKfTcly1fmlx2wl5tqj30ge0au745.jpg)\n",
    "\n",
    "所以我们的任务就是给出一张图片，我们希望区别出其到底属于 0 到 9 这 10 个数字中的哪一个。\n",
    "\n",
    "## 多分类问题\n",
    "前面我们讲过二分类问题，现在处理的问题更加复杂，是一个 10 分类问题，统称为多分类问题，对于多分类问题而言，我们的 loss 函数使用一个更加复杂的函数，叫交叉熵。\n",
    "\n",
    "### softmax\n",
    "提到交叉熵，我们先讲一下 softmax 函数，前面我们见过了 sigmoid 函数，如下\n",
    "\n",
    "$$s(x) = \\frac{1}{1 + e^{-x}}$$\n",
    "\n",
    "可以将任何一个值转换到 0 ~ 1 之间，当然对于一个二分类问题，这样就足够了，因为对于二分类问题，如果不属于第一类，那么必定属于第二类，所以只需要用一个值来表示其属于其中一类概率，但是对于多分类问题，这样并不行，需要知道其属于每一类的概率，这个时候就需要 softmax 函数了。\n",
    "\n",
    "softmax 函数示例如下\n",
    "\n",
    "![](https://ws4.sinaimg.cn/large/006tKfTcly1fmlxtnfm4fj30ll0bnq3c.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于网络的输出 $z_1, z_2, \\cdots z_k$，我们首先对他们每个都取指数变成 $e^{z_1}, e^{z_2}, \\cdots, e^{z_k}$，那么每一项都除以他们的求和，也就是\n",
    "\n",
    "$$\n",
    "z_i \\rightarrow \\frac{e^{z_i}}{\\sum_{j=1}^{k} e^{z_j}}\n",
    "$$\n",
    "\n",
    "如果对经过 softmax 函数的所有项求和就等于 1，所以他们每一项都分别表示属于其中某一类的概率。\n",
    "\n",
    "## 交叉熵\n",
    "交叉熵衡量两个分布相似性的一种度量方式，前面讲的二分类问题的 loss 函数就是交叉熵的一种特殊情况，交叉熵的一般公式为\n",
    "\n",
    "$$\n",
    "cross\\_entropy(p, q) = E_{p}[-\\log q] = - \\frac{1}{m} \\sum_{x} p(x) \\log q(x)\n",
    "$$\n",
    "\n",
    "对于二分类问题我们可以写成\n",
    "\n",
    "$$\n",
    "-\\frac{1}{m} \\sum_{i=1}^m (y^{i} \\log sigmoid(x^{i}) + (1 - y^{i}) \\log (1 - sigmoid(x^{i}))\n",
    "$$\n",
    "\n",
    "这就是我们之前讲的二分类问题的 loss，当时我们并没有解释原因，只是给出了公式，然后解释了其合理性，现在我们给出了公式去证明这样取 loss 函数是合理的\n",
    "\n",
    "交叉熵是信息理论里面的内容，这里不再具体展开，更多的内容，可以看到下面的[链接](http://blog.csdn.net/rtygbwwwerr/article/details/50778098)\n",
    "\n",
    "下面我们直接用 mnist 举例，讲一讲深度神经网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "from torchvision.datasets import mnist # 导入 pytorch 内置的 mnist 数据\n",
    "\n",
    "from torch import nn\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
      "Processing...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "# 使用内置函数下载 mnist 数据集\n",
    "train_set = mnist.MNIST('../../data/mnist', train=True, download=True)\n",
    "test_set  = mnist.MNIST('../../data/mnist', train=False, download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看看其中的一个数据是什么样子的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a_data, a_label = train_set[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABAElEQVR4nGNgGMyAWUhIqK5jvdSy\n/9/rGRgYGFhgEnJsVjYCwQwMDAxPJgV+vniQgYGBgREqZ7iXH8r6l/SV4dn7m8gmCt3++/fv37/H\ntn3/iMW+gDnZf/+e5WbQnoXNNXyMs/5GoQoxwVmf/n9kSGFiwAW49/11wynJoPzx4YIcRlyygR/+\n/i2XxCWru+vv32nSuGQFYv/83Y3b4p9/fzpAmSyoMnohpiwM1w5h06Q+5enfv39/bcMiJVF09+/f\nv39P+mFKiTtd/fv3799jgZiBJLT69t+/f/8eDuDEkDJf8+jv379/v7Ryo4qzMDAwMAQGMjBc3/y3\n5wM2V1IfAABFF16Aa0wAOwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x10ABFAB70>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里的读入的数据是 PIL 库中的格式，我们可以非常方便地将其转换为 numpy array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(28, 28)\n"
     ]
    }
   ],
   "source": [
    "a_data = np.array(a_data, dtype='float32')\n",
    "print(a_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们可以看到这种图片的大小是 28 x 28"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   3.  18.\n",
      "   18.  18. 126. 136. 175.  26. 166. 255. 247. 127.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.  30.  36.  94. 154. 170. 253.\n",
      "  253. 253. 253. 253. 225. 172. 253. 242. 195.  64.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.  49. 238. 253. 253. 253. 253. 253.\n",
      "  253. 253. 253. 251.  93.  82.  82.  56.  39.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.  18. 219. 253. 253. 253. 253. 253.\n",
      "  198. 182. 247. 241.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.  80. 156. 107. 253. 253. 205.\n",
      "   11.   0.  43. 154.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.  14.   1. 154. 253.  90.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0. 139. 253. 190.\n",
      "    2.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  11. 190. 253.\n",
      "   70.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  35. 241.\n",
      "  225. 160. 108.   1.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  81.\n",
      "  240. 253. 253. 119.  25.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "   45. 186. 253. 253. 150.  27.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.  16.  93. 252. 253. 187.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0. 249. 253. 249.  64.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "   46. 130. 183. 253. 253. 207.   2.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  39. 148.\n",
      "  229. 253. 253. 253. 250. 182.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  24. 114. 221. 253.\n",
      "  253. 253. 253. 201.  78.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.  23.  66. 213. 253. 253. 253.\n",
      "  253. 198.  81.   2.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.  18. 171. 219. 253. 253. 253. 253. 195.\n",
      "   80.   9.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.  55. 172. 226. 253. 253. 253. 253. 244. 133.  11.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0. 136. 253. 253. 253. 212. 135. 132.  16.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]]\n"
     ]
    }
   ],
   "source": [
    "print(a_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以将数组展示出来，里面的 0 就表示黑色，255 表示白色\n",
    "\n",
    "对于神经网络，我们第一层的输入就是 28 x 28 = 784，所以必须将得到的数据我们做一个变换，使用 reshape 将他们拉平成一个一维向量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def data_tf(x):\n",
    "    x = np.array(x, dtype='float32') / 255\n",
    "    x = (x - 0.5) / 0.5 # 标准化，这个技巧之后会讲到\n",
    "    x = x.reshape((-1,)) # 拉平\n",
    "    x = torch.from_numpy(x)\n",
    "    return x\n",
    "\n",
    "train_set = mnist.MNIST('./data', train=True, transform=data_tf, download=True) # 重新载入数据集，申明定义的数据变换\n",
    "test_set = mnist.MNIST('./data', train=False, transform=data_tf, download=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([784])\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "a, a_label = train_set[0]\n",
    "print(a.shape)\n",
    "print(a_label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "# 使用 pytorch 自带的 DataLoader 定义一个数据迭代器\n",
    "train_data = DataLoader(train_set, batch_size=64, shuffle=True)\n",
    "test_data = DataLoader(test_set, batch_size=128, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用这样的数据迭代器是非常有必要的，如果数据量太大，就无法一次将他们全部读入内存，所以需要使用 python 迭代器，每次生成一个批次的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a, a_label = next(iter(train_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([64, 784])\n",
      "torch.Size([64])\n"
     ]
    }
   ],
   "source": [
    "# 打印出一个批次的数据大小\n",
    "print(a.shape)\n",
    "print(a_label.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 使用 Sequential 定义 4 层神经网络\n",
    "net = nn.Sequential(\n",
    "    nn.Linear(784, 400),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(400, 200),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(200, 100),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(100, 10)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=784, out_features=400)\n",
       "  (1): ReLU()\n",
       "  (2): Linear(in_features=400, out_features=200)\n",
       "  (3): ReLU()\n",
       "  (4): Linear(in_features=200, out_features=100)\n",
       "  (5): ReLU()\n",
       "  (6): Linear(in_features=100, out_features=10)\n",
       ")"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "交叉熵在 pytorch 中已经内置了，交叉熵的数值稳定性更差，所以内置的函数已经帮我们解决了这个问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 定义 loss 函数\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.SGD(net.parameters(), 1e-1) # 使用随机梯度下降，学习率 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 0, Train Loss: 0.525527, Train Acc: 0.830690, Eval Loss: 0.214004, Eval Acc: 0.929292\n",
      "epoch: 1, Train Loss: 0.169223, Train Acc: 0.948527, Eval Loss: 0.156571, Eval Acc: 0.951048\n",
      "epoch: 2, Train Loss: 0.119509, Train Acc: 0.962537, Eval Loss: 0.141246, Eval Acc: 0.955301\n",
      "epoch: 3, Train Loss: 0.093633, Train Acc: 0.970349, Eval Loss: 0.096926, Eval Acc: 0.970036\n",
      "epoch: 4, Train Loss: 0.077827, Train Acc: 0.975413, Eval Loss: 0.088236, Eval Acc: 0.971025\n",
      "epoch: 5, Train Loss: 0.062835, Train Acc: 0.980211, Eval Loss: 0.090155, Eval Acc: 0.973200\n",
      "epoch: 6, Train Loss: 0.053678, Train Acc: 0.983109, Eval Loss: 0.084136, Eval Acc: 0.974189\n",
      "epoch: 7, Train Loss: 0.056607, Train Acc: 0.982343, Eval Loss: 0.075727, Eval Acc: 0.976562\n",
      "epoch: 8, Train Loss: 0.040552, Train Acc: 0.986774, Eval Loss: 0.065600, Eval Acc: 0.980024\n",
      "epoch: 9, Train Loss: 0.034272, Train Acc: 0.989272, Eval Loss: 0.121962, Eval Acc: 0.963212\n",
      "epoch: 10, Train Loss: 0.030490, Train Acc: 0.990005, Eval Loss: 0.067141, Eval Acc: 0.979233\n",
      "epoch: 11, Train Loss: 0.027200, Train Acc: 0.991188, Eval Loss: 0.160441, Eval Acc: 0.953521\n",
      "epoch: 12, Train Loss: 0.023948, Train Acc: 0.991904, Eval Loss: 0.076049, Eval Acc: 0.980123\n",
      "epoch: 13, Train Loss: 0.018909, Train Acc: 0.993503, Eval Loss: 0.065272, Eval Acc: 0.980518\n",
      "epoch: 14, Train Loss: 0.017229, Train Acc: 0.994386, Eval Loss: 0.067790, Eval Acc: 0.981309\n",
      "epoch: 15, Train Loss: 0.014564, Train Acc: 0.995253, Eval Loss: 0.067104, Eval Acc: 0.981804\n",
      "epoch: 16, Train Loss: 0.013621, Train Acc: 0.995819, Eval Loss: 0.076764, Eval Acc: 0.980716\n",
      "epoch: 17, Train Loss: 0.012969, Train Acc: 0.995836, Eval Loss: 0.154731, Eval Acc: 0.963805\n",
      "epoch: 18, Train Loss: 0.012531, Train Acc: 0.996202, Eval Loss: 0.098053, Eval Acc: 0.975574\n",
      "epoch: 19, Train Loss: 0.010139, Train Acc: 0.996635, Eval Loss: 0.072089, Eval Acc: 0.982002\n"
     ]
    }
   ],
   "source": [
    "# 开始训练\n",
    "losses = []\n",
    "acces = []\n",
    "eval_losses = []\n",
    "eval_acces = []\n",
    "\n",
    "for e in range(20):\n",
    "    train_loss = 0\n",
    "    train_acc = 0\n",
    "    net.train()\n",
    "    for im, label in train_data:\n",
    "        im = Variable(im)\n",
    "        label = Variable(label)\n",
    "        # 前向传播\n",
    "        out = net(im)\n",
    "        loss = criterion(out, label)\n",
    "        # 反向传播\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        # 记录误差\n",
    "        train_loss += loss.data[0]\n",
    "        # 计算分类的准确率\n",
    "        _, pred = out.max(1)\n",
    "        num_correct = float((pred == label).sum().data[0])\n",
    "        acc = num_correct / im.shape[0]\n",
    "        train_acc += acc\n",
    "        \n",
    "    losses.append(train_loss / len(train_data))\n",
    "    acces.append(train_acc / len(train_data))\n",
    "    # 在测试集上检验效果\n",
    "    eval_loss = 0\n",
    "    eval_acc = 0\n",
    "    net.eval() # 将模型改为预测模式\n",
    "    for im, label in test_data:\n",
    "        im = Variable(im)\n",
    "        label = Variable(label)\n",
    "        out = net(im)\n",
    "        loss = criterion(out, label)\n",
    "        # 记录误差\n",
    "        eval_loss += loss.data[0]\n",
    "        # 记录准确率\n",
    "        _, pred = out.max(1)\n",
    "        num_correct = flot((pred == label).sum().data[0])\n",
    "        acc = num_correct / im.shape[0]\n",
    "        eval_acc += acc\n",
    "        \n",
    "    eval_losses.append(eval_loss / len(test_data))\n",
    "    eval_acces.append(eval_acc / len(test_data))\n",
    "    print('epoch: {}, Train Loss: {:.6f}, Train Acc: {:.6f}, Eval Loss: {:.6f}, Eval Acc: {:.6f}'\n",
    "          .format(e, train_loss / len(train_data), train_acc / len(train_data), \n",
    "                     eval_loss / len(test_data), eval_acc / len(test_data)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画出 loss 曲线和 准确率曲线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1132fb390>]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH4RJREFUeJzt3X98XHWd7/HXZyaTZJI0mbaJbZMU2grKFkEKgfUiKP4G\nVi0oIuhVWeGy9SHuetXHLru6Prz7w1Xv6vWuopVFrqJeQVSQ1QL+uC6uImtTWqHll22BNumv9Eea\n378mn/vHOWmnIT+m7SSTOef9fDzmkfPjO3M+PQzvOfOd7znH3B0REYmWRLELEBGRwlO4i4hEkMJd\nRCSCFO4iIhGkcBcRiSCFu4hIBCncJVbMbK2Z/e0JPvffzeyGQtckMhPKil2ASL7M7DngBnf/+Ym+\nhruvKVxFInOXjtwlMsxMBysiIYW7lAQz+xZwCvBvZtZjZn9pZsvMzM3sejPbAfy/sO3dZrbHzA6b\n2a/M7Myc1/mGmf1DOH2JmbWZ2UfNbJ+Z7TazP82znoSZfcLMng+fe4eZ1YXrKs3s22Z2wMw6zWy9\nmS0K111nZtvNrNvMnjWzdxd4V4kACncpEe7+HmAH8BZ3r3H3z+WsfjXwR8Cbwvn7gdOBFwGPAt+Z\n4qUXA3VAE3A9cIuZzc+jpOvCx2uAFUAN8OVw3fvC11wKLATWAP1mVg38C3CZu88DLgQ25bEtkeOm\ncJco+JS797p7P4C73+7u3e4+CHwKePnYUfUEhoG/c/dhd18H9AAvzWOb7wa+4O7b3b0H+GvgmrBr\naJgg1E9z96y7b3D3rvB5o8DLzCzt7rvdfcuJ/qNFpqJwlyjYOTZhZkkz+4yZbTOzLuC5cFX9JM89\n4O4jOfN9BEfh02kEns+Zf55ggMIi4FvAg8CdZrbLzD5nZil37wXeSXAkv9vMfmJmZ+SxLZHjpnCX\nUjLZJUxzl78LWA28nqBrZFm43Apcyy7g1Jz5U4ARYG/4LeB/uPtKgq6XNwPvBXD3B939DcAS4Cng\nXwtclwigcJfSspegf3sq84BB4ABQBXx6hmr5LvDfzWy5mdWE27nL3UfM7DVmdpaZJYEugm6aUTNb\nZGarw773QYIuoNEZqk9iTuEupeSfgE+EI1A+NkmbOwi6SNqBJ4BHZqiW2wm6X34FPAsMAB8K1y0G\nvk8Q7E8CD4VtE8BHCI76DxL8EPyBGapPYs50sw4RkejRkbuISAQp3EVEIkjhLiISQQp3EZEIKtqF\nlurr633ZsmXF2ryISEnasGHDfndvmK5d0cJ92bJltLa2FmvzIiIlycyen76VumVERCJJ4S4iEkEK\ndxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiaCSC/en9nTxuQee4nDfcLFLERGZs0ou3Hcc6OMr/76N\nHQf7il2KiMicVXLh3phJA9De2V/kSkRE5q6SC/emMNx3KdxFRCZVcuGeqUqRTiUV7iIiUyi5cDcz\nGjOV7DqscBcRmUzJhTsE/e7tnQPFLkNEZM4qyXBvyqTVLSMiMoW8wt3MLjWzp81sq5ndPMH6S8zs\nsJltCh+fLHypRzVm0nR0DzI4kp3JzYiIlKxpb9ZhZkngFuANQBuw3szuc/cnxjX9D3d/8wzU+AJj\nwyH3HB7g1IXVs7FJEZGSks+R+wXAVnff7u5DwJ3A6pkta2qNmUpAY91FRCaTT7g3ATtz5tvCZeNd\naGaPmdn9ZnbmRC9kZjeaWauZtXZ0dJxAuWFBR8a660dVEZGJFOoH1UeBU9z9bOBLwL0TNXL3W929\nxd1bGhqmvb/rpBbXBUfu+lFVRGRi+YR7O7A0Z745XHaEu3e5e084vQ5ImVl9waocp6IsScO8CoW7\niMgk8gn39cDpZrbczMqBa4D7chuY2WIzs3D6gvB1DxS62FzBWHeFu4jIRKYdLePuI2Z2E/AgkARu\nd/ctZrYmXL8WuAr4gJmNAP3ANe7uM1g3TZlKnt7TPZObEBEpWdOGOxzpalk3btnanOkvA18ubGlT\na6xL88unOnB3wi8NIiISKskzVCHolukfztKpm3aIiLxASYc7aKy7iMhESjbcdV13EZHJlWy4j52l\nqnAXEXmhkg33BdXlVJQl2HVYZ6mKiIxXsuFuZjRprLuIyIRKNtwh+FFV3TIiIi9U4uFeqXAXEZlA\niYd7mn3dgwyNjBa7FBGROaXkw909uGmHiIgcVdLh3qQTmUREJlTS4d6oE5lERCZU0uG+RDftEBGZ\nUEmHe2UqSX1NObsOK9xFRHKVdLjD2E079IOqiEiu0g/3Op3IJCIyXumHe3iW6gzf+ElEpKREINwr\n6RvKcrhfN+0QERlT8uGuse4iIi9U8uF+dKy7flQVERkToXDXkbuIyJiSD/eF1eWUlyUU7iIiOUo+\n3BMJo7GuUn3uIiI5Sj7cQTftEBEZL0Lhrh9URUTGRCbc93YPMJzVTTtERCAi4d6UqdRNO0REckQi\n3DUcUkTkWNEKd136V0QEyDPczexSM3vazLaa2c1TtDvfzEbM7KrClTi9xjqdpSoikmvacDezJHAL\ncBmwErjWzFZO0u6zwE8LXeR00uVJFlSXa6y7iEgonyP3C4Ct7r7d3YeAO4HVE7T7EPADYF8B68tb\nY6ZSfe4iIqF8wr0J2Jkz3xYuO8LMmoArga9O9UJmdqOZtZpZa0dHx/HWOiXdtENE5KhC/aD6ReCv\n3H3Kgebufqu7t7h7S0NDQ4E2HWjMpGk/pJt2iIgAlOXRph1YmjPfHC7L1QLcaWYA9cDlZjbi7vcW\npMo8NGXS9A5l6RoYoS6dmq3NiojMSfmE+3rgdDNbThDq1wDvym3g7svHps3sG8CPZzPY4dix7gp3\nEYm7abtl3H0EuAl4EHgS+J67bzGzNWa2ZqYLzFdjphLQiUwiIpDfkTvuvg5YN27Z2knaXnfyZR2/\nJp2lKiJyRCTOUAWor6kglTTadSKTiEh0wj2RMJZoOKSICBChcAedyCQiMiZi4a4jdxERiFi4N2XS\n7OkaYEQ37RCRmItUuDdm0ow67O0eLHYpIiJFFblwBw2HFBGJVLg36UQmEREgYuG+JLxph67rLiJx\nF6lwr64oI1OV0pG7iMRepMIdxq7rrrNURSTeohfuGusuIhK9cG/KVKrPXURiL3Lh3phJ0z0wQtfA\ncLFLEREpmkiGO8Bu9buLSIxFNtzV7y4icRa5cB+7aYf63UUkziIX7g3zKihLmI7cRSTWIhfuyYSx\nuE7XdReReItcuMPYWHf9oCoi8RXJcG/KpNXnLiKxFslwb8xUsqdrgOyoF7sUEZGiiGi4p8mOOvu6\n1TUjIvEU2XAHjXUXkfiKZLgfHeuuI3cRiadIhvuSOt2RSUTiLZLhPq8yRW1lmcJdRGIrkuEOuq67\niMRbZMM9GOuuPncRiafIhruO3EUkzvIKdzO71MyeNrOtZnbzBOtXm9ljZrbJzFrN7KLCl3p8GjNp\nDvcP0zM4UuxSRERm3bThbmZJ4BbgMmAlcK2ZrRzX7BfAy939HOD9wG2FLvR4NWaCETO7dfQuIjGU\nz5H7BcBWd9/u7kPAncDq3Abu3uPuY+f6VwNFP++/eb6u6y4i8ZVPuDcBO3Pm28JlxzCzK83sKeAn\nBEfvL2BmN4bdNq0dHR0nUm/ejp6lqh9VRSR+CvaDqrvf4+5nAFcAfz9Jm1vdvcXdWxoaGgq16Qm9\naF4lSd20Q0RiKp9wbweW5sw3h8sm5O6/AlaYWf1J1nZSkgljca1u2iEi8ZRPuK8HTjez5WZWDlwD\n3JfbwMxOMzMLp88FKoADhS72eOm67iISV2XTNXD3ETO7CXgQSAK3u/sWM1sTrl8LvB14r5kNA/3A\nO3N+YC2axkwlG3YcKnYZIiKzbtpwB3D3dcC6ccvW5kx/FvhsYUs7eY2ZND95fDfZUSeZsGKXIyIy\nayJ7hioE4T6cdfb3DBa7FBGRWRXpcD96XXf1u4tIvEQ63HVHJhGJq4iHu27aISLxFOlwn1eZYl5l\nmc5SFZHYiXS4g8a6i0g8RT7cdV13EYmjGIS7LkEgIvETg3BPc6hvmL4h3bRDROIj8uHepEv/ikgM\nRT7cNdZdROJI4S4iEkGRD/dF8ypImMJdROIl8uFelkywuLaSdvW5i0iMRD7cQWPdRSR+4hPuhxXu\nIhIfsQn33Z0DjI4W/eZQIiKzIhbh3pSpZCg7qpt2iEhsxCLcG3XTDhGJmViFu85SFZG4iFm468hd\nROIhFuFeW1lGTUWZumVEJDZiEe5mpkv/ikisxCLcQWPdRSRe4hXu+kFVRGIiNuHelElzsHeI/qFs\nsUsREZlxsQn3xkwlgLpmRCQW4hPudRoOKSLxEZ9w11h3EYmRvMLdzC41s6fNbKuZ3TzB+neb2WNm\n9riZPWxmLy98qSdncV0lZui67iISC9OGu5klgVuAy4CVwLVmtnJcs2eBV7v7WcDfA7cWutCTlUom\nWDRPY91FJB7yOXK/ANjq7tvdfQi4E1id28DdH3b3Q+HsI0BzYcssDJ3IJCJxkU+4NwE7c+bbwmWT\nuR64f6IVZnajmbWaWWtHR0f+VRaI7sgkInFR0B9Uzew1BOH+VxOtd/db3b3F3VsaGhoKuem8NGXS\n7Dqsm3aISPTlE+7twNKc+eZw2THM7GzgNmC1ux8oTHmF1ZhJMzQyyoHeoWKXIiIyo/IJ9/XA6Wa2\n3MzKgWuA+3IbmNkpwA+B97j7M4UvszA0HFJE4mLacHf3EeAm4EHgSeB77r7FzNaY2Zqw2SeBhcBX\nzGyTmbXOWMUn4chZqgp3EYm4snwaufs6YN24ZWtzpm8AbihsaYXXpNvtiUhMxOYMVYC6dIqq8qSu\nDikikRercA9u2qHhkCISfbEKd9BNO0QkHmIX7k06S1VEYiB24d5Yl2Z/zxADw7pph4hEV/zCPRwx\n8/Se7iJXIiIyc2IX7hefXs/C6nI+fNcmOvt0pqqIRFPswv1FtZV87T3n0X6onw98+1GGs6PFLklE\npOBiF+4ALcsW8Jm3n8Vvtx/gkz/ajLsuJCYi0ZLXGapR9LZzm9nW0cMtv9zGixtquOHiFcUuSUSk\nYGIb7gAffcNL2d7Ryz+ue5IVDdW89oxFxS5JRKQgYtktMyaRMD5/9cs5s7GWD/3fjTy1p6vYJYmI\nFESswx2gqryM2957PjWVZVz/jVb29wwWuyQRkZMW+3AHWFxXyW3vPZ8DvYPceEerTnASkZKncA+d\n1VzHF64+h0d3dHLzDx7TCBoRKWkK9xyXn7WEj73xJdy7aRe3/HJrscsRETlhsR4tM5EPvuY0tnX0\n8s8/fYYVDTVcftaSYpckInLcdOQ+jpnxT287i/NOnc9HvreJx9o6i12SiMhxU7hPoDKV5GvvOY+F\n1RX8tzta2XNYd24SkdKicJ9EfU0FX7+uhZ6BEW64Yz19QyPFLklEJG8K9ymcsbiWL71rFU/s6uIj\nd/2e0VGNoBGR0qBwn8Zrz1jEx/9kJQ9s2cPnf/Z0scsREcmLRsvk4f2vXMbWfcFFxlbU1/D285qL\nXZKIyJR05J4HM+PvVp/JhS9eyF//8HHWP3ew2CWJiExJ4Z6nVDLBV959Lk3z0/zZtzbwH3/oKHZJ\nIiKTUrgfh0xVObdfdz516RTv+frv+PPvbmRft4ZJisjco3A/Tsvrq7n/Ly7mw68/nQc27+F1n3+I\nbz3yvEbSiMiconA/AZWpJB9+/Ut44MMXc3ZzHX9772be9tWH2bLrcLFLExEBFO4nZUVDDd++/o/5\n4jvPoe1QH2/98m/4hx8/Qe+gTngSkeLKK9zN7FIze9rMtprZzROsP8PMfmtmg2b2scKXOXeZGVes\nauIXH7mEd56/lNt+/Syv/8JDPLhlT7FLE5EYmzbczSwJ3AJcBqwErjWzleOaHQT+HPjngldYIuqq\nUnz6yrP4wQcupC6d4s++tYEbvtlK26G+YpcmIjGUz5H7BcBWd9/u7kPAncDq3Abuvs/d1wPDM1Bj\nSTnv1Pn824cu4m8uP4PfbN3PG77wK2791TaGs6PFLk1EYiSfcG8CdubMt4XLjpuZ3WhmrWbW2tER\n3XHiqWSCG1/1Yn7+0VfzytPq+fS6p3jLl37NhucPFbs0EYmJWf1B1d1vdfcWd29paGiYzU0XRVMm\nzW3va+Fr7zmPw/3DXLX2Yf7mnsc53Bf7LzgiMsPyubZMO7A0Z745XCZ5etOZi7notHr+18+e4f88\n/BwPbt7DO89fylXnNbOioabY5YlIBOVz5L4eON3MlptZOXANcN/MlhU91RVlfOLNK7nvplfy8qUZ\n1j60jdd+/iGu+urD3LV+Bz0aPikiBWTu059ZaWaXA18EksDt7v6PZrYGwN3XmtlioBWoBUaBHmCl\nu3dN9potLS3e2tpagH9CadrXNcAPN7Zzd+tOtnX0kk4luexli7mqpZlXLF9IImHFLlFE5iAz2+Du\nLdO2yyfcZ0Lcw32Mu7NxZyd3t7bx49/vontwhKUL0rz93Gbefm4zSxdUFbtEEZlDFO4lqH8oy0+f\n2MPdrW38Ztt+3OG/rFjIO1qauexlS0iXJ4tdoogUmcK9xLUd6uOHj7bz/Q1t7DjYR01FGW8+ewnv\naGnm3FPmY6ZuG5E4UrhHxOio87vnDnJ3axvrHt9N/3CWFfXVXLmqiStWNanbRiRmFO4R1DM4wrrH\ndvP9DW38LrwbVMup87liVRN/ctYS5leXF7lCEZlpCveIazvUx4827eLeje38YV8PqaRxyUtfxJWr\nmnjtGS+iMqX+eZEoUrjHhLvzxO4u7t3Yzo827WJf9yDzKsq4/KwlXLGqiT9evkDDKkUiROEeQ9lR\n57fbDnDPxnYe2Lyb3qEsS+oqees5jVy5qokzFtcWu0QROUkK95jrH8rysyf38qON7Tz0TAcjo84Z\ni+dx5aom3npOI0vq0sUuUUROgMJdjjjQM8hPHt/NPRvb2bijE4AV9dWcc0qGVafMZ9XSDGcsnkdZ\nUjfmEpnrFO4yoef293L/5j08uuMQG3ccYn/PEADpVJKzm+uCsD8lw6pTMrxoXmWRqxWR8fIN93yu\nCikRsqy+mg9c8mIg+DG27VB/GPSdbNzZydd/vZ3hbPCB35RJh0EfBP6ZjbVUlE09CmdwJMvhvmE6\n+4fp7Bums2+Izv7hcNkQnX3D9A9nefVLGnjTmYs1qkdkhujIXY4xMJxly64uNu44xMadnWza0Ul7\nZz8A5ckEKxtreVlTLSNZD8I7DOzDYZj3D2cnfe1kwsikUwAc6B2itrKMK1Y1cXXLUl7WVDcr/z6R\nUqduGSmYvV0DwZF9eIT/5J4u0qkkmaoUmXQ5dVUpMulUMF9VTt3YdLqcTFXqyHxNRRlmxuio89vt\nB/he607u37yHoZFRVi6p5eqWZq5Y1USmSidjiUxG4S4l4XDfMPf9vp27Wneyub2L8mSCN565iKtb\nlnLRafUaoy8yjsJdSs6WXYe5u7WNeze109k3TFMmzdvPa+Yd5+nSxyJjFO5SsgaGs/z8yb3ctX4n\nv94aXPr4lact5OqWpfoRVmJP4S6R0N7Zz/db27h7w07aDvVTW1nGW17eyLmnzGd5QzUr6qvVRy+x\nonCXSMn9EfaBzXsYHBk9sm5BdTnL64OgDwK/hhUN1Zy6sGraoZsipUbj3CVSEgnjlafV88rT6hnO\njrLzYB/P7u9le0cv2/f3sr2jh4ee6eDuDW1Hn2PQND/NivqaIPzD4F9WX8Wi2kpSOiNXIkzhLiUn\nlUywoqGGFQ01vO6Pjl3XPTDMs/t7eXZ/L9s6esMPgB7WP3eQvqFjx+BnqlI01FRQX1NB/bwK6mvK\nqa+poGFeRc7ychZWV1Bepg8CKS0Kd4mUeZUpzm7OcHZz5pjl7s7erkG27+/huf197OseYH/PIPu7\nh+joGeSxtk72dw/SOzTxSViZqlQQ9uEHwMLqcuZXl7Ng7FF1dD5TlVJ3kBSdwl1iwcxYXFfJ4rpK\nLnzx5O36h7Ls7xlkX/dgEP7hB8DYdEf3IJvbD3Owd4iugZFJX6emoowFYx8AVanwbzCf+8Ewvyr4\nW5dOkdSYfikghbtIjnR5kqULqvIaVz+cHaWzb5hDfUMc7D36ONQ7xMG+sb/DdPQM8szeHg72Dk16\neQYzyKRTzA8/AILQP/ZDIffbweLaStLl+nYgk1O4i5ygVDIR9M/Pq8j7Of1D2SPBP/ahMPYhkPuh\n0Haoj8fbhzjUO8xQdnTC11pYXU7T/DRNmeDRmEkfmW+en6YuncJM3wbiSuEuMovS5UmayoMAzoe7\n0zeUDT4Ewg+DAz1D7D7cT3vnAO2d/Tyzt5tfPr2PgeFjPwSqy5NHw39+GP7hozadIp1Kki5PBn9T\nSV3qIWIU7iJzmJlRXVFGdUXZlF1F7s7B3iHaO/tpP9Qf/M2Z3rizk86+4Sm3VVGWIF2epCqVpDIn\n9I98AJQnqSpPUplKMq8yRW1lGXXpFLXpFLWVKWrTR+drysv0YVFkCneRCDAzFtZUsLCm4gUjhcb0\nDI6wKwz93sER+oeyDAxn6RvK0j8cPobCx/DRdT2DI3R0Dx6zvmdohKnOf0xYMHKpNl1GbWVwZdDa\nnPnKVJJUMkFZ0ihPJkgljVRZglQyQXm4fGw6Fa4vG5svM8oSCcoSRjJhlCXDv4lE+NeO+RvXrimF\nu0hM1FSU8ZJF83jJonkn/Vqjo0734Ahd/cG1/LsGhunqD+aD6bHlR9ts398TLOsfYSg7SnZ0ds6O\nT4Yhn7Qw8JPBdCJclrDgJLlkwkiE80enx5ZztH3CqCpPUlNRduQbTDAdzNdUBtO1laljls/2uRIK\ndxE5bomEUZcOjsiXnuBrZEed4exo+HBGsqMMhdPD2VGGRkYZGWszcuy6kVEnOzrKSNbJjno470eX\njzrZ7CTLw8eoO6OjkHVnNJzPOkenx9o4R6bHnnuwd4jnD/TRPTBC98DwMZfDmEx5WeLIB8F/fcWp\n3HDxihPcc/lRuItIUQRH1MlIXOVzaGSUnsEg6IPAD6aDZSP0DI7QFa7rGRg5rhFWJyqvcDezS4H/\nDSSB29z9M+PWW7j+cqAPuM7dHy1wrSIic1J5WYIFZcE5CHPFtJ1AZpYEbgEuA1YC15rZynHNLgNO\nDx83Al8tcJ0iInIc8unhvwDY6u7b3X0IuBNYPa7NauAODzwCZMxsSYFrFRGRPOUT7k3Azpz5tnDZ\n8bbBzG40s1Yza+3o6DjeWkVEJE+zOjbH3W919xZ3b2loaJjNTYuIxEo+4d4Ox4x2ag6XHW8bERGZ\nJfmE+3rgdDNbbmblwDXAfePa3Ae81wKvAA67++4C1yoiInmadiiku4+Y2U3AgwRDIW939y1mtiZc\nvxZYRzAMcivBUMg/nbmSRURkOnmNc3f3dQQBnrtsbc60Ax8sbGkiInKizKe6+s9MbtisA3j+BJ9e\nD+wvYDmFNtfrg7lfo+o7Oarv5Mzl+k5192lHpBQt3E+GmbW6e0ux65jMXK8P5n6Nqu/kqL6TM9fr\ny4du6S4iEkEKdxGRCCrVcL+12AVMY67XB3O/RtV3clTfyZnr9U2rJPvcRURkaqV65C4iIlNQuIuI\nRNCcDnczu9TMnjazrWZ28wTrzcz+JVz/mJmdO4u1LTWzX5rZE2a2xcz+YoI2l5jZYTPbFD4+OVv1\nhdt/zsweD7fdOsH6Yu6/l+bsl01m1mVmHx7XZtb3n5ndbmb7zGxzzrIFZvYzM/tD+Hf+JM+d8v06\ng/X9TzN7KvxveI+ZTXiH7OneDzNY36fMrD3nv+Plkzy3WPvvrpzanjOzTZM8d8b3X0G5+5x8EFzq\nYBuwAigHfg+sHNfmcuB+wIBXAP85i/UtAc4Np+cBz0xQ3yXAj4u4D58D6qdYX7T9N8F/6z0EJ2cU\ndf8BrwLOBTbnLPsccHM4fTPw2Un+DVO+X2ewvjcCZeH0ZyeqL5/3wwzW9yngY3m8B4qy/8at/zzw\nyWLtv0I+5vKR+5y+SYi77/bwVoLu3g08yQTXsJ/j5spNVl4HbHP3Ez1juWDc/VfAwXGLVwPfDKe/\nCVwxwVPzeb/OSH3u/lN3HwlnHyG4KmtRTLL/8lG0/TcmvF3o1cB3C73dYpjL4V6wm4TMNDNbBqwC\n/nOC1ReGX5fvN7MzZ7UwcODnZrbBzG6cYP2c2H8EVxqd7H+oYu6/MYv86FVO9wCLJmgzV/bl+wm+\njU1kuvfDTPpQ+N/x9km6tebC/rsY2Ovuf5hkfTH333Gby+FeEsysBvgB8GF37xq3+lHgFHc/G/gS\ncO8sl3eRu59DcI/bD5rZq2Z5+9Oy4DLSbwXunmB1sfffC3jw/XxOjh82s48DI8B3JmlSrPfDVwm6\nW84BdhN0fcxF1zL1Ufuc//8p11wO9zl/kxAzSxEE+3fc/Yfj17t7l7v3hNPrgJSZ1c9Wfe7eHv7d\nB9xD8NU311y4ycplwKPuvnf8imLvvxx7x7qrwr/7JmhT7PfidcCbgXeHH0AvkMf7YUa4+153z7r7\nKPCvk2y32PuvDHgbcNdkbYq1/07UXA73OX2TkLB/7uvAk+7+hUnaLA7bYWYXEOzvA7NUX7WZzRub\nJvjRbfO4ZnPhJiuTHi0Vc/+Ncx/wvnD6fcCPJmiTz/t1RpjZpcBfAm91975J2uTzfpip+nJ/x7ly\nku0Wbf+FXg885e5tE60s5v47YcX+RXeqB8FojmcIfkX/eLhsDbAmnDbglnD940DLLNZ2EcHX88eA\nTeHj8nH13QRsIfjl/xHgwlmsb0W43d+HNcyp/Rduv5ogrOtylhV1/xF80OwGhgn6fa8HFgK/AP4A\n/BxYELZtBNZN9X6dpfq2EvRXj70P146vb7L3wyzV963w/fUYQWAvmUv7L1z+jbH3XU7bWd9/hXzo\n8gMiIhE0l7tlRETkBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIR9P8BgS8DiKl5eLQA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11305b080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title('train loss')\n",
    "plt.plot(np.arange(len(losses)), losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1134fad68>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYXFWd7vHv2/ck3Z3Onc4FAhpIAkrENvg4jKMiCM5o\nwJnHg3qQyUSR5xFGzzhnhnFuzp1xdBjnyJiDGo0cFK9IcKKIOEecOSIk0JALIEkIJN2VpHPpzq2r\nr7/zR+1uiqIvlXR1Kul6P8/TT+291tpVa+9U9q/W3mutrYjAzMysrNgVMDOz04MDgpmZAQ4IZmaW\ncEAwMzPAAcHMzBIOCGZmBjggmOVF0mpJf17sepiNJ3kcgk10knYCH4qInxS7LmanM7cQrORJqih2\nHcxOBw4INqFJugs4G7hf0lFJfyRpoaSQtErSi8BPk7LflrRHUoekhyVdmPU+X5X0t8nyWyTtlvQJ\nSfskpSStHKEOKyU9LemIpB2SPpKTv0JSs6TDkrZLuipJny7pK5JaJR2S9P1xOERmgxwQbEKLiOuB\nF4F3RURtRHw6K/s3gCXAO5L1HwKLgNnA48DdI7z1WcBUYB6wCrhD0rRhyu4DfguoB1YCt0u6BEDS\ncuBrwP8EGoA3AzuT7e4CJgMXJnW6Pa+dNjtJbipbKftURBwbWImINQPLkj4FHJI0NSI6hti2B/jr\niOgF1ks6ClwAPJJbMCL+PWv1Z5J+DPw6maCzClgTEQ8m+S3J5zcCVwMzIuLQwLYnt5tm+XELwUrZ\nroEFSeWSbksu2RzmpV/pM4fZ9kASDAYcB2qHKijpakmPSDooqR14Z9b7LgC2D7HZAuBgVjAwG3cO\nCFYKhutKl53+fmAF8HYyl4IWJukaywdLqga+C3wGmBMRDcD6rPfdBbxqiE13AdMlNYzl881OhAOC\nlYK9wHmjlKkDuoADZK7b/32BPrsKqAbagF5JVwNXZuV/GVgp6XJJZZLmSVocESky9zT+TdI0SZWS\n3lygOpkNyQHBSsE/AH8mqV3SHw5T5mvAC2Su4W9liHsBJyMijgC/D3wLOESmJbIuK/9RkhvNQAeZ\n+wTnJNnXk7lX8QyZG9MfL0SdzIbjgWlmZga4hWBmZgkHBDMzAxwQzMws4YBgZmbAGTZSeebMmbFw\n4cJiV8PM7IyycePG/RExa7RyZ1RAWLhwIRs2bCh2NczMziiSXsinnC8ZmZkZkEdAkLQmmeJ38zD5\nkvSvkrZJempgFsck7ypJzyZ5t2alT5f0oKTnktfhZok0M7NTJJ8WwleBq0bIv5rMlMGLgBuBL0Bm\nsjDgjiR/KfA+SUuTbW4FHoqIRcBDybqZmRXRqAEhIh4GDo5QZAXwtch4BGhIpu5dDmyLiB0R0Q3c\nk5Qd2GZtsrwWuOZkd8DMzAqjEPcQ5pE1jTCwO0kbLh0ysz6mkuU9wJzh3lzSjZI2SNrQ1tZWgOqa\nmdlQin5TOTKTKQ07oVJE3BkRTRHRNGvWqL2mzMzsJBUiILSQeZjHgPlJ2nDpAHuTy0oDT4baV4B6\nmJnZGBRiHMI64GZJ9wCXAh0RkZLUBiySdC6ZQHAdmal/B7a5Abgteb2vAPUwMzstRQQ9fUFXbx/p\nnn66evvo6u2nq6efdG8fXUnay/J6++nq6Rt8fc8l81k4c8q41nPUgCDpG8BbgJmSdgN/CVQmO7ma\nzNOf3glsI/MYwZVJXq+km4EHgHIyz43dkrztbcC3JK0iMwf9ewu4T2Y2gUUEXb39HO/u43h3L+me\nvmS5j87uPjp7BpZ7B9PTvX2DF6Yj571e/t5Zy1np/RH09PXT3Zv8JctdWesvy0/SsvPH8qQBCS45\nZ9q4B4Qz6nkITU1N4ZHKZme2rt4+Ojp7ONzZm3lN93C4M/PXkfxl53V09nAk3cvx7t7BE37/CZ62\nqirKKMt6GKqynoyqnIekZq8qK7Oqooyq8rLMa85y9Qh5VRVlVJeXUV1ZTnVF1mtFGdUV5dRUZl6r\nKzNpNYP5mbSainIqy/WyupwoSRsjomm0cmfU1BVmVnwRwbHuvpedwDuyTuiDr+neIfO7evtHfP+a\nyjLqayqZOqmS+kmVzKmv4dWzK5hcVcHkqnImV5VTU1k+uDypqoLJleVMqsr8Ta4qZ3JlxeByTWU5\n5WVjejR2yXBAMDMgc6I/eKyb1vY0Le3HaWlP09reScuhTvYcTr/sxN47wk90CeqqK5g6OXNSnzqp\nktl1tYMn+KmTKqmvqaD+ZesDAaCC6oryU7jXls0BwaxE9PT1s6cjTUtykm9t78wsJ3+t7Z2ke17+\n631SZTlzG2ponDqJ+dMmDZ7gs0/uuWl11RWU+Rf5GckBwewM0NXbx9F0L8e6+jjS1ZNZ7u7lSJJ2\nNEk7OrDclSynezjW1Ud7Zzf7jnS94sbmzNpq5jXUsPisOt52wWzmTZvE3IZJzEv+GiZXjunatZ1Z\nHBDMiiAiONzZy/5jXew/0sWBY93sP5pZ3n+s+2VpB452c7Srd9T3lKC2qoLamgqmVFdQW11BXU0F\ns+tqqJ80lcapyYk+Oek3Tq2hptKXZ+wlDghmJ6mnrz/5VZ75pZ75Vd4zuHysq5ej6V6OdPVmTv5H\nuzhwrIv9R7o5cKyLnr5XXoeXYPrkKmbWVjOjtoqL5zcwo7aKGVOqqKuppLY6c7Kvq6l4xfKkynJf\nqrExcUAwG8Kxrl6e2XOEranDPJ06zPNtxwYv1QwEgNF6y0Dyq726gvqaSmbUVjGrtpolZ9Uzo7aa\nmbWZE//M2mpm1lUxY0o106dUuUeMFY0DgpW0iGDP4TRPpw6ztfVwEgCOsPPAscHr7fU1Fbx6di2z\n62o4b2bmkkxdckmmtualSzO11ZU56/7VbmcWBwQrGT19/Wzbd5StrZlf/QO//g8d7xksc/b0ySxt\nrOfa181jSWM9S+fWM3dqjW+sWklwQLAJKyLY1NLB+k17+PlzbTy39yjdfZnLPNUVZSw+q453XHgW\nS+fWs6SxnsVn1VFXU1nkWpsVjwOCTSgRQfOudn64eQ/rN6XYfaiTijKx/NzprLxsIUsb67lwbj0L\nZ0yhorzos7+bnVYcEOyM198fPLHrEOs37eGHm1K0dqSpLBeXvXomv3/5Iq5cOoeGyVXFrqbZac8B\nwc5I/f3BhhcOsX5Tih9t3sOew2mqyst48/kz+cSVF/D2JXOYOtmXf8xOhAOCjatjXb1saungxYPH\nmVKV6X3z0l+mX/3kqvK8btr29QePPn+QH27OBIF9R7qoqijjLefP4tbXLOZtS2ZT73sAZifNAcEK\npq8/2LbvKM27DtG8q50nXmznV3uPjDpVcXmZMl04k+6a9TWZ7pt1g104K+no7OHBrXvYf7Sbmsoy\n3nrBbK5+TSNvWzyb2mp/jc0Kwf+T7KTtO5Km+cV2mndl/p7a3TE4xUJ9TQUXL2jgygvP4nULGjhv\n1hQ6ezLz8RxJRu8eSSejetPJ8sCI33Qv+46k2d7WO1i+oly8dfFs3nlRI29dPIvJVf7qmhWa/1dZ\nXtI9fWxu6cj88t/VTvOL7bS0dwJQUSYWN9Zx7evmsWxBA8vObuDcGVMKOiArIjwWwGycOSDYsI51\n9fLAlj18v7mV/7dt/+Ac+PMaJrHs7AZW/tpCli1o4KJ5U8d9kjQHA7Px54BgL9PT18/Pn2vj+0+0\n8uOte0j39DN/2iRWXXYurz9nGsvObmB2XU2xq2lm48ABwYgIHn+xnfuaW/jBUykOHuumYXIlv33J\nfK593Txef840/0I3KwEOCCVse9tR7nuihe83t/LiweNUV5Tx9qVzuGbZPH7j/FlUVXgkr1kpySsg\nSLoK+BxQDnwpIm7LyZ8GrAFeBaSB34uIzZIuAL6ZVfQ84C8i4l8kfQr4MNCW5H0yItaPZWdsdPuO\npLn/yRT3Nbfw1O4OygRvelVmRO87LpzjuXzMStioAUFSOXAHcAWwG3hM0rqI2JpV7JNAc0RcK2lx\nUv7yiHgWWJb1Pi3AvVnb3R4RnynMrthwjnX18qPNe/h+cwv/tW0//QEXzavnz35zCe++eC6z631P\nwMzyayEsB7ZFxA4ASfcAK4DsgLAUuA0gIp6RtFDSnIjYm1XmcmB7RLxQmKrbSPr7g0d3HuTbG3bz\nw80pjnf3sWD6JD761lezYtlcXj27rthVNLPTTD4BYR6wK2t9N3BpTpkngfcAP5e0HDgHmA9kB4Tr\ngG/kbHeLpA8CG4BPRMSh3A+XdCNwI8DZZ5+dR3VL266Dx/ne4y185/Fd7DrYSW11BSuWzeW3L5nv\nm8NmNqJC3VS+DficpGZgE/AE0DeQKakKeDfwJ1nbfAH4GyCS188Cv5f7xhFxJ3AnQFNT0yiTIJSm\n492ZS0Lf3rCbX+w4gARvetUMPnHFBbzjwrOYVOUHqZvZ6PIJCC3Agqz1+UnaoIg4DKwEUOYn6PPA\njqwiVwOPZ19Cyl6W9EXgByda+VIWETy28xDf2biLf38qxbHuPs6ZMZlPXHE+114yj/nTJhe7imZ2\nhsknIDwGLJJ0LplAcB3w/uwCkhqA4xHRDXwIeDgJEgPeR87lIkmNEZFKVq8FNp/cLpSWlvZOvrdx\nN995fDcvHDjOlKpyfvO1jfzO6xfwhoW+JGRmJ2/UgBARvZJuBh4g0+10TURskXRTkr8aWAKslRTA\nFmDVwPaSppDpofSRnLf+tKRlZC4Z7Rwi3xLpnj5+tHkP39m4m//avp8IeON50/n9ty3iqovOYopn\n+zSzAlDEmXNZvqmpKTZs2FDsapwyEcEDW/byd+u3sutgJ/OnTeJ3Xj+f375kPgum+5KQmeVH0saI\naBqtnH9anqZ+tfcIf3X/Fv5r2wHOn1PLV1a+gd9YNKugM4iamWVzQDjNdBzv4faf/Iq7HnmB2uoK\n/urdF/KBS8/2A+HNbNw5IJwm+vqDrz/6Iv/842fp6Ozh/ZeezR9ccQHTp/jh8GZ2ajggnAZ+sf0A\nf3X/Fp7Zc4Q3njedv3zXhSxprC92tcysxDggFNHuQ8f5+/VPs37THuY1TOLfPnAJV190lruOmllR\nOCAUQWd3H1/42Xb+98+2I8EfXHE+N775vHF/6piZ2UgcEE6hiOD+p1L8w/qnSXWkedfFc/mTqxcz\nt2FSsatmZuaAcKpsbungr+/fyqM7D3Lh3Ho+d93rWH7u9GJXy8xskAPCKXDHf2zjMz9+lmmTq/iH\n97yG9zYtoNzjCczsNOOAMM7ufHg7//TAs7zr4rn87TUXMXWSn0hmZqcnB4RxdNcvdvL365/hN1/b\nyO3vvdiDy8zstOYz1Dj59oZd/Pl9W3j7ktn8y39b5mBgZqc9n6XGwbonW/nj7z7Fry+ayefffwmV\nDgZmdgbwmarAHtiyh//xzWaaFk7nzuubPLbAzM4YDggF9LNftXHL15/gNfOmsuZ33+BHV5rZGcUB\noUB+sf0AN35tA6+eXcvalcup9UNrzOwM44BQABtfOMSqtY9x9vTJ3LVqOVMnu2upmZ15HBDGaHNL\nB7/7lUeZXVfN3R+6lBm11cWukpnZSXFAGINn9xzh+i//kvqaSu7+8BuZXV9T7CqZmZ00B4STtKPt\nKB/40i+pLC/j6x++lHmeoM7MznAOCCdh18HjfOBLvyQi+PqHL+WcGVOKXSUzszHLKyBIukrSs5K2\nSbp1iPxpku6V9JSkRyVdlJW3U9ImSc2SNmSlT5f0oKTnktdphdml8bWnI837v/QIx7p6uWvVpbx6\ndl2xq2RmVhCjBgRJ5cAdwNXAUuB9kpbmFPsk0BwRrwU+CHwuJ/+tEbEsIpqy0m4FHoqIRcBDyfpp\nre1IF+//0iMcOtbD11ZdytK5fsylmU0c+bQQlgPbImJHRHQD9wArcsosBX4KEBHPAAslzRnlfVcA\na5PltcA1ede6CNqPd3P9l39Jqj3NV1a+gWULGopdJTOzgsonIMwDdmWt707Ssj0JvAdA0nLgHGB+\nkhfATyRtlHRj1jZzIiKVLO8Bhgwgkm6UtEHShra2tjyqW3iH0z18cM2j7Nh/jC9+sIk3LPSDbcxs\n4inUTeXbgAZJzcAtwBNAX5J3WUQsI3PJ6aOS3py7cUQEmcDxChFxZ0Q0RUTTrFmzClTdE/OF/7ud\nLa2HWf3fL+GyRTOLUgczs/GWz/wKLcCCrPX5SdqgiDgMrASQJOB5YEeS15K87pN0L5lLUA8DeyU1\nRkRKUiOwb4z7Mm52tB3lvJlTeNvi0a6CmZmdufJpITwGLJJ0rqQq4DpgXXYBSQ1JHsCHgIcj4rCk\nKZLqkjJTgCuBzUm5dcANyfINwH1j25Xxk+pI0+hxBmY2wY3aQoiIXkk3Aw8A5cCaiNgi6aYkfzWw\nBFgrKYAtwKpk8znAvZlGAxXA1yPiR0nebcC3JK0CXgDeW7jdKqzW9k6WNrpHkZlNbHlNyRkR64H1\nOWmrs5Z/AZw/xHY7gIuHec8DwOUnUtli6OrtY//Rbua6hWBmE5xHKo9iT0cagMapnqfIzCY2B4RR\ntLZnAoJbCGY20TkgjCLV0Qm4hWBmE58Dwiha2zMBwS0EM5voHBBG0dqRZvqUKmoq/XxkM5vYHBBG\nkWrv9OUiMysJDgijSHWkaZzqy0VmNvE5IIyitb2TuQ1uIZjZxOeAMIKjXb0cTvf6hrKZlQQHhBGk\n2t3l1MxKhwPCCFo7PCjNzEqHA8II3EIws1LigDCC1vZOJJhT74BgZhOfA8IIWjvSzKmrobLch8nM\nJj6f6UaQ6uik0V1OzaxEOCCMINWeZq4HpZlZiXBAGEZE0NrhaSvMrHQ4IAzj0PEe0j397nJqZiXD\nAWEYL0177RaCmZUGB4RhpAYfnekWgpmVhrwCgqSrJD0raZukW4fInybpXklPSXpU0kVJ+gJJ/yFp\nq6Qtkj6Wtc2nJLVIak7+3lm43Rq7wSeluYVgZiWiYrQCksqBO4ArgN3AY5LWRcTWrGKfBJoj4lpJ\ni5PylwO9wCci4nFJdcBGSQ9mbXt7RHymkDtUKK3taSrLxcwp1cWuipnZKZFPC2E5sC0idkREN3AP\nsCKnzFLgpwAR8QywUNKciEhFxONJ+hHgaWBewWo/jlrbO2mcOomyMhW7KmZmp0Q+AWEesCtrfTev\nPKk/CbwHQNJy4BxgfnYBSQuB1wG/zEq+JbnMtEbStBOq+ThLucupmZWYQt1Uvg1okNQM3AI8AfQN\nZEqqBb4LfDwiDifJXwDOA5YBKeCzQ72xpBslbZC0oa2trUDVHV1re9pdTs2spIx6DwFoARZkrc9P\n0gYlJ/mVAJIEPA/sSNYryQSDuyPie1nb7B1YlvRF4AdDfXhE3AncCdDU1BR51HfM+vqDvYfTbiGY\nWUnJp4XwGLBI0rmSqoDrgHXZBSQ1JHkAHwIejojDSXD4MvB0RPxzzjaNWavXAptPdicKbf/RLnr7\ng0a3EMyshIzaQoiIXkk3Aw8A5cCaiNgi6aYkfzWwBFgrKYAtwKpk818Drgc2JZeTAD4ZEeuBT0ta\nBgSwE/hI4XZrbFqSQWnz3OXUzEpIPpeMSE7g63PSVmct/wI4f4jt/hMYsptORFx/QjU9hVLtHpRm\nZqXHI5WHMDAozTOdmlkpcUAYQmt7mslV5dRPyqsBZWY2ITggDCEzKK2GzD1xM7PS4IAwhFRHp8cg\nmFnJcUAYQmuHn5RmZqXHASFHd28/+492eZZTMys5Dgg59h5OE+EeRmZWehwQcrQMPinNAcHMSosD\nQg4/GMfMSpUDQo7WZJSyLxmZWalxQMiR6uikYXIlk6rKi10VM7NTygEhR6o97TmMzKwkOSDkaGnv\n9CynZlaSHBBypDrcQjCz0uSAkOV4dy8dnT3uYWRmJckBIYt7GJlZKXNAyDI4BsHPUjazEuSAkKXV\no5TNrIQ5IGRpbU8jwVluIZhZCXJAyJLq6GRWbTWV5T4sZlZ6fObLkupI0+jLRWZWovIKCJKukvSs\npG2Sbh0if5qkeyU9JelRSReNtq2k6ZIelPRc8jqtMLt08jwozcxK2agBQVI5cAdwNbAUeJ+kpTnF\nPgk0R8RrgQ8Cn8tj21uBhyJiEfBQsl40EeFpK8yspOXTQlgObIuIHRHRDdwDrMgpsxT4KUBEPAMs\nlDRnlG1XAGuT5bXANWPakzHq6Oyhs6fPXU7NrGTlExDmAbuy1ncnadmeBN4DIGk5cA4wf5Rt50RE\nKlneA8wZ6sMl3Shpg6QNbW1teVT35AwOSvM9BDMrUYW6qXwb0CCpGbgFeALoy3fjiAgghsm7MyKa\nIqJp1qxZBansUDwozcxKXUUeZVqABVnr85O0QRFxGFgJIEnA88AOYNII2+6V1BgRKUmNwL6T2oMC\nGRiUNs8tBDMrUfm0EB4DFkk6V1IVcB2wLruApIYkD+BDwMNJkBhp23XADcnyDcB9Y9uVsWntSFNZ\nLmbWVhezGmZmRTNqCyEieiXdDDwAlANrImKLpJuS/NXAEmCtpAC2AKtG2jZ569uAb0laBbwAvLew\nu3ZiUu2dzKmvoaxMxayGmVnR5HPJiIhYD6zPSVudtfwL4Px8t03SDwCXn0hlx1NrR9qznJpZSfNI\n5USqo9PPQTCzkuaAAPT3B3s60u5yamYlzQEB2H+0i56+YK67nJpZCXNAIHP/APC0FWZW0hwQyPQw\nAnwPwcxKmgMCL7UQPCjNzEqZAwKZUcqTKsuZOqmy2FUxMysaBwRe6nKamXXDzKw0OSCQmenUg9LM\nrNQ5IJC0ENzl1MxKXMkHhO7efvYd6fKgNDMreSUfEPYeThMBc93l1MxKXMkHhJQHpZmZAQ4Ig09K\ncwvBzEpdyQeEgWcpu4VgZqXOAaG9k6mTKplSndejIczMJqySDwjucmpmllHyAaG13c9BMDMDBwS3\nEMzMEiUdEDq7+zh0vMctBDMzSjwgtLrLqZnZoLwCgqSrJD0raZukW4fInyrpfklPStoiaWWSfoGk\n5qy/w5I+nuR9SlJLVt47C7tro0u5y6mZ2aBR+1pKKgfuAK4AdgOPSVoXEVuzin0U2BoR75I0C3hW\n0t0R8SywLOt9WoB7s7a7PSI+U6B9OWGDLQQHBDOzvFoIy4FtEbEjIrqBe4AVOWUCqFPmgQK1wEGg\nN6fM5cD2iHhhjHUumIEWwpyp1UWuiZlZ8eUTEOYBu7LWdydp2T4PLAFagU3AxyKiP6fMdcA3ctJu\nkfSUpDWSpg314ZJulLRB0oa2trY8qpu/VEcns+qqqa4oL+j7mpmdiQp1U/kdQDMwl8wlos9Lqh/I\nlFQFvBv4dtY2XwDOS8qngM8O9cYRcWdENEVE06xZswpU3YyW9k7musupmRmQX0BoARZkrc9P0rKt\nBL4XGduA54HFWflXA49HxN6BhIjYGxF9SUvii2QuTZ1SqY60byibmSXyCQiPAYsknZv80r8OWJdT\n5kUy9wiQNAe4ANiRlf8+ci4XSWrMWr0W2HxiVR+biCDVnnmWspmZ5dHLKCJ6Jd0MPACUA2siYouk\nm5L81cDfAF+VtAkQ8McRsR9A0hQyPZQ+kvPWn5a0jMwN6Z1D5I+rw+lejnX3uYeRmVkiryk+I2I9\nsD4nbXXWcitw5TDbHgNmDJF+/QnVtMBa2wcGpTkgmJlBCY9UHngwji8ZmZlllGxAGHgwji8ZmZll\nlGxASHV0UlEmZtV5UJqZGZRyQGhPM6e+hvIyFbsqZmanhZINCC3tnZ7l1MwsS8kGBA9KMzN7uZIM\nCP39wZ6OtHsYmZllKcmAcOBYN919/e5hZGaWpSQDQqrDg9LMzHKVZEAYGKXc6JlOzcwGlWhASAal\nuYVgZjaoJANCqqOT6ooypk2uLHZVzMxOGyUZEFo70sxtmETmiZ9mZgYlGhBSHpRmZvYKJRkQWts9\nKM3MLFfJBYTevn72HUn7WcpmZjlKLiDsPdJFf0CjexiZmb1MyQWElJ+UZmY2pJILCC0DAcGXjMzM\nXqbkAkKqIzMozZeMzMxervQCQnsndTUV1FZXFLsqZmanlbwCgqSrJD0raZukW4fInyrpfklPStoi\naWVW3k5JmyQ1S9qQlT5d0oOSnktepxVml0bW2pH2LKdmZkMYNSBIKgfuAK4GlgLvk7Q0p9hHga0R\ncTHwFuCzkqqy8t8aEcsioikr7VbgoYhYBDyUrI+7VIcHpZmZDSWfFsJyYFtE7IiIbuAeYEVOmQDq\nlJkLohY4CPSO8r4rgLXJ8lrgmrxrPQat7WnfPzAzG0I+AWEesCtrfXeSlu3zwBKgFdgEfCwi+pO8\nAH4iaaOkG7O2mRMRqWR5DzBnqA+XdKOkDZI2tLW15VHd4aV7+jh4rNs9jMzMhlCom8rvAJqBucAy\n4POS6pO8yyJiGZlLTh+V9ObcjSMiyASOV4iIOyOiKSKaZs2aNaZKDvYw8j0EM7NXyCcgtAALstbn\nJ2nZVgLfi4xtwPPAYoCIaEle9wH3krkEBbBXUiNA8rrvZHciXwOD0vwsZTOzV8onIDwGLJJ0bnKj\n+DpgXU6ZF4HLASTNAS4AdkiaIqkuSZ8CXAlsTrZZB9yQLN8A3DeWHclHa9JCmOd7CGZmrzBqZ/yI\n6JV0M/AAUA6siYgtkm5K8lcDfwN8VdImQMAfR8R+SecB9ybPHagAvh4RP0re+jbgW5JWAS8A7y3w\nvr3CwKMzz/I9BDOzV8hrdFZErAfW56StzlpuJfPrP3e7HcDFw7znAZJWxamS6uhkZm0V1RXlp/Jj\nzczOCCU1UtnPQTAzG15JBQQPSjMzG15pBQS3EMzMhlUyAeFwuocjXb1uIZiZDaNkAkKq3YPSzMxG\nUjIBobVj4ElpbiGYmQ2lZALCQAvBj840MxtayQSE1vZOysvE7Dq3EMzMhlI6AaGjkzl11ZSXqdhV\nMTM7LZVMQEj5OQhmZiMqnYDQ0Umj5zAyMxtWSQSEiKC1I+1ZTs3MRlASAeHAsW66e/vdQjAzG0FJ\nBITBQWluIZiZDaskAsLgoDSPUjYzG1ZJBAQ/OtPMbHSlERA60lRVlDFjSlWxq2JmdtoqiYBw7swp\nXLNsLsmjPM3MbAh5PULzTHfd8rO5bvnZxa6GmdlprSRaCGZmNjoHBDMzA/IMCJKukvSspG2Sbh0i\nf6qk+yWeZFz1AAAGCklEQVQ9KWmLpJVJ+gJJ/yFpa5L+saxtPiWpRVJz8vfOwu2WmZmdqFHvIUgq\nB+4ArgB2A49JWhcRW7OKfRTYGhHvkjQLeFbS3UAv8ImIeFxSHbBR0oNZ294eEZ8p6B6ZmdlJyaeF\nsBzYFhE7IqIbuAdYkVMmgDpluvHUAgeB3ohIRcTjABFxBHgamFew2puZWcHkExDmAbuy1nfzypP6\n54ElQCuwCfhYRPRnF5C0EHgd8Mus5FskPSVpjaRpQ324pBslbZC0oa2tLY/qmpnZySjUTeV3AM3A\nXGAZ8HlJ9QOZkmqB7wIfj4jDSfIXgPOS8ings0O9cUTcGRFNEdE0a9asAlXXzMxy5RMQWoAFWevz\nk7RsK4HvRcY24HlgMYCkSjLB4O6I+N7ABhGxNyL6kpbEF8lcmjIzsyLJZ2DaY8AiSeeSCQTXAe/P\nKfMicDnwc0lzgAuAHck9hS8DT0fEP2dvIKkxIlLJ6rXA5tEqsnHjxv2SXsijzkOZCew/yW1PBddv\nbFy/sXH9xu50ruM5+RRSRIxeKNMl9F+AcmBNRPydpJsAImK1pLnAV4FGQMBtEfF/JF0G/JzMfYWB\newqfjIj1ku4ic7kogJ3AR7ICRMFJ2hARTeP1/mPl+o2N6zc2rt/YnQl1HE1eU1dExHpgfU7a6qzl\nVuDKIbb7TzIBYqj3vP6EampmZuPKI5XNzAworYBwZ7ErMArXb2xcv7Fx/cbuTKjjiPK6h2BmZhNf\nKbUQzMxsBA4IZmYGTMCAkMfMrJL0r0n+U5IuOYV1G3b216wyb5HUkTUL7F+cqvoln79T0qbkszcM\nkV/M43dB1nFplnRY0sdzypzS45dMu7JP0uastOmSHpT0XPI63LQsI35Xx7F+/yTpmeTf715JDcNs\nO+J3YRzrl9dMyEU8ft/MqttOSc3DbDvux6/gImLC/JEZJ7GdzJQYVcCTwNKcMu8EfkimO+wbgV+e\nwvo1Apcky3XAr4ao31uAHxTxGO4EZo6QX7TjN8S/9R7gnGIeP+DNwCXA5qy0TwO3Jsu3Av84TP1H\n/K6OY/2uBCqS5X8cqn75fBfGsX6fAv4wj3//ohy/nPzPAn9RrONX6L+J1kLIZ2bWFcDXIuMRoEFS\n46moXEyM2V+LdvxyXA5sj4iTHbleEBHxMJnZfbOtANYmy2uBa4bYNJ/v6rjULyJ+HBG9yeojZKaj\nKYphjl8+inb8BiQzMbwX+EahP7dYJlpAyGdm1nzKjLthZn8d8KakOf9DSRee0oplRo7/RNJGSTcO\nkX9aHD8yU6gM9x+xmMcPYE68NOp+DzBniDKny3H8PTItvqGM9l0YT6PNhHw6HL9fB/ZGxHPD5Bfz\n+J2UiRYQzggaevbXAY8DZ0fEa4H/BXz/FFfvsohYBlwNfFTSm0/x549KUhXwbuDbQ2QX+/i9TGSu\nHZyWfbsl/SmZh1jdPUyRYn0X8poJ+TTwPkZuHZz2/5dyTbSAkM/MrPmUGTcaZvbXARFxOCKOJsvr\ngUpJM09V/SKiJXndB9zLK2ehLerxS1wNPB4Re3Mzin38EnsHLqMlr/uGKFPs7+HvAr8FfCAJWq+Q\nx3dhXER+MyEX+/hVAO8BvjlcmWIdv7GYaAFhcGbW5FfkdcC6nDLrgA8mvWXeCHTEOE6qly255jjk\n7K9ZZc5KyiFpOZl/owOnqH5TlHnUKZKmkLn5mDsLbdGOX5Zhf5kV8/hlWQfckCzfANw3RJl8vqvj\nQtJVwB8B746I48OUyee7MF71y74nNdxMyEU7fom3A89ExO6hMot5/Mak2He1C/1HphfMr8j0QPjT\nJO0m4KZkWWSeEb2dzCysTaewbpeRuXzwFJkHCjUn9c2u383AFjK9Jh4B3nQK63de8rlPJnU4rY5f\n8vlTyJzgp2alFe34kQlMKaCHzHXsVcAM4CHgOeAnwPSk7Fxg/Ujf1VNUv21krr8PfAdX59ZvuO/C\nKarfXcl36ykyJ/nG0+n4JelfHfjOZZU95cev0H+eusLMzICJd8nIzMxOkgOCmZkBDghmZpZwQDAz\nM8ABwczMEg4IZmYGOCCYmVni/wORoioh1wf08AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112bfc780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(len(acces)), acces)\n",
    "plt.title('train acc')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1136d2860>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcnHWV6P/P6ep9rU66O0svWTskIaST0GIEAgrRAQZF\nR/0JCopXxAgIqPwcXK46rxnv3HG8XkdFIpuigogIihqHmXFECEtISDoJIWTpzlLdWbo6ve/buX9U\nVadSqU5Xd9de5/169StVz1LPt59Un/rW9znP+YqqYowxJnWkxboBxhhjossCvzHGpBgL/MYYk2Is\n8BtjTIqxwG+MMSnGAr8xxqQYC/zGTJGIPC8it8S6HcZMlgV+k1RE5LCIrA/D69wsIpvD0SZj4o0F\nfmOMSTEW+E3SEJGfA1XA70WkW0S+5F2+VkReFpF2EdkpIu/02+dmEWkQkS4ROSQiHxORZcBG4B3e\n12kP4dhpIvI1ETkiIs0i8jMRKfKuyxaRX4jIKW8btorIrPGOH4FTY8wZLPCbpKGqNwFHgfeqar6q\nfltEyoE/Av8EzADuAX4jIqUikgd8H7haVQuAi4E6Vd0LbABe8b6OM4TD3+z9eRewEMgHfuhd9wmg\nCKgEZnpfu2+840/zNBgzIQv8JtndCGxS1U2qOqqq/wlsA67xrh8FVohIjqoeV9U9UzzOx4DvqmqD\nqnYDXwauF5F0YAhPwF+sqiOq+rqqdob5+MaEzAK/SXbzgA97h1javcM2lwJzVLUH+AieHvhxEfmj\niCyd4nHmAkf8nh8B0oFZwM+B54AnROSYiHxbRDLCfHxjQmaB3ySbwHKzLuDnqur0+8lT1f8NoKrP\nqeq7gTnAW8CD47zORI7h+ZDxqQKGgZOqOqSq/6Cqy/EM51wLfHyC4xsTMRb4TbI5iWeM3ecXwHtF\n5G9ExOG90PpOEakQkVkicp13rH0A6MYz9OJ7nQoRyQzxuL8EPi8iC0QkH/hfwK9UdVhE3iUiF4iI\nA+jEM/QzOsHxjYkYC/wm2fwz8DXvsM49quoCrgO+ArjxfAP4//G899OAL+DprbcClwOf9b7OfwN7\ngBMi0hLCcR/BM6TzAnAI6Ac+5103G3gKT9DfC/zVu+25jm9MxIhNxGKMManFevzGGJNiLPAbY0yK\nscBvjDEpJqTALyJXicg+ETkoIvcGWf8xEdklIru9t8bXeJdXishfRORNEdkjIneF+xcwxhgzORNe\n3PWmoO0H3g00AluBG1T1Tb9tLgb2qmqbiFwNfFNV3y4ic/DcKLNdRAqA14H3++8bTElJic6fP386\nv5cxxqSU119/vUVVS0PZNj2EbS4CDqpqA4CIPIEnPW4seKvqy37bvwpUeJcfB457H3eJyF6g3H/f\nYObPn8+2bdtCab8xxhhARI5MvJVHKEM95Xhyn30avcvG8yngT0EaNR9YDWwJtpOI3Coi20Rkm9vt\nDqFZxhhjpiKsF3dF5F14Av/fByzPB34D3O1XnOoMqvqAqtaqam1paUjfVowxxkxBKEM9TXjKyfpU\neJedQURWAg/hKTF7ym95Bp6g/5iqPj295hpjjJmuUHr8W4Fqbw2STOB64Fn/DUSkCngauElV9/st\nF+BhPBd+vxu+ZhtjjJmqCXv83iJTd+ApK+sAHlHVPSKywbt+I/B1PPXGf+SJ9Qyrai1wCXATsFtE\nfBNMfEVVN4X/VzHGGBOKuKzVU1tbq5bVY4wxoROR170d7gnZnbvGGJNikibwDw6Pcv/z9byw31JB\njTHmXJIm8Gc4hAdeqOcPu47FuinGGBPXkibwiwg1lU52ujpi3RRjjIlrSRP4AWoqnBxo7qJnYDjW\nTTHGmLiVXIG/sohRhTearNdvjDHjSarAv7LCCcDOxvYYt8QYY+JXUgX+kvwsKopzbJzfGGPOIakC\nP+C5wGs9fmOMGVfyBf6KIhrb+mjpHoh1U4wxJi4lYeD3jPPvsl6/McYElXSBf0V5EWkCdTbOb4wx\nQSVd4M/LSqe6rMB6/MYYM46kC/zgyeff6WonHiuPGmNMrCVp4HfS1juEq7Uv1k0xxpi4k5yB33uB\nt86Ge4wx5ixJGfjPm11AVnoau1wW+I0xJlBSBv4MRxrnzy20G7mMMSaIkAK/iFwlIvtE5KCI3Btk\n/cdEZJeI7BaRl0WkJtR9I6Wm0snupg6GR0ajdUhjjEkIEwZ+EXEA9wFXA8uBG0RkecBmh4DLVfUC\n4B+BByaxb0SsqnTSPzTKgebuaBzOGGMSRig9/ouAg6raoKqDwBPAdf4bqOrLqtrmffoqUBHqvpEy\nVqnTxvmNMeYMoQT+csDl97zRu2w8nwL+NNl9ReRWEdkmItvc7unPmzt/Zi6F2ek2zm+MMQHCenFX\nRN6FJ/D//WT3VdUHVLVWVWtLS0vD0RZqKp1WusEYYwKEEvibgEq/5xXeZWcQkZXAQ8B1qnpqMvtG\nSk2Fk/0nu+gbHInWIY0xJu6FEvi3AtUiskBEMoHrgWf9NxCRKuBp4CZV3T+ZfSOpptLJyKiy55j1\n+o0xxmfCwK+qw8AdwHPAXuBJVd0jIhtEZIN3s68DM4EfiUidiGw7174R+D2CqqkoAqDOLvAaY8yY\n9FA2UtVNwKaAZRv9Ht8C3BLqvtFSVpjN3KJsdjVaj98YY3yS8s5dfysrbCpGY4zxl/SBv6bSyZFT\nvbT1DMa6KcYYExdSIPB7xvmt12+MMR5JH/gvKC9CBBvnN8YYr6QP/AXZGSwqzbfSDcYY45X0gR88\nN3LtbLSpGI0xBlIk8K+qLKKle5CmdpuK0RhjUiLw+yp12ji/McakSOBfOqeATEeajfMbYwwpEviz\n0h0sm1topRuMMYYUCfzgqdvzRlMHI6N2gdcYk9pSKPA76Rkcod5tUzEaY1Jb6gT+Ss8FXhvuMcak\nupQJ/AtL8ijISrcLvMaYlJcygT8tTbigoshSOo0xKS9lAj94hnv2Hu+kf8imYjTGpK7UCvwVToZH\nlTePd8a6KcYYEzMhBX4RuUpE9onIQRG5N8j6pSLyiogMiMg9Aes+LyJ7ROQNEfmliGSHq/GTtcp7\ngXeXjfMbY1LYhIFfRBzAfcDVwHLgBhFZHrBZK3An8J2Afcu9y2tVdQXgwDPhekzMLsqmrCCLnTbO\nb4xJYaH0+C8CDqpqg6oOAk8A1/lvoKrNqroVGAqyfzqQIyLpQC5wbJptnpaaSqdl9hhjUloogb8c\ncPk9b/Qum5CqNuH5FnAUOA50qOp/BNtWRG4VkW0iss3tdofy8lOyqtJJQ0sPHX3BPqOMMSb5RfTi\nrogU4/l2sACYC+SJyI3BtlXVB1S1VlVrS0tLI9amlRWeqRh323CPMSZFhRL4m4BKv+cV3mWhWA8c\nUlW3qg4BTwMXT66J4bWy3HOB1+bgNcakqlAC/1agWkQWiEgmnouzz4b4+keBtSKSKyICXAnsnVpT\nw6MoN4OFJXlWusEYk7LSJ9pAVYdF5A7gOTxZOY+o6h4R2eBdv1FEZgPbgEJgVETuBpar6hYReQrY\nDgwDO4AHIvS7hKym0slLB1ti3QxjjImJCQM/gKpuAjYFLNvo9/gEniGgYPt+A/jGNNoYdisrinhm\nRxMnOvqZXRSz2wqMMSYmUurOXR+r1GmMSWUpGfiXzykkPU3sAq8xJiWlZODPznCwdE4BuyzwG2NS\nUEoGfvAUbNvl6mDUpmI0xqSY1A38lU66BoZpaOmJdVOMMSaqUjbw+yp1Wt0eY0yqSdnAv6g0n9xM\nh43zG2NSTsoGfkeacEF5EXVWs8cYk2JSNvCDZ7hn77FOBoZtKkZjTOpI6cC/ssLJ4Mgobx3vinVT\njDEmalI68NdUeko02zi/MSaVpHTgL3fmUJKfSZ3LxvmNMakjpQO/iFBT4bTSDcaYlJLSgR884/z1\n7m66+m0qRmNMakj5wF9TWYQq7G6y4R5jTGqwwF/hu4PXAr8xJjWkfOAvzstk3sxcK91gjEkZKR/4\nwTPObymdxphUEVLgF5GrRGSfiBwUkXuDrF8qIq+IyICI3BOwzikiT4nIWyKyV0TeEa7Gh0tNRRHH\nOvpp7uyPdVOMMSbiJgz8IuIA7gOuBpYDN4jI8oDNWoE7ge8EeYl/A/5dVZcCNcDeabU4AsYqdVrd\nHhNDJzv7OWmdDxMFofT4LwIOqmqDqg4CTwDX+W+gqs2quhU4IydSRIqAy4CHvdsNqmrcjamcP7cI\nR5rYcI+JqS8+uZO7n6iLdTNMCggl8JcDLr/njd5loVgAuIGfiMgOEXlIRPKCbSgit4rINhHZ5na7\nQ3z58MjJdLBkVoFNvm5i6mBzN/tPWt0oE3mRvribDqwB7lfV1UAPcNY1AgBVfUBVa1W1trS0NMLN\nOtuqyiJ2utpRtakYTfT1D41worOfUz2DdNrNhCbCQgn8TUCl3/MK77JQNAKNqrrF+/wpPB8Ecaem\nwkln/zCHT/XGuikmBTW19409PtJi70ETWaEE/q1AtYgsEJFM4Hrg2VBeXFVPAC4ROc+76ErgzSm1\nNMJWem/ksnF+Ewuu1tPB/vApmwfaRFb6RBuo6rCI3AE8BziAR1R1j4hs8K7fKCKzgW1AITAqIncD\ny1W1E/gc8Jj3Q6MB+GSEfpdpWTIrn7xMB7/Z3sR7V84lLU1i3SSTQlxtp3v8h1ss8JvImjDwA6jq\nJmBTwLKNfo9P4BkCCrZvHVA7jTZGRbojjb+/eilf/90efvDfB7lrfXWsm2RSSGNrL5npaThzMjhk\nPX4TYXbnrp+b1s7j71aX870/7+cvbzXHujkmhbjaeqkozmFBSR5H7DpTTPx570keeKE+1s2ICgv8\nfkSEb33gApbNLuSuJ3ZwxHpeJkpcrX1UFueyoCTPhnpi5GevHOE7/7GfoZHRWDcl4izwB8jJdPDj\nmy5ERPjMz1+nb9AmYjeR52rrpXJGDvNm5llKZ4zUu7sZHE6NObgt8AdROSOXf7t+FftOdvHlp3dZ\nbr+JqK7+Idp7h7w9/lzAUjqjrX9oZCylti4FMvss8I/jneeV8YX1S/ht3TEefflwrJtjkpir1RNw\nKmfkMm+m58Z2S+mMrkMtPfj6d3VHLfCntNvftZj1y8r4pz/uZevh1lg3xyQpV5und19ZnMt8X+C3\ncf6oqnd3A1DuzEmJObgt8J9DWprw3Y+sonJGLrc9tt3KNpuI8N28VTkjh5xMB7MLsy2lM8oa3D2I\nwHWr5lLv7k76aywW+CdQmJ3BxhsvpLt/mNse287gcPJf8TfR1djWR0FWOkU5GQDMm5lrKZ1RVu/u\nZm5RDmsXzvTMwZ3kJdot8IfgvNkFfPtDK9l2pI3/tSnuphMwCe5oay8VM3IR8dwtbimd0dfg7mFR\nWf7YHNzJXqnXAn+I3lszl1suXcBPXz7MMzsaY90ck0Rcrb1UFueMPbeUzuhSVRrc3SwsyaMoN4OF\nJXkW+M1p9169lLcvmMGXn97Nm8c6Y90ckwRUlca2Pipn5I4ts5TO6DrZOUDP4AiLyvIBqKl0Upfk\nJdot8E9CuiONH350Dc6cTD7zi2209w7GukkmwbV0D9I3NEKVX+C3lM7o8mX0LCr1nPdVlU7cXQMc\n70jeZA4L/JNUWpDFj25cw4mOfu7+VR2jo8nbKzCRN5bKOeP0UI+ldEbX6cB/uscPsDOJh3ss8E/B\nmqpivvHe83l+n5vv/flArJtjEthYKmfx6R6/L6XTJgWKjgZ3D/lZ6ZQVZAGwbE4BmY60pB7nt8A/\nRR97exUfvrCC7//5AH/eezLWzTEJqtFbh7/CL/CDJ6XThnqio97dzcLSvLGsqqx0B8vmFlrgN2cT\nEf7x/StYUV7I3b+qs6/lZkpcrb2U5GeRk+k4Y7mldEZPg7tnbJjHZ1VFEbubOhhJ0qFcC/zTkJ3h\n4P6PXYgjzVPJs3dwONZNMgnGV5UzkKV0Rkfv4DBN7X0sLMk7Y/mqKie9gyMcaE7OSp0hBX4RuUpE\n9onIQRG5N8j6pSLyiogMiMg9QdY7RGSHiPwhHI2OJ5UzcvnBDas50NzFvb/ZndQpYCb8fHX4A1lK\nZ3Qc8n6r8qVy+ozdyJWkBdsmDPwi4gDuA64GlgM3iMjygM1agTuB74zzMncBSXvL67rqUr74nvN4\nducxntjqinVzTIIYGVWOtfeN2+MHS+mMtHq3N/AHDPUsKMmjMDs9aQu2hdLjvwg4qKoNqjoIPAFc\n57+Bqjar6lbgrO+lIlIB/C3wUBjaG7due+cils0p5JntTbFuikkQxzv6GB7VoD1+S+mMjvrmbkQ8\nF9P9iYj3Rq7krNkTSuAvB/y7sY3eZaH6HvAlIKmrm4kI7zqvlO1H2+gesLF+MzH/OvyBLKUzOhpa\neqgsziU7w3HWutWVTvad6EzKa3cRvbgrItcCzar6egjb3ioi20Rkm9vtjmSzIubS6hKGR5VX60/F\nuikmAfjX4Q/GUjojr77Zk8oZTE2lk1GFN5qSrzxLKIG/Caj0e17hXRaKS4D3ichhPENEV4jIL4Jt\nqKoPqGqtqtaWlpaG+PLx5cJ5xeRkOHjxQGJ+cJnocrX2kiYwx5kddL2ldEbW6KhyqOXsVE4f3x28\nda62aDYrKkIJ/FuBahFZICKZwPXAs6G8uKp+WVUrVHW+d7//VtUbp9zaOJeV7mDtwhm8eKAl1k0x\nCcDV2sucohwyHMH/DC2lM7KOd/bTNzQybo+/JD+LiuIcdibhOP+EgV9Vh4E7gOfwZOY8qap7RGSD\niGwAEJHZItIIfAH4mog0ikhhJBsery6tLqWhpYfGNhubNefmague0eNjKZ2RVd98Zo2eYHyVOpNN\nSGP8qrpJVZeo6iJV/ZZ32UZV3eh9fMLbsy9UVaf3cWfAazyvqteG/1eIL5dVlwCw2Xr9ZgKu1t4z\nqnIGspTOyGpwTxz4V1c6aWrvo7kruSp12p27Yba4LJ/Zhdk23GPOqX9ohOaugXEv7IKldEZavbuH\ngux0SvIzx93mdKXO5BruscAfZiLCpdUlvFTfkrR1Psz0+YqzBUvl9LGUzshqaOlmUWn+WHG2YFbM\nLcKRJklXotkCfwSsqy6hvXeIN5qSq5dgwidYHf5gLKUzcuqbe8a9sOuTk+ngvFkFSXcHrwX+CLhk\nsXec/6AN95jgGoPU4Q9mQUkeRyzwh133wDAnOvvPOb7v47vAm0yTLlngj4CS/CzOn1vIC/stn98E\n52rrIys9jVLv5B/jmTczj5buQbospTOsDo1ToyeY1ZVOuvqHOZREH8AW+CNkXbWnfEOPlW8wQbha\ne6kozjnn+DKcTuk8bCmdYRU4z+65jN3IlUSVOi3wR8i66hKGRpQth6x8gzmbpw7/uYd5wFI6I6XB\n3Y0jTaiaOfH/weKyfPIyHUk1zm+BP0IunFdMdkYaL+y3cX5ztvHq8AeylM7IqHf3UFmcQ1b62cXZ\nAjnShAsqipLqRi4L/BGSneHgogUz7QKvOUtH3xAdfUMTZvSApXRGSr27O6TxfZ9VlcXsPd5J/9BI\nBFsVPRb4I+iy6hIONndzrL0v1k0xccQVYkaPj6V0hpevONtEqZz+VlUWMTSivHk8OSp1WuCPoHXV\nniqjVr7B+Gscy+EPLfBbSmd4NbX3MTA8OukeP5A0N3JZ4I+gJbPyKSvI4kUb7jF+xiZgCbnHbymd\n4TSW0VMWeuCfXZTNrMKspBnnt8AfQb7yDZsPuJPq5g8zPa62Xgqy0ynKzQhpe0vpDC/fPLsLS0If\n6gHPBOzW4zchWVddQlvvEHuOJcfYoJm+iapyBrKUzvBqcHfjzM1gRt74xdmCWVXl5PCpXtp7ByPU\nsuixwB9hvvINLx60u3iNh6sttFROH0vpDK96dzcLS/ImvHku0KoK34xcid/rt8AfYWUF2SybU8iL\nls9vAFWlsa03pFROH0vpDK8G9/jTLZ7LBRVFiCRHiWYL/FGwrrqE14+00Tto5RtSnbt7gP6h0ZAz\nenwspTM8OvuHaO4aYOEUAn9BdgaLS/OTYg5eC/xRsK66hMGRUbYcao11U8w59A4OR/wGnclm9PhY\nSmd4NIwVZ5vchV2fVZVOdjZ2oJrYyRohBX4RuUpE9onIQRG5N8j6pSLyiogMiMg9fssrReQvIvKm\niOwRkbvC2fhE8bb5M8hMT7Phnjh3y6Pb+Pyv6iJ6jMYQ6/AHspTO8GiYQiqnv5pKJ609g2Mf4Ilq\nwsAvIg7gPuBqYDlwg4gsD9isFbgT+E7A8mHgi6q6HFgL3B5k36SXneHg7QtmsNku8Mat/qERth5u\n5aWDLRFNvfXdtVsx6R6/d+J1G+eflnp3N+lpMqmsKn+rfJU6E7xgWyg9/ouAg6raoKqDwBPAdf4b\nqGqzqm4FhgKWH1fV7d7HXcBeoDwsLU8w66pL2H+ymxMdyTVpc7J483gnQyNKZ/8wDRHMnnG19lFa\nkEV2xsTFwfz5UjoPWWbPtDS4e6iamUuGY2qj3OfNLiArPS3hSzSH8tuXAy6/541MIXiLyHxgNbBl\nnPW3isg2Ednmdidfz/jSxd7yDXYXb1zy/0PefjRyF+9cbb1UFk9umAcspTNcPKmcUxvmAchwpLGi\nvCjhSzRH5eKuiOQDvwHuVtWgdzKp6gOqWquqtaWlpdFoVlQtnV1ASX4WLx5Ivg+1ZLCzsZ2ygiwK\nstPZEcHeXKh1+ANZSuf0jYwqh1t6WVQ2tQu7PqsqnbzR1MHQyGiYWhZ9oQT+JqDS73mFd1lIRCQD\nT9B/TFWfnlzzkkdamnDp4plsPhDZMWQzNXWudlZXOVlV6WRHhHr8wyOjHGvvn3RGj4+ldE5PY1sv\ngyOjLJpGjx88F3gHhkfZd6IrTC2LvlAC/1agWkQWiEgmcD3wbCgvLp5b4x4G9qrqd6fezOSwrrqU\nUz2D7D1h5RviSWvPIEdO9bKqspg1VcXsP9lFdwSmzDze0c/IqE46o8fHUjqn53Rxtun1+FdXJv4d\nvBMGflUdBu4AnsNzcfZJVd0jIhtEZAOAiMwWkUbgC8DXRKRRRAqBS4CbgCtEpM77c03Efps4t67a\nW77ByjTHFV/hrVWVTlZXORlV2BWBP+rJ1uEPZCmd09MwVpxtej3+iuIcZuRlJnTgTw9lI1XdBGwK\nWLbR7/EJPENAgTYDkyuIkcTKCrM5b1YBLx5ws+HyRbFujvGqc7Uj4rklf2TEMwy3w9XOxd46S+Hi\nmmQd/kD+KZ0ryovC1q5UUe/uZkZeJsWTLM4WSEQ8N3IlcOC3O3ejbF11CVsPt9E3mBxTuCWDOlc7\nS8oKyM/ylEpeVJoXkXF+V2sfjjRhTlH2lPa3lM7pqXf3TPmO3UA1FU4OursT9tuXBf4oW7eklMHh\nUV47bOUb4oGqsrOxfezGHIDVVcVsP9oe9tvyXW29zHVmkz7FHPJ5M311+S3wT0XDNFM5/dVUFqEK\nuxsTs2CbBf4ou2j+DDIdaWy2tM644KmvPsSqKv/A77kt/2hreFMnXa29Ux7fB8jNTGdWYZaldE5B\nR+8QLd2D076w6+PrKOxI0OEeC/xRlpPp4G0Liu0Cb5zwjdPWVPgFfu/8quHO559sHf5g5s/Ms5TO\nKahv8WT0hKvH78zNZP7M3IQd57fAHwOXLi7lrRNdNHda+YZYq3O1k5PhYMms0wHhvNkF5GY6wjrO\n3z80grtrYMqpnD6W0jk19c3TK84WjKdSpwV+EyJfWqeVb4i9Ha52LqgoOmPc3ZEm1FQ42R7GHn/j\nNDN6fCylc2oaWnrIcMiUymWMp6bSycnOAY53JF6lTgv8MbB8TiEz8zJtuCfGBoZH2Husc+yGHH+r\nq5zsPd4ZtuwrXxnfyVblDGRVOqemvrmbeTPzpnxhPRjfOH8iDvdY4I+BtDThksUlvHigJeEndEhk\ne493MTgySk2QwL+mqpjhUeWNY+HJ2nBNsQ5/IEvpnJqGlvClcvosm1NIhkMS8gKvBf4YWVddQkv3\nAG8lcL2PRFfnHcNfFSTw+7J8wjXO72rtJTsjjdL8rGm9ji+l08b5Qzc0MsqRUz1Tmm7xXLIzHCyb\nU2g9fhO6ddWeCqRWrTN26lyeipzBbqgqyc+iakYu24+E54/6aGsvFcW5eMpXTZ0vpfNQiw31hMrV\n2svQiE5pgvWJrKp0sruxg5EEK7xogT9GZhdlU12Wb+P8MVTn8ty4NV4wXl3lZPvRtrAMx7la+8J2\nYdFSOidnrEZPmId6wJMG3DM4wkFv1lCisMAfQ+uqS3ntUGvEJ/g2Z2vvHeTwqd6g4/s+a6qKae4a\n4HgYZk2bah3+YCylc3LGqnKGKYffn29IsM4Vucl7IsECfwytqy5hYHiUrVa+Iep8lRWDZfT4rPb+\nUU93Rq6O3iG6+oenffOWj6V0Tk6Du4eS/CyKcjPC/toLZuZRkJ1OnSuxSjdY4I+hty+cQYZD2GzD\nPVHnX5FzPEtnF5KVnjbtO3jDldHjYymdk1Pv7o7IMA94MvRWVToTrkSzBf4Yys1M58J5xbxggT/q\ndrraqS7LpyB7/F5gZnoaKyuKpp3ZM1aHP0xDPZbSOTmeVM7wD/P41FQ42X+yi97B8E/eEykW+GNs\nXXUpe4934u4aiHVTUoaqUudqP6M+z3hWVxXzxrFOBoanfh1munX4A1lKZ+haewZp7RkMew6/v1WV\nTkZGlT3HEmdmPQv8MXaZN63zJSvfEDVHW3tpC6jIOZ7VlU4Gh0d5cxp/1K7WPopyMig8x7eLybCU\nztA1+C7sRrDHv7LSM1xYF+aifpFkgT/Gzp9bSHFuBi9YPn/U1PlNtTiR1VXTr9TpyegJX40YsJTO\nUPlSOSMZ+MsKsil35lCXQAXbQgr8InKViOwTkYMicm+Q9UtF5BURGRCReyazb6rzlW/YbOUboqbO\n1U52RhrnzSqYcNvZRdnMLcqe1m35063DH4yldIam3t1NZnoa5WEszhbMqkpncvX4RcQB3AdcDSwH\nbhCR5QGbtQJ3At+Zwr4pb111Cc1dA+w/mVg3gSSqOlc7F5QXhVywa3VVMduPTO0Cr6rS2NYXtvF9\nH0vpDE29u4cFM/NwpEV26u+ayiKa2vsS5lpdKO/8i4CDqtqgqoPAE8B1/huoarOqbgUC34UT7mvg\nUivfEDUXhw7fAAAZjUlEQVSDw6PsOdYZ0jCPz+oqJ03tfVOaP8HdNcDA8GhYywGDpXSGqiGCqZz+\nVnkn70mUuj2hBP5ywOX3vNG7LBQh7ysit4rINhHZ5nanVgAsd+awqDTPyjdEwd7jnQwOj479oYZi\nbJx/Cn/Uvoyeigj0+MFSOs9lcHiUI629ER3f91lRXogjTRJmYpa4ubirqg+oaq2q1paWlsa6OVG3\nrrqULYdOWfmGCPP9YYaS0eNz/lxv+d0pjOH66vCHe4zfUjondrS1l5FRjUqPPzcznSWzChLmRq5Q\nAn8TUOn3vMK7LBTT2TelrKsuoX9odMpjySY0dUfbKcnPYm6Qipzjyc5wsHxu0ZRKN/gmbK8I81CP\npXROrD4KqZz+3ja/mK2HW2ntGYzK8aYjlMC/FagWkQUikglcDzwb4utPZ9+UsnbhTDIcwiMvHUqo\nOwATzUQVOcezpsrJrsZ2hkdGJ7Wfq7WXsoIssjMck9ovFPNnWmbPuUSyKmcwN62dR//QKI++fDgq\nx5uOCQO/qg4DdwDPAXuBJ1V1j4hsEJENACIyW0QagS8AXxORRhEpHG/fSP0yiSwvK50vvPs8/vxW\nM9d+fzNvNCVW0adE0NE7RENLz1jxtclYXVVM/9DopCfOCWdVzkALSiyX/1zq3d2UFWSdsyxHOFXP\nKmD9sjIefeVw3HfeQhrjV9VNqrpEVRep6re8yzaq6kbv4xOqWqGqharq9D7uHG9fE9xn37mIx29Z\nS+/gCB/40Us8+EIDowk2wUM8Gxvfn0RGj4+viudk6/aEsw5/IEvpPLcGd3fUhnl8Nly+iPbeIX61\n1TXxxjEUNxd3jcc7Fs3kT3et413nlfGtTXv5xE9eo7lr+vXgTWgVOcdTUZxDSX7WpC7wDo2Mcrwj\n/Dn8PpbSOT5Vpd7dE7VhHp/a+TOonVfMQy8eYmiSw4LRZIE/DhXnZfLjmy7kWx9YwdbDrVz9vRf5\ny1vNsW5WwqtztbOoNH9KNXNEhDVVzkmldB5v72dUw1ecLZCldI7vVM8gHX1DUe/xg6fX39Texx93\nHY/6sUNlgT9OiQgfe/s8fn/HpZQWZPHJn27lH36/x9I9p8hXkXMqwzw+q6uKOdTSQ1uIWRtjVTnD\nnMrpYymd44v2hV1/Vywto7osn41/rY/bMiwW+ONc9awCfnv7Jdx88Xx+8tJhPvCjlznYPLkLjAYa\n2/po7RmcZuD3jvOHOM3e6Tr8kRnjt5TO8UU7ldNfWprwmcsX8daJLp7fH583o1rgTwDZGQ6++b7z\neeTmWpo7+7n2B5t5fMvRuO1NxKMdk6jIOZ6VFUU40kK/kcvV1kt6mjCnKHIFwiylM7gGdzdZ6WmU\nOyNbnG0876uZy5yibDY+Xx+T40/EAn8CuWLpLP501zreNn8GX3lmN5/9xXbae+P/ZpF4UHe0naz0\nNM6bPXFFzvHkZqazdHZB6IG/tY+5zpyIFgizlM7g6t09LCjJIy3CxdnGk5mexqcuXcCWQ63TnsEt\nEizwJ5iywmwe/eRFfPWaZfz5rZNc9b0XeaX+1KRfp7N/iO1H23hym4t//tNebnl0K1d853k+8/Nt\nSXkPQZ2rjQvKi8gIsSLneFZXeeZXHQkhzTYSdfgDWUpncA3ubhaVRX+Yx9/1F1VRmJ3Oxr/GX68/\nPdYNMJOXliZ8+rKFrF04kzuf2MFHH3qV29+5mLvWV58R2FSV5q4BDjZ3j/3Uuz3/NvuVj810pDG/\nJJdFZfm8Un+K5/ac5N3LZ3HXldWsKJ986mO8GRoZ5Y1jnXx87bxpv9bqymJ+8epRDjZ3T/jtwdXa\nx/plZdM+5rn4p3Qmw/9VOAwMj3C0tZf31cyNaTvys9L5+Dvmc9/zB6mPwT0F52KBP4FdUFHEHz53\nKf/w+z388C8Heam+hfcsn+0J8u5uGpq76Ro4fQdhQVY6i8ryuWxJKYvL8llUms/isnwqi3PGatN3\n9g/x05cO89CLDVz7ZnJ8ALx1vIvB4VFqpjG+77Nmnm9GrrZzBv6+wRFaugcilsrp45/Smcj/R+F0\n5FQvo0rMe/wAN18ynwdfbODBFxr43x9cGevmjLHAn+DystL59odquGxJKV9+ejc7jr7FrMIsFpfl\n84E15Swuy2exN8CXFmRNWKOmMDuDO6+s5uZL5p/xAbB+2SzuXp+YHwB13iyc6VzY9Zk/MxdnbgY7\njrZz/UVV4243Vo45wjM/WUrn2Xzz7C4siX3gL8nP4sO1FTy5tZEvvHsJZYWhFweMJAv8SeLalXO5\nYmkZw6Malkm9/T8AHn3pMA++2MC1P0jMD4AdrnZK8jPDEoRFhNWVzgkrdZ5O5Yxsj99SOs9WH8Mc\n/mA+vW4hj285ysMvHeLLVy+LdXMAu7ibVHIz08MS9P0VZmfwuSur2XzvFXzx3Ut47dAprv3BZm55\nNHEuAu+cYkXO8aypKuZAczcdfeNfUB0L/BG6ecufpXSeqd7dzZyibPKy4qNfO29mHldfMIfHXz1K\nZ5xchLfAb0IS+AGw9XCr9wNgK7sb4/cDoKNviHp3DzUV0x/m8fHNyLXrHLMtudr6yMlwUJKfGbbj\njsdSOs8Uixo9E/ns5YvoGhjmsVePxropgAV+M0ljHwB//y7uec8Sth5u470/jN8PgF1TmHFrIisr\nixDhnPn8rtZeKopzwvYt41wspfM0VaWhOb4yaABWlBdx6eISHnnpEAPDsS+7YoHfTElBdgZ3XHH2\nB8Cdv9wRVzeV1XmD88ow9vgLszOoLss/5zi/qy1yVTkDWZXO09zdA3QNDLOwJL56/OAp3ubuGuCZ\n7bGfhNACv5kW/w+AO6+sZtPu41z1vRfZHCcTx+9sbGdRaR5FOeG99rGmqpgdR9uDls1QVRpbe6mK\nUuC3Kp2n1Td7zkE8pHIGumTxTFaUF/LACw0h3QAYSRb4TVgUZGfwhXcv4be3X0J+djo3Prwl5tVE\nfRU5w5G/H2h1lZOOvqGgwbajb4iugeGIp3L6WErnaQ0t3lTOOBvqAU9G2IbLF9HQ0sN/vnkipm0J\nKfCLyFUisk9EDorIvUHWi4h837t+l4is8Vv3eRHZIyJviMgvRSQ+EllNRKwo99xU5qsm+t4fxG4a\nyca2Plq6B8dmzwon3wXe7UHG+V2tfUDkUzl9LKXztPrmHnIyHMyJk3z5QFedP5uqGbnc/9eGmBZZ\nnDDwi4gDuA+4GlgO3CAiywM2uxqo9v7cCtzv3bccuBOoVdUVgAPPhOsmifmqif7sf1xEZ/8QH/jR\nS9z3l4NR/3pbN1aRszjsr724NJ+CrPSgBbgiXYc/GEvp9Gho6WZhaeyKs00k3ZHGpy9byE5XO682\ntMasHaH0+C8CDqpqg6oOAk8A1wVscx3wM/V4FXCKyBzvunQgR0TSgVzgWJjabuLcZUtKee7uy3jP\n8tn863P7+MiPXxnLb4+Gna52MtPTWDpn6hU5x5OWJqyqcgbN7Il0Hf5gLKXTo97dHZfDPP4+fGEF\nJfmZ/PiF2BVvCyXwlwP+Mwc3epdNuI2qNgHfAY4Cx4EOVf2PYAcRkVtFZJuIbHO743PyAjN5ztxM\nfvjR1fzfj9Sw70QXV33vBZ7c5orK19w6Vzsr5hZOuyLneFZXOnnrRCe9g8NnLHe19eLMzaAgzDfT\nnYuldEL/0AiNbX0sirMc/kDZGQ5uvng+z+9zs/d4Z0zaENGLuyJSjOfbwAJgLpAnIjcG21ZVH1DV\nWlWtLS0tjWSzTJSJCB9YXcGf7l7HivIivvTULjb84nVOdQ9MvPMUDY2MsrupIyLDPD6rq4oZVdjp\nOvMahqu1L6rDPGApnQCHT/WgGp8XdgPdtHY+eZkOfhyjks2hBP4moNLveYV3WSjbrAcOqapbVYeA\np4GLp95ck8gqinN5/NNr+co1S/nLW27+JoKTyO870cXA8GhYb9wK5Cv6FjgVYzTq8AeylE6/VM44\n7/EDFOVmcMNFVfx+1/GoDn/6hBL4twLVIrJARDLxXJx9NmCbZ4GPe7N71uIZ0jmOZ4hnrYjkiucW\nxiuBvWFsv0kwjjTh1ssW8bs7LmFmXiaf/OlWvvrM7rOGS6bLd2E3Ehk9PsV5mSwsyTtjnH90VGmM\nQY/fUjrjqypnKD61bgECPLz5UNSPPWHgV9Vh4A7gOTxB+0lV3SMiG0Rkg3ezTUADcBB4ELjNu+8W\n4ClgO7Dbe7wHwv1LmMSzbE4hv7vjEj69bgGPv3aUa7+/eSxYh0Odq50ZeeGpyHkungu8bWPXLJq7\nBhgcGaUiSqmcPqme0tnY1svvdh6jckYOOZmOWDcnJHOKcrhuVTlPbD1Ka09073YPaYxfVTep6hJV\nXaSq3/Iu26iqG72PVVVv966/QFW3+e37DVVdqqorVPUmVY3cwK5JKNkZDr76t8t57Ja30z80wgfv\nf5kfPX8wLBd+68JckXM8a6qKaekepLHNk7t/OpUz+pN8V5cV8Nf9bg6c7Ir6sWPp5foW3vfDlzjZ\n0c8/vf+CWDdnUjZcvpD+oVF+9srhqB7X7tw1MXfxohL+dPdlXLViNt/+93188cmd0ypk1dk/RL27\nOywTr0xktfcagq9uT7Tq8AfzlWuWIQIf2vgKWw/HLkc8WlSVhzcf4qaHX2NGXia/u+MSLl+SWIkh\n1bMKWL+sjEdfPhz24c5zscBv4kJRTgY/vGE1X3z3Ep7e0cRHH9xCyxSzfnY3dqAanhm3JnLerAJy\nMhxj4/y+u3bLndHv8S+fW8jTn72YmXmZ3PjQFv79jdiWBYik/qERvvDkTv7xD29y5dIynrnt4oTI\n5glmw+WLaOsd4smtrok3DhML/CZuiAifu7Ka+z66hj3HOnj/fS+x78Tkhy181wrCWYN/POmONFZW\nFI3dwetq62VWYRbZGbEZZ66ckctTn72Y5XML+exjr/PzVw7HpB2R1NTex4c2vswzO5r4/PolbLzx\nwqjeMxFutfNnUDuvmAdfPMTQyGhUjmmB38Sdv105hyc/8w4Gh0f54P0vTzrlc8fRdhaW5FGUG51g\nsGZeMXuOddI/NIIrilU5xzMjL5PHb1nLFeeV8T9/t4d/fe6tmNaFCadX6k/x3h9s5khLLw99vJa7\n1lfHbXmGydhw+SKa2vv4467jUTmeBX4Tl1ZWOPndHZcwb2Yun3p0Kw9vPhRS8PJV5IzGMI/P6kon\nw6PKnmMdNLZFP5UzmJxMBz++6UKuf1sl9/2lni89tStqvclIUFV+8tIhbnx4C8W5Gfz2jktYv3xW\nrJsVNlcsLaO6LJ+Nf62Pyod0fExKaUwQc4py+PWGd/D5X9Xxj394k3p3N//wvvPPWYLhWEc/Ld0D\nEb1xK5CvUudrh9o43tEX9VTO8aQ70vjnv7uA2UXZfO+/DuDuHuC+j66Jm7loQ9U/NMJXntnN09ub\nWL9sFv/3IzUJPbQTTFqacNf6at481snA8GjEhwqtx2/iWm5mOvd/7EJue+ciHt9ylJt/8hodvePX\no/HNuBWN8X2f0oIsKmfk8MfdxxjV2KRyjkdEuHv9Ev757y7ghf1ubnjw1SlfNI+FY+19fHjjKzy9\nvYm711fzwE2JPZ5/LteunMuXrloaletDFvhN3EtLE7501VL+z4dreO1QKx/40Uvjliaoc7WRmZ7G\nsjmFUW3j6spi3mjyFNyKRSrnRG64qIoHbqpl/8kuPnT/ywlxh++rDZ7x/EMtPTz48VruXr8kKcbz\n44EFfpMwPnhhBY9/ei3tfUO8/76XeLn+7Okdd7o6OH9uIZnp0X1rr/YbWorHwA+wfvksHrtlLR19\nQ3zw/pfZ3RibCXImoqr89KVD3PjQFopyM/jt7Zfw7iQaz48HFvhNQnnb/Bn89rZLKCvI4uMPv8Yv\nXzs6tm54rCJn9IZ5fHzj/BkOYXaczv4EcOG8Yp767MVkpTv4yAOv8Nf94SuBfqp7gIPNXZzs7Kdn\nYHhKFyn7h0a459e7+Obv3+TyJaX89vZLWByH8+cmusS6ymMMUDUzl9/cdjGfe3wHX356Nwebu/nK\nNcvYd7KLvqGRmAT+5XM83zLmFGXjiPPhiEWl+Txz28V84idb+dRPt/IvH1zJBy+smNRrdA8M80ZT\nBztd7exq7KDO1U5Te98Z26QJ5GelU5CdQUF2OgXZ6Wc8z89OpzA7w7ssndxMB/c/X8/Oxg7uvLKa\nu69MjlTNeGSB3ySkwuwMHv5ELf/0x708vPkQh1p6WLtwBhCdO3YDZaan8fYFM8hPkIyZssJsnvzM\nWjb84nW++OudnOzq57OXLwpa22hweJS3TnSys9EX6Ns50NyNr0NfUZzDqionn7h4HrMKs+keGKar\nf5ju/mG6+j0Tz/ueu7sHaGjp8a4bZjAgxTTPm4b6N+fPjsZpSFkSjzd21NbW6rZt2ybe0BjgF68e\n4RvP7mFUleLcTF7/2vqIF2cLpndwGEESpjokeIL6Pb/eybM7j/GJd8zjf167nMOnescCfF1jB3uP\ndY4F6Jl5maysKKKm0klNhZOVFUXMzM+a8vEHhkf8PiSGmV2UTWnB1F8vlYnI66paG8q2idE9MeYc\nblw7j/kz87jtsdd5x8KZMQn64Ek9TTSZ6Wl87yOrmFWYxYMvHuKXW10MDnuCfG6mgwvKi7j5kvlj\nQb6iOCes5zcr3UFWvoOSaXx4mMmzHr9JGp39Q6SJJMxwS7z59TYXu5s6WFFexKpKJ4tK8+P+eoU5\nzXr8JiUVJumNPdHy4dpKPlxbOfGGJuFZOqcxxqQYC/zGGJNiQgr8InKViOwTkYMicm+Q9SIi3/eu\n3yUia/zWOUXkKRF5S0T2isg7wvkLGGOMmZwJA7+IOID7gKuB5cANIrI8YLOrgWrvz63A/X7r/g34\nd1VdCtTgmbDdGGNMjITS478IOKiqDao6CDwBXBewzXXAz7yTrr8KOEVkjogUAZcBDwOo6qCqtoex\n/cYYYyYplMBfDvhPBtnoXRbKNgsAN/ATEdkhIg+JSF6wg4jIrSKyTUS2ud3hqx9ijDHmTJG+uJsO\nrAHuV9XVQA9w1jUCAFV9QFVrVbW2tLQ0ws0yxpjUFUrgbwL8k3srvMtC2aYRaFTVLd7lT+H5IDDG\nGBMjodzAtRWoFpEFeIL59cBHA7Z5FrhDRJ4A3g50qOpxABFxich5qroPuBJ4c6IDvv766y0icmQS\nv4e/EuDsQu3xw9o3Pda+6bH2TU88t29eqBtOGPhVdVhE7gCeAxzAI6q6R0Q2eNdvBDYB1wAHgV7g\nk34v8TngMRHJBBoC1o13zCmP9YjItlBvW44Fa9/0WPumx9o3PfHevlCFVLJBVTfhCe7+yzb6PVbg\n9nH2rQMS/kQZY0yysDt3jTEmxSRj4H8g1g2YgLVveqx902Ptm554b19I4rIsszHGmMhJxh6/McaY\nc7DAb4wxKSYhA/90qoVGqX2VIvIXEXlTRPaIyF1BtnmniHSISJ335+tRbuNhEdntPfZZ053F8hyK\nyHl+56VORDpF5O6AbaJ6/kTkERFpFpE3/JbNEJH/FJED3n+Lx9n3nO/XCLbvX71VcXeJyDMiEnQW\n+oneCxFs3zdFpMnv//CacfaN1fn7lV/bDotI3Tj7Rvz8hZ2qJtQPnnsJ6oGFQCawE1gesM01wJ8A\nAdYCW6LcxjnAGu/jAmB/kDa+E/hDDM/jYaDkHOtjeg4D/r9PAPNief7wFBtcA7zht+zbwL3ex/cC\n/zJO+8/5fo1g+94DpHsf/0uw9oXyXohg+74J3BPC/39Mzl/A+v8DfD1W5y/cP4nY459ytdBoNVBV\nj6vqdu/jLjylqAML28W7mJ5DP1cC9ao61Tu5w0JVXwBaAxZfBzzqffwo8P4gu4byfo1I+1T1P1R1\n2Pv0VTylVGJinPMXipidPx8REeD/A34Z7uPGSiIG/ulUC406EZkPrAa2BFl9sfdr+J9E5PyoNgwU\n+C8ReV1Ebg2yPl7O4fWM/wcXy/MHMEu9pUnwfCuZFWSbeDmP/wPPN7hgJnovRNLnvP+Hj4wzVBYP\n528dcFJVD4yzPpbnb0oSMfAnDBHJB34D3K2qnQGrtwNVqroS+AHw2yg371JVXYVnEp3bReSyKB9/\nQt4yH+8Dfh1kdazP3xnU850/LnOjReSrwDDw2DibxOq9cD+eIZxVwHE8wynx6AbO3duP+7+lQIkY\n+KdTLTRqRCQDT9B/TFWfDlyvqp2q2u19vAnIEJGSaLVPVZu8/zYDz+D5Su0v5ucQzx/SdlU9Gbgi\n1ufP66Rv+Mv7b3OQbWJ6HkXkZuBa4GPeD6ezhPBeiAhVPamqI6o6Cjw4znFjff7Sgb8DfjXeNrE6\nf9ORiIF/rFqot0d4PZ7qoP6eBT7uzUxZi1+10Gjwjgk+DOxV1e+Os81s73aIyEV4/i9ORal9eSJS\n4HuM5yLgGwGbxfQceo3b04rl+fPzLPAJ7+NPAL8Lsk0o79eIEJGrgC8B71PV3nG2CeW9EKn2+V8z\n+sA4x43Z+fNaD7ylqo3BVsby/E1LrK8uT+UHT8bJfjxX+7/qXbYB2OB9LHjmCa4HdgO1UW7fpXi+\n9u8C6rw/1wS08Q5gD54shVeBi6PYvoXe4+70tiEez2EenkBe5LcsZucPzwfQcWAIzzjzp4CZwJ+B\nA8B/ATO8284FNp3r/Rql9h3EMz7uew9uDGzfeO+FKLXv59731i48wXxOPJ0/7/Kf+t5zfttG/fyF\n+8dKNhhjTIpJxKEeY4wx02CB3xhjUowFfmOMSTEW+I0xJsVY4DfGmBRjgd8YY1KMBX5jjEkx/w+M\nPsT8gwlEaAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x113561ac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(len(eval_losses)), eval_losses)\n",
    "plt.title('test loss')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1137a9828>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4XPV56PHvq91arGW02LJsyZLlDWM2YSwgDoSEGJJC\nIG0C2WkSSm+gyb1PklLam7T33rQkadOQloSSlISskBBMnIRAIJjN2ODdlo1lLZZtSbZHm7VZ+7z3\njzkjBlmSR9Is0sz7eR491sw5M+c349F7fvOe97xHVBVjjDGxIy7SAzDGGBNeFviNMSbGWOA3xpgY\nY4HfGGNijAV+Y4yJMRb4jTEmxljgN8aYGGOB30QdEWkQkXcH4Xk+JSKvBmNMxswmFviNMSbGWOA3\nUUVEfgIsAX4rIj0i8mXn/vUi8pqInBGRfSJyjd9jPiUi9SLSLSJHReSjIrIKeAiodJ7nzATbu0NE\n3nQeWy8ifzVm+c0isldEukSkTkQ2OvfniMgPRaRZRDpE5KkQvSXGnEOsZYOJNiLSAHxGVZ93bi8C\n9gMfB54BrgMeA1YCZ4GTwOWqWi0iC4EcVT0oIp9ynufqSbb1PuAwUA9sAP4AXK2qu0VkHfAc8OfA\nn4CFQIaqHhaR3wM9wF3Ov1eq6kvBfSeMGV9CpAdgTBh8DHhaVZ92bj8nIjuBG4EnAA+wRkSOq+pJ\nvDuCgKjq7/1uviQifwTeAewGPg08oqrPOcubAJydyw2AS1U7fI+d3kszZuos1WNiQTHwF06a54yT\ntrkaWKiqvcCH8c68T4rI70VkZaBPLCI3iMh2EWl3nvdGINdZvBioG+dhi4F2v6BvTFhZ4DfRaGz+\n8gTwE1XN8vtJU9X7AVT1WVV9D95UzGHg+xM8z9uISDLwa+BfgQJVzQKeBsRvu2XjPPQEkCMiWdN4\nbcbMmAV+E41OA6V+t38K/JmIvFdE4kUkRUSuEZEiESlwDsCmAQN48+0ev+cpEpGkCbaTBCQDLcCw\niNwAXO+3/L+BO0TkOhGJE5FFIrLSSSf9AfiuiGSLSKKIbAjaqzfmPCzwm2j0L8A/OGmdL6rqCeBm\n4D68QfoE8CW8n/844H8BzUA78E7gr53neQE4CJwSkdaxG1HVbuBvgF8CHcBHgM1+y98A7gD+HejE\nm8cvdhZ/HBjC+w3DDXwhSK/dmPOyqh5jjIkxNuM3xpgYY4HfGGNijAV+Y4yJMRb4jTEmxszKM3dz\nc3O1pKQk0sMwxpg5Y9euXa2qmhfIurMy8JeUlLBz585ID8MYY+YMETkW6LqW6jHGmBhjgd8YY2KM\nBX5jjIkxFviNMSbGWOA3xpgYY4HfGGNijAV+Y4yJMRb4jTFmFnittpUfbj2KxxP6jsmz8gQuY0z0\nGhrxcKqzn+YzfZzs7KfpTN/o74nxwvpSF5VlLpbnZxAXJ+d/wijQMzDMl57YT1JCHLevW0JKXHxI\nt2eB3xgTNKpKW+8gJ894A/rJTm9Qbz7TT7Pzu7t7gLGXAclOTaQwax5d/UM8e/A0ADlpSawvzaGy\nLJfKUhdleWmIROeO4Ot/OExzZx+/+qtKUhJDG/TBAr8xZpraegZ482Q3h052ev9t7qKhrZeBYc/b\n1ktOiGNR1jwKs+axoTyPwqx5LMqax8KsFAqz5lGYOY95SW8Fu8aOs2yra2NbfRvb69p4+sApAPIz\nkllf6uLKMu83giU5qVGxI3itrpWfbD/Gp69eSkVJTli2OSuvwFVRUaHWq8eY2WHEoxxt7eXNk10c\nOtnl/be5C3f3wOg6C+ansGphBsvy052gPm802GenJk47QKsqx9rOsq2+bXRn0OJstzAzhfVlLq4s\ny6WyzMWirHlBeb3h1DswzHu//TIJccIfPr/hbTvAqRKRXapaEci6NuM3xozqGRjmsC+4n+zi0Mlu\nqk910T/kncUnxAnL8tO5elkuqwvns2qh9ycnbaLr0c+MiFCSm0ZJbhq3r1uCqlLX0su2ula21bfx\nYnULT+5uAmBJTiqffcdSPl5ZEpKxhMLXnzlM05k+Hr+zckZBf6oCCvwishF4AIgHfqCq949Zng08\nApQB/cBfqmqVs+x/Ap8BFDgA3KGq/UF7BcaYSakqXX3DtPT009I9SEvPAC3dA7Q6//p+WnsG3jaL\nz0pNZNWC+XxkXbET5L0z+uSE8AWosUS8O55l+el8vLIEj0c54u5mW10bT+xq5GtPv8mHLl8c0TEG\naltdGz/edow7riph3dLwpHh8zhv4RSQeeBB4D9AI7BCRzap6yG+1+4C9qnqLiKx01r9ORBYBfwOs\nVtU+EfklcBvwoyC/DmNQVVSJmUqQsQ40dvLHQ6fGCeiDDI54zlk/IU7Iy0gmNz2ZBZkpXLgok6Ls\neaMz+YWZKbM+hx4XJ6xcMJ+VC+azMHMed/10F/sbO7k8TLny6To7OMyXf72PYlcqX3rvirBvP5AZ\n/zqgVlXrAUTkMeBmwD/wrwbuB1DVwyJSIiIFftuYJyJDQCrQHKzBG+NzqLmLv/7ZLq4sc/Evt66N\n9HAi4h9/e5DdxztwpSU7AT2Jsvx08jKSyUtPftu/uenJZM5LjKqd5PrSHES8M+nZHvi/8Uw1J9r7\nePzO9aQmhT/jHsgWFwEn/G43AleMWWcfcCvwioisA4qBIlXdJSL/ChwH+oA/quofx9uIiNwJ3Amw\nZMmSKb0IE9t+v/8kX/zVPvqGRujsG+JrH7gwqgJaIFSVI6e7+ci6JXztlgsjPZyIyEpNYtWC+Wyr\na+NvriuP9HAmtL2+jR+91sCnrizhilJXRMYQrDN37weyRGQvcA+wBxhxcv83A0uBQiBNRD423hOo\n6sOqWqGqFXl5AV09zMQ4j0f55rOH+dzPd7O6cD5/d8NKzpwd4s1TXREZz9CIh+FxUirh0NI9QHf/\nMMvy0yOy/dlifamL3cc76B8aifRQxnV2cJi//fV+luSk8uWN4U/x+AQS+JuAxX63i5z7Rqlql6re\noaoXA58A8oB64N3AUVVtUdUh4EngyqCM3MS0rv4hPvvjnTy4pY7bLl/Mzz97BTddXAh4v+pHwl/9\nZBdfeHxvRLZd6+4BiPnAX1nmYmDYw94TZyI9lHF989lqjrWd5Rt/vjYiKR6fQAL/DqBcRJaKSBLe\ng7Ob/VcQkSxnGXgreF5W1S68KZ71IpIq3qNE1wFvBm/4JhbVtfTwgQe38tKRFv7vzRfwL7deSHJC\nPAsz51HiSmV7ffgDf+/AMC8faWHXsY6wbxugtsUb+MvzMyKy/dli3dK38vyzzRtH2/nRaw18srKY\n9RFK8ficd5ejqsMicjfwLN5yzkdU9aCI3OUsfwhYBTwqIgocBD7tLHtdRJ4AdgPDeFNAD4fklZiY\nsOWwm7/5xR4SE+L46WeuOOcPqLLMxe/2n2TEo8SHMc//+tE2hj3Kyc5+uvqHmJ+SGLZtg3fGn56c\nQMH85LBud7bJnJfIBYXzI7Lzn0zf4AhffmIfRdnz+PLGlZEeTmB1/Kr6NPD0mPse8vt9G7B8gsd+\nFfjqDMZoDKrK916q45vPVrNqwXwe/sRlFGWnnrPe+lIXv3jjBAebO1lblBW28b1a81agqXX3cOmS\n7LBt27fNsvz0WV9+GQ6VpS4efe0Y/UMjYel7E4hvPltNQ9tZfv7ZK0hLjvx5s9aW2cx6fYMj3POL\nPXzjmWred+FCfv3XV44b9MH7Rw/h/6q/tbaVYpd3TDWnu8O6bfAG/mV5sZ3f96ksczE44mF3hNJu\nY+1oaOeHrx3l4+uLubIsN9LDASzwm1museMsH/zea/z+wEm+vHEF/3H7JZOe2p4/P4WyvDS2hfGr\nvru7n+rT3Xz48sUkJ8RRc7onbNsG74Fud/dAzB/Y9akoySFOmBXpHm+KZz+LsuZx7w2RT/H4WOA3\ns9b2+jZu+s+tnOg4yyOfvJz/cc2ygFIZlWUudhxtZyhMpZVba1sB2FCex7L8dI64wxv4raLn7ean\nJHLhosyw7vwn8m9/rOZoay/f+ODaWZHi8bHAb2YdVeXH2xr42A9eJys1kac+dxXXrswP+PGVpbn0\nDo5woKkzdIP082pNG9mpiaxeOJ/y/HRqw5zqscB/rvVlLvaeOEPfYOTq+Xcda+e/tx7lo1cs4cpl\nsyPF42OB38wqA8Mj3PvrA3zlNwfZsDyPpz53FWVTzF2vL/Werh+OPL+qsrW2lSuX5RIXJ5QXZNDc\n2U93/1DIt+1T6+4hKT6Oxdlzry1xqKwvdTE0ohErr+0fGuFLv9pPYeY8/u7GVREZw2Qs8JtZw93V\nz+0Pb+fxnSf43LVlfP8TFdMqi3SlJ7OiICMsOd66ll5OdfVztTOjW17graOvDWO6p9bdw9LcNBLi\n7c/Z5/KSHOLjhG31rRHZ/reeO0J9ay9f/+Ba0mdRisdn9o3IxJzBYQ8/e/0YD/yphoEhDw9+5FLe\nt3bhjJ6zsszF4ztOMDjsISkhdAHx1ZoWgNHAX+6kW2pO93BJmEo6a909XLgoMyzbmivSkxO8ef4I\nnMi161gHP3ilntvXLeHq8tmV4vGxKYKJGFXlmapTXP/vL/FPvz3E6oXzeepzV8046IP3q37f0Aj7\nGkN76v6rtW0Uu1JZnOMt5Vyck+qt7HGHJ8/fPzTCiY6zlt8fR2WZi/2NnfQODIdtm/1DI3zpiX0s\nzJzHfTfOniqesSzwm4jYe+IMH/qvbdz1010kxMfxyKcq+NlnrmDFguC0HPBv0RsqwyMette3cZXf\ngbv4OKEsL50jYSrprG/pRdUO7I6nstTFsEfZGcY8/78/d4T6ll7u/+CFZIT57O2psMBvwupE+1nu\n+cUePvDgVo629vK1W9bwzOffwbtWFgT1rFP/Fr2hsq/xDD0Dw6NpHp/lBelhO4nL16PHAv+5LivO\nJiFOwpbu2XO8g++/Us/t6xbzjvLZ3WHYcvwmLDr7hvjullp+uLUBEbj72mXcdU1ZSA98VZa5+Mn2\n0J26/2pNGyJwZdnb+wWVF2Tw1N5muvuHQj7rq3X3ECewNDctpNuZi9KSE7hocVbYTuT69vM15KYn\nz8oqnrFsxm9CamjEw4+2HuWab27h4Vfq+bOLCnnxS9fwxfeuCHm1Q2Wpi8FhD7uPh+ar/tbaVi5c\nlElW6tsvNO47wBuOyp46dw+Lc1JnTU+a2aay1MWBpk56Qpznd3f180pNC39RURT2Bn3TYYHfhMRb\nB25f5h9/e4hVC+fz27uv5t8+dBELM8NTb76u1Dl1PwRf9XsHhtl9vONt+X2fcqeksyYMgd969Exu\nfamLEY+yo6E9pNvZvK8Zj8ItlxSFdDvBYqkeE3R7T5zhn3//Jm80tLMsP51HPlXBtSvyw945MpSn\n7vvaMI/N7wMsyUklKSEu5Hn+4REPR1t7uWbF7M4nR9Jlxdkkxgvb69q4dkXgZ39P1ZO7m1hblDln\njrVY4DdBoarUtfTynT/VsHlfM7npSXztljV8uGJxRE8sWl/m4pFXj9I3ODJpc7eperWmjeSEOC4r\nPrdW31fZE+oZ//H2swyOeCibI8EmEuYlxXPJ4uyQ9u2pPtXNoZNdfPXPVodsG8Fmgd9My9CIh4PN\nXexsaGfXsQ52NHTQ2jNAckIcd1+7jL96Z+msKGerLHXxXy/Vs/NYe1ArLbbWtrJuac6EufXlBens\nbAhtGaH16AnM+tIc/nNLbcgukLNpTxPxccKfXVQY9OcOFQv8JiCdfUPsPt7BroYOdh5rZ++JM/QP\nebtfFmXP4x3luVxWnM27VxWwIDMlwqN9y+UlOaMlfcEK/L42zLdcumjCdcrz0/nN3mZ6BoZDdhDb\nSjkDs77MxXdeqGXH0XauW1UQ1Of2eJTf7G3incvzyE2fO1c/s8BvzqGqNHb0sfNYOzsbOtjZ0MER\ndzeq3jTG6oXzuX3dEiqKc6goyaZg/uwJ9GOlJSewtii4eX5fG+bx8vs+5X49ey5eHJorgdW6eyiY\nnzwnqkgi6dIl2SQlxLGtri3ogX97fRsnO/u5bw6UcPqzwG8Ab7+cp/Y28VJ1CzuPtXO6awDw9jy5\nZEkW71u7kIribC5anDWr+ooHorLMxUMv1Qdt9u3fhnkivmZtNae7Qxb469w9NtsPQEpiPJcszmL7\n0eDn+Z/c00RGcgLvWR3cHUqoza2/YBN0wyMentzTxHf+VENjRx+FmSlcsdTF5SXZXFacw4oFGWG9\naHkoVJbm8uCWOnY0tM+4smNsG+aJjFb2hOgAr+9g+gcnSTeZt1SWuXjgTzV0nh0iMzU435D6Bkf4\nw4GTvG/twjl3HoUF/hg14lE272vigedraGg7y4WLMvm/N6/hmhV5UXfB7mCW9I1twzyR0cqeEJV0\nnurqp2dg2Gb8AaosdfHt52t4/Wgb11+wICjP+cdDp+gdHOEDl8y9na8F/hjj8Si/P3CSbz9/hLqW\nXlYuyODhj1/Ge1YHt1fObBLMkr6xbZgnU56fHrILgfgqeqyUMzAXL8kiOSGO7fXtQQv8m/Y0UZiZ\nwvqlrvOvPMtY4I8RHo/y7MFTfPv5GqpPd1Oen853P3opGy9YMGnKIlqsL3Pxny/UzLikb2wb5sks\nL0hn875megeGg35cxEo5pyY5IZ7LioNXz9/SPcArNa3cuaF0Tv79WMuGKKeqPHfoNO//j1f565/t\nZsjj4Tu3X8IzX9jAjRcunJMf2umoLHXhUXijfvqn7o/Xhnkyy/JDdzWuWncP81MSyJtDJYSRVlnq\n4s2TXXT0Ds74uTbva2bEo9w6B9M8YIE/aqkqW6rd3PzgVj774530Dg7zrQ9dxB+/sIGbLiqc8wds\np+qSJVnekr4ZzPj2NXaO24Z5IssLvLPxIyHI89c4FT3Rmp4LhfVOF9XXj868b8+mPY2sWTR/tGx3\nrrFUT5TxVp208a3nqtl9/AxF2fP4xgfXcsuli0iM4WuypiTGc9mS7Bn1Zn+1phUR78wxEEtyUkmK\njwvJjL/O3cN1q0LXeyYaXVSUxbzEeLbXt7FxzfTz/DWnu6lq6uJ/v3/utGgYywJ/lOjqH2LH0Xb+\n66V63mhopzAzhX++5UL+/LKikF5zdi6pLHPx788f4czZwXNaKQfC14Y5Oy2wxybEx1Galxb0GX9H\n7yBtvYOU58/N2WakJCXEUVEys50/eGv34+OEm+ZQi4axLPDPQapK05k+p0eO9+za6tPeM2vzM5L5\nPzdfwIcvX0xywtyqLQ61yjIX33oOtte3T3nG52vD/NkNpVN6XHlBBnuCfD0Aa9UwfetLXXzz2Wra\negZwTeP4iMej/GZPE+8ozyUvY+4eXwko8IvIRuABIB74gareP2Z5NvAIUAb0A3+pqlUisgJ43G/V\nUuArqvrtYAw+VgyPeDh8qpudDe3sOObtl3Oqqx+AtKR4Li3OZuOaBaMtFObaySTh4vuqv62udcqB\n/42j7RO2YZ7M8vx0fhvkyh6r6Jm+9aVv5flvvHDhlB+//WgbzZ39/O0Ns/dC6oE47ydRROKBB4H3\nAI3ADhHZrKqH/Fa7D9irqreIyEpn/etUtRq42O95moBNQX4NUadnYJg9x709cnYea2fv8TP0Do4A\nsDAzhcuX5lBRnM1lxdmsXJAR0bbHc8noV/1pHOB9paZ1wjbMkyl3DvDWtfSwtig4rRtq3T2kJMax\nKCs8F7SJJmuLMklNimdbXdu0Av9Te5pIT07g+tXBORcgUgKZgqwDalW1HkBEHgNuBvwD/2rgfgBV\nPSwiJSJSoKqn/da5DqhT1WPBGXr0+en2Y/z89eMcPtWFR0EEVi6YzwcvK+Ky4mwqSnLsj32GfF/1\nW3sGptRN8XxtmCfiq/o4cjq4gb80Nz1mSnGDKTE+joqSnGldh7d/aIQ/HDjFxjULgnpth0gIJPAv\nAk743W4Erhizzj7gVuAVEVkHFANFgH/gvw34xUQbEZE7gTsBlixZEsCwoktn3xBf3XyQ8vx07r52\nGRUlOVyyJGtW9LSPJpVOSd/2+jbevzawg3OBtGGeSLFT2VPjDt4B3lp3z5S/eZi3VJa6+Pozh2np\nHphSnv65Q6fpHhies7X7/oKVI7gfyBKRvcA9wB5gxLdQRJKAm4BfTfQEqvqwqlaoakVeXuxdSu6V\nmhZGPMr/+8Aa/tf1K9iwPM+CfghcuCiTNOerfqBeq/WuO9X8PrxV2VNzOjglnWcHh2k602f5/Rnw\n3/lPxaY9TSzMTBk9TjCXBRL4m4DFfreLnPtGqWqXqt6hqhcDnwDygHq/VW4Ado9J/Rg/Ww63kDkv\nMWQtfI1XYnwcly/NmVKe/5Wa1vO2YZ5MeUFG0Gb89S29gB3YnYk1hfNJT06YUuBv7RngpSMt3HRx\nYVSk2AIJ/DuAchFZ6szcbwM2+68gIlnOMoDPAC+rapffKrczSZon1nk8yktH3GxYnmcHasOgstRF\nfUsvp53KqMkE2oZ5MuX56Zxo7+Ps4PC0Hu/PtwOxwD99CfFxXD7Fg/y/HW3RUBTCkYXPeaOMqg4D\ndwPPAm8Cv1TVgyJyl4jc5ay2CqgSkWq8s/vP+x4vIml4K4KeDPbgo0VVcyetPYO8a2XspbgiYSpf\n9QNtwzwZX+uGYJzBW+vuIT5OKHGlzfi5YlllWeA7f/CmeVYvnM+KBdFx0lxA00tVfVpVl6tqmap+\nzbnvIVV9yPl9m7N8hareqqodfo/tVVWXqnaG5iXMfVsOtyACG4J4MXAzsQsKM8lISQgozz+VNswT\n8TVrC0aev9bdQ7Er1c7GniFfnj6QnX+tu4f9jZ3cGkUXvbFPzyzwQrWbi4qypnUmoZm6+DjhiqWu\ngL7qv1rbxpKcwNowT6TElUpivHAkCHn+WncPy/IszTNTvp1/IIF/055G4oQ53aJhLAv8EdbaM8D+\nxjMzvjKUmZrKMhfH2s7SfKZvwnV8bZivLp/+bB+cyp7cdGpnOOMfGvFwrO2s5feDwLvzzznvtz6P\nR3lqTzNXl+eRPz8lTKMLPQv8EfbykRZU4V0rLfCHk6/D5mR/+FNtwzyZ8oL0Gc/4j7X1MuxRC/xB\nsr7URUPbWU52Trzzf6OhnaYzfVFRu+/PAn+EbaluITc9mQsKp1cqaKZn5YIMslMTJ033TLUN82TK\n8zNo7JhZZY/16AmuQPL8m3Y3kZoUz/UXFIRrWGFhgT+Chkc8vHykhWtW5EVFbfBcEufL808y499a\n28qawsDbME9meUE6qlDn7p32c4xeZ9dy/EGxeuF8Muclsr1u/Auz9A+N8PSBk2xcs4DUpOhqZGyB\nP4L2nDhDZ9+Q5fcjpLLMRdOZPk60nz1nma8Nc6CXWTwfX8+emZzIVevuoTAzJejX741VcXHCuklO\n5nv+TV+Lhuio3fdngT+Cthx2Ex8nMz54aKbHV88/3qzf14b5HUH6vyn2VfbM4ABvbUsPZZbmCarK\nUhfH28/SNM5B/k27myiYnzz6OYkmFvgjaEt1CxXF2WTOs548kVCen05uetK4M77ptmGeSKKvsmea\nM36PR72lnBb4g2r0ZL4xO/82p0XDBy5eFJXXp7bAHyGnOvt582QX11o1T8SICFeUunitrhVVfduy\nrbWtXF4y9TbMk1lWkD7tGX/TmT76hzx2ucUgW1Ew/kH+3+0/ybBHp9WRdS6wwB8hW6rdAJbfj7DK\nUhenuwY42vrWQVdfG+Zg5fd9ludncKLjLH2DI+dfeQy73GJoTHSQ/8k9TaxaOJ+VC6Kz2s4Cf4Rs\nOeymMDNltI+LiYzRPL/fjM/XhjlY+X2fcl9lT8vUZ/11VsoZMmMP8te19LDvxBluuSR6ztQdywJ/\nBAwMj7C1tpVrV+YjEn35w7mkNDeN/Izkt834XqlpJWsGbZgn4tvJT6eyp9bdQ05aEjlBKC01b+er\n5/ft/J/a00ScwM0XR2eaByzwR8TOhg56B0cszTMLiAiVZS6217ejqqNtmK8qm34b5okUu9KmXdlj\nPXpCZ3lBOq60JLbXteHxKJv2NHHVslwKoqhFw1gW+CNgy2E3SfFxXLks+srE5qLKUhetPQPUuntG\n2zAHO78P3sqepblp1Jye2oxfVa2UM4REhPWlLrbXt7HzWAeNHX3cEmUtGsayM0Ei4IVqN1eU5kTd\n2YBzlX+e31fcE+z8vk95fgZVzVPrUN7WO8iZs0OW3w+h9aU5/P7ASR740xHmJcbz3gsWRHpIIWUz\n/jA71tZLfUuvpXlmkSU5qRRmprCtro1Xalpn3IZ5MuUF6Rxvn1plj/XoCT3fzn9rbRsb1yyI+rOj\nLfCH2YvV3gt7WDfO2UNEWF/m/ar/en1bSNI8PuX5GVOu7LHAH3pleenkOtfDiPY0D1jgD7st1W6W\n5qZRkmuXzptNKktddJwdojtIbZgnMp3Knlp3D6lJ8RRmRu/BxkgTEd65PI9FWfNCuuOfLaL7+8ws\n0zc4wra6Nj5yxZJID8WM4fuqLwJXhrA3S0luGglxMqXLMPpaNVjpb2j9n5svoG9oJCpbNIxlgT+M\nttW3MjDssfz+LFSUncqSnFQy5yUGpQ3zRHyVPVMp6ax194R0Z2S80pIToj637xMbr3KW2HK4hXmJ\n8VxRmhPpoZhx/OdHLgnLRcyXF2RwMMDKnu7+IU519Vsppwkqy/GHiaqypdrNVctySU4IXuMvEzxr\ni7LC0ptlWX46x9rP0j90/sqeupbe0ccYEywW+MOkrqWHxo4+rl2ZF+mhmAhbXhB4ZY9V9JhQsMAf\nJlsOe8s4r7H8fswr91X2BJDnr3X3kBgvFIfovAITmyzwh8kLh92sKMhgUda8SA/FRFiJy6nsCaCk\ns9bd410/3v5UTfDYpykMuvuH2NHQbhddMQAkJcRREmBlT12LXXXLBJ8F/jDYWtvKsEe5doXl943X\n8oL08zZrGxge4VhbrwV+E3QW+MNgy+EWMlISuDRI1281c9+y/AyOn6ey52hrLx61A7sm+Czwh5iv\njHNDeR6Jlqc1juUF6XjOU9ljFT0mVAKKRCKyUUSqRaRWRO4dZ3m2iGwSkf0i8oaIrPFbliUiT4jI\nYRF5U0Qqg/kCZruDzV24uwcsv2/exnfRdF9wH0+tuwcRbwMxY4LpvIFfROKBB4EbgNXA7SKyesxq\n9wF7VXWoq6DjAAAToElEQVQt8AngAb9lDwDPqOpK4CLgzWAMfK540bmo+juXW37fvGWp07PnyCR5\n/lp3D0XZ80hJtBP+THAFMuNfB9Sqar2qDgKPATePWWc18AKAqh4GSkSkQEQygQ3AfzvLBlX1TNBG\nPwdsqW5hbVEmeRnJkR6KmUV8lT2T1fLb5RZNqAQS+BcBJ/xuNzr3+dsH3AogIuuAYqAIWAq0AD8U\nkT0i8gMRGbcfsYjcKSI7RWRnS0vLFF/G7NTRO8ie4x120pYZV3l+OjUTpHpGPEp9q1X0mNAI1tHG\n+4EsEdkL3APsAUbwNoG7FPieql4C9ALnHCMAUNWHVbVCVSvy8qIjLfJyTQsexco4zbjKCzI41tY7\nbmVPY8dZBoc9FvhNSAQS+JuAxX63i5z7Rqlql6reoaoX483x5wH1eL8dNKrq686qT+DdEcSELYfd\nuNKSuKgoK9JDMbNQeb63sqfeacTmzyp6TCgFEvh3AOUislREkoDbgM3+KziVO74m5p8BXnZ2BqeA\nEyKywll2HXAoSGOf1UY8yktHWnjn8jziYuDCDmbqlhd4K3vGa90wGvjzMsI6JhMbztuPX1WHReRu\n4FkgHnhEVQ+KyF3O8oeAVcCjIqLAQeDTfk9xD/AzZ8dQD9wR5NcwK+1rPEPH2SGusTJOM4GS3FTi\nJ7gaV627h9z0ZDJTEyMwMhPtAroQi6o+DTw95r6H/H7fBiyf4LF7gYoZjHFOevGwmziBDeXRf/1O\nMz3JCfGUuFLHnfHXuHsotzSPCRE7lTREXqh2c1lxNlmpobuMn5n7yvMzzpnxqyp1bmvOZkLHAn8I\nuLv6qWrqsjJOc17LC9JpaOtlYPityh539wDdA8MW+E3IWOAPgRePeM9DsIuqm/NZVpBxTmWPVfSY\nULPAHwIvVrspmJ/MqoVWkWEmt9y5Gpd/6wYL/CbULPAH2dCIh1eOtHLtinxErIzTTG5pbhrxcfK2\nZm217h4ykhPItzYfJkQs8AfZzoYOugeGrRunCUhyQjzFrtRzZvxl+ek2cTAhY4E/yF6sdpMYL1y1\nzMo4TWCW52e8rWdPrV1u0YSYBf4g21LtZt3SHNKTAzpFwhjKC9I51naWgeEROvuGaOkesMBvQsqi\nUxA1dpzlyOkePlSx+PwrG+MoL8hgxKMcbe2ld8Bb1mntmE0oWeAPohernTJOy++bKfCdoXvkdA99\ng8Pe+wos8JvQscAfRFsOu1mSk0pp7riXHDBmXKV5acQJ1J7upm9ohKSEOIqyUyM9LBPFLMcfJE/s\nauSFajfXry6wagwzJd6ePWkcOd1DrbuHUqfE05hQsRl/EPxq5wm+/Ov9XFWWyxffu+L8DzBmjPKC\ndGrc3QyOeOz6DSbkbMY/Q790gv7Vy3L5wScr7MLYZlrK8zNoaDtLY0efVfSYkLMZ/wz8cscJ/vZJ\nb9D//ics6JvpKy9IZ8SjgLVqMKFngX+aHt9xnHufPMA7yvN4+OOXWdA3M1Ke/1ZfJwv8JtQs8E/D\nY294g/6G5Rb0TXD4KnvA27/HmFCywD9Fv3jjOH/35AHeuTyP/7Kgb4IkJTGeYlcaqkpygn2mTGhZ\n4J+Cn79+nPs2HeCaFXk89DEL+ia4PllZjJPmNyakLPAH6GevH+PvN1Vx7Yo8Hvr4ZTYrM0H3qauW\nRnoIJkZY4A/AT7cf4x+equJdK/P53scutaBvjJnTrI7/PH6yrYF/eKqK6yzoG2OihAX+Sfx4WwP/\n+zcHefeqfL5rQd8YEyUs8E/gx9sa+IoT9B/8qAV9Y0z0sBz/OB59rYGvbj7Iu1cV8N2PXkpSgu0f\njTHRwwL/GD/cepR/+u0h3rO6gAc/YkHfGBN9LKr5efS1Bv7pt4d47wUW9I0x0csim2NoxMM/P/0m\nG5bn8R+3W9A3xkSvgKKbiGwUkWoRqRWRe8dZni0im0Rkv4i8ISJr/JY1iMgBEdkrIjuDOfhgqjnd\nw8Cwhz+/rMiCvjEmqp03xy8i8cCDwHuARmCHiGxW1UN+q90H7FXVW0RkpbP+dX7Lr1XV1iCOO+iq\nmjsBWFM4P8IjMcaY0ApkarsOqFXVelUdBB4Dbh6zzmrgBQBVPQyUiEhBUEcaYlVNnaQnJ1Diss6I\nxpjoFkjgXwSc8Lvd6Nznbx9wK4CIrAOKgSJnmQLPi8guEblzoo2IyJ0islNEdra0tAQ6/qCpaupk\ndeF84uxap8aYKBesZPb9QJaI7AXuAfYAI86yq1X1YuAG4HMismG8J1DVh1W1QlUr8vLygjSswIx4\nlEMnu1hTmBnW7RpjTCQEUsffBCz2u13k3DdKVbuAOwBERICjQL2zrMn51y0im/Cmjl6e8ciDqL6l\nh/4hD2sWWX7fGBP9Apnx7wDKRWSpiCQBtwGb/VcQkSxnGcBngJdVtUtE0kQkw1knDbgeqAre8IPj\nQJNzYHeRzfiNMdHvvDN+VR0WkbuBZ4F44BFVPSgidznLHwJWAY+KiAIHgU87Dy8ANnm/BJAA/FxV\nnwn+y5iZqqYuUhLjKMuza50aY6JfQC0bVPVp4Okx9z3k9/s2YPk4j6sHLprhGEOuqrmT1QvnE28H\ndo0xMSDmz1TyeJRDzV2W5jHGxIyYD/wNbb30DAxbRY8xJmbEfOCvau4C7MCuMSZ2xHzgP9jUSVJ8\nHOUFdmDXGBMbYj7wVzV3snJhBonxMf9WGGNiRExHO1WlqskO7BpjYktMB/7Gjj46+4bswK4xJqbE\ndOCvGj1j11o1GGNiR2wH/uZOEuKE5QUZkR6KMcaETUwH/gNNXSwvyCAlMT7SQzHGmLCJ2cCvqhxs\n6rQ0jzEm5sRs4D/V1U9b76BV9BhjYk7MBv4Djd4DuxdYRY8xJsbEbOCvau4iTmD1Qkv1GGNiS8wG\n/oNNnSzLT2dekh3YNcbElpgN/FXNnXbiljEmJsVk4Hd393O6a4AL7MCuMSYGxWTgP9jkbcV8oQV+\nY0wMisnA72vVsLrQDuwaY2JPbAb+5k5Kc9NITw7oksPGGBNVYjPwWytmY0wMi7nA3947SNOZPmvV\nYIyJWTEX+A82O62YrZTTGBOjYi7wVzkVPdaqwRgTq2Iw8HeyJCeVzNTESA/FGGMiIvYCf7O1YjbG\nxLaYCvydfUMcaztraR5jTEyLqcB/qNmb37dSTmNMLIupwD96cXU7Y9cYE8MCCvwislFEqkWkVkTu\nHWd5tohsEpH9IvKGiKwZszxeRPaIyO+CNfDpqGrupDAzBVd6ciSHYYwxEXXewC8i8cCDwA3AauB2\nEVk9ZrX7gL2quhb4BPDAmOWfB96c+XBnpqqp0zpyGmNiXiAz/nVArarWq+og8Bhw85h1VgMvAKjq\nYaBERAoARKQIeB/wg6CNehp6B4apb+21E7eMMTEvkMC/CDjhd7vRuc/fPuBWABFZBxQDRc6ybwNf\nBjyTbURE7hSRnSKys6WlJYBhTc2hk12owoVFlt83xsS2YB3cvR/IEpG9wD3AHmBERN4PuFV11/me\nQFUfVtUKVa3Iy8sL0rDe8taBXZvxG2NiWyB9iZuAxX63i5z7RqlqF3AHgIgIcBSoBz4M3CQiNwIp\nwHwR+amqfiwIY5+SqqYu8jKSyZ+fEu5NG2PMrBLIjH8HUC4iS0UkCbgN2Oy/gohkOcsAPgO8rKpd\nqvp3qlqkqiXO416IRNAH74zfrrhljDEBBH5VHQbuBp7FW5nzS1U9KCJ3ichdzmqrgCoRqcZb/fP5\nUA14OvoGR6hxd1v9vjHGEFiqB1V9Gnh6zH0P+f2+DVh+nud4EXhxyiMMgsOnuvAoVsppjDHEyJm7\nVdaqwRhjRsVG4G/sJCcticJMO7BrjDGxEfibO7mgcD7egiNjjIltUR/4B4ZHOHK629I8xhjjiPrA\nX3O6h6ERtRO3jDHGEfWB/4Bzxq7V8BtjjFfUB/6qpk4yUhJYnDMv0kMxxphZIfoDf3MXawoz7cCu\nMcY4ojrwD414ePNkFxcWWZrHGGN8ojrw17p7GBz2cIG1ajDGmFFRHfhHWzHbgV1jjBkV1YH/YHMX\naUnxLHWlRXooxhgza0R14K9q6uSCwkzi4uzArjHG+ERt4B/xKAebu7hgkeX3jTHGX9QG/qOtPfQN\njdgZu8YYM0bUBv6qJmvFbIwx44nawH+gqZOUxDjK8uzArjHG+IvawF/V1MmqhfNJiI/al2iMMdMS\nlVHR41EOOa0ajDHGvF1UBv7j7WfpHhhmjVX0GGPMOaIy8B+wM3aNMWZCURn4q5o7SYqPozw/I9JD\nMcaYWScqA//Bpi5WLMggKSEqX54xxsxI1EVGVaWqudPSPMYYM4GoC/yNHX2cOTtkB3aNMWYCURf4\nDzY7B3atlNMYY8YVdYG/qqmLhDhhxQI7sGuMMeOJvsDf3El5QQYpifGRHooxxsxKAQV+EdkoItUi\nUisi946zPFtENonIfhF5Q0TWOPenOLf3ichBEfmnYL8Af6pKVVMna+xSi8YYM6HzBn4RiQceBG4A\nVgO3i8jqMavdB+xV1bXAJ4AHnPsHgHep6kXAxcBGEVkfrMGPdbprgNaeQavoMcaYSQQy418H1Kpq\nvaoOAo8BN49ZZzXwAoCqHgZKRKRAvXqcdRKdHw3O0M/11jV2bcZvjDETCSTwLwJO+N1udO7ztw+4\nFUBE1gHFQJFzO15E9gJu4DlVfX28jYjInSKyU0R2trS0TO1VOKqaO4kTWLXQAr8xxkwkWAd37wey\nnAB/D7AHGAFQ1RFVvRjvjmCdL/8/lqo+rKoVqlqRl5c3rUFUNXVSlpdOalLCtB5vjDGxIJAI2QQs\n9rtd5Nw3SlW7gDsARESAo0D9mHXOiMgWYCNQNYMxT6iqqYvKMlcontoYY6JGIDP+HUC5iCwVkSTg\nNmCz/woikuUsA/gM8LKqdolInohkOevMA94DHA7e8N8yOOzh6vJc3rl8et8WjDEmVpx3xq+qwyJy\nN/AsEA88oqoHReQuZ/lDwCrgURFR4CDwaefhC5374/HuZH6pqr8LwesgKSGOf/2Li0Lx1MYYE1VE\nNWRFNtNWUVGhO3fujPQwjDFmzhCRXapaEci6UXfmrjHGmMlZ4DfGmBhjgd8YY2KMBX5jjIkxFviN\nMSbGWOA3xpgYY4HfGGNizKys4xeRFuDYNB+eC7QGcTjBZuObGRvfzNj4ZmY2j69YVQNqXTArA/9M\niMjOQE9iiAQb38zY+GbGxjczs318gbJUjzHGxBgL/MYYE2OiMfA/HOkBnIeNb2ZsfDNj45uZ2T6+\ngERdjt8YY8zkonHGb4wxZhIW+I0xJsbMycAvIhtFpFpEakXk3nGWi4h8x1m+X0QuDfP4FovIFhE5\nJCIHReTz46xzjYh0ishe5+crYR5jg4gccLZ9zsUPIvkeisgKv/dlr4h0icgXxqwT1vdPRB4REbeI\nVPndlyMiz4lIjfNv9gSPnfTzGsLxfVNEDjv/f5t8V8Mb57GTfhZCOL5/FJEmv//DGyd4bKTev8f9\nxtbgXFN8vMeG/P0LOlWdUz94rwJWB5QCScA+YPWYdW4E/gAIsB54PcxjXAhc6vyeARwZZ4zXAL+L\n4PvYAOROsjyi7+GY/+9TeE9Oidj7B2wALgWq/O77BnCv8/u9wNcnGP+kn9cQju96IMH5/evjjS+Q\nz0IIx/ePwBcD+P+PyPs3Zvm/AV+J1PsX7J+5OONfB9Sqar2qDgKPATePWedm4MfqtR3IEpGF4Rqg\nqp5U1d3O793Am8CicG0/SCL6Hvq5DqhT1emeyR0Uqvoy0D7m7puBR53fHwU+MM5DA/m8hmR8qvpH\nVR12bm4HioK93UBN8P4FImLvn4+ICPAh4BfB3m6kzMXAvwg44Xe7kXODaiDrhIWIlACXAK+Ps/hK\n52v4H0TkgrAODBR4XkR2icid4yyfLe/hbUz8BxfJ9w+gQFVPOr+fAgrGWWe2vI9/ifcb3HjO91kI\npXuc/8NHJkiVzYb37x3AaVWtmWB5JN+/aZmLgX/OEJF04NfAF1S1a8zi3cASVV0L/AfwVJiHd7Wq\nXgzcAHxORDaEefvnJSJJwE3Ar8ZZHOn3723U+51/VtZGi8jfA8PAzyZYJVKfhe/hTeFcDJzEm06Z\njW5n8tn+rP9bGmsuBv4mYLHf7SLnvqmuE1Iikog36P9MVZ8cu1xVu1S1x/n9aSBRRHLDNT5VbXL+\ndQOb8H6l9hfx9xDvH9JuVT09dkGk3z/HaV/6y/nXPc46EX0fReRTwPuBjzo7p3ME8FkICVU9raoj\nquoBvj/BdiP9/iUAtwKPT7ROpN6/mZiLgX8HUC4iS50Z4W3A5jHrbAY+4VSmrAc6/b6Sh5yTE/xv\n4E1V/dYE6yxw1kNE1uH9v2gL0/jSRCTD9zveg4BVY1aL6HvomHCmFcn3z89m4JPO758EfjPOOoF8\nXkNCRDYCXwZuUtWzE6wTyGchVOPzP2Z0ywTbjdj753g3cFhVG8dbGMn3b0YifXR5Oj94K06O4D3a\n//fOfXcBdzm/C/Cgs/wAUBHm8V2N92v/fmCv83PjmDHeDRzEW6WwHbgyjOMrdba7zxnDbHwP0/AG\n8ky/+yL2/uHdAZ0EhvDmmT8NuIA/ATXA80COs24h8PRkn9cwja8Wb37c9xl8aOz4JvoshGl8P3E+\nW/vxBvOFs+n9c+7/ke8z57du2N+/YP9YywZjjIkxczHVY4wxZgYs8BtjTIyxwG+MMTHGAr8xxsQY\nC/zGGBNjLPAbY0yMscBvjDEx5v8DD4zKq+qWgTkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1135c4fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(len(eval_acces)), eval_acces)\n",
    "plt.title('test acc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到我们的三层网络在训练集上能够达到 99.9% 的准确率，测试集上能够达到 98.20% 的准确率"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习：看一看上面的训练过程，看一下准确率是怎么计算出来的，特别注意 max 这个函数**\n",
    "\n",
    "**自己重新实现一个新的网络，试试改变隐藏层的数目和激活函数，看看有什么新的结果**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}