machinelearning_notebook/6_pytorch/1_NN/4-deep-nn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 深层神经网络\n",
    "前面一章我们简要介绍了神经网络的一些基本知识，同时也是示范了如何用神经网络构建一个复杂的非线性二分类器，更多的情况神经网络适合使用在更加复杂的情况，比如图像分类的问题，下面我们用深度学习的入门级数据集 MNIST 手写体分类来说明一下更深层神经网络的优良表现。\n",
    "\n",
    "## MNIST 数据集\n",
    "mnist 数据集是一个非常出名的数据集，基本上很多网络都将其作为一个测试的标准，其来自美国国家标准与技术研究所, National Institute of Standards and Technology (NIST)。 训练集 (training set) 由来自 250 个不同人手写的数字构成, 其中 50% 是高中学生, 50% 来自人口普查局 (the Census Bureau) 的工作人员，一共有 60000 张图片。 测试集(test set) 也是同样比例的手写数字数据，一共有 10000 张图片。\n",
    "\n",
    "每张图片大小是 28 x 28 的灰度图，如下\n",
    "\n",
    "![](https://ws3.sinaimg.cn/large/006tKfTcly1fmlx2wl5tqj30ge0au745.jpg)\n",
    "\n",
    "所以我们的任务就是给出一张图片，我们希望区别出其到底属于 0 到 9 这 10 个数字中的哪一个。\n",
    "\n",
    "## 多分类问题\n",
    "前面我们讲过二分类问题，现在处理的问题更加复杂，是一个 10 分类问题，统称为多分类问题，对于多分类问题而言，我们的 loss 函数使用一个更加复杂的函数，叫交叉熵。\n",
    "\n",
    "### softmax\n",
    "提到交叉熵，我们先讲一下 softmax 函数，前面我们见过了 sigmoid 函数，如下\n",
    "\n",
    "$$s(x) = \\frac{1}{1 + e^{-x}}$$\n",
    "\n",
    "可以将任何一个值转换到 0 ~ 1 之间，当然对于一个二分类问题，这样就足够了，因为对于二分类问题，如果不属于第一类，那么必定属于第二类，所以只需要用一个值来表示其属于其中一类概率，但是对于多分类问题，这样并不行，需要知道其属于每一类的概率，这个时候就需要 softmax 函数了。\n",
    "\n",
    "softmax 函数示例如下\n",
    "\n",
    "![](https://ws4.sinaimg.cn/large/006tKfTcly1fmlxtnfm4fj30ll0bnq3c.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于网络的输出 $z_1, z_2, \\cdots z_k$，我们首先对他们每个都取指数变成 $e^{z_1}, e^{z_2}, \\cdots, e^{z_k}$，那么每一项都除以他们的求和，也就是\n",
    "\n",
    "$$\n",
    "z_i \\rightarrow \\frac{e^{z_i}}{\\sum_{j=1}^{k} e^{z_j}}\n",
    "$$\n",
    "\n",
    "如果对经过 softmax 函数的所有项求和就等于 1，所以他们每一项都分别表示属于其中某一类的概率。\n",
    "\n",
    "## 交叉熵\n",
    "交叉熵衡量两个分布相似性的一种度量方式，前面讲的二分类问题的 loss 函数就是交叉熵的一种特殊情况，交叉熵的一般公式为\n",
    "\n",
    "$$\n",
    "cross\\_entropy(p, q) = E_{p}[-\\log q] = - \\frac{1}{m} \\sum_{x} p(x) \\log q(x)\n",
    "$$\n",
    "\n",
    "对于二分类问题我们可以写成\n",
    "\n",
    "$$\n",
    "-\\frac{1}{m} \\sum_{i=1}^m (y^{i} \\log sigmoid(x^{i}) + (1 - y^{i}) \\log (1 - sigmoid(x^{i}))\n",
    "$$\n",
    "\n",
    "这就是我们之前讲的二分类问题的 loss，当时我们并没有解释原因，只是给出了公式，然后解释了其合理性，现在我们给出了公式去证明这样取 loss 函数是合理的\n",
    "\n",
    "交叉熵是信息理论里面的内容，这里不再具体展开，更多的内容，可以看到下面的[链接](http://blog.csdn.net/rtygbwwwerr/article/details/50778098)\n",
    "\n",
    "下面我们直接用 mnist 举例，讲一讲深度神经网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "from torchvision.datasets import mnist # 导入 pytorch 内置的 mnist 数据\n",
    "\n",
    "from torch import nn\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用内置函数下载 mnist 数据集\n",
    "train_set = mnist.MNIST('../../data/mnist', train=True, download=True)\n",
    "test_set  = mnist.MNIST('../../data/mnist', train=False, download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看看其中的一个数据是什么样子的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "a_data, a_label = train_set[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABAElEQVR4nGNgGMyAWUhIqK5jvdSy/9/rGRgYGFhgEnJsVjYCwQwMDAxPJgV+vniQgYGBgREqZ7iXH8r6l/SV4dn7m8gmCt3++/fv37/Htn3/iMW+gDnZf/+e5WbQnoXNNXyMs/5GoQoxwVmf/n9kSGFiwAW49/11wynJoPzx4YIcRlyygR/+/i2XxCWru+vv32nSuGQFYv/83Y3b4p9/fzpAmSyoMnohpiwM1w5h06Q+5enfv39/bcMiJVF09+/fv39P+mFKiTtd/fv3799jgZiBJLT69t+/f/8eDuDEkDJf8+jv379/v7Ryo4qzMDAwMAQGMjBc3/y35wM2V1IfAABFF16Aa0wAOwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x7FF1658A5278>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(5)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里的读入的数据是 PIL 库中的格式，我们可以非常方便地将其转换为 numpy array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(28, 28)\n"
     ]
    }
   ],
   "source": [
    "a_data = np.array(a_data, dtype='float32')\n",
    "print(a_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们可以看到这种图片的大小是 28 x 28"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   3.  18.\n",
      "   18.  18. 126. 136. 175.  26. 166. 255. 247. 127.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.  30.  36.  94. 154. 170. 253.\n",
      "  253. 253. 253. 253. 225. 172. 253. 242. 195.  64.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.  49. 238. 253. 253. 253. 253. 253.\n",
      "  253. 253. 253. 251.  93.  82.  82.  56.  39.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.  18. 219. 253. 253. 253. 253. 253.\n",
      "  198. 182. 247. 241.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.  80. 156. 107. 253. 253. 205.\n",
      "   11.   0.  43. 154.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.  14.   1. 154. 253.  90.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0. 139. 253. 190.\n",
      "    2.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  11. 190. 253.\n",
      "   70.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  35. 241.\n",
      "  225. 160. 108.   1.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  81.\n",
      "  240. 253. 253. 119.  25.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "   45. 186. 253. 253. 150.  27.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.  16.  93. 252. 253. 187.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0. 249. 253. 249.  64.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "   46. 130. 183. 253. 253. 207.   2.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  39. 148.\n",
      "  229. 253. 253. 253. 250. 182.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.  24. 114. 221. 253.\n",
      "  253. 253. 253. 201.  78.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.  23.  66. 213. 253. 253. 253.\n",
      "  253. 198.  81.   2.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.  18. 171. 219. 253. 253. 253. 253. 195.\n",
      "   80.   9.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.  55. 172. 226. 253. 253. 253. 253. 244. 133.  11.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0. 136. 253. 253. 253. 212. 135. 132.  16.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]\n",
      " [  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.\n",
      "    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]]\n"
     ]
    }
   ],
   "source": [
    "print(a_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以将数组展示出来，里面的 0 就表示黑色，255 表示白色\n",
    "\n",
    "对于神经网络，我们第一层的输入就是 28 x 28 = 784，所以必须将得到的数据我们做一个变换，使用 reshape 将他们拉平成一个一维向量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def data_tf(x):\n",
    "    x = np.array(x, dtype='float32') / 255\n",
    "    x = (x - 0.5) / 0.5 # 标准化，这个技巧之后会讲到\n",
    "    x = x.reshape((-1,)) # 拉平\n",
    "    x = torch.from_numpy(x)\n",
    "    return x\n",
    "\n",
    "train_set = mnist.MNIST('../../data/mnist', train=True, transform=data_tf, download=True) # 重新载入数据集，申明定义的数据变换\n",
    "test_set = mnist.MNIST('../../data/mnist', train=False, transform=data_tf, download=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([784])\n",
      "tensor(5)\n"
     ]
    }
   ],
   "source": [
    "a, a_label = train_set[0]\n",
    "print(a.shape)\n",
    "print(a_label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "# 使用 pytorch 自带的 DataLoader 定义一个数据迭代器\n",
    "train_data = DataLoader(train_set, batch_size=64, shuffle=True)\n",
    "test_data = DataLoader(test_set, batch_size=128, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用这样的数据迭代器是非常有必要的，如果数据量太大，就无法一次将他们全部读入内存，所以需要使用 python 迭代器，每次生成一个批次的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "a, a_label = next(iter(train_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([64, 784])\n",
      "torch.Size([64])\n"
     ]
    }
   ],
   "source": [
    "# 打印出一个批次的数据大小\n",
    "print(a.shape)\n",
    "print(a_label.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用 Sequential 定义 4 层神经网络\n",
    "net = nn.Sequential(\n",
    "    nn.Linear(784, 400),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(400, 200),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(200, 100),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(100, 10)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=784, out_features=400, bias=True)\n",
       "  (1): ReLU()\n",
       "  (2): Linear(in_features=400, out_features=200, bias=True)\n",
       "  (3): ReLU()\n",
       "  (4): Linear(in_features=200, out_features=100, bias=True)\n",
       "  (5): ReLU()\n",
       "  (6): Linear(in_features=100, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "交叉熵在 pytorch 中已经内置了，交叉熵的数值稳定性更差，所以内置的函数已经帮我们解决了这个问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义 loss 函数\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.SGD(net.parameters(), 1e-1) # 使用随机梯度下降，学习率 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/bushuhui/.virtualenv/dl/lib/python3.5/site-packages/ipykernel_launcher.py:22: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number\n",
      "/home/bushuhui/.virtualenv/dl/lib/python3.5/site-packages/ipykernel_launcher.py:25: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number\n",
      "/home/bushuhui/.virtualenv/dl/lib/python3.5/site-packages/ipykernel_launcher.py:41: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number\n",
      "/home/bushuhui/.virtualenv/dl/lib/python3.5/site-packages/ipykernel_launcher.py:44: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 0, Train Loss: 0.166705, Train Acc: 0.947978, Eval Loss: 0.129106, Eval Acc: 0.959157\n",
      "epoch: 1, Train Loss: 0.117714, Train Acc: 0.962836, Eval Loss: 0.097123, Eval Acc: 0.969838\n",
      "epoch: 2, Train Loss: 0.092098, Train Acc: 0.970532, Eval Loss: 0.098194, Eval Acc: 0.969541\n",
      "epoch: 3, Train Loss: 0.074442, Train Acc: 0.975880, Eval Loss: 0.077213, Eval Acc: 0.975574\n",
      "epoch: 4, Train Loss: 0.062742, Train Acc: 0.979594, Eval Loss: 0.149892, Eval Acc: 0.955301\n",
      "epoch: 5, Train Loss: 0.052319, Train Acc: 0.983276, Eval Loss: 0.124755, Eval Acc: 0.961531\n",
      "epoch: 6, Train Loss: 0.045134, Train Acc: 0.985091, Eval Loss: 0.085263, Eval Acc: 0.975178\n",
      "epoch: 7, Train Loss: 0.038610, Train Acc: 0.987423, Eval Loss: 0.063986, Eval Acc: 0.980123\n",
      "epoch: 8, Train Loss: 0.033068, Train Acc: 0.988906, Eval Loss: 0.074201, Eval Acc: 0.977453\n",
      "epoch: 9, Train Loss: 0.029478, Train Acc: 0.990155, Eval Loss: 0.066254, Eval Acc: 0.980123\n",
      "epoch: 10, Train Loss: 0.024885, Train Acc: 0.992237, Eval Loss: 0.067818, Eval Acc: 0.979727\n",
      "epoch: 11, Train Loss: 0.020706, Train Acc: 0.993237, Eval Loss: 0.174131, Eval Acc: 0.958070\n",
      "epoch: 12, Train Loss: 0.019527, Train Acc: 0.993553, Eval Loss: 0.066838, Eval Acc: 0.982199\n",
      "epoch: 13, Train Loss: 0.016248, Train Acc: 0.994620, Eval Loss: 0.080457, Eval Acc: 0.978738\n",
      "epoch: 14, Train Loss: 0.017617, Train Acc: 0.994603, Eval Loss: 0.064320, Eval Acc: 0.982496\n",
      "epoch: 15, Train Loss: 0.012970, Train Acc: 0.995985, Eval Loss: 0.079791, Eval Acc: 0.979925\n",
      "epoch: 16, Train Loss: 0.012162, Train Acc: 0.995736, Eval Loss: 0.083829, Eval Acc: 0.979727\n",
      "epoch: 17, Train Loss: 0.011916, Train Acc: 0.996185, Eval Loss: 0.079493, Eval Acc: 0.981507\n",
      "epoch: 18, Train Loss: 0.008972, Train Acc: 0.997385, Eval Loss: 0.074135, Eval Acc: 0.981507\n",
      "epoch: 19, Train Loss: 0.008857, Train Acc: 0.997018, Eval Loss: 0.074056, Eval Acc: 0.983188\n"
     ]
    }
   ],
   "source": [
    "# 开始训练\n",
    "losses = []\n",
    "acces = []\n",
    "eval_losses = []\n",
    "eval_acces = []\n",
    "\n",
    "for e in range(20):\n",
    "    train_loss = 0\n",
    "    train_acc = 0\n",
    "    net.train()\n",
    "    for im, label in train_data:\n",
    "        im = Variable(im)\n",
    "        label = Variable(label)\n",
    "        # 前向传播\n",
    "        out = net(im)\n",
    "        loss = criterion(out, label)\n",
    "        # 反向传播\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        # 记录误差\n",
    "        train_loss += loss.data[0]\n",
    "        # 计算分类的准确率\n",
    "        _, pred = out.max(1)\n",
    "        num_correct = float((pred == label).sum().data[0])\n",
    "        acc = num_correct / im.shape[0]\n",
    "        train_acc += acc\n",
    "        \n",
    "    losses.append(train_loss / len(train_data))\n",
    "    acces.append(train_acc / len(train_data))\n",
    "    # 在测试集上检验效果\n",
    "    eval_loss = 0\n",
    "    eval_acc = 0\n",
    "    net.eval() # 将模型改为预测模式\n",
    "    for im, label in test_data:\n",
    "        im = Variable(im)\n",
    "        label = Variable(label)\n",
    "        out = net(im)\n",
    "        loss = criterion(out, label)\n",
    "        # 记录误差\n",
    "        eval_loss += loss.data[0]\n",
    "        # 记录准确率\n",
    "        _, pred = out.max(1)\n",
    "        num_correct = float((pred == label).sum().data[0])\n",
    "        acc = num_correct / im.shape[0]\n",
    "        eval_acc += acc\n",
    "        \n",
    "    eval_losses.append(eval_loss / len(test_data))\n",
    "    eval_acces.append(eval_acc / len(test_data))\n",
    "    print('epoch: {}, Train Loss: {:.6f}, Train Acc: {:.6f}, Eval Loss: {:.6f}, Eval Acc: {:.6f}'\n",
    "          .format(e, train_loss / len(train_data), train_acc / len(train_data), \n",
    "                     eval_loss / len(test_data), eval_acc / len(test_data)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画出 loss 曲线和 准确率曲线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fa44b7aa588>]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8VfWd//HXJzuQEEIStrCEzYVNwKgYLC60CnYqWrXF2nGprbXWLtPpWDvtr7XOTKe21a6OVatttbVqtbS0YtHWKq0IEkCBiGAIWyJLCFtCCNk+vz/uQa8xIReynCT3/Xw88shZvveeTw6X9/3e7zn3HHN3REQkPiSEXYCIiHQdhb6ISBxR6IuIxBGFvohIHFHoi4jEEYW+iEgcUeiLAGb2MzP7fyf42BfM7JMdXZNIZ0gKuwCR9jKzLcAn3f2vJ/oc7n5Tx1Uk0n2ppy+9npmpcyMSUOhLj2ZmjwAjgT+ZWbWZ3Wpm+WbmZnaDmW0Dng/a/s7MdprZATNbYmYTo57nl2b238H0eWZWZmb/bma7zWyHmV0fYz0JZvZ1M9saPPZhM8sM1qWZ2a/NrNLM9pvZCjMbHKy7zsxKzazKzDab2dUdvKtEAIW+9HDu/q/ANuBD7p7u7t+NWn0ucCpwUTD/DDAeGASsAn5zjKceAmQCecANwD1mlhVDSdcFP+cDY4B04KfBumuD5xwBZAM3AYfNrB/wY2Cuu2cAhcCrMWxL5Lgp9KU3u93dD7n7YQB3f8jdq9z9CHA7cNrRXngL6oE73L3e3RcB1cDJMWzzauBudy9192rgq8D8YIipnkjYj3P3Rndf6e4Hg8c1AZPMrI+773D34hP9o0WORaEvvdn2oxNmlmhm3zGzTWZ2ENgSrMpp5bGV7t4QNV9DpNfelmHA1qj5rUROmBgMPAIsBh4zs7fM7Ltmluzuh4CPEun57zCzp83slBi2JXLcFPrSG7R2qdjo5R8D5gHvJzLEkh8stw6u5S1gVNT8SKAB2BV8aviWu08gMoTzL8A1AO6+2N0/AAwF3gAe6OC6RACFvvQOu4iMnx9LBnAEqAT6At/upFp+C/ybmY02s/RgO4+7e4OZnW9mk80sEThIZLinycwGm9m8YGz/CJGhpKZOqk/inEJfeoP/Bb4enBHz5VbaPExkqKUceB1Y1km1PERkGGcJsBmoBT4XrBsCPEkk8NcDLwZtE4AvEfmUsJfIAejPdFJ9EudMN1EREYkf6umLiMQRhb6ISBxR6IuIxBGFvohIHOl2F6LKycnx/Pz8sMsQEelRVq5cucfdc9tq1+1CPz8/n6KiorDLEBHpUcxsa9utNLwjIhJXFPoiInFEoS8iEkcU+iIicUShLyISRxT6IiJxRKEvIhJHek3o76+p40d/fZN15QfCLkVEpNvqdl/OOlEJCcaP/raRxqYmJuW1dttTEZH41mt6+v3TkpkyfAAvbaoMuxQRkW6r14Q+wMxx2by2fT/VRxrabiwiEod6V+iPzaGhyXlls3r7IiIt6VWhP31UFilJCbxUotAXEWlJTKFvZnPMbIOZlZjZbS2sn2Vmq8yswcyuaLZupJk9a2brzex1M8vvmNLfKy05kYJRWbxUsqezNiEi0qO1GfpmlgjcA8wFJgBXmdmEZs22AdcBj7bwFA8D33P3U4Ezgd3tKbgtM8fl8MbOKiqrj3TmZkREeqRYevpnAiXuXurudcBjwLzoBu6+xd3XAE3Ry4M3hyR3fy5oV+3uNR1TessKx2YD8HKphnhERJqLJfTzgO1R82XBslicBOw3s9+b2Woz+17wyaHTTM7LJCM1SeP6IiIt6OwDuUnA+4AvA2cAY4gMA72Lmd1oZkVmVlRRUdG+DSYmcNaYbJZu0ri+iEhzsYR+OTAian54sCwWZcCrwdBQA/AHYHrzRu5+v7sXuHtBbm6bt3hsU+HYbLZW1lC2r1NHkkREepxYQn8FMN7MRptZCjAfWBjj868ABpjZ0SS/AHj9+Ms8PjPH5QCwVEM8IiLv0mboBz30W4DFwHrgCXcvNrM7zOwSADM7w8zKgCuB+8ysOHhsI5Ghnb+Z2VrAgAc65095x0mD08lJT+UlDfGIiLxLTBdcc/dFwKJmy74RNb2CyLBPS499DpjSjhqPm5lRODabpZsqcXfMrCs3LyLSbfWqb+RGmzkum4qqI5Tsrg67FBGRbqPXhn7h2Mi4vr6dKyLyjl4b+iMG9mXEwD661LKISJReG/oQuermstJKGhqb2m4sIhIHenXoF47Loaq2gXVvHQy7FBGRbqF3h35wHR6N64uIRPTq0M9JT+WUIRm8rHF9ERGgl4c+RM7iWbFlL7X1jWGXIiISujgI/WyONDSxatu+sEsREQldrw/9s8YMJDHBdB0eERHiIPQz0pKZMjxT1+ERESEOQh8i5+uvKTtAVW192KWIiIQqLkK/cFw2jU3OK5v3hl2KiEio4iL0p4/MIjUpQbdQFJG4Fxehn5acSEF+lm6hKCJxLy5CHyLn67+xs4o91UfCLkVEJDRxE/pv30JR384VkTgWU+ib2Rwz22BmJWZ2WwvrZ5nZKjNrMLMrWljf38zKzOynHVH0iZicl0lGWhJLdR0eEYljbYa+mSUC9wBzgQnAVWY2oVmzbcB1wKOtPM1/AUtOvMz2S0wwZozJ1vn6IhLXYunpnwmUuHupu9cBjwHzohu4+xZ3XwO858L1ZnY6MBh4tgPqbZeZY7PZvvcw2/fWhF2KiEgoYgn9PGB71HxZsKxNZpYA3AV8uY12N5pZkZkVVVRUxPLUJ6Tw7XF99fZFJD519oHcm4FF7l52rEbufr+7F7h7QW5ubqcVM35QOrkZqTpfX0TiVlIMbcqBEVHzw4NlsTgbeJ+Z3QykAylmVu3u7zkY3BXMjMKx2bxUUom7Y2ZhlCEiEppYevorgPFmNtrMUoD5wMJYntzdr3b3ke6eT2SI5+GwAv+omWNz2FN9hI27qsMsQ0QkFG2Gvrs3ALcAi4H1wBPuXmxmd5jZJQBmdoaZlQFXAveZWXFnFt0eheN0C0URiV+xDO/g7ouARc2WfSNqegWRYZ9jPccvgV8ed4UdbHhWX0Zl92Xppj184pzRYZcjItKl4uYbudEKx2azvHQvDY3vOcNURKRXi9PQz6HqSANryw+EXYqISJeK09CPjOvrOjwiEm/iMvSz01M5ZUiGDuaKSNyJy9CHyFU3i7buo7a+MexSRES6TByHfjZ1DU2s3Lov7FJERLpM3Ib+GfkDSUwwDfGISFyJ29DPSEvmtOGZOpgrInElbkMfIuP6a8r2c7C2PuxSRES6RFyHfuHYHJoclpfuDbsUEZEuEdehP33UANKSEzSuLyJxI65DPzUpkTPyB+qmKiISN+I69CEyxLNxVzW7q2rDLkVEpNMp9INLMryss3hEJA7EfehPysukf1oSS3ULRRGJA3Ef+okJxowx2bykcX0RiQMxhb6ZzTGzDWZWYmbvud2hmc0ys1Vm1mBmV0Qtn2pmL5tZsZmtMbOPdmTxHWXmuBzK9h1mW2VN2KWIiHSqNkPfzBKBe4C5wATgKjOb0KzZNuA64NFmy2uAa9x9IjAH+KGZDWhv0R1t5tFbKKq3LyK9XCw9/TOBEncvdfc64DFgXnQDd9/i7muApmbLN7r7m8H0W8BuILdDKu9AY3PTGZSRqvP1RaTXiyX084DtUfNlwbLjYmZnAinAphbW3WhmRWZWVFFRcbxP3W5mRuHYbF7eVElTk3f59kVEukqXHMg1s6HAI8D17v6eG9O6+/3uXuDuBbm54XwQKByXQ+WhOjbsqgpl+yIiXSGW0C8HRkTNDw+WxcTM+gNPA19z92XHV17XmTkuB4Bni3eFXImISOeJJfRXAOPNbLSZpQDzgYWxPHnQfgHwsLs/eeJldr68AX244JRB/GLpZqp01U0R6aXaDH13bwBuARYD64En3L3YzO4ws0sAzOwMMysDrgTuM7Pi4OEfAWYB15nZq8HP1E75SzrAF2aPZ39NPQ+/vDXsUkREOoW5d68DlwUFBV5UVBTa9j/xyxWs2raPf9x6PhlpyaHVISJyPMxspbsXtNUu7r+R25x6+yLSmyn0mzltxAAuOGUQD/yjVGP7ItLrKPRboN6+iPRWCv0WnDZiAOefnMsD/yil+khD2OWIiHQYhX4rvvD+k9hfU8+vlm4JuxQRkQ6j0G/FVPX2RaQXUugfg3r7ItLbKPSPQb19EeltFPptONrbf/jlLWGXIiLSbgr9NkwdMYDzTs7lgSXq7YtIz6fQj8EXZo9nn3r7ItILKPRjMG1klnr7ItIrKPRjpN6+iPQGCv0YRff2D6m3LyI9lEL/OLzT29c1eUSkZ1LoH4ejvf37l2xSb19EeqSYQt/M5pjZBjMrMbPbWlg/y8xWmVmDmV3RbN21ZvZm8HNtRxUeFvX2RaQnazP0zSwRuAeYC0wArjKzCc2abQOuAx5t9tiBwDeBs4AzgW+aWVb7yw7PtJFZnHuSevsi0jPF0tM/Eyhx91J3rwMeA+ZFN3D3Le6+Bmhq9tiLgOfcfa+77wOeA+Z0QN2h+sL71dsXkZ4pltDPA7ZHzZcFy2IR02PN7EYzKzKzooqKihifOjzT1dsXkR6qWxzIdff73b3A3Qtyc3PDLicmR3v7jyxTb19Eeo5YQr8cGBE1PzxYFov2PLZbe6e3r/P2RaTniCX0VwDjzWy0maUA84GFMT7/YuBCM8sKDuBeGCzrFb7w/vHsPVSn3r6I9Bhthr67NwC3EAnr9cAT7l5sZneY2SUAZnaGmZUBVwL3mVlx8Ni9wH8ReeNYAdwRLOsVpo/MYpZ6+yLSg5i7h13DuxQUFHhRUVHYZcRs5dZ9XH7vUm6bewo3nTs27HJEJE6Z2Up3L2irXbc4kNuTnT5KvX0R6TkU+h3gC7MjY/u/1ti+iHRzCv0OcLS3f9+SUmrq1NsXke5Lod9Bjvb2/+/vm8IuRUSkVQr9DnL6qCwunz6c/3uhhGWllWGXIyLSIoV+B/rWvImMHNiXf3v8VfbX1IVdjojIeyj0O1B6ahI/uWo6e6qP8JWn1tDdTocVEVHod7DJwzP5j4tOZnHxLn6zfFvY5YiIvItCvxN88pwxvG98Dv/159fZuKsq7HJERN6m0O8ECQnGXR85jYy0JD736Gpq6xvDLklEBFDod5pBGWl878rT2LCrim8vWh92OSIigEK/U51/8iBuOGc0D7+8lWeLd4ZdjoiIQr+z3TrnZCYO68+tT61h54HasMsRkTin0O9kqUmJ/PiqaRypb+LfHn+Vxiadxiki4VHod4Gxuel865KJvFxayc9e1GUaRCQ8Cv0ucmXBcP5lylDufm4jq7btC7scEYlTMYW+mc0xsw1mVmJmt7WwPtXMHg/WLzez/GB5spn9yszWmtl6M/tqx5bfc5gZ/3PZZIb0T+Pzv13Nwdr6sEsSkTjUZuibWSJwDzAXmABcZWYTmjW7Adjn7uOAHwB3BsuvBFLdfTJwOvDpo28I8SizTzI/vmoqOw7U8rUF63SZBhHpcrH09M8ESty91N3rgMeAec3azAN+FUw/Ccw2MwMc6GdmSUAfoA442CGV91CnjxrIF2eP50+vvcWTK8vCLkdE4kwsoZ8HbI+aLwuWtdgmuJH6ASCbyBvAIWAHsA34fks3RjezG82syMyKKioqjvuP6GluPn8cZ40eyDcXFlNaUR12OSISRzr7QO6ZQCMwDBgN/LuZjWneyN3vd/cCdy/Izc3t5JLCl5hg/HD+VFKSEvj8Y6s50qDLNIhI14gl9MuBEVHzw4NlLbYJhnIygUrgY8Bf3L3e3XcDLwFt3q09HgzN7MOdl09hXflBvr94Q9jliEiciCX0VwDjzWy0maUA84GFzdosBK4Npq8AnvfIUcptwAUAZtYPmAG80RGF9wYXTRzCx2eM5IF/bOaFDbvDLkdE4kCboR+M0d8CLAbWA0+4e7GZ3WFmlwTNHgSyzawE+BJw9LTOe4B0Mysm8ubxC3df09F/RE/29Q9O4KTB6Xz5d69RUXUk7HJEpJez7nbaYEFBgRcVFYVdRpfasLOKS376T84ak80vrzuDhAQLuyQR6WHMbKW7tzl8rm/kdgMnD8ng6x88lSUbK/jOX97Q+fsi0mmSwi5AIj4+YxTrd1Zx/5JS9tfU8e3LJpOUqPdkEelYCv1uwsz4n0snkd0vhZ88X8L+mnp+fNU00pITwy5NRHoRdSW7ETPj3y88mW9+aALPvr6Lax96RdfoEZEOpdDvhq6fOZofzZ/Kyq37+Oh9y9hdpZuviEjHUOh3U/Om5vHzawvYsucQV9z7MlsrD4Vdkoj0Agr9buy8kwfx6KfO4mBtPZff+zLFbx0IuyQR6eEU+t3ctJFZPHnT2SQnGvPvW8ay0sqwSxKRHkyh3wOMG5TBk58pZFD/VK556BWeLd4Zdkki0kMp9HuIvAF9+N1NhZw6tD83/XolT6zY3vaDRESaUej3IAP7pfDoJ89i5rgcbn1qDT97cZO+vSsix0Wh38P0S03iwWvP4EOnDeM7z7zBtxetp6lJwS8isdE3cnuglKQEfvTRqQzsm8wD/9hM5aE67rx8Csm6bIOItEGh30MlJBi3XzKRnPRU7npuI/tr6rnnY9Ppk6LLNohI69Q17MHMjM/NHs9/XzqJv2/YzccfXM7+mrqwyxKRbkyh3wt8fMYo7vnYdNaWHeCy/1tKye6qsEsSkW4qptA3szlmtsHMSszsthbWp5rZ48H65WaWH7Vuipm9bGbFZrbWzNI6rnw56uLJQ3n0U2dRVVvPpfcs5W/rd4Vdkoh0Q22GvpklErnt4VxgAnCVmU1o1uwGYJ+7jwN+ANwZPDYJ+DVwk7tPBM4DdNnITlKQP5CFt5xDfk5fPvlwEff8vUSndIrIu8TS0z8TKHH3UnevAx4D5jVrMw/4VTD9JDDbzAy4EFjj7q8BuHuluzd2TOnSkmED+vC7TxfyoSnD+N7iDXzut6s5XKddLiIRsYR+HhD99c+yYFmLbYIbqR8AsoGTADezxWa2ysxubWkDZnajmRWZWVFFRcXx/g3STJ+URH40fypfmXMKT6/dwRU/W0r5/sNhlyUi3UBnH8hNAs4Brg5+X2Zms5s3cvf73b3A3Qtyc3M7uaT4YGZ85ryxPHhtAdsqa7jkJ//klc17wy5LREIWS+iXAyOi5ocHy1psE4zjZwKVRD4VLHH3Pe5eAywCpre3aIndBacMZsFnZ5LZJ5mrf76MR5dvC7skEQlRLKG/AhhvZqPNLAWYDyxs1mYhcG0wfQXwvEeOIC4GJptZ3+DN4Fzg9Y4pXWI1blA6Cz47k8KxOfzngrX8vz+so76xKeyyRCQEbYZ+MEZ/C5EAXw884e7FZnaHmV0SNHsQyDazEuBLwG3BY/cBdxN543gVWOXuT3f8nyFtyeyTzEPXncGnZ43hkWVb+fjPl1NZfSTsskSki1l3O6WvoKDAi4qKwi6jV/vD6nK+8tQactJTeeCaAiYM6x92SSLSTma20t0L2mqnb+TGoUun5fG7m86mscm5/N6lLFq7I+ySRKSLKPTj1JThA1h4y0xOHZrBzb9Zxd3PbtAlmkXigEI/jg3qn8Zvb5zBRwqG8+PnS7jxkZUcrNUXpkV6M4V+nEtNSuTOy6fwzQ9N4O8bdnPB91/g0eXbaNDZPSK9kkJfMDOunzmaBTcXMjqnH/+5YC0f/PE/WbJR344W6W0U+vK2KcMH8MSnz+beq6dzuL6Rax56hWsfeoWNu3SpZpHeQqEv72JmzJ08lOe+NIuvXXwqq7btY84Pl/C1BWvZo/P6RXo8hb60KDUpkU/NGsOL/3E+15ydz2MrtnPe917g3hc2UVuvq3aK9FQKfTmmgf1SuP2SiSz+4ixmjBnInX95g9l3vcjC197StfpFeiCFvsRk3KB0fn7tGTz6ybPo3yeZz/92NR++dykrt+4LuzQROQ4KfTkuheNy+PPnzuG7V0yhfN9hLr93Kbc8uorte2vCLk1EYqBr78gJO3SkgfuWlHL/kk00NcH15+Rz83njyOyTHHZpInEn1mvvKPSl3XYcOMz3F2/kqVVl9E9L4lPvG8P154wmPTUp7NJE4oZCX7pc8VsH+MFzG/nr+t1k9U3mxlljubZwFH1TFP4inU2hL6F5bft+7n5uIy9urCAnPYWbzh3Lx2eMIi05MezSRHothb6EbuXWvdz93EZeKqlkUEYqnz1/HPPPHEFqksJfpKN16PX0zWyOmW0wsxIzu62F9alm9niwfrmZ5TdbP9LMqs3sy7H+AdLznT5qIL/55Aweu3EG+dn9+ObCYs773gv8ZvlW6hp0QTeRMLQZ+maWCNwDzAUmAFeZ2YRmzW4A9rn7OOAHwJ3N1t8NPNP+cqUnmjEmm8c/PYNf33AWQzLT+NqCdVxw1ws8sWK7ruYp0sVi6emfCZS4e6m71wGPAfOatZkH/CqYfhKYbWYGYGaXApuB4o4pWXoiM+Oc8Tn8/jOF/OL6MxjYL4Vbn1rD++9+kd+vKqNRN3AR6RKxhH4esD1qvixY1mKb4EbqB4jcKD0d+ArwrWNtwMxuNLMiMyuqqNDlfHszM+P8kwfxx8/O5IFrCuiTksSXnniNC38QubSD7t4l0rk6+xu5twM/cPfqYzVy9/vdvcDdC3Jzczu5JOkOzIwPTBjM0587h3uvnk5igvH5367mgrte4Bcvbab6SEPYJYr0SrGcQF0OjIiaHx4sa6lNmZklAZlAJXAWcIWZfRcYADSZWa27/7TdlUuvkJAQuZTzhROH8My6HTz4z81860+vc9ezG7myYDjXFeYzKrtf2GWK9BptnrIZhPhGYDaRcF8BfMzdi6PafBaY7O43mdl84MPu/pFmz3M7UO3u3z/W9nTKpry6fT+/eGkzT6/ZQaM7s08ZxPUzR1M4NpvgUJGINBPrKZtt9vTdvcHMbgEWA4nAQ+5ebGZ3AEXuvhB4EHjEzEqAvcD89pUv8WzqiAH8aP40/vPiU/n1sq08unwbf12/nJMHZ3DdzHwunZpHnxSd6y9yIvTlLOn2ausbWfjaW/zipS2s33GQAX2TuerMkfzrjFEMG9An7PJEugV9I1d6HXfnlc17+cVLW3j29Z2YGXMmDeETM/OZPjJLQz8S1zpseEekuzAzzhqTzVljstm+t4ZHlm3lsVe28fSaHUwZnsn1M/OZO2morvEjcgzq6UuPVlPXwFOryvnlS5vZVHGItOQEZozJ5tyTcjn3pFxG5/TTJwCJCxrekbjS1OS8XFrJc6/vYsnGCkr3HAJgeFaft98ACsfl6Br/0msp9CWubaus4cU3K1iysYKlJXs4VNdIUoJRkJ/FuScN4tyTcjl1aIY+BUivodAXCdQ1NLFy6z5e3FjBixsrWL/jIAC5GanMGp/LuSfn8r5xOWT1Swm5UpETp9AXacWug7Us2VjBkjf38I83K9hfU48ZTBk+gIsmDmbe1DzydCqo9DAKfZEYNDY5a8r28+LGCl7YUMGr2/cDcNbogVw2LY+5k4fqRu/SIyj0RU7Atsoa/vhqOQtWl1O65xApiQnMPnUQl07L47yTc3XXL+m2FPoi7eDurC0/wILV5fzptbfYU11HZp9kPjhlKJdOzaNgVBYJCToILN2HQl+kgzQ0NvHPkj38YXU5i4t3cbi+keFZfbh0ah6XThvGuEEZYZcootAX6QyHjjTw3Ou7WLC6nH+8WUGTw+S8TOZNHcYlpw1jUP+0sEuUOKXQF+lku6tq+fNrO/jDq+WsKTtAgsHZY7OZM2koF00czKAMvQFI11Hoi3Shkt3V/PHVcp5eu4PSikOYwRn5A5k7aQhzJg1haKZOAZXOpdAXCYG78+buahat3cEza3eyYVcVANNGDuDiSUOZM2kIIwb2DblK6Y0U+iLdwKaKav6ybieL1u6g+K3IN4En52Uyd/IQ5k4ayugc3QpSOkaHhr6ZzQF+ROTOWT939+80W58KPAycTuTeuB919y1m9gHgO0AKUAf8h7s/f6xtKfSlt9pWWcMz63awaN1OXgu+BHbKkAwunjyUuZOGMH6wzgKSE9dhoW9miUTukfsBoIzIPXKvcvfXo9rcDEyJukfuZe7+UTObBuxy97fMbBKw2N3zjrU9hb7Eg/L9h/nLup08s3YHRVv3ATBuUDpTRwwgNyOV3PRUcoLfuRkp5Kan0b9Pki4QJ63qyNA/G7jd3S8K5r8K4O7/G9VmcdDm5eBG6juBXI96cou8WiuBoe5+pLXtKfQl3uw6WMvi4p38Zd1ONu85xJ7qI9Q3vvf/ZUpiAjnpKeRmpJKTnhp5c4iazklPZfygdF04Lk515J2z8oDtUfNlwFmttQlupH4AyAb2RLW5HFjVUuCb2Y3AjQAjR46MoSSR3mNw/zSuOTufa87OByIHgw8crqei6kjkpzrye0913dvzOw7Usqb8AJXVR2hq9v5w6tD+FI7NpnBsNmeMHkj/NF07SN7RJXeUMLOJwJ3AhS2td/f7gfsh0tPvippEuiszY0DfFAb0TWlznL+xydlXE3kz2F11hDXb9/NyaSWPLNvKg//cTILB5OEDKBybzdljsinIz6Jvim4kE89i+dcvB0ZEzQ8PlrXUpiwY3skkMpSDmQ0HFgDXuPumdlcsIm9LTDBy0iNDO6cOhXNPyuVzs8dTW9/Iqm37WLapkqWbKnlgSSn3vrCJ5ERj6ogBnD02h8Kx2UwbOeC4LiJX19DE7qpadh6oZefB4PeBWnYcrGX3wVqGDejDpdPyeN+4HJISEzrxL5cTFcuYfhKRA7mziYT7CuBj7l4c1eazwOSoA7kfdvePmNkA4EXgW+7++1gK0pi+SMc7dKSBoq37WLppD8s2VbK2/ABNDqlJCRTkZ1E4NocZY7LJ7JMcFeiH3wn2g7XsPHCEPdXvPRyXlpzAkP5pDOqfxsZdVeyvqScnPZVLThvGh6fnMXFYfx2A7gIdfcrmxcAPiZyy+ZC7/4+Z3QEUuftCM0sDHgGmAXuB+e5eamZfB74KvBn1dBe6++7WtqXQF+l8Bw7Xs2LzXpZuqmTppj28sbOqxXZZfZMZ3D+NIZlpDM1Mi0wH80My0xjav8+7ziqqa2ji7xvK+irTAAAHzklEQVR2s2BVOc+/sZu6xibGD0rnsul5XDo1j2G6OU2n0ZezRCRmew/Vsby0krrGJgb3fyfg05JP/P4B+2vqeHrtDhasKqdo6z7MYMbobC6bnsfcSUPI0AHmDqXQF5FuY1tlDQtWl7NgdRlbKmtITUrgwolD+PC0PN43XuP/HUGhLyLdjruzevt+Fqwq509r3grG/1P40GnD+PC04UzK0/j/iVLoi0i3dnT8/w+ry/nb+sj4f5/kRDL7JNO/TxL905Lp3yc5Mp+WRP8+ycGyyLpIu3eWZaQlkxjHdzPryC9niYh0uJSkBC6aOISLJg7hQE09i9btYNPuag7W1nPwcAMHDtez62Atb+6u4uDhBg7W1tNWHzUlKYHkBCMpMYHkRCMpIYGkRCM5MYGkdy1/d5ujv4dkpnH6qCwK8rN67eWw1dMXkR6hqck5VBd5Mzj6JnDwcD0Haxs4eLieA4frqa1vpL7RaWxqor7JaWhsoqHR356ub3QamiLLjv6Obrdtbw2H6xsByBvQ5+03gNNHZXHKkP7d+pOEevoi0qskJBgZacmRs36yOmcbDY1NrN9RRdHWvRRt3ccrm/ey8LW3AOiXksi0kVlvvxFMG5lFemrPi1D19EVEWuHulO8/zMqt+yjaso+irft4Y+dB3CHB4JQh/d/+JFCQP5C8EL+HoAO5IiKdoKq2ntXb9lO0dR8rt+5l9bb91NRFhoSSE40EO/pD5HfCO9NmRmIC77SJmjaDicMy+clV006oLg3viIh0goy0ZGadlMusk3KByJDQGzurKNqyl11VR2hyxz1yMbyj003uwXzk00Nk/p3pJodGd0YO7PxPCgp9EZF2SEpMYFJeJpPyMsMuJSb6GpyISBxR6IuIxBGFvohIHFHoi4jEEYW+iEgcUeiLiMQRhb6ISBxR6IuIxJFudxkGM6sAtrbjKXKAPR1UTmdQfe2j+tpH9bVPd65vlLvnttWo24V+e5lZUSzXnwiL6msf1dc+qq99unt9sdDwjohIHFHoi4jEkd4Y+veHXUAbVF/7qL72UX3t093ra1OvG9MXEZHW9caevoiItEKhLyISR3pk6JvZHDPbYGYlZnZbC+tTzezxYP1yM8vvwtpGmNnfzex1Mys2sy+00OY8MztgZq8GP9/oqvqiathiZmuD7b/n/pQW8eNgH64xs+ldWNvJUfvmVTM7aGZfbNamS/ehmT1kZrvNbF3UsoFm9pyZvRn8bvF23WZ2bdDmTTO7tgvr+56ZvRH8+y0wswGtPPaYr4VOrO92MyuP+je8uJXHHvP/eyfW93hUbVvM7NVWHtvp+69DuXuP+gESgU3AGCAFeA2Y0KzNzcDPgun5wONdWN9QYHownQFsbKG+84A/h7wftwA5x1h/MfAMYMAMYHmI/947iXzxJLR9CMwCpgPropZ9F7gtmL4NuLOFxw0ESoPfWcF0VhfVdyGQFEzf2VJ9sbwWOrG+24Evx/Dvf8z/751VX7P1dwHfCGv/deRPT+zpnwmUuHupu9cBjwHzmrWZB/wqmH4SmG1m1hXFufsOd18VTFcB64G8rth2B5sHPOwRy4ABZjY0hDpmA5vcvT3f0m43d18C7G22OPp19ivg0hYeehHwnLvvdfd9wHPAnK6oz92fdfeGYHYZMLyjtxurVvZfLGL5/95ux6ovyI6PAL/t6O2GoSeGfh6wPWq+jPeG6tttghf9ASC7S6qLEgwrTQOWt7D6bDN7zcyeMbOJXVpYhAPPmtlKM7uxhfWx7OeuMJ/W/7OFvQ8Hu/uOYHonMLiFNt1lP36CyCe3lrT1WuhMtwTDTw+1MjzWHfbf+4Bd7v5mK+vD3H/HrSeGfo9gZunAU8AX3f1gs9WriAxXnAb8BPhDV9cHnOPu04G5wGfNbFYINRyTmaUAlwC/a2F1d9iHb/PI5/xuef6zmX0NaAB+00qTsF4L9wJjganADiJDKN3RVRy7l9/t/y9F64mhXw6MiJofHixrsY2ZJQGZQGWXVBfZZjKRwP+Nu/+++Xp3P+ju1cH0IiDZzHK6qr5gu+XB793AAiIfo6PFsp8721xglbvvar6iO+xDYNfRIa/g9+4W2oS6H83sOuBfgKuDN6b3iOG10CncfZe7N7p7E/BAK9sNe/8lAR8GHm+tTVj770T1xNBfAYw3s9FBT3A+sLBZm4XA0bMkrgCeb+0F39GC8b8HgfXufncrbYYcPcZgZmcS+XfoyjelfmaWcXSayAG/dc2aLQSuCc7imQEciBrK6Cqt9rDC3oeB6NfZtcAfW2izGLjQzLKC4YsLg2WdzszmALcCl7h7TSttYnktdFZ90ceILmtlu7H8f+9M7wfecPeyllaGuf9OWNhHkk/kh8iZJRuJHNX/WrDsDiIvboA0IkMCJcArwJgurO0cIh/z1wCvBj8XAzcBNwVtbgGKiZyJsAwo7OL9NybY9mtBHUf3YXSNBtwT7OO1QEEX19iPSIhnRi0LbR8SefPZAdQTGVe+gchxor8BbwJ/BQYGbQuAn0c99hPBa7EEuL4L6yshMh5+9HV49Iy2YcCiY70Wuqi+R4LX1hoiQT60eX3B/Hv+v3dFfcHyXx59zUW17fL915E/ugyDiEgc6YnDOyIicoIU+iIicUShLyISRxT6IiJxRKEvIhJHFPoiInFEoS8iEkf+P5l97Mx45o99AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title('train loss')\n",
    "plt.plot(np.arange(len(losses)), losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'train acc')"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xt8VPWd//HXh1whhEASruF+FagoCHgDwUsRe9Eqtqu9WbXr9mK77dbtw253bdf+uq7Vrlu7tn3Y1nqprbfWilYrglpBC3JHLkICIiRckgAhCZD75/fHnNgxJjCQSc4k834+HvOYM+d8z8xnTibvOfM93zlj7o6IiCSHHmEXICIinUehLyKSRBT6IiJJRKEvIpJEFPoiIklEoS8ikkQU+iJRzOwXZvYfYdch0lFM4/SluzCzncAX3X1x2LWIJCrt6UvSMLPUsGsQCZtCX7oFM3sEGA48a2bVZvZtMxtpZm5mN5rZLuDloO2TZrbPzA6b2WtmNjnqfh40s/8XTM81s2Iz+5aZlZrZXjO7/jg1XG9mW8ysysx2mNk/tVh+hZmtM7NKM9tuZvOD+blm9hsz22Nmh8zsTx2wiUQAhb50E+7+OWAX8HF37+3uP4paPAeYCFwa3H4BGAcMANYAjx7nrgcBOUABcCNwn5n1a6NtKfAxoA9wPXCPmU0DMLOZwMPAvwJ9gQuAncF6jwC9gMlBTffE9KRFToE+7koy+L67H2m+4e4PNE+b2feBQ2aW4+6HW1m3Hrjd3RuA582sGpgALG/Z0N3/HHXzr2a2CJhN5I3lRuABd38pWF4SPP5g4DIgz90PNa97ak9T5MS0py/JYHfzhJmlmNl/B90rlfx9bzu/jXUPBIHf7CjQu7WGZnaZmS03s4NmVgF8JOp+hwHbW1ltGHAwKvBFOpRCX7qTtoaiRc//NHAFcAmRbpuRwXxrzwObWQbwB+BuYKC79wWej7rf3cCYVlbdDeSaWd/2PL5IrBT60p3sB0afoE02UAscINKP/l9xeux0IAMoAxrM7DJgXtTyXwPXm9nFZtbDzArM7DR330vkGMPPzKyfmaWZ2QVxqknkAxT60p3cAfy7mVWY2S1ttHkYeJdIn/pmWumbPxXuXgV8HXgCOETkE8XCqOVvEhzcBQ4T6bcfESz+HJFjB28TORj8jXjUJNIafTlLRCSJaE9fRCSJKPRFRJKIQl9EJIko9EVEkkjCfSM3Pz/fR44cGXYZIiJdyurVq8vdvf+J2iVc6I8cOZJVq1aFXYaISJdiZu/G0k7dOyIiSUShLyKSRBT6IiJJRKEvIpJEFPoiIklEoS8ikkQU+iIiSSThxumLiHQ1pZU1vLK1lANH6sjvnUH/7Az6B9e5WemkpSTO/rVCX0TkJLk7m/ZUsmRLKUve3s+G4tZ+XvnvcrPSye+dTv/sDPJ7Z7z3xvD368iy3F7ppHbwG4RCX0QkBjX1jbyxvZzFW0p5eUsp+yprMIMzh/XllnnjuXjiQEbk9aK8qo6y6lrKqmopr45cmqfLqmpZu6uCsqpajtU3fuAxPlTQh+e+NrtDn4dCX0SkDaWVNSx5u5QlW/azrKicmvomeqWncMG4/lw0cQAXThhA/+yM960zPC+V4Xm9TnjfR2ob3v+GUF1HVnpKRz2V9yj0RUQCzd02i7fsZ8mWUt4qiXTbFPTtyT9MH8ZFEwdyzuhcMlLbH85ZGalkZaQyIi+r3fd1MhT6IpJUmpqc8upa9hyuYW/FMfYermHv4WPsOVzD6p2H3uu2mTqsL/966QQunjiACQOzMbOwS48Lhb6IhGpPxTGe27CHP2/YS3l1HdmZqfTpmUafzDT69EwNrtPo08r8nOB278xUUnoY7s6BI3Xsrahhz+FjkVCvrGFvRRDsFTXsr6yhoen9vw2ekdqDIX17MnV4Xy46bQAXnjaA/N4ZbVTctSn0RaTTlVfX8vxbe3l2/R5W7jwEwJShOZw9KpfKmgaqauopqTjGlr31VNbUU1XTcML7zM5IpbahibrGpvfNT0/pwaCcTAbnZDJzVC6DcjIZkpPJ4Jyekem+PenXK63b7MmfiEJfRDrF4WP1vLhxH89u2MPrReU0OYwb0JtvfXg8Hz9jCCPz2+7bbmxyqmsbqDwWeROoPNYQXNdTWROZf/hYPRlpPRjSHOY5PRncN5PcXun06JEcgR4Lhb6IdJijdQ0s3lLKwnV7eG1bGXWNTQzP7cWX547h8jMKmDAoO6b7Selh5PSMdOdI+yj0RSSuahsa+evWMhau38OSLaUcq29kYJ8MPnfuCC4/YwhThuYkTVdKIlLoiwillTW8sf0A5dW1mBkGmBFc23vTvG+ZRbWJ3M/KnYd4cdM+qmoa6NcrjaumFXD5GUOYMTJXXSwJQqEvkoSqaxtYseMAy4rKeb2onG37q+Nyv9kZqcybPIjLzxzCeWPyEuqcMxKh0BdJAvWNTazfXcGyonKWFZazbncFDU1ORmoPZo7KZcG0oZw/Np9hub3AwXHcwYl8YSlyHZnPe/Pf366pyRnQJyMuX1ySjqPQF+mG3J3C0mqWFUb25JfvOMCRukbMYEpBDjddMJpZY/OZNqIfmWkK6WSi0Bfp4pqanMPH6jlwpJb1uw9H9uaLyimrqgVgZF4vPjG1gNnj8jlndB59e6WHXLGESaEvkkDcnaN1jRw8Useho3UcPPL3S+R2PQeP1HLoSD0Hj9ZxKJgf/QXT3Kx0zh+bz6yxeZw3JuiyEQko9EVCVFpZw/J3DrJ8xwFW7DhA8aFj1DY0tdo2pYfRr1cauVnp9OuVzrgBvemXlU5ecLtfVhrjB2YzcVAfjZSRNin0RTpRdMgv33GAHWVHAOidkcqMkf24eOJAcrPSye2VTr+sdHKz0sjNivy4RnZmqsJc2k2hL9KB9lfWBAF/kBU7DrCjPBLy2RmpzBiVyzUzhnHO6DwmDe7T4b+YJAIKfZG4Ol7IzxyVyzUzFfISLoW+SDu5O4u3lPKTJdvYWFIJ/D3kr505PBLyQ/qQoq4ZSQAKfZFT5O68XnSAuxZtZf3uCkbm9eLfPnIa547OV8hLwlLoi5yC1e8e5K4Xt7J8x0GG5GRy54LTWTBtqLpsJOEp9EVOwqY9h/nxom28/HYp+b3T+d7HJ/Hps4fr1APSZSj0RWJQVFrNPS9t489v7SWnZxrfnj+BL5w3kl7p+heSrkWvWJHj2H3wKP+7uJCn1xbTMy2Fr180lhtnj9aPeUiXpdAXacX+yhp++nIhj6/cjZlx46xRfGnOGPK66Y9lS/KIKfTNbD7wEyAF+JW7/3eL5SOAB4D+wEHgs+5eHCy7E/ho0PQH7v54nGoXibuDR+r4xV+389AbO2lscv5hxjC+dtE4BuVkhl2aSFycMPTNLAW4D/gwUAysNLOF7r45qtndwMPu/pCZXQTcAXzOzD4KTAPOBDKAV83sBXevjPcTETlV7s7Gkkqee2sPjy7fxdG6Bj4xtYBvXDye4Xk6WZl0L7Hs6c8Eitx9B4CZPQZcAUSH/iTgX4LpV4A/Rc1/zd0bgAYz2wDMB56IQ+0ip6y+sYkVOw6yaPM+Xtq8n72Ha0jpYVw6eSD/8uHxjB0Q2w92i3Q1sYR+AbA76nYxcHaLNuuBq4h0AV0JZJtZXjD/e2b2Y6AXcCHvf7MAwMxuAm4CGD58+Ek+BZHYVNc28Nq2MhZt2sfLb5dSWdNAZloP5ozvzy3zJnDRaQPol6VzzUv3Fq8DubcA/2dmXwBeA0qARndfZGYzgDeAMuBvQGPLld39fuB+gOnTp3vL5SKnqqyqlsVb9rNo0z5e336AuoYmcrPSuXTyIOZNHsSssfn0TNcYe0kesYR+CTAs6vbQYN573H0PkT19zKw3sMDdK4JlPwR+GCz7HbCt/WWLtO2d8iMs2rSPRZv3s2bXIdxhWG5PPn/OCOZNHsRZI/rpFAmStGIJ/ZXAODMbRSTsrwE+Hd3AzPKBg+7eBHyHyEie5oPAfd39gJlNAaYAi+JYvwgAO8qqeXptCX/ZuI/C0moAPlTQh29eMp55kwcyYWA2Zgp6kROGvrs3mNnNwItEhmw+4O6bzOx2YJW7LwTmAneYmRPp3vlqsHoasDT4Z6skMpSzIf5PQ5LR4aP1PLthD39YU8zaXRX0MDhndB6fOXs4H548iIK+PcMuUSThmHtidaFPnz7dV61aFXYZkqDqG5t4bVsZf1hTzOLNpdQ1NjFhYDYLzirgE2cWMKCPxtNLcjKz1e4+/UTt9I1c6RI27TnMH9eU8My6Esqr68jNSucz5wxnwbShTB7SR103IjFS6EvCKquq5Zl1JTy1upi391WRlmJcfNpAFpw1lLkT+pOm0xiLnDSFviSUmvpGFm/Zzx/XlPDXbWU0NjlnDOvLD66YzMemDNE4epF2UuhLQjhQXctPXy7ij2uKqaxpYFCfTG66YDQLphXo27EicaTQl1DVNzbxyN/e5Z7F2zhW18jHpgzm6rOGce6YPI2lF+kACn0JzdLCMm5/djOFpdXMHpfP9z4+SXv1Ih1MoS+dbteBo/zgz5t5afN+huf24pefn84lEwdoBI5IJ1DoS6c5UtvAfa8U8aul75CaYnx7/gRunDVKvy8r0okU+tLh3J1n1u3hjhe2sL+yliunFnDrZacxUF+kEul0Cn3pUG8VH+b7z25i9buHOL0gh5995izOGtEv7LJEkpZCXzpEeXUtd/1lK0+s3k1eVjo/WjCFq88aSg+NyBEJlUJf4qquoYmH/7aTnywu5Fh9I1+cNYqvXTyOPplpYZcmIij0JU7cnVe3lvGDP29mR9kR5ozvz398bBJjB/QOuzQRiaLQl3apa2jiuQ17uP+1Hby9r4qReb349XXTueg0DcEUSUQKfTkllTX1PPbmLh5YtpN9lTWMG9CbH109hSvOHKIhmCIJTKEvJ2VPxTF+8/o7/P7N3VTXNnDemDzuWHA6c8b110FakS5AoS8x2VhymF8t3cFzG/biwEdPH8w/zh7N6UNzwi5NRE6CQl/a5O78dVsZv1y6g9eLDpCVnsJ1543k+vNHMrRfr7DLE5FToNCXD6hraGLh+j388rUdbN1fxcA+Gdx62WlcO3M4OT019FKkK1Poy3sOH6vndyt28eAb77C/spbTBmXz40+ewcfPGEJ6qn6lSqQ7UOgLAGt3HeKGB1dy6Gg9s8bmc9fVZzB7XL6GXYp0Mwp94bVtZXzpt6vJ753BIzeezYcKdHBWpLtS6Ce55zbs4ZuPr2PsgGweumEGA7J15kuR7kyhn8QeWf4utz2zkRkjcvnlddN1kFYkCSj0k5C7c++SIu5ZvI1LJg7g/z49jcw0fYtWJBko9JNMU5Nz+3ObefCNnVw1rYA7F0whLUUjc0SShUI/idQ1NPGvT63nmXV7+OKsUfzbRybq1AkiSUahnySO1jXw5d+u4a/byvj2/Al8ec4YDccUSUIK/SRQcbSOGx5cybrdFdxx1elcO3N42CWJSEgU+t3cvsM1fP6BFewsP8p9n57GZacPDrskEQmRQr8be6f8CJ/91Qoqjtbx4PUzOG9sftgliUjIYhq2YWbzzWyrmRWZ2a2tLB9hZkvMbIOZvWpmQ6OW/cjMNpnZFjO719SR3Ck2lhzm6p+/wbH6Rn5/0zkKfBEBYgh9M0sB7gMuAyYB15rZpBbN7gYedvcpwO3AHcG65wHnA1OADwEzgDlxq15a9bftB7jm/uVkpqXw5JfOZcrQvmGXJCIJIpY9/ZlAkbvvcPc64DHgihZtJgEvB9OvRC13IBNIBzKANGB/e4uWtr24aR/X/eZNBuVk8tSXz2VMf/0wuYj8XSyhXwDsjrpdHMyLth64Kpi+Esg2szx3/xuRN4G9weVFd9/S8gHM7CYzW2Vmq8rKyk72OUjgiVW7+fJvVzNpcB+e/KdzGZzTM+ySRCTBxOurmLcAc8xsLZHumxKg0czGAhOBoUTeKC4ys9ktV3b3+919urtP79+/f5xKSi73v7adbz+1gfPH5vPoF8+mX1Z62CWJSAKKZfROCTAs6vbQYN573H0PwZ6+mfUGFrh7hZn9I7Dc3auDZS8A5wJL41C7EDmPzl0vbuVnr27no1MGc8+nztQPnohIm2JJh5XAODMbZWbpwDXAwugGZpZvZs339R3ggWB6F5FPAKlmlkbkU8AHunfk1DQ2Od/900Z+9up2rp05nHuvmarAF5HjOmFCuHsDcDPwIpHAfsLdN5nZ7WZ2edBsLrDVzLYBA4EfBvOfArYDbxHp91/v7s/G9ykkp7qGJr7+2Fp+t2IXX5k7hv+68kOk6Dw6InIC5u5h1/A+06dP91WrVoVdRkI7WtfAl367hte2lfFvHzmNmy4YE3ZJIhIyM1vt7tNP1E7fyO1iDh+t5/oH32Td7gruXHA6/zBD59ERkdgp9LuQ0soaPv/Am+woO8LPPjON+R/SeXRE5OQo9LuIXQeO8tlfr6C8upYHvjCDWeN0WgUROXkK/S5g674qPvfrFdQ2NPHoF89m6vB+YZckIl2UQj/Brdl1iOt/s5LMtB48+aVzGT8wO+ySRKQLU+gnsKWFZdz08GoG9MngtzeezbDcXmGXJCJdnEI/QT3/1l7++bG1jOnfm4dvnMmA7MywSxKRbkChn4B+/+Yuvvv0W0wb3o9ff2EGOT3Twi5JRLoJhX6C+fmr27nzL28zd0J/fv6Zs+iZnhJ2SSLSjSj0E8idf3mbn7+6nY+fMYQff/IMnUdHROJOoZ8gnly1m5+/up1Pnz2cH1yh8+iISMfQrmQC2La/iv94ZiPnjs5T4ItIh1Loh+xoXQNffXQNvTNS+cm1ZyrwRaRDqXsnZLc9s4mismoeueFsDcsUkQ6nPf0QPblqN0+tLuZrF43TuXREpFMo9ENSuL+K257ZxDmjc/nni8eFXY6IJAmFfgiO1jXwlUfXkJWRwr3XTFU/voh0GvXph+B7QT/+wzfMZEAf9eOLSOfRnn4n+8PqYp5cXczNF45l9rj+YZcjIklGod+Jikqr+Pc/beTsUerHF5FwKPQ7ybG6Rr7y6Bp6padw77VTSU3RpheRzqc+/U7yvYUbKSyt5qHrZzJQ/fgiEhLtbnaCP64p5olVxXx17lguGK9+fBEJj0K/gxWVVvHdpzcyc1Qu37hE/fgiEi6Ffgc6VtfIVx9dS6/0FH6qfnwRSQDq0+9A31+4ia37q3joBvXji0hi0K5nB3l6bTGPr9rNVy8cwxz144tIglDod4Ci0upIP/7IXL55yfiwyxEReY9CP84i/fhryEzTeHwRSTzq04+z/3w20o//4PUzGJSjfnwRSSzaDY2jP60t4bGVu/nK3DHMnTAg7HJERD4gptA3s/lmttXMiszs1laWjzCzJWa2wcxeNbOhwfwLzWxd1KXGzD4R7yeRCMqqavnu028xY2Q//uXD6scXkcR0wtA3sxTgPuAyYBJwrZlNatHsbuBhd58C3A7cAeDur7j7me5+JnARcBRYFMf6E8Z9rxRR09DEj64+Q/34IpKwYkmnmUCRu+9w9zrgMeCKFm0mAS8H06+0shzgauAFdz96qsUmqpKKY/xuxS4+edZQRuVnhV2OiEibYgn9AmB31O3iYF609cBVwfSVQLaZ5bVocw3w+9YewMxuMrNVZraqrKwshpISy72LCwH4uk6XLCIJLl79ELcAc8xsLTAHKAEamxea2WDgdODF1lZ29/vdfbq7T+/fv2t9kemd8iM8taaYz5wznCF9e4ZdjojIccUyZLMEGBZ1e2gw7z3uvodgT9/MegML3L0iqsmngKfdvb595Saee17aRnpKD74yd2zYpYiInFAse/orgXFmNsrM0ol00yyMbmBm+WbWfF/fAR5ocR/X0kbXTle2ZW8lz27Yw/Xnj6R/dkbY5YiInNAJQ9/dG4CbiXTNbAGecPdNZna7mV0eNJsLbDWzbcBA4IfN65vZSCKfFP4a18oTwI8XbaN3Rir/dMGYsEsREYlJTN/IdffngedbzLstavop4Kk21t3JBw/8dnlrdx1i8Zb93DJvPDm90sIuR0QkJhpQfop+vGgbeVnpXH/+qLBLERGJmUL/FPxt+wGWFZXz5bljyMrQ6YtEpOtQ6J8kd+fuRVsZ1CeTz54zIuxyREROikL/JL2ytZTV7x7iaxePJTMtJexyREROikL/JDQ1OXe/uI3hub341PRhJ15BRCTBKPRPwgsb97F5byXf/PA40nRSNRHpgpRcMWpscv7npa2MG9Cby8/odiNQRSRJKPRj9PTaEraXHeFb88aT0sPCLkdE5JQo9GNQ19DE/y7exukFOVw6eVDY5YiInDKFfgweX7mL4kPH+Na88ZhpL19Eui6F/gkcq2vkpy8XMXNkLnPGd63TPouItKTQP4FHlu+ktKqWWy6doL18EenyFPrHUVVTz89f3c4F4/szc1Ru2OWIiLSbQv84fr3sHQ4dreeWeePDLkVEJC4U+m04dKSOXy19h0snD2TK0L5hlyMiEhcK/Tb84rXtHKlr4FvzJoRdiohI3Cj0W1FaWcNDb+zkE2cWMH5gdtjliIjEjUK/Ffe9UkRDo/ONS8aFXYqISFwp9FvYffAov3tzF5+cPowReVlhlyMiElcK/RbuXVKImfH1i8eGXYqISNwp9KNsL6vmD2uK+ezZIxic0zPsckRE4k6hH+Wel7aRmZbCVy4cE3YpIiIdQqEf2F9Zw3Mb9nLdeSPJ750RdjkiIh1CoR9YWlgOwMemDA65EhGRjqPQDywrLCMvK52Jg/qEXYqISIdR6APuzrKiA5w/Np8e+lUsEenGFPrA2/uqKK+uZda4/LBLERHpUAp9YFnQnz9boS8i3ZxCH1haVM6Y/lkamy8i3V7Sh35NfSNvvnOA2eP0U4gi0v0lfeivefcQNfVNzBqrrh0R6f5iCn0zm29mW82syMxubWX5CDNbYmYbzOxVMxsatWy4mS0ysy1mttnMRsav/PZbVlROag/jnDF5YZciItLhThj6ZpYC3AdcBkwCrjWzSS2a3Q087O5TgNuBO6KWPQzc5e4TgZlAaTwKj5dlReVMHd6X3hmpYZciItLhYtnTnwkUufsOd68DHgOuaNFmEvByMP1K8/LgzSHV3V8CcPdqdz8al8rj4NCROt4qOcysserPF5HkEEvoFwC7o24XB/OirQeuCqavBLLNLA8YD1SY2R/NbK2Z3RV8cngfM7vJzFaZ2aqysrKTfxan6I3tB3BH4/NFJGnE60DuLcAcM1sLzAFKgEYgFZgdLJ8BjAa+0HJld7/f3ae7+/T+/Ttvr3tZURnZmamcMTSn0x5TRCRMsYR+CTAs6vbQYN573H2Pu1/l7lOB7wbzKoh8KlgXdA01AH8CpsWl8nZyd5YWlnPu6DxSU5J+EJOIJIlY0m4lMM7MRplZOnANsDC6gZnlm1nzfX0HeCBq3b5m1rz7fhGwuf1lt9+7B45SfOiYvoUrIknlhKEf7KHfDLwIbAGecPdNZna7mV0eNJsLbDWzbcBA4IfBuo1EunaWmNlbgAG/jPuzOAVLiyKnXpilL2WJSBKJaZyiuz8PPN9i3m1R008BT7Wx7kvAlHbU2CGWFZZR0LcnI/N6hV2KiEinScrO7IbGJt7YfoDZ4/Ix06mURSR5JGXobyg5TFVNg4ZqikjSScrQX1ZYjhmcP0ahLyLJJWlD/0NDcuiXlR52KSIinSrpQr+6toE1uw6pa0dEklLShf6KHQdoaHJm61TKIpKEki70lxaWk5nWg7NG9gu7FBGRTpd0ob+sqJyZo/LISP3Aed9ERLq9pAr9vYePUVRara4dEUlaSRX6ywqbT72g0BeR5JRcoV9UTn7vDE4blB12KSIioUia0G9qcl4vKmfW2DydekFEklbShP7b+6oor67TWTVFJKklTegvK4r8DOMsHcQVkSSWNKG/tLCccQN6MygnM+xSRERCkxShX1PfyJvvHNSoHRFJekkR+qvfPURtQ5N+GlFEkl5ShP7SwnLSUoyzR+WFXYqISKiSIvSXFZUxdXg/sjJi+nVIEZFuq9uH/sEjdWzaU6lTL4iIkASh/3pROe469YKICCRB6C8rLKdPZipThvYNuxQRkdB169B3d5YVlXPemHxSeujUCyIi3Tr03yk/QknFMXXtiIgEunXoLyuKnEpZ4/NFRCK6degvLSxnWG5PRuRlhV2KiEhC6Lah39DYxPLtB5g1VmfVFBFp1m1Df31xBVW1DeraERGJ0m1Df2lhOWZw3hidekFEpFm3Df1lheVMKcihb6/0sEsREUkY3TL0q2rqWbu7QkM1RURaiCn0zWy+mW01syIzu7WV5SPMbImZbTCzV81saNSyRjNbF1wWxrP4tizfcZDGJtdBXBGRFk542kkzSwHuAz4MFAMrzWyhu2+OanY38LC7P2RmFwF3AJ8Llh1z9zPjXPdxLSsso2daCtNG6NQLIiLRYtnTnwkUufsOd68DHgOuaNFmEvByMP1KK8s71dKics4enUtGakqYZYiIJJxYQr8A2B11uziYF209cFUwfSWQbWbNw2YyzWyVmS03s0+09gBmdlPQZlVZWdlJlP9BeyqOsaPsiH4AXUSkFfE6kHsLMMfM1gJzgBKgMVg2wt2nA58G/tfMxrRc2d3vd/fp7j69f//29cMvK2w+9YL680VEWorlp6RKgGFRt4cG897j7nsI9vTNrDewwN0rgmUlwfUOM3sVmApsb3flbVhaVM6A7AzGD+zdUQ8hItJlxbKnvxIYZ2ajzCwduAZ43ygcM8s3s+b7+g7wQDC/n5llNLcBzgeiDwDHVVOT83pRObPG5mOmUymLiLR0wtB39wbgZuBFYAvwhLtvMrPbzezyoNlcYKuZbQMGAj8M5k8EVpnZeiIHeP+7xaifuNq8t5KDR+o0Pl9EpA0x/VK4uz8PPN9i3m1R008BT7Wy3hvA6e2sMWbNp1LWQVwRkdZ1q2/kLissZ8LAbAb0yQy7FBGRhNRtQr+mvpE3dx5U146IyHF0m9CvrKln/uRBXDxxQNiliIgkrJj69LuCAdmZ3Hvt1LDLEBFJaN1mT19ERE5MoS8ikkQU+iIiSUShLyKSRBT6IiJJRKEvIpJEFPoiIklEoS8ikkTM3cOu4X2R2GD7AAAFE0lEQVTMrAx4tx13kQ+Ux6mcjqD62kf1tY/qa59Erm+Eu5/w16MSLvTby8xWBb/UlZBUX/uovvZRfe2T6PXFQt07IiJJRKEvIpJEumPo3x92ASeg+tpH9bWP6mufRK/vhLpdn76IiLStO+7pi4hIGxT6IiJJpEuGvpnNN7OtZlZkZre2sjzDzB4Plq8ws5GdWNswM3vFzDab2SYz++dW2sw1s8Nmti643NbafXVwnTvN7K3g8Ve1stzM7N5gG24ws2mdWNuEqG2zzswqzewbLdp06jY0swfMrNTMNkbNyzWzl8ysMLju18a61wVtCs3suk6s7y4zezv4+z1tZn3bWPe4r4UOrO/7ZlYS9Tf8SBvrHvf/vQPrezyqtp1mtq6NdTt8+8WVu3epC5ACbAdGA+nAemBSizZfAX4RTF8DPN6J9Q0GpgXT2cC2VuqbCzwX8nbcCeQfZ/lHgBcAA84BVoT4995H5IsnoW1D4AJgGrAxat6PgFuD6VuBO1tZLxfYEVz3C6b7dVJ984DUYPrO1uqL5bXQgfV9H7glhr//cf/fO6q+Fst/DNwW1vaL56Ur7unPBIrcfYe71wGPAVe0aHMF8FAw/RRwsZlZZxTn7nvdfU0wXQVsAQo647Hj7ArgYY9YDvQ1s8Eh1HExsN3d2/Mt7XZz99eAgy1mR7/OHgI+0cqqlwIvuftBdz8EvATM74z63H2RuzcEN5cDQ+P9uLFqY/vFIpb/93Y7Xn1BdnwK+H28HzcMXTH0C4DdUbeL+WCovtcmeNEfBvI6pbooQbfSVGBFK4vPNbP1ZvaCmU3u1MIiHFhkZqvN7KZWlseynTvDNbT9zxb2Nhzo7nuD6X3AwFbaJMp2vIHIJ7fWnOi10JFuDrqfHmijeywRtt9sYL+7F7axPMztd9K6Yuh3CWbWG/gD8A13r2yxeA2R7oozgJ8Cf+rs+oBZ7j4NuAz4qpldEEINx2Vm6cDlwJOtLE6Ebfgej3zOT8jxz2b2XaABeLSNJmG9Fn4OjAHOBPYS6UJJRNdy/L38hP9fitYVQ78EGBZ1e2gwr9U2ZpYK5AAHOqW6yGOmEQn8R939jy2Xu3ulu1cH088DaWaW31n1BY9bElyXAk8T+RgdLZbt3NEuA9a4+/6WCxJhGwL7m7u8guvSVtqEuh3N7AvAx4DPBG9MHxDDa6FDuPt+d2909ybgl208btjbLxW4Cni8rTZhbb9T1RVDfyUwzsxGBXuC1wALW7RZCDSPkrgaeLmtF3y8Bf1/vwa2uPv/tNFmUPMxBjObSeTv0JlvSllmlt08TeSA38YWzRYCnw9G8ZwDHI7qyugsbe5hhb0NA9Gvs+uAZ1pp8yIwz8z6Bd0X84J5Hc7M5gPfBi5396NttInltdBR9UUfI7qyjceN5f+9I10CvO3uxa0tDHP7nbKwjySfyoXIyJJtRI7qfzeYdzuRFzdAJpEugSLgTWB0J9Y2i8jH/A3AuuDyEeBLwJeCNjcDm4iMRFgOnNfJ22908Njrgzqat2F0jQbcF2zjt4DpnVxjFpEQz4maF9o2JPLmsxeoJ9KvfCOR40RLgEJgMZAbtJ0O/Cpq3RuC12IRcH0n1ldEpD+8+XXYPKJtCPD88V4LnVTfI8FrawORIB/csr7g9gf+3zujvmD+g82vuai2nb794nnRaRhERJJIV+zeERGRU6TQFxFJIgp9EZEkotAXEUkiCn0RkSSi0BcRSSIKfRGRJPL/ARFv7V2ZbvTRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(len(acces)), acces)\n",
    "plt.title('train acc')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'test loss')"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl83GW1+PHPmUkme7N3Sdqme6GF7oS2QCmCUFBoRVAQpUWvKMp1l4t6f+DlXlfcroqKXtlRQBSoUq1lsWyldKEtlNKVLkm3ZJI0eybJPL8/Zr7pNM0ymfnOft6vV16dzHxnvk+n6ckz5/uc84gxBqWUUqnBEesBKKWUih4N+koplUI06CulVArRoK+UUilEg75SSqUQDfpKKZVCNOgrFSIR+ZeI/Fusx6HUUGjQV0lFRPaLyCU2vM4KEXnFjjEpFU806CulVArRoK+Shog8DIwF/ioizSJym//++SLymog0iMhWEVkc8JwVIrJPRJpE5D0RuUFEzgR+Ayzwv05DEOd2iMh/isgBETkuIg+JSL7/sUwReURE3P4xbBCREf2dPwJvjVI9NOirpGGM+QRwELjSGJNrjPmhiJQDzwL/AxQBXwP+LCKlIpID/By43BiTBywEthhjdgCfBdb5X6cgiNOv8H9dBEwAcoFf+h9bDuQDY4Bi/2u39Xf+MN8GpQakQV8lu48Dq4wxq4wxXmPMGmAjcIX/cS9wlohkGWOOGGO2h3ieG4CfGGP2GWOagW8A14lIGtCJL9hPMsZ0G2M2GWMabT6/UkHRoK+SXQVwrT+t0uBP1ZwPjDLGtAAfxTfzPiIiz4rIGSGepww4EPD9ASANGAE8DKwGHhORwyLyQxFJt/n8SgVFg75KNr3bxh4CHjbGFAR85Rhjvg9gjFltjHk/MAp4F/hdP68zmMP4fsFYxgJdwDFjTKcx5r+MMdPwpXA+CNw4yPmViggN+irZHMOXU7c8AlwpIpeJiNN/UXWxiIwWkREistSfW+8AmvGlW6zXGS0iriDP+0fgyyIyXkRyge8CjxtjukTkIhE5W0ScQCO+dI93kPMrFREa9FWy+R7wn/5UzteMMYeApcA3gRp8M/+v4/vZdwBfwTdLrwMuBG7xv84LwHbgqIjUBnHe+/ClcV4C3gPagX/3PzYSeBJfwN8BrPUfO9D5lYoI0U1UlFIqdehMXymlUogGfaWUSiEa9JVSKoVo0FdKqRSSFusB9FZSUmLGjRsX62EopVRC2bRpU60xpnSw4+Iu6I8bN46NGzfGehhKKZVQROTA4EdpekcppVKKBn2llEohGvSVUiqFaNBXSqkUokFfKaVSiAZ9pZRKIRr0lVIqhWjQVypOvLanlj3Hm2M9DJXkNOgrFSe++qet/OKF3bEehkpyGvSVigPGGGqaOqht7oj1UFSS06CvVBxobOuiy2twN3tiPRSV5DToKxUH3C0d/j816KvI0qCvVBywgn19iwfdwlRFkgZ9peKAldbp8hoa27piPBqVzDToKxUHrPRO79tK2U2DvlJxoC7gAq7m9VUkadBXKg4EBnpdwaMiSYO+UnHA3eIh2+UEoE5n+iqCNOgrFQfczR1MLM3tua1UpGjQVyoO1LV4GJWfSV5Gmub0VURp0FcqDtQ2eyjOdVGU69L0joooDfpKxZjXa6hv9VCck0FxjkuXbKqI0qCvVIydaOuk22soynFRlJOhq3dURGnQVyrGrBx+ca6L4hxN76jI0qCvVIxZq3WKczIo9uf0tf+OihQN+krFWF3ATL8ox6X9d1REadBXKsZqraCf46I41wVo/x0VORr0lYoxq+9Oof9CLmhVroocDfpKxZi7pYP8rHTSnQ6Kc3wz/VpdwaMiRIO+UjHmbvH0pHWsP3WmryIlqKAvIktEZKeI7BGR2/t4fJGIbBaRLhG5ptdjY0XknyKyQ0TeEZFx9gxdqeTgbu7omeEX5VhBX3P6KjIGDfoi4gTuAS4HpgHXi8i0XocdBFYAf+jjJR4C7jbGnAlUAsfDGbBSyaauxVeNC5CR5iQvI03TOypigpnpVwJ7jDH7jDEe4DFgaeABxpj9xphtgDfwfv8vhzRjzBr/cc3GmFZ7hq5UcnA3eyjyp3UA7b+jIiqYoF8OHAr4vsp/XzCmAA0i8hcReVNE7vZ/clBKAd3+vjslOQFBX6tyVQRF+kJuGnAB8DXgHGACvjTQKUTkZhHZKCIba2pqIjwkpeJHQ6sHrzmZywdfZa62V1aREkzQrwbGBHw/2n9fMKqALf7UUBfwNDCn90HGmN8aY+YZY+aVlpYG+dJKJb6T1bgZPfcV57h0IxUVMcEE/Q3AZBEZLyIu4DpgZZCvvwEoEBErkr8PeGfow1QqOVkXbItzTs3p17dq/x0VGYMGff8M/VZgNbADeMIYs11E7hKRqwBE5BwRqQKuBe4Vke3+53bjS+08LyJvAQL8LjJ/FaUST38z/c5uQ2O79t9R9ksL5iBjzCpgVa/77gi4vQFf2qev564BZoQxRqWSltVj55ScvtV/p9lXqauUnbQiV6kYcjd7EIHC7JPBXfvvqEjSoK9UDLlbOijISifNefK/opXf1xU8KhI06CsVQ3UtnlPy+RCY3tGgr+ynQV+pGKpt9pySzwftv6MiS4O+UjFU1+KhJPfUoJ+R5iQ3I03TOyoiNOgrFUPu5o7TZvpAz165StlNg75SMdLtNTS0dfas1glUlOPSnL6KCA36SsWIr+qW09I74G/FoDN9FQEa9JWKEWsm31d6x9dpUy/kKvtp0FcqRqxq3OI+0jvFuRnUtWj/HWU/DfpKxYg10y/uJ72j/XdUJGjQVypGepqt9ZPeCTxGKbto0E8SXq/hsp++xBMbDw1+sIoL7uYORKAgu68lmxk9xyhlJw36SeJ4Uwc7jzVx/6v7Yz0UFSR3i4eibBdOh5z2mPbfUZGiQT9JVNX79pvfcaSRXceaYjwaFQx3Hy0YLJreUZGiQT9JVDe09dx+Zkuwu1mqWPI1W9Ogr6JLg36SqKr3Bf1zxhXyzJbDutQvAdS2dPS5XBMgM93Xf6dWc/rKZhr0k0RVfRvFOS6uO2csVfVtbDpQH+shqUEMNNMHq0BLZ/rKXhr0k0RVfSvlhVlcdtZIMtMdPK0pnrjW2e2lobWz35w+aNM1FRka9JNEdUMbowuzyM1I45IzR/DstiN0dntjPSzVj/rW0zdE7604x0WtNl1TNtOgnwSMMVTXt1FekAXAslnl1Ld28tKumhiPTPWnpxp3gJm+9t9RkaBBPwnUNnvo6PL2BP1FU0opyE7n6S2HYzwy1Z+BqnEtRTnaf0fZT4N+ErCWa44uzAbAlebgA2ePYs07R2np0N4t8chalTPQhdySXO2/o+ynQT8JWIVZ5YVZPfctm11Oe6eXf75zNFbDUgM4OdPvP6eva/VVJGjQTwLV/jX6gUF/7thCyguyePpNTfHEI3ezB6dDyM9K7/cY3SBdRYIG/SRQVd/GsMw0hmWeDCAOh3DVrDJe2VOrBT5xyN3ioTDbhaOPvjuWkp6mazrTV/bRoJ8EfMs1s0+7f9mscrq9hr9t1dl+vHE3dwx4ERdOzvS16ZqyU1BBX0SWiMhOEdkjIrf38fgiEdksIl0ick0fjw8TkSoR+aUdg1ansgqzeps6Mo8zRubpKp44NFg1LmhOX0XGoEFfRJzAPcDlwDTgehGZ1uuwg8AK4A/9vMx/Ay+FPkzVH2uN/ug+gj74LuhuOdTA/tqWKI9MDcTd0n+HTYvVf0fTO8pOwcz0K4E9xph9xhgP8BiwNPAAY8x+Y8w24LQSUBGZC4wA/mnDeFUvDa2dtHi6e9bo93bVzDJE4Bmd7ccVd3NHT85+IEU5rp69dJWyQzBBvxwI3I6pyn/foETEAfwY+Nogx90sIhtFZGNNjVaRDsXJNfp9B/2ygiwqxxXxzJZqLfKJE54uL43tXYPO9EGbrin7RfpC7ueAVcaYqoEOMsb81hgzzxgzr7S0NMJDSi5WS+W+LuRals0uZ19tC29XN0ZrWGoAJ/vuDB70i3Ncmt5Rtgom6FcDYwK+H+2/LxgLgFtFZD/wI+BGEfn+kEaoBtRTmNVPegfgirNG4XJq58140VONG8RMXzttKrsFE/Q3AJNFZLyIuIDrgJXBvLgx5gZjzFhjzDh8KZ6HjDGnrf5RoatuaCPH5aQgu/8in/zsdBZPLeWvWw/T7dUUT6z1VOMGldPPwN3Soak5ZZtBg74xpgu4FVgN7ACeMMZsF5G7ROQqABE5R0SqgGuBe0VkeyQHrU6qqm+jvDALkf6LfMCX4jne1MG6ve4ojUz1x0rXBJPTL87x9d9p0h5KyiZpwRxkjFkFrOp13x0BtzfgS/sM9BoPAA8MeYRqQL7lmv3n8y3vO2M4eRlpPL2lmvMnl0RhZKo/VrFVyQB9dyxW3r+u2XNKxbVSodKK3ARXVd86YD7fkpnu5LKzRvKPt4/S3tkdhZGp/ribO0hzCMOyBp9znazK1WWbyh4a9BNYY3snje1dfVbj9mXZrHKaO7p4fsfxCI9MDaTOX5g1WEoOTnbh1BU8yi4a9BNYdf3Aa/R7WzCxmOF5GbqKJ8ZqmwevxrX0pHd0BY+yiQb9BNbTUjmI9A6A0yFcObOMf+08zonWzkgOTQ2griW4alzQpmvKfhr0E1jvHbOCsWxWOZ3dhlVvH4nUsNQggum7Y8lMd5Ljcmp6R9lGg34Cq6pvJSPNQUkQlZ2Ws8qHMaE0h6ff1BRPrNQ1D95hM1BRrm6QruyjQT+BVTcEt0Y/kIiwbFY569+r47D/k4KKno6ubpo6uoKqxrUU52RoekfZRoN+Aquqbws6nx9o6awyAFbq5ipRN5RqXIv231F20qCfwIItzOqtojiH2WMLNMUTA0OpxrVop01lJw36CarV04W7xRP0cs3els0q592jTew82mTzyNRAeqpxh5DTL87NoK7Fo/13lC006CcoKx8fSnoH4AMzRuF0iK7ZjzK3v8NmURAtGCzFOS483V7tv6NsoUE/QVUNsTCrt5LcDM6fVMLKLYfxaufNqDmZ0x9aegd8q36UCpcG/QRlBf1gWzD0ZdnsMqob2th0sN6uYalB1DZ7SHcKeRlB9ToETv6C0BU8yg5JE/SPNbbz6Yc28sru2lgPJSqqG9pIdwrD8zJDfo1Lp40kK92pF3SjqK6lg+KcjCEtsz3Zf0fX6qvwJU3Qz89K55XdtTy341ishxIVVfVtjMrPwukIPnj0lpORxvunjeDZt47g6TptT3sVAe4h9N2xFGn/HWWjpAn6melO5k8o4qVdqbGxenV9a8j5/EDLZpfR0NqZMu9brLlbhlaNCye3VdT0jrJD0gR9gAunlLKvtoVDda2xHkrEhVqY1dsFk0spynHpKp4ocbd0DKkaF07239GZvrJDUgX9RVNKAVib5LPWjq5ujjd1hFSY1Vu608EHzh7FczuO0axLAiPO13cn+OWalqJcl+b0lS2SKuiPL8lhdGFW0gf9ww3tQHgrdwItm11Ge6eX1W8fteX1VN/aO7tp8XQPOacP1gbpOtNX4UuqoC8iXDillHV73Ul9YXKoffQHM2dsIWOKsjTFE2GhVONaSrQVg7JJUgV98KV4mju62JzEa8+rG3zXLOy4kAu+X5ZLZ5bz6p5aapo0hRApoVTjWoq06ZqySdIF/YUTi0lzSFKvRqmqb8MhMDI/9DX6vS2bXYbXwLPbtPNmpLhDqMa1+Hrqa/8dFb6kC/p5menMqShM6rx+tX+NfrrTvn++ScPzGFuUzev76mx7TXUqa6Y+1NU7ACU5GXi6vXqxXYUt6YI++JZubj/cmLSpCruWa/Y2t6KQTQfrdTYZIdbuVyGt3rHW6muKR4UpaYM+wMu7k3O2X93QZls+P9CcikJqmjp6+vooe7mbPbjSHOS4nEN+bpH231E2ScqgP23UMEpyXUmZ1+/s9nLkRJttyzUDzR1bCMCmA8l7ETyW3C0eSnJcQ+q7Y7FSQrqCR4UrKYO+wyFcMLmUl3bXJl3b4KMn2vEa+1buBJo6Mo8cl1ODfoS4mzt6ZuxDZaWEdIN0Fa6ggr6ILBGRnSKyR0Ru7+PxRSKyWUS6ROSagPtnicg6EdkuIttE5KN2Dn4gi6aUUNfiYfvhxmidMip6WioXhF+N25vTIcweW6hBP0LqWjw9HTOHyprp12pOX4Vp0KAvIk7gHuByYBpwvYhM63XYQWAF8Ide97cCNxpjpgNLgJ+JSEG4gw7GBZN9ef2XkiyvX90Qfh/9gcypKOTdo4206CoR29U2e0JauQO+/jvZ2n9H2SCYmX4lsMcYs88Y4wEeA5YGHmCM2W+M2QZ4e92/yxiz23/7MHAcKLVl5IMoyc3grPJhrN2ZZEHfP9MvK7BvjX6gOWML8BrYeqghIq+fyupC6LAZqDhXq3JV+IIJ+uXAoYDvq/z3DYmIVAIuYG8fj90sIhtFZGNNjX1B+sIppWw+WE9je6dtrxlrVfWtDM/LICNt6CtAgjFbL+ZGRKuni7bO7pCqcS1FORnUatM1FaaoXMgVkVHAw8BNxpjTmuIYY35rjJlnjJlXWmrfB4FFk0vp8hpe2+O27TVjLVLLNS35WelMGZGrWyjarKcwK5yZvvbfUTYIJuhXA2MCvh/tvy8oIjIMeBb4ljHm9aENLzxzKgrJzUhLqrx+VX0b5Ta0VB7I3IpCNh+oT7qVT7HU04IhxJy+9VwN+ipcwQT9DcBkERkvIi7gOmBlMC/uP/4p4CFjzJOhDzM06U4HCycWs3ZnTVJUmXZ7DUdORHamD76um43tXeytaY7oeVJJONW4Fl9Pfe2/o8IzaNA3xnQBtwKrgR3AE8aY7SJyl4hcBSAi54hIFXAtcK+IbPc//SPAImCFiGzxf82KyN+kH4umlFLd0Ma+2pZonjYijje109ltItKCIdDcCs3r2602jL47luIcl/bfUWFLC+YgY8wqYFWv++4IuL0BX9qn9/MeAR4Jc4xhsVoyrN1Zw8TS3FgOJWzWyp1Iz/THl+RQmJ3O5oP1XFc5NqLnShV1YXTYtFgXgetaPORlptsyLpV6krIiN9CYomwmlOQkRV6/KkpBX0SYo0VatnI3d5CZ7iDbFdQ8q0/F2n9H2SDpgz74Ujyv73PT3tkd66GExSrMKotwegd8F8H31rRQrwHGFu4wqnEtxdppU9kgJYL+hVNKae/0smF/YveKr6pvozjHFdZsMVhWXv/NQzrbt4O7ObzCLDjZXln776hwpETQP3dCEa40R8JX51bVt0as/UJvM0cX4HSIpnhs4uu7E17Qtz4paHpHhSMlgn62K43KcUUJn9ePdGFWoCyXk+llwzTo28Td3BFWNS74/k2yXU5N76iwpETQB1+KZ9exZo6cSMwNQowxVEdox6z+zBlbyNZDJ+jqPq2IWg2BMcbXSz/M9A74UjxaoKXCkTJBf5F/6WaibqxS2+yho8vL6AhX4waaW1FIW2c37x5tito5k1GLp5uOLm9PTj4cxbkZmt5RYUmZoD9lRC4jh2Xy0q7aWA8lJFX1rQDRnelrkZYt6nr67oSX3gHfCh63Nl1TYUiZoC8iLJpSwsu7axIyXWEt1xxdFL2gX5afychhmRr0w1RrtWCwYaav6R0VrpQJ+uBL8TS2d7G16kSshzJkJ3fMil7QFxHmVmiRVrjqbOiwaSnOceFu0f47KnQpFfTPn1SCQ2BtAub1q+vbGJaZFvXy+zkVhVQ3tHH0RHtUz5tM3P6Zvj05fReeLi8tnsQuNFSxk1JBvyDbxcwxBQl5Mde3XDN6F3EtVpHWZu2vH7KTbZXDz+lbyz41r69ClVJBH3xLN7dWNSRce4FoFmYFmjZqGBlpDk3xhMHd7CHb5STLFf5uZz2tGBLs51fFj5QL+oumlGIMvLIncVbxWGv0o1WYFciV5mDm6AKd6Ych3L1xA1mvU6cFWipEKRf0Z44uID8rPaHy+g2tnbR4uqN6ETfQ7IoC3q4+kfAN62Kl1oZqXEtRz0xf0zsqNCkX9J0O4fzJJby0K3F20+pZrhmDnD7A3LGFdHYb3q5OvFVP8aCuxUOJDRdxQfvvqPClXNAHuHByKcebOhKm0tQqzIpFege0SCtc7maPLSt34GT/HU3vqFClZNBPtJYM0do8pT8luRmMK87WoB8CY4w/p29Pegd8KR6d6atQpWTQH5mfydQReQnTdbOqvo0cl5P8rNhtkTenopDNB+sTJiUWL5o6uvB0e22pxrUUa9BXYUjJoA9w4dRSNrxXT6sn/jeZrm5oo7wwCxGJ2RjmVhRS2+zhYF1rzMaQiOysxrX4WjHohVwVmpQN+osml+Lp9vL6PneshzIo33LN2FzEtWiRVmjsrMa1FOdmaE5fhSxlg/68cYVkpifGblpV9a0xW65pmTw8j9yMNM3rD5G14UmJjTn94hwXtdp/R4UoZYN+ZrqTBROKeWl3fBdpNbZ30tjeFbOLuBanQ5g9toBNBxpiOo5EY+Xe7ZzpF+Vo/x0VupQN+uBbxfNebQsH3fGbp662umvGOOiDbyetnUcbaWrvjPVQEkZdBIK+tRJIUzwqFCkd9C/0L91cG8ereKrrY1uYFWhuRSFeA1sPaZFWsGqbO8jNSCMzPfy+OxZrJVCtXsxVIUjpoD++JIfRhVlxndePxY5Z/Zk1tgARLdIaCjv77lisTw0601ehSOmgLyJcOKWUdXtr8XTF525a1Q1tZKQ5bNlUO1zDMtOZOiKPTbqCJ2h2VuNaepqu6Vp9FYKggr6ILBGRnSKyR0Ru7+PxRSKyWUS6ROSaXo8tF5Hd/q/ldg3cLoumlNLi6Y7b2WtVfezX6AeaU1HImwfr8Xp15Ugw3C0eW/roB7JeT9M7KhSDBn0RcQL3AJcD04DrRWRar8MOAiuAP/R6bhFwJ3AuUAncKSKF4Q/bPgsnFpPmkLitzq1uaIuL1I5lzthCmtq72FPTHOuhJAR3c4et1bjg67+Tla79d1RogpnpVwJ7jDH7jDEe4DFgaeABxpj9xphtQO8cyWXAGmNMnTGmHlgDLLFh3LbJy0xnTkVh3Ob146EwK9Bcbb4WtJN9d+xPzekG6SpUwQT9cuBQwPdV/vuCEdRzReRmEdkoIhtraqIffC+cUso7RxqpaYqvj8utni7cLZ6Yr9EPNK44m6Iclwb9IDS2ddHlNbbn9AFKcrX/jgpNXFzINcb81hgzzxgzr7S0NOrnt5ZuvhxnKZ7DDbHtrtkXEWHO2EI2a9AflNWCwc5qXIuv02Z8TVJUYggm6FcDYwK+H+2/LxjhPDdqpo0aRnGOK+TdtIwx1DR18M7hRjq77VsFdMgqzIqjnD74Ujz7als0vTCISFTjWopytP+OCk1aEMdsACaLyHh8Afs64GNBvv5q4LsBF28vBb4x5FFGmMMhLJpSytpdNXi9Bofj1JUynd1ejp5op6q+jcMNbVQ3tFFd7/vT+r7Dv+TzixdP5svvn2LLuOKpMCtQT/O1A/VcMm1EjEcTv9wR6LBpsdI7xpi4WdmlEsOgQd8Y0yUit+IL4E7gPmPMdhG5C9hojFkpIucATwGFwJUi8l/GmOnGmDoR+W98vzgA7jLG1EXo7xKWRVNKeOrNan723C483eaU4H6sqZ3eva1K8zIoK8jizFHDuGTaCMryM1n19lEeXX+Az100kYy08Cswq+rbSHcKw/PsTw+EY8bofNIcwuaDGvQHYqVf7F6yCb5PDx3+/ju5GcHM3ZTyCeqnxRizCljV6747Am5vwJe66eu59wH3hTHGqLhgcimuNAc/f2EP6U5hVH4W5QVZnDephPLCLEYXZFFWkEV5YRaj8jP7LKufODyXT/z+DZ7ddoSr5/T5dgxJdUMbZQVZp33yiLXMdCfTy4bpxdxBWOmXyKR3TlblatBXQ6E/LX4luRms/fpiBKE0LwNnCIH2/EklTCzN4cHX9tsS9OOhpXJ/5lQU8sc3DtLZ7SXdGRfrAeKOu8VDXmYarjT73x/r4rC7pYOxxfGV/lPxTf+3BhiVn8XI/MyQAj74VrYsXziOrVUneNOGVgXV9fFVmBVobkUh7Z1edhxpjPVQ4pa7xRORlTsQMNPXi+lqiDTo2+zqOaPJzUjjwdf2h/U6HV3dHG/qiLuLuBYt0hqcu7kjIqkdOBn03bqCRw2RBn2b5Wakcc3c0Tz71hGON7WH/DqHG3zPjYc++n0ZlZ9FWX6mBv0B1LV4bG/BYLFWBGmBlhoqDfoRcOOCCjq7DX9cf2jwg/txcrlmfAZ98OX1tUirf7XNkWnBAJDtSvP139ECLTVEGvQjYEJpLoumlPLo+gMhF2vFUx/9/sytKOTwiXaOnGiL9VDijtdrqG+1v8NmoKIcl6Z31JBp0I+QFQsrON7UwT/ePhrS86sb2nA6hFH5mTaPzD5zxlpFWrpvbm8n2jrpjlDfHUux9t9RIdCgHyGLpwynojg75Au6VfVtjByWSVocL4ecVjaMzHSH5vX7YAXjSKV3wLdtoq7eUUMVvxElwTkcwifmV7DxQD1vVw99T9lq/+Yp8Szd6WDG6ALdSasP7ubIVeNainIyes6jVLA06EfQtfPGkJXuDGm2X93Qxug4zudb5lYUsr36BO2d3bEeSlypi8ZMP6D/jlLB0qAfQflZ6Vw9p5xnth4e0sfwzm4vR07E/0wfYO7YQrq8hm1VQ/80k8xqraAfyZy+v/9Oq0d/4argadCPsOULx+Hp8vLYhoNBP+foiXa8Jr6Xa1rmaJFWn6y+O4URDPpalatCoUE/wqaMyGPBhGIeWXeAriCXb1b19NGPz2rcQEU5LiaU5LBZ8/qncLd0kJ+VHtG+RFbqqFbz+moINOhHwfKF4zh8op3ndhwP6vjqONwxayCz/TtpaW75JHcEq3EtRf6LxDrTV0OhQT8KLjlzOOUFWUFf0LUKs0YVxO8a/UBzKwpxt3g44G6N9VDihru5I6IXceHk9QJdq6+GQoN+FKQ5HXx8fgXr9rnZebRp0OOr69sYMSzDlo1YokGbr52ursUT0cIsCOi/o1W5agg06EfJdeeMISPNwYPr9g96bFUct1Tuy+ThueRlpOl6/QDuZg/FEWqrbMl2pZGZ7tD+O2pINOhHSWGOi6UzCbdIAAAZjElEQVSzynhqczUnWjsHPLa6oY3yOG2p3BeHQ5itzdd6dPf03YnsTB98xV+a3lFDoUE/ipYvHEdbZzd/2tR/981ur+HIibaEuYhrmTu2kJ3HmmhsH/gXWipoaPXgNZFdo28pztWma2poNOhH0fSyfM4ZV8hD6w7Q7e17pcvxpnY6u01CpXfAl9c3BrYe0uZr1mqaogind8C3ZFZX76ih0KAfZcsXjuNgXSv/2tn38s1E6KPfl5lj8nEIrNvrjvVQYq7WP/MuiVJ6R4O+GgoN+lF22fSRjBiWwQP9LN+sStCgn5eZzgWTS3lyU1XIewgki5Mz/Sild1o6tEZCBU2DfpSlOx3ccG4FL++uZW9N82mPW4VZiVCN29ty/x4Cq7eHtodAsnC3RL7DpqUox0V7p/bf8XoN/9p5nNf21uovwEFo0I+B6yvH4nI6eHjdgdMeq6pvpTjHRZYrMdboB7pwynDGFGXx0Gun/71SiXVhtTA7PeLnSvX+O+2d3Ty6/gCX/GQtK+7fwMd+t55l97zKmneOafDvhwb9GCjNy+ADM0bx5KYqmju6Tnmsqj7xVu5YnP49BN7YX8e7RxtjPZyYcbd0UJidHpUNcEpSdIN0d3MHP12zi4Xff4FvPfU2ORlp/Pz62Xzv6rOpa/Xw6Yc2cvn/vsyz2470u2giVWnQj5HlC8fR3NHFnzdVnXJ/ImyeMpCPzPMVoT3Ux6eYVBGNalyL1X8nVTZT2VvTzDefeouF33+B/31+N7PHFPDYzfNZeet5XDWzjOsrx/LiVxfzk4/MxNPt5fN/2MylP13LU29WBd3wMNlp0I+RWWMKmDmmgAfX7cfrn4kYY3yFWQm2XDNQQbaLq2b6i9DaUnPNfm0UqnEtqdB/xxjDG+/V8W8PbuSSn6zlyU1VXD2nnOe+sojfrziH+ROKEZGe49OcDq6eM5o1X76QX35sNulOB19+fCvv+/FaHnvjIJ6u1A7+QQV9EVkiIjtFZI+I3N7H4xki8rj/8fUiMs5/f7qIPCgib4nIDhH5hr3DT2wrFlawr6aFV/bUAr5g0dHlZXQCVeP2xSpC6/0pJlXURaHDpsXqv5OMOf2ubi/PbjvCsl+9xkfuXcfGA3X8+0WTePU/3sf3rp7BpOF5Az7f6RA+OKOMVV+4gN9+Yi4F2enc/pe3WHz3izy0bn/K7vY2aNAXESdwD3A5MA24XkSm9TrsU0C9MWYS8FPgB/77rwUyjDFnA3OBz1i/EBRccfYoSnJdPd03re6aiTzTBzirPJ/ZYwt4+PUDPZ9iUkk0OmxarP47dqR3dh5t4siJNhtGFZ6Wji7uf/U9Lvrxv/j8HzbT0Orhv5dOZ93tF/OVS6dSmje0T1EOh3Dp9JE88/nzeOCmcxhVkMUdz2zngh++yP+9vI9WT9fgL5JE0oI4phLYY4zZByAijwFLgXcCjlkKfNt/+0ngl+L7vGWAHBFJA7IAD5C6V/h6yUhz8rHKsfzixT0cdLee7KNflNhBH2D5gnF86fEtvLKnlkVTSmM9nKjp6vbS0NbZk2uPBjv67xyqa+VDv3qVUfmZrP7SoqhchO6tpqmD+199j0deP0BjexdzKwr51hXTeP+0ETgdMvgLDEJEWDx1OBdOKeX1fXX84oXd/M+zO/jVv/byqfPHc+OCCvIyI7/iKtaC+ZctBwKbxVT57+vzGGNMF3ACKMb3C6AFOAIcBH5kjKnrfQIRuVlENorIxpqamiH/JRLZDfMrcIrw0Lr9ATtmJX7Qv/zskRTnuFLugm59ayfGnFxVEw3FueG1YjDG8I2/vIWny8vemhYe29B/b6hI6ejq5iP3ruPXa/eycGIJf75lAX++ZSFLzhppS8APJCIsmFjMHz49nz/fsoCZo/O5e/VOzvv+C/zyhd1Jn/aJ9K/zSqAbKAPGA18VkQm9DzLG/NYYM88YM6+0NHVmhQAjhmVy2VkjeWLjIXYfayY/Kz0pZhsZaU6uqxzD8+8e41Bd6myu0lONG6WcvnWucIL+ExsP8cqeWu68ajqV44r42XO7TltKHGn3v7qf92pb+P3yefzmE3OZW1EUlfPOrSji/psq+eut53PuhGJ+9M9dXPazl3ixnzYpySCYoF8NjAn4frT/vj6P8ady8gE38DHgH8aYTmPMceBVYF64g042KxaOo7G9i5Vbq5Nilm+54dwKBHh0ffCbwic6K7cejWpcS1FO6J02j55o53/+toNzxxdxQ+VYvvmBM6lt9nDv2r02j7J/x5va+cXzu7nkzOG874wRUTtvoLNH5/O7G+fxyKfOxekQbrp/A59+aGNSTliCCfobgMkiMl5EXMB1wMpex6wElvtvXwO8YHzlcAeB9wGISA4wH3jXjoEnk3kVhUwbNYzObpOwhVl9KSvI4v3TRvD4hoNJ/5HZYuXWo3UhF3zLNt0hbKRijOFbT71Fp9fLDz48A4dDmDWmgCtnlvG7l/dx9ER7BEZ7urv/sRNPt5dvfaD3+pDoO39yCf/44iL+Y8kZvLK7lkt+spZfPJ9cKZ9Bg74/R38rsBrYATxhjNkuIneJyFX+w34PFIvIHuArgLWs8x4gV0S24/vlcb8xZpvdf4lEJyKsWDgOIKELs/qyfME46ls7+du2I7EeSlRYaZZoLdkEKM7N8PffGVpKZuXWwzz/7nG+dulUxpXk9Nx/22VT8Xrhx//cafdQT7OtqoEnN1dx03njGR8whlhypTm4ZfFEnv/qhVxy5gh+vCa5Uj5B5fSNMauMMVOMMRONMd/x33eHMWal/3a7MeZaY8wkY0yltdLHGNPsv3+6MWaaMebuyP1VEttVs8o4Z1wh508qifVQbLVgYjGThufy8Lr9sR5KVLibOxDxFalFi3X9YCgpnpqmDu5cuZ3ZYwu46bzxpzw2piibFeeN48nNVbxzOHKL7Ywx/Ndf36E4x8Wt75sUsfOEqqwgi3tumMPDn6rsSfncnAQpH63IjROZ6U7+9NmFXHxmbHKakSLi68ezteoEW1JggxV3i4eibJftK04GEkpV7rdXbqe1o5u7r5nR51g/v3gSwzLT+d7fd9g2zt5Wbj3MpgP1fP2yqQyL48ULF0wu7Un5vJwEKR8N+irirp5TTo7LyUPr9sd6KBHnbo5e3x2L1fIh2A3S//7WEZ596whfvGRyv1Wt+dnpfOHiyby8u5a1u+xfRt3m6eb7f3+X6WXDuGbumMGfEGOBKZ+LzxzOj9fsYkmCpnw06KuIy8tM5+o5o/nb1iNJ3xisrsUT1Yu4cHKmXxtEeqeh1cP/e2Y708uGcfOi01ZPn+IT8yuoKM7mu8/usL1T5W/W7uXIiXbuvHJ6VD8VhausIItf3TCXhz9ViUMSM+WjQV9FxY0LKvB0e3l8Y+QKfxrbO2PeRre2pSOqyzVhaD317/rbOzS0evjhNTNIH6Tq1pXm4LbLzmDnsSZb+yhVN7Txm7V7+eCMUVSOj856fLtdMLmUv3/pAm5bMpWXd9fy/p/6Uj4n2jpp83TT0dVNZ7eXbq+Ju77+wbRhUCpsk0fksWBCMY++fpDPLJpo++xu17Emrvn1a0wozeW+FedEPcViicVMP9vlJDPdMWjQf/Hd4/xlczVfeN8kppflB/XaV5w9ktljC/jRP3fywZmjyHaFHzK+/3ffqu1vXHFm2K8VSxlpTj63eBJLZ5XznWff4cdrdvHjNbv6PFYEHCI4ev7033bIKffPGJ3P/TdVRnTcGvRV1Ny4oIJbHt3M8zuOcen0kba97rHGdlbc9wbpTgc7jjTy4V+/xoM3VTK2OLrdSju7vTS0dkb9F46I+PrvDJDeaWzv5JtPvcWUEbl8fggrZUSE//zAmXz41+v43Uvv8cVLJoc11jfeq+OvWw/zhYsnJ00hYrk/5bNur5u3qhvwGvAag9dr+rzdbQzG+LZ47LltDN1eE5UOuxr0VdS8f9oIRuVn8vDrB2wL+s0dXXzygQ00tHXyxGcW0N7Zzace3MjVv36NB246h7PKg5vR2qG+1SrMim56B/xVuQNcyP3eqnc51tjOrz9+HhlpQ9uKc25FEZefNZJ7X9rL9eeOYXheZkhj9HoNd/1tO6PyM/nshQNfT0hECyYWs2BicayHMSjN6auoSXM6+Fjl2H43hR+qrm4vn390M+8ebeKeG+ZwVnk+88YV8edbFpCR5uCj966LyMqT/lgz7WgWZlkG6r/z2p5a/vjGQf7tggnMGlMQ0uv/x5Iz8HR5+ema3SGP8clNVbxd3cjtl59hS5pIhUaDvoqq6yrHku6UPjeFHwpjDP/vmbdZu6uG/1l2FhdNHd7z2KThefzlcwsZW5zDpx7YELXNXGJRjWspzu27/06rp4v/+Ms2xpfk8JX3Twn59ceV5PDx+RU8vuEgu481Dfn5Te2d/HD1u8ytKOSqmWUhj0OFT4O+iqrSvAyuOHsUf95URUsYnRx/9a+9/PGNQ3z+oolcXzn2tMdHDMvk8c/M59wJRXz1T1u558U9EV9FUWs1W4vyhVzov//O3at3cqiujR98eAaZ6UNL6/T2hYsnk5ORxvf+PvT2Wb98YQ+1zR7uvHLaKVsbqujToK+i7sYFFTR1dPHUm72btQbn6TeruXv1TpbOKuNrl07t97hhmencv6KSpbPKuHv1Tu54ZntEl3SenOnHIqd/ev+djfvreOC1/SxfUGHL0siiHBefv2gSL7x7nNf8W3wG473aFu579T2unTuaGaNDSy8p+2jQV1E3Z2wh08uG8fC6A0Oefa/b6+brT25l/oQifnjNjEFnja40Bz/9yCw+s2gCD79+gFse2RSx8nl3swenQ8jPin5LAevThZXiae/s5rY/b6MsP4vblpxh23lWLBxHeUEW31m1I+itML/z7A5cTgdfX9L/L2gVPRr0VdSJCDcuqGDnsSbWv3faRmr92n2sic88vJGK4hzu/fi8oFehOBzCN644kzuvnMaaHce44f/WUx+BjcTdLR4Ks104YlBh2rv/zv8+v5t9NS384MMzyMmw76JpZrqT25ZMZfvhRp7eMvgntZd21fDcjmPc+r7JIa/6UfbSoK9i4qqZ5eRnpQd9Qfd4Yzsr7t9ARrqTB246h/zsoc+mbzpvPPd8bA5vVZ/gw795zfbSeXdzR0wu4kJgVW4H26oa+O1L+/jovDGcP9n+rq1Xzijj7PJ8frR654Cfmrq6vfz3396hojibT54/zvZxqNBo0FcxkeVy8pF5o1m9/eigm3W0dHTxyQc3UNfi4b7l54RVwHLF2aN4+JOV1DZ1cPWvX2P74RMhv1ZvsajGtZT4awOONXZw25PbKMl18c0PRKbi1eEQvnnFmRw+0c59r77X73GPrj/I7uPNfPOKM4dcG6AiR4O+ipmPz6+g2xj+8Eb/2yl2dXu59Q+beedwI/fcMJuzR4dfbHXuhGKevGUhaQ7ho/e+ziu7g78oGcjrNeyvbWH19qP8/Pnd7D7eHLP2D9Z5f/H8bt492sR3P3R2RK8tLJhYzCVnjuDXL+7ts4lefYuHn6zZxXmTirl0WnK1C090WiGhYqaiOIfFU0r54xsHufWiSbjSTp2DGGO4c+V2XtxZw3c+dJat+6dOGeFby3/T/RtYcf8b3H3tDD40e3S/x9c0dbDrWBPvHm1i59FGdh5tYtexZtoC0htji7JZcpZ97SWGItvlJCPNweET7SybVRaVfRluv/wMLvvZS/z8+d3819KzTnnsp8/toqm9kzs+OF2XaMYZDfoqpm5cMI6bHtjAP7YfPa1o5zdr9/Ho+oN89sKJ3HBuhe3nHpWfxROfXcDND23ky49v5VhjB5+YX8Hu483sPNroD/C+r8ANSopzXEwdmcd1lWM4Y2QeU0cOY/LwXFsvmA6ViFCSm0FHVzd3Xjk9KuecNDyX6yvH8Oj6gyxfOI4JpbkA7DzaxKPrD/Lx+RVMHdl3v34VOxJvbT/nzZtnNm7cGOthqCjxeg2Lf/QvRgzL4E+fXdhz/zNbqvniY1u4cmYZ//vRWRFdEdPR1c1Xn9h62j6+WelOpozIZao/sJ8xMo8pI/IozYv+OvxgPPfOMYpzXcweWxi1c9Y0dbD47hc5f3IJ935iHsYYPv779bxd3ci/vraYwhilu1KRiGwyxswb7Did6auYcjh82yl+Z9UO3jncyLSyYazf5+brf9pG5fgifnTtjIgvgcxIc/Lz62ZTOb6IhtZOpo7M44yReYwpzI7J8stQXRKD3HlpXga3LJ7Ij/65izfeq6Oh1cOre9x8+8ppGvDjlM70Vcw1tHo497vPc/Wccj51/niu/tVrlORl8JdbFkZ1g3EVmjZPNxf5P601tHXicjpY9cULBt2kRdlLZ/oqYRRku1g2q5yn3zzMy7trcaU5ePCmSg34CSLL5eSrl07h609uA+ChT1ZqwI9j+i+j4sInFlTQ1tmNu9nDfSvOYUxRdDdAUeG5es5oKscXsWxWGYumlMZ6OGoAOtNXceGs8ny+dcWZnFWer025EpDTITz26fno6sz4p0FfxY1PL0q+3ZRSSSJd9E5lmt5RSqkUokFfKaVSSFBBX0SWiMhOEdkjIrf38XiGiDzuf3y9iIwLeGyGiKwTke0i8paIaH9VpZSKkUGDvog4gXuAy4FpwPUiMq3XYZ8C6o0xk4CfAj/wPzcNeAT4rDFmOrAY6LRt9EoppYYkmJl+JbDHGLPPGOMBHgOW9jpmKfCg//aTwMXi67J0KbDNGLMVwBjjNsZEZtsipZRSgwom6JcDhwK+r/Lf1+cxxpgu4ARQDEwBjIisFpHNInJbXycQkZtFZKOIbKypqRnq30EppVSQIn0hNw04H7jB/+eHROTi3gcZY35rjJlnjJlXWqqFHUopFSnBBP1qYEzA96P99/V5jD+Pnw+48X0qeMkYU2uMaQVWAXPCHbRSSqnQBFOctQGYLCLj8QX364CP9TpmJbAcWAdcA7xgjDEishq4TUSyAQ9wIb4Lvf3atGlTrYgEt3Fq30qA0LZCig4dX3h0fOHR8YUnnscX1KYTgwZ9Y0yXiNwKrAacwH3GmO0ichew0RizEvg98LCI7AHq8P1iwBhTLyI/wfeLwwCrjDHPDnK+sPI7IrIxmE5zsaLjC4+OLzw6vvDE+/iCEVQbBmPMKnypmcD77gi43Q5c289zH8G3bFMppVSMaUWuUkqlkGQM+r+N9QAGoeMLj44vPDq+8MT7+AYVdztnKaWUipxknOkrpZTqhwZ9pZRKIQkZ9MPp+hmFsY0RkRdF5B1/Z9Ev9nHMYhE5ISJb/F939PVaER7nfn/X0y0ictpO9OLzc/97uE1EolZUJyJTA96bLSLSKCJf6nVMVN9DEblPRI6LyNsB9xWJyBoR2e3/s7Cf5y73H7NbRJZHcXx3i8i7/n+/p0Skzy3JBvtZiOD4vi0i1QH/hlf089wB/79HcHyPB4xtv4hs6ee5EX//bGWMSagvfLUCe4EJgAvYCkzrdczngN/4b18HPB7F8Y0C5vhv5wG7+hjfYuBvMX4f9wMlAzx+BfB3QID5wPoY/nsfBSpi+R4Ci/BVk78dcN8Pgdv9t28HftDH84qAff4/C/23C6M0vkuBNP/tH/Q1vmB+FiI4vm8DXwvi33/A/++RGl+vx38M3BGr98/Or0Sc6YfT9TPijDFHjDGb/bebgB2c3qAuESwFHjI+rwMFIjIqBuO4GNhrjAmnSjtsxpiX8BUeBgr8OXsQWNbHUy8D1hhj6owx9cAaYEk0xmeM+afxNUAEeB1fC5WY6Of9C0Yw/9/DNtD4/LHjI8Af7T5vLCRi0A+n62dU+dNKs4H1fTy8QES2isjfRWR6VAfmY4B/isgmEbm5j8eDeZ+j4Tr6/88W6/dwhDHmiP/2UWBEH8fEy/v4SXyf3Poy2M9CJN3qTz/d1096LB7evwuAY8aY3f08Hsv3b8gSMegnBBHJBf4MfMkY09jr4c340hUzgV8AT0d7fMD5xpg5+DbH+byILIrBGAYkIi7gKuBPfTwcD+9hD+P7nB+X659F5FtAF/BoP4fE6mfh18BEYBZwBF8KJR5dz8Cz/Lj/vxQoEYN+OF0/o0JE0vEF/EeNMX/p/bgxptEY0+y/vQpIF5GSaI3Pf95q/5/HgafwfYwOFMz7HGmXA5uNMcd6PxAP7yFwzEp5+f883scxMX0fRWQF8EHgBv8vptME8bMQEcaYY8aYbmOMF/hdP+eN9fuXBlwNPN7fMbF6/0KViEG/p+unfyZ4Hb4un4Gsrp8Q0PUzGoPz5/9+D+wwxvykn2NGWtcYRKQS379DNH8p5YhInnUb3wW/t3sdthK40b+KZz5wIiCVES39zrBi/R76Bf6cLQee6eOY1cClIlLoT19c6r8v4kRkCXAbcJXxtTbv65hgfhYiNb7Aa0Qf6ue8wfx/j6RLgHeNMVV9PRjL9y9ksb6SHMoXvpUlu/Bd1f+W/7678P1wA2TiSwnsAd4AJkRxbOfj+5i/Ddji/7oC+Cy+vYIBbgW241uJ8DqwMMrv3wT/ubf6x2G9h4FjFHx7I+8F3gLmRXmMOfiCeH7AfTF7D/H98jmCb4/nKnz7QhcDzwO7geeAIv+x84D/C3juJ/0/i3uAm6I4vj348uHWz6G1oq0MX8fbfn8WojS+h/0/W9vwBfJRvcfn//60/+/RGJ///gesn7mAY6P+/tn5pW0YlFIqhSRiekcppVSINOgrpVQK0aCvlFIpRIO+UkqlEA36SimVQjToK6VUCtGgr5RSKeT/AxYn+g7gTQWjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(len(eval_losses)), eval_losses)\n",
    "plt.title('test loss')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'test acc')"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd8m+W5+P/P5SkvyfGIR3ZIQggkJCEEyi50QKGQBOiBLlZLFz0dpwNKD79z6KAt7bfn9LSHAi27h9Ew2zLL7GBl2NnDSSBesR0ntiU7li3p/v2hR46ieMjWo2H5er9efll6lm7Jsi7d47pvMcaglFJKZSS7AEoppVKDBgSllFKABgSllFIWDQhKKaUADQhKKaUsGhCUUkoBGhCUUkpZNCCoCUNE3hORD9lwnatF5O92lEmpVKIBQSmlFKABQU0QIvIgMB34k4h4ROQ71vZTReSfItIhIrUick7YOVeLyG4RcYvIHhH5lIgcB/wW+IB1nY4hHu8aEdlqnbtbRL4Qsf8SEakRkS4R2SUi51vbS0TkXhFpEpGDIvJUnF4SpY4iOnWFmihE5D3gc8aYv1r3pwAbgM8AzwPnAY8A84EeoBk42RizXUSqgBJjzGYRudq6zhnDPNaFwDZgN3AW8BxwhjFmnYgsB14CLgNeBqqAImPMNhH5C+ABvmj9Ps0Y87q9r4RSg8tKdgGUSqJPA88aY5617r8kImuAjwGrgQBwgojsNcY0EwwQUTHG/CXs7usi8iJwJrAOuA64xxjzkrW/EcAKOhcApcaYg6Fzx/bUlBo9bTJSE9kM4HKruajDav45A6gyxnQD/0Lwm3qziPxFROZHe2ERuUBE3hKRA9Z1PwaUWbunAbsGOW0acCAsGCiVUBoQ1EQS2T5aDzxojCkO+ykwxvwEwBjzgjHmwwSbdLYBdw9xnSOISC7wOPBzoMIYUww8C0jY4x4zyKn1QImIFI/huSkVMw0IaiJpAWaH3X8I+LiIfFREMkXEISLniMhUEamwOn4LAC/B9vxA2HWmikjOEI+TA+QCbYBPRC4APhK2//fANSJynohkiMgUEZlvNUs9B/yviEwSkWwROcu2Z6/UCDQgqInkNuD7VvPQt4wx9cAlwPcIfnjXA98m+H+RAXwTaAIOAGcDX7Ku8wqwGdgnIvsjH8QY4wb+FXgMOAh8EngmbP87wDXAL4FOgv0EM6zdnwH6CdZIWoGv2/TclRqRjjJSSikFaA1BKaWURQOCUkopQAOCUkopiwYEpZRSwDjLVC4rKzMzZ85MdjGUUmpcWbt27X5jTPlIx42rgDBz5kzWrFmT7GIopdS4IiLvR3OcNhkppZQCNCAopZSyaEBQSikFaEBQSill0YCglFIK0ICglFLKogFBKaUUoAFBKaVSWktXL//5p830+wMjHxwjDQhKKZWiXtveygX//Tcefbeerc1dcX+8cZWprJRSE0G/P8AvXtzBb1/fxfzKIn79yaXMmVwY98fVgKCUUimkseMQ//rweta+f5BPnjKdWy5agCM7MyGPrQFBKaVSxEtbWvjWH2vxBwy/unIJF59YndDH1z4EpVRc7W3v4bev70pIp+h41ecL8IM/b+HzD6xhWkkef/7qGQkPBqA1BKVUHAUChq8/up51ezvY1erhZ5ctQkSSXayUUn+ghxv+bx21DZ1cfdpMbvrYfHKzEtNEFEkDglJprqWrl7f3HODji6oS/mG8el0D6/Z2cMqsEv64toEpk/L4+ofmJbQMqezZjc189/ENCPDbTy/l/BOqkloeDQhKpbmH39nLf/11J5kiXLgocR84nT39/OS5bZw0YxIPf/5Uvr16A//1151UF+fxiWXTElaOVNTb7+dHf9nKg2+9z4nTivn1lUuYVpKf7GJpQFAq3bV0eQH4/57ZxGnHlDKpICchj/uLl7bT0dPHrZcsJyNDuG3VQlq6evneExupdDo4a96IC3ilpT37u/nKH9axpbmLz585i29/dD45WanRnZsapVBKxU2b20tpQQ4dPf3c+uctCXnMTY2dPPTW+3z2AzM5vtoFQE5WBnd8Ojie/ksPrWVzU2dCypJKnq5p5KJf/Y2mzkP8/qpl3HzhgpQJBqABQam01+bxsqDayZc/OIcn1zfy6rbWuD5eIGD496c3UVKQwzc+fGR/QZEjm/uuWY4zL5tr7n2Xxo5DcS1LqjjU5+fGxzfwtUdqOK7KybP/eibnHVeR7GIdRZuMlEpz+91ejikv4IYPzuH5Tc1878mNvPiNsyhyZMfl8VavbWD93g5+cfmJuPKOfoxKl4N7rzmZy+94k2vufYc/fvG0QY9LFXWtbh56a29Mw2bf2XOAna0evnzOMXzzw/PIykzN7+IaEJRKY8YY2txeyotyycnK4GeXnciq//0Htz23jR+vXGj743X09PGT57dx8sxJrFo6Zcjj5lc6ufMzJ3HVve/whQfXcP+1y5M21HI4z25sHkgUK3KM/ePSmZfN/dcu5+wU7zfRgKBUGus65KPPH6C8MBeAxdOKue6MWdz9tz1ctKiK044ps/Xxfv7idjoP9XPrJSeMOMT1tDll/OyyRXzj0Vq+s3oDv/zEYjIyUiNHwecPcPsL27nzjd0smV7MHZ86iUqXI9nFirvUrLcopWzR5ukFoLwod2DbNz98LDNL87nx8Y309Plse6yNDZ384e29fPYDMziuyhnVOSuXTOXbHz2Wp2uauP3F7baVJRbtHi+fvecd7nxjN585dQaPXv+BCREMQAOCUmmt1R0cchoeEPJyMvnJpYvYe6CHX7y4w5bHCQQM3396E6UFuUd1JI/ky+ccw5XLp3PHa7t46K33bSnPWNXWd/Dx//k7a94/yO2XLeIHK05IqVFA8TZxnqlSE1CbFRAmhwUEgFNnl/LpU6dzzz/2sG7vwZgf57E19dTWd3DzhfNxjrKzWkT4wSXHc+78ydzy9Cb+uqUl5vKMxSPv7OXy376JiPDEl07j8gmYPKcBQak0FgoI5YVHN3l89/z5VDkdfGf1Brw+/5gf42B3Hz99fhvLZ5awYvHQHcnDycrM4H+uXMLx1S6++vB6aus7xlye0fL6/Nz0xAZufGIjp8wu4c9fPYMTprgS9vipRAOCUmmszeMlJzMDZ97R40eKHNn8eNVC6lo9/PqVujE/xu0vbqer18etK46Paa6kgtwsfn/1MkoLc7ju/nfZ294z5mtFq6njEJ/47Zs8/E49Xz7nGO67ZnnCMrlTkQYEpdJYaMjpUB/U5xw7mVVLp3DHa7vY0jT6JRpr6zt4+J29XH3aTOZXRteRPJzJRQ7uu2Y5/X7D1fe+w8HuvpivOZR/7trPx//n7+xq6+a3nz6J75w/n8wUGeWULFEFBBE5X0S2i0idiNw4yP4ZIvKyiGwQkddEZGrYvp+JyGYR2SoivxLrnSkiJ4nIRuuaA9uVUvZpc3spi+g/iHTLRQsozs/h26trR5V85bcykssKc/n6h+bGWtQBcyYX8rurltHQcYjPPbCG3v6xN2cNxhjDXW/s4tO/e5vi/Gye+srpnH9Cpa2PMV6NGBBEJBP4DXABsAC4UkQWRBz2c+ABY8wi4FbgNuvc04DTgUXACcDJwNnWOXcAnwfmWj/nx/pkVHp6ZVsL//7UpqQtsPL8pmauvvcd/uOZzTzyzl7W7z1It9e+4Zrx1Ob2DuQgDKU4P4cfXHI8m5u6uOuN3VFf+9F369nQ0Mn3LzzO9qznk2eW8MtPLGbd3oN849Ea/AFjy3W7vT5u+L/1/PjZbXz0+EqevuGMhKxVPF5Ek5i2HKgzxuwGEJFHgEuA8FmyFgDftG6/Cjxl3TaAA8gBBMgGWkSkCnAaY96yrvkAsAJ4LqZno9KOzx/g35/aTGPHIfr9AW5btTChc/q/vbudrz68nkn5Obyz5wA9fYe/rU4ryePYCifzK4s4trKI+ZVFzCwrIDuFpiXY7/GyZPqkEY+7YGEVH1tYyX+/vJOPHl854ofkge4+fvbCNk6ZVRK3lb0uXFRFc+dx/PAvW/neExv52KIqygpzKCvMpaQgZ9Sv8+42D194cC272jzceMF8vnDWbF2sJ0I0AWEKUB92vwE4JeKYWmAV8N/ASqBIREqNMW+KyKtAM8GA8GtjzFYRWWZdJ/yagw5PEJHrgesBpk+fHkVxVTr5y8ZmGjsOcfqcUh55t57Z5QVcf9YxCXns9/Z384WH1jKtJJ8nv3Q6RY4sGg4eYtu+Lrbvc7Otxc2OfW5e3d468A02JzOD2eUFVpA4HCyqXI6Ef/j4/AHau/uOyEEYzn9efAL/3PU63318A4994QPDtqff/sI23L0+frBi5IzkWHzuzNk0d/by+7/v4dE19UfsK87PprQgGCDKCnMptYJFaWEOpQW5lBcFf5cV5fLPuv3822O1ZGUKD153CqfPsTdDO13YNXXFt4Bfi8jVwBtAI+AXkTnAcUCoT+ElETkTiHqKQ2PMXcBdAMuWLbOn3qjGhWBb726OKS/ggWtP4asPr+O257Yxs7SAjxwf3zbfzp5+rr3vXQS49+qTceUHm0Sml+YzvTT/iMf3+vzsau1me0sX2/a52b7PzTt7DvBUTdPAMfMri3jua2cmNCgc6O7DGKIOCOVFudxy0QK++VgtD7z5HtecPmvQ49bvPcgj79bzuTNmMa+iyMYSD+7fL1rA1afNpNXdS5u7j/ZuL/ut3+2ePto8Xrbt62K/p4/OQ/1DXmfhFBd3fHopUyclfyGaVBVNQGgEwjM0plrbBhhjmgjWEBCRQuBSY0yHiHweeMsY47H2PQd8AHiQw0Fi0Gsq9c9d7Wxu6uInqxaSmSH84vLFNB58k689UsMfv/iBuI0V7/cH+NIf1lJ/sIeHrjuFGaUFwx6fm5XJgmonC6qPHGXTeaifHS1u/vDW+zxV00RXry+hs3oOZCmP0IcQbuWSKTxT28TPnt/Oh46rOGoVL3/AcMvTm5lclMvXErgU5rSS/KhWFOvzBTjY00eb20t7dx/73V7au71kZmTwqVOm48hOvQn0Ukk0jXDvAnNFZJaI5ABXAM+EHyAiZSISutZNwD3W7b3A2SKSJSLZBDuUtxpjmoEuETnVGl30WeBpG56PSiN3vrGbssJcViwJtibm5WRy91XLmJSfzefuX8O+zl7bH9MYwy1Pb+Kfu9r5yapFnDK7dMzXcuVlc/LMEs615r1v6bK/vMNp8xw9bcVIRIQfrwwG4Jue2IgxR1bKH35nLxsbO7n5wgUU5qbe3Jg5WRlUOB2cMMXF2fPKufSkqVx/1jFcd8YsDQZRGDEgGGN8wA3AC8BW4DFjzGYRuVVELrYOOwfYLiI7gArgR9b21cAuYCPBfoZaY8yfrH1fBn4H1FnHaIeyGrC1uYs3drRxzekzj/hHnlzk4HdXnYy7t5/PPfCurZOzAfzub3t4+J16vvLBY7j0pKkjnxCFSmcwSzgeAWw4Q01bMZLq4jxuvGA+f6/bz2Nh7fbtHi+3v7CdD8wu5eMJXJtZJU5UId4Y8yzwbMS2W8Juryb44R95nh/4whDXXENwKKpSR7n7jd3k52Ty6VNmHLVvQbWTX125hM8/sIZvPlrL/35qqS3TJr+0pYUfP7eVjy2s5N8+fGzM1wupcAY/kBNeQ7ACQtkomoxCPrl8On+qbeKHf97K2fMmU+ly8LPnt9Pt9XHrJbFlJKvUlTrj45SyNHUc4pnaJv7l5GkDnbmRzjuugpsvXMDzm/fxsxdinzZ5c1MnX3tkPYumuPjF5fbOy19h1RCSERCKcrPIyxl9U0lGhvDTSxfRHwjw/ac2sm7vQR5dU891Z8xibgI6klVypF4joJrw7v3HHgxw3RmDj3IJufb0mexu8/Db13cxu7yAT4xxdsqWrl6uu28NxXnZ3P3ZZWP6AB2OIzuT4vxs9iWhD2E0/QeRZpYV8K2PHMsP/7KV9Xs7qHQ6+Op59mUkq9SjNQSVUrp6+3n4nXouXFg14vBAEeE/Lj6eM+eWcfOTG3lrd/uoH6+nz8fn7l9DV28/v7vqZCY747MQSqXTwb5Ob1yuPZRopq0YyTWnz2LxtGLau/v4/kXHpWRHsrKPBgSVUv7v7b14vD6uP2t2VMdnZ2bw608uZUZpAV98aC179ndH/ViBgOGbj9ayqamTX12x5Khho3aa7HTQ6k5sDWG/O7YaAkBmhnDHp5fy88tP5MKF2pGc7jQgqJTR5wtw7z/2cPqc0lHlGLjysrnnqpPJEOG6+96loye6GTJvf3E7z2/ex80fO44PLagYa7GjUunMTfwoI8/I8xhFo8qVx2UnTdWO5AlAA4JKGU/XNNLS5R3T1BTTS/O58zMn0XDwEF96aB19vuEnwntsTT13vLaLT54yfcS+CjtUOh3s93jxJWiCvt5+P+5eX8w1BDWxaEBQKcEYw91/2838yiLOmju2eWZOnlnCTy9byJu72/n3pzYdlVQV8tbudm5+ciNnzCnjPy9OzBDKyU4HAQP7PfGb3z9c2yBrKSs1Eg0IKiW8tr2NHS0ero9xBsqVS6by1XPn8Oiaeu7+29FTOe/Z380XH1rL9JJ8fvOppQmbmXQgOS1BI43GkqWslAYElRLufGMXVS4HH7dhKuVvfGgeFy6q4rbntvHC5n0D2zt6+rjOmrDunqtPTui8QpWuxGYrt41hHiOlNCCopNvQ0MFbuw9w7emzbPnGnpEh/OLyE1k0tZivP1LDpsZO+nwBvvTQOuoP9nDnZ5aNOGGd3RKdnDbWaSvUxKYBQSXdnW/spig3iyuWjy2xbDCO7Ezu/uxJlBQEF2z/tz/W8ubu4IR1y2eV2PY40SotyCErQxIaEESgZAIvGK9GTwOCSqq97T08t7GZT5463fZlGIMT4S3D0+vjT7VN3PDBObZNWDdaGRnC5KLchPYhlBbkkJVCq7ep1Kdphyqpfv/33WRmCNcOsRhLrI6rcnLvNct5970DfOnsxKy0NpQKlyOhNYSxTGqnJjYNCCppDnb38diaBi5ZPGWgjT0els8qSUozUaSKIgd1bZ6EPFabDVnKauLR+qRKmgffep9D/f6op6kY7ypdDloSOMpIA4IaLQ0IKil6+/3c/8/3+OCx5QlZlzcVVDgduL0+ur32LuoTyRgT80ynamLSgKCS4vF1DbR3941pmorxqtIV/ICOd8dyV6+PPl9AcxDUqGlAUAnnDxh+97c9LJrq4tTZyW/bT5SKosTkIui0FWqsNCCohHtpSwt79nfHPE3FeFPh0oCgUpsGBJVwd72xi2kleZx/fGWyi5JQoZFU8V4oJzSPkWYpq9HSgKASas17B1i3t4PPnTF7wiVNFeZmUZiblbgaQmH8hvKq9DSx/iNV0t35xm6K87O5fFlyMoaTrcKZm5CAkJOZgTNP04zU6GhAUAmzq83DX7e28NlTZ5CfMzE/rCpdjriPMgrlIEyk/hllDw0IKmF+97fd5GRm8NnTZia7KElT4Yx/clqbx0uZ9h+oMdCAMA5sauzkte2tyS5GTNrcXh5f18ilJ02d0HPsVDgdtLq9BAKDr+Zmhza3l/JCneVUjZ4GhHHgO6s38L0nNia7GDG5/5/v0e8P8PkzJ8Y0FUOpdDrwBQzt3fFbSlOnrVBjFVVAEJHzRWS7iNSJyI2D7J8hIi+LyAYReU1EplrbPygiNWE/vSKywtp3n4jsCdu32N6nlh42N3WypbmLfV29CVug3W7dXh8PvvU+H1lQwayyxC5Mk2rivVCOP2A40O3VLGU1JiMGBBHJBH4DXAAsAK4UkQURh/0ceMAYswi4FbgNwBjzqjFmsTFmMXAu0AO8GHbet0P7jTE1sT+d9PP42kYAAubw+PLxZvXaBjoP9U+oaSqGUuEMflDHKyC0d3sJGE1KU2MTTQ1hOVBnjNltjOkDHgEuiThmAfCKdfvVQfYDXAY8Z4zpGWthJ5o+X4CnahqZlB9cOKapIzEzZdrtH3X7mV1WwEkzJiW7KEk3sLZynAKCZimrWEQTEKYA9WH3G6xt4WqBVdbtlUCRiJRGHHMF8HDEth9ZzUy/FJFB38Eicr2IrBGRNW1tbVEUN328tr2VA919fM5qd0/UAu1229nq4djKiTGj6UjKC3PJEOI20kgDgoqFXZ3K3wLOFpH1wNlAI+AP7RSRKmAh8ELYOTcB84GTgRLgu4Nd2BhzlzFmmTFmWXl5uU3FHR8eX9dAWWEu/3JycK3h5s5DSS7R6PX2+3m/vZu5kwuTXZSUkJWZQVlh/JbS1CxlFYtoAkIjEL76+VRr2wBjTJMxZpUxZglws7WtI+yQTwBPGmP6w85pNkFe4F6CTVPK0u7x8vLWVlYuqaa0IIe87Eyax2ENYc/+bgIG5kyQNQ+iUeF00NIVn/6gUD9TWZEOO1WjF01AeBeYKyKzRCSHYNPPM+EHiEiZiISudRNwT8Q1riSiuciqNSDBdMoVwKbRFz99PVPbhC9guPSkqYgIVcWOcVlD2NHiBmBehdYQQoIBIX41hMLcrAmbCa5iM2JAMMb4gBsINvdsBR4zxmwWkVtF5GLrsHOA7SKyA6gAfhQ6X0RmEqxhvB5x6T+IyEZgI1AG/DCmZ5JmVq9tYOEUF/MrnQBUuRzjsoZQ1+ohM0Mm/HDTcJWu+DYZaf+BGquovkYYY54Fno3YdkvY7dXA6iHOfY+jO6Exxpw7moJOJFuautjc1MV/Xnz8wLYqVx5/37k/iaUamx0tbmaU5pOblZnsoqSMiiIHHT399Pb7cWTb+7oEs5Q1IKix0UzlFPT4ugayM4WLT6we2FblctDqHn/JaTtbPdqhHCG0UE5rHPoRdC1lFQsNCCmm3x/gqfWNfOi4CiYVHO4YrHLlETDQ6h4/yWlen5/323uYO1k7lMNVOuOXi6BNRioWGhBSzGvb22jv7uPSpUeuF1BlfascTx3Le/Z34w8Y5mqH8hHilZzW2+/H3evTgKDGTANCilm9tp6ywhzOPvbInIuq4lBAGD8dyztbPABaQ4hQURRqMrL3b3k4B0EDghobDQgp5EB3H69sa2XF4ilkRywvWeXMA6B5HE1fsbPFTYbA7HIdYRTOmZeFIzvD9szzUA6C1hDUWGlASCHP1DTS7w/mHkRy5mWRnzO+ktN2tnqYUVpg+0ia8U5EqHTav3KaTluhYqUBIYWsXtfACVOcHFflPGqfiFi5COOnD2Fnq4c5OsJoUJPjkJymAUHFSgNCitja3MWmxi4uWzr04vNVrrxxU0Po8wV4b3+3ZigPoTIO01e0ub2IQEmBTluhxkYDQop4fK2Ve7D4qBy+AeOphvBeeze+gNEO5SFUuoJNRsbYt5Rmm8dLSX7OUf1PSkVL3zkpoN8fXPfgvPkVw367CyaneekfB8lpAyOMtIYwqAqngz5fgI6e/pEPjpLmIKhYaUBIAa9vb2O/p4/LBulMDldVnIcZJ8lpO1rciMAx5RoQBjOwcprbviZADQgqVhoQUsDqtQ2D5h5EGkhoGgfNRnWtHqaX5OsIoyEMZCvb2Cek8xipWGlASLID3X28vK1l0NyDSNWuYC7CeFhKc0eLW/sPhlFhBQS7RhoZY9iv8xipGGlASLLhcg8iHa4hpHZA6PcH2LO/W/sPhjHZajLa12lP85/b68PrC2hAUDHRgJBkj69r5PjqwXMPIjkdWRTkZNKU4k1G71sjjHTI6dByszIpKcixrQ9BcxCUHTQgJNG2fV1sbOwcsTM5RESCwxVTvIawQ+cwikqF00GLTX9LncdI2UEDQhKFcg8uGSb3IFJ1cR5NKR4QdrZ4dIRRFCqd9q2cpjUEZQcNCEnS7w/w5Pomzp0/eVSZpZVOB80dqd1ktKPVzbRJ+eTl6Aij4VTYmK2sAUHZQQNCkryxo439Hi+XnTRtVOdVFefR5knt5LS6Fl0lLRoVTgft3fb8Lds8XrIzBVdetg0lUxOVBoQkWb22gdKCHM4ZIfcgUrXLgTH2DVe0m88fYPd+D3MrtP9gJJXW39KORMNQDoKI2FAyNVFpQEiCg919/HVrCyuWjJx7ECnVh56+195Dv99oDSEKdianaZaysoMGhCR4praJfr+JenRRuOpiKzktRQNCXasb0DmMohHKRbBj5TQNCMoOGhCSYPXahqhzDyKl+vQVoSGnug7CyAZqCHYEBM1SVjbQgJBg2/e52djYyaXDrHswHKcjm8LcrJSdvmJnq4epk/LIz8lKdlFSXklBDtmZEnNA8AcM7R6dx0jFTgNCgj2+roGsDOGSxdVjvkYqJ6ftbHEzTzuUoyIiTC5y0Brj0NMD3X0EjA45VbHTgJBAPn+AJ9Y1cu78yZTG8G0uVRfK8fkD7G7r1g7lUbAjuGsOgrKLBoQEemNnKPdgbM1FIcGAkHo1hL0HeujzB7T/YBQqbVhbuc2jAUHZI6qAICLni8h2EakTkRsH2T9DRF4WkQ0i8pqITLW2f1BEasJ+ekVkhbVvloi8bV3zURFJ+4VgQ7kHH5w/OabrVLmCyWl9vtRKTgt1KGuTUfQqnLEvpXl4HiOHXcVSE9SIAUFEMoHfABcAC4ArRWRBxGE/Bx4wxiwCbgVuAzDGvGqMWWyMWQycC/QAL1rn/BT4pTFmDnAQuM6G55OyDnb38dctrVwSxboHI6kaSGhKrVpCaMjpMVpDiFqFM5eePj8er2/M1wgFhLKitP9OpeIsmk+m5UCdMWa3MaYPeAS4JOKYBcAr1u1XB9kPcBnwnDGmR4LplOcCq6199wMrRlv48eRPG5ro8wdibi6C4PQVQMo1G+1s9TClOI/CXB1hFK3QMOJYmo3a3F4Kc7N0ZJeKWTTvoClAfdj9BuCUiGNqgVXAfwMrgSIRKTXGtIcdcwXw/6zbpUCHMSb0tajBepyjiMj1wPUA06dPj6K49rvz9V08sa6RqZPymFaSz/SS/LDf0Q2xXL22gQVVThZUjz73IFKV9SGSagFhR4tHE9JGqWIgW9nLnDFOF645CMoudn2l+BbwaxG5GngDaAT8oZ0iUgUsBF4Y7YWNMXcBdwEsW7Zs7A2tMXhsTT3uXh8i8Nbudrr7/EfsLyvMORwgJgV/Ty3JY3pJPlWuPOpaPWxo6OSWiyJb2sZmICCk0Kyn/oBhV5uHM+aUJrso44odS2m2uXuUdi07AAAbWklEQVQ1B0HZIpqA0AiET8k51do2wBjTRLCGgIgUApcaYzrCDvkE8KQxpt+63w4Ui0iWVUs46pqpoqu3n11t3fzbh+fx1fPmYozhQHcf9QcPsfdAD/XWz94DPax9/yB/3tCMP3A4bmVnCvk5WTHnHoQrcmRTlJuVUjWEvQd66PMFdFK7UbIjW7nN7WV+Zew1T6WiCQjvAnNFZBbBD+0rgE+GHyAiZcABY0wAuAm4J+IaV1rbATDGGBF5lWC/wiPAVcDTY30S8bSxoROAxdOLgWAyUWlhLqWFuSyeVnzU8f3+AM0dvdQfDAaJ0M/CKa6Ycg8iVaZYLsLOFmsOI+1QHpW8nEycjqyY+xDOnKs1BBW7EQOCMcYnIjcQbO7JBO4xxmwWkVuBNcaYZ4BzgNtExBBsMvpK6HwRmUmwhvF6xKW/CzwiIj8E1gO/j/nZxEFNfbCis2jK0R/+g8nOzGB6aT7TS/M5PY7lqirOS6kaws5WncNorGJJTuvt99PV69M+BGWLqPoQjDHPAs9GbLsl7PZqDo8Yijz3PQbpMDbG7CY4giml1dR3MLusAFd+ai08UuV0sLW5K9nFGLCzxU21y0GRI7Vep/GgwumgZYxrIuz36FrKyj6aqTwMYww19R2cOEjTULJVFTvYn0LJaTtbPczR/oMxqXA6aBljDUGnrVB20oAwjH1dvbS5vZw41ZXsohylKoVWTvMHDHWtHuZpc9GYVDodtHm8RwxGiNZAUprWEJQNNCAMo9bqP0jJGoIrdZLTGg724PUFNAdhjCqcuQNTWI+WzmOk7KQBYRg19Z1kZ4otyWR2O5yclvyRRjsHFsXRJqOxqIhh6GmohlBaqNNWqNhpQBhGbX0HC6qc5GZlJrsoR0ml6St26LKZMYllnew2t9daaEf/lVXs9F00BH/AsKEhNTuUAQpzsyjKzUqJhXLqWjxUOh04dYTRmFTGkK3c5taV0pR9NCAMYVebh+4+PydOTc2AAMGRRk0pMH3FzladwygWpYW5ZGYILWNYOU3nMVJ20oAwhJoU7lAOqXTl2bJAeywC1gijudp/MGaZGUJ5Ye6Y+xA0ICi7aEAYQm19B0WOLGaXFSS7KEOqdjlo6khuQGjsOMShfr/WEGJU4Rr9ymnGGA0IylYaEIZQ29DBiVOLyciQZBdlSFWuvKQnp+2w5jCapwEhJhVFuaMOCG6vD68voH0IyjYaEAbR2+9nW7ObE6elXkJauCobFleJ1eE5jLTJKBZjmc9Is5SV3TQgDGJzUye+gEnpDmUIdipDcoee7mzxUOHMxZWnI4xiUeF00NXr41DEWhvD0YCg7KYBYRA19daU1yncoQypkZy2s9WtHco2GMu6CBoQlN00IAyitr6DKpeDydY/aaqqtKavSFbHcmiEkU55HbuxrJw2EBC0D0HZRAPCIEIdyqmuMDeLIkcW+5JUQ2jsOERPn595OstpzCpdwQ/1UQUEj5fsTNHmOmUbDQgRDnb38X57z8AKaamu2pVHU5L6EOqsDmUdchq7gfmMRvG3bHN7KSvMTemRcGp80YAQoabBSkgbBzUEiG21rVjtbNVlM+1SmJtFfk7mqLKVNQdB2U0DQoTa+g5EYGEKroEwmOri5K2tvKPFQ3lRLsX5OtNmrESESufoktN0HiNlNw0IEWrrO5g7uZDC3KhWF026Smce+z19eH3RD1e0y85Wj9YObFThdIxulJHOY6RspgEhjDGG2obOcdNcBIdzEVo6x7Ym71gZY6hrcWuHso1G0/znDxgOdPdpQFC20oAQpuHgIQ5096X0hHaRkpWL0NTZS3efX4ec2miyM5dWdy/GjLyU5sGePvwBowFB2UoDQpjQDKepnpAWLllLae4cmMNIawh2qXQ66PcHv/mPRHMQVDxoQAhTU99BblYGx1aOnw+5wzWERAcEa8ip1hBsM5psZc1SVvGgASFMbX0HJ0xxjavlCAtys3A6shLeZLSz1U1ZYQ6TCnSEkV1CmfGtUQw91YCg4mH8fPLFWb8/wKam8dWhHFJdnJf4GoIuimO7gbWVo6kheIIBoUybjJSNNCBYdrS46e0PpPyU14OpdCU2FyE4wkiXzbTb5KJcRKLLVm5zeynIyaRgnAyPVuNDVAFBRM4Xke0iUiciNw6yf4aIvCwiG0TkNRGZGrZvuoi8KCJbRWSLiMy0tt8nIntEpMb6WWzXkxqL2nEyw+lgqlx5Cc1W3tfVi9vrY652KNsqOzOD0oLoFsrRLGUVDyMGBBHJBH4DXAAsAK4UkQURh/0ceMAYswi4FbgtbN8DwO3GmOOA5UBr2L5vG2MWWz81MTyPmNXUH2RSfjbTS/KTWYwxqXI5EpqctkM7lOOmwqkBQSVPNDWE5UCdMWa3MaYPeAS4JOKYBcAr1u1XQ/utwJFljHkJwBjjMcb02FJym9XWd3LitGJExt9EYaGRRomqJYSGnGpAsF+l08G+aDqVNUtZxUE0AWEKUB92v8HaFq4WWGXdXgkUiUgpMA/oEJEnRGS9iNxu1ThCfmQ1M/1SRAZ9d4vI9SKyRkTWtLW1RfWkRsvj9bGj1T0uO5Qh8bkIda0eSgtyKNUOTdtVuKKbz0jnMVLxYFen8reAs0VkPXA20Aj4gSzgTGv/ycBs4GrrnJuA+db2EuC7g13YGHOXMWaZMWZZeXm5TcU90qbGTowZn/0HEL6UZmI6lne0uDVDOU4qihwc6B6++c/r89N5qF9rCMp20QSERmBa2P2p1rYBxpgmY8wqY8wS4GZrWwfB2kSN1dzkA54Cllr7m02QF7iXYNNUUtRaGcqLxskMp5ESmZxmjGFnq0czlOMktFDOcLkI+z3BTGYNCMpu0QSEd4G5IjJLRHKAK4Bnwg8QkTIRCV3rJuCesHOLRST01f5cYIt1TpX1W4AVwKZYnkgsahs6mFaSN26bQPJzsnDlZdOcgKU0W7q8uHt9OuQ0TqJZSlOT0lS8jBgQrG/2NwAvAFuBx4wxm0XkVhG52DrsHGC7iOwAKoAfWef6CTYXvSwiGwEB7rbO+YO1bSNQBvzQtmc1SjV7O1g8bVKyHt4WVS5HQmoIoUVxtMkoPqJJTjs8j1Fqr/mtxp+oslqMMc8Cz0ZsuyXs9mpg9RDnvgQsGmT7uaMqaZy0dvXS1NnLteO0uSikKkHJaaE5jLTJKD4qikI1hKGbjLSGoOJlwmcq1zaM34S0cFXFiUlO29nqZlJ+NqU6h1FcFOdnk5OVEVWTUWmh/g2UvTQg1HeQmSEcXz3OawhOB+3dffT2xzc5bWeLh7kVReMyX2M8CC2lOVxwb/P0UlKQM64mYVTjw4R/R9U2dHBsRRF5OZkjH5zCqoqDuQijWZN3tIwx7Ghxa0JanFU4c0fsQ9AcBBUPEzogBAKG2voOFk8f381FcHjoaVMcRxq1ub109fo0IMRZhdNB60gBQfsPVBxM6ICwp72brl4fi8dphnK4gekruuLXsbyzVTuUEyE4fcXQS2nqtBUqXiZ0QAglpI2nNZSHEpq+Ip41hB3WHEZzNAchripdDnr7A3Qd8h21zxijNQQVNxM+IOTnZKbFmPq8nEyK87PjOtJoZ6sHV162tl/HWWjltBb30X9Lj9dHb39A/wYqLiZ0QKhp6GThFBeZGekxYqbSGd9chJ0tbuZVFOoIozgbWFt5kOCuOQgqniZsQPD6/Gxt6hr3+Qfh4rmUZnCEkYc5umxm3A0EhEE6ljUgqHiasAFhW7ObPn8grQJCZRynr9jv6aPzUD/ztP8g7iY7gx/2LYPVEDwaEFT8TNiAUJNGHcoh1a7g1MnxSE47vCiO1hDizZEd7A8arA/h8DxGGhCU/SZsQKit76C8KHdguGY6qLRGGsWjYzk05FRnOU2MYLby0fMZtbm9ZGcKrrzsJJRKpbsJGxBqGjo4cer4XDJzKNWh5LQ4dCzvaHHjdGQxWZsqEqLCOfjKaW1uL2WFuWSkyUAIlVomZEDoPNTP7rZuFk8b3/MXRQpNXxGvGoLOYZQ4Fc7cwQOCJxgQlIqHCRkQNloznKZT/wEcHp0Sj47lulaPdignUKXTwX6PF58/cMR2TUpT8TQhA0JN/UEAFqXBlBXh8nIymZSfbXsuwn6PlwPdfTrkNIEqXA4C5vCoohCd2E7F0wQNCJ3MLi9Iy465Slee7UtphhbF0UntEie0UE54858/YGjv7tMagoqbCRcQjDHU1HekxYR2g6mOQy5CnbVspk5qlzihpTTDV0472NOHP2A0IKi4mXABobmzl/0eb9r1H4RUxmEpzS3NXRTlZlHh1A+iRKkIzWcU1rGsWcoq3iZcQEinGU4HU12cx8GeftuS0/r9AV7c3MKZ88p0hFEClRbkkJUhR0xfoQFBxduECwg1DR3kZGZwXFV6Nn/YPdLojR1ttHf3sXLJVFuup6KTkSFMLsodvIagncoqTiZeQNjbwXHVTnKzxveSmUOpKg4FBHuajZ5c38ik/GzOnlduy/VU9CpcRyan6TxGKt4mVEDwBwwbGztZPDW9EtLChRbKsWOkUVdvPy9taeHjJ1aTkzWh3iopITh9xZE1hPycTApys5JYKpXOJtR/eV2rh54+f9r2H0D4UpqxB4TnN+7D6wuwcsmUmK+lRi84fcXhUUaalKbibUIFhHTvUIbgTJmT8rNp6oi9yeiJ9Q3MKitIqynCx5MKpwOP14fHG1xKU5PSVLxNqIBQ09BBkSOLWaUFyS5KXFW5Yl8op7HjEG/tPsDKJVN0dFGSVLqsdRGs2l6bR2sIKr6iCggicr6IbBeROhG5cZD9M0TkZRHZICKvicjUsH3TReRFEdkqIltEZKa1fZaIvG1d81ERybHrSQ2ltj44w2m6zxRZXRx7ctpT6xsBWLFYm4uSZSAXwfpbapORircRA4KIZAK/AS4AFgBXisiCiMN+DjxgjFkE3ArcFrbvAeB2Y8xxwHKg1dr+U+CXxpg5wEHgulieyEgO9fnZts89IZo/Yk1OM8bw5PpGls2YxPTSfBtLpkZjICC4e/H6/HQe6tcmIxVX0dQQlgN1xpjdxpg+4BHgkohjFgCvWLdfDe23AkeWMeYlAGOMxxjTI8E2iHOB1dY59wMrYnomI9jc1Ik/YNK6/yCkypVHR08/h/rGlpy2uamLulYPK5dq7SCZBtZW7vTS7ukDdMipiq9oAsIUoD7sfoO1LVwtsMq6vRIoEpFSYB7QISJPiMh6EbndqnGUAh3GGN8w17TVwJKZaTzkNCQ00mistYQn1jWSk5nBRQur7SyWGqWC3CyKcrNo6erVLGWVEHZ1Kn8LOFtE1gNnA42AH8gCzrT2nwzMBq4ezYVF5HoRWSMia9ra2sZcwNqGTqpdDiY702fJzKFUxbCUps8f4JnaJs6dPxlXfvrNBjveTHbmsq9TA4JKjGgCQiMwLez+VGvbAGNMkzFmlTFmCXCzta2D4Df/Gqu5yQc8BSwF2oFiEcka6pph177LGLPMGLOsvHzs2bK19R0TorkIDtcQmsYQEP5Wt5/9Hq82F6WISpeDFnevZimrhIgmILwLzLVGBeUAVwDPhB8gImUiErrWTcA9YecWi0jok/xcYIsxxhDsa7jM2n4V8PTYn8bw2j1e9h7omTABITR18r4xNBk9ua6R4vxsPnjsZLuLpcagwumgJayGUFqgAUHFz4gBwfpmfwPwArAVeMwYs1lEbhWRi63DzgG2i8gOoAL4kXWun2Bz0csishEQ4G7rnO8C3xSROoJ9Cr+37VlF2GAtmTkRRhhBMDmtpCBn1DUEj9fHi1v2cdGiKp2qIkVUOh20ur20dPUyKT9b/y4qrqKaFMUY8yzwbMS2W8Jur+bwiKHIc18CFg2yfTfBEUxxV1PfQYbAwinp36EcUuVyjLoP4bmNzfT261QVqaTC6cAXMGzf59bmIhV3E+LrRm1DB3MnF02oScGqXI5RT1/x5PpGZpTms3T6pDiVSo1WKBdhc1OXBgQVdxPiE/IbH5pH56H+ZBcjoapceax5/2DUxzd3HuLN3e3867lzdaqKFBLqDzrU79ekNBV3EyIgTJTO5HCVLsdAclpezshrPzxd04QxaHNRiglftlRrCCreJkST0URUPYqFcowxPLmukaXTi5lZlt4T/4035YW5hKbe0oCg4k0DQpoaWCgnio7lLc1dbG9xs3KpLpOZarIyMyizmoo0IKh404CQpgaS06LoWH5yXSPZmcJFC6viXSw1BqF+hPLC9M+yV8mlASFNVQxMjDZ8DcHnD/B0bRPnHDuZSQVxn4FcjcHkIisgaA1BxZkGhDTlyM6kNIrktH/saqfN7WWVdianrNBCORoQVLxpQEhjVcWOEaeveHJdA05HFucep1NVpKql0ycxZ3IhxXk62aCKrwkx7HSiqnTm0XCwZ8j93V4fL2xuYcWSKeRmjTw0VSXHqqVTWaUd/ioBtIaQxkZaSvOFzfs41O9nlc5sqpRCA0Jaq3Q56DzUT0+fb9D9T65vZFpJHstm6FQVSikNCGmtephchJauXv5Rt5+Vi6foVBVKKUADQloLjV9v7jg6IDxd00jAwAodXaSUsmhASGOHawhHjzR6Yl0jJ04rZnZ5YaKLpZRKURoQ0liFNX49ssloa3MX2/a5NfdAKXUEDQhpLDcrk7LCnKMCwpPrG8nKED5+YnWSSqaUSkUaENJclSvviCYjf8DwdE0j5xxbTolOVaGUCqMBIc1VRiyl+eaudlq6vKxcoolOSqkjaUBIc9URS2k+sb6BIkcW5+lUFUqpCDp1RZqrdOXR1euj2+tDBJ7ftI+LT6zGka1TVSiljqQBIc0dXjmtl02NnfT0+TX3QCk1KG0ySnOVzsNLaT6xvpEpxXksn1mS5FIppVKRBoQ0V10cTE7b0NDJ33e2sWJJNRkZOlWFUupoGhDS3GRnMDntvn++R8Cgo4uUUkPSgJDmgslpubS5vSya6mLOZJ2qQik1OA0IE0CVNcndSu1MVkoNI6qAICLni8h2EakTkRsH2T9DRF4WkQ0i8pqITA3b5xeRGuvnmbDt94nInrB9i+15SipSlctBpk5VoZQawYjDTkUkE/gN8GGgAXhXRJ4xxmwJO+znwAPGmPtF5FzgNuAz1r5DxpihPuy/bYxZPfbiq2hcfdpMzpxXTlmhLtKulBpaNDWE5UCdMWa3MaYPeAS4JOKYBcAr1u1XB9mvkui0OWV85tQZyS6GUirFRRMQpgD1YfcbrG3haoFV1u2VQJGIlFr3HSKyRkTeEpEVEef9yGpm+qWIDPr1VUSut85f09bWFkVxlVJKjYVdncrfAs4WkfXA2UAj4Lf2zTDGLAM+CfyXiBxjbb8JmA+cDJQA3x3swsaYu4wxy4wxy8rLy20qrlJKqUjRBIRGYFrY/anWtgHGmCZjzCpjzBLgZmtbh/W70fq9G3gNWGLdbzZBXuBegk1TSimlkiSagPAuMFdEZolIDnAF8Ez4ASJSJiKha90E3GNtnxRqChKRMuB0YIt1v8r6LcAKYFPsT0cppdRYjTjKyBjjE5EbgBeATOAeY8xmEbkVWGOMeQY4B7hNRAzwBvAV6/TjgDtFJEAw+PwkbHTSH0SkHBCgBviijc9LKaXUKIkxJtlliNqyZcvMmjVrkl0MpZQaV0RkrdWXOyzNVFZKKQVoQFBKKWUZV01GItIGvD/G08uA/TYWx25avtho+WKj5YtNqpdvhjFmxHH74yogxEJE1kTThpYsWr7YaPlio+WLTaqXL1raZKSUUgrQgKCUUsoykQLCXckuwAi0fLHR8sVGyxebVC9fVCZMH4JSSqnhTaQaglJKqWFoQFBKKQWkYUCIYrnPXBF51Nr/tojMTGDZponIqyKyRUQ2i8jXBjnmHBHpDFta9JZElc96/PdEZKP12EfNEyJBv7Jevw0isjSBZTs27HWpEZEuEfl6xDEJff1E5B4RaRWRTWHbSkTkJRHZaf2eNMS5V1nH7BSRqxJYvttFZJv193tSRIqHOHfY90Icy/cfItIY9jf82BDnDvu/HsfyPRpWtvdEpGaIc+P++tnOGJM2PwQn39sFzAZyCC7csyDimC8Dv7VuXwE8msDyVQFLrdtFwI5ByncO8OckvobvAWXD7P8Y8BzBSQlPBd5O4t96H8GEm6S9fsBZwFJgU9i2nwE3WrdvBH46yHklwG7r9yTr9qQEle8jQJZ1+6eDlS+a90Icy/cfwLei+PsP+78er/JF7P8FcEuyXj+7f9KthhDNcp+XAPdbt1cD51lTcMedCa4Bsc667Qa2cvTqc6nuEoLrZxtjzFtAcWgq8wQ7D9hljBlr5rotjDFvAAciNoe/x+4nOL17pI8CLxljDhhjDgIvAecnonzGmBeNMT7r7lsE1zhJiiFev2hE878es+HKZ31ufAJ42O7HTZZ0CwjRLPc5cIz1T9EJlJJgVlPVEuDtQXZ/QERqReQ5ETk+oQUDA7woImtF5PpB9kfzGifCFQz9j5jM1w+gwhjTbN3eB1QMckyqvI7XEqzxDWak90I83WA1ad0zRJNbKrx+ZwItxpidQ+xP5us3JukWEMYFESkEHge+bozpiti9jmAzyInA/wBPJbh4ZxhjlgIXAF8RkbMS/PgjkuBCTRcDfxxkd7JfvyOYYNtBSo7tFpGbAR/whyEOSdZ74Q7gGGAx0EywWSYVXcnwtYOU/1+KlG4BYcTlPsOPEZEswAW0J6R0wcfMJhgM/mCMeSJyvzGmyxjjsW4/C2RLcLW5hDCHlzxtBZ7k6KVNo3mN4+0CYJ0xpiVyR7JfP0uLHF4RsApoHeSYpL6OInI1cBHwKStoHSWK90JcGGNajDF+Y0wAuHuIx03265cFrAIeHeqYZL1+sUi3gDDicp/W/dCIjsuAV4b6h7Cb1eb4e2CrMeb/DXFMZahPQ0SWE/wbJSRgiUiBiBSFbhPsfIxc2vQZ4LPWaKNTgc6w5pFEGfKbWTJfvzDh77GrgKcHOeYF4CMSXGZ2EsHX+oVEFE5Ezge+A1xsjOkZ4pho3gvxKl94n9TKIR43mv/1ePoQsM0Y0zDYzmS+fjFJdq+23T8ER8HsIDgC4WZr260E3/wADoJNDXXAO8DsBJbtDILNBxsILhtaY5X3i8AXrWNuADYTHDXxFnBaAss323rcWqsModcvvHwC/MZ6fTcCyxL89y0g+AHvCtuWtNePYGBqBvoJtmNfR7BP6mVgJ/BXoMQ6dhnwu7Bzr7Xeh3XANQksXx3B9vfQezA06q4aeHa490KCyveg9d7aQPBDviqyfNb9o/7XE1E+a/t9ofdc2LEJf/3s/tGpK5RSSgHp12SklFJqjDQgKKWUAjQgKKWUsmhAUEopBWhAUEopZdGAoJRSCtCAoJRSyvL/A/qsGwZGnblTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(len(eval_acces)), eval_acces)\n",
    "plt.title('test acc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到我们的三层网络在训练集上能够达到 99.9% 的准确率，测试集上能够达到 98.20% 的准确率"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习：看一看上面的训练过程，看一下准确率是怎么计算出来的，特别注意 max 这个函数**\n",
    "\n",
    "**自己重新实现一个新的网络，试试改变隐藏层的数目和激活函数，看看有什么新的结果**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}