machinelearning_notebook/6_pytorch/0_basic/1-Tensor-and-Variable.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensor and Variable\n",
    "\n",
    "PyTorch的简洁设计使得它入门很简单，在深入介绍PyTorch之前，本节将先介绍一些PyTorch的基础知识，使得读者能够对PyTorch有一个大致的了解，并能够用PyTorch搭建一个简单的神经网络。部分内容读者可能暂时不太理解，可先不予以深究，后续的课程将会对此进行深入讲解。\n",
    "\n",
    "本节内容参考了PyTorch官方教程[^1]并做了相应的增删修改，使得内容更贴合新版本的PyTorch接口，同时也更适合新手快速入门。另外本书需要读者先掌握基础的Numpy使用，其他相关知识推荐读者参考CS231n的教程[^2]。\n",
    "\n",
    "[^1]: http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html\n",
    "[^2]: http://cs231n.github.io/python-numpy-tutorial/\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 把 PyTorch 当做 NumPy 用\n",
    "\n",
    "PyTorch 的官方介绍是一个拥有强力GPU加速的张量和动态构建网络的库，其主要构件是张量，所以我们可以把 PyTorch 当做 NumPy 来用，PyTorch 的很多操作好 NumPy 都是类似的，但是因为其能够在 GPU 上运行，所以有着比 NumPy 快很多倍的速度。通过本次课程，你能够学会如何像使用 NumPy 一样使用 PyTorch，了解到 PyTorch 中的基本元素 Tensor 和 Variable 及其操作方式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个 numpy ndarray\n",
    "numpy_tensor = np.random.randn(10, 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用下面两种方式将numpy的ndarray转换到tensor上"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "pytorch_tensor1 = torch.Tensor(numpy_tensor)\n",
    "pytorch_tensor2 = torch.from_numpy(numpy_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用以上两种方法进行转换的时候，会直接将 NumPy ndarray 的数据类型转换为对应的 PyTorch Tensor 数据类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同时我们也可以使用下面的方法将 pytorch tensor 转换为 numpy ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 如果 pytorch tensor 在 cpu 上\n",
    "numpy_array = pytorch_tensor1.numpy()\n",
    "\n",
    "# 如果 pytorch tensor 在 gpu 上\n",
    "numpy_array = pytorch_tensor1.cpu().numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要注意 GPU 上的 Tensor 不能直接转换为 NumPy ndarray，需要使用`.cpu()`先将 GPU 上的 Tensor 转到 CPU 上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch Tensor 使用 GPU 加速\n",
    "\n",
    "我们可以使用以下两种方式将 Tensor 放到 GPU 上"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第一种方式是定义 cuda 数据类型\n",
    "dtype = torch.cuda.FloatTensor # 定义默认 GPU 的 数据类型\n",
    "gpu_tensor = torch.randn(10, 20).type(dtype)\n",
    "\n",
    "# 第二种方式更简单，推荐使用\n",
    "gpu_tensor = torch.randn(10, 20).cuda(0) # 将 tensor 放到第一个 GPU 上\n",
    "gpu_tensor = torch.randn(10, 20).cuda(0) # 将 tensor 放到第二个 GPU 上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用第一种方式将 tensor 放到 GPU 上的时候会将数据类型转换成定义的类型，而是用第二种方式能够直接将 tensor 放到 GPU 上，类型跟之前保持一致\n",
    "\n",
    "推荐在定义 tensor 的时候就明确数据类型，然后直接使用第二种方法将 tensor 放到 GPU 上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "而将 tensor 放回 CPU 的操作非常简单"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "cpu_tensor = gpu_tensor.cpu()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也能够访问到 Tensor 的一些属性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([10, 20])\n",
      "torch.Size([10, 20])\n"
     ]
    }
   ],
   "source": [
    "# 可以通过下面两种方式得到 tensor 的大小\n",
    "print(pytorch_tensor1.shape)\n",
    "print(pytorch_tensor1.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.FloatTensor\n",
      "torch.cuda.FloatTensor\n"
     ]
    }
   ],
   "source": [
    "# 得到 tensor 的数据类型\n",
    "print(pytorch_tensor1.type())\n",
    "print(gpu_tensor.type())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "# 得到 tensor 的维度\n",
    "print(pytorch_tensor1.dim())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n"
     ]
    }
   ],
   "source": [
    "# 得到 tensor 的所有元素个数\n",
    "print(pytorch_tensor1.numel())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习**\n",
    "\n",
    "查阅以下[文档](http://pytorch.org/docs/0.3.0/tensors.html)了解 tensor 的数据类型，创建一个 float64、大小是 3 x 2、随机初始化的 tensor，将其转化为 numpy 的 ndarray，输出其数据类型\n",
    "\n",
    "参考输出: float64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "float64\n"
     ]
    }
   ],
   "source": [
    "# 答案\n",
    "x = torch.randn(3, 2)\n",
    "x = x.type(torch.DoubleTensor)\n",
    "x_array = x.numpy()\n",
    "print(x_array.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor的操作\n",
    "Tensor 操作中的 api 和 NumPy 非常相似，如果你熟悉 NumPy 中的操作，那么 tensor 基本是一致的，下面我们来列举其中的一些操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1.],\n",
      "        [1., 1.],\n",
      "        [1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones(3, 2)\n",
    "print(x) # 这是一个float tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.FloatTensor\n"
     ]
    }
   ],
   "source": [
    "print(x.type())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1, 1],\n",
      "        [1, 1],\n",
      "        [1, 1]])\n"
     ]
    }
   ],
   "source": [
    "# 将其转化为整形\n",
    "x = x.long()\n",
    "# x = x.type(torch.LongTensor)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1.],\n",
      "        [1., 1.],\n",
      "        [1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "# 再将其转回 float\n",
    "x = x.float()\n",
    "# x = x.type(torch.FloatTensor)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-1.8509,  0.5228, -1.3782],\n",
      "        [ 0.9726,  1.7519, -0.3425],\n",
      "        [-0.0131,  2.1198, -1.1388],\n",
      "        [ 0.2897,  1.2477, -0.2862]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(4, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 沿着行取最大值\n",
    "max_value, max_idx = torch.max(x, dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0.5228, 1.7519, 2.1198, 1.2477])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 每一行的最大值\n",
    "max_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1, 1, 1, 1])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 每一行最大值的下标\n",
    "max_idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([-2.7063,  2.3820,  0.9679,  1.2512])\n"
     ]
    }
   ],
   "source": [
    "# 沿着行对 x 求和\n",
    "sum_x = torch.sum(x, dim=1)\n",
    "print(sum_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 1, 1, 4, 3])\n",
      "torch.Size([1, 1, 1, 1, 4, 3])\n",
      "tensor([[[[[[-1.8509,  0.5228, -1.3782],\n",
      "            [ 0.9726,  1.7519, -0.3425],\n",
      "            [-0.0131,  2.1198, -1.1388],\n",
      "            [ 0.2897,  1.2477, -0.2862]]]]]])\n"
     ]
    }
   ],
   "source": [
    "# 增加维度或者减少维度\n",
    "print(x.shape)\n",
    "x = x.unsqueeze(0) # 在第一维增加\n",
    "print(x.shape)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 1, 4, 3])\n"
     ]
    }
   ],
   "source": [
    "x = x.unsqueeze(1) # 在第二维增加\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 3])\n",
      "tensor([[-1.8509,  0.5228, -1.3782],\n",
      "        [ 0.9726,  1.7519, -0.3425],\n",
      "        [-0.0131,  2.1198, -1.1388],\n",
      "        [ 0.2897,  1.2477, -0.2862]])\n"
     ]
    }
   ],
   "source": [
    "x = x.squeeze(0) # 减少第一维\n",
    "print(x.shape)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 3])\n"
     ]
    }
   ],
   "source": [
    "x = x.squeeze() # 将 tensor 中所有的一维全部都去掉\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 4, 5])\n",
      "torch.Size([4, 3, 5])\n",
      "torch.Size([5, 3, 4])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(3, 4, 5)\n",
    "print(x.shape)\n",
    "\n",
    "# 使用permute和transpose进行维度交换\n",
    "x = x.permute(1, 0, 2) # permute 可以重新排列 tensor 的维度\n",
    "print(x.shape)\n",
    "\n",
    "x = x.transpose(0, 2)  # transpose 交换 tensor 中的两个维度\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 4, 5])\n",
      "torch.Size([12, 5])\n",
      "torch.Size([3, 20])\n"
     ]
    }
   ],
   "source": [
    "# 使用 view 对 tensor 进行 reshape\n",
    "x = torch.randn(3, 4, 5)\n",
    "print(x.shape)\n",
    "\n",
    "x = x.view(-1, 5) # -1 表示任意的大小，5 表示第二维变成 5\n",
    "print(x.shape)\n",
    "\n",
    "x = x.view(3, 20) # 重新 reshape 成 (3, 20) 的大小\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(3, 4)\n",
    "y = torch.randn(3, 4)\n",
    "\n",
    "# 两个 tensor 求和\n",
    "z = x + y\n",
    "# z = torch.add(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外，pytorch中大多数的操作都支持 inplace 操作，也就是可以直接对 tensor 进行操作而不需要另外开辟内存空间，方式非常简单，一般都是在操作的符号后面加`_`，比如"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 3])\n",
      "torch.Size([1, 3, 3])\n",
      "torch.Size([3, 1, 3])\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones(3, 3)\n",
    "print(x.shape)\n",
    "\n",
    "# unsqueeze 进行 inplace\n",
    "x.unsqueeze_(0)\n",
    "print(x.shape)\n",
    "\n",
    "# transpose 进行 inplace\n",
    "x.transpose_(1, 0)\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]])\n",
      "tensor([[2., 2., 2.],\n",
      "        [2., 2., 2.],\n",
      "        [2., 2., 2.]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones(3, 3)\n",
    "y = torch.ones(3, 3)\n",
    "print(x)\n",
    "\n",
    "# add 进行 inplace\n",
    "x.add_(y)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习**\n",
    "\n",
    "访问[文档](http://pytorch.org/docs/0.3.0/tensors.html)了解 tensor 更多的 api，实现下面的要求\n",
    "\n",
    "创建一个 float32、4 x 4 的全为1的矩阵，将矩阵正中间 2 x 2 的矩阵，全部修改成2\n",
    "\n",
    "参考输出\n",
    "$$\n",
    "\\left[\n",
    "\\begin{matrix}\n",
    "1 & 1 & 1 & 1 \\\\\n",
    "1 & 2 & 2 & 1 \\\\\n",
    "1 & 2 & 2 & 1 \\\\\n",
    "1 & 1 & 1 & 1\n",
    "\\end{matrix}\n",
    "\\right] \\\\\n",
    "[torch.FloatTensor\\ of\\ size\\ 4x4]\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1  1  1  1\n",
      " 1  2  2  1\n",
      " 1  2  2  1\n",
      " 1  1  1  1\n",
      "[torch.FloatTensor of size 4x4]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 答案\n",
    "x = torch.ones(4, 4).float()\n",
    "x[1:3, 1:3] = 2\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variable\n",
    "tensor 是 PyTorch 中的完美组件，但是构建神经网络还远远不够，我们需要能够构建计算图的 tensor，这就是 Variable。Variable 是对 tensor 的封装，操作和 tensor 是一样的，但是每个 Variabel都有三个属性，Variable 中的 tensor本身`.data`，对应 tensor 的梯度`.grad`以及这个 Variable 是通过什么方式得到的`.grad_fn`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 通过下面这种方式导入 Variable\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_tensor = torch.randn(10, 5)\n",
    "y_tensor = torch.randn(10, 5)\n",
    "\n",
    "# 将 tensor 变成 Variable\n",
    "x = Variable(x_tensor, requires_grad=True) # 默认 Variable 是不需要求梯度的，所以我们用这个方式申明需要对其进行求梯度\n",
    "y = Variable(y_tensor, requires_grad=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = torch.sum(x + y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(-6.3913)\n",
      "<SumBackward0 object at 0x7f8a18db6e10>\n"
     ]
    }
   ],
   "source": [
    "print(z.data)\n",
    "print(z.grad_fn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面我们打出了 z 中的 tensor 数值，同时通过`grad_fn`知道了其是通过 Sum 这种方式得到的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.]])\n",
      "tensor([[1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "# 求 x 和 y 的梯度\n",
    "z.backward()\n",
    "\n",
    "print(x.grad)\n",
    "print(y.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过`.grad`我们得到了 x 和 y 的梯度，这里我们使用了 PyTorch 提供的自动求导机制，非常方便，下一小节会具体讲自动求导。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习**\n",
    "\n",
    "尝试构建一个函数 $y = x^2 $，然后求 x=2 的导数。\n",
    "\n",
    "参考输出：4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "提示：\n",
    "\n",
    "$y = x^2$的图像如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnB0lEQVR4nO3dd3hUVf7H8feZyaRDAkkIhCSEEFpAekcUBbvoYqPYcFXsZd2iq/tTd11dy9pdC9ZVKVbEiqKgINICRFoChIQ0IIUQkpBCMnN+fyS6igSGkMm5M/N9PU8eyWSY+VwDn1zOPfccpbVGCCGEddlMBxBCCHFkUtRCCGFxUtRCCGFxUtRCCGFxUtRCCGFxAZ540ejoaJ2UlOSJlxZCCJ+0du3aUq11zOG+5pGiTkpKIi0tzRMvLYQQPkkpldvc12ToQwghLE6KWgghLE6KWgghLE6KWgghLE6KWgghLE6KWgghLE6KWgghLM4yRV1b7+Tlpdn8sKPUdBQhhDhmSzKLeX15DgcbXK3+2pYp6gCb4uVl2by6LMd0FCGEOGYvfLeD//6wE4ddtfprW6eo7TYuGhrPkq3F7NlfazqOEEK4LbukitU5ZVwyPAGlfLioAS4ZloBLw/tr801HEUIIt72Tlo/dprhoSLxHXt9SRZ0UHcbo5CjeScvH5ZItwoQQ1lfvdPHB2gJO7dOJTu2DPfIelipqgKkjEsgvq2FF9l7TUYQQ4qi+ySimtOogU4cneOw9LFfUZ/TrTESIg7mr80xHEUKIo5q3Jo/O7YM5uddhVyhtFZYr6mCHncmDu/LV5iLKDhw0HUcIIZq1q7yG77aVcPGweALsnqtTyxU1wJThCRx0upi/vtB0FCGEaNZ7aQVo3TgRwpMsWdR9u7RnYEIk76zJQ2u5qCiEsB6nS/NuWj4npkST0DHUo+9lyaIGmDo8gW1FVazPLzcdRQghfmN5VimF5TVM8eBFxJ9YtqgnDYwjNNDOPLmoKISwoHlr8ugQ6uD0frEefy/LFnV4UACTBsTxyY+7qaytNx1HCCF+VlpVx6ItRVwwJJ6gALvH38+yRQ0wZUQCNfVOPvlxt+koQgjxs/nrCql36jYZ9gCLF/XghEh6x7Zj3hoZ/hBCWIPWmrmr8xiSGEmv2HZt8p6WLmqlFNNHJrKhYD8bCspNxxFCCFZk7yW79ACXjuzWZu9p6aIGmDykKyEOO3NWyVm1EMK82avyiAhxcM6ALm32npYv6vbBDs4bGMeC9F1UyEVFIYRBJZV1fLlpDxcNjSfY4fmLiD+xfFEDXDoqkZp6JwvkTkUhhEHvrc2nwaWZPjKxTd/XK4p6QHwk/bu2Z/YquVNRCGGGy6WZsyqPUckd6RET3qbv7VZRK6X+oJTarJTapJSaq5TyzKKrR3DpyG5k7qlkXd6+tn5rIYRg6fYSCvbVtOlFxJ8ctaiVUl2BW4FhWuv+gB2Y6ulghzpvYBzhQQHMXikXFYUQbW/2qjyiwgI5o1/nNn9vd4c+AoAQpVQAEArs8lykwwsLCmDy4K58unE35dWy/KkQou3s3l/D4sxiLhmeQGBA248YH/UdtdaFwL+BPGA3sF9r/dWhz1NKzVRKpSml0kpKSlo/KTB9ZCIHG1y8v7bAI68vhBCH886afJwuzbThbXsR8SfuDH10AM4HugNxQJhS6rJDn6e1nqW1Hqa1HhYT45mdDvp2ac+QxEjmyEVFIUQbaXC6mLc6n5N6xZAY5dnlTJvjzjn8RCBHa12ita4HPgTGeDZW8y4d2Y3s0gOyp6IQok0szixmT0Utl7bxlLxfcqeo84BRSqlQpZQCJgAZno3VvHMGdCEixCEXFYUQbWL2qjxi2wcxoU8nYxncGaNeBbwPrAM2Nv2eWR7O1axgh52Lh8bz5eY9FFXUmoohhPADO0sP8N22EqaNSPTonohH49Y7a63v01r30Vr311pfrrWu83SwI7lsVDecWsv6H0IIj3prZS4BNsX0EeaGPcBL7kw8VFJ0GCf3imHO6jwONrhMxxFC+KDqgw28l5bPmf0706l9m9/j9yteWdQAV4zu1rhAyuY9pqMIIXxQ40JwDVwxOsl0FO8t6pN7dSKxYyhvrcg1HUUI4WO01ry5Ipc+ndsxPKmD6TjeW9R2m+KyUYms3llGxu4K03GEED5kbe4+MnZXcMXoJBonu5nltUUNcMmwBIICbLwpZ9VCiFb03xW5tAsO4HeD40xHAby8qCNDAzl/UBwfrS9kf41sKiCEOH7FFbV8sXE3Fw9NIDQwwHQcwMuLGuCK0UnU1Dtl/Q8hRKuYu7pxc4DLR7f9cqbN8fqi7t81giGJkby9MheXS9b/EEK0XL3TxZzVuZzUK4bu0WGm4/zM64sa4MoxSeSUHmBZVqnpKEIIL/bV5iKKKuq40kJn0+AjRX1m/85Ehwfy5g87TUcRQnixN1fsJL5DCON7m1vX43B8oqiDAuxMH5HI4q3F7Cw9YDqOEMILbd61n1U5ZVwxuht2m/kpeb/kE0UNjet/BNgUb8hZtRCiBV5fvpPQQDtThpld1+NwfKaoO7UP5twBcbyXlk9FrUzVE0K4r6Syjo/Td3HhkHgiQh2m4/yGzxQ1wO/HdufAQSfvpclUPSGE++asyuOg08WMsUmmoxyWTxX1CfERDE/qwBs/5OCUqXpCCDfUNTh5a2Uup/SOoUdMuOk4h+VTRQ1w1dju5JfV8HVGkekoQggv8OmPuymtquOqsd1NR2mWzxX16amxdI0M4fXlOaajCCEsTmvNa8tzSOkUzrie0abjNMvnijrAbuOK0d1YmV3G5l37TccRQljYmp372LyrgqvGWmOVvOb4XFEDTB2eSIjDzhvLd5qOIoSwsNeX5xAR4uCCwfGmoxyRTxZ1RKiDi4bGsyB9F6VVRrd3FEJYVH5ZNV9u3sP0kYmEBNpNxzkinyxqgBljkzjodDF7pWyAK4T4rTdX7EQpxeWjrLWux+H4bFH3iAlnfO8Y3lqZS22903QcIYSFVNU1MG9N48a1cZEhpuMclc8WNcC145IprapjQXqh6ShCCAuZtzqPytoGZo5LNh3FLT5d1GN6RJHapT0vL8uRtaqFEAA0OF28vnwnI7p3ZGBCpOk4bvHpolZKMfOkZLKKq/huW4npOEIIC/h80x4Ky2u85mwafLyoAc4Z0IUuEcHMWpptOooQwjCtNbOW7iA5JoxT+1hrzekj8fmidtht/H5sd1Zk72VjgdwAI4Q/W5ldxqbCCq4dl4zNYmtOH4nPFzXA1BEJtAsK4OVlclYthD97eVk20eGBTB7c1XSUY+IXRd0u2MG0kYl8tnE3BfuqTccRQhiwvaiSxZnFXDE6iWCHtW9wOZRfFDXAjDFJKBp3cRBC+J9XluUQ7LBxmRfc4HIovynquMgQJg2MY97qPPbXyA4wQviT4spa5q8v5OKhCXQMCzQd55j5TVEDXDOucQeYuavltnIh/MmbP+RS73Jx9YnWXXP6SPyqqPvFRTA2JYrXl+dQ1yC3lQvhDw7UNfD2qlxOT40lKTrMdJwW8auiBrj+5B4UVdTx0Xq5rVwIfzB3dR7l1fVcd3IP01FazO+K+sSUaPp3bc+L32XLvopC+Li6BievLMthVHJHhiR2MB2nxdwqaqVUpFLqfaVUplIqQyk12tPBPEUpxY3jU8gpPcDCTXtMxxFCeNBH6wvZU1HLjeNTTEc5Lu6eUT8NLNRa9wEGAhmei+R5Z/TrTHJ0GC98l4XWclYthC9yujQvfZdN/67tLb0fojuOWtRKqQjgJOBVAK31Qa11uYdzeZTdprju5GQ2FVawbHup6ThCCA/4cvMesksPcMPJKZbeD9Ed7pxRdwdKgNeVUuuVUq8opX5z6VQpNVMplaaUSispsf5KdZMHx9O5fTDPf5tlOooQopVprXn+2yySo8M4s39n03GOmztFHQAMAV7QWg8GDgB3HfokrfUsrfUwrfWwmJiYVo7Z+gIDbFwzrjsrs8tYl7fPdBwhRCv6PquUTYUVXHdyMnYvWnypOe4UdQFQoLVe1fT5+zQWt9ebNiKRyFAHL3y7w3QUIUQren7JDmLbB/E7L1t8qTlHLWqt9R4gXynVu+mhCcAWj6ZqI2FBAVw5OolFW4rYVlRpOo4QohWsz9vHiuy9XDsumaAA71p8qTnuzvq4BZitlNoADAIe8liiNjZjTBIhDjsvylm1ED7h+W93EBHiYNqIRNNRWo1bRa21Tm8afx6gtf6d1tpnBnU7hAUybUQiC37cRd5eWQJVCG+2dU8li7YUceWYJMKCAkzHaTV+d2fi4fx0weGF72QGiBDe7NnF2wkLtPP7sUmmo7QqKWogtn0wU4cn8P7aAtlYQAgvlVVcyWcbd3PlmCQiQ71vKdMjkaJucn3Tgi0vfidj1UJ4o+cWZxHisHONF+0u7i4p6iZxkSFcNDSBd9cUsGd/rek4QohjkFN6gI9/3MVlo7p55cYARyNF/Qs3ju+BS2s5qxbCy/xnSRYOu41rffBsGqSofyWhYygXDOnK3NV5FFfIWbUQ3iBvbzXz1xdy6chuxLQLMh3HI6SoD3HTKSk0uDSzlmabjiKEcMPz32b9vNCar5KiPkS3qDDOHxTH26tyKa2qMx1HCHEEBfuqeX9tAdOGJxDbPth0HI+Roj6Mm05Joa7BxcvL5KxaCCt74dsdKIVXb7PlDinqw+gRE86kAXG8tSKXvXJWLYQl7Sqv4b20Ai4elkBcZIjpOB4lRd2MWyekUFvv5CUZqxbCkp5dnIVGc+N43z6bBinqZqV0asfvBnXlzRU7Ka6UGSBCWEne3mreS8tn2ohE4juEmo7jcVLUR3DrhJ7UOzXPL5F51UJYyTOLt2O3KW46xbs3rXWXFPURJEWHcdGQeOasymNXeY3pOEIIYEdJFR+uK+CyUd18eqbHL0lRH8UtE1LQaJ5bIivrCWEFT3+9naAAOzf4wdj0T6SojyK+QyhThyfy7pp8Wa9aCMO27qnkkw27mDE2iehw37wL8XCkqN1w0ykp2GyKZxZvNx1FCL/25KJthAUGMNNH1/RojhS1GzpHBHP5qG58uK6A7JIq03GE8EubCvezcPMerj6xOx18cIW8I5GidtMN43sQFGDnqa/lrFoIE55YtI2IEAdXj+tuOkqbk6J2U3R4EDPGJvHJhl1k7K4wHUcIv7I2dx+LM4uZeVIy7YMdpuO0OSnqY3DdScm0CwrgsS+3mo4ihN/QWvPIF5lEhwdxlY/theguKepjEBkayA3jU1icWcyq7L2m4wjhF5ZsLWb1zjJum9iT0EDf2Vn8WEhRH6MZY5KIbR/Ewwsz0VqbjiOET3O6NI98sZWkqFCmDk8wHccYKepjFBJo5w8Te7E+r5yvthSZjiOET/tofSFbiyr50xm9cdj9t67898iPw0VD4+kRE8ajCzNpcLpMxxHCJ9XWO3li0TZO6BrB2f27mI5jlBR1CwTYbfz5jD7sKDnAB+sKTMcRwie9vTKXwvIa7jqrDzabMh3HKCnqFjqjXyyDEyN5ctF2auudpuMI4VMqaut5bkkW43pGMzYl2nQc46SoW0gpxZ1n9mFPRS1v/LDTdBwhfMqs77Ipr67nzjP7mI5iCVLUx2FUchSn9I7h+SVZlFcfNB1HCJ9QXFHLq9/nMGlgHP27RpiOYwlS1MfpzrP6UFXXwDPfyDKoQrSGf3+1lQaXiz+d3st0FMuQoj5OfTq355JhCby5Yqcs2CTEcdq8az/vrS3gytFJdIsKMx3HMqSoW8Edp/ciKMDGw19kmo4ihNfSWvPgZxlEhji4ZUJP03EsRYq6FXRqF8yNp6Tw1ZYiVuyQW8uFaIlvMor5Ycdebp/Yi4gQ/1t46UikqFvJ1Sd2Jy4imH9+tgWXS24tF+JY1DtdPPR5BskxYUwfmWg6juVIUbeSYIedO8/qw+ZdFXy4vtB0HCG8yuyVuWSXHuCes/v69a3izXH7/4hSyq6UWq+U+tSTgbzZeQPjGJQQyWNfZlJ9sMF0HCGsbfZsSEpC22ycfs4o/lyaxql9OplOZUnH8qPrNiDDU0F8gVKK/zu3L0UVdcxamm06jhDWNXs2zJwJubkorYnbX8wNcx5BzZljOpkluVXUSql44BzgFc/G8X5Du3XknAFdeOm7bHbvrzEdRwhruuceqK7+1UO2mprGx8VvuHtG/RTwF6DZpeKUUjOVUmlKqbSSkpLWyOa17jqzDy6t+dfnMl1PiMPKyzu2x/3cUYtaKXUuUKy1Xnuk52mtZ2mth2mth8XExLRaQG+U0DGU607uwcc/7mKl7AQjxG8lNjOzo7nH/Zw7Z9RjgfOUUjuBecCpSqm3PZrKB9xwcg+6RoZw/8ebZc1qIQ5x8B8PUOsI+vWDoaHw4INmAlncUYtaa/1XrXW81joJmAos1lpf5vFkXi4k0M7/nZtK5p5K3l6ZazqOEJYyK34UfznjZmrj4kEp6NYNZs2CSy81Hc2SZMKiB53RL5ZxPaN5fNE2SqvqTMcRwhIKy2t4bkkW9VOmEVyYDy4X7NwpJX0Ex1TUWutvtdbneiqMr1FKcd+kftQcdPLYwq2m4whhCQ991jjL955z+hpO4j3kjNrDUjqFc/WJ3XknLZ/0/HLTcYQwanlWKZ9t3M1N41OI7xBqOo7XkKJuA7dM6EmndkHct2CTrAMi/Fa908V9H28msWMo156UbDqOV5GibgPhQQHcfXZffizYz7w1+abjCGHE68tzyCqu4t5zUwl22E3H8SpS1G3k/EFxjEruyMNfZFBSKRcWhX8p2FfNk4u2M7FvJyb0lfU8jpUUdRtRSvHg5BOorXfxz8+2mI4jRJvRWnPvgs0oBX8/vz9KKdORvI4UdRvqERPODeN7sCB9F0u3+fdt9sJ/LNy0h8WZxdxxWi+6RoaYjuOVpKjb2A3je5AcHcbfPtpEbb3TdBwhPKqytp77P9lMapf2zBiTZDqO15KibmPBDjv/nNyfvLJqnlssO5cL3/b4V9sorqzjXxecQIBsCNBi8n/OgDE9orlgSFdeWrqDbUWVpuMI4RE/5pfz3xU7uWJUNwYmRJqO49WkqA255+y+hAUFcM/8jTK3WvicBqeLv364kU7tgvjjGb1Nx/F6UtSGRIUHcffZfVmzcx9z18gavMK3vLY8hy27K7h/Uj/aB8uO4sdLitqgi4fGM6ZHFP/6PJPCctkNRviG7JIqHv9qGxP7xnJm/86m4/gEKWqDlFI8fMEAnC7NXz/ciNYyBCK8m9Ol+cv7GwgKsPHQZJkz3VqkqA1LjArlzjN7s3RbCe+vLTAdR4jj8uaKnaTl7uPeSf3o1D7YdByfIUVtAVeMTmJEUkce+HQLRRW1puMI0SK5ew/w6MKtjO8dw4VDupqO41OkqC3AZlM8ctEA6hpc3DNfhkCE93G5NHd+sIEAm+JfF5wgQx6tTIraIrpHh/HnM3rzdUYxC9J3mY4jxDGZvTqPldll3HNOX7pEyG3irU2K2kKuGtudIYmR3P/JZoorZQhEeIeCfdU8/HkG43pGM2V4guk4PkmK2kLsNsWjFw2k+qCTu2UWiPACLpfmz+9tAJAhDw+SoraYlE7h/KVpCEQ2GRBW9+r3OazI3su9k1Jlay0PkqK2oN+P7c7YlCge+HQLO0sPmI4jxGFl7K7gsS+3cnpqLJcMkyEPT5KitiCbTfHviwcSYFPc/k46DU6X6UhC/EptvZM/vJNO+xCHDHm0ASlqi+oSEcKDk08gPb+c/yzZYTqOEL/y+FdbydxTyWMXDSAqPMh0HJ8nRW1hkwbG8btBcTyzeDvr8/aZjiMEAD9klfLyshwuG5XIKX1k/8O2IEVtcX8/vz+x7YK4490fqT7YYDqO8HP7q+v543s/khwdxj1np5qO4zekqC0uIsTB45cMYufeA/zjE9kUV5ijtebujzZSUlnHk1MGERJoNx3Jb0hRe4HRPaK44eQezFuTz4L0QtNxhJ+aszqPzzbs5o7Te8mOLW1MitpL3HFaL4Z168DdH24ku6TKdBzhZ7bsquDvn2zhpF4xXH9SD9Nx/I4UtZcIsNt4ZtpgHAE2bpqzXnYwF22mqq6Bm+esIzLEwROXDMRmk6l4bU2K2ovERYbwxCUDydhdwT8/k/Fq4Xlaa/42fyM79x7gmWmDiZapeEZIUXuZU/vEMvOkZN5e2TheKIQnvZdWwEfpu7h9Yi9GJUeZjuO3pKi90J/P6M2ghEju+mADuXvlFnPhGduKKrn3402M6RHFTaekmI7j16SovZDDbuO56YNRCm54ex01B2W8WrSuytp6rn97LeFBATw1dRB2GZc2SoraS8V3COWpqYPI2FPBXz/cIEuiilbjcmnuePdHcvdW89z0IXRqJ3sfmiZF7cVO7RPLHRN78VH6Ll5fvtN0HOEjnluSxaItRfztnL4yLm0RRy1qpVSCUmqJUmqLUmqzUuq2tggm3HPTKSmcnhrLg59nsGLHXtNxhJf7JqOIJ7/exgWDuzJjTJLpOKKJO2fUDcAftdapwCjgJqWU3ORvETab4vFLBpIUFcrNc9ZRWF5jOpLwUtklVdw+L53ULu15SJYutZSjFrXWerfWel3TryuBDED2greQdsEOZl0xjLoGF9e/tVZuhhHHrKqugeveWkuAXfHS5UMJdsg6HlZyTGPUSqkkYDCw6jBfm6mUSlNKpZWUlLRSPOGuHjHhPDllEBsL98t+i+KYuFyaP76bzo6SKv4zfYhsqWVBbhe1Uioc+AC4XWtdcejXtdaztNbDtNbDYmJiWjOjcNNpqbHccVovPlxfyHOLs0zHEV7ikYWZfLm5iL+dk8qYlGjTccRhBLjzJKWUg8aSnq21/tCzkcTxuOXUFHaWHuDxRdtIjArl/EEySiWaN3d1Hi8tzebyUd24amyS6TiiGe7M+lDAq0CG1voJz0cSx0Mpxb8uPIER3Tvy5/c3sDa3zHQkYVHLtpfwt482Mb53DPdNSpWLhxbmztDHWOBy4FSlVHrTx9keziWOQ1CAnZcuG0rXyBCufXOt3GYufmNbUSU3vr2Onp3CeXbaYALsckuFlbkz6+N7rbXSWg/QWg9q+vi8LcKJlusQFshrM4bj0pqr3ljD/up605GERZRU1nHV62sIDrTz6ozhtAt2mI4kjkJ+jPqw7tFhvHTZUPLLqpn5VppM2xMcqGvgmjfT2HugjlevHEbXyBDTkYQbpKh93MjkKP598UBW5ZRx69z1NDhdpiMJQ+oanFz/9lo2Fe7n2WlDGBAfaTqScJMUtR84f1BX7p+UyldbivirzLH2S06X5o53fmTZ9lIeuXAAp6XGmo4kjoFb0/OE95sxtjv7qut5+pvtRIY6uPvsvnKV309orfnbR5v4bONu/nZOXy4aGm86kjhGUtR+5PaJPSmvPsjLy3LoEBbIjeNlMXh/8NiXW5m7Oo+bTunBNeOSTccRLSBF7UeUUtw3qR/lNfU8unArkSGBTB+ZaDqW8KCXl2bz/Lc7mD4ykT+d3tt0HNFCUtR+xmZT/PvigVTWNnDPRxsJsCkuGZ5gOpbwgNe+z+HBzzM4Z0AXHji/vwx1eTG5mOiHHHYbz186hJN6xvCXDzbwzpo805FEK3v1+xz+8ekWzurfmaemyFZa3k6K2k8FO+y8dPlQxveO4c4PNjJvtZS1r3hlWTYPfLqFs0/ozDPTBuOQuw69nnwH/Viww86Llw3llN4x3PXhRuaskrL2dq8sy+afn2VwzgldeHqqlLSvkO+inwt22Hnx8sayvnv+RmavyjUdSbTQy0v/V9JPTR0kJe1D5DspCApoLOtT+3TinvmbeP7bLLkpxotorXnsy8yfLxw+LSXtc+S7KYCmsr5sKOcNjOPRhVt54NMMXC4pa6trcLq464ON/GfJDqaNSOCZqbISni+S6XniZ4EBNp6aMoio8EBeW57D3gN1PHbRQAID5C++FdXWO7ll7noWbSni1lNT+MNpvWQKno+Soha/YrMp7j03lZh2QTy6cCv7qut54dIhhAXJHxUr2V9Tz7X/TWNNbhl/P68fV45JMh1JeJCcKonfUEpx4/gUHr1wAN9vL2H6yysprqg1HUs0KdhXzZSXVrA+fx/PThssJe0HpKhFsy4ZnsCsy4exvbiK855bzoaCctOR/N6anWWc/9xyCstreH3GCM4dEGc6kmgDUtTiiCamxvLBDWOw2xQXv7iCBemFpiP5rXmr85j+8koiQhx8dNNYTuwpO4b7CylqcVR9u7Tn45vHMjA+ktvmpfPYl5kyI6QNNThd3P/xZu76cCOjkqOYf+NYesSEm44l2pAUtXBLVHgQb18zkmkjEvjPkh3MfGst+2tkH0ZP21tVx1VvrOGNH3Zy9YndeX3GcCJCZY9DfyNFLdwWGGDjockncP+kVL7dWszZTy9jXd4+07F81g87Sjnr6WWsyinj0QsH8H/npsocaT8l33VxTJRSzBjbnfeuH41ScPGLK3jh2x0yFNKKGpwunli0jUtfWUV4cADzbxwjS9H6OSlq0SKDEzvw2a3jOLNfZx5ZmMmVr6+mpLLOdCyvt3t/DdNfXsUz32znwiHxfHLzifSLizAdSxgmRS1aLCLEwXPTB/PQ5BNYnVPGWU8vY+Gm3aZjeSWtNQvSCznr6WVs2rWfJ6cM5N8XD5QbjQQgRS2Ok1KK6SMT+fjmE+nULojr317HjbPXUlwpN8i4a1d5DVf/N43b5qWTFBXGp7ecyOTBsgGt+B/liVXShg0bptPS0lr9dYW11TtdzFqazdPfbCfEYef/zk3lwiFdZf2JZrhcmjmr83j4i0ycLs2fzujNjDFJshuLn1JKrdVaDzvs16SoRWvLKq7izg82sDZ3Hyf1iuH+Sakky7zfX9m6p5J7F2xiVU4ZY1Oi+NfkASRGhZqOJQySohZtzuXSvLUyl0cXZlLX4OLy0d24bUJPIkMDTUczqqSyjie/3sa81XmEBwVwzzl9uWRYgvyrQ0hRC3NKKut4YtE23lmTR7tgB7dO6Mnlo7r53dKptfVOXluew/NLdlBb7+SyUY0/uDqE+fcPLvE/UtTCuMw9FTz4WQbLtpfSPTqMW05NYdLAOJ/fiaSuwcn8dYU8uziLwvIaJvaN5a9n95FbwMVvSFELS9Ba8+22Eh75IpPMPZXEdwjhupN7cPHQeIIddtPxWlX1wQbmrs7n5aXZ7Kmo5YSuEdx1Vh/GpshCSuLwpKiFpWitWZxZzHNLslifV050eBDXjOvO1OEJXj+Gvbeqjjmr8nhteQ77qusZldyRG8enMK5ntIxDiyOSohaWpLVmZXYZz3+bxbLtpQQG2Dirf2emDEtgVHIUNi+ZpuZ0ab7PKuWdNXks2lJEvVMzoU8nbjylB0O7dTQdT3iJIxW13PYkjFFKMbpHFKN7RLFlVwXvrMlj/vpCFqTvIrFjKJcMi2fSwDi6RYWZjnpYO0qq+Dh9F++vLaCwvIYOoQ6uGJ3E1OEJ9IxtZzqe8CFyRi0spbbeycJNe5i3Jo+V2WUA9OwUzoS+sZyW2olBCR2M3RDS4HSxNncfX2cU8XVGMTmlBwAY1zOaKcMTOC01lqAA3xprF23nuIc+lFJnAk8DduAVrfXDR3q+FLVoDfll1SzaUsTXGUWszimjwaWJCgtkZHJHBiVEMjixA/3jIggJ9Ew5HqhrYGPhftbnlZOev49VOWWUV9fjsCtGJUdxWmosE/rG0jUyxCPvL/zLcRW1UsoObANOAwqANcA0rfWW5n6PFLVobftr6vluWwmLM4pYm7eP/LIaAOw2RZ/O7egV246EjqEkdAghoWMoiR1D6RgWSFCArdmLeFpr6hpclFbVkV9WQ35ZNfn7qskvqyZzTyXbiir5afXWblGhDO3WgYl9YxnXM5p2wbJ4v2hdxztGPQLI0lpnN73YPOB8oNmiFqK1RYQ4OG9gHOcNbNzMtbSqjvS8ctLzGz9W55TxUXohh5532BSEBgYQEmgnNNCO1lB90EnNwQZq6p0cuoy2TUGXiBCSY8I4PTWWwYkdGJgQSUe5MUUY5E5RdwXyf/F5ATDy0CcppWYCMwESExNbJZwQzYkOD2JiaiwTU2N/fuxgg4td5TXk76smr6ya8up6ag46qT7opPpgA9UHnSgFoYF2QhwBjf8NtNMxLJCEDo1n4V0ig33+JhzhfVpt1ofWehYwCxqHPlrrdYVwV2CAjaToMJKirTlLRIiWcufUoRD45T5A8U2PCSGEaAPuFPUaoKdSqrtSKhCYCnzs2VhCCCF+ctShD611g1LqZuBLGqfnvaa13uzxZEIIIQA3x6i11p8Dn3s4ixBCiMOQy9tCCGFxUtRCCGFxUtRCCGFxUtRCCGFxHlk9TylVAuS28LdHA6WtGMckXzkWXzkOkGOxIl85Dji+Y+mmtY453Bc8UtTHQymV1tzCJN7GV47FV44D5FisyFeOAzx3LDL0IYQQFidFLYQQFmfFop5lOkAr8pVj8ZXjADkWK/KV4wAPHYvlxqiFEEL8mhXPqIUQQvyCFLUQQlicJYtaKfWAUmqDUipdKfWVUirOdKaWUEo9ppTKbDqW+UqpSNOZWkopdbFSarNSyqWU8rqpVEqpM5VSW5VSWUqpu0znOR5KqdeUUsVKqU2msxwPpVSCUmqJUmpL05+t20xnaimlVLBSarVS6semY/l7q76+FceolVLttdYVTb++FUjVWl9vONYxU0qdDixuWir2EQCt9Z2GY7WIUqov4AJeAv6ktfaa3YtbskGzlSmlTgKqgDe11v1N52kppVQXoIvWep1Sqh2wFvidN35fVOMOymFa6yqllAP4HrhNa72yNV7fkmfUP5V0kzDAej9N3KC1/kpr3dD06Uoad8fxSlrrDK31VtM5WujnDZq11geBnzZo9kpa66VAmekcx0trvVtrva7p15VABo17tHod3aiq6VNH00er9ZYlixpAKfWgUiofuBS413SeVvB74AvTIfzU4TZo9spC8FVKqSRgMLDKcJQWU0rZlVLpQDGwSGvdasdirKiVUl8rpTYd5uN8AK31PVrrBGA2cLOpnEdztONoes49QAONx2JZ7hyLEK1NKRUOfADcfsi/pr2K1tqptR5E47+cRyilWm1YqtV2IT9WWuuJbj51No27y9znwTgtdrTjUErNAM4FJmgrXhD4hWP4nngb2aDZoprGcz8AZmutPzSdpzVorcuVUkuAM4FWueBryaEPpVTPX3x6PpBpKsvxUEqdCfwFOE9rXW06jx+TDZotqOkC3KtAhtb6CdN5jodSKuanWV1KqRAaL1y3Wm9ZddbHB0BvGmcZ5ALXa6297gxIKZUFBAF7mx5a6Y2zVwCUUpOBZ4EYoBxI11qfYTTUMVBKnQ08xf82aH7QbKKWU0rNBcbTuKRmEXCf1vpVo6FaQCl1IrAM2Ejj33WAu5v2aPUqSqkBwH9p/PNlA97VWv+j1V7fikUthBDifyw59CGEEOJ/pKiFEMLipKiFEMLipKiFEMLipKiFEMLipKiFEMLipKiFEMLi/h95yyGcg55E7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = np.arange(-3, 3.01, 0.1)\n",
    "y = x ** 2\n",
    "plt.plot(x, y)\n",
    "plt.plot(2, 4, 'ro')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([4.])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "\n",
    "# 答案\n",
    "x = Variable(torch.FloatTensor([2]), requires_grad=True)\n",
    "y = x ** 2\n",
    "y.backward()\n",
    "print(x.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一次课程我们将会从导数展开，了解 PyTorch 的自动求导机制"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}