machinelearning_notebook/6_pytorch/0_basic/Tensor-and-Variable.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensor and Variable\n",
    "这是 PyTorch 基础的第二课，通过本次课程，你能够学会如何像使用 NumPy 一样使用 PyTorch，了解到 PyTorch 中的基本元素 Tensor 和 Variable 及其操作方式。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 把 PyTorch 当做 NumPy 用\n",
    "PyTorch 的官方介绍是一个拥有强力GPU加速的张量和动态构建网络的库，其主要构件是张量，所以我们可以把 PyTorch 当做 NumPy 来用，PyTorch 的很多操作好 NumPy 都是类似的，但是因为其能够在 GPU 上运行，所以有着比 NumPy 快很多倍的速度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个 numpy ndarray\n",
    "numpy_tensor = np.random.randn(10, 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用下面两种方式将numpy的ndarray转换到tensor上"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "pytorch_tensor1 = torch.Tensor(numpy_tensor)\n",
    "pytorch_tensor2 = torch.from_numpy(numpy_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用以上两种方法进行转换的时候，会直接将 NumPy ndarray 的数据类型转换为对应的 PyTorch Tensor 数据类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同时我们也可以使用下面的方法将 pytorch tensor 转换为 numpy ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 如果 pytorch tensor 在 cpu 上\n",
    "numpy_array = pytorch_tensor1.numpy()\n",
    "\n",
    "# 如果 pytorch tensor 在 gpu 上\n",
    "numpy_array = pytorch_tensor1.cpu().numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要注意 GPU 上的 Tensor 不能直接转换为 NumPy ndarray，需要使用`.cpu()`先将 GPU 上的 Tensor 转到 CPU 上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch Tensor 使用 GPU 加速\n",
    "\n",
    "我们可以使用以下两种方式将 Tensor 放到 GPU 上"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第一种方式是定义 cuda 数据类型\n",
    "dtype = torch.cuda.FloatTensor # 定义默认 GPU 的 数据类型\n",
    "gpu_tensor = torch.randn(10, 20).type(dtype)\n",
    "\n",
    "# 第二种方式更简单，推荐使用\n",
    "gpu_tensor = torch.randn(10, 20).cuda(0) # 将 tensor 放到第一个 GPU 上\n",
    "gpu_tensor = torch.randn(10, 20).cuda(1) # 将 tensor 放到第二个 GPU 上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用第一种方式将 tensor 放到 GPU 上的时候会将数据类型转换成定义的类型，而是用第二种方式能够直接将 tensor 放到 GPU 上，类型跟之前保持一致\n",
    "\n",
    "推荐在定义 tensor 的时候就明确数据类型，然后直接使用第二种方法将 tensor 放到 GPU 上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "而将 tensor 放回 CPU 的操作非常简单"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cpu_tensor = gpu_tensor.cpu()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也能够访问到 Tensor 的一些属性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([10, 20])\n",
      "torch.Size([10, 20])\n"
     ]
    }
   ],
   "source": [
    "# 可以通过下面两种方式得到 tensor 的大小\n",
    "print(pytorch_tensor1.shape)\n",
    "print(pytorch_tensor1.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.FloatTensor\n"
     ]
    }
   ],
   "source": [
    "# 得到 tensor 的数据类型\n",
    "print(pytorch_tensor1.type())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "# 得到 tensor 的维度\n",
    "print(pytorch_tensor1.dim())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n"
     ]
    }
   ],
   "source": [
    "# 得到 tensor 的所有元素个数\n",
    "print(pytorch_tensor1.numel())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习**\n",
    "\n",
    "查阅以下[文档](http://pytorch.org/docs/0.3.0/tensors.html)了解 tensor 的数据类型，创建一个 float64、大小是 3 x 2、随机初始化的 tensor，将其转化为 numpy 的 ndarray，输出其数据类型\n",
    "\n",
    "参考输出: float64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "float64\n"
     ]
    }
   ],
   "source": [
    "# 答案\n",
    "x = torch.randn(3, 2)\n",
    "x = x.type(torch.DoubleTensor)\n",
    "x_array = x.numpy()\n",
    "print(x_array.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor的操作\n",
    "Tensor 操作中的 api 和 NumPy 非常相似，如果你熟悉 NumPy 中的操作，那么 tensor 基本是一致的，下面我们来列举其中的一些操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1  1\n",
      " 1  1\n",
      "[torch.FloatTensor of size 2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones(2, 2)\n",
    "print(x) # 这是一个float tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.FloatTensor\n"
     ]
    }
   ],
   "source": [
    "print(x.type())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1  1\n",
      " 1  1\n",
      "[torch.LongTensor of size 2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 将其转化为整形\n",
    "x = x.long()\n",
    "# x = x.type(torch.LongTensor)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1  1\n",
      " 1  1\n",
      "[torch.FloatTensor of size 2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 再将其转回 float\n",
    "x = x.float()\n",
    "# x = x.type(torch.FloatTensor)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "-0.8203 -0.0328  1.8283\n",
      "-0.1734 -0.1873  0.9818\n",
      "-1.8368 -2.2450 -0.4410\n",
      "-0.8005 -2.1132  0.7140\n",
      "[torch.FloatTensor of size 4x3]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(4, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 沿着行取最大值\n",
    "max_value, max_idx = torch.max(x, dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1.8283\n",
       " 0.9818\n",
       "-0.4410\n",
       " 0.7140\n",
       "[torch.FloatTensor of size 4]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 每一行的最大值\n",
    "max_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 2\n",
       " 2\n",
       " 2\n",
       " 2\n",
       "[torch.LongTensor of size 4]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 每一行最大值的下标\n",
    "max_idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.9751\n",
      " 0.6212\n",
      "-4.5228\n",
      "-2.1997\n",
      "[torch.FloatTensor of size 4]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 沿着行对 x 求和\n",
    "sum_x = torch.sum(x, dim=1)\n",
    "print(sum_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 3])\n",
      "torch.Size([1, 4, 3])\n"
     ]
    }
   ],
   "source": [
    "# 增加维度或者减少维度\n",
    "print(x.shape)\n",
    "x = x.unsqueeze(0) # 在第一维增加\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 1, 4, 3])\n"
     ]
    }
   ],
   "source": [
    "x = x.unsqueeze(1) # 在第二维增加\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 4, 3])\n"
     ]
    }
   ],
   "source": [
    "x = x.squeeze(0) # 减少第一维\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 3])\n"
     ]
    }
   ],
   "source": [
    "x = x.squeeze() # 将 tensor 中所有的一维全部都去掉\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 4, 5])\n",
      "torch.Size([4, 3, 5])\n",
      "torch.Size([5, 3, 4])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(3, 4, 5)\n",
    "print(x.shape)\n",
    "\n",
    "# 使用permute和transpose进行维度交换\n",
    "x = x.permute(1, 0, 2) # permute 可以重新排列 tensor 的维度\n",
    "print(x.shape)\n",
    "\n",
    "x = x.transpose(0, 2)  # transpose 交换 tensor 中的两个维度\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 4, 5])\n",
      "torch.Size([12, 5])\n",
      "torch.Size([3, 20])\n"
     ]
    }
   ],
   "source": [
    "# 使用 view 对 tensor 进行 reshape\n",
    "x = torch.randn(3, 4, 5)\n",
    "print(x.shape)\n",
    "\n",
    "x = x.view(-1, 5) # -1 表示任意的大小，5 表示第二维变成 5\n",
    "print(x.shape)\n",
    "\n",
    "x = x.view(3, 20) # 重新 reshape 成 (3, 20) 的大小\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = torch.randn(3, 4)\n",
    "y = torch.randn(3, 4)\n",
    "\n",
    "# 两个 tensor 求和\n",
    "z = x + y\n",
    "# z = torch.add(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外，pytorch中大多数的操作都支持 inplace 操作，也就是可以直接对 tensor 进行操作而不需要另外开辟内存空间，方式非常简单，一般都是在操作的符号后面加`_`，比如"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 3])\n",
      "torch.Size([1, 3, 3])\n",
      "torch.Size([3, 1, 3])\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones(3, 3)\n",
    "print(x.shape)\n",
    "\n",
    "# unsqueeze 进行 inplace\n",
    "x.unsqueeze_(0)\n",
    "print(x.shape)\n",
    "\n",
    "# transpose 进行 inplace\n",
    "x.transpose_(1, 0)\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = torch.ones(3, 3)\n",
    "y = torch.ones(3, 3)\n",
    "print(x)\n",
    "\n",
    "# add 进行 inplace\n",
    "x.add_(y)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习**\n",
    "\n",
    "访问[文档](http://pytorch.org/docs/0.3.0/tensors.html)了解 tensor 更多的 api，实现下面的要求\n",
    "\n",
    "创建一个 float32、4 x 4 的全为1的矩阵，将矩阵正中间 2 x 2 的矩阵，全部修改成2\n",
    "\n",
    "参考输出\n",
    "$$\n",
    "\\left[\n",
    "\\begin{matrix}\n",
    "1 & 1 & 1 & 1 \\\\\n",
    "1 & 2 & 2 & 1 \\\\\n",
    "1 & 2 & 2 & 1 \\\\\n",
    "1 & 1 & 1 & 1\n",
    "\\end{matrix}\n",
    "\\right] \\\\\n",
    "[torch.FloatTensor\\ of\\ size\\ 4x4]\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1  1  1  1\n",
      " 1  2  2  1\n",
      " 1  2  2  1\n",
      " 1  1  1  1\n",
      "[torch.FloatTensor of size 4x4]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 答案\n",
    "x = torch.ones(4, 4).float()\n",
    "x[1:3, 1:3] = 2\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variable\n",
    "tensor 是 PyTorch 中的完美组件，但是构建神经网络还远远不够，我们需要能够构建计算图的 tensor，这就是 Variable。Variable 是对 tensor 的封装，操作和 tensor 是一样的，但是每个 Variabel都有三个属性，Variable 中的 tensor本身`.data`，对应 tensor 的梯度`.grad`以及这个 Variable 是通过什么方式得到的`.grad_fn`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 通过下面这种方式导入 Variable\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_tensor = torch.randn(10, 5)\n",
    "y_tensor = torch.randn(10, 5)\n",
    "\n",
    "# 将 tensor 变成 Variable\n",
    "x = Variable(x_tensor, requires_grad=True) # 默认 Variable 是不需要求梯度的，所以我们用这个方式申明需要对其进行求梯度\n",
    "y = Variable(y_tensor, requires_grad=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "z = torch.sum(x + y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "-2.1379\n",
      "[torch.FloatTensor of size 1]\n",
      "\n",
      "<SumBackward0 object at 0x10da636a0>\n"
     ]
    }
   ],
   "source": [
    "print(z.data)\n",
    "print(z.grad_fn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面我们打出了 z 中的 tensor 数值，同时通过`grad_fn`知道了其是通过 Sum 这种方式得到的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable containing:\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "[torch.FloatTensor of size 10x5]\n",
      "\n",
      "Variable containing:\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "    1     1     1     1     1\n",
      "[torch.FloatTensor of size 10x5]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 求 x 和 y 的梯度\n",
    "z.backward()\n",
    "\n",
    "print(x.grad)\n",
    "print(y.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过`.grad`我们得到了 x 和 y 的梯度，这里我们使用了 PyTorch 提供的自动求导机制，非常方便，下一小节会具体讲自动求导。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小练习**\n",
    "\n",
    "尝试构建一个函数 $y = x^2 $，然后求 x=2 的导数。\n",
    "\n",
    "参考输出：4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "提示：\n",
    "\n",
    "$y = x^2$的图像如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd4VFX+x/H3mcmkQwJJCIQkhBBaRHpHFAW76FpQwIZbWLuuW3TVVXddXcvaXQvWVSn2riguKIi0AJGWACEhDUghhCSkkMyc3x+J/hQpQ8jk3DvzfT1PniWzw8znMvjh5txzz1Faa4QQQtiHw3QAIYQQR0eKWwghbEaKWwghbEaKWwghbEaKWwghbEaKWwghbEaKWwghbEaKWwghbEaKWwghbCbIFy8aGxurU1JSfPHSQgjhl1avXl2utY7z5rk+Ke6UlBQyMjJ88dJCCOGXlFL53j5XhkqEEMJmpLiFEMJmpLiFEMJmpLiFEMJmpLiFEMJmpLiFEMJmpLiFEMJmLFPc9Y1uXlicy3fbyk1HEUKIo7You5RXluaxv8nj8/eyTHEHORQvLMnlpSV5pqMIIcRRe/abbfz3u+24nMrn72Wd4nY6uGhYIos2l7Jrb73pOEII4bXcshpW5lVw8YgklAqg4ga4eHgSHg3vrC40HUUIIbz2ZkYhTofioqGJ7fJ+lirulNgIxqTG8GZGIR6PNh1HCCGOqNHt4d3VRZzSrwtdOoa2y3taqrgBpo5MorCijmW5u01HEUKII/pfVinlNfuZOiKp3d7TcsV9+nFdiQpzMXdlgekoQghxRPNWFdC1Yygn9fFqRdY2YbniDnU5OX9Id77cWELFvv2m4wghxCHtqKzjmy1lTBmeSJCz/erUcsUNcMmIJPa7Pby/tth0FCGEOKS3M4rQunliRXuyZHH379aRQUnRvLmqAK3lIqUQwnrcHs1bGYWckBZLUufwdn1vSxY3wNQRSWwpqWFtYaXpKEII8QtLc8oprqzjkna8KPkDyxb35EEJhAc7mScXKYUQFjRvVQGdwl2cdlx8u7+3ZYs7MiSIyQMT+Pj7nVTXN5qOI4QQPyqvaWDBphIuGJpISJCz3d/fssUNcMnIJOoa3Xz8/U7TUYQQ4kfvrymm0a2NDJOAxYt7SFI0feM7MG+VDJcIIaxBa83clQUMTY6mT3wHIxksXdxKKaaPSmZd0V7WFclFSiGEectyd5Nbvo9LR/UwlsHSxQ1w/tDuhLmczFkhZ91CCPNmryggKszF2QO7Gctg+eLuGOri3EEJfJi5gyq5SCmEMKisuoEvNuziomGJhLra/6LkDyxf3ACXjk6mrtHNh3InpRDCoLdXF9Lk0UwflWw0hy2Ke2BiNAO6d2T2CrmTUghhhsejmbOigNGpnekVF2k0i1fFrZT6g1Jqo1Jqg1JqrlKqfRad/YlLR/Uge1c1awr2tPdbCyEEi7eWUbSnzuhFyR8csbiVUt2BG4HhWusBgBOY6utgBzp3UAKRIUHMXi4XKYUQ7W/2igJiIoI5/biupqN4PVQSBIQppYKAcGCH7yIdXERIEOcP6c4n63dSWSvLvQoh2s/OvXUszC7l4hFJBAeZH2E+YgKtdTHwb6AA2Ans1Vp/eeDzlFIzlVIZSqmMsrKytk8KTB+VzP4mD++sLvLJ6wshxMG8uaoQt0czbYTZi5I/8GaopBNwHtATSAAilFKXHfg8rfUsrfVwrfXwuDjf7ATRv1tHhiZHM0cuUgoh2kmT28O8lYWc2CeO5Jj2Xb71ULw5558E5Gmty7TWjcB7wFjfxjq0S0f1ILd8n+xJKYRoFwuzS9lVVc+lhqcA/pQ3xV0AjFZKhSulFDARyPJtrEM7e2A3osJccpFSCNEuZq8oIL5jCBP7dTEd5UfejHGvAN4B1gDrW37PLB/nOqRQl5MpwxL5YuMuSqrqTcUQQgSA7eX7+GZLGdNGJrfrnpJH4lUSrfXdWut+WusBWuvLtdYNvg52OJeN7oFba1m/RAjhU68vzyfIoZg+0jrDJGCTOycPlBIbwUl94pizsoD9TR7TcYQQfqh2fxNvZxRyxoCudOnY7vccHpYtixvgijE9mhd82bjLdBQhhB9qXtiuiSvGpJiO8gu2Le6T+nQhuXM4ry/LNx1FCOFntNa8tiyffl07MCKlk+k4v2Db4nY6FJeNTmbl9gqydlaZjiOE8COr8/eQtbOKK8ak0DyZzlpsW9wAFw9PIiTIwWty1i2EaEP/XZZPh9AgfjUkwXSUg7J1cUeHB3Pe4AQ+WFvM3jrZZEEIcexKq+r5fP1OpgxLIjw4yHScg7J1cQNcMSaFuka3rF8ihGgTc1c2b5Zw+Rjzy7ceiu2Le0D3KIYmR/PG8nw8Hlm/RAjReo1uD3NW5nNinzh6xkaYjnNIti9ugCvHppBXvo8lOeWmowghbOzLjSWUVDVwpYXPtsFPivuMAV2JjQzmte+2m44ihLCx15ZtJ7FTGBP6WmddkoPxi+IOCXIyfWQyCzeXsr18n+k4Qggb2rhjLyvyKrhiTA+cDutNAfwpvyhuaF6/JMiheFXOuoUQrfDK0u2EBzu5ZLi11iU5GL8p7i4dQzlnYAJvZxRSVS9TA4UQ3iurbuCjzB1cODSRqHCX6ThH5DfFDfDrcT3Zt9/N2xkyNVAI4b05KwrY7/YwY1yK6She8aviPj4xihEpnXj1uzzcMjVQCOGFhiY3ry/P5+S+cfSKizQdxyt+VdwAV43rSWFFHV9llZiOIoSwgU++30l5TQNXjetpOorX/K64T0uPp3t0GK8szTMdRQhhcVprXl6aR1qXSMb3jjUdx2t+V9xBTgdXjOnB8twKNu7YazqOEMLCVm3fw8YdVVw1zpqrAB6K3xU3wNQRyYS5nLy6dLvpKEIIC3tlaR5RYS4uGJJoOspR8cvijgp3cdGwRD7M3EF5jdHtMYUQFlVYUcsXG3cxfVQyYcFO03GOil8WN8CMcSnsd3uYvVw2FBZC/NJry7ajlOLy0dZel+Rg/La4e8VFMqFvHK8vz6e+0W06jhDCQmoampi3qnkj4IToMNNxjprfFjfA78anUl7TwIeZxaajCCEsZN7KAqrrm5g5PtV0lFbx6+Ie2yuG9G4deWFJnqzVLYQAoMnt4ZWl2xnZszODkqJNx2kVvy5upRQzT0wlp7SGb7aUmY4jhLCAzzbsoriyzrZn2+DnxQ1w9sBudIsKZdbiXNNRhBCGaa2ZtXgbqXERnNLP2mtuH47fF7fL6eDX43qyLHc364vkhhwhAtny3Ao2FFfxu/GpOCy+5vbh+H1xA0wdmUSHkCBeWCJn3UIEsheW5BIbGcz5Q7qbjnJMAqK4O4S6mDYqmU/X76RoT63pOEIIA7aWVLMwu5QrxqQQ6rLXDTcHCojiBpgxNgVF8y4XQojA8+KSPEJdDi6z4Q03BwqY4k6IDmPyoATmrSxgb53skCNEICmtruf9tcVMGZZE54hg03GOWcAUN8BvxzfvkDN3pdwGL0Qgee27fBo9Hn5zgn3W3D6cgCru4xKiGJcWwytL82hoktvghQgE+xqaeGNFPqelx5MSG2E6TpsIqOIGuPqkXpRUNfDBWrkNXohAMHdlAZW1jfz+pF6mo7QZr4pbKRWtlHpHKZWtlMpSSo3xdTBfOSEtlgHdO/LcN7myL6UQfq6hyc2LS/IYndqZocmdTMdpM96ecT8BzNda9wMGAVm+i+RbSimunZBGXvk+5m/YZTqOEMKHPlhbzK6qeq6dkGY6Sps6YnErpaKAE4GXALTW+7XWlb4O5kunH9eV1NgInv0mB63lrFsIf+T2aJ7/JpcB3Tvaaj9Jb3hzxt0TKANeUUqtVUq9qJSy9Qi/06H4/UmpbCiuYsnWctNxhBA+8MXGXeSW7+Oak9JstZ+kN7wp7iBgKPCs1noIsA+47cAnKaVmKqUylFIZZWXWX4nv/CGJdO0YyjNf55iOIoRoY1prnvk6h9TYCM4Y0NV0nDbnTXEXAUVa6xUt379Dc5H/jNZ6ltZ6uNZ6eFxcXFtm9IngIAe/Hd+T5bkVrCnYYzqOEKINfZtTzobiKn5/UipOGy8mdShHLG6t9S6gUCnVt+WhicAmn6ZqJ9NGJhMd7uLZr7eZjiKEaEPPLNpGfMcQfmXzxaQOxdtZJTcAs5VS64DBwP2+i9R+IkKCuHJMCgs2lbClpNp0HCFEG1hbsIdlubv53fhUQoLsvZjUoXhV3FrrzJZhkIFa619prf1mbGHG2BTCXE6ek7NuIfzCM19vIyrMxbSRyaaj+EzA3Tl5oE4RwUwbmcyH3++gYLcs+SqEnW3eVc2CTSVcOTaFiJAg03F8JuCLG/jxAsaz38gMEyHs7KmFW4kIdvLrcSmmo/iUFDcQ3zGUqSOSeGd1kWy0IIRN5ZRW8+n6nVw5NoXocPsv3Xo4Utwtrm5ZgOa5b2SsWwg7enphDmEuJ7+18e7t3pLibpEQHcZFw5J4a1URu/bWm44jhDgKeeX7+Oj7HVw2uodfbJRwJFLcP3HthF54tJazbiFs5j+LcnA5HfwuAM62QYr7Z5I6h3PB0O7MXVlAaZWcdQthBwW7a3l/bTGXjupBXIcQ03HahRT3Aa47OY0mj2bW4lzTUYQQXnjm65wfF44LFFLcB+gRE8F5gxN4Y0U+5TUNpuMIIQ6jaE8t76wuYtqIJOI7hpqO026kuA/iupPTaGjy8MISOesWwsqe/XobSuFX25J5Q4r7IHrFRTJ5YAKvL8tnt5x1C2FJOyrreDujiCnDk0iIDjMdp11JcR/CjRPTqG9087yMdQthSU8tzEGjuXZCYJ1tgxT3IaV16cCvBnfntWXbKa2WGSZCWEnB7lrezihk2shkEjuFm47T7qS4D+PGib1pdGueWSTzuoWwkicXbsXpUFx3sn9tAuwtKe7DSImN4KKhicxZUcCOyjrTcYQQwLayGt5bU8Rlo3sE1EySn5LiPoIbJqah0Ty9SFYOFMIKnvhqKyFBTq4JwLHtH0hxH0Fip3CmjkjmrVWFsl63EIZt3lXNx+t2MGNcCrGRgXGX5MFIcXvhupPTcDgUTy7cajqKEAHtsQVbiAgOYmaArElyKFLcXugaFcrlo3vw3poicstqTMcRIiBtKN7L/I27+M0JPekUACsAHo4Ut5eumdCLkCAnj38lZ91CmPDogi1Ehbn4zfiepqMYJ8XtpdjIEGaMS+HjdTvI2lllOo4QAWV1/h4WZpcy88RUOoa6TMcxTor7KPz+xFQ6hATx8BebTUcRImBorXnw82xiI0O4ys/3kvSWFPdRiA4P5poJaSzMLmVF7m7TcYQICIs2l7JyewU3TepNeLD/7tx+NKS4j9KMsSnEdwzhgfnZaK1NxxHCr7k9mgc/30xKTDhTRySZjmMZUtxHKSzYyR8m9WFtQSVfbioxHUcIv/bB2mI2l1Tzp9P74nJKXf1A/iRa4aJhifSKi+Ch+dk0uT2m4wjhl+ob3Ty6YAvHd4/irAHdTMexFCnuVghyOvjz6f3YVraPd9cUmY4jhF96Y3k+xZV13HZmPxwOZTqOpUhxt9Lpx8UzJDmaxxZspb7RbTqOEH6lqr6RpxflML53LOPSYk3HsRwp7lZSSnHrGf3YVVXPq99tNx1HCL8y65tcKmsbufWMfqajWJIU9zEYnRrDyX3jeGZRDpW1+03HEcIvlFbV89K3eUwelMCA7lGm41iSFPcxuvXMftQ0NPHk/2TZVyHawr+/3EyTx8OfTutjOoplSXEfo35dO3Lx8CReW7ZdFqAS4hht3LGXt1cXceWYFHrERJiOY1lS3G3gltP6EBLk4IHPs01HEcK2tNbc92kW0WEubpjY23QcS5PibgNdOoRy7clpfLmphGXb5FZ4IVrjf1mlfLdtNzdP6kNUmCwkdThS3G3kNyf0JCEqlH9+ugmPR26FF+JoNLo93P9ZFqlxEUwflWw6juV5XdxKKadSaq1S6hNfBrKrUJeTW8/sx8YdVby3tth0HCFsZfbyfHLL93HHWf3l1nYvHM2f0E1Alq+C+INzByUwOCmah7/IpnZ/k+k4Qljb7NmQkoJ2ODjt7NH8uTyDU/p1MZ3KFrwqbqVUInA28KJv49ibUoq/ndOfkqoGZi3ONR1HCOuaPRtmzoT8fJTWJOwt5Zo5D6LmzDGdzBa8PeN+HPgLICsqHcGwHp05e2A3nv8ml51760zHEcKa7rgDamt/9pCjrq75cXFERyxupdQ5QKnWevURnjdTKZWhlMooKytrs4B2dNsZ/fBozb8+k+mBQhxUQcHRPS5+xpsz7nHAuUqp7cA84BSl1BsHPklrPUtrPVxrPTwuLq6NY9pLUudwfn9SLz76fgfLZaccIX4p+RAzRw71uPiZIxa31vqvWutErXUKMBVYqLW+zOfJbO6ak3rRPTqMez7aKGt2C3GA/f+4l3pXyM8fDA+H++4zE8hmZN6Nj4QFO/nbOelk76rmjeX5puMIYSmzEkfzl9Ovpz4hEZSCHj1g1iy49FLT0WzhqIpba/211vocX4XxN6cfF8/43rE8smAL5TUNpuMIYQnFlXU8vSiHxkumEVpcCB4PbN8upX0U5Izbh5RS3D35OOr2u3l4/mbTcYSwhPs/bb4d5I6z+xtOYl9S3D6W1iWS35zQkzczCsksrDQdRwijluaU8+n6nVw3IY3ETuGm49iWFHc7uGFib7p0COHuDzfIOiYiYDW6Pdz90UaSO4fzuxNTTcexNSnudhAZEsTtZ/Xn+6K9zFtVaDqOEEa8sjSPnNIa7jonnVCX03QcW5PibifnDU5gdGpnHvg8i7JquVApAkvRnloeW7CVSf27MLG/rEdyrKS424lSivvOP576Rg///HST6ThCtButNXd9uBGl4O/nDUApZTqS7Ulxt6NecZFcM6EXH2buYPGWwF4WQASO+Rt2sTC7lFtO7UP36DDTcfyCFHc7u2ZCL1JjI7jzgw3UN7pNxxHCp6rrG7nn442kd+vIjLEppuP4DSnudhbqcvLP8wdQUFHL0wtlZ3jh3x75cgul1Q3864LjCZINEtqM/EkaMLZXLBcM7c7zi7expaTadBwhfOL7wkr+u2w7V4zuwaCkaNNx/IoUtyF3nNWfiJAg7nh/vcztFn6nye3hr++tp0uHEP54el/TcfyOFLchMZEh3H5Wf1Zt38PcVbIGsfAvLy/NY9POKu6ZfBwdQ2XH9rYmxW3QlGGJjO0Vw78+y6a4UnbLEf4ht6yGR77cwqT+8ZwxoKvpOH5JitsgpRQPXDAQt0fz1/fWo7UMmQh7c3s0f3lnHSFBDu4/X+Zs+4oUt2HJMeHcekZfFm8p453VRabjCHFMXlu2nYz8Pdw1+Ti6dAw1HcdvSXFbwBVjUhiZ0pl7P9lESVW96ThCtEr+7n08NH8zE/rGceHQ7qbj+DUpbgtwOBQPXjSQhiYPd7wvQybCfjweza3vriPIofjXBcfLEImPSXFbRM/YCP58el++yirlw8wdpuMIcVRmryxgeW4Fd5zdn25Rclu7r0lxW8hV43oyNDmaez7eSGm1DJkIeyjaU8sDn2Uxvncsl4xIMh0nIEhxW4jToXjookHU7ndzu8wyETbg8Wj+/PY6ABkiaUdS3BaT1iWSv7QMmcimC8LqXvo2j2W5u7lrcrpsRdaOpLgt6NfjejIuLYZ7P9nE9vJ9puMIcVBZO6t4+IvNnJYez8XDZYikPUlxW5DDofj3lEEEORQ3v5lJk9tjOpIQP1Pf6OYPb2bSMcwlQyQGSHFbVLeoMO47/3gyCyv5z6JtpuMI8TOPfLmZ7F3VPHzRQGIiQ0zHCThS3BY2eVACvxqcwJMLt7K2YI/pOEIA8F1OOS8syeOy0cmc3E/2jzRBitvi/n7eAOI7hHDLW99Tu7/JdBwR4PbWNvLHt78nNTaCO85KNx0nYElxW1xUmItHLh7M9t37+MfHssmwMEdrze0frKesuoHHLhlMWLDTdKSAJcVtA2N6xXDNSb2Yt6qQDzOLTccRAWrOygI+XbeTW07rIzvaGCbFbRO3nNqH4T06cft768ktqzEdRwSYTTuq+PvHmzixTxxXn9jLdJyAJ8VtE0FOB09OG4IryMF1c9bKDvGi3dQ0NHH9nDVEh7l49OJBOBwy9c80KW4bSYgO49GLB5G1s4p/firj3cL3tNbc+f56tu/ex5PThhArU/8sQYrbZk7pF8/ME1N5Y3nzeKMQvvR2RhEfZO7g5kl9GJ0aYzqOaCHFbUN/Pr0vg5Oiue3ddeTvllvihW9sKanmro82MLZXDNednGY6jvgJKW4bcjkdPD19CErBNW+soW6/jHeLtlVd38jVb6wmMiSIx6cOxinj2pYixW1TiZ3CeXzqYLJ2VfHX99bJErCizXg8mlve+p783bU8PX0oXTrI3pFWc8TiVkolKaUWKaU2KaU2KqVuao9g4shO6RfPLZP68EHmDl5Zut10HOEnnl6Uw4JNJdx5dn8Z17Yob864m4A/aq3TgdHAdUopudfVIq47OY3T0uO577Mslm3bbTqOsLn/ZZXw2FdbuGBId2aMTTEdRxzCEYtba71Ta72m5dfVQBYgWzhbhMOheOTiQaTEhHP9nDUUV9aZjiRsKreshpvnZZLerSP3y1KtlnZUY9xKqRRgCLDCF2FE63QIdTHriuE0NHm4+vXVcnOOOGo1DU38/vXVBDkVz18+jFCXrENiZV4Xt1IqEngXuFlrXXWQ/3+mUipDKZVRVlbWlhmFF3rFRfLYJYNZX7xX9qsUR8Xj0fzxrUy2ldXwn+lDZQsyG/CquJVSLppLe7bW+r2DPUdrPUtrPVxrPTwuLq4tMwovnZoezy2n9uG9tcU8vTDHdBxhEw/Oz+aLjSXceXY6Y9NiTccRXgg60hNU80DXS0CW1vpR30cSx+KGU9LYXr6PRxZsITkmnPMGy+UIcWhzVxbw/OJcLh/dg6vGpZiOI7zkzRn3OOBy4BSlVGbL11k+ziVaSSnFvy48npE9O/Pnd9axOr/CdCRhUUu2lnHnBxuY0DeOuyeny8VIG/FmVsm3WmultR6otR7c8vVZe4QTrRMS5OT5y4bRPTqM3722Wm6LF7+wpaSaa99YQ+8ukTw1bQhBTrkXz07k0/JTnSKCeXnGCDxac9Wrq9hb22g6krCIsuoGrnplFaHBTl6aMYIOoS7TkcRRkuL2Yz1jI3j+smEUVtQy8/UMmSYo2NfQxG9fy2D3vgZeunI43aPDTEcSrSDF7edGpcbw7ymDWJFXwY1z19Lk9piOJAxpaHJz9Rur2VC8l6emDWVgomw/ZldS3AHgvMHduWdyOl9uKuGvMsc7ILk9mlve/J4lW8t58MKBnJoebzqSOAZHnA4o/MOMcT3ZU9vIE//bSnS4i9vP6i+zCAKE1po7P9jAp+t3cufZ/bloWKLpSOIYSXEHkJsn9aaydj8vLMmjU0Qw106QxfEDwcNfbGbuygKuO7kXvx2fajqOaANS3AFEKcXdk4+jsq6Rh+ZvJjosmOmjkk3HEj70wuJcnvl6G9NHJfOn0/qajiPaiBR3gHE4FP+eMojq+ibu+GA9QQ7FxSOSTMcSPvDyt3nc91kWZw/sxr3nDZChMT8iFycDkMvp4JlLh3Ji7zj+8u463lxVYDqSaGMvfZvHPz7ZxJkDuvL4JbL1mL+R4g5QoS4nz18+jAl947j13fXMWynl7S9eXJLLvZ9s4qzju/LktCG45K5IvyOfaAALdTl57rJhnNw3jtveW8+cFVLedvfiklz++WkWZx/fjSemSmn7K/lUA1yoy8lzlzeX9+3vr2f2inzTkUQrvbD4/0v78amDpbT9mHyygpCg5vI+pV8X7nh/A898nSM36diI1pqHv8j+8ULkE1Lafk8+XQG0lPdlwzh3UAIPzd/MvZ9k4fFIeVtdk9vDbe+u5z+LtjFtZBJPTpWV/gKBTAcUPwoOcvD4JYOJiQzm5aV57N7XwMMXDSI4SIrAiuob3dwwdy0LNpVw4ylp/OHUPjLlL0BIcYufcTgUd52TTlyHEB6av5k9tY08e+lQIkLkr4qV7K1r5Hf/zWBVfgV/P/c4rhybYjqSaEdyKiV+QSnFtRPSeOjCgXy7tYzpLyyntKredCzRomhPLZc8v4y1hXt4atoQKe0AJMUtDuniEUnMunw4W0trOPfppawrqjQdKeCt2l7BeU8vpbiyjldmjOScgQmmIwkDpLjFYU1Kj+fda8bidCimPLeMDzOLTUcKWPNWFjD9heVEhbn44LpxnNBbdmQPVFLc4oj6d+vIR9ePY1BiNDfNy+ThL7Jlxkk7anJ7uOejjdz23npGp8bw/rXj6BUXaTqWMEiKW3glJjKEN347imkjk/jPom3MfH01e+tkH0tf213TwFWvruLV77bzmxN68sqMEUSFyx6RgU6KW3gtOMjB/ecfzz2T0/l6cylnPbGENQV7TMfyW99tK+fMJ5awIq+Chy4cyN/OSZc52gKQ4hZHSSnFjHE9efvqMSgFU55bxrNfb5OhkzbU5Pbw6IItXPriCiJDg3j/2rGy9K74GSlu0SpDkjvx6Y3jOeO4rjw4P5srX1lJWXWD6Vi2t3NvHdNfWMGT/9vKhUMT+fj6EzguIcp0LGExUtyi1aLCXDw9fQj3n388K/MqOPOJJczfsNN0LFvSWvNhZjFnPrGEDTv28tglg/j3lEFy45M4KClucUyUUkwflcxH159Alw4hXP3GGq6dvZrSarlhx1s7Kuv4zX8zuGleJikxEXxywwmcP0Q29BWHpnyxCtzw4cN1RkZGm7+usLZGt4dZi3N54n9bCXM5+ds56Vw4tLusn3EIHo9mzsoCHvg8G7dH86fT+zJjbIrsVhOglFKrtdbDvXquFLdoazmlNdz67jpW5+/hxD5x3DM5nVSZd/wzm3dVc9eHG1iRV8G4tBj+df5AkmPCTccSBklxC+M8Hs3ry/N5aH42DU0eLh/Tg5sm9iY6PNh0NKPKqht47KstzFtZQGRIEHec3Z+LhyfJTyVCiltYR1l1A48u2MKbqwroEOrixom9uXx0j4BbKra+0c3LS/N4ZtE26hvdXDa6+R+yThGB/Q+Z+H9S3MJysndVcd+nWSzZWk7P2AhuOCW449d3AAAHMklEQVSNyYMS/H6nloYmN++vKeaphTkUV9YxqX88fz2rn9yyLn5BiltYktaar7eU8eDn2WTvqiaxUxi/P6kXU4YlEupymo7Xpmr3NzF3ZSEvLM5lV1U9x3eP4rYz+zEuTRaGEgcnxS0sTWvNwuxSnl6Uw9qCSmIjQ/jt+J5MHZFk+zHw3TUNzFlRwMtL89hT28jo1M5cOyGN8b1jZRxbHJYUt7AFrTXLcyt45usclmwtJzjIwZkDunLJ8CRGp8bgsMm0OLdH821OOW+uKmDBphIa3ZqJ/bpw7cm9GNajs+l4wiaOprjltixhjFKKMb1iGNMrhk07qnhzVQHvry3mw8wdJHcO5+LhiUwelECPmAjTUQ9qW1kNH2Xu4J3VRRRX1tEp3MUVY1KYOiKJ3vEdTMcTfsyrM26l1BnAE4ATeFFr/cDhni9n3KK16hvdzN+wi3mrClieWwFA7y6RTOwfz6npXRic1MnYDSpNbg+r8/fwVVYJX2WVkle+D4DxvWO5ZEQSp6bHExLkX2P1ov206VCJUsoJbAFOBYqAVcA0rfWmQ/0eKW7RFgoralmwqYSvskpYmVdBk0cTExHMqNTODE6KZkhyJwYkRBEW7Juy3NfQxPrivawtqCSzcA8r8iqorG3E5VSMTo3h1PR4JvaPp3t0mE/eXwSWth4qGQnkaK1zW158HnAecMjiFqItJHUO59cn9OTXJ/Rkb10j32wpY2FWCasL9vDZ+l0AOB2Kfl070Ce+A0mdw0nqFEZS53CSO4fTOSKYkCDHIS8Kaq1paPJQXtNAYUUdhRW1FO6ppbCiluxd1WwpqeaH1Wp7xIRzSr8uTOofz/jesXQIlc0MhDneFHd3oPAn3xcBo3wTR4iDiwpzce6gBM4d1Lw5bnlNA5kFlWQWNn+tzKvgg8xiDvwB0qEgPDiIsGAn4cFOtIba/W7q9jdR1+jmwGXEHQq6RYWRGhfBaenxDEnuxKCkaDrLjTLCQtrs4qRSaiYwEyA5ObmtXlaIg4qNDGFSejyT0uN/fGx/k4cdlXUU7qmloKKWytpG6va7qd3vpnZ/E7X73SgF4cFOwlxBzf8b7KRzRDBJnZrP0rtFh/r9TUHC/rwp7mLgp9tvJLY89jNa61nALGge426TdEIcheAgBymxEaTEWnMWihBtxZtTi1VAb6VUT6VUMDAV+Mi3sYQQQhzKEc+4tdZNSqnrgS9ong74stZ6o8+TCSGEOCivxri11p8Bn/k4ixBCCC/IVRghhLAZKW4hhLAZKW4hhLAZKW4hhLAZKW4hhLAZn6zHrZQqA/Jb+dtjgfI2jGOSvxyLvxwHyLFYkb8cBxzbsfTQWsd580SfFPexUEpleLtCltX5y7H4y3GAHIsV+ctxQPsdiwyVCCGEzUhxCyGEzVixuGeZDtCG/OVY/OU4QI7FivzlOKCdjsVyY9xCCCEOz4pn3EIIIQ7DksWtlLpXKbVOKZWplPpSKZVgOlNrKKUeVkpltxzL+0qpaNOZWkspNUUptVEp5VFK2W4GgFLqDKXUZqVUjlLqNtN5joVS6mWlVKlSaoPpLMdCKZWklFqklNrU8nfrJtOZWkspFaqUWqmU+r7lWP7u0/ez4lCJUqqj1rqq5dc3Aula66sNxzpqSqnTgIUtS+M+CKC1vtVwrFZRSvUHPMDzwJ+01rbZDbo1G15bmVLqRKAGeE1rPcB0ntZSSnUDummt1yilOgCrgV/Z8XNRzRubRmita5RSLuBb4Cat9XJfvJ8lz7h/KO0WEYD1/nXxgtb6S611U8u3y2nePciWtNZZWuvNpnO00o8bXmut9wM/bHhtS1rrxUCF6RzHSmu9U2u9puXX1UAWzXvc2o5uVtPyravly2e9ZcniBlBK3aeUKgQuBe4ynacN/Br43HSIAHWwDa9tWRD+SimVAgwBVphN0npKKadSKhMoBRZorX12LMaKWyn1lVJqw0G+zgPQWt+htU4CZgPXm8p5JEc6jpbn3AE00XwsluXNsQjR1pRSkcC7wM0H/LRtK1prt9Z6MM0/WY9USvlsGKvNdnk/WlrrSV4+dTbNu+/c7cM4rXak41BKzQDOASZqK15Q+Imj+EzsxqsNr0X7axkPfheYrbV+z3SetqC1rlRKLQLOAHxyAdmSQyVKqd4/+fY8INtUlmOhlDoD+Atwrta61nSeACYbXltQywW9l4AsrfWjpvMcC6VU3A+zxpRSYTRfCPdZb1l1Vsm7QF+aZzHkA1drrW13hqSUygFCgN0tDy234+wYAKXU+cBTQBxQCWRqrU83m8p7SqmzgMf5/w2v7zMcqdWUUnOBCTSvRFcC3K21fsloqFZQSp0ALAHW0/zfOsDtLXvc2opSaiDwX5r/fjmAt7TW//DZ+1mxuIUQQhyaJYdKhBBCHJoUtxBC2IwUtxBC2IwUtxBC2IwUtxBC2IwUtxBC2IwUtxBC2IwUtxBC2Mz/AbHHGCe52T3ZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x = np.arange(-3, 3.01, 0.1)\n",
    "y = x ** 2\n",
    "plt.plot(x, y)\n",
    "plt.plot(2, 4, 'ro')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([4.])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "\n",
    "# 答案\n",
    "x = Variable(torch.FloatTensor([2]), requires_grad=True)\n",
    "y = x ** 2\n",
    "y.backward()\n",
    "print(x.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一次课程我们将会从导数展开，了解 PyTorch 的自动求导机制"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}