Improve description of PyTorch

6_pytorch/0_basic/imgs/autograd_Variable.svg

@@ -1,2 +0,0 @@
-<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
-<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="231px" height="231px" version="1.1" content="&lt;mxfile userAgent=&quot;Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36 OPR/48.0.2685.35&quot; version=&quot;7.5.4&quot; editor=&quot;www.draw.io&quot;&gt;&lt;diagram id=&quot;07382e78-ac41-cbda-a75f-370a405a7521&quot; name=&quot;第 1 页&quot;&gt;1ZZdb5swFIZ/DZebwCYkuWyytruZNCnSdjk52IBVg5kxg+zX7xjMhwtVWy2RttzEfo99fPw+NuDhY94+KlJmXyRlwkM+bT38yUMoCPwt/Bnl0is7tO+FVHFqB03Cif9mVvStWnPKKmegllJoXrpiLIuCxdrRiFKycYclUrirliRlC+EUE7FUv3Oqs2EX20n/zHiaDSsHkd3fmcRPqZJ1YdfzEE66Xx/OyZDLbrTKCJXNTML3Hj4qKXXfytsjE8bbwbZ+3sML0bFuxQr9lgmon/CLiJoNFXd16cvgRZNxzU4liU2/Ad4ePmQ6F9ALoEmqsieQ8JZB1kPChThKIVU33WwfxTHolVbyic0iNDpHmwgilFSZmdrlswUxpVn74qaC0So4gkzmTKsLDLETUGTdtacP7Wy/mViORy2bcRxFYs9POuaePISGtXHdUrxiaSS0cUZC/XNvo5+1HAIfqu4W3MGAYFe2UxBaqe5c0mTIBDX0yfrQgll3AkdLXyHo2G86X4nWTBWdgvzwSlD2GwcKjpZQos2SSXgFJOGNkKSK0P8YCd75ryLZrlyTayDZ3BDJj6R4PxX/n6GCXCohWlLZ3+iiRH9NBYVrVEitpSHz8RtRnJwhxRv5gI3aheC+RgpZsGfvHCsRwVNDJgbbGegHA4XDK/7OBnJOqVlmlbqD2lRov1GQf6Wn4fbZ1fOXkIP9yt3D76cM3emDoovNvtrw/R8=&lt;/diagram&gt;&lt;/mxfile&gt;"><defs/><g transform="translate(0.5,0.5)"><rect x="10" y="10" width="210" height="210" fill="#fff2cc" stroke="#d6b656" stroke-dasharray="3 3" pointer-events="none"/><rect x="45" y="90" width="65" height="40" rx="6" ry="6" fill="#ffffff" stroke="#000000" stroke-dasharray="1 4" pointer-events="none"/><g transform="translate(59.5,100.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="35" height="19" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 37px; white-space: nowrap; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;"><font style="font-size: 18px">data</font></div></div></foreignObject><text x="18" y="16" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><rect x="130" y="90" width="70" height="40" rx="6" ry="6" fill="#ffffff" stroke="#000000" stroke-dasharray="1 4" pointer-events="none"/><g transform="translate(146.5,100.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="36" height="19" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 38px; white-space: nowrap; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;"><font style="font-size: 18px">grad</font></div></div></foreignObject><text x="18" y="16" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><rect x="70" y="150" width="95" height="40" fill="#ffffff" stroke="#000000" stroke-dasharray="1 4" pointer-events="none"/><g transform="translate(86.5,160.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="61" height="19" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 63px; white-space: nowrap; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;"><font style="font-size: 18px">grad_fn</font></div></div></foreignObject><text x="31" y="16" fill="#000000" text-anchor="middle" font-size="12px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><g transform="translate(21.5,31.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="187" height="26" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 20px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 189px; white-space: nowrap; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;"><font style="font-size: 24px">autograd.Variable</font></div></div></foreignObject><text x="94" y="23" fill="#000000" text-anchor="middle" font-size="20px" font-family="Helvetica">[Not supported by viewer]</text></switch></g></g></svg>

6_pytorch/0_basic/ref_dynamic-graph.ipynb

@@ -1,100 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# 动态图和静态图\n",
-    "目前神经网络框架分为[静态图框架和动态图框架](https://blog.csdn.net/qq_36653505/article/details/87875279)，PyTorch 和 TensorFlow、Caffe 等框架最大的区别就是他们拥有不同的计算图表现形式。 TensorFlow 使用静态图，这意味着我们先定义计算图，然后不断使用它，而在 PyTorch 中，每次都会重新构建一个新的计算图。通过这次课程，我们会了解静态图和动态图之间的优缺点。\n",
-    "\n",
-    "对于使用者来说，两种形式的计算图有着非常大的区别，同时静态图和动态图都有他们各自的优点，比如动态图比较方便debug，使用者能够用任何他们喜欢的方式进行debug，同时非常直观，而静态图是通过先定义后运行的方式，之后再次运行的时候就不再需要重新构建计算图，所以速度会比动态图更快。"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "![](https://ws3.sinaimg.cn/large/006tNc79ly1fmai482qumg30rs0fmq6e.gif)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## PyTorch"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# pytorch\n",
-    "import torch\n",
-    "first_counter = torch.Tensor([0])\n",
-    "second_counter = torch.Tensor([10])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "while (first_counter < second_counter):\n",
-    "    first_counter += 2\n",
-    "    second_counter += 1"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "tensor([20.])\n",
-      "tensor([20.])\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(first_counter)\n",
-    "print(second_counter)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "可以看到 PyTorch 的写法跟 Python 的写法是完全一致的，没有任何额外的学习成本\n",
-    "\n",
-    "上面的例子展示如何使用静态图和动态图构建 while 循环，看起来动态图的方式更加简单且直观，你觉得呢？"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.7.9"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

6_pytorch/1-tensor.ipynb

@@ -0,0 +1,726 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Tensor and Variable\n",
+    "\n",
+    "\n",
+    "张量(Tensor)是一种专门的数据结构，非常类似于数组和矩阵。在PyTorch中，我们使用张量来编码模型的输入和输出，以及模型的参数。\n",
+    "\n",
+    "张量类似于`NumPy`的`ndarray`，不同之处在于张量可以在GPU或其他硬件加速器上运行。事实上，张量和NumPy数组通常可以共享相同的底层内存，从而消除了复制数据的需要(请参阅使用NumPy的桥接)。张量还针对自动微分进行了优化，在Autograd部分中看到更多关于这一点的内介绍。\n",
+    "\n",
+    "`variable`是一种可以不断变化的变量，符合反向传播，参数更新的属性。PyTorch的`variable`是一个存放会变化值的内存位置，里面的值会不停变化，像装糖果（糖果就是数据，即tensor）的盒子，糖果的数量不断变化。pytorch都是由tensor计算的，而tensor里面的参数是variable形式。\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Tensor基本用法\n",
+    "\n",
+    "PyTorch基础的数据是张量(Tensor)，PyTorch 的很多操作好 NumPy 都是类似的，但是因为其能够在 GPU 上运行，所以有着比 NumPy 快很多倍的速度。本节内容主要包括 PyTorch 中的基本元素 Tensor 和 Variable 及其操作方式。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.1 Tensor定义与生成"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import torch\n",
+    "import numpy as np"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# 创建一个 numpy ndarray\n",
+    "numpy_tensor = np.random.randn(10, 20)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "可以使用下面两种方式将numpy的ndarray转换到tensor上"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "pytorch_tensor1 = torch.Tensor(numpy_tensor)\n",
+    "pytorch_tensor2 = torch.from_numpy(numpy_tensor)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "使用以上两种方法进行转换的时候，会直接将 NumPy ndarray 的数据类型转换为对应的 PyTorch Tensor 数据类型"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "同时也可以使用下面的方法将 `PyTorch Tensor` 转换为 `NumPy ndarray`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# 如果 pytorch tensor 在 cpu 上\n",
+    "numpy_array = pytorch_tensor1.numpy()\n",
+    "\n",
+    "# 如果 pytorch tensor 在 gpu 上\n",
+    "numpy_array = pytorch_tensor1.cpu().numpy()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "需要注意 GPU 上的 Tensor 不能直接转换为 NumPy ndarray，需要使用`.cpu()`先将 GPU 上的 Tensor 转到 CPU 上"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.2 PyTorch Tensor 使用 GPU 加速\n",
+    "\n",
+    "我们可以使用以下两种方式将 Tensor 放到 GPU 上"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# 第一种方式是定义 cuda 数据类型\n",
+    "dtype = torch.cuda.FloatTensor # 定义默认 GPU 的 数据类型\n",
+    "gpu_tensor = torch.randn(10, 20).type(dtype)\n",
+    "\n",
+    "# 第二种方式更简单，推荐使用\n",
+    "gpu_tensor = torch.randn(10, 20).cuda(0) # 将 tensor 放到第一个 GPU 上\n",
+    "gpu_tensor = torch.randn(10, 20).cuda(1) # 将 tensor 放到第二个 GPU 上"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "使用第一种方式将 tensor 放到 GPU 上的时候会将数据类型转换成定义的类型，而是用第二种方式能够直接将 tensor 放到 GPU 上，类型跟之前保持一致\n",
+    "\n",
+    "推荐在定义 tensor 的时候就明确数据类型，然后直接使用第二种方法将 tensor 放到 GPU 上"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "而将 tensor 放回 CPU 的操作如下"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "cpu_tensor = gpu_tensor.cpu()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Tensor 属性的访问方式"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([10, 20])\n",
+      "torch.Size([10, 20])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 可以通过下面两种方式得到 tensor 的大小\n",
+    "print(pytorch_tensor1.shape)\n",
+    "print(pytorch_tensor1.size())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.FloatTensor\n",
+      "torch.cuda.FloatTensor\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 得到 tensor 的数据类型\n",
+    "print(pytorch_tensor1.type())\n",
+    "print(gpu_tensor.type())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "2\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 得到 tensor 的维度\n",
+    "print(pytorch_tensor1.dim())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "200\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 得到 tensor 的所有元素个数\n",
+    "print(pytorch_tensor1.numel())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Tensor的操作\n",
+    "Tensor 操作中的 API 和 NumPy 非常相似，如果熟悉 NumPy 中的操作，那么 tensor 基本操作是一致的，下面列举其中的一些操作"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2.1 基本操作"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[1., 1.],\n",
+      "        [1., 1.],\n",
+      "        [1., 1.]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = torch.ones(3, 2)\n",
+    "print(x) # 这是一个float tensor"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.FloatTensor\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(x.type())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[1, 1],\n",
+      "        [1, 1],\n",
+      "        [1, 1]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 将其转化为整形\n",
+    "x = x.long()\n",
+    "# x = x.type(torch.LongTensor)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[1., 1.],\n",
+      "        [1., 1.],\n",
+      "        [1., 1.]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 再将其转回 float\n",
+    "x = x.float()\n",
+    "# x = x.type(torch.FloatTensor)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[-1.2200,  0.9769, -2.3477],\n",
+      "        [ 1.0125, -1.3236, -0.2626],\n",
+      "        [-0.3501,  0.5753,  1.5657],\n",
+      "        [ 0.4823, -0.4008, -1.3442]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = torch.randn(4, 3)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# 沿着行取最大值\n",
+    "max_value, max_idx = torch.max(x, dim=1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([0.9769, 1.0125, 1.5657, 0.4823])"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# 每一行的最大值\n",
+    "max_value"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tensor([1, 0, 2, 0])"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# 每一行最大值的下标\n",
+    "max_idx"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([-2.5908, -0.5736,  1.7909, -1.2627])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 沿着行对 x 求和\n",
+    "sum_x = torch.sum(x, dim=1)\n",
+    "print(sum_x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([4, 3])\n",
+      "torch.Size([1, 4, 3])\n",
+      "tensor([[[-1.2200,  0.9769, -2.3477],\n",
+      "         [ 1.0125, -1.3236, -0.2626],\n",
+      "         [-0.3501,  0.5753,  1.5657],\n",
+      "         [ 0.4823, -0.4008, -1.3442]]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 增加维度或者减少维度\n",
+    "print(x.shape)\n",
+    "x = x.unsqueeze(0) # 在第一维增加\n",
+    "print(x.shape)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([1, 1, 4, 3])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = x.unsqueeze(1) # 在第二维增加\n",
+    "print(x.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([1, 4, 3])\n",
+      "tensor([[[-1.2200,  0.9769, -2.3477],\n",
+      "         [ 1.0125, -1.3236, -0.2626],\n",
+      "         [-0.3501,  0.5753,  1.5657],\n",
+      "         [ 0.4823, -0.4008, -1.3442]]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = x.squeeze(0) # 减少第一维\n",
+    "print(x.shape)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([4, 3])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = x.squeeze() # 将 tensor 中所有的一维全部都去掉\n",
+    "print(x.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([3, 4, 5])\n",
+      "torch.Size([4, 3, 5])\n",
+      "torch.Size([5, 3, 4])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = torch.randn(3, 4, 5)\n",
+    "print(x.shape)\n",
+    "\n",
+    "# 使用permute和transpose进行维度交换\n",
+    "x = x.permute(1, 0, 2) # permute 可以重新排列 tensor 的维度\n",
+    "print(x.shape)\n",
+    "\n",
+    "x = x.transpose(0, 2)  # transpose 交换 tensor 中的两个维度\n",
+    "print(x.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([3, 4, 5])\n",
+      "torch.Size([12, 5])\n",
+      "torch.Size([3, 20])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 使用 view 对 tensor 进行 reshape\n",
+    "x = torch.randn(3, 4, 5)\n",
+    "print(x.shape)\n",
+    "\n",
+    "x = x.view(-1, 5) # -1 表示任意的大小，5 表示第二维变成 5\n",
+    "print(x.shape)\n",
+    "\n",
+    "x = x.view(3, 20) # 重新 reshape 成 (3, 20) 的大小\n",
+    "print(x.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[-3.1321, -0.9734,  0.5307,  0.4975],\n",
+      "        [ 0.8537,  1.3424,  0.2630, -1.6658],\n",
+      "        [-1.0088, -2.2100, -1.9233, -0.3059]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = torch.randn(3, 4)\n",
+    "y = torch.randn(3, 4)\n",
+    "\n",
+    "# 两个 tensor 求和\n",
+    "z = x + y\n",
+    "# z = torch.add(x, y)\n",
+    "print(z)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2.2 `inplace`操作\n",
+    "另外，pytorch中大多数的操作都支持 `inplace` 操作，也就是可以直接对 tensor 进行操作而不需要另外开辟内存空间，方式非常简单，一般都是在操作的符号后面加`_`，比如"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "torch.Size([3, 3])\n",
+      "torch.Size([1, 3, 3])\n",
+      "torch.Size([3, 1, 3])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = torch.ones(3, 3)\n",
+    "print(x.shape)\n",
+    "\n",
+    "# unsqueeze 进行 inplace\n",
+    "x.unsqueeze_(0)\n",
+    "print(x.shape)\n",
+    "\n",
+    "# transpose 进行 inplace\n",
+    "x.transpose_(1, 0)\n",
+    "print(x.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[1., 1., 1.],\n",
+      "        [1., 1., 1.],\n",
+      "        [1., 1., 1.]])\n",
+      "tensor([[2., 2., 2.],\n",
+      "        [2., 2., 2.],\n",
+      "        [2., 2., 2.]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "x = torch.ones(3, 3)\n",
+    "y = torch.ones(3, 3)\n",
+    "print(x)\n",
+    "\n",
+    "# add 进行 inplace\n",
+    "x.add_(y)\n",
+    "print(x)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 练习题\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 查阅[PyTorch的Tensor文档](http://pytorch.org/docs/tensors.html)了解 tensor 的数据类型，创建一个 float64、大小是 3 x 2、随机初始化的 tensor，将其转化为 numpy 的 ndarray，输出其数据类型\n",
+    "* 查阅[PyTorch的Tensor文档](http://pytorch.org/docs/tensors.html)了解 tensor 更多的 API，创建一个 float32、4 x 4 的全为1的矩阵，将矩阵正中间 2 x 2 的矩阵，全部修改成2"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 参考\n",
+    "* http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html\n",
+    "* http://cs231n.github.io/python-numpy-tutorial/"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

6_pytorch/0_basic/2-autograd.ipynb → 6_pytorch/2-autograd.ipynb

@@ -5,7 +5,17 @@
    "metadata": {},
    "source": [
     "# 自动求导\n",
-    "这次课程我们会了解 PyTorch 中的自动求导机制，自动求导是 PyTorch 中非常重要的特性，能够让我们避免手动去计算非常复杂的导数，这能够极大地减少了我们构建模型的时间，这也是其前身 Torch 这个框架所不具备的特性，下面我们通过例子看看 PyTorch 自动求导的独特魅力以及探究自动求导的更多用法。"
+    "\n",
+    "自动求导是 PyTorch 中非常重要的特性，能够让我们避免手动去计算非常复杂的导数，这能够极大地减少构建模型的时间。 PyTorch 的 Autograd 模块实现了深度学习的算法中的反向传播求导数，在张量（Tensor类）上的所有操作， Autograd 都能为他们自动提供微分，简化了手动计算导数的复杂过程。\n",
+    "\n",
+    "在PyTorch 0.4以前的版本中， PyTorch 使用 `Variabe` 类来自动计算所有的梯度 `Variable` 类主要包含三个属性 \n",
+    "* Variable 所包含的 Tensor；\n",
+    "* grad：保存 data 对应的梯度，grad 也是个 Variable，而不是 Tensor，它和 data 的形状一样；\n",
+    "* grad_fn：指向一个 Function 对象，这个 Function 用来反向传播计算输入的梯度;\n",
+    "\n",
+    "从 PyTorch 0.4版本起， `Variable` 正式合并入 `Tensor` 类，通过 `Variable` 嵌套实现的自动微分功能已经整合进入了 `Tensor` 类中。虽然为了的兼容性还是可以使用 `Variable`（tensor）这种方式进行嵌套，但是这个操作其实什么都没做。\n",
+    "\n",
+    "以后的代码建议直接使用 `Tensor` 类进行操作，因为官方文档中已经将 `Variable` 设置成过期模块。"
    ]
   },
   {
@@ -14,8 +24,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import torch\n",
-    "from torch.autograd import Variable"
+    "import torch"
    ]
   },
   {
@@ -28,7 +37,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -40,7 +49,7 @@
     }
    ],
    "source": [
-    "x = Variable(torch.Tensor([2]), requires_grad=True)\n",
+    "x = torch.tensor([2.0], requires_grad=True)\n",
     "y = x + 2\n",
     "z = y ** 2 + 3\n",
     "print(z)"
@@ -67,7 +76,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -93,7 +102,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [
     {
@@ -106,16 +115,16 @@
     }
    ],
    "source": [
-    "# 定义Variable\n",
-    "x = Variable(torch.FloatTensor([1,2]), requires_grad=False)\n",
-    "b = Variable(torch.FloatTensor([5,6]), requires_grad=False)\n",
-    "w = Variable(torch.FloatTensor([[1,2],[3,4]]), requires_grad=True)\n",
+    "# 定义变量\n",
+    "x = torch.tensor([1,2], dtype=torch.float, requires_grad=False)\n",
+    "b = torch.tensor([5,6], dtype=torch.float, requires_grad=False)\n",
+    "w = torch.tensor([[1,2],[3,4]], dtype=torch.float, requires_grad=True)\n",
     "print(w)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 9,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -132,7 +141,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 10,
    "metadata": {},
    "outputs": [
     {
@@ -178,7 +187,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "上面数学公式就更加复杂，矩阵乘法之后对两个矩阵对应元素相乘，然后所有元素求平均，有兴趣的同学可以手动去计算一下梯度，使用 PyTorch 的自动求导，我们能够非常容易得到 x, y 和 w 的导数，因为深度学习中充满大量的矩阵运算，所以我们没有办法手动去求这些导数，有了自动求导能够非常方便地解决网络更新的问题。"
+    "上面数学公式的具体含义是：矩阵乘法之后对两个矩阵对应元素相乘，然后所有元素求平均。使用 PyTorch 的自动求导，能够非常容易得到 对 `w` 的导数，因为深度学习中充满大量的矩阵运算，所以手动去求这些导数比较费时间和精力，有了自动求导能够非常方便地解决网络更新的问题。"
    ]
   },
   {
@@ -192,7 +201,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 28,
+   "execution_count": 12,
    "metadata": {},
    "outputs": [
     {
@@ -205,22 +214,22 @@
     }
    ],
    "source": [
-    "m = Variable(torch.FloatTensor([[2, 3]]), requires_grad=True) # 构建一个 1 x 2 的矩阵\n",
-    "n = Variable(torch.zeros(1, 2)) # 构建一个相同大小的 0 矩阵\n",
+    "m = torch.tensor([[2, 3]], dtype=torch.float, requires_grad=True) # 构建一个 1 x 2 的矩阵\n",
+    "n = torch.zeros(1, 2) # 构建一个相同大小的 0 矩阵\n",
     "print(m)\n",
     "print(n)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
+   "execution_count": 13,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "tensor(2., grad_fn=<SelectBackward>)\n",
+      "tensor(2., grad_fn=<SelectBackward0>)\n",
       "tensor([[ 4., 27.]], grad_fn=<CopySlices>)\n"
      ]
     }
@@ -247,7 +256,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "下面我们直接对 n 进行反向传播，也就是求 n 对 m 的导数。\n",
+    "下面我们直接对 `n` 进行反向传播，也就是求 `n` 对 `m` 的导数。\n",
     "\n",
     "这时我们需要明确这个导数的定义，即如何定义\n",
     "\n",
@@ -326,7 +335,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 17,
    "metadata": {},
    "outputs": [
     {
@@ -338,14 +347,14 @@
     }
    ],
    "source": [
-    "x = Variable(torch.FloatTensor([3]), requires_grad=True)\n",
+    "x = torch.tensor([3], dtype=torch.float, requires_grad=True)\n",
     "y = x * 2 + x ** 2 + 3\n",
     "print(y)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 18,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -354,7 +363,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 19,
    "metadata": {},
    "outputs": [
     {
@@ -371,7 +380,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 20,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -380,7 +389,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 24,
+   "execution_count": 21,
    "metadata": {},
    "outputs": [
     {
@@ -446,28 +455,17 @@
     "\\frac{\\partial k_1}{\\partial x_0} & \\frac{\\partial k_1}{\\partial x_1}\n",
     "\\end{matrix}\n",
     "\\right]\n",
-    "$$\n",
-    "\n",
-    "参考答案：\n",
-    "\n",
-    "$$\n",
-    "\\left[\n",
-    "\\begin{matrix}\n",
-    "4 & 3 \\\\\n",
-    "2 & 6 \\\\\n",
-    "\\end{matrix}\n",
-    "\\right]\n",
-    "$$"
+    "$$\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 25,
+   "execution_count": 22,
    "metadata": {},
    "outputs": [],
    "source": [
-    "x = Variable(torch.FloatTensor([2, 3]), requires_grad=True)\n",
-    "k = Variable(torch.zeros(2))\n",
+    "x = torch.tensor([2, 3], dtype=torch.float, requires_grad=True)\n",
+    "k = torch.zeros(2)\n",
     "\n",
     "k[0] = x[0] ** 2 + 3 * x[1]\n",
     "k[1] = x[1] ** 2 + 2 * x[0]"
@@ -475,31 +473,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "tensor([2., 3., 4.], requires_grad=True)\n",
-      "tensor([2., 0., 0.])\n"
-     ]
-    }
-   ],
-   "source": [
-    "# demo to show how to use `.backward`\n",
-    "x = torch.tensor([2,3,4], dtype=torch.float, requires_grad=True)\n",
-    "print(x)\n",
-    "y = x*2\n",
-    "\n",
-    "y.backward(torch.tensor([1, 0, 0], dtype=torch.float))\n",
-    "print(x.grad)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 26,
+   "execution_count": 23,
    "metadata": {},
    "outputs": [
     {
@@ -530,8 +504,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 30,
-   "metadata": {},
+   "execution_count": 24,
+   "metadata": {
+    "scrolled": true
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -545,11 +521,45 @@
    "source": [
     "print(j)"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([2., 3., 4.], requires_grad=True)\n",
+      "tensor([2., 0., 0.])\n"
+     ]
+    }
+   ],
+   "source": [
+    "# demo to show how to use `.backward`\n",
+    "x = torch.tensor([2,3,4], dtype=torch.float, requires_grad=True)\n",
+    "print(x)\n",
+    "y = x*2\n",
+    "\n",
+    "y.backward(torch.tensor([1, 0, 0], dtype=torch.float))\n",
+    "print(x.grad)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 参考资料\n",
+    "* [PyTorch 的 Autograd](https://zhuanlan.zhihu.com/p/69294347)\n",
+    "* [PyTorch学习笔记之自动求导（AutoGrad)](https://zhuanlan.zhihu.com/p/102942725)\n",
+    "* [Pytorch Autograd (自动求导机制)](https://www.cnblogs.com/wangqinze/p/13418291.html)"
+   ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -563,7 +573,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.9"
+   "version": "3.9.7"
   }
  },
  "nbformat": 4,

6_pytorch/1_NN/2-logistic-regression.ipynb → 6_pytorch/4-logistic-regression.ipynb

@@ -239,7 +239,9 @@
   {
    "cell_type": "code",
    "execution_count": 14,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "np_data = np.array(data, dtype='float32') # 转换成 numpy array\n",
@@ -260,7 +262,9 @@
   {
    "cell_type": "code",
    "execution_count": 15,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 定义 logistic 回归模型\n",
@@ -335,7 +339,9 @@
   {
    "cell_type": "code",
    "execution_count": 17,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 计算loss, 使用clamp的目的是防止数据过小而对结果产生较大影响。\n",
@@ -476,7 +482,9 @@
   {
    "cell_type": "code",
    "execution_count": 31,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 使用 torch.optim 更新参数\n",
@@ -616,7 +624,9 @@
   {
    "cell_type": "code",
    "execution_count": 35,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 使用自带的loss\n",
@@ -704,7 +714,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -718,7 +728,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.9"
+   "version": "3.9.7"
   }
  },
  "nbformat": 4,

6_pytorch/1_NN/4-deep-nn.ipynb → 6_pytorch/5-deep-nn.ipynb

@@ -66,7 +66,9 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "import numpy as np\n",
@@ -80,7 +82,9 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 使用内置函数下载 mnist 数据集\n",
@@ -98,7 +102,9 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "a_data, a_label = train_set[0]"
@@ -261,7 +267,9 @@
   {
    "cell_type": "code",
    "execution_count": 9,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "def data_tf(x):\n",
@@ -298,7 +306,9 @@
   {
    "cell_type": "code",
    "execution_count": 11,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "from torch.utils.data import DataLoader\n",
@@ -317,7 +327,9 @@
   {
    "cell_type": "code",
    "execution_count": 12,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "a, a_label = next(iter(train_data))"
@@ -346,7 +358,9 @@
   {
    "cell_type": "code",
    "execution_count": 14,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 使用 Sequential 定义 4 层神经网络\n",
@@ -399,7 +413,9 @@
   {
    "cell_type": "code",
    "execution_count": 16,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 定义 loss 函数\n",
@@ -495,7 +511,9 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "import matplotlib.pyplot as plt\n",
@@ -653,7 +671,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -667,7 +685,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.7"
+   "version": "3.5.4"
   }
  },
  "nbformat": 4,

6_pytorch/1_NN/3-nn-sequential-module.ipynb → 6_pytorch/5-nn-sequential-module.ipynb

@@ -41,7 +41,9 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "import torch\n",
@@ -87,7 +89,9 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "def plot_decision_boundary(model, x, y):\n",
@@ -117,7 +121,9 @@
   {
    "cell_type": "code",
    "execution_count": 5,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 变量\n",
@@ -171,7 +177,9 @@
   {
    "cell_type": "code",
    "execution_count": 7,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "def plot_logistic(x):\n",
@@ -234,7 +242,9 @@
   {
    "cell_type": "code",
    "execution_count": 9,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 定义两层神经网络的参数\n",
@@ -293,7 +303,9 @@
   {
    "cell_type": "code",
    "execution_count": 11,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "def plot_network(x):\n",
@@ -388,7 +400,9 @@
   {
    "cell_type": "code",
    "execution_count": 13,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# Sequential\n",
@@ -448,7 +462,9 @@
   {
    "cell_type": "code",
    "execution_count": 16,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 通过 parameters 可以取得模型的参数\n",
@@ -502,7 +518,9 @@
   {
    "cell_type": "code",
    "execution_count": 18,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "def plot_seq(x):\n",
@@ -556,7 +574,9 @@
   {
    "cell_type": "code",
    "execution_count": 19,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 将参数和模型保存在一起\n",
@@ -573,7 +593,9 @@
   {
    "cell_type": "code",
    "execution_count": 20,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 读取保存的模型\n",
@@ -637,7 +659,9 @@
   {
    "cell_type": "code",
    "execution_count": 23,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 保存模型参数\n",
@@ -769,7 +793,9 @@
   {
    "cell_type": "code",
    "execution_count": 27,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "class module_net(nn.Module):\n",
@@ -791,7 +817,9 @@
   {
    "cell_type": "code",
    "execution_count": 28,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "mo_net = module_net(2, 4, 1)"
@@ -843,7 +871,9 @@
   {
    "cell_type": "code",
    "execution_count": 31,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 定义优化器\n",
@@ -887,7 +917,9 @@
   {
    "cell_type": "code",
    "execution_count": 33,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 保存模型\n",
@@ -920,7 +952,9 @@
   {
    "cell_type": "code",
    "execution_count": 34,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "net = nn.Sequential(\n",
@@ -1021,7 +1055,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -1035,7 +1069,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.7"
+   "version": "3.5.4"
   }
  },
  "nbformat": 4,

6_pytorch/1_NN/5-param_initialize.ipynb → 6_pytorch/6-param_initialize.ipynb

@@ -26,7 +26,9 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "import numpy as np\n",
@@ -37,7 +39,9 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 定义一个 Sequential 模型\n",
@@ -53,7 +57,9 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 访问第一层的参数\n",
@@ -96,7 +102,9 @@
   {
    "cell_type": "code",
    "execution_count": 5,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "# 定义一个 Tensor 直接对其进行替换\n",
@@ -138,7 +146,9 @@
   {
    "cell_type": "code",
    "execution_count": 7,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "for layer in net1:\n",
@@ -173,7 +183,9 @@
   {
    "cell_type": "code",
    "execution_count": 8,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "class sim_net(nn.Module):\n",
@@ -206,7 +218,9 @@
   {
    "cell_type": "code",
    "execution_count": 9,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "net2 = sim_net()"
@@ -304,7 +318,9 @@
   {
    "cell_type": "code",
    "execution_count": 12,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "for layer in net2.modules():\n",
@@ -331,7 +347,9 @@
   {
    "cell_type": "code",
    "execution_count": 13,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "from torch.nn import init"
@@ -436,7 +454,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -450,7 +468,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.7"
+   "version": "3.5.4"
   }
  },
  "nbformat": 4,

README.md

@@ -50,25 +50,26 @@
       - [nn/param_initialize](6_pytorch/1_NN/5-param_initialize.ipynb)
       - [optim/sgd](6_pytorch/1_NN/optimizer/6_1-sgd.ipynb)
       - [optim/adam](6_pytorch/1_NN/optimizer/6_6-adam.ipynb)
+9. [Deep Learning](7_deep_learning/README.md)
    - CNN
-      - [CNN Introduction](6_pytorch/2_CNN/CNN_Introduction.pptx)
+      - [CNN Introduction](7_deep_learning/1_CNN/CNN_Introduction.pptx)
       - [CNN simple demo](demo_code/3_CNN_MNIST.py)
-      - [cnn/basic_conv](6_pytorch/2_CNN/1-basic_conv.ipynb)
-      - [cnn/batch-normalization](6_pytorch/2_CNN/2-batch-normalization.ipynb)
-      - [cnn/lr-decay](6_pytorch/2_CNN/3-lr-decay.ipynb)
-      - [cnn/regularization](6_pytorch/2_CNN/4-regularization.ipynb)
-      - [cnn/vgg](6_pytorch/2_CNN/6-vgg.ipynb)
-      - [cnn/googlenet](6_pytorch/2_CNN/7-googlenet.ipynb)
-      - [cnn/resnet](6_pytorch/2_CNN/8-resnet.ipynb)
-      - [cnn/densenet](6_pytorch/2_CNN/9-densenet.ipynb)
+      - [cnn/basic_conv](7_deep_learning/1_CNN/1-basic_conv.ipynb)
+      - [cnn/batch-normalization](7_deep_learning/1_CNN/2-batch-normalization.ipynb)
+      - [cnn/lr-decay](7_deep_learning/2_CNN/1-lr-decay.ipynb)
+      - [cnn/regularization](7_deep_learning/1_CNN/4-regularization.ipynb)
+      - [cnn/vgg](7_deep_learning/1_CNN/6-vgg.ipynb)
+      - [cnn/googlenet](7_deep_learning/1_CNN/7-googlenet.ipynb)
+      - [cnn/resnet](7_deep_learning/1_CNN/8-resnet.ipynb)
+      - [cnn/densenet](7_deep_learning/1_CNN/9-densenet.ipynb)
    - RNN
-      - [rnn/pytorch-rnn](6_pytorch/3_RNN/pytorch-rnn.ipynb)
-      - [rnn/rnn-for-image](6_pytorch/3_RNN/rnn-for-image.ipynb)
-      - [rnn/lstm-time-series](6_pytorch/3_RNN/time-series/lstm-time-series.ipynb)
+      - [rnn/pytorch-rnn](7_deep_learning/2_RNN/pytorch-rnn.ipynb)
+      - [rnn/rnn-for-image](7_deep_learning/2_RNN/rnn-for-image.ipynb)
+      - [rnn/lstm-time-series](7_deep_learning/2_RNN/time-series/lstm-time-series.ipynb)
    - GAN
-      - [gan/autoencoder](6_pytorch/4_GAN/autoencoder.ipynb)
-      - [gan/vae](6_pytorch/4_GAN/vae.ipynb)
-      - [gan/gan](6_pytorch/4_GAN/gan.ipynb)
+      - [gan/autoencoder](7_deep_learning/3_GAN/autoencoder.ipynb)
+      - [gan/vae](7_deep_learning/3_GAN/vae.ipynb)
+      - [gan/gan](7_deep_learning/3_GAN/gan.ipynb)
 
 
 

references_tips/InstallPython.md

@@ -41,10 +41,26 @@ bash ./Anaconda3-2020.11-Linux-x86_64.sh
 
 参考这里的[conda安装和软件源设置说明](https://mirrors.bfsu.edu.cn/help/anaconda/)
 
-```
-conda config --set show_channel_urls yes
-conda config --add channels https://mirrors.bfsu.edu.cn/anaconda/pkgs/main/
-conda config --add channels https://mirrors.bfsu.edu.cn/anaconda/pkgs/free/
+
+各系统都可以通过修改用户目录下的 `.condarc` 文件。Windows 用户无法直接创建名为 `.condarc` 的文件，可先执行 `conda config --set show_channel_urls yes` 生成该文件之后再修改。
+
+Linux下，打开文件编辑器 `gedit ~/.condarc`，然后把下面的内容拷贝到这个文件中：
+```
+channels:
+  - defaults
+show_channel_urls: true
+default_channels:
+  - https://mirrors.bfsu.edu.cn/anaconda/pkgs/main
+  - https://mirrors.bfsu.edu.cn/anaconda/pkgs/r
+  - https://mirrors.bfsu.edu.cn/anaconda/pkgs/msys2
+custom_channels:
+  conda-forge: https://mirrors.bfsu.edu.cn/anaconda/cloud
+  msys2: https://mirrors.bfsu.edu.cn/anaconda/cloud
+  bioconda: https://mirrors.bfsu.edu.cn/anaconda/cloud
+  menpo: https://mirrors.bfsu.edu.cn/anaconda/cloud
+  pytorch: https://mirrors.bfsu.edu.cn/anaconda/cloud
+  pytorch-lts: https://mirrors.bfsu.edu.cn/anaconda/cloud
+  simpleitk: https://mirrors.bfsu.edu.cn/anaconda/cloud
 ```