Improve cross-entropy loss equations

5_nn/2-mlp_bp.ipynb

@@ -1026,7 +1026,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.5.4"
+   "version": "3.7.9"
   }
  },
  "nbformat": 4,

5_nn/3-softmax_ce.ipynb

@@ -196,7 +196,7 @@
     "  & = & -a_j a_i\n",
     "\\end{eqnarray}\n",
     "\n",
-    "当u，v都是变量的函数时的导数推导公式：\n",
+    "当$u$，$v$都是变量的函数时的导数推导公式：\n",
     "$$\n",
     "(\\frac{u}{v})' = \\frac{u'v - uv'}{v^2} \n",
     "$$\n",
@@ -222,21 +222,41 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "参数更新方程为\n",
+    "### 3.4 参数更新\n",
+    "\n",
+    "误差与参数的偏导为：\n",
     "$$\n",
     "\\frac{\\partial C}{\\partial w_{ij}} = (-y_i + a_i) x_i\n",
     "$$\n",
     "\n",
+    "误差项为：\n",
+    "$$\n",
+    "\\delta_i = -(-y_i + a_i)\n",
+    "$$\n",
+    "\n",
+    "参数跟新公式为：\n",
+    "$$\n",
+    "w_{ij} = w_{ij} + \\eta \\delta_i x_i\n",
+    "$$\n",
+    "\n",
+    "\n",
     "其中\n",
     "$$\n",
+    "a_i = \\frac{e^{z_i}}{\\sum_k e^{z_k}}\n",
+    "$$\n",
+    "\n",
+    "$$\n",
     "z_i = \\sum_{j} w_{ij} x_{j} + w_b\n",
-    "$$\n"
+    "$$\n",
+    "\n"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "### 3.4 二次代价函数的更行方程\n",
+    "\n",
     "最为对比，使用二次代价函数的更新方程为：\n",
     "\n",
     "$$\n",
@@ -245,7 +265,9 @@
     "\n",
     "$$\n",
     "w_{ji} = w_{ji} + \\eta \\delta_j x_{ji}\n",
-    "$$"
+    "$$\n",
+    "\n",
+    "需要注意这里 $w_{ji}$ 和上面的定义不太一样！"
    ]
   },
   {
@@ -286,7 +308,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.5.4"
+   "version": "3.7.9"
   }
  },
  "nbformat": 4,