diff --git a/5_nn/2-mlp_bp.ipynb b/5_nn/2-mlp_bp.ipynb index 7a79ea5..7774d13 100644 --- a/5_nn/2-mlp_bp.ipynb +++ b/5_nn/2-mlp_bp.ipynb @@ -1026,7 +1026,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.5.4" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/5_nn/3-softmax_ce.ipynb b/5_nn/3-softmax_ce.ipynb index 76a294e..5ca8168 100644 --- a/5_nn/3-softmax_ce.ipynb +++ b/5_nn/3-softmax_ce.ipynb @@ -196,7 +196,7 @@ " & = & -a_j a_i\n", "\\end{eqnarray}\n", "\n", - "当u,v都是变量的函数时的导数推导公式:\n", + "当$u$,$v$都是变量的函数时的导数推导公式:\n", "$$\n", "(\\frac{u}{v})' = \\frac{u'v - uv'}{v^2} \n", "$$\n", @@ -222,21 +222,41 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "参数更新方程为\n", + "### 3.4 参数更新\n", + "\n", + "误差与参数的偏导为:\n", "$$\n", "\\frac{\\partial C}{\\partial w_{ij}} = (-y_i + a_i) x_i\n", "$$\n", "\n", + "误差项为:\n", + "$$\n", + "\\delta_i = -(-y_i + a_i)\n", + "$$\n", + "\n", + "参数跟新公式为:\n", + "$$\n", + "w_{ij} = w_{ij} + \\eta \\delta_i x_i\n", + "$$\n", + "\n", + "\n", "其中\n", "$$\n", + "a_i = \\frac{e^{z_i}}{\\sum_k e^{z_k}}\n", + "$$\n", + "\n", + "$$\n", "z_i = \\sum_{j} w_{ij} x_{j} + w_b\n", - "$$\n" + "$$\n", + "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ + "### 3.4 二次代价函数的更行方程\n", + "\n", "最为对比,使用二次代价函数的更新方程为:\n", "\n", "$$\n", @@ -245,7 +265,9 @@ "\n", "$$\n", "w_{ji} = w_{ji} + \\eta \\delta_j x_{ji}\n", - "$$" + "$$\n", + "\n", + "需要注意这里 $w_{ji}$ 和上面的定义不太一样!" ] }, { @@ -286,7 +308,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.5.4" + "version": "3.7.9" } }, "nbformat": 4,