Browse Source

Improve cross-entropy loss equations

pull/10/MERGE
bushuhui 3 years ago
parent
commit
2b4e6d75e1
2 changed files with 28 additions and 6 deletions
  1. +1
    -1
      5_nn/2-mlp_bp.ipynb
  2. +27
    -5
      5_nn/3-softmax_ce.ipynb

+ 1
- 1
5_nn/2-mlp_bp.ipynb View File

@@ -1026,7 +1026,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.4"
"version": "3.7.9"
}
},
"nbformat": 4,


+ 27
- 5
5_nn/3-softmax_ce.ipynb View File

@@ -196,7 +196,7 @@
" & = & -a_j a_i\n",
"\\end{eqnarray}\n",
"\n",
"当u,v都是变量的函数时的导数推导公式:\n",
"当$u$$v$都是变量的函数时的导数推导公式:\n",
"$$\n",
"(\\frac{u}{v})' = \\frac{u'v - uv'}{v^2} \n",
"$$\n",
@@ -222,21 +222,41 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"参数更新方程为\n",
"### 3.4 参数更新\n",
"\n",
"误差与参数的偏导为:\n",
"$$\n",
"\\frac{\\partial C}{\\partial w_{ij}} = (-y_i + a_i) x_i\n",
"$$\n",
"\n",
"误差项为:\n",
"$$\n",
"\\delta_i = -(-y_i + a_i)\n",
"$$\n",
"\n",
"参数跟新公式为:\n",
"$$\n",
"w_{ij} = w_{ij} + \\eta \\delta_i x_i\n",
"$$\n",
"\n",
"\n",
"其中\n",
"$$\n",
"a_i = \\frac{e^{z_i}}{\\sum_k e^{z_k}}\n",
"$$\n",
"\n",
"$$\n",
"z_i = \\sum_{j} w_{ij} x_{j} + w_b\n",
"$$\n"
"$$\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.4 二次代价函数的更行方程\n",
"\n",
"最为对比,使用二次代价函数的更新方程为:\n",
"\n",
"$$\n",
@@ -245,7 +265,9 @@
"\n",
"$$\n",
"w_{ji} = w_{ji} + \\eta \\delta_j x_{ji}\n",
"$$"
"$$\n",
"\n",
"需要注意这里 $w_{ji}$ 和上面的定义不太一样!"
]
},
{
@@ -286,7 +308,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.4"
"version": "3.7.9"
}
},
"nbformat": 4,


Loading…
Cancel
Save