machinelearning_notebook/4_logistic_regression/1-Least_squares.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 最小二乘\n",
    "\n",
    "## 1. 最小二乘的基本原理\n",
    "\n",
    "最小二乘法（Least Squares）是一种数学优化技术，它通过最小化误差的平方和找到一组数据的最佳函数匹配， 最小二乘法通常用于曲线拟合、求解模型。很多其他的优化问题也可通过最小化能量或最大化熵用最小二乘形式表达。\n",
    "\n",
    "![ls_theory](images/least_squares.png)\n",
    "\n",
    "最小二乘原理的一般形式为：\n",
    "$$\n",
    "L = \\sum (V_{obv} - V_{target}(\\theta))^2\n",
    "$$\n",
    "其中\n",
    "* $V_{obv}$是观测的多组样本值\n",
    "* $V_{target}$是假设拟合函数的输出值\n",
    "* $\\theta$为构造模型的参数\n",
    "* $L$是目标函数\n",
    "\n",
    "如果通过调整模型参数$\\theta$，使得$L$下降到最小则表明，拟合函数与观测最为接近，也就是找到了最优的模型。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 示例\n",
    "\n",
    "假设我们有下面的一些观测数据，希望找到它们内在的规律。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVZElEQVR4nO3df4xcV3nG8eepY2D50W7SrJCzwbVFkdMoEXbYRqFugTgFh4KKCS0tbVFaIdyq0EIAF4dWgkqtsiUhKVLVSIYEgppSaGIMSlJCFBshkJp2Hbt2ghOVQhyyMfGisgSoBYl5+8fcjb3jnfXO7Jz749zvR1p55u7s3tc38TNnzjn3HEeEAADt8TNVFwAAKBfBDwAtQ/ADQMsQ/ADQMgQ/ALTMGVUXsBRnn312rFmzpuoyAKBR9u7d+92IGOs+3ojgX7NmjaampqouAwAaxfbhhY7T1QMALUPwA0DLEPwA0DIEPwC0DMEPAC3TiFk9ANAEu/ZN69q7H9bjs8d0zuiItm1epy0bxqsu6xQEPwAMwa5907p650Ede+q4JGl69piu3nlQkmoX/nT1AMAQXHv3w8+E/pxjTx3XtXc/XFFFvRH8ADAEj88e6+t4lQh+ABiCc0ZH+jpeJYIfAIZg2+Z1Glm5Yt6xkZUrtG3zuoF+365909o4uVtrt9+pjZO7tWvf9DDKlMTgLgAMxdwA7jBm9aQeKE4W/LafI+krkp5dnOe2iPig7U9KeqWk7xcv/cOI2J+qDgAoy5YN40MJ5sUGimsd/JJ+LGlTRPzQ9kpJX7X9b8X3tkXEbQnPDQCNlXqgOFkff3T8sHi6sviKVOcDgFykHihOOrhre4Xt/ZKOSronIu4rvvW3tg/YvsH2s3v87FbbU7anZmZmUpYJALUy7IHibkmDPyKOR8R6SedKutj2BZKulnSepF+WdJak9/f42R0RMRERE2Njp2wgAwDZ2rJhXNdccaHGR0dkSeOjI7rmiguHdgdwKbN6ImLW9h5Jl0fEdcXhH9v+hKT3lVEDADTJsAaKF5KsxW97zPZo8XhE0qslPWR7VXHMkrZIeiBVDQCAU6Vs8a+SdIvtFeq8wXw2Iu6wvdv2mCRL2i/pTxLWAADokiz4I+KApA0LHN+U6pwAgNNjyQYAaBmCHwBahuAHgJYh+AGgZQh+AGgZlmUG0ApN2QhdSl8rwQ8ge03aCL2MWunqAZC9Jm2EXkatBD+A7DVpI/QyaiX4AWSvSRuhl1ErwQ+gcfrdiDz1+vbDVEatDO4CaJRBBj+HuRF6amXU6oj674Y4MTERU1NTVZcBoAY2Tu7W9AL93eOjI/radtaAPJntvREx0X2crh4AjdKkgdq6IvgBNEqTBmrriuAH0ChNGqitKwZ3ATRKkwZq64rgB9A4KTcibwO6egCgZQh+AGiZZMFv+zm2/8P2f9l+0PZfF8fX2r7P9jdsf8b2s1LVAAA4VcoW/48lbYqIl0paL+ly25dI+jtJN0TEL0r6nqS3JawBANAl2eBudG4J/mHxdGXxFZI2Sfq94vgtkj4k6cZUdQDAnCZtxpJS0j5+2yts75d0VNI9kv5H0mxEPF285DFJC15121ttT9mempmZSVkmgBaYW+NnevaYQifW+DndAm85Shr8EXE8ItZLOlfSxZLO6+Nnd0TERERMjI2NpSoRQEs0aTOW1EqZ1RMRs5L2SHq5pFHbc11M50pq39stgNKxxs8JKWf1jNkeLR6PSHq1pEPqvAH8VvGyKyV9PlUNADCHNX5OSNniXyVpj+0Dkv5T0j0RcYek90t6j+1vSPp5STclrAEAJLHGz8lSzuo5IGnDAse/qU5/PwCUhjV+TmCtHgCtMcgaPzlOASX4AaCHQbZ5bAKCHwC0cMt+sSmgBD+QsRw/6mO+Xi377tCf0/QpoAQ/sIhcP+rXTdVvrr1a9itsHY845fVNnwJK8AOLyPWjfh3Mhf307DFZnYW8pGreXHu14I9HaGTlinn/D+QwBZT1+IFFcLdnGn+166Cu+sx+TRfXsbtNXfZSCr1a8OOjI7rmigs1Pjoin/S86W/6tPiBRZwzOvJMOHUfx2B27ZvWrf/+6Clh363MN9dtm9ed0qc/17LPcZtHgh9YxGKBgNPrNVPmdKEvlfvm2rabuwh+YBFtC4Rh6nemzMmqeHPNsWXfC8EPnEadAqHq2S/96HemzJzxmv+9ckDwAw3RtKml/cyUsaTfv2S1/mbLhSVV127M6gEqtGvftDZO7tba7Xdq4+TuRXeDatpGIv3MlLnhd9YT+iWixQ9UpN8WfNOmlrZtpkyT0OIHKtJvC75pG4ls2TCe5Rz4HNDiByrSbwu+iVNLadnXEy1+oCL9tuBpQWNYaPEDFRmkBU8LGsNA8AMV4eYwVIXgBypECx5VSNbHb/tFtvfY/rrtB22/qzj+IdvTtvcXX7+RqgYAwKlStviflvTeiLjf9gsk7bV9T/G9GyLiuoTnBpCRJi1V0QTJgj8ijkg6Ujz+ge1DkvgvBaAvTVuqoglKmc5pe42kDZLuKw690/YB2zfbPrPHz2y1PWV7amZmpowyAdRQ05aqaILkwW/7+ZJul/TuiHhS0o2SXixpvTqfCD6y0M9FxI6ImIiIibGxsdRlAqippi1V0QRJZ/XYXqlO6N8aETslKSKeOOn7H5N0R8oaANRLv/31Ze6C1paxhJSzeizpJkmHIuL6k46vOullb5T0QKoaANTLXH/99OwxhU701y+2Kum2zes0snLFvGMplqoYpLamStnVs1HSWyVt6pq6+WHbB20fkHSppKsS1gCgRgbpry9rqYo2jSWknNXzVXX2V+h2V6pzAqi3Qfvry7jRrU1jCSzSBqA0dV5aus61DRvBD6A0ZfXXD6LOtQ0ba/UAKE2dF6arc23D5lhkt/u6mJiYiKmpqarLAIBGsb03Iia6j9PVAwAtQ/ADQMsQ/ADQMgzuIittueW+7fjvvDwEP7Kx1OV76x4ada+vaizTvHx09SAbS7nlvu7rsdS9vjpo09IKqRD8yMZSbrmve2jUvb46aNPSCqkQ/MjGUm65r3to1L2+OmjT0gqpEPzIxlJuua97aAyzvl37prVxcrfWbr9TGyd3Z9Nd1KalFVIh+JGNpSzfW/fQGFZ9OY8VlLVMc85YsgGtU/dZM8Oob+Pk7gV3rRofHdHXtm8aVqmouV5LNhD8QE0t5w1g7fY7tdC/bEv61uTrhlon6qtX8DOPH1imFJ8gljtXvcx9atE89PEDhUEGQ1P1pS93WmfdxzJQLYIf0OABnmre/XKndTIAisUk6+qx/SJJn5L0QkkhaUdEfNT2WZI+I2mNpEckvTkivpeqDuRtWN0siwX4Yr8v1bz7YXTVlLFPLZopZYv/aUnvjYjzJV0i6R22z5e0XdK9EfESSfcWz4G+DbObZdAAT3VfAF01SClZ8EfEkYi4v3j8A0mHJI1LeoOkW4qX3SJpS6oakLdhdrMMGuCpApquGqRUyqwe22skbZB0n6QXRsSR4lvfUacrCOjbMLtZtm1eN28WjbS0AE+5TytdNUglefDbfr6k2yW9OyKetP3M9yIibC94I4HtrZK2StLq1atTl4kGGuaUxaUEeK/xBAIaTZP0Bi7bKyXdIenuiLi+OPawpFdFxBHbqyR9OSIWbVZxAxcW0j3XXeq00lN0iZR5LmBYSt9s3Z2m/U2SDs2FfuELkq4sHl8p6fOpakDeyuwHZ7lk5KRnV4/tuyT9aUQ8MuDv3ijprZIO2t5fHPuApElJn7X9NkmHJb15wN8PlNbNwnLJyMliffyfkPQl27dI+nBEPNXPL46Ir6qzNMhCLuvnd6G96rKgWtOWQKjLdUM99ezqiYh/lXSRpJ+VNGX7fbbfM/dVWoVorTotLdykefV1um6op9P18f9E0o8kPVvSC7q+gKTq1K/epHn1dbpuqKfF+vgvl3S9OoOxF0XE/5VWFQaS28f7KvvVF5u6WXeMR+B0Fuvj/0tJvx0RD5ZVDAa33GV866iqfvWmX8umjUegfIv18f8aod8cOX68X6hf3ZIuPW8s6Xmbfi2bNB6BarAscyZy/Hi/ZcO43vSy8XlTw0LS7Xunkw5UNv1aNmk8AtVgB65M5Prxfs9DM6dsIbiU5ZKXI4dr2ZTxCFSDFn8mcv14X0XrO9drCcyhxZ+JlKtEVqmK1neu1xKYk3SRtmFhkbb2YnE0YHC9FmmjxY9ao/UNDB/Bj9pjoBIYLgZ3AaBlCH4AaBmCHwBahuAHgJYh+AGgZQh+AGgZgh8AWobgB4CWSXYDl+2bJb1e0tGIuKA49iFJb5c0U7zsAxFxV6oa0D5N3IWsiTWj2VLeuftJSf8g6VNdx2+IiOsSnhct1cSds5pYM5ovWVdPRHxF0v+m+v1AtybunNXEmtF8VfTxv9P2Ads32z6zgvMjU03cOauJNaP5yg7+GyW9WNJ6SUckfaTXC21vtT1le2pmZqbXy4Bn9Fqjv847ZzWxZjRfqcEfEU9ExPGI+Kmkj0m6eJHX7oiIiYiYGBtLu7k28tDEnbOaWDOar9RlmW2viogjxdM3SnqgzPMjb01cu7+JNaP5ku3AZfvTkl4l6WxJT0j6YPF8vaSQ9IikPz7pjaAnduBCnTD9Ek1R+g5cEfGWBQ7flOp8QBmYfokccOcu0AemXyIHBD/QB6ZfIgcEP9AHpl8iBwQ/0AemXyIHpU7nBJqO6ZfIAcHfUGVOKWT64nxbNoy3+u+P5iP4SzSsAC1zSiHTF4H80MdfkrkAnZ49ptCJAN21b7rv31XmlEKmLwL5ocVfksUCtN+Wc5lTCnOavkiXFdBBi78kwwzQMqcU5jJ9cZifuICmI/hLMswALXNK4ULnsqRLz2vWiql0WQEnEPwL2LVvWhsnd2vt9ju1cXL3UFqFwwzrLRvGdc0VF2p8dESWND46omuuuDBJt8WWDeN608vG5ZOOhaTb9043qrWcU5cVsFz08XdJNYtl2PO/y5xSuOehGXWv4Tro+ERVzhkd0fQCId+0LitgGAj+LsMchO3W1PnfObSWt21eN+8NXeKOW7QXXT1dcgi5YcthgLfM7jGg7mjxd6FL4FS5tJab+okLGLZsg3/QOdu5hFwvg1wX1qcB8pJl8C9ngDbnkFvudcnhGgDINPiXO0Cba8ilHLgG0BxZDu4yQLswrgsAKWHw277Z9lHbD5x07Czb99j+7+LPM1OcO4dZKClwXQBIaVv8n5R0edex7ZLujYiXSLq3eD507JK0MK4LAClhH39EfMX2mq7Db5D0quLxLZK+LOn9wz53zgO0y8F1ASBJjui+GX+Iv7wT/HdExAXF89mIGC0eW9L35p4v8LNbJW2VpNWrV7/s8OHDyeoEgBzZ3hsRE93HKxvcjc47Ts93nYjYERETETExNtaslSABoM7KDv4nbK+SpOLPoyWfHwBar+x5/F+QdKWkyeLPz5d8fiwRu1UB+UoW/LY/rc5A7tm2H5P0QXUC/7O23ybpsKQ3pzo/BscG60DeUs7qeUuPb12W6pwYDu7wBfKW5Z27WB7u8AXyluVaPW00zD55lqYG8kaLPwNzffLTs8cUOtEnP+ieuEu5wzfFvsQAykHwZ2CxPvlBnG63qmG/0QAoF109GUjRJ7/Y0tQM/gLNRos/A2WvusngL9BsBH8Gylp1c65fv9c6Gwz+As1AV08Gylh1s/umrm4s7ww0B8GfidTbRS7Urz9nnCUdgEYh+LEkvfrvLelr2zeVWwyAZaGPH0vCto1APgh+LAnbNgL5oKsHS8K2jUA+CH4sWeoBZADloKsHAFqGFn8P7EAFIFcE/wLYgQpAzujqWcCwV7sEgDoh+BfAImQAckbwL4CblQDkrJLgt/2I7YO299ueqqKGxXCzEoCcVTm4e2lEfLfC8/fEzUoAcsasnh64WQlArqrq4w9JX7K91/bWhV5ge6vtKdtTMzMzJZcHAPmqKvh/NSIukvRaSe+w/YruF0TEjoiYiIiJsbGx8isEgExVEvwRMV38eVTS5yRdXEUdANBGpQe/7efZfsHcY0mvkfRA2XUAQFtVMbj7Qkmfsz13/n+OiC9WUAcAtFLpwR8R35T00rLPCwDo4M5dAGgZgh8AWobgB4CWIfgBoGUIfgBoGYIfAFqG4AeAliH4AaBlCH4AaJns1+PftW+aDVUA4CRZB/+ufdO6eudBHXvquCRpevaYrt55UJIIfwCtlXVXz7V3P/xM6M859tRxXXv3wxVVBADVyzr4H5891tdxAGiDrIP/nNGRvo4DQBtkHfzbNq/TyMoV846NrFyhbZvXVVQRAFQv68HduQFcZvUAwAlZB7/UCX+CHgBOyLqrBwBwKoIfAFqG4AeAliH4AaBlCH4AaBlHRNU1nJbtGUmHq66jBs6W9N2qi6gRrsd8XI/5uB7SL0TEWPfBRgQ/OmxPRcRE1XXUBddjPq7HfFyP3ujqAYCWIfgBoGUI/mbZUXUBNcP1mI/rMR/Xowf6+AGgZWjxA0DLEPwA0DIEf83ZfpHtPba/bvtB2++quqY6sL3C9j7bd1RdS9Vsj9q+zfZDtg/ZfnnVNVXJ9lXFv5UHbH/a9nOqrqluCP76e1rSeyPifEmXSHqH7fMrrqkO3iXpUNVF1MRHJX0xIs6T9FK1+LrYHpf055ImIuICSSsk/W61VdUPwV9zEXEkIu4vHv9AnX/Urd5gwPa5kl4n6eNV11I12z8n6RWSbpKkiPhJRMxWWlT1zpA0YvsMSc+V9HjF9dQOwd8gttdI2iDpvopLqdrfS/oLST+tuI46WCtpRtIniq6vj9t+XtVFVSUipiVdJ+lRSUckfT8ivlRtVfVD8DeE7edLul3SuyPiyarrqYrt10s6GhF7q66lJs6QdJGkGyNig6QfSdpebUnVsX2mpDeo84Z4jqTn2f6DaquqH4K/AWyvVCf0b42InVXXU7GNkn7T9iOS/kXSJtv/VG1JlXpM0mMRMfcp8DZ13gja6tclfSsiZiLiKUk7Jf1KxTXVDsFfc7atTv/toYi4vup6qhYRV0fEuRGxRp1Bu90R0doWXUR8R9K3ba8rDl0m6esVllS1RyVdYvu5xb+dy9Tiwe5est9sPQMbJb1V0kHb+4tjH4iIu6orCTXzZ5Jutf0sSd+U9EcV11OZiLjP9m2S7ldnRtw+sXTDKViyAQBahq4eAGgZgh8AWobgB4CWIfgBoGUIfgBoGYIf6FOxYuq3bJ9VPD+zeL6m4tKAJSH4gT5FxLcl3Shpsjg0KWlHRDxSWVFAH5jHDwygWEZjr6SbJb1d0vpiiQCg9rhzFxhARDxle5ukL0p6DaGPJqGrBxjca9VZ+veCqgsB+kHwAwOwvV7Sq9XZFe0q26uqrQhYOoIf6FOx6uON6uyN8Kika9XZ/ANoBIIf6N/bJT0aEfcUz/9R0i/ZfmWFNQFLxqweAGgZWvwA0DIEPwC0DMEPAC1D8ANAyxD8ANAyBD8AtAzBDwAt8/8F/I4RSbbD3wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 生成数据\n",
    "data_num = 50\n",
    "X = np.random.rand(data_num, 1)*10\n",
    "Y = X * 3 + 4 + 4*np.random.randn(data_num,1)\n",
    "\n",
    "# 画出数据的分布\n",
    "plt.scatter(X, Y)\n",
    "plt.xlabel(\"X\")\n",
    "plt.ylabel(\"Y\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 数学原理\n",
    "有$N$个观测数据为：\n",
    "$$\n",
    "\\mathbf{X} = \\{x_1, x_2, ..., x_N \\} \\\\\n",
    "\\mathbf{Y} = \\{y_1, y_2, ..., y_N \\}\n",
    "$$\n",
    "其中$\\mathbf{X}$为自变量，$\\mathbf{Y}$为因变量。\n",
    "\n",
    "希望找到一个模型能够解释这些数据，假设使用最简单的线性模型来拟合数据：\n",
    "$$\n",
    "y = ax + b\n",
    "$$\n",
    "那么问题就变成求解参数$a$, $b$能够使得模型输出尽可能和观测数据有比较小的误差。\n",
    "\n",
    "如何构建函数来评估模型输出与观测数据之间的误差是一个关键问题，这里我们使用观测数据与模型输出的平方和来作为评估函数（也被称为损失函数Loss function):\n",
    "$$\n",
    "L = \\sum_{i=1}^{N} \\{y_i - (a x_i + b)\\}^2 \\\\\n",
    "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2 \n",
    "$$\n",
    "\n",
    "使误差函数最小，那么我们就可以求出模型的参数:\n",
    "$$\n",
    "\\frac{\\partial L}{\\partial a} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i \\\\\n",
    "\\frac{\\partial L}{\\partial b} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b)\n",
    "$$\n",
    "\n",
    "即当偏微分为0时，误差函数为最小，因此我们可以得到:\n",
    "$$\n",
    "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i = 0 \\\\\n",
    "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) = 0 \\\\\n",
    "$$\n",
    "\n",
    "将上式调整一下顺序可以得到：\n",
    "$$\n",
    "a \\sum x_i^2 + b \\sum x_i = \\sum y_i x_i \\\\\n",
    "a \\sum x_i + b N = \\sum y_i\n",
    "$$\n",
    "\n",
    "上式中$\\sum x_i^2$, $\\sum x_i$, $\\sum y_i$, $\\sum y_i x_i$都是已知的数据，而参数$a$, $b$是我们想要求得未知参数。通过求解二元一次方程组，我们即可求出模型的最优参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 求解程序"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a = 2.951846, b = 4.837623\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAl2ElEQVR4nO3deXhU5d3/8fdXjBIBAQUpghbaKiCLBIPSxg2o4CMugI+t1gWsFmvrhgpF+yju0opL+/yqfagbKrgBRkUoVQERVJAlhl1RgxKpIoqKQoVw//44MyEJWSaZM3POmfm8rouL5GQy8+Vw5TN37tWcc4iISPTsFXQBIiLSMApwEZGIUoCLiESUAlxEJKIU4CIiEbV3Ol+sVatWrkOHDul8SRGRyFuyZMnnzrnWVa+nNcA7dOjA4sWL0/mSIiKRZ2brq7uuLhQRkYhSgIuIRJQCXEQkotLaB16dHTt2sGHDBrZv3x50KVmtcePGtG/fnpycnKBLEZEEBR7gGzZsoFmzZnTo0AEzC7qcrOScY/PmzWzYsIGOHTsGXY6IJCjwAN++fbvCO2BmxoEHHsimTZuCLkUkFAqXlXLXrLV8smUbB7fIZdTATgzOaxd0WXsIPMABhXcI6P9AxFO4rJTrpi1n244yAEq3bOO6acsBQhfiGsQUEangrllry8M7btuOMu6atTagimqmAPdZhw4d+Pzzz5N+jIgE45Mt2+p1PUgKcBGRCg5ukVuv60FSgAMlJSV07tyZ4cOHc/jhh3PuuefyyiuvUFBQwGGHHcaiRYv44osvGDx4MD169KBPnz4UFxcDsHnzZgYMGEDXrl25+OKLqXjC0RNPPMHRRx9Nz549ueSSSygrK6upBBEJiVEDO5Gb06jStdycRowa2KlBz1e4rJSCcbPpOOYlCsbNpnBZqR9lAiEZxCx31VVQVOTvc/bsCffdV+fD1q1bx7PPPsvDDz9M7969mTx5MvPnz+eFF17gjjvu4JBDDiEvL4/CwkJmz57NBRdcQFFRETfffDPHHnssN954Iy+99BIPPfQQAKtXr+bpp59mwYIF5OTk8Lvf/Y5JkyZxwQUX+PvvExFfxQcq/ZiFkuoB0ToD3MwaA/OAfWOPn+KcG2tmjwInAF/FHjrcOVeUdEUB6dixI927dwega9eu9O/fHzOje/fulJSUsH79eqZOnQpAv3792Lx5M19//TXz5s1j2rRpAAwaNIiWLVsC8Oqrr7JkyRJ69+4NwLZt2zjooIMC+JeJSH0NzmvnS8DWNiCalgAH/gP0c85tNbMcYL6ZzYx9bZRzbkrSVcQl0FJOlX333bf847322qv887322oudO3fWe4Wic45hw4Zx5513+lqniERHqgdE6+wDd56tsU9zYn+y7ij74447jkmTJgEwd+5cWrVqxf7778/xxx/P5MmTAZg5cyZffvklAP3792fKlCl89tlnAHzxxResX1/tjpAikqFSPSCa0CCmmTUysyLgM+Bl59zC2JduN7NiM7vXzPat4XtHmNliM1sc5ZV+N910E0uWLKFHjx6MGTOGiRMnAjB27FjmzZtH165dmTZtGoceeigARxxxBLfddhsDBgygR48enHTSSWzcuDHIf4KIpJnfA6JVWcVZE3U+2KwF8BxwObAZ+DewDzABeN85d0tt35+fn++qHuiwevVqunTpUr+qJSX0fyHiPz+W5ZvZEudcftXr9ZqF4pzbYmZzgJOdc+Njl/9jZo8A19arIhGRLODXgGh16uxCMbPWsZY3ZpYLnASsMbO2sWsGDAZWpKRCERGpViIt8LbARDNrhBf4zzjnppvZbDNrDRhQBPw2dWWKiEhVdQa4c64YyKvmer+UVCQiIgnRUnoRkYhSgIuIRJQCvB5OOeUUtmzZUutjbrzxRl555ZUGPf/cuXM59dRT63zciSeeSNXpmFXdd999fPfddw2qQ0SiQQGeAOccu3btYsaMGbRo0aLWx95yyy38/Oc/T09htVCAi2S+yAV4KrZmvOeee+jWrRvdunXjvth+LCUlJXTq1IkLLriAbt268fHHH1c6iOHWW2+lU6dOHHvssZxzzjmMH+9Nix8+fDhTpnjbw3To0IGxY8fSq1cvunfvzpo1awBYtGgRP/3pT8nLy+NnP/sZa9fWftLHtm3bOPvss+nSpQtDhgxh27bd+yhceuml5Ofn07VrV8aOHQvAX//6Vz755BP69u1L3759a3yciERbuLaTrUMqtmZcsmQJjzzyCAsXLsQ5xzHHHMMJJ5xAy5Ytee+995g4cSJ9+vSp9D1vv/02U6dO5Z133mHHjh306tWLo446qtrnb9WqFUuXLuX+++9n/PjxPPjgg3Tu3JnXX3+dvffem1deeYXrr7++fKfD6jzwwAPst99+rF69muLiYnr16lX+tdtvv50DDjiAsrIy+vfvT3FxMVdccQX33HMPc+bMoVWrVjU+rkePHg26ZyJBisqBw7C71n9/sZUfHNDU91oj1QJPxVl18+fPZ8iQITRp0oSmTZsydOhQXn/9dQB++MMf7hHeAAsWLOCMM86gcePGNGvWjNNOO63G5x86dCgARx11FCUlJQB89dVXnHXWWXTr1o2RI0eycuXKWmucN28e5513HgA9evSoFLzPPPMMvXr1Ii8vj5UrV7Jq1apqnyPRx4mEWbwRV7plG47djTg/D0nwS+GyUm58diknzJ3G3AkjaPLeat9rjVSAp/usuiZNmiT9HPFtaRs1asTOnTsBuOGGG+jbty8rVqzgxRdfZPv27Q167g8//JDx48fz6quvUlxczKBBg6p9rkQfJxJ2kTlw+PvvefeW8cz420XcMetvfNakJTllO32vNVIBnoqtGY877jgKCwv57rvv+Pbbb3nuuec47rjjav2egoKC8uDdunUr06dPr9drfvXVV7Rr5/0a9eijj9b5+Ipb1q5YsaL8OLevv/6aJk2a0Lx5cz799FNmzpxZ/j3NmjXjm2++qfNxIlES+gOHv/8eJkyAww5jdOF9fNb0AM7/xS2ced5drPzBTwB/a41UH/iogZ0q9YFD8lsz9urVi+HDh3P00UcDcPHFF5OXl1fe3VGd3r17c/rpp9OjRw/atGlD9+7dad68ecKvOXr0aIYNG8Ztt93GoEGD6nz8pZdeyoUXXkiXLl3o0qVLeX/7kUceSV5eHp07d+aQQw6hoKCg/HtGjBjBySefzMEHH8ycOXNqfJxIlBzcIpfSagIw8AOHv/8eHn0Ubr8dPvoI+vRhZP/f81zrI8Cs0kP9rLVe28kmy4/tZMMygLF161aaNm3Kd999x/HHH8+ECRMqDS5GkbaTlXSr789z1YkM4DXi7hzaPZiBzKrBfcwxcPPNMGAAhUWf+FarL9vJhkEqt2asjxEjRrBq1Sq2b9/OsGHDIh/eIunWkFllfh44nJTvv4eJE73gXr/eC+4JE2DAgPIWdzpqjVyAh0W8T1pEGqahB/4G2oirLrj//ncYOHCPrhJIfa2hGMRMZzeOVE//B5JuoR+QrGjHDnjwQejUCUaMgDZtYOZMePNNOPnkasM7HQIP8MaNG7N582YFSICcc2zevJnGjRsHXYpkkVQf+OuLeHAffjj85jdw0EEwYwa89VagwR0XeBdK+/bt2bBhA1E+8DgTNG7cmPbt2wddhmSRVMwq882OHfDYY3DbbVBSAr17w/33hyK0Kwo8wHNycujYsWPQZYhImoVmQLKi6oL7b3+D//qvUAV3XOABLiLZKyyzytixAx5/3AvuDz+E/PxQB3dc4H3gIiKB2bEDHn7YG5y86CI48ECYPh0WLYJTTgl1eIMCXESy0Y4d8Mgj0LnznsE9aFDogzuuzgA3s8ZmtsjM3jGzlWZ2c+x6RzNbaGbrzOxpM9sn9eWKiCShYnD/+tfQsiW8+GLkgjsukRb4f4B+zrkjgZ7AyWbWB/gTcK9z7ifAl8BFKatSRCQZO3d6S967dKkc3G+/DaeeGrngjqtzENN5E7S3xj7Nif1xQD/gV7HrE4GbgAf8L1FEpLKE91DZuROeeMIbnHz/fejVC154IdKhXVFCfeBm1sjMioDPgJeB94EtzrmdsYdsAKodSjazEWa22MwWa663iCQroUMddu70lrx37gwXXgjNm3vBvXgxnHZaRoQ3JBjgzrky51xPoD1wNNA50Rdwzk1wzuU75/Jbt27dsCpFRGJqPdShYnAPHw777w/PP59xwR1Xr3ngzrktZjYH+CnQwsz2jrXC2wPhO9NIRDJOdXulNNpVRp/50+GhS2DdOsjL84I7A0O7ojoD3MxaAzti4Z0LnIQ3gDkH+G/gKWAY8HwqCxURgcqHOjTaVcYZq+Zy+RtP0fHLjdCzJxQWwumnZ3RwxyXShdIWmGNmxcDbwMvOuenAH4CrzWwdcCDwUOrKFBHxjBrYiaaNYMiK2bz84KXc89K9bNsnl7fueQiWLoUzzsiK8IbEZqEUA3nVXP8Arz9cRCQ9du5k8IrZ/PyJsTT96ENWHdSRMefdTJ+Rv2Zwr+zbjE17oYhI+O3cCU89BbfeCu++S9Mjj4S/PMcRp5/OuL0SW1AeluMY/aQAF5HwKiuDJ58sD2569IBp07xukgSDGxp2fFsUKMBFJHzKyrwW9y231Du4q2tpN/T4trBTgEvWyMRfoTNOPLhvvRXWrvWCe+pUGDw4oRZ3TS3tquEdF8rj2+pBAS5ZIVN/hQ6bBr9JlpXB0097Le61a6F793oFd1xNLe1GZpRVc2xjqI5vawAFuGSFTP0VOgzioV26ZRuGt1ESJPgmWV1wT5kCQ4bUK7jjampRlzlHbk6jcB7flgTtBy5ZIVInoEfI/xQuZ+TTReULa6q2ccuXuFcVH5zs1g3OPRf22ccL7qIiOPPMBoU31NyibtcilzuHdqddi1yswudRf/NWC1yyQsXVe1WvS8MULitl0lsf7RHaVVV6kywrg2ee8Vrca9Z4AZ5Ei7uq2g5KDs3xbT5SgEtWCPUJ6BFQ08yOusIbYm+SZWXw7LNecK9e7QX3s8/C0KG+BHdcKA9KTiEFuGSFbPvB9lN9Z3ZU1KQR3Mca6H5ZSoO7okxsaddEAS5ZI0w/2FGa0ljfmR0Ae+0q47wNb3PtwmfY/4N3oWtXr+skif5t2ZMCXCTNojalsT4zO/Zyu7ijbA1nz3wUVq1ScKeY7qiIDwqXlVIwbjYdx7xEwbjZlU+HqaLWAwlCKJGZHXu5XZz/0UKKplzL2XeP9h7w9NNQXAxnnaXwThHdVZEkJXTEVwVRm9I4amAncnMaVbpWPrPjyLYs+NHnfPDSddz65K3s33hvL7iXL4df/ELBnWK6uyJJqm+LuqYWbVinNA7Oa7fnHOrBXRm87g1vqfsvfwm7dnlL4IuLFdxppD5wkSTVt0UdxSmN5QPAu3Z5S9yHXQErV0KXLl5w//d/Q6NGdT+R+EpvkyJJqm+LutoWbdhXBe7a5S24OfJIr4W9a5e3knL5cq8FrvAOhFrgIklqSIs6TFMaa7Vrl7eN6803w4oV3mnvTz7pDUwqtAOnFrhIkiLZoq5LxRb3WWd5J+JMnuyF+NlnK7xDQi1wER9EpkVdl1274LnnvBb38uVei3vyZK/bRKEdOnW2wM3sEDObY2arzGylmV0Zu36TmZWaWVHszympL1dEUiI+ONmzpzcg+f33MGmS1+I+5xyFd0gl0gLfCVzjnFtqZs2AJWb2cuxr9zrnxqeuPBFJqV27oLDQa3EXF0OnTl5wp2hgMkpbCERBnQHunNsIbIx9/I2ZrQZ0x0WirGpwH344PPFESvu3o7aFQBTUaxDTzDoAecDC2KXLzKzYzB42s5Y1fM8IM1tsZos3bdqUXLUikpx4H3evXt7+JNu3e8G9apV3sEIKu0qitoVAFCQc4GbWFJgKXOWc+xp4APgx0BOvhX53dd/nnJvgnMt3zuW3bt06+YpFpP4qBvfQofDdd/D4495inBQHd1zUthCIgoRmoZhZDl54T3LOTQNwzn1a4ev/AKanpEIRaTjn4Pnnva6SoiI47DAvuM8+G/ZObhJaffuz03kqUrb0tScyC8WAh4DVzrl7KlxvW+FhQ4AV/pcnIg3inNfH3auXd1zZt9/CY495XSXnnedLeNdnAy+ofVMsPzWktqhKpAulADgf6FdlyuCfzWy5mRUDfYGRqSxURBIQb3HHg3vr1t3Bff75SQd3XEP6s9O14Cmb+toTmYUyH7BqvjTD/3JEpEGcgxde8LpKli2Dn/wEJk6EX/3Kt9CuqKH92elY8JRNfe1aSi8SZfHgPuooGDwYvvnGC+7Vq+GCC1IS3hDuLXHDXJvfFOAiUVQxuM84A77+Gh59NOXBHZeu/uyGCHNtftNeKCJR4hxMnw433QRLl8KPf+wF97nnpjy0K4p3g4RxpkeYa/ObuRpOlU6F/Px8t3jx4rS9nkjGqBrcP/oR3HCDLzNKJPzMbIlzLr/qdXWhiIRZPLh794bTT4ctW+CRR2DNGhg+XOGd5RTgImFUMbhPOw2+/BIefnh3cOfkBF2hhIACXCRMnIOXXoKjj/aC+4svdgf3hRcquKUS/f4loZQtS6HLOQczZnh93IsXQ8eO8NBD3uKbDA7trPt/9pkCXEIn0W1Hw/7Dn1B9zsHMmV5wv/121gQ3aHtZP6gLRUInkaXQYd/vos764i3uY46BQYNg0yZ48EFYuxZ+/euMD2/IriXvqaIAl9BJZCl02H/4a6zvn2u8FnefPpWD+9134aKLsiK447JpyXuqqAtFQieRbUfD/sO/Rx3OceIHS7hqwWTY+C788Ifwj394qyb32SeYIgOWzu1lM5Va4BI6iSyFDvt+F+V1OMcJHyzhucev5dEpN9Fm21decL/7Llx8cULhXbislIJxs+k45iUKxs0OTTdRsrJpyXuqKMAldBLZdjTsP/yjBhzOSR8t47nHr2Xis2Np/e2X3HjKFSya9WbCwQ3h7+tPRrq2l81kWkovkRXKWSjOwb/+5c0qeestNrZow1+POYs3jj2VkYO61bu+gnGzq+1maNcilwVj+vlUtIRdTUvp1QcukZWOvaUTViW4OfRQmDCBtsOGcczKTcybtZaRTxdx16y19XqjCXtfvwRLAS4S06AWvXPw8stecL/5phfc//d/3nL3ffZJeq6zBvqkNuoDl4zTkEG/evc1x1vcBQUwcCCUlnrB/d57MGJEeR93stMdw97XL8FSgEtGaeigX8JBG29xH3usF9wbNsDf/75HcMcl2wWigT6pTZ1dKGZ2CPAY0AZwwATn3F/M7ADgaaADUAL8wjn3ZepKlUzm14BkbUFc2/PVGbTOwSuveF0lb7wBhxziBffw4bDvvjU+rx9dIKHq65dQSaQFvhO4xjl3BNAH+L2ZHQGMAV51zh0GvBr7XKTe/Jwq19AWb43zyps39oL7uONgwAD4+GN44AGvxX3JJbWGN6gLRFKrzgB3zm10zi2NffwNsBpoB5wBTIw9bCIwOEU1Sobzc1l8Qxf47BG0ztHv42JeeOY6OOkkWL9+d3D/9rd1BnecukAkleo1C8XMOgB5wEKgjXNuY+xL/8brYhGpNz+nyo0a2KnSrA9IrMVbfo7iP9fQ4Z23GL3wKY4sWQHt28P993sbTCUY2tU9twJbUiHhADezpsBU4Crn3NdmVv4155wzs2pXBJnZCGAEwKGHHppctZKR/Jwql8iBttX2t/c8mMFfrGHwjJtg/nxfglsk1RJaiWlmOcB0YJZz7p7YtbXAic65jWbWFpjrnKu1maOVmFKdqnOlwWs1p6KrYY/Xco4TS1cwflUhrZYthHbt4PrrvZ0BFdwSEg1eiWleU/shYHU8vGNeAIYB42J/P+9TrZJlEmk1+6W8v905fvpRMVfNn8wxG1by2f6t4G9/U3BLpCTShVIAnA8sN7Oi2LXr8YL7GTO7CFgP/CIlFUpWSFc/8SdfflcpuDc2PZAbTvotz/QYwNrfDUn564v4qc4Ad87NB6yGL/f3txzJVIFvPOUczJ3LtGf/SN6Hxfy76QHlwf2fvfehXUiXpgd+3yTUtBeKpFzgZx/OmeMtwJk3j86t23DrwEt5ottJ/Gdvb9VkWOdlB37fJPS0lF5SLrDjz+bOhRNPhH79YN06+N//JfejErrf+UdatWoe+nnZYT82ToKnFngIZdqvzWnfEnXuXK/F/dprbGvVhr+fdhl/P7wfrbY2Z9TqzZGZl62tZKUuCvCQycRfm9O2Jeprr3nBPXcutG1L8ehbOH+vI/nKeSsso3YvtZWs1EVdKCGTib82V7cfiAF9O7f25wVeew369vW6S9auhb/8Bd5/n0tbFpSHd1yU7qX2UZG6KMBDJhN/bR6c144zj2pXaSqTA6YuKU3ubMd587z+7RNPhDVryoObK66A3NzI30vtoyJ1URdKyGTqr81z1myi6prfRLZ5rda8eV5XyZw58IMfwH33eXtx51a+R5lwL6PSXy/BUAs8ZDL112ZfWsOvvw79+8MJJ8Dq1V5wf/ABXHnlHuENmXsvReLUAg+ZdC4rT6ekWsOvv+61uGfPhjZt4N57vb24qwntijL1XorEJbSZlV+0mVX2atCGVfPne8H96qtecI8Z43WV7LdfeooWCYkGb2Yl4od6tYarBvc993gtbgW3SCUKcEmbOgfkFizwgvuVVxTcIgnQIKYEb8EC79iyY4+F4mK4+25vcHLkSIW3SC0U4BKcN97wDgquGNwffghXX63gFkmAAlzSLx7cBQXwzjswfrzX4lZwi9SL+sAlfd580+vj/te/oHVrL7h/+1to0iToykQiSQEuqVc1uO+6Cy69VMEtkiQFuKTOm2/CzTfDrFkKbpEUUICL/956y2txz5oFrVrBn/8Mv/udglvEZwpw8c9bb3kt7n/+c3dwX3opNG0adGUiGanOADezh4FTgc+cc91i124CfgNsij3seufcjFQVKSG3cKHX4o4H95/+5LW4kwjuKJ5KFMWaJdoSaYE/Cvw/4LEq1+91zo33vSKJjoULvRb3zJm+BTdE81SiKNYs0VfnPHDn3DzgizTUIlGxaBGccgr06eN9PG6ctwBn9GhfukuieCpRFGuW6EtmIc9lZlZsZg+bWUvfKpLwWrQIBg2CY47ZHdwlJfCHP/jazx3Fk3SiWLNEX0MD/AHgx0BPYCNwd00PNLMRZrbYzBZv2rSppodJmFUM7oULUxbccTXtER7mk3SiWLNEX4MC3Dn3qXOuzDm3C/gHcHQtj53gnMt3zuW3bu3TIbaSHm+/Daeeuju477zT6ypJUXDHRfEknSjWLNHXoGmEZtbWObcx9ukQYIV/JUng3n7bG5x86SU44AAvuH//e2jWLC0vH8WTdKJYs0RfnSfymNmTwIlAK+BTYGzs8554h4uXAJdUCPQa6USekFu82Avu6dO94L72WrjssrQFd7pp2p9ERYNP5HHOnVPN5Yd8qUrCoWpw33FHRgc3aNqfZAZtJ5vNliyB00+H3r29LV5vv93r477uuowOb9C0P8kMWkqfjZYs8VrcL74ILVt6wX3ZZbD//kFXljaa9ieZQC3wbLJ0KZxxBuTnewcH33abNx3w+uuzKrxB0/4kMyjAs0E8uI86Cl5/fXdw//GPWRfccZr2J5lAXSiZbNkyr6vk+eehRQu49Va4/HJo3jzoygKnaX+SCRTgAUvJVLYagrvwg63c9cASBVbM4Lx2Wf3vl+hTgDeAX6Hr+1S2qsF9yy1wxRXQvLmmzYlkIPWB11M8CEu3bMOxOwgLl5XW+7l8m8pWVARDhkCvXvDaa15wl5TADTeUd5do2pxI5lELvJ5qC8L6tmSTnspWVOS1uAsLvaC++Wavxd2ihf+vFSJaQSniUQu8nvwMwgZPZSsqgqFDIS8P5szxgrukBG68sdrwTuq1QsbP34BEok4BXk9+BmG9p7K9887u4J492zvGrI7gru21DOjbOVo7RKorSGS3jA7wwmWlFIybTccxL1EwbrYvrTQ/5w8PzmvHnUO7065FLga0a5HLnUO779kd8M47cOaZ0LNn5eAeO7bO4K74Wmce1Q6rcM0BU5eURqr1mkldQSLJytg+8FTNuvB7/nCtU9mKi73ukWnTvAU3Y8fCVVclHNpVzVmziap7Tza0/z4oB7fIpbSasI5aV5CIHzI2wP0cbKwq5fOHi4u9mSRTp+4O7iuv9PYtSUImtF5HDexU6Y0ZtIJSslfGBngkw6pqcN94o9fiTjK44zKh9aoVlCK7ZWyARyqsli/3gnvKlJQEd1ymtF61glLEE/oAb+ic30iEVcXgbtbMW3hz1VXeoQp1aMh9UetVJLOEOsCTGYgMdVitWOEF97PP1ju4Ifn7Eop7ICJJq/NMTD/V90zMgnGzq+0GadcilwVj+vlZWnpUDe4rr4SRIxMO7riMuy8iUqsGn4kZpEgORFZn5crdwd20KfzP/zQouOMy5r6ISFLqXMhjZg+b2WdmtqLCtQPM7GUzey/2t7+jbTGRX/69ciX88pfQvTvMmOGdfPPhh972rg0Mb8iA+yIivkhkJeajwMlVro0BXnXOHQa8Gvvcd5E9NWXlSjj77MrBXVLinYRz4IFJP31k74uI+KrOLhTn3Dwz61Dl8hnAibGPJwJzgT/4WRiEfCCyOqtWeV0lzzwDTZp4p7tffbUvoV1R5O6LiKREQoOYsQCf7pzrFvt8i3OuRexjA76Mf17N944ARgAceuihR61fv96XwkNl1SqvW+Tpp73gvuKKlAS3iGSnmgYxk97MynnvADW+CzjnJjjn8p1z+a1bR2vnuzqtWgXnnAPdusH06TBmjNfHffvtCm8RSbmGzkL51MzaOuc2mllb4DM/iwq91au9FvdTT8F++3nBffXV0KpV0JWJSBZpaIC/AAwDxsX+ft63isKsanD/4Q9wzTWhDm6dXiOSueoMcDN7Em/AspWZbQDG4gX3M2Z2EbAe+EUqiwzcmjVecD/5ZGSCG1K3pa6IhEMis1DOqeFL/X2uJXyqBvfo0V5wR6QvP5Vb6opI8EK9EjMwa9fuDu7c3MgFd5xWbIpkNgV4RRWDu3FjuPZa708ag9vPPutIbakrIvWmAAcvuG+7DSZPDiy4wf8+60S21NUgp0h0ZfShxnV69104/3w44gjv3Mlrr/WWvP/pT4F0l/h94npdhybH3zBKt2zDsfsNI0qHHItks+xsgb/7rtfinjTJa3Ffc40X3gcdFGhZqeizrm3/bw1yikRbdgV4SIM7Lt191hrkFIm27OhCee89GDYMunTxji+7+mpvyfuf/xya8Ib07TJYuKyUgnGza9z/QIOcItGQ2S3w997zWtxPPAH77usF97XXQps2QVdWrXTsMlh1oLQqbUsrEh2ZGeDr1u0O7n328U6/GTUqtMFdUarPrKyu3zuunWahiERKZgV41eC+6qrIBHe61NS/baDzNEUiJjMCfN06bwvXxx/3gvvKK73g/sEPgq4sdLS4RyRzRHsQ8/334cILoXNnb4fAK6+EDz6Au+9WeNdAx7GJZI5otsDff99rcT/2GOTkeCfgjB6t0E6AjmMTyRzRCnAFty9SPVAqIukRjQCvGtyXX+4Fd9u2QVcmIhKYaAT47bd7OwQ2ILi1WZOIZKqETqX3S35+vlu8eHH9v/GTT8Cs3i3u6hat5OY0qrShk4hI2KXsVPq0OPjgBnWX+L27n4hImEQjwBtImzWJSCbL6ACvaXGKFq2ISCZIKsDNrMTMlptZkZk1oHM7tbRoRUQymR+zUPo65z734Xl8p0UrIpLJojGNMAlatCIimSrZPnAH/MvMlpjZiOoeYGYjzGyxmS3etGlTki8nIiJxyQb4sc65XsB/Ab83s+OrPsA5N8E5l++cy28dwEHBIiKZKqkAd86Vxv7+DHgOONqPokREpG4NDnAza2JmzeIfAwOAFX4VJiIitUtmELMN8JyZxZ9nsnPun75UJSIidWpwgDvnPgCO9LEWERGph4xeiSkikskU4CIiEaUAFxGJKAW4iEhEKcBFRCJKAS4iElEKcBGRiFKAi4hElAJcRCSiIrMfeOGyUh3MICJSQSQCvHBZKddNW15+wnzplm1cN205gEJcRLJWJLpQ7pq1tjy847btKOOuWWsDqkhEJHiRCPBPtmyr13URkWwQiQA/uEVuva6LiGSDSAT4qIGdyM1pVOlabk4jRg3sFFBFIiLBi8QgZnygUrNQRER2i0SAgxfiCmwRkd0i0YUiIiJ7UoCLiESUAlxEJKIU4CIiEaUAFxGJKHPOpe/FzDYB69P2guHVCvg86CJCRPejMt2PynQ/4IfOudZVL6Y1wMVjZoudc/lB1xEWuh+V6X5UpvtRM3WhiIhElAJcRCSiFODBmBB0ASGj+1GZ7kdluh81UB+4iEhEqQUuIhJRCnARkYhSgKeJmR1iZnPMbJWZrTSzK4OuKQzMrJGZLTOz6UHXEjQza2FmU8xsjZmtNrOfBl1TkMxsZOxnZYWZPWlmjYOuKWwU4OmzE7jGOXcE0Af4vZkdEXBNYXAlsDroIkLiL8A/nXOdgSPJ4vtiZu2AK4B851w3oBFwdrBVhY8CPE2ccxudc0tjH3+D98OZ1Rucm1l7YBDwYNC1BM3MmgPHAw8BOOe+d85tCbSo4O0N5JrZ3sB+wCcB1xM6CvAAmFkHIA9YGHApQbsPGA3sCriOMOgIbAIeiXUpPWhmTYIuKijOuVJgPPARsBH4yjn3r2CrCh8FeJqZWVNgKnCVc+7roOsJipmdCnzmnFsSdC0hsTfQC3jAOZcHfAuMCbak4JhZS+AMvDe2g4EmZnZesFWFjwI8jcwsBy+8JznnpgVdT8AKgNPNrAR4CuhnZk8EW1KgNgAbnHPx38qm4AV6tvo58KFzbpNzbgcwDfhZwDWFjgI8TczM8Po3Vzvn7gm6nqA5565zzrV3znXAG5ya7ZzL2haWc+7fwMdm1il2qT+wKsCSgvYR0MfM9ov97PQniwd1axKZQ40zQAFwPrDczIpi1653zs0IriQJmcuBSWa2D/ABcGHA9QTGObfQzKYAS/FmcC1DS+r3oKX0IiIRpS4UEZGIUoCLiESUAlxEJKIU4CIiEaUAFxGJKAW4iEhEKcBFRCLq/wMqlHCjfISp9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "N = X.shape[0]\n",
    "\n",
    "S_X2 = np.sum(X*X)\n",
    "S_X  = np.sum(X)\n",
    "S_XY = np.sum(X*Y)\n",
    "S_Y  = np.sum(Y)\n",
    "\n",
    "A1 = np.array([[S_X2, S_X], \n",
    "               [S_X, N]])\n",
    "B1 = np.array([S_XY, S_Y])\n",
    "\n",
    "# numpy.linalg模块包含线性代数的函数。\n",
    "# 使用这个模块，可以计算逆矩阵、求特征值、解线性方程组以及求解行列式等。\n",
    "coeff = np.linalg.inv(A1).dot(B1)\n",
    "\n",
    "print('a = %f, b = %f' % (coeff[0], coeff[1]))\n",
    "\n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = coeff[0] * x_min + coeff[1]\n",
    "y_max = coeff[0] * x_max + coeff[1]\n",
    "\n",
    "plt.scatter(X, Y, label='original data')\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 如何使用迭代的方法求出模型参数\n",
    "\n",
    "当数据比较多的时候，或者模型比较复杂，无法直接使用解析的方式求出模型参数。因此更为常用的方式是，通过迭代的方式逐步逼近模型的参数。\n",
    "\n",
    "### 2.1 梯度下降法\n",
    "在机器学习算法中，对于很多监督学习模型，需要对原始的模型构建损失函数，接下来便是通过优化算法对损失函数进行优化，以便寻找到最优的参数。在求解机器学习参数的优化算法中，使用较多的是基于梯度下降的优化算法(Gradient Descent, GD)。\n",
    "\n",
    "梯度下降法有很多优点，其中最主要的优点是，**在梯度下降法的求解过程中只需求解损失函数的一阶导数，计算的代价比较小，这使得梯度下降法能在很多大规模数据集上得到应用。**\n",
    "\n",
    "梯度下降法的含义是通过当前点的梯度方向寻找到新的迭代点。梯度下降法的基本思想可以类比为一个下山的过程。假设这样一个场景：\n",
    "* 一个人被困在山上，需要从山上下来，找到山的最低点，也就是山谷；\n",
    "* 但此时山上的浓雾很大，导致可视度很低。因此，下山的路径就无法全部确定，他必须利用自己周围的信息去找到下山的路径。\n",
    "* 以他当前的所处的位置为基准，寻找这个位置最陡峭的地方，然后朝着山的高度下降的地方走\n",
    "* 每走一段距离，都反复采用同一个方法，最后就能成功的抵达山谷。\n",
    "\n",
    "\n",
    "一般情况下，这座山最陡峭的地方是无法通过肉眼立马观察出来的，而是需要一个工具来测量；同时，这个人此时正好拥有测量出最陡峭方向的能力。所以，此人每走一段距离，都需要一段时间来测量所在位置最陡峭的方向，这是比较耗时的。那么为了在太阳下山之前到达山底，就要尽可能的减少测量方向的次数。这是一个两难的选择，如果测量的频繁，可以保证下山的方向是绝对正确的，但又非常耗时；如果测量的过少，又有偏离轨道的风险。所以需要找到一个合适的测量方向的频率，来确保下山的方向不错误，同时又不至于耗时太多！\n",
    "\n",
    "\n",
    "![gradient_descent](images/gradient_descent.png)\n",
    "\n",
    "如上图所示，得到了最优解。$x$,$y$表示的是$\\theta_0$和$\\theta_1$，$z$方向表示的是花费函数，很明显出发点不同，最后到达的收敛点可能不一样。当然如果是碗状的，那么收敛点就应该是一样的。\n",
    "\n",
    "对于最小二乘的损失函数\n",
    "$$\n",
    "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2\n",
    "$$\n",
    "\n",
    "更新的策略是：\n",
    "$$\n",
    "\\theta^1 = \\theta^0 - \\eta \\triangledown L(\\theta)\n",
    "$$\n",
    "其中$\\theta$代表了模型中的参数，例如$a$, $b$\n",
    "\n",
    "此公式的意义是：$L$是关于$\\theta$的一个函数，我们当前所处的位置为$\\theta_0$点，要从这个点走到L的最小值点，也就是山底。首先我们先确定前进的方向，也就是梯度的反向，然后走一段距离的步长，也就是$\\eta$，走完这个段步长，就到达了$\\theta_1$这个点！\n",
    "\n",
    "最终的更新方程是：\n",
    "\n",
    "$$\n",
    "a^1 = a^0 + 2 \\eta [ y - (ax+b)]*x \\\\\n",
    "b^1 = b^0 + 2 \\eta [ y - (ax+b)] \n",
    "$$\n",
    "\n",
    "下面就这个公式的几个常见的疑问：\n",
    "\n",
    "* **$\\eta$是什么含义？**\n",
    "$\\eta$在梯度下降算法中被称作为学习率或者步长，意味着我们可以通过$\\eta$来控制每一步走的距离，以保证不要步子跨的太大，错过了最低点。同时也要保证不要走的太慢，导致太阳下山了，还没有走到山下。所以$\\eta$的选择在梯度下降法中往往是很重要的。\n",
    "![gd_stepsize](images/gd_stepsize.png)\n",
    "\n",
    "* **为什么要梯度要乘以一个负号？**\n",
    "梯度前加一个负号，就意味着朝着梯度相反的方向前进！梯度的方向实际就是函数在此点上升最快的方向，而我们需要朝着下降最快的方向走，自然就是负的梯度的方向，所以此处需要加上负号。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 示例代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch    0: loss = 821.866782, a = 3.383589, b = 1.395455\n",
      "epoch  100: loss = 694.178445, a = 3.058880, b = 4.323617\n",
      "epoch  200: loss = 707.108900, a = 2.868004, b = 4.733422\n",
      "epoch  300: loss = 694.373048, a = 2.908494, b = 4.808539\n",
      "epoch  400: loss = 693.019312, a = 2.916707, b = 4.817341\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAm+0lEQVR4nO3dd5hU1f3H8fdXJLICCgoaAXWx0dGFBYmICqgYsSCJJsaCiQZb7EFRo4CiYETEHomNxK7g2iAkFMWK0qT7s0HCCtIERBalnN8fZ5Zd1i0zO+XeO/t5PQ8Pu3fvzHy5PPuZM+eeYs45REQkenYJugAREakeBbiISEQpwEVEIkoBLiISUQpwEZGI2jWTL9aoUSOXm5ubyZcUEYm8mTNnrnbONS57PKMBnpuby4wZMzL5kiIikWdmS8s7ri4UEZGIUoCLiESUAlxEJKIy2gdeni1btrBs2TI2b94cdCk1Wp06dWjWrBm1a9cOuhQRiVPgAb5s2TLq169Pbm4uZhZ0OTWSc441a9awbNkymjdvHnQ5IhKnwAN88+bNCu+AmRl77703q1atCroUkVAomF3I3RM/5et1RTRpkMOAXi3ok9c06LJ+IvAABxTeIaD/AxGvYHYhN46bR9GWbQAUrivixnHzAEIX4rqJKSJSyt0TP90R3sWKtmzj7omfBlRRxRTgKZabm8vq1auTPkdEgvH1uqKEjgdJAS4iUkqTBjkJHQ+SAhxYsmQJLVu25IILLuCwww7jnHPOYdKkSXTt2pVDDz2Ujz76iLVr19KnTx/at29Ply5dmDt3LgBr1qzhxBNPpE2bNlx00UWU3uHo6aefpnPnzhxxxBFcfPHFbNu2raISRCQkBvRqQU7tWjsdy6ldiwG9WlTr+QpmF9J1+BSaD3yTrsOnUDC7MBVlAiG5ibnD1VfDnDmpfc4jjoBRo6o87fPPP+ell17iiSeeoFOnTjz77LO8++67vPbaa9x5553sv//+5OXlUVBQwJQpUzj//POZM2cOQ4YM4eijj+bWW2/lzTff5PHHHwdg0aJFvPDCC7z33nvUrl2byy67jGeeeYbzzz8/tf8+EUmp4huVqRiFku4bolUGuJnVAaYBu8XOf9k5N8jMngKOBdbHTr3AOTcn6YoC0rx5c9q1awdAmzZt6NmzJ2ZGu3btWLJkCUuXLmXs2LEA9OjRgzVr1rBhwwamTZvGuHHjAOjduzcNGzYEYPLkycycOZNOnToBUFRUxD777BPAv0xEEtUnr2lKArayG6IZCXDgB6CHc26jmdUG3jWzCbGfDXDOvZx0FcXiaCmny2677bbj61122WXH97vssgtbt25NeIaic45+/foxbNiwlNYpItGR7huiVfaBO29j7NvasT81biv7bt268cwzzwDw1ltv0ahRI/bYYw+OOeYYnn32WQAmTJjAt99+C0DPnj15+eWXWblyJQBr165l6dJyV4QUkSyV7huicd3ENLNaZjYHWAn8xzk3PfajO8xsrpnda2a7VfDY/mY2w8xmRHmm3+DBg5k5cybt27dn4MCBjBkzBoBBgwYxbdo02rRpw7hx4zjggAMAaN26NUOHDuXEE0+kffv2nHDCCSxfvjzIf4KIZFiqb4iWZaVHTVR5slkD4BXgCmANsAL4GTAa+MI5d1tlj8/Pz3dlN3RYtGgRrVq1SqxqSQv9X4ikXiqm5ZvZTOdcftnjCY1Ccc6tM7OpwEnOuRGxwz+Y2ZPAnxOqSESkBkjVDdHyVNmFYmaNYy1vzCwHOAFYbGb7xY4Z0AeYn5YKRUSkXPG0wPcDxphZLXzgv+ice8PMpphZY8CAOcAl6StTRETKqjLAnXNzgbxyjvdIS0UiIhIXTaUXEYkoBbiISEQpwBNw8skns27dukrPufXWW5k0aVK1nv+tt97ilFNOqfK84447jrLDMcsaNWoUmzZtqlYdIhINCvA4OOfYvn0748ePp0GDBpWee9ttt3H88cdnprBKKMBFsl/kAjwdSzOOHDmStm3b0rZtW0bF1mNZsmQJLVq04Pzzz6dt27b873//22kjhttvv50WLVpw9NFHc/bZZzNihB8Wf8EFF/Dyy355mNzcXAYNGkSHDh1o164dixcvBuCjjz7iF7/4BXl5eRx11FF8+mnlO30UFRXx29/+llatWnHGGWdQVFSyjsKll15Kfn4+bdq0YdCgQQDcf//9fP3113Tv3p3u3btXeJ6IRFu4lpOtQjqWZpw5cyZPPvkk06dPxznHkUceybHHHkvDhg357LPPGDNmDF26dNnpMR9//DFjx47lk08+YcuWLXTo0IGOHTuW+/yNGjVi1qxZPPzww4wYMYLHHnuMli1b8s4777DrrrsyadIkbrrpph0rHZbnkUceYffdd2fRokXMnTuXDh067PjZHXfcwV577cW2bdvo2bMnc+fO5corr2TkyJFMnTqVRo0aVXhe+/btq3XNRIIUlQ2HIf21RqoFno696t59913OOOMM6tatS7169ejbty/vvPMOAAceeOBPwhvgvffe4/TTT6dOnTrUr1+fU089tcLn79u3LwAdO3ZkyZIlAKxfv54zzzyTtm3bcs0117BgwYJKa5w2bRrnnnsuAO3bt98peF988UU6dOhAXl4eCxYsYOHCheU+R7zniYRZcSOucF0RjpJGXCo3SUiV0rU2XbeCwm83pbzWSAV4pveqq1u3btLPUbwsba1atdi6dSsAt9xyC927d2f+/Pm8/vrrbN68uVrP/dVXXzFixAgmT57M3Llz6d27d7nPFe95ImEXpQ2H7574KU1WLOHe10fw9uj+HPPVrJTXGqkAT8fSjN26daOgoIBNmzbx/fff88orr9CtW7dKH9O1a9cdwbtx40beeOONhF5z/fr1NG3qP0Y99dRTVZ5fesna+fPn79jObcOGDdStW5c999yTb775hgkTJux4TP369fnuu++qPE8kSiKz4fDixQz4523857HL6PXZB/y9Ux8W7HswkNpaI9UHPqBXi536wCH5pRk7dOjABRdcQOfOnQG46KKLyMvL29HdUZ5OnTpx2mmn0b59e/bdd1/atWvHnnvuGfdrXn/99fTr14+hQ4fSu3fvKs+/9NJL+f3vf0+rVq1o1arVjv72ww8/nLy8PFq2bMn+++9P165ddzymf//+nHTSSTRp0oSpU6dWeJ5IlDRpkENhOQEYmg2HFy2C22+H55+n1667MfrIvvy90xmsqdtgxymprDWh5WSTlYrlZMNyA2Pjxo3Uq1ePTZs2ccwxxzB69Oidbi5GkZaTlUxL9Pe57EAG8I24YX3bBXsjc+FCH9wvvAC77w6XX874Xudw3dSvU1JrSpaTDYN0Ls2YiP79+7Nw4UI2b95Mv379Ih/eIplWnVFlqdxwOCXKBvf118N110HjxpwM/Nhw77TWGrkWuKSP/i8kk7oOn1Jud0jTBjm8NzDka+UtWOCD+8UXfXBfcYUP7tiw3VQLdQvcOYdfVlyCksk3chGI0A3J0hYsgNtug5degrp1YeBAuPbatAV3VQIfhVKnTh3WrFmjAAmQc441a9ZQp06doEuRGiTdG/6m1Pz58JvfQLt2MH68D+6vvoI77wwsvCEELfBmzZqxbNkyorzhcTaoU6cOzZo1C7oMqUHSMaos5ebPL2lx16sHN97oW9x77x10ZUAIArx27do0b9486DJEJMNCd0OytHnzfHC//LIP7ptvhmuuCU1wFws8wEWk5grLqLId5s71wT12LNSvH9rgLqYAFxEpG9x/+YsP7r32CrqySinARaTm+uQTH9zjxsEee8Att8DVV4c+uItVOQrFzOqY2Udm9omZLTCzIbHjzc1supl9bmYvmNnP0l+uiEgKzJkDffvCEUfApElw662wZIkP84iEN8Q3jPAHoIdz7nDgCOAkM+sC3AXc65w7BPgWuDBtVYqIpEJxcOflweTJJcE9ZAg0bBh0dQmrsgvF+QHaG2Pf1o79cUAP4Hex42OAwcAjqS9RRGRnCa+JNHu2b10XFMCee8KgQXDVVZEM7dLi6gM3s1rATOAQ4CHgC2Cdc25r7JRlQLlXz8z6A/0BDjjggGTrFZEaLqE1VGbP9q3rV1/1wT14sA/uKva2jYq4ZmI657Y5544AmgGdgZbxvoBzbrRzLt85l9+4cePqVSkiEhPXpg6zZsHpp0OHDvDWWz64lyzxLe8sCW9IcBSKc26dmU0FfgE0MLNdY63wZkD49jQSkaxT6Roqs2b5Fvdrr/mgHjIErrwyq0K7tHhGoTQ2swaxr3OAE4BFwFTg17HT+gGvpqlGEZEdylsrpe2Kz/nna3dAx44wbZrv716yxN+kzNLwhvha4PsBY2L94LsALzrn3jCzhcDzZjYUmA08nsY6RUSAnddQabf8M65671mO/+Jjfqy/p1/i9YorfH93DRDPKJS5QF45x7/E94eLiGRMn7ymNFj4CbsOHcrRiz9kQ059Fl42gNZ33lxjgruYZmKKSHR8/DEMGcJxb77phwAOHcoeV1xB6z32qPKhYdmOMZUU4CISfh995G9Ijh/vZ0recQf86U9++nscqrN9WxQowEUkvKZP98E9YYIP7jvv9MFdv36FDymvpV3Z0EMFuEgEZONH6KxVjeCGilvaZcO7WKi3b4uDAlxqhGz9CB02Sb9JfvihD+5//cuvwT1sGFx+eZXBXayilnYtM7aVs21jKLdvS4ACXGqEbP0IHQbFoV24rgjDL5QECb5JfvCBD+6JE31wDx8Ol10Wd3AXq6hFvc05cmrXCvf2bdUQ+KbGIpkQyR3QI+AvBfO45oU5FMauY9k27k+muJf1wQdw0klw1FEwc6YP7iVL4IYbEg5vqLhF3bRBDsP6tqNpgxys1PdRf/NWC1xqhCYNcnaETNnjUj0Fswt55sP//iS0yyr3TfL9932L+9//9ru633WXb3HXq5dUTZVtlBy67dtSQAEuNUIkdkAPsYpGdlQV3lDmTfL99/3CUv/5jw/uv/4VLr006eAuFuqNktPAXDkd++mSn5/vZsyYkbHXEylNo1Cqp+wNYOAn/ckVyaldy3dVbFrig3vSJGjcGK6/3gd33brpKzyLmNlM51x+2eNqgUuNEaaP0FF6M0l0ZEexpg1yGLbPeo65/oKS4L77bgV3CinARTIsakMaExnZYcA5XQ5g6N7rfB/35Mmwzz4wYgRccomCO8U0CkUkBQpmF9J1+BSaD3yTrsOnUDC74uXx49qQIEQSGdnxj0N/YOgDV8Mxx8C8eXDPPfDVV3DddQrvNFALXCRJibaoozakMa6RHdOm+Rb3lCmw774+uC+5BHbfPcDKs59a4CJJSrRFXVGLNqxDGvvkNa14DPXbb0OPHnDssbBgAYwcCV9+Cddeq/DOALXARZKUaIs6ikMaf3ID+O23ofu5fr/Jn/8c7r0X+vdXaGeYWuAiSUq0RV1pizbs3noLjjvO/1m8GEaN8i3uq69WeAdALXCRJFWnRR2mIY1xKd7Z/e23Yb/9fHD37w854ez2qSkU4CJJytrZf86VBPe0aT6477sP/vhHBXdIKMBFUiByLerKOAdTp/rgfucdH9z33w8XXaTgDpkq+8DNbH8zm2pmC81sgZldFTs+2MwKzWxO7M/J6S9XRNLGOT8M8NhjoWdP+OILH9xfful3eld4h048LfCtwHXOuVlmVh+YaWb/if3sXufciPSVJyJpVxzcgwfDu+9CkybwwAO+xV2nTkpfKkpLCERBlQHunFsOLI99/Z2ZLQJ0xUWizjk/1X3wYHjvPWjaFB58EC68MOXBDdFbQiAKEhpGaGa5QB4wPXboT2Y218yeMLOGFTymv5nNMLMZq1atSq5aEUmec345127d4IQT/AYKDz0En3/uty9LQ3hD9JYQiIK4A9zM6gFjgaudcxuAR4CDgSPwLfR7ynucc260cy7fOZffuHHj5CsWkeopDu6jj4YTT4SlS31wf/GF30whTcFdLGpLCERBXKNQzKw2Pryfcc6NA3DOfVPq538H3khLhSKSnOLgHjzYb2HWrBk8/DD84Q+w227VftpE+7MzuStSTelrj2cUigGPA4uccyNLHd+v1GlnAPNTX56IVJtzfpPgo46CXr1g2TJ45BHfVXLppUmH943j5lG4rghHSX92ZaswDujVgpzatXY6lo4lBKpTW1TF04XSFTgP6FFmyOBfzWyemc0FugPXpLNQEYlT6eA+6SQoLPTB/dlnfoXAJIK7WHX6szO1hEBN6muPZxTKu/h12ssan/pyRKTaioN78GCYPh323x/+9je44IKUhHZp1e3PzsSEp5rU167FrESizjmYMAG6dIFf/hKWL4dHH/VdJRdfnPLwhnAviRvm2lJNAS4SVc7B+PE+uE8+Gb75BkaP9l0l/fvDz36WtpfOVH92dYS5tlTTWigiUVPc4h48GD7+GA480Ad3v35pDe3SwryAV5hrSzVzlewqnWr5+fluxowZGXs9kaxS3OIePBhmzIDcXLj5Zjj//IwFtwTDzGY65/LLHlcXikjYOQdvvAGdO8Mpp8Dq1fDYY/B//+fXK1F411gKcJGwKg7uTp3g1FNhzZqS4L7wQqhdO+gKJWAKcJGwcQ5ef70kuNeuhccfh08/VXDLTnQTU0KppkyF3klxcA8ZArNmwUEHwRNPwLnnZm1o18j/5xRSgEvoxLvsaNh/+eOuzzl47TUf3LNn++B+8kk455ysDW7Q8rKpoC4UCZ14pkKHfb2LuOpzDl59FTp2hD59YMMGH9yLF/vZk1kc3lCzpryniwJcQieeqdBh/+WvtD7noKAAOnQoCe6nnqoxwV2sJk15TxcFuIROPFOhw/7LX14d5rbT9qPJkJcHZ5wBGzfCmDE+uPv1g11rVo9mTZryni4KcAmdeKZCh/2Xv3Qd5rbT6//e582nruLRV+6ETZvgH/+ARYv8JJwqgrtgdiFdh0+h+cA36Tp8Smi6iZJVk6a8p4sCXEInnmVHw/7LP6BXC3bf1ej16fuMf/JKHn3lTnbf+iMzbxsFCxfCeefF1eIOe19/MjK1vGw201R6iazQjkLZvh1eeYX1A//Cnp8v5su9mvLP48/jiD9fwumdDkzoqboOn1LuLjZNG+Tw3sAeqapYQq6iqfQ1q9NNskom1pZOyPbtMG4c3HYbzJvHnocdBk8/zdwW3fj3pM95aux8/jr5i4TeaMLe1y/BUheKSEy1+5q3b4eXX4bDD4czz4Qff4Snn4aFCylofRw3vrqw2l0gYe/rl2ApwCXrVCeIq9XXvH07vPRSSXBv2QLPPAMLFvhJOLVqJT3cMex9/RIsBbhklere9EsoaLdvhxdfhPbt4ayzYOtWePZZH9y/+x3UKgncZLtAdKNPKlNlH7iZ7Q/8A9gXcMBo59x9ZrYX8AKQCywBznLOfZu+UiWbpeqGZGVBXNnzxRW027b5rpLbb/dh3aoVPPecb33XqlXu45s0yCn3JmQiXSCh6+uX0IinBb4VuM451xroAlxuZq2BgcBk59yhwOTY9yIJS+VQueq2eCvta962DV54wbe4f/tb3wJ/7jmYN89/X0F4g7pAJL2qDHDn3HLn3KzY198Bi4CmwOnAmNhpY4A+aapRslwqp8VX96ZfeUFbtxaMYjG0a+eDGuD55+MK7mLqApF0SmgYoZnlAnnAdGBf59zy2I9W4LtYRBKWyqFyA3q12GmFO4ivxVt6H8UVazdy7n8/4rqPXmSPrz6D1q19C/zXv4ZdEr9tpC4QSZe4A9zM6gFjgaudcxvMbMfPnHPOzMqdEWRm/YH+AAcccEBy1UpWSkU/cbF4NrStqL+9T/uf02fxNBhzm1+fpE2bpIJbJN3imolpZrWBN4CJzrmRsWOfAsc555ab2X7AW865Sps5mokp5Sm7LjT4VnM6uhrKe626tWBM3a/I/+dDJcE9aBD86lcKbgmFas/ENN/UfhxYVBzeMa8B/YDhsb9fTVGtUsPE02pOldL97bts38api6Zx5fsvcPDaZdC2rR/X3bevglsiIZ4ulK7AecA8M5sTO3YTPrhfNLMLgaXAWWmpUGqETPUTf72uiFqx4L4iFtyLGudyaZ8beWTsUAW3REqVAe6cexewCn7cM7XlSLYKxcJTW7dywVfvcu6kf3Lw2kIWNc7lkj43MvGwX9CkYd1QhncorpuElhazkrQLfO/DrVv9uO3bb2fQZ5+xeJ/mXNznJv59WBec7RLacdmBXzcJvfA1OSTrBLb92datfuOE1q39xgk5OTBuHIvHv838I3uC7RLqcdlh3zZOgqcWeAhl28fmjC+JWrw2ye23w+ef89l+BzPijJtYkN+dP+e28v3tHfdPz2unkJaSlaoowEMmGz82p3Kcd6W2bvWrAQ4dCp9/zroWbfjLmbfwZvNOONsFNvwQqWuZsesmkaUulJDJxo/N5U1TN6B7y8apeYGtW/2u7i1b+l3d69WDggJ697uPNw460od3TJSupdZRkaoowEMmGz8298lryq86Nt1pKJMDxs4sTG5vxy1b4MknfXD//vewxx7w6qswaxacfjpfr99c7sOici21jopURV0oIZOtH5unLl5F2Tm/8SzzWq4tW/yON0OHwpdfQocOPrhPPRVKLfGQDddS66hIZdQCD5ls/dickk8WW7bAE09Aixbwhz9Agwbw2mswYwacdtpO4Q3Zey1FiinAQyZbPzYntbfjli3w+OM+uC+8EPbaC15/3Qd3mVZ3adl6LUWKxbWYVapoMauaq1oLVm3Z4sdxDx0KS5ZAfr5fZKp37wpDWyQbVXsxK5FUSGjBqh9/9MF9xx0lwf3gg3DyyQpukVIU4JIxVd6Q+/FHGDPGB/fSpdCpEzz0EPzylwpukXKoD1yC9+OPMHo0HHYY9O8P++4L48fD9OlqdYtUQgEuwSkO7kMPhYsvhp//3Af3hx+q1S0SBwW4ZN6PP8Kjj5YE9377wYQJ8MEHCm6RBCjAJXN++AH+9jcf3JdcAk2awL/+5YP7pJMU3CIJUoBL+pUO7ksv9cE9cSK8/z706qXgFqkmBbikzw8/wCOPwCGH+OBu1qwkuE88UcEtkiQNI5TU++EHP3Ny2DBYtgyOOspPgT/+eIW2SAopwCV1Nm/2wT18eElwP/kk9Oyp4BZJgyoD3MyeAE4BVjrn2saODQb+CKyKnXaTc258uoqUkCsO7mHDoLAQunZNOrijuCtRFGuWaIunBf4U8CDwjzLH73XOjUh5RRIdmzfDY4/54P76azj6aD+TskePpFrcUdyVKIo1S/RVeRPTOTcNWJuBWiQqNm+GBx6Agw+GK67wf0+eDNOmpaS7JIq7EkWxZom+ZEah/MnM5prZE2bWMGUVSXiVDu4rr/R/T5kCb7+ddKu7tCjuShTFmiX6qhvgjwAHA0cAy4F7KjrRzPqb2Qwzm7Fq1aqKTpMwKyqC+++Hgw7ywX3IISXB3b17ym9QJrV2eECiWLNEX7UC3Dn3jXNum3NuO/B3oHMl5452zuU75/IbN07RJraSGUVFcN99PrivusovNjV1atqCu1gUd9KJYs0SfdUaRmhm+znnlse+PQOYn7qSJHBFRX6tkrvughUr4Ljj4Lnn/N8ZkNDa4SERxZol+qrckcfMngOOAxoB3wCDYt8fgd9cfAlwcalAr5B25Am58oJ70KCMBXemadifREW1d+Rxzp1dzuHHU1KVhMOmTSXB/c03vnvk+efh2GODrixtNOxPsoHWQqnJNm2CkSN9H/e110KbNr5/e8qUrA5v0LA/yQ6aSl8TbdrkVwe86y5YudKP3X7pJejWLejKMkbD/iQbqAVek3z/PdxzDzRvDtddB+3a+ck3kybVqPAGDfuT7KAArwm+/x5GjPDB/ec/Q/v28M47NTK4i2nYn2QDdaFks++/h4cfhrvvhlWr4IQT/KiSrl2DrixwGvYn2aDKYYSppGGEP5WWoWwVBHfB7rkKLJEIqvYwQvmpVIVuyoeybdxYEtyrV/tdbwYNgqOO0rA5kSykPvAEFQdh4boiHCVBWDC7MOHnStlQto0b/YiS5s3hhhugY0e/bdnEiX5ThVS+loiEhlrgCaosCBNtySY9lG3jRnjoId/iXrPG7+w+aBB06ZL61woRzaAU8dQCT1Aqg7DaQ9m++85vW5abCwMHQqdO8MEHMGFCueGd1GuFTCo/AYlEnQI8QakMwoSHsn33nd/9JjcXbrwROneGDz+sNLgrey0DureM1gqR6goSKZHVAV4wu5Cuw6fQfOCbdB0+JSWttFSOH+6T15RhfdvRtEEOBjRtkMOwvu1+2h2wYQPceacP7ptu8mE9fTqMHw9HHhn3a/2qY1NKLwDrgLEzCyPVes2mriCRZGVtH3i6Rl2kevxwn7ymFT92wwZ48EE/e3LtWjj5ZN/H3bnC5dcrNXXxKsoOGq1u/31QmjTIobCcsI5aV5BIKmRtgKfyZmNZlYZuKmzY4Lcuu+ce+PZb6N3bB3enTkk9bTa0Xgf0arHTGzNoBqXUXFkb4JEMqw0b/NZlI0f64D7lFLj11qSDu1g2tF41g1KkRNYGeKTCav163+IuDu5TT/XBnf+TiVdJyZbWa9o/AYlEROgDvLpjfiMRVuvXl7S4163zwT1okJ+IU4XqXBe1XkWyS6gDPJkbkaEOq/Xr/WbB997rg/u003yLO47ghuSvSyiugYgkLdQBnuyNyNCF1bp1PrhHjfJfn366D+4OHRJ6mnTeoBWR6Ah1gEfyRmR5ioP73nt967uawV0sa66LiCSlyok8ZvaEma00s/mlju1lZv8xs89ifzdMR3GRn/69bh0MHuwn4Awe7DcLnjULCgqqHd6QBddFRFIinpmYTwEnlTk2EJjsnDsUmBz7PuUiu2vKt9/6m5G5uTBkCPToAbNnwyuvQF5e0k8f2esiIilVZReKc26ameWWOXw6cFzs6zHAW8ANqSwMQn4jsjzffuv7t0eN8mO6+/b1XSWHH57Sl4ncdRGRtIhrR55YgL/hnGsb+36dc65B7GsDvi3+vpzH9gf6AxxwwAEdly5dmpLCQ2XtWh/a992X1uAWkZqpoh15kl7Myvl3gArfBZxzo51z+c65/MaNo7XyXZXWroVbbvFdJbff7rcu++QTGDtW4S0iaVfdUSjfmNl+zrnlZrYfsDKVRYXe2rV+8s399/slXn/9ax/k7dsHXZmI1CDVDfDXgH7A8Njfr6asojBbs8YPBSwO7jPP9MHdrl3QlVVIu9eIZK8qA9zMnsPfsGxkZsuAQfjgftHMLgSWAmels8jArVlT0uLeuDESwQ3pW1JXRMIhnlEoZ1fwo54priV8Vq/2wf3AA/D99yXB3bZt0JXFRTM2RbJbqGdiBqZscJ91lg/uNm2CriwhmrEpkt0U4KWtXu03UXjgAdi0KZDgTmWfdaSW1BWRhCnAAVat8sH94IM+uH/zGx/crVtntIxU91nHs6SubnKKRFdWb2pcpVWr4IYboHlz+Otf/bKu8+fDc89lPLwh9TuuV7VpcvEbRuG6IhwlbxhR2uRYpCarmS3wVatgxAjf4i4qgrPPhr/8BVq1CrSsdPRZV7akrm5yikRbzQrwlSt9cD/0UKiCu1im+6x1k1Mk2mpGF8rKlTBggO8queceOOMMWLgQnnkmNOENmVtlsGB2IV2HT6lw/QPd5BSJhuxuga9cCXffDQ8/DJs3w+9+51vcLcK57GomVhkse6O0LC1LKxId2Rng33xTEtw//BD64C4t3dvAldfvXaypRqGIREp2BfiKFT64H3nEB/c55/jgPuywoCsLjYr6tw14b2CPzBYjIknJjgBfscIPA/zb33xwn3su3Hyzgrscmtwjkj2ifRNzxQq49lp/c/K++/zMycWLYcwYhXcFtB2bSPaIZgt8+fKSFveWLSUt7kMPDbqy0NN2bCLZI1oBvnw53HUXPPqoD+7zzvPBfcghQVcWKem+USoimRGNAC8b3Oef74P74IODrkxEJDDRCPCBA/2km2oEtxZrEpFsFdeu9KmSn5/vZsyYkfgDly6FbdvgoIMSelh5k1ZyatfaaUEnEZGwS9uu9Blx4IEJhzekfnU/EZEwiUaAV5MWaxKRbJbVAV7R5BRNWhGRbJBUgJvZEjObZ2ZzzKwandvppUkrIpLNUjEKpbtzbnUKniflNGlFRLJZNIYRJkGTVkQkWyXbB+6Af5vZTDPrX94JZtbfzGaY2YxVq1Yl+XIiIlIs2QA/2jnXAfglcLmZHVP2BOfcaOdcvnMuv3Hjxkm+nIiIFEsqwJ1zhbG/VwKvAJ1TUZSIiFSt2gFuZnXNrH7x18CJwPxUFSYiIpVL5ibmvsArZlb8PM865/6VkqpERKRK1Q5w59yXwOEprEVERBKQ1TMxRUSymQJcRCSiFOAiIhGlABcRiSgFuIhIRCnARUQiSgEuIhJRCnARkYhSgIuIRFRk1gMvmF2ojRlEREqJRIAXzC7kxnHzduwwX7iuiBvHzQNQiItIjRWJLpS7J366I7yLFW3Zxt0TPw2oIhGR4EUiwL9eV5TQcRGRmiASAd6kQU5Cx0VEaoJIBPiAXi3IqV1rp2M5tWsxoFeLgCoSEQleJG5iFt+o1CgUEZESkQhw8CGuwBYRKRGJLhQREfkpBbiISEQpwEVEIkoBLiISUQpwEZGIMudc5l7MbBWwNGMvGF6NgNVBFxEiuh470/XYma4HHOica1z2YEYDXDwzm+Gcyw+6jrDQ9diZrsfOdD0qpi4UEZGIUoCLiESUAjwYo4MuIGR0PXam67EzXY8KqA9cRCSi1AIXEYkoBbiISEQpwDPEzPY3s6lmttDMFpjZVUHXFAZmVsvMZpvZG0HXEjQza2BmL5vZYjNbZGa/CLqmIJnZNbHflflm9pyZ1Qm6prBRgGfOVuA651xroAtwuZm1DrimMLgKWBR0ESFxH/Av51xL4HBq8HUxs6bAlUC+c64tUAv4bbBVhY8CPEOcc8udc7NiX3+H/+Ws0Qucm1kzoDfwWNC1BM3M9gSOAR4HcM796JxbF2hRwdsVyDGzXYHdga8Drid0FOABMLNcIA+YHnApQRsFXA9sD7iOMGgOrAKejHUpPWZmdYMuKijOuUJgBPBfYDmw3jn372CrCh8FeIaZWT1gLHC1c25D0PUExcxOAVY652YGXUtI7Ap0AB5xzuUB3wMDgy0pOGbWEDgd/8bWBKhrZucGW1X4KMAzyMxq48P7GefcuKDrCVhX4DQzWwI8D/Qws6eDLSlQy4BlzrniT2Uv4wO9pjoe+Mo5t8o5twUYBxwVcE2howDPEDMzfP/mIufcyKDrCZpz7kbnXDPnXC7+5tQU51yNbWE551YA/zOzFrFDPYGFAZYUtP8CXcxs99jvTk9q8E3dikRmU+Ms0BU4D5hnZnNix25yzo0PriQJmSuAZ8zsZ8CXwO8DricwzrnpZvYyMAs/gms2mlL/E5pKLyISUepCERGJKAW4iEhEKcBFRCJKAS4iElEKcBGRiFKAi4hElAJcRCSi/h9DXZEW7ONw8AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "n_epoch = 500          # epoch size\n",
    "a, b = 1, 1            # initial parameters\n",
    "epsilon = 0.001         # learning rate\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    data_idx = list(range(N))\n",
    "    random.shuffle(data_idx)\n",
    "    \n",
    "    for j in data_idx:\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (Y[j]-a*X[j]-b)**2\n",
    "    \n",
    "    if i % 100 == 0:\n",
    "        print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n",
    "    \n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n",
    "\n",
    "plt.scatter(X, Y, label='original data')\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 如何可视化迭代过程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "/* global mpl */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function () {\n",
       "    if (typeof WebSocket !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof MozWebSocket !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert(\n",
       "            'Your browser does not have WebSocket support. ' +\n",
       "                'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "                'Firefox 4 and 5 are also supported but you ' +\n",
       "                'have to enable WebSockets in about:config.'\n",
       "        );\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = this.ws.binaryType !== undefined;\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById('mpl-warnings');\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent =\n",
       "                'This browser does not support binary websocket messages. ' +\n",
       "                'Performance may be slow.';\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = document.createElement('div');\n",
       "    this.root.setAttribute('style', 'display: inline-block');\n",
       "    this._root_extra_style(this.root);\n",
       "\n",
       "    parent_element.appendChild(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen = function () {\n",
       "        fig.send_message('supports_binary', { value: fig.supports_binary });\n",
       "        fig.send_message('send_image_mode', {});\n",
       "        if (fig.ratio !== 1) {\n",
       "            fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n",
       "        }\n",
       "        fig.send_message('refresh', {});\n",
       "    };\n",
       "\n",
       "    this.imageObj.onload = function () {\n",
       "        if (fig.image_mode === 'full') {\n",
       "            // Full images could contain transparency (where diff images\n",
       "            // almost always do), so we need to clear the canvas so that\n",
       "            // there is no ghosting.\n",
       "            fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "        }\n",
       "        fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "    };\n",
       "\n",
       "    this.imageObj.onunload = function () {\n",
       "        fig.ws.close();\n",
       "    };\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_header = function () {\n",
       "    var titlebar = document.createElement('div');\n",
       "    titlebar.classList =\n",
       "        'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n",
       "    var titletext = document.createElement('div');\n",
       "    titletext.classList = 'ui-dialog-title';\n",
       "    titletext.setAttribute(\n",
       "        'style',\n",
       "        'width: 100%; text-align: center; padding: 3px;'\n",
       "    );\n",
       "    titlebar.appendChild(titletext);\n",
       "    this.root.appendChild(titlebar);\n",
       "    this.header = titletext;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = (this.canvas_div = document.createElement('div'));\n",
       "    canvas_div.setAttribute(\n",
       "        'style',\n",
       "        'border: 1px solid #ddd;' +\n",
       "            'box-sizing: content-box;' +\n",
       "            'clear: both;' +\n",
       "            'min-height: 1px;' +\n",
       "            'min-width: 1px;' +\n",
       "            'outline: 0;' +\n",
       "            'overflow: hidden;' +\n",
       "            'position: relative;' +\n",
       "            'resize: both;'\n",
       "    );\n",
       "\n",
       "    function on_keyboard_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.key_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    canvas_div.addEventListener(\n",
       "        'keydown',\n",
       "        on_keyboard_event_closure('key_press')\n",
       "    );\n",
       "    canvas_div.addEventListener(\n",
       "        'keyup',\n",
       "        on_keyboard_event_closure('key_release')\n",
       "    );\n",
       "\n",
       "    this._canvas_extra_style(canvas_div);\n",
       "    this.root.appendChild(canvas_div);\n",
       "\n",
       "    var canvas = (this.canvas = document.createElement('canvas'));\n",
       "    canvas.classList.add('mpl-canvas');\n",
       "    canvas.setAttribute('style', 'box-sizing: content-box;');\n",
       "\n",
       "    this.context = canvas.getContext('2d');\n",
       "\n",
       "    var backingStore =\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        this.context.webkitBackingStorePixelRatio ||\n",
       "        this.context.mozBackingStorePixelRatio ||\n",
       "        this.context.msBackingStorePixelRatio ||\n",
       "        this.context.oBackingStorePixelRatio ||\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        1;\n",
       "\n",
       "    this.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n",
       "        'canvas'\n",
       "    ));\n",
       "    rubberband_canvas.setAttribute(\n",
       "        'style',\n",
       "        'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n",
       "    );\n",
       "\n",
       "    // Apply a ponyfill if ResizeObserver is not implemented by browser.\n",
       "    if (this.ResizeObserver === undefined) {\n",
       "        if (window.ResizeObserver !== undefined) {\n",
       "            this.ResizeObserver = window.ResizeObserver;\n",
       "        } else {\n",
       "            var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n",
       "            this.ResizeObserver = obs.ResizeObserver;\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n",
       "        var nentries = entries.length;\n",
       "        for (var i = 0; i < nentries; i++) {\n",
       "            var entry = entries[i];\n",
       "            var width, height;\n",
       "            if (entry.contentBoxSize) {\n",
       "                if (entry.contentBoxSize instanceof Array) {\n",
       "                    // Chrome 84 implements new version of spec.\n",
       "                    width = entry.contentBoxSize[0].inlineSize;\n",
       "                    height = entry.contentBoxSize[0].blockSize;\n",
       "                } else {\n",
       "                    // Firefox implements old version of spec.\n",
       "                    width = entry.contentBoxSize.inlineSize;\n",
       "                    height = entry.contentBoxSize.blockSize;\n",
       "                }\n",
       "            } else {\n",
       "                // Chrome <84 implements even older version of spec.\n",
       "                width = entry.contentRect.width;\n",
       "                height = entry.contentRect.height;\n",
       "            }\n",
       "\n",
       "            // Keep the size of the canvas and rubber band canvas in sync with\n",
       "            // the canvas container.\n",
       "            if (entry.devicePixelContentBoxSize) {\n",
       "                // Chrome 84 implements new version of spec.\n",
       "                canvas.setAttribute(\n",
       "                    'width',\n",
       "                    entry.devicePixelContentBoxSize[0].inlineSize\n",
       "                );\n",
       "                canvas.setAttribute(\n",
       "                    'height',\n",
       "                    entry.devicePixelContentBoxSize[0].blockSize\n",
       "                );\n",
       "            } else {\n",
       "                canvas.setAttribute('width', width * fig.ratio);\n",
       "                canvas.setAttribute('height', height * fig.ratio);\n",
       "            }\n",
       "            canvas.setAttribute(\n",
       "                'style',\n",
       "                'width: ' + width + 'px; height: ' + height + 'px;'\n",
       "            );\n",
       "\n",
       "            rubberband_canvas.setAttribute('width', width);\n",
       "            rubberband_canvas.setAttribute('height', height);\n",
       "\n",
       "            // And update the size in Python. We ignore the initial 0/0 size\n",
       "            // that occurs as the element is placed into the DOM, which should\n",
       "            // otherwise not happen due to the minimum size styling.\n",
       "            if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n",
       "                fig.request_resize(width, height);\n",
       "            }\n",
       "        }\n",
       "    });\n",
       "    this.resizeObserverInstance.observe(canvas_div);\n",
       "\n",
       "    function on_mouse_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.mouse_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousedown',\n",
       "        on_mouse_event_closure('button_press')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseup',\n",
       "        on_mouse_event_closure('button_release')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'dblclick',\n",
       "        on_mouse_event_closure('dblclick')\n",
       "    );\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousemove',\n",
       "        on_mouse_event_closure('motion_notify')\n",
       "    );\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseenter',\n",
       "        on_mouse_event_closure('figure_enter')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseleave',\n",
       "        on_mouse_event_closure('figure_leave')\n",
       "    );\n",
       "\n",
       "    canvas_div.addEventListener('wheel', function (event) {\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        on_mouse_event_closure('scroll')(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.appendChild(canvas);\n",
       "    canvas_div.appendChild(rubberband_canvas);\n",
       "\n",
       "    this.rubberband_context = rubberband_canvas.getContext('2d');\n",
       "    this.rubberband_context.strokeStyle = '#000000';\n",
       "\n",
       "    this._resize_canvas = function (width, height, forward) {\n",
       "        if (forward) {\n",
       "            canvas_div.style.width = width + 'px';\n",
       "            canvas_div.style.height = height + 'px';\n",
       "        }\n",
       "    };\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n",
       "        event.preventDefault();\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus() {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'mpl-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'mpl-button-group';\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'mpl-button-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        var button = (fig.buttons[name] = document.createElement('button'));\n",
       "        button.classList = 'mpl-widget';\n",
       "        button.setAttribute('role', 'button');\n",
       "        button.setAttribute('aria-disabled', 'false');\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "\n",
       "        var icon_img = document.createElement('img');\n",
       "        icon_img.src = '_images/' + image + '.png';\n",
       "        icon_img.srcset = '_images/' + image + '_large.png 2x';\n",
       "        icon_img.alt = tooltip;\n",
       "        button.appendChild(icon_img);\n",
       "\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    var fmt_picker = document.createElement('select');\n",
       "    fmt_picker.classList = 'mpl-widget';\n",
       "    toolbar.appendChild(fmt_picker);\n",
       "    this.format_dropdown = fmt_picker;\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = document.createElement('option');\n",
       "        option.selected = fmt === mpl.default_extension;\n",
       "        option.innerHTML = fmt;\n",
       "        fmt_picker.appendChild(option);\n",
       "    }\n",
       "\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', { width: x_pixels, height: y_pixels });\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_message = function (type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function () {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function (fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1], msg['forward']);\n",
       "        fig.send_message('refresh', {});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n",
       "    var x0 = msg['x0'] / fig.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n",
       "    var x1 = msg['x1'] / fig.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0,\n",
       "        0,\n",
       "        fig.canvas.width / fig.ratio,\n",
       "        fig.canvas.height / fig.ratio\n",
       "    );\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch (cursor) {\n",
       "        case 0:\n",
       "            cursor = 'pointer';\n",
       "            break;\n",
       "        case 1:\n",
       "            cursor = 'default';\n",
       "            break;\n",
       "        case 2:\n",
       "            cursor = 'crosshair';\n",
       "            break;\n",
       "        case 3:\n",
       "            cursor = 'move';\n",
       "            break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_message = function (fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n",
       "    for (var key in msg) {\n",
       "        if (!(key in fig.buttons)) {\n",
       "            continue;\n",
       "        }\n",
       "        fig.buttons[key].disabled = !msg[key];\n",
       "        fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n",
       "    if (msg['mode'] === 'PAN') {\n",
       "        fig.buttons['Pan'].classList.add('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    } else if (msg['mode'] === 'ZOOM') {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.add('active');\n",
       "    } else {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message('ack', {});\n",
       "};\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function (fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            var img = evt.data;\n",
       "            if (img.type !== 'image/png') {\n",
       "                /* FIXME: We get \"Resource interpreted as Image but\n",
       "                 * transferred with MIME type text/plain:\" errors on\n",
       "                 * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "                 * to be part of the websocket stream */\n",
       "                img.type = 'image/png';\n",
       "            }\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src\n",
       "                );\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                img\n",
       "            );\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        } else if (\n",
       "            typeof evt.data === 'string' &&\n",
       "            evt.data.slice(0, 21) === 'data:image/png;base64'\n",
       "        ) {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig['handle_' + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\n",
       "                \"No handler for the '\" + msg_type + \"' message type: \",\n",
       "                msg\n",
       "            );\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\n",
       "                    \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n",
       "                    e,\n",
       "                    e.stack,\n",
       "                    msg\n",
       "                );\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "};\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function (e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e) {\n",
       "        e = window.event;\n",
       "    }\n",
       "    if (e.target) {\n",
       "        targ = e.target;\n",
       "    } else if (e.srcElement) {\n",
       "        targ = e.srcElement;\n",
       "    }\n",
       "    if (targ.nodeType === 3) {\n",
       "        // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "    }\n",
       "\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    var boundingRect = targ.getBoundingClientRect();\n",
       "    var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n",
       "    var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n",
       "\n",
       "    return { x: x, y: y };\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys(original) {\n",
       "    return Object.keys(original).reduce(function (obj, key) {\n",
       "        if (typeof original[key] !== 'object') {\n",
       "            obj[key] = original[key];\n",
       "        }\n",
       "        return obj;\n",
       "    }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function (event, name) {\n",
       "    var canvas_pos = mpl.findpos(event);\n",
       "\n",
       "    if (name === 'button_press') {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * this.ratio;\n",
       "    var y = canvas_pos.y * this.ratio;\n",
       "\n",
       "    this.send_message(name, {\n",
       "        x: x,\n",
       "        y: y,\n",
       "        button: event.button,\n",
       "        step: event.step,\n",
       "        guiEvent: simpleKeys(event),\n",
       "    });\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.key_event = function (event, name) {\n",
       "    // Prevent repeat events\n",
       "    if (name === 'key_press') {\n",
       "        if (event.key === this._key) {\n",
       "            return;\n",
       "        } else {\n",
       "            this._key = event.key;\n",
       "        }\n",
       "    }\n",
       "    if (name === 'key_release') {\n",
       "        this._key = null;\n",
       "    }\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.key !== 'Control') {\n",
       "        value += 'ctrl+';\n",
       "    }\n",
       "    else if (event.altKey && event.key !== 'Alt') {\n",
       "        value += 'alt+';\n",
       "    }\n",
       "    else if (event.shiftKey && event.key !== 'Shift') {\n",
       "        value += 'shift+';\n",
       "    }\n",
       "\n",
       "    value += 'k' + event.key;\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n",
       "    if (name === 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message('toolbar_button', { name: name });\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "\n",
       "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n",
       "// prettier-ignore\n",
       "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";/* global mpl */\n",
       "\n",
       "var comm_websocket_adapter = function (comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.binaryType = comm.kernel.ws.binaryType;\n",
       "    ws.readyState = comm.kernel.ws.readyState;\n",
       "    function updateReadyState(_event) {\n",
       "        if (comm.kernel.ws) {\n",
       "            ws.readyState = comm.kernel.ws.readyState;\n",
       "        } else {\n",
       "            ws.readyState = 3; // Closed state.\n",
       "        }\n",
       "    }\n",
       "    comm.kernel.ws.addEventListener('open', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('close', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('error', updateReadyState);\n",
       "\n",
       "    ws.close = function () {\n",
       "        comm.close();\n",
       "    };\n",
       "    ws.send = function (m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function (msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        var data = msg['content']['data'];\n",
       "        if (data['blob'] !== undefined) {\n",
       "            data = {\n",
       "                data: new Blob(msg['buffers'], { type: data['blob'] }),\n",
       "            };\n",
       "        }\n",
       "        // Pass the mpl event to the overridden (by mpl) onmessage function.\n",
       "        ws.onmessage(data);\n",
       "    });\n",
       "    return ws;\n",
       "};\n",
       "\n",
       "mpl.mpl_figure_comm = function (comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = document.getElementById(id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm);\n",
       "\n",
       "    function ondownload(figure, _format) {\n",
       "        window.open(figure.canvas.toDataURL());\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element;\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error('Failed to find cell for figure', id, fig);\n",
       "        return;\n",
       "    }\n",
       "    fig.cell_info[0].output_area.element.on(\n",
       "        'cleared',\n",
       "        { fig: fig },\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function (fig, msg) {\n",
       "    var width = fig.canvas.width / fig.ratio;\n",
       "    fig.cell_info[0].output_area.element.off(\n",
       "        'cleared',\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "    fig.resizeObserverInstance.unobserve(fig.canvas_div);\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable();\n",
       "    fig.parent_element.innerHTML =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "    fig.close_ws(fig, msg);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.close_ws = function (fig, msg) {\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width / this.ratio;\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message('ack', {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () {\n",
       "        fig.push_to_output();\n",
       "    }, 1000);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'btn-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'btn-group';\n",
       "    var button;\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'btn-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        button = fig.buttons[name] = document.createElement('button');\n",
       "        button.classList = 'btn btn-default';\n",
       "        button.href = '#';\n",
       "        button.title = name;\n",
       "        button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message pull-right';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = document.createElement('div');\n",
       "    buttongrp.classList = 'btn-group inline pull-right';\n",
       "    button = document.createElement('button');\n",
       "    button.classList = 'btn btn-mini btn-primary';\n",
       "    button.href = '#';\n",
       "    button.title = 'Stop Interaction';\n",
       "    button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n",
       "    button.addEventListener('click', function (_evt) {\n",
       "        fig.handle_close(fig, {});\n",
       "    });\n",
       "    button.addEventListener(\n",
       "        'mouseover',\n",
       "        on_mouseover_closure('Stop Interaction')\n",
       "    );\n",
       "    buttongrp.appendChild(button);\n",
       "    var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n",
       "    titlebar.insertBefore(buttongrp, titlebar.firstChild);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._remove_fig_handler = function (event) {\n",
       "    var fig = event.data.fig;\n",
       "    if (event.target !== this) {\n",
       "        // Ignore bubbled events from children.\n",
       "        return;\n",
       "    }\n",
       "    fig.close_ws(fig, {});\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (el) {\n",
       "    el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (el) {\n",
       "    // this is important to make the div 'focusable\n",
       "    el.setAttribute('tabindex', 0);\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    } else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (event, _name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager) {\n",
       "        manager = IPython.keyboard_manager;\n",
       "    }\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which === 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "};\n",
       "\n",
       "mpl.find_output_cell = function (html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i = 0; i < ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code') {\n",
       "            for (var j = 0; j < cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] === html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "};\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel !== null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target(\n",
       "        'matplotlib',\n",
       "        mpl.mpl_figure_comm\n",
       "    );\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAiUAAAFuCAYAAABeGmjZAAAAAXNSR0IArs4c6QAAIABJREFUeF7t3QmUHVWdx/Fvp5ushNVAWGQLSATZF20GxwVlXHDQGRAUlV0Q0RDAyBJ2QljFAAKKoKCyCIoOoiPgsAdkcYOAYIMIsonsJIQk3T3nphrsJL286lf13q2q7zsnRw+579a9n/+F/nXVraoW/CiggAIKKKCAAhEItEQwBoeggAIKKKCAAgpgKHERKKCAAgoooEAUAoaSKMrgIBRQQAEFFFDAUOIaUEABBRRQQIEoBAwlUZTBQSiggAIKKKCAocQ1oIACCiiggAJRCBhKoiiDg1BAAQUUUEABQ4lrQAEFFFBAAQWiEDCURFEGB6GAAgoooIAChhLXgAIKKKCAAgpEIWAoiaIMDkIBBRRQQAEFDCWuAQUUUEABBRSIQsBQEkUZHIQCCiiggAIKGEpcAwoooIACCigQhYChJIoyOAgFFFBAAQUUMJS4BhRQQAEFFFAgCgFDSRRlcBAKKKCAAgooYChxDSiggAIKKKBAFAKGkijK4CAUUEABBRRQwFDiGlBAAQUUUECBKAQMJVGUwUEooIACCiiggKHENaCAAgoooIACUQgYSqIog4NQQAEFFFBAAUOJa0ABBRRQQAEFohAwlERRBgehgAIKKKCAAoYS14ACCiiggAIKRCFgKImiDA5CAQUUUEABBQwlrgEFFFBAAQUUiELAUBJFGRyEAgoooIACChhKXAMKKKCAAgooEIVAaULJ+PHju9dee+0oUB2EAgooUK/AHXfc8QywSr39+H0FiiRQmlDS3t7ePXPmzCLZO1YFFFCgX4GWlpY7gG0kUqBKAoaSKlXbuSqgQGEEDCWFKZUDzVDAUJIhpl0poIACWQkYSrKStJ8iCRhKilQtx6qAApURMJRUptROtJeAocTloIACCkQoYCiJsCgOKXcBQ0nuxB5AAQWKJHDmpH0Z9uw8OplNK2PoWnk4k2dc0PApGEoaTu4BIxAwlERQBIeggAJxCIRA0vXMc8CCXgNqY9j4cQ0PJoaSONaEo2isgKGksd4eTQEFIhaYsevuLOh+fokRtrWsyKTLL27oyA0lDeX2YJEIGEoiKYTDUECB5gt8Y9ed6O6eu8RAWhjFwVdc2dABGkoayu3BIhEwlERSCIehgALNF8jyTEm9e1MMJc1fD46g8QKGksabe0QFFIhUIKs9JVn0YyiJdJE4rFwFYgglI4FbgOFAGzADCFvdHwNeBrqBp4CPDSThY+ZzXSd2rkBlBN46w9E9h9aW0UO6+yaLMy6GksosOSfaSyCGUBLGMBqYDYwB7gM2B/4ATASWvMDbRwkNJa5rBRSIRSCLvSmGkliq6TgaKRBDKOk93xWA3/WEkvC/hpJGrgaPpYACmQh4piQTRjupoEAsoWQUcCewLjAF+BbwV+AFoBM4FbjKyzcVXKFOWYECCrinpIBFc8hRCMQSSt7EGAf8BNgZaO3ZS7Ia8BtgB6BjMbW9gfCHCRMmtHd0LP7XURg7CAUUqKBAvXtTvHxTwUXjlIktlISSnAXcCvR+KMDZwM0DnS1xT4mrWQEFyiRgKClTNZ1LrQIxhJJwdmQ+8BIwFrgd+DzwKPAqsGxPSPkMMKu/iRlKai257RRQoAgChpIiVMkxZi0QQyjZGAjPbx7W8+dc4NfA1T2TDf/8HODbA03eUJL10rA/BRRopoChpJn6HrtZAjGEkkzmbijJhNFOFFAgEgFDSSSFcBgNFTCUNJTbgymggAK1CRhKanOyVbkEDCXlqqezUUCBkggYSkpSSKeRSsBQkorLxgoooEBjBAwljXH2KHEJGEriqoejUUABBRYKGEpcCFUUMJRUserOWQEFchV468FpzKaVMUN6qZ+hJNcS2XmkAoaSSAvjsBRQoH+BLH7o5+WbxSPmPVOSV3XsN3YBQ0nsFXJ8CiiwiEBWP/TzYs3iZXyGkryqY7+xCxhKYq+Q41NAgUUEsvqhnxfrN3bdie7uuUt038IoDr6i99szBh6Bl2/yqpD9xixgKIm5Oo5NAQWWEMjqh35etFmFJkNJXhWy35gFDCUxV8exKVARgTR7RLL6oZ8XbVaXlwwleVXIfmMWMJTEXB3HpkAFBNL+EE/bvhmEb4assa88y6tjV6Jr/Agmz7gg1VAMJam4bFwSAUNJSQrpNBQoqsBQzny8dWalew6tLaOHdMttrl5//zsccwx8//tw6aWwyy6pD2coSU3mF0ogYCgpQRGdggJFFoh9j0gq2xdfhFNOgRkzoK0NpkyByZNh6aVTdRMaG0pSk/mFEggYSkpQRKegQJEFhnKmJLr5zp0L55wDJ50Er70G++8PU6fCSisNeaiGkiHT+cUCCxhKClw8h65AGQSKsEekX+fOTvjhD+Goo+CJJ2DXXeHEE2HChLpLYyipm9AOCihgKClg0RyyAmUTiH6PyOLg3d3wq1/BYYfBfffBdtsll2222CKz0hhKMqO0owIJGEoKVCyHqoACEQj89rfw9a/DzTfDppsmYWT77TMfmKEkc1I7LIBADKFkJHALMBxoA2YA4d65rYGLgPD3lwDHD+TZ3t7ePXPmzAKQO0QFFCikwMMPw5FHwlVXwVprwbRpyeWaYcNymY6hJBdWO41cIIZQEsYwGpgNjAHuAzYHrgP2BP4M3A7s2/N3fZIaSiJfaQ5PgaIKPP00HH88XHABLLdcsn8kbGQdMSLXGRlKcuW180gFYgglvWlWAH4HbAP8HNiq5y+/AoR76qb352goiXSFOSwFiirwyitw+ulwxhnJDA4+GA49FJZdtiEzMpQ0hNmDRCYQSygZBdwJrAtMAX4LTAU+2eP1KWA74EBDSWQryOEo0ESBNI+nr3mY8+bB+efDCSdAeO7IPvskD0JbZZWau8iioaEkC0X7KJpALKHkTbdxwE+ArwGH1xBK9gbCHyZMmNDe0dFRNH/Hq4ACQxTI/Fbiri644ork+SKPPgr//d/JvpH11x/iCOv7mqGkPj+/XUyB2EJJUDwLuLfnrEjvyzdjgZM8U1LMheaoFchaINOHrt1wQ3JHze9+B+99L5x6KrznPVkPOVV/hpJUXDYuiUAMoSScHZkPvASE4BE2tX4OuBDYo2ej623AF93oWpJV5zQUyEAgk8fThxASnjVy/fWw4YZw8snw8Y+HZ7xnMML6ujCU1Ofnt4sp0Px/82Bj4GIg3FcX/pwLnAeEX1NCMAm3BP8AOHYgYje6FnMBOmoFhipQ15mScHkm3EUTXpa3+urJ/pHPfx5aW4c6nMy/ZyjJnNQOCyAQQyjJhMlQkgmjnShQGIEh7Sl57rnkMfDnnQdjxsARR8CBB8KosNc+ro+hJK56OJrGCBhKGuPsURRQIAeBmh9PP3s2nHlmslck3F0zaVJy2Wb55XMYVTZdGkqycbSXYgkYSopVL0ergAJpBObPh4sugmOPpevZZ3lwrXW4/V3v4PXR4+haeTiTZ4SHR8f5MZTEWRdHla+AoSRfX3tXQIEmCJz51X14x70dtN93Lyu8+hqPvH11bt1gPZ5fNjw8+s1PG8PGj4s2mBhKmrBwPGTTBQwlTS+BA4hNIJcHcsU2yRKP58c77cC2N85k1Rde5KkVl+OWjSfy5Li3Ad1LzLqtZUUmXR722cf3MZTEVxNHlL+AoSR/Y49QIIEhbZ4s0PwaOdSGh7v77oPDD4drr+WFsWO4daP16Vht/IC397YwioOvuLKRLDUfy1BSM5UNSyRgKClRMZ1K/QJ13WZa/+FL0UMIIzz7Ol3d4dFDvT85XS55/PHkMfAXXwwrr8z1q6/CfWuNp7uGt/d6pqQUS85JlEjAUFKiYjqV+gUyeSBX/cMobA/f3HMPOuc83+elkjCpTEPACy/A9Olw9tkwfHjyRNaDDmLG3gewoDuMYfFP+M9d70s4OYWkjKrnmZKMIO2mUAKGkkKVy8HmLeCZktqFF7880z2qZcBAEnrO5HLJ668nQSQEknCr75e/DEceCW8L+0agv0twraOXo+X1bjq759DaMtq7b2ovtS0VaJiAoaRh1B6oCALuKamtSn07LX4mYsm+6jpT0tkJl1wCRx8Nf/877LZb8iTWtdde4kA1P7+ktuk2pZVnSprC7kGbLGAoaXIBPHx8AjH8QGv4JtGUZejvjNLA3Qzxckl3N/ziF8km1lmzYPvtk3fUbLZZylEXq7mhpFj1crTZCBhKsnG0FwUyEyjC2Zr+9t70hzCM5WD8qPTPBLnjjmSvyK23wuabwymnwIc+lJl1zB0ZSmKujmPLS8BQkpes/SrQSyDNmY8i7Gvp/0zJ4pdwWmgdvSIHfe/76dbDn/+cvJfm6qthnXVg2jT49Kehhjtq0h0o3taGknhr48jyEzCU5GdrzwosFEh75qMIdwDltpn0qafguOPgwgthhRWS/SNf/GJyd03FPoaSihXc6S4UMJS4EBTIWSDtmY+07XMefr/dZ7r35uWX4bTT4BvfSM6GHHIIHHoojB3brOk1/biGkqaXwAE0QcBQ0gR0D1ktgbRnPtKeWSm05htvwHnnwYknQggm4azIUUfB+PGFnlYWgzeUZKFoH0UTMJQUrWKOt3ACQznzkelZiBjFurrgsstg6lR47DHYeedk38h668U42qaMyVDSFHYP2mQBQ0mTC+Dhyy9QqTMfg5Uz3N573XXJHTV//CO8731w6qmw9daDfbNyf28oqVzJnXAke0rWBy4ClgXmAZOBm4H5wKyeKt0D7DNQxdrb27tnzpxpURWIUqD0Zz5qUb/nniSM/N//wUYbJbf3fuQjA74wr5Zuy9rGUFLWyjqvgQRiOFOyJjASeAiYCFwDhHO4zwA1X1g2lLjQFYhUoKMjuUxzxRWwxhrJU1jD01hbWyMdcBzDMpTEUQdH0ViBGEJJ7xmH8bwZRp42lDR2MXg0BTIV+Mc/kgBy/vmwzDLJ+2kOOABGht9B/AwmYCgZTMi/L6NAbKFkR2B/4KPAG8D9wOvAVOAmL9+UcQk6p9IJvPZacmtvuMV3wYKFb+5deNlmueUWmWqaB8qVzqiGCRlKakCySekEYgol4TLO9cDHgA5gVeApYAPgWmBT4OXFKrA3EP4wYcKE9o5wmtiPAgo0R2D+fLjgguThZ//8J+y1Fxx7LKy22hLjcfPv4CUylAxuZIvyCcQSSpYBbgGmANf1wRz2mRwHhA2vfX7cU1K+xemMCiIQ7qi56qrksfDhF4Mdd4STToINwu8TfX+Gcpt0QTQyG6ahJDNKOyqQQAyhJOx2C2dCfg6c12O3PDCn5xJOOGNyO7AF8IKhpECry6GWX+DGG5NLM3ffDdtsk9xRs+22g8477QPlBu1wsQZluDRkKElbdduXQSCGULIDcHWv23+D6wHAd4BOoKvnLMnPBgL3TEkZlqNzKIxAeMbIYYfB//4vTJwIJ5/Mmb+5hmH/mE8ns2llDF0rD+/3rcB5nikpy6UhQ0lh/m1woBkKxBBKMpmOoSQTRjtRYGCBv/0teQz8D38Iq6yS7B/ZYw/OPORLdD3zHLCg1/fbGDZ+XJ/BJM/gkGfgaeTyMJQ0UttjxSJgKImlEo5DgZgFnn8+2SdyzjkwalRyluSrX4XRoxeOeihBIK8HyuV9aahRZTKUNEra48QkYCiJqRqORYHYBObMgRkzFl6eYe5cOPDAZEPriisuMtKYgsBQAlJs7GE8hpIYq+KY8hYwlOQtbP8KFFEgPF/k+9+HY46Bp5+Gz30ueRDamuHO/SU/MQWBPC8NNbKUhpJGanusWAQMJbFUwnEoEINAuL33f/4HDj8cHnwQPvpRmD4dNtlkwNHFFgTyujTUyBIZShqp7bFiETCUxFIJx6FAswVuvx2mTIHwYsuttkpu7/3AB2oeVRmCQM2TbUBDQ0kDkD1EdAKGkuhK4oAUaLDAAw8kZ0bCGZJ11002tO60k2/vbXAZFj+coaTJBfDwTREwlDSF3YMqEIHAk08me0a+9z0YNy75//vsA0stFcHgHIKhxDVQRQFDSRWrXsI5l+EJng0ry0svJZdmvvlNaGuDr30NDj4Yll66YUPo60DWcFEVQ0lTl6MHb5KAoaRJ8B42O4HYNllmN7OMewq39H7rWzBtGrz6Kuy/f/IgtJVWyvhA6buzhkuaGUrSryO/UXwBQ0nxa1j5GdRyO2qMv4U3bEydnfCjHyUB5PHHYZdd4MQTk/0jkXxqqWEkQ23YMAwlDaP2QBEJGEoiKoZDGZrAYA/uivG38IaMKdze+6tfJU9fve8++OAHk8s2W245NOgcvzVYDXM8dLRdG0qiLY0Dy1HAUJIjrl03RmCw37IH+/vGjHLRo+Q+prvuSt7ee9NNsOmmSRj58IejvaMmd49mFLnOYxpK6gT064UUMJQUsmwOurfAYGcdYvwtPKsxLX4JaJkRr7PnnG646ipYa63kMs1nPgPDhkW9aAarYdSDz2lwhpKcYO02agFDSdTlcXC1Cgz04K4YfwvPYky9f5CPfn0u7Q/8hY0efYJ5I4Yz6uRTko2sI0bUStj0dj58bdESGEqaviQdQBMEDCVNQPeQjRWI8bfwLMYUgk3LvGfZ6qFH2eLhR2np7ubed6zD7yduyZd+elljkT1a5gKGksxJ7bAAAoaSAhTJIdYvkNVv4VneMfNmXwu6XwO6w3thaWtZmq6VhzN5xgUDT3rePP7vPVvzngceZOS8+dy39tu5Y8P1mD1qJC2M4uArrqwfzR6aKmAoaSq/B2+SgKGkSfAeNg6BNCEji7Mbi886dZ9dXfDjH8ORR8Kjj/KX1Vbmto0m8sIy/3rwWVvLiky6/OI4gB3FkAUMJUOm84sFFjCUFLh4Dr0+gbSBIIt9IIuPOFWfN9yQ3FHzu9/Bttty+SrL82RL2MC6oFe3bQwbP27wMy310fntBggYShqA7CGiE4ghlKwPXAQsC8wDJgM3AzsAZwDhv7qnAN8dSK+9vb17Zni7qZ/KCKQ5y9EXSqpAAGR1x0zvsdTU5+9/nzxr5LrrYMMN4eST4eMfX3h7b1aXpSqzaAo0UUNJgYrlUDMTiCGUrAmMBB4CJgLXAO8E7gfCe9NfBe4B/g14vr+ZG0oyWxOF6CjtWY6+JlVTIOj1xbQhphbIAfucfmzyFNbwNNbVV4fjj4cvfAFaW2vp2jYFFzCUFLyADn9IAjGEkt4DD+N5Bvgv4CBg556/DGdMQjDp95YCQ8mQ6l/YL2URENL2kUUQWhy8rz5HvdHJe/72BJvPehjGjIEjjoADD4RRowpbLweeXsBQkt7MbxRfILZQsiOwP3AhsG1PMAnK4ZJO+PXwdM+UFH/RZTGDtGc5+jrmUEJGHpdL3uyzZf4rbPmXx9jiz7MYER529tWvJpdtVlghCzL7KJiAoaRgBXO4mQjEFErCZZzrgY8Bm9YYSvYGwh8mTJjQ3tHRkQmKncQvkPYsR38zGihk1LtnpWbFBQvgwgvh2GPh2Wdh993huONgjTVq7sKG5RMwlJSvps5ocIFYQskywC3AFOA6YJuesyO9L9/cC1zqmZLBi1qFFkM5y5HGJe/+F44lvDDv6qvh8MPh4Ydhhx3gpJNgo43SDNW2JRUwlJS0sE5rQIEYQkm4LHMt8HPgvJ7RtgGzgPf3bHQNgSQEFTe6uqDfEsjjUsqbnWd1Jqbfct16K0yZAnfeCe9+d/LCvPe9z+oq8JaAocTFUEWBGEJJuPX36p4Q8mYNQhj59549JOGW4FOB7wxUIDe6lm/5NuzySR90WexZ6bMi99+fnBn5xS/gHe+A6dPhU5+K9u295VtVxZmRoaQ4tXKk2QnEEEoymY2hJBPGaDppyOWTAWab+ZmSJ56AY46Biy+GlVZK9o/stRcstdSQzZsZ2oY8aL9Ys4ChpGYqG5ZIwFBSomKWaSqZh4KUOJmFohdfTM6GnHUWDB+eXLKZPDm51beOT2bjq2MMfjVfAUNJvr72HqeAoSTOutQ9qqL/Fp3b5ZN+ZPvyCk2HPTuPzu45tLaMru1FeW/2//rrcM45ycbV2bPhgAOS99WMG1d3bUMHzQ5tmUzCTgYUMJS4QKooYCgpYdXL8Ft0I3/oZurV2QmXXAJHHw1//zt89rNwwgmwzjqZrrRGh7ZMB29nNQkYSmpislHJBAwlJStoWX6LToLCP4DOXhVqZdj4lTJ/2VwmASjc3nvttcnDzmbNgu23T95Rs9lmuaywTMacy8jsNCsBQ0lWkvZTJAFDSZGqVeNYy/BbdCNDSd1e4bbesFck3Oa7+ebJ7b0f+lCN1Rpas0zP7gxtCH4rZwFDSc7Adh+lgKEkyrLUN6gy/BbdyDkM+VgPPZS8l+anP00uz0ybBp/+NIRHxDfgk+dzWhowfA8xiIChxCVSRQFDSQmrXobfous+e5Girqm9nn46eQz8d78Lyy+f7B/Zb7/k7ho/CmQkYCjJCNJuCiVgKClUuWofbNF/ix7y2YvaiRZpWZPXyy/DaafBmWcm3z3kEDj0UFgmvCXBjwLZChhKsvW0t2IIGEqKUafKjTL12Ys8hd54A84/P7mL5qWX4ItfTM6OjB+f51Htu+IChpKKL4CKTt9QUtHCF2HaNZ29yHMiXV1w2WUwdSo89hjstFOybyQ8Ht6PAjkLGEpyBrb7KAUMJVGWxUE1XeC66+DrX4c//CF5UV64oya8OM+PAg0SMJQ0CNrDRCVgKImqHA6m6QL33ps8a+SGG2CjjZJnjXz0o74wr+mFqd4ADCXVq7kzBkOJq0CBIPDII8llmssvhzXWSPaP7LYbtLbqo0BTBAwlTWH3oE0WMJQ0uQAevskC//gHnHhispF16aWT99N8+cswcmSTB+bhqy5gKKn6Cqjm/A0l1ay7s37tNfjGN5JbfBcsgEmTkss2yy2njQJRCBhKoiiDg2iwgKGkweAerskC8+cnDz0LDz977jnYc0849lhYffUmD8zDK7CogKHEFVFFAUNJFatexTmHF+ZddVXyWPiODvjP/4STToINN6yihnMugIChpABFcoiZC8QQSq4CtgN+DezaM8PHgJeBbuAp4GODzby9vb175syZgzXz70sq8NYzTZhNK2PoWnn4v94mfNNNye29d90F22yT3N677bYNlxhwjA0fjQeMXcBQEnuFHF8eAjGEkvcDY4HdFgslE4G5tU7aUFKrVPna9ff015XahvH5F+fCr34FEyfC9Omw445Nub03qifUlm8JlHJGhpJSltVJDSIQQygJQwzBZH9Diet1KAKLvydn7Ow5/Nv9D7PB356kZdVVk/0je+wBbW1D6T6T7zT6XT6ZDNpOmipgKGkqvwdvkkCsoeSvwAtAJ3AqEC7xDPjxTMlgQuX9+zffKDzyjXm8+8EONu34G52tw7hr4kTee+dvYfTopk++kW89bvpkHUAmAoaSTBjtpGACsYaSVXv2kqwG/AbYAejow3ZvIPxhwoQJ7R1hA6Ofygl8a6fd2Pjhe9nqz4/Q1tnFH9Zdk9++c10WjFyFSZdfHIWHZ0qiKEOhBmEoKVS5HGxGArGGkt7TOxu4ebCzJZ4pyWhFFKmb8HyRiy/m1UlfZenZc3hgzdWY+a538MqYcGakjWHjx/1rs2uT5+WekiYXoICHN5QUsGgOuW6BGEPJGGAY8CqwLHAr8Blg1kCzNZTUvRaK00G4vfeaa+Dww+GBB+AjH+EHy4/iha6xdHbPobVl9KJ330Qys6a/9TgSB4dRm4ChpDYnW5VLIIZQci2wNRDCSNhHsiNwUQ9zCCfnAN8ejN1QMphQSf7+9tuT23vD/265ZXJ77wc/mOvkvJU3V14770fAUOLSqKJADKEkE3dDSSaM8Xby4IPJmZGf/zxsIEoefLbzzrnf3utll3iXRNlHZigpe4WdX18ChhLXRdwCTz6ZPAb+oovgbW+Do4+GffeF4cMbMm43qDaE2YP0IWAocVlUUcBQUsWqF2HOL72UXJr55jehtRW+9jU4+GAYG56z17iPt/I2ztojLSpgKHFFVFHAUFLFqsc857lz4dxzYdo0eOUV2G8/OOooWHnlpozaMyVNYfeghCuTLXcA24ihQJUEDCVVqnbMc+3shEsvhalT4fHHYZdd4MQTYd11mzpq95Q0lb/SBzeUVLr8lZ28oaQEpS/03SHh9t5f/zq5o+ZPf0rupAmXbcKdNZF8vJU3kkJUbBiGkooV3OkuFDCUNHEhZBEmGvmbfBbjXYT77ruTMHLjjbDJJkkY2X773O+oaWLJPbQCNQsYSmqmsmGJBAwlTSpmVmGiUXseshrvQu6//AWOPBKuvBLWXDO5TPPZz8Kw8FgaPwoosPA3RveUuBAqKGAoaVLRswoTjbo7JJPxPvssHH88fOc7sMwyyf6RAw6AESNyq0LmZ3dyG6kdK7CogKHEFVFFAUNJk6qeVZjIJCzUYFDXeF99FU4/Hc44A7q6YPJkmDIFlg1vEcjvk+nZnfyGac8K9ClgKHFhVFHAUNKkqmcVJhr1g3dI4503LzkrEs6OPP887L138iC0VcNLoPP/DGnM+Q/LIyhQk4ChpCYmG5VMwFCSsqBZXQ7IMkw04u6Qb+65B51z/rmEVuvot3HQ976/6D8PZ0PCfpGwb+SRR+BTn0oeCz9xYkrt+prXdXanvkP7bQXqFjCU1E1oBwUUMJSkKFqWQSIcthFhIsX0Bmxa81mH3/wmuaPm3nth223h1FOhvT2rYaTqp+Yxp+rVxgo0RsBQ0hhnjxKXgKEkRT2q/ENu0LMOf/gDHHZY8syRDTaAk0+GHXZo6u29WYfIFEvFpgrULWAoqZvQDgooYChJUbRBfzCn6KtoTfsLZMvPGcFey46CH/0IVlst2T/yhS9AW1sUUyzS2agowBxENAKGkmhK4UAaKFDZUDKUvSFlOlOSdv6Ln3UY9cY83v3AI2zyyGO0hZfkHXEEfOUrMGpUA5evh1KgvAKGkvLW1pn1L1DJUDLU0/pD/V5sC3Co8wjfG/7kbDZ56E9s9dDDtHZ10RZu7z38cFhhhdim6Xhmif3MAAAU9ElEQVQUKLSAoaTQ5XPwQxSoZCip54xHGS4HDGn+CxbARRclt/Q+8wzsvjscdxysscYQl55fU0CBgQQMJa6PKgpUMpRUeW9IWOSp5h9emPeznyVnQx56CD7+cZg+HTbaqIr/vjhnBRomYChpGLUHikgghlByFbAd8Gtg1x6brYGLgJHAJcDxg5m1t7d3z5w5c7BmC/9+SGcKauq5GI1qnv+ttyZPXr3zTth66+T23ve9rxiTdJQKFFzAUFLwAjr8IQnEEEreD4wFdusVSu4C9gT+DNwO7AvcN9AM04SSoe6pGJJwhF8adP6zZiVnRq65BtZbLzkz8l//1dTbeyNkdEgK5CpgKMmV184jFYghlASaEEz27wkl4RnkPwe26jH7CrA0MD2rUBL6KcPekHrWVJ/zP/RoOOYYuPhiGDcu2T8SHg2/1FL1HMrvKqDAEAQMJUNA8yuFF4gxlGwJTAU+2aP7qZ7LOwdmGUoKX7ksJ/Dii8nDzs46Kwkg4ZLNQQfB0iEL+lFAgWYIGEqaoe4xmy1Q9FCyNxD+MGHChPaOjo5mexbr+HPnwtlnJ++lmT0bvvQlmDo1OUviRwEFmipgKGkqvwdvkkCMoaSvyzdhz8lJninJaJV0dsIPfgBHHw1PPAGf/SyccAKss05GB7AbBRSoV8BQUq+g3y+iQIyhJDjeDezRs9H1NuCLWW50LWKhMhlzuL33l79M3lFz//3woQ/BKafA5pv32X3ap75mMkY7UUCBhQKGEhdCFQViCCXXAuEW4DHAC8AngBHAhT23BP8AOHaw4qS5+2awvkr59+G23vD23ltugc02S8LIhz/c71QHvUOnlEhOSoF4BAwl8dTCkTROIIZQkslsDSX9MIYHnh15JPzkJ7D22jBtGuyyCwwbNqB7zc8yyaR6dqKAAosLGEpcE1UUMJSUsOrhLMfSj73E1rPuZaNHH2PuiBGMDmdG9tsPRoSTUIN/Uj31dfDubKGAAikFDCUpwWxeCgFDSSnK+K9JfGv/3dn8ljvZ8uFHFv7De9Zfm3vWfwcL3r4qk2dcUPNsazlT4p6TmjltqEBqAUNJajK/UAIBQ0kJirhwCm+8Ad/+NnOmTGHkvHn8aZ23c+cG6zF7VHhSP7S1rMikyy+uebaD7SkZ7O9rPpANFVCgTwFDiQujigKGkqJXvasLLr88eb7IX//Kw6uvxm0brcuLYxd98FkLozj4iitTzXagp97WciYl1cFsrIACiwgYSlwQVRQwlBS56tdfn9xR8/vfw7//+8IX5s0481wWdD+/xKzSnikZjMU9J4MJ+fcK1CdgKKnPz28XU8BQUsS63Xtv8qyRG26Ad70reUT8xz628IV5eV9WefPsyYLuF4Gu3MNPEcvjmBXIQsBQkoWifRRNwFBSpIo9+mhymeayy+Dtb0+ewvq5z0Fr6yKzyOtlg30Hnt6HbmPY+HGpNtQWid+xKtBIAUNJI7U9ViwChpJYKjHQOJ57Lgkg55+fvCTviCPgwANhZLKJtVGf/vaRwDDaWpana+XhBpJGFcPjlF7AUFL6EjvBPgQMJTEvi9degzPPhNNOg/nzYdKkZA/J8ss3ZdTuI2kKuwetqIChpKKFr/i0DSUxLoAQQC68EI49FsJZkj32SP5/uGTTxI933DQR30NXTsBQUrmSO+HwzqeyKJTiMfPhhXnhcfDh8sxf/gKf+ARMnw4bbhhFmfLeRBvFJB2EApEIGEoiKYTDaKiAoaSh3AMc7OabYcoUuOsuaG9PXpj33vfGMrq3xpHXJtroJuqAFGiygKGkyQXw8E0RMJQ0hb3XQf/0Jzj8cPjlL2H99ZMzI5/85MLbe/0ooEB1BQwl1a19lWdemp98jbp8k9n7Xh5/HI4+Gi65BMaPh+OOgz33hLa2Kq9H566AAj0ChhKXQhUFDCUpqp7JnooXXkjOhpx9dvLG3nA3zUEHwejRKUZiUwUUKLuAoaTsFXZ+fQkYSlKsi7ruPnn9dTjrrCSQhP//5S8nG1rf9rYUI7CpAgpURcBQUpVKO8/eAoaSFOthSM/pWLAguUQTLtU89RTstlvyILS11kpxZJsqoEDVBAwlVau48w0CsYeS+cCsnlLdA+zTX9kasack1ZmScHvvNdckm1gfeAD+4z+Sd9RsuqkrTwEFFBhUwFAyKJENSigQeyh5Bhhfi3sjQknNe0pmzkz2itx2G2yxRXJ773bb1TIN2yiggAILBQwlLoQqChhKUlZ9wOd0PPhgsk/kZz+DCRNg2jTYeWcYNizlUWyugAJVFzCUVH0FVHP+sYeSN4D7gdeBqcBNzbx80+8SCXtFwmPgw6PhV1wRjjkG9t0Xhg+v5qpy1gooULeAoaRuQjsooEDsoWRV4ClgA+BaIGzIeLmX895A+MOECRPaOzo6GluCl1+GU09NXpoXzoYceigccgiMHdvYcXg0BRQonYChpHQldUI1CMQeSnpP4RrgOCBseF3i04g9JW8d9I034Nxz4cQT4ZVXYL/94KijYOWVayC3iQIKKDC4gKFkcCNblE8g5lCyPDAHCJdwwhmT24EtgBeaFkq6uuDSS2HqVPjb3+DTn06CyXrrlW9lOCMFFGiqgKGkqfwevEkCMYeSbYDvAJ1AV89Zkp/155TrmZJwe++vfw2HHQZ//CN84APJHTVbbdWksnlYBRQou4ChpOwVdn59CcQcSlJVLLdQcvfdye29N94IG2+chJHwzBFfmJeqPjZWQIF0AoaSdF62LoeAoaS/OoZNs0ceCT/+MayxRnKZJjyN1dt7y7HynYUCkQsYSiIvkMPLRcBQsjjrs88mj4H/9rdhmWWS/SNf+hKMHJlLAexUAQUU6PM0dkvLHUC4jO1HgcoIGEreLPWrr8IZZ8Dpp0PY0Bre3Bsu2yy7bGUWgxNVQIF4BDxTEk8tHEnjBAwl8+bBBRfA8cfDP/8Je+2VPAhttdUaVwWPpIACCiwmYChxSVRRoLqhJJwNufLKZN/II4/AJz8JJ50E73xnFdeBc1ZAgcgEDCWRFcThNESguqFk/nzYcENYaaXkqazbeOm2ISvOgyigQE0ChpKamGxUMoHqhpJQyGeeSZ7C6u29JVvWTkeB4gsYSopfQ2eQXqDaoSS9l99QQAEFGiJgKGkIsweJTKDyoeTMSfsy7Nl5dDKbVsbQtfJwJs+4ILIyORwFFKiagKGkahV3vkGg0qEkBJKuZ54DFvRaDW0MGz/OYOK/Hwoo0FQBQ0lT+T14kwQqHUpm7Lo7C7qfX4K+rWVFJl1+cZNK4mEVUECBsNXNh6e5DqonUOlQ8o1dd6K7e+4SVW9hFAdfcWX1VoMzVkCBaAQMJdGUwoE0UKDSocQzJQ1caR5KAQVSCRhKUnHZuCQClQ4l7ikpySp2GgqUUMBQUsKiOqVBBSodSoLOW3ffdM+htWW0d98MumRsoIACjRAwlDRC2WPEJlD5UBJbQRyPAgooEAQMJa6DKgoYSqpYdeesgALRCxhKoi+RA8xBIPZQsgNwBjAMOAX4bn8G7e3t3TNnzsyByC4VUECBxgsYShpv7hGbLxBzKGkD7gc+ALwK3AP8G7Dkg0UAQ0nzF5MjUECB7AQMJdlZ2lNxBGIOJeG1vZOBnXs4wxmTEEwu64vXUFKcRedIFVBgcAFDyeBGtiifQMyhZCdgW+CgHvYQUFqB0w0l5VuIzkgBBRYVMJS4IqooUPRQsjcQ/jBhwoT2jo6OKtbQOSugQAkFDCUlLKpTGlQg5lDS1+Wbe4FL+5nV08BfB51xeRqsBPyjPNNJNRPnnoqrNI2rVve1gVVKUz0nokANAjGHkrDRdRbw/p6NriGQhKDS50bXGuZatibhVqPgUcWPc69i1aHKda9mxZ115QRiDiWhGP/Zs4ck3BJ8KvCdylWo/wlX+T/Qzr2a/yJUue7VrLizrpxA7KGkcgVJMeGwl+bCFO3L1NS5l6matc+lynWvXcmWChRYwFBS4OI5dAUUUEABBcokYCgpUzWdiwIKKKCAAgUWMJQUuHgOXQEFFFBAgTIJGEqKV831gYuAZYF5PU+9vbl406hrxKOBB3ue7ntYXT0V68vr9tR+xZ7ab1as4Q95tOEBivuGF+cCvwCmDLknv6iAAlELGEqiLk+fg1sTGAk8BEwErgHWK9406hrxNCD8gA7PpalSKLkVOAS4C6jKMztW6JnvhsAC4E5gL+C+ulaQX1ZAgSgFDCVRlqXmQYX6PQOMB7pr/laxG4YAdnJPGAuhrCqh5F3AacBHi12+1KMPZ4XCM4o2BuYDtwGfAJ5K3ZNfUECB6AUMJdGXaMAB7gjsX7EfVD8Hvtbz4LgqhZJPAZ8HRvQ85TNcwjun2Mu35tF/FTihJ3iH5xWdVPM3baiAAoUSMJQUqlyLDDZcxrke+BhQlZf+hBAWXtIYQskePZevqnKmJLyg8lxgk54nHId9ROEyxh+Lu4RrGvnywA+BzwJdwG+APXue9lxTBzZSQIHiCBhKilOr3iNdBrilZ8PfdcWcwpBGPR34HNAJLA0sBZxSkd+cwysFjgB26JEL874f+MGQJIvzpZ17guikXvMOr5+4pDhTcKQKKFCrgKGkVql42rUC1wLhMsZ58Qyr4SOp2pmS8C6osME1vAtqDnATcHDPP2s4fgMP2A7MAN7bc/nmzXmHDa9+FFCgZAKGkuIVNPymfPVip6/DD6qXijeVukZctVASsMIGz3C2KHyuBI6rS7A4Xw4bm8Olu7CZ+6fA1OIM3ZEqoEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AT+HziZ3m5Hl1FsAAAAAElFTkSuQmCC\" width=\"570.3895891961337\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib nbagg\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.animation as animation\n",
    "\n",
    "n_epoch = 300          # epoch size\n",
    "a, b = 1, 1             # initial parameters\n",
    "epsilon = 0.0001         # learning rate\n",
    "\n",
    "fig = plt.figure()\n",
    "imgs = []\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    data_idx = list(range(N))\n",
    "    random.shuffle(data_idx)\n",
    "    \n",
    "    for j in data_idx[:10]:\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "\n",
    "    if i<80 and i % 5 == 0:\n",
    "        x_min = np.min(X)\n",
    "        x_max = np.max(X)\n",
    "        y_min = a * x_min + b\n",
    "        y_max = a * x_max + b\n",
    "\n",
    "        img = plt.scatter(X, Y, label='original data')\n",
    "        img = plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "        imgs.append(img)\n",
    "        \n",
    "ani = animation.ArtistAnimation(fig, imgs)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 如何使用批次更新的方法？\n",
    "\n",
    "如果有一些数据包含比较大的错误（异常数据），因此每次更新仅仅使用一个数据会导致不精确，同时每次仅仅使用一个数据来计算更新也导致计算效率比较低。\n",
    "\n",
    "\n",
    "* [梯度下降方法的几种形式](https://blog.csdn.net/u010402786/article/details/51188876)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 如何拟合多项式函数？\n",
    "\n",
    "需要设计一个弹道导弹防御系统，通过观测导弹的飞行路径，预测未来导弹的飞行轨迹，从而完成摧毁的任务。按照物理学，可以得知模型为:\n",
    "$$\n",
    "y = at^2 + bt + c\n",
    "$$\n",
    "我们需要求解三个模型参数$a, b, c$。\n",
    "\n",
    "损失函数的定义为：\n",
    "$$\n",
    "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEPCAYAAABoekJnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd41fXZx/H3zVKR6QAsVEQRDCpW2uKgSgRFqRW0g4eq\niILWAYriAvUpqWgF666jpVXER1uLVgsoIDOiVsU6URBTLAgooVUEEQfjfv74/mIONECSM35nfF7X\nda6c8zsjN7k0n3y3uTsiIiK1VSfuAkREJLcpSEREJCkKEhERSYqCREREkqIgERGRpChIREQkKRkJ\nEjO738zKzeythGvNzWyGmS02s2fMrGnCcyPNrMzMFplZr4TrXczsLTN7z8zuyETtIiKyY5lqkYwH\nTtzm2ghglrt3BOYAIwHMrBPQDygCegP3mplF77kPGOzuHYAOZrbtZ4qISIZlJEjc/XlgzTaX+wIT\novsTgFOj+32AR919k7svBcqArmbWCmjs7q9Er3so4T0iIhKTOMdIWrh7OYC7rwJaRNdbA8sTXrcy\nutYaWJFwfUV0TUREYpRNg+3aq0VEJAfVi/F7l5tZS3cvj7qtVkfXVwLfTnhdm+ja9q5XycwUTCIi\nteDutvNXVcpki8SiW4XJwNnR/YHApITr/c2sgZm1A9oD86Pur7Vm1jUafD8r4T1Vcnfd3Bk1alTs\nNWTLTT8L/Sz0s9jxrTYy0iIxsz8BxcCeZvYBMAoYAzxmZoOAZYSZWrj7QjObCCwENgIXeeW/bgjw\nILArMNXdp2eifhER2b6MBIm7n76dp47fzutvAm6q4vqrwKEpLE1ERJKUTYPtkibFxcVxl5A19LOo\npJ9FJf0skmO17RPLdmbm+fpvExFJFzPDs3iwXURE8pCCREREkqIgERGRpChIREQkKQoSERFJioJE\nRESSoiApAO+/D/Pmwbp1cVciIvkozk0bJc2WLYPRo+Fvf4P27WHBAmjdGr77XejSpfLWvPn2P2PL\nFlizBlav3vr22Wfhvd26we6777yWDz6AOXPC9+rbN3X/RhGJn4IkD61cCb/+NTz6KFxwAbz3Huyx\nB2zaBO++C6+9Bq++CpMnwxtvwFdfQZ06ULfu1l/r1IG1a6FxY2jRYutbw4YhpF5/HQ47DI47LtyO\nOio8t2oVzJ0bbnPmhNbQccfB88+Hzz3llLh/SiKSKlrZnkdWr4YxY+DBB2HQILj6ath77x2/Z8uW\nEDCbN4f7235t2hQaNNj++zdsgL//vTI03noLWraETz6B7t2hR49wO/hgMIOXXgohMns2dO6c0n++\niKRAbVa2K0hi5B5aD2VlW98+/BCOOAJ694bi4vAX/vasXg0zZsC0aTB9Opx+OlxzDeyzT8b+GVtZ\nvz50qR10UGjZVOXPf4aRI+Hll0PoiEj2UJAkyOYgmTULrr8+dC81bgwdOsCBB1beWraEF16AqVND\n19HRR4dQ6d0bDjgg/AKePj2ER1lZ+Iv/pJPgRz+Cb30r7n9d9ZSUwDPPhFbMrrvGXY2IVFCQJMjG\nIHn1VRgxApYuhRtuCMHQpMmO37N2bQieadPCbc2aEDa9e4fwOProHXc9ZSt3+PnPw3jJI4+Ebq9t\nbdoEEyaE0N1/f7jzTnWHiaSbgiRBNgVJWRlcdx089xz88pcweDDUr1/zz3EPwdKsWeprjMMXX4Su\nu5NPDj+XCu7w5JNw7bWhdXbjjfDmm6EV85OfhEH+vfaKq2qR/Kbdf7NExS/8hQvhwgvDTKbDDguB\ncsEFtQsRCH+150uIAOy2W5iafP/9MHFiuDZnDhx5ZAiL228PXV/dusFFF4UZZ/XrQ1FRaJ1s3Bhv\n/SISqEWSBHeYNCmMZZSXhymv5eXhVrdu+Gv6tNPCwPKee6a1lJz2xhtwwglw6KFhvcno0fA//xO6\nvaryzjtw2WWwfHkImxNPrLprTERqTl1bCdIdJP/6FwwdGr4OGRIW+rVsWXmrziI9qTRrFixZAuec\nU70xH3d46im46qowbnTssWG68bHHhqnG2wshEdkxBUmCdAXJ11/DLbfAbbfBFVfA8OG5OdidT5Yt\ng2efDdvAPPtsWMNyzDGV4y8HHhh3hSK5Q0GSIB1BMndu6Ktv3x5++1vYb7+UfrykyMqVYWLDnDkw\nZUrltix9+0LXrmqtiOyIgiRBTYLkww/DoO8LL8Auu4RuqUaNwteK+/Pmhdudd4ZfSOqTzw1btsAr\nr4SxrEmTQmvllFPC1OPjjou7OpHsoyBJsLMgWbIkTDH9619h8eLQBdKjR/jF8/nn4bZ+feX9Nm3g\n8stDqEju+uc/Q6Dcc08YpL/ttjB7TESCnAwSM7sMGAxsARYA5wC7A38B2gJLgX7uvjZ6/UhgELAJ\nGObuM7bzuf8VJJ9+Gn6BPPZYmGHVty/8+MfhL1ONcxSWtWvh/PPDFO2//CVMKRaRHFxHYmbfAi4G\nurh7Z8JuxD8HRgCz3L0jMAcYGb2+E9APKAJ6A/ea7byT6euv4a67oGPH8Bfp3XeHfvTf/z78VaoQ\nKTxNm4Y9vy65JMz0euCBMBNMRGouG4Yd6wK7m1k9YDdgJdAXmBA9PwE4NbrfB3jU3Te5+1KgDOi6\nvQ92h8cfD9NBp00LU0zHj4cf/GD7GwpK4TCDc8+F0tLQxXXmmeGcFRGpmVjPI3H3D83sVuADYAMw\nw91nmVlLdy+PXrPKzFpEb2kNvJjwESuja1Xq1i1sc37vvWHBm0hVDj4Y5s+HSy8Nh3WNHRtmejVo\nEFbSN2hQeX/ffcOEDBGpFGuQmFkzQuujLbAWeMzMzgC27WSoVadDixYldO4cZmPVr19McXFxUvVK\n/mrYEMaNC+Mlv/td6A79+uuwDUvF1w0bQsBMn67t7yV/lJaWUlpamtRnxDrYbmY/BU509/OixwOA\nI4EeQLG7l5tZK2CuuxeZ2QjA3X1s9PrpwCh3f7mKz86aTRslP7jDr34VdiueOVPriCQ/5dxgO6FL\n60gz2zUaNO8JLAQmA2dHrxkITIruTwb6m1kDM2sHtAfmZ7ZkKVRmYQfiYcPCyvm3367e+9av10C+\n5LdYg8Td5wOPA68DbwIGjAPGAieY2WJCuIyJXr8QmEgIm6nARWp2SKYNHRrGUY4/Hl58cfuvW7Qo\nDOA3axZmh23ZkrkaRTIp9nUk6aKuLUm3adPgrLPg4YfDNPIKb74ZzlB59tnQehkwAM44A9q2DdOM\na3uMgEgm5OSCxHRRkEgmvPBCWNR6113hGOTRo8OWLJdfHhY8VuyEsGED/OxnYdr5X/6i1fSSvRQk\nCRQkkikLFoRjj+vUgauvDidgVhUUGzfCwIFhb7fJk3d+zLJIHBQkCRQkkkkbNkC9ejvfJWHLljDG\nMn9+mEasI4Ml2+TirC2RvNCwYfW22qlTp3LDyGOOgRUr0l+bSLrFuiBRpBCZhcH45s1DmMyfD3vv\nHXdVIrWnri2RGF19dTiDfsoUnXEj2UFdWyI5ZvRoWL06nLgpkqvUIhGJ2ZIlcOSRYXfqww6Luxop\ndGqRiOSgAw6A22+H/v3DaZwiuUYtEpEsMWBAWH8yblzclUghU4tEJIfdcw/MmRMOYxPJJWqRiGSR\n+fPhlFPCNiv77ht3NVKItLI9gYJEctXYsfDUUzB3blgtD7BqVdjXq+L2ySfhyOhjj4Xu3aFdO00f\nltRQkCRQkEiu2rIlrHxv2TKEwwsvwKefwlFHheOju3WDPfaA55+HefPCLsN16oRQOfZYOO00neAo\ntacgSaAgkVz20UcwZkw4T75bNygqCmFRFXd4//0QKrNnw3PPhS3uO3XKbM2SHxQkCRQkUqgefhiu\nuAKeeAKOPjruaiTXaNaWiHDmmfDgg9C3bxhrEUk3BYlIHjrppBAi550H48fHXY3kO+3+K5KnjjgC\nSkvDwH15edggUjO7JB00RiKS51auDC2Unj3httu2P2gvAhojEZEqtG4dZnK9+mro6tqyJe6KJN8o\nSEQKQLNmYUpwWRlccIHCRFJLQSJSIBo1gqefhoULYciQsP5EJBViDxIza2pmj5nZIjN7x8yOMLPm\nZjbDzBab2TNm1jTh9SPNrCx6fa84axfJNY0bw9Sp8OabcPHFChNJjdiDBLgTmOruRcBhwLvACGCW\nu3cE5gAjAcysE9APKAJ6A/eaaR6KSE00aRK6uV55BS69VGEiyYs1SMysCXCMu48HcPdN7r4W6AtM\niF42ATg1ut8HeDR63VKgDOia2apFcl/TpvDMM/D3v8PllytMJDlxt0jaAf8xs/Fm9pqZjTOzhkBL\ndy8HcPdVQIvo9a2B5QnvXxldE5EaatYMZswImz5edZXCRGov7gWJ9YAuwBB3/4eZ3U7o1tr2P+la\n/SdeUlLyzf3i4mKKi4trV6VInmreHGbODGtMrrwSfvMbLVosNKWlpZSWlib1GbEuSDSzlsCL7r5/\n9PgHhCA5ACh293IzawXMdfciMxsBuLuPjV4/HRjl7i9X8dlakChSTZ98Ar16hZ2G77hDYVLIcm5B\nYtR9tdzMOkSXegLvAJOBs6NrA4FJ0f3JQH8za2Bm7YD2wPzMVSySn/bYA2bNCic0DhmidSZSM7Fv\nkWJmhwF/BOoD7wPnAHWBicC3gWVAP3f/NHr9SGAwsBEY5u4ztvO5apGI1NC6dXDyydCxI4wbp+1U\nCpHOI0mgIBGpnfXrw7nx++4LDzwAdevGXZFkUs51bYlI9qlYAf/hhzBgAGzaFHdFku3UIhGRKn35\nJfz4x+H+d78LGzb89611a7j3XqgX9/xPSRl1bSVQkIgk76uv4M47w9eGDf/7dscdcOSRMHp03JVK\nqihIEihIRNKvvBy6dIGHHgprUST3aYxERDKqZcsQImedFUJFCpOCRESS0rMnDBoUBua1/qQwKUhE\nJGmjRoXB+bFj465E4qAxEhFJiRUr4Hvfg7/+NWy1IrlJYyQiEps2beAPf4DTTw97d0nhUItERFJq\n+HBYsgT+9jdt/piL1CIRkdiNGRNWxd9+e9yVSKZoPaqIpFSDBjBxInTvHo71PffcuCuSdFOQiEjK\ntWsHs2dDjx5hB+FBg+KuSNJJQSIiaXHggSFMevYMYyXnnBN3RZIuChIRSZsOHcKBWRVhcvbZcVck\n6aAgEZG06thx65bJwIFxVySppiARkbTr2LGyZVKnTthORfKHgkREMuKgg0KYHH982E7lvPPirkhS\nRQsSRSSjFi+Gvn1D6+T228N0YckeWpAoIlmvY0d4+eWwN1ePHrBqVdwVSbIUJCKScU2bwpNPQq9e\nYaPHl16KuyJJhrq2RCRWU6bA4MFw440aN8kGOmo3gYJEJHe89x6ceioccwzcdRfsskvcFRWunB0j\nMbM6ZvaamU2OHjc3sxlmttjMnjGzpgmvHWlmZWa2yMx6xVe1iKRKhw5h3GTZMvjf/427GqmprAgS\nYBiwMOHxCGCWu3cE5gAjAcysE9APKAJ6A/eaaaNqkXzQuDE8+CCMHw+LFsVdjdRE7EFiZm2AHwJ/\nTLjcF5gQ3Z8AnBrd7wM86u6b3H0pUAZ0zVCpIpJmrVrBddfBxReDeqZzR+xBAtwOXAkk/mfT0t3L\nAdx9FdAiut4aWJ7wupXRNRHJE0OGwOrV8PjjcVci1RXrynYzOxkod/c3zKx4By+t1d8mJSUl39wv\nLi6muHhH30JEskG9enD33XDGGdC7NzRqFHdF+a20tJTS0tKkPiPWWVtm9mvgTGATsBvQGHgS+B5Q\n7O7lZtYKmOvuRWY2AnB3Hxu9fzowyt1fruKzNWtLJIcNGACtW4cTFyVzcnr6r5l1By539z5mdjPw\nsbuPNbOrgebuPiIabH8EOILQpTUTOLCqxFCQiOS2jz6Czp3h+efDanjJjLRN/zWz/6vOtRQaA5xg\nZouBntFj3H0hMJEww2sqcJHSQiQ/7bMPXHONBt5zQbVaJGb2mrt3SXhcF1jg7p3SWVwy1CIRyX0b\nN8Lhh0NJCfz0p3FXUxhS3iKJFv99BnQ2s3XR7TNgNTApiVpFRHaqfn245x4YPhw+/zzuamR7qtsi\nucndR2agnpRRi0Qkf5xxBrRtC7/+ddXPu4fTFyV5aR1sN7PWQFsSpgy7+7waVZhBChKR/PHhh2Hg\n/dvfhi++CLcvv6y8X79+2PRx2LBwAqPUXtqCxMzGAP0Jg9ybo8vu7n1qXGWGKEhE8ssHH8C//w27\n7fbftw8+CGfB77JL2GZl333jrjZ3pTNIFgOd3f2r2haXaQoSkcKyeTPcfDPcdhvcemtYh6LurppL\nZ5BMA37m7utrW1ymKUhECtMbb4QQ6dABfvc72HvvuCvKLSkPEjP7LWF7ktbAYcBs4JtWibtfUrtS\n009BIlK4vvwybEf/yCNw//1hqxWpnnQEycAdvdndJ+zo+TgpSETk2WfhtNNgyRJo3jzuanJDTm+R\nkmoKEhEBOOssOOQQuOqquCvJDekcI1nAf+/Auxb4B3CDu39ck2+aCQoSEQF4/XXo0wfefz9ME5Yd\nS+dRu9OAp4EzotsUQoisAh6syTcUEcmkww+H9u11vkk61WqvrcRrZrbA3Q9NW4W1pBaJiFSYMgWu\nvx7mz9eU4J1JZ4ukrpl9c6StmX0fqBs93FSTbygikmknnwxr14Yt6SX1qtsi+T7wANAIMGAdcC7w\nDnCyu09MZ5G1oRaJiCS67z6YMQOefDLuSrJb2mdtmVlTAHdfW8PaMk5BIiKJPv8c9tsPXnoJDjgg\n7mqyVzrWkZzp7g+b2fCqnnf322pYY8YoSERkW9dcA+vXw113xV1J9krHGMnu0dfG27mJiOSMoUPh\n4YdhzZq4K8kvWpAoIgVlwAA49FAtUNyedJ7Z3sHMZpvZ29HjzmZ2XW2KFBGJ02WXwW9/G47xldSo\n7vTfPwAjgY0A7v4W4XwSEZGc0qWLFiimWnWDpKG7z9/mmtaPiEhOGj48nFui3u/UqG6Q/MfMDiDa\nb8vMfgp8lLaqRETSSAsUU6u6CxL3B8YBRwNrgH8BZ7j7svSWV3sabBeRHbnvPpg0CaZN07YpidK5\nRcpKYDxwI/AoMBPY4Vkl1WFmbcxsjpm9Y2YLzOyS6HpzM5thZovN7JmKhZDRcyPNrMzMFplZr2Rr\nEJHCNHgwLF+ule6pUN0WyXTgU+A1YHPFdXe/NalvbtYKaOXub5hZI+BVoC9wDvCxu99sZlcDzd19\nhJl1Ah4Bvg+0AWYBB1bV9FCLRER25tlnw3TghQuhUaO4q8kO6TyP5G13P6TWlVW3GLO/AXdHt+7u\nXh6FTam7H2RmIwB397HR66cBJe7+chWfpSARkZ0aODCc637LLXFXkh3S2bX1dzNL61bxZrYf8B3g\nJaClu5cDuPsqoEX0stbA8oS3rYyuiYjUym9+Aw89BAsWxF1J7qq3oycTTkasB5xjZu8DXxF2AHZ3\n75yKIqJurceBYe6+3sy2bUrUqmlRUlLyzf3i4mKKi4trW6KI5KkWLcJZJRdeCPPmQZ3q/nmdJ0pL\nSyktLU3qM3a2aWPbHb05FbO2zKwe8BQwzd3vjK4tAooTurbmuntRFV1b04FR6toSkWRs2QJHHQXn\nnw+DBsVdTbzSvo18OpjZQ8B/3H14wrWxwCfuPnY7g+1HELq0ZqLBdhFJgddfh5NOCgPve+4ZdzXx\nybkgMbNuwDygogvNgWuA+cBE4NvAMqCfu38avWckMJiwXcswd5+xnc9WkIhIjQwbFs4t+eMf464k\nPjkXJOmkIBGRmlq3DoqK4LHH4Oij464mHumctSUikveaNIFbbw0D75u0m2C1qUUiIpLAHU48Eb74\nIpxbst9+4da2bfjaokV+b6mirq0EChIRqa1168JU4KVLw23Zssr7n38OJSX5ezBWbYJkh+tIREQK\nUZMm8KMfVf3cihVw/PHw9ddwnY73AxQkIiI10qYNlJZCjx6weTOMGhV3RfFTkIiI1FCrVjB3LvTs\nGcLkV7/K73GTnVGQiIjUQsuWMGdO6ObatAluvLFww0RBIiJSSy1abB0mY8cWZphoHYmISBL22gtm\nz4ZZs+CKKwrzHHgFiYhIkvbcMwTJtGnh+N5CoyAREUmBPfYI4yQ33FB4rRIFiYhIivTtC19+Cc88\nE3clmaUgERFJkTp14NprYfTowmqVKEhERFKoXz/497/h2WfjriRzFCQiIilUty6MHBnGSgqFgkRE\nJMXOPBP++U948cW4K8kMBYmISIrVrw9XXx1mcRUCbSMvIpIGX34JBxwATz0Fhx8edzXVpxMSRUSy\nxK67hpXuhdAqUYtERCRNPv8c9t8/7Md18MFxV1M9apGIiGSR3XeHSy+Fm26Ku5L0UotERCSN1q0L\nYyUvvgjt28ddzc6pRSIikmWaNIEhQ2DMmLgrSZ+cDBIzO8nM3jWz98zs6rjrERHZkUsugSlT4Ikn\n4q4kPXLuYCszqwPcDfQEPgReMbNJ7v5uvJWJiFRtjz1g+nT44Q/h66+hf/+4K0qtnAsSoCtQ5u7L\nAMzsUaAvoCARkax1+OEwcyb06gVffQUDB8ZdUerkYpC0BpYnPF5BCBcRkax2yCFhKvAJJ4Qw+cUv\n4q4oNXIxSKqtpKTkm/vFxcUUFxfHVouICMBBB8HcudCzZ+jmGjo03npKS0spLS1N6jNybvqvmR0J\nlLj7SdHjEYC7+9htXqfpvyKStZYuDWFy0UVw+eVxV1OpNtN/c7FF8grQ3szaAh8B/YGfx1uSiEjN\n7LdfOLOkRw/44otwIJbV6Nd39si5IHH3zWY2FJhBmL58v7svirksEZEaa9MmhEmvXvDJJ3DLLeGU\nxVyTc11b1aWuLRHJFWvWwCmnQLt28MADYRv6uGhlu4hIDmreHGbMgE8/hT59wmaPuURBIiKSBRo2\nhCefhFatwiD8xx/HXVH1KUhERLJEvXqha6t7dzjmGFi+fOfvyQY5N9guIpLPzGDsWNh7b/jBD8LW\nKkVFcVe1YwoSEZEsdMUVIUx69gxbq2TzwVgKEhGRLDVwYJjBdcIJMGsWdOoUd0VVU5CIiGSx00+H\nLVvg+ONh9uzs7OZSkIiIZLkzz9w6TA46KO6KtqYgERHJAWedBe6VYdKxY9wVVVKQiIjkiIEDQ8uk\nZ8+wHX2HDnFXFChIRERyyDnnVIbJ3LnQvn3cFSlIRERyzuDBsHlzmM313HNh88c4KUhERHLQL34B\na9eGnYPnzYO99oqvFu3+KyKSw0aMCOMls2dD48bJf15tdv9VkIiI5DB3OP98WLIEnn4adt01uc9T\nkCRQkIhIodi8Gfr3D18nTgybP9aWziMRESlAdevCww/D+vWhdZLpv6EVJCIieWCXXeCJJ+Cdd+Cq\nqzIbJgoSEZE80agRTJ0abn/6U+a+r8ZIRETyzLx5cPbZsHhxzc9/1xiJiIhw7LFhxfv48Zn5fmqR\niIjkofnz4Sc/gbKymk0JVotEREQA6NoVunSB3/8+/d8rtiAxs5vNbJGZvWFmfzWzJgnPjTSzsuj5\nXgnXu5jZW2b2npndEU/lIiK54frrYcwY+Pzz9H6fOFskM4CD3f07QBkwEsDMOgH9gCKgN3CvmVU0\ns+4DBrt7B6CDmZ2Y+bJFRHLDYYeF8ZK7707v94ktSNx9lrtviR6+BFTsX9kHeNTdN7n7UkLIdDWz\nVkBjd38let1DwKmZrFlEJNeUlMCtt4YNHtMlW8ZIBgFTo/utgeUJz62MrrUGViRcXxFdExGR7Sgq\ngt694Y40DgakdRt5M5sJtEy8BDhwrbtPiV5zLbDR3f+c6u9fUlLyzf3i4mKKi4tT/S1ERLLeqFFh\n8H3oUNhzz62fKy0tpbS0NKnPj3X6r5mdDZwH9HD3r6JrIwB397HR4+nAKGAZMNfdi6Lr/YHu7n7h\ndj5b039FRCIXXADNmoXB9x3Jqem/ZnYScCXQpyJEIpOB/mbWwMzaAe2B+e6+ClhrZl2jwfezgEkZ\nL1xEJAdddx384Q+walXqPzu2FomZlQENgI+jSy+5+0XRcyOBwcBGYJi7z4iufxd4ENgVmOruw3bw\n+WqRiIgkuPTS8HVH4yU6jySBgkREZGurVkGnTvDaa7DfflW/Jqe6tkREJLNatYLhwytbJqmiIBER\nKSBXXgmLFsGUKan7THVtiYgUmFmz4NxzYeFCaNhw6+fUtSUiIjt1/PFw1FFwww2p+Ty1SERECtBH\nH0HnzuEQrKKiyutqkYiISLXssw/88pdw0UXJn++uIBERKVAXXhg2c3zkkeQ+R11bIiIFbP58OPXU\nMPDerJkWJG5FQSIiUj0XXgh16sA99yhItqIgERGpnjVrwor3yZOha1cNtouISA01bw5jx4aWSW2k\n9TwSERHJDQMGwGefwauv1vy96toSEZFvaB2JiIhknIJERESSoiAREZGkKEhERCQpChIREUmKgkRE\nRJKiIBERkaQoSEREJCkKEhERSUrsQWJml5vZFjPbI+HaSDMrM7NFZtYr4XoXM3vLzN4zszviqVhE\nRBLFGiRm1gY4AViWcK0I6AcUAb2Be82sYrn+fcBgd+8AdDCzEzNcck4qLS2Nu4SsoZ9FJf0sKuln\nkZy4WyS3A1duc60v8Ki7b3L3pUAZ0NXMWgGN3f2V6HUPAadmrNIcpv9JKulnUUk/i0r6WSQntiAx\nsz7AcndfsM1TrYHlCY9XRtdaAysSrq+IromISIzSuo28mc0EWiZeAhy4DriG0K0lIiI5LJZt5M3s\nEGAWsIEQLm0ILY+uwCAAdx8TvXY6MIowjjLX3Yui6/2B7u5e5VEsZqY95EVEaiEnj9o1s38BXdx9\njZl1Ah4BjiB0Xc0EDnR3N7OXgEuAV4CngbvcfXpcdYuISPackOiElgnuvtDMJgILgY3ARQknVA0B\nHgR2BaYqRERE4pcVLRIREcldcU//TTkzO8nM3o0WLV4ddz1xMbM2ZjbHzN4xswVmdkncNcXNzOqY\n2WtmNjnuWuJkZk3N7LFowe87ZnZE3DXFxcwuM7O3o4XOj5hZg7hryhQzu9/Mys3srYRrzc1shpkt\nNrNnzKxpdT4rr4LEzOoAdwMnAgcDPzezg+KtKjabgOHufjBwFDCkgH8WFYYRukwL3Z2EruEi4DBg\nUcz1xMJj6EAmAAADf0lEQVTMvgVcTBif7Uzo6u8fb1UZNZ7wuzLRCGCWu3cE5gAjq/NBeRUkhFlf\nZe6+zN03Ao8SFjgWHHdf5e5vRPfXE35ZFOy6m2gXhR8Cf4y7ljiZWRPgGHcfDxAt/F0Xc1lxqgvs\nbmb1gIbAhzHXkzHu/jywZpvLfYEJ0f0JVHPRd74FybaLGbVoETCz/YDvAC/HW0msKnZRKPRBwXbA\nf8xsfNTNN87Mdou7qDi4+4fArcAHhOUHn7r7rHiril0Ldy+H8Mco0KI6b8q3IJFtmFkj4HFgWNQy\nKThmdjJQHrXQLLoVqnpAF+Aed+9CWMs1It6S4mFmzQh/gbcFvgU0MrPT460q61TrD698C5KVwL4J\njysWOhakqLn+OPB/7j4p7npi1A3oY2bvA38GjjOzh2KuKS4rCFsT/SN6/DghWArR8cD77v6Ju28G\nngCOjrmmuJWbWUuAaH/D1dV5U74FyStAezNrG82+6A8U8gydB4CF7n5n3IXEyd2vcfd93X1/wn8T\nc9z9rLjrikPUbbHczDpEl3pSuBMQPgCONLNdox3Ge1J4Ew+2baFPBs6O7g8EqvUHaLYsSEwJd99s\nZkOBGYSQvN/dC+0/DADMrBtwBrDAzF4nNFGv0SJOIewO8YiZ1QfeB86JuZ5YuPt8M3sceJ2w+Pl1\nYFy8VWWOmf0JKAb2NLMPCFtRjQEeM7NBhG2p+lXrs7QgUUREkpFvXVsiIpJhChIREUmKgkRERJKi\nIBERkaQoSEREJCkKEhERSYqCRCQFoq3ZL4zu7xMdziZSELSORCQFoo0xp7j7oTGXIpJxebWyXSRG\nNwH7m9lrwD+BInc/1MwGErbi3h1oT9httgEwAPgS+KG7f2pm+wP3AHsRNlI8z93fi+HfIVJj6toS\nSY0RwJJoR91tt6s/mBAmXYEbgfXR614CKvb8GgcMdffvR++/L1OFiyRLLRKR9Jvr7huADWb2KfBU\ndH0BcKiZ7U7YdfaxaPNAgPox1ClSKwoSkfT7KuG+JzzeQvh/sA6wJmqliOQcdW2JpMZnQOPofo0O\nznL3z4B/mdlPK66ZWecU1iaSVgoSkRRw90+AF8zsLeBmtn+y3PaunwkMNrM3zOxtoE8ayhRJC03/\nFRGRpKhFIiIiSVGQiIhIUhQkIiKSFAWJiIgkRUEiIiJJUZCIiEhSFCQiIpIUBYmIiCTl/wExAjxr\nuL00TgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0e276b03c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "pa = -20\n",
    "pb = 90\n",
    "pc = 800\n",
    "\n",
    "t = np.linspace(0, 10) \n",
    "y = pa*t**2 + pb*t + pc + np.random.randn(np.size(t))*15\n",
    "\n",
    "\n",
    "plt.plot(t, y)\n",
    "plt.xlabel(\"time\")\n",
    "plt.ylabel(\"height\")\n",
    "plt.savefig(\"fig-res-missle_taj.pdf\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 如何得到更新项?\n",
    "\n",
    "$$\n",
    "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n",
    "$$\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial L}{\\partial a} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i^2 \\\\\n",
    "\\frac{\\partial L}{\\partial b} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i \\\\\n",
    "\\frac{\\partial L}{\\partial c} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c)\n",
    "\\end{eqnarray}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 程序"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch    0: loss = 2.56931e+07, a =   -3.84417, b =    7.00335, c =    4.33039\n",
      "epoch  500: loss = 1.05229e+06, a =   -30.8718, b =    238.361, c =    426.765\n",
      "epoch 1000: loss =     349168, a =   -25.9534, b =    172.499, c =    590.056\n",
      "epoch 1500: loss =     121559, a =   -23.1517, b =    134.984, c =    683.042\n",
      "epoch 2000: loss =    47995.8, a =   -21.5563, b =     113.62, c =    735.995\n",
      "epoch 2500: loss =    24281.5, a =   -20.6477, b =    101.455, c =     766.15\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEACAYAAABRQBpkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4TGf7wPHvk1pDkklKQm0Jat/KW6qxBLWmsVSpVi2l\n2krbV3Wz9idVoa2qly5aRYtWFW1tURRNaWipUvseEVEhZJBErPfvjxnTBCGymCz357rmMueZc87c\nMyZzz/Pc5zzHiAhKKaXyNxdnB6CUUsr5NBkopZTSZKCUUkqTgVJKKTQZKKWUQpOBUkop7iAZGGOm\nG2NijTHbUrR5GmNWGmP2GmNWGGM8Ujw2zBiz3xiz2xjTJkV7fWPMNmPMPmPM/7LupSillMqoO+kZ\nfAm0va5tKLBKRKoCa4BhAMaYGkB3oDrQHvjUGGPs20wB+otIFaCKMeb6fSqllLrL0p0MROQ3IP66\n5k7ATPv9mUBn+/2OwFwRuSwih4H9QENjTCnATUQ22deblWIbpZRSTpLZmoG3iMQCiMhxwNveXgaI\nTrFejL2tDHA0RftRe5tSSiknyuoCss5toZRSuVCBTG4fa4zxEZFY+xDQCXt7DFAuxXpl7W1ptd+U\nMUaTi1JKZYCImNuv9a877RkY++2axUBf+/0+wKIU7T2MMYWMMX5AZWCjfSjpjDGmob2g3DvFNjcl\nInoTYdSoUU6PIafc9L3Q90Lfi1vfMiLdPQNjzBwgALjXGHMEGAW8C8w3xvQDorAdQYSI7DLGzAN2\nAZeAYPk3wheBr4AiwDIRWZ6hyJVSSmWZdCcDEXkqjYceSWP9ccC4m7RvBmqn93mVUkplPz0DOZcI\nCAhwdgg5hr4X/9L34l/6XmSOyej40t1gjJGcHJ9SSuVExhjkDgvImT2aSCmVjXx9fYmKinJ2GCqH\nqlChAocPH86SfWnPQKkczP4Lz9lhqBwqrc9HRnoGWjNQSimlyUAppZQmA6WUUmgyUErlQC1atGDG\njBnpWvfXX3+lXLlyt19R3ZImg9zi0CFYuxbOnnV2JEoBtiOdXF1dcXd357777uOZZ54hKSnJKbH8\ne7mUW5s5cyZNmzbN5mhyJz20NKeLioJ33oGFC6FyZdi+HcqUgQYNCKtbFP8HOmFp2Aw8PQGwJluJ\nOBIBgH95fyxFLHD1KsTHYz16gIiDvxB4pRKcOAHnzkH9+uDvD8WK3T6WI0dgzRrbc3XqlJ2vWuUC\nxhjCwsJo0aIFJ06coE2bNowbN4533nnH2aGlSUTSnTjyG+0Z5ABh+8KwJltTtVkP7SJscCBhj9XC\n6uMB+/bB77/DmTNY535FWPPS+B8vxIj5A7HeXw4KFcLqUZgRT5TE/+En8G/6FCOeKoX1Pi8oXBhr\nzUqMeL8d/h8vJmzZ/7Du+gvi4myJxscHa/NGhPxfU6wrFsG1X3fHj2P9ehphL7ezJaL//Ad++gmC\ng2HJEie8UyqnuXZYo7e3N23btmXr1q2Oxy5evMjrr79OhQoVKF26NMHBwVy4cAEAq9VKUFAQ3t7e\n3HvvvQQFBRETk+YExqkkJyfTt29fvLy8qFWrFps2bUr1+HvvvUflypVxd3enVq1aLFy4EIA9e/Yw\ncOBANmzYgJubG15eXgAsW7aM+vXr4+HhQYUKFXj77bcz/b7kSs6eXe82M+9JXrF071KJPx+fqi3+\nfLyjPXhpsO3x2FiJHzxQgrsUlvjXXpT46P3/Pmbf5vrlgUsGyoHju2XgwgESdyJKLseflsun4iQu\nep8MnN9XDh3fc+t9nIqR4M+C5PDwYAnu5yPxXq4iFStKvI+7BL/oK/ETx8nSlZ9IfNJpW+AbNoiU\nKCHxm9bJ0r1L79I7mD/l5L8BX19fWb16tYiIREdHS+3atWXw4MGOx1955RXp1KmTWK1WSUhIkI4d\nO8rw4cNFROTUqVPyww8/SHJysiQkJEj37t2lc+fOjm0DAgJk+vTpN33eIUOGSLNmzcRqtcrRo0el\nVq1aUq5cOcfjCxYskOPHj4uIyLx586RYsWKO5a+++kqaNm2aan+//vqr7NixQ0REtm/fLqVKlZJF\nixZl9u25K9L6fNjb7+j7Vk86yywRiImB/fth/37CDi3Hf/8FLNEnoVEjaN8e60P1WHFsHWuj1hLa\nKhRLEQvWZCsjVo+wLZ+9yIllC3ht23g6/naSGa3vpX3AsyQVLUhcUhzHzh0jIjqCkq4liToThaWw\nhQtXLpB0KYmkS0mcv3weAIO5oQt8Va4CUKxgMYoXKo5rQVdcC7pS6J5CxCbGUtu7NtFnomlXuR1l\n3ctStEBRlu5ZxPOlAvkx8U8mtJ2IZ1HP1PEWsWD9ehojlg4m9P3NRCTv/3dIyu7acFVglcC793+R\nB932pLOsGvLIwN+Zn58fp06dAiAhIYFWrVrx/fff4+7uDkDx4sXZvn07fn5+AGzYsIGePXty6NCh\nG/a1detWWrVq5dhfixYt6NWrF/369bth3UqVKvHZZ5/RunVrAL744gveeecdjhw5ctM4H3jgAUaP\nHk1QUBAzZ85k+vTprF27Ns3XNXjwYFxcXJgwYcIdvBvOkZUnnWnNIKNWrYLRo2HzZnBzgypV4P77\n8a9cmxEBfxBa620sG7dhnRDKCPdNvHOxCdVbPUivY4/RoHITFv01F9+zLrT/eSZHCiYR52oo6enO\n14GJtK74MNvOR1HClMCnmA81S9akpW9LBiwdwOIei6nsVdnxpX7p6iXGrB3Dm/5vMj5ivOPLGnB8\ngb/y0Cu8v/59Xm/8OgXvKehIIgdOH6DnDz0Z02IMghB1Joq4pDjOXU6i4/qXsRS28O3O7yjnXo7y\nHuXxKeZD69mteaLmE2wotIEPq/XH8kQf/H/6MXWiSJE4wvaFaaLITk7+sbRo0SJatGjBunXreOqp\np4iLi8Pd3Z2TJ0+SlJREgwYNHOtevXrV8cV1/vx5XnnlFVasWIHVakVESEhISNeY/rFjxyhbtqxj\nuUKFCqkenzVrFhMnTnRM05CYmEhcXFya+9u4cSNDhw5lx44dXLx4kYsXL9KtW7c7fStyvzvtStzN\nGzmxi/znn7K0Wz2Jr+4nMneuyJkzIvLvkM+Vq1dk87HN0mZ2G3nlp1fk/sn3S+2Paorr24XlvpGu\n0viFgkII8uaTJWTO8CD5beFkiT55UOIS4yR4abBExkemGsK5tu+bPXazIaNry7d6LD37vNZ+9MxR\n2X1yt6w4sEK+2PyFvLzsZSEEqfVJLSkWWkx8RhaR5m96S98f+4j/dH+Z/fds6bew37/7O3dSgse3\nkPjKZUUCAiR+07obXp9KW478G7BLOUwkIjJixAjHUM/Vq1elWLFicuzYsZtu+84770iLFi3kxIkT\nIiKydetWcXFxkStXrojIrYeJKlasKCtWrHAsT5061TFMFBUVJYULF5b169c7Hq9Xr55jXzNnzrxh\nmKhSpUoyadIkuXjxoojYhrd69eqV/jfCidL6fJCBYSKnf+HfMjgn/iHcMMa/b5/E9+gsSxt5Svwn\nEyR48QsSlxgnO0/slM82fSZ1p9QV/+n+4j7OXcpPLC+tZ7UWQpAJERNk49GNcib5jP2LdqBERv2d\nJV/qt6pDpLtGkWKfh+MP31ECOZ10WqJj98nP7avKRyEd5OnvnxZCkGKhxaTCxArS5cNG8s5jJeS7\nrtWl7+ftJXLyaAnuWkTiBz4jcvLkLWNUNrkpGZw8eVKKFSsm27ZtExHbl2r37t0dX/hHjx51fIm/\n+eab0qFDB0lOTpZTp05J586d050MhgwZIgEBARIfHy/R0dFSp04dRzLYtWuXFC1aVPbt2ydXrlyR\nGTNmSIECBRz7Wr58ufj5+Tm++EVEfHx8ZNasWSIi8scff4i3t7cmg5x2c+YfQnzSaQn+4VmJ37JB\n5IUXJL60pwx4+z+ycMu3EvJLiDT/srkUHF1Qyn9YXipNqiSj1oySlQdWysnEkzf91Z0dX+oZldY+\nR60ZdccJJP58vMixYxJfuawET2ojkfGRMvDzINn0SA35NrCCvPH549JqZitxH+suhCCdZrWXKa8H\nyPYqFjn1YagEL34hzeSjcnYy8PPzS5UMRESCg4Pl8ccfFxGR5ORkGT58uFSsWFE8PDykRo0a8tFH\nH4mIyLFjxyQgIECKFy8uVatWlalTp6ZKBi1atEgzGSQlJUnv3r3FYrFIzZo15YMPPkhVQB45cqR4\neXlJyZIl5bXXXkuVWC5evCiPPvqo43ERW8G5QoUK4u7uLkFBQfLyyy/ny2SQrwvIYfvC8C/3MJYV\nv8KyZRAbizXuKBEmmsDN54jyMLwQBPd5V2KJx3ESLiXyQOkHaFKuCU3KN6Gse1nqT61P5KBIfC2+\nADcWWu3LzXyb0bZS21w7dn6rsX//8v6M+O55QoevwlK1LtZ/Ihnx/P2EPjcXi6sX1mQrw1cPp2v1\nroxZN4bSxUuzMfI3Tlv/oeHJQpypUp5hge+y/OAKxrYai6WIRWsNdjprqbqVrCwgO/3X/61uZPOv\novg9WyV4YHmJr1NF5OOPJW7+TOn6aYAMW/CCNPmisRQfW1wenvawEILM3T5Xki4m/bttGmPu+XHo\nw/Gaf/5Z5LPPRC5cSFeP4tiZGJk/c4j07WXrNVhGFpQnxtaX6Yvflu3/3HooLb/I7r8Blbul9flA\newY3uukvzLMniJj8OoH/W8bBV5/hufv+pEhBV1ZHrqaiZ0XaV25Pm0ptqO1Tm9C1obzh/0aqI3XS\n+vWf8kgeZXO7X/jX3rs3KjzJqBVDaRh9ld/ObOdnn0TuxZUixdwZ+tDrrL24n3Gtxjl6Ivmlx6A9\nA3Ur2jO4Azf8Ml25WJ7u4y5j+1eV5lMaifs4d2k3u50Qgmw4siHt7dI5xq/S71bv8ZXoI/Lnl6Hy\n6mu1bL2GYS7S+61aMnvB/8mARc/mmx5DVvwNqLwrrc8H+bpncOyYbf6eiAgoXNg2107x4lCsGFZX\nFwZeWojP8bN8XWQ/FCtG5zrd6FytCw1KN2DM2jE3/PrXMevsl+5eQ+PXGPX9y9SKOs/KUxv53SuR\nEi7FefH+nuy1XGF8m/F5tkemPQN1K1nZM8jdyeDgQfjxR8LWTsN/43EsrYOgZUu4ehXruZMsTPiT\nExdOM4ftHCeB2ALJfPfoV3R94GnucblHh3tysFv938jBg8xcPJrBFxdT/GpB2lTrwFN1e+FiXGjh\n1yJPJXBNBupWctwwETAY2AFsA74BCgGewEpgL7AC8Eix/jBgP7AbaHOL/d7Y/4mPFxkzRqRuXREf\nH5HnnpP4pQskePHzEn8+XuIS4+TD9R9K6Q9Ki+VdiwxYPECW7F0iA5cO1GJvLpKe8yQio/6W/oP8\n5KNOpaXVpw+J+zh3qfZxNflh1w9y6cqlPDGEdNO/AaXs0vp84IzzDID7gENAIfvyd0Af4D3gTXvb\nEOBd+/0awBZsU2H4Agew91Busu9/X92FC7L0g+ckvmwJkb59RdatE7l8WeLPx8uiPYtkwc4FUmlS\nJSk+trhUmlRJvv77a0m+lHzbM3FV7nLD/2fSaQkeHyDx93lJzNQPZOyvoeL9vrd4j/eW+p/Xl7+O\n/SUiuTfxazJQt5ITk0GUvSdQAFgMPALsAXzs65QC9tjvDwWGpNj+J6BRGvsWuXpVZP58kcqVJT6w\nlQTPesLxR73t+DZ5cOqDUvbDstLg8wYy5tcxQggSGR/peFNy65eAurk0/z9//kSkZk2Rp56SyOjt\nQgjSb2E/KfF+CWk5s6V88ecX8vyS53PdjwJNBupWclQysD0v/wXOAbHAbHtb/HXrnLb/+xHwVIr2\nacBjaexXpHFj25DQypUiInIy8aS0m91Omn3ZTAq/U1ieXfSs/HXsrzSP+1f5SGKixD/XW4Kf9JDI\nb6dI8Oed5PjqxTJ34Rh55JNG4hXqIXX+V1VW7g7LNZ+R/JAM1q1bJ9WqVXN2GDcVHh4uZcuWTff6\nt5pGIztkZTLI9KylxhgL0AmoAJwB5htjegLXVzUyVAUL8fYmuUYV9v/wGffumc/yy8vxLubN5n82\nszt4N9VKVruh2BjaKlQLwfmQ1eUiIzoWJ/TMh1hmzCX0ahIjKgwgdFdpnki6SmShEkyoGEMbayD+\nPg1ZF7WODvd3YPmB5XrkWAb4+vpy4sQJChQogIhtttG+ffsyefLkW27n4uLCgQMHqFixIgBNmjRh\n9+7d2RLjM888Q7ly5Rg9enSG95FdV0bz8/Nj+vTptGzZMtP7Cg8PJzw8PFP7yIornT0CHBKR0yJy\nBfgReBiINcb4ABhjSgEn7OvHACmvXl3W3nZTrSe/weIyS1lVYRUXy11kdpfZNCrTiMhBkXy08SPH\nH23KL/5rCeHa5R9V/uD4HDzVD1auxLLqN0I/2UPEzDHw9994rvsT6d2LPfIiRbZu5/9WDKPKx1X4\n+/jfvL7ydcfV5q79uPAv7+/kV3RzN70yXrKVsH1hd3Uf1y57efbsWc6dO8fZs2dvmwiubaeyVkBA\nACEhIY5bhtxpV+L6G9AQ2A4UAQzwFfAitgLyEEm7gFwI8OM2BWTXUFcZvmq4xCbEajFYZdgNn51J\n78nA7sVk5c+fy5MLnhSPcR5S65NasubQmpt/ps6ds9Wv7jJuMgyQFX8HWbGP62ctTenAgQPSvHlz\n8fDwkJIlS0qPHj1ERKRZs2ZijJFixYqJm5ubzJs374ahGF9fXxk/frzUqVNHihcvLs8++6zExsZK\n+/btxc3NTVq3bi1Wq9Wxfrdu3aRUqVJisVikefPmsmvXLhGxTW1dsGBBKVy4sLi5uUnHjh1FxDZJ\nXteuXaVkyZJSsWJFmTx5smNf58+flz59+oinp6fUrFlTxo8fn2oSvOutXLlSqlWrJhaLRV566SVp\n3ry5Y5jo4MGD0rJlS7n33nulZMmS0rNnTzljn/K+V69e4uLiIq6uruLm5ibjx4+/6WvZuXPnLf8P\nbvb5SNHulJrBKGyHiW4DZgIFAS9gFbZDS1cClhTrD7MngdseWro3bq/jBWoxWGXUTT87s6bK0gct\nIuvXS8zZGHkp7CUhBOnwdQf5M+ZP23arpkh8r24i99wj8tJLIleu3NXPXFp/7FlRI8vsPm6VDJ58\n8kkZO3asiIhcuHBBIiIiHI8ZY+TQoUOO5fDw8FRfuL6+vtK4cWM5efKkHDt2TLy9vaVBgwby999/\ny4ULF6Rly5YyevRox/pffvmlJCYmysWLF2Xw4MFSr149x2N9+/aVt956y7F89epVadCggYwZM0Yu\nX74skZGRUqlSJVlpr0ne7pKaKcXFxYmbm5v88MMPcvnyZZk4cWKq6bIPHDggq1atkkuXLklcXJw0\nb9481WVBfX19Zc2aNan2eavXcjM5Lhlk1w3QX/4qey1bZrue89IFErw0WHbE7hD/6f5SelwJaf26\njyx60F0Gvt1I4vdvF2naVOJ7d0817XZ2S+uPXUQkMj5SCCFLbimPwEsvX19fcXNzE09PT7FYLOLp\n6SnTpk0TEZHevXvL888/L0ePHr1hO2OMHDx40LF8s2QwZ84cx3LXrl0lODjYsfzRRx9Jly5dbhpT\nfHy8GGPk7NmzInJjMvjjjz+kQoUKqbYZN26c9OvXT0RsF865lhhEUl8453qzZs2Sxo0bp2orW7Zs\nmgXkhQsXSv369VO9zrSS6c1ey81kZTLI8Ze91GKwylbt22OdP5sRHz9G6GMfYzlynqVL3RlaIIna\nD7Thzad34lr4Ik/+/gafzJ3ChDEdCJ1RA8sjhZ0atjXZyviI8UQOirzhcqd3so8Rq0fcMBXLnbh2\n2cvrjR8/npEjR9KwYUO8vLx49dVXeeaZZ9K9Xx8fH8f9okWL3rCckJAA2C6lOXz4cBYsWEBcXBzG\n2K4DHhcXh5ub2w37jYqKIiYmBi8vL8D2Y/jq1as0a9YMuP0lNVM6duwY5cqVS9WWcvnEiRMMGjSI\ndevWkZCQwJUrVxzPezN3+lqyWlYUkLOVFoNVdou47wqhw37G8sZb0KULlpYdePfzA/h2fZadL+1m\nZLORHD17lEpf1KJOnzfxKGqB9u3h7FmnxJvy6Dlfi6/jB9P1BeHs3gdwrQd/A29vb6ZOnUpMTAyf\nffYZwcHBHDp06I72nR7ffPMNS5YsYc2aNVitVg4fPpxyZOGGYnW5cuWoWLEip0+f5vTp08THx3Pm\nzBmWLFkCwH333Ud0dLRj/aioqDSfu3Tp0hw5ciRVW8pthw8fjouLCzt37sRqtfL111+ner+uj23O\nnDm3fC3ZLccnA7AlBD3ET2WXwCqBWBr4w/79tvmuXnoJi2dpAqsEco/LPbT0a0nTCk2ZFjSNkWtD\nqNt0J0vqubK0R32sMQdT7etOj8jJiKw4ei67j8BbsGABMTG2gwQtFgsuLi64uNi+bkqVKpVliSEh\nIYHChQvj6elJYmIiw4YNS/Ul6+Pjk+q5GjZsiJubG++//z7JyclcuXKFnTt38ueffwLQrVs3xo0b\nh9Vq5ejRo3z88cdpPndgYCC7du1i4cKFXLlyhUmTJnH8+HHH4+fOnaN48eK4ubkRExPD+PHjU21/\n/ftw7ty5W76W7JYrkoFSd4WrKxQqlKrp2i/osa3G0r9+f/a9tI8ybmUYen8U//dgAr3erkv8gR2p\n1s3uQ1IDqwTeMJxzpz+YsmIfAEFBQbi7uztuXbt2BWDTpk00atQId3d3OnfuzOTJk/H19QUgJCSE\n3r174+XlxYIFC27Y5/VfgLf6Quzduzfly5enTJky1KpVi4cffjjV4/3792fnzp14eXnx2GOP4eLi\nwtKlS9m6dSt+fn54e3szYMAAztp7eaNGjaJ8+fL4+fnRrl07evfuneZz33vvvcyfP58hQ4ZQokQJ\nDh48SJMmTRyPjxo1is2bN2OxWAgKCnK8N9cMHTqUd955By8vLz788EP69Olzy9eS3XL3rKVKZbO0\nptleF7WO5MvJjFwQzJmk00zu9Dm/xm/J8tqWzlqqbkWnsFYqh7hy9QqTR7bm1cK/0Lpiaya3n0y1\nEtWybP+aDNStZGUy0GEipTLh3MVzHGhchT3htUiI2o//DH/af9OevXF7U613N2oJSmWGJgOlMshx\nRE7rd6k6bSHLppylc8lmlHUrS93P6jJ01VASLibk+OktlAIdJlIqw26oJ3z9Ndb3RxPx9TjK+dzP\n4/Me58yFM9T2rs28bvPwKpr2MeZp0WEidStaM1Aqp+rVC4oWhalTOWw9jN8kP+p418G1kCsftf+I\n/9z3nzvanSYDdStaM1Aqp/rkE1izBut3Mx1nCPuX9+fp2k8T9G0QbWa3Yf+p/ak20XqCyhHudP6K\nu3kjH1zYQ+U98b+tkuCuRSV+3zbbsn1CuMPxhyV4abAUGVNExq4dKxcvX7ztbKEVKlQQbNcC0Zve\nbrhdP8/SNXDncxPpMJFSWSxsXxj+P2zCErYafvkFChSwXXfj76UEHi3KHxsW8HiB7yl6CWrdU5oZ\nVd/E0qI9+PmBzvWvsoDWDJTKKa5ehbZtwcfH9gUfEQFWKzRuDP7+RNavSMU/nsTbuNHltDfjFibg\nebkANGtmu3XpYttWqQzQmoFSOYWLC8yaBffeC02bwpIlEBcHYWFYXw3mgyvriBwUSVC97lxq0Zya\nL7vw5sQOxLd8GH77DRo2hF27tJ6g7hrtGSh1F11/ve5ry52qdmLwisEkXEpgcY/F1F21HevwVxkx\nqimhPafr9O3qjugwkVI5XFpzHUUciaBNpTaErg3l3Yh3GfzQYM7s387YMRuwfD4THn3UiVGr3EaT\ngVJ5wNqotTT/qjn1fOox5/4hVH96MIwdC3dwcRiVv2nNQKlczpps5bsd33HwvwdxL+xOk80v8v6U\np1k8YwjWcaMgxY8jrSeorKQ9A6VyiJvVE17+6WWirFEkJsZTZUs0U9yfxDLhE6wXz+rlYFWadJhI\nqVwsrXrCb1G/EXMuhuGrh1Hl+CVmJ7ZlYidvQluN1USgbkqTgVJ5WJQ1iqfmP8H6Y38QfvJRmk9e\nZDuEVanraM1AqTzMo4gHde+rz1uN3qSdVxifvN4MuXrV2WGpPCJLkoExxsMYM98Ys9sYs9MY08gY\n42mMWWmM2WuMWWGM8Uix/jBjzH77+m2yIgal8rKU12Ie3e49Vj21nP8r+gedRlbi221zsCZbb1hf\ni8vqTmRVz2ASsExEqgN1gT3AUGCViFQF1gDDAIwxNYDuQHWgPfCpudUVr5VSRByJSFUs9q/Whp0v\n7qTAaSuvfT+A3j/0diQEvZiOyohM1wyMMe7AFhGpdF37HqC5iMQaY0oB4SJSzRgzFNuMeu/Z1/sJ\nCBGRP26yb60ZKHUrViurezSid6MYSpSqyLxu85n8x2Q9yiifc1bNwA+IM8Z8aYz5yxgz1RjjCviI\nSCyAiBwHvO3rlwGiU2wfY29TSt0pi4VW3/7O9hWVKHMsgWqfVOPxGo9rIlB3rEAW7aM+8KKI/GmM\nmYhtiOj6n/QZ+okfEhLiuB8QEEBAQEDGolQqr/L0xOXHhfgOfZB3yjxMhzkdmNxuMgMaDHB2ZOou\nCQ8PJzw8PFP7yIphIh9gg4hUtC83wZYMKgEBKYaJfhGR6jcZJloOjNJhIqUyxnGyWv3XsQR1Y10z\nX4K8VxFUJYgu1bvQ0q/lTedCCqwS6MSoVXZyyjCRfSgo2hhTxd7UCtgJLAb62tv6AIvs9xcDPYwx\nhYwxfkBlYGNm41Aqv3IUl0v7wapVNN0Qw/ajQRw9G82I1SMYuHSgFpfVbWXJSWfGmLrANKAgcAh4\nBrgHmAeUA6KA7iJita8/DOgPXAIGicjKNParPQOl7tTZsxAYiFStwlfBD/P6qjep61OXGZ1mMD5i\nvBaX8wE9A1kpZZOQAEFBUL48O997jU7zHuNg/EH2vLiHqiWqOjs6lc00GSil/pWUBJ06YfXx4M1u\nFmIS/mHTsU2s6r2KOj51nB2dykaaDJRSqVitxxkxvBGh0VXwqNeIsZfDCS30O1+faMJjcSVtCaNM\nGfj0UyiQFQcXqpxA5yZSSqUScWIzoeM2YmnaGlOoMCNKdGGeZQB9Sv/OyOZXuTrwBYiMhLff1iks\n8jntGShSMT8QAAAZ/klEQVSVD+06sYuWs1pSv3R95jT9HzRvzohhDQntPVOLy3mADhMppdLtROIJ\nHpn1COcunKNxQT8+/WA3lt+3go+Ps0NTmaTJQCl1Rw5bD+M3yQ+vIl58ndiG9r+fguXL9ToJuZzW\nDJRS6WZNtjI+YjyRgyIJ8A2gn3s4H3gfQN5919mhKSfQZKBUPpTyesu+Fl+md5pOm0ptmN2wKK0i\nQzi+evEN62txOW/TYSKl8qG0rre8JnINs5ePZ/3xjfw6IIJqlR9KlTi0uJw7aM1AKZVpIsJbwx/i\nfwU3M+ep71lxcKUmglxGawZKqUwzxjDm7XV8uKMMnb7rTB2fOpoI8gFNBkqpG1ivJvF392YsXlmC\n18L+y5hfx6C99LxNzz9XSqXiqBF0/ghLg5P8/lgTmru8S6Q1ks+DPqeAi35t5EVaM1BKpXJDcXnf\nPo4ENadLn8KYEiX48YkfKedRzrG+Xiwn59GagVIq0wKrBKauEVSpQvnF4fw+5RK1zxbhgc8fYOeJ\nnYBeLCcv0Z6BUip99u5FWrVkzOuNeC9pJd89/h3L9i/TI41yIO0ZKKWyT9WqmFWreWv8H4x3f5xH\nv32UxuUaayLIIzQZKKXSr1o1rMt+YMfa+cy1DOC5Jc8xbfM0Z0elsoAeFqCUSjdrspURR2cR+lY4\nlm69KNu2HW1XvMKJxBMMbzbc2eGpTNCagVIq3VIdaXTmDPTuzfbkI3RoHUvDco35ouMXeBX1cqyv\nRxo5h9YMlFLZKtWRRh4e8OOP1PbvwtYpQvSx3TSe3pgTiScAPdIot9GegVIq85YsIen5fnR92Yed\nRc7yU8+f+HTTp3qkkZPoRHVKKefZt4/LXTrR89GLzHM9xJbnt1CvVD1nR5UvOXWYyBjjYoz5yxiz\n2L7saYxZaYzZa4xZYYzxSLHuMGPMfmPMbmNMm6yKQSnlRFWqkBC+knvPXuLZS7VpNasVe+P2Ojsq\nlU5ZWTMYBOxKsTwUWCUiVYE1wDAAY0wNoDtQHWgPfGqMuaMMppTKeazJVkZsfJexQ39m6ufHeMG3\nG42mNWJb7DZnh6bSIUuSgTGmLNABSHnAcSdgpv3+TKCz/X5HYK6IXBaRw8B+oGFWxKGUcp6IIxG2\nGkGFqpiRbxE69QDDmgyl1cxW7Inb4+zw1G1kVc9gIvAGkHKA30dEYgFE5DjgbW8vA0SnWC/G3qaU\nysVSHWn04otw4gRD/qnEhLYTaDy9Mb8e/jXV+nopzZwl0yedGWMCgVgR2WqMCbjFqhmqBIeEhDju\nBwQEEBBwq6dQSuUIBQrAxx9Dz5703r0bl3YutPumHd93+54OVTqkupSmyrzw8HDCw8MztY9MH01k\njBkLPA1cBooCbsCPwH+AABGJNcaUAn4RkerGmKGAiMh79u2XA6NE5I+b7FuPJlIqN+vVC8qUgXff\n5cfdP/LUD08x9dGp/H70dz3sNBs5/dBSY0xz4DUR6WiMeR84JSLvGWOGAJ4iMtReQP4GaIRteOhn\n4P6bfetrMlAql/vnH6hTB377DapWZf7O+XRf0J0ZHWfwzAPPODu6PCunnYH8LtDaGLMXaGVfRkR2\nAfOwHXm0DAjWb3yl8qjSpWH4cHj5Zazn4wk/HM4P3X8geFkwc7bPcXZ0KgU96Uwplb0uXcLasA4j\nnqlA6HNzsRSx8EvkLwTOCeSzwM/oXa+3syPMc3Jaz0AppaBgQSLe6kPo5B1YrhQEoIVfC5b3XM6g\n5YMY8vMQrMnWVJvokUZ3n/YMlFJ3R8+eUKECjB3raNoeu53Ws1tTy7sWC7rNx1LUM9WRRlpgzhin\nF5CzmiYDpfKQY8dsxeRy5eD8edstOZldRRNo1TWJGqddmF5rOOOrnSK01VhNBJmgyUAplbMdOQIn\nT0LRoqlue5OOEDCrJceT44j8/SF8P/8Oypd3drS5ltYMlFI5W/ny0KAB1KgBfn5QqhR4eODjWY6W\nldvgU8yHZxufxNr4AZg1C/TH4F2jPQOllFOlrBH8c+4fWsxsQR1XX+ZNO4vFrzp89hmULOnsMHMV\n7RkopXIdxwR3RSxUL1md1b1X83dSJO+N7QAVK0LduvDTT84OM8/TnoFSKsfZFruN5l82Z3L7yfSy\nlocuXeDgQaxFjV5TOR20gKyUyjPWHl5Lu2/a8UXQF/T8YAXWmpUYUeuEHnKaDpoMlFJ5SnhkOB3m\ndODjWm+wec4EQqdFYnHT+sHtaM1AKZWnBPgFMKfrHPpvHc3DlMeydJWzQ8qzNBkopXIsa7KVnw/+\nzHddv+PZmgdZOXOUHm6aTTJ9cRullMoO109L4QJ0vtSDFUs+oWnHl5wdXp6jNQOlVI4Uti8M//L+\nqYrFsz7swyvx37J+0DaqlajmxOhyNi0gK6XytsREZrUtxchObvz6bAR+nn7OjihH0gKyUipvK1aM\n3s1e5tFT99JiZguOnTvmeEinvc4cTQZKqdzlpZcY+1U0pYuWpOXMlsQlxTnqC/7l/Z0dXa6lw0RK\nqdynVy+stSrTynMxFy9f5KFyDzG+9Xg9Gc0uI8NEejSRUir3GTwYS6dOLNi8mopTquJe2J2iBYo6\nO6pcTYeJlFK5T/36WKtV4IM5L3Hg5QMcTzxO9wXduXL1irMjy7U0GSilch1rspURnT0I/e4klTwr\nEtEvgs3HNvPckufQoeWM0WSglMp1Io5EEPrMbCynEuG33yhVvBTr+68n/HA4o38d7ezwciUtICul\ncq8pU2DRItv1DowhNiGWBz5/gFcbv8rrD7/uWM2abM1XU1875TwDY0xZY8waY8xOY8x2Y8x/7e2e\nxpiVxpi9xpgVxhiPFNsMM8bsN8bsNsa0yWwMSql8qn9/iI6GH38EwKe4D8t6LiMkPIQvt3wJoIed\nplOmewbGmFJAKRHZaowpDmwGOgHPAKdE5H1jzBDAU0SGGmNqAN8ADwJlgVXA/TfrAmjPQCl1W7/+\nCr16wa5dULw4AL8d+Y3Ws1vzRdAXbIjekO+ugeCUnoGIHBeRrfb7CcBubF/ynYCZ9tVmAp3t9zsC\nc0XksogcBvYDDTMbh1Iqn2reHFq0gJAQR1OT8k34qtNX9PqxF4FVAvNVIsioLC0gG2N8gXrA74CP\niMSCLWEA3vbVygDRKTaLsbcppVTGjB8Ps2bB9u2AbWhobdRaPmr/Ed3nd2dH7A4nB5jzZdlJZ/Yh\nogXAIBFJMMZcP76TofGekBTZPiAggICAgIyGqJTKq7y9YfRoGDgQ68rFjPjlLcfQkPW8lWZfNWPL\n81uoYKng7EizRXh4OOHh4ZnaR5YcTWSMKQAsBX4SkUn2tt1AgIjE2usKv4hIdWPMUEBE5D37esuB\nUSLyx032qzUDpVT6XL0KjRsT1vsh/Pu/7RgaEhEGLh3I+qPr2TRgE4ULFHZyoNnPaVNYG2NmAXEi\n8mqKtveA0yLyXhoF5EbYhod+RgvISqmssGULtGtnKybfe6+j+crVK3Rf0J0iBYowu8tsXEzePsXK\nKcnAGOMPrAW2YxsKEmA4sBGYB5QDooDuImK1bzMM6A9cwjastDKNfWsyUErdmUGDIDERpk1L1Xz+\n0nnqT61Pu0rtmNhuoqM9L56DoBe3UUqps2ehenWYPx8efjjVQwdPH6TB1Aa81ewtXnv4tRsurZlX\n6KylSinl7g4TJsDAgbB5MxT492uuklclwvuG4z/DH4/CHmw5viXPJYKM0p6BUirvEYG2beH8eahd\nG3x9bbcKFcDXl4Wn19Nl3mMsfXJpnhoeukaHiZRS6pqzZ2HtWjh82HaLioLDh7EeO8SI/5yhZqNH\neZNVbHx2IzW8azg72iylyUAppW7BUSOoFowlsCujnizNp6472PL8Fsq6l3V2eFlGk4FSSt1C2L4w\n/Mv722oEx48jLVvQ64mCHCjjSkS/CO5xucfZIWYJTQZKKXUnYmO5+EgL2nU9zwONuzCh7YfOjihL\nOGWiOqWUyrV8fCi0Opzvl7jy3fov+HD9hFQPW5OthO0Lc1Jwd5cmA6VU/ubtjeeKX1kcXpqRK4fy\nw67vgfx3HQQdJlJKKYBTp1j2VEO6No5i8dPLWLh3Ua49B0FrBkoplRmnTzOhXzVef+Akfz33Fw+U\nfsDZEWWI1gyUUioTrK4uHHqkAQMOlyDo2yBOJp50dkh3jSYDpZQixTkI/b5hyiZvahhvWs5qiTXZ\n6uzQ7gpNBkopBUQcibDVCFy9uGfESOYvLMjFyxcZuXqks0O7K7RmoJRS17tyBapX58CkUfjvfJXv\nHv+OAN8AZ0eVblozUEqprHDPPTBsGJUnfMmcx+bQZW4Xth7fmmqVvHYOgiYDpZS6maefhgMHaBXr\nyvCmw2k1qxXRZ6KBvHkOgg4TKaVUWqZMgbAwZMkS+i3qR/jhcFb3Wc2E9RNy9DkIep6BUkplpeRk\nqFQJli7lUp1aNPmyCRtjNhI5KBJfi6+zo0uT1gyUUiorFSkCr78OoaEkXkqkRokalC5emoFhA/Pc\nIafaM1BKqVtJTMRazZcR4x4h9PEp7D+1n/bftKdNpTZ8Gvhpjhwq0p6BUkpltWLFiHihA6Err2Ap\nYuHBMg/yQZsP2BizkZUHVzo7uiyjPQOllLqds2dttYMNG6ByZQBeWvYSUWeiWNRjES4mZ/2u1p6B\nUkplB3d3ePFFePddR9PEthM5k3yG0b+OdmJgWcdpycAY084Ys8cYs88YM8RZcSilVLr897+wZAn8\n8AMABe8pyHMNnmPaX9NYtGeRY7XcejKaU5KBMcYF+BhoC9QEnjTGVHNGLEoplS5eXrB8ua2HMHcu\nAI9WeZSHyj5E/8X92RO3J1efjOaUmoEx5iFglIi0ty8PBURE3rtuPa0ZKKVylh07oE0bGDcO+vTB\nmmzlse8eI8oaRcuKLRnferzTjzDKTTWDMkB0iuWj9jallMrZatWCNWtg5EiYOhVLEQszOs3gkPUQ\nMWdj8Cjs4ewIM6SAswO4nZCQEMf9gIAAAgICnBaLUkoBUK0a/PILtGqFNdnK+EpR7HlxD02/bMq4\ndeMY3mz4XQ0nPDyc8PDwTO3DmcNEISLSzr6sw0RKqVzHum8bI0Y1JbTBG1heH8m22G08NO0hFnRb\nQIcqHZwWV24aJtoEVDbGVDDGFAJ6AIudFItSSmVIBNGEjlmPZeosGDOGOt61mdVlFr0X9uZ4wnFn\nh3dHnHbSmTGmHTAJW0KaLiLv3mQd7RkopXK+f/6xFZVbt4YPPiBk7WjCD4ezqvcqCrjc/dF4nbVU\nKaWcJT4egoLAz48r076g4Vf++Jf3Z3L7yY5VrMlWIo5EEFglMFtDyU3DREoplbd4esLKlWC1ck/n\nLsx/9Cu+3Pols/+eDeT8C+Joz0AppbLS5cswYADs3s2az4YQGPYUYU+F8f2u7+/aBXF0mEgppXIC\nERg6FJYsIfS9Rxn513h2B++mWsm7M9GCJgOllMpBrOPfYfjm94lp9SD7Lx5nff/1ObZnoDUDpZTK\nBtZkKyNqHGdsm/f4eswuLiUn8vi8x3PsFdI0GSilVDaIOBJhqxH0C8Zt3Id8/2USW47+ydwdc50d\n2k3pMJFSSt0NX3/NtBkvMamjN3+8uBXXgq7Z9lRaM1BKqRxMZs6k988DKdSmPdN7f59tz6M1A6WU\nysFMnz5MaTWR9X8tZtby926/wV2kyUAppe6i4s88z8t+TzA4fBi7N/3kaHf2FdI0GSil1F321PMf\nU9dSlce+CSJxz7YccXay1gyUUsoJ4s/H03hCdersP0fJwMcJ7Tgpy85B0JqBUkrlEp5FPfn+udXM\nr5hEta+XY0m47NR4NBkopZQTWJOtfLrpU5Y+uZQ3659m6xPN4dw5p8WjyUAppe6yazWC0FahBFYJ\n5P9ajaat/yFOPt4ekpOdEpPWDJRS6i4L2xeGf3l/R41ARGj/dTuKb9nBgsONYN48KJDxi+LoSWdK\nKZVLxSXF8cBn9Zj+uw9t3OrBtGlg7uj73EELyEoplUuVcC3BrC6z6dvwGMf3b4E337RNhX2XaDJQ\nSqkcooVfC5r5NufJ3q5cXRYGc+YAd+eENE0GSimVg3zc4WP2nDnIO280grfewnru5F05IU1rBkop\nlcNsi91Go2mN+HZbVX7+j4XQlxfe0QlpWjNQSqk8oI5PHSa0mUCXGn8TPGs3Fopk+3NqMlBKqRzG\nmmxl54mdPF79cboFJmGdMjHbnzNTycAY874xZrcxZqsx5ntjjHuKx4YZY/bbH2+Tor2+MWabMWaf\nMeZ/mXl+pZTKa1KekDaj0wzO3+tBjx0hWE/FZOvzZrZnsBKoKSL1gP3AMABjTA2gO1AdaA98aozj\ngNkpQH8RqQJUMca0zWQMSimVZzgul1nEglthN759cgGb74NFnw3O1ufNVDIQkVUictW++DtQ1n6/\nIzBXRC6LyGFsiaKhMaYU4CYim+zrzQI6ZyYGpZTKSwKrBKYqFj9U9iH+W/d5Zkcu5Ko1PtueNytr\nBv2AZfb7ZYDoFI/F2NvKAEdTtB+1tymllErDsMc+JLmkhYmTemTbc9x28gtjzM+AT8omQIARIrLE\nvs4I4JKIfJvVAYaEhDjuBwQEEBAQkNVPoZRSOVoBlwLM7jGfet8F0HDbUprWedTxmDXZypT5U7hw\n8EKmniPT5xkYY/oCA4CWInLB3jYUEBF5z768HBgFRAG/iEh1e3sPoLmIDExj33qegVJK2X3+alOG\nuW9ix2uHuM/tvlTF5pRDS3f9PANjTDvgDaDjtURgtxjoYYwpZIzxAyoDG0XkOHDGGNPQXlDuDSzK\nTAxKKZVfPDd4Di0PXKXtl604bD1800SQUZnqGRhj9gOFgFP2pt9FJNj+2DCgP3AJGCQiK+3tDYCv\ngCLAMhEZdIv9a89AKaVSsA4eSC232cTck0jkoEh8Lb43rKNTWCulVB5njdpL35G12VDTg8DqQXzY\n9sMbegY6HYVSSuVh1mQrI3ZM5qsqb9Az2oNT508xfPVwrMnWTO9bewZKKZVLOK6QZoqSXK8WDfpe\n4L+tRlDWvSyBVQId6+kwkVJK5RerVrHlzd606XGJv57/i3Ie5RwPaTJQSqn85MknCa18jF/uL8jK\nXitxMbaRf60ZKKVUfvLhhwz5fCeJZ+P4eOPHmdqVJgOllMqtSpemwFujmL2sMG/98hZ/HP0jw7vS\nZKCUUrnZwIFUPn6JUcUCCfo2iJOJJzO0G60ZKKVUbrdxI9K5E63HVOHM5ST+fP5PrRkopVS+07Ah\nplNnZm3xI+FSQoZ2oclAKaXygrFjcV38Ey1ca2Zoc00GSimVB1iLGkYMqs3Yzw9kaHtNBkoplQdE\nHIkgdOB8LL0GZGh7LSArpVQeoyedKaWUyhBNBkoppTQZKKWU0mSglFIKTQZKKaXQZKCUUgpNBkop\npdBkoJRSCk0GSimlyKJkYIx5zRhz1RjjlaJtmDFmvzFmtzGmTYr2+saYbcaYfcaY/2XF8yullMqc\nTCcDY0xZoDUQlaKtOtAdqA60Bz41xlw7NXoK0F9EqgBVjDFtMxtDfhAeHu7sEHIMfS/+pe/Fv/S9\nyJys6BlMBN64rq0TMFdELovIYWA/0NAYUwpwE5FN9vVmAZ2zIIY8Tz/o/9L34l/6XvxL34vMyVQy\nMMZ0BKJFZPt1D5UBolMsx9jbygBHU7QftbcppZRyogK3W8EY8zPgk7IJEGAkMBzbEJFSSqlcLMNT\nWBtjagGrgCRsCaIsth5AQ6AfgIi8a193OTAKW13hFxGpbm/vATQXkYFpPIfOX62UUhlwp1NYZ9n1\nDIwxkUB9EYk3xtQAvgEaYRsG+hm4X0TEGPM78F9gExAGTBaR5VkShFJKqQy57TDRHRBsPQREZJcx\nZh6wC7gEBKe4Ss2LwFdAEWCZJgKllHK+HH2lM6WUUndHjjwD2RjTzhizx35i2hBnx+Msxpiyxpg1\nxpidxpjtxpj/OjsmZzPGuBhj/jLGLHZ2LM5kjPEwxsy3n9S50xjTyNkxOYsxZrAxZof9ZNZvjDGF\nnB3T3WKMmW6MiTXGbEvR5mmMWWmM2WuMWWGM8UjPvnJcMjDGuAAfA22BmsCTxphqzo3KaS4Dr4pI\nTaAx8GI+fi+uGYRt+DG/m4RtmLU6UBfY7eR4nMIYcx/wMrZ6ZR1sQ989nBvVXfUltu/KlIYCq0Sk\nKrAGGJaeHeW4ZIDtaKT9IhIlIpeAudhOYst3ROS4iGy130/A9gefb8/LsJ/t3gGY5uxYnMkY4w40\nFZEvAewnd551cljOdA9QzBhTAHAFjjk5nrtGRH4D4q9r7gTMtN+fSTpP7M2JyeD6E9b0xDTAGOML\n1AP+cG4kTnXtbPf8XujyA+KMMV/ah8ymGmOKOjsoZxCRY8AE4Ai2Q9utIrLKuVE5nbeIxILtByXg\nnZ6NcmIyUNcxxhQHFgCD7D2EfMcYEwjE2ntKxn7LrwoA9YFPRKQ+tnN9hjo3JOcwxliw/RKuANwH\nFDfGPOXcqHKcdP14yonJIAYon2L52sls+ZK967sAmC0ii5wdjxP5Ax2NMYeAb4EWxphZTo7JWY5i\nmwbmT/vyAmzJIT96BDgkIqdF5ArwA/Cwk2NytlhjjA+AfT64E+nZKCcmg01AZWNMBftRAT2A/Hzk\nyAxgl4hMcnYgziQiw0WkvIhUxPaZWCMivZ0dlzPYhwCijTFV7E2tyL9F9SPAQ8aYIvaZkVuR/4rp\n1/eUFwN97ff7AOn6EZmVJ51lCRG5Yox5CViJLVlNF5H89p8LgDHGH+gJbDfGbMHW3RuuJ+opbGfx\nf2OMKQgcAp5xcjxOISIbjTELgC3YTnDdAkx1blR3jzFmDhAA3GuMOYJt2p93gfnGmH7YpgDqnq59\n6UlnSimlcuIwkVJKqbtMk4FSSilNBkoppTQZKKWUQpOBUkopNBkopZRCk4FSSik0GSillAL+H3Bw\nuXyJPhDcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0e27112160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_epoch = 3000              # epoch size\n",
    "a, b, c = 1.0, 1.0, 1.0     # initial parameters\n",
    "epsilon = 0.0001            # learning rate\n",
    "\n",
    "N = np.size(t)\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    for j in range(N):\n",
    "        a = a + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)*t[j]**2\n",
    "        b = b + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)*t[j]\n",
    "        c = c + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (y[j] - a*t[j]**2 - b*t[j] - c)**2\n",
    "    \n",
    "    if i % 500 == 0:\n",
    "        print(\"epoch %4d: loss = %10g, a = %10g, b = %10g, c = %10g\" % (i, L, a, b, c))\n",
    "    \n",
    "    \n",
    "y_est = a*t**2 + b*t + c \n",
    "\n",
    "\n",
    "plt.plot(t, y, 'r-', label='Real data')\n",
    "plt.plot(t, y_est, 'g-x', label='Estimated data')\n",
    "plt.legend()\n",
    "plt.savefig(\"fig-res-missle_est.pdf\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. 如何使用sklearn求解线性问题？\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  (100, 1)\n",
      "Y:  (100, 1)\n",
      "a = 2.993255, b = 2.768899\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEPCAYAAACzwehFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcVNWd9/HPD+lumqVZtMUdcAUTCZCBR6PRxogaM1ET\nMyqTTCAi6jhGH03cx4CPSUbN68kkmkcFxC0RYUxiII7KEu0YtzSuOGHRaJq40u2GgiAN/Xv+uNXY\n3VR17XXvrfq+X6960V1ddftXt5r7q3PO75xj7o6IiEhPeoUdgIiIRJ+ShYiIpKVkISIiaSlZiIhI\nWkoWIiKSlpKFiIik1TvsAMysGVgPtANt7j7BzAYDC4BhQDNwqruvDy1IEZEKF4WWRTvQ4O5j3X1C\n4r7LgGXufhDwMHB5aNGJiEgkkoWxYxwnAXcmvr4TOLmkEYmISBdRSBYOLDWz5WZ2ZuK+oe6+DsDd\n3wZ2DS06EREJf8wCONzd3zKzemCJma0hSCCdaU0SEZEQhZ4s3P2txL+tZvY7YAKwzsyGuvs6M9sN\naEn2XDNTEhERyYG7WzaPD7Ubysz6mln/xNf9gGOBF4FFwNTEw6YAC1Mdw91je5sxY0boMSj+8OOo\nxPjjHHs5xJ+LsFsWQ4H7Ei2E3sDd7r7EzJ4G/svMzgDWAqeGGaSISKULNVm4+9+AMUnufw84pvQR\niYhIMlGohqpYDQ0NYYeQF8UfrjjHH+fYIf7x58Jy7b+KAjPzOMcvIhIGM8PjNMAtIiLxoGQhIiJp\nKVmIiEhaShYiIpKWkoWIiKSlZCEiImkpWYiISFpKFiIikpaShYiUrdbWVpYvX05ra2vYocSekoWI\nlKV77lnAsGEjmTTpHIYNG8k99ywIO6RY03IfIlJ2WltbGTZsJJs2PQKMBlZQWzuRtWtXU19fH3Z4\nodNyHyIiQHNzM9XVwwkSBcBoqqqG0dzcHF5QMadkISJlZ/jw4WzZ0gysSNyzgra2tQwfPjy8oGJO\nyUJEyk59fT1z595Ebe1E6urGUVs7kblzb1IXVB40ZiEiZau1tZXm5maGDx+uRNFJLmMWShYiUlaK\nmSDKJflogFtEKloxy2UrvRRXLQsRKQvFLJctt1JctSxEpGIVs1xWpbhKFiJSJopZLqtSXCULESkT\nxSyXVSluRMYszKwX8DTwurufaGaDgQXAMKAZONXd1yd5nsYsRKQLVUOlF9vSWTO7EPg8UJdIFtcB\n77r79WZ2KTDY3S9L8jwlCxGRLMVygNvM9gJOAG7tdPdJwJ2Jr+8ETi51XCIi8qnQkwXwn8DFQOcm\nwlB3Xwfg7m8Du4YRmIiIBHqH+cvN7CvAOnd/3swaenhoyr6mmTNnbv+6oaGBhoaeDiMiUnkaGxtp\nbGzM6xihjlmY2Y+BbwFbgVpgAHAf8A9Ag7uvM7PdgEfcfVSS52vMQkQkS7Ebs3D3K9x9H3ffFzgd\neNjd/wX4PTA18bApwMKQQhQREaIxZpHMtcAkM1sDfCnxvYiIhCQSpbO5UjeUiEj2YtcNJSIi8aBk\nISIiaSlZiIhIWkoWIlJyra2tLF++nNbW1rBDkQwpWYhISVX6jnNxpWooESmZcttxLq5UDSUikZCq\nm0k7zsWXkoWIFFRP3UzacS6+1A0lIgWTSTfTPfcsYNq0c6mqGkZb21rmzr2JyZNPCzXuSpNLN1So\nq86KSHnp6GbatGnHbqaOZDF58mkcc8zRZbHjXCVRshCRgunazRS0LJJ1M9XX1ydNEuWybWk50piF\niBRMfX09c+feRG3tROrqxlFbO5G5c2/K6MKvktpo05iFiBRcti0EldSWlsYsRCQSUnUzpTJr1hw2\nbRpCspJaJYtoUMtCRELV2trKPvscyObNBjSilkXxaVKeiMROc3MzNTX7AjcDE4FxwGFcccX3lCgi\nRMlCREL1aQXVKGA18H369Knm7LOn53Q8LVJYHEoWInnQhSl/XSuojqO29rvcdtstObUqVFFVPBqz\nEMlRx0zk6urgk3ExZiJX0ryDfF9rSSqq1q2DxYvh298uzPFCojELkRJpbW1l2rRz2bTpEdavf4ZN\nmx5h2rRzC9rCqLRPyfX19YwfPz7nC3tPFVV5aW+HpUvhn/4JRo6ERx+Ftrb8jhlDShYiOSj26qml\nSEblpLW1lR/96CfAuxRskcK334b/+A/Yf3+4+GI4+mhYuxZuvRWqqgoTeIwoWYjkoNirp2op7+R6\nWvq8IBVV7e2wZAl84xswahS8+iosWADPPQf/+q9QV1fAVxMvShYiOchnWYtMaCnvHWW29HmOFVVv\nvQU//nHQirjsMpg0KWhFzJkD48eDZdW9X57cPbQbUAP8GXgOeBGYkbh/MLAEWAMsBgameL6LhKml\npcWbmpq8paWl4MeeN2++19YO8bq6sV5bO8TnzZtf8N8RFy0tLV5bO8ThBQd3eMFra4d0Oe9Zn69t\n29wfesj9a19zHzTIffp09+XL3dvbi/xqwpe4dmZ1vQ69GsrM+rr7x2a2E/A4cD5wCvCuu19vZpcC\ng939siTP9bDjFykmVUMFli9fzqRJ57B+/TPb76urG8eyZbMYP3789uc/99xzAIwdOzb1+XrzTbj9\n9mDsYcgQOPtsmDwZBgwozguLoFyqoUJtWXS+AX2Bp4HxBO3IoYn7dwNWp3hOYdKsSCfFbC3EXbHO\nzbx5871Pn0Her99B3qfPoB1aBelaFh2tioEDxyVvVWzd6v7gg5+2Is46y/3ppwv6GuKEHFoWUUgS\nvQi6oT4E/iNx3/vdHvNeiucW8vyJpL/oVLBinZuWlhavqhrgMNhhnMNgr6rqv0NCStXN1GMieeMN\n92uucR82zP0f/sF99mz3Dz8sSNxxlkuyCL0bqoOZ1QH3EXRD/cndh3T62bvuvnOS5/iMGTO2f9/Q\n0EBDQ0MJopVypGWyUyvmuVmyZAnHHfc14Mntx4bDWLz4Po499tgd4ujeVdW9i6oX2/ha7YHMGb8X\ng1esgNNOg7POgnHj8oozzhobG2lsbNz+/dVXX511N1Rklih39w/NrBE4HlhnZkPdfZ2Z7Qa0pHre\nzJkzSxShlLtMtgStVMnOTe/e+xTw3OxB5zJh2D3po5Itfd5RCbUHSzmDpziTm2nd3ELvk8+D//5v\n6N+/APHFW/cP0ldffXXWxwi1dNbMdjGzgYmva4FJwCpgETA18bApwMJQApSKonLV1JKdm48+WsOz\nzz6f97HHjh1LdXVrl2NXV7/D2LFj0z952zbqm5p46eARvMhxDK+6gdNrNvLy3Xcz4MILad20SWt3\nFUq2/VaFvAGHAM8CzxP8pVyZuH8IsIygdHYJMCjF8wvXiSfiKldNpaWlxS+77HKHWofRDkMcrtuh\nfDVXHee9X7/RmZ33115zv/pq9733dp8wwf3WW73l1Ve7DL5r/Ck14jxmkQuVzkoxVFK5aiY6Fkzs\n1WtvNm58Cfh3YDpQv0P5aj7Snvdt2+DBB2H2bHjssaDcdfp0GDMm6bGCMZbfAP2AjdTWnqLxpwRt\nqypSANluCRqGUiW0zmtUfTr43ECQLArbTZfyvL/2GsydG9z23DMYrL7nHujXL+WxgmVRBhFM2RoO\nNONep/GnPGi5D5GYKeVqtMnWqIIh9Ot3RMGXOOli61b4/e/hq18NWg7vvAP33w9PPQVnnNFjogDo\n378/mza9BTwCPAP8hs2b32LLli2Fj7VSZNtvFaUbGrOQbsp9Ql0my14U8nctXrw46e9bvHhxcc7x\n2rXuP/iB+557uh96qPttt7lv2JD1YZqamry29pBEzPMTYywHeE3NjhP+KhFxnJSXz03JQjqrhAHN\npqYmHzhwXOIiGNzq6sZ6U1NTQX9P53NZVdXfq6sHFm/Qv63NfeFC9698xX3IEPfzznN/4YW8Dvlp\nUn0kkSiKn1zjRMlCKlYpP3GHqRSvM9nv6NNnUOFbE83N7lddFbQiDjvM/Y473DduLNjh582b7zU1\ndQ4HFj25xk0uyUJjFlIWKmX/h1RLowMFm0+Q7FxWV49g8ODB+Y9PbN0KCxfCCScEM6o/+CCocHri\nCZgyBfr2zTP6T02efBrPPfcUNTUtaO5M/pQspCxU0oS6yZNPY+3a1SxbNou1a1cD5DzgnWwzoVzO\nZapNibZrboarroJhw+D66+H004MqpxtugEMOyTjebI0aNYrbb7+laPuOVJRsmyJRuqFuKOmkEifU\n5dMt1dMYTzbnMuVxtmxxv+8+9+OPd995Z/fzz3d/8cW8X3NPUhU4lHvhQ7bQmIVUukq7KOQ64J1J\nksnkXCY7zkE1A33DhRe67767+xFHuN91l/vHHxfsNadSCQUOhZJLstCkPCkrmU6oK5dZ2l27jIJJ\nc5l0v2WyaGIm57LjOG2bRvFVfstZzGb8lg189NZb9Fu2DA4+OM9XmJnOkweD17SCadMmcswxR8f6\n/Y0SjVlIxSnlpLZiy3Uv8EKN8exrxhUbV7KWPfnf/IxfcSQH1NSx0w03lCxRQOUUOIQq26ZIlG6o\nG0qyVKrS01J3heXyO3Me49myxf3Xv3Y/9lj3nXf2VV/+so+p+XQexi23zA7l9RfrfS3Hrk00ZiHS\ns2JPaotbv3lWF8JXXnG//HL33XZzP/JI91/9yn3Tpi7HueWW2aG9/mIUOMTt/cyUkoVIGsX6BNqx\nNEafPoPKa2LgJ5+433uv+6RJ7rvs4n7RRe6rViV9aBQmRhayFRCF11MsuSQLDXBLReno4582bSJV\nVcNoa1ubd9195yW8N2/eQrB/12hivdPeK6/AnDlwxx0wcmSw0uuiRdCnT8qnRGGnwUKuGByF1xMl\n2s9CKlKhqqGS7U0dLOG9BngrXnt4b9kSzK6ePRteeIGPv/ENXjn6aHY76iiAtOer3PYwL7fX01ku\n+1mE3pWUzw11Q0nIko2BwH7et++BXlMzyG+5ZXbYIab38svul1zivuuu/snhh/sLl1/uV37/ku19\n9dXVA72qqn9G/fblNjGy3F5PB7RTnkhqxZhbsWrVKsaOPZRPPllI0KJYQVXVFzFrp6ZmBFu3vsHc\nuTcxefJpBfl9BbNlC/zud0ErYsUKmDKF+3ffg1Muv5otW+qBN4CZwCVk21qKyxyWTOOMy+vJhloW\nIikUo6ql45jBvgm13qfPcK+tHeJVVf2jOyj60kvuF1/svuuu7hMnus+f7755c9LB3GBp75bE92Md\nmspm1dZyrXLKFKqGkqgKs1a9GFUtyY5ZUzPIFyxYUJL9JjKNsampyVteey1IChMnBkni4ovd16zp\n8timpibv1+9z3brTRicSxAsOgxOJI2LJLwflXOWUqVyShWZwS9GFPWO6GLN7kx2zpmYEgwYNisTq\nt/fcs4Av7X0ATx7xj/je+/D2NdfAOefA3/8erPp64IFdHj98+HDa21/rEje8DJxKVdUXqapqo67u\nuLJYtVWzvXOUbXaJ0g21LCIvCp/iStWy6DhmqIOimzf7+lmzvLFXb3+bIX4tl/j+LMro9c6bN9+r\nqwc67OfQ13v37ufXXPMjb2lpKatZzFH4mwwb6oaSqCnVNqDpFHN2b7Jjlvziunq1+/e+515f7x9M\nmODf7jvCq/gk63PeMbmwaHtsR0S5VjllKpdkEWo1lJntBdwFDAXagTnufoOZDQYWAMOAZuBUd1+f\n5PkeZvySXpRq1YtR1VKsSpmMjrt5M/z2t0FF0+rVMHUqTJ9Oa11dZM55sRTivJdjlVOmYlcNBewG\njEl83Z+gNm8kcB1wSeL+S4FrUzy/YJlWiqfSP8VlK22lzqpVwbIbu+wSLMNx773BshxJjlGO57zS\nK5kKgbi1LLozs98Bv0jcjnL3dWa2G9Do7iOTPN6jFL+kVsmf4rKRsiW25nnqH300aEW89BJ85zsw\nbRrst1+Px2pubqZ///5s2LChLM59lFqqcZZLyyIy1VBmNhwYAzwFDHX3dQDu/jawa3iRSSHU19cz\nfvz4jDcm6nE/5wwU4hhh6F6pM5Iqrt+2E4NGj4a77oLzzw8qmn784x4TBQTn/K9/fZXPf/6Iklei\nFev8q5IpPJFYSNDM+gO/Bi5w9w1m1r25kLL5MHPmzO1fNzQ00NDQUIwQpUQ6FuWrrg4258lm9nPH\nJ+lnn32eCy+8LKdjhG348OH0+uRvfJMfcxYPcQCr+GX7Bj5c8ig7jx+f1bHC2j0un/cwnVx3Bqx0\njY2NNDY25neQbPutCn0jSFgPESSKjvtWEbQuIBjXWJXiuQXovZOwdVQOrVy5MueSxo5+7AEDxjrU\nOlwXv7LIv/zF/YILfHP/Ab6kV5V/s3aED+gzeHuffLYVVmFUopWiLLWcx2NKhTiWzhJUQ/20233X\nAZe6BrjLXufBypqaQV5bOyLri1u6pSqiuDzF9gv/2rXud93lfvjh7rvv7n7lle6vvrpDYshlUDeM\n+QSlSlDlNO8jDLFLFsDhwDbgeeA54FngeGAIsIygOmoJMCjF8wt+EqV0kl/kax0eyerilnzl10+X\nqijGBTKfi9W8efN9XE2d31xd762Yv/G5z7nfd1+wXWmK35Vvi6tUn8I14S0eYpcs8r0pWeQvzE9o\nyS7ytbWf9ZqauqwubqmSTv/+ny3KBTLn0s2PP/b1N97oT/Tq7W9Q7/+Hf/dhPJD2Yprvp/VSv8fq\nJoo+JQvJStj16qk+ha5cuTLri1v3C9Qtt8wuygUyp0/OL77o/t3vug8Z4u8ffrhP7ruv70Rbxhf+\nOH5aVzdRtClZSMaicgEq5KfQUlygMv6Uv3Gj+x13uB92mPsee7hfdZV7c3PO573Qn9Z1Ma9sShaS\nsais2eQerwtX2ov9ihXu553nPmSI+wknuC9c6N7W1uUYuV74C3WeMm1Rxul9kewoWUjGotKyiKPu\nF/sFt93hfttt7oce6r7nnu4/+IH72rU9HiOsC3Gm73vYXZRSXEoWkpVCdW1U4ifQlpYWf/Huu/3j\nM84IWhH/+I/uixbt0IqImkxalPogUf5ySRaRmMEt4Zg8+TSOOebovNZsKuZs3UjauBEWLKB+9mzq\n33gjWJ/p+edh773DjiwjmcyA7lhSI5j1DZ2X1ND6S5UrUgsJZksLCYarEhZ161hCZL8NGxhy770w\nfz4ccQScdRYcfzz0jt/nrY4EX1U1jLa2tTsk+Ep4XytdLgsJxu8vXSIj1SfQ5557jsGDB8d+ldN7\nb7+Th8/+N6a1OxvbN/P6KacwesUK2GuvsEPLy+TJpzFmzGiampqYMGECo0aN6vLz+vp65s69iWnT\nJnZJKHF+L6UAsu23itINjVmEKlnfdlXVgPgPjD77rH88ZYq/i/nvaPATuN978WzZ9NurGkoo5H4W\nZvYAcK67N5cudWVH3VDh69ylsWXL32hvd7ZseZTYdV9s2BB0Mc2eDevW8fqXv8yx8x5n1Ucvbn9I\nXd04li2bxfgsV3+Nkih3MWnPk9Ip9H4WtwNLzOxKM6vKLzQpV5Mnn8batatZtmwWCxcuoLZ2P2K1\n18Czz8I558A++8D998PMmfDqq9Rccw3NW98kGAiGclkKuxj7QRRi74p77lnAsGEjS77vhmShp2YH\nwVan1wEvAN8HLuq4ZduEKcaNmHdDlVszPzYllx9+6D57tvvnP+++zz7u11zj/vrrOzysHNc4KvR7\nVIj5GLH5uykjFHqeBVAN/ABYDVwNzOi4ZfuLinGLc7Io10lPkb7APv20+1lnuQ8a5H7yye4PPOC+\ndWuPTym3hO5e2Pk1hbjIR2k1gUpR0GRBsFT4SuBaoG+2By7FLa7JIpP/ZHG+SEUq9g8/dJ81y33c\nOPdhw9x/+EP3N94IO6qCy/acF+I9KtRFXi2L0it0svgT8JlsD1jKW1yTRbr/ZOXa6iiZ9nb35cvd\np08PWhFf/7r7Qw+lbUVESfeLeU8X97D+Xgp5kY90i7QMFbwbKuq3uCaLnv6T6VNWdrpcRNevd7/5\nZvexY92HD3f/0Y/c33wz7BCz1v3if955F6RMBmH/vcRt1WAJKFnESKr/ZOq/zdy8efO9ts9gb+h3\nkN+2U41/0ref+ymnuC9e7L5tW9jh5WTHi/8jHuwemDwZROHvRRf5+MklWWgGd0hSrcuUydo9Au+8\n8gpPTTmDx9v2ZiBbmMPZHNR+F0033xzrGv0dZ8X3A/YmWalrfX19JP5e6uvrY33OJTM9zbOQIquv\nr2f8+PFd/qN1LLVQWzuRurpx1NZO1FILHdyhqQnOPJNBY8cy0au5hBvZn79yLT/nveoR0Z7TkYGu\nF3+AjcBrpJrvob8XKRUtJBhRms3ayfr1cPfdwezqjz6C6dN556tfZZ/xR0ZyJnK+ui/0N23at5g7\n91cpF/4D/b1IdnKZwa1kIdHU0YqYNQt++1s49thgpdejj4ZeQYM43eqpcdb94q9kIIWkZFEByv6i\n8cEHn7YiNm4MEsSUKTB0aNKHR+18RC2ebMQ5dslOLski9IqmfG7EuBoqF2U7/6K93f2JJ9ynTnUf\nOND91FPdly2LXUVTnN+fKMeuaqvCI46ls8BcYB2wotN9g4ElwBpgMTAwxXMLewYjLOx6+qJ4/333\nG290P+QQ9/33d7/+evd168KOKidxfn+iHHuUk1ic5ZIsolANdTtwXLf7LgOWuftBwMPA5SWPKmKK\nsVpoKNzhiSdg6lQYPhweewx+9jNYswYuvhh23TXsCHMS5/cnqrG3trYybdq5bNr0COvXP8OmTY8w\nbdq5ea1uK7kLPVm4+2PA+93uPgm4M/H1ncDJJQ0qgnYsqYzZ/Iv334cbb4TRo4NE8dnPwssvB3tI\ndBq0jqs4vz9RjT2qSaxSRfV/6K7uvg7A3d8G4vlxs4BiWU/vDo8/HgxQjxgRtChuvDFoRXz/+xDl\n2LtJt2dDLN+fhKjGHtUkVqkiUQ1lZsOA37v76MT377n7kE4/f9fdd07yPJ8xY8b27xsaGmhoaChB\nxOGJRcXKe+/BL38ZVDRt3RpUNH3727FKDp11lOhWVwcXr55KdJO9P7F4z4hmnOVcHl1KjY2NNDY2\nbv/+6quvjmfpbJJksQpocPd1ZrYb8Ii7j0ryPI9C/FFU8v/4Ha2I2bNh0SL4yleCJHHkkWDZVehF\nSb7bkGaTaCS5KCaxuCv0tqqlZIlbh0XA1MTXU4CFpQ4ozkq6ReV77wUD1J/5DJx5JowZA3/9azBX\n4qijYp0oIL9+cw3QFkayZXGk9EJPFmY2D3gCONDM/m5m3yHYcGmSma0BvpT4XjJQkguUO/zpT/Av\n/wL77gtPPw233AKrVsFFF8EuuxTud4Wg8/hEPv3muSaaQuxpLVJw2dbaRulGBc2zyFRRl6x+5x33\nn/7UfeTI4PbTnwb3lZFkdf257tmQy/wFzSuQUiCOk/LyuSlZ7KjgE6za293/+Ef3b34zmF39rW+5\nP/pocH+ZSbcpVS6ziLNJNFGeHCflJZdkof0sykxHGeS0aRO7VJBk3d/7zjtw113BgHWvXnD22XDD\nDTBkSPrnxtSOe0l82m2Ua595qn1Lsv396q+XsEWiGipXqoZKLacKEnf44x+DBPHAA3DiiUFF0+GH\n0/rOO2VfkZJv5VPHMXI9T4X4/SKZ0EKCkpPWVat87QUXeNt++7kffLD7z3/u/u67239eSf3o+ewp\nXYjzVMg9rUVSIYduKLUsQhRq/bg7NDay9sp/Z+CTT/Jg1WDmWBvTb5/N5H8+vUuMpfi0G6Va+lxi\nKeR5itK5kPKklkWMhPZpvaXF/Sc/cT/gAG8bOdIvrOrrg/hTygHVolZXJZRDy6UU50mkUFA1VDyU\nvOqlvd39D39wP+20oKJpyhT3xx/3pj//Oe0FrtixlvJcFHNfBFUySZzkkixCn5RXiTKZrFWQiVkt\nLfCTn8CBB8IFF8ARR8Df/gZ33AFf+ALDR4xIO+Gs2IvMlWpl0WLPao/qYnwiBZNtdonSjTJtWeTV\nLbNtW7DL3KmnBq2IqVPdn3wy5byITAZUW1pafPHixb548eKCf1IuxSfycmm9iBQK6oaKj1QX6Zwv\nbOvWuV97rft++wU7z/3iF8FOdBno6QJXivGEYlcAaTxBpKtckoWqoUKUrOpl+fLlTJp0DuvXP7P9\ncXV141i2bBbjx4/veoD2dnj44WBexNKl8PWvB/MiJkwoyAJ+paz7L2YFkOYviHSVSzWUZnCHqL6+\nfoeLVdeF64IL2w4L1739djDuMGcO9O8fzK6eMwcGDixofKWcUZzsXBTy2AWZ1S5SwdSyiKCkG76c\n9k/whz/ArFnBv6ecErQixo8v2jLg5faJXPMXRAK5tCyULCKq48I2cNMm+i5YwO73389OgwcHrYhv\nfhPq6koSh3YqEyk/Shblor0dli7ltat+QP/ly1lUNYRbe7Vx7m2zusyuLhV9IhcpL0oWcffWW3D7\n7TBnDm0DBnDR6le4s20pH/EF0nUBpbqgR/FCH8WYRCpJnLdVrVzt7fDQQ0El08EHQ3Mz3Hsvz996\nK7/sOzKRKKCnyWqpJpzlMxGtWLu1lXTLVxEpnGxrbaN0I8bzLHzbNvcf/tB92DD3cePcZ81y//DD\n7T9ON9+iY27EypUrkz4u1f2ZTBYr1twKLYkhEg1ouY8Y6dULeveG3/wGnnkmqGwaMGD7j3taPqLz\np/OxYw8F9qT7chlNTU057/9crD28S7W0h4gUnuZZhOnSS3v8cbJd1jpfzIP5D43ACXSflzFhwoT0\n8zWSKObciozmkIhIJKllEXH19fVdtvTc8dN5A336DKWm5qguLZBRo0ZlvbBda2sr77//ftrFBfN5\nLVpsTySeVA0VM6kmyj3zzGNs2LAh52qojvkU1dXD+fjjlzDbiT599i3K3ApVQ4mES6WzFaLQE+WS\nJaA+fY5i4cIFjB07Vhd0kTJTdmtDmdnxwM8Iusvmuvt1IYcUCcnGMvKRbJyiunoEgwcPVqIQESDC\nLQsz6wW8BHwJeBNYDpzu7qs7PaYiWxaFVm5rQIlIz8ptUt4E4GV3X+vubcB84KSQYypLGngWkXSi\n3LI4BTjO3c9KfP8tYIK7n9/pMWXbsghjEFgDzyKVoezGLDIxc+bM7V83NDTQ0NAQWiyF0rkyacuW\n5pKt9FpzyvMvAAAIl0lEQVTMPSVEJDyNjY00NjbmdYwotywOBWa6+/GJ7y8jmKJ+XafHlF3LQuMH\nIlJs5TZmsRzY38yGmVk1cDqwKOSYiq6US2IUa7FAESk/kU0W7r4NOA9YAvwFmO/uq8KNqvi6LokB\nxVoSQ6u/ikg2ItsNlYly7IaC4u9Op64ukcpWkQPc5ajQk+66K+ZigSJSnpQsIqqYlUla/VVEshXZ\nMQspHk3CE5FsacyigmkSnkhl0qqzIiKSVrnNsxARkYhQsqgwmognIrlQsqggmognIrnSmEWF0EQ8\nEemgMYuYKWWXUCnXnBKR8qNkEZJSdwmVas0pESlP6oYKQVhdQsVec0pE4kFrQ8VEWGszFXvNKREp\nX0oWIQhzbSbthiciudCYRQi0NpOIxI3GLEKktZlEJAxaG0pERNLSPAsRESkKJQsREUlLyUJERNJS\nsogprR4rIqWkZBFDWj1WREpN1VAxo9VjRSRfsaqGMrNvmNn/mNk2MxvX7WeXm9nLZrbKzI4NK8Yo\n0uqxIhKGMLuhXgS+Bvyx851mNgo4FRgFfBm4ycyyyoDlTKvHikgYQksW7r7G3V8GuieCk4D57r7V\n3ZuBl4EJpY4vqrRUiIiEIYoLCe4JPNnp+zcS90mCVo8VkVIrarIws6XA0M53AQ5c6e6/L8TvmDlz\n5vavGxoaaGhoKMRhI0+rx4pIphobG2lsbMzrGKFXQ5nZI8D33P3ZxPeXAe7u1yW+fwiY4e5/TvLc\niquGEhHJV6yqobrpHPQi4HQzqzazEcD+QFM4YYmICIRbOnuymb0GHArcb2YPArj7SuC/gJXAA8C5\naj6IiIQr9G6ofKgbSkQke3HuhpII0bpTItKdkoV0oXWnRCQZdUPJdlp3SqQyqBtK8qJ1p0QkFSUL\n2U7rTolIKkoWsp3WnRKRVDRmITtobW3VulMiZSyXMQslCxGRCqMBbhERKQolCxERSUvJQkRE0lKy\nEBGRtJQsREQkLSULERFJS8lCRETSUrIQEZG0lCxERCQtJQsREUlLyUJERNJSshARkbSULEREJC0l\nCxERSSu0ZGFm15vZKjN73sx+Y2Z1nX52uZm9nPj5sWHFKCIigTBbFkuAz7j7GOBl4HIAMzsYOBUY\nBXwZuMnMslp3PS4aGxvDDiEvij9ccY4/zrFD/OPPRWjJwt2XuXt74tungL0SX58IzHf3re7eTJBI\nJoQQYtHF/Q9O8YcrzvHHOXaIf/y5iMqYxRnAA4mv9wRe6/SzNxL3iYhISHoX8+BmthQY2vkuwIEr\n3f33icdcCbS5+z3FjEVERHIX6h7cZjYVmA4c7e6fJO67DHB3vy7x/UPADHf/c5LnawNuEZEcZLsH\nd2jJwsyOB/4vcKS7v9vp/oOBu4H/RdD9tBQ4wMPMaiIiFa6o3VBp3AhUA0sTxU5Pufu57r7SzP4L\nWAm0AecqUYiIhCvUbigREYmHqFRD5aynyX1RZmbHm9lqM3vJzC4NO55smNleZvawmf3FzF40s/PD\njilbZtbLzJ41s0Vhx5ItMxtoZvcm/u7/Ymb/K+yYsmFmF5rZ/5jZCjO728yqw46pJ2Y218zWmdmK\nTvcNNrMlZrbGzBab2cAwY+xJivizvm7GPlmQYnJflJlZL+AXwHHAZ4DJZjYy3KiyshW4yN0/AxwG\n/FvM4ge4gKCrM45+Djzg7qOAzwGrQo4nY2a2B/BdYJy7jyboCj893KjSup3g/2pnlwHL3P0g4GGi\nfd1JFn/W183YJ4seJvdF2QTgZXdf6+5twHzgpJBjypi7v+3uzye+3kBwsYrNXBgz2ws4Abg17Fiy\nlfgE+EV3vx0gMXn1w5DDytZOQD8z6w30Bd4MOZ4euftjwPvd7j4JuDPx9Z3AySUNKgvJ4s/luhn7\nZNHNGcCDYQeRge4TD18nRhfbzsxsODAG2KG0OcL+E7iYYM5P3IwA3jGz2xPdaLPNrDbsoDLl7m8S\nVEH+nWDC7QfuvizcqHKyq7uvg+DDE7BryPHkI6PrZiyShZktTfRvdtxeTPz71U6P6ZjcNy/EUCuK\nmfUHfg1ckGhhRJ6ZfQVYl2gZWeIWJ72BccD/c/dxwMcEXSKxYGaDCD6VDwP2APqb2T+HG1VBxPGD\nR1bXzTBLZzPm7pN6+nlict8JwNElCSh/bwD7dPp+r8R9sZHoQvg18Et3Xxh2PFk4HDjRzE4AaoEB\nZnaXu3875Lgy9Trwmrs/nfj+10CcCiSOAV519/cAzOy3wBeAuH3IW2dmQ919nZntBrSEHVC2sr1u\nxqJl0ZPE5L6LgRM7ZoHHwHJgfzMblqgEOR2IW1XObcBKd/952IFkw92vcPd93H1fgvP+cIwSBYmu\nj9fM7MDEXV8iXgP1fwcONbM+idWkv0Q8Bui7t0IXAVMTX08Bov6BqUv8uVw3Yz/PwsxeJpjc1zEL\n/Cl3PzfEkDKSeLN+TpCw57r7tSGHlDEzOxx4FHiRoPntwBXu/lCogWXJzI4CvufuJ4YdSzbM7HME\ng/NVwKvAd9x9fbhRZc7MZhAk6jbgOeDMRKFHJJnZPKAB2BlYB8wAfgfcC+wNrAVOdfcPwoqxJyni\nv4Isr5uxTxYiIlJ8se+GEhGR4lOyEBGRtJQsREQkLSULERFJS8lCRETSUrIQEZG0lCxECiixfPur\niWUtOpayftXM9kn3XJEoU7IQKSB3fx24Cbgucde1wC3u/vfwohLJnybliRRYYt2spwn2ETgTGOPu\n28KNSiQ/sVhIUCRO3H2rmV0CPAQco0Qh5UDdUCLFcQLBpj6HhB2ISCEoWYgUmJmNIVhN9VDgIjMb\nGnJIInlTshApvJsINoR6HbieYGc4kVhTshApIDObDqx194cTd90MjDSzL4YYlkjeVA0lIiJpqWUh\nIiJpKVmIiEhaShYiIpKWkoWIiKSlZCEiImkpWYiISFpKFiIikpaShYiIpPX/AbYVECAw8J1aAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fbe67244630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import linear_model\n",
    "import numpy as np\n",
    "\n",
    "# load data\n",
    "# generate data\n",
    "data_num = 100\n",
    "X = np.random.rand(data_num, 1)*10\n",
    "Y = X * 3 + 4 + 8*np.random.randn(data_num,1)\n",
    "\n",
    "print(\"X: \", X.shape)\n",
    "print(\"Y: \", Y.shape)\n",
    "\n",
    "# create regression model\n",
    "regr = linear_model.LinearRegression()\n",
    "regr.fit(X, Y)\n",
    "\n",
    "a, b = np.squeeze(regr.coef_), np.squeeze(regr.intercept_)\n",
    "\n",
    "print(\"a = %f, b = %f\" % (a, b))\n",
    "\n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n",
    "\n",
    "plt.scatter(X, Y)\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r')\n",
    "plt.xlabel(\"X\")\n",
    "plt.ylabel(\"Y\")\n",
    "plt.savefig(\"fig-res-sklearn_linear_fitting.pdf\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. 如何使用sklearn拟合多项式函数？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([800.,  90., -20.])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fitting polynomial functions\n",
    "\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "t = np.array([2, 4, 6, 8])\n",
    "\n",
    "pa = -20\n",
    "pb = 90\n",
    "pc = 800\n",
    "\n",
    "y = pa*t**2 + pb*t + pc\n",
    "\n",
    "model = Pipeline([('poly', PolynomialFeatures(degree=2)),\n",
    "                  ('linear', LinearRegression(fit_intercept=False))])\n",
    "model = model.fit(t[:, np.newaxis], y)\n",
    "model.named_steps['linear'].coef_\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 参考资料\n",
    "* [梯度下降法](https://blog.csdn.net/u010402786/article/details/51188876)\n",
    "* [如何理解最小二乘法？](https://blog.csdn.net/ccnt_2012/article/details/81127117)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}