machinelearning_notebook/4_logistic_regression/1-Least_squares.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 最小二乘\n",
    "\n",
    "## 1. 最小二乘的基本原理\n",
    "\n",
    "最小二乘法（Least Squares）是一种数学优化技术，它通过最小化误差的平方和找到一组数据的最佳函数匹配， 最小二乘法通常用于曲线拟合、求解模型。很多其他的优化问题也可通过最小化能量或最大化熵用最小二乘形式表达。\n",
    "\n",
    "![ls_theory](images/least_squares.png)\n",
    "\n",
    "最小二乘原理的一般形式为：\n",
    "$$\n",
    "L = \\sum (V_{obv} - V_{target}(\\theta))^2\n",
    "$$\n",
    "其中\n",
    "* $V_{obv}$是观测的多组样本值\n",
    "* $V_{target}$是假设拟合函数的输出值\n",
    "* $\\theta$为构造模型的参数\n",
    "* $L$是目标函数\n",
    "\n",
    "如果通过调整模型参数$\\theta$，使得$L$下降到最小则表明，拟合函数与观测最为接近，也就是找到了最优的模型。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 示例\n",
    "\n",
    "假设我们有下面的一些观测数据，我们希望找到他们内在的规律。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVZElEQVR4nO3df4xcV3nG8eepY2D50W7SrJCzwbVFkdMoEXbYRqFugTgFh4KKCS0tbVFaIdyq0EIAF4dWgkqtsiUhKVLVSIYEgppSaGIMSlJCFBshkJp2Hbt2ghOVQhyyMfGisgSoBYl5+8fcjb3jnfXO7Jz749zvR1p55u7s3tc38TNnzjn3HEeEAADt8TNVFwAAKBfBDwAtQ/ADQMsQ/ADQMgQ/ALTMGVUXsBRnn312rFmzpuoyAKBR9u7d+92IGOs+3ojgX7NmjaampqouAwAaxfbhhY7T1QMALUPwA0DLEPwA0DIEPwC0DMEPAC3TiFk9ANAEu/ZN69q7H9bjs8d0zuiItm1epy0bxqsu6xQEPwAMwa5907p650Ede+q4JGl69piu3nlQkmoX/nT1AMAQXHv3w8+E/pxjTx3XtXc/XFFFvRH8ADAEj88e6+t4lQh+ABiCc0ZH+jpeJYIfAIZg2+Z1Glm5Yt6xkZUrtG3zuoF+365909o4uVtrt9+pjZO7tWvf9DDKlMTgLgAMxdwA7jBm9aQeKE4W/LafI+krkp5dnOe2iPig7U9KeqWk7xcv/cOI2J+qDgAoy5YN40MJ5sUGimsd/JJ+LGlTRPzQ9kpJX7X9b8X3tkXEbQnPDQCNlXqgOFkff3T8sHi6sviKVOcDgFykHihOOrhre4Xt/ZKOSronIu4rvvW3tg/YvsH2s3v87FbbU7anZmZmUpYJALUy7IHibkmDPyKOR8R6SedKutj2BZKulnSepF+WdJak9/f42R0RMRERE2Njp2wgAwDZ2rJhXNdccaHGR0dkSeOjI7rmiguHdgdwKbN6ImLW9h5Jl0fEdcXhH9v+hKT3lVEDADTJsAaKF5KsxW97zPZo8XhE0qslPWR7VXHMkrZIeiBVDQCAU6Vs8a+SdIvtFeq8wXw2Iu6wvdv2mCRL2i/pTxLWAADokiz4I+KApA0LHN+U6pwAgNNjyQYAaBmCHwBahuAHgJYh+AGgZQh+AGgZlmUG0ApN2QhdSl8rwQ8ge03aCL2MWunqAZC9Jm2EXkatBD+A7DVpI/QyaiX4AWSvSRuhl1ErwQ+gcfrdiDz1+vbDVEatDO4CaJRBBj+HuRF6amXU6oj674Y4MTERU1NTVZcBoAY2Tu7W9AL93eOjI/radtaAPJntvREx0X2crh4AjdKkgdq6IvgBNEqTBmrriuAH0ChNGqitKwZ3ATRKkwZq64rgB9A4KTcibwO6egCgZQh+AGiZZMFv+zm2/8P2f9l+0PZfF8fX2r7P9jdsf8b2s1LVAAA4VcoW/48lbYqIl0paL+ly25dI+jtJN0TEL0r6nqS3JawBANAl2eBudG4J/mHxdGXxFZI2Sfq94vgtkj4k6cZUdQDAnCZtxpJS0j5+2yts75d0VNI9kv5H0mxEPF285DFJC15121ttT9mempmZSVkmgBaYW+NnevaYQifW+DndAm85Shr8EXE8ItZLOlfSxZLO6+Nnd0TERERMjI2NpSoRQEs0aTOW1EqZ1RMRs5L2SHq5pFHbc11M50pq39stgNKxxs8JKWf1jNkeLR6PSHq1pEPqvAH8VvGyKyV9PlUNADCHNX5OSNniXyVpj+0Dkv5T0j0RcYek90t6j+1vSPp5STclrAEAJLHGz8lSzuo5IGnDAse/qU5/PwCUhjV+TmCtHgCtMcgaPzlOASX4AaCHQbZ5bAKCHwC0cMt+sSmgBD+QsRw/6mO+Xi377tCf0/QpoAQ/sIhcP+rXTdVvrr1a9itsHY845fVNnwJK8AOLyPWjfh3Mhf307DFZnYW8pGreXHu14I9HaGTlinn/D+QwBZT1+IFFcLdnGn+166Cu+sx+TRfXsbtNXfZSCr1a8OOjI7rmigs1Pjoin/S86W/6tPiBRZwzOvJMOHUfx2B27ZvWrf/+6Clh363MN9dtm9ed0qc/17LPcZtHgh9YxGKBgNPrNVPmdKEvlfvm2rabuwh+YBFtC4Rh6nemzMmqeHPNsWXfC8EPnEadAqHq2S/96HemzJzxmv+9ckDwAw3RtKml/cyUsaTfv2S1/mbLhSVV127M6gEqtGvftDZO7tba7Xdq4+TuRXeDatpGIv3MlLnhd9YT+iWixQ9UpN8WfNOmlrZtpkyT0OIHKtJvC75pG4ls2TCe5Rz4HNDiByrSbwu+iVNLadnXEy1+oCL9tuBpQWNYaPEDFRmkBU8LGsNA8AMV4eYwVIXgBypECx5VSNbHb/tFtvfY/rrtB22/qzj+IdvTtvcXX7+RqgYAwKlStviflvTeiLjf9gsk7bV9T/G9GyLiuoTnBpCRJi1V0QTJgj8ijkg6Ujz+ge1DkvgvBaAvTVuqoglKmc5pe42kDZLuKw690/YB2zfbPrPHz2y1PWV7amZmpowyAdRQ05aqaILkwW/7+ZJul/TuiHhS0o2SXixpvTqfCD6y0M9FxI6ImIiIibGxsdRlAqippi1V0QRJZ/XYXqlO6N8aETslKSKeOOn7H5N0R8oaANRLv/31Ze6C1paxhJSzeizpJkmHIuL6k46vOullb5T0QKoaANTLXH/99OwxhU701y+2Kum2zes0snLFvGMplqoYpLamStnVs1HSWyVt6pq6+WHbB20fkHSppKsS1gCgRgbpry9rqYo2jSWknNXzVXX2V+h2V6pzAqi3Qfvry7jRrU1jCSzSBqA0dV5aus61DRvBD6A0ZfXXD6LOtQ0ba/UAKE2dF6arc23D5lhkt/u6mJiYiKmpqarLAIBGsb03Iia6j9PVAwAtQ/ADQMsQ/ADQMgzuIittueW+7fjvvDwEP7Kx1OV76x4ada+vaizTvHx09SAbS7nlvu7rsdS9vjpo09IKqRD8yMZSbrmve2jUvb46aNPSCqkQ/MjGUm65r3to1L2+OmjT0gqpEPzIxlJuua97aAyzvl37prVxcrfWbr9TGyd3Z9Nd1KalFVIh+JGNpSzfW/fQGFZ9OY8VlLVMc85YsgGtU/dZM8Oob+Pk7gV3rRofHdHXtm8aVqmouV5LNhD8QE0t5w1g7fY7tdC/bEv61uTrhlon6qtX8DOPH1imFJ8gljtXvcx9atE89PEDhUEGQ1P1pS93WmfdxzJQLYIf0OABnmre/XKndTIAisUk6+qx/SJJn5L0QkkhaUdEfNT2WZI+I2mNpEckvTkivpeqDuRtWN0siwX4Yr8v1bz7YXTVlLFPLZopZYv/aUnvjYjzJV0i6R22z5e0XdK9EfESSfcWz4G+DbObZdAAT3VfAF01SClZ8EfEkYi4v3j8A0mHJI1LeoOkW4qX3SJpS6oakLdhdrMMGuCpApquGqRUyqwe22skbZB0n6QXRsSR4lvfUacrCOjbMLtZtm1eN28WjbS0AE+5TytdNUglefDbfr6k2yW9OyKetP3M9yIibC94I4HtrZK2StLq1atTl4kGGuaUxaUEeK/xBAIaTZP0Bi7bKyXdIenuiLi+OPawpFdFxBHbqyR9OSIWbVZxAxcW0j3XXeq00lN0iZR5LmBYSt9s3Z2m/U2SDs2FfuELkq4sHl8p6fOpakDeyuwHZ7lk5KRnV4/tuyT9aUQ8MuDv3ijprZIO2t5fHPuApElJn7X9NkmHJb15wN8PlNbNwnLJyMliffyfkPQl27dI+nBEPNXPL46Ir6qzNMhCLuvnd6G96rKgWtOWQKjLdUM99ezqiYh/lXSRpJ+VNGX7fbbfM/dVWoVorTotLdykefV1um6op9P18f9E0o8kPVvSC7q+gKTq1K/epHn1dbpuqKfF+vgvl3S9OoOxF0XE/5VWFQaS28f7KvvVF5u6WXeMR+B0Fuvj/0tJvx0RD5ZVDAa33GV866iqfvWmX8umjUegfIv18f8aod8cOX68X6hf3ZIuPW8s6Xmbfi2bNB6BarAscyZy/Hi/ZcO43vSy8XlTw0LS7Xunkw5UNv1aNmk8AtVgB65M5Prxfs9DM6dsIbiU5ZKXI4dr2ZTxCFSDFn8mcv14X0XrO9drCcyhxZ+JlKtEVqmK1neu1xKYk3SRtmFhkbb2YnE0YHC9FmmjxY9ao/UNDB/Bj9pjoBIYLgZ3AaBlCH4AaBmCHwBahuAHgJYh+AGgZQh+AGgZgh8AWobgB4CWSXYDl+2bJb1e0tGIuKA49iFJb5c0U7zsAxFxV6oa0D5N3IWsiTWj2VLeuftJSf8g6VNdx2+IiOsSnhct1cSds5pYM5ovWVdPRHxF0v+m+v1AtybunNXEmtF8VfTxv9P2Ads32z6zgvMjU03cOauJNaP5yg7+GyW9WNJ6SUckfaTXC21vtT1le2pmZqbXy4Bn9Fqjv847ZzWxZjRfqcEfEU9ExPGI+Kmkj0m6eJHX7oiIiYiYGBtLu7k28tDEnbOaWDOar9RlmW2viogjxdM3SnqgzPMjb01cu7+JNaP5ku3AZfvTkl4l6WxJT0j6YPF8vaSQ9IikPz7pjaAnduBCnTD9Ek1R+g5cEfGWBQ7flOp8QBmYfokccOcu0AemXyIHBD/QB6ZfIgcEP9AHpl8iBwQ/0AemXyIHpU7nBJqO6ZfIAcHfUGVOKWT64nxbNoy3+u+P5iP4SzSsAC1zSiHTF4H80MdfkrkAnZ49ptCJAN21b7rv31XmlEKmLwL5ocVfksUCtN+Wc5lTCnOavkiXFdBBi78kwwzQMqcU5jJ9cZifuICmI/hLMswALXNK4ULnsqRLz2vWiql0WQEnEPwL2LVvWhsnd2vt9ju1cXL3UFqFwwzrLRvGdc0VF2p8dESWND46omuuuDBJt8WWDeN608vG5ZOOhaTb9043qrWcU5cVsFz08XdJNYtl2PO/y5xSuOehGXWv4Tro+ERVzhkd0fQCId+0LitgGAj+LsMchO3W1PnfObSWt21eN+8NXeKOW7QXXT1dcgi5YcthgLfM7jGg7mjxd6FL4FS5tJab+okLGLZsg3/QOdu5hFwvg1wX1qcB8pJl8C9ngDbnkFvudcnhGgDINPiXO0Cba8ilHLgG0BxZDu4yQLswrgsAKWHw277Z9lHbD5x07Czb99j+7+LPM1OcO4dZKClwXQBIaVv8n5R0edex7ZLujYiXSLq3eD507JK0MK4LAClhH39EfMX2mq7Db5D0quLxLZK+LOn9wz53zgO0y8F1ASBJjui+GX+Iv7wT/HdExAXF89mIGC0eW9L35p4v8LNbJW2VpNWrV7/s8OHDyeoEgBzZ3hsRE93HKxvcjc47Ts93nYjYERETETExNtaslSABoM7KDv4nbK+SpOLPoyWfHwBar+x5/F+QdKWkyeLPz5d8fiwRu1UB+UoW/LY/rc5A7tm2H5P0QXUC/7O23ybpsKQ3pzo/BscG60DeUs7qeUuPb12W6pwYDu7wBfKW5Z27WB7u8AXyluVaPW00zD55lqYG8kaLPwNzffLTs8cUOtEnP+ieuEu5wzfFvsQAykHwZ2CxPvlBnG63qmG/0QAoF109GUjRJ7/Y0tQM/gLNRos/A2WvusngL9BsBH8Gylp1c65fv9c6Gwz+As1AV08Gylh1s/umrm4s7ww0B8GfidTbRS7Urz9nnCUdgEYh+LEkvfrvLelr2zeVWwyAZaGPH0vCto1APgh+LAnbNgL5oKsHS8K2jUA+CH4sWeoBZADloKsHAFqGFn8P7EAFIFcE/wLYgQpAzujqWcCwV7sEgDoh+BfAImQAckbwL4CblQDkrJLgt/2I7YO299ueqqKGxXCzEoCcVTm4e2lEfLfC8/fEzUoAcsasnh64WQlArqrq4w9JX7K91/bWhV5ge6vtKdtTMzMzJZcHAPmqKvh/NSIukvRaSe+w/YruF0TEjoiYiIiJsbGx8isEgExVEvwRMV38eVTS5yRdXEUdANBGpQe/7efZfsHcY0mvkfRA2XUAQFtVMbj7Qkmfsz13/n+OiC9WUAcAtFLpwR8R35T00rLPCwDo4M5dAGgZgh8AWobgB4CWIfgBoGUIfgBoGYIfAFqG4AeAliH4AaBlCH4AaJns1+PftW+aDVUA4CRZB/+ufdO6eudBHXvquCRpevaYrt55UJIIfwCtlXVXz7V3P/xM6M859tRxXXv3wxVVBADVyzr4H5891tdxAGiDrIP/nNGRvo4DQBtkHfzbNq/TyMoV846NrFyhbZvXVVQRAFQv68HduQFcZvUAwAlZB7/UCX+CHgBOyLqrBwBwKoIfAFqG4AeAliH4AaBlCH4AaBlHRNU1nJbtGUmHq66jBs6W9N2qi6gRrsd8XI/5uB7SL0TEWPfBRgQ/OmxPRcRE1XXUBddjPq7HfFyP3ujqAYCWIfgBoGUI/mbZUXUBNcP1mI/rMR/Xowf6+AGgZWjxA0DLEPwA0DIEf83ZfpHtPba/bvtB2++quqY6sL3C9j7bd1RdS9Vsj9q+zfZDtg/ZfnnVNVXJ9lXFv5UHbH/a9nOqrqluCP76e1rSeyPifEmXSHqH7fMrrqkO3iXpUNVF1MRHJX0xIs6T9FK1+LrYHpf055ImIuICSSsk/W61VdUPwV9zEXEkIu4vHv9AnX/Urd5gwPa5kl4n6eNV11I12z8n6RWSbpKkiPhJRMxWWlT1zpA0YvsMSc+V9HjF9dQOwd8gttdI2iDpvopLqdrfS/oLST+tuI46WCtpRtIniq6vj9t+XtVFVSUipiVdJ+lRSUckfT8ivlRtVfVD8DeE7edLul3SuyPiyarrqYrt10s6GhF7q66lJs6QdJGkGyNig6QfSdpebUnVsX2mpDeo84Z4jqTn2f6DaquqH4K/AWyvVCf0b42InVXXU7GNkn7T9iOS/kXSJtv/VG1JlXpM0mMRMfcp8DZ13gja6tclfSsiZiLiKUk7Jf1KxTXVDsFfc7atTv/toYi4vup6qhYRV0fEuRGxRp1Bu90R0doWXUR8R9K3ba8rDl0m6esVllS1RyVdYvu5xb+dy9Tiwe5est9sPQMbJb1V0kHb+4tjH4iIu6orCTXzZ5Jutf0sSd+U9EcV11OZiLjP9m2S7ldnRtw+sXTDKViyAQBahq4eAGgZgh8AWobgB4CWIfgBoGUIfgBoGYIf6FOxYuq3bJ9VPD+zeL6m4tKAJSH4gT5FxLcl3Shpsjg0KWlHRDxSWVFAH5jHDwygWEZjr6SbJb1d0vpiiQCg9rhzFxhARDxle5ukL0p6DaGPJqGrBxjca9VZ+veCqgsB+kHwAwOwvV7Sq9XZFe0q26uqrQhYOoIf6FOx6uON6uyN8Kika9XZ/ANoBIIf6N/bJT0aEfcUz/9R0i/ZfmWFNQFLxqweAGgZWvwA0DIEPwC0DMEPAC1D8ANAyxD8ANAyBD8AtAzBDwAt8/8F/I4RSbbD3wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 生成数据\n",
    "data_num = 50\n",
    "X = np.random.rand(data_num, 1)*10\n",
    "Y = X * 3 + 4 + 4*np.random.randn(data_num,1)\n",
    "\n",
    "# 画出数据的分布\n",
    "plt.scatter(X, Y)\n",
    "plt.xlabel(\"X\")\n",
    "plt.ylabel(\"Y\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 数学原理\n",
    "有$N$个观测数据为：\n",
    "$$\n",
    "\\mathbf{X} = \\{x_1, x_2, ..., x_N \\} \\\\\n",
    "\\mathbf{Y} = \\{y_1, y_2, ..., y_N \\}\n",
    "$$\n",
    "其中$\\mathbf{X}$为自变量，$\\mathbf{Y}$为因变量。\n",
    "\n",
    "我们希望找到一个模型能够解释这些数据，假设使用最简单的线性模型来拟合数据：\n",
    "$$\n",
    "y = ax + b\n",
    "$$\n",
    "那么问题就变成求解参数$a$, $b$能够使得模型输出尽可能和观测数据有比较小的误差。\n",
    "\n",
    "如何构建函数来评估模型输出与观测数据之间的误差是一个关键问题，这里我们使用观测数据与模型输出的平方和来作为评估函数（也被称为损失函数Loss function):\n",
    "$$\n",
    "L = \\sum_{i=1}^{N} \\{y_i - (a x_i + b)\\}^2 \\\\\n",
    "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2 \n",
    "$$\n",
    "\n",
    "使误差函数最小，那么我们就可以求出模型的参数:\n",
    "$$\n",
    "\\frac{\\partial L}{\\partial a} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i \\\\\n",
    "\\frac{\\partial L}{\\partial b} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b)\n",
    "$$\n",
    "\n",
    "即当偏微分为0时，误差函数为最小，因此我们可以得到:\n",
    "$$\n",
    "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i = 0 \\\\\n",
    "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) = 0 \\\\\n",
    "$$\n",
    "\n",
    "将上式调整一下顺序可以得到：\n",
    "$$\n",
    "a \\sum x_i^2 + b \\sum x_i = \\sum y_i x_i \\\\\n",
    "a \\sum x_i + b N = \\sum y_i\n",
    "$$\n",
    "\n",
    "上式中$\\sum x_i^2$, $\\sum x_i$, $\\sum y_i$, $\\sum y_i x_i$都是已知的数据，而参数$a$, $b$是我们想要求得未知参数。通过求解二元一次方程组，我们即可求出模型的最优参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 求解程序"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a = 2.951846, b = 4.837623\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAl2ElEQVR4nO3deXhU5d3/8fdXjBIBAQUpghbaKiCLBIPSxg2o4CMugI+t1gWsFmvrhgpF+yju0opL+/yqfagbKrgBRkUoVQERVJAlhl1RgxKpIoqKQoVw//44MyEJWSaZM3POmfm8rouL5GQy8+Vw5TN37tWcc4iISPTsFXQBIiLSMApwEZGIUoCLiESUAlxEJKIU4CIiEbV3Ol+sVatWrkOHDul8SRGRyFuyZMnnzrnWVa+nNcA7dOjA4sWL0/mSIiKRZ2brq7uuLhQRkYhSgIuIRJQCXEQkotLaB16dHTt2sGHDBrZv3x50KVmtcePGtG/fnpycnKBLEZEEBR7gGzZsoFmzZnTo0AEzC7qcrOScY/PmzWzYsIGOHTsGXY6IJCjwAN++fbvCO2BmxoEHHsimTZuCLkUkFAqXlXLXrLV8smUbB7fIZdTATgzOaxd0WXsIPMABhXcI6P9AxFO4rJTrpi1n244yAEq3bOO6acsBQhfiGsQUEangrllry8M7btuOMu6atTagimqmAPdZhw4d+Pzzz5N+jIgE45Mt2+p1PUgKcBGRCg5ukVuv60FSgAMlJSV07tyZ4cOHc/jhh3PuuefyyiuvUFBQwGGHHcaiRYv44osvGDx4MD169KBPnz4UFxcDsHnzZgYMGEDXrl25+OKLqXjC0RNPPMHRRx9Nz549ueSSSygrK6upBBEJiVEDO5Gb06jStdycRowa2KlBz1e4rJSCcbPpOOYlCsbNpnBZqR9lAiEZxCx31VVQVOTvc/bsCffdV+fD1q1bx7PPPsvDDz9M7969mTx5MvPnz+eFF17gjjvu4JBDDiEvL4/CwkJmz57NBRdcQFFRETfffDPHHnssN954Iy+99BIPPfQQAKtXr+bpp59mwYIF5OTk8Lvf/Y5JkyZxwQUX+PvvExFfxQcq/ZiFkuoB0ToD3MwaA/OAfWOPn+KcG2tmjwInAF/FHjrcOVeUdEUB6dixI927dwega9eu9O/fHzOje/fulJSUsH79eqZOnQpAv3792Lx5M19//TXz5s1j2rRpAAwaNIiWLVsC8Oqrr7JkyRJ69+4NwLZt2zjooIMC+JeJSH0NzmvnS8DWNiCalgAH/gP0c85tNbMcYL6ZzYx9bZRzbkrSVcQl0FJOlX333bf847322qv887322oudO3fWe4Wic45hw4Zx5513+lqniERHqgdE6+wDd56tsU9zYn+y7ij74447jkmTJgEwd+5cWrVqxf7778/xxx/P5MmTAZg5cyZffvklAP3792fKlCl89tlnAHzxxResX1/tjpAikqFSPSCa0CCmmTUysyLgM+Bl59zC2JduN7NiM7vXzPat4XtHmNliM1sc5ZV+N910E0uWLKFHjx6MGTOGiRMnAjB27FjmzZtH165dmTZtGoceeigARxxxBLfddhsDBgygR48enHTSSWzcuDHIf4KIpJnfA6JVWcVZE3U+2KwF8BxwObAZ+DewDzABeN85d0tt35+fn++qHuiwevVqunTpUr+qJSX0fyHiPz+W5ZvZEudcftXr9ZqF4pzbYmZzgJOdc+Njl/9jZo8A19arIhGRLODXgGh16uxCMbPWsZY3ZpYLnASsMbO2sWsGDAZWpKRCERGpViIt8LbARDNrhBf4zzjnppvZbDNrDRhQBPw2dWWKiEhVdQa4c64YyKvmer+UVCQiIgnRUnoRkYhSgIuIRJQCvB5OOeUUtmzZUutjbrzxRl555ZUGPf/cuXM59dRT63zciSeeSNXpmFXdd999fPfddw2qQ0SiQQGeAOccu3btYsaMGbRo0aLWx95yyy38/Oc/T09htVCAi2S+yAV4KrZmvOeee+jWrRvdunXjvth+LCUlJXTq1IkLLriAbt268fHHH1c6iOHWW2+lU6dOHHvssZxzzjmMH+9Nix8+fDhTpnjbw3To0IGxY8fSq1cvunfvzpo1awBYtGgRP/3pT8nLy+NnP/sZa9fWftLHtm3bOPvss+nSpQtDhgxh27bd+yhceuml5Ofn07VrV8aOHQvAX//6Vz755BP69u1L3759a3yciERbuLaTrUMqtmZcsmQJjzzyCAsXLsQ5xzHHHMMJJ5xAy5Ytee+995g4cSJ9+vSp9D1vv/02U6dO5Z133mHHjh306tWLo446qtrnb9WqFUuXLuX+++9n/PjxPPjgg3Tu3JnXX3+dvffem1deeYXrr7++fKfD6jzwwAPst99+rF69muLiYnr16lX+tdtvv50DDjiAsrIy+vfvT3FxMVdccQX33HMPc+bMoVWrVjU+rkePHg26ZyJBisqBw7C71n9/sZUfHNDU91oj1QJPxVl18+fPZ8iQITRp0oSmTZsydOhQXn/9dQB++MMf7hHeAAsWLOCMM86gcePGNGvWjNNOO63G5x86dCgARx11FCUlJQB89dVXnHXWWXTr1o2RI0eycuXKWmucN28e5513HgA9evSoFLzPPPMMvXr1Ii8vj5UrV7Jq1apqnyPRx4mEWbwRV7plG47djTg/D0nwS+GyUm58diknzJ3G3AkjaPLeat9rjVSAp/usuiZNmiT9HPFtaRs1asTOnTsBuOGGG+jbty8rVqzgxRdfZPv27Q167g8//JDx48fz6quvUlxczKBBg6p9rkQfJxJ2kTlw+PvvefeW8cz420XcMetvfNakJTllO32vNVIBnoqtGY877jgKCwv57rvv+Pbbb3nuuec47rjjav2egoKC8uDdunUr06dPr9drfvXVV7Rr5/0a9eijj9b5+Ipb1q5YsaL8OLevv/6aJk2a0Lx5cz799FNmzpxZ/j3NmjXjm2++qfNxIlES+gOHv/8eJkyAww5jdOF9fNb0AM7/xS2ced5drPzBTwB/a41UH/iogZ0q9YFD8lsz9urVi+HDh3P00UcDcPHFF5OXl1fe3VGd3r17c/rpp9OjRw/atGlD9+7dad68ecKvOXr0aIYNG8Ztt93GoEGD6nz8pZdeyoUXXkiXLl3o0qVLeX/7kUceSV5eHp07d+aQQw6hoKCg/HtGjBjBySefzMEHH8ycOXNqfJxIlBzcIpfSagIw8AOHv/8eHn0Ubr8dPvoI+vRhZP/f81zrI8Cs0kP9rLVe28kmy4/tZMMygLF161aaNm3Kd999x/HHH8+ECRMqDS5GkbaTlXSr789z1YkM4DXi7hzaPZiBzKrBfcwxcPPNMGAAhUWf+FarL9vJhkEqt2asjxEjRrBq1Sq2b9/OsGHDIh/eIunWkFllfh44nJTvv4eJE73gXr/eC+4JE2DAgPIWdzpqjVyAh0W8T1pEGqahB/4G2oirLrj//ncYOHCPrhJIfa2hGMRMZzeOVE//B5JuoR+QrGjHDnjwQejUCUaMgDZtYOZMePNNOPnkasM7HQIP8MaNG7N582YFSICcc2zevJnGjRsHXYpkkVQf+OuLeHAffjj85jdw0EEwYwa89VagwR0XeBdK+/bt2bBhA1E+8DgTNG7cmPbt2wddhmSRVMwq882OHfDYY3DbbVBSAr17w/33hyK0Kwo8wHNycujYsWPQZYhImoVmQLKi6oL7b3+D//qvUAV3XOABLiLZKyyzytixAx5/3AvuDz+E/PxQB3dc4H3gIiKB2bEDHn7YG5y86CI48ECYPh0WLYJTTgl1eIMCXESy0Y4d8Mgj0LnznsE9aFDogzuuzgA3s8ZmtsjM3jGzlWZ2c+x6RzNbaGbrzOxpM9sn9eWKiCShYnD/+tfQsiW8+GLkgjsukRb4f4B+zrkjgZ7AyWbWB/gTcK9z7ifAl8BFKatSRCQZO3d6S967dKkc3G+/DaeeGrngjqtzENN5E7S3xj7Nif1xQD/gV7HrE4GbgAf8L1FEpLKE91DZuROeeMIbnHz/fejVC154IdKhXVFCfeBm1sjMioDPgJeB94EtzrmdsYdsAKodSjazEWa22MwWa663iCQroUMddu70lrx37gwXXgjNm3vBvXgxnHZaRoQ3JBjgzrky51xPoD1wNNA50Rdwzk1wzuU75/Jbt27dsCpFRGJqPdShYnAPHw777w/PP59xwR1Xr3ngzrktZjYH+CnQwsz2jrXC2wPhO9NIRDJOdXulNNpVRp/50+GhS2DdOsjL84I7A0O7ojoD3MxaAzti4Z0LnIQ3gDkH+G/gKWAY8HwqCxURgcqHOjTaVcYZq+Zy+RtP0fHLjdCzJxQWwumnZ3RwxyXShdIWmGNmxcDbwMvOuenAH4CrzWwdcCDwUOrKFBHxjBrYiaaNYMiK2bz84KXc89K9bNsnl7fueQiWLoUzzsiK8IbEZqEUA3nVXP8Arz9cRCQ9du5k8IrZ/PyJsTT96ENWHdSRMefdTJ+Rv2Zwr+zbjE17oYhI+O3cCU89BbfeCu++S9Mjj4S/PMcRp5/OuL0SW1AeluMY/aQAF5HwKiuDJ58sD2569IBp07xukgSDGxp2fFsUKMBFJHzKyrwW9y231Du4q2tpN/T4trBTgEvWyMRfoTNOPLhvvRXWrvWCe+pUGDw4oRZ3TS3tquEdF8rj2+pBAS5ZIVN/hQ6bBr9JlpXB0097Le61a6F793oFd1xNLe1GZpRVc2xjqI5vawAFuGSFTP0VOgzioV26ZRuGt1ESJPgmWV1wT5kCQ4bUK7jjampRlzlHbk6jcB7flgTtBy5ZIVInoEfI/xQuZ+TTReULa6q2ccuXuFcVH5zs1g3OPRf22ccL7qIiOPPMBoU31NyibtcilzuHdqddi1yswudRf/NWC1yyQsXVe1WvS8MULitl0lsf7RHaVVV6kywrg2ee8Vrca9Z4AZ5Ei7uq2g5KDs3xbT5SgEtWCPUJ6BFQ08yOusIbYm+SZWXw7LNecK9e7QX3s8/C0KG+BHdcKA9KTiEFuGSFbPvB9lN9Z3ZU1KQR3Mca6H5ZSoO7okxsaddEAS5ZI0w/2FGa0ljfmR0Ae+0q47wNb3PtwmfY/4N3oWtXr+skif5t2ZMCXCTNojalsT4zO/Zyu7ijbA1nz3wUVq1ScKeY7qiIDwqXlVIwbjYdx7xEwbjZlU+HqaLWAwlCKJGZHXu5XZz/0UKKplzL2XeP9h7w9NNQXAxnnaXwThHdVZEkJXTEVwVRm9I4amAncnMaVbpWPrPjyLYs+NHnfPDSddz65K3s33hvL7iXL4df/ELBnWK6uyJJqm+LuqYWbVinNA7Oa7fnHOrBXRm87g1vqfsvfwm7dnlL4IuLFdxppD5wkSTVt0UdxSmN5QPAu3Z5S9yHXQErV0KXLl5w//d/Q6NGdT+R+EpvkyJJqm+LutoWbdhXBe7a5S24OfJIr4W9a5e3knL5cq8FrvAOhFrgIklqSIs6TFMaa7Vrl7eN6803w4oV3mnvTz7pDUwqtAOnFrhIkiLZoq5LxRb3WWd5J+JMnuyF+NlnK7xDQi1wER9EpkVdl1274LnnvBb38uVei3vyZK/bRKEdOnW2wM3sEDObY2arzGylmV0Zu36TmZWaWVHszympL1dEUiI+ONmzpzcg+f33MGmS1+I+5xyFd0gl0gLfCVzjnFtqZs2AJWb2cuxr9zrnxqeuPBFJqV27oLDQa3EXF0OnTl5wp2hgMkpbCERBnQHunNsIbIx9/I2ZrQZ0x0WirGpwH344PPFESvu3o7aFQBTUaxDTzDoAecDC2KXLzKzYzB42s5Y1fM8IM1tsZos3bdqUXLUikpx4H3evXt7+JNu3e8G9apV3sEIKu0qitoVAFCQc4GbWFJgKXOWc+xp4APgx0BOvhX53dd/nnJvgnMt3zuW3bt06+YpFpP4qBvfQofDdd/D4495inBQHd1zUthCIgoRmoZhZDl54T3LOTQNwzn1a4ev/AKanpEIRaTjn4Pnnva6SoiI47DAvuM8+G/ZObhJaffuz03kqUrb0tScyC8WAh4DVzrl7KlxvW+FhQ4AV/pcnIg3inNfH3auXd1zZt9/CY495XSXnnedLeNdnAy+ofVMsPzWktqhKpAulADgf6FdlyuCfzWy5mRUDfYGRqSxURBIQb3HHg3vr1t3Bff75SQd3XEP6s9O14Cmb+toTmYUyH7BqvjTD/3JEpEGcgxde8LpKli2Dn/wEJk6EX/3Kt9CuqKH92elY8JRNfe1aSi8SZfHgPuooGDwYvvnGC+7Vq+GCC1IS3hDuLXHDXJvfFOAiUVQxuM84A77+Gh59NOXBHZeu/uyGCHNtftNeKCJR4hxMnw433QRLl8KPf+wF97nnpjy0K4p3g4RxpkeYa/ObuRpOlU6F/Px8t3jx4rS9nkjGqBrcP/oR3HCDLzNKJPzMbIlzLr/qdXWhiIRZPLh794bTT4ctW+CRR2DNGhg+XOGd5RTgImFUMbhPOw2+/BIefnh3cOfkBF2hhIACXCRMnIOXXoKjj/aC+4svdgf3hRcquKUS/f4loZQtS6HLOQczZnh93IsXQ8eO8NBD3uKbDA7trPt/9pkCXEIn0W1Hw/7Dn1B9zsHMmV5wv/121gQ3aHtZP6gLRUInkaXQYd/vos764i3uY46BQYNg0yZ48EFYuxZ+/euMD2/IriXvqaIAl9BJZCl02H/4a6zvn2u8FnefPpWD+9134aKLsiK447JpyXuqqAtFQieRbUfD/sO/Rx3OceIHS7hqwWTY+C788Ifwj394qyb32SeYIgOWzu1lM5Va4BI6iSyFDvt+F+V1OMcJHyzhucev5dEpN9Fm21decL/7Llx8cULhXbislIJxs+k45iUKxs0OTTdRsrJpyXuqKMAldBLZdjTsP/yjBhzOSR8t47nHr2Xis2Np/e2X3HjKFSya9WbCwQ3h7+tPRrq2l81kWkovkRXKWSjOwb/+5c0qeestNrZow1+POYs3jj2VkYO61bu+gnGzq+1maNcilwVj+vlUtIRdTUvp1QcukZWOvaUTViW4OfRQmDCBtsOGcczKTcybtZaRTxdx16y19XqjCXtfvwRLAS4S06AWvXPw8stecL/5phfc//d/3nL3ffZJeq6zBvqkNuoDl4zTkEG/evc1x1vcBQUwcCCUlnrB/d57MGJEeR93stMdw97XL8FSgEtGaeigX8JBG29xH3usF9wbNsDf/75HcMcl2wWigT6pTZ1dKGZ2CPAY0AZwwATn3F/M7ADgaaADUAL8wjn3ZepKlUzm14BkbUFc2/PVGbTOwSuveF0lb7wBhxziBffw4bDvvjU+rx9dIKHq65dQSaQFvhO4xjl3BNAH+L2ZHQGMAV51zh0GvBr7XKTe/Jwq19AWb43zyps39oL7uONgwAD4+GN44AGvxX3JJbWGN6gLRFKrzgB3zm10zi2NffwNsBpoB5wBTIw9bCIwOEU1Sobzc1l8Qxf47BG0ztHv42JeeOY6OOkkWL9+d3D/9rd1BnecukAkleo1C8XMOgB5wEKgjXNuY+xL/8brYhGpNz+nyo0a2KnSrA9IrMVbfo7iP9fQ4Z23GL3wKY4sWQHt28P993sbTCUY2tU9twJbUiHhADezpsBU4Crn3NdmVv4155wzs2pXBJnZCGAEwKGHHppctZKR/Jwql8iBttX2t/c8mMFfrGHwjJtg/nxfglsk1RJaiWlmOcB0YJZz7p7YtbXAic65jWbWFpjrnKu1maOVmFKdqnOlwWs1p6KrYY/Xco4TS1cwflUhrZYthHbt4PrrvZ0BFdwSEg1eiWleU/shYHU8vGNeAIYB42J/P+9TrZJlEmk1+6W8v905fvpRMVfNn8wxG1by2f6t4G9/U3BLpCTShVIAnA8sN7Oi2LXr8YL7GTO7CFgP/CIlFUpWSFc/8SdfflcpuDc2PZAbTvotz/QYwNrfDUn564v4qc4Ad87NB6yGL/f3txzJVIFvPOUczJ3LtGf/SN6Hxfy76QHlwf2fvfehXUiXpgd+3yTUtBeKpFzgZx/OmeMtwJk3j86t23DrwEt5ottJ/Gdvb9VkWOdlB37fJPS0lF5SLrDjz+bOhRNPhH79YN06+N//JfejErrf+UdatWoe+nnZYT82ToKnFngIZdqvzWnfEnXuXK/F/dprbGvVhr+fdhl/P7wfrbY2Z9TqzZGZl62tZKUuCvCQycRfm9O2Jeprr3nBPXcutG1L8ehbOH+vI/nKeSsso3YvtZWs1EVdKCGTib82V7cfiAF9O7f25wVeew369vW6S9auhb/8Bd5/n0tbFpSHd1yU7qX2UZG6KMBDJhN/bR6c144zj2pXaSqTA6YuKU3ubMd587z+7RNPhDVryoObK66A3NzI30vtoyJ1URdKyGTqr81z1myi6prfRLZ5rda8eV5XyZw58IMfwH33eXtx51a+R5lwL6PSXy/BUAs8ZDL112ZfWsOvvw79+8MJJ8Dq1V5wf/ABXHnlHuENmXsvReLUAg+ZdC4rT6ekWsOvv+61uGfPhjZt4N57vb24qwntijL1XorEJbSZlV+0mVX2atCGVfPne8H96qtecI8Z43WV7LdfeooWCYkGb2Yl4od6tYarBvc993gtbgW3SCUKcEmbOgfkFizwgvuVVxTcIgnQIKYEb8EC79iyY4+F4mK4+25vcHLkSIW3SC0U4BKcN97wDgquGNwffghXX63gFkmAAlzSLx7cBQXwzjswfrzX4lZwi9SL+sAlfd580+vj/te/oHVrL7h/+1to0iToykQiSQEuqVc1uO+6Cy69VMEtkiQFuKTOm2/CzTfDrFkKbpEUUICL/956y2txz5oFrVrBn/8Mv/udglvEZwpw8c9bb3kt7n/+c3dwX3opNG0adGUiGanOADezh4FTgc+cc91i124CfgNsij3seufcjFQVKSG3cKHX4o4H95/+5LW4kwjuKJ5KFMWaJdoSaYE/Cvw/4LEq1+91zo33vSKJjoULvRb3zJm+BTdE81SiKNYs0VfnPHDn3DzgizTUIlGxaBGccgr06eN9PG6ctwBn9GhfukuieCpRFGuW6EtmIc9lZlZsZg+bWUvfKpLwWrQIBg2CY47ZHdwlJfCHP/jazx3Fk3SiWLNEX0MD/AHgx0BPYCNwd00PNLMRZrbYzBZv2rSppodJmFUM7oULUxbccTXtER7mk3SiWLNEX4MC3Dn3qXOuzDm3C/gHcHQtj53gnMt3zuW3bu3TIbaSHm+/Daeeuju477zT6ypJUXDHRfEknSjWLNHXoGmEZtbWObcx9ukQYIV/JUng3n7bG5x86SU44AAvuH//e2jWLC0vH8WTdKJYs0RfnSfymNmTwIlAK+BTYGzs8554h4uXAJdUCPQa6USekFu82Avu6dO94L72WrjssrQFd7pp2p9ERYNP5HHOnVPN5Yd8qUrCoWpw33FHRgc3aNqfZAZtJ5vNliyB00+H3r29LV5vv93r477uuowOb9C0P8kMWkqfjZYs8VrcL74ILVt6wX3ZZbD//kFXljaa9ieZQC3wbLJ0KZxxBuTnewcH33abNx3w+uuzKrxB0/4kMyjAs0E8uI86Cl5/fXdw//GPWRfccZr2J5lAXSiZbNkyr6vk+eehRQu49Va4/HJo3jzoygKnaX+SCRTgAUvJVLYagrvwg63c9cASBVbM4Lx2Wf3vl+hTgDeAX6Hr+1S2qsF9yy1wxRXQvLmmzYlkIPWB11M8CEu3bMOxOwgLl5XW+7l8m8pWVARDhkCvXvDaa15wl5TADTeUd5do2pxI5lELvJ5qC8L6tmSTnspWVOS1uAsLvaC++Wavxd2ihf+vFSJaQSniUQu8nvwMwgZPZSsqgqFDIS8P5szxgrukBG68sdrwTuq1QsbP34BEok4BXk9+BmG9p7K9887u4J492zvGrI7gru21DOjbOVo7RKorSGS3jA7wwmWlFIybTccxL1EwbrYvrTQ/5w8PzmvHnUO7065FLga0a5HLnUO779kd8M47cOaZ0LNn5eAeO7bO4K74Wmce1Q6rcM0BU5eURqr1mkldQSLJytg+8FTNuvB7/nCtU9mKi73ukWnTvAU3Y8fCVVclHNpVzVmziap7Tza0/z4oB7fIpbSasI5aV5CIHzI2wP0cbKwq5fOHi4u9mSRTp+4O7iuv9PYtSUImtF5HDexU6Y0ZtIJSslfGBngkw6pqcN94o9fiTjK44zKh9aoVlCK7ZWyARyqsli/3gnvKlJQEd1ymtF61glLEE/oAb+ic30iEVcXgbtbMW3hz1VXeoQp1aMh9UetVJLOEOsCTGYgMdVitWOEF97PP1ju4Ifn7Eop7ICJJq/NMTD/V90zMgnGzq+0GadcilwVj+vlZWnpUDe4rr4SRIxMO7riMuy8iUqsGn4kZpEgORFZn5crdwd20KfzP/zQouOMy5r6ISFLqXMhjZg+b2WdmtqLCtQPM7GUzey/2t7+jbTGRX/69ciX88pfQvTvMmOGdfPPhh972rg0Mb8iA+yIivkhkJeajwMlVro0BXnXOHQa8Gvvcd5E9NWXlSjj77MrBXVLinYRz4IFJP31k74uI+KrOLhTn3Dwz61Dl8hnAibGPJwJzgT/4WRiEfCCyOqtWeV0lzzwDTZp4p7tffbUvoV1R5O6LiKREQoOYsQCf7pzrFvt8i3OuRexjA76Mf17N944ARgAceuihR61fv96XwkNl1SqvW+Tpp73gvuKKlAS3iGSnmgYxk97MynnvADW+CzjnJjjn8p1z+a1bR2vnuzqtWgXnnAPdusH06TBmjNfHffvtCm8RSbmGzkL51MzaOuc2mllb4DM/iwq91au9FvdTT8F++3nBffXV0KpV0JWJSBZpaIC/AAwDxsX+ft63isKsanD/4Q9wzTWhDm6dXiOSueoMcDN7Em/AspWZbQDG4gX3M2Z2EbAe+EUqiwzcmjVecD/5ZGSCG1K3pa6IhEMis1DOqeFL/X2uJXyqBvfo0V5wR6QvP5Vb6opI8EK9EjMwa9fuDu7c3MgFd5xWbIpkNgV4RRWDu3FjuPZa708ag9vPPutIbakrIvWmAAcvuG+7DSZPDiy4wf8+60S21NUgp0h0ZfShxnV69104/3w44gjv3Mlrr/WWvP/pT4F0l/h94npdhybH3zBKt2zDsfsNI0qHHItks+xsgb/7rtfinjTJa3Ffc40X3gcdFGhZqeizrm3/bw1yikRbdgV4SIM7Lt191hrkFIm27OhCee89GDYMunTxji+7+mpvyfuf/xya8Ib07TJYuKyUgnGza9z/QIOcItGQ2S3w997zWtxPPAH77usF97XXQps2QVdWrXTsMlh1oLQqbUsrEh2ZGeDr1u0O7n328U6/GTUqtMFdUarPrKyu3zuunWahiERKZgV41eC+6qrIBHe61NS/baDzNEUiJjMCfN06bwvXxx/3gvvKK73g/sEPgq4sdLS4RyRzRHsQ8/334cILoXNnb4fAK6+EDz6Au+9WeNdAx7GJZI5otsDff99rcT/2GOTkeCfgjB6t0E6AjmMTyRzRCnAFty9SPVAqIukRjQCvGtyXX+4Fd9u2QVcmIhKYaAT47bd7OwQ2ILi1WZOIZKqETqX3S35+vlu8eHH9v/GTT8Cs3i3u6hat5OY0qrShk4hI2KXsVPq0OPjgBnWX+L27n4hImEQjwBtImzWJSCbL6ACvaXGKFq2ISCZIKsDNrMTMlptZkZk1oHM7tbRoRUQymR+zUPo65z734Xl8p0UrIpLJojGNMAlatCIimSrZPnAH/MvMlpjZiOoeYGYjzGyxmS3etGlTki8nIiJxyQb4sc65XsB/Ab83s+OrPsA5N8E5l++cy28dwEHBIiKZKqkAd86Vxv7+DHgOONqPokREpG4NDnAza2JmzeIfAwOAFX4VJiIitUtmELMN8JyZxZ9nsnPun75UJSIidWpwgDvnPgCO9LEWERGph4xeiSkikskU4CIiEaUAFxGJKAW4iEhEKcBFRCJKAS4iElEKcBGRiFKAi4hElAJcRCSiIrMfeOGyUh3MICJSQSQCvHBZKddNW15+wnzplm1cN205gEJcRLJWJLpQ7pq1tjy847btKOOuWWsDqkhEJHiRCPBPtmyr13URkWwQiQA/uEVuva6LiGSDSAT4qIGdyM1pVOlabk4jRg3sFFBFIiLBi8QgZnygUrNQRER2i0SAgxfiCmwRkd0i0YUiIiJ7UoCLiESUAlxEJKIU4CIiEaUAFxGJKHPOpe/FzDYB69P2guHVCvg86CJCRPejMt2PynQ/4IfOudZVL6Y1wMVjZoudc/lB1xEWuh+V6X5UpvtRM3WhiIhElAJcRCSiFODBmBB0ASGj+1GZ7kdluh81UB+4iEhEqQUuIhJRCnARkYhSgKeJmR1iZnPMbJWZrTSzK4OuKQzMrJGZLTOz6UHXEjQza2FmU8xsjZmtNrOfBl1TkMxsZOxnZYWZPWlmjYOuKWwU4OmzE7jGOXcE0Af4vZkdEXBNYXAlsDroIkLiL8A/nXOdgSPJ4vtiZu2AK4B851w3oBFwdrBVhY8CPE2ccxudc0tjH3+D98OZ1Rucm1l7YBDwYNC1BM3MmgPHAw8BOOe+d85tCbSo4O0N5JrZ3sB+wCcB1xM6CvAAmFkHIA9YGHApQbsPGA3sCriOMOgIbAIeiXUpPWhmTYIuKijOuVJgPPARsBH4yjn3r2CrCh8FeJqZWVNgKnCVc+7roOsJipmdCnzmnFsSdC0hsTfQC3jAOZcHfAuMCbak4JhZS+AMvDe2g4EmZnZesFWFjwI8jcwsBy+8JznnpgVdT8AKgNPNrAR4CuhnZk8EW1KgNgAbnHPx38qm4AV6tvo58KFzbpNzbgcwDfhZwDWFjgI8TczM8Po3Vzvn7gm6nqA5565zzrV3znXAG5ya7ZzL2haWc+7fwMdm1il2qT+wKsCSgvYR0MfM9ov97PQniwd1axKZQ40zQAFwPrDczIpi1653zs0IriQJmcuBSWa2D/ABcGHA9QTGObfQzKYAS/FmcC1DS+r3oKX0IiIRpS4UEZGIUoCLiESUAlxEJKIU4CIiEaUAFxGJKAW4iEhEKcBFRCLq/wMqlHCjfISp9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "N = X.shape[0]\n",
    "\n",
    "S_X2 = np.sum(X*X)\n",
    "S_X  = np.sum(X)\n",
    "S_XY = np.sum(X*Y)\n",
    "S_Y  = np.sum(Y)\n",
    "\n",
    "A1 = np.array([[S_X2, S_X], \n",
    "               [S_X, N]])\n",
    "B1 = np.array([S_XY, S_Y])\n",
    "\n",
    "# numpy.linalg模块包含线性代数的函数。\n",
    "# 使用这个模块，可以计算逆矩阵、求特征值、解线性方程组以及求解行列式等。\n",
    "coeff = np.linalg.inv(A1).dot(B1)\n",
    "\n",
    "print('a = %f, b = %f' % (coeff[0], coeff[1]))\n",
    "\n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = coeff[0] * x_min + coeff[1]\n",
    "y_max = coeff[0] * x_max + coeff[1]\n",
    "\n",
    "plt.scatter(X, Y, label='original data')\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 如何使用迭代的方法求出模型参数\n",
    "\n",
    "当数据比较多的时候，或者模型比较复杂，无法直接使用解析的方式求出模型参数。因此更为常用的方式是，通过迭代的方式逐步逼近模型的参数。\n",
    "\n",
    "### 2.1 梯度下降法\n",
    "在机器学习算法中，对于很多监督学习模型，需要对原始的模型构建损失函数，接下来便是通过优化算法对损失函数进行优化，以便寻找到最优的参数。在求解机器学习参数的优化算法中，使用较多的是基于梯度下降的优化算法(Gradient Descent, GD)。\n",
    "\n",
    "梯度下降法有很多优点，其中最主要的优点是，**在梯度下降法的求解过程中只需求解损失函数的一阶导数，计算的代价比较小，这使得梯度下降法能在很多大规模数据集上得到应用。**\n",
    "\n",
    "梯度下降法的含义是通过当前点的梯度方向寻找到新的迭代点。梯度下降法的基本思想可以类比为一个下山的过程。假设这样一个场景：\n",
    "* 一个人被困在山上，需要从山上下来(i.e. 找到山的最低点，也就是山谷)。\n",
    "* 但此时山上的浓雾很大，导致可视度很低。因此，下山的路径就无法全部确定，他必须利用自己周围的信息去找到下山的路径。\n",
    "* 这个时候，他就可以利用梯度下降算法来帮助自己下山。\n",
    "    - 具体来说就是，以他当前的所处的位置为基准，寻找这个位置最陡峭的地方，然后朝着山的高度下降的地方走\n",
    "    - 然后每走一段距离，都反复采用同一个方法，最后就能成功的抵达山谷。\n",
    "\n",
    "\n",
    "一般情况下，这座山最陡峭的地方是无法通过肉眼立马观察出来的，而是需要一个工具来测量；同时，这个人此时正好拥有测量出最陡峭方向的能力。所以，此人每走一段距离，都需要一段时间来测量所在位置最陡峭的方向，这是比较耗时的。那么为了在太阳下山之前到达山底，就要尽可能的减少测量方向的次数。这是一个两难的选择，如果测量的频繁，可以保证下山的方向是绝对正确的，但又非常耗时；如果测量的过少，又有偏离轨道的风险。所以需要找到一个合适的测量方向的频率，来确保下山的方向不错误，同时又不至于耗时太多！\n",
    "\n",
    "\n",
    "![gradient_descent](images/gradient_descent.png)\n",
    "\n",
    "如上图所示，得到了最优解。$x$,$y$表示的是$\\theta_0$和$\\theta_1$，$z$方向表示的是花费函数，很明显出发点不同，最后到达的收敛点可能不一样。当然如果是碗状的，那么收敛点就应该是一样的。\n",
    "\n",
    "对于最小二乘的损失函数\n",
    "$$\n",
    "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2\n",
    "$$\n",
    "\n",
    "我们更新的策略是：\n",
    "$$\n",
    "\\theta^1 = \\theta^0 - \\eta \\triangledown L(\\theta)\n",
    "$$\n",
    "其中$\\theta$代表了模型中的参数，例如$a$, $b$\n",
    "\n",
    "此公式的意义是：$L$是关于$\\theta$的一个函数，我们当前所处的位置为$\\theta_0$点，要从这个点走到L的最小值点，也就是山底。首先我们先确定前进的方向，也就是梯度的反向，然后走一段距离的步长，也就是$\\eta$，走完这个段步长，就到达了$\\theta_1$这个点！\n",
    "\n",
    "更新的策略是：\n",
    "\n",
    "$$\n",
    "a^1 = a^0 + 2 \\eta [ y - (ax+b)]*x \\\\\n",
    "b^1 = b^0 + 2 \\eta [ y - (ax+b)] \n",
    "$$\n",
    "\n",
    "下面就这个公式的几个常见的疑问：\n",
    "\n",
    "* **$\\eta$是什么含义？**\n",
    "$\\eta$在梯度下降算法中被称作为学习率或者步长，意味着我们可以通过$\\eta$来控制每一步走的距离，以保证不要步子跨的太大，错过了最低点。同时也要保证不要走的太慢，导致太阳下山了，还没有走到山下。所以$\\eta$的选择在梯度下降法中往往是很重要的。\n",
    "![gd_stepsize](images/gd_stepsize.png)\n",
    "\n",
    "* **为什么要梯度要乘以一个负号？**\n",
    "梯度前加一个负号，就意味着朝着梯度相反的方向前进！梯度的方向实际就是函数在此点上升最快的方向，而我们需要朝着下降最快的方向走，自然就是负的梯度的方向，所以此处需要加上负号。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 示例代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch    0: loss = 821.866782, a = 3.383589, b = 1.395455\n",
      "epoch  100: loss = 694.178445, a = 3.058880, b = 4.323617\n",
      "epoch  200: loss = 707.108900, a = 2.868004, b = 4.733422\n",
      "epoch  300: loss = 694.373048, a = 2.908494, b = 4.808539\n",
      "epoch  400: loss = 693.019312, a = 2.916707, b = 4.817341\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAm+0lEQVR4nO3dd5hU1f3H8fdXJLICCgoaAXWx0dGFBYmICqgYsSCJJsaCiQZb7EFRo4CiYETEHomNxK7g2iAkFMWK0qT7s0HCCtIERBalnN8fZ5Zd1i0zO+XeO/t5PQ8Pu3fvzHy5PPuZM+eeYs45REQkenYJugAREakeBbiISEQpwEVEIkoBLiISUQpwEZGI2jWTL9aoUSOXm5ubyZcUEYm8mTNnrnbONS57PKMBnpuby4wZMzL5kiIikWdmS8s7ri4UEZGIUoCLiESUAlxEJKIy2gdeni1btrBs2TI2b94cdCk1Wp06dWjWrBm1a9cOuhQRiVPgAb5s2TLq169Pbm4uZhZ0OTWSc441a9awbNkymjdvHnQ5IhKnwAN88+bNCu+AmRl77703q1atCroUkVAomF3I3RM/5et1RTRpkMOAXi3ok9c06LJ+IvAABxTeIaD/AxGvYHYhN46bR9GWbQAUrivixnHzAEIX4rqJKSJSyt0TP90R3sWKtmzj7omfBlRRxRTgKZabm8vq1auTPkdEgvH1uqKEjgdJAS4iUkqTBjkJHQ+SAhxYsmQJLVu25IILLuCwww7jnHPOYdKkSXTt2pVDDz2Ujz76iLVr19KnTx/at29Ply5dmDt3LgBr1qzhxBNPpE2bNlx00UWU3uHo6aefpnPnzhxxxBFcfPHFbNu2raISRCQkBvRqQU7tWjsdy6ldiwG9WlTr+QpmF9J1+BSaD3yTrsOnUDC7MBVlAiG5ibnD1VfDnDmpfc4jjoBRo6o87fPPP+ell17iiSeeoFOnTjz77LO8++67vPbaa9x5553sv//+5OXlUVBQwJQpUzj//POZM2cOQ4YM4eijj+bWW2/lzTff5PHHHwdg0aJFvPDCC7z33nvUrl2byy67jGeeeYbzzz8/tf8+EUmp4huVqRiFku4bolUGuJnVAaYBu8XOf9k5N8jMngKOBdbHTr3AOTcn6YoC0rx5c9q1awdAmzZt6NmzJ2ZGu3btWLJkCUuXLmXs2LEA9OjRgzVr1rBhwwamTZvGuHHjAOjduzcNGzYEYPLkycycOZNOnToBUFRUxD777BPAv0xEEtUnr2lKArayG6IZCXDgB6CHc26jmdUG3jWzCbGfDXDOvZx0FcXiaCmny2677bbj61122WXH97vssgtbt25NeIaic45+/foxbNiwlNYpItGR7huiVfaBO29j7NvasT81biv7bt268cwzzwDw1ltv0ahRI/bYYw+OOeYYnn32WQAmTJjAt99+C0DPnj15+eWXWblyJQBr165l6dJyV4QUkSyV7huicd3ENLNaZjYHWAn8xzk3PfajO8xsrpnda2a7VfDY/mY2w8xmRHmm3+DBg5k5cybt27dn4MCBjBkzBoBBgwYxbdo02rRpw7hx4zjggAMAaN26NUOHDuXEE0+kffv2nHDCCSxfvjzIf4KIZFiqb4iWZaVHTVR5slkD4BXgCmANsAL4GTAa+MI5d1tlj8/Pz3dlN3RYtGgRrVq1SqxqSQv9X4ikXiqm5ZvZTOdcftnjCY1Ccc6tM7OpwEnOuRGxwz+Y2ZPAnxOqSESkBkjVDdHyVNmFYmaNYy1vzCwHOAFYbGb7xY4Z0AeYn5YKRUSkXPG0wPcDxphZLXzgv+ice8PMpphZY8CAOcAl6StTRETKqjLAnXNzgbxyjvdIS0UiIhIXTaUXEYkoBbiISEQpwBNw8skns27dukrPufXWW5k0aVK1nv+tt97ilFNOqfK84447jrLDMcsaNWoUmzZtqlYdIhINCvA4OOfYvn0748ePp0GDBpWee9ttt3H88cdnprBKKMBFsl/kAjwdSzOOHDmStm3b0rZtW0bF1mNZsmQJLVq04Pzzz6dt27b873//22kjhttvv50WLVpw9NFHc/bZZzNihB8Wf8EFF/Dyy355mNzcXAYNGkSHDh1o164dixcvBuCjjz7iF7/4BXl5eRx11FF8+mnlO30UFRXx29/+llatWnHGGWdQVFSyjsKll15Kfn4+bdq0YdCgQQDcf//9fP3113Tv3p3u3btXeJ6IRFu4lpOtQjqWZpw5cyZPPvkk06dPxznHkUceybHHHkvDhg357LPPGDNmDF26dNnpMR9//DFjx47lk08+YcuWLXTo0IGOHTuW+/yNGjVi1qxZPPzww4wYMYLHHnuMli1b8s4777DrrrsyadIkbrrpph0rHZbnkUceYffdd2fRokXMnTuXDh067PjZHXfcwV577cW2bdvo2bMnc+fO5corr2TkyJFMnTqVRo0aVXhe+/btq3XNRIIUlQ2HIf21RqoFno696t59913OOOMM6tatS7169ejbty/vvPMOAAceeOBPwhvgvffe4/TTT6dOnTrUr1+fU089tcLn79u3LwAdO3ZkyZIlAKxfv54zzzyTtm3bcs0117BgwYJKa5w2bRrnnnsuAO3bt98peF988UU6dOhAXl4eCxYsYOHCheU+R7zniYRZcSOucF0RjpJGXCo3SUiV0rU2XbeCwm83pbzWSAV4pveqq1u3btLPUbwsba1atdi6dSsAt9xyC927d2f+/Pm8/vrrbN68uVrP/dVXXzFixAgmT57M3Llz6d27d7nPFe95ImEXpQ2H7574KU1WLOHe10fw9uj+HPPVrJTXGqkAT8fSjN26daOgoIBNmzbx/fff88orr9CtW7dKH9O1a9cdwbtx40beeOONhF5z/fr1NG3qP0Y99dRTVZ5fesna+fPn79jObcOGDdStW5c999yTb775hgkTJux4TP369fnuu++qPE8kSiKz4fDixQz4523857HL6PXZB/y9Ux8W7HswkNpaI9UHPqBXi536wCH5pRk7dOjABRdcQOfOnQG46KKLyMvL29HdUZ5OnTpx2mmn0b59e/bdd1/atWvHnnvuGfdrXn/99fTr14+hQ4fSu3fvKs+/9NJL+f3vf0+rVq1o1arVjv72ww8/nLy8PFq2bMn+++9P165ddzymf//+nHTSSTRp0oSpU6dWeJ5IlDRpkENhOQEYmg2HFy2C22+H55+n1667MfrIvvy90xmsqdtgxymprDWh5WSTlYrlZMNyA2Pjxo3Uq1ePTZs2ccwxxzB69Oidbi5GkZaTlUxL9Pe57EAG8I24YX3bBXsjc+FCH9wvvAC77w6XX874Xudw3dSvU1JrSpaTDYN0Ls2YiP79+7Nw4UI2b95Mv379Ih/eIplWnVFlqdxwOCXKBvf118N110HjxpwM/Nhw77TWGrkWuKSP/i8kk7oOn1Jud0jTBjm8NzDka+UtWOCD+8UXfXBfcYUP7tiw3VQLdQvcOYdfVlyCksk3chGI0A3J0hYsgNtug5degrp1YeBAuPbatAV3VQIfhVKnTh3WrFmjAAmQc441a9ZQp06doEuRGiTdG/6m1Pz58JvfQLt2MH68D+6vvoI77wwsvCEELfBmzZqxbNkyorzhcTaoU6cOzZo1C7oMqUHSMaos5ebPL2lx16sHN97oW9x77x10ZUAIArx27do0b9486DJEJMNCd0OytHnzfHC//LIP7ptvhmuuCU1wFws8wEWk5grLqLId5s71wT12LNSvH9rgLqYAFxEpG9x/+YsP7r32CrqySinARaTm+uQTH9zjxsEee8Att8DVV4c+uItVOQrFzOqY2Udm9omZLTCzIbHjzc1supl9bmYvmNnP0l+uiEgKzJkDffvCEUfApElw662wZIkP84iEN8Q3jPAHoIdz7nDgCOAkM+sC3AXc65w7BPgWuDBtVYqIpEJxcOflweTJJcE9ZAg0bBh0dQmrsgvF+QHaG2Pf1o79cUAP4Hex42OAwcAjqS9RRGRnCa+JNHu2b10XFMCee8KgQXDVVZEM7dLi6gM3s1rATOAQ4CHgC2Cdc25r7JRlQLlXz8z6A/0BDjjggGTrFZEaLqE1VGbP9q3rV1/1wT14sA/uKva2jYq4ZmI657Y5544AmgGdgZbxvoBzbrRzLt85l9+4cePqVSkiEhPXpg6zZsHpp0OHDvDWWz64lyzxLe8sCW9IcBSKc26dmU0FfgE0MLNdY63wZkD49jQSkaxT6Roqs2b5Fvdrr/mgHjIErrwyq0K7tHhGoTQ2swaxr3OAE4BFwFTg17HT+gGvpqlGEZEdylsrpe2Kz/nna3dAx44wbZrv716yxN+kzNLwhvha4PsBY2L94LsALzrn3jCzhcDzZjYUmA08nsY6RUSAnddQabf8M65671mO/+Jjfqy/p1/i9YorfH93DRDPKJS5QF45x7/E94eLiGRMn7ymNFj4CbsOHcrRiz9kQ059Fl42gNZ33lxjgruYZmKKSHR8/DEMGcJxb77phwAOHcoeV1xB6z32qPKhYdmOMZUU4CISfh995G9Ijh/vZ0recQf86U9++nscqrN9WxQowEUkvKZP98E9YYIP7jvv9MFdv36FDymvpV3Z0EMFuEgEZONH6KxVjeCGilvaZcO7WKi3b4uDAlxqhGz9CB02Sb9JfvihD+5//cuvwT1sGFx+eZXBXayilnYtM7aVs21jKLdvS4ACXGqEbP0IHQbFoV24rgjDL5QECb5JfvCBD+6JE31wDx8Ol10Wd3AXq6hFvc05cmrXCvf2bdUQ+KbGIpkQyR3QI+AvBfO45oU5FMauY9k27k+muJf1wQdw0klw1FEwc6YP7iVL4IYbEg5vqLhF3bRBDsP6tqNpgxys1PdRf/NWC1xqhCYNcnaETNnjUj0Fswt55sP//iS0yyr3TfL9932L+9//9ru633WXb3HXq5dUTZVtlBy67dtSQAEuNUIkdkAPsYpGdlQV3lDmTfL99/3CUv/5jw/uv/4VLr006eAuFuqNktPAXDkd++mSn5/vZsyYkbHXEylNo1Cqp+wNYOAn/ckVyaldy3dVbFrig3vSJGjcGK6/3gd33brpKzyLmNlM51x+2eNqgUuNEaaP0FF6M0l0ZEexpg1yGLbPeo65/oKS4L77bgV3CinARTIsakMaExnZYcA5XQ5g6N7rfB/35Mmwzz4wYgRccomCO8U0CkUkBQpmF9J1+BSaD3yTrsOnUDC74uXx49qQIEQSGdnxj0N/YOgDV8Mxx8C8eXDPPfDVV3DddQrvNFALXCRJibaoozakMa6RHdOm+Rb3lCmw774+uC+5BHbfPcDKs59a4CJJSrRFXVGLNqxDGvvkNa14DPXbb0OPHnDssbBgAYwcCV9+Cddeq/DOALXARZKUaIs6ikMaf3ID+O23ofu5fr/Jn/8c7r0X+vdXaGeYWuAiSUq0RV1pizbs3noLjjvO/1m8GEaN8i3uq69WeAdALXCRJFWnRR2mIY1xKd7Z/e23Yb/9fHD37w854ez2qSkU4CJJytrZf86VBPe0aT6477sP/vhHBXdIKMBFUiByLerKOAdTp/rgfucdH9z33w8XXaTgDpkq+8DNbH8zm2pmC81sgZldFTs+2MwKzWxO7M/J6S9XRNLGOT8M8NhjoWdP+OILH9xfful3eld4h048LfCtwHXOuVlmVh+YaWb/if3sXufciPSVJyJpVxzcgwfDu+9CkybwwAO+xV2nTkpfKkpLCERBlQHunFsOLI99/Z2ZLQJ0xUWizjk/1X3wYHjvPWjaFB58EC68MOXBDdFbQiAKEhpGaGa5QB4wPXboT2Y218yeMLOGFTymv5nNMLMZq1atSq5aEUmec345127d4IQT/AYKDz0En3/uty9LQ3hD9JYQiIK4A9zM6gFjgaudcxuAR4CDgSPwLfR7ynucc260cy7fOZffuHHj5CsWkeopDu6jj4YTT4SlS31wf/GF30whTcFdLGpLCERBXKNQzKw2Pryfcc6NA3DOfVPq538H3khLhSKSnOLgHjzYb2HWrBk8/DD84Q+w227VftpE+7MzuStSTelrj2cUigGPA4uccyNLHd+v1GlnAPNTX56IVJtzfpPgo46CXr1g2TJ45BHfVXLppUmH943j5lG4rghHSX92ZaswDujVgpzatXY6lo4lBKpTW1TF04XSFTgP6FFmyOBfzWyemc0FugPXpLNQEYlT6eA+6SQoLPTB/dlnfoXAJIK7WHX6szO1hEBN6muPZxTKu/h12ssan/pyRKTaioN78GCYPh323x/+9je44IKUhHZp1e3PzsSEp5rU167FrESizjmYMAG6dIFf/hKWL4dHH/VdJRdfnPLwhnAviRvm2lJNAS4SVc7B+PE+uE8+Gb75BkaP9l0l/fvDz36WtpfOVH92dYS5tlTTWigiUVPc4h48GD7+GA480Ad3v35pDe3SwryAV5hrSzVzlewqnWr5+fluxowZGXs9kaxS3OIePBhmzIDcXLj5Zjj//IwFtwTDzGY65/LLHlcXikjYOQdvvAGdO8Mpp8Dq1fDYY/B//+fXK1F411gKcJGwKg7uTp3g1FNhzZqS4L7wQqhdO+gKJWAKcJGwcQ5ef70kuNeuhccfh08/VXDLTnQTU0KppkyF3klxcA8ZArNmwUEHwRNPwLnnZm1o18j/5xRSgEvoxLvsaNh/+eOuzzl47TUf3LNn++B+8kk455ysDW7Q8rKpoC4UCZ14pkKHfb2LuOpzDl59FTp2hD59YMMGH9yLF/vZk1kc3lCzpryniwJcQieeqdBh/+WvtD7noKAAOnQoCe6nnqoxwV2sJk15TxcFuIROPFOhw/7LX14d5rbT9qPJkJcHZ5wBGzfCmDE+uPv1g11rVo9mTZryni4KcAmdeKZCh/2Xv3Qd5rbT6//e582nruLRV+6ETZvgH/+ARYv8JJwqgrtgdiFdh0+h+cA36Tp8Smi6iZJVk6a8p4sCXEInnmVHw/7LP6BXC3bf1ej16fuMf/JKHn3lTnbf+iMzbxsFCxfCeefF1eIOe19/MjK1vGw201R6iazQjkLZvh1eeYX1A//Cnp8v5su9mvLP48/jiD9fwumdDkzoqboOn1LuLjZNG+Tw3sAeqapYQq6iqfQ1q9NNskom1pZOyPbtMG4c3HYbzJvHnocdBk8/zdwW3fj3pM95aux8/jr5i4TeaMLe1y/BUheKSEy1+5q3b4eXX4bDD4czz4Qff4Snn4aFCylofRw3vrqw2l0gYe/rl2ApwCXrVCeIq9XXvH07vPRSSXBv2QLPPAMLFvhJOLVqJT3cMex9/RIsBbhklere9EsoaLdvhxdfhPbt4ayzYOtWePZZH9y/+x3UKgncZLtAdKNPKlNlH7iZ7Q/8A9gXcMBo59x9ZrYX8AKQCywBznLOfZu+UiWbpeqGZGVBXNnzxRW027b5rpLbb/dh3aoVPPecb33XqlXu45s0yCn3JmQiXSCh6+uX0IinBb4VuM451xroAlxuZq2BgcBk59yhwOTY9yIJS+VQueq2eCvta962DV54wbe4f/tb3wJ/7jmYN89/X0F4g7pAJL2qDHDn3HLn3KzY198Bi4CmwOnAmNhpY4A+aapRslwqp8VX96ZfeUFbtxaMYjG0a+eDGuD55+MK7mLqApF0SmgYoZnlAnnAdGBf59zy2I9W4LtYRBKWyqFyA3q12GmFO4ivxVt6H8UVazdy7n8/4rqPXmSPrz6D1q19C/zXv4ZdEr9tpC4QSZe4A9zM6gFjgaudcxvMbMfPnHPOzMqdEWRm/YH+AAcccEBy1UpWSkU/cbF4NrStqL+9T/uf02fxNBhzm1+fpE2bpIJbJN3imolpZrWBN4CJzrmRsWOfAsc555ab2X7AW865Sps5mokp5Sm7LjT4VnM6uhrKe626tWBM3a/I/+dDJcE9aBD86lcKbgmFas/ENN/UfhxYVBzeMa8B/YDhsb9fTVGtUsPE02pOldL97bts38api6Zx5fsvcPDaZdC2rR/X3bevglsiIZ4ulK7AecA8M5sTO3YTPrhfNLMLgaXAWWmpUGqETPUTf72uiFqx4L4iFtyLGudyaZ8beWTsUAW3REqVAe6cexewCn7cM7XlSLYKxcJTW7dywVfvcu6kf3Lw2kIWNc7lkj43MvGwX9CkYd1QhncorpuElhazkrQLfO/DrVv9uO3bb2fQZ5+xeJ/mXNznJv59WBec7RLacdmBXzcJvfA1OSTrBLb92datfuOE1q39xgk5OTBuHIvHv838I3uC7RLqcdlh3zZOgqcWeAhl28fmjC+JWrw2ye23w+ef89l+BzPijJtYkN+dP+e28v3tHfdPz2unkJaSlaoowEMmGz82p3Kcd6W2bvWrAQ4dCp9/zroWbfjLmbfwZvNOONsFNvwQqWuZsesmkaUulJDJxo/N5U1TN6B7y8apeYGtW/2u7i1b+l3d69WDggJ697uPNw460od3TJSupdZRkaoowEMmGz8298lryq86Nt1pKJMDxs4sTG5vxy1b4MknfXD//vewxx7w6qswaxacfjpfr99c7sOici21jopURV0oIZOtH5unLl5F2Tm/8SzzWq4tW/yON0OHwpdfQocOPrhPPRVKLfGQDddS66hIZdQCD5ls/dickk8WW7bAE09Aixbwhz9Agwbw2mswYwacdtpO4Q3Zey1FiinAQyZbPzYntbfjli3w+OM+uC+8EPbaC15/3Qd3mVZ3adl6LUWKxbWYVapoMauaq1oLVm3Z4sdxDx0KS5ZAfr5fZKp37wpDWyQbVXsxK5FUSGjBqh9/9MF9xx0lwf3gg3DyyQpukVIU4JIxVd6Q+/FHGDPGB/fSpdCpEzz0EPzylwpukXKoD1yC9+OPMHo0HHYY9O8P++4L48fD9OlqdYtUQgEuwSkO7kMPhYsvhp//3Af3hx+q1S0SBwW4ZN6PP8Kjj5YE9377wYQJ8MEHCm6RBCjAJXN++AH+9jcf3JdcAk2awL/+5YP7pJMU3CIJUoBL+pUO7ksv9cE9cSK8/z706qXgFqkmBbikzw8/wCOPwCGH+OBu1qwkuE88UcEtkiQNI5TU++EHP3Ny2DBYtgyOOspPgT/+eIW2SAopwCV1Nm/2wT18eElwP/kk9Oyp4BZJgyoD3MyeAE4BVjrn2saODQb+CKyKnXaTc258uoqUkCsO7mHDoLAQunZNOrijuCtRFGuWaIunBf4U8CDwjzLH73XOjUh5RRIdmzfDY4/54P76azj6aD+TskePpFrcUdyVKIo1S/RVeRPTOTcNWJuBWiQqNm+GBx6Agw+GK67wf0+eDNOmpaS7JIq7EkWxZom+ZEah/MnM5prZE2bWMGUVSXiVDu4rr/R/T5kCb7+ddKu7tCjuShTFmiX6qhvgjwAHA0cAy4F7KjrRzPqb2Qwzm7Fq1aqKTpMwKyqC+++Hgw7ywX3IISXB3b17ym9QJrV2eECiWLNEX7UC3Dn3jXNum3NuO/B3oHMl5452zuU75/IbN07RJraSGUVFcN99PrivusovNjV1atqCu1gUd9KJYs0SfdUaRmhm+znnlse+PQOYn7qSJHBFRX6tkrvughUr4Ljj4Lnn/N8ZkNDa4SERxZol+qrckcfMngOOAxoB3wCDYt8fgd9cfAlwcalAr5B25Am58oJ70KCMBXemadifREW1d+Rxzp1dzuHHU1KVhMOmTSXB/c03vnvk+efh2GODrixtNOxPsoHWQqnJNm2CkSN9H/e110KbNr5/e8qUrA5v0LA/yQ6aSl8TbdrkVwe86y5YudKP3X7pJejWLejKMkbD/iQbqAVek3z/PdxzDzRvDtddB+3a+ck3kybVqPAGDfuT7KAArwm+/x5GjPDB/ec/Q/v28M47NTK4i2nYn2QDdaFks++/h4cfhrvvhlWr4IQT/KiSrl2DrixwGvYn2aDKYYSppGGEP5WWoWwVBHfB7rkKLJEIqvYwQvmpVIVuyoeybdxYEtyrV/tdbwYNgqOO0rA5kSykPvAEFQdh4boiHCVBWDC7MOHnStlQto0b/YiS5s3hhhugY0e/bdnEiX5ThVS+loiEhlrgCaosCBNtySY9lG3jRnjoId/iXrPG7+w+aBB06ZL61woRzaAU8dQCT1Aqg7DaQ9m++85vW5abCwMHQqdO8MEHMGFCueGd1GuFTCo/AYlEnQI8QakMwoSHsn33nd/9JjcXbrwROneGDz+sNLgrey0DureM1gqR6goSKZHVAV4wu5Cuw6fQfOCbdB0+JSWttFSOH+6T15RhfdvRtEEOBjRtkMOwvu1+2h2wYQPceacP7ptu8mE9fTqMHw9HHhn3a/2qY1NKLwDrgLEzCyPVes2mriCRZGVtH3i6Rl2kevxwn7ymFT92wwZ48EE/e3LtWjj5ZN/H3bnC5dcrNXXxKsoOGq1u/31QmjTIobCcsI5aV5BIKmRtgKfyZmNZlYZuKmzY4Lcuu+ce+PZb6N3bB3enTkk9bTa0Xgf0arHTGzNoBqXUXFkb4JEMqw0b/NZlI0f64D7lFLj11qSDu1g2tF41g1KkRNYGeKTCav163+IuDu5TT/XBnf+TiVdJyZbWa9o/AYlEROgDvLpjfiMRVuvXl7S4163zwT1okJ+IU4XqXBe1XkWyS6gDPJkbkaEOq/Xr/WbB997rg/u003yLO47ghuSvSyiugYgkLdQBnuyNyNCF1bp1PrhHjfJfn366D+4OHRJ6mnTeoBWR6Ah1gEfyRmR5ioP73nt967uawV0sa66LiCSlyok8ZvaEma00s/mlju1lZv8xs89ifzdMR3GRn/69bh0MHuwn4Awe7DcLnjULCgqqHd6QBddFRFIinpmYTwEnlTk2EJjsnDsUmBz7PuUiu2vKt9/6m5G5uTBkCPToAbNnwyuvQF5e0k8f2esiIilVZReKc26ameWWOXw6cFzs6zHAW8ANqSwMQn4jsjzffuv7t0eN8mO6+/b1XSWHH57Sl4ncdRGRtIhrR55YgL/hnGsb+36dc65B7GsDvi3+vpzH9gf6AxxwwAEdly5dmpLCQ2XtWh/a992X1uAWkZqpoh15kl7Myvl3gArfBZxzo51z+c65/MaNo7XyXZXWroVbbvFdJbff7rcu++QTGDtW4S0iaVfdUSjfmNl+zrnlZrYfsDKVRYXe2rV+8s399/slXn/9ax/k7dsHXZmI1CDVDfDXgH7A8Njfr6asojBbs8YPBSwO7jPP9MHdrl3QlVVIu9eIZK8qA9zMnsPfsGxkZsuAQfjgftHMLgSWAmels8jArVlT0uLeuDESwQ3pW1JXRMIhnlEoZ1fwo54priV8Vq/2wf3AA/D99yXB3bZt0JXFRTM2RbJbqGdiBqZscJ91lg/uNm2CriwhmrEpkt0U4KWtXu03UXjgAdi0KZDgTmWfdaSW1BWRhCnAAVat8sH94IM+uH/zGx/crVtntIxU91nHs6SubnKKRFdWb2pcpVWr4IYboHlz+Otf/bKu8+fDc89lPLwh9TuuV7VpcvEbRuG6IhwlbxhR2uRYpCarmS3wVatgxAjf4i4qgrPPhr/8BVq1CrSsdPRZV7akrm5yikRbzQrwlSt9cD/0UKiCu1im+6x1k1Mk2mpGF8rKlTBggO8queceOOMMWLgQnnkmNOENmVtlsGB2IV2HT6lw/QPd5BSJhuxuga9cCXffDQ8/DJs3w+9+51vcLcK57GomVhkse6O0LC1LKxId2Rng33xTEtw//BD64C4t3dvAldfvXaypRqGIREp2BfiKFT64H3nEB/c55/jgPuywoCsLjYr6tw14b2CPzBYjIknJjgBfscIPA/zb33xwn3su3Hyzgrscmtwjkj2ifRNzxQq49lp/c/K++/zMycWLYcwYhXcFtB2bSPaIZgt8+fKSFveWLSUt7kMPDbqy0NN2bCLZI1oBvnw53HUXPPqoD+7zzvPBfcghQVcWKem+USoimRGNAC8b3Oef74P74IODrkxEJDDRCPCBA/2km2oEtxZrEpFsFdeu9KmSn5/vZsyYkfgDly6FbdvgoIMSelh5k1ZyatfaaUEnEZGwS9uu9Blx4IEJhzekfnU/EZEwiUaAV5MWaxKRbJbVAV7R5BRNWhGRbJBUgJvZEjObZ2ZzzKwandvppUkrIpLNUjEKpbtzbnUKniflNGlFRLJZNIYRJkGTVkQkWyXbB+6Af5vZTDPrX94JZtbfzGaY2YxVq1Yl+XIiIlIs2QA/2jnXAfglcLmZHVP2BOfcaOdcvnMuv3Hjxkm+nIiIFEsqwJ1zhbG/VwKvAJ1TUZSIiFSt2gFuZnXNrH7x18CJwPxUFSYiIpVL5ibmvsArZlb8PM865/6VkqpERKRK1Q5w59yXwOEprEVERBKQ1TMxRUSymQJcRCSiFOAiIhGlABcRiSgFuIhIRCnARUQiSgEuIhJRCnARkYhSgIuIRFRk1gMvmF2ojRlEREqJRIAXzC7kxnHzduwwX7iuiBvHzQNQiItIjRWJLpS7J366I7yLFW3Zxt0TPw2oIhGR4EUiwL9eV5TQcRGRmiASAd6kQU5Cx0VEaoJIBPiAXi3IqV1rp2M5tWsxoFeLgCoSEQleJG5iFt+o1CgUEZESkQhw8CGuwBYRKRGJLhQREfkpBbiISEQpwEVEIkoBLiISUQpwEZGIMudc5l7MbBWwNGMvGF6NgNVBFxEiuh470/XYma4HHOica1z2YEYDXDwzm+Gcyw+6jrDQ9diZrsfOdD0qpi4UEZGIUoCLiESUAjwYo4MuIGR0PXam67EzXY8KqA9cRCSi1AIXEYkoBbiISEQpwDPEzPY3s6lmttDMFpjZVUHXFAZmVsvMZpvZG0HXEjQza2BmL5vZYjNbZGa/CLqmIJnZNbHflflm9pyZ1Qm6prBRgGfOVuA651xroAtwuZm1DrimMLgKWBR0ESFxH/Av51xL4HBq8HUxs6bAlUC+c64tUAv4bbBVhY8CPEOcc8udc7NiX3+H/+Ws0Qucm1kzoDfwWNC1BM3M9gSOAR4HcM796JxbF2hRwdsVyDGzXYHdga8Drid0FOABMLNcIA+YHnApQRsFXA9sD7iOMGgOrAKejHUpPWZmdYMuKijOuUJgBPBfYDmw3jn372CrCh8FeIaZWT1gLHC1c25D0PUExcxOAVY652YGXUtI7Ap0AB5xzuUB3wMDgy0pOGbWEDgd/8bWBKhrZucGW1X4KMAzyMxq48P7GefcuKDrCVhX4DQzWwI8D/Qws6eDLSlQy4BlzrniT2Uv4wO9pjoe+Mo5t8o5twUYBxwVcE2howDPEDMzfP/mIufcyKDrCZpz7kbnXDPnXC7+5tQU51yNbWE551YA/zOzFrFDPYGFAZYUtP8CXcxs99jvTk9q8E3dikRmU+Ms0BU4D5hnZnNix25yzo0PriQJmSuAZ8zsZ8CXwO8DricwzrnpZvYyMAs/gms2mlL/E5pKLyISUepCERGJKAW4iEhEKcBFRCJKAS4iElEKcBGRiFKAi4hElAJcRCSi/h9DXZEW7ONw8AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "n_epoch = 500          # epoch size\n",
    "a, b = 1, 1            # initial parameters\n",
    "epsilon = 0.001         # learning rate\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    data_idx = list(range(N))\n",
    "    random.shuffle(data_idx)\n",
    "    \n",
    "    for j in data_idx:\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (Y[j]-a*X[j]-b)**2\n",
    "    \n",
    "    if i % 100 == 0:\n",
    "        print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n",
    "    \n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n",
    "\n",
    "plt.scatter(X, Y, label='original data')\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 如何可视化迭代过程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "/* global mpl */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function () {\n",
       "    if (typeof WebSocket !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof MozWebSocket !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert(\n",
       "            'Your browser does not have WebSocket support. ' +\n",
       "                'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "                'Firefox 4 and 5 are also supported but you ' +\n",
       "                'have to enable WebSockets in about:config.'\n",
       "        );\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = this.ws.binaryType !== undefined;\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById('mpl-warnings');\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent =\n",
       "                'This browser does not support binary websocket messages. ' +\n",
       "                'Performance may be slow.';\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = document.createElement('div');\n",
       "    this.root.setAttribute('style', 'display: inline-block');\n",
       "    this._root_extra_style(this.root);\n",
       "\n",
       "    parent_element.appendChild(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen = function () {\n",
       "        fig.send_message('supports_binary', { value: fig.supports_binary });\n",
       "        fig.send_message('send_image_mode', {});\n",
       "        if (fig.ratio !== 1) {\n",
       "            fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n",
       "        }\n",
       "        fig.send_message('refresh', {});\n",
       "    };\n",
       "\n",
       "    this.imageObj.onload = function () {\n",
       "        if (fig.image_mode === 'full') {\n",
       "            // Full images could contain transparency (where diff images\n",
       "            // almost always do), so we need to clear the canvas so that\n",
       "            // there is no ghosting.\n",
       "            fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "        }\n",
       "        fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "    };\n",
       "\n",
       "    this.imageObj.onunload = function () {\n",
       "        fig.ws.close();\n",
       "    };\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_header = function () {\n",
       "    var titlebar = document.createElement('div');\n",
       "    titlebar.classList =\n",
       "        'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n",
       "    var titletext = document.createElement('div');\n",
       "    titletext.classList = 'ui-dialog-title';\n",
       "    titletext.setAttribute(\n",
       "        'style',\n",
       "        'width: 100%; text-align: center; padding: 3px;'\n",
       "    );\n",
       "    titlebar.appendChild(titletext);\n",
       "    this.root.appendChild(titlebar);\n",
       "    this.header = titletext;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = (this.canvas_div = document.createElement('div'));\n",
       "    canvas_div.setAttribute(\n",
       "        'style',\n",
       "        'border: 1px solid #ddd;' +\n",
       "            'box-sizing: content-box;' +\n",
       "            'clear: both;' +\n",
       "            'min-height: 1px;' +\n",
       "            'min-width: 1px;' +\n",
       "            'outline: 0;' +\n",
       "            'overflow: hidden;' +\n",
       "            'position: relative;' +\n",
       "            'resize: both;'\n",
       "    );\n",
       "\n",
       "    function on_keyboard_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.key_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    canvas_div.addEventListener(\n",
       "        'keydown',\n",
       "        on_keyboard_event_closure('key_press')\n",
       "    );\n",
       "    canvas_div.addEventListener(\n",
       "        'keyup',\n",
       "        on_keyboard_event_closure('key_release')\n",
       "    );\n",
       "\n",
       "    this._canvas_extra_style(canvas_div);\n",
       "    this.root.appendChild(canvas_div);\n",
       "\n",
       "    var canvas = (this.canvas = document.createElement('canvas'));\n",
       "    canvas.classList.add('mpl-canvas');\n",
       "    canvas.setAttribute('style', 'box-sizing: content-box;');\n",
       "\n",
       "    this.context = canvas.getContext('2d');\n",
       "\n",
       "    var backingStore =\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        this.context.webkitBackingStorePixelRatio ||\n",
       "        this.context.mozBackingStorePixelRatio ||\n",
       "        this.context.msBackingStorePixelRatio ||\n",
       "        this.context.oBackingStorePixelRatio ||\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        1;\n",
       "\n",
       "    this.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n",
       "        'canvas'\n",
       "    ));\n",
       "    rubberband_canvas.setAttribute(\n",
       "        'style',\n",
       "        'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n",
       "    );\n",
       "\n",
       "    // Apply a ponyfill if ResizeObserver is not implemented by browser.\n",
       "    if (this.ResizeObserver === undefined) {\n",
       "        if (window.ResizeObserver !== undefined) {\n",
       "            this.ResizeObserver = window.ResizeObserver;\n",
       "        } else {\n",
       "            var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n",
       "            this.ResizeObserver = obs.ResizeObserver;\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n",
       "        var nentries = entries.length;\n",
       "        for (var i = 0; i < nentries; i++) {\n",
       "            var entry = entries[i];\n",
       "            var width, height;\n",
       "            if (entry.contentBoxSize) {\n",
       "                if (entry.contentBoxSize instanceof Array) {\n",
       "                    // Chrome 84 implements new version of spec.\n",
       "                    width = entry.contentBoxSize[0].inlineSize;\n",
       "                    height = entry.contentBoxSize[0].blockSize;\n",
       "                } else {\n",
       "                    // Firefox implements old version of spec.\n",
       "                    width = entry.contentBoxSize.inlineSize;\n",
       "                    height = entry.contentBoxSize.blockSize;\n",
       "                }\n",
       "            } else {\n",
       "                // Chrome <84 implements even older version of spec.\n",
       "                width = entry.contentRect.width;\n",
       "                height = entry.contentRect.height;\n",
       "            }\n",
       "\n",
       "            // Keep the size of the canvas and rubber band canvas in sync with\n",
       "            // the canvas container.\n",
       "            if (entry.devicePixelContentBoxSize) {\n",
       "                // Chrome 84 implements new version of spec.\n",
       "                canvas.setAttribute(\n",
       "                    'width',\n",
       "                    entry.devicePixelContentBoxSize[0].inlineSize\n",
       "                );\n",
       "                canvas.setAttribute(\n",
       "                    'height',\n",
       "                    entry.devicePixelContentBoxSize[0].blockSize\n",
       "                );\n",
       "            } else {\n",
       "                canvas.setAttribute('width', width * fig.ratio);\n",
       "                canvas.setAttribute('height', height * fig.ratio);\n",
       "            }\n",
       "            canvas.setAttribute(\n",
       "                'style',\n",
       "                'width: ' + width + 'px; height: ' + height + 'px;'\n",
       "            );\n",
       "\n",
       "            rubberband_canvas.setAttribute('width', width);\n",
       "            rubberband_canvas.setAttribute('height', height);\n",
       "\n",
       "            // And update the size in Python. We ignore the initial 0/0 size\n",
       "            // that occurs as the element is placed into the DOM, which should\n",
       "            // otherwise not happen due to the minimum size styling.\n",
       "            if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n",
       "                fig.request_resize(width, height);\n",
       "            }\n",
       "        }\n",
       "    });\n",
       "    this.resizeObserverInstance.observe(canvas_div);\n",
       "\n",
       "    function on_mouse_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.mouse_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousedown',\n",
       "        on_mouse_event_closure('button_press')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseup',\n",
       "        on_mouse_event_closure('button_release')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'dblclick',\n",
       "        on_mouse_event_closure('dblclick')\n",
       "    );\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousemove',\n",
       "        on_mouse_event_closure('motion_notify')\n",
       "    );\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseenter',\n",
       "        on_mouse_event_closure('figure_enter')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseleave',\n",
       "        on_mouse_event_closure('figure_leave')\n",
       "    );\n",
       "\n",
       "    canvas_div.addEventListener('wheel', function (event) {\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        on_mouse_event_closure('scroll')(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.appendChild(canvas);\n",
       "    canvas_div.appendChild(rubberband_canvas);\n",
       "\n",
       "    this.rubberband_context = rubberband_canvas.getContext('2d');\n",
       "    this.rubberband_context.strokeStyle = '#000000';\n",
       "\n",
       "    this._resize_canvas = function (width, height, forward) {\n",
       "        if (forward) {\n",
       "            canvas_div.style.width = width + 'px';\n",
       "            canvas_div.style.height = height + 'px';\n",
       "        }\n",
       "    };\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n",
       "        event.preventDefault();\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus() {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'mpl-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'mpl-button-group';\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'mpl-button-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        var button = (fig.buttons[name] = document.createElement('button'));\n",
       "        button.classList = 'mpl-widget';\n",
       "        button.setAttribute('role', 'button');\n",
       "        button.setAttribute('aria-disabled', 'false');\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "\n",
       "        var icon_img = document.createElement('img');\n",
       "        icon_img.src = '_images/' + image + '.png';\n",
       "        icon_img.srcset = '_images/' + image + '_large.png 2x';\n",
       "        icon_img.alt = tooltip;\n",
       "        button.appendChild(icon_img);\n",
       "\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    var fmt_picker = document.createElement('select');\n",
       "    fmt_picker.classList = 'mpl-widget';\n",
       "    toolbar.appendChild(fmt_picker);\n",
       "    this.format_dropdown = fmt_picker;\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = document.createElement('option');\n",
       "        option.selected = fmt === mpl.default_extension;\n",
       "        option.innerHTML = fmt;\n",
       "        fmt_picker.appendChild(option);\n",
       "    }\n",
       "\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', { width: x_pixels, height: y_pixels });\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_message = function (type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function () {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function (fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1], msg['forward']);\n",
       "        fig.send_message('refresh', {});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n",
       "    var x0 = msg['x0'] / fig.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n",
       "    var x1 = msg['x1'] / fig.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0,\n",
       "        0,\n",
       "        fig.canvas.width / fig.ratio,\n",
       "        fig.canvas.height / fig.ratio\n",
       "    );\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch (cursor) {\n",
       "        case 0:\n",
       "            cursor = 'pointer';\n",
       "            break;\n",
       "        case 1:\n",
       "            cursor = 'default';\n",
       "            break;\n",
       "        case 2:\n",
       "            cursor = 'crosshair';\n",
       "            break;\n",
       "        case 3:\n",
       "            cursor = 'move';\n",
       "            break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_message = function (fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n",
       "    for (var key in msg) {\n",
       "        if (!(key in fig.buttons)) {\n",
       "            continue;\n",
       "        }\n",
       "        fig.buttons[key].disabled = !msg[key];\n",
       "        fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n",
       "    if (msg['mode'] === 'PAN') {\n",
       "        fig.buttons['Pan'].classList.add('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    } else if (msg['mode'] === 'ZOOM') {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.add('active');\n",
       "    } else {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message('ack', {});\n",
       "};\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function (fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            var img = evt.data;\n",
       "            if (img.type !== 'image/png') {\n",
       "                /* FIXME: We get \"Resource interpreted as Image but\n",
       "                 * transferred with MIME type text/plain:\" errors on\n",
       "                 * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "                 * to be part of the websocket stream */\n",
       "                img.type = 'image/png';\n",
       "            }\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src\n",
       "                );\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                img\n",
       "            );\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        } else if (\n",
       "            typeof evt.data === 'string' &&\n",
       "            evt.data.slice(0, 21) === 'data:image/png;base64'\n",
       "        ) {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig['handle_' + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\n",
       "                \"No handler for the '\" + msg_type + \"' message type: \",\n",
       "                msg\n",
       "            );\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\n",
       "                    \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n",
       "                    e,\n",
       "                    e.stack,\n",
       "                    msg\n",
       "                );\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "};\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function (e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e) {\n",
       "        e = window.event;\n",
       "    }\n",
       "    if (e.target) {\n",
       "        targ = e.target;\n",
       "    } else if (e.srcElement) {\n",
       "        targ = e.srcElement;\n",
       "    }\n",
       "    if (targ.nodeType === 3) {\n",
       "        // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "    }\n",
       "\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    var boundingRect = targ.getBoundingClientRect();\n",
       "    var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n",
       "    var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n",
       "\n",
       "    return { x: x, y: y };\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys(original) {\n",
       "    return Object.keys(original).reduce(function (obj, key) {\n",
       "        if (typeof original[key] !== 'object') {\n",
       "            obj[key] = original[key];\n",
       "        }\n",
       "        return obj;\n",
       "    }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function (event, name) {\n",
       "    var canvas_pos = mpl.findpos(event);\n",
       "\n",
       "    if (name === 'button_press') {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * this.ratio;\n",
       "    var y = canvas_pos.y * this.ratio;\n",
       "\n",
       "    this.send_message(name, {\n",
       "        x: x,\n",
       "        y: y,\n",
       "        button: event.button,\n",
       "        step: event.step,\n",
       "        guiEvent: simpleKeys(event),\n",
       "    });\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.key_event = function (event, name) {\n",
       "    // Prevent repeat events\n",
       "    if (name === 'key_press') {\n",
       "        if (event.key === this._key) {\n",
       "            return;\n",
       "        } else {\n",
       "            this._key = event.key;\n",
       "        }\n",
       "    }\n",
       "    if (name === 'key_release') {\n",
       "        this._key = null;\n",
       "    }\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.key !== 'Control') {\n",
       "        value += 'ctrl+';\n",
       "    }\n",
       "    else if (event.altKey && event.key !== 'Alt') {\n",
       "        value += 'alt+';\n",
       "    }\n",
       "    else if (event.shiftKey && event.key !== 'Shift') {\n",
       "        value += 'shift+';\n",
       "    }\n",
       "\n",
       "    value += 'k' + event.key;\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n",
       "    if (name === 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message('toolbar_button', { name: name });\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "\n",
       "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n",
       "// prettier-ignore\n",
       "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";/* global mpl */\n",
       "\n",
       "var comm_websocket_adapter = function (comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.binaryType = comm.kernel.ws.binaryType;\n",
       "    ws.readyState = comm.kernel.ws.readyState;\n",
       "    function updateReadyState(_event) {\n",
       "        if (comm.kernel.ws) {\n",
       "            ws.readyState = comm.kernel.ws.readyState;\n",
       "        } else {\n",
       "            ws.readyState = 3; // Closed state.\n",
       "        }\n",
       "    }\n",
       "    comm.kernel.ws.addEventListener('open', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('close', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('error', updateReadyState);\n",
       "\n",
       "    ws.close = function () {\n",
       "        comm.close();\n",
       "    };\n",
       "    ws.send = function (m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function (msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        var data = msg['content']['data'];\n",
       "        if (data['blob'] !== undefined) {\n",
       "            data = {\n",
       "                data: new Blob(msg['buffers'], { type: data['blob'] }),\n",
       "            };\n",
       "        }\n",
       "        // Pass the mpl event to the overridden (by mpl) onmessage function.\n",
       "        ws.onmessage(data);\n",
       "    });\n",
       "    return ws;\n",
       "};\n",
       "\n",
       "mpl.mpl_figure_comm = function (comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = document.getElementById(id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm);\n",
       "\n",
       "    function ondownload(figure, _format) {\n",
       "        window.open(figure.canvas.toDataURL());\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element;\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error('Failed to find cell for figure', id, fig);\n",
       "        return;\n",
       "    }\n",
       "    fig.cell_info[0].output_area.element.on(\n",
       "        'cleared',\n",
       "        { fig: fig },\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function (fig, msg) {\n",
       "    var width = fig.canvas.width / fig.ratio;\n",
       "    fig.cell_info[0].output_area.element.off(\n",
       "        'cleared',\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "    fig.resizeObserverInstance.unobserve(fig.canvas_div);\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable();\n",
       "    fig.parent_element.innerHTML =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "    fig.close_ws(fig, msg);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.close_ws = function (fig, msg) {\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width / this.ratio;\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message('ack', {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () {\n",
       "        fig.push_to_output();\n",
       "    }, 1000);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'btn-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'btn-group';\n",
       "    var button;\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'btn-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        button = fig.buttons[name] = document.createElement('button');\n",
       "        button.classList = 'btn btn-default';\n",
       "        button.href = '#';\n",
       "        button.title = name;\n",
       "        button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message pull-right';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = document.createElement('div');\n",
       "    buttongrp.classList = 'btn-group inline pull-right';\n",
       "    button = document.createElement('button');\n",
       "    button.classList = 'btn btn-mini btn-primary';\n",
       "    button.href = '#';\n",
       "    button.title = 'Stop Interaction';\n",
       "    button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n",
       "    button.addEventListener('click', function (_evt) {\n",
       "        fig.handle_close(fig, {});\n",
       "    });\n",
       "    button.addEventListener(\n",
       "        'mouseover',\n",
       "        on_mouseover_closure('Stop Interaction')\n",
       "    );\n",
       "    buttongrp.appendChild(button);\n",
       "    var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n",
       "    titlebar.insertBefore(buttongrp, titlebar.firstChild);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._remove_fig_handler = function (event) {\n",
       "    var fig = event.data.fig;\n",
       "    if (event.target !== this) {\n",
       "        // Ignore bubbled events from children.\n",
       "        return;\n",
       "    }\n",
       "    fig.close_ws(fig, {});\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (el) {\n",
       "    el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (el) {\n",
       "    // this is important to make the div 'focusable\n",
       "    el.setAttribute('tabindex', 0);\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    } else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (event, _name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager) {\n",
       "        manager = IPython.keyboard_manager;\n",
       "    }\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which === 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "};\n",
       "\n",
       "mpl.find_output_cell = function (html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i = 0; i < ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code') {\n",
       "            for (var j = 0; j < cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] === html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "};\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel !== null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target(\n",
       "        'matplotlib',\n",
       "        mpl.mpl_figure_comm\n",
       "    );\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAiUAAAFuCAYAAABeGmjZAAAAAXNSR0IArs4c6QAAIABJREFUeF7t3QmUHVWdx/Fvp5ushNVAWGQLSATZF20GxwVlXHDQGRAUlV0Q0RDAyBJ2QljFAAKKoKCyCIoOoiPgsAdkcYOAYIMIsonsJIQk3T3nphrsJL286lf13q2q7zsnRw+579a9n/+F/nXVraoW/CiggAIKKKCAAhEItEQwBoeggAIKKKCAAgpgKHERKKCAAgoooEAUAoaSKMrgIBRQQAEFFFDAUOIaUEABBRRQQIEoBAwlUZTBQSiggAIKKKCAocQ1oIACCiiggAJRCBhKoiiDg1BAAQUUUEABQ4lrQAEFFFBAAQWiEDCURFEGB6GAAgoooIAChhLXgAIKKKCAAgpEIWAoiaIMDkIBBRRQQAEFDCWuAQUUUEABBRSIQsBQEkUZHIQCCiiggAIKGEpcAwoooIACCigQhYChJIoyOAgFFFBAAQUUMJS4BhRQQAEFFFAgCgFDSRRlcBAKKKCAAgooYChxDSiggAIKKKBAFAKGkijK4CAUUEABBRRQwFDiGlBAAQUUUECBKAQMJVGUwUEooIACCiiggKHENaCAAgoooIACUQgYSqIog4NQQAEFFFBAAUOJa0ABBRRQQAEFohAwlERRBgehgAIKKKCAAoYS14ACCiiggAIKRCFgKImiDA5CAQUUUEABBQwlrgEFFFBAAQUUiELAUBJFGRyEAgoooIACChhKXAMKKKCAAgooEIVAaULJ+PHju9dee+0oUB2EAgooUK/AHXfc8QywSr39+H0FiiRQmlDS3t7ePXPmzCLZO1YFFFCgX4GWlpY7gG0kUqBKAoaSKlXbuSqgQGEEDCWFKZUDzVDAUJIhpl0poIACWQkYSrKStJ8iCRhKilQtx6qAApURMJRUptROtJeAocTloIACCkQoYCiJsCgOKXcBQ0nuxB5AAQWKJHDmpH0Z9uw8OplNK2PoWnk4k2dc0PApGEoaTu4BIxAwlERQBIeggAJxCIRA0vXMc8CCXgNqY9j4cQ0PJoaSONaEo2isgKGksd4eTQEFIhaYsevuLOh+fokRtrWsyKTLL27oyA0lDeX2YJEIGEoiKYTDUECB5gt8Y9ed6O6eu8RAWhjFwVdc2dABGkoayu3BIhEwlERSCIehgALNF8jyTEm9e1MMJc1fD46g8QKGksabe0QFFIhUIKs9JVn0YyiJdJE4rFwFYgglI4FbgOFAGzADCFvdHwNeBrqBp4CPDSThY+ZzXSd2rkBlBN46w9E9h9aW0UO6+yaLMy6GksosOSfaSyCGUBLGMBqYDYwB7gM2B/4ATASWvMDbRwkNJa5rBRSIRSCLvSmGkliq6TgaKRBDKOk93xWA3/WEkvC/hpJGrgaPpYACmQh4piQTRjupoEAsoWQUcCewLjAF+BbwV+AFoBM4FbjKyzcVXKFOWYECCrinpIBFc8hRCMQSSt7EGAf8BNgZaO3ZS7Ia8BtgB6BjMbW9gfCHCRMmtHd0LP7XURg7CAUUqKBAvXtTvHxTwUXjlIktlISSnAXcCvR+KMDZwM0DnS1xT4mrWQEFyiRgKClTNZ1LrQIxhJJwdmQ+8BIwFrgd+DzwKPAqsGxPSPkMMKu/iRlKai257RRQoAgChpIiVMkxZi0QQyjZGAjPbx7W8+dc4NfA1T2TDf/8HODbA03eUJL10rA/BRRopoChpJn6HrtZAjGEkkzmbijJhNFOFFAgEgFDSSSFcBgNFTCUNJTbgymggAK1CRhKanOyVbkEDCXlqqezUUCBkggYSkpSSKeRSsBQkorLxgoooEBjBAwljXH2KHEJGEriqoejUUABBRYKGEpcCFUUMJRUserOWQEFchV468FpzKaVMUN6qZ+hJNcS2XmkAoaSSAvjsBRQoH+BLH7o5+WbxSPmPVOSV3XsN3YBQ0nsFXJ8CiiwiEBWP/TzYs3iZXyGkryqY7+xCxhKYq+Q41NAgUUEsvqhnxfrN3bdie7uuUt038IoDr6i99szBh6Bl2/yqpD9xixgKIm5Oo5NAQWWEMjqh35etFmFJkNJXhWy35gFDCUxV8exKVARgTR7RLL6oZ8XbVaXlwwleVXIfmMWMJTEXB3HpkAFBNL+EE/bvhmEb4assa88y6tjV6Jr/Agmz7gg1VAMJam4bFwSAUNJSQrpNBQoqsBQzny8dWalew6tLaOHdMttrl5//zsccwx8//tw6aWwyy6pD2coSU3mF0ogYCgpQRGdggJFFoh9j0gq2xdfhFNOgRkzoK0NpkyByZNh6aVTdRMaG0pSk/mFEggYSkpQRKegQJEFhnKmJLr5zp0L55wDJ50Er70G++8PU6fCSisNeaiGkiHT+cUCCxhKClw8h65AGQSKsEekX+fOTvjhD+Goo+CJJ2DXXeHEE2HChLpLYyipm9AOCihgKClg0RyyAmUTiH6PyOLg3d3wq1/BYYfBfffBdtsll2222CKz0hhKMqO0owIJGEoKVCyHqoACEQj89rfw9a/DzTfDppsmYWT77TMfmKEkc1I7LIBADKFkJHALMBxoA2YA4d65rYGLgPD3lwDHD+TZ3t7ePXPmzAKQO0QFFCikwMMPw5FHwlVXwVprwbRpyeWaYcNymY6hJBdWO41cIIZQEsYwGpgNjAHuAzYHrgP2BP4M3A7s2/N3fZIaSiJfaQ5PgaIKPP00HH88XHABLLdcsn8kbGQdMSLXGRlKcuW180gFYgglvWlWAH4HbAP8HNiq5y+/AoR76qb352goiXSFOSwFiirwyitw+ulwxhnJDA4+GA49FJZdtiEzMpQ0hNmDRCYQSygZBdwJrAtMAX4LTAU+2eP1KWA74EBDSWQryOEo0ESBNI+nr3mY8+bB+efDCSdAeO7IPvskD0JbZZWau8iioaEkC0X7KJpALKHkTbdxwE+ArwGH1xBK9gbCHyZMmNDe0dFRNH/Hq4ACQxTI/Fbiri644ork+SKPPgr//d/JvpH11x/iCOv7mqGkPj+/XUyB2EJJUDwLuLfnrEjvyzdjgZM8U1LMheaoFchaINOHrt1wQ3JHze9+B+99L5x6KrznPVkPOVV/hpJUXDYuiUAMoSScHZkPvASE4BE2tX4OuBDYo2ej623AF93oWpJV5zQUyEAgk8fThxASnjVy/fWw4YZw8snw8Y+HZ7xnMML6ujCU1Ofnt4sp0Px/82Bj4GIg3FcX/pwLnAeEX1NCMAm3BP8AOHYgYje6FnMBOmoFhipQ15mScHkm3EUTXpa3+urJ/pHPfx5aW4c6nMy/ZyjJnNQOCyAQQyjJhMlQkgmjnShQGIEh7Sl57rnkMfDnnQdjxsARR8CBB8KosNc+ro+hJK56OJrGCBhKGuPsURRQIAeBmh9PP3s2nHlmslck3F0zaVJy2Wb55XMYVTZdGkqycbSXYgkYSopVL0ergAJpBObPh4sugmOPpevZZ3lwrXW4/V3v4PXR4+haeTiTZ4SHR8f5MZTEWRdHla+AoSRfX3tXQIEmCJz51X14x70dtN93Lyu8+hqPvH11bt1gPZ5fNjw8+s1PG8PGj4s2mBhKmrBwPGTTBQwlTS+BA4hNIJcHcsU2yRKP58c77cC2N85k1Rde5KkVl+OWjSfy5Li3Ad1LzLqtZUUmXR722cf3MZTEVxNHlL+AoSR/Y49QIIEhbZ4s0PwaOdSGh7v77oPDD4drr+WFsWO4daP16Vht/IC397YwioOvuLKRLDUfy1BSM5UNSyRgKClRMZ1K/QJ13WZa/+FL0UMIIzz7Ol3d4dFDvT85XS55/PHkMfAXXwwrr8z1q6/CfWuNp7uGt/d6pqQUS85JlEjAUFKiYjqV+gUyeSBX/cMobA/f3HMPOuc83+elkjCpTEPACy/A9Olw9tkwfHjyRNaDDmLG3gewoDuMYfFP+M9d70s4OYWkjKrnmZKMIO2mUAKGkkKVy8HmLeCZktqFF7880z2qZcBAEnrO5HLJ668nQSQEknCr75e/DEceCW8L+0agv0twraOXo+X1bjq759DaMtq7b2ovtS0VaJiAoaRh1B6oCALuKamtSn07LX4mYsm+6jpT0tkJl1wCRx8Nf/877LZb8iTWtdde4kA1P7+ktuk2pZVnSprC7kGbLGAoaXIBPHx8AjH8QGv4JtGUZejvjNLA3Qzxckl3N/ziF8km1lmzYPvtk3fUbLZZylEXq7mhpFj1crTZCBhKsnG0FwUyEyjC2Zr+9t70hzCM5WD8qPTPBLnjjmSvyK23wuabwymnwIc+lJl1zB0ZSmKujmPLS8BQkpes/SrQSyDNmY8i7Gvp/0zJ4pdwWmgdvSIHfe/76dbDn/+cvJfm6qthnXVg2jT49Kehhjtq0h0o3taGknhr48jyEzCU5GdrzwosFEh75qMIdwDltpn0qafguOPgwgthhRWS/SNf/GJyd03FPoaSihXc6S4UMJS4EBTIWSDtmY+07XMefr/dZ7r35uWX4bTT4BvfSM6GHHIIHHoojB3brOk1/biGkqaXwAE0QcBQ0gR0D1ktgbRnPtKeWSm05htvwHnnwYknQggm4azIUUfB+PGFnlYWgzeUZKFoH0UTMJQUrWKOt3ACQznzkelZiBjFurrgsstg6lR47DHYeedk38h668U42qaMyVDSFHYP2mQBQ0mTC+Dhyy9QqTMfg5Uz3N573XXJHTV//CO8731w6qmw9daDfbNyf28oqVzJnXAke0rWBy4ClgXmAZOBm4H5wKyeKt0D7DNQxdrb27tnzpxpURWIUqD0Zz5qUb/nniSM/N//wUYbJbf3fuQjA74wr5Zuy9rGUFLWyjqvgQRiOFOyJjASeAiYCFwDhHO4zwA1X1g2lLjQFYhUoKMjuUxzxRWwxhrJU1jD01hbWyMdcBzDMpTEUQdH0ViBGEJJ7xmH8bwZRp42lDR2MXg0BTIV+Mc/kgBy/vmwzDLJ+2kOOABGht9B/AwmYCgZTMi/L6NAbKFkR2B/4KPAG8D9wOvAVOAmL9+UcQk6p9IJvPZacmtvuMV3wYKFb+5deNlmueUWmWqaB8qVzqiGCRlKakCySekEYgol4TLO9cDHgA5gVeApYAPgWmBT4OXFKrA3EP4wYcKE9o5wmtiPAgo0R2D+fLjgguThZ//8J+y1Fxx7LKy22hLjcfPv4CUylAxuZIvyCcQSSpYBbgGmANf1wRz2mRwHhA2vfX7cU1K+xemMCiIQ7qi56qrksfDhF4Mdd4STToINwu8TfX+Gcpt0QTQyG6ahJDNKOyqQQAyhJOx2C2dCfg6c12O3PDCn5xJOOGNyO7AF8IKhpECry6GWX+DGG5NLM3ffDdtsk9xRs+22g8477QPlBu1wsQZluDRkKElbdduXQSCGULIDcHWv23+D6wHAd4BOoKvnLMnPBgL3TEkZlqNzKIxAeMbIYYfB//4vTJwIJ5/Mmb+5hmH/mE8ns2llDF0rD+/3rcB5nikpy6UhQ0lh/m1woBkKxBBKMpmOoSQTRjtRYGCBv/0teQz8D38Iq6yS7B/ZYw/OPORLdD3zHLCg1/fbGDZ+XJ/BJM/gkGfgaeTyMJQ0UttjxSJgKImlEo5DgZgFnn8+2SdyzjkwalRyluSrX4XRoxeOeihBIK8HyuV9aahRZTKUNEra48QkYCiJqRqORYHYBObMgRkzFl6eYe5cOPDAZEPriisuMtKYgsBQAlJs7GE8hpIYq+KY8hYwlOQtbP8KFFEgPF/k+9+HY46Bp5+Gz30ueRDamuHO/SU/MQWBPC8NNbKUhpJGanusWAQMJbFUwnEoEINAuL33f/4HDj8cHnwQPvpRmD4dNtlkwNHFFgTyujTUyBIZShqp7bFiETCUxFIJx6FAswVuvx2mTIHwYsuttkpu7/3AB2oeVRmCQM2TbUBDQ0kDkD1EdAKGkuhK4oAUaLDAAw8kZ0bCGZJ11002tO60k2/vbXAZFj+coaTJBfDwTREwlDSF3YMqEIHAk08me0a+9z0YNy75//vsA0stFcHgHIKhxDVQRQFDSRWrXsI5l+EJng0ry0svJZdmvvlNaGuDr30NDj4Yll66YUPo60DWcFEVQ0lTl6MHb5KAoaRJ8B42O4HYNllmN7OMewq39H7rWzBtGrz6Kuy/f/IgtJVWyvhA6buzhkuaGUrSryO/UXwBQ0nxa1j5GdRyO2qMv4U3bEydnfCjHyUB5PHHYZdd4MQTk/0jkXxqqWEkQ23YMAwlDaP2QBEJGEoiKoZDGZrAYA/uivG38IaMKdze+6tfJU9fve8++OAHk8s2W245NOgcvzVYDXM8dLRdG0qiLY0Dy1HAUJIjrl03RmCw37IH+/vGjHLRo+Q+prvuSt7ee9NNsOmmSRj58IejvaMmd49mFLnOYxpK6gT064UUMJQUsmwOurfAYGcdYvwtPKsxLX4JaJkRr7PnnG646ipYa63kMs1nPgPDhkW9aAarYdSDz2lwhpKcYO02agFDSdTlcXC1Cgz04K4YfwvPYky9f5CPfn0u7Q/8hY0efYJ5I4Yz6uRTko2sI0bUStj0dj58bdESGEqaviQdQBMEDCVNQPeQjRWI8bfwLMYUgk3LvGfZ6qFH2eLhR2np7ubed6zD7yduyZd+elljkT1a5gKGksxJ7bAAAoaSAhTJIdYvkNVv4VneMfNmXwu6XwO6w3thaWtZmq6VhzN5xgUDT3rePP7vPVvzngceZOS8+dy39tu5Y8P1mD1qJC2M4uArrqwfzR6aKmAoaSq/B2+SgKGkSfAeNg6BNCEji7Mbi886dZ9dXfDjH8ORR8Kjj/KX1Vbmto0m8sIy/3rwWVvLiky6/OI4gB3FkAUMJUOm84sFFjCUFLh4Dr0+gbSBIIt9IIuPOFWfN9yQ3FHzu9/Bttty+SrL82RL2MC6oFe3bQwbP27wMy310fntBggYShqA7CGiE4ghlKwPXAQsC8wDJgM3AzsAZwDhv7qnAN8dSK+9vb17Zni7qZ/KCKQ5y9EXSqpAAGR1x0zvsdTU5+9/nzxr5LrrYMMN4eST4eMfX3h7b1aXpSqzaAo0UUNJgYrlUDMTiCGUrAmMBB4CJgLXAO8E7gfCe9NfBe4B/g14vr+ZG0oyWxOF6CjtWY6+JlVTIOj1xbQhphbIAfucfmzyFNbwNNbVV4fjj4cvfAFaW2vp2jYFFzCUFLyADn9IAjGEkt4DD+N5Bvgv4CBg556/DGdMQjDp95YCQ8mQ6l/YL2URENL2kUUQWhy8rz5HvdHJe/72BJvPehjGjIEjjoADD4RRowpbLweeXsBQkt7MbxRfILZQsiOwP3AhsG1PMAnK4ZJO+PXwdM+UFH/RZTGDtGc5+jrmUEJGHpdL3uyzZf4rbPmXx9jiz7MYER529tWvJpdtVlghCzL7KJiAoaRgBXO4mQjEFErCZZzrgY8Bm9YYSvYGwh8mTJjQ3tHRkQmKncQvkPYsR38zGihk1LtnpWbFBQvgwgvh2GPh2Wdh993huONgjTVq7sKG5RMwlJSvps5ocIFYQskywC3AFOA6YJuesyO9L9/cC1zqmZLBi1qFFkM5y5HGJe/+F44lvDDv6qvh8MPh4Ydhhx3gpJNgo43SDNW2JRUwlJS0sE5rQIEYQkm4LHMt8HPgvJ7RtgGzgPf3bHQNgSQEFTe6uqDfEsjjUsqbnWd1Jqbfct16K0yZAnfeCe9+d/LCvPe9z+oq8JaAocTFUEWBGEJJuPX36p4Q8mYNQhj59549JOGW4FOB7wxUIDe6lm/5NuzySR90WexZ6bMi99+fnBn5xS/gHe+A6dPhU5+K9u295VtVxZmRoaQ4tXKk2QnEEEoymY2hJBPGaDppyOWTAWab+ZmSJ56AY46Biy+GlVZK9o/stRcstdSQzZsZ2oY8aL9Ys4ChpGYqG5ZIwFBSomKWaSqZh4KUOJmFohdfTM6GnHUWDB+eXLKZPDm51beOT2bjq2MMfjVfAUNJvr72HqeAoSTOutQ9qqL/Fp3b5ZN+ZPvyCk2HPTuPzu45tLaMru1FeW/2//rrcM45ycbV2bPhgAOS99WMG1d3bUMHzQ5tmUzCTgYUMJS4QKooYCgpYdXL8Ft0I3/oZurV2QmXXAJHHw1//zt89rNwwgmwzjqZrrRGh7ZMB29nNQkYSmpislHJBAwlJStoWX6LToLCP4DOXhVqZdj4lTJ/2VwmASjc3nvttcnDzmbNgu23T95Rs9lmuaywTMacy8jsNCsBQ0lWkvZTJAFDSZGqVeNYy/BbdCNDSd1e4bbesFck3Oa7+ebJ7b0f+lCN1Rpas0zP7gxtCH4rZwFDSc7Adh+lgKEkyrLUN6gy/BbdyDkM+VgPPZS8l+anP00uz0ybBp/+NIRHxDfgk+dzWhowfA8xiIChxCVSRQFDSQmrXobfous+e5Girqm9nn46eQz8d78Lyy+f7B/Zb7/k7ho/CmQkYCjJCNJuCiVgKClUuWofbNF/ix7y2YvaiRZpWZPXyy/DaafBmWcm3z3kEDj0UFgmvCXBjwLZChhKsvW0t2IIGEqKUafKjTL12Ys8hd54A84/P7mL5qWX4ItfTM6OjB+f51Htu+IChpKKL4CKTt9QUtHCF2HaNZ29yHMiXV1w2WUwdSo89hjstFOybyQ8Ht6PAjkLGEpyBrb7KAUMJVGWxUE1XeC66+DrX4c//CF5UV64oya8OM+PAg0SMJQ0CNrDRCVgKImqHA6m6QL33ps8a+SGG2CjjZJnjXz0o74wr+mFqd4ADCXVq7kzBkOJq0CBIPDII8llmssvhzXWSPaP7LYbtLbqo0BTBAwlTWH3oE0WMJQ0uQAevskC//gHnHhispF16aWT99N8+cswcmSTB+bhqy5gKKn6Cqjm/A0l1ay7s37tNfjGN5JbfBcsgEmTkss2yy2njQJRCBhKoiiDg2iwgKGkweAerskC8+cnDz0LDz977jnYc0849lhYffUmD8zDK7CogKHEFVFFAUNJFatexTmHF+ZddVXyWPiODvjP/4STToINN6yihnMugIChpABFcoiZC8QQSq4CtgN+DezaM8PHgJeBbuAp4GODzby9vb175syZgzXz70sq8NYzTZhNK2PoWnn4v94mfNNNye29d90F22yT3N677bYNlxhwjA0fjQeMXcBQEnuFHF8eAjGEkvcDY4HdFgslE4G5tU7aUFKrVPna9ff015XahvH5F+fCr34FEyfC9Omw445Nub03qifUlm8JlHJGhpJSltVJDSIQQygJQwzBZH9Diet1KAKLvydn7Ow5/Nv9D7PB356kZdVVk/0je+wBbW1D6T6T7zT6XT6ZDNpOmipgKGkqvwdvkkCsoeSvwAtAJ3AqEC7xDPjxTMlgQuX9+zffKDzyjXm8+8EONu34G52tw7hr4kTee+dvYfTopk++kW89bvpkHUAmAoaSTBjtpGACsYaSVXv2kqwG/AbYAejow3ZvIPxhwoQJ7R1hA6Ofygl8a6fd2Pjhe9nqz4/Q1tnFH9Zdk9++c10WjFyFSZdfHIWHZ0qiKEOhBmEoKVS5HGxGArGGkt7TOxu4ebCzJZ4pyWhFFKmb8HyRiy/m1UlfZenZc3hgzdWY+a538MqYcGakjWHjx/1rs2uT5+WekiYXoICHN5QUsGgOuW6BGEPJGGAY8CqwLHAr8Blg1kCzNZTUvRaK00G4vfeaa+Dww+GBB+AjH+EHy4/iha6xdHbPobVl9KJ330Qys6a/9TgSB4dRm4ChpDYnW5VLIIZQci2wNRDCSNhHsiNwUQ9zCCfnAN8ejN1QMphQSf7+9tuT23vD/265ZXJ77wc/mOvkvJU3V14770fAUOLSqKJADKEkE3dDSSaM8Xby4IPJmZGf/zxsIEoefLbzzrnf3utll3iXRNlHZigpe4WdX18ChhLXRdwCTz6ZPAb+oovgbW+Do4+GffeF4cMbMm43qDaE2YP0IWAocVlUUcBQUsWqF2HOL72UXJr55jehtRW+9jU4+GAYG56z17iPt/I2ztojLSpgKHFFVFHAUFLFqsc857lz4dxzYdo0eOUV2G8/OOooWHnlpozaMyVNYfeghCuTLXcA24ihQJUEDCVVqnbMc+3shEsvhalT4fHHYZdd4MQTYd11mzpq95Q0lb/SBzeUVLr8lZ28oaQEpS/03SHh9t5f/zq5o+ZPf0rupAmXbcKdNZF8vJU3kkJUbBiGkooV3OkuFDCUNHEhZBEmGvmbfBbjXYT77ruTMHLjjbDJJkkY2X773O+oaWLJPbQCNQsYSmqmsmGJBAwlTSpmVmGiUXseshrvQu6//AWOPBKuvBLWXDO5TPPZz8Kw8FgaPwoosPA3RveUuBAqKGAoaVLRswoTjbo7JJPxPvssHH88fOc7sMwyyf6RAw6AESNyq0LmZ3dyG6kdK7CogKHEFVFFAUNJk6qeVZjIJCzUYFDXeF99FU4/Hc44A7q6YPJkmDIFlg1vEcjvk+nZnfyGac8K9ClgKHFhVFHAUNKkqmcVJhr1g3dI4503LzkrEs6OPP887L138iC0VcNLoPP/DGnM+Q/LIyhQk4ChpCYmG5VMwFCSsqBZXQ7IMkw04u6Qb+65B51z/rmEVuvot3HQ976/6D8PZ0PCfpGwb+SRR+BTn0oeCz9xYkrt+prXdXanvkP7bQXqFjCU1E1oBwUUMJSkKFqWQSIcthFhIsX0Bmxa81mH3/wmuaPm3nth223h1FOhvT2rYaTqp+Yxp+rVxgo0RsBQ0hhnjxKXgKEkRT2q/ENu0LMOf/gDHHZY8syRDTaAk0+GHXZo6u29WYfIFEvFpgrULWAoqZvQDgooYChJUbRBfzCn6KtoTfsLZMvPGcFey46CH/0IVlst2T/yhS9AW1sUUyzS2agowBxENAKGkmhK4UAaKFDZUDKUvSFlOlOSdv6Ln3UY9cY83v3AI2zyyGO0hZfkHXEEfOUrMGpUA5evh1KgvAKGkvLW1pn1L1DJUDLU0/pD/V5sC3Co8wjfG/7kbDZ56E9s9dDDtHZ10RZu7z38cFhhhdim6Xhmif3MAAAU9ElEQVQUKLSAoaTQ5XPwQxSoZCip54xHGS4HDGn+CxbARRclt/Q+8wzsvjscdxysscYQl55fU0CBgQQMJa6PKgpUMpRUeW9IWOSp5h9emPeznyVnQx56CD7+cZg+HTbaqIr/vjhnBRomYChpGLUHikgghlByFbAd8Gtg1x6brYGLgJHAJcDxg5m1t7d3z5w5c7BmC/9+SGcKauq5GI1qnv+ttyZPXr3zTth66+T23ve9rxiTdJQKFFzAUFLwAjr8IQnEEEreD4wFdusVSu4C9gT+DNwO7AvcN9AM04SSoe6pGJJwhF8adP6zZiVnRq65BtZbLzkz8l//1dTbeyNkdEgK5CpgKMmV184jFYghlASaEEz27wkl4RnkPwe26jH7CrA0MD2rUBL6KcPekHrWVJ/zP/RoOOYYuPhiGDcu2T8SHg2/1FL1HMrvKqDAEAQMJUNA8yuFF4gxlGwJTAU+2aP7qZ7LOwdmGUoKX7ksJ/Dii8nDzs46Kwkg4ZLNQQfB0iEL+lFAgWYIGEqaoe4xmy1Q9FCyNxD+MGHChPaOjo5mexbr+HPnwtlnJ++lmT0bvvQlmDo1OUviRwEFmipgKGkqvwdvkkCMoaSvyzdhz8lJninJaJV0dsIPfgBHHw1PPAGf/SyccAKss05GB7AbBRSoV8BQUq+g3y+iQIyhJDjeDezRs9H1NuCLWW50LWKhMhlzuL33l79M3lFz//3woQ/BKafA5pv32X3ap75mMkY7UUCBhQKGEhdCFQViCCXXAuEW4DHAC8AngBHAhT23BP8AOHaw4qS5+2awvkr59+G23vD23ltugc02S8LIhz/c71QHvUOnlEhOSoF4BAwl8dTCkTROIIZQkslsDSX9MIYHnh15JPzkJ7D22jBtGuyyCwwbNqB7zc8yyaR6dqKAAosLGEpcE1UUMJSUsOrhLMfSj73E1rPuZaNHH2PuiBGMDmdG9tsPRoSTUIN/Uj31dfDubKGAAikFDCUpwWxeCgFDSSnK+K9JfGv/3dn8ljvZ8uFHFv7De9Zfm3vWfwcL3r4qk2dcUPNsazlT4p6TmjltqEBqAUNJajK/UAIBQ0kJirhwCm+8Ad/+NnOmTGHkvHn8aZ23c+cG6zF7VHhSP7S1rMikyy+uebaD7SkZ7O9rPpANFVCgTwFDiQujigKGkqJXvasLLr88eb7IX//Kw6uvxm0brcuLYxd98FkLozj4iitTzXagp97WciYl1cFsrIACiwgYSlwQVRQwlBS56tdfn9xR8/vfw7//+8IX5s0481wWdD+/xKzSnikZjMU9J4MJ+fcK1CdgKKnPz28XU8BQUsS63Xtv8qyRG26Ad70reUT8xz628IV5eV9WefPsyYLuF4Gu3MNPEcvjmBXIQsBQkoWifRRNwFBSpIo9+mhymeayy+Dtb0+ewvq5z0Fr6yKzyOtlg30Hnt6HbmPY+HGpNtQWid+xKtBIAUNJI7U9ViwChpJYKjHQOJ57Lgkg55+fvCTviCPgwANhZLKJtVGf/vaRwDDaWpana+XhBpJGFcPjlF7AUFL6EjvBPgQMJTEvi9degzPPhNNOg/nzYdKkZA/J8ss3ZdTuI2kKuwetqIChpKKFr/i0DSUxLoAQQC68EI49FsJZkj32SP5/uGTTxI933DQR30NXTsBQUrmSO+HwzqeyKJTiMfPhhXnhcfDh8sxf/gKf+ARMnw4bbhhFmfLeRBvFJB2EApEIGEoiKYTDaKiAoaSh3AMc7OabYcoUuOsuaG9PXpj33vfGMrq3xpHXJtroJuqAFGiygKGkyQXw8E0RMJQ0hb3XQf/0Jzj8cPjlL2H99ZMzI5/85MLbe/0ooEB1BQwl1a19lWdemp98jbp8k9n7Xh5/HI4+Gi65BMaPh+OOgz33hLa2Kq9H566AAj0ChhKXQhUFDCUpqp7JnooXXkjOhpx9dvLG3nA3zUEHwejRKUZiUwUUKLuAoaTsFXZ+fQkYSlKsi7ruPnn9dTjrrCSQhP//5S8nG1rf9rYUI7CpAgpURcBQUpVKO8/eAoaSFOthSM/pWLAguUQTLtU89RTstlvyILS11kpxZJsqoEDVBAwlVau48w0CsYeS+cCsnlLdA+zTX9kasack1ZmScHvvNdckm1gfeAD+4z+Sd9RsuqkrTwEFFBhUwFAyKJENSigQeyh5Bhhfi3sjQknNe0pmzkz2itx2G2yxRXJ773bb1TIN2yiggAILBQwlLoQqChhKUlZ9wOd0PPhgsk/kZz+DCRNg2jTYeWcYNizlUWyugAJVFzCUVH0FVHP+sYeSN4D7gdeBqcBNzbx80+8SCXtFwmPgw6PhV1wRjjkG9t0Xhg+v5qpy1gooULeAoaRuQjsooEDsoWRV4ClgA+BaIGzIeLmX895A+MOECRPaOzo6GluCl1+GU09NXpoXzoYceigccgiMHdvYcXg0BRQonYChpHQldUI1CMQeSnpP4RrgOCBseF3i04g9JW8d9I034Nxz4cQT4ZVXYL/94KijYOWVayC3iQIKKDC4gKFkcCNblE8g5lCyPDAHCJdwwhmT24EtgBeaFkq6uuDSS2HqVPjb3+DTn06CyXrrlW9lOCMFFGiqgKGkqfwevEkCMYeSbYDvAJ1AV89Zkp/155TrmZJwe++vfw2HHQZ//CN84APJHTVbbdWksnlYBRQou4ChpOwVdn59CcQcSlJVLLdQcvfdye29N94IG2+chJHwzBFfmJeqPjZWQIF0AoaSdF62LoeAoaS/OoZNs0ceCT/+MayxRnKZJjyN1dt7y7HynYUCkQsYSiIvkMPLRcBQsjjrs88mj4H/9rdhmWWS/SNf+hKMHJlLAexUAQUU6PM0dkvLHUC4jO1HgcoIGEreLPWrr8IZZ8Dpp0PY0Bre3Bsu2yy7bGUWgxNVQIF4BDxTEk8tHEnjBAwl8+bBBRfA8cfDP/8Je+2VPAhttdUaVwWPpIACCiwmYChxSVRRoLqhJJwNufLKZN/II4/AJz8JJ50E73xnFdeBc1ZAgcgEDCWRFcThNESguqFk/nzYcENYaaXkqazbeOm2ISvOgyigQE0ChpKamGxUMoHqhpJQyGeeSZ7C6u29JVvWTkeB4gsYSopfQ2eQXqDaoSS9l99QQAEFGiJgKGkIsweJTKDyoeTMSfsy7Nl5dDKbVsbQtfJwJs+4ILIyORwFFKiagKGkahV3vkGg0qEkBJKuZ54DFvRaDW0MGz/OYOK/Hwoo0FQBQ0lT+T14kwQqHUpm7Lo7C7qfX4K+rWVFJl1+cZNK4mEVUECBsNXNh6e5DqonUOlQ8o1dd6K7e+4SVW9hFAdfcWX1VoMzVkCBaAQMJdGUwoE0UKDSocQzJQ1caR5KAQVSCRhKUnHZuCQClQ4l7ikpySp2GgqUUMBQUsKiOqVBBSodSoLOW3ffdM+htWW0d98MumRsoIACjRAwlDRC2WPEJlD5UBJbQRyPAgooEAQMJa6DKgoYSqpYdeesgALRCxhKoi+RA8xBIPZQsgNwBjAMOAX4bn8G7e3t3TNnzsyByC4VUECBxgsYShpv7hGbLxBzKGkD7gc+ALwK3AP8G7Dkg0UAQ0nzF5MjUECB7AQMJdlZ2lNxBGIOJeG1vZOBnXs4wxmTEEwu64vXUFKcRedIFVBgcAFDyeBGtiifQMyhZCdgW+CgHvYQUFqB0w0l5VuIzkgBBRYVMJS4IqooUPRQsjcQ/jBhwoT2jo6OKtbQOSugQAkFDCUlLKpTGlQg5lDS1+Wbe4FL+5nV08BfB51xeRqsBPyjPNNJNRPnnoqrNI2rVve1gVVKUz0nokANAjGHkrDRdRbw/p6NriGQhKDS50bXGuZatibhVqPgUcWPc69i1aHKda9mxZ115QRiDiWhGP/Zs4ck3BJ8KvCdylWo/wlX+T/Qzr2a/yJUue7VrLizrpxA7KGkcgVJMeGwl+bCFO3L1NS5l6matc+lynWvXcmWChRYwFBS4OI5dAUUUEABBcokYCgpUzWdiwIKKKCAAgUWMJQUuHgOXQEFFFBAgTIJGEqKV831gYuAZYF5PU+9vbl406hrxKOBB3ue7ntYXT0V68vr9tR+xZ7ab1as4Q95tOEBivuGF+cCvwCmDLknv6iAAlELGEqiLk+fg1sTGAk8BEwErgHWK9406hrxNCD8gA7PpalSKLkVOAS4C6jKMztW6JnvhsAC4E5gL+C+ulaQX1ZAgSgFDCVRlqXmQYX6PQOMB7pr/laxG4YAdnJPGAuhrCqh5F3AacBHi12+1KMPZ4XCM4o2BuYDtwGfAJ5K3ZNfUECB6AUMJdGXaMAB7gjsX7EfVD8Hvtbz4LgqhZJPAZ8HRvQ85TNcwjun2Mu35tF/FTihJ3iH5xWdVPM3baiAAoUSMJQUqlyLDDZcxrke+BhQlZf+hBAWXtIYQskePZevqnKmJLyg8lxgk54nHId9ROEyxh+Lu4RrGvnywA+BzwJdwG+APXue9lxTBzZSQIHiCBhKilOr3iNdBrilZ8PfdcWcwpBGPR34HNAJLA0sBZxSkd+cwysFjgB26JEL874f+MGQJIvzpZ17guikXvMOr5+4pDhTcKQKKFCrgKGkVql42rUC1wLhMsZ58Qyr4SOp2pmS8C6osME1vAtqDnATcHDPP2s4fgMP2A7MAN7bc/nmzXmHDa9+FFCgZAKGkuIVNPymfPVip6/DD6qXijeVukZctVASsMIGz3C2KHyuBI6rS7A4Xw4bm8Olu7CZ+6fA1OIM3ZEqoEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AQMJbnR2rECCiiggAIKpBEwlKTRsq0CCiiggAIK5CZgKMmN1o4VUEABBRRQII2AoSSNlm0VUEABBRRQIDcBQ0lutHasgAIKKKCAAmkEDCVptGyrgAIKKKCAArkJGEpyo7VjBRRQQAEFFEgjYChJo2VbBRRQQAEFFMhNwFCSG60dK6CAAgoooEAaAUNJGi3bKqCAAgoooEBuAoaS3GjtWAEFFFBAAQXSCBhK0mjZVgEFFFBAAQVyEzCU5EZrxwoooIACCiiQRsBQkkbLtgoooIACCiiQm4ChJDdaO1ZAAQUUUECBNAKGkjRatlVAAQUUUECB3AT+HziZ3m5Hl1FsAAAAAElFTkSuQmCC\" width=\"570.3895891961337\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib nbagg\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.animation as animation\n",
    "\n",
    "n_epoch = 300          # epoch size\n",
    "a, b = 1, 1             # initial parameters\n",
    "epsilon = 0.0001         # learning rate\n",
    "\n",
    "fig = plt.figure()\n",
    "imgs = []\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    data_idx = list(range(N))\n",
    "    random.shuffle(data_idx)\n",
    "    \n",
    "    for j in data_idx[:10]:\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "\n",
    "    if i<80 and i % 5 == 0:\n",
    "        x_min = np.min(X)\n",
    "        x_max = np.max(X)\n",
    "        y_min = a * x_min + b\n",
    "        y_max = a * x_max + b\n",
    "\n",
    "        img = plt.scatter(X, Y, label='original data')\n",
    "        img = plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "        imgs.append(img)\n",
    "        \n",
    "ani = animation.ArtistAnimation(fig, imgs)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 如何使用批次更新的方法？\n",
    "\n",
    "如果有一些数据包含比较大的错误（异常数据），因此每次更新仅仅使用一个数据会导致不精确，同时每次仅仅使用一个数据来计算更新也导致计算效率比较低。\n",
    "\n",
    "\n",
    "* [梯度下降方法的几种形式](https://blog.csdn.net/u010402786/article/details/51188876)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 如何拟合多项式函数？\n",
    "\n",
    "需要设计一个弹道导弹防御系统，通过观测导弹的飞行路径，预测未来导弹的飞行轨迹，从而完成摧毁的任务。按照物理学，可以得知模型为:\n",
    "$$\n",
    "y = at^2 + bt + c\n",
    "$$\n",
    "我们需要求解三个模型参数$a, b, c$。\n",
    "\n",
    "损失函数的定义为：\n",
    "$$\n",
    "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlFUlEQVR4nO3dd3xUZd7+8c83kwahkxAgBEIglAAiEqQJFkQRC+quggXRRdHVteuuuvv4PG55Vte1rXVRVPDRFUFdrEgRdakSqhBaCC2BFHoo6ffvj4z+cJciZJKTyVzv1yuvnDlzMnON5crJfc65jznnEBGR0BLmdQAREal5Kn8RkRCk8hcRCUEqfxGREKTyFxEJQeFeB/gpYmNjXVJSktcxRESCypIlS3Y65+KO9lxQlH9SUhLp6elexxARCSpmtuVYz2nYR0QkBKn8RURCkMpfRCQEqfxFREKQyl9EJASp/EVEQpDKX0QkBKn8Q4hzjpkZeSzYuMvrKCLisaC4yEuqbtPOgzw6bRX/2rCTyPAwJo/rR6+2Tb2OJSIe0Z5/HVdUWs4zM9dz4bPfsHzrXh4Z3oX4RlGMe2sJO/YdDvj7Oef4x7dbmb9xZ8BfW0QCR3v+ddjX6wt4dNoqtuw6xGU9W/O7i7vSolE053RuwRUvzuOWSelMuXUA9SJ9AXm/otJy7p+ygk9X7gDgxgFJ/GZYl4C9vogEjso/SJVXOOaszefjlds5WFwOVN6O07nKpcKiUhZv3kNybAz/N7YvZ6XE/vCzneIb8rdrenHzpHQemLKCF67thZlVKU/+/iJumZTOypx9/HpYZwoKi3lj3ma+2VDAsyNP57Q2Tar0+iISWCr/ILP7YAmTF2/j7UVbyN5zmNgGkcQ1jMYAM/8XRpjBAxd04pbByUSF/+ee95Cu8Tw0rAt//nwtnWY35O7zU475fiuy93JG26Y0rhdx1G1W5ezj5onp7C8qZfzoNIamxle+R5d4Hpy6gitfms+d56Vwx7kdCPdppFGkNlD5BwHnHCuz9zFpwRY+XrmdkrIK+iU345HhXRmaGk/EKRbquMHJrMsr5JlZ6+kU34CLerQCoLS8gq/XFTB1STaz1+ZRWu4IDzMGdIzlou4tGZoaT2yDKACmr8rl3snLaVo/gqm3DSC1daMfXv+slFim3z2YRz9axTOz1vPlunyeubonyXENqv4PRUSqxJxzXmc4obS0NBdqUzqXVziWbd3DjIw8ZmbksWnnQepH+rjyjARG90uic8uGAXmfotJyrn11IWt2FPL01T1ZunUPHy7bzs4DxcQ2iOTy0xMY2DGWhVm7+HxVLlt3HyLMoE9SM5LjGvCPb7fSq20T/j66Ny0aRh/zfT5ZuZ3ffriK4rJyHr6oK6P7tSMsrGpDTSJyfGa2xDmXdtTnVP7VY31eIX/4JIP8/cUcOZxuZhjQIDqclo2iadk4mvhG0f7lKPYcLGVmRh6z1uSx62AJ4WFG/w7NuaBbS0ac3ppG0UcfeqmK/MIiLn9hHtv3FREeZgzp2oKf907knM5xP/qrwjnHmh2FTF+dy/RVO1ifd4ARp7fmiZ+dRnTEiQ/q5u0v4jfvr+SrdQWc1TGWv/z8NFo3qRfwzyMilVT+NaiiwvHm/M08Pn0tDaPC6ZPUDIf74UBs5T9ux/7DZeTuLyJ3fxElZRU/eo0GUeGc0zmOC7q15JzOcdVS+P8uq+AAC7N2c2G3eJr7h3ROZH9R6UlnqzwVdBt//DQDX5jx2GXduKJXwkkdcF6YtYu3Fm4hrV1Tru/X7pSHvUTqOpV/DcndV8QDU1YwN3MnQ7q04PGfnUZcw+MXqXOOvYdKf/hFEOkLIy2p6VEP0tYlW3cd4v4py1m8eQ8Xdovnf6/occJfOqu37+Mv09fx9foCYiJ9HCwpJzk2hkeGd2VI1xZVPmNJpK5R+deA78e0S8oq+K9LUrnmzESV0QmUVzgmzM3ir1+sJ9xn9ElqRt/kZvRLbk6PhMY/7NFv3XWIp2auY9ry7TSuF8Ht53RgzIAk5mXu5E+frSGr4CADOzbndxen0rVVoxO8q0joUPkHQEFhMf/aUEBJWQWlFY7y8grKKhxlFY7vcvbx6cod9ExswrMjT6d9bIynWYPN+rxCJi3YzKKs3WzIPwBA/Ugfvds1Ja5hFB+v2I4vzLhpYHtuO7vDj045LS2v4O2FW3h29gb2Hy7l6rRE7r+g8wn/4hIJBSr/Kth7qIRXvs5i4vzNHC4tP+o2ET7j9nM68qvzOmr8uYp2Hijm2027WZi1i0VZu9m08yA/692Ge85PIb7Rsc8m2neolL99uYGJ8zdTL8LH3eenMGZAkv59SEhT+Z+CwqJSXp+7mdf+lcWBkjIu69maWwYl07xBJL4wIyIsDJ+v8nuEz3TxUjVxzp3U8FlWwQEe+ziDr9cXkNKiAY9d1o0BHWNP/IMidZDK/yQcLinnrYWbefmrjew5VMqF3eK5b2jngJ1XL9XPOcesNfn8/pPVbNt9mIt7tOK3F3fVaaUSco5X/rrCl8rTMxdu2sWHS3P4fFUuB4rLGNwpjgcu6KQ5aYKQmTE0NZ5BKbGM/yaLF+dk8uXafO4b2ombB7XXgXgRQrz8M/ML+WBpDtOWbydn72FiIn1c1KMVo/okkpbUzOt4UkXRET7uGpLCFb0SeOzjjMozg3Ye4A8jumuYTkJeQMrfzO4FbqbyOqbvgJuAVsC7QHNgCTDaOVdiZlHAJKA3sAsY6ZzbHIgcP9XGggM88sF3LNq0mzCDwZ3i+PWwzlyQ2lLTD9dBic3q8+oNvXlqxnpemJNJ3v5iXri2F/UjQ3rfR0Jclf/rN7ME4C4g1Tl32MzeA0YBw4FnnHPvmtkrwFjgZf/3Pc65jmY2CngCGFnVHD9FeYXjjXmbePKLdURH+Pjt8K6M6NX6uHPSSN1gZjxwYWdaNo7m0WmruGb8Qibc2OeHCepEQk2g/vYNB+qZWThQH9gBnAdM9T8/EbjcvzzC/xj/80OsBgZhN+08yMi/L+CPn65hUEosM+8dzC2Dk1X8Ieb6fu34++g01uUV8rOX57N550GvI4l4osp7/s65HDP7K7AVOAzMoHKYZ69zrsy/WTaQ4F9OALb5f7bMzPZROTT0o/v+mdk4YBxA27ZtTznf93Pt/OWLtUT6wnj66p4nPZeM1C1DU+N555Z+jH1zMT97eT4vXHsG8Y2iOFBcRmFRGYVFpewvKqO8wjG8eysa16/+uZVEaloghn2aUrk33x7YC0wBhlX1dZ1z44HxUHmq56m8Rt7+Iu58Zxnfbt7NeV1a8Ocrexz3QiEJHWe0bcr7vxzAmDe+5ZpXFx5zu6dmrOOR4V21wyB1TiCOeJ0PbHLOFQCY2QfAQKCJmYX79/7bADn+7XOARCDbP0zUmMoDvwEXExXOgeIynvz5afy8dxv9zys/khzXgH/ePpCZGXlERYTRMCqChtHhNIyu/L7zQDG//ySD+95bwbuLt/HHy7vTKV7Xe0jdUOWLvMysL/A60IfKYZ83gXRgMPD+EQd8VzrnXjKzO4Aezrnb/Ad8r3TOXX2896jKRV4VFU43DZFTVlHheC99G49PX8uBojLGDmrPXeelEBOlM4Wk9qv2K3zN7DEqz9gpA5ZRedpnApWnejbzr7veOVdsZtHAW0AvYDcwyjmXdbzXrw0Tu0lo232whCc+X8vk9G20bhzNMyNPp29yc69jiRyXpncQCZAlW3bz4NSV5Ow5zCvX9+bcLi28jiRyTMcrf13mKHISerdrxtTbBpAS34Bxb6Xz2Xc7vI4kckpU/iInqVlMJO/c0o+ebZrwq3eWMnVJtteRRE6ayl/kFDSKjmDS2DMZ0CGWB6as4K0Fm72OJHJSVP4ip6h+ZDivjUnj/K7x/Ne01bzy9UavI4n8ZCp/kSqIjvDx8vVncGnP1jz++Vr+8EkGxWVHv+ObSG2i8hepoghfGM+OPJ0x/dsxYe4mLn1+Lt9l7zvhz+XsPcwXq3MJhjPupO5R+YsEgC/MeGxEd964qQ/7Dpdy+UvzeHrmekrKKv5j23W5hdw3eTln/2UOt761hFlr8j1ILKFO5S8SQOd2bsGMe85mRM/W/G32Bi5/cR5rc/fjnGNR1i5ueuNbLnz2G6avzuWG/km0a16fZ2et196/1Dhdoy4SYI3rR/D0yNO5sHtLfvvhd1z6/FxSWjQkY8d+msVEct/QTtzQvx1N6keS2roRD0xZwYyMPC7s1tLr6BJCdIWvSDXafbCExz5ezbrcQq7r25af90780d3iysorGPrMN0RH+Pj0zrM0D5UElG7gLuKRZjGRPDeq1zGfD/eFced5HbnvvRXMyMhlWPdWNZhOQpnG/EU8dlnP1iTHxvDsrA1UVNT+v8SlblD5i3gs3BfGXUNSWJtbyPTVuV7HkRCh8hepBS7t2ZoOcTE8p71/qSEqf5FawBdm3DUkhXV5hXy2SjOFSvVT+YvUEpec1pqOLRrw3KwNlGvvX6qZyl+klvCFGXcPSWFD/gE+1X0CpJqp/EVqkYt7tKJTfAOem7Vee/9SrVT+IrVIWJhx95BObCw4yNiJi1mXW+h1JKmjVP4itczwHi15ZHgXlmzZw0XPfcNvpq4kd1+R17GkjtH0DiK11J6DJbwwJ5NJCzbjCzPGntWeW8/uQKPoCK+jSZDQDdxFglDTmEj+65JUvrz/HC5IbcmLczZyzpNfMXVJtmYBlSpT+YvUconN6vO3a3rx8a/OIjk2hgemrOCeycspLCr1OpoEMZW/SJDo0aYxk2/tz31DO/Hxiu1c8vxcVmbv9TqWBKmAlL+ZNTGzqWa21szWmFl/M2tmZjPNbIP/e1P/tmZmfzOzTDNbaWZnBCKDSCj4/krgybf2p7Ssgp+9PJ9Xv8nSlBBy0gK15/8cMN051wXoCawBHgJmO+dSgNn+xwAXASn+r3HAywHKIBIy+iQ147O7B3Fu5xb86bM13PTmYnYeKPY6lgSRKpe/mTUGBgMTAJxzJc65vcAIYKJ/s4nA5f7lEcAkV2kh0MTMNIm5yElqUj+Sv4/uzR9GdGNB1i5GvDCPTTsPeh1LgkQg9vzbAwXAG2a2zMxeM7MYIN459/016rlAvH85Adh2xM9n+9f9iJmNM7N0M0svKCgIQEyRusfMGN0/ifdvG0BRaTlXvTKf1dv3eR1LgkAgyj8cOAN42TnXCzjI/x/iAcBVnpd2UoOSzrnxzrk051xaXFxcAGKK1F092jTmvdv6E+kLY9T4hSzevNvrSFLLBaL8s4Fs59wi/+OpVP4yyPt+OMf/Pd//fA6QeMTPt/GvE5Eq6BDXgCm/HEBcwyhGT1jEnLX5J/4hCVlVLn/nXC6wzcw6+1cNATKAj4Ax/nVjgGn+5Y+AG/xn/fQD9h0xPCQiVZDQpB5Tbu1PxxYNuGVSOtOWa79Kji5QN3C/E3jbzCKBLOAmKn+xvGdmY4EtwNX+bT8DhgOZwCH/tiISIM0bRPGPW/px88R0/8VgZVzfr53XsaSWCUj5O+eWA0ebP2LIUbZ1wB2BeF8RObqG0RFM/MWZ/Oqdpfzun6uocI4b+id5HUtqEV3hK1JHRUf4eOm63gxNjefRaat5e9EWryNJLaLyF6nDIsPDePHaMxjSpQW//XAVkxdv9TqS1BIqf5E6LjI8jJeuP4NzOsfx0AffMSV924l/SOo8lb9ICIgK9/HK9b05q2Msv35/JR8szfY6knhM5S8SIqIjfLx6Qxr9k5vzwJQVOg00xKn8RUJIdISPCWP60CepGfdOXs47i3QMIFSp/EVCTL1IH6/f2IfBneJ45MPvePKLtbozWAhS+YuEoJiocF67IY1RfRJ5cc5G7p28nJKyCq9jSQ0K1BW+IhJkwn1h/PnKHiQ2q8+TX6wjb38xr4zuTeN6ukF8KNCev0gIMzPuOLcjz4zsSfqW3Vz1ynxy9h72OpbUAJW/iHBFrzZMvOlMduwr4ooX55GZX+h1JKlmKn8RAWBAx1im3jaACgfjJi2hsKjU60hSjVT+IvKDzi0b8uK1vdiy+xAPTlmps4DqMJW/iPxI3+TmPHxRF6avzmX8N1lex5FqovIXkf8w9qz2XNyjFU9MX8v8jTu9jiPVQOUvIv/BzHji56eRHNeAO99Zxo59OgOorlH5i8hRNYgK55Xre1NUWs7tby/VRWB1jMpfRI6pY4sGPHlVT5Zt3cufPs3wOo4EkMpfRI5reI9WjBuczMQFW3hqxjp2HSj2OpIEgKZ3EJET+vWFncnec4jnv8zk799kcUmPVozu347TE5tgZl7Hk1Og8heREwr3hfHSdb1Zn1fIWwu28MHSbD5YlkOPhMaM7teOy05vTXSEz+uYchIsGC7iSEtLc+np6V7HEBG/wqJS/rksh0kLtrAh/wDdExrxz9sHEu7TSHJtYmZLnHNpR3tO/6ZE5KQ1jI5gdP8kZtw7mCd+1oNVOfv5x7e6MUwwCVj5m5nPzJaZ2Sf+x+3NbJGZZZrZZDOL9K+P8j/O9D+fFKgMIlKzzIyr0xLpn9ycp2auZ++hEq8jyU8UyD3/u4E1Rzx+AnjGOdcR2AOM9a8fC+zxr3/Gv52IBCkz49FLU9l/uJRnZ23wOo78RAEpfzNrA1wMvOZ/bMB5wFT/JhOBy/3LI/yP8T8/xHS6gEhQ69qqEdf2bctbC7ewLlfTQQeDQO35Pwv8Gvj+EsDmwF7nXJn/cTaQ4F9OALYB+J/f599eRILY/UM70yAqnN9/slqzgQaBKpe/mV0C5DvnlgQgz5GvO87M0s0svaCgIJAvLSLVoGlMJPeen8K8zF3MzMjzOo6cQCD2/AcCl5nZZuBdKod7ngOamNn31xG0AXL8yzlAIoD/+cbArn9/UefceOdcmnMuLS4uLgAxRaS6XdevHSktGvDHT9dQXFbudRw5jiqXv3PuYedcG+dcEjAK+NI5dx0wB/i5f7MxwDT/8kf+x/if/9Lpb0SROiHCF8ajl6aydfchJszd5HUcOY7qPM//N8B9ZpZJ5Zj+BP/6CUBz//r7gIeqMYOI1LBBKXEMTY3nhS8zydtf5HUcOYaAlr9z7ivn3CX+5Szn3JnOuY7Ouaucc8X+9UX+xx39z+tWQSJ1zG+Hd6Ws3PHE9LVeR5Fj0BW+IhJwSbEx3DyoPR8szeFPn2ZQXqGR3dpGE7uJSLW4b2gnDhaX8eq/NrF51yGeG3U69SNVObWF9vxFpFqE+8J4bER3/ufSVGavyePqvy8gd5+OAdQWKn8RqVY3DmzPa2PS2FRwkMtfnMeqnH1eRxJU/iJSA87rEs+U2wZgBlf/fQGzdBGY51T+IlIjUls3YtodA+kQ14Bb3krn05U7vI4U0lT+IlJjWjSKZvKt/ejdtin3vrec9M27vY4UslT+IlKj6keG8+oNaSQ0qcctk9LJKjjgdaSQpPIXkRrXNCaSN2/qQ5gZN76xmJ0Hir2OFHJU/iLiiXbNY3htTBr5hUXcPDGdwyWaCK4mqfxFxDO92jbluVG9WJG9l7vfXaYrgWuQyl9EPHVht5Y8ekkqMzLy+OOnGV7HCRm61lpEPHfTwPZk7znMhLmbWL19P0O7xjOkawuS4xp4Ha3OsmCYSj8tLc2lp6d7HUNEqlFFhePlrzfy8YrtrPXfB7h9bAxDurTgvK4tODOpGeE+DVacDDNb4pxLO+pzKn8RqW227T7EnHX5zFqTz8KNuygpr+DKXgk8PfJ0r6MFleOVv36Nikitk9isPjf0T2LSL85k6aNDua5vWz5cnqNrAgJI5S8itVqDqHDuOb8TEb4wxn+jez8FispfRGq9uIZRXNW7DR8szSFft4YMCJW/iASFcYOTKauoYMI83Rg+EFT+IhIU2jWPYXiPVry9cCv7Dpd6HSfoqfxFJGjcdnYHDhSX8faiLV5HCXoqfxEJGt0TGjMoJZbX526mqFRzAVWFyl9Egsovz+7AzgPFvL802+soQU3lLyJBpX+H5vRs05hXv8nSRHBVUOXyN7NEM5tjZhlmttrM7vavb2ZmM81sg/97U/96M7O/mVmmma00szOqmkFEQoeZcdvZHdi86xDTV+V6HSdoBWLPvwy43zmXCvQD7jCzVOAhYLZzLgWY7X8McBGQ4v8aB7wcgAwiEkIu6NaS9rExvPx1JsEwRU1tVOXyd87tcM4t9S8XAmuABGAEMNG/2UTgcv/yCGCSq7QQaGJmraqaQ0RChy/MuHVwMqty9jMvc5fXcYJSQMf8zSwJ6AUsAuKdczv8T+UC8f7lBGDbET+W7V/37681zszSzSy9oKAgkDFFpA644owEWjSM4qWvMr2OEpQCVv5m1gB4H7jHObf/yOdc5d9lJ/W3mXNuvHMuzTmXFhcXF6iYIlJHRIX7GDc4mfkbdzF3w06v4wSdgJS/mUVQWfxvO+c+8K/O+344x/89378+B0g84sfb+NeJiJyU0f3bkdCkHo9PX0OFzvw5KYE428eACcAa59zTRzz1ETDGvzwGmHbE+hv8Z/30A/YdMTwkIvKTRYX7uP+CTqzK2c/HK7d7HSeoBGLPfyAwGjjPzJb7v4YDjwNDzWwDcL7/McBnQBaQCbwK3B6ADCISoi4/PYGurRrx1xnrKCmr8DpO0KjyPXydc3MBO8bTQ46yvQPuqOr7iogAhIUZvxnWmRvfWMzbi7Zw08D2XkcKCrrCV0SC3tmd4hjQoTnPf5lJYZFm/PwpVP4iEvTMjIcu6sLugyW629dPpPIXkTrhtDZNuOS0Vrz2r02629dPoPIXkTrjwQs7U1pewbOzN3gdpdZT+YtIndGueQzX9W3L5MXb2FhwwOs4tZrKX0TqlDuHpBAdHsYTn6/VpG/HofIXkToltkEUt53dgRkZeYwcv5BVOfu8jlQrqfxFpM65/dyO/OmK7mzMP8ClL8zlgSkryNNB4B9R+YtIneMLM67r2445D57DuEHJfLR8O+f+9Suen71B9/71U/mLSJ3VKDqCh4d3ZeZ9gxmcEsdTM9cz5Kmv2bTzoNfRPKfyF5E6r13zGF4Z3Zt3x/Vj/+FS/vRphteRPKfyF5GQ0S+5Obef25FZa/KZvzG07wGg8heRkHLTwCRaN47mfz8L7XsAqPxFJKRER/h4cFhnVuXs56MVoXsPAJW/iIScET0T6J7QiCe/WBeyZ/+o/EUk5ISFGY8M70rO3sO8MW+z13E8ofIXkZA0oEMsQ7q04KU5mew+WOJ1nBqn8heRkPXw8C4cKi3nbyE4C6jKX0RCVscWDRnZJ5H/W7iFrBCbBVTlLyIh7Z7zU4gKD+Mv09d5HaVGqfxFJKS1aBjNbWd3YPrqXGaszvU6To1R+YtIyLt5UDLJcTGMe2sJ4yalh8TcPyp/EQl59SJ9fHbXIB68sDPzMncy9Omveezj1ew9VHfPAlL5i4hQeeXvHed25KsHz+WqtEQmzt/M2U9+xYS5mygpq/A6XsB5Vv5mNszM1plZppk95FUOEZEjxTWM4s9X9uDzuwdzWpvG/OGTDEaOX0BhUanX0QLKk/I3Mx/wInARkApcY2apXmQRETmazi0b8tbYvjx/TS++y97HL95czKGSMq9jBYxXe/5nApnOuSznXAnwLjDCoywiIsd0ac/WPDeqF0u27OHmiel1Zi4gr8o/Adh2xONs/7ofmNk4M0s3s/SCgoIaDScicqSLT2vFU1f3ZEHWLm59awnFZcH/C6DWHvB1zo13zqU559Li4uK8jiMiIe6KXm14/MoefL2+gDveXhr0B4G9Kv8cIPGIx23860REaq2RfdryhxHdmLUmn3smL6OsPHh/AYR79L6LgRQza09l6Y8CrvUoi4jITza6fxLFZRX88dM1xER+x5NX9fQ60inxZM/fOVcG/Ar4AlgDvOecW+1FFhGRk3XzoGRuHZzMlCXZZOYH54Rwno35O+c+c851cs51cM79yascIiKn4uZByYSHGZMXb/U6yimptQd8RURqs7iGUQxNjWfqkuygPPtH5S8icopGndmWPYdKmbE6z+soJ03lLyJyigZ1jCWhST3eDcKhH5W/iMgpCgszRvVJZF7mLrbsCq5poFX+IiJVcFVaImEG7y7eduKNaxGVv4hIFbRsHM15XeKZkp5NaRBd9KXyFxGpomvOTGTngWJmrwmeA78qfxGRKjq7UxytGkfzzrfBM/Sj8hcRqaJwXxhXpSXyrw0FbNt9yOs4P4nKX0QkAK5OawPAe+nBsfev8hcRCYA2Tetzdqc43kvfFhSzfar8RUQCZFSftuTtL+ardbX/BlQqfxGRABnStQVxDaP4x7e1/4pflb+ISIBE+MK4qncb5qzLZ/PO2n3Fr8pfRCSAbhyQRL0IH3/4JMPrKMel8hcRCaAWjaK5a0gKs9fm8+Xa2nvRl8pfRCTAbhrYnuS4GH7/cUatnetf5S8iEmCR4WH8z6Xd2LzrEK/9a5PXcY5K5S8iUg0Gd4rjwm7xvPBlJtv3HvY6zn9Q+YuIVJPfXZxKhXP872drvI7yH1T+IiLVJLFZfX55Tgc+WbmD+Rt3eh3nR1T+IiLV6LazO9CmaT3+56PVtWq+f5W/iEg1io7w8eglqazPO8BbC7Z4HecHVSp/M3vSzNaa2Uoz+9DMmhzx3MNmlmlm68zswiPWD/OvyzSzh6ry/iIiwWBoajyDO8XxzMz1FBQWex0HqPqe/0ygu3PuNGA98DCAmaUCo4BuwDDgJTPzmZkPeBG4CEgFrvFvKyJSZ5kZ/31pKgdLypgwt3ac+lml8nfOzXDOlfkfLgTa+JdHAO8654qdc5uATOBM/1emcy7LOVcCvOvfVkSkTusQ14Bh3VvyzqItHCwuO/EPVLNAjvn/Avjcv5wAHHlHg2z/umOtFxGp88ae1Z79RWW8vzTb6ygnLn8zm2Vmq47yNeKIbX4LlAFvByqYmY0zs3QzSy8oqP1zY4uInMgZbZvSM7EJb8zbTEWF8zTLCcvfOXe+c677Ub6mAZjZjcAlwHXOue8/TQ6QeMTLtPGvO9b6o73veOdcmnMuLS4u7qQ/mIhIbWNm3HxWezbtPMiXa/M9zVLVs32GAb8GLnPOHXnX4o+AUWYWZWbtgRTgW2AxkGJm7c0sksqDwh9VJYOISDC5qHtLWjeO5rW5WZ7mqOqY/wtAQ2CmmS03s1cAnHOrgfeADGA6cIdzrtx/cPhXwBfAGuA9/7YiIiEh3BfGmAFJLMzazert+zzLYf9/pKb2SktLc+np6V7HEBEJiH2HS+n/59kM696Sp68+vdrex8yWOOfSjvacrvAVEalhjetFcHVaIh+v2E7+/iJPMqj8RUQ8cNPAJMoqHJM8mvJB5S8i4oF2zWM4v2s8by/awuGSmr/bl8pfRMQjY89qz55DpXywrOYv+lL5i4h4pG/7ZnRPaMTrczfV+EVfKn8REY+YGWPPas/GgoNMW5FDTZ59qfIXEfHQxT1akxwbw72TV3DJ83OZvHhrjRwD0Hn+IiIeO1RSxj+XbWfSgs2szS2kcb0IRvZJ5Pq+7WjbvP4pv+7xzvNX+YuI1BLOOb7dtJtJC7YwfXUuFc4xvEcrXrimF2Z20q93vPIPr3JaEREJCDOjb3Jz+iY3J3dfEe98u5XyiopTKv4TUfmLiNRCLRtHc9/QTtX2+jrgKyISglT+IiIhSOUvIhKCVP4iIiFI5S8iEoJU/iIiIUjlLyISglT+IiIhKCimdzCzAqAqt7uJBXYGKE6wCLXPHGqfF/SZQ0VVPnM751zc0Z4IivKvKjNLP9b8FnVVqH3mUPu8oM8cKqrrM2vYR0QkBKn8RURCUKiU/3ivA3gg1D5zqH1e0GcOFdXymUNizF9ERH4sVPb8RUTkCCp/EZEQVKfL38yGmdk6M8s0s4e8zlPdzCzRzOaYWYaZrTazu73OVFPMzGdmy8zsE6+z1AQza2JmU81srZmtMbP+XmeqbmZ2r/+/61Vm9g8zi/Y6U6CZ2etmlm9mq45Y18zMZprZBv/3poF4rzpb/mbmA14ELgJSgWvMLNXbVNWuDLjfOZcK9APuCIHP/L27gTVeh6hBzwHTnXNdgJ7U8c9uZgnAXUCac6474ANGeZuqWrwJDPu3dQ8Bs51zKcBs/+Mqq7PlD5wJZDrnspxzJcC7wAiPM1Ur59wO59xS/3IhlYWQ4G2q6mdmbYCLgde8zlITzKwxMBiYAOCcK3HO7fU0VM0IB+qZWThQH9jucZ6Ac859A+z+t9UjgIn+5YnA5YF4r7pc/gnAtiMeZxMCRfg9M0sCegGLPI5SE54Ffg1UeJyjprQHCoA3/ENdr5lZjNehqpNzLgf4K7AV2AHsc87N8DZVjYl3zu3wL+cC8YF40bpc/iHLzBoA7wP3OOf2e52nOpnZJUC+c26J11lqUDhwBvCyc64XcJAADQXUVv5x7hFU/uJrDcSY2fXepqp5rvLc/ICcn1+Xyz8HSDzicRv/ujrNzCKoLP63nXMfeJ2nBgwELjOzzVQO7Z1nZv/nbaRqlw1kO+e+/6tuKpW/DOqy84FNzrkC51wp8AEwwONMNSXPzFoB+L/nB+JF63L5LwZSzKy9mUVSeXDoI48zVSszMyrHgdc45572Ok9NcM497Jxr45xLovLf8ZfOuTq9R+icywW2mVln/6ohQIaHkWrCVqCfmdX3/3c+hDp+kPsIHwFj/MtjgGmBeNHwQLxIbeScKzOzXwFfUHlmwOvOudUex6puA4HRwHdmtty/7hHn3GfeRZJqcifwtn/HJgu4yeM81co5t8jMpgJLqTyrbRl1cKoHM/sHcA4Qa2bZwH8DjwPvmdlYKqe2vzog76XpHUREQk9dHvYREZFjUPmLiIQglb+ISAhS+YuIhCCVv4hICFL5i4iEIJW/iEgI+n8b461CWnDOfAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "pa = -20\n",
    "pb = 90\n",
    "pc = 800\n",
    "\n",
    "t = np.linspace(0, 10) \n",
    "y = pa*t**2 + pb*t + pc + np.random.randn(np.size(t))*15\n",
    "\n",
    "\n",
    "plt.plot(t, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 如何得到更新项?\n",
    "\n",
    "$$\n",
    "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n",
    "$$\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial L}{\\partial a} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i^2 \\\\\n",
    "\\frac{\\partial L}{\\partial b} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i \\\\\n",
    "\\frac{\\partial L}{\\partial c} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c)\n",
    "\\end{eqnarray}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 程序"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch    0: loss = 2.45413e+07, a =   -3.07866, b =    6.90614, c =    4.30938\n",
      "epoch  500: loss = 1.14068e+06, a =   -28.6118, b =     222.36, c =    434.835\n",
      "epoch 1000: loss =     399358, a =   -23.4814, b =    153.657, c =    605.164\n",
      "epoch 1500: loss =     165221, a =   -20.5589, b =    114.526, c =    702.159\n",
      "epoch 2000: loss =    92877.7, a =   -18.8947, b =    92.2414, c =    757.394\n",
      "epoch 2500: loss =    71458.8, a =   -17.9469, b =    79.5512, c =    788.849\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5LElEQVR4nO3dd3xN9xvA8c83EmKLvZtQo/YKRYugZn5WCVVKB6pGf12pDqWqgw4UpYoatWKPn6LkGqUlsVqzRuwVK4gg4/n9ca80IUHkJjfJfd6v133de8/5nnOek8tzz/2uY0QEpZRSzsXF0QEopZRKfZr8lVLKCWnyV0opJ6TJXymlnJAmf6WUckKujg7gUeTPn188PT0dHYZSSqUr27dvvygiBRJaly6Sv6enJ8HBwY4OQyml0hVjzPHE1mm1j1JKOSFN/kop5YQ0+SullBNKF3X+SqmUFxkZyalTp7h165ajQ1FJ5O7uTvHixXFzc3vkbTT5K6UAOHXqFDlz5sTT0xNjjKPDUY9IRLh06RKnTp3Cy8vrkbfTah+lFAC3bt0iX758mvjTGWMM+fLlS/IvNk3+SqlYmvjTp8f53DT5OxMRWLYM1q93dCRKKQfT5O8sDh2CFi2gbVvr89atjo5IqftkypSJatWqUalSJf7zn/9w9erVx9rPtGnT6N+//0PLeXp6cvHixQeW+eKLLx4rhrROk39Gd+sWDB0KlSvDn3/C119D0aLQrh2cOmX/44nATz+BxWL/fasML2vWrOzatYs9e/aQN29exo8f7+iQNPmrdGj1aqhUCT79FDp0gAMH4N13YflyuHHD+ivg5k37HS8iArp0gd69oXFjGDjQvvtXTqVu3bqcPn0agCNHjtCiRQtq1qzJs88+y4EDBwBYvnw5derUoXr16jRt2pTz588/cJ+XLl2iWbNmVKxYkddee424dzJs164dNWvWpGLFikyaNAmAQYMGERERQbVq1XjxxRcTLZcemfRwG8datWqJzu1zj+hoWLkS5s61JvK7n6OI9REWBr//DmXLwvjx0LRp/O1XrIA2baBjR5g3D5Lb0Hf2rPXLJDgYvvgCzp2DMWOgXDn45ReoVSt5+1cpbv/+/Tz11FPWN//9L+zaZd8DVKsGo0c/sEiOHDm4ceMG0dHRdOnShVdffZUWLVrQpEkTJk6cSJkyZdi6dSsffPABgYGBXLlyhTx58mCMYfLkyezfv59vv/2WadOmERwczLhx4+Ltf+DAgeTPn59PPvmE//3vf/j6+hIaGkr+/Pm5fPkyefPmJSIiAm9vbzZs2EC+fPliY7orsXKOFu/zszHGbBeRBP/z6ZV/GjZy80gsIfGrTyy7ljDy81aMbFcAy8A2sHYtHD8OJ09iubWfViU2YIk8BLdvw/Dh8NdfWEpnotWsVvH35euL5YtetHKbj2XYK/GPEWJh5OaR1jcXL8Kvv8KD6l537gRvb9i3D5YsgUGDrP/Jf/sNwsOhbl0YNgyiouzyd1EZ192r7MKFC3P+/Hmee+45bty4wZYtW+jUqRPVqlWjT58+nD17FrCOTWjevDmVK1fm66+/Zu/evQ/c/8aNG+nWrRsArVu3xsPDI3bd999/T9WqVXn66ac5efIkhw4dSnAfj1ouzRORNP+oWbOmZGQjfh8hgUcD4y0LPBoovZf1lvwj80vgkXUi27ZJYJ9mkt8fCfREAttVlfyf5ZTAf9bEls8/Mr98u+Vb6za2/T10ef9akv89JHDa0H+Xj8gvvcc2l8Duz4q4uVl/S7i6SmCnWjLi2w4i58//G+iiRSLZsomUKCGya9f9J3f5skjXrtZ91K4tcvBgCvwFlT3s27fP0SFI9uzZRUQkPDxcnnnmGRkzZoyEhYVJ4cKFEyzfsGFDWbp0qYiIWCwWadiwoYiI/Pzzz9KvX7/7yletWlWOHDkS+97Dw0NCQ0PFYrFI/fr1JTw8PHa/FoslXkx3j5FYOUdL6PMDgiWRvKpX/qkowSv5EAtHLh/Bb4Ff7DpLiAW/+X40ul2UTy5Xoe2UprQcUxvf/GtoZErzy9D2jHuhNMXzl6bZ3FYU+LoATWdaq3W++v0rIiIjaDyjMa7DXGk8ozERkRF8vulzIqMjaTKjCXm+ysNzM5+jUPZCrPH2oMLtnLQ8PJSnR5bBd3ozXt5ykyfmraZD8d+Z+9Zz3Fq+GMs7z+NXegfeYxcx8vnCWNpVhT59rG0JVapgWTyKkTdW33/SHh4wa5a1aunQIetP/3HjICYmpf/cKh3Lli0b33//Pd9++y3ZsmXDy8uL+fPnA9YL1t27dwMQFhZGsWLFAJg+ffpD99ugQQNmz54NwK+//sqVK1di9+Ph4UG2bNk4cOAAf/75Z+w2bm5uREZGPrRceqPTO6SAkZtH4n2nAD5fzrHWhRuDpcANjhS/ytfFrhEQ5MmzkUUJKBfFG0W20zPz0zRyf5IWM5uRJ1N2QiPDEANd9w4Fd+s+V5WxPv+a7Rx5Lt3CI6sHedzz4JnHk8OXD1OpYCW8i3qTJVMWsrhmYdvpbfxx6g/qFKtDnWJ1iJZoomOi2Xp6KzvP7aRcvnIUzlGYK3fCuF6qMJkv3GBrxGEw8HX1u9UzwgushO0rMVkN5fOX56ePn8CcPMlnUXsYEvgXXV5qxz8f9KHz8u50KN8BS4gFHy+f2L+FJcRC0Jkg/P384Zln4LXXYMAAWLoUpk6FEiVS9bNR6Uf16tWpUqUKc+bMYdasWfTt25fhw4cTGRlJly5dqFq1KkOHDqVTp054eHjQuHFjQkJCHrjPIUOG8MILL1CxYkXq1atHyZIlAWjRogUTJ07kqaeeoly5cjz99NOx2/Tu3ZsqVapQo0YNpk6dmmi59EYbfJNh5OaReBf1jp/sjq5j7tzBLAr7g4A1uWlUqjFzc53k9RK76HilCEcz32RTzksIEBOnjTVrJOS4DaE5oMY5F9plqkDRSvUILV2EkTu+p2e1nsz8ayYBHQNij2cJseC3wI++tfoyIXhC7LrHWj7vefq61WNC1B9832ocpTxKcSH8ApO2T2LFoRVUK1SNvNnyEnIlhBNhJ4iW6NjYDYZy+cvhmduTTSc28VGDj+hRtQcHLx7Eb4EfHcp3oEulLta4bV1BLaPeJKgY+PeYBN26Ja3BecMG+OEHqF8f+vaFJExmpRKXUIOhSj+S2uDr8Pr8R3mk1Tr/u/Xmd+vR122dJ3k/cpUhDZFOA4uI6zBXyfxZZmEosQ+v0V5SdmxZYSjiO8tX1h+1yKljf8va3yZJ/uG5ZPCUbpJ/hHWf9+4/7vvE1iW5zj+R5XGPMThwcLwykdGRcuzKMem+qLswFPGe5C2NpzeWQl8XineuLkNdpO7kutJjUQ/J9WUuWbB3wb/H+DKv9H6tkAR6ItK+vciFC7HrRvw+IuE/+M6dIi1aWNsPcuSwPpctK7JsmUhMjN0/X2eTFur81eNLap2/wxP7ozwcmfwTa4wd8fsIiYyOlPFbx0v2z7NLuS+KisuQfxNfls+ySLFviwlDkVa/tJItJ7bItVvXEkyoiSXy3st6J3rsxOJq+UtLuyyPbWxO4Esh7vt7vxguhl+UHot7CEORyj9UlqfGPSVmqIn9u+T+Mrdk+SyLvLP6HZm2Y6rkH5pdAsu4imTPLoF+3tb3K8aJ3Lnzb0BHjvzbaOzhITJypMjNmyLLl4uUK2dd3qSJyO7dSflo1T00+advSU3+Wu3zEHerSgKaTKTBnhtMvryWd8ICqJSpKPuiz3Kd27FlK13LSm+fd3i6ehuu3LrCi4tejFfFAlj3dU81TLxqkTjHDToThH99/1Q/Z0ikSssWk3dR7wTPI+45xj3vWkVrsevcLj7b+Bm/Hf2NbG7ZuBlpHfyVxz0PN++E0ziiMFs5zYI5MQQXA+9L7viUbACFC8OcOVhKGYI61sP/3cWQJ8+/gUZGwsSJ1lHMV6/CK69Yu7gWKpR6f6wMQqt90rekVvto8rdJLNlZ9q/kzpZNjDJbcRG4ZateLnsRGodAgXAYXxv6ZW3AhGz7COiUvpL840jsbzV3z1wWHVj0SF8K3zz3DZExkWw8vpFlB5cRdjsMAM+cJagsBdlw7S9mbSmC78ZzWHo3w6/4FgL8FsQ7ZjxXrsBnn8HYsZAtm/XLoH9/bQ9IAk3+6ZvW+T+mu9UYa4+sleDTwdIzoKu4DnURbFU5OT5xFYYi7aY2l9OHd4pcvCiBfy+z1s8f+i3ePh5UXZORPXS8wgPaFfqv7C85vsgh9abUkxxf5IitJir5XUnJ/nl2mbV71gOPEfu3PXjw33aBChVE1q1L+RPPILTaJ33TOv+HSCh5rD2yVvqu6Csd5nYQl6Eu1sQzBCnfDxnW5ymZuHRwgvXbD01ESkSS/qWw+vBqWR+yXupPqR+vAbnaxGrSc3FP8fjKQ9YdWRdvm3j7j4kRWbpUxMvL+k+8UyeREydS7XzTK03+6Zsm/4eIe4W/8dhGaT+3vbh8ak34WYZkkrIDrY2Tb/d6QiQo6IE9blTyPOjLM26Dct6v8srry1+X+lPqxzYeu3zqIvWn1BePrzzibRNvX/t/lRGfPifi7m4dhfzNN9or6AHSQvJ3cXGRqlWrxj6+/PLLRMsuXrxY9u7dG/t+8ODB8ttvvyU7hitXrsj48eOTvN2QIUPk66+/fmi5uCOG7Xl8Tf42iSWWt1e/LV3md4lN+AxB6r+eWWZVRpZXcZf8g7PI4Ok9YxO8Xt2nvgd94Z69flYmBk0Ur9Fe8X4R9FneR/KOyJvwl3RIiEibNtZ/7r16iURGOvDs0q6kJP+U+n/xsMQYV48ePWT+/PnJOl5CQkJCpGLFiknezl7J/3GP75DkD7wF7AX2AHOwjkv1ArYCh4F5QGZb2Sy294dt6z0ftv/HSf5x//OHhodK///1F9dh1nr7TJ9mkjL+7sJQxP85RFq2lMDJH8X2r793e5W6HpZY7n42b616y9rNdmy52C8Ct2Fu0mZ2m3ifpYhYr/g/+sj6T751a5EbN1LzlNKFpCT/lPpFnFhifP/99+Wpp56SypUryzvvvCObN28WDw8P8fT0lKpVq8rhw4fjfRk88cQTMmjQIKlatarUrFlTtm/fLs2aNZNSpUrJhAkTRETk+vXr0rhxY6levbpUqlRJlixZIiIinTt3Fnd3d6lataq8++67IiIycuRIqVWrllSuXFk++eST2LiGDx8uZcqUkfr160uXLl0STP5Hjx6Vp59+WipVqiQfffRR7Dk+6vETK3evVE/+QDEgBMhqex8A9LQ9d7Etmwj0tb1+A5hoe90FmPewYzxutc+8PfMk82eZY6sKSo8pLaO+bCsLqrhJfn8jgz9vKvm/yqtX+OlIYklnxq4ZMsQyRDy+8hCGIpk/yyx9V/SVgSsHxv9cJ0yQwFJGRnQuHn+COhUvebz565vS8OeGD3xUmVBF3Ia5SclRJcVtmJtUmVDlgeXf/PXNh8Zwb7XP3Llz5eLFi1K2bFmJsVXZXblyRUTuv/K/N/n/8MMPIiLy3//+VypXrizXrl2TCxcuSMGCBUVEJDIyUsLCwkREJDQ0VEqXLi0xMTH3XXmvXr1aevXqJTExMRIdHS2tW7eWDRs2SHBwsFSqVEnCw8MlLCxMSpcunWDy/89//iPTp08XEZFx48bFJv9HPX5i5R70+d31oORvr7l9XIGsxphIIBtwFmgMdLWtnw4MBSYAbW2vARYA44wxxhaoXf2n7H/ImTknlyIu0bt0Z36ceArL6aX4vZiZgPYB+NTsiE+c7oj3diP08fJJvGuhcoigM0HxPisfLx8COgYQdCaIhk80ZHzQeF6q+hLz9sxj8o7JRMZE8kPwD/T37s+nPp+yvXk5/C7lJGDmBahXD1atgiefdPBZpU8e7h4UyVmEE2EnKJm7JB7uHg/f6CHu3skrrqioKNzd3Xn11Vfx9fXF19f3kfbVpk0bACpXrsyNGzfImTMnOXPmJEuWLFy9epXs2bPz4YcfsnHjRlxcXDh9+nSCN4NZs2YNa9asoXr16gDcuHGDQ4cOcf36ddq3b0+2bNniHe9emzdvZuHChQB0796d999/H7BeeD/K8RMrV7hw4Uf6OyQm2clfRE4bY74BTgARwBpgO3BVRO7OEHYK6y8EbM8nbdtGGWPCgHxAvBtpGmN6A72B2MmXkurPU39ijGFwlmZM+HseXcKyE/R2ZwJa98anVGMgfvLQRJ/2JTQe4u7nFvdLvGfVnnSa34kXKr3A/w79j9FbRzM+aDyZXDIxsfVEgkoFwVcz8alXDwICoGhRLEcDCTq/A/+cLaw3w4mKst7sxiP5SS29Gd1i9EPL3B3HMbjBYCYET2BIwyEp8n/I1dWVbdu2sW7dOhYsWMC4ceMIDAx86HZZsmQBwMXFJfb13fdRUVHMmjWL0NBQtm/fjpubG56enty6deu+/YgIH3zwAX369Im3fPRDbkwTl0lg7qpHPf6jlkuqZE/pbIzxwHo17wUUBbIDLZK7XxGZJCK1RKRWgQIFkry9JcSCX0BHAn4vwrAP1hBwog5+3bLg7dsnNvHf5ePlk64GWan7JfSLYH6n+ZTIXYIjA4/wSrVXiIyJ5E7UHXou7cmCG9to1zmGdU+6gI8Plubl8Avsi/fQn+D5560jhXv3hvLlYebMf++UpgDiDeAb5jOMgI4B8aYlt6cbN24QFhZGq1atGDVqVOx0zjlz5uT69euPvd+wsDAKFiyIm5sbFouF48ePJ7jf5s2bM3Xq1Ni7eZ0+fZoLFy7QoEEDlixZQkREBNevX2f58uUJHqd+/frMnTsXsCbypB4/sXLJZY/5/JsCISISKiKRwCKgPpDHGHP3l0Vx4LTt9WmgBIBtfW7gkh3iiCfoTBABvjPwOZEJfv4Zn9l/EOC3gKAzQfY+lEoD/Ov7J1ht51/fn/XH1rPsn2UMbjCYPFnz0KNKD06EneBa5A2atwil1ZeV8HslBwHVhuMzfYP19oUhIdYb3pcqBS+9BI0awUPuEuVMHlT9lhx37+R19zFo0CCuX7+Or68vVapU4ZlnnuG7774DoEuXLnz99ddUr16dI0eOJPlYL774IsHBwVSuXJkZM2ZQvnx5APLly0f9+vWpVKkS7733Hs2aNaNr167UrVuXypUr07FjR65fv06NGjXo3LkzVatWpWXLlnh7eyd4nDFjxjB+/HgqV64ce0/ipBw/sXLJllhjwKM+gDpYe/pkAwzW+v0BwHziN/i+YXvdj/gNvgEPO0ay+vlHRz/+tirdS6yBePXh1TJz90wp8k0RYSiSbXg2Gb5huFy+eTl+4390tMhPP0lglZwy4lkXEX9/kevXHXhGKSct9PNXjy/V7+QlIluxNtzuAP7G+mtiEvA+8LYx5jDWOv0ptk2mAPlsy98GBiU3hgdy0ZuVObPErlB3ndtFsZzFiIyJ5KWqLxEZE8nHlo8pObok289sp+P8jtYqDBcXLE1K49c1M96Vm8PIkVChAmzc6OAzUyp5dGI35ZQs9/TysoRYeD7geaoXrs764+txMS64urjSq0Yv5uyZ8+8XyJYt1vaA48dh4UJo1crRp2I3OrFb+pbUid30slg5pYR+ESz0W0jzJ5tzaMAhetXoRWR0JGO3jaVkrpKUzlvaeg/mIrfh99+tV//t2mGZPpSRm0c6+GzsJz1cDKr7Pc7npslfOaUHNRCX8ihFpwqdyO2eG++i3uw4t4NSY0qx5cQWa3XQ9b8hMBBL87L47f0U778vO+gs7Mvd3Z1Lly7pF0A6IyJcunQJd3f3JG2nN3BX6h53q4QWdLLeP2Dennn0XNqTFYdW4GJcaDW7FX1r9WXms+cI2FYDn69HQExJeOMNR4eeLMWLF+fUqVOEhoY6OhSVRO7u7hQvXjxJ22jyV+oe91YJda7UmYLZC7L6yGou3bzElJ1TGPXnKBqUbID31IUQ+Sr06wc3boB/+h0v4ubmhpeXl6PDUKlEG3yVSoK7DcOFcxRm/8X9eLh70KhkQ95Ye5WmU9fDW2/Bl19iObMl3d2hTWU82uCrlB3crQ5a6LeQff32Ma7lOK7fuc7if5bQ4olNfPZeHWTUKCwtyuM3twPeRRMe9BPrxAlYskRHDyuH0OSv1CO6tzqoX+1+rH5xNT2q9qBozqJ8kn0rnl8UoEPd4wRMvobPtPVw5879O9qzxzpquHRpaN8eEpkWQKmUpNU+StlBVEwUbee0ZeXhlQC8HlaWgjv/oZFraXy+WwyVKsGmTVh+8Cfo1Fb8d2WHXr2siT9XLti+HRKY/Eup5NBqH6VS2Kbjm9h2Zhvv1n2XrK5ZmZT7MN82duc/dY6w9vlqUKMGlh4N8fPchnezntYqn1GjYPBg2LkTli519CkoJ6NX/kolU2Kjhb08vNhxdgeZxND1RG5+9YwiwC8An/It/904Kso6YCxbNtixQ6cjUXalV/5KpaDERgv7VfBjcefF5HDPxcwnrlKkgBcVS9SMv7Grq/Xqf/dua+OvUqlEr/yVSkGWEAt+8/14qsBTbDqxiRxuOWj+ZHP61upLk1JNrIWiorD4eBFUKAr/gNN69a/sRq/8lXKA2OqgTgFsfHkjP7f9mdvRt1m4fyEtZrXg550/W8ud3IRfszC8t5+DRYscHLVyFnrlr1QKGbl5JN5FvePNIRR4NJCJwRNZdWQV1+9cp36J+hy8dJCADnPw8R1grQbavVuv/pVdPOjKX5O/Ug4QGh5K05lN+ev8X+TLmo/lLyyn7u/HoGtXmDcP/PwcHaLKALTaR6k0Zs+FPZy5foaulbty5dYV6k2txzse2xjeIT+WH/whOjq2rCXEkqGmjVZpgyZ/pVJZ3K6hszrMYlmXZbi7uvPd1tFMqB5N+6ePY5k+NF7Zh04VoVQS6ayeSqWye7uGti7bmpVdVzJnzxzWHl3LmagrtAr5nDfX3GHK7qnxyiplL1rnr1QacuPODT78sRNjL68C4JUn2jOlp/YAUo9H6/yVSidyZM5B+9bvkRt3ct2GqccW8/J7ZYk6edzRoakMRpO/UmmIJcSC38LOLH5pJcf7H6FJVAmm5ThE0bGezPy4DYSFxSurDcHqcWnyVyoNidsekKdoKdYOP8Hgyv25mtXwkuty3ulRGJk2DcvRQG0IVsmidf5KpQOnrp2izZTn2HntAGUvwiWPzMzvvBCfir6ODk2lYVrnr1Q6VzxXcYL/u5fnvJryT36IjLxD5tf6gF4Uqcdkl+RvjMljjFlgjDlgjNlvjKlrjMlrjPnNGHPI9uxhK2uMMd8bYw4bY/4yxtSwRwxKZXQbjm1g5/ldvFr9VW64u9Cg2RlafF+btSP6QExMbDltC1CPwl5X/mOAVSJSHqgK7AcGAetEpAywzvYeoCVQxvboDUywUwxKZVhxB4ZNbjOZpS8sxc0tM6tLC63CJ7GwW024cEEHhalHluw6f2NMbmAXUEri7MwYcxBoJCJnjTFFgPUiUs4Y86Pt9Zx7yyV2DK3zV84usUnifgj+geX7lxAp0bxwJBtrKmYhoPNCHRSmgAfX+dtjhK8XEAr8bIypCmwH3gQKxUno54BCttfFgJNxtj9lWxYv+RtjemP9ZUDJkiXtEKZS6Zd/ff/7ljUu1ZjGpRqzL3QfjSY/w+wnr9DwaCQNr+Z2QIQqvbFHtY8rUAOYICLVgXD+reIBwPaLIEk/MURkkojUEpFaBQoUsEOYSmVM52+cJyaTC5Vzl2VDsUjq/ujNZctKR4el0jh7JP9TwCkR2Wp7vwDrl8F5W3UPtucLtvWngRJxti9uW6aUSqK7dfzzO81n95sHeLPCK2wrHMMTv7Xmx2n97yurDcHqrmQnfxE5B5w0xpSzLWoC7AOWAT1sy3oAS22vlwEv2Xr9PA2EPai+XymVuLiDwowxjO40hfENRiAuhtePjeedMa0REW0IVvexyyAvY0w1YDKQGTgKvIz1iyUAKAkcB/xE5LIxxgDjgBbATeBlEXlga642+CqVNBfPh9BiZBW257pB1UzFOJ3lts4O6oRSusEXEdkFJHSAJgmUFaCfPY6rlEpY/kJebB12Fp8hT7Ap52mKRuSmXP5yD99QOQ0d4atUBrXxQhD7C7rQ6UpRzsSEUXlUWbae2vrwDZVT0OSvVAYUOyisUwABI0OYfKwKV6PDqTelLoN+G3RfWW0Idj6a/JXKgOLdLSxzZl79cRsLD1WncJgwYssIOs7rSFRMlDYEOzGd1VMpZ3HrFlFtfOnisY6FFcArjxfXbl9jfqf52hCcQemsnkopcHfHdelyFlxszH8OQsjVEABK5C7xkA1VRqTJXylnkjUrlvHv8oeXGz12weWIS9T8sSabjm9ydGQqlWnyV8qJWEIs+C1/iYAXlzDtdktmLILw29dpPKMxXRZ0wRJiua+8NgZnTJr8lXIisQ3BFVrBsmV0q/0aC+cKJW+5M2/vPHzn+BJ4NBBAG4MzOG3wVcqZicCXX3Lnk4/o+1oRphY5S5ZMWXi77tv8tOMnHRWczmmDr1IqYcbAhx+SedpMJk8N5au/CnI7+jZf/v4lPav21MSfgWnyV0pBt26YVaupfTCcnHcMLrgw6s9RzN0z19GRqRSiyV8pBYDFy+D3YmaW/pqHdWuL4u7qzouLXmTyjsmODk2lAE3+SinA1hjcZSE+o5fQ6I+z/LH3afK65+X1Fa/z7R/fxiurvYDSP03+SinAeqtIHy8faNAARo6k8ux17JDelMhVgnfXvMvHgR8D2gsoo9DePkqp+4lA586wcCFXVy2h4ZGP+ev8XzxX6jl2ntupvYDSCe3to5RKGmNgyhQoV448L77K1hYLeSr/U/x29DfK5StHI89Gjo5QJZMmf6VUwnLmhEWLICKCP/q1ITQ8lOqFq7P55GbazG1DdEy0oyNUyaDJXymVuPLlsYx7B7+K+wk49wzbe2/nhYovsOKfFTSd2ZQ70XccHaF6TJr8lVIPFPRkNgJc/PAZtQTzySfMbjiG3jV6s/7Yep6Z+gzhd8Jjy2ovoPRDk79S6oH86/vjM3wWdOwIw4dDiRL8OD+Cdzy7EnQmiDqT63Al4or2Akpn7HIDd6VUBufqCvPnw9698MMPMGMG38y8Qc4XijCUvZQfV55oidYbw6QjeuWvlHp0FSvC+PFw+jSMH8+Q3R50/Qsu3LyAwVAufzlHR6gekSZ/pVTS5coFb7yBZflY1lTLwUu74GLERbx/8uZE2AlHR6cegd2SvzEmkzFmpzFmhe29lzFmqzHmsDFmnjEms215Ftv7w7b1nvaKQSmVeiwhFvwWdibgxaVMD/NhrCUrZ6+fpfZPtTl65aijw1MPYc8r/zeB/XHejwBGiciTwBXgVdvyV4ErtuWjbOWUUulM7I1hSjWG0aPpv+k2E2405FLEJer8VIeDFw/GltVeQGmPXZK/MaY40BqYbHtvgMbAAluR6UA72+u2tvfY1jexlVdKpSOxcwEBVKkCffrQZ9QmJlb7mEsRl6g7pS57LuzRXkBplL2u/EcD/kCM7X0+4KqIRNnenwKK2V4XA04C2NaH2corpdKzzz6DXLl4dcwmfm47lbDbYdT+qTbPBzyvcwGlQclO/sYYX+CCiGy3Qzxx99vbGBNsjAkODQ21566VUikhXz749FNYt44exz14o9YbRERFcCvqFvmy6fVdWmOPK//6QBtjzDFgLtbqnjFAHmPM3XEExYHTttengRIAtvW5gUv37lREJolILRGpVaBAATuEqZRKca+/DhUqYBnRl7l75tLPux+3o2/z7M/P8vf5vx0dnYoj2clfRD4QkeIi4gl0AQJF5EXAAnS0FesBLLW9XmZ7j219oKSHeaWVUg/n5obl0574PXOWgKh2jGs1juntpnPjzg2e+fkZ/QJIQ1Kyn//7wNvGmMNY6/Sn2JZPAfLZlr8NDErBGJRSqSyoiBBwph4+X8yBM2foVqUb09pO41bULZ75+Rn2XNgTW1Z7ATmO3sxFKWV/hw9bRwN36QLTrZ37Zu6eSc+lPcmZOSe/v/I7oeGh+C3w08bgFPSgm7no3D5KKft78kl45x348ksoUABGjKB71e4IwstLXqbOT3XI4pqFhX4LNfE7iCZ/pVTKGDYMrl+Hb7+1/hKYNYuXqr7E1tNb+SHoB1yMCyVyl3B0lE5L5/ZRSqUMV1cYOxa+/x6WL4cGDbAEBRCwN4DXa75OeGQ49afW5/jV446O1Clp8ldKpawBA2DZMiwR+/Bb+AIB1T5ngu8EJvpOJDQ8lHpT6nHm+hlHR+l0NPkrpVJe69YEDetDQGBefNq/DcuX07tmb8a2HMvFiItUn1idxfsXx9tEewKlLE3+SqlU4d9xND5L/4Ly5aFtW5g/n361+7Gm2xqu3LpCp/mdWHZgGYDOB5QKNPkrpVJPkSKwYQPUqwfdu8PmzTT0bMjyF5ZjjOH5+c/z/m/vaxfQVKDJXymVurJnh6VLoWRJ6y+Af/6h+ZPNWdBpAdEx0YzcMpJeNXpp4k9hmvyVUqkvXz749VdwcYGWLeHCBXJlyUWOzDkA+PaPb1l7dK2Dg8zYNPkrpRyjdGlrF9CzZ7H0bIjffD+WdlnK2JZjuRN9B9/Zvqw7us7RUWZYmvyVUo5Tpw7Mnk3QtQME/F0On5IN6F+7P0MbDuV29G2GbhhKepiCJj3S5K+Ucqx27fD3G4PPL5utU0IAnzT8hAG1B/D7id95bdlr8YprF1D70OkdlFKON3AgHDsGo0bBzp2YNm0Y7fsG+y/uZ+quqeTMkpPRLUbHdgEN6Bjg6IjTPZ3VUymVNsTEwFdfwdy58Ld13v/Isk/SwC+cP13P0qFcezae3KRdQJPgQbN6arWPUiptcHGBDz+Ev/6CkBAYNw43r9IEfneREldh0cHFtHiyhSZ+O9Hkr5RKezw9oV8/WLWKP7ct5mauLBQIh1l//cLk7ZMdHV2GoMlfKZVmWUIs+K3syfy2s9j+c2byRWehz4o+zPl7jqNDS/c0+Sul0qygM0HWOv4az1Oi4yusnxJNVtesDFw1kEs3Lzk6vHRNk79SKs3yr+//bx3/u+9S8VwMv95ow5WIKzSc1pCIyIjYstoFNGk0+Sul0ofSpaFTJ54dv4LBtd9jb+hems5sSlRMlM4C+hg0+Sul0o/334fr1xmyKzcDaw9ky8kt1JlcR2cBfQw6yEsplX5Urw7NmsHo0Yw5dozgM8FsObWFJl5NNPEnkV75K6XSl/ffh/PnsUz6kH8u/0PFAhVZF7KOIZYhjo4sXdHkr5RKX3x8sLQoh9/ZMQR0mENw72AqFqjIsI3DGLttrKOjSzeSnfyNMSWMMRZjzD5jzF5jzJu25XmNMb8ZYw7Znj1sy40x5ntjzGFjzF/GmBrJjUEp5USMIaitNwFzY/DZcQV3V3fW91xP0ZxFGbR2EEcuH3F0hOmCPa78o4B3RKQC8DTQzxhTARgErBORMsA623uAlkAZ26M3MMEOMSilnIh/r2n4uJWxzgUkQv5s+VnfYz0INJreKN4YAO0CmrBkJ38ROSsiO2yvrwP7gWJAW2C6rdh0oJ3tdVtghlj9CeQxxhRJbhxKKSeSKRP4+8OOHbDOesOXMvnK8FXTrzh17RQ+0324HXVbu4A+gF3r/I0xnkB1YCtQSETO2ladAwrZXhcDTsbZ7JRt2b376m2MCTbGBIeGhtozTKVURtC9u/WG8F9+GbtoQJ0BDH52MH9f+Juak2pqF9AHsFvyN8bkABYC/xWRa3HXiXXe6CTNHS0ik0SklojUKlCggL3CVEplFFmywHvvQWAgrP33fr/DGg/Dx9OHvaF7qZC/gib+RNgl+Rtj3LAm/lkissi2+Pzd6hzb8wXb8tNAiTibF7ctU0qppHnjDXjiCWv3z5gYwFrH//eFv6lSqAobT2zkk8BPHBxk2mSP3j4GmALsF5Hv4qxaBvSwve4BLI2z/CVbr5+ngbA41UNKKfXosmSBzz6z1v3PmxfvTl/bXttGpYKV+GzTZ0wI0n4l90r2nbyMMc8Am4C/gRjb4g+x1vsHACWB44CfiFy2fVmMA1oAN4GXReSBt+nSO3kppRIVEwM1asC1a4yc9hreJerGVvWEhodSeUJlbkbeZF+/fRTPVdzBwaauB93JS2/jqJRK/1atgpYtYcwY6/2A49hzYQ/1ptTjybxPsunlTWTPnN1BQaY+vY2jUipja94cGje2VgFdi9ffhEoFKzG341x2ndtFy1ktiZGY2HXOPAZAk79SKv0zBkaMgIsX4euv71vdqkwr+tbqy6YTm+i+qDuA048B0Fk9lVIZQ61a0LkzfPedtRdQkfhjR8e1GsfxsOPM3jObiKgINp3Y5NRjAPTKXymVcXz+Ody5A59+et8qYwyLOi+iZO6SLD6wmHbl2jlt4gdN/kqpjKR0aXj9dZg8GQ4evG/15hObCb8TTq4suZi6ayoL9y10QJBpgyZ/pVTGMngwZM0KgwZBnN6Md+v453eaz/oe63E1rnRZ2IU1R9Y4MFjH0eSvlMpYCha0jvhdsgQaNbIOAAOCzgTF1vFXL1KdGe1nEBUTxScW5xwBrMlfKZXxfPABTJwI+/dbG4Jffhl/r27x6vg7V+rMoPqD2Hp6K2+teive5s7QBVSTv1Iq48mUCfr0gUOH4N13YfZsKFsWhg+HiIjYYsMbD6dOsTqM3jqaMX+OAZynC6iO8FVKZXxHjljn/1+0CEqWtM4CWqYMAGG3wqg0oRKnr51mQJ0BzP57dobpAqojfJVSzq10aVi4ENavh6tX4Z13Ylflds/N2u5rccvkxvdbv6dXjV4ZIvE/jCZ/pZTzaNgQPvwQli8HiyV28ZnrZ3B3dQdg1B+jCDwa6KgIU40mf6WUcxk4EEqUsLYFxMTE1vEv6byEj5/9mFvRt2g7ry2WEMvD95WOafJXSjmXrFnhiy+sXUDnzInXBXRoo6G0eLIFt6JusXB/xh4Apg2+SinnExMD3t7WieAOHLB+IdhcjrhMrUm1uBN9hx19dlAwe0EHBpo82uCrlFJxubjAN9/AiRPw/ffxVuXNmpdFnRdxPvw8zWY2IyomKnZdRur/r8lfKeWcfHzA19daBXTxYrxV1QpX492677L7/G66LuwKZLz+/5r8lVLOa+RICA+HYcPuW/Vl0y9pW64t8/fNp2NAx9h7A2eUbqCa/JVSzuupp+C112DCBPjnn/tWB3QKoHiu4izcv5BOFTplmMQPmvyVUs5u6FBwd7fOB3SPzSc2czPyJtlcszFp+yRWHlqZ+vGlEE3+SinnVriwdRbQRYtg6dLYxXfr+Bd0WsDSF5YSLdF0mNchwwwA0+SvlFJvvw3lykG7dtC+PRw6FK//f9NSTfm00afcjr7N+KDxjo7WLjT5K6VUtmywc6f1NpBr10KFCvjPP4NP7qqxRT5u8DHNSjfjf4f+x46zOxwYrH1o8ldKKbAO9PrwQzh8GF55BcaOhSefhNGj4c4dXIwLv7T/BXdXd1rPas3VW1djN02P/f8dlvyNMS2MMQeNMYeNMYMcFYdSSsVTqBD8+CPs3m0dBfzWW9YJ4a5do0D2AgxvPJxz4edoPbs1IpJu+/87JPkbYzIB44GWQAXgBWNMBUfEopRSCapUCVavhrlzITgYWreG8HD61+7PG7XeYMvJLTSb2Szd9v931JV/beCwiBwVkTvAXKCtg2JRSqnEde5svRPYli3Qpg1ERDCu1TjK5y/P2pC1+Jb1TXeJHxyX/IsBJ+O8P2VbFssY09sYE2yMCQ4NDU3V4JRSKp5OnWD6dOs9ADp0YP2hNYSGh5InSx5m7J7B0gNLH76PNCbNNviKyCQRqSUitQoUKODocJRSzq5bN/jpJywHVuE3ow3z289mTfc1GAx+C/zSXf9/RyX/00CJOO+L25YppVTa9eqrBPVrR8CsO/h8+BPeharzTbNvuBN9hzFbxzg6uiRxddBxg4AyxhgvrEm/C9DVQbEopdQj8393MWQaZR0YljMnb06ZguWYhV8P/UrwmWBqFU1w+vw0xyFX/iISBfQHVgP7gQAR2euIWJRSKsneegveew9+/hlz8CA/t/2ZwjkK03lBZ8JuhTk6ukfisDp/EVkpImVFpLSIfO6oOJRS6rG88w64usLkyeTNmhffsr4cu3KM3it6c/cOiWl58FeabfBVSqk0rVAhaNsWpk2D27fpVKETWd2yErA3gEnbJ6X5wV+a/JVS6nH16gWXLsGSJfh4+bC0y1LcXNzot7Ifzwc8n6YHf2nyV0qpx/Xcc/DEE/DTTwA0KdWEAbUHEC3RuLq48nTxpx0cYOI0+Sul1ONycbHeCWzdOjhyBEuIhRl/zaBr5a6E3gyly4Iujo4wUZr8lVIqOV5+GVxcsEwdHDvPz6wOs+hUoRPL/lnG8A3DHR1hgjT5K6VUchQrBr6+BG1fTkC72bF1/L90+IUyecvwxe9fcPpa2hvDqslfKaWSq1cv/FffwOfv67GLMmfKzPIXlmOMofvi7kTHRDswwPtp8ldKqeRq0QKKF4dJk+ItLpe/HGNbjsVyzEKfFX3irXP0GABN/koplVyurta7f61ZA8eOxVv1crWXaeTZiCk7p/DDth8A0sQYAE3+SillD6+8Yn2eMiXeYmMMizsvpmD2ggxYNYD3f3s/TdwARpO/UkrZwxNPWKt/pk6FqKh4q/K452Gh30JiJIaRW0bSt1Zfhw/+0uSvlFL20qsXnDkDv/5636rI6EiyuWUDYPSfo7GEWFI7ung0+SullL34+kLhwvc1/N6t41/SeUlsPX/H+R0d+gWgyV8ppezFzc066GvlSjh8OHZx0JkgAjoG8Fzp5/ilwy9ESzReebzYdnqbw0LV5K+UUvY0YABky2ad89/Gv75/bB1/2Xxl+a7Zd2w/ux13V3dHRanJXyml7KpIEfjkE1ixAv73vwSL9K7ZG9+yvry/9n32XNiTygFaafJXSil7e/NNKFcO/vtfuH37vtXGGKa0mYKriytt57bldtS/ZVJr8Jcmf6WUsrfMmeH77631/t99l2CRgtkL8uGzH3L0ylFeWvwSkLqDv8zd242lZbVq1ZLg4GBHh6GUUknToQOsXg0HDkCJEgkWaTOnDcv/WU63Kt1YdXiVXQd/GWO2i0iCd5TXK3+llEop330HMTHWm70nYs7zc8ibNS+//PULL1d7OdUGf2nyV0qplOLpCYMGwbx5YEm4T/+209uIkRgAvt/6far1/dfkr5RSKcnf3/olMGAAREbGW3W3jn+R3yI+evYjbkffpt28dqnyBaDJXymlUlLWrDB6NOzdCz/8EG/V3cFfPl4+fNLwE6oVroaLccFyLI0nf2PM18aYA8aYv4wxi40xeeKs+8AYc9gYc9AY0zzO8ha2ZYeNMYOSc3yllEoX2rSB5s2t/f/Pn49dHHfwV+ZMmZnRbgY3I2+yN3QvKd0ZJ7lX/r8BlUSkCvAP8AGAMaYC0AWoCLQAfjDGZDLGZALGAy2BCsALtrJKKZVxGQNjxsCNGzBqVKLFKheqzLBGw1i0fxGz/p6VoiElK/mLyBoRuTt36Z9AcdvrtsBcEbktIiHAYaC27XFYRI6KyB1grq2sUkplbOXKWbt+/vij9UsgEe/We5cncj/B6yte59S1U7HL7T34y551/q8Ad+cxLQacjLPulG1ZYsuVUirje+stuHoVpk9PtEgml0wMbzyc8Mhw2s9rj4ikyOCvhyZ/Y8xaY8yeBB5t45T5CIgC7PY7xRjT2xgTbIwJDg0NtddulVLKcerWhdq1rVVAMTGJFutWpRtv1nmT4DPBtJ7dOkXu/PXQ5C8iTUWkUgKPpQDGmJ6AL/Ci/NtCcRqIO5ytuG1ZYssTOu4kEaklIrUKFCiQ5BNTSqk0xxh4+204dCjRSd/uGtV8FKU8SvHr4V95vebrdh/8ldzePi0Af6CNiNyMs2oZ0MUYk8UY4wWUAbYBQUAZY4yXMSYz1kbhZcmJQSml0pXnn7dO9ZDInD93rT+2nrBbYXzwzAdM3D7R7n3/k1vnPw7ICfxmjNlljJkIICJ7gQBgH7AK6Cci0bbG4f7AamA/EGArq5RSzsHV1Trga/162LUrwSJ36/jnd5rPF02+IKBjAH4L/Oz6BaATuymlVGq7ehWKF7f+Ckig8Xfk5pF4F/WOV9VjCbEQdCYI//r+j3yYB03spslfKaUcYeBAmDgRjh+33gAmBeisnkoplda8+SZERcH48Q45vCZ/pZRyhNKlrdM+TJwIN28+vLydafJXSilHeestuHQJZs5M9UNr8ldKKUdp0ABq1LDO+vmAQV8pQZO/Uko5ijHWq/8DB2D2bEjFDjia/JVSypH8/KBsWejeHWrWhClTUqUNQJO/Uko5UubMsGOHdbbPqCh47TXrGID33oOjR1PssJr8lVLK0bJnh969Yfdu2LABmja1zvv/5JPQuXOKVAe52n2PSimlHo8x1kbgBg3g9GmYNMn6a8AYux9Kk79SSqVFxYrBp5+m2O612kcppZyQJn+llHJCmvyVUsoJafJXSiknpMlfKaWckCZ/pZRyQpr8lVLKCWnyV0opJ5QubuNojAkFjidjF/mBi3YKJ71wtnN2tvMFPWdnkZxzfkJECiS0Il0k/+QyxgQndh/LjMrZztnZzhf0nJ1FSp2zVvsopZQT0uSvlFJOyFmS/yRHB+AAznbOzna+oOfsLFLknJ2izl8ppVR8znLlr5RSKg5N/kop5YQydPI3xrQwxhw0xhw2xgxydDwpzRhTwhhjMcbsM8bsNca86eiYUosxJpMxZqcxZoWjY0kNxpg8xpgFxpgDxpj9xpi6jo4ppRlj3rL9u95jjJljjHF3dEz2ZoyZaoy5YIzZE2dZXmPMb8aYQ7ZnD3scK8Mmf2NMJmA80BKoALxgjKng2KhSXBTwjohUAJ4G+jnBOd/1JrDf0UGkojHAKhEpD1Qlg5+7MaYYMBCoJSKVgExAF8dGlSKmAS3uWTYIWCciZYB1tvfJlmGTP1AbOCwiR0XkDjAXaOvgmFKUiJwVkR2219exJoRijo0q5RljigOtgcmOjiU1GGNyAw2AKQAickdErjo0qNThCmQ1xrgC2YAzDo7H7kRkI3D5nsVtgem219OBdvY4VkZO/sWAk3Hen8IJEuFdxhhPoDqw1cGhpIbRgD8Q4+A4UosXEAr8bKvqmmyMye7ooFKSiJwGvgFOAGeBMBFZ49ioUk0hETlre30OKGSPnWbk5O+0jDE5gIXAf0XkmqPjSUnGGF/ggohsd3QsqcgVqAFMEJHqQDh2qgpIq2z13G2xfvEVBbIbY7o5NqrUJ9a++Xbpn5+Rk/9poESc98VtyzI0Y4wb1sQ/S0QWOTqeVFAfaGOMOYa1aq+xMeYXx4aU4k4Bp0Tk7q+6BVi/DDKypkCIiISKSCSwCKjn4JhSy3ljTBEA2/MFe+w0Iyf/IKCMMcbLGJMZa+PQMgfHlKKMMQZrPfB+EfnO0fGkBhH5QESKi4gn1s84UEQy9BWhiJwDThpjytkWNQH2OTCk1HACeNoYk83277wJGbyRO45lQA/b6x7AUnvs1NUeO0mLRCTKGNMfWI21Z8BUEdnr4LBSWn2gO/C3MWaXbdmHIrLScSGpFDIAmGW7sDkKvOzgeFKUiGw1xiwAdmDt1baTDDjVgzFmDtAIyG+MOQUMAb4CAowxr2Kd2t7PLsfS6R2UUsr5ZORqH6WUUonQ5K+UUk5Ik79SSjkhTf5KKeWENPkrpZQT0uSvlFJOSJO/Uko5of8DYFgMPLNjsOMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_epoch = 3000              # epoch size\n",
    "a, b, c = 1.0, 1.0, 1.0     # initial parameters\n",
    "epsilon = 0.0001            # learning rate\n",
    "\n",
    "N = np.size(t)\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    for j in range(N):\n",
    "        a = a + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)*t[j]**2\n",
    "        b = b + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)*t[j]\n",
    "        c = c + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (y[j] - a*t[j]**2 - b*t[j] - c)**2\n",
    "    \n",
    "    if i % 500 == 0:\n",
    "        print(\"epoch %4d: loss = %10g, a = %10g, b = %10g, c = %10g\" % (i, L, a, b, c))\n",
    "    \n",
    "    \n",
    "y_est = a*t**2 + b*t + c \n",
    "\n",
    "\n",
    "plt.plot(t, y, 'r-', label='Real data')\n",
    "plt.plot(t, y_est, 'g-x', label='Estimated data')\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. 如何使用sklearn求解线性问题？\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  (100, 1)\n",
      "Y:  (100, 1)\n",
      "a = 2.743186, b = 4.076122\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmaElEQVR4nO3de3TU1bn/8fdOCBBEjBcOQgDBG15qhRKpknrDS9RiQVSEKsV6oe3vtNW2h0qtp+iyLbTaY/15a4OAUhWoyg9j8cYSPRa8VC6KoiAWKxJBsBJEiZDA/v2xEwxhkswk38v+znxea7lIxmRmz0zmmWee/ey9jbUWERFJnry4ByAiIq2jAC4iklAK4CIiCaUALiKSUArgIiIJ1S7KGzvooINsnz59orxJEZHEW7JkycfW2q6NL480gPfp04fFixdHeZMiIolnjHk/1eUqoYiIJJQCuIhIQimAi4gklAK4iEhCKYCLiCRUpF0oIiLZYu6ySm55ehUfVlXTo6iQ8WX9GD6gONIxKICLiGRo7rJKfjHnDaprdgJQWVXNL+a8ARBpEFcJRUQkQ7c8vWp38K5XXbOTW55eFek4lIGLiLSgcbmksqo65c992MTlYVEAFxFpRqpyiQFSHYXTo6gw0rGphCIi0oxU5RILmEY/V1iQz/iyfpGNC5SBi0iafOi6iENTZRELFBcVqgtFRPzmS9dFHJqqeRcXFbJowpAYRvQllVBEpEW+dF3EYXxZPwoL8ve4LI5ySSoK4CLSoqbKCFF3XcRh+IBiJo04juKiQgywf6cCOrTL4yezX6N08gLmLquMbWxpB3BjTL4xZpkx5m913/c1xrxijHnXGDPbGNM+vGGKSJya6q6IuusiLsMHFLNowhBuu6Q/X9Tsoqq6BsuXpaS4gngmGfg1wNsNvv8dcJu19nBgM3BlkAMTEX/4XEaIkm+lpLQCuDGmJ/BN4N667w0wBHik7kfuB4aHMD4R8UDjMkJxUSGTRhyX9ROYjflWSkq3C+WPwM+Bfeu+PxCostbW1n2/Dkj5TBpjxgHjAHr37t3qgYpIvIYPKM65gN1YUx0pcZWSWszAjTFDgY3W2iWtuQFrbbm1tsRaW9K1615ncoqIJIZvpaR0MvBS4FvGmPOAjkAX4HagyBjTri4L7wnENxUrIhKB+k8gvixoMtamWtHfxA8bcxrwX9baocaYh4FHrbWzjDF/ApZba+9u7vdLSkqsTqUXEcmMMWaJtbak8eVt6QO/DvipMeZdXE18ahuuS0REMpTRUnpr7fPA83VfrwEGBT8kERFJh1ZiiogklDazEpGclfQdFhXARSQnZcMOiyqhiEhO8m1ZfGsogItITvJtWXxrKICLSE7Khh0WFcBFJCf5tiy+NTSJKZKmpHcsyJ58WxbfGgrgImnIho6FbBPEG2rSd1hUCUUkDdnQsZBN6t9QK6uqvTgZJy7KwEXSkA0dC0GKu5zU3BtqkjPqTCkDF0lDNnQsBMWH7FdvqI4CuEgasqFjISg+lJP0huoogIukQWdCfsmH7FdvqI5q4CJpSnrHQlB8OBcyG1oAg6AALiIZGV/Wb4+WSogn+9UbqgK4iGRI2a8/FMBFJGPKfv2gSUwRkYRSBi4iEpKwFzwpgItkmbhXSWpMX15/2PvnqIQikkV8WCWpMTlRLHhSABfJIj6skmwsV8cUxYInBXCRLOLDKsl0bzvbxxTFcn8FcJEs4uMeIbk6piiW+yuAi2QRH/cIScqYALbtqA2sDh7F/jnqQhHJIj6ukvR5TDdWrKCqumb35Zu31QTaKRL2gidjrQ3tyhsrKSmxixcvjuz2RCS3tdQqWDp5QcqNuYqLClk0YUiUQ22WMWaJtbak8eXKwEUkK6XTh+3jBGsmVAMXkYzMXVZJ6eQF9J0wj9LJC7w9hzKdVkEfJ1gzoQAuImnzcVFOU9LJrn2cYM2EAriIpM3HRTlNSSe7juSkpQ0b4I47YNeu4K6zjmrgIpK2JNWM0z14IpROkZ07Yf58mDIFKiqgthZOPBFOOCHQm1EGLiJpS1LNOJZzTCsr4de/hsMOg3PPhRdegGuvhZUrAw/eoAxcRDLgy3Fq6Yrk4ImdO+Gpp6C8HP72N1cqOfNM+P3vYdgw6NAhtJtWABcJkY/bqLaFj4tyYrN2LUybBlOnwrp10K0b/PzncNVVLgOPQIsB3BjTEXgB6FD3849YaycaY/oCs4ADgSXAGGvtjjAHK5IkUewHHYecPk6tthbmzXPZ9lNPgbVQVga33w7nnw8FBZEOJ50a+HZgiLX2eKA/cI4x5kTgd8Bt1trDgc3AlaGNUiSBktSxIS3417/ghhugd28YPhyWLYPrr4c1a+DJJ2HEiMiDN6SRgVu31v6zum8L6v6zwBDg23WX3w/cCNwT/BBFkilJHRuSQk2N6yCZMgWeeQaMcROT48bBeedBu5Yr0F4cqWaMyceVSQ4H7gL+CVRZa2vrfmQdkHJUxphxwDiA3r17t3W8IonRo6gw5T4bPnZsSAPvvgv33gv33QcffQQ9e8LEiXDFFdCrV9pX482Ratbandba/kBPYBBwVLo3YK0tt9aWWGtLunbt2rpRiiTQ6Uel/ntv6vIkScpy+rRt3w6zZ7vukSOOgFtvdX3b8+a58snEiRkFb4imhJZRF4q1tsoY8xxwElBkjGlXl4X3BBL+DIoE67mVmzK6PCmyanJ21SpXIrn/fvj4YzjkENfH/d3vQo8ebbpqL45UM8Z0NcYU1X1dCJwFvA08B1xU92NjgccCG5VIFsjWGnjiJ2e/+AIeeghOOw2OOsp1kJx6qusqWbMGfvnLNgdv8OdIte7Ac8aY5cCrwHxr7d+A64CfGmPexbUSTg1sVCJZIEmrFjOR2Demt96Cn/wEiovh0ktd7/akSfDBB/DII64dMC+4xelRbJSVThfKcmBAisvX4OrhIpJC0lYtpitRk7PbtrngXF4Oixa5Vr8RI+Dqq+H009scsJvrMoli0ZNWYoqEJIwXsA8rOxPxxrR8uatt/+UvsGULHHkk3HILjB0LATVTpDMXEPaiJwVwkRAF+QL2ZfLQ2+X0n3/uOknKy+GVV9weJBde6Pq2TznF9XEHqLm5gKgeCwVwkYTwIWDU82o5/bJlLmg/+CBs3QpHHw233QZjxsCBB4Z2sz7MBSiAiySEDwHDG1u3wsyZLnAvWQIdO8LIkS7bHjw48Gw7FR/mArQfuIhHmlsgk61dLWmzFl591U1Adu8O3/se7NjhTrv58EPXy11aGknwBj+OY1MGLuKJlmrciZg8DMOWLa48MmUKvPYadOoEo0a5bHvQoMgCdmM+zAUogIt4oqUad5gBw4fulj1YCy+/7IL27NmuHbB/f7jnHhg9GvbbL76xNRD3XIACuIgn0qlxhxEwfOluAWDzZnjgAVfbfvNN6NwZLrvMlU0GDowt2/aVArhIRFrKcuOaFIu9u8VaWLjQZdsPP+yWup9wggvio0bBvvs2++vefXqIkAK4SATSyXLjqnHH0d0yd1klf57zDwa/+ARj3niGPpvWQpcubsvWq6925ZI0r8ebTw8xUAAXiUA6WW5ck2KRZv7WsvDeR2h3+93MXbmQDjtrWdqjH+VDf8JJv/gB5w8+IqOri/3TQ8wUwEUikG6WG8ekWCSZ/8aNrs1vyhS+sXo1Wzrsw0P9z2XW8WWs6toHgP994YOMA3iu98YrgOeAXK4R+sKHRR9NCS3z37ULFixwtey5c90RZd/4Bj898nzm9Stle0GHPX4806A7d1klecaw09q9/p8Pj2sUFMCzXK7XCH3hew93oJn/hg0wfbo7lmzNGjjgAPjhD11t++ijeWXyAra38c2s/u86VfBu6+OapIQnEQE8SQ+ob3K9RuiLoLJcn14L9WOprKqmwO5i8HvLuOKt+Zy86mXyamvdgQm//jVccIFb6l4niDezVH/XAPnGMGnEca1+TJKW8HgfwJP2gPom12uEPmlrluvTa6F+LF0++YgfLZ/PJcvn0/PTjfy7sAvTTxhO75//mLOGn5zyd4N4M2vq73eXtW16LJKW8HgfwJP2gPrG59qrZMab18LOnSz643383xcfZ8g/XyXf7uLvh/Tnt6dfwfwjvk5NfgHFK2s4q5mraOubWVh/10lLeLwP4El7QH3je+1V0hf7a2HtWpg2DaZO5ZZ169i0TxF/+vqFzP7q2azdv3ukYwrr7zppCY/3ATxpD6hvfNhwR4IRy2uhthbmzXOdJE8+6S4rK+P6U6/ir937U5ufOoS0NKa21vLD+rtOWsLjfQBP2gPqo7g33JFgRPpaeO89mDrVZdzr17tT2n/5S7jySujTh0HLKvl/c96gNsVEYktjCqqW39LfdWveJJKW8HgfwJP2gIqEpa2vhRYDWk0NVFS4bHv+fLdx1Hnnufa/886Ddl+Gi4ZjqayqJr+uH7s4jTFFUctvy5tEkhIeY1P0UYalpKTELl68OLLbk+ziUwtd0jQOaAAGuPTE3vz6K4WuZ3v6dLdislcvl2lfcYX7OmB9J8wjVdQxwHuTvxnIbZROXpCy3FRcVMiiCUMCuY0oGWOWWGtLGl/ufQYuAn610CVR46y3fW0NZ69+iXNnPQXvL4f8fBg61B2SUFbmvg9JFLX82Cd8I6Ij1SQRmvvYLS2rD1yH/nsd1y+Yykt3j+XOit9zyOYN/PnsK1yHydy5rlQSYvCGaI4iy5Xj55SBSyLkSkYVii++4PL3FlH20uOc+MGb1OTlM//wrzOz/zks7NMfTB7f69EjsuFEMa8V9oSvL+U8BXBJhGxuJw0tGKxY4Q5J+MtfmPjJJ/yrqDuTT72cR447g4/32X/3jxXH8BiGPVEY9vFzvpTzFMAlEbK1nTTwYLBtmzvVZsoUWLQICgpgxAgYN46pVQfywD/W7TGBmA2PYVPCepPwZkUsCuCSENnaThpYMFi+fHe2zZYtcOSRcMstMHYsdO0KwM3AwL4HZd1jGDWfynkK4JIYSerPTVebgsHnn7sT28vL4ZVXoEMHuOgi17d9yikpDwDOxscwaj6V89SFIhKjVnVLLF0KP/gBdO/u+rW3boXbboPKSnei+6mn6vT2EEXRRZMuZeDiBV9m9aOWdm1/61aYOdNl20uWuP21L7nEZduDBytgR8incp4CuMTOp1n9qDUbDKyFxYtd0J4505VMjjsO7rgDLr0U9t+/hWuXsPhSilIAl9j5NKsfh72CwZYtcPfdblLytdegUycYNcqtkhw0SNm27KYALrHzaVY/NtbCyy+7bHv2bKiuhgED4J574Nvfhi5d4h6heEgBXGLn06x+5DZvdq1/U6bAm29C584wZozLtgcOjHt04jl1oUjsfJrVj4S18Pe/u0Ddowdccw0UFrogvn49/PnPCt6SlhYzcGNML2AG0A2wQLm19nZjzAHAbKAP8C9gpLV2c3hDlWzl06x+qD7+GGbMcIF65UpXFrniCtdJ0r9/3KPLCdnW7dTifuDGmO5Ad2vtUmPMvsASYDhwOfCJtXayMWYCsL+19rrmrkv7gUvOsRaef94F7UcfhR074KSTXNAeORL22SfQm8u2ABWkVHuiFxbkM2nEcd4/Rq3eD9xaux5YX/f1VmPM20AxMAw4re7H7geeB5oN4CI5Y+NGuP9+F7hXr4aiIvj+913g/spXQrnJXG7HTEc2djtlNIlpjOkDDABeAbrVBXeADbgSS6rfGQeMA+jdu3erByr+UJbXhF274NlnXdCeO9cdUXbyyfDf/+2WuBeGOymbjQEqSNnY7ZR2ADfGdAYeBa611n5qGvSiWmutMSZlLcZaWw6UgyuhtG24ErfWZHlZH/A3bGDFb26n6MH7Kd68ni2F+7Jp5OUc/sufwNFHRzaMuAOU789zNnY7pdWFYowpwAXvB621c+ou/qiuPl5fJ98YzhDFJ5mejFMf8CurqrF8GfDnLquMYLQh2rkTnnoKRoxgV69eHHvnZNbuexA/Pn88g/7P/ZzfdwRzv4i2dzvOU2iS8DxnY7dTiwHcuFR7KvC2tfZ/GvyvCmBs3ddjgceCH574JtMsL+uOQqushJtvhkMPhXPPhYULmTl4BKdf/WdGj55ExTGnsr1d+1Du49xllZROXkDfCfMonbxgr+AYZ4BKwvM8fEAxk0YcR3FRIQZ3kEUSJjCbk04JpRQYA7xhjHmt7rLrgcnAX40xVwLvAyNDGaF4JdOPoXF/rE9Xsx//a2tdtl1eDvPmuVr3mWfCrbfCsGHc8Kv5KU9ZD/I+plO6CqMdM92ySFP3tbKqmtLJC7wpq7R2DxNfy0PpdKEsBJrafOGMYIcjvsv0ZJwk1B2bCo6F6yspe2UeTJsG69ZBt25w3XVuC9fDDtv9+1Hcx3QnKDMJUC0FpUzmO5p6DOp/r/7f8Y+8nvL3feZzd49WYkpGMv0YmoS6Y8Pg2G5nLWe/8xJ3PfTfnDX0RLj5Zj7qfTi/uOwmjhjzZ0q7nMncTzvu8ftR3MegP8mkU7POpCyS6jFIpWan5abHV7RqzHHxuTykvVAkY5lkeUlYZflhVTU9qzZwyfL5jHxjPt0++4QNnQ/gzpMu4ajrr+Gal6siL100FnSWn05Gn8mbRqrHoKmMfPO2mlaNOS4+lwEVwCV0vuydvJcdO6CigllzfscJq5dgjeG5Qwdyfdl/8vyhJRx8QGd4Y1vgpYvWCPpQ53SCUqZvGo0fgz4T5rVqbL7xuQyoEorknnffhQkToFcvuPhijtv6IXedciml35/GVRdN5NnDv077Du0ZX9YvlNJFc50kTQm6gyKdlsO2loaKCgsyutxXPpcBlYFLTqh4ZQ2Lb59O2YsVlL6/nF35+eQNHQrjxtGprIxeyzeQ//QqTKMSyC1Prwos+2rrZFiQWX46GX1bS0M3futYxj/8OjW7vuzRKcgz3PitYwO5D1HxuQzY4mZWQdJmVhK5VatY/ZvbOPDRmRyw7VM+2K8bs756No9/rYyfjj2txRdhkBsglU5ekPLNoLiokEUThmR0XUGIojXO1/a7pGn1ZlYiifPFF27nv/JyeOEF+ubl88zhX2dm/3NY2Kc/1rjKYTp7hASZffk2GRbF3IS38x9ZQgHcY8peMrRihdtIasYMd9LNYYfB5MmctK6YTfvsfQBwOoEzyOfA58kwSSZNYnoqCXtLeGHbNrdta2mp26b17rvh7LPdroDvvAPXXUf74h4pf7WlwBn0c+DzZJgkkwK4p3xePBC3ucsq+c41U5gxcChbD/wPuPxyd9rNrbe6vUpmzYIhQyDP/Xm3NnAG/RwkaS+O1nbLSLRUQvGUb/VSL3z2Gctu+RN9p97LjMpVbM8v4Il+pcwZeB4XXjua4V/rmfLXWlvHDuM5SEJN2Oel47KnnA7gPteYVS9tYOlSNyH50EMM2LqVdw7szU1nXM2cY4ewpXBfANY8806TARxaFzhz9TnQwRDJkbMB3PcsI+iVd4mzdSvMnOkC95Il0LEja886n2u7DGJp8VFg9txfLYxPJrn6HOjTX3LkbA3c9xpzkuqlgbEWXn3VnRvZvTt873tuufsdd8CHHzJ68PdZ2vPovYI3hJMV5+RzQLwHQ0hmcjYDT0KWkYR6aSC2bIEHH3TZ9uuvQ6dOMHq0C+SDBu0O2M09N2FlxTnzHDSQq588kihnA3iu1je9YS28/LIL2rNnQ3U1DBgA99wD3/42dNn7OLKmnrP9OxXkXJANk89Lx2VPORvAlWXE5JNP4IEHXOBesQI6d4YxY2DcOBg4sNlfbeo5m3j+3ntr+DxBnQS5+MkjiXI2gCvLiJC1sHChC9oPPwzbt8MJJ7hVk6NGuSCehnSfM98nqH2kN7xk0mZWEp6PP3bL2qdMgZUrXVnksstcbbt//9Bu1rdNo3wX5IZd9denN4NgaTOrLOPti8RaeP55l23PmeO6SE46CaZPh4svhn32CX0ISZig9un5C7LvW59+oqUA3gSfXmCNx1RZVY2B3Sehe/Ei2bgR7rsP7r0XVq+GoiL+eeFlTPyPwSzqeDA9NhQy/p0qhg8IP4D7PkHtW5AL8g1Pi4CilbN94M3xcSOphmOCL4N3vVh62HftgvnzYeRI6NnTndh+8MEwYwYVT7zK0MMuZmHHgyN/DH3fNMq3NQhB9n0n4dNPNlEAT8G3FxikHlNjkb1I1q+HSZPg8MPdzn8LFsCPfgRvvQUvvABjxvC7/10b22MYxAKcMDdz8i3IBfmGp0VA0VIJJQXfXmDp3naoL5KdO+GZZ9yEZEWF+/700+E3v4ELLoCOHff48bAfw5ZKXG1pgwu7xOFbiSfIjiy150ZLATwF315g9bedakz1QnuRVFbCtGmutr12LXTtCj/7GVx1FRxxRMbjDeIxDDvAhl3H9THIBdX3rfbcaCmAp+DTC6ypiUtg9/fFQb9IamvhqadcJ8m8ea7WfdZZbr/tYcOgffsWryLMxzDsABv2p4dsD3JaBBQdBfAUfHmBNc40LSEGbXAZ9tSp7r/KSjched11cOWV7niyDIT5GIYdYKP4BKYgJ0FQAG+CDy+wVJlmffAObEFKTY3LssvLXdYNUFbmdgAcOhQKClp91WE9hmEHWJ8+gYk0RwHcY63JNOcuq+Smx1eweVsNAEWFBdz4rWP3DqTvvefq2tOnu66S4mK44QaXbR9ySGD3IQxhB1hfPoGJtEQB3GOZZppzl1Uy/pHXqdn5ZaW8qrqG8Q+/DsDwY7u6DpLycte/nZcH553nNpI691xol4w/hygCrA+fwERakoxXbI7KNNO85elVewTvej3+Xcln186Alc/Bxo1sO7gHD505lqlHnE5e716M79mP4QkJ3vUUYEUUwL2WaabZsLTSvraGsndeZNTypyl9fzm1Jg+GfYuXhlzAVRsO5PP69wQfluGLSKsogHsuk0yzR1EhHf65mlGvP81Fbz7LAdWf8sF+3fj9Kd9hUek3eey3I/mvyQv4fOeeZZls3avCx/1sRIKkAJ4Nqqvh0Ud57OE7OWjpK9Tk5fPMEScy6/gyFvbpT7v8fG65+HjAz1WmQWkYsPcrLODzHbW7S0pxbxglEgYF8CRbscItbZ8xAzZv5qDDDmPFj3/Bjzsczz/z3CEJjbtQfFxlGoTGPfNV1TV7/Uy2ftKQ3KUAnoLXH723bXOn2pSXw4svuj7tESNcJ8lpp3FsXh7PNvPr2drjnM5mX5AdnzRE6rUYwI0x04ChwEZr7VfqLjsAmA30Af4FjLTWbg5vmNHxba/m3V5/3WXbDzzgTnHv188tbf/Od9z+JGnK1h7ndANz0j9piDSUTgZ+H3AnMKPBZROAZ621k40xE+q+vy744UXPqw3pP/vMndheXg7/+Ad06AAXXeSy7ZNPBmNadbWNJ0brt05NckBvabMvyI5PGiINtRjArbUvGGP6NLp4GHBa3df3A8+TJQHci0m+pUtd0H7oIdi6FY45Bv74R3d6+wEHBHpTzX3igORk6qlKQwV5hs4d21G1rcb78Yu0Rmtr4N2stevrvt4AdGvqB40x44BxAL17927lzYWvvu7d1BHPoX/0/vRTmDnTlUmWLHH7a19yicu2Tzqp1dl2S5r6xHFjxQq21+7yr5TUhGwtDYk0p82TmNZaa4xp8mh7a205UA7uVPq23l4YUp3K3VBoH72thVdfdUF75kz4/HP46lfhzjvh0kuhqCj422ykqU8WSezi0OpMyTWtDeAfGWO6W2vXG2O6AxuDHFTUmutgCGXb1qoqePBBF7hffx06dYLRo122fcIJoWXbqaRTO25IXRwi/mhtAK8AxgKT6/59LLARxaCpoGQguG1brYWXXnJBe/Zst/hmwAD4059c8O7SJZjbaUHjFsnTj+rKo0sq92or7FiQt3tHw4bUxSHij3TaCGfiJiwPMsasAybiAvdfjTFXAu8DI8McZNhCXdzyySeu9a+83C286dzZTUaOGwcDB7b9+jOQasLy0SWVXDiwmOdWbtqjdgxkZb+4SDZJpwtldBP/64yAxxKbwBe3WAsLF7qg/fDDsH07DBrksu9Ro1wQj0FTE5bPrdzU5CeNTCYFvV4AJZKFtBKTADsYPv7YLWufMgVWrnRlkSuvhKuvhv79gx94hjJtkcxkUtDbBVAiWUwBvE6rOxisheefd9n2nDmwY4dr+5s+HS6+GPbZJ/CxtlaYpSKvFkCJ5Ii8uAeQWBs3wu9/D0ceCUOGuPMkv/99eOMNt0fJ5Zd7FbzBlYoKC/L3uCyounZTWXxlVTWlkxcwd1llm29DRPakDDwTu3bBs8+6bPuxx9yBwCefDBMnwoUXQqHfHRphLnZprh1R5RSRcBhro1tbU1JSYhcvXhzZ7QVm/XpXErn3XncY8IEHwtixcNVVcPTRcY/OCy0thgLXU9+atkxfJkd9GYfkHmPMEmttSePLlYE3ZedOeOYZl20//rj7/vTT4be/hQsucBtLyW4Ns/umMvHWLALyZXLUl3GINKQaeGOVlXDzzXDooe7E9kWL4Gc/g3fegQULXBuggndKwwcUs2jCEIqbmBRtzWRpc5OjUfJlHCINKQMHqK2FJ5907X/z5rla91lnuf22hw2D9u3jHmGiZNJX31JZwovdIT0ah0hDuR3A166FqVPdf5WVcPDBMGGC690+9NC4R9csn+ux6U6WplOW8OUIOF/GIdJQ7gXwmhqXZZeXu9Y/gHPOgTvugKFD3RFlzfAhcKYT+OIeZzp99en0jvtyBJwv4xBpKHcC+HvvuS6SadNgwwYoLoYbbnDZ9iGHpHUVvkxktRT4bpj7Bg++vHb33ua+TrilU5bwZZ9vX8Yh0lB2B/AdO6CiwmXb8+dDXp6bmBw3Ds49F9pldvd9WW3YXOCbu6xyj+Bdz8dVkemWJXzZ59uXcYjUy84ulNWr4brroFcvt5x95Uq46SZ4/33XEnj++RkHb/BnIqupumuPosJmTxXybcItzJWhIrkgewL49u0wa5Zb1n7kkfCHP8DgwfDEE6588qtfQc+ebbqJ5gJnlJoLfM0F6f0Km6/vR234gGImjTiO4qJCDG6hz6QRxynLFUlTIksoDSfoTtyxiZs3vcThTz4K//439OkDv/kNfPe70L17oLfry0RWc/XY5hbSRHjQT9pUlhBpvcQF8LnLKrlx9mJOW/F3Rr/+NF//4E1q8vKpHHIOxeN/DGee6WrdIfBpIqupwDe+rB/Xzn4t5e9UpThhR0SSK1kBfMUKdvzwVzy/9BmKvviM9/bvzqTTLufRr5xBh+IeLDo7oOPPmuF7xjh8QDE3Pb5Cx6GJ5IBkBPCHHoK77oIXX2RYfjuePnIwM48v4+Xex2GNy7aNZxN0cZp4/rFelHpEJFzJCOAVFa6+/Yc/cMHmvrxVs/fSdmWXX/Kp1CMi4UlGAC8vh333BWMYl2LbUmWXe/O91CMibZeMAN6ly+4vlV1+Ke7l8iISL+8DeFNBKtcDlS/L+kUkPl4HcAWppoW1rF9ZvUhyeL0SU5voNy2MZf31b5iVVdVYvnzD1IHEIn7yOoD7sveIj8JY1q83TJFk8TqA+7L3iI/C2AhKb5giyeJ1ANdudU0LYyMovWGKJIvXk5hqGWxe0N04vmzWJSLp8TqAgxakRElvmCLJ4n0Al2jpDVMkObyugYuISNMUwEVEEkoBXEQkoRTARUQSSgFcRCShFMBFRBKqTQHcGHOOMWaVMeZdY8yEoAYlIiIta3UfuDEmH7gLOAtYB7xqjKmw1r4V1OBE0qVtcCUXtSUDHwS8a61dY63dAcwChgUzLJH0aRtcyVVtCeDFwAcNvl9Xd5lIpLQNruSq0CcxjTHjjDGLjTGLN23aFPbNSQ7SNriSq9oSwCuBXg2+71l32R6steXW2hJrbUnXrl3bcHMiqWkbXMlVbQngrwJHGGP6GmPaA6OAimCGJZI+7RsvuarVXSjW2lpjzA+Bp4F8YJq1dkVgIxNJk7bBlVxlrLWR3VhJSYldvHhxINeltjERyRXGmCXW2pLGlydyP/D6trH6zoP6tjFAQVxEckYil9KrbUxEJKEBXG1jIiIJDeBqGxMRSWgAV9uYiEhCJzHVNiYiktAADjo9XUQkkSUUERFRABcRSSwFcBGRhFIAFxFJKAVwEZGEinQzK2PMJuD9Zn7kIODjiIbjk1y935C79133O/e05b4fYq3d60CFSAN4S4wxi1PtuJXtcvV+Q+7ed93v3BPGfVcJRUQkoRTARUQSyrcAXh73AGKSq/cbcve+637nnsDvu1c1cBERSZ9vGbiIiKRJAVxEJKG8CODGmHOMMauMMe8aYybEPZ6oGGN6GWOeM8a8ZYxZYYy5Ju4xRckYk2+MWWaM+VvcY4mKMabIGPOIMWalMeZtY8xJcY8pKsaYn9T9nb9pjJlpjOkY95jCYIyZZozZaIx5s8FlBxhj5htjVtf9u38QtxV7ADfG5AN3AecCxwCjjTHHxDuqyNQCP7PWHgOcCPxnDt13gGuAt+MeRMRuB56y1h4FHE+O3H9jTDHwY6DEWvsVIB8YFe+oQnMfcE6jyyYAz1prjwCerfu+zWIP4MAg4F1r7Rpr7Q5gFjAs5jFFwlq73lq7tO7rrbgXc05scm6M6Ql8E7g37rFExRizH3AKMBXAWrvDWlsV66Ci1Q4oNMa0AzoBH8Y8nlBYa18APml08TDg/rqv7weGB3FbPgTwYuCDBt+vI0eCWEPGmD7AAOCVmIcSlT8CPwd2xTyOKPUFNgHT60pH9xpj9ol7UFGw1lYCtwJrgfXAFmvtM/GOKlLdrLXr677eAHQL4kp9COA5zxjTGXgUuNZa+2nc4wmbMWYosNFauyTusUSsHfA14B5r7QDgcwL6KO27uprvMNybWA9gH2PMZfGOKh7W9W4H0r/tQwCvBHo1+L5n3WU5wRhTgAveD1pr58Q9noiUAt8yxvwLVzIbYox5IN4hRWIdsM5aW/8p6xFcQM8FZwLvWWs3WWtrgDnA4JjHFKWPjDHdAer+3RjElfoQwF8FjjDG9DXGtMdNbFTEPKZIGGMMrh76trX2f+IeT1Sstb+w1va01vbBPd8LrLVZn41ZazcAHxhj+tVddAbwVoxDitJa4ERjTKe6v/szyJEJ3DoVwNi6r8cCjwVxpbEfamytrTXG/BB4GjczPc1auyLmYUWlFBgDvGGMea3usuuttU/ENyQJ2Y+AB+uSlTXAd2MeTySsta8YYx4BluK6r5aRpcvqjTEzgdOAg4wx64CJwGTgr8aYK3Fbao8M5La0lF5EJJl8KKGIiEgrKICLiCSUAriISEIpgIuIJJQCuIhIQimAi4gklAK4iEhC/X+9I5pX4O8IFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "from sklearn import linear_model\n",
    "import numpy as np\n",
    "\n",
    "# load data\n",
    "# generate data\n",
    "data_num = 100\n",
    "X = np.random.rand(data_num, 1)*10\n",
    "Y = X * 3 + 4 + 8*np.random.randn(data_num,1)\n",
    "\n",
    "print(\"X: \", X.shape)\n",
    "print(\"Y: \", Y.shape)\n",
    "\n",
    "# create regression model\n",
    "regr = linear_model.LinearRegression()\n",
    "regr.fit(X, Y)\n",
    "\n",
    "a, b = np.squeeze(regr.coef_), np.squeeze(regr.intercept_)\n",
    "\n",
    "print(\"a = %f, b = %f\" % (a, b))\n",
    "\n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n",
    "\n",
    "plt.scatter(X, Y)\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. 如何使用sklearn拟合多项式函数？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([800.,  90., -20.])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fitting polynomial functions\n",
    "\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "t = np.array([2, 4, 6, 8])\n",
    "\n",
    "pa = -20\n",
    "pb = 90\n",
    "pc = 800\n",
    "\n",
    "y = pa*t**2 + pb*t + pc\n",
    "\n",
    "model = Pipeline([('poly', PolynomialFeatures(degree=2)),\n",
    "                  ('linear', LinearRegression(fit_intercept=False))])\n",
    "model = model.fit(t[:, np.newaxis], y)\n",
    "model.named_steps['linear'].coef_\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 参考资料\n",
    "* [梯度下降法](https://blog.csdn.net/u010402786/article/details/51188876)\n",
    "* [如何理解最小二乘法？](https://blog.csdn.net/ccnt_2012/article/details/81127117)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}