machinelearning_notebook/4_logistic_regression/1-Least_squares.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 最小二乘（Generalized Least Squares）\n",
    "\n",
    "## 1. 最小二乘的基本原理\n",
    "\n",
    "最小二乘法（generalized least squares）是一种数学优化技术，它通过最小化误差的平方和找到一组数据的最佳函数匹配。 最小二乘法通常用于曲线拟合、求解模型。很多其他的优化问题也可通过最小化能量或最大化熵用最小二乘形式表达。\n",
    "\n",
    "最小二乘原理的一般形式为：\n",
    "$$\n",
    "L = \\sum (V_{obv} - V_{target}(\\theta))^2\n",
    "$$\n",
    "其中$V_{obv}$是我们观测的多组样本值，$V_{target}$是我们假设拟合函数的输出值，$\\theta$为构造模型的参数。$L$是目标函数，如果通过调整模型参数$\\theta$，使得$L$下降到最小则表明，拟合函数与观测最为接近，也就是找到了最优的模型。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 示例\n",
    "\n",
    "假设我们有下面的一些观测数据，我们希望找到他们内在的规律。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZ00lEQVR4nO3db7BdVXnH8d9jSPWi1gTJMHCDTaZaKMqY6B1Gm6nVUAuOKBlU1LZM2mHkTW0V20jQF/CiHWLpiL6wzmQEG6eOoEBDRh2RERynvHC8ISgiMlLkTy5griPxHxlI4OmLsy/cnJx/e5+999prre9nJpN7d27OWWefc5+19rOetba5uwAA+XhR6AYAANpF4AeAzBD4ASAzBH4AyAyBHwAyc1zoBkzixBNP9HXr1oVuBgBEZe/evb909zX9x6MI/OvWrdP8/HzoZgBAVMzs4UHHSfUAQGYI/ACQGQI/AGSGwA8AmSHwA0BmoqjqAYAlu/ct6Opb79djBw/plFUz2nbOadqycTZ0s6JC4AcQjd37FnT5zffo0OFnJUkLBw/p8pvvkSSCfwmkegBE4+pb738+6C85dPhZXX3r/YFaFCcCP4BoPHbwUKnjGIzADyAap6yaKXUcgxH4AURj2zmnaWbliqOOzaxcoW3nnBaoRXFichdANJYmcKnqmQ6BH0BUtmycJdBPiVQPAGSGwA8AmSHwA0BmCPwAkBkCPwBkhqoeAJhSbBvHEfgBYAoxbhxHqgcAphDjxnEEfgCYQowbxxH4AWAKMW4cR+AHgCnEuHEck7sAMIUYN44j8APAlGLbOI7ADwAlxVa334/ADwAlxFi334/JXQAoIca6/X6Nj/jNbIWkeUkL7n6ema2XdL2kV0raK+kid3+m6XYAodSdFog9zdBFZc5pjHX7/doY8X9E0n3Lvv+UpGvc/dWSnpR0cQttAIJYSgssHDwk1wtpgd37FjrxeCh/TmOs2+/XaOA3s7WS3inpC8X3JmmzpBuLH9klaUuTbQBCqjstkEKaoWvKntMY6/b7NZ3q+Yykj0t6efH9KyUddPcjxff7JQ28njKzSyRdIkmvetWrmm0l0JC60wIppBmqaDK9Vfacxli336+xwG9m50k64O57zeytZf+/u++UtFOS5ubmvN7WAe04ZdWMFgYEkKppgbofLwZNV9FUOaex1e33azLVs0nSu83sIfUmczdL+qykVWa21OGslURyEsmqOy2QQpqhrKbTWzme08YCv7tf7u5r3X2dpA9Iut3d/0bSHZLeW/zYVkm3NNUGILQtG2d11QVnanbVjEzS7KoZXXXBmZVHi3U/XgyaTm/leE5DLOC6TNL1ZvavkvZJujZAG4DW1J0WiD3NUFYb6a3czmkrC7jc/bvufl7x9YPufpa7v9rd3+fuT7fRBgBxyjEV0zS2bADQaSlU0XQNgR9A5+WWimkae/UAQGYI/ACQGQI/AGSGHD+AzmEH0mYR+AF0Sgo3Ouk6Uj0AOoUdSJtH4AfQKbnuQNomAj+ATknhRiddR+AH0Cls0dA8JncBdApbNDSPwA+gc9iioVmkegAgMwR+AMgMgR8AMkOOH8lj+T+6ru3PKIEfSWP5P7ouxGeUVA+SxvJ/jLJ734I27bhd67d/Q5t23K7d+xZab0OIzygjfiSN5f8YpitXgyE+o4z4kTSW/2OYrlwNhviMEviRNJb/Y5imRtpl00chPqOkepA0lv9jmFNWzWhhQJCfZqRdJX0U4jNq7t7Yg9dlbm7O5+fnQzcDQEL6g7TUG2lfdcGZlYPuph23D+xMZlfN6M7tmyu3tSoz2+vuc/3HGfEDyFITI+1YigkI/AA6qY1FTXVvBldX+qjp107gRzJYoRtWnee/K6WWZW0757SB6aMyE7VtvHaqepCEpV+WhYOH5HrhlyXEgpwc1X3+u1JqWdaWjbO66oIzNbtqRqZebr/snEEbr72xEb+ZvUTS9yS9uHieG939CjNbL+l6Sa+UtFfSRe7+TFPtQB5G/bJ0eYSYirrPf5Olll1PH7UxT9DkiP9pSZvd/fWSNkg618zeJOlTkq5x91dLelLSxQ22AZmIZVItVXWf/yYWNcVyVdjGgq7GAr/3/K74dmXxxyVtlnRjcXyXpC1NtQH5YIVuWHWf/yYWNcWSPmpjQVejOX4zW2Fmd0s6IOk2Sf8n6aC7Hyl+ZL+kgddEZnaJmc2b2fzi4mKTzUQCWKEbVt3nv45ceb9YrgqbeO39Gq3qcfdnJW0ws1WS/kfS6SX+705JO6XeAq5GGohksEI3rCbOf1dLLdvQ9D2HWynndPeDZnaHpDdLWmVmxxWj/rWSupVgQ7S4QXdYXT//g0otTdLbTl/TyPN1uby4sVSPma0pRvoysxlJb5d0n6Q7JL23+LGtkm5pqg0AsGTLxlm9542zsmXHXNJNexdqn+AtM5Ec4p4ATeb4T5Z0h5n9SNIPJN3m7l+XdJmkj5nZA+qVdF7bYBsA4Hl3/HRR/XnjJiZ4J51IDlVp1Fiqx91/JGnjgOMPSjqrqedF2rp8+bxcLO3MTVsTvJM+T6j1J6zcRTRiqcOOpZ05aqvsd9LnCVVpROBHNGKpw46lnTlqq+z3baevOWouYdjzhFp/QuBHNGKpw46lnTlqo0Z+974F3bR34ai5BJP0njceW/UUav0Ju3MiGrHUYcfSztjUNW/SdNnpoCs+V29ieVBblv5Pm/NBBH5Eo44tb9sQSztjEtM2zYM6/aXj67d/45jgHmL9A4Ef0Yhlde64dlLxU14su6/u3rcgk44pGV2yfLJfCtdpEfgRla6vDl0yrJ0xjVy7JJZ5k6tvvX9o0F8udKdF4AeWaXo0HsvIdVJtXb3EMm9SpiMK2WlR1QMU2qi/j2XkOok21yuEqn4pu51CmY4oZKdF4AcKbdTfT1q3HWL/lrLaXK/QRhlmvyod26AOauUK08oXHV3VH3qyn1QPUGhjND5JxU8s8wBtX720Pb9TJS03bGJ/0LGQ7yWBHyi0kUeepDIplnmAWPLuVVXt2IZ1UF167wj8iaBEcHpt1d+PG7nGMg+Q+nqFlDs2An8CYkkN1K3uzq4r6wRiCThdOV+jTPMZSbljM/fu39Vwbm7O5+fnQzejszbtuH1goJhdNaM7t28O0KLm9Xd2Uu+XsukJvzak/NqqqBq8R51HabIOK/YraTPb6+5z/ccZ8ScgltRAnWLJg1cRw0h6uSaD4zRXs8M+I1fuuVdPH3luoseMZcFgWQT+BMSSGqhqUGBJpbMbFjRjCTiDAvOlN9ytj95wt2Zr6ASm6eCHfRYOHjp8zLFUBg2Too4/AaEWt7RhWC31K2ZWDvz5mDq7FG7YMmwnSumFTmDdFGsRpungy34WYhs0TCO7wB/DwpiyQixuacuwEZ+Zou/sUrhhy7hgubwTqNKpTXOjkmEDotXHxz9omFZWqZ6Uq19iSQ2UNfRy/anDuub9GxrPgzeZv550NDtpG0JMRA5LMw5SJZ0yTWXNqMVUqVbrTCqrwJ/ahGDsFQeTGDV/MWoHzDrOS9MDhUnmZiZtQ6hBzaDAPErZdMq0E92jBkSp/+6MklXgT2VCUEr76mW5siO+Os9L2YFC2Q5nktc2aRtCDWqWB+aFg4dG7kUvVUunNHE1m+oV8qSyCvwpVb+kdvUyTNkRX53npcxAoUqHM8lrG9aGhYOHtGnH7c//fN2DmjKd2PIguvT/BnUCdaVTcrjSbVpWgT+llXgpXb2MU2Z0Vud5KTNQqNrhjHtto3LoyzuXOgc101w1DeoE6gzQuVzpNi2rqp6Uql+mqXZIWZ3npUyZbFMd8aA2LLfUudRZ0ltXtdGWjbO6c/tm/XzHO3Xn9s21/J6lUAnVBVmN+KV0cnspXb3Uqc7zUibN1FQasT+HPshjBw/Vutq3y1eTXW5bTLIL/KmIbVl/W+o+L5MOFJrsiJfaMGxPpqXOpa5BTZfnwrrctpgQ+COWytVL3UKclzY64jau8nbvW9Dvnz5yzPGuXE1ypVuPxgK/mZ0q6UuSTlJvcn+nu3/WzE6QdIOkdZIeknShuz/ZVDuAaVStbmlC053LoN0sJWn18St1xbte24lBBle69WhsW2YzO1nSye5+l5m9XNJeSVsk/Z2kX7n7DjPbLmm1u1826rHYljkfXSrVy2175By3907dsG2Zh1b1mNk3zWxd1Sd098fd/a7i699Kuk/SrKTzJe0qfmyXep0BoN37FrTtaz88atOybV/7YbD9lHKrIGHiNB+jyjm/KOnbZvZJMxu8q9GEig5ko6TvSzrJ3R8v/ukJ9VJBgK7cc68OP3f0Fejh51xX7rm39GPVsRlfboGQEuF8DA387v41SW+Q9IeS5s3sX8zsY0t/Jn0CM3uZpJskfdTdf9P3HK4hK7zN7BIzmzez+cXFxUmfDhEbtE/6qOPD1LXdcW6BMOXtvXG0cQu4npH0e0kvlvTyvj9jFVcKN0n6srvfXBz+RZH/X5oHODDo/7r7Tnefc/e5NWvWTPJ0gKT6UjS5BcKUFjhitKFVPWZ2rqRPS9oj6Q3u/lSZBzYzk3StpPvc/dPL/mmPpK2SdhR/31K20Rhv1CRplyZQl1t9/Eo9+dSxo/th+6cPU1eKZumcXLnn3uevOl6yMu3F7pQI52FUOecnJb3P3csnWHs2SbpI0j1mdndx7BPqBfyvmtnFkh6WdGHFx8cQo/YzkdTZvU6ueNdrte3GH+rwsy9k/1auMF3xrteWepy6F/k8feS5579+8qnDrZ2vrnbQiN/QwO/ufz7NA7v7/0qyIf989jSPjdHGpTq6uqtnXTXadS7yCbULKpuRoUms3E1QlVRHVypV6kg1pLBvTS7bbiMMAn+CxqU6ctjrJPZ9a3IrJUW70p6pytSoapTcKlWmFep85VZKinYx4k/QJKmOJicNU5qU7D+Xr5hZKTPp0hvufn4f/CZeG5uRlZPSZ64Nje3VUyf26olHyvvbtP3aCGaTSfkzN61he/UQ+FGrlDf6Svm1xYz3ZbjSm7QBVaQ8KZnya4sZ70t5BH7UKuVJyZRfW8x4X8oj8GOksrtcplw1lPJrixnvS3lU9SSiiYnAKqtHU75DUmqvLZXJ49TelzYwuZuApqoamDQ7WiqBUqISJhdM7iaszDbEZVI3TJq9oK49/rsit7uL4WikehIwaYAum7oJtV1BF8W8d86gKxU69bwx4i+pjlv61W3Sqoayo7xUJs1yvg3jsCuVVUPucZBjp54jAn8JXb3cnzRAlw1eKdyRKffbMA7r7N2VRKeOarJO9ZSdrOvq5f6kVQ1VUjex35Gprvcs1r1zhnXqvz50WNe8f0Myk9UoJ9vAX6VUscuX+5ME6FiD1zTqvg1jbIFyVGcfe6eO6rIN/FVGgrFPdsYavKZR53sWY6DMsbPHeNkG/iojwS7/Ek2atmoyeHWxzr3L71kbcuzsMV62gb9qvlvq3i9RF+7P2oU2DNLV96xNMV6poFnZrtxNaeViF1bYdqENberi1Q3Qb9jK3WxH/CmNBLsw6VylDbEGz7qvbmI9D4hXtoFfSucSuAuTzmXb0NXU0CTqLOuN+TwgXkkv4KprlW0XV+su14UVtmXbEPNeMXVeYcV8HhCvZEf8k46kxl1mxzAi60LaqmwbupCeqqrOK6yYzwPilWzgn+RyfJKg3tXVuv26kLYq04YupKeqqrNENObzgHglm+qZZCQ1yWU2I7JmdCE9VVWdexjFfB4Qr2RH/JOMpCYJ6ozImtGF9NQ06rrCiv08IE7JBv5JLscnCeq5r/xsUhfSU13AeUDbGkv1mNl1ZnbAzH687NgJZnabmf2s+Ht1U88/6nJ8qUpn4eAhWd//6w/qKWxNDADLNbZy18zeIul3kr7k7q8rjv27pF+5+w4z2y5ptbtfNu6x6ly5O2jFrkly9YI6l9kAUtH6yl13/56Zres7fL6ktxZf75L0XUljA3+dBk3oLgX9FLcWAIB+bVf1nOTujxdfPyHppGE/aGaXmNm8mc0vLi7W1gCqdADkLtjkrru7mQ3NM7n7Tkk7pV6qp+zjD1uYRZVOOtjjBqim7RH/L8zsZEkq/j7QxJOMus8qddNp6Or9j4EYtB3490jaWny9VdItTTzJuNW2VOkcq+v7EfVjjxugusZSPWb2FfUmck80s/2SrpC0Q9JXzexiSQ9LurCJ5x6Xx6du+mgx7EfUj7kaoLomq3o+OOSfzm7qOZeQxy+nrv2I2sy58x4D1SW5Vw95/HLqGD23nXPnPQaqSzLwk8cvZ9gouczoue2cO+8xUF2ye/XUmcdPvWywjv2IQuTcmasBqkk28NclxonPsurYIbIrOffUO2mgDgT+MWK5Ecu0ph09d2EX0xw6aaAOSeb460TZ4GS6kHOnth+YDCP+MbqSwqhD02mQ0Dl3OmlgMoz4x0ilbDCHLQ7qqE4CckDgH6MLKYw65JAGSaWTBppGqmcCoVMYdcghDcL9a4HJEPhLiLlUMKW5ilFS6KSBppHqmVDsOXLSIACWMOIfY2mUP2i0HFM9P2kQAEsI/CMMujF7v5hy5KRBAEgE/pEGVcL0G5cjj3leAECaCPwjjBvNj8uRs4UAgC5icneEUaP5Ser5c6idBxAfAv8IwyphPvP+Dbpz++axo/YcaucBxIfAP8K0q3bZQgBAFyWb469rUnVUJcy45+jCVsUA0C/JwN/GpOq451jqFA4dflYrzPSsu2ap6gHQAUkG/jZunjJu4nZ5p/Cs+/MjfYI+gNCSzPG3Mak66jmo5gHQZUkG/jYmVUc9B9U8ALosycDfxoZko56Dah4AXZZkjr+NDcnGPQfVPAC6ytw9dBvGmpub8/n5+aBtKFseyh49AEIzs73uPtd/PMkRf92qlId2bSdMOiIAS4Lk+M3sXDO738weMLPtIdpQRuxVOrHfRAZAvVoP/Ga2QtLnJL1D0hmSPmhmZ7TdjjJir9KJveMCUK8QI/6zJD3g7g+6+zOSrpd0foB2TCz2Kp3YOy4A9QoR+GclPbrs+/3FsaOY2SVmNm9m84uLi601bpDY71cbe8cFoF6dreN3953uPufuc2vWrAnalml36Qwt9o4LQL1CVPUsSDp12fdri2Od1rUqnTK40TqA5UIE/h9Ieo2ZrVcv4H9A0l8HaEeyhpVuEugBSAECv7sfMbMPS7pV0gpJ17n7vW23I1Xc5xfAOEFy/O7+TXf/E3f/Y3f/txBtSBWlmwDG6ezkLqqhdBPAOAT+xFC6CWAcAn9iKN0EMA6btCWG0k0A4xD4E0TpJoBRSPUAQGYI/ACQmaxSPdyMBAAyCvxVV7TSWQBITTapniorWrlzFYAUZRP4q6xoZfsDACnKJvBXWdHK9gcAUpRN4K+yopXtDwCkKJvAX+UuWmx/ACBF2VT1SOVXtLL9AYAUZRX4q2D7AwCpIfBjaqx1AOJC4MdUuNUjEJ9sJnfRDNY6APEh8GMqrHUA4kPgx1RY6wDEh8CPqbDWAYgPk7uYCmsdgPgQ+DE11joAcSHVAwCZIfADQGYI/ACQGQI/AGSGwA8AmTF3D92GscxsUdLDJf7LiZJ+2VBzuirH1yzxunOS42uWpnvdf+Tua/oPRhH4yzKzeXefC92ONuX4miVed+h2tCnH1yw187pJ9QBAZgj8AJCZVAP/ztANCCDH1yzxunOS42uWGnjdSeb4AQDDpTriBwAMQeAHgMwkFfjN7Fwzu9/MHjCz7aHb0wYzO9XM7jCzn5jZvWb2kdBtaouZrTCzfWb29dBtaYuZrTKzG83sp2Z2n5m9OXSb2mBmlxaf7x+b2VfM7CWh29QEM7vOzA6Y2Y+XHTvBzG4zs58Vf6+e9nmSCfxmtkLS5yS9Q9IZkj5oZmeEbVUrjkj6Z3c/Q9KbJP1DJq9bkj4i6b7QjWjZZyV9y91Pl/R6ZfD6zWxW0j9JmnP310laIekDYVvVmP+SdG7fse2SvuPur5H0neL7qSQT+CWdJekBd3/Q3Z+RdL2k8wO3qXHu/ri731V8/Vv1AkHym+Ob2VpJ75T0hdBtaYuZvULSWyRdK0nu/oy7HwzaqPYcJ2nGzI6TdLykxwK3pxHu/j1Jv+o7fL6kXcXXuyRtmfZ5Ugr8s5IeXfb9fmUQAJczs3WSNkr6fuCmtOEzkj4u6bnA7WjTekmLkr5YpLi+YGYvDd2oprn7gqT/kPSIpMcl/drdvx22Va06yd0fL75+QtJJ0z5gSoE/a2b2Mkk3Sfqou/8mdHuaZGbnSTrg7ntDt6Vlx0l6g6TPu/tGSb9XDZf9XVfktM9Xr+M7RdJLzexvw7YqDO/V309dg59S4F+QdOqy79cWx5JnZivVC/pfdvebQ7enBZskvdvMHlIvpbfZzP47bJNasV/SfndfuqK7Ub2OIHV/Kenn7r7o7ocl3SzpzwK3qU2/MLOTJan4+8C0D5hS4P+BpNeY2Xoz+wP1Jn/2BG5T48zM1Mv53ufunw7dnja4++Xuvtbd16n3Pt/u7smPAN39CUmPmtlpxaGzJf0kYJPa8oikN5nZ8cXn/WxlMKm9zB5JW4uvt0q6ZdoHTOZm6+5+xMw+LOlW9Wb9r3P3ewM3qw2bJF0k6R4zu7s49gl3/2a4JqFB/yjpy8Xg5kFJfx+4PY1z9++b2Y2S7lKvim2fEt2+wcy+Iumtkk40s/2SrpC0Q9JXzexi9banv3Dq52HLBgDIS0qpHgDABAj8AJAZAj8AZIbADwCZIfADQGYI/EBJxY6oPzezE4rvVxffrwvcNGAiBH6gJHd/VNLn1auvVvH3Tnd/KFijgBKo4wcqKLbJ2CvpOkkfkrSh2E4A6LxkVu4CbXL3w2a2TdK3JP0VQR8xIdUDVPcO9bYJfl3ohgBlEPiBCsxsg6S3q3fXs0uXdk8EYkDgB0oqdoj8vHr3PnhE0tXq3SgEiAKBHyjvQ5Iecffbiu//U9KfmtlfBGwTMDGqegAgM4z4ASAzBH4AyAyBHwAyQ+AHgMwQ+AEgMwR+AMgMgR8AMvP/xr22No2bkysAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# generate data\n",
    "data_num = 100\n",
    "X = np.random.rand(data_num, 1)*10\n",
    "Y = X * 3 + 4 + 8*np.random.randn(data_num,1)\n",
    "\n",
    "# draw original data\n",
    "plt.scatter(X, Y)\n",
    "plt.xlabel(\"X\")\n",
    "plt.ylabel(\"Y\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 数学原理\n",
    "有$N$个观测数据为：\n",
    "$$\n",
    "\\mathbf{X} = \\{x_1, x_2, ..., x_N \\} \\\\\n",
    "\\mathbf{Y} = \\{y_1, y_2, ..., y_N \\}\n",
    "$$\n",
    "其中$\\mathbf{X}$为自变量，$\\mathbf{Y}$为因变量。\n",
    "\n",
    "我们希望找到一个模型能够解释这些数据，假设我们使用最简单的线性模型来拟合数据：\n",
    "$$\n",
    "y = ax + b\n",
    "$$\n",
    "那么问题就变成求解参数$a$, $b$能够使得模型输出尽可能和观测数据有比较小的误差。\n",
    "\n",
    "如何构建函数来评估模型输出与观测数据之间的误差是一个关键问题，这里我们使用观测数据与模型输出的平方和来作为评估函数（也被称为损失函数Loss function):\n",
    "$$\n",
    "L = \\sum_{i=1}^{N} \\{y_i - (a x_i + b)\\}^2 \\\\\n",
    "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2 \n",
    "$$\n",
    "\n",
    "使误差函数最小，那么我们就可以求出模型的参数:\n",
    "$$\n",
    "\\frac{\\partial L}{\\partial a} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i \\\\\n",
    "\\frac{\\partial L}{\\partial b} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b)\n",
    "$$\n",
    "既当偏微分为0时，误差函数为最小，因此我们可以得到:\n",
    "$$\n",
    "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i = 0 \\\\\n",
    "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) = 0 \\\\\n",
    "$$\n",
    "\n",
    "将上式调整一下顺序可以得到：\n",
    "$$\n",
    "a \\sum x_i^2 + b \\sum x_i = \\sum y_i x_i \\\\\n",
    "a \\sum x_i + b N = \\sum y_i\n",
    "$$\n",
    "通过求解二元一次方程组，我们即可求出模型的最优参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 求解程序"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a = 2.763945, b = 6.154651\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmPUlEQVR4nO3deZRU1bn38e9D0xFQIiBooBlNlJZJGpm00QjkOuGAJG00MUDURa7GOCQvityrmMFIolGSdY0JywkTjNoMLYqGyBSEZVQQRBBRoxBoFRBpEGm0gf3+UVVNdVFznWmfej5ruaSL6qpdp5vf2WfvZ+8jxhiUUkrZp5nfDVBKKZUfDXCllLKUBrhSSllKA1wppSylAa6UUpZq7uWbtW/f3nTv3t3Lt1RKKeutWrXqE2NMh8THPQ3w7t27s3LlSi/fUimlrCcim5M9rkMoSillKQ1wpZSylAa4UkpZytMx8GQaGhrYunUr+/fv97spRa1FixZ07tyZ0tJSv5uilMqS7wG+detWWrduTffu3RERv5tTlIwx7Ny5k61bt9KjRw+/m6OUypLvAb5//34Nb5+JCMcddxw7duzwuymqiNSsruWeBRv5sK6eTm1aMvHcnoyuKPO7WVbxPcABDe8A0J+B8lLN6lpum/Mm9Q0HAaitq+e2OW8CaIjnQCcxlVKeu2fBxsbwjqlvOMg9Czb61CI7aYA7rHv37nzyyScFP0epMPuwrj6nx1VyGuBKKc91atMyp8dVchrgwKZNmygvL2f8+PGcfPLJfP/732fhwoVUVlZy0kkn8eqrr/Lpp58yevRo+vXrx9ChQ1m7di0AO3fu5JxzzqF3795cc801xN/h6K9//SuDBw+mf//+/OhHP+LgwYOpmqBUUZl4bk9alpY0eaxlaQkTz+3pU4vsFIhJzEY33QRr1jj7mv37w7RpGZ/23nvvUV1dzSOPPMKgQYN44oknWL58OfPmzePXv/41Xbp0oaKigpqaGhYvXszYsWNZs2YNP//5zxk2bBh33HEH8+fP5+GHHwZgw4YNPPXUU6xYsYLS0lKuu+46Zs6cydixY539fEpZKDZRqVUohQlWgPuoR48e9O3bF4DevXszcuRIRIS+ffuyadMmNm/ezOzZswEYMWIEO3fuZM+ePSxbtow5c+YAMGrUKNq2bQvAokWLWLVqFYMGDQKgvr6e448/3odPplQwja4o08AuULACPIuesluOOuqoxj83a9as8etmzZpx4MCBnFcoGmMYN24cd999t6PtVEqpGB0Dz9KZZ57JzJkzAVi6dCnt27fnq1/9KmeddRZPPPEEAC+88AK7du0CYOTIkcyaNYvt27cD8Omnn7J5c9IdIZVSKi/B6oEH2J133slVV11Fv379aNWqFTNmzABgypQpXHHFFfTu3ZszzjiDrl27AtCrVy9+9atfcc4553Do0CFKS0t54IEH6Natm58fQykVIhJfNeG2gQMHmsQbOmzYsIFTTjnFszao1PRnoVQwicgqY8zAxMd1CEUppSylQyhKKRVl2wZbGuBKKYWdG2zpEIpSSmHnBlsa4EophZ0bbGmAK6UUdm6wpQGegwsuuIC6urq0z7njjjtYuHBhXq+/dOlSLrzwwozPO/vss0ksx0w0bdo09u3bl1c7lCpGNm6wpZOYWTDGYIzh+eefz/jcX/ziFx60KLNp06Zx5ZVX0qpVK7+bopQVbNxgy7oeeM3qWiqnLqbHpPlUTl1Mzeragl/zvvvuo0+fPvTp04dp0f1YNm3aRM+ePRk7dix9+vRhy5YtTW7E8Mtf/pKePXsybNgwrrjiCu69914Axo8fz6xZs4DIjRumTJnCgAED6Nu3L2+//TYAr776KqeffjoVFRWcccYZbNyYfpKkvr6eyy+/nFNOOYVLL72U+vrDY3LXXnstAwcOpHfv3kyZMgWAP/zhD3z44YcMHz6c4cOHp3yeUqqp0RVlrJg0gg+mjmLFpBGBDm+wrAfuRpnPqlWrePTRR3nllVcwxjBkyBC++c1v0rZtW959911mzJjB0KFDm3zPa6+9xuzZs3njjTdoaGhgwIABnHbaaUlfv3379rz++uv88Y9/5N577+Whhx6ivLycl156iebNm7Nw4UImT57cuNNhMg8++CCtWrViw4YNrF27lgEDBjT+3V133UW7du04ePAgI0eOZO3atdxwww3cd999LFmyhPbt26d8Xr9+/fI6ZkqFhW1134ms6oG7UeazfPlyLr30Uo4++miOOeYYxowZw0svvQRAt27djghvgBUrVnDJJZfQokULWrduzUUXXZTy9ceMGQPAaaedxqZNmwDYvXs3VVVV9OnTh5tvvpn169enbeOyZcu48sorAejXr1+T4H366acZMGAAFRUVrF+/nrfeeivpa2T7PKWKRaxDWFtXj+Fwh9CJq3qvWBXgXpf5HH300QW/Rmxb2pKSEg4cOADA7bffzvDhw1m3bh3PPvss+/fvz+u1P/jgA+69914WLVrE2rVrGTVqVNLXyvZ5ShUTG+u+E2Ud4CJSIiKrReS56Nc9ROQVEXlPRJ4Ska+418wIN8p8zjzzTGpqati3bx+ff/45c+fO5cwzz0z7PZWVlY3Bu3fvXp577rmc3nP37t2UlUUu0x577LGMz4/fsnbdunWNt3Pbs2cPRx99NMceeyzbtm3jhRdeaPye1q1b89lnn2V8nsqe0/MvbsznFLtcjqmNdd+JcumB3whsiPv6N8D9xphvALuAq51sWDJulPkMGDCA8ePHM3jwYIYMGcI111xDRUVF2u8ZNGgQF198Mf369eP888+nb9++HHvssVm/5y233MJtt91GRUVFY688nWuvvZa9e/dyyimncMcddzSOt5966qlUVFRQXl7O9773PSorKxu/Z8KECZx33nkMHz487fNUdpy+3A7D5XvQ5HpMbaz7TpTVdrIi0hmYAdwF/BS4CNgBfM0Yc0BETgfuNMacm+51nNhONiiTDnv37uWYY45h3759nHXWWUyfPr3J5KKNdDvZ1CqnLqY2Sc+srE1LVkwa4fvrqdyPaWJRBEQ6hHeP6Ru4icxU28lmW4UyDbgFaB39+jigzhgT6z5uBZJ+YhGZAEwAGm92UIig3EdvwoQJvPXWW+zfv59x48ZZH94qPacvt8Nw+Z4PNztguR5TG+u+E2UMcBG5ENhujFklImfn+gbGmOnAdIj0wHP9/qCKjUmr4tCpTcukvbt8L7edfj0buL3bXz7HNCgdwnxlMwZeCVwsIpuAJ4ERwO+BNiISOwF0BvIevPPyrkAqOf0ZpOf0/IuNy7YL5XbVRzEe04wBboy5zRjT2RjTHbgcWGyM+T6wBPhO9GnjgGfyaUCLFi3YuXOnBoiPjDHs3LmTFi1a+N2UwBpdUcbdY/pS1qYlQmRctZCxUqdfzwZuDxsV4zHN6Z6Y0SGU/2eMuVBETiTSI28HrAauNMZ8ke77k01iNjQ0sHXrVq1L9lmLFi3o3LkzpaWlfjdFhZRO3Oav0ElMAIwxS4Gl0T+/DwwutGGlpaX06NGj0JdRSgXcxHN7Jq36CPMQh9us2gtFKWWvMFR9BI0GuFLKM7ZXfQSNVXuhKKWUOkwDXCmlLKUBrpRSltIxcKWUa4Kyd1FYaYArpVzh9tJ5pUMoSimXhOGGCUGnAa6UckWx7rjoJQ1wpZQrwnDDhKDTAFdKuaIYdwf0mk5iKqVcoUvn3acBrpRyjS6dd5cOoSillKU0wJVSylIa4EopZSkdA1fW0GXZKuiS/o52LIFnn4Wrr4ZmzvaZNcCVFXRZtgq6+N/RDnt3MeL15+j04ArMlnWIMdC/Pwwa5Oh7aoArK6Rblq0BroJwdfbwrJf5zmuLGPX2cgZvWU8zDO8c15WHR4zlmj/cAr16Of6eGuDKCrosW6Xi69XZxx/D7NlQXc0z/1zWGNq/r7yC+eXDeK99VwS4xoXwBg1wZYlObVomvaO5LstWnl+dxYU2y5aBMdCrF4+O/AFPdhvCux26NXm6m7+jGuDKCnpHc5WKW1dn8cMyfUrq+dWhdzj15X80CW3uuAOqqqB3b45bXcvWOW+Ch7+jGuDKCrosW6XixtVZzepafvf4Pxm+fjkXvv1S45j2nhNP5qtxoR3Pj99RMca49uKJBg4caFauXOnZ+ymlwi9xDBwiPd+7x/TNPTw//hjmzOH1302n//traYbh3eO6ML98GPN7DmPfSeWsmDTC4U+QmYisMsYMTHxce+BKKasV3PONhjbV1fDPf4IxtD6uC3+ovJz5PYc1GdOWgE2aa4ArpVzlRYlfzptmxYf2smVw6BCcckrjmPb4Z7c5Mizj9mfXAFeBE4Sa3mLm5PEP1AKsbduaVo/EQvt//xcuu6zJmPbEL9sUPGnuxWfXAFeBEqh/8EXI6ePv+wKsbdsiPe2nn84Y2vGcmJD04rNnDHARaQEsA46KPn+WMWaKiPQAngSOA1YBPzDGfOlIq1TR8v0ffJFz+vh7UeJ3RLimC+1Y9YhIxvcodC9zLxafZdMD/wIYYYzZKyKlwHIReQH4KXC/MeZJEfkTcDXwoGMtU0VJV1z6y+nj71aJX+JVwr2PL6N79Sb6/+vFyETkoUNQXp5zaDvJi8VnGQPcROoM90a/LI3+Z4ARwPeij88A7kQDXBVIV1z6y+nj78YCrNhVQvvPd3HeOy9zwdvLGbJlHSXG/9CO58Xis6zGwEWkhMgwyTeAB4B/A3XGmAPRp2wFkl5riMgEYAJA165dC22vCjldcekvp4+/44tbtm1j+JLZTUL7vXad+b/Tv8vz5ZUseOg6X0M7nhcLe3JayCMibYC5wO3AY8aYb0Qf7wK8YIzpk+77dSGPyoZWofgrcMc/NqYdq9M+FAnt+eXDmF8+jHfadwMRytq09GWRjRccWchjjKkTkSXA6UAbEWke7YV3BmqdaaoqdnojXH8F4vhv3354IjJ+TPt//odFfb/J9W98Sf2BQ41PF2B4eQdXmhK4E1qcbKpQOgAN0fBuCfwX8BtgCfAdIpUo44Bn3GyoUirk0oQ2VVXQpw+IMBL4dumbzPzXf4iNHxhg9qpaBnZr52i45lJW6UfQZ9MD7wjMiI6DNwOeNsY8JyJvAU+KyK+A1cDDLrZTKRVGsdCuroalSyOh3bPnEaGdaMnbO0gc/HWj3DTbskq/1i9kU4WyFqhI8vj7wGA3GqXCL8iXpfFsaadV8gzteF6Vm2b7Pn6tX9CVmMpztqy2tKWdVnAgtON5VW6a7fv4tX7B2VskK5WFdL2VILGlnYG1fTv86U8wciR07AjXXgu1tTB5MqxdCxs2wC9+AX375lz6N/HcnrQsLWnymBvlpsPLO5DYsmTvk+rE4fb6Be2BK8/ZstrSlnYGyvbtMHduZCIyvqc9eXJk75Ece9qpeFFjXbO6ltmrapuMtQvw7dOOrNLxa/2CBrjynC2rLW1pp+927DhcPZIY2lVVR/SwnZpXcLvcMdkVmCEygZqsLbHvCVoVilKOsmW1pS3t9EUstKurYcmSSGiffHLK0I6xaV4h2ck79niPSfOPCGk/6uc1wJXnbLm/ZaZ2Fl2FSp6hHc+W3SZrVtcicESpYowhGCcfDXDli0Cs9stCqnba1JMsiAOhHc+WeYV7FmxMGd7x/D75aICrUHK7d2xLTzJb8cerV/MvuMu8S/9XXmwa2rfdFpmIzKNqJMaWeYVcTih+nnw0wFXoeNE7tqUnmY2a1bX89i8v8c23VjDq7ZcY+p83aW4O8Vm3E2ntQGjH82teIdcTeqoTTarn+kUDXIWOF73jbHuSgR4n37ED5s6l7Ld/Ztn7a2huDvHvdmU8OLSK58sr2XNSL1bcNtLRt/Rj/iOfE3qyE01piYCBhkOHB1f8ntTWAFeh40XvOJueZCDHyaOh3TimffAg7dt2agztDR16NPa0Zfd+V5rg9fxHPif0VCeaZI/5eULWAFeh48U4azY9ycCMkycJbU46CSZNgqoqrnz+E2qThHXQxqXzle8JPdWJJjBXUGiAB06gL7kt4dU4a6aepK/j5BlCm379GnvaEw/Vhrre3ZaJ03xogAdIIC+5PeD0SSsodeaeB0eq0L711shEZFxoxwvK8UqnkN+RMC/IyumWaoXSW6qlVzl1cdJ/8GG+VVTiSQsi/7juHtM3UAGSD08+2yefHN57JD60q6rShrYf8g3hdMcRsjvx2H5l68gt1ZS7wlSalq3AjBO7wLWebarQvvXWSHCfempeoe1myBVydZnqd+TOeev54sChrF7TloVjudIAD5Awj9VB8oAIy0krVfg5Fhyx0K6uhsWLI6H9jW8UHNrx7U8M2JufWsNNT62hzIEwL+REnep3oa6+4YjHwnLyz5YGeICEeawuVQ/s2JalSf8h2nTScm3uwuXQjpdq5z1wJswLOVHnsqgm29cMC2sD3PYxrWRsmEzKV6oeWIvSZrQsLbH6pOXoMJCHoR0vU+jFh3k+J6dCri5TdWxalDZj1z67T/6FsjLAw1ytEdaxupSXwfsauP+7/V0/abl5ws+2d5myDUlCe8txZcwbNIbXBn2L0eMvYPSAzo60NZVcern5nJwKubpMt6gmrFes2bIywMM28RXGq4lE6Xpg6Xb8c+K4uH3Cz6Z3mdiGfR9+zKrb53HGztUc/9qKxp72xrHXMqmknNXtujX2tF+Zuw5EXP2dSBaw6eQ6TFHo1WW6jk3Y/+2kY2WAh2XiC8J9NREv1x6Yk8cl1xN+rieObD7bPQs2ctSeXVz8zsuMens5Z2x+g+bmEFuOK4NbbokMj/Tvz1W/WXLEycCLzkl8wNbW1afdCxvyG6Zw4+oyrFes2bIywMNUrRG2q4lUcu2BOXlccjnh53PiSPvZdu6EuXO5+89/bAztD9p25M9Dvs3z5cNYf/yJlH21FRM5ntEijndOcjkZxYdh7PuShblTwxTFcOXpNisDPEzVGmG6msgkl96Sk8cllxN+vieOJp8tGtpMqoZFi+DgQU5s16lJaMdPRMafJJzsnBRyFZMszJ0M2mK58nSblQEepmqNMF1NOMnJ45LLCT/vE0cstKsPhzZf/3rj8MhK04H/m7su5Rhz7CThZOfEqasYN4YpiuXK021WBjiEZ+wrTFcTTnLyuORyws/pxJEqtCdOjCxj79+/sac9GkCkcVgimQ/r6h3tnAT56i7IbbOJtQEeFmG6mnCS08cl2xN+xhPHzp1QUxNZxp4Y2lVVUFGRsk471oZUe97EThJOdU6CfHUX5LbZRAM8AMJyNeE0P45LshPH5CHHM+r1v8Ok3EM7GS+uumpW1/L5FweOeDwoV3d65emMjAEuIl2Ax4ETiExGTzfG/F5E2gFPAd2BTcBlxphd7jVVqfzlXI3RtUWkp11dDbcvggMH4MQT8w7txNcH9666ku3eB9C2VSlTLuodiM6CXnk6I+N2siLSEehojHldRFoDq4gM6Y0HPjXGTBWRSUBbY8yt6V5Lt5MtHkEqEct6W9fY8EhsTDsW2pddVnBoe6kYtyUOu7y3kzXGfAR8FP3zZyKyASgDLgHOjj5tBrAUSBvgqjjUrK5lYvUbjTd/ra2rZ2L1G4A/JWJpKx7ie9rxof2zn0WC25LQjqcThMUjpzFwEekOVACvACdEwx3gYyJDLEpx57z1Te7cDZE7ed85b33OAe5ETz4xuNrU7+Gcd/7FqI3L4fa1oQjteDpBWDyyDnAROQaYDdxkjNkjcb/gxhgjIknHYkRkAjABoGvXroW1Vlkh2faw6R5PxanFHp3atGTvR9s5592XuTC6jL300EFq23WMhHZVFQwYYHVox9MJwuKRVYCLSCmR8J5pjJkTfXibiHQ0xnwUHSffnux7jTHTgekQGQN3oM2qSBS82OPTT6GmhjnzHqfdK8spPXSQzW2+xkODLmVhn7P4wXWXUubyLn9+0AnC4pFNFYoADwMbjDH3xf3VPGAcMDX6/2dcaWGRSzeEEKSJwnhtW5Um3ae5bavSnF4nr7HcaGhTXQ0LF8KBA5zQowfvjP0RU77Si5fbRHb5a9uqNDQ97mS0NLU4ZNMDrwR+ALwpImuij00mEtxPi8jVwGbgMldaWMTSDSEAgd1LYspFvZk46w0aDh6+4CotEaZc1Dun18l6LDdJaNOjR5PhkbfWfMiaOW9C9Hjt2tfg2fEK6olW2U/vSh9g6crBgECXijkRWmnL/7q1TB7asbuxJ4xp+1Va58md6VXo6V3pLZTPEEJQSsWcuIRPHMvt+ZUGfi1vM2Dy75uG9k9/mjS04/lVWqebNik3aYAHWKYhhGIoFRvdrSWjT9gM/3z6yNCuqoLTTstqLNuv0jqtyVZuauZ3A1RqE8/tScvSkiaPxcrB0v2d9XbtgkcfhfPPhxNOgKuugo0bI6H92mvw73/Db34DAwdmPRHp1/FKdYII24lW+UN74AGWTTmYm5Njnk6+7dp1eEz7xRcjPe3u3XPuaaeSeCyPbVmKCNz81JrGfbjd+Gxak50bnfDNjU5iqqQ8mXxLFdqxvUcKDO1UvJ5Y1FDKjk74ppZqElMDXCXlWtVGfGgvXAgNDZHQjlWPuBTa8XSzp2DSn0tqWoWicuLo5Fuq0L7pJs9CO55OLAaT/lxypwGukiq4amPXLnjmmcidaxJDu6oqpwlIp+lmT8GkP5fcaRVKkahZXUvl1MX0mDSfyqmLqVldm/b5eVVt7NoFjz0GF1wQqR754Q9hw4ZIaL/6Krz/Pvz2tzBokK/L2ENdwWMx/bnkTnvgAePGhFc+u/plvSFSrKcdm4hsaIBu3QLR004lbJs9hWWSNGw/Fy/oJGaAuDUL7/jkUKrQjlWPBDC044Ul8EArN4qFTmJaIJdl17mEkCOTQ6lC+8YbI8Ed8NCOcWqP8aDQpfrFTQM8QLIN2lxDKO/JoZCEdjybAy/ZSVsrN4pb0QZ4EC+jsw3aXEMop9WAdXWHq0cSQ7uqytcJSDdur5bp8aBIddJuk2Lvda3cKA5FGeBBvYzONmhzDaGMk0MBDu0YJ2+vZmOpWqqT9lHNm9GytESX6hepUAR4rj2zoF5GZzsLn08IHbG9a10dzJgRGR75xz8CGdrxnPqZ2bo3SaqT8+76Bu7/bv/AXU0qb1gf4Pn0zIJ8GZ3NPtp5h1Cspx0f2l27Bja04zn1M7O1VC3dSVtvn1a8rA/wfHpmtl5Gx+QUQqlC+4YbIhORAQ7teE7+zGwMPFuvHJS7rA/wfHpmQf7HkO1wUNoQKjC0gzjBG+SfmRdsvXJQ7rI+wPMdD4bg/WMoaKKurg7mzYtMRCaGdlUVDB6cVU87qBO8Qf2ZecnGKwflLutXYoZpJVrOKyZThXZVVU6hXVAbLBfEqw2lEoV2JWaYemZZDQfFQru6GhYsyLunXVAbEtgagk5fbdh6HJS9rA9wCM+lZarhoJOPOgiPP+5aaGfThlRDUkEdcsmGk+WkNh8HZS8rAtypnk3Qe0jxE3Vf3b+Xb733Che9s4KzNq2Bhi8jof2Tn0QmIh0M7VRtiEk3WRjUmvpsOFlOavNxUPYKfIBn27PJFM429JBGn3gMXb6ykfpZTzLo3ZUcdfAA+77WiZKfXO9qaDdpQ45DUkGuqc/EydJEm4+DslfgAzybnk024RzYHtLu3U0mIk/78kvo0gVujAyPtBoyxPM67VyGpGyuqXeyNNHm46DsFfg78mTTs0kXzrm8jmd274a//AUuugiOPx7GjoU33oDrr4eXX4bNm+F3v4OhQwO/yMbmu6iMrijj7jF9KWvTEiFSaZNv9ZLNx0HZK/A98Gx6NtmEs+89pFhPOzYRGetpX3/94YnIZoE/nx7B9iogpybAbT8Oyk6BD/BsLnOzCWdfVvKFNLQThaUKqFB6HJTXMga4iDwCXAhsN8b0iT7WDngK6A5sAi4zxuxyo4HpejaxicvaunoEiF+SlBjOnvWQUoX2j398eCIyBKGtlPJfxpWYInIWsBd4PC7Afwt8aoyZKiKTgLbGmFszvZmTKzGTrcCMhXiZ15evqUL7O9/R0FZKFSzvlZjGmGUi0j3h4UuAs6N/ngEsBTIGuJOSTVzGwtuTJd+7d8Ozz0aqRxJ72lVVMGSIhrZSylX5joGfYIz5KPrnj4ETUj1RRCYAEwC6du2a59sdyZeqkj17Dpf8xUK7c2cNbaWULwqexDTGGBFJOQ5jjJkOTIfIEEqur59qgY5nVSWx0K6uhr//XUPbBUFfIatUUOUb4NtEpKMx5iMR6Qhsd7JRMekW6LhaVaKh7RkbVsgqFVT5Bvg8YBwwNfr/ZxxrUZx0C3Ri49yO9dziQ3vBAvjii0hoX3ddZCLSktC2rTcb2BWySlkgmzLCvxGZsGwvIluBKUSC+2kRuRrYDFzmRuMyjXMXXHebKrSvvdaq0I6xsTcbqBWySlkmmyqUK1L81UiH23IEV8a59+xpWj0SH9pVVZHl6xaFdjynerNe9uJ9XyGrlMUCnVSO7S+xZw/MnAmXXBLZe+TKK+H11yOhvWJFZO+R+++HM86wNrzBmd5srBdfW1eP4XAvvmZ1rUOtbEr3EFEqf4FeSl/Q6slYTzs2EfnFF1BWFoqedipO9Ga9HpPWPUSUyl+gAxxyHOdOFdr//d9w2WXUHNWFe158lw/n7aLTsqWhCwonKnP8GJPWPUSUyk/gAzyjDKEd62nbOMGXKyd6s0EZk7atmkYpP9gZ4OlCu6oKTj/9iOGRYilXK7Q368uujQmK4WSrlBPsCfDPPjtcPZJlaMfTcrXsBGFMulhOtkoVyo4Av+46eOSRSGh36pR1aMcLytCAE9weXvB7TFpPtkplx44yjM6dI6G9fDls2QLTpkFlZU5VJGEpV/O6zM8PqU6qNp5slXKTHQE+eXJeoR3Pyfsf+imb+3/aLiwnW6XcZscQikP8HhpwQjEMLwRhHF4pGxRVgMfYXKIWprH8dMJwslXKbXYMoTjI9jFkHV5QSsUUTQ88/gbIiWwqUdPhBaVUTFEEeLIbICeyaQxZhxeUUlAkAZ6sciNRpjFkm8fNlVLhVBQBnql3nWkMWZd2K6WCqCgmMdP1rrOpBy+G2mullH2KIsBTVW5M+25/VkwakbEXXQy110op+xRFgBe6ClOXdiulgijwY+BOTR6mq9zI9B5B2GJVKaUSBTrAvZg8zPQesXCvbzhIiQgHjaFMq1CUUgEQ6AD3Yl/oTBOU8eF+0JjGnreGt1LKb4EeA/di8jDde2j1iVIqyAId4F5MHqZ7D60+UUoFWaAD3IuNm9K9h1afKKWCLNBj4F5s3JTpPbT6RCkVVGKM8ezNBg4caFauXOnZ+yWTa1mi7oGilPKbiKwyxgxMfDzQPXCn5VOWGLSd//SEopSKKWgMXETOE5GNIvKeiExyqlFusb2qxPabUSilnJV3gItICfAAcD7QC7hCRHo51TA32F5VYvsJSCnlrEJ64IOB94wx7xtjvgSeBC5xplnusL2qxPYTkFLKWYUEeBmwJe7rrdHHmhCRCSKyUkRW7tixo4C3K5zt95O0/QSklHKW63XgxpjpxpiBxpiBHTp0cPvt0ip0V0K/2X4CUko5q5AqlFqgS9zXnaOPBVrQqkpyoTc0VkrFKyTAXwNOEpEeRIL7cuB7jrRKAalLBjWwlVJQQIAbYw6IyPXAAqAEeMQYs96xlhU5vQ+nUiqTgsbAjTHPG2NONsZ83Rhzl1ONUloyqJTKLNCbWRUzLRlUSmWiAR5QWjKolMpEAzygtGRQKZVJUW1mZRMtGVRKZaIBHmBaMqiUSkeHUJRSylIa4EopZSkrh1D0pgZKKWVhgOe7QlFDXykVNtYNoeSzQlHvZKOUCiPrAjyfFYq6LF0pFUbWBXg+KxR1WbpSKoysC/B8VijqsnSlVBhZF+D53FVHl6UrpcLIuioUyH2Foi5LV0qFkZUBng9dlq6UCpuiCXCVmdbKK2UXDXAF6C3clLKRdZOYyh1aK6+UfTTAFaC18krZSANcAVorr5SNNMAVoLXyStlIJzEVoLXyStlIA1w10lp5peyiQyhKKWUpDXCllLKUBrhSSllKA1wppSylAa6UUpYSY4x3byayA9icw7e0Bz5xqTlBVYyfGfRzF5Ni/MxQ2OfuZozpkPigpwGeKxFZaYwZ6Hc7vFSMnxn0c/vdDi8V42cGdz63DqEopZSlNMCVUspSQQ/w6X43wAfF+JlBP3cxKcbPDC587kCPgSullEot6D1wpZRSKWiAK6WUpQIZ4CJynohsFJH3RGSS3+3xgoh0EZElIvKWiKwXkRv9bpNXRKRERFaLyHN+t8UrItJGRGaJyNsiskFETve7TV4QkZujv9/rRORvItLC7za5QUQeEZHtIrIu7rF2IvKiiLwb/X/bQt8ncAEuIiXAA8D5QC/gChHp5W+rPHEA+JkxphcwFPhxkXxugBuBDX43wmO/B/5ujCkHTqUIPr+IlAE3AAONMX2AEuByf1vlmseA8xIemwQsMsacBCyKfl2QwAU4MBh4zxjzvjHmS+BJ4BKf2+Q6Y8xHxpjXo3/+jMg/6NBvzi0inYFRwEN+t8UrInIscBbwMIAx5ktjTJ2vjfJOc6CliDQHWgEf+tweVxhjlgGfJjx8CTAj+ucZwOhC3yeIAV4GbIn7eitFEGTxRKQ7UAG84nNTvDANuAU45HM7vNQD2AE8Gh06ekhEjva7UW4zxtQC9wL/AT4Cdhtj/uFvqzx1gjHmo+ifPwZOKPQFgxjgRU1EjgFmAzcZY/b43R43iciFwHZjzCq/2+Kx5sAA4EFjTAXwOQ5cTgdddMz3EiInsE7A0SJypb+t8oeJ1G8XXMMdxACvBbrEfd05+ljoiUgpkfCeaYyZ43d7PFAJXCwim4gMlY0Qkb/62yRPbAW2GmNiV1iziAR62H0L+MAYs8MY0wDMAc7wuU1e2iYiHQGi/99e6AsGMcBfA04SkR4i8hUikxzzfG6T60REiIyJbjDG3Od3e7xgjLnNGNPZGNOdyM95sTEm9D0yY8zHwBYR6Rl9aCTwlo9N8sp/gKEi0ir6+z6SIpi8jTMPGBf98zjgmUJfMHA3NTbGHBCR64EFRGapHzHGrPe5WV6oBH4AvCkia6KPTTbGPO9fk5SLfgLMjHZS3gd+6HN7XGeMeUVEZgGvE6m6Wk1Il9WLyN+As4H2IrIVmAJMBZ4WkauJbKt9WcHvo0vplVLKTkEcQlFKKZUFDXCllLKUBrhSSllKA1wppSylAa6UUpbSAFdKKUtpgCullKX+P1MWfxqwynBsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "N = X.shape[0]\n",
    "\n",
    "S_X2 = np.sum(X*X)\n",
    "S_X  = np.sum(X)\n",
    "S_XY = np.sum(X*Y)\n",
    "S_Y  = np.sum(Y)\n",
    "\n",
    "A1 = np.array([[S_X2, S_X], \n",
    "               [S_X, N]])\n",
    "B1 = np.array([S_XY, S_Y])\n",
    "# numpy.linalg模块包含线性代数的函数。使用这个模块，可以计算逆矩阵、求特征值、解线性方程组以及求解行列式等。\n",
    "coeff = np.linalg.inv(A1).dot(B1)\n",
    "\n",
    "print('a = %f, b = %f' % (coeff[0], coeff[1]))\n",
    "\n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = coeff[0] * x_min + coeff[1]\n",
    "y_max = coeff[0] * x_max + coeff[1]\n",
    "\n",
    "plt.scatter(X, Y, label='original data')\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 如何使用迭代的方法求出模型参数\n",
    "\n",
    "当数据比较多的时候，或者模型比较复杂，无法直接使用解析的方式求出模型参数。因此更为常用的方式是，通过迭代的方式逐步逼近模型的参数。\n",
    "\n",
    "### 2.1 梯度下降法\n",
    "在机器学习算法中，对于很多监督学习模型，需要对原始的模型构建损失函数，接下来便是通过优化算法对损失函数进行优化，以便寻找到最优的参数。在求解机器学习参数的优化算法中，使用较多的是基于梯度下降的优化算法(Gradient Descent, GD)。\n",
    "\n",
    "梯度下降法有很多优点，其中最主要的优点是，**在梯度下降法的求解过程中只需求解损失函数的一阶导数，计算的代价比较小，这使得梯度下降法能在很多大规模数据集上得到应用。**\n",
    "\n",
    "梯度下降法的含义是通过当前点的梯度方向寻找到新的迭代点。梯度下降法的基本思想可以类比为一个下山的过程。假设这样一个场景：\n",
    "* 一个人被困在山上，需要从山上下来(i.e. 找到山的最低点，也就是山谷)。\n",
    "* 但此时山上的浓雾很大，导致可视度很低。因此，下山的路径就无法确定，他必须利用自己周围的信息去找到下山的路径。\n",
    "* 这个时候，他就可以利用梯度下降算法来帮助自己下山。\n",
    "    - 具体来说就是，以他当前的所处的位置为基准，寻找这个位置最陡峭的地方，然后朝着山的高度下降的地方走\n",
    "    - 同理，如果我们的目标是上山，也就是爬到山顶，那么此时应该是朝着最陡峭的方向往上走。\n",
    "    - 然后每走一段距离，都反复采用同一个方法，最后就能成功的抵达山谷。\n",
    "\n",
    "\n",
    "我们同时可以假设这座山最陡峭的地方是无法通过肉眼立马观察出来的，而是需要一个复杂的工具来测量，同时，这个人此时正好拥有测量出最陡峭方向的能力。所以，此人每走一段距离，都需要一段时间来测量所在位置最陡峭的方向，这是比较耗时的。那么为了在太阳下山之前到达山底，就要尽可能的减少测量方向的次数。这是一个两难的选择，如果测量的频繁，可以保证下山的方向是绝对正确的，但又非常耗时，如果测量的过少，又有偏离轨道的风险。所以需要找到一个合适的测量方向的频率，来确保下山的方向不错误，同时又不至于耗时太多！\n",
    "\n",
    "\n",
    "![gradient_descent](images/gradient_descent.png)\n",
    "\n",
    "如上图所示，得到了局部最优解。x,y表示的是$\\theta_0$和$\\theta_1$，z方向表示的是花费函数，很明显出发点不同，最后到达的收敛点可能不一样。当然如果是碗状的，那么收敛点就应该是一样的。\n",
    "\n",
    "对于某一个损失函数\n",
    "$$\n",
    "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2\n",
    "$$\n",
    "\n",
    "我们更新的策略是：\n",
    "$$\n",
    "\\theta^1 = \\theta^0 - \\eta \\triangledown L(\\theta)\n",
    "$$\n",
    "其中$\\theta$代表了模型中的参数，例如$a$, $b$\n",
    "\n",
    "此公式的意义是：$L$是关于$\\theta$的一个函数，我们当前所处的位置为$\\theta_0$点，要从这个点走到L的最小值点，也就是山底。首先我们先确定前进的方向，也就是梯度的反向，然后走一段距离的步长，也就是$\\eta$，走完这个段步长，就到达了$\\theta_1$这个点！\n",
    "\n",
    "我们更新的策略是：\n",
    "\n",
    "$$\n",
    "a^1 = a^0 + 2 \\eta [ y - (ax+b)]*x \\\\\n",
    "b^1 = b^0 + 2 \\eta [ y - (ax+b)] \n",
    "$$\n",
    "\n",
    "下面就这个公式的几个常见的疑问：\n",
    "\n",
    "* **$\\eta$是什么含义？**\n",
    "$\\eta$在梯度下降算法中被称作为学习率或者步长，意味着我们可以通过$\\eta$来控制每一步走的距离，以保证不要步子跨的太大，错过了最低点。同时也要保证不要走的太慢，导致太阳下山了，还没有走到山下。所以$\\eta$的选择在梯度下降法中往往是很重要的。\n",
    "![gd_stepsize](images/gd_stepsize.png)\n",
    "\n",
    "* **为什么要梯度要乘以一个负号？**\n",
    "梯度前加一个负号，就意味着朝着梯度相反的方向前进！梯度的方向实际就是函数在此点上升最快的方向，而我们需要朝着下降最快的方向走，自然就是负的梯度的方向，所以此处需要加上负号。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 示例代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch    0: loss = 7670.746142, a = 2.354092, b = 3.762462\n",
      "epoch   50: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  100: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  150: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  200: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  250: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  300: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  350: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  400: loss = 8594.865640, a = 1.749804, b = 6.543725\n",
      "epoch  450: loss = 8594.865640, a = 1.749804, b = 6.543725\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnIklEQVR4nO3deZBU1fnw8e/DMDrDIouAwQEBlX2RgWFQR4xAosYlIEncFZL4wzIxRpPXBFMVMav8EsqQVCX5FRVN+L3RREVDiMvrL4q+KvUWy7AjIoSAzMgOAwIDzMB5/7g90z1D9/R2t3P7+VRRzHT3dJ/b0/Pcc5/znHPEGINSSin7tAu6AUoppXKjAVwppSylAVwppSylAVwppSylAVwppSzV3s8X69Gjh+nfv7+fL6mUUtarrq7eb4zp2fp2XwN4//79WblypZ8vqZRS1hORHclu1xSKUkpZSgO4UkpZSgO4UkpZytcceDINDQ3U1NRw4sSJoJtS0EpKSujTpw/FxcVBN0UplaHAA3hNTQ2dO3emf//+iEjQzSlIxhgOHDhATU0NAwYMCLo5SqkMBR7AT5w4ocE7YCLC+eefz759+4Juiiogi1bX8ss3NvNJXT0Xdi3l0esGM7W8LOhmWSXwAA5o8A4B/R0oPy1aXctjL6+nvuE0ALV19Tz28noADeJZ0EFMpZTvfvnG5ubg3aS+4TS/fGNzQC2ykwZwl/Xv35/9+/fn/RilouyTuvqsblfJaQBXSvnuwq6lWd2uktMADmzfvp0hQ4YwY8YMBg0axF133cWbb75JVVUVAwcOZPny5Rw8eJCpU6cyatQoLr/8ctatWwfAgQMHuPbaaxk+fDj33XcfiTsc/fnPf6ayspLRo0dz//33c/r06VRNUKqgPHrdYEqLi1rcVlpcxKPXDQ6oRXYKxSBms4cfhjVr3H3O0aNh3ry0D9u6dSsvvvgizzzzDOPGjeO5557j/fffZ/Hixfz85z+nb9++lJeXs2jRIpYsWcK9997LmjVr+NGPfsRVV13F448/zquvvsrTTz8NwKZNm3j++edZunQpxcXFfOMb3+DZZ5/l3nvvdff4lLJQ00ClVqHkJ1wBPEADBgxg5MiRAAwfPpzJkycjIowcOZLt27ezY8cOXnrpJQAmTZrEgQMHOHLkCO+++y4vv/wyADfeeCPdunUD4K233qK6uppx48YBUF9fT69evQI4MqXCaWp5mQbsPIUrgGfQU/bKueee2/x1u3btmr9v164djY2NWc9QNMYwffp0nnzySVfbqZRSTTQHnqEJEybw7LPPAvDOO+/Qo0cPzjvvPK6++mqee+45AF5//XUOHToEwOTJk1m4cCF79+4F4ODBg+zYkXRFSKWUykm4euAh9sQTT/C1r32NUaNG0aFDBxYsWADA7NmzueOOOxg+fDhXXnklF110EQDDhg3jpz/9Kddeey1nzpyhuLiY3/72t/Tr1y/Iw1BKRYgkVk14raKiwrTe0GHTpk0MHTrUtzao1PR3oVQ4iUi1Maai9e2aQlFKKUtpCkUppWJsW2BLA7hSSmHnAluaQlFKKexcYEsDuFJKYecCWxrAlVIKOxfY0gCehRtuuIG6uro2H/P444/z5ptv5vT877zzDjfddFPax11zzTW0Lsdsbd68eRw/fjyndihViGxcYEsHMTNgjMEYw2uvvZb2sT/+8Y99aFF68+bN4+6776ZDhw5BN0UpK9i4wJZ1PfBFq2upmrOEAbNepWrOEhatrs37OZ966ilGjBjBiBEjmBdbj2X79u0MHjyYe++9lxEjRrBz584WGzH85Cc/YfDgwVx11VXccccdzJ07F4AZM2awcOFCwNm4Yfbs2YwZM4aRI0fy4YcfArB8+XKuuOIKysvLufLKK9m8ue1Bkvr6em6//XaGDh3KLbfcQn19PCf3wAMPUFFRwfDhw5k9ezYAv/nNb/jkk0+YOHEiEydOTPk4pVRLU8vLWDprEv+ecyNLZ00KdfAGy3rgXpT5VFdX88c//pFly5ZhjGH8+PF89rOfpVu3bmzZsoUFCxZw+eWXt/iZFStW8NJLL7F27VoaGhoYM2YMY8eOTfr8PXr0YNWqVfzud79j7ty5/OEPf2DIkCG89957tG/fnjfffJMf/OAHzSsdJvP73/+eDh06sGnTJtatW8eYMWOa7/vZz35G9+7dOX36NJMnT2bdunU89NBDPPXUU7z99tv06NEj5eNGjRqV03umVFTYVvfdmlU9cC/KfN5//31uueUWOnbsSKdOnZg2bRrvvfceAP369TsreAMsXbqUKVOmUFJSQufOnbn55ptTPv+0adMAGDt2LNu3bwfg8OHDfOUrX2HEiBE88sgjbNy4sc02vvvuu9x9990AjBo1qkXgfeGFFxgzZgzl5eVs3LiRDz74IOlzZPo4pQpFU4ewtq4eQ7xD6MZVvV+sCuB+l/l07Ngx7+doWpa2qKiIxsZGAH74wx8yceJENmzYwD/+8Q9OnDiR03P/+9//Zu7cubz11lusW7eOG2+8MelzZfo4pQqJjXXfrWUcwEWkSERWi8grse8HiMgyEdkqIs+LyDneNdPhRZnPhAkTWLRoEcePH+fYsWP87W9/Y8KECW3+TFVVVXPgPXr0KK+88kpWr3n48GHKypzLtD/96U9pH5+4ZO2GDRuat3M7cuQIHTt2pEuXLuzZs4fXX3+9+Wc6d+7Mp59+mvZxKnNuj794MZ5T6LJ5T22s+24tmx74t4FNCd//J/ArY8ylwCHg6242LBkvynzGjBnDjBkzqKysZPz48dx3332Ul5e3+TPjxo3ji1/8IqNGjeILX/gCI0eOpEuXLhm/5ve+9z0ee+wxysvLm3vlbXnggQc4evQoQ4cO5fHHH2/Ot1922WWUl5czZMgQ7rzzTqqqqpp/ZubMmVx//fVMnDixzcepzLh9uR2Fy/ewyfY9tbHuu7WMlpMVkT7AAuBnwHeAm4F9wGeMMY0icgXwhDHmuraex43lZMMy6HD06FE6derE8ePHufrqq5k/f36LwUUb6XKyqVXNWUJtkp5ZWddSls6aFPjzqezf09ZFEeB0CJ+cNjJ0A5mplpPNtAplHvA9oHPs+/OBOmNMU/exBkh6xCIyE5gJNG92kI+w7KM3c+ZMPvjgA06cOMH06dOtD96qbW5fbkfh8j0XXnbAsn1Pbaz7bi1tABeRm4C9xphqEbkm2xcwxswH5oPTA8/258OqKSetCsOFXUuT9u5yvdx2+/ls4PVqf7m8p2HpEOYqkxx4FfBFEdkO/BWYBPwa6CoiTSeAPkDOyTs/dwVSyenvoG1uj7/YOG07X15XfRTie5o2gBtjHjPG9DHG9AduB5YYY+4C3ga+HHvYdODvuTSgpKSEAwcOaAAJkDGGAwcOUFJSEnRTQmtqeRlPThtJWddSBCevmk+u1O3ns4HXaaNCfE+z2hMzlkL5X8aYm0TkYpweeXdgNXC3MeZkWz+fbBCzoaGBmpoarUsOWElJCX369KG4uDjopqiI0oHb3OU7iAmAMeYd4J3Y19uAynwbVlxczIABA/J9GqVUyD163eCkVR9RTnF4zaq1UJRS9opC1UfYaABXSvnG9qqPsLFqLRSllFJxGsCVUspSGsCVUspSmgNXSnkmLGsXRZUGcKWUJ7yeOq80haKU8kgUNkwIOw3gSilPFOqKi37SAK6U8kQUNkwIOw3gSilPFOLqgH7TQUyllCd06rz3NIArpTyjU+e9pSkUpZSylAZwpZSylAZwpZSylObAlTV0WrYKO78/oxrAlRV0WrYKuyA+o5pCUVbQadmqLYtW11I1ZwkDZr1K1ZwlLFpd63sbgviMag9cWUGnZatUwnJ1FsRnVHvgygo6LVulEparsyA+oxrAlRV0WrZKxaueb7ZpmSA+o5pCUVbQadkqlQu7llKbJFjn0/PNJS0TxGdUjDGePXlrFRUVZuXKlb69nlIq+loHW3B6vk9OG5lz8KyasyTpSaGsaylLZ03Kua25EpFqY0xF69u1B66UspoXPV9bBs01gCulPOXH5Ba3F81yKy3j9bFrAFehozMug+Xm+x+WEr9sPXrd4KRpmWwGJP04dq1CUaHS9KGvravHEP/QBzExoxC5/f6HpcQvW1PLy3hy2kjKupYiOLnvbHPqfhx72h64iJQA7wLnxh6/0BgzW0QGAH8FzgeqgXuMMadca5kqSG196MPcY4sKt99/L0v8wp6W8SOPnkkP/CQwyRhzGTAauF5ELgf+E/iVMeZS4BDwdddapQqWLYNHUeX2++/F5BZbrtL8mNiTNoAbx9HYt8WxfwaYBCyM3b4AmOpaq1TB0hmXwXL7/fdicostaRk/JvZklAMXkSIRWQPsBf4J/AuoM8Y0xh5SAyS91hCRmSKyUkRW7tu3z4UmqyjTGZfBcvv9dyOX3JotV2leHHtrGVWhGGNOA6NFpCvwN2BIpi9gjJkPzAdnIk8ObVQFRGdcBsuL9z+sJX5+8HpP0KzKCI0xdSLyNnAF0FVE2sd64X2AcCWglLV0I9xghf39T1biJ8DEIT09eb0wl7WmTaGISM9YzxsRKQU+D2wC3ga+HHvYdODvHrVRKaWaTS0v40tjy5CE2wzwUnWt6wOZ2QyYBrEmeSY58N7A2yKyDlgB/NMY8wrwfeA7IrIVp5Twae+aqZRScW9/uI/W+VgvBjIzHTANqjImbQrFGLMOKE9y+zag0otGqegL82VpIlvaWWj8GsjM9HWCmr+gMzGV72yp47WlnYXIr3LTTF8nqMoYDeDKd7bU8drSzkLkV7npxCE9W+TaU71OUPMXNIAr39lSx2tLOwuRHzXWi1bX8lJ1bYtcuwBfGnt2lU5Q8xd0NULlO1vqeG1pp23cGlfwutwx2RWYwRlATdaWpp/xc7xEA7jynRtLdfrBlnbaxKblZZOdvJtuHzDr1bOCdBD18xrAle9smW2Zrp1aoZI9W1abXLS6FoGzShWbJA5qQ3AnHw3gKhBhn+3XJFU7bepJhokt4wq/fGNzyuCdKOiTjwZwFUle945t6Ulmyq+rCVvGFbI5oQR58tEqFBU5ftRv29KTzISf9e5BVWtkO809mxNKkCcfDeAqcvyo38607jeI9TGy5We9ux/lf63lcoJKdqIpLhKK27WsCg96UFtTKCpy/OgdZ1KhYkue3O+rCb/HP3JJd6UawE52W5C/Sw3gKnL8yLNmUkljS57clrx0rnI9QaU60YTpd6cBPGS0NC1/ftVvp+tJ2pInj3q9e5RPUBrAQ8SWS263uX3SCkuduS2BIyzvV1vy+YxE+QQlxvi3y1lFRYVZuXKlb69nm6o5S5L+wZd1LWXprEkBtMh7rU9a4PxxeT2w5YcoH1sucg3Cbb2PkNmJx/YrWxGpNsZUtL5de+AhYsslt5tsyRPnwoaebSIvg1w+V5epPiNPLN7IycYzGT1noBPHTp+GzZth6FCQ1msb5kcDeIjYcsmdq2QBIionrVTBz5YZp8kC7CPPr+Hh59dQ5kIwz+dEneqzUFffcNZtgZ/8jYHaWli+PP5v5Ur49FP46CMYONDVl9MAHiJRztWl6oF1KS1O+odo00krCmMXqVbeA3eCeT4n6lQdm2xfyxN1dU6ATgzYu3Y59xUXw2WXwT33QGUl9Ojh+stbG8Btz2klY9sldzZS9cBKittRWlxk9UkrCmmgdEEvMZjncnLK5+oyVcempLgdh477ePI/eRLWrm0ZrDcnTHYaNAgmT3aCdWWlE7xLSrxpS4yVATwKPZ5UbLnkzlbKy+DjDfzqttGen7S8POFn2rvMtA1BdE6y6eXmcnLK5+qyrUk1nl2xnjkDW7a0DNZr1sCpU879F1wA48fHe9cVFdCtW/6vmyUrA3gUejyJong10VpbPbC2Vvxz433x+oSfSe8y0zYE1TlJFmDbkm2aIt+ry7Y6Nq787eza1TJYr1gBhw8793Xs6ATohx+O96779HF9QDIXVgbwqAx8QbSvJhJl2wNz833J9oSf7Ykjk2PLtA1BdU4SA2xtXX2ba2FDbmkKL64uc3rOI0egurplwK6pce4rKoJRo+D22+PBeuhQ5/YQsjKAR6laI2pXE6lk2wNz833J5oSfy4kjk2NL1Ybaunqq5ixpfrzbnZNsTkaJwbDp55IFc7fSFL5ceZ46BevXtwzWmzY51SIAl1wCEybEg3V5OZTaE0esDOBRqtaI0tVEOtn0ltx8X7I54ed64kh3bG3lmBNPEm52TvK5ikkWzN0MtJ5ceRoD//pXy2C9apUz+AhOFcj48XDbbU6wHjcOzj8/r+MImpUBPErVGlG6mnCTm+9LNid8r06o6XLMTScJNzsnbl3FeJH6cKVte/e2DNbLl8OhQ859paUwdiw8+GC8d92vXyjy1m6yMoBDdKo1onQ14SY335dsTvhenVBb55iT+aSu3tXOSZiv7rJu29GjTm86MVjv2OHc164djBgBX/pSPFgPHw7trQ1vGYv+EYZclK4m3OT2+5LpCd/LE2pTG1KtedN0knCrcxLmq7s229bYCBs3tgzWGzY4pX0A/fs7qZBvfcsJ1mPGOJUiBUgDeAhE5WrCbUG8L36cUP246lq0upZjJxvPuj0sV3fN78GpRvoc3sPoXR8xds9Wbj5ZAz9eD/Wx4N69uxOkp06N56179Qq07WGSdjVCEekL/DdwAc5g9HxjzK9FpDvwPNAf2A7caow51NZz6WqEKihhq7X3c+GoJt06FDP75uHBdhb273dqrJcvZ/eb73Lu6mq6HXPqrU+fcy5FY8fE0yDjx8PFF0cub52LVKsRZhLAewO9jTGrRKQzUA1MBWYAB40xc0RkFtDNGPP9tp5LA3jhCFPALLRlXUOzLPHx47B6dctUyLZtzn0iMGxYPFhXVsLIkc76IeosOS8na4zZBeyKff2piGwCyoApwDWxhy0A3gHaDOCqMCxaXcujL66l4YzTOaitq+fRF9cCwUxOKpRa+yaBDF6ePu3UVycG63XrnNsB+vZ1gvT99zv/jx0LnTt7154CkVUOXET6A+XAMuCCWHAH2I2TYlGKJxZvbA7eTRrOGJ5YvDHrgOlGTz7M1Rhe8Hzw0hjYufPsJVOPHXPu79LFyVXPmhXPW/fu7c5rqxYyDuAi0gl4CXjYGHNEEvJSxhgjIklzMSIyE5gJcNFFF+XXWmWFZMvDtnV7Km5N9ghzNYYXXB8kPXSoOW/d/G/PHue+c86B0aPhq1+Np0IGDnRK+5TnMgrgIlKME7yfNca8HLt5j4j0NsbsiuXJ9yb7WWPMfGA+ODlwF9qsCoRbqY9Cq7XPq5LmxImzl0z96KP4/UOGwHXXxYP1qFFw7rkeHYlKJ20AF6er/TSwyRjzVMJdi4HpwJzY/3/3pIUFrq0UQpgGChN161CcdJ3mbh2yG6ByK/XR9J48sXhj81VASXG0e4gZlWCeOeOsZ50YrNeuhYbY7653b6cSZMaMeN66a1evm66ykEkPvAq4B1gvImtit/0AJ3C/ICJfB3YAt3rSwgLWVgoBCO0qhrNvHs6jC9fScDp+wVVcJMy+eXhWz+N26uNk45nmrw8db/Dt/QrNiTbZVl9Hjjj3de7sLJn6ne84QbuyEsqC7wyotmVShfI+kKoQc7K7zVGJ2kohNH2d7L6gA7hbk2HCuC5ItgJbLvjw4bO3+vrkE+e+9u2d3WLuuiueChk8OLRLpqrUdCZmiOWSQghLZYUbsyijsC6ILyeOU6eckr2mQL1sGXz4Yfz+gQNh4sR4sB492vOtvpQ/NICHWLoUQiFUVti+LojrJ44zZ2Dr1pY969Wr41t99erlpECaetcVFc50dBVJGsBDLF0KoZAqK/IVVCVK3ieO3bvP3uqrrs65r0MHJ0A/9FC8d33RRTr1vIBoAA+xTFIIXg6OhWbwzQWt38supcWIwCPPr2leh9uLY8vqxHH06NlbfX38sXNfUZEz1fzWW1tu9RWxJVOj9JnzQ9q1UNyka6HYI8rrh/h9bEmD0ohezhKpicH6gw/iS6ZefHHLdULKy50ed4RF+TOXr5wXs3KTBnB7hGZBJA/4fmzGOIs4td7q68QJ5/7zz2+5At+4cc72XwUmyp+5fOW8mJUqTFFeP8TzY9u37+ytvg4edO4rKXEmxDzwQDxoDxigeWui/ZnzigZwlVSU1w9x9diOH2+51deyZbB9u3Nfu3bO1l633NJyqy9dMjWpKH/mvKIBvEBkOzgU5fVDcj62xkYnT916q6+mJVP79XOC9De/Gd/qq1MnD48kWqL8mfOKBvCQ8WIUPpfZgFHeqzOjYzPG2TQ3MVhXVzs9bnDWBKmshJtvji+Z+pnP+H8wRKdyI8qfOa/oIGaIeDUKr4NDLSUNeP1Kz14ydW9sgc1zz3WqQBKrQi69NBR5a63cKAw6iGmBbKZdZ9Pr0sGhuEWra3ni+ZVcXLuFa3d9xGW7PqL8F1vgUGydEBGnvvqGG1pu9XXOOcE2PIVC221ItaQBPEQyDbTZpkQKenDo9GlnXZBYr3rI4rdYsXsbxWec925Xp/NZe+EgXqu8gQe+f6dTIXLeeQE3OrlkJ209ORe2gg3gYcwbZhpos+11RWVwKO3vzBioqTl7ydSjR537zzuP/d0G8FblNNb2HsTa3gPZ09mptxbggYkT/T+oDKU6aXdNsfZ6QZycVWEG8MCW+Ewj00Cbba8rCoNDyX5nP3/u/9FzWSNVB7fFS/h273Z+oLjYWXVv+vR4KmTQIL7/i3esvBpJddI+t307SouLrD85q9xEIoBn25sOa94w00CbS0rErVX9gjLv1fUM2vEBl8Xy1qN3beGSgzXxBwweDJ//fDxYX3ZZ0q2+bL0aSXVyPlzfwK9uG231yVnlzvoAnktvOsx5w0wCra1BKGNnzjj7MCakQv6nejXnnGkEYF/HrqzpPZiXh09kXe9B/O//+iZ065bRU9t6NdLWSdv2k7PKnfUBPJfetO2DerYGoZR27Wo5k3HFivhWX506QUUFL1z1Jd7vfjFrew9iV+cezSV8ZV1LMw7eTWwMeJE/aaucWB/Ac+lNh/mPIdN0kJdByNMB3iNHzl4ytSaWCmnf3tnl/M4746mQIUOgqIhOq2v5vyH9nfkhcidt5QrrA3iu+WAI3x9DGAZXXW3DqVOwfn3LYL1pk1MtAnDJJTBhQnwT3dGjoTT57y2svzM/2XjloLxl/UzMKM1EC8OMyZzbYEzyrb5OnnTu79mz5UzGceOcZVQDFsZyUqVai+xMzCj1zMIwuJpxG/bsaZ56vuef71K6dhXn1X8KQGNJKe3HVcCDD8YDdr9+oZh6nsjtKx49GSi/WR/AITqXlmEYXE3Whg6n6rnmWA3MnRvvXe/YAYBp146DPfuzetCVsckxg6jpPYCffnl06H8nbpaThiH9pQqPFQHcrZ5N2HtIYRhc/d7kS/jT/FcYsvPD5prrQfs/psjEtvrq39/JWX/rW1BZyefePsK/jrd6ktMEXlOfCTeveMI6t0BFW+gDeKY9m3TB2YYeku/pIGOczQcSSvimrFrFlHongB0q6cyHFw1hy9SpDJnyOSdv3atXi6fY9uqrSZ86DDX16bh5xROG9JcqPKEP4Jn0bDIJzrb0kDxNB+3ff/aSqfv3O/eVlDgbENx/f3PeutvFF3NFmrx1GNI+uXLzisfm90HZK/QBPJOeTSbBueB6SMePO1UgicF62zbnPhEYNszZjKCphG/EiJy2+gpD2idXbl7x2Pw+KHuFPoBn0rPJJDhHuod0+vTZW32tXx/f6qtvXydIN/Wux46Fzp1deWnbq4DcuuKx/X1Qdgp9AM+kZ5NJcI5MD8kY2Lmz5dTz6mo4dsy5v0sXJ0jPmhWvt+7d29MmRaUKKF/6Pii/pQ3gIvIMcBOw1xgzInZbd+B5oD+wHbjVGHPIiwa21bNpGrisratHgMQpSa2Ds7U9pEOHzs5b79nj3HfOOc5WX1/7Wsutvtq1C7bNSilfpJ2JKSJXA0eB/04I4L8ADhpj5ojILKCbMeb76V7MzZmYyWZgNgXxMluCc2snTsCaNS2D9ZYt8fuHDIkH6vHjna2+kiyZqpSKlpxnYhpj3hWR/q1ungJcE/t6AfAOkDaAuynZwGVT8LZio94zZ2Dz5ngaZPlyWLcOGmK7q/Tu7QTpr37VCdgVFU56RCmlYnLNgV9gjNkV+3o3cEGqB4rITGAmwEUXXZTjy53NuqqS2tqWPesVK+BTZ+o5nTs7uervfjfewy6z7OpBKeW7vAcxjTFGRFLmYYwx84H54KRQsn3+VBN0Ql1VcviwsxdjYsD+JLbreXGxs1vMPffEg/XgwQWdtw77DFmlwirXAL5HRHobY3aJSG9gr5uNatLWBJ3QVJWcPOmkPhKD9Ycfxu8fOBAmTowH69GjnUkzCrBjhqxSYZVrAF8MTAfmxP7/u2stStDWBJ2mPLevPbczZ1oumbpsmTPoeOqUc3+vXk7e+q674nnr7t29a08StvVmbZkhq1QYZVJG+BecAcseIlIDzMYJ3C+IyNeBHcCtXjQuXZ7b87rb3bvPzlvX1Tn3dezoBOhvfzveu+7bN9AlU23szVo3lqFUiGRShXJHirsmu9yWs/ia5/7007O3+tq507mvqMgp2bvttniwHjrUuT1E3OrN+tmLD/VYhlIhF+qZmJ7luRsaYMOGePne8uXOVPSmmviLL4aqqniwLi+HDh3ye00fuNGb9bsXH5qxDKUsFOoA7srsSWOcRZwSe9arVjmTZgB69HCC9Fe+Ep963qOHB0fjPTd6s37npK2dIatUCIQ6gEMOee59+1oG6+XL4eBBABpLStj4mUtZMfJ6Pr50BFfdeQPX3nh56Lb6ypUbvdkgctK6hohSuQl9AG/TsWNObzoxWG/f7tzXrh0MHw633AKVlSzpMoCH1p3iaELn8sVlh3my7JPIBA83erNhyUnbVk2jVBDs2ZW+sTG+ZGpT7nrDBqe0D5xNcxN3PR8zBjp1av7xMOz4boNka8yUFhfx5LSRvgXQMLRBqTCxe1f6//gPeO45Z5MCgG7dnCA9ZUo8b31Bytn8gJarZSoMOWmtDVcqM3YE8EGD4L774rvHXHJJ1nnrsKQG3OB1eiHonLSebJXKjB0B/NFH83+KiJSr2ThZJ1tROtkq5aWCWUFpankZT04bSVnXUgQn921jTrWt9EJUPHrdYEqLW06SsvFkq5TX7OiBuyTo1IAbCiG9EIY8vFI2KKgA3sTmErVCSS9E4WSrlNcKJoXSpCmHXFtXjyGeQ160ujbopmVE0wtKqSYF0wNP3AC5NZtK1DS9oJRqUhABPNnEkNZsyiFrekEpBQUSwJNVbrSWLodsc95cKRVNBRHA0/Wu0+WQC6H2Willn4IYxGyrd51JPXgh1F4rpexTEAE8VeXGvNtGs3TWpLS96EKovVZK2acgAni+szBT9eCjVnutlLJL6HPgbg0etlW5ke41orKOilIqWkIdwP0YPEz3Gk3Bvb7hNEUinDaGMq1CUUqFQKgDuB/rQqcboEwM7qeNae55a/BWSgUt1DlwPwYP23oNrT5RSoVZqAO4H4OHbb2GVp8opcIs1AHcj4Wb2noNrT5RSoVZqHPgfizclO41tPpEKRVW9uxK75JsyxJ1DRSlVNDs3pXeJbmUJYZt5T89oSilmuSVAxeR60Vks4hsFZFZbjXKK7ZXldi+GYVSyl05B3ARKQJ+C3wBGAbcISLD3GqYF2yvKrH9BKSUclc+PfBKYKsxZpsx5hTwV2CKO83yhu1VJbafgJRS7songJcBOxO+r4nd1oKIzBSRlSKyct++fXm8XP5s30/S9hOQUspdnteBG2PmG2MqjDEVPXv29Prl2pTvqoRBs/0EpJRyVz5VKLVA34Tv+8RuC7WwVZVkQzc0VkolyieArwAGisgAnMB9O3CnK61SQOqSQQ3YSinII4AbYxpF5EHgDaAIeMYYs9G1lhU43YdTKZVOXjlwY8xrxphBxphLjDE/c6tRSksGlVLphXoxq0KmJYNKqXQ0gIeUlgwqpdLRAB5SWjKolEqnoBazsomWDCql0tEAHmJaMqiUaoumUJRSylIawJVSylJWplB0UwOllLIwgOc6Q1GDvlIqaqxLoeQyQ1F3slFKRZF1ATyXGYo6LV0pFUXWBfBcZijqtHSlVBRZF8BzmaGo09KVUlFkXQDPZVcdnZaulIoi66pQIPsZijotXSkVRVYG8FzotHSlVNQUTABX6WmtvFJ20QCuAN3CTSkbWTeIqbyhtfJK2UcDuAK0Vl4pG2kAV4DWyitlIw3gCtBaeaVspIOYCtBaeaVspAFcNdNaeaXsoikUpZSylAZwpZSylAZwpZSylAZwpZSylAZwpZSylBhj/HsxkX3Ajix+pAew36PmhFUhHjPocReSQjxmyO+4+xljera+0dcAni0RWWmMqQi6HX4qxGMGPe6g2+GnQjxm8Oa4NYWilFKW0gCulFKWCnsAnx90AwJQiMcMetyFpBCPGTw47lDnwJVSSqUW9h64UkqpFDSAK6WUpUIZwEXkehHZLCJbRWRW0O3xg4j0FZG3ReQDEdkoIt8Ouk1+EZEiEVktIq8E3Ra/iEhXEVkoIh+KyCYRuSLoNvlBRB6Jfb43iMhfRKQk6DZ5QUSeEZG9IrIh4bbuIvJPEdkS+79bvq8TugAuIkXAb4EvAMOAO0RkWLCt8kUj8F1jzDDgcuCbBXLcAN8GNgXdCJ/9Gvg/xpghwGUUwPGLSBnwEFBhjBkBFAG3B9sqz/wJuL7VbbOAt4wxA4G3Yt/nJXQBHKgEthpjthljTgF/BaYE3CbPGWN2GWNWxb7+FOcPOvKLc4tIH+BG4A9Bt8UvItIFuBp4GsAYc8oYUxdoo/zTHigVkfZAB+CTgNvjCWPMu8DBVjdPARbEvl4ATM33dcIYwMuAnQnf11AAgSyRiPQHyoFlATfFD/OA7wFnAm6HnwYA+4A/xlJHfxCRjkE3ymvGmFpgLvAxsAs4bIz5n2Bb5asLjDG7Yl/vBi7I9wnDGMALmoh0Al4CHjbGHAm6PV4SkZuAvcaY6qDb4rP2wBjg98aYcuAYLlxOh10s5zsF5wR2IdBRRO4OtlXBME79dt413GEM4LVA34Tv+8RuizwRKcYJ3s8aY14Ouj0+qAK+KCLbcVJlk0Tkz8E2yRc1QI0xpukKayFOQI+6zwH/NsbsM8Y0AC8DVwbcJj/tEZHeALH/9+b7hGEM4CuAgSIyQETOwRnkWBxwmzwnIoKTE91kjHkq6Pb4wRjzmDGmjzGmP87veYkxJvI9MmPMbmCniAyO3TQZ+CDAJvnlY+ByEekQ+7xPpgAGbxMsBqbHvp4O/D3fJwzdpsbGmEYReRB4A2eU+hljzMaAm+WHKuAeYL2IrInd9gNjzGvBNUl56FvAs7FOyjbgqwG3x3PGmGUishBYhVN1tZqITqsXkb8A1wA9RKQGmA3MAV4Qka/jLKt9a96vo1PplVLKTmFMoSillMqABnCllLKUBnCllLKUBnCllLKUBnCllLKUBnCllLKUBnCllLLU/wePL85DrKS2HAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_epoch = 500          # epoch size\n",
    "a, b = 1, 1             # initial parameters\n",
    "epsilon = 0.01         # learning rate\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    for j in range(N):\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (Y[j]-a*X[j]-b)**2\n",
    "    \n",
    "    if i % 50 == 0:\n",
    "        print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n",
    "    \n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n",
    "\n",
    "plt.scatter(X, Y, label='original data')\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 如何可视化迭代过程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "/* global mpl */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function () {\n",
       "    if (typeof WebSocket !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof MozWebSocket !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert(\n",
       "            'Your browser does not have WebSocket support. ' +\n",
       "                'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "                'Firefox 4 and 5 are also supported but you ' +\n",
       "                'have to enable WebSockets in about:config.'\n",
       "        );\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = this.ws.binaryType !== undefined;\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById('mpl-warnings');\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent =\n",
       "                'This browser does not support binary websocket messages. ' +\n",
       "                'Performance may be slow.';\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = document.createElement('div');\n",
       "    this.root.setAttribute('style', 'display: inline-block');\n",
       "    this._root_extra_style(this.root);\n",
       "\n",
       "    parent_element.appendChild(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen = function () {\n",
       "        fig.send_message('supports_binary', { value: fig.supports_binary });\n",
       "        fig.send_message('send_image_mode', {});\n",
       "        if (fig.ratio !== 1) {\n",
       "            fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n",
       "        }\n",
       "        fig.send_message('refresh', {});\n",
       "    };\n",
       "\n",
       "    this.imageObj.onload = function () {\n",
       "        if (fig.image_mode === 'full') {\n",
       "            // Full images could contain transparency (where diff images\n",
       "            // almost always do), so we need to clear the canvas so that\n",
       "            // there is no ghosting.\n",
       "            fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "        }\n",
       "        fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "    };\n",
       "\n",
       "    this.imageObj.onunload = function () {\n",
       "        fig.ws.close();\n",
       "    };\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_header = function () {\n",
       "    var titlebar = document.createElement('div');\n",
       "    titlebar.classList =\n",
       "        'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n",
       "    var titletext = document.createElement('div');\n",
       "    titletext.classList = 'ui-dialog-title';\n",
       "    titletext.setAttribute(\n",
       "        'style',\n",
       "        'width: 100%; text-align: center; padding: 3px;'\n",
       "    );\n",
       "    titlebar.appendChild(titletext);\n",
       "    this.root.appendChild(titlebar);\n",
       "    this.header = titletext;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = (this.canvas_div = document.createElement('div'));\n",
       "    canvas_div.setAttribute(\n",
       "        'style',\n",
       "        'border: 1px solid #ddd;' +\n",
       "            'box-sizing: content-box;' +\n",
       "            'clear: both;' +\n",
       "            'min-height: 1px;' +\n",
       "            'min-width: 1px;' +\n",
       "            'outline: 0;' +\n",
       "            'overflow: hidden;' +\n",
       "            'position: relative;' +\n",
       "            'resize: both;'\n",
       "    );\n",
       "\n",
       "    function on_keyboard_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.key_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    canvas_div.addEventListener(\n",
       "        'keydown',\n",
       "        on_keyboard_event_closure('key_press')\n",
       "    );\n",
       "    canvas_div.addEventListener(\n",
       "        'keyup',\n",
       "        on_keyboard_event_closure('key_release')\n",
       "    );\n",
       "\n",
       "    this._canvas_extra_style(canvas_div);\n",
       "    this.root.appendChild(canvas_div);\n",
       "\n",
       "    var canvas = (this.canvas = document.createElement('canvas'));\n",
       "    canvas.classList.add('mpl-canvas');\n",
       "    canvas.setAttribute('style', 'box-sizing: content-box;');\n",
       "\n",
       "    this.context = canvas.getContext('2d');\n",
       "\n",
       "    var backingStore =\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        this.context.webkitBackingStorePixelRatio ||\n",
       "        this.context.mozBackingStorePixelRatio ||\n",
       "        this.context.msBackingStorePixelRatio ||\n",
       "        this.context.oBackingStorePixelRatio ||\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        1;\n",
       "\n",
       "    this.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n",
       "        'canvas'\n",
       "    ));\n",
       "    rubberband_canvas.setAttribute(\n",
       "        'style',\n",
       "        'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n",
       "    );\n",
       "\n",
       "    // Apply a ponyfill if ResizeObserver is not implemented by browser.\n",
       "    if (this.ResizeObserver === undefined) {\n",
       "        if (window.ResizeObserver !== undefined) {\n",
       "            this.ResizeObserver = window.ResizeObserver;\n",
       "        } else {\n",
       "            var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n",
       "            this.ResizeObserver = obs.ResizeObserver;\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n",
       "        var nentries = entries.length;\n",
       "        for (var i = 0; i < nentries; i++) {\n",
       "            var entry = entries[i];\n",
       "            var width, height;\n",
       "            if (entry.contentBoxSize) {\n",
       "                if (entry.contentBoxSize instanceof Array) {\n",
       "                    // Chrome 84 implements new version of spec.\n",
       "                    width = entry.contentBoxSize[0].inlineSize;\n",
       "                    height = entry.contentBoxSize[0].blockSize;\n",
       "                } else {\n",
       "                    // Firefox implements old version of spec.\n",
       "                    width = entry.contentBoxSize.inlineSize;\n",
       "                    height = entry.contentBoxSize.blockSize;\n",
       "                }\n",
       "            } else {\n",
       "                // Chrome <84 implements even older version of spec.\n",
       "                width = entry.contentRect.width;\n",
       "                height = entry.contentRect.height;\n",
       "            }\n",
       "\n",
       "            // Keep the size of the canvas and rubber band canvas in sync with\n",
       "            // the canvas container.\n",
       "            if (entry.devicePixelContentBoxSize) {\n",
       "                // Chrome 84 implements new version of spec.\n",
       "                canvas.setAttribute(\n",
       "                    'width',\n",
       "                    entry.devicePixelContentBoxSize[0].inlineSize\n",
       "                );\n",
       "                canvas.setAttribute(\n",
       "                    'height',\n",
       "                    entry.devicePixelContentBoxSize[0].blockSize\n",
       "                );\n",
       "            } else {\n",
       "                canvas.setAttribute('width', width * fig.ratio);\n",
       "                canvas.setAttribute('height', height * fig.ratio);\n",
       "            }\n",
       "            canvas.setAttribute(\n",
       "                'style',\n",
       "                'width: ' + width + 'px; height: ' + height + 'px;'\n",
       "            );\n",
       "\n",
       "            rubberband_canvas.setAttribute('width', width);\n",
       "            rubberband_canvas.setAttribute('height', height);\n",
       "\n",
       "            // And update the size in Python. We ignore the initial 0/0 size\n",
       "            // that occurs as the element is placed into the DOM, which should\n",
       "            // otherwise not happen due to the minimum size styling.\n",
       "            if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n",
       "                fig.request_resize(width, height);\n",
       "            }\n",
       "        }\n",
       "    });\n",
       "    this.resizeObserverInstance.observe(canvas_div);\n",
       "\n",
       "    function on_mouse_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.mouse_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousedown',\n",
       "        on_mouse_event_closure('button_press')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseup',\n",
       "        on_mouse_event_closure('button_release')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'dblclick',\n",
       "        on_mouse_event_closure('dblclick')\n",
       "    );\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousemove',\n",
       "        on_mouse_event_closure('motion_notify')\n",
       "    );\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseenter',\n",
       "        on_mouse_event_closure('figure_enter')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseleave',\n",
       "        on_mouse_event_closure('figure_leave')\n",
       "    );\n",
       "\n",
       "    canvas_div.addEventListener('wheel', function (event) {\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        on_mouse_event_closure('scroll')(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.appendChild(canvas);\n",
       "    canvas_div.appendChild(rubberband_canvas);\n",
       "\n",
       "    this.rubberband_context = rubberband_canvas.getContext('2d');\n",
       "    this.rubberband_context.strokeStyle = '#000000';\n",
       "\n",
       "    this._resize_canvas = function (width, height, forward) {\n",
       "        if (forward) {\n",
       "            canvas_div.style.width = width + 'px';\n",
       "            canvas_div.style.height = height + 'px';\n",
       "        }\n",
       "    };\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n",
       "        event.preventDefault();\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus() {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'mpl-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'mpl-button-group';\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'mpl-button-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        var button = (fig.buttons[name] = document.createElement('button'));\n",
       "        button.classList = 'mpl-widget';\n",
       "        button.setAttribute('role', 'button');\n",
       "        button.setAttribute('aria-disabled', 'false');\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "\n",
       "        var icon_img = document.createElement('img');\n",
       "        icon_img.src = '_images/' + image + '.png';\n",
       "        icon_img.srcset = '_images/' + image + '_large.png 2x';\n",
       "        icon_img.alt = tooltip;\n",
       "        button.appendChild(icon_img);\n",
       "\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    var fmt_picker = document.createElement('select');\n",
       "    fmt_picker.classList = 'mpl-widget';\n",
       "    toolbar.appendChild(fmt_picker);\n",
       "    this.format_dropdown = fmt_picker;\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = document.createElement('option');\n",
       "        option.selected = fmt === mpl.default_extension;\n",
       "        option.innerHTML = fmt;\n",
       "        fmt_picker.appendChild(option);\n",
       "    }\n",
       "\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', { width: x_pixels, height: y_pixels });\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_message = function (type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function () {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function (fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1], msg['forward']);\n",
       "        fig.send_message('refresh', {});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n",
       "    var x0 = msg['x0'] / fig.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n",
       "    var x1 = msg['x1'] / fig.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0,\n",
       "        0,\n",
       "        fig.canvas.width / fig.ratio,\n",
       "        fig.canvas.height / fig.ratio\n",
       "    );\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch (cursor) {\n",
       "        case 0:\n",
       "            cursor = 'pointer';\n",
       "            break;\n",
       "        case 1:\n",
       "            cursor = 'default';\n",
       "            break;\n",
       "        case 2:\n",
       "            cursor = 'crosshair';\n",
       "            break;\n",
       "        case 3:\n",
       "            cursor = 'move';\n",
       "            break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_message = function (fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n",
       "    for (var key in msg) {\n",
       "        if (!(key in fig.buttons)) {\n",
       "            continue;\n",
       "        }\n",
       "        fig.buttons[key].disabled = !msg[key];\n",
       "        fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n",
       "    if (msg['mode'] === 'PAN') {\n",
       "        fig.buttons['Pan'].classList.add('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    } else if (msg['mode'] === 'ZOOM') {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.add('active');\n",
       "    } else {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message('ack', {});\n",
       "};\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function (fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            var img = evt.data;\n",
       "            if (img.type !== 'image/png') {\n",
       "                /* FIXME: We get \"Resource interpreted as Image but\n",
       "                 * transferred with MIME type text/plain:\" errors on\n",
       "                 * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "                 * to be part of the websocket stream */\n",
       "                img.type = 'image/png';\n",
       "            }\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src\n",
       "                );\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                img\n",
       "            );\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        } else if (\n",
       "            typeof evt.data === 'string' &&\n",
       "            evt.data.slice(0, 21) === 'data:image/png;base64'\n",
       "        ) {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig['handle_' + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\n",
       "                \"No handler for the '\" + msg_type + \"' message type: \",\n",
       "                msg\n",
       "            );\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\n",
       "                    \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n",
       "                    e,\n",
       "                    e.stack,\n",
       "                    msg\n",
       "                );\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "};\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function (e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e) {\n",
       "        e = window.event;\n",
       "    }\n",
       "    if (e.target) {\n",
       "        targ = e.target;\n",
       "    } else if (e.srcElement) {\n",
       "        targ = e.srcElement;\n",
       "    }\n",
       "    if (targ.nodeType === 3) {\n",
       "        // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "    }\n",
       "\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    var boundingRect = targ.getBoundingClientRect();\n",
       "    var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n",
       "    var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n",
       "\n",
       "    return { x: x, y: y };\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys(original) {\n",
       "    return Object.keys(original).reduce(function (obj, key) {\n",
       "        if (typeof original[key] !== 'object') {\n",
       "            obj[key] = original[key];\n",
       "        }\n",
       "        return obj;\n",
       "    }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function (event, name) {\n",
       "    var canvas_pos = mpl.findpos(event);\n",
       "\n",
       "    if (name === 'button_press') {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * this.ratio;\n",
       "    var y = canvas_pos.y * this.ratio;\n",
       "\n",
       "    this.send_message(name, {\n",
       "        x: x,\n",
       "        y: y,\n",
       "        button: event.button,\n",
       "        step: event.step,\n",
       "        guiEvent: simpleKeys(event),\n",
       "    });\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.key_event = function (event, name) {\n",
       "    // Prevent repeat events\n",
       "    if (name === 'key_press') {\n",
       "        if (event.key === this._key) {\n",
       "            return;\n",
       "        } else {\n",
       "            this._key = event.key;\n",
       "        }\n",
       "    }\n",
       "    if (name === 'key_release') {\n",
       "        this._key = null;\n",
       "    }\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.key !== 'Control') {\n",
       "        value += 'ctrl+';\n",
       "    }\n",
       "    else if (event.altKey && event.key !== 'Alt') {\n",
       "        value += 'alt+';\n",
       "    }\n",
       "    else if (event.shiftKey && event.key !== 'Shift') {\n",
       "        value += 'shift+';\n",
       "    }\n",
       "\n",
       "    value += 'k' + event.key;\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n",
       "    if (name === 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message('toolbar_button', { name: name });\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "\n",
       "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n",
       "// prettier-ignore\n",
       "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";/* global mpl */\n",
       "\n",
       "var comm_websocket_adapter = function (comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.binaryType = comm.kernel.ws.binaryType;\n",
       "    ws.readyState = comm.kernel.ws.readyState;\n",
       "    function updateReadyState(_event) {\n",
       "        if (comm.kernel.ws) {\n",
       "            ws.readyState = comm.kernel.ws.readyState;\n",
       "        } else {\n",
       "            ws.readyState = 3; // Closed state.\n",
       "        }\n",
       "    }\n",
       "    comm.kernel.ws.addEventListener('open', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('close', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('error', updateReadyState);\n",
       "\n",
       "    ws.close = function () {\n",
       "        comm.close();\n",
       "    };\n",
       "    ws.send = function (m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function (msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        var data = msg['content']['data'];\n",
       "        if (data['blob'] !== undefined) {\n",
       "            data = {\n",
       "                data: new Blob(msg['buffers'], { type: data['blob'] }),\n",
       "            };\n",
       "        }\n",
       "        // Pass the mpl event to the overridden (by mpl) onmessage function.\n",
       "        ws.onmessage(data);\n",
       "    });\n",
       "    return ws;\n",
       "};\n",
       "\n",
       "mpl.mpl_figure_comm = function (comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = document.getElementById(id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm);\n",
       "\n",
       "    function ondownload(figure, _format) {\n",
       "        window.open(figure.canvas.toDataURL());\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element;\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error('Failed to find cell for figure', id, fig);\n",
       "        return;\n",
       "    }\n",
       "    fig.cell_info[0].output_area.element.on(\n",
       "        'cleared',\n",
       "        { fig: fig },\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function (fig, msg) {\n",
       "    var width = fig.canvas.width / fig.ratio;\n",
       "    fig.cell_info[0].output_area.element.off(\n",
       "        'cleared',\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "    fig.resizeObserverInstance.unobserve(fig.canvas_div);\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable();\n",
       "    fig.parent_element.innerHTML =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "    fig.close_ws(fig, msg);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.close_ws = function (fig, msg) {\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width / this.ratio;\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message('ack', {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () {\n",
       "        fig.push_to_output();\n",
       "    }, 1000);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'btn-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'btn-group';\n",
       "    var button;\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'btn-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        button = fig.buttons[name] = document.createElement('button');\n",
       "        button.classList = 'btn btn-default';\n",
       "        button.href = '#';\n",
       "        button.title = name;\n",
       "        button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message pull-right';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = document.createElement('div');\n",
       "    buttongrp.classList = 'btn-group inline pull-right';\n",
       "    button = document.createElement('button');\n",
       "    button.classList = 'btn btn-mini btn-primary';\n",
       "    button.href = '#';\n",
       "    button.title = 'Stop Interaction';\n",
       "    button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n",
       "    button.addEventListener('click', function (_evt) {\n",
       "        fig.handle_close(fig, {});\n",
       "    });\n",
       "    button.addEventListener(\n",
       "        'mouseover',\n",
       "        on_mouseover_closure('Stop Interaction')\n",
       "    );\n",
       "    buttongrp.appendChild(button);\n",
       "    var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n",
       "    titlebar.insertBefore(buttongrp, titlebar.firstChild);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._remove_fig_handler = function (event) {\n",
       "    var fig = event.data.fig;\n",
       "    if (event.target !== this) {\n",
       "        // Ignore bubbled events from children.\n",
       "        return;\n",
       "    }\n",
       "    fig.close_ws(fig, {});\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (el) {\n",
       "    el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (el) {\n",
       "    // this is important to make the div 'focusable\n",
       "    el.setAttribute('tabindex', 0);\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    } else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (event, _name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager) {\n",
       "        manager = IPython.keyboard_manager;\n",
       "    }\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which === 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "};\n",
       "\n",
       "mpl.find_output_cell = function (html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i = 0; i < ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code') {\n",
       "            for (var j = 0; j < cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] === html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "};\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel !== null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target(\n",
       "        'matplotlib',\n",
       "        mpl.mpl_figure_comm\n",
       "    );\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAggAAAFaCAYAAAB7UqUCAAAAAXNSR0IArs4c6QAAEJNJREFUeF7t2KERwDAMBMG4/6bDbXAF/AaLaGVwk/P5CBAgQIAAAQKXwCFCgAABAgQIELgFBII3QYAAAQIECDwCAsGjIECAAAECBASCN0CAAAECBAi0gD8IbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwICYe7kFiZAgAABAi0gENrIBAECBAgQmBMQCHMntzABAgQIEGgBgdBGJggQIECAwJyAQJg7uYUJECBAgEALCIQ2MkGAAAECBOYEBMLcyS1MgAABAgRaQCC0kQkCBAgQIDAnIBDmTm5hAgQIECDQAgKhjUwQIECAAIE5AYEwd3ILEyBAgACBFhAIbWSCAAECBAjMCQiEuZNbmAABAgQItIBAaCMTBAgQIEBgTkAgzJ3cwgQIECBAoAUEQhuZIECAAAECcwI/n0wBWzkAx+oAAAAASUVORK5CYII=\" width=\"432.2077922077922\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib nbagg\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.animation as animation\n",
    "\n",
    "n_epoch = 3000          # epoch size\n",
    "a, b = 1, 1             # initial parameters\n",
    "epsilon = 0.001         # learning rate\n",
    "\n",
    "fig = plt.figure()\n",
    "imgs = []\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    for j in range(N):\n",
    "        a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n",
    "        b = b + epsilon*2*(Y[j] - a*X[j] - b)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (Y[j]-a*X[j]-b)**2\n",
    "    #print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n",
    "    \n",
    "    if i % 50 == 0:\n",
    "        x_min = np.min(X)\n",
    "        x_max = np.max(X)\n",
    "        y_min = a * x_min + b\n",
    "        y_max = a * x_max + b\n",
    "\n",
    "        img = plt.scatter(X, Y, label='original data')\n",
    "        img = plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n",
    "        imgs.append(img)\n",
    "        \n",
    "ani = animation.ArtistAnimation(fig, imgs)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 如何使用批次更新的方法？\n",
    "\n",
    "如果有一些数据包含比较大的错误（异常数据），因此每次更新仅仅使用一个数据会导致不精确，同时每次仅仅使用一个数据来计算更新也导致计算效率比较低。\n",
    "\n",
    "\n",
    "* [梯度下降方法的几种形式](https://blog.csdn.net/u010402786/article/details/51188876)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 如何拟合多项式函数？\n",
    "\n",
    "需要设计一个弹道导弹防御系统，通过观测导弹的飞行路径，预测未来导弹的飞行轨迹，从而完成摧毁的任务。按照物理学，可以得知模型为:\n",
    "$$\n",
    "y = at^2 + bt + c\n",
    "$$\n",
    "我们需要求解三个模型参数$a, b, c$。\n",
    "\n",
    "损失函数的定义为：\n",
    "$$\n",
    "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVR0lEQVR4nO3df6xcZZnA8e9jKVrUUH40LNy2tokNBnVd8AZxmxiXugF/hBLXH7i7Blk2/QcUwaBF/5B/jDUYUaPLpgFdzBILiyw0SkSWYja7WVlaSkQorA0I7aVIUYpGqlB89o85Q2/rzPRO58ycmXO+n6S5c95z5sw77e0zZ573ed8TmYkkqVleUXUHJEmjZ/CXpAYy+EtSAxn8JamBDP6S1EBHVN2BuTj++ONz2bJlVXdDkibKli1bnsnMRZ32TUTwX7ZsGZs3b666G5I0USLi8W77TPtIUgMZ/CWpgQz+ktRABn9JaiCDvyQ10ERU+2j83Lp1hqvueIQn9+zlpIULuPyskzn31KmquyVpjgz+NdQtMPfb3uv8V9zyAHtffAmAmT17ueKWBwD8AJAmhMF/zPUKzJ32AR0D8+bHf833tszMub2t02tfdccjLx/ftvfFl7jqjke6PkfSeIlJWM9/eno66zLJq5+rbzgwkAMsmD+PL77/zV33vWr+K3j2+Rf/5HXnRfBSh3/rbu0LF8znD/v+2PG1L73xfrr91iyYP6/jc/wAkEYvIrZk5nTHfQb/0Tk4XQKt4Pg3b5064Oq73d4tkE8tXAC0rtJHrddrd/sgmVq4gP9ee+bQ+ybpQL2Cv9U+I9QtXfLde3Z0bO8U+AGe3LOXJ/sM/PMi+mrv5sk9e7n8rJNZMH/eAe0L5s/rGPjbz7l16wwr121i+dofsHLdJm7dOtPX60oqlzn/EeoWsLsFzW5O6nH13S1d0+3bRb/fOk5auODlFM7Baaqr7nikY5+OXjC/6wBxp/OYIpKGz+A/oH5y+CctXNBXuqRbIO81HnDlOW8EOgfU6dcdO+f2budv7zv31KmOQbrTcyLo+M3myo0PHvD+/FCQRsec/wD6zeH3294e2O2n2qfMAHk45+/0nF4DxJ30Gmz2A0CaOwd8h2Tluk19D3y20yOD1tpPkm5/T/1y4FjqT6/gb9pnDroF5n5z+E/u2ds1XdKtvQ4uP+vkvspSu2n/fdf5g1IaFYP/IfSazdpvDr89UNs03QaIob+5CictXODsYqkkBv9ZOl1R9prN2u2KtlsOvx3wmqjXN5t+BpudXSyVo5E5/35m0x4caNoCeGzde0tbL0cH6vb3t3ztD5xdLM2RA76zdKvQ6XdZBAcfq1HmILtUd87wnaVb2qDbwONLmR1nszY5hVOlfmcXt8cEZvbsJWdtO8NYTVfr4N9pSYF+l0WYWriAL77/zUwtXEDM2vbKsRrnnjrV8d9jqstg+ryInmMEUlOVMuAbEZcC/wgk8ABwAXAisAE4DtgCfDQzX4iIVwLfAd4K/Ar4cGb+oox+zNatKmThUfM7XuX3mk1b5zLMSdTP7OJuYzb9XgRIdTPwlX9ETAGfAKYz803APOA84EvA1Zn5euBZ4MLiKRcCzxbtVxfHla5beieTjmmDK895o1f4E6zfbwRNLbuV2soq9TwCWBARLwJHAbuAM4G/LfZfD1wJXAOsLh4D3Ax8IyIiSx557nZl99zeF7n6w3/RdQDQYD+5+vlGcPlZJ1uRpUYbOPhn5kxEfBl4AtgL/IhWmmdPZu4rDtsJtP9XTQE7iufui4jnaKWGnpl93ohYA6wBWLp0ad/96jYBq70qpf/Jm2GuE8wOnizmB4PqbuDgHxHH0LqaXw7sAf4NOHvQ82bmemA9tEo9+31+twlYVuk0T6cP+5XrNvUcCHYWsequjGqfdwGPZebuzHwRuAVYCSyMiPaHy2KgXVs3AywBKPYfTWvgt1TdcsD+5xV0Tws+uWfvIWcRS3VQRs7/CeCMiDiKVtpnFbAZuBv4AK2Kn/OB24rjNxbb/1Ps31R2vr/N9I666ZUW7PXBINXFwFf+mXkPrYHb+2iVeb6CVrrmM8BlEbGdVk7/uuIp1wHHFe2XAWsH7YPUr26Txdo33emkvbCct6NUHZRS7ZOZnwc+f1Dzo8DpHY79PfDBMl5XOlzdBoLb7Z3Gi/7qDYscC1BtuKqnGqvXvRWg8z2KXVFUddG4hd2kw+WKopo0LuwmlaDbWIDrB2kSGfylOep3RVGrgzTODP7SHLl+kOrEAV+pD/2uHySNK4O/NKBeZaOuEaRxZfCXStDpG0G3e0q0j5eqZM5fGhLXCNI4M/hLQ+IaQRpnpn2kIem1eJxjAaqaV/7SkHSbF9BeI2hmz16S/WMBLhKnUTL4S0PSbV7A3Q/v7jkW4MqhGgXTPtIQdaoCuvTG+zse++SevVYIaWS88pdGrNf9AqwQ0qgY/KUR63UjGSuENCoGf2nEet1fute3AqlM5vylCnRbI+jys052nSCNhMFfGiOHur2kVBaDvzRmun0rcGKYymTwlyaAJaAqmwO+0gSwBFRlM/hLE8ASUJXNtI80AVwkTmUr5co/IhZGxM0R8XBEbIuIt0fEsRFxZ0T8vPh5THFsRMTXI2J7RPw0Ik4row9SnblInMpWVtrna8APM/MNwFuAbcBa4K7MXAHcVWwDvBtYUfxZA1xTUh+k2jrcReKkbgZO+0TE0cA7gI8BZOYLwAsRsRp4Z3HY9cCPgc8Aq4HvZGYCPym+NZyYmbsG7YtUZ/0uEif1UsaV/3JgN/DtiNgaEddGxKuBE2YF9KeAE4rHU8COWc/fWbQdICLWRMTmiNi8e/fuErop1Y/LQehwlRH8jwBOA67JzFOB37E/xQNAcZWf/Zw0M9dn5nRmTi9atKiEbkr102uROO8LoF7KCP47gZ2ZeU+xfTOtD4NfRsSJAMXPp4v9M8CSWc9fXLRJ6lO3sQDAgWD1NHDOPzOfiogdEXFyZj4CrAIeKv6cD6wrft5WPGUjcHFEbADeBjxnvl86fJ3GAlau29R1INgyUEF5df4fB26IiCOBR4ELaH2ruCkiLgQeBz5UHHs78B5gO/B8caykEjkpTIdSSvDPzPuB6Q67VnU4NoGLynhdSZ31mhQmgcs7SLXUayBYApd3kGrJ+wLoUAz+Uk15XwD1YvCXGsT7AqjNnL/UIN4XQG0Gf6lBLAFVm2kfqUEOVQLqeEBzeOUvNcih1gJySYjmMPhLDdJtLaBzT51yPKBhTPtIDdOtBNTxgGbxyl8S4L0BmsbgLwlwSYimMe0jCXBJiKYx+Et6WbfxANWPaR9JaiCv/CUdkpO/6sfgL6knF4OrJ9M+knpy8lc9Gfwl9eTkr3oy+Evqyclf9WTwl9STk7/qyQFfST31mvxlFdDkMvhLOqROk7+sAppspn0kHRargCZbacE/IuZFxNaI+H6xvTwi7omI7RFxY0QcWbS/stjeXuxfVlYfJI2OVUCTrcwr/0uAbbO2vwRcnZmvB54FLizaLwSeLdqvLo6TNGGsAppspQT/iFgMvBe4ttgO4Ezg5uKQ64Fzi8eri22K/auK4yVNEKuAJltZA75fBT4NvLbYPg7Yk5n7iu2dQHsEaArYAZCZ+yLiueL4Z2afMCLWAGsAli5dWlI3JZXFKqDJNnDwj4j3AU9n5paIeOfAPSpk5npgPcD09HSWdV5J5bEKaHKVkfZZCZwTEb8ANtBK93wNWBgR7Q+XxcBM8XgGWAJQ7D8a+FUJ/ZA0BqwCmgwDB//MvCIzF2fmMuA8YFNm/h1wN/CB4rDzgduKxxuLbYr9mzLTK3upJqwCmgzDrPP/DHBZRGynldO/rmi/DjiuaL8MWDvEPkgaMauAJkOpM3wz88fAj4vHjwKndzjm98AHy3xdSePj8rNOPiDnD1YBjSOXd5BUKm8EPxkM/pJK543gx5/BX9LIWP8/Pgz+kkbC+v/x4qqekkbC+v/xYvCXNBLW/48Xg7+kkbD+f7wY/CWNhKuAjhcHfCWNhPX/48XgL2lketX/WwY6WgZ/SZWzDHT0zPlLqpxloKNn8JdUOctAR8/gL6lyloGOnsFfUuUsAx09B3wlVc4y0NEz+EsaC93KQC0BHQ6Dv6SxZQno8JjzlzS2LAEdHoO/pLFlCejwGPwljS1LQIfH4C9pbFkCOjwO+EoaW5aADo/BX9JYswR0OAZO+0TEkoi4OyIeiogHI+KSov3YiLgzIn5e/DymaI+I+HpEbI+In0bEaYP2QVKztEtAZ/bsJdlfAnrr1pmquzYxysj57wM+lZmnAGcAF0XEKcBa4K7MXAHcVWwDvBtYUfxZA1xTQh8kNYgloIMbOPhn5q7MvK94/FtgGzAFrAauLw67Hji3eLwa+E62/ARYGBEnDtoPSc1hCejgSq32iYhlwKnAPcAJmbmr2PUUcELxeArYMetpO4u2g8+1JiI2R8Tm3bt3l9lNSRPOEtDBlRb8I+I1wPeAT2bmb2bvy8wEsp/zZeb6zJzOzOlFixaV1U1JNWAJ6OBKqfaJiPm0Av8NmXlL0fzLiDgxM3cVaZ2ni/YZYMmspy8u2iRpTiwBHdzAwT8iArgO2JaZX5m1ayNwPrCu+HnbrPaLI2ID8DbguVnpIUmak143g9ehlXHlvxL4KPBARNxftH2WVtC/KSIuBB4HPlTsux14D7AdeB64oIQ+SJL6MHDwz8z/AqLL7lUdjk/gokFfV5J0+JzhK6lWnPk7NwZ/SbXhzV/mzlU9JdWGM3/nzuAvqTac+Tt3Bn9JteHM37kz+EuqDWf+zp0DvpJq41Azf60E2s/gL6lWet38xUqg/Uz7SGoEK4EOZPCX1AhWAh3I4C+pEawEOpDBX1IjWAl0IAd8JTWC9wA4kMFfUmN4D4D9TPtIUgN55S+p8Zo4+cvgL6nRmjr5y7SPpEZr6uQvg7+kRmvq5C+Dv6RGa+rkL4O/pEZr6uQvB3wlNVpTJ38Z/CU1XhMnf5n2kaQGqiz4R8TZEfFIRGyPiLVV9UOSmqiStE9EzAO+Cfw1sBO4NyI2ZuZDVfRHkjqp88zfqq78Twe2Z+ajmfkCsAFYXVFfJOlPtGf+zuzZS7J/5u+tW2eq7lopqgr+U8COWds7i7aXRcSaiNgcEZt379490s5JUt1n/o7tgG9mrs/M6cycXrRoUdXdkdQwdZ/5W1XwnwGWzNpeXLRJ0lio+8zfqoL/vcCKiFgeEUcC5wEbK+qLJP2Jus/8raTaJzP3RcTFwB3APOBbmflgFX2RpE7qPvM3MrPqPhzS9PR0bt68uepuSNJEiYgtmTndad/YDvhKkobHtX0kqU91mPxl8JekPtTlto+mfSSpD3WZ/GXwl6Q+1GXyl8FfkvpQl8lfBn9J6kNdJn854CtJfajL5C+DvyT1qQ63fTT4S1JJJqn+3+AvSSWYtPp/B3wlqQSTVv9v8JekEkxa/b/BX5JKMGn1/wZ/SSrBpNX/O+ArSSWYtPp/g78klWSS6v9N+0hSAxn8JamBDP6S1EDm/CVpBMZt6QeDvyQN2Tgu/WDaR5KGbByXfjD4S9KQjePSDwMF/4i4KiIejoifRsS/R8TCWfuuiIjtEfFIRJw1q/3som17RKwd5PUlaRKM49IPg1753wm8KTP/HPg/4AqAiDgFOA94I3A28E8RMS8i5gHfBN4NnAJ8pDhWkmprHJd+GGjANzN/NGvzJ8AHisergQ2Z+QfgsYjYDpxe7NuemY8CRMSG4tiHBumHJI2zcVz6ocxqn38AbiweT9H6MGjbWbQB7Dio/W2dThYRa4A1AEuXLi2xm5I0euO29MMhg39E/AfwZx12fS4zbyuO+RywD7ihrI5l5npgPcD09HSWdV5J0hyCf2a+q9f+iPgY8D5gVWa2g/QMsGTWYYuLNnq0S1LjVDX5a9Bqn7OBTwPnZObzs3ZtBM6LiFdGxHJgBfC/wL3AiohYHhFH0hoU3jhIHyRpUrUnf83s2Uuyf/LXrVuHf008aLXPN4DXAndGxP0R8c8AmfkgcBOtgdwfAhdl5kuZuQ+4GLgD2AbcVBwrSY1T5eSvQat9Xt9j3xeAL3Rovx24fZDXlaQ6qHLylzN8JakiVU7+MvhLUkWqnPzlqp6SVJEqJ38Z/CWpQlVN/jLtI0kNZPCXpAYy+EtSA5nzl6QxNOxlHwz+kjRmRnHPX9M+kjRmRrHsg8FfksbMKJZ9MPhL0pgZxbIPBn9JGjOjWPbBAV9JGjOjWPbB4C9JY2jYyz6Y9pGkBjL4S1IDGfwlqYEM/pLUQAZ/SWqgyMyq+3BIEbEbeHyAUxwPPFNSdyZF095z094v+J6bYpD3/LrMXNRpx0QE/0FFxObMnK66H6PUtPfctPcLvuemGNZ7Nu0jSQ1k8JekBmpK8F9fdQcq0LT33LT3C77nphjKe25Ezl+SdKCmXPlLkmYx+EtSA9U6+EfE2RHxSERsj4i1Vfdn2CJiSUTcHREPRcSDEXFJ1X0alYiYFxFbI+L7VfdlFCJiYUTcHBEPR8S2iHh71X0atoi4tPi9/llEfDciXlV1n8oWEd+KiKcj4mez2o6NiDsj4ufFz2PKeK3aBv+ImAd8E3g3cArwkYg4pdpeDd0+4FOZeQpwBnBRA95z2yXAtqo7MUJfA36YmW8A3kLN33tETAGfAKYz803APOC8ans1FP8CnH1Q21rgrsxcAdxVbA+stsEfOB3YnpmPZuYLwAZgdcV9GqrM3JWZ9xWPf0srIAxvQfAxERGLgfcC11bdl1GIiKOBdwDXAWTmC5m5p9JOjcYRwIKIOAI4Cniy4v6ULjP/E/j1Qc2rgeuLx9cD55bxWnUO/lPAjlnbO2lAIGyLiGXAqcA9FXdlFL4KfBr4Y8X9GJXlwG7g20Wq69qIeHXVnRqmzJwBvgw8AewCnsvMH1Xbq5E5ITN3FY+fAk4o46R1Dv6NFRGvAb4HfDIzf1N1f4YpIt4HPJ2ZW6ruywgdAZwGXJOZpwK/o6RUwLgq8tyraX3wnQS8OiL+vtpejV62avNLqc+vc/CfAZbM2l5ctNVaRMynFfhvyMxbqu7PCKwEzomIX9BK7Z0ZEf9abZeGbiewMzPb3+pupvVhUGfvAh7LzN2Z+SJwC/CXFfdpVH4ZEScCFD+fLuOkdQ7+9wIrImJ5RBxJa3BoY8V9GqqICFp54G2Z+ZWq+zMKmXlFZi7OzGW0/o03ZWatrwgz8ylgR0ScXDStAh6qsEuj8ARwRkQcVfyer6Lmg9yzbATOLx6fD9xWxklrewP3zNwXERcDd9CqDPhWZj5YcbeGbSXwUeCBiLi/aPtsZt5eXZc0JB8HbigubB4FLqi4P0OVmfdExM3AfbSq2rZSw6UeIuK7wDuB4yNiJ/B5YB1wU0RcSGtp+w+V8lou7yBJzVPntI8kqQuDvyQ1kMFfkhrI4C9JDWTwl6QGMvhLUgMZ/CWpgf4ftyiOCs1E0MQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "pa = -20\n",
    "pb = 90\n",
    "pc = 800\n",
    "\n",
    "t = np.linspace(0, 10) \n",
    "y = pa*t**2 + pb*t + pc + np.random.randn(np.size(t))*5\n",
    "\n",
    "\n",
    "plt.scatter(t, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 如何得到更新项?\n",
    "\n",
    "$$\n",
    "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n",
    "$$\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial L}{\\partial a} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i^2 \\\\\n",
    "\\frac{\\partial L}{\\partial b} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i \\\\\n",
    "\\frac{\\partial L}{\\partial c} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c)\n",
    "\\end{eqnarray}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 程序"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch    0: loss = 2.60133e+07, a =   -4.09558, b =    7.02074, c =    4.32858\n",
      "epoch  100: loss = 2.50396e+06, a =   -33.1825, b =    282.352, c =    192.658\n",
      "epoch  200: loss = 1.94167e+06, a =   -35.4465, b =    297.516, c =    270.663\n",
      "epoch  300: loss = 1.58138e+06, a =   -34.3201, b =    280.422, c =    328.378\n",
      "epoch  400: loss = 1.26769e+06, a =   -32.8956, b =     261.08, c =    378.356\n",
      "epoch  500: loss = 1.01347e+06, a =   -31.5673, b =    243.259, c =      422.8\n",
      "epoch  600: loss =     810004, a =   -30.3731, b =    227.263, c =    462.482\n",
      "epoch  700: loss =     647465, a =   -29.3051, b =    212.962, c =    497.935\n",
      "epoch  800: loss =     517653, a =   -28.3507, b =    200.183, c =    529.611\n",
      "epoch  900: loss =     413978, a =   -27.4979, b =    188.764, c =    557.914\n",
      "epoch 1000: loss =     331175, a =    -26.736, b =    178.562, c =    583.202\n",
      "epoch 1100: loss =     265038, a =   -26.0552, b =    169.446, c =    605.798\n",
      "epoch 1200: loss =     212210, a =   -25.4469, b =    161.301, c =    625.987\n",
      "epoch 1300: loss =     170010, a =   -24.9033, b =    154.023, c =    644.025\n",
      "epoch 1400: loss =     136297, a =   -24.4177, b =    147.521, c =    660.143\n",
      "epoch 1500: loss =     109363, a =   -23.9838, b =    141.711, c =    674.544\n",
      "epoch 1600: loss =    87842.3, a =   -23.5961, b =    136.519, c =    687.411\n",
      "epoch 1700: loss =    70645.3, a =   -23.2497, b =    131.881, c =    698.908\n",
      "epoch 1800: loss =    56901.8, a =   -22.9402, b =    127.737, c =     709.18\n",
      "epoch 1900: loss =      45917, a =   -22.6637, b =    124.034, c =    718.359\n",
      "epoch 2000: loss =    37135.8, a =   -22.4166, b =    120.725, c =     726.56\n",
      "epoch 2100: loss =    30115.2, a =   -22.1958, b =    117.769, c =    733.887\n",
      "epoch 2200: loss =    24501.2, a =   -21.9985, b =    115.127, c =    740.434\n",
      "epoch 2300: loss =      20011, a =   -21.8223, b =    112.767, c =    746.284\n",
      "epoch 2400: loss =    16419.1, a =   -21.6648, b =    110.659, c =    751.511\n",
      "epoch 2500: loss =    13544.9, a =   -21.5241, b =    108.774, c =    756.181\n",
      "epoch 2600: loss =    11244.5, a =   -21.3983, b =    107.091, c =    760.354\n",
      "epoch 2700: loss =    9402.78, a =    -21.286, b =    105.587, c =    764.082\n",
      "epoch 2800: loss =    7927.77, a =   -21.1856, b =    104.243, c =    767.414\n",
      "epoch 2900: loss =    6746.04, a =   -21.0959, b =    103.042, c =     770.39\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmc0lEQVR4nO3dfXyU5Z3v8c8vkAZHMSqPFUxCrW4VIqjQ6tH11RRawQfAPZVld+rRXbbTUx9W1m4RmbZie4a16Munqt2TU9yy67QstRbB6taKse5ZqwWsFfDhCEoiKBKoRjGYBvI7f8wdSMg9gZDJTGbm+3698mLua66572sC/Oaa6/5d12XujoiIFJeSXDdARESyT8FfRKQIKfiLiBQhBX8RkSKk4C8iUoQG5roBh2Po0KFeVVWV62aIiOSVdevW7XT3YWHP5UXwr6qqYu3atbluhohIXjGz+nTPadhHRKQIKfiLiBQhBX8RkSKUF2P+ItL3Wltb2bp1Kx9//HGumyI9NGjQIEaPHk1paelhv0bBX0QA2Lp1K4MHD6aqqgozy3Vz5DC5O7t27WLr1q2MGTPmsF+nYR/pucWLoa6uc1ldXapc8tbHH3/MkCFDFPjzjJkxZMiQHn9jU/AvNOkC80UXkVwWp+quKkpuKaHqriqSy+LpyxcvJrk+2bl8fTJ1vkmTYNasA9epq0sdT5qU3fcqGafAn5+O5O/N8mFJ54kTJ3pR5vkvXpwKqDU1B8rq6mDNGgCSFU3EtydpaGqgoryCxMgo0bqdJNf/hPj0CA0tjVSUDSOxshmmTye2+yc0dxgSjLTClYPOZenHv+1a3nIaS499g+a2lgPlJWXUDphJdMEyksvixNfcSsPgNio+LCExaT7RhvL07Z03r69+S5Ihr7zyCqeddlqumyFHKOzvz8zWufvEsPrq+WdTN73y0PLNm0nGp1N164hU7/vWESTj02HSJJIVTcQ2LKK+qR7HqW+qJ7ZhEVeftZ3YtFbqW3akylt2EJvWyvUlv+oU4AGaS6G27Xfh5ce81inwAzS3tRAvqSO5PknsjbuoP7YNN6g/to3YG3eRrGhK216RwzFgwAAmTJjAuHHjuPTSS3n//feP6Dw//vGPufbaaw9Zr6qqip07d3ZbZ9GiRUfUhv5OwT+b0g2XTJkSWp6sGRoayJND3ya+PRketN/9ZWjQ3rVnV2iT9vm+8HLaQssbWhqJr47T3Nrc+RqtzVzf8L/TtjftEJLkpz6673PUUUfx4osvsmHDBk444QTuu+++Xp0vExT8pfdqamD58lSg/853Un8uXw433EDyBzGqHp1CyUKj6tEpJH8QSwX4sN736jgNTQ2hl0gXzNMZYAN6VF5RXpH22rv27Apt7/UPf5XYijmdv6WsmENy0Wx9KOSrLNz3Offcc9m2bRsAmzdvZurUqZx99tn8+Z//Oa+++ioAq1at4nOf+xxnnnkmU6ZM4d133+32nLt27eJLX/oSY8eO5e/+7u/oOOw9c+ZMzj77bMaOHUttbS0A8+fPZ8+ePUyYMIFoNJq2Xj5S8O+NI+j9JIe+TdU1rZSUfI+qaw70isOGUeqbwpflaB/jDzMgzV/pECJEWjuXRVohVvLZ8PIRFxMpKetcXlJGYnKCCitP+/7C7GJP+IfCxyvSfihIP5euI9Pxfk8v7Nu3j9WrVzN9+nQAYrEYP/jBD1i3bh233347V199NQDnn38+zz33HL///e+ZPXs2iw/xzeOWW27h/PPPZ+PGjVx22WU0NBzoyDzwwAOsW7eOtWvXcs8997Br1y5uvfXW/d9Gkslk2nr5SMG/N9L1fjZvDv1QSC6anQp21pQK8tZEbMUcrn/8+tBhlHSBvKJsGImR0fCgvfvPQoP23Zs+Te24BVSWV2IYleWV1I5bwP3PHhde/sJIah8vpbJseKq8bDi1j5cS3XkiicvuJVIa6XyN0ghDBh7bo1/frgEt4d9s9vwybQaS9CM1NfD1r8P3vpf6MwOBv72XPXLkSN59912++MUvsnv3bp599lkuv/xyJkyYwNe+9jXeeecdIDU34cILL6S6uprbbruNjRs3dnv+Z555hq985SsAXHzxxRx//PH7n7vnnnsYP34855xzDm+99Ravv/566DkOt15/p0levdGx9/P1r8MPf5g6BpLx6V0ybuJTS2n2rsGueU9L2NnZ521EBpR1ybhJtNWkMmvGLeia7dNQznnTRu0fGqooryAxOUG0OvWVNUqi80VmQzSsvGEx0dkriXb8D/25VOZONMjcOfgarFpFrGRFl/Ye1TaAXXT+cOtO/cDdxDYs2n9Po76pntjORTBuAVEguT6Z9v1JFtXVpf7Nf/vbqT9ranr9AdDey25ububCCy/kvvvu46qrruK4447jxRdf7FL/uuuu44YbbmD69Ok8/fTTLFy48Iiu+/TTT/Pkk0/y29/+lkgkwuc///nQvPnDrZcPFPx7q2Pv59vfhpqa1DDOtFaaW3YABDc+y2hu2x1+DgdC0nQrS44jMfPebgJ5SNBuL+9tMAxLzezwnztaHe16jeoohARmgNiKOV0/FMqOCb0RPcAG0Fza+d5FcynEtydh0WZi+w58wLQPFbFqFdEFy3rzjqUn2r/ltg/11NRkdOgnEolwzz33MHPmTK6++mrGjBnDz372My6//HLcnZdeeonx48fT1NTEqFGjAFi6dOkhz3vBBRfwk5/8hG9961s8/vjjvPfeewA0NTVx/PHHE4lEePXVV3nuuef2v6a0tJTW1lZKS0u7rZdvNOxzKIcY108ui1P1UYKSm6Hqo0Qq/311PHQ4I91N1CGRIaHDKInL7iVaHWXL3C203dzGlrlb+n0PN6y90Z0nhg4h3V3xtdAhqnQ3rRuaGoiX1KVNQZUsWrOmc6Bv/xYczEHJhDPPPJMzzjiDn/70pySTSZYsWcL48eMZO3YsjzzyCAALFy7k8ssv5+yzz2bo0KGHPOfNN9/MM888w9ixY3n44YepqEjdO5s6dSp79+7ltNNOY/78+Zxzzjn7XxOLxTjjjDOIRqPd1ss3muR1KAf3cDocJ999stPwBKTG3Q9OwewoUhrpNL4fKY1Qe2kqY6BghzLSTVZbtix0Qlp8eoT64FtTR5XllTQ0NeB0/TdrGP/2F/9WuL/DLNAkr/zW00leCv7tuptN235jt+O4fk0NVd89nnp/v8upBrixz7r+XivLhpO4+A4FqHZpfufJR75H7Ji6Lh+qtcE9jrAsqCEcxZ6StrQzkuXQFPzzm2b4Hkq6YZzNm9PmLYelZwI0eFPoJfaZh6dJttXk3TBOn5o3r+v4cE0N0ROnhmYgRRvKSUxOhP5uGVja7YxkzSUQ6az4ev7dDOMAXXr4yaFvh96srJ25hPjqeGgvtLK8ksTkhHr4faGuLjST6ooLPwodDoLU35e+ERyaev75TcM+h6M94B80jJNcnyT+82tooIkKykn89/uI//KG8PHnYAgntioWOoavQN9H0gwVVf3nX4QPwVESulRFZdlwtszvfjZosVHwz28a9mnXXZZOyOSU5Ppk6ASssMAP0NCyg2h1lNpLazsPTyjw9600Q0WJy+4NzxzqZo0iDQdJMctI8DezfzCzjWa2wcx+amaDzGyMmT1vZpvM7N/N7BNB3bLgeFPwfFUm2tBFd2uPHDw5pa6O+C9vCE/PTDfLtrwSCE9tlOxLl05aWTY8tP4JR51AbFWs89ISq2L6AJCi0evgb2ajgL8HJrr7OGAAMBv4PnCnu38aeA+YE7xkDvBeUH5nUC/z0q09Qmr2bdU3SigZ8L+o+kYJyfh0Gj4O7+Hvoy08B39y18lVkkNr1hBNrGTL/HdTH8Tz3yWaWEmirSb8BvGePaFLasR/cehlgKXvtC/p3P5z6623pq27YsUKXn755f3H3/nOd3jyySd73Yb333+f+++/v8evW7hwIbfffvsh6x1zzDF9cv2eytSwz0DgKDMbCESAd4AvAA8Fzy8FZgaPZwTHBM9Ptr7aPihseOe34csOn2CR0FO0D+VoaKefS5c5dO7XQr8R/DHNchMN3pTaxUxrCx1SXwybtS/v0P4zf/78tHUPDv7f/e53mTJlSq/bkK3gm+vr9zr4u/s24HaggVTQbwLWAe+7+96g2lZgVPB4FPBW8Nq9Qf0hB5/XzGJmttbM1jY2Nh5Z48KGdyLPhQ7vcNRRaXv4GtrJY2m+EVTYcaHVK8or0m6Uk6wIT+0tRsn1yawOm82fP5/TTz+dM844g3/8x3/k2WefZeXKlXzzm99kwoQJbN68mauuuoqHHkr1N6uqqrjpppuYMGECEydO5IUXXuDCCy/k5JNP5p//+Z8B2L17N5MnT+ass86iurp6/6zh+fPns3nzZiZMmMA3v/lNAG677TYmTZrEGWecwc0337y/XYlEglNPPZXzzz+f1157LbTtb775Jueeey7V1dV861vf2l9+uNdPV6/X3L1XP8DxwFPAMKAUWAF8BdjUoc5JwIbg8QZgdIfnNgNDu7vG2Wef7T321FP+4LnHeOU/DXdbaF75T8P9wXOPcVtozkK6/NhC8wdfetAr76xM1b+z0h986cGeX1fywoMvPeiRRKTTv4FIIrL/30DYv5HKOytz3ew+9fLLLx923b76HZWUlPj48eP3/yxbtsx37tzpp556qre1tbm7+3vvvefu7ldeeaX/7Gc/2//ajseVlZV+//33u7v73Llzvbq62j/44APfsWOHDx8+3N3dW1tbvampyd3dGxsb/eSTT/a2tjZ/8803fezYsfvP+6tf/cq/+tWveltbm+/bt88vvvhi/81vfuNr1671cePG+UcffeRNTU1+8skn+2233dblPV166aW+dOlSd3e/9957/eijj+7R9dPVO1jY3x+w1tPE1Uws7DYFeNPdGwHM7GHgPOA4Mxvoqd79aGBbUH9b8GGwNRgmKgcyviB2+/DOwYurndB2VOgKkxXlFeGLlUlBilZHYePLB+1DPJdodZQrHr4i9DUNTQ1aUTSQbkOfdOWHq33Yp6O9e/cyaNAg5syZwyWXXMIll1xyWOdq3wugurqa3bt3M3jwYAYPHkxZWRnvv/8+Rx99NAsWLOCZZ56hpKSEbdu2hW4G88QTT/DEE09w5plnAqke++uvv86HH37IZZddRiQS6XS9g/3Xf/0XP//5zwG44ooruPHGG4FUx/twrp+u3siRIw/r95BOJsb8G4BzzCwSjN1PBl4G6oAvB3WuBNq/q6wMjgmefyr4hMqoIxnekSJSV0f0ulq2XPIkbQudLZc8SfS6WqirS7tRjjKEDkj3O0pX3hsDBw7kd7/7HV/+8pd59NFHmTp16mG9rqwsdaO/pKRk/+P2471795JMJmlsbGTdunW8+OKLjBgxInR5Znfnpptu2n8fYtOmTcyZM6dLve6E3dY83Osfbr2eysSY//Okbty+AKwPzlkL3AjcYGabSI3pLwlesgQYEpTfAKS/o9ML6Xogf9zzR93AlW5XpUy3UQ67dytDKJCYnMhaJ2r37t00NTVx0UUXceedd/KHP/wBgMGDB/Phhx8e8XmbmpoYPnw4paWl1NXVUV9fH3reCy+8kAceeIDdu1NLsm/bto0dO3ZwwQUXsGLFCvbs2cOHH37IqlWrQq9z3nnnsWxZajZ5+25gPbl+unq9lZH1/N39ZuDmg4rfAD4bUvdj4PJMXLc7FeUVoUsvaHhHgG73K4guXhy6Uc4Vr4Zv5J1ujadC1v7/J9NDYO07ebWbOnUq119/PTNmzODjjz/G3bnjjjsAmD17Nl/96le555579t/o7dF7iEa59NJLqa6uZuLEiXzmM58BYMiQIZx33nmMGzeOadOmcdttt/HKK69w7rnnAqlUzQcffJCzzjqLv/zLv2T8+PEMHz6cSWn2L7777rv567/+a77//e8zY8aMHl//xhtvDK3XWwW7vEN7NoKWXpBMqbqrKnwtJzuOxKlXh+6qFvoh009peYf8puUdAlp6QTIt3VDHRSMuUGqo5J2CDf6gpRcks6LVUWo/NZfKD0owh8oPSqj91Fwe2/OHLhv47N92UqSfKujgL5JRaTKE+irtMRfyYRhYujqSvzcFf5HDlSZDqMLKQ6tXlFfk1cqhgwYNYteuXfoAyDPuzq5duxg0aFCPXlewN3xFsiW5LB66l/OVLaex9Ng38mYjmdbWVrZu3ZqRHHLJrkGDBjF69GhKSzuPP3Z3wzcjqZ4ixSzaUB6aGhrfeE/41pKldfTHu0+lpaWMGTMm182QLFHPX6SPlNxSErq1pGG03Ry+yYxIJhVlqqdIrnW3BEI+3QuQwqTgL9JHEpMToRvJXHTKRVojSHJOwV+kj6TbWvKx5/5NawRJzumGr0hfCTaSiXbcYexzdVzxmy+EVi/GNYIkdxT8RfpKmsXjKv5QmXbRQRYvJlnRlPfrBEn/p2EfkSzrbjlkbSEp2aLgL5Jl6dYIilZHiW9Pap0gyQoN+4hkW7BGUHT5k6klIurqYNYsGDGloNYJkv5NPX+RbOtmF7Fsbo8oxU3BXyTb5s07EPjb1dTAvHlpt5BMjIxqYphklIK/SD8SbSindtyCzpsQjVsAv3yU2Io5nW8Er5hDctHsXDdZ8pTW9hHJA1W3jqC+ZUeX8sqy4WyZ/24OWiT5QGv7iOS5hpbGHpWLHIqCv0ge0I1gyTQFf5E8kG6RuETzOSSXxTvfCF4Wh8WLc9RSyRcK/iJ5IN0icZQM0IxgOSIZmeRlZscBPwLGAQ78LfAa8O9AFbAFmOXu75mZAXcDFwHNwFXu/kIm2iFSsNIsElf1n3+RdkZwlER22yh5JVM9/7uB/3D3zwDjgVeA+cBqdz8FWB0cA0wDTgl+YsAPM9QGkcKVZm5AupVANSNYDqXXwd/MyoELgCUA7v4nd38fmAEsDaotBWYGj2cA/+opzwHHmdkne9sOkWKkG8FypDLR8x8DNAL/Yma/N7MfmdnRwAh3fyeosx0YETweBbzV4fVbg7JOzCxmZmvNbG1jo9LZRMJoRrAcqUwE/4HAWcAP3f1M4CMODPEA4KmZZD2aTebute4+0d0nDhs2LAPNFCk8mhEsRyoTN3y3Alvd/fng+CFSwf9dM/uku78TDOu0T0/cBpzU4fWjgzIR6al584hCl5u7VVt+RHNLS6ey5rYW4qV1RLPYPOm/et3zd/ftwFtm9mdB0WTgZWAlcGVQdiXwSPB4JfA/LOUcoKnD8JCIZIBmBMuhZGo9/+uApJl9AngD+BtSHyzLzWwOUA/MCuo+RirNcxOpVM+/yVAbRCRQUV6RfqtIETIU/N39RSBs8aDJIXUduCYT1xWRcInJCWIr5tDcdmDoJ1JSRmKycv8lRTN8RQpQuhnB0Z0nKgtIAC3pLFKYFi+GSZM6TwyrqyP5wFxip77W5RtB7YCZRBcsy0FDpS91t6Szgr9IEdG+AMVF6/mLCKAsIDlAwV+kiGg5CGmn4C9SRNLuC6AsoKKTqTx/EckD0Z0nwuOlxKeX09DSSEXZMBIrm4mefyIsXkyyoon49iQNTQ1UlFeQGBkl2lCeWlVUCoqCv0gxSbMvAGvWkKxoIrZh0f79Aeqb6ontXATjFmhJiAKkbB8RAaDqrqrQWcGV5ZVsmbsl+w2SXlO2j4gcUroNYLQxTGFS8BcRQJlAxUbBX0SA7jeGkcKj4C8iQPqNYaIN5VoPqADphq+IdCu5PklsVYzm1ub9ZZHSCLWX1hKt1reC/kw3fEXkiMVXxzsFfoDm1mbiq+M5apFkgoK/iHSrIST9s7tyyQ8K/iLSrYqy4T0ql/yg4C8i3UpcfEf4ekAX35GjFkkmKPiLSLei1VFqZy6h0ssxh0ovp3bmEt3szXMK/iJySNGdJ7LlvlLa2r7NlvtS20ECSgHNY1rYTUS6V1cHs2bB8uWpbSFramDWLJLRamJDnt2/JWR9Uz2xFXNg1SptCZkH1PMXke6tWXMg8EPqz+XLiQ9e02kvYIDmthbiJXU5aKT0lHr+ItK9sLX8a2poeOaj0OraEjI/ZKznb2YDzOz3ZvZocDzGzJ43s01m9u9m9omgvCw43hQ8X5WpNohI9mghuPyWyWGf64FXOhx/H7jT3T8NvAfMCcrnAO8F5XcG9UQkz2hLyPyWkeBvZqOBi4EfBccGfAF4KKiyFJgZPJ4RHBM8PzmoLyJ5JLrzRGofL6WybHhqIbiy4dQ+nsoEUhZQ/5epMf+7gHnA4OB4CPC+u+8NjrcCo4LHo4C3ANx9r5k1BfV3djyhmcWAGEBFhb5GivQ7abaETD4wl9iprykLqJ/rdc/fzC4Bdrj7ugy0Zz93r3X3ie4+cdiwYZk8tYhkwrx5BzKA2tXUEB+7XVlAeSATPf/zgOlmdhEwCDgWuBs4zswGBr3/0cC2oP424CRgq5kNBMqBXRloh4j0A+myfZQF1L/0uufv7je5+2h3rwJmA0+5exSoA74cVLsSeCR4vDI4Jnj+Kc+HTQVE5LAoCyg/9OUkrxuBG8xsE6kx/SVB+RJgSFB+AzC/D9sgIlmmLKD8kNFJXu7+NPB08PgN4LMhdT4GLs/kdUWk/4juPBEeLyU+vZyGlkYqyoaRWNlM9PwTc9006UAzfEUks9JkAbFmDcmhbxNfHaehqYGK8goSkxNaHTRHFPxFJLPSLAeRHPp2p72A65vqia2KAegDIAe0sJuIZEX8F9eG7wX8i2tz1KLipuAvIlnR4E09Kpe+peAvIlmhFND+RcFfRLIiMTlBpDTSqSxSGlEKaI4o+ItIVkSro9R+ai6VH5Sk9gL+oITaT83Vzd4cUbaPiGRHXR3R62qJLn8ytSZQ+/aQI6Z0XSNI+px6/iKSHWm2g2TNGkCbwWeb5cOyOhMnTvS1a9fmuhki0keSi2YT27ei02qgkZIyagfM1DLQvWBm69x9Ythz6vmLSM7FS+q0DHSWKfiLSM5pGejsU/AXkZzTHIDsU/AXkZzTMtDZp+AvIjnX3Wbw0jeU5y8iuadloLNOqZ4i0m8pBbR3lOopInlJKaB9R8FfRPotpYD2HQV/Eem3lALadxT8RaTfUgpo31HwF5F+SymgfUepniLSf3WTAqploHtHwV9E+q9587qW1dSk8v+XxYlvTx7I/x8ZJdpQHv4a6aLXwz5mdpKZ1ZnZy2a20cyuD8pPMLNfm9nrwZ/HB+VmZveY2SYze8nMzuptG0SkuCQrmohtWER9Uz2OU99UT2zDIpIV2gz+cGVizH8v8A13Px04B7jGzE4H5gOr3f0UYHVwDDANOCX4iQE/zEAbRKSIxLcnaS7tXNZcmiqXw9Pr4O/u77j7C8HjD4FXgFHADGBpUG0pMDN4PAP4V095DjjOzD7Z23aISPFoaGroUbl0ldFsHzOrAs4EngdGuPs7wVPbgRHB41HAWx1etjUoO/hcMTNba2ZrGxs1oUNEDlD+f+9lLPib2THAz4G57v5Bx+c8tYBQjxYRcvdad5/o7hOHDRuWqWaKSAFIjIwSae1cFmlNlcvhyUjwN7NSUoE/6e4PB8Xvtg/nBH/uCMq3ASd1ePnooExE5LBEG8qpHbeAyvLKVP5/eSW14xaksn3ksPQ61dPMDFgCvOLud3R4aiVwJXBr8OcjHcqvNbNlwOeApg7DQyIihzZvHlEgimb6HqlM9PzPA64AvmBmLwY/F5EK+l80s9eBKcExwGPAG8Am4P8AV2egDSIiACTXJ6m6q4qSW0qouquK5HplAIXpdc/f3f8vYGmenhxS34FrentdEZGDJdcnia2K0dzaDJDK/18VA9AGMAfR2j4iUjDiv7h2f+Bv19zaTPwX1+aoRf2Xgr+IFIwGD5/hm668mCn4i0jBUP7/4VPwF5GCkZicIFIa6VQWKY1o/f8QCv4iUjCi1VFqPzWXyg9KMIfKD0qo/dRc3ewNoSWdRaRw1NURva6W6PInU0s/19XBrFkwYorW/z+Iev4iUjjWrIHlyw8E+pqa1PGaNbB4cerDoKO6ulR5EVLPX0QKR7rNX4JvAcn4dOLTIzS0NFJRNozEymaiiZXZb2c/oOAvIkUhOfRtYtNaaW5JLTNW37KD2LQyGPo2xXhHQMM+IlIU4qvjNLe1dCprbmshvjqeoxblloK/iBQFbQDTmYK/iBSFirLwfUHSlRc6BX8RKQqJthoiJWWdyiIlZSTaijMFVMFfRIpCdMEyamcu6bwBzMwlRBcsy3XTcsJSKyz3bxMnTvS1a9fmuhkiInnFzNa5+8Sw55TqKSLFbfFikhVNxLcnaWhqoKK8gsTIaGpLyLB5AwVCwz4iUtSSFU3ENiyivqkex1MbwGxYRLKisJeBVvAXkaIW356kubRzWXNpqryQKfiLSFEr1vx/BX8RKWrFugGMgr+IFLXEyCiR1s5lkdZUeSFT8BeRohZtKKd23ILO+f/jFqSyfQqY8vxFRApUd3n+Oev5m9lUM3vNzDaZ2fxctUNEpBjlJPib2QDgPmAacDrwV2Z2ei7aIiKSTnJ9kqq7qii5pYSqu6pIri+c9M9c9fw/C2xy9zfc/U/AMmBGjtoiItJFctFsYivmdJ78tWIOyUWzc920jMhV8B8FvNXheGtQJiLSL8RL6sI3fympS/OK/NJvs33MLGZma81sbWNjY66bIyJFpqElPO6kK883uQr+24CTOhyPDsr2c/dad5/o7hOHDSvOzRZEJHcKffJXroL/GuAUMxtjZp8AZgMrc9QWEZEuEpMT4Zu/TE7kqEWZlZPg7+57gWuBXwGvAMvdfWMu2iIiEia680RqHy+lsmx4avJX2XBqHy8luvPEXDctI3K2nr+7PwY8lqvri4h0a80aoomVRGs6bPP4uTpYswZq8n/rR83wFREpUP1yhq+IiOSOtnEUEemJAtn2UT1/EZEeKJRtHxX8RUR6oFC2fVTwFxHpgULZ9lHBX0SkBwpl5q+Cv4hIDxTKto8K/iIiPVAo2z5qkpeISIHSJC8Rkb62eDHUHbTWf11dqrwf0iQvEZFMmDSJZHw68ekRGloaqSgbRmJlM9FE/1ywWMFfRCQDkkPfJjatleaWHQDUt+wgNq0Mhr5Nf7wVrGEfEZEMiK+Oh2/7uDqeoxZ1T8FfRCQD8m3yl4K/iEgGVJSFbzebrjzXFPxFRDIg0VYTvu1jW//c+EXBX0QkA6ILllE7c0nnyV8zlxBdsCzXTQulSV4iIgVKk7xERKQTBX8RkSKkSV4iIn2tH279qJ6/iEgf649bPyr4i4j0sf649WOvgr+Z3WZmr5rZS2b2CzM7rsNzN5nZJjN7zcwu7FA+NSjbZGbze3N9EZF80B9n//a25/9rYJy7nwH8P+AmADM7HZgNjAWmAveb2QAzGwDcB0wDTgf+KqgrIlKw+uPWj70K/u7+hLvvDQ6fA0YHj2cAy9y9xd3fBDYBnw1+Nrn7G+7+J2BZUFdEpGD1x60fMznm/7fA48HjUcBbHZ7bGpSlK+/CzGJmttbM1jY2NmawmSIi2dUft348ZKqnmT0JjAx5Ku7ujwR14sBeIGN3L9y9FqiF1AzfTJ1XRCTr5s0jCkRJ5Lol+x0y+Lv7lO6eN7OrgEuAyX5grYhtwEkdqo0OyuimXEREsqS32T5TgXnAdHdv7vDUSmC2mZWZ2RjgFOB3wBrgFDMbY2afIHVTuH/ucSYi0tdyuO9vb2f43guUAb82M4Dn3P1/uvtGM1sOvExqOOgad98HYGbXAr8CBgAPuPvGXrZBRCQ/5XDf314Ff3f/dDfPJaDrAJe7PwY81pvriogUglzu+6sZviIiOZLLfX8V/EVEciSXM38V/EVEciSX+/4q+IuI5Egu9/1V8BcRyZFc7vurPXxFRAqU9vAVEZFOFPxFRPqbLMz8VfAXEelvJk2CWbMOfADU1aWOJ03K2CUU/EVE+puaGpI/iFH16BRKFhpVj04h+YMY1GQuC0jBX0Skn0muTxJ74y7qj23DDeqPbSP2xl0k12duz18FfxGRfia+Ok5za3OnsubW5owu+6DgLyLSz2Rj2QcFfxGRfqbCwrd3TFd+JBT8RUT6mcRl9xIpjXQqi5RGSFx2b8auoeAvItLPRKuj1F5a23nZh0triVZnbpV/Le8gIlKgtLyDiIh0ouAvIlKEFPxFRIqQgr+ISBFS8BcRKUJ5ke1jZo1AfS9OMRTYmaHm5Itie8/F9n5B77lY9OY9V7p76IbAeRH8e8vM1qZLdypUxfaei+39gt5zseir96xhHxGRIqTgLyJShIol+NfmugE5UGzvudjeL+g9F4s+ec9FMeYvIiKdFUvPX0REOlDwFxEpQgUd/M1sqpm9ZmabzGx+rtvT18zsJDOrM7OXzWyjmV2f6zZli5kNMLPfm9mjuW5LNpjZcWb2kJm9amavmNm5uW5TXzOzfwj+XW8ws5+a2aBctynTzOwBM9thZhs6lJ1gZr82s9eDP4/PxLUKNvib2QDgPmAacDrwV2Z2em5b1ef2At9w99OBc4BriuA9t7seeCXXjciiu4H/cPfPAOMp8PduZqOAvwcmuvs4YAAwO7et6hM/BqYeVDYfWO3upwCrg+NeK9jgD3wW2OTub7j7n4BlwIwct6lPufs77v5C8PhDUgFhVG5b1ffMbDRwMfCjXLclG8ysHLgAWALg7n9y9/dz2qjsGAgcZWYDgQjwdo7bk3Hu/gzwx4OKZwBLg8dLgZmZuFYhB/9RwFsdjrdSBIGwnZlVAWcCz+e4KdlwFzAPaMtxO7JlDNAI/Esw1PUjMzs6143qS+6+DbgdaADeAZrc/YnctiprRrj7O8Hj7cCITJy0kIN/0TKzY4CfA3Pd/YNct6cvmdklwA53X5frtmTRQOAs4IfufibwERkaCuivgnHuGaQ++E4Ejjazr+S2Vdnnqdz8jOTnF3Lw3wac1OF4dFBW0MyslFTgT7r7w7luTxacB0w3sy2khva+YGYP5rZJfW4rsNXd27/VPUTqw6CQTQHedPdGd28FHgb+W47blC3vmtknAYI/d2TipIUc/NcAp5jZGDP7BKmbQytz3KY+ZWZGahz4FXe/I9ftyQZ3v8ndR7t7Fam/46fcvaB7hO6+HXjLzP4sKJoMvJzDJmVDA3COmUWCf+eTKfCb3B2sBK4MHl8JPJKJkw7MxEn6I3ffa2bXAr8ilRnwgLtvzHGz+tp5wBXAejN7MShb4O6P5a5J0keuA5JBx+YN4G9y3J4+5e7Pm9lDwAukstp+TwEu9WBmPwU+Dww1s63AzcCtwHIzm0NqaftZGbmWlncQESk+hTzsIyIiaSj4i4gUIQV/EZEipOAvIlKEFPxFRIqQgr+ISBFS8BcRKUL/H+EOooBJSBGJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_epoch = 3000              # epoch size\n",
    "a, b, c = 1.0, 1.0, 1.0    # initial parameters\n",
    "epsilon = 0.0001            # learning rate\n",
    "\n",
    "N = np.size(t)\n",
    "\n",
    "for i in range(n_epoch):\n",
    "    for j in range(N):\n",
    "        a = a + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)*t[j]**2\n",
    "        b = b + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)*t[j]\n",
    "        c = c + epsilon*2*(y[j] - a*t[j]**2 - b*t[j] - c)\n",
    "\n",
    "    L = 0\n",
    "    for j in range(N):\n",
    "        L = L + (y[j] - a*t[j]**2 - b*t[j] - c)**2\n",
    "    \n",
    "    if i % 200 == 0:\n",
    "        print(\"epoch %4d: loss = %10g, a = %10g, b = %10g, c = %10g\" % (i, L, a, b, c))\n",
    "    \n",
    "    \n",
    "y_est = a*t**2 + b*t + c \n",
    "\n",
    "\n",
    "plt.plot(t, y, 'rx', label='Real data')\n",
    "plt.plot(t, y_est, 'go', label='Estimated data')\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. 如何使用sklearn求解线性问题？\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  (100, 1)\n",
      "Y:  (100, 1)\n",
      "a = 3.376138, b = 0.051810\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmFUlEQVR4nO3de5zUVf3H8ddhQQHNthRJFnBJjCJN0c3ULVOslLywmeIlyVuieUMxCn5qmpquqYukqJBYeAVEXEAsvKCRt3K5iYqIogIrCoqAF4SFPb8/ziDL7uzud2a+15n38/Hwwc6wM98zM/L5nvl8P+d8jLUWERFJnjZRD0BERLKjAC4iklAK4CIiCaUALiKSUArgIiIJ1TbMg+2yyy62tLQ0zEOKiCTe7NmzP7TWdmp8f6gBvLS0lJqamjAPKSKSeMaYd9PdrxSKiEhCKYCLiCSUAriISEIpgIuIJJQCuIhIQoVahSIikq+q59Zy44xFvLdmPV2KOzD0iF5U9CkJ9JgK4CIiOaqeW8vwyQtYX7cZgNo16xk+eQFAoEFcKRQRkRzdOGPRl8F7i/V1m7lxxqJAj6sALiKSo/fWrM/ofr8ogIuI5KhLcYeM7veLAriISI6GHtGLDu2KtrmvQ7sihh7RK9Dj6iKmiEiOtlyoVBWKiBSsKErx/FLRpyT0sSqAi0gsRFWKl2TKgYtILERVipdkCuAiEgtRleIlmQK4iMRCVKV4SaYALiKxEFUpXpLpIqaIxEJUpXhJ5jmAG2OKgBqg1lp7tDGmBzAe2BmYDQy01m4MZpgiUgiiKMVLskxSKIOBhQ1u3wCMsNb2BD4GzvJzYCIiYaqeW0t55Ux6DJtOeeVMqufWRj2kVnkK4MaYrsBRwF2p2wboC0xK/co4oCKA8YkIyQwuSbKlBr12zXosW2vQ4/4+e02h3AL8HvhK6vbOwBpr7abU7eVA2u89xphBwCCA7t27Zz1QkUKlBS7Ba6kGveF7HLeVoq3OwI0xRwMrrbWzszmAtXaMtbbMWlvWqVOnbJ5CpKBpgUvwvNSgx3GW7mUGXg4ca4z5OdAe2AkYCRQbY9qmZuFdgXh/1xBJKC1wyY2XWXOX4g7Upnk/G9age52lh6nVGbi1dri1tqu1thQ4CZhprf0V8DRwfOrXTgOmBDZKkQKmBS7Z8zpr9lKDHscTaS4Lef4ADDHGvInLiY/1Z0gi0pAWuGTPa/qpok8J1x+3NyXFHTBASXEHrj9u721m1nE8kWa0kMda+wzwTOrnJcAB/g9JRBrSApfsZTJrbq0GfegRvba5mAzRn0i1ElMkAeK6wCVuVRmNecltexXHE6kCuIhkJQnljX7PmuN2IlUAF5GshF2Vkc1sP46zZj8pgItIVsKsysh2th/3FE+utJ2siGQlzKqMbBYzxXHhjd8UwEUkK2GWN2Yz249yBWtYe9cohSIiWQkzv5xNNUlUC2/CvLirAC4iWQurKiObahI/SwgzEebFXaVQRCT2vKyUbCyqFaxhzvw1AxeRRMh0th92CeGWihfbzN8HMfNXABeRvBVWiqdx3ruxoGb+CuAikrV8r7P2Kl3ee4uSAN8XBXARyUoSltKHpbn8tgGeG9Y3sOPqIqaIZEWdgraKaqtZBXARyUocGxxEJaqKFwVwEclKHBscRCWbMkc/KAcuIlmJY4ODKEWx1awCuIhkJd+3ak0CBXARyVrcGhzE0quvwt//DpWV0NbfkKscuIiI36yFJ56Afv1gr73g9tth/nzfD6MZuIgUpEAWIW3YAA8+CFVVsGABdO4M11wD554Lu+ziz8AbUAAXkYLj+yKkjz6CO+6A226DDz5ws+6774ZTToHtt/dz6NtQCkVECo5vi5DeeAN++1vo1g2uuAL23RcefxxefhnOOCPQ4A2agYtIAcppEZK1MGsW3HwzPPootGsHp54Kl1ziZt4hUgAXkYKTVbOHujp46CEXuOfMcTntK66A885zue4IKIUiIgUno6Xva9bAX/4CPXrAr34Fn30Go0fD0qXwpz9FFrxBM3ARKUCeFiEtWQIjR8LYsS5oH3aYC9z9+kGbeMx9FcBFJG9kUhrY7CKk5593ZYCPPOIC9cknu/x2nz4Bjz5zCuAikhdyKg3ctMkF7KoqePFFKC6G3/8eLrgASjIrKwyzyYUCuMSWur1IJrLqBv/JJy5FMnIkvPMO7LEH3HornH467LhjxmMIu8mFArjEkrq9bKUTmTcZlQYuXeoC9ZgxsG4d/PCHbvZ97LFQVNT09z3K6iSSg1YDuDGmPTAL2D71+5OstVcaY3oA44GdgdnAQGvtRt9HKAUp7H8IcaUTmXeeSgNralygnjjR3T7+eBgyBA44oMXn9noSDbvJhZdLqRuAvtbafYB9gSONMQcCNwAjrLU9gY+BswIZoRQkdXtx1LbMu2ZLA3/SE6ZMgUMOge9/3y2+GTwY3noLxo/3FLyHT15A7Zr1WLaeRKvn1jb53bCbXLQawK3zaepmu9R/FugLTErdPw6oCGKAUpiS1u2lem4t5ZUz6TFsOuWVM9P+486GTmTeNe6Ks0dHGL9pDhUnHw4VFfDuu24RzvLl7s/dd/f0vJmcRMNureYpB26MKcKlSXoCo4C3gDXW2k2pX1kO6Puc+CZJ3V6CTHNktWKwgFX0KaGis4FRo2DknbB6tZthT5gAxx3naT/uxumSdO8/pD+Jht3kwlMAt9ZuBvY1xhQDjwDf9noAY8wgYBBA9+7dsxiiFKKou71kcuGwuRnaxRPmceOMRTmNO0knssjNn+/y2w8+6MoCf/ELl98++GAwxtNTpDsZG1zKobHmTqJhNrnIqArFWrvGGPM0cBBQbIxpm5qFdwXSfme01o4BxgCUlZWlex9E0oqq20umM+qW0hm5zsajPpHFXn09/OtfLnA/9RTssIPbe3vwYFcSmKF0J2MLTYJ4XE6iXqpQOgF1qeDdAfgp7gLm08DxuEqU04ApQQ5UJCyZVsC09DW7tcd6obZlaaxfD/fdByNGwMKF0KWLa1k2aBB87WtZP21zJ2OL6zQft5Oolxn4bsC4VB68DTDRWvuoMeY1YLwx5lpgLjA2wHGKhCbTC4fp0hxeHysZWrnStSe7/XZYtcrtv33vvTBgAGy3Xc5P39zJuKS4A88N65vz8/ut1QBurX0ZaLIJgLV2CdBy/Y1IAmV64bBhmqO5mbguOubotdfcbPvee13bsqOPdvntQw/1nN/2ImnXHOKxpZZIjGRTClbRp4TnhvXllhP3DbWMLK9ZC08+CT//OXz3uy5lcvrpLmUybZrbHdDH4A1NSxFLijtw/XF7xyJdko6W0os0ksuFQ1109MGGDW6BTVWVa022665w9dWudVkAjYEba+maQ9y2NTDWhlcYUlZWZmtqakI7nogkyEcfuf22b7sNVqxws+4hQ1xj4Pbtox5dk+okcN+uwpihG2NmW2vLGt+vFIqIRGvxYjj/fNcY+LLLYO+9YcYMWLAAzjwzFsEb4rmtgVIoIiGK21fwyFgL//mPW9I+bZprDPyrX7kZdyuNgaN6D+O4rYECuEhItLMgrjHwpEkuv11TAzvv7Gbd558P3/hGqw+P8j2M47YGSqFIJILa/CnO4vgVPDRr1sBNN7nVkaec4vbgvvNOty/3Ndd4Ct4Q7XsY9kZVXmgGLqEr1Jlorl/BE5l+efvtrY2BP/3Ulf7dfrsrDcyiMXCUaYxsKoyC/swUwCV0hdqsIZev4Ik76b3wgkuTTJ7sAvVJJ7nGwPvtl9PTRp3GyGRbgzA+M6VQJHRxvBgUhly+gici/bJ5s8tvH3yw++/JJ2HoUDcLv/fenIM3xDON0ZwwPjPNwCV0Uc+iopLLIp9Yn/Q++QTuvtulSt5+G775TfjrX+GMM7JqDNySJC2UCuMzUwCX0CVtvwk/ZbuzYCxPesuXu0A9ZgysXQvl5e5CZf/+OTUGbk1SdmcM4zNTCkVCl7T9JuJg6BG9aFe07b4f7YpMNCe92bNdzXaPHq6O+4gj4MUX4dlnXdebAIN3koSR7tEMXCKRlFlUrnytQmi860WY7VHq610z4Koq+Pe/4StfgQsvhIsugtLSEAfiv6AqRcJI9yiAiwTEzyqEG2csoq5+24hdV2+Dr9z5/HMYN85t5bp4sVvuftNN8JvfwFe/GtxxPco1+AZdKRL0REUpFJGA+FmFEPpFzBUr4PLLXcA+7zwoLnY7BC5ZApdeGpvgPXzyAmrXrMeyNfhmsigsEdU9LdAMXCQgfgbd0C5ivvyym20/8IBb9t6/vwvY5eW+7b3tV8rCj/UEsa7u8UAzcJGANBdcswm6gV4Qs9Y1Bv7pT2GffWDiRDj7bHjjDXjkEfjhD30N3rnOmrfwI/j6+RlFQQFcJCB+Bt1AKne++MItcd9rL+jXz7Utu/56WLbM7cnds2f2z90MP1MWfgTfJC0MSkcpFJGA+F2F4NsFsVWr3H4ko0a5n/fZB+65B0480ZfGwC3xM2Xhx3qCJC0MSkcBXCRAsSqXXLjQ5bfvuce1LTvqKLf/dgC9JZvjZy7fr+Abq88oQwrgMZHIneakidh9jtbCzJmufvuxx1x3m9NOg4svhu98J/Th+L0KN8nB1w8K4DGQuJ3mJK1YfY4bN25tDDx/vmsM/Kc/ucbAnTqFO5YGkp6yiBsF8BhI6vaqsZttRiwWn+Pq1a4x8K23ulru3r3hrrvc0veY9JYs9FmznxTAYyCJtaixmm3GRKSf45tvwi23sGns3bT9Yj3/Ke3DI6dfxCEXnkrFfl2DP75EQgE8BuK401xrs+tsZ5v5PGsP/XO01m0gVVUFU6ZQX9SWqb0PZXRZfxZ1KgXgn4+8AsYU7GeS71QHHgNxq0X1stgim9mmn4s44ii0z3HTJpffPuAAOOQQmDULLruMiqH3M6Tf4C+DN7ReY53vn0m+UwCPgbhtr+plsUU2iyiSvu9EawL/HNeuddu37rEHnHyyawx8xx1u4c0117CgvmPah7V0Us33zyTfKYUSE3G6sONldp1NOVgSc/2ZCuRzfOcd1zjhrrtc95sf/9itlDzqqG0aA2eTwonTZ6JUTuY0A5cmvMyus5ltJn3fidZUz62lvHImPYZNp7xyZu5piP/+FwYMcDPuW2+FY46Bmhp45hn3c6Ou7tmkcOLymSiVkx0FcGnCayCo6FPCc8P68nblUTw3rG+rs6W45fr95FsA2rzZdXIvL4cDD4THH4ff/c5t43r//bD//s0+NJuTalw+E6VysqMUijQR1GKLfF7EkXMN+KefusbAt9ziGgP36OGaBJ9xhut+00hz6YZMUzhx+UzilMpJklYDuDGmG3AP0BnXxGmMtXakMebrwASgFHgHGGCt/Ti4oUqYgsrJxynXn43mAmfWAWj5cpceGT3aXaQ8+GC48UaoqGi2t6TfNfhx+EziWEqbBF5SKJuAS621vYEDgfONMb2BYcBT1to9gadSt0XyVktpkoxzyXPmwKmnupn2TTfBz34GL7wAzz0Hv/xli42B8zHdEJdUTtK0GsCttSustXNSP38CLARKgP7AuNSvjQMqAhqjSCy0FDg9BaD6epg2ze3+t//+MGUKXHABvPWWa6Jw4IGexpGP6Ya4ldImRUY5cGNMKdAH+C/Q2Vq7IvVX7+NSLOkeMwgYBNC9e/esByoStZYCZ4u55M8/d1u4jhjhutx07erSJGefnVVvyXxNN8QhlZM0ngO4MWZH4GHgYmvtOtNg/2BrrTXG2HSPs9aOAcYAlJWVpf0dkSRoLXA2CUDvvw9XXOEW23z0kZt1P/AAHH88tGuX9Tj83pI1XxRiHbmnMkJjTDtc8L7fWjs5dfcHxpjdUn+/G7AymCGKxIPnPO2CBXDmmbD77vDnP7uekrNmwUsvuRWUOQRvULohnUKtI/dShWKAscBCa21Vg7+aCpwGVKb+nBLICEViosU0ibWuZruqyv3ZsaNLkQweDHvuGchYCjlgNxaLrXwj4CWFUg4MBBYYY+al7vs/XOCeaIw5C3gXGBDICEVipEng/OILV79dVQWvvgq77QbXXQfnnANf/3p0Ay0w+Xhh14tWA7i19lmguYZ5h/s7HJGEWLUK7rzT7UmyciV873swbhycdFLgjYGlqXy9sNsaLaUXycTrr7vZdffu8Mc/QlkZPPkkzJsHv/61gndEDvt2pyazzEK4sKul9CKtsdZtIHXzzTB9Omy/vQvWF1/sWpZJpKrn1vLw7FoalrgZ4Jf75/91AgVw8V3elHNt3AgTJrj89rx5rhnwVVe5xsC77hr16IA8eq9zkO4CpgWefn1VNAMKkQK4+CovemV+/PHWxsDvvQff+Q787W9u6XtMGgND8t9rv04+hXoBE5QDz3u+71HdikTv0/HWW3DhhW6l5PDhLj3y2GPwyivwm9/EKnhDst9rP+u247KneRQUwPNYFIsbwpwN+XJy2tIY+LjjXL326NFwwgkwfz488QT069ekcUJcJHnm6efJp5A3worn/5niiyhmaGHNhnI+OW3a5PLbBx4IP/qRu0g5fDi8+y784x+uLDDmkjjz3HLSTVfyB9mdfAp5Zapy4HksihlaWPt0ZL3ybt0611ty5EhYuhR69oRRo+C002CHHXwdY9CStidK45x9OtmefAp1ZaoCeB6LYnFDWB1eMj45vfuuawz8t7+5xsCHHOIuUh59dGxTJK2JSzcdr9KddBuK88knrhTA81hUM7RcZkNeKxM8n5z+9z9XBjhpkrt94olwySVuAU4eSNLMs6VvfiUxP/nElQJ4HkvaDC2TsrgWT06bN8PUqS5wP/us23N7yBBXYdKtW3gvSLbR3Em3pLgDzw3rG8GIkk8BPM8laYaWSV473clp2I+6csxzj8AJt7iSwNJS1yT4zDPTNgaWcCUtZ58ECuAFLk4r+TLNa395cqqtTeWzR8OaNa6ypLLSNQZuq//F4yJp3wiTQP93F7C4reTL+KLrvHluf5Lx412/yeOOc6mSgw4KdqCStSR9I0yCZF5+F1/EbSWf58bA06dD377Qpw9UV8P558PixfDQQwreUlA0Ay9gcVvJ1+JX7PXrtzYGXrTILXf/y19c15vi4kjGG4U4pbwkegrgBcyvOnE/g0qTr9gffOD23b7jDvjwQ98aAydR3FJeEj0F8ALTMNgWd2xHuzaGuvqtOylnWhUQWFB55RU3277vPqirg2OOgUsvdcveTXMNovJbofZ9lOYpgBeQxsH248/raFdkKO7QjrXr67KaPfsaVKx1G0hVVcGMGdChA5x1lmuc8K1vZfZceShuKa90lOIJlwJ4AUkXbOs2W3bYvi3zrvxZVs/pS1DZsMGlRaqq3Mz7G9+Aa6+Fc8+FnXdu9eGFEjTi3vdRKZ7wqQqlgAQxg8tpR7wPP3SBevfd3WIbY9xOgO+8A5dd5jl4h71lblTivm1q3KqaCoECeAEJYvvRrILKokWuLVn37nDFFbDffi51Mn++2xVw++09H7+Qgkbct01NQoon3yiFEkNBpQSCWMrseXWdtfDvf7uFN48+6oL0wIEuv/3d72Z9/EILGnFeCBP3FE8+UgCPmSDziEEtZW4xqNTVwcSJLr89Zw7ssgtceaWbgXfunNNxQUEjTrTXSfiUQomZoFMCFX1KGHpEL7oUd+C9Neu5ccaiYPLFH3/sFtr06OGaAX/+OYwZ45ooXHWVL8Eb4p8XLiRxT/HkI83AYybolEDglQJLlrhuN2PHwmefweGHu8B95JGBNE7QBknxEucUTz5SAI+ZoFMCgSwGsRZeeMHlt6uroagITj7ZNU7Yd9+cx9yaJAeNQimBlGAohRIzQacEfJ3hb9rk8tsHHQTl5fD00/CHP7gywHHjQgneSVZIJZASDAXwmAk6j+hLKeG6dW6Ze8+erkXZRx+5xsDLlsF110GXLr6MNd8VUgmkBEMplBgKMiWQU6XA0qVbGwOvW+f2JRk50jUGLipq/fGyjUIrgRT/KYAXmKwu+r30kisDfOghd/uEE1zjhO9/P4QRBy+qPLRKICVXCuAFyNMMf/NmmDbNBe7//Ad22skturnoIreCMk9EuX+H6qYlV60GcGPM3cDRwEpr7V6p+74OTABKgXeAAdbaj4MbZjIlssLgs8/cfiS33AJvvun2KRkxwu1VstNOLT40ia83yi1aVQIpufIyA/8HcBtwT4P7hgFPWWsrjTHDUrf/4P/wkitxO7O99x7cdhvceadbhPODH7gLkr/4hafGwIl7vSlR56GTXAIp0Wu1CsVaOwtY3eju/sC41M/jgAp/h5V8QVQYVM+tpbxyJj2GTae8cqY/5Wbz5sGvfw2lpXDDDa7X5HPPwYsvulx3Kni3duykVlQEscGXSFiyLSPsbK1dkfr5faDZddHGmEHGmBpjTM2qVauyPFzy+D2z87VmuL4eHnvMrZLs0wcmT3Z7kyxeDJMmwcEHZ3zsqGey2dJSfEmynOvArbUWsC38/RhrbZm1tqxTp065Hi40uc52/Z7Z+TLDXb/elQDutRccdZTb1vWGG2D5clcO+M1vZn3spM5ktX+HJFm2VSgfGGN2s9auMMbsBqz0c1BR8yOf63eFQU4z3JUr4fbb3X+rVrlZ9333wYABnhoDezl2utfbro3h842b6DFseqwv0CkPLUmVbQCfCpwGVKb+nOLbiCLSsIKijTFsttt+qci0MsHvCoOsaoZfe82VAd53n2tbdswxrn77xz/OqDGwl2M3fr1f7dCOzzZu4uPP64DkXNQUSRIvZYQPAocCuxhjlgNX4gL3RGPMWcC7wIAgBxm0xjPuxsF7i0zzuX7O7DzP6K2FJ590gftf/3KNgc84w9Vw98pu9u/12A1fb3nlTNasr9vm79VBXcRfrQZwa+3JzfzV4T6PJTLpcrzpRJnPbXVGv2EDPPigC9wLFrj9tq+9Fs45xzVRCPLYaWST8kliHblIlLQSE28z6zhUJqSd0X/0kavdvu02eP992Htv+Pvf3XauGfSWzOrYLcg05ZPUOnKRKGk3QpoPKkXGxLcy4Y034LzzoFs3uPxy2GcfePxx1xj49NN9Dd7ZyLQ8L6l15CJR0gyc5nO8sQva1sKsWS5NMm2aqyA59VTXOGGvvaIe3TYyTbsktY5cJEoK4CRgT4q6OrcTYFUVzJ7tctpXXOFm4D71lgxCJmkX7cwnkjkF8JRY1gKvWeMW3vz1r26xTa9eMHo0DBzoqkvyiHbmE8mcAnjE0lZeFG/c2hj400+hb19eGPpnhn7ahdolG+gy8oV4fUPwQey/BYnEkAJ4ShQlbI0rLzq/Oof291yJXfQ8pk0bV0kyZAjVtlPq9zYA+VuhEctvQSIxpgBOdCVsN85YxMYNG+n3xguc/dIj7PfeItZuvwP3/ehEBt5/I5SkZqWVMyPbs9orP0+AqgcX8UYBnIg29f/kE3725HjOrJlKt7Uf8E7xbvzxJ+cwae+fsH67Dgws2XrcuFdo+HkCVD24iHcK4IQcIJctcxclx4zhynXr+F/X3lzb9yye6PkD6tu4uumSRpUXflRoBDmr9fMEGGWHHJGkUQAnpBK22bPh5pth4kR3+/jjeeaoU/ntG21brbzItUIj6FmtnyfAuH/bEIkTrcQkwE396+th6lS3+19ZGTz6qNtU6q23YPx4Dh14tKe9qHPdszroVY5+7gWe1H3FRaKgGTgBlLB99hmMG+caAy9e7BoDV1XBWWc1aQzstfIilwqNoGe1ftZwqx5cxDsFcHzMD69YsbUx8OrVcMABMGECHHecp8bAQQk6ReTnCVD14CLeGdvM3tdBKCsrszU1NaEdz4vG+WHIYh+U+fNhxAh44AHYtMl1ch8yxPWWzKBxQktjzCWg+fIaRSQyxpjZ1tqyxvcX/Aw866qH+nqYMcOlRp58EnbYAc49FwYPhj32yHo8jYP1Yd/uxMOza3O6AKlZrUh+KvgAnnF++IsvXIuyqipYuBC6dIHKShg0CL72tZzGkq5a5P4XlzbpGJ1NWZ3XHLoW0YgkR8EHcM/54ZUr4Y47YNQo1xh4333h3ntdY+DttvNlLOm+DTSX4AqirE6LaESSpeDLCFstIVy40M2uu3eHq67i2Z334KSTr6N8wE1Uf/cw34I3ZBaUgyirU1MFkWQp+Bl42vzwz75FxerX4ednwz//Ce3b8/bRJ3Derj9m4U67uQeu/cL32Wlz3wYM287Egyqr0yIakWRJfAD3I2f7ZX5440bXGPiMS+Dll2HXXeHqq3msvIILn1jWpFu930u8m6uB/uX+JTz9+qrA89LZ9LH08t4rry4SjEQHcL9yttOfeYXl14+g4vlqOn+6mnV79GKnsWPhlFOoXvgRwycvaBK8t/Bzdhp1tUgmi2i8vvfKq4sEJ9EBPOeNjxYvZsllf+aw6vF0rNvArNI+/O7nF1OzZxnX9/keFe3bpz1GQ37noqPcEzuTE4jX916bU4kEJ9EBPKucrbXw7LNuY6mpU+napi2P9D6Usd/vzxudSt3vbKr/MsC09Fz5uMTb6wnE63uvvHqwlJ4qbIkO4BnlbOvqYNIkV79dUwM77wyXX0756m+xasem9dtbAkxzxygypqBXMnp97/1exq+AtZXSU5LoMkJPuwiuXQs33eRWR55yCqxb5/YqWboUrr6a7bp2SfvcWwJMc8e4ecA+Bf2PxOsOjn7u9LglYNWuWY9la8Cqnlub8XPlA5V9SqIDeIvbrL79NlxyCXTtCkOHQs+eMG2aq+s+5xzo2BFoPcDkupVrvvL6vvj5/ilgbUvpKUl0CgXS5GxffBEGXAIPPwxt2sBJJ7lAvt9+zT4eWr5wp2a76YWxFW5DCljbCqURicRa4gM4AJs3Q3W1y28//zwUF7tZ9wUXuBl4GsqlJo8C1ra0d7okOoXCJ5+4/pJ77gnHHw/vv+9uL1vmNphqIXgrl5o8gXVOSiil9ySZM/Dly+HWW2H0aHeRsrzclQUeeywUFaV9SMMZdxtjAl9VKf6LeqFTHCm9V9iSFcDnzNnaGLi+3s26hwyBH/ygxYc1LrcKY1WlBEMBS2SrnFIoxpgjjTGLjDFvGmOG+TWohqrn1vK706/jxe57w/77UzdlKlx4oWsMPGFCq8Eb0lcvpFOouVQRSaasA7gxpggYBfQDegMnG2N6+zUw2Dpz/unz0+i25gOuPexMDv7tP6geeCmUlnp+Hi8z60LOpYpIMuWSQjkAeNNauwTAGDMe6A+85sfAYOvMefiRF7K2/Y5sblP05f2ZfI1uaTVlvbXKpUpaqlSSuMslgJcAyxrcXg40yWcYYwYBgwC6d++e0QG2zJxXd/zqNvenC8Ytaa7cSlfspTlapi5JEHgZobV2jLW2zFpb1qlTp4we21xO2kBGJX8qt3LvV3nlTHoMm0555UyVTLZCqz4lCXKZgdcC3Rrc7pq6zzdDj+jFJRPmNekLack8jVLI1QuaTWZOqz4lCXKZgb8E7GmM6WGM2Q44CZjqz7Ccij4loTb1hfycqWo2mbnmvv2pUkniJOsAbq3dBFwAzAAWAhOtta/6NbAtSkL8h5SvKzQ1m8ycVn1KEuSUA7fWPmat/Za1dg9r7Z/9GlRDYf5DyteZqmaTmdN1E0mC2K/EDHP5dL7OVLXpUXYK+bqJJEPsAzj4/w+pufrefN3tTnuIiOSnRARwP7VUkZHPM1XNJkXyT7K3k81Ca13SlfcUkaQouBl4a3luzVRFJCkKbgauigwRyRcFF8AzLUvMx4U9IpIfCi6FkklFhpagi0icFVwAB+957tYueIqIRKkgA7hX2Szs0R7SIhKWgsuBZyLTC575upeKiMSTAngLMr3gma97qYhIPCmF0oJMl6Dn614quVBKSSQ4CuCtyGRhT77upZItVfGIBEspFB9pD+ltKaUkEizNwH2kXf+2pZSSSLAUwH2mvVS2UkpJJFhKoUhglFISCZZm4BIYpZREgqUALoFSSkkkOEqhiIgklAK4iEhCKYCLiCSUAriISEIpgIuIJJSx1oZ3MGNWAe9m8JBdgA8DGk5cFeJrBr3uQlKIrxlye927W2s7Nb4z1ACeKWNMjbW2LOpxhKkQXzPodUc9jjAV4muGYF63UigiIgmlAC4iklBxD+Bjoh5ABArxNYNedyEpxNcMAbzuWOfARUSkeXGfgYuISDMUwEVEEiqWAdwYc6QxZpEx5k1jzLCoxxMGY0w3Y8zTxpjXjDGvGmMGRz2msBhjiowxc40xj0Y9lrAYY4qNMZOMMa8bYxYaYw6KekxhMMZckvr/+xVjzIPGmPZRjykIxpi7jTErjTGvNLjv68aYJ4wxi1N/fi3X48QugBtjioBRQD+gN3CyMaZ3tKMKxSbgUmttb+BA4PwCed0Ag4GFUQ8iZCOBf1lrvw3sQwG8fmNMCXARUGat3QsoAk6KdlSB+QdwZKP7hgFPWWv3BJ5K3c5J7AI4cADwprV2ibV2IzAe6B/xmAJnrV1hrZ2T+vkT3D/ovN9I2xjTFTgKuCvqsYTFGPNV4BBgLIC1dqO1dk2kgwpPW6CDMaYt0BF4L+LxBMJaOwtY3eju/sC41M/jgIpcjxPHAF4CLGtwezkFEMgaMsaUAn2A/0Y8lDDcAvweqI94HGHqAawC/p5KHd1ljNkh6kEFzVpbC9wELAVWAGuttY9HO6pQdbbWrkj9/D7QOdcnjGMAL2jGmB2Bh4GLrbXroh5PkIwxRwMrrbWzox5LyNoC+wF3WGv7AJ/hw9fpuEvlfPvjTmBdgB2MMadGO6poWFe/nXMNdxwDeC3QrcHtrqn78p4xph0ueN9vrZ0c9XhCUA4ca4x5B5cq62uMuS/aIYViObDcWrvlG9YkXEDPdz8B3rbWrrLW1gGTgYMjHlOYPjDG7AaQ+nNlrk8YxwD+ErCnMaaHMWY73EWOqRGPKXDGGIPLiS601lZFPZ4wWGuHW2u7WmtLcZ/zTGtt3s/IrLXvA8uMMb1Sdx0OvBbhkMKyFDjQGNMx9f/74RTAxdsGpgKnpX4+DZiS6xPGrqmxtXaTMeYCYAbuKvXd1tpXIx5WGMqBgcACY8y81H3/Z619LLohSYAuBO5PTVKWAGdEPJ7AWWv/a4yZBMzBVV3NJU+X1RtjHgQOBXYxxiwHrgQqgYnGmLNw22oPyPk4WkovIpJMcUyhiIiIBwrgIiIJpQAuIpJQCuAiIgmlAC4iklAK4CIiCaUALiKSUP8P8E+7kB8xqKkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "from sklearn import linear_model\n",
    "import numpy as np\n",
    "\n",
    "# load data\n",
    "# generate data\n",
    "data_num = 100\n",
    "X = np.random.rand(data_num, 1)*10\n",
    "Y = X * 3 + 4 + 8*np.random.randn(data_num,1)\n",
    "\n",
    "print(\"X: \", X.shape)\n",
    "print(\"Y: \", Y.shape)\n",
    "\n",
    "# create regression model\n",
    "regr = linear_model.LinearRegression()\n",
    "regr.fit(X, Y)\n",
    "\n",
    "a, b = np.squeeze(regr.coef_), np.squeeze(regr.intercept_)\n",
    "\n",
    "print(\"a = %f, b = %f\" % (a, b))\n",
    "\n",
    "x_min = np.min(X)\n",
    "x_max = np.max(X)\n",
    "y_min = a * x_min + b\n",
    "y_max = a * x_max + b\n",
    "\n",
    "plt.scatter(X, Y)\n",
    "plt.plot([x_min, x_max], [y_min, y_max], 'r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. 如何使用sklearn拟合多项式函数？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([800.,  90., -20.])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fitting polynomial functions\n",
    "\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "t = np.array([2, 4, 6, 8])\n",
    "\n",
    "pa = -20\n",
    "pb = 90\n",
    "pc = 800\n",
    "\n",
    "y = pa*t**2 + pb*t + pc\n",
    "\n",
    "model = Pipeline([('poly', PolynomialFeatures(degree=2)),\n",
    "                  ('linear', LinearRegression(fit_intercept=False))])\n",
    "model = model.fit(t[:, np.newaxis], y)\n",
    "model.named_steps['linear'].coef_\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}