{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 最小二乘(Generalized Least Squares)\n", "\n", "## 1. 最小二乘的基本原理\n", "\n", "最小二乘法(generalized least squares)是一种数学优化技术,它通过最小化误差的平方和找到一组数据的最佳函数匹配。 最小二乘法通常用于曲线拟合、求解模型。很多其他的优化问题也可通过最小化能量或最大化熵用最小二乘形式表达。\n", "\n", "最小二乘原理的一般形式为:\n", "$$\n", "L = \\sum (V_{obv} - V_{target}(\\theta))^2\n", "$$\n", "其中$V_{obv}$是我们观测的多组样本值,$V_{target}$是我们假设拟合函数的输出值,$\\theta$为构造模型的参数。$L$是目标函数,如果通过调整模型参数$\\theta$,使得$L$下降到最小则表明,拟合函数与观测最为接近,也就是找到了最优的模型。\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.1 示例\n", "\n", "假设我们有下面的一些观测数据,我们希望找到他们内在的规律。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAA9qklEQVR4nO2df5hcVZnnv29Xbkg3unQiGTY0CUGXCWtkSEsr7LIzKzhjFAboESXysKMz4zPsrs7zDJHtNazOEFxcsoOK7jM7urjMjK6IicC0ARyRkbi78hiwYyfEKNkJQhLKKNGko6SbpLr73T/q3sqtW+ece879Xd3v53n66e5bt26dc6vqfc95fxIzQxAEQRAAoKfsAQiCIAjVQZSCIAiC0EKUgiAIgtBClIIgCILQQpSCIAiC0GJB2QNIw5lnnskrV64sexiCIAhdxY4dO37OzEtVj3W1Uli5ciXGxsbKHoYgCEJXQUT7dY+J+UgQBEFoIUpBEARBaCFKQRAEQWiRm1IgokVE9DQR7SKiPUR0u3/8b4noeSLa6f+s8Y8TEf03ItpHRM8Q0RvzGpsgCIKgJk9H8wkAVzDzy0TkAfgOEf29/9gIMz8QOf8dAM73fy4B8Fn/tyAIglAQuSkFblbae9n/1/N/TNX3rgXwRf9524mon4iWMfOhvMYoCIJQNqPjddz12F78ZGIKZ/f3YmTtKgwPDpQ2nlx9CkRUI6KdAF4C8DgzP+U/9HHfRHQ3EZ3mHxsAcDD09Bf9Y9Fr3kREY0Q0dvjw4TyHLwiCkCuj43Xc+tBu1CemwADqE1O49aHdGB2vlzamXJUCM88w8xoA5wB4MxG9AcCtAC4A8CYASwB82PGa9zDzEDMPLV2qzL0QBEHoCu56bC+mGjNtx6YaM7jrsb0ljaig5DVmniCibQDezsyf8A+fIKK/AfAf/P/rAJaHnnaOf0wQBCERVTPNRPnJxJTT8SLIM/poKRH1+3/3AvgdAM8S0TL/GAEYBvAD/ylbAbzXj0K6FMAx8ScIgpCUKppmopzd3+t0vAjyNB8tA7CNiJ4B8D00fQqPALiPiHYD2A3gTAB3+Od/HcCPAewD8HkAH8hxbIIgzHGqaJqJMrJ2FXq9WtuxXq+GkbWrShpRvtFHzwAYVBy/QnM+A/hgXuMRBKH7SGP+qaJpJkowlyqZuLq6IJ4gCHOXwPwTrPYD8w8AK6F5dn8v6goFUKZpRsXw4ECl/BxS5kIQhEqS1vxTRdNMNyA7BUGYp8z1yJwqmma6AVEKgjAPSWuaKYIszD9VM810A2I+EoR5iETmCDpkpyAI8xCJzBF0iFIQhHmIROYIOsR8JAjzEDHNCDpkpyAI8xAxzQg6RCkIwjylCqaZqofFzkdEKQiCUArdEBY7HxGlIAgFI6vjJqaw2Pl4P6qCKAVBKBBZHZ+iG8Ji5yMSfSQIBdINSWNFUcVeAoIoBUEoFFkdn0LCYquJmI+ErqLK9nibsbkkjVV1rlmNS8Jiq4koBaFrqLI93nZsI2tXtZ0HqFfHVZ1r1uOqQlis0I6Yj4Suocr2eNuxDQ8O4M53XoiB/l4QgIH+Xtz5zgs7BGNV51rVccUxOl7HZZuewHkbHsVlm56oVJ/mqiE7BaFrqLI93mVsNqvjKs01bC5izTlVeA90VHXXVVVkpyB0DVWOVsl6bFWZayBQ6waFAFTjPdDRrbubshClIHQNRUeruJgcsh7byNpV8GrUdsyrUeGROSqBGqXqEUNV2nV1A7kpBSJaRERPE9EuItpDRLf7x88joqeIaB8RbSaihf7x0/z/9/mPr8xrbEJ3YmuPz4LoCjkwOegUQy5jiy7NTUv1nDAJzrzfg6yoyq6rWyDmfD5pREQATmfml4nIA/AdAH8K4EMAHmLmrxDR5wDsYubPEtEHAPwGM/87InoPgN9j5nWm1xgaGuKxsbFcxi/Mby7b9IQydHSgvxdPbrhizr9+1caRhqhPAWjubqquzPKEiHYw85Dqsdx2CtzkZf9fz/9hAFcAeMA//gUAw/7f1/r/w3/8rb5iEQRn0kablG1yKPv1A+ZCglmRO8y5QK7RR0RUA7ADwD8D8N8BPAdggpmn/VNeBBC8MwMADgIAM08T0TEArwHw8zzHKMw9sog20SWZ9RDhvA2P5p5oVZXOaHMlwUzyIezJVSkw8wyANUTUD+DvAFyQ9ppEdBOAmwBgxYoVaS8nzEGyqL6pSjIDgBnf3Jp3WKNtkpsNaTOQRaDOLwrJU2DmCSLaBuBfAOgnogX+buEcAMG+vg5gOYAXiWgBgDMA/EJxrXsA3AM0fQpFjF9IRlJhlFaIZWF6ia6Qe4haCiEgrzLPwfynGjOo+a87kHCFLjH6git5Rh8t9XcIIKJeAL8D4EcAtgF4l3/a+wB8zf97q/8//Mef4Ly84ELuuEbvpH1emKyiTYYHB/Dkhivw/KarMKv5KGZt4w/PH2juTIIdQhIhLjH6git55iksA7CNiJ4B8D0AjzPzIwA+DOBDRLQPTZ/Bvf759wJ4jX/8QwA25Dg2IWeSCqMshFgeztGiwhqzFuJVcVgL3UNu5iNmfgbAoOL4jwG8WXH8FQDvzms8QrEkFUZ5mH6ycI5maeM3kbUQ7+/zcHSy0XFcYvQFHVL7SMiFpNEzWUXdZO0cLSoKJ8uoo9HxOl5+ZbrjeBmZ0UL3IGUuhFxIasKpclx82Mfw5IYrcos6ymr+dz22F43ZTl/I6QsXiJNZ0CI7BSEXbFbWpiijsuLiy25sk+X8dSanY1Od5iRBCBClIDhjKzhNJpy4UMm8BbFqDgAqEb5pM/+su7zNJcpW7N2OmI8EJ7IIGQXKDZXUzWHj1j2FjylJOQ7b96DKpri8yOrzOZ8RpSA4kZUwLzNUUjeHCY1ZJa8xJRVgWXd5SzLuqnYxk7yM9Ij5SHAiK2Ge1rShM//YmA2SjDUPkpbjsH0P8jCjVD1DWvIy0iNKQXAiKzt1mrh/lWAaeWAXwGhF25iElW4Oi/s8vNKYzT0XIcBFgIUFvKrkBtD+HuiE99j+I3j0mUOt3IX+Xg8br1ltLdBdFFkZtv356kfJEjEfCU5kZadOY9pQCabGDHeEX0bNBoHZoz4xhWhN9l6vhtuuXl1oiWXbLOmomUmlEKLvgU54f2n7gbZktompBka+usvaBGSryD46uhvrN+8s3LY/H/0oWSM7BcGJLEMmk0YZuZgCgnOjK2dGs3MYAx3F5vJSAtGV8+UXLMWDO+ranUlwvmrlCwA1IswyK98Dl3vUmGXrwn42K/HR8Tru236go1FcXgUEw5Qd0jwXEKUgOFN2KWWdYNKdC6hXzoFCcO0glsQsojLnPLijjusuHsC2Zw93XEvVLSzKLDOe33SV8jGXewTYKxEbs99dj+3Vdg4twu9R9uez2xGlIHQdKsHk1ajNpwCcElaj43WtgHR1QCZ1tOrMOduePaxUSqrzo5zR62kfU92jYGekwtbmbrMSN91TG79H+HWE4hGlIHQdOsGkOxYIGhWuDsi8I4bijoc5fnIao+N1beJgMN6wuWrz9w6iMdOuGrwet1pIcStx3S6FACu/R94mJsGMKAWhK9EJpuixyzY9oV1xezXC5RcsbTmfbRraJBXirlExNuafxozZF6C6R0PnLsHtD+9JHH1kg26XcuOlK6x2FBI+Wi6iFIQ5jUnAzMwyNj99sGVysmm1mTTk0TUEV9cONIqrAM3C3q7zA4SP9/d5OG1BD45NNaQMR5chSkEojSLi2E0r7lmGtqOazoyRNL/CNSomej6g9geY/Ap5YMp/CEdSHZ1soNer4e51a7RzLKpHheAGdXPHy6GhIR4bGyt7GEICVNE1vV4t87wAmygeHQQoo3vKSMoa/Ng3lc1yFvd5GP/zt+X62jaJczXN8bjoLileVw5EtIOZh1SPyU5BKIU4J+PoeB0bt+5p1SNa3Ofhtqvdbd/B+bds2aUUWiZ0Zoy0JpgkgnBCoRB0x7MUtFGlqruHuuNx5i3dvcxLWYgSikeUglAY4S+kKY59dLyOka/uagsvPTrZaJaygHu4YnC+y44hLzNG0jBMW/t7mjBPlcC0CY0F9DuFpB3j8ghVlRBYO6TMhVAI0VINOs7u79V2DAuibZIQlNWoUbTAxSmCx/Isb5G0iqdt+Yak19dVbLVJgOv1arjhkuWZdozLo9KpVFC1Q3YKQiHYrDgDIbJ+807tOWnCFXU7hjx8GTqShmHaOqqTXl8nMHU7AFWJjaFzl+TaMS5tqKqEwNohSqEi5GnrTHPtrMZl+uIR0HZtU72ftOGKZdfGSROGaePL0F2/hwjnbXjUWZnMMKPXq1kp0azKS+QVqiohsHbkZj4iouVEtI2IfkhEe4joT/3jG4moTkQ7/Z8rQ8+5lYj2EdFeIlqb19iqRp7dotJcO8tx6b54A/29eH7TVXhywxUtgTKydhW8nk4zj1dzy7zVMTw4gCc3XNHxukWQdxVP1fWBpnA3vYem96fIyrFAfvdIKqjakVtIKhEtA7CMmb9PRK8GsAPAMIDrAbzMzJ+InP96APcDeDOAswH8A4BfZ2atzWGuhKQGGbVRkhRry/LaWY7LNQRVF30EdH8FzLwjYGxCSKPvYVEhwrZI9FG+lBKSysyHABzy//4VEf0IgOnuXwvgK8x8AsDzRLQPTQXx3bzGWBXytHWmuXaW4wqbbYKSEmEnn40pIm30SF6dyFyvmXcVz/D1z9vwqPKc6HtYtlktSl73SCqoxlOIT4GIVgIYBPAUgMsA/AkRvRfAGIBbmPkomgpje+hpL0KhRIjoJgA3AcCKFSvyHXhB5GnrTHPtrMelcvTaCHZTXwHbAmoqhbJ+807cvHmnsdZR0de0eU0Xwe3yHorAFIACQlKJ6FUAHgRwMzP/EsBnAbwOwBo0dxKfdLkeM9/DzEPMPLR06dKsh1sKedo601w7j3G5hgWG/Ro6bHYuun4KQLyvRNeoPs01k5DExyN2dMGVXHcKROShqRDuY+aHAICZfxZ6/PMAHvH/rQNYHnr6Of6xOU+eW/c0185jXK4mKZtQVtPOJa57WYCpz7BuZxOnjLIuA61TqLds0Sf1VcEsJHb8bMn7fuamFIiIANwL4EfM/KnQ8WW+vwEAfg/AD/y/twL4MhF9Ck1H8/kAns5rfFUjz617mmtnXYbA1SQVJ3hNq17Xukeq1zLtbGzKW2cZA697rRlmowmuTLOQZBFnSxH3M0/z0WUAfh/AFZHw078got1E9AyAywGsBwBm3gNgC4AfAvgGgA+aIo+E8kgTqupqzjDtAuLCI21LNJhey7Sz0YV/xl0zKaZs7Kpm5koWcbYUcT/zjD76Dpp5SVG+bnjOxwF8PK8xCdmQpmOWqzlDV17ZJlTSZZWuU0ymnU00oira7jJr231cQb8qZuZmGcEmZqhisrIlo1lwJu0H08WckcYmbmoLyZH/r7tYPSadUgo6tgVj+vS6NYnHactAjLmqipm5WUWwiRmqSRFZ2aIUBGeKLhcQp0SiHb+YgWNTDZzR68GrUVtPYlXzegaw7dnD2tcGOnsdhxvKBALqzndemDrZ0ISpG1tVI4qyaqQj/ZybFNGYSJSC4MzI2lUdpa1dm79nRXQFGW5EMzHVgNdDWNznYWKyYXQM1yemcNmmJ5Sr+6hSUvV9thFQac0fqgTAuJ7SNuRplskq+qnoYnZVNVUVEU0mSkFIRtRbpPeB5kqcM7kxy+hbuKDVnUxXugOwN0kkEVBZmT+yjiQqwiyTxZiL3J1W3VSVdzSZ9FMQnLnrsb1tJhkgea8DXWJY3GMBruU64iKGbCI5dILIJKDiokZs5poH3RIdVGQSXrfck7yQnYLgTFZbedOKDLArh2GTKxAW1lETTJJ5JLHrmu5ZmSvTbukxUGQSXrfck7wQpSA4k9VWPm5FZmO3NzlfAbWwDrbfOlNS3DySCCjTPSvTidpNPQaKSsLrpnuSB2I+EpzJaitvWpHZrtaGBwfa6v0v7vPQ3+u11f4HoDTNmOYRZ85x7clgeq0yV6ZSG6mT+X5PZKcgOJPVVj5uRZZFdU8b00x0HoB7Jdc4TPdMZ8oqYmVahdpIVWO+35PcmuwUwVxpsjNfMTV2AbLppZykUZApQinrktij43Xc/vCetlBaoNwGN8Lcp5QmO8LcJ4tY7kVeT0vw9/d62HjN6rZrpO0tncSZbHrMtGtwvR+6gn2q+5CE8D3IKqdBmPuIUhASkUUXtKhAPDE923ZOUseiTXVUk2kmLqJJ5QROcj90ORann7YgE4UQHk9QN6lqMfdC9RBHs5CItHH3uuffvHln6jj9uIS2OKehTfXT6G4iSWx7Fg5mlwZAtuMS5jeyUxASkTbuPqmJJs3YADufgE0uQ3SnkUTAJwl9jNZ5evmV6Va5EZcGQPMl5l5wR3YKBVBWtmocacZlyuq1WTXHRdakWc3qrh04l227zj254Qp8et0aq/DEJFnOrqGP0T4WRycbbfWngPYGQCbmS8y94I4ohZxJ05CmyuO6/AJ1f+zLL1hqtWpOYqKxRXVtwqmidy73PpoHoWvskyS23fbaAbZNg+IaAM2nmHvBHTEf5UxVS/6mHZeu1PS2Zw9bmUWSmGh0qKJ+7nznhcrmN0lMUzYO76Sx7S7OdFslqWoAJNFHgi2iFHKmqnVU4nwCccLN9Py7162xqg0UCERdvoLNalbnvwh6G6hyDtIoZdO9ybsMg02dp/B9K7M3s9C9iPkoZ5LYmrNG5TvQvf4ZvZ6VWck0r7BZBGj2Fg4Escp042pGCRPnv8i6HWSZpkCVScirUUdZD1EEQhpkp5AzRXRKMqFbSV938UBb97BgXETJC9FFV6mAfbmIpKvaOKGfZXGzsk2B8738glAMslPImTSr4CzQCbJtzx5WjmsiUm4hIK4QnWpeRdSlj9uJZVncrAqmQNdCfILgiuwUCqBM265JkKnG5VKcLW5etiUm0pTLsN2xpFldB+PTVQlL4xAXoS5UjdyUAhEtB/BFAGehGfxxDzN/hoiWANgMYCWAFwBcz8xHiYgAfAbAlQAmAfwBM38/r/HNBWyEjM580kOE0fG6MrQyKmS9GuH4iWmct+FRa2E2Ol5vi/qJjil8nq2JyTRf3fG0gjiuZEZah7hqnt2IKLy5Q547hWkAtzDz94no1QB2ENHjAP4AwLeYeRMRbQCwAcCHAbwDwPn+zyUAPuv/FhTYChldE5oZZuX5USHbt7CG4ydnMDHVML5OFNPKOpzjYGunj5uvaixZCOK43IBFntoCGxWSkyenKxmanAVzXeHNN3LzKTDzoWClz8y/AvAjAAMArgXwBf+0LwAY9v++FsAXucl2AP1EtCyv8XU7tvb6wPZfI+q4hs6+H9it7163BpMnOwWijV/AZGf/0vYDrSgonYkpejyJf8L0HNts7jh/wdHJRkcEkipKKVoa2/b63cB872k81yjEp0BEKwEMAngKwFnMfMh/6KdompeApsI4GHrai/6xQ6FjIKKbANwEACtWrMhv0BXHxek5PDiA9Zt3Ol0HMK/244RZXEx9sJokAlQtPWq+eStYbScZh+6x4LV1vaHDK/z+Pk8r0AOiK37bzGNgbpSbqIIDXsiO3KOPiOhVAB4EcDMz/zL8GDc7/Dh1+WHme5h5iJmHli5Vl1qYD7jmPyTJl4gr5mZiZO0qdO5N2plqzCgVAnDKvFU3KIS4cegeC/ImomPZuHVPxwr/5Vem4dXiZtJ+r2yF4VwpN1GFXBwhO3JVCkTkoakQ7mPmh/zDPwvMQv7vl/zjdQDLQ08/xz8mKHANtUwSmqn7UpN/PRPDgwO48dIVsYpBh0pwR0lSArvXq7V6C0SZmGp0vGZjlnH6wgWt0FuVGQ5ov1c2wtAUmlzVAoo65ntP47lGbkrBjya6F8CPmPlToYe2Anif//f7AHwtdPy91ORSAMdCZiYhgmv+Q/T8/l4Pi7werDf0L9AVlrvx0hVWDsQ7hi/EjZeu0ApS+ONwEdwBNSJcd7G5N3NgxgleP7hHA44r2GNTjVZuwCevvyhWAMYV+yNAm2NQdtZ0EsrOxRGyJU+fwmUAfh/AbiLa6R/7TwA2AdhCRO8HsB/A9f5jX0czHHUfmiGpf5jj2LoSVdifrs+wCl2tIV20SNoY/9HxOh7cUdcK+F6vho3XrFa+hqlQHtA0Lz24o46hc5coQ1ejXccCwa3KtA7GssjrUfoPdIX8dPck+PuWLbuUczftJMrOmk6K1FmaO+SmFJj5O4DWevBWxfkM4IN5jafbyTLsz0XwpPmymxyu0WqdqteIa6mpG3Pc/HSCXfWapkJ+JkzKx2RWEaetUDaS0dwluAjyuESiJM3sk6C7XmA+UREe+xm+iWtisuEUfWQjWE2CPaskrCQ7LV20U3+fB0CSxIT80SoFIvo6gA8w8wvFDUfQYbuCjNtR2GYaZ4FrMbro2CemGuj1arh73Rqn8htpiuC57ozihLTr9XSuFGZJEhOKweRo/hsA3ySij/hRRELOmKJObMP+4hKJdLkHNhFFruiiUi6/YKl1s/lg7C7d1JJGw7hG/eThFD42pc6JODbVkCQxoRC0OwVm/ioR/T2APwMwRkT/C8Bs6PFP6Z4ruKNbBY7tP4Jtzx7u6CAGqAVd3I5C9zgj+9Xm8OAAxvYfwf1PHcQMM2pEeOOKM9pKdts0mw+K9wGnCvaZuqklMdskWYXrhPTGrXsSm3hMu5wi/A1inhLiQlJPAjgO4DQAr478CBmiEzD3bT/QEhKMU557Xdhf3I7C1NQeyDZGPhp9NMOMJ587ol3txo09KL8x0N/bsduJrphdS0wnWYXrhPHEVCPx7sG0y8k7Sawbw2GF7DH5FN4O4FNo5g+8kZknCxvVPMS0go/+P9Df23LURld2l1+wFJu/dxCNmVPP9GrU2lGYSk1nbbN2Kffg0sYz7Yo5uGfh3sWmcemwaY8JuIWUmnY5Y/uP4L7tB2J3i0np1nBYIVtM0UcfAfBuZt5T1GDmM7YCBjglqFRCfPPTBzuFXOhfk9C5bNMTmQoFF7NGtNl8knLgNitmVQ5D3Lh06CrQqnC5FyrndLDrCo+WAGMCnysSDisAZp/CbxY5kPmOSsDERQmpVnaN2c5nNGa5TbDrImKyFgq2ii7s5LaJ1tEJ48mT08oeEWFcdi9xq3CVEps8Oa1NgBsdr+P2h/e0Hu/v9bDxmtVWQl01bgaw7dnDVnOxIcvWpUL3InkKFUElYC6/YKmyj3IgqFyEddy5o+N19GhMKUmFwsjaVVi/eWdsxUNXJ3dw7sate1p9HoBTZazD5wSETUZxENC2SzE5X6NKTNWUJ4i4GnlgV5tZb2KqgZGv7jKON3jNInJLyu4nLlQDUQoVQrVKHjp3iVYguZicTII9EGQqhWAbyqkao84OHsW1FhHQvFd3Pba3TSkAdg16TIT9NarnxvlZdCawux7b26YQAqK7ON1rFpFbkrasiTA3EKVQIEnC/UzmFGXrzB4CCG0CKE6w60wqQdG5ux7bi/WbdyrHHCc07xi+sKXYbMNqbbE1d9majKJjGR2vK+sXxflZVO+Zrp+F7XiDyLO8nMwBUsNIEKVQEHlko5pq+NgonziTSlB0zjRml4gVQrNcA3MzGSvtStTWBm4ysQTRR9FaTKbdk+qacQrftKuzHW8QeSareCFPRCkURF7hfrqVXdw1bUwqumY04THHtdOMvs7RyVOlK9IKNFsbuE4gR01FYeJ2F2FBHpd4GNRxqvUQZiKBAF4PZTJeQcgKUQoFUbVwvzih1+vVtI+Hx6yL8w96GNiU3Ui68rW1gSdxoJrel+hzTYmHwZ2ZmGrA6yEsWljDcb/vdTj6KFoI0KuRkwlQELJClEJBVC3czyT0BvrNPQ3O6D1VCktnXgmOJ+mT7KoYbMtYuygg3ftVI+rIJLdNPGzMMn6tbyH2fKx9ta8qBAgAPQTMcmeZcUHIk9x7NAtN4oq0Fd2C0VTuIigLMbJ2VdNxHeG4nw8QnK+7jul1ABRa3M217IXu/frk9RdZlxZRoVIgul3bLJ/6jIhCEIpClEJBDA/qWxaWUXNGJfS8GuH4iemWYgKAVy3q3Ew2ZrglvE3KbnS8juMnpp3GVZXsWdP7FUVXwVWFSoGY5ixVUIWiEfNRxrgkOgWUUXMmalLp7/Pw8ivTLdNF1LwTJRBkLl3MbKhS9qxteGaSxMMwcfkmVVGUwvxAlEKGJA07jYvgyYuw0Lts0xMd5RmCpvcqvwH7zwknqoVR1VGKo5udqa6Jh2HiaihVSVEKcx9RChmSdMUfF8FTBKZcBV0kkknpua5u55IzNbpbjAu/1ZXtALpbUQrdifgUMsQUaWNyIsdF8ORN0KJTRWBL1zmUdTZvl9VtEKs/VxRCEv/Q8OAAdt72Nnx63RorP4Yg5IXsFDJEZxsO2kYC6tX1gCFZKSkuJTV0LTqBZuXRoMSFjnAp77hYe4Ax1Zhte76q/o8LVeoWltY/ZOvHqNKchblFbjsFIvprInqJiH4QOraRiOpEtNP/uTL02K1EtI+I9hLR2rzGlSe6KJS4LmFJewrrcF2tmkw9RycbrWvo6O/zsOb2b+LmzTtbrzkx1QAYWNznta16X4koBJsxmBgdr2PkgV1tcx15YFeukVum8OGiWmZKhzQhL/I0H/0tgLcrjt/NzGv8n68DABG9HsB7AKz2n/NXRFRTPLfSqMIYdSvwsJBwCX+0wbW1ZFpH5tHJRoctHGjuAPoWLmjLDci6peTtD+/pqD7amGHc/nA+vaHiBHLeLTOBZK1DBcGW3MxHzPx/iGil5enXAvgKM58A8DwR7QPwZgDfzWt8eRHd/l+26QmrTOYsq1PG+TaiJgeXDmJpx5J1zX5VQxvT8bTEmYdG1q7CyFd3tTU7UtU3SkPVSqYIc4syfAp/QkTvBTAG4BZmPgpgAMD20Dkv+sc6IKKbANwEACtWrMh5qOkpo3FJUt9G2EZ9/MS0cvWfZCxhuqlmv8pubyWQo177jIPIqlYyRZhbEOcY4eLvFB5h5jf4/58F4Odomtn/M4BlzPxHRPSXALYz85f88+4F8PfM/IDp+kNDQzw2Npbb+LMiiVMwjSNRVQFV16RFV3nTpTGNjl6vlmn0jOqeqMI4A2pEuOGS5bhj+MJEr6VS5qct6FG+XnAfdTtD031O8tlQjU0ilQRbiGgHMw+pHit0p8DMPwv+JqLPA3jE/7cOYHno1HP8Y11B3Bfb1TSUtveCajXu2s4xfI0kSXSL+zzcdrVd/2FAfQ/DcwgyrgOzTHBPrrt4AJufPqjsTT3DjC9tP4CHdryI//LO33B6D3RmokVeT0fehk2LVJXpDoDyfQ6X3NZ9nsL3Jk6ZSKSS4ELRO4VlzHzI/3s9gEuY+T1EtBrAl9H0I5wN4FsAzmdm4zK1CjuFPFZtrqvNNNcMrmsSFKbnBgQ7kSRJaKp7qOogZxp7nPJyfU/O2/CocmdFAO5et0YrZHX3StU1bZHXo/R9qM5N+nkqalchiqe7KGWnQET3A3gLgDOJ6EUAtwF4CxGtQfMz/wKAfwsAzLyHiLYA+CGAaQAfjFMIVSGPukW2jkSXL6LJmRy3EzE5MKNN7pOguoeqlb+K+sRUaye2csOj2vOmGjO4ZcsubVvRKCa7vWuLVF1Yss40pwthTnJ/i6irlUdXQaE88ow+ukFx+F7D+R8H8PG8xpMVUUHsapaxwcaRmKahvOraJkGRdyewNDWeCM17MTw4oC0XEhA8ZiO0kgYIRO9z3JhsSfp5KiJSqYyCjkJ+SJkLB1Qx6i4lkm25/IKlHde17faVJlZdJyiyTq6LkqbGEwOtOd9wyXLzySHi7lWa3JEgNLXXqxkVQn+vZ11y+4xeL1G/jSLyJiREdm4hZS4cUAli1Vc+bTbygzvqbdclANdd3G6ycP0i2kQT6QRFXmGkwa4r7Uo6mHMQZXTfUwdgc8k4oZUmd8Sm3enGa1a3zjWV3PZ6CMdPdpY1D8ZoooiQaAmRnVuIUnDAJET6ez0cm2rkYl9nANuePdx2zPWLaCOkTILCRkC6+DhslNTiPg99CxfE5k6E53zH8IW4Y/jCtrH0aEw4eQotm3an4T4bYaIltydPTivLmt+yZZfy+WGKyAspIxdHyA9RCg6YfAinn7YAO297W+rXsN0BuH4RXYRUHLrwURsfR/DcOD9Cr1frCGnVKZJJvz2oLgxYF4FTRgKhjR8mqoDP0zjQZ5itdgxRxRCYzbJSDN2UkCjEIz4FB0xCJCv7aZwNOCjGtn7zTizyetDf61nZvBd56rc63JPZBl3tn9sf3hPr4wg/14RqLoEymWrMdNjdj042jAXhhgcHcN3FAy3fRY2owxyXNbriiEG+gkvxurg+13F+JCmgJ7ggSsGB4cEBLO7zlI9lZYqI63kc/nIfnWzgxPQs7l63xijYb/z8dzvKVQPNN991taxzcOtqDYWVZZwJC1ArqagyUbkLAnOKStAFfprAhDTDjAd31HMVimFHNdAeluoqlFWfiTBxC5K8C+iJ0plbiFJw5LarV+caiWOKekny5R4dr+PJ544oH5tFU2C4fHldd0RhZRn3XEIz8iqKjTIBTplTovMpq6ro8OAAntxwhbJarsvrB58JXZRW3IIk7+ggqdo6txCfgiNF2E91Tt0kX+6NW80lpF0TjXS28v5eDyemZzuStsJCPq5BPQN4cEcdQ+cusYq0UqGKjy87ZDKL1w/mk8Q3knd0UNn3V8gW2SkkIFgBhvsEFIFrzPnoeN2q0qnLqk5n3tp4zWpcd/FAm70/EPLByj3ODKIbyxm9apOdjqgwsr1vuuY5pqY6NmSVK5A0dyLvPJMiciGE4hCl0EW4frldtu+2WcUmwbTt2cNGM0n0uTrCQn10vI7jJ6c7zukB0KO5SFQY2dy3j47uxvpQ57hgB/XR0d2p7eVZCmWbBUlUiQHItIlTlLyVjlAsYj7qIlxNVy7b93C5CJtxJDVvhZ9r04Dorsf2KovineFXYbUxp8Tdt9HxOu7bfkCp0O5/6mBHjoNrCYciQzZ15U/ufOeFmZQkUSEhqXMLUQpdhkuWbZwNPwyj2drSVL467svuaru2ybXQKZqJyYZRGKlyKXRC8a7H9mrbpuqyrV3t5Wmyo10oqw5RUfMT8keUgiNZlAguqsywTujqInmOTjZaoaX1iSmMPLALYHT0MADUTmnXhLrhwQGM7T/SWo0H+QMAWr0H4rKRVcLItVigScDrCtpV1V4uTl8hLeJTcCCLeOysY7pNTlCd/X/AUqA1ZrijhLXJKe3qCFXlD2x++iBGHtjVuj8qgRxnr3YNkdQJeEKzyF432cvF6SukJdcmO3lTdJOdLJrfZNlAx6WBSnh3ckavh+Mnp2Mb2OggAM9vuirRc8PYNO8JqBFhltlqZ6VrkAM077PK1KTqgXDjpSs66ihV3V4urToFGyrTjrPbyWJrnsU1TPWDVPbjqKCYmGrA6yEs7vMwMdkwFptTcXZ/byaC0mXOM8x4IUYRBWPSKYSgzASgNinp5tNN9nJx+gppEaXgQBZJQLYNdEyRMnHVRaPCVtfZrG/hAoz/+du01/Vq1OZTAJqCdeVrejPptOXiCI9m80bvkarkdBhd97NAgXaT4AfU8zf1dRYEW8Sn4EAW8dhx14jzOdiUfIgqKdtQ0WjRuHVvWo51b17ekZD25HNHMilrYJPMFjDD3PKfrNzwaEdOwX3bD2jvi6rMREA3OmBVn5EvbT8gtYeETBCl4ECably214hzksYJMZWSsnE+6orGPfrMIa1AjZIkTDN8L0wd2Bb3ecaieCaTUVB/SEU3OmBtFgZSe0hIyrw1HyW1iWdhZjBdI25VbzK56PoimEJF4/wTNoXoTl2zB6+79eut8NIbLlne6oamI3wvdH0DAIAZTmMJCIT+XGoEY6t8u3EXJJTPvFQKrnHsRRLnc1AJN6+H8KpFC1oNVMb2H+mwL9/5zguxceueljN5kdeDsf1HjHZ4VyZD5blnmPGl7QcAIFYxhOeoK7Z3zMIJHvUbhIV+WQ7YPCKXbH0x3bgLEspnXoakmsJCR9auSpXVmxabkELX8NJer4brLh7oUAAq52sUVfVTF2pEeO7OK63ONc09rltbMMcqOVvzCg+1CTaQMFTBhCkkNTelQER/DeB3AbzEzG/wjy0BsBnASgAvALiemY8SEQH4DIArAUwC+ANm/n7cayRVCqY49mjGr9dDAKFN6Gb1xTZFGNkqIdtYf11mrgkCcPe6NQBOKcX+Pg8vvzLdkdRmIhxKGjc33eO6XAKGezvRotC9Ny45Fzok+khIQ1l5Cn8L4C8BfDF0bAOAbzHzJiLa4P//YQDvAHC+/3MJgM/6v3NBt/2uESlDN6OkrSUTZ75y8VvY2o1dFQJwahdh2jnpylAEhJ3HNmY73dy7Mf5e994E9yuN2bLbQmiF7iFX8xERrQTwSGinsBfAW5j5EBEtA/BtZl5FRP/D//v+6Hmm6yfdKei29S4mEpes3rAz17Rit81qDq8S44RyQA8BqsW9yYSkMh0FuyQAsSYdALjsdUvw7qEVxnOTZHPbUmY2su0uLs/5C4KKKmU0nxUS9D8FcJb/9wCAg6HzXvSPdSgFIroJwE0AsGLFCucBhBvABwI67EuwTaaydeJFFZBJgNus+l2u1wY3k9HCZjCvh7BwQQ+On+xUhr1eDUSdET9TjRls3LrH2s/w3R8f0bYDDbC9566UHVCgCgpQkUWUUDeV4hCqTWl5CtzcojhvU5j5HmYeYuahpUs7+/maiDaAn2FuRagMDw4ok6m8Hmpm9oZwCWW07S8M2CmauOvViOAp3tVZNP0igTmnv9cDCG0KIZhlkDsxMamO+JmYaljPycb1EPRyyJqyewfb5mGkjRLKusiiML8pWin8zDcbwf/9kn+8DmB56Lxz/GOZEickVIlld737Itz1rosSJ6zZrgJtFU3c9WaYEYoMVT4e7AKiEUtR+V1USCMDuHnzzkStLk1UoYx0uFPaJ6+/KFVGvK4ibtnKT5hbFG0+2grgfQA2+b+/Fjr+J0T0FTQdzMfi/AlJiBMSpi140q24TUy5S/SMS70gHXFJacFKUxXGmidZmndGx+uxvRiywtZ0k8ZZbjKFVUH5CXOH3JQCEd0P4C0AziSiFwHchqYy2EJE7wewH8D1/ulfRzMcdR+aIal/mMeYTIlhedmfTXblsJN3bP8RK2Fha6dOy1RjBvc9dQA3XrJC2aoyz9fduHVPJnH8SXoxJH0t289N0qgh024gi0KNghCQm/mImW9g5mXM7DHzOcx8LzP/gpnfysznM/NvM/MR/1xm5g8y8+uY+UJmzqVJgqkYXV5b8LBJCjhlVw4rBNuCZlEnuQldU/uA/l4vthgdM3D/0weNCiGuZpGKWszgJqYaqcxIOr9LjSjzhK6iTDem3UAWhRoFIWBeFcQzFaPTfenqE1Opbd2BXfmFTVfhuTuvNFbtDIgKFp2TXMcsQ1sEjgBsvGY1rrt4AHHifMbgKR7o78Xzm67CrENY8+I+D59890Wx3d+ic9d1l1Ohey9nmTOPyDF9brL0j5iKGmZRqFEQAuZd7SPd9t1kq8/KlGQqPqd7XVOv4nBYbZTAT6HrKDY8OIDbH96Tyiw0eXIao+N1az9Hr1fDbVevbr0Ho+N13Lx5p/LcsJ/H1axXhDklrqEPgEzDXy+/YGmrllT0ePAaogSELJhXOwUTcbX905oEoit9G4JOYbpexYB6xxAOs42uIO9et6bVYvKoJuTUlqOTDdz60G5cfsHSWFMUAXjjijOwceserNzwKFZueBS3P7wHfar4WZwS4EnMM3mbU2zfyyzNSNuePex0XBCSMu92CjrCkSG6L3uaaA6XfIUAm1W8rohfXNmIrITVVGMG2549jOsuHsD9Tx3UKq+gOU+Yo5MN1HoIXg+1lRMJC/AkkTV5l8RweS+zigCSCCOhKEQphAgEqK48QRLzg43JKFpqwqZ6KdC+I9AJPF24pI0wWdznWe0m6hNTbQ16XJiZZfyTPg99CxcoBXhSU1Ce5hQXQZyVycr2Pkhms5AWUQoKkjRkUX0ZAWDkq7uMFUV1K32dIolW2ATQ8jvE9XMO2+NNfoBPr1vTuoZN/Z4eRTkMFyYmG61e0VGq2BxHd+9M/RzSYnMfyi7rIcwN5mU/BRtcVly6AnsAY8qQXmwqwW1TKhqA8nWDvgJxHdrWb96p3JGEC7TF1e6P1lPSYdr9xBWEq9rqV/d+593PIe4+mPqESME9IUyVCuJ1DS7mB50z1ERcFnPUxxHNa7j1od1Y5PUoXzcu2ewnE1MYHhyIjfwJOG3Bqdc5fWENXq0Hx6YaOLu/F8dPTLe6uUUJ72ouv2ApNj99sGPX5NUodjUd9RGEy5KUQVllvOM+k+J3ELJAlEIMNqvUJF+68Eo8zkmsWgGaSlXErdsDO/RAjJ1atSKe5WaOg01f5U9ef1HbvRo6d0lbS9DFfV4rRNVEFc0iVQwBlcxmIQtEKRiwFUau9YgW93mt6488sKtlfqlPTGHkgV0d189ypRe2Q8fZqXU7oJs378TY/iO4Y/hCY1/lqNDMo8RDFQRzVcxbVfS/CN2H5CkYsI2Rj8txCOPVCLddvRoAcPvDezrs8Y0Z7qgYmtVKjwhtPoxoHkN/r4dFXg/W+69vUnRf2n4AHx3drc0J2HjNautxxWUsZ2EWcc2KdrluHmWrk4xXMpuFLBBHswFdL2dV17XwatF0R8ORPSsNphegvctZVkXwXtB0i7NpBh+lRoTn7rwy1UrZprl9nIKK88/YvEZS8nDu5jleQQDE0ZwYFxtt2DRiEhTB4zar3GBXEgiXsOCdPDmdKCN5dLzeFrIaXBNw73gU5CWksa/bmIbiKsPG+RjyND/l4dyturlMmNuIUjCQ1Eare97lFyx1Xo0HwiUqeE11g0wEwjP4O83uw7U6qgoboRrM+5Ytu7QJciahmWdUTh7OXYkiEspEfAoGkthoVeWtg+dte/awsxDuIVLak4cHB1oOaxcC4Zmk7EaUS1+7ONXzAb3wZKDNlj48OBBbjVUnNE0VRtOSR52lPMcrCHHITiEGF9NI1BYc7QG9PsHKfoYZ60PRPmFuu3p1otV+VivOF35hd51wqY+gqmvgBxhZu0qb9R01C8VFeemEZp5ROXnkLEgUkVAmohQyRGcL/tCWnYlMPQEM4L7tBzB07pI2YRMVSLY+gUB4xoXRLu7z8Mupaa3Jxka5qBRl8NojD+zCujcth6mpQ9gstPI1eqVgEpp5J5tlnbNQVnKcIAASfZQJrn0SwtgWvwPiI1ps6hTZRjQF543tP6Ks428zHpsx9VAzIc4EAbh73RptWY4aUUeinCAIeiT6KAfCisBFsIcZ8Ms/2PZAjluZq8wOXo1w+sIFrbIU0RVnsBpd5PXgxPQsZrkpZK+7+NTqVzU+m/IUNmOOUwhAc2djamiTR0e1uURVkuuE7kCUQgKiJpEkCoGAtlW2jWKwKRcNuJsdGMArjdnW688w48EddQydu0QrjE9fuMBKsLhme0cJzEImf4w4YPVUsUSIUG3EfJQAGzNNHKoS2IEw71tYw/GT7WadLJOXPjq620oJDfT3an0VcQl84XnFOcOjTXai1WBNPS4C01JWAi6rVXVVVudSOVVQYTIflaIUiOgFAL8CMANgmpmHiGgJgM0AVgJ4AcD1zHzUdJ2ySmfrMp2TohL4eQmVj47u1voIohDMK/2w0DZl4QJoK4QXRWfiCt+D/j4PL78y3aE8brx0RUdUVlKyyiSuUkayS1a+MH+oqk/hcmb+eej/DQC+xcybiGiD//+HyxhY3JbbxiTi4mdQJV7lUYVzdLxurRAAtAS0bqUfLuBnysJ9csMVLSGvSkBrzDBOP20Bdt52qtHO6Hi9LVT16GQDPdSsz6Tzj6Qlq0ziKmUkS+VUwZUqJa9dC+AL/t9fADBc1kDiCuHZFMBTKQTTc5LmDtgWTgsUnS3h/IoggU9FY4bxoS07rfpamxLQovPfuHVPR+7CLAPHNLuNLMgqk7hKGcl5JNcJc5uylAID+CYR7SCim/xjZzHzIf/vnwI4q5yhxX+phwcHcN3FA6bw+g6CrGadcE3a/9m2QqdLBnONqKOaqsn+PMv6VIPovGyzdXWmJvZ/sqpGmmRsRV0nC6RyquBKWUrhXzHzGwG8A8AHiei3wg9y09GhXFIS0U1ENEZEY4cPH85lcDZf6m3PHnbOLxgeHMDI2lXwetpFqNdjF94Zxba0N2C/Su31aoli/hmdikG1Io1buQY7Hxt0c01KVqvqqq3OA6X+/KarWp9DQdBRik+Bmev+75eI6O8AvBnAz4hoGTMfIqJlAF7SPPceAPcATUdzHuOzKTPgYgroODcqPRXL7DQd334yMdXx/P4+T1tV9fRQtNNUYwa3bNmlLKvR3+tpV/DAqYgh05hNYbNJyndnaZLJKpNYMpKFbqbw6CMiOh1ADzP/yv/7cQAfA/BWAL8IOZqXMPN/NF2rrOgjwC0sNRz+ZxMiaBu9orvW4j4PrzRm25PYegggtDX1icta/jeRyJ6o89c0hyQkCfWV0EpBcKdq0UdnAfg7alYQXQDgy8z8DSL6HoAtRPR+APsBXF/C2FrERf+odhO1HsJMRGDa7jDCx29/eI9V9IpuR8OMjucHgjxakG54cAC3bNmlHNP9Tx1sUwrBa6vCS23NIyZla1r1L1aEpIrDVBCyp3ClwMw/BnCR4vgv0NwtdAU6E4HqmE0/58BfMTpe15p5okJTNwZT9m+0cmtwTHeuat7RHAJb80jSUN9gN1CVhDBBmMtImYsU6HYTJkGlqnUUXvGaHKdxHd8C4orzRXcdwe4hiqmJTpI8irj4/ThfTh65G4IgtFOlPIU5z+h4HQ/uqLcpBALais+ZTCi2phKbPIrw69xwyXLlObrjSbEJ9ZXwSUEoF9kpFIhqpcxohrcG6Ewo/b2etXAMm5V0O4bwriPwG9z/1EHMMKNGhBsuWZ5Z+Yjwa8Zl18puQBDKRZRCgdg4mXUmlI3XrHZ6rbDt36aL1x3DF2auBKKouqwlzdEQBCEfRCkUiO1KGcguxr1yMfMWORqCIJSHlM4ukCpVzywDKeMsCNWgankK85bKrdoLpkqF4gRBUCNKoWDmoyM1yC/Q7UmljLMgVAdRCkKuxNUzkqxkQagWohSEXDGV7B6YZ+YzQegGRCkIuaLzFxAgzmVBqCCS0SzkSpUazgiCEI8oBSFXqtZwRhAEM2I+EnJlvofhCkK3IUpByJ35GIYrCN2KmI8EQRCEFqIUBEEQhBaiFARBEIQWohQEQRCEFqIUBEEQhBZdXTqbiA4D2B86dCaAn5c0nCKZL/ME5s9cZZ5zjyrP9VxmXqp6oKuVQhQiGtPVCJ9LzJd5AvNnrjLPuUe3zlXMR4IgCEILUQqCIAhCi7mmFO4pewAFMV/mCcyfuco85x5dOdc55VMQBEEQ0jHXdgqCIAhCCkQpCIIgCC26TikQ0RIiepyI/tH/vVhz3jeIaIKIHokcP4+IniKifUS0mYgWFjNyNxzm+T7/nH8koveFjn+biPYS0U7/59eKG308RPR2f3z7iGiD4vHT/Pdnn/9+rQw9dqt/fC8RrS104AlIOlciWklEU6H38HOFD94Bi3n+FhF9n4imiehdkceUn+MqknKeM6H3c2txo3aAmbvqB8BfANjg/70BwH/VnPdWAFcDeCRyfAuA9/h/fw7Avy97TknnCWAJgB/7vxf7fy/2H/s2gKGy56GZWw3AcwBeC2AhgF0AXh855wMAPuf//R4Am/2/X++ffxqA8/zr1MqeU05zXQngB2XPIcN5rgTwGwC+COBdoePaz3HVftLM03/s5bLnEPfTdTsFANcC+IL/9xcADKtOYuZvAfhV+BgREYArADwQ9/wKYDPPtQAeZ+YjzHwUwOMA3l7M8FLxZgD7mPnHzHwSwFfQnG+Y8PwfAPBW//27FsBXmPkEMz8PYJ9/vaqSZq7dROw8mfkFZn4GwGzkud30OU4zz66gG5XCWcx8yP/7pwDOcnjuawBMMPO0//+LAKra/cVmngMADob+j87nb/xt6p9VTMjEjbvtHP/9Oobm+2fz3CqRZq4AcB4RjRPR/yai38x7sClI875003uadqyLiGiMiLYT0XCmI8uISnZeI6J/APBPFQ99JPwPMzMRdW1Mbc7zvJGZ60T0agAPAvh9NLezQvdwCMAKZv4FEV0MYJSIVjPzL8semJCYc/3v5WsBPEFEu5n5ubIHFaaSSoGZf1v3GBH9jIiWMfMhIloG4CWHS/8CQD8RLfBXZOcAqKccbmIymGcdwFtC/5+Dpi8BzFz3f/+KiL6M5ra3KkqhDmB56H/V+xCc8yIRLQBwBprvn81zq0TiuXLTCH0CAJh5BxE9B+DXAYzlPmp30rwv2s9xBUn1+Qt9L39MRN8GMIimj6IydKP5aCuAIDrhfQC+ZvtE/0u2DUAQEeD0/IKxmedjAN5GRIv96KS3AXiMiBYQ0ZkAQEQegN8F8IMCxmzL9wCc70eCLUTTuRqNxAjP/10AnvDfv60A3uNH7JwH4HwATxc07iQknisRLSWiGgD4K8vz0XTCVhGbeepQfo5zGmdaEs/Tn99p/t9nArgMwA9zG2lSyvZ0u/6gaWv9FoB/BPAPAJb4x4cA/M/Qef8XwGEAU2ja/db6x1+LphDZB+CrAE4re04p5/lH/lz2AfhD/9jpAHYAeAbAHgCfQcUidABcCeD/oblK+oh/7GMArvH/XuS/P/v89+u1oed+xH/eXgDvKHsuec0VwHX++7cTwPcBXF32XFLO803+d/E4mru+PabPcVV/ks4TwL8EsBvNiKXdAN5f9lxUP1LmQhAEQWjRjeYjQRAEISdEKQiCIAgtRCkIgiAILUQpCIIgCC1EKQiCIAgtRCkIQkYQ0XIiep6Ilvj/L/b/X1ny0ATBGlEKgpARzHwQwGcBbPIPbQJwDzO/UNqgBMERyVMQhAzxM8h3APhrAH8MYA0zN8odlSDYU8naR4LQrTBzg4hGAHwDwNtEIQjdhpiPBCF73oFmhdM3lD0QQXBFlIIgZAgRrQHwOwAuBbDer3ArCF2DKAVByAi/kdFnAdzMzAcA3AXgE+WOShDcEKUgCNnxxwAOMPPj/v9/BeCfE9G/LnFMguCERB8JgiAILWSnIAiCILQQpSAIgiC0EKUgCIIgtBClIAiCILQQpSAIgiC0EKUgCIIgtBClIAiCILT4/6Mxet+YPjmWAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import sklearn\n", "from sklearn import datasets\n", "\n", "# load data\n", "d = datasets.load_diabetes()\n", "\n", "X = d.data[:, 2]\n", "Y = d.target\n", "\n", "# draw original data\n", "plt.scatter(X, Y)\n", "plt.xlabel(\"X\")\n", "plt.ylabel(\"Y\")\n", "plt.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.2 数学原理\n", "有$N$个观测数据为:\n", "$$\n", "\\mathbf{X} = \\{x_1, x_2, ..., x_N \\} \\\\\n", "\\mathbf{Y} = \\{y_1, y_2, ..., y_N \\}\n", "$$\n", "其中$\\mathbf{X}$为自变量,$\\mathbf{Y}$为因变量。\n", "\n", "我们希望找到一个模型能够解释这些数据,假设我们使用最简单的线性模型来拟合数据:\n", "$$\n", "y = ax + b\n", "$$\n", "那么问题就变成求解参数$a$, $b$能够使得模型输出尽可能和观测数据有比较小的误差。\n", "\n", "如何构建函数来评估模型输出与观测数据之间的误差是一个关键问题,这里我们使用观测数据与模型输出的平方和来作为评估函数(也被称为损失函数Loss function):\n", "$$\n", "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2 \\\\\n", "L = \\sum_{i=1}^{N} \\{y_i - (a x_i + b)\\}^2\n", "$$\n", "\n", "使误差函数最小,那么我们就可以求出模型的参数:\n", "$$\n", "\\frac{\\partial L}{\\partial a} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i \\\\\n", "\\frac{\\partial L}{\\partial b} = -2 \\sum_{i=1}^{N} (y_i - a x_i - b)\n", "$$\n", "既当偏微分为0时,误差函数为最小,因此我们可以得到:\n", "$$\n", "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) x_i = 0 \\\\\n", "-2 \\sum_{i=1}^{N} (y_i - a x_i - b) = 0 \\\\\n", "$$\n", "\n", "将上式调整一下顺序可以得到:\n", "$$\n", "a \\sum x_i^2 + b \\sum x_i = \\sum y_i x_i \\\\\n", "a \\sum x_i + b N = \\sum y_i\n", "$$\n", "通过求解二元一次方程组,我们即可求出模型的最优参数。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.3 求解程序" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "a = 949.435260, b = 152.133484\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAABOZ0lEQVR4nO2deXgV1fnHPyfhAgkqAUELAQRbCwooARQrLoCtWKkSpaLUDbVFq9a6/FC0KmhRaN2te+tWVxQ0RdFaFSyKyiaLoqAoIMSNAgElQbKc3x9zJ8ydzHJm7tw15/M8eZLMneWcufd+5z3vec/7CiklGo1Go8kvCjLdAI1Go9FEjxZ3jUajyUO0uGs0Gk0eosVdo9Fo8hAt7hqNRpOHtMh0AwA6dOggu3fvnulmaDQaTU6xePHi/0kpOzq9lhXi3r17dxYtWpTpZmg0Gk1OIYRY5/aadstoNBpNHqLFXaPRaPIQLe4ajUaTh/j63IUQrYG5QKv4/tOllBOFEI8CRwFb47uOlVIuFUII4E7gOKA6vv39oA2rra1lw4YN7NixI+ihmghp3bo1Xbp0IRaLZbopGo0mACoTqj8Aw6SU3wshYsDbQohX4q+Nl1JOt+3/S2C/+M8g4L7470Bs2LCB3Xffne7du2M8LzTpRkrJpk2b2LBhAz169Mh0czQaTQB8xV0amcW+j/8bi/94ZRsbCfwzftx7QogSIUQnKeVXQRq2Y8cOLewZRgjBnnvuycaNGzPdFI0m66lYUsnNr67iy6oaOpcUMX54T8rLSjPWHiWfuxCiUAixFPgWeE1KOT/+0o1CiOVCiNuFEK3i20qB9ZbDN8S32c85TgixSAixyE08tLBnHv0eaDT+VCyp5KrnP6CyqgYJVFbVcNXzH1CxpDJjbVISdyllvZSyH9AFOEQI0Qe4CugFHAy0B64McmEp5YNSyoFSyoEdOzrG4Gs0Gk1OcPOrq6iprU/YVlNbz82vrspQiwJGy0gpq4A5wLFSyq+kwQ/AI8Ah8d0qga6Ww7rEtzVrunfvzv/+97+k99FomiMVSyoZPHU2PSbMYvDU2Rm1iJ34sqom0PZ04CvuQoiOQoiS+N9FwC+AlUKITvFtAigHPowfMhM4UxgcCmwN6m/XaDQak2x0edjpXFIUaHs6ULHcOwFzhBDLgYUYPveXgCeFEB8AHwAdgMnx/V8GPgdWA38HLoi81Wli7dq19OrVi7Fjx/LTn/6U0047jddff53Bgwez3377sWDBAjZv3kx5eTkHHngghx56KMuXLwdg06ZNHHPMMfTu3Zvf/va3WCtePfHEExxyyCH069eP8847j/r6ercmaDTNnmx0edgZP7wnRbHChG1FsULGD++ZoRapRcssB8octg9z2V8CFybfNAuXXAJLl0Z6Svr1gzvu8N1t9erVPPfcczz88MMcfPDBPPXUU7z99tvMnDmTm266ia5du1JWVkZFRQWzZ8/mzDPPZOnSpVx//fUcfvjhXHfddcyaNYuHHnoIgI8//php06Yxb948YrEYF1xwAU8++SRnnnlmtP3TaLKIZCJJstHlYcfsSzZFy2RF4rBspkePHvTt2xeA3r17c/TRRyOEoG/fvqxdu5Z169YxY8YMAIYNG8amTZvYtm0bc+fO5fnnnwdgxIgRtGvXDoA33niDxYsXc/DBBwNQU1PDXnvtlYGeaTTpwXSrmNa36VYBlMSvc0kRlQ5CnkmXhxPlZaUZFXM7uSHuChZ2qmjVqlXj3wUFBY3/FxQUUFdXF3jlppSSs846iylTpkTaTo0mW/Fyq6iI4fjhPRMeDpB5l0cuoHPLJMkRRxzBk08+CcCbb75Jhw4d2GOPPTjyyCN56qmnAHjllVfYsmULAEcffTTTp0/n22+/BWDz5s2sW+eatVOj8SXfI0nKy0qZclJfSkuKEEBpSRFTTuqbVVZyNpIblnsWM2nSJM455xwOPPBAiouLeeyxxwCYOHEiY8aMoXfv3hx22GF069YNgAMOOIDJkydzzDHH0NDQQCwW45577mGfffbJZDc0OUqyLo90EIVbJdtcHrmAsEZxZIqBAwdKe7GOjz/+mP333z9DLdJY0e9F9jJ46mxH4SwtKWLeBMeYh7RjfwCB4VbR1nfyCCEWSykHOr2mLXeNJofRkSQaN7S4azQ5jI4k0bihJ1Q1mhwmGxfPaLIDbblrNDmMdnlo3NDirtHkONng8si2XOYaLe4ajSZJciEcszmife4RcNxxx1FVVeW5z3XXXcfrr78e6vxvvvkmv/rVr3z3GzJkCPaQUjt33HEH1dXVodqhSSTbFw+li1xI7NUc0ZZ7EkgpkVLy8ssv++57ww03pKFF/txxxx2cfvrpFBcXZ7opOY22VneRC+GYzZG8sdxTYUXddttt9OnThz59+nBHPL/N2rVr6dmzJ2eeeSZ9+vRh/fr1CUU2/vznP9OzZ08OP/xwxowZwy233ALA2LFjmT7dqCXevXt3Jk6cSP/+/enbty8rV64EYMGCBfzsZz+jrKyMww47jFWrvC2fmpoaTj31VPbff39OPPFEamp2fZl+//vfM3DgQHr37s3EiRMBuOuuu/jyyy8ZOnQoQ4cOdd1P44+2VneRjbnMNXliuafCilq8eDGPPPII8+fPR0rJoEGDOOqoo2jXrh2ffvopjz32GIceemjCMQsXLmTGjBksW7aM2tpa+vfvz4ABAxzP36FDB95//33uvfdebrnlFv7xj3/Qq1cv3nrrLVq0aMHrr7/O1Vdf3Zhx0on77ruP4uJiPv74Y5YvX07//v0bX7vxxhtp37499fX1HH300SxfvpyLL76Y2267jTlz5tChQwfX/Q488MBQ96w5oa3VXejEXkkgJTQ0QGGh/74ByQvLPRVW1Ntvv82JJ55ImzZt2G233TjppJN46623ANhnn32aCDvAvHnzGDlyJK1bt2b33Xfn+OOPdz3/SSedBMCAAQNYu3YtAFu3buXkk0+mT58+XHrppaxYscKzjXPnzuX0008H4MADD0wQ5WeffZb+/ftTVlbGihUr+OijjxzPobpf1GSzv1qlbUGs1Wzta1Tt0om9QiAlvPwyDBoEDzyQkkvkheWebiuqTZs2SZ/DTB1cWFhIXV0dANdeey1Dhw7lhRdeYO3atQwZMiTUudesWcMtt9zCwoULadeuHWPHjmXHjh2h94uabPZXq7ZN1VrN1r5G3a5sCMfMCaSEV16BSZNg4ULo3h06dkzJpfLCck+Fz++II46goqKC6upqtm/fzgsvvMARRxzheczgwYN58cUX2bFjB99//z0vvfRSoGtu3bqV0lLjC/Loo4/67m9NK/zhhx82lvjbtm0bbdq0oW3btnzzzTe88sorjcfsvvvufPfdd777pZJs9lertk3VWs3WvmZru/zI1lGQL6aoH3oojBgBGzfCP/4Bn3wCJ5+ckkvmheWeCp9f//79GTt2LIcccggAv/3tbykrK2t0oThx8MEHc8IJJ3DggQey995707dvX9q2bat8zSuuuIKzzjqLyZMnM2LECN/9f//733P22Wez//77s//++zf69w866CDKysro1asXXbt2ZfDgwY3HjBs3jmOPPZbOnTszZ84c1/1SSTb7q4O0TcVazaa+WhcaueWCzYb3wI1sHQV5IiX8+9+Gpb5ggWGp/+MfcOaZELDQT1DyJuVvtqyQ+/7779ltt92orq7myCOP5MEHH0yY6MxFok75m81paqNuW7b01SntrhPZ8B64kS33Ugkp4dVXDVGfPx/22QeuucYQ9ZYtI7tMs0j5my0+v3HjxvHRRx+xY8cOzjrrrJwX9lSQ7uiKIA/+qNs2fnhPxk9fRm39LiMqVijSHkni5Iaxk+0RLtk0CnLFSdQffBDOOitSUVfBV9yFEK2BuUCr+P7TpZQThRA9gGeAPYHFwBlSyp1CiFbAP4EBwCbgFCnl2hS1P+swfeAad9KZ7CroUD4lbbMPjjMwWPYSQAE5kQ8mq9MbSwn/+Y8h6u+9B926ZUzUTVQs9x+AYVLK74UQMeBtIcQrwGXA7VLKZ4QQ9wPnAvfFf2+RUv5ECHEq8BfglDCNk1IihAhzqCYiUuW2S9dIK0xx5ijbdvOrq6htSLyHtQ1SuTh0VLgJY1a6NFzIynh6J1F/4AEYOzZjom7iGy0jDb6P/xuL/0hgGDA9vv0xoDz+98j4/8RfP1qEUOjWrVuzadOmlImLxh8pJZs2baJ169YZa0Oy0RGZHspn+vom+ZD3Pavi6U1RHzwYjj0WvvzSEPVPP4Vx4zIu7KDocxdCFGK4Xn4C3AN8BlRJKeviu2wAzDtcCqwHkFLWCSG2Yrhu/hekYV26dGHDhg1s3LgxyGGaiGndujVdunTJyLWjiI5ws1gLhKDHhFkpd0dkiyshX/K+Z3xuTUp47TXDUn/3XejaFe6/H84+OysE3YqSuEsp64F+QogS4AWgV7IXFkKMA8YBdOvWrcnrsViMHj16JHsZTQ4TxqVix2koD1AfHxGmOpwuSldCshFhGRfGXEZKeP11Q9TfeWeXqI8dC/EFidlGoGgZKWWVEGIO8DOgRAjRIm69dwHM8XIl0BXYIIRoAbTFmFi1n+tB4EEwQiHDd0GTasKKSrJiFIVLw26xFgjRKOwmQR8Yqpj9r6mtpzB+3dKQFnNOxnjnA06ift99hqWepaJu4utzF0J0jFvsCCGKgF8AHwNzgF/HdzsL+Ff875nx/4m/Pltqx3nOYopKZXzhiykqfr7vsMdZiWrlcXlZKfMmDGPN1BE0uHwUo/aBW/sPxkjBtNjDiHGurijNWUxRP+IIOOYY+OILQ9Q//RTOPz/rhR3U0g90AuYIIZYDC4HXpJQvAVcClwkhVmP41B+K7/8QsGd8+2XAhOibrUkXYUUlCjFKxSRgutLTRi3G2TIxm/dYRf0Xv4B16+Dee2H16pwRdRNft4yUcjlQ5rD9c+AQh+07gNQkS9CknbCikgqXShSTgOkKp4tajEuKY2yprm2yPStivPMBKWH2bMP98vbb0KWLIernnJNTgm4lb1aoalJD2GiPqKJEop4ETFfUSJRRMhVLKvl+R12T7ZlY6Zp3SAlz5hii/tZbUFoK99wD556bs6JukhdZITWpI6xrJJvjqq0++HkThqUsSiaq/jsthAJo07KFnkwNi2mpH3UUHH00fP65IeqffQYXXJDzwg7actf4oGLpekXFZCquOtOJ5KLsv5srZ2tNUzeNxgcp4c03DUt97lzDUr/7bsNSz+BivVSgxb0ZoyqAXq4RvxC9VAuqUx+ArAgbVOm/ynuQLQuh0k3kD2jT/TJ3LnTunLeibqLdMs2UKEIVIbMhem59mDRzRdrbFCZNgup7kM0urlQR1ecTMCz1IUNg2DAj6uVvfzPcLxdemLfCDlrcmy1RiXImQ/Tc+lDl4q5IVZvCClHUVZ/CtDtbqxpF8vk0RX3oUKPi0V13GaJ+0UV5Leom2i3TTIlKlJN1Gbi5VVSG42HamgrCpklQfQ9SMX+Q7Stek/p8/ve/hvvlzTehUydD1H/3u2Yh6Fa0uDdTovLjJhM37iQw46cvA0ljdIiX6Lj1oV1xjB21DWlLDRtEiKxC7ZQKARLfAzcRXrRuM7OWf9UY+15SFGPSCb2VhTnIAykTk9OhPp92Ub/zTkPUi/J7bsIN7ZZppkTlx03GZeAkMLX1sknYn304broTKqtqsOeSLooVMvH43mlNDau66tXuvnESdvt74CbCT7z3RcKipqqaWsY/t0zZtaL6QLqm4gMunbY0Gt93AAJ9PufONfzpQ4bAypWGqH/2GVx8cbMVdtCWe7MlylC9sFExQdwq5r52S1ZiVBKS0CQpV6rE3G7JDu3VkRmLK11HCub+TpYoQKEQNEjp+B4EuUdBioCoWMYVSyp58r0vmhSOSlWiNStKn8+5cw1Lfc4c+NGP4I47jFzqzVjQrWhxb8ZkOgWsm8C47QvOlqwp7EErCoVxNzi5SWYsrmTUgFLmrNzY5FwqhakbpGTN1BGOrwW5R6D+MFBxp9386irXioDpmBdw/Xy+9ZYh6rNna1H3QIu7JmM4CUysUCT43GGX6FQsqXQVuqCTq2EnFN3cJHNWbnR8uKgUpm5bFHN9zekemSMVJ1TnTFQsY697qjIvYL1OJFhFfe+94fbb4bzztKi7oMVdkzHcBMZtmykYTgSdCE51hIvfdivbd9ZRsaRSuWD30F4dmbZwPbX1iRIfKwiWa8Zv5OY2ahCgNC8Qmevm7bcNUX/jjV2iPm4cFBcnf+48Rou7JqO4CYx92+Cps10t4FihYGivjo2TrCqFMcKKcdAoDhW3Sm29t6/c6R4N3Kc917+4InS0jApuo4bTDu2mZOEnva7ALuq33WZY6lrUldDirskJvISivkEybcH6RleOSgm9sKGgQUM/3cr82QkqhFHMl7j5ya3bS4pjtGpRwNaa2vSlR5g3zxD111/Xop4EWtw1SZOOOGgvC7hB4lphyc09EDY+P2iUkX1/cPaXe/ndU4FX/Lw18mdLdS1FsUJuP6Wfax8jy5H/zjuGqL/2Guy1F9x6q1EgQ4t6KEQ2VMAbOHCgXLRoUaaboQmBUzRIUaww8rhylagTNwQ4RqNkYnFO2Q3/cSy60a44xpLrjknptVUWUBW6bPeLRkrqXtpF/cortagrIoRYLKUc6PSattw1SeE3mVaxpJJJM1c05ntpVxxj4vHBfcPm/pc/u8xRfLxwcw8k69oII2hVDsLutj3Kh4/94eh2D922+7mN3O6lZx/efdcQ9f/8xxD1W24xRL1NG6X+ZDKlcy6gxV0TGOsXyysOumJJJeOfW5YQ1rilutZIMUDwMDlz/yAWfKrSDoQN/1P1TycTXugkfCohmeBuuYetIOXUh3bLFnPU0/cYot6xYyBR9zovZEdenGxBpx/QBMK+hN6NziVFrhWEzOiQMJjpDgqFPfHALszXUpl2IGzWQtVl9WHP75ahUmUhVFGskDGDukZaQcrah/6VH3P/k3/iqLNHwpIlcPPNsGYNXH65srA7nRfSl2Y6l9CWuyYQKhagKQaXTlvquk8yYXJuFnwqfP1uhA3/U52QDXt+N+Fzs8idUh8M3Kd9pBWkyipXcsm8pzhqzfv8r7gtNw05h6tfuiuQoDudV3V7c0WLe8Sk0heYzLmjapfXF0hAwrm98qkkm34302X8kgn/U/H1u52/QAh6TJgV+KFQLyVFsUKlh2FUaSl+sW0Nv/n3owxZs5hNRXtw05CzebxsBO33asfVIYUdmm9lqqD4umWEEF2FEHOEEB8JIVYIIf4Y3z5JCFEphFga/znOcsxVQojVQohVQojhqexANhFp9ZgIzx1lu9y+QKUlRU0KTo8f3pNYQVP3Saww2EpKN9JR6NqNVFdHcjo/GCLt9R56vT9py5Q5fz788pc8eN8fOPCbT5kyZCxHnP8QDw4aBW3aJH2PmmNlqjCoWO51wOVSyveFELsDi4UQr8Vfu11KeYt1ZyHEAcCpQG+gM/C6EOKnUsrgMWw5RiqXYSdz7ijbFSSm2Ty3U7QMGKtOczXaIdUjB/v5nUIXnd5Dr/cn5Yni5s+H66+HV16BDh3gL39h3uByXnprAzVVNZ4rhoOQ6VFbruAr7lLKr4Cv4n9/J4T4GPC6iyOBZ6SUPwBrhBCrgUOAdyNob1aTSl9gMueOsl3WL5a51N86maUyxE822iFVlYmCnjPVYmk9f48Jsxz3sb+HGRG+BQuMkMZXXoE994SpU436pLvtxvHA8YN/GvklM53RNBcI5HMXQnQHyoD5wGDgIiHEmcAiDOt+C4bwv2c5bAMODwMhxDhgHEC3bt3CtD3rSKUvMJlzR90upwlNFYH2ymuuOpJwejBcOm0pl0xbGtoyTMU5Va4ZRICDvIdpE74FCwxL/eWXm4i6JvMoh0IKIXYDZgCXSCm3AfcBPwb6YVj2twa5sJTyQSnlQCnlwI4dOwY5NGtJpS8wmXOnol1Bw9Gsfn83VEYSbvncwX8uwa0gdDLnDEOYOZCs8jMvWAAjRsCgQYYrZsoUI6Txyiu1sGcRSpa7ECKGIexPSimfB5BSfmN5/e/AS/F/K4GulsO7xLflPakcEidz7lS0K6irRyWE0msk4VfNyMSrDqjbSMPvoRJ15SG3B+Plz7ov7soGP/ObT8wiNvnPDF41n61Fu7PhDxPofePVsPvuaWtDPpHqVba+4i6EEMBDwMdSytss2zvF/fEAJwIfxv+eCTwlhLgNY0J1P2BBZC3OclI5JE7m3KGWh3sQ1NXjJ6BeVmjQvDJO1/Iaaaik5Y0yhtrtWvVSerq2MuZnXriQry+7iiFvv8GW1rvz1yPP5LH+v6Jht92Zsnob5WVa3IOSjlW2Km6ZwcAZwDBb2ONfhRAfCCGWA0OBSwGklCuAZ4GPgH8DFzaHSJlcJJkQyaBuAi+r3C8sT3XpvNe1vEYabmGHfucMi9fq2qxaabloERx/PBxyCEWLF/DXI8/k8PMf4t6fjWZ7q+LsamuOkY5VtirRMm9DkyLzAC97HHMjcGMS7dKkgWRCJIO6CdxC9FRirYNYzW4PGK+Rhj0CyF7GLmrftl/is4yvtFy82Ih+eeklaN8ebryRwzfux3etmmZpDNNWnfQrPats9QrVZkyyH7AgboJkfMZe5d6k7f9RA5zb5PZwMSs4mW2645R+odupSqmPGyhjKy0XLzaiX158Edq1gxtvhIsugj32YI+ps/kugogrnfTLIB2rbLW4N2PSvYzb72FgrwAkJWytqaVtUYxYoUioGepUJFoCc1ZudL02NK1Fai1MYQrNlJP6euYuTxav6kwZiYB5/31D1GfONER98mT4wx9gjz0ad4mqIEfK663mCJEVOPFAi3szZvzwnk1S8gYtshwVdovOWtCiqqaWWIGgXXGMqupazwnQyqoaBk+d7Wht2x8uTnVZVYQmWbeC00Iwv5qvKgRul4KoO7U5mRFNupN+ZasLKB3RT1rcmzv22RT3ub6U4jdpWtsgKW7ZorFakVkM2wnVoX4YoYnKrRB15Eugdi1ZYoj6v/4FJSXw5z8bot62bcrbnM7RYra7gFId/aTzuTdjbn51VYKrA8LnWndbIOT3mknQNAp+ES4qkQduguIlNH5RDip9TQVK0RdLlkB5OfTvD//9ryHqa9fCNdf4CntUpHMxVnPP+64t92ZMVENkLwsJ1NIUqMSaW0XX7toI048wfk+ve5ZJS9HzvVy61LDUKyoMS/2GG+Dii9Mm6FbSuRirued91+LejIlqiOxnIan4tb0mGcFZdM1hrZuLxq8fYYTG655lcrLQqV0HfPM5Vy6YBn+ZZwj59dcbol5SktK2+JGuxVjNPe+7FvdmTFQz9mEsJL9shtZoGVN0wTlNsFc//CbUggqN17XcKk+lw1K0tuuAbz7n4nee5thP3qV2tz2yRtTTTToiUrIZLe7NmKiGyH4WUhTZDFVcHvZ+QPDMlX543TM3F1E6LMXyslL2+OQjCv98I0eteJvvWrfh4/MuY/+p1zY7UTfJhnw8mURIn9Vy6WDgwIFy0aJFmW6GJiROuV/M1acQTa1TN9dLaUmRa0y6V0RN1Kl8K5ZUcv2LKxJCOCFNdV2XLTP86M8/b7hfLr0U/vjHZivqzQkhxGIp5UCn17TlrokkFrh1rKBRwEuKYkw6oXfCOZKt/Rpm0tTrNS8rPuj9cEts5nQfwmC9B9aY+D/3qGfYs/cbor7HHjBxIlxyiRZ1DaDFvdkTRVUku7D9UNeQsE/YCTSVbJBeLg+/CBynyc4w98MtRr9NqxaRCLu1PfVS0uvbNVz8wtMM++QdanfbndjEiYal3q5dUtfS5Bc6zr2Zk2zcttvxl0xbmnSct9/CJr/JMZVsj3brPkxsdBQhdyqFRHp9u4Z7X7iJfz/yBw5fu5Q7DxvD8Zc9biT50sKusaEt92ZOsnHbYV0fybQN1HzmKrHwdss/jFCHCbmz59H5fkddYxoIeyGRnhvXcvG8pxmxah7bWhZz52FjeOjgkWxrvRviB9dLaJo5WtwDkK15KpJpV7Jx22FcH6q4ndtrEtWO6RJym/S1W/5hhDpoyJ1XHh2Tmtp6pj/2bx56458M+3Au37Us4s7DTuXhgSPZWrSrOEZzidnWBEe7ZRRJprBFNrdraC/n+rVDe3VUsmLDuD5UcTq3YFdysCD3vryslCkn9aW0pAiBe4GQMMvjVc9t4udu+unGtdxdMZUn7vwtR6x5n3sPH8Ph5z/M7UecniDszSlmWxMcbbkrkq2pSpNtl1uK3DkrNypZsWFcH244jUCmnNTXsYhGGJePysRu2NjoIJPGbg+7/Tau44/vPMNxK99me8vWPDr0dMZOv5PO62rY7dVVbI0wg6Qm/9Hirki25qnw85n7iZTX8bef0k/J3RDU9eGEm3/fzK3uFLOezMPV696kenm8/aFpF/W7fzaapw4bxYTTDoP27Slvnx1ZDDW5hRZ3RbIhT4WTILm1q21RLKmEXU7l5wqFSIgcccqXbu4f1P/vNwKJ8uGa6VSwpo++y5efc/E7zzBi5dtUt2zNQ0ecyr1lx1PcaW8maKtckyRa3BXJdJ4KN0EaNaA0oZqQ2S4hwifssvbL3FdVDMNavX7iHeXDNdMutvJWWxm44D46v/Yi1bHW/HPob+h47QR+N7QPv0v51TXNBT2hqkjQSbOocROkOSs3OraryiECA5wTdvn1Kx15sf1yq0eZBzxjLraPPoIxY6BPH7q8M5uCq65ity/XM3b2E4wY2ie119Y0O7TlHoB0pSp1wkuQnNoVJImVX79Ul/4nE5KpOoJIJhTVbJ9bNqVkJn492/HRR0ZhjGnToE0bmDABLr8c9txTue0aTVB8xV0I0RX4J7A3RrDCg1LKO4UQ7YFpQHdgLTBaSrlFCCGAO4HjgGpgrJTy/dQ0Pz9QEQs3t0SBEFQsqXQM6bOLZaxQsP2HOnpMmKUsjhVLKh2LUZttsu6n6rrx6q/b9mTXGPilMkh24tepn3z8sSHqzzwDxcWGqF92GXTooNzudJOtazk0wVGx3OuAy6WU7wshdgcWCyFeA8YCb0gppwohJgATgCuBXwL7xX8GAffFf2scUBULt2IW9VI67m8Xy+KWhWzfWU9VTa3ndex4WbrWGHlVP7Zff53aEsUEqF9seeuYs4fSLnbVO+v8+2kX9SuvNCz1LBZ1yPxEsyZafH3uUsqvTMtbSvkd8DFQCowEHovv9hhQHv97JPBPafAeUCKE6BR1w/MFVX+26RsvFE0rWLv5v8vLSpk3YRi3n9KP6p1NhU3Fb+7lh37ivS8aFxN5Fau2EsZ/73WMas1SP3/6luraJou/nBaIOa0mbTz/ypVw2mnQuzfMnGmI+tq1MGVK1gs76Jqj+UYgn7sQojtQBswH9pZSfhV/6WsMtw0Ywr/ectiG+LavLNsQQowDxgF069YtaLvzhiCTe+VlpaGq/XhZ336i55dewLTuhACn0gCFcbeRaf2GaYfba+a13Wq3Wi3ukuKYqzCb2C1wP2vf5Meb1nPlwunw19mGpX7FFYal3tF59W+2kq1rOTThUBZ3IcRuwAzgEinlNmGxIKWUUggRqOqHlPJB4EEwinUEOTafCBriFyYk0C/plRdm+TivN8hLAE23kZ9Ihknda8bd29syaeYKfqhrSBD9WIEgViiorff+qFnvlZ+o7btpA3945xlO+HgusnXrnBV1k2xYy6GJDqVQSCFEDEPYn5RSPh/f/I3pbon//ja+vRLoajm8S3ybxoGgIX5hQgLdvpwifj4vystKOe3QbjR1BqnhJMB2wqTuLYoVUu9SRayqprbJNWsbJG1atmgM+XRyb0HivXK7b/tu2sDtL97Caw9dwLGfvsdnZ55Hi3VrYerUBGFXdRllC1GGm2oyj6+4x6NfHgI+llLeZnlpJnBW/O+zgH9Ztp8pDA4FtlrcNxobQePn7fuXFMVoHSvgUo/86W4JuE47tJvSRNnk8r6cdmg3V0Ek3o4gAmxSKASjBnjXTjXdI+b1zXtUGtCi3FpTy7wJw1gzdQS3jj7IV8js923fTRu47aVbee2hCxj+6bv8/ZATKar8gp8+em8Taz1bE815kem1HJpoUXHLDAbOAD4QQiyNb7samAo8K4Q4F1gHjI6/9jJGGORqjFDIs6NscD7gFG6mmsIW3HO5uEU3JBsjXrGkkhmLK12FuihWyKQTejtewyuhGBhumxmLKxm4T3vHkEl7FSJTgJ1WzpptaR0rcPSvuyU8c7sn5t9/u/9lLpz3NCM/+i8/tIjx94PL+fshJ9G6tBPnu7hgMr0KNiyZXMuhiRZdIDvNeBWTDvqlClM0OgzJFJpWKZVnnsfeZpX+OT0oIZqi3HzyCUyejHzySWoKY/yzbAR/P+QkNrUp8T1fjwmzHOcpBLBm6gj1Nmg0HugC2VlEEIvOb0FJmKLRYXA7nwDXh4i17W3jrqOq6tpA0TIq0RtelmboxTiffgqTJ8MTT0CrVojLLuPNY0/n8YWb2FxVo5Ru1y06p6Q4BujFQprUo8U9zaiGm/m5XFRXjkZB0CgKe9uramopihVy+yn9AqVFSCZ6I6h7oWJJJU8/+QajX32M8hVvIlu1pMVll8H//R/svTfHAccdrXw6x7BQc7teLKRJBzpxWArwipLwS5Bl4regxC12XSUCJihuURRDe3X0Lepsb3uQ6kphozeCRqm89q+3kWPH8uRtZ3Pcynk8NHAkQ85/mIrfXAJ77+15rBtba5xj6rfW1OrFQpq0oC33iHGzyhat28yclRubVBQCZ8Hys/DdXpdEb/2Vl5WyaN1mnp6/nnopKRSC/t3aJqQathd1dmu7PUe8V3WlMBPBgazi1ath8mSGPv44tQUxHh44kgcHncT/2rQDYNLMFSmpTZuOxULa7aPR4h4xblbZk+990ShiEhpFzc1/6+eS8CoeDdF+ue3RMvVSMu+zzU32M61Pv7abwq1SXSmoe0VpTmP1arjxRnj8cWjZkkcHnMADh4xi427tEo6rqqkNnIvHxCvLZRDXVBi020cDWtwjx8uitv/vFfUxtFdHpi1cn7CiMlYoGi18L/GI+sutugwfgpXnS9aCNe+ZWSXKK6b+y6oa+OwzY6L08cchFoOLL4YrruCRRz9io8I1g4Qyeo06Fq3bnPCwh2gXC+VqGKYmWrS4R4xfLhYrpog5ifG0BeubipXlXy/xGDx1dqRf7iDuAnt5vjBpjFUsWKcYeDe6bfmKKxbPgFteSxB1fvQjAMYPr1cK14Rg98Jp1GGOgqytFeC5kCsoOkeMBrS4R46TRe0X1eJkadU2ND2itkEquSyi/nKrPrCsk7kq7hS3NMbVO+scc9RbURlNdNvyFRe9O42TPpyNaBmDP/zBEPVOiUlKnR5G1TvrXBdCVSyp5PoXVzS+XlIUY9IJvUPXipXAnJUbfY9VReeI0YAW98hxEoqhvTo61jk1hTCI6PrtW7GkkgIXF0XYL7dK8jAIPplr7jtp5opG3zbsSr9r3cfE6opxo2vV11z0zjRGffgG9YUtWDvmHH5yyw1UfN3AzY99zJdV7zcZTdgfRm6LzYb26sj46csS3GVVNbWMf26ZZ3vNz0I61iZkut6vJjvQ4p4CnKzWgfu0d3VTBHHleAm0KUhOwq4aQujURjc/sZ2guV7AuFc3v7oqQdxBrdCHnURRL6TFxX+gxZVX8pNOnQLPQ7i5lm5+dZVjZkn7qMqpvU6RUiZRWtVRlCTU5D46/UAIog4zcxKtWIEAQYKQ+C15d1uuXygEYwZ1Zc7Kja5tVkmLYLWancI5wyaZUl2q79a/LlVfc9G7zzLqwzdoEAVMG3Ace/35Wo49ZkBjuy9/dpnjQy9oqga3tgZpb5T3TtO80ekHIiQVYWZulpbTNrcydF6uCjM5l1ebg0RYCIxl9FIai3KSfcCp+ojtrgu7qD9RdhwVx5zO2ScP5ljLA8ltNON0Tr8Ht9coy6+9JmaklLaqNalEi3tAUhVm5jYB6XdOlcRcbkUtrG32K5Nnv86W6l0pBZIVJlUfsSmsXaq+5sJ3n+XXcVF/4dATGP3c3ZzduXOTFKR+E68qRb7NBWhmnpzCAkG9bcI7ViBc22sn6sRuGo0TWtwDkm1hZn7iVRQrdH3d2ma3OHEzh7pKOoSwlqiqj3hi32K2XnMz5ctfa7TUHx08mkvHDoXOnX37aMf+AFFZgFZVU0usQNA6XnAcEqNl7AnT7NWf9MSmJl1ocQ9ItoWZeYlXaYl3TvW2RbHGv93cFub2MHVMgwq86/5r18JNN3HMI49QX1DI84NO4NaykRR27er7IPEq0Wf3c6suQKttkOxV3JIVNyRa304J0wAKBDRI//TIGk2U6MRhAfFLZpXu0mpuDxVz6F9eVsr44T2NCVob2+Px5Ob+bufxug40raEaWRKsdevgvPNgv/3gscfg/PMp/PwzTn7ned6756zG/nnh9n7dOvogx0VVqjg9CNxGUQ1y12dEC7smXWhxD4hXKbJMlFZzEq9YoWD7D3WNDxiA3Vo3HaTV1stGEfZ6aFUsqWT7D3WB2pWUm8oq6o8+Cuefb6QO+NvfoDSYOAYpHeeWsdIJpweBV5911kdNutFuGRe8oibcXAiZyOlh91eXFMf4fkddk4RXfn53r4gd1aX5VkK5qdatgylT4OGHQQgYNw4mTIAuXYKfy4Jq8rEwC9Cs+K1X0Mv/NelEi7sDYcMd/SJOUoVVvAZPnd1k2bxZXNrJry7jx1gXLFlxylPjR+BJwy++gJtuilzUwxB0AZoVt3QKJnr5vyadaHF3IKwF7hdxkg68Yt3dIme8Hl5Brc1Ak4ZffGFY6g89ZIj6735niHrXroGumSrsoze/sE+3dAqgo2Q06Uf73B3wigzxmiz1izhJNWbpPSdMX7PbxKmbTziItWnGevsK+xdfwO9/Dz/5iWGt/+53Ro71e+7JKmEPM39SXlbK0onHcMcp/ZT8/BpNqtCWuwNuvlOzHBw4W7ulPgU0whAk1YFb6T0wMi1eOm2pp1hbUxD7xWqDpKa2IeF4p/wqCaxfb1jq//iH8f9vfwtXXdUo6NlUPSjZ+RNVP3829VmTX/ha7kKIh4UQ3wohPrRsmySEqBRCLI3/HGd57SohxGohxCohxPBUNTyVuEVN2IXTbu2GrfnpRlDr0cuFsqW6tvEcbpQUx+h3/X+4ZNrSxmtW1dSChHbFsQQrdIdN2D3bsH49XHAB/PjHhrCfe65hqd97b4Kwj5++LKGv46cvS2mkkVfYarpK4aU7ukrTfFBxyzwKHOuw/XYpZb/4z8sAQogDgFOB3vFj7hVCFDocm9U4hc+5WcTWL3uQsDsVghZSTnbCbkt1bRNfMRgWeXHLFqyZOqIxtlyp0Pf69XDhhYb7xSrq990H3bolHHf9iyuaZFusrZdc/+KKpPrkhp+wqhYyTwZdKFuTSnzdMlLKuUKI7ornGwk8I6X8AVgjhFgNHAK8G76JmcE+rHbL8Gf/sget+emFn+/fPpT3i9aIsi2e+WA2bNjlfpESzjnHcL/ss4/r+Z0KY3htTxY/t8v44T0Z/9yyhKIpTvljkiHbUllo8otkfO4XCSHOBBYBl0sptwClwHuWfTbEtzVBCDEOGAfQzWbFZSOZKIAQ1vdv9eFu/6HO0RoP0xYrTte6tt/uHPvQFPj735VFPR04+bWVhNU+Ox1x0FO2pbLQ5BdK+dzjlvtLUso+8f/3Bv6H4Yb+M9BJSnmOEOJu4D0p5RPx/R4CXpFSTvc6f67kcw8z+ZXMhJlTxke3Yg9umQZVskb64ZtvfMMGmDrVEPWGBkPUr77aVdSd7olT+KCJmY9+cnnfwG13y1PfqkWB4/XM++g2UvO6z2E+G3459DUaL7zyuYcSd7fXhBBXAUgpp8RfexWYJKX0dMtki7inowhH0C+vapk2e6EIp3OEWUzVrjjGxONd6oNWVhqi/uCDjaL+nxPO5voPql3z0psraK3ujqJYIaMGlDJtwXrH2rEmxbECbjrpwEDviZtItyuOsaO2wfW98SrKYc/FDk1X8Zp98iqQAsE+czqyRmMncnEXQnSSUn4V//tSYJCU8lQhRG/gKQw/e2fgDWA/KaWn2ZgN4p4KKyqo9ZfMOc3zen3hvY41MUcGnueyi/rZZ8PVV1OxJaZUUcqr7X4PoaDviVeVp9tP6ecqlkGqKLWOFTjODURZcSldVr5+gOQWSVViEkI8DQwBOgghNgATgSFCiH4Yn921wHkAUsoVQohngY+AOuBCP2HPFlKRF0Z1wizIF8pr0tQvTYLXRJ0A/y9zZSX85S+GqNfXN4o63bsDcPMzTVMVeFni9rabk9HdJ8xy3a+mtp7Ln13WGLMfNuVv55Iiz8lvp/vsFg7r5vJyC50N83lKR96iVFQZ02QOlWiZMQ6bH/LY/0bgxmQalQ7SUZVeZcIsmcLNTuf2+sKHrgz05Ze7LPX6ehg71hD1Hj0Sdksmh47AuBflZaWuaRxMzNdUxCfsRLj9Pvu1SZWwn6d0RNZkIvGdJnU0y/QDTjHOQVK7qjK0V8cm51Wt/pNMrLPbFz7wIqsvv4Q//hH23deITT/jDPjkE2Pi1CbskFwOHQmNfR4zSD0Fgd+9SmbtgRkSWRQr9BT2kqKYcqrgtkWxUPn+0xF3r0Mz84tmmX7ASVCdvrrJri6dsbgy4bwCGDUg0RUQ9AulEv3i9oVXLWfHV18Z7pcHHoDa2l2W+r77urbp5ldXJW3Zmn02o2KenP8FKqf0E59k1h6olDGcdELvxn29UgXHCgTbdzZNx2y20Yt0hOLq0Mz8olmKu5cYlBTF2FpTm/RkktsDZM7KjQnbgn6hVMTG6wvvKXRxUa+//35kbS0z+vyc6cPP4LQxQynf1z2Cw+9h0644RnHLFr6x99Y+Ty7vy+TyvgnuswIX10gqxUeljKE1z78Ve6rg6p11jumYL392mePxVpQfzEmQibUcmtTRLMXdy8feplULlk48JulrqFrkQb9QQcTGD1M4azdUcvnSf/HrRS8h6uqo6HM0dxw6mvUlPwLgQwfrUjW8sihW2CSU0u2BUB0v+2fd1/owcosYycRCMpWIJ/uDtIfLRHG9lEoWvF3gTXdUVAKfjgeIJn00S5+7lxhE5V/085GaSasunbaU1rECSopiSj7h1jHnt8xaM1WFiiWV3PbYfzlnxl3MfeC3jHrnBSp6HcUJFz3E5cde3Cjs0NSvbZ2z8MKpL+ZDoaa2volfekt1rWfirPKyUkYNKG307RcK0cTNFTVuSeTMFBBBknz51aH1m2fRicY0QWiWlnt5WSnXv7jCMTY5qiG+l0Vut0C3VNdSFCv0LQZx2t/fbZJmF4wndCDr9euv2fGHP/Kf+S/Sor6O5/sM4+6fncIX7Tq5HmJ96Pm5hsDZsrX328md7uWmMOcxTNdMvZTMWFzJwH3ap6WMoTnxbrY7aKigX+4fP8Mi1dEsOhQyv2iWljvAxON7R5qe145XlEaYCJmKJZXM+2yz42sNGF98Xwvu66/hssugRw9+/c4LvNjrSIb97gGuOO4ST2GHxIeenwgJjEghOyoPBdjlprD3J1NZFMvLSpk3YZhjdtAg1zc/E25RRX6GRaqjWXSWyvyiWVrukB7/otvkZZgv6aSZ3qlvPa2sr7+Gm282whl37oQzzuA3ew5jQYv2Tc5TUhTjh7rEZfl2sfYrBC3B0aIOIkJOFmmmQ/WiuL7ZnzBzB6mOZsn0/dVES7O13GGXRWbNU54OgsYsVyypVMrs2MTK+uYbuPxyI4Txjjtg9GhYuRIeeYTfnDbMceQy6YTejBpQmuAPN8XatKSd/NC+bcGI8Q6CXVRU75tbEQ6v4hwqRBVrHjb2PupiMHbSEUuvSR/NWtwzRdAvaZBhcWVVjSHq//d/xkIjq6g/+qhROANvgZmzcqOn+8F+rBtWca5YUsn2nXVN9ikAClxOYhcVlft2TcUHXGqpJGWOaK6p+CDpycgoxVXFsLA/jIBIi8HYSfXDQ5Nemq1bJpMEdQmpDos7bN/CefOfp+7OX9Ni5044/XS45hrYbz/XdoR1G1mPVSlkcvOrqxyTh7WNZ51UcVP43beKJZU8+d4Xjg+mp+evbxIjH3QyMp2hgm6Tm1NO6hs66ZwfOhQyv9DiniGCrJr083Hvub2KcQue58z3Z9GyvpaXDxzGEz8/gwUt9qTzjPWMH14MqH9pg/p2VWL13R4YVdW1nqLilFTNTdy8CoS7rZ4N6k9OZrVrEDKV5yVd/dOknmYr7lGkNk1XelQ38Syu2sS4Bc9zxpJZtKqrpeKAo7j7sFNZ035XG8xC08hdGRr9QtyCLqwqLytl0brNjdaxGX8ONJYD9Ftd6iQqQUPzvITaLfFXtvqT9eSmJlmapbhHEc8bdUyw14PCbtke0GIHd3zxOl2efpSWdTsdRd2KkzvEywoMOjx3ij+ftmA90xaub7y2k7D6+XODWq9eZQnHDOraJNdLNvuTdZ4XTbIoFetINeku1hFFEY0oC3EoF2L49ls+HT+xUdRf6TuUu342mk9Kwo0WvKo3BUGlCIhJoRA0SKk00glSDcl04TjlYD/t0G5N8tRkuz9Zl+DTqJBUsY58JIohbxTn8MrPkmChbtwIt9xC3V1/Y98ffuBfBxzF3T87hc/37EKsQNCudQuqqmsDF8TuXFIUieAF6XO9lKz1eaCYbXIT9iAFwu0joFwRRj25qUmWZinuUQx5VQtxeEV2+GVT3PHl1zBhAtx9N9TU8MaBQ/nrwF/z2Z678p3XNkiKW7ZgyXXHuJ43VigSfO5gCGT3PYsicS35Tfhasa/OtN8jp1S5VtyqIZkPwlwScHDuv1/dVY1GhWYZ5x5FPK/fOfySPHktxW9XvZUr33yUtx44F/76VygvhxUrOH/4pQnCbmIPUbQn1zrl4K6cckjXJguT5n22OZLl5iqLmkzqpWyM3+4+YVaTmPQn3/vC9b44Lf83ycWJRqfPyBPvfaETg2kioVmKezLVeVTP4Zenw0mM2lVv5Yr/Psrb95/LefNnsOnoY+Gjj+CJJ6BXL6UVhG7JtWYt/8pVGO2ECQ+03guvikztimMJGSXtbfJyxZj5XZzIxYlGlVw7OreLJiw575YJ6zOOYvjudQ4/n7zVldGueiu/W/gCZy1+iaLaH3j9oGEUXHsNPx81JOFYv0yTXv57lYRdu85ZwI+verkxrHHMoK6N1ZHcsN4Lt7zlAFISqC0mpnjnU0EJ1YdoLo5KNJknp8U9m1OU+vnkxw/vyV+eeJvT35nBWe+/RPHOHcw64EgeGXY6S9p0ovOnDbxZ8UET/+uUk/oyaeaKxknT1rECFq3b7OmnDkq1Ja1wvZQ88d4XAL4Cb+2jU9/NKld+2P3qVvHO1ERjKiJtVOcqcnFUosk8OR0K6RWOOH54zyZfRkifKHiGsnVrDbfeSu2dd1FYU81LvY7gkWFn8GHbzo4x6dbjRw0obSLkTpOMdpyyPQahUAg+m3Kc0r5effer3mT2MZsmFVMVlqgyqa7DHzVeeIVC+oq7EOJh4FfAt1LKPvFt7YFpQHdgLTBaSrlFCCGAO4HjgGpgrJTyfb8GhhV3rzjoolhhk+LEiMQFPVF9Qb0iYqyvXT1oL0a8/jTcdRds3w6nnALXXgsHHKAcK+620tILAdx+Sj9g18OtpDjG9zvqEiJo/LCGMPpZsm6vu8WiS4KXCUwXbu9NkJh9N3S0jCYZko1zfxS4G/inZdsE4A0p5VQhxIT4/1cCvwT2i/8MAu6L/04JbsPaQiGaWENOIpZsrg4/t1CjH3rTJrjtNjghLuqjRxui3rt347lU/apBhR12WfVeIxm39AAm1klSFXeY23xELsZvu7035v1Kxh2Ya6GbmtxByS0jhOgOvGSx3FcBQ6SUXwkhOgFvSil7CiEeiP/9tH0/r/OHtdzdhstBXA9BVmlaJy29LOjGVaqbNxuiftdd8P33TUTdarX5iatJgQAnY9vLNePkkjFHLYCvqwRg8I/bc/LAbp77hlmdq0omV5eqjqpS2X+NxolUrFDd2yLYXwN7x/8uBdZb9tsQ39ZE3IUQ44BxAN26dQvcAGuhZVNorb521UU1qpNV9geJlxBv/+pbI9WuKeonn2yIep8+oc6XgDQWJVndS7ECQcsWBWzf2fShVhQrRIimESo1tfVMmrlC2Q//7uebXcv8maje86BkeuLcr/apSRRRLbmUIkGT3SQd5y4N0z+wr0BK+aCUcqCUcmDHjk3rbXphXfwBhjCaERXlZaWOi2piBcJYqWkhSAidSkxy25rvuGzu48x74Fy48Ub45S9h+XKYNi1B2FXOVygEMYd3pwFj3sB0k5QUxUCQIOxmL83Y+yqHQuAAVTW1yqMcFde8gJQsuMl0bU/VOP5ko1r8Fr5pNEEIa7l/I4ToZHHLfBvfXglYl1B2iW+LFL9sgW5+XadtqlaRl1XWtuY7zl1YwdmLZ7L7zhoqf/Er2tw2pYmgq54PjAeWR+BM4wNNiKZZH+2HBUkPkAwSuGTaUm5+dVWkFmc2pL+1fq7c3IGqhoKbdZ6pHO6a/CSsuM8EzgKmxn//y7L9IiHEMxgTqVv9/O1h8Puyq6TPDYqTQO6x43tD1BfNZI+d1bzR9yjENdcwbPTPQ50vKH6Lk0zLzyl8MpVE6TapWFLpmws+KlRdIslMCnu5mLLhIabJH1RCIZ8GhgAdgG+AiUAF8CzQDViHEQq5OR4KeTdwLEYo5NlSSt+Z0qATqn7x7amOSbaL+ss9B3Pn4DF8v9/+yqFsKjHOUSEEnDaom2MJulRSUhRj6cRjQh/vdY+ijv9OV4pdr88uOM9b6IlajRtJTahKKce4vHS0w74SuDBY84LjtQQ9VUPb8rJSYtuq+OaGv/Drt2ewx85qZvUczN8Gj2Flx+7GTvHETyZuFqzTZLAbbtExJiqLk6SEpxes9xR2AcoROyaFBYJ6j8ZV1dRSsaQy9H13m5coFCJy0U2XS8TLOr/9lH55k1pBk3lyMnGYV9Iuty9PZVUNg6fODjc5VVUFkyYxYuRgzpn9OHsc/0tYtoybxt6wS9hdsE/8uU0Gu9EgcU2WJYBJJ/Rm1IBS3FN1GXiJcGlJEWumjqAhgLC3K45x68kHubbNxN73wVNn02PCLKX3wu29bJAych+01+cmyglNr+RvUSS002hMcja3jNviDy9fdmBfcFUV3HGH8bN1K4waBdddR0X9ntz8snq4pflgcYtn97Lg3VxNZoWh8rJSrn9xRVLuluqddVQsqVSeByiKFTLx+N6N70HFkkoumbbUcV/rPEjQcMZ0lJrzKwwCRBp2ObRXx4TRnXW7eQ0t5pooyEnL3Qu/3OJKIXRVVXD99dC9u/H76KNh6VKYPp2K+j0TLG8VzMpBEvd4dicL3hreabfobj+lX2PpuC0uoY6qbKmu5arnP2Bor46+edkF0L9bWybNXEH3CbPoPmEW17+4gmKnuE12CXGYcMYo8u57YR9FuRFl2OWclRsDbddowpKzlrsb1kgGty+ta/RBVRXceSfcfrthqZ90Elx3HRx0UOMuKvHudlSsardkZ37L+aMSnZraeuas3MioAaU8PX+960PILPJhZUt1LYUFgliBSEjzYBXiMJEgqU5VEOS9jCpiRUfEaNJF3ok77BJCt8iEJsP6rVt3iXpVFZx4oiHq/fo17uKVL93EngJAJVsjJFrobsLlFqanIgrtimNK1n1lVU1CoY8g1DdI9iiOUdyyhaMQh3WxpNJNEURQo3IFqd4HvVJVkyx5Ke4mvoUdHER9zujzuWZtC758ppLO/97cuO/455Z5ZlB0s7zdHgj2jIJAo1/er96q1V/t5Se/45R+jedQyY9S4JCmIAhV1bWNtVztZGORDbd755VPPllU7kOm0y1o8oOczueugqMFtO9uRt6X224zRL283JgoZS/HLx5IaiwFLOx4xUOrpLgFHK9r5jX3StQ1fnhPLp221HGEYI2P9ourt+erccNrNOIXj51t1qhbbHuq88n73QevWHgd766xkorEYTlDwrB+69ZEUR85EiZOhLIyAG6eOttx0s8Lvxzk9jkAqziaFlnrWIHjdf0WHX1ZVUN5WalvpIpJqxa7rtOmZSGxwgK21tTSuaSI7T/UNVZ3smMdZQzt1ZFpC9Y3GcXECoWvdWv3oZvzBZkS+EylH/ZzNWm/vCYK8l7cAdi2jY+umkyXR+5nj5rvmHvAYdTfcy1Df3Nswm5hvjxWy9hvMtTJIvNKIeBnR5t+2lIfP66ThdogjRh5lbqnt44+KEGMBu7TPqHUX7viWGNopBfZ6G7IxtDDdISAavKf/Bb3bdvgrrvYefMtHLBtK6/9ZBB3DB7Dih/9hKKPYYpt9WTQfC/timOAIVrjpy9rdGtUVtUwfvoyIFG0orS8rH5aPz+uWxjiJdOWsmjdZiaX9/Wse2oXv7CCmO2JsbLFbZSN8xOa3CM/xX3bNvjb3+DWW2HLFubvfxh/OXE0H/7oJ427OImKat5uMNwQE483im5c/+KKJv7q2nrZJENiVNkZhSDBx293L7QtiiEEXBq/vtc1zQU1boIy6YTeboc2wU8co3A3pEqAUzWqCNPeXKxWpck+8mtC1SbqHH88TJxIj+e+dnRxOFVhsn4Zve6MNRKlu4dLAxKrHkWVLGytS/WoMAnJzOLXyQinSuItv4gdv/mLVCb3SsUkZrqSkWmaL/k/obptG9x9tyHqmzc3ijoDBgDQ+TXFeHcSXQ5+GfzM0EU/zFGCKRJWAa3eWRdqhak1IZdVlCF45RQzrj0Z/7OKy8VvZORnLafSrZOKScxsd0Np8pvcFvfvvttlqW/eDL/6lSHqAxMfZGF9mG7HDe3VMbB1bIqEXUC98rJ4YYqg+XcyowG3ykJBUBFHs9+XP7vMdaGUl/ilMookFZOYOupFk0lyO7fMjBnwpz/BYYfBwoXw4otNhB28s0i6YU/Li+W4OSs3BhbTAiEcswuWl5U2TswGwRTBMOkQ7By6b7ukjgd3EZSQkAGyvKzUN/ukm/h5ZVRMllTksUllezUaP3Lbcj/tNOjbt9H94kUQl4NT8WprioBLQ1ja9VJyqSU6xcrE43uHsr6jsgDXblI7jzUFg70o+fjhPV1X8drdLX4Ty27il8ooklRMYuqoF00mya8J1Yhw87X7Fc5QQQC3WyZjTVQncq14Ve+x0q44xraaOldXiNPEsh2vidpYoeCUg7sybeF6z1Wu5uTkaX9/t0nyMRO/CcdsCVdUJdfaq8ktvCZUtbhbUEkO5oZqkjDwj8BQyQOjGoFj7rdo3WbHPOIq7VFpk8qDz3ywuaVLKBSiyYIpjUbjTv5HyySBVdCDCLSV0viyfNUapX7uFKfhfKxQ0KZli8Z0AXYL0LQOW8cK+KGugQZpiOWoAbvcUU7tU0kboNJmlRFN55Iiz8IYqaiwlE/oUYAmCM1a3O2uhjDCLiDB6lUReJU0txDc/yuBHbUNjdevl5IZiysZuE97V1Ft07KFkkAkuwDL9DV7zVfoiUZ3sjF1gya7adZuGRX3hx9OqXtNUS5uWcj2nYnukigXsVxT8YHSw6S0pMjVl++3kMvaL79JX3uxDnv2S68c+25zEWGJysrNFmtZZ4rUOJEyn7sQYi3wHVAP1EkpBwoh2gPTgO7AWmC0lHKL13nSnvLXkiwrykebk3CnShyuqfjA1YduR+BteVvF12tVJZCQMMyOm+vIeg9KimN8v6OuyUPgtEO7NYkiCktUK0OzaYWp22dVZTJck7+k2uc+VEr5P8v/E4A3pJRThRAT4v9fGcF1AuM3lFVxNQTxwzstwElF1sGKJZXKwg40Cq2b5W1NdOa1qnLehGGNYu20EKm2XtKmVQuWTtxVsKNiSWVCiOSW6loKhJGQzG3+IFmiWhmaTStMdaZITVBSsYhpJPBY/O/HgPIUXEMJv6LMfsW0wVnYvY4JG3tesaSSwVNn02PCrIRFP077WVen+uFUZNuJ2nrJZc8uVao767UQyd7/STNXNIl9b5Cw1cX6j4KoVoZm0wrTVBcL1+QfyYq7BP4jhFgshBgX37a3lPKr+N9fA3sneY3Q+H05y8tKGTWglCCL781Vqm4iGcaSMgW7Mu4XN0cYTgIfZEVqoRBNskd6+WcbJK73wt4v1dWXbi4cGf/x6mtYoloZmk0rTMOsstY0b5IV98OllP2BXwIXCiGOtL4oDYe+o4knhBgnhFgkhFi0cePGJJvhjMqXc87KjYHj08vLShk/vCexgkQpjBWohRXa8RthWFG1GotihaFixiVNBd7JQvSzJM2RiApufQ1LVFZutlnL5sN5zdQRjZ9DjcaNpHzuUsrK+O9vhRAvAIcA3wghOkkpvxJCdAK+dTn2QeBBMCZUk2mHGyrLv4MMsZvsa1dBB7NXZULVa4RhP76kOOaaRbKNJTqnpraey59d5pjuoKQo5mpRw64IF682e4Vrhkk7HKWrI6pUAjqvuiaXCR0tI4RoAxRIKb+L//0acANwNLDJMqHaXkp5hde5MhUtA8HCIa1hZyqhaarRFm7nalccY0dtQ+JipgIBgoRl/n6rUE+3RaLYJzm9+hCGMCGmOqRPowlOqqJl9gZeEEbGxBbAU1LKfwshFgLPCiHOBdYBo5O4RtL4Ras4WfeFBYJ6m/CpWvzW7de/uEIp2sJthCFl0wLdpiDbE3eVl5Vy+bPLHNv09Pz1CeJuXtsprFHV7eD10PSywts5hELqiUGNJnpCi7uU8nPgIIftmzCs95zAbejttE2l3qq1KLWb+8Qufm5t8FrNac9UaW5z29ep3/YYdFW3Q9gQU9M6z5aFQRpNPtOs0w+YuFn3XoLjlEvGXpTaDb8KUCZ+SczsowDTmrfjVYwjTBy+X/y331xHKmL/NRpNIrldrCNDVCypZMbiygRhF5CQpMvLNaHqglCJw7deZ8ygro77uG0Pi0qIqQ7b02gyi7bcQ+BkuUqMsEoTN9dESVFMWeSs7ho3C946CjD96k/PX0+9lBQKwZhBXSNb1m+9pt9qSW2dazSZRYt7CFQmU91cE5NO6B3oWlbfuEpVn8nlfSMXcztOVZfCxvhrNJrUoMU9BKqWK0QXI511MdcKMf4ajSZzNOuUv2HJpmyBmUCnn9VosgNdiSliss6KTjPZlFBLo9E4o8U9JM1xwtCMT3cb6+n0sxpN9qDFXaOEX74YvcpUo8kutLhrlPBKNVzazNxSGk0uoMVdo4SbP91eIFyj0WQHeoWqRolsKlyh0Wj80eKuUSLbCldoNBpvtFtGo0RzD//UaHINLe4aZZpj+KdGk6tot4xGo9HkIVrcNRqNJg/R4q7RaDR5iBZ3jUajyUO0uGs0Gk0ekhUpf4UQG4F1lk0dgP9lqDnppLn0E5pPX3U/849s7us+UsqOTi9khbjbEUIscstRnE80l35C8+mr7mf+kat91W4ZjUajyUO0uGs0Gk0ekq3i/mCmG5Ammks/ofn0Vfcz/8jJvmalz12j0Wg0yZGtlrtGo9FokkCLu0aj0eQhGRN3IUR7IcRrQohP47/buez3byFElRDiJdv2HkKI+UKI1UKIaUKIlulpeTAC9POs+D6fCiHOsmx/UwixSgixNP6zV/pa748Q4th4+1YLISY4vN4q/v6sjr9f3S2vXRXfvkoIMTytDQ9B2L4KIboLIWos7+H9aW98ABT6eaQQ4n0hRJ0Q4te21xw/x9lIkv2st7yfM9PX6gBIKTPyA/wVmBD/ewLwF5f9jgaOB16ybX8WODX+9/3A7zPVl2T7CbQHPo//bhf/u138tTeBgZnuh0vfCoHPgH2BlsAy4ADbPhcA98f/PhWYFv/7gPj+rYAe8fMUZrpPKeprd+DDTPchwn52Bw4E/gn82rLd9XOcbT/J9DP+2veZ7oPfTybdMiOBx+J/PwaUO+0kpXwD+M66TQghgGHAdL/jswCVfg4HXpNSbpZSbgFeA45NT/OS4hBgtZTycynlTuAZjP5asfZ/OnB0/P0bCTwjpfxBSrkGWB0/X7aSTF9zCd9+SinXSimXAw22Y3Ppc5xMP3OCTIr73lLKr+J/fw3sHeDYPYEqKWVd/P8NQLZWkVDpZymw3vK/vT+PxId/12aZWPi1O2Gf+Pu1FeP9Uzk2m0imrwA9hBBLhBD/FUIckerGJkEy70suvafJtrW1EGKREOI9IUR5pC2LiJRWYhJCvA78yOGlP1n/kVJKIUTOxmSmuJ+nSSkrhRC7AzOAMzCGiZrc4Sugm5RykxBiAFAhhOgtpdyW6YZpQrNP/Hu5LzBbCPGBlPKzTDfKSkrFXUr5c7fXhBDfCCE6SSm/EkJ0Ar4NcOpNQIkQokXcQuoCVCbZ3NBE0M9KYIjl/y4YvnaklJXx398JIZ7CGE5mi7hXAl0t/zu9D+Y+G4QQLYC2GO+fyrHZROi+SsNJ+wOAlHKxEOIz4KfAopS3OjjJvC+un+MsJKnPn+V7+bkQ4k2gDMOHnzVk0i0zEzBn088C/qV6YPzLMgcwZ7ADHZ9mVPr5KnCMEKJdPJrmGOBVIUQLIUQHACFEDPgV8GEa2qzKQmC/eORSS4xJRHvkgLX/vwZmx9+/mcCp8QiTHsB+wII0tTsMofsqhOgohCgEiFt6+2FMNmYjKv10w/FznKJ2Jkvofsb71yr+dwdgMPBRyloalgzOVu8JvAF8CrwOtI9vHwj8w7LfW8BGoAbDLzY8vn1fDDFYDTwHtMr07HSS/Twn3pfVwNnxbW2AxcByYAVwJ1kWUQIcB3yCYbX8Kb7tBuCE+N+t4+/P6vj7ta/l2D/Fj1sF/DLTfUlVX4FR8fdvKfA+cHym+5JkPw+Ofxe3Y4zCVnh9jrP1J2w/gcOADzAibD4Azs10X5x+dPoBjUajyUP0ClWNRqPJQ7S4azQaTR6ixV2j0WjyEC3uGo1Gk4docddoNJo8RIu7RqPR5CFa3DUajSYP+X/7AausF2y+xgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "N = X.shape[0]\n", "\n", "S_X2 = np.sum(X*X)\n", "S_X = np.sum(X)\n", "S_XY = np.sum(X*Y)\n", "S_Y = np.sum(Y)\n", "\n", "A1 = np.array([[S_X2, S_X], \n", " [S_X, N]])\n", "B1 = np.array([S_XY, S_Y])\n", "# numpy.linalg模块包含线性代数的函数。使用这个模块,可以计算逆矩阵、求特征值、解线性方程组以及求解行列式等。\n", "coeff = np.linalg.inv(A1).dot(B1)\n", "\n", "print('a = %f, b = %f' % (coeff[0], coeff[1]))\n", "\n", "x_min = np.min(X)\n", "x_max = np.max(X)\n", "y_min = coeff[0] * x_min + coeff[1]\n", "y_max = coeff[0] * x_max + coeff[1]\n", "\n", "plt.scatter(X, Y, label='original data')\n", "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. 如何使用迭代的方法求出模型参数\n", "\n", "当数据比较多的时候,或者模型比较复杂,无法直接使用解析的方式求出模型参数。因此更为常用的方式是,通过迭代的方式逐步逼近模型的参数。\n", "\n", "### 2.1 梯度下降法\n", "在机器学习算法中,对于很多监督学习模型,需要对原始的模型构建损失函数,接下来便是通过优化算法对损失函数进行优化,以便寻找到最优的参数。在求解机器学习参数的优化算法中,使用较多的是基于梯度下降的优化算法(Gradient Descent, GD)。\n", "\n", "梯度下降法有很多优点,其中最主要的优点是,在梯度下降法的求解过程中只需求解损失函数的一阶导数,计算的代价比较小,这使得梯度下降法能在很多大规模数据集上得到应用。\n", "\n", "梯度下降法的含义是通过当前点的梯度方向寻找到新的迭代点。梯度下降法的基本思想可以类比为一个下山的过程。假设这样一个场景:\n", "* 一个人被困在山上,需要从山上下来(i.e. 找到山的最低点,也就是山谷)。\n", "* 但此时山上的浓雾很大,导致可视度很低。因此,下山的路径就无法确定,他必须利用自己周围的信息去找到下山的路径。\n", "* 这个时候,他就可以利用梯度下降算法来帮助自己下山。\n", " - 具体来说就是,以他当前的所处的位置为基准,寻找这个位置最陡峭的地方,然后朝着山的高度下降的地方走\n", " - 同理,如果我们的目标是上山,也就是爬到山顶,那么此时应该是朝着最陡峭的方向往上走。\n", " - 然后每走一段距离,都反复采用同一个方法,最后就能成功的抵达山谷。\n", "\n", "\n", "我们同时可以假设这座山最陡峭的地方是无法通过肉眼立马观察出来的,而是需要一个复杂的工具来测量,同时,这个人此时正好拥有测量出最陡峭方向的能力。所以,此人每走一段距离,都需要一段时间来测量所在位置最陡峭的方向,这是比较耗时的。那么为了在太阳下山之前到达山底,就要尽可能的减少测量方向的次数。这是一个两难的选择,如果测量的频繁,可以保证下山的方向是绝对正确的,但又非常耗时,如果测量的过少,又有偏离轨道的风险。所以需要找到一个合适的测量方向的频率,来确保下山的方向不错误,同时又不至于耗时太多!\n", "\n", "\n", "![gradient_descent](images/gradient_descent.png)\n", "\n", "如上图所示,得到了局部最优解。x,y表示的是$\\theta_0$和$\\theta_1$,z方向表示的是花费函数,很明显出发点不同,最后到达的收敛点可能不一样。当然如果是碗状的,那么收敛点就应该是一样的。\n", "\n", "对于某一个损失函数\n", "$$\n", "L = \\sum_{i=1}^{N} (y_i - a x_i - b)^2\n", "$$\n", "\n", "我们更新的策略是:\n", "$$\n", "\\theta^1 = \\theta^0 - \\alpha \\triangledown L(\\theta)\n", "$$\n", "其中$\\theta$代表了模型中的参数,例如$a$, $b$\n", "\n", "此公式的意义是:$L$是关于$\\theta$的一个函数,我们当前所处的位置为$\\theta_0$点,要从这个点走到L的最小值点,也就是山底。首先我们先确定前进的方向,也就是梯度的反向,然后走一段距离的步长,也就是$\\alpha$,走完这个段步长,就到达了$\\theta_1$这个点!\n", "\n", "我们更新的策略是:\n", "\n", "FIXME: 和后面的公式表达一样,好对比\n", "$$\n", "a^1 = a^0 + 2 \\alpha [ y - (ax+b)]*x \\\\\n", "b^1 = b^0 + 2 \\alpha [ y - (ax+b)] \n", "$$\n", "\n", "下面就这个公式的几个常见的疑问:\n", "\n", "* **$\\alpha$是什么含义?**\n", "$\\alpha$在梯度下降算法中被称作为学习率或者步长,意味着我们可以通过$\\alpha$来控制每一步走的距离,以保证不要步子跨的太大,错过了最低点。同时也要保证不要走的太慢,导致太阳下山了,还没有走到山下。所以$\\alpha$的选择在梯度下降法中往往是很重要的。\n", "![gd_stepsize](images/gd_stepsize.png)\n", "\n", "* **为什么要梯度要乘以一个负号?**\n", "梯度前加一个负号,就意味着朝着梯度相反的方向前进!梯度的方向实际就是函数在此点上升最快的方向,而我们需要朝着下降最快的方向走,自然就是负的梯度的方向,所以此处需要加上负号。\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.2 示例代码" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "epoch 0: loss = 2590736.867664, a = 18.689017, b = 148.815329\n", "epoch 1: loss = 2557255.189453, a = 36.831348, b = 148.828655\n", "epoch 2: loss = 2525130.800853, a = 54.614824, b = 148.822534\n", "epoch 3: loss = 2494255.916234, a = 72.046438, b = 148.816532\n", "epoch 4: loss = 2464581.753419, a = 89.133153, b = 148.810649\n", "epoch 5: loss = 2436061.445196, a = 105.881792, b = 148.804882\n", "epoch 6: loss = 2408649.957111, a = 122.299046, b = 148.799229\n", "epoch 7: loss = 2382304.015732, a = 138.391470, b = 148.793687\n", "epoch 8: loss = 2356982.039720, a = 154.165492, b = 148.788256\n", "epoch 9: loss = 2332644.073594, a = 169.627411, b = 148.782932\n", "epoch 10: loss = 2309251.724102, a = 184.783402, b = 148.777713\n", "epoch 11: loss = 2286768.099073, a = 199.639520, b = 148.772598\n", "epoch 12: loss = 2265157.748666, a = 214.201696, b = 148.767584\n", "epoch 13: loss = 2244386.608923, a = 228.475748, b = 148.762669\n", "epoch 14: loss = 2224421.947529, a = 242.467375, b = 148.757851\n", "epoch 15: loss = 2205232.311697, a = 256.182166, b = 148.753129\n", "epoch 16: loss = 2186787.478095, a = 269.625597, b = 148.748500\n", "epoch 17: loss = 2169058.404731, a = 282.803039, b = 148.743962\n", "epoch 18: loss = 2152017.184726, a = 295.719754, b = 148.739515\n", "epoch 19: loss = 2135637.001889, a = 308.380901, b = 148.735155\n", "epoch 20: loss = 2119892.088045, a = 320.791536, b = 148.730882\n", "epoch 21: loss = 2104757.682020, a = 332.956616, b = 148.726693\n", "epoch 22: loss = 2090209.990240, a = 344.881000, b = 148.722587\n", "epoch 23: loss = 2076226.148871, a = 356.569449, b = 148.718562\n", "epoch 24: loss = 2062784.187440, a = 368.026633, b = 148.714617\n", "epoch 25: loss = 2049862.993883, a = 379.257126, b = 148.710750\n", "epoch 26: loss = 2037442.280956, a = 390.265414, b = 148.706960\n", "epoch 27: loss = 2025502.553972, a = 401.055894, b = 148.703244\n", "epoch 28: loss = 2014025.079785, a = 411.632875, b = 148.699602\n", "epoch 29: loss = 2002991.857005, a = 422.000581, b = 148.696032\n", "epoch 30: loss = 1992385.587369, a = 432.163154, b = 148.692533\n", "epoch 31: loss = 1982189.648231, a = 442.124651, b = 148.689103\n", "epoch 32: loss = 1972388.066142, a = 451.889051, b = 148.685741\n", "epoch 33: loss = 1962965.491447, a = 461.460255, b = 148.682445\n", "epoch 34: loss = 1953907.173892, a = 470.842084, b = 148.679215\n", "epoch 35: loss = 1945198.939172, a = 480.038285, b = 148.676048\n", "epoch 36: loss = 1936827.166411, a = 489.052532, b = 148.672944\n", "epoch 37: loss = 1928778.766513, a = 497.888424, b = 148.669902\n", "epoch 38: loss = 1921041.161364, a = 506.549490, b = 148.666919\n", "epoch 39: loss = 1913602.263848, a = 515.039190, b = 148.663996\n", "epoch 40: loss = 1906450.458648, a = 523.360914, b = 148.661131\n", "epoch 41: loss = 1899574.583794, a = 531.517985, b = 148.658322\n", "epoch 42: loss = 1892963.912935, a = 539.513662, b = 148.655569\n", "epoch 43: loss = 1886608.138306, a = 547.351138, b = 148.652870\n", "epoch 44: loss = 1880497.354362, a = 555.033542, b = 148.650225\n", "epoch 45: loss = 1874622.042047, a = 562.563943, b = 148.647632\n", "epoch 46: loss = 1868973.053689, a = 569.945349, b = 148.645090\n", "epoch 47: loss = 1863541.598476, a = 577.180708, b = 148.642599\n", "epoch 48: loss = 1858319.228507, a = 584.272908, b = 148.640157\n", "epoch 49: loss = 1853297.825385, a = 591.224784, b = 148.637763\n", "epoch 50: loss = 1848469.587337, a = 598.039110, b = 148.635417\n", "epoch 51: loss = 1843827.016840, a = 604.718609, b = 148.633117\n", "epoch 52: loss = 1839362.908723, a = 611.265949, b = 148.630862\n", "epoch 53: loss = 1835070.338746, a = 617.683744, b = 148.628653\n", "epoch 54: loss = 1830942.652617, a = 623.974558, b = 148.626486\n", "epoch 55: loss = 1826973.455442, a = 630.140902, b = 148.624363\n", "epoch 56: loss = 1823156.601589, a = 636.185240, b = 148.622282\n", "epoch 57: loss = 1819486.184948, a = 642.109985, b = 148.620242\n", "epoch 58: loss = 1815956.529568, a = 647.917504, b = 148.618242\n", "epoch 59: loss = 1812562.180670, a = 653.610117, b = 148.616282\n", "epoch 60: loss = 1809297.895998, a = 659.190096, b = 148.614361\n", "epoch 61: loss = 1806158.637522, a = 664.659671, b = 148.612477\n", "epoch 62: loss = 1803139.563458, a = 670.021025, b = 148.610631\n", "epoch 63: loss = 1800236.020598, a = 675.276301, b = 148.608822\n", "epoch 64: loss = 1797443.536950, a = 680.427596, b = 148.607048\n", "epoch 65: loss = 1794757.814654, a = 685.476969, b = 148.605309\n", "epoch 66: loss = 1792174.723185, a = 690.426435, b = 148.603605\n", "epoch 67: loss = 1789690.292815, a = 695.277972, b = 148.601934\n", "epoch 68: loss = 1787300.708335, a = 700.033517, b = 148.600297\n", "epoch 69: loss = 1785002.303019, a = 704.694970, b = 148.598692\n", "epoch 70: loss = 1782791.552830, a = 709.264191, b = 148.597119\n", "epoch 71: loss = 1780665.070844, a = 713.743007, b = 148.595576\n", "epoch 72: loss = 1778619.601901, a = 718.133206, b = 148.594065\n", "epoch 73: loss = 1776652.017457, a = 722.436540, b = 148.592583\n", "epoch 74: loss = 1774759.310646, a = 726.654730, b = 148.591130\n", "epoch 75: loss = 1772938.591527, a = 730.789459, b = 148.589707\n", "epoch 76: loss = 1771187.082519, a = 734.842379, b = 148.588311\n", "epoch 77: loss = 1769502.114022, a = 738.815108, b = 148.586943\n", "epoch 78: loss = 1767881.120197, a = 742.709234, b = 148.585602\n", "epoch 79: loss = 1766321.634920, a = 746.526311, b = 148.584288\n", "epoch 80: loss = 1764821.287889, a = 750.267864, b = 148.583000\n", "epoch 81: loss = 1763377.800890, a = 753.935387, b = 148.581737\n", "epoch 82: loss = 1761988.984199, a = 757.530345, b = 148.580499\n", "epoch 83: loss = 1760652.733134, a = 761.054173, b = 148.579286\n", "epoch 84: loss = 1759367.024737, a = 764.508280, b = 148.578096\n", "epoch 85: loss = 1758129.914584, a = 767.894044, b = 148.576931\n", "epoch 86: loss = 1756939.533728, a = 771.212818, b = 148.575788\n", "epoch 87: loss = 1755794.085750, a = 774.465927, b = 148.574668\n", "epoch 88: loss = 1754691.843935, a = 777.654671, b = 148.573570\n", "epoch 89: loss = 1753631.148552, a = 780.780322, b = 148.572493\n", "epoch 90: loss = 1752610.404243, a = 783.844130, b = 148.571438\n", "epoch 91: loss = 1751628.077518, a = 786.847318, b = 148.570404\n", "epoch 92: loss = 1750682.694337, a = 789.791085, b = 148.569391\n", "epoch 93: loss = 1749772.837797, a = 792.676607, b = 148.568397\n", "epoch 94: loss = 1748897.145906, a = 795.505037, b = 148.567423\n", "epoch 95: loss = 1748054.309442, a = 798.277504, b = 148.566469\n", "epoch 96: loss = 1747243.069899, a = 800.995115, b = 148.565533\n", "epoch 97: loss = 1746462.217508, a = 803.658956, b = 148.564616\n", "epoch 98: loss = 1745710.589343, a = 806.270090, b = 148.563716\n", "epoch 99: loss = 1744987.067493, a = 808.829561, b = 148.562835\n", "epoch 100: loss = 1744290.577312, a = 811.338391, b = 148.561971\n", "epoch 101: loss = 1743620.085732, a = 813.797581, b = 148.561125\n", "epoch 102: loss = 1742974.599648, a = 816.208114, b = 148.560295\n", "epoch 103: loss = 1742353.164357, a = 818.570953, b = 148.559481\n", "epoch 104: loss = 1741754.862068, a = 820.887041, b = 148.558683\n", "epoch 105: loss = 1741178.810465, a = 823.157303, b = 148.557902\n", "epoch 106: loss = 1740624.161324, a = 825.382646, b = 148.557135\n", "epoch 107: loss = 1740090.099189, a = 827.563959, b = 148.556384\n", "epoch 108: loss = 1739575.840097, a = 829.702112, b = 148.555648\n", "epoch 109: loss = 1739080.630352, a = 831.797961, b = 148.554926\n", "epoch 110: loss = 1738603.745351, a = 833.852341, b = 148.554219\n", "epoch 111: loss = 1738144.488447, a = 835.866073, b = 148.553526\n", "epoch 112: loss = 1737702.189871, a = 837.839962, b = 148.552846\n", "epoch 113: loss = 1737276.205677, a = 839.774796, b = 148.552180\n", "epoch 114: loss = 1736865.916748, a = 841.671348, b = 148.551527\n", "epoch 115: loss = 1736470.727823, a = 843.530375, b = 148.550887\n", "epoch 116: loss = 1736090.066576, a = 845.352619, b = 148.550259\n", "epoch 117: loss = 1735723.382723, a = 847.138809, b = 148.549644\n", "epoch 118: loss = 1735370.147167, a = 848.889657, b = 148.549041\n", "epoch 119: loss = 1735029.851171, a = 850.605863, b = 148.548450\n", "epoch 120: loss = 1734702.005574, a = 852.288113, b = 148.547871\n", "epoch 121: loss = 1734386.140028, a = 853.937078, b = 148.547303\n", "epoch 122: loss = 1734081.802266, a = 855.553417, b = 148.546747\n", "epoch 123: loss = 1733788.557403, a = 857.137775, b = 148.546201\n", "epoch 124: loss = 1733505.987262, a = 858.690785, b = 148.545666\n", "epoch 125: loss = 1733233.689723, a = 860.213068, b = 148.545142\n", "epoch 126: loss = 1732971.278103, a = 861.705231, b = 148.544628\n", "epoch 127: loss = 1732718.380559, a = 863.167870, b = 148.544125\n", "epoch 128: loss = 1732474.639507, a = 864.601570, b = 148.543631\n", "epoch 129: loss = 1732239.711074, a = 866.006902, b = 148.543147\n", "epoch 130: loss = 1732013.264567, a = 867.384429, b = 148.542673\n", "epoch 131: loss = 1731794.981959, a = 868.734701, b = 148.542208\n", "epoch 132: loss = 1731584.557401, a = 870.058256, b = 148.541752\n", "epoch 133: loss = 1731381.696752, a = 871.355623, b = 148.541306\n", "epoch 134: loss = 1731186.117121, a = 872.627321, b = 148.540868\n", "epoch 135: loss = 1730997.546436, a = 873.873858, b = 148.540438\n", "epoch 136: loss = 1730815.723022, a = 875.095730, b = 148.540018\n", "epoch 137: loss = 1730640.395202, a = 876.293427, b = 148.539605\n", "epoch 138: loss = 1730471.320908, a = 877.467426, b = 148.539201\n", "epoch 139: loss = 1730308.267311, a = 878.618197, b = 148.538805\n", "epoch 140: loss = 1730151.010464, a = 879.746199, b = 148.538416\n", "epoch 141: loss = 1729999.334955, a = 880.851882, b = 148.538036\n", "epoch 142: loss = 1729853.033583, a = 881.935688, b = 148.537663\n", "epoch 143: loss = 1729711.907037, a = 882.998050, b = 148.537297\n", "epoch 144: loss = 1729575.763593, a = 884.039393, b = 148.536938\n", "epoch 145: loss = 1729444.418820, a = 885.060132, b = 148.536587\n", "epoch 146: loss = 1729317.695300, a = 886.060674, b = 148.536242\n", "epoch 147: loss = 1729195.422355, a = 887.041420, b = 148.535904\n", "epoch 148: loss = 1729077.435792, a = 888.002761, b = 148.535573\n", "epoch 149: loss = 1728963.577645, a = 888.945081, b = 148.535249\n", "epoch 150: loss = 1728853.695944, a = 889.868757, b = 148.534931\n", "epoch 151: loss = 1728747.644477, a = 890.774156, b = 148.534619\n", "epoch 152: loss = 1728645.282573, a = 891.661642, b = 148.534314\n", "epoch 153: loss = 1728546.474885, a = 892.531568, b = 148.534014\n", "epoch 154: loss = 1728451.091187, a = 893.384282, b = 148.533720\n", "epoch 155: loss = 1728359.006178, a = 894.220124, b = 148.533433\n", "epoch 156: loss = 1728270.099288, a = 895.039428, b = 148.533150\n", "epoch 157: loss = 1728184.254503, a = 895.842521, b = 148.532874\n", "epoch 158: loss = 1728101.360185, a = 896.629725, b = 148.532603\n", "epoch 159: loss = 1728021.308903, a = 897.401353, b = 148.532337\n", "epoch 160: loss = 1727943.997275, a = 898.157714, b = 148.532077\n", "epoch 161: loss = 1727869.325813, a = 898.899110, b = 148.531821\n", "epoch 162: loss = 1727797.198769, a = 899.625836, b = 148.531571\n", "epoch 163: loss = 1727727.523995, a = 900.338184, b = 148.531326\n", "epoch 164: loss = 1727660.212803, a = 901.036437, b = 148.531086\n", "epoch 165: loss = 1727595.179837, a = 901.720875, b = 148.530850\n", "epoch 166: loss = 1727532.342938, a = 902.391770, b = 148.530619\n", "epoch 167: loss = 1727471.623027, a = 903.049391, b = 148.530392\n", "epoch 168: loss = 1727412.943986, a = 903.694001, b = 148.530170\n", "epoch 169: loss = 1727356.232544, a = 904.325856, b = 148.529953\n", "epoch 170: loss = 1727301.418168, a = 904.945210, b = 148.529740\n", "epoch 171: loss = 1727248.432959, a = 905.552309, b = 148.529531\n", "epoch 172: loss = 1727197.211551, a = 906.147396, b = 148.529326\n", "epoch 173: loss = 1727147.691014, a = 906.730709, b = 148.529125\n", "epoch 174: loss = 1727099.810763, a = 907.302481, b = 148.528928\n", "epoch 175: loss = 1727053.512464, a = 907.862939, b = 148.528735\n", "epoch 176: loss = 1727008.739955, a = 908.412309, b = 148.528546\n", "epoch 177: loss = 1726965.439156, a = 908.950808, b = 148.528360\n", "epoch 178: loss = 1726923.557995, a = 909.478653, b = 148.528179\n", "epoch 179: loss = 1726883.046328, a = 909.996055, b = 148.528000\n", "epoch 180: loss = 1726843.855869, a = 910.503219, b = 148.527826\n", "epoch 181: loss = 1726805.940116, a = 911.000348, b = 148.527655\n", "epoch 182: loss = 1726769.254286, a = 911.487641, b = 148.527487\n", "epoch 183: loss = 1726733.755249, a = 911.965292, b = 148.527322\n", "epoch 184: loss = 1726699.401462, a = 912.433493, b = 148.527161\n", "epoch 185: loss = 1726666.152915, a = 912.892430, b = 148.527003\n", "epoch 186: loss = 1726633.971067, a = 913.342287, b = 148.526848\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "epoch 187: loss = 1726602.818794, a = 913.783243, b = 148.526696\n", "epoch 188: loss = 1726572.660334, a = 914.215474, b = 148.526548\n", "epoch 189: loss = 1726543.461235, a = 914.639153, b = 148.526402\n", "epoch 190: loss = 1726515.188306, a = 915.054449, b = 148.526259\n", "epoch 191: loss = 1726487.809572, a = 915.461529, b = 148.526119\n", "epoch 192: loss = 1726461.294221, a = 915.860553, b = 148.525981\n", "epoch 193: loss = 1726435.612569, a = 916.251683, b = 148.525846\n", "epoch 194: loss = 1726410.736010, a = 916.635074, b = 148.525714\n", "epoch 195: loss = 1726386.636980, a = 917.010879, b = 148.525585\n", "epoch 196: loss = 1726363.288915, a = 917.379249, b = 148.525458\n", "epoch 197: loss = 1726340.666217, a = 917.740330, b = 148.525334\n", "epoch 198: loss = 1726318.744212, a = 918.094267, b = 148.525212\n", "epoch 199: loss = 1726297.499121, a = 918.441201, b = 148.525093\n", "epoch 200: loss = 1726276.908024, a = 918.781270, b = 148.524975\n", "epoch 201: loss = 1726256.948827, a = 919.114611, b = 148.524861\n", "epoch 202: loss = 1726237.600231, a = 919.441356, b = 148.524748\n", "epoch 203: loss = 1726218.841703, a = 919.761637, b = 148.524638\n", "epoch 204: loss = 1726200.653451, a = 920.075580, b = 148.524530\n", "epoch 205: loss = 1726183.016388, a = 920.383312, b = 148.524424\n", "epoch 206: loss = 1726165.912113, a = 920.684955, b = 148.524320\n", "epoch 207: loss = 1726149.322881, a = 920.980630, b = 148.524218\n", "epoch 208: loss = 1726133.231583, a = 921.270455, b = 148.524118\n", "epoch 209: loss = 1726117.621717, a = 921.554545, b = 148.524021\n", "epoch 210: loss = 1726102.477370, a = 921.833014, b = 148.523925\n", "epoch 211: loss = 1726087.783192, a = 922.105974, b = 148.523831\n", "epoch 212: loss = 1726073.524378, a = 922.373533, b = 148.523739\n", "epoch 213: loss = 1726059.686650, a = 922.635798, b = 148.523648\n", "epoch 214: loss = 1726046.256229, a = 922.892873, b = 148.523560\n", "epoch 215: loss = 1726033.219826, a = 923.144863, b = 148.523473\n", "epoch 216: loss = 1726020.564620, a = 923.391866, b = 148.523388\n", "epoch 217: loss = 1726008.278237, a = 923.633982, b = 148.523305\n", "epoch 218: loss = 1725996.348741, a = 923.871308, b = 148.523223\n", "epoch 219: loss = 1725984.764612, a = 924.103938, b = 148.523143\n", "epoch 220: loss = 1725973.514733, a = 924.331966, b = 148.523064\n", "epoch 221: loss = 1725962.588374, a = 924.555481, b = 148.522987\n", "epoch 222: loss = 1725951.975181, a = 924.774575, b = 148.522912\n", "epoch 223: loss = 1725941.665156, a = 924.989333, b = 148.522838\n", "epoch 224: loss = 1725931.648652, a = 925.199842, b = 148.522765\n", "epoch 225: loss = 1725921.916352, a = 925.406186, b = 148.522694\n", "epoch 226: loss = 1725912.459264, a = 925.608447, b = 148.522625\n", "epoch 227: loss = 1725903.268703, a = 925.806706, b = 148.522556\n", "epoch 228: loss = 1725894.336285, a = 926.001043, b = 148.522489\n", "epoch 229: loss = 1725885.653913, a = 926.191534, b = 148.522424\n", "epoch 230: loss = 1725877.213768, a = 926.378257, b = 148.522360\n", "epoch 231: loss = 1725869.008296, a = 926.561285, b = 148.522297\n", "epoch 232: loss = 1725861.030202, a = 926.740692, b = 148.522235\n", "epoch 233: loss = 1725853.272442, a = 926.916548, b = 148.522174\n", "epoch 234: loss = 1725845.728205, a = 927.088926, b = 148.522115\n", "epoch 235: loss = 1725838.390917, a = 927.257893, b = 148.522057\n", "epoch 236: loss = 1725831.254223, a = 927.423516, b = 148.522000\n", "epoch 237: loss = 1725824.311982, a = 927.585863, b = 148.521944\n", "epoch 238: loss = 1725817.558260, a = 927.744997, b = 148.521889\n", "epoch 239: loss = 1725810.987324, a = 927.900983, b = 148.521835\n", "epoch 240: loss = 1725804.593630, a = 928.053883, b = 148.521783\n", "epoch 241: loss = 1725798.371821, a = 928.203757, b = 148.521731\n", "epoch 242: loss = 1725792.316718, a = 928.350666, b = 148.521680\n", "epoch 243: loss = 1725786.423315, a = 928.494668, b = 148.521631\n", "epoch 244: loss = 1725780.686770, a = 928.635821, b = 148.521582\n", "epoch 245: loss = 1725775.102403, a = 928.774181, b = 148.521535\n", "epoch 246: loss = 1725769.665688, a = 928.909804, b = 148.521488\n", "epoch 247: loss = 1725764.372248, a = 929.042743, b = 148.521442\n", "epoch 248: loss = 1725759.217848, a = 929.173052, b = 148.521397\n", "epoch 249: loss = 1725754.198393, a = 929.300783, b = 148.521353\n", "epoch 250: loss = 1725749.309922, a = 929.425986, b = 148.521310\n", "epoch 251: loss = 1725744.548603, a = 929.548712, b = 148.521268\n", "epoch 252: loss = 1725739.910728, a = 929.669010, b = 148.521226\n", "epoch 253: loss = 1725735.392708, a = 929.786928, b = 148.521186\n", "epoch 254: loss = 1725730.991072, a = 929.902512, b = 148.521146\n", "epoch 255: loss = 1725726.702459, a = 930.015810, b = 148.521107\n", "epoch 256: loss = 1725722.523617, a = 930.126866, b = 148.521069\n", "epoch 257: loss = 1725718.451397, a = 930.235724, b = 148.521031\n", "epoch 258: loss = 1725714.482753, a = 930.342429, b = 148.520995\n", "epoch 259: loss = 1725710.614734, a = 930.447022, b = 148.520959\n", "epoch 260: loss = 1725706.844482, a = 930.549546, b = 148.520923\n", "epoch 261: loss = 1725703.169232, a = 930.650042, b = 148.520889\n", "epoch 262: loss = 1725699.586303, a = 930.748549, b = 148.520855\n", "epoch 263: loss = 1725696.093102, a = 930.845107, b = 148.520822\n", "epoch 264: loss = 1725692.687115, a = 930.939754, b = 148.520789\n", "epoch 265: loss = 1725689.365908, a = 931.032529, b = 148.520757\n", "epoch 266: loss = 1725686.127121, a = 931.123468, b = 148.520726\n", "epoch 267: loss = 1725682.968469, a = 931.212608, b = 148.520695\n", "epoch 268: loss = 1725679.887739, a = 931.299985, b = 148.520665\n", "epoch 269: loss = 1725676.882782, a = 931.385632, b = 148.520635\n", "epoch 270: loss = 1725673.951521, a = 931.469585, b = 148.520606\n", "epoch 271: loss = 1725671.091938, a = 931.551877, b = 148.520578\n", "epoch 272: loss = 1725668.302080, a = 931.632540, b = 148.520550\n", "epoch 273: loss = 1725665.580052, a = 931.711608, b = 148.520523\n", "epoch 274: loss = 1725662.924017, a = 931.789111, b = 148.520496\n", "epoch 275: loss = 1725660.332194, a = 931.865080, b = 148.520470\n", "epoch 276: loss = 1725657.802855, a = 931.939547, b = 148.520445\n", "epoch 277: loss = 1725655.334325, a = 932.012540, b = 148.520420\n", "epoch 278: loss = 1725652.924980, a = 932.084089, b = 148.520395\n", "epoch 279: loss = 1725650.573243, a = 932.154222, b = 148.520371\n", "epoch 280: loss = 1725648.277584, a = 932.222968, b = 148.520347\n", "epoch 281: loss = 1725646.036521, a = 932.290353, b = 148.520324\n", "epoch 282: loss = 1725643.848614, a = 932.356405, b = 148.520301\n", "epoch 283: loss = 1725641.712465, a = 932.421150, b = 148.520279\n", "epoch 284: loss = 1725639.626719, a = 932.484614, b = 148.520257\n", "epoch 285: loss = 1725637.590060, a = 932.546823, b = 148.520236\n", "epoch 286: loss = 1725635.601209, a = 932.607801, b = 148.520215\n", "epoch 287: loss = 1725633.658927, a = 932.667572, b = 148.520194\n", "epoch 288: loss = 1725631.762010, a = 932.726160, b = 148.520174\n", "epoch 289: loss = 1725629.909289, a = 932.783590, b = 148.520154\n", "epoch 290: loss = 1725628.099627, a = 932.839883, b = 148.520135\n", "epoch 291: loss = 1725626.331922, a = 932.895062, b = 148.520116\n", "epoch 292: loss = 1725624.605102, a = 932.949149, b = 148.520097\n", "epoch 293: loss = 1725622.918128, a = 933.002166, b = 148.520079\n", "epoch 294: loss = 1725621.269989, a = 933.054135, b = 148.520061\n", "epoch 295: loss = 1725619.659702, a = 933.105075, b = 148.520043\n", "epoch 296: loss = 1725618.086312, a = 933.155007, b = 148.520026\n", "epoch 297: loss = 1725616.548894, a = 933.203951, b = 148.520009\n", "epoch 298: loss = 1725615.046544, a = 933.251927, b = 148.519993\n", "epoch 299: loss = 1725613.578388, a = 933.298953, b = 148.519977\n", "epoch 300: loss = 1725612.143573, a = 933.345050, b = 148.519961\n", "epoch 301: loss = 1725610.741272, a = 933.390234, b = 148.519945\n", "epoch 302: loss = 1725609.370679, a = 933.434524, b = 148.519930\n", "epoch 303: loss = 1725608.031013, a = 933.477937, b = 148.519915\n", "epoch 304: loss = 1725606.721512, a = 933.520492, b = 148.519900\n", "epoch 305: loss = 1725605.441435, a = 933.562205, b = 148.519886\n", "epoch 306: loss = 1725604.190064, a = 933.603092, b = 148.519872\n", "epoch 307: loss = 1725602.966697, a = 933.643171, b = 148.519858\n", "epoch 308: loss = 1725601.770653, a = 933.682456, b = 148.519845\n", "epoch 309: loss = 1725600.601270, a = 933.720964, b = 148.519831\n", "epoch 310: loss = 1725599.457903, a = 933.758711, b = 148.519818\n", "epoch 311: loss = 1725598.339923, a = 933.795710, b = 148.519806\n", "epoch 312: loss = 1725597.246722, a = 933.831977, b = 148.519793\n", "epoch 313: loss = 1725596.177703, a = 933.867527, b = 148.519781\n", "epoch 314: loss = 1725595.132289, a = 933.902373, b = 148.519769\n", "epoch 315: loss = 1725594.109917, a = 933.936530, b = 148.519757\n", "epoch 316: loss = 1725593.110038, a = 933.970011, b = 148.519746\n", "epoch 317: loss = 1725592.132118, a = 934.002830, b = 148.519734\n", "epoch 318: loss = 1725591.175638, a = 934.034999, b = 148.519723\n", "epoch 319: loss = 1725590.240092, a = 934.066532, b = 148.519712\n", "epoch 320: loss = 1725589.324986, a = 934.097441, b = 148.519702\n", "epoch 321: loss = 1725588.429841, a = 934.127738, b = 148.519691\n", "epoch 322: loss = 1725587.554189, a = 934.157436, b = 148.519681\n", "epoch 323: loss = 1725586.697574, a = 934.186546, b = 148.519671\n", "epoch 324: loss = 1725585.859554, a = 934.215081, b = 148.519661\n", "epoch 325: loss = 1725585.039694, a = 934.243051, b = 148.519651\n", "epoch 326: loss = 1725584.237575, a = 934.270467, b = 148.519642\n", "epoch 327: loss = 1725583.452786, a = 934.297341, b = 148.519633\n", "epoch 328: loss = 1725582.684926, a = 934.323683, b = 148.519624\n", "epoch 329: loss = 1725581.933606, a = 934.349504, b = 148.519615\n", "epoch 330: loss = 1725581.198445, a = 934.374814, b = 148.519606\n", "epoch 331: loss = 1725580.479074, a = 934.399623, b = 148.519598\n", "epoch 332: loss = 1725579.775131, a = 934.423941, b = 148.519589\n", "epoch 333: loss = 1725579.086264, a = 934.447779, b = 148.519581\n", "epoch 334: loss = 1725578.412130, a = 934.471144, b = 148.519573\n", "epoch 335: loss = 1725577.752394, a = 934.494048, b = 148.519565\n", "epoch 336: loss = 1725577.106730, a = 934.516498, b = 148.519557\n", "epoch 337: loss = 1725576.474819, a = 934.538504, b = 148.519550\n", "epoch 338: loss = 1725575.856351, a = 934.560074, b = 148.519542\n", "epoch 339: loss = 1725575.251023, a = 934.581218, b = 148.519535\n", "epoch 340: loss = 1725574.658540, a = 934.601943, b = 148.519528\n", "epoch 341: loss = 1725574.078613, a = 934.622259, b = 148.519521\n", "epoch 342: loss = 1725573.510962, a = 934.642172, b = 148.519514\n", "epoch 343: loss = 1725572.955312, a = 934.661691, b = 148.519507\n", "epoch 344: loss = 1725572.411396, a = 934.680825, b = 148.519501\n", "epoch 345: loss = 1725571.878952, a = 934.699579, b = 148.519494\n", "epoch 346: loss = 1725571.357726, a = 934.717963, b = 148.519488\n", "epoch 347: loss = 1725570.847468, a = 934.735982, b = 148.519482\n", "epoch 348: loss = 1725570.347937, a = 934.753646, b = 148.519476\n", "epoch 349: loss = 1725569.858895, a = 934.770959, b = 148.519470\n", "epoch 350: loss = 1725569.380112, a = 934.787931, b = 148.519464\n", "epoch 351: loss = 1725568.911360, a = 934.804566, b = 148.519458\n", "epoch 352: loss = 1725568.452420, a = 934.820872, b = 148.519453\n", "epoch 353: loss = 1725568.003077, a = 934.836856, b = 148.519447\n", "epoch 354: loss = 1725567.563120, a = 934.852523, b = 148.519442\n", "epoch 355: loss = 1725567.132345, a = 934.867881, b = 148.519436\n", "epoch 356: loss = 1725566.710551, a = 934.882934, b = 148.519431\n", "epoch 357: loss = 1725566.297542, a = 934.897690, b = 148.519426\n", "epoch 358: loss = 1725565.893128, a = 934.912154, b = 148.519421\n", "epoch 359: loss = 1725565.497121, a = 934.926331, b = 148.519416\n", "epoch 360: loss = 1725565.109340, a = 934.940228, b = 148.519411\n", "epoch 361: loss = 1725564.729607, a = 934.953850, b = 148.519407\n", "epoch 362: loss = 1725564.357747, a = 934.967203, b = 148.519402\n", "epoch 363: loss = 1725563.993590, a = 934.980291, b = 148.519398\n", "epoch 364: loss = 1725563.636972, a = 934.993120, b = 148.519393\n", "epoch 365: loss = 1725563.287729, a = 935.005696, b = 148.519389\n", "epoch 366: loss = 1725562.945702, a = 935.018023, b = 148.519385\n", "epoch 367: loss = 1725562.610739, a = 935.030106, b = 148.519380\n", "epoch 368: loss = 1725562.282686, a = 935.041949, b = 148.519376\n", "epoch 369: loss = 1725561.961396, a = 935.053559, b = 148.519372\n", "epoch 370: loss = 1725561.646724, a = 935.064938, b = 148.519368\n", "epoch 371: loss = 1725561.338531, a = 935.076093, b = 148.519365\n", "epoch 372: loss = 1725561.036676, a = 935.087027, b = 148.519361\n", "epoch 373: loss = 1725560.741026, a = 935.097744, b = 148.519357\n", "epoch 374: loss = 1725560.451449, a = 935.108250, b = 148.519354\n", "epoch 375: loss = 1725560.167816, a = 935.118547, b = 148.519350\n", "epoch 376: loss = 1725559.890000, a = 935.128641, b = 148.519347\n", "epoch 377: loss = 1725559.617879, a = 935.138535, b = 148.519343\n", "epoch 378: loss = 1725559.351332, a = 935.148234, b = 148.519340\n", "epoch 379: loss = 1725559.090241, a = 935.157740, b = 148.519337\n", "epoch 380: loss = 1725558.834492, a = 935.167059, b = 148.519333\n", "epoch 381: loss = 1725558.583972, a = 935.176193, b = 148.519330\n", "epoch 382: loss = 1725558.338570, a = 935.185146, b = 148.519327\n", "epoch 383: loss = 1725558.098179, a = 935.193922, b = 148.519324\n", "epoch 384: loss = 1725557.862695, a = 935.202525, b = 148.519321\n", "epoch 385: loss = 1725557.632013, a = 935.210957, b = 148.519318\n", "epoch 386: loss = 1725557.406033, a = 935.219223, b = 148.519315\n", "epoch 387: loss = 1725557.184657, a = 935.227324, b = 148.519313\n", "epoch 388: loss = 1725556.967789, a = 935.235266, b = 148.519310\n", "epoch 389: loss = 1725556.755334, a = 935.243051, b = 148.519307\n", "epoch 390: loss = 1725556.547200, a = 935.250681, b = 148.519305\n", "epoch 391: loss = 1725556.343298, a = 935.258160, b = 148.519302\n", "epoch 392: loss = 1725556.143538, a = 935.265492, b = 148.519299\n", "epoch 393: loss = 1725555.947836, a = 935.272678, b = 148.519297\n", "epoch 394: loss = 1725555.756105, a = 935.279723, b = 148.519295\n", "epoch 395: loss = 1725555.568265, a = 935.286628, b = 148.519292\n", "epoch 396: loss = 1725555.384233, a = 935.293396, b = 148.519290\n", "epoch 397: loss = 1725555.203932, a = 935.300030, b = 148.519288\n", "epoch 398: loss = 1725555.027283, a = 935.306533, b = 148.519285\n", "epoch 399: loss = 1725554.854212, a = 935.312908, b = 148.519283\n", "epoch 400: loss = 1725554.684644, a = 935.319156, b = 148.519281\n", "epoch 401: loss = 1725554.518507, a = 935.325281, b = 148.519279\n", "epoch 402: loss = 1725554.355730, a = 935.331284, b = 148.519277\n", "epoch 403: loss = 1725554.196244, a = 935.337169, b = 148.519275\n", "epoch 404: loss = 1725554.039980, a = 935.342937, b = 148.519273\n", "epoch 405: loss = 1725553.886873, a = 935.348591, b = 148.519271\n", "epoch 406: loss = 1725553.736857, a = 935.354133, b = 148.519269\n", "epoch 407: loss = 1725553.589869, a = 935.359566, b = 148.519267\n", "epoch 408: loss = 1725553.445846, a = 935.364891, b = 148.519265\n", "epoch 409: loss = 1725553.304728, a = 935.370111, b = 148.519263\n", "epoch 410: loss = 1725553.166456, a = 935.375227, b = 148.519262\n", "epoch 411: loss = 1725553.030969, a = 935.380242, b = 148.519260\n", "epoch 412: loss = 1725552.898213, a = 935.385158, b = 148.519258\n", "epoch 413: loss = 1725552.768130, a = 935.389977, b = 148.519257\n", "epoch 414: loss = 1725552.640666, a = 935.394701, b = 148.519255\n", "epoch 415: loss = 1725552.515767, a = 935.399331, b = 148.519253\n", "epoch 416: loss = 1725552.393381, a = 935.403869, b = 148.519252\n", "epoch 417: loss = 1725552.273456, a = 935.408317, b = 148.519250\n", "epoch 418: loss = 1725552.155943, a = 935.412678, b = 148.519249\n", "epoch 419: loss = 1725552.040792, a = 935.416952, b = 148.519247\n", "epoch 420: loss = 1725551.927954, a = 935.421142, b = 148.519246\n", "epoch 421: loss = 1725551.817384, a = 935.425249, b = 148.519244\n", "epoch 422: loss = 1725551.709033, a = 935.429274, b = 148.519243\n", "epoch 423: loss = 1725551.602858, a = 935.433220, b = 148.519242\n", "epoch 424: loss = 1725551.498814, a = 935.437088, b = 148.519240\n", "epoch 425: loss = 1725551.396858, a = 935.440879, b = 148.519239\n", "epoch 426: loss = 1725551.296947, a = 935.444595, b = 148.519238\n", "epoch 427: loss = 1725551.199039, a = 935.448238, b = 148.519237\n", "epoch 428: loss = 1725551.103094, a = 935.451809, b = 148.519235\n", "epoch 429: loss = 1725551.009073, a = 935.455309, b = 148.519234\n", "epoch 430: loss = 1725550.916935, a = 935.458739, b = 148.519233\n", "epoch 431: loss = 1725550.826644, a = 935.462102, b = 148.519232\n", "epoch 432: loss = 1725550.738161, a = 935.465399, b = 148.519231\n", "epoch 433: loss = 1725550.651449, a = 935.468630, b = 148.519229\n", "epoch 434: loss = 1725550.566474, a = 935.471797, b = 148.519228\n", "epoch 435: loss = 1725550.483199, a = 935.474901, b = 148.519227\n", "epoch 436: loss = 1725550.401591, a = 935.477945, b = 148.519226\n", "epoch 437: loss = 1725550.321615, a = 935.480927, b = 148.519225\n", "epoch 438: loss = 1725550.243239, a = 935.483851, b = 148.519224\n", "epoch 439: loss = 1725550.166431, a = 935.486717, b = 148.519223\n", "epoch 440: loss = 1725550.091158, a = 935.489527, b = 148.519222\n", "epoch 441: loss = 1725550.017389, a = 935.492280, b = 148.519221\n", "epoch 442: loss = 1725549.945095, a = 935.494980, b = 148.519220\n", "epoch 443: loss = 1725549.874246, a = 935.497625, b = 148.519219\n", "epoch 444: loss = 1725549.804812, a = 935.500219, b = 148.519219\n", "epoch 445: loss = 1725549.736765, a = 935.502761, b = 148.519218\n", "epoch 446: loss = 1725549.670077, a = 935.505253, b = 148.519217\n", "epoch 447: loss = 1725549.604720, a = 935.507696, b = 148.519216\n", "epoch 448: loss = 1725549.540668, a = 935.510090, b = 148.519215\n", "epoch 449: loss = 1725549.477894, a = 935.512437, b = 148.519214\n", "epoch 450: loss = 1725549.416373, a = 935.514737, b = 148.519214\n", "epoch 451: loss = 1725549.356080, a = 935.516992, b = 148.519213\n", "epoch 452: loss = 1725549.296990, a = 935.519202, b = 148.519212\n", "epoch 453: loss = 1725549.239078, a = 935.521369, b = 148.519211\n", "epoch 454: loss = 1725549.182321, a = 935.523493, b = 148.519211\n", "epoch 455: loss = 1725549.126696, a = 935.525574, b = 148.519210\n", "epoch 456: loss = 1725549.072179, a = 935.527615, b = 148.519209\n", "epoch 457: loss = 1725549.018750, a = 935.529615, b = 148.519208\n", "epoch 458: loss = 1725548.966385, a = 935.531575, b = 148.519208\n", "epoch 459: loss = 1725548.915064, a = 935.533497, b = 148.519207\n", "epoch 460: loss = 1725548.864766, a = 935.535381, b = 148.519206\n", "epoch 461: loss = 1725548.815470, a = 935.537227, b = 148.519206\n", "epoch 462: loss = 1725548.767155, a = 935.539037, b = 148.519205\n", "epoch 463: loss = 1725548.719803, a = 935.540811, b = 148.519205\n", "epoch 464: loss = 1725548.673394, a = 935.542550, b = 148.519204\n", "epoch 465: loss = 1725548.627910, a = 935.544255, b = 148.519203\n", "epoch 466: loss = 1725548.583330, a = 935.545926, b = 148.519203\n", "epoch 467: loss = 1725548.539638, a = 935.547564, b = 148.519202\n", "epoch 468: loss = 1725548.496816, a = 935.549169, b = 148.519202\n", "epoch 469: loss = 1725548.454846, a = 935.550743, b = 148.519201\n", "epoch 470: loss = 1725548.413712, a = 935.552285, b = 148.519201\n", "epoch 471: loss = 1725548.373396, a = 935.553797, b = 148.519200\n", "epoch 472: loss = 1725548.333882, a = 935.555279, b = 148.519200\n", "epoch 473: loss = 1725548.295154, a = 935.556732, b = 148.519199\n", "epoch 474: loss = 1725548.257196, a = 935.558156, b = 148.519199\n", "epoch 475: loss = 1725548.219993, a = 935.559552, b = 148.519198\n", "epoch 476: loss = 1725548.183531, a = 935.560920, b = 148.519198\n", "epoch 477: loss = 1725548.147793, a = 935.562261, b = 148.519197\n", "epoch 478: loss = 1725548.112766, a = 935.563576, b = 148.519197\n", "epoch 479: loss = 1725548.078435, a = 935.564864, b = 148.519196\n", "epoch 480: loss = 1725548.044787, a = 935.566128, b = 148.519196\n", "epoch 481: loss = 1725548.011808, a = 935.567366, b = 148.519195\n", "epoch 482: loss = 1725547.979484, a = 935.568579, b = 148.519195\n", "epoch 483: loss = 1725547.947802, a = 935.569769, b = 148.519195\n", "epoch 484: loss = 1725547.916751, a = 935.570935, b = 148.519194\n", "epoch 485: loss = 1725547.886316, a = 935.572078, b = 148.519194\n", "epoch 486: loss = 1725547.856486, a = 935.573198, b = 148.519193\n", "epoch 487: loss = 1725547.827248, a = 935.574296, b = 148.519193\n", "epoch 488: loss = 1725547.798592, a = 935.575373, b = 148.519193\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "epoch 489: loss = 1725547.770504, a = 935.576428, b = 148.519192\n", "epoch 490: loss = 1725547.742975, a = 935.577462, b = 148.519192\n", "epoch 491: loss = 1725547.715992, a = 935.578476, b = 148.519192\n", "epoch 492: loss = 1725547.689545, a = 935.579470, b = 148.519191\n", "epoch 493: loss = 1725547.663624, a = 935.580444, b = 148.519191\n", "epoch 494: loss = 1725547.638217, a = 935.581399, b = 148.519191\n", "epoch 495: loss = 1725547.613314, a = 935.582335, b = 148.519190\n", "epoch 496: loss = 1725547.588906, a = 935.583252, b = 148.519190\n", "epoch 497: loss = 1725547.564983, a = 935.584152, b = 148.519190\n", "epoch 498: loss = 1725547.541534, a = 935.585033, b = 148.519189\n", "epoch 499: loss = 1725547.518551, a = 935.585897, b = 148.519189\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAABQaklEQVR4nO2deXhU1fnHPyfDQBJUAoILYbW1oAgSCWKlWsAFiwJxReoCbrRqa12KRasCikqLCtq64W5FBUFTFBU33PgJCLKJQEEFISIiEBASIMv5/XHnhjs3d587S5LzeR4eJnfucs6dme99z3ve875CSolCoVAo6hdZ6W6AQqFQKMJHibtCoVDUQ5S4KxQKRT1EibtCoVDUQ5S4KxQKRT2kUbobANCyZUvZoUOHdDdDoVAo6hSLFi36SUrZyuq9jBD3Dh06sHDhwnQ3Q6FQKOoUQoj1du8pt4xCoVDUQ5S4KxQKRT1EibtCoVDUQ1x97kKIbOBjoEls/+lSytFCiGeB3wI7YrsOl1IuEUII4EFgAFAW2/6F34ZVVFSwceNG9uzZ4/dQRYhkZ2fTpk0botFoupuiUCh84GVCdS/QT0q5SwgRBT4VQrwVe2+klHK6af/fAUfG/vUCHo3974uNGzdy4IEH0qFDB7TnhSLVSCnZunUrGzdupGPHjulujkKh8IGruEsts9iu2J/R2D+nbGODgedjx80TQuQJIQ6XUm7y07A9e/YoYU8zQggOPvhgtmzZku6mKBQZT/HiEibMXs33peW0zsthZP9OFBXkp609nnzuQoiIEGIJ8CPwrpRyfuytu4UQy4QQE4UQTWLb8oENhsM3xraZzzlCCLFQCLHQTjyUsKcf9RkoFO4ULy7hlleXU1JajgRKSsu55dXlFC8uSVubPIm7lLJKStkdaAMcL4Q4BrgF6Az0BFoAf/NzYSnlZClloZSysFUryxh8hUKhqBNMmL2a8oqquG3lFVVMmL06TS3yGS0jpSwF5gBnSCk3SY29wDPA8bHdSoC2hsPaxLY1aDp06MBPP/2U8D4KRUOkeHEJvcd/QMdRs+g9/oO0WsRWfF9a7mt7KnAVdyFEKyFEXux1DnAasEoIcXhsmwCKgC9jh8wELhUaJwA7/PrbFQqFQicTXR5mWufl+NqeCrxY7ocDc4QQy4DP0XzubwBThBDLgeVAS2BcbP83gW+AtcATwDWhtzpFrFu3js6dOzN8+HB+9atfcdFFF/Hee+/Ru3dvjjzySBYsWMC2bdsoKiqiW7dunHDCCSxbtgyArVu3cvrpp9OlSxeuvPJKjBWvXnjhBY4//ni6d+/OH/7wB6qqquyaoFA0eDLR5WFmZP9O5EQjcdtyohFG9u+UphZ5i5ZZBhRYbO9ns78Erk28aQauvx6WLAn1lHTvDpMmue62du1aXnnlFZ5++ml69uzJiy++yKeffsrMmTO55557aNu2LQUFBRQXF/PBBx9w6aWXsmTJEsaOHctvfvMb7rjjDmbNmsVTTz0FwMqVK5k6dSpz584lGo1yzTXXMGXKFC699NJw+6dQZBCJRJJkosvDjN6XTIqWyYjEYZlMx44d6dq1KwBdunThlFNOQQhB165dWbduHevXr2fGjBkA9OvXj61bt7Jz504+/vhjXn31VQDOPPNMmjdvDsD777/PokWL6NmzJwDl5eUccsghaeiZQpEadLeKbn3rbhXAk/i1zsuhxELI0+nysKKoID+tYm6mboi7Bws7WTRp0qTmdVZWVs3fWVlZVFZW+l65KaVk2LBh3HvvvaG2U6HIVJzcKl7EcGT/TnEPB0i/y6MuoHLLJMhJJ53ElClTAPjwww9p2bIlBx10ECeffDIvvvgiAG+99Rbbt28H4JRTTmH69On8+OOPAGzbto31622zdioUrtT3SJKignzuPacr+Xk5CCA/L4d7z+maUVZyJlI3LPcMZsyYMVx++eV069aN3NxcnnvuOQBGjx7N0KFD6dKlCyeeeCLt2rUD4Oijj2bcuHGcfvrpVFdXE41Gefjhh2nfvn06u6GooyTq8kgFYbhVMs3lURcQxiiOdFFYWCjNxTpWrlzJUUcdlaYWKYyozyJz6T3+A0vhzM/LYe4oy5iHlGN+AIHmVlHWd+IIIRZJKQut3lOWu0JRh1GRJAo7lLgrFHUYFUmisENNqCoUdZhMXDyjyAyU5a5Q1GGUy0NhhxJ3haKOkwkuj0zLZa5Q4q5QKBKkLoRjNkSUzz0EBgwYQGlpqeM+d9xxB++9916g83/44YecddZZrvv16dMHc0ipmUmTJlFWVhaoHYp4Mn3xUKqoC4m9GiLKck8AKSVSSt58803Xfe+8884UtMidSZMmcfHFF5Obm5vuptRplLW6n7oQjtkQqTeWezKsqAceeIBjjjmGY445hkmx/Dbr1q2jU6dOXHrppRxzzDFs2LAhrsjGXXfdRadOnfjNb37D0KFDue+++wAYPnw406drtcQ7dOjA6NGjOe644+jatSurVq0CYMGCBfz617+moKCAE088kdWrnS2f8vJyLrzwQo466ijOPvtsysv3/5iuvvpqCgsL6dKlC6NHjwbgoYce4vvvv6dv37707dvXdj+FO8pa3U8m5jJX1BPLPRlW1KJFi3jmmWeYP38+Ukp69erFb3/7W5o3b86aNWt47rnnOOGEE+KO+fzzz5kxYwZLly6loqKC4447jh49eliev2XLlnzxxRc88sgj3HfffTz55JN07tyZTz75hEaNGvHee+9x66231mSctOLRRx8lNzeXlStXsmzZMo477ria9+6++25atGhBVVUVp5xyCsuWLeO6667jgQceYM6cObRs2dJ2v27dugW6Zw0JZa3uRyX2ykzqhbgnmnXOik8//ZSzzz6bpk2bAnDOOefwySefMGjQINq3b19L2AHmzp3L4MGDyc7OJjs7m4EDB9qe/5xzzgGgR48eNamBd+zYwbBhw1izZg1CCCoqKhzb+PHHH3PdddcB0K1btzhRnjZtGpMnT6ayspJNmzbx1VdfWYq21/3CJpOjK7y0zc/ioUzta1jtUuGYmUm9EPdUW1G64CeCnjo4EolQWVkJwO23307fvn157bXXWLduHX369Al07m+//Zb77ruPzz//nObNmzN8+HD27NkTeL+wyWR/tde2ebVWM7WvYbcrE8IxFfHUC597Mnx+J510EsXFxZSVlbF7925ee+01TjrpJMdjevfuzeuvv86ePXvYtWsXb7zxhq9r7tixg/x87Qfy7LPPuu5vTCv85Zdf1pT427lzJ02bNqVZs2Zs3ryZt956q+aYAw88kJ9//tl1v2SSyf5qr23zmoY2U/uaqe1yQ0UoeadeWO7J8Pkdd9xxDB8+nOOPPx6AK6+8koKCAtatW2d7TM+ePRk0aBDdunXj0EMPpWvXrjRr1szzNW+++WaGDRvGuHHjOPPMM133v/rqq7nssss46qijOOqoo2r8+8ceeywFBQV07tyZtm3b0rt375pjRowYwRlnnEHr1q2ZM2eO7X7JJJP91X7a5sVazaS+Gt0wdrlgM+EzsCNTR0EJ8e23kJsLhx4a+qnrTcrfTPFr7tq1iwMOOICysjJOPvlkJk+eHDfRWRcJO+VvJqepDbttmdJXq7S7VmTCZ2BHptzLUFi7Fu65B55/Hq69Fh58MNBpGkTK30zx+Y0YMYKvvvqKPXv2MGzYsDov7Mkg1dEVfh78YbdtZP9OjJy+lIqq/UZUNCJSHkli5YYxk+kRLpk0CgrM//4Hd98NU6ZANAp/+hPcfHNSLuUq7kKIbOBjoEls/+lSytFCiI7Ay8DBwCLgEinlPiFEE+B5oAewFRgipVyXlNZnILoPXGFPKqMr/A7lk9I28+A4DYNlJwEUUCciXOpKemNLVq7URP2ll6BJE/jLX2DkSDjssKRd0ovlvhfoJ6XcJYSIAp8KId4CbgQmSilfFkI8BlwBPBr7f7uU8pdCiAuBfwBDgjROSokQIsihipBIltsuVSOtIGGyYbZtwuzVVFTH38OKaplQmG4Q7ISxLrk06mQ8/YoVcNddMG0a5OTATTfBX/8KhxyS9Eu7RstIjV2xP6OxfxLoB0yPbX8OKIq9Hhz7m9j7p4gACp2dnc3WrVuTJi4Kd6SUbN26lezs7LS1IdHoiHQP5dN9fZ36kPe9ThXKXrYMzj8fjjkGZs2CUaNg3Tr45z9TIuzg0ecuhIiguV5+CTwMfA2USikrY7tsBPQ7nA9sAJBSVgohdqC5bn7y07A2bdqwceNGtmzZ4ucwRchkZ2fTpk2btFw7jOgIO4s1Swg6jpqVdHdEprgS6stCo0yZW7Nl8WLNUn/tNTjoILjtNrj+ejj44JQ3xZO4SymrgO5CiDzgNaBzohcWQowARgC0a9eu1vvRaJSOHTsmehlFHSaMlcdWQ3mAqtiIMNnhdGG6EhKNCMt4YazLLFyoifrMmdCsGYwerfnVmzdPW5N8RctIKUuFEHOAXwN5QohGMeu9DaCPl0uAtsBGIUQjoBnaxKr5XJOByaCFQgbvgiLZBBWVRMUoDJeG2WLNEqJG2HUSTVVhh97/8ooqIrHr5ge0mOtljHd9YMECGDsW3nxTE/I774TrrtMEPs24+tyFEK1iFjtCiBzgNGAlMAc4L7bbMOC/sdczY38Te/8DqRzndRZdVEpiC190UXHzfQc9zkhYK4+LCvKZO6of344/k2qbr2LYPnBj/0EbKegWexAxrqsrSustn30GZ5wBvXrB/PlazPq6dXD77Rkh7OAt/cDhwBwhxDLgc+BdKeUbwN+AG4UQa9F86k/F9n8KODi2/UZgVPjNVqSKoKIShhglYxIwVelpwxbjTJmYbfB88gmcdhqceCIsWgT/+Ie2yvSWWzQfewbh6paRUi4DCiy2fwMcb7F9D3B+KK1TpJ2gopIMl0oYk4CpCqcLW4zzcqNsL6udJbROxHjXBz78UHO5zJmjRbvcdx/88Y8QQhLBZFFvVqgqkkPQaI+wokTCngRMVdRImFEyxYtL2LWnstb2dKx0bVBICR98oIn6xx9rC44mToQRI7R8MBlOvcgKqUgeQV0jmRxXbfTBzx3VL2lRMmH132ohFEDTxo3UZGoykBLeeQdOOglOPVXLA/PQQ/DNN1pYYx0QdlCWu8IFL5auU1RMuuKq051ILsz+27lydpQ7F3NR+ERKePttLfpl/nxo0wYefhguvxzSuJAvKErcGzBeBdDJNeIWopdsQbXqA5ARYYNe+h921af6RMoe0FLCG29o7peFC6F9e3j8cRg2TMsDU0dRbpkGShihipDeED27PoyZuSLlbQqSJsHrZ5DJLq5kEdb30xEpobgYevSAQYNg61Z48kktc+OIEXVa2EGJe4MlLFFOZ4ieXR9KbdwVyWpTUCEKu+pTkHZnalWjpBoN1dUwYwYUFMDZZ8PPP8Mzz8Dq1XDFFdC4ceLXyACUW6aBEpYoJ+oysHOreBmOB2lrMgiaJsHrZ5AM90Smr3hNitFQVaWJ+l13wZdfwq9+pRXLGDoUGtU/Kax/PVJ4Iiw/biJx41YCM3L6UpDURIc4iY5dH5rnRtlTUZ2y1LB+hMgo1FapECD+M7AT4YXrtzFr2aaa2Pe8nChjBnXxLMx+HkjpmJwOdZ6hqkpLuXvXXVpe9aOOghdfhAsugEjE/fg6inLLNFDC8uMm4jKwEpiKKlkr7M88HNfdCSWl5ZhzSedEI4we2CWlqWG9rno1u2+shN38GdiJ8Avzvotb1FRaXsHIV5Z6dq14fSDdVrycG6YuSa7v24JQvp+VlfCf/8DRR8Pvfw9ZWTB1Kixfrlnr9VjYQVnuDZYwQ/WCRsX4GWLr+5otWYlWSUhCraRcyRJzsyXbt3MrZiwqsR0p6PtbWaIAESGoltLyM/Bzj/wUAfFiGRcvLmHKvO9qFY5KVqI1Iwl9PysqtDJ2d9+txah36wbTp2v+9ayGY88qcW/ApDsFrJ3A2O0L1pasLux+KwoFcTdYuUlmLCrh3B75zFm1pda5vBSmrpaSb8efafmen3sE3h8GXtxpE2avtq0ImIp5Ad/fz4oKzYd+zz3agqOCAi2v+qBBDUrUdZS4K9KGlcBEIyLO5w77Rad4cYmt0PmdaAs6oWjnJpmzaovlw8VLYepmOVHb96zukT5SscKrT9qLZex0T73MCxivk1T27YNnn9VEff16KCyESZPgrLOgAZfpVOKuSBt2AmO3TRcMK/xOtCU7wsVtu5Hd+yopXlziuWB3386tmPr5Biqq4iU+muUv14ybZWw3ahDgaV4g6XVi9+6Fp56C8eNhwwYt/e6jj2qpeBuwqOsocVekFTuBMW/rPf4DWws4GhH07dyqZpLVS2GMoGLsN4rDi1ulosrZV251jwrbt2Ds6ysCR8t4wW7UcNEJ7TxZ+Elb67BnDzzxhJZut6RES7/75JNaKl4l6jUocVfUCZyEoqpaMnXBhhpXjpcSekFD7fyGftqV+TPjVwjDmC+x85Mbt+flRmnSKIsd5RXpT49QVgaTJ2tFpjdt0hJ7Pfcc9OunRN0CJe6KhElFHLSTBVwtsa2wZOceCBqf7zeKw7w/WPvLnfzuycApft4Y+bO9rIKcaISJQ7rb9jHpOfJ374bHHoMJE2DzZujbV4tT79MnnPPXU0QmVMArLCyUCxcuTHczFAGwigbJiUZCjyv3EnVihwDLaJR0LM4puPMdy6IbzXOjLL7j9KRe28sCqojNdrdopKTcy1274JFHtMIYW7Zo6XfvuEOz2BUACCEWSSkLrd5TlrsiIdwm04oXlzBm5oqafC/Nc6OMHujfN6zvf9O0pZbi44SdeyBR10YQQSu1EHa77WEKpvnhaHcP7ba7uY3s7mWgPuzcqaXavf9+LZlX//6aqJ94YmLnbWAocVf4xvjDcoqDLl5cwshXlsaFNW4vq9BSDOA/TE7f348Fn6y0A0HD/7z6pxMJL7QSPi8hmWBvuQetIOWrDzt2aEUxJk6E7dthwABN1Hv1Suy8DZSGF9mvSAjzEno7Wufl2FYQ0qNDgqCnO4g4TKDp7yUz7UDQrIVel9UHPb9dhkovC6FyohGG9mobagUpT33Yvh3GjNHyqOtul88/h1mzagm7r/M2cJTlrvCFFwtQF4Mbpi6x3SeRMDk7Cz4Zvn47gob/eZ2QDXp+O+Gzs8itUh8Utm+R1ApSNdu3btUWGz30kOaKOftsuP12bWVpIudVAErcQyeZvsBEzh1Wu5x+QALizu2UTyXRMLl0l/FLJPzPi6/f7vxZQtBx1CzfD4UqKcmJRjw9DMNKS2HXh6Oi++DWW+Ff/9ImTc87D267DY49NqHz1vfKVH5xdcsIIdoKIeYIIb4SQqwQQvwltn2MEKJECLEk9m+A4ZhbhBBrhRCrhRD9k9mBTCKZ1WMSOXeY7bL7AeXn5dQqOD2yfyeiWbXdJ9GIv5WUdqSi0LUdya6OZHV+0ETa6TN0+nxSmSkTavfh4N2l3P7xs8y8/2JtVemZZ2oZGl95xbOwW50X6n9lqiC4hkIKIQ4HDpdSfiGEOBBYBBQBFwC7pJT3mfY/GngJOB5oDbwH/EpKaTuWry+hkPoKSTNBklqFee4w2+U39NEuWgbSZ3WHRbIjNryELpo/w1SFpnqleHEJT03/jMHvTuHiJW/RpKoCMXQo/P3vWl71BM5b178/YZBQKKSUchOwKfb6ZyHESsDpLg4GXpZS7gW+FUKsRRP6z3y3vI6RTF9gIucOs11Gd4i+1N84meVliJ9otEOyKhP5PWeys2oaz99x1CzLfcyfYbrdVfGN+56i5yZQ9PjjWnKviy/W3DGdwhm1NUQx94Mvn7sQogNQAMwHegN/EkJcCiwEbpJSbkcT/nmGwzZi8TAQQowARgC0a9cuSNszjmT6AhM5d9jtsprQ9CLQTnnNvSaasnow3DB1CddPXeKYSybV5/RyTT8C7OczTLvwbdyo5X154gmtYMall2qi/stfpq9NDRDPoZBCiAOAGcD1UsqdwKPAL4DuaJb9/X4uLKWcLKUslFIWtmrVys+hGUsyfYGJnDsZ7fIbjmb0+9vhZSRhl88d3OcS7ApCJ3LOIASZA6kTfubvvoOrr4Zf/EJLF3DJJfC//8HTTythTwOeLHchRBRN2KdIKV8FkFJuNrz/BPBG7M8SoK3h8DaxbfWeZA6JEzl3Mtrl19XjJYTSaSThVs1Ix6kOqN1Iw+2hEnb6WrsH403T7Bd3ZYK7xXa0sW6dlkv92We1HS+/HG65RYtbV9iS7HkDV3EXQgjgKWCllPIBw/bDY/54gLOBL2OvZwIvCiEeQJtQPRJYEFqLM5xkDokTOXeoy8Px7+pxE1AnK9RvXhmrazmNNLyk5Q0zhtruWlVSOrq20ulusXo4PvLkbApK3qH9rBlapaOrroJRo6BtW5ezKVKxytaLW6Y3cAnQzxT2+E8hxHIhxDKgL3ADgJRyBTAN+Ap4G7jWKVJGkT4SCZH06yZwssrdwvK8Lp13upbTSMMu7NDtnEFxWl2bqSstjZ9Bh20l3DdrIm8+ehWHzXoNrrlGK2v38MNK2D2SilW2XqJlPoVaReYB3nQ45m7g7gTapUgBiVTQ8esmsEsL6yVEz4/VbPeAcRppmCOAzGXswvZtuyU+y8SVlt+XlvOLrRu49rNpDP7qI/ZFojzbYyCTe53Lggcv8XUuFcaYmlW2aoVqAybRL5gfN0EiPmOncm/S9Pe5PazbZPdw0Ss46W2aNKR74HZ6Jd/FDZRxKy2/+orH336AU5fOYU+0MU/2LOKJ48/mp6bNyffZVpX0SyMVq2yVuDdgUr2M2+1hYK4AJCXsKK+gWU6UaETE1Qy1KhItgTmrttheG2rXIjUWptCF5t5zuia86MwJp+pMGRUBs3w53HUXTJ9Ov+wcnjrxPB7tUcS23GZAsLamrd5qhpH0AicocW/QjOzfqVZKXr9FlsPCbNEZC1qUllcQzRI0z41SWlbhOAFaUlpO7/EfWFrb5oeLVV1WL0KTqFvBaiGYW81XL4Tm7liyRBP1V1+FAw+EW2+l0fXX02rDXnJmr0YkcP5UJ/3KVBdQKqKflLg3dMyzKWkqRek2aVpRLclt3KimWpFdSgXwPtQPIjRhuRXCjnwJpV2LFmmi/t//QrNmWvrdv/wFWrTQztMycddJKkeLme4CSnb0k8rn3oCZMHt1nKsDgudat1sg5Paejt80Cm4RLl4iD+wExUlo3KIcvPQ1GSQUfbFgAQwcCIWF8NFHMHasFrs+dmyNsIdFKhdjNfS878pyb8CENUR2spDAW5oCL7HmRtE1uzaC9COI39PpnqXTUgz0Wc6bpwn4229rIj5uHPz5z3DQQUlqZWoXYzX0vO9K3BswYQ2R3SwkL35tp0lGsBZdfVhr56Jx60cQoXG6Z+mcLPT1Wc6dq4n6u+9Cy5Za+t1rrtH86ykgVYuxGnred+WWacCENUR2spC8Wk96+Tw933jz3Ch5OdG43OOApcvDqR9ubhK/OeGdrpVOS9HTZ/nRR3DKKfCb38DSpTBhAnz7LfztbykT9lRSJ/LxJBFluTdgwhoiu1lIYWQz9OLyMPcD/GeudMPpntm5iFJhKdq2q3tr+OADuPNOTdwPOwweeAD+8AfIzU16u9JJJuTjSSeuxTpSQX0p1tFQcSoQAeHUOg1ScMQpoibsVL7Fi0sY+/qKuBBOSGOhDCnhvfc0Uf/0U2jdWrPQr7oKchqGW6IhkFCxDkX9J4xY4OxoVo2A5+VEGTOoS9w5Eq39GmTS1Ok9Jyve7/2wS2xmdR+CYLwHrjHxUsLs2Zqof/YZtGkD//43XHEFZGcn1A5F3UKJewMnjKpIZmHbW1kdt0/QCTQv2SCdXB5uEThWk51B7oddjH7TJo1CEXZje/S8NLXaJSW8+aYm6gsWQLt28OijcNll0KRJQm1Q1E3UhGoDJ9G4bbvjr5+6JOE4b7eFTW6TY16yPZqt+yCx0WFMpPopJBLXrrdXaYuOCgvhrLPgxx+1Ckhr1sAf/6iEvQGjLPcGTqJx20FdH4m0Dbz5zL3Ewpst/yBCHSTkzpxHZ9eeypo0EF4KiQhZzen/m8df/u8l+PFbrfrR009rdUqjUdvrKhoOynL3QbpWH7qRSLucVml6sWLdIkESWRFod259EtVrFaq5o/oxaUh3T2FxQVat+g25M+fR315WEZffB+ILiRgRspoBqz7lzWeu4/Hiezigah889xysWqW5YJSwK2IocfdIIoUtMrldfTtb16/t27mVJys2iOvDK1bnFuxPDubn3pvj6O0KhASJjfZ6bh2vxUeMhUSyqqsY+NVHzH7qTzzy3/FEqyoZOXgkX7z9f1oB6kZqEK6IR30jPJKpqUoTbZdditw5q7Z4cjcEcX3YYRWlcu85XS2LaARx+XiZ2A0aG+1n0tjrw651Xg5FXQ+lzZuv0vLx++iw5Tv+17I9fxp0M0t7ncpNvzu6wcRsK/yjxN0jmZqnws1n7iZSTsdPHNLdU+4VXdjs4t29rAi08+/rudWtYtYTebg63ZtkL4/3kkfngAg8tG8ZHP1HCtesga5d4ZFX+NU55/DvLDXgVrijxN0jmZCnwkqQ7NrVLCeaUMIuq/JzESHifOhW+dL1/f3GtLuNQMJ8uKY7FaxVHp1oRNC0cSN27yrnsm8/5S/zX6HpxvXQvbuWV33wYK0ItULhEfVt8Ui681TY+db7dm5l2S4h7BN2GXHrV1FBfs0+5hhrK5+331wtOm7iHWSi0450p4K18tHfN/golrRez5rp13PrjPtoemhLLcTxiy/g7LOVsCt8o74xHvE7aRY2doI0Z9UWy3aVmpbB67gl7LLqVyrE0E28w3y4ZoKLreYhOPZU5ub9j8Hn94ERI+CQQ+CNN+Dzz2HQIBBpqp6iqPMot4wPUpWq1AonQbJql58kVm798rr0P5E0Bm651cNIAqW3zy6bUiITv76/F3v2wJNPaul2S0rg17+GyZPh9NOVoCtCwVXchRBtgeeBQ9GCFSZLKR8UQrQApgIdgHXABVLK7UIIATwIDADKgOFSyi+S0/z6gRexsPONZwlB8eISy5A+K7/u7r2VdBw1y7MoFS8usSxGrbfJuJ9XP7ZTf+22JyqobqkMEp34teqnJeXlmoj/4x+waZOWfvfZZ7VUvBkg6plac1ThHy+WeyVwk5TyCyHEgcAiIcS7wHDgfSnleCHEKGAU8Dfgd8CRsX+9gEdj/yss8CoWdsUsqqS03N8slrmNI+zeV0VpeYXjdcw4WbrGGHmvIZlu/bVqSxgToG6x5dlRaw+lWezK9lUGCz0tK4PHHoN//hM2b4bf/hamTIE+fTJC1CH9E82KcHH1uUspN+mWt5TyZ2AlkA8MBp6L7fYcUBR7PRh4XmrMA/KEEIeH3fD6gld/tu4bj1gIgZ3/W/frThzSnbJ9tYXNi9/cyQ/9wrzvahYTORWrNhLEf+90jNfVuW7+9O1lFbUmia0msc0pfV3Pv2uXVhSjY0e46SY45hj48EPtX9++GSPskP6JZkW4+PK5CyE6AAXAfOBQKeWm2Fs/oLltQBP+DYbDNsa2bTJsQwgxAhgB0K5dO7/trjf4mdwrKsjnhqlLfJ0HnK1vN9Fzi8nWrTshtMSEZiIxt5Fu/QZph917+rXtarcaLe683KitMOuYLXCvK0nBwl//88/w8MNw//3w00+aL/2OO6B3b0/nSweZMNGsCA/P0TJCiAOAGcD1UsqdxvekVvHDV9UPKeVkKWWhlLKwVSvrJfANAb8hfkFCAt2SXjkxsn8n3GzL8ooqS2GH/W6jEgdhd2uH3Xt63L25LWNmrqhlce/aU0k04m4lG++VV1GL89fv2AF33w0dOsAtt0DPnlpe9dmzM1rYIdxwU0X68STuQogomrBPkVK+Gtu8WXe3xP7/Mba9BGhrOLxNbJvCAr8hfkFCAu1+nCJ2PieKCvK56IR2rgJvh5UAmwmSutcYd2+mtLyi1jUrqiVNGzeqCfm0cm9B/L3yImo1oaMdm2pFpzt0gNtugxNP5MP/vEHvk/9Kx+KtGZVozo50r+VQhIuruMeiX54CVkopHzC8NRMYFns9DPivYfulQuMEYIfBfaMw4Td+3rx/Xk6U7GgWNzjkT7dLwHXRCe08TZSNK+rKRSe0sxVEYu3wI8A6ESE4t4dz7VTdPaJfX79H+T4tyh3lFTULrO6/4FhXIXNLiiaAuSO6U/Ta49C+PYwZo02QLlpE8Z2PcfXqSMYlmnMi3Ws5FOHixefeG7gEWC6EWBLbdiswHpgmhLgCWA9cEHvvTbQwyLVooZCXhdng+oBVuJldHVAr7HK52EU3JBojXry4hBmLSmyFOicaYcygLpbXcEooBprbZsaiEgrbt7AMmTRXIdIFWN/XKjY+O5pl6V+3S3hmd0/01zdNW1qr783LdnD98lnQYajmXz/3XM1i795dO+/4DzIy0Zwb6VzLoQgXVSA7xTgVk/b7owpSNDoIiRSa9lIqTz+Puc1e+mf1oIRwinJb9aFF2Q6uWvAal37xBrmVexHnn6+Jeteuccd0HDXLco5BAN+OP9N3GxQKK1SB7AzCT4petwUlQYpGB8G2GhDYPkSMbW8Wcx2VllX4ipbxEr3hZGmGtRinqCCfJlu3sP3OeyiaN5Psyn2U9B9E0/vvgaOPtjzGLjonL1crpqEWCymSjRL3FOM13MzN5eJ15WgY+M2IaW57aXkFOdEIE4d095UWIZFMnH7dC7Ziu2kT/POf/O7xx2HvXrjoIrj1Vtp27ux4PrsBsZRqsZAiNajEYUnAaWGN13AztwUldrHrXiJg/GIXRdG3cyvPRZ31tvuprhQ0esNv2UGrxUoTn53D10Mv1xYf/etfMGSIVsru+efBRdhBm7y1264WCylSgbLcQ8bOKlu4fhtzVm2pVVEIrAXLzcK3e18SvvVXVJDPwvXbeGn+BqqkJCIEx7VrxoxFJZbWp1uSM9if2MypulKQieAgVrFRbFvv/JE/zpvBkGWzyZKS1487nfuOO4fKjkcwclduzTJsN5xGHalYLKTcPgol7iFjZ5VNmfddjYhJqBE1uwlJN5eE3ft6eGCYP25ztEyVlMz9elut/YxFnZ3argu3l+pKft0rQcoOfl9aTpsdm7l63iucv+w9AKZ3PZVHfn0+G5vFFl77dJ04Zbn045oKgnL7KECJe+g4WdTmv52iPvp2bsXUzzdQUbX/yGhE1Fj4TuIR9o/bzzJ8P+X5ErVg9XumV4lyiqm3Pec33/DQ+w9zxhfvUC0EU489nUdPOI/vDzqk1q5+QhmdRh0L12+Le9hDuIuFMrXeryK1KHEPGS/1MXV0wbES46kLNtQWK8OfTuLRO+QYaz/uAnN5viBpjL1YsFYx8G7timPtWi1NwH/+w4BII146bgD/7nkuPxzU0vE8fu6F1ahDHwUZWyvAcSGXX1SOGAUocQ8dK4vaLarFytKqqK59REW19OSyCPvH7fWBZZzM9eJOsUtjXLav0jJHvRE/o4k4q3j1ak3Up0yBxo3hz38mMnIkB2yWRGavRhhS+9othCpeXMLY11fUvJ+XE2XMoC6Ba8VKYM6qLZ764oVMqPerSD9K3EPGymrt27lV3OQjxAuOH9F127d4cQlZNi6KoD/ukf07ccPUJa6Z4fxO5ur7jpm5oibPPOxPv2vcR8foinFDwP5RQ/ZONgz4E63fnsm+SJTXep9L89tv4XenFWjXaR1/LbvFZn07t2Lk9KVx7rLS8gpGvrLUsb36dyEVaxPcqlopGgZK3JOAldVa2L6FrZvCjyvHSaB1QbISdq8hhFZttPMTm/Gb6wW0ezVh9uo4cQdvhT6cqJnP+PJLGHcTcto0WjRqwhPHn80TPc9ma9M8cj7ezN6W1iMEO9fShNmr44RdxzyqsmqvVaSUTphWdRglCRV1HyXuAQgSieLkprAsiZclQBAnJG4Cbeeq0JNzTZi9mhumLrFss9sk7LiirjUPKK/hnF7x6kby6orJiUa4q2MVnHcezJhBRW5TnjjhfJ4oHMz23GY1+7nNQ1h9Znb59L221xgpZWxv2Fa1yhGjUOLuk2SEmdlZWlbb7MrQObkq9ORcTm32E2Eh0JbRS6ktyknUMvTqI3ZyXejRMn3KNnLPl6/RetxsOOggVl31F4Yf1JsfGuVaHue3yLfTKMtre/VIKWVVK5KJEnefJCvMzM7ScjunF1eFXVELY5vdyuSZr7O9bH9KgUSFyauP2Cm2f+6pB8Gdd8Lrr0NenpZ+97rruOLxxfzgsViJ2wI0PU9OJEtQZZrwjmYJf+0NMbGbQmGFEnefZFqYmZurIicasX3f2Ga7OHE9h7qXdAhBLVGvPmKrh0CvH9fw4NzX4ZYPoHlzuOsu+POfoVmzWn00Y36AeFmAVlpeQTRLkB0rOA7x0TLmhGnRiPDlWlMowkKJu08yLczMSbzy85xzqjfLida8tosT17cHqWPqV+C9zFuAJsKHffkFNy+YSq81C+Hgg+Gee+Daa+Ggg+KOsfu8IkLUSgHsdQFaRbXkkNzGrLgz3vq2SpgGkCWgWrqnR1YowkQlDvOJWzIrv0mrEsXuoaIP/YsK8hnZv5M2QWtidyyeXN/f7jxO1wFSmgSraNc3zH3vHmZMGUmv0vXwj3/AunVavVKTsIP953X/BcdaLqryitWDwG4UVS33f0eUsCtShRJ3nziVIrPKLpjs0mpW4hWNCHbvrax5wAAckF17kFZRJWtE2OmhVby4hN17K321K1Q3lZQwZw707Qsnn6yFN95/P3z7Ldx8MxxwgO2hfkrH2WWstMLqQeDUZ5X1UZFqlFvGBqeoCTsXQjpyepj91Xm5UXbtqaxxCZjdJmZ0QXKK2PEaW24kFDeVlPD++9pE6SefwOGHw6RJcNVVkGsd/WKF17DAIAvQjLitV1DL/xWpRIm7BUHDHd0iTpKFUbx6j/+g1rJ5vbi0lV9dxo4xLlgyYpWnxo2EJw2lhHfe0UT9//4P8vO1nOpXXgnZ2cHP6wG/C9CM2KVT0FHL/xWpRIm7BUEtcLeIk1TgFOtuFznj9PDya20mNGkoJbz1libq8+dD27bwyCNw+eXQpIn/8yWIefTmFvZpl04BVJSMIvUon7sFTpEhTpOlbhEnyUYvvWeF7mu2mzi18wn7sTb1WG/fwi6lFp9+/PFw5pnwww/w+ONa5sarr06bsAeZPykqyGfJ6NOZNKS7Jz+/QpEslOVugZ3vVC8HB9bWbr5LAY0g+El1YFd6D7RMi3rqATuMKYjdYrVBUl5RHXe8VX4VR6qr4b//1WLTFy9md5v2PHjuX3mm4284ZNuBjFyxJW2CmOj8iVc/v6qYpEgWrpa7EOJpIcSPQogvDdvGCCFKhBBLYv8GGN67RQixVgixWgjRP1kNTyZ2URNm4TRbu0Frftrh13p0cqFsL6uoOYcdeblRuo99h+unLqm5Zml5BUhonhuNs0L3mITdSxtqqK6G6dOhoADOOQd+/plFYydSeNFDTP5lHyoijSgpLWfk9KVJjTRyCltNVSm8VEdXKRoOXtwyzwJnWGyfKKXsHvv3JoAQ4mjgQqBL7JhHhBARi2MzGqvwOTuL2Phj9xN25wW/hZQTnbDbXlZRy1cMmkWe27gR344/syZ23muh7ziqqmDqVOjWDc4/H/buhf/8B1au5EqOppz4r0pFlWTs6ysS6pMdbsIaqH8+UYWyFcnE1S0jpfxYCNHB4/kGAy9LKfcC3woh1gLHA58Fb2J6MA+rrep9Qu0fe5jZ+Nx8/+ahvFu0Rpht8ZUzXBf1ceNg5Uo46ih48UW44AKIaIJuVRjDaXuiuLldRvbvxMhXlsYVTbHKH5MImZbKQlG/SGRC9U9CiGUxt03z2LZ8YINhn42xbbUQQowQQiwUQizcsiW8KjTJImyXixfsrETd92+2OK1GDnmGFANhtsXTKKWyUrPMjz4aLrpIE/Jp07RFSEOH1gh7srFyv3gSVvPsdMhBT6kYHSgaLkJ6iOSIWe5vSCmPif19KPATmhv6LuBwKeXlQoh/A/OklC/E9nsKeEtKOd3p/IWFhXLhwoUJdSQVBJn8SmTCzCrjo12xB7tMg34KXNiRE434cy9VVMALL2jl7L7+Go49Fu64A4qKICvL8p5YhQ/qRIRgaK+2jCvq6rvtdhWVmjTKsryefh/tRmpO9znId8OqbSqyRuEVIcQiKWWh5XtBxN3uPSHELQBSyntj780GxkgpHd0ymSLuYUcuhPHj9VqmTQDfjj/T8RxBFlM1z40yeqC3+qDs28ficQ9yyL8nkr99E6taH0npX0fxw8mnMeHdNXEraI3ujpxohHN75DN1wQbL2rE6udEs7jmnm6/PxE6km+dG2VNRbfvZdBw1y3aexZyLHWqv4tX7pKcKtvs++fnOqcgahZnQxV0IcbiUclPs9Q1ALynlhUKILsCLaH721sD7wJFSSkezMRPEPRlWlF/rL5Fz6ud1+sE7Haujjwx8LUbauxeefZaysePI3bSRpYcdyYO9h/LBL3oSjWTVqijl1Ha3h5Dfz8ROpAUwcUh3W7G0u1dWVZSyo1mWcwNW+wb9PqXKylcPkLqFk7i7TqgKIV4C+gAthRAbgdFAHyFEd7Tv7jrgDwBSyhVCiGnAV0AlcK2bsGcKycgL43XCzM8PymnS1C1NgtNEXVwxaa/93bMHnn4axo+HDRv4tu1RTDhvDB8e0QNiq3KdLHFz2/XJ6A6jZtnuV15RxU3TltqWCzTjlKLZb+lDu3BYO5eXXehskO9TKvIWJaPKmCJ9eImWGWqx+SmH/e8G7k6kUakgFVXpveR+9/uDMia3sjq30w8+tMpA5eXw5JOaqH//PfTuDU89xZnv7a0Rdb8IqJkUtkvjoKO/50V8fEX1GDDfZ7c2eSXo9ykVkTXpSHynSB4NMv2AVYyzn9SuXunbuVWt83qt/pNIrLPdDz7hiJ+yMpg4EY44Aq67Dn75Sy1r4yefwGmnEckK/nWSUNPnob3aej7O7V4lsvZAD4nMiUYchT0vJ+o5VXCznGigfP+piKxRoZn1iwaZfsCuKr2ZRFeXzlhUEndeAZzbI94V4PcH5SX6xe4H77WcXS1274ZHH4UJE+DHH7W86i+/DL/9bU2bJsxenbBlq/dZj4qZMv87vJzSTXwSWXvgpYzhmEFdavZ1ShUczRLs3lc7HbPeRieCjkD8kGlVxhSJ0SDF3UkM8nKi7CivSHgyye4BMmdVfEy/3x+UF7Fx+sF7ETpdrHds3so1q97hinmv0qR0G5x2Gtx+O5x0Uty+bg+b5rlRchs3qhG+3XsrLcMQjX0eV9SVcUVd49xnWTaukWSKj5cyhsY8/0bMqYLL9lVapmO+adpSy+ONBH4w+yAVDxBF6miQ4u7kY2/apBFLRp+e8DW8WuR+f1B+xMYNq4lcgHEvzeOC+TO5asFrNN/zMx//ohA58Rl+O3xQrWPdom9yopFaoZR2D4SyWNk/477Gh5FdxEiyF5IFnacwP0g72kwUV0npyYI3C7zujgpL4FPxAFGkjgbpc3cSg7D8i24+Un3V5A1Tl5AdzSIvJ+rJJ5wdtf7IjDVTvWA173DPi5/x/U238t6/hnPzx8/zRX5nBl9yP5eeN4ZbfzjA8lgnrPqiPxTKK6pq+aW3l1U4Js4qKsjn3B75NfnxI0LUcnOFjV0SOT0FhJ8kX251aN3mWVSiMYUfGqTlXlSQz9jXV1jGJoc1xHeyyM0W6PayCnKiEddiEBc98VmtNLugPaH9Wq9G985Be3Zxxef/5bJFMzlo727e/WUvHuw9lC8P+2XN/saHnptrCKwtW3O/rdzpTm4KfR5Dd81UScmMRSUUtm+RkjKG+sS73m6/oYJuuX/cDItkR7OoUMj6RYO03AFGD+yS1FwxTlEaQSJkiheXMPfrbZbvVaP98P1YcN+XlpNXvpObPv4Pcx+9jL/830vMbX8sZw5/kKvOvT1O2CH+oecmQgItUsiMl4cC7HdTmPuTriyKRQX5zB3VzzI7qJ/r698Ju8pcboZFsqNZVJbK+kWDtNwhNf5Fu8nLID/SMTOdU9/6srK2bOHOeVM4+7Nicvft4c1Ovfn3iUNYdUhHLayvMn5Zvlms3QpBS7C0qP2IkJVFmu5QvTCur/cnyNxBsqNZ0n1/FeHSYMUdwk3P6we/P9LixSW2SbWMuA7RN2+G+++HRx7h4rIy3jzqZCadcAFrWrUH9of1LVy/jSnzvquxUs1i7SW1sFVbmuVEPfVDxywqXu+b3YrfRJfWhyWuQQ2LZEezqFDI+kWDdcukE7+LifwMiy0t6h9+gBtvhI4dNXEvKkJ89RUVL0yh7MjOtdxGc1ZtcXQ/mF1OdhjFuXhxCbv3VdbaJwvIsjmJWVS83Lfbipdzg6GSlD6iua14ecKTkWGmfdZdPcYCKGbMqYqBUIvBmElHWmtF8mjQlnu68Gu5+RkWG5fx8/338I9/wOTJWhreiy+GW2+FX/1KawfWLhwvw3PjqMdLIZMJs1dbJg9rFss66cUidbtvxYtL4kYcOuUVVbw0f0OtGHm/k5GpDBW0m9y895yugZPOuaFCIesXStzThB+XkJuP24gEHnthDlXXTOesz98kIqsoGXg+/7vsT4xZuY/vn15D67yNjj9av8NzL+4CuwdGaVmFo6hYuVLsxM2pQLjd6lm//uRUufLSleclXa5KRfg0WHEPI7VpqtKj2omn+cefv+NHrp73Cucvf5csKXml66k8csL5bD74cJi/syZDo9vkq1/fblFBPgvXb6uxjvX4c6CmHKDb6lIrUfEbmuck1HaJvzLVn6wmNxWJ0iDFPYx43rBjgp0eFHaWrR573ab0B66Z9wrnLX8fgGndTuPRE86npNkh2skt3CFOVqDf4blV/PnUBRuY+vmGGleMlbC6+XP9Wq92Iw6BlozMnOslk/3JanJTkSieinUkm1QX6wijiEaYhTj8FGIwPgS6lG9h2EcvUrT8A6qF4OVj+/NYr/PYdFDtGHMrnKo3+cFLERCdiBBUS+lppOOnGpLuwrHKwX7RCe1q5anJdH+yKsGn8EJCxTrqI2EMecM4h1N+FisLVf/BH7b5O+77bCqDV3xIZaQRr/QcyEM9ioi0bUv53krwGG7YOi8nFMHz0+cqKVnn8kDR22Qn7Pryf7AeMTmNgOqKMKrJTUWiNEhxD2PI67UQh1Nkh1usuFk0p/3nHe6Z/TyDVn7MvkiUZwoHMfn4c2jcJp95sdGC1XmjEQEyviqSADocnBOKa8nPhK95dab5HlmlyjViVw1JfxDWJQEH6/671V1VKLzQIMU9jMUgbudw88l7WYpf86BYsQLGjeOFl6eyJ9qYJ3oW8eTxZ/NT0+YACFOIonlyc0hPrfiFeWGSVTqDIBEZXhY16VRJGTdiMedqsQpl1MlPUsWsdGH1HXlh3nc176vcLopEaJDiHsaQ1+0cbpOBbmKUE41wV8cquOACmD4dmjZlym8vZGK3s9iW2yxuX/NowSq5VnY0y1Y0zQQJD4T998IuMga03O5OycOcXDFzR/XzFFNfV/DygFdl7hRBqfPiHtRnHMbw3ekcbj55J1dGn7IS7vnyNVqPexsOPFBbeHTDDRz43R7KX10ONpkmnfz3Xqzq/efM4he3vFlj+Q/t1bamOpIdxnthl7ccQEp8tUVHF+/6VFDC60O0Lo5KFOmnTqcfyOT81m753K2WenffvJZni8fx7L/+wIGffcIHQ/7IGTf8h46Vv6b3E1oa3HvP6UpeTrTmmOxoFgvXb/OUX90rZRXVcZb/C/O+47bi5Z6Pt+u7XuXKDae6s07ZNpOJORVAGN8xr6ONujgqUaSfOh0K6RSOqMeBm6sMpSr6wEsom25tt/pqCTfOm8rJaxZQmn0ATxUO5rkeA9mZfUDcOXOiEc7tkV9rwtFqktFMXk6UvaZsj36ICMHX9w7wtK9T392qN+l9zKRJxWSFJXqZVFfhjwonnEIhXcVdCPE0cBbwo5TymNi2FsBUoAOwDrhASrldCCGAB4EBQBkwXEr5hVsDg4q7Uxy0eQVnNEuAIC6/SVg/UKeIGMeHyWefwZ13wttvsyPnQB7veTbPH3cWu5rk2l7PbqWlEwKYOKQ7sP/hlpcbZdeeyrgIGjeMIYxufXPKzGgViy7xXyYwVdgZEX5i9u1Q0TKKREhU3E8GdgHPG8T9n8A2KeV4IcQooLmU8m9CiAHAn9HEvRfwoJSyl1sDw7bc/QhgkEVHOoEtuk8/hbFj4b33oGVL+OtfOeaHIxxFPVEmDenuOJJxmgSFeMs9UUu2Li0mAmcjQkdZ2Ip0kJC4x07QAXjDIO6rgT5Syk1CiMOBD6WUnYQQj8dev2Tez+n8QcXdTmT8uB78rNI0Tlo6PUBsHxgffaSJ+pw5cMghfPn7Efyl+Ql8U4aruOpkCbAytp1cM1YuGV2MAFdXCUDvX7Tg/MJ2jvsm8qB0I50PBK8rcJPZf4XCimSsUD3UINg/AIfGXucDGwz7bYxtqyXuQogRwAiAdu3a+W6AsdCyLrRGX7vXyUWvk1XmB4mTEMdFN0ipifnYsfDxx3DYYTBxIq8ffxY3v7WW8jL388UhtUVJRvdSNEvQuFEWu/fVfqjlRCMIUTtCpbyiijEzV3j2w3/2zTbbMn86YU3omkl3bU+vcfxhRLXUtVGNInNJOFpGaqa/71lZKeVkKWWhlLKwVStvuVB0jFEyoAmjHlGhVwoyR6JEs4S2UtOAnxA6r/U/IfbAkBLeeQdOOglOOQXWroWHHoJvvoHrr2f8x985ni8iBFGLT6cabd5AX+mZlxMFQZyw673UI0lKLQqBA5SWV3jukxfXvJ5LPmzSXdvTHKETtAaqG5kc/aWoewQV980xdwyx/3+MbS8B2hr2axPbFipuP3arcLkJ5x/LhPOODRxC59Uqy2mUxYQDSuDEE6F/f1i/Hh5+GL7+Gv78Z8jJ8XS+KimpqHZ+X7fKzUUwzDqcqlA6CVw/dUlooYI6mZD+1lg56f4Ljk2oYpFdWGW6H2KK+kVQt8xMYBgwPvb/fw3b/ySEeBltQnWHm789CG4/di/pc/3imj9FSs7/YSm3LnqF5iuWQrt28NhjMHw4NGni/3wecFucpFt+VuGTySRMt0nx4hLXXPBh4dUlksgKZycXUyY8xBT1B1dxF0K8BPQBWgohNgKj0UR9mhDiCmA9cEFs9zfRImXWooVCXpaENjsm7UqWf9bW7yolp6+dz5/nvkTXzV+z7dB8xp9zI08fcRKtth/EyK9+8lwQIxmUV1QxZf53XNSrnWPelmRcd8zMFaHEgQfJBR/0Wl6/N0FXODtZ5yqHuyJMXN0yUsqhUsrDpZRRKWUbKeVTUsqtUspTpJRHSilPlVJui+0rpZTXSil/IaXsKqVMSpJ2p0K+yRraGl09AI2QnLF6Lm8+ex2TXx3HgXvL+OuA6zn+kod57Mh+7ItEbX2m5slgJ+yKR+vk5URr3QszUsJLCzY4CruTL9mOiEvjSssrEnLP2M1zRIQIPewwVS4RJ+tcFahWhEmdTD/gtATd7sdTUlqesC+4qCCfuTf3Yd1xu1k76xYeK76XJpX7uOHMGznlqseY3vVUKiPxgyGzQNhNBttRLal5oJgRwJhBXTi3R36tJftmqhxmRPPzcvh2/JlU+1gc1Tw3yv3nH2vbNh1z3/0s4bf7LKulDD2CxOl7E+b8gVNainSlVlDUT+ps4jC7YbGTLzshF01VFUybBuPGwVdfsa5VOx4Y+Ffe6HwS1VnOlrP+YLFbLGQM5zSjh3faVRgqKshn7OsrEnK3lO2rpHhxied5gJxohNEDu9R8BsWLS7h+6hLLfY3zIH7dZalwU7gVBgFCDbvs27lVXFpf43b9GkrMFWFQJy13J6yGtkZ8D7UrK+GFF6BLF/j979m5t4obzx5Fv+H/YubRfVyFHfZXDpLYx7NbWfDG8E6zRTdxSPea0nHbbUIdvbK9rIJbXl1O386tXF08AjiuXTPGzFxBh1Gz6DBqFmNfX0GuVdwm+4U4iNsj2W4K8yjKjjDdM3NWbfG1XaEISp213O0wRjIkVNihshKmTIG774Y1a6BbN5g+nQGrm7Fx515fbfJiVdslO3MrEReW6JRXVDFn1RbO7ZFfU+jDCqsiH9vLKohkCaJZIi5XjVGIg0SCJLvUnJ+1C2FFrKiIGEWqqHfiDvuFMFBhh4oKeP55uOcebcFRQQG89hrFbXsw4d01lOy0/xGaUwB4ydYI8Ra6nXDZhel5EYXmuVFP1n1JaXlcoQ8/VFVLDsqNktu4kaUQB3WxJNNN4UdQw3IFeb0PaqWqIlHqnVvGiK9h/b59MHkyuzv8Aq68kqVlEW6+9C6Kn5xJcfuejJzhPHzXXSVm14ndhGNEiLhJM8B2stFp5aKT6Ewa0p11489k8R2nu058ghaZk0hoZmlZRc1Cn7mj+sWJUSZGgtjdO6d88oni5T6olaqKMKjT+dy94GoB7d0LTz8N994LGzawtHUnJp54IR8eUQhCxH6IknKH5aJOGQG9pLgFLBOg6XnNnRJ1jezfiRumLrEcIRgTWbnlDjfnq7HDaTTiljgr06xRu8Rzyc4n73YfnOoUqMRkCiPJSBxWZ7Ad1u/ZA08+CePHQ0kJ/PrX3HDqtbzWqgsY4r3dLFm3HOTmOQBzQehbXl1OdjTLcrLRbdHR96XlFBXku0aq6DRptP86TRtHiEay2FFeQeu8HHbvraTUpkqSMW95386tmLpgQ6088NGIcLVuzT50Y7qIdJBsn77TdZ2uofzyijCo9+Kuo1tLW7eU8sf/vc8f5s0g56fNWmKv556Dfv0ovuVN3+c1WsZuk6FWFplTCgE3O1p3K+S7+HGtLNRqqcXIe6l7ev8Fx8aJUWH7FoyZuaLmYdA8N1oTGulEurM7WpGJoYdqpaoiDBqEuBcvLuHOlz/nnM9f5w8LXqXV7lLmt+9GxT3/4jdXnldjqfvN99I8N1pz/pHTl9a4NUpKyxk5Xat5ahSOMC0vo5/WrWi0XRji9VOXsHD9NsYVdbXte15OtJb4JWPpfSYIbKa4jepTEXBF+qj/4r5rFyW3juWdj6fRsmwHn7Y/lmsHj2JB22PI35rDXIMLxk++l2hEMHpgFwDGvr6ilr+6okpy/dQlTJi9ukYkwkgWBtqzyOjjN7sXmuVEEQJuiF3f6Zr6gho7QRkzqIvndrmJYxjuhmQJcLJGFUHamy53kaJ+UX8nVHfu1FLt3n8/bN3KRx2P46ETL2RRm6NrdrGqwmT8MTrdmUlDutf82Do4uDQgvupRWMnC1tlUj/JSdNmMXkIvEeH0UnrPraKR2/xFsgpVO7UtLWUYFQqPNKwJ1R074F//ggcegO3bYcAARrT/He8c1LHWrlY+TKPLwekHr7/vxerUXQ+6SBgFtGxfZaAVpsWLSywLcYP/yil6XHsi/mcvLhe3kZGbtZxMt04yJjEz3Q2lqN/UH3Hfvl2rdDRpEpSWwsCBcMcdUFjIgMUlfBLAh2nnqujbuZVv61gXCbOAOuVlcUIXQf11IqMBv9kgrfAijnq/b5q21HahlJP4JTOKJBmTmCrqRZFO6v4ipm3b4PbboUMHGDMG+vSBL76AmTOhUButBMm2Z5WWVz9uzqotvsU0SwjLRShFBfk1E7N+0EXQzxJ6O044onlCx4O9CEqIW5RVVJDvmn3STvycMiomSjIWWSWzvQqFG3Xbci8uhksugV274Lzz4Lbb4NhjLXf143KwKoZtTBFwQwBLu0pKbjBEpxgZPbBLIOs7LAtw3VZv59EfeCWl5bWKko/s34mRryytFf8Otd0tbhPLduKXzCiSZExiqqgXRTqp2+LevTucdRb8/e9wzDGhndbOV3rjtCWBXCg6Epgy7zsK27eIEw2zsHj1mesi6BaB0zw3ys7ySltXiJeHhNUDT7/2yOlLGdKzbe11+waM7pYOB9uLu5P4JTuKJOyYdxX1okgn9TdaJgBGy9QvXpOEgXsEhltUCXiPwNH3W7h+m2UecS/t8dKmLKEtjHJCABOHdLdNlxARotaCKYVCYU/DipbxiVHQ/Qi0kfzYsnyvNUrdLGWr4Xw0ImjauFFNugCzBahbh9nRLPZWVlMtNbE8t8d+a9SqfV7SBnhps5uwgzbScCqMkYwKS/WJTFlkpagbNGhxN7saggi7gDir14vAe0lzC/6H8xLYU1Fdc/0qKZmxqITC9i1sRbVp40aeBCLRBVi6u8VpvkJNNNqTiakbFJlNg3bLeHF/uGFMqmVc7v99aTm5jSPs3hfvLglzEcttxcs9PUzy83JsffluC7mM/XKb9DUX6zBnv3TKsa+7bMISqrCs3EyxllWmSIUVTm6ZhMRdCLEO+BmoAiqllIVCiBbAVKADsA64QEq53ek86Ur523HUrIRqj5qxEu5kicNtxcttfehmBM6Wt1F8nVZVAnEJw8zYuY6M9yAvN8quPZW1HgIXndCuVhRRUMJaGZpJK0ztvqtWD2dFwyHZPve+UsqfDH+PAt6XUo4XQoyK/f23EK7jG7ehrBdXgx8/vNUCnGRkHSxeXOJZ2IEaobWzvI2JzpxWVeoFOIoXl1guRKqokjRt0oglo0+Pa6sxRHJ7WQVZQktIZjd/kChhrQzNpBWmKlOkwi/JWMQ0GHgu9vo5oCgJ1/CEW1Fmt2LaYC3sTscEjT0vXlxiW4nJvJ9xdaobVkW2raioktw4bYmnurNOC5HM/R8zc0Wt2PdqCTtsrP8wCGtlaCatMM3ESlaKzCZRcZfAO0KIRUKIEbFth0opN8Ve/wAcmuA1AuP24ywqyOfcHvlO4dm10Fep2olkEEvKT1k1PytSI0LUyh7p5J+tlvah6uZ+eV19aefCkbF/ySghF9bK0ExaYRpklbWiYZOouP9GSnkc8DvgWiHEycY3pebQtzTxhBAjhBALhRALt2zZkmAzrPHy45yzaovv+PSignxG9u9ENCteCqNZ3sIKzbiNMIx4tRpzopFAMeMSbzVE3SxJfSTiBbu+BiUsKzfTrGX94WxVo1ahMJOQz11KWRL7/0chxGvA8cBmIcThUspNQojDgR9tjp0MTAZtQjWRdtjhZfm3nyF2rX3NKmhh9nqZUHUaYZiPz8uN2maRbGqIzimvqOKmaUst0x3k5URtLWrYH+Hi1GancM0gaYfDdHWEtTJUrTBV1GUCR8sIIZoCWVLKn2Ov3wXuBE4BthomVFtIKW92Olc6C2T7CYc0hp15CU3zGm1hd67muVH2VFTHL2bKEiCIKw7itgr1YlMkinmS06kPQQgSYqpC+hQK/yQrWuZQ4DWhZUxsBLwopXxbCPE5ME0IcQWwHrgggWskjFu0ipV1H8kSVJmEz6vFb9w+9vUVnqIt7EYYUtYu0K0LsjlxV1FBPjdNW2rZppfmb4gTd/3aVmGNXt0OTg9NJyu8uUUopJoYVCjCJ7C4Sym/AWqlYJRSbkWz3usEdkNvq21GQXYLTSteXGLrPjGLn10bnFZzmjNV6tvs9rXqtzkG3avbIWiIqW6dZ8rCIIWiPtOg0w/o2Fn3ToJjlUvGXJTaDrcKUDpuSczMowDdmjfjVIwjSBy+W/y321xHMmL/FQpFPHW/WEcaKF5cwoxFJXHCLiAuSZeTa8KrC8JLHL7xOkN7tbXcx257ULyEmKqwPYUivSjLPQBWlqtEC6vUsXNN5OVEPYuc0V1jZ8EbRwG6X/2l+RuokpKIEAzt1Ta0Zf3Ga7qtllTWuUKRXpS4B8DLZKqda2LMoC6+rmX0jXup6jOuqGvoYm7GqupS0Bh/hUKRHJS4B8Cr5QrhxUhnXMy1hxh/hUKRPhp0yt+gZFK2wHSg0s8qFJmBqsQUMhlnRaeYTEqopVAorFHiHpCGOGGox6fbjfVU+lmFInNQ4q7whFu+GLXKVKHILJS4KzzhlGo4v4G5pRSKuoASd4Un7Pzp5gLhCoUiM1ArVBWeyKTCFQqFwh0l7gpPZFrhCoVC4Yxyyyg80dDDPxWKuoYSd4VnGmL4p0JRV1FuGYVCoaiHKHFXKBSKeogSd4VCoaiHKHFXKBSKeogSd4VCoaiHZETKXyHEFmC9YVNL4Kc0NSeVNJR+QsPpq+pn/SOT+9peStnK6o2MEHczQoiFdjmK6xMNpZ/QcPqq+ln/qKt9VW4ZhUKhqIcocVcoFIp6SKaK++R0NyBFNJR+QsPpq+pn/aNO9jUjfe4KhUKhSIxMtdwVCoVCkQBK3BUKhaIekjZxF0K0EEK8K4RYE/u/uc1+bwshSoUQb5i2dxRCzBdCrBVCTBVCNE5Ny/3ho5/DYvusEUIMM2z/UAixWgixJPbvkNS13h0hxBmx9q0VQoyyeL9J7PNZG/u8OhjeuyW2fbUQon9KGx6AoH0VQnQQQpQbPsPHUt54H3jo58lCiC+EEJVCiPNM71l+jzORBPtZZfg8Z6au1T6QUqblH/BPYFTs9SjgHzb7nQIMBN4wbZ8GXBh7/Rhwdbr6kmg/gRbAN7H/m8deN4+99yFQmO5+2PQtAnwNHAE0BpYCR5v2uQZ4LPb6QmBq7PXRsf2bAB1j54mku09J6msH4Mt09yHEfnYAugHPA+cZttt+jzPtXyL9jL23K919cPuXTrfMYOC52OvngCKrnaSU7wM/G7cJIQTQD5judnwG4KWf/YF3pZTbpJTbgXeBM1LTvIQ4HlgrpfxGSrkPeBmtv0aM/Z8OnBL7/AYDL0sp90opvwXWxs6XqSTS17qEaz+llOuklMuAatOxdel7nEg/6wTpFPdDpZSbYq9/AA71cezBQKmUsjL290YgU6tIeOlnPrDB8Le5P8/Ehn+3Z5hYuLU7bp/Y57UD7fPzcmwmkUhfAToKIRYLIT4SQpyU7MYmQCKfS136TBNta7YQYqEQYp4QoijUloVEUisxCSHeAw6zeOvvxj+klFIIUWdjMpPcz4uklCVCiAOBGcAlaMNERd1hE9BOSrlVCNEDKBZCdJFS7kx3wxSBaR/7XR4BfCCEWC6l/DrdjTKSVHGXUp5q954QYrMQ4nAp5SYhxOHAjz5OvRXIE0I0illIbYCSBJsbmBD6WQL0MfzdBs3XjpSyJPb/z0KIF9GGk5ki7iVAW8PfVp+Dvs9GIUQjoBna5+fl2EwicF+l5qTdCyClXCSE+Br4FbAw6a32TyKfi+33OANJ6Ptn+F1+I4T4EChA8+FnDOl0y8wE9Nn0YcB/vR4Y+7HMAfQZbF/Hpxgv/ZwNnC6EaB6LpjkdmC2EaCSEaAkghIgCZwFfpqDNXvkcODIWudQYbRLRHDlg7P95wAexz28mcGEswqQjcCSwIEXtDkLgvgohWgkhIgAxS+9ItMnGTMRLP+2w/B4nqZ2JErifsf41ib1uCfQGvkpaS4OSxtnqg4H3gTXAe0CL2PZC4EnDfp8AW4ByNL9Y/9j2I9DEYC3wCtAk3bPTCfbz8lhf1gKXxbY1BRYBy4AVwINkWEQJMAD4H5rV8vfYtjuBQbHX2bHPZ23s8zrCcOzfY8etBn6X7r4kq6/AubHPbwnwBTAw3X1JsJ89Y7/F3WijsBVO3+NM/Re0n8CJwHK0CJvlwBXp7ovVP5V+QKFQKOohaoWqQqFQ1EOUuCsUCkU9RIm7QqFQ1EOUuCsUCkU9RIm7QqFQ1EOUuCsUCkU9RIm7QqFQ1EP+H3dCV00ywQvdAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "n_epoch = 500 # epoch size\n", "a, b = 1, 1 # initial parameters\n", "epsilon = 0.01 # learning rate\n", "\n", "for i in range(n_epoch):\n", " for j in range(N):\n", " a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n", " b = b + epsilon*2*(Y[j] - a*X[j] - b)\n", "\n", " L = 0\n", " for j in range(N):\n", " L = L + (Y[j]-a*X[j]-b)**2\n", " print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n", " \n", "x_min = np.min(X)\n", "x_max = np.max(X)\n", "y_min = a * x_min + b\n", "y_max = a * x_max + b\n", "\n", "plt.scatter(X, Y, label='original data')\n", "plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. 如何可视化迭代过程" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "application/javascript": [ "/* Put everything inside the global mpl namespace */\n", "/* global mpl */\n", "window.mpl = {};\n", "\n", "mpl.get_websocket_type = function () {\n", " if (typeof WebSocket !== 'undefined') {\n", " return WebSocket;\n", " } else if (typeof MozWebSocket !== 'undefined') {\n", " return MozWebSocket;\n", " } else {\n", " alert(\n", " 'Your browser does not have WebSocket support. ' +\n", " 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n", " 'Firefox 4 and 5 are also supported but you ' +\n", " 'have to enable WebSockets in about:config.'\n", " );\n", " }\n", "};\n", "\n", "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n", " this.id = figure_id;\n", "\n", " this.ws = websocket;\n", "\n", " this.supports_binary = this.ws.binaryType !== undefined;\n", "\n", " if (!this.supports_binary) {\n", " var warnings = document.getElementById('mpl-warnings');\n", " if (warnings) {\n", " warnings.style.display = 'block';\n", " warnings.textContent =\n", " 'This browser does not support binary websocket messages. ' +\n", " 'Performance may be slow.';\n", " }\n", " }\n", "\n", " this.imageObj = new Image();\n", "\n", " this.context = undefined;\n", " this.message = undefined;\n", " this.canvas = undefined;\n", " this.rubberband_canvas = undefined;\n", " this.rubberband_context = undefined;\n", " this.format_dropdown = undefined;\n", "\n", " this.image_mode = 'full';\n", "\n", " this.root = document.createElement('div');\n", " this.root.setAttribute('style', 'display: inline-block');\n", " this._root_extra_style(this.root);\n", "\n", " parent_element.appendChild(this.root);\n", "\n", " this._init_header(this);\n", " this._init_canvas(this);\n", " this._init_toolbar(this);\n", "\n", " var fig = this;\n", "\n", " this.waiting = false;\n", "\n", " this.ws.onopen = function () {\n", " fig.send_message('supports_binary', { value: fig.supports_binary });\n", " fig.send_message('send_image_mode', {});\n", " if (mpl.ratio !== 1) {\n", " fig.send_message('set_dpi_ratio', { dpi_ratio: mpl.ratio });\n", " }\n", " fig.send_message('refresh', {});\n", " };\n", "\n", " this.imageObj.onload = function () {\n", " if (fig.image_mode === 'full') {\n", " // Full images could contain transparency (where diff images\n", " // almost always do), so we need to clear the canvas so that\n", " // there is no ghosting.\n", " fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n", " }\n", " fig.context.drawImage(fig.imageObj, 0, 0);\n", " };\n", "\n", " this.imageObj.onunload = function () {\n", " fig.ws.close();\n", " };\n", "\n", " this.ws.onmessage = this._make_on_message_function(this);\n", "\n", " this.ondownload = ondownload;\n", "};\n", "\n", "mpl.figure.prototype._init_header = function () {\n", " var titlebar = document.createElement('div');\n", " titlebar.classList =\n", " 'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n", " var titletext = document.createElement('div');\n", " titletext.classList = 'ui-dialog-title';\n", " titletext.setAttribute(\n", " 'style',\n", " 'width: 100%; text-align: center; padding: 3px;'\n", " );\n", " titlebar.appendChild(titletext);\n", " this.root.appendChild(titlebar);\n", " this.header = titletext;\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n", "\n", "mpl.figure.prototype._init_canvas = function () {\n", " var fig = this;\n", "\n", " var canvas_div = (this.canvas_div = document.createElement('div'));\n", " canvas_div.setAttribute(\n", " 'style',\n", " 'border: 1px solid #ddd;' +\n", " 'box-sizing: content-box;' +\n", " 'clear: both;' +\n", " 'min-height: 1px;' +\n", " 'min-width: 1px;' +\n", " 'outline: 0;' +\n", " 'overflow: hidden;' +\n", " 'position: relative;' +\n", " 'resize: both;'\n", " );\n", "\n", " function on_keyboard_event_closure(name) {\n", " return function (event) {\n", " return fig.key_event(event, name);\n", " };\n", " }\n", "\n", " canvas_div.addEventListener(\n", " 'keydown',\n", " on_keyboard_event_closure('key_press')\n", " );\n", " canvas_div.addEventListener(\n", " 'keyup',\n", " on_keyboard_event_closure('key_release')\n", " );\n", "\n", " this._canvas_extra_style(canvas_div);\n", " this.root.appendChild(canvas_div);\n", "\n", " var canvas = (this.canvas = document.createElement('canvas'));\n", " canvas.classList.add('mpl-canvas');\n", " canvas.setAttribute('style', 'box-sizing: content-box;');\n", "\n", " this.context = canvas.getContext('2d');\n", "\n", " var backingStore =\n", " this.context.backingStorePixelRatio ||\n", " this.context.webkitBackingStorePixelRatio ||\n", " this.context.mozBackingStorePixelRatio ||\n", " this.context.msBackingStorePixelRatio ||\n", " this.context.oBackingStorePixelRatio ||\n", " this.context.backingStorePixelRatio ||\n", " 1;\n", "\n", " mpl.ratio = (window.devicePixelRatio || 1) / backingStore;\n", "\n", " var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n", " 'canvas'\n", " ));\n", " rubberband_canvas.setAttribute(\n", " 'style',\n", " 'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n", " );\n", "\n", " var resizeObserver = new ResizeObserver(function (entries) {\n", " var nentries = entries.length;\n", " for (var i = 0; i < nentries; i++) {\n", " var entry = entries[i];\n", " var width, height;\n", " if (entry.contentBoxSize) {\n", " width = entry.contentBoxSize.inlineSize;\n", " height = entry.contentBoxSize.blockSize;\n", " } else {\n", " width = entry.contentRect.width;\n", " height = entry.contentRect.height;\n", " }\n", "\n", " // Keep the size of the canvas and rubber band canvas in sync with\n", " // the canvas container.\n", " canvas.setAttribute('width', width * mpl.ratio);\n", " canvas.setAttribute('height', height * mpl.ratio);\n", " canvas.setAttribute(\n", " 'style',\n", " 'width: ' + width + 'px; height: ' + height + 'px;'\n", " );\n", "\n", " rubberband_canvas.setAttribute('width', width);\n", " rubberband_canvas.setAttribute('height', height);\n", "\n", " // And update the size in Python. We ignore the initial 0/0 size\n", " // that occurs as the element is placed into the DOM, which should\n", " // otherwise not happen due to the minimum size styling.\n", " if (width != 0 && height != 0) {\n", " fig.request_resize(width, height);\n", " }\n", " }\n", " });\n", " resizeObserver.observe(canvas_div);\n", "\n", " function on_mouse_event_closure(name) {\n", " return function (event) {\n", " return fig.mouse_event(event, name);\n", " };\n", " }\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mousedown',\n", " on_mouse_event_closure('button_press')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseup',\n", " on_mouse_event_closure('button_release')\n", " );\n", " // Throttle sequential mouse events to 1 every 20ms.\n", " rubberband_canvas.addEventListener(\n", " 'mousemove',\n", " on_mouse_event_closure('motion_notify')\n", " );\n", "\n", " rubberband_canvas.addEventListener(\n", " 'mouseenter',\n", " on_mouse_event_closure('figure_enter')\n", " );\n", " rubberband_canvas.addEventListener(\n", " 'mouseleave',\n", " on_mouse_event_closure('figure_leave')\n", " );\n", "\n", " canvas_div.addEventListener('wheel', function (event) {\n", " if (event.deltaY < 0) {\n", " event.step = 1;\n", " } else {\n", " event.step = -1;\n", " }\n", " on_mouse_event_closure('scroll')(event);\n", " });\n", "\n", " canvas_div.appendChild(canvas);\n", " canvas_div.appendChild(rubberband_canvas);\n", "\n", " this.rubberband_context = rubberband_canvas.getContext('2d');\n", " this.rubberband_context.strokeStyle = '#000000';\n", "\n", " this._resize_canvas = function (width, height, forward) {\n", " if (forward) {\n", " canvas_div.style.width = width + 'px';\n", " canvas_div.style.height = height + 'px';\n", " }\n", " };\n", "\n", " // Disable right mouse context menu.\n", " this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n", " return false;\n", " });\n", "\n", " function set_focus() {\n", " canvas.focus();\n", " canvas_div.focus();\n", " }\n", "\n", " window.setTimeout(set_focus, 100);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'mpl-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'mpl-button-group';\n", " continue;\n", " }\n", "\n", " var button = (fig.buttons[name] = document.createElement('button'));\n", " button.classList = 'mpl-widget';\n", " button.setAttribute('role', 'button');\n", " button.setAttribute('aria-disabled', 'false');\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", "\n", " var icon_img = document.createElement('img');\n", " icon_img.src = '_images/' + image + '.png';\n", " icon_img.srcset = '_images/' + image + '_large.png 2x';\n", " icon_img.alt = tooltip;\n", " button.appendChild(icon_img);\n", "\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " var fmt_picker = document.createElement('select');\n", " fmt_picker.classList = 'mpl-widget';\n", " toolbar.appendChild(fmt_picker);\n", " this.format_dropdown = fmt_picker;\n", "\n", " for (var ind in mpl.extensions) {\n", " var fmt = mpl.extensions[ind];\n", " var option = document.createElement('option');\n", " option.selected = fmt === mpl.default_extension;\n", " option.innerHTML = fmt;\n", " fmt_picker.appendChild(option);\n", " }\n", "\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "};\n", "\n", "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n", " // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n", " // which will in turn request a refresh of the image.\n", " this.send_message('resize', { width: x_pixels, height: y_pixels });\n", "};\n", "\n", "mpl.figure.prototype.send_message = function (type, properties) {\n", " properties['type'] = type;\n", " properties['figure_id'] = this.id;\n", " this.ws.send(JSON.stringify(properties));\n", "};\n", "\n", "mpl.figure.prototype.send_draw_message = function () {\n", " if (!this.waiting) {\n", " this.waiting = true;\n", " this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " var format_dropdown = fig.format_dropdown;\n", " var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n", " fig.ondownload(fig, format);\n", "};\n", "\n", "mpl.figure.prototype.handle_resize = function (fig, msg) {\n", " var size = msg['size'];\n", " if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n", " fig._resize_canvas(size[0], size[1], msg['forward']);\n", " fig.send_message('refresh', {});\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n", " var x0 = msg['x0'] / mpl.ratio;\n", " var y0 = (fig.canvas.height - msg['y0']) / mpl.ratio;\n", " var x1 = msg['x1'] / mpl.ratio;\n", " var y1 = (fig.canvas.height - msg['y1']) / mpl.ratio;\n", " x0 = Math.floor(x0) + 0.5;\n", " y0 = Math.floor(y0) + 0.5;\n", " x1 = Math.floor(x1) + 0.5;\n", " y1 = Math.floor(y1) + 0.5;\n", " var min_x = Math.min(x0, x1);\n", " var min_y = Math.min(y0, y1);\n", " var width = Math.abs(x1 - x0);\n", " var height = Math.abs(y1 - y0);\n", "\n", " fig.rubberband_context.clearRect(\n", " 0,\n", " 0,\n", " fig.canvas.width / mpl.ratio,\n", " fig.canvas.height / mpl.ratio\n", " );\n", "\n", " fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n", "};\n", "\n", "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n", " // Updates the figure title.\n", " fig.header.textContent = msg['label'];\n", "};\n", "\n", "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n", " var cursor = msg['cursor'];\n", " switch (cursor) {\n", " case 0:\n", " cursor = 'pointer';\n", " break;\n", " case 1:\n", " cursor = 'default';\n", " break;\n", " case 2:\n", " cursor = 'crosshair';\n", " break;\n", " case 3:\n", " cursor = 'move';\n", " break;\n", " }\n", " fig.rubberband_canvas.style.cursor = cursor;\n", "};\n", "\n", "mpl.figure.prototype.handle_message = function (fig, msg) {\n", " fig.message.textContent = msg['message'];\n", "};\n", "\n", "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n", " // Request the server to send over a new figure.\n", " fig.send_draw_message();\n", "};\n", "\n", "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n", " fig.image_mode = msg['mode'];\n", "};\n", "\n", "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n", " for (var key in msg) {\n", " if (!(key in fig.buttons)) {\n", " continue;\n", " }\n", " fig.buttons[key].disabled = !msg[key];\n", " fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n", " if (msg['mode'] === 'PAN') {\n", " fig.buttons['Pan'].classList.add('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " } else if (msg['mode'] === 'ZOOM') {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.add('active');\n", " } else {\n", " fig.buttons['Pan'].classList.remove('active');\n", " fig.buttons['Zoom'].classList.remove('active');\n", " }\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Called whenever the canvas gets updated.\n", " this.send_message('ack', {});\n", "};\n", "\n", "// A function to construct a web socket function for onmessage handling.\n", "// Called in the figure constructor.\n", "mpl.figure.prototype._make_on_message_function = function (fig) {\n", " return function socket_on_message(evt) {\n", " if (evt.data instanceof Blob) {\n", " /* FIXME: We get \"Resource interpreted as Image but\n", " * transferred with MIME type text/plain:\" errors on\n", " * Chrome. But how to set the MIME type? It doesn't seem\n", " * to be part of the websocket stream */\n", " evt.data.type = 'image/png';\n", "\n", " /* Free the memory for the previous frames */\n", " if (fig.imageObj.src) {\n", " (window.URL || window.webkitURL).revokeObjectURL(\n", " fig.imageObj.src\n", " );\n", " }\n", "\n", " fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n", " evt.data\n", " );\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " } else if (\n", " typeof evt.data === 'string' &&\n", " evt.data.slice(0, 21) === 'data:image/png;base64'\n", " ) {\n", " fig.imageObj.src = evt.data;\n", " fig.updated_canvas_event();\n", " fig.waiting = false;\n", " return;\n", " }\n", "\n", " var msg = JSON.parse(evt.data);\n", " var msg_type = msg['type'];\n", "\n", " // Call the \"handle_{type}\" callback, which takes\n", " // the figure and JSON message as its only arguments.\n", " try {\n", " var callback = fig['handle_' + msg_type];\n", " } catch (e) {\n", " console.log(\n", " \"No handler for the '\" + msg_type + \"' message type: \",\n", " msg\n", " );\n", " return;\n", " }\n", "\n", " if (callback) {\n", " try {\n", " // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n", " callback(fig, msg);\n", " } catch (e) {\n", " console.log(\n", " \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n", " e,\n", " e.stack,\n", " msg\n", " );\n", " }\n", " }\n", " };\n", "};\n", "\n", "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n", "mpl.findpos = function (e) {\n", " //this section is from http://www.quirksmode.org/js/events_properties.html\n", " var targ;\n", " if (!e) {\n", " e = window.event;\n", " }\n", " if (e.target) {\n", " targ = e.target;\n", " } else if (e.srcElement) {\n", " targ = e.srcElement;\n", " }\n", " if (targ.nodeType === 3) {\n", " // defeat Safari bug\n", " targ = targ.parentNode;\n", " }\n", "\n", " // pageX,Y are the mouse positions relative to the document\n", " var boundingRect = targ.getBoundingClientRect();\n", " var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n", " var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n", "\n", " return { x: x, y: y };\n", "};\n", "\n", "/*\n", " * return a copy of an object with only non-object keys\n", " * we need this to avoid circular references\n", " * http://stackoverflow.com/a/24161582/3208463\n", " */\n", "function simpleKeys(original) {\n", " return Object.keys(original).reduce(function (obj, key) {\n", " if (typeof original[key] !== 'object') {\n", " obj[key] = original[key];\n", " }\n", " return obj;\n", " }, {});\n", "}\n", "\n", "mpl.figure.prototype.mouse_event = function (event, name) {\n", " var canvas_pos = mpl.findpos(event);\n", "\n", " if (name === 'button_press') {\n", " this.canvas.focus();\n", " this.canvas_div.focus();\n", " }\n", "\n", " var x = canvas_pos.x * mpl.ratio;\n", " var y = canvas_pos.y * mpl.ratio;\n", "\n", " this.send_message(name, {\n", " x: x,\n", " y: y,\n", " button: event.button,\n", " step: event.step,\n", " guiEvent: simpleKeys(event),\n", " });\n", "\n", " /* This prevents the web browser from automatically changing to\n", " * the text insertion cursor when the button is pressed. We want\n", " * to control all of the cursor setting manually through the\n", " * 'cursor' event from matplotlib */\n", " event.preventDefault();\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n", " // Handle any extra behaviour associated with a key event\n", "};\n", "\n", "mpl.figure.prototype.key_event = function (event, name) {\n", " // Prevent repeat events\n", " if (name === 'key_press') {\n", " if (event.which === this._key) {\n", " return;\n", " } else {\n", " this._key = event.which;\n", " }\n", " }\n", " if (name === 'key_release') {\n", " this._key = null;\n", " }\n", "\n", " var value = '';\n", " if (event.ctrlKey && event.which !== 17) {\n", " value += 'ctrl+';\n", " }\n", " if (event.altKey && event.which !== 18) {\n", " value += 'alt+';\n", " }\n", " if (event.shiftKey && event.which !== 16) {\n", " value += 'shift+';\n", " }\n", "\n", " value += 'k';\n", " value += event.which.toString();\n", "\n", " this._key_event_extra(event, name);\n", "\n", " this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n", " return false;\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n", " if (name === 'download') {\n", " this.handle_save(this, null);\n", " } else {\n", " this.send_message('toolbar_button', { name: name });\n", " }\n", "};\n", "\n", "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n", " this.message.textContent = tooltip;\n", "};\n", "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n", "\n", "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n", "\n", "mpl.default_extension = \"png\";/* global mpl */\n", "\n", "var comm_websocket_adapter = function (comm) {\n", " // Create a \"websocket\"-like object which calls the given IPython comm\n", " // object with the appropriate methods. Currently this is a non binary\n", " // socket, so there is still some room for performance tuning.\n", " var ws = {};\n", "\n", " ws.close = function () {\n", " comm.close();\n", " };\n", " ws.send = function (m) {\n", " //console.log('sending', m);\n", " comm.send(m);\n", " };\n", " // Register the callback with on_msg.\n", " comm.on_msg(function (msg) {\n", " //console.log('receiving', msg['content']['data'], msg);\n", " // Pass the mpl event to the overridden (by mpl) onmessage function.\n", " ws.onmessage(msg['content']['data']);\n", " });\n", " return ws;\n", "};\n", "\n", "mpl.mpl_figure_comm = function (comm, msg) {\n", " // This is the function which gets called when the mpl process\n", " // starts-up an IPython Comm through the \"matplotlib\" channel.\n", "\n", " var id = msg.content.data.id;\n", " // Get hold of the div created by the display call when the Comm\n", " // socket was opened in Python.\n", " var element = document.getElementById(id);\n", " var ws_proxy = comm_websocket_adapter(comm);\n", "\n", " function ondownload(figure, _format) {\n", " window.open(figure.canvas.toDataURL());\n", " }\n", "\n", " var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n", "\n", " // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n", " // web socket which is closed, not our websocket->open comm proxy.\n", " ws_proxy.onopen();\n", "\n", " fig.parent_element = element;\n", " fig.cell_info = mpl.find_output_cell(\"
\");\n", " if (!fig.cell_info) {\n", " console.error('Failed to find cell for figure', id, fig);\n", " return;\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_close = function (fig, msg) {\n", " var width = fig.canvas.width / mpl.ratio;\n", " fig.root.removeEventListener('remove', this._remove_fig_handler);\n", "\n", " // Update the output cell to use the data from the current canvas.\n", " fig.push_to_output();\n", " var dataURL = fig.canvas.toDataURL();\n", " // Re-enable the keyboard manager in IPython - without this line, in FF,\n", " // the notebook keyboard shortcuts fail.\n", " IPython.keyboard_manager.enable();\n", " fig.parent_element.innerHTML =\n", " '';\n", " fig.close_ws(fig, msg);\n", "};\n", "\n", "mpl.figure.prototype.close_ws = function (fig, msg) {\n", " fig.send_message('closing', msg);\n", " // fig.ws.close()\n", "};\n", "\n", "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n", " // Turn the data on the canvas into data in the output cell.\n", " var width = this.canvas.width / mpl.ratio;\n", " var dataURL = this.canvas.toDataURL();\n", " this.cell_info[1]['text/html'] =\n", " '';\n", "};\n", "\n", "mpl.figure.prototype.updated_canvas_event = function () {\n", " // Tell IPython that the notebook contents must change.\n", " IPython.notebook.set_dirty(true);\n", " this.send_message('ack', {});\n", " var fig = this;\n", " // Wait a second, then push the new image to the DOM so\n", " // that it is saved nicely (might be nice to debounce this).\n", " setTimeout(function () {\n", " fig.push_to_output();\n", " }, 1000);\n", "};\n", "\n", "mpl.figure.prototype._init_toolbar = function () {\n", " var fig = this;\n", "\n", " var toolbar = document.createElement('div');\n", " toolbar.classList = 'btn-toolbar';\n", " this.root.appendChild(toolbar);\n", "\n", " function on_click_closure(name) {\n", " return function (_event) {\n", " return fig.toolbar_button_onclick(name);\n", " };\n", " }\n", "\n", " function on_mouseover_closure(tooltip) {\n", " return function (event) {\n", " if (!event.currentTarget.disabled) {\n", " return fig.toolbar_button_onmouseover(tooltip);\n", " }\n", " };\n", " }\n", "\n", " fig.buttons = {};\n", " var buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " var button;\n", " for (var toolbar_ind in mpl.toolbar_items) {\n", " var name = mpl.toolbar_items[toolbar_ind][0];\n", " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", " var image = mpl.toolbar_items[toolbar_ind][2];\n", " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", "\n", " if (!name) {\n", " /* Instead of a spacer, we start a new button group. */\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", " buttonGroup = document.createElement('div');\n", " buttonGroup.classList = 'btn-group';\n", " continue;\n", " }\n", "\n", " button = fig.buttons[name] = document.createElement('button');\n", " button.classList = 'btn btn-default';\n", " button.href = '#';\n", " button.title = name;\n", " button.innerHTML = '';\n", " button.addEventListener('click', on_click_closure(method_name));\n", " button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n", " buttonGroup.appendChild(button);\n", " }\n", "\n", " if (buttonGroup.hasChildNodes()) {\n", " toolbar.appendChild(buttonGroup);\n", " }\n", "\n", " // Add the status bar.\n", " var status_bar = document.createElement('span');\n", " status_bar.classList = 'mpl-message pull-right';\n", " toolbar.appendChild(status_bar);\n", " this.message = status_bar;\n", "\n", " // Add the close button to the window.\n", " var buttongrp = document.createElement('div');\n", " buttongrp.classList = 'btn-group inline pull-right';\n", " button = document.createElement('button');\n", " button.classList = 'btn btn-mini btn-primary';\n", " button.href = '#';\n", " button.title = 'Stop Interaction';\n", " button.innerHTML = '';\n", " button.addEventListener('click', function (_evt) {\n", " fig.handle_close(fig, {});\n", " });\n", " button.addEventListener(\n", " 'mouseover',\n", " on_mouseover_closure('Stop Interaction')\n", " );\n", " buttongrp.appendChild(button);\n", " var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n", " titlebar.insertBefore(buttongrp, titlebar.firstChild);\n", "};\n", "\n", "mpl.figure.prototype._remove_fig_handler = function () {\n", " this.close_ws(this, {});\n", "};\n", "\n", "mpl.figure.prototype._root_extra_style = function (el) {\n", " el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n", " el.addEventListener('remove', this._remove_fig_handler);\n", "};\n", "\n", "mpl.figure.prototype._canvas_extra_style = function (el) {\n", " // this is important to make the div 'focusable\n", " el.setAttribute('tabindex', 0);\n", " // reach out to IPython and tell the keyboard manager to turn it's self\n", " // off when our div gets focus\n", "\n", " // location in version 3\n", " if (IPython.notebook.keyboard_manager) {\n", " IPython.notebook.keyboard_manager.register_events(el);\n", " } else {\n", " // location in version 2\n", " IPython.keyboard_manager.register_events(el);\n", " }\n", "};\n", "\n", "mpl.figure.prototype._key_event_extra = function (event, _name) {\n", " var manager = IPython.notebook.keyboard_manager;\n", " if (!manager) {\n", " manager = IPython.keyboard_manager;\n", " }\n", "\n", " // Check for shift+enter\n", " if (event.shiftKey && event.which === 13) {\n", " this.canvas_div.blur();\n", " // select the cell after this one\n", " var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n", " IPython.notebook.select(index + 1);\n", " }\n", "};\n", "\n", "mpl.figure.prototype.handle_save = function (fig, _msg) {\n", " fig.ondownload(fig, null);\n", "};\n", "\n", "mpl.find_output_cell = function (html_output) {\n", " // Return the cell and output element which can be found *uniquely* in the notebook.\n", " // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n", " // IPython event is triggered only after the cells have been serialised, which for\n", " // our purposes (turning an active figure into a static one), is too late.\n", " var cells = IPython.notebook.get_cells();\n", " var ncells = cells.length;\n", " for (var i = 0; i < ncells; i++) {\n", " var cell = cells[i];\n", " if (cell.cell_type === 'code') {\n", " for (var j = 0; j < cell.output_area.outputs.length; j++) {\n", " var data = cell.output_area.outputs[j];\n", " if (data.data) {\n", " // IPython >= 3 moved mimebundle to data attribute of output\n", " data = data.data;\n", " }\n", " if (data['text/html'] === html_output) {\n", " return [cell, data, j];\n", " }\n", " }\n", " }\n", " }\n", "};\n", "\n", "// Register the function which deals with the matplotlib target/channel.\n", "// The kernel may be null if the page has been refreshed.\n", "if (IPython.notebook.kernel !== null) {\n", " IPython.notebook.kernel.comm_manager.register_target(\n", " 'matplotlib',\n", " mpl.mpl_figure_comm\n", " );\n", "}\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%matplotlib nbagg\n", "\n", "import matplotlib.pyplot as plt\n", "import matplotlib.animation as animation\n", "\n", "n_epoch = 3000 # epoch size\n", "a, b = 1, 1 # initial parameters\n", "epsilon = 0.001 # learning rate\n", "\n", "fig = plt.figure()\n", "imgs = []\n", "\n", "for i in range(n_epoch):\n", " for j in range(N):\n", " a = a + epsilon*2*(Y[j] - a*X[j] - b)*X[j]\n", " b = b + epsilon*2*(Y[j] - a*X[j] - b)\n", "\n", " L = 0\n", " for j in range(N):\n", " L = L + (Y[j]-a*X[j]-b)**2\n", " #print(\"epoch %4d: loss = %f, a = %f, b = %f\" % (i, L, a, b))\n", " \n", " if i % 50 == 0:\n", " x_min = np.min(X)\n", " x_max = np.max(X)\n", " y_min = a * x_min + b\n", " y_max = a * x_max + b\n", "\n", " img = plt.scatter(X, Y, label='original data')\n", " img = plt.plot([x_min, x_max], [y_min, y_max], 'r', label='model')\n", " imgs.append(img)\n", " \n", "ani = animation.ArtistAnimation(fig, imgs)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. 如何使用批次更新的方法?\n", "\n", "如果有一些数据包含比较大的错误(异常数据),因此每次更新仅仅使用一个数据会导致不精确,同时每次仅仅使用一个数据来计算更新也导致计算效率比较低。\n", "\n", "\n", "* [梯度下降方法的几种形式](https://blog.csdn.net/u010402786/article/details/51188876)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. 如何拟合多项式函数?\n", "\n", "需要设计一个弹道导弹防御系统,通过观测导弹的飞行路径,预测未来导弹的飞行轨迹,从而完成摧毁的任务。按照物理学,可以得知模型为:\n", "$$\n", "y = at^2 + bt + c\n", "$$\n", "我们需要求解三个模型参数$a, b, c$。\n", "\n", "损失函数的定义为:\n", "$$\n", "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n", "$$\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVM0lEQVR4nO3dbYxcZ3mH8evGzssGqiwhVkrWTu2qVlDAgOkoCnVV0ZgqDiDsuhSCSjGUykINLQQEdeBD6SenCgJSQdNaARpalJeGNLEoJaQxqAWJlDVOgSSkWOHFXhKyvDggcEkc7n6Ys/HamRl7ds7MmTnn+kmRZ845O/PM7uY/Z+/nfs5EZiJJapanVT0ASdLoGf6S1ECGvyQ1kOEvSQ1k+EtSAy2vegAn4+yzz87Vq1dXPQxJmih79+79QWau6LRvIsJ/9erVzM7OVj0MSZooEfGdbvss+0hSAxn+ktRAhr8kNZDhL0kNZPhLUgNNRLeP+nPbvjmuvuMBvnfoMOdOT/HOS85ny/qZvrdLqq+YhKt6tlqtbGqrZ69g7rQP4Mpbv8bhx5948jGmTlnGH/zmDJ/cO3fS23duXQfQ13P7hiGNl4jYm5mtjvsM//HQT5AvBHOnfaef8jR+/PPHn/L4yyJ4osPPutv26alT+MWRX/b13Cd6w5A0Wob/mLtt31xfQT4zPQXA3KHDIxvjyTx3rzcM3wCk0esV/tb8R6zTGf7VdzxwTGACHH78iadsW/C9JYR+v2f+3fR67kOHn/pGdfjxJ7j6jgcA/yKQxoln/kPSTxmnW8h3s5Sz735r/mX/1XH867RMJA2fZ/4jdnwZZ+7QYa689WucfsrTOp7h91t37/VG8t5XPhfoHKitXzvrpLd3e/xe+3rNN3R63e/dfe8xr2/h+wT4BiANmeE/oDLKOE9kdjwz7hXkC7rt6xSeW9bP9LW93+eG/v6ysUwkVceyzwC6TdQupYyz8KYx6WHX7c2wrDLRJH5PpKpY9hlQt572bmf4Synj9Dr7niTdXkcZZaKr73jARWlSSQz/E+hWv4funS+DlHHqaOH1DVom+t6hwz1/HnX/PkplsuyzSD8li15dL3Uq4wyb33NpeFzkdRL6rd8H8IHXvLDrSldDZ+m6/Sx2bl3HFTfdQ7ffWOcJpGP1Cn+v6lnoVb/v5NzpKbasn2Hn1nXMTE8RtM8+DZvB9fq+nluc/R+v1zyBpKdqZM2/U6mh3/r9Qs26LhO146bb9/Wdl5y/pHkCy0HSsRp35r9QUpg7dJjk6ITh9BmndDx+4azTs/vx0O2vgpkufxGcOXVKx5/3bfvmRjpuadw07sy/W3nntOVP63qG79n9eOmnnTSCnm2jUlOVcuYfEVdExL0R8fWIuCEiTo+INRFxd0Tsj4ibIuLU4tjTivv7i/2ryxhDJ7ftm2PDVXtYs+Pf2HDVHm7bN9e1vPPo4cc9w59g3f4iONRhHQEcLQcd//shNcXA3T4RMQN8AbggMw9HxM3Ap4GXAbdm5o0R8ffA/2TmtRHxZ8DzM/PNEXEZ8PuZ+Zpez7GUbp+lXCb5izsu7us5NP42XLXHy0+rsUbR7bMcmIqI5cAZwEPAxcAtxf7rgS3F7c3FfYr9GyO6tNQMoFt5J7P9P/liiydwVS/vvOT8jj/vXuUgqQkGDv/MnAPeB3yXdug/CuwFDmXmkeKwg8DC6dQMcKD42iPF8c86/nEjYntEzEbE7Pz8fN/jsrwjWFo5CDqXDKU6GXjCNyKeSftsfg1wCPgXYNOgj5uZu4Bd0C779Pv1505Pdfxzf6E/37Bvjk4/726riM+dnvISEmqEMso+LwW+lZnzmfk4cCuwAZguykAAK4GFU6c5YBVAsf9M4IcljOMY3f7ct7wj6P370a1kaElIdVJG+H8XuCgizihq9xuB+4DPAa8qjtkG3F7c3l3cp9i/J4dwjQlX36qXXr8f3UqGdgipTkq5tk9E/DXwGuAIsA/4U9q1/RuBs4ptr8vMX0TE6cA/AeuBHwGXZeaDvR5/XK/nr3qyQ0h14YXdpD7YJqy68MJuUh+W2iEkTZLGXd5BOhlL6RDy4nGaJJ75SyepW4fQ7z5nhReP08Qx/KWT1K0c9LlvzNsaqolj2UfqQ6dy0BU33dPxWOcCNM4Mf2lAvVaTOxegcWXZRxqQcwGaRIa/NCDnAjSJLPtIJXAuQJPG8JeGxLkAjTPLPtKQOBegcWb4S0PiXIDGmWUfaYicC9C48sxfGrFzp6f62i4Ng+EvjdiJPmXOD4zRKFj2kUZsoQzUqdvHzw/WqBj+UgU6zQUAPT8/2PBXmSz7SGOk1+cHS2Uy/KUx4mSwRsXwl8ZIr8lgJ4JVJmv+0hjpNhkMOBGsUhn+0pjpNBm84ao9TgSrVJZ9pAngRLDKZvhLE8CJYJWtlPCPiOmIuCUivhER90fEiyPirIi4MyK+Wfz7zOLYiIi/jYj9EfHViHhRGWOQ6syJYJWtrDP/a4DPZOZzgBcA9wM7gLsycy1wV3Ef4FJgbfHfduDaksYg1Va3K4QCXh5aSxKZOdgDRJwJ3AP8ei56sIh4AHhJZj4UEc8GPp+Z50fEPxS3bzj+uG7P0Wq1cnZ2dqBxSnW04ao9HT8wZmZ6ii/uuLiCEWmcRMTezGx12lfGmf8aYB74WETsi4jrIuLpwDmLAv1h4Jzi9gxwYNHXHyy2HT/o7RExGxGz8/PzJQxTqh8ngrVUZYT/cuBFwLWZuR74GUdLPAAUfxH09SdGZu7KzFZmtlasWFHCMKX6cSJYS1VG+B8EDmbm3cX9W2i/GXy/KPdQ/PtIsX8OWLXo61cW2yT1yYlgLdXA4Z+ZDwMHIuL8YtNG4D5gN7Ct2LYNuL24vRt4fdH1cxHwaK96v6TunAjWUg084QsQES8ErgNOBR4E3kj7jeVm4DzgO8CrM/NHERHAh4BNwM+BN2Zmz9lcJ3yl/jgRLOg94VvK5R0y8x6g0xNs7HBsApeX8bySOnMiWCfiCl+phpwI1okY/lINnehzgiWv6inV0Ik+J7jTdjWL4S/VVKdLQ/sB8Vpg2UdqkF4fEK9mMfylBrELSAsMf6lB7ALSAsNfapATdQF5SYjmcMJXapATdQE5Gdwchr/UMJ26gKD3ZLDhXz+WfSQBTgY3jeEvCXAyuGkMf0mAl4RoGmv+koDek8GqH8Nf0pO6TQZ7PaD6Mfwl9WQLaD1Z85fUk9cDqifDX1JPtoDWk+EvqSdbQOvJ8JfUky2g9eSEr6Se/FSwejL8JZ2QnwpWP5Z9JC2JXUCTzfCXtCR2AU220sI/IpZFxL6I+FRxf01E3B0R+yPipog4tdh+WnF/f7F/dVljkDQ6dgFNtjLP/N8K3L/o/t8AH8jM3wB+DLyp2P4m4MfF9g8Ux0maMHYBTbZSwj8iVgIvB64r7gdwMXBLccj1wJbi9ubiPsX+jcXxkibIlvUz7Ny6jpnpKQKYmZ5i59Z1TvZOiLK6fT4IvAv4leL+s4BDmXmkuH8QWPiNmAEOAGTmkYh4tDj+B4sfMCK2A9sBzjvvvJKGKalMXghucg185h8RrwAeycy9JYznSZm5KzNbmdlasWJFmQ8taYgWWkDnDh0mOdoC6ofBj5cyyj4bgFdGxLeBG2mXe64BpiNi4S+LlcDCT34OWAVQ7D8T+GEJ45A0BmwBnQwDh39mXpmZKzNzNXAZsCcz/wj4HPCq4rBtwO3F7d3FfYr9ezIzBx2HpPFgC+hkGGaf/18Cb4+I/bRr+h8ptn8EeFax/e3AjiGOQdKI2QI6GUq9vENmfh74fHH7QeDCDsf8H/CHZT6vpPHxzkvOP+ayD2AL6Djy2j6SSuVnAU8Gw19S6WwBHX+Gv6SR8Cqg48ULu0kaCVtAx4vhL2kkbAEdL4a/pJGwBXS8GP6SRsKrgI4XJ3wljYQtoOPF8Jc0Mt1aQME20FEz/CVVzjbQ0bPmL6lytoGOnuEvqXK2gY6e4S+pcraBjp7hL6lytoGOnhO+kipnG+joGf6SxkKvNlCVz/CXNNbs/x8Ow1/S2LL/f3ic8JU0tuz/Hx7DX9LYsv9/eAx/SWPL/v/hMfwljS37/4fHCV9JY8v+/+EZOPwjYhXwceAcIIFdmXlNRJwF3ASsBr4NvDozfxwRAVwDvAz4OfCGzPzKoOOQVE/d+v9tAR1MGWWfI8A7MvMC4CLg8oi4ANgB3JWZa4G7ivsAlwJri/+2A9eWMAZJDbLQAjp36DDJ0RbQ2/bNVT20iTFw+GfmQwtn7pn5U+B+YAbYDFxfHHY9sKW4vRn4eLZ9CZiOiGcPOg5JzWEL6OBKnfCNiNXAeuBu4JzMfKjY9TDtshC03xgOLPqyg8U2SToptoAOrrTwj4hnAJ8E3paZP1m8LzOT9nxAP4+3PSJmI2J2fn6+rGFKqgFbQAdXSvhHxCm0g/8TmXlrsfn7C+Wc4t9Hiu1zwKpFX76y2HaMzNyVma3MbK1YsaKMYUqqCVtABzdw+BfdOx8B7s/M9y/atRvYVtzeBty+aPvro+0i4NFF5SFJOqEt62fYuXUdM9NTBDAzPcXOrevs9ulDtCsyAzxAxG8D/wV8DfhlsfndtOv+NwPnAd+h3er5o+LN4kPAJtqtnm/MzNlez9FqtXJ2tuchkqTjRMTezGx12jdwn39mfgGILrs3djg+gcsHfV5J6sT+/5PjCl9JteEloE+e1/aRVBv2/588w19Sbdj/f/IMf0m1Yf//yTP8JdWG/f8nzwlfSbXhJaBPnuEvqVa6XQJax7LsI0kN5Jm/pMZwAdhRhr+kRnAB2LEs+0hqBBeAHcvwl9QILgA7luEvqRFcAHYsw19SI7gA7FhO+EpqBBeAHcvwl9QYLgA7yvCX1HhN7P83/CU1WlP7/53wldRoTe3/N/wlNVpT+/8Nf0mN1tT+f8NfUqM1tf/fCV9JjdbU/n/DX1LjNbH/v7KyT0RsiogHImJ/ROyoahyS1ESVnPlHxDLgw8DvAQeBL0fE7sy8r4rxSFIndV78VdWZ/4XA/sx8MDMfA24ENlc0Fkl6ioXFX3OHDpMcXfx12765qodWiqrCfwY4sOj+wWKbJI2Fui/+GttWz4jYHhGzETE7Pz9f9XAkNUzdF39VFf5zwKpF91cW256Umbsys5WZrRUrVox0cJJU98VfVYX/l4G1EbEmIk4FLgN2VzQWSXqKui/+qqTbJzOPRMRbgDuAZcBHM/PeKsYiSZ3UffFXZGbVYzihVquVs7OzVQ9DkiZKROzNzFanfWM74StJGh4v7yBJfarD4i/DX5L6UJdP/rLsI0l9qMviL8NfkvpQl8Vfhr8k9aEui78Mf0nqQ10WfznhK0l9qMviL8NfkvpUh0/+suwjSQ1k+EtSA1n2kaSSTNLKX8NfkkowaSt/LftIUgkmbeWv4S9JJZi0lb+GvySVYNJW/hr+klSCSVv564SvJJVg0lb+Gv6SVJJJWvlr2UeSGsjwl6QGsuwjSSMwbqt/DX9JGrJxXP1r2UeShmwcV/8OFP4RcXVEfCMivhoR/xoR04v2XRkR+yPigYi4ZNH2TcW2/RGxY5Dnl6RJMI6rfwc9878TeF5mPh/4X+BKgIi4ALgMeC6wCfi7iFgWEcuADwOXAhcAry2OlaTaGsfVvwOFf2Z+NjOPFHe/BKwsbm8GbszMX2Tmt4D9wIXFf/sz88HMfAy4sThWkmprHFf/llnz/xPg34vbM8CBRfsOFtu6bX+KiNgeEbMRMTs/P1/iMCVptLasn2Hn1nXMTE8RwMz0FDu3rhvvbp+I+A/gVzvsek9m3l4c8x7gCPCJsgaWmbuAXQCtVivLelxJqsK4rf49Yfhn5kt77Y+INwCvADZm5kJIzwGrFh22sthGj+2SpBEZtNtnE/Au4JWZ+fNFu3YDl0XEaRGxBlgL/DfwZWBtRKyJiFNpTwrvHmQMkqT+DbrI60PAacCdEQHwpcx8c2beGxE3A/fRLgddnplPAETEW4A7gGXARzPz3gHHIEkTq6qVv3G0UjO+Wq1Wzs7OVj0MSSrV8St/od0FVNZkcETszcxWp32u8JWkilS58tfwl6SKVLny1/CXpIpUufLX8JekilS58tdLOktSRar83F/DX5IqVNXKX8s+ktRAhr8kNZBlH0kaQ8Ne+Wv4S9KYGcVn/lr2kaQxM4qVv4a/JI2ZUaz8NfwlacyMYuWv4S9JY2YUK3+d8JWkMTOKlb+GvySNoWGv/LXsI0kNZPhLUgMZ/pLUQIa/JDWQ4S9JDRSZWfUYTigi5oHvDPAQZwM/KGk4k6Jpr7lprxd8zU0xyGv+tcxc0WnHRIT/oCJiNjNbVY9jlJr2mpv2esHX3BTDes2WfSSpgQx/SWqgpoT/rqoHUIGmveamvV7wNTfFUF5zI2r+kqRjNeXMX5K0iOEvSQ1U6/CPiE0R8UBE7I+IHVWPZ9giYlVEfC4i7ouIeyPirVWPaVQiYllE7IuIT1U9llGIiOmIuCUivhER90fEi6se07BFxBXF7/XXI+KGiDi96jGVLSI+GhGPRMTXF207KyLujIhvFv8+s4znqm34R8Qy4MPApcAFwGsj4oJqRzV0R4B3ZOYFwEXA5Q14zQveCtxf9SBG6BrgM5n5HOAF1Py1R8QM8BdAKzOfBywDLqt2VEPxj8Cm47btAO7KzLXAXcX9gdU2/IELgf2Z+WBmPgbcCGyueExDlZkPZeZXits/pR0Iw7sg+JiIiJXAy4Hrqh7LKETEmcDvAB8ByMzHMvNQpYMajeXAVEQsB84AvlfxeEqXmf8J/Oi4zZuB64vb1wNbyniuOof/DHBg0f2DNCAIF0TEamA9cHfFQxmFDwLvAn5Z8ThGZQ0wD3ysKHVdFxFPr3pQw5SZc8D7gO8CDwGPZuZnqx3VyJyTmQ8Vtx8GzinjQesc/o0VEc8APgm8LTN/UvV4hikiXgE8kpl7qx7LCC0HXgRcm5nrgZ9RUilgXBV17s203/jOBZ4eEa+rdlSjl+3e/FL68+sc/nPAqkX3Vxbbai0iTqEd/J/IzFurHs8IbABeGRHfpl3auzgi/rnaIQ3dQeBgZi78VXcL7TeDOnsp8K3MnM/Mx4Fbgd+qeEyj8v2IeDZA8e8jZTxoncP/y8DaiFgTEafSnhzaXfGYhioignYd+P7MfH/V4xmFzLwyM1dm5mraP+M9mVnrM8LMfBg4EBHnF5s2AvdVOKRR+C5wUUScUfyeb6Tmk9yL7Aa2Fbe3AbeX8aC1/QD3zDwSEW8B7qDdGfDRzLy34mEN2wbgj4GvRcQ9xbZ3Z+anqxuShuTPgU8UJzYPAm+seDxDlZl3R8QtwFdod7Xto4aXeoiIG4CXAGdHxEHgr4CrgJsj4k20L23/6lKey8s7SFLz1LnsI0nqwvCXpAYy/CWpgQx/SWogw1+SGsjwl6QGMvwlqYH+Hw0jphXTh/5hAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "#t = np.array([2, 4, 6, 8])\n", "t = np.linspace(0, 10) # Add random noise\n", "\n", "pa = -20\n", "pb = 90\n", "pc = 800\n", "\n", "y = pa*t**2 + pb*t + pc\n", "\n", "\n", "plt.scatter(t, y)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 5.1 如何得到更新项?\n", "\n", "$$\n", "L = \\sum_{i=1}^N (y_i - at_i^2 - bt_i - c)^2\n", "$$\n", "\n", "\\begin{eqnarray}\n", "\\frac{\\partial L}{\\partial a} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i^2 \\\\\n", "\\frac{\\partial L}{\\partial b} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c) t_i \\\\\n", "\\frac{\\partial L}{\\partial c} & = & - 2\\sum_{i=1}^N (y_i - at_i^2 - bt_i -c)\n", "\\end{eqnarray}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. 如何使用sklearn求解线性问题?\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X: (442, 1)\n", "Y: (442,)\n", "a = 949.435260, b = 152.133484\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAABIW0lEQVR4nO2de5xVVfn/388MB5jBy4CSwQii5Ve+ot+YJDXpIljiJXW+UpiZpZnkL828fEksCyxNirxmaZqpqSmmNuGlzAQzrwgOqCgoKogjKgmDF0YYZtbvj332sM+evfZee599rrPer9e8ZmaffVlrn3M++1nPetbziFIKi8VisVQXNaVugMVisVjSx4q7xWKxVCFW3C0Wi6UKseJusVgsVYgVd4vFYqlC+pW6AQA77rijGjVqVKmbYbFYLBXFokWL/qOUGhr0WlmI+6hRo1i4cGGpm2GxWCwVhYis0r1m3TIWi8VShVhxt1gslirEirvFYrFUIZHiLiIDRWSBiCwRkaUicn52+w0i8qqILM7+jM1uFxG5QkRWiMgzIvLJAvfBYrFYLD5MJlQ3AROVUu+LSAZ4RET+ln1tmlLqDt/+hwK7Z3/2A67K/rZYLBZLkYgUd+VkFns/+28m+xOWbewo4I/Z454QkQYRGaaUWpN3ay0Wi6VMaWltY/b9y3mjvYPhDXVMm7QHzU2NJWuPkc9dRGpFZDHwNvCAUurJ7EsXZl0vl4rIgOy2RmC15/DXs9v855wqIgtFZOHatWuT98BisVhKTEtrG+fe9Sxt7R0ooK29g3PvepaW1raStclI3JVSXUqpscDOwL4ishdwLjAa+BQwBDgnzoWVUtcopcYppcYNHRoYg2+xWCwVwez7l9PR2ZWzraOzi9n3Ly9Ri2IuYlJKtYvIfOAQpdSvsps3icj1wP9l/28DRngO2zm7zWKxWBJRbi4PP2+0d8TaXgxMomWGikhD9u864IvAMhEZlt0mQDPwXPaQucA3slEz+wMbrL/dYrEkpRxdHn6GN9TF2l4MTNwyw4D5IvIM8BSOz/0e4BYReRZ4FtgRuCC7/33AK8AK4Frgu6m32mKx9BnK0eXhZ9qkPajL1OZsq8vUMm3SHiVqkVm0zDNAU8D2iZr9FXBq/k2zWCzVQj5ulXJ0efhx+1JOrqOySBxmsViqF9et4lrfrlsFMBK/4Q11tAUIeSldHkE0NzWW1TyATT9gsVgKSr5ulXJ0eVQC1nK3WCqcao8kKUeXRyVgxd1iqWDydXkUgzTcKuXm8qgErFvGYqlgbCSJRYe13C2WCsZGklh0WHG3WCoYG0li0WHdMhZLBWNdHhYd1nK3WCoY6/Kw6LDibrFUOOXg8ij3cMy+iBV3i8WSF5UQjtkXseJusSTEWqsOYeGYffF+lAtW3C2WBFhrdSuVEI7ZF7HRMhZLAiph8VCxKMdc5hYr7hZLIqy1uhUbjpkHSkFXV/R+CbBuGUtJKGd/tUnb4iweKte+ptUuG46ZAKXgb3+DmTPhhBPgu+nXNLLibik65eyvNm3btEl75OwHwdZqufY17XaVQzhmReAV9aeeglGjYOjQglzKumUsRaec/dWmbWtuauSio/emsaEOARob6rjo6L17CVy59rVc2xVFS2sb42fNY9fp9zJ+1ryyqqMaiivq++8Phx8Oa9fC738PL74IX/lKQS5pLXdL0Slnf3WctplYq+XUV68bRmn2KYf3QEe5joJCUQr+/nfHUl+wwLHUf/97+MY3IJMp6KWt5W4pOuUcXZF228qlr64wtoUIO5THe6CjokYbrqh/+tNw2GHw1ltw7bWwfDmcdFLBhR2suFtKQLGjK+IM5dNu27RJe5CplZxtmVopeiRJkDD6KfcIl3IaBWnxivqhh8Kbb8I11zjul29/G/r3L1pTIsVdRAaKyAIRWSIiS0Xk/Oz2XUXkSRFZISJzRKR/dvuA7P8rsq+PKnAfLBWGqb86DfwWqzuU1wl8QdrmN5XDTOcCESaAhX4P0qJcRkGBKAX33w8HHOCI+po1W0X95JOLKuouJj73TcBEpdT7IpIBHhGRvwFnAZcqpW4TkauBk4Crsr/XK6U+LiJfBX4BHFOg9lsqlGJFVyRZGp9m22bfv5zO7lw17+xWRV+arwvdbGyo49HpE4vWjnwwjVAqKkrBP/7h+NSfeAJGjoTf/c4JbyyBoHuJtNyVw/vZfzPZHwVMBO7Ibr8RaM7+fVT2f7KvHyQiueNSi8WQfKMjSj2UL/X1XaphoVExR3yRuKI+fjwccgi88YYj6i+9BFOnllzYwTBaRkRqgUXAx4HfAC8D7UqpLdldXgfcO9wIrAZQSm0RkQ3ADsB/Umy3pQ+QRnSEzmKtEWHX6fcWfMFNuVRKqpaFRiWPp1cKHnjAsdQffxxGjICrr4YTTywLQfdiJO5KqS5grIg0AH8BRud7YRGZCkwFGDlyZL6ns1QhaWQbDBrKA3Qpx1VS6HC6NF0J+a4oLbkwVjJKwT//6Yj6Y49tFfUTToABA0rdukBixbkrpdpFZD7waaBBRPplrfedAXe83AaMAF4XkX7A9sA7Aee6BrgGYNy4cSWYYrKYklRU8hWjNFwafou1RqRH2F0KlZ7W7X9HZxe12es2JrSYKzLGuxoIEvWrrnIs9TIVdReTaJmhWYsdEakDvgi8AMwHvpzd7ZvAX7N/z83+T/b1eUopK94VStxok3yP85JWdERzUyOPTp/Iq7MOp1vzUUzbB+7tPzgjBddiTyLGFRXjXQ24ov7Zz8LBB8Nrrzmi/tJLcMopZS/sYBbnPgyYLyLPAE8BDyil7gHOAc4SkRU4PvXrsvtfB+yQ3X4WMD39ZluKRVJRSUOMCjEJWKxwurTFuFwmZqser6h/8YuwahX89rewYkXFiLpLpFtGKfUM0BSw/RVg34DtHwKFSZZgKTpJRaUQLpU0JgGLFU6Xthg31GdYv7Gz1/ayiPGuBpSCefMc98sjj8DOOzui/q1vVZSge7G5ZSyhJI32SCtKJO1JwGJFjaQZJdPS2sb7H27ptb0UK12rDqVg/nxH1P/9b2hshN/8xkkRUKGi7mLTD1hCSeoaKee4aq8P/tHpEwsWJZNW/4MWQgEM6t/PTqYmxbXUP/95OOggeOUVR9RfftnJrV7hwg7WcrdEYGLphkXFlCquutQFMtLsv86Vs6Gjt5vGEoFS8NBDjqX+8MOOpX7llY6lPnBgqVuXKlbc+zCmAhjmGokK0Su0oAb1ASiLsEGT/qdd9amaSP0B7bpfHn4Yhg+vWlF3sW6ZPkoaoYpQ2hA9XR9mzl1a9DYlSZNg+h6Us4urUKT1+QQcS/3AA2HiRCfq5de/dtwvp55atcIOVtz7LGmJcilD9HR9aNe4KwrVpqRClHbVpyTtLteqRql8Pl1RnzDByc54xRWOqJ92WlWLuot1y/RR0hLlfF0GOreKyXA8SVsLQdI0CabvQSHmD8p9xWten89//ctxvzz0EAwb5oj6ySf3CUH3YsW9j5KWHzefuPEggZl2xxJQ9ESHhImOrg+D6zN82NldtNSwcYTIK9RBqRAg9z3QifDCVeu495k1PbHvDXUZZh45xliY4zyQSjE5nejz6Rf1yy93RL2uuucmdFi3TB8lLT9uPi6DIIHp7FK9wv78w3HXndDW3oE/l3RdppYZR4wpampY01WvfvdNkLD73wOdCN/8xGs5i5raOzqZ9uclxq4V0wfSeS3Pcuacxen4vmMQ6/P58MOOP/3AA2HZMkfUX34ZTj+9zwo7WMu9z5JmqF7SqJg4bhV3X78lq3AqCSnolZSrUGLut2QnjB7KnYvatCMFd/8gSxSgVoRupQLfgzj3KE4REBPLuKW1jVueeK1X4ahCJVrzYvT5fPhhx1KfPx8++lG47DInl3ofFnQvVtz7MKVOAasTGN2+EGzJusIet6JQEndDkJvkzkVtTN6nkfnL1vY6l3//ILqV4tVZhwe+FucegfnDwMSdNvv+5dqKgMWYF9B+Pv/9b0fU582zoh6CFXdLyQgSmEyt5PjcYavotLS2aYUu7uRq0glFnZtk/rK1gQ8Xk8LU29dltK8F3SN3pBKE6ZyJiWUcdk9N5gW810kFr6jvtBNceil85ztW1DVYcbeUDJ3A6La5ghFE3IngQke4RG338sHmLbS0tmkXkLnt9bqB5jy1ms6uXInP1MTLNRM1ctONGgSM5gVSc9088ogj6g8+uFXUp06F+vr8z13FWHG3lBSdwPi3jZ81T2sBZ2qFCaOH9kyymhTGSCrGcaM4TNwqnV3hvvKgezRulyGcf/fSxNEyJuhGDcftP9LIws97XYFf1C+5xLHUragbYcXdUhGECUVXt2LOgtU9rhyTEnpJQ0Hjhn7qyvz5iSuEacyX6Pzk3u0N9RkG9KthQ0dn8dIjPPqoI+r//KcV9Tyw4m7Jm2LEQYdZwN0KbYUlnXsgaXx+3Cgj//4Q7C8P87sXgrD4eW/kz/qNndRlarn0mLHaPqaWI/+xxxxRf+AB+MhH4OKLnQIZVtQTIeVQAW/cuHFq4cKFpW6GJQFB0SB1mdrU48pNok50CARGo5RicU7TT/8RWHRjcH2G1p8cXNBrmyygqtVsj4pGyute+kX9nHOsqBsiIouUUuOCXrOWuyUvoibTWlrbmDl3aU++l8H1GWYcEd837O5/9u1LAsUnDJ17IF/XRhJBaw8Qdt32NB8+/oej7h7qtke5jXT3MrQPjz/uiPo//uGI+q9+5Yj6oEFG/SllSudKwIq7JTbeL1ZYHHRLaxvT/rwkJ6xx/cZOJ8UA8cPk3P3jWPCFSjuQNPzP1D+dT3hhkPCZhGSC3nJPWkEqqA+Dlyzi87f+xhH1oUNjiXrYeaE88uKUCzb9gCUW/iX0OoY31GkrCLnRIUlw0x3Uij/xwFbc1wqZdiBp1kLTZfVJz6/LUGmyEKouU8ux+41ItYKUtw+fbHuBq2/5EZ8/8ShobYXZs+HVV+Hss42FPei8ULw005WEtdwtsTCxAF0xOHPOYu0++YTJ6Sz4Qvj6dSQN/zOdkE16fp3w6SzyoNQH43YZkmoFqaa2ZZzx6J/4/KtP85/67fn5gd/ih/dcEUvQg85rur2vYsU9ZQrpC8zn3Gm1K+wLJJBz7rB8Kvmm3y11Gb98wv9MfP2689eIsOv0e2M/FLqUoi5Ta/QwTCstxRfffZWv/f0GDnx1Ee/UbcfPDzyRm5oOZ8hHBvPDhMIOfbcyVVwi3TIiMkJE5ovI8yKyVES+n90+U0TaRGRx9ucwzzHnisgKEVkuIpMK2YFyItXqMSmeO8126b5AjQ11vQpOT5u0B5ma3u6TTG28lZQ6ilHoWkehqyMFnR8ckQ57D8Pen6JlynzySTj0UK656nv8z1svcdGBJ/DZU67jmv0mw6BBed+jvliZKgkmlvsW4Gyl1NMisi2wSEQeyL52qVLqV96dRWRP4KvAGGA48E8R+S+lVPwYtgqjkMuw8zl3mu2KE9PsnjsoWgacVaeVGu1Q6JGD//xBoYtB72HY+1PwRHFPPgnnnw9/+xvsuCP84hc8Or6Ze/79Oh3tHaErhuNQ6lFbpRAp7kqpNcCa7N/vicgLQNhdPAq4TSm1CXhVRFYA+wKPp9DesqaQvsB8zp1mu7xfLHepv3cyy2SIn2+0Q6EqE8U9Z6HF0nv+XaffG7iP/z0sifAtWOCENP7tb7DDDjBrllOfdJttOAI4Yvx/pX7JUmc0rQRi+dxFZBTQBDwJjAdOE5FvAAtxrPv1OML/hOew1wl4GIjIVGAqwMiRI5O0vewopC8wn3On3a6gCU0TgQ7La246kgh6MJw5ZzFnzFmc2DIsxDlNrhlHgOO8h0UTvgULHEv9vvt6ibql9BiHQorINsCdwBlKqXeBq4CPAWNxLPuL41xYKXWNUmqcUmrc0KFD4xxathTSF5jPuQvRrrjhaF6/vw6TkYQunztEzyXoCkLnc84kJJkDKSs/84IFcPjhsN9+jivmoouckMZzzrHCXkYYWe4iksER9luUUncBKKXe8rx+LXBP9t82YITn8J2z26qeQg6J8zl3IdoV19VjEkIZNpKIqmbkElYHVDfSiHqopF15SPdgPPt2/eKucvAzP3TzvWQu+Bnjlz/Jhrptef170xlz4Q9h222L1oZqotCrbCPFXUQEuA54QSl1iWf7sKw/HuB/geeyf88F/iQil+BMqO4OLEitxWVOIYfE+Zw70fLwEOK6eqIENMwKjZtXJuhaYSMNk7S8acZQ667VpVSoa6tkfuannuLNs87lwEceZP3Abfnl577BjZ/8Et3bbMtFK96lucmKe1yKscrWxC0zHjgemOgLe/yliDwrIs8AE4AzAZRSS4HbgeeBvwOn9oVImUoknxDJuG6CMKs8KizPdOl82LXCRhq6sMOocyYlbHVtWa20XLgQjjgC9t2XukUL+OXnvsFnTrmO3356Ch8MqC+vtlYYxVhlaxIt8wj0KjIPcF/IMRcCF+bRLksRyCdEMq6bQBeiZxJrHcdq1j1gwkYa/gggfxm7tH3bUYnPSr7SctEiJ/rlnntgyBC48EI+s3Z33hvQO0tjkrbapF/FWWVrV6j2YfL9gMVxE+TjMw4r96Z8/0/eJ7hNuoeLW8HJbdNlx4xN3E5TGiPcQCVbablokRP9cvfdMHgwXHghnHYabLcd282ax3spRFzZpF8OxVhla8W9D1PsZdxRDwN/BSClYENHJ9vXZcjUSk7N0KAi0QqYv2yt9trQuxaptzCFKzQXHb13aO7yfAmrzlSSCJinn3ZEfe5cR9QvuAC+9z3YbrueXdIqyFHweqsVQmoFTkKw4t6HmTZpj14peeMWWU4Lv0XnLWjR3tFJpkYYXJ+hfWNn6ARoW3sH42fNC7S2/Q+XoLqsJkKTr1shaCFYVM1XE2K3y0DUg9qcz4im2Em/ytUFVIzoJyvufR3/bIp+rq+gRE2adnYr6vv366lW5BbDDsJ0qJ9EaNJyK6Qd+RKrXa2tjqj/9a/Q0AA/+5kj6ttvX/A2F3O0WO4uoEJHP9l87n2Y2fcvz3F1QPJc67oFQlGvucRNoxAV4WISeaATlDChiYpyMOlrITCKvmhtheZm+OQn4V//ckR95Uo477xIYU+LYi7G6ut5363l3odJa4gcZiGBWZoCk1hzr+j6XRtJ+pHE7xl2z0ppKYa+l4sXO5Z6S4tjqf/0p3D66UUTdC/FXIzV1/O+W3Hvw6Q1RI6ykEz82mGTjBAsuu6wVueiiepHEqEJu2elnCwMateeb73COQvmwC8edYT8/PMdUW9oKGhboijWYqy+nvfdinsfJq0Z+yQWUlQ2Q2+0jCu6EJwmOKwfURNqcYUm7Fq6ylPFsBS97drzrVc4/bFbOeTFx+ncZruyEfViU4yIlHLGinsfJq0hcpSFlEY2QxOXh78fED9zZRRh90znIiqGpdjc1Mh2Lz5P7c8u5PNLH+G9gYN44Ttn8d+zftznRN2lHPLxlBJREavlisG4cePUwoULS90MS0KCcr+4q08hnVqnOtdLY0OdNiY9LKIm7VS+La1tnH/30pwQTihSXdclSxw/+l13Oe6XM8+E73+/z4p6X0JEFimlxgW9Zi13SyqxwAMzNT0C3lCXYeaRY3LOkW/t1ySTpmGvhVnxce+HLrFZ0H1IgvceeGPif7ZrFxNvv9oR9e22gxkz4IwzrKhbACvufZ40qiL5hW3Tlu6cfZJOoJlkgwxzeURF4ARNdia5H7oY/UED+qUi7N72dCnF6Ldf5fS/3MrEFx+jc5ttycyY4VjqgwfndS1LdWHj3Ps4+cZt644/Y87ivOO8oxY2RU2OmWR79Fv3SWKj0wi5MykkMvrtV/ntX37O36//Hp9ZuZjLDziWI866yUnyZYXd4sNa7n2cfOO2k7o+8mkbmPnMTWLh/ZZ/EqFOEnLnz6Pz/odbetJA+AuJ7LF2Jac/eiuHL3+Ud/vXc/kBx3Ldp47i3YHbIJu0l7D0cay4x6Bc81Tk065847aTuD5M0Z07bBLVj+sS0k36+i3/JEIdN+QuLI+OS0dnF3fc+Heue/CPTHzuYd7rX8flB3yVP4w7ig11W4tj9JWYbUt8rFvGkHwKW5RzuyaMDq5fO2H0UCMrNonrw5Sgcwtbk4PFuffNTY1cdPTeNDbUIegLhCRZHm96bpcod9N/rV3JlS2zuPnyb/PZV5/mt585ls+c8gcu/ezXc4S9L8VsW+JjLXdDyjVVab7t0qXInb9srZEVm8T1oSNoBHLR0XsHFtFI4vIxmdhNGhsdZ9JY97Dbfe0qvv/YbRy27BE+6D+QGyZ8nRPuuJzhqzrY5v7lbEgxg6Sl+rHibki55qmI8plHiVTY8ZceM9bI3RDX9RGEzr/v5lYPilnP5+Eadm8KvTze/9D0i/qVn57Cnw6YzPTjDoAhQ2geUh5ZDC2VhRV3Q8ohT0WQIOnatX1dJq+EXUHl52pFciJHgvKlu/vH9f9HjUDSfLiWOhWs66Pf+Y1XOP2x2zh82SNs7D+Q6z77VX7bdAT1w3ZiurXKLXlixd2QUuep0AnS5H0ac6oJue0SSZ6wy9svd19TMUxq9UaJd5oP11K72JoHbGDcgqsY/sDdbMwM5I8TvsbQH0/n5Al7cXLBr27pK9gJVUPiTpqljU6Q5i9bG9iu9oAIDAhO2BXVr2LkxY7KrZ5mHvCSudiefx6OPRb22oudH5tHzbnnss0bqzlh3s0cPmGvwl7b0uewlnsMipWqNIgwQQpqV5wkVlH9Ml36n09IpukIIp9QVLd9umxK+Uz8hrbj+eedwhhz5sCgQTB9Opx9Nuywg3HbLZa4RIq7iIwA/gjshBOscI1S6nIRGQLMAUYBK4EpSqn1IiLA5cBhwEbgBKXU04VpfnVgIhY6t0SNCC2tbYEhfX6xzNQKH2zawq7T7zUWx5bWtsBi1G6bvPuZum7C+qvbnu8ag6hUBvlO/Ab1kxdecET9ttugvt4R9bPOgh13NG53sSnXtRyW+JhY7luAs5VST4vItsAiEXkAOAF4UCk1S0SmA9OBc4BDgd2zP/sBV2V/WwIwFQtdMYsupQL394tlff9aPtjcRXtHZ+h1/IRZut4YeVM/dlR/g9qSxgRoVGz5wEywh9Ivdhs3b4nup1/UzznHsdTLWNSh9BPNlnSJ9Lkrpda4lrdS6j3gBaAROAq4MbvbjUBz9u+jgD8qhyeABhEZlnbDqwVTf7brG6+V3hWsdf7v5qZGHp0+kUuPGcvGzb2FzcRvHuaHvvmJ13oWE4UVq/aSxH8fdoxpzdIof/r6jZ29Fn8FLRALWk3ac/5ly+C442DMGJg71xH1lSvhoovKXtjB1hytNmL53EVkFNAEPAnspJRak33pTRy3DTjCv9pz2OvZbWs82xCRqcBUgJEjR8Ztd9UQZ3KvuakxUbWfMOs7SvSi0gu41p0IBJUGqM26jVzrN0k7dK+519bVbvVa3A31Ga0wu/gt8Chr3+Vj76zmnKfugF/Ocyz1H/zAsdSHBq/+LVfKdS2HJRnG4i4i2wB3Amcopd4VjwWplFIiEqvqh1LqGuAacIp1xDm2mogb4pckJDAq6VUYbvm4sDcoTABdt1GUSCZJ3evG3fvbMnPuUjZt6c4R/UyNkKkVOrvCP2reexUlaru98zrfe+w2jnzhYdTAgRUr6i7lsJbDkh5GoZAiksER9luUUndlN7/luluyv9/Obm8DRngO3zm7zRJA3BC/JCGBui+nZM8XRnNTI8ftP5LeziAzggTYT5LUvXWZWro0VcTaOzp7XbOzWzGof7+ekM8g9xbk3ivdfdvtnde59O5f8cB13+WQl57g5W98h36rVsKsWTnCbuoyKhfSDDe1lJ5Icc9Gv1wHvKCUusTz0lzgm9m/vwn81bP9G+KwP7DB476x+IgbP+/fv6Euw8BMDWeG5E/XJeA6bv+RRhNlFzTvzXH7j9QKItl2xBFgl1oRJu8TXjvVdY+413fvUWNMi3JDRyePTp/Iq7MO5+Ipn4gUMv992+2d17nknot54LrvMumlx7l23/+lru01/uuG3/ay1ss10VwYpV7LYUkXE7fMeOB44FkRWZzd9kNgFnC7iJwErAKmZF+7DycMcgVOKOSJaTa4GggKNzNNYQv6XC666IZ8Y8RbWtu4c1GbVqjrMrXMPHJM4DXCEoqB47a5c1Eb43YZEhgy6a9C5Apw0MpZty0DMzWB/nVdwjPdPXH//vXV93Hqo7dy1PP/YlO/DNd+qplr9z2agY3DOEXjgin1KtiklHIthyVdbIHsIhNWTDrulypJ0egk5FNo2qRUnnsef5tN+hf0oIR0inLz4otwwQWoW26hozbDH5sO59p9j+adQQ2R59t1+r2B8xQCvDrrcPM2WCwh2ALZZUQciy5qQUmSotFJ0J1PQPsQ8bZ9+6zrqH1jZ6xoGZPojTBLM/FinJdeggsugJtvhgEDkLPO4qFDvs5NT73DuvYOo3S7uuichvoMYBcLWQqPFfciYxpuFuVyMV05mgZxoyj8bW/v6KQuU8ulx4yNlRYhn+iNuO6FltY2br3lQabcfyPNSx9CDehPv7POgv/7P9hpJw4DDjvI+HSBYaHudrtYyFIMbOKwAhAWJRGVIMslakGJLnbdJAImLrooigmjh0YWdfa3PU51paTRG3GjVB746yOoE07glktO5LBlj3LduKM48JQ/0PK1M2CnnUKP1bGhIzimfkNHp10sZCkK1nJPGZ1VtnDVOuYvW9urohAEC1aUha97XZG+9dfc1MjCVeu49cnVdClFrQifHLl9Tqphf1FnXdv9OeLDqislmQiOZRWvWAEXXMCEm26isybDH8YdxTX7Hc1/Bg0GYObcpQWpTVuMxULW7WOx4p4yOqvslide6xExBT2ipvPfRrkkwopHQ7pfbn+0TJdSPPryul77udZnVNtd4TaprhTXvWI0p7FiBVx4Idx0E/Tvzw37HMnv9p3M2m0G5xzX3tEZOxePS1iWyziuqSRYt48FrLinTphF7f8/LOpjwuihzHlqdc6Kykyt9Fj4YeKR9pfbdBk+xCvPl68F694zt0pUWEz9G+0d8PLLzkTpTTdBJgOnnw4/+AHX3/A8aw2uGSeUMWzUsXDVupyHPaS7WKhSwzAt6WLFPWWicrF4cUUsSIznLFjdW6w8/4aJx/hZ81L9csdxF/jL8yVJY2xiwQbFwOsYuX4NP1h0J/zqgRxR56MfBWDapC6jcE2Idy+CRh3uKMjbWoHQhVxxsTliLGDFPXWCLOqoqJYgS6uzu/cRnd3KyGWR9pfb9IHlncw1cafo0hhv3LwlMEe9F5PRxMj1azjt8Tkc/dw8pH8Gvvc9R9SH5SYpDXoYbdy8RbsQqqW1jfPvXtrzekNdhplHjklcK1YB85etjTzWFJsjxgJW3FMnSCgmjB4aWOfUFcI4ohu1b0trGzUaF0XSL7dJ8jCIP5nr7jtz7tIe3zZsTb/r3cfF64rRMaL9TU57bA6Tn3uQrtp+rDz2W3z8Vz+l5c1uZt/4Am+0P91rNOF/GOkWm00YPZRpdyzJcZe1d3Qy7c9LQtvrfhaKsTah1PV+LeWBFfcCEGS1jttliNZNEceVEybQriAFCbtpCGFQG3V+Yj9xc72Ac69m3788R9zBrNCHn1xRr6Xf6d+j3znn8PFhw2LPQ+hcS7PvXx6YWdI/qgpqb1CklEuaVnUaJQktlY9NP5CAtMPMgkQrUyMg5AhJ1JJ33XL9WhGO3W8E85et1bbZJC2C12oOCudMmmTKdKm+rn87t7/JaY/fzuTnHqRbapizz2F85Gc/5pCD9+lp99m3Lwl86MVN1aBra5z2pnnvLH0bm34gRQoRZqaztIK26crQhbkq3ORcYW2OE2EhOMvolXIW5eT7gDP1EftdF35Rv7npMFoO/jonfmU8h3geSLrRTNA5ox7cYaOsqPa6uJFS1qq2FBIr7jEpVJiZbgIy6pwmibl0RS28bY4qk+e/zvqNW1MK5CtMpj5iV1h3bn+TUx+/nS9nRf0v+x/JlD9fyYnDh/dKQRo18WpS5NtdgObmyamtEbp8E96ZGtG210/aid0sliCsuMek3MLMosSrLlOrfd3bZl2cuJtD3SQdQlJL1NRHPGPvejacN5vmZx7osdRvGD+FM0+YAMOHR/bRj/8BYrIArb2jk0yNMDBbcBxyo2X8CdP81Z/sxKalWFhxj0m5hZmFiVdjQ3hO9e3rMj1/69wW7vYkdUzjCrx2/5Ur4ec/5+Drr6erppa79juSi5uOonbEiMgHSViJPr+f23QBWme34iP1/Vn601zrOyhhGkCNQLeKTo9ssaSJTRwWk6hkVsUuraZ7qLhD/+amRqZN2sOZoPXxQTae3N1fd56w60DvGqqpJcFatQq+8x3YfXe48UY45RRqX3mZrzx2F0/85ps9/QtD935dPOUTgYuqTAl6EOhGUd1q62fECrulWFhxj0lYKbJSlFYLEq9MrfDBpi09DxiAbQb2HqR1dqkeEQ57aLW0tvHBpi2x2pWXm8or6jfcAKec4qQO+PWvoTGeOMYpHafLWBlE0IMgrM8266Ol2Fi3jIawqAmdC6EUOT38/uqG+gzvf7ilV8KrKL97WMSO6dJ8L4ncVKtWwUUXwR/+ACIwdSpMnw477xz/XB5Mk48lWYDmJWq9gl3+bykmVtwDSBruGBVxUii84jV+1rxey+bd4tJBfnWVPca7YMlLUJ6aKGJPGr72Gvz856mLehLiLkDzokun4GKX/1uKiRX3AJJa4FERJ8UgLNZdFzkT9vCKa23GmjR87TXHUr/uOkfUTz7ZEfURI2Jds1D4R29RYZ+6dApgo2Qsxcf63AMIiwwJmyyNijgpNG7pvSBcX7Nu4lTnE45jbbqx3pHC/tpr8P/+H3z84461fvLJTo713/ymrIQ9yfxJc1Mji2cczGXHjDXy81sshcJa7gHofKduOTgItnYbIwpoJCFOqgNd6T1wMi2eOWdxqFh7UxBHxWqDoqOzO+f4oPwqOaxe7Vjqv/+98/+3vw3nntsj6OVUPSjf+RNTP3859dlSXURa7iLyBxF5W0Se82ybKSJtIrI4+3OY57VzRWSFiCwXkUmFangh0UVN+IXTb+0mrfmpI671GOZCWb+xs+ccOhrqM4w9/x+cMWdxzzXbOzpBweD6TI4V+qFP2EPbsHo1fPe78LGPOcJ+0kmOpf7b3+YI+7Q7luT0ddodSwoaaRQWtlqsUnjFjq6y9B1M3DI3AIcEbL9UKTU2+3MfgIjsCXwVGJM95rciUhtwbFkTFD6ns4i9X/Y4YXcmxC2knO+E3fqNnb18xeBY5PX9+/HqrMN7YsuNCn2vXg2nnuq4X7yiftVVMHJkznHn3720V7bFzi7F+XcvzatPOqKE1bSQeT7YQtmWQhLpllFKPSwiowzPdxRwm1JqE/CqiKwA9gUeT97E0uAfVusy/Pm/7HFrfoYR5fv3D+WjojXSbEtoPpjXX9/qflEKvvUtx/2yyy7a8wcVxgjbni9Rbpdpk/Zg2p+X5BRNCcofkw/llsrCUl3k43M/TUS+ASwEzlZKrQcagSc8+7ye3dYLEZkKTAUY6bPiypFSFEBI6vv3+nA/2LQl0BpP0hYvQdf68dhtOeS6i+Daa41FvRgE+bWNhNU/O51y0FO5pbKwVBdG+dyzlvs9Sqm9sv/vBPwHxw39M2CYUupbInIl8IRS6ubsftcBf1NK3RF2/krJ555k8iufCbOgjI+6Yg+6TIMmWSOjiMw3/vrrMGuWI+rd3Y6o//CHWlEPuidB4YMubj76C5r3jt12XZ76Af1qAq/n3kfdSC3sPif5bETl0LdYwgjL555I3HWvici5AEqpi7Kv3Q/MVEqFumXKRdyLUYQj7pfXtEybv1BE0DmSLKYaXJ9hxhGa+qBtbY6oX3NNj6j/48gTOf/Zjdq89O4KWq+7oy5Ty+R9GpmzYHVg7ViX+kwNPz/6f2K9JzqRHlyf4cPObu17E1aUw5+LHXqv4nX7FFYgBeJ95mxkjcVP6uIuIsOUUmuyf58J7KeU+qqIjAH+hONnHw48COyulAo1G8tB3AthRcW1/vI5p3vesC982LEu7sgg9Fx+UT/xRPjhD2lZnzGqKBXW9qiHUNz3JKzK06XHjNWKZZwqSgMzNYFzA2lWXCqWlW8fIJVFXpWYRORW4EBgRxF5HZgBHCgiY3E+uyuB7wAopZaKyO3A88AW4NQoYS8XCpEXxnTCLM4XKmzSNCpNQthEnUD0l7mtDX7xC0fUu7p6RJ1RowCYfVvvVAVhlri/7e5k9Kjp92r36+js4uzbl/TE7CdN+Tu8oS508jvoPuvCYXUuL13obJLPUzHyFhWiypildJhEyxwbsPm6kP0vBC7Mp1HFoBhV6U0mzPIp3Bx07rAvfOLKQG+8sdVS7+qCE05wRH3XXXN2yyeHjuDci+amRm0aBxf3NRPxSToR7r/PUW0yJennqRiRNaVIfGcpHH0y/UBQjHOc1K6mTBg9tNd5Tav/5BPrrPvCx15k9cYb8P3vw267ObHpxx8PL77oTJz6hB3yy6GjoKfPx+5nnoIg6l7ls/bADYmsy9SGCntDXcY4VfD2dZlE+f6LEXdvQzOriz6ZfiBIUIO+uvmuLr1zUVvOeQWYvE+uKyDuF8ok+kX3hTctZ8eaNY775Xe/g87OrZb6brtp2zT7/uV5W7Zun92omFuefA2TU0aJTz5rD0zKGM48ckzPvmGpgjM1wgebe6djdtsYRjFCcW1oZnXRJ8U9TAwa6jJs6OjMezJJ9wCZv2xtzra4XygTsQn7wocKXVbUu66+GtXZyZ17fYE7Jh3PccdOoHk3fQRH1MNmcH2G+v79ImPvvX2+oHlvLmjeO8d9VqNxjRRSfEzKGHrz/HvxpwreuHlLYDrms29fEni8F+MHcx6UYi2HpXD0SXEP87EPGtCPxTMOzvsaphZ53C9UHLGJwhXOztfbOHvxX/nywnuQLVto2esgLtt/CqsbPgrAcwHWpWl4ZV2mtlcope6BsDFb9s+7r/dhpIsYKcVCMpOIJ/+DdFfNRHGXUkYWvF/gXXdUWgJfjAeIpXj0SZ97mBik5V+M8pG6SavOnLOYgZkaGuoyRj7hgZngt8xbM9WEltY2LrnxX3zrzit4+HffZvJjf6Fl9Oc58rTrOPuQ03uEHXr7tb1zFmEE9cV9KHR0dvXyS6/f2BmaOKu5qZHJ+zT2+PZrRXq5udJGl0TOTQERJ8lXVB3aqHkWm2jMEoc+abk3NzVy/t1LA2OT0xrih1nkfgt0/cZO6jK1kcUgjrv28V5pdsF5QseyXt98kw+/933+8eTd9Ovawl17TeTKTx/Da4OHaQ/xPvSiXEMQbNn6+x3kTg9zU7jzGK5rpksp7lzUxrhdhhSljKE78e62O26oYFTunyjDotDRLDYUsrrok5Y7wIwjxqSantdPWJRGkgiZltY2Hn15XeBr3Thf/EgL7s034ayzYNdd+fJjf+Hu0Z9j4sm/4weHnREq7JD70IsSIcGJFPJj8lCArW4Kf39KlUWxuamRR6dPDMwOGuf67mdCF1UUZVgUOprFZqmsLvqk5Q7F8S/qJi+TfElnzg1PfRtqZb35Jsye7YQzbt4Mxx/P13aYyIJ+Q3qdp6Euw6Ytucvy/WIdVQhaQaBFHUeEgizSUofqpXF9tz9J5g4KHc1S6vtrSZc+a7nDVovMm6e8GMSNWW5pbTPK7NjLynrrLTj7bCeE8bLLYMoUWLYMrr+erx03MXDkMvPIMUzepzHHH+6KtWtJB/mhI9uCE+MdB7+omN43XRGOsOIcJqQVa5409j7tYjB+ihFLbykefVrcS0XcL2mcYXFbe4cj6v/3f85CI6+o33CDUziDcIGZv2xtqPvBf6wOrzi3tLbxweYtvfapAWo0J/GLisl9O6/lWc70VJJyRzTntTyb92RkmuJqYlj4H0ZAqsVg/BT64WEpLn3WLVNK4rqETIfFO36wnu88eRdbLv8y/TZvhq9/Hc47D3bfXduOpG4j77EmhUxm3788MHnY9tmskyZuiqj71tLaxi1PvBb4YLr1ydW9YuTjTkYWM1RQN7l50dF7J046F4UNhawurLiXiDirJqN83Dt80M7UBXfxjafvpX9XJ/f9z0Ru/sLxLOi3A8PvXM20SfWA+Zc2rm/XJFZf98Bo39gZKipBSdV04hZWIFy3ejauPzmf1a5xKFWel2L1z1J4+qy4p5HatFjpUXXiWd/+DlMX3MXxrfcyYEsnLXt+nisP+CqvDtnaBrfQNGprhsaoELe4C6uamxpZuGpdj3Xsxp8DPeUAo1aXBolK3NC8MKHWJf4qV3+yndy05EufFPc04nnTjgkOe1D4Lds9+33IZa/9k51vvYH+WzYHirqXIHdImBUYd3geFH8+Z8Fq5jy1uufaQcIa5c+Na72GlSU8dr8RvXK9lLM/2eZ5seSLUbGOQlPsYh1pFNFIsxCHcSGGt9/mpWkzekT9b3tP4IpPT+HFhmSjhbDqTXEwKQLiUitCt1JGI5041ZBcF05QDvbj9h/ZK09NufuTbQk+iwl5FeuoRtIY8qZxjrD8LDkW6tq18KtfseWKX7Pbpk38dc/Pc+Wnj+GVHXYmUyMMHtiP9o2dsQtiD2+oS0Xw4vS5SylWRjxQ3DbphD1OgXD/CKhShNFOblrypU+KexpDXtNCHGGRHVHZFD98402YPh2uvBI6Onjwfybwy3Ff5uUdtuY77+xW1PfvR+tPDtaeN1MrOT53cARy1A51qbiWoiZ8vfhXZ/rvUVCqXC+6akjug7CSBByC+x9Vd9ViMaFPxrmnEc8bdY6oJE9hS/EHb9zAOQ/dwL9/dxL88pfQ3AxLl3LKpDNzhN3FH6LoT651zKdGcMy+I3otTHr05XWpLDc3WdTk0qVUT/z2qOn39opJv+WJ17T3JWj5v0slTjQGfUZufuI1mxjMkgp9Utzzqc5jeo6oPB1BYjR44wZ+8K8beOTqk/jOk3fyzkGHwPPPw803w+jRRisIdcm17n1mjVYY/SQJD/Tei7CKTIPrMzkZJf1tCnPFuPldgqjEiUaTXDs2t4slKRXvlknqM05j+B52jiifvNeVMXjjBk5+6i98c9E91HVu4p+fmEjNj8/jC5MPzDk2KtNkmP/eJGHX1nPW8LFz7+sJazx2vxE91ZF0eO+FLm85gFLEaouLK97VVFDC9CFaiaMSS+mpaHEv5xSlUT75aZP24Bc3P8LXH7uTbz59D/WbP+TePT/H9RO/TuugYQx/qZuHWp7t5X+96Oi9mTl3ac+k6cBMDQtXrQv1U8dloyetcJdS3PzEawCRAu/tY1Df3SpXUfj96l7xLtVEYyEibUznKipxVGIpPRUdChkWjjht0h69voxQPFEIDWUbORAuvpjOy6+gtmMj94z+LNdPPJ7nth8eGJPuPX7yPo29hDxoktFPULbHONSK8PJFhxntG9b3qOpNbh/LaVKxUGGJJpPqNvzREkZYKGSkuIvIH4AvAW8rpfbKbhsCzAFGASuBKUqp9SIiwOXAYcBG4ASl1NNRDUwq7mFx0HWZ2l7FiZHcBT1pfUHDImK8r/1wv49w+D9vhSuugA8+gGOOgR//GPbc0zhWXLfSMgwBLj1mLLD14dZQn+H9D7fkRNBE4Q1hjLJkda/rYtEV8csEFgvdexMnZl+HjZax5EO+ce43AFcCf/Rsmw48qJSaJSLTs/+fAxwK7J792Q+4Kvu7IOiGtbUivayhIBHLN1dHlFuoxw/9zjtwySVwZFbUp0xxRH3MmJ5zmfpV4wo7bLXqw0YyuvQALt5JUhN3mG4+ohLjt3XvjXu/8nEHVlropqVyMHLLiMgo4B6P5b4cOFAptUZEhgEPKaX2EJHfZf++1b9f2PmTWu664XIc10OcVZreScswC7pnleq6dY6oX3EFvP9+L1H3Wm1R4upSIxBkbIe5ZoJcMu6oBYh0lQCM/9gQvjJuZOi+SVbnmlLK1aWmo6pC9t9iCaIQK1R38gj2m8BO2b8bgdWe/V7Pbusl7iIyFZgKMHLkyNgN8BZadoXW62s3XVRjOlnlf5CECfEHa952Uu26ov6Vrziivtdeic6Xg3IWJXndS5kaoX+/Gj7Y3PuhVpepRaR3hEpHZxcz5y419sM//so6bZk/F9N7HpdST5xH1T51SSOqpZJSJFjKm7zj3JVj+sf2FSilrlFKjVNKjRs6tHe9zTC8iz/AEUY3oqK5qTFwUU2mRpyVmh7ihNCZxCRv3/EeZz18E4/+7iS48EI49FB45hmYMydH2E3OVytCJuDd6caZN3DdJA11GRByhN3tpRt73x5QCBygvaPTeJRj4poXKMiCm1LX9jSN4883qiVq4ZvFEoeklvtbIjLM45Z5O7u9DfAuodw5uy1VorIF6vy6QdtMraIwq2z7jvc46akWTlw0l203d9D2xS8x6JKLegm66fnAeWCFBM70PNBEemd99B8WJz1APijgjDmLmX3/8lQtznJIf+v9XOncgaaGgs46L1UOd0t1klTc5wLfBGZlf//Vs/00EbkNZyJ1Q5S/PQlRX3aT9LlxCRLI7T583xH1hXPZbvNGHtz788h55zFxyhcSnS8uUYuTXMsvKHyykKTpNmlpbYvMBZ8Wpi6RfCaFw1xM5fAQs1QPJqGQtwIHAjsCbwEzgBbgdmAksAonFHJdNhTySuAQnFDIE5VSkTOlcSdUo+LbCx2T7Bf1+/YYz+Xjj+X93f/bOJTNJMY5LUTguP1GBpagKyQNdRkWzzg48fFh9yjt+O9ipdgN++xC8LyFnai16MhrQlUpdazmpYMC9lXAqfGaF5+wJeiFGto2NzWSebedt376C778yJ1st3kj9+4xnl+PP5ZlQ0c5O2UTP7noLNigyWAduugYF5PFSUrBrQtWhwq7gHHEjkttjdAV0rj2jk5aWtsS33fdvEStSOqiWyyXSJh1fukxY6smtYKl9FRk4rCwpF26L09bewfjZ81LNjnV3g4zZ3L4UeP51ryb2O6IQ2HJEn5+wk+3CrsG/8SfbjJYR7dCmyxLgJlHjmHyPo3oU3U5hIlwY0Mdr846nO4Ywj64PsPFX/mEtm0u/r6PnzWPXaffa/Re6N7LbqVS90GHfW7SnNAMS/6WRkI7i8WlYnPL6BZ/hPmyY/uC29vhssucnw0bYPJk+MlPaOnagdn3mYdbug8WXTx7mAWvczW5FYaamxo5/+6leblbNm7eQktrm/E8QF2mlhlHjOl5D1pa2zhjzuLAfb3zIHHDGYtRai6qMAiQatjlhNFDc0Z33u3uNayYW9KgIi33MKJyixuF0LW3w/nnw6hRzu+DDoLFi+GOO2jp2iHH8jbBrRyk0MezB1nw3vBOv0V36TFje0rHrdeEOpqyfmMn5971LBNGD43Myy7AJ0duz8y5Sxk1/V5GTb+X8+9eSn1Q3CZbhThJOGMaeffD8I+idKQZdjl/2dpY2y2WpFSs5a7DG8mg+9Jqow/a2+Hyy+HSSx1L/eij4Sc/gU98omcXk3h3PyZWtS7ZWdRy/rREp6Ozi/nL1jJ5n0ZufXK19iHkFvnwsn5jJ7U1QqZGctI8eIU4SSRIoVMVxHkv04pYsRExlmJRdeIOW4VQF5nQa1i/YcNWUW9vh//9X0fUx47t2SUsX7qLPwWASbZGyLXQdcKlC9MzEYXB9Rkj676tvSOn0EccuroV29VnqO/fL1CIk7pYCummiCOoabmCTO+DXalqyZeqFHeXyMIOAaI+f8opnLeyH2/c1sbwv6/r2Xfan5eEZlDUWd66B4I/oyDQ45ePqrfq9VeH+ckvO2ZszzlM8qPUBKQpiEP7xs6eWq5+yrHIhu7eheWTzxeT+1DqdAuW6qCi87mbEGgB7baNk/flkkscUW9udiZK+UjgFw8UHZ4CFn7C4qFNUtwCgdd185qHJeqaNmkPzpyzOHCE4I2Pjoqr9+er0RE2GomKxy43a1QX217ofPJR9yEsFt7Gu1u8FCJxWMWQM6zfsCFX1I86CmbMgKYmAGbPmhc46RdGVA5y/xyAVxxdi2xgpibwulGLjt5o76C5qTEyUsVlQL+t1xnUv5ZMbQ0bOjoZ3lDHB5u29FR38uMdZUwYPZQ5C1b3GsVkaiXSuvX70N35glIJfKnSD0e5mqxf3pIGVS/uALz7Ls+fewE7X38123W8x8N7HkDXb37MhK8dkrNbki+P1zKOmgwNssjCUghE2dGun7Yxwo8bZKF2KydG3qTu6cVTPpEjRuN2GZJT6m9wfaYnNDKMcnQ3lGPoYTFCQC3VT3WL+7vvwhVXsHn2r9jz3Q088PH9uGz8sSz96MepewEu8q2ejJvvZXB9BnBEa9odS3rcGm3tHUy7YwmQK1ppWl5eP22UH1cXhnjGnMUsXLWOC5r3Dq176he/pIJY7omxysVtVI7zE5bKozrF/d134de/hosvhvXrefK/D+AX/zuF5z768Z5dgkTFNG83OG6IGUc4RTfOv3tpL391Z5fqlSExreyMIuT4+P3uhe3rMojAmdnrh13TXVCjE5SZR47RHdqLKHFMw91QKAEu1KgiSXsrsVqVpfyorglVn6hzxBEwYwa7/vnNQBdHUBUm75cx7M54I1FGhbg0ILfqUVrJwlZqqkclSUjmFr/ORzhNEm9FRexEzV8UMrlXISYxi5WMzNJ3qf4J1XffhSuvdER93boeUWeffQAY/oBhvDu5LoeoDH5u6GIU7ijBFQmvgG7cvCXRClNvQi6vKEP8yiluXHs+/mcTl0vUyCjKWi6kW6cQk5jl7oayVDeVLe7vvbfVUl+3Dr70JUfUx+U+yJL6MHXHTRg9NLZ17IqEX0DD8rKE4Yqg+3c+owFdZaE4mIij2++zb1+iXSgVJn6FjCIpxCSmjXqxlJLKzi1z553wox/BAQfAU0/B3Xf3EnYIzyKpw5+WF89x85etjS2mNSKB2QWbmxp7Jmbj4IpgknQIfvbfbXBex4NeBBXkZIBsbmqMzD6pE7+wjIr5Uog8NoVsr8USRWVb7scdB3vv3eN+CSOOyyGoeLU3RcCZCSztLqU40xOd4mXGEWMSWd9pWYAr3zE7jzcFg78o+bRJe2hX8frdLVETyzrxK2QUSSEmMW3Ui6WUVNeEakrofO1RhTNMEOBSz2Ssi+lErpew6j1eBtdneLdji9YVEjSx7CdsojZTKxzzqRHMeWp16CpXd3LyuGsf75V8zCVqwrFcwhVNqbT2WiqLsAlVK+4eTJKD6TBNEgbRERgmeWBMI3Dc/RauWheYR9ykPSZtMnnwuQ82XbqEWpFeC6YsFoue6o+WyQOvoMcRaC+N2WX5pjVKo9wpQcP5TK0wqH+/nnQBfgvQtQ4HZmrYtKWbbuWI5eR9trqjgtpnkjbApM0mI5rhDXWhhTEKUWGpmrCjAEsc+rS4+10NSYRdIMfqNRF4kzS3EN//q4APO7t7rt+lFHcuamPcLkO0ojqofz8jgch3AZbraw6br7ATjXrKMXWDpbzp024ZE/dHFEGpe11Rru9fywebc90laS5iOa/lWaOHSWNDndaXH7WQy9uvqElff7EOf/bLsBz7urmIpKRl5ZaLtWwzRVqCKJjPXURWAu8BXcAWpdQ4ERkCzAFGASuBKUqp9WHnKXrKX0+yrDQfbUHCXShxOK/lWa0P3Y8Qbnl7xTdsVSWQkzDMj8515L0HDfUZ3v9wS6+HwHH7j+wVRZSUtFaGltMKU91n1WQy3FK9FNrnPkEp9R/P/9OBB5VSs0Rkevb/c1K4TmyihrImroY4fvigBTiFyDrY0tpmLOxAj9DqLG9vorOwVZWPTp/YI9ZBC5E6uxSDBvRj8YytBTtaWttyQiTXb+ykRpyEZLr5g3xJa2VoOa0wtZkiLXEpxCKmo4Abs3/fCDQX4BpGRBVljiqmDcHCHnZM0tjzltY2xs+ax67T781Z9BO0n3d1ahRBRbaD6OxSnHX7YqO6s2ELkfz9nzl3aa/Y924FGzTWfxqktTK0nFaYFrpYuKX6yFfcFfAPEVkkIlOz23ZSSq3J/v0msFOe10hM1JezuamRyfs0EmfxvbtKVSeSSSwpV7Dbsn5xd4QRJPBxVqTWivTKHhnmn+1WaO+Fv1+mqy91LhyV/Qnra1LSWhlaTitMk6yytvRt8hX3zyilPgkcCpwqIp/zvqgch36giSciU0VkoYgsXLt2bZ7NCMbkyzl/2drY8enNTY1Mm7QHmZpcKczUmIUV+okaYXgxtRrrMrWJYsYVvQU+yEKMsiTdkYgJur4mJS0rt9ysZffh/Oqsw3s+hxaLjrx87kqptuzvt0XkL8C+wFsiMkwptUZEhgFva469BrgGnAnVfNqhw2T5d5whdq99/SoYYPaaTKiGjTD8xzfUZ7RZJAd5onM6Ors4+/YlgekOGuoyWosatka4hLU5LFwzSdrhNF0daaUSsHnVLZVM4mgZERkE1Cil3sv+/QDwU+Ag4B3PhOoQpdQPws5VqmgZiBcO6Q07MwlNM4220J1rcH2GDzu7cxcz1QgIOcv8o1ahft0XieKf5AzrQxKShJjakD6LJT6FipbZCfiLOBkT+wF/Ukr9XUSeAm4XkZOAVcCUPK6RN1HRKkHWfW2N0OUTPlOL37v9/LuXGkVb6EYYSvUu0O0Ksj9xV3NTI2ffviSwTbc+uTpH3N1rB4U1mrodwh6aYVb44IBQSDsxaLGkT2JxV0q9AnwiYPs7ONZ7RaAbegdtM6m36i1KrXOf+MVP14aw1Zz+TJXuNt2+Qf32x6Cbuh2Shpi61nm5LAyyWKqZPp1+wEVn3YcJTlAuGX9Rah1RFaBcopKY+UcBrjXvJ6wYR5I4/Kj476i5jkLE/lssllwqu1hHiWhpbePORW05wi6Qk6QrzDVh6oIwicP3XufY/UYE7qPbnhSTEFMbtmexlBZruScgyHJVOGGVLjrXRENdxljkvO4anQXvHQW4fvVbn1xNl1LUinDsfiNSW9bvvWbUaklrnVsspcWKewJMJlN1romZR46JdS2vb9ykqs8FzXunLuZ+gqouJY3xt1gshcGKewJMLVdIL0a67GKuDWL8LRZL6ejTKX+TUk7ZAkuBTT9rsZQHthJTypSdFV1kyimhlsViCcaKe0L64oShG5+uG+vZ9LMWS/lgxd1iRFS+GLvK1GIpL6y4W4wISzXc2MfcUhZLJWDF3WKEzp/uLxBusVjKA7tC1WJEORWusFgs0VhxtxhRboUrLBZLONYtYzGir4d/WiyVhhV3izF9MfzTYqlUrFvGYrFYqhAr7haLxVKFWHG3WCyWKsSKu8VisVQhVtwtFoulCimLlL8ishZY5dm0I/CfEjWnmPSVfkLf6avtZ/VRzn3dRSk1NOiFshB3PyKyUJejuJroK/2EvtNX28/qo1L7at0yFovFUoVYcbdYLJYqpFzF/ZpSN6BI9JV+Qt/pq+1n9VGRfS1Ln7vFYrFY8qNcLXeLxWKx5IEVd4vFYqlCSibuIjJERB4QkZeyvwdr9vu7iLSLyD2+7buKyJMiskJE5ohI/+K0PB4x+vnN7D4vicg3PdsfEpHlIrI4+/OR4rU+GhE5JNu+FSIyPeD1Adn3Z0X2/Rrlee3c7PblIjKpqA1PQNK+isgoEenwvIdXF73xMTDo5+dE5GkR2SIiX/a9Fvg5Lkfy7GeX5/2cW7xWx0ApVZIf4JfA9Ozf04FfaPY7CDgCuMe3/Xbgq9m/rwb+X6n6km8/gSHAK9nfg7N/D86+9hAwrtT90PStFngZ2A3oDywB9vTt813g6uzfXwXmZP/eM7v/AGDX7HlqS92nAvV1FPBcqfuQYj9HAf8D/BH4sme79nNcbj/59DP72vul7kPUTyndMkcBN2b/vhFoDtpJKfUg8J53m4gIMBG4I+r4MsCkn5OAB5RS65RS64EHgEOK07y82BdYoZR6RSm1GbgNp79evP2/Azgo+/4dBdymlNqklHoVWJE9X7mST18rich+KqVWKqWeAbp9x1bS5zifflYEpRT3nZRSa7J/vwnsFOPYHYB2pdSW7P+vA+VaRcKkn43Aas///v5cnx3+/bjMxCKq3Tn7ZN+vDTjvn8mx5UQ+fQXYVURaReRfIvLZQjc2D/J5XyrpPc23rQNFZKGIPCEizam2LCUKWolJRP4JfDTgpR95/1FKKRGp2JjMAvfzOKVUm4hsC9wJHI8zTLRUDmuAkUqpd0RkH6BFRMYopd4tdcMsidkl+73cDZgnIs8qpV4udaO8FFTclVJf0L0mIm+JyDCl1BoRGQa8HePU7wANItIvayHtDLTl2dzEpNDPNuBAz/874/jaUUq1ZX+/JyJ/whlOlou4twEjPP8HvQ/uPq+LSD9ge5z3z+TYciJxX5XjpN0EoJRaJCIvA/8FLCx4q+OTz/ui/RyXIXl9/jzfy1dE5CGgCceHXzaU0i0zF3Bn078J/NX0wOyXZT7gzmDHOr7ImPTzfuBgERmcjaY5GLhfRPqJyI4AIpIBvgQ8V4Q2m/IUsHs2cqk/ziSiP3LA2/8vA/Oy799c4KvZCJNdgd2BBUVqdxIS91VEhopILUDW0tsdZ7KxHDHpp47Az3GB2pkvifuZ7d+A7N87AuOB5wvW0qSUcLZ6B+BB4CXgn8CQ7PZxwO89+/0bWAt04PjFJmW374YjBiuAPwMDSj07nWc/v5XtywrgxOy2QcAi4BlgKXA5ZRZRAhwGvIhjtfwou+2nwJHZvwdm358V2fdrN8+xP8oetxw4tNR9KVRfgcnZ928x8DRwRKn7kmc/P5X9Ln6AMwpbGvY5LtefpP0EDgCexYmweRY4qdR9Cfqx6QcsFoulCrErVC0Wi6UKseJusVgsVYgVd4vFYqlCrLhbLBZLFWLF3WKxWKoQK+4Wi8VShVhxt1gslirk/wOoz+qScO0jTQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn import linear_model\n", "from sklearn import datasets\n", "import numpy as np\n", "\n", "# load data\n", "d = datasets.load_diabetes()\n", "\n", "X = d.data[:, np.newaxis, 2]\n", "Y = d.target\n", "print(\"X: \", X.shape)\n", "print(\"Y: \", Y.shape)\n", "\n", "# create regression model\n", "regr = linear_model.LinearRegression()\n", "regr.fit(X, Y)\n", "\n", "a, b = regr.coef_, regr.intercept_\n", "print(\"a = %f, b = %f\" % (a, b))\n", "\n", "x_min = np.min(X)\n", "x_max = np.max(X)\n", "y_min = a * x_min + b\n", "y_max = a * x_max + b\n", "\n", "plt.scatter(X, Y)\n", "plt.plot([x_min, x_max], [y_min, y_max], 'r')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 7. 如何使用sklearn拟合多项式函数?" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([800., 90., -20.])" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Fitting polynomial functions\n", "\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.linear_model import LinearRegression\n", "from sklearn.pipeline import Pipeline\n", "\n", "t = np.array([2, 4, 6, 8])\n", "\n", "pa = -20\n", "pb = 90\n", "pc = 800\n", "\n", "y = pa*t**2 + pb*t + pc\n", "\n", "model = Pipeline([('poly', PolynomialFeatures(degree=2)),\n", " ('linear', LinearRegression(fit_intercept=False))])\n", "model = model.fit(t[:, np.newaxis], y)\n", "model.named_steps['linear'].coef_\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 2 }