Lhy_Machine_Learning/README.md

李宏毅2021/2022春季机器学习课程课件及作业
===========================

[![BILIBILI](https://raw.githubusercontent.com/Fafa-DL/readme-data/main/Bilibili.png)](https://space.bilibili.com/46880349)

****

## 重磅须知
```
（重磅须知，统一说明）为方便所有网课资料与优质电子书籍的实时更新维护，创建了一个在线实时网盘文件夹，放在公众号【啥都会一点的研究生】，本节课对应序号【05】。

UP将2021&2022所有作业的数据资料整理打包好了，由于文件太大，已同步放在上述所提在线网盘。

在线网盘能满足该课程所需资料的全部需求，链接挂掉也会及时更新，祝大家学习顺利。

2022仅在2021基础上进行小补充，2021内容变成了前置知识，UP会在视频标题打上2022的标签；

ppt/pdf支持直链下载。
```

## 更新日志

|日期|项目|
|---|---
|2021/03/16|更新HW1、HW2，同步更新助教范例|
|2021/03/26|更新HW3、HW4课件、代码、范例；release页发布HW1-HW4数据|
|2021/04/01|更新选修内容To Learn More，基本是李老师今年不打算讲而以前讲过的知识点(旧视频)|
|2021/04/09|更新GAN 及 HW05|
|2021/04/16|更新Self-Supervised Learning 及 HW06|
|2021/04/30|更新Explainable AI&Adversarial Attack 及 HW07&HW08|
|2021/05/06|更新Domain Adaptation 及 HW09&HW10|
|2021/05/21|更新RL 及 HW11|
|2021/05/28|更新Quantum ML|
|2021/06/04|更新Life-Long&Compression 及 HW12|
|2021/06/11|更新Meta Learning 及 HW13&HW14|
|2021/06/18|更新HW15，随着李老师课程结语视频上传，2021机器学习基本结束啦|
|2021/12/20|更新Github排版，删除repo中的ppt/pdf直接提供下载链接，总资料放入[百度云盘-提取码：sv1i](https://pan.baidu.com/s/13cxyIbvF0bEyytANLf58NQ)|
|2022/02/17|2022春季机器学习课程仅在21基础上进行小补充，UP同步更新官网补充内容|
|2022/02/21|更新Lecture 1:Introductionof Deep Learning补充内容，Github排版大更新|
|2022/02/25|更新Lecture 2:What to do if my network fails to train补充内容与HW2|
|2022/03/05|更新Lecture 3:Images input，HW3|
|2022/03/13|更新Lecture 4 Sequence as input，HW4<br>UP将2021&2022所有作业的数据资料整理打包好放在公众号【啥都会一点的研究生】|
|2022/03/18|更新Lecture 5 Sequence to sequence，HW5，相应Data放在公众号维护的网盘中|
|2022/04/05|更新Lecture 7以及HW6|
|2022/04/16|更新HW7|
|2022/04/23|更新HW8|
|2022/04/30|更新Lecture 9 & Lecture10，HW9|
|2022/05/06|更新HW10|
|2022/05/13|更新HW11，部分Lecture10教学视频|

****
	
|B站主页|[啥都会一点的研究生](https://space.bilibili.com/46880349)|
|---|---|
|人工智能技术探讨群1|[78174903](https://jq.qq.com/?_wv=1027&k=lY5KVICA)|
|人工智能技术探讨群2|[571218304](https://jq.qq.com/?_wv=1027&k=ZCDCT3xV)|
|人工智能技术探讨群3|[584723646](https://jq.qq.com/?_wv=1027&k=bakez5Yz)|

****

|名称|项目|
|---|---|
|2021课程主页|[李宏毅2021春季机器学习](https://speech.ee.ntu.edu.tw/~hylee/ml/2021-spring.html)|
|2022课程主页|[李宏毅2022春季机器学习](https://speech.ee.ntu.edu.tw/~hylee/ml/2022-spring.php)|
|B站视频合集|[(强推)李宏毅2021春机器学习课程](https://www.bilibili.com/video/BV1Wv411h7kN)|
|2021视频及资料目录|[链接](https://github.com/Fafa-DL/Lhy_Machine_Learning/tree/main/2021%20ML)
|2022视频及资料目录|[链接](https://github.com/Fafa-DL/Lhy_Machine_Learning/tree/main/2022%20ML)

****

|章节|2021前置知识|2022补充|选修|作业|
|---|---|---|---|---|
|Lecture 1|[（上）机器学习基本概念简介](https://www.bilibili.com/video/BV1Wv411h7kN?p=3)<br/>[（下）机器学习基本概念简介](https://www.bilibili.com/video/BV1Wv411h7kN?p=4)|Video:<br/>[2022-机器学习相关规定](https://www.bilibili.com/video/BV1Wv411h7kN?p=1)<br/>[2022-Colab教学](https://www.bilibili.com/video/BV1Wv411h7kN?p=5)<br/>[2022-Pytorch Tutorial 1](https://www.bilibili.com/video/BV1Wv411h7kN?p=6)<br/>[2022-Pytorch Tutorial 2](https://www.bilibili.com/video/BV1Wv411h7kN?p=7)<br/><br/>PDF:<br/>[Rules](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/rule%20(v2).pdf)<br/>[Chinese class course intro](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/introduction%20(v2).pdf)<br/>[Pytorch Tutorial 1](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Pytorch%20Tutorial%201.pdf)<br/>[Pytorch Tutorial 2](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Pytorch%20Tutorial%202.pdf)<br/>[Colab Tutorial](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Colab%20Tutorial%202022.pdf)<br/>[Environment Setup](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/EnvironmentSetup.pdf)|[深度学习简介](https://www.bilibili.com/video/BV1Wv411h7kN?p=13)<br/>[反向传播](https://www.bilibili.com/video/BV1Wv411h7kN?p=14)<br/>[预测-宝可梦](https://www.bilibili.com/video/BV1Wv411h7kN?p=15)<br/>[分类-宝可梦](https://www.bilibili.com/video/BV1Wv411h7kN?p=16)<br/>[逻辑回归](https://www.bilibili.com/video/BV1Wv411h7kN?p=17)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=11)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/HW01.pdf)<br/>[Code](https://colab.research.google.com/drive/1FTcG6CE-HILnvFztEFKdauMlPKfQvm5Z#scrollTo=YdttVRkAfu2t)<br/>[Submission](https://www.kaggle.com/t/a3ebd5b5542f0f55e828d4f00de8e59a)|
|Lecture 2|[（一）局部最小值 (local minima) 与鞍点 (saddle point)](https://www.bilibili.com/video/BV1Wv411h7kN?p=19)<br/>[（二）批次 (batch) 与动量 (momentum)](https://www.bilibili.com/video/BV1Wv411h7kN?p=20)<br/>[（三）自动调整学习率 (Learning Rate)](https://www.bilibili.com/video/BV1Wv411h7kN?p=21)<br/>[（四）损失函数 (Loss) 也可能有影响](https://www.bilibili.com/video/BV1Wv411h7kN?p=22)|Video:<br/>[2022-再探宝可梦、数码宝贝分类器 — 浅谈机器学习原理](https://www.bilibili.com/video/BV1Wv411h7kN?p=23)<br/><br/>PDF:<br/>[Theory](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/theory%20(v7).pdf)|[Gradient Descent (Demo by AOE)](https://www.bilibili.com/video/BV1Wv411h7kN?p=24)<br/>[ Beyond Adam (part 1)](https://www.bilibili.com/video/BV1Wv411h7kN?p=26)<br/>[ Beyond Adam (part 2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=27)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=28)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/hw2_slides%202022.pdf)<br/>[Code](https://colab.research.google.com/drive/1hmTFJ8hdcnqRz_0oJSXjTGhZLVU-bS1a?usp=sharing)<br/>[Submission](https://www.kaggle.com/c/ml2022spring-hw2)|
|Lecture 3|[卷积神经网络CNN](https://www.bilibili.com/video/BV1Wv411h7kN?p=31)|Video:<br/>[为什么用了验证集还是过拟合](https://www.bilibili.com/video/BV1Wv411h7kN?p=32)<br/>[鱼与熊掌可以兼得的机器学习](https://www.bilibili.com/video/BV1Wv411h7kN?p=33)<br/><br/>PDF:<br/>[Validation](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/validation.pdf)<br/>[Why Deep](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/whydeep%20(v3).pdf)|[Spatial Transformer Layer](https://www.bilibili.com/video/BV1Wv411h7kN?p=34)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=35)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Machine%20Learning%20HW3%20-%20Image%20Classification.pdf)<br/>[Code](https://colab.research.google.com/drive/15hMu9YiYjE_6HY99UXon2vKGk2KwugWu)<br/>[Submission](https://www.kaggle.com/c/ml2022spring-hw3b)|
|Lecture 4|[自注意力机制(Self-attention)（上）](https://www.bilibili.com/video/BV1Wv411h7kN?p=41)<br/>[自注意力机制(Self-attention)（下）](https://www.bilibili.com/video/BV1Wv411h7kN?p=42)|Video:<br/>[None]<br/><br/>PDF:<br/>[None]|[RNN(part 1)](https://www.bilibili.com/video/BV1Wv411h7kN?p=40)<br/>[RNN(part 2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=41)<br/>[GNN(part 1)](https://www.bilibili.com/video/BV1Wv411h7kN?p=42)<br/>[GNN(part 2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=43)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=45)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Machine%20Learning%20HW4.pdf)<br/>[Code](https://colab.research.google.com/drive/1gC2Gojv9ov9MUQ1a1WDpVBD6FOcLZsog?usp=sharing)<br/>[Submission](https://www.kaggle.com/c/ml2022spring-hw4)|
|Lecture 5|[类神经网络训练不起来怎么办（五）批次标准化](https://www.bilibili.com/video/BV1Wv411h7kN?p=48)<br/>[Transformer（上）](https://www.bilibili.com/video/BV1Wv411h7kN?p=49)<br/>[Transformer（下）](https://www.bilibili.com/video/BV1Wv411h7kN?p=50)|Video:<br/>[各式各样神奇的自注意力机制 (Self-attention) 变型](https://www.bilibili.com/video/BV1Wv411h7kN?p=51)<br/><br/>PDF:<br/>[xformer](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/xformer%20(v8).pdf)|[NAT model](https://www.bilibili.com/video/BV1Wv411h7kN?p=52)<br/>[Pointer network](https://www.bilibili.com/video/BV1Wv411h7kN?p=53)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=54)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/HW05.pdf)<br/>[Code](https://colab.research.google.com/drive/1Tlyk2vCBQ8ZCuDQcCSEWTLzr1_xYF9CL#scrollTo=Le4RFWXxjmm0)<br/>[Submission](https://ml.ee.ntu.edu.tw/hw5/)|
|Lecture 6|[GAN（一）基本概念介绍](https://www.bilibili.com/video/BV1Wv411h7kN?p=58)<br/>[GAN（二）理论介绍与WGAN](https://www.bilibili.com/video/BV1Wv411h7kN?p=59)<br/>[GAN（三）生成器效能评估与条件式生成](https://www.bilibili.com/video/BV1Wv411h7kN?p=60)<br/>[GAN（四）Cycle GAN](https://www.bilibili.com/video/BV1Wv411h7kN?p=61)|Video:<br/>[None]<br/><br/>PDF:<br/>[None]|[Theory of GAN (part 1)](https://www.bilibili.com/video/BV1Wv411h7kN?p=62)<br/>[Theory of GAN (part 2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=63)<br/>[Theory of GAN (part 3)](https://www.bilibili.com/video/BV1Wv411h7kN?p=64)<br/>[Deep Generative Model (part 1)](https://www.bilibili.com/video/BV1Wv411h7kN?p=65)<br/>[Deep Generative Model (part 2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=66)<br/>[FLOW-based Model](https://www.bilibili.com/video/BV1Wv411h7kN?p=67)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=70)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Machine%20Learning%20HW6.pdf)<br/>[Code](https://colab.research.google.com/drive/10lHBPFoNhTiiPe-yZ7SwAV1wwrkGc4En?usp=sharing)<br/>|
|Lecture 7|[自监督学习（一）芝麻街与进击的巨人](https://www.bilibili.com/video/BV1Wv411h7kN?p=71)<br/>[自监督学习（二）BERT简介](https://www.bilibili.com/video/BV1Wv411h7kN?p=72)<br/>[自监督学习（三）BERT的奇闻轶事](https://www.bilibili.com/video/BV1Wv411h7kN?p=73)<br/>[自监督学习（四）GPT的野望](https://www.bilibili.com/video/BV1Wv411h7kN?p=74)|Video:<br/>[如何有效的使用自督导式模型](https://www.bilibili.com/video/BV1Wv411h7kN?p=75)<br/>[语音与影像上的神奇自督导式学习模型](https://www.bilibili.com/video/BV1Wv411h7kN?p=76)<br/><br/>PDF:<br/>[Recent Advance of Self-supervied learning for NLP](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/PLM.pdf)<br/>[SSL for Speech and Image](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/SSL_speech_image%20(v9).pdf)|[BERT (part 1)](https://www.bilibili.com/video/BV1Wv411h7kN?p=77)<br/>[BERT (part 2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=78)<br/>[GPT-3](https://www.bilibili.com/video/BV1Wv411h7kN?p=80)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=82)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/hw7_slides.pdf)<br/>[Code](https://colab.research.google.com/drive/1QloQ42zYwX_ETAs2GIkeh8qFj0UjHXfH?usp=sharing)<br/>[Submission](https://www.kaggle.com/c/ml2022spring-hw7)|
|Lecture 8|[自编码器 (Auto-encoder) (上) – 基本概念](https://www.bilibili.com/video/BV1Wv411h7kN?p=83)<br/>[自编码器 (Auto-encoder) (下) – 领结变声器与更多应用](https://www.bilibili.com/video/BV1Wv411h7kN?p=84)<br/>[Anomaly Detection (1_7)](https://www.bilibili.com/video/BV1Wv411h7kN?p=85)<br/>[Anomaly Detection (2_7)](https://www.bilibili.com/video/BV1Wv411h7kN?p=86)<br/>[Anomaly Detection (3_7)](https://www.bilibili.com/video/BV1Wv411h7kN?p=87)<br/>[Anomaly Detection (4_7)](https://www.bilibili.com/video/BV1Wv411h7kN?p=88)<br/>[Anomaly Detection (5_7)](https://www.bilibili.com/video/BV1Wv411h7kN?p=89)<br/>[Anomaly Detection (6_7)](https://www.bilibili.com/video/BV1Wv411h7kN?p=90)<br/>[Anomaly Detection (7_7)](https://www.bilibili.com/video/BV1Wv411h7kN?p=91)|Video:<br/>[None]<br/><br/>PDF:<br/>[None]<br/>|[PCA](https://www.bilibili.com/video/BV1Wv411h7kN?p=92)<br/>[t-SNE](https://www.bilibili.com/video/BV1Wv411h7kN?p=93)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=95)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Machine%20Learning%20Homework%208%20Anomaly%20Detection.pdf)<br/>[Code](https://colab.research.google.com/drive/16J23Uqkclro8zvp5Y1EXFtEWOvMA9YXC#scrollTo=JoW1UrrxgI_U)<br/>[Submission](https://www.kaggle.com/competitions/ml2022spring-hw8/)|
|Lecture 9|[机器学习的可解释性 (上) – 为什么神经网络可以正确分辨宝可梦和数码宝贝](https://www.bilibili.com/video/BV1Wv411h7kN?p=96)<br/>[机器学习的可解释性  (下) –机器心中的猫长什么样子](https://www.bilibili.com/video/BV1Wv411h7kN?p=77)|Video:<br/>[自然语言处理上的对抗式攻击 Part1](https://www.bilibili.com/video/BV1Wv411h7kN?p=98)<br/><br/>PDF:<br/>[Adversarial Attack for NLP](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Attacks-in-NLP-Draft.pdf)<br/>|[None]|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=100)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Machine%20Learning%20HW9.pdf)<br/>[Code](https://colab.research.google.com/drive/1DvQX9apokZHZNfZeG7brJS6XbsGpPFYU?usp=sharing)|
|Lecture 10|[来自人类的恶意攻击 (Adversarial Attack) (上) – 基本概念](https://www.bilibili.com/video/BV1Wv411h7kN?p=101)<br/>[2021 - 来自人类的恶意攻击 (Adversarial Attack) (下) – 类神经网络能否躲过人类深不见底的恶意](https://www.bilibili.com/video/BV1Wv411h7kN?p=102)|Video:<br/>[自然语言处理上的对抗式攻击 Part2](https://www.bilibili.com/video/BV1Wv411h7kN?p=103)<br/>[自然语言处理上的对抗式攻击 Part3](https://www.bilibili.com/video/BV1Wv411h7kN?p=104)<br/>[自然语言处理上的模仿攻击 (Imitation Attack) 以及后门攻击 (Backdoor Attack)](https://www.bilibili.com/video/BV1Wv411h7kN?p=105)<br/><br/>PDF:<br/>[Adversarial Attack for NLP](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Attacks-in-NLP-Draft.pdf)|[More about Adversarial Attack (1_2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=106)<br/>[More about Adversarial Attack (2_2)](https://www.bilibili.com/video/BV1Wv411h7kN?p=107)|[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=109)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/Machine%20Learning%20HW10%20Adversarial%20Attack.pdf)<br/>[Code](https://colab.research.google.com/drive/1BcYS4bAInDCTo4Ilsc6w4o_SEoFTuHSO?usp=sharing)<br/>[Submission](https://ml.ee.ntu.edu.tw/hw10/)|
|Lecture 11|[ 概述领域自适应 (Domain Adaptation)](https://www.bilibili.com/video/BV1Wv411h7kN?p=110)|Video:<br/>[None]<br/><br/>PDF:<br/>[More about self-supervised learning](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/More%20self-supervised%20(v2).pdf)||[Video](https://www.bilibili.com/video/BV1Wv411h7kN?p=112)<br/>[Slide](https://speech.ee.ntu.edu.tw/~hylee/ml/ml2022-course-data/hw11_slides%20(ML2022).pdf)<br/>[Code](https://colab.research.google.com/drive/1N2MceW-Z8olOSQpF-TGvqBq83tRdMOzu)<br/>[Submission](https://www.kaggle.com/competitions/ml2022-spring-hw11)|

****
[![BILIBILI](https://raw.githubusercontent.com/Fafa-DL/readme-data/main/gzh.jpg)](https://space.bilibili.com/46880349)