Lecture Note : About this course (PPTX, PDF) 

Homework :  Create your web site

Readings :

• 十三分鐘略懂 AI 技術：機器學習、深度學習技術原理及延伸應用 Youtube 13'27", 2021

• 機器學習案例：Alpha Go 2017

• 為何機器學習需要數學基礎：The Mathematics of Machine Learning

• 為何機器學習需要Python : Python、R、Java、 C++ ：從業界反饋看機器學習語言趨勢

• Deep Learning in Self-Driving Cars, 2021

• Football Games Analysis from video stream with Machine Learning, 2021

• Artificial Intelligence, Machine Learning, and Deep Learning — What the Difference? 2020

• Understanding the differences between AI, machine learning, and deep learning, 2017

• What's the Difference Between Artificial Intelligence, Machine Learning, and Deep Learning? nVidia, 2016

Lecture Note : Introduction to Machine Learning (PPTX, PDF)

Homework :  Python (I)

Readings :

• Contemporary Classification of Machine Learning Techniques (Part 1), 2019

• The history of machine learning, 2019

• Main concepts behind Machine Learning, 2018

• Machine Learning: A Primer - an introduction for both technical and non-technical readers, 2018

• Cousins of Artificial Intelligence, 2018

• A casual intro to Machine Learning, 2017

• A Friendly Introduction to Machine Learning, Udacity 2016. Youtube video 30'52"

• How Machine Learning Works, As Explained by Google, 2015

• Chapter 1 Introduction, Introduction to Machine Learning, 2nd, E. Alpaydin, MIT Press, 2010

• Machine Learning - Wikipedia

Lecture Note : Machine Learning for Pattern Recognition (PPTX, PDF)

Homework : Python (III)

Readings :

• Machine Learning for Image Recognition, News 2017.

• Machine Learning in Computer Vision, Applied Artificial Intelligence, Vol. 15, pp. 693-705, 2001.

• Classification: Probabilistic Generative Model (pdf, video) Machine Learning課程，台大電機系李宏毅教授，2017.

Lecture Note : Machine Learning Problems and Algorithms (PPTX, PDF)

Homework : P1 Scikit-Learn

Readings :

• Part 1 Overview, Goals, Learning Types, and Algorithms , Machine Learning: An In-Depth, Non-Technical Guide. 2017.

• Top 10 Machine Learning Algorithms, 2016

• 關於 Machine Learning 與 Statistical Modeling 的那些事, 2016.

• Thinking about Data, Victor Lavrenko, 2015. Lectures 2 & 3 of the Introductory Applied Machine Learning (IAML) course at the University of Edinburgh (25 部Youtube影片約80分鐘)

• Chapter 1 Introduction, Introduction to Machine Learning, 2nd, E. Alpaydin, MIT Press, 2010.

• Chapter 1 Introduction, Machine Learning: An Algorithmic Perspective, S. Marsland, Chapman and Hall/CRC, 2009.

Lecture Note : Supervised Learning (PPTX, PDF)

Homework : P2 Trees and Forests

Readings :

• Chapter 2 Supervised Learning, 2nd, E. Alpaydin, MIT Press, 2010.

• Chapters 1-7, Machine Learning: An Algorithmic Perspective, S. Marsland, Chapman and Hall/CRC, 2009.

• Part 1 Classification: Chapters 1-7, Machine Learning in Action. Peter Harrington, Manning Publications Co., 2012.

• Supervised Learning - Wikipedia

• Comparing supervised learning algorithms, 2015

• Predictive modeling, supervised machine learning, and pattern classification - the big picture, 2014

• Supervised learning superstitions cheat sheet

• KNN (K-Nearest Neighbor)

  • Create a K-Nearest Neighbors Algorithm from Scratch in Python, 2022

• Decision Tree

  • Implications of Information Theory in Machine Learning, 2021

  • Decision Tree in Machine Learning, Towards Data Science, 2018

• SVM (Support Vector Machine)

  • 支持向量机，维基百科。

  • 机器学习与数据挖掘-支持向量机(SVM)

  • Support Vector Machine簡介

  • LIBSVM -- A Library for Support Vector Machines, NTU.

  • Support Vector Machines (SVM) and the Multi-Dimensional Wizardry, 2022

  • Understand Support Vector Machines: part 1, part 2, 2021

  • An Introduction to Support Vector Machine, 2020

  • Machine Learning Youtube channel, StatQuest with Josh Starmer, 2019

    • Support Vector Machines, Clearly Explained!!!  2019. Youtube 20’31"

    • Support Vector Machines Part 2: The Polynomial Kernel  2019

    • Support Vector Machines Part 3: The Radial (RBF) Kernel  2019

  • Support Vector Machine (SVM): A Visual Simple Explanation — Part 1, 2019

  • Support Vector Machine (SVM)  pdf,video,   Machine Learning課程，李宏毅，台大電機系，2016.  

  • SVM, Xianxu Hou's blog, 2015

Lecture Note : Linear Regression (PPTX, PDF)

Homework : P4 SVM

Readings :

• Linear Regression - Wikipedia

• A Beginner’s Guide to Regression Analysis in Machine Learning, 2021

• Fundamentals of Linear Regression for Machine Learning, 2021

• 線性迴歸的運作原理，資料科學・機器・人, 2017。 Source : How linear regression works, Data Science and Robots Blog, 2016. Youtube video.

• Algorithms From Scratch: Linear Regression - Detailing and Building a Linear Regression model from scratch, Kurtis Pykes, 2020. Towards Data Science.    (Scikit-Learn)

• Least Squares Linear Regression In Python, 2019

• Simple and Multiple Linear Regression in Python，2017

• An Introduction to Gradient Descent and Linear Regression, 2014. (Python)

• Chapter 8 Predicting numeric values: regression, Machine Learning in Action. by Peter Harrington, Manning Publications Co., 2012

• Lecture Series: Linear Regression, 3 部Youtube影片 約50分鐘, 2016

• Lesson 1: Simple Linear Regression, STAT 501, Eberly College of Science, The Pennsylvania State University.

• Regression: Case Study, pdf, video, Machine Learning，李宏毅，台大電機，2016

• Gradient descent

  • What Is a Gradient in Machine Learning? 2021

  • How to Implement Gradient Descent Optimization from Scratch, 2021

  • Understanding Gradient Descent And Its Variants, Medium, 2019.

• Gradient descent tutorials

  • A Gentle Introduction To Partial Derivatives and Gradient Vectors, 2021

  • A Gentle Introduction To Gradient Descent Procedure, 2021

  • Higher-Order Derivatives, 2021

• Gradient descent implementation

  • Implementing Gradient Descent in Python from Scratch, 2022

  • How to Implement Gradient Descent Optimization from Scratch, 2021

  • How to implement Gradient Descent in Python, 2019

Lecture Note : Logistic Regression (PPTX, PDF)

Homework :  P5 Linear Regression

Readings :

• Guide to an in-depth understanding of logistic regression, 2016

  • An Intuitive Guide To Exponential Functions & e

  • Demystifying the Natural Logarithm (ln)

• D.G. Kleinbaum and M. Klein, Logistic Regression: A Self-Learning Text, 3rd, 2010.

• Chapter 5 Logistic regression, Machine Learning in Action, by Peter Harrington, Manning Publications Co., 2012  (PDF)

• Logistic regression, Wikipedia

• Breaking it Down: Logistic Regression, 2022

  • Exploring the fundamentals of logistic regression with NumPy, TensorFlow, and the UCI Heart Disease Dataset

• Logistic Regression From Scratch With Python, 2021

• Logistic Regression With Gradient Descent in Excel, 2020

  • So you can better understand how Logistic Regression works

• Logistic Regression for Machine Learning, 2020

• Logistic Regression - A gentle introduction to Logistic Regression, Towards Data Science, 2020

• Intuitively, How Can We (Better) Understand Logistic Regression, 2020

• How To Implement Logistic Regression From Scratch in Python, 2019

• Logistic Regression Tutorial for Machine Learning, 2019

• 👍 StatQuest: Logistic Regression, Youtube channel, StatQuest with Josh Starmer, 2018-2020

• Logistic Regression with Python, 2019

• Machine Learning From Scratch: Logistic Regression. Classification Fundamentals in Python.  2018

• Practical aspects — Logistic Regression in layman terms, 2018

• Classification: Logistic Regression pdf, pptx, video, Machine Learning課程，李宏毅，台大電機，2017.

Lecture Note : Logistic Regression (PPTX, PDF)

Homework :  P6 Logistic Regression

Readings :

• Essential guide to Multi-Class and Multi-Output Algorithms in Python, 2021

• Logistic and Softmax Regression, Xianxu Hou's blog, 2015

• Deep Learning Tutorial: Linear Regression, Logistic Regression, Softmax regression.

• What is Softmax Regression and How is it Related to Logistic Regression? 2016

• Multiclass classification, Wikipedia

• 一步一步学习Deep Learning (2) — Softmax Regression (Matlab)，2015

• Softmax Regression, Logistic Regression, Stanford UFLDL(Unsupervised Feature Learning and Deep Learning) Tutorial

Lecture Note : Neural Information Processing (PPTX, PDF)

Homework : P7 Clustering

Readings :

• Every Machine Learning Algorithm Can Be Represented as a Neural Network. The Algorithm of Algorithms. Medium, 2020

• A Beginners Guide to Neural Nets - Towards Data Science, 2020

• Neural Networks are Function Approximation Algorithms, by Jason Brownlee, 2020

• A Comprehensive Guide on Activation Functions - Taking a modern tour over the many curves of data-science, 2020

• Introduction to Artificial Neural Networks(ANN), 2019

• Neural Network Activation Function Types - Understanding what really happens in a neural network, 2019

• Understanding Neural Network Neurons - Explaining What These Smart Components Doing?  2019

• The Perceptron — A Building Block of Neural Networks - AND HOW IT CAN COMPUTE LOGICAL STATEMENTS, 2019

• Introduction To Artificial Intelligence — Neural Networks, 2019

• Machine Learning for Beginners: An Introduction to Neural Networks, 2019

• Neural Networks — An Intuition, 2018

• The differences between Artificial and Biological Neural Networks, 2018

• Demystifying Neural Networks: A Mathematical Approach (Part 1), 2018

• The Scuffle Between Two Algorithms -Neural Network vs. Support Vector Machine, 2018

• Introduction to Artificial Neural Networks : Part 1, Part 2 - Learning, 2014

• Machine Learning: An Algorithmic Perspective, S. Marsland, Chapman and Hall/CRC, 2009

  • Chapter 2 Linear Discriminants (PDF)

  • Chapter 3 The Multi-Layer Perceptron (PDF)

• Chapter 11 Multilayer Perceptrons, Introduction to Machine Learning, 2nd, E. Alpaydin, MIT Press, 2010   (PDF)

Lecture Note : Backpropagation (PPTX, PDF)

Homework : P8 PyTorch and Neueal Networks

Readings

• Neural Networks Backpropagation Made Easy, 2021

• Neural Networks Fundamentals, 2021

• Do You Understand Gradient Descent and Backpropagation? Most Don’t. 2020

• Machine Learning - Neural Networks Tutorial, Paul Tero of Existor Ltd, 2015

• Backpropagation in 5 Minutes, Youtube 5'28"  2017   (中文字幕)

• Neural Networks, Youtube videos (中文字幕)

  • 1. What is Neural Network (19’13")

  • 2. How Neural Networks Learn (21’01")

  • 3. What Is Backpropagation really doing? (13’54")

  • 4. Backpropagation Calculus (10’18")

• Backpropagation單元 pdf, video,  Machine Learning課程，李宏毅，台大電機，2017

• Chapter 3 The Multi-Layer Perceptron (PDF), Machine Learning: An Algorithmic Perspective, S. Marsland, Chapman and Hall/CRC, 2009

• Neural Networks & The Backpropagation Algorithm, Explained, 2016

• 資料科學・機器・人, 2017：神經網路的運作原理，反向傳播的運作原理。

  • How Deep Neural Networks Work, B. Rohrer, 2017. Youtube 24'37". Blog. How neural networks work.

• Backpropagation - Wikipedia

Readings Advanced

• A Gentle Introduction To Math Behind Neural Networks - Math Concepts Behind Neural Networks From Scratch, 2019

• Understanding the math behind Neural Networks, 2019

• Introduction to Neural Networks - A detailed overview of neural networks with a wealth of examples and simple imagery, 2019

• Calculating the Backpropagation of a Network - A beginner’s guide to the math behind the backpropagation algorithm, 2019

• Understanding the 3 Primary Types of Gradient Descent, 2018

• Why momentum really works, Distill, 2017

• The Evolution of Gradient Descent, Youtube 9'18", 2017. (此英文的簡短介紹不僅有gradient descent，還有其他深度神經網路常用的方法)

Lecture Note : Deep Learning Introduction (PPTX, PDF)

Homework : P9 Learning of Neural Networks. Project Announcement

Readings

• Neural network (backpropagation) playground by TensorFlow (1 hidden layer, activation=tanh, batch size=10)

• Deep Learning State of the Art 2020, MIT course, Youtube video: 1:27:41, 2020

• Intuitive Deep Learning : Introduction, 2019

  • Part 1a: Introduction to Neural Networks - What is Deep Learning? A very gentle and intuitive introduction to Neural Networks and how they work! 

  • Part 1b: Introduction to Neural Networks - How do we make neural networks work? Some nitty-gritty details to take note of! 

  • Part 2: CNNs for Computer Vision - What are Convolutional Neural Networks? How can we apply Neural Networks to recognize images?  

• Loss Functions Explained, 2019

• Gradient Descent in Deep Learning, 2019

• Intro to Deep Learning - Neural networks for newbies, novices, and neophytes, 2019

• The Most Intuitive and Easiest Guide for Artificial Neural Network - Demystifying neural networks for complete starters, 2019

• Understanding Deep Learning through Energy Landscapes, 2018

• What and why NN. CNN, RNN, applications Part 4: Neural Networks & Deep Learning, Machine Learning for Humans, by V. Maini, Medium, 2017

• 資料科學・機器・人, 2017：解密深度學習，卷積神經網路的運作原理。

• Machine Learning and Deep Learning : Differences, 2018

• Deep Learning模型最近若干年的重要进展，2018

• Deep Learning for Computer Vision – Introduction to Convolution Neural Networks, 2016

• Machine Learning，李宏毅，台大電機，2016

  • Brief Introduction of Deep Learning pdf, video

  • Tips for deep learning pdf, video

  • Why deep pdf, video

Readings Advanced

• V. Kumar and M. L, “Deep Learning as a Frontier of Machine Learning: A Review,” International Journal of Computer Applications, vol. 182, no. 1, pp. 22–30, Jul. 2018

• J. Schmidhuber, "Deep Learning in Neural Networks: An Overview," arXiv:1404.7828v4, 2014

Lecture Note : Convolutional Neural Network (CNN) (PPTX, PDF)

Homework : Project proposal

Readings :

• 【機器學習2021】卷積神經網路 (Convolutional Neural Networks, CNN)，台大李宏毅Youtube 2021.  ML2021 week3 CNN slides.  55'38"

• Convolutional Neural Networks Explained (What Is Computer Vision), Youtube 2020.  10'46"

• Intermediate Topics in Neural Networks - A detailed overview of neural architecture, activation functions, loss functions, output units. 2020

• Advanced Topics in Deep Convolutional Neural Networks, 2019

  • Neural Network 3D Simulation, Youtube 2’44"

• An introduction to Convolutional Neural Networks, 2019

• An Introduction to Convolutional Neural Networks - A simple guide to what CNNs are, how they work, and how to build one from scratch in Python. 2019

• Deep Learning with Convolutional Neural Networks, 2019

• Computer Vision: A Study On Different CNN Architectures and their Applications, 2019

• A Gentle Introduction to the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), 2019

• A Gentle Introduction to Pooling Layers for Convolutional Neural Networks, 2019

• Gentle Dive into Math Behind Convolutional Neural Networks - Mysteries of Neural Networks Part V, 2019

• Convolutional Neural Network: A Step By Step Guide, 2019

• Simple Introduction to Convolutional Neural Networks, 2019

• Convolutional Neural Network - Learn Convolutional Neural Network from basic and its implementation in Keras, 2019

• Intuitive Deep Learning : Introduction, 2019

  • Part 1a: Introduction to Neural Networks - What is Deep Learning? A very gentle and intuitive introduction to Neural Networks and how they work! 

  • Part 1b: Introduction to Neural Networks - How do we make neural networks work? Some nitty-gritty details to take note of! 

  • Part 2: CNNs for Computer Vision - What are Convolutional Neural Networks? How can we apply Neural Networks to recognize images?  

• Convolution: An Exploration of a Familiar Operator’s Deeper Roots, 2018

• Convolutional Neural Networks for Beginners - Practical Guide with Python and Keras, 2018

• Image Classification in 10 Minutes with MNIST Dataset, 2018

• An Introduction to Convolutional Neural Networks, 2018

• (ReLU) An Overview to Vanishing Gradient Problem, 2018

• Ted’s Lab, 2018

  • Part 1: Introduction To Deep Learning

  • Part 2: Deep Neural Networks as Computational Graphs

  • Part 3: How to classify MNIST digits with different neural network architectures - Getting started with neural networks and Keras

• Tutorial: Derivation of Convolutional Neural Network from Fully Connected Network, 2018

• 解析卷积神经网络—深度学习实践手册，2017 (51MB)

• How Convolutional Neural Networks Work, B. Rohrer, 2016. Youtube 26'13".   Blog.

• Convolutional Neural Network pdf, video, Machine Learning課程，李宏毅，台大電機，2016

• Convolutional Neural Network, Wiki