Deep Learning知识要点

Deep Learning知识要点

Topic:
matrix factorization
collaborative filtering
CTR预估
one shot learning （单样本学习）+ 孪生网络
强化学习 RL
对抗生成网络 GAN
GCN/GNN
Meta-learning法。从先前的学习经验中提炼出基本的参数和结构配置。
Transfer learning法。从先前的学习经验中提炼出可以重用的一些知识
machine translation
LSTM/GRU/GMU…
Attention
Transformer
encoder-decoder
factorization machine
auto-encoder
sequence to sequence
copy net
highway
conventional/ recurrent / recursive neural network
memory network/Neural Turing Machines
Momentum

Method:
DSSM
Wide & Deep
GAN各种变形
FM各种变形
GLU/GTU/GMU/GRU
ResNet
copynet
实验的trick:
dropout
L1/L2 正则
batch norm
clip by norm/value
learning rate decay
预处理语言模型：
word2vec, fasttext, glove, elmo, gpt, bert

CNN(卷积神经网络)、RNN(循环神经网络)、DNN(深度神经网络)的内部网络结构区别
https://www.zhihu.com/question/34681168
http://www.cnblogs.com/pinard/p/6418668.html
http://blog.csdn.net/Real_Myth/article/details/53780314
https://www.jiqizhixin.com/articles/2017-03-30-4
http://www.jianshu.com/p/9dc9f41f0b29
https://www.jiqizhixin.com/articles/2017-11-04-3

反向传播：
https://zhuanlan.zhihu.com/p/24801814
http://blog.csdn.net/pennyliang/article/details/6695355
https://www.zhihu.com/question/27239198/answer/154510111
https://zhuanlan.zhihu.com/p/25081671

max pooling:
https://www.zhihu.com/question/23437871
http://blog.csdn.net/jiejinquanil/article/details/50042791
http://blog.csdn.net/pangjiuzala/article/details/71840947

LSTM:
https://www.zhihu.com/question/41949741
http://blog.csdn.net/jerr__y/article/details/58598296

machine translation:
https://zhuanlan.zhihu.com/p/25366912