PRACTICE DATASETS

PART 1. ARTIFICIAL NEURAL NETWORKS (ANN)

Datasets & Templates:

• Artificial_Neural_Networks

Additional Reading:

• Yann LeCun et al., 1998, Efficient BackProp

• By Xavier Glorot et al., 2011, Deep sparse rectifier neural networks

• CrossValidated, 2015, A list of cost functions used in neural networks, alongside applications

• Andrew Trask, 2015, A Neural Network in 13 lines of Python (Part 2 – Gradient Descent)

• Michael Nielsen, 2015, Neural Networks and Deep Learning

PART 2. CONVOLUTIONAL NEURAL NETWORKS (CNN)

Datasets & Templates:

• Convolutional_Neural_Networks

Additional Reading:

• Yann LeCun et al., 1998, Gradient-Based Learning Applied to Document Recognition

• Jianxin Wu, 2017, Introduction to Convolutional Neural Networks

• C.-C. Jay Kuo, 2016, Understanding Convolutional Neural Networks with A Mathematical Model

• Kaiming He et al., 2015, Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

• Dominik Scherer et al., 2010, Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition

• Adit Deshpande, 2016, The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)

• Rob DiPietro, 2016, A Friendly Introduction to Cross-Entropy Loss

• Peter Roelants, 2016, How to implement a neural network Intermezzo 2

PART 3. RECURRENT NEURAL NETWORKS (RNN)

Datasets & Templates:

• Recurrent_Neural_Networks

• Homework_Challenge

Additional Reading:

• Oscar Sharp & Benjamin, 2016, Sunspring

• Sepp (Josef) Hochreiter, 1991, Untersuchungen zu dynamischen neuronalen Netzen

• Yoshua Bengio, 1994, Learning Long-Term Dependencies with Gradient Descent is Difficult

• Razvan Pascanu, 2013, On the difficulty of training recurrent neural networks

• Sepp Hochreiter & Jurgen Schmidhuber, 1997, Long Short-Term Memory

• Christopher Olah, 2015, Understanding LSTM Networks

• Shi Yan, 2016, Understanding LSTM and its diagrams

• Andrej Karpathy, 2015, The Unreasonable Effectiveness of Recurrent Neural Networks

• Andrej Karpathy, 2015, Visualizing and Understanding Recurrent Networks

• Klaus Greff, 2015, LSTM: A Search Space Odyssey

• Xavier Glorot, 2011, Deep sparse rectifier neural networks

PART 4. SELF ORGANIZING MAPS (SOM)

Datasets & Templates:

• Self_Organizing_Maps

• Mega_Case_Study

Additional Reading:

• Tuevo Kohonen, 1990, The Self-Organizing Map

• Mat Buckland, 2004?, Kohonen's Self Organizing Feature Maps

• Nadieh Bremer, 2003, SOM – Creating hexagonal heatmaps with D3.js

PART 5. BOLTZMANN MACHINES (BM)

Datasets & Templates:

• Boltzmann_Machines

Additional Reading:

• Yann LeCun, 2006, A Tutorial on Energy-Based Learning

• Jaco Van Dormael, 2009, Mr. Nobody

• Geoffrey Hinton, 2006, A fast learning algorithm for deep belief nets

• Oliver Woodford, 2012?, Notes on Contrastive Divergence

• Yoshua Bengio, 2006, Greedy Layer-Wise Training of Deep Networks

• Geoffrey Hinton, 1995, The wake-sleep algorithm for unsupervised neural networks

• Ruslan Salakhutdinov, 2009?, Deep Boltzmann Machines

PART 6. AUTOENCODERS (AE)

Datasets & Templates:

• AutoEncoders

Additional Reading:

• Malte Skarupke, 2016, Neural Networks Are Impressively Good At Compression

• Francois Chollet, 2016, Building Autoencoders in Keras

• Chris McCormick, 2014, Deep Learning Tutorial – Sparse Autoencoder

• Eric Wilkinson, 2014, Deep Learning: Sparse Autoencoders

• Alireza Makhzani, 2014, k-Sparse Autoencoders

• Pascal Vincent, 2008, Extracting and Composing Robust Features with Denoising Autoencoders

• Salah Rifai, 2011, Contractive Auto-Encoders: Explicit Invariance During Feature Extraction

• Pascal Vincent, 2010, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion

• Geoffrey Hinton, 2006, Reducing the Dimensionality of Data with Neural Networks