PRACTICE DATASETS

PART 1. ARTIFICIAL NEURAL NETWORKS (ANN)

Datasets & Templates:

• Artificial_Neural_Networks

Additional Reading:

• Yann LeCun et al., 1998, Efficient BackProp

• By Xavier Glorot et al., 2011, Deep sparse rectifier neural networks

• CrossValidated, 2015, A list of cost functions used in neural networks, alongside applications

• Andrew Trask, 2015, A Neural Network in 13 lines of Python (Part 2 – Gradient Descent)

• Michael Nielsen, 2015, Neural Networks and Deep Learning

PART 2. CONVOLUTIONAL NEURAL NETWORKS (CNN)

Datasets & Templates:

• Convolutional_Neural_Networks

Additional Reading:

• Yann LeCun et al., 1998, Gradient-Based Learning Applied to Document Recognition

• Jianxin Wu, 2017, Introduction to Convolutional Neural Networks

• C.-C. Jay Kuo, 2016, Understanding Convolutional Neural Networks with A Mathematical Model

• Kaiming He et al., 2015, Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

• Dominik Scherer et al., 2010, Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition

• Adit Deshpande, 2016, The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)

• Rob DiPietro, 2016, A Friendly Introduction to Cross-Entropy Loss

• Peter Roelants, 2016, How to implement a neural network Intermezzo 2

PART 3. RECURRENT NEURAL NETWORKS (RNN)

Datasets & Templates:

• Recurrent_Neural_Networks
• Homework_Challenge

Additional Reading:

• Oscar Sharp & Benjamin, 2016, Sunspring
• Sepp (Josef) Hochreiter, 1991, Untersuchungen zu dynamischen neuronalen Netzen
• Yoshua Bengio, 1994, Learning Long-Term Dependencies with Gradient Descent is Difficult
• Razvan Pascanu, 2013, On the difficulty of training recurrent neural networks
• Sepp Hochreiter & Jurgen Schmidhuber, 1997, Long Short-Term Memory
• Christopher Olah, 2015, Understanding LSTM Networks
• Shi Yan, 2016, Understanding LSTM and its diagrams
• Andrej Karpathy, 2015, The Unreasonable Effectiveness of Recurrent Neural Networks
• Andrej Karpathy, 2015, Visualizing and Understanding Recurrent Networks
• Klaus Greff, 2015, LSTM: A Search Space Odyssey
• Xavier Glorot, 2011, Deep sparse rectifier neural networks

PART 4. SELF ORGANIZING MAPS (SOM)

Datasets & Templates:

• Self_Organizing_Maps
• Mega_Case_Study

Additional Reading:

• Tuevo Kohonen, 1990, The Self-Organizing Map
• Mat Buckland, 2004?, Kohonen's Self Organizing Feature Maps
• Nadieh Bremer, 2003, SOM – Creating hexagonal heatmaps with D3.js

PART 5. BOLTZMANN MACHINES (BM)

Datasets & Templates:

• Boltzmann_Machines

Additional Reading:

• Yann LeCun, 2006, A Tutorial on Energy-Based Learning
• Jaco Van Dormael, 2009, Mr. Nobody
• Geoffrey Hinton, 2006, A fast learning algorithm for deep belief nets
• Oliver Woodford, 2012?, Notes on Contrastive Divergence
• Yoshua Bengio, 2006, Greedy Layer-Wise Training of Deep Networks
• Geoffrey Hinton, 1995, The wake-sleep algorithm for unsupervised neural networks
• Ruslan Salakhutdinov, 2009?, Deep Boltzmann Machines

PART 6. AUTOENCODERS (AE)

Datasets & Templates:

• AutoEncoders

Additional Reading:

• Malte Skarupke, 2016, Neural Networks Are Impressively Good At Compression
• Francois Chollet, 2016, Building Autoencoders in Keras
• Chris McCormick, 2014, Deep Learning Tutorial – Sparse Autoencoder
• Eric Wilkinson, 2014, Deep Learning: Sparse Autoencoders
• Alireza Makhzani, 2014, k-Sparse Autoencoders
• Pascal Vincent, 2008, Extracting and Composing Robust Features with Denoising Autoencoders
• Salah Rifai, 2011, Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
• Pascal Vincent, 2010, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
• Geoffrey Hinton, 2006, Reducing the Dimensionality of Data with Neural Networks