Overview
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is designed with an intuitive programming model based on the layer abstraction. A variety of popular deep learning models are supported, namely feed-forward models including convolutional neural networks (CNN), energy models like restricted Boltzmann machine (RBM), and recurrent neural networks (RNN). Many built-in layers are provided for users. SINGA architecture is sufficiently flexible to run synchronous, asynchronous and hybrid training frameworks. SINGA also supports different neural net partitioning schemes to parallelize the training of large models, namely partitioning on batch dimension, feature dimension or hybrid partitioning.
Audience
This course is directed at researchers, engineers and developers seeking to utilize Apache SINGA as a deep learning framework.
After completing this course, delegates will:
- understand SINGA’s structure and deployment mechanisms
- be able to carry out installation / production environment / architecture tasks and configuration
- be able to assess code quality, perform debugging, monitoring
- be able to implement advanced production like training models, embedding terms, building graphs and logging
Course Outline
Introduction
Installation
Quick Start
Programming
- NeuralNet
- Layer
- Param
- TrainOneBatch
- Updater
Distributed Training
Data Preparation
Checkpoint and Resume
Python Binding
Performance test and Feature extraction
Training on GPU
Examples
- Feed-forward models
- CNN
- MLP
- RBM + Auto-encoder
- Vanilla RNN for language modelling
- Char-RNN