Deep Learning - Papers
Contents_Index
- UNDERSTANDING / GENERALIZATION / TRANSFER3
- OPTIMIZATION / TRAINING TECHNIQUES4
- UNSUPERVISED / GENERATIVE MODELS1
- CONVOLUTIONAL NEURAL NETWORK MODELS2
- IMAGE: SEGMENTATION / OBJECT DETECTION2
- IMAGE / VIDEO / ETC4
- NATURAL LANGUAGE PROCESSING / RNNS1
- SPEECH / OTHER DOMAIN2
- REINFORCEMENT LEARNING / ROBOTICS1
- MORE PAPERS FROM 20167
- NEW PAPERS6
- OLD PAPERS1
- HW / SW / DATASET5
- BOOK / SURVEY / REVIEW5
- APPENDIX: MORE THAN TOP 10011
Understanding / Generalization / Transfer
3_ENTRIES-
Distilling the knowledge in a neural network (2015), G. Hinton et al. [pdf]
-
How transferable are features in deep neural networks? (2014), J. Yosinski et al. [pdf]
-
Learning and transferring mid-Level image representations using convolutional neural networks (2014), M. Oquab et al. [pdf]
-
Visualizing and understanding convolutional networks (2014), M. Zeiler and R. Fergus [pdf]
- **Deep neural networks are easily fooled
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2015), A. Nguyen et al. [pdf]
- **CNN features off-the-Shelf
CNN features off-the-Shelf: An astounding baseline for recognition (2014), A. Razavian et al. [pdf]
Optimization / Training Techniques
4_ENTRIES- **Batch normalization
Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), S. Loffe and C. Szegedy [pdf]
- **Delving deep into rectifiers
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (2015), K. He et al. [pdf]
Unsupervised / Generative Models
1_ENTRIES-
Pixel recurrent neural networks (2016), A. Oord et al. [pdf]
-
Improved techniques for training GANs (2016), T. Salimans et al. [pdf]
-
Unsupervised representation learning with deep convolutional generative adversarial networks (2015), A. Radford et al. [pdf]
-
Generative adversarial nets (2014), I. Goodfellow et al. [pdf]
-
Auto-encoding variational Bayes (2013), D. Kingma and M. Welling [pdf]
-
Building high-level features using large scale unsupervised learning (2013), Q. Le et al. [pdf]
Convolutional Neural Network Models
2_ENTRIES-
Rethinking the inception architecture for computer vision (2016), C. Szegedy et al. [pdf]
-
Inception-v4, inception-resnet and the impact of residual connections on learning (2016), C. Szegedy et al. [pdf]
-
Identity Mappings in Deep Residual Networks (2016), K. He et al. [pdf]
-
Deep residual learning for image recognition (2016), K. He et al. [pdf]
-
Spatial transformer network (2015), M. Jaderberg et al., [pdf]
-
Going deeper with convolutions (2015), C. Szegedy et al. [pdf]
-
**Very deep convolutional networks for large-scale image recognit…
- **Return of the devil in the details
Return of the devil in the details: delving deep into convolutional nets (2014), K. Chatfield et al. [pdf]
- **OverFeat
OverFeat: Integrated recognition, localization and detection using convolutional networks (2013), P. Sermanet et al. [pdf]
Image: Segmentation / Object Detection
2_ENTRIES-
Fully convolutional networks for semantic segmentation (2015), J. Long et al. [pdf]
-
Fast R-CNN (2015), R. Girshick [pdf]
-
Rich feature hierarchies for accurate object detection and semantic segmentation (2014), R. Girshick et al. [pdf]
-
Spatial pyramid pooling in deep convolutional networks for visual recognition (2014), K. He et al. [pdf]
-
Semantic image segmentation with deep convolutional nets and fully connected CRFs, L. Chen et al. [pdf]
-
Learning hierarchical features for scene labeling (2013), C. Farabet et al. [[pdf]](https://h…
- **You only look once
You only look once: Unified, real-time object detection (2016), J. Redmon et al. [pdf]
- **Faster R-CNN
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015), S. Ren et al. [pdf]
Image / Video / Etc
4_ENTRIES-
Image Super-Resolution Using Deep Convolutional Networks (2016), C. Dong et al. [pdf]
-
A neural algorithm of artistic style (2015), L. Gatys et al. [pdf]
-
Deep visual-semantic alignments for generating image descriptions (2015), A. Karpathy and L. Fei-Fei [pdf]
-
Long-term recurrent convolutional networks for visual recognition and description (2015), J. Donahue et al. [pdf]
-
Large-scale video classification with convolutional neural networks (2014), A. Karpathy et al. [pdf]
-
Two-stream convolutional networks for action recognition in videos (2014), K. Simonyan et al. [[p…
- **Show, attend and tell
Show, attend and tell: Neural image caption generation with visual attention (2015), K. Xu et al. [pdf]
- **Show and tell
Show and tell: A neural image caption generator (2015), O. Vinyals et al. [pdf]
- **DeepFace
DeepFace: Closing the gap to human-level performance in face verification (2014), Y. Taigman et al. [pdf]:
Natural Language Processing / RNNs
1_ENTRIES-
Neural Architectures for Named Entity Recognition (2016), G. Lample et al. [pdf]
-
Exploring the limits of language modeling (2016), R. Jozefowicz et al. [pdf]
-
Teaching machines to read and comprehend (2015), K. Hermann et al. [pdf]
-
Effective approaches to attention-based neural machine translation (2015), M. Luong et al. [pdf]
-
Conditional random fields as recurrent neural networks (2015), S. Zheng and S. Jayasumana. [pdf]
-
Memory networks (2014), J. Weston et al. [pdf]
-
Neural turing machines (2014), A. Graves et al. [pdf]
-
**Neural machine translation b…
Speech / Other Domain
2_ENTRIES-
End-to-end attention-based large vocabulary speech recognition (2016), D. Bahdanau et al. [pdf]
-
Speech recognition with deep recurrent neural networks (2013), A. Graves [pdf]
-
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition (2012) G. Dahl et al. [pdf]
-
Acoustic modeling using deep belief networks (2012), A. Mohamed et al. [pdf]
- **Deep speech 2
Deep speech 2: End-to-end speech recognition in English and Mandarin (2015), D. Amodei et al. [pdf]
- **Deep neural networks for acoustic m...
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups (2012), G. Hinton et al. [pdf]
Reinforcement Learning / Robotics
1_ENTRIES-
End-to-end training of deep visuomotor policies (2016), S. Levine et al. [pdf]
-
Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection (2016), S. Levine et al. [pdf]
-
Asynchronous methods for deep reinforcement learning (2016), V. Mnih et al. [pdf]
-
Deep Reinforcement Learning with Double Q-Learning (2016), H. Hasselt et al. [pdf]
-
Mastering the game of Go with deep neural networks and tree search (2016), D. Silver et al. [pdf]
-
Continuous control with deep reinforcement learning (2015), T. Lillicrap et al. [pdf]
-
Deep learning for detecting robotic grasps (2015), I. Lenz et al. [[pdf]](http://w…
- **Human-level control through deep re...
Human-level control through deep reinforcement learning (2015), V. Mnih et al. [pdf]
More Papers from 2016
7_ENTRIES-
Layer Normalization (2016), J. Ba et al. [pdf]
-
Learning to learn by gradient descent by gradient descent (2016), M. Andrychowicz et al. [pdf]
-
Domain-adversarial training of neural networks (2016), Y. Ganin et al. [pdf]
-
Colorful image colorization (2016), R. Zhang et al. [pdf]
-
Generative visual manipulation on the natural image manifold (2016), J. Zhu et al. [pdf]
-
Dynamic memory networks for visual and textual question answering (2016), C. Xiong et al. [pdf]
-
Stacked attention networks for image question answering (2016), Z. Yang et al. [pdf]
-
**Hybrid…
- **Texture networks
Texture networks: Feed-forward synthesis of textures and stylized images (2016), D Ulyanov et al. [pdf]
- **SqueezeNet
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size (2016), F. Iandola et al. [pdf]
- **Binarized neural networks
Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1 (2016), M. Courbariaux et al. [pdf]
- **Google's neural machine translation...
Google's neural machine translation system: Bridging the gap between human and machine translation (2016), Y. Wu et al. [pdf]
New papers
6_ENTRIESNewly published papers (< 6 months) which are worth reading
-
Convolutional Sequence to Sequence Learning (2017), Jonas Gehring et al. [pdf]
-
A Knowledge-Grounded Neural Conversation Model (2017), Marjan Ghazvininejad et al. [pdf]
-
Deep Photo Style Transfer (2017), F. Luan et al. [pdf]
-
Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017), T. Salimans et al. [pdf]
-
Deformable Convolutional Networks (2017), J. Dai et al. [pdf]
-
Mask R-CNN (2017), K. He et al. [pdf]
-
Learning to discover cross-domain relations with generative adversarial networks (2017), T. Kim et al. [pdf]
-
Wasserstein GAN (2017), M. Arjovsky et al. [pdf]
-
Understan…
- MobileNets
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017), Andrew G. Howard et al. [pdf]
- Accurate, Large Minibatch SGD
Accurate, Large Minibatch SGD:Training ImageNet in 1 Hour (2017), Priya Goyal et al. [pdf]
- Deep voice
Deep voice: Real-time neural text-to-speech (2017), S. Arik et al., [pdf]
- Batch renormalization
Batch renormalization: Towards reducing minibatch dependence in batch-normalized models (2017), S. Ioffe. [pdf]
Old Papers
1_ENTRIESClassic papers published before 2012
-
An analysis of single-layer networks in unsupervised feature learning (2011), A. Coates et al. [pdf]
-
Deep sparse rectifier neural networks (2011), X. Glorot et al. [pdf]
-
Natural language processing (almost) from scratch (2011), R. Collobert et al. [pdf]
-
Recurrent neural network based language model (2010), T. Mikolov et al. [pdf]
-
Learning mid-level features for recognition (2010), Y. Boureau [pdf]
-
A practical guide to training restricted boltzmann machines (2010), G. Hinton [pdf]
-
Understanding the difficulty of training deep feedforward neura…
- Stacked denoising autoencoders
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion (2010), P. Vincent et al. [pdf]
HW / SW / Dataset
5_ENTRIES- TensorFlow
TensorFlow: Large-scale machine learning on heterogeneous distributed systems (2016), M. Abadi et al. [pdf]
- MatConvNet
MatConvNet: Convolutional neural networks for matlab (2015), A. Vedaldi and K. Lenc [pdf]
Book / Survey / Review
5_ENTRIES-
On the Origin of Deep Learning (2017), H. Wang and Bhiksha Raj. [pdf]
-
Neural Network and Deep Learning (Book, Jan 2017), Michael Nielsen. [html]
-
Deep learning (Book, 2016), Goodfellow et al. [html]
-
Tutorial on Variational Autoencoders (2016), C. Doersch. [pdf]
-
Deep learning (2015), Y. LeCun, Y. Bengio and G. Hinton [pdf]
- Deep Reinforcement Learning
Deep Reinforcement Learning: An Overview (2017), Y. Li, [pdf]
- Neural Machine Translation and Sequen...
Neural Machine Translation and Sequence-to-sequence Models(2017): A Tutorial, G. Neubig. [pdf]
- Deep learning in neural networks
Deep learning in neural networks: An overview (2015), J. Schmidhuber [pdf]
- Representation learning
Representation learning: A review and new perspectives (2013), Y. Bengio et al. [pdf]
Appendix: More than Top 100
11_ENTRIES(2016)
-
A character-level decoder without explicit segmentation for neural machine translation (2016), J. Chung et al. [pdf]
-
Dermatologist-level classification of skin cancer with deep neural networks (2017), A. Esteva et al. [html]
-
Weakly supervised object localization with multi-fold multiple instance learning (2017), R. Gokberk et al. [pdf]
-
Brain tumor segmentation with deep neural networks (2017), M. Havaei et al. [pdf]
-
Adversarially learned inference (2016), V. Dumoulin et al. [web][pdf]
-
Understanding convolutional neural networks (2016), J. Koushik [pdf]
-
Adaptive computation time for recurrent neural networks (2016), A. Graves [[pdf]](http://arxiv.org/pdf/1603.089…
- Professor Forcing
Professor Forcing: A New Algorithm for Training Recurrent Networks (2016), A. Lamb et al. [pdf]
- Taking the human out of the loop
Taking the human out of the loop: A review of bayesian optimization (2016), B. Shahriari et al. [pdf]
- Inside-outside net
Inside-outside net: Detecting objects in context with skip pooling and recurrent neural networks (2016), S. Bell et al. [pdf].
- Ask your neurons
Ask your neurons: A neural-based approach to answering questions about images (2015), M. Malinowski et al. [pdf]
- Mind's eye
Mind's eye: A recurrent visual representation for image caption generation (2015), X. Chen and C. Zitnick. [pdf]
- Towards AI-complete question answering
Towards AI-complete question answering: A set of prerequisite toy tasks (2015), J. Weston et al. [pdf]
- Ask me anything
Ask me anything: Dynamic memory networks for natural language processing (2015), A. Kumar et al. [pdf]
- Deep compression
Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding (2015), S. Han et al. [pdf]
- Beyond short snippents
Beyond short snippents: Deep networks for video classification (2015) [pdf]
- Finding function in form
Finding function in form: Compositional character models for open vocabulary word representation (2015), W. Ling et al. [pdf]