A Review about Deep Learning Methods and Applications

Year 2017, Volume: 3 Issue: 3, 47 - 64, 27.12.2017

Abdulkadir Şeker , Banu Diri Hasan Hüseyin Balık

Abstract

Artificial neural networks were used in the solution of many
problems in the field of machine learning. However, in the period called
"AI Winter", studies in this area have come to a halt due to
especially hardware limitations and other problem. Artificial neural networks,
which started to become a popular area at beginning of the 2000s, have switched
from shallow networks to deep networks thanks to GPU developments. This
approach has been successfully used in a wide range of fields from image
processing to natural language processing, from medical applications to
activity identification. In this study, it is described the history of the deep
learning, methods and the implementations separated by the application areas.
In addition, information has been given to the libraries used in recent years
and working groups focused on deep learning. The aim of this study both
explains the developments in deep learning to researchers and provides possible
fields study with deep learning.

Keywords

Deep learning, Feature engineering, Artificial neural networks

Derin Öğrenme Yöntemleri Ve Uygulamaları Hakkında Bir İnceleme

Abstract

Makine öğrenmesi alanında yapay sinir ağları birçok problemin çözümünde sıklıkla kullanılmıştır. Ancak ―Yapay Zeka Kış Uykusu‖ olarak da adlandırılan dönemde başta donanımsal kısıtlamalar ve diğer problemler sebebiyle bu alandaki çalışmalar durma noktasına gelmiştir. 2000’lerin başında tekrar gözde bir alan olmaya başlayan yapay sinir ağları, GPU gelişmeleriyle birlikte sığ ağlardan derin ağlara geçiş yapmıştır. Bu yaklaşım görüntü işlemeden, doğal dil işlemeye, medikal uygulamalardan aktivite tanımaya kadar oldukça geniş bir yelpazede başarıyla kullanılmaya başlanmıştır. Bu çalışmada, derin öğrenmenin tarihçesi, kullanılan yöntemler ve uygulama alanlarına göre ayrılmış çalışmalar anlatılmıştır. Ayrıca son yıllarda kullanılan kütüphaneler ve derin öğrenme üzerine yoğunlaşan çalışma grupları hakkında da bilgiler verilmiştir. Bu çalışmanın amacı, hem araştırmacılara derin öğrenme konusundaki gelişmeleri anlatmak, hem de derin öğrenme ile çalışılacak muhtemel konuları vermektir.

Keywords

Derin öğrenme, Özellik mühendisliği, Yapay sinir ağları

References

• [1] J. Schmidhuber, ―Deep learning in neural networks: An overview,‖ Neural Networks, vol. 61, pp. 85–117, 2015.
• [2] Y. LeCun, Y. Bengio, and G. Hinton, ―Deep learning,‖ Nature, vol. 521, pp. 436–444, 2015.
• [3] A. L. Blum and P. Langley, ―Selection of relevant features and examples in machine learning,‖ Artif. Intell., vol. 97, no. 1–2, pp. 245–271, Dec. 1997.
• [4] Y. Bengio, A. Courville, and P. Vincent, ―Representation Learning: A Review and New Perspectives,‖ IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798–1828, 2013.
• [5] L. Deng and D. Yu, ―Deep Learning: Methods and Applications,‖ Found. Trends® Signal Process., vol. 7, no. 3–4, pp. 197–387, 2014.
• [6] Y. Bengio, ―Learning Deep Architectures for AI,‖ Found. trends® Mach. Learn., vol. 2, no. 1, pp. 1–127, 2009.
• [7] H. A. Song and S.-Y. Lee, ―Hierarchical Representation Using NMF,‖ in International Conference on Neural Information Processing., 2013, pp. 466–473.
• [8] A. G. Ivakhnenko and V. G. Lapa, ―Cybernetic Predicting Devices,‖ 1966.
• [9] Tim Dettmers, ―Deep Learning in a Nutshell: History and Training Parallel Forall,‖ 2015. [Online]. Available: https://devblogs.nvidia.com/parallelforall/deeplearning-nutshell-history-training/. [Accessed: 20- Mar-2017].
• [10] K. . N. Fukushima, ―A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position.,‖ Biol. Cybern., vol. 36, no. 4, pp. 193–202, 1980.
• [11] Y. LeCun et al., ―Backpropagation Applied to Handwritten Zip Code Recognition,‖ Neural Comput., vol. 1, no. 4, pp. 541–551, Dec. 1989.
• [12] LeCun Yann, Boser B, and Denker J S, ―Handwritten Digit Recognition with a BackPropagation Network,‖ 1989.
• [13] G. E. Hinton, P. Dayan, B. J. Frey, and R. M. Neal, ―The wake-sleep algorithm for unsupervised neural networks,‖ Science, vol. 268, no. 5214, pp. 1158–61, May 1995.
• [14] S. Hochreiter and J. Schmidhuber, ―Long ShortTerm Memory,‖ Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
• [15] C. Cortes and V. Vapnik, ―Support-Vector Networks,‖ Mach. Learn., vol. 20, no. 3, pp. 273– 297, 1995.
• [16] I. N. Aizenberg, N. N. Aizenberg, and J. Vandewalle, ―Multiple-Valued Threshold Logic and Multi-Valued Neurons,‖ in Multi-Valued and Universal Binary Neurons, Boston, MA: Springer US, 2000, pp. 25–80.
• [17] G. E. Hinton, ―Learning multiple layers of representation,‖ Trends Cogn. Sci., vol. 11, no. 10, pp. 428–434, Oct. 2007.
• [18] D. Ciresan, A. Giusti, L. M. Gambardella, and J. Schmidhuber, ―Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images,‖ in Advances in neural information processing systems, 2012, pp. 2843–2851.
• [19] D. Cireşan, U. Meier, J. Masci, and J. Schmidhuber, ―Multi-column deep neural network for traffic sign classification,‖ Neural Networks, vol. 32, pp. 333–338, 2012.
• [20] D. C. Ciresan, U. Meier, L. M. Gambardella, and J. Schmidhuber, ―Convolutional Neural Network Committees for Handwritten Character Classification,‖ in 2011 International Conference on Document Analysis and Recognition, 2011, pp. 1135–1139.
• [21] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, ―Improving Neural Networks by Preventing Co-adaptation of Feature Detectors,‖ Neural Evol. Comput., Jul. 2012.
• [22] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ―ImageNet Classification with Deep Convolutional Neural Networks,‖ in Advances in neural information processing systems, 2012, pp. 1097– 1105.
• [23] ―The Race For AI: Google, Twitter, Intel, Apple In A Rush To Grab Artificial Intelligence Startups,‖ cbinsights, 2017. [Online]. Available: https://www.cbinsights.com/blog/top-acquirers-aistartups-ma-timeline/. [Accessed: 21-Apr-2017].
• [24] D. H. Hubel and T. N. Wiesel, ―Receptive fields and functional architecture of monkey striate cortex,‖ J. Physiol., vol. 195, no. 1, pp. 215–243, Mar. 1968.
• [25] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, ―Gradient-based learning applied to document recognition,‖ Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
• [26] Y. Le Cun et al., ―Handwritten digit recognition: applications of neural network chips and automatic learning,‖ IEEE Commun. Mag., vol. 27, no. 11, pp. 41–46, Nov. 1989.
• [27] D. Cireşan, U. Meier, and J. Schmidhuber, ―Multi-column Deep Neural Networks for Image Classification,‖ Feb. 2012.
• [28] D. C. Cirean, U. Meier, J. Masci, and L. M. Gambardella, ―Flexible, High Performance Convolutional Neural Networks for Image Classification,‖ in Proceedings of the TwentySecond international joint conference on Artificial Intelligence, 2012, pp. 1237–1242.
• [29] ―Results of ILSVRC2014,‖ 11-Nov-2014. [Online]. Available: http://arxiv.org/abs/1311.2524. [Accessed: 26-Apr-2017]. [30] S. S. Farfade, M. Saberian, and L.-J. Li, ―Multiview Face Detection Using Deep Convolutional Neural Networks,‖ in Proceedings of the 5th ACM on International Conference on Multimedia Retrieval., 2015, pp. 643–650.
• [31] E. Grefenstette, P. Blunsom, N. de Freitas, and K. M. Hermann, ―A Deep Architecture for Semantic Parsing,‖ Apr. 2014.
• [32] Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil, ―Learning semantic representations using convolutional neural networks for web search,‖ in Proceedings of the 23rd International Conference on World Wide Web - WWW ’14 Companion, 2014, pp. 373–374.
• [33] N. Kalchbrenner, E. Grefenstette, and P. Blunsom, ―A Convolutional Neural Network for Modelling Sentences,‖ Apr. 2014.
• [34] Y. Kim, ―Convolutional Neural Networks for Sentence Classification,‖ Aug. 2014.
• [35] R. Collobert and J. Weston, ―A unified architecture for natural language processing:Deep neural networks with multitask learning,‖ in Proceedings of the 25th international conference on Machine learning - ICML ’08, 2008, vol. 20, no. 1, pp. 160–167.
• [36] I. Wallach, M. Dzamba, and A. Heifets, ―AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery,‖ Oct. 2015.
• [37] J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson, ―Understanding Neural Networks Through Deep Visualization,‖ Jun. 2015.
• [38] C. J. Maddison, A. Huang, I. Sutskever, and D. Silver, ―Move Evaluation in Go Using Deep Convolutional Neural Networks,‖ Dec. 2014.
• [39] C.-S. Lee et al., ―Human vs. Computer Go: Review and Prospect [Discussion Forum],‖ IEEE Comput. Intell. Mag., vol. 11, no. 3, pp. 67–72, Aug. 2016.
• [40] J. L. Elman, ―Finding Structure in Time,‖ Cogn. Sci., vol. 14, no. 2, pp. 179–211, Mar. 1990.
• [41] T. Mikolov, ―Recurrent neural network based language model,‖ in Interspeech, 2010.
• [42] A. Graves and N. Jaitly, ―Towards End-To-End Speech Recognition with Recurrent Neural Networks.,‖ in ICML, 2014, pp. 1764–1772.
• [43] A. Karpathy and L. Fei-Fei, ―Deep VisualSemantic Alignments for Generating Image Descriptions,‖ in CVPR, 2015, pp. 3128–3137.
• [44] Y. Bengio, P. Simard, and P. Frasconi, ―Learning long-term dependencies with gradient descent is difficult,‖ IEEE Trans. Neural Networks, vol. 5, no. 2, pp. 157–166, Mar. 1994.
• [45] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, ―LSTM: A Search Space Odyssey,‖ Mar. 2015.
• [46] F. A. Gers, J. Schmidhuber, and F. Cummins, ―Learning to forget: continual prediction with LSTM.,‖ Neural Comput., vol. 12, no. 10, pp. 2451– 71, Oct. 2000.
• [47] F. A. Gers and J. Schmidhuber, ―Recurrent nets that time and count,‖ in Proceedings of the IEEEINNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, 2000, pp. 189–194 vol.3.
• [48] A. Graves and J. Schmidhuber, ―Framewise phoneme classification with bidirectional LSTM and other neural network architectures,‖ Neural Networks, vol. 18, no. 5–6, pp. 602–610, Jul. 2005.
• [49] A. Graves, A. Mohamed, and G. Hinton, ―Speech recognition with deep recurrent neural networks,‖ in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 6645–6649.
• [50] S. Fernández, A. Graves, and J. Schmidhuber, ―An Application of Recurrent Neural Networks to Discriminative Keyword Spotting,‖ in International Conference on Artificial Neural Networks, 2007, pp. 220–229.
• [51] M. Baccouche, F. Mamalet, C. Wolf, C. Garcia, and A. Baskurt, ―Sequential Deep Learning for Human Action Recognition,‖ Springer, Berlin, Heidelberg, 2011, pp. 29–39.
• [52] D. Eck and J. Schmidhuber, ―Learning the Long-Term Structure of the Blues,‖ Springer, Berlin, Heidelberg, 2002, pp. 284–289.
• [53] S. Hochreiter, M. Heusel, and K. Obermayer, ―Fast model-based protein homology detection without alignment,‖ Bioinformatics, vol. 23, no. 14, pp. 1728–1736, Jul. 2007.
• [54] H. Mayer, F. Gomez, D. Wierstra, I. Nagy, A. Knoll, and J. Schmidhuber, ―A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks,‖ in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006, pp. 543–548.
• [55] J. Schmidhuber, F. Gers, and D. Eck, ―Learning Nonregular Languages: A Comparison of Simple Recurrent Networks and LSTM,‖ Neural Comput., vol. 14, no. 9, pp. 2039–2041, Sep. 2002.
• [56] A. Graves, ―Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks.‖
• [57] P. Smolensky, ―Information Processing in Dynamical Systems: Foundations of Harmony Theory,‖ 1986.
• [58] G. E. Hinton and R. R. Salakhutdinov, ―Reducing the Dimensionality of Data with Neural Networks,‖ Science (80-. )., vol. 313, no. 5786, pp. 504–507, Jul. 2006.
• [59] R. Salakhutdinov and G. Hinton, ―Deep Boltzmann Machines,‖ in International Conference on Artificial Intelligence and Statistics, 2009, pp. 3– 11.
• [60] M. A. Carreira-Perpiñán and G. E. Hinton, ―On Contrastive Divergence Learning,‖ Artif. Intell. Stat., vol. 10, 2005.
• [61] H. Larochelle and Y. Bengio, ―Classification using discriminative restricted Boltzmann machines,‖ in Proceedings of the 25th international conference on Machine learning - ICML ’08, 2008, pp. 536–543.
• [62] R. Salakhutdinov, A. Mnih, and G. Hinton, ―Restricted Boltzmann machines for collaborative filtering,‖ in Proceedings of the 24th international conference on Machine learning - ICML ’07, 2007, pp. 791–798.
• [63] A. Coates, A. Ng, and H. Lee, ―An Analysis of Single-Layer Networks in Unsupervised Feature Learning,‖ in PMLR, 2011, pp. 215–223.
• [64] G. E. Hinton and R. R. Salakhutdinov, ―Replicated Softmax: an Undirected Topic Model,‖ in Advances in Neural Information Processing Systems 22 , 2009, pp. 1607–1614.
• [65] G. E. Hinton, S. Osindero, and Y.-W. Teh, ―A Fast Learning Algorithm for Deep Belief Nets,‖ Neural Comput., vol. 18, no. 7, pp. 1527–1554, Jul. 2006.
• [66] M. Ranzato, F. J. Huang, Y.-L. Boureau, and Y. LeCun, ―Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition,‖ in 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8.
• [67] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, ―Greedy layer-wise training of deep networks,‖ Proceedings of the 19th International Conference on Neural Information Processing Systems. MIT Press, pp. 153–160, 2006.
• [68] R. Salakhutdinov and G. Hinton, ―Semantic hashing,‖ Int. J. Approx. Reason., vol. 50, pp. 969– 978, 2009.
• [69] G. W. Taylor, G. E. Hinton, and S. T. Roweis, ―Modeling Human Motion Using Binary Latent Variables,‖ in Advances in Neural Information Processing Systems 19 , 2006, pp. 1345–1352.
• [70] Y. Bengio, ―Learning Deep Architectures for AI,‖ Found. Trends® Mach. Learn., vol. 2, no. 1, pp. 1–127, 2009.
• [71] C.-Y. Liou, W.-C. Cheng, J.-W. Liou, and D.-R. Liou, ―Autoencoder for words,‖ Neurocomputing, vol. 139, pp. 84–96, 2014.
• [72] A. B. L. Larsen and S. K. Sønderby, ―Generating Faces with Torch,‖ 13-Nov-2015. [Online]. Available: http://arxiv.org/abs/1506.05751. [Accessed: 27-Apr-2017].
• [73] D. P. Kingma and M. Welling, ―Auto-Encoding Variational Bayes,‖ Dec. 2013.
• [74] A. Krizhevsky and G. E. Hinton, ―Using Very Deep Autoencoders for Content Based Image Retrieval,‖ in European Symposium on Artificial Neural Networks, 2011, pp. 489–494.
• [75] A. Seker and A. Gürkan YUKSEK, ―Stacked Autoencoder Method for Fabric Defect Detection,‖ Sci. Sci. J., vol. 38, no. 2, 2017. [76] ―Stacked Autoencoders,‖ Stanford University, 2013. [Online]. Available: http://ufldl.stanford.edu/wiki/index.php/Stacked_Aut oencoders. [Accessed: 27-Apr-2017].
• [77] G. Mi, Y. Gao, and Y. Tan, ―Apply Stacked Auto-Encoder to Spam Detection,‖ Adv. Swarm Comput. Intell., vol. 9141, pp. 3–15, 2015.
• [78] L. Deng, M. Seltzer, D. Yu, A. Acero, A. Mohamed, and G. Hinton, ―Binary Coding of Speech Spectrograms Using a Deep Auto-encoder,‖ in Interspeech 2010, 2010, p. 1692–1695. [79] O. Lyudchik, ―Outlier detection using autoencoders,‖ 2016.
• [80] S. Yadav and S. Subramanian, ―Detection of Application Layer DDoS attack by feature learning using Stacked AutoEncoder,‖ in International Conference on Computational Techniques in Information and Communication Technologies, 2016, pp. 361–366.
• [81] S. Zhou, Q. Chen, and X. Wang, ―Active deep learning method for semi-supervised sentiment classification,‖ Neurocomputing, vol. 120, pp. 536– 546, 2013.
• [82] S. Zhou, Q. Chen, and X. Wang, ―Fuzzy deep belief networks for semi-supervised sentiment classification,‖ Neurocomputing, vol. 131, pp. 312– 322, 2014.
• [83] Y. Goldberg and O. Levy, ―word2vec Explained: deriving Mikolov et al.’s negativesampling word-embedding method,‖ Feb. 2014.
• [84] L. You, Y. Li, Y. Wang, J. Zhang, and Y. Yang, ―A deep learning-based RNNs model for automatic security audit of short messages,‖ in 2016 16th International Symposium on Communications and Information Technologies (ISCIT), 2016, pp. 225– 229.
• [85] M. Yousefi-Azar and L. Hamey, ―Text Summarization Using Unsupervised Deep Learning,‖ Expert Syst. Appl., vol. 68, pp. 93–105, Feb. 2017.
• [86] A. Sboev, T. Litvinova, D. Gudovskikh, R. Rybka, and I. Moloshnikov, ―Machine Learning Models of Text Categorization by Author Gender Using Topic-independent Features,‖ Procedia Comput. Sci., vol. 101, pp. 135–142, 2016.
• [87] M. L. Brocardo, I. Traore, I. Woungang, and M. S. Obaidat, ―Authorship verification using deep belief network systems,‖ Int. J. Commun. Syst., p. e3259, 2017.
• [88] Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil, ―A Latent Semantic Model with ConvolutionalPooling Structure for Information Retrieval,‖ in CIKM, 2014.
• [89] J. Gao, L. Deng, M. Gamon, and X. He, ―Modeling interestingness with deep neural networks,‖ 14/304,863, 2014.
• [90] Y.-B. Kim, K. Stratos, R. Sarikaya, and M. Jeong, ―New Transfer Learning Techniques For Disparate Label Sets,‖ in Association for Computational Linguistics , 2015.
• [91] O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, ―Show and Tell: A Neural Image Caption Generator,‖ arXiv, vol. 32, no. 1, pp. 1–10, Nov. 2014.
• [92] A.-R. Mohamed, T. N. Sainath, G. Dahl, B. Ramabhadran, G. E. Hinton, and M. A. Picheny, ―DEEP BELIEF NETWORKS USING DISCRIMINATIVE FEATURES FOR PHONE RECOGNITION,‖ in Acoustics, Speech and Signal Processing , 2011, p. 5060–5063.
• [93] G. Hinton et al., ―Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups,‖ IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82–97, Nov. 2012.
• [94] Y. LeCun, C. Cortes, and C. Burges, ―MNIST handwritten digit database.‖ [Online]. Available: http://yann.lecun.com/exdb/mnist/. [Accessed: 30- Apr-2017].
• [95] D. C. Cireşan, A. Giusti, L. M. Gambardella, and J. Schmidhuber, ―Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks,‖ in Medical Image Computing and Computer-Assisted Intervention, 2013, pp. 411–418.
• [96] ―ImageNet,‖ Stanford Vision Lab, 2016. [Online]. Available: http://image-net.org/. [Accessed: 30-Apr-2017].
• [97] O. Russakovsky et al., ―ImageNet Large Scale Visual Recognition Challenge,‖ Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, Dec. 2015.
• [98] Q. V. Le et al., ―Building high-level features using large scale unsupervised learning,‖ arXiv Prepr. arXiv1112.6209, Dec. 2011.
• [99] K. He, X. Zhang, S. Ren, and J. Sun, ―Deep Residual Learning for Image Recognition,‖ in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
• [100] R. Girshick, J. Donahue, T. Darrell, and J. Malik, ―Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,‖ in The IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 580–587.
• [101] C. Farabet, C. Couprie, L. Najman, and Y. LeCun, ―Learning Hierarchical Features for Scene Labeling,‖ IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1915–1929, Aug. 2013.
• [102] P. Sermanet, K. Kavukcuoglu, S. Chintala, and Y. Lecun, ―Pedestrian Detection with Unsupervised Multi-stage Feature Learning,‖ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3626–3633.
• [103] S. K. Zhou, H. Greenspan, and D. Shen, Deep learning for medical image analysis. .
• [104] M. Havaei et al., ―Brain tumor segmentation with Deep Neural Networks,‖ Med. Image Anal., vol. 35, pp. 18–31, Jan. 2017.
• [105] A. Işın, C. Direkoğlu, and M. Şah, ―Review of MRI-based Brain Tumor Image Segmentation Using Deep Learning Methods,‖ Procedia Comput. Sci., vol. 102, pp. 317–324, 2016.
• [106] G. Urban, M. Bendszus, F. A. Hamprecht, and J. Kleesiek, ―Multi-modal Brain Tumor Segmentation using Deep Convolutional Neural Networks,‖ in MICCAI BraTS (Brain Tumor Segmentation) Challenge. Proceedings, winning contribution, 2014, pp. 31–35.
• [107] A. Prasoon, K. Petersen, C. Igel, F. Lauze, E. Dam, and M. Nielsen, ―Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network,‖ in MICCAI, 2013, pp. 246–253.
• [108] Q. Zhang et al., ―Deep learning based classification of breast tumors with shear-wave elastography,‖ Ultrasonics, vol. 72, pp. 150–157, 2016.
• [109] W. Sun, B. Zheng, and W. Qian, ―Automatic Feature Learning Using Multichannel ROI Based on Deep Structured Algorithms for Computerized Lung Cancer Diagnosis,‖ Comput. Biol. Med., 2017.
• [110] J. Arrowsmith and P. Miller, ―Trial Watch: Phase II and Phase III attrition rates 2011–2012,‖ Nat. Rev. Drug Discov., vol. 12, no. 8, pp. 569–569, Aug. 2013.
• [111] ―Merck Molecular Activity Challenge,‖ Kaggle. [Online]. Available: https://www.kaggle.com/c/MerckActivity/details/win ners. [Accessed: 30-Apr-2017].
• [112] G. E. Dahl, N. Jaitly, and R. Salakhutdinov, ―Multi-task Neural Networks for QSAR Predictions,‖ Jun. 2014.
• [113] ―Tox21 Data Challenge Final Subchallenge Leaderboard,‖ 2014. [Online]. Available: https://tripod.nih.gov/tox21/challenge/leaderboard.js p. [Accessed: 01-May-2017].
• [114] T. Unterthiner, A. Mayr, M. Steijaert OpenAnalytics, J. K. Wegner Johnson, H. Ceulemans, and S. Hochreiter, ―Deep Learning as an Opportunity in Virtual Screening,‖ in Workshop on Deep Learning and Representation Learning, 2014.
• [115] D. Chicco, P. Sadowski, and P. Baldi, ―Deep autoencoder neural networks for gene ontology annotation predictions,‖ in Proceedings of the 5th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics - BCB ’14, 2014, pp. 533–540.
• [116] B. Alipanahi, A. Delong, M. Weirauch, and B. Frey, ―Predicting the sequence specificities of DNAand RNA-binding proteins by deep learning,‖ Nat. Biotechnol., 2015.
• [117] A. Sathyanarayana et al., ―Sleep Quality Prediction From Wearable Data Using Deep Learning,‖ JMIR mHealth uHealth, vol. 4, no. 4, p. e125, Nov. 2016.
• [118] Y. Tkachenko, ―Autonomous CRM Control via CLV Approximation with Deep Reinforcement Learning in Discrete and Continuous Action Space,‖ ArXiv, Apr. 2015.
• [119] B. Onyshkevych, ―Deep Exploration and Filtering of Text (DEFT).‖ [Online]. Available: http://www.darpa.mil/program/deep-exploration-andfiltering-of-text. [Accessed: 01-May-2017].
• [120] ―The NVIDIA DGX-1 Deep Learning System,‖ NVIDIA. [Online]. Available: http://www.nvidia.com/object/deep-learningsystem.html. [Accessed: 01-May-2017].
• [121] X. Niu, Y. Zhu, and X. Zhang, ―DeepSense: A novel learning mechanism for traffic prediction with taxi GPS traces,‖ in 2014 IEEE Global Communications Conference, 2014, pp. 2745–2750.
• [122] X. Ma, H. Yu, Y. Wang, Y. Wang, and M. González, ―Large-Scale Transportation Network Congestion Evolution Prediction Using Deep Learning Theory,‖ PLoS One, vol. 10, no. 3, p. e0119044, Mar. 2015.
• [123] D. K. Kim and T. Chen, ―Deep Neural Network for Real-Time Autonomous Indoor Navigation,‖ Nov. 2015.
• [124] ―Comparison of deep learning software.‖ [Online]. Available: https://en.wikipedia.org/wiki/Co mparison_of_deep_learning_software. [Accessed: 01-May-2017].
• [125] Theano Development Team, ―Theano: A {Python} framework for fast computation of mathematical expressions,‖ arXiv e-prints, vol. abs/1605.02688.
• [126] Jason Brownlee, ―Popular Deep Learning Libraries,‖ 2016. [Online]. Available: http://machinelearningmastery.com/popular-deeplearning-libraries/. [Accessed: 01-May-2017].
• [127] ―DeepLearning 0.1 documentation.‖ [Online]. Available: http://deeplearning.net/tutorial/contents [Accessed: 01-May-2017].
• [128] Y. Jia et al., ―Caffe: Convolutional Architecture for Fast Feature Embedding,‖ Jun. 2014.
• [129] R. Collobert, C. Farabet, and K. Kavukcuoğlu, ―Torch | Scientific computing for LuaJIT.,‖ NIPS Workshop on Machine Learning Open Source Software, 2008. [Online]. Available: http://torch.ch/. [Accessed: 01-May-2017].
• [130] ―NVIDIA DIGITS.‖ [Online]. Available: https://developer.nvidia.com/digits [Accessed: 01-May-2017].
• [131] ―TensorFlow.‖ [Online]. Available: https://www.tensorflow.org/ [Accessed: 01-May-2017].
• [132] ―Deeplearning4j: Open-source, Distributed Deep Learning for the JVM.‖ [Online]. Available: https://deeplearning4j.org/ [Accessed: 01-May-2017].
• [133] Deniz Yuret, ―Welcome to Knet.jl’s documentation!‖ [Online]. Available: http://denizyuret.github.io/Knet.jl/latest/. Accessed: 17-Aug-2017].
• [134] Deniz Yuret, ―Julia ve Knet ile Derin Öğrenmeye Giriş,‖ 2016. [Online]. Available: http://www.denizyuret.com/2016/09/julia-ve-knetile-derin-ogrenmeye-giris.html. [Accessed: 17-Aug-2017].