Enhanced Sequence Modeling with Deep Learning

Abstract text:

Enhanced Sequence Modeling with Deep Learning

Bálint Gyires-Tóth1

1 Budapest University of Technology and Economics, Department of Telecommunications and Media Informatics

Deep learning based sequence modeling has shown superior results to previous analytical and machine learning methods. Deep learning paradigm provides several approaches to create robust models for sequential data. Time delay and recurrent neural networks, including Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) based models, are intended to be used for such purpose. Convolutional neural networks (CNNs) have also achieved outstanding results in sequence modeling lately. The CNN’s structure allows efficient training of very deep neural networks – even with more than hundred layers –, which are likely to have better performance than shallow models. The WaveNet, introduced by DeepMind, have utilized some novel ideas in CNNs and thus achieved state of the art results in time-domain regression of audio signals (eg. music and speech). Inspired by WaveNet, similar architectures resulted SOTA solutions in various sequence modeling tasks, like speech recognition and natural language processing.

In this presentation, the fundamentals of sequence modeling with recurrent and convolutional neural networks are introduced and the theoretical and practical details of WaveNet type architectures are discussed.

Bálint Gyires-Tóth

End of talk: 6/22/2018 9:40:00 AM

Start of talk: 6/22/2018 9:10:00 AM

Abstract

Slides