LEARNING WITH MEMORY - FROM RECURRENT NEURAL NETWORKS TO MEMORY AUGMENTED NEURAL NETWORKS
ABSTRACT |
| Neural networks with memory capabilities have been introduced to solve several machine learning problems which require to model sequential data such as time series. The most common models are Recurrent Neural Networks and their variants such as Long-Short Term Memories (LSTM) and Gated Recurrent Units (GRU). More recently, alternative solutions exploiting Transformers or Memory Augmented Neural Networks (MANN) have been suggested to overcome the limitations of RNNs. They have distinguishing characteristics that make them substantially different from the early models. The former directly compares the elements in the input sequence through self-attention mechanisms. The latter uses an external element-wise addressable memory. Differently from RNNs, in MANNs, state to state transitions are obtained through read/write operations and a set of independent states is maintained. An important consideration of Memory Augmented Networks the number of parameters is not tied to the size of the memory. They have been defined both with episodic and permanent external memories. The course will discuss these new memory networks models and their applications. |
SPEAKERS |
|
Federico Becattini (federico.becattini@unifi.it) obtained his PhD in 2018 from the University of Florence under the supervision of Prof. Alberto Del Bimbo and Prof. Lorenzo Seidenari. Currently he is a PostDoc at MICC, where he is involved in numerous collaborations, mostly focusing on Autonomous Driving and Scene Understanding. He attended several international conferences both as speaker and volunteer, as well as summer schools. He served to the scientific community as a reviewer for scientific journals and conferences.
|
