ÇмúÇà»ç

Dr. Nan Duan

Principal Research Manager, Microsoft Research

Professor Kee-Eung Kim, ±è±âÀÀ ±³¼ö

KAIST, Çѱ¹°úÇбâ¼ú¿ø

Abstract: Recent advances in deep reinforcement learning have shown remarkable achievements. We now have reinforcement learning agents that surpass human expert level in many tasks, such as arcade video games and go. Yet, these are all achieved in simulated environments, where trial-and-error is regarded as cheap. In order to make a tangible impact on real-world environments, this basic assumption behind reinforcement learning has to be addressed with top priority. Fortunately, there is a growing body of research on this matter, exemplified by approaches such as sim2real and off-line reinforcement learning. In this talk, I will go over the challenges that need to be addressed to make reinforcement learning practical in real-world scenarios, and present our work on off-line reinforcement learning inspired by cross-validation in supervised learning.

Bio: Kee-Eung Kim received the B.S. degree in computer science from KAIST, Daejeon, South Korea, in 1995, and the ScM and Ph.D. degrees in computer science from Brown University, Providence, RI, USA, in 1998 and 2001, respectively. From 2001 to 2006, he was a Senior Software Engineer with Samsung SDS, South Korea, and a Senior Research Staff Member with the Samsung Advanced Institute of Technology, South Korea. In 2006, he joined the Faculty of Computer Science Department with KAIST, where he is currently a Professor of Graduate School of AI. His research interests are representations and algorithms for sequential decision making problems in artificial intelligence and machine learning, including Markov decision processes and reinforcement learning.

Title: Learning to represent data across domains

Professor Seunghoon Hong, È«½ÂÈÆ ±³¼ö

KAIST, Çѱ¹°úÇбâ¼ú¿ø

Abstract: How can we describe noisy data using a compact and meaningful set of features? Learning to represent data has been the core problem of machine learning, and has achieved remarkable progress in various data domains, such as images, languages, audio signals, etc. However, learning to align such representations ‘across’ multiple heterogeneous domains still remains an open problem, although it is essential to learn from multi-modal input sources (e.g., observations taken from multiple sensors). In this talk, I will introduce the problem of cross-domain disentanglement learning, and our recent work that enable unsupervised training of aligned representations across domains.

Bio: Seunghoon Hong is an assistant professor at the School of Computing, KAIST. Before joining KAIST, he had been a postdoctoral fellow at the University of Michigan and visiting research faculty at Google Brain team. His research interests lie in the intersection of machine learning and computer vision, with a specific focus on learning with least supervision and deep generative models. He received the B.S. and Ph.D. degree from the Department of Computer Science and Engineering at POSTECH, Pohang, Korea in 2011 and 2017, respectively.

Title: Deep Diffusion-Invariant Wasserstein Distributional Classification

Professor Junseok Kwon, ±ÇÁؼ® ±³¼ö 

Chung-Ang University, Áß¾Ó´ëÇб³

Abstract: We present a novel classification method called deep diffusion-invariant Wasserstein distributional classification (DeepWDC). DeepWDC represents input data and labels as probability measures to address severe perturbations in input data. It can output the optimal label measure in terms of diffusion invariance, where the label measure is stationary over time and becomes equivalent to a Gaussian measure. Furthermore, DeepWDC minimizes the 2-Wasserstein distance between the optimal label measure and Gaussian measure, which reduces the Wasserstein uncertainty. Experimental results demonstrate that DeepWDC can substantially enhance the accuracy of several baseline deterministic classification methods and outperforms state-of-the-art-methods on 2D and 3D data containing various types of perturbations (e.g., rotations, impulse noise, and down-scaling).

Bio: Junseok Kwon received the PhD degree in electrical engineering and computer science in 2013, the MSc degree in 2008 in the topic of object tracking (supervised by Prof. Kyoung Mu Lee), and the BSc degree in 2006 from the Seoul National University, Korea. He is an associate professor in the School of Computer Science and Engineering at Chung-Ang University, Seoul, Korea. He was a postdoctoral researcher under Prof. Luc Van Gool in the Computer Vision Laboratory at ETH Zurich from 2013 to 2014. His research interests include visual tracking, visual surveillance, and Monte Carlo Sampling method and its variants.

Title: MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures

Professor Sungju Hwang, Ȳ¼ºÁÖ ±³¼ö 

KAIST, Çѱ¹°úÇбâ¼ú¿ø

Abstract: Regularization and transfer learning are two popular techniques to enhance model generalization on unseen data, which is a fundamental problem of machine learning. Regularization techniques are versatile, as they are task- and architecture-agnostic, but they do not exploit a large amount of data available. Transfer learning methods learn to transfer knowledge from one domain to another, but may not generalize across tasks and architectures, and may introduce new training cost for adapting to the target task. To bridge the gap between the two, we propose a transferable perturbation, MetaPerturb, which is meta-learned to improve generalization performance on unseen data. MetaPerturb is implemented as a set-based lightweight network that is agnostic to the size and the order of the input, which is shared across the layers. Then, we propose a meta-learning framework, to jointly train the perturbation function over heterogeneous tasks in parallel. As MetaPerturb is a set-function trained over diverse distributions across layers and tasks, it can generalize to heterogeneous tasks and architectures. We validate the efficacy and generality of MetaPerturb trained on a specific source domain and architecture, by applying it to the training of diverse neural architectures on heterogeneous target datasets against various regularizers and fine-tuning. The results show that the networks trained with MetaPerturb significantly outperform the baselines on most of the tasks and architectures, with a negligible increase in the parameter size and no hyperparameters to tune.

Bio: Sung Ju Hwang is an associate professor in the Graduate School of Artificial Intelligence and School of Computing at KAIST. Prior to working at KAIST, he was an assistant professor in the School of Electric and Computer Engineering at UNIST, and before that he was a postdoctoral research associate at Disney Research. He completed his Ph.D. study in computer science at University of Texas at Austin, under the supervision of Professor Kristen Grauman. His research interests are mainly on developing novel models and algorithms for tackling new challenges in deploying artificial intelligence systems to various real-world application domains, with the special focus on low-resource learning, on-device learning, safe/secure learning, and large-scale learning. Sung Ju Hwang is also a co-founder of AItrics, an AI startup company located in Seoul.

Professor Taesup Moon, ¹®Å¼· ±³¼ö

Sungkyunkwan University, ¼º±Õ°ü´ëÇб³ 

Abstract: In this talk, I will present a recently developed novel regularization-based continual learning method, dubbed as Adaptive Group Sparsity based Continual Learning (AGS-CL), using two group sparsity-based penalties. Our method selectively employs the two penalties when learning each node based on its importance, which is adaptively updated after learning each new task. By utilizing the proximal gradient descent method for learning, the exact sparsity and freezing of the model is guaranteed, and thus, the learner can explicitly control the model capacity as the learning continues. Furthermore, as a critical detail, we re-initialize the weights associated with unimportant nodes after learning each task in order to prevent the negative transfer that causes the catastrophic forgetting and facilitate efficient learning of new tasks. Throughout the extensive experimental results, we show that our AGS-CL uses much less additional memory space for storing the regularization parameters, and it significantly outperforms several state-of-the-art baselines on representative continual learning benchmarks for both supervised and reinforcement learning tasks. This work will be also presented in the upcoming NeurIPS 2020 conference.

Bio: Taesup Moon received the B.S. degree in electrical engineering from Seoul National University, Seoul, Korea, in 2002 and the M.S. and Ph.D. degrees in electrical engineering from Stanford University, Stanford, CA, USA, in 2004 and 2008, respectively. From 2008 to 2012, he was a Scientist at Yahoo! Labs, Sunnyvale, CA, and he held a Post-Doctoral Researcher appointment with the Department of Statistics, UC Berkeley from 2012 to 2013. From 2013 to 2015, he was a Research Staff Member with Samsung Advanced Institute of Technology (SAIT), and from 2015 to 2017, he was an Assistant Professor at the Department of Information and Communication Engineering, Daegu-Gyeongbuk Institute of Science and Technology (DGIST). Currently, he is an Associate Professor at the Department of Electrical and Computer Engineering, Sungkyunkwan University (SKKU), Suwon, South Korea. His current research interests are in machine/deep learning, signal processing, information theory, and various (big) data science applications.

Title: Towards machines that reconstruct and understand 3D scenes

Professor Jaesik Park, ¹ÚÀç½Ä ±³¼ö

POSTECH, Æ÷Ç×°ø°ú´ëÇб³

Abstract: Recovering and understanding a 3D scene is an essential task for artificial intelligence systems to interact with the real-world. This talk will cover my recent work regarding 3D reconstruction, learning-based geometric features, 3D semantic segmentation, and point cloud registration. Moreover, the talk will introduce the new version of Open3D equipped with a machine learning module.

Bio: Jaesik Park is an Assistant Professor at POSTECH. He received his Bachelor’s degree from Hanyang University in 2009, and he received his Master’s degree and Ph.D. degree from KAIST in 2011 and 2015, respectively. He worked at Intelligent Systems Lab in Intel as a Research Scientist. His research interests cover various 3D vision topics. He is a co-founder of Open3D and a recipient of the Microsoft Research Asia Fellowship and two Samsung Humantech Awards. He serves as an Area Chair for ICCV 2019, CVPR 2020, CVPR 2021.

Title: Learning architectures for binary networks and beyond

Professor Jonghyun Choi, ÃÖÁ¾Çö ±³¼ö

GIST, ±¤ÁÖ°úÇбâ¼ú¿ø

Abstract: The large computational complexity of the algorithms is one of the bottlenecks to deploy AI systems such as visual recognition models. Neural networks with 1-bit precision would have very high memory efficiency with computational benefits. I will introduce a few ideas to make better architectures for binary networks using search. In addition, we introduce simpler training methods for the searched better architectures. We show that the searched architectures outperform state of the art binary networks and the simpler training regime further improves the utilization of the smaller networks outperforming the state of the binary networks with higher FLOPs.

(Relevant paper link: https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123570562.pdf)

Bio: Jonghyun Choi is an assistant professor in the AI graduate school/EECS in GIST (Gwangju Institute of Science and Technology). Prior to GIST, he has been a research scientist in Allen Institute for Artificial Intelligence (AI2) in 2016-2018 and currently affiliated research scientist there. He got a PhD degree from University of Maryland, College Park in 2015 with distinguished dissertation fellowship under the supervision of Prof. Larry S. Davis. His research interest includes supervision- and resource-efficient computer vision and learning methods, and high level visual reasoning.

Title: Mixture Density Object Detector: LEARNING BOUNDING BOXES WITHOUT GROUND TRUTH ASSIGNMENT

Professor Nojun Kwak, °û³ëÁØ ±³¼ö 

Seoul National University, ¼­¿ï´ëÇб³

Abstract: In multi-object detection using neural networks, most methods train a network based on ground truth assignment, which makes the training too heuristic and complicated. In this paper, we reformulate the multi-object detection task as a problem of density estimation of bounding boxes. Instead of using a ground-truth-assignment-based method, we train a network by estimating the probability density of bounding boxes in an input image using a mixture model. For this purpose, we propose a novel network for object detection called Mixture Density Object Detector (MDOD), and the corresponding objective function for the density-estimation-based training. Unlike the ground-truth-assignment-based methods, our proposed method gets rid of the cumbersome processes of matching between ground truth boxes and their predictions as well as the heuristic anchor design. It is also free from the problem of foreground-background imbalance. We applied MDOD to MS COCO dataset. Our proposed method not only deals with multi-object detection problems in a new approach but also improves detection performances through MDOD.

Bio: NOJUN KWAK received the B.S., M.S., and Ph.D. degrees in electrical engineering and computer science from Seoul National University, Seoul, in 1997, 1999, and 2003, respectively. From 2003 to 2006, he was with Samsung Electronics, Seoul. From 2007 to 2013, he was a Faculty Member with the Department of Electrical and Computer Engineering, Ajou University, Suwon, South Korea. Since 2013, he has been with the Graduate School of Convergence Science and Technology, Seoul National University, where he is currently a Professor. His current research interests include feature learning by deep neural networks and their applications in various areas of pattern recognition, computer vision, and image processing.

Title: Neural Motion Feature Learning for Video Understanding

Professor Minsu Cho, Á¶¹Î¼ö ±³¼ö

POSTECH, Æ÷Ç×°ø°ú´ëÇб³

Abstract: Motion plays a crucial role in understanding videos and most state-of-the-art neural models for video classification incorporate motion information typically using optical flows extracted by a separate off-the-shelf method. As the frame-by-frame optical flows require heavy computation, incorporating motion information has remained a major computational bottleneck for video understanding. In this work, we replace external and heavy computation of optical flows with internal and light-weight learning of motion features. We propose a trainable neural module for effective motion feature extraction. Inserted in the middle of any neural network, it learns to establish correspondences across frames and convert them into motion features, which are readily fed to the next downstream layer for better prediction. We demonstrate that the proposed method provides a significant gain on four standard benchmarks for action recognition with only a small amount of additional cost, outperforming the state of the art on Something-Something-V1&V2 datasets.

Bio: Minsu Cho is an associate professor of Computer Science and Engineering at POSTECH, South Korea, leading POSTECH Computer Vision Lab. Before joining POSTECH in the fall of 2016, he has worked as a postdoc (2012-2015) and a starting researcher (2015-2016) in Inria (the French National Institute for computer science and applied mathematics) and ENS (École Normale Supérieure), Paris, France. He completed his Ph.D. in 2012 at Seoul National University, Korea. His research lies in the areas of computer vision and machine learning, especially in the problems of visual correspondence, action recognition, and object recognition with minimal supervision. In general, he is interested in the relationship between correspondence and supervision in visual learning. He is an editorial board member of the International Journal of Computer Vision and is serving area chairs in top computer vision conferences including CVPR 2018, ICCV 2019, CVPR 20-2021, IJCAI 2021.

Title: Neural Interface for World Knowledge

Abstract: Modern natural language and machine learning problems are increasingly dependent on external world knowledge, and thus how we encode and interact with knowledge data has become very important as well. In this talk, I will discuss different approaches in the literature for handling world knowledge, and in particular, our recent work on creating a memory interface for accessing massive unstructured knowledge data. I will then argue both observed and hypothetical advantages of such a memory-based approach, especially how it can help us to perform more complex interaction with world knowledge including language reasoning.

Bio: Minjoon Seo is an Engineering Manager at Naver and an incoming Assistant Professor at KAIST Graduate School of AI. He finished his Ph.D. at the University of Washington, advised by Hannaneh Hajishirzi and Ali Farhadi. His research interest is in natural language processing and machine learning, and in particular, how knowledge data can be encoded (e.g. external memory and language model), accessed (e.g. question answering and dialog), and produced (e.g. scientific reasoning). His study was supported by Facebook Fellowship and AI2 Key Scientific Challenges Award. He previously co-organized MRQA 2018, MRQA 2019 and RepL4NLP 2020.

Professor Kyomin Jung, Á¤±³¹Î ±³¼ö

Seoul National University, ¼­¿ï´ëÇб³

Abstract Recently various Deep Neural Network (DNN) models have shown excellence in question answering (QA) tasks. In this talk, I will present basic ideas of QA systems with DNN models, and present our work on them. Specifically we investigate the human-like QA system that generates answers in the natural language form in aligning with the given context. Furthermore, our research aims to fully incorporate external knowledge to the QA system for integrating deep neural network-based models and human-generated expertise knowledge-base. To this end, we have developed model frameworks that detect supporting sentences in knowledge-based QA.

Bio: Kyomin Jung is an associate professor in the Electrical and Computer Engineering department at Seoul National University (SNU), and has a joint appointment in the Mathematics department SNU. He worked at KAIST Computer Science department from 2009 to 2013. He received Ph.D.

at MIT department of Mathematics in 2009, and B.Sc. at Seoul National University department of Mathematics in 2003. During his Ph.D., he worked at Microsoft Research Cambridge (2008), IBM T.J. Watson Research (2007), and Bell Labs (2006) as research internships. He is a recipient of the excellent new faculty funding from NRF Korea in 2012.

His research area includes machine learning and natural language processing, and their applications to various services. His work on machine learning and natural language processing have been published in major journals and conferences including IEEE PAMI, NIPS, ICML, AAAI, ACL, NAACL, ICDM, SIGMETRICS, and ICDE.

Professor Gunhee Kim, ±è°ÇÈñ ±³¼ö

Seoul National University, ¼­¿ï´ëÇб³

Abstract: We explore the task of improving persona consistency of dialogue agents. Recent models tackling consistency often train with additional Natural Language Inference (NLI) labels or attach trained extra modules to the generative agent for maintaining consistency. However, such additional labels and training can be demanding. Also, we find even the best-performing persona-based agents are insensitive to contradictory words. Inspired by social cognition and pragmatics, we endow existing dialogue agents with public self-consciousness on the fly through an imaginary listener. Our approach, based on the Rational Speech Acts framework (Frank and Goodman, 2012), can enforce dialogue agents to refrain from uttering contradiction. We further extend the framework by learning the distractor selection, which has been usually done manually or randomly. Results on Dialogue NLI (Welleck et al., 2019) and PersonaChat (Zhang et al., 2018) dataset show that our approach reduces contradiction and improves consistency of existing dialogue models. Moreover, we show that it can be generalized to improve context-consistency beyond persona in dialogues.

Bio: Gunhee Kim is an associate professor in the Department of Computer Science and Engineering of Seoul National University from 2015. He was a postdoctoral researcher at Disney Research for one and a half years. He received his PhD in 2013 under supervision of Eric P. Xing from Computer Science Department of Carnegie Mellon University. Prior to starting PhD study in 2009, he earned a master’s degree under supervision of Martial Hebert in Robotics Institute, CMU. His research interests are solving computer vision and web mining problems that emerge from big image data shared online, by developing scalable and effective machine learning and optimization techniques. He is a recipient of 2014 ACM SIGKDD doctoral dissertation award, and 2015 Naver New faculty award.

Title: Conversation Model and its Evaluation

Professor JinYeong Bak, ¹ÚÁø¿µ ±³¼ö

Sungkyunkwan University, ¼º±Õ°ü´ëÇб³

Abstract: Many neural network-based open-domain conversation models have seen successes in recent years. Despite these recent successes, the open-domain conversation models still have challenges to imitate human-level conversations. One of the challenges is the consideration of speakers in the conversations. Knowing information about a speaker, such as her linguistic style or personal information can help predict her response, and knowing more about both speakers from their previous conversations can help predict the contents of their conversation. I focus on modeling speakers and their behaviors in the conversations that enable the conversation model to generate human-like responses. In this talk, I will introduce two recent research projects on conversation modeling: generating personalized conversational responses (EMNLP 2019) and evaluating conversational responses without human-labeled data (ACL 2020). I will also show future directions of my research to build conversation models and its evaluation.

Bio: JinYeong Bak is an assistant professor in the colleague of computing at Sungkyunkwan University. His research interests analyzing human behaviors from their conversations and building open-domain conversation models from the insights of the analysis. He worked at Microsoft Research Asia as a research intern and United Nations Global Pulse Lab Jakarta as a junior data scientist. He received Ph.D. and M.S. from the KAIST and a B.S. from Sungkyunkwan University. His research has been published in ACL, EMNLP, CHI, and ICWSM. His personal homepage: https://nosyu.github.io and lab homepage: https://hli.skku.edu

Professor Hong-Goo Kang, °­È«±¸ ±³¼ö

Yonsei University, ¿¬¼¼´ëÇб³

Abstract: Speech is one of the most effective media to control or communicate with digital devices. The main goal of speech interface research is to provide naturalness similar to human-to-human communication. Thanks to the advances in deep learning-based technology, speech interface systems can be applied for various applications such as AI speakers, robots, and so on. This talk briefly explains the core modules for natural speech interface, then introduces recent research trends and directions, from a perspective of speech enhancement and speech synthesis techniques.

Bio: Hong-Goo Kang is a professor of Electrical and Electronic engineering at Yonsei University. Prior to joining Yonsei, he had been a Senior Technical Staff Member at AT&T Labs-Research (Florham Park, NJ) for 6.5 years, after PhD from Yonsei. In 2008~20009 and 2015~2016, respectively, he worked for Broadcom (Irvine, CA) and Google (Mountain View, CA) as a visiting scholar, where he participated in various projects on speech signal processing. His research interests include speech/audio signal processing, machine learning, and human computer interface.

Title: A REVIEW OF ON-DEVICE FULLY NEURAL END-TO-END AUTOMATIC SPEECH RECOGNITION ALGORITHMS

Dr. Chanwoo Kim, ±èÂù¿ì »ó¹« 

Samsung Electronics, »ï¼ºÀüÀÚ

Abstract: In this presentation, we review various end-to-end automatic speech recognition algorithms and their optimization techniques for on-device applications. Conventional speech recognition systems comprise a large number of discrete components such as an acoustic model, a language model, a pronunciation model, a text-normalizer, an inverse-text normalizer, a decoder based on a Weighted Finite State Transducer (WFST), and so on. To obtain sufficiently highspeech recognition accuracy with such conventional speech recognition systems, a very large language model (up to 100 GB) is usually needed. Hence, the corresponding WFST size becomes enormous, which prohibits their on-device implementation. Recently, fully neural network end-to-end speech recognition algorithms have been proposed. Examples include speech recognition systems based on Monotonic Chunk-wise Attention (MoCha), Recurrent Neural Network Transducer (RNN-T), transformer-based speech recognition systems, and so on. These fully neural network-based systems require much smaller memory footprints compared to conventional algorithms, therefore their on-device implementation has become feasible. In this paper, we review such end-to-end speech recognition models. We extensively discuss their structures, performance, and advantages compared to conventional algorithms.

Bio: Dr. Chanwoo Kim has been a vice president at Samsung research leading the speech processing Lab since Feb. 2018. He was a senior software engineer at Google speech team between Feb. 2013 and Feb. 2018. He worked for acoustic modeling for speech recognition systems and enhancing noise robustness using deep learning techniques. He was a speech scientist at the Microsoft Speech Team from 2011 to 2013. Dr. Kim received his Ph. D. from the Language Technologies Institute of School of Computer Science Carnegie Mellon University in 2010. He received his B.S and M.S. degrees in Electrical Engineering from Seoul National University in 1998 and 2001, respectively. Dr. Kim’s doctoral research was focused on enhancing the robustness of automatic speech recognition systems in noisy environments. Between 2003 and 2005 Dr. Kim was a Senior Research Engineer at LG Electronics, where he worked primarily on embedded signal processing and protocol stacks for multimedia systems. Prior to his employment at LG, he worked for EdumediaTek and SK Teletech as a R&D engineer.