Title: Vision-centric Multi-modal Learning: Introduction and Applications
People live by interacting with the world through signals sensed by the five senses. Ultimately, in order to develop artificial intelligence that can communicate and empathize at the same level as humans, multimodal machine perception and understanding like humans must be developed as a key stepping stone.
This tutorial first introduces multimodal learning setups, and presents the development patterns to incorporate multimodal information, such as vision, sound, and natural language data. Also, this tutorial showcases a few interesting recent applications.
Tae-Hyun Oh is an assistant professor with Electrical Engineering (adjunct with Graduate School of AI and Dept. of Creative IT Convergence) at POSTECH, South Korea. He was jointly affiliated with OpenLab, POSCO-RIST, South Korea, as a research director in 2021-2023. He received the B.E. degree (First class honors) in Computer Engineering from Kwang-Woon University, South Korea in 2010, and the M.S. and Ph.D. degrees in Electrical Engineering from KAIST, South Korea in 2012 and 2017, respectively.
Before joining POSTECH, he was a postdoctoral associate at MIT CSAIL, Cambridge, MA, US, and was with Facebook AI Research, Cambridge, MA, US. He was a research intern at Microsoft Research in 2014 and 2016. He serves as an associate editor for the Visual Computer journal. He was a recipient of Microsoft Research Asia fellowship, Samsung HumanTech thesis gold award, Qualcomm Innovation awards, top research achievement awards from KAIST, and CVPR'20 and ICLR'22 outstanding reviewer awards.
Title: 3D Generation via 2D Priors and Neural Rendering: Recent Advances, Limitations, and Future Directions
Generative models for texts and images have gained widespread popularity due to their remarkable advancements in producing highly realistic outputs. Meanwhile, the challenge of generating 3D content remains unresolved primarily due to limited availability of large-scale data. However, recent attempts to generate 3D using 2D priors have yielded surprising success, although some failure cases persist. In this tutorial, I will provide an overview of the recent advances in 3D generation using 2D priors, explaining the fundamental building blocks in the pipeline: NeRF, Diffusion Models, and Score Distillation Loss. Additionally, we will discuss the limitations of current pipelines and outline potential future research directions aimed at overcoming these challenges.
Minhyuk Sung is an assistant professor in the School of Computing at KAIST, affiliated with the Graduate School of AI and the Metaverse Program. Before joining KAIST, he was a Research Scientist at Adobe Research. He received his Ph.D. from Stanford University under the supervision of Professor Leonidas J. Guibas. His research interests lie in vision, graphics, and machine learning, with a focus on 3D geometric data processing. His academic services include serving as a program committee member in Eurographics 2022, SIGGRAPH Asia 2022 and 2023, and AAAI 2023.
Title: Causal Inference
To achieve desired outcomes, it is essential for an intelligent agent to possess the capability of utilizing accessible data to make rational decisions. In order to do so effectively, the agent must go beyond merely estimating associations found in the given data and be capable of inferring the consequences of its actions. Causality-focused researchers in the field of AI have long been exploring formal representations of observations and experiments, as well as mathematical methods for deriving the effects of actions. In this tutorial, I will provide an introduction to the fundamental concept of causal inference and discuss its application to some problems in AI and ML.
Sanghack Lee is an Assistant Professor at the Graduate School of Data Science at Seoul National University. Prior to that, he was an Associate Research Scientist at Columbia University, working with Professor Elias Bareinboim on the intersection of causal inference and AI. Lee received his Ph.D. in Information Sciences and Technology from Pennsylvania State University. His research focuses on developing techniques for (1) unifying data sets from various conditions to answer causal questions, (2) uncovering causal relationships from various data types, and (3) creating decision-making algorithms that incorporate causal domain knowledge. He was the recipient of a Best Paper Award at the UAI conference in 2019.
Title: AI Fairness: Model-centric and Data-centric Approaches
AI fairness is a critical element in Responsible AI that needs to be supported in all steps of machine learning. In this tutorial, we study various notions of AI fairness and cover recent model-centric and data-centric approaches for mitigating unfairness. The data-centric approaches are not only studied in the AI community, but also in the data management community. We then discuss other recent trends in AI fairness.
Steven Euijong Whang is an associate professor at KAIST EE and AI. His research interests include Data-centric AI and Responsible AI. Previously he was a Research Scientist at Google Research and co-developed the data infrastructure of the TensorFlow Extended (TFX) machine learning platform. Steven received his Ph.D. in computer science in 2012 from Stanford University and his B.S. in computer science from KAIST in 2003. He received a Google AI Focused Research Award (2018, the first in Asia) and was a Kwon Oh-Hyun Endowed Chair Professor (2020-2023).
Title: Large Language Models: Introduction and Recent trends
In this tutorial, I will introduce the recent advances in the field of natural language processing regarding the development of large language models. The focus will be on exploring the methodology of pretraining and fine-tuning, as well as prompting-based approaches, to utilize these large language models effectively. Additionally, we discuss other recent trends in this field including symbolic distillation, neural theory-of-mind, and memory augmentation.
Jinyoung Yeo is an assistant professor at Yonsei University in the AI and CS department. He also holds the position of Chief AI Officer at Market Designers Inc., where he is involved in the development of AI tutors for English education. He earned his Ph.D. in Computer Science and Engineering from POSTECH. His research primarily revolves around the advancement of Natural Language Processing techniques, with a particular emphasis on Dialogue Agents and Commonsense Reasoning.
이재호 교수 (포항공과대학교)
Title: Recent Trends in Neural Data Compression: From Generative Models to Neural Fields
Neural data compression aims to automate the design of data compression algorithms by modeling the encoder and decoder using neural networks and training them with some dataset. In this two-part tutorial, I give a general overview of neural data compression. In part 1, I introduce basic concepts and algorithms for neural data compression. In part 2, I describe recent trends in neural data compression for visual signals, from the approaches that use generative models for giving highly realistic reconstructions to the methods that use neural fields for a modality-agnostic compression paradigm.
Jaeho Lee is an assistant professor at the electrical engineering department of POSTECH. He leads a research group, called Efficient Learning Lab, which focuses on developing theories and algorithms for making machine learning practices more efficient in terms of computation or memory. Before joining POSTECH, he was a postdoctoral researcher at KAIST, working with Prof. Jinwoo Shin. Even before that, he was at the lovely cornfields of the University of Illinois at Urbana-Champaign, where he completed his graduate studies under the supervision of Prof. Maxim Raginsky.