On-Device AI - What and Why? (introductory video)
Artificial Intelligence, Internet of Things, and Edge Computing, all happy together
- Small IoT devices, wearables, and smartphones do not only sense its environment but also analyze the data and make decisions all by itself by performing AI computations.
Less dependence on the cloud, more intimate interaction with users
- User data is consumed locally, which addresses both privacy and network bandwidth issues.
- A user receives a response to her actions in real time from a nearby edge device, instead of from a cloud far away.
Enablers
- Low-power AI accelerators, lightweight software platforms, model compression techniques, adaptive and federated learning all enable the magic to happen on the edge.
Adaptive, Compressed, Unsupervised and Federated ML
Training an ML model without sharing raw data and labels. After training, the model should be compressed and adapted for a test domain.
The number of IoT sensors around us is growing rapidly, making various types of data born in edge devices. The amount of data in a small edge device, however, is not big enough to train an ML model. Simply sending all raw data from the edge to the cloud can generate big data for training, but this causes privacy issues. We investigate how to make an efficient and secure distributed learning framework by using multiple constrained data sources.
ImageNet-ES: Dataset to explore domain shifts via environment and sensor factors
MEBQAT: Meta learning for adaptive quantization
HypeMeFed: Federated learning with Resource Heterogeneity
UpCycling: Semi-supervised 3D object detection without Raw data
FedSIM: Semi-supervised, Meta federated learning
On-device AI Systems and Applications
ML model is a cool "module," but we want "holistic" application systems.
Recent technology development has brought big data, cloud, edge AI chips, IoT sensors, and learning techniques. Given lots of powerful hammers, we investigate how to orchestrate them to enable interesting and useful applications on the edge. In this perspective, it is important to deeply analyze characteristics of applications, sensors and systems and determine when/where to execute AI algorithms for efficient operation without sacrificing user experiences.
SleepXViT: Explainable Steep Staging
MIRROR: On-device Generative AI for Fashion
SlAction: On-device sleep video analytics
PointSplit: On-device 3D object detection
ScripPainter: Vision-based on-device software testing
MARVEL / SnapLink: Cloud-edge joint design for Mixed Reality
MiCrowd: Vision-based on-MCU Crowd counting
Funded Projects
On-going
Center for Optimizing Hyperscale AI Models and Platforms, NRF Engineering Research Center (ERC), 2023.06 - 2030.02.
Joint Design of Application, Deep Learning and Systems for On-device Deep Video Understanding, PI, NRF Young Researcher (selected as Innovative Research Lab), 2023.03 - 2028.02.
Development of Beyond X-verse Core Technology for Hyper-realistic Interactions by Synchronizing the Real World and Virtual Space, IITP, 2023.01 - 2027.12.
Development of the Artificial Intelligence Technology to Enhance Individual Soldier Surveillance Capabilities, IITP, 2023.04 - 2026.12.
Interpretation of Sleep BioSignals based on Artificial Intelligence, SNU AI-Bio Convergence Research (ABC), 2023. 03 - 2026.02.
Performance Optimization of Federated Learning via Efficient Client Selection in Open-Set Environments, PI, Samsung Electronics, 2024.09 - 2025.08.
Ambient Healthcare: IoT-based Personalized Edge AI System for Remote Patient Monitoring, PI, SNU Creative-Pioneering Researcher, 2022.08 - 2025.06.
Completed
Quantization-Aware Domain Adaptation for Neural Networks, PI, Google, 2023.10 - 2024.09.
Research on Federated Learning with Unclassifiable Data, PI, Samsung Electronics, 2023.09 - 2024.08.
Research on Distributed Learning and Extended-vision based 3D Object Detection Model for Autonomous Driving in 5G Networks, NRF Mid-career Researcher, 2020.09 - 2023.02
Stabilization of Intelligent Marine Transportation System, MOF, 2022.08 - 2023.02.
Self-collected Sleep Data Construction for Monitoring Sleep and Pumonary Disorders based on Artificial Intelligence, NIA Dataset Construction, 2022.05 - 2022.12
Research on Floating Population Estimation using IoT Sensors on Electric Scooters, PI, MaaS Asia, 2021.07 - 2022.06
Development of Structured/Unstructured Data Analysis Model for Advanced DA/BI Services, PI, Hyperlounge, 2021.10 - 2022.02
Research on IoT-based Ambient Artificial Intelligence, PI, SNU New Faculty Startup, 2020.03 - 2021.12
Development of Edge AI Model for SK Magic IoT Devices, SK Networks, 2020.11 - 2021.10
Development of Real-time Detection Algorithm based on Time-series Data, PI, SK Hynix, 2020.12 - 2021.09