Yichen Peng

I am a research assistant professor at the Koike Lab, Institute of Science Tokyo (ex. Tokyo Institute of Technology), working with Prof. Koike Hideki, and Prof. Erwin Wu. I am also a visiting scholar at Keio University working with Prof. Kai Kunze. Currently, I am visiting KLab, advised by Prof. Kris Kitani, Carnegie Mellon University. My research focuses on computer vision, human motion understanding, and multimodal generative models, with applications in skill training and human motion generation.

I received my Master and Ph.D. in Computer Science from the Japan Advanced Institute of Science and Technology (JAIST) in 2021, 2024 respectively, supervised by Prof. Kazunori Miyata. (Funded by JST-SPRING) I worked closely with Prof. Haoran Xie, and Prof. Tsukasa Fukasato in Waseda University, on sketch-based 2D/3D Generation & Interface.

GitHub  /  Google Scholar  /  LinkedIn  /  CV

Email: yichen.p.8896[at]m.isct.ac.jp

Research Interests

My research focuses on computer vision, human motion understanding (Pose Estimation/Motion Capture), and multimodal generative models, with applications in skill training(Piano/Skiing/Golf) and human motion generation (Speech/Conversation Gesture). I am also interested in sketch-based 2D/3D Generation & Interface.

Publications

PiaMuscle: Improving Piano Skill Acquisition by Cost-effectively Estimating and Visualizing Activities of Miniature Hand Muscles

LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Model

Dual-modal 3d human pose estimation using insole foot pressure sensors

Emage: Towards unified holistic co-speech gesture generation via expressive masked audio gesture modeling

Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis

DiffFaceSketch: Sketch-guided latent diffusion model for high-fidelity face image synthesis

Dualmotion: Global-to-local casual motion design for character animations

Design and source code from Jon Barron's website