Erzhen Hu

I am a Ph.D. candidate in Computer Science at the University of Virginia, advised by Prof. Seongkook Heo. Before that, I obtained M.S. in Statistics from University of Virginia, and B.A. in Sociology from Shanghai University.

I have had the fortunate opportunity to intern with amazing researchers at Microsoft Research and Google. My work has been supported by Google PhD Fellowship.

About

As we approach the everyday use of virtual world, my research explores how to enable more flexible and frictionless way of connecting with people in virtual and hybrid spaces.

My research includes exploring Human-AI communication paradigms by augmenting human-human interactions through multi-modal agents, leveraging LLMs, and applying advanced 2D and 3D computer vision methods for multi-user scenarios, video-mediated interactions and XR applications. I also investigate theory-driven approaches to improve spontaneous and structured conversations, and turn-taking dynamics in small to medium-sized groups.

Updates

Jun 2024: I have passed the Qualifying Exam and have become a Ph.D. candidate!
Jan 2024: Excited to start a research internship as Student Researcher at Google.
May 2023: Excited to start an internship as Research Intern at Microsoft Research.
Mar 2023: Presented OpenMic at the Video Sharing session and ThingShare at the Human-AI collaboration session on Apr 25th (Tue). Check out the CHI 2023 program!
Jan 2023: Two first-author papers accepted at CHI 2023.
Nov 2022: Serving as CHI 2023 Late-Breaking Work (LBW) Program Committee member.
Oct 2022: Reviewed CHI 2023 papers (1*special recognition) and SVed at UIST 2022.
May 2022: Presented FluidMeet paper at CHI 2022 in New Orleans.
Mar 2022: Presented FluidMeet at the Virtual Social Presence Seminar. You can watch the recorded 40-min talk here.

Selected Publications

Experiencing Thing2Reality: Transforming 2D Content into Conditioned Multiviews and 3D Gaussian Objects for XR Communication

Erzhen Hu, Mingyi Li, Xun Qian, Alex Olwal, David Kim, Seongkook Heo, Ruofei Du

UIST 2024 Demo

Leverage multi-view diffusion models and 3D Gaussian Splatting to enhance XR communication and collaboration around objects. A user can identify 2D content from the web browser or video seethrough, and the system will automatically transform the 2D content into 3D objects and multi-view images for XR communication.

[Demo] [30s Preview] [Video Figure] [Project Page]

ThingShare: Ad-Hoc Digital Copies of Physical Objects for Sharing Things in Video Meetings

Erzhen Hu, Jens Emil Grønbæk, Wen Ying, Ruofei Du, Seongkook Heo

CHI 2023

Leverage deep learning (instance segmentation) to enrich communication centered around objects and elevate real-time collaboration in remote settings.

[Paper] [30s Preview] [Video Figure] [Talk] [Project Page]

OpenMic: Utilizing Proxemic Metaphors for Conversational Floor Transitions in Multiparty Video Meetings

Erzhen Hu, Jens Emil Grønbæk, Austin Houck, Seongkook Heo

CHI 2023

Employ proxemic metaphors within the virtual conversational space to enhance turn-taking dynamics in groups of varying sizes (small to medium-sized groups).

[Paper] [30s Preview] [Video Figure] [Talk] [Project Page]