My research is focused on developing methodologies capable of effectively harnessing various sources of data with applications in 2D/3D/4D computer vision, generative models, and neural rendering.
My recent works aim for scalable 3D representation learning and 3D/4D view synthesis without requiring large-scale real-world multi-view datasets, utilizing
During my graduate study, I interned at Apple, Meta Reality Labs, and Google, conducting research related to above topics and gaining experiences in large-scale training for GANs and diffusion models.
I am on the job market, looking for research scientist / engineer positions in industry or post-doc positions in academia starting from summer 2024. Happy to discuss potential fits. Email /
Google Scholar /
GitHub /
CV
IllumiNeRF provides a simpler approach than traditional inverse rendering for 3D relighting: relit source views than run a latent NeRF.
IllumiNeRF: 3D Relighting without Inverse Rendering
@article{zhao2024illuminerf,
author = {Xiaoming Zhao and Pratul P. Srinivasan and Dor Verbin and Keunhong Park and Ricardo Martin Brualla and Philipp Henzler},
title = {{IllumiNeRF: 3D Relighting without Inverse Rendering}},
journal = {ArXiv},
year = {2024},
}
GoMAvatar introduces Gaussians-on-Mesh (GoM) representation for real-time, memory-efficient, and high-quality animatable human modeling.
GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh
@inproceedings{wen2024GoM,
title={{GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh}},
author={Jing Wen and Xiaoming Zhao and Zhongzheng Ren and Alex Schwing and Shenlong Wang},
booktitle={CVPR},
year={2024},
}
NeRFDeformer automatically modifies a NeRF representation based on a single RGB-D observation of a non-rigid transformed version of the original scene.
NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows
@inproceedings{tang2024nerfdeformer,
title={{NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows}},
author={Zhenggang Tang and Zhongzheng Ren and Xiaoming Zhao and Bowen Wen and Jonathan Tremblay and Stan Birchfield and Alex Schwing},
booktitle={CVPR},
year={2024},
}
PGDVS provides an analysis framework for generalized dynamic view synthesis and finds with consistent depth estimations, scene-specific appearance optimization is NOT required.
Pseudo-Generalized Dynamic View Synthesis from a Video
@inproceedings{Zhao2024PGDVS,
title={{Pseudo-Generalized Dynamic View Synthesis from a Video}},
author={Xiaoming Zhao and Alex Colburn and Fangchang Ma and Miguel Angel Bautista and Joshua M. Susskind and Alexander G. Schwing},
booktitle={ICLR},
year={2024},
}
OPlanes provides more flexibility than voxel grids and enables to better leverage correlations than per-point classification.
Occupancy Planes for Single-view RGB-D Human Reconstruction
@inproceedings{Zhao2023Oplanes,
title={{Occupancy Planes for Single-view RGB-D Human Reconstruction}},
author={Xiaoming Zhao and Yuan-Ting Hu and Zhongzheng Ren and Alexander G. Schwing},
booktitle={AAAI},
year={2023},
}
GMPI guarantees to be view-consistent and enables fast training (in less than half a day at a resolution of 10242) and high FPS during inference.
Generative Multiplane Images: Making a 2D GAN 3D-Aware
@inproceedings{zhao2022gmpi,
title = {Generative Multiplane Images: Making a 2D GAN 3D-Aware},
author = {Xiaoming Zhao and Fangchang Ma and David Güera and Zhile Ren and Alexander G. Schwing and Alex Colburn},
booktitle = {ECCV},
year = {2022},
}
Carefully designed initialization and alignment procedures enable benefiting from both classical and recent learning-based texture optimization techniques.
Initialization and Alignment for Adversarial Texture Optimization
@inproceedings{zhao2022tex,
title = {Initialization and Alignment for Adversarial Texture Optimization},
author = {Xiaoming Zhao and Zhizhen Zhao and Alexander G. Schwing},
booktitle = {ECCV},
year = {2022},
}
REDO enables class-agnostic geometry reconstruction for dynamic objects from RGB-D videos.
Class-agnostic 4D Reconstruction from Videos
@inproceedings{ren2021redo,
title = {Class-agnostic Reconstruction of Dynamic Objects from Videos},
author = {Zhongzheng Ren$^\ast$ and Xiaoming Zhao$^\ast$ and Alexander G. Schwing},
booktitle = {NeurIPS},
year = {2021},
note = {$^\ast$ equal contribution},
}
A well-trained visual odometry module can be a drop-in replacement for GPS and Compass sensor in PointGoal navigation.
The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
@inproceedings{Zhao2021pointnav,
title={{The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation}},
author={Xiaoming Zhao and Harsh Agrawal and Dhruv Batra and Alexander G. Schwing},
booktitle={ICCV},
year={2021},
}
Meta-learning and few-shot learning strategy can be utilized to mitigate the data scarcity issue in characterizing the specificity of less-studied kinases for protein-peptide binding prediction.
Mitigating Data Scarcity in Protein Binding Prediction Using Meta-Learning
@inproceedings{luo2019mitigating,
title={{Mitigating Data Scarcity in Protein Binding Prediction Using Meta-Learning}},
author={Luo, Yunan and Ma, Jianzhu and Zhao, Xiaoming and Su, Yufeng and Liu, Yang and Ideker, Trey and Peng, Jian},
booktitle={Proc. RECOMB},
year={2019},
}
A deep learning-based thermodynamic model is introduced for protein-RNA binding prediction.
Integrating Thermodynamic and Sequence Contexts Improves Protein-RNA Binding Prediction
@article{su2019integrating,
title={{Integrating thermodynamic and sequence contexts improves protein-RNA binding prediction}},
author={Su, Yufeng and Luo, Yunan and Zhao, Xiaoming and Liu, Yang and Peng, Jian},
journal={PLOS Computational Biology},
year={2019},
}
@inproceedings{Zhao2020ICMLOpenWorldWS,
title={{Learning from Synthesized Demonstrations}},
author={Xiaoming Zhao and Yang Liu and Jian Peng},
booktitle={Proc. ICML Workshop on Learning in Artificial Open Worlds},
year={2020},
}
@inproceedings{Zhao2019AAAIRLGameWS,
title={{Approximation Gradient Error Variance Reduced Optimization}},
author={Wei-Ye Zhao and Yang Liu and Xiaoming Zhao and Jie-Lin Qiu and Jian Peng},
booktitle={Proc. AAAI Workshop on Reinforcement Learning in Games},
year={2019},
}
