🚩 I will be joining Shanghai Jiao Tong University as a tenure-track assistant professor.🚩 I am recruiting RA, Master, and PhD students, drop me an email if interested.
[2022/04] One paper (QAS) was accepted to Nature Partner Journals Quantum Information (NPJ QI).
[2022/03] Two papers (GreedyNASv2 & DyRep) were accepted to CVPR 2022.
[2022/01] One paper (ReLoss) was accepted to ICLR 2022.
[2021/12] We released MMRazor - a model compression toolkit for model slimming and AutoML, which includes 3 mainstream technologies NAS, pruning, and KD. MMRazor can be easily applied to various projects (e.g., MMDet and MMCls) in OpenMMLab.
[2021/11] We released GreedyNASv2 at arXiv.
[2021/03] One paper about NAS was accepted to CVPR 2021. The NAS benchmark in our paper was released at github.
[2021/01] One paper about channel number search (pruning) was accepted to ICLR 2021 as spotlight.
[2020/11] One paper about NAS was released at arXiv. Our TopoNAS explicitly learns the topology for differentiable NAS (DARTS), and enjoys significant efficiency improvement on obtained architectures.
[2020/10] One paper about quantum architecture search (QAS) was released at arXiv. Our QAS implicitly learns a rule that can well suppress the influence of quantum noise and the barren plateau.
[2020/02] One paper about NAS was accepted to CVPR 2020.
@article{huang2025dist,
author={Huang, Tao and You, Shan and Wang, Fei and Qian, Chen and Xu, Chang},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
title={DIST+: Knowledge Distillation From a Stronger Adaptive Teacher},
year={2025},
volume={},
number={},
pages={1-15},
doi={10.1109/TPAMI.2025.3554235}}
MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders
Jiajun Cao, Yuan Zhang,Tao Huang, Ming Lu, Qizhe Zhang, Ruichuan An, Ningning Ma, Shanghang Zhang IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
@misc{minimax2025minimax01scalingfoundationmodels,
title={MiniMax-01: Scaling Foundation Models with Lightning Attention},
author={MiniMax and Aonian Li and Bangwei Gong and Bo Yang and Boji Shan and Chang Liu and Cheng Zhu and Chunhao Zhang and Congchao Guo and Da Chen and Dong Li and Enwei Jiao and Gengxin Li and Guojun Zhang and Haohai Sun and Houze Dong and Jiadai Zhu and Jiaqi Zhuang and Jiayuan Song and Jin Zhu and Jingtao Han and Jingyang Li and Junbin Xie and Junhao Xu and Junjie Yan and Kaishun Zhang and Kecheng Xiao and Kexi Kang and Le Han and Leyang Wang and Lianfei Yu and Liheng Feng and Lin Zheng and Linbo Chai and Long Xing and Meizhi Ju and Mingyuan Chi and Mozhi Zhang and Peikai Huang and Pengcheng Niu and Pengfei Li and Pengyu Zhao and Qi Yang and Qidi Xu and Qiexiang Wang and Qin Wang and Qiuhui Li and Ruitao Leng and Shengmin Shi and Shuqi Yu and Sichen Li and Songquan Zhu and Tao Huang and Tianrun Liang and Weigao Sun and Weixuan Sun and Weiyu Cheng and Wenkai Li and Xiangjun Song and Xiao Su and Xiaodong Han and Xinjie Zhang and Xinzhu Hou and Xu Min and Xun Zou and Xuyang Shen and Yan Gong and Yingjie Zhu and Yipeng Zhou and Yiran Zhong and Yongyi Hu and Yuanxiang Fan and Yue Yu and Yufeng Yang and Yuhao Li and Yunan Huang and Yunji Li and Yunpeng Huang and Yunzhi Xu and Yuxin Mao and Zehan Li and Zekang Li and Zewei Tao and Zewen Ying and Zhaoyang Cong and Zhen Qin and Zhenhua Fan and Zhihang Yu and Zhuo Jiang and Zijia Wu},
year={2025},
eprint={2501.08313},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2501.08313},
}
EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
Xiaohuan Pei*, Tao Huang*, Chang Xu In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2025
@inproceedings{zhang2024unveiling,
author = {Zhang, Yuan and Xiao, Fei and Huang, Tao and Fan, Chun-Kai and Dong, Hongyuan and Li, Jiawen and Wang, Jiacong and Cheng, Kuan and Zhang, Shanghang and Guo, Haoyuan},
booktitle = {Advances in Neural Information Processing Systems},
title = {Unveiling the Tapestry of Consistency in Large Vision-Language Models},
year = {2024}
}
LocalMamba: Visual State Space Model with Windowed Selective Scan
Tao Huang, Xiaohuan Pei, Shan You, Fei Wang, Chen Qian, Chang Xu Efficient Deep Learning for Foundation Models, European Conference on Computer Vision (ECCV) Workshop, 2024.
@article{huang2024localmamba,
title={Localmamba: Visual state space model with windowed selective scan},
author={Huang, Tao and Pei, Xiaohuan and You, Shan and Wang, Fei and Qian, Chen and Xu, Chang},
journal={arXiv preprint arXiv:2403.09338},
year={2024}
}
Active Generation for Image Classification
Tao Huang*, Jiaqi Liu*, Shan You, Chang Xu European Conference on Computer Vision (ECCV), 2024.
@inproceedings{huang2024active,
title={Active Generation for Image Classification},
author={Huang, Tao and Liu, Jiaqi and You, Shan and Xu, Chang},
booktitle={European Conference on Computer Vision},
year={2024},
organization={Springer}
}
Training-free Composite Scene Generation for Layout-to-Image Synthesis
Jiaqi Liu,Tao Huang, Chang Xu European Conference on Computer Vision (ECCV), 2024.
@inproceedings{liu2024training,
title={Training-free Composite Scene Generation for Layout-to-Image Synthesis},
author={Liu, Jiaqi and Huang, Tao and Xu, Chang},
booktitle={European Conference on Computer Vision},
year={2024},
organization={Springer}
}
Diverse and Tailored Image Generation for Zero-Shot Multi-Label Classification
Kaixin Zhang, Zhixiang Yuan, Tao Huang Knowledge-Based Systems (KBS), Volume 299, 5 September 2024, 112077.
@inproceedings{yuan2024positive,
title={Positive label is all you need for multi-label classification},
author={Yuan, Zhixiang and Zhang, Kaixin and Huang, Tao},
booktitle={2024 IEEE International Conference on Multimedia and Expo (ICME)},
pages={1--6},
year={2024},
organization={IEEE}
}
FreeKD: Knowledge Distillation via Semantic Frequency Prompt
Yuan Zhang,Tao Huang, Jiaming Liu, Tao Jiang, Kuan Cheng, Shanghang Zhang IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
@inproceedings{zhang2024freekd,
title={FreeKD: Knowledge Distillation via Semantic Frequency Prompt},
author={Zhang, Yuan and Huang, Tao and Liu, Jiaming and Jiang, Tao and Cheng, Kuan and Zhang, Shanghang},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={15931--15940},
year={2024}
}
Knowledge Diffusion for Distillation
Tao Huang, Yuan Zhang, Mingkai Zheng, Shan You, Fei Wang, Chen Qian, Chang Xu Advances in Neural Information Processing Systems (NeurIPS), 2023.
@inproceedings{huang2023knowledge,
author = {Huang, Tao and Zhang, Yuan and Zheng, Mingkai and You, Shan and Wang, Fei and Qian, Chen and Xu, Chang},
booktitle = {Advances in Neural Information Processing Systems},
editor = {A. Oh and T. Naumann and A. Globerson and K. Saenko and M. Hardt and S. Levine},
pages = {65299--65316},
publisher = {Curran Associates, Inc.},
title = {Knowledge Diffusion for Distillation},
url = {https://proceedings.neurips.cc/paper_files/paper/2023/file/cdddf13f06182063c4dbde8cbd5a5c21-Paper-Conference.pdf},
volume = {36},
year = {2023}
}
Avatar Knowledge Distillation: Self-ensemble Teacher Paradigm with Uncertainty
Yuan Zhang, Weihua Chen, Yichen Lu,Tao Huang, Xiuyu Sun, Jian Cao Proceedings of the 31th ACM International Conference on Multimedia (ACM MM), 2023.
@inproceedings{zhang2023avatar,
title={Avatar knowledge distillation: self-ensemble teacher paradigm with uncertainty},
author={Zhang, Yuan and Chen, Weihua and Lu, Yichen and Huang, Tao and Sun, Xiuyu and Cao, Jian},
booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
pages={5272--5280},
year={2023}
}
Masked Distillation with Receptive Tokens
Tao Huang*, Yuan Zhang*, Shan You, Fei Wang, Chen Qian, Jian Cao, Chang Xu International Conference on Learning Representations (ICLR), 2023.
@inproceedings{
huang2023masked,
title={Masked Distillation with Receptive Tokens},
author={Tao Huang and Yuan Zhang and Shan You and Fei Wang and Chen Qian and Jian Cao and Chang Xu},
booktitle={The Eleventh International Conference on Learning Representations },
year={2023},
url={https://openreview.net/forum?id=mWRngkvIki3}
}
Knowledge Distillation from A Stronger Teacher
Tao Huang, Shan You, Fei Wang, Chen Qian, Chang Xu Advances in Neural Information Processing Systems (NeurIPS), 2022.
@inproceedings{huang2022knowledge,
author = {Huang, Tao and You, Shan and Wang, Fei and Qian, Chen and Xu, Chang},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {33716--33727},
publisher = {Curran Associates, Inc.},
title = {Knowledge Distillation from A Stronger Teacher},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/da669dfd3c36c93905a17ddba01eef06-Paper-Conference.pdf},
volume = {35},
year = {2022}
}
Quantum circuit architecture search for variational quantum algorithms
Yuxuan Du,Tao Huang, Shan You, Min-Hsiu Hsieh, Dacheng Tao Nature Partner Journals Quantum Information (NPJ QI), 2022.
@article{du2022quantum,
author = {Du, Yuxuan and Huang, Tao and You, Shan and Hsieh, Min-Hsiu and Tao, Dacheng},
date = {2022/05/23},
doi = {10.1038/s41534-022-00570-y},
id = {Du2022},
isbn = {2056-6387},
journal = {npj Quantum Information},
number = {1},
pages = {62},
title = {Quantum circuit architecture search for variational quantum algorithms},
url = {https://doi.org/10.1038/s41534-022-00570-y},
volume = {8},
year = {2022},
bdsk-url-1 = {https://doi.org/10.1038/s41534-022-00570-y}
}
GreedyNASv2: Greedier Search with a Greedy Path Filter
Tao Huang, Shan You, Fei Wang, Chen Qian, Changshui Zhang, Xiaogang Wang, Chang Xu IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
@inproceedings{huang2022greedynasv2,
title={Greedynasv2: greedier search with a greedy path filter},
author={Huang, Tao and You, Shan and Wang, Fei and Qian, Chen and Zhang, Changshui and Wang, Xiaogang and Xu, Chang},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={11902--11911},
year={2022}
}
DyRep: Bootstrapping Training with Dynamic Re-parameterization
Tao Huang, Shan You, Bohan Zhang, Yuxuan Du, Fei Wang, Chen Qian, Chang Xu IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
@inproceedings{huang2022dyrep,
title={Dyrep: bootstrapping training with dynamic re-parameterization},
author={Huang, Tao and You, Shan and Zhang, Bohan and Du, Yuxuan and Wang, Fei and Qian, Chen and Xu, Chang},
booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
pages={588--597},
year={2022}
}
Relational Surrogate Loss Learning
Tao Huang, Zekang Li, Hua Lu, Yong Shan, Shusheng Yang, Yang Feng, Fei Wang, Shan You, Chang Xu International Conference on Learning Representations (ICLR), 2022.
@inproceedings{
huang2022relational,
title={Relational Surrogate Loss Learning},
author={Tao Huang and Zekang Li and Hua Lu and Yong Shan and Shusheng Yang and Yang Feng and Fei Wang and Shan You and Chang Xu},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/forum?id=dZPgfwaTaXv}
}
Data Agnostic Filter Gating for Efficient Deep Networks
Hongyan Xu, Xiu Su, Shan You,Tao Huang, Fei Wang, Chen Qian, Changshui Zhang, Chang Xu, Dadong Wang, Arcot Sowmya IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022.
@inproceedings{xu2022data,
title={Data agnostic filter gating for efficient deep networks},
author={Xu, Hongyan and Su, Xiu and You, Shan and Huang, Tao and Wang, Fei and Qian, Chen and Zhang, Changshui and Xu, Chang and Wang, Dadong and Sowmya, Arcot},
booktitle={ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={3503--3507},
year={2022},
organization={IEEE}
}
Prioritized Architecture Sampling with Monto-Carlo Tree Search
@inproceedings{su2021prioritized,
title={Prioritized architecture sampling with monto-carlo tree search},
author={Su, Xiu and Huang, Tao and Li, Yanxi and You, Shan and Wang, Fei and Qian, Chen and Zhang, Changshui and Xu, Chang},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={10968--10977},
year={2021}
}
Locally Free Weight Sharing for Network Width Search🚩 Spotlight
Xiu Su, Shan You,Tao Huang, Fei Wang, Chen Qian, Changshui Zhang, Chang Xu International Conference on Learning Representations (ICLR), 2021.
@inproceedings{
su2021locally,
title={Locally Free Weight Sharing for Network Width Search},
author={Xiu Su and Shan You and Tao Huang and Fei Wang and Chen Qian and Changshui Zhang and Chang Xu},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=S0UdquAnr9k}
}
GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet
@inproceedings{you2020greedynas,
title={Greedynas: Towards fast one-shot nas with greedy supernet},
author={You, Shan and Huang, Tao and Yang, Mingmin and Wang, Fei and Qian, Chen and Zhang, Changshui},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={1999--2008},
year={2020}
}
Talks
Peking University: "Efficient Large Models", Jul. 2024.
TNNLS 2024 Tutorial: "Efficient and Secure Foundation Models", Jun. 2024.
TMLR Young Scientist Seminar@HKBU: "Knowledge Distillation from A Stronger Teacher", Jul. 2022. [Slides]
SenseTime Titan Open Course: "Automated Machine Learning", May 2020.
Teaching
2024, Guest Lecturer for COMP5329 Deep Learning, The University of Sydney
2024, Tutor for COMP5329 Deep Learning, The University of Sydney
Academic Services
Conference Events:
2024, Co-organizor, "ESGMFM '24: The 1st Workshop on Efficiency, Security, and Generalization of Multimedia Foundation Models", ACM MM 2024 Workshop
2024, Co-organizor, "Efficient and Secure Foundation Models", IJCNN 2024 Tutorial
Reviewer for Conferences:
NeurIPS, ICLR, ICML, CVPR, ICCV, ECCV, AAAI, ACM MM.
