Biography

My name is Tao Huang (黄涛). I am a final-year PhD candidate at The University of Sydney, advised by Prof. Chang Xu. Before that, I was a Researcher at SenseTime, supervised by Dr. Shan You. In 2020, I received my Bachelor's degree in Computer Science at Huazhong University of Science and Technology, China.

My major research interests lie within efficient machine learning algorithms and their applications in computer vision tasks:

• Efficient architectures: NAS, handcrafts, pruning, Rep
• Knowledge distillation (KD)
• Efficient training
• Efficient large models

Tao Huang
PhD Candidate
The University of Sydney
Email: huntocn[at]gmail.com

     

News

• [2024/09] One paper was accepted to NeurIPS 2024.
• [2024/07] I gave a talk about "Efficient Large Models" at Peking University. Many thanks to Prof. Shanghang Zhang for the invitation.
• [2024/07] Two papers were accepted to ECCV 2024.
• [2024/06] I co-organized TNNLS 2024 Tutorial "Efficient and Secure Foundation Models" and gave a talk.
• [2024/06] One paper was accepted to journal Knowledge-Based Systems.
• [2024/03] Our PU-MLC was accepted to ICME 2024 as oral.
• [2024/03] We released ActGen & LocalMamba & EfficientVMamba at arXiv.
• [2024/02] One paper (FreeKD) was accepted to CVPR 2024.
• [2024/02] We released DIST+.
• [2023/09] One paper (DiffKD) was accepted to NeurIPS 2023.

Publications    [Google Scholar]

2024

• NeurIPSUnveiling the Tapestry of Consistency in Large Vision-Language Models
  
  Yuan Zhang, Fei Xiao, Tao Huang, Chun-Kai Fan, Hongyuan Dong, Jiawen Li, Jiacong Wang, Kuan Cheng, Shanghang Zhang, Haoyuan Guo
  Advances in Neural Information Processing Systems (NeurIPS), 2024.

  ArXiv Bib Code

                      @inproceedings{zhang2024unveiling,
                          author = {Zhang, Yuan and Xiao, Fei and Huang, Tao and Fan, Chun-Kai and Dong, Hongyuan and Li, Jiawen and Wang, Jiacong and Cheng, Kuan and Zhang, Shanghang and Guo, Haoyuan},
                          booktitle = {Advances in Neural Information Processing Systems},
                          title = {Unveiling the Tapestry of Consistency in Large Vision-Language Models},
                          year = {2024}
                      }
                  

• ECCVWLocalMamba: Visual State Space Model with Windowed Selective Scan
  
  Tao Huang, Xiaohuan Pei, Shan You, Fei Wang, Chen Qian, Chang Xu
  Efficient Deep Learning for Foundation Models, European Conference on Computer Vision (ECCV) Workshop, 2024.

  ArXiv Bib Code

                      @article{huang2024localmamba,
                          title={Localmamba: Visual state space model with windowed selective scan},
                          author={Huang, Tao and Pei, Xiaohuan and You, Shan and Wang, Fei and Qian, Chen and Xu, Chang},
                          journal={arXiv preprint arXiv:2403.09338},
                          year={2024}
                      }
                  

• ECCVActive Generation for Image Classification
  
  Tao Huang, Jiaqi Liu, Shan You, Chang Xu
  European Conference on Computer Vision (ECCV), 2024.

  ArXiv Bib Code

                      @inproceedings{huang2024active,
                          title={Active Generation for Image Classification},
                          author={Huang, Tao and Liu, Jiaqi and You, Shan and Xu, Chang},
                          booktitle={European Conference on Computer Vision},
                          year={2024},
                          organization={Springer}
                      }
                  

• ECCVTraining-free Composite Scene Generation for Layout-to-Image Synthesis
  
  Jiaqi Liu, Tao Huang, Chang Xu
  European Conference on Computer Vision (ECCV), 2024.

  ArXiv Bib Code

                      @inproceedings{liu2024training,
                          title={Training-free Composite Scene Generation for Layout-to-Image Synthesis},
                          author={Liu, Jiaqi and Huang, Tao and Xu, Chang},
                          booktitle={European Conference on Computer Vision},
                          year={2024},
                          organization={Springer}
                      }
                  

• KBSDiverse and Tailored Image Generation for Zero-Shot Multi-Label Classification
  
  Kaixin Zhang, Zhixiang Yuan, Tao Huang
  Knowledge-Based Systems (KBS), Volume 299, 5 September 2024, 112077.

  ArXiv Bib KBS Code

                      @article{ZHANG2024112077,
                          title = {Diverse and tailored image generation for zero-shot multi-label classification},
                          journal = {Knowledge-Based Systems},
                          volume = {299},
                          pages = {112077},
                          year = {2024},
                          issn = {0950-7051},
                          doi = {https://doi.org/10.1016/j.knosys.2024.112077},
                          url = {https://www.sciencedirect.com/science/article/pii/S0950705124007111},
                          author = {Kaixin Zhang and Zhixiang Yuan and Tao Huang},
                          keywords = {Zero-shot multi-label learning, Deep generative model, Diffusion model, Synthetic data},
                      }
                  

• ICMEPositive Label Is All You Need for Multi-Label Classification🚩 Oral
  
  Zhixiang Yuan, Kaixin Zhang, Tao Huang
  IEEE International Conference on Multimedia and Expo (ICME), 2024.

  ArXiv Bib Code

                      @inproceedings{yuan2024positive,
                          title={Positive label is all you need for multi-label classification},
                          author={Yuan, Zhixiang and Zhang, Kaixin and Huang, Tao},
                          booktitle={2024 IEEE International Conference on Multimedia and Expo (ICME)},
                          pages={1--6},
                          year={2024},
                          organization={IEEE}
                      }
                  

• CVPRFreeKD: Knowledge Distillation via Semantic Frequency Prompt
  
  Yuan Zhang, Tao Huang, Jiaming Liu, Tao Jiang, Kuan Cheng, Shanghang Zhang
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.

  ArXiv Bib Code

                      @inproceedings{zhang2024freekd,
                          title={FreeKD: Knowledge Distillation via Semantic Frequency Prompt},
                          author={Zhang, Yuan and Huang, Tao and Liu, Jiaming and Jiang, Tao and Cheng, Kuan and Zhang, Shanghang},
                          booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
                          pages={15931--15940},
                          year={2024}
                      }
                  

2023

• NeurIPSKnowledge Diffusion for Distillation
  
  Tao Huang, Yuan Zhang, Mingkai Zheng, Shan You, Fei Wang, Chen Qian, Chang Xu
  Advances in Neural Information Processing Systems (NeurIPS), 2023.

  ArXiv Bib Poster Code

                      @inproceedings{huang2023knowledge,
                          author = {Huang, Tao and Zhang, Yuan and Zheng, Mingkai and You, Shan and Wang, Fei and Qian, Chen and Xu, Chang},
                          booktitle = {Advances in Neural Information Processing Systems},
                          editor = {A. Oh and T. Naumann and A. Globerson and K. Saenko and M. Hardt and S. Levine},
                          pages = {65299--65316},
                          publisher = {Curran Associates, Inc.},
                          title = {Knowledge Diffusion for Distillation},
                          url = {https://proceedings.neurips.cc/paper_files/paper/2023/file/cdddf13f06182063c4dbde8cbd5a5c21-Paper-Conference.pdf},
                          volume = {36},
                          year = {2023}
                      }                       
                  

• ACM MMAvatar Knowledge Distillation: Self-ensemble Teacher Paradigm with Uncertainty
  
  Yuan Zhang, Weihua Chen, Yichen Lu, Tao Huang, Xiuyu Sun, Jian Cao
  Proceedings of the 31th ACM International Conference on Multimedia (ACM MM), 2023.

  ArXiv Bib Code

                      @inproceedings{zhang2023avatar,
                          title={Avatar knowledge distillation: self-ensemble teacher paradigm with uncertainty},
                          author={Zhang, Yuan and Chen, Weihua and Lu, Yichen and Huang, Tao and Sun, Xiuyu and Cao, Jian},
                          booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
                          pages={5272--5280},
                          year={2023}
                      }
                  

• ICLRMasked Distillation with Receptive Tokens
  
  Tao Huang*, Yuan Zhang*, Shan You, Fei Wang, Chen Qian, Jian Cao, Chang Xu
  International Conference on Learning Representations (ICLR), 2023.

  ArXiv Bib Code

                      @inproceedings{
                          huang2023masked,
                          title={Masked Distillation with Receptive Tokens},
                          author={Tao Huang and Yuan Zhang and Shan You and Fei Wang and Chen Qian and Jian Cao and Chang Xu},
                          booktitle={The Eleventh International Conference on Learning Representations },
                          year={2023},
                          url={https://openreview.net/forum?id=mWRngkvIki3}
                      }
                  

2022

• NeurIPSKnowledge Distillation from A Stronger Teacher
  
  Tao Huang, Shan You, Fei Wang, Chen Qian, Chang Xu
  Advances in Neural Information Processing Systems (NeurIPS), 2022.

  ArXiv Bib Slides Poster 解读 Code

                      @inproceedings{huang2022knowledge,
                          author = {Huang, Tao and You, Shan and Wang, Fei and Qian, Chen and Xu, Chang},
                          booktitle = {Advances in Neural Information Processing Systems},
                          editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
                          pages = {33716--33727},
                          publisher = {Curran Associates, Inc.},
                          title = {Knowledge Distillation from A Stronger Teacher},
                          url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/da669dfd3c36c93905a17ddba01eef06-Paper-Conference.pdf},
                          volume = {35},
                          year = {2022}
                      }
                  

• NPJ QIQuantum circuit architecture search for variational quantum algorithms
  
  Yuxuan Du, Tao Huang, Shan You, Min-Hsiu Hsieh, Dacheng Tao
  Nature Partner Journals Quantum Information (NPJ QI), 2022.
  

  ArXiv Bib Nature Code

                  @article{du2022quantum,
                      author = {Du, Yuxuan and Huang, Tao and You, Shan and Hsieh, Min-Hsiu and Tao, Dacheng},
                      date = {2022/05/23},
                      doi = {10.1038/s41534-022-00570-y},
                      id = {Du2022},
                      isbn = {2056-6387},
                      journal = {npj Quantum Information},
                      number = {1},
                      pages = {62},
                      title = {Quantum circuit architecture search for variational quantum algorithms},
                      url = {https://doi.org/10.1038/s41534-022-00570-y},
                      volume = {8},
                      year = {2022},
                      bdsk-url-1 = {https://doi.org/10.1038/s41534-022-00570-y}
                  }
                  

• CVPRGreedyNASv2: Greedier Search with a Greedy Path Filter
  
  Tao Huang, Shan You, Fei Wang, Chen Qian, Changshui Zhang, Xiaogang Wang, Chang Xu
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  

  ArXiv Bib Poster

                      @inproceedings{huang2022greedynasv2,
                          title={Greedynasv2: greedier search with a greedy path filter},
                          author={Huang, Tao and You, Shan and Wang, Fei and Qian, Chen and Zhang, Changshui and Wang, Xiaogang and Xu, Chang},
                          booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
                          pages={11902--11911},
                          year={2022}
                      }
                  

• CVPRDyRep: Bootstrapping Training with Dynamic Re-parameterization
  
  Tao Huang, Shan You, Bohan Zhang, Yuxuan Du, Fei Wang, Chen Qian, Chang Xu
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  

  ArXiv Bib Poster Code

                      @inproceedings{huang2022dyrep,
                          title={Dyrep: bootstrapping training with dynamic re-parameterization},
                          author={Huang, Tao and You, Shan and Zhang, Bohan and Du, Yuxuan and Wang, Fei and Qian, Chen and Xu, Chang},
                          booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
                          pages={588--597},
                          year={2022}
                      }
                  

• ICLRRelational Surrogate Loss Learning
  
  Tao Huang, Zekang Li, Hua Lu, Yong Shan, Shusheng Yang, Yang Feng, Fei Wang, Shan You, Chang Xu
  International Conference on Learning Representations (ICLR), 2022.
  

  ArXiv Bib Poster Slides Code

                      @inproceedings{
                          huang2022relational,
                          title={Relational Surrogate Loss Learning},
                          author={Tao Huang and Zekang Li and Hua Lu and Yong Shan and Shusheng Yang and Yang Feng and Fei Wang and Shan You and Chang Xu},
                          booktitle={International Conference on Learning Representations},
                          year={2022},
                          url={https://openreview.net/forum?id=dZPgfwaTaXv}
                      }
                  

• ICASSPData Agnostic Filter Gating for Efficient Deep Networks
  
  Hongyan Xu, Xiu Su, Shan You, Tao Huang, Fei Wang, Chen Qian, Changshui Zhang, Chang Xu, Dadong Wang, Arcot Sowmya
  IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022.

  ArXiv Bib

                      @inproceedings{xu2022data,
                          title={Data agnostic filter gating for efficient deep networks},
                          author={Xu, Hongyan and Su, Xiu and You, Shan and Huang, Tao and Wang, Fei and Qian, Chen and Zhang, Changshui and Xu, Chang and Wang, Dadong and Sowmya, Arcot},
                          booktitle={ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
                          pages={3503--3507},
                          year={2022},
                          organization={IEEE}
                      }
                  

2021

• CVPRPrioritized Architecture Sampling with Monto-Carlo Tree Search
  
  Xiu Su*, Tao Huang*, Yanxi Li, Shan You, Fei Wang, Chen Qian, Changshui Zhang, Chang Xu
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  

  ArXiv Bib Poster NAS-Bench-Macro

                      @inproceedings{su2021prioritized,
                          title={Prioritized architecture sampling with monto-carlo tree search},
                          author={Su, Xiu and Huang, Tao and Li, Yanxi and You, Shan and Wang, Fei and Qian, Chen and Zhang, Changshui and Xu, Chang},
                          booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
                          pages={10968--10977},
                          year={2021}
                      }
                  

• ICLRLocally Free Weight Sharing for Network Width Search🚩 Spotlight
  
  Xiu Su, Shan You, Tao Huang, Fei Wang, Chen Qian, Changshui Zhang, Chang Xu
  International Conference on Learning Representations (ICLR), 2021.
  

  ArXiv Bib OpenReview

                      @inproceedings{
                          su2021locally,
                          title={Locally Free Weight Sharing for Network Width Search},
                          author={Xiu Su and Shan You and Tao Huang and Fei Wang and Chen Qian and Changshui Zhang and Chang Xu},
                          booktitle={International Conference on Learning Representations},
                          year={2021},
                          url={https://openreview.net/forum?id=S0UdquAnr9k}
                      }
                  

2020

• CVPRGreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet
  
  Shan You*, Tao Huang*, Mingmin Yang*, Fei Wang, Chen Qian, Changshui Zhang
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  

  ArXiv Bib Poster Video Slides Code

                      @inproceedings{you2020greedynas,
                          title={Greedynas: Towards fast one-shot nas with greedy supernet},
                          author={You, Shan and Huang, Tao and Yang, Mingmin and Wang, Fei and Qian, Chen and Zhang, Changshui},
                          booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
                          pages={1999--2008},
                          year={2020}
                      }
                  

Manuscripts

• arXivSparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference
  
  Yuan Zhang*, ChunKai Fan*, Junpeng Ma*, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, Shanghang Zhang
  arXiv preprint arXiv:2410.04417 (2024).

  ArXiv Code

• arXivEfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
  
  Xiaohuan Pei*, Tao Huang*, Chang Xu
  arXiv preprint arXiv:2403.09977 (2024).

  ArXiv Code

• arXivNot All Steps are Equal: Efficient Generation with Progressive Diffusion Models
  
  Wenhao Li, Xiu Su, Shan You, Tao Huang, Fei Wang, Chen Qian, Chang Xu
  arXiv preprint arXiv:2312.13307 (2023).

  ArXiv

• DIST+: Knowledge Distillation from A Stronger Teacher
  
  Tao Huang, Shan You, Fei Wang, Chen Qian, Chang Xu
  

  PDF

• arXivLightViT: Towards Light-Weight Convolution-Free Vision Transformers
  
  Tao Huang, Lang Huang, Shan You, Fei Wang, Chen Qian, Chang Xu
  arXiv preprint arXiv:2207.05557 (2022).

  ArXiv Bib Code

• arXivExplicitly Learning Topology for Differentiable Neural Architecture Search
  
  Tao Huang, Shan You, Yibo Yang, Zhuozhuo Tu, Fei Wang, Chen Qian, Changshui Zhang
  arXiv preprint arXiv:2011.09300 (2020).

  ArXiv

Grants

• 2023-2024, Efficient Language-and-Vision Models, 1000 USD, Google Cloud Research Credits Award

Talks

• Peking University: "Efficient Large Models", Jul. 2024.
• TNNLS 2024 Tutorial: "Efficient and Secure Foundation Models", Jun. 2024.
• TMLR Young Scientist Seminar@HKBU: "Knowledge Distillation from A Stronger Teacher", Jul. 2022. [Slides]
• SenseTime Titan Open Course: "Automated Machine Learning", May 2020. [Recording]

Teaching

• 2024-2025, Guest Lecturer for COMP5329 Deep Learning, The University of Sydney
• 2024-2025, Tutor for COMP5329 Deep Learning, The University of Sydney

Academic Services

• 2024, Co-organizor, "ESGMFM '24: The 1st Workshop on Efficiency, Security, and Generalization of Multimedia Foundation Models", ACM MM 2024 Workshop
• 2024, Co-organizor, "Efficient and Secure Foundation Models", IJCNN 2024 Tutorial

Education

• The University of Sydney, Sydney, Australia

  Jul. 2022 – Present
  Ph.D. student at School of Computer Science
  Advisor: Prof. Chang Xu
  
• Huazhong University of Science and Technology, Wuhan, China

  Sep. 2016 – Jun. 2020
  B.E. in Computer Science and Technology
  

Selected Experiences

• SenseTime, Beijing, China
  
  Research Intern

  Jul. 2022 - Present
  Researcher

  Jul. 2020 – Jun. 2022
  Research Intern

  Aug. 2019 – Jul. 2020
  • Research on model compression algorithms (NAS, KD, pruning, etc.).
  • Research and development on auto-driving (smart carbin) scenes such as face verification and drowsiness detection.
  • Applied NAS to face verification task on large­scale industrial datasets, which significantly improves the performance.
• Horizon Robotics, Beijing, China
  Computer Vision Research Intern

  May 2019 – Aug. 2019
  • Development on object detection framework: anchor-­free detection method, detection in traffic scene.
  • Research on knowledge distillation methods for face alignment, object detection.

Selected Awards

• NeurIPS 2023 Scholar Award

  Oct. 2023
• Outstanding Graduate award, Huazhong University of Science and Technology

  Jun. 2020
• Outstanding Graduate Thesis award, Huazhong University of Science and Technology

  Jun. 2020