Publications

All publications and implementations are made publicly available. If you cannot obtain a particular resource, feel free to drop me an email.

* indicates equal contribution.

Liliang Ren, Congcong Chen, Haoran Xu, Young Jin Kim, Adam Atkinson, Zheng Zhan, Jiankai Sun, Baolin Peng, Liyuan Liu , Shuohang Wang, Hao Cheng, Jianfeng Gao, Weizhu Chen, and Yelong Shen (2025). Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation. arXiv:2507.06607 [cs].

Feng Yao, Zilong Wang, Liyuan Liu , Junxia Cui, Li Zhong, Xiaohan Fu, Haohui Mai, Vish Krishnan, Jianfeng Gao, and Jingbo Shang (2025). Training Language Models to Generate Quality Code with Program Analysis Feedback. arXiv:2505.22704 [cs].

Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu , Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen (2025). Reinforcement Learning for Reasoning in Large Language Models with One Training Example. arXiv:2504.20571 [cs].

Yufan Zhuang, Liyuan Liu , Chandan Singh, Jingbo Shang, and Jianfeng Gao (2025). Vector-ICL: In-context Learning with Continuous Vector Representations. arXiv:2505.14827 [cs].

Yufan Zhuang, Chandan Singh, Liyuan Liu , Jingbo Shang, and Jianfeng Gao (2024). Vector-ICL: In-context Learning with Continuous Vector Representations. Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025).

Rongzhi Zhang, Kuang Wang, Liyuan Liu , Shuohang Wang, Hao Cheng, Chao Zhang, and Yelong Shen (2024). LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy. arXiv:2410.03111 [cs].

Liyuan Liu , Jianfeng Gao, and Weizhu Chen (2024). GRIN: GRadient-INformed MoE. arXiv:2409.12136 [cs].

Qingru Zhang, Xiaodong Yu, Chandan Singh, Xiaodong Liu, Liyuan Liu , Jianfeng Gao, Tuo Zhao, Dan Roth, and Hao Cheng (2024). Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering. arXiv:2409.10790 [cs].

Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu , Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang, Lijuan Wang , Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward, Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Jilong Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang, Ziyi Yang, Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou (2024). Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv:2404.14219 [cs].

Yufan Zhuang, Liyuan Liu , Chandan Singh, Jingbo Shang, and Jianfeng Gao (2024). Learning a Decision Tree Algorithm with Transformers. arXiv:2402.03774 [cs].

Qingru Zhang, Chandan Singh, Liyuan Liu , Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao (2023). Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs. Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024).

Chengyu Dong, Liyuan Liu , Hao Cheng, Jingbo Shang, Jianfeng Gao, and Xiaodong Liu (2023). Fast-ELECTRA for Efficient Pre-training. Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024).

Suyu Ge, Yunan Zhang, Liyuan Liu , Minjia Zhang, Jiawei Han, and Jianfeng Gao (2023). Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs. Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024). Outstanding Paper Honorable Mention.

Liyuan Liu , Jianfeng Gao, and Weizhu Chen (2023). Sparse Backpropagation for MoE Training. WANT@NeurIPS 2023. Selected as Oral.

Liyuan Liu , Chengyu Dong, Xiaodong Liu, Bin Yu, and Jianfeng Gao (2023). Bridging Discrete and Backpropagation: Straight-Through and Beyond. Proceedings of the Proceeding of Thirty-seventh Annual Conference on Neural Information Processing Systems (NeurIPS 2023). Selected as Oral.

Chengyu Dong, Liyuan Liu , and Jingbo Shang (2022). Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting. Proceedings of the Thirty-sixth Annual Conference on Neural Information Processing Systems (NeurIPS 2022). Selected as Oral.

Chengyu Dong, Liyuan Liu , Hao Cheng, Jingbo Shang, Jianfeng Gao, and Xiaodong Liu (2022). Understand and modularize generator optimization in ELECTRA-style pretraining. Proceedings of the Fortieth International Conference on Machine Learning (ICML 2023).

Chengyu Dong, Liyuan Liu , and Jingbo Shang (2022). Toward Student-oriented Teacher Network Training for Knowledge Distillation. Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024).

Li Sha, Liyuan Liu , Yiqing Xie, Heng Ji, and Jiawei Han (2022). Piled: An identify-and-localize framework for few-shot event detection. arXiv:2202.07615 [cs].

Xiaotao Gu, Zihan Wang, Zhenyu Bi, Yu Meng, Liyuan Liu , Jiawei Han, and Jingbo Shang (2021). Multi-head or Single-head? An Empirical Comparison for Transformer Training. the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2021).

Liyuan Liu , Jialu Liu, and Jiawei Han (2021). Multi-head or Single-head? An Empirical Comparison for Transformer Training. arXiv:2106.09650 [cs].

Tao Chen, Haochen Shi, Liyuan Liu , Siliang Tang, Jian Shao, Zhigang Chen, and Yueting Zhuang (2020). Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training. Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021).

Zichao Li*, Liyuan Liu* , Chengyu Dong, and Jingbo Shang (2020). Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training. arXiv:2010.08034 [cs].

Xiaotao Gu, Liyuan Liu , Hongkun Yu, Jing Li, Chen Chen, and Jiawei Han (2020). On the Transformer Growth for Progressive BERT Training. arXiv:2010.12562 [cs].

Xiaodong Liu, Kevin Duh, Liyuan Liu , and Jianfeng Gao (2020). Very Deep Transformers for Neural Machine Translation. arXiv:2008.07772 [cs].

Chengyu Dong, Liyuan Liu , Zichao Li, and Jingbo Shang (2020). Towards Adaptive Residual Network Training: A Neural-ODE Perspective. the Thirty-seventh International Conference on Machine Learning (ICML 2020).

PDF Code Supplemental

Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang, Liyuan Liu , and Xiang Ren (2020). Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling. the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020).

Yuning Mao, Liyuan Liu , Qi Zhu, Xiang Ren, and Jiawei Han (2020). Facet-Aware Evaluation for Extractive Text Summarization. the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020).

Honglei Zhuang, Fang Guo, Chao Zhang, Liyuan Liu , and Jiawei Han (2020). Joint Aspect-Sentiment Analysis with Minimal User Guidance. the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2020).

Liyuan Liu , Xiaodong Liu, Jianfeng Gao, Weizhu Chen, and Jiawei Han (2020). Understanding the Difficulty of Training Transformers. the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020). Selected as Oral.

Liyuan Liu , Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han (2020). On the Variance of the Adaptive Learning Rate and Beyond. the Eighth International Conference on Learning Representations (ICLR 2020).

Jingbo Shang*, Xinyang Zhang*, Liyuan Liu , Sha Li, and Jiawei Han (2020). NetTaxo: Automated Topic Taxonomy Construction from Large-Scale Text-Rich Network. the 2020 Web Conference (WWW 2020).

Qinyuan Ye*, Liyuan Liu* , Maosen Zhang, and Xiang Ren (2019). Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction. the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019).

PDF Code Blog (3rd-Party)

Zihan Wang*, Jingbo Shang*, Liyuan Liu* , Lihao Lu, Jiacheng Liu, and Jiawei Han (2019). CrossWeigh: Training Named Entity Tagger from Imperfect Annotations. the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019).

PDF Code Blog (3rd-Party) Video (3rd-Party)

Liyuan Liu , Zihan Wang, Jingbo Shang, Dandong Yin, Heng Ji, Xiang Ren, Shaowen Wang, and Jiawei Han (2019). Raw-to-End Name Entity Recognition in Social Media. arXiv:1908.05344 [cs].

Liyuan Liu , Jingbo Shang, and Jiawei Han (2019). Arabic Named Entity Recognition: What Works and What's Next. the Fourth Arabic Natural Language Processing Workshop (WANLP 2019).

Ying Lin, Liyuan Liu , Heng Ji, Dong Yu, and Jiawei Han (2019). Reliability-aware Dynamic Feature Composition for Name Tagging. the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019).

Jingbo Shang, Jiaming Shen, Liyuan Liu , and Jiawei Han (2019). Constructing and Mining Heterogeneous Information Networks from Massive Text. Conference tutorial at the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2019).

PDF Tutorial Page Slides

Yujin Yuan, Liyuan Liu , Siliang Tang, Zhongfei Zhang, Yueting Zhuang, Shiliang Pu, Fei Wu, and Xiang Ren (2019). Cross-Relation Cross-Bag Attention for Distantly-Supervised Relation Extraction. the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI 2019).

Jingbo Shang*, Liyuan Liu* , Xiaotao Gu, Xiang Ren, Teng Ren, and Jiawei Han (2018). Learning Named Entity Tagger using Domain-Specific Dictionary. the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018).

PDF Code Blog Blog (Chinese) Doc

Liyuan Liu , Xiang Ren, Jingbo Shang, Jian Peng, and Jiawei Han (2018). Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling. the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018). Selected as Oral.

PDF Code Blog Doc

Huan Gui, Qi Zhu, Liyuan Liu , Aston Zhang, and Jiawei Han (2018). Expert Finding in Heterogeneous Bibliographic Networks with Locally-trained Embeddings. arXiv:1803.03370 [cs].

Liyuan Liu , Jingbo Shang, Xiang Ren, Frank Fangzheng Xu, Huan Gui, Jian Peng, and Jiawei Han (2018). Empower Sequence Labeling with Task-Aware Neural Language Model. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI 2018)..

PDF Code (new) Code (old) Blog

Jingbo Shang, Xiyao Shi, Meng Jiang, Liyuan Liu , Timothy Hanratty, and Jiawei Han (2018). Contrast Subgraph Mining from Coherent Cores. arXiv:1802.06189 [cs].

Qi Zhu, Hongwei Ng, Liyuan Liu , Ziwei Ji, Bingjie Jiang, Jiaming Shen, and Huan Gui (2017). Wikidata Vandalism Detection - The Loganberry Vandalism Detector at WSDM Cup 2017. arXiv:1712.06922 [cs].

Carl Yang, Liyuan Liu , Mengxiong Liu, Zongyi Wang, Chao Zhang, and Jiawei Han (2017). Graph Clustering with Embedding Propagation. the 2020 IEEE International Conference on Big Data (IEEE BigData 2020).

Liyuan Liu* , Xiang Ren*, Qi Zhu, Shi Zhi, Huan Gui, Heng Ji, and Jiawei Han (2017). Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach. the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017).

PDF Code Blog Slides

Chao Zhang, Liyuan Liu , Dongming Lei, Quan Yuan, Honglei Zhuang, Tim Hanratty, and Jiawei Han (2017). TrioVecEvent: Embedding-based Online Local Event Detection in Geo-tagged Tweet Streams. the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2017).

Liyuan Liu , Linli Xu, Zhen Wang, and Enhong Chen (2015). Community Detection Based on Structure and Content: A Content Propagation Perspective. the 2015 IEEE International Conference on Data Mining (ICDM 2015).