Liyuan Liu

Ph.D. Candidate @ DMG


Hi there!

Welcome to Liyuan Lucas Liu (刘力源)’s webpage! I am a Ph.D. student at the University of Illinois at Urbana-Champaign (UIUC). My Thesis advisor is Prof. Jiawei Han, and my undergraduate advisor is Prof. Linli Xu. If you are going to visit UIUC, please let me buy you a bubble tea. House Milk Tea and Black Sugar Milk Tea (w. no additional sugar) are pretty good .

I would join Microsoft Research @ Redmond in Feb, 2022 .


  • Resource-Economical Methods
  • Broaden Supervision Sources
  • Automate Training Workflows


  • Ph.D. in Computer Science (in progress)

    University of Illinois at Urbana-Champaign

  • B.Eng. in Computer Science, 2016

    University of Science and Technology of China

Things I do

... and want to do

Getting better resource productivity is crucial to accommodate the ever-growing resource consumption of neural networks, and such productivity is in great demand to advance cutting-edge models or apply them to real-world applications. Correspondingly, my research is formed as a “pincer movement”, attacking the productivity regarding the two major types of training cost, i.e., annotation cost and computational cost:

Fun Facts

  • Received more than 3,000 GitHub stars in total! It is ranked 2,041 among all github users (according to Gitstar).
    • Although I doubt the Gitstar ranking is incomplete and outdated , it is still nice to have such an encouragement .
  • Torch-Scope has been downloaded by more than 32,000 times.
    • Although this number cannot reflect the actual user number , there must be someone other than myself using this package .
  • Won the topcoder Arabic NER challenge.
    • Since I know nothing about Arabic, my model has surely surpassed me on that . On second thought, why I am happy for being worse than PC…
  • Love skiing (can do black run, but still a rookie); DJI fans (proudly own Mavic Pro, Mavic Mini & Spark); love to watch Texas Hold’em (but seldom play); play Sheng Ji (双扣) and Mafia (狼人杀) with families & friends.

Selected Publications

List of all publications >>

* indicates equal contribution.

Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training
Understanding the Difficulty of Training Transformers
On the Variance of the Adaptive Learning Rate and Beyond
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations
Learning Named Entity Tagger using Domain-Specific Dictionary
Empower Sequence Labeling with Task-Aware Neural Language Model
Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach

Highlighted Honors

List of all honors »

Winner of the Topcoder Arabic NER Challenge

Ranked 1st among 137 registrants and 220 submissions.

Guo Moruo Scholarship

Highest honor for USTC undergraduate students.

Google Excellent Scholarship

Only 58 graduate and undergraduate students shortlisted nationwide.

Reading Group

The latest reading group is available at Google Group. The reading lists are:

DMG Group Meeting

The latest DMG group meeting notification is available at Google Group.

PC Member

I’ve served as a PC Member for ACL2020, WWW2020, AAAI2020, IJCAI2020, EMNLP2019, LLD2019, ACL2019, NAACL2019, AAAI2019, and EMNLP2018.


… stay in touch!