Li Dong 董力

VibeVoice Technical Report
Zhiliang Peng*, Jianwei Yu*, Wenhui Wang*, Yaoyao Chang*, Yutao Sun#*, Li Dong*, Yi Zhu, Weijiang Xu, Hangbo Bao, Zehua Wang, Shaohan Huang, Yan Xia, Furu Wei.
arXiv:2508.19205, 2025.
pdf  bib  code  demo  checkpoints

Reward Reasoning Model
Jiaxin Guo#*, Zewen Chi*, Li Dong*, Qingxiu Dong#, Xun Wu, Shaohan Huang, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2025.
pdf  bib  checkpoints

Native Hybrid Thinking Models
Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong#, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2025.
pdf  bib

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
Yinsicheng Jiang, Yao Fu, Yeqi Huang, Ping Nie, Zhan Lu, Leyang Xue, Congjie He, Man-Kit Sit, Jilong Xue, Li Dong, Ziming Miao, Dayou Du, Tairan Xu, Kai Zou, Edoardo Ponti, Luo Mai.
Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2025.
pdf  bib

Scaling Laws of Synthetic Data for Language Models
Zeyu Qin, Qingxiu Dong#, Xingxing Zhang, Li Dong, Xiaolong Huang, Ziyi Yang, Mahmoud Khademi, Dongdong Zhang, Hany Hassan Awadalla, Yi R. Fung, Weizhu Chen, Minhao Cheng, Furu Wei.
Conference on Language Modeling (COLM), 2025.
pdf  bib

Reinforcement Pre-Training
Qingxiu Dong#*, Li Dong*, Yao Tang#, Tianzhu Ye#, Yutao Sun#, Zhifang Sui, Furu Wei.
arXiv:2506.08007, 2025.
pdf  bib

Rectified Sparse Attention
Yutao Sun#*, Tianzhu Ye#*, Li Dong*, Yuqing Xia*, Jian Chen#, Yizhao Gao, Shijie Cao, Jianyong Wang, Furu Wei.
arXiv:2506.04108, 2025.
pdf  bib  code

On-Policy RL with Optimal Reward Baseline
Yaru Hao, Li Dong, Xun Wu, Shaohan Huang, Zewen Chi, Furu Wei.
arXiv:2505.23585, 2025.
pdf  bib   code merged to verl

Imagine while Reasoning in Space: Multimodal Visualization-of-Thought
Chengzu Li*, Wenshan Wu*, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, Furu Wei.
International Conference on Machine Learning (ICML), 2025.
pdf  bib

Differential Transformer
Tianzhu Ye#*, Li Dong*, Yuqing Xia*, Yutao Sun#*, Yi Zhu, Gao Huang, Furu Wei.
International Conference on Learning Representations (ICLR), Oral, 2025.
pdf  bib  code

Data Selection via Optimal Control for Language Models
Yuxian Gu#, Li Dong, Hongning Wang, Yaru Hao, Qingxiu Dong#, Furu Wei, Minlie Huang.
International Conference on Learning Representations (ICLR), Oral, 2025.
pdf  bib  code

Self-Boosting Large Language Models with Synthetic Preference Data
Qingxiu Dong#, Li Dong, Xingxing Zhang, Zhifang Sui, Furu Wei.
International Conference on Learning Representations (ICLR), 2025.
pdf  bib

Semi-Parametric Retrieval via Binary Bag-of-Tokens Index
Jiawei Zhou#, Li Dong, Furu Wei, Lei Chen.
International Conference on Learning Representations (ICLR), 2025.
pdf  bib

WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale
Jiaxi Li, Xingxing Zhang, Xun Wang, Xiaolong Huang, Li Dong, Liang Wang, Si-Qing Chen, Wei Lu, Furu Wei.
arXiv:2502.16684, 2025.
pdf  bib

Multimodal Latent Language Modeling with Next-Token Diffusion
Yutao Sun#*, Hangbo Bao*, Wenhui Wang*, Zhiliang Peng*, Li Dong*, Shaohan Huang, Jianyong Wang, Furu Wei.
arXiv:2412.08635, 2024.
pdf  bib  code

RedStone: Curating General, Code, Math, and QA Data for Large Language Models
RedStone v-team.
arXiv:2412.03398, 2024.
pdf  bib  code

You Only Cache Once: Decoder-Decoder Architectures for Language Models
Yutao Sun#*, Li Dong*, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei.
Neural Information Processing Systems (NeurIPS), Oral, 2024.
pdf  bib  code

Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2024.
pdf  bib

Multi-Head Mixture-of-Experts
Xun Wu, Shaohan Huang, Wenhui Wang, Shuming Ma, Li Dong, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2024.
pdf  bib

Direct Preference Knowledge Distillation for Large Language Models
Yixing Li#, Yuxian Gu#, Li Dong, Dequan Wang, Yu Cheng, Furu Wei.
arXiv:2406.19774, 2024.
pdf  bib  code

Towards Optimal Learning of Language Models
Yuxian Gu#, Li Dong, Yaru Hao, Qingxiu Dong#, Minlie Huang, Furu Wei.
arXiv:2402.17759, 2024.
pdf  bib  code

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Shuming Ma*, Hongyu Wang*, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, Furu Wei.
arXiv:2402.17764, 2024.
pdf  bib

Kosmos-E: Learning to Follow Instruction for Robotic Grasping
Zhi Wang, Xun Wu, Shaohan Huang, Li Dong, Wenhui Wang, Shuming Ma, Furu Wei.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.
pdf  bib  code

Kosmos-G: Generating Images in Context with Multimodal Large Language Models
Xichen Pan#, Li Dong, Shaohan Huang, Zhiliang Peng#, Wenhu Chen, Furu Wei.
International Conference on Learning Representations (ICLR), 2024.
pdf  bib  code

Kosmos-2: Grounding Multimodal Large Language Models to the World
Zhiliang Peng#*, Wenhui Wang*, Li Dong*, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
International Conference on Learning Representations (ICLR), 2024.
pdf  bib  code  demo

Knowledge Distillation of Large Language Models
Yuxian Gu#, Li Dong, Furu Wei and Minlie Huang.
International Conference on Learning Representations (ICLR), 2024.
pdf  bib  code

BioCLIP: A Vision Foundation Model for the Tree of Life
Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth G Campolongo, Chan Hee Song, David Edward Carlyn, Li Dong, Wasila M Dahdul, Charles Stewart, Tanya Berger-Wolf, Wei-Lun Chao, Yu Su.
International Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
pdf  bib  code  model  demo  Best Student Paper Award

BitNet: Scaling 1-bit Transformers for Large Language Models
Hongyu Wang*, Shuming Ma*, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Ruiping Wang, Yi Wu, Furu Wei.
arXiv:2310.11453, 2023.
pdf  bib

Kosmos-2.5: A Multimodal Literate Model
Tengchao Lv, Yupan Huang, Jingye Chen, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, Shaoxiang Wu, Guoxin Wang, Cha Zhang, Furu Wei.
arXiv:2309.11419, 2023.
pdf  bib

Large Language Model for Science: A Study on P vs. NP
Qingxiu Dong#*, Li Dong*, Ke Xu*, Guangyan Zhou, Yaru Hao, Zhifang Sui, Furu Wei.
arXiv:2309.05689, 2023.
pdf  bib  code

Retentive Network: A Successor to Transformer for Large Language Models
Yutao Sun#*, Li Dong*, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, Furu Wei.
arXiv:2307.08621, 2023.
pdf  bib  code

LongNet: Scaling Transformers to 1,000,000,000 Tokens
Jiayu Ding*, Shuming Ma*, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Furu Wei.
arXiv:2307.02486, 2023.
pdf  bib  code

Language Is Not All You Need: Aligning Perception with Language Models
Shaohan Huang*, Li Dong*, Wenhui Wang*, Yaru Hao*, Saksham Singhal*, Shuming Ma*, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi#, Johan Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2023.
pdf  bib  MetaLM

Augmenting Language Models with Long-Term Memory
Weizhi Wang#, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2023.
pdf  bib  code

Optimizing Prompts for Text-to-Image Generation
Yaru Hao*, Zewen Chi#*, Li Dong, Furu Wei.
Neural Information Processing Systems (NeurIPS), Spotlight, 2023.
pdf  bib  code  demo

Extensible Prompts for Language Models
Tao Ge, Jing Hu, Li Dong, Shaoguang Mao, Yan Xia, Xun Wang, Si-Qing Chen, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2023.
pdf  bib

Pre-Training to Learn in Context
Yuxian Gu#, Li Dong, Furu Wei and Minlie Huang.
Association for Computational Linguistics (ACL), 2023.
pdf  bib  code

A Length-Extrapolatable Transformer
Yutao Sun#, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei.
Association for Computational Linguistics (ACL), 2023.
pdf  bib  code

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers
Damai Dai#, Yutao Sun#, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei.
Findings of Association for Computational Linguistics (Findings of ACL), 2023.
pdf  bib

Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning
Barun Patra*, Saksham Singhal*, Shaohan Huang*, Zewen Chi#, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song.
Association for Computational Linguistics (ACL), 2023.
pdf  bib

GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Jian Yang, Shuming Ma, Li Dong, Shaohan Huang, Haoyang Huang, Yuwei Yin, Dongdong Zhang, Liqun Yang, Furu Wei and Zhoujun Li.
Association for Computational Linguistics (ACL), 2023.
pdf  bib

Magneto: A Foundation Transformer
Hongyu Wang*, Shuming Ma*, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng#, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei.
International Conference on Machine Learning (ICML), 2023.
pdf  bib  code

Semi-Offline Reinforcement Learning for Optimized Text Generation
Changyu Chen, Xiting Wang, Yiqiao Jin, Victor Ye Dong, Li Dong, Rui Yan, Jim Cao, Yi Liu.
International Conference on Machine Learning (ICML), 2023.
pdf  bib

Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
Wenhui Wang*, Hangbo Bao#*, Li Dong*, Johan Bjorck, Zhiliang Peng#, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, Furu Wei.
International Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
pdf  bib  code  VL-BEiT

Non-Contrastive Learning Meets Language-Image Pre-Training
Jinghao Zhou#, Li Dong, Zhe Gan, Lijuan Wang, Furu Wei.
International Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
pdf  bib

Generic-to-Specific Distillation of Masked Autoencoders
Wei Huang*, Zhiliang Peng#*, Li Dong, Furu Wei, Jianbin Jiao, Qixiang Ye.
International Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
pdf  bib  code

A Unified View of Masked Image Modeling
Zhiliang Peng#, Li Dong, Hangbo Bao#, Furu Wei, Qixiang Ye.
Transactions on Machine Learning Research, 2023.
pdf  bib  code

Visually-Augmented Language Modeling
Weizhi Wang#, Li Dong, Hao Cheng, Haoyu Song#, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei.
International Conference on Learning Representations (ICLR), 2023.
pdf  bib  code

Corrupted Image Modeling for Self-Supervised Visual Pre-Training
Yuxin Fang#, Li Dong, Hangbo Bao#, Xinggang Wang, Furu Wei.
International Conference on Learning Representations (ICLR), Spotlight, 2023.
pdf  bib  code

Prototypical Calibration for Few-shot Learning of Language Models
Zhixiong Han#, Yaru Hao, Li Dong, Yutao Sun#, Furu Wei.
International Conference on Learning Representations (ICLR), 2023.
pdf  bib

Structured Prompting: Scaling In-Context Learning to 1,000 Examples
Yaru Hao*, Yutao Sun#*, Li Dong, Zhixiong Han, Yuxian Gu, Furu Wei.
arXiv:2212.06713, 2022.
pdf  bib  code

BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers
Zhiliang Peng#, Li Dong, Hangbo Bao#, Qixiang Ye, Furu Wei.
arXiv:2208.06366, 2022.
pdf  bib  code

Language Models are General-Purpose Interfaces
Yaru Hao*, Haoyu Song#*, Li Dong*, Shaohan Huang, Zewen Chi#, Wenhui Wang, Shuming Ma, Furu Wei.
arXiv:2206.06336, 2022.
pdf  bib

On the Representation Collapse of Sparse Mixture of Experts
Zewen Chi#, Li Dong, Shaohan Huang, Damai Dai#, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2022.
pdf  bib  code

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
Hangbo Bao#*, Wenhui Wang*, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Furu Wei.
Neural Information Processing Systems (NeurIPS), 2022.
pdf  bib  code

BEiT: BERT Pre-Training of Image Transformers
Hangbo Bao#, Li Dong, Songhao Piao, Furu Wei.
International Conference on Learning Representations (ICLR), Oral, 2022.
pdf  bib  code

AdaPrompt: Adaptive Model Training for Prompt-based NLP
Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang.
Findings of Empirical Methods in Natural Language Processing (Findings of EMNLP), 2022.
pdf  bib

CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation
Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei.
Findings of Empirical Methods in Natural Language Processing (Findings of EMNLP), 2022.
pdf  bib

Knowledge Neurons in Pretrained Transformers
Damai Dai#, Li Dong, Yaru Hao#, Zhifang Sui, Furu Wei.
Association for Computational Linguistics (ACL), 2022.
pdf  bib

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
Zewen Chi#*, Shaohan Huang*, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei.
Association for Computational Linguistics (ACL), 2022.
pdf  bib

StableMoE: Stable Routing Strategy for Mixture of Experts
Damai Dai#, Li Dong, Shuming Ma, Bo Zheng#, Zhifang Sui, Baobao Chang, Furu Wei.
Association for Computational Linguistics (ACL), 2022.
pdf  bib  code

Controllable Natural Language Generation with Contrastive Prefixes
Jing Qian#, Li Dong, Yelong Shen, Furu Wei, Weizhu Chen.
Findings of Association for Computational Linguistics (Findings of ACL), 2022.
pdf  bib

CLIP Models are Few-Shot Learners: Empirical Studies on VQA and Visual Entailment
Haoyu Song#, Li Dong, Weinan Zhang, Ting Liu, Furu Wei.
Association for Computational Linguistics (ACL), 2022.
pdf  bib

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption
Tianyu Chen*, Hangbo Bao#*, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, Furu Wei.
Findings of Association for Computational Linguistics (Findings of ACL), 2022.
pdf  bib

Swin Transformer V2: Scaling Up Capacity and Resolution
Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, Baining Guo.
International Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
pdf  bib  code

Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training
Bo Zheng#, Li Dong, Shaohan Huang, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song and Furu Wei.
Empirical Methods in Natural Language Processing (EMNLP), 2021.
pdf  bib  code

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Zewen Chi#, Li Dong, Shuming Ma, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei.
Empirical Methods in Natural Language Processing (EMNLP), 2021.
pdf  bib

Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei.
Empirical Methods in Natural Language Processing (EMNLP), 2021.
pdf  bib

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
Zewen Chi#, Li Dong, Bo Zheng#, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei.
Association for Computational Linguistics (ACL), 2021.
pdf  bib  code

Consistency Regularization for Cross-Lingual Fine-Tuning
Bo Zheng#, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi#, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei.
Association for Computational Linguistics (ACL), 2021.
pdf  bib  code

Learning to Sample Replacements for ELECTRA Pre-Training
Yaru Hao#, Li Dong, Hangbo Bao#, Ke Xu, Furu Wei.
Findings of Association for Computational Linguistics (Findings of ACL), 2021.
pdf  bib  code

Memory-Efficient Differentiable Transformer Architecture Search
Yuekai Zhao#, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen.
Findings of Association for Computational Linguistics (Findings of ACL), 2021.
pdf  bib

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei.
Findings of Association for Computational Linguistics (Findings of ACL), 2021.
pdf  bib  code

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains
Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei.
Findings of Association for Computational Linguistics (Findings of ACL), 2021.
pdf  bib  code

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Zewen Chi#, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou.
North American Association for Computational Linguistics (NAACL), 2021.
pdf  bib  code

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Yaru Hao#, Li Dong, Furu Wei, Ke Xu.
AAAI Conference on Artificial Intelligence (AAAI), 2021.
pdf  bib  code  Best Paper Runner-up

DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
arXiv:2106.13736, 2021.
pdf  bib  code

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei.
arXiv:2012.15547, 2020.
pdf  bib  code

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Hangbo Bao#, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon.
International Conference on Machine Learning (ICML), 2020.
pdf  bib  code

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou.
Neural Information Processing Systems (NeurIPS), 2020.
pdf  bib  code

Cross-Lingual Natural Language Generation via Pre-Training
Zewen Chi#, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, Heyan Huang.
AAAI Conference on Artificial Intelligence (AAAI), 2020.
pdf  code  bib

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Xiujun Li, Xi Yin, Chunyuan Li, Xiaowei Hu, Pengchuan Zhang, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao.
European Conference on Computer Vision (ECCV), 2020.
pdf  bib  code  blog

Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Zhongli Li#, Wenhui Wang, Li Dong, Furu Wei, Ke Xu.
Association for Computational Linguistics (ACL), 2020.
pdf  code  bib

Unified Language Model Pre-training for Natural Language Understanding and Generation
Li Dong*, Nan Yang*, Wenhui Wang*, Furu Wei*, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon.
Neural Information Processing Systems (NeurIPS), 2019.
pdf  code  bib

Visualizing and Understanding the Effectiveness of BERT
Yaru Hao#, Li Dong, Furu Wei, Ke Xu.
Empirical Methods in Natural Language Processing (EMNLP), 2019.
pdf  bib

Data-to-text Generation with Entity Modeling
Ratish Puduppully, Li Dong, Mirella Lapata.
Association for Computational Linguistics (ACL), 2019.
pdf  code  bib

Learning to Ask Unanswerable Questions for Machine Reading Comprehension
Haichao Zhu#, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu.
Association for Computational Linguistics (ACL), 2019.
pdf  data  bib

Data-to-Text Generation with Content Selection and Planning
Ratish Puduppully, Li Dong, Mirella Lapata.
AAAI Conference on Artificial Intelligence (AAAI), 2019.
pdf  code  data  bib

Coarse-to-Fine Decoding for Neural Semantic Parsing
Li Dong, Mirella Lapata.
Association for Computational Linguistics (ACL), 2018.
pdf  code  bib  Best Paper Honourable Mention

Confidence Modeling for Neural Semantic Parsing
Li Dong, Chris Quirk, Mirella Lapata.
Association for Computational Linguistics (ACL), 2018.
pdf  code  bib

Learning to Paraphrase for Question Answering
Li Dong, Jonathan Mallinson, Siva Reddy, Mirella Lapata.
Empirical Methods in Natural Language Processing (EMNLP), 2017.
pdf  bib

Learning to Generate Product Reviews from Attributes
Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, Ke Xu.
European Chapter of the Association for Computational Linguistics (EACL), 2017.
pdf  code  data  bib

Language to Logical Form with Neural Attention
Li Dong, Mirella Lapata.
Association for Computational Linguistics (ACL), 2016.
pdf  code  bib  slides

Long Short-Term Memory-Networks for Machine Reading
Jianpeng Cheng, Li Dong, Mirella Lapata.
Empirical Methods in Natural Language Processing (EMNLP), 2016.
pdf  code  bib

Solving and Generating Chinese Character Riddles
Chuanqi Tan, Furu Wei, Li Dong, Weifeng Lv, Ming Zhou.
Empirical Methods in Natural Language Processing (EMNLP), 2016.
pdf  bib

Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction
Yichun Yin, Furu Wei, Li Dong, Kaimeng Xu, Ming Zhang, Ming Zhou.
International Joint Conference on Artificial Intelligence (IJCAI), 2016.
pdf  bib

Question Answering over Freebase with Multi-Column Convolutional Neural Networks
Li Dong, Furu Wei, Ming Zhou, Ke Xu.
Association for Computational Linguistics (ACL), 2015.
pdf  bib  slides

A Hybrid Neural Model for Type Classification of Entity Mentions
Li Dong, Furu Wei, Hong Sun, Ming Zhou, Ke Xu.
International Joint Conference on Artificial Intelligence (IJCAI), 2015.
pdf  bib  slides

Ranking with Recursive Neural Networks and Its Application to Multi-document Summarization
Ziqiang Cao, Furu Wei, Li Dong, Sujian Li, Ming Zhou.
AAAI Conference on Artificial Intelligence (AAAI), 2015.
pdf  bib

Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification
Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, Ke Xu.
Association for Computational Linguistics (ACL), Short paper, 2014.
pdf  data  bib

Adaptive Multi-Compositionality for Recursive Neural Models with Applications to Sentiment Analysis
Li Dong, Furu Wei, Ming Zhou, Ke Xu.
AAAI Conference on Artificial Intelligence (AAAI), 2014.
pdf  bib  slides

A Joint Segmentation and Classification Framework for Sentiment Analysis
Duyu Tang, Furu Wei, Bing Qin, Li Dong, Ting Liu, Ming Zhou.
Empirical Methods in Natural Language Processing (EMNLP), 2014.
pdf  bib

The Automated Acquisition of Suggestions from Tweets
Li Dong, Furu Wei, Yajuan Duan, Xiaohua Liu, Ming Zhou, Ke Xu.
AAAI Conference on Artificial Intelligence (AAAI), 2013.
pdf  slides  data  bib

MoodLens: An Emoticon-Based Sentiment Analysis System for Chinese Tweets
Jichang Zhao, Li Dong, Junjie Wu, Ke Xu.
Knowledge Discovery and Data Mining (KDD), Demo paper, 2012.
pdf  demo  poster  video  data  bib

Model as a Game: On Numerical and Spatial Consistency for Generative Games
Jingye Chen, Yuzhong Zhao, Yupan Huang, Lei Cui, Li Dong, Tengchao Lv, Qifeng Chen, Furu Wei.
ICCV-25 Workshop on Human-Interactive Generation and Editing (HiGen), 2025.
pdf  bib  Best Paper Runner-up

Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei.
Sixth Conference on Machine Translation (WMT), 2021.
pdf  bib  code  (The system submission is based on DeltaLM.)

Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension
Hangbo Bao#, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Lei Cui, Songhao Piao, Ming Zhou.
Proceedings of the 2nd Workshop on Machine Reading for Question Answering (MRQA), 2019.
pdf  bib  code

Splusplus: A Feature-Rich Two-stage Classifier for Sentiment Analysis of Tweets
Li Dong, Furu Wei, Yichun Yin, Ming Zhou, Ke Xu.
International Workshop on Semantic Evaluation (SemEval), 2015.
pdf  bib

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
Haoran Li*, Qingxiu Dong#*, Zhengyang Tang*, Chaojun Wang*, Xingxing Zhang*, Haoyang Huang*, Shaohan Huang, Xiaolong Huang, Zeqiang Huang, Dongdong Zhang, Yuxian Gu#, Xin Cheng, Xun Wang, Si-Qing Chen, Li Dong, Wei Lu, Zhifang Sui, Benyou Wang, Wai Lam, Furu Wei.
Transactions on Machine Learning Research (TMLR), 2025.
pdf  bib

BitNet: 1-bit Pre-training for Large Language Models
Hongyu Wang, Shuming Ma, Lingxiao Ma, lei wang, Wenhui Wang, Li Dong, Shaohan Huang, Huaijie Wang, Jilong Xue, Ruiping Wang, Yi Wu, and Furu Wei.
Journal of Machine Learning Research (JMLR), 2025.
pdf  bib  code

DeepNet: Scaling Transformers to 1,000 Layers
Hongyu Wang*, Shuming Ma*, Li Dong, Shaohan Huang, Dongdong Zhang, Furu Wei.
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024.
pdf  bib  code

Generic-to-Specific Distillation of Masked Autoencoders
Wei Huang*, Zhiliang Peng#*, Li Dong, Furu Wei, Jianbin Jiao, Qixiang Ye.
IEEE Transactions on Circuits and Systems for Video Technology, 2024.
pdf  bib  code

Transforming Wikipedia into Augmented Data for Query-Focused Summarization
Haichao Zhu#, Li Dong, Furu Wei, Bing Qin, Ting Liu.
IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 2022.
pdf  bib

Adaptive Multi-Compositionality for Recursive Neural Network Models
Li Dong, Furu Wei, Ke Xu, Shixia Liu, Ming Zhou.
IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 2015.
online

A Statistical Parsing Framework for Sentiment Classification
Li Dong, Furu Wei, Shujie Liu, Ming Zhou, Ke Xu.
Computational Linguistics, 2015.
pdf  bib  slides

A Joint Segmentation and Classification Framework for Sentence Level Sentiment Classification
Duyu Tang, Bing Qin, Furu Wei, Li Dong, Ting Liu, Ming Zhou.
IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 2015.
online

Unraveling the origin of exponential law in intra-urban human mobility
Xiao Liang, Jichang Zhao, Li Dong, Ke Xu.
Scientific Reports, 2013.
online

Performance of Local Information Based Link Prediction: A Sampling Perspective
Jichang Zhao, Xu Feng, Li Dong, Xiao Liang, Ke Xu.
Journal of Physics A: Mathematical and Theoretical. 45 345001, 2012.
online

Learning Natural Language Interfaces with Neural Models
The University of Edinburgh, 2019.
online  AIMatters (invited)  AAAI/ACM SIGAI Doctoral Dissertation Award (Runner-up)

ICCV-25 Workshop on Human-Interactive Generation and Editing, Best Paper Runner-up, 2025

CVPR-24 Best Student Paper Award, 2024

AAAI-21 Best Paper Runner-up, 2021

2019 AAAI/ACM SIGAI Doctoral Dissertation Award (Runner-up), 2020

ACL-18 Best Paper Honourable Mention, 2018

Adeptmind Scholar Fellowship, 2018

Microsoft Research Asia Fellowship, 2015

Graduate Student Scholarship 1st, 2012

AirBus Scholarship, 2011

National Scholarship, 2010/2014

Qian-Shen Scholarship 1st, 2009