Kewei Tu （屠可伟）

Associate Professor and Assistant Dean
School of Information Science and Technology
ShanghaiTech University

[Research Interests] [Talks] [Publications] [Teaching] [Students] [Biography] [Contact Me]

Research Interests

My research interest is in natural language processing, machine learning, and artificial intelligence in general. My current research focuses on 1) combining symbolic, probabilistic and neural approaches to the representation, learning and utilization of linguistic structures, and 2) integrating linguistic structures into transformers and large language models.

Selected Recent Talks

Neuralizing Statistical Approaches to NLP (Last update: Nov. 2023)
Neuralizing Symbolic Approaches to NLP (Last update: Jul. 2022)
Unsupervised Natural Language Parsing (EACL 2021 Tutorial)
Unsupervised Natural Language Parsing (Last update: Jul. 2019)
Representation and Learning of Grammars Beyond the Language Domain (Last update: Nov. 2017)

(Full Publication List)

Selected Recent Publications

Haoyi Wu and Kewei Tu, "Layer-Condensed KV Cache for Efficient Inference of Large Language Models". In the 62th Annual Meeting of the Association for Computational Linguistics (ACL 2024), Bangkok, Thailand, August 11–16, 2024. (Code)
Yida Zhao, Chao Lou, and Kewei Tu, "Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models". In the 62th Annual Meeting of the Association for Computational Linguistics (ACL 2024), Bangkok, Thailand, August 11–16, 2024. (Code)
Xiang Hu, Pengyu Ji, Qingyang Zhu, Wei Wu, and Kewei Tu, "Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale". In the 62th Annual Meeting of the Association for Computational Linguistics (ACL 2024), Bangkok, Thailand, August 11–16, 2024. (Code)
Xiang Hu, Qingyang Zhu, Kewei Tu, Wei Wu, "Augmenting transformers with recursively composed multi-grained representations". In the Twelfth International Conference on Learning Representations (ICLR 2024), Vienna, Austria, May 7-11, 2024. (Code)
Chaoyi Ai and Kewei Tu, "Frame Semantic Role Labeling Using Arbitrary-Order Conditional Random Fields". In the 38th Annual AAAI Conference on Artificial Intelligence (AAAI 2024), Vancouver, Canada, February 20-27, 2024. (Poster)(Code)
Tianyu Yu, Chengyue Jiang, Chao Lou, Shen Huang, Xiaobin Wang, Wei Liu, Jiong Cai, Li Yangning, Yinghui Li, Kewei Tu, Hai-Tao Zheng, Ningyu Zhang, Pengjun Xie, Fei Huang, and Yong Jiang, "SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding". In the 38th Annual AAAI Conference on Artificial Intelligence (AAAI 2024), Vancouver, Canada, February 20-27, 2024. (Code)
Chao Lou and Kewei Tu, "AMR Parsing with Causal Hierarchical Attention and Pointers". In the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), Singapore, December 6-10, 2023. (Poster)(Code)
Zhaohui Yan, Songlin Yang, Wei Liu, and Kewei Tu, "Joint Entity and Relation Extraction with Span Pruning and Hypergraph Neural Networks". In the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), Singapore, December 6-10, 2023. (Slides)(Poster)(Code)
Zhuo Chen, Chengyue Jiang, and Kewei Tu, "Using Interpretation Methods for Model Enhancement". In the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), Singapore, December 6-10, 2023. (Poster)(Code)
Wei Liu, Songlin Yang, Yoon Kim, and Kewei Tu, "Simple Hardware-Efficient PCFGs with Independent Left and Right Productions". In Findings of EMNLP, 2023. (Slides)(Poster)(Code)
Pengyu Ji, Songlin Yang, and Kewei Tu, "Improving Span Representation by Efficient Span-Level Attention". In Findings of EMNLP, 2023. (Poster)(Code)
Haoyi Wu, Wenyang Hui, Yezeng Chen, Weiqi Wu, Kewei Tu, and Yi Zhou, "Conic10K: A Challenging Math Problem Understanding and Reasoning Dataset". In Findings of EMNLP, 2023. (Slides)(Poster)(Code)
Chengyue Jiang, Wenyang Hui, Yong Jiang, Xiaobin Wang, Pengjun Xie and Kewei Tu, "Recall, Expand, and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing". In the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023), Toronto, Canada, July 9-14, 2023. (Slides)(Poster)(Code)
Zixia Jia, Zhaohui Yan, Wenjuan Han, Zilong Zheng and Kewei Tu, "Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field". In the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023), Toronto, Canada, July 9-14, 2023. (Slides)(Poster)(Code)
Songlin Yang and Kewei Tu, "Don't Parse, Choose Spans! Continuous and Discontinuous Constituency Parsing via Autoregressive Span Selection". In the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023), Toronto, Canada, July 9-14, 2023. (Slides)(Code)
Weiqi Wu, Chengyue Jiang, Yong Jiang, Pengjun Xie and Kewei Tu, "Do PLMs Know and Understand Ontological Knowledge?". In the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023), Toronto, Canada, July 9-14, 2023. (Outstanding Paper Award) (Slides)(Poster)(Code&Data)
Jiong Cai, Shen Huang, Yong Jiang, Zeqi Tan, Pengjun Xie and Kewei Tu, "Improving Low-resource Named Entity Recognition with Graph Propagated Data Augmentation". In the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023), Toronto, Canada, July 9-14, 2023. (Poster)(Code)
Chao Lou and Kewei Tu, "Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints". In the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023), Toronto, Canada, July 9-14, 2023. (Slides)(Poster)(Code)
Haoyi Wu and Kewei Tu, "Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation". In Findings of ACL, 2023. (Slides)(Poster)(Code)
Wei Liu, Songlin Yang and Kewei Tu, "Structured Mean-Field Variational Inference for Higher-Order Span-Based Semantic Role Labeling". In Findings of ACL, 2023. (Slides)(Poster)(Code)
Xiang Hu, Xinyu Kong, and Kewei Tu, "A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification". In the Eleventh International Conference on Learning Representations (ICLR 2023), May 1–5, 2023. (Code)
Chengyue Jiang, Yong Jiang, Weiqi Wu, Pengjun Xie, and Kewei Tu, "Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field". In the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), December 7–11, 2022. (Slides)(Poster)(Code)
Songlin Yang, Wei Liu, and Kewei Tu, "Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs". In 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022), Seattle, Washington, July 10-15, 2022. (Slides)(Code)
Hao Zhou, Gongshen Liu, and Kewei Tu, "Improving Constituent Representation with Hypertree Neural Networks". In 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022), Seattle, Washington, July 10-15, 2022. (Slides)(Poster)(Code)
Xinyu Wang, Min Gui, Yong Jiang, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, and Kewei Tu, "ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition". In 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022), Seattle, Washington, July 10-15, 2022. (Slides)(Poster)(Code)
Songlin Yang and Kewei Tu, "Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks". In the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022), Dublin, Ireland, May 22-27, 2022. (Slides)(Poster)(Code)
Chao Lou, Songlin Yang, and Kewei Tu, "Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing". In the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022), Dublin, Ireland, May 22-27, 2022. (Slides)(Poster)(Code)
Songlin Yang and Kewei Tu, "Headed-Span-Based Projective Dependency Parsing". In the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022), Dublin, Ireland, May 22-27, 2022. (Slides)(Poster)(Code)
Zixia Jia, Zhaohui Yan, Haoyi Wu, and Kewei Tu, "Span-based Semantic Role Labeling with Argument Pruning and Second-Order Inference". In the 36th AAAI Conference on Artificial Intelligence (AAAI 2022), Vancouver, Canada, February 22 – March 1, 2022. (Slides)(Poster)(Code)

Selected Older Publications

Chengyue Jiang, Zijian Jin, and Kewei Tu, "Neuralizing Regular Expressions for Slot Filling". In the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021), November 7-11, 2021. (Slides)(Poster)(Code)
Songlin Yang, Yanpeng Zhao, and Kewei Tu, "Neural Bi-Lexicalized PCFG Induction". In the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), August 1-6, 2021. (Slides)(Code)
Xinyu Wang, Yong Jiang, Zhaohui Yan, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, and Kewei Tu, "Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor". In the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), August 1-6, 2021. (Slides)(Code)
Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, and Kewei Tu, "Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning". In the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), August 1-6, 2021. (Slides)(Code)
Songlin Yang, Yanpeng Zhao, and Kewei Tu, "PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols". In the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2021), June 6-11, 2021. (Slides)(Poster)(Code)
Wenjuan Han, Yong Jiang, Hwee Tou Ng, and Kewei Tu, "A Survey of Unsupervised Dependency Parsing". In the 28th International Conference on Computational Linguistics (COLING 2020), December 8-13, 2020. (Slides)
Chengyue Jiang, Yinggong Zhao, Shanbo Chu, Libin Shen, and Kewei Tu, "Cold-start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks". In the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), November 16–20, 2020. (Slides)(Code)
Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, and Kewei Tu, "Learning Numeral Embedding". In Findings of EMNLP, 2020. (Code)
Xinyu Wang and Kewei Tu, "Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training". In the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL-IJCNLP 2020), December 4-7, 2020. (Slides)(Code)
Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Fei Huang, and Kewei Tu, "Structure-Level Knowledge Distillation For Multilingual Sequence Labeling". In the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), July 5–10, 2020. (Slides)(Code)
Xinyu Wang, Jingxian Huang, and Kewei Tu, "Second-Order Semantic Dependency Parsing with End-To-End Neural Networks". In the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), Florence, Italy, July 28 – August 2, 2019. (Poster)(Code)(Code v2)(Erratum)
Wenjuan Han, Yong Jiang, and Kewei Tu, "Enhancing Unsupervised Generative Dependency Parser with Contextual Information". In the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), Florence, Italy, July 28 – August 2, 2019. (Poster)
Yanpeng Zhao, Liwen Zhang, and Kewei Tu, "Gaussian Mixture Latent Vector Grammars". In the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), Melbourne, Australia, July 15–20, 2018. (Supplementary material)(Slides)(Code)
Jun Mei, Yong Jiang, and Kewei Tu, "Maximum A Posteriori Inference in Sum-Product Networks". In the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI 2018), New Orleans, Lousiana, USA, February 2–7, 2018. (Supplementary material)(Slides)(Code)
Jiong Cai, Yong Jiang, and Kewei Tu, "CRF Autoencoder for Unsupervised Dependency Parsing". In the Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), Copenhagen, Denmark, September 7–11, 2017. (Code)(Poster)
Yong Jiang, Wenjuan Han, and Kewei Tu, "Unsupervised Neural Dependency Parsing". In the Conference on Empirical Methods in Natural Language Processing (EMNLP 2016), Austin, Texas, USA, November 1-5, 2016. (Poster)(Reimplementation)
Kewei Tu, "Stochastic And-Or Grammars: A Unified Framework and Logic Perspective". In the 25th International Joint Conference on Artificial Intelligence (IJCAI 2016), New York City, USA, July 9-15, 2016. (Supplementary material)(Slides)(Poster)
Kewei Tu, Meng Meng, Mun Wai Lee, Tae Eun Choe, and Song-Chun Zhu, "Joint Video and Text Parsing for Understanding Events and Answering Queries". In IEEE MultiMedia, vol. 21, no. 2, pp. 42-70, 2014.(arXiv)(Website)
Kewei Tu, Maria Pavlovskaia, and Song-Chun Zhu, "Unsupervised Structure Learning of Stochastic And-Or Grammars". In Advances in Neural Information Processing Systems 26 (NIPS 2013), Lake Tahoe, Nevada, USA, December 5-10, 2013. (Supplementary material)(Poster)(Datasets)(Code)
Kewei Tu and Vasant Honavar, "Unambiguity Regularization for Unsupervised Learning of Probabilistic Grammars". In Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing and Natural Language Learning (EMNLP-CoNLL 2012), Jeju, Korea, July 12-14, 2012. (Slides)
Kewei Tu and Vasant Honavar, "On the Utility of Curricula in Unsupervised Learning of Probabilistic Grammars". In Proceedings of the Twenty-second International Joint Conference on Artificial Intelligence (IJCAI 2011), Barcelona, Catalonia, Spain, July 16-22, 2011. (Supplementary material)(Slides)

Teaching

CS 274A: Natural Language Processing (Spring 2022-2024)
CS 181/281: Artificial Intelligence (Spring 2015, 2017, 2018; Fall 2018-2023)
CS 121/240: Algorithm Design and Analysis (Fall 2014, 2015, 2017; Spring 2019-2021)
CS 101: Programming Language and Data Structure (Fall 2016)

Textbook

屠可伟、王新宇、曲彦儒、俞勇：《动手学自然语言处理》 (Hands-on Natural Language Processing)

Students

Current Students

PhD students: Chengyue Jiang, Chao Lou
Master students: Zhuo Chen, Wei Liu, Chaoyi Ai, Haoyi Wu, Wenyang Hui, Yida Zhao, Pengyu Ji

Alumni (year of graduation, initial position)

PhD: Jiong Cai (2024, Huawei), Zhaohui Yan (2023, 军科院), Xinyu Wang (2023, Alibaba DAMO Academy), Zixia Jia (2023, Beijing Institute for General AI), Liwen Zhang (2021, Baidu), Ge Wang (2021, 中国商飞), Yixian Liu (2020, Tencent), Wenjuan Han (2019, Postdoc@National University of Singapore), Yong Jiang (2019, Alibaba DAMO Academy)
Master: Songlin Yang (2023, PhD at MIT), Zechuan Hu (2022, NetEase), Zhao Li (2021, 金沙江创投), Jun Li (2020, Leyan Technologies), Jun Mei (2019, Pony.ai), Chen Zhu (2018, PhD at University of Maryland), Yanpeng Zhao (2018, PhD at University of Edinburgh), Yang Zhou (2018, Leyan Technologies), Shanbo Chu (2017, Leyan Technologies), Jun Li (2016, Tencent)

Biography

Kewei Tu received BS and MS degrees in Computer Science and Technology from Shanghai Jiaotong University, China in 2002 and 2005 respectively and received a PhD degree in Computer Science from Iowa State University, USA in 2012. During 2012-2014, he worked as a postdoctoral researcher at Departments of Statistics and Computer Science of the University of California, Los Angeles, USA. Since 2014, he has been an assistant professor and then an associate professor with the School of Information Science and Technology at ShanghaiTech University, Shanghai, China. He has around 100 publications in major conferences and journals including ACL, EMNLP, NAACL, AAAI, etc. He served as a PC member at many NLP and AI conferences, as a (senior) area chair at several conferences such as EMNLP, AAAI and AACL, and as an action editor of ACL Rolling Review. He received an outstanding paper award at ACL 2023, two best system paper awards at SemEval 2022 and SemEval 2023, and best paper nominations at multiple top conferences.

Contact Me

Email	tukw shanghaitech.edu.cn
Tel	+86-21-20685089
Office	393 Middle Huaxia Road SIST 1A-304B / 1A-215 Pudong, Shanghai, 201210, China

Last update: June 3, 2024