I am currently a postdoctoral researcher at the Natural Language Processing Laboratory of Fudan University, working under the supervision of Prof. Xuanjing Huang. My postdoctoral tenure is expected to conclude in September 2025. Prior to joining Fudan NLP Lab, I got both my B.S. (2017) and Ph.D. (2023) degree in Computer Science from East China Normal University, advised by Prof. Yuanbin Wu and Xiaoling Wang.

Research

My primary research interests include the architectural design of LLMs, with a particular focus on Transformer position encoding, multimodal LLMs, and CUDA programming.

Publication

(* indicates equal contribution, and 📧 indicates corresponding author)

• Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs

  Tao Ji, Bin Guo, Yuanbin Wu, Qipeng Guo, Lixing Shen, Zhan Chen, Xipeng Qiu, Qi Zhang, Tao Gui📧

  In arXiv. [paper]

• AntLM: Bridging Causal and Masked Language Models

  Xinru Yu*, Bin Guo*, Shiwei Luo*, Jie Wang, Tao Ji📧, Yuanbin Wu📧

  In BabyLM Shared Task 2024. [paper]

• Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs

  Shuo Li*, Tao Ji*, Xiaoran Fan*, Linsheng Lu, Leyi Yang, Yuming Yang, Zhiheng Xi, Rui Zheng, Yuran Wang, Xiaohui Zhao, Tao Gui, Qi Zhang, Xuanjing Huang

  In ICLR 2025. [paper]

• Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models

  Yufang Liu*, Tao Ji*, Changzhi Sun, Yuanbin Wu, Aimin Zhou

  In EMNLP 2024. [bib][paper]

• Generation with Dynamic Vocabulary

  Yanting Liu, Tao Ji📧, Changzhi Sun, Yuanbin Wu📧, Xiaoling Wang📧

  In EMNLP 2024. [bib][paper]

• LongHeads: Multi-Head Attention is Secretly a Long Context Processor

  Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji📧, Tao Gui📧, Qi Zhang📧, Xuanjing Huang

  In Findings of EMNLP 2024. [bib][paper]

• Poly-Visual-Expert Vision-Language Models

  Xiaoran Fan*, Tao Ji*, Changhao Jiang*, Shuo Li*, Senjie Jin*, Sirui Song, Junke Wang, Boyang Hong, Lu Chen, Guodong Zheng, Ming Zhang, Caishuang Huang, Rui Zheng, Zhiheng Xi, Yuhao Zhou, Shihan Dou, Junjie Ye, Hang Yan, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang

  In COLM 2024. [paper]

• Length Generalization of Causal Transformers without Position Encoding

  Jie Wang*, Tao Ji*, Yuanbin Wu, Hang Yan, Tao Gui, Qi Zhang, Xuanjing Huang, Xiaoling Wang

  In Findings of ACL 2024. [bib][paper]

• Rehearsal-free Continual Language Learning via Efficient Parameter Isolation

  Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, Wenqiu Zeng

  In ACL 2023. [bib][paper]

• Typology Guided Multilingual Position Representations: Case on Dependency Parsing

  Tao Ji, Yuanbin Wu, Xiaoling Wang

  In Findings of ACL 2023. [bib][paper]

• Zero-Shot Event Detection Based on Ordered Contrastive Learning and Prompt-Based Prediction

  Senhui Zhang, Tao Ji, Wendi Ji, Xiaoling Wang

  In Findings of NAACL 2022. [bib][paper]

• Explore Unsupervised Structures in Pretrained Models for Relation Extraction

  Xi Yang*, Tao Ji*, Yuanbin Wu

  In Findings of EMNLP 2022. [bib][paper]

• Word Reordering for Zero-shot Cross-lingual Structured Prediction

  Tao Ji, Yong Jiang, Tao Wang, Zhongqiang Huang, Fei Huang, Yuanbin Wu and Xiaoling Wang

  In EMNLP 2021. [bib][paper]

• A Unified Encoding of Structures in Transition Systems

  Tao Ji, Yong Jiang, Tao Wang, Zhongqiang Huang, Fei Huang, Yuanbin Wu and Xiaoling Wang

  In EMNLP 2021. [bib][paper]

• Generating CCG Categories

  Yufang Liu, Tao Ji, Yuanbin Wu, Man Lan

  In AAAI 2021. [bib] [paper]

• Graph-based Dependency Parsing with Graph Neural Networks

  Tao Ji, Yuanbin Wu, Man Lan

  In ACL 2019. [bib] [paper]

• AntNLP at CoNLL 2018 Shared Task: A Graph-based Parser for Universal Dependency Parsing

  Tao Ji, Yufang Liu, Yijun Wang, Yuanbin Wu, Man Lan

  In CoNLL 2018. [bib] [paper]

• ECNU at EPE 2017: Universal Dependencies Representations Parser

  Tao Ji, Yuekun Yao, Qi Zheng, Yuanbin Wu, Man Lan

  In IWPT 2017. [bib] [paper]

• A Fast and Lightweight System for Multilingual Dependency Parsing

  Tao Ji, Yuanbin Wu, Man Lan.

  In CoNLL 2017. [bib] [paper]

Thesis

Ph.D. Thesis: Multilingual Dependency Parsing, 2023, grade 4.0 A

Bachelor Thesis: Dependency Parsing Based on Recurrent Neural Networks, 2017, grade 4.0 A

Education / Experience

• 2023 - Present: Postdoc at Fudan University

• 2020 - 2021: Research Intern at Alibaba DAMO Academy

• 2017 - 2023: Ph.D. student at East China Normal University

• 2013 - 2017: B.Eng from East China Normal University

Selected Awards

• Postdoctoral Fellowship Program of CPSF (C), 2023

• Shanghai "Super Postdoc" Incentive Program, 2023

• Rank 1 at CCL2021 shared task: Multi-source Cross-domain Dependency Parsing, 2021

• Rank 2 at NLPCC2019 shared task: Cross-domain Dependency Parsing, 2019

• National Scholarship, 2018

• First Prize in Chinese undergraduate computer design contest (8th), 2015

• Silver Medal in ACM-ICPC Asia EC-FINAL Contest, 2015

• Silver Medal in ACM-ICPC Asia Xian Regional Contest, 2014

Contact

taoji[at]fudan.edu.cn

A5029, Interdisciplinary Building No.2, Fudan University (Jiangwan Campus)

2005 Songhu Road, Shanghai 200438, China