Zhiqi Huang

photo @ Yellowstone

Hi, I am an Applied Researcher at Capital One. I completed my Ph.D. under the mentorship of my advisor, Prof. James Allan, at the Center for Intelligent Information Retrieval (CIIR), Manning College of Information and Computer Sciences, University of Massachusetts Amherst. My research endeavors lie at the intersection of information retrieval, natural language processing, and machine learning (IR/NLP/ML).

I received my master’s in Statistics from the University of Maryland, College Park, and my bachelor’s in Applied Math from Sun Yat-sen University.

news

Oct 29, 2025	Our paper, Distillation versus Contrastive Learning: How to Train Your Rerankers, has been accepted at the IJCNLP-AACL 2025.
Oct 15, 2025	I will be presenting our work, Confidence-Based Response Abstinence: Improving LLM Trustworthiness via Activation-Based Uncertainty Estimation, at the 2nd UncertaiNLP Workshop at EMNLP 2025. See you in Suzhou, China!
Aug 29, 2025	Check out our work on uncertainty quantification, Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question Answering, which will be presented at the Reliable ML Workshop at NeurIPS 2025.
Jul 10, 2025	Looking for a tool for uncertainty quantification in LLMs? Check out TruthTorchLM (code) — to be presented at EMNLP 2025.
Feb 20, 2025	Check our survey, A Survey of Model Architectures in Information Retrieval, for the development of information retrieval (IR) modeling and challenges in the era of large language models (LLMs).

selected publications

SIGIR

Soft Prompt Decoding for Multilingual Dense Retrieval

Zhiqi Huang, Hansi Zeng, Hamed Zamani, and James Allan

In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023

PDF
EMNLP

Language Concept Erasure for Language-invariant Dense Retrieval

Zhiqi Huang, Puxuan Yu, Shauli Ravfogel, and James Allan

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

Abs PDF

Multilingual models aim for language-invariant representations but still prominently encode language identity. This, along with the scarcity of high-quality parallel retrieval data, limits their performance in retrieval. We introduce LANCER, a multi-task learning framework that improves language-invariant dense retrieval by reducing language-specific signals in the embedding space. Leveraging the notion of linear concept erasure, we design a loss function that penalizes cross-correlation between representations and their language labels. LANCER leverages only English retrieval data and general multilingual corpora, training models to focus on language-invariant retrieval by semantic similarity without necessitating a vast parallel corpus. Experimental results on various datasets show our method consistently improves over baselines, with extensive analyses demonstrating greater language agnosticism.
EMNLP

Confidence-Based Response Abstinence: Improving LLM Trustworthiness via Activation-Based Uncertainty Estimation

Zhiqi Huang, Vivek Datla, Chenyang Zhu, Alfy Samuel, Daben Liu, Anoop Kumar, and Ritesh Soni

In Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025), Nov 2025

Abs PDF

We propose a method for confidence estimation in retrieval-augmented generation (RAG) systems that aligns closely with the correctness of large language model (LLM) outputs. Confidence estimation is especially critical in high-stakes domains such as finance and healthcare, where the cost of an incorrect answer outweighs that of not answering the question. Our approach extends prior uncertainty quantification methods by leveraging raw feed-forward network (FFN) activations as auto-regressive signals, avoiding the information loss inherent in token logits and probabilities after projection and softmax normalization. We model confidence prediction as a sequence classification task, and regularize training with a Huber loss term to improve robustness against noisy supervision. Applied in a real-world financial industry customer-support setting with complex knowledge bases, our method outperforms strong baselines and maintains high accuracy under strict latency constraints. Experiments on Llama 3.1 8B model show that using activations from only the 16th layer preserves accuracy while reducing response latency. Our results demonstrate that activation-based confidence modeling offers a scalable, architecture-aware path toward trustworthy RAG deployment.