Siu Hin Ng

Author directory

2026

Dense retrievers can blur the distinctions between short, jargon-heavy records that share most of their vocabulary. We study Lex-TM, a training-free layer that interpolates dense cosine similarity with the Jensen–Shannon similarity of topic distributions from an LDA model whose topic–word prior is scaled by a Herfindahl–Hirschman (HHI) exclusivity score. On NTCIR-13 MedWeb, cast as retrieving tweets that share a query’s exact symptom-label set, Lex-TM reaches an English MRR of 0.8151 against 0.7081 for BM25+dense reciprocal rank fusion, a significant paired difference; in Japanese and Chinese the differences from fusion are not significant. With Qwen3-Embedding-0.6B, routing improves on dense retrieval significantly in English and Chinese. Three findings bound these gains. SPLADE++ and BGE-M3’s multi-vector mode match or exceed Lex-TM. In English and Chinese the gain depends on the label strings in the indexed tweets: without them, routing no longer improves on dense retrieval. The HHI prior brings no benefit: on short texts it becomes a rarity weight, and a symmetric prior performs as well across five Gibbs seeds. Outside MedWeb, routing matches dense retrieval on Chinese medical QA and lowers nDCG@10 on ChatDoctor, FiQA and long biomedical abstracts. Code, seeds and a changelog of fixed bugs are released.
Search
Co-authors
    Venues
    Fix author