Salekul Islam

Author directory

2026

Large-scale annotated datasets underpin progress in natural language understanding, yet human annotation remains a persistent bottleneck. Large Language Models promise to bridge this gap, but their reliability as annotators is poorly understood. We evaluate 14 open-weight instruction-tuned models and 5 matched base–instruct pairs across 51,189 instances spanning 10 heterogeneous datasets, revealing that annotation failures are structurally predictable rather than random. We identify three recurring failure modes: Safety Hallucination, Intensity Collapse, and Entailment Bias. Safety hallucination can precede alignment rather than being introduced by it: some base models flag up to 100% of benign text as hate speech. Alignment benefits scale inversely with base model capability, yielding a +21.6pp gain for the weakest base model yet a negligible +0.5pp shift for the strongest. KL divergence between base and instruct distributions is 2.0x higher on incorrect predictions, providing a lightweight instance-level signal for alignment-associated errors. Inter-prompt unanimity achieves 72–94% accuracy at 71–90% coverage on clean tasks, yet can actively degrade performance on tasks dominated by shared systematic bias. We translate these findings into a practical deployment framework with pre-screening diagnostics and tiered quality-routing policies, establishing principled boundaries for when LLM annotation can be trusted and when human intervention remains necessary.
The exponential growth of scientific literature has made personalized, factually grounded paper discovery increasingly difficult. Existing systems suffer from limited personalization, cold-start fragility, or hallucinated explanations. We introduce HybridScholar, an integrated architecture combining a CrossAttention-gated fusion of collaborative filtering and RAG learning candidate-specific gate weights per candidate with a Chain-of-Verification (CoVe) pipeline for factually grounded explanations. Evaluated on the Scholar Inbox dataset, HybridScholar substantially outperforms (NDCG@10=0.2991) strong baselines including LightGCN and Scholar-Inbox. Cold-start analysis shows the RAG fallback prevents hard failure for sparse users (NDCG@10=0.283), with cold NDCG rising monotonically as interactions grow. The CoVe pipeline reduces hallucination rate by 23.8%, confirmed by a human evaluation study showing CoVe-revised explanations are preferred by experts on factual accuracy (76.7% vs. 36.0%) and by end-users on helpfulness (78.8%, p<0.0001). Overall, HybridScholar improves ranking quality, cold-start robustness, and factual reliability for large-scale scholarly research discovery, demonstrated on 15,881 users across 239,618 papers.