Anneswa Ghosh
Author directory2026
Similar Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents
Sriram Selvam | Anneswa Ghosh
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Sriram Selvam | Anneswa Ghosh
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Search APIs expose ranked snippets, URLs, and metadata on which agents decide whether to answer, search again, or fetch pages. We evaluate these interfaces as decision surfaces on a fixed sample of 100 questions from the 254-question SEALQA-HARD subset, using one frozen GPT-5.4 agent, a fixed orchestration harness, and a shared page-fetch backend across Brave, Tavily, and Firecrawl. A Kimi-K2.6 oracle labels visible URL-level evidence;a separate answer audit yields 25, 25, and 26 correct answers out of 100. These counts indicate similar observed accuracy but do not establish equivalence. Under the tested configurations, Brave exposes more pre-fetch support alongside a larger snippet surface; Tavily has a larger rank-1 share among trajectory-pooled supporting observations; and Firecrawl is associated with broader exploration. First-search query-level metrics distinguish support availability from ranking, while contradiction exposure complements contradiction-to-gold ratios. A retrospective oracle union covers 44/100 questions, versus 26/100 for the best individual provider: 18 percentage points of headroom. The observed evidence and action differences motivate evaluating search APIs jointly with agent policy and retrieval budget.
Eval-Pair Matrix: Answer-Paired Meta-Evaluation of LLM Judges for Grounded RAG
Sriram Selvam | Anneswa Ghosh
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Sriram Selvam | Anneswa Ghosh
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
LLM-as-a-judge evaluation is widely used for retrieval-augmented generation (RAG), but reusing a model as generator and judge complicates identifying self-leniency. We introduce Eval-Pair Matrix, a controlled meta-evaluation protocol for source-grounded RAG. Starting from GaRAGe, we induce one hidden answer-causal contradiction per record, generate answers from perturbed passages with GPT, Grok, and Gemini, and use the same models as blind judges against the original passages. The experiment contains 300 core records, 897 labeled outputs, and 2,683 verdicts in a crossed 3×3 matrix; the primary analysis uses 275 validated records. We estimate same-model effects by pairing judges on the exact same candidate answer. Diagonal and off-diagonal F1 are similar, and aggregate paired recall is near zero for the three tested endpoints (−0.5 pp; 95% cluster-bootstrap CI [−2.7,+1.7]). Matching judges flag answers that avoided the induced claim less often (−4.3 pp). A targeted human evaluation classifies reviewed apparent false positives as alternate source-error detections, adoption-label mistakes, or unclear cases; none were adjudicated as genuine false alarms. We recommend full matrices, answer-paired effects, behavior strata, and label-task alignment.