Maksim Silchenko
Author directory2026
QEbreak at WMT26: A Pre-Registered Audit of Omission and Addition Asymmetry in Machine Translation Quality Metrics
Maksim Silchenko
Proceedings of the Eleventh Conference on Machine Translation
Maksim Silchenko
Proceedings of the Eleventh Conference on Machine Translation
Reference-free quality estimation (QE) metrics see only the source and the hypothesis, so a translation that silently drops source content leaves nothing for them to flag. QEbreak, a pre-registered contrastive challenge set submitted to the WMT26 Automated Translation Quality Evaluation task (Subtask 4), measures this blind spot: 2,886 segments over 10 language directions, built as matched {base, omission, addition} triples, a fact-free verbosity ladder, and numeric-contradiction pairs as a positive control. Every hypothesis, endpoint, and equivalence margin was frozen in git-timestamped commits before any score existed. Scores from 31 systems confirm the deficit and refute our mechanism. Reference-free QE metrics under-penalize omission (family asymmetry A = −0.51; a current QE baseline scores the omission at or above its own base in 19.5 percent of matched contests, strictly above in 14.3), and the edit-direction by reference-availability interaction is significant (β = −0.069, 95% CI [−0.081, −0.057]). But reference-based neural metrics are asymmetric in the same direction (A = −0.42), outside the pre-registered ±0.30 equivalence margin, so omission under-penalization belongs to learned metrics as a class; reference access softens it but does not remove it. Every returned metric penalizes pure verbosity, and two systems score corrupted numerals above correct ones.
thylinao at ChildSafeAds 2026: A Six-Member Ensemble and What Each Level of Data Access Provided
Maksim Silchenko
Proceedings of the Natural Legal Language Processing Workshop 2026
Maksim Silchenko
Proceedings of the Natural Legal Language Processing Workshop 2026
This paper describes the thylinao system for the ChildSafeAds shared task on monitoring sponsored segments in child-facing YouTube videos. The system is a six-member ensemble: two fine-tuned encoders, a zero-shot prompted Qwen3-32B, a LoRA-tuned Qwen3-14B used for ST2 only, a TF-IDF baseline, and a level-2 stacker for the compliance sub-task, blended at equal weight behind a decision layer whose every parameter is fitted on channel-grouped out-of-fold predictions. The system scores mean macro-F1 0.6537 on the evaluation split; inference over the 503 test instances is dominated by one measured 14-minute GPU pass of the 32B member. The zero-shot member is the most valuable and nearly the cheapest, while removing the fine-tuned DeBERTa changes nothing measurable. An ablation over the task’s four data access levels, run on the zero-shot member, finds the channel name adds nothing and the product-page crawl helps product identification but not, measurably, compliance. Validating these changes also surfaced properties of the evaluation, reported as feedback: the metric’s divisor dominates score variance and is recoverable in closed form from a published baseline row, and the benchmark is channel-disjoint but substantially advertiser-overlapping. The final submitted entry additionally contained three labels set by hand, which the task Terms prohibit; this was disclosed to the organisers and is documented in Section 7. The system-only score is reported throughout and no rank is claimed.
thylinao at BAREC Shared Task 2026: Ensembling at the Noise Floor
Maksim Silchenko
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Maksim Silchenko
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Coverage Is Not Knowledge: Why Closed-Book Knowledge Base Construction Looks Selection Bound
Maksim Silchenko
Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026)
Maksim Silchenko
Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026)
This paper describes the first place system of the AKBC 2026 shared task on predicting complete knowledge base entries from language models, built on a single 31.3B parameter base model in completion mode with no training and no retrieval. It scores 0.7060 on the official test set; the runner up scores 0.6908, and a composite taking the best per relation score of every other ranked team would reach 0.7011.
The largest single improvement I found is not an algorithm. Asking the model to recall facts about an entity before committing to an answer, rather than asking for the answer directly, moved hasArea from 0.8100 to 0.8700. Three earlier configurations, across two frames and two aggregation methods, had all returned exactly 0.8100, which I had read as evidence that the answers were absent from the candidate pool. That reading was incorrect.
The main analytical result is a correction to how this field measures headroom, and it corrects my own earlier conclusion as much as anyone else’s. Oracle coverage analyses report that hasCapacity is selection bound: the recitation frame puts an acceptable answer in the pool for 84.5 percent of validation subjects while the aggregator realises 36.1 percent. That inference does not survive a null control. A pool of one hundred guesses spanning one decade contains about twenty one mutually exclusive 5 percent tolerance windows, so it contains something acceptably close to almost any plausible target whether or not the model knows the answer. Permuting gold values across subjects measures that coincidence rate directly: 0.355 for hasCapacity against 0.031 for hasArea. Chance corrected, the findable headroom beyond the rank one answer falls from 0.443 to 0.203. Three instruments that are not functions of vote frequency then agree that the relation is knowledge bound.
The largest single improvement I found is not an algorithm. Asking the model to recall facts about an entity before committing to an answer, rather than asking for the answer directly, moved hasArea from 0.8100 to 0.8700. Three earlier configurations, across two frames and two aggregation methods, had all returned exactly 0.8100, which I had read as evidence that the answers were absent from the candidate pool. That reading was incorrect.
The main analytical result is a correction to how this field measures headroom, and it corrects my own earlier conclusion as much as anyone else’s. Oracle coverage analyses report that hasCapacity is selection bound: the recitation frame puts an acceptable answer in the pool for 84.5 percent of validation subjects while the aggregator realises 36.1 percent. That inference does not survive a null control. A pool of one hundred guesses spanning one decade contains about twenty one mutually exclusive 5 percent tolerance windows, so it contains something acceptably close to almost any plausible target whether or not the model knows the answer. Permuting gold values across subjects measures that coincidence rate directly: 0.355 for hasCapacity against 0.031 for hasArea. Chance corrected, the findable headroom beyond the rank one answer falls from 0.443 to 0.203. Three instruments that are not functions of vote frequency then agree that the relation is knowledge bound.