01 / The task
A challenge set is scored by everyone else's metric
In this subtask the participant does not submit a metric. The participant submits the segments that every submitted metric is then asked to score, and receives the scores back.
The WMT26 Automated Translation Quality Evaluation task collects metric systems that assign a quality score to a machine translation. Its 4th subtask invites challenge sets: collections of source sentences, hypotheses and references built to test 1 specific weakness. The organizers score every submitted segment with every submitted metric system and return the scores to the builder. There is no leaderboard for a challenge set. The deliverable is the analysis, published as a system paper alongside the organizers' findings.
3 constraints shape the design. A cap of 1 million tokens, counted over source, hypothesis and reference together. 7 published fields per row, and nothing else. Scoring at the segment level, 1 score per row, higher is better. The format carries 1 hypothesis per row, so a correct and a corrupted translation of the same source have to travel as separate rows, and the pairing has to be rebuilt after the scores come back.
The weakness under test
A reads only the source and the hypothesis. A reference-based metric can notice that translated content is missing because the reference still contains it. A reference-free metric sees a hypothesis that is shorter, fluent and internally consistent, with nothing in it to flag. The registered concern was that such a metric penalizes omission less than it penalizes a comparable amount of added content, and that a best-of-n reranker built on it will therefore sometimes prefer the incomplete translation.
3 hypotheses were registered.
- HB, the headline. Reference-free QE metrics under-penalize meaning omission relative to fluent addition, while reference-based metrics penalize the 2 edit directions roughly symmetrically.
- HA, secondary. Reference-free QE scores do not decrease, or even increase, as fluent, fact-free, source-unsupported material is appended to a realistic base translation.
- HC, a positive control. The reference-based neural frontier catches a single load-bearing numeric contradiction while surface and reference-free metrics diverge. This proves the harness can detect a break at all.
The shipped set, 2,886 segments interactive, pick an arm
- Segments
- 2,886
- Directions
- 10
- Groups
- 1,137
- Rows per group
- 6, 3 or 2
How this page is organised. Sections 02 and 03 describe what was built and what was frozen before any score existed. Sections 04 to 07 report what came back. Section 08 lists what was not done. Every number is quoted from the camera-ready paper at the precision it was recorded at.
02 / The challenge set
3 arms, with 1 matched triple at the centre
Every headline item holds the source fixed and changes the hypothesis by exactly 1 information unit, in 1 of 2 directions.
Arm B, the headline: matched omission and addition triples
308 triples on 6 directions, 924 segments. Each triple pairs a base translation with an omission member that removes 1 and an addition member that appends 1 fact-free, source-unsupported unit. The 3 members travel as separate rows, paired by opaque group identifiers, which is the design the organizers endorsed. The triple supports both primary endpoints of section 03: the signed interaction and the preference-flip rate.
The 2 directions are matched on information units, not on edit size. Additions change a median of 2.0x as many characters as omissions remove, counted in whitespace-stripped characters (1.4x in the LLM-generated stratum, 2.0x and 2.5x in the 2 deterministic strata), and 79% of triples add more characters than they remove. A length-controlled refit therefore accompanies the primary interaction in section 04.
1 source, 1 triple, 3 separate rows static diagram
Arm A, secondary: a verbosity dose ladder
76 series on en-de and en-ru, 456 segments. Each series appends 1 to 5 fluent, fact-free filler units to a base translation, giving 6 segments per series including the dose-0 base. Fillers are screened to contain no digits, dates or named entities, and to be entailment-neutral against both the source and the base, so the ladder isolates pure unsupported length. At the maximum dose the hypothesis is a median of 6.4x the base length.
Arm C, the positive control: numeric minimal pairs
753 pairs on all 10 directions, 1,506 segments. Each pair contrasts a correct hypothesis with a copy whose single load-bearing numeral is corrupted: digit substitution in 440 pairs, transposition in 213, a magnitude shift in 100. Numeral selection follows a scoring rule frozen in the registration, with units and separators weighted up and years penalized, so the corrupted numeral is never a researcher degree of freedom. The arm exists to prove the harness has teeth. If the reference-based frontier missed a flat numeric contradiction, the harness would be under suspicion rather than the metrics.
Validity gates
Every generated item passes deterministic gates before it can ship, and items that fail are dropped rather than repaired. The omission gate requires the base hypothesis to the omitted-member hypothesis while the dropped unit stays un-entailed by the remaining text, which blocks omissions that merely rephrase. The addition gate requires the appended unit to carry no digit, date or named entity, and to be entailment-neutral against both source and base: neutral at or above 0.70, entailment and contradiction at or below 0.15 each. Subtle LLM-generated triples obey symmetric length caps of 25% of the base's characters in either direction; the realized worst cases are 22.9% and 24.4%. No gate reads any metric score.
Gate survival is low by design. In the LLM-generation runs, 1,042 raw candidates were drawn, 861 parsed into complete triples, and the gates admitted 89 of those: 10.3% overall, between 3.2% on en-ja and 20.0% on en-ru.
Gate survival in the LLM-generation runs animated
3 provenance strata
Arm B ships in 3 strata, defined only from metadata written at generation time. S1, 89 triples: subtle perturbations generated by Llama-3.1-8B on real WMT24 system outputs under the length caps, where the omission typically drops a single detail such as a street number. S2, 100 triples: deterministic whole-unit edits on real WMT24 system-output bases. S3, 119 triples: deterministic edits on reference-verbatim bases. Real-MT bases come from the median-ranked WMT24 system per direction. The registered secondary prediction is a gradient for the reference-free family: a deeper asymmetry and a higher flip rate in S1 than in S2 and S3 pooled, with no prediction between S2 and S3, and no stratified test in the confirmatory family.
Blind submission
The uploaded rows carry exactly the 7 published fields under a . All internal metadata was stripped, identifiers are opaque hashes with no label, phenomenon or dose token, and row order was de-correlated so that the base member sits at no fixed position: base-first fell from 100% to 44.0%, against a 43.3% chance level. A private keymap, never uploaded, restores the pairing when scores return. The structure that a contrastive set cannot remove, such as pair members sharing a source, is stated in section 08.
03 / Pre-registration
Everything that could be frozen was frozen before the first score
The hypotheses, endpoints, equivalence margin, multiplicity rule, strata and orientation manifest were committed to a timestamped git history before the upload. Whatever the scores said, the reported outcome would be the registered one.
What was committed, and when static diagram
The 2 primary endpoints
N1, the signed interaction. For each perturbed Arm B member, the signed drop is the base score minus the member score, so a positive value means the metric penalized the edit. Within each metric the drops are replaced by their , which makes the response invariant to any monotone rescaling of a metric. The registered 3-way design reduces to an identifiable model, because every shipped Arm B edit carries exactly 1 information unit:
The reported quantity is β, the coefficient on the product. Because the response is a percentile, β has a plain reading: the addition-minus-omission gap in mean percentile drop for reference-reading metrics, minus the same gap for reference-free metrics, in percentile points. The registered prediction is β < 0. The point estimate is ordinary least squares on 10,220 observations, from 6 reference-free metrics with 3,444 observations and 11 reference-reading metrics with 6,776. Every interval and p-value comes from a over the 268 distinct source sentences, 2,000 replicates. The 12 returned systems whose reference usage no public source documents sit out of this 1 design, since R is undefined for them.
N2, the preference flip. Pflip is the share of matched {base, omission} pairs in which a reference-free metric scores the omission at or above its own base. That is the contest a best-of-n adjudicates. The registered prediction was a high rate for reference-free QE and a rate near 0 for reference-based and LLM-judge metrics.
The anchor
HB is a contrast, so it needs the reference-based family to be symmetric in fact. For metric m, bom,m is the mean raw oriented drop over its omission members and badd,m the same over its addition members, both in the metric's own units. The scale-free relative asymmetry is
The registered is ±0.30, tested by 2 one-sided tests at α = 0.05, with a : the family's pooled raw drop, in its native units, must be positive for the family to anchor anything. The refutation leg fires when the reference-based family's 95% CI lies entirely below −0.30. In that case the claim that the missing reference is what produces the asymmetry is refuted, whatever the reference-free family does.
Multiplicity, orientation, anti-circularity
1 test per metric on Arm A (28 metrics) and on Arm C (31), plus the single pooled Arm B β, make 60 confirmatory tests under control at q = 0.05, applied once, with the family size fixed by the returned roster before any score value is inspected. The harness fails closed on any returned metric name that matches no manifest pattern. Everything outside that family, the strata and the sensitivity refits included, is secondary or exploratory and is never reported as a break.
A local CometKiwi pre-filter, a stand-in for the audit-target family, labeled every row with a break prediction but never gated selection, and there was no selection step for it to influence. The packager ran with no per-direction cap, and the set came in at 231,582 tokens against the 1M budget, so every item that passed the validity gates shipped, apart from 1 deterministic triple dropped by a metric-free rule that keeps the LLM triple when both were built on the same source. The label split of the shipped set: Arm B, 4 of 308 triples predicted to break; Arm A, 0 of 76 series; Arm C, 89 of 753 pairs. HB is reported as confirmed only if the effect also holds in the 304 Arm B triples predicted not to break and on the 189 triples whose bases are real WMT24 outputs.
| Quantity | Registered prediction | Decision rule | Outcome |
|---|---|---|---|
| N1 β, Arm B, primary | β < 0 | OLS point estimate; source-clustered bootstrap 95% CI; 1 test in the confirmatory family | −0.069 [−0.081, −0.057], confirmed |
| N2 Pflip, Arm B, primary | high for reference-free QE; near 0 for reference-based and LLM judges | clustered bootstrap CI; descriptive | 0.013 to 0.195 for reference-free QE; not near 0 elsewhere |
| Anchor A, reference-based | inside [−0.30, 0.30]; reference-free A < 0 | established if the 90% CI sits inside the margin and the teeth precondition holds; refutes the mechanism if the 95% CI lies below −0.30 | −0.42 [−0.49, −0.35]: refutation leg fires; reference-free −0.51 |
| Strata gradient, secondary | A(S1) < A(S2+S3); Pflip(S1) > Pflip(S2+S3) | ordering of point estimates, CIs reported; never in the confirmatory family | holds as an ordering |
| Anti-circularity strata | effect persists on the 304 triples predicted not to break and on the 189 real-MT triples | confirmation of HB requires both | both hold |
| Arm A ρ | ρ ≥ 0 for reference-free QE (HA) | one-sided clustered bootstrap test of H0: ρ ≥ 0, in the confirmatory family | ρ from −0.37 to −1.00 for all 28 metrics: HA refuted |
| Arm C τb | reference-based frontier catches, reference-free trails (HC) | one-sided test of H0: τb ≤ 0, in the confirmatory family | holds; 2 MQM-LLM variants invert |
| Multiplicity | 28 Arm A tests, 31 Arm C tests, 1 Arm B test | Benjamini-Hochberg at q = 0.05, applied once | 60 tests; 58 reject at q < 0.001 |
The frozen prediction table
The registration also fixed a verdict for every arm-by-family cell before any score existed. 4 of the 16 cells were contradicted by the data, and the paper marks them rather than reframing them. Select a cell to read the registered verdict and what the return showed.
Registered verdicts, 4 arms by 4 families interactive, select a cell
Arm B omission · reference-free QE · registered: break-hard
Confirmed in direction, refuted as a mechanism
The family under-penalizes omission (A = −0.51) and the interaction is significant, so the predicted break appears. The registration attributed it to the missing reference, and the anchor cell for the reference-based family refuted that attribution.
04 / Results, Arm B
The interaction is significant, and the anchor refutes the mechanism
The organizers returned segment scores from 31 systems on 6 August 2026. 58 of the 60 confirmatory tests reject at q < 0.001. The 2 exceptions are on Arm C.
28 systems cover all 2,886 segments. cuni-v14-regression covers 3 directions, and the 2 Lexicala QE ensembles cover 3 of the 4 directions that carry Arm C only. The roster is a superset: some teams submitted more than the permitted 2 systems, and the organizers had not designated main submissions by the camera-ready deadline, so the family of 60 tests stands on the returned roster. 12 systems carry opaque names whose reference usage no public source documents. They enter every per-metric endpoint but sit out the reference-availability axis. All ingestion checks passed: every returned identifier matched the private keymap, and per-system counts matched the organizers' coverage table exactly.
N1: β = −0.069
The signed interaction is significant in the predicted direction: β = −0.069, source-clustered bootstrap 95% CI [−0.081, −0.057], q < 0.001 in the confirmatory family. On its own scale the coefficient reads as follows. For reference-free metrics, an addition sits on average at the 56.1st percentile of that metric's drops and an omission at the 43.9th, a gap of 12.2 points. For reference-reading metrics the gap is 5.2 points, 52.6 against 47.4. β is the difference of the 2 gaps before rounding: 5.24 − 12.17 = −6.9 points. Per family, the gap is 18.3 points for reference-free QE, 13.7 for reference-based neural, 3.4 for LLM judges, and −8.6 for surface metrics, which penalize omission more.
The sensitivity refits leave the interaction intact. Under the registered z-score transform it is −0.24 standard deviations [−0.29, −0.20]. With edit magnitude as a covariate it is β = −0.073 [−0.084, −0.061], so the 2.0x character imbalance of section 02 does not explain it. The registration's mixed-effects form, refitted with source nested in language pair, returns −0.069 [−0.086, −0.052], and adding random slopes gives −0.070 [−0.086, −0.054]. Nothing turns on the estimator.
Reading β on its own scale scroll driven
Figure 6, scroll driven. Every value shown is a cell mean from the fitted model. The stages are staged for reading, not derived on the fly.
Stage 1 / reference-free metrics
Within each metric, every drop is replaced by its percentile among that metric's drops, so 50 is the median penalty. For the 6 reference-free metrics, additions land on average at the 56.1st percentile and omissions at the 43.9th.
Stage 2 / the reference-free gap
The distance between the 2 marks is 12.2 percentile points. An omission is, on average, a noticeably smaller penalty than an addition for a metric that cannot see the reference.
Stage 3 / reference-reading metrics
For the 11 metrics that read a reference, additions sit at 52.6 and omissions at 47.4: a gap of 5.2 points, in the same direction, less than half the size.
Stage 4 / the coefficient
β is the reference-reading gap minus the reference-free gap: 5.24 − 12.17 = −6.9 points. The gap is 6.9 percentile points wider without a reference, and the bootstrap interval, [−8.1, −5.7] in the same units, excludes 0.
Stage 5 / per family
Split the reference-reading pool by family and the gap is 18.3 points for reference-free QE, 13.7 for reference-based neural metrics, 3.4 for LLM judges, and −8.6 for surface metrics, which penalize omission more than addition.
The anchor
The family asymmetries carry the larger result. Reference-free QE: A = −0.510 [−0.573, −0.442], with per-metric values of −1.28 for xCOMET-XL-QE, −0.59 for xCOMET-XXL-QE, −0.22 for cuni-v14-regression and +0.05 for COMETKiwi22. Reference-based neural: A = −0.422 [−0.494, −0.346]. LLM judges: −0.135 [−0.207, −0.067]. The 12 unidentified systems: −0.323 [−0.395, −0.250]. Surface metrics alone over-penalize omission, at +0.362 [+0.297, +0.435], since n-gram precision plus a brevity penalty charges missing content twice.
The teeth precondition holds. The pooled reference-based drop, in the family's native units and dominated by the 2 xCOMET scales from 0 to 100, has 95% CI [+11.5, +13.4], and every member's mean drop is positive in both edit directions. But the reference-based 90% and 95% intervals lie entirely below the −0.30 margin, so the registered refutation leg fires. The claim that the missing reference is what produces the asymmetry is refuted: the asymmetry appears in every learned family. Reference access reduces it, which is the significant interaction, and surface overlap inverts it.
Relative asymmetry A by family, with the registered margin interactive, show the per-metric values
What this page does not take up. The finding that the asymmetry belongs to learned metrics as a class, rather than to reference-freeness, is reported here as the pre-registered outcome and nothing more. Why learned metrics behave this way is a separate question. It is outside the scope of the paper, and it is not discussed on this page.
05 / Results, N2
The flip rate reaches 1 contest in 5 for a current baseline
Pflip is the deployment-relevant number: the rate at which a reference-free metric scores the omission at or above its own base, which is exactly the contest a best-of-n reranker adjudicates.
Over 308 matched pairs, or 182 for the partial-coverage CUNI system: xCOMET-XL-QE 0.195 [0.154, 0.235], xCOMET-XXL-QE 0.114 [0.079, 0.152], cuni-v14-regression 0.093 [0.055, 0.137], COMETKiwi22 0.013 [0.003, 0.027]. Part of the first 3 rates is exact ties, mostly at the scale ceiling. Strictly above the base, the rates are 0.143, 0.071, 0.044 and 0.013. A reranker guided by xCOMET-XL-QE rates the translation that silently dropped content at least as high as its base in 1 matched contest in 5, and strictly higher in 1 in 7. COMETKiwi22 almost never flips and is also the family's most symmetric member. The failure is graded inside the family, and the deep asymmetry is carried by the xCOMET QE baselines.
The registration expected the rate to be near 0 for reference-based and LLM-judge metrics. Computed the same way for every metric, as an exploratory reading, it is not. The family means are 0.088 for reference-based neural, where xCOMET-XL-ref reaches 0.208; 0.130 for LLM judges, where gemba-poly reaches 0.325; 0.074 for surface metrics; and 0.218 for the unidentified systems, against 0.104 for reference-free QE. The flip, like the asymmetry, is shared by learned metrics as a class.
308 matched contests, 1 metric at a time interactive, pick a metric and a rule
- Matched contests
- 308
- Flipped
- 60
- Rate
- 0.195
- 95% CI
- [0.154, 0.235]
| Metric | Pairs | Omission drop | Addition drop | Am | Gap, pts | Pflip | Strictly above |
|---|---|---|---|---|---|---|---|
| xCOMET-XL-QE | 308 | +10.2 | +46.1 | −1.28 | +38.1 | 0.195 | 0.143 |
| xCOMET-XXL-QE | 308 | +28.9 | +52.9 | −0.59 | +22.8 | 0.114 | 0.071 |
| cuni-v14-regression | 182 | +15.9 | +19.9 | −0.22 | +7.7 | 0.093 | 0.044 |
| COMETKiwi22 | 308 | +0.185 | +0.177 | +0.05 | +0.2 | 0.013 | 0.013 |
06 / Results, Arms A and C
The length hypothesis is refuted, and the control holds
Every returned metric penalizes fluent, fact-free filler. The reference-based frontier catches a single corrupted numeral. 2 participant systems score the corrupted member above the correct one.
Arm A: no returned metric rewards pure length
HA is refuted outright, and the registered negative is reported as such. Every returned metric penalizes filler. The within-source Spearman ρ, the mean over series of the per-series correlation between dose and oriented score, runs from −0.37 for fluency2-gemini35-esa to −1.00 for BLEU, chrF, spBLEU and YiSi-1, all at q < 0.001 against H0: ρ ≥ 0. COMETKiwi22 reaches ρ = −0.86 [−0.89, −0.83]; xCOMET-XL-QE and xCOMET-XXL-QE reach −0.65 and −0.70. The length-gaming attack this arm operationalizes does not work on the modern roster. Reference-based curves are exploratory context on this arm, since reference-verbatim bases inflate the dose-0 intercept, and the conclusion is scoped to en-de and en-ru as registered.
Within-source Spearman ρ against filler dose, 28 metrics animated
Arm C: the harness has teeth, and 2 systems invert the control
The reference-based neural frontier catches the single corrupted numeral: COMET22, BERTScore and YiSi-1 at τb = +1.00, xCOMET-XXL-ref at +0.95. Surface metrics detect the token change deterministically, at +1.00. The reference-free baselines trail visibly: COMETKiwi22 +0.77, xCOMET-XL-QE +0.77, xCOMET-XXL-QE +0.79. That is the registered pattern. 1 cell of the frozen prediction table was wrong here: surface metrics were predicted to break on this arm and instead held perfectly. Contrastive accuracy counts metric ties as half credit; tie rates stay at or below 0.19 and are highest for span-derived scorers.
2 participant systems invert the control. MQM-LLM, at τb = −0.21 and contrastive accuracy 0.397, and MQM-LLM Confidence Aware, at −0.12 and 0.439, score the corrupted member above the correct one more often than not. They are the only 2 confirmatory non-rejections on the board. The orientation cross-check flags both, and their strongly negative Arm A slopes confirm that the declared orientation is right, so this is a system-level failure on load-bearing numerals rather than a sign error in the manifest.
Numeric-contradiction contrastive accuracy, 31 systems animated
07 / Where the effect lives
Subtle edits on real system output cut deepest
The pre-declared gradient holds as an ordering, and both anti-circularity strata hold.
In S1, the 89 triples with subtle LLM perturbations on real system outputs, the reference-free asymmetry deepens to A = −0.604 [−0.747, −0.462] and mean Pflip rises to 0.154 [0.107, 0.203], against −0.474 [−0.590, −0.359] and 0.104 [0.074, 0.135] in S2 and −0.538 [−0.634, −0.449] and 0.058 [0.035, 0.085] in S3. Pooled, S2 and S3 give −0.510 and 0.079. The interaction strengthens to β = −0.117 [−0.151, −0.082] inside S1. The registered prediction is an ordering, and it holds on both axes. The A intervals overlap, so the gradient is a direction rather than a separated effect.
Both anti-circularity confirmation strata hold as well. In the 304 triples the local CometKiwi pre-filter predicted would not break: β = −0.069 [−0.081, −0.057], reference-free A = −0.503, mean Pflip = 0.094. On the 189 triples with real WMT24 system-output bases: β = −0.078 [−0.094, −0.061], A = −0.487, mean Pflip = 0.129. The measured deficit is not an artifact of pre-filter-guided construction, and it is present on real MT output.
Reference-free QE by stratum animated
08 / Limitations
What was not done, and what the set cannot certify
Item validity rests on machine gates
A blind human validation study was built before submission: a 270-item single-annotation sheet plus a 60-item double-annotated overlap, sampled with a guaranteed minimum of S1 items and sealed so annotators see only an identifier, the source and the hypothesis. The protocol targets per-direction item precision of at least 95% and reports Cohen's κ and Krippendorff's α on the overlap. The annotation had not been carried out by the camera-ready deadline, so the paper reports the protocol and no human precision number. The machine gates certify entailment relations and the absence of entities, digits and dates. They cannot certify that a dropped unit is one a human reader would miss, or that an appended unit reads as natural, and every per-item claim inherits that limit.
- Coverage and stratum size
- Arm B covers 6 of the 10 directions. S1 is small on en-ja, 4 triples, and en-ru, 6. Any stratum below 20 matched pairs gets point estimates only, and per-direction readings are exploratory. Numeric pairs use Latin and Cyrillic digit contexts only.
- Edit size and dose
- Additions change a median of 2.0x the characters omissions remove. If a metric responds to raw edit magnitude, this pushes A in the predicted direction, which is why the magnitude refit sits beside the primary interaction. Every Arm B edit carries 1 unit, so no dose-interaction claim is made.
- Circularity disclosed, not eliminated
- CometKiwi labeled, never gated, the Arm A and B items. Members of a pair share a source, group size correlates with arm, and control members are reference-verbatim. An honest metric scores each row independently; a deliberately adversarial scorer could exploit these properties.
The reviews, and what changed for the camera-ready
2 reviewers, both with recommendation 5 of 5. The 1st asked for the notation to be defined before use, for the specification of the model behind β and its scale, and for every figure to be cited from the body. The 2nd asked for the anti-circularity counts to appear in the plan section, and for a shorter paper.
The revision answered each request, and it also corrected statements in the submitted version that the author's own verification passes found: the gate-survival range, the description of the estimator, the premise of the omission gate, and the record of the post-return commit. Every number in the camera-ready has a script and an output file beside it in the repository.
The submitted version is preserved alongside the camera-ready, so the 2 can be compared line by line.
09 / Coda
What transfers past this task
The measurement is specific to WMT26's roster. The discipline that produced it is not.
- A blind protocol with a private keymap. The rows carry no label, phenomenon or dose token, and the pairing lives in a file that is never uploaded. A metric can only be scored on what it was given.
- A fail-closed orientation manifest. The harness raises on any returned metric name it cannot place, so the family size and every sign are fixed by the roster before a score value is read.
- An equivalence anchor with a teeth precondition. The contrast that the headline hypothesis needed was itself a registered, falsifiable claim, and the registration said in advance what would count as its refutation.
- A positive control arm. Arm C exists so that a null result elsewhere cannot be blamed on a harness that detects nothing.
Together these let a registered mechanism claim be refuted in public while the measurement that refuted it stands. The paper is to appear in the WMT26 proceedings, and the poster will be presented at WMT26, at EMNLP 2026 in Budapest, October 2026.
Silchenko, QEbreak at WMT26: A Pre-Registered Audit of Omission and Addition Asymmetry in Machine Translation Quality Metrics. Proceedings of the 11th Conference on Machine Translation (WMT26), challenge-set submission to the Automated Translation Quality Evaluation task.