How do you put a number on a Magic position?

Evidence: 7 sets, 5,109,204 games, 31.4M card episodes
Purpose: Give an agent a way to tell whether a play improved its position

The question

An agent that plays Magic needs to know whether what it just did helped. That means a number over board states. The moment you try to define that number honestly, the game stops cooperating.

You cannot read a card's value off the card

The convenient answer would be that a card's worth is a function of its printed attributes — rarity, mana cost, type. Then you could score any card ever printed, including ones released tomorrow.

Differences between cards are 98.8% real signal rather than noise. Rarity, cost and type explain 3% of it. Guessing that every card is exactly average beats using them. Two Final Fantasy rares, both three mana, both colourless, identical on every attribute printed on the card, win at 41.4% and 63.6%.

So card value has to be measured from outcomes rather than derived. Which immediately creates a worse problem.

Every measurement is contaminated by the deck

Measure a card's win rate and you are also measuring the decks it tends to appear in. Good cards live in good decks. The effect is large and almost perfectly uniform: every card gains roughly 9.5 points of win rate when the cards around it are strong. Rarity's own signal is so swamped by this that rarities come out in the wrong order — mythics beat commons beat rares beat uncommons.

The usual response is to try to remove the confound. The better response is to notice that the agent never asks the contaminated question. It never wants to know whether a card is good in the abstract. It wants to know which of the four cards in this hand, right now to play — and all four share a deck, a draw, and a game. The deck effect is a common term across every option being compared, so it cancels.

That resolves the problem but constrains the answer: these numbers are usable ordinally, within one hand at one moment, and never as absolute magnitudes compared across decks.

What you condition on is what you would teach

You still have to decide what event a card's value is measured against, and this turns out to be a choice about behaviour rather than a technical detail.

Measured when the card is…	What it actually captures
In your hand	Rises steadily with turn number. Holding cards late correlates with winning — because the game went long, which usually means you were ahead.
On the battlefield	Survivorship. You only observe the card in games where it lived long enough to be seen, so it is credited for the winning positions that let it survive.
Cast	The act of playing it. Conditions on having done something, so both confounds drop out.

An agent optimising against the first measurement learns to hoard its hand and feel good about surviving to turn twelve. The measurement does not merely describe behaviour; it selects it. Picking the conditioning event is the closest thing here to writing down what you want the agent to want.

Some of the game cannot be scored from a position at all

The most useful result was a hard limit. Grading a value function on whether it recovers the real worth of a known-good play:

Play	Simple model	With 16 more features
Removal spell	29%	120%
Attack	~31%	~31%

Sixteen carefully chosen additions moved attacking by nothing at all, while removal went from barely credited to fully credited.

The asymmetry is not a modelling failure. A removal spell's entire value is written into the board you can see immediately afterwards — a creature is gone. An attack's value mostly is not. It sits in damage that matters three turns from now, in the blocks the opponent must now consider, in a race that has not resolved. Magic divides into effects that a snapshot can price and effects that only a sequence can price, and no amount of additional snapshot detail moves an effect from the second group to the first.

One coefficient makes the same point in miniature. All else equal, a board is worth measurably less at the end of your turn than at the end of your opponent's — you are tapped out and it is about to be their move. Obvious once said, and invisible to every earlier model here, because none of them recorded whose turn the snapshot came from.

Where that leaves the division of labour

The measurement owns statics: what a frozen position is worth. The language model owns dynamics — rules, sequencing, what happens over the next three turns — because that is the part it reads natively and the part the numbers structurally cannot reach. And a value is consumed as a difference between two positions at a decision point, never as a level, because the levels are calibrated against a population the agent does not stand in.

Which is also why the number is deliberately not shown to the agent by default. Hand a language model a score and ask it to act on the score, and it becomes an executor for the statistics — at which point the statistics may as well play alone. It is available as an experiment, switched off, so that if it ever gets switched on the effect can be measured rather than assumed.

← Writing