Single-player Is Not a Tag: Three Bugs in Our Analyzer

On 2026-07-26 we found three defects in the code that scores Steam store pages for this site. The clearest one: the analyzer told 44 pages that Singleplayer and Single-player were a redundant tag pair. One of those is a real community tag. The other is a Steam feature declaration that was never in the tag cloud at all. It was comparing a tag against a feature and calling it a duplicate.

Two of the three are outright bugs and are fixed. The third is a limit we knew about and had not thought through. All three changed numbers we published, so this post is the disclosure and the postmortem in one.

Not from tests, and not from monitoring. We found them by opening live Steam pages next to our own analyzer output while writing store page teardowns, and noticing that the advice did not describe the page.

The Stardew Valley report raised a warning titled “Duplicate and redundant tag entries waste valuable tag slots” and quoted, as evidence, a list pairing Singleplayer with Single-player, Multiplayer with Multi-player, and five variations on co-op. Open the store page and those pairs do not exist. The community tag cloud holds 20 entries and no duplicates. Pull the raw Steam appdetails payload we stored with the analysis and Single-player, Multi-player, Co-op, Online Co-op, LAN Co-op, Shared/Split Screen Co-op and Shared/Split Screen all come back under categories. They are feature declarations, set in Steamworks, printed in the Features column on the right of the store page, and never editable as tags.

The same check on Balatro and Hollow Knight: Silksong gave the same answer. Touch Only Option, Mouse Only Option and Silksong’s Single-player are all in categories, none of them in the tag cloud.

The model was not hallucinating. It read its input correctly and reasoned correctly from it. The defect was in the code that assembles the input, which is the part nobody looks at when an AI tool says something wrong. That is the transferable part of this: when an LLM tool gives bad advice, check what you handed it before you blame the model. A prompt that quietly asserts something false will produce confident, well-written, wrong output, and it will do it consistently enough to look like a feature.

It also explains why our own evaluation would never have caught this. Every stored analysis was produced from the same corrupted input, so the data agreed with itself. The only check that worked was comparing one page’s output to the page.

fetch.ts merged three different things into a single tags array with no marker for which was which:

1. 01User tags, the community tag cloud, editable by the developer.
2. 02Genres, Steam’s official genre list.
3. 03Categories, which are features: Single-player, LAN Co-op, Steam Achievements, Mouse Only Option, Family Sharing.

analyze.ts then flattened the array into one comma-separated string for the prompt, so the model saw Roguelike, Action, Single-player, Steam Achievements, Mouse Only Option with no way to tell which entries were tags.

Across the 373-page research corpus, 146 of 549 tag issues (26.6%) name a string that can only be a Steam feature, spread across 129 pages (34.6%). 136 of those issues, on 121 pages, are of the remove-it or it-is-diluting-you type, and 22 of the 55 critical tag issues are among them. Count names that exist both as a feature and as a community tag, such as Multiplayer and Co-op, and it rises to 171 issues (31.1%) on 152 pages (40.8%). The lower pair is the one to quote.

Balatro’s report said its accessibility tags, naming Touch Only Option and Mouse Only Option, were diluting its genre signal. Silksong’s said to remove a duplicate Single-player. A developer who followed that advice would go into Steamworks and try to delete their local co-op declaration and their accessibility options. Those fields drive store filtering and accessibility surfacing, so acting on the advice would have cost the game discoverability and made its accessibility support invisible to the players who filter for it. That is worse than useless advice. No developer is known to have acted on it, and we are not claiming harm that happened. The advice was wrong, and following it would have hurt.

The same merge corrupted the score. computeMetrics() set the tag floor from the length of that padded array:

Across 555 stored analyses the median game carried 8 features and 3 genres, so the count was padded by a median of 11 entries. A typical page was collecting a tag floor it had not earned, worth roughly 11 to 12 points of tag subscore. Only pages genuinely using all 20 slots were unaffected.

There was a second-order effect. The prompt reports tag slot usage from the same count, so a game with 8 real tags was shown to the model as “19/20 slots used” and was never told to add tags. It now reads “8/20”, which is why the fixed analyzer produces more add-a-tag recommendations on thin pages. Our tag optimizer and the tag guide work from the community cloud only.

Steam’s appdetails endpoint returns detailed_description as compiled HTML. The developer types BBCode into Steamworks, Steam compiles it, and the API hands out the compiled result.

Our rubric awarded 11 of the 30 deterministic description points for structure: 7 for formatting, 4 for embedded media. Both tests were regexes looking for the BBCode: [h1], [list], [b] for formatting, [img] and [previewyoutube] for media. They cannot match compiled HTML, so they never matched anything.

Measured over all 555 stored analyses:

The description floor is 8 points for a short description of 100 to 300 characters, 8 for more than 150 words, 7 for formatting, 4 for media, 3 for a trailer. With two of those unreachable, the ceiling was 19 out of 30 and 473 of the 555 analyses (85.2%) sat at exactly 19. The average page lost 9.9 of the 11 points it could not reach.

The scoring loss was not the worst part. The floor block in the prompt also states what it found, so on a page full of section headers and animated GIFs the model was told formatting: no, embedded media: no and then asked to judge the description. It believed us. In the research corpus, the wall-of-text description theme sits on 89 pages before you filter for that contamination and 26 after.

FULL_SCREENSHOT_LIMIT = 10 in analyze.ts. The prompt sends the first 10 screenshots and stops, which was a deliberate cost and latency decision and is still a defensible one. It is listed here because we had not carried the consequence through to what we were willing to claim.

188 of the 555 stored analyses (33.9%) carry more than 10 screenshots. In the 373-page corpus it is 121 pages (32.4%), the largest gallery holds 48, and 681 individual screenshots on those pages were never sent to the model. The deterministic screenshot floor also saturates at 8 images, so both halves of the screenshot score stop moving somewhere around the eighth picture.

That matters because our store page audit reported that screenshot subscores plateau above 12 images with a median difference of exactly 0 at p = 1.00. Read literally, that is a claim about Steam pages. It is not. It is a claim about a rubric that stops looking at the tenth image. The post now says so, and restricts the surviving finding to the step between 7 and 8 screenshots, which happens inside the visible range.

Everything above was fixed in code on 2026-07-26. Nothing was re-scored.

The two corrections push in opposite directions. A typical released game loses about 11 to 12 points of tag subscore and gains up to 11 points of description subscore, and since the overall score weights those at roughly 0.20 and 0.25, the headline number for many pages barely moves. The diagnosis moves completely. The same page that used to be told its description was its weak section and its tags were fine now gets the opposite reading, which is the reading we believe.

Three consequences worth stating plainly:

• —Scores from before 2026-07-26 are not comparable to scores after it. Same page, same rubric name, different instrument.
• —The leaderboard currently mixes both regimes. Entries analyzed before the fix carry inflated tag subscores and depressed description subscores. We have not re-run it.
• —We chose not to re-score the 373-page research corpus. Re-running it costs real money and we decided disclosure was the better spend. That is a budget decision, not a methodological one, and you should weigh the published figures accordingly.

The Steam store page audit had a headline finding that description is the weakest part of the average store page, resting on description carrying the lowest median subscore. That claim was an artifact of Bug 2 and has been withdrawn from the post rather than quietly edited out. On the corrected floor the description median moves from 64 to about 75 while tags moves the other way, which probably inverts the ranking. The post now leads on the largest issue theme in the data, the weak or buried opening hook, which does not depend on either broken floor.

The teardown posts state their analysis dates, which is the honest mitigation, but their figures are pre-fix and will not reproduce if those pages are analyzed today. The Stardew Valley teardown already documented the tag conflation and warned readers not to delete their feature declarations, so it stands as written.

• —TagData carries a source of user, genre or category, set at all three merge points in fetch.ts. The field is optional and a missing source is treated as user, so stored analyses do not regress.
• —The prompt emits three labelled lines instead of one flat list, with an explicit instruction never to recommend removing a Steam feature.
• —computeMetrics counts only source === 'user' for the tag floor, and the slot count follows.
• —The response parser only ever sees community tag names, so a suggested removal that names a feature is dropped as a hallucination. That is the path that produced the Stardew Valley recommendation.
• —The formatting and media checks accept HTML and BBCode. Regression tests cover both, including one that asserts the old 19-of-30 ceiling on genuinely unformatted text so the bug cannot come back silently.

Four rules we are keeping, and would suggest to anyone shipping a deterministic scorer in front of a model.

Test third-party payloads against a real captured response, not a fixture you wrote from the documentation. Both regexes passed unit tests, because we wrote the tests and the BBCode fixtures from the same wrong assumption.

Alert on prevalence. A check that fires on 0.0% or 100% of production rows is almost always broken rather than informative, and a 0.0% match rate across 555 rows should have raised something long before a human noticed.

Keep provenance through every transformation. Merging three lists of strings into one array destroyed the single fact the model needed, and no amount of prompt engineering downstream could put it back.

Read the tool’s output against reality on a schedule, not only when a blog post makes you look closely. If you want to try that on your own page, the analyzer runs the fixed rubric and shows you the evidence behind every issue it raises, which is the part you should be checking.