Steam Page A/B Testing: How to Test Without Native Tools

Google Play developers can upload two icons, pick a 95% confidence level, and let live traffic crown a winner. Steam developers get nothing of the kind. There is no native way to split traffic between two capsules, two trailers, or two descriptions, and Valve has given no hint that one is coming. The workarounds exist, though, and the difference between a useful test and an expensive guess comes down to method.

Google Play Console’s “store listing experiments” split live store traffic across icon, screenshot, and description variants, with a selectable confidence level of 90%, 95%, 98%, or 99%. That is real experimentation infrastructure, built into the platform. Steam has no equivalent; every visitor sees the same page.

So indie devs improvise, and the improvisations fall into three families:

1. 01Off-platform split tests. Paid social ads randomize who sees capsule A versus capsule B; you measure click-through rate.
2. 02Qualitative tests. Show your assets to people who match your audience and watch where they get confused. Costs nothing.
3. 03Sequential before/after tests. Change one thing on the live page, compare fixed time windows, and control for confounders.

The third is what most indies actually do. A November 2024 interview study of ten indie developers found they prefer sequential testing over split tests before release and typically run experiments lasting “a weekend to a week,” mostly because they lack the user access and analytics budget for anything fancier. Sequential testing is a legitimate methodology. Done sloppily, it’s how a developer convinces themselves a worse capsule is better.

Is page testing worth this much effort? Yes. Chris Zukowski surveyed 208 games in the February 2025 Steam Next Fest and found 68-88% of wishlists came from people who never played the demo. They judged the capsule, the screenshots, and the description, nothing else.

Steam’s native reporting is the entire first-party measurement stack now, so learn exactly what it can and cannot see.

UTM analytics. Steamworks records five parameters (utm_source, utm_campaign, utm_medium, utm_content, utm_term), and a link only needs utm_source or utm_campaign present to be tracked. No prior Steamworks setup is required; you append parameters to your store URL and the report starts populating. The full reference is in Valve’s UTM documentation.

The report splits traffic three ways, and the split decides what you can actually measure:

Conversion attribution uses a 72-hour window after the click, “even if they visit other pages in between.” Visit counts update hourly, but conversions are only finalized four days after the visit.

Valve states the blind spots plainly: the visitor must be logged into Steam in the browser where they opened the link, only app pages and sale pages are supported, and demo or DLC downloads without the base game don’t count as conversions. In Zukowski’s 2021 UTM experiments, 90.35% of visitors arriving through his tagged links were not logged into Steam, mostly mobile social traffic. Among the logged-in tracked visits, average wishlist conversion was 8.43%. A coordinated #ScreenshotSaturday test across nine developers produced five wishlists total. Here’s what that funnel does to a respectable pile of clicks:

Traffic reports. Steamworks defines an impression as your game brand being “displayed on screen to a player,” and Valve explicitly warns this “doesn’t guarantee the player has seen your game, only that it was visible on screen.” Visits are “unique page loads.” Some impression types have no measurable click-through rate at all because they never link to your store page.

Steam won’t randomize traffic for you, but Meta and Reddit will. The setup: two ad sets, identical audience, identical budget, identical copy and placement, run at the same time. The only thing that differs is the image, capsule A versus capsule B. The metric is click-through rate. Your store page never changes; you’re testing the creative.

The framing that matters: you are buying data, not customers. For most indies the unit economics of ads-as-acquisition don’t close; a $15 game leaves only about $8.50 of margin to acquire each buyer with. As a testing instrument, though, ads are cheap. A 2025 GameDev.net Facebook-ads guide reports most indie teams invest $3,000-$12,000 in pre-launch campaigns aiming for $1-2 per wishlist. A capsule split test needs a small fraction of that, because you only need enough clicks per variant to see a clear CTR gap, not enough conversions to fill a wishlist funnel. Where ads sit in your overall spend, and the full break-even math behind that $8.50, is a question for our indie game marketing budget guide; here they’re a measurement device.

Three rules keep the test honest. Change one variable per test. Run variants simultaneously, never one after the other, because day-of-week and audience mood shift constantly. And judge on CTR, not on wishlists: as the funnel above shows, ad-driven UTM wishlist counts are too small to mean anything at indie budgets.

Most page problems don’t need statistics to find, and paying an ad network to discover that your capsule is unreadable at thumbnail size is a waste of money.

The test I recommend first: screenshot a real Steam search results grid, drop your capsule into it at actual size, and send it to ten people who play games in your genre. Ask one question: “what kind of game is this?” If the answers scatter across three genres, you have a legibility problem no split test between two illegible variants will fix. Do the same with your short description: show it for five seconds, take it away, ask what the game is. Hesitation is your answer.

Run the automated check before the human one. Our capsule validator catches sizing, contrast, and readability problems, and a full page audit flags weak or missing elements before you put a single dollar behind a variant. The logic is simple: ads are for choosing between two good options. Qualitative checks and audits are for catching the bad option before you pay to test it. And if the verdict is that your capsule needs a redesign rather than a tweak, the design principles live in our capsule design guide; this post is about how to test, not what to draw.

These cheap tests target exactly the assets that, per the Next Fest survey above, do most of the selling.

When the change ships to your real page, you’re running a sequential test, and the structure is everything.

1. 01One change per test. If you swap the capsule and rewrite the description in the same week, you’ve learned nothing attributable.
2. 02Equal comparison windows. The interviewed indie devs run a weekend to a week. On a live page where daily wishlists are noisy single or double digits, I’d push for two weeks on each side of the change.
3. 03Hold everything else still. No announcements, no discounts, no festival entries inside either window.
4. 04Pin your traffic sources. Keep the same UTM-tagged links in your Discord, your social bios, and your press kit before and after, tagged identically. Then you can compare the same source against itself instead of comparing a quiet week of organic traffic against a loud one.
5. 05Log the change date. Future you will not remember.

One constraint on what you can test: since September 1, 2022, base capsules may only contain artwork, the game name, and an official subtitle. No review scores, no award logos, no discount text. Non-compliant capsules risk losing eligibility for featuring in official Steam sales and events. An ad-test winner that breaks these rules can’t ship, so only test variants you’re allowed to use.

Two notes. First, this post deliberately avoids telling you what a “good” CTR or visit-to-wishlist rate is; those reference numbers live in our CTR benchmarks and store page conversion benchmarks. For testing purposes the absolute number barely matters, the before/after delta does. Second, on wishlist quality: at GDC 2025, Valve reps confirmed wishlists are weighted by account quality, so a thousand wishlists from inactive or bot-like accounts are worth less than they look. A giveaway-driven spike during your test window is noise wearing a costume.

Every one of these has fooled a developer I’ve talked to.

The visibility round you forgot about. Ship an update mid-test and Steam may grant extra storefront exposure. Your “after” window inflates for reasons that have nothing to do with the new capsule. Update visibility rounds are great; just don’t run them inside a test window.

Festival and sale bleed. A themed fest or seasonal sale changes both the volume and the composition of your traffic. Fest-week visitors and normal-week visitors are two different populations.

The external-traffic myth. At GDC 2025, Valve reps said the claim that external traffic boosts Steam’s algorithm “just isn’t true.” So when a Reddit post goes semi-viral during your after window, it raises your numbers directly, but there is no algorithmic multiplier to credit your page change with. Attribute the spike to the post, not the capsule.

Data breakpoints. Steamworks traffic data has hard historical seams: overall traffic starts September 23, 2014; region and ownership data starts October 17, 2018; cookie-adjusted impression and visit reports only run from March 23, 2021. If your game has been in Early Access for years, comparisons that straddle those dates are invalid.

Tiny samples. The nine-developer #ScreenshotSaturday test produced five wishlists. Five. Any conclusion drawn from numbers that small is a coin flip with a spreadsheet attached.

The case studies that get passed around are developer-reported before/afters, not controlled experiments, and I’m going to present them that way because the caveat is part of the lesson.

Imagine Earth (Serious Bros) replaced its Early Access capsule art in June 2015 and went from selling 0-3 copies a day to 40-60, roughly 20x, with the devs saying nothing else changed. Zukowski, who documented the case, explicitly caveats that it was not a controlled experiment.

Kingdom Workshop's developer is the cleanest sequential example I know: methodical updates to tags, capsule art, and page assets took daily wishlists from about 5 to about 20, a 4x lift, with no streamer coverage and no viral posts during the period. One change at a time, measured patiently. That’s Method 3 done right.

Tower Factory's visit-to-purchase conversion rose from 6.8% in its first month to 13.7% after a year of iteration, on the way to 52,000 copies sold. The trajectory is the lesson here, not the multiple: a year-long window is confounded by accumulating reviews, discounts, and content updates, so don’t credit the page work alone.

None of these would survive peer review. But the direction is consistent across every case I’ve seen: when the starting page is weak, asset changes move numbers a lot.

Week 1: audit and qualitative. Run your page through the store page checklist and the capsule validator, then do the thumbnail-in-a-grid test with ten genre players. Pick exactly one hypothesis, for example “my capsule reads as a puzzle game when it’s a roguelike.”

Week 2: ad split test. Two capsule variants, identical ad sets, simultaneous run, small budget. Judge CTR only, and only once the gap is unambiguous. If the variants tie, your hypothesis was wrong; that’s a result too, and it cost you lunch money.

Week 3: ship the winner. Confirm it complies with the 2022 capsule rules, update the page, log the date, and go quiet: no announcements, no discounts. Keep your owned-channel UTM links identical to the prior weeks.

Week 4 and onward: read it properly. Let the full window run, add four days for conversion finalization, then compare equal-length before and after windows on Trusted Visits and daily wishlists. Walk the false-positive list above before you celebrate.

Then pick the next variable and run it again. Kingdom Workshop’s 4x didn’t come from one inspired swap, it came from months of exactly this unglamorous loop. Start with the audit; it’s the only step in the plan that’s free, and it usually finds the first hypothesis for you.