Jump to a section
Leave with a decision
Write a test brief that can survive an inconclusive result.
Start with the visual explanation, work through the example, then try the exercise on your app.
A screenshot test starts before design
One hypothesis should connect the change to the outcome.
Problem
Visitors may not understand how group expenses are settled.
Change
Show the settlement screen earlier and make its headline explicit.
Measure
Choose the relevant conversion measure and define the decision rule before launch.
Learn
Review the result and uncertainty. Keep a record even if there is no clear winner.
Work through it · fictional example
Test a claim, not a designer’s favourite colour.
Trail Notes wants to compare a feature-led opening screenshot with a memory-led one. Both proposed versions show the same supported journal workflow.
Why this changes the decision
The hypothesis is that the memory promise explains the value more clearly to the selected audience. Name the primary metric, store/market, guardrails, owner and review rule before launch. Keep other planned changes separate where possible so the result answers the intended question.
What to check before calling it a win
Use the experiment platform’s current methodology to interpret the result. If evidence is inconclusive, record that outcome, inspect the assumptions and choose whether to continue, redesign or stop. Do not keep peeking until a favourable number appears.
Your turn · a five-minute start
Make one useful decision for your app.
Write the hypothesis, what stays constant and what you will do if neither version has convincing evidence.
What a useful answer looks like
A useful plan defines the inconclusive path. “Run until it wins” does not.
Start with a documented uncertainty
A test should resolve a decision. Record the observation that suggests the current screenshot could be misunderstood, and explain how the proposed version addresses it. “Make it better” is not a hypothesis. An explicit hypothesis helps the team learn even when the result is inconclusive, because it explains what was actually tested.
Define control, treatment and audience
Save the exact assets for each version. Specify what changes and what stays fixed. Record the store, market, locale and eligible audience in the native console. If you change the icon, headline and entire screenshot sequence together, the result describes the package; it cannot isolate which element mattered.
No clear winner. Now what?
An inconclusive test is not permission to declare your favourite design the winner.
Was the test informative?
Check implementation, exposure, traffic quality and the uncertainty in the result.
Not enough evidence
Keep the result inconclusive. Extend only if the test plan and available traffic justify it.
Evidence against the hypothesis
Revisit the problem and audience. A new colour is not a new explanation.
Choose the metric and decision process first
Use the platform’s current result guidance and write down how you will respond to a clear improvement, clear decline or inconclusive result. Set a review cadence without promising that a fixed number of days will produce certainty. Small traffic can limit what a test tells you. Avoid repeatedly stopping the test at the first favorable movement.
Control the operational context
Coordinate product releases, paid campaigns and localized updates with whoever owns the experiment. Not every source of change can be removed, but it can be recorded. Confirm the correct assets are visible after launch. Keep screenshots of setup and results so the decision can be explained later.
Write a useful conclusion
State the hypothesis, conditions, measured result, uncertainty and decision. Do not turn one result into a universal rule about screenshot design. If a treatment is rolled out, verify that the assets were applied correctly and monitor the relevant outcome. A record of unsuccessful experiments can prevent your next team from repeating the same unsupported idea.
Write the hypothesis before the design request
Illustrative hypothesis: for visitors interested in keeping a hiking diary, showing an actual journal entry in the first frame may explain Trail Notes more clearly than a generic landscape. Compare the current opening frame with a journal-led version and document the audience and primary metric.
| Part of the brief | What to record |
|---|---|
| Control | The current opening screenshot |
| Treatment | The proposed journal-led first frame |
| Held constant | The remaining sequence and agreed surrounding assets |
| Measurement | The native platform metric and its exact definition |
| Review | The platform result, uncertainty and planned decision |
| Context | Releases, traffic changes and other campaigns during the test |
An inconclusive test still needs a decision
A small observed difference is not automatically a winner. Check whether the experiment can answer the question with the traffic available. You may keep the control, reduce the number of variants or return to a clearer hypothesis. Do not turn an inconclusive result into a claimed lift for a case study.
Your next steps
- Document one uncertain communication problem.
- Save control and treatment files with clear names.
- Define the result interpretation before launch.
- Keep a conclusion that includes uncertainty and next action.
Sources and platform guidance
Platform documentation checked 29 September 2026. Verify the applicable console settings before publishing.
Questions and answers
Should every test run for a week?
No. Required evidence depends on traffic, effect size and the platform’s method. Follow the actual result guidance rather than a universal calendar rule.
Can a test prove every future audience will prefer a version?
No. Its evidence is bounded by the tested audience, assets and conditions.
Work directly with Gabriel
Your screenshots need a message before a makeover.
I review the opening promise and sequence, then prepare designer-ready messaging and a plan for the next test. You get a brief that says more than “make it pop”. Your designer deserves that much.
Sharpen my screenshot test →Or email gabriel@asoprofessional.com. Bring your app link, market and the decision you need help with.
Want a first opinion? Get a free video teardown of your listing.