From ad creative to A/B test hypotheses

It is 9:30 on a Tuesday. A performance marketer opens last week's numbers for eight banners: two pulled ahead on click-through, three landed below the account average, and three barely spent anything at all. The team lead asks one question: "So what are we testing this week?" The answer usually arrives in the same shape: "Let's shorten that headline and see what happens." That is not a hypothesis, it is a hunch. A test calendar built on hunches leaves you, three months later, with a pile of results and no explanation for any of them.

This piece walks through turning creative-analysis findings into hypotheses you can actually test: why the report alone is not a hypothesis, the four parts a usable one is made of, which weak score deserves a test slot, how many elements should change at once, where video hypotheses come from, which thresholds have to be written down before launch, when a winning variant has not really won, and what survives a finished test. The order is deliberate — each section takes the previous one's output as its input.

Why is an analysis report not a hypothesis on its own?

A creative analysis hands you a diagnosis: this banner scores low on composition, text density is high, the call to action sits below the visual hierarchy. All of that can be true and none of it is testable yet. A finding is an observation; a hypothesis is a prediction with a mechanism inside it — change this, and that behaviour moves in this direction, for this reason. Creative Intelligence produces an AI Score that rates every banner and video from zero to ten across nine dimensions, including visual, composition, colour, copy, call to action and attention. That table is raw material, not a test plan.

Ignore the difference and your test calendar quietly becomes a fix-it list, where every week the reddest row in the report gets patched. Patching the red row may not move performance at all, because that dimension may not be decisive in that placement. Tracking which visual wins and why, in a repeatable way, is a different job from closing report gaps one by one: the first produces learning, the second produces activity.

What four parts make a usable hypothesis?

A workable creative hypothesis carries four parts: which element changes, what the current variant is, what the proposed variant is, and what effect you expect. When all four are not written down, the post-test argument is guaranteed — the distance between "we expected exactly this" and "this result is surprising" is a single sentence written before launch. The A/B Test Hypotheses output fills in precisely those fields: every analysis proposes three tests, and each one carries the element to change, its current state, the proposed state and the expected impact as separate entries. The hypothesis is a stored record you can return to, not a comment that disappears when the tab closes.

You have to add a fifth part yourself: the measurement metric. "Better performance" is not a metric. If the change concerns attention in the first second, measure view rate; if it concerns the call to action, measure click-through; if it concerns the offer wording, measure cost per conversion. A team that picks the metric after the fact will crown whichever number happens to look good. A test run against the wrong metric returns a correct answer to the wrong question, and nothing in the report flags the mistake.

Which weak score is actually worth a test?

Not every low score deserves a slot. Three filters help: how decisive that dimension is in this placement, how expensive the fix is, and where the current score sits in the industry distribution. Industry-Specific Checks answer the first question directly — a weighted checklist runs for each of twenty industries, and the creative is also scored against the target placement you select. What matters in an in-feed visual is not weighted the same way as what matters in a display banner served across a content network, and scoring both against one generic checklist is misleading from the start.

Industry Benchmarking closes the third question: your score is positioned against the average and the percentile bands of analyses in the same industry and placement. A contrast score of six means nothing by itself; it means something once you know most analyses in that placement sit above eight. The Prioritised Recommendations list then ranks suggestions as high, medium or low priority, so what goes to the front of the queue stops being a matter of taste and the weekly stand-up is not spent arguing about order.

How many elements should change in one test?

The textbook answer is one, and most of the time it holds: change two things at once and you learn which variant won without learning why. In creative work, though, a rigid single-variable rule can get expensive. On a low-traffic campaign, one headline swap will burn the budget without producing a difference you can trust. The reasonable move there is a concept test: race two genuinely different approaches, then run single-variable refinements inside the winning concept. Reversing that order — fine-tuning first, questioning the concept later — is the most common way to lose a quarter.

Isolation gets harder still in campaigns you hand to automation. In campaign types where the algorithm assembles creative combinations itself, you do not control which variant reaches which audience, so a hypothesis you genuinely want to test has to be isolated in a separate campaign.

Opening a new creative set to the full budget at once is an unnecessary risk as well. Starting with a small share and widening in stages caps the damage when the hypothesis is wrong, and when it is right you can repeat the decision at scale with confidence.

How do copy variants become hypotheses?

The most common mistake in copy testing is generating variants at random: three headlines get written, nobody can describe what separates them, a winner emerges and no one can say what was learned. Variants should differ by angle — one leads with the benefit, one answers the objection, one carries urgency. AI Copy Variants does exactly that: three sets of headline, body and call to action are generated from the analysis findings, each built from a different angle. When the test closes, the winning angle is readable, and that reading carries into the next campaign brief.

Alongside angle there is a measurable form dimension. Copy and CTA Analysis reads the headline, body copy and call to action inside the image automatically, and evaluates word count and readability separately. Those two values turn the "proposed variant" field into something concrete: instead of "let's shorten the headline", you write "cut the fourteen-word headline to seven words". The first is a discussion; the second is an instruction you can execute and repeat next quarter.

Where do video hypotheses come from?

In video, the creative is not a single surface but a timeline. For a banner, "the call to action is weak" is a complete finding; for video, the same sentence has to answer when. Video Ad Analysis extracts six key frames from the video and produces seven scores, including opening impact, pacing, narrative and call-to-action timing. The Frame Timeline then marks the strongest and weakest frame, and shows where the opening, the product reveal, the call and the close actually land along that timeline.

Hypotheses that come out of this are naturally time-bound: "the product first appears at second six; moving it to second two will reduce drop-off in the first three seconds." That single sentence already contains the element, the current variant, the proposed variant and the expected effect. In campaigns built around video reach and conversion, the opening seconds decide most of the measurement, so opening hypotheses usually enter the queue first, and end-frame hypotheses wait until the opening is solid.

Which thresholds must be written down before launch?

Once the hypothesis is written, the decisions about how the test ends belong on paper too — before it starts. Otherwise the rules drift as the results arrive, and the process degrades into looking until the desired answer appears. One line per field below is enough; what matters is that it was decided in advance rather than negotiated afterwards.

Decision area	Written before launch	What happens if it is not
Primary metric	The single metric that decides the winner	Every variant wins on something and the call stalls
Minimum volume	Impressions and clicks needed per variant	Noise gets mistaken for a result
Minimum duration	Full days to run and which weekdays are covered	A weekday-weekend gap looks like a winner
Stop rule	The bad outcome that ends the test early	A losing variant runs until the budget is gone
Winning margin	The size of gap that justifies a rollout	The creative process is rebuilt for a trivial difference

These thresholds help on a single channel and become mandatory the moment the same hypothesis runs across several. In a setup that brings several ad platforms into one dashboard, the same creative behaves differently by placement, and if the thresholds are not written jointly each channel declares its own winner.

If budgets are managed by rules, one more threshold is needed: which automations pause during the test window. Backtesting a rule against historical data first keeps budget from shifting mid-test and stops two variants from competing under unequal conditions.

When has the winning variant not actually won?

The riskiest moment in a test is when the result matches the expectation, because that is the moment nobody wants to check it. There are a few familiar ways a winner has not really won: the test was stopped before the learning phase ended; the two variants shared an audience and ate each other's impressions; the measurement window fell across a holiday or a promotional period; or one variant did not fully go live on day one and simply received fewer impressions.

That last case is more common than it looks. A variant getting disapproved on policy grounds, or the account being restricted, corrupts the test data quietly, because the report only shows lower delivery and never the reason. Creative fatigue is a similar silent distorter: two weeks in, frequency on the winning variant climbs and the gap closes. This is why adding a line to the exit decision — read the result once more two weeks later — is what tells you whether the change you rolled out actually held.

How does a creative hypothesis pair with the landing page?

If the promise made in the creative is not met on the screen after the click, your test winner only produces a more expensive misunderstanding. Every creative hypothesis deserves a twin: a hypothesis that tests the same promise on the landing page. Landing Page Analysis evaluates the page section by section from desktop and mobile screenshots — headline, value proposition, call to action and trust signals are scored separately, each with a concrete fix. When the angle in the creative and the headline on the page say different things, the next test picks itself.

The technical side runs alongside it. Technical Page Measurement reports performance, accessibility, SEO and best-practice scores together with LCP, FCP, CLS and TBT. Running a headline test on a page with a badly slow mobile LCP means the thing you are measuring is not the headline. And a setup that matches ad spend to revenue recorded in the CRM lets you validate the winner against closed deals rather than click-through, which matters because some creatives buy cheap clicks and expensive customers.

What is left when the test ends?

The real output of a test is not the winning image but a sentence that has been confirmed or refuted. Teams that collect those sentences build in-house knowledge about their own audience within a couple of quarters and stop guessing which angle holds in which placement; teams that do not rebuild the same test every quarter. Side-by-Side Comparison makes the collecting easy: put two or three creatives against each other, let the strengths be weighed, pick the winner, and the result stays on record, so months later you can revisit why a variant came out ahead.

The next campaign starts from there as well. Bulk Analysis scores up to twenty creatives in one ZIP in a single pass, so you can see which members of a new set already follow the rules you have validated, and the PDF Report hands the same findings to a colleague or a client as they are. Leaving a proven creative in one placement is waste, too: the logic of getting many pieces out of a single asset applies to creative as well, because a validated angle keeps working when the format changes.

If you want creative scores, hypotheses and test outcomes living in one place, open a free Rocketly account and start by analysing your current creative set.

FAQ

How many hypotheses should run at the same time?

The number of parallel tests should never exceed what your traffic can support at a meaningful volume per variant. On small budgets one hypothesis at a time is enough; on high-volume accounts two or three can run in separate campaigns. Queued hypotheses are not discarded, they wait in the priority list.

Has the test failed when the hypothesis is refuted?

No. A refuted hypothesis still produces information and closes off a direction you no longer need to retry. A failed test is one you can learn nothing from, which usually happens when the measurement metric was left vague or the difference between variants was never described.

Can I run an AI-suggested hypothesis as it comes?

Treat it as a first draft. The element, the current variant, the proposed variant and the expected effect arrive filled in; brand tone, legal limits and placement constraints are yours to add. Making it shippable usually takes a few sentences of editing, not a rewrite.

Do video and banner hypotheses share one queue?

They share the prioritisation logic but should not compete inside the same test. Video hypotheses hang on a timeline and banner hypotheses on single-frame composition, so their measurement metrics diverge; comparing the two in one test blurs where the difference came from.