Rolling out an ad rule in stages: start with 10%

Thursday morning, 9:40. A performance marketer at a mid-sized ecommerce brand opens the dashboard to see what a single rule did overnight — the one switched to fully automatic the evening before: "if 7-day ROAS is below 1.0, pause the campaign." The logic was sound, the threshold defensible, nobody argued with it. Yet nine of the fourteen active campaigns are now paused, including two launched at the start of the week that had not exited the learning phase, and one retargeting set producing the cheapest acquisition cost of the month. (The numbers describe an example account.) The rule's judgment was not wrong. Its scope was. It went live across the entire account on night one.

This piece is about reaching the same rule, at the same threshold, by widening scope in stages instead. In order: why a correct rule still should not open account-wide, what "start with 10%" is actually ten percent of, which campaigns belong in the first stage, why backtesting and simulation cannot replace staged rollout, which mode the first stage should run in, what conditions must be met before moving up a stage, which measurements mean anything while scope grows, what conflicts surface as the rule spreads, how to roll back, and the habits that quietly wreck the whole process.

Why does a correct rule still not open across the whole account?

A condition–action rule compresses two separate things into one sentence: a judgment ("below 1.0 ROAS, this campaign is losing money") and a scope ("which campaigns that judgment applies to"). Most teams debate the judgment for hours and never debate the scope. But a flawed judgment usually reveals itself on a handful of campaigns and gets corrected; a flawed scope applies that same flaw everywhere, in one night. Staged rollout exists to separate the two: test the judgment while keeping the blast radius small.

The second reason is comparability. Open a rule everywhere at once and there is no other side to compare against. If spend drops the following week, you cannot say whether that came from the rule, from seasonality, or from the platform reweighting delivery. Split the scope deliberately and the campaigns the rule touches and the ones it does not live through the same week under the same conditions. This is not a clean experiment — campaigns are not randomly assigned, so no causal claim survives — but observation at least gets a reference point.

The third reason is trust. Most teams that resist automation are not resisting automation; they are resisting loss of control. Reading a few days of "here is what the rule would have done" on a small set moves that objection out of abstraction and onto specific decisions. The same reasoning settles the broader question of when to hand control to automated campaign types: observe narrowly first, widen second.

What is "start with 10%" ten percent of?

In practice the phrase means one of three different things, and the choice changes what your results mean. Ten percent of campaign count: with ten campaigns, the rule runs on one. Ten percent of spend: the rule runs on campaigns carrying a tenth of daily budget, which usually means a handful of small ones. Or ten percent of time: the rule runs on everything, but only during certain hours of the week.

All three are legitimate, and they answer different questions. Splitting by campaign count is the cleanest way to test the judgment, because a variety of campaign types enters the sample. Splitting by spend is a risk control — you know the ceiling on what a wrong action can cost before you start. Splitting by time only makes sense for metrics that degrade in a specific window, such as CPM spikes outside business hours. In Rocketly, restricting a rule to chosen days and hours is a separate setting and should not be confused with splitting scope.

A practical recommendation: pick the first stage by campaign count, but check its spend share while picking. If ten percent of your campaigns happens to carry half your spend, you have opened a large risk under the label of a small stage. Widening across several platforms simultaneously is its own mistake; managing Meta, Google and TikTok campaigns from one dashboard makes the work easier, but delivery mechanics differ per platform, so stages should advance one platform at a time.

Which campaigns belong in the first stage?

The first stage has to satisfy two conflicting requirements at once: representative enough for the rule to fire, small enough that a wrong action stays affordable. That is why starting with your best and worst campaigns is wrong for two separate reasons. On your best campaign, one bad pause is expensive. On your worst, the rule fires every single time and teaches you nothing about its discriminating power.

The campaigns in the middle — sitting just above and just below the threshold — carry the most information. Rocketly's "pick the threshold from your data" view makes this visible: it shows how the metric is distributed across your campaigns, turns the median into a threshold in one click, and tells you how many campaigns that threshold would catch. Choosing the first stage from the middle of that distribution surfaces the rule's behavior on borderline cases in week one.

The second filter is data maturity. Campaigns in the learning phase, freshly launched, or converting in single digits per day do not belong in stage one; their metrics swing day to day and the rule mistakes noise for signal. The third filter is budget structure: in accounts mixing campaign-level and ad-set-level budgets, the same rule acts in two different places, so limiting the first stage to one budget type removes the ambiguity. And if the goal is simply to stop obvious waste, cutting wasted search terms with negative keywords often pays off faster than any automation rule.

Why do backtesting and simulation not replace the staged rollout?

Rocketly offers two checks before a rule is switched on. "Backtest before you turn it on" shows how many times and on which campaigns the rule would have fired over the past 28 days, touching nothing. "Simulate with your own numbers" lets you enter sample metrics and see whether the condition matches and how the guardrails would clip the change. Both validate the rule's logic and should absolutely run before a staged rollout begins.

But both answer the same question: what would the rule have done in the past. Neither answers what happens after the rule changes something. A backtest does not know that a paused campaign's budget will flow to a neighbor. It does not know that pausing a set frees the audience and raises delivery cost on the sibling set. These second-order effects only appear once the rule actually acts, which is exactly why staged rollout exists.

The same distinction shows up in creative work: scoring an asset before it goes live, the way analyzing ad creatives before publishing does, narrows risk without telling you how the asset behaves in real delivery. The rule engine is no different. Pre-checks shorten the staged rollout; they do not cancel it.

Which mode should the first stage run in?

Every Rocketly rule runs in one of three modes: preview only, with my approval, or automatic. For stage one the right mode is almost always preview only. The rule evaluates campaigns, reports how many matched and what would change, and writes nothing to the account. Its purpose is not to apply decisions but to make you read the list of them.

The work at this stage is a daily habit: open the match list and answer one question per row — would I have done this myself? Rows where the answer is no point at what needs fixing, and it is rarely the threshold. Usually it is the lookback window or a campaign type that should have been excluded. A week of lists replaces guesswork about where the setting should land.

Stage two switches the mode to approval. Matches land in an approval queue, you see the reason and the exact change before anything happens, and nothing touches a campaign until you say so. This is not a waiting room — it is where the rule teaches you the most. The ratio of approved to rejected decisions is the truest signal of whether the rule is ready. Just as building a structured creative testing system records why a winner won, the approval queue records why a rule decided what it decided.

What has to be true before moving up a stage?

Stages should advance on conditions, not on the calendar. "It has been two weeks, let us open it up" is the most common way staged rollout falls apart, because elapsed time proves nothing on its own. The table below shows how an example progression can be structured. The percentages shift with account size; what does not shift is that every row has an exit condition.

Stage	Scope	Mode	Before advancing
1	About 10% of campaigns	Preview only	One full data week, every decision on the list actually read
2	About 25%	With approval	Rejection rate clearly low and rejection reasons not repeating
3	About 50%	With approval	Fewer changes clipped by guardrails, no conflict warnings
4	Full scope	Automatic	No decision in the previous stage required a rollback

The last row matters most. Going automatic should be delayed not because the rule works well, but because a rule that fails does not announce it. If a single rollback occurred during a stage, that stage repeats. The discipline looks rigid but it is cheap: another week of waiting costs far less than one wrong action applied account-wide.

What actually tells you something while scope grows?

This is where staged rollouts get misread most often. Looking at total account ROAS after a rule opens tells you close to nothing, because creative, seasonality, competition and platform delivery all moved in the same week. The only comparison with meaning is at the action level: each applied action's before and after compared across equal windows. Rocketly's "measurement after the action" view does exactly this, and states plainly that it is observational and claims no causality.

The second source is the rule's own metric. If the rule runs on ROAS, platform-reported conversion value misleads most B2B and long-cycle businesses. Decisions need to reconcile with revenue closed in the CRM, and tying ad spend back to CRM revenue has to exist from day one of the rollout, not be bolted on later.

Third is data integrity. As scope widens, the number of campaigns the rule touches grows and so does measurement error. Where browser-side measurement is incomplete, thresholds are hard to trust without server-side tracking and the Conversions API. Closing the loop back to the platform belongs to the same chain: without sending offline conversions back to the ad platform, the ROAS the rule sees and the revenue the business sees stay two different numbers.

What conflicts appear as scope widens?

A single rule running alone looks fine. Add a second and third and you get decisions that cancel each other on the same campaign: one trims budget for weak ROAS while another raises it for high conversion volume. Rocketly flags these rule pairs in its "catch conflicting rules" view and resolves them from one screen. Execution order is configurable too, so you decide which rule acts first.

The second conflict type is scope overlap. Moving from 10% to 50%, some newly included campaigns already fall under another rule. Defining scope as explicitly listed campaign sets rather than name patterns or tags removes most of this confusion. The third is between account-level policy and rule settings: when working hours, minimum data days and pause approval are defined account-wide, no rule can loosen those limits, so a looser setting written at rule level simply has no effect.

The fourth comes from targeting. Audiences built from CRM lists create their own performance profile; bringing your best customers into ad platforms as matched audiences pulls the ROAS distribution upward, and the same threshold fires far too late on those campaigns. Keeping them in a separate rule set is the right answer.

How do you roll a stage back?

The rollback plan has to exist before the rollout starts. It has three levels. First, reversing a single decision: an applied action, or an entire run, can be undone within 24 hours and the budget returns to its previous value. Second, dropping the rule one stage: automatic back to approval, approval back to preview only. Third, the master switch: automation ships off by default, and switching it off never deletes your rules, it only stops them running.

Which level you use depends on whether the failure is isolated or systemic. One unexpected action on one campaign: reverse it and continue. Two wrong decisions of the same type in a row: drop the rule a stage. Several rules behaving oddly at once: the problem is in the data layer, not the rules, so kill the master switch and check the sync.

There is also a silent form of failure — the rule that never fires at all. A rule producing no results for a long stretch is usually broken: the threshold was set out of reach, or the scope emptied out. Rocketly notifies marketing owners when a rule repeatedly produces nothing, so broken automation does not sit quietly in the background.

Which habits break a staged rollout?

First, shortening the ladder. When week one goes well, the pull to skip stages two and three and go straight to automatic is strong, and giving in turns staged rollout into a ceremony. Second, editing the rule mid-stage. Change the threshold or the lookback window halfway and your observation is void; if a change is genuinely needed, reset the stage and start it again.

Third, widening scope and changing the rule in the same week. Add a new platform, a new campaign type and a bigger scope together and nothing can be attributed to anything. Fourth, leaving measurement plumbing for later: if naming conventions and UTM-based campaign tracking are inconsistent, you cannot separate touched from untouched campaigns in any report. Fifth, assuming data health; a rule built before sync freshness, 28-day coverage and account currency are verified produces decisions from stale numbers.

Teams that take staged rollout seriously share one habit: they record not what the rule did but why it did it. When every action logs its reason and its outcome, "why did this budget drop" is answerable three months later without guesswork. If you want to try a first rule in preview mode on a small slice of your campaigns, you can open a Rocketly account and run it without touching a single budget.

FAQ

How long should the first stage last?

At least one full data week, because day-of-week performance alone can push a metric above and below a threshold. Time is necessary but not sufficient: the stage ends when every decision on the list has been read and questioned.

I have few campaigns and 10% is one campaign. What then?

In small accounts, split the mode instead of the scope. Run the rule on all campaigns in preview only; nothing changes, but the decision list becomes real. Move straight to approval mode once you trust it.

The backtest looked clean. Do I still need stages?

Yes. A backtest shows what the rule would have done historically; it does not show how budget and delivery shift after an action lands. Second-order effects only appear once the rule really acts.

What should I watch after the rule goes automatic?

Three things: how many decisions needed a rollback, how many changes the guardrails clipped, and whether the rule went silent. All three flag broken logic far earlier than account-level totals do.