Geo Holdout Test for Google Ads: How It Works and When to Run One

A geo holdout test for Google Ads answers a question your dashboard can't: would those conversions have happened without the ads? You turn ads off in some regions, leave them on in comparable ones, and measure the gap. That gap is your incremental impact, and it's often smaller than what the platform reports.

This matters most if you suspect brand search and retargeting are taking credit they didn't earn. Below you'll find how the method works, how to set it up in Google Ads, how to read the numbers without fooling yourself, where it breaks down, and why your search terms should be tidy before you start.

Why Reported Conversions Don't Prove Your Ads Worked

Attribution decides which touchpoint gets credit for a conversion that already happened. Incrementality asks a different question: how many of those conversions happened because of the ads, and how many would have happened anyway? Google Ads reports attribution. It can't directly show you the world where your ads never ran.

The gap is widest in two places. Branded search is the first: someone types your company name, clicks your ad, and buys. Many of those people were headed to your site regardless, and without the ad they'd have clicked your organic listing. Retargeting is the second: you show ads to people who already visited and were likely to return. The platform logs the conversion and credits the ad, even if the ad changed nothing.

This doesn't mean these campaigns are worthless. Brand campaigns can defend against competitors bidding on your name, and retargeting can nudge some fence-sitters. It means you can't tell how much of the reported result is real from the report alone.

A holdout test fixes this by creating the missing comparison. Two terms to keep straight:

• Holdout (control): the regions where your ads are paused or suppressed for the test.
• Treatment (test): the regions where your ads keep running as normal.

If treatment regions convert meaningfully more than holdout regions, relative to how they behaved before, the ads are doing real work. If the two groups look about the same, much of your reported performance was credit for conversions that were coming anyway.

How a Geo Holdout Test Works

The logic is simple. Split your geographic markets into two groups, change ad exposure in only one group, and compare outcomes over the same period. Because both groups live through the same weeks, the same news cycle, and the same seasonal swings, the difference between them is much more likely to come from the ads than from outside noise.

How it differs from a Google Ads experiment

People often confuse this with a Google Ads campaign experiment. They aren't the same thing. As of this writing, Google's campaign experiments (the successor to Drafts and Experiments) are built to test a change, such as a new bidding strategy or setting, by splitting a campaign's traffic between an original and a trial version. Check the current Google Ads Help page for exact wording, since the feature names and options have shifted over the years. The point is that an experiment compares two ways of running ads. A geo holdout compares running ads against not running them, and it splits by place, not by auction or user.

Tools for geo-based analysis

You don't need special software to run a basic version, but analysis tooling exists. Google has published open-source work in this area, including the older GeoexperimentsResearch R package and Trimmed Match for geo-experiment analysis, and more recently Meridian, its open-source marketing mix modeling framework. Meridian is a modeling tool rather than a geo-test runner, and tool names and maintenance status change, so confirm what's current on Google's developer pages before you build a process around any of them. A spreadsheet can handle a simple test.

An illustrative example

Suppose, hypothetically, you sell online across 20 comparable metro areas. You pause your search campaigns in 10 and keep them running in the other 10 for six weeks. Before the test, the two groups produced similar weekly conversion totals. During the test, the running group's conversions hold steady while the paused group drops by 15%. That 15% drop is a rough read on how much the ads contributed in those markets. Real tests need more care than this, which the next sections cover, but the shape is the same.

Setting Up the Test in Google Ads

Work through the setup in this order, because each choice constrains the next.

1. Choose one KPI. Pick conversions, revenue, or leads before you start, and make sure you can measure it by region from a source outside Google Ads too, such as your CRM or store data.
2. Set the measurement window. Decide the pre-test baseline period, the test period, and any cooldown afterward.
3. Select and match regions. Pair or group markets with similar historical volume, seasonality, and conversion rates, then assign them to test and control.
4. Apply the change at campaign level. Use location targeting or location exclusions on the campaigns in the holdout regions.

Matching markets and avoiding spillover

Look at at least several months of history by region. Markets that move together before the test are good candidates to be compared during it. Avoid putting neighboring metros in opposite groups when people commute or travel between them. Someone in a holdout city who works in a treatment city will still see your ads, which blurs the difference you're trying to measure.

Location settings need a deliberate choice

Google Ads lets you target people who are in your locations, or people in or showing interest in them. The "interest" option means someone outside a region can see your ads because they searched about it, and a person physically in a holdout region could still be reached by a campaign aimed elsewhere. Check the current Google Ads Help wording for these options and choose the narrower, presence-based setting for a clean test.

Duration and freezing everything else

Run the test through at least one full buying cycle for your business. For a product people research for a month before purchasing, a two-week test will miss delayed conversions. I won't give you a magic number of weeks, because the right length depends on your sales cycle and volume. Treat any universal figure with suspicion unless it names a source.

During the test, freeze everything else you can: bids, budgets, promotions, landing pages, and major creative. Any change in only one group becomes an alternative explanation for your result.

Reading the Results Without Fooling Yourself

Don't compare raw totals between groups. Treatment and control were never identical, so start from each group's pre-test baseline and measure the change. This is the logic of difference-in-differences: take the change in the treatment group, subtract the change in the control group, and what remains is your estimate of the ads' effect.

From there, convert it into business terms:

• Incremental conversions = the conversions the treatment group gained or kept relative to what the control group's trend predicts it would have had without ads.
• Incremental CPA = ad spend in treatment regions ÷ incremental conversions.
• iROAS (incremental return on ad spend) = incremental revenue ÷ ad spend.

Compare these against the platform's own numbers. The platform figure usually divides spend by all attributed conversions, while the incremental figure divides it only by the conversions the ads caused, so incremental CPA is typically the higher of the two.

An illustrative worked example

These numbers are made up. Say your treatment regions spent $10,000 and Google Ads reported 200 conversions, a platform CPA of $50. Your holdout regions, adjusted for baseline differences, tell you that without ads the treatment regions would still have produced 120 conversions. Incremental conversions are 80, so incremental CPA is $10,000 ÷ 80 = $125. The campaign still may be worth running if a conversion is worth more than $125 to you, but the decision looks very different at $125 than at $50.

Report a range, not a point

Small markets are noisy. A few unusual days in one city can swing a result. Instead of reporting "80 incremental conversions," report a range, such as 50 to 110, using a confidence interval from your analysis method. If the range includes zero, you haven't shown the ads work, and you haven't shown they don't. You've shown the test couldn't tell.

Common Mistakes and Limits of Geo Testing

Most failed geo tests fail for the same handful of reasons:

• Too few regions. With only two or three markets per group, one odd city can decide the outcome.
• Mismatched markets. If the groups behaved differently before the test, the comparison starts biased.
• Ending early. Stopping the moment the chart looks good, or after a week, misses delayed conversions and locks in noise.
• Changing budgets mid-test. Any shift in spend in one group muddies the cause.
• Ignoring other channels. A TV flight, email blast, or social campaign running only in certain regions can masquerade as an ad effect.

What the method can't do

Pausing ads in some regions costs you the conversions you'd have gotten there, and you should budget for that loss as the price of learning. Accounts with low conversion volume may never produce a readable result, because the noise swamps the signal. Online-only national brands face extra spillover: people travel, use VPNs, share links, and see ads on mobile networks that don't map neatly to a region.

A geo holdout also doesn't tell you which keywords work. It measures a campaign or channel as a whole. If you want to know whether one keyword earns its place, you need a different tool, such as search term analysis and conversion data at the keyword level.

When not to bother

If your spend is small or you get only a handful of conversions a week, skip the test. Cheaper checks do more for you: compare brand impressions against organic clicks, look at the search terms report for waste, or run a short on/off period on a single campaign and watch total business results. Save the geo holdout for accounts where the spend is large enough that a wrong assumption is expensive.

Clean Search Terms Before You Test

A holdout test measures your account as it is. If a chunk of your spend goes to irrelevant queries, that waste inflates your cost per conversion and adds noise to the comparison. Cleaning up first gives you a baseline that reflects what your ads do when pointed at the right people.

Keep three terms distinct. Search terms are the actual queries people typed. Keywords are what you bid on. Negative keywords are the terms you tell Google not to match, so your ads stay away from those searches. The Search Terms Report connects them: it shows what people typed, so you can decide what to exclude and what deserves to become a keyword.

A practical pre-test routine:

1. Open the Search Terms Report for the campaigns in the test and sort by spend.
2. Add negatives for clearly irrelevant queries, choosing the match type deliberately.
3. Promote high-intent search terms that convert into keywords with fitting match types.
4. Stop. Once the test starts, freeze these changes until it ends.

This is where Keywordme fits. It's a Chrome extension that works inside the Search Terms Report, so you can remove junk search terms, add negatives, build keyword lists, and apply match types without exporting to a spreadsheet. It doesn't run geo tests, and you still review each decision yourself. It just makes the cleanup before the test faster.

Measure Real Impact, Then Protect It With a Clean Account

A geo holdout test replaces reported credit with measured impact: what your ads added beyond what would have happened anyway. Run it when your spend justifies the cost of pausing ads somewhere, and read the result as a range, not a single number. Before you start, tidy the Search Terms Report so wasted queries aren't skewing the baseline.

Optimize Google Ads Campaigns 10X Faster. Without Leaving Your Account. Keywordme lets you remove junk search terms, build high-intent keyword lists, and apply match types instantly, right inside Google Ads. No spreadsheets, no switching tabs, just quick, seamless optimization. Start your free 7-day trial (then just $12/month) and take your Google Ads game to the next level.