Incrementality Testing for Paid Search: How to Tell What Your Ads Actually Cause

Incrementality testing for paid search answers one question: how many conversions happened because of your ads, and how many would have happened anyway? Google Ads reports conversions it can connect to a click. It cannot tell you which of those customers were already on their way to your site.

That gap matters most when you suspect your reported ROAS looks too good. This article covers how incrementality differs from attribution, why search is prone to overcounting, which test designs you can realistically run, how to read the numbers, and where search term cleanup fits in.

Incremental Conversions vs. Conversions Google Ads Reports

Incrementality is the difference in outcomes between a group exposed to your ads and a comparable group that wasn't. The unexposed group is the control group (also called a holdout). If the exposed group converts at a meaningfully higher rate, the gap is your incremental effect. Everything else would have happened without the ads.

Attribution works on a different question. It takes conversions that already occurred and divides credit among the touchpoints involved: last click, data-driven, and so on. It assumes the ad deserves some credit and argues about how much. It never tests whether the ad deserved any.

That's why a model can be perfectly consistent and still wrong about causation. Attribution tells you who was in the room when the sale happened. Incrementality asks whether the sale would have happened if they'd stayed home.

An illustrative example

Suppose a store runs a branded search campaign on its own name. It reports 500 conversions a month at a 12x ROAS, which looks fantastic. Now suppose that when the campaign is paused in a test, the organic listing picks up most of those clicks, and total orders barely move. The reported ROAS was real as a record of what happened after clicks. It wasn't a measure of what the ads caused. (These numbers are made up to show the logic, not drawn from a real account.)

The misconception to drop is that a high conversion count or low CPA proves your ads drove the sales. Those metrics describe activity inside the platform. They can be strong when incrementality is high, and equally strong when it's close to zero.

Why Paid Search Is Especially Prone to Overcounting

Search captures demand that already exists. Someone typing your brand name, or a bottom-funnel phrase like "buy [product] near me," has usually made up their mind before your ad appears. Display or social ads often create interest. Search more often intercepts it, and interception is easy to mistake for creation.

Branded and non-branded behave differently

Branded campaigns often have the lowest incrementality because those searchers were heading to you anyway, especially if your organic listing ranks first. Non-branded campaigns tend to reach people who haven't chosen you yet. But that's a tendency, not a rule. A business with weak organic rankings, or aggressive competitors bidding on its name, may find its branded ads are quite incremental. Test it rather than assuming either way.

The well-known eBay search studies from the early 2010s are often cited here, and they found little incremental value in branded search for that particular company. They describe one very large, very well-known brand, so treat them as a reason to test, not a verdict on your account.

Overlap inflates the numbers

A customer might click an ad, get an email, then come back through direct traffic. Each channel's own reporting can claim some credit. Organic listings, email, and direct traffic all overlap with paid search, so platform-reported results can add up to more than your actual sales.

Loose matching adds noise

Broad match and loosely matched search terms (the actual queries people typed, as opposed to the keywords you bid on) can bring in clicks from low-intent queries. Some of those clicks convert for reasons unrelated to the ad, and many don't convert at all, which drags down efficiency. That's a search terms report problem, and cleaning it up won't fix your measurement. It does keep your spend from being diluted, which matters for the next sections.

Test Designs You Can Run for Paid Search

Search is hard to test at the user level because you can't easily hide ads from a random half of the people searching. So most practical designs work at the region or time level instead.

A geo holdout (geo experiment) pauses or reduces ads in a set of matched regions while ads continue elsewhere. You then compare outcomes between the two. It suits local or national advertisers with enough volume spread across many regions, and it's the design most small-to-mid advertisers can actually run.

A time-based on/off test switches ads off for a stretch and compares sales to a period when they were on. It's simple, and it's the weakest option. Seasonality, promotions, pay cycles, and news events all change demand between periods, and you can't separate those effects from the ads. A short pause that shows no drop proves much less than it seems to.

Google also offers its own tools. As of November 2026, check Google Ads Help for current eligibility and requirements on Conversion Lift, since availability and minimum spend thresholds can change and vary by account. Google has also published open-source measurement tools for geo-based analysis, including Meridian for marketing mix modeling. Names and maintenance status change, so confirm what's current before building a plan around any of them.

Here's how the main options compare:

• Geo holdout: moderate setup effort (matching regions, tracking outcomes by region). Budget: you give up some sales in held-out regions. Main risk: poorly matched regions or spillover between them.
• Time-based on/off: low setup effort. Budget: you give up sales during the pause. Main risk: seasonality and promotions that confound the result.
• Google-provided lift tools: setup depends on the tool and your eligibility. Budget: may require minimum spend. Main risk: limited availability and less visibility into how the measurement works.

Running a Basic Geo Holdout Step by Step

You don't need a data science team for this, but you do need discipline before launch. Most failed tests fail because the setup was loose, not because the math was hard.

1. Pick one clear question. For example: do my non-brand search campaigns drive sales beyond what organic would deliver? Testing branded, non-branded, and Shopping at once makes the result unreadable.
2. Choose a metric from your own data. Use orders or leads from your store or CRM, broken out by region. Google Ads conversions alone put you right back inside the platform's own accounting.
3. Match your regions. Group regions into test and control sets with similar past sales trends and similar size. Look at several months of history so the two groups moved together before the test.
4. Set the test length. It should cover at least one full sales cycle, plus enough time to collect a meaningful number of conversions. A product people buy within a day needs less time than a B2B lead that closes in six weeks.
5. Freeze other big changes. Avoid new promotions, major bid strategy switches, big budget moves, or site changes in either group while the test runs. Anything that hits one group differently muddies the comparison.
6. Decide the decision rule in advance. Write down what result would make you cut budget, hold it, or increase it. For instance: if incremental CPA comes in above your break-even CPA, you'll reduce spend on that campaign type.

That last step is the one people skip. Without it, a disappointing result gets explained away ("the test was too short") and a good one gets celebrated, regardless of what the data says. Pre-committing keeps the test honest.

Reading Results Without Fooling Yourself

Incremental lift is the test outcome minus the control outcome, divided by the control outcome. Here's an illustrative example with made-up numbers: ads stay on in the test regions, which produce 1,100 orders. The matched control regions, where ads were paused, produce 1,000 orders (scaled to the same size). Lift is (1,100 − 1,000) ÷ 1,000 = 10%. Only about 100 of the 1,100 orders were incremental.

Now turn that into efficiency. If you spent $10,000 in the test regions, incremental CPA is spend divided by incremental conversions: $10,000 ÷ 100 = $100 per incremental order. If Google Ads reported 600 conversions for that spend, the reported CPA would look like about $16.67. Same spend, very different picture. For incremental ROAS, divide the incremental revenue by spend instead.

Check confidence before you act

A small or noisy test can show a lift that isn't real, or a flat result that hides a real effect. Look at how many conversions sit behind each group and how much results varied between regions before the test. If the difference you measured is smaller than normal week-to-week swings, treat it as inconclusive, not as proof of zero or of success.

Common ways results go wrong

• Contaminated control regions: people travel, and some national campaigns or other channels still reach held-out areas.
• Mid-test promotions: a sale in one group but not the other invalidates the comparison.
• Treating one result as permanent: bidding, budgets, competitors, and demand all shift. Retest periodically instead of filing one answer forever.

What to Fix Before and After You Test

Clean the account first. If your search terms report is full of irrelevant queries, they dilute spend and add conversions of questionable quality, which makes both reported and measured results harder to interpret. Review the report, separate search terms from the keywords that triggered them, and add negative keywords (terms that stop your ads from showing for matching queries) for the junk you find. Do this before the test starts, not during, since mid-test changes can contaminate results.

After the test, act on what it showed. Depending on the result, that might mean shifting budget away from a low-lift campaign type, tightening match types where loose matching is pulling in weak queries, or splitting branded and non-branded into separate campaigns so you can control and measure them independently.

For the cleanup work, Keywordme is a Chrome extension that lets you add negatives and keywords and apply match types from inside the Search Terms Report in Google Ads. It doesn't run incrementality tests, and it doesn't make changes on its own. You review and choose what to add.

If your account is small

Not every account has the volume for a clean geo test. If you only get a handful of conversions per region, the noise will swamp the signal. In that case, lean on steady search term hygiene and business-level metrics: total revenue, new customer counts, and blended cost of acquisition. Watch how they move as you change spend. It's less rigorous than a holdout, but it beats trusting platform conversions alone.

Start Narrow: One Test, One Cleaner Report

Platform-reported conversions show what happened after people clicked. Incrementality tests show what your ads caused. Both are useful, but only one tells you whether the next dollar belongs in search.

You don't need to test everything. Pick one campaign type, one question, and one metric from your own data, and write down your decision rule before you begin. Clean up your search terms report first so the spend you're measuring is spend you actually want.

Optimize Google Ads Campaigns 10X Faster. Without Leaving Your Account. Keywordme lets you remove junk search terms, build high-intent keyword lists, and apply match types instantly, right inside Google Ads. No spreadsheets, no switching tabs, just quick, seamless optimization. Start your free 7-day trial (then just $12/month) and take your Google Ads game to the next level.