Guide to Export 25000 Rows in GSC API without Coding
Use analytics for better SEO / Practical playbook / 2026–2027 edition

Search Console data at scale. Export it. Query it. Trust it.

Go past the 1,000-row table. Export with the API, keep your own history in BigQuery, and run 12 SQL investigations that end in a decision you can defend.

Four steps. Four rungs. Climb only as far as the question needs.

QUESTIONGRAINQUERYDECISION What do I need to know?Which rows can answer it?Did it return what I expect?What happens next? RUNG 1Search Console UI1,000-row table RUNG 2API Explorer25,000 rows per request RUNG 3Scripted APIRepeatable daily pulls RUNG 4BigQuery exportDaily, retained, joinable
Teaching illustration built on Google’s documented limits. It shows no site data.
4 rungs, 1 ruleThe Data Ladder and the Grain Rule decide what to use and how to read it.
12 SQL investigationsEach with its grain, a reading guide, what to verify and the next action.
Updated for 2026Hourly API data, the branded filter, export gaps and the generative AI report.
Inside this guide

Search Console is the closest thing you have to Google’s own record of how your pages perform. The interface shows a table of 1,000 rows, so most people go looking for more rows. That is rarely the real problem.

The real problem is grain: whether a number describes a property, a page, a query, or a pool of queries Google will never show you. Two honest exports can disagree and both be right. I would rather ship a 1,000-row finding I can explain than a two-million-row dashboard nobody can.

This guide is the third step of the technical evidence trail on this site. First understand how a page renders, then check what bots actually did in your logs, then pull Search Console data at scale to confirm both. If your finding began in rendering or logs, this is where you test it against demand.

The working method

Question → Grain → Query → Decision. Name the question, choose the grain that can answer it, write the smallest query that returns it, then decide what to investigate or change. A number is a lead, not a verdict.

01Answer first

Three questions, three direct answers.

How do I export more than 1,000 rows from Search Console?

Use the Search Analytics API. One request returns up to 25,000 rows, and startRow pages through the rest. Google exposes at most 50,000 rows per day per search type, taken from the top by clicks, so it is not a complete export.

What does the BigQuery bulk export add?

A daily, ongoing copy of your performance data in your own warehouse. You can query it with SQL, keep it past the 16-month window and join it with other data. It starts on the day you set it up, does not backfill, and still hides anonymised queries.

Which one should I use?

The lowest rung that answers the question. Use the interface to look, the API to extract and BigQuery to retain and join. Each step up adds setup, cost and something to maintain.

The rest of this guide shows how to do each, how to read the numbers without fooling yourself, and what to run once the data is in place.

02Decision guide

Which Search Console data path do you need: UI, API or BigQuery?

Choose by the question, not by how impressive the tool sounds. I treat the choice as four rungs, and I start on the lowest one that can answer.

Rung 1

Search Console interface

Filter a page or query and read the answer. Right for a single page, a quick look or a first hunch.

Rung 2

API Explorer, no code

Paste a short request, authorise, and read back up to 25,000 rows per request. Right for an occasional filtered extract.

Rung 3

Scripted API or connector

A pagination loop or connector you maintain. Right when the same pull repeats every day or week.

Rung 4

BigQuery bulk export

A daily warehouse copy you own. Right for retained history, SQL and joins with crawl, analytics or CRM data.

What are you trying to do?

Pick one and the matching rung lights up.
Choose what you are trying to do.
Search Console performance data: which path fits which need.
What you needInterfaceAPIBigQuery export
Investigate one page or query this morningBest first stopPossibleUnnecessary
A filtered extract of more than 1,000 rowsTable shows 1,000 rowsBest fitOverkill unless already running
The same report, refreshed dailyManualScripted request or connectorScheduled SQL view
History beyond the 16-month windowNoOnly if you archive it continuouslyBest ongoing option, from setup day
Very large query × URL analysisLimitedTop 50,000 rows per day per search typeBest fit
Joins with crawl, logs, GA4 or CRMNoBuild your own pipelineSQL-ready warehouse
Data from before you set anything upAbout 16 monthsAbout 16 monthsNo automatic backfill
The idea to keep

The Data Ladder: climb only as far as the question needs. A higher rung has to earn its setup, its bill and its upkeep.

That is also why I would not enable Cloud billing to answer a one-page question. If you do need rung 4, the sooner you switch it on the better, because the export cannot recover days before it started.

032025–2026 update

What changed in Search Console data in 2025 and 2026?

Six changes affect how you extract and read the data. None removes the need to decide the grain first, and two of them are traps if you ignore them.

WhenWhat changedWhat it means for your pipeline
Apr 2025The Search Analytics API gained hourly data: an hour dimension and a hourly_all data state, covering up to the last 10 days. Hourly rows can be partial.Use it to check a launch or incident within hours. Do not report from it, because hourly rows can still change.
2025Fresh-data requests now return a metadata object with first_incomplete_date or first_incomplete_hour.Store it with the pull. Flag or drop every date after it before you compare periods.
Oct–Nov 2025Query groups arrived, then a branded queries filter in the Performance report, announced 20 Nov 2025.Handy for a quick look. Google’s classification is not a regex and eligibility is limited, so keep your own brand pattern in SQL when you need a definition others can inspect. My regex guide covers that fallback.
28 Feb and 1 Mar 2026Google confirmed these two days were missing from bulk exports for some properties and would not be recovered.Test for missing days automatically (investigation 05) and keep an independent daily archive of site totals, so a gap is visible and explainable.
3 Jun 2026Generative AI performance reports launched for Search (AI Overviews and AI Mode) and Discover. They show impressions by page, country, device and date. They show no queries and no clicks.A separate dataset that lives in the interface with a CSV export. At the time of writing I found no documented API or bulk-export access. My SEO checklist shows how to read it next to the rest of your evidence.
2026Clicks from AI Overviews and AI Mode are counted inside regular Web search data, with no filter to separate them.Expect the same in your exported WEB rows. You cannot isolate AI feature traffic from these tables.
The idea to keep

Freshness is a flag, not a feature. Recent days look weak because they are incomplete, not because demand fell. Carry the flag with the data.

04Foundation

Why don’t my Search Console numbers match? Decide the grain first.

They match when you compare the same grain. Google calculates impressions, clicks and position differently when you group by property than when you group by page, and the API and the export return different slices of the same data.

The Grain Rule

One question, one grain. Write the grain, the date window and the search type before you write the query. If two numbers disagree, compare their grain before you suspect either of them.

Eight things to settle before the first query

DecisionWhy it changes the answerWhere it shows up
Property or page grainA search that shows three of your pages is one property impression but three page impressions.API aggregationType; the site and URL tables in BigQuery.
Search typeWeb, Image, Video, News, Discover and Google News are separate surfaces.API type; BigQuery search_type, stored in capitals such as WEB.
Position mathPosition is impression-weighted, and exported sums are zero-based.SUM(sum_position) / SUM(impressions) + 1 on the URL table; sum_top_position on the site table.
Hidden queriesAnonymised queries have no query string, yet they still count in totals.is_anonymized_query; query is empty for those rows.
Dropped rowsAsking for page and query together can drop some rows.API only. Query without them when you need accurate totals.
Time zoneSearch Console dates follow Pacific Time, not your office clock.Every source. Convert before comparing with GA4 or logs.
FreshnessThe latest days are provisional, and data usually trails by two to three days.dataState, first_incomplete_date, your date window.
Row ceilingThe API exposes the top rows by clicks, up to 50,000 per day per search type.API only. The bulk export is not bound by this ceiling.

One week, three honest totals

Here is the mechanism in numbers. Every row is correct. They answer different questions.

Synthetic numbers to show the mechanism. They do not come from a real property.
GrainImpressionsWhat it tells you
Property grain, all queries10,000The number to report as your total.
Sum of URL rows12,600Higher, because one search can show several of your URLs. Never sum URL rows to get a site total.
Visible query rows only4,300Lower, because anonymised queries have no string. This is the share you can actually inspect.

I call the missing slice Hidden Share: anonymised impressions divided by all impressions. Investigation 03 measures it on your own property, because every query-level finding you publish is a statement about the visible share only.

Manikandan’s review point

Before you accept a finding, write four things: the unit of analysis, the baseline period, the smallest sample you would trust, and what evidence would prove you wrong.

05Rung 2 · No code

How do I export more than 1,000 rows from Search Console without coding?

Use Google’s API Explorer. Sign in, enter your property, paste a short request, and read back up to 25,000 rows per request. This is the route my 2023 tutorial taught, updated for what the API does today.

  1. Open the Search Analytics query reference ↗ and scroll to the “Try it” panel at the bottom.
  2. Enter siteUrl exactly as Search Console lists it: sc-domain:example.com for a Domain property, or https://www.example.com/ for a URL-prefix property.
  3. Paste a request body from the lab below. Start with the quick sample, then move to the daily pull when completeness matters.
  4. Authorise with Google OAuth 2.0 using an account that has access to the property, then run the request. A 200 response means it worked.
  5. Page through the results. Repeat the request with startRow at 25,000, then 50,000, until a response returns zero rows.
  6. Save the response as a .json file and open it on your own machine, for example with Excel’s Power Query (Data → Get Data → From File → From JSON), then expand rows and keys.
What I changed since 2023

The earlier version of this guide sent you to a free online converter. The response holds your real queries and URLs, so I now keep that step on your own computer.

Request lab

Paste any of these into the Request body field. Dates are in Pacific Time. The patterns and settings are illustrations, so adjust them to your property.

Request body · quick sample
{
  "startDate": "2026-07-01",
  "endDate": "2026-09-30",
  "dimensions": ["query", "page"],
  "type": "web",
  "rowLimit": 25000,
  "startRow": 0
}
Fast for a first look. Across a long range you get the top rows by clicks, not every row. Use the daily pull when completeness matters.

The limits to respect

LimitWhat Google documentsWhat I do about it
Rows per response1 to 25,000, with 1,000 as the default.Always set rowLimit and page with startRow.
Rows exposedUp to 50,000 per day per search type, sorted by clicks. The API returns top rows, not a guarantee of every row.Pull one day at a time, and never call the result a complete export.
Page and query togetherSome data may be dropped when you group by both.Run a second request without them for accurate totals.
FreshnessData is usually available after two to three days; final data is the default.Use dataState: all only with the incomplete-date flag.
DatesPacific Time.Convert before you compare with GA4 or server logs.

When the same pull becomes routine, move to rung 3: a short script that loops over days and pages, or a connector you trust. Write each day’s result to your own storage as it arrives, because a missing day cannot always be fetched again later.

06Rung 4 · Ongoing export

How do I set up the Search Console bulk export to BigQuery?

Create a Google Cloud project with billing, enable the BigQuery APIs, give Search Console’s export service account two roles, then switch on Bulk data export in Search Console settings. Tables appear within about 48 hours.

01 · CLOUD

Project and billing

Create or pick a Google Cloud project and attach billing. There is a free usage level, but storage and queries above it are charged. Set a budget alert now, not after the first bill.

02 · APIS

Enable BigQuery

Under APIs & Services, enable the BigQuery API and the BigQuery Storage API in that project.

03 · ACCESS

Authorise the export

Add search-console-data-export@system.gserviceaccount.com as a principal with the BigQuery Job User and BigQuery Data Editor roles. Removing it later stops the export.

04 · SEARCH CONSOLE

Settings → Bulk data export

As a property owner, enter the project ID, keep a dataset name that starts with searchconsole, and choose the location carefully. It cannot be changed afterwards.

05 · VERIFY

Wait, then inspect

The first export runs up to 48 hours after a successful setup and includes that day’s data. Look for the site table, the URL table and the export log.

06 · PROTECT

Leave the tables alone

If you set a partition expiration, keep it at 14 days or longer. Do not change the exported schema, for example by adding a column, because that makes the export fail. One property per destination dataset.

What you get

TableGrainPosition fieldUse it for
searchdata_site_impressionProperty, by day, query, country, device and search typesum_top_positionSite totals, brand split, query demand, Hidden Share.
searchdata_url_impressionURL, by day, query, country, device and search typesum_positionPage performance, query × URL work, cohorts.
ExportLogOne row per export runNoneFreshness and gap checks.
Three traps

No backfill: the export starts on the day you configure it, so pull whatever history the interface and API still hold into your own archive now. Gaps happen: two days in early 2026 never arrived for some properties. The raw tables belong to Google: build views on top, never edit them.

People often assume more history exists than does. In a small, self-selected LinkedIn poll I ran in 2023, 31% picked 16 months as the longest history you can get from the interface, 30% said 12 months, 9% said 36, and the remaining 30% thought it counted from the day the property was created. The poll proves nothing about the industry, but it matches the questions I still see. Check the real limit before you promise anyone a trend.

Who should not do this

Skip rung 4 if your question is one page or one query, or if nobody will own a Cloud project. The export is a commitment to billing, permissions, retention and monitoring. If that commitment is unwelcome, rung 3 with a daily archive covers more than most teams expect.

07Model

How do you build a Search Console dataset you can trust?

Never point a dashboard at the raw tables. Put validated views on top, add one URL key, and run three checks every day.

Raw Google tables→Validated views→One URL key→Business dimensions

The URL key

Most bad joins come from URL variants: query strings, trailing slashes, capitals, a migrated path. A single normalised key, defined once in a view, removes most of them.

BigQuery Standard SQL · view
CREATE OR REPLACE VIEW `your-project.seo.v_url_daily` AS
SELECT
  data_date,
  search_type,
  device,
  country,
  url,
  LOWER(REGEXP_REPLACE(REGEXP_REPLACE(url, r'[?#].*$', ''), r'/+$', '')) AS url_key,
  query,
  is_anonymized_query,
  clicks,
  impressions,
  sum_position
FROM `your-project.searchconsole.searchdata_url_impression`;
Lowercasing can merge paths that differ only by case, so check your platform before you rely on it. Keep the original url beside the key.

The minimum data dictionary

Field groupWhat to storeWhy it matters
Date and search typedata_date, search_typeLike-for-like comparisons. Web and Discover behave differently.
Query and privacyquery, is_anonymized_queryKeeps the hidden pool separate and visible.
Landing URLurl plus url_keyStable joins through replatforming and URL changes.
Audiencecountry, deviceMarket and device cohorts.
Numeratorsclicks, impressions, sum_positionRecalculate ratios from sums. Never average ratios across rows.
Page intelligencedirectory, template, owner, intent clusterTurns rows into something a team can act on.

Three checks to run every day

Freshness.The newest data_date in both tables matches what the export log says was published.
Volume.Yesterday’s impressions sit inside the range your own history sets. Alert outside it, and do not alert on days after the incomplete-data flag.
Completeness.No dates are missing in the last 60 days. Investigation 05 does exactly this.

Once a month, compare one complete week of property-level clicks in your table with the Performance report. Expect them to be close, and write down the tolerance you accept so the next analyst does not rediscover it.

Keep the bill small

  • Filter on data_date first. The tables are partitioned by day, so a date filter limits what BigQuery reads.
  • Select only the columns you need. LIMIT does not reduce the bytes scanned.
  • Run a dry run to see the bytes before you execute, and set a maximum bytes billed on shared queries.
  • Pre-aggregate anything a dashboard repeats into a small scheduled table, rather than recomputing raw rows on every view.
08Hands-on

12 SQL investigations for Search Console data in BigQuery.

Each one names the question, the table and grain, the query, how to read it, what to verify and what to do next. They are written for the export described above. Replace your-project and the brand pattern, and read the bytes estimate before you run anything.

Window convention

“Latest complete 28 days” means 30 days ago through 3 days ago in Pacific Time, because the newest days trail. Pin literal dates in any saved report so a number does not change when someone reopens it.

12 showing
01PerformanceBeginner

Which URLs earn the most search demand right now?

Table and grain: URL table · URL × day · all queries included

A clean baseline for the latest complete 28 days. Per-URL sums are valid; position is weighted by impressions.

Show the SQL, the checks and the next action
BigQuery Standard SQL
-- Latest complete 28 days, Pacific Time
SELECT
  url,
  SUM(clicks) AS clicks,
  SUM(impressions) AS impressions,
  ROUND(SAFE_DIVIDE(SUM(clicks), SUM(impressions)) * 100, 2) AS ctr_pct,
  ROUND(SAFE_DIVIDE(SUM(sum_position), SUM(impressions)) + 1, 1) AS avg_position
FROM `your-project.searchconsole.searchdata_url_impression`
WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY)
                    AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
  AND search_type = 'WEB'
GROUP BY url
HAVING SUM(impressions) >= 100
ORDER BY clicks DESC
LIMIT 100;
VerifyDo not add URL rows to get a site total. Use the site table for totals.
Next actionPick the five URLs that matter to the business and track them against this baseline.
02SegmentationBeginner

How much of my demand is branded?

Table and grain: Site table · query × day

Three groups: your brand pattern, everything else, and the hidden pool. The hidden pool stays separate on purpose.

Show the SQL, the checks and the next action
BigQuery Standard SQL
-- Replace the pattern with your own brand names and common misspellings
SELECT
  CASE
    WHEN is_anonymized_query THEN 'hidden (anonymised)'
    WHEN REGEXP_CONTAINS(LOWER(query), r'\b(acme|acme[\s-]?analytics)\b') THEN 'brand'
    ELSE 'non-brand'
  END AS query_group,
  SUM(clicks) AS clicks,
  SUM(impressions) AS impressions,
  ROUND(SAFE_DIVIDE(SUM(sum_top_position), SUM(impressions)) + 1, 1) AS avg_position
FROM `your-project.searchconsole.searchdata_site_impression`
WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY)
                    AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
  AND search_type = 'WEB'
GROUP BY query_group
ORDER BY impressions DESC;
VerifyRead twenty non-brand queries by hand for brand variants you missed. Google’s branded filter is a quick cross-check where you have it. My regex guide has tested brand patterns.
Next actionTrack non-brand growth, not total clicks, as your signal of new category discovery.
03Hidden ShareBeginner

How much of my demand can I never inspect?

Table and grain: Site table · month × device

Hidden Share by month and device. A high share means query-level findings describe only a minority of your demand.

Show the SQL, the checks and the next action
BigQuery Standard SQL
SELECT
  FORMAT_DATE('%Y-%m', data_date) AS month,
  device,
  SUM(impressions) AS impressions,
  SUM(IF(is_anonymized_query, impressions, 0)) AS hidden_impressions,
  ROUND(SAFE_DIVIDE(SUM(IF(is_anonymized_query, impressions, 0)), SUM(impressions)) * 100, 1) AS hidden_share_pct
FROM `your-project.searchconsole.searchdata_site_impression`
WHERE data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 90 DAY)
  AND search_type = 'WEB'
GROUP BY month, device
ORDER BY month, device;
VerifyCompare months only when the export covered every day of each. The current month is incomplete.
Next actionQuote Hidden Share beside any query-level chart, and lean on page-level evidence when it is high.
04OpportunityBeginner

Which queries sit just off page one?

Table and grain: URL table · query × URL

Query and URL pairs averaging positions 4 to 10 with real impression volume. Weighting removes the noise of averaging row positions.

Show the SQL, the checks and the next action
BigQuery Standard SQL
SELECT
  query,
  url,
  SUM(impressions) AS impressions,
  SUM(clicks) AS clicks,
  ROUND(SAFE_DIVIDE(SUM(sum_position), SUM(impressions)) + 1, 1) AS avg_position
FROM `your-project.searchconsole.searchdata_url_impression`
WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY)
                    AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
  AND search_type = 'WEB'
  AND query IS NOT NULL
GROUP BY query, url
HAVING SUM(impressions) >= 200
  AND SAFE_DIVIDE(SUM(sum_position), SUM(impressions)) + 1 BETWEEN 4 AND 10
ORDER BY impressions DESC
LIMIT 100;
VerifyAverage position blends every result position a page showed. It is not a rank. Check the live results before you act.
Next actionChoose pages where the query fits the page’s purpose, then improve the snippet, the direct answer and internal links.
05Export QABeginner

Did every day arrive?

Table and grain: Site table and ExportLog · day

Lists dates in the last 60 days with no rows, then shows what the export log says it published. This is the check that would have caught the early-2026 gap.

Show the SQL, the checks and the next action
BigQuery Standard SQL
-- 1) Dates in the last 60 days with no rows at all
WITH expected AS (
  SELECT day
  FROM UNNEST(GENERATE_DATE_ARRAY(
    DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 60 DAY),
    DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY))) AS day
),
seen AS (
  SELECT DISTINCT data_date
  FROM `your-project.searchconsole.searchdata_site_impression`
  WHERE data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 60 DAY)
)
SELECT expected.day AS missing_date
FROM expected
LEFT JOIN seen ON seen.data_date = expected.day
WHERE seen.data_date IS NULL
ORDER BY missing_date;

-- 2) What the export log says was published, and when
SELECT
  namespace,
  MAX(data_date) AS newest_data_date,
  MAX(publish_time) AS last_publish_time,
  COUNT(*) AS logged_exports
FROM `your-project.searchconsole.ExportLog`
GROUP BY namespace
ORDER BY namespace;
VerifyA day with no search impressions also returns no rows, so on very small properties cross-check the log. Run SELECT * FROM the export log with LIMIT 5 once to confirm the column names in your project.
Next actionAlert on any missing date before a dashboard refreshes, and record each gap in your data dictionary.
06DecayIntermediate

Which URLs lost the most clicks?

Table and grain: URL table · URL × two 28-day windows

Compares the latest complete 28 days with the 28 before. A minimum on prior clicks stops tiny pages from dominating the list.

Show the SQL, the checks and the next action
BigQuery Standard SQL
-- recent = 30 to 3 days ago, prior = 58 to 31 days ago (Pacific Time)
WITH w AS (
  SELECT
    url,
    SUM(IF(data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), clicks, 0)) AS recent_clicks,
    SUM(IF(data_date <  DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), clicks, 0)) AS prior_clicks,
    SUM(IF(data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), impressions, 0)) AS recent_impressions,
    SUM(IF(data_date <  DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), impressions, 0)) AS prior_impressions
  FROM `your-project.searchconsole.searchdata_url_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 58 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
    AND search_type = 'WEB'
  GROUP BY url
)
SELECT
  url,
  prior_clicks,
  recent_clicks,
  recent_clicks - prior_clicks AS click_delta,
  ROUND(SAFE_DIVIDE(recent_clicks - prior_clicks, prior_clicks) * 100, 1) AS click_change_pct,
  recent_impressions - prior_impressions AS impression_delta
FROM w
WHERE prior_clicks >= 20
ORDER BY click_delta ASC
LIMIT 50;
VerifyCheck seasonality, a launch or migration inside either window, and the incomplete-date flag before blaming a page.
Next actionOpen the top losers in the Performance report, compare their queries, then decide whether to refresh, merge or leave each one.
07OpportunityIntermediate

Which queries gained impressions without gaining clicks?

Table and grain: Site table · query × two 28-day windows

Demand grew by at least half while clicks stayed flat. That is visibility your page is not converting into visits.

Show the SQL, the checks and the next action
BigQuery Standard SQL
WITH w AS (
  SELECT
    query,
    SUM(IF(data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), impressions, 0)) AS recent_impressions,
    SUM(IF(data_date <  DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), impressions, 0)) AS prior_impressions,
    SUM(IF(data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), clicks, 0)) AS recent_clicks,
    SUM(IF(data_date <  DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), clicks, 0)) AS prior_clicks
  FROM `your-project.searchconsole.searchdata_site_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 58 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
    AND search_type = 'WEB'
    AND query IS NOT NULL
  GROUP BY query
)
SELECT
  query,
  prior_impressions,
  recent_impressions,
  recent_impressions - prior_impressions AS impression_gain,
  prior_clicks,
  recent_clicks
FROM w
WHERE prior_impressions >= 100
  AND recent_impressions >= prior_impressions * 1.5
  AND recent_clicks <= prior_clicks * 1.1
ORDER BY impression_gain DESC
LIMIT 50;
VerifyCheck the live results for features that answer the query without a click, and whether position fell as impressions rose.
Next actionRework the title, summary and first answer for the intent behind these queries.
08DiscoveryAdvanced

Which queries appeared for the first time?

Table and grain: Site table · query × recent window against the 180 days before it

Queries visible in the latest 28 days that were absent from the previous 180 days of the dataset.

Show the SQL, the checks and the next action
BigQuery Standard SQL
WITH recent AS (
  SELECT query, SUM(impressions) AS impressions, SUM(clicks) AS clicks
  FROM `your-project.searchconsole.searchdata_site_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
    AND search_type = 'WEB'
    AND query IS NOT NULL
  GROUP BY query
  HAVING SUM(impressions) >= 20
),
earlier AS (
  SELECT DISTINCT query
  FROM `your-project.searchconsole.searchdata_site_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 210 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 31 DAY)
    AND search_type = 'WEB'
    AND query IS NOT NULL
)
SELECT r.query, r.impressions, r.clicks
FROM recent AS r
LEFT JOIN earlier AS e ON e.query = r.query
WHERE e.query IS NULL
ORDER BY r.impressions DESC
LIMIT 100;
VerifyYour export only holds history from setup day. With under seven months of data, “new” means new to this dataset. A query can also be absent simply because it was hidden before.
Next actionGroup recurring themes into content briefs and ignore one-off oddities.
09OverlapAdvanced

Which queries are split across several of my URLs?

Table and grain: URL table · query × URL

Queries where at least two URLs each hold 20% or more of the impressions. These are overlap candidates. They are not proof of cannibalisation.

Show the SQL, the checks and the next action
BigQuery Standard SQL
WITH qu AS (
  SELECT query, url, SUM(impressions) AS impressions, SUM(clicks) AS clicks
  FROM `your-project.searchconsole.searchdata_url_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
    AND search_type = 'WEB'
    AND query IS NOT NULL
  GROUP BY query, url
  HAVING SUM(impressions) >= 20
),
shares AS (
  SELECT
    query,
    url,
    impressions,
    SUM(impressions) OVER (PARTITION BY query) AS query_impressions,
    SAFE_DIVIDE(impressions, SUM(impressions) OVER (PARTITION BY query)) AS impression_share
  FROM qu
),
agg AS (
  SELECT
    query,
    ANY_VALUE(query_impressions) AS query_impressions,
    COUNTIF(impression_share >= 0.2) AS urls_with_20pct_share,
    STRING_AGG(url, ' | ' ORDER BY impressions DESC LIMIT 3) AS top_urls
  FROM shares
  GROUP BY query
)
SELECT *
FROM agg
WHERE query_impressions >= 200
  AND urls_with_20pct_share >= 2
ORDER BY query_impressions DESC
LIMIT 50;
VerifySeveral URLs can serve different intents legitimately. Check the live results and whether the winning URL swaps from day to day.
Next actionWhere the intent is the same, consolidate or sharpen the page roles. Where it differs, leave the pages alone.
10SegmentationIntermediate

Which directories are gaining or losing share?

Table and grain: URL table · first path segment × two 28-day windows

The first path segment as a quick cohort, with change and share of recent clicks. For real work, replace the regex with your own URL map.

Show the SQL, the checks and the next action
BigQuery Standard SQL
WITH d AS (
  SELECT
    COALESCE(REGEXP_EXTRACT(url, r'^https?://[^/]+(/[^/?#]*)'), '/') AS directory,
    SUM(IF(data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), clicks, 0)) AS recent_clicks,
    SUM(IF(data_date <  DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), clicks, 0)) AS prior_clicks,
    SUM(IF(data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), impressions, 0)) AS recent_impressions,
    SUM(IF(data_date <  DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), impressions, 0)) AS prior_impressions
  FROM `your-project.searchconsole.searchdata_url_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 58 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
    AND search_type = 'WEB'
  GROUP BY directory
)
SELECT
  directory,
  prior_clicks,
  recent_clicks,
  ROUND(SAFE_DIVIDE(recent_clicks - prior_clicks, prior_clicks) * 100, 1) AS click_change_pct,
  ROUND(SAFE_DIVIDE(recent_clicks, SUM(recent_clicks) OVER ()) * 100, 1) AS share_of_recent_clicks,
  prior_impressions,
  recent_impressions
FROM d
WHERE prior_clicks + recent_clicks >= 20
ORDER BY recent_clicks DESC;
VerifyA migration or a new URL pattern moves pages between cohorts. Keep the URL map under version control.
Next actionShow each section owner their direction, then pick one lagging section to investigate.
11SegmentationIntermediate

Where does mobile trail desktop?

Table and grain: URL table · directory × device

CTR and weighted position side by side for each directory that has meaningful volume on both devices.

Show the SQL, the checks and the next action
BigQuery Standard SQL
WITH d AS (
  SELECT
    COALESCE(REGEXP_EXTRACT(url, r'^https?://[^/]+(/[^/?#]*)'), '/') AS directory,
    SUM(IF(device = 'MOBILE', impressions, 0)) AS mobile_impressions,
    SUM(IF(device = 'DESKTOP', impressions, 0)) AS desktop_impressions,
    SUM(IF(device = 'MOBILE', clicks, 0)) AS mobile_clicks,
    SUM(IF(device = 'DESKTOP', clicks, 0)) AS desktop_clicks,
    SUM(IF(device = 'MOBILE', sum_position, 0)) AS mobile_sum_position,
    SUM(IF(device = 'DESKTOP', sum_position, 0)) AS desktop_sum_position
  FROM `your-project.searchconsole.searchdata_url_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
    AND search_type = 'WEB'
  GROUP BY directory
)
SELECT
  directory,
  mobile_impressions,
  desktop_impressions,
  ROUND(SAFE_DIVIDE(mobile_clicks, mobile_impressions) * 100, 2) AS mobile_ctr_pct,
  ROUND(SAFE_DIVIDE(desktop_clicks, desktop_impressions) * 100, 2) AS desktop_ctr_pct,
  ROUND(SAFE_DIVIDE(mobile_sum_position, mobile_impressions) + 1, 1) AS mobile_position,
  ROUND(SAFE_DIVIDE(desktop_sum_position, desktop_impressions) + 1, 1) AS desktop_position
FROM d
WHERE mobile_impressions >= 500
  AND desktop_impressions >= 500
ORDER BY mobile_impressions DESC
LIMIT 50;
VerifyDevice mix and query mix differ, so a gap is a hint to test on a phone, not a verdict.
Next actionTest the mobile layout and answer position for the directories with the widest gap.
12Early warningAdvanced

Are my pages ranking for fewer distinct queries than before?

Table and grain: URL table · URL × query × two 28-day windows

Counts the distinct visible queries per URL in each window, with a fixed minimum so the threshold stays stable between windows.

Show the SQL, the checks and the next action
BigQuery Standard SQL
WITH qp AS (
  SELECT
    url,
    query,
    IF(data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY), 'recent', 'prior') AS period,
    SUM(impressions) AS impressions
  FROM `your-project.searchconsole.searchdata_url_impression`
  WHERE data_date BETWEEN DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 58 DAY)
                      AND DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 3 DAY)
    AND search_type = 'WEB'
    AND query IS NOT NULL
  GROUP BY url, query, period
  HAVING SUM(impressions) >= 5
),
b AS (
  SELECT
    url,
    COUNTIF(period = 'prior') AS prior_queries,
    COUNTIF(period = 'recent') AS recent_queries
  FROM qp
  GROUP BY url
)
SELECT url, prior_queries, recent_queries, recent_queries - prior_queries AS query_change
FROM b
WHERE prior_queries >= 10
ORDER BY query_change ASC
LIMIT 50;
VerifyAnonymisation and the impression threshold both bias the count. Compare only like-for-like windows.
Next actionReview pages whose query breadth contracted for intent drift, before traffic visibly falls.

Every query was syntax-checked in a BigQuery SQL parser. None has been run against a private property, so read the bytes estimate and test on a short date range first.

09Beyond Search Console

What changes when you join Search Console data with crawls, logs, GA4 and CRM?

You can ask why, not just what. Search Console says which pages earn demand. The other sources say whether those pages can be crawled, were fetched, held attention and led anywhere.

Write four things down before any join runs: the matching rule, the time window, who owns each source, and one limit you will state in the report.

+ Crawl data

Relate demand to status code, canonical, indexability, depth and template from a Screaming Frog crawl.

Join: url_key to the crawler URLCrawl depth is a measurement from your crawl, not a ranking factor.

+ Server logs

Compare verified Googlebot requests with search demand for each URL. My log file analysis guide covers the verification step first.

Join: url_key to the requested path on the same hostA fetch shows access. It does not show indexing or ranking.

+ GA4

Relate landing-page demand to engagement and key events once both sides use the same normalised URL.

Join: landing page to url_key, with aligned datesSearch Console clicks are not GA4 sessions. Expect the totals to differ.

+ CRM and sales

Connect landing pages to pipeline or revenue through a governed page-to-campaign model that someone owns.

Join: a maintained landing-page modelKeep modelled value separate from observed attribution.

+ Topic taxonomy

Group queries into families with reviewed n-grams or entity mapping to find coverage and intent gaps.

Join: query to a reviewed topic tableRead the taxonomy by hand before you trust it.

+ AI visibility research

Compare search demand with answers you sample yourself: the question, the date, the service and the cited URL. My SEO checklist shows how to record them.

Join: your saved citations to the landing URLNo Search Console field links a row to an AI citation.
Search Console dataset→Governed URL and topic map→Approved outside data→Reviewable SEO action

Two streams, search behaviour and technical or business evidence, meet in a card that names each source. Nothing is asserted that no source supports.

10Agentic workflows

Can AI agents query Search Console data safely?

Yes, inside a boundary: a read-only identity, approved views, a cost cap, and a person who reviews the evidence before anything changes.

It is the same principle as the rest of my work: AI learns to operate trusted SEO tools, and people decide what the evidence supports. My SEO checklist applies it to audits. Here it applies to your warehouse.

Human question→Read-only approved views→SQL plus validation→Reviewed action
Identity.A separate read-only service identity, never the one that owns the export.
Scope.Approved views only, with an allowlist of columns and a capped date window.
Cost.A maximum bytes billed on every agent query, and a dry run first.
Evidence.Every finding returns its SQL, the affected URLs, the window and the incomplete-date flag.
Review.Nothing ships on an agent’s say-so. A person verifies before a ticket exists.

Know what an agent cannot repair. It cannot recover a missing day, reveal an anonymised query, or pull generative AI report data that no documented API returns. Several community-built Search Console MCP servers exist. Treat each one as software that holds your credentials: read its source, give it read-only access to a single property, and revoke it when you stop using it.

A prompt worth testing on your own views
Compare the last two complete 28-day windows for /services/ using only the
approved views. Exclude dates after first_incomplete_date. Group by non-brand
query cohort and rank by impression decline. Return the exact SQL, the affected
URLs, two alternative explanations and the checks I must run before I escalate.
11Where this breaks

Debug the definition before you rewrite the strategy.

SymptomWhat to inspectNext action
Totals differ between interface, API and BigQueryGrain, search type, Pacific Time dates, anonymised rows.Re-run both at the same grain and window.
The last days look like a crashIncomplete data, first_incomplete_date, export lag.Exclude dates after the flag. Re-check in three days.
Query rows sum to less than the totalAnonymised queries.Measure Hidden Share (investigation 03). Do not spread it across queries.
Position looks too good or too badAveraged row positions, or a zero-based sum used as is.Use the weighted formula and add one.
The export stoppedRemoved service account, changed schema, short partition expiry, two properties on one dataset.Check the export status in Search Console settings, then IAM and the table schema.
The bill jumpedNo date filter, wide column lists, dashboards recomputing raw rows.Partition filter, pre-aggregated tables, a byte cap.
A join returns too many rowsURL variants, and many-to-many joins at query grain.Join on url_key after aggregating each side.
Everything looks like cannibalisationSeveral URLs per query is normal when intents differ.Check intent and live results before merging (investigation 09).
12Put it to work

Your first 30 days, and what to watch in 2027.

DAYS 1–7

Find your rung

Run one complete day through the API Explorer. Compare its totals with the interface. If you need history or joins, start the bulk export now.

DAYS 8–21

Make it trustworthy

Create the URL key view. Run investigations 01, 03 and 05. Write your grain, window and search type at the top of every saved query.

DAYS 22–30

Decide something

Run two or three of 04, 06 and 10. Write each as Question, Grain, Query, Decision with an owner. Schedule the daily freshness check.

Measure the plan by what it changes: time from question to answer, the share of findings you can trace back to a query, and the number of recommendations someone acted on. Row counts are not on that list.

Looking towards 2027

Keep the method. Recheck the moving parts.

  • Will Google add the generative AI performance data to the API or the bulk export?
  • Will branded and query-group classifications become something you can export and query?
  • Will the bulk export gain a backfill, or a way to repair a missing day?
  • Will an official Search Console MCP server arrive, and with what permissions model?
13Questions

Keep the limits visible.

Is the Search Console API still limited to 25,000 rows?

Each response is capped at 25,000 rows, and you page with startRow. Across all pages Google exposes up to 50,000 rows per day per search type, taken from the top by clicks, so pagination does not make the API unlimited.

Does BigQuery give me every search query?

No. It is a fuller daily dataset than the API’s top rows, but anonymised queries still have no query string. Investigation 03 measures how much is hidden on your property.

Can I get data from before I turned the export on?

Not automatically. The bulk export does not backfill. Pull whatever history the interface and the API still hold, about 16 months, into your own archive straight away.

Is the BigQuery export free?

Billing must be enabled on the Cloud project. Google describes a free usage level, and storage and query use above it is charged. Set a budget alert and filter on data_date in every query.

How do I calculate average position in BigQuery?

On the URL table use SUM(sum_position) / SUM(impressions) + 1, with SAFE_DIVIDE to guard against zero impressions. On the site table use sum_top_position in the same pattern.

Can I see AI Overviews or AI Mode data through the API or BigQuery?

Clicks and impressions from those features sit inside regular Web search data, with no filter to split them. The generative AI performance report shows impressions by page, country, device and date in the interface. At the time of writing I found no documented API or bulk-export access to it.

Do I need to know SQL or code?

No for rungs 1 and 2, since the interface and the API Explorer need none. SQL becomes useful at rung 4, and the 12 investigations are written to be copied, adjusted and read.

Can Search Console and GA4 numbers be joined?

You can compare landing-page cohorts once URLs are normalised and dates aligned. Search Console clicks and GA4 sessions measure different things, so they will not reconcile one to one.

Why do my dates look off by a day?

Search Console dates follow Pacific Time. Convert them before you compare with GA4, server logs or a business calendar.

Can AI agents run these queries for me?

Yes, inside a boundary: read-only access, approved views, a cost cap, and a person who reviews the evidence before anything changes.

Put it to work

Start with one rung and one question.

Pick the lowest rung that answers your question, write the grain at the top of the query, and run investigation 03 first so you know how much of your demand you can actually inspect.

Keep reading

All writing →

Questions about this guide?

Send me a note. I read every one.

Let’s connect ↗

1 thought on “Search Console data at scale: export it, query it, trust it”

Comments are closed.

Scroll to Top