Export Google SERP to spreadsheet

Table of Contents

Before You Start
The Automatic Way
The Alternative (Manual) Way

Before You Start #

While loading all the results you may see the following message. If you see it, be sure to click the link to refresh the page and get all the results.

Also, currently Google only displays up to 300 results. It won’t show all the pages that are indexed.

We experienced this issue before, and it looks like it is a common issue: https://support.google.com/websearch/thread/8663620/i-can-search-more-than-300-records?hl=en

The Automatic Way #

Go to Google and search for the site that you want to get all the URLs. Use the following format: site:mightyminnow.com (or whatever site that you want)
Scroll to the bottom. First, more results will automatically appear, but after 3 or 4 ajax loads, a “More results” button will appear.
Open your browser console and paste the following code:

				
					let $links = document.getElementById('rcnt').querySelectorAll('a');
let $button = document.querySelector( '#botstuff a[role="button"]' );
$button.click();
window.scrollTo(0, document.body.scrollHeight);

let interval = setInterval( function() {
	window.scrollTo(0, document.body.scrollHeight);
	$newLinks = document.getElementById( 'rcnt' ).querySelectorAll( 'a' );
	if ( $newLinks.length == $links.length ) {
		console.log( 'FINISHED' );
		clearInterval( interval );

		let str = "";
		for ( let $link of $links ) {
		  str += "\n" + $link.href;
		}

		console.log( str );
	} else {
		$links = $newLinks;
		$button.click();
	}
}, 3000 );

The script will automatically start clicking in the “More results” button every 3 seconds until all the results are loaded.
At the end, all the links will appear in your browser console. Now you can copy/paste those links in your spreadsheet.

The Alternative (Manual) Way #

Note: Originally, we had to click on the More results button again and again and again. The current script does this automatically, but in case that something goes wrong, refresh your page and just click on the More results all the needed times, and execute this original code:

				
					let str = "";
let links = document.getElementById('rcnt').querySelectorAll('a');

for ( let link of links ) {
  str += "\n" + link.href;
}

console.log( str );