Keeping GoogleBot Happy

One of the interesting side effects I hadn’t considered when we rolled out some fairly significant caching updates on GitHub in the beginning of January was how much Google’s crawler would take full advantage of the speed increase.

The graph lays it all out pretty clearly what happens when your site is more responsive. The number of pages the bot is able to index goes up dramatically when the time spent downloading the page is reduced dramatically. More pages indexed on Google inevitably means more revenue for us as our traffic grows, so this is a solid win beyond making sure existing GitHubbers are happy.

Have feedback on this post? Let @github know on Twitter.
Need help or found a bug? Contact us.

Changelog

Subscribe

GitHub Universe logo

GitHub Universe

October 16-17 in San Francisco
Get tickets today

Discover new ways to build better

Try Marketplace apps free for 14 days

Learn more