May 2018 Newsletter
Summary
Twitter announced changes to their API, which will affect our ability to index tweets. Also, join us for a technical SEO webinar May 29!
Changes coming to Twitter’s API
Twitter may be limiting access to their API this summer
Many of you have listed Twitter handles in your search sites, in order to show relevant tweets in your results pages. Twitter announced they will be deprecating the API we use to index your tweets, and replace it with two APIs: a very limited API and an unlimited "enterprise" API. The new APIs have been reported as unfinished, and there is no pricing published for the enterprise API.
The deprecation of the current API was originally set for June 19, but has been postponed for an unspecified period of time.
What’s the impact on your search?
We currently index close to two thousand twitter accounts. We will investigate options, but the likely outcome is that when they deprecate their current API we will no longer show Tweets in our search results. There would be no action for you to take. If you have strong feelings about this potential change, please reach out to us and we can discuss.
Technical SEO Webinar May 30
Join us to learn how search engines index your sites
Our upcoming technical webinar on how search engines index your websites will now be held on May 29, at 1pm ET. Learn about how crawlers take in content, and how they interact with XML sitemaps and robots.txt files.
Did you miss the Intro session a couple weeks ago? Watch the recording (60 minutes, YouTube)</td>
SEO Pro Tip
Use the robots meta tag to manage* indexing of individual pages
Many websites have topic pages that list all the items they have for that particular area, such as https://search.gov/tagged/indexing/. These pages are helpful to crawlers, which will use the links on them to discover new pages to add to the search index. And while it’s great to include those new pages in the search results, the topic page itself is not very helpful as a search result.
To block indexing of these topic pages, place a noindex robots meta tag in the <head> of the page. On the Search.gov topic pages, we want crawlers not to index the page, but still follow the links on it, so our robots meta tag looks like this:
`<meta name="robots" content="noindex, follow">`
Release Notes
Want to learn about the latest features, fixes, and focuses of the Search team? We post monthly Release Notes on our website.