April 2018 Newsletter
Summary
Two webinars coming in May - an intro session and one about how search engines work. Read tips about avoiding spider traps in your site.
Two Webinars in May
Come learn more about search engines, and how to make the best use of Search.gov.
May 3 (1pm-2pm ET): An Introduction to Search.gov
New to our service or want a refresher? Join us for a general tour of the Admin Center and an overview of search optimization good practices. Register now
May 10 (1pm-2pm ET): How Search Engines Index Your Websites
A more advanced session looking at how search engines work, and the role of xml sitemaps and robots.txt files. If you are new, we recommend you attend the Introduction session first. Register now
Recommended Reading
Is your website a spider trap?
Sometimes dynamic link development can cause search engine spiders to get caught in an infinite loop of links inside your website. For instance, if a page has a ‘real’ URL, but can be accessed through a number of other links with different query parameters, each one of these links will look new to the spider and it will keep crawling and crawling and crawling. Allowing search engines to get around your site easily is a ranking factor. Read more
- Field Guide to Spider Traps
- Avoid the SEO Spider Trap
- Fixing Spider Traps to Improve Crawl Efficiency
Note: links to articles are not an endorsement of the website.
XML Sitemaps and Robots.txt files: Quick Tips
- XML Sitemaps advertise to search engines what content is on your website.
Tip: Be sure the URLs in your sitemap match the protocol of the sitemap itself and the pages on the list
Robots.txt files help search engines find your sitemaps and indicate what you do and do not want indexed.
Tip: Be sure that the sitemap for a given subdomain is listed on the robots.txt file for that subdomain.
Release Notes
Want to learn about the latest features, fixes, and focuses of the Search team? We post monthly Release Notes on our website.