Gbyte blog: Index Drupal 9 pages with IndexNow using Simple XML sitemap

Drupal CMS

Bing 410 Gone

Microsoft recently killed their search engine’s public API responsible for accepting sitemap ping requests (those where you let search engines know your XML sitemap’s content has changed). They did so completely unannounced leading to logs filling up and users unsurprisingly assuming their sitemap submitting code was somehow at fault. It wasn’t.

It became apparent that this was a step for Microsoft towards switching to the IndexNow protocol instead.

IndexNow supplements XML sitemaps

With IndexNow you now can quickly notify all participating search engines (Bing, Yandex) about a change that happened on your page eliminating the need to wait for them to come and scan your sitemap. The benefits of this approach are

Instant submission of changes including creating, deleting and updating of content
An alleged greener approach to indexing content as sitemap scanning requests get deprioritized
Only one search engine needs to be notified and it will notify the others for you

Still, this new approach is more of a supplement than a revolution:

Can I submit all URLs for my site?

Use IndexNow to submit only URLs having changed (added, updated, or deleted) recently, including all URLs if all URLs have been changed recently. Use sitemaps to inform search engines about all your URLs. Search engines will visit sitemaps every few days.