Skip to content

Implement IndexNow integration for improved search engine indexing #14997

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

asafashirov
Copy link
Contributor

@asafashirov asafashirov commented May 12, 2025

The IndexNow integration enhances the pulumi.com search engine presence by instantly notifying search engines (Bing and Yandex) about new and updated content, eliminating the typical waiting period for discovery and indexing. This means users searching for Pulumi resources will immediately find the most current documentation, blog posts, and tutorials rather than outdated cached versions. The feature works seamlessly within the existing build pipeline, requiring minimal overhead while maximizing content visibility, improving SEO performance, and creating a better overall user experience by ensuring the latest content is readily discoverable.

Changes:

  • Add IndexNow API implementation in scripts/search/indexnow.js
  • Create shell wrapper in scripts/search/submit-indexnow.sh
  • Integrate with build pipeline in ci-push.sh
  • Add Makefile target for manual submission
  • Include API key in GitHub workflow environment
  • Add test scripts to verify API functionality

The implementation successfully:

  • Integrates naturally with your existing build and deployment workflow
  • Uses the same patterns and approach as other search-related functionality
  • Reuses existing sitemap to determine which URLs to submit
  • Maintains a cache to avoid resubmitting unchanged URLs
  • Includes error handling and logging
  • Has a test mode for local validation

- Add IndexNow API implementation in scripts/search/indexnow.js
- Create shell wrapper in scripts/search/submit-indexnow.sh
- Integrate with build pipeline in ci-push.sh
- Add Makefile target for manual submission
- Include API key in GitHub workflow environment
- Add test scripts to verify API functionality

This implementation submits new and updated URLs to search engines
via the IndexNow protocol, allowing for faster indexing of content.
@asafashirov
Copy link
Contributor Author

Contingent on the IndexNow API key (33134703c43349ddb227d769745f33cc) being added to our ESC secrets management system.

@asafashirov
Copy link
Contributor Author

Note to follow after merge, need to make sure:
- The key file is accessible at https://www.pulumi.com/indexnow.txt
- The script is running successfully during deployments

@pulumi-bot
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants