Unexpected Error at Ponder Runtime API Endpoint

[koo-one]

Hello,

I'm building a custom subgraph using Ponderlib that listens to a forked chain and communicates with two external endpoints:

1. The official Aave V3 subgraph- ethereum mainnet, (which start with https://gateway.thegraph.com/api....)
2. Morpho's API (https://blue-api.morpho.org/graphql)

Background

• When querying Aave’s subgraph, I occasionally encounter getaddrinfo ENOTFOUND gateway.thegraph.com. I reached out to Aave, but haven't received a clear response regarding rate limits or potential causes.
• When querying Morpho’s API, I ran into 502 errors. After contacting Morpho, they mentioned a rate limit of 5k requests per 5 minutes. However, they also noted that if the request truly hit the rate limit, I should see a 403 error instead of a 502.
• The Graph has not yet provided a definitive response regarding these DNS errors or additional constraints beyond the known 100k free monthly query limit.
• Both scenarios involve sending approximately 60 requests per minute, and errors have been observed at various cumulative counts (e.g., around 5000, 8000 requests, etc.).

Given Morpho’s clearly stated rate limits—and their clarification that a rate limit error would return a 403—I suspect their endpoint is not the core issue. Likewise, Aave hasn't confirmed any constraints besides the monthly free limit on The Graph.

Question / Issue

• I’m beginning to think these errors might be related to The Graph infrastructure or potentially Ponderlib itself (e.g., how it batches or times queries).
• Has anyone experienced similar issues with Ponderlib or The Graph returning DNS-related errors (ENOTFOUND) or unexpected 502 responses under query loads?
• Are there known best practices or configurations within Ponderlib to avoid hitting potential limits or causing unexpected DNS failures?

Any advice, guidance, or references to documentation would be greatly appreciated!

Thank you in advance.

[typedarray]

It looks like you're trying to fetch data from external sources from within indexing functions, which we generally recommend against. Ponder works best when indexing functions are simple; only transforming decoded EVM logs/transactions/blocks and doing RPC requests via context.client. As you're seeing, when you try to query external data sources, things often go wrong, and the core indexing process suffers.

Two possible directions here:

1. Fetch the external data on-the-fly in API functions. This way, the core indexer won't crash if the external sources fail.
2. Use the "offchain tables" pattern, where a separate service/system takes care of fetching data from external sources and writing them to the database, then you can query/join across ponder-managed tables and the external "offchain" data tables. Basic example.

The errors you're seeing with these external sources have nothing to do with Ponder. From your error logs, it looks like you're making requests using a normal Node.js HTTP client - Ponder is "unaware" that you're doing this and will not interfere. So, I can't help you with those errors - I'm confident you'd have the same issues if you queried these endpoints from any Node.js app.