Skip to content

fix: cache getAcceptsAddresses #431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fmhall wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix: cache getAcceptsAddresses #431

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
56dbdc7
cache getAcceptsAddresses
fmhall Jan 5, 2026
e180046
build recipient filter first
fmhall Jan 5, 2026
2eb749c
use MV for first txs
fmhall Jan 5, 2026
5df46dc
Merge branch 'main' into mason/cache-accepts-addresses
fmhall Jan 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion apps/scan/src/services/db/resources/accepts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { scanDb } from '@x402scan/scan-db';

import { mixedAddressSchema } from '@/lib/schemas';
import { createCachedQuery, createStandardCacheKey } from '@/lib/cache';

import type { Chain } from '@/types/chain';
import type { AcceptsNetwork, ResourceOrigin } from '@x402scan/scan-db';
Expand All @@ -10,7 +11,7 @@ interface GetAcceptsAddressesInput {
tags?: string[];
}

export const getAcceptsAddresses = async (input: GetAcceptsAddressesInput) => {
const getAcceptsAddressesUncached = async (input: GetAcceptsAddressesInput) => {
const { chain, tags } = input;
const accepts = await scanDb.accepts.findMany({
include: {
Expand Down Expand Up @@ -58,3 +59,16 @@ export const getAcceptsAddresses = async (input: GetAcceptsAddressesInput) => {
{} as Record<string, Array<ResourceOrigin>>
);
};

/**
* Get accepts addresses grouped by origin (cached)
* This is used to determine which addresses are "bazaar" sellers
*/
export const getAcceptsAddresses = createCachedQuery({
queryFn: getAcceptsAddressesUncached,
cacheKeyPrefix: 'accepts:addresses',
createCacheKey: input =>
createStandardCacheKey(input as Record<string, unknown>),
dateFields: [],
tags: ['accepts', 'bazaar'],
});
13 changes: 4 additions & 9 deletions apps/scan/src/services/transfers/sellers/stats/bucketed.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ const getBucketedSellerStatisticsUncached = async (
Math.floor(timeRangeMs / numBuckets / 1000)
);

// Use the recipient_first_transaction materialized view for fast lookups
// This avoids scanning all hypertable chunks to compute MIN(block_timestamp)
const sql = Prisma.sql`
WITH all_buckets AS (
SELECT generate_series(
Expand All @@ -46,13 +48,6 @@ const getBucketedSellerStatisticsUncached = async (
(${bucketSizeSeconds} || ' seconds')::interval
) AS bucket_start
),
seller_first_transactions AS (
SELECT
recipient,
MIN(block_timestamp) AS first_transaction_date
FROM "TransferEvent"
GROUP BY recipient
),
bucket_stats AS (
SELECT
to_timestamp(
Expand All @@ -61,14 +56,14 @@ const getBucketedSellerStatisticsUncached = async (
COUNT(DISTINCT t.recipient)::int AS total_sellers,
COUNT(DISTINCT CASE
WHEN to_timestamp(
floor(extract(epoch from sft.first_transaction_date) / ${bucketSizeSeconds}) * ${bucketSizeSeconds}
floor(extract(epoch from rft.first_transaction_date) / ${bucketSizeSeconds}) * ${bucketSizeSeconds}
) = to_timestamp(
floor(extract(epoch from t.block_timestamp) / ${bucketSizeSeconds}) * ${bucketSizeSeconds}
)
THEN t.recipient
END)::int AS new_sellers
FROM "TransferEvent" t
LEFT JOIN seller_first_transactions sft ON t.recipient = sft.recipient
LEFT JOIN recipient_first_transaction rft ON t.recipient = rft.recipient
Copy link
Contributor

@vercel vercel bot Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code references a materialized view recipient_first_transaction that doesn't exist in the database, causing this query to fail at runtime.

View Details
📝 Patch Details 
diff --git a/packages/internal/databases/transfers/prisma/migrations/20260105182134_add_recipient_first_transaction_mv/migration.sql b/packages/internal/databases/transfers/prisma/migrations/20260105182134_add_recipient_first_transaction_mv/migration.sql
new file mode 100644
index 00000000..a22f78d6
--- /dev/null
+++ b/packages/internal/databases/transfers/prisma/migrations/20260105182134_add_recipient_first_transaction_mv/migration.sql
@@ -0,0 +1,11 @@
+-- Create materialized view for first transaction date by recipient
+CREATE MATERIALIZED VIEW recipient_first_transaction AS
+SELECT 
+  recipient,
+  MIN(block_timestamp) AS first_transaction_date
+FROM "TransferEvent"
+GROUP BY recipient;
+
+-- Create index for fast lookups
+CREATE UNIQUE INDEX recipient_first_transaction_idx
+ON recipient_first_transaction (recipient);

Analysis

Missing materialized view recipient_first_transaction referenced in seller statistics queries

What fails: The seller statistics functions getBucketedSellerStatistics() in apps/scan/src/services/transfers/sellers/stats/bucketed.ts (line 66) and getOverallSellerStatistics() in apps/scan/src/services/transfers/sellers/stats/overall.ts (line 33) reference a materialized view recipient_first_transaction that does not exist in the database, causing runtime SQL errors.

How to reproduce:

1. Call getBucketedSellerStatistics() with any valid input
2. The queryRaw() function executes the SQL query against the transfers database
3. PostgreSQL returns: relation "recipient_first_transaction" does not exist

Result: Runtime error - PostgreSQL cannot execute the query due to missing materialized view

Expected: The materialized view should exist and contain recipient addresses with their first transaction timestamps

Root cause: Commit 2eb749c5 (use MV for first txs) refactored seller statistics from using inline CTE seller_first_transactions to a materialized view recipient_first_transaction, but the corresponding database migration was never created.

Fix: Created migration packages/internal/databases/transfers/prisma/migrations/20260105182134_add_recipient_first_transaction_mv/migration.sql that creates the missing materialized view with proper indexing for query performance.

${transfersWhereClause(input)}
GROUP BY bucket_start
)
Expand Down
18 changes: 7 additions & 11 deletions apps/scan/src/services/transfers/sellers/stats/overall.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,28 +13,24 @@ const getOverallSellerStatisticsUncached = async (
input: z.infer<typeof sellerStatisticsInputSchema>
) => {
const { startDate, endDate } = getTimeRangeFromTimeframe(input.timeframe);

// Use the recipient_first_transaction materialized view for fast lookups
// This avoids scanning all hypertable chunks to compute MIN(block_timestamp)
const sql = Prisma.sql`
WITH seller_first_transactions AS (
SELECT
recipient,
MIN(block_timestamp) AS first_transaction_date
FROM "TransferEvent"
GROUP BY recipient
),
filtered_transfers AS (
WITH filtered_transfers AS (
SELECT DISTINCT t.recipient
FROM "TransferEvent" t
${transfersWhereClause(input)}
)
SELECT
COUNT(DISTINCT ft.recipient)::int AS total_sellers,
COUNT(DISTINCT CASE
WHEN sft.first_transaction_date >= ${startDate ?? Prisma.sql`'1970-01-01'::timestamp`}
AND sft.first_transaction_date <= ${endDate ?? Prisma.sql`NOW()`}
WHEN rft.first_transaction_date >= ${startDate ?? Prisma.sql`'1970-01-01'::timestamp`}
AND rft.first_transaction_date <= ${endDate ?? Prisma.sql`NOW()`}
THEN ft.recipient
END)::int AS new_sellers
FROM filtered_transfers ft
LEFT JOIN seller_first_transactions sft ON ft.recipient = sft.recipient
LEFT JOIN recipient_first_transaction rft ON ft.recipient = rft.recipient
Copy link
Contributor

@vercel vercel bot Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code references a materialized view recipient_first_transaction that doesn't exist in the database, causing this query to fail at runtime.

View Details
📝 Patch Details 
diff --git a/packages/internal/databases/transfers/prisma/migrations/20260105000000_recipient_first_transaction/migration.sql b/packages/internal/databases/transfers/prisma/migrations/20260105000000_recipient_first_transaction/migration.sql
new file mode 100644
index 00000000..45829f39
--- /dev/null
+++ b/packages/internal/databases/transfers/prisma/migrations/20260105000000_recipient_first_transaction/migration.sql
@@ -0,0 +1,11 @@
+-- CreateMaterializedView
+CREATE MATERIALIZED VIEW recipient_first_transaction AS
+SELECT 
+  recipient,
+  MIN(block_timestamp) AS first_transaction_date
+FROM "TransferEvent"
+GROUP BY recipient;
+
+-- CreateIndex
+CREATE UNIQUE INDEX recipient_first_transaction_idx
+ON recipient_first_transaction (recipient);

Analysis

Missing materialized view recipient_first_transaction causes SQL runtime error

What fails: Queries in getOverallSellerStatistics() and getBucketedSellerStatistics() fail at runtime when executing the SQL that references the undefined materialized view recipient_first_transaction.

How to reproduce:

  1. Call the /trpc/sellers.getOverallSellerStatistics endpoint (or getOverallSellerStatistics() directly)
  2. The query attempts to execute a LEFT JOIN against recipient_first_transaction

Result: PostgreSQL throws error: relation "recipient_first_transaction" does not exist

Expected behavior: The materialized view should exist and be queryable. This view was referenced in the refactoring commit (2eb749c) but was never created in a migration.

Root cause: Commit 2eb749c5 ("use MV for first txs") refactored the code to replace the inline CTE seller_first_transactions with a LEFT JOIN to recipient_first_transaction, but the corresponding migration file to create this materialized view was never added.

Fix: Added migration 20260105000000_recipient_first_transaction that creates the materialized view with the schema expected by the queries:

CREATE MATERIALIZED VIEW recipient_first_transaction AS
SELECT 
  recipient,
  MIN(block_timestamp) AS first_transaction_date
FROM "TransferEvent"
GROUP BY recipient;

This allows the statistics queries to efficiently look up each recipient's first transaction date without scanning the entire hypertable for every query.

`;

const result = await queryRaw(
Expand Down