Skip to content

Sink for BigQuery #86

@Ifechukwudaniel

Description

@Ifechukwudaniel

This issue tracks the implementation of a Sink that writes data to Google BigQuery.

The goal is to allow the SDK to persist decoded on-chain data directly into BigQuery tables for analysis, indexing, and integration with other Google Cloud tools.

✅ Tasks

  • Implement Sink interface for BigQuery
  • Support dynamic schema generation based on table.Columns
  • Support Flush() that inserts rows to BigQuery using streaming inserts
  • Handle dataset/table creation if they don’t exist
  • Ensure batching is efficient and avoids quota limits
  • Add configuration options:
    • projectID
    • datasetID
    • tablePrefix or tableName
  • Add authentication support using Application Default Credentials or service account
  • Write unit tests for the sink
  • Write integration test using test project or emulator (if available)
  • Add usage example in documentation

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions