Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DatastreamToSpanner] Spanner Exception handling #2185

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

darshan-sj
Copy link
Contributor

@darshan-sj darshan-sj commented Feb 13, 2025

Cagtegorize Spanner Exceptions into Retryable and Permanent Errors.

Retryable errors will be retried through DLQ, Permanent Errors will not be retried and put into severe/ folder of DLQ.

  • SpannerMigrationException with ErrorCodes defined in spanner-common module.
  • SpannerExceptionParser (in spanner-common) parses a SpannerException and creates a SpannerMigrationException with appropriate ErrorCode.
  • SpannerExceptionClassifier (in datastream-to-spanner) classifies a SpannerMigrationException into Retryable or Permanent error based on the SpannerMigrationException.ErrorCode

Copy link

codecov bot commented Feb 13, 2025

Codecov Report

Attention: Patch coverage is 92.70073% with 10 lines in your changes missing coverage. Please review.

Project coverage is 47.18%. Comparing base (40141f5) to head (7ee6bef).
Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
.../migrations/exceptions/SpannerExceptionParser.java 88.88% 2 Missing and 4 partials ⚠️
...ort/v2/templates/SpannerTransactionWriterDoFn.java 70.00% 1 Missing and 2 partials ⚠️
.../templates/spanner/SpannerExceptionClassifier.java 88.88% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2185      +/-   ##
============================================
+ Coverage     46.96%   47.18%   +0.21%     
- Complexity     4047     4412     +365     
============================================
  Files           876      879       +3     
  Lines         52194    52366     +172     
  Branches       5502     5529      +27     
============================================
+ Hits          24515    24709     +194     
+ Misses        25921    25893      -28     
- Partials       1758     1764       +6     
Components Coverage Δ
spanner-templates 69.05% <90.74%> (+0.14%) ⬆️
spanner-import-export 65.72% <ø> (+0.01%) ⬆️
spanner-live-forward-migration 76.86% <90.74%> (+0.35%) ⬆️
spanner-live-reverse-replication 79.06% <93.25%> (+0.28%) ⬆️
spanner-bulk-migration 88.01% <93.25%> (+0.07%) ⬆️
Files with missing lines Coverage Δ
...he/beam/it/gcp/spanner/SpannerResourceManager.java 80.95% <100.00%> (+4.48%) ⬆️
...grations/exceptions/SpannerMigrationException.java 100.00% <100.00%> (ø)
.../templates/spanner/SpannerExceptionClassifier.java 88.88% <88.88%> (ø)
...ort/v2/templates/SpannerTransactionWriterDoFn.java 51.41% <70.00%> (+1.16%) ⬆️
.../migrations/exceptions/SpannerExceptionParser.java 88.88% <88.88%> (ø)

... and 11 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant