Add deferred pagination mode to GenericTransfer by dabla · Pull Request #44809 · apache/airflow

Commits on Jan 2, 2025

refactor: Refactored the GenericTransfer operator to support paginated reads (in deferred mode) and introduce a SQLExecuteQueryTrigger

davidblain-infrabel
authored and
potiuk
committed
refactor: updated provider dependencies

dabla
authored and
potiuk
committed
refactor: Added TestSQLExecuteQueryTrigger and moved test code which allows you to run deferrable operators in test common test utils

davidblain-infrabel
authored and
potiuk
committed
refactor: Fixed static checks

davidblain-infrabel
authored and
potiuk
committed
refactor: Fixed static checks

davidblain-infrabel
authored and
potiuk
committed
refactor: Fixed static checks

davidblain-infrabel
authored and
potiuk
committed
refactor: Reformatted GenericTransfer

davidblain-infrabel
authored and
potiuk
committed
refactor: Moved source and destination hooks into cached properties

davidblain-infrabel
authored and
potiuk
committed
refactor: Moved imports to type checking block

davidblain-infrabel
authored and
potiuk
committed
refactor: Fixed execute method of GenericTransfer

davidblain-infrabel
authored and
potiuk
committed
refactor: Refactored get_hook method of GenericTransfer which checks if hook is instance of DbApiHook instead of checking presence of get_records and insert_rows method

davidblain-infrabel
authored and
potiuk
committed
refactor: Remove white lines from mock_context

davidblain-infrabel
authored and
potiuk
committed
refactor: Reformatted get_hook in GenericTransfer operator

davidblain-infrabel
authored and
potiuk
committed
refactor: Added sql.pyi for SQLExecuteQueryTrigger

davidblain-infrabel
authored and
potiuk
committed
refactor: Reformatted SQLExecuteQueryTrigger definition

davidblain-infrabel
authored and
potiuk
committed
refactor: Added alias in SQLExecuteQueryTrigger definition

davidblain-infrabel
authored and
potiuk
committed
Merge branch 'main' into feature/paginated-generic-transfer
dabla
authored
Merge branch 'main' into feature/paginated-generic-transfer
dabla
authored

Commits on Jan 7, 2025

Commits on Jan 8, 2025

Commits on Jan 9, 2025

Merge branch 'main' into feature/paginated-generic-transfer
dabla
authored

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add deferred pagination mode to GenericTransfer#44809

Commits on Jan 2, 2025

refactor: Refactored the GenericTransfer operator to support paginated reads (in deferred mode) and introduce a SQLExecuteQueryTrigger

refactor: updated provider dependencies

refactor: Added TestSQLExecuteQueryTrigger and moved test code which allows you to run deferrable operators in test common test utils

refactor: Fixed static checks

refactor: Fixed static checks

refactor: Fixed static checks

refactor: Reformatted GenericTransfer

refactor: Moved source and destination hooks into cached properties

refactor: Moved imports to type checking block

refactor: Fixed execute method of GenericTransfer

refactor: Refactored get_hook method of GenericTransfer which checks if hook is instance of DbApiHook instead of checking presence of get_records and insert_rows method

refactor: Remove white lines from mock_context

refactor: Reformatted get_hook in GenericTransfer operator

refactor: Added sql.pyi for SQLExecuteQueryTrigger

refactor: Reformatted SQLExecuteQueryTrigger definition

refactor: Added alias in SQLExecuteQueryTrigger definition

Merge branch 'main' into feature/paginated-generic-transfer

Merge branch 'main' into feature/paginated-generic-transfer

Commits on Jan 7, 2025

Merge branch 'main' into feature/paginated-generic-transfer

refactor: Added unit test for GenericTransfer using deferred pageable reads

Merge branch 'main' into feature/paginated-generic-transfer

Commits on Jan 8, 2025

Merge branch 'main' into feature/paginated-generic-transfer

refactor: Moved generic_transfer operator definition in provider.yaml from standard provider to common-sql provider and removed dependency of common-sql provider in standard provider as normally th…

refactor: Renamed typo of module which allows you to run deferrable operators within unit tests

refactor: Reformatted xcom_pull method from mock_context

Commits on Jan 9, 2025

Merge branch 'main' into feature/paginated-generic-transfer