Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚡️ Speed up method Timer.finish by 22% in PR #45125 (tolik0/airbyte-cdk/add-per-partition-with-global-fallback) #45655

Open
wants to merge 1 commit into
base: tolik0/airbyte-cdk/add-per-partition-with-global-fallback
Choose a base branch
from

Conversation

codeflash-ai[bot]
Copy link

@codeflash-ai codeflash-ai bot commented Sep 18, 2024

⚡️ This pull request contains optimizations for PR #45125

If you approve this dependent PR, these changes will be merged into the original PR branch tolik0/airbyte-cdk/add-per-partition-with-global-fallback.

This PR will be automatically closed if the original PR is merged.


📄 Timer.finish() in airbyte-cdk/python/airbyte_cdk/sources/declarative/incremental/global_substream_cursor.py

📈 Performance improved by 22% (0.22x faster)

⏱️ Runtime went down from 1.35 millisecond to 1.11 millisecond

Explanation and details

Here's an optimized version of the program.

Changes made.

  1. Removed unnecessary division and __ceil__() call by using integer arithmetic to achieve the ceiling division.
  2. Used the None type directly to check if _start is set, which is slightly faster.

Correctness verification

The new optimized code was tested for correctness. The results are listed below.

🔘 (none found) − ⚙️ Existing Unit Tests

✅ 1018 Passed − 🌀 Generated Regression Tests

(click to show generated tests)
# imports
import time
# function to test
from typing import Optional

import pytest  # used for our unit tests
from airbyte_cdk.sources.declarative.incremental.global_substream_cursor import \
    Timer

# unit tests

def test_timer_started_and_stopped_normally_short_duration():
    """Test that the timer correctly calculates elapsed time for a short duration."""
    timer = Timer()
    timer._start = time.perf_counter_ns()
    time.sleep(1)
    codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation

def test_timer_started_and_stopped_normally_long_duration():
    """Test that the timer correctly calculates elapsed time for a long duration."""
    timer = Timer()
    timer._start = time.perf_counter_ns()
    time.sleep(3)
    codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation

def test_timer_not_started():
    """Test that the function raises a RuntimeError if the timer is not started."""
    timer = Timer()
    with pytest.raises(RuntimeError, match="Global substream cursor timer not started"):
        timer.finish()
    # Outputs were verified to be equal to the original implementation

def test_elapsed_time_just_below_one_second():
    """Test that the timer rounds up just below one second to 1 second."""
    timer = Timer()
    timer._start = time.perf_counter_ns()
    time.sleep(0.999)
    codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation

def test_elapsed_time_exactly_one_second():
    """Test that the timer correctly calculates exactly one second."""
    timer = Timer()
    timer._start = time.perf_counter_ns()
    time.sleep(1)
    codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation

def test_elapsed_time_just_above_one_second():
    """Test that the timer rounds up just above one second to 2 seconds."""
    timer = Timer()
    timer._start = time.perf_counter_ns()
    time.sleep(1.001)
    codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation


def test_high_frequency_start_and_stop():
    """Test the timer's stability and correctness under high frequency start and stop."""
    timer = Timer()
    for _ in range(1000):
        timer._start = time.perf_counter_ns()
        time.sleep(0.001)
        codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation

def test_high_precision_timing():
    """Test the timer's precision for very short durations."""
    timer = Timer()
    timer._start = time.perf_counter_ns()
    time.sleep(1e-6)  # 1 microsecond
    codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation

def test_consistent_results():
    """Test that the timer returns consistent results for the same duration."""
    timer = Timer()
    timer._start = time.perf_counter_ns()
    time.sleep(2)
    codeflash_output = timer.finish()
    for _ in range(10):
        codeflash_output = timer.finish()
    # Outputs were verified to be equal to the original implementation

🔘 (none found) − ⏪ Replay Tests

…e-cdk/add-per-partition-with-global-fallback`)

Here's an optimized version of the program.



Changes made.
1. Removed unnecessary division and `__ceil__()` call by using integer arithmetic to achieve the ceiling division.
2. Used the `None` type directly to check if `_start` is set, which is slightly faster.
@codeflash-ai codeflash-ai bot added the ⚡️ codeflash Optimization PR opened by Codeflash AI label Sep 18, 2024
Copy link

vercel bot commented Sep 18, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Sep 18, 2024 4:05pm

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@octavia-squidington-iii octavia-squidington-iii added CDK Connector Development Kit community labels Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit ⚡️ codeflash Optimization PR opened by Codeflash AI community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants