Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] Fix parallelism deriving heuristic to ensure parallelism stays w/in min/max bounds #47695

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open

[Data] Fix parallelism deriving heuristic to ensure parallelism stays w/in min/max bounds #47695

wants to merge 23 commits into from

Commits on Oct 16, 2024

  1. Making block-size configs' defaults env-var configurable 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    2a73848 View commit details
    Browse the repository at this point in the history

  2. Bumping min-block-size default to 16Mb 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    d79e54e View commit details
    Browse the repository at this point in the history

  3. Fixing parallelism auto-detection heuristic to respect min/max bounds… 

    … for parallelism based on target block-sizes

Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7145ff8 View commit details
    Browse the repository at this point in the history

  4. lint 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    17e3273 View commit details
    Browse the repository at this point in the history

  5. Tidying up telemetry 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    4a55419 View commit details
    Browse the repository at this point in the history

  6. Updating tests 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    bdf8017 View commit details
    Browse the repository at this point in the history

  7. Tidying up further 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    f3fa1b3 View commit details
    Browse the repository at this point in the history

  8. Updated more tests 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    d51f40d View commit details
    Browse the repository at this point in the history

  9. Fixing more tests 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    6d37dda View commit details
    Browse the repository at this point in the history

  10. Tidying up 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    15edd4e View commit details
    Browse the repository at this point in the history

  11. Fixed stats test 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    aa11d73 View commit details
    Browse the repository at this point in the history

  12. Fixing more tests 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    ef3a8e6 View commit details
    Browse the repository at this point in the history

  13. lint 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7cf4ced View commit details
    Browse the repository at this point in the history

  14. Fixed NPE 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    da5d0ae View commit details
    Browse the repository at this point in the history

  15. Fixing test; 

    Updating fixtures

Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    6989bd0 View commit details
    Browse the repository at this point in the history

  16. Fixing string interpolation 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    3e17e4a View commit details
    Browse the repository at this point in the history

  17. Updated pydoc 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    79f6b05 View commit details
    Browse the repository at this point in the history

  18. Made kwargs explicit; 

    Fixed incorrect usage of `_autodetect_parallelism`;

Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7b9f43e View commit details
    Browse the repository at this point in the history

  19. Assert parallelism boundaries are overlapping; 

    Tidying up

Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    cdf2246 View commit details
    Browse the repository at this point in the history

  20. Fixing more tests 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    8f30820 View commit details
    Browse the repository at this point in the history

  21. Fixing some more tests 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    3d7ebf4 View commit details
    Browse the repository at this point in the history

  22. Override target_min_block_size when target_max_block_size < `targ… 

    …et_min_block_size`

Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    a3be14f View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Fixing even more tests 

    Signed-off-by: Alexey Kudinkin <[email protected]>
    @alexeykudinkin
    alexeykudinkin committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    eaab6c6 View commit details
    Browse the repository at this point in the history