Update all pre-commit hook version and fix the resulting errors

[MGAMZ]

Motivation

As mentioned in #23, the pre-commit hooks' version are out-of-date. So doing pre-commit autoupdate for this repo. This introduces many lint errors and static check fails. This PR also fixes them.

pre-commit Upgrade

The following hooks' have been upgraded:

• https://github.com/pre-commit/pre-commit v4.0.0 -> v4.3.0
• https://github.com/PyCQA/flake8 7.1.1 -> 7.3.0
• https://github.com/PyCQA/isort 5.11.5 -> 7.0.0
• https://github.com/pre-commit/mirrors-yapf v0.32.0 -> v0.43.0
• https://github.com/pre-commit/pre-commit-hooks v5.0.0 -> v6.0.0
• https://github.com/executablebooks/mdformat 0.7.9 -> 1.0.0
• https://github.com/myint/docformatter 06907d0 -> v1.7.7
• https://github.com/asottile/pyupgrade v3.0.0 -> v3.21.0
• https://github.com/pre-commit/mirrors-mypy v1.2.0 -> v1.18.2
• https://github.com/astral-sh/uv-pre-commit 0.9.5 -> 0.9.7

Remove mdformat-openmmlab and mdformat_frontmatter as they are no longer maintained and not support mdformat=1.0.0.

fix-encoding-pragma has been removed as pyupgrade now provides the same effect.

flake8 now operates with --max-line-length=90 to avoid too much auto lint modifications.

Lint Auto Fixes

• CONTRIBUTING.md
• README.md
• docs/en/advanced_tutorials/basedataset.md
• docs/en/advanced_tutorials/cross_library.md
• docs/en/advanced_tutorials/data_transform.md
• docs/en/advanced_tutorials/initialize.md
• docs/en/advanced_tutorials/model_analysis.md
• docs/en/advanced_tutorials/registry.md
• docs/en/design/evaluation.md
• docs/en/design/infer.md
• docs/en/design/logging.md
• docs/en/design/visualization.md
• docs/en/examples/train_a_gan.md
• docs/en/migration/param_scheduler.md
• docs/en/notes/changelog.md
• docs/en/notes/contributing.md
• docs/en/tutorials/hook.md
• docs/en/tutorials/param_scheduler.md
• mmengine/_strategy/deepspeed.py
• mmengine/config/config.py
• mmengine/dataset/utils.py
• mmengine/fileio/backends/local_backend.py
• mmengine/fileio/file_client.py
• mmengine/hooks/checkpoint_hook.py
• mmengine/model/test_time_aug.py
• mmengine/runner/checkpoint.py
• mmengine/utils/dl_utils/torch_ops.py
• mmengine/visualization/visualizer.py
• tests/test_analysis/test_jit_analysis.py
• tests/test_dataset/test_base_dataset.py
• tests/test_optim/test_optimizer/test_optimizer_wrapper.py

New flake8 Reports Several F824 Errors

global name / nonlocal name is unused: name is never assigned in scope.
mmengine/fileio/backends/registry_utils.py and mmengine/fileio/io.py are modified to fix this error.
When locally modifying a global dict, it's not necessary to manually declare that the dict is a global variable, as it won't create a new local variable.

Fix mypy Errors

No overload variant of "join" matches argument types "SimplePath", "str" [call-overload]

1. In mmengine/utils/package_utils.py, osp.join now receives a forced str type variable.
2. In mmengine/structures/instance_data.py, static annotation is modified.

Argument 2 to "copy" has incompatible type "str | Path"; expected "str" [arg-type]

1. In mmengine/fileio/backends/local_backend.py, shutil.copy(src, dst) is modified to shutil.copy(str(src), str(dst))
2. In mmengine/fileio/backends/local_backend.py, shutil.copytree(src, dst) is modified to shutil.copytree(str(src), str(dst))

Fix mmengine/runner/runner.py:1428: error: "Any" not callable [misc].
Fix mmengine/runner/runner.py:1439: error: Argument "num_workers" to "worker_init_fn" has incompatible type "Any | None"; expected "int" [arg-type].
dataloader_cfg.get('num_workers') is modified to dataloader_cfg.get('num_workers', 0)
The position of # type: ignore is modified to ensure it is effective.
The similar issue also happens in mmengine/runner/_flexible_runner.py, which is also modified.

Accomplish TODO: if filepath or filepaths are Path, should return Path.

This TODO is described in mmengine/fileio/backends/local_backend.py, this causes implementations and static annotations modification on func: join_path, copyfile, copytree, copyfile_from_local, copytree_from_local, copyfile_to_local, copytree_to_local.

Now the fileio can dynamically operates pathlib.Path and str inputs and will return different types.

tests/test_fileio/test_backends/test_local_backend.py is modified to align with the updated fileio API.

test_join_path is also influenced.

Fix TestEmptyCacheHook Test

The annotation in TestEmptyCacheHook suggestes it wants to test the times of calling of torch.cuda.empty_cache Hook. The test logic hopes runner.train will call max_epochs times. To achieve this, the test script specifies val_interval to 1e6, but this cannot override the final validation on training last epoch, see the EpochBasedTrainLoop below:

if (self.runner.val_loop is not None
        and self._epoch >= self.val_begin
        and (self._epoch % self.val_interval == 0
                or self._epoch == self._max_epochs)):
    self.runner.val_loop.run()

Merely specifying val_interval to 1e6 cannot prevent entering if when self._epoch == self._max_epochs, i.e., when training done. To safely prevent entering this if block, set val_begin to infinite seems to be an option.

BC-breaking

There shouldn't be any BC-breaking.

[read-the-docs-community]

Documentation build overview

📚 onedl-mmengine | 🛠️ Build #30252754 | 📁 Comparing d893fcd against latest (aab4a17)

🔍 Preview build

Show files changed (30 files in total): 📝 30 modified | ➕ 0 added | ➖ 0 deleted

File	Status
advanced_tutorials/basedataset.html	📝 modified
advanced_tutorials/cross_library.html	📝 modified
advanced_tutorials/initialize.html	📝 modified
advanced_tutorials/model_analysis.html	📝 modified
advanced_tutorials/registry.html	📝 modified
design/evaluation.html	📝 modified
design/infer.html	📝 modified
design/logging.html	📝 modified
design/visualization.html	📝 modified
examples/train_a_gan.html	📝 modified
migration/param_scheduler.html	📝 modified
notes/changelog.html	📝 modified
notes/contributing.html	📝 modified
api/generated/mmengine.fileio.LocalBackend.html	📝 modified
_modules/mmengine/_strategy/deepspeed.html	📝 modified
_modules/mmengine/config/config.html	📝 modified
_modules/mmengine/dataset/utils.html	📝 modified
_modules/mmengine/fileio/file_client.html	📝 modified
_modules/mmengine/fileio/io.html	📝 modified
_modules/mmengine/hooks/checkpoint_hook.html	📝 modified
_modules/mmengine/model/test_time_aug.html	📝 modified
_modules/mmengine/runner/_flexible_runner.html	📝 modified
_modules/mmengine/runner/checkpoint.html	📝 modified
_modules/mmengine/runner/runner.html	📝 modified
_modules/mmengine/structures/instance_data.html	📝 modified
_modules/mmengine/utils/package_utils.html	📝 modified
_modules/mmengine/visualization/visualizer.html	📝 modified
_modules/mmengine/fileio/backends/local_backend.html	📝 modified
_modules/mmengine/fileio/backends/registry_utils.html	📝 modified
_modules/mmengine/utils/dl_utils/torch_ops.html	📝 modified

[MGAMZ]

There're some pytest failures, I'm trying to figure out what happened.

============================================================== short test summary info ==============================================================
FAILED tests/test_fileio/test_backends/test_local_backend.py::TestLocalBackend::test_copyfile_0 - AssertionError: '/tmp/tmpk0503p6e/test.txt.bak' != PosixPath('/tmp/tmpk0503p6e/test.txt.bak')
FAILED tests/test_fileio/test_backends/test_local_backend.py::TestLocalBackend::test_copyfile_from_local_0 - AssertionError: '/tmp/tmpaqcephbs/test.txt.bak' != PosixPath('/tmp/tmpaqcephbs/test.txt.bak')
FAILED tests/test_fileio/test_backends/test_local_backend.py::TestLocalBackend::test_copyfile_to_local_0 - AssertionError: '/tmp/tmpjjlhckku/test.txt.bak' != PosixPath('/tmp/tmpjjlhckku/test.txt.bak')
FAILED tests/test_fileio/test_backends/test_local_backend.py::TestLocalBackend::test_copytree_0 - AssertionError: '/tmp/tmpbfgej59f/dir100' != PosixPath('/tmp/tmpbfgej59f/dir100')
FAILED tests/test_fileio/test_backends/test_local_backend.py::TestLocalBackend::test_copytree_from_local_0 - AssertionError: '/tmp/tmplswa4rdf/dir100' != PosixPath('/tmp/tmplswa4rdf/dir100')
FAILED tests/test_fileio/test_backends/test_local_backend.py::TestLocalBackend::test_copytree_to_local_0 - AssertionError: '/tmp/tmpd0zpjgca/dir100' != PosixPath('/tmp/tmpd0zpjgca/dir100')
FAILED tests/test_fileio/test_fileclient.py::TestFileClient::test_http_backend[http-None] - urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
FAILED tests/test_fileio/test_fileclient.py::TestFileClient::test_http_backend[None-http] - urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
FAILED tests/test_fileio/test_io.py::test_copyfile - AssertionError: assert '/tmp/tmp3fiklerp/test.txt.bak' == PosixPath('/tmp/tmp3fiklerp/test.txt.bak')
FAILED tests/test_fileio/test_io.py::test_copytree - AssertionError: assert '/tmp/tmpdajzvqqi/dir100' == PosixPath('/tmp/tmpdajzvqqi/dir100')
FAILED tests/test_fileio/test_io.py::test_copyfile_from_local - AssertionError: assert '/tmp/tmpukbefd3j/test.txt.bak' == PosixPath('/tmp/tmpukbefd3j/test.txt.bak')
FAILED tests/test_fileio/test_io.py::test_copytree_from_local - AssertionError: assert '/tmp/tmpqnmdk6_e/dir100' == PosixPath('/tmp/tmpqnmdk6_e/dir100')
FAILED tests/test_fileio/test_io.py::test_copyfile_to_local - AssertionError: assert '/tmp/tmp7i0xqlhk/test.txt.bak' == PosixPath('/tmp/tmp7i0xqlhk/test.txt.bak')
FAILED tests/test_fileio/test_io.py::test_copytree_to_local - AssertionError: assert '/tmp/tmpu4aegp01/dir100' == PosixPath('/tmp/tmpu4aegp01/dir100')
FAILED tests/test_hooks/test_empty_cache_hook.py::TestEmptyCacheHook::test_with_runner - AssertionError: 5 != 4
FAILED tests/test_strategies/test_fsdp.py::TestStrategy::test_run_strategy - torch.multiprocessing.spawn.ProcessRaisedException: 
FAILED tests/test_visualizer/test_vis_backend.py::TestDVCLiveVisBackend::test_close - AttributeError: 'Live' object has no attribute '_dvc_file'. Did you mean: 'dvc_file'?
FAILED tests/test_visualizer/test_vis_backend.py::TestAimVisBackend::test_experiment - ImportError: Please run "pip install aim" to install aim
FAILED tests/test_visualizer/test_vis_backend.py::TestAimVisBackend::test_add_config - ImportError: Please run "pip install aim" to install aim
FAILED tests/test_visualizer/test_vis_backend.py::TestAimVisBackend::test_add_image - ImportError: Please run "pip install aim" to install aim
FAILED tests/test_visualizer/test_vis_backend.py::TestAimVisBackend::test_add_scalar - ImportError: Please run "pip install aim" to install aim
FAILED tests/test_visualizer/test_vis_backend.py::TestAimVisBackend::test_add_scalars - ImportError: Please run "pip install aim" to install aim
FAILED tests/test_visualizer/test_vis_backend.py::TestAimVisBackend::test_close - ImportError: Please run "pip install aim" to install aim
====================================== 23 failed, 34 passed, 167 deselected, 879 warnings in 329.35s (0:05:29) ======================================

[MGAMZ]

This PR includes many modifications, but there should be no BC-breaking. Most of the modifications are introduced to make CICD happy.

[Copilot]

Pull Request Overview

This PR applies automated code formatting improvements across multiple files using pre-commit hooks. The changes primarily focus on code style consistency without altering functionality.

• Updated pre-commit hook configurations with newer versions of linting tools
• Reformatted code to comply with line length and style guidelines (removing unnecessary whitespace, adjusting line breaks)
• Enhanced type hints for better clarity
• Fixed documentation formatting (removing extra spaces, correcting escaped characters)

Reviewed Changes

Copilot reviewed 28 out of 41 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
.pre-commit-config.yaml	Updated pre-commit hook versions and configuration (flake8, isort, yapf, mypy, etc.)
tests/test_optim/test_optimizer/test_optimizer_wrapper.py	Reformatted conditional expressions for better readability
tests/test_hooks/test_empty_cache_hook.py	Updated config attribute from val_interval to val_begin
tests/test_fileio/test_io.py	Changed path concatenation from osp.join to Path division operator
tests/test_fileio/test_backends/test_local_backend.py	Updated test expectations to match new path handling implementation
tests/test_dataset/test_base_dataset.py	Reformatted comparison expressions across multiple lines
tests/test_analysis/test_jit_analysis.py	Reformatted dictionary comprehension for better readability
mmengine/visualization/visualizer.py	Reformatted assertion statement with awkward line breaks
mmengine/utils/package_utils.py	Added explicit string conversion for path compatibility
mmengine/utils/dl_utils/torch_ops.py	Reformatted multi-line assignment for better readability
mmengine/structures/instance_data.py	Enhanced type hints using cast and get_args, simplified type aliases
mmengine/runner/runner.py	Reformatted function calls and added default parameter value
mmengine/runner/_flexible_runner.py	Same reformatting as runner.py
mmengine/runner/checkpoint.py	Reformatted dictionary comprehension
mmengine/model/test_time_aug.py	Reformatted dictionary comprehension
mmengine/hooks/checkpoint_hook.py	Reformatted multi-line assertion
mmengine/fileio/io.py	Removed unused global variable declaration
mmengine/fileio/file_client.py	Reformatted function signature line break
mmengine/fileio/backends/registry_utils.py	Removed unused global variable declaration
mmengine/fileio/backends/local_backend.py	Enhanced return type annotations and implementation to preserve Path types
mmengine/dataset/utils.py	Reformatted dictionary comprehension
mmengine/config/config.py	Reformatted dictionary comprehensions
mmengine/_strategy/deepspeed.py	Reformatted multi-line assertion
Multiple documentation files	Fixed spacing issues, corrected escaped markdown characters, reformatted tables

Comments suppressed due to low confidence (1)

mmengine/config/config.py:270

• ConfigDict.reduce_ex returns tuple of size 5 and tuple of size 6.

    def __reduce_ex__(self, proto):

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The return type annotation str in FileClient.join_path is inconsistent with the updated LocalBackend.join_path which now returns Union[str, Path]. This creates an API inconsistency where the backend implementation returns a different type than what the client interface declares. The return type should be updated to Union[str, Path] to match the backend implementation.

[Copilot]

[nitpick] The line break placement creates awkward formatting where bboxes[:, and 3] are split across lines. Consider reformatting to either keep the array indexing on one line or break at a more natural boundary, such as: assert (bboxes[:, 0] <= bboxes[:, 2]).all() and (bboxes[:, 1] <= bboxes[:, 3]).all() on a single line or breaking before the and operator.