Fix: proper return types for MultiIndex.swaplevel & MultiIndex.union

[realzachbrownjohn]

• Closes MultiIndex method corrections #1436
• Tests added: None yet

This is my first contribution so I would appreciate your guidance. I think I have fixed the issue, but I'm confused on where to add test(s), or if I need to at all. Would it be acceptable to use Dr-Irv's reproduction script, or perhaps a modified copy?

[cmp0xff]

Please look for test_multiindex to place the tests. I think you need to add one or more new test functions in tests/indexes/test_indexes.py.

[realzachbrownjohn]

Thank you for the feedback! I will work on that shortly.

[realzachbrownjohn]

I've tried my best to spend a bit of time unpacking this. Ultimately, I didn't think it was best to include all types of Index in the overload because not all types of Index are valid to union with MultiIndex (only those with tuples). Referring to tuples within Indexes wasn't supported so I had to add that support in _typing.pyi.

Happy to adjust recognising that Dr-Irv never mentioned any issue with using the complete Index type.

Additionally, I've attempted to add a test in tests/indexes/test_indexes.py covering the different cases I could think of.

[realzachbrownjohn]

I can see now that all checks have failed, looks related to my extension of S1 (in adding support for tuple[Hashable, ...] I seem to have violated some sort of constraint). Any guidance would be greatly appreciated.

[cmp0xff]

• You can mark the PR as a draft if you plan to work on it for a while.
  • Just click Re-request review on the top-right corner of the PR when you are done.
• Please run pre-commit locally to avoid simple mistakes
• Please use poe to do further tests locally, e.g. poe test_all

[realzachbrownjohn]

Ended up removing MultiIndex-Index unions from assumption for returning a MultiIndex - was causing too many problems and Dr-Irv said it was okay to ignore. It was either violating the overload standard or confusing mypy in testing.

swaplevel tests added, MultiIndex tests removed some cases relating to different types of Indexes (since we no longer try checking for those), and S1 changes reverted.

[realzachbrownjohn]

I'm not sure why it failed. I ran poe test_all locally several times; all successful.

Running the test case locally individually it failed, seems to be a runtime error because Pandas won't accept an Index unionising a MultiIndex (as it should). Removing that test case.

[cmp0xff]

Please also run pre-commit locally.

[cmp0xff]

If MultiIndex.union does not produce an Index, please remove this overload

[cmp0xff]

This extra empty line can also be removed

[cmp0xff]

Would you?

[realzachbrownjohn]

Would you?

That extra newline disappeared once I sync'd with the up-to-date master. It shouldn't be there anymore, or at least it isn't on my end.

[realzachbrownjohn]

Please also run pre-commit locally.

My mistake. I had ran a pre-commit after committing, and didn't commit the refactoring it made.

[cmp0xff]

Great, now all checks pass. We are almost there.

[cmp0xff]

I suppose Index should be removed. It is not tested, either.

Suggested change

	self,
	other: Self | Index | list[HashableT],
	sort: bool | None = ...,
	self, other: list[HashableT] | Self, sort: bool | None = ...

[realzachbrownjohn]

Index can't be removed; the combined scope of the overloaded union definitions in MultiIndex need to at least match the scope of the base definition, and removing Index breaks that. I will add a test for it instead.

[Dr-Irv]

If the type checkers are complaining about mismatched signatures between the parent class declaration Index.union() and the subclass MultiIndex.union(), you can use @override where you import override from typing_extensions, since we know we want to override the signature from the base class

[realzachbrownjohn]

It might be complaining about a signature mismatch?

Method "union" overrides class "Index" in an incompatible manner
Parameter 2 type mismatch: base parameter is type "list[HashableT@union] | Index[Any]", override parameter is type "MultiIndex | list[HashableT@union]"
Type "list[HashableT@union] | Index[Any]" is not assignable to type "MultiIndex | list[HashableT@union]"
Type "Index[Any]" is not assignable to type "MultiIndex | list[HashableT@union]"
"Index[Any]" is not assignable to "MultiIndex"
"Index[Any]" is not assignable to "list[HashableT@union]"

In any case, I tried @override above the code to no avail. Gives the same error.

[Dr-Irv]

OK, but I think there is a better way to do this whole thing. Don't add union to MultiIndex. Rather, change the definition of union in core/indexes/base.pyi to:

    def union(
        self, other: list[HashableT] | Self, sort: bool | None = None
    ) -> Self: ...

I think that will work.

[realzachbrownjohn]

I agree. Will do that now.

[cmp0xff]

Would you?