-
Notifications
You must be signed in to change notification settings - Fork 17
Async/Sync Tests Rewrite #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Luc1412
wants to merge
41
commits into
master
Choose a base branch
from
lib-tests-refac
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
ab17fc2
Create test_methods.py
trevorflahardy 11e45c3
Correct Concatenate error
trevorflahardy 06af63e
Update for consistency with "is"
trevorflahardy 7414b06
Fix ReconstructAble.from_dict typing
trevorflahardy 540b688
Correct type narrowing for async and sync reconstructable checks
trevorflahardy a5be0ae
Black isort for recent changes
trevorflahardy d257a80
Start mock python pytests
trevorflahardy ae25d17
Upgrade imports for Python 3.9
Luc1412 6abfb8b
Update project for pytest logging output, if needed for some tests
trevorflahardy 3fd0541
Update gitignore for build files in newer Python versions
trevorflahardy 6606092
Start moving test methods
trevorflahardy 944f1a0
Add README for test maintainers.
trevorflahardy d9eb490
Remove now unused file
trevorflahardy 02c50fa
Move internal cosmetic helpers to new file
trevorflahardy f21c97a
Migrate cosmetic test functions, update the cosmetic handling of test…
trevorflahardy c0e46ca
Black isort
trevorflahardy 2cdd1d3
Add clarification for test_methods
trevorflahardy cbf3c59
Rename all test_async -> test
trevorflahardy be9652e
Updates to tests
trevorflahardy c44e319
Refactor how similar tests behave for better maintainability
trevorflahardy 3ce6882
test_cosmetic_lego_kits -> test_cosmetic_lego_kit
trevorflahardy f704111
Black isort
trevorflahardy 925bb2b
Update some terminology for error handling and documentation
trevorflahardy da124eb
Add response flags to cosmetic tests
trevorflahardy d38a187
Fix import
trevorflahardy 49c1a0e
Fix import
trevorflahardy 03ed4f3
Remove log
trevorflahardy 5f89f5e
Fixes to tests
trevorflahardy dbfdd64
Revert some changes from master manually
trevorflahardy a4f24fe
Add back annotation for len return
trevorflahardy 6dbd588
Remove duplicated `async with` statement causing test to fail
trevorflahardy 3500aba
Rename test_x to validate_x so pyright doesn't pickup validation func…
trevorflahardy 72acb8c
Rename `_test` to `_validate` to be more consistent with other test v…
trevorflahardy 6887549
Remove "synchronize" test condition to remove duplicate tests running.
trevorflahardy 53d1c09
Remove unneeded reconstruction test, as the HybridClient now covers t…
trevorflahardy da4a0ab
Remove reference to deleted file
trevorflahardy 796ed4a
Cleanup wording in README.md
trevorflahardy 1d263d7
Add missing filename in README.md
trevorflahardy b074e91
black and isort
Luc1412 3ee8cfd
delete test_sync_methods.py
Luc1412 e8221dd
Update tests/README.md
trevorflahardy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,7 +2,7 @@ name: PyPI Release | |
|
||
on: | ||
release: | ||
types: [ published ] | ||
types: [published] | ||
workflow_dispatch: | ||
|
||
jobs: | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -87,4 +87,7 @@ fabric.properties | |
|
||
# Development Test Files | ||
_.py | ||
.env | ||
.env | ||
|
||
# Build files | ||
/build/* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
# Fortnite API Library Tests | ||
|
||
This library makes a large effort to ensure that the responses given from the API are stably transformed to their | ||
respective Python objects. Every client, both in the `Client` and `SyncClient` classes are tested. This file | ||
outlines how the tests are laid such that all these edge cases are handled. | ||
|
||
## Generic Library Tests | ||
|
||
Many tests in the main `/tests` directory are generic-related object-related tests. These ensure basic functionality surrounding how the more-complex objects of the library are constructed and function. | ||
|
||
| Test File | Purpose and Logic | | ||
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| `test_account.py` | Ensures that an `Account` object is created properly and its dunder methods work as expected. | | ||
| `test_aes.py` | Ensures that `Aes` object initializes properly by checking known dynamic keys and hashes. | | ||
| `test_asset.py` | Ensures that the rules regulating `Asset` resizing are correct and that the asset reading functions function correctly. | | ||
| `test_beta.py` | Ensures that a user with the `beta` flag disabled on a `Client` cannot call beta methods. This validates that the beta flag decorator works as expected. | | ||
| `test_proxy.py` | Ensures that the `TransformerListProxy` class initializes properly, transforms to expected objects as needed, and has the same interface as a typical `py.List` would. | | ||
| `test_ratelimits.py` | Ensures that the library's handling of rate limits is correct, and related exceptions are raised as expected. | | ||
| `test_repr.py` | The library uses a dynamic decorator to create the `__repr__` dunder by taking advantage of the `__slots__` on a class. This test ensures that the dynamic function works as expected. | | ||
| `test_methods.py` | The handling of all the functions on the `Client` and `SyncClient` class. See Edge Cases below for more information. | | ||
|
||
### Edge Case Library Tests | ||
|
||
Due to the library's complexity, especially considering that it fully supports both `async` and `sync` functionality, | ||
many edge cases are tested. Mainly, these tests are related to the `Client` and `SyncClient` classes, and the methods | ||
defined on them. | ||
|
||
#### Definition and Tests for the Hybrid Client | ||
|
||
##### Test Client Hybrid: `test_client_hybrid.py` | ||
|
||
The tests define a custom `ClientHybrid` class (in `./client/test_client_hybrid.py`). This class wraps a `Client` to act as an intermediatory between a requested API call and the actual method. When an API call is requested, the `ClientHybrid` will call **both** the async `Client` version and the `SyncClient` version of the method. The results are then compared to ensure that they are the same. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any reason for |
||
|
||
Thus, all tests that make API calls will import and use the `ClientHybrid`. | ||
|
||
As an example, consider the user requesting to call `fetch_aes()` using the `ClientHybrid`: | ||
|
||
1. The `ClientHybrid` class is initialized as a context manager, the same as you would with a `Client`. | ||
2. The `fetch_aes()` method is called on the `ClientHybrid`. | ||
3. The sync method of `fetch_aes()` is called on an internally held `SyncClient` class. | ||
4. The async method of `fetch_aes()` is called on the `Client` itself. | ||
5. The result, if reconstructable or comparable, is checked to ensure that both returned objects are the same. | ||
6. The result of the async method call is returned as the final value. | ||
|
||
This approach, although loop blocking in nature, ensures that the results from both the `Client` and `SyncClient` are the same. | ||
|
||
##### Test Client: `test_client.py` | ||
|
||
The tests defined here ensure that the client's behavior surrounding initialization work as expected. This is, but is not limited to, context manager use, custom passed HTTP session management, etc. | ||
|
||
#### Tests for the Methods on the Client: `test_methods.py` | ||
|
||
Every method, except for those defined in `test_stats.py` and `cosmetics/*.py` (more on this directory after) on the `Client` is tested here. This uses the `ClientHybrid`, as described above. | ||
|
||
This logic has been separated out of the conventional cosmetic tests due to the nature of the stats endpoints themselves. The `Client` must have API key while using them, unlike any other endpoint, and have been clustered together accordingly. | ||
|
||
#### Cosmetic Tests: `/cosmetics/*.py` | ||
|
||
A majority of the definitions in this library relate to the cosmetics of Fortnite. Thus, the tests for them are inherently large. To combat this, and to future proof the readability and maintainability of the library, these tests have been separated from others of the `Client` to `cosmetics/test_cosmetic_functions.py` and the associated internal helper functions to `cosmetics/cosmetic_utils.py`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,151 @@ | ||
""" | ||
MIT License | ||
|
||
Copyright (c) 2019-present Luc1412 | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. | ||
""" | ||
|
||
from __future__ import annotations | ||
|
||
import inspect | ||
import logging | ||
from collections.abc import Callable, Coroutine | ||
from typing import TYPE_CHECKING, Any, Generic, TypeVar | ||
|
||
import pytest | ||
import requests | ||
from typing_extensions import Concatenate, ParamSpec, TypeAlias | ||
|
||
import fortnite_api | ||
from fortnite_api import ReconstructAble | ||
|
||
P = ParamSpec('P') | ||
T = TypeVar('T') | ||
|
||
if TYPE_CHECKING: | ||
Client: TypeAlias = fortnite_api.Client | ||
SyncClient = fortnite_api.SyncClient | ||
|
||
CoroFunc = Callable[P, Coroutine[Any, Any, T]] | ||
|
||
log = logging.getLogger(__name__) | ||
|
||
|
||
class HybridMethodProxy(Generic[P, T]): | ||
def __init__( | ||
self, | ||
hybrid_client: ClientHybrid, | ||
sync_client: SyncClient, | ||
async_method: CoroFunc[Concatenate[Client, P], T], | ||
sync_method: Callable[Concatenate[SyncClient, P], T], | ||
) -> None: | ||
self.__hybrid_client = hybrid_client | ||
self.__sync_client = sync_client | ||
|
||
self.__async_method = async_method | ||
self.__sync_method = sync_method | ||
|
||
@property | ||
def __name__(self) -> str: | ||
return self.__async_method.__name__ | ||
|
||
def _validate_results(self, async_res: T, sync_res: T) -> None: | ||
assert type(async_res) is type(sync_res), f"Expected {type(async_res)}, got {type(sync_res)}" | ||
|
||
if isinstance(async_res, fortnite_api.Hashable): | ||
assert isinstance(sync_res, fortnite_api.Hashable) | ||
assert async_res == sync_res | ||
log.debug('Hashable comparison passed for method %s.', self.__async_method.__name__) | ||
|
||
if isinstance(async_res, fortnite_api.ReconstructAble): | ||
assert isinstance(sync_res, fortnite_api.ReconstructAble) | ||
|
||
sync_res_narrowed: ReconstructAble[Any, fortnite_api.SyncHTTPClient] = sync_res | ||
async_res_narrowed: ReconstructAble[Any, fortnite_api.HTTPClient] = async_res | ||
|
||
async_raw_data = sync_res_narrowed.to_dict() | ||
sync_raw_data = sync_res_narrowed.to_dict() | ||
assert async_raw_data == sync_raw_data | ||
log.debug('Raw data equality passed for method %s', self.__async_method.__name__) | ||
|
||
async_reconstructed = type(async_res_narrowed).from_dict(async_raw_data, client=self.__hybrid_client) | ||
sync_reconstructed = type(sync_res_narrowed).from_dict(sync_raw_data, client=self.__sync_client) | ||
|
||
assert isinstance(async_reconstructed, type(sync_reconstructed)) | ||
assert type(async_reconstructed) is type(async_res_narrowed) | ||
assert type(sync_reconstructed) is type(sync_res_narrowed) | ||
log.debug('Reconstructed data equality passed for method %s', self.__async_method.__name__) | ||
|
||
async def __call__(self, *args: P.args, **kwargs: P.kwargs) -> T: | ||
# Call the sync method first | ||
sync_result = self.__sync_method(self.__sync_client, *args, **kwargs) | ||
|
||
# Call the async method | ||
async_result = await self.__async_method(self.__hybrid_client, *args, **kwargs) | ||
|
||
log.debug('Validating results for %s', self.__async_method.__name__) | ||
self._validate_results(async_result, sync_result) | ||
return async_result | ||
|
||
|
||
class ClientHybrid(fortnite_api.Client): | ||
"""Denotes a "client-hybrid" that calls both a async and sync | ||
client when a method is called. | ||
|
||
Pytest tests are not called in parallel, so although this is a | ||
blocking operation it will not affect the overall performance of | ||
the tests. | ||
""" | ||
|
||
def __init__(self, *args: Any, **kwargs: Any) -> None: | ||
super().__init__(*args, **kwargs) | ||
|
||
kwargs.pop('session', None) | ||
session = requests.Session() | ||
self.__sync_client: fortnite_api.SyncClient = fortnite_api.SyncClient(*args, session=session, **kwargs) | ||
self.__inject_hybrid_methods() | ||
|
||
def __inject_hybrid_methods(self) -> None: | ||
# Walks through all the public coroutine methods in this class. If it finds one, | ||
# it will mark it as a hybrid proxy method with it and its sync counterpart. | ||
for key, value in fortnite_api.Client.__dict__.items(): | ||
if inspect.iscoroutinefunction(value): | ||
sync_value = getattr(fortnite_api.SyncClient, key, None) | ||
if sync_value is not None and inspect.isfunction(sync_value): | ||
setattr(self, key, HybridMethodProxy(self, self.__sync_client, value, sync_value)) | ||
|
||
async def __aexit__(self, *args: Any) -> None: | ||
# We need to ensure that the sync client is also closed | ||
self.__sync_client.__exit__(*args) | ||
return await super().__aexit__(*args) | ||
|
||
|
||
@pytest.mark.asyncio | ||
async def test_hybrid_client(): | ||
hybrid_client = ClientHybrid() | ||
|
||
# Walk through all coroutines in the normal client - ensure that | ||
# every coro on the normal is a proxy method on the hybrid client. | ||
for key, value in fortnite_api.Client.__dict__.items(): | ||
if inspect.iscoroutinefunction(value) and not key.startswith('_'): | ||
assert hasattr(hybrid_client, key) | ||
|
||
method = getattr(hybrid_client, key) | ||
assert isinstance(method, HybridMethodProxy) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be nice to directly embed links to those files within the readme for easy access.