Skip to content

Conversation

ivanpauno
Copy link
Member

Fixes #528 (comment).

test_module_scope.py / test_function_scope.py don't work when --count=N is passed, because now the test order is different.
I moved them to the example folder and I'm now running then with the pytester plugin to avoid the issue.

I'm also running the other examples we have in that folder, ignoring one that would introduce a dependency on launch_ros (we have to move that one somewhere else later).
I've also fixed some issues in the existing examples.

…sed.

* [launch_pytest] Test all the examples.

Signed-off-by: Ivan Santiago Paunovic <[email protected]>
Signed-off-by: Ivan Santiago Paunovic <[email protected]>
@ivanpauno ivanpauno added the bug Something isn't working label Nov 3, 2021
@ivanpauno ivanpauno requested review from Blast545 and hidmic November 3, 2021 20:38
@ivanpauno ivanpauno self-assigned this Nov 3, 2021
Copy link
Contributor

@Blast545 Blast545 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after green CI

# this function can use assertions to validate the output or return a boolean.
# pytest generates easier to understand failures when assertions are used.
assert output == 'hello_world\n', 'process never printed hello_world'
assert process_tools.wait_for_output_sync(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious question, why was this assert removed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function always returns None in this case, though I think that's confusing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See a9cc81c

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function always returns None in this case, though I think that's confusing.

It was not only confusing, it was also blocking unnecessarely untile the timeout expired.

Copy link
Contributor

@hidmic hidmic Nov 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ivanpauno meta: we clearly need a less terse idiom to write predicates. If we were to assume the predicate passed if it returns None, that'd be practical yet surprising. If we were to simply return the result of evaluating the expression, we'd lose detailed assertion message (most of which are provided pytest rewriting logic). So how about splitting use cases? E.g.:

# Wait for incoming process output with a timeout
output = wait_for_output_sync(process, timeout=10)
output = await wait_for_output(process, timeout=10)

# Wait for incoming process output matching a predicate with a timeout
def predicate(output):
    return 'foo' in output
output = wait_for_output_sync(process, predicate, timeout=10)
output = await wait_for_output(process, predicate, timeout=10)

# Assert a statement on process output with a timeout 
def statement(output):
    assert output == 'foo'
assert_output_sync(process, statement, timeout=10)
await assert_output(process, statement, timeout=10)

In this case, it would read:

def hello_world_found():
     assert output.splitlines() == ['hello_world'], 'process never printed hello_world'
assert_output_sync(launch_context, hello_world_proc, hello_world_found, timeout=5)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like the right approach to me.
I will address it in a follow-up PR so we solve the CI issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, merge!

Signed-off-by: Ivan Santiago Paunovic <[email protected]>
@ivanpauno
Copy link
Member Author

CI using --retest-until-fail 2:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status

Signed-off-by: Ivan Santiago Paunovic <[email protected]>
@ivanpauno
Copy link
Member Author

  • Windows Build Status

Copy link
Contributor

@hidmic hidmic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but for one meta comment

# this function can use assertions to validate the output or return a boolean.
# pytest generates easier to understand failures when assertions are used.
assert output == 'hello_world\n', 'process never printed hello_world'
assert process_tools.wait_for_output_sync(
Copy link
Contributor

@hidmic hidmic Nov 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ivanpauno meta: we clearly need a less terse idiom to write predicates. If we were to assume the predicate passed if it returns None, that'd be practical yet surprising. If we were to simply return the result of evaluating the expression, we'd lose detailed assertion message (most of which are provided pytest rewriting logic). So how about splitting use cases? E.g.:

# Wait for incoming process output with a timeout
output = wait_for_output_sync(process, timeout=10)
output = await wait_for_output(process, timeout=10)

# Wait for incoming process output matching a predicate with a timeout
def predicate(output):
    return 'foo' in output
output = wait_for_output_sync(process, predicate, timeout=10)
output = await wait_for_output(process, predicate, timeout=10)

# Assert a statement on process output with a timeout 
def statement(output):
    assert output == 'foo'
assert_output_sync(process, statement, timeout=10)
await assert_output(process, statement, timeout=10)

In this case, it would read:

def hello_world_found():
     assert output.splitlines() == ['hello_world'], 'process never printed hello_world'
assert_output_sync(launch_context, hello_world_proc, hello_world_found, timeout=5)

@ivanpauno ivanpauno merged commit 92400b4 into master Nov 5, 2021
@delete-merged-branch delete-merged-branch bot deleted the ivanpauno/launch-pytest-fix-retest-until-fail branch November 5, 2021 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants