Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AfterScenario Hook runs out of order vs Step hooks when there has been an error #650

Open
Johnlon opened this issue Oct 20, 2024 · 0 comments
Labels
🐛 bug Defect / Bug

Comments

@Johnlon
Copy link
Member

Johnlon commented Oct 20, 2024

👓 What did you see?

We have a test in suite_context_test.go tht makes an assertion on what I consider the wrong behaviour.

		name: "fail_then_undefined_fails_scenario", afterStepCnt: 2, beforeStepCnt: 2,
		body: `
				When step fails
				Then step is undefined`,
		log: `
				>>>> Before suite
				>> Before scenario "test"
				Before step "step fails"
				After step "step fails", error: oops, status: failed
				<< After scenario "test", error: oops
				Before step "step is undefined"
				After step "step is undefined", error: step is undefined, status: undefined
				<<<< After suite`,
	},

Above we can see that the 'AfterScenario' hook is expected called before the scenario is actually over and the remaining steps have their BeforeStep/AfterStep hooks called.

✅ What did you expect to see?

I am expecting the hooks to properly nest.
We should not call the After Scenario until all scenarios have been processed.

I expect this...

				>>>> Before suite
				>> Before scenario "test"
				Before step "step fails"
				After step "step fails", error: oops, status: failed
				Before step "step is undefined"
				After step "step is undefined", error: step is undefined, status: undefined
				<< After scenario "test", error: oops
				<<<< After suite`,

📦 Which tool/library version are you using?

v0.14.1 and also main

🔬 How could we reproduce it?

Just inspect the test cases in TestTestSuite_Run in suite_contexr_test.go

I believe this logic I plain wrong.

📚 Any additional context?

Seems same since at least v0.13

The formatters like events/cucumber look ok - but the hook order is bogus.

@Johnlon Johnlon added the 🐛 bug Defect / Bug label Oct 20, 2024
@Johnlon Johnlon changed the title AfterScenario Hook runs too early when there has been an error AfterScenario Hook runs out of order vs Step hooks when there has been an error Oct 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Defect / Bug
Projects
None yet
Development

No branches or pull requests

1 participant