Allow functionality of Suite, Parameterized and BlockJUnit4ClassRunnerWithParameters to be reused in custom Runners

[PeterWippermann]

As of version 4.12 there is much encapsulated functionality in Suite, Parameterized and BlockJUnit4ClassRunnerWithParameters, which is hidden in private methods and not available for custom implementations of Runner for reuse.

I find it very interesting to implement a parameterized Suite. But since accessing the code of the already existing classes is not possible, I'd have to copy a lot of private methods from there. Instead I'd like to refactor the classes and make this functionality publicly available via static methods, e.g. from Parameterized: public static allParameters(TestClass).

If I prepare a pull request for these changes, would it be accepted by the team? Does is make sense at all in regard to the upcoming release of JUnit5? When will JUnit 4.13 be released?

[kcooney]

We don't have a schedule for 4.13. In the past we did releases when someone had time to do the work. Perhaps we should start by tagging bugs and pull requests that we want to go in 4.13.

Adding public methods to these classes that would only used for code reuse seems odd. The average user would be confused about when to use them. Extracting a class with helper methods might make more sense. The only downside is we would need to support these methods forever so it might make maintenance a pain.

What would a parameterized suite look like? What happens when you try to run one class of the suite in isolation in an IDE?

JUnit5 doesn't have a Runner abstraction, and instead has other extension points. Have you looked at those?

[PeterWippermann]

Hi @kcooney ,
I didn't say, that I wanted to contribute a parameterized suite yet - that might be a second step. First of all I wanted to assure that you would allow to make the code accessible that I'm interested in. If not, it would make no sense for me to take the next steps.

What would a parameterized suite look like? What happens when you try to run one class of the suite in isolation in an IDE?

I have solutions for these, but I think they should be left as implementation details to the users. Again, first of all we need to have the functionality of Suite, Parameterized etc. accessible to reuse it.

Extracting a class with helper methods might make more sense.

That would be ok for me. I also thought about separate helper classes in the first place, but I thought the functionality of these would be too tightly coupled to the original classes. In addition, users in general use @RunWith(Parameterized.class), so they don't even see the public API of these classes.

How many of these helper classes would you like to have and how would you name them?
E.g. from Parameterized I'd like to extract the following:

• allParameters(TestClass)
• normalizeParameter(Object)
• produceTestName(String, int, Object[])
• getNamePatternForParameters(TestClass)

From Suite I only need:

• getSuiteClasses(Class<?>)

From BlockJUnit4ClassRunnerWithParameters I need at least:

• createTest(TestClass, Object[])

JUnit5 doesn't have a Runner abstraction, and instead has other extension points. Have you looked at those?

Yes, I had a look at these and I find them very promising! However I wanted to provide a near-term solution for JUnit 4. When will JUnit 5 be released?

[PeterWippermann]

@kcooney , I'd appreciate a feedback on short notice :-)

[marcphilipp]

@PeterWippermann JUnit 5 M1 was released last week. The roadmap for the GA release is available here: https://github.com/junit-team/junit5/wiki#roadmap

[kcooney]

I am sick so I have been slow to respond.

I guess it depends on how willing the team is to support these proposed public methods going forward given that we would almost certainly not implement a parameterized suite in JUnit 4.x. I haven't had time to take a look at those APIs.

Given that 4.13 won't be released for several months you would have to copy and paste anyway for the time being, so I suggest not waiting on us to add the APIs.

[PeterWippermann]

@kcooney I understand, although that's a bit disappointing.
Given the release plan for JUnit 5 (linked by @marcphilipp) and its GA by "the end of 2016", do you plan to maintain JUnit 4 afterwards at all?

[marcphilipp]

Yes, we will maintain JUnit 4 afterwards.

[kcooney]

Didn't mean to imply that we would not maintain 4.x. Most of the team is focused on 5.x so responses to pull requests has been slower than normal. Also, for larger changes, we have sometimes asked if the user has considered 5.x, which slows discussions and reduces the number of commits.

One advantage to trying what you want in 5.x is it would give us additional feedback on the 5.x APIs.

[PeterWippermann]

That's good news!
May I propose, that we just do both?

1. I will have a look at JUnit 5 and try out, how I can solve these kind of problems there. I should have taken a look into the future of JUnit anyway much earlier ;-)
2. Let me start a pull request for my proposed changes in JUnit4. I'd be very happy if these could make it into 4.13, so I can build a ParameterizedSuite on top of that.

[kcooney]

@PeterWippermann I couldn't find most of the methods you wanted to extract. Why don't you send us a pull where you just change them to public static methods, and we'll see which ones make sense to make visible externally and where they belong?

[PeterWippermann]

@kcooney I started PR #1348 . Please have a look 😊

[PeterWippermann]

For the period before this issue can be solved, I set up a project that provides exactly this functionality and makes it available for reuse: https://github.com/PeterWippermann/parameterized-suite