Improvements to Test result Output API

[cspray]

The problem

Currently the output displayed by the CLI command when showing test output is implemented by the DefaultResultPrinter. It is setup directly in the AsyncUnitFrameworkRunner. This presents significant extensibility issues and a tight coupling to the Symfony Console component. It would be impossible for a user to replace the defaullt output. Additionally, our end users should not be aware of our dependence on the Syfmony Console component. Plugins, the CLI and other public facing APIs should be resilient to the underlying libraries used to implement the framework potentially changing.

The solution

TerminalOutput

To solve these problems will require an approach with multiple facets. The first is to introduce the concept of a TerminalOutput interface that abstracts away the sending of data to the console when CLI commands are run. We're intentionally this name to signify that this output is intended to be sent to a terminal's STDOUT. It is expected that the vast majority, if not all, use cases will involve outputting some information to a terminal to let the developer know if tests passed. This output should be separated from the output we might expect from build artifacts related to code coverage or metrics.

This abstraction now allows us to hide the fact that we're relying on Symfony Console at the moment for terminal output. Wrappers can be constructed for most of the Symfony Console functionality and provides an opportunity for other wrappers to be constructed for better libraries. This will require a bit more work on part of the implementing developer to provide wrappers or adapters but this is work effectively done by an internal contributor to AsyncUnit and should not impact end consumers.

TerminalOutputPlugin

The second facet is to introduce the concept of a TerminalOutputPlugin that abstracts away which specific implementation should show test result output. This Plugin could be picked up by automagically in the compilation step and any of this type found can be automatically loaded. During the plugin registration process we can ensure that at least 1 TerminalOutputPlugin is loaded and, if not, ensure that the default test result printer is registered so that there's always input in the console.

Tradeoffs and other Concerns

• While this abstraction allows us to hide the fact that we're not dealing with Symfony Console it also makes it harder for our default implementation, and other potential implementations, to use the SymfonyStyle component. Even if we provided an OutputInterface wrapper the SymfonyStyle requires an InputInterface in its constructor. It could be possible that during implementation a reasonable solution is found for this problem but on the surface it seems intractable; either effectively prevent the use of the SymfonyStyle object or create a leaky abstraction.

• This also does not account for the fact that some strings could be formatted in such a way that relies on Symfony Console's string formatting utilities. For example, some strings could use <fg=green>text</> to signify their text color. Obviously this type of string formatting would only work if used with the Symfony Console project. This proposal does not address this problem and a new proposal would have to be constructed to address the problems with the use of this formatting.