New formatter idea: worker-oriented view

[howardjohn]

Today, dots-v2 offers a great view of what is happening as tests execute. IMO, superior to others since it can show multiple packages making progress in parallel.

However, it only shows what Go's test output shows us, which misses a lot. Go has a lot of other diagnostic tools, most notably tracing and actiongraph (I talk a bit more about these here; there is no documentation in Go). These can be joined with the existing test data to give some pretty compelling visualizations, IMO.

Below is an example of a prototype I put together:

https://asciinema.org/a/DSUj5zSDHGG4XKKPp8rEy7Ycm

(note: its choppy as I turned down the write frequency to meet asciinema size limits)

Obviously the formatting is just a placeholder 🙂. But we can see a much different than others would show us.

This currently shows a liner per thread, how many actions, how long the current action has lasted, and the action (with a number of completed tests, for test run actions). At the bottom, we also show the current and expected total amount of each action type (clearly, there is a bug here as we execute more than we expect 🙂 ).

Most notably, we can see that the actual execution of the tests accounts for a trivial amount of time - almost all time is spent on linking.

The three data sources provide:

• Actiongraph: gives us the execution plan, letting us know what packages are going to be built, tested, etc
• Trace: gives us the high level execution details. What is being executed on what thread.
• Test data: gives us details of a test execution.

Together I think these can provide an even richer experience than what is possible today.

I have a POC at https://github.com/howardjohn/gotestsum/tree/dotsv3. Its very hacky.

[dnephin]

This is awesome! Thank you for working on this!

I've used the actiongraph before to debug build times, but had not considered using it to augment gotestsum. I really like this idea. I see you attempted to use a file descriptor to receive the data. did that not end up working for some reason?

Let me know if there is anything I can help with!

[howardjohn]

Yeah I initially tried to pass pipes to it then give go /dev/fd/3. I don't really know why it doesn't work TBH, but the reads from the same pipe in gotestsum never return anything for actiongraph. For debug-trace it worked though afaik. go handles these differently, with actiongraph it open+closes at the start, then open+closes at the end. With trace it just opens, appends a bunch for the lifetime of the program, then closes, which could be related. Probably a way to make it work, but I couldn't figure it out so moved to tmp file for now. Probably something we should track down if/when we move forward with this, though

[howardjohn]

This really showcases golang/go#61233 well. So much time spent "running" packages with no tests at all!

Spent some time refining the UI

[afbjorklund]

Here is a little visualization tool, that I did:
https://github.com/afbjorklund/go-test-trace

It can help show some of the concurrency issues, as an addition to what is already shown by this tool.

[ianb-mp]

This looks very cool. In my case, all tests are in the same package (with many tests running simultaneously via t.parallel()). It'd be nice to have a way of showing which tests are 'in progress' with a runtime for each.