Test project setup

[nikmd23]

Unit/Integration Testing:

Till now we have ben prototyping what Glimpse v2 would look like and have been avoiding writing unit tests. This is primarily because we have been making big changes and generally trying different concepts, hence ease of making changes has been a bigger priority over reliability. But now we are moving more towards locking down some of our concepts, setting up CI, brining more people into the project, etc and as such need to get tests in place.

• Server Side - Here we are probably looking at using a combination of xUnit, AutoFixture and NSubstitute. Additionally, we will want to have integration tests which try and exercise the system end to end, etc. The Asp.Net and MVC teams have quite a few patterns in place for some of the tests we are looking at, so probably makes sense to look at how they are doing some of this and if its applicable. Lastly, getting Music Store setup so that we can have Glimpse running off that and maybe have some automated integration tests.
• Client Side - The client is in a similar state to the server but has something that is approaching integration tests. I've spent the time building a fairly advanced data generator which means the client can fully run without the server - its really cool. That said, their isn't any automated tests on this which allows us to repeat test cases. Additionally we are looking at needing unit tests - maybe something like Jasmine, Jest and Karma.

Tasks
This issue does not require more than a few tests being written. It's aim is not to improve test coverage, but rather put in place the infrastructure required to allow tests to be written:

• Create appropriate test projects
  • Including client side
  • Including integration tests
  • Setting up music store
• Build script(s) to execute said tests that work cross-platform

Extension/Hook Points:

We have various extensibility points that we are working on for Glimpse - part of the concept of Glimpse as a platform. We have been bitten in the past by not implementing enough concrete examples/implementations on top of these, that we found issues when people started to really build out on top of them. These involves both client and server side work and can priorities these as we go along. The benefit of this work is that it will actually help plugin authors with real code they can look at, additionally it can help drive the documentation.

[nikmd23]

Update/Work in progress: Welp, the state of testing in .NET kind of sucks right now.

Testing Framework

@xunit added support for Beta 6 today, but we are already on Beta 7. @bradwilson has started a branch to get Beta 7 support up and running, but due to problems with Beta 7, it doesn't appear to be working. We may need to revisit this one in a few weeks.

Mocking Framework

I should have realized this sooner, but currently there are no mocking frameworks that support CoreCLR. (Including @nsubstitute.) Not all tests will need/require mocks though. We could either:

• Hand write all our mocks. 😷 This makes me sick.
• Only support running our tests on Windows. 😞 This makes me sad.
• Mark tests that don't leverage mocks with an [xPlat] trait attribute. [Trait]'s are sealed in xUnit 2, so we'll need to either use the [Trait("xPlat", true)] syntax, or create an ITraitDiscoverer (example here and here) that makes the [xPlat] syntax possible. 😕 This isn't a great solution, but it will work for now. Hopefully we get xPlat mocking frameworks soon enough.

Edit: It looks like NSubstitute might be getting closer to a version that runs on core.

Auto Mocking

Clearly we can't use @AutoFixture or auto-mocking if we can't use mocking in general. AutoFixture does do more than just auto mocking, but we'll have to determine if there's enough value without being coupled to a mocking framework.

Resolution

Wait until @xunit supports Beta 7, and then revisit.

[bradwilson]

Only support running our tests on Windows. 😞 This makes me sad.

dnx451 works on Mono.

[avanderhoorn]

Not that this really helps, but I wonder if with the advent of the pre-compiler hooks, we will see mocking frameworks tapping into this to create their proxies at compile time... have to think about that one @davidfowl @anurse @lodejard

[nikmd23]

That's true @bradwilson, thanks for reminding me.

[bradwilson]

Clearly someone is going to need to step up and fix Castle Proxy, because everybody is using the same engine. One fix could fix them all. :-p

[nikmd23]

And that work is already well underway. 😄

[jonorossi]

@bradwilson 😉 I've been working there slowly. xUnit,net is actually the thing currently blocking me at the moment castleproject/Core#90 (comment).

[avanderhoorn]

As an update to this... I just brought over the work that @rynowak did which sets up the ground work for our integration tests.