The commonality between these tests is they were all making use of an attribute that extended ExceptionBaseAttribute defined in the UnitTesting namespace. This extension compared the exception message, rather than just the exception type as provided by the vanilla attributes.
My development machine was running VS 2013 the build server VS 2012. After a bit of digging it was clear I was looking at a breaking change in the VS 2012 test runner.
So what could I do about it?
- Upgrade the build server? Not viable
- Ignore the tests? Plain stupid
- Toggle exception attributes with preprocessor directives? Hacky
There was a fourth option. MSTest legacy mode.
To enable it I first had to create a solution level file with a .runsettings extension, of which I placed the following contents:
This was then checked in and referenced in the build definition under Automated Tests -> Test Source -> Run Settings:
From this point onwards all tests were running as green as ever.
Perhaps this helped you? Have you experienced any MSTest breaking changes?