Bitslapping contest : LX vs Compile on Test integration in VS2005
Round 3: LX on offense, compile still recovering
See the previous rounds (1a, 1b, 2a, 2b) in this bitslap.
Dennis is pretty harsh on not giving me any points for great punches. The way he talks he can sell a refrigerator to an eskimo.
1. One stop shop
VS2005 offers you a single GUI to work from. Everything is integrated and is available out-of-the-box. Dennis has to bring in like a zillion separate frameworks, add-ins and other stuff. Moreover, he has to keep up with all the different versions, downloads, and hope that they work together.
2. Generate test code from any class or method
With VS2005 you can right-click any piece of code and generate unit test code.
You can pick any complete type or its members and create unit test code for those in a project of your choice, be it existing or new, C# or Visual Basic.
Through the settings button there is control over the naming conventions and whether all tests have documentation and Assert.Inconclusive statements inserted by default.
As an added bonus VS2005 can also generate reflection based code so you can test private members of your type, in those cases where you do not want to expose the members just for the sake of testability.
///A test for BeginReceiveMessage ()
public void BeginReceiveMessageTest()
VentriloConnection target = new VentriloConnection();
ProtocolTests.KillerApps_Gaming_Protocols_Ventrilo_VentriloConnectionAccessor accessor = new ProtocolTests.KillerApps_Gaming_Protocols_Ventrilo_VentriloConnectionAccessor(target);
Assert.Inconclusive("A method that does not return a value cannot be verified.");
The code fragment above uses the generated ...VentriloConnectionAccessor class that is able to call the members that are protected, internal or private. The signature and naming is the same as for the real object (VentriloConnection).
The generated code offers an excellent starting point for your unit tests.
3. Test timeouts
Since Dennis did not show me any evidence of NUnit tests running faster than VS2005, I'll discard that: "Objection, your honor, subjective evidence and hearsay". The reply from any decent judge would be: "Granted in full".
But, apparently NUnit and TestDriven.NET do not know timeouts. VS2005 has default timeouts configurable by this dialog:
and the TimeoutAttribute to overrule that per test.
4. Generic tests
Visual Studio 2005 has the notion of generic tests, where you can run any test harness to test your code or application. This could be a custom test harness to test your Windows Forms application or any command-line runable tool or program. Let's say you have created a Setup.exe program to bootstrap your .msi Windows installer program for your application. You create a generic test and run the setup with the desired arguments
You can specify additional files that need to be deployed, such as an answer file for the setup example. Also, you can influence the environment variables for the duration of the test run. And yes, these tests can also be terminated if they do not finish within the alloted time interval.
The test will fail or succeed depending on the return code of the application that is run. A 0 (zero) return code indicates success, non-zero failure.
5. Ordered tests
There is yet another test type, that allows you to bundle any type of test (be it another manual, unit, load, generic or web tests) into an ordered list. While this may seem weird from a unit testing perspective (why would you have the results of unit tests be significant to the outcome of others. Not really separation of tests). But if you take into account the other test types it is a true addition to your testing framework. Going back to the previous example, you can generate two generic tests: one for setup and one for uninstall. Run these in order, plus optional tests to check a correct and complete uninstall, and you have a pretty neat mechanism to automate the testing of installation packages.
You can specify that the ordered run should continue even if one of the test in the list has failed. I bet Dennis will be misusing unit tests for this.
Right, let's let Dennis take a look at this. This time I'll let him give the points and haven't given these myself. I think these are all winners.