Spec Flow and Friends 2

Ok so I said I was going to cover Machine Specifications this week but I thought a look at some spec flow tests in the wild so to speak might be helpful, and it gives me a chance to showcase a home project.

A long time ago now I wrote a small application that watched C’# solutions, when a save was made, the application built the solution and ran all the tests, and displayed the result in a popup window.  I have used this on many projects now, but have started to find problems with the way it operates, and decided to do a project rewrite, as the original was all legacy code, and I wanted to provide better support for the application.

I put together two features that covered the basic functionality I wanted,

  1. Feature: nTestRunner solution change
  2.     As a developer
  3.     In order to get rapid feedback
  4.     When I save a file, the program should be compiled and
  5.     all tests run and the results stored in a file in the
  6.     same format as nunit, so I can use beacons to view the
  7.     results of the test.
  8.  
  9.     The file watcher watches the solution file, and the immediate
  10.     project directories, when a change in these watched areas is
  11.     detected then the watcher is stopped, the build and test cycle
  12.     started, and the results written to the xml results file in nunit
  13.     format, if the runner display is set then it is activated with
  14.     the results from the build test cycle.
  15.     
  16. Scenario: with default configuration
  17.     Given the program is running with no argument
  18.     When a change event is received from the watcher
  19.     Then the watcher is switched off
  20.     And the build is triggered
  21.     And the tests are run
  22.     And the results are stored
  23.     And the watcher is restarted
  24.  
  25. Scenario: with test runner configuration set
  26.     Given the program is running with MSpec argument
  27.     When a change event is received from the watcher
  28.     Then only projects with the specified runner are tested
  29.  
  30. Scenario: with the display runner configuration set
  31.     Given the program is running with Growl argument
  32.     When a change event is received from the watcher
  33.     Then the runner display is called
  34.     And the runner display is given the test results

 

I have laid out the feature files in the same manor, this is my documentation, so there is a large amount of text, that is pure description, then we get to the scenarios, these the above file has been tweaked so that the each is scenario uses a similar syntax, this will assist me when I write the step definitions

  1. Feature: nTestRunner Program Startup
  2.     As a developer
  3.     In order to get rapid feedback
  4.     When I save a file, the program should be compiled and
  5.     all tests run and the results stored in a file in the
  6.     same format as nunit, so I can use beacons to view the
  7.     results of the test.
  8.  
  9.     This feature covers the startup of the program,
  10.     calling nTestRunner
  11.         starts the program up in default mode, it will scan
  12.         up the directory chain looking for a solution file,
  13.         starting at its current directory it is this file
  14.         that the applicaion uses to decide what is a test
  15.         project, and it is the file that is passed to msbuild,
  16.         it is assumed that all test files in the solution want
  17.         to be run, regredless of the test runner used, all
  18.         results are combined into an nunit result format.  
  19.         No display is made when the tests are run, only the file
  20.         is written to the same directory as the found solution file.
  21.         
  22.     calling nTestRunner -Path | -P [path to solution file]
  23.         starts the program up with the solution file path set
  24.  
  25.     calling nTestRunner -Test | -T [Test runner name]
  26.         starts the program up with only the specified test runner
  27.  
  28.     calling nTestRunner -Display | -D [Runner Display Name]
  29.         starts the program up with the specified display runner
  30.  
  31. Scenario: Startup without arguments
  32.     Given that the program is not running
  33.     When the program is run with arguments ''
  34.     Then the user sees text containing 'nTestRunner version 1.0'
  35.     And the user sees text containing 'Watching Files'
  36.     And the user sees text containing 'nTestRunner.sln'
  37.     And the user sees text containing 'nTestRunner.features.csproj'
  38.     And the user sees text containing 'nTestRunner.Spec.csproj'
  39.  
  40. Scenario: Startup with path arguments
  41.     Given that the program is not running
  42.     When the program is run with arguments '-Path,C:\TestSolution.sln'
  43.     Then the user sees text containing 'TestSolution.sln'
  44.     And the user sees text containing 'TestProject1.csproj'
  45.     And the user sees text containing 'TestProject2.csproj'
  46.  
  47. Scenario: Startup with test runner arguments
  48.     Given that the program is not running
  49.     When the program is run with arguments '-Test,MSpec'
  50.     Then the user sees text containing 'Running tests with MSpec'
  51.     
  52. Scenario: Startup with display arguments
  53.     Given that the program is not running
  54.     When the program is run with arguments '-Display,Growl'
  55.     Then the user sees text containing 'Displaying results in Growl'

This second file has not undergone any changes as yet, and is still in the raw form from the initial project outline, although it should not take much tweaking to get it into shape.

With the feature files in place, I get an overview of the project, and what I need to do to make everything work, I tend to only worry at this point about exceptions but to focus on the “happy path”, because that is what the user will care about, getting the happy path in the features, will ensure the application works, and give the user the quickest feedback and allows them to answer the question.

Does what I have asked for make sense?

Next step is to choose a feature to start on, in this case it was easy, I need the start-up features before I can even think about anything else, the step definitions are actually very simple consisting of just 3 steps

  1. [Binding]
  2. public class StartupStepDefinations
  3. {
  4.     TestConsole _console;
  5.     Runner _runner;
  6.  
  7.     [Given(@"that the program is not running")]
  8.     public void GivenThatTheProgramIsNotRunning()
  9.     {
  10.         _runner = null;
  11.     }
  12.  
  13.     [When(@"the program is run with arguments '(.*)'")]
  14.     public void WhenTheProgramIsRunWith(string arguments)
  15.     {
  16.         var args = arguments.Split(',');
  17.  
  18.         _console = new TestConsole();
  19.  
  20.         _runner = new Runner(args, _console);
  21.     }
  22.  
  23.     [Then(@"the user sees text containing '(.*)'")]
  24.     public void ThenTheUserSeesTextContaining(string expectedText)
  25.     {
  26.         Assert.Contains(expectedText, _console.Output);
  27.     }
  28. }

 

and that’s it, the only interesting item in the list is the “WhenTheProgramIsRunWith” step, as this splits the argument string at the comma to produce the argument array that is usually passed into a console application.

The test console is simply a class that has tales the place of the normal standard out for this application, the actual implementation will simply call console.write with any string given to it, this version however aggregates the strings sent to it which it returns when the output property is called.

I will next time cover machine specifications, and show how I judge when to move from this high level whole system tests to the lower level specification tests.

Advertisements

About Duncan Butler

Trying to be a very agile software developer, working in C# with Specflow, Nunit and Machine Specifications, and in the evening having fun with Ruby and Rails
This entry was posted in Uncategorized. Bookmark the permalink.

3 Responses to Spec Flow and Friends 2

  1. Darren Cauthon says:

    You might want to avoid using private members in your SpecFlow definitions. I recorded a short screencast on the subject a while back:

    • duncanbutler says:

      Darren thanks for the feedback and the screencast

      I only tend to use the context classes when my step definations start to flow across classes, I think the local storeage of data helps reading the step definations, especially when first starting out. But I totally agree that once you start to reuse step definations and have then across classes then using the context is a must, I also like the way you refactor you classes to improve navigation at the end of your demo.

  2. Darren Cauthon says:

    Ok, cool, just passing the idea along. 🙂 This is one of the ideas I’m always passing to developers when I see people tweeting or blogging about SpecFlow. I think the private member issue is a big “gotcha” that keeps .Net developers from moving past the demo SpecFlow use and into full-app SpecFlow testing.

    Thanks for your posts on MSpec, too. I’m getting into that myself now, and it was a good read.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s