The SpecFlow+ Runner is out of maintenance. Read more about it here.

Migrating away from the SpecFlow+ Runner

The last official version of SpecFlow thats supports the SpecFlow+ Runner is 3.9.40. If you add a reference to any newer version of SpecFlow in your project, then it is not guaranteed that the runner will continue to function as intended.

The information below outlines the differences between the options available as an alternative to the SpecFlow+ Runner and how to migrate your project to them safely.

Overview

Here is a simple overview that breaks down each alternative option to the SpecFlow+ Runner and its parallelisation and reporting features.

Runner	Parallelisation	Reporting
NUnit	Feature level	SpecFlow+ LivingDoc, NUnit report (prior to SpecFlow 3)
xUnit	Feature level	SpecFlow+ LivingDoc
MSTest	Feature level	SpecFlow+ LivingDoc, MsTest Test Execution Report (prior to SpecFlow 3)

Migration

NUnit

1. Packages

   • Remove the following packages from your project:
     • SpecRun.SpecFlow
   • Add the following packages from your project:
     • for SpecFlow to generate the code behind files discoverable by the NUnit execution engine
       • SpecFlow.NUnit
       • NUnit
     • for Test Discovery & Execution
       • NUnit3TestAdapter
       • Microsoft.NET.Test.Sdk

2. Parallelisation

   • NUnit offers parallel execution only at feature level when used with SpecFlow. This means that if you set the LevelOfParallelism to 2, then 2 features will execute their tests at the same time. This can be achieved by doing the following:

     • Add a new C# file to your project, for example AssemblyInfo.cs

       using NUnit.Framework;
       
       [assembly: Parallelizable(ParallelScope.Fixtures)]
       [assembly: LevelOfParallelism(2)]
       

     • Note: SpecFlow does not support scenario level parallelization with NUnit (when scenarios from the same feature execute in parallel). If you configure a higher level NUnit parallelization than “Fixtures” your tests will fail with runtime errors.

3. Reporting

• If you are using SpecFlow 3, then you can generate reports using SpecFlow+ LivingDoc
  • SpecFlow+LivingDoc for Azure DevOps: If your team uses Azure DevOps then we suggest installing our dedicated extension to help you generate and share LivingDoc within the familiar Azure DevOps interface.
  • SpecFlow+LivingDoc Generator: If you want to generate a self-hosted HTML documentation with no external dependencies so you have the freedom to share it as you wish, then we suggest the SpecFlow plugin and command-line tool.
• If you are using SpecFlow 2 or lower, Then see here.

xUnit

1. Packages
   • Remove the following packages from your project:
     • SpecRun.SpecFlow
   • Add the following packages from your project:
     • for SpecFlow to generate the code behind files discoverable by the xUnit execution engine
       • SpecFlow.xUnit
       • xUnit
     • for Test Discovery & Execution
       • xunit.runner.visualstudio
       • Microsoft.NET.Test.Sdk
2. Parallelisation
   • By default xUnit runs all SpecFlow features in parallel with each other. No additional configuration is necessary.
3. Reporting

• you can generate reports using SpecFlow+ LivingDoc
  • SpecFlow+LivingDoc for Azure DevOps: If your team uses Azure DevOps then we suggest installing our dedicated extension to help you generate and share LivingDoc within the familiar Azure DevOps interface.
  • SpecFlow+LivingDoc Generator: If you want to generate a self-hosted HTML documentation with no external dependencies so you have the freedom to share it as you wish, then we suggest the SpecFlow plugin and command-line tool.

MSTest

1. Packages

   • Remove the following packages from your project:
     • SpecRun.SpecFlow
   • Add the following packages from your project:
     • for SpecFlow to generate the code behind files discoverable by the MSTest execution engine
       • SpecFlow.MSTest
       • MSTest.TestFramework
     • for Test Discovery & Execution
       • MSTest.TestAdapter
       • Microsoft.NET.Test.Sdk

2. Parallelisation

   • MSTest offers parallel execution only at feature level when used with SpecFlow. This can be achieved by doing the following:

     • Add a new C# file to your project, for example AssemblyInfo.cs

       using Microsoft.VisualStudio.TestTools.UnitTesting;
       [assembly: Parallelize(Scope = ExecutionScope.ClassLevel)]
       

       • Note: SpecFlow does not support scenario level parallelization with MsTest (when scenarios from the same feature execute in parallel). If you configure a higher level MsTest parallelization than ‘ClassLevel’ your tests will fail with runtime errors.

3. Reporting

• If you are using SpecFlow 3, then you can generate reports using SpecFlow+ LivingDoc
  • SpecFlow+LivingDoc for Azure DevOps: If your team uses Azure DevOps then we suggest installing our dedicated extension to help you generate and share LivingDoc within the familiar Azure DevOps interface.
  • SpecFlow+LivingDoc Generator: If you want to generate a self-hosted HTML documentation with no external dependencies so you have the freedom to share it as you wish, then we suggest the SpecFlow plugin and command-line tool.
• If you are using SpecFlow 2 or lower, Then see here.