Frequently Asked Questions

General

Is SpecFlow an implementation of Cucumber?

SpecFlow is using the same Gherkin language to define your Scenarios as Cucumber. It has the same and sometimes more features than Cucumber but is limited to the .NET ecosystem.

Can SpecFlow replace unit tests?

Unit tests are a testing technique, so if you ask if you can replace unit tests that ensure technical excellence with SpecFlow, the answer is no. However, If you are thinking about the unit tests that check for functionality, you can probably use SpecFlow for them. But since unit tests are usually used for technical excellence, the easy answer is no.

Is it possible to introduce BDD during a project?

Yes, it is. People understand the purpose and benefits of BDD most of the time rather quickly. You need to consider that for BDD, typically, the automation part is a bit more complex because we are usually automating the application via an external interface.

Are there any tools/services recommended for the Discovery and Formulation phase?

For Discovery: Any tool/service that lets you collaborate in real-time. Examples would be:

For Formulation: We recommend our own SpecFlow Gherkin Editor. Details can be found here

Should developers use Scenarios as their source of the requirements?

No. From our experience, when you have only concrete examples for implementing, a developer needs to abstract how the feature works again. Developers need the conceptual definition of the requirement to implement it and use the examples to see if the implementation is doing the correct behavior.

In a typical Scrum iteration, when would the Requirement workshop and Example mapping usually take place?

As these meetings are about defining your user stories/product backlog entries, they need to happen before planning for a sprint/iteration. One possibility is to hold the Discovery and Example mapping sessions in your refinement meetings.

How would you approach getting everyone involved with tests earlier when this is a big shift from current workflow?

Don't talk about tests; start talking about concrete examples of how the application should behave. From our experience, there will always be questions for exceptional cases, and they need to be addressed. Convince them that it is better to answer these as early as possible and not when half of the features are already implemented. Depending on the answer, a more extensive rewrite might be needed.

Should the BDD approach only be used for a new project?

No. You can start with BDD at any time. You can start small by just thinking and writing down the behavior of the application with concrete examples. After that, you can go further with formulating them in Gherkin and somewhen automate them.

What is the difference between Cucumber and Gherkin?

Gherkin is the language you are using to write and formulate your scenarios. Cucumber is a tool to automate these scenarios on programming languages like Java, JavaScript, Ruby, and a lot more

Can you give recommendations to approach a Shift Left Testing approach with SpecFlow?

Check out this article.

Is it possible to mix SpecFlow tests and standard unit tests in the same VS project?

Our recommendation is to have two separate projects—one for unit tests and one for SpecFlow scenarios. SpecFlow integrates into the various test runners to be able to enable all its features. Combining SpecFlow tests and standard unit tests may cause some of these features not to function as expected. Before and After TestRun hooks are an example of this.

Is it possible to use SpecFlow+ LivingDoc for .NET Framework projects?

Yes, it can be used for .NET Framework projects and.NET Core or .NET projects. You only need a .NET Core runtime available to run SpecFlow+ LivingDoc.

What is the Price/Cost of SpecFlow?

SpecFlow is free and largely open source! Thanks to the backing of Tricentis. Just give it a go and enjoy our tools.

Who are the people behind SpecFlow?

Meet us on any of our webinars or social media. You can read more about us here.

Is SpecFlow actively maintained and supported?

SpecFlow and the rest of our tools are constantly under development. You can check out our GitHub project, our public roadmap, feature requests, and our webinars to learn about what we are currently working on.

Is SpecFlow suitable for DevOps testing?

SpecFlow supports a development technique (BDD) with a heavy focus on testing. The SpecFlow framework uses common unit test runners (nUnit, xUnit, and MSTest) for test execution of its .dlls. It can therefore be very nicely integrated into any build server or automated CI/CD pipeline.

What is SpecFlow?

SpecFlow allows .NET development teams to define, manage and execute automated acceptance tests as business readable specifications. SpecFlow is based on Gherkin language and aims to bridge the communication gap between domain experts and developers. Acceptance tests in SpecFlow follow the BDD paradigm of defining specifications with examples, so that they are also easily understood by non-technical users. Acceptance tests can then be tested automatically as needed, while their specification serves as a living documentation of the system. SpecFlow integrates with Visual Studio, but can be also used from the command line (e.g. on a build server). SpecFlow has a range of other products under the SpecFlow+ product range. These products add additional functionalities to SpecFlow, such as a Gherkin editor , advanced reporting (LivingDoc) and much more.

How do I install SpecFlow?

To install SpecFlow and SpecFlow+, you first need to install the SpecFlow Visual Studio extension.. Once you have done this, you can then add SpecFlow to each of your projects. For more details, see the Getting Started article, where you learn how to install and use SpecFlow and also see how sample project works with SpecFlow.

What is the difference between SpecFlow and SpecFlow+?

SpecFlow is open source and covers all your basic needs for managing, automating and testing specifications by example. SpecFlow+ is a series of extensions that introduce additional features, such as the Gherkin editor for Gherkin specifications, and living documentation. SpecFlow and SpecFlow+ products are free to use. We only ask your to register for a free account using a free Microsoft account.

Do you have a public product roadmap?

Please check our roadmap page for an updated overview of what is coming up and what the team is working on right now.

How can I contribute to SpecFlow's development?

Details on how you can contribute to SpecFlow's development can be found here. There are a number of ways you can contribute to SpecFlow such as working on bug fixes, feature requests, and writing documentation.

I have a feature request, will you implement it?

We appreciate any feedback on how you are using SpecFlow and how your experience could be improved. You can submit feedback on our feature request list. We will evaluate all suggestions we receive, and factor this feedback into future development cycles.

Who are the developers behind SpecFlow?

SpecFlow was originally created by a team at TechTalk, lead by Christian Hassa and Gaspar Nagy, but recently SpecFlow and its add on components, SpecFlow+, are owned and sponsored by Tricentis. Since open-source developers from around the world have contributed to the project; an overview can be found here. Please visit our about us page to get to know the team behind SpecFlow.

Can I use the SpecFlow name for my own projects?

We are always happy to see projects that extend SpecFlow’s feature set, and SpecFlow is open source to specifically encourage such contributions from the community. We have however reserved the rights to the name “SpecFlow” as well as the official SpecFlow logo. We need to retain control over this aspect of the project, as we share responsibility for supporting and maintaining SpecFlow, along with all other contributors in the community. When naming your project, one of our primary concerns is therefore that the name of your project avoids any confusion concerning project ownership and avoids creating the impression that your project is officially supported by us.

How do I enable tracing in SpecFlow?

You can enable traces for SpecFlow. Once tracing is enabled, a new SpecFlow pane is added to the output window showing diagnostic messages. To enable tracing, select Tools | Options | SpecFlow from the menu in Visual Studio and set Enable Tracing to 'True'.

Upgrade error : "Trace listener failed. -> The ScenarioContext...."

To resolve this error, make sure you have regenerated the .feature.cs files after upgrading. If you do not do this, you will receive this exception when accessing ScenarioContext.Current.

To regenerate these files:

1. Open a feature file in your solution. If you see a popup informing you that the feature files were generated with an earlier version of SpecFlow, click on Yes to regenerate these files. Depending on the size of your project, this may take a while.

2. If you are using an earlier version of Visual Studio, you need to force the feature files to be regenerated. Right-click on your project, and select Regenerate Feature Files from the menu.

Visual Studio error: "Missing [assembly:GeneratorPlugin] attribute"

If you are receiving this error, try setting the Generation Mode in SpecFlow to "OutOfProcess". To do so:

1. Select Tools | Options from the menu in Visual Studio.

2. Select SpecFlow from the list on the left.

3. Locate the Generation Mode setting and set it to "OutOfProcess".

Steps are not recognized even though there are matching step definitions

The SpecFlow Visual Studio integration caches the binding status of step definitions. If the cache is corrupted, steps may be unrecognised and the highlighting of your steps may be wrong (e.g. bound steps showing as being unbound). To delete the cache:

1. Close all Visual Studio instances.

2. Navigate to your %TEMP% folder and delete any files that are prefixed with specflow-stepmap-, e.g. specflow-stepmap-SpecFlowProject-607539109-73a67da9-ef3b-45fd-9a24-6ee0135b5f5c.cache.

3. Reopen your solution.

Visual Studio error: Cannot find custom tool `SpecFlowSingleFileGenerator`

If Visual Studio displays the error message Cannot find custom tool 'SpecFlowSingleFileGenerator' on this system, when right-clicking on a feature file and selecting Run Custom Tool, make sure the SpecFlow extension is installed and enabled. To enable the extension in Visual Studio, select Tools | Extensions and Updates..., select the "SpecFlow for Visual Studio" extension, then select Enable.

Unable to find plugin in the plugin search path when saving / generating feature files

SpecFlow searches for plugins in the NuGet packages folder. This is detected relative to the reference to TechTalk.SpecFlow.dll. If this DLL is not loaded from the NuGet folder, the plugins will not be found. A common problem is that the NuGet folder is not yet ready (e.g. not restored) when opening the solution, but TechTalk.SpecFlow.dll in located in the bin\Debug folder of the project. In this case, Visual Studio may load the assembly from the bin\Debug folder instead of waiting for the NuGet folder to be properly restored. Once this has happened, Visual Studio remembers that it loaded the assembly from bin\Debug, so reopening the solution may not solve this issue. The best way to fix this issue is as follows:

1. Make sure the NuGet folders are properly restored.

2. Close Visual Studio.

3. Delete the bin\Debug folder from your project(s).

4. Reopen your solution in Visual Studio.

How can I contact support?

If you have a general issue, suggestion or question concerning SpecFlow products, please visit our support page.

I have a feature request; where can I submit it?

Please direct all your feature requests here, where the SpecFlow community gets to add and votes on existing feature requests.

BDD

How would you best describe the tester role in BDD – would they concentrate on not-automated tests?

Yes, using BDD would enable them to have time for more manual testing like exploratory testing or release testing. They need less time to spend on regression testing because the scenarios are automated and can be executed whenever needed. But they will also get involved more earlier in the discussions of how a feature should work. With their unique views on requirements and thoughts about concrete examples, testers help identify missing information in the requirements. These examples can then be used for your scenarios.

Why do most of the people from the IT industry think BDD is a testing tool?

We think this happened because the automation part of BDD is the most accessible part of the three phases (Discovery, Formulation, Automation), and you get something concrete out of it. Discovery and Formulation are the complex parts that need adjustments on how you and your team are working.

Writing Scenarios

Is there an OR keyword?

No, there is not. The reason is that the intention of Gherkin is to make it possible for everyone to write understandable scenarios. If there were an Or, there would be branches on how you read a Scenario and add unnecessary complexity.

Do Scenarios and Features need to be completely independent?

Yes, they need to be independent. Every test runner is running scenarios in a different order.

Which is good practice when writing Scenarios? "When I perform an action," or "When a user performs an action."

The recommendation is to use the third person form like "When a user performs an action." The reason is that this makes it clear which person/role is the current user. When you write it in the first person, it is not clear which role you are in at the moment.

How do you do data-driven testing with an external source like CSV, Excel in Scenarios?

You can use SpecFlow.External Data for this. More information here.

Does Given in the Background replace the Given that would be in the Scenario?

No, Background steps are not replacing the Given steps in a scenario. First, the steps in the Background are executed, and then the steps in the Scenario.

If there are no Given steps, does that affect SpecFlow?

No. It is possible to have no Given steps in a scenario.

How many steps/lines should an ideal Scenario have?

It always depends on your application, but if you need to scroll to read a single scenario, you should try to make it shorter.

Is it acceptable to write a "Given When Then When Then" scenario?

No, this is a bad practice. Having multiple "When Then" blocks is a sign that you are testing various behaviors at once. You can probably put every "When Then" block into a different scenario. This makes it also easier to find out what part is broken if the Scenarios start failing.

What is the difference between a table and a Scenario Outline?

Scenario Outlines and Tables, also called Data Tables are two completely different things. Scenario Outlines give you the possibility to parameterize your Scenarios so that you don't have to duplicate a Scenario multiple times to showcase minor changes. Tables are a parameter for your step definitions. They give you the possibility to pass more parameters to a step definition in a more structured way. You can access the Table in a non-typesafe way. To get a more typesafe API, you can use the extension methods in SpecFlow.Assist namespace. You can read more about it here.

Is there a way to parameterize a feature the same way as a scenario outline?

No, there is not. What is possible is to parameterize all features/scenarios with the Target- Feature of the SpecFlow+ Runner.

Is it possible to use two tables in StepDefinition?

No, it is not possible. The main idea is that one step should represent one action. If a step's text indicates more actions than that, it is a sign that the step should be split up.

Automation

How can you have optional parameters in step definitions?

SpecFlow doesn't support optional parameters in the step definition methods. You have to write two methods, but you can simply call one from the other. 
  [Given(@"Add two numbers (.*) (.*) (.*)")]
  public void Testtheconditionwith(string a, string b, string c)
  {

  }

  [Given(@"Add two numbers (.*) (.*)")]
  public void Testtheconditionwith(string a, string b)
  {
     Testtheconditionwith(a,b,null)
  }

What is the Scenario Context, and in what ways can it be used?

It's the best way to share information across a scenario. E.g., you need to save a login name at the beginning of a Scenario and validate it at the end. You would use ScenarioContext to access that info throughout the Scenario.

How do you refactor a project that has step definitions that call on other step definitions?

The suggestions is to just call the method name itself. E.g: QuestionAskingStepDefinition.ThenTheSuggestionListShouldBeFound()

How and when is the ScenarioContext instantiated?

It is done automatically by the SpecFlow runtime. You only have to request it via constructor injection to get the correct instance.

How many Step definitions can there be for one PageObject class?

There is no limit on how many step definitions you can have.

How do you group your step definitions in classes?

We recommend grouping them by feature/area they are automating.

If an 'AND' is used at a step, the Visual Studio plugin creates another 'Given' attribute.

There are only Given/When/Then steps. An And step is always of the type in which block they appear. So, if you have And Steps after a Given steps, they are Given steps.

How can you execute Features and Scenarios in order?

SpecFlow does not influence the execution order of your scenarios. Your test runner handles this. Please check the documentation of your test runner to see if it is possible to run tests in a particular order.

Is it possible to create dependencies between tests?

No, this is not possible. Execution order is in the hands of the test runner (NUnit, xUnit, MSTest, SpecFlow+ Runner), and SpecFlow can't manipulate this. Currently, no runner supports this use case.

Can we pair a particular feature file with only one step definition class?

Technically yes, but it is not recommended at all. When you rename the feature title or move a scenario into another file, everything will break.

Why should we not use Thread. Sleep?

Because the duration of the action you are waiting can differ over time. Somewhen your scenarios will break because the time you wait isn't enough.

Should the application code and test code be in different solutions?

If possible, it is highly recommended that test code and application code are in the same solution.

Is it a good practice to have a separate Step Definition file for each Feature file?

No, and it is not necessary. Step definitions are global. This means you define it per project and not only for a single feature file.

What is the difference between the Driver pattern and the PageObject pattern?

These two patterns are solving different requirements. The Page Object Model is a pattern that is often used to abstract your Web UI with Selenium to automate it easier. More information here here. The Driver Pattern is an additional layer between your step definitions and your automation code. With it, you can easily reuse your automation logic. More information here.

Are there any hooks for Rules?

No, there aren't hooks for Rules.

What is the lifetime of fields/members of a step definition class?

Every Scenario that gets executed has its own instances of step definition classes. A step definition class and all its fields and members are alive and in memory as long as the Scenario in which they were used is executed. After a scenario ended, they are released and not used anymore.

Are there any drawbacks or advantages for specific test runners?

Every test runner has some unique features. Depending on your needs, choose the appropriate one. You can get an overview at here

Do we need to check in the feature.cs file?

No, you do not need to. The `feature.cs` files are generated when the project is built so that they would be generated in a CI/CD pipeline.

How do you regenerate the feature code-behind files when you are changing test runner?

Rebuild your project that contains SpecFlow and change the test runner. Everything will then regenerate

Can you have multiple Given/When/Then attributes attached to a single method in your step definitions class?

Yes, you can. This is especially useful to have slightly different formulations for the same automation. You can see an example here

If the step definitions are global, what does that mean for private state in the step definition classes, setup/mocking in BeforeScenario bound methods, etc.?

Every Scenario that gets executed has its own instances of step definition classes. A step definition class and all its fields and members are alive and in memory as long as the Scenario in which they were used is executed. After a scenario ended, they are released and not used anymore

Is there a limit for the column numbers in a table?

No, there is no limit as long as you are not using anonymous tuples. If you are using these, you are limited to 7 columns.

What are the advantages of using a custom class over the FeatureContext in context injection?

A custom class gives you type-safe access to your data. If you are using the Feature/ScenarioContext, you always need the correct string key and then cast to the right datatype to get to your data. This can be error-prone.

How can I use CreateSet for a complex data model?

This is not possible. If you get into this need, we suggest rechecking the Formulation of your table if all columns are essential. If it is, we recommend in as a first step to use CreateSet to convert a row of the table to a POCO for the row, and as the second step, convert the POCO to your entities/classes

How do I share step definitions between multiple projects?

You can configure in the specflow.json file in which assemblies SpecFlow should look for step definitions. More details are here

How do I write some messages to the test output?

Since SpecFlow 3.8, there is a dedicated API for writing messages to output regardless of which unit test runner you are using.It is called ISpecFlowOutputHelper, more information here

How do I initialize a Selenium WebDriver for a whole Feature?

One way to do it is this: 
              [Binding]
           public class SpecflowHooks
           {
               private static IWebDriver driver;
               private readonly IObjectContainer container;

               public SpecflowHooks(IObjectContainer container)
               {
                   this.container = container;
               }

               [BeforeFeature]
               public static void OneTime()
               {
                   driver = new ChromeDriver(…);
               }

               [BeforeScenario]
               public void SetUp()
               {
                   container.RegisterInstanceAs(driver);
               }

               [AfterScenario]
               public void TearDown()
               {
               }

               [AfterFeature]
               public static void FeatureTearDown()
               {
                   if (driver == null)
                       return;

                   driver.Close();
                   driver.Dispose();
                   driver = null;
               }
           }

What is the difference between the Before/After- Scenario and the Before/After- ScenarioBlock Hooks?

The current ScenarioBlock changes when you switch from one step type (Given/When/Then) to another. The hooks are called when this happens.

How can I retry a failed Scenario?

The easiest way is to use the SpecFlow+ Runner. It has a built-in retry functionality. You can find the documentation at here.

Is it possible to mark a step as failed and carry on running the test?

No, this is not possible.

How can I skip/ignore scenarios at runtime?

You need at least SpecFlow 3.1 to do it. 
                [Binding]
              public sealed class StepDefinitions
              {
                 private readonly IUnitTestRuntimeProvider _unitTestRuntimeProvider;

                 public CalculatorStepDefinitions(IUnitTestRuntimeProvider unitTestRuntimeProvider)
                 {
                       _unitTestRuntimeProvider = unitTestRuntimeProvider;
                 }

                 [When("your binding")]
                 public void YourBindingMethod()
                 {
                       _unitTestRuntimeProvider.TestIgnore("This scenario is always skipped");
                 }
              }

How can I switch test runner?

You need to switch the NuGet package that configures the test runner in your project. You find the list of them here

Is it possible to mark a step as failed and carry on running the test?

No, this is not possible.

SpecFlow+ Excel

When will SpecFlow+ Excel be available for SpecFlow 3?

We do not currently have a schedule for .NET Core support for SpecFlow+ Excel

SpecFlow+ Runner

What are the advantages of SpecFlow+ Runner?

Some of the key features of SpecFlow+ Runner are:
  • Support for multiple targets, allowing you to write a single test to target different environments (e.g. x86 and x64, various browsers).
  • Configuration file transformations, which can also be used in conjunction with targets. This allows you to transform your configuration file for different platforms or web browsers, or set up a separate database instance for each thread using placeholders
  • Advanced reporting options using CSHTML templates. You can configure the output to meet your specific needs, both by customizing the formatting and determining which data to include and how it should be laid out. 3 default templates are included to get you started that output your test reports as either HTML, JSON or XML.
  • Adaptive test scheduling mode priorities previously failing tests over stable tests based on your execution history. Note that this feature requires you to set up a SpecFlow+ Runner server.
  • Parallelisation and isolation options for multi-threaded test execution. You can isolate threads by AppDomain, SharedAppDomain or Process.

I have already activated a SpecFlow account, but the SpecFlow+Runner is still asking me to sign up

1. Add the following system environment variable in windows Variable name: SPECFLOW_PLUS_USE_PROXY_CREDENTIALS Variable value: 1

2. Restart the application If you run the tests with Visual Studio please restart your Visual Studio (exit, open again). This is necessary to pick up the new environment variable setting.

3. Run the SpecFlow+Runner tests again

I am getting a proxy error after signing up for a SpecFlow account, what should I do?

To resolve this issue: 1. Add the following system environment variable in windows Variable name: SPECFLOW_PLUS_USE_PROXY_CREDENTIALS Variable value: 1

2. Restart the application If you run the tests with Visual Studio please restart your Visual Studio (exit, open again). This is necessary to pick up the new environment variable setting.

3. Run the SpecFlow+Runner tests again

Issue due to having multiple solutions with different versions of SpecFlow+ installed

When loading a solution containing SpecFlow+, Visual Studio caches the SpecFlow+ components. If you open a new solution containing a SpecFlow+ project, Visual Studio will retain the cached version of the SpecFlow+ components in memory. If the version used by the two projects is different, this means that there will be a version mismatch. You can avoid this issue by either upgrading all your solutions to the same version of SpecFlow+, or by restarting Visual Studio to flush the cache before opening the new solution.

Tests are not displayed in the Test Explorer window when using SpecFlow+ Runner

Note: As of Visual Studio 2017 15.7 the temporary files are no longer used. The following only applies to earlier versions of Visual Studio. The Visual Studio Test Adapter cache may also get corrupted, causing tests to not be displayed. If this happens, try clearing your cache as follows: 1. Close all Visual Studio instances

2. Navigate to your %TEMP%\VisualStudioTestExplorerExtensions\ folder and delete any sub-folders related to SpecFlow/SpecRun, i.e. that have "SpecFlow" or "SpecRun" in their name.

3. Reopen your solution and ensure that it builds.

Visual Studio 2015: Tests are not displayed in the Test Explorer window when using SpecFlow+ Runner

Visual Studio 2015 handles solution-level NuGet packages differently (those registered in the .nuget\packages.config file of the solution). As a result, solution-level NuGet packages must be listed in the projects that use them, otherwise Test Explorer cannot recognise the test runner. To fix this issue, either re-install the SpecFlow+ Runner NuGet packages, or add the dependency on the SpecRun.Runner package ( package id="SpecRun.Runner" version="1.2.0" ) to the packages.config file of your SpecFlow projects. You might need to restart Visual Studio to see your tests.

SpecFlow Gherkin Editor

Why should I use the Gherkin Editor?

The Gherkin Editor enables you to quickly collaborate, draft and share Gherkin feature files with your team and business stakeholder. The editor allows organizing all your Gherkin feature file drafts in just one place and simplifies the collaboration with your non-technical team members and business stakeholders.

How can I create an account for the Gherkin Editor?

You can create an account by clicking on Create an account on the top right corner of the editor and sign in with your Microsoft account. If you don't have a Microsoft account yet, you are able to create one for free during the sign-in process. In case you use a corporate or student account and run into admin consent issues, please use your personal account instead or send this link to your admin to grant you access.

Can I use my existing SpecFlow account for the Gherkin Editor?

Yes, only one SpecFlow account is required to use all our SpecFlow+ products. Simply click on Sign In on the top right corner of the Gherkin Editor and use the same Microsoft account you used for other SpecFlow+ products

How can I store a feature file online?

You can save and store feature files associated with your SpecFlow account by clicking the Sign in to save button on the top right of the Gherkin Editor.

How can I share my feature files in the Gherkin Editor?

You can share your feature files by clicking the Share button on the top right corner of the Gherkin Editor. Please note this is only available to users who have created a free account and are signed in. In general, you can choose between three access levels: 1. "Only me" ~ only you can view and edit the feature file with this link.

2. "Anyone with the link" ~ anyone with the link can view and edit the feature file.

3. "Anyone with the link - read only" ~ anyone with the link can only view the feature file.

Is the Gherkin Editor free to use?

Yes, the Gherkin Editor is free to use and doesn't require an account sign-up. Only for saving and sharing feature files, a free account sign-up is required.

How can I upload an existing feature file?

There is no upload feature in the Gherkin Editor yet. The simplest way to get your existing feature files into the Gherkin Editor is to manually copy/paste them into the editor view and then save them.

How can I delete my feature files?

Click on the Feature Files button at the top of the Gherkin Editor to view all your feature files. From here you can delete, download and view the access level for each feature file.

What is a Gherkin Editor?

In general, a Gherkin Editor is a text editor that allows you to write feature files using Gherkin syntax. Gherkin is a "Business Readable" language created especially for behavior descriptions. Its simplicity allows non-technical users to write specifications and it also allows you to remove logic details from behavior tests. Gherkin serves two purposes: serving as your project's specification/documentation and automated tests. The SpecFlow Gherkin Editor gives you additional tools to assist you in writing better Gherkin and also gives you the ability to share Gherkin feature files so you can collaborate with your team more easily.

SpecFlow+ LivingDoc

What is SpecFlow+LivingDoc?

SpecFlow+LivingDoc is a set of tools that allows you to share and collaborate on Gherkin Feature Files with stakeholders who may not be familiar with developer tools. There are two ways to produce living documentation depending on how you want to use and share it. You have the option to use the Azure DevOps extension which helps you build living documentation in Azure DevOps environment or use the LivingDoc Generator which is a stand-alone generator with no external dependencies. Please check the documentation page for more info.

How can I share the generated living documentation?

If you generated your LivingDoc using the CLI tool , the output file will be an HTML file with no external dependencies, therefore you can share this HTML file as you wish. Please check our documentation page for a few ideas on how to share LivingDoc.

What’s the difference between SpecFlow+ LivingDoc Generator and the SpecFlow+ LivingDoc Azure DevOps extension?

The LivingDoc Azure DevOps extension integrates into the Azure DevOps environment and allows you to build, view, and share LivingDoc. The LivingDoc Generator on the other hand, is a stand-alone LivingDoc generator with no external dependencies, it allows you to generate LivingDoc HTML files that are not tied to particular environment. Please check our documentation to learn more.

Do I need to use Microsoft Azure DevOps to generate LivingDoc?

You can generate living documentation by using the CLI tool which is a stand-alone product with no external dependencies. You can find all the information you need regarding how to create, and use LivingDoc here.

My JSON file name is FeatureData.json instead of TestExecution.json, what should I do?

This indicates that you are using an older version of SpecFlow+LivingDoc. Please check our migration guide here to upgrade to the latest version.

SpecMap

List of known issues

  • It is not possible to drag & drop items from the backlog query to the map in TFS when using Edge.
  • Issues can arise when using templates in languages other than English. This may also result in an empty default query. You can however define your own queries and use these instead.
  • SpecMap requires certain fields to be present in your template; if the fields are missing, errors will occur. This means that custom templates are not currently supported.
  • Synchronization issues can occur in setups with high latency, e.g. where the server is in a different country from the users. This is more likely to occur using the on-premise (TFS) extension. Please open a support request if you are experiencing long delays, particularly when receiving messages that the map has been updated and your changes cannot be saved as a result.
  • SpecMap is not fully compatible with the new navigation layout in Azure DevOps (VSTS) (currently only available as a preview). The backlog queries are empty when not using the default team. A workaround is to define your own queries.

    How is SpecMap licensed?

    In order to use SpecMap, you need to sign up for a free personal SpecFlow account. SpecMap is free of charge and will remain free forever.

    Is it possible to restrict access to specific maps per user?

    Access to projects and work items is determine by a user’s TFS project/work item permissions. Users without permission to access a project cannot access the maps in that project. If a user has access to a project, but not to certain work items, the user can browse the maps, but will not see the details (title, etc) of the mapped work items they are not permitted to access. There is currently no means of restricting access to a particular map outside of these options. We intend to add access settings for maps in the future, allowing you to grant read-only access or deny access to specific maps.

    Is it possible to use custom TFS templates with SpecMap?

    Since January, 2020 SpecMap is free forever. To properly list SpecMap as a free extension in the Azure DevOps marketplace we've changed the configuration of the extension and removed the Paid flag in version 0.5.109. This configuration change of the flags is not automatically applied when SpecMap is installed on an on-premise Azure Devops Server (formerly Team Foundation Server). Users using any SpecMap version prior to 0.5.109 will see the following an error message In order to fix this issue the Flags value of the extension has to be manually updated in the Azure Devops Database, using the following steps: 1. Connect to the Azure Devops Server Configuration Database

    2. Create a backup before making any changes

    3. Locate the [Gallery].[tbl_Extension] table

    4. Locate the row with the [ExtensionName] "SpecMap"

    5. Update the value of the [Flags] from 276 (Public, Paid, Validated) to 260 (Public, Validated)

    6. Attempt to update again SpecMap which should succeed now

    Are external network connections required for SpecMap on TFS ("offline")?

    SpecMap only communicates with the server where the extension is installed. This is either Azure Devops or the on-premise TFS server where it is hosted. All data is stored on this server, and SpecMap does not send nor store data elsewhere. External network connections are only used by the monitoring functions, which connect to an external Azure AppInsights service. However, this is not required for SpecMap to function. We only collect information on users/and usage, which can help us troubleshoot issues with SpecMap.

    SpecFlow Account

    Can I use other platforms than Microsoft to sign up for a SpecFlow account?

    Microsoft is the only supported platform to use to sign-up for a SpecFlow account. But, we are considering other options at the moment and there is a feature request for it.

    How do I sign out/deactivate a SpecFlow account?

    We currently do not have this feature in our product range. Please reach out to support at [email protected] and we will take care of it for you.

    Where is the user data for the SpecFlow account stored?

    The user data is stored in Azure, in the West Europe data center region (Netherlands, EU), you can read more information here

    Where can I find your privacy policy?

    Our privacy policy can be found here.

    How can I delete my SpecFlow account?

    If you would like to delete your SpecFlow account, please email us at [email protected]. Note that deleting your account will prevent you from using any of the licensed products (SpecFlow+ Runner, SpecFlow+ LivingDoc, SpecMap). Deleting your account will not affect any data stored locally in Azure DevOps. Even after your account is deleted, you will still see the SpecFlow+ LivingDoc and SpecMap entries in Azure DevOps. To remove these entries, you need to uninstall the extensions. This will also remove any data specific to the extension (i.e. your maps in SpecMap and your living documentation in SpecFlow+ LivingDoc).

    Do I need to sign-up for a SpecFlow account if I have an existing valid license key?

    No, you don't need to do anything right now. Only once the license keys expire each of your team members will be asked to create a free personal SpecFlow account.

    What kind of permission are required for creating a SpecFlow account?

    We request the minimum possible permission set necessary to authenticate the user with a Microsoft account, which consists of:

    - "Sign you in and read your profile" The "User.Read" permission is required by the SpecFlow account to allow users to sign-in to the SpecFlow application, and allows the SpecFlow application to read the basic profile information of the signed-in user. More info here

    - "Maintain access to data you have given it access to" This permission appears on all Microsoft consent screens automatically and it is not used by the SpecFlow account. More info here

    What if I am working in an offline environment?

    We no longer give out offline licenses.

    Do I need to create a SpecFlow account to use SpecFlow+ Runner on a build server

    SpecFlow+ Runner recognizes the most popular build servers and does not require a product activation when running in build server mode.

    How can I sign up for a free SpecFlow account?

    The first time you use one of our products, you will be asked to sign in with your Microsoft account and set up your SpecFlow account. Once you have set up your free SpecFlow account, you can use it to sign in all SpecFlow+ products. Learn more about the sign up here.