# Build Servers You can set up SpecFlow+ Runner to execute your tests whenever a build is created on your build server. **Note:** When running tests on a build server with SpecRun.exe (legacy), use the `buildserverrun` [command line option](../Usage/SpecFlowPlus-Runner-Command-Line.md). For general information on configuring builds, refer to the vendor-specific documentation. The following sections only cover configuring the build to execute your SpecFlow+ tests, not the full configuration process: ## SpecFlow+ and TFS/VSTS By default, TFS looks for a profile named TFS.srprofile to execute your tests; if none is found, TFS uses Default.srprofile instead. If you have changed the name of your profile, you need to enter the name of your profile in your [runner settings](Configuring-SpecFlowPlus-Runner-Visual-Studio-Test-Explorer-Integration.html#specflow-runner-settings) (`` element) and enter the path to your `.runsettings` file in the **Run Settings File** field. ### SpecFlow+ LivingDoc If you are using TFS/VSTS to build your projects, you may want to includes a step to [build your living documentation|Generating Documentation](https://docs.specflow.org/projects/specflow-livingdoc/en/latest/Generating/Building-the-Documentation.html) using SpecFlow+ LivingDoc. ### TFS 2015+/VSTS *Note: General information on running tests with your builds in TFS can be found [here](https://www.visualstudio.com/en-gb/docs/test/continuous-testing/getting-started/getting-started-with-continuous-testing).* To configure your build process to execute tests using SpecFlow+ Runner: 1. Open TFS and switch to your build definition. 1. Click on **Add build step** to add a new step to the build definition. Click on **Test** in the list of categories and click on **Add** next to "Visual Studio Test". 1. Configure the build step as follows: * Enter the path to your **Test Assembly** (the DLL containing your compiled specifications project with the test bindings). * If you are using both MSTest and SpecFlow+ Runner, enter the path to NuGet package folder in the **Path to Custom Test Adapters** field. If you do not enter the path here, only your MSTest tests will be executed as TFS cannot find the SpecFlow+Runner test adapter. * If you have renamed your .srprofile file (i.e. the name is not `TFS.srprofile` or `Default.srprofile`), ensure the profile is specified in your [.runsettings file](Configuring-SpecFlowPlus-Runner-Visual-Studio-Test-Explorer-Integration.md) and enter the name of the .runsettings file in the **Run Settings File** field. 1. Save your changes. 1. You may also want to add an additional build step to generate living documentation from your feature files, see [Generating Documentation](https://docs.specflow.org/projects/specflow-livingdoc/en/latest/Generating/Generating-Documentation.html). Once the test run is complete, links to the report file and logs generated by SpecFlow+ Runner are available from the test run as attachments. ### TFS and XAML To configure your build process in TFS with XAML: 1. Define your build definition as described [here](https://msdn.microsoft.com/en-us/library/ms181716%28v=vs.120%29.aspx) 1. In order for the tests to execute, the binaries (DLLs in the various SpecFlow packages) need to be available to the build agent. Either: * Enable NuGet restore, in which cases the files are downloaded automatically * Check in the DLLs in the corresponding SpecFlow, SpecRun and SpecFlow+ packages. These DLLs are located in the corresponding sub-folders of your solution's packages directory. 1. Enter the name of your .srprofile file in the **Run Settings File** field if the name is not `TFS.srprofile` or `Default.srprofile` (see above). #### Known Issues * The build agents cache the test adapters, which means that the last test adapter to be used is used for each build. You can thus only use a single SpecFlow+ Runner version with the same build agent, as the cached version is always used. If you want to use a different version of SpecFlow+ Runner for different builds, you need to define separate build agents. * Upgrading SpecFlow to a newer version requires a restart to purge the cache. ## SpecFlow+ and TeamCity Information on on configuring build steps in TeamCity can be found [here](https://www.jetbrains.com/help/teamcity/visual-studio-tests.html). Select `VSTest` as your test engine type and configure your build step as described in the TeamCity documentation. You may also want to install the custom logger (the link is on the documentation page). ## SpecFlow+ and Jenkins *Note: General information on installing Jenkins can be found [here](https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins).* The following steps assume that you are using the [Git plugin](https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin) to handle your source files. You will also need to install the [MS Build plugin](https://wiki.jenkins-ci.org/display/JENKINS/MSBuild+Plugin). You probably also want to install the [HTML Publisher plugin](https://wiki.jenkins-ci.org/display/JENKINS/HTML+Publisher+Plugin) to handle reports. If Visual Studio is not installed on the build server, you will also need to install [nuget.exe](https://dist.nuget.org/win-x86-commandline/latest/nuget.exe) and the [Agents for Visual Studio](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser#install). To configure your build process in Jenkins to execute tests using SpecFlow+ Runner: 1. Start the Jenkins web interface. 1. Click on **New Item** to add a new project if you have not already set up a project: 1. Enter a name for your project. 1. Choose "Freestyle project" from the list. 1. On the Jenkins main screen, select **Manage Jenkins** and then **Global Tool Configuration**. 1. Add a configuration for MSBuild. Give it a meaningful **Name** (e.g. “MSBuild”) and enter the **Path to MSBuild**. 1. Add a configuration for VSTest as well, including the **Path to VSTest**. 1. Select **Configure** From the project's main page. 1. Enable **Git** under **Source Code Management**. 1. Enter your **Repository URL**, e.g. “https://github.com/Me/MyProject.git”. You can also enter your login credentials. 1. Specify which branch to build under **Branches to build**, e.g. “\*/master”. 1. Enable **Delete workspace before build starts** under **Build Environment**, otherwise your test results may be incorrect due to the existing TRX files. 1. Click on the **Advanced** button. 1. Click on **Add** next to **Patterns for files to be deleted**: Exclude the following: .git/\*\*, TestResults, TestResult/\*.html 1. Click on **Add build step** under **Build** and select **Execute Windows batch command**. 1. Add the following **Command**: “C:\PATH\nuget.exe” restore “C:\PATH\MySpecs.sln” Replace “PATH” with the full path to nuget.exe and your solution. This ensures that your NuGet packages are restored each build. 1. Select the MSBuild configuration you defined in the **MSBuild Version** field under **Build a Visual Studio project or solution using MSBuild**. 1. Enter the path to your solution in the **MSBuild Build File** field. 1. Configure your unit tests to run with VSTest.console: * Select the configuration you defined earlier in the **VStest Version** field. * **Test Files**: Enter the path to the assemblies to be tested here. * **Specify a logger for test results**: Select trx. * **Command Line Arguments**: Enter `/TestAdapterPath:”C:\PATH\packages”` (replace “C:\PATH\” with the path to your solution’s directory). ## SpecFlow+ Execution Reports (optional) The SpecFlow+ execution reports are stored in the TestResults directory of your project. To be able to easily access these reports from the project overview in Jenkins: 1. Under **Post-build Actions**, click on **Add post-build action** and select “Publish HTML reports”. 1. Enter the path to your project’s TestResults directory under **HTML dir to archive**. 1. Set the **Index Pages** to “ProjectName_ProfileName\*.html” (replace “ProjectName and ProfileName with the name of your project and profile respectively) to reflect the SpecFlow+ report naming convention. The results of your tests are also visualized by the Jenkins interface, e. g. in the build history and graphs. This information is taken from the TRX file generated by MSTest, which needs to be published as well: 1. Under **Post-build Actions**, click on **Add post-build action** and select “Publish MSTest test result report”. 1. Enter “\*\*/TestResults/\*.trx” in the **Test report TRX file** field under **Publish MSTest test result report**. ## SpecFlow+ and AppVeyor *Note: General information on AppVeyor can be found [here](https://www.appveyor.com/docs/).* To configure your AppVeyor build process to execute your test using SpecFlow+ Runner: 1. Create a new `appveyor.yml` file or open your existing file. 1. Add the following `before_build` section to restore the NuGet packages:
  before_build:
    - appveyor DownloadFile https://dist.nuget.org/win-x86-commandline/latest/nuget.exe
    - appveyor DownloadFile https://raw.githubusercontent.com/appveyor/ci/master/scripts/nuget-restore.cmd
    - nuget-restore MyProject.sln
Replace `MyProject.sln` with the full path to your solution, relative to the location of the appveyor.yml file.
3. Add the following `test_script` section to execute the tests and generate the test output for AppVeyor: 
  test_script:
    - ps: vstest.console /logger:Appveyor "AssemblyPath\MyAssembly.dll" /TestAdapterPath:"PackagesPath\packages"
Replace `AssemblyPath` with the path to your assembly and replace `MyAssembly.dll` with the name of your assembly.
Replace `PackagesPath` with the path to the packages folder of your solution.
4. Save `appveyor.yml` and use the file to build your application with AppVeyor. A sample project is available [here](https://github.com/techtalk/SpecFlow.Plus.Examples/tree/master/AppVeyor_Support).