diff --git a/docs/core/testing/index.md b/docs/core/testing/index.md index a956ef9a9b33a..61c5c640c9c0d 100644 --- a/docs/core/testing/index.md +++ b/docs/core/testing/index.md @@ -13,52 +13,21 @@ ms.assetid: 815ac74c-4bd9-4a94-a87c-78288b27c0e2 # Unit Testing in .NET Core -By [Steve Smith](http://ardalis.com) and [Bill Wagner](https://github.com/BillWagner) - -.NET Core has been designed with testability in mind, so that creating -unit tests for your applications is easier than ever before. This article -briefly introduces unit tests (and how they differ from other kinds of tests). -Linked resources demonstrates how to add a test project to your solution and -then run unit tests using either the command line or Visual Studio. +.NET Core has been designed with testability in mind, so that creating unit tests for your applications is easier than ever before. This article briefly introduces unit tests (and how they differ from other kinds of tests). Linked resources demonstrates how to add a test project to your solution and then run unit tests using either the command line or Visual Studio. ## Getting Started with Testing -Having a suite of automated tests is one of the best ways to ensure a -software application does what its authors intended it to do. There are -many different kinds of tests for software applications, including integration -tests, web tests, load tests, and many others. At the lowest level are -unit tests, which test individual software components or methods. Unit -tests should only test code within the developer’s control, and should -not test infrastructure concerns, like databases, file systems, or -network resources. Unit tests may be written using -[Test Driven Development (TDD)](http://deviq.com/test-driven-development/), -or they can be added to existing code to confirm its correctness. In -either case, they should be small, well-named, and fast, since ideally -you will want to be able to run hundreds of them before pushing your -changes into the project’s shared code repository. +Having a suite of automated tests is one of the best ways to ensure a software application does what its authors intended it to do. There are many different kinds of tests for software applications, including integration tests, web tests, load tests, and many others. At the lowest level are unit tests, which test individual software components or methods. Unit tests should only test code within the developer’s control, and should not test infrastructure concerns, like databases, file systems, or network resources. Unit tests may be written using [Test Driven Development (TDD)](http://deviq.com/test-driven-development/), or they can be added to existing code to confirm its correctness. In either case, they should be small, well-named, and fast, since ideally you will want to be able to run hundreds of them before pushing your changes into the project’s shared code repository. > [!NOTE] -> Developers often struggle with coming up with good names for their -> test classes and methods. As a starting point, the ASP.NET product -> team follows -> [these conventions](https://github.com/aspnet/Home/wiki/Engineering-guidelines#unit-tests-and-functional-tests). +> Developers often struggle with coming up with good names for their test classes and methods. As a starting point, the ASP.NET product team follows [these conventions](https://github.com/aspnet/Home/wiki/Engineering-guidelines#unit-tests-and-functional-tests). -When writing unit tests, be careful you don’t accidentally introduce -dependencies on infrastructure. These tend to make tests slower and -more brittle, and thus should be reserved for integration tests. You -can avoid these hidden dependencies in your application code by following -the [Explicit Dependencies Principle](http://deviq.com/explicit-dependencies-principle/) -and using [Dependency Injection](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/dependency-injection) -to request your dependencies from the framework. You can also keep your -unit tests in a separate project from your integration tests, and ensure -your unit test project doesn’t have references to or dependencies on -infrastructure packages. +When writing unit tests, be careful you don’t accidentally introduce dependencies on infrastructure. These tend to make tests slower and more brittle, and thus should be reserved for integration tests. You can avoid these hidden dependencies in your application code by following the [Explicit Dependencies Principle](http://deviq.com/explicit-dependencies-principle/) and using [Dependency Injection](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/dependency-injection) to request your dependencies from the framework. You can also keep your unit tests in a separate project from your integration tests, and ensure your unit test project doesn’t have references to or dependencies on infrastructure packages. Learn more about unit testing in .NET Core projects: -Try this [walkthrough creating unit tests with xUnit and the .NET CLI](unit-testing-with-dotnet-test.md). - -The XUnit team has written a tutorial that shows +* Try this [walkthrough creating unit tests with xUnit and the .NET CLI](unit-testing-with-dotnet-test.md). +* The XUnit team has written a tutorial that shows [how to use xUnit with .NET Core and Visual Studio](http://xunit.github.io/docs/getting-started-dotnet-core.html). - -If you prefer using MSTest, try the [walkthrough creating unit tests with MSTest and the .NET CLI](unit-testing-with-mstest.md). +* If you prefer using MSTest, try the [walkthrough creating unit tests with MSTest and the .NET CLI](unit-testing-with-mstest.md). +* For a additional information and examples on how to use selective unit test filtering, see [Running selective unit tests](../testing/selective-unit-tests.md). diff --git a/docs/core/testing/selective-unit-tests.md b/docs/core/testing/selective-unit-tests.md new file mode 100644 index 0000000000000..4b0757a9d811d --- /dev/null +++ b/docs/core/testing/selective-unit-tests.md @@ -0,0 +1,104 @@ +--- +title: Running selective unit tests | Microsoft Docs +description: Shows you how to use a filter expression to run selective unit tests with the dotnet test command. +keywords: .NET, .NET Core, unit test, selective test +author: smadala +ms.author: mairaw +ms.date: 03/22/2017 +ms.topic: article +ms.prod: .net-core +ms.devlang: dotnet +ms.assetid: 13d01272-bbf8-456c-a97a-560001d1a7f2 +--- + +# Running selective unit tests + +The following examples use `dotnet test`. If you're using `vstest.console.exe`, replace `--filter ` with `--testcasefilter:`. + +## MSTest + +```csharp +namespace MSTestNamespace +{ + using Microsoft.VisualStudio.TestTools.UnitTesting; + + [TestClass] + public class UnitTestClass1 + { + [Priority(2)] + [TestMethod] + public void TestMethod1() + { + } + + [TestCategory("CategoryA")] + [Priority(3)] + [TestMethod] + public void TestMethod2() + { + } + } +} +``` + +| Expression | Result | +| ---------- | ------ | +| `dotnet test --filter Method` | Runs tests whose `FullyQualifiedName` contains `Method`. Available in `vstest 15.1+`. | +| `dotnet test --filter Name~TestMethod1` | Runs tests whose name contains `TestMethod1`. | +| `dotnet test --filter ClassName=MSTestNamespace.UnitTestClass1` | Runs tests which are in class `MSTestNamespace.UnitTestClass1`.
**Note:** The `ClassName` value should have a namespace, so `ClassName=UnitTestClass1` won't work. | +| `dotnet test --filter FullyQualifiedName!=MSTestNamespace.UnitTestClass1.TestMethod1` | Runs all tests except `MSTestNamespace.UnitTestClass1.TestMethod1`. | +| `dotnet test --filter TestCategory=CategoryA` | Runs tests which are annotated with `[TestCategory("CategoryA")]`. | +| `dotnet test --filter Priority=3` | Runs tests which are annotated with `[Priority(3)]`.
**Note:** `Priority~3` is an invalid value, as it isn't a string. | + +**Using conditional operators | and &** + +| Expression | Result | +| ---------- | ------ | +| dotnet test --filter "FullyQualifiedName~UnitTestClass1|TestCategory=CategoryA" | Runs tests which have `UnitTestClass1` in `FullyQualifiedName` **or** `TestCategory` is `CategoryA`. | +| `dotnet test --filter "FullyQualifiedName~UnitTestClass1&TestCategory=CategoryA"` | Runs tests which have `UnitTestClass1` in `FullyQualifiedName` **and** `TestCategory` is `CategoryA`. | +| dotnet test --filter "(FullyQualifiedName~UnitTestClass1&TestCategory=CategoryA)|Priority=1" | Runs tests which have either `FullyQualifiedName` containing `UnitTestClass1` **and** `TestCategory` is `CategoryA` **or** `Priority` is 1. | + +## xUnit + +```csharp +namespace XUnitNamespace +{ + public class TestClass1 + { + [Trait("Category", "bvt")] + [Trait("Priority", "1")] + [Fact] + public void foo() + { + } + + [Trait("Category", "Nightly")] + [Trait("Priority", "2")] + [Fact] + public void bar() + { + } + } +} +``` + +| Expression | Result | +| ---------- | ------ | +| `dotnet test --filter DisplayName=XUnitNamespace.TestClass1.Test1` | Runs only one test, `XUnitNamespace.TestClass1.Test1`. | +| `dotnet test --filter FullyQualifiedName!=XUnitNamespace.TestClass1.Test1` | Runs all tests except `XUnitNamespace.TestClass1.Test1`. | +| `dotnet test --filter DisplayName~TestClass1` | Runs tests whose display name contains `TestClass1`. | + +In the code example, the defined traits with keys `Category` and `Priority` can be used for filtering. + +| Expression | Result | +| ---------- | ------ | +| `dotnet test --filter XUnit` | Runs tests whose `FullyQualifiedName` contains `XUnit`. Available in `vstest 15.1+`. | +| `dotnet test --filter Category=bvt` | Runs tests which have `[Trait("Category", "bvt")]`. | + +**Using conditional operators | and &** + +| Expression | Result | +| ---------- | ------ | +| dotnet test --filter "FullyQualifiedName~TestClass1|Category=Nightly" | Runs tests which has `TestClass1` in `FullyQualifiedName` **or** `Category` is `Nightly`. | +| `dotnet test --filter "FullyQualifiedName~TestClass1&Category=Nightly"` | Runs tests which has `TestClass1` in `FullyQualifiedName` **and** `Category` is `Nightly`. | +| dotnet test --filter "(FullyQualifiedName~TestClass1&Category=Nightly)|Priority=1" | Runs tests which have either `FullyQualifiedName` containing `TestClass1` **and** `Category` is `CategoryA` **or** `Priority` is 1. | diff --git a/docs/core/tools/dotnet-test.md b/docs/core/tools/dotnet-test.md index cb3c160f882f5..6f21d691431b2 100644 --- a/docs/core/tools/dotnet-test.md +++ b/docs/core/tools/dotnet-test.md @@ -50,7 +50,7 @@ List all of the discovered tests in the current project. `--filter ` -Filters out tests in the current project using the given expression. For more information on filtering support, see [Running selective unit tests in Visual Studio using TestCaseFilter](https://aka.ms/vstest-filtering). +Filters out tests in the current project using the given expression. For more information, see the [Filter option details](#filter-option-details) section. For additional information and examples on how to use selective unit test filtering, see [Running selective unit tests](../testing/selective-unit-tests.md). `-a|--test-adapter-path ` @@ -92,9 +92,45 @@ Run the tests in the project in the current directory: Run the tests in the `test1` project: -`dotnet test ~/projects/test1/test1.csproj` +`dotnet test ~/projects/test1/test1.csproj` + +## Filter option details + +`--filter ` + +`` has the format `[|&]`. + +`` is an attribute of the `Test Case`. The following are the properties supported by popular unit test frameworks: + +| Test Framework | Supported properties | +| :------------: | --------------------------------------------------------------------------------------------------------- | +| MSTest |
  • FullyQualifiedName
  • Name
  • ClassName
  • Priority
  • TestCategory
| +| Xunit |
  • FullyQualifiedName
  • DisplayName
  • Traits
| + +The `` describes the relationship between the property and the value: + +| Operator | Function | +| :------: | --------------- | +| `=` | Exact match | +| `!=` | Not exact match | +| `~` | Contains | + +`` is a string. All the lookups are case insensitive. + +An expression without an `` is automatically considered as a `contains` on `FullyQualifiedName` property (for example, `dotnet test --filter xyz` is same as `dotnet test --filter FullyQualifiedName~xyz`). + +Expressions can be joined with conditional operators: + +| Operator | Function | +| :------: | :------: | +| `|` | OR | +| `&` | AND | + +You can enclose expressions in paranthesis when using conditional operators (for example, `(Name~TestMethod1) | (Name~TestMethod2)`). + +For a additional information and examples on how to use selective unit test filtering, see [Running selective unit tests](../testing/selective-unit-tests.md). ## See also -* [Target frameworks](../../standard/frameworks.md) -* [Runtime IDentifier (RID) catalog](../rid-catalog.md) +[Frameworks and Targets](../../standard/frameworks.md) +[.NET Core Runtime IDentifier (RID) catalog](../rid-catalog.md) diff --git a/docs/toc.md b/docs/toc.md index b0e9466d24a8e..7d88efc6064a4 100644 --- a/docs/toc.md +++ b/docs/toc.md @@ -38,6 +38,7 @@ ## [Unit Testing](core/testing/index.md) ### [Unit testing with dotnet test and xUnit](core/testing/unit-testing-with-dotnet-test.md) ### [Unit testing with dotnet test and MSTest](core/testing/unit-testing-with-mstest.md) +### [Running selective unit tests](core/testing/selective-unit-tests.md) ## [Versioning](core/versions/index.md) ### [.NET Core Support](core/versions/lts-current.md)