Skip to content

Running selective unit tests documentation - #1776

Merged
guardrex merged 12 commits into
dotnet:masterfrom
guardrex:guardrex/selective-unit-tests
May 18, 2017
Merged

guardrex merged 12 commits into
dotnet:masterfrom
guardrex:guardrex/selective-unit-tests

Conversation

@guardrex

@guardrex guardrex commented Mar 22, 2017 •

Copy link
Copy Markdown
Contributor

Fixes #1489

@mairaw @BillWagner This seems like a good approach:

  • It doesn't clutter up the options on the dotnet test command with more --filter info. It places the option details in a section at the bottom of the topic out of the way titled Filter option details. It keeps the option details in that new section focused on the option details and doesn't include the full-blown examples.
  • It creates a doc over in the testing area to hold the remaining content with the lengthy examples. I took the original content (giving the original author credit) and reorganized the content to my liking.
  • I link to the testing area doc from the dotnet command --filter option and from the Filter option details section.

What da ya tinks?

If the structure of all of this looks fine, then my only remaining question is whether or not the testing area doc should be more of a walkthrough tutorial or should it be left as it is ... as more of a reference doc. I think it's ok as a ref doc. It's fairly straightforward the way Satya wrote it with some SillyRex:tm: changes.

btw- Rendering the table cell content that contains pipes as <code> HTML markup. I can't escape a pipe in a table cell.

@guardrex guardrex added this to the March 2017 milestone Mar 22, 2017
@guardrex guardrex self-assigned this Mar 22, 2017
@dnfclas

dnfclas commented Mar 22, 2017

Copy link
Copy Markdown

@guardrex,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

@mairaw

mairaw commented Mar 23, 2017

Copy link
Copy Markdown
Contributor

One of my intentions on adding this doc here is that we'd replace the aka.ms link that is used on the CLI tools to go here instead of the CLI repo (before it was to a blog post). So that's why I was planning to have the topic both covering the conceptual part as well as the examples.

@piotrpMSFT @livarcocc any thoughts on this?

@guardrex

Copy link
Copy Markdown
Contributor Author

@mairaw By "conceptual part," do you mean the content I have in the Filter option details section of the dotnet test doc? It could be in both places. I could leave that section over there and place it at the top of the new doc.

@mairaw mairaw removed this from the March 2017 milestone Mar 29, 2017
@guardrex guardrex removed the WIP label May 13, 2017

@rpetrusha rpetrusha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guardrex, this looks good. Again, a few nits.

Comment thread docs/core/testing/index.md Outdated
@@ -62,3 +62,5 @@ The XUnit team has written a tutorial that shows
[how to use xUnit with .NET Core and Visual Studio](http://xunit.github.io/docs/getting-started-dotnet-core.html).

If you prefer using MSTest, try the [walkthrough creating unit tests with MSTest and the .NET CLI](unit-testing-with-mstest.md).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

walkthrough, [Unit testing with dotnet test and MSTest](...)

Each of these items after the "Learn..." paragraph could be a bulleted list item; otherwise, they appear independent of one another.

| --- | --- |
| <code>dotnet test --filter "FullyQualifiedName~UnitTestClass1&#124;TestCategory=CategoryA"</code> | Runs tests which have `UnitTestClass1` in `FullyQualifiedName` **or** `TestCategory` is `CategoryA`. |
| `dotnet test --filter "FullyQualifiedName~UnitTestClass1&TestCategory=CategoryA"` | Runs tests which have `UnitTestClass1` in `FullyQualifiedName` **and** `TestCategory` is `CategoryA`. |
| `dotnet test --filter "(FullyQualifiedName~UnitTestClass1&TestCategory=CategoryA)|Priority=1"` | Runs tests which have either `FullyQualifiedName` containing `UnitTestClass1` **and** `TestCategory` is `CategoryA` **or** `Priority` is 1. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't recognized as a table cell because of the | operator. %7C may work.

| Expression | Result |
| --- | --- |
| `dotnet test --filter XUnit` | Runs tests whose `FullyQualifiedName` contains `XUnit`. Available in `vstest 15.1+`. |
| `dotnet test --filter Category=bvt` | Runs tests which has `[Trait("Category", "bvt")]`. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests which has --> tests which have

Comment thread docs/core/tools/dotnet-test.md Outdated
`--filter <EXPRESSION>`

Filters out tests in the current project using the given expression. For more information on filtering support, see [Running selective unit tests in Visual Studio using TestCaseFilter](https://aka.ms/vstest-filtering).
Filters out tests in the current project using the given expression. For option details, see the [Filter option details](#filter-option-details) section. For a additional information and examples on how to use selective unit test filtering, see [Running selective unit tests](../testing/selective-unit-tests.md).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"For more information" seems to me to be preferable to "For option details", since we're repeating "option details" in the title.
a additional --> additional

Comment thread docs/core/tools/dotnet-test.md Outdated

`--filter <EXPRESSION>`

`<Expression>` is of the format `<property><operator><value>[|&<Expression>]`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: is of --> has

@guardrex

Copy link
Copy Markdown
Contributor Author

@rpetrusha Updates complete. I found a few other little things, too. I did some additional 🌻table beautification 🌻; so in addition to checking the %7C table rendering, we'll need to look at how well these other tables render before we merge.

@rpetrusha rpetrusha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some additional comments, and unfortunately %7C didn't work.

Comment thread docs/core/tools/dotnet-test.md Outdated
`--filter <EXPRESSION>`

Filters out tests in the current project using the given expression. For more information on filtering support, see [Running selective unit tests in Visual Studio using TestCaseFilter](https://aka.ms/vstest-filtering).
Filters out tests in the current project using the given expression. For for information, see the [Filter option details](#filter-option-details) section. For additional information and examples on how to use selective unit test filtering, see [Running selective unit tests](../testing/selective-unit-tests.md).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for for --> for more

| ---------- | ------ |
| `dotnet test --filter Method` | Runs tests whose `FullyQualifiedName` contains `Method`. Available in `vstest 15.1+`. |
| `dotnet test --filter Name~TestMethod1` | Runs tests whose name contains `TestMethod1`. |
| `dotnet test --filter ClassName=MSTestNamespace.UnitTestClass1` | Runs tests which are in class `MSTestNamespace.UnitTestClass1`.<br>**Note:** The `ClassName` value should have namespace, so `ClassName=UnitTestClass1` won't work. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have namespace -> have a namespace

| `dotnet test --filter ClassName=MSTestNamespace.UnitTestClass1` | Runs tests which are in class `MSTestNamespace.UnitTestClass1`.<br>**Note:** The `ClassName` value should have namespace, so `ClassName=UnitTestClass1` won't work. |
| `dotnet test --filter FullyQualifiedName!=MSTestNamespace.UnitTestClass1.TestMethod1` | Runs all tests except `MSTestNamespace.UnitTestClass1.TestMethod1`. |
| `dotnet test --filter TestCategory=CategoryA` | Runs tests which are annotated with `[TestCategory("CategoryA")]`. |
| `dotnet test --filter Priority=3` | Runs tests which are annotated with `[Priority(3)]`.<br>**Note:** `Priority~3` is an invalid value as is int not a string. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as it int not?

| ---------- | ------ |
| <code>dotnet test --filter "FullyQualifiedName~UnitTestClass1&#124;TestCategory=CategoryA"</code> | Runs tests which have `UnitTestClass1` in `FullyQualifiedName` **or** `TestCategory` is `CategoryA`. |
| `dotnet test --filter "FullyQualifiedName~UnitTestClass1&TestCategory=CategoryA"` | Runs tests which have `UnitTestClass1` in `FullyQualifiedName` **and** `TestCategory` is `CategoryA`. |
| `dotnet test --filter "(FullyQualifiedName~UnitTestClass1&TestCategory=CategoryA)%7CPriority=1"` | Runs tests which have either `FullyQualifiedName` containing `UnitTestClass1` **and** `TestCategory` is `CategoryA` **or** `Priority` is 1. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

%7C didn't work. Possibly try & notation?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<code> tags might lick it. That was our go-to workaround for markdown escapes that don't work in inline code.

@mairaw

mairaw commented May 16, 2017

Copy link
Copy Markdown
Contributor

For this one, if we want to replace the link to https://aka.ms/vstest-filtering with the work being done in this PR, would we send them to the new topic or the dotnet test page?

@guardrex

Copy link
Copy Markdown
Contributor Author

@mairaw Does it depend on the context of the link?

Command-related 👉 docs/core/tools/dotnet-test.md
Unit testing-related 👉 docs/core/testing/index.md

... or perhaps both if both are relevant?

@mairaw

mairaw commented May 16, 2017

Copy link
Copy Markdown
Contributor

see the output of dotnet test --help @guardrex

@guardrex

Copy link
Copy Markdown
Contributor Author

@mairaw ah ... I thought you meant there was more than one. Shouldn't that go to the docs/core/testing/selective-unit-tests.md because it pertains to the filtering expression that would be used with the --filter option?

@guardrex

Copy link
Copy Markdown
Contributor Author

@rpetrusha I stripped out the other <code> blocks in favor of markdown inline code with &#124; pipes. If that doesn't work on this pass, then I'll add back the <code> blocks to see if that will fix it.

@rpetrusha rpetrusha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guardrex, the formatting is still problematic. Also -- and I have no idea why -- the TOC is no longer building properly. In the last build, I distinctly remember seeing the selective unit tests topic; now it's gone, though the TOC file is still in your PR.


| Expression | Result |
| ---------- | ------ |
| `dotnet test --filter "FullyQualifiedName~TestClass1&#124;Category=Nightly"` | Runs tests which has `TestClass1` in `FullyQualifiedName` **or** `Category` is `Nightly`. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This renders as dotnet test --filter "FullyQualifiedName~UnitTestClass1&#124;TestCategory=CategoryA"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't been able to get back to this until just now: I was over doing a review for [@]mairaw. I'm free to look at this one again now.

It was a test on my last commit to see it NOT correctly convert that &#124; ...... so now, I'll go back over to <code> tags. I think that will lick it on the next pass if the 🙏 server gods 🙏 bless us! lol

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw - [@]mairaw is working on TOC issues right now. Not sure if it's affecting what ur seeing. #2079 (comment)

| ---------- | ------ |
| `dotnet test --filter "FullyQualifiedName~TestClass1&#124;Category=Nightly"` | Runs tests which has `TestClass1` in `FullyQualifiedName` **or** `Category` is `Nightly`. |
| `dotnet test --filter "FullyQualifiedName~TestClass1&Category=Nightly"` | Runs tests which has `TestClass1` in `FullyQualifiedName` **and** `Category` is `Nightly`. |
| `dotnet test --filter "(FullyQualifiedName~TestClass1&Category=Nightly)&#124;Priority=1"` | Runs tests which have either `FullyQualifiedName` containing `TestClass1` **and** `Category` is `CategoryA` **or** `Priority` is 1. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This renders as dotnet test --filter "(FullyQualifiedName~UnitTestClass1&TestCategory=CategoryA)&#124;Priority=1"

@guardrex

Copy link
Copy Markdown
Contributor Author

@rpetrusha

Copy link
Copy Markdown
Contributor

@guardrex, yes, this looks good. It's ready to merge.

@guardrex
guardrex merged commit ae036cf into dotnet:master May 18, 2017
@guardrex
guardrex deleted the guardrex/selective-unit-tests branch May 18, 2017 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants