Skip to content

Poll: Should runnable samples include output as comments? #4571

Closed

Description

@BillWagner

this comment highlights a question we've been wrestling with: should runnable samples include the output as comments or not?

The advantage of including comments is that you can see what code should do without running it. The advantage of not including the comments is that the samples take less space, and after running them, you can see the output and the code without scrolling.

Anytime we've asked, the responses have been almost evenly split. Try these two pages and vote:

Example where comments are included Vote 👎

Example without comments for output Vote 👍

Activity

  1. bryanjhogan commented on Mar 7, 2018

    @bryanjhogan

    I like seeing the comments because it lets me quickly see the output of some method and there might be many examples on a page. Sometimes I just want to glance at the docs to help choose between a few alternatives, e.g. the date formatting.

    Suggestions - would it be possible to leave the comments in, but as soon as a user edits the code or runs the sample, then remove the block of comments from the sample?

  2. svick commented on Mar 7, 2018

    @svick
    Member

    I think being able to see the output without running the code makes sense when the specific string matters, like with Console.WriteLine(dat1.ToString(CultureInfo.InvariantCulture));. But in general, I think it's mostly unnecessary, like with Console.WriteLine(dat1.Equals(DateTime.MinValue)); or the foreach loop.

  3. ChrisMaddock commented on Mar 8, 2018

    @ChrisMaddock
    Contributor

    Option 3: it would be really cool if the page could load in this state in the first place! Presumably sample outputs could be cached?

    This option requires some development, of course. 😄

    image

  4. pkulikov commented on Mar 9, 2018

    @pkulikov
    Contributor

    @ChrisMaddock that would be perfect! Then one can edit code immediately, without the necessity to run it first as it is now.

  5. mairaw commented on Mar 9, 2018

    @mairaw
    Contributor

    Adding @LadyNaggaga and @yunyunyunyun here for some suggestions on the interactive sample experience.

  6. LadyNaggaga commented on Mar 9, 2018

    @LadyNaggaga
    Contributor

    @ChrisMaddock I like the idea of preloading the output. However I am worried that people might not run the code.
    @BillWagner I am okay with the comments in the output because you also let the user know they can run the code. Have user not responded well to it ?

  7. BillWagner commented on Mar 9, 2018

    @BillWagner
    MemberAuthor

    @LadyNaggaga The small sample in the votes above mirrors the responses I've gotten every time I show this in a conference: it's almost exactly split evenly for "I'd rather run the code" and "i'd like to know what it does when the page loads". No consensus is emerging. 😕

    I like @ChrisMaddock 's idea if we can implement it. It would give users both in a way.

    If that proves really difficult to implement, @svick 's guidance can help us do better in the content.

  8. sputier commented on Mar 9, 2018

    @sputier
    Contributor

    TL;DR; I think we should keep comments

    As a developer with a bit of experience, I think the comments in 101 samples are "distracting", and I appreciate them in the advanced code samples.

    ... But 101 samples are not designed for those who already know the language : beginners probably need comments in basic code samples as much as I appreciate comments in advanced samples.

  9. SpaceShot commented on Mar 9, 2018

    @SpaceShot

    At first I was going to come in here and say obviously Yes. Then I tried both interactive samples and I'm definitely more undecided. Removing the comments tidies the overall length of the document, and running the code is "cool". However, after that wears off, the seasoned dev might want to just see the output and get back to work. Sure it's just a few seconds but I see that angle too.

  10. jkaster commented on Mar 9, 2018

    @jkaster

    I'm a huge fan of documentation, code comments, and reference implementations for users. However, this site provides the Run button in situ, so the user can look at the sample, mentally challenge themselves to see if they understand it, then hit Run. Or, just hit Run to see the output anyway. So I think omitting the comments when the output can be shown directly below the sample is the way to go.

  11. scottaddie commented on Mar 9, 2018

    @scottaddie
    Member

    My thoughts mirror those of @sputier. ☝️ I do, however, think there's a point at which the comments become unnecessarily noisy. We should always try to limit the number of them.

  12. nemec commented on Mar 9, 2018

    @nemec

    I vote for, at the very least, caching the response 😐
    Service Unavailable message

    Until Blazor launches and we can run the samples client-side!

  13. rkeithhill commented on Mar 10, 2018

    @rkeithhill

    One thing I don't like about the commented output is that the copy button copies the code and the commented output. That is just more lines of code I have to delete.

    BTW why does the copy button go away after you run the code snippet? That's unfortunate.

  14. IEvangelist commented on Mar 10, 2018

    @IEvangelist
    Member

    I prefer it without the comments. They seem a bit redundant making them less useful. Also, if the user is to change the variables - are the comments with the expected output going to change too? I think it is clean and simple enough to have it where upon executing the Run the Output is displayed.

    Comments just muddy the waters for me, especially when they're simply stating the expectations. The beauty of this tool is that the user can interact with it, learn from it, edit and re-run and see it for themselves. I do want to take a moment to thank everyone on the team for this, it is utterly amazing and I love where you're going with it. Thank you!

  15. SeanKilleen commented on Mar 10, 2018

    @SeanKilleen
    Contributor

    It's a tough call for sure. My instinct is to recommend removing comments. I know that they can be helpful, especially for beginners, but I think losing them is the right way to go because:

    • The concept of clicking a green "run" button doesn't seem to be a far-removed concept that would confuse people
    • Those who are beginners may not be in such a hurry that they would need to see comments at a glance. Running doesn't seem to be a heavy lift from a time perspective.
    • By keeping comments in a code sample, especially one that can be edited where the comments would then be incorrect, we perpetuate the idea to beginners that trivial comments like that might be appropriate to document their production code, when oftentimes testing the result of code would be preferred.

    My last thought there has been gnawing on me so I'll just say: why not present the comments in the form of xUnit tests instead? That might be closer to the behavior that we want the community to model. At first I thought I'd be joking about this, but now that I've said it...

  16. 3 remaining items

  17. Felienne commented on Mar 10, 2018

    @Felienne

    Thanks @bryanjhogan for sharing our work!

    Please do include the example outputs. I understand it will make the code longer, but for novices this is of very high importance. One of the activities that makes learners better at programming is 'tracing' which basically entails executing the code in your brain and predicting the outcome. The example output helps with that a lot!

    And yes there is the "run" button, but that might not be all that easy to find for novices and also it takes quite some time to run.

    I could also see how you would make two versions.

  18. guntbert commented on Mar 10, 2018

    @guntbert

    Being able to check my results against the "correct" results is well worth some effort (IMO).
    Thats why I'd prefer comments even to @ChrisMaddock's excellent idea.
    One thing comes to my mind: make displaying the expected results quasi optional by

    • providing them as a "tooltip", shown on hover or
    • having a section of the page fold/unfold (UX hint: down arrow or plus sign)
  19. crowcoder commented on Mar 11, 2018

    @crowcoder
    Contributor

    Agree with @guntbert , I'd like to see how the output in a folding section looks.

  20. mika76 commented on Mar 11, 2018

    @mika76

    I vote keep the comments. Quicker to see and react. Lots of times I just need a refresher on exact output, and comments would be much quicker to see.

  21. rkeithhill commented on Mar 11, 2018

    @rkeithhill

    How about always showing the output pane but prepopulated with the expected output until the user presses the Run button? When the Run button is pressed the new output overwrites the comment output.

    image

  22. BillWagner commented on Mar 21, 2018

    @BillWagner
    MemberAuthor

    First of all, thank you to everyone that responded. The votes, ideas, and comments have given us a lot to think about.

    One clear message is that this question isn's so simple as always display comments, or never display comments.
    The right choice depends on several factors:

    1. The intended audience
    2. The goal of the article
    3. The nature of the output

    There are likely other factors we haven't discovered yet.

    Let's start by creating guidance for when to include comments, or output when using the current version of the REPL.

    When to display the expected output without running the sample

    • Articles intended for beginners should provide output so that readers can compare the output of their work with the expected answer.
    • Samples where the output is integral to the topic should display that output. For example, articles on text formatting should show the text format without running the sample.
    • When both the sample and the expected output is short, consider showing the output. It saves a bit of time.

    Notice that I didn't use the phrase 'comments' above. We haven't experimented with showing the expected output in the article text. That is worth testing.

    When to exclude expected output from the sample

    • Articles where the sample generates a larger output should not include that in comments. It obscures the code once the sample has been run.
    • Articles where the sample demonstrates a topic, but the output isn't integral to understanding. For example, code that runs a LINQ query to explain query syntax and then display every item in the output collection.

    Other notes from comments

    @SeanKilleen mentioned the concept of using xUnit tests:

    My last thought there has been gnawing on me so I'll just say: why not present the comments in the form of xUnit tests instead? That might be closer to the behavior that we want the community to model. At first I thought I'd be joking about this, but now that I've said it...

    We've thought about this. The concern for shorter topics and beginner topics is that it introduces more requirements to understand the sample. "I came here to learn how to format strings. Now, I have to learn xUnit first." For samples that represent full programs, we agree that explaining test strategies is an important part!

    New site ideas

    This has also generated great ideas for updates to the REPL. I've created new issues at https://github.com/dotnet/try for these features

    Thanks again. I'm creating a PR that updates our contributor guide with this information. We'll gather data and continue to improve.

  23. self-assigned this
    on Mar 21, 2018
  24. added and removed
    discussionIndicates issues that are being discussed
    on Mar 21, 2018
  25. added this to the milestone on Mar 21, 2018
  26. rpetrusha commented on Mar 21, 2018

    @rpetrusha
    Contributor

    There's also one other issue that's emerged recently as a result of your PR on string comparison, @BillWagner. For culture-sensitive comparison, .NET relies on information supplied by the operating system, which can be very different across particularly .NET Core versions. In that case, it would seem, output either shouldn't be included, or should be included only with the caveat that it reflects output for a specified platform using a specified culture.

  27. added a commit that references this issue on Mar 31, 2018
    202c7d2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions