StackOverflowException on web project code analysis

[PatrickHofman]

Version Used:
17.3.6
17.4.0 Preview 3

Steps to Reproduce:

1. Start Visual Studio
2. Open solution file containing a web project
3. Crash

Expected Behavior:
Visual Studio to open and fully load, not crash.

Actual Behavior:
Visual Studio crashes after the solution is opened with no error message shown.

When inspecting a Windows dump file created on crash, it shows this call stack ending up in a stack overflow (on both versions mentioned, the call stack is the same):

Unhandled exception at 0x00007FF83EB9805D (Microsoft.CodeAnalysis.CSharp.ni.dll) in devenv.exe.20804.dmp: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x000000967FA48FD8).

>	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Syntax.BinaryExpressionSyntax.GetNodeSlot(int index)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.ChildSyntaxList.ItemInternal(Microsoft.CodeAnalysis.SyntaxNode node, int index)	Unknown
 	[External Code]	
 	[The 2 frame(s) above this were repeated 2442 times]	
 	[External Code]	
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.UserFunctionExtensions.WrapUserFunction.AnonymousMethod__0((System.__Canon, System.__Canon) input, System.Threading.CancellationToken token)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.TransformNode<(Microsoft.CodeAnalysis.DotnetRuntime.Extensions.SyntaxValueProviderExtensions.SyntaxTreeInfo, Microsoft.CodeAnalysis.DotnetRuntime.Extensions.GlobalAliases), (Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)>..ctor.AnonymousMethod__0((Microsoft.CodeAnalysis.DotnetRuntime.Extensions.SyntaxValueProviderExtensions.SyntaxTreeInfo, Microsoft.CodeAnalysis.DotnetRuntime.Extensions.GlobalAliases) i, System.Threading.CancellationToken token)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.TransformNode<(Microsoft.CodeAnalysis.DotnetRuntime.Extensions.SyntaxValueProviderExtensions.SyntaxTreeInfo, Microsoft.CodeAnalysis.DotnetRuntime.Extensions.GlobalAliases), (Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)>.UpdateStateTable(Microsoft.CodeAnalysis.DriverStateTable.Builder builder, Microsoft.CodeAnalysis.NodeStateTable<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)> previousTable, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.DriverStateTable.Builder.GetLatestStateTableForNode<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)>(Microsoft.CodeAnalysis.IIncrementalGeneratorNode<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)> source)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.TransformNode<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>), (Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)>.UpdateStateTable(Microsoft.CodeAnalysis.DriverStateTable.Builder builder, Microsoft.CodeAnalysis.NodeStateTable<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)> previousTable, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.DriverStateTable.Builder.GetLatestStateTableForNode<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)>(Microsoft.CodeAnalysis.IIncrementalGeneratorNode<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>)> source)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.CombineNode<(Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>), Microsoft.CodeAnalysis.Compilation>.UpdateStateTable(Microsoft.CodeAnalysis.DriverStateTable.Builder graphState, Microsoft.CodeAnalysis.NodeStateTable<((Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>), Microsoft.CodeAnalysis.Compilation)> previousTable, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.DriverStateTable.Builder.GetLatestStateTableForNode<((Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>), Microsoft.CodeAnalysis.Compilation)>(Microsoft.CodeAnalysis.IIncrementalGeneratorNode<((Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>), Microsoft.CodeAnalysis.Compilation)> source)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.TransformNode<((Microsoft.CodeAnalysis.SyntaxTree, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.SyntaxNode>), Microsoft.CodeAnalysis.Compilation), Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>.UpdateStateTable(Microsoft.CodeAnalysis.DriverStateTable.Builder builder, Microsoft.CodeAnalysis.NodeStateTable<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax> previousTable, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.DriverStateTable.Builder.GetLatestStateTableForNode<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>(Microsoft.CodeAnalysis.IIncrementalGeneratorNode<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax> source)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.BatchNode<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>.UpdateStateTable(Microsoft.CodeAnalysis.DriverStateTable.Builder builder, Microsoft.CodeAnalysis.NodeStateTable<System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>> previousTable, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.DriverStateTable.Builder.GetLatestStateTableForNode<System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>>(Microsoft.CodeAnalysis.IIncrementalGeneratorNode<System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>> source)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.CombineNode<Microsoft.CodeAnalysis.Compilation, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>>.UpdateStateTable(Microsoft.CodeAnalysis.DriverStateTable.Builder graphState, Microsoft.CodeAnalysis.NodeStateTable<(Microsoft.CodeAnalysis.Compilation, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>)> previousTable, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.DriverStateTable.Builder.GetLatestStateTableForNode<(Microsoft.CodeAnalysis.Compilation, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>)>(Microsoft.CodeAnalysis.IIncrementalGeneratorNode<(Microsoft.CodeAnalysis.Compilation, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>)> source)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.SourceOutputNode<(Microsoft.CodeAnalysis.Compilation, System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.CSharp.Syntax.ClassDeclarationSyntax>)>.UpdateStateTable(Microsoft.CodeAnalysis.DriverStateTable.Builder graphState, Microsoft.CodeAnalysis.NodeStateTable<(System.Collections.Generic.IEnumerable<Microsoft.CodeAnalysis.GeneratedSourceText>, System.Collections.Generic.IEnumerable<Microsoft.CodeAnalysis.Diagnostic>)> previousTable, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.DriverStateTable.Builder.GetLatestStateTableForNode<(System.Collections.Generic.IEnumerable<Microsoft.CodeAnalysis.GeneratedSourceText>, System.Collections.Generic.IEnumerable<Microsoft.CodeAnalysis.Diagnostic>)>(Microsoft.CodeAnalysis.IIncrementalGeneratorNode<(System.Collections.Generic.IEnumerable<Microsoft.CodeAnalysis.GeneratedSourceText>, System.Collections.Generic.IEnumerable<Microsoft.CodeAnalysis.Diagnostic>)> source)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.SourceOutputNode<(System.__Canon, System.Collections.Immutable.ImmutableArray<System.__Canon>)>.AppendOutputs(Microsoft.CodeAnalysis.IncrementalExecutionContext context, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.GeneratorDriver.UpdateOutputs(System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.IIncrementalGeneratorOutputNode> outputNodes, Microsoft.CodeAnalysis.IncrementalGeneratorOutputKind outputKind, Microsoft.CodeAnalysis.GeneratorRunStateTable.Builder generatorRunStateBuilder, System.Threading.CancellationToken cancellationToken, Microsoft.CodeAnalysis.DriverStateTable.Builder driverStateBuilder)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.GeneratorDriver.RunGeneratorsCore(Microsoft.CodeAnalysis.Compilation compilation, Microsoft.CodeAnalysis.DiagnosticBag diagnosticsBag, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.dll!Microsoft.CodeAnalysis.GeneratorDriver.RunGenerators(Microsoft.CodeAnalysis.Compilation compilation, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.FinalizeCompilationAsync(Microsoft.CodeAnalysis.SolutionState solution, Microsoft.CodeAnalysis.Compilation compilationWithoutGenerators, Microsoft.CodeAnalysis.SolutionState.CompilationTracker.CompilationTrackerGeneratorInfo generatorInfo, Microsoft.CodeAnalysis.Compilation compilationWithStaleGeneratedTrees, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoFromScratchAsync(Microsoft.CodeAnalysis.SolutionState solution, Microsoft.CodeAnalysis.SolutionState.CompilationTracker.CompilationTrackerGeneratorInfo generatorInfo, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetOrBuildCompilationInfoAsync(Microsoft.CodeAnalysis.SolutionState solution, bool lockGate, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetCompilationSlowAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetCompilationAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.GetMetadataReferenceAsync(Microsoft.CodeAnalysis.SolutionState.ICompilationTracker tracker, Microsoft.CodeAnalysis.ProjectState fromProject, Microsoft.CodeAnalysis.ProjectReference projectReference, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.GetMetadataReferenceAsync(Microsoft.CodeAnalysis.ProjectReference projectReference, Microsoft.CodeAnalysis.ProjectState fromProject, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.FinalizeCompilationAsync(Microsoft.CodeAnalysis.SolutionState solution, Microsoft.CodeAnalysis.Compilation compilationWithoutGenerators, Microsoft.CodeAnalysis.SolutionState.CompilationTracker.CompilationTrackerGeneratorInfo generatorInfo, Microsoft.CodeAnalysis.Compilation compilationWithStaleGeneratedTrees, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoFromScratchAsync(Microsoft.CodeAnalysis.SolutionState solution, Microsoft.CodeAnalysis.SolutionState.CompilationTracker.CompilationTrackerGeneratorInfo generatorInfo, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetOrBuildCompilationInfoAsync(Microsoft.CodeAnalysis.SolutionState solution, bool lockGate, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetCompilationSlowAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetCompilationAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.GetMetadataReferenceAsync(Microsoft.CodeAnalysis.SolutionState.ICompilationTracker tracker, Microsoft.CodeAnalysis.ProjectState fromProject, Microsoft.CodeAnalysis.ProjectReference projectReference, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.GetMetadataReferenceAsync(Microsoft.CodeAnalysis.ProjectReference projectReference, Microsoft.CodeAnalysis.ProjectState fromProject, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.FinalizeCompilationAsync(Microsoft.CodeAnalysis.SolutionState solution, Microsoft.CodeAnalysis.Compilation compilationWithoutGenerators, Microsoft.CodeAnalysis.SolutionState.CompilationTracker.CompilationTrackerGeneratorInfo generatorInfo, Microsoft.CodeAnalysis.Compilation compilationWithStaleGeneratedTrees, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoFromScratchAsync(Microsoft.CodeAnalysis.SolutionState solution, Microsoft.CodeAnalysis.SolutionState.CompilationTracker.CompilationTrackerGeneratorInfo generatorInfo, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetOrBuildCompilationInfoAsync(Microsoft.CodeAnalysis.SolutionState solution, bool lockGate, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetCompilationSlowAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetCompilationAsync(Microsoft.CodeAnalysis.SolutionState solution, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.CodeAnalysis.Workspaces.dll!Microsoft.CodeAnalysis.SolutionState.GetCompilationAsync(Microsoft.CodeAnalysis.ProjectState project, System.Threading.CancellationToken cancellationToken)	Unknown
 	Microsoft.WebTools.ProjectSystem.dll!Microsoft.WebTools.ProjectSystem.Web.Telemetry.WebProjectDetailsTelemetryCollector.CollectDataAndPostTelemetryAsync()	Unknown
 	[External Code]	

[CyrusNajmabadi]

@joperezr we probably need to port dotnet/roslyn#64322 over to the runtime repo. I'll contact you today about the process around that.

[PatrickHofman]

I have checked another issue related to the dotnet/roslyn#64322 fix, and the repro project there also uses ANTLR, just as we do.

[CyrusNajmabadi]

@PatrickHofman can you please attach the project? That would be helpful with validating the fix. Or at least the large file antlr produces.

[PatrickHofman]

I can't put the original files here, since they are confidential. I have been digging a little deeper, and since I know this might be ANTLR related, I isolated the grammar G4 files. I narrowed it down to a grammar with a single rule, and the original list of tokens. I could delete a fair amount of tokens, until suddenly the solution did open and build. I then isolated the failing generated ANTLR code and put it in a separate project file. It seems the number of string concatenations done is the problem here.

I will attach a single file that once put in a default .NET 6 Console App will make VS crash.

ConsoleApp16.zip

[CyrusNajmabadi]

hey @PatrickHofman we're creating fixes for this right now. One thing that would help is if you can add a little more info on the impact to VS. When you say it 'crashes' can you be specific on exactly the experience you are getting? Do you mean that the process completely dies and VS disappears? Or is it more that you get a message saying there was an error?

Similarly, what happens if you try to compile from the command line? Thanks!

[CyrusNajmabadi]

@PatrickHofman Thanks so much for your example file. Would it be possible to have the original antlr file that generated that? i just want to get a sense for how big the grammar is that causes this. Thanks!

[CyrusNajmabadi]

@PatrickHofman @sharwell Does antlr have a mode where it doesn't emit a string as such an unbalanced binary-tree? e.g. if it can just emit it as one long string, instead of 2000 concats. (or if it can emit it with a lower number of concats). Just looking to see if there's a workaround to help you and others in the interim while this is worked on.

[joperezr]

@PatrickHofman as a workaround, you can disable the Json Source generator from being automatically added to your project if you are not using it by adding the following target into your .csproj:

  <Target Name="RemoveJsonSourceGenerator"
    BeforeTargets="CoreCompile">
    <ItemGroup>
      <AnalyzersByFileName Include="@(Analyzer -> '%(FileName)')">
        <OriginalIdentity>%(Identity)</OriginalIdentity>
      </AnalyzersByFileName>
      <AnalyzersToRemoveByFileName Include="System.Text.Json.SourceGeneration" />
    </ItemGroup>

    <ItemGroup>
      <AnalyzersToRemove Include="@(AnalyzersByFileName)" Condition="'@(AnalyzersToRemoveByFileName)' == '@(AnalyzersByFileName)' and '%(Identity)' != ''" />
      <Analyzer Remove="%(AnalyzersToRemove.OriginalIdentity)" />
    </ItemGroup>
  </Target>

This is of course not the right fix, as this would not work in case you actually do use the Json Source generator, but it is a temporary workaround that people can use while we prepare a servicing fix for 6.0

[CyrusNajmabadi]

I'm going to move this to the runtime repo as this is not an issue with roslyn (as evident with this in the callstack: Microsoft.CodeAnalysis.DotnetRuntime.Extensions).

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.