Repository navigation
Description
edit: known issue when running test compiled in Debug mode code.
work around:
build.cmd -test -c Release
I'm getting this when running ./build -tests
V Reverse slicing should not work with at symbol in 1st slice index [2ms]
V Reverse slicing should not work with at symbol in 2nd slice index [3ms]
V Reverse slicing should not work with at symbol in both slice index [2ms]
V Fixed index 4d slicing should not be available in 47 [5s 249ms]
V Else branch is a function that returns int while if branch is string [5s 297ms]
V Reverse slicing should work with overloaded infix hat [5s 528ms]
The active test run was aborted. Reason: Test host process crashed : Process is terminated due to StackOverflowException.
Test run in progress.Results File: C:\dev\src\github.com\Microsoft\masterfsharp\artifacts\TestResults\Debug\FSharpSuite.Tests_net472.xml
Test Run Aborted.
Total tests: Unknown
Passed: 97
Total time: 1.3206 Minutes
Deferred Messages
Detailed Build Summary
======================
============================== Build Hierarchy (IDs represent configurations) =====================================================
Id : Exclusive Time Total Time Path (Targets)
-----------------------------------------------------------------------------------------------------------------------------------
0 : 80.547s 80.547s C:\dev\src\github.com\Microsoft\masterfsharp\tests\fsharp\FSharpSuite.Tests.fsproj (VSTest)
============================== Node Utilization (IDs represent configurations) ====================================================
Timestamp: 1 Duration Cumulative
-----------------------------------------------------------------------------------------------------------------------------------
637146961850490571: 0 80.549s 80.549s # (scale 1:1000)
-----------------------------------------------------------------------------------------------------------------------------------
Utilization: 100.0 Average Utilization: 100.0
Build FAILED.
0 Warning(s)
0 Error(s)
Time Elapsed 00:01:20.58
Command failed to execute with exit code 1: C:\Program Files\dotnet\dotnet.exe test C:\dev\src\github.com\Microsoft\masterfsharp\tests\fsharp\FSharpSuite.Tests.fsproj -c Debug -f net472 -v n --test-adapter-path . --logger "nunit;LogFilePath=C:\dev\src\github.com\Microsoft\masterfsharp\artifacts\TestResults\Debug\FSharpSuite.Tests_net472.xml" /bl:C:\dev\src\github.com\Microsoft\masterfsharp\artifacts\log\Debug\FSharpSuite.Tests_net472.binlog --no-restore --no-build
System.Management.Automation.RuntimeException: Command failed to execute with exit code 1: C:\Program Files\dotnet\dotnet.exe test C:\dev\src\github.com\Microsoft\masterfsharp\tests\fsharp\FSharpSuite.Tests.fsproj -c Debug -f net472 -v n --test-adapter-path . --logger "nunit;LogFilePath=C:\dev\src\github.com\Microsoft\masterfsharp\artifacts\TestResults\Debug\FSharpSuite.Tests_net472.xml" /bl:C:\dev\src\github.com\Microsoft\masterfsharp\artifacts\log\Debug\FSharpSuite.Tests_net472.binlog --no-restore --no-build
at Exec-CommandCore, C:\dev\src\github.com\Microsoft\masterfsharp\eng\build-utils.ps1: line 50
at Exec-Console, C:\dev\src\github.com\Microsoft\masterfsharp\eng\build-utils.ps1: line 119
at TestUsingNUnit, C:\dev\src\github.com\Microsoft\masterfsharp\eng\build.ps1: line 260
at <ScriptBlock>, C:\dev\src\github.com\Microsoft\masterfsharp\eng\build.ps1: line 345
at <ScriptBlock>, <No file>: line 1
I've seen similar error running tests in VS and Rider.
Also, I'm not able to use nunit GUI test runner which gives:
FSharpSuite.Tests.dll (TestFixtureSetUp):
Could not load file or assembly 'FSharpSuite.Tests, Version=10.7.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. Strong name signature could not be verified. The assembly may have been tampered with, or it was delay signed but not fully signed with the correct private key. (Exception from HRESULT: 0x80131045)
I've tried loading artifacts\bin\FSharpSuite.Tests\Debug\net472\FSharpSuite.Tests.dll in it.
Sorry if this is known issue, looking for a work around.
Activity
cartermp commentedon Jan 15, 2020
The NUnit GUI runner issue sounds like a typical .NET Framework binding redirects problem. I'd be shocked if that issue is in this repo since it looks like a very very old program that hasn't been updated for NUnit 3.
As for the test itself, I'm not sure what the issue is here but 5 seconds for running some of these tests is way too long, which would indicate something is going on. @TIHan touched the compiler assertion framework last, but none of those changes looked like they would hit this. We don't see this in CI either (and CI machines are generally not that great). Does this reproduce for you?
smoothdeveloper commentedon Jan 15, 2020
Thanks for the feedback
The program is up to date and for nunit 3, it is called testcentric-gui (on nuget), and it generally works way faster than any of the in IDE runners around.
It is a 2017 laptop with dual core i7 with 16gb of RAM, and I have two fsharp.sln open, one in vs2019 and the other in Rider.
The tests running a little slow is ok, I'll give it another shot to see if this happens in another place in the test list, and giving my laptop more breathing room while doing so, I'll report back soon.
smoothdeveloper commentedon Jan 15, 2020
Here is an updated run, I've switched to "high performance" and closed all the IDEs
after running the tests

it took shorter for this particular test, but overall the figures look a bit volatile to draw a conclusion.
As it failed a bit later in the test suite, this indicates it is not determistic about a given test.
I'll try to work around by running tests in IDE in smaller bunch and append any info that may be relevant to the issue.
If there is anything I can do to help troubleshooting, or if anyone running the tests hit this, please let me know.
abelbraaksma commentedon Jan 15, 2020
I think a recent change in testcentric was that now by default they're running parallel, could you check if parallel vs sequential makes a difference?
smoothdeveloper commentedon Jan 15, 2020
@abelbraaksma thanks.
I'm not able to even run any of them in testcentric-gui, it seems that it has issue with delay signed assemblies as the message indicates, despite as @cartermp was saying, the beginning of the error message looks like a binding redirect.
If there is a way to disable signing on the unit test assemblies locally in the build, I could confirm if this is the issue with the gui runner (which can be useful but is not mandatory for contribution to this repository).
I'm able to run small batches of tests I need in the IDE, and I suspect the issue may be environment performance related: the issue is not blocking me.
It would be good if someone else using the repository can confirm the issue is showing up on equivalent hardware though.
abelbraaksma commentedon Jan 15, 2020
I'll try to have a go at it, haven't run the tests of F# in a while, but I do remember reporting binding issues before, maybe I see something familiar and it rings a bell (my setup is similar to yours, except for the cpu, which in my case is dual cpu and 12 core each, and not a laptop, so I hope it goes a bit faster ;).
Though I have to say, given your configuration, the average runtime of each test seems rather high
KevinRansom commentedon Jan 15, 2020
Yes, stack-overflow can happen running the tests with debug. Many work, but some don't. By default the debug build disables use of the .tail command. I suppose this was originally done to improve the debugger experience, but it does mean there is a chance that some looping code will blow the stack.
Kevin
smoothdeveloper commentedon Jan 16, 2020
@KevinRansom what's the way to run the tests in release build? this should be mentionned in https://github.com/dotnet/fsharp/blob/master/TESTGUIDE.md (seems the instructions are outdated there) or https://github.com/dotnet/fsharp/blob/master/DEVGUIDE.md
yatli commentedon Apr 19, 2020
@smoothdeveloper hello fellow Elektronaut!
-c Releasewould do.smoothdeveloper commentedon Apr 19, 2020
Thanks @yatli!
I actually updated the dev/test guides some time back, and now just updated the comment at the top of the issue with the workaround in case someone else hits the same issue.
cartermp commentedon Sep 2, 2020
Will close this out as by design for now, it kind of sucks, but at least the proper steps are documented.
abelbraaksma commentedon Sep 2, 2020
FYI, the test guides have very recently been revised, and this particular issue, and a few others, are addressed, plus more details on using release vs debug builds.