[android] Connection issues in Sockets and Functional networking tests

[matouskozak]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=735679
Build error leg or test failing: System.Net.Sockets.Tests, System.Net.Ping.Functional.Tests, System.Net.NameResolution.Pal.Tests
Affected CI: android_[arm/arm64] Release AllSubset_Mono (runtime-extra-platforms)
Range of commits: 5795e8c...a2b7214

Stack trace example from System.Net.Sockets.Tests:

07-10 02:54:56.978 12860 18040 I DOTNET  : 1) 	[FAIL] System.Net.Sockets.Tests.SendTo_ArraySync.Datagram_UDP_ShouldImplicitlyBindLocalEndpoint   Test name: System.Net.Sockets.Tests.SendTo_ArraySync.Datagram_UDP_ShouldImplicitlyBindLocalEndpoint
07-10 02:54:56.978 12860 18040 I DOTNET  :    Assembly:  [System.Net.Sockets.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
07-10 02:54:56.978 12860 18040 I DOTNET  :    Exception messages: System.Net.Sockets.SocketException : Network is unreachable   Exception stack traces:    at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Net.Sockets.Socket.SendTo(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, EndPoint remoteEP)
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Net.Sockets.Tests.SocketHelperArraySync.<>c__DisplayClass11_0.<SendToAsync>b__0() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketTestHelper.cs:line 92
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Threading.Tasks.Task`1[[System.Int32, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].InnerInvoke()
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Threading.Tasks.Task.<>c.<.cctor>b__292_0(Object obj)
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
07-10 02:54:56.978 12860 18040 I DOTNET  : --- End of stack trace from previous location ---
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
07-10 02:54:56.978 12860 18040 I DOTNET  : --- End of stack trace from previous location ---
07-10 02:54:56.978 12860 18040 I DOTNET  :    at System.Net.Sockets.Tests.SendTo`1.<Datagram_UDP_ShouldImplicitlyBindLocalEndpoint>d__7[[System.Net.Sockets.Tests.SocketHelperArraySync, System.Net.Sockets.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]].MoveNext() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendTo.cs:line 90
07-10 02:54:56.978 12860 18040 I DOTNET  : --- End of stack trace from previous location ---
07-10 02:54:56.978 12860 18040 I DOTNET  :    Execution time: 0.031889
07-10 02:54:56.978 12860 18040 I DOTNET  : 2) 	[FAIL] System.Net.Sockets.Tests.SendTo_ArraySync.Datagram_UDP_AccessDenied_Throws_DoesNotBind   Test name: System.Net.Sockets.Tests.SendTo_ArraySync.Datagram_UDP_AccessDenied_Throws_DoesNotBind
07-10 02:54:56.978 12860 18040 I DOTNET  :    Assembly:  [System.Net.Sockets.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
07-10 02:54:56.978 12860 18040 I DOTNET  :    Exception messages: Assert.Equal() Failure: Values differ
07-10 02:54:56.978 12860 18040 I DOTNET  : Expected: AccessDenied
07-10 02:54:56.978 12860 18040 I DOTNET  : Actual:   NetworkUnreachable   Exception stack traces:    at System.Net.Sockets.Tests.SendTo`1.<Datagram_UDP_AccessDenied_Throws_DoesNotBind>d__8[[System.Net.Sockets.Tests.SocketHelperArraySync, System.Net.Sockets.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]].MoveNext() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendTo.cs:line 105
07-10 02:54:56.978 12860 18040 I DOTNET  : --- End of stack trace from previous location ---
07-10 02:54:56.978 12860 18040 I DOTNET  :    Execution time: 0.0532853

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "'System.Net.Sockets.Tests' END OF WORK ITEM LOG: Command exited with 1",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=735679
Error message validated: ['System.Net.Sockets.Tests' END OF WORK ITEM LOG: Command exited with 1]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/12/2024 8:22:53 AM UTC

Report

Build	Definition	Test	Pull Request
745393	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
743450	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
743391	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
743077	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
742744	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
742370	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
742110	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
741767	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
740838	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution	#104541
740555	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution	#104870
740287	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
739967	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
739577	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
739539	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
739937	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
738878	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution	#100356
737515	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution
735679	dotnet/runtime	System.Net.Sockets.Tests.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	3	18

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[matouskozak]

Could the failures be caused by #104251 @antonfirsov ? (cc: @simonrozsival)

[wfurt]

unlikely @matouskozak .

       [Fact]
        public async Task Datagram_UDP_ShouldImplicitlyBindLocalEndpoint()
        {
            using var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
            byte[] buffer = new byte[32];

            Task sendTask = SendToAsync(socket, new ArraySegment<byte>(buffer), GetGetDummyTestEndpoint());

That tests sends data to random site that should follow default route and succeed. So this looks more like environmental problem to me. (or test design) If this is persistent we should investigate of the test systems have full internet access.

[matouskozak]

unlikely @matouskozak .

       [Fact]
        public async Task Datagram_UDP_ShouldImplicitlyBindLocalEndpoint()
        {
            using var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
            byte[] buffer = new byte[32];

            Task sendTask = SendToAsync(socket, new ArraySegment<byte>(buffer), GetGetDummyTestEndpoint());

That tests sends data to random site that should follow default route and succeed. So this looks more like environmental problem to me. (or test design) If this is persistent we should investigate of the test systems have full internet access.

Thank you @wfurt for the investigation. I opened an infrastructure issue for it dotnet/dnceng#3405.

[matouskozak]

The internet connection to the devices was restored. Closing.