Skip to content

System.Threading.Tasks.Dataflow tests timing out on coreclr Linux_musl arm #71475

Description

@radical

Frequency:

Day Run OS Notes
8/12-8/26 (logs available and verified)
8/5-8/11 PR #73472 6x Alpine.314.Arm64.Open + 1x Ubuntu.1804.Arm64.Open Specific to the PR
7/30 PR #49576 Alpine.314.Arm32.Open (logs not available for verfication)
7/1-7/22 (logs available and verified)
7/16 PR #72296 Debian.10.Arm32.Open System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestProducerConsumerAsyncEnumerable
7/13 PR #72108 Debian.10.Arm32.Open System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestProducerConsumerAsyncEnumerable
7/12 PR #71175 Debian.10.Arm32.Open System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestProducerConsumerAsyncEnumerable
7/7 Official run Work item killed without any tests reported as hanged - log
7/6 Official run Debian.11.Arm32.Open System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestProducerConsumerAsyncEnumerable
4/1-6/30 (logs not available for verification)
6/30 PR #71474 Alpine.314.Arm32.Open
6/30 PR #71164 Alpine.314.Arm32.Open
6/30 PR #71460 Alpine.314.Arm32.Open
6/29 PR #71440 Debian.10.Arm32.Open
6/15 PR #70756 2 failures - perhaps different root cause or caused by the PR?
6/4-6/6 PR #70240 13 failures - likely caused by the PR itself
5/13 PR #69291 2 failures - perhaps different root cause or caused by the PR?
5/6 PR #67894 Alpine.314.Arm64.Open
4/11 PR #67666 27 failures -- most likely caused by the PR itself
4/7 PR #67694 8 failures -- likely caused by the PR itself
let timeouts = (friendlyNamePrefix : string, includePR : bool) {
cluster('engsrvprod.kusto.windows.net').database('engineeringdata').WorkItems
| where Status == "Timeout"
| where FriendlyName startswith friendlyNamePrefix
//| where Finished > datetime('2021-09-07')
| distinct JobId, WorkItemId, Name, FriendlyName, ConsoleUri
| join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
    | where ((Branch == 'refs/heads/main') or (Branch == 'refs/heads/master') or (includePR and (Source startswith "pr/")) or (Branch startswith 'refs/heads/release/6.0'))
    | where Type startswith "test/functional/cli/"
        and not(Properties contains "runtime-staging")
    | summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName) by JobId
| project-rename JobType = Type) on JobId
| order by Finished desc
};
timeouts('System.Threading.Tasks.Dataflow', true);

Build, and Log:

  Starting:    System.Threading.Tasks.Dataflow.Tests (parallel test collections = on, max threads = 4)
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:02:11
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:04:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:06:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:08:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:10:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:12:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:14:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:16:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:18:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:20:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:22:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:24:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:26:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:28:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:30:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:32:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:34:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:36:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:38:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:40:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:42:12
   System.Threading.Tasks.Dataflow.Tests: [Long Running Test] 'System.Threading.Tasks.Dataflow.Tests.TransformManyBlockTests.TestOrdering_Sync_OrderedEnabledAsyncEnumerable', Elapsed: 00:44:12

...
[EXECUTION TIMED OUT]
Exit Code:-3Executor timed out after 2700 seconds and was killed
['System.Threading.Tasks.Dataflow.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions