Skip to content

Regressions in System.Tests.Perf_Array for ARM64 #68667

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare ae9f1ca5717045293f726877de18a8f6cf792c5b
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Reverse - Duration of single invocation 86.87 ns 141.59 ns 1.63 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.Reverse


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 141.5902091930573 > 91.26066230361728.
IsChangePoint: Marked as a change because one of 4/26/2022 6:18:08 AM, 4/27/2022 8:46:50 AM falls between 4/19/2022 6:32:34 PM and 4/27/2022 8:46:50 AM.
IsRegressionStdDev: Marked as regression because -131.54038213427378 (T) = (0 -142.15692031098698) / Math.Sqrt((0.22062057934126858 / (35)) + (0.5115350008666868 / (3))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (3) - 2, .025) and -0.6369040400935537 = (86.84499324887872 - 142.15692031098698) / 86.84499324887872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare ae9f1ca5717045293f726877de18a8f6cf792c5b
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Reverse - Duration of single invocation 171.70 ns 286.87 ns 1.67 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.Reverse(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 286.8743261808392 > 180.59975390646315.
IsChangePoint: Marked as a change because one of 4/26/2022 6:18:08 AM, 4/27/2022 8:46:50 AM falls between 4/19/2022 6:32:34 PM and 4/27/2022 8:46:50 AM.
IsRegressionStdDev: Marked as regression because -227.30060518890855 (T) = (0 -286.39017242473375) / Math.Sqrt((1.9498782004738482 / (35)) + (0.5830247902076874 / (3))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (3) - 2, .025) and -0.6580417757183983 = (172.72795934266873 - 286.39017242473375) / 172.72795934266873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline b42adaddff9047e32c9d7f4145746dbb579758f2
Compare ae9f1ca5717045293f726877de18a8f6cf792c5b
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Reverse - Duration of single invocation 173.49 ns 236.54 ns 1.36 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.Reverse(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.54396131211246 > 182.10393200384888.
IsChangePoint: Marked as a change because one of 4/26/2022 6:18:08 AM, 4/27/2022 8:46:50 AM falls between 4/19/2022 6:32:34 PM and 4/27/2022 8:46:50 AM.
IsRegressionStdDev: Marked as regression because -126.16357924033326 (T) = (0 -236.0946824240651) / Math.Sqrt((1.7492659048874621 / (35)) + (0.6067386409791822 / (3))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (3) - 2, .025) and -0.3668204454251107 = (172.73277058028975 - 236.0946824240651) / 172.73277058028975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions