Skip to content

Regressions in System.Text.Tests.Perf_Encoding and Perf_Encoders #70501

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare 06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 265.26 ns 297.67 ns 1.12 0.69 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 297.6709341236771 > 276.9819006858682.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsRegressionStdDev: Marked as regression because -5.7326841542386395 (T) = (0 -311.8049826288513) / Math.Sqrt((280.8213364247804 / (11)) + (719.7321643610898 / (34))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (34) - 2, .025) and -0.1436918839458107 = (272.63023110132076 - 311.8049826288513) / 272.63023110132076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.25094
Baseline c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare 06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 96.30 ns 110.85 ns 1.15 0.66 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.8524935605853 > 98.68546502434354.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsRegressionStdDev: Marked as regression because -7.850515874716991 (T) = (0 -116.21095716185361) / Math.Sqrt((61.42400947097666 / (11)) + (84.2728866224184 / (34))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (34) - 2, .025) and -0.23734486762653884 = (93.9196178869422 - 116.21095716185361) / 93.9196178869422 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions