HttpClient automatically adds Request-Id HTTP header

[shravan2x]

I have an application that sends HTTP requests via HttpClient from an Asp.NET Core project. For unknown reasons, all sent requests seem to have a Request-Id HTTP header. I'm not able to repro this in a regular console application. The docs don't have any information on disabling it.

They seem to be added by the framework https://github.com/dotnet/runtime/search?q=RequestIdHeaderName&unscoped_q=RequestIdHeaderName .

• Is there a temporary fix for this issue?
• What is the right way to fix this issue in the long-term?
• Can we treat changes to the default HTTP headers sent as breaking changes? Issues like HttpRequestMessage does not respect TryAddWithoutValidation #35335 and this silently cause hard-to-debug issues.

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

[davidsh]

For unknown reasons, all sent requests seem to have a Request-Id HTTP header.

HttpClient API does not add "Request-Id" HTTP request header.

How did you confirm that HttpClient was adding this request header? Was it by inspecting request headers received by the server? It's quite possible that a front-end proxy etc. is adding these headers on your behalf.

[stephentoub]

HttpClient API does not add "Request-Id" HTTP request header.

Actually, DiagnosticHandler does:

runtime/src/libraries/System.Net.Http/src/System/Net/Http/DiagnosticsHandler.cs

Line 271 in 4168844

request.Headers.TryAddWithoutValidation(DiagnosticsHandlerLoggingStrings.RequestIdHeaderName, currentActivity.Id);

but only if the diagnostics are enabled:

runtime/src/libraries/System.Net.Http/src/System/Net/Http/HttpClientHandler.cs

Line 220 in 4b03513

return DiagnosticsHandler.IsEnabled() ?

[davidfowl]

See this thread for clarity dotnet/aspnetcore#21012 (comment)

[davidfowl]

The docs are lacking in this area cc @noahfalk @shirhatti

[shravan2x]

@davidsh You can see how surprised I was when I realized the framework was doing this, and yet again just now when I heard that this is intentional behavior, and that there are no plans to fix it.

@davidfowl

That's fair, but we went with an opt-out model so that the "it just works when I call my other ASP.NET Core app or backend that supports the new W3C trace context" is on by default.

I strongly disagree with this. Not only is it sending behavior modifying headers without explicitly opting-in, it leaks private information about my HTTP client.

In what universe is it OK to post data that I didn't explicitly ask for to a remote location????

As much I hate to say it, I agree with @karlra's strongly worded views on this.

Even if it had been well documented, surely we don't expect every user of .NET to use a proxy, identify the weird header, google it, and then add code to opt-out? Why can't we ask users to add a line of code in their Startup.cs if they actually want such behavior?

I doubt the vast majority of Asp.NET Core users know or care about features like distributed tracing. Such a feature should have been designed to be opt-in. I'm confident you'll hear the same opinion if a public poll was conducted for this.

[davidsh]

@karelz @stephentoub

[davidfowl]

At this point if we decide to revert the behavior the question will be:

• What is the gesture to turn this on for an outbound calls?
• What about scenarios where you don't control how the HttpClient is constructed?

cc @SergeyKanzhelev @lmolkova

[shravan2x]

What is the gesture to turn this on for an outbound calls?

Following the usual pattern, something along the lines of services.AddDiagnostics(options => options.IsActivityPropagationEnabled = true); seems expected.

What about scenarios where you don't control how the HttpClient is constructed?

I'm unsure what such a scenario might be. However, I think having it be controlled by a flag like the example above is reasonable.

[davidfowl]

Following the usual pattern, something along the lines of services.AddDiagnostics(options => options.IsActivityPropagationEnabled = true); seems expected.

This would be HttpClientFactory specific? What would it be applied to? Which HttpClients?

I'm unsure what such a scenario might be. However, I think having it be controlled by a flag like the example above is reasonable.

A global static? Where do I set it, which HttpClients does it affect? Today the HttpClients that are affected are ones that run within the current activity. That's why it can be disabled on a granular basis.

[karlra]

I strongly disagree with this. Not only is it sending behavior modifying headers without explicitly opting-in, it leaks private information about my HTTP client.

I obviously also disagree strongly with this. Imagine the number of issues that will be raised on here about it.....The concatenation of the header that takes place if you try to set this header yourself is even worse.

In what universe is it OK to post data that I didn't explicitly ask for to a remote location????

As much I hate to say it, I agree with @karlra's strongly worded views on this.

Why, you hold a particular dislike for me? :)

[davidfowl]

The concatenation of the header that takes place if you try to set this header yourself is even worse.

I forgot about that. @karlr can you file another issue for that?