[Android] Deserialization issue when consuming ASMX Web Service

[steveisok]

Transferred customer issue from dotnet/android#7230

Description

I'm upgrading an existing Xamarin Traditional (Android) application into .NET 6 but I'm running into deserialization problems. Our Android app integrates with a .NET Framework 4.8 ASMX Web Service on the backend. This has been working fine for years but doesn't work after we move to .NET 6.

Reproduction Steps

I was able to create a repro project on my GitHub: https://github.com/adolfo1981/Net6AndroidWebServiceTest

My GitHub repro repo consists of the following projects:

• HelloAndroid (NET6 - .net6.0-android Target Framework Moniker)
• WebServiceServer (.NET Framework 4.8)
• WebServiceClient (.NET 6) --> referenced by HelloAndroid project to consume ASMX Web Services

1. Create a Xamarin Android (NET 6) project similar to @jamesmontemagno https://github.com/jamesmontemagno/net6-mobile-samples/tree/main/HelloAndroid
2. Create a .NET Framework 4.8 WebServiceServer application following instructions like this: https://www.c-sharpcorner.com/article/how-to-create-a-web-service-project-in-net-using-visual-studio/
3. Create a .NET 6 (Class Library) WebServiceClient and add a Connected Service -> Web Service pointing to a wsdl or deployed asmx file
4. From Main Activity create an instance of the generated soap client and call HelloWorldAsync method:

var soapClient = new WebServiceSoapClient(WebServiceSoapClient.EndpointConfiguration.WebServiceSoap);
//Async test
var helloResponse = await soapClient.HelloWorldAsync();

Expected behavior

Get string response from HelloWorldAsync() method

Actual behavior

Exception:
The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:HelloWorldResponse. The InnerException message was 'There was an error deserializing the object of type ServiceReference1.HelloWorldResponseBody. Unexpected end of file. Following elements are not closed: HelloWorldResult, HelloWorldResponse, Body, Envelope. Line 1, position 298.'. Please see InnerException for more details.

StackTrace:

at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName)
   at System.Runtime.Serialization.DataContractSerializer.ReadObject(XmlDictionaryReader reader, Boolean verifyObjectName)
   at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader, XmlObjectSerializer serializer) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 657
   at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 652
   at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.DeserializeParameterPart(XmlDictionaryReader reader, PartInfo part, Boolean isRequest) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 521

Regression?

Yes, this worked in legacy Xamarin Android and should work in the .NET versions.

Known Workarounds

None

Configuration

.NET 6, .NET 7
Android (any architecture)

Other information

IMPORTANT:

The repro project points to my local IP (Bottom of Reference.cs file) and needs to be changed to IP of the user running the code. I could not use localhost because I was getting connectivity issues and only worked when setting my local IP.

Things that I've tried:

• Generating the Web Service Client with 'Reuse referenced assemblies' enabled and disabled and that had no effect. Kept getting deserialization issue
• Tried changing the HelloAndroid.csproj TargetFramework from .net6,0-android to .net6.0-android31.0 and nothing changed even after deleting /bin and /obj folders in this project

Tagging subscribers to 'arch-android': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

I'm upgrading an existing Xamarin Traditional (Android) application into .NET 6 but I'm running into deserialization problems. Our Android app integrates with a .NET Framework 4.8 ASMX Web Service on the backend. This has been working fine for years but doesn't work after we move to .NET 6.

Reproduction Steps

I was able to create a repro project on my GitHub: https://github.com/adolfo1981/Net6AndroidWebServiceTest

My GitHub repro repo consists of the following projects:

• HelloAndroid (NET6 - .net6.0-android Target Framework Moniker)
• WebServiceServer (.NET Framework 4.8)
• WebServiceClient (.NET 6) --> referenced by HelloAndroid project to consume ASMX Web Services

1. Create a Xamarin Android (NET 6) project similar to @jamesmontemagno https://github.com/jamesmontemagno/net6-mobile-samples/tree/main/HelloAndroid
2. Create a .NET Framework 4.8 WebServiceServer application following instructions like this: https://www.c-sharpcorner.com/article/how-to-create-a-web-service-project-in-net-using-visual-studio/
3. Create a .NET 6 (Class Library) WebServiceClient and add a Connected Service -> Web Service pointing to a wsdl or deployed asmx file
4. From Main Activity create an instance of the generated soap client and call HelloWorldAsync method:

var soapClient = new WebServiceSoapClient(WebServiceSoapClient.EndpointConfiguration.WebServiceSoap);
//Async test
var helloResponse = await soapClient.HelloWorldAsync();

Expected behavior

Get string response from HelloWorldAsync() method

Actual behavior

Exception:
The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:HelloWorldResponse. The InnerException message was 'There was an error deserializing the object of type ServiceReference1.HelloWorldResponseBody. Unexpected end of file. Following elements are not closed: HelloWorldResult, HelloWorldResponse, Body, Envelope. Line 1, position 298.'. Please see InnerException for more details.

StackTrace:

at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName)
   at System.Runtime.Serialization.DataContractSerializer.ReadObject(XmlDictionaryReader reader, Boolean verifyObjectName)
   at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader, XmlObjectSerializer serializer) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 657
   at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 652
   at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.DeserializeParameterPart(XmlDictionaryReader reader, PartInfo part, Boolean isRequest) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 521

Regression?

Yes, this worked in legacy Xamarin Android and should work in the .NET versions.

Known Workarounds

None

Configuration

.NET 6, .NET 7
Android (any architecture)

Other information

IMPORTANT:

The repro project points to my local IP (Bottom of Reference.cs file) and needs to be changed to IP of the user running the code. I could not use localhost because I was getting connectivity issues and only worked when setting my local IP.

Things that I've tried:

• Generating the Web Service Client with 'Reuse referenced assemblies' enabled and disabled and that had no effect. Kept getting deserialization issue
• Tried changing the HelloAndroid.csproj TargetFramework from .net6,0-android to .net6.0-android31.0 and nothing changed even after deleting /bin and /obj folders in this project

Author:	steveisok
Assignees:	-
Labels:	os-android, area-System.ServiceModel
Milestone:	8.0.0

[steveisok]

@simonrozsival Please triage this when you have a moment.

[simonrozsival]

Triage: From the customer's report it's clear that there is an Android specific issue (the same code works in a .NET 6 console application) somewhere in the code that processes the response from the server. At a first glance this could be one of the following issues:

• the HTTP client doesn't return the full body of the response and this causes the XML reader to fail because the content stream it's processing is missing closing elements
• there is some problem with the encoding of the response string and the XML reader can't properly identify some closing tags
• there is some Android specific problem in the XML parser itself

We should investigate further to identify the cause of the problem and fix it to align the behavior of all supported platforms.

The repro project didn't work on my PC out-of-the-box because one of the server project failed to load. I'll give it another try later this week.

[adrianpbv]

Same problem here. I'm using a third party soap web service and when I try to make a request i get [DOTNET] Error in deserializing body of reply message for operation 'x'. I'm running the application on android. Also i notice that the tool dotnet-svcutil doesn't work properly using .net6-anyPlatform. So, when generating the file at the moment of adding the Web Service Reference maybe the System.Xml.Serialization package is not added to the project. The screenshot shows the Error List when I tried to add dotnet-svcutil to the project.

[quiksilva7]

Is there a solution to this yet? I am running into the exact same issue running .NET 7. Web Services run fine in a Windows environment, as soon as I try executing the same code in an Android environment I receive the error posted in this issue.

[simonrozsival]

I was able to reproduce the bug in both .NET 6 and .NET 7. It appears to be a problem with response content decompression in AndroidMessageHandler. For some reason the decompressed XML is incomplete /cc @grendello @jonathanpeppers

There are two workarounds that might apply in some scenarios:

• disable compression in the Andorid client
• add the following property to the .csproj file to switch from the AndroidMessageHandler to SocketsHttpHandler that decompresses the content correctly: <UseNativeHttpHandler>false</UseNativeHttpHandler>

[grendello]

@simonrozsival if the decompression finished then the payload is probably "valid" (in the sense that the compressed data contains all that was sent/received), but it of course depends on the compression algorithm used. If it's GZip (most likely?), then the decompression may return a short result if one block of data fails to decompress. All the preceding blocks will still be decompressed. That begs the question - is the compressed data length (as reported by the Content-Length header) equal to how much data was actually fetched? You could test it by disabling automatic decompression in the client and do the decompression step separately, after receiving the content.

If the length is different, then there might be a bug in that we somehow fail to read the last chunk of data from the service. If that's the case, then it might be a problem with the underlying Java client (a short read caused by something) or in our content stream handling. I find it hard to believe the latter is the case, since we've had this code for many years without any such issues, but nothing is impossible - perhaps some subtle semantics of stream reading changed between the mono/mono and dotnet BCLs?

Another test would be to see if changing the compression algorithm makes things work? The underlying Java client supports only GZip and Deflate IIRC, both should also be supported by the ASMX service server.

[akoeplinger]

@adrianpbv the second issue you mentioned about dotnet-svcutil is unrelated, looks like you added a PackageReference to it but that won't work since it is a tool that is supposed to be installed via dotnet tool install --global dotnet-svcutil and then used on the command line: https://learn.microsoft.com/en-us/dotnet/core/additional-tools/dotnet-svcutil-guide?tabs=dotnetsvcutil2x

[simonrozsival]

@grendello Yes, it's gzip. The Content-Length is 311 and from what I can tell, WCF trims the decoded message to this length:

[DOTNET] Content-Length: 311
[DOTNET] Decoded message has 311 bytes: '<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><HelloWorldResponse xmlns="http://tempuri.org/"><HelloWorldResult>Hello World</HelloWorldResu'

It reminds me of another issue with the iOS native handler: dotnet/wcf#4903. I think this is a bug in WCF which doesn't correctly work with native handlers 🤔

[grendello]

@simonrozsival hm, that sounds a bit like WCF takes the compressed data length and applies it to the decompressed content length?