Using DataContractMessageSerializer with SSB

Apr 14, 2009 at 3:03 PM
Edited Apr 14, 2009 at 3:07 PM

I've just switched to using the DataContractSerializer with my project as I would like to use generics in my message objects.

Anyway I found that this worked fine with messages below a certain size, 960bytes when serialised.  After that the MemoryStream that is used to serialize the message objects increases it's buffer size from the size of the contained data to double it.  This means that when you do something like

messageEnvelope.Payload = payload.GetBuffer() ;

You end up copying the entire allocated buffer, including all the trailing nulls, into the message rather than just the bit your interested in.  In my case a 990 byte message copies 1996 bytes onto the wire.

While this isn't a problem with the XmlSerializer, that seems quite content to ignoring the nulls on the end of the stream, the DataContractSerializer has a fit and throws the Exception:

"There was an error deserializing the object of type System.Object. The data at the root level is invalid. Line 1, position 1."

Anyway it's easy to fix, if you change the chunk of code in SimpleServiceBus.Bus.Pipeline.PipelineComponents.MessageBodySerializationComponent down around line 36 to something along the lines of

 var payload = new MemoryStream();
svc.Serialize(messageEnvelope.Body, payload);
var data = new byte[payload.Length];
Array.Copy(payload.GetBuffer(), data, data.Length);
messageEnvelope.Payload = data;

Everything works fine.

BTW in the same file there are two Exceptions being thrown relating to serialization, and the terms serializing and deserializing are in the wrong exceptions.


Apr 14, 2009 at 3:24 PM

Just noticed that there is the same issue in SimpleServiceBus.Endpoints.Msmq.Transport.MsmqTransport::Send (~line 177), but that's just wasted bytes, the BinaryFormatter does not have the same issue as the DataContractSerializer.

There is a final use of GetBuffer in SimpleServiceBus.Endpoints.Msmq.Transport::CreateTransportMessageFromMsmqMessage but that always seems to have a capacity that matches it's length, so is not a problem at the minute.

Apr 17, 2009 at 11:33 PM
Thanks for the fix, it has been applied to the trunk.
Jul 17, 2009 at 12:16 AM

We noticed a similar issue when the message starts to be larger (about 70 kB). The issue happened in MsmqTransport.cs, line 220:

var payload = new MemoryStream();
StreamHelper.CopyStream(msmqMessage.BodyStream, payload);
result.Payload = payload.GetBuffer();

The issue is that the MemoryStream allocates (in our case) 128 kB and then when you call GetBuffer() you get the full 128 kB, even if the message itself is only 70 kB. We only notices this on DataContractSerializer as well. Our quick fix is to update the first line above with this:

var payload = new MemoryStream((int)msmqMessage.BodyStream.Length);

We're not sure if there are any issues of asking the BodyStream for its length before you read it, but it works fine so we assume the fix is correct. Obviously we want to minimize the number of in-memory buffers that get allocated and copied.

/Hakan and Konstantin