Sergey # Blog

Few thoughts about the current.

Welcome to Binary-in-ETW!

One of the most questionable features of ETW, an excellent tracing mechanism, is mixing of serialization with channel itself. This is why we built Binary-in-ETW library that has been released as part of Tx (LINQ to Logs and Traces). Now you can use JSON or Bond’s Compact Binary serializations methods or any another ones that could serialize to byte array like Avro, ProtoBuf, BJSON, etc.

Conceptually we are using ETW event just as an envelope.

Writing a mixed stream into Binary-in-ETW

This is how you can write your items of mixed types into the Binary-in-ETW:

1
2
3
4
5
6
7
8
9
10
var observer = new BinaryEtwObserver();

// Log a scalar instance
observer.OnNext("Hello");

// Log an instance of a complex .NET type
observer.OnNext(new MyClass { Value = "Some value " + DateTime.UtcNow });

// Log a bonded object
observer.OnNext(new ABondClass { StringValue = "Bond: Some another value " + DateTime.UtcNow });

Reading from Binary-in-ETW

This is how you can read from ETL files:

1
2
3
4
5
6
7
8
9
var playback = new Playback();

playback.AddSequentialBondEtlFiles(@"c:\Data\mixed_000001.etl");
  
playback.GetObservable<string>().Dump();
playback.GetObservable<MyClass>().Dump();
playback.GetObservable<ABondClass>().Dump();

playback.Run();

Advanced usage (custom serialization)

If you want to use your own serialization or you just want to store binary data as is, you need to write to BinaryEventSource directly, and write an ITypeMap implementation that could deserialize stored binary data.

1
2
3
4
5
6
7
BinaryEventSource.Log.Write(
  occurenceTime : DateTime.UtcNow, // Event creation time; value will be ignored
  receiveTime : DateTime.UtcNow, // Time when event was written  
    protocol : "My Serialization Protocol", // Identifier of the serialization protocol. Value is used when you want to deserialize back
    source : "My Application", // Identifier of your source; value will be ignored
    eventData : new byte[] { ... }, // Payload
    manifestId : "daf0be6e-da1e-5a6a-0d49-69782745c886"); // Identifier of the payload type

The following protocol identifiers are being used by the library: "JSON", "BOND_V1", "BOND_V2", "BOND" for JSON, BOND Compact Binary V1, BOND Compact Binary V2 serializations and are reserved. Do not use these names when giving a name for your custom protocol.

Next step is to create ITypeMap implementation. You can find several samples here.

And the usage is:

1
2
3
4
5
6
7
8
9
10
var playback = new Playback();

((IPlaybackConfiguration)playback).AddInput(
  () => BinaryEtwObservable.FromSequentialFiles(@"c:\Data\mixed_000001.etl"),
  typeof(MyCustomTypeMap),
  typeof(GeneralPartitionableTypeMap));
  
playback.GetObservable<MyCustomType>().Dump();

playback.Run();

Summary

If you already use ETW for tracing, consider this extension to log your custom objects, so you do not have to create multiple ETW events that ultimately serve the same purpose. Or if you would want to use this as a redundant output together with the primary ones (e.g. logging of raw web requests or post to an external services) this solution will be an excellent choice too.

Enjoy!

Comments