Monday, April 18, 2016

Yet another short async/await example

The async/await pattern is here for some time and this is going to be yet another short example that could possibly make it easier for someone to grasp the idea.

First, since we talk about possible async operations, let’s start with a Windows.Forms/WPF application where there is at least a chance to see that an operation is async – we will start a long running operation and while it lasts, we the GUI will remain responsive. On the other hand, trying anything async from a console app doesn’t make much sense.

So let’s have a Windows.Forms app with two buttons. Assign something instant to the second button, something like MessageBox.Show so that you can press it and make sure the app remains responsive. We will then make the first button run a long operation.

Let’s start with this long running operation:

private int SlowOperation()
{    
    Thread.Sleep( 5000 );    
    return 17;
}

It is obvious that running this from a click handler would freeze the application for 5 seconds.

Without async/await, you still can run this asynchronously. There are numerous ways to do it, a possibly most basic one is to create a delegate and run it asynchronously, this is something delegates support since .NET has been released:

Func<int> f = () => SlowOperation();
f.BeginInvoke(     
    ar =>    
    {        
        Func<int> s = (Func<int>)ar.AsyncState;        
        var result = s.EndInvoke( ar );        
        MessageBox.Show( result.ToString() );    
    },     
    f );

This looks ugly. Not only I need a pair of BeginInvoke/EndInvoke methods but also I need to feed both with valid arguments. In particular, if the delegate returns a value, a common practice is to pass the delegate as the “async state” so that it can be later picked up and used to end the async call. This basic pattern of Begin…/End… is called the Asynchronous Programming Model (APM).

However, this works. You can run the app and make sure it stays responsive and after 5 seconds it just shows a message box with the result of the long call.

As .NET evolved, this basic pattern has been evolving too. We had background workers, we had so called Event-Based Asynchronous Pattern (EAP) which is obsolete nowadays. A newest addition to the toolbox is the Task-based Asynchronous Pattern (TAP) which is based on the Task class that represents both initiation and completion of an async method.

And this is what this tutorial aims to – to show how this ugly code from above can be rewritten to a more modern style. We will refactor the code in three steps.

Step one. This is where we introduce the Task class. It also evolves but at the time I write this, you can pretty much easily convert anything to a task with Task.Run facade. It uses the thread pool to queue the async operation and immediately starts it.

The problem with bare Task.Run is that it returns a task and I still to supply a delegate to provide a code that runs when the task completes:

Task<int> task = Task.Run( () => SlowOperation() );
task.ContinueWith( t => MessageBox.Show( t.Result.ToString() ) );

This still doesn’t look good but definitely better than the APM version.

Btw. Stephen Cleary has a nice explanation on how and why the Task.Run should be used only to invoke async methods rather than inside their implementations.

Step two. In this step I rewrite my original SlowOperation to already support the async model. This way I won’t need the Task.Run anymore:

private Task<int> SlowOperation2()
{    
    return Task.Delay( 5000 )        
               .ContinueWith( t => 17 );
}

Note, that I also had to change the blocking Thread.Sleep to a non blocking Task.Delay.

Now I go back to the client code and I rewrite it to

SlowOperation2()
    .ContinueWith( t => MessageBox.Show( t.Result.ToString() ) );

Step three. This is where the async/await is finally introduced – it is merely a syntactic sugar over .ContinueWith.

First, I rewrite the SlowOperation2. Instead of the lambda that shows the continuation, the code is now clean.

private async Task<int> SlowOperation2()
{    
    await Task.Delay( 5000 );    
    return 17;
}

Then I rewrite the client code to also replace the ContinueWith with async/await. This time it is not that easy because to be able to use await, my method has to be marked with async.

private async void button1_Click( object sender, EventArgs e )
{    
    var result = await SlowOperation2();    
    MessageBox.Show( result.ToString() );
}

Happy asyncing/awaiting your code.

Friday, January 15, 2016

DI, Factories and the Composition Root Revisited

Some time ago I published an entry on Local Factories and what role they play in a clean architecture stack where the Composition Root configures dependencies between abstractions and implementations. The article can be found here.

There are dozens of questions about this article from people and I decided to refresh the concept by simplyfing the code a little bit.

The general idea of the Local Factory pattern is to hide the implementation details of a factory but in the same time make the factory the only legitimate source of instances. The goal the pattern tries to achieve is to create a stable API for creating instances, allow possible DI usage but not to be dependant on any specific implementation (including the DI).

Let’s start with the service:

// service contract
// no ioc here
public interface IService
{
    void Foo();
}

Then, the factory:

// the Local Factory
// still no ioc here
public class ServiceFactory
{
    private static Func<IService> _provider;
   
    // the factory is the only legal provider of service instances
    // but in fact it delegaes the creation yet elsewhere
    public IService CreateService()
    {
        if ( _provider != null )
            return _provider();
        else
            throw new ArgumentException();
    }
   
    public static void SetProvider( Func<IService> provider )
    {
        _provider = provider;
    }
}

Note how smart the factory is.

Until an actual provider is configured (and it will be configured in the Composition Root), the factory doesn’t even know how to create instances. This approach is a simplification compared to the one I presented last time – my approach that time was to have a factory provider to create factory to create instances. A redundancy that can be easily eliminated by moving the volatile part of the implementation to the factory and eliminating the factory provider from the big picture.

The client code relies on the factory:

// the client uses the factory
// no ioc here in the client
public class ServiceClient
{
    public void ServiceUsageExample()
    {
        var sf = new ServiceFactory();
        var service = sf.CreateService();
       
        service.Foo();
    }
}

Note that up to this point there is no DI, just a simple dependency from the client to the factory that returns instances. In a real world application you would probably have multiple local factories, each factory belongs to a specific subsystem in a specific layer and doesn’t interfere with other layers and other subsystems.

This is where such local factory differs from the Service Locator, a God-like factory that creates instances of multiple classes from multiple layers but in the same time it introduces an unwanted association to itself. A local factory on the other hand is a part of its local ecosystem: the factory together with the abstraction (interface) consitute the API for their future clients.

Now it is the time however to configure the factory from the Composition Root. The CR should:

  • know what actual implementation of the interface should be used
  • configure the factory provider somehow, for example to return an instance of a fixed, known type or maybe to use a DI container to return one

First, a concrete implementation of the service. Note that the factory is unaware of what actual type will be used which of course means that the type can be defined anywhere in the solution stack, for example in a different assembly from the one the interface/factory are defined.

public class ServiceImpl : IService
{
    public void Foo()
    {
        Console.WriteLine( "serviceimpl:foo" );
    }
}

And last but not least, the Composition Root:

public class Program
{
    public static void Main()
    {
        CompositionRoot();
       
        var client = new ServiceClient();
        client.ServiceUsageExample();       
    }
   
    public static void CompositionRoot()
    {
        // this is the only place in the code that is aware of the ioc       
        // but could be as well configured to not to use ioc at all
       
        // no-ioc:
        ServiceFactory.SetProvider( () => new ServiceImpl() );
       
        // ioc:
        /*
        var container = new UnityContainer();
        container.Register<IService, ServiceImpl>();
        ServiceFactory.SetProvider( () => container.Resolve<IService>() );
        */
    }                                                           
}

Note that the Composition Root is the only spot in the code that is aware of the possible DI container. We could say that the CR delegates the implementation down the application stack to the Local Factory which otherwise would not know how to create instances. And whether or not a DI container is used – this fact is known only to the CR.

Note also that the possibility to have a simple provider that returns an instance of a known type is suitable for unit testing.

Hope the idea is clearer this time.

Specified value has invalid Control characters

One of sites started to report this exception from a DotnetOpenAuth powered OAuth2 token endpoint:

Parameter name: value
   at System.Net.WebHeaderCollection.CheckBadChars(String name, Boolean isHeaderValue)
   at System.Net.WebHeaderCollection.Set(String name, String value)
   at DotNetOpenAuth.Messaging.Channel.ReadFromRequest(HttpRequestBase httpRequest)
   at DotNetOpenAuth.Messaging.Channel.TryReadFromRequest[TRequest](HttpRequestBase httpRequest, TRequest& request)
   at DotNetOpenAuth.OAuth2.AuthorizationServer.HandleTokenRequest(HttpRequestBase request)

Upon further investigation it turned out that following snippet from the ReadFromRequest method fails:

foreach (string name in httpRequest.Headers)
{
     httpDirectRequest.Headers[name] = httpRequest.Headers[name];
}

This seems to use the [string, string] indexer on the Headers collection which in turn tries to validate both header name and value in CheckBadChars.

The ARR turned out to be the culprit, when it acts as a reverse proxy, it adds a couple of headers, including the X-ARR-SSL header that contains the information on the actual SSL cert used by ARR. And one of our development sites apparently used a certificate generated by hand from an internal cert authority with an invalid character in the cert’s Subject name.

Lesson learned, the cert should never contain any non-ASCII chars in the subject name.

Monday, November 2, 2015

String trimming WCF inspector for JSON binding

In my last blog entry I presented a string trimming WCF inspector for SOAP binding. Unfortunately, the approach presented there doesn’t work for Web bound WCF services that use JSON as the communication language.

The difficulty here is that the documentation of possible differences in message inspectors for XML vs JSON is very sparse. Fortunately, the previously mentioned blog entry by Carlos Figueira brings some invaluable details. In particular, Carlos shows that internally a JSON WCF represents the incoming data using XML, unfortunately however, there approach from the SOAP inspector doesn’t work this time. This is because requests are never SOAP envelopes.

There are two possible approaches, either stick with the XML that represents incoming JSON data or work at the JSON level. I took this as a challenge to get the JSON from the XML, modify it and give it back to the message builder. To trim strings I need a completely different approach then, I no longer walk over XML nodes, rather I have a JSON I need to rewrite to another JSON but the actual structure of the data is not known (the inspector works for different WCF methods of different signatures not known to the inspector).

My proposal is as follows, it uses JSON.NET to rewrite JSON with a help of an auxiliary json converter that does actual trimming.

/// <summary>
 /// WCF inspektor trimujÄ…cy stringi dla webhttpbinding
 /// </summary>
 /// <remarks>
 /// https://code.msdn.microsoft.com/WCF-REST-Message-Inspector-c4b6790b
 /// </remarks>
 public class StringTrimmingWebMessageInspector : IDispatchMessageInspector
 {
     #region IDispatchMessageInspector Members
 
     public object AfterReceiveRequest(
         ref System.ServiceModel.Channels.Message request,
         IClientChannel channel, InstanceContext instanceContext )
     {
         if ( !request.IsEmpty )
         {
             WebContentFormat messageFormat = this.GetMessageContentFormat( request );
             if ( messageFormat == WebContentFormat.Json )
             {
                 MemoryStream ms            = new MemoryStream();
                 XmlDictionaryWriter writer = JsonReaderWriterFactory.CreateJsonWriter( ms );
                
                 request.WriteMessage( writer );
                 writer.Flush();
 
                 ms.Close();
 
                 string messageBody = Encoding.UTF8.GetString(ms.ToArray());
 
                 string convertedBody = JObject.Parse( messageBody )
                    .ToString( Newtonsoft.Json.Formatting.None, new TrimmingConverter() );
 
                 ms = new MemoryStream( Encoding.UTF8.GetBytes( convertedBody ) );
                 XmlDictionaryReader reader = 
                    JsonReaderWriterFactory.CreateJsonReader( ms, XmlDictionaryReaderQuotas.Max ); 
 
                 Message newMessage = Message.CreateMessage( reader, int.MaxValue, request.Version );
                 newMessage.Properties.CopyProperties( request.Properties );
                 request = newMessage; 
             }               
         }
 
         return null;
     }
 
     private WebContentFormat GetMessageContentFormat( Message message )
     {
         WebContentFormat format = WebContentFormat.Default;
         if ( message.Properties.ContainsKey( WebBodyFormatMessageProperty.Name ) )
         {
             WebBodyFormatMessageProperty bodyFormat;
             bodyFormat = (WebBodyFormatMessageProperty)message.Properties[WebBodyFormatMessageProperty.Name];
             format = bodyFormat.Format;
         }
 
         return format;
     }
 
     public void BeforeSendReply( ref System.ServiceModel.Channels.Message reply, object correlationState )
     {
     }
 
     #endregion
 }
 
 public class TrimmingConverter : JsonConverter
 {
     public override bool CanConvert( Type objectType )
     {
         return objectType == typeof( string );
     }
     public override bool CanRead { get { return true; } }
     public override object ReadJson( 
        JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer )
     {
         if ( reader.Value != null && reader.Value is string )
             return ( (string)reader.Value ).Trim();
         else
             return reader.Value;
     }
     public override bool CanWrite { get { return true; } }
     public override void WriteJson( 
        JsonWriter writer, object value, JsonSerializer serializer )
     {
         if ( value is string )
             serializer.Serialize( writer, ( (string)value ).Trim() );
         else
             serializer.Serialize( writer, value );
     }
 }

String trimming WCF message inspector for XML binding

How often your users copy-paste data from somewhere to your web app and then post something that is incorrect because some extra characters are copied from the source text? Is user name really “ John ” “ Smith ”?

Sooner or later at the server side, the input strings are validated but often what you do first is a simple trimming so that “ John” becomes “John”. The question is then not “to trim or not to trim” but rather “when the trimming should occur to be as ellegant as possible”.

String trimming on HTTP input is a convenient solution. The data that is processed by an MVC controller, a WebApi controller or a WCF service code is already trimmed so that you don’t have to put any extra effort on trimming at the business level of your code. This approach is common and solutions for MVC or WebAPI can easily be found.

My specific problem was that I needed the same for WCF and unfortunately, I couldn’t find anything ready to use. My previous experience with WCF message inspectors suggested that a smart inspector is a way to go. A series of blog entries by Carlos Figueria is a valuable source of information on that topic, however this specific requirement of a trimming inspector has no previous solutions (or I couldn’t find any).

Without any further ado, let’s see how such message inspector is implemented:

/// <summary>
/// WCF inspektor trimujÄ…cy stringi
/// </summary>
public class StringTrimmingMessageInspector : IDispatchMessageInspector
{
    #region IDispatchMessageInspector Members
 
    public object AfterReceiveRequest(
        ref System.ServiceModel.Channels.Message request,
        IClientChannel channel, InstanceContext instanceContext )
    {
        if ( !request.IsEmpty )
        {
            MessageBuffer buffer = request.CreateBufferedCopy( Int32.MaxValue );
            request = buffer.CreateMessage();
 
            XmlDocument xmlDoc = new XmlDocument();
 
            using ( MemoryStream ms = new MemoryStream() )
            {
                XmlWriter writer = XmlWriter.Create( ms );
                request.WriteMessage( writer ); // the message was consumed here
                writer.Flush();
                ms.Position = 0;
                xmlDoc.Load( ms );
            }
 
            XmlNamespaceManager manager = new XmlNamespaceManager( xmlDoc.NameTable );
            manager.AddNamespace( "s", "http://schemas.xmlsoap.org/soap/envelope/" );
 
            XmlNode bodyNode = xmlDoc.SelectSingleNode( "//s:Envelope/s:Body", manager );
            this.ChangeNode( bodyNode );
 
            Message newMessage = Message.CreateMessage( request.Version, request.Headers.Action, bodyNode.FirstChild );
            newMessage.Properties.CopyProperties( request.Properties );
 
            request = newMessage;
        }
 
        return null;
    }
 
    private void ChangeMessage( XmlDocument doc )
    {
        if ( doc == null ) return;
 
        ChangeNode( doc.DocumentElement );
    }
 
    private void ChangeNode( XmlNode node )
    {
        if ( node == null ) return;
        if ( node.NodeType == XmlNodeType.Text ) node.InnerText = node.InnerText.Trim();
 
        foreach ( XmlNode childNode in node.ChildNodes )
            ChangeNode( childNode );
    }
 
    public void BeforeSendReply( ref System.ServiceModel.Channels.Message reply, object correlationState )
    {
    }
 
    #endregion
}

The inspector assumes the message is always an XML document and this is true for XML based bindings. A request is always a SOAP envelope that has headers and a body. Inside the inspector I retrieve the body node of an incoming message and I run a simple recursive function on the node that walks over the tree of XML nodes and whenever it find a node of text type, it trims its contents. I then recreate the message using traversed node as the body.

To apply the inspector, I also need a simple endpoint behavior

/// <summary>
/// Podstawowa infrastruktura rozszerzania
/// </summary>
public class StringTrimmingBehavior : IEndpointBehavior
{
    #region IEndpointBehavior Members
 
    public void AddBindingParameters( ServiceEndpoint endpoint, 
       System.ServiceModel.Channels.BindingParameterCollection bindingParameters )
    {
    }
 
    public void ApplyClientBehavior( ServiceEndpoint endpoint, 
       System.ServiceModel.Dispatcher.ClientRuntime clientRuntime )
    {
    }
 
    public void ApplyDispatchBehavior( ServiceEndpoint endpoint, 
       System.ServiceModel.Dispatcher.EndpointDispatcher endpointDispatcher )
    {
        endpointDispatcher.DispatchRuntime.MessageInspectors
           .Add( new StringTrimmingMessageInspector() );
    }
 
    public void Validate( ServiceEndpoint endpoint )
    {
    }
 
    #endregion
}
And this is it, please feel free to use/modify this for your own purposes.