Monday, April 24, 2017

Using ngrok to expose any local server to the Internet

Ngrok is here for some time and it's a great tool that makes it possible to expose a website from local machine to the Internet by creating a tunnel from ngrok's infrastructure to your machine. This post adds 2 cents to this story. The story begins when you have a web app but not necessarily on your local machine but somewhere nearby at your local network. Sure, you can log into the local server and run ngrok there. But another approach is to have a HTTP proxy, from your machine to the local server. One of simplest and cleanest proxies that do the job would be this tiny node.js snippet
var httpProxy = require('http-proxy');

httpProxy.createProxyServer({
    target       : 'http://a.server.in.your.local.network:80', 
    changeOrigin : true, 
    autoRewrite  : true 
})
.listen(3000);
The two options make sure that both host and location headers are correctly rewritten. Please consult the list of other available options of the http-proxy module if you need more specific behavior. Then just invoke the ngrok as usual
ngrok http 3000
navigate to the address returned by ngrok and you will get the response from the server behind the proxy.

Friday, March 10, 2017

Visual Studio 2017 doesn't seem to support Silverlight but Silverlight itself is supported till 2021

This has been a great week as VS2017 has finally got its RTM version. Unfortunately, for some developers news aren't that great.

It seems the new VS drops the support for the Silverlight:

Silverlight projects are not supported in this version of Visual Studio. To maintain Silverlight applications, continue to use Visual Studio 2015.

Great. Note, though, that Silverlight 5 support lasts till 2021.

This effectively means that starting from now, till 2021, people will have to stick with VS2015 for some projects, despite VS2017, VS2019, VS2021 or any other newer version is released.

Nice one, Microsoft!

Friday, February 3, 2017

Calling promises in a sequence

One of the top questions about promises I was asked lately is a question on how to call promises in a sequence so that the next starts only after the previous one completes.

The question is asked mostly by people with C# background where a Task has to be started in an explicit way. In contrast, in Javascript, as soon as you have the promise object, it the execution has already started. Thus, following naive approach doesn’t work:

function getPromise(n) {
    return new Promise( (res, rej) => {
        setTimeout( () => {
            console.log(n);
            res(n);
        }, 1000 );
    });
}

console.log('started');

// in parallel
Promise.all( [1,2,3,4,5].map( n => getPromise(n)) )
    .then( result => {
        console.log('finished with', result);
    });

The above code yields 1,2,3,4,5 as soon as 1 second passes and there are no further delays, although people expect them here. The problem with this approach is that as soon as getPromise is called, the promise code starts. There are then 5 timers in total, started in almost the very same time, the time it takes the map function to loop over the source array.

In order to call promises in a sequence we have to completely rethink the approach. Each next promise should be invoked from within the then handler of the previous promise as then is called only where a promise is fulfilled. A simple structural trick is to use map.reduce in order to loop over the array so that each time we have the access to the previous “value” (the previous promise) and the current value (current number/whatever from the array). The map.reduce needs an “initial value” and it should be a promise so we just feed it with a dummy, resolved promise.

function getPromise(n) {
    return new Promise( (res, rej) => {
        setTimeout( () => {
            console.log(n);
            res(n);
        }, 1000 );
    });
}

console.log('started');

// in a sequence
[1,2,3,4,5].reduce( (prev, cur) => {
    return prev.then( result => {
        return getPromise(cur);
    });
}, Promise.resolve() )
    .then( result => {
        console.log('finished with', result);
    });

A neat trick worth of remembering.

Thursday, January 12, 2017

Node.js 7.x async/await example with express and mssql

Async/await has finally found its home in node 7.x and it’s just great, we’ve been waiting for ages. Not only it works great, with Visual Studio Code you can write and debug async/await like any other code. Just make sure you have runtimeArgs in your launch.json as a temporary fix until it’s not required somewhere in future:

{
    // Use IntelliSense to learn about possible Node.js debug attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "node",
            "request": "launch",
            "name": "Launch Program",
            "program": "${workspaceRoot}/app.js",
            "cwd": "${workspaceRoot}",
            "runtimeArgs": [
                "--harmony"
            ]
        },
        {
            "type": "node",
            "request": "attach",
            "name": "Attach to Process",
            "port": 5858
        }
    ]
}

Async/await could possibly sound like a ephemeral alien until you realize that a lot of good code has already been written with Promises async/await is a sugar over. In an example below I read the data from a simple Sql Server table using node’s mssql module:

CREATE TABLE [dbo].[Parent](
 [ID] [int] IDENTITY(1,1) NOT NULL,
 [ParentName] [nvarchar](150) NOT NULL,
 CONSTRAINT [PK_Parent] PRIMARY KEY CLUSTERED
(
 [ID] ASC
)
) ON [PRIMARY]

I also need a connection string somewhere

// settings.js
module.exports.connectionString = 'server=.\\sql2012;database=databasename;user id=user;password=password';

Note that since tedious doesn't support integrated auth at the moment, you are stuck with sql's username/pwd authentication. The mssql on the other hand uses tedious internally.

Here goes an example of the "old-style" code, written with promises  
   var conn = new sql.Connection(settings.connectionString);
   conn.connect()
     .then( () => {

         var request = new sql.Request(conn);

         request.query('select * from Parent')
              .then( recordset => {

                  recordset.forEach( r => {
     // do something with single record
                  });

    conn.close();
              })
              .catch( err => {
                  console.log( err );
              });
     })
     .catch( err => {
         console.log(err);
     });  
(note that you could possibly avoid nesting thens by just returing the request.query promise so that the next, chained then would refer to it.) However, with async/await the same becomes
try {
 var conn = new sql.Connection(settings.connectionString);
 await conn.connect();
 
 var request = new sql.Request(conn);
 var recordset = await request.query('select * from Parent') 
 
 recordset.forEach( r => {
  // do something with single record
 });
 
 conn.close();
}
catch ( err ) {
 console.log( err );
}
Pretty impressive, if you ask me. If you don't mind occasional awaits, the code is clean, no .thens, no .catches. Remember that express middlewares can be async too
var app = express();

app.get('/', async (req, res) => {

      // code that awaits

});
All this means the callback hell is hopefully gone forever.

Friday, December 16, 2016

Combining sync/async WCF client/server that share an interface-based contract

Having a shared interface between WCF client and server is handy, often you just put such contract in a shared assembly and you just build your service out of the contract and then use the ChannelFactory to automatically create clients.

// shared contract
[ServiceContract]
public interface IService1
{
  [OperationContract]
  string DoWork( string s );
}
// server
public class Service1 : IService1
{
  public string DoWork( string s )
  {
     return s + " from wcf";
  }
}
// client
var binding  = new BasicHttpBinding();
var endpoint = new EndpointAddress( "http://localhost:55748/Service1.svc" );
 
var factory = new ChannelFactory<IService1>( binding, endpoint );
var channel = factory.CreateChannel();
 
var result  = channel.DoWork("foo");

The problem starts when you decide to go async at the server or the client but in the same time keep the other sync – in MVC or in Windows.Forms you just change the method from Foo Method() to async Task<Foo> Method(), here in WCF you just can’t change the signature of the interface method as it affects both the client and the server.

For example, I you to be async at the server side so that you switch to the async interface of your data provider, and you change the interface to

[ServiceContract]
public interface IService1
{
  [OperationContract]
  Task<string> DoWork( string s );
}

// oops, the client doesn't compile anymore as the supposed contract has just changed
// the client is not "smart enough" to notice that in fact this is still the same contract 
// at the conceptual level

Fortunately, a simple solution exists, mentioned here. The solution is to have two interfaces, a sync one and an async one. Both interfaces have to share the name, however they can exist in the very same assembly, in adjacent namespaces. It turns out that both the client and the server could implement any of the two interfaces and wiring still succeeds.

namespace Sync
{
    // shared contract
    [ServiceContract]
    public interface IService1
    {
        [OperationContract]
        string DoWork( string s );
    }
}

namespace Async
{
    [ServiceContract]
    public interface IService1
    {
        [OperationContract]
        Task<string> DoWork( string s );
    }
}

An example async client, using the async interface, would be

  
private async void button1_Click( object sender, EventArgs e )
{
    var binding  = new BasicHttpBinding();
    var endpoint = new EndpointAddress( "http://localhost:55748/Service1.svc" );

    var factory = new ChannelFactory<Async.IService1>( binding, endpoint );
    var channel = factory.CreateChannel();

    var result = await channel.DoWork( "foo" );

    MessageBox.Show( result );
}

An example async server would be

public class Service1 : IService1
{
    public async Task<string> DoWork( string s )
    {
        await Task.Delay( 5000 );
        return s + " from wcf";
    }
}

The conclusion is: the shared assembly should just provide both interfaces, the sync and the async one, which lets both the client and the server freely decide which interface they implement. In a sense, the client and the server still share the same interface although technically these are two different interfaces.

Thursday, August 25, 2016

SignedXml::CheckSignature and dreadful “The profile for the user is a temporary profile” on .NET 4.6.2

Sometimes, out of nowhere, your well-tested and working-for-years code just stops working just because something changes somewhere you can really have control over.

This post documents one of such cases. The case is about checking digital signatures of signed XML documets. For years we’ve been creating and validating documents with self-contained signatures. I’ve blogged on that in a three part tutorial. In our scenario, IIS hosted ASP.NET applications were receiving signed XML documents and processing them.

This worked as a charm until .NET 4.6.2. In .NET 4.6.2 the SignedXml::CheckSignature( X509Certificate2 ) method we’ve been using for years doesn’t work anymore in our scenario, it throws “The profie for the user is a temporary profile” exception.

What is a temporary profile? You see, when an app pool is configued in IIS to use a custom account (a domain account in our case) and the Load User Profile is turned on, the ASP.NET worker process is supposed to load the profile of the app pool account. The existence of the profile changes some details on how temporary paths are resolved and how keys are imported to cert stores. IIS has an unfortunate way of handling this requirement – instead of creating a regular profile, it creates a temporary profile. People often ask what these profiles are and how to turn them off.

Unfortunately, there is no easy way to change this behavior of creating temporary profiles rather than regular profiles (or we don’t know how to do it). A workaround exists – you can just log into the server as the app pool account with an interactive session (a direct login or a remote desktop session). A local profile is then created at the server and further loading the profile by ASP.NET use the newly created, valid profile rather than a temporary one (e.g. the workaround is mentioned here at Paul Stovell blog). But suppose you have a farm where servers are cloned on demand (we use the Web Farm Framework). Suppose also you have multiple apps. We have like ~150 servers and ~50 apps on each, this means logging 150*50 times to different servers only to have profiles created correctly.

Unfortunately also, to be able to use cryptographic API, you often just have to turn on loading user profiles, otherwise the crypto system just doesn’t work sometimes (private keys cannot be accessed for certs that are loaded from local files). In our case, turning off the profile raises an instant exception “Object is in invalid state”.

What all it means is that .NET 4.6.2 changes the way SignedXml::CheckSignature( X509Certificate2 ) works and changes it in a fundamental way. Before 4.6.2 the method works always, regardless of whether the profile is a temporary profile or not, as long as the profile is loaded (Load User Profile = true). In .NET 4.6.2 the method doesn’t work if the profile is a temporary profile.

One of our first attempts was to follow the previously mentioned workaround and somehow automate the profile creation so that profiles are there when IIS requests them. This however doesn’t solve the issue on already existing servers.

But then another approach was tested. Because we check signatures on self-contained signed documents, X509Certificate2 instances we pass to the method are retrieved from the SignedXml:

// Create a new SignedXml object and pass it  // the XML document class.
SignedXml signedXml = new SignedXml( xd );
 
// Load the first <signature> node.   signedXml.LoadXml( (XmlElement)messageSignatureNodeList[0] );

// load certificate 
foreach ( KeyInfoClause clause in signedXml.KeyInfo ) 

    if ( clause is KeyInfoX509Data ) 
    { 
        if ( ( (KeyInfoX509Data)clause ).Certificates.Count > 0 ) 
        { 
            certificate = (X509Certificate2)( (KeyInfoX509Data)clause ).Certificates[0]; 
        } 
    } 
}

if ( certificate != null ) 

    // Check the signature and return the result. Throws the “user profile is a temporary profile” 
    return signedXml.CheckSignature( certificate, true ); 

else 
    return false;

The exception possibly is related to the possible leakage of private keys when temporary profiles are involved. Maybe one of .NET BCL developers was just oversensitive here, implementing a bunch of new features.

There are no private keys, however, in our certificates! Since certificates are here to verify signatures, private keys are not included, we only have public keys. How about using the overload of the CheckSignature that only expects an AsymmetricAlgorithm?

// Create a new SignedXml object and pass it 
// the XML document class.
SignedXml signedXml = new SignedXml( xd );

// Load the first  node.  
signedXml.LoadXml( (XmlElement)messageSignatureNodeList[0] );

AsymmetricAlgorithm rsa = null;

// load certificate
foreach ( KeyInfoClause clause in signedXml.KeyInfo )
{
	if ( clause is KeyInfoX509Data )
	{
        	if ( ( (KeyInfoX509Data)clause ).Certificates.Count > 0 )
	        {
        	      certificate = ((KeyInfoX509Data)clause).Certificates[0] as X509Certificate2;
	        }
        }
}

if ( certificate == null )
{
	Message = "No KeyInfoX509Data clause in the signature";
        return false;
}

if ( certificate.PublicKey == null || certificate.PublicKey.Key == null )
{
	Message = "The KeyInfoX509 clause doesn't contain any valid public key";
        return false;
}

rsa = certificate.PublicKey.Key;

// Check the signature and return the result. 
return signedXml.CheckSignature( rsa );

Yes, as you can expect, this works correctly, again regardless on whether the profile is a regular profile or a temporary one.

Monday, April 18, 2016

Yet another short async/await example

The async/await pattern is here for some time and this is going to be yet another short example that could possibly make it easier for someone to grasp the idea.

First, since we talk about possible async operations, let’s start with a Windows.Forms/WPF application where there is at least a chance to see that an operation is async – we will start a long running operation and while it lasts, we the GUI will remain responsive. On the other hand, trying anything async from a console app doesn’t make much sense.

So let’s have a Windows.Forms app with two buttons. Assign something instant to the second button, something like MessageBox.Show so that you can press it and make sure the app remains responsive. We will then make the first button run a long operation.

Let’s start with this long running operation:

private int SlowOperation()
{    
    Thread.Sleep( 5000 );    
    return 17;
}

It is obvious that running this from a click handler would freeze the application for 5 seconds.

Without async/await, you still can run this asynchronously. There are numerous ways to do it, a possibly most basic one is to create a delegate and run it asynchronously, this is something delegates support since .NET has been released:

Func<int> f = () => SlowOperation();
f.BeginInvoke(     
    ar =>    
    {        
        Func<int> s = (Func<int>)ar.AsyncState;        
        var result = s.EndInvoke( ar );        
        MessageBox.Show( result.ToString() );    
    },     
    f );

This looks ugly. Not only I need a pair of BeginInvoke/EndInvoke methods but also I need to feed both with valid arguments. In particular, if the delegate returns a value, a common practice is to pass the delegate as the “async state” so that it can be later picked up and used to end the async call. This basic pattern of Begin…/End… is called the Asynchronous Programming Model (APM).

However, this works. You can run the app and make sure it stays responsive and after 5 seconds it just shows a message box with the result of the long call.

As .NET evolved, this basic pattern has been evolving too. We had background workers, we had so called Event-Based Asynchronous Pattern (EAP) which is obsolete nowadays. A newest addition to the toolbox is the Task-based Asynchronous Pattern (TAP) which is based on the Task class that represents both initiation and completion of an async method.

And this is what this tutorial aims to – to show how this ugly code from above can be rewritten to a more modern style. We will refactor the code in three steps.

Step one. This is where we introduce the Task class. It also evolves but at the time I write this, you can pretty much easily convert anything to a task with Task.Run facade. It uses the thread pool to queue the async operation and immediately starts it.

The problem with bare Task.Run is that it returns a task and I still to supply a delegate to provide a code that runs when the task completes:

Task<int> task = Task.Run( () => SlowOperation() );
task.ContinueWith( t => MessageBox.Show( t.Result.ToString() ) );

This still doesn’t look good but definitely better than the APM version.

Btw. Stephen Cleary has a nice explanation on how and why the Task.Run should be used only to invoke async methods rather than inside their implementations.

Step two. In this step I rewrite my original SlowOperation to already support the async model. This way I won’t need the Task.Run anymore:

private Task<int> SlowOperation2()
{    
    return Task.Delay( 5000 )        
               .ContinueWith( t => 17 );
}

Note, that I also had to change the blocking Thread.Sleep to a non blocking Task.Delay.

Now I go back to the client code and I rewrite it to

SlowOperation2()
    .ContinueWith( t => MessageBox.Show( t.Result.ToString() ) );

Step three. This is where the async/await is finally introduced – it is merely a syntactic sugar over .ContinueWith.

First, I rewrite the SlowOperation2. Instead of the lambda that shows the continuation, the code is now clean.

private async Task<int> SlowOperation2()
{    
    await Task.Delay( 5000 );    
    return 17;
}

Then I rewrite the client code to also replace the ContinueWith with async/await. This time it is not that easy because to be able to use await, my method has to be marked with async.

private async void button1_Click( object sender, EventArgs e )
{    
    var result = await SlowOperation2();    
    MessageBox.Show( result.ToString() );
}

Happy asyncing/awaiting your code.