C# Puzzle No.13 (advanced)

Using LinqToSQL is fun and speeds up a lot of things. However, there are issues with LinqToSQL which confuse people. Ultimately, some of these people tend to think that LinqToSQL is useless because things that should be obvious are not such obvious. Over a year ago I wrote a short blog entry on one of such basic issues.

Another such "not-so-obvious" thing is the issue of "ordering". One of the common requirements is to be able to order by the name of the property and not by the property itself. For example, if you use ObjectDataSource, then you know that the sorting parameter is passed to the object responsible for data retrieving by name of the parameter.

Let's take a look at two LinqToSQL ordering attempts:

   1: ConcreteDataContext ctx = new ConcreteDataContext();

   2:  

   3: /* this is easy */

   4: var list = from elem in ctx.TheTable

   5:            orderby elem.Property

   6:            select elem;

   7:  

   8: /* this will compile

   9:    however it does not work

  10: */

  11: string PropertyName = "Property";

  12: var list = from elem in ctx.TheTable

  13:            orderby PropertyName

  14:            select elem;

In the example code above, the first linq clause is obvious - the ordering uses the property "Property" in a direct way. However, in in the second clause we use the name of the property.

You are to answer two following questions:

1. Why the second clause does not produce correct results, although it compiles correctly?

2. How is it possible then to build generic linq expressions which sort objects by names of their properties.

By generic I mean that following solution is not acceptable:

   1: switch ( PropertyName )

   2: {

   3:    case "Property":

   4:  

   5:       return from elem in ctx.TheTable

   6:              orderby elem.Property

   7:              select elem;

   8:  

   9:    case "AnotherProperty":

  10:  

  11:       return from elem in ctx.TheTable

  12:              orderby elem.AnotherProperty

  13:              select elem;

  14:  

  15:    ...

  16:  

  17: }

Beware when you upgrade ...

It's a real pity when you realize that you are truly disappointed by a new version of a software you use.

When Windows Media Player 11 came out, I've upgraded from WMP 10 immediately. To me, a new version always means new functions, enhanced compatibility, stability and security.

I got no problems with WMP 11 until I've bought two great soundtracks by Jeremy Soule from DirectSong. These soundtracks are protected by the DRM - you are required to obtain a valid license from the soundtrack provider to be able to play it with the Windows Media Player.

To be able to backup your downloads, DirectSong says that:

Our current licensing is set up to allow unlimited Soundtrack CDs to be burned per machine download.

Great, this does not work in WMP 11 - you learn from the WMP that the license does not allow you to burn, synchronize or even play the music (which does not stop the music from actually playing!).

After few nervous hours spent trying to find out what's the problem, I've learned from someone's blog that you have to downgrade to WMP 10 to get this to work.

I don't remeber beeing so disappointed by a new version of the Microsoft's software for a long time. A new version which breaks the backward compatibility is not something which I could accept.

Inject application-specific code into ASP.NET module

We often write ASP.NET modules to customize ASP.NET processing pipeline - extend security, etc.

   1: public class ExampleModule : IHttpModule

   2:  {

   3:      #region IHttpModule Members

   4:  

   5:      public void Dispose() { }

   6:  

   7:      public void Init( HttpApplication context )

   8:      {

   9:          context.BeginRequest += new EventHandler( context_BeginRequest );

  10:      }

  11:  

  12:      void context_BeginRequest( object sender, EventArgs e )

  13:      {

  14:          // extend the processing pipeline by providing

  15:          // custom code for particular events, in this case

  16:          // the BeginRequest event

  17:      }

  18: }

However, as modules are static, in a sense that a module is active or inactive in processing pipeline, applications which "host" modules are dynamic, in a sense that different applications work under different conditions.

The issue we've faced recently was to inject a custom, application specific code into module:

   1: void context_BeginRequest( object sender, EventArgs e )

   2: {

   3:     // how to call an application-defined code

   4:     // but do it so that the module is not

   5:     // rewritten everytime application changes

   6:     

   7:     // ?

   8: }

The solution we've came up with was rather simple: we've used a custom provider to provide application-specific code.

   1: <system.web>

   2:     <specificProvider defaultProvider="ApplicationSpecificProvider">

   3:         <providers>

   4:             <add name="ApplicationSpecificProvider" type="Namespace.ApplicationSpecificProvider" />

   5:         </providers>

   6:     </specificProvider>

This way we can configure different providers on the application level but still be able to write generic module code:

   1: void context_BeginRequest( object sender, EventArgs e )

   2: {

   3:     HttpApplication app = (HttpApplication)sender;

   4:  

   5:     SpecificProvider.DoSpecificOperation( app );

   6: }

C# Puzzle No.12 (intermediate)

Consider following class which stores the information about external urls.

   1: class LinkInfo

   2: {

   3:      public int    Ord { get; set; }

   4:      public string Url { get; set; }

   5: }

In your application you somehow retrieve the list of such items:

   1: List<LinkInfo> l = new List<LinkInfo>()

   2:      { 

   3:        new LinkInfo() { Ord = 1, Url = "" },

   4:        new LinkInfo() { Ord = 2, Url = "test1" },

   5:        new LinkInfo() { Ord = 3, Url = "test1" },

   6:        new LinkInfo() { Ord = 4, Url = "test2" },

   7:        new LinkInfo() { Ord = 5, Url = "" }

   8:      };

Your goal is to use Linq to filter this list in a special way:

• if the Url is empty - the item is always returned


• if the Url is nonempty - only one item with such Url is returned

In the above case the Linq expression should return:

   1: { 

   2:   new LinkInfo() { Ord = 1, Url = "" },

   3:   new LinkInfo() { Ord = 2, Url = "test1" },

   4:   new LinkInfo() { Ord = 4, Url = "test2" },

   5:   new LinkInfo() { Ord = 5, Url = "" }

   6: };

or

   1: { 

   2:   new LinkInfo() { Ord = 1, Url = "" },

   3:   new LinkInfo() { Ord = 3, Url = "test1" },

   4:   new LinkInfo() { Ord = 4, Url = "test2" },

   5:   new LinkInfo() { Ord = 5, Url = "" }

   6: }

 

How effective is your solution? Can you not only think of just a solution but of an effective one?

NTE_BAD_DATA (0x80090005) on CryptImportKey

The CryptImportKey docs says that NTE_BAD_DATA can occur when importing a key when

Either the algorithm that works with the public key to be imported is not supported by this CSP, or an attempt was made to import a session key that was encrypted with something other than one of your public keys.

This has been exactly the case in my scenario. I generate RSA public/private key pair using .NET and RSACryptoServiceProvider:

   1: RSACryptoServiceProvider rsa = new RSACryptoServiceProvider( 2048 );

   2:  

   3: File.WriteAllBytes( @"capipublic.key", rsa.ExportCspBlob( false ) );

   4: File.WriteAllBytes( @"capiprivate.key", rsa.ExportCspBlob( true ) );

and then try to use these keys to encrypt/decrypt data in C++ using CryptoAPI.

It seems that the default MS_DEF_PROV provider is uncapable of importing a 2048-bit key and it just returns with NTE_BAD_DATA.

However, initializing the crypto context (CryptAcquiteContext) with more powerful CSP is enough, in this case the the MS_STRONG_PROV.

More on Crypto Service Providers here.

Password recovery mechanism for applications which store data in password-protected files

The issue

If your application stores data in files, you probably think of a mechanism to protect the data with user-provided passwords.

The common issue with password-protected files is that a lost password cannot be easily recovered. However, if the application provides a backdoor or magic unlock code it can be easily compromised and password-removing tools can be easily written.

Let's summarize few assumptions on the encrypt - recovery issue:

• the application should be able to use any known encryption mechanism to protect files
• the password-recovery mechanism should not be built into the application code but rather involve an additional actions from the application's producer (for example - I can send the file to the application's producer and they magically unlock it for me)
• the password-recovery mechanism should not allow the application's producer to retrieve the password in an plain, explicit form (because I might have used the same password to protect my files and my bank account)
• the password-recovery mechanism should provide some form of authorization so that no one is able to steal my files and ask the producer to unlock them
• the password-recovery mechanism should be as automatized as it can be so that it does not involve a physical person to unlock each single file

The idea

I use standard cryptography mechanisms to fulfill above requirements: SHA512 will be used to compute hashes, AES will be used to encrypt file data and RSA will be used to recover passwords.

The idea works like this: when the user decides to protect the file with a password, AES is used to encrypt the file data. However, the user is asked to provide both password and email which will then be used to authorize unlock requests.

The application then stores the data in an encrypted form containing three sections:

1. (E) Base64 form of the email provided by the user (so that it can be easily extracted from the encrypted document)
2. (C) AES encoded file content with SHA512(Password) is used as an actual password
3. (S) RSA encoded tuple (Email, SHA512(Password)) (RSA's public key is provided together with the application)

When a password protected file is open, the user is asked to enter his/her password:

 

Note, that the EMail info is extracted from the (E) section of the document. Although it can be easily modified in the encrypted file (it's not encrypted but only base64ed), the email is also stored in one-way RSA signature of the document [(S) section].

After user provides the password, SHA512 of it is used to decrypt the file content using AES as the decryption algorithm.

Then, the triple - decrypted document content, user email and user password - exists in the application's memory. User can then invoke change password operation which, again, recomputes all three sections of the encrypted file, the (E), (C) and (S) and stores the document.

Recovering lost passwords

To recover a lost password, user clicks the [I lost my password] link on the password input window. The application contacts provider's webservice passing three parameters to the webservice - the encrypted filename, the email extracted from the public (E) section of the document and the contents of the (S) section of the document.

Note, that the webservice knows the RSA's private key used to encrypt the (S) section of the document. The (S) section is then decoded.

There are three possibilities:

• the contents of the (S) section sent to the webservice cannot be decrypted. An email is sent from the server to the provided user email address saying that "sorry but the request cannot be processed"
• the content of the (S) section is succesfully decrypted, however the user email provided to the webservice from the (E) section and the decoded user email recovered from the (S) section do not match. An email is sent to the provided email address saying "sorry but the document context does not seem to be auhtorized to provided user address"
• the content of the (S) section is succesfully decrypted and both emails are the same. This means that the user is authorized to access the document data and an email is sent saying "dear user, this is the unlock code for your file" with the SHA512(Password) decrypted from the (S) section of the document

Note how the requirements are fulfilled. The email is stored independently in two sections of the document and the unlock code is generated only when these two addresses match. On the other hand, the provider's service get's no access to plain password but rather it knows the SHA512 hash of it.

The user is then asked to enter the unlock code (which in fact is the SHA512 hash of his password):

Since the application now knows the SHA512 hash of the password, it uses it to AES-decrypt the document data (remember that the hash is used to encrypt the data, not the password itself).

Note, that very few users will try to unlock files encrypted with an email which they are not able to use anymore. As a emergency procedure you can ask users to send such files directly to the service provider where the authorization part can be skipped but then some other authorization mechanisms have to be used.

The implementation

As usual, the reference implementation is provided (fileencrecov.zip) containing two Visual Studio 2008 C# projects: a dummy application and a recovery webservice. The webservice does not send real emails! Instead, a dummy implementation is provided and "emails" are stored in the application's server directory so you can open them with the Notepad and copy/paste the unlock code to the application.

Where to go from there

The idea and the implementations are provided "as is" which means that I do not take any responsibility of any holes in the recovery protocol which can be used to compromise it. You are free to use the code, however, if you wish to use it in a closed-source application you have to notify me of it.

You are also free to yet tune the protocol so that for example salts are used when computing hashes so that the provider's webservice cannot easily store the requests and perform a dictionary attack to retrieve users' plain passwords.

If you belive that the idea or the implementation is flawed in any way, please drop a note so others do not use something which has some fundamental problems.

LinqToSQL: how to replace DELETE with UPDATE

Today I wrote a small web application which uses LinqToSQL and LinqToSQLDataSource to connect to the data source. This time, however, I thought that I could use a common pattern for deleting: instead of physically deleting records from the database I was going to update them (and set object.Deleted = true).

My first successive approach was to plug into LinqtoSQLDataSource processing pipeline:

   1: protected void TheLinqDataSource_Deleting( object sender, LinqDataSourceDeleteEventArgs e )

   2:  {

   3:      /* an item to delete is passed in arguments */

   4:      ACTIVITY Item = e.OriginalObject as ACTIVITY;

   5:      

   6:      /* create a new context */

   7:      ActivityMonitorClassesDataContext context = 

   8:         new ActivityMonitorClassesDataContext();

   9:      

  10:      /* retrieve the item once again in the new context */

  11:      ACTIVITY ItemToModify = 

  12:         ( from i in context.ACTIVITies where i.ID == Item.ID select i ).Single();

  13:  

  14:      /* update it */

  15:      ItemToModify.Deleted = true;

  16:      context.SubmitChanges();

  17:  

  18:      /* cancel the Deleting event from the processing pipeline */

  19:      e.Cancel = true;

  20:  }

This works like a charm but then I thought Wait! Why do I hack the view's processing pipeline? Couldn't I just pretend that the object is deleted and rather modify the way LinqToSQL actually deletes data?

So I removed the TheLinqDataSource_Deleting event and started to search for a place where I could plug into Linq's processing pipeline.

It seems that there are few partial methods generated in the LINQ model:

   1: [System.Data.Linq.Mapping.DatabaseAttribute(Name="ActivityMonitor")]

   2: public partial class ActivityMonitorClassesDataContext : 

   3:     System.Data.Linq.DataContext

   4: {

   5:     

   6:     private static System.Data.Linq.Mapping.MappingSource 

   7:         mappingSource = new AttributeMappingSource();

   8:     

   9: #region Extensibility Method Definitions

  10: partial void OnCreated();

  11: partial void InsertACTIVITY(ACTIVITY instance);

  12: partial void UpdateACTIVITY(ACTIVITY instance);

  13: partial void DeleteACTIVITY(ACTIVITY instance);

  14: #endregion

  15:  

  16: ...

If you provide an explicit implementation of these methods (Ben Hall explains on how Linq knows if these methods are implemented), you can alter the way Linq inserts, updates and deletes items.

At first I thought that following will work:

   1: public partial class ActivityMonitorClassesDataContext

   2: {

   3:     partial void DeleteACTIVITY( ACTIVITY instance )

   4:     {

   5:         instance.Deleted = true;

   6:  

   7:         /* redirect delete to update */

   8:         this.ExecuteDynamicUpdate( instance );

   9:     }

  10: }

however, it does not work. Linq knows that the entity should be deleted so the query generated for the update clause does not contain any new values and you end up with SqlException saying that there is an error near WHERE (the update query looks like this: "UPDATE ... SET WHERE ..." with just a blank space between SET and WHERE).

I would still love to see the code above working, in the meantime however, following code works:

   1: public partial class ActivityMonitorClassesDataContext

   2: {

   3:     partial void DeleteACTIVITY( ACTIVITY instance )

   4:     {

   5:         /* duplicate the context with its transaction */

   6:         ActivityMonitorClassesDataContext context = 

   7:             new ActivityMonitorClassesDataContext( this.Connection );

   8:         context.Transaction = this.Transaction;

   9:  

  10:         /* attach and modify the instance */

  11:         context.ACTIVITies.Attach( instance );

  12:  

  13:         instance.Deleted = true;

  14:  

  15:         context.SubmitChanges();

  16:     }

  17: }

I would love to learn a cleaner way to make it work.