Tuesday, July 3, 2007

Tiny ASP.NET Ajax Framework Contest Continued...

Few days ago we've held a tiny contest of three ASP.NET Ajax frameworks. This time, three more contestants await to be compared:

These three above were chosen because the integration of an existing application is as simple as placing a single Ajax Manager on a web form - few other frameworks (and here) were rejected because they are either dead now, force you to use specific set of components or rely on writing client-side javascript code.

The result chart is available here:

  • ComfortASP.NET has a terrible requests-per-second ratio. It served only 35 (!) requests per second, while sending 6 and receiving 83 MB of data
  • Anthem.NET was able to process 133 request per second, sending 6 and receving 121 MB of data
  • Telerik's RAD Ajax did 292 requests per second, sending 7 and receiving 172 MB (!) of data

Gathering all results together, we had 2 non-Ajax contestants: pure Web.Forms application, Javascript version of it and 6 Ajax frameworks: Microsoft's ASP.NET AJAX, Ajaxium, VisualWebGUI, ComfortASP.NET, Anthem.NET and Telerik's RAD Ajax.

Regarding requests-per-second we have following results:

 

 

requests-per-second

reference

VWG

422

198,1220657

Javascript

400

187,7934272

Telerik

292

137,0892019

ASP.NET AJAX

222

104,2253521

Web.Forms

213

100

Anthem

133

62,44131455

Ajaxium

111

52,11267606

Comfort

35

16,43192488

 

Regarding number-of-bytes-sent we have following results:

 

 

sent data

reference

Javascript

1938800

24,23560589

VWG

2066800

25,83564589

Ajaxium

5029800

62,87407185

ASP.NET AJAX

5600600

70,00925023

Comfort

5880400

73,50683767

Anthem

6161400

77,01942549

Telerik

7293200

91,16727918

Web.Forms

7999800

100

 

Regarding number-of-bytes-received we have following results:

 

 

received data

reference

VWG

23348807

29,11700146

Javascript

37121000

46,29153905

Ajaxium

47756301

59,55423272

Web.Forms

80189600

100

Comfort

83742600

104,4307491

Anthem

121547800

151,575516

ASP.NET AJAX

129531800

161,5319194

Telerik

172294800

214,8592835

 

Few comments:

  • the overall winner is VWG, however the application had to be completely rewritten in order to run under this framework
  • the javascript version did not win (even in the bytes-received test!) and though this may seem surprising, it is caused by the script size
  • the Ajaxium, Anthem and Comfort reduce the requests-per-second ratio singificantly. The latter two are able to correctly handle dynamic, non-ajax content sent to the browser
  • regarding the requests-per-second, Telerik is significantly better than ASP.NET AJAX, however it downloads huge amout of data comparing to all other frameworks

Please feel free to study these resulsts and pick up your own winner. Thanks again to Krzysiek Owczarek for contributing to this contest.

5 comments:

Vlad said...

I'm just curious:

- did you use Telerik RadAjaxManager or RadAjaxPanel for your tests? If you are familiar with Telerik AJAX concept you will probably know that using the manager you can select which control will instantiate ajax and which will be updated on this particular ajax call.

- do you refer to AJAX.NET or ASP.NET AJAX? AJAX.NET is a bit different than ASP.NET AJAX.

- why you didn't use triggers for the UpdatePanel. Without triggers your test will be equivalent to the postback-based one.

- I'm not so sure however it seems that VWG does not deal with important things like ViewState, EventValidation, etc.

- Can you use VWG framework to ajaxify existing ASP.NET Page similar to Telerik RadAjax or Microsoft ASP.NET AJAX?

Vlad
www.telerik.com

Wiktor Zychla said...

Vlad,

1. we used RadAjaxManager with no further enhancements. we've decided to take the simplest route with each framework, and for telerik's this was the simplest method of ajaxing the form

2. thanks for pointing out the naming collision - I've updated the article to correctly name the Microsoft's ASP.NET Ajax. AJAX.NET has been "disqualified" because it relies on writing client-side javascripts.

3. I am sure that using triggers could singificantly reduce the amount of data sent to and from the server, however we've decided that the form's logic is just too complicated and the application of triggers should just require too much caution.

4. I think that vwg does not have to deal with viewstate or event validation because the state is persistent on the application server. thus, even with self-prepared request, you cannot alter it in any way which would be impossible using the web browser as the request source. as it sends your activity to the application server and the server responds with the data used to alter the view, all the data is stored on the application server.

5. no, you cannot ajaxify existing applications easily with vwg. that's its only shortcoming comparing to other ajax frameworks and that's also the reason the contest is not quite fair - we've rewritten the application for vwg and used only ajax managers for other frameworks. I just wanted to know how vwg compares to other, much simpler ways of ajaxifying your web applications.

regarding the telerik's rad ajax - to us it's the real and viable alternative to microsoft's asp.net ajax - telerik works faster and is able to handle few scenarios which confuse microsoft's.

thank you for your comments, keep up your good work on your framework.

Regards,
Wiktor

Anonymous said...

RE: "no, you cannot ajaxify existing applications easily with vwg."

I believe that, with VWG, you can use "gateway handlers" to, for example, render images, etc... and use VWG apps like AJAX handlers.

Moreover, VWG is, of course, an AJAX implementation and one can define his/her own events, etc... and add them to the framework.

The limitations that I found with VWG were inherited from WinForms (i.e. WPF enabled complex compositions, for example) although I've read that future versions of VWG will enable this type of rendering.

Some of the other limitations of VWG are, for example, the "drag and drop model" which is far too limited; i.e. the DoDragDrop function (part of WinForms) is far more powerful.

3DTechnikCom said...

Hello Wiktor,

thank you for this research and the effort you put in this. For the reason that this report is round about four years old and VWG came up with a few new versions:
Do you have some current studies? Or have you heard of an actual study like this?

Any information regarding speed comparisons between these frameworks is highly appreciated.

Thank you in advance,
Timo

Wiktor Zychla said...

Timo,

I do not have any current data on this. I guess it's not only VWG which has been updated since then but also I know that at least one of the frameworks does not exist at the moment (Ajaxium).

Thanks for your note,
Regards,
Wiktor