I think this is a useful design pattern that can be valuable to most of our clients dealing with time-sensitive data.

It is time-consuming for an application to check if there are new payments on every request. This check is better off being uncoupled from the actual user request. Then, the check should be handled backend by a cron job or similar every second, minute, hour or whatever suits the data in the system.

The Paypal example has these advantages:

• Faster response times — does not wait for a backend account check
• Server less sensitive for extreme load because each request uses less cpu power
• Server can handle more users because each request ends faster

Any other pros or cons you like to add?

Technorati Tags: usability, performance, user experience, paypal, response time

Recently I was called out to debug some javascript code that did not perform. There was really not a big problem in Firefox, but Internet Explorer hung using 20 seconds or more each time the script was activated by the user.

It was one of those applications where you click on a little plus to expand a list of units in a subgroup. The list that caused the trouble contained 1,416 rows.

The subgroup was invisible and already rendered. On click a display style property changed from

to

. Unfolding the list took 19 seconds in IE. WHAT?

A first look

A brief investigation hinted something was wrong:

• Content/markup ratio: 23%/77%
• Page contained 27,000 elements in the DOM
• W3C HTML Validator unable to validate
• Firefox HTML validator gave 6 HTML errors and 15,700 warnings
• No JavaScript errors, though

There were plenty of transparent spacer gifs, and the spacer gifs were used in empty table cells. A brief count in the DOM revealed that 5,400 of the 7,200 images were transparent gifs. Uh oh.

Most important pitfall: TOO many elements in DOM

JavaScript really was not the problem here, because there was nothing but an innocent

statement. The problem had to be rendering related and I got suspicious as the hidden list contained:

• One giant table element with
  display:none
• Inside that 1,416 rows
• Each of the rows had a table cell with one new table inside it.
• Each nested table had 15 cells (four of which contained only transparent spacer.gifs)
• Tables nested six levels deep on the page

All this equals bloated HTML and 27,000 total HTML elements on the page. Here’s how we got down to 11,000 elements without removing any content:

Strategy for reducing element count

These were my recommendations for reducing element count and thereby improving performance:

Avoid JavaScript. We found that in this situation, the javascript performance had big variations on different browsers (and on the same browser, with different people logged in). In this case, we were able to avoid JavaScript, which is not always the case.

Remove validation errors and warnings. Browsers are coded after the HTML specifications, HTML specifications tell browser vendors how their code should interpret valid HTML. If the code is not valid, sometimes browsers guess what they think the source code means. The only thing specified in the HTML standard is how valid code behaves. Invalid code is left for the browser to do whatever it pleases. If, for instance, a

element is nested directly in a

element, it’s not allowed. But what will the browser do about it? Render the element? Or silently ignore it? Can you submit forms at all from the page? If your form cannot submit, it can be a big risk to use HTML that does not validate.

In this particular case, there were 6 errors and 15,700 warnings (all of which could be easily fixed).

Remove elements without content. Originally each of the 1416 rows had its own table element. Additionally, a lot of transparent images were placed in empty table cells. All of this clutter was removed. We ended up using six table columns in the parent table, compared to originally 15 columns in each of the 1416 child tables. This really adds up when you consider that in our baseline example we had 1416 rows. All in all we reduced HTML footprint with 40%.

Result

Long-term findings

• Don’t use JavaScript for general page layout rendering. Think really really hard if it can be done without JavaScript. One example: Print should be done with CSS
• If there is no way around it, use standard JavaScript libraries were browser bugs, etc are hidden for the programmer
• Tables used for layout are evil. Use tables for table-based content ONLY
• HTML code is really messy and hard to maintain. Should be cleaned up to avoid similar situations in the future. Clean up nested tables, table cells with transparent gifs. Use CSS where appropriate could shave up to 50-70% of HTML footprint
• Many HTML validation errors. Code improperly nested, forgotten end-tags: Ideally, the pages should have no validation errors. It’s an achievable work worth the effort and very educational way of working with the code

Long-term benefits

In general, several benefits could be gained by raising the knowledge of HTML, CSS, Javascript, web standards, and browser issues.

• Include easier maintenance
• Lower risk of change
• Lower risk of browser incompatibilities
• Lower risk of browser crashes

Browser tools

These tools I used for quick investigation of the HTML page:

• HTML validator Firefox extension by Marc Gueury
• Firebug Firefox extension by Joe Hewitt
• Content/markup ratio with pageinfo bookmarklet by Slayeroffice (Steve Chipman) who has a cat named Javascript!
• Counting of HTML elements with JavaScript shell bookmarklet by Jesse Ruderman:
  document.getElementsByTagName(‘*’).length

  could be done dicectly from the browser address bar by typing:

  javascript:alert(document.getElementsByTagName(‘*’).length);

More info on my firefox extensions in the article: Web developer’s collection of browser tools (Nov 30th, 2005).

Technorati Tags: webstandards, w3c, browser errors, code, performance, javascript, html, dom, spacer.gif,

We were on holiday in Italy but as my wife is a big Madonna fan, I called in vain to buy tickets. Here’s my experience:

1. Called the number, from my cell phone, phone responded “network busy” (the busy network must have been at Billetlugen).
2. Tried again as soon as possible (usually every 10 seconds for the next 3 hours).
3. Eventually I got through and was presented with this menu:
   • Press 1 for purchase of Madonna tickets
   • Press 2 for purchase of other tickets
   • Press 3 for other
4. I pressed 1 and got the message “there are currently no available salespersons. Try again later”. Then, the line was disconnected! DISCONNECTED????
5. I simply couldn’t believe that this was intentionally implemented in their system, so I stubbornly called up again until I got through. It happened all three times I got through to Billetlugen.

I have never heard of broken queue system this way. It’s the worst user experience I ever had. They bloody disconected just when I thought i got through the bottleneck.

I wasted over three hours this way. I join Michael Heilemann in his point that here is definately room for improvement (For a start, when people call in, try NOT to disconnect them before they have purchased tickets :) .

A few of my friends and colleagues had the exact same experience, and they also experienced website dead as a zombie. I was also told that 500,000 people had tried simultaneously to buy tickets which made the systems break down.

On Billetlugen’s homepage I found this image “Powered by iTicket and .Net”. Not the best advertisement for Microsoft .Net, that Billetlugen breaks down.

One last thing: I called in vain every 10 seconds for the first two hours. thats about 700 calls. Ironic that they could have served me in one call…

More info on Michael Heilemann’s blog: Boycot Billetlugen

PS. If anybody have two tickets to spare, please let me know!

Technorati Tags: usability, user experience, concert tickets, ticket systems, billetlugen, michael heilemann, madonna, horsens

]]> http://justaddwater.dk/2006/05/13/ticket-systems/feed/ 7 http://justaddwater.dk/2006/01/14/ajax-performance-stats-roi-and-business-value/ http://justaddwater.dk/2006/01/14/ajax-performance-stats-roi-and-business-value/#comments Fri, 13 Jan 2006 23:10:30 +0000 Jesper Rønn-Jensen http://justaddwater.dk/2006/01/14/ajax-performance-stats-roi-and-business-value/ AJAX usability metrics), AJAX ROI faceoff, where a traditional webapp is compared to an AJAX webapp. For the visually oriented, there is a video comparing the two applications.]]> How do you build a system that can deliver and update content to 100,000 people simultaneously? Via Ajaxian.com I saw this article from MacRumors on what traffic they got when Steve Jobs delivered his keynote on MacWorld a few days ago.

On January 10th, 2006, MacRumors.com successfully delivered live updates of the Macworld San Francisco Keynote speech to over 100,000 people simultaneously using the latest web technologies.

“By far the best mac keynote coverage I have ever seen. No constant page refreshing. No ‘page will update every three minutes’ even though they never do. I just let the page update every minute like it was supposed to. It was actually weird to get coverage without some sort of connection failure.” – MacRumors member, war

The live coverage web solution is basically a transscript updated frequently during the keynote. One article is about the traffic they experienced:

Our site was linked to widely across the web, including prominent links from Digg and Slashdot — two popular technology sites. These links caused no noticable “spike” in our number of visitors, illustrating the magnitude of traffic that MacRumorsLive was already receiving.

We peaked at approximately 103,000 simultaneous web visitors and 6,000 IRC viewers during the Keynote speech and transmited over 32 GB of data in a three hour period. If not for the efficiency of the MacRumorsLive AJAX update system, the same webcast would have required approximately twice as many servers and would have had to transfer almost 6 times as much data (196 GB).

With peak loads that reaches insane levels of almost 4,000 hits per second:

Hits/second (2)	3,812
IRC users	6,079
Simultaneous visitors	109,000
Bandwidth (3)	approx. 77Mbit/sec

Hits per second from ~8am to ~12:30pm. Average over each minute.

They also posted an article about how they built the system which is insanely interesting

Traditionally, websites have coped with visitors during the keynotes by swapping to text-only pages, using HTML Meta Refresh to reload the page at set intervals, usually every three minutes, for example, MacRumors’ coverage from MacWorld in January 2005. As a result, the sites lose their identity and branding, and cannot make best use of the additional traffic for site promotion.

Even with all this simplification the sheer number of visitors can still take the server down. To provide the best coverage during live events an improved system is required.

As this system is retaining the normal layout of the site, all the necessary objects (CSS, images & Javascript) also need to be loaded. This increases the load placed on the web servers considerably in comparison to the old system, which only had a single text file to display. Even with the ability to use multiple servers, the web server software needs to be improved.

By using PHP to generate the necessary HTML & XML files every ten seconds, rather than each time the file is requested, it is possible to use small & simple web servers to deliver the public site, without the overhead of database usage. Two web server software packages are employed – lighttpd and thttpd – with Apache still being used for the administration interface.

The efficiency of these packages over Apache is considerable and allows quite modest hardware to be used to serve a large number of visitors. Benchmarks of lighttpd compared to various other packages can be found on their website. In our tests, thttpd was able to sustain in excess of 3,300 hits per second, whereas Apache managed only 600 hits per second for 20 seconds, before overloading the server due to memory usage. [1]

The use of these more specialised web servers allows a vast number of simultaneous visitors to be supported – as the system stands at the moment, in excess of 200,000.

This reminds me of Ajaxinfo (the guys behind AJAX usability metrics), AJAX ROI faceoff, where a traditional webapp is compared to an AJAX webapp. For the visually oriented, there is a video comparing the two applications at the end of the article.

Recently on Developer.com, I looked at ways for developers to communicate the value of Ajax in dollars and cents. Arguing for or against a technology usually creates some stress for developers, who tend to speak a different language from business decision makers.

Developers talk about Usability Bandwidth Ramp-up Cost of Ownership

Managers talk about Efficiency Transaction costs Training costs Return on Investment (ROI)

Right at the point. Performance website or not, the metrics he mentions above are almost like something Jakob Nielsen could have done. Conclusions:

Although every new technology should be greeted with a healthy amount of skepticism, there are clearly demonstrable, quantifiable advantages to using an Ajax architecture in a Web application. These cost savings originate primarily from time savings, but also from reductions in bandwidth requirements.

A representative test case showed that a business can save between 500 and 2,800 man hours per year on a 10-step hypothetical process, saving roughly 4 seconds per step (a between 30% and 70% reduction in labor costs).

Although the benefits of improved application architecture extend beyond mere time savings, when included in the decision making process, an ROI approach such as this can help make a solid business case for Ajax.



The benefits listed here are very similar to benefits when we’re arguing for web-standards: Reduced bandwith, training costs, cost of ownership, and increased return on investment. But for the time being I still believe that AJAX has increased learning costs for developers. (Although most developers are really keen on using AJAX and webstandards it still requires expert knowledge regarding libraries, backend integration and browser compatibility — to name a few).
I’d be happy to hear your comments on this. Please share more info here. Also check out Ajaxinfo’s Call for AJAX Studies.

UPDATE: Just saw Jonathan Boutelle’s Bandwidth Savings with AJAX blog post, that covers the same story and has a good point on AJAX fit frequently used applications:

The old-school RIA technology vendors always made the claim that using the RIA paradigm saves you bandwidth. Numbers on this were always hard to get, and hard to trust.

Theoretically it always made sense that an RIA would consume less bandwidth (in exchange for a larger initial download). The longer a user continues to use the application, the greater this benefit (so an RIA word processor might make more sense than an RIA news site).

Now, watching a keynote presentation takes at least an hour, so it’s definitely long enough to provide benefit. What we see in this case is really different though: AJAX is basically being used as a broadcast technique, grabbing new data every minute so that the user doesn’t have to refresh the entire page. For covering live events that have a large audience, this makes a ton of sense. Expect to see more content sites “pushing” content out to their users this way.

Technorati Tags: macrumor, ajax, performance, ajaxian, macworld, architecture, webdevelopment

]]> http://justaddwater.dk/2006/01/14/ajax-performance-stats-roi-and-business-value/feed/ 22