Form Generation: Performance Impact

I was recently asked about the performance impact of using Metawidget. Metawidget dynamically generates forms at runtime, so it's expected there will be a performance impact over hard-coding those forms statically: rendering speed and memory consumption will probably get worse. Conversely, deployment size will probably get better (screens can be generated on-demand, rather than pre-compiled and shipped with the app).

Of course, everybody hates micro-benchmarks. They're easy to quote, but rarely very accurate, and almost impossible to make meaningful. The best advice is try Metawidget with your application, in your environment, to see if it performs acceptably.

That said, and with huge pinches of salt, I thought I'd give it a go. Metawidget already ships with 3 examples of 'retrofitted' applications - that is, existing applications that have been upgraded to use Metawidget. These make for great 'before and after' tests of two versions of the same application where Metawidget is the only variable.

I took the existing JBoss Seam Hotel Booking sample, the JBoss Seam DVD Store sample, and the JBoss Seam Groovy Booking sample, and wrote a Webtest to cycle over them 100 times. I then wrote a similar Webtest (some of the ids are slightly different) to cycle over the Metawidget version. These webtests are under /test/src/web/examples/faces.

Next I...

1. took my PC (Intel Core 2 Duo 8400, 4GB RAM, Windows XP SP3)


2. built the samples from JBoss Seam 2.2.0.GA


3. deployed them on a fresh copy of JBoss 5.1.0.GA and Java 6 update 17


4. set all logging to WARN (I have found logging skews the benchmarks)


5. ran the Webtests

...and timed the result. I did this a bunch of times and it generally came to...

Seam Booking Test (seam-booking-performance-test.xml)
test-without-metawidget	7m 44s
	7m 55s
	7m 50s
test-with-metawidget	6m 56s
	6m 42s
	6m 48s
Seam DVD Store Test (seam-dvdstore-performance-test.xml)
test-without-metawidget	5m 17s
	5m 17s
	5m 21s
test-with-metawidget	5m 40s
	5m 47s
	5m 38s
Seam Groovy Booking Test (seam-groovybooking-performance-test.xml)
test-without-metawidget	48m 1s
	47m 50s
test-with-metawidget	45m 1s
	44m 6s
	44m 33s

Finally I did a quick graph of the results (using GraphJam):

As you can see, the results are all over the place. The Hotel Booking and Groovy Booking samples actually run faster with Metawidget (14% faster and 7% faster, respectively), and the DVD Store sample runs slower (7% slower). The latter result is expected, the former is a surprise. I tried it on another PC just to confirm, and I'd encourage you to try it yourself.

What does this mean? As I said at the start - not very much. Does it mean your application will run faster with Metawidget? No. As they say your mileage may vary. Does it mean Metawidget probably won't have an appreciable impact on the performance of your User Interface and so you should take a look at it? Definitely :)

Feedback welcome!