Thursday, March 1, 2012

Metawidget @ JavaOne 2011: Parleys Presentation

Parleys have posted videos from sessions at JavaOne 2011. This includes our talk on DRY UIs: Let the metadata do the work:


This is the same talk we gave at the JBoss Booth, but with a 10 minute Question and Answer session at the end. Please try the Full Screen mode if you find the coding examples difficult to read.

My thanks to all those who attended, and to my co-speaker Dan Allen.

Friday, February 24, 2012

Metawidget and AeroGear

Recently I've been working with the JBoss AeroGear team. AeroGear is a new JBoss initiative focussed on providing examples, tutorials, and techniques to help developers get going with all kinds of mobile development.

We've been creating a 'proof of concept' integration between JBoss Forge and Metawidget for scaffolding mobile apps. Like the Forge/Metawidget/JSF scaffold, it uses static generation and doesn't require Metawidget at runtime. Unlike the JSF scaffold, however, the AeroGear scaffold generates a POH5/JQuery app. It's heavily based on Jay Balunas' excellent AeroGear Maven archetype - thanks Jay!

The Story So Far

An update on our progress. The plugin is hosted at https://github.com/forge/scaffold-aerogear. First download the latest version of JBoss Forge and run it. Then install the plugin:

$ forge git-plugin git://github.com/forge/scaffold-aerogear.git

Setup your project:

$ new-project --named test --topLevelPackage com.test --type war
$ persistence setup
? [provider=ARG (of type org.jboss.forge.spec.javaee.jpa.api.JPAProvider)]: HIBERNATE
? [container=ARG (of type org.jboss.forge.spec.javaee.jpa.api.JPAContainer)]: JBOSS_AS7
$ scaffold setup --scaffoldType aerogear
$ entity --named Company
$ field string --named name
$ field boolean --named publiclyListed
$ field custom --named color
? [type=The qualified Class to be used as this field's type (of type java.lang.String)]: java.awt.Color
$ entity --named Employee
$ field string --named firstName
$ field string --named lastName
$ field custom --named dateOfBirth
? [type=The qualified Class to be used as this field's type (of type java.lang.String)]: java.util.Date
$ field int --named salary
$ field manyToOne --named company --fieldType com.test.model.Company
$ scaffold from-entity com.test.model.* --scaffoldType aerogear

This will generate a complete app, ready to import into your favourite IDE and deploy. To save time, you can download the generated app here (click the link, then use File > Download to download the ZIP).

Kick The Tyres

Deploy the app (I tested on JBoss AS 7.0.2.Final, please try this exact version) and open it in an HTML5-compliant browser or mobile device (I tested on Chrome):


The opening screen displays a menu generated from the entities in your domain model. Here we're developing a little employee database. Click on Company:


This screen allows you to search for existing companies. But we need to create some first! Click on Create:


Fill out the fields as shown above. Things to note:
  • All the form fields and labels have been generated by Metawidget
  • This includes a custom PlaceholderWidgetProcessor for HTML5 placeholders such as 'Your Color'
  • It also includes correct HTML5 controls, such as checkbox and the new color input. Unfortunately Chrome doesn't seem to recognize this yet
Click Save to return to the search screen:


Things to note:
  • The company has been saved to the database and redisplayed in the search results, using JQuery, JSON and JPA
  • Metawidget has generated the 'name' field as an HTML5 search control
  • Metawidget has generated appropriate columns for the search results table
  • Search fields work as expected. If you click 'publicly listed' and hit Search, you'll only see companies where 'publicly listed' has been set to true. This uses JQuery, JSON, JPA and the Criteria API
Click on a row in the table to view and edit it:


Things to note:
  • Metawidget has generated a read-only view of the entity
  • This includes using HTML5 output controls
Next, click back to the Home screen, click on Employee and click Create:


Things to note:
  • Metawidget has generated HTML5 date and number controls where appropriate. Again, Chrome's support is limited
  • There is a many-to-one relation between an employee and their company. This is presented as a dropdown. Possible values are populated using JQuery, JSON and JPA
  • The values in the dropdown use the toString of the company object. This uses a custom ObjectMapperProvider
Click Save and create some more employees:


Searching on a many-to-one relation also works:


Things to note:
  • The 'Company' column in the table uses the toString of the company object
The Road Ahead

That's it so far! We hope this gives a compelling proof of concept for generating mobile applications using AeroGear, JBoss Forge and Metawidget. But there's lots more to do, so please join the community and start contributing!

Wednesday, February 1, 2012

Metawidget v2.0: Runtime and Static Form Generator

Version 2.0 of Metawidget is now available! We bumped the version number to reflect the fact we've broken binary compatibilty: this release no longer supports JDK 1.4. It also represents a significant maturing of the static form generation code:
  • Drop support for JDK 1.4 (was causing a number of issues with frameworks that scan the classpath)
  • Implement long-standing refactorings (taking the opportunity now that we've broken binary compatibility)
  • Static Metawidget improvements (ready for Forge CR1)
  • Bug fixes, documentation and unit tests
Existing code should not be impacted too much. Breaking refactorings were:

  • There is no longer a separate Java5Inspector: it has been rolled into PropertyTypeInspector
  • FacesInspector has been renamed FacesAnnotationInspector
All existing examples have been migrated as a point of reference.

As always, the best place to start is the Reference Documentation:

http://metawidget.org/doc/reference/en/pdf/metawidget.pdf


Your continued feedback is invaluable to us. Please download it and let us know what you think.

Friday, January 27, 2012

What Do You Want In A Generic DAO API?

I was recently involved in a discussion around a 'generic DAO API'. There's been a lot of Java EE goodness in recent years. Specifications such as JPA, EJB, CDI and JSF have made great strides to simplifying web application development. But anybody attempting to build a CRUD application in Java EE 6 will find there are still significant pieces missing.

Like many others, Kennard Consulting have had their own, proprietary 'generic DAO API' for many years. It's got all the usual features like CRUD, support for relationships, pagination and sorting, bookmarkable URLs, and so on. After 5 years of living with it, it's easy to be retrospective. There are certainly some things we'd do differently!

But I'll just highlight a few things that have worked out quite well. They're a bit unusual, but I think they're things any 'generic DAO API' should at least consider:

Custom EL Resolver

A common (anti?) pattern I see is this:

@Named
public class PersonView
   extends GenericView {

   // Just for @Named
}

In most domains, there are lots of entities. And if your GenericView is any good, it should be able to supply most of the common operations for most of them. But then you still have to derive a skeleton class from the (abstract?) GenericView just to expose it to JSF. This results in a huge number of boilerplate View classes (one per entity).

For our API, we instead wrote a custom EL resolver that knows how to map an EL name to a GenericView automatically (in cases where one hasn't been explicitly @Named). This has saved us a significant amount of code.

To QBE or not to QBE?

Query By Example (QBE) is a great way to develop search screens for CRUD applications. You can specify all the search fields as a Java Object, then write some generic code to translate that into JPA-QL. I would highly recommend a generic DAO API allow you to query using a partially populated version of an entity.

But don't just limit it to the entity class itself! For example, if my entity is:

public class Person {

   public String getName() { ... }
   public Date getDateOfBirth() { ... }
}

Then sure it's great to be able to do:

Person search = new Person();
search.setName( "John Smith" );
Person found = queryByExample( search );

But it's also very useful to be able to create a specialized Search class:

public class PersonSearch {

   public String getName() { ... }

   public Date getBornAfter() { ... }
   public Date getBornBefore() { ... }
}

So that I can do:

PersonSearch search = new PersonSearch();
search.setBornAfter( "1980-01-01" );
search.setBornBefore( "1990-01-01" );
Person found = queryByExample( search );

For this to work, you'll need to be able to annotate the PersonSearch class a little:

public class PersonSearch {

   public String getName() { ... }

   @Search( value = ComparisonType.GREATER_THAN, field = "dateOfBirth" )
   public Date getBornAfter() { ... }
   @Search( value = ComparisonType.LESS_THAN, field = "dateOfBirth" )
   public Date getBornBefore() { ... }
}

But that in itself opens all kinds of interesting possibilities:

public class PersonSearch {

   @Search( value = ComparisonType.CONTAINS )
   public String getName() { ... }

   @Search( value = ComparisonType.GREATER_THAN, field = "dateOfBirth" )
   public Date getBornAfter() { ... }
   @Search( value = ComparisonType.LESS_THAN, field = "dateOfBirth" )
   public Date getBornBefore() { ... }
}

So you can do:

Person search = new Person();
search.setName( "Smith" );
Person found = queryByExample( search );

We've found such a capability very handy.

Sticky searches

How you relate your entity to your view to your search object is open to a lot of personal preference. We have the view bean manage both the entity and the search object, but that's just us. However one thing our users have asked for is the entity and the search object should have different lifecycles. In particular, the search should be session-based, so that every time the user comes back to the CRUD screen it's still there for them.

Automatic trimming

Whitespace before/after input is a frequent source of confusion.

For example, if you double-click a word in an e-mail in order to paste it into your browser, a trailing space will often come along for the ride. Equally if you drag to highlight text in your browser, you'll often pickup some preceding whitespace. This can cause searches to miss, logins to fail, and users to create 'duplicate' entries (that differ only in their whitespace).

On top of that, browsers are vague on how to render whitespace immediately following a HTML tag and immediately before actual content. For example:

<textarea> Foo</textarea>

Most browsers trim this away, which means your redisplayed values no longer match your original values.

On balance, automatically trimming all input seems to save a lot of headaches.

Security

Whilst 'security through obscurity' is a bit frowned upon, it's a great safety net! Particuarly when you get down to the level of 'make sure user A can't see record B, even though user C should be able to see it'. It's easy to miss some of the rules. But a few things have really helped:

  • Unguessable IDs: we use UUIDs for all our identifiers. For practical purposes these are unguessable - avoiding the situation where, say, a user who loads a customer record with an ID of 123 might try to load one with an ID of 124. To minimize the performance impact we store each UUID as a 128-bit integer, not as a String.
  • Obscured IDs: we encrypt every ID (whether on the URL, or inside the HTML) relative to the logged-in user, preventing a scenario where one user can snoop an ID and try it under their own login.
  • Temporary names: we use randomly generated names for all our HTML fields (at least in production). Furthermore these change with every page display/POST back.
In conclusion, these are just a few things that have worked out well for us over the past 5 years. They may not apply to your particular 'generic DAO API', but we think they're worth considering. Feedback welcome!

Tuesday, January 17, 2012

Bulletproof Backups with the ReadyNAS Duo (Update)

I thought I'd do an update to one of my most popular blog posts: Bulletproof Backups with the ReadyNAS Duo:

I've had my ReadyNAS Duo nearly four years now, and have experimented with all sorts of things and reached some conclusions:

RSnapshot

I struggled for a couple years with versioned backups. Because you have quite a lot of disk space, you can get away with making multiple, complete copies of your data for daily and weekly backups. In fact, using the built-in ReadyNAS software it's the best you can do.

But if you've got thousands of files and most of them don't change often, this feels very wasteful. A while ago I discovered the rsnapshot add-on. rsnapshot only copies files that have changed, and creates hard links for those that haven't. So suddenly you can have many days/weeks worth of versioned backups in very little disk space. Awesome!

However:

Hard Links Are Evil

A warning about hard links. If your ReadyNAS ever decides it needs to check the disk (fsck), all those thousands of rsnapshot hard links will bring it to its knees. This has only happened to me once, thankfully!

Your options in this case seem to be either: kill fsck, delete all your rsnaphot backups (and hence the hard links) and run fsck again. Or wait out the mysterious blue pulsing light:

Mysterious Blue Pulsing Light

Sometimes the ReadyNAS just sits there ominously with a blue pulsing power light. You can't connect to it during these times. Worse, this mystery state can last for hours! If you're like me you installed RAIDar once when you first bought your ReadyNAS, and then forgot all about it. However RAIDar is very useful for telling you what your NAS is doing during these pulsing light times, and stopping you resetting it in impatience.

Root That Sucker

I resisted the EnableRootSSH and ToggleSSH add-ons for years, because if you use them and subsequently mess something up, you probably can't ask for support. But if you're careful they can be very useful.

For example, when configuring rsnapshot, I found it necessary to manually edit /etc/cron.d/rsnapshot and set it to:

0 9,18 * * * root /c/addons-config/rsnapshot/hourly.sh
0 14 * * * root /c/addons-config/rsnapshot/daily.sh
0 16 * * 1 root /c/addons-config/rsnapshot/weekly.sh

This is because by default rsnapshot runs at midnight, but I have my ReadyNAS configured to power down overnight.

Bringing The Horse To Water

Although there's lots of rsnapshot goodness once your data is on the ReadyNAS, actually getting it there can be troublesome. I went through a lot of different backup programs. Some of them are truly awful. I won't name and shame them, but here are some problems to watch out for:
  • do they have a special driver that syncs the data? These frequently BSOD'ed my PC
  • do they mess up the folder case when they sync, especially if you have two folders with different case at different places in your heirarchy (e.g. C:\foo\Finances versus C:\foo\bar\finances)
  • do they have catalogs/indexes/other mechanisms that make them horribly slow to copy thousands of files
  • can they do file filtering based on wildcard matching
  • can they exclude folders based on wildcard matching, especially nested folders (e.g. *\tmp\)
The absolute best I found was SyncBackSE. It does it all, and does it fast, and does it without any nasty surprises!

Teaching The Horse To Drink

Even with a great product like SyncBackSE, you still need to take care how you schedule your backups. If you've got thousands of files, even if they don't change often, your backup software still has to scan over them every time.

I settled on configuring half a dozen backup jobs. Critical files were scanned more often (say, hourly) but in small groups (say, just My Documents). Less critical files were scanned less often (say, daily) in larger groups (say, C:\Program Files).

Exactly What It Says On The Tin

The ReadyNAS is a great Network Attached Storage device (e.g. hard drives in a box), but it's pretty lousy for anything else. It's really underpowered if you try to use it as a server, or load it up with too many add-ons or streaming services. If that's what you're after, you really need more than 256MB RAM and a slow CPU. Apparently the Duo v2 is faster, but it's still only 256MB RAM.

Your Mileage May Vary

Some things I got right the first time:
  • it's definitely too slow to use as an 'active' drive that you run programs off.
  • keeping all paths relative to some distant drive letter (like N:) is handy for restores.
  • despite some misgivings, I've had no problems at all powering down the ReadyNAS and swapping drive trays as a way to take off-site backups. However, remember that the drives are EXT3 formatted. This makes them a pain to try and read in Windows, and the ReadyNAS doesn't do NTFS. So it can still take a while to restore all that data into a usable form.
But all in all, the ReadyNAS Duo is a complete win!

Tuesday, January 10, 2012

Metawidget v1.35: New Static Form Generation

Version 1.35 of Metawidget, with new static form generation is now available! This release includes the following enhancements:
  • First version of StaticMetawidget
  • Support JPA @Temporal
  • Output HTML <label> tags around labels
  • Refactor StrutsWidgetBuilder/SpringWidgetBuilder into WidgetProcessors
  • Bug fixes, documentation and unit tests
As always, the best place to start is the Reference Documentation:

http://metawidget.org/doc/reference/en/pdf/metawidget.pdf


Your continued feedback is invaluable to us. Please download it and let us know what you think.

Friday, December 23, 2011

Forge and Metawidget Discussed On JBoss Community Asylum

Metawidget, and the work we're doing with the JBoss Forge team, has gotten a nice mention on the latest episode of the JBoss Community Asylum podcast. Here's an excerpt (taken from 35m.22s):

Max: "Right now if you download Forge right now, you get a JSF Metawidget thing"

Lincoln: "Right you get a default scaffolding provider we call them, and that one right now is using - like you said - JSF and Metawidget which we're actually working on the next version of, so that... right now when you run the scaffold it basically looks at your database objects, your JPA objects, it runs through them all and generates a bunch of view files for the Create, Read, Update, Delete operations. You know, a simple interface."

Max: "So this is kind of like what seam-gen did before, right?"

Lincoln: "Right. Kind of like that. And when you look at the output of the scaffold that you would download, maybe from Beta 3, you will see this Metawidget tag and Metawidget is..."

Max: "Yeah why don't you just tell what Metawidget is?"

Lincoln: "Metawidget is a really cool framework actually, it..."

Emmanuel: "We should interview... er... forgot his name..."

Max: "Richard? Richard something? Kennard"

Lincoln: "Richard Kennard"

Max: "Yeah Kennard"

Lincoln: "...it's a really cool framework for, basically you just point this tag at an Object or a bean and it runs through that bean and looks at all the properties and builds up an interface that shows up on the page."

Max: "Yeah, and in your JSF you just have one [line] and say this is the bean I want to do and it builds the UI. And it does it for... er... Swing, and..."

Lincoln: "You can do Swing, you can do GWT, you can do JavaFX I think. Lots of stuff."

Max: "So actually it is pretty powerful, but it's not..."

Lincoln: "The problem is then you've got this tag in there and it's... if you want to customize that then you're learning the Metawidget framework. And so what we are doing right now actually is we're customizing Metawidget so that, using the same inspection process that Metawidget provides, generate real code. Materializes it into real XML, real JSF pages. Then you can go in and modify those pages just like you would have done by yourself."


Listen to the full podcast here.