Friday, March 25, 2011

Metawidget: Full Speed To Maven

The next release of Metawidget (v1.15) represents a significant rearrangement of our source tree, as we migrate the codebase from Ant to Maven.

I'm not going to knock Ant: it's been a great build tool for us, and it's much more flexible than Maven in many regards. But there are some compelling reasons for us to switch:

  • Metawidget wires up existing technologies, and there are lots of them.
    I really like the idea of a single metawidget.jar that's easy to deploy and get running, but there are some use cases where this isn't sufficient. For example, perhaps you want to save space by deploying only those Metawidget plugins you actually use. Or perhaps you need to deploy different plugins on different application tiers.

    We need a path from the easy-to-use but monolithic metawidget.jar to this 'fine-grained' use case. Maven gives us this path. So with v1.15 you'll be able to do either...


    Or you can specify fine-grained dependencies...

       <artifactId>...other modules...</artifactId>

    Maven will automatically drag in related dependencies for you, such as org.metawidget.modules:metawidget-core.
  • Metawidget needs lots of high quality examples.
    Whether it's examples on how to run Metawidget on Android, or in GWT Hosted Mode, or with JSF 2, or how to write custom Inspectors or WidgetBuilders - high quality examples are critical. Maven lets us compose each small piece of Metawidget (every example, every plugin) as a standalone project. This makes it easy to use tools such as m2eclipse to open the individual pom.xml in your IDE and start experimenting!

  • Some great tools play nicer with Maven.
    Tools such as Jenkins and Sonar, which should have a direct bearing on maintaining our code quality.
Don't Panic!

To ally any concerns, allow me to stress:

  • Metawidget v1.15 will be backwards compatible with v1.10.
    In fact, the binaries will be almost identical. Only the build system has been rewritten.
  • Metawidget will not require Maven to use.
    We will still be packaging and distributions that you can download, just as before.
All up, I don't think anything will be lost (apart from sweat and tears rearranging the code) and hopefully a lot will be gained. I've branched the old v1.10 codebase in SVN and checked in an early copy of the Maven version. I'd say it's about 75% done (and has nearly 100 POMs!)

I'd be most grateful to anybody who could check it out and provide feedback.

Wednesday, March 9, 2011

Grokking Seam Forge: Part 2

JBoss have just released Seam 3.0.0.CR2, which includes Alpha 2 of Seam Forge. It's an alpha, so expect pain, but let's take it for a spin!

  • First, download Seam Forge from here
  • Unzip to a folder of your choosing
  • Run bin\forge
Seam Forge is command-line driven, similar to Rails and Roo (though with loftier goals :). Type the sections in bold:

[no project] forge-1.0.0.Alpha2 $ new-project --named MyApp --topLevelPackage com.myapp
Use [/forge-1.0.0.Alpha2/MyApp] as project directory? [Y/n] y
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/resources/META-INF/forge.xml
***SUCCESS*** Created project [MyApp] in new working directory [/forge-1.0.0.Alpha2/MyApp]

[MyApp] MyApp $ persistence setup --provider HIBERNATE --container JBOSS_6
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/resources/META-INF/persistence.xml
***SUCCESS*** Installed [forge.spec.jpa] successfully.
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/resources/META-INF/persistence.xml

[MyApp] MyApp $ new-entity --named Person
In which package you'd like to create this @Entity, or enter for default: [com.myapp.domain]
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/java/com/myapp/domain/
Created @Entity [com.myapp.domain.Person]
Picked up type : com.myapp.domain.Person

[MyApp] $ new-field string --fieldName firstName
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/java/com/myapp/domain/
Added field to com.myapp.domain.Person: @Column private String firstName;
[MyApp] $ new-field string --fieldName surname
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/java/com/myapp/domain/
Added field to com.myapp.domain.Person: @Column private String surname;
[MyApp] $ cd ..
[MyApp] domain $ scaffold from-entity
The [forge.maven.WebResourceFacet] facet requires the following packaging type [war], but is currently [jar], would you like to change the packaging to [war]? (Note: this could break other plugins in your project.) [Y/n] y
Packaging updated to [war]
***SUCCESS*** Installed [forge.maven.WebResourceFacet] successfully.
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/WEB-INF/beans.xml
***SUCCESS*** Installed [forge.spec.cdi] successfully.
The [forge.spec.jsf] facet depends on the following missing facets: [forge.spec.servlet]. Would you like to attempt installation of these facets as well? [Y/n] y
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/WEB-INF/web.xml
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/index.html
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/WEB-INF/faces-config.xml
***SUCCESS*** Installed [forge.spec.jsf] successfully.
***SUCCESS*** Scaffolding installed.
No scaffold type was provided, use Forge default? [Y/n] y
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/WEB-INF/beans.xml
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/java/org/jboss/seam/forge/persistence/
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/java/org/jboss/seam/forge/persistence/
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/resources/forge-template.xhtml
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/resources/forge.css
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/resources/favicon.ico
***INFO*** [/forge-1.0.0.Alpha2/MyApp/src/main/java/com/myapp/domain/] File exists, re-run with `--overwrite` to replace existing files.
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/java/com/myapp/view/
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/scaffold/person/view.xhtml
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/scaffold/person/create.xhtml
Wrote /forge-1.0.0.Alpha2/MyApp/src/main/webapp/scaffold/person/list.xhtml
***SUCCESS*** Generated UI for [com.myapp.domain.Person]


  • Open Eclipse (with m2eclipse installed)

  • Choose File > Import > Existing Maven Projects and import the pom.xml at \forge-1.0.0.Alpha2\MyApp\pom.xml
  • Edit src/main/resources/META-INF/persistence.xml and remove the line that says <non-jta-data-source/>
  • (a bug)
  • Right click the pom.xml and choose Run As > Maven package

  • Take the WAR it generates under target/MyApp-1.0.0-SNAPSHOT.war and deploy it under JBoss AS 6.0.0.Final

  • Open your browser at http://localhost:8080/MyApp-1.0.0-SNAPSHOT/scaffold/person/list.jsf:
  • Click Create:
And that's it! Seam Forge has created a full web application for you, complete with JPA back-end and JSF front-end, without you having to write a single line of code. Of course there's much more to it than that, but hopefully this gives you a taste for what JBoss and the Seam Team are cooking up!

Monday, March 7, 2011

MySQL Performance: Leveraging SQL_CALC_FOUND_ROWS and FOUND_ROWS inside JPA-QL

There are lots of articles on the Web about how to optimize paginated displays. The basic problem is: in order to paginate well, you must display the total number of hits to the user:

1...30 (of 54,342)

But finding that total number of hits can be slow. In particular doing select count(*) in MySQL is notoriously slow. There are lots of possible solutions. A promising, MySQL-specific one is select FOUND_ROWS(). Using this via native SQL is easy. But exposing it inside JPA-QL (which generates its SQL automatically) is tricky. The rules for using it are:

  1. Clients must run a normal query, using LIMIT for pagination, and prefix it as: select SQL_CALC_FOUND_ROWS * from myTable

  2. Immediately afterwards, before any other SQL statements are run, clients call select FOUND_ROWS() to obtain the number of rows there would have been if no LIMIT had been set
A further advantage of SQL_CALC_FOUND_ROWS is that it can be used with having queries, whereas count() cannot.

Here's one way to leverage this inside Hibernate-based JPA. We'll be using Interceptors. Like all the solutions to this problem, this one isn't perfect, but it may suit your purposes:

package com.kennardconsulting.core.hibernate.interceptor;

import org.hibernate.EmptyInterceptor;
import org.hibernate.HibernateException;
import org.hibernate.SessionFactory;

* Hibernate interceptor to make use of MySQL's proprietary <code>SQL_CALC_FOUND_ROWS</code> hint.
* The <code>count()</code> method in MySQL is notoriously slow
* ( as it scans the entire index.
* <code>SQL_CALC_FOUND_ROWS</code> was introduced to allow clients to obtain a <code>count()</code>
* in a more performant way.
* <p>
* The rules are:
* <p>
* <ol>
* <li>Clients must run a normal query, using <code>LIMIT</code> for pagination, and prefix it with
* <code>SQL_CALC_FOUND_ROWS</code></li>
* <li>Immediately afterwards, <em>before any other SQL statements are run</em>, clients call
* <code>FOUND_ROWS()</code> to obtain the number of rows there <em>would</em> have been, if no
* limit had been set</li>
* </ol>
* <p>
* A further advantage of <code>SQL_CALC_FOUND_ROWS</code> is that it can be used with
* <code>having</code>, where <code>count()</code> cannot.
* <p>
* Other approaches we tried:
* <p>
* <ul>
* <li>Hibernate supports ScrollableResults, but on MySQL that seems to just bring back the entire
* results and is very slow</li>
* <li>You can try embedding count(*) as an extra column in the query, but that negates the indexes</li>
* <li>You can try 'show table status where name = Member' but that varies wildly and is
* disconcerting for the user (for 60,000 rows, +/- 3,000 rows)</li>
* <li>You can make 'count()' setCacheable( true )</li>
* </ul>
* <strong>Note: this is not a general purpose Interceptor. It assumes it will be used to execute a
* single select (possibly with secondary selects) and then thrown away. It cannot be
* re-used.</strong>

public class MySQLCalcFoundRowsInterceptor
   extends EmptyInterceptor {

   // Private statics

   private final static String   SELECT_PREFIX         = "select ";

   private final static String   CALC_FOUND_ROWS_HINT   = "SQL_CALC_FOUND_ROWS ";

   private final static String   SELECT_FOUND_ROWS      = "select FOUND_ROWS()";

   // Private members

   private SessionFactory      mSessionFactory;

   private byte            mSQLStatementsPrepared;

   private Long            mFoundRows;

   // Constructor

   public MySQLCalcFoundRowsInterceptor( SessionFactory sessionFactory ) {

      mSessionFactory = sessionFactory;

   // Public methods

   public String onPrepareStatement( String sql ) {

      switch ( mSQLStatementsPrepared ) {

         case 0: {

            // First time, prefix CALC_FOUND_ROWS_HINT

            StringBuilder builder = new StringBuilder( sql );
            int indexOf = builder.indexOf( SELECT_PREFIX );

            if ( indexOf == -1 ) {
               throw new HibernateException( "First SQL statement did not contain '" + SELECT_PREFIX + "'" );

            builder.insert( indexOf + SELECT_PREFIX.length(), CALC_FOUND_ROWS_HINT );
            return builder.toString();

         case 1: {

            // Before any secondary selects, capture FOUND_ROWS. If no secondary selects are
            // ever executed, getFoundRows() will capture FOUND_ROWS just-in-time when called
            // directly

            return sql;

            // Pass-through untouched
            return sql;

   public long getFoundRows() {

      if ( mFoundRows == null ) {

      return mFoundRows;

   // Private methods

   private void captureFoundRows() {

      // Sanity checks

      if ( mFoundRows != null ) {
         throw new HibernateException( "'" + SELECT_FOUND_ROWS + "' called more than once" );

      if ( mSQLStatementsPrepared < 1 ) {
         throw new HibernateException( "'" + SELECT_FOUND_ROWS + "' called before '" + SELECT_PREFIX + CALC_FOUND_ROWS_HINT +"'" );

      // Fetch the total number of rows

      mFoundRows = ( (Number) mSessionFactory.getCurrentSession().createSQLQuery( SELECT_FOUND_ROWS ).uniqueResult() ).longValue();

To use this Interceptor inside JPA-QL:

SessionFactory sessionFactory = ((org.hibernate.Session) mEntityManager.getDelegate()).getSessionFactory();
MySQLCalcFoundRowsInterceptor foundRowsInterceptor = new MySQLCalcFoundRowsInterceptor( sessionFactory );
Session session = sessionFactory.openSession( foundRowsInterceptor );

try {
   org.hibernate.Query query = session.createQuery( ... )   // Note: JPA-QL, not createNativeQuery!
   query.setFirstResult( ... );
   query.setMaxResults( ... );

   List entities = query.list();
   long foundRows = foundRowsInterceptor.getFoundRows();

} finally {

   // Disconnect() is good practice, but close() causes problems. Note, however, that
   // disconnect could lead to lazy-loading problems if the returned list of entities has
   // lazy relations


Hope that helps!

Thursday, March 3, 2011

Eclipse Helios: Missing semicolon JavaScript Warnings

If your Eclipse Problems tab looks like this after upgrading to 3.6.2 (Helios)...
...and you're having a hard time turning that off, here's what appears to be the answer:

The Suspend all validators checkbox on the Window > Preferences > Validation screen is broken (or partially broken, its behaviour is a bit erratic). So ignore it.

Instead, uncheck that box (and leave it unchecked) and explictly turn off Client-side Javascript validation in the box underneath:

Hope that helps somebody! I've reported this as bug 338759.

Tuesday, March 1, 2011

Metawidget and Drools

Blogger Sergiy has put together a terrific example of integrating Metawidget with JBoss Drools.

He uses Metawidget to render a Drools ViewDefiniton for use in a weather forecasting application. To quote:

"One of the key aspects of Metawidget is its concept of inspectors that are responsible for extracting meaningful metadata from business objects. Metawidget comes with a number of default inspectors that allow extracting metadata based on Java reflection, annotations, etc. But what if we want something special? :)

For instance, how about using business rules as the source of meaningful metadata for a UI?"

He then describes in detail the steps necessary to create an application like this:

Read the full blog post here.

Thanks Sergiy!