On the value of reflection

The research methodologies of Action Research and Reflective Practice instruct that 'reflections' (as in contemplation, not as in type introspection) from a previous software development phase should drive the planning for the next phase (much like the industry methodology of Iterative Development). It is perhaps worthwhile to reinforce the value of this reflection. After all, reflection is expensive. Conducting experiments, interviews and case studies consumes valuable time and resources, and it is legitimate to question whether its benefits outweigh its cost.

One of the most important factors in software development is scope: deciding what to include and what to leave out. Scope creep and feature bloat are recognised risks, impacting development costs and release schedules. Good architects carefully apply rules of thumb: every design decision should 'carry its own weight', and strive to 'kill several birds with one stone'. But an implicit difficulty in evaluating this is knowing what the 'birds' are. Once out of its initial planning phases, software development has a tendency to lurch from immediate issue to immediate issue, dealing with each new requirement as it arises. Considering new requirements in isolation invariably means the burden of large-scale redesign to satisfy any one requirement will seem onerous: a smaller-scale, less impactful alternative will always seem the better option. Reflection, on the other hand, allows the practitioner to consider many weeks worth of problems in a holistic light: he can see all the birds at once, and an approach that once seemed over-engineered now appears justified. Surfacing all the issues at the same time clears a path forward that otherwise would have seemed prohibitive.

This phenomena is analogous to neural networks. While progressing to solve a given problem, a neural network may get trapped, still short of the best solution, in a local minima. The local minima itself does not represent the best answer, but none of the immediate ways out of the minima are enough of an improvement to overcome the walls of the valley. It takes a combined push, a sort of disruptive excitation, to escape the trough so that a better solution can be found:

So much for the theory - is it demonstrable in practice? Here I will give personal testimony. One of the themes from Metawidget's alpha cycle reflections was support for 1-to-M relations. In itself, this seemed a corner case: difficult to support within the current architecture without a slippery slope of requirements around sorting, pagination and summary-to-detail navigation. Another theme was support for third-party UI components. The most challenging case study indicated this would have improved adoption, though it was not a primary factor. A third issue was around supporting the SWT library: the current design of 'return null to render nothing, return a dummy Metawidget to trigger nesting' was backwards for SWT's purposes, though this was being worked around in a sub-optimal way.

Individually, none of these requirements seemed enough to justify a significant reworking of the widget creation subsystem. Indeed, the theme of 1-to-M relations gnawed at me for months with no obvious solution within the existing architecture. It was only reframing it within the context of the additional requirements of 'supporting third-party components' and 'turning widget creation inside out' that a new path presented itself (see WidgetBuilders). Looking back, I realise I was probably especially resistant to seeing this path because it was in an area I had already considered and decided against.

In summary, I have found explicit reflection to be an enlightening and worthwhile use of a project's time. It is easy to skip this phase in the heady rush of pumping out release and release, but when one takes the time to properly pause for breath important insights can be gained.

A Unified Theory

I just had an interesting exchange with the guys from the OpenXava project. We discussed differences in our two approaches, as well as those of other UI generation projects, and what it would take to unify them all under a JSR one day. I think we're a long way from that day, primarily because UI generation isn't particularly 'mainstream' yet (at least, not in the sense of ORM). Still, it's often said in physics that even though we don't know what the Unified Theory is, we know something about what features it must have.

Can we say something similar about UI generation? I'll list here all those features I think are being explored, either by Metawidget, OpenXava or one of the other projects, and see if we can update this page over the years to form consensus.

Static or Runtime

Should the generation happen statically or at runtime? If runtime, how do you allow customisation? If statically, how do you allow re-running the generation when the domain model changes (without losing any customisations)?

Static: n/a
Runtime: Metawidget, Naked Objects, Woko

Modeling Language

Should the generator have its own modeling language, which developers use to describe the UI, or should it try and derive the UI automatically? Do modeling languages introduce error-prone duplication? Is automatic derivation too inflexible? Is there enough metadata to drive automatic derivation, or do we have to 'guess and fill in the gaps'?

Yes: n/a
No: Metawidget, Naked Objects, Woko

Production or Prototype

Should we expect UI generation to be able to be used in production applications, or only during a prototyping phase?

Prototype: n/a
Production: Metawidget, Naked Objects, Woko

Customisation

What sort of customisations of the generated UI are important? Graphics? Layouts? How should we facilitate them?

Pluggable: Metawidget, Woko
Search-based: Naked Objects

Bounds of Generation

Should we try to automatically generate the whole UI, or just pieces of it? Is generating the whole UI flexible enough? Is just generating pieces useful enough?

Whole UI: Naked Objects, Woko
Just pieces: Metawidget

Multiple Platforms

Is supporting multiple platforms (eg. desktop, web, mobile) important?

Yes: Naked Objects, Metawidget
No: Woko

Consistency

Given the same domain model, should we try and produce a consistent UI across all platforms? Does this risk a 'lowest common denominator'? Does tailoring uniquely to each platform introduce too much work for the developer?

Yes: n/a
No: Metawidget, Naked Objects

Diverse Architectures

Should the generator care about diverse architectures? Is mandating the technology stack of the application too restrictive? Or should we try to enforce 'good coding' that way? Does supporting multiple versions of everything introduce too much complexity? Is the ability to retrofit existing applications an important goal?

Yes: Metawidget
No: Naked Objects, Woko

Third Party Components

Should we support third-party UI components? What if they are not available on all platforms (eg. desktop, web)?

Support third-party: Metawidget
No explicit support: Naked Objects, Woko

Non-Domain Model Objects

Should we support modelling objects that are not strictly part of the domain? 'Solution space' objects, such as search screens that are user-centric (even role-centric) and not persisted to long-term storage?

Support non-domain model objects: Metawidget
No explicit support: Naked Objects

---

Convergence

Following conversations (some below, some in newer blog entries) with various parties, we note some convergence with future releases of products. Specifically:

Naked Objects: 4+ supports deriving metadata from different sources. Newer viewers may allow generating just pieces of the UI, and also modelling non-domain objects. Future versions want to support third-party components.

Woko: 2+ will allow more diverse architectures.

Death by Annotations

Every so often I read how 'we're all going to be drowning in annotations' or how someone's 'annotations are exploding' and consequently how 'OMG annotations suck'. I'd agree it's a problem. But, seeing as how Metawidget is probably going to make it worse, not better, I thought I should offer a balanced view.

Here's the basic problem. Here's a piece of Real Code, from one of my production systems, that uses JPA, JAXB, Hibernate extensions and Metawidget:

@OneToMany( cascade = CascadeType.ALL )
@JoinColumn( name = "survey_id" )
@IndexColumn( name = "number", base = 1 )
@Cascade( org.hibernate.annotations.CascadeType.DELETE_ORPHAN )
@UiComesAfter( "dateExtension" )
@XmlElementWrapper
@XmlElement( name = "question" )
public List getQuestions()
{
   return mQuestions;
}

A lot of people are going to look at that, think it looks really messy, and turn off. And I agree - it does look really messy. But it's important to consider what we've gained here, and what the alternative is:

• it's less code, and more typesafe, than 4 separate XML configuration files


• the metadata is closer to the thing the metadata is referring to


• whilst it may seem like a lot of annotations for little actual code (the bit in bold), that's backward thinking: annotations are code, albeit declarative code. And it's a lot less code to write something declaratively than procedurally

So whilst I don't necessarily think annotations are as good as life can get, I do think they're better than anything we ever had before. The challenge now is how do we make them better?

Metawidget and Rebinding

One of the cool features of Metawidget is automatic binding between the business model and the UI, using whatever technologies the underlying platform supports (such as BeansBinding).

Up until now, however, there has been a bit of a 'gotcha' with this.

For Metawidgets that don't use automatic binding, the general approach is to call setToInspect and then setValue to populate the generated UI from business model values. This technique has an implicit side effect: the values can also be repopulated as many times as you like with new source objects, without re-calling setToInspect. This allows the Metawidget to be generated once and reused many times, mitigating the performance cost of generation.

For Metawidgets that do use automatic binding, however, setValue is never used. Setting new values requires re-calling setToInspect (and re-running generation) for every new source object.

To avoid this the new release of Metawidget supports a second, lightweight version of setToInspect called rebind. This function is only relevant when using automatic binding. Using rebind, a Metawidget can update the values in its generated widgets without re-running generation. This allows the Metawidget to be generated once and reused many times.

The downside of rebind is that the rebound source object must have exactly the same set of field names as the original object, else the call will fail. It becomes the responsibility of the caller to ensure this consistency.

For an example of rebinding, see the GWT Address Book sample application.

Metawidget: Javassist versus @UiComesAfter

Update: the APIs shown in this blog entry have changed slightly in newer releases of Metawidget. Specifically PropertyStyles are now set as object instances, not as classes. Please download the latest documentation from http://metawidget.org

There's been some feedback that having to annotate your business model with @UiComesAfter like this...

public class Person {
   public String name;

   @UiComesAfter( "name" )
   public int age;

   @UiComesAfter( "age" )
   public boolean retired;
}

...to order the UI fields is a bit annoying. I agree. The problem is the Java classfile specification doesn't maintain the original declaration order of methods and fields, so we can't extract this information from reflection.

Of course, Metawidget can extract it from other places instead, such as hibernate-config.xml or metawidget-metadata.xml files, where available - but failing that you have to provide the ordering information somehow.

Well, thanks to an idea by those guys from the Tapestry team, there is now another option: JavassistPropertyStyle.

If you have Javassist available in your environment, the new JavassistPropertyStyle can use Javassist to extract debug information from the Java classfile and order the fields according to source code line number! No more @UiComesAfter! You use it like this:

<propertyTypeInspector xmlns="java:org.metawidget.inspector.propertytype"
   config="org.metawidget.inspector.impl.BasePropertyInspectorConfig">
   <propertyStyle>
      org.metawidget.inspector.impl.propertystyle.javassist.JavassistPropertyStyle
   </propertyStyle>
</propertyTypeInspector>

There are downsides. First, if you accidentally recompile your business model without debug information (like, say, when you go from development to production) all your fields will quietly unorder themselves. To protect from this, JavassistPropertyStyle fails early and fails hard if there is no debug information.

Second, JavassistPropertyStyle uses the following sorting algorithm:

• superclass public fields come first, sorted by name


• superclass methods come next, sorted by getter line number (or, if no getter, setter line number)


• public fields come next, sorted by name


• methods come last, sorted by getter line number (or, if no getter, setter line number)

This algorithm is less flexible than @UiComesAfter, which can interleave superclass and subclass properties. However, it is possible to use both @UiComesAfter and JavassistPropertyStyle together to get the best of both worlds!

@UiComesAfter versus setFields

Update: the information shown in this blog entry has changed slightly in newer releases of Metawidget. Specifically you can now use an InspectionResultProcessor to implement a .setFields approach if you so desire. Please download the latest documentation from http://metawidget.org

I was recently asked why Metawidget uses @UiComesAfter and @UiHidden annotations to define the ordering and visibility of fields, rather than something like:

metawidget.setToInspect(myLoginBean);
metawidget.setFields("login","password");

There are a few reasons:

1. Separation of Concerns

In Metawidget's architecture, Inspectors inspect objects (at runtime) and Metawidgets display them. The Metawidget itself has no compile-time knowledge of what it is displaying. This has all sorts of useful side effects.

For example, in the Swing Address Book example application, the code says:

metawidget.setToInspect(contact);

At compile time, contact is of type Contact. But Contact is actually an abstract class. At runtime, contact can be either a BusinessContact or a PersonalContact. It would not be possible to say...

metawidget.setToInspect(contact);
metawidget.setFields("firstname","lastname","dateOfBirth");

...without introducing compile-time dependencies on the type of object, which means we could no longer just have a single ContactDialog class - we'd need two separate BusinessContactDialog and PersonalContactDialog classes.

Think of Metawidget like JPA's EntityManager. If you were building a very generic CRUD app, you could use a single EntityManager instance and a single Metawidget instance to manage your whole app.

2. Duplication of Declaration

Metawidget is very big on avoiding duplicate declarations between the UI and the application. If you do...

metawidget.setFields("firstname","lastname","dateOfBirth");

...you are effectively restating that the business class has a firstname field. If ever that field name changes (or is deleted), it now has to be changed in at least two places - more if you have several screens using that Metawidget.

3. Not just Java

It's a shame the Java bytecode specification doesn't retain the ordering of fields and methods, but not all technologies have that limitation. If you use HibernateInspector or XmlInspector, for example, their XML is inherently ordered and the inspectors use that information, instead of you having to use @UiComesAfter. SQL schemas also retain field ordering (though we don't have a SqlSchemaInspector yet).

And there are some other clever approaches we can try. Tapestry 5's ClassFactory, for example, uses Javassist to extract line numbering information and sort getter methods that way.

Conclusion

I think these are compelling reasons to avoid the setFields approach. However, feedback is very welcome!

Why A Runtime User Interface Generator

I was recently asked why Metawidget works at runtime, instead of pre-compiling and statically outputting generated code. There are two reasons.

Easier to Maintain

I think people like static code generators because they give the illusion of being productive: you click a few buttons and bam! you suddenly have this whole app with mountains of code written for you, all ready to go.

But I do think it's an illusion. Let's look at what, typically, you have to do with a piece of code:

• write it
• debug it
• unit test it
• update it
• document it
• hand it over

Whilst code generators might ease the first one (write) they're no help with the third (unit test) and they actually make the other four much worse. This is because the code they generate is usually quite poor (it being generated by generic algorithms) and because nobody, not even the developer who runs the tool, initially understands the code it produces.

Runtime generators, on the other hand, are great when it comes to updating, documenting and handing over, because a) they haven't produced a mountain of code for someone to trawl through and b) documentation on how the runtime generation process works is normally very good (eg. it's in the generator's user manual).

So I'd much prefer a runtime approach to a compile-time one. If you compare systems like JPA to those that pre-generate mountains of SELECT/INSERT/DELETE stored procedures I'm sure you'd agree.

More Powerful

There are lots of important properties of a system and the relationships between its parts that are only discernable at runtime, not statically at compile-time.

For example, the Address Book sample application adapts its screens dynamically at runtime depending on whether the incoming object is a PersonalContact or a BusinessContact. And there are @FacesNotHiddenInRole annotations that modify a UI based on the currently logged-in user.

Finally, if you're inspecting 'live' objects (as opposed to source files) you can automatically do a lot of the binding between UI and application layer, further reducing all that boilerplate code.

Useful Bounds of Automatic UI Generation

A recent post on the Metawidget forums asked why Metawidget doesn't 'do more'. Given we can automatically create forms from domain objects, and given there are solutions that can automatically persist domain objects (eg. JPA), why not 'fill in the gap' and automatically create the entire CRUD app?

And indeed, there are solutions that do this. The most notable being http://www.nakedobjects.org. The drawback with these solutions is they necessarily create very generic UIs, which bear little resemblance to how they would have appeared and functioned had they been designed by hand, with due consideration to their problem domain. For example, with Naked Objects you get this...


...which is simply not how many clients want their apps to look.

Good UI design is both art and science. When you try to 'fill in the gap' you realize there's a lot of 'art' in that gap, and trying to automate that art results in less effective UIs.

So Metawidget is not trying to be that solution. I believe that solution is less useful for many real world apps. Instead, Metawidget tries to identify the bounds between where UI generation can be useful and practical and where it becomes too generic and impractical, and stays within those bounds. I call these the 'Useful Bounds of Generation'.

Staying within the Useful Bounds of Generation lets you apply Metawidget to a large category of real world applications that fully automated CRUD solutions simply aren't interested in. You can even retrofit an existing app and remove lots of your boilerplate code.

Element.getAttribute

To tell Metawidget's Inspectors we don't want a label on a field, ideally we'd like to do...

@UiLabel( "" )

...or...

<property name="foo" label="">

Unfortunately, according to the DOM Element.getAttribute specification, it does not distinguish between 'an attribute having no value (eg. foo="")' and 'an attribute not existing'. In both cases getAttribute returns an empty String.

So to be able to specify an empty String without it just being ignored as a non-existant attribute, we need to put a String with a space...

@UiLabel( " " )

...or...

<property name="foo" label=" ">

...which is a bit sucky I know.

Suggested improvements welcome :)

Update: After playing around with label="null" for a bit, I noticed DOM supplies an Element.hasAttribute method for this purpose, so you can differentiate after all.

Concurrent MetaInspector

I spent today trying different ways of utilising concurrency within MetaInspector.

It's tempting because Inspectors are inherently threadsafe and immutable, so they can safely be run in parallel. There's a minor complication with having to make sure the order of merging is maintained, and I had to move some methods from private to protected static to allow access from a private static inner class, but otherwise it wasn't too bad (I mostly just used CyclicBarrier).

Then I noticed it didn't work.

It worked most of the time, but sometimes one or more Inspectors would fail. I tracked this down to not being the Inspector's fault, but the object that was being inspected: some of the getters in my JSF application use lazy-initialization, and some of them use FacesContext while doing so. FacesContext is a non-inheritable ThreadLocal, so isn't available to those Inspectors running on a different Thread.

So you can't just say to MetaInspector 'here are some Inspectors' and 'here's how many Threads to run them across'. Still, you can say 'here are some Inspectors and here are the groups I want you to run them in' and leave it up to the developer to specify which ones can safely run concurrently.

That does work.

However now the inspector-config.xml looks a bit weird. You don't have to use it if you don't want to, but I'm not sure this feature 'carries its own weight' enough to be part of the core API. You have this confusing interface with setInspectors and setConcurrentInspectors, and it may not be clear which ones run when:

<metaInspector>
<inspectors>
...
</inspectors>
<concurrentInspectors>
<list>
...
</list>
<list>
...
</list>
</concurrentInspectors>
</metaInspector>

There are other interesting approaches too, such as just using concurrency for the merging whilst letting the Inspectors run on the main Thread.

Still, in the interests of K.I.S.S., I'm leaving it all out. It just smells too much like premature optimization. Keen developers can still implement their own ConcurrentMetaInspector if they need to.

Hook, Line and Sinker

I've been making a conscious effort to pepper the Metawidget base classes with lots of subclass 'hooks' - to the point of making more fine-grained methods and classes than I usually would.

This is because I anticipate Metawidget being extended in an unusually large number of ways, by an unusally diverse range of third-parties. They'll be those who want to keep all the components but rework the layout, those who want to keep the layout but change the components, those who want to keep both the layout and the components but alter the validation etc. etc.

Ask Your Father

Where possible, all Inspectors try and traverse their parent first to pick up any useful attributes such as @label. I tried having MetaInspector do this too.

It has the advantage that something like XmlInspector's metawidget-metadata.xml doesn't need to declare the whole path for cases like FooController/foo. It can just do...

<entity type="foo">

...rather than...

<entity type="FooController">
<property name="theFoo" type="foo">
</entity>

<entity type="foo"/>

The problem is all Inspectors must act the same: I can't remove the parent lookup from all the other Inspectors, because they may be used independent of MetaInspector. Therefore, we have MetaInspector doing the parent lookup and then all the child Inspectors doing 'parent parent' lookups.

Also, having found the parent, traversing from the parent to the child generically is horribly inefficient, as the best we can do is create a DOM and look for its child. We cannot, say, use JavaBean calls to adjust the objToInspect.

Both of these problems can be worked around (at the expense of increasing the complexity of the Inspector API) by introducing a separate inspectParent method, and this has some interesting advantages.

For example when using manually overriden widgets, we don't want to inspect children - we only want to grab some parent details. But now we are looking at an Inspector API with both an inspectParent and an inspect method, where the latter inspects both the parent and the child. I feel this overlap would be confusing for Inspector implementors.

We could try inspectParent and inspectChildren methods, but then the normal case of inspecting both parent and children needs double the calls into the Inspector layer, and would have to return ill-formed chunks of inspection-result.xsd.

On balance, I've decided to keep the Inspector API simple (eg. just inspect) at the expense of a slightly more complicated metawidget-metadata.xml and slightly less efficient manually-specified widgets, because both of the latter cases are used much less frequently than the first.

Death of a WidgetBuilder

Things consolidated a bit today.

Until now I've had a Metawidget interface with a MetawidgetImpl class. This acted as a go-between between instantiating platform-specific WidgetBuilders, and running and collating results from multiple Inspectors.

In turn, the platform-specific WidgetBuilders (SwingWidgetBuilder, FacesWidgetBuilder, etc) implemented a WidgetBuilder interface and extended a generic WidgetBuilderImpl class which took care of some of the generic 'plumbing' of iterating over inspection results.

Then we had another class that extended each target framework and made use of the WidgetBuilder. So you have:

• FacesMetawidgetComponent extends javax.faces.UIComponent (platform-specific base class)
• FacesWidgetBuilder extends WidgetBuilder (generic base class for iterating inspection results)
• MetawidgetImpl (collates inspection results)
• JavaBeanInspector implements Inspector

This worked, but over time I've ended up with such a close relationship between FacesWidgetBuilder and FacesMetawidgetComponent it no longer justifies the separation. So I've collapsed the two. It has the downside of some duplication between, say, FacesWidgetBuilder and SwingWidgetBuilder in how they iterate over inspection results - but I think I can live with that.

In addition, I collapsed the separation between Metawidget and the Inspectors into simply a MetaInspector that both implements Inspector and collates multiple results. This is really neat because it means we can efficiently support apps that only need a single Inspector (they don't use MetaInspector) and formalize a common XSD for inspection results - both from the 'bottom-level' Inspectors and the 'top-level' MetaInspector.

It means there's no longer a separate Metawidget class per se, but I think I can live with that too.

As a final bonus it allows for pluggable implementations of, say, remoting (RemoteInspector) and alternative merging algorithms.

Phew! A lot of refactoring for one day...

Abandon XPaths, All Ye Who Enter Here

I've been trying really hard to hang on to using XPaths as the way to tell the Inspectors what to inspect. It just feels 'right', given that XML is a natural choice for what to return from Inspectors.

However:

1. Whilst XPaths work quite well for object graphs, which are inherently nested, they are bad for targets like struts-config.xml which are not nested - and nesting them involves recombining lots of different potential combinations
2. Whilst the approach is cute, it is not true: the XML returned by an Inspector may bear no relation to what that Inspector inspects: in the case of struts-config.xml, the format may be quite different, and in the case of object graphs there is no XML
3. We actually don't want the expressiveness of XPaths. We want person/current, but we don't want WidgetBuiders being able to ask for, say, starts-with( @type, 'org.example' )
4. We would have to ship JXPath (no biggie)

Instead, I'm going to try simply using an array of names (eg. 'person', 'current'). This is easier all round to understand, and less prone to abuse.

Traverse Objects, not Classes

Another big design goal for Metawidget is that it do its work at runtime: I've long had a bee in my bonnett about those who espouse the benefits of static code generation. Yes, static code generation seems to get you started quickly, but in reality it generates volumes of code that is often of poor quality (very verbose, few comments, etc) and still has to be read and understood, documented, handed-over etc.

Runtime frameworks, on the other hand, generate zero code and whilst the frameworks themselves still need to be understood, there is usually lots of documentation how to do so. They also have the benefit of being able to work with the dynamic, runtime nature of a system, which is always richer than the static, compile-time information.

So for example a class Foo might declare a method getBar() which at compile-time is declared to return a Bar, but at runtime turns out (in some instances) to return an ExtendedBar. By examining runtime information, Metawidget can generate better, more accurate interfaces.

Still, this begs the question: if we are traversing objects in favour of classes, what happens if getBar() returns null? Should we stop traversing, or should we fall back to traversing the class heirarchy?

The latter approach is tempting, because it means we can display user interfaces for null objects: it seems a logical way to allow initializing new objects. However, in practice it's awkward because, given we already know the underlying data model is null, we have nowhere to send the data the user interface captures.

So for now I'm going with the first approach: traverse objects, not classes, and stop if we hit null. We'll see how it works out.

XML Is Your Friend

Though less efficient than returning a binary format, XML seems a natural choice for what Inspectors should return:

• it's technology-neutral, so any underlying platform can produce it, and any technology can be sensibly mapped to it (from object graphs to database schemas)
• it can be serialized across process boundaries (so the back-end can Inspect and send results to the front-end)
• it prevents accidentally keeping references to 'live' objects, and accidentally abusing those references

I have some uncertainty about whether to return the XML as a String or a DOM or a SAX Source. I'm going for a DOM for now, but we'll have to see how it shakes out.

WidgetBuilders Run The Show

One of the design goals for Metawidget is separating the 'gathering of metadata' from the 'application of metadata to construct user interfaces'. So far, I've settled on two broad abstractions: Inspectors and WidgetBuilders.

The Inspectors will implement some common interface and allow a uniform way to inspect a variety of back-ends (JavaBeans, XML configuration files, annotated POJOs, etc.) to look for metadata. The WidgetBuilders will be tailored to each platform (FacesWidgetBuilder, SwingWidgetBuilder, etc.) and construct a native UI. Metawidget itself will sit in the middle, co-ordinating the interaction between the Inspectors and the WidgetBuilders.

From there, something became apparent pretty quickly.

Originally, I was going to have the Inspectors mine as much as metadata as they could from the back-end, then pass the result to the WidgetBuilder for construction. This certainly has an elegance about it.

However, in practice the developer must be able to override certain portions of the finished interface, either to insert new components or hide existing ones. In the latter case, the developer may choose to hide a portion of the interface (intending, perhaps, to move it to a separate screen) that cuts out a whole branch of the Inspector's metadata.

If the Inspectors have already inspected that metadata, this could be very inefficient. Furthermore, unless the Inspectors are told where to stop, how do they know how far to go? Should they traverse every possible piece of metadata they find? How do they know what their WidgetBuilder will ultimately need? It may depend which UI screen the WidgetBuilder is serving?

It seems the best way to solve this is to flip things on their head: have the WidgetBuilders drive the Inspectors. The user will instruct the WidgetBuilder what aspect they want to render, the WidgetBuilder will see what's involved (minus those parts of the UI the user has opted to override), and will then proceed to ask the Inspectors (via Metawidget) for metadata. Based on what they get back, the WidgetBuilders may later ask for further metadata.

This 'feedback cycle' is perhaps a little less intuitive than the input/process/output approach, but I think works much neater in practice.