Bulletproof Backups with the ReadyNAS Duo (Update)

I thought I'd do an update to one of my most popular blog posts: Bulletproof Backups with the ReadyNAS Duo:

I've had my ReadyNAS Duo nearly four years now, and have experimented with all sorts of things and reached some conclusions:

RSnapshot

I struggled for a couple years with versioned backups. Because you have quite a lot of disk space, you can get away with making multiple, complete copies of your data for daily and weekly backups. In fact, using the built-in ReadyNAS software it's the best you can do.

But if you've got thousands of files and most of them don't change often, this feels very wasteful. A while ago I discovered the rsnapshot add-on. rsnapshot only copies files that have changed, and creates hard links for those that haven't. So suddenly you can have many days/weeks worth of versioned backups in very little disk space. Awesome!

However:

Hard Links Are Evil

A warning about hard links. If your ReadyNAS ever decides it needs to check the disk (fsck), all those thousands of rsnapshot hard links will bring it to its knees. This has only happened to me once, thankfully!

Your options in this case seem to be either: kill fsck, delete all your rsnaphot backups (and hence the hard links) and run fsck again. Or wait out the mysterious blue pulsing light:

Mysterious Blue Pulsing Light

Sometimes the ReadyNAS just sits there ominously with a blue pulsing power light. You can't connect to it during these times. Worse, this mystery state can last for hours! If you're like me you installed RAIDar once when you first bought your ReadyNAS, and then forgot all about it. However RAIDar is very useful for telling you what your NAS is doing during these pulsing light times, and stopping you resetting it in impatience.

Root That Sucker

I resisted the EnableRootSSH and ToggleSSH add-ons for years, because if you use them and subsequently mess something up, you probably can't ask for support. But if you're careful they can be very useful.

For example, when configuring rsnapshot, I found it necessary to manually edit /etc/cron.d/rsnapshot and set it to:

0 9,18 * * * root /c/addons-config/rsnapshot/hourly.sh
0 14 * * * root /c/addons-config/rsnapshot/daily.sh
0 16 * * 1 root /c/addons-config/rsnapshot/weekly.sh

This is because by default rsnapshot runs at midnight, but I have my ReadyNAS configured to power down overnight.

Bringing The Horse To Water

Although there's lots of rsnapshot goodness once your data is on the ReadyNAS, actually getting it there can be troublesome. I went through a lot of different backup programs. Some of them are truly awful. I won't name and shame them, but here are some problems to watch out for:

• do they have a special driver that syncs the data? These frequently BSOD'ed my PC
• do they mess up the folder case when they sync, especially if you have two folders with different case at different places in your heirarchy (e.g. C:\foo\Finances versus C:\foo\bar\finances)
• do they have catalogs/indexes/other mechanisms that make them horribly slow to copy thousands of files
• can they do file filtering based on wildcard matching
• can they exclude folders based on wildcard matching, especially nested folders (e.g. *\tmp\)

The absolute best I found was SyncBackSE. It does it all, and does it fast, and does it without any nasty surprises!

Teaching The Horse To Drink

Even with a great product like SyncBackSE, you still need to take care how you schedule your backups. If you've got thousands of files, even if they don't change often, your backup software still has to scan over them every time.

I settled on configuring half a dozen backup jobs. Critical files were scanned more often (say, hourly) but in small groups (say, just My Documents). Less critical files were scanned less often (say, daily) in larger groups (say, C:\Program Files).

Exactly What It Says On The Tin

The ReadyNAS is a great Network Attached Storage device (e.g. hard drives in a box), but it's pretty lousy for anything else. It's really underpowered if you try to use it as a server, or load it up with too many add-ons or streaming services. If that's what you're after, you really need more than 256MB RAM and a slow CPU. Apparently the Duo v2 is faster, but it's still only 256MB RAM.

Your Mileage May Vary

Some things I got right the first time:

• it's definitely too slow to use as an 'active' drive that you run programs off.
• keeping all paths relative to some distant drive letter (like N:) is handy for restores.
• despite some misgivings, I've had no problems at all powering down the ReadyNAS and swapping drive trays as a way to take off-site backups. However, remember that the drives are EXT3 formatted. This makes them a pain to try and read in Windows, and the ReadyNAS doesn't do NTFS. So it can still take a while to restore all that data into a usable form.

But all in all, the ReadyNAS Duo is a complete win!

Metawidget v1.35: New Static Form Generation

Version 1.35 of Metawidget, with new static form generation is now available! This release includes the following enhancements:

• First version of StaticMetawidget
• Support JPA @Temporal
• Output HTML <label> tags around labels
• Refactor StrutsWidgetBuilder/SpringWidgetBuilder into WidgetProcessors
• Bug fixes, documentation and unit tests

As always, the best place to start is the Reference Documentation:

http://metawidget.org/doc/reference/en/pdf/metawidget.pdf


Your continued feedback is invaluable to us. Please download it and let us know what you think.

Forge and Metawidget Discussed On JBoss Community Asylum

Metawidget, and the work we're doing with the JBoss Forge team, has gotten a nice mention on the latest episode of the JBoss Community Asylum podcast. Here's an excerpt (taken from 35m.22s):

Max: "Right now if you download Forge right now, you get a JSF Metawidget thing"

Lincoln: "Right you get a default scaffolding provider we call them, and that one right now is using - like you said - JSF and Metawidget which we're actually working on the next version of, so that... right now when you run the scaffold it basically looks at your database objects, your JPA objects, it runs through them all and generates a bunch of view files for the Create, Read, Update, Delete operations. You know, a simple interface."

Max: "So this is kind of like what seam-gen did before, right?"

Lincoln: "Right. Kind of like that. And when you look at the output of the scaffold that you would download, maybe from Beta 3, you will see this Metawidget tag and Metawidget is..."

Max: "Yeah why don't you just tell what Metawidget is?"

Lincoln: "Metawidget is a really cool framework actually, it..."

Emmanuel: "We should interview... er... forgot his name..."

Max: "Richard? Richard something? Kennard"

Lincoln: "Richard Kennard"

Max: "Yeah Kennard"

Lincoln: "...it's a really cool framework for, basically you just point this tag at an Object or a bean and it runs through that bean and looks at all the properties and builds up an interface that shows up on the page."

Max: "Yeah, and in your JSF you just have one [line] and say this is the bean I want to do and it builds the UI. And it does it for... er... Swing, and..."

Lincoln: "You can do Swing, you can do GWT, you can do JavaFX I think. Lots of stuff."

Max: "So actually it is pretty powerful, but it's not..."

Lincoln: "The problem is then you've got this tag in there and it's... if you want to customize that then you're learning the Metawidget framework. And so what we are doing right now actually is we're customizing Metawidget so that, using the same inspection process that Metawidget provides, generate real code. Materializes it into real XML, real JSF pages. Then you can go in and modify those pages just like you would have done by yourself."

Listen to the full podcast here.

You Can't Spell Forge Without Metawidget

JBoss have just released Beta 4 of JBoss Forge. There are a host of new features in this release. Most exciting for me being the new UI scaffolding.

Scaffolding has been significantly rewritten to use a new static Metawidget. This allows Forge to output pure JSF tags, with no runtime dependencies on any non-Java EE libraries. The UI includes creating, updating, deleting, pagination and searching. It also supports one-to-many, many-to-many, many-to-one and one-to-one relationships:The code it generates is very clean:

<h:panelGrid columnClasses="label,component,required" columns="3">
   <h:outputLabel for="customerBeanCustomerFirstName" value="First Name:"/>
   <h:panelGroup>
      <h:inputText id="customerBeanCustomerFirstName" value="#{customerBean.customer.firstName}"/>
      <h:message for="customerBeanCustomerFirstName" styleClass="error"/>
   </h:panelGroup>
   ...

Behind the scenes, Forge is still using the same Metawidget pipeline as before - the same Inspectors, WidgetBuilders, WidgetProcesors and Layouts. This means the scaffolding is pluggable to adapt to your needs - including custom UI libraries, custom layouts, even custom langauges (Ceylon, anyone?).

The new static Metawidget is also used to generate Java code for the JSF backing beans. Again using WidgetBuilders and outputting very clean code:

String firstName = this.search.getFirstName();
if (firstName != null && !"".equals(firstName)) {
   predicatesList.add(builder.like(root.<String>get("firstName"), '%' + firstName + '%'));
}
String lastName = this.search.getLastName();
if (lastName != null && !"".equals(lastName)) {
   predicatesList.add(builder.like(root.<String>get("lastName"), '%' + lastName + '%'));
}

Please give it a try!

• Download Forge Beta4


• Install it


• Run it


• Copy and execute this command:
  
  $ run-url https://raw.github.com/forge/core/master/showcase/posale.fsh

For the lazy, I've uploaded the project that Forge generates. You can download it here.

Pleasing Dick Wall

In Episode 371 of the Java Posse, Dick Wall has this to say about D.R.Y. (at 47m.30s):

"That actually, I hate to say, but it's an affliction that's very very bad in the Java world. And you see developers drop it as soon as they see another programming environment and really grok the idea of Don't Repeat Yourself. A lot of the flagship libraries in Java are really bad for this.

Wicket is one that springs to mind... everybody loves Wicket, [but] now I've been exposed to a bunch of different stuff I look at it and you have to repeat things 3, 4 or 5 times sometimes. When you're adding a field on to a Web page you've gotta name it on the page and then you come into the code and you write your handle on it with the exact same kind of component that is, and the name, and even worse, you know, taking the current Java mindset of 'you should be working for the compiler' and not the other way around.

If you get it wrong, it complains at you. It doesn't try and fix it for you."

This is, of course, exactly the problem that Metawidget is designed to address. I talk about it in my JavaOne video here (at 5m.30s).

Unfortunately work has stalled on the Wicket version while I've been working on JBoss Forge integration. Wicket actually makes completely dynamic controls, like Metawidget, tricky because of its approach of 'a Java file and a static HTML file'.

However, I was fortunate to have lunch with Wicket guru Juliano Viana at JavaOne 2011. He deep dived into the Wicket internals and found MarkupContainer.getAssociatedMarkupStream for me. It looks like this can free up the requirement of a static HTML file. So hopefully there is a way to get around this Wicket limitation.

Metawidget @ JavaOne 2011: JBoss Booth Video

JBoss have posted videos from sessions at their mini-theater in the JavaOne Exhibition Hall. This includes our talk on DRY UIs: Let the metadata do the work:



The video is available in High Definition, so please try the Full Screen mode if you find the coding examples difficult to read. A very similar talk, given as a JavaOne conference session, is available on Parleys.

Should HTML.next Have A Higher Level of Abstraction?

HTML brings a lot of great features to the table. One of them is device independence. The HTML author gets to write:

<button value="OK"/>

And the device gets to choose how to render that using a native widget:

I think this is a great level of abstraction. But in my opinion the W3C made a mistake when they specified:

<input type="checkbox"/>

See that type attribute? It's referring to the widget type. But it really should be referring to the data type. Then we could write:

<input type="boolean"/>
<input type="date"/>
<input type="integer"/>
<input type="color"/>

And the device gets to choose how to render that. If it were a simple device, or it didn't understand the data type, it could just render a text box. But better implementations could offer nicer UIs. Date pickers. Spinners. Color pickers. Browsers could compete by offering better widgets.

Think how much time our industry has wasted re-implementing HTML date pickers and spinners. Hundreds of widget libraries across all development platforms (PHP libraries, Java libraries, .NET libraries, etc).

I came to this realisation while using Metawidget. Typically you tell Metawidget to generate an entire form, composed of many widgets:

<m:metawidget value="#{myForm}"/> ...which gives...

But I've found it suprisingly useful to still use Metawidget even when dealing with single widgets:

<m:metawidget value="#{myForm.name}"/>
<m:metawidget value="#{myForm.age}"/>
<m:metawidget value="#{myForm.dateOfBirth}"/>

Here we are deferring the actual widget choice to Metawidget. So Metawidget can choose it for us based on the best widgets available on the target device, our locale, our accessibility requirements etc. Maybe that's a plain text box. Or maybe it's a RichFaces date picker. Or an ExtGWT spinner.

It's nice to defer that choice, because now all the widgets can automatically upgrade over time as the platform improves. If this were in HTML too, it'd save an enormous amount of re-implementing. And it would provide a new battleground for browser competition.

Better for developers. Better for browser vendors. Better for consumers.