Orbeon Search

Comments? Feedback?

This wiki does not yet support public comments (a limitation of Google Sites), so we encourage you to post your comments either:

On Twitter by responding to @orbeon.

On our community mailing list: subscribe sending an email to ops-users-subscribe@ow2.org (content of subject/body doesn't matter), you'll get a response with the email to use to send your message to the community mailing list.

Recent site activity

How Orbeon Forms Works, Part I

This is a post for techies, trying to shed light on little-known aspects of Orbeon Forms.

Why does Orbeon Forms require a server runtime?

We often get asked if, after a form is created with Orbeon Forms, it can be deployed without a server-side runtime. The answer, for the moment, is no, and here is why.

In 2005, when we started implementing the XForms engine which currently ships with Orbeon Forms, Ajax had just been named as such. We thought hard about whether we should write a full client-side XForms engine in JavaScript (like more recent projects such as Ubiquity XForms and XSLTForms), or pick a hybrid approach where some code would run on the client, and some on the server.

At the time, we decided that writing a full engine in pure JavaScript would be hard, and that we would be very late to the market if we went that way. The reason was that Java was simply way more productive for us than JavaScript, and we think this is still the case (we have now started using Scala on the server as well). That has proven a good choice since our XForms engine has been mature for a long time now, and that (unfortunately, in our opinion), plain JavaScript implementations are just getting there as we write.

So let's see now the hybrid approach work.

The hybrid approach

Once you have decided on a hybrid approach, how do you implement it? It depends how much you run on the server. In Orbeon Forms, the answer is: a lot! The server is smart, and the client is rather dumb (although that balance might progressively change in the future).

So the tree of user interface controls, XForms models and instances, all that resides on the server. This means the following:

  • When an XForms page is requested, the server must produce the initial HTML markup that will run in the browser.
  • The server must keep state information, including some XForms controls state (like repeat indexes, selected switch cases, visible dialogs) and XML instance data.
  • After the page has loaded, the client must send updates to the server, the server then must update its state in response, and tell the client about what needs to change.

We cover these three points below.

Initialization

When you request an XForms page, Orbeon Forms:

  • Passes the source of the page (HTML and XForms) to the XForms engine.
  • Analyzes the XForms page, unless already done (it might be cached already).
  • Creates XForms models and the tree of user interface controls.
  • Runs XForms initialization.
  • Produces HTML output matching the state of the XForms controls just after initialization.

The resulting HTML also contains links to CSS and JavaScript needed to properly display the page.

Typically, requests for a given XForms page produce different HTML depending on what happens during XForms initialization. For example, the page might fetch data from a database, a list is latest news, etc.

NOTE: It could have been possible to always produce the same HTML, and then, with a subsequent roundtrip to the server, or by loading say some JSON data structure embedded in the page, initialize the page with its initial data on the client. This would have some drawbacks:

    • Latency would likely increase if another Ajax request is needed.
    • The page might appear to flicker, as it would be in an intermediate state until JavaScript can run to fully initialize the page.
    • This wouldn't work at all with clients that do not support script.

In most cases, producing the initial HTML on the server is the best solution.

Note that there are some exceptions to this rule. Some UI components, like trees, menus, dialogs, the rich test editor, etc. do need client-side initialization, simply because they are implemented by JavaScript libraries that do not have a server-side component at all.

State

Before a page is fully sent to the browser, Orbeon Forms makes sure that instance values and controls are kept "somewhere". This works through a mechanism of caches and stores:

  • First, Java/Scala data structures representing the running page are kept around as much as possible.
  • Second, when these data structures take too much place, they are serialized out to a memory store.
  • Finally, when that store becomes too full, the state is serialized out on disk.

This ensures that pages with which users are currently interacting remain responsive.

UI updates

As the user interacts with an XForms page through activation of buttons, tabbing through fields, etc., the client-side JavaScript runtime collects information, and after a little while sends it to the server, usually through an Ajax request (but it can also be an HTML form post in cases like file upload).

The server then:

  • goes through the list of changes (like "user clicked on the Save button")
  • stores data into instances and dispatches appropriate events to the tree of control
  • as needed runs the XForms rebuild, recalculate, revalidate, and refresh operations

Once all that has happened, the server has two things to contemplate:

  • a "before" state, or the XForms instances and controls before the user acted on it
  • an "after" state, or the XForms instances and controls after the user acted on it and all the events and actions that might have run in consequence have been processed

The server then compares the two states sends the differences to the client. The client updates the HTML DOM appropriately, and voilà!

Now it might sound easy, but it's not quite so:

  • You have to properly compute the difference between two trees.
  • The server might end up having huge amounts of updates, like when lots of new repeat iterations suddenly appear because data was queried from a database.
  • The client must interpret these changes and apply them efficiently to the HTML DOM, without erasing information the user might have created in the period of time the Ajax request was processed.

Orbeon Forms is starting to implement heuristics to minimize markup in page and Ajax response sizes, for example sending incremental updates when possible, and full HTML updates when too many changes have occurred.

NOTE: Orbeon Forms also has a variation on this with the "noscript" mode, which doesn't use JavaScript or Ajax at all, but uses full client-server roundtrips and always produces HTML. This is a topic for a later post!