XForms - Engine Architecture

This document contains basic information about the architecture of the Orbeon Forms XForms engine (AKA XForms processor).

[STATUS: VERY PRELIMINARY AND INCOMPLETE]

See also: XForms - Core XForms Engine Improvements

The Orbeon Forms XForms engine at a glance

The XForms engine has the following characteristics:
  • Dynamically transforms (X)HTML+XForms documents on the server
  • It is currently designed to run within an XML pipeline (but technically doesn't have to be so)
  • Produces as a result (X)HTML documents that can be processed by regular browsers
  • The server-side layer
    • is in control of the XForms intelligence
      • state of the controls
      • state of the instances
      • model logic
      • event dispatch and flow
      • execution of actions
    • handles state information
      • static state (sharable between page loads)
      • dynamic state
  • The client-side (in the browser) is either

XForms engine lifecycle

The XForms engine's lifecycle is divided into two separate phases:

  • Page initialization
    • during which an XForms page is processed, transformed, and sent to the client
    • results in a new HTML page
  • Incoming actions AKA updates
    • during which an XForms document is updated following user actions on the client
    • most of the time results in an updated HTML page
      • DOM updates on the client in Ajax mode
      • full page update in Noscript mode
    • can also result in
      • client-side script execution
      • navigation to another page
      • display of alert message

Page initialization

During initial page load, XHTML+XForms content is transformed into:
  • Static state information
  • Dynamic state information
  • Initial XHTML/HTML output
    • HTML contains static + dynamic state information (usually reference to state information)
    • links to client-side JavaScript as needed
    • NOTE: engine does not dynamically generate JavaScript, only links to XForms engine JS files, and includes user scripts.

Incoming actions

Incoming events:
  • Client provides:
    • references to static + dynamic state information through a UUID
    • actions to perform on the server
      • Ajax mode:
        • provided most of the time through Ajax requests
        • sometimes (uploads, submission with replace="all") through form POSTs
      • Noscript mode
        • through form POSTs
  • XForms server:
    • identifies static + dynamic state to find or recreate XForms document
    • processes client actions
    • generates new dynamic state
  • Server sends back:
    • Ajax mode:
      • Ajax response
      • actions to perform on the client
    • Noscript mode:
      • new updated XHTML page containing static + dynamic state information (or reference)

Server processing of client events

The client can send multiple actions at a time to the server, in particular because there is a small delay to send some events (e.g. value changes), so that the client can pack them together and save overall latency.

The server processes each client-side event in turn:
  • Principle: whether the client sends 2 events at once, or in 2 Ajax requests, the result should be the same.
  • In Noscript mode
    • events are reordered so that "activation" events come last (client might send them in another order)
    • special handling of checkboxes blanking takes place
  • If there are contiguous value change events for a given controls, they are combined
  • The event target is obtained and checked
  • In Noscript mode
    • DOMActivate or value change events are created depending on the actual target object
  • Event objects are created, checking that targets exist 
  • Check that the event is allowed on its target
    • each target type has a list of events it accepts
    • in addition, explicit events can be specified by the form author
  • ClientEvents.processEvents()
    • Strict value changes check if the value is different. If it is the same, the event is ignored.
    • If the target is within a repeat, xxforms-repeat-focus is dispatched
    • For XFormsOutput, DOMFocusIn translates into:
      • DOMFocusIn
      • If the output is not readonly, followed by DOMActivate (not sure why)
    • A value change with focus change translates into:
      • Setting the external value into the instance
      • DOMFocusOut
      • DOMFocusIn
  • Between events, full deferred event handling takes place, including a refresh takes place to update the controls and their state.
    • If an event causes e.g. a control to become non-relevant, and a subsequent event tries to access that control, the second event must be rejected.
  • Since 2009-10-29, all events coming from the client are created ahead of time to point to the targets. Before dispatching, check that the event's target is still a valid object.
    • We want to handle controls that just "move" in a repeat. Scenario:
      • repeat with 2 iterations has xforms:input and xforms:trigger
      • assume repeat is sorted on input value
      • use changes value in input and clicks trigger
      • client sends 2 events to server
      • client processes value change and sets new value
      • refresh takes place and causes reordering of rows
      • client processes DOMActivate on trigger, which now has moved position, e.g. row 2 to row 1
      • DOMActivate is dispatched to proper control (i.e. same as input was on)
      • On the other hand, if the repeat iteration has disappeared, or was removed and recreated, the event is
        not dispatched.


Glossary

  • Static state: TODO
  • Dynamic state: TODO
  • TODO: term for "page generation" or "live XForms document" or ...

State information

Once a page has been requested, it starts its life:
  • Each new incoming action goes from one dynamic state to
    • the same dynamic state if "nothing" changed
    • a new dynamic state if "something" changed
  • State store
    • keeps static state
    • keeps initial dynamic state for
      • browser back diff compute
      • possibly: xforms:reset?

XForms engine initialization

Main phases triggered by XFormsToXHTML (oxf:xforms-to-xhtml processor):
  • NOTE: See graphics below as well.
  • Annotator (XFormsAnnotatorContentHandler == XFACH)
    • adds ids on all the XForms elements (xforms:*, xxforms:*, exforms:*, xbl:*) which don't have any
    • gathers namespace information on same elements to accelerate ns lookups later
    • finds AVTs on non-XForms elements
      • this only handles elements outside labels, etc.; those are handled separately atm
      • adds ids to those elements
      • produces xxforms:attribute elements
    • finds title information and produces xxforms:text elements
  • Extractor (XFormsExtractorContentHandler == XFECH)
    • extracts XForms content only
    • preserves contents of
      • inline XForms instances (xforms:instance)
      • inline XML Schemas (xs:schema)
      • xforms:label, xforms:hint, xforms:help, xforms:alert (as they may contain XHTML which is processed separately)
  • Store input XHTML into SAXStore
  • Static state (XFormsStaticState == XFSS)
    • initialization extracts:
      • top-level attribute information
      • controls document
      • models document
      • scripts
      • XBL components and bindings
      • full XHTML document if present (Noscript mode)
      • properties
      • static instances
        • this is information about read-only shared instances obtained after xforms-ready has been dispatched
        • we just keep the information about those instances, not the 
      • analysis
        • [?] check for mandatory single-node/node-set bindings on controls
        • gather static control information
        • gather repeat hierarchy information
        • gather static itemset information
        • extract event handlers
        • components
          • generate static shadow trees
            • full tree for HTML output
            • compact tree for internal processing
          • create factories for custom controls components (XFormsComponentControl)
          • gather static XBL scoping information (xxbl:scope="inner|outer")
        • XPath dependency analysis
  • ContainingDocument (XFormsContainingDocument == XFCD)
    • is passed XFSS
    • create XFormsControls
      • minimal initialization
    • create and index XFormsModel objects for top-level object
    • run models initialization
      • dispatch xforms-model-construct
        • TODO
      • dispatch xforms-model-construct-done
        • TODO
      • build controls tree
        • TODO
        • components
          • create container
          • add all container models
          • run models initialization for all container models (except there are no sub-trees of controls)
            • -> FIXME: xforms-ready / xxforms-ready should be dispatched after all is done
      • dispatch xforms-ready
        • TODO
      • dispatch xxforms-ready
        • add shared instance information to XFSS if not already present
  • Create state information associated with completed XForms initialization
    • get encoded static state from XFSS
    • get encoded dynamic state from XFCD
  • Rendering
    • register top-level element handlers
    • replay input XHTML from SAXStore through handlers

XForms Document Processing into the Static State Document
XBL Bound Element Processing into Shadow Trees
(red color shows changes planned)




Controls trees

XForms controls are organized in trees contained by XFormsControls.

NOTE: In the future, we are considering using subtrees, e.g. for repeat iterations and components.

XFormsControls contains:

  • One or two trees (ControlsTree):
    • initial tree
    • current tree
  • A reference to the XFCD
  • A reference to a XFormsContextStack (XFCS)
The reason for the existence of an initial vs. current tree is for purposes of comparison: upon responding to an Ajax request, the difference between the initial state (when the request came in) and the new state (at the time of the response) must be computed. This is done by comparing the two trees of controls in XFormsServer.

There are two ways the differences can be computed:
  • For "local" changes (switch case change, dialog visibility):
    • each control has its own "initial" and "current" version of the local data
  • For heavier changes (bindings, control values, repeat iterations)
    • we make sure the current control tree is cloned to the initial control tree before the changes are made
NOTE: "Local" changes are an optimization which adds complexity. We might want to find a better solution for avoiding duplication of the whole tree yet keep things simpler.

Building the tree

This is done only once per the life of XFormsControls.

The tree of controls contains:
  • every control, even non-relevant ones
  • except for repeat iterations:
    • if there is no iteration for a repeat, then none of the controls within xforms:repeat are present
    • those missing controls are added/removed dynamically depending on changes to repeat node-sets
NOTE: 2011-04 We plan to improve this and don't build non-relevant subtrees at all.

Updating the tree

Control bindings are updated if dirty:
  • at the beginning of a refresh
  • TODO

Controls and handlers

XForms controls are implemented as follows:

  • Each XForms control is represented by an object of a class derived from XFormsControl
    • e.g. XFormsInputControl
  • During rendering (first page load, or in the case of the Noscript mode every time the server sends by an HTML page)
    • a hierarchy of "handlers" (short for "XML element handler") is in charge of rendering each control to XHTML
    • handlers also render the XHTML head, body, handle AVTs on XHTML elements, etc.
    • the base class for XForms handlers is XFormsBaseHandler, which itself derives from a non-XForms related ElementHandler base class
    • e.g. XFormsInputHandler handles the rendering of XFormsInputControl
  • In addition, controls have a client-side counterpart in xforms.js and related JS files

Control lifecycle

A control (XFormsControl) follows this lifecycle:

  • creation (constructor)
  • setBindingContext()
    • set or update the control's binding
    • control relevance is determined at this time, and calls
      • onCreate
      • onDestroy
      • onBindingUpdate
    • must only be called once per control when
      • the tree/subtree is constructed
      • control bindings are updated during refresh
  • TODO: update following
    • saveBinding()
      • save the control's current binding-related information (relevance, MIPs)
      • for comparison purposes when dispatching UI events during refresh
    • saveValue()
      • save the control's current value
      • for comparison purposes when dispatching UI events during refresh
    • markDirty()
    • evaluate()
      • evaluate the control's value and any value-related properties that are needed at refresh time
    • destruction (garbage collection)

Evaluation time for values:

  • some values are needed for refresh events, so they are evaluated at refresh time
  • some aspects of the control, like LHHA are evaluated lazily because they are usually not needed at refresh time, only when sending changes to the client

Element identifiers (ids)

Basics:
  • Controls have a "static id", which is the id available in the source XHTML+XForms document.
  • Controls also have an "effective id", which is an id scoped depending on repeats and components
    • repeats: a suffix is added, e.g. my-control·1-2 -> my-control within 2nd iteration of 1st iteration of repeats
    • components: a prefix is added, e.g. my-component$my-control -> my-control within the my-component instance of a component
    • prefix and suffixes can be combined, e.g. my-component$my-control·1-2
  • Controls have a prefixed id, which is the effective id without the suffix, e.g. my-component$my-control
  • If no id is present on a control, then an id is automatically added. This is necessary for:
    • rendering: controls must be matched against the original XHTML+XForms document
    • HTML DOM in the browser: Ajax communication must match the client representation and the server representation
  • Models and instances also must have ids for internal use. If no id is present, then an id is automatically added.


Two-pass submission

A two-pass submission is a submission that isn't executed right away by the server as it receives an event from the client, but instead where the server asks the client to do a POST to the server, upon which the submission happens. This happens with:
  • Replace="all" submissions. One major reason for this is this double requirement:
    • XForms requires checking first that the submission can take place. E.g. the XML document to submit might not be valid, in which case the submission is stopped with xforms-submit-error. In this scenario, the web page must stay as it was before, therefore this check must be done through an Ajax request, not an HTML form submission.
    • The most natural way, from the web browser's perspective, to then load a new page is to do an HTTP POST through an HTML form submission. 
(Other possibilities can be contemplated, such as receiving the HTML in the Ajax response and loading the resulting HTML in the current page. Each possibility has pros and cons.)

For instance, in the PFC wizard example, each step if the wizard is implemented with a separate page, and you go from one page to the next with an xforms:submission replace="all". When the client tells the server that the user pressed the next button, the server responds with:

<xxf:event-response xmlns:xxf="http://orbeon.org/oxf/xml/xforms">
    <xxf:dynamic-state>...</xxf:dynamic-state>
    <xxf:action>
        <xxf:control-values/>
        <xxf:server-events>...</xxf:server-events>
        <xxf:submission method="POST" replace="all"/>
    </xxf:action>
</xxf:event-response>

Upon receiving this response, the client will do a form POST to the server, passing the value in the xxf:server-events element received in the above Ajax response to the server in the $server-events parameter.
  • Replace="none | instance" submissions that require a file upload. In that case, the server responds with <xxf:submission method="POST" replace="none"/> (instead of replace="all") to indicate that the client should run a "background" POST.


Asynchronous and delayed submission

Two types of submissions use the xxf:server-events action in an Ajax response:
  • Asynchronous submissions (<xforms:submission mode="asynchronous">). When executing an asynchronous submission, the server doesn't wait for the result fo the submission before sending a response to the client, but tells the client that it needs to "poll" the server after a certain delay:
<xxf:server-events delay="9964" discardable="true" show-progress="false">...</xxf:server-events>

After receiving this action from the server, the client sends the server events in an Ajax request only if it didn't send another Ajax request before the specified delay, hence the discardable="true". The server always responds with show-progress="false" when the xxf:server-events is related to an asynchronous submission. This means that the client won't show a loading indicator when those "polling" Ajax requests are sent to the server, which seems like a reasonable default.
  • Delayed events (<xforms:dispatch delay="1000">). The value of the show-progress attribute in the action below is controlled by the xxforms:show-progress="true|false" attribute on xforms:dispatch.
<xxf:server-events delay="983" show-progress="true">...</xxf:server-events>


Comments