Projects‎ > ‎XForms‎ > ‎

XForms State Improvements

Status

This project is completed as of 2010-06-01.

Rationale

As of May 2010, the XForms dynamic state is serialized:
  • each time an XForms page is requested
  • each time an Ajax/noscript response is produced
This was done this way for historical reasons, and it is not optimal in theory because the XForms document is usually in cache and serializing state at every request is wasteful.

The goal of this project is to get rid of unneeded state serializations to gain significant performance.

Initial impact analysis (2010-05-20)

Issues this project addresses

  • conceptual simplification of XForms state management
  • code simplification
  • significant performance improvements especially for forms with large instances
  • solves current issue of server errors (due to missing state) upon long-running 2nd passes of submission with replace="all"
  • will also help solving issue of background uploads
  • makes it much easier to implement <xf:reset>

Change to the order in which cache/store processing occurs [DONE]

Currently, with server state handling:
  • (static state UUID, dynamic state UUID) sent by the client are used to hit XFormsStateStore first
  • the state store returns the serialized state, which can be very large
  • the serialized state is used to index XFormsDocumentCache
This requires that the serialized state be available in all cases, as it is used as a cache key.

What needs to happen instead:
  • (static state UUID, dynamic state UUID) sent by the client are used to hit XFormsDocumentCache first
  • if the document is not found in cache, then XFormsStateStore is used
  • when a document is expired from XFormsDocumentCache, it is serialized to XFormsStateStore

Change how state is identified [DONE]

Currently, there is a mechanism which keeps the last two dynamic states of the document in store in addition to the initial dynamic state. States are kept in serialized form.

So since we want to keep extra serializations, we cannot keep the last two states!

NOTE: The historical rationale for keeping the last two dynamic states was to address scenarios such as the following:
  • user clicks button, dynamic state sent is d1
  • server performs actions, and triggers two-pass submission, returns state d2
  • client starts HTML form upload with state d2
  • assume upload is very slow, e.g. because it includes uploads
  • user keeps interacting with the page, causing dynamic states d3, d4, etc. to be created
  • when HTML form upload reaches server, server must be aware of state d2
  • for this reason, more than one state must be kept (we kept 2, but even that is not right!)
(A more elaborate algorithm was devised, but never implemented. That algorithm is in fact not needed with the much better and simpler solution described below.)

The solution consists in, instead of sending the client (static state UUID, dynamic state UUID), a single document UUID identifying the interaction between the user and the document.

NOTE: The document UUID is already created by the XForms engine.

This requires the following:
  • initial HTML stores document UUID instead of (static state UUID, dynamic state UUID)
  • XFormsDocumentCache is indexed by document UUID
  • store must also keep track of the (document UUID) -> (static state UUID, dynamic state UUID)

Change to submission [DONE]

With the older "last two dynamic states" strategy, if the second pass of a submission reached the server and the document had already moved on to a new state, there was no conflict because the document would be recreated from serialized state exactly as it was after the first pass.

In the example above, even if the user interacted with the form and caused the document to be in state d4, the submission would perform the second pass in state d2. Consistency was ensured.

With the newer solution, there are cases where this will not work as expected. Consider the following:
  • browser initiates second pass
  • user causes Ajax request which clears instance to submit
  • slow second pass reaches document
  • second pass tries to use modified data
Note that similar inconsistencies were already possible in the past, e.g. with:
  • <xf:send .../>
  • <xf:delete .../>
But that was something the form author could control. With the new method, the user might cause unneeded actions.

One initial simple solution is similar to the way we handle asynchronous submissions:
  • as of xforms-serialize completion, the callable for the submission must be stored
  • the request is picked up when the 2nd phase reaches the document
  • we must be careful about where to store the request, as there is a chance the 2nd pass will never reach the server!
NOTE: This solution doesn't work because the 2nd phase might send new uploaded files. So for now we have to accept that between the time a submission with replace="all" is sent and its completion, data might change.

In addition, there is a synchronization issue to address in XFormsServer:
  • old method
    • if the user was in state d3 but submission state d2, a new document was created just for the purpose of submission
    • therefore Ajax interactions and the 2nd pass would not block each other
  • new method
    • 2nd pass reaches the same document, therefore is synchronized with Ajax requests
    • if the replace="all" is slow (e.g. calling report service, slow external site, etc.), user actions are blocked until the submission completes
  • solution
    • completion of the 2nd pass must occur outside of the synchronization block
    • the submission stores a callable in the document
    • after completion of the synchronization block, the callable is run

Change to client state handling [DONE]

We do not think that client state handling is very useful at this point, but the above changes unfortunately require some work to make it work again.

Solution:
  • HTML must contain document UUID + serialized static and dynamic state
  • document UUID is sent to server along with serialized static + dynamic state
  • document cache can then be checked first, and store used if not in cache
  • server returns new serialized dynamic state
NOTE: In theory you can imagine optimizations, such as not sending the static state unless the document is not found in cache.


Initial implementation notes

  • branch: xforms-state-improvements
  • XFormsDocumentCache
    • use of pool is no longer needed
    • cache is greatly simplified
  • MemoryCache needs ability to call back when entry is expired
    • old Cacheable -> CacheableInputOutput
    • new Cacheable interface created
    • if cache object instanceof Cacheable, then cache calls back an expired() method
    • this has been merged to the master branch as of 2010-05-20
  • XFormsStateManager
    • lifecycle methods
    • uses XFormsDocumentCache to find/store document
    • uses XFormsStateStore to obtain state to recreate a document
    • when session expires, docs must leave document cache too
      • throw new OXFException("Got xxforms-all-events-required event without initial dynamic state.");
  • XFormsContainingDocument
    • upon expire() use uses XFormsStateStore to serialize state
    • don't use initial/current dynamic state UUIDs, instead just keep a change sequence number
  • XFormsStateStore
    • mapping document UUID -> static state UUID
    • mapping document UUID + "-C" -> current state
    • remove pinned, previousKey, and "last 2 entries" stuff
    • previousKey no longer needed
    • addOrReplaceOne() doesn't assume the value is constant
    • created lifecycle interface
    • initial state stored into store as UUID + "-I"
  • XFormsServer
    • some refactoring
    • create new XFCD with initial state for all events (browser back)
  • HTML stores single document UUID
    • modify XHTMLBodyHandler
    • check JS code and make changes there

Comments