Definitions

ID

A text identifier that uniquely identifies a document. IDs are generated by adapters and, for new records created remotely, by clients. An id created by a client must contain sufficient information for the adapter to be able to create the corresponding document in the correct location.

Version

A text identifier for a particular version of a document. Versions are only ever compared for equality, they do not need to support any kind of ordering. The only requirement is that the version must change when a document changes. Typical implementations include

• an increasing counter
• a hash of the document
• a timestamp

Versions are always generated by adapters.

Sequence

An increasing counter maintained by the syncer to order modifications and identify changes that have occurred since a client last synced.

ClientNs

Client namespace, i.e., the database and collection where a document should be stored on the client. This is expressed in the usual MongoDB form db.collection where collection may itself contain embedded periods.

Pull

The pull stage of synchronization ‘pulls’ modified records from the adapter down to the client. The client begins this process by retrieving the list of modified ids from the syncer. For each id, the client verifies whether it already has the correct version of the document. If not, it adds the id to the list of ids that need to be pulled. Once it has collected sufficient ids, it requests the documents by making a POST request to the adapter at the endpoint

/lowla/pull

with a json body in the form

{
  "ids": [
    "<id>", "<id>", {...}
  ]
}

The adapter should generate a response of the form

[
  {
    "id": "<id>",
    "clientNs": "<clientNs>",
    "deleted": true|false
  },
  {
    // Document JSON
  },
  {...}
]

Notes

• The response consists of an array of javascript objects. The objects usually alternate between objects containing metadata and objects giving actual document contents. The exception to this is that when the metadata object has deleted: true, there is no following document and the next object, if any, will again be metadata.
• For performance or scalability reasons, the adapter need not return all of the requested documents. The client will re-request any ids that were not returned. This continues until the adapter returns an empty response (i.e., an array containing no objects.) At this point the client will move on to any remaining ids that need to be pulled. However, the client will be unable to update its sync sequence in this situation and it will request these documents again the next time it retrieves a list of modifications from the syncer.
• As long as pulls are successful, the client will update its sync sequence as it pulls documents. Thus if a sync terminates before the client can retrieve all the modifications provided by the syncer, the client is able to resume from where it left off on the next sync.

Push

The push stage of synchronization ‘pushes’ modified records from the client directly up to the adapter. The client collects the necessary change data and then makes a POST request to the adapter at the endpoint

/_lowla/push

with a request body of the form

{
  "documents": [
    {
      "_lowla" : {
        "id": "<id>",
        "version": "<version>",
        "deleted": true [optional]
      },
      "ops" : {
        "$set": {
          "fieldToModify": "newvalue",
          ...
        },
        "$unset": {
          "fieldToDelete": "dummyvalue",
          ...
        }
      }
    },
    {...}
  ]
}

The adapter should generate a response of the form

[
  {
    "id": "<id>",
    "version": "<version>"
    "clientNs": "<clientNs>",
    "deleted": true|false,
	"clientId": "<id that was pushed>"
  },
  {
    // Document JSON
  },
  {...}
]

Notes

• The response consists of an array of javascript objects. The objects usually alternate between objects containing metadata and objects giving actual document contents. The exception to this is that when the metadata object has deleted: true, there is no following document and the next object, if any, will again be metadata.
• The client may not (and usually will not) include all modifications in a single push request.
• For performance or scalability reasons, the adapter need not return responses for all of the pushed documents. The client will re-push any documents that were not returned. This continues until the adapter returns an empty response (i.e., an array containing no objects.) At this point the client will move on to any remaining ids that need to be pushed. However, the client will push these documents again the next time it performs a push sync.
• If the client is not able to generate suitable ids for new documents, the adapter may return a record using a different id than the value that was pushed. In this case, the original id will be returned in the metadata in the clientId property. If this property is set, the client should delete the original document under the old id and insert the returned document under the new id.
• The adapter must eventually provide a response for every document pushed. It can choose to reject a document by returning deleted: true in the metadata, but it cannot simply ignore a document as the client will continue to push it until it receives a response.
• It is the job of the adapter to handle conflicts when a document is edited in multiple places. It can detect conflicts by comparing the version on an incoming document with the current version in the back-end. For most applications a simple strategy suffices such as always accepting the incoming document or always rejecting the incoming document. In either case, the adapter must send the document that it ultimately stored in the back-end back to the client.