Up - API Reference - Home
When you have a client connected to the server, such as an app or an api connection, you cache the incoming data. You want to make sure this data is up-to-date, even when other users or processes are modifying that data at the same time. To accomplish this, we provide the sync API.
The sync API is a real-time API based on
socket.io. We provide examples using the python librarypython-socketio.
Syncing happens two ways; an initial sync to get you up-to-date, then a continuous flow of sync-events.
For the initial sync we make use of a cursor; an ever-increasing number attached to each update. You remember your current cursor (provided when you logged in), call the initialization, then handle the resulting list of changes. Afterwards, you update your cursor so a subsequent request only fetches new changes (the subsequent request is only needed when the connection was lost).
For the sync-events you also get a list of changes, but this will usually just be one update.
Finally, we also require the JWT you use for the /api calls, and it may get updated in the sync process.
Now there are two types of data - project-data, and global unicat data; for example, 'assets' and 'cc.users', respectively. These are synced independently, so we need two cursors.
The sync returns a list of updates - you must filter these based on the ones you are interested in; then call a /api method to actually retrieve the updated data.
This same list of updates can be manually retrieved by calling /sync/get as a regular POST request. We can also fetch the latest cc and current cursors through a POST request to /sync/cursors.
For the upcoming examples, we use the async version of python-socketio.
You connect to /sync/io:
import socketio
sio = socketio.AsyncClient()
sio.jwt = <jwt-token>
def get_auth():
return {"JWT": sio.jwt}
await sio.connect(
"https://unicat.app",
socketio_path="/sync/io",
auth=get_auth,
)
There are three events that must be handled: connect, authorization, and sync.
connect is called whenever a successful connection is established, this is the time and place to start your initial sync which also subscribes you to sync updates. This is done by emitting an initsync message to the server. This returns a list of sync-updates.
authorization is called when the JWT changes.
sync is called when data changes serverside. The data given is a list of sync-updates.
cc_cursor = 0
project_gid = "<project-gid>"
project_cursor = 0
@sio.on("connect")
async def on_sync_connect():
await sio.emit(
"initsync",
{
"cc.cursor": cc_cursor,
"project": project_gid,
"cursor": project_cursor,
},
callback=handle_sync_data,
)
@sio.on("authorization")
async def on_sync_authorization(data):
sio.jwt = data["JWT"]
@sio.on("sync")
async def on_sync_message(syncdata):
await handle_sync_data(syncdata)
async def handle_sync_data(syncdata=None):
if not syncdata: # nothing to sync
return
for item in syncdata:
# TODO: handle event, update cursor
print(item)
The server supports just one call, initsync (besides connect and disconnect, obviously).
await sio.emit(
"initsync",
{
"cc.cursor": cc_cursor,
"project": project_gid,
"cursor": project_cursor,
},
callback=handle_sync_data,
)
cc.cursor a cursor for global unicat data.
These are optional, but if you provide one, you must also provide the other.
project a project gid. You can only sync for one project at a time.
cursor a cursor for the current project data.
You must provide a callback function to handle the data returned from the server. This function should also update the cursors.
This function can also be used to handle incoming data on sync events.
Sync data is a list of objects that must be handled in-order. You only need to handle the updates that are relevant to you.
Each object has the following form:
{
"type": "<type>",
"cursor": <cursor>,
"action": "UPDATE",
"data_type": "<data type>",
"data_key": "<data key>",
"data": <json data or null>,
}
type is either cc or the current project gid.
cursor the cursor-value for this update. If you handled (or skipped) this update, update your local cursor to this value.
action is one of INSERT, UPDATE, or DELETE. Use these to update your cached data accordingly.
data_type is the data type, and the possible values are different for the cc and project types, but they match the data types returned from the /api. In addition, we also get sync data from jobs.
data_key a unique key for the data type, usually but not always the gid.
data associated data, usually null, except for data type jobs, then it looks like:
{
"type": "cc",
"cursor": <cursor>,
"action": "UPDATE",
"data_type": "jobs",
"data_key": "<job gid>",
"data": {
"project_gid": "<project_gid>",
"job": "backup_project",
"status": "processing",
"info": {},
"created_on": 1613565541.4195538,
"updated_on": 1613565543.2836412,
},
}
cc updatescc.version changes when we push a server-side update. The key is the new version.
cc.users key <gid>.
cc.projects key <gid>.
cc.projects_members key <project-gid>/<user-gid>. The data from the api is returned as a list, so it doesn't really have a key.
jobs key <gid>. The associated data provides the job name, status, and extra info.
records key <gid>.
definitions key <gid>.
classes key <gid>.
fields key <gid>.
layouts key <gid>.
assets key <gid>.
queries key <gid>.
ccapi = UnicatApi(…, …)
async def handle_sync_data(syncdata=None):
if syncdata is None:
print("-- sync error - auth?")
return
if not syncdata:
print("-- nothing to sync")
return
for item in syncdata:
await handle_sync_dataitem(item)
async def handle_sync_dataitem(item):
if item["data_type"] == "jobs":
pass
if item["data_type"] == "cc.version":
ccapi.cc_version = item["data_key"]
…
# we check if this item interests us, by looking in the ccapi cache
elif item["data_key"] in ccapi.data[item["data_type"]]:
if item["action"] == "DELETE":
del(ccapi.data[item["data_type"]][item["data_key"]])
elif item["action"] == "UPDATE":
if item["data_type"] == "assets":
# fetching the asset auto-updates the ccapi cache
success, result = await ccapi.call("/assets/get", {"asset": item["data_key"]})
…
# always update the local cursors
if item["type"] == "cc":
ccapi.cc_cursor = item["cursor"]
else:
ccapi.project_cursor = item["cursor"]
Note that we make an api-call for each updated item now; that is not very efficient. You should collect all relevant gids from the complete syncdata, then fetch them in bulk (a single call per data type).
Requires JWT.
Get sync data manually.
This may trigger a warning when there are too many sync updates to quickly retrieve. In this case it is better to re-initialize, so you are automatically in sync and have the latest cursors.
If you use /sync/get to keep your own CMS in sync with Unicat, you can ignore that warning, handle the sync entries, then do another /sync/get request with the latest cursor.
POST /sync/get
Authorization: Bearer <JWT>
{
"cc_cursor": <cc_cursor>,
"project": "<project gid>",
"cursor": <cursor>,
}
These parameters are all optional, but at least one must be provided. If you provide either of project or cursor, you must also provide the other.
cc_cursor last known cursor for global unicat data.
project gid for the project you are syncing.
cursor last known cursor for project-data.
Authorization: <JWT>
{
"success": true,
"result": {
"sync": [
{
"type": "<type>",
"cursor": <cursor>,
"action": "UPDATE",
"data_type": "<data type>",
"data_key": "<data key>",
"data": <json data or null>,
},
…
],
"warning": "Sync limit reached - better to reinitialize."
},
"data": {}
}
sync is the list of sync updates. See the 'Sync data' section above.
warning is only given when the sync limit was reached - you should re-initialize, or keep requesting this endpoint with the latest cursor, until the warning disappears.
400 Bad request - missing argument
401 Unauthorized - missing or expired JWT
403 Forbidden - no access to this data
Requires JWT.
Get current sync cursors.
POST /sync/cursors
Authorization: Bearer <JWT>
{
"project": "<project gid>",
}
project gid for the project you are syncing.
Authorization: <JWT>
{
"success": true,
"result": {
"cc.cursor": <cc cursor>,
"cursor": <project cursor>,
},
"data": {}
}
cc.cursor is the latest cc cursor.
cursor is the latest project cursor, if you specified a project gid.
401 Unauthorized - missing or expired JWT
403 Forbidden - no access to this data