Sync API

Samples

Sample application that uses Sync API

Sync API is a set of methods that let applications reliably track all changes of particular entities in Space. To track the changes, Sync API uses etags (entity versions). This approach prevents synchronization issues (e.g., due to bugs or network errors) and ensures that the data retrieved by an application is always up-to-date and consistent with the latest changes in Space.

Suppose you're developing an application that integrates Space with a third-party issue tracker. In such a scenario, your application can use Sync API to maintain a consistent state of issues – in sync with Space.

Another example of where Sync API might be helpful is applications that must track specific Space notifications, e.g., chat messages in some channel. The main job of such an application is to ensure it doesn't miss any single message in the channel. Here, an effective strategy is to use Sync API in pair with webhooks. Webhooks notify when a new message is posted, while Sync API can be periodically invoked to double-check for any missed messages (e.g., the messages could be sent during network disruptions).

Sync API is currently available for the following Space entities:

For these entities, Sync API lets you:

Get records of a particular entity
Reliably receive and process record updates

Use Sync API

For all supported entities, Sync API provides the Get sync batch method. You can find all sync methods in the API Playground using the sync filter:

For example, this is how you can use the Get sync batch method to get all issues in a project:

var latestEtag = etag
var hasMore = true

while (hasMore) {
    // get a batch of issues using Sync API
    val batch = spaceClient.projects.planning.issues.getSyncBatch(
        project = ProjectIdentifier.Id(projectId),
        // we get not all issues at once but in batches of `batchSize`
        // starting with the issue that has the `latestEtag`
        batchInfo = SyncBatchInfo.SinceEtag(
            etag = latestEtag,
            batchSize = 50
        )
    )
    // do something with batch.data
    // ...

    // `latestEtag` is now the etag of the last issue in the batch
    latestEtag = batch.etag
    hasMore = batch.hasMore
}

Find the full example here.

A single request that returns all issues in the MY-PRJ project:

GET https://mycompany.jetbrains.space/api/http/projects/key:MY-PRJ/planning/issues/sync-batch?batchInfo={etag:0,batchSize:50}
Authorization: Bearer here_goes_your_token
Accept: application/json

How Sync API works

Sync API operates not entities but records. A record represents an entity at a particular point in time. It contains the entity's data and its etag – the entity version. Each time the entity is created, updated, or deleted, the etag value is increased.

When making a call, you specify the etag of the last record you received. A returned record will contain Space entities starting from the specified etag. To get all entities, specify etag equal 0.

All Sync API methods return the following data structure:

Field	Type	Description
`data`	For chat messages: `ChannelItemSyncRecord` For emojis: `CustomEmojiInfo` For issues: `Issue` For teams: `TD_Team`	A batch of Space entities. The entity type depends on the initial Sync API call. For details on these entity types, refer to the API Playground.
`etag`	`string`	The etag of the last entity in the batch.
`hasMore`	`boolean`	Specifies whether there are more entities to retrieve.

Field

Type

Description

data

For chat messages: ChannelItemSyncRecord

For emojis: CustomEmojiInfo

For issues: Issue

For teams: TD_Team

A batch of Space entities. The entity type depends on the initial Sync API call. For details on these entity types, refer to the API Playground.

etag

string

The etag of the last entity in the batch.

hasMore

boolean

Specifies whether there are more entities to retrieve.

To reduce the size of transferred data, Sync API returns data in batches of a limited size (defined by the call). To get all entities, you need to make several calls. For example, a Space instance contains 250 issues that your application wants to retrieve. If a batch size is 100, the application has to make 3 calls. Note that it's possible for an entity to be returned multiple times if it gets updated between two API requests.

Use Sync API to get chat messages

Sync API works a bit differently with chat messages. For applications that sync messages, it is important to preserve message order. The problem is that if a chat message is updated, it gets a new etag instead of the one given it by creation. As a result, this message will be returned after other messages (created during the interval between the message creation and update). This means that applications cannot simply iterate over records from Sync API and send messages one by one without disrupting the order.

To address this issue, Space stores not only the latest chat message etag but all its etag history, starting from the initial etag given the message by its creation. At the same time, Space doesn't store message text history – it keeps only the most recent text version. Thus, if you request a message by its earlier etag, you will still receive the latest version of the message text, though the datetime and the modType (CREATED, UPDATED, or ARCHIVED) fields will be accurate for the requested etag.

A Get sync batch call for chat messages returns records with the data: ChannelItemSyncRecord field, which in turn consists of the following fields:

Field	Type	Description
`chat`	`ChannelItemRecord`	Chat message data including its `id`, `text`, and other fields. Note that the `archived: Boolean` field indicates whether the message is deleted in addition to the `modType = "ARCHIVED"` state.
`etag`	`string`	The etag of the message.
`modType`	`string`	The message state: `CREATED`, `UPDATED`, or `ARCHIVED` (deleted).

Sync API call that gets chat messages looks the same as for other Space entities. The only difference is that you get a somewhat different data structure in the response:

spaceClient.chats.messages.syncBatch.getSyncBatch(
    batchInfo = SyncBatchInfo.SinceEtag(
        etag = latestEtag,
        batchSize = batchSize
    ),
    channel = ChannelIdentifier.Channel(ChatChannel.FromName("My chat channel"))
) {
    chatMessage {
        text()
        id()
        archived()
        created()
        edited()
    }
    etag()
    modType()
}

GET https://mycompany.jetbrains.space/api/http/chats/messages/sync-batch?batchInfo={etag:0,batchSize:50}&channel=channel:name:My%20chat%20channel&$fields=data(chatMessage(text,id,archived,created,edited),etag,modType),etag,hasMore
                    Authorization: Bearer here_goes_the_token
                    Accept: application/json

How message states work

Consider an example:

12:00 - Message A is created (id = A, etag = 1, text = abc).
12:05 - Message B is created (id = B, etag = 2, text = def).
12:10 - Message C is created (id = C, etag = 3, text = ghi).
12:15 - Message B is updated (id = B, etag = 4, text = 123).

After this, say at 12:20, we use Sync API to get all messages: etag = 0. Space will return the following records:

`data.chatMessage.id`	A	B	C
`data.etag`	1	2	3
`data.modType`	CREATED	CREATED	CREATED
`data.chatMessage.text`	abc	123	ghi
`data.chatMessage.created`	12:00	12:05	12:10
`data.chatMessage.edited`	—	12:15	—

Even though message B was already updated, it is returned in the CREATED state and with the old etag. This is because, for such cases, Sync API conflates records into one.

Now, imagine we performed Sync at 12:13 – after all three messages are created but before message B is updated (our latest etag is now 3, and the text of message B is def). At 12:20, we want to update the messages, so, we send Sync API request with etag = 3. In this case, Space will return just one record:

`data.chatMessage.id`	B
`data.etag`	4
`data.modType`	UPDATED
`data.chatMessage.text`	123
`data.chatMessage.created`	12:05
`data.chatMessage.edited`	12:15

For deleted messages, Space always returns an ARCHIVED record with an empty string as text.

Last modified: 08 August 2023