Skip to content

Use the newest lastSyncAt instead of the eldest one during sync #327

@lmatayoshi

Description

@lmatayoshi

Current behavior for syncing process is determined by:

The eldest lastSyncAt of the Collector is sent to the server in order to fetch all the entries that have been updated later than the eldest lastSyncAt. That forces the Collector to update the lastSyncAt of all the remaining entries so the eldest lastSyncAt gets updated and turns into the newest value in the next sync.

However, In order for the current approach to work, it is mandatory that the sync process completes entirely so the lastSyncAt of the remaining entries gets updated. Otherwise, next sync will fetch already fetched entries again. This situation gets worse in instances such as Nigeria, in which there are a lot of Antibiotic Consumption Stats (more than 70000) and syncing processes can last up to 8 hours.

Let's change the current approach and sent the newest lastSyncAt to the server. By doing this, it won't be necessary to repeat the whole syncing process again if it gets interrupted for any reason. Be careful with not updating lastSyncAt when uploading changes to the server (remoteUpload and remoteUploadUpdate) to avoid missing entries in the next sync that other Collectors could potentially have created during the upload.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions