Skip to content

[Feature] Optimize /content add endpoints #98

@jcace

Description

@jcace
  • Calling POST /api/v1/datasets/contents/ (i.e, from edge-vertex) takes longer depending on how many contents are being published. In my testing, I saw it take several seconds with only ~20 contents to be added
  • Additionally, importing from singularity where we had 8000+ contents took a solid 5+ minutes
  • We need to optimize the db insert queries to speed it up - this should be a very fast operation. Currently it does each one individually to keep track of success/fail, so would be good to investigate if we can somehow batch them together for one db transaction instead

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions