Dev/composition ids

[tom-kuchler]

Adding prioritization of older requests in the queuing.

[tostocker]

Looks good - I only have one question. Now that we have a sorted insertion that is O(n) it would probably make sense to use a better data structure for the compute_queue and io_queue so we get something like O(log(n))?

[tom-kuchler]

Looks good - I only have one question. Now that we have a sorted insertion that is O(n) it would probably make sense to use a better data structure for the compute_queue and io_queue so we get something like O(log(n))?

I though about that too.
The issue is if we want to use a single layer map we need to add some secondary key, since the composition ID is not unique.
For this we could extend the composition id with a second id that is local to the composition.
I think that could be a reasonable quick fix.
Alternatively we could add a second layer, where we have a top level queue holding a per composition data structure which then hold the actual queues for the functions.
The two level version could eliminate the issue of having to carry the keys around everywhere, which would be nicer, but it also makes taking work from the queue more complicated.

I think the first one could be done fairly quickly, if you think that would be a worthwhile thing to have.
Otherwise I would open an issue to make sure it does not get lost and address it later.

[tostocker]

Looks good - I only have one question. Now that we have a sorted insertion that is O(n) it would probably make sense to use a better data structure for the compute_queue and io_queue so we get something like O(log(n))?

I though about that too. The issue is if we want to use a single layer map we need to add some secondary key, since the composition ID is not unique. For this we could extend the composition id with a second id that is local to the composition. I think that could be a reasonable quick fix. Alternatively we could add a second layer, where we have a top level queue holding a per composition data structure which then hold the actual queues for the functions. The two level version could eliminate the issue of having to carry the keys around everywhere, which would be nicer, but it also makes taking work from the queue more complicated.

I think the first one could be done fairly quickly, if you think that would be a worthwhile thing to have. Otherwise I would open an issue to make sure it does not get lost and address it later.

Ok I see - let's merge and open an issue.