RFE: batch size >1

[ppbrown]

I am officially requesting what someone THOUGHT #56 was asking for:

I would like

model.answer_question(enc_image, PROMPT, tokenizer)

to be able to accept an array of image tensors, rather than only a single one.

Right now, I'm using moondream for captioning large (100,000+ image) batches. Im running more than one instance of moondream, which somewhat improves my throughput, from 4 im/sec to maybe 7im/sec.

I Imagine though, that if it properly supported batch mode, throughput might go up to 10im/sec or more?

I am informed that some other "automodel" models support this sort of thing, so it would be awesome if moondream did too.