feat(adapter): add optional update for progress tracking on timer

[findkim]

Inference can be slow depending on the model performance, the number of GPUs the model is running on, the context length or tokens to generate or parallelism. For slow inference times, an interval based progress tracking may be a frustrating user experience to wait 30mins or longer for the first progress update.

This PR adds an optional parameter to ProgressTracking adapter to update progress on a timed interval, not enabled by default. ProgressTrackingInterceptor tracks the last status that was posted, and skips sending a request / writes to file if there has not been a change. The timed update is run as a asyncio task that is stopped on post_eval_hook

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[agronskiy]

question / concern: since this is asyncio inside multithreaded Flask, could you clarify how this task would behave? Is the asyncio loop executed on in the main Flask thread where the object is created, and would that keep blocking? There are suggestions on SO to have an asyncio on a separate thread to avoid issues with asyncio + threading marriage.

[findkim]

Thanks for the quick review! I’ve verified asyncio.create_task working with fastapi but have not with flask, this is a good callout to do extra testing

[findkim]

I've added a test on the adapter layer and found that asyncio.create_task errors with no running event loop which tracks with your series of questions about the asyncio loop. Once I reworked the background timer to run as a thread and removed asyncio.sleep, the Flask server seems to running as expected and the timer thread does not block incoming requests. Lmk what you think!

[agronskiy]

I think we can merge this since you tested this on this adapter.