Add the ability for ollama ports to stay open while it's still running with .WithLifetime(Persistent)

[JohnGalt1717]

Related to an existing integration?

Yes

Existing integration

Ollama

Overview

Right now you can specify a port to use, and you can specify persistence, but there is no way to specify that the instance keeps the port assigned. It would be nice to either have an extension for .WithPort(int, bool proxy = true) or add an optional parameter on AddOllama().

This would solve the problem so that the instance stays available for external calls while the aspire host isn't running.

Usage example

See above.

Breaking change?

No

Alternatives

I can't find any.

Additional context

This means you can use the instance and API while doing other things and makes Ollama with persistent lifetime work as you'd expect.

Help us help you

No, just wanted to propose this

[aaronpowell]

You can specify a port as the final argument of the AddOllama method:

var ollama = builder.AddOllama("ollama", 12345);

Alternatively, you can call the WithEndpoint method exposed by Aspire and provide a port, which would override the one that we do initially.

Would either of these do what you're after?

[JohnGalt1717]

Unfortunately not because when you shut down the aspire host, the port mapping is removed even though the container is not because of WithLifetime(Persistent)

Basically when you set WithLifetime(Persistent), the container should be up and be able to be used even when the aspire host isn't running which means that the ports should stay available.

Interestingly, the UI.WithLifetime(Persistent) does keep the ports open, but it can't access the container for the ollama server because those ports were removed.

[aaronpowell]

ah yes, because that port is the port that is used by the proxy in the app host, not the port assigned in Docker when the container is started.

I feel like this problem is a by-design, my understanding is that persistent containers are persistent from the app host perspective, but they aren't meant to be then used by other systems outside of the app host/aspire app. @davidfowl is this the right assumption that I have?

[davidfowl]

We've implemeneted a mititgation in 9.1 dotnet/aspire#7063 (comment)

[JohnGalt1717]

On some of the implementations there is a flag "useProxy" that if you set to false, tells it not to proxy the ports but to directly expose them.

Is it possible to enable this for ollama as an optional parameter like others have done in the mean time?

[davidfowl]

If you want a workaround right now, then you should be able to turn off the proxy like this:

var ollama = builder.AddOllama("ollama", 12345).WithEndpoint("http", e => e.IsProxied = false);