Pydantic AI: Concurrency-limited models can keep their slot when a streamed request ends early

QFinch assessment

Applications that wrap a model with ConcurrencyLimitedModel or limitmodelconcurrency can permanently lose shared concurrency capacity when a streamed request releases its slot from a different task than the one that acquired it. This can happen when a stream ends early, and also when a stream is fully consumed using the default streamtext() debouncing. In an application that exposes an affected streaming endpoint to network clients and shares a long-lived model limiter across requests, a client can repeatedly start a stream and disconnect. The completed requests retain their slots, eventually preventing subsequent requests that share the limiter from proceeding. Agent-level maxconcurrency and non-streaming model requests are not affected by this defect.

Technical context

The built-in limiter uses anyio.CapacityLimiter, which associates each acquired slot with its borrowing task. Pydantic AI's streaming lifecycle can acquire the slot on the task consuming the stream and run cleanup on another internal task. The limiter rejects that release, so the slot remains occupied even after the request has ended. Cleanup can raise a RuntimeError; a later request on the borrowing task can also fail because that task still holds a slot. Early termination includes stopping iteration, a consumer exception, and cancellation. Fully consuming streamtext() with its default debounceby=0.1 can also reach the cross-task release path. Fully consumed streams must therefore not be assumed safe.