Class EmbeddingRequestPacer

Namespace

MailFathom.Application.Emails.Embeddings.Limits

Assembly

MailFathom.Application.dll

Spaces embedding requests out so a deployment never sends faster than it declared it would.

public sealed class EmbeddingRequestPacer

Inheritance

object

EmbeddingRequestPacer

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

A rate ceiling and the spend ceiling beside it bound different things and neither substitutes for the other: the budget decides how much a period may cost, this decides how quickly that cost is allowed to accumulate. It exists because a provider quota is stated per minute rather than per month, and because being refused with a rate-limit response costs an attempt, a retry, and a place in a circuit-breaker window that other work is measured in.

It is also not the concurrency limit. How many calls may be in flight at once is the resilience pipeline's AiProviderInvocation budget, which is the one mechanism that owns that question; a second limiter counting in-flight calls here would make two settings answer for one behaviour.

The pacing is a slot reservation rather than a token bucket that refills on a timer: a caller takes the next free slot, moves the marker forward by one interval, and waits for its own slot to arrive. Nothing polls, nothing spins, and every wait is a single cancellable delay measured on the injected TimeProvider — which is what lets a test prove that the ceiling binds and then releases, rather than proving it against a wall clock.

Properties

IsUnpaced

Gets whether this pacer delays nothing, which is what a rate of zero asked for.

Methods

Create(int, TimeProvider)

Builds a pacer from the rate a deployment declared.

WaitForSlotAsync(CancellationToken)

Waits until this deployment is allowed to send its next embedding request.