Embedding generation

A chunk is a passage of text. A vector is where that passage lands in a space a model defines, and two vectors of one space can be compared, which is what makes semantic search possible at all. This page describes how MailFathom turns the first into the second: what it declares, what it calls, and what it does when the call fails.

Nothing here decides when to embed, and nothing here stores a vector. Those belong to the worker and to the schema — automatic embedding is what decides that a newly synchronized message should be embedded. What this owns is the one boundary that talks to a provider.

An instance that embeds nothing is a working instance

Writing no Embeddings section is a supported deployment rather than an omission. No vectors are produced, no provider is called, no credential is needed, and lexical search answers exactly as it did before. That is the state an operator who has not chosen a provider should be left in, rather than being made to choose one to start the service.

Chunking happens either way. Chunks reach no network and cost nothing an operator has to consent to, and they are what a later activation embeds — so an instance that turns embedding on later has the passages already.

Declaring is free; activating is what spends

Which model an instance embeds with is a configuration value, so a reviewer, a chart, and a git diff all see it. Editing that value starts nothing: it says what this deployment intends, and an explicit activation is what computes the profile identity from it, states the estimate, takes the confirmation, and begins producing vectors.

That split is deliberate and is the whole of ADR 0006's answer to a hazard worth naming: embedding is the first thing MailFathom does that costs money per unit of mail. A configuration edit that re-embedded a mailbox at the next restart would be an invoice arriving a month later for a decision nobody was shown.

An operator who edits the model and expects the change to take effect has to know that it did not. The activation command is what makes it so, and it is deliberately not automatic.

What a declaration says

Each entry of Embeddings:Endpoints declares a whole geometry — provider, model, model version, dimension, distance metric, and how a passage is prepared — beside the endpoint address and the credential. Configuration reference is the inventory.

The geometry is repeated per endpoint rather than stated once for the chain, and that is the point rather than duplication to tidy away. It is what makes a disagreement expressible, and therefore refusable.

The chain is one vector space reached several ways

Embeddings:Endpoints is ordered, and a failing endpoint falls through to the next. What every entry must share is the geometry: the same model, the same width, the same metric, the same preparation. One model offered both by a vendor's own API and by a cloud deployment of it is the case this exists for — the endpoint fails, the vector space does not.

A chain whose entries disagree is refused at startup, naming both aliases and the property they differ on. The refusal is not a restriction to work around. A fallback on a different model does not produce a degraded vector; it produces a point in a different space, and a distance computed against it is a number with no meaning. Written under the active profile, those vectors would make retrieval slightly worse rather than fail — which is the hardest possible failure to attribute.

An operator who genuinely wants a different model when the first is unavailable is asking for a second profile and a switch between them. That is a deliberate operation, not a fallback.

Falling through is logged and changes nothing about what is written: the vectors a fallback returns belong to the same profile, because its identity is the same.

The endpoint's name is what everything else calls it

Everything else about an endpoint is either an address or a credential, and neither may be written down: an address identifies a tenant and a resource, so a failure message that named one would put it in every log line. Alias is the name the operator chose, and it is what a log record, a metric tag, a resilience circuit, and a failure message use.

Vector width is a database decision

pgvector stores far more than it indexes: a vector column holds up to 16000 dimensions and an HNSW index covers 2000. So a model is not merely large or small — it is indexable, or it is stored and searched exactly, which is correct but linear in the number of vectors.

The index that covers it belongs to one profile rather than to the table, which is why a width is a property of a generation instead of of the schema. Stored email schema describes its shape and what maintaining one costs.

AllowTrimVectors decides which, and it is off by default. With it off, a declared dimension above what an index covers is refused at startup, naming the dimension and the ceiling, rather than quietly producing an instance whose semantic search never becomes fast. With it on, the declared width is what the profile records and what the stored vectors have — because a trimmed vector occupies a different space than the full one, and a profile claiming the model's nominal width would be describing vectors that do not exist.

Where the endpoint can produce the narrower vector itself, that is used in preference to cutting one down: SupportsRequestedDimension asks for the declared width, and a model trained to answer at it returns a vector already normalized for it. Where the adapter must shorten a wider answer, it renormalizes — dropping the tail of a unit vector leaves one shorter than unit length, and a cosine distance between vectors of differing lengths is a number rather than an error.

Two providers, one client

OpenAI and Azure OpenAI are the two this release reaches, and one client construction serves both: an Azure resource's v1 data plane is OpenAI-compatible, so a cloud deployment is the same client pointed at the resource's own /openai/v1/ address with the deployment's name as the routed model. The choice is deliberate beyond embeddings — the chat model the answering feature will need is served by this same wiring rather than by a second one.

Support beyond these two is deliberately out of scope and is tracked separately.

Authentication has two shapes

An endpoint carries either a provider key or a Microsoft Entra credential, never both and never neither, and startup refuses any other combination.

A key is a secret reference like every other credential this deployment holds, resolved per request — so rotating it behind an unchanged reference takes effect on the next call, with no cache to invalidate and no restart.

A Microsoft Entra credential exists for the deployment where there is no secret to provision at all. Four shapes are supported and they are the whole of the set: managed identity, workload identity, client secret, and client certificate. Every one of them is non-interactive by construction, because MailFathom is a background service with nobody at a keyboard: a credential that opens a browser or prints a device code would surface as a request that never returns.

DefaultAzureCredential is deliberately not used. Its chain reaches both of those interactive shapes and the developer-tool credentials of whoever last signed in on the host — which would let a deployed service authenticate as an operator's own account because a stale sign-in happened to be there. MailFathom composes its four explicitly and reaches nothing else.

The token a Microsoft Entra credential fetches is cached by the credential, which is what it exists to do, so the credential is built once per endpoint. One consequence is worth stating: rotating the client secret of a registered application takes effect at the next restart, while rotating a provider key takes effect on the next call.

What a failing call is classified as

A provider that fails says six different things, and collapsing them into "the call failed" gets the next two decisions wrong. A rate limit answered with an immediate retry is how an account gets throttled harder; a refused credential repeated is how the same refusal is bought again while the account carries the requests.

Classification	What it means	Repeated?
Credential rejected	The endpoint refused the credential presented	No — rotate or correct it
Rate limited	The deployment is over its allowed rate	Yes, after a backoff
Request timed out	The endpoint did not answer within the configured time	Yes
Transport faulted	The request never reached an answer	Yes
Request refused	The endpoint rejected the request itself — a model it does not serve, an input beyond what it accepts	No — correct the declaration
Vector shape unexpected	The answer is not in the declared space: a width nothing declared, a count that does not match the passages, a component that is not a finite number	No

A caller's own cancellation and a host shutdown are absent from that table on purpose. Both are this system's own decision rather than a remote party's answer, so reporting one as a provider failure would let it open a circuit against a healthy endpoint.

Falling through the chain follows the same reasoning but answers a different question. An unreachable, throttled, slow, or credential-refusing endpoint says nothing about the next one, which is a different address with a different credential, so the chain continues. So does an endpoint the resilience pipeline declined to call at all — a circuit it opened after repeated failures, or a concurrency budget already spent — because an unavailable first endpoint is precisely the condition a fallback exists for, and ending the request there would make the whole chain unusable for as long as its first entry stayed broken. An answer of the wrong shape is not: every endpoint declares the same geometry, so a width nothing declared means the declaration is wrong and asking the next endpoint would buy a second paid call to learn the same thing.

A vector whose length does not match the profile's dimension is therefore a failure at the adapter, named as such, rather than a row the database rejects later with no provider in sight.

Bounds every call carries

• A batch bound. MaxPassagesPerRequest is applied before the provider sees a request, and a caller reads the same number from the port so it can cut its own work to it. A batch beyond it is refused rather than split, because splitting would spend the caller's budget on a number of requests it never chose.
• A per-passage bound. InputCharacterLimit is what a passage is cut to. It is deliberately not a second setting beside the profile's: what the model sees decides what a vector means, so a rule able to cut a passage differently from the one the profile records would produce vectors in a space nothing declared.
• An explicit timeout. RequestTimeout bounds one request to one endpoint, applied by MailFathom rather than left to whatever the provider library defaults to. A deadline that expires is reported as a timeout rather than as a cancellation, because a cancellation would tell the pipeline that this system stopped the work.
• One retry layer. The call runs under the AiProviderInvocation resilience pipeline, and both the provider library's own retry policy and the standard HTTP resilience handler are switched off for it. Outbound resilience holds the rule and why two layers multiply rather than add.
• A bounded response. The transport refuses a body larger than the declared geometry could fill, and refuses redirects — a moved endpoint answering with one would carry the key or the bearer token to whatever host it named.

What never reaches a log

No passage, no vector, no credential, and no provider response body. A provider's own error text quotes the request that produced it, and the request is mail text; the classification, the endpoint alias, and the counts are what a log record carries instead. Vectors inherit the classification of the mail they derive from and are not treated as anonymous.

Proving it without spending

Almost everything downstream of this boundary is provable at zero provider cost, and a deterministic in-repository generator is what makes that true. It derives a vector from the text alone — reproducible, of the declared width, and of unit length — so the schema, the worker, the backfill, and the generation switch are all testable against a real database and no provider at all. Its profile names a provider of its own, so a deployment that activated it by accident is visible in the profile row rather than in the quality of its search results.

What only a real provider can establish is much smaller: that the adapter speaks the protocol, authenticates, classifies a real refusal, and returns the width the profile claims. Those tests exist, and they are skipped unless somebody asks — the Integration tests workflow turns them on through an input that defaults to off. Asking for them without a credential configured fails the run rather than skipping, because a run somebody requested and which then quietly proved nothing is worse than one that never started.