AI request info (proto)

Warning

This API feature is currently work-in-progress. API features marked as work-in-progress are not considered stable, are not covered by the threat model, are not supported by the security team, and are subject to breaking changes. Do not use this feature without understanding each of the previous points.

data.ai.v3.RequestInfo

[data.ai.v3.RequestInfo proto]

Request attributes published as typed dynamic metadata by the request info AI filter. Values are client-declared and unverified, apart from estimated_input_tokens, which Envoy computes. A value the client sent that Envoy cannot use, such as one of the wrong type, out of range, or a string over 256 bytes, is ignored.

{
  "input_llm_protocol": ...,
  "model": ...,
  "stream": {...},
  "max_output_tokens": {...},
  "message_count": {...},
  "tool_count": {...},
  "estimated_input_tokens": {...}
}

input_llm_protocol

(type.ai.v3.LLMProtocol) The route’s declared wire API. When unspecified, only model and stream are read.

model

(string) The requested model.

stream

(BoolValue) Whether a streamed response was requested.

max_output_tokens

(UInt64Value) The requested output token cap.

message_count

(UInt32Value) Number of messages in the conversation.

tool_count

(UInt32Value) Number of tools declared.

estimated_input_tokens

(UInt64Value) Envoy’s estimate of the input tokens, published only when token_estimation is configured: ceil(tokens_per_byte * request payload bytes). A size heuristic, not a tokenizer result, and independent of the wire API.