.. _envoy_v3_api_file_envoy/data/ai/v3/request_info.proto:

AI request info (proto)
=======================


.. warning::
   This API feature is currently work-in-progress. API features marked as work-in-progress are not considered stable, are not covered by the :ref:`threat model <arch_overview_threat_model>`, are not supported by the security team, and are subject to breaking changes. Do not use this feature without understanding each of the previous points.







.. _envoy_v3_api_msg_data.ai.v3.RequestInfo:

data.ai.v3.RequestInfo
----------------------


:repo:`[data.ai.v3.RequestInfo proto] <api/envoy/data/ai/v3/request_info.proto#L27>`

Request attributes published as typed dynamic metadata by the :ref:`request info AI
filter <envoy_v3_api_msg_extensions.http.ai_filters.request_info.v3.RequestInfo>`. Values are
client-declared and unverified, apart from ``estimated_input_tokens``, which Envoy computes.
A value the client sent that Envoy cannot use, such as one of the wrong type, out of range,
or a string over 256 bytes, is ignored.



.. code-block:: json
  :force:

  {
    "input_llm_protocol": ...,
    "model": ...,
    "stream": {...},
    "max_output_tokens": {...},
    "message_count": {...},
    "tool_count": {...},
    "estimated_input_tokens": {...}
  }

.. _envoy_v3_api_field_data.ai.v3.RequestInfo.input_llm_protocol:


input_llm_protocol
  (:ref:`type.ai.v3.LLMProtocol <envoy_v3_api_enum_type.ai.v3.LLMProtocol>`) The route's declared wire API. When unspecified, only ``model`` and ``stream`` are read.


.. _envoy_v3_api_field_data.ai.v3.RequestInfo.model:


model
  (`string <https://developers.google.com/protocol-buffers/docs/proto#scalar>`_) The requested model.


.. _envoy_v3_api_field_data.ai.v3.RequestInfo.stream:


stream
  (`BoolValue <https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#boolvalue>`_) Whether a streamed response was requested.


.. _envoy_v3_api_field_data.ai.v3.RequestInfo.max_output_tokens:


max_output_tokens
  (`UInt64Value <https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#uint64value>`_) The requested output token cap.


.. _envoy_v3_api_field_data.ai.v3.RequestInfo.message_count:


message_count
  (`UInt32Value <https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#uint32value>`_) Number of messages in the conversation.


.. _envoy_v3_api_field_data.ai.v3.RequestInfo.tool_count:


tool_count
  (`UInt32Value <https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#uint32value>`_) Number of tools declared.


.. _envoy_v3_api_field_data.ai.v3.RequestInfo.estimated_input_tokens:


estimated_input_tokens
  (`UInt64Value <https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#uint64value>`_) Envoy's estimate of the input tokens, published only when :ref:`token_estimation
  <envoy_v3_api_field_extensions.http.ai_filters.request_info.v3.RequestInfo.token_estimation>`
  is configured: ``ceil(tokens_per_byte * request payload bytes)``. A size heuristic, not a
  tokenizer result, and independent of the wire API.