Dynamic modules route specifier

Overview

The DynamicModuleRouteSpecifier configuration specifies a route specifier backed by a dynamic module. For each request the module keeps the route that route matching resolved, refines it, replaces it with one of the route templates it declares, drops it, or lets route matching carry on with the next route.

The module is invoked while the route is being resolved, and again whenever the route is recomputed, so it must be able to reach a decision from the request and the stream info alone. The call is synchronous and cannot be time boxed, so a module must not block or perform I/O.

Route templates

A module cannot build a route out of thin air. Instead it selects one of the route_templates the specifier declares. Each template is an ordinary Route that is built and validated once, when the specifier is configured, so an invalid template is rejected at configuration load rather than on the request path. A template inherits from the virtual host and the route configuration the specifier is configured on, exactly like a configured route does.

When the module selects a template, Envoy evaluates it against the request like a configured route. The match must hold, and the action is resolved the usual way, including weighted_clusters and cluster specifier plugins. A match that does not hold is a failure, because the rewrites of the action are only correct for a request the match accepts.

Decisions

The module records one of five decisions with set_decision:

• Unspecified is the default, in effect when the module records no decision: Envoy generates a new route based on what the setter callbacks recorded, from the selected template, or from the route the specifier was given when no template was selected, with the recorded properties applied on top. With nothing recorded it uses the route the specifier was given, unchanged.

• PassThrough uses the route the specifier was given, unchanged, ignoring whatever the module recorded.

• NoRoute uses no route, so the request is handled as if nothing had matched.

• Error reports that the module could not decide.

• ReusePrevious uses the previous route of the stream unchanged, without building a new route.

The hook returns a status rather than the decision: Continue hands the produced route to the route specifiers configured after this one, StopIteration makes it the final route, and StopIterationAndSkipRoute drops the route and lets route matching carry on with the next route, ignoring the recorded decision.

A decision Envoy cannot honor is handled by the configured failure_policy, which passes the request through to the route table, drops the route, or lets route matching carry on with the next route. The SDK reports an error when the module panics, so a panic is handled by the same policy rather than crashing Envoy.

Stream consistent decisions

Envoy resolves the route more than once per stream, for example after a filter clears the route cache. On a later resolution the module reads the route the connection manager last installed with get_previous_route and get_previous_route_metadata, which are null on the first resolution, after an internal redirect, and when a filter installed a null route. Reading a marker the module wrote into the metadata of a route it built earlier, through set_route_metadata_string, is how a module keeps a stream on the version it first served without depending on route object identity.

ReusePrevious uses that previous route unchanged and builds nothing. When the stream has no previous route the decision is rejected and counted in reuse_previous_rejected, and the failure policy applies.

A module that builds routes from versioned state can lease that version for as long as any route it built is alive. set_route_user_data records a u64 on the produced route, which forces the route to be wrapped even with no other override, and on_route_specifier_route_destroy fires with that value when the route is destroyed. The hook may run on any thread and after the stream is gone, since another owner may retain the route, so a module must treat it as a lease release only. The specifier_instance_id lets two specifier instances sharing one module tell their routes apart.

Shadowing a routing change

A module receives the route that route matching resolved through its context, so it can shadow a routing change on its own without any support from Envoy. In a dry run the module computes the route it would ask for, compares it against the resolved route, counts the outcome on its own metrics, and records PassThrough so that routing stays unchanged. Once the counts give confidence, the same module records its overrides or selects its template to apply the decision. The route_specifier_shadow.rs test module under test/extensions/dynamic_modules/test_data/rust shows both runs, comparing the cluster name.

Migrating routing to a module

A module can take over the routing of a virtual host one step at a time, while the route table it replaces stays in place as a fallback. The pattern is a catch all route placed first in the route table, carrying the specifier at the route level. The catch all route matches every request, so the specifier runs first for each one. For a request the module owns it records its overrides or selects its template, and for a request the module leaves to the route table it returns the StopIterationAndSkipRoute status, which drops the catch all route and lets route matching carry on with the routes below it. Setting failure_policy to CONTINUE_MATCHING makes a module failure fall back to the route table as well, rather than to the catch all route’s own action. When the module owns the whole virtual host, the routes below the catch all route can be removed and the failure_policy changed to NO_ROUTE.

Route overrides

The properties that are built from other extensions, such as the retry policy and the request mirroring policies, are declared as route_overrides. Each override is built and validated once when the specifier is configured, and the module selects one by override_id. An override that replaces no property is rejected, so a module can rely on a declared override changing something.

An override may also carry tracing and metadata, which are route level properties valid on any route including a direct response or a redirect, unlike the retry policy and the other route entry properties. Module selected route metadata can change authorization decisions, rate limiting descriptors and access log fields, so the override set is static configuration chosen by trusted in process code.

Notes

• The request headers are read-only. A recorded path, authority or header mutation is applied through the header transforms of the route the decision produces, so that Envoy applies it at the right point of the request lifetime.

• get_cluster_host_count reports whether a cluster is routable from the current worker and returns host counts at a priority level. It uses getThreadLocalCluster(), so it can return false even when the cluster is configured but not yet warmed on the worker.

• Custom counters, gauges and histograms can be defined during configuration and recorded during resolution, and are emitted under the metrics_namespace prefix of DynamicModuleConfig.

Statistics

The specifier emits statistics rooted at <metrics_namespace>.route_specifier.<stat_prefix>., where stat_prefix is the stat_prefix of the specifier, sharing the metrics_namespace of the module-defined metrics above.

Name	Type	Description
decision_pass_through	Counter	Requests for which the module kept the resolved route.
decision_has_override	Counter	Requests the default decision resolved from the route the specifier was given with the recorded overrides applied.
decision_has_template	Counter	Requests for which the module selected a route template.
decision_no_route	Counter	Requests for which the module dropped the route.
decision_error	Counter	Requests for which the module could not decide.
decision_reuse_previous	Counter	Requests for which the module reused the previous route.
route_skipped	Counter	Requests for which the module let route matching carry on with the next route.
unknown_status	Counter	Requests for which the module returned a status Envoy does not know.
runtime_skipped	Counter	Requests outside runtime_fraction.
failure_module_error	Counter	Decisions not honored because the module reported an error.
failure_template_not_selected	Counter	Decisions not honored because a template selection named an identifier that is not declared.
failure_template_match_failed	Counter	Decisions not honored because the match of the selected template did not hold.
failure_override_without_route	Counter	Decisions not honored because there was no route to refine.
failure_override_on_non_route_entry	Counter	Decisions not honored because route entry properties were recorded for a direct response.
failure_route_metadata	Counter	Decisions not honored because a typed metadata factory rejected the recorded metadata.
reuse_previous_rejected	Counter	ReusePrevious decisions rejected because the stream had no previous route.
route_destroy	Counter	Routes built with user data that were destroyed and fired the route destroy hook.
on_route_duration	Histogram	Time in microseconds the module spent deciding.
specifier_duration	Histogram	Time in microseconds the specifier spent on a request.

Configuration

• This extension should be configured with the type URL type.googleapis.com/envoy.extensions.router.route_specifiers.dynamic_modules.v3.DynamicModuleRouteSpecifier.

• v3 API reference

Attention

Dynamic modules run in-process with the same privileges as Envoy. Only load modules you trust. This extension is currently under active development. Capabilities and ABI are expected to evolve.

Configuration example

route_config:
  virtual_hosts:
  - name: default
    domains: ["*"]
    route_specifiers:
    - name: envoy.router.route_specifiers.dynamic_modules
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.router.route_specifiers.dynamic_modules.v3.DynamicModuleRouteSpecifier
        dynamic_module_config:
          name: my_route_specifier
          do_not_close: true
        specifier_name: my_specifier_impl
        stat_prefix: my_specifier
        failure_policy: PASS_THROUGH
        route_templates:
        - template_id: canary
          route:
            match:
              prefix: "/"
            route:
              cluster: canary_service
              timeout: 5s
    routes:
    - match:
        prefix: "/"
      route:
        cluster: web_service