Skip to content

Commit

Permalink
Hints -> Routing, vet for correctness, and note that we don't do the …
Browse files Browse the repository at this point in the history
…new trafficDistribution feature

Signed-off-by: Flynn <[email protected]>
  • Loading branch information
kflynn committed Feb 25, 2025
1 parent 249c74a commit a7e6190
Show file tree
Hide file tree
Showing 18 changed files with 492 additions and 366 deletions.
36 changes: 0 additions & 36 deletions linkerd.io/content/2.12/features/topology-aware-hints.md

This file was deleted.

47 changes: 47 additions & 0 deletions linkerd.io/content/2.12/features/topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Topology Aware Routing
description: |-
Linkerd's implementation of Kubernetes topology aware routing enables endpoint consumption based on a node's zone label.
---

Kubernetes clusters are increasingly deployed in multi-zone environments with
network traffic often relying on obscure endpoint matching and routing for a
given service. Linkerd's implementation of Kubernetes ([Topology Aware
Routing][topology aware routing]) provides users with a mechanism for
asserting more control over endpoint selection and routing within a cluster
that spans multiple zones. Users can now implement routing constraints and
prefer endpoints in a specific zone in order to limit cross-zone networking
costs or improve performance through lowered cross-zone latency and bandwidth
constraints.

The goal of topology aware routing is to to provide a simpler way for users to
prefer endpoints by basing decisions solely off the node's
`topology.kubernetes.io/zone` label. If a client is in `zone-a`, then it
should prefer endpoints marked for use by clients in `zone-a`. When the
feature is enabled and the label set, Linkerd's destination controller will
attempt to find endpoints whose `routing.ForZones` field matches the client's
zone.

To get started with topology aware routing take a look at the [enabling
topology aware routing](../../tasks/enabling-topology-aware-routing/) task
documentation.

[topology aware routing]:
https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/

{{< note >}}

Topology aware routing requires that the cluster have the `TopologyAwareHints`
feature gate enabled (which is the default starting with Kubernetes 1.24), and
that Linkerd's `endpointSlice` feature be turned on (this is the default
starting with Linkerd stable-2.12).

{{< /note >}}

{{< note >}}

Starting in Kubernetes 1.31, Kubernetes also has the `trafficDistribution`
feature available as an alternative to topology aware routing.
`trafficDistribution` is not yet supported by Linkerd.

{{< /note >}}
Original file line number Diff line number Diff line change
@@ -1,28 +1,32 @@
---
title: "Enabling Topology Aware Hints"
title: "Enabling Topology Aware Routing"
description: |-
Enable topology aware hints to allow Linkerd to intelligently choose same-zone endpoints
Enable topology aware routing to allow Linkerd to intelligently choose same-zone endpoints
---

[Topology aware hints](../../features/topology-aware-hints/) allow Linkerd
users to specify endpoint selection boundaries when a request is made against
a service, making it possible to limit cross-zone endpoint selection and lower
the associated costs and latency.
[Topology aware routing](../../features/topology-aware-routing/) allows
Linkerd users to specify endpoint selection boundaries when a request is made
against a service, making it possible to limit cross-zone endpoint selection
and lower the associated costs and latency.

There are three requirements to successfully enabling topology aware hints with Linkerd:
There are three requirements to successfully enabling topology aware routing
with Linkerd:

1. `TopologyAwareHints` feature gate MUST be enabled on the Kubernetes
cluster.
2. Each Kubernetes node should be labeled with the
1. The `TopologyAwareHints` feature gate MUST be enabled on the Kubernetes
cluster. (This feature gate is enabled by default in Kubernetes 1.24 and
later.)

2. Each Kubernetes node needs to be assigned to a zone, using
`"topology.kubernetes.io/zone` label.

3. Relevant Kubernetes services will need to be modified with the
`service.kubernetes.io/topology-aware-hints=auto` annotation.

When Linkerd receives a set of endpoint slices and translates them to an
address set, a copy of the `Hints.forZones` field is made from each endpoint
slice if one is present. This field is only present if it's set by the
endpoint slice controller, and there are several
[safeguards][topology-aware-hints-safeguards] that Kubernetes checks before
[safeguards][topology-aware-routing-safeguards] that Kubernetes checks before
setting it. To have the endpoint slice controller set the `forZones` field
users will have to add the `service.kubernetes.io/topology-aware-hints=auto`
annotation to the service.
Expand All @@ -40,17 +44,23 @@ node the client is on and limits cross-node and cross-zone communication.

## Constraints

Kubernetes places some [constraints][topology-aware-hints-constraints] on
topology aware hints that you should review before enabling topology aware
hints on Linkerd.
Kubernetes places some [constraints][topology-aware-routing-constraints] on
topology aware routing that you should review before trying to enable topology
aware routing on Linkerd.

The main things to be aware of are:

- Linkerd assumes that traffic to a given zone will be roughly proportional to
the capacity of the nodes in the zone (which can be a concern when using
horizontal pod autoscaling, as the HPA may start nodes in the "wrong" zone);
and
- The Kubernetes endpoint slice controller does not take...
- Topology aware routing assumes that traffic to a given zone will be roughly
proportional to the capacity of the nodes in the zone. This can be a concern
when using horizontal pod autoscaling, as HPA may start nodes in the "wrong"
zone.

- Services with `internalTrafficPolicy` set to `Local` ignore topology aware
routing, by design.

- If you have workloads running on Nodes labeled with `control-plane` or
`master` role, topology aware routing will not route to endpoints on these
Nodes.

## Configuring Topology Aware Routing

Expand All @@ -63,5 +73,5 @@ time="2021-08-27T14:04:35Z" level=info msg="Establishing watch on endpoint [defa
time="2021-08-27T14:04:35Z" level=debug msg="Filtering through addresses that should be consumed by zone zone-b" addr=":8086" component=endpoint-translator remote="127.0.0.1:49846" service="nginx-deploy-svc.default.svc.cluster.local:80"
```

[topology-aware-hints-safeguards]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards
[topology-aware-hints-constraints]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#constraints
[topology-aware-routing-safeguards]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards
[topology-aware-routing-constraints]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#constraints
36 changes: 0 additions & 36 deletions linkerd.io/content/2.13/features/topology-aware-hints.md

This file was deleted.

47 changes: 47 additions & 0 deletions linkerd.io/content/2.13/features/topology-aware-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Topology Aware Routing
description: |-
Linkerd's implementation of Kubernetes topology aware routing enables endpoint consumption based on a node's zone label.
---

Kubernetes clusters are increasingly deployed in multi-zone environments with
network traffic often relying on obscure endpoint matching and routing for a
given service. Linkerd's implementation of Kubernetes ([Topology Aware
Routing][topology aware routing]) provides users with a mechanism for
asserting more control over endpoint selection and routing within a cluster
that spans multiple zones. Users can now implement routing constraints and
prefer endpoints in a specific zone in order to limit cross-zone networking
costs or improve performance through lowered cross-zone latency and bandwidth
constraints.

The goal of topology aware routing is to to provide a simpler way for users to
prefer endpoints by basing decisions solely off the node's
`topology.kubernetes.io/zone` label. If a client is in `zone-a`, then it
should prefer endpoints marked for use by clients in `zone-a`. When the
feature is enabled and the label set, Linkerd's destination controller will
attempt to find endpoints whose `routing.ForZones` field matches the client's
zone.

To get started with topology aware routing take a look at the [enabling
topology aware routing](../../tasks/enabling-topology-aware-routing/) task
documentation.

[topology aware routing]:
https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/

{{< note >}}

Topology aware routing requires that the cluster have the `TopologyAwareHints`
feature gate enabled (which is the default starting with Kubernetes 1.24), and
that Linkerd's `endpointSlice` feature be turned on (this is the default
starting with Linkerd stable-2.12).

{{< /note >}}

{{< note >}}

Starting in Kubernetes 1.31, Kubernetes also has the `trafficDistribution`
feature available as an alternative to topology aware routing.
`trafficDistribution` is not yet supported by Linkerd.

{{< /note >}}
Original file line number Diff line number Diff line change
@@ -1,28 +1,32 @@
---
title: "Enabling Topology Aware Hints"
title: "Enabling Topology Aware Routing"
description: |-
Enable topology aware hints to allow Linkerd to intelligently choose same-zone endpoints
Enable topology aware routing to allow Linkerd to intelligently choose same-zone endpoints
---

[Topology aware hints](../../features/topology-aware-hints/) allow Linkerd
users to specify endpoint selection boundaries when a request is made against
a service, making it possible to limit cross-zone endpoint selection and lower
the associated costs and latency.
[Topology aware routing](../../features/topology-aware-routing/) allows
Linkerd users to specify endpoint selection boundaries when a request is made
against a service, making it possible to limit cross-zone endpoint selection
and lower the associated costs and latency.

There are three requirements to successfully enabling topology aware hints with Linkerd:
There are three requirements to successfully enabling topology aware routing
with Linkerd:

1. `TopologyAwareHints` feature gate MUST be enabled on the Kubernetes
cluster.
2. Each Kubernetes node should be labeled with the
1. The `TopologyAwareHints` feature gate MUST be enabled on the Kubernetes
cluster. (This feature gate is enabled by default in Kubernetes 1.24 and
later.)

2. Each Kubernetes node needs to be assigned to a zone, using
`"topology.kubernetes.io/zone` label.

3. Relevant Kubernetes services will need to be modified with the
`service.kubernetes.io/topology-aware-hints=auto` annotation.

When Linkerd receives a set of endpoint slices and translates them to an
address set, a copy of the `Hints.forZones` field is made from each endpoint
slice if one is present. This field is only present if it's set by the
endpoint slice controller, and there are several
[safeguards][topology-aware-hints-safeguards] that Kubernetes checks before
[safeguards][topology-aware-routing-safeguards] that Kubernetes checks before
setting it. To have the endpoint slice controller set the `forZones` field
users will have to add the `service.kubernetes.io/topology-aware-hints=auto`
annotation to the service.
Expand All @@ -40,17 +44,23 @@ node the client is on and limits cross-node and cross-zone communication.

## Constraints

Kubernetes places some [constraints][topology-aware-hints-constraints] on
topology aware hints that you should review before enabling topology aware
hints on Linkerd.
Kubernetes places some [constraints][topology-aware-routing-constraints] on
topology aware routing that you should review before trying to enable topology
aware routing on Linkerd.

The main things to be aware of are:

- Linkerd assumes that traffic to a given zone will be roughly proportional to
the capacity of the nodes in the zone (which can be a concern when using
horizontal pod autoscaling, as the HPA may start nodes in the "wrong" zone);
and
- The Kubernetes endpoint slice controller does not take...
- Topology aware routing assumes that traffic to a given zone will be roughly
proportional to the capacity of the nodes in the zone. This can be a concern
when using horizontal pod autoscaling, as HPA may start nodes in the "wrong"
zone.

- Services with `internalTrafficPolicy` set to `Local` ignore topology aware
routing, by design.

- If you have workloads running on Nodes labeled with `control-plane` or
`master` role, topology aware routing will not route to endpoints on these
Nodes.

## Configuring Topology Aware Routing

Expand All @@ -63,5 +73,5 @@ time="2021-08-27T14:04:35Z" level=info msg="Establishing watch on endpoint [defa
time="2021-08-27T14:04:35Z" level=debug msg="Filtering through addresses that should be consumed by zone zone-b" addr=":8086" component=endpoint-translator remote="127.0.0.1:49846" service="nginx-deploy-svc.default.svc.cluster.local:80"
```

[topology-aware-hints-safeguards]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards
[topology-aware-hints-constraints]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#constraints
[topology-aware-routing-safeguards]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards
[topology-aware-routing-constraints]: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#constraints
36 changes: 0 additions & 36 deletions linkerd.io/content/2.14/features/topology-aware-hints.md

This file was deleted.

Loading

0 comments on commit a7e6190

Please sign in to comment.