Hey @tanujd11 from a user perspective can you share what you like to happen on the data plane ( from gateway to multiple backend endpoints with different topology info )

I understand this is very useful for optimizing East West traffic within a cluster, is that also the case for north South ?

I think for an Envoy gateway running in us-east-1/us-east-1a should prefer the same zone backend to prevent cross zonal traffic. I think this behaviour could be made as default as cross zone communication is obviously costly. WDYT?

thanks, here's something more to think about

• opt in or default - K8s service has a opt in annotation https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/#enabling-topology-aware-routing service.kubernetes.io/topology-mode
• Is this a soft preference (weighted) or hard preference (priority)

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

there's a new field in the Service spec (trafficDistribution.preferClose) https://kubernetes.io/docs/concepts/services-networking/service/#traffic-distribution that we could consider using to automate priority amongst endpoints within a Service

there's a new field in the Service spec (trafficDistribution.preferClose) https://kubernetes.io/docs/concepts/services-networking/service/#traffic-distribution that we could consider using to automate priority amongst endpoints within a Service

Could be an option when this new field is stable and corresponding K8s version is adopted by massive companies.

Before that, IMO it's better to do load balancing accross endpoints in the cluster via Envoy's capability.

Currently EG has implemented locality weighted load balancing ¹, one BackendRef is translated to one LocalityLbEndpoints.

locality := &endpointv3.LocalityLbEndpoints{
	Locality: &corev3.Locality{
		Region: fmt.Sprintf("%s/backend/%d", clusterName, i),
  	},
	LbEndpoints: endpoints,
	Priority:    0,
}
  
// Set locality weight
var weight uint32
if ds.Weight != nil {
	weight = *ds.Weight
} else {
	weight = 1
}

Actually endpoints inside a LocalityLbEndpoints may be running in different zone, cross zone cost can't be saved in this way.

Through Envoy's capability, priority levels ² or zone aware routing ^{3 4} can archive the goal to save cross zone cost.

priority levels

1. Backend endpoint should be set with correct zone, it can be retrieved from EndpointSlice, inherited from Node topology.kubernetes.io/zone label.
2. Envoy's command options should be set with --service-zone option, means which zone Envoy Pod is running in.
3. EG rearranges EDS resources for each Envoy, if Envoy and Backend endpoint are in same zone, priority as 0, else 1.

zone aware routing

This approach is mutually exclusive with locality weighted load balancing, since in the case of locality aware LB, we rely on the management server to provide the locality weighting, rather than the Envoy-side heuristics used in zone aware routing.

1. Backend endpoint should be set with correct zone, it can be retrieved from EndpointSlice, inherited from Node topology.kubernetes.io/zone label.
2. Envoy's command options should be set with --service-zone option, value meaning which zone Envoy Pod is running in.
3. Envoy's bootstrap config should be set with cluster_manager. local_cluster_name, means which fleet Envoy Pod belongs to, it will be irKey in implementation.
4. Add cluster corresponding to cluster_manager. local_cluster_name to CDS resources.
5. Design a mechanism to discover Envoy Pods belongs to cluster_manager. local_cluster_name as endpoints and add them to EDS resources.
6. Both Envoy and Backend cluster are not in panic mode ⁵.

personal preference

Since step 1 and 2 is required by both, priority levels can work with implemented locality weighed load balancing, but zone aware routing can't. Apparently priority levels are easier to implement. But it requires EDS resources should be arranged in xds/cache module for individual Envoy. No matter EG do this, or create new xDS Hook API, like PostEndpointModify(ClusterLoadAssignment, Node) which allow extension server to do this.

thanks for outlining the steps @aoledk ! we currently have #3055 open to get explicit priority per backendRef and program that into the xds cluster resource.

In the future, we can use this issue to make sure we track the auto priority work, the field in k8s preferClose could be the knob for users to say they want to opt in to this feature

Hi @aoledk, regarding:

priority levels
[...]
EG rearranges EDS resources for each Envoy, if Envoy and Backend endpoint are in same zone, priority as 0, else 1.

Is this option viable? Can our XDS server produce different EDS for different envoy pods that are part of the same Envoy deployment?

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

Hi @aoledk! Are you still looking into implementing it yourself? If not, I’m interested in this feature and can work on bringing it to life.

@flyik recently I'm busy with bringing in EG, you can go ahead.

@flyik I've unassigned myself, you can assign to yourself.

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

keep

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

keep

Is anyone working on this who would like to collaborate? I'm interested in support for both Traffic Distribution and Topology Aware Routing

thanks @jukie assigning this one to you.
Rethinking #1909 (comment), we may need to make this opt in, using a field such as 'ZoneAwareRoutingEnabled' in the EnvoyProxy resource

@arkodg wouldn't opting in via the Kubernetes Service annotation (service.kubernetes.io/topology-mode: Auto) be a better place so that it can be configured per-service?

Updating getIREndpointsFromEndpointSlice() and DestinationEndpoint should be pretty simple to include topology info from Kubernetes but a topic with more complexity would be how individual Envoy instances should discover their locality zone. Before I get started, any preferences on the approach there?

The approach I thought of was to include the --service-zone arg for the Envoy proxy deployment. It would default to --service-zone $SERVICE_ZONE and we'd inject this env var separately through an init container or mutating webhook. In either setup it'd fail open for an empty env var resulting in no zone being set and falling back to current logic.

I think there are two parts here

1. Enabling settings in envoy proxy to enable the zone aware routing feature
2. Enabling zone aware routing per backend service using trafficDistribution.preferClose

I was referring to 1. to be opt in, in case setting those fields has an impact on common case performance, and might affect users not interested in the feature, and if it isn't, default is fine

Hi! Will there be an option to choose between Zone-Aware Routing and Locality-Weighted Load Balancing in the future? It looks like the approach with priorities has a lot to offer—starting with more than one "layer" for balancing (region/zone/subzone vs. just zone in the case of ZAR), the ability to manually set fallback locations, and the option to specify weights across them.

@flyik yes, my PR is still very much WIP but the end result won't hardcode one or the other like it does currently.

From the contributors call it sounds like there's a strong preference against adding a Mutating Webhook so I was curious if anyone has another suggestion for how to handle zone discovery from the envoy proxy instances. An initContainer is the other option I had in mind.

The challenge is that we can't use downwardAPI to share underlying node topology labels and inject as an env var for example so we'll need something else to populate the --service-zone arg or bootstrap configuration.

thanks for digging into this @jukie, +1 to the init container approach of injecting an env var and reusing it in the proxy container

Is anyone interested in owning that piece of work? That can happen separately from my PR and would amount to the goal of getting the envoy proxy instances launched with --service-zone=$SOME_ZONE or --service-zone=$(cat /some/shared/file/from/init)".

sure @jukie feel free to create a GH issue or subtask for it, and someone else from the community can pick that piece up