IstioCon 2023 - Virtual

Optimizing resource provisioning and performance in large, multi-cluster service meshes is difficult. Cluster schedulers/autoscalers assist, but a series of complementary problems lie at finer granularities after the containers have been provisioned: microbursts of requests, resource interference, tail latency in dependent services, high bandwidth cost.
IP networks have long developed techniques of optimizing these traffic flows - an area known as Traffic Engineering. We present an analogue at the Service level - Service-Layer Traffic Engineering (SLATE) - a system that automatically optimizes the flow of requests
in these service meshes. This system globally optimizes request routing, minimizing request latency & bandwidth cost based on administrator intent. We show that using methods like call graph and compute cost prediction, SLATE better reacts to sudden changes in service load, can prioritize certain requests over others through the mesh, and more.