Autothrottle: A Practical Bi-Level Approach to Resource Management for SLO-Targeted Microservices


Zibo Wang, University of Science and Technology of China and Microsoft Research; Pinghe Li, ETH Zurich; Chieh-Jan Mike Liang, Microsoft Research; Feng Wu, University of Science and Technology of China; Francis Y. Yan, Microsoft Research
Awarded Outstanding Paper!


Achieving resource efficiency while preserving end-user experience is non-trivial for cloud application operators. As cloud applications progressively adopt microservices, resource managers are faced with two distinct levels of system behavior: end-to-end application latency and per-service resource usage. Translating between the two levels, however, is challenging because user requests traverse heterogeneous services that collectively (but unevenly) contribute to the end-to-end latency. We present Autothrottle, a bi-level resource management framework for microservices with latency SLOs (service-level objectives). It architecturally decouples application SLO feedback from service resource control, and bridges them through the notion of performance targets. Specifically, an application-wide learning-based controller is employed to periodically set performance targets—expressed as CPU throttle ratios—for per-service heuristic controllers to attain. We evaluate Autothrottle on three microservice applications, with workload traces from production scenarios. Results show superior CPU savings, up to 26.21% over the best-performing baseline and up to 93.84% over all baselines.

NSDI '24 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {295485,
author = {Zibo Wang and Pinghe Li and Chieh-Jan Mike Liang and Feng Wu and Francis Y. Yan},
title = {Autothrottle: A Practical {Bi-Level} Approach to Resource Management for {SLO-Targeted} Microservices},
booktitle = {21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24)},
year = {2024},
isbn = {978-1-939133-39-7},
address = {Santa Clara, CA},
pages = {149--165},
url = {},
publisher = {USENIX Association},
month = apr

Presentation Video