Explore Data Placement Algorithm for Balanced Recovery Load Distribution

Authors: 

Yingdi Shan, Zhongguancun Laboratory and Tsinghua University; Kang Chen and Yongwei Wu, Tsinghua University

Abstract: 

In distributed storage systems, the ability to recover from failures is critical for ensuring reliability. To improve recovery speed, these systems often distribute the recovery task across multiple disks and recover data units in parallel. However, the use of fine-grained data units for better load balancing can increase the risk of data loss.

This paper systematically analyzes the recovery load distribution problem and proposes a new data placement algorithm that can achieve load balancing without employing fine-grained data units. The problem of finding an optimal data placement for recovery load balancing is formally defined and shown to be NP-hard. A greedy data placement algorithm is presented, and experimental results demonstrate its superior performance compared to conventional techniques, with up to 2.4 times faster recovery. Furthermore, the algorithm supports low-overhead system expansion.

USENIX ATC '23 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

This content is available to:

BibTeX
@inproceedings {288754,
author = {Yingdi Shan and Kang Chen and Yongwei Wu},
title = {Explore Data Placement Algorithm for Balanced Recovery Load Distribution},
booktitle = {2023 USENIX Annual Technical Conference (USENIX ATC 23)},
year = {2023},
isbn = {978-1-939133-35-9},
address = {Boston, MA},
pages = {233--240},
url = {https://www.usenix.org/conference/atc23/presentation/shan},
publisher = {USENIX Association},
month = jul
}

Presentation Video