RESAR Storage: a System for Two-Failure Tolerant, Self-Adjusting Million Disk Storage Clusters

Seminar Slides: Download (PDF)

When: Monday, September 16, 2013
Where: PGH 232
Time: 11:00 AM

Speaker: Prof. Darrell Long, University of California, Santa Cruz

Host: Dr. Jehan-Francois Paris

The demand for large-scale storage is greater than ever. The wideavailability of broadband networking has made cloud based storage a vibrant and growing market. Additionally, as we explore exascale high performance computing (HPC) systems with exabytes of data, power considerations become a significant factor. Most existing systems rely on replication to protect user data, maintaining as many as six copies. This high overhead leads to unnecessary costs in equipment, maintenance and energy. While storage appliances using era- sure coding schemes are available, their long rebuild times and lack of continuity of service during rebuild make them unsuitable as building blocks for large scale storage systems.

We present RESAR (Robust, Efficient, Scalable, Autonomous Reliable) storage, a reliable distributed storage volume provider that scales to millions of drives. We implemented our system and tested it on a large-scale emulation platform called Megatux. Our results show that RESAR is capable of scaling to millions of drives, and it’s rebuild performance benefits from this scale by distributing the recovery across many disks. In our emulations, the work of rebuilding a one terabyte hard drive was distributed across 400 disks and completed in less than four minutes with no interruption of service. With an annual durability of 99.999999% and a storage overhead cost of 20%, RESAR has great promise for both exascale HPC and cloud storage.

Joint work with my Ph.D. student Igancio Coderí, Thomas Kroeger of Sandia National Laboratory, and Thomas Schwarz of Universidad Catòlica del Uruguay.

Bio:
Darrell D. E. Long is a Fellow of the Institute of Electrical and Electronics Engineers and of the American Association for the Advancement of Science He is the Director of the Storage Systems Research Center at the University of California, Santa Cruz where he is Professor of Computer Science and holds the Kumar Malavalli Endowed Chair. He received his B.S. degree in Computer Science from San Diego State University, and his M.S. and Ph.D. from the University of California, San Diego. His current research interests in the storage systems area include high performance storage systems, archival storage systems and exascale file systems. His research also includes computer system reliability, video-on-demand, applied machine learning and computer security.

He served as the Vice Chair and then Chair of the University of California Committee on Research Policy. He has served on the University of California President’s Council on the National Laboratories, and on the Science & Technology, National Security and Intelligence committees. He currently serves on the Science & Technology committee for both Los Alamos and Lawrence Livermore National Laboratories. He served on the National Research Council Standing Committee on Technology Insight-Gauge, Evaluate and Review.