Kyle Guichard and Venus So, Bill.com
In the early days of the pandemic, the Internet was melting as everyone adjusted to working remotely and adjusting to sudden increases in traffic. Remember the days of routine outages and meltdowns of various top brands? We saw a significant increase in customer activity as we helped small businesses manage their back-office processes from their home offices and mobile phones. During this time, we experienced issues with many of our service providers.
However, instead of melting down, we improved our production resilience, safeguarded against customer-facing incidents, and increased our availability. In this talk, we will discuss how we managed to grow our business and improve our availability by focusing on three things:
- Leading a cross-functional culture change towards extreme ownership of production
- Improved resilience in our payments platform with additional redundancy, self-healing, and retries
- Focusing on third-party management and business processes to scale
Kyle Guichard, Bill.com
Kyle Guichard is a VP of Engineering at Bill.com leading the Data Platform and Technology Operations team in the Bay Area. With over 15 years of experience administering mission critical systems from payroll to managing back-office payments, he is passionate about leveraging the cloud and maximizing resilience in systems.
Venus So, Bill.com
Venus is Sr. Director of Engineering at Bill.com leading Digital Payments and Risk teams. She has over 10 years of experience leading mission critical engineering teams from small startups to Fortune 500 companies. At Bill.com, her team went through explosive growth—operational with 130+ countries with $96B+ money moved as the start-up scaled to a successful public company.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Kyle Guichard and Venus So},
title = {Soft Failures, Hard Goals - Accelerating Payments at Scale During the Pandemic},
booktitle = {SREcon20 Americas (SREcon20 Americas)},
year = {2020},
url = {https://www.usenix.org/conference/srecon20americas/presentation/guichard},
publisher = {USENIX Association},
month = dec
}