Today, I got the privilege to present my team work at USENIX LISA 15. TubeMogul grew from few servers to over two thousands servers and handling over one trillion http requests a month, processed in less than 50ms each. To keep up with the fast growth, the SRE team had to implement an efficient Continuous Delivery infrastructure that allowed to do over 10,000 puppet deployment and 8,500 application deployment in 2014. In this presentation, we will cover the nuts and bolts of the TubeMogul operations engineering team and how they overcome challenges.

During Puppet Camp Paris, I got the privilege to present the Continuous Delivery Workflow of TubeMogul’s Operations Engineering Team. In few years, we went from few servers to over two thousands nodes fully managed by Puppet. In our presentation, we went over the challenges we faced as well as the implementation of our workflow to improve our day to day operation while still moving fast.

With our operations continuous delivery workflow using Git, Gerrit, and Jenkins, we have been able to manage over 10,000 Puppet deployment in 2014.

The International Earth Rotation And Reference Systems Service (IERS) announced that a positive leap second will be introduced on the last day of June 2015 (Official Bulletin C 49) making the day with 86,401 seconds.

In 2012, a similar event created major outages on most of the internet with only few avoiding problems. See this Forbes post from July 2012: +1: Google Aces ‘Leap Second’ While Reddit, LinkedIn And More Went Down Saturday.