During Puppet Camp Paris, I got the privilege to present the Continuous Delivery Workflow of TubeMogul's Operations Engineering Team. In few years, we went from few servers to over two thousands nodes fully managed by Puppet. In our presentation, we went over the challenges we faced as well as the implementation of our workflow to improve our day to day operation while still moving fast.
- Enterprise software company for digital branding
- Over 27 Billion Ads Served in 2014
- Over 30 Billion Ad Auctions per day
- Bid processed in less than 50 ms
- Bid served in less than 80 ms (include network round trip)
- 5 PB of monthly video traffic served
- 1.1 EB of data stored
- Ensure the smooth day to day operation of the platform
- Provide a cost effective and cutting edge infrastructure
- Team composed of SREs, SEs, and DBAs
- Managing over 2,500 servers (virtual and physical)
Five Years Of Puppets!
- 2008 - 2010: Use SVN, Bash scripts and custom templates.
- 2010: Managing about 250 instances. Start looking at Puppet.
- 2011: Started with Puppet 0.25 then upgraded to 2.7 by EOY on 400 servers with 2 contributors.
- 2012: 800 servers managed by Puppet. 4 contributors.
- 2013: 1,000 servers managed by Puppet. 6 contributors.
- 2014: 1,500 servers managed by Puppet. Workflow using Git, Gerrit and Jenkins. 9 contributors. Start migration to 3.7.
- 2015: 2,000 servers managed by Puppet. 13 contributors.