Its supposed to be THE day of the year for online retailers: Cyber Monday. For online commerce companies, you plan for increased capacity, fail over plans and so forth. But the best laid plans can always go awry. They did for Bonobos – an online mens clothing retailer with fanatical fans for the men’s pants. It was selling millions of dollars of ‘premium’ pants in a time when CNN and the likes are reporting gloom and doom – who knew that people had money and were spending it on PANTS!
Fast forward to Cyber Monday 2011. Bonobos had expanded their product line and men had been telling their better halves that THIS is what they want for Christmas or Hanukkah or whatever occasion that is coming up for them. People were buying and buying. Then all the plans for the best sales were going awry. The site was chugging, coming to a near halt. Transactions were timing out BUT they were getting charged – over and over and over. It was a bad mix of people spending money and not having their order being recorded – and some getting charged twice or more! Their UX lead spelled it out pretty well in their Quora explanation – of which I read with a sinking heart the week after it happened. It was really, really painful to read as an IT professional. They go into how their commerce system, Magento, just didn’t record the transactions. Oddly, I have worked quite a bit with Magento and this powerful Open Source platform does have its snags – this one I saw quite a bit. But Bonobos had planned for capacity spikes with their hosting partner – Rackspace Cloud and testing load and things seemed to go well. But what happens when you get roughly 10x the traffic to tested to? Little things become big things. And from what was explained on Quora their cache was incorrectly configured – eeks!
But Bonobos has a service motto – People before Profit. Yes they have awesome customer service, but how do you think they did in an epic collapse of their website on the one day it shouldn’t fail? It was all hands on deck and make things right – no matter the cost. The site failed so badly, they just brought it down and started to refund and raincheck everyone – at a cost of millions. But if you follow the twitter stream, people were mad then when Bonobos got back to them customers were amazingly accepting and understanding. This is what I define as a successful failure. I would venture that most companies would TRY to do what Bonobos did but very, very few have the A-players across-the-board that make success a remote possibility.
Bonobos took care of everyone – down to the last customer with refunds, rain checks and credits. The folks from Magento assigned a A-team to help them get out (hint – the power of partnerships matter). Their CTO started to address the resolution from day-one – NOTE it was their CTO’s third week on the job come Cyber Monday. Knowing quite a few CTO’s – I can’t imagine the stress and adrenalin that was flowing through his blood. The feedback from customers, tech gurus and retail magicians has been uniformly the same: #SUCCESS. Now they aren’t out of the woods yet, the site is back up and I think they have a more robust testing plan with some better load testing that goes a bit deeper and more end to end. I have to say that up until this point I haven’t been compelled to go out and buy a pair of their pants – but when my father in law called looking for last minute gift ideas for me I told him: Bonobos.com. From inside the company to the partners, this is a prime example of how to get it done right when things go south: hire right (the best), never settle for less than the best, partner right and have an open dialogue with employees and customers – and stick to those ideals when it actually goes south.
Successful Failure: A Bonobos case study
by Jonathan Distad on January 5th, 2012 | ~ minute read
