Phoenix Project: A Novel about IT, Devops, and Helping Your Business Win

von Gene Kim

Auf Amazon anschauen (opens new window)

  • Situations like this only reinforce my deep suspicion of developers: They’re often carelessly breaking things and then disappearing, leaving Operations to clean up the mess.

  • A half hour later, we finally write on the whiteboard: “a ‘change’ is any activity that is physical, logical, or virtual to applications, databases, operating systems, networks, or hardware that could impact services being delivered.”

  • But we’ll keep trying things until we have a system that works,

  • Any improvement made after the bottleneck is useless, because it will always remain starved, waiting for work from the bottleneck. And any improvements made before the bottleneck merely results in more inventory piling up at the bottleneck.”

  • But I learned most of this already in business school. I don’t see how this could possibly be relevant to managing it Operations. it is not like running a factory.” “Oh, really?” he turns to me, frowning intensely. “Let me guess. You’re going to say that it is pure knowledge work, and so therefore, all your work is like that of an artisan. Therefore, there’s no place for standardization, documented work procedures, and all that high-falutin’ ‘rigor and discipline’ that you claimed to hold so near and dear.”

  • “Your job as vp of it Operations is to ensure the fast, predictable, and uninterrupted flow of planned work that delivers value to the business while minimizing the impact and disruption of unplanned work, so you can provide stable, predictable, and secure it service.” Listening to him, I wonder if I should be writing this down.

  • “The First Way helps us understand how to create fast flow of work as it moves from Development into it Operations, because that’s what’s between the business and the customer. The Second Way shows us how to shorten and amplify feedback loops, so we can fix quality at the source and avoid rework. And the Third Way shows us how to create a culture that simultaneously fosters experimentation, learning from failure, and understanding that repetition and practice are the prerequisites to mastery.”

  • He continues, “Remember, outcomes are what matter—not the process, not controls, or, for that matter, what work you complete.”

  • “When you brought up the word commitment, it reminded me of something Erik asked me last week that stuck with me,” I say. “He asked on what basis do we decide whether we can accept a new project. When I said that I didn’t know, he took me on another tour of mrp-8 manufacturing plant. He took me to Allie, the Manufacturing Resource Planning Coordinator, and asked her how she decides on whether to accept a new order.” I flip back to my notes. “She said that she would first look at the order and then look at the bill of materials and routings. Based on that, she would look at the loadings of the relevant work centers in the plant and then decide whether accepting the order would jeopardize any existing commitments.

  • “Obviously,” he continues, “every work center is made up of four things: the machine, the man, the method, and the measures.

  • ‘Improving daily work is even more important than doing daily work.’ The Third Way is all about ensuring that we’re continually putting tension into the system, so that we’re continually reinforcing habits and improving something. Resilience engineering tells us that we should routinely inject faults into the system, doing them frequently, to make them less painful.

  • “Sensei Rother calls this the Improvement Kata,” he continues. “He used the word kata, because he understood that repetition creates habits, and habits are what enable mastery. Whether you’re talking about sports training, learning a musical instrument, or training in the Special Forces, nothing is more to mastery than practice and drills. Studies have shown that practicing five minutes daily is better than practicing once a week for three hours. And if you want to create a genuine culture of improvement, you must create those habits.”

  • “A critical part of the Second Way is making wait times visible, so you know when your work spends days sitting in someone’s queue—or worse, when work has to go backward, because it doesn’t have all the parts or requires rework.

  • Patty says, “What that graph says is that everyone needs idle time, or slack time. If no one has slack time, wip gets stuck in the system. Or more specifically, stuck in queues, just waiting.”

  • “I’ll tell you what a good day feels like. It feels like the end of the year when we’re beating the pants off the competition—when we haven’t closed the books yet, but everyone knows that it’s going to be a monster quarter. All the sales people would have hit quota, and the ones on top would have hit their accelerators. A good day feels like my staff panicking about the size of the commission checks we’re going to be writing them. “I’m not worried, because those big commission checks would mean the company is making money,” he says, smiling even more broadly. “Steve would be excited to announce to Wall Street and the analysts how well the company is performing—all made possible because we had a winning strategy, and also because we had the right plan and the ability to operate and execute. It means that we got all the parts of this organization to click as a team and win.

  • CFO GOALS Health of company Revenue Market share Average order size Profitability Return on assets Health of Finance Order to cash cycle Accounts receivable Accurate and timely financial reporting Borrowing costs

  • “It can do anything you want,” I reply, smiling. “That’s a big magic wand,” she says, laughing. “I want accurate and timely order information from our stores and online channels. I want to press a button and get it, instead of running it through the circus we’ve created. I’d use that data to create marketing campaigns that continually do a/b testing of offers, finding the ones that our customers jump at. When we find out what works, we’d replicate it across our entire customer list. By doing this, we’d be creating a huge and predictable sales funnel for Ron. “I’d use that information to drive our production schedule, so we can manage our supply and demand curves. We’d keep the right products on the right store shelves and keep them stocked. Our revenue per customer would go through the roof. Our average order sizes would go up. We’d finally increase our market share and start beating the competition again.”

  • “The longer the product development cycle, the longer the company capital is locked up and not giving us a return. Dick expects that on average, our r&d investments return more than ten percent. That’s the internal hurdle rate. If we don’t beat the hurdle rate, the company capital would have been better spent being invested in the stock market or gambled on racehorses.

  • As Erik keeps reminding me, a great team performs best when they practice. Practice creates habits, and habits create mastery of any process or skill. Whether it’s calisthenics, sports training, playing a musical instrument, or in my experience, the endless drilling we did in the Marines. Repetition, especially for things that require teamwork, creates trust and transparency.

  • We make it to the plant in record time and are now standing on the catwalk. He begins the speech that I’ve been expecting. “A manufacturing plant is a system. The raw materials start on one side, and a million things need to go just right in order for it to leave as finished goods as scheduled out the other side. Everything works together. If any work center is warring with the other work centers, especially if Manufacturing is at war with Engineering, every inch of progress will be a struggle.”

  • “In manufacturing, we have a measure called takt time, which is the cycle time needed in order to keep up with customer demand. If any operation in the flow of work takes longer than the takt time, you will not be able to keep up with customer demand.”

  • “We studied all the works of Sensei Taiichi Ohno, Steven Spear, and Mike Rother.

  • “Oh, really?” he says, deadpan. “Let me tell you a story. Back in 2009, I was a board director at a technology company, where one of our engineers went to the Velocity Conference and came back raving like a madman, full of dangerous, impossible ideas. He saw a presentation given by John Allspaw and his colleague Paul Hammond that flipped the world on its head. Allspaw and Hammond ran the it Operations and Engineering groups at Flickr. Instead of fighting like cats and dogs, they talked about how they were working together to routinely do ten deploys a day! This is in a world when most it organizations were mostly doing quarterly or annual deployments. Imagine that. He was doing deploys at a rate one thousand times faster than the previous state of the art.