Tag Archives: agile project management

Whatever Happened To Software Engineering?

map-clip-artI’ve been working some on my mailing list to help Agile teams do better and one of the recurring topics from readers has been how to fit “normal” software engineering principles into an Agile framework. How can you do database design, for instance, if every 2 weeks the database is actually being used?

Of course there’s an easy answer: we incrementally do things in Agile. So, for instance, you might use all those database design skills, but a little at a time, instead of all at once. The first sprint you sketch out the tables, maybe add a few fields. Second sprint you change up the tables some, add some more fields. Third sprint you’re working a bit with cardinality. And so on. The concept is that we do the same stuff, only a bit at a time instead of becoming a bottleneck.

This is a good enough answer, and it works for most stuff. But I think it’s also masking a conflict that many people have: software is supposed to be engineering, not just little dips and drabs of stuff added in here or there. To hear many YAGNI pundits and others, you don’t actually do anything until the last minute, and then only in support of the exact thing you’re working on. Most engineering disciplines, however, encourage you to solve the entire problem. If you’re building a bridge, architect the bridge first, dang it. If you’re going to the moon, you’d better have some technical work around the entire trip. You don’t just launch into orbit and figure it out from there. Building incrementally is fine. But there exists fields of inquiry where there’s a long sequential process of refinement over the entire problem domain — one in which you gain execution advantages by working the entire problem at once. We seem to either have tossed this fact away or are purposefully ignoring it.

I began noticing a problem. When we talk about these things, people shut down. I suspect many folks in software engineering, especially Agile coaching, don’t have an engineering background! This can lead to a severe disadvantage when dealing with certain areas:

  • Modeling I don’t see many teams sketch, much less model. That’s a shame, because visual information is a much more effective way to discuss technical matters. Add in a bit of formal training, say 30 minutes, and teams can sketch in UML. Then you can link diagrams. There’s something to be said for lightly sketching problems on the whiteboard. Take a picture if you want to keep it. Start using a modeling tool if you realize that you’re having a group discussion around highly technical stuff. You can sketch, you can use UML, you can use a modeling tool, all without having to become a waterfall BDUF project. Really, you can. It’s the only way to go for complex projects.

  • Process Analysis Does anybody remember structured process analysis? Not to see most teams. The way most people teach Scrum and Agile is that a list of stuff appears — I guess from the sky, brought down by a dove to the Product Owner. The team only works on the stuff in front of it. Don’t spend a minute thinking about the big picture! After all, your project could end at any point in time, and you don’t want to spend one minute on things that you’ll never need.

    Of course, there ARE projects that could end at any moment, but for most of us, we’re brought on to address some kind of system: a website, a business problem, an internal need, and so on. The team, and project, has some cohesion. You’re going to be here in six months, and you’re going to be working on the same thing. For those kinds of projects, spending some time doing process analysis is a no-brainer. You get back much more than you put in. I’m not talking about anything waterfall-like. I’m simply talking about building a process model, over time, of who does what with the system and why. This can help you cut to the chase and decide which stories should be prioritized first. It can help you define and have a common understanding of your stories. It can help the Product Owner make economic decisions about the backlog, and it can help the team interject creativity into the solution, instead of just being a bunch of order-takers. Great stuff for couple hours or so each sprint.

  • Lean Startup Another one of those end-to-end, outside-the-team areas where the team and the Product Owner need to work. What hypotheses is this work supposed to be testing? What are the revenue streams? How are we addressing the gap between market and PO? In a startup world, all these things are critical — and there’s a sequential engineering-type endeavor that can take you from point A to point B. It’s a mistake not to do it.

When we teach things like good architecture or proper database design, many times we view the developer as the center of the universe and the engineering practice as something that’s completed in toto before any other work can occur. This has caused tremendous pushback from Agile teams, rightly so, because it creates handoffs, bottnecks, and unecessary documentation. But are we overreacting the other way? Isn’t there a place for long-format, sequential, detail-oriented work, even in Agile teams in complex domains? Especially in Agile teams dealing in complex domains?

Each of these has a few areas in common: 1) they’re about more than the work directly in font of the team, 2) they’re about having the experts, the team, assist the organization in its work, 3) they’re sequential, 4) they build on themselves as work gets done, 5) they’re not the work itself, and 6) they end up discovering things for the PO and the organization that wouldn’t be discovered otherwise. Because of these attributes, they tend to “fall between the cracks” when teams adopt Agile practices. They are traditionally part of what people employ engineers to do.

How about you? Are there engineering processes you miss seeing in your Agile teams? What are you doing to make up for it?

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

It’s Not Always a Narrative

I was reading a commentary on financial issues the other day that was extremely poorly-written. It wasn’t that the author couldn’t write. He had a great ability to evoke emotions. It wasn’t that his topic was bad. He was trying to explain why the economy was doing so poorly.

His problem was that he wanted it all to fit into a narrative.

We humans do that. Whatever problem we’re trying to work through, we want a narrative to explain it to us. Even in our personal lives, we have a deep desire for it “to just all make sense” — meaning that we can connect events in our life in some kind of greater meaning.

The problem is, many times when dealing with thousands or millions of independent agents each making the best decision they can, it doesn’t necessarily fit into any narrative at all. It just is.

I see flaw many times when people describe marketing to kids in college. Marketing is some great evil force out to control their mind. It has desires, needs, goals, and it reacts when things happen.

These are all not just examples of anthropomorphism — treating some random object as if it were a person. These are examples of trying to make some external system fit into some simpler narrative that we can understand and relate to. We try to make stories out of everything.

This is especially bad in business. Nobody wants to know that a random bunch of really smart people, executing adaptively in a certain context, created a unique machine that made a lot of money for a short period of time (usually until conditions changed and the team was unable to adapt) But everybody wants to read a gripping narrative about how Steve Jobs, or Bill Gates, or some other guy was able to beat adversity and champion a killer idea and maintain a vision until he was rewarded.

We love a narrative.

Business doesn’t work like that, though. Yes, you can sell books like that, but that’s not the way business works. In the startup community, we even have a name for these TV shows and books that glorify some hero working through adversity to make it happen — business porn. The idea is that the average reader comes by, takes a look for a few minutes, is suitably and superficially stimulated, then moves on to other things. Nothing of lasting value was exchanged. If anything, in many consumer’s minds, an unrealistic expectation was created of how things work.

One of the neat parts about helping technology teams is watching how really smart people solve problems. We tend to do this in narratives. One of the reasons user stories are so powerful is that they play into this natural tendency.

Narratives can be powerful tools for solving problems. But they also can create their own problems if used incorrectly. We need to be able to leverage our natural proclivity to narratives without being overcome by it.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Why your programming skills suck at project management

I was working with an Agile program doing planning last week, and we were all busy creating our release plans and beginning to scope out our product backlog. Every team was using big visible indicators of work, usually post-it notes and blotter paper.

Except one. As we started socializing our work, talking about dependencies and identifying opportunities to refactor, one team came up and presented their backlog on the projector using a spreadsheet. Everybody stopped while one person went down each line, item-by-item, for 100 items.

This kind of meeting, where somebody grabs a projector and folks stare at a wall for hours, is one of my pet peeves. But these guys were new and I saw it as an opportunity to coach them later on. I timeboxed their presentation to limit the damage, then we all managed to complete.

At the end of the day, when we were getting feedback, I got quite a surprise! The group who didn’t physically work with their backlog thought that all this post-it and physical stuff was so much rework. Why not everybody just complete their work ahead of time and we could all show it on a projector? Ten teams, all with hundreds of items on spreadsheets to go down line by line, all day long.


They also thought the 3 hours we spent doing the same work took too long! Go figure.

I had to come up with a respectful way to tell them that this is a bad idea.

“You know, the way you think about backlogs, a big hunk of data to be created and managed, is not actually the way they work. The wonderful skills you have as a programmer or analyst are actually not applicable here. The goal here is not to work with the data, it’s to work with what is between your ears. Your brain. We are optimizing the transfer of information using humans, not computers. That means that there’s a whole new set of rules to deal with.”

I don’t think they got it, but most of the other teams knew what was going on and had my back. So it worked out.

But there is a teachable moment here. As technologists we are used to moving data around on machines. This is a completely different universe from optimizing the behavior of people to understand and attack a problem. People are not computers.

There are a lot of little consequences from this. For instance, people can only compare so many items at a time, so we usually limit comparisons to 2 or 3 items. People can only remember so many items in a meeting, so we limit our lists to 20 or 30 items. People read more by body language than by words, so we put them physically together while planning. People are more likely to make good tough decisions early in the day, before they reach “decision fatigue”, so we front-load the work. And so on.

Some of this stuff, of course, is unproven pscyho-babble. Some is not. What the Agile community has been doing is empirically testing these various theories to see which ones work well and which ones do not. As practitioners we have a big list of “games” and principles the games are built on. We add some in, see how the team reacts, then adapt. That’s why Agile is different from other methodologies. Agile is a marketing term around best practices for iterative and incremental development. That vagueness is its strength, but it can also drive newbies nuts. It is entirely different from the kind of yes-no, off-on boolean logic most of us deal with in our jobs everyday.

Or to paraphrase a famous presidential campaign, it’s the people, stupid.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Trusting the System

I remember when I first learned and really grokeed Object-Oriented Analysis and Design. I had read a bunch of books. I had even designed and coded some systems. Some had went well, but some, well, were a bit more complex than they needed to be. See “Architecture Astronaut

Then one day I picked up use cases. Not the god-awful templated 30-page nightmare that most companies make out of use-cases; the real deal. trigger-actor-goal-context process analysis. The kind you can do on a napkin. Maybe sketch something on the back of it if you’re having problems. Specifically, how process analysis fits into Object-Oriented Analysis. Process analysis tells you what needs to happen. OOA tells you what are the logical pieces to make it happen. Semi-structured, all from plain English.

I realized at that point that both OOA and process analysis helped me do one very important thing: I no longer had to keep the entire problem in my head at one time. I had a system that I could use to pick up any problem and drive it towards some kind of resolution. It might be an ugly resolution, but I could get there. I didn’t have to be a genius to solve tough business problems. I just had to follow the system.

I had a similar experience with Agile. As a young PM, I had a desire to plan out everything my team was doing. After all, the more I planned, the less risk there was, right? How could you do anything truly complex and important without having a detailed plan of how you were going to attack it?

At the time, I hated making developers suffer, so I held back. I would secretly plan a bit out ahead in order to make myself feel better, but unless we needed it, I let the team do their work. This wasn’t out of any feelings of comfort. Heck, I spent most of my time as a PM worried about the 17 thousand ways things could go wrong. It was the mid 90s, and we were charging tons of money to solve complex problems. Lots could go wrong.

But what I found out over time, as I adopted more and more “Agile” practices without even knowing it (“Agile” didn’t exist at the time), was that all of my planning was really just a Stupid Human Trick. I felt that by treating people like computers, by programming out everything that was going to happen, that somehow I was reducing risk. But what really happened was that people would always find the problems I missed, and if I left them alone, would go fix them. Turns out people were pretty good at identifying and structuring their own work as they went along. Much better than I was by myself, doing it for them.

Eventually I learned there was a system. There was a minimum amount of structure that I could (had) to provide for teams to succeed. As long as I did that, they would always rock and roll. My job as a PM (and later SM) was to get obstacles out of the way, provided the most minimum amount of structure necessary, and watch the magic happen.

I didn’t have to be a genius to solve tough technology development problems. I just had to follow the system.

As a coach, when I explain these and other concepts like them to folks, many times I get the “crazy uncle” smile. Yes, Daniel, they say, of course it’s all that. (Meanwhile wondering how much I am billing and how did we get stuck with this nice, but misguided guy who thinks the world works on magic)

I think it’s one of those things you have to see to believe. Once you tackle a super-complex business problem using simple tools, you realize you can do this over and over again. Just trust the system. Once you see a team solve a ton of development problems while delivering a product the customer wants, you realize you can do this over and over again. Just trust the system.

Yes, we take this too far. Sometimes people don’t really understand that there is a balance between structure and chaos. Sometimes they want to say silly things like “It’s emergent” to describe conversations and processes they don’t understand. When you see the magician do a few tricks, you might start thinking he can do anything. He cannot.

But most people have the opposite problem. They come from school, where you had to understand the complete problem and solution to get a grade. They want to do well where they are, so they want to be able to describe just how each piece of the process works to create a solution. It is completely counter-intuitive to think that by doing simple things in a system, you can achieve enormously complex goals. But for many of the really cool things in life, that’s how it works.

Trust the system.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Who Moved My SMEs?

Working in a team in a large organization? Having to use SMEs? What’s the best way to do that?

SMEs (Subject-Matter Experts, pronounced “Smees”) are an integral part of any large group of developers. They are the guys who know a bunch about how a certain thing operates and are helping the team dealing with it.

Many times they have a special role and title. A Project Manager is an example of a SME — sometimes PMs have multiple teams, so they “visit” each team and help them. A DBA could be a SME, as could an expert in some business process. SMEs know so much they have to get spread around.

SMEs might not have deliverables. Maybe you need the online marketing expert to sit in on team planning sessions while you build-out a new website. They’re spending 2 hours a week with you, but you don’t get a new widget when they’re done.

The problem for Agile teams is twofold. First, is this person on your team or not? From a pigs-and-chickens standpoint, obviously they’re part of the gang. The entire reason for them to be there is to speak up and participate as much as possible. But from a team commitment standpoint, the most they can do is provide feedback on whether they think stories can be accomplished or not. So the answer is “it depends”. They’re a little of both. Pigkens. Chickpigs.

Weirder still is the viewpoint from the SMEs standpoint. So you have a team of specialized experts. They are tasked out to dozens of teams all over the organization. Is this an Agile or Scrum team?

To be more specific: does this team make group commitments to clear, testable deliverables inside of fixed timeboxes? Can they predict when any item on their backlog will be delivered? Is there a Product Owner that prioritizes work and elaborates on how it is to be completed?

I think the answer is no, they are not a Scrum team in any kind of traditional sense. They can certainly use many Agile practices, though, like stand-ups, paired-work, 40-hour-work-week, and so forth.

They also can certainly use Kanban to track and somewhat prioritize their work. Perhaps over a long period and lots of data, you can even start to spot and predict patterns.

The bigger question is: do you really need SMEs? I think in a large organization you end up having them one way or another — it’s just the way the math works out. If you are employing 1,000 people, while 90% of them might be perfectly cross-trained, I can guarantee you that 100 or more are going to be there especially because of a targeted knowledge area that they have mastered that is not easy to share. I’m all for making as many things as possible into skillsets; things like project management, database work, or user interface design. Teams should have various skillsets, and organizations should manage and encourage cross-training.

But as noble of a goal as that is, you just can’t do it with everything, especially in BigCorps. This is the difference between how we’d like the world to act and how it actually is. We run into this in the Product Owner role quite often.

SME’s, and SME groups, are probably here to stay. The question now is: are we managing them as efficiently as we should?

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

The Perfect Agile/Scrum Stand-up

When people first arrive in Agile/Scrum, one of the first things they ask is “Doesn’t Agile mean we don’t need requirements?” followed by a close second of “Scrum is too rigid. We definitely don’t want something so restricting”

It seems like you’re either a cowboy or a Nazi. There are no in-betweens.

It’s like this: Agile is all about dynamic process tailoring. That means, yes, that teams control the degree of paperwork and process they need to accomplish. But it’s also like this: there are some “recipes” that the community has found over and over again that work. You’d be a complete fool to ignore them.

What happens with high-performing teams is that they have very few things they do that are defined and reproducible. But for those very few things, they are very disciplined. Being Agile, strangely enough, means being extremely disciplined about a very few things and leaving the rest of the details to make up as you go along.

The crux of all this ambivalence is, in my opinion, the stand-up. Since nothing proves a point like actually seeing it, in the video that follows, note the following things:

  • It happens quickly. Stand-ups are high-energy. People are standing-up, moving around. Nobody is sitting in their chair with their head in their notebook or cellphone.
  • Nobody is reporting to anybody. This isn’t a status report. There are no questions and answers.
  • We point at the things we are doing. It’s all about Visual Process Control. That means the things you are doing are publicly visible, they represent status at any time, and they are manipulable.
  • Decisions are made after the stand-up. The stand-up is a game of identifying tokens: what has happened, what is going to happen, what obstacles individuals are facing. After the tokens are all identified, the “real” work begins: the tokens are resolved. Technical discussions and ad-hoc discussions begin.
  • There is no Project Manager. If you can look at a stand-up and find a project manager, it’s a bad stand-up. Stand-ups are about team commitments, risks, and issues. Not roles.
  • Everybody is in the same room. Body language and other non-verbal communication channels count. A lot. Everybody isn’t standing around talking into a PolyCom.
  • It’s scalable. If done correctly, team stand-ups scale very easily. A manager could attend several stand-ups each day, get a close-up view of how the entre program is going, and still have most of the day free for coordination and planning activities.
  • The team is small. You can’t have an Agile team with 30 people on it. Do not do this. It does not work.
  • The story board is not that complicated. This is about the limit of complexity for a story board that I am comfortable with. Past this is drudgery. If your storyboard has the colors of a Christmas tree and a sufficient number of post-its to draw a 40-foot picture of Godzilla, you’re over-complicating things. Stop that.
  • It’s a problem-solving exercise. Immediately after the stand-up is over, the Product Owner talks to anybody with questions or problems and works them out.
  • We work out problems visually. Notice how they pull the story/task over to the flip-pad and work through the problem? See how the options and structures are all presented visually? Here’s a tip: doodling or drawing structures and flow on paper has about a hundred times more effectiveness that trying to write it down or vocally discuss it. Model things.
  • Big Picture is not present. Nobody is looking at long-range issues. Nobody is talking about things happening 4 months from now. Nobody is tapping their foot and asking “are you through yet?” The entire thing works on local optimization. That means that inside of sprints teams locally optimize to finish their commitments. It’s only at the boundary of sprints — demos, retros, and IPAs — that the big picture is important.

Finally, notice it doesn’t match the video. The perfect Agile/Scrum stand-up demonstration does not exist. Teams are different. Everybody has to adapt. As I said in the beginning, Agile is all about adapting. The point is to try things the “best” way — as the community has determined them — and then adapt for your particular situation. That way you don’t miss out on anything, and you also have good reasons for the way you’ve adapted.

Want to rock out your Agile team? Here are the e-books for you.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Tyranny of the Tools

As an agile coach, it’s been both a privilege and a pain to watch groups of extremely smart people blow themselves up.

The sad thing is that you can explain how this is all going to happen, you can plead with them not to head down this path, yet they do it anyway. What could have been a kick-ass team instead becomes a boring slog down old project management road.

And it all begins with automation.

That may sound strange. After all, we’re technologists, damn it. Our job is to take repetitive manual jobs and make the computers do them, right?

When I train people, we use post-its and index cards. We use painter’s tape on a wall. If we have a token for work that is to be performed — a story or a task — we jot a title down and plop it on the wall. This is called visual process control. It’s the result of decades of research and experimentation.

Or to put it bluntly, when my wife wants me to mow the lawn, she can ask, she can email, or she can make a bright red note and stick it on the refrigerator where I have to look at it every time I get a snack. Guess which one gets results. Make me look at it — even when I’m doing something else — and I’ll figure out a way to make it go away.

We find that things we look at — even in a casual, off-hand manner — are the things we think about. We also find that as we see things, we work on them in our mind in the background. We make a lot more progress letting our subconscious mind work on things while we’re busy with more important stuff, like technology development.

Still, there’s nothing like the gleam in a programmer’s eye the first time he sees a story/kanban board and physical risk/issue list. Note cards! On a wall! It’s like 1960 or something in here!

Then the gears start turning. Watching, I feel a terrible sadness, as if thousands of man-hours all screamed out at once, then were suddenly silenced.

There’s a reason why there are 40-thousand software implementations of Scrum/Agile on the web. Everybody who’s ever sat in a team room looking at the wall wanted to “fix” it.

Obviously a spreadsheet could handle all these lists and totaling up of things. And it’s not too hard to make a replica of a storyboard using some HTML5 and AJAX tools. Heck, given a few days, we could make a thing of beauty out of this — and make it easier to use too! Hell, I could be adding and completing stories on my iPhone! Taking some old, cranky, ad-hoc system using bubble gum and bailing wire and replacing it with a database and a decent front-end? This is the entire point of modern life! This is what we do! Why suffer?


So then — if we’re lucky — somebody does up a spreadsheet. Then everybody has to update the spreadsheet. Notice how the focus has moved. Instead of sitting in a room doing other things while the status of our project beckons us on the wall, we’re sitting in our cubicles filling out a spreadsheet once a day. The focus is on remembering to update a tool, not thinking about the status of our project. The tool starts running the project. It becomes the central source for finding out things. Not the people.

If we’re not lucky somebody writes a check for a more “powerful” tool. “Powerful” Agile tools usually have all sorts of fields, checkboxes, and whiz-bangs. They promise all kinds of benefits for teams — track your actual time! Track code changes against tasks! Roll up dozens of projects in a single bound!

So the job becomes completing the tool. Stand up? Well, that’s a reason to verify the tool has everything updated. Sprint closing? A detailed check to make sure the tool is in sync. Retrospective? A time to talk about updates to the tool. Do we need new fields? New reports?

It’s not us working the project. It’s us working the tool. Meanwhile, we still have the project to deliver. Agile becomes just another way to micro-manage and control.

Of course, I’m no Luddite. I love tools — as a means to communicate. There’s no better way to look at a team from a distance (when you can’t actually walk into their team room) than some sort of easily accessible online tool. So the team still works like it always has, using bubble gum and bailing wire, and each day somebody takes the stuff on the board and transfers it to a tool. The tool is never used except my remote workers as a way to visit the team room.

This is all just extreme fantasy on my part, of course. I’ve never seen a company that bought a tool that didn’t end up servicing the tool instead of actually being Agile. But I continue to hope.

There are people who love tools. These are the people who build them, sell them, or have a full-time job to maintain them. To them, tools are the answer to everything. How could you not love something that organizes so much data so easily? Weird thing — these people are also not the people who are actually doing the work.

When we create Agile teams, we create teams of real, live people. That means our control and communication systems must be built around the qualities and capabilities of people, not robots. We are doing much more in a team room than simply transferring abstract pieces of metadata around about a project. We are gaining a common emotional understanding of the problems and the players, getting a deep feeling for what might go wrong and what we need to worry about, adjusting our work patterns, our communication techniques, and our problem solving skills to this particular team, this particular problem, in this particular situation. There’s a hell of a lot of things that don’t appear on the walls — but that the walls remind us of. An Agile team isn’t a team that moves stories across a wall. It’s a team that looks each other in the eye and talks about things the Product Owner wants and how they’re going to give it to them. It’s a team that munches down on hot dogs and the conversation reminds somebody of something important so they reach up to add a new task on the wall. The stories, walls, and data are all just props to facilitate ubiquitous in-person human interaction. We take the ubiquitous in-person human interaction part out of it, we’ve killed the very thing we’re trying to improve.

If you’re looking for more balance and traction in your Agile team, take a look at the book I wrote on beign a ScrumMaster.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

BigCorp Agile and Feedback Loops

I’m working with a new large client — a great company with outstanding folks which will go unnamed — and I’ve had a bit of troubling adapting to the corporate world after a year or so working startups and providing Agile coaching advice to small teams. After thinking about it for a while, I believe it all boils down to feedback loops.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Goldilocks Agile and Your Grandma

Goldilocks was very tired by this time, so she went upstairs to the bedroom. She lay down in the first bed, but it was too hard. Then she lay in the second bed, but it was too soft. Then she lay down in the third bed and it was just right. Goldilocks fell asleep.

As she was sleeping, the three bears came home.

Let’s say you were writing a letter to your grandma. Would you get out the user manual to the word processor you use? Perhaps spend a few hours going over how to create a Table of Authorities? Those manuals can be huge! Modern desktop word processors deliver hundreds or thousands of features, but 99% of people use 1% of them. You’re not writing a doctoral thesis; you’re just writing a letter to your grandma.

Does that mean we should throw word processors out the door? Never use them? Of course not, it just means that we separate the tool from what we are trying to accomplish.

Being a good grandkid, you decide to write a letter to your grandma every day. After a couple of weeks of this, one day your word processor breaks — you can’t use it any more. Do you stop writing letters to your grandma?

Probably not. Not if you love your grandma and have decided this is an important thing to do for her. You’ll just pick up another tool — ye old pencil and paper has been known to work very well for this — and write a letter to grandma. The tool is replaceable.

Let’s say you are in an auto accident. For the entire day, you’ve been sedated and didn’t get to write your grandma. Would you then say that your plan to write your grandma was bad, broken, or somehow defective?

Probably not. Not if you love your grandma and have decided this is an important thing to do for her. You’ll just start up again tomorrow.

You change jobs and find yourself with a new spouse and life. One long day, after spending time writing your grandma instead of doing other things, you ask the key question here: why are you spending all this time worrying about your grandma? Don’t you have better things to do? Seems like you are focusing entirely much on something that is not important.

At this point, you may decide that yes, you love your grandma, but it’s not effective spending an hour everyday writing her to let her know that. You could accomplish the same thing with a phone call every other day.

So what the hell does all this talk about grandmas have to do with Agile?

Just this: that we consistently confuse the tool, the ritual, and the principles.

An underlying principle of agile is to aggressively attack communication risk in order to tighten the feedback loop. The tool (one of many) is the conversation. The rituals include the story conversation, backlog sizing, or the daily stand-up.

So what if you’re in a team that’s distributed? Does that mean that stand-ups are dead? I don’t know, because I’m not you. Nobody else knows either. What I know is that the principle of aggressively attacking communication risk doesn’t go away simply because people live in different parts of the world, no more than you stopped loving your grandma because your word processor broke.

What if you’re in a team that’s writing exploratory code for a startup? Should you still use TDD? I don’t know, because I’m not you. Nobody else knows either. What I know is that the principle is that the later we test the poorer job we do of it, and that by knowing exactly what the code is supposed to do, we can use tests to make sure we are only writing code to do what is necessary. I also know the principle from the startup world is that 1-in-20 startups succeed, and that coding is a very, very, very small part of making people happy — even in products that require massive amounts of coding. My guess is that in unfunded startups with no traction you’re better off focusing on finding users and a market than in writing tight code, ’cause most of the time it doesn’t matter anyway. But that’s just a guess, and it’s just for a particular type of programming in a very particular kind of situation. The principle of startups being a long-shot doesn’t go away because of the principle of verifying code to make sure you only write what is necessary. You just need to have a conversation about principles to get that sorted out, not tools or rituals. Would you still write your grandma a letter everyday if she lost the ability to read? Probably not — but perhaps so. You have to figure it out.

What about if you’re in a marketing team without hard deadlines or a fixed product? What can you do to work better? The principle of a tight feedback loop doesn’t require a product. What do you think you’ll do today? Did you do it? That’s not just a stand-up, that’s a feedback loop. Maybe that’s all you need. Maybe the other rituals and tools don’t help you very much. Then don’t use them. Never give up your principles, but always be willing to ditch rituals or tools that aren’t working for you.

“But that’s not real Agile!” I can hear you say. If by a pre-ordained set of rituals, nope, it isn’t. But guess what? I favor Individuals and interactions over processes and tools, even if the processes and tools are “the real agile.” I’m funny like that.

A very smart man named Nietzsche once said “God is dead.” By that he wasn’t declaring himself an atheist: what he was saying was that any structure — tools and rituals — we put around the necessarily fuzzy concept of “God” is inherently broken. it’s self-contradictory at best, and actually harmful at worse. [Insert long rant about the harms and evils of organized religions from Nietzsche here.] In his eyes, it’s our inclination to keep trying to create structures and rituals out of the things we don’t understand that is itself the cause of much pain and suffering, not what people actually believe by themselves. We create our own prisons.

Agree or not with the old philosopher, what he said resonates in a lot more areas than just religion. Building technology products is a fuzzy and unsolved problem. We are always seeking to locally optimize for performance. Because of this, over and over again people discover the same principles at work: focus on customers, tight feedback loops beat loose feedback loops, visible work tracking encourages problem solving, etc. These folks write up these principles into various rituals: pair-programming, sprint planning, release planning, etc.

Other folks come along and learn the rituals, never realizing that the rituals are just instantiations of the principles. Perhaps they feel like they’re not expected to know. Perhaps their bosses think they’re unable of working on their own. Perhaps they are afraid of being in charge of their own destiny. I don’t know — but I know it is very sad to watch.

This is what we call “Cargo Cult Agile” — going through the motions hoping that, somehow, by magic the rituals will make things improve.

They never do.

I wish this was the end of things, but it gets worse: other folks come along and take these rituals and turn them into tools. The tools have all sorts of shiny widgets and do all kinds of neat data processing. There are seminars, workbooks, how-to guides, contests, and all sorts of other things to do with the tools.

We have started reading the word processor manual in order to write grandma’s letter. We are so far removed from our goal; instantiating some kind of regular practice to show our love for our grandma; that we may never find our way back. The tool becomes an end to itself. We are lost in the weeds, and we become the enemy of the thing we’re trying to help. We’ve become the bears instead of Goldilocks.

The thing is, it’s not like you don’t need tools. If you have 100 teams, you can’t visit them all at the same time. It is useful for many reasons to have a tool. It’s also not like you don’t need rituals: having something to train and get people started with is critical. But it’s not an all-in or all-out question: to phrase it like that is to be dishonest. A little bit of tools and rituals is incredibly awesome: it lets you do a zillion times as much. But too much or doing something simply because it’s on a list? It’ll kill performance faster than anything I know.

Just like Goldilocks, it’s up to you to experiment and try things until it is just right.

Then hope the bears never show up.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Agile Metrics You’re Not Supposed to See

Now that I’ve ticked off half the agile community, let’s be fair. I should offer something back. You guys can trash this, copy it, or ignore it, whatever works for you. Just don’t say I never gave you anything.

Let me count the ways metrics have been presented to agile teams.

  • Agile Metrics don’t exist – Metrics are for waterfall teams
  • Nothing to see here, please move along. – Metrics for agile teams is just like metrics for regular teams
  • They’re like regular metrics, only cuter – Look! We use colored pens!
  • What we really need is an agile checklist – We need to know if people are doing agile the way they’re supposed to be doing agile. If so, there won’t be other problems
  • It’s all in there – Existing agile artifacts give you everything you need

Let’s cut the crap. The only metrics I really care about is: is the team worth the money they are getting paid? Do things get delivered in a way that the customer is ecstatic about? Is the team happy and productive?

I can make the obvious statement that if you’re worried about something, you should track it — at least until you’re not worried about it any more. But I’m not going to do that.

Or I can give you another platitude: you can’t manage what you don’t measure, you can’t measure what you don’t define, and you can’t define something without a common vocabulary and grammar.

But I won’t do that either.

Instead of defects-per-hogshead, or man-hours-per-story-point, let’s get down to brass tacks. Let’s make it personal. What do I use to measure teams? Certainly when working with a dozen teams or more I have to have some kind of insight into what’s going on. Something I can share with others. Some way of telling if I’m doing a good job, what the real problems are.

Well, what is it?

I’m about to show you two metrics. One is mostly useless. The second is terribly vague and confusing. Put together they do a pretty job of nailing down how to keep an eye on the agility of several teams at once. And most folks don’t want to see them.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.