Category Archives: Modern Program Management

Whatever Happened To Software Engineering?

map-clip-artI’ve been working some on my mailing list to help Agile teams do better and one of the recurring topics from readers has been how to fit “normal” software engineering principles into an Agile framework. How can you do database design, for instance, if every 2 weeks the database is actually being used?

Of course there’s an easy answer: we incrementally do things in Agile. So, for instance, you might use all those database design skills, but a little at a time, instead of all at once. The first sprint you sketch out the tables, maybe add a few fields. Second sprint you change up the tables some, add some more fields. Third sprint you’re working a bit with cardinality. And so on. The concept is that we do the same stuff, only a bit at a time instead of becoming a bottleneck.

This is a good enough answer, and it works for most stuff. But I think it’s also masking a conflict that many people have: software is supposed to be engineering, not just little dips and drabs of stuff added in here or there. To hear many YAGNI pundits and others, you don’t actually do anything until the last minute, and then only in support of the exact thing you’re working on. Most engineering disciplines, however, encourage you to solve the entire problem. If you’re building a bridge, architect the bridge first, dang it. If you’re going to the moon, you’d better have some technical work around the entire trip. You don’t just launch into orbit and figure it out from there. Building incrementally is fine. But there exists fields of inquiry where there’s a long sequential process of refinement over the entire problem domain — one in which you gain execution advantages by working the entire problem at once. We seem to either have tossed this fact away or are purposefully ignoring it.

I began noticing a problem. When we talk about these things, people shut down. I suspect many folks in software engineering, especially Agile coaching, don’t have an engineering background! This can lead to a severe disadvantage when dealing with certain areas:

  • Modeling I don’t see many teams sketch, much less model. That’s a shame, because visual information is a much more effective way to discuss technical matters. Add in a bit of formal training, say 30 minutes, and teams can sketch in UML. Then you can link diagrams. There’s something to be said for lightly sketching problems on the whiteboard. Take a picture if you want to keep it. Start using a modeling tool if you realize that you’re having a group discussion around highly technical stuff. You can sketch, you can use UML, you can use a modeling tool, all without having to become a waterfall BDUF project. Really, you can. It’s the only way to go for complex projects.

  • Process Analysis Does anybody remember structured process analysis? Not to see most teams. The way most people teach Scrum and Agile is that a list of stuff appears — I guess from the sky, brought down by a dove to the Product Owner. The team only works on the stuff in front of it. Don’t spend a minute thinking about the big picture! After all, your project could end at any point in time, and you don’t want to spend one minute on things that you’ll never need.

    Of course, there ARE projects that could end at any moment, but for most of us, we’re brought on to address some kind of system: a website, a business problem, an internal need, and so on. The team, and project, has some cohesion. You’re going to be here in six months, and you’re going to be working on the same thing. For those kinds of projects, spending some time doing process analysis is a no-brainer. You get back much more than you put in. I’m not talking about anything waterfall-like. I’m simply talking about building a process model, over time, of who does what with the system and why. This can help you cut to the chase and decide which stories should be prioritized first. It can help you define and have a common understanding of your stories. It can help the Product Owner make economic decisions about the backlog, and it can help the team interject creativity into the solution, instead of just being a bunch of order-takers. Great stuff for couple hours or so each sprint.

  • Lean Startup Another one of those end-to-end, outside-the-team areas where the team and the Product Owner need to work. What hypotheses is this work supposed to be testing? What are the revenue streams? How are we addressing the gap between market and PO? In a startup world, all these things are critical — and there’s a sequential engineering-type endeavor that can take you from point A to point B. It’s a mistake not to do it.

When we teach things like good architecture or proper database design, many times we view the developer as the center of the universe and the engineering practice as something that’s completed in toto before any other work can occur. This has caused tremendous pushback from Agile teams, rightly so, because it creates handoffs, bottnecks, and unecessary documentation. But are we overreacting the other way? Isn’t there a place for long-format, sequential, detail-oriented work, even in Agile teams in complex domains? Especially in Agile teams dealing in complex domains?

Each of these has a few areas in common: 1) they’re about more than the work directly in font of the team, 2) they’re about having the experts, the team, assist the organization in its work, 3) they’re sequential, 4) they build on themselves as work gets done, 5) they’re not the work itself, and 6) they end up discovering things for the PO and the organization that wouldn’t be discovered otherwise. Because of these attributes, they tend to “fall between the cracks” when teams adopt Agile practices. They are traditionally part of what people employ engineers to do.

How about you? Are there engineering processes you miss seeing in your Agile teams? What are you doing to make up for it?

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

The Perfect Agile/Scrum Stand-up

When people first arrive in Agile/Scrum, one of the first things they ask is “Doesn’t Agile mean we don’t need requirements?” followed by a close second of “Scrum is too rigid. We definitely don’t want something so restricting”

It seems like you’re either a cowboy or a Nazi. There are no in-betweens.

It’s like this: Agile is all about dynamic process tailoring. That means, yes, that teams control the degree of paperwork and process they need to accomplish. But it’s also like this: there are some “recipes” that the community has found over and over again that work. You’d be a complete fool to ignore them.

What happens with high-performing teams is that they have very few things they do that are defined and reproducible. But for those very few things, they are very disciplined. Being Agile, strangely enough, means being extremely disciplined about a very few things and leaving the rest of the details to make up as you go along.

The crux of all this ambivalence is, in my opinion, the stand-up. Since nothing proves a point like actually seeing it, in the video that follows, note the following things:

  • It happens quickly. Stand-ups are high-energy. People are standing-up, moving around. Nobody is sitting in their chair with their head in their notebook or cellphone.
  • Nobody is reporting to anybody. This isn’t a status report. There are no questions and answers.
  • We point at the things we are doing. It’s all about Visual Process Control. That means the things you are doing are publicly visible, they represent status at any time, and they are manipulable.
  • Decisions are made after the stand-up. The stand-up is a game of identifying tokens: what has happened, what is going to happen, what obstacles individuals are facing. After the tokens are all identified, the “real” work begins: the tokens are resolved. Technical discussions and ad-hoc discussions begin.
  • There is no Project Manager. If you can look at a stand-up and find a project manager, it’s a bad stand-up. Stand-ups are about team commitments, risks, and issues. Not roles.
  • Everybody is in the same room. Body language and other non-verbal communication channels count. A lot. Everybody isn’t standing around talking into a PolyCom.
  • It’s scalable. If done correctly, team stand-ups scale very easily. A manager could attend several stand-ups each day, get a close-up view of how the entre program is going, and still have most of the day free for coordination and planning activities.
  • The team is small. You can’t have an Agile team with 30 people on it. Do not do this. It does not work.
  • The story board is not that complicated. This is about the limit of complexity for a story board that I am comfortable with. Past this is drudgery. If your storyboard has the colors of a Christmas tree and a sufficient number of post-its to draw a 40-foot picture of Godzilla, you’re over-complicating things. Stop that.
  • It’s a problem-solving exercise. Immediately after the stand-up is over, the Product Owner talks to anybody with questions or problems and works them out.
  • We work out problems visually. Notice how they pull the story/task over to the flip-pad and work through the problem? See how the options and structures are all presented visually? Here’s a tip: doodling or drawing structures and flow on paper has about a hundred times more effectiveness that trying to write it down or vocally discuss it. Model things.
  • Big Picture is not present. Nobody is looking at long-range issues. Nobody is talking about things happening 4 months from now. Nobody is tapping their foot and asking “are you through yet?” The entire thing works on local optimization. That means that inside of sprints teams locally optimize to finish their commitments. It’s only at the boundary of sprints — demos, retros, and IPAs — that the big picture is important.

Finally, notice it doesn’t match the video. The perfect Agile/Scrum stand-up demonstration does not exist. Teams are different. Everybody has to adapt. As I said in the beginning, Agile is all about adapting. The point is to try things the “best” way — as the community has determined them — and then adapt for your particular situation. That way you don’t miss out on anything, and you also have good reasons for the way you’ve adapted.

Want to rock out your Agile team? Here are the e-books for you.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Kanban Vs. Defect Hell

Do you have a huge pool of defects, bugs, and fixes waiting to work on? So much that you find it impossible to balance it with other project work?

I’ve heard three versions of the same story about how teams fixed this problem over the last year. I’m always cautious of over-generalization, but once our sample size gets to three it’s time to share.

The first part of the story goes like this: after releasing super-cool whiz-bang 4.0, the team is given a lot of fixes to make. For a while, everybody works on them, but management has other ideas: it’s time to work on super-cool whiz-bang 5.0, guys! Surely you can do this other stuff in the background.

So the “other stuff” is taken up by one or two guys, or it’s farmed off to India, or a couple of guys come in on the weekends for a bit to try to tidy it all up. There are a lot of plans for how it’s all taken care of.

Only it doesn’t work the was it was planned.

Instead, the load becomes bigger than the guys can handle. Not only stuff from 4.0 is in the list, there is still stuff from 3.0, 2.0, and 1.0 that hasn’t been fixed. The list gets huge.

Acting logically, you set up a triage. Things come into the list. You take a bit of time to estimate how big it is. Is it 1 day? 2 weeks? 3 months? Once you know roughly how big items are, then the business guys can make decisions about priorities. So the next thing to set up some kind of field for priorities, maybe low, medium, and high. Maybe forced ranking. Of course you use some kind of online system. After all, this is the 21st century! All important things must happen online somewhere. If it’s not online, it’s not important.

But wait a dang minute, here. Some things are mission-critical fixes, like when production tanks. You can’t keep working on fixing a report when the whole thing is broken. So you make a rule that when production tanks, everybody stops work and fixes it. Then they go back to whatever they were working on.

After a bit of time, you notice that not only is the list huge, it just keeps growing bigger and bigger — much faster than you could ever hope to catch. You’ve just entered Defect Hell, where a big dump-truck comes by once a day and piles an endless stream of crap on top of your head.

This is bad for everybody. For customers, things are broken and never get fixed. For other people in the company, they start losing faith in the ability of the team to fix problems quickly. For the team itself, they’re on a demoralizing death-march without end.

Bad. Bad. Bad. And if you’ve worked in a big organization at all, you’ve seen it.

In most places, the response to this problem is to simply keep over-complicating the plumbing. Perhaps you add another field to your list, like “severity”, or “impact”, or “will cause loss of customers” Maybe you create little automated email systems, flashing lights, or loud sirens. The hidden message is obvious: you dumb defect fixers need us to help kick you in the butt so that you’ll work faster.

That never works, so then, sadly, organizations will invent completely new ways of fixing the problem. Instead of complicating the existing system, another system will grow up alongside the first in order to “really once and for all fix the defect problem” Perhaps it has red flags, or little stuffed animals that explode. The safe money says it’s something that looks very serious and has a lot of complexity underneath. Maybe there’s a class or a little gold star you get to wear.

The problem here is outhouse process engineering, or the idea that a few smart guys can sit around with a team of super-users and pinpoint the exact nature of the defect-tracking system to a degree necessary in order to design a one-size-fits-all solution.

Yes, Six-Sigma Guys, I’m looking at you.

But you are by no means alone. Not at all. All of us in technology believe we can over-generalize and then over-apply those generalizations.

And we are most always wrong when we do it. Technology development, no matter how much we’d like it to be, just ain’t manufacturing. It’s R&D.

Here’s how the second part of the story is turning out for some folks:

Take a look at your data. How long does it take to fix a defect on average? For our story let’s say the answer is 25 days.

Throw out all the other complexity. We know on average it takes 25 days to fix any defect. Yes, it will vary by the individual case, but the variance is not important right now. Addressing it causes more problems than it solves.

Second, eliminate any sort of up-front sizing, prioritizing, or tool-based data nightmares. Simply make a list. Once again, the overhead creates more problems than it solves.

Third, create a kanban board with some steps you think might be common for all defects. On one side might be “start” on the other side “test” Don’t worry if it’s perfect, just throw it out there. If it doesn’t work for you, change it. Immediately. It’s your board, not anybody else’s. It doesn’t exist as a rule anywhere.

Fourth, pick a number that represents your maximum capacity, i.e., the amount of stuff you can work on at any one time. let’s say you’ve got a couple of people working on things and everybody decides that number is 3. But it could be 7. Just like the layout of your board, don’t spend a second over-engineering it. Pick something and then adapt. it’s important to get buy-in from your customers on this, but the critical thing is that you have to pick some number that everybody agrees on. This number represents work saturation — when you have 3 defects on the board, you’re loaded. You’re not allowed to add more on the left until one comes off the other end.

At this point, you have a simple question for people submitting things to the list: what would you like to have done in 25 days? They can flip for it, arm-wrestle, conduct decision-facilitation meetings — it doesn’t matter. From the team’s standpoint, just let them know. The business must self-organize to work through this.

There are a lot of objections to this setup, mostly because it ditches a lot of complexity that we just “know” is important. What about things that have to happen immediately? Well, make sure they get pulled next. What about things that finish early? The way kanban works is that once something is finished, the next item at the top is pulled. What about people trying to submit new feature requests disguised as bugs? That’s a common worry, but as it turns out it only happens a very small percentage of the time, maybe 2 or 3 percent of the total submissions. When the team finds one, let them escalate if you like. It’s not a big enough deal to add complexity to the system. Why are we giving up the step of sizing the effort before it goes in the list? Because teams that look at the data are finding they spend between 25-40 percent of their time simply sizing defects that may never get fixed. What about things that stay on the list and nobody ever puts it in the top three? Things on the list over six months get booted. If they’re important, somebody will put them back on the list. If not, they won’t.

Once all the objections are satisfied, what I’m hearing is that this is good for everybody. For the teams, it gives them some amount of sanity: they know what they are working on and have freedom to adapt and improve how they solve problems.

For the business partners, they start having to have real conversations about how much they can do and what needs to be done next, instead of just tagging everything “urgent” and throwing it in an electronic bucket somewhere. It also cuts a lot of the cruft out. The question of “what do we want done in 25 days” is a solid question with real impact. Checking little boxes on a form somewhere is just an exercise in frustration and futility.

Also the business starts to note that things are arriving as scheduled. Many things are done in much less than 25 days, but somehow even the outliers get pulled in under the limit. The business starts trusting the team more.

And, after a few months, as the team relaxes into a flow and starts self-optimizing, the 25-day lead time starts decreasing. It’s 22 days, then 18 days. As the lead time decreases, velocity through the system increases.

In all three stories I’ve heard, on huge messes it took perhaps a year or two to completely fix. You don’t clean up a problem that took years to accumulate in a week. But even within a few months, it was clear to everybody that things were on the right track. Every one of these guys were extremely happy — to the point of recommending kanban for just about everything in the world.

Which, of course, is the problem. The same “disease” that caused folks to create cumbersome tracking and allocation systems in an effort to help defect teams can also cause successful defect teams to create systems that may or may not work for other folks. Beware over-generalization and premature optimization.

So no, there’s no magic potion, but as they say in the news business, a pattern of three makes a trend, which makes it newsworthy. I am putting this in my arsenal of suggestions for any type of team that has a situation that looks like this, including program-level defect teams, where this defect hell thing can drive everybody crazy.

Whatever it’s actual value for your particular situation, it’s an nice tool to have sitting in the tool-chest.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

The Case For Nothing

The TSA announced this week that they are going to start being more aggressive with their pat-downs. This, in addition to the virtual strip searches they want to perform, has made me want to stop flying commercially forever.

Folks seem to enjoy this kind of abuse of my freedoms, though — folks from all sides. When I’m in a conservative room and I complain about conservationists taking away my private property rights (or government telling me I have to buy insurance), everybody agrees. When I’m in a liberal room and I complain about the TSA or internet monitoring, everybody agrees.

Everybody seems to agree: whatever they want to do is more important than my freedom of action and my personal property. They know better than me, and they are very willing to decide that I need to make sacrifices to make them happy about something or another.

But I don’t want to talk about politics. Or at least not directly. I’d much rather try to go meta and talk about general principles. What I would like to talk about is the reaction of individuals when organizations do stupid things, because I see people doing this same thing to each other at all levels of organization, from tiny teams of a dozen or so up to the size of the United States Federal Government. And the response is always the same.


Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Accounting — Agile Project Management

One of the first questions we get when implementing and training agility in organizations is about accounting. So much of what happens in an organization revolves around the numbers — risk/reward, investment/return, burn-rate, etc. Many times developers and technology management gets wrapped up in all the goodness of change and then we forget to bring the accounting folks along.

In a lot of organizations, you can get away with glossing over some accounting principles because the project has a charter and a budget that is independent of the way the project is ran. However, it’s totally natural for accountants to want to see inside the project as much as possible. And once you start running multiple projects in a program a lot of this hidden detail raises it’s ugly head.

We wrote an agile project-tracking tool last year as part of an effort to provide some answers to these questions for a large corporation that was looking to have more control and insight. While it’s not publicly available, we did learn several lessons that might be useful to others.

So if you’re trying to figure out how accounting concepts apply, here are some terms and their meanings in accounting terms:

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Is Agile Something Teams Do? Or Something We Do To Teams?

“Well nobody asked you your opinion,” the coach glared at me, “so you can sit down and shut up. And if you can’t sit down and shut up, you can leave the room.”

I could feel everyone’s eyes on me for just a second. Then they all found something else interesting to stare at. I glanced at my project manager. She was playing with her pen, her eyes were bug-eyed and her eyebrows raised. She looked as if she expected elephants to fly out from her ears at any moment.

Welcome to my Tuesday morning.

Talk about being completely blindsided — I had no idea this was coming.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

CrossFire Charts: Measuring Agile in Matrixed Environments

A CrossFire Chart which shows team performance against budget

The CrossFire chart allows teams to measure budgeted performance
against traditional agile metrics

Here’s a little graph I picked up last week. I don’t think anybody else has published this, so I’m calling it a “CrossFire”. It was created by a team who had a lot of matrixed team members and wanted to track budgeted performance (the amount of time people reported in the official time-tracking system) with agile performance (iteration burn-down).

Turns out you can find out some very interesting things from these CrossFire graphs.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Why I Love Modeling

I’ve been a big fan of modeling for almost all of my career.

I’m not talking about wearing a funny green dress and covering yourself in paper Snoopys — not that there’s anything wrong with that, mind you. It’s just not the kind of modeling that I like so much. Plus the wife says I always stretch her dresses when I wear them.

Fashion model wearing a green dress with a large number of paper cutouts in the shape of the cartoon character Snoopy

Hey — at those rates, I’d give it a shot!

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

My Product Owner is a Robot


But I don’t argue with him that much…

So a while back I was exposed to two agile teams that were building their own product owner robots. You know how it is, you need a product owner, of course, and sometimes there just isn’t one around.

One of these teams I think is going to succeed, while another will struggle. I’ll describe both teams and you can see if you make the same call. Then some commentary.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.