« Structuring Larger F# Functional Projects| Main | Agile Metrics You're Not Supposed to See »

Agile Ruined My Life

| | Comments (116)

I read the reply to my comment on a popular hacker board with sadness:

(disclaimer: Agile consultants ruined the software group I work in.) Making good software is hard, and anyone claiming to have a magical process that guarantees good software is selling snake oil. I can appreciate your wanting to make a buck, but would also seriously appreciate it if you could find some other industry besides software development to go screw up

Reminded me of an email I received back in May:

[We] started working on [agile technique X] when [author]'s [famous book] was just a draft. I was on that project and worked on Agile Projects for a decade. (Next time you meet [famous guy], ask about me, I just finished reviewing his forthcoming [another famous book]). I am a founding member of the Agile Society of [place] and have organized conferences on Agile. I've attended XP Conf as well. I've probably worked in more agile projects than you ever have (not that it particularly matters). So let us first dispense with the notion that your notion of what constitutes "true" agile and its scamsters is somehow the only standard....


Do you deny that the whole Scrum Master idea is a scam within the Agile Camp?

Scamster? Ron Jeffries the guru/founder of Agile couldn't write a Sudoku implementation with his favorite technique "TDD" and refactoring over five weeks. Fraud.

Robert Martin (another "guru" and agile consultant) claims that any code not written with TDD is "stone age " code including such things as Unix and such people as Norvig and Linus and Zawinski who've built more code than he can dream of. Dalke poked holes in his TDD "kata" which never got answered Fraud.

I could go on and on. And these are the gurus. But that isn't the point. i *saw* "agile consultants" evolve from some naive but well meaning people (like Kent) to scamsters like X and co and tose are just at the top. Practically every single "Scrum MAster" is a fraud. The more intelligent among them admit that two days of listening to a higher level shyster teach nothing and it is just a signal to dumb managers to improve their chances of getting a project. Yet they go along. That in my eyes is a scam like chiroproctors or reiki people claiming to be doctors. Agile was amovement founded by scamsters and propagated mostly by scamsters.

I've had many such conversations over the years.There are some seriously pissed off people about Agile out there. Why? Isn't agile supposed to be warmth, apple pie, motherhood, goodness and all of that? Why so much anger?

The easy answer -- and the answer most agile-lovers would give -- is that these folks are simply non-hackers. Bad attitude, poor skills, interpersonal conflicts -- the reasons are many and diverse as to why a small percentage of folks are just going to get ticked off at anything you try to do.

I don't accept that. Or rather, while it may be true, it is also an excuse for non-action. I view every piece of feedback as a cause for some kind of action.

And the thing is, it's not just the people who are being trained. I've done my fair share of complaining about various pieces of agile, and I've seen many other coaches -- in private-- grumble and complain as well.

So it's time to get honest. Take a good look at ourselves.

Here are the problems I see and hear about:

  • Fake success stories - People think they can take some lame project that's mostly done, apply a little agile, then proclaim how great it was? Come on, folks, this isn't fooling anybody. Everybody knows exactly what it is: propaganda. Making it worse, many times the experts brought in are the last to know what a pointless photo-op exercise it was, leading them to "learn" incorrect things from the experience as well, then "sharing" that knowledge with new teams, continuing the cycle of crap.
  • Trainers who can't do the work - I have good friends who teach agile and haven't coded or led a team in years, so to them I apologize. But if you're going to train something, you should be able to do it. And I mean do it to a very high level of expertise. An agile coach should be able to code, perform analysis, manage the project, test -- anything that needs doing on a project. If she can't, then how can you talk to her about your particular situation? If your agile trainer was a BA last week, or never slung code in his life, or is a professional trainer, or -- let's be brutally honest -- is making less than the members of the team are, you've got a dud. It seems like common sense but it bears repeating: you can't train something you haven't done. And "done" means a bunch of times, not just on the pilot project. I had a company once that wanted me to train several people to be agile coaches -- people who never knew agile before I walked in the door two weeks before. Hell, if I could do that I'd be printing money, but it doesn't work like that. Does anybody go to school to be a famous baseball coach? Or do they learn to play baseball first and then only some of them realize that they have a talent for coaching? Nine women can't have a baby in one month, no matter how much you wish it were so.
  • Inflexibility on the part of adherents - I worked with a lady who wrote an article asking "why are we complaining about Scrum teams not succeeding when they're really not doing scrum?" This attitude -- that there is a list of things that must be perfectly done and failure is a result of not doing them -- is basically religious in nature. You can never do enough. If the team fails? Well, it wasn't agile enough. It's nonsense, that's what it is. Lots of great agile teams fail. And lots of teams who are not agile do very well.
  • "Feel good" agile - One of my friends went to an agile conference. She told me she left one class because it was about "the use of haiku in team-building". While I love poetry, seems a bit fluffy to me (and to her). I have several friends who, god love them, are hippies. It's all bunnies and floaty clouds and harmonics and karma. These things may have an important part in life, but I don't want to sing Kumbaya, I want to have a fun and productive team. Don't get me wrong -- I love unusual and off-the-wall techniques. But the agile community has at times embraced the far fringe of wackiness too. It's hard enough getting extremely detail-oriented analytical people to stand up and talk to each other every day. Getting them to put on a puppet show for their showcase is just a bridge too far. We need to tone it down.
  • Magic Bullet Syndrome - One of the first things they tell you in Scrum class is that Scrum is not a magic bullet. Then they spend the rest of the time telling you how it's the best thing since sliced bread. We've all met and worked with the guys who already have the answer -- you just need to ask the question. The solutions have already been determined for whatever problem you might have. These people are extremely annoying. It's like talking to a wall. Bad, bad. I knew a guy once (another famous guy) that would love to stand up and give an impassioned plea for just doing scrum. Whatever the problem, whatever the actual situation, you could count on him to bloviate about Scum. Not only ineffective, but highly annoying. You don't know whether to laugh or cry
  • Reversal of team dominance - I know a lot of guys who teach agile. Sadly, many of them impose agile on teams, not train them. You come in with a big stick, then proceed to beat people with it until they "conform". The dynamic is backwards -- the outsider is somehow in charge instead of the team. One guy (famous author again) basically put it like this to me when I told him the team wasn't succeeding: I'm here to demonstrate certain practices and to show that they work, not to just stop everything and attend to what the team is dealing with today. Sadly, upper-level management encourages this kind of behavior. Many clients will ask me to provide a schedule an a checklist for how I'm going to make their teams agile. I tell them look, I can provide the information, and I can coach the team as it works the problems, but the problems are going to be people problems, not technology problems. And guess what? People don't respond very well to being treated like machines.
  • Cargo Cult Agile - There are a lot of teams doing cargo cult agile out there, also called theater agile. It's where everybody knows their lines, the rituals, and where to stand and how to act. It's like an orchestrated pagent. It's an awful, lifeless thing. Blech
  • Non-answers to questions - Can agile work with distributed teams? It depends. Can we use fortnights to estimate projects? It depends. Can agile work with embedded software? It depends. Argghhh! Everything depends. Sometimes no matter how hard I try to be forthcoming, honest, and direct, I end up sounding like those guys from The Matrix the first time you watched it. They said a lot, but it didn't really mean much. It all sound like just so much gobbledygook.I hate giving advice like that. I know folks hate hearing it.
  • Conflicting Advice - Can you architect and design before you code with agile or not? Can you have requirements? Can you work on requirements ahead of the sprint you are in? Can you roll-up multiple projects into usable program management metrics? I could list a dozen more questions which have multiple answers, depending on who you talk to. One guy says do it this way. Another guy says do it that way. It's enough to drive anybody nuts.
  • Scamsters - I spoke at an Agile conference back in 2009. One of the first things I said to the audience was "I don't read agile books. They are a waste of time". Wow! You could almost hear the groans throughout the crowd! But I'm serious: 99% of agile books out there are just people telling stories about stuff. Stories are great -- love to hear them. But I can't trust the authors of most of these books to tell honest stories and learn honest lessons from them. Instead they have a theme, an argument, a point-of-view. And everything in that book is going to support that theme, that point of view. Heck, it's just good book-writing. The problem is, real life doesn't have a theme. Or if it does, it would be amazingly incredible and preposterously improbable if your book matched up with what was going on with my organization. The early agile books were so funny that I couldn't read them -- I always started laughing t oo much. When Bob Martin started trashing developers who didn't use TDD I realized that many "agile experts" had were jumping the shark. Yes, there are a lot of folks selling you things you don't need by convincing you that you need it. A fair word for that is "scamster"
  • It's the same, only different - One of the things I hated most about agile is when management decides to "be" agile, only they don't want to change anything. So then you're teaching a team that they are in control, that they are responsible for important decisions about how much work they can do in each iteration and how to do it -- only they aren't. This destroys morale faster than anything. A while back I turned down teaching TDD to a team. Why? Because somebody up above had decided the team was doing TDD, not the team. The class would have been three days of me trying to share things that the team had no desire to hear and wasn't going to practice. A lot of other coaches -- the vast majority, probably, would have taken that gig, but I'm not going to be part of the problem. Courage isn't just for teams.
  • Little Gold Star Syndrome - A two-day class and a little gold star, or your name on some website, doesn't mean jack squat in terms of what you can do. Let's just be honest about that. The training might be great, but the idea that getting a little gold star sticker on your head is going to make you significantly better is bogus.



When training agile, one of the first things I do is go over definitions.

What's Iterative Development?
What Incremental Development?
What's Scrum?
What's Agile

The answers -- both the ones I give and the class's -- are interesting.

Iterative development is doing things in iterations. Little bits of work done over and over again. Agile is big on time-boxes, but iterations can be done based on features too. The idea is that you do everything you need to do to deploy. Then you do it again. Over time the product gets better and better and the team begins to have experience in all phases of development.

Incremental Development is doing things in little atomic pieces, called increments. Say you want a checkbook program, so increment 1 might be logging in. Increment 2 could be writing a check, etc.

So far, so good. Most folks think that iterative and incremental development are good ideas. If not, then welcome to 2010. There are other ways of doing things, sure. But most folks are already at this point.

So what's Scrum? It's a standardized version of project management tools for iterative and incremental development, that's what. It has a board, a test, a class. It's a monolithic thing. When we talk scrum we have concrete terms and concepts to discuss (like them or not, separate subject)

Our final question: what's Agile? Usually a couple people have ideas. "It's TDD" one might say. Another might say "There's a manifesto I think"

After a long pause I tell the class what sounds like a joke but isn't.

Agile has no definition.

Nada. Zip. Bupkis.

There's no standards board, there's no test, there's no approved workbook, there's no checklist.

Agile is nothing like Scrum. Personally, I think that's a good thing.

Agile is a set of best practices around running iterative and incremental development teams. It's a marketing term. Sure, there's a manifesto, and there are experts (I'm one of them), and there are conferences, and books, and classes, and god knows what else. But it's just best practices.

It's based on three things: 1) principles not practices, 2) attention to people, and 3) always be adapting

To some, this might be so fuzzy as to mean nothing. If so, I apologize. I can assure you that there really is a structure and line of progress to learning agile. I can also assure you that teams that "get it" are happier and produce a lot more than teams who don't

But it IS an art, not a science. You don't just read a book or take a class and suddenly you are agile. It's more like playing jazz piano. You learn a bit, you do a bit, you take an honest inventory of what works and what doesn't, then you learn a bit more. And so on. It's the doing , the reflecting, and the adapting that count the most. You don't learn to play the piano by watching a film of somebody else playing, reading a book about it, or going to a conference. And you don't learn by making yourself into a robot, following a series of rules without exception. Would you try to play the piano by dressing up like a pianist, renting Carnegie Hall, and simply acting as much like a great piano player as possible? Yet every day some poor schmucks are sitting in a stand-up that lasts for an hour and is more of a brutal daily status report than something collaborative. And we call that agile.

The commenter from yesterday went on to say that he was working in a development group that was happy and productive. Then they were bought out by a larger firm who decided to "do agile" on them. Productivity went down the tubes, morale suffered, and people were told to adapt or get lost.

Iterative and incremental development isn't for everybody. Lots of teams do things completely ad-hoc. Lots of teams are happy with waterfall. Lots of folks just don't care to change. These are all good reasons why agile might be a bad idea.

My standard for what agile isn't universal, sure. but I'm very happy teaching best practices for iterative and incremental development. You can call that agile, you can call it Joe. Whatever it is, helping people see things and try things they haven't seen or tried before -- and then letting them decide whether it's working for them or not -- is a pretty good business to be in. But there are a LOT of problems in this business, and ignoring them won't make them go away. Over time there can be an us-versus-them attitude that sets up between any two groups of people. We must always be on guard for this. If you're not a servant to the team, you shouldn't be in the room. That's just as bluntly as I can put it. The problems listed above are owned by all of us, and it's our job to make sure we address them as best we can.

UPDATE: If you are interested in the discussion, after reading the comments here you'll want to read the follow-up, "Agilholics Anonymous", which tracks the impact the article had on the net 2 weeks after it was published.

UPDATE2: For those of you coming in months later, here's the current corpus of Agile material as rated on Amazon. No need to single out any one author or book, just take a random look at some of the reviews and see if they strike you as being unbiased.

116 Comments

Great article. A lot to chew on. I completely agree that a 2 day course doesn't make you a "Master" at anything. Time and experience shows Mastery. I especially liked your point about Agile in that it's based on three things: 1) principles not practices, 2) attention to people, and 3) always be adapting.

These are crucial to communicating to people out there!
Keep up the good work.

Best,
Peter Saddington
www.whitebarrel.com/blog

So where do I go to learn this stuff properly AND get certified for the bosses to believe I can do this?

London hints preferred :)

Good analysis. One thing that makes things with some good (like agile programming) go so bad, is the "secret formula" mindset endemic in middle management. If you doubt this mindset exists, visit the business books section of any major book store and look at all the books with titles like "Secrets of...," "Keys to...," etc. This opens the door to everything from unrealistic expectations to scammers in agile as well as everywhere else.

Skillsmatter in London run fantastic courses, or you can try to get some Thoughtworks training.

Thank you, thank you, thank you for this post. While I don't agree with most of it, I too am sick of the snake oil salespeople selling Agile golden bullets. It's nice to hear from someone who's actually _done_ lots of Agile projects.

I'm taking the approach of simply cherry picking Agile techniques that suit my organisation as opposed to trying to convert them to the new religion X.

Cheers,
Martijn

Hi Dave,

What I recommend is to combine classes with immediate hands-on practice. The practice is more important, of course, but the bosses and such like the classes, and there's stuff you need to learn anyway. To continue the piano analogy, you could learn to play the piano by just winging it, but the knowledge work also helps. Just be aware that the piano book isn't really teaching you to play piano. It's just giving you some terms and concepts you'll use.

I teach and coach agile as a side business to my startup -- it pays the bills. So -- shameless plug coming -- write me an email and let's talk! I've consulted with companies both in Europe and the Americas. Usually we work on the ground inside the team(s), and combine _very_ brief classroom work with whatever the team is already doing. The team comes first, of course. So if the boss wants you to get the gold star, by all means go get it, but follow-up immediately with something more useful and practical, like hands-on coaching.

There's a lot more I could add -- like the purpose of a coach or how to set up a successful intervention, but a great part of that is dependent on the circumstances involved. That's why it's best to start a conversation with somebody who's been there, done that. Beats some kind of generic advice that might not apply to everybody.

So my advice is to find somebody who knows how to help people make the transition and knows what to expect (like me), and then listen closely to what they have to say.

The problem is that this is a business -- in other words, people try to standardize it and make it into a money-making machine, just like any other business. But technology is extremely unique; generalities don't hold so well. That's why it's best to form a partnership with a person or two instead of getting some kind of cookie-cutter product from a company. That's my opinion, anyway. If I had to learn agile, I'd look around for somebody like me and then form a relationship that would last for a while. I know that sounds self-serving, but that's how I would go about it. I certainly wouldn't buy an agile tool (like so many places do)

To Dave Hodgkinson: London hints

Learn this stuff properly: Extreme Tuesday Club

http://xpday-london.editme.com/eXtremeTuesdayClub

Get certified so the bosses will believe you can do it: Agile Developer Skills Workshop coming up in London:

http://wizewerx.com/skills-workshops/

Cheers,
Dave (no relation)

"Time and experience shows Mastery."

IMHO time and experience show age. Results show mastery.

Nice article!

Much like pre-Snowbird, my last few engagements over the past 4 years (just by way of recent examples), I always pushed solving the underlying business problem and providing solutions, as that was what the folks I was talking to needed. It just so happens that I use a lightweight, agile approach. It just so happened that the client was cool with that (and even if they weren't, I would find a way to do it anyway ).

One engagement (i was leading the team for about 2 years) was with a company devoid of much effective s/w processes (inefficient waterfall and huge business/IT schism). Now they have an effective (distributed) agile process, they have a lightweight set of tools, and a good relationship between the business and IT. As agile as I would like? No. But on a scale of 0 to 10, I am very happy that they are probably a 6-7, and even happier that they continue to try and improve. After all, they are in charge of what practices they chose to adopt within their own constraints. I simply shared with them how I have had success doing projects. Some things I suggested we adopt were rejected. Some of those bit the team later -- which was a better lesson than had I forced it down their throats.

However, many other times, I think, people/orgs are seeking to just become *more agile.* Or they want their people to be certified in this or that. For some reason, they probably think this will help them succeed. And maybe it will. or me, it is a bit nebulous. I always liked to tie all education/training with actual doing so that the lessons would sink in and so that people had a reason to succeed.

Unfortunately, these orgs seeking "more agile" may not necessarily tie the agile transformation to any sort of project success. Rather, they just want to learn better techniques in the hope that it will foster improvement to their bottom line. Sometimes it is just throwing a bone to the folks to try and take their mind off of the truly waterfall world in which they operate.

At times, the agile hype is almost like "If I wear these jeans, I will be buff and have great-looking people surrounding me." As Madison avenue has brainwashed most of us, that technique is very effective.

Frankly, this is a normal process of a good idea being turned into a movement being turned into a fad being turned into a money-making opportunity resulting in disillusionment and spotty success.

Warts and all, I'll happily take the rewards that having agile "out there" has provided "humanity" -- even if it is abused at times.

The smart and skeptical will prevail and the easily duped will be duped. The beauty of a free market :-)

Caveat Emptor!

BTW: your email insert has a fraudulent line:
"Agile was a movement founded by scamsters and propagated mostly by scamsters."

We didn't found a movement. We published how we found success at doing software development. We all had different approaches and we found the common threads that we felt might be useful for others to know about.

I might be wrong, but I think true movements just "happen," they are not "founded."

(Who knows, maybe movements that are purposefully founded deserve skepticism by default, I'll leave that for others to ponder and research :-)

Jon,

Thanks for the comment.

One of the interesting things I've noticed is that there is a difference between the "thing" and "the way the thing is perceived"

I'll use an example that many folks know -- use cases. Use cases can be a lightweight, adaptive, iterative, agile approach to having a conversation. Just write bullet points on a story card and draw a flow on the back (if required). Or they can be a god-awful monstrosity created by the devil himself and designed to cause pain and suffering upon mankind.

Both are true -- and neither is. Depends on your experience. When people think of "agile" and "scrum", they don't think of it the way folks in the industry do. They think of the perception, not the details. So this post dealt with perceptions. And for a lot of folks, agile is something somebody else decided they are going to do. It is foisted upon them.

As you know, most teams that get it are happy and productive. But there are many, many ways to screw it up. And when it all goes to crap, the word that people remember is going to be "agile". Fair or not.

I think the angry email guy speaks for a lot of folks who just aren't speaking out about their feelings. So I decided to "lay the cards on the table" and see what kind of response we get.

I appreciate the feedback.

rof,l re: your comment...

Or they [use cases] can be a god-awful monstrosity created by the devil himself and designed to cause pain and suffering upon mankind.

I remember in one engagement near Minneapolis, this one woman insisted that all use cases be done to 3 levels. Frankly, I was using a blend of FDD and XP and didn't have the foggiest idea what she meant. What's a level? Why always 3? So she brought out a book on the subject and the author -- who worked at the company, IIRC.

We mutually decided that maybe I wasn't the right guy for doing the rest of the training if they had a rigid idea about how to do things. It turned out that there was an internal rift between the guys that brought us in, and this woman. She "won." I left without finishing the last day, and wouldn't be back the following week.

BTW: I like what you are doing. I guess I need to get out more. I didn't realize there was so much simmering discontent behind Agile. Even if I haven't run across it myself, I can "feel" the pain through what you have posted :-)

Top article.

I work in a small team. We are 'agile' (self proclaimed).

And this is what it means to be agile for us:

We use Kanban to keep a well radiated idea of whats going on.

We think it's important to limit the focus of the dev group at any one time, such that we are not all working on disparate aspects of the codebase.

We like to write our code as cleanly and concisely as possible, so that the next guy who comes along can read our intent quickly.

Thats about it.

We still have to work really hard at, 1. staying agile and 2. our code. It's not suddenly any easier, it's just easier where it ought to be, so that the hard work goes into outcomes that have short and long term value. That is, we are dispensing with yak-shaving. http://catb.org/jargon/html/Y/yak-shaving.html

It feels better this way vs. more waterfally, long release cycle development practices. This is where it feels warm and fluffy.

You can have warm and fluffy feelings with cold hard scientific analysis. If the analysis says "you're doing better" and "you keep doing better", you'll get the warm and fluffies, at least if improving the quality, quantity and overall value of your product is what makes you happy.

If you like writing cryptic code that scares the pants off your co-workers, you wont like "being agile". If you like ruling your teams with an iron fist, you wont like it much either.

I agree with many of the comments that you make, especially about the snake oil often peddled by Agile consultants. It is disappointing that you stopped short of calling the Scrum certification process what it really is -- a pyramid scheme.

I worked for Ivar Jacobson for a while. You might remember Ivar -- he's one of the guys that invented use cases.

Just like you, I had a client once that _insisted_ that we were going to do "stories" as these huge multiple-page use cases. I told her that yes, we would be doing stories, and yes, requirements for a formal system were a different thing than co-located teams, but no, we wouldn't necessarily be doing things to that level of detail. At least not for development.

I mean damn. We argued for a half hour. I didn't try to argue. It was me trying to convince her that her paperwork would be done and it was her telling me I didn't know what use cases were!

Finally in exasperation I point to my badge. See that? Know who that guy is? Would you like for me to get him on the phone? Do you think maybe I know what I'm talking about?

Later we laughed about it, but I hate name-dropping and using that kind of stick. But some folks are just stubborn as mules. I know because I am one of these people. :)

A lot of the problems here are definitions. Coming to understand what people are talking about is half the battle.

So the team switched to user stories

After I left, they created an Access database for "managing" user stories. Then they created a multi-page MS Word template for "creating" user stories. At one point they had several thousand "user stories". They programmed a special SharePoint site. They bought some expensive tool.

It was never use cases. Use cases were never the problem. It was the corporate culture that was the problem. Sigh.

I've been lucky to do intensive coaching followed by periods where I just do what I want. For the last several months I've been learning functional programming and working on a startup. I think if I did coaching all the time, or if I made a business out of helping folks with Agile, I probably wouldn't see a lot of negative responses either. Folks have tendency to keep that kind of stuff to themselves.

Wow. I feel really bad that people have such strong feelings that they're willing to write off an entire population of people, most of whom they've never met. I think I understand most of the sentiments in this piece, but I'd like to call out a thing or two.

First, "But the agile community has at times embraced the far fringe of wackiness too. It's hard enough getting extremely detail-oriented analytical people to stand up and talk to each other every day. Getting them to put on a puppet show for their showcase is just a bridge too far. We need to tone it down."

I find this unfair, and it smacks of entitlement. "I don't want to think about what to listen to and what not to listen to, so stop saying things that don't interest me." Nonsense. Wackiness keeps fields from becoming stale. Copernicus was wacky. Einstein was wacky. Hawking is wacky. If you don't like wackiness, then don't pay attention to it. Let other people pay attention to the wackiness, apply bits of it, have success, and then maybe you'll want to pay attention to it, but don't tell us to shut it down because you don't like it.

Next, "I could list a dozen more questions which have multiple answers, depending on who you talk to. One guy says do it this way. Another guy says do it that way. It's enough to drive anybody nuts."

Honestly, what do you expect? With a diverse group of people with differing experiences, backgrounds, and cultures, do you expect them all to agree on issues of style and approach? There will always be schools of practice, but I don't see how you can expect a group like this to agree on highly contextual issues like "can we do traditional architecture together with evolutionary design". That just sounds like expecting way too much.

Next, "I worked with a lady who wrote an article asking "why are we complaining about Scrum teams not succeeding when they're really not doing scrum?" This attitude -- that there is a list of things that must be perfectly done and failure is a result of not doing them -- is basically religious in nature."

I apologise, but there's a basic flaw in reasoning here. There are some people who attempt to apply some convenient, short list of agile practices, find they don't help, then say "agile didn't work". I imagine you and I both have seen it happen. Pointing this out does not equate to promoting a religious adherence to agile. I have started using an analogy that I think communicates the problem well: I want to treat agile software development practices as tools in a toolbox that I use to solve problems as they arise; others seem to want to treat agile software development practices as levels to conquer in a video game. The latter is doomed to failure, on average; the former has a better chance of succeeding, on average. This is a more nuanced way of saying "if you don't actually /do/ agile, then it probably won't work", where "do agile" means "incrementally apply useful practices to solve real problems". Perhaps I have an advantage in the way I define "doing agile".

I have to say that you are spot on when it comes to cargo cult, trainers who can't do the work, magic bullet, reversal of team dominance, and non-answers to questions. Still, I want to leave you with one final point.

You have treated authors terribly unfairly here. You have decided that 99% of agile authors are incapable of writing honestly. As an agile author, I have to invite you to kiss my fat, white ass. I wrote honestly. I used examples that, as closely as I could remember, reflected what I have done on real projects and in real codebases. Where I was forced to conjecture, I admitted as much. Where I was honor-bound to admit that my approach might be unsound, but was my best guess at the time, I did that. When you put a number like "99%" on such a claim, you perpetrate the exact same kind of bullshit that you've written this entire article pointing out to people. You lose considerable credibility. Or worse, you /don't/ lose considerable credibility, because some of your readers would rather take your utterly unsubstantiated claim at face value. With that statement, you've launched your very own brand of snake oil. What shall you call it?


J.B.

Thanks for your comment.

I'll say this one more time: this post was about how people perceive agile. It's not about what a great book you or I wrote, or whether doing unique things can't help stale teams. It's about how people -- you and I -- take really good stuff from other people and turn it into crap. One of the ways we do that, unfortunately, is by relying on books, authors, and speakers to do our thinking for us.

So yes, I was unkind to authors. There is a large segment of the developer community that is unkind to authors, especially authors that fall into the "self-help" or "process" categories. That might not be fair, but there it is. Might as well get it out and deal with it.

I'm glad you got your dander up. That was the point. Many things about the community suck. Let's talk about them and fix them. If you found me offensive. I apologize. My point was to empathize and speak out for the folks who are angry about agile, not to be fair to everybody in the world. I'm not the most diplomatic person, regrettably.

I think I'll pass on kissing your fat white ass. But I appreciate the offer.

Note: This is a clean site. My kids read these articles. Vehement disagreement and passionately advocating your position is welcome. Name-calling and trashing is each other is not. J.B.'s comment was borderline. I ruthlessly delete comments that cross the line.

Fair enough - but how did it ruin your life?

Largely agree with your conclusions, in particular that to take upon yourself to promote Agile is also, of necessity, to accept the responsibility to explain it well and clearly. I do hope and wish the Agile community can turn its critical faculties on itself. There is some evidence it can do so. In 2008 the Agile conference had a track called "Questioning Agile", which I thought was one of the most interesting parts of the conference; I learned a lot there. (I'm hoping the track, which went missing the past two years, is brought back into the program in 2011.)

Some of your comments miss the mark. I've attended the haiku workshop you mention; I think we geeks vastly underestimate to what extent our work is literary in nature. More to the point, this was a session at a conference where about 40 other things were going on at the same time. I commend your friend for walking out when she realized the session wasn't a fit for her - I think her assessment of it as "fluffy" might have been a tad hasty.

Laurent,

Thanks for the comment.

I'm getting a lot of flack about the haiku! Never thought that would happen.

As I said in the paragraph, I'm in favor of trying new things. I think the problem is that different people are in different spots when it comes to their ability to try new things. Many times the "true believers" are willing to be a lot more avant-guarde than the average developer. For better or worse, fairly or unfairly, this reflects poorly on the community.

To give an example, I knew a developer in a recent team that refused to touch the nerf ball. The first day I was there he walks in and says "I'm not passing some silly ball around". I spent several weeks with that team, and the guy never touched the nerf ball. (Other team members would put it in front of him and remove it when it was his time to speak)

Now the nerf ball thing is silly -- it's just a stupid ball. Passing a token keeps us from speaking all at once. But to him, software engineering was a dignified profession and he wasn't about to stoop to a "bunch of silly games".

I grok that. Can't do much with it, but I grok it.

The problem, like I said, is that when you're trying to learn new things, there can be a bridge too far. Agile just kind of throws best practices from all kinds of industries into one big bucket. That's the beauty of agile, but it also has drawbacks.

I had a team once that painted their retrospective: they took a big sheet of paper and drew what they thought of the team. There was a landscape, a river, a road, some birds. Hey, I thought it was cool. There's no way I could get most of my teams to try it, but it the idea looked interesting enough to experiment with.

I hope that explains it better.

I love this article, but perhaps not the mis-guided title.
It outlines simply that implementing Agile badly has a negative affect on the workers and business that project management is supposed to support.
Although I have not seen the article proclaiming the Waterfall methodology ruined my live and lost me millions of dollars.
I feel the significant difference that agile should proclaim is it's continuing flexibility. Small steps and iterations that can and should be evaluated regularly mean that you can change as the development progresses. This is not only true of the product development but should also be true of the Agile implementation and how the team works and functions.
As soon as you have a group member saying that 'I have done agile and it works this way' then it is doomed to failure. Return to the manifesto, put things in and take things out as required and the Agile principals can be of great beneficial effect.
Having said that, it is not for everyone and there are teams that I would say outright, Agile will not work for those people, they wont get it and wont contribute to the process. It is not the golden ticket but it can be wonderful if you get one.

Wow, you sure wrote a lot of words defending a managerial process. That must mean it's good.

For me you were a little bit extreme, agile is quite good, though I've to agree with you about two days to become a SCRUM Master, it does not exist in practice.
The critical point with agile are:
1 Jrs will never able to work with/implement it.
2 Only years and having worked in several projects will give you the expertise to make run one successful agile project, the same way one senior struts developer will never be a good system architect.
3 The team must have the 'I can do it' attitude and be a experienced and organized team (this is one of the most hard to put together).
4 Governmental environment for sure is not for agile, due the lazy mentality.
5 Follow the book does not help if you don't have a good experience.
6 Does not exist one standard way to make run agile projects though just conventions, good practices, nor formula.
7 Use TDD with average team, will not help, you'll need a good team.
8 Need have access to the people that have the knowledge about what is being developed, no access means lack of information and fail.
9 At last one person must have expertise in all project phases, like from make the interviews up to deploy in production environment, and know most of configurations for the environment to be used, just code does not help.
Because this is not easy implement agile in medium to big projects. It's expensive and very costly for the company have several good guys working with it, and means that after project done, they will need keep a few just to enhance the system after or do the maintenance.
Before start one project you should know these points before to decide for what methodology you should go.
Managerial process is very problematic in all way, there are several problems that will avoid you to reach the deadline date with the system working nearly impossible.
Of course there are successful histories in all methodologies, though doesn't mean that it will work all the time.

"Although I have not seen the article proclaiming the Waterfall methodology ruined my live and lost me millions of dollars."

Charette, Robert N., "Why Software Fails", IEEE Spectrum, Sept. 2005.
http://spectrum.ieee.org/computing/software/why-software-fails

I completely agree with you on Agile...I do keep the same thinking...

Daniel,

Perhaps the haiku thing and the nerf ball thing are deeper than we realize. They remind me of the essay in the classic Peopleware about how the smell of popcorn is "not professional".

Cognitive scientists recognize two kinds of intelligence, "fluid" and "crystallized". The latter is declarative knowledge, like a doctor learning their anatomy. The former is harder to define, more creative. The activity of crafting good software depends crucially on fluid intelligence, even as it requires a good dose of declarative knowledge.

Now to me, someone who lets their status hangups get in the way of learning new things, when fluid intelligence is at such a premium in our work - they are the one acting unprofessionally.

I can see where you're coming from - pushing people beyond their limits doesn't do good. And I have heard about situations that really crossed the line: one manager I heard of insisted that in "standups", so-called, people were to report in with personal information, like what they did over the week-end. Or again one "company retreat" where we were forced to go on stage and put on a play lauding the merits of the company.

To me the principle is clear: if it helps write better code, I'll take a stab at it, with humility. If it doesn't have to do with the job, it's optional and I'm allowed to pass. To be on the safe side, I operate on the rule that "pass" is always a legitimate move.

Let's be clear that "dignified" doesn't write code or fix bugs or deepen our understanding of how we do either. It's a misplaced hangup. I can respect it, but I don't have to condone it.

I really agree on the three principles, that's what should never be forgotten.

Agile should be agile in adoption as well - it's very hard to start from scratch and adopt each and every "agile technique" at once, and it should be a developer's demand beyond the manager's will - if something works well, there may be no reason to change its.

… and the trainers like to mention Toyota quality – the original of Agile. :)

Agile is a rehash of already existing concepts. It doesn't add much to the whole development challenge as a whole.

What developers need are new tools, new ways to accomplish their goals, and not some "remixes".

Pragmatic model-driven development (hence not UML/MDA) coupled with code generation is one step forward to a better development future. It does require some mindset shift but pragmatic approaches are within reach of the mere developer mortal. Check ABSE and its AtomWeaver IDE for an example of a flexible model-driven approach that supports iterative development.

In many organization 'agile adoption' just translates to order 'work 50% faster without any change'. Instead of hiring consultants, money can be much better spend by buying new server/profiler developers were asking for last six months.

And agile is like MacDonald staff training, it is made to squeeze last bit of juices out of developers. All that meetings, micro managing, pair programing, it pushes until burnout.

Btw I think I found definition for agile: bulls*t

@RuiCurado

I am not sure what you mean by agile being a rehash of existing concepts. Or that agile does not address the challenges in development.

Could you expand on both of those points? Maybe in a separate blog that you link here so we don't overwhelm this post?

Given the code-generator references, I am wondering if you are postulating that the impediments to development are in the creation of the code itself?

Color me curious :-)

I often hear statements about agile that seem to be based on the assumption, "Geez, everything else in our business is managed and performing well, now we just need a great software development methodology." If that is the case, then there is a whole business model out there voraciously hunting these companies down who will repeatedly chalk any failures up to, "You're not doing it right."

Well then I guess I just been doing iterative/incremental dev all my career without having but an etiquette to it. It is a little like design patterns, they are nothing more then standardized algorythms. I been in IT for over 14 years and there are always things that showed up in the litterature that seemed mysterious like Agile dev or design patterns and mostly they turn out be just be etiquettes on stuff I already know. But in an interview, when you get asked the question and you don't know what the etiquette means. It does make you look like a dinosaur.

Caouellet,

What I hear from teams that "get it" is usually something like this "Oh wow! This is just the way we used to do things? Remember? Back when everything worked so well? We just didn't have a name for it then"

So I know where you're coming from. Many times I've been unable to recall the name for something that I've done for years. Had an interviewer once ask me about how to normalize a database. I could describe it, and even remembered that there were five normal forms, but heck if I could remember the names of any of them.

Having said that, just remember that "best practices" is also a moving target. The trick is being able to keep an open mind to try new best practices. Many of them will not work, or will make you look silly. So what? If it improves your work, it's worth looking a little silly. You don't have to learn the jargon, but you do have to be willing to experiment a bit. As you know, in this business there is no standing still: either you're learning more and becoming more valuable, or you're falling behind.

Hi,
Great article, I am totally agree, after doing scrum for almost 2 years I found that sprints sucks, sprints are a good way to deliver to the client/final user something usable but working on fixed sprints may lead to time wasting, what happens if we can handle for instance 15 points per sprint but the stories in the backlog does not allow the team to fit 15 points in the sprint, we take more or less points, but we never change the sprint’s time frame, if we take less points, we are wasting time, if we take more points, then we compromise the team’s sanity, usually when the team is under pressure, the team fails.
QA and bug fixing process usually does not finish in the same sprint, so the team finishes working in two or three sprints at the same time, It’s easy to be discouraged when the sprints are not closed.
Having a fixed time frame and the possibility of go back and fix something from the past it’s the perfect environment for the student syndrome and the Parkinson’s law, also it seems like there is not ambition or reward to finish something quick(with quality).
Also, the way the story points are calculated, that is just think on the effort required to accomplish the user story can be tricky a lot of times we underestimate or forget to calculate the effort required to integrate the user story with the rest of the application.
Also it seems like we forgot that we can work on parallel, and that we can create a strong code foundation(framework) so we can build the app fast and with repeatable results.

This is a great article and the discussion here is very interesting. As an outsider to Agile, and having seen, heard and read many things about it, here's my point of view on it:

From what I've been able to tell so far, there's no "Agile". It just doesn't exist. Why? Because basically, what we have are a set of vague principles and nothing else. No clear methodology. No definite tools. No nothing. It seems to me that the reason why it's all so confusing is because people just slap whatever they feel like around the basic principles and call that "Agile", so what we end up with is a random mish-mash of things that more often than not don't have much to do with each other.

The almost-religious attitude also gets to me. "Be adaptable" and "listen to the customer" doesn't have any practical meaning to me. You might as well take a few days to tell me "do a good job"! Principles are fine and all, but what truly matters is how to put them into practice -- that is, without actual techniques, principles are useless. You won't become a master at piano by listening to piano, reading about it, and watching people do it, but you sure can tell that they are actually playing piano! There's nothing "avant-garde" or "revolutionary" or whatever about it and most people will be able to wrap their mind around at least the basic concepts of it because "playing piano" and most of all "learning how to play piano" implies using specific tools and techniques in certain ways, and that doing so will yield certain precise, measurable and repeatable results (making less mistakes, being able to play musics of higher complexity, being able to play faster, etc). I have yet to see any of that in Agile.

I can relate to the people who don't like the nerf ball, the haikus or whatever other "far-out" ideas the "Agile people" can come up with, and I can tell why I don't like it either with one simple sentence: what the hell does that have to do with software development?! I love poetry, I write some in my spare time, but seriously, what does it have to do with analysis, programming, code, the customer, or anything else? It has nothing to do with being uptight about my job. What practical advantages will it give me and my team? How can it be useful to software development? And, most of all, where's the cold, hard data to back it up? There's only so much faith and marketspeak one can take before being tempted to throw the whole thing out the window. I'm willing to try it, but I'm not willing to risk wasting my time and the time of my team if nobody can come up with anything better than "you have to try it to understand". No, I don't have to. This is a tool. It is not special, and it does not deserve any special treatment. I will not start using it if I don't have any way to tell if I'm doing it right, or even if I'm doing it at all!

In closing, I think that the Agile community would gain a lot by realizing that what me and a lot of people like me need are not "anecdotes", "coaches" and "stories" but techniques, clear structures and measurements. We need to have a way to tell what Agile is, what it is not and what are its pros and cons before we try it, because we're not going to risk our projects just because some people told us "you need to try it to understand" and "it just works". Also, I have heard many times the argument that "obviously", when Agile doesn't work in a team, it's because they were "doing it wrong" or "were not really using Agile". But I think maybe it's time for the Agile gurus to get their heads out of their a** and entertain the idea that maybe, just maybe, it's because their paradigm 1) isn't the right one for every situation and 2) isn't complete yet and needs some more work. The onus is on them, not us.

So thank you for your article, and have a nice day!

We've recently switched over to scrum at my company, and for the second time in my career I've been stuck doing this. It's so tiring on developers, all these iterations. Two week cycles, over and over, on mini tasks, makes you feel more like a code monkey than an actual developer.

It is good to vent, even better when it is productive venting (and this is.. for the most part)

We risk becoming one big agile echo chamber.

I personally enjoy hearing failure stories at conferences because it keeps us honest.. but that clearly isn't the case with everyone as I watched half the room walk out during one this year.

"1) principles not practices, 2) attention to people, and 3) always be adapting"

I wish I'd read more of this from other "experts", along with hiring for talents instead of skills.

Anonymous,

Yes, it can be very tiring.

But perhaps you are looking at it the wrong way.

A very smart man told me something back when I was a teenager: software and technology is one of the few businesses where we create our own reality as we go along. His point was that the way we work on making technology is almost completely up to us. Come in late, leave early, code in the park, go for a swim, have a funny hat day -- whatever we dream up is what our work is like. This is much different from a carpenter, who has to go to the job site every day at a prescribed time, work with the same materials in the same fashion, doing the same thing (mostly) his grandfather might have done.

Yes, I know this is the real world and it doesn't work that way, but it works that way a helluva lot more than most folks realize. We create our own little prisons with the systems we create, and then we unhappily live in them.

So if I were you (or your team) and I found this agile stuff just grinding the hell out of me? I'd be bringing that up every stand-up. It's an obstacle, an impediment to team performance. It needs to be fixed.

(Hated to give you a scrummy answer, but dude, people are what make software happen. If it's a prison ship, that's no good for anybody, including the folks writing the checks)

Hope things brighten up.

David you hit on a couple of good points.

I really don't mean to be negative, but most of us really don't want to hear that some of our concepts aren't working too well. It's easier on the mind to learn something, "nail it down", and then take it for granted. Many, many times I have seen agile trainers and supporters put down teams because they are not doing TDD. That would be fine if those same people had just gotten off a few TDD teams that were styling and profiling, but many times the folks making the judgment haven't coded anything much in years. When we do this, it's not helping anybody -- it's just badgering people with phrases we've learned to parrot. Developers see through this.

Another pet peeve is when some asshole falls back on "it's emergent" You ask something like "how can we make sure the design is in place for this new space shuttle code?" and the answer is "it's emergent". That may be true, but it's not an answer. It's just crap. It's like saying you can get to the moon by jumping up in the air a little ways, then keep learning how to jump higher. Something is missing somewhere

The "hiring for talent" point is especially insightful. Agile and Scrum critics will tell you "sure, if you had smart people, it wouldn't matter what process you used."

This is an unfair criticism of the agile movement, but there is a great deal of truth in there. Probably the best response I can give is that agile principles let you know when crappy teams are working for you a lot faster that other things do. But that implies that you're willing to get rid of the team and hire for talent. I'm not seeing a lot of that in industry. Industry wants little cookie-cutter developers all making the same rate with the same benefits package. Uniformity is central. The idea of a great 7-person team doing the work of 50 just doesn't compute with these folks.

@Ben

"without actual techniques, principles are useless" - agreed; here are some specific you could try: TDD, refactoring, fully scripted builds, continuous integration, user stories, acceptance tests, personas, velocity-based planning, retrospectives, burndown charts, standup meetings, pair programming, information radiators, team auditions, five why's, CRC cards, coding dojos...

"what the hell does [haiku writing] have to do with software development" - there's a reason we call it *writing* software; it's a literary activity. Deliberate practice at this kind of thing will improve your fluid intelligence.

I share your concern that 10 years on we still lack some hard empirical evidence backing up some of the Agile talk. I'm working on fixing that, personally, and know a few people who are. But the lack of "scientific" studies is not too surprising, 10 years isn't such a long time.

Consider that "software engineering" has been around for 40 years, and can hardly be said to have solved the issues it set out to solve. See http://bit.ly/d9h0Au

Again, note that no one in the Agile community claims that everyone should be into haiku writing; it's one of the 40 things you could take an interest in at the main Agile conference a couple years ago. Some of the more mainstream topics of the conference I've listed above.

The issue that I have with agile is the fact that it has become so intertwined with Scrum (and to a lesser extent extreme programming and a few others). As you've pointed out, agile is nothing more than a set of principles. Just because I do Kanban instead of Scrum, doesn't mean that I'm not agile. Different people, different processes. However I've been told multiple times by various individuals that because I wasn't doing process "X", I wasn't agile (note that X is whatever process they're doing).

Also I find it funny that in the manifesto it says not to worry about the process/tools, but instead worry about the interaction/individuals. And yet everyone seems to focus on the process and the tools...

"Whatever it is, helping people see things and try things they haven't seen or tried before -- and then letting them decide whether it's working for them or not -- is a pretty good business to be in."

Thanks for the great read Daniel.

I was the stage producer for "Questioning Agile" at Agile2008 in Toronto. (Not many people know that it was originally called "Counterpoint".)

There were some great talks on that stage that invited criticism and others that provoked honest questions about what notions of Agile meant in different contexts. One of the talks invited people to come up with facets of Agile, and the result was a list of over 60 ideas (posted in a newsletter earlier this year: http://www.quardev.com/newsletter/2010-03-11-979104237)

I offered to run the stage again for Agile 2009, but the conference president brushed off the idea entirely. I hope this post prompts them to consider having one in 2011 in Salt Lake City, just like it's prompted dialogue here.

I think that's a great idea, Jon.

We keep telling teams how important the retrospective is, yet do we ever sit down and have a retrospective on the agile community? I can't imagine that we would get it perfect the first time around.

Another idea along the same lines that I've found useful is having two "experts" argue about something. Anything. Many concepts are difficult to understand without context, and for some reason an argument (hopefully a friendly one!) provides the context that some folks need to make a decision.

Maybe I'm wrong, but people on the fence want to hear the skeptics. They want to know where all the warts are. I think perhaps people on the inside of the community either forget that or are uncomfortable with a lot of self-criticism. It can be very disruptive.

One of the things that bug me about conferences is the predictable format. We had a problem. We did X. Now everything is great. That gets a bit old. Surely the community isn't full of super-star performers that hit homeruns every game. Last I checked the CHAOS study was still mostly accurate -- most teams fail in one way or another. The agile community should be big enough to make these failure modes a high profile part of the conversation. Instead I think we've drifted way too far into boosterism. My opinion, for what it's worth.

9 women CAN all have babies in one month. I don't get it?

This article and comments make several bells ring in my head. Especially the point about Agile=Scrum. Agile is, if I understand the concept correctly, about tailoring a process that works here and now, not about following this or that existing system. Whereas Scrum is often understood as the following of the rules described in this or that book or article, without clear understanding why these rules were created and whether they are suitable for the particular project. As a result, new methodology does not help much (or even makes things more difficult, because people force themselves to follow things they don't completely grok) and everybody gets disappointed and starts blaming "agile" in general.

Great Insights!, although I do not agree completely on all the points, there are several that I consider some comments very honest, direct and justified.
However I do have some questions/comments I would like to make:
1 - Through all the time there has been good/bad SW Projects and good/bad SW Dev teams, what are the specific and general points that those teams and/or projects that are/were key to succeed? Incredibly most of the time those teams can share the same kind of answers: Communication, Early feed-back, Collaboration, etc. Those are not invented but honest ones came from real teams, so the question is:
Because some other person that tried to understand these topics and try to explain/interpret to other people, does that means those recommendations are invalid?
Are these themes or concepts only valid if they comes from the guys that lived the project?
I don't think so...
I mean those principles, guidelines, ideas, or advices are good one independently from who is the channel/person that provides them.
I agree that not all the scrum-masters have the knowledge or experience to become real mentors, but the concepts, ideas and advices still have the same value.

2 - I thinks is not very clear to confirm or assert that a scrum master have to have the specific technical knowledge to become useful or provide value to a team.
There has been around the world also many teams that are leaded by a scrum-master that do not know the specifics of a language or technology used by the team/project that at the end succeed.
There has been also interesting stories that comes from the members of those teams that agree that is not a requirement to have that kind of knowledge to get a successful project, even more they do not recommend that, because they consider it provides a good balance and health to have someone in the team with another point of view, that should not be entirely immersed in the technology, of course this guy have know the general terms and has to have a good idea of the stuff that the team is doing, but not necessarily to be an expert on the technology.

To assert that it is strictly a requirement is like to assert that in order to be a pilot of a car, the drive has to be a mechanical engineer, or in order to be an airplane pilot, the pilot should have to have experience building airplanes.
This same idea applies to other areas like sports (football, soccer, box, tennis, etc.), rocket science, arts (theater, movies, etc.).

There are snake-oil salesman. No argument. However, when you decide to paint 99% of a community with the same brush, you're simply selling your own brand of snake oil. It's not about being diplomatic. It's about doing exactly what you tell others not to to. It's not about being fair. It's about wrongly maligning people you don't know about work you haven't evaluated. That's just as reckless as the things you accuse others of doing. Worse, perhaps, because you appear to know better.

To the earlier posters, the reason "the community" tends to focus on tools, processes, and procedures is "the community" is pretty much completely staffed with and run by process driven analytical thinkers. Agile is more about interactions, relationships, and communication; "fuzzy" concepts that tend to make the average engineer break out in hives.

It's a completely personal observation, but I have noticed the teams with the most difficulty in "getting" agile concepts are those staffed by engineers and technicians almost exclusively. The teams with a good mixture of backgrounds, training, and expertise are usually the ones claiming it to be nothing more than what they've been doing for years.

J.B. -- we've already established that I'm not going to kiss your ass, so my patience is wearing a bit thin. To the best of my knowledge, aside from Uncle Bob (and just how does one become an agile uncle, anyway?) I haven't mentioned any author or book by name. If I said all politicians are crooks, hell if I would expect my senator to call. You are getting your genres confused.

Yes, I made some sweeping statements. It's a rant -- the purpose is to demonstrate emotion and to see if others in the audience felt the same way. If I started adding 17 caveats for every statement I made the thing would be as boring as an encyclopedia. People generally feel this way. There are a lot of them like this.

Why don't you take all this anger and vitriol and go do something useful with it, like doing a better job as an author, or a better job as an agile spokesperson? Look, for better or worse these emotions are just reality. You can continue to vent on me or you can learn from this. Not going to phase me one way or another, I'm not going to be bullied into saying my feelings were invalid, but if I were you, hell if I'd spend time acting like jerk to some random guy on the internet because he stepped on my toes when I could be doing something more useful with myself.

I hate to get personal, and I don't know you from Adam, but is this how you act in retrospectives? Somebody says they feel like all of X is a certain way, and you go on the attack? I'd like to remind you that nobody brought you into this discussion. You came voluntarily. Try to add something of value. If the only thing you've got is "all generalizations are false" then let's agree to that and move on.

And as for the "you're in this too" argument, all I have to say is that I don't have a book, don't have a brand, don't have a tool, don't have a certification, and don't have a membership or an alliance to anything that might cause a conflict of interest. If next week "Best practices of iterative and incremental development" starts going by the name Pretzel, I can immediately pivot and be a Pretzel consultant. All I provide is the mentor-mentee (or master-apprentice) relationship that has been missing in this industry for way too long.

I think we're done here.

"Robert Martin (another "guru" and agile consultant) claims that any code not written with TDD is "stone age " code including such things as Unix and such people as Norvig and Linus and Zawinski who've built more code than he can dream of. Dalke poked holes in his TDD "kata" which never got answered Fraud."

This is a lie, and the author is the fraud. What I said was: "Anyone who continues to think that TDD slows you down is living in the stone age. " This is from the article: http://blog.objectmentor.com/articles/2009/10/06/echoes-from-the-stone-age

You should have published the author's name so that he could take responsibility for his careless accusations and flagrant inaccuracies.

Thanks for dropping by, Bob.

It was a private conversation from several months ago. I stripped out as much as I thought I had to. The purpose of quoting him was to provide feedback into the feelings of some developers, not to relate facts. This is an editorial, not a newspaper article.

If you'd like, I'll email him and hook you two up. As you can see, he's a pretty angry guy.

I think people can be totally honest, but wrong. Large-scale madness is a repeating pattern in our society. I've been working in agile teams for the last seven years. It's nicer than working in waterfall teams for the most part, but it's far from ideal (for me) in most cases, and I think it's far from the final word on software development.

I think we should all entertain the possibility we might be wrong. That's why the context-driven school works for me. "No best practices" and the means to answer "it depends" in a more meaningful way. The same thinking lets me maintain a healthy suspicion of my own practices.

Good call on that on my friend. But what if you're a "Fake Success Story?"

I was working for a company and they tried to implement Agile. It destroyed the team because they never understood it. First it was by the book, strict and no exceptions. Then after the first 30 day cycle they realised it wasn't working so they changed it. Another 30 day cycle the changes weren't working and the team were getting annoyed with the whole idea. 30/60 days on everyone had left or was looking for new jobs.

This is a great article, I have come to loathe agile dogma being rammed down the throats of the young and impressionable and parroted back to me as if its a law of nature, so it's nice to read an intelligent and honest article. I've been leading a small team of top devs for a couple of years, and I reckon:

• Do what works for your team, not just what others say is necessary. Never do anything which cannot be shown to actually help your team.

• TDD is useful for some situations, but its no substitute for using your intelligence or doing good research. Do _enough_ design up front, no more and definitely no less.

• Scrum is an embarrassing victim of its own hype, that makes developers appear jingoistic, entitled, self-indulgent and out of touch with reality.

• Iteration needs to be a fluid process driven by the scope of the problem being addressed and the needs and deadlines of the business, not artificial story points and arbitrary timelines.

• Oh and Bob, I enjoyed your "Clean Code" book, but I lost a lot of respect for you when you declared that developers who don't use TDD are 'unprofessional'.

I am convinced that 10 years from now TDD will have disappeared into the dustbin of programming history along with RUP, UML, Scrum, and Agile with a big 'A'. Meanwhile, "professional" programmers will still be busy getting stuff done for real customers, not writing books and pontificating at conferences!

We learned in college about RapidDev tools. Sure they can get you RTM faster but they're often buggy and bloated. We were taught to stay away unless there was some reason the project had to be rushed. That was 15 years ago and still rings true today.

Sounds like a Scrum guy's perspective - according to the post the most important aspect of software development is collaboration and learning.

There is nothing wrong with that, but the agile manifesto is somewhat broader, and most methodologies seem to take that into account (even though the manifesto is by no means a "standard", of course).

Correct me if I'm wrong - the agile manifesto seems to be the first document that says that minimalism is a desirable principle in software development. It has some more things to say about what you should deliver, when you should deliver (continuously), and those are about the attitude of engineering, not of management and collaboration.

That is not to say that collaboration/project management is unimportant, but it seems to me that many people are only willing to adopt the management practices of agile. How can you claim that your team can continuously deliver "potentially shippable increments", without proper attention to engineering? For example: Even with TDD you cannot guarantee that you haven't introduced code into your product that does not serve a purpose (from a business POV), but at least your team will have control over the product you are building at all times.

I don't know of any team that can say with confidence: "We can produce a list of all the features build into this product within five minutes". But if you keep adding/removing/changing features with each iteration, having at least some tests to document this is an essential prerequisite.

Keep Posting Man, Thumbs Up!

First let me tell you i had no idea of what Agile mean when I first read the post.

Now, if I understand it correctly, I think Agile is for programming the same as ITIL is for System Administration, a list of best-practices.

I took the ITIL courses this year and I recall thinking "but... i'm already doing this, maybe not exactly this but is really similar". After reading all (the post and the comments) I think is really like that, best-practices for developers,programmers, etc.
I think we all have to be a bit human here and realize that best-practices and reality are too far to use them sometimes.
After a few ideas I got from here and what wikipedia and google is telling me about Agile and TDD i've just learned that I'm already using it! Since always!! (thank god for my teachers though).

The kicker for me when it comes to Agile is how evangelized it is -- both in print and on the web -- as demonstrated by many comments in this thread.

Many people have succeeded using an Agile approach, just as many people have succeeded using Waterfall (egad!). Many projects of both types have also failed. At the end of the day, it comes down to team dynamics throughout the company. A great Agile dev team with a poor PM will fail just the same as a "traditional" team would fail with a poor PM. Before taking up Agile as a way to fix a seemingly broken project, first perform a frank analysis of the company itself to ensure that it's not the people themselves that caused the failure! If the failure turns out to be personnel-related, the failure will repeat itself regardless of the process used.

Agile provides a mechanism for a very fluid approach to software development that is useful when customer interaction is frequent. It also defines a lot of very good practices which can be boiled down to "pay attention to the code you write, stupid!". Just be sure that, before jumping head-first into a new process, the process is a right fit. No Irish screwdrivers, please.

Remember: good people will develop good products, even if they don't stand during meetings!

Waterfall still works (gasp!) in situations where your problem & solution are mostly known.

I find that Agile works when your problem is mostly known, and your solution is mostly unknown.

You can also run with mostly unknown solution & problem, although that takes a bit more progressive thinking than I've seen from the Agile community so far. (startups are king at this approach, and generally skip out on the big conferences)

As long as this is a healthy debate, let's keep going. If it devolves into name calling I generally tune out.

-David

David -- you continue to be spot on.

The big open secret in the XP and Agile communities is that these philosophies evolved around business web development. TDD especially seems fitted for fixed-problem, OOP environments. Unknown problem and solutions lead more to functional programming and really unusual project configurations.

I think perhaps where I take my leave of the agile community is that I focus on startups. To me, a startup is the ultimate agile team: small group of guys, fixed timeboxes (mostly), and either delivering value or dying. In my mind this is what we should be looking at. Sure, a lot of it doesn't translate, but there are opportunities for cross-pollination that I think both communities should not overlook.

Some introspection is a good thing for all of us. But many times both the system and our minds are wired against effective introspection. I just completed another blog article on why agile feedback is so hard to get. I'll bet you for every hundred people that read this rant on agile adoption, maybe only one or two read the next one on measuring and promoting agile (best practices) inside an organization. It easy to rant against the other guy. It's much tougher to actually change something for the better or to take an honest look at something you've created. At least that's the way it is for me.

I'd love to pick the conversation up at the bottom of the next article if anybody is game. My opinion is that we should move from ranting to beginning to have some practical ideas for working through some of these issues.

http://www.whattofix.com/blog/archives/2010/09/agile-metrics-y.php

I have been around for many years in software development, I had my share fair of great and bad projects. Some my fault , some others, and some of a mix of each other. I had talks to my fellow peers about this. One thing is true, the authors of these books, mostly their 'golden' project was in reality a complete failure.

More than often, you hear about how great it is, in reality, its lipstick on a pig. Back in the 1990s it was R.A.D., before then there was the typical design, develop, and release. To me, its all the same. You *have* to have an idea what you want done, to even start developing it.

To those authors that like to sell the idea of 'Agile' or Extreme Programming, I present this. In the 18 years of software development, I have completed more projects than you. Your method of development is great for management, but not getting it done.

I prescribe to the "Get It Done." Method. Let your good developers do things they need to do, and Let your 'corporate/average/newbie' learn as much from the senior ones in order for them to come up with the best way for them to 'Get It Done.'

@Laurent Bossavit

/* I share your concern that 10 years on we still lack some hard empirical evidence backing up some of the Agile talk. I'm working on fixing that, personally, and know a few people who are. But the lack of "scientific" studies is not too surprising, 10 years isn't such a long time. */

Do we pay attention to the evidence that is available or do we bend the evidence to fit our talk?

"The significant development process measures are the use of
• An early prototype
• Daily builds"

[Early prototype: The extent to which an early prototype was shown to customers, as measured by the percentage of the product’s final functionality that was complete when the first prototype was released.

Daily builds: if design changes were integrated into code base and compiled daily]

MacCormack, A., Kemerer, C.F., Cusumano, M., and B. Crandall, "Trade-offs between Productivity and Quality in Selecting Software Development Practices." IEEE Software 20(5) 2003 78-85

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.61.1633

May I suggest that "Agile software development" simply means:

"Get to the next working prototype as quickly as possible."

This seems to me to be the common nub of everything I consider to be "Agile."

If you start with that nub, any added processes and methodologies can still be consider members of the class called: Agile. Processes and methodologies that do not focus on "Getting to the next prototype as quickly as possible," should not be considered Agile.

Very good post. I'm a software architect and I continuously see this behavioral pattern in self-proclaimed architects. They do fancy UML component, interaction, package diagrams but when it's time to deliver the goods they always fail because they are mere diagrammers who don't write frequently code. As an architect, you have to get your hands dirty to now what works and what doesn't, to get a fell of the current technologies and tools available. You really nailed the point, sadly, but that's the truth with these "experts".

Nice article. I agree that Agile is misused in a lot of places for a very dogmatic process. Real agile is just being flexible and doing iterative and incremental development.

Scrum on the other hand I think adds too many flaws to the process. Here is my take on what you could encounter if you use scrum straight up:
http://hoothoothoot.wordpress.com/2010/08/31/its-a-marathon-not-a-sprint/


Let me put this another way:

1. Let us consider the possibility that "At least SOME highly paid consultants that come in and preach something, and then leave, are scammers".

2. Further let us consider that the buzz is all on Agile right now. There's not a lot of call for "Waterfall consultants". When agile loses its shine, and its losing it, believe me, then the fraudsters will move on.

3. Like all fads in programming, and Agile IS a fad, of turning the pet peeves and best practices of a small cadre of elitists into a big fad-powered cadre of elitists. That doesn't mean it contains NOTHING good. Customer involvement is good. Keeping the "user stories" readable by non programmers is good. Scrums are better than having people run off all on their own all the time. But they are NOT the only way to solve the problem, and NOT the best way for EVERYONE, merely the best way for many people right now. Let's hope the name agile dies, and that the good ideas inside it continue. Unit testing is good. TDD is not all its cracked up to be.

4. Anybody who doesn't GET Agile, or LIKE Agile is merely not a very good programmer. This is my favorite reaction from the agile community, and the one that does more damage to its credibility than any accusation made by any Agile-sceptic, such as myself.

Warren

Agree with Gist of article.

No silver bullet solution and so on.

Citing Resistance of Change to Agile Practices as a valid reason to stay with Good Old Days of Waterfall, IMO, is living in Fantasy World of Alice in Wonderland.

Any or all changes are painful. Yes, coming to work every day and every week and saying that what I did yesterday really did not finish and have no outlook to when the TASK will be done is PAINFUL. Facing reality on day by day basis is painful. Postponing Reality check to end of month is potentially fatal to a project.

Interesting post. I think you mean "bloviate about Scrum", rather than "bloviate about Scum", which is suggestive of something else! :)

The rest could do with a check, too. Sorry to be picky, but I think that a well written and checked document carries more weight and is more likely to be listened to than one with several errors in it. And this should be listened to, as it's good, honest stuff.

The following advertisement appeared at the top of this page: "Learn SCRUM in 10 Minutes Free, fast-paced and entertaining HD-quality video on SCRUM." :-)

LOL!

David, you've figured out my master plan: make money by picking on a topic, then have advertisers pitch people that stop by. NOT!

In all fairness guys, I've made something like 30 bucks so far off these ads. If anybody has a problem with that being a conflict of interest, I'm happy to donate it to somebody. Maybe there is a support group for ex-XP'ers. :) Or a charity for folks who know everything. Maybe there's a nursing home for old agilstas. They can have daily stand-ups, work in sprints, and do pair-applesauce-eating.

I wish there would be more interest in "where do we go from here", but alas, people seem like they want to hear things that are either totally positive or totally negative. 50 thousand people want to drop by and read an expert honestly talk about problems in his field. 17 and a half want to continue the conversation about what's good about it and how can we make it better. Something wrong with that.

Your post was a delight. Thanks. However, I have a short observation. It becoming harmful to consider Scrum as Agile, in the sense that if Scrum does not exists than agile does not exists. Agile is many other things also. It is Evo, RUP, AUP, DASDM, CrystalClear, Lean, XP, Kanban and so forth. I would say that even Batman is agile. Now, putting all these in the same salad bowl is not fair. In my humble opinion, it is a difference between being agile in a XP, RUP, or Kanban way. You mentioned scumster so, as far as just in Scrum you find Masters (except Star Wars, of course) a better title will be: Scrum Ruined My Life.

Ioan,

Thanks for the comment.

I'm not sure I followed you completely, but this post was really not meant to be about Scrum. It was more about the tendency people have to take good ideas and run too far with them. You are right that Agile covers a lot more ground than just Scrum -- and wherever there are people teaching best practices for iterative and incremental development, there are people making broad generalizations that just don't work. And there are industries set up to cater to the people that feel that way. People react to this, even if most of it is silent. I just wanted to take an honest look at some of the reactions.

And as far as the title, the idea is that when we implement these things we can hurt people and make them pretty angry. We can destroy teams, break friendships, and tank companies. I don't think a lot of agile folks want to talk about that, because it's just automatically assumed that "agile is goodness". But anytime you muck around with how technology is developed -- if you're really changing things, that is -- you can really screw things up big time. And never know.

This is a great article! It tells us much about the pit falls of slavishly following any methodology (not only Agile).

Any software team should have the talent and experience to modify their approach as system evolves. Leading the team is about setting goals, supporting advancement, managing uncertainty and delivering quality. Either you've got it and you can do it; or you don't and you can't.

Interesting article, and very interesting comments.

A few points to consider:

I wonder if the quality of the people on a team are considered? A team of complete nimwits is likely to fail regardless of the consultant or process they use. Likewise a team of stars will often succeed in spite of the process used. The interesting space is how techniques like Scrum compare with more traditional development for "Average" developers.

Should we consider development organizations as organic constructs. They are living organisms that can get sick, die, and flourish. They need food, water, exercise. They can have enemies and friends.

Techniques like Agile development are ways to manipulate the environment and affect organizations. Consider it like exercise, or eating right. Or perhaps a bad virus (or even cancer?) Agile techniques should have a positive effect on an organization's "health"

Managing the individuals or the process to achieve a goal like "doing agile" is likely to be counterproductive and might even kill the organization. Introducing these techniques and paying attention to the organizations "organic" reaction is more likely to be successful.

Mike

Agile for Agile is really not a great idea. The business must drive the development model and more importantly the overall process. Here is a great white paper, focused on Software Development Management but echoes what was written in the blog.

http://www.parasoft.com/jsp/products/article.jsp?articleId=3222&redname=agilesdm&referred=AgileSDM

I was very pleased to come across this article which basically affirmed my own beliefs concerning the state of software development today. In the 70s, the management gurus came up with Management by Objective, the be-all end-all management technique that would earn practicioners vast sums on the lecture circuit. Why should software development be any different and not have its magic bullets and sacrosanct buzz words; Agile, Waterfall, Scrum, whatever.

Management by Objective did not transform bad managers into good managers, it just gave them a smokescreen and stacks of management reports to hide behind. I've seen the same outcome with Agile, Waterfall and Scrum. Good managers don't care what you call it, they understand the fundamentals of what good management is, a few basic operating principles and work more and talk less.

I led a very small but very productive development team for a mid-size engineering firm. We didn't use Agile, we didn't even use Microsoft Project, which many companies insist on today. What we did was fully define the task before commiting a single keystroke to code. There was no iterative or incremental development either. It was a more modular approach to cut through the spaghetti and meatball coding of todays development environment. Yes, defining requirements took considerable time, but the result was solid software that ran first time, every time and always exceeded the users' expectations.

There is no magic bullet. Either your project manager has a talent for task definition, organization, leadership and communication or he doesn't. No amount of "tools" or "methods" can compensate for that if its lacking.

Tony,

Thank you for your great comment.

I agree -- there are some universal principles. Good teams tend to do well.

The only thing I would add is that "best practices" is not just "stuff we used to do" Hopefully 50 thousand people trying to do things better since the 1970s would have come up with some good ideas. It's not like the whole thing is bullshit.

So for all those "we did it a lot better back when and we didn't have one standup!" I'm with you! But dudes, pay attention. The times they are a changing. Simply because there are a lot of problems with the way agile is being implemented doesn't mean you should toss the whole thing out of the window. If all you got from the article was that agile is a scam? You have well and truly missed my point. Yes, agile is a marketing term for best practices for iterative and incremental development. But unless you think you have nothing to learn, you should be hooking up with some good agile resources, no matter how screwed up parts of the thing are.

Wow! The Rush Lumbaugh of Agile has been found!

[long personal attack which has been edited out]

Crawl back into your hole. We don't need your help trying to improve the world of software development.

How about those "anonymous quotes" in your post? Why don't you own up that these are your own comments? Are worried such groundless defamation might get you in trouble?
http://en.wikipedia.org/wiki/Defamation

Boyd I have the email chain for all of these quotes. The blog, which has been around for several years, is anything but an attack blog. It's just me rambling on about things that interest me. You are here as my guest. Please remember that.

This particular article was started by this conversation on HackerNews: http://news.ycombinator.com/item?id=1666724

It was one of many I've had over the years with developers. I felt it was time for somebody "on the other side" to acknowledge that we know what they are talking about. Time to take an honest accounting of where we are. For that I do not apologize.

Now if you have something worthwhile to contribute, please stay, otherwise not.

Nice article, thanks for sharing.

I think Going to agile is not just applying some practices & principles in your team like pair programming and iterative process model, before going to agile we should study about target organization deeply, we should know organization culture. As mentioned agile is not defined process or set of practices, instead agile is set of mindset that should be adjust and adopt in your organization. Playing agile in your organization is more art than science.

I think the problem with adopting agile is that there is no defined process, step and rules. In the other hand these factors are most notable feature of agile. There is no definition; you should learn how to play and improve your process in continuous manner.

>Another pet peeve is when some asshole falls back on
> "it's emergent" You ask something like "how can we make
> sure the design is in place for this new space shuttle
> code?" and the answer is "it's emergent". That may be
> true, but it's not an answer. It's just crap. It's like
> saying you can get to the moon by jumping up in the air
> a little ways, then keep learning how to jump higher.
> Something is missing somewhere

The space shuttle seems to be the last refuge of waterfall advocacy! This argument reminds me of the claim by Intelligent Design advocates that the human eye had to have been created all at once because there's no way it could have evolved from something simpler. This is basically a failure of imagination as much as ignorance of biological research (see http://bit.ly/EyeAndID1).

Does anybody really think shuttles are created all at once from a Big Design Up Front without individual hardware and software components being iteratively built and tested in isolation? If we get past the FUD and think about it, maybe we can see the shuttle differently (see http://bit.ly/LifeDeathAgility1).

And consider the Challenger disaster in relation to Agile principles (see http://bit.ly/LifeDeathAgility2).

Tom the point isn't that iteration and incremental development doesn't exist in complex projects like the space shuttle. In fact, just the opposite,as you point out. The point is that agile advocates consistently either overlook or don't understand all the engineering disciplines -- some of which are done in some sort of sequential order -- involved in space missions. So they just throw out "it's emergent" -- a true statement but a useless one for the technicians who need to make it happen.

Dan, Thank you very much !

We had just give up to adopt canonical agile technics

Regards
Nelson Rosa
POA-RS

It is a heathy sign that this kind of discussion can take place. If no one is allowed to say that the emperor has no clothes, it is not science nor engineering, its religion. Stripping away the religion will help us find the parts that create the value.

Brilliant and creative people try new ideas and sometimes stumble on something successful. But anything as complex as a software project as thousands of measurable elements. The hot project got enough of them right to show value, but many of the elements of the project could be "wrong" and most of them are arbitrary choices. It would take hundreds of projects trying different permutations of the elements to sort out which ones created the value. As luck would have it, we have now done thousands of Agile projects, and that should allow us to share results and speculate on which elements are working and which are not.

I like Agile, and think it is going to be a part of good programming for years to come, but I don't like the time-boxing in Scrum. I think Agilists try too hard to take back all the control from managers. I think the tools are really important, despite what the manifesto says. I love doing the tests before the code. I love the fast feedback. I think having a flexible team that can do different jobs helps prevent bottlenecks. I think the Lean concept of Waste is really useful.

So, let's throw stones are what feels false and share the bits that seem to work.

- Tomo

I worked for a start up company. There was three software developers, myself included. We rapidly developed a service based product that required no operational staff. There was developers and sales people. I could dream up solutions to make the company more money and implement them.

Then one day the owner hired a project manager and made her CIO. She said our new development methodology would be agile. Innovation went out the window. In three weeks the first developer left. The second month I left. The third month the third developer quit. After that, she was fired.

The only folks that like agile are management, because if forces milestones to be completed by a target date. This doesn't result in better software, it just results paying someone to frustrate developers.

I posted a rather lenghty commentary on this topic on my blog. http://www.gregerwikstrand.com/blog/ Basically, I agree that Agile methods might not be fit for everything but I disagree with the statement that a coach has to be a master at everything he coaches.

I don't know enough "agile" to be PRO or CON. What I do know is that a few of the agile "techniques" that have been described to me sound VERY similar to successful Project Management/Development techniques that I have used over 30+ years of leading/driving system/application/database development teams.

And a SCRUM is where the two guys of the Rugby "2nd line" sticks their heads (and shoulders) between the sweaty butts of the "Hooker" and two "Props" and "pushes like Hell" to get them to "hook" the ball back between our legs to the guy waiting to pick it up and run with it. A "good analogy" for a supposedly organized "management process"?? Not exactly precision teamwork like American football!!

Ye Olde Goate
"Olde Goates don't 'fade away', they just 'butt out'."
Still Buttin'

And ss for "the emporer has no clothes" -- that's exactly what I thought when I found out that Java has no data structures/declaratives. Object bias?!

There's plenty of companies who don't use agile and still have successful AND failing projects, irrespective of delivery technique.

Clearly a two day course is going to teach you bugger all, but you have to go into this kind of thing with your eyes open, just as in the rest of life.

Software development is bloody hard. The best thing you can arm yourself with is a team of very clever, hard working guys who bend over backwards for their clients.

If you don't like Agile, stop bitching do your own thing.

I think that you, like many others on both side of the discussion, are trying to over simplify something you have not spent the time to truly understand.
The one piece you are missing here is answering the "why". Why are you even doing software development? The answer to this questions will help you understand the motivations behind each of the different perspectives and perhaps begin to let go some of the anger.
You also need to gain a better understanding of what agile is all about - it is not about practices and it is not about 2 day classes - it is about creating the ability to respond to changing needs in competitive and business landscapes. Sometimes this is done with software. Sometimes not.
Lastly, I am proud that you share these postings and the responses with your kids. It is too bad that they may get the impression that you are bigoted, because though this is only a development methodology you rail against here, you do not set a good example for those other aspects of life that really matter.

Shane,

Thanks for the comment. I don't know who you're talking about "bitching", but I appreciate your directness.

Bitching and moaning are part of living, Shane. How are we supposed to improve things that we do not talk about?

I'm not going to weigh into the "just have smart people" debate once again. It's a big part of success, yes, but there's also a need to fail quickly and adapt, even with teams who are not super-coders. Some sort of process created from first principles is probably the best way to do this.

Matt,

Thanks for the comment! Seemed nice until the little nasty-gram there at the end.

Addressing the crux of your argument, yes, I understand what I'm talking about. I've been both a developer and a coach/mentor for over 20 years, worked and drove high-performance/high-profile teams. As an agile coach, I've taught and practiced all of the major agile/xp practices. I've coached over 50 teams, and had eyes on another couple hundred teams as they transitioned to agile. I've spoke at agile conferences and know some of the agile thought-leaders. I don't know a lot of stuff, but I know agile -- and I know how to help teams deliver business value.

As for "you're setting a bad example for your kids", I remind you that personal attacks are off-limits, whether it's to me or to other commenters. We've had a few drive-by slanderings already on this article. What's the old saying? If you can't argue facts, just slime the other guy.

We're not going to play that game here. Everybody's opinion and experiences about issues, topics, and principles are welcome. Other stuff not so much.

Thanks again for the comment. Best of luck in your work.

Very nice post.

Agile is just a big marketing word. I personally know authors who just put in 'Agile' and 'Lean' in their titles for marketing reasons... but that's ok! More people will read his book.
Because of this Agile marketing success lots of people get involved and as a result you end up with a lot of 'agile coaches'. Some bad, some good just like in any other industry.

I totally agree with you on 'trainers can't do the work' and 'fake success stories'.
I personally know people who are positioned as top notch 'agile coaches' or trainers but have very limited experience doing actual work. Sure they can talk about it and give entertaining talks about it...

But not having done the actual work yourself or not being an expert on performing the work does not mean you cannot add lots of value to teams or organizations. I remember this great tennis coach that coached top of the world players winning al major tournaments, but never played at a high level himself.

Cheers,
Cesario

Beautiful essay - 1000 points of light.

Unfortunately, my brain can only process 10 at at time - and discussion is virtually impossible. So - please feed it to us in smaller pieces.

Thank you,
Eric Houston

On second thought - It seems that Agile had nothing to do with it. I seems that scamsters ruined your life. Well join the crowd! Why must you bash Agile for human failings?

Eric,

Scamsters were listed as one item of several. If you think it all boils down to just "scamsters" then yes, you need it explained in detail.

You are closer to the truth when you observe that my complaints are part of human nature and not specifically related to Agile. People have a tendency to make a mess of things in any system. Sounds like a great topic for a future post!

Thanks for the feedback.

The key is "a baby" as opposed to "babies."
Having a baby is a nine month task for one woman;
that task cannot be divided into 9 parts and yield one baby.

Great Article!

Thank you!

As with every development method (methodology is only used to sound important), you will always find disciples who become so involved in the process itself, that they actually think it is the proper following of the process that guarantees project success. Anyone who has ever developed will know that the process is only a (probably small) part of the puzzle.
I think you are right to point out that the word "agile " has been gobbled up by marketing departments and is now mostly associated with specific methods such as Scrum, XP and so on.
You are touching an important part about the agile development "movement" (can't think of a better word"). It is about taking into account that both the customer and the developer are human beings. The issue of "big design first" approach is that it portrays the customer as the all knowing oracle and the developer as the coding machine just waiting to be fed another project requirement. But we all know the reality is different. The customer does not know everything and often doesn't even know what he wants in the first place and developers are generally intelligent life forms that require a certain amount of autonomy to come up with creative solutions to the customers problems. And this is exactly what the agile manifesto is all about. It is all about frequent interaction between customer and developer discussing requirements and delivered software after a release, team autonomy and embracing change as a necessary part of every project.
And no, there is no silver bullets, just increments to better software development.

I have been doing Agile (specifically Scrum) for the past two and a half years. I have experienced quite a few of the things you mentioned. great article! and its not dishing the process, its the way "some" people go about implementing it.

Thank you for your article. I have experienced much of what you describe. For many people, it becomes about following a process and not about delivering a product that satisfies the customer in a time frame and at a cost acceptable to the customer and the organization. Whatever process accomplishes that for your organization is, I believe, the "right" process.

I love your emphasis on incremental and iterative development. I would add that continuous build and testing are integral to incremental and iterative development. I've seen these patterns applied in all manner of processes, including waterfall, and have been key to managing risk and insuring that we deliver our product.

Great article Daniel and thanks for the honesty. However, I think one other label for why there is a growth with anti-agile movement should be the Evangelical Finger Pointing as Chris here demonstrates.

Chris R. Chapman replied to comment from Warren | September 7, 2010 11:20 PM | Reply

"Although I have not seen the article proclaiming the Waterfall methodology ruined my live and lost me millions of dollars."

Charette, Robert N., "Why Software Fails", IEEE Spectrum, Sept. 2005.
http://spectrum.ieee.org/computing/software/why-software-fails

In answering the question to there being no known article Chris points to the IEEE Spectrun article in which nowhere does it mention waterfall. The actual reasons are summarised as:

* Unrealistic or unarticulated project goals
* Inaccurate estimates of needed resources
* Badly defined system requirements
* Poor reporting of the project's status
* Unmanaged risks
* Poor communication among customers, developers, and users
* Use of immature technology
* Inability to handle the project's complexity
* Sloppy development practices
* Poor project management
* Stakeholder politics
* Commercial pressures


Why therefore does Chris assume waterfall was responsible?

Sadly, I believe this stems from the 'religious fervour' that surround the newly proclaimed 'agile methods' in that it is ok to express any negative view about all that is wrong with software development as long as they point to the waterfall methodology - even if this is completely fallacious. With the emphasis always towards promoting their own ideology and not in the addressing the real issues that prevent developing quality software.

In fact the software crisis of 1968 is still alive and kicking 40 years on. Many of the reasons why software development projects fail have had little or nothing to do with which methodology is used and it appears that the new approaches are still failing to address the very same issues that has dogged software projects that used the waterfall method.

I have to say though that as a Software Engineer I get very concerned as to the calibre of the developers who propose to be 'experts', supposedly capable of writing quality software systems, yet cannot be honest - even to themselves.

Andy,

You are exactly correct: a big portion of not doing things right is failure to understand both the good and bad parts of the old way.

One of the first things I learned as an Agile coach is that the word "waterfall" is bandied around in much the same way the word "witch" was bandied around in 1500 -- as a general excuse for all the evils of the world.

In fact it got so bad that I actually had to counsel some of the coaches that I was teaching that while "waterfall" -- the idea that long projects can be done top-down monolithically -- is a bad thing, it's not like you can't do anything sequentially. For some reason, though, to some folks this seemed like an incredible idea. Sequentially? What? We can't have a rigid order to the whole project! Why, why, that's waterfall!

Life is sequential. You can only work on one thing at a time. Release dates happen in an order. Teams depend on other teams.

The problem here -- and it's a really nasty one -- is that once you realize that you have to have some kind of sequential actions in your project, guess what? That means that you have to start describing -- gasp -- how software development actually takes place. Design, Analysis, Test, Coding, Incremental modeling, etc. This is a place where, sadly, many trainers are not able to go with the team, so they just throw out the word "waterfall" and hope people will go away.

Well the emperor has no clothes. Let's get that out in the open and then deal with it raionally.

Thanks for the comment, Andy!

I think your opinions are a reflection of how many people feel. I have always tried to teach hard work, good engineering, etc.. I am not showing off now, but consider this situation: I've got about 11 years experience building systems and companies, but I programmed for the first time when I was 10. I have a computer science degree, top of my class, and a computational masters degree, I've written and spoke and applied agile for years. When a mate comes up to me and says, 'hey! meet so-and-so. He's got he's a scrum master!' I am always polite but, yes, the words scam jump to mind.

But, what the hell, we are all moving on. Let everybody do what they want to do and let's the rest of us just focus on solid engineering.

Jamie
http://www.financialagile.com/reflections/9-general/37-beliefs-matter

PS - wrap all this up in an essay. Don't just off-load but contribute. Get this wrapped up, get a talk done, and get on the conference circuit. Don't call people frauds (and quoting someone else calling someone a fraud is a bit of a cowards way of name-calling! I hope you are ready to confront these people to their faces).

Wrap this up and people will listen. You can help move our levels of understanding forward.

Thanks and well done. Jamie.

Jamie,

Thanks for the comment.

You've brought up a good point -- next steps.

I had no idea the article would get the response from the community it has: tens of thousands of visits. So I never really planned next steps.

And if you knew me, you'd know that I'm not really the type of person to call others frauds. Get upset, maybe. Identify with emailers and posters who are frustrated with agile? Absolutely. But I think a lot of the failures I see from the agile community are very well-meaning people who, for various reasons, are not able to take their analysis to the next level.

I have no intention of being "the Rush Lumbugh of the agile movement" as one commenter here said. I'm just some schmuck who teaches agile and also gets fed up with a lot of stuff in the community. I love businesses and startups, I love helping folks, but I have no intention of being "the guy" -- doing the speech, writing the book, etc. I'm happy for now just being another code monkey who also writes articles.

Daniel thank you for your reply. Yes the waterfall method, in its basic form (see note i), is actually the logical sequence of activities as to how human beings naturally solve problems. Therefore I would not agree that waterfall is the 'old ways'. Whether people like it or not waterfall is still being applied in any development process whether that be software or not.

Let us take TDD. It matters not how small the steps are in order to develop the tests the developers are employing the waterfall method (in it's basic form at least) because that is the way we work.

So I think it is fair to say that waterfall and agile share many important characteristics no matter how certain people wish to try to disguise the fact - importantly both, on original inception at least, were intended for the development of relatively small pieces (see note ii). Waterfall employed an iterative approach (see note iii) whereas agile has refactoring. Nevertheless both have failed to address the issues and complexities surrounding the emergence for the development of enterprise-wide solutions.

I would be interested to see any metrics that compared economic costs of up-front documented analysis/design specifications and refactoring - that also includes all manpower costs as well as costs involved in disseminating knowledge of the systems being developed to stakeholders of all levels of technical and business expertise - meetings and training new people on the systems that they have to develop and maintain also has it's costs.

I am not sure agile addresses the dissemination of knowledge to all stakeholders in the most cost effective way. In fact this cost is overlooked in the cost estimation processes anyway and should in my view be included. A debate in itself perhaps?

Notes:
i. By basic form I mean excluding the documentation requirement.
ii. Royce's paper written in 1970 was describing an approach for developing large software systems as was considered then - large systems today do not compare.
iii. The wikipedia representation of waterfall is plain wrong - obviously written by some agile zealot. See:
http://en.wikipedia.org/wiki/Iterative_and_incremental_development
http://en.wikipedia.org/wiki/Waterfall_model
Notice the first article states ‘Iterative and Incremental development is at the heart of a cyclic software development process developed in response to the weaknesses of the waterfall model.’
And the second does not include any mention of the iterations described in Royces paper. See:
http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf
These two wikipedia articles cleverly try to hide the fact the waterfall is indeed iterative.

Dear messr DanielMarkham,

I envy you

- you have the fortune of being involved in endeavours that are part-of-the-big-picture projects.
- you live and work in an extremely organised and well-funded environment.
- your customers' requirements never changes more than three times a week; in fact, your customers' requirements never change a bit within the three years each of your projects took to finish.
- In fact, your management has successfully trained you and your customers extremely well to "FIX the SYSTEM" to avoid allowing customers change their requirements.
- You are delivering software to maintain very well behaved and predictable processes and you "FIX the SYSTEM" by ensuring you factor out badly behaving and low
predictability areas.

I have been thro the same type of training you have been thro in the earlier part of my programming life. I used to code in Fortran and Cobol and have even written code on punch cards. Avoid the unpredictable projects and make sure you educate your customers to present to you a framework that is predictable and well-behaved and of low variability.

However, I pity the companies or clients you work for. You have managed to so successfully and authoritatively educated them and framed them into a nutshell of proper electronic behaviour that precludes them of the vast other possibilities and avenues of profits.

I on the other hand, though envying the stability and predictability of your work and social life, still prefer the excitement of being subject to the brutality of market conditions. No "FIX the SYSTEM" management buffer between myself and colleagues with the jungle out there.

So, we have no choice. Yes, we fire-fight, we succumb to frequent changes in the market place. We write software as an aid to engineers, scientists and business planners who themselves fire-fight. They have no choice because they hustle for business and every little increase in profits for the company counts.

I envy you because you are insulated from the angst and emotions of making a company profitable. To be profitable, the companies I worked for must respond very quickly. Products are discarded, plans are discarded. You have trained the CEO of your company, or at least saliently chained down his/her options to respond quickly to changes. Oh yes, you work in a set-up where the funds have been determined by the US Congress and the goals are set in stone. I envy you.

But still, I also pity you. To me programming is like making love. The excitement of unpredictability, the tinge of uncertainty. I am sorry you would have none of that. You are missing out a lot on life.

Thanks for commenting, but what the heck are you talking about?

I'm an agile coach who firmly believes in working in a fast-changing environment where the customer is as close as possible.

Did you even read the entire essay and comments? Or just get a feeling you knew what it was about and sat down and wrote this diatribe?

For somebody who took as much time as you did to write the comment, it might have served you better to understand WTF you are commenting about.

I was going to delete this, but its not really that offensive -- just so off-target as to almost be comical.

Remember folks, when we get up a head of emotional steam its very easy to get on a high horse and ride off into the sunset. In a lot of situations, the way we humans practice and teach Agile is broken -- the emperor has no clothes. Let's just get that fact out in the open and deal with it. Nobody is saying to throw out the baby with the bathwater. Honest, open discussion beats, well, the kind of knee-jerk thing we see so often from "agilistas". In my opinion, going on like our commenter has just done here is part of the problem, not the solution. Next we'll be bandying around the word "waterfall" like people in the 14th century used the term "witch"

If agile is to improve we need to have open and honest discussions about the problems. We can't insist on lumping folks into one camp or another. We need to get away from personal attacks and deal with the issues involved. Right?

Sigh.

Great thoughts Daniel. There's a need to expose the BS around Agile and point out the hypocrisy in those who rigidly oppose any hybridization of the Agile movement. Agile was born out of asking questions and seeking flexibility....it's ironic, maybe even sad, that some of it's very highest proponents are so inflexible and dogmatic now. Aren't these the attitudes that Agile sought to change?

Leave a comment


Comment Policy: I really, really, really enjoy comments, but if all you have to offer is general platitudes like how happy you are to have found my site and what a wonderful place it is, I will delete your comment and report your comment as spam. Please try to either tell me I am wrong, sympathize with my point, expand on what I'm saying, or offer your own experiences or opinions. If you just want a link your best bet is to just ask for one. Probably won't work, but at least be honest about it. No name-calling and please keep the profanity as low as possible. If your grandma can't read it or you wouldn't say it in person, don't write it here. Thanks.

About this Entry

This page contains a single entry by DanielBMarkham published on September 7, 2010 9:42 AM.

Structuring Larger F# Functional Projects was the previous entry in this blog.

Agile Metrics You're Not Supposed to See is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.





Share Bookmark this on Delicious

Recent Comments

  • Christopher R. Goldsbury: Great thoughts Daniel. There's a need to expose the BS read more
  • DanielBMarkham: Thanks for commenting, but what the heck are you talking read more
  • Blessed Geek: Dear messr DanielMarkham, I envy you - you have the read more
  • Andy: Daniel thank you for your reply. Yes the waterfall method, read more
  • DanielBMarkham: Jamie, Thanks for the comment. You've brought up a good read more
  • Jamie: PS - wrap all this up in an essay. Don't read more
  • Jamie: I think your opinions are a reflection of how many read more
  • DanielBMarkham: Andy, You are exactly correct: a big portion of not read more
  • Andy: Great article Daniel and thanks for the honesty. However, I read more
  • Walt Wallach: Thank you for your article. I have experienced much of read more

Information you might find handy
(other sites I have worked on)





Recently I created a list of books that hackers recommend to each other -- what are the books super hackers use to help guide them form their own startups and make millions? hn-books might be a site you'd like to check out.
On the low-end of the spectrum, I realized that a lot of people have problems logging into Facebook, of all things. So I created a micro-site to help folks learn how to log-in correctly, and to share various funny pictures and such that folks might like to share with their friends. It's called (appropriately enough) facebook login help