« Understanding Flowcharts| Main | The Mythical Man-Month »

Programmer as Market

| | Comments (25)

Something has been bugging me about Microsoft and other tech vendors for some time. As I was reading and responding to a comment over on Hacker News it finally jelled in my mind.

The commenter wanted to know if TDD/BDD was useful in startups. Of course, the answer, like all answers to these questions, is "it depends". But feeling a bit punchy, I just said "no", then outlined why the answer was no.

I got a bit of pushback, much of it along the lines of "but TDD/BDD is _good_ development. Surely you're not telling us to be bad developers!"

Yes, I am.

Fact is, startups are all about making things people want, not being a great developer. Your program is number 50 on the list of important things for you to do. You might write 10 lines of Javascript and make a million bucks. Or you might write the cleanest, most gorgeous code anybody has ever seen and make zero.

It has nothing to do with code. It has to do with finding and meeting customer needs at scale. In fact, the odds are 20-1 that your entire effort is worthless no matter what or how much you code, so don't write code to be maintained. Don't write code to be pretty. Write code to get it done as quickly as possible so you can move on to the real game: customers.

This doesn't go over well with most folks. I know it didn't go over well with me.

The problem is that we developers have become a market, and like all markets, people are trying to sell us stuff. These people do not have our customers in mind -- for them we are the customer. Instead they have us in mind.

People try to sell us stuff based on whether or not we want to consume it, not whether or not it's useful to the people as programmers we are trying to help. This creates a bit of perverse incentives.

Don't like talking to people? No problem. You can read a hundred programming books and none of them will tell you that you need to do that -- even though "talking to people" has to be the most important thing a programmer does. Like cool little widgets instead of fuzzy problems? No problem. There are myriad vendors out there who will sell you all sorts of cool little widgets. You can spend the next ten years becoming a widget expert.

You see, what I found with Microsoft -- and what's true in the industry as a whole -- is that they could care less about helping me help other folks. What they want to do is meet whatever pre-existing prejudices I might have as a programmer. Wonder what cool features C# 5.0 will have? Wonder who in the real world cares aside from us programmers?

This feeling was a long time coming, and like every person that has been conned, it took me a while to wake up. What? You mean all those things I spent money for were useless?

Last week I went through my office cleaning up. You know how many tools and languages and other pieces of crap I've bought over the years? A zillion. Just how many calendar controls does one person need, anyway? Will there be some calendar control Apocalypse, where everyone will suddenly have to write their own day-planner?

I have spent hundreds of thousands of dollars on software, tools, books, and equipment that tell me what I want to hear -- that programming is a complicated yet noble profession which involves mastering lots of technical details and being plugged into the latest technology. But programming is helping people, not any of that stuff.

The same goes for TDD/BDD -- and a lot of other things. We get sold this idea that programming is high craftsmanship. That there are "good" programmers and "bad" programmers. The "good" programmers all do things this certain way.

Yes, some things work better than others. No doubt about it. But people who sell us stuff don't necessarily give us all the times when their stuff shouldn't be used. For them, it's good for almost everything. They have no incentive to provide context to us, because many times the context in which what they are selling is useful is very small, but the context in which we might imagine their stuff is useful can be very large indeed.

Do you know how many hundreds of millions of dollars -- perhaps billions -- I have seen spent on enterprise tools which all boil down to "your programmers need help to do things the right way. Our tools will change their behavior so you don't have to"

Never works, but still folks spend. Marketers are selling folks on a vision of what they would like the world to be, instead of what the world really is.

Yes, I would to live in a world where having some cool new set of programming tools would mean that I have an awesome startup, or that I'm able to help people simply by clicking a button somewhere, but that world doesn't exist anywhere outside a seminar, conference or sales pitch. In the real world, real people grapple with fuzzy difficult questions, they don't always get along, and they implement suboptimal solutions that are better than nothing.

I have never been in a situation helping somebody where having some little cool piece of tech actually helped. Sure, I've thought that having the tech helped. You betcha. I've been programmed by vendors to think "Gee, good thing I have X! This will make the problem much easier to solve!" but it's never actually been the case. (The only exceptions I would make to that statement would be relational databases and web applications, which standardized information storage and application distribution)

On the contrary, I have seen hundreds of programmers and dozens of teams get caught wasting huge chunks of time with stuff somebody bought that is getting in the way of their work. I have seen teams who are basically paralyzed because of all the "help" in the form of tools, languages, tech, and methodologies they have purchased. I had one client that had a hundred people working on a problem. It had reached the point where they were thinking their problem was insolvable. After prototyping for a week, I showed them how to solve their problem very simply.

Their reply? This won't work because it doesn't match up with our tools! Not that the customer would hate it, or that it failed to solve the problem. They couldn't implement it because their existing tools selection prevented them from solving the very problem the tools were supposed to help them solve!

I have good friends who constantly tweet about getting hung up by tool X or technology Y -- instead of solving somebody's problem. They keep feeling like that by mastering this latest tech somehow their productivity will shoot ahead. But it never does. There's just a new set of magical tools and tech that come out next year. This is a treadmill that will never stop. Vendors will keep making things and promising that problems will go away, increasing the overall complexity and depth -- as long as programmers have money to spend.

A lot of tools are very useful, no doubt. But we need to be extremely careful that we are not being taken into some kind of imaginary marketing world that doesn't exist for anybody else just for purposes of feeling like we're cool. We do so at our own peril.

25 Comments

Oh my god... I could not have said it better myself! I've had this thought in mind for awhile now.

I used to work in a startup, and the other developers would berate me... calling me a duct tape programmer. Startup went belly-up because the idea did not meet any market need-- not because how the code was implemented!

Hi, I'm curious about this piece of the artice:
"[I] Wonder who in the real world cares aside from us programmers?"

Who do you think should directly care besides programmers?

I've been creating web apps for a few years now and have first-hand experience with your point. My language of choice is coldfusion. You can imagine the pressure I feel to learn something else, PHP, Python, anything. In fact, I have begun to learn both these languages, and they do fit niches I sometimes need to fill. But I can use coldfusion to solve problems because I have been using it for a long time and I am fairly resourceful. I use it because I don't have enough time. The vast majority of my time is spent trying to show my market that I have a solution they can use, and get them to pay for it. I love programming. I actually find it entertaining. That's a good thing, but it doesn't pay the bills by itself.

By the way, I wish I could write as well as you, and I admire your ability to flesh out a topic in such detail. Thanks.

This is more a Microsoft ecosystem thing than it is a developer thing in general. I've seen several MS developers come to a similar realization. And in fact, the very fact that they are "MS developers" - not simply developers - is part of that strange ecosystem. I've seen it remarked on more than once that MS and the vendors that surround their products keep "their" developers in a state of constant churn, trying to keep up with the latest practices and tools, so much so that they never have time to look at what the rest of the industry is doing.

I have never "bought a language" in 12+ years of programming everything from air traffic control systems to web-based startups, and and I've bought precious few tools. But the basic practices of quality work - such as TDD and continuous integration - have stood me in good stead all along the way. I think you're lumping good practices and people *selling* practices together, but in my experience there's really no need to conflate the two. You can have one without the other - I have!

P.S. Is there really someone *selling* TDD/BDD out there? As long as I've been using it - a good decade or so - it's just been something you *do* in order to quickly come to the simplest possible solution to a problem. I don't recall ever being given an opportunity to pay for it.

I mean, OK, there are a few good books about it which are worth picking up to get beyond the basics, but other than that I can't imagine how anyone could sell it to me as a product.

Avdi,

Thanks for the comment! Yes, I have lumped the two together, and for purposes of this article I'm happy with lumping them together. The churn I see in Microsoft turning out new stuff every quarter is exactly the same churn I see in management philosophies, programming movements, or author worship -- perhaps only at a slower pace.

I think my turning point was when I realized that for any project I had to start with zero -- no preconceived ideas about language, delivery platform, development practices -- nothing. Then add in little things at a time, eventually ending up with the barest of minimum tech and practices to meet the immediate need. This is a completely different viewpoint than most developers have. It's not unusual for developers to sit around _before they even understand what the project is doing_ and talk about huge swaths of architecture and practice. I have found that kind of behavior to be very counterproductive.

I've coded a lot of systems, and I've watched a lot of teams using all kinds of tech and methods. What seems to be consistent is that they over-constrain themselves by these hidden assumptions they get from others, without even realizing what they are doing. That's true no matter which piece of the puzzle you talk about. I don't think it's limited to just Microsoft.

Avdi,

I just turned down an opportunity to take a couple of weeks and teach TDD/BDD to a team. It's a great way of doing things, and I'm a big fan -- when it's necessary. But this team didn't want to change, already were having a lot of problems, and my role would have just been an arm of management forcing people to do what they didn't want to.

Now those two weeks of consulting had value to somebody.

I think you overstate the case when you lump all of this just into "things that people sell." There is a marketplace of ideas out there, and folks are marketing to developers at all times, whether or not money is directly changing hands or not. The point is to remember that, once you become the customer instead of the _real_ customer, things get out of whack quickly. Not that TDD or whatever is bad or good. That's too far down in the weeds for what I'm talking about. We could sit around imagining various scenarios until the cows came home in which TDD would be awesome or terrible. But that's the problem: too much imagination and very little pragmatism.

I suspect the real reason why there is a programming-industrial complex (analogous to the military-industrial complex) is because it behooves us in the industry to make distinctions between good and bad programming. How can you charge 1st world labor rates for programming when the same programming is done in 2nd world and 3rd world countries for 1/10th the rate? There must be either a real or perceived value to paying someone $125/hr rather than someone else just $10/hr to program a solution. So Ruby/Python arise to displace PHP, because fewer people do Ruby/Python, because the 2nd world and 3rd world have tons of PHP and ASP developers. When Ruby/Python gets commoditized, we move onto the next great thing.

Good programmers put the time in to know their tools, and don't waste their customers time and money learning a tool that may not help them solve the problem.

TDD/BDD are an ingrained part of my workflow at this point, which actually allow me to execute far quicker than I can without them. Same with pair programming.

I think it is wrong to assume that these programmers who get hung up on all these tools and heavy weight processes would be performing any better without them.

Great article...I can relate to my working experience

Sometimes I feel too many cutting edge, cool tools confuse me and others rather than help us focus on our goals...I agree...know your tools and keep'em simple

I definitely know the tendency you're talking about - I think most developers suffer from it, especially toward the beginning of their careers. I still suffer from it occasionally, but it was much worse when I was a younger developer. I had read so many awesome books about patterns and practices that I wanted to apply them all at once and completely map out everything I'd use for the project before writing a single line of code!

And unfortunately, rather than helping developers get *over* this tendency, a lot of organizations actually enshrine it as a virtue and give senior people titles like "architect" and expect them to over-think things for a living.

But here's why I think it's odd that you chose TDD to pick on this article: if there's one mental practice that, more than anything else, helped cure me of this affliction, it was TDD. TDD is perhaps the ultimate in developer pragmatism. TDD says: "STOP THINKING. Stop architecting. Stop researching methodologies. Identify ONE concrete feature. Write a test that will pass when the feature is working. Now write the simplest code to make that test pass."

Of all the practices that are being promulgated (and sometimes sold), TDD more than any of the others has forced developers to shut up, stop over-thinking things, and just write the damn code to make the system work. So I see it as a strange practice to choose as an example.

Avdi,

Happy you found my mention of TDD uncomfortable! Now explain to me when you wouldn't use TDD and your journey to master will be complete (Sorry, watched too many Star Wars movies over the weekend)

The thing is that you are right in a general sense: iterating, measuring, and feedback loops will tell you when you are overthinking or overconstraining yourselves. TDD is just a form of that at a very small scale.

I'm happy you like TDD and use it no matter what. I'm hearing from several folks like that. More power to you guys. As for me, I'm willing to give anything up -- except my hungarian variable naming practice, which you will have to pry from my cold, dead hands. (wink)

Everything in moderation. I don't TDD my spikes, and a lot of my tests these days would be unrecognizable to an old-school TDD-er because they take place at a much higher level than the unit. I just think you're painting with a pretty broad brush when you say developers do these things because of marketing.

As long as you are using *real* hungarian and not Systems Hungarian[1], I got no problem with that! :-)

[1] http://www.joelonsoftware.com/articles/Wrong.html

Daniel,

I think I know what you are getting at, and I think context is also important when talking about TDD. Picking up TDD in javascript, ruby or python is very easy. You can tests every method. Working together with a person with experience with TDD, you can get the gist of it in about half an hour.

I cannot say the same about .NET development. The static nature of the language creates a big overhead to even get to the point where you can TDD. One needs to know a collection of techniques before one can start doing it correctly. Is this what you are talking about? If that is the case, I agree that this would slow down development.

That said, I guess the idea that you are getting is that the main goal is to get a product done quickly and make money. Once a product makes money, there is the justification to refactor and create tests. In a way, it is like a meta-TDD where you don't even touch TDD until you have a justification to maintain it. I can see value in this thinking.

That said, due to my creative personality, I find TDD good in that it helps me keep focused on what has to be done, and there is a clear indicator that I have reached the goal. Otherwise I would end up just adding "one more thing" for eternity. :)

Hi Hugo,

I certainly didn't want to turn this into anything specifically to do with TDD or BDD. The problem is that for _anything_ like this, once we start talking about how easy it is for the coders and stop talking about how much value the customer is going to get, we've drifted off the rails.

So all you TDD/BDD lovers, I'm with you guys. Like I said, it's a great technique -- when you need it. But as one other commenter has pointed out, you always don't need it.

Keep the focus on value, not the tools. If, for instance, you're in a startup situation, there are 20-1 odds you'll never see that code again. Factor that into whatever you decide to do.

Although sometimes it works out. The new Microsoft language F# was created in part to address the geekiest programming market in the world: functional programmers working in the financial sector.

However, even though not in that category, I have adopted F# and have been learning Haskell. The reason being, after some initial learning curve, I find I can 1) prototype much, much faster than with other languages, and 2) the chances of the prototype being viable and maintainable are much higher.

The creators of Python and Ruby were attempting to do the same thing. I'm not an experienced programmer in either, so I can't say how well those attempts worked out. I know Microsoft's F# decision did help me.

The main place where I think things go wrong is not in the initial attempt, but in the later bloat. Which is why I think smaller, simpler programs in continuously evolving and specialized languages are the way forward. If you can't afford to re-write your code when a better language emerges, chances are the goal or methodology of writing it was flawed in the first place.

-Neil

Great comment, Neil.

I wonder how much F# had to do with my slow realization. There's something about sitting down and coding a solution in just a few airtight lines of code that really makes you wonder why you were spending so much time building architectural castles in the sky.

Context plays a huge role here. The type of advice I would give a team in a BigCorp kind of environment is completely different than you or I doing exploratory programming.

I'd like to move on to Haskell and Erlang. Right now I really like having all those MS and third party libraries to pull from, though.

My critique of your post comes from a communist position. I am not here to argue the merits of TDD/BDD/CDD/KDD/GDD, or whatever.

I would like to simply point out a simple hypocrisy that is inherent in your argument. Isn't it interesting that most of these "million dollar 10 line" javascript pro-customer developers that you are talking about build off the work of programmers who actually do care about writing good code?

Most of these new millionaires build off the work of various free software projects which are built without the monetary "motivator".

It boils down to the fact that a "customer" to you is someone who pays. I think you should concentrate on the idea of "user" instead of "customer". Though they may be often the same thing, sometimes a customer doesn't exist for a product, but there are potentially a million users.

Would you claim software without a customer, but has millions of users is not valuable?

Mempco,

Thanks for the comment! Although I am a bit confused by your reasoning.

I meant no difference between "user" or "customer". Money need not change hands for you to create something that provides value to somebody. Concentrate on the value, not on the tools, methodology, architecture, widgets, or anything else.

Funny this was posted on Reddit a few hours after "How to use BDD to discover value-add for your startup"[1]

[1]: http://blog.interfacevision.com/index.php/2011/02/how-to-use-bdd-to-discover-value-add-for-your-startup/

I might be interested in re-reading this article after some editing. It's too rant-y and contradictory for me (all these tools are useless, except for relationship databases and web applications, and a lot of our tools are very useful), and seems to presume the worst out of all parties involved.

I agree with you, wholeheartedly, with one small proviso:

Development using agile processes has been, for me and the last dozen teams I've worked with, the most efficient, fastest, best way (pick any three) to get code out the door.

The reality we like to ignore about our craft is, first and foremost, it's still a craft. Whereas broad practices can be repeated, we're still almost as dependent upon individual heroics as we believe ourselves to be. It's hard to get teams whose members have never exercised professional discipline to do so. Hell, it's hard to get such teams to function as teams; that's one of the leading causes of startup death right there. Not always obviously, not always visibly, to be sure. But if the team — even if my only teammates are me, myself and I — fail to write code that works, that delights the customer and shows him how different and revolutionary things can be if only he uses our product, then the startup will eventually fail. The only variable at that point is how much VC money, gallons of printers' ink and gigabytes of industry-PR server space will be flushed in the process.

If you're not doing something that nobody has done before, or at least without significantly improving the customer experience, then you're not a startup. At best, you're a Ponzi scheme hoping to ride somebody else's coat-tails into the Magical World of "Me, Too." And that's not where any professional would want to go.

Chris,

Thesis: because programmers are markets to tools and other vendors, programmers have started concentrating on what's better or worse for them, the programmer, instead of what is better or worse for the customer/user, which is the entire reason for programming in the first place. This has created a perverse sense of what is appropriate or not for a given type of project.

There are extremely few tools that are truly useful no matter what type of the project because of their transformational nature: relational databases and web applications are probably the only two I can come up with. Sadly, many programmers would have a hard time keeping their same opinion of transformational tools under 20 or 40 items. Something is out of whack.

Thanks for the comment. Hope that cleared it up for you. You make a good point about editing, but it's a blog entry, not an essay. Plus it's a rant. I barely checked the dang thing for spelling, much less terseness or logical flow. Money back if you don't like it.

Thanks for the article! I think you have a valid point, except that one of the goals of TDD/BDD/etc is to ensure the software does something the customer needs. A programmer, before sitting down to code, should talk with the product lead, a QA engineer, the CTO, or the customer to find out what the software needs to do, and if the programmer starts by writing assertions about what the program needs to do to satisfy the customer, then as the program continues to evolve, the team developing the software knows that the customer will continue to be satisfied.

Another way of looking at it is this: if you are writing tests that your product owner or customer do not find valuable, you may be writing tests the wrong way.

I agree with Avdi: don't use TDD for spikes. But other code, even if it has a shelf life of a couple months, still will be maintained.

"You might write 10 lines of Javascript and make a million bucks. Or you might write the cleanest, most gorgeous code anybody has ever seen and make zero.

It has nothing to do with code. It has to do with finding and meeting customer needs at scale"

Seems like you are talking explicitly about customers, not users. Unless I missed it, nothing in your article talked about users specifically, but explicitly about customers.

Which is interesting considering much of your article is talking about how all these software vendors sell to companies widget x, or tool y, without regard how it can help you, the vendor of software, help your customers.

In other words, these vendors seem to be writing software/tools/widgets quickly to sell to software companies. By your assessment, they are doing exactly what you recommend. And they make a boat load of money.

However, if they focused on the user instead, then they would be writing tools to help you write software that is beneficial to the user.

So please use the term user instead of customer. They are often not the same person and its an important distinction.

Leave a comment


Comment Policy: I really, really, really enjoy comments, but if all you have to offer is general platitudes like how happy you are to have found my site and what a wonderful place it is, I will delete your comment and report your comment as spam. Please try to either tell me I am wrong, sympathize with my point, expand on what I'm saying, or offer your own experiences or opinions. If you just want a link your best bet is to just ask for one. Probably won't work, but at least be honest about it. No name-calling and please keep the profanity as low as possible. If your grandma can't read it or you wouldn't say it in person, don't write it here. Thanks.

About this Entry

This page contains a single entry by DanielBMarkham published on February 20, 2011 9:49 AM.

Understanding Flowcharts was the previous entry in this blog.

The Mythical Man-Month is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.





Share Bookmark this on Delicious

Recent Comments

  • mempko: "You might write 10 lines of Javascript and make a read more
  • James Kingsbery: Thanks for the article! I think you have a valid read more
  • DanielBMarkham: Chris, Thesis: because programmers are markets to tools and other read more
  • Jeff Dickey: I agree with you, wholeheartedly, with one small proviso: Development read more
  • Chris Morris: I might be interested in re-reading this article after some read more
  • AP²: Funny this was posted on Reddit a few hours after read more
  • DanielBMarkham: Mempco, Thanks for the comment! Although I am a bit read more
  • mempko: My critique of your post comes from a communist position. read more
  • DanielBMarkham: Great comment, Neil. I wonder how much F# had to read more
  • TechNeilogy: Although sometimes it works out. The new Microsoft language F# read more

Information you might find handy
(other sites I have worked on)





Recently I created a list of books that hackers recommend to each other -- what are the books super hackers use to help guide them form their own startups and make millions? hn-books might be a site you'd like to check out.
On the low-end of the spectrum, I realized that a lot of people have problems logging into Facebook, of all things. So I created a micro-site to help folks learn how to log-in correctly, and to share various funny pictures and such that folks might like to share with their friends. It's called (appropriately enough) facebook login help