Tag Archives: epistemology

It’s All Bullshit

One of the key indicators of reaching mastery level in any field is realizing that’s it’s all bullshit — and moving forward anyway.

Take one of my fields, technology development, as an example.

Did you know that 70% of all IT-related projects fail to meet their objectives? Or that Winston Royce was the man that created the waterfall development model? Or that the military uses “command and control” to tell soldiers what to do? (and that this is a bad thing) How about that Use Cases are going to destroy your productivity?

These factoids, and many more, are commonly used in presentations dealing with technology development.

And they’re all bullshit.

Success and failure as measure by the 1994 CHAOS study, is a relative thing. Winston Royce never wanted people to use the waterfall method; he was just describing sequential activities. In fact, he said you should iterate through again. The military, especially ground combat units, give most operational responsibility to people on the ground engaged in combat, not generals — exactly the opposite from what a non-military person would think. Use cases are a form of process analysis, not a huge MS Word document template to be filled out. As a form of analysis, you can complete them with any level of formality or documentation you like: lots of meetings and paperwork, or a napkin and a 15-minute conversation over lunch.

It was a recent conversation on Google+ that got me thinking of this. Internet friend Laurent was complaining that the statistics used by academics discussing technology development were bad, mungled, and meant nothing. He seemed very upset that such arbitrary numbers could be used.

This piqued me, because I never thought of academics (or any other thought leader) in the technology development arena to be doing anything more than quasi-informed guesswork. And that was fine with me. But to Laurent it was very disturbing.

One of the things I’ve learned as I’ve gotten older is that there’s theory, then there’s the real world. In the real world you can use any kind of theory you’d like. Just pull it down, apply it, and see if it works for you. Keep what works. Throw away the rest. But you can’t focus on the theory. You always have to focus on pragmatism: is this working for me right now. It’s this two-track approach, where academics and other thought leaders scope out a possible path forward, then millions of us try it out in practice, that slowly moves us forward.

Most things we do are not physics. We don’t need a theory of life, the universe, and everything. We have work to do. These things do not exist in the absolute, easy-to-digest way we might want them to. We need a solution to the problem we’re working on right now.

Some people just give up on all theory — all they want is results. Some people, many times without realizing it themselves, start focusing on how things ought to work based on their pet theories without looking at how it’s really working. Confirmation bias is a powerful thing.

People screw up when they focus either too much on how a few teams worked or how the theory works. You need both tracks.

But this is true in many more areas aside from technology development. There are things we can do that work most all of the time, there are things that are going to be highly contextual, and there’s the system of common understanding about why things work they way they do. These are completely different areas!

So yes, at some level you realize that, heck, even physics isn’t physics. Physics has many more unanswered questions than any sane person would be comfortable with. The soft sciences are a lot more soft than you were taught, and that evolutionary behavior theory sometimes can amount to a lot of speculative bullshit while hand-waving than the TV science shows let on.

But then you realize that it’s this way everywhere else too. And you move on. After all, you have work to do.

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Why I Hate Science

TL;DR -> The word “science” is being used in so many contexts as to be very bad for the application of real science. Scientists have themselves to blame for this. They need to create better standards of communication both internally and externally if they want their profession to retain the rightful luster it deserves.

I love science, when the word means the use of Bayesian models to come up with new and observable reproducible patterns of behavior. I am deeply intrigued by science, when the word means the groups of people and political processes used to do this model creation. And I hate science, when the word means something used as a club to beat up people you simply disagree with.

That’s the problem: the word “science” can be used in all of those contexts, and it’s never clear exactly which one a person is using. Sometimes it looks like it just means “anything a scientist is saying.”

Listening to some pedantic columnists (you know who you are), people out in the real world are a bunch of babbling idiots, swigging down cheap beer and going to tractor pulls while thinking that science is just a bunch of hogwash for nerds. Usually the explanation is that they got this way because of a poor public education system and a media that promotes conflict over truth.

It’s not unusual in school to hear this religion of science put another way, science is the beautiful application of reason and logic that slowly and inexorably moves from ignorance to truth.


First, there are things that are known and predictable out to 3 or 4 sigmas. I don’t think anybody believes these things are wishy-washy. But then there are scientists who are experts in one area who go off the deep end picking up little pieces of theory here and there and speculating ad naseum. I don’t think you need to be a kuckle-dragger to tell to see that there is a vast difference.

This kind of pseudo-scientific slime is everywhere. Pick up about any book in the “science” section and you’ll read some of the wildest speculation: what happened 50 thousand years ago to bring about domestication of animals, how the human sex drive evolved, where all the dark matter has gone to, or how it is that conservatives have different brains than liberals.

Don’t get me wrong: there’s nothing wrong with such speculation. In fact, it’s required for science to progress. What I’m concerned about is that the average scientific layman reader — the person who assumes they are just as up-to-date on scientific matters as possible — has no freaking idea where the real science leaves off and where the bullshit begins. One minute the scientists is talking about Carbon-14 dating of stoneware. The next minute he’s talking about tool use in prehistoric societies. Then he mentions inbreeding with neanderthals. Each of these topics has a greatly different aspect of speculation versus reproducible science, but the reader is left to judge the entire essay by what, exactly? Popularity of the scientist? His or her own political opinions and how they agree with those of the scientist? The number of papers published? The name of the publication it appears in? Phase of the moon?

Whatever it is, it has nothing to do with the eventual veracity of the statements made. But I’m not sure Mr. Smarter-than-thou reading it has any clue about that. And it’s an ethical slip that scientists need to stop making.

Once again, don’t get me wrong: I love pseudo-science. Proudly sitting on my shelf is the latest edition of “Secrets of Antigravity Propulsion” I get it down when I need an interesting diversion and laugh. I’m not saying it’s false, and I’m not saying it’s true. In fact, that’s what makes it so much fun to me: while we may have very good guesses about how true it is, we don’t know. As much as we want certainty, there are things we simply do not know. Non-reproducible phenomenon, such as some atmospheric things commonly called UFOs, could be anything. We’ll be speculating about this stuff for centuries. Fun stuff.

And yes, I’ll watch a good UFO show — although they are hard to come by anymore. Probably the best over the last few years was one the History Channel did on commercial pilot sightings. Trained professionals with lots of time in the air seeing unusual things that are cross-checked by ground radar? Very interesting. Fun material for speculation.

But I know these things are wishy-washy. I know that this is all so much BS. My problem is that when I run into statements from scientists that are just as flimsy — perhaps in a less spectacular arena — it’s not so clear.

Take evolutionary psychology, the idea that aspects of our personality exist because natural selection made them the best fit for previous environments which might not exist any more. Great idea — you can certainly hear the music and hum along — but at the end of the day you can make the argument that everything about personalities are evolutionary in nature. ADHD? Useful for surviving in the jungle. Adultery? Useful as a gene transfer mechanism. And so on. You can just throw darts at a dartboard of human behavior and pull some kind of theory out of the air for why evolution made it that way.

Not exactly much of a science. Personally I like Elvis sightings a bit better. Much more entertaining.

How about Anthropology? A fine field of science if there ever was one. Except it’s reached the point where it has deconstructed itself. Anthropologists are unable to make firm comparisons between cultures, thereby reducing everything they are doing to either observation or complete speculation (and sometimes politically-biased speculation at that). Come now, if you can’t compare things, where is the science? I’d argue that private property rights are a critical part of human cultural evolution — that there is a great ditch which is crossed when the private property rights of the individual are recognized. But I don’t think such an opinion is very popular among Anthropologists. Perhaps so. Beats me. It’s their mess. They should clean it up.

The flowering of this idiocy, of course, is when people have strong political or personal opinions and want science to tell them that other people with different opinions are idiots. That way they can appropriately look down on them. I was reading a great article in Mother Jones the other day about homosexuality, science, and politics. The tl;dr version is this: it’s not as simple as “I was born that way” or “It’s all a choice”. For some it might not be a choice. For others it might be perfectly fine to change sexual preferences because, well, just because. Sexuality is fluid. The idea that you’re gay or straight and were born that way and must be that way for life is an artifact of political discourse over the past century or so. But both sides want to use “science” to bolster their side and attack the other. Science, ever so flexible, becomes a big stick to hit the other guy with.

There are many other topics like this. Pick your favorite. The social sciences are full of various ideas that have consensus that are of unknown truthfulness. Economists use all sorts of complex mathematical equations to beat each other up — more and more in public forums, sadly — instead of trying to reach useful conclusions. Got a pet cause? Get you some scientists who are willing to be advocates. The truth doesn’t matter. All that matters is that you are the ones standing for science and the other guys are not.

Average Joe Sixpack is not an idiot, no matter what the surveys may show. He’s aware of this at an instinctual level and for him, it all becomes just so much noise. Show him a new cell phone that uses some tech: then he’ll be a believer. It’s a wonderfully pragmatic approach. Don’t talk. Show me. Show me something that is immediately observable. It’s not his job nor should he care about the fine differences in certainty between inflationary theory and certainty that the Higgs-Boson particle has been found. The ironically tragic part of all of this is that it’s not Joe Sixpack who is harmed by this mess: it’s the guy who loves science and reads as much about it as he can. The guy that can make the biggest difference in a democracy. He’s the real victim. Joe is already using a heuristic that’s imminently practical and workable.

Science needs a new set of ethics. It needs to be clear when reporting medical studies what types of methods were used, what was brought into the study a priori, the sample size, and how it was reviewed. All the data must be open. Scientists should be clear when they are speaking on matters of reproducible science — such as the absorption spectrum of CO2, the Greenhouse principle, the fact that man changes his environment, and black-body physics — and when they are totally speculating, like predicting the global temperature in 100 years. These are different types of information. (I hate to use global warming, but it’s in vogue.)

If scientists want to be considered in the same boat as the guys who think aliens built the pyramids, then fine, they should keep going the way they are going. But I doubt that. So I would encourage, no beg, scientists to adopt much stricter standards of disseminating information. Scientists should love science first. They should not be public advocates for any cause. It’s a conflict of interest. They should only perform open and reproducible experiments. They should encourage criticism. They should reject herd mentality and stop playing the game of trying to publish papers with the least amount of controversial material in them. Peer review should mean something. And the public — of any persuasion — should be brought into the creation and review process as much as humanly possible. Scientists may not like this; it’s a totally new way of doing things. But it’s critically needed.

In addition to the comments and links provided in the article, one of our readers (Lee Killough) sent in some suggested reading. I can’t vouch for any of these, but the list looks interesting. I thought it was worth including. Here it is, along with his summaries. Looks like great resources for further investigation. Thanks Lee!

Brief description:

Scientism and Values: A compendium of economists’ and philosphers’ take on the misapplication of science to questions of value, like conflating means and ends. Just because science shows the most efficient means towards certain ends, does not mean it has any special moral authority towards those ends. Mises.org has put the pdf online, but I have a hardcopy. Murray Rothbard was one of the contributors. I did not read all the comments but I noticed property rights were mentioned, so this book is essential reading.[PDF]

The Mismeasure of Desire: A legally-minded philosopher questions essentialist assumptions of gay genes and critically analyzes some important “gay research”. I pointed you to a radio interview he gave, but the book is available too. Whether homosexuality is a choice, is a separate question from its legal protection, but the two are often conflated. [Yes, and as the linked article above clearly indicated, this use science as a weapon has no impact at all on how we should treat each other. Different subject entirely.]

Against Method: Physicist Paul Feyerabend questions the arbitraryness of the scientific method. This book shook the foundations of science, around the same time as Kuhn’s The Structure of Scientific Revolutions. [Kuhn's book is considered by many to be one of the most important books about how science is practiced written in the last 100 years.]

Who Rules In Science: A moderate in the philosphy of science debates tries to get to the bottom of the debate and proposes realism as an alternative to the extremes in the debate. The Sokal Hoax and other controversies are mentioned.

The End of Science: Horgan surveys a lot of problems in science and concludes pessimistically that it can never have all the answers, despite the confidence of some of its outspoken proponents. (He later wrote a book which expands this idea further into neuroscience/psychiatry.)

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

The Gap

The internet has a democracy problem. It has too much of it.

What? You might ask: how could having too much of a democracy be a bad thing?

Because pure democracies are non-functional. Some folks think if we each had a voting button on our iPhone that somehow the world would be a fairer place, but it wouldn’t. One week we’d be invading Egypt, the next week Libya. One month we’d be bailing out banks. The next month we’d be throwing all bankers in jail. One month we’d elect one bunch of guys, the next month we’d throw them all in jail. People are fickle and easily moved by emotion — pure democracy is the mob, which never is a good thing.

That’s why many nations have a representative republic: we elect people who then make decisions for us for a period of time. Using this “proxy”, we manage to slow down the tempers of the masses and get things done (or so the theory goes)

But you’d be lucky to get that far in a conversation online. Voting? Good thing! The more of it, the better.

I love watching groups of people reach decisions, and I love seeing how some structures work and some don’t. What I’m seeing online, however, is that hundreds of millions of folks are having discussions in which, although they know nothing of the background, they are perfectly willing to apply some 3rd grade social studies ideas to how the solution is supposed to work. As we continue to democratize everything online, this lack of foundational knowledge is going to really hurt all of us.

There exists a gap between what’s required to do stuff folks want to do and the actual knowledge they have to go make it happen.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Markham’s Scale of Ignorance

Yesterday a couple of readers gave me grief for all the “knowns”, “unknown knowns”, “unknown unknowns”, etc.. On top of that , the definitions got a little loose in the essay.

So instead of fixing the essay (Gad no! This is the internet! 2-hours work constitutes long-term commitment) I thought I would enumerate the scale of what you can know and what you can’t. I’m probably reinventing something from somewhere, but I think these distinctions are important enough to restate. One of the authors from the articles I quote came to the conclusion that you can deal with any amount of unknowns simply by knowing the questions. Hell no. That’s totally whacked.

Ten years ago I sat in the office of a high-ranking procurement officer in the military. He was a fast-riser, had a masters in mathematics and was a very sharp guy. I was explaining to him that the way the software development was going on a certain project was troublesome. The people, technology, process, environment, and bureaucracy were not working together. Instead various misunderstandings, agendas, confusion, and ignorance was causing chaos and poor performance.

It was a complicated discussion, made more so because each of the varying factors – people, technology, process, bureaucracy, environment – were pretty dang complicated in their own right. The way they all worked together — or were supposed to work together — was even more complex. Remember, this guy was probably a genius. Literally responsible for tens of billions of dollars. But he had no concept of what he didn’t know. It was like trying to explain String Theory to Julius Caesar. We just had no place to meet. Sure, given a few weeks of gaining some common understanding, this guy would be teaching me something. There was no stupidity at work — he was a brilliant man. He wasn’t even classically ignorant — it wasn’t like I could give him a class and a couple of tests and somehow that would fix things. We simply couldn’t communicate.

I’ll never forget what he said.

“I’m not sure I’m following you completely, but you see, I’m on top of the whole thing. I can ask any questions I like and get an answer”

My thoughts were: yes! But you neither know the correct questions, what the answers might imply, or how the answers to one question might lead to other questions!

Simply asking and answering questions is not enough. This guy had the magic power — whatever he asked, you can be sure that somebody was going to work as hard as they could to come up with an answer. And the project was still hosed up.

So in the interest of simplifying the discussion of how ignorant we all are in various ways, I propose the following scale:

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Back to the Darkness

In the modern world the stupid are cocksure while the intelligent are full of doubt. – Bertrand Russell

I haven’t been blogging much lately, and it all has to do with flying. Fifteen years ago a 22-year-old kid at the airport told me something that’s been resonating with me ever since.

In my mid 30s, I had decided to learn to fly. So I went to the airport, where I met a young flight instructor. He seemed like a nice enough fellow. He asked me about my desire to fly. Immediately I went through a list of things I’d like to do — first I’d like to learn how to navigate, and then I’d like to try out landing on grass strip, and then….

“Hold on a second there, hoss,” he said, “Right now you don’t even know what you don’t know.”

As I learned to fly, first as a private pilot, then getting my instrument, commercial, tailwheel, high-performance, and complex ratings (including trying out twins, seaplanes, and stunt planes, among other things) I thought a lot about what he said: I didn’t even know what I didn’t know.

He was right.

His point was that while I was very aware of what I’d like to do in terms an outsider would understand, I had no domain experience at all in aviation, aside from watching a few movies and being a passenger. It’s not just that I didn’t know the questions, I wouldn’t understand the questions, wouldn’t understand what they meant, what the answers might mean, or how the questions fit together. There was no basis for us to have an intelligent conversation.

So we spent a lot of time doing things not on my list: flying slow, approaching a stall, reading METARs, talking about maps, talking about priorities in an emergency (aviate, navigate, communicate), talking about all sorts of domain concepts, talking and learning. We created a common model understood overtly, tacitly, and functionally from which we could start to have a conversation.

You see, I thought I knew what I wanted, and I was right in a way, but in a more important way I was worse than ignorant — an ignorant person can be taught, he simply needs to be exposed to information — I was a stranger in a strange land. I was just some guy with a boatload of terms and stories that all kind of fit together in my world-view but had little credence in his — even though the terms were the same. First I had to learn and be able to physically and symbolically manipulate concepts about what I didn’t know, aviation, and then we could start talking about what I’d like to know.

The reason I haven’t blogged much lately because I am beginning to feel that the vast majority of what we say and do in the world is horribly incomplete. We’re all like kindergarteners to somebody else. We don’t know what we don’t know. This has very mportant consequences

This may have been meant as a political sidestep,
but is there something very profound here as well?

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

Ceterum autem censeo, Carthaginem esse delendam

(Note use of Wave for comments at bottom. Feedback appreciated)

I do not believe in evolution.

When I was in fourth grade, my entire class took a break for an hour while I had a debate with my friend David Hillon about evolution. He argued for evolution and science, and I argued for skepticism and faith.

The world, you see, was only a few thousand years old. The Great Flood made the Grand Canyon. There were too many gaps in the fossil record to demonstrate evolution. Great flood stories are part of every culture. After all, we already knew how things came about — the King James Bible — and as long as we understood that was true we could speculate freely about the rest of it.

As long as it supported long-established religious dogma, it was worthy of study.

I have been thinking about my fourth-grade experience and Cato The Elder quite a bit over the last several years.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

DGE Roundup 2

Over the summer I’ve read a bunch of books arguing for and against the existence of God. I’ve had to cut the series short due to work requirements, but I thought I’d draw some conclusions. This is that last of those posts.

So I think it’s fair for the reader to say, “Ok Daniel. What have you learned from all of this?”

I’ve learned quite a bit, actually.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

DGE Review 4: Jesus, Interrupted, by Bart D. Ehrman

(This is the fourth in my “Does God Exist” series of reviews. There will be six or seven books on whether God exists or not. I’ll read them for you, give a recap here, and then try to draw it all into some conclusions at the end. This is not meant as a religious discussion, more of an examination of the way smart people argue about really tough subjects.)

What would the Bible, the Christian’s holy book, look like if it were dissected by critical historians? That’s the question Bart Ehrman asks in his book “Jesus Interrupted”

It’s not a flattering sight.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

DGE Review 3: Atheist Delusions, by David Bentley Hart

(This is the third in my “Does God Exist” series of reviews. There will be six or seven books on whether God exists or not. I’ll read them for you, give a recap here, and then try to draw it all into some conclusions at the end. This is not meant as a religious discussion, more of an examination of the way smart people argue about really tough subjects.)

“The problem with those Christians,” my friend told me one night, “is that they want to run all of our lives. I’m gay, and they even want to tell me I can’t get married!”

I’ve had this discussion, or ones like it, many times before. There’s usually some initial charge that involves current politics, like opposition to gay marriage, or abortion. Then “the list” comes out. We all know “the list” by now: the crusades, burning of the Pagan temples, the Thirty Year’s War, trials for witchcraft and sorcery, and my favorite, The Inquisition (which Mel Brooks made into a wonderful musical, by the way)

This video is much too silly for this article,
but the song is stuck in my head. Now it can be stuck in yours!

The conclusion is then “Christianity brings out the worst in people” or better yet “religion is a meme”. It seems very fashionable among modern authors to go down the list, often at great length, in order to draw the conclusion that all religion is a sort of evolutionary hangover that mankind suffers from. Once we completely free ourselves from such superstitious silliness, only then will we able to move forward together.

David Bentley Hart is having none of it.

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.

DGE Review 2: “God’s Problem,” by Bart D. Ehrman

(This is the second in my “Does God Exist” series of reviews. There will be six or seven books on whether God exists or not. I’ll read them for you, give a recap here, and then try to draw it all into some conclusions at the end. This is not meant as a religious discussion, more of an examination of the way smart people argue about really tough subjects.)

If God exists, then why do people suffer like they do?

It’s a reasonable question, first put forward in writing by the Greek philosopher Epircurus before much of the Christian or Jewish Bible was written.

Is God willing to prevent evil but not able? Then he is impotent.

Is God able but not willing? Then he is malevolent.

Is he both able and willing? Whence, then, evil?

For anybody who has had a deeply-held religion, the problem of why evil and suffering exists can be a deal-breaker. For Ehrman it was.

When you’re a hammer, the world is your nail, and when you’re a professor of religious studies and the bible everything must go back there. So, like lemmings to the cliff, the second book in our series also discusses the question of God’s existence and suffering through the lens of Christianity and Judaism. But there is a big difference: these questions regarding suffering and evil are common across all religions, everywhere, so we have a broader mandate and there are many answers to choose from.

Bart takes us on a tour of the Bible, however, and finds that there are many answers to the question of why evil exists. Not all of them make much sense and they are not consistent with each other. In some cases original authors made one point and other authors came back later and revised it to make another.

So what are the biblical reasons for evil and suffering?

Continue reading

If you've read this far and you're interested in Agile, you should take my No-frills Agile Tune-up Email Course, and follow me on Twitter.