« Craigslist Spambot Attack| Main | Answering PG'S Arc Challenge: On the Road to a DSL »
The Outsider
I was reading John Graham-Cumming today -- he makes the case that amateurs have a long tradition of helping scientists. He uses several examples, including his own discovery of a data error in some climate data -- an error that was promptly acknowledge and fixed by the scientists involved.
Others have made the case that as science gets more and more complex, the amateur really has little to offer professional scientists. There's simply too much complexity nowadays.
I don't buy that, and here's why:
As a solution developer and management consultant, I've worked with teams all over the spectrum of human endeavor. Programmers especially have to learn a new discipline every time they step into a contract and begin delivering a solution.
Many, many times -- too many to count -- I've seen ignorant amateur outsiders lead a group of people to a breakthrough in one area or another. Previously irreconcilable issues become easy to handle. Problems that seemed insurmountable suddenly become trivial.
As a coach and trainer, I see this even inside of delivery teams themselves. Teams careen off the rails and nobody knows why, yet everybody is very intelligent and acting in the best manner possible. Deadlines are missed even though heroic efforts are being made. Then suddenly an outsider comes in, asks the team to explain how things work, and everything gets better.
Why is that?
First there's the issue of simultaneous cognition. The human brain can only hold about 4 or 5 orders of complexity in it's head at the same time -- and that's genius level. And it's not a issue of knowledge, it's an issue of span of control. Although I may understand how electricity works, how a network card works, how a computer works, how the internet works, and how web pages work, I do not keep all of that in my head while I'm making a new web page. Instead I just work at the higher abstraction level, leaving the rest of it to "just work".
This means that to work at a really high level -- say cutting-edge science -- you have to work on lots of assumptions about what's true most of the time. Most of the time, of course, you don't have to worry about these things. But every now and then there's some coupling between a low-level detail and a high-level one and it's completely off your radar. A fresh face, which has to understand even the basics before they can get the advanced stuff, is the prompt needed to remember and apply the coupling.
Then there's a close relative, hidden assumptions. Technology teams are really good at this but it infects any group of specialists. A new effort begins, people begin to assume roles and help each other out using patterns from previous experiences. Quickly a lot of "rules" are made even though they may never be expressed: Joe is going to do builds, Sue is the person for user-interface design, we only make database changes on Tuesdays. These hidden rules can quickly get very complex and begin to hurt team performance. If you're doing it the wrong way, no matter how hard you try you're not going to suddenly do it the right way. It's just going to hurt more.
The outsider here simply asks why folks believe the things they believe. Why is Sue the person for UI work? Doesn't anybody else know how interface design. Sue speaks up and says she's actually an expert on database design. Suddenly we realize the team is out of whack. One of my favorite recurring examples of this is the team that does all the easy work first -- after all, it's important to get momentum! They quickly find themselves in sprint 7 with really tough problems and lots of refactoring to do. The unwritten rule was the team was so poor that they needed encouragement more than they needed to accomplish things.
Outsiders, amateurs, and ignorant people are extremely important in any kind of technical endeavor because they force us to unmask and deal with hidden assumptions. The more technical, the higher the level of abstraction, the closer and smaller the group involved, the more outsiders are needed. Companies that employ true consultants (and not just consultants as staff augmentation) understand this: they bring in people who are great facilitators, great designers, great people at handling and organizing complex systems, but mostly great people at asking stupid questions. Because it's the stupid questions that trip up the smartest of us.
As someone who is studying to be a scientist, I know that this is nonsense. Amateurs help all the time. It was amateurs that helped discover that cilia were important in the asymmetry of human beings (our organs). I see all the time posters around campus of things that amateurs have done in their research. Amateurs are a tremendous help, not to mention they're willing to work for free to get their feet wet.
Yeah, I'd say that science does suffer from a degree of groupthink, but I must stress that word degree. There are lots of cases of scientists disagreeing with each other and publishing reports with contrary information. So I don't want to say that science will never change, but it will be slow. One of the major reasons for this is the way that we teach science. Most of the classes that I take just regurgitate information rather than showing you how the conclusions were drawn. What would be much more effective, rather, is a presentation of how the information was discovered and the debate that occurred over the idea. It would make for much more interesting classes and would spur new ideas. I don't know why our schools don't take this approach. I'd say national standards for schools, but it's an approach that we need to introduce to more people if we want our schools to change.