Is Modeling Science?

Let's look at the simplest model, F=ma. Its a set of symbols that, when executed, reflects how force, mass, and acceleration work in the real world. We look at the model, we make measurements, we plug the measurements in to the model and run it, and we observe how close the model is to reality. Over time, we forget that these are just symbols that match up with our observations, and start thinking about them as "laws of nature' Mostly, this doesn't matter, especially with simple models like F=ma.

F=ma is a model of a single theory with simple, discreet variables. It has high fidelity, which means that when we do something in the real world the model reflects the actions in the world. Since models are collections of theories and interdependencies and need to be "executed", all computer programs are models. That's why some of us use UML and OOAD tools to organize (model) our systems prior to developing them.

We can do lots of useful things with it. Given a model for friction, reflection, and a couple other things, we could construct a fairly neat mathematical game of billards. Rather, it would be a model of the billiards game, which generally satisfies the desire that the model has enough fidelity to make the game enjoyable. I love playing pool on the computer.

But it's not billiards.

Certainly we could add relativistic equations, slope of the ground, noise, seismic vibrations, bare spots in the cloth, imperfections in the balls, spin of the balls. The list goes on, quite literally to infinity. Given any model of billiards, it is always possible to add more factors to the equations. These factors would make the model have more or less fidelity to some real world system in question. Sure, some of the variables, like the barometric pressure, have little to do with my _enjoyment_ of the game, but they make the model more faithful nonetheless.

Does mastering a flight simulator teach you how to fly? Does it teach you how airplanes work?

Probably the model that has the most programmers and dollars thrown at it is Microsoft's Flight Simulator. It's a gorgeous product that has the physics of flying down flat. Flight physics are well-known, and a lot of test-pilot time has been spent observing, measuring, predicting, and tweaking models. So in most cases, FS is going to behave like a real airplane will. What if you flew inside a tornado? Or tried to use the plane as a submarine? In cases where no observations have been made, the computer continues to apply theories, even if the results are wrong. It's not like a big red light goes on or anything. The computer doesn't make any judgments about the quality of the theories in the system. It just runs them.

I have a model I use for software estimation. I plug in the use cases, and some more stuff, and the computer predicts how long the project will take. There are some really fancy models for software project planning, like COCOMO II , or SLIM. I usually plug some numbers in, take a look to see if I like the results, and keep poking around until it looks about right. I have a feeling most people use software estimation models like this. Same thing happens in financial forecasting models for businesses. In the end, your model is just a reflection of your pre-existing opinion wrapped up in a bunch of computational rules. Nothing wrong with that -- it certainly helps organize your thoughts. It's a way of reviewing the interdependencies that you already knew existed but forgot or were unable to correlate all at once in your head. But it's not learning -- the stuff that's in the model is the stuff you put in there. It's just a way of organizing your theories on how the world works.

Models are really useful.

Science is about the theories that make up the models, not the models themselves.

Things get really strange when you introduce recursion or Cellular Automata-type attributes into your model. Whenever you have a huge number of discrete objects that are all interdependent on each other, you have a sort of CA program going on. Scientists do this when you there are a huge number of particles -- like the molecules in the atmosphere. So you see this in weather forecasts and Global Warming models for example.

You also see this in games like the Sims --you have thousands of little people all acting and reacting to each other. Also in a lot of computer strategy games.

CA is a strange animal. Sometimes initial conditions are important. Sometimes they are not. Sometimes small perturbations drastically change the entire system. Sometime even huge changes fail to change the system much at all. They are non-predictable in a way that simple models are not. And great amounts of complexity can come from very simple rules.

When everything depends on everything else, ad infinitum, small errors in any supporting theory propagate into the model quickly, making CA models difficult if not impossible to maintain fidelity. Take a look at the Sims. It has to be highly "tweaked" to make sure the game is fun, that is, that it looks and acts like real people might act. Systems such as these ALWAYS run out of control -- that is, limiting factors have to be randomly added to prevent some sort of runaway condition in the model. Too many people, one strategy always works, etc. In fact, the only two end states for a complex model such as this is runaway in one direction or stabilizing in a chaotic fashion. (There's a third state of oscillation, but I consider that stabilized for our purposes)

Ever notice how the 10-day weather forecasts trends towards seasonally average conditions once you get past a few days? That's because the model has been tweaked to gravitate towards averages to prevent a runaway condition. In fact, runaway models are the default -- even in systems where the observation is that the system is in stasis, models will indicate things getting quickly out of control. Because of the computationally difficult nature of CA modeling, it's not uncommon for their predictive nature to only be good for a short time from a measured initial condition. With lots of practice, models can be improved, but there is a natural limit to their ability to be faithful to reality. This is the nature on the type of programming involved, and is NOT related to whatever the subject of the model is.

The natural thing to do when your model is whacked is to add more theories. After all, certainly the more stuff you put in there the closer the model will be to observations, right?

One might have assumed that among different processes there would be a vast range of different levels of computational sophistication. But the remarkable assertion that the Principle of Computational Equivalence makes is that in practice this is not the case, and that instead there is essentially just one highest level of computational sophistication, and this is achieved by almost all processes that do not seem obviously simple -- Stephen Wolfram, A New Kind of Science, page 717

Wolfram's book is called "A New Kind of Science", and that's exactly what it is -- it is not science as we know it. There is a rich and fertile field in modeling and CA, but it's not science.

Let's review:

• Models are collections of theories which are always provisional


• Models are incomplete by nature


• Models are by nature heuristic, not scientific<\li>
  
• Once any sort of CA or recursion is introduced into a model, they become non-predictive. That is, the only way to see what the model does is to run it<\li>
  
• While models can be more or less faithful to reality, adjusting the model involves operating on multiple theories simultaneously, which violates the precept of science that you observe, theorize, and test one theory at a time


• To make a complex model more faithful, it is necessary to run it hundreds or thousands of times, comparing it's predictive value to what actually happens. This gives you the heuristic, or rule-of-thumb, information you need to tweak the model


• Because of these factors, creating a model is not like testing a hypothesis or making a discovery

I love models, and I get the feeling a lot of scientists are really digging them too. But CA models are not science. There is no observation, theory, and test involved. It's just a reflection of the common wisdom of the scientists creating a model. There's nothing wrong with that, and certainly there might be some insight gained from using models because the computer is able to do so many calculations so quickly, but it ain't science.

We've reached the point where scientists are using models to capture huge numbers of theories and their inter-relatedness. This is a great thing, as long as it leads to more observation, theory, and testing, which it can. But it can also be a form of intellectual tom-foolery, whether intentional or not. You put together a bunch of theories into a cool program that does all sorts of graphs and charts and announce what the ocean's temperature is going to be in 200 years. There's no way to check your prediction except for waiting for 200 years and seeing how it goes, so you're off the hook. You can claim great discoveries and observations when in fact nothing of significance is going on.

I'm of the opinion that we may be seeing the same phenomenon in string theory. People throw every open-ended theory they have into a big pot and come up with a super huge, amorphous pot of intellectual stew. Little bit of everything in there. With at least 10^500 universes, and a theory that can spin and dance like string theory, it appears you have something when, in fact, there may be nothing of new value there at all.

We need to keep funding research into the atmosphere and theoretical physics, but we also need a healthy dose of skepticism when it comes to what we're paying for. It's easy to appear to be doing something, even fool yourself into thinking you're doing something, when in fact you're just re-hashing stuff that's decades old and was probably not that good of a model to begin with. It's the age of the computer model, and computers will be able to create and amalgamate models with such speed and precision that it can make your head spin. But we can't confuse speed and accuracy with intelligence and insight. Our future depends on it.