« My week| Main | Pop Music Hacking »

Putting the fun in functional programming

| | Comments (17)

A funny thing happened to me over the past two years of playing around with F#.

Programming became fun again.

I've always enjoyed programming, but in the OOP world, programming quickly became more about structure and frameworks than about solutions. When I learned C++ in Windows, moving from C, I went from a world in which small things happened in a particular order to a large world in which your code had to fit inside a rigid framework. The Windows message pump was butt-ugly, but at least you could look at it and understand it from top to bottom. MFC did the same thing wrapped up in classes. While simpler, it was also farther removed from reality.

And that's the way most OOP solutions were constructed - abstracting the problem away from the metal far enough that you were dealing in abstracts. Need to make a new windows form? It's CForm, or System.Windows.Form.Fom -- whatever it is, you just make a new one. It knows all about how to run forms on Windows. OOP is a beautiful thing.

But with abstractions come gaps in knowledge. A lot of the problems I see that Windows programmers have is related to not understanding what's going on under the hood. Need to do a lot of string building? Just use the "+" operator over and over again. Never mind that each string addition could cause a GC. (There is a StringBuilder class for just this scenario) Need to run a bunch of things in parallel? I'm sure there's some objects out there for that too.

I like good Object-Oriented Programming. On a good project you design the solution, create the scaffold, then plug in the details. Everything has a place. It's easy to share the codebase. A defined problem has a defined (and architected) solution. Planned changes lead to built-in flexibility in your model -- if you're smart. OOP methodologies are software engineering.

So it was with some trepidation that I ventured into scripting and functional code. It all seemed so idiotically simple. After all, who couldn't write a function? Who couldn't refactor a function? These were the simplest of skills, not worth much discussion after you learn and begin applying best principles of coupling and cohesion.

But I found out some really neat things. When I first wrote my first functional code, I would write these butt-ugly imperative loops and such. I might end up with 80 lines of code.

But then I started refactoring, condensing, generalizing. As I worked my code, hidden similarities revealed themselves. Pieces came together in unexpected ways. Like some kind of weird magic trick, 80 lines of code became 50, then 20, then maybe 15 lines or so.

And it was a thing of beauty. It was impossible not to realize this -- being able to express a solution to perhaps a complex problem in just a few lines gives you the feeling of getting closer to knowing some kind of ultimate truth. It's one thing to understand currying and functional composition and all of that. But it's another thing entirely to watch these tools come together in these unexpected ways to make these elegant solutions that you didn't see before.

In OOP, you know the answer before you start. It's all laid out in your OOAD. In FP, even though you have the same general understanding of the answer before you begin, there's a "revealing" that occurs as the code tightens up. It gives you the feeling of solving a puzzle, not just slinging code.

That was fun enough, and I went on for several months annoying all my friends with how "clean" F# was to code. It got better, though. Sometime in the last year or so I found myself doing something frequently that I very, very rarely did before.

I found myself re-using code.

Everybody talks about re-using code. It was supposed to be one of the huge benefits of OOP, but it never worked out that way. The problem was that OOP code usually lived in these huge frameworks. They had to inherit from such-and-such a form. They had to use all these libraries. And an object was a monolithic thing. In for a penny, in for a pound. If done correctly, you ended up with these nicely-sized hunks of data and code -- which were specifically constructed for that particular problem in that particular environment.

But FP isn't like that. I remember the time I wanted a function to slice lists. Given a linked-list, slice it into 2 pieces at midpoint x. I could have developed the solution, but I just googled it and found a neat little piece of OCAML code that did it in just a few lines.

On another project I had a requirement to take some data objects and convert them into JSON to send over the wire. The "old" way would have been to write a "ToJson" method on each object, hand-writing in the return string (or using some kind of code generation to automate it). This time, however, I decided to bite the bullet. I wrote a generic app that takes any .NET object and makes a JSON string -- even complex objects. As part of that, I ended up with a nice little function that will walk a .NET object, providing a callback. That means you can do all sorts of things, not just write JSON. You could take the same code and use it for binary persistence. Or for runtime reflection. There are all sorts of little uses.

I became a code-snippet collector.

It's like collecting stamps. I have this neat piece of code to create a irregular-shaped window. One for transparent bitmap buttons. Another one for playing disk sounds. Here are some type extensions to handle the old-style collections. Here's another one for posting messages from a worker thread back to the GUI.

These are all things that I have done before, and quite frankly, they're not that hard to implement. Before -- if I had the time -- I might write up some of this in a class. Say 100 lines of code. But who the heck wants all of that trouble? Most of this stuff is just a few lines of F#. Small enough to be portable and easy-to-use, yet complex enough to warrant keeping around. Any simpler and it wouldn't be worth keeping. Any more verbose and it'd be a pain to keep and reuse.

I've also found that these pieces come together wonderfully in "exploratory" programming. Not traditional business programming, where you know the problem, you've defined the problem, you've designed the solution, and now you go code. Nope. Exploratory programming is much more like painting a picture. Does that look good over there? How about like this?

And frankly, exploratory programming is more fun than fixed-problem programming.

F# has this wonderful ability to cross-cut the existing .NET libraries to create ad-hoc solutions. Instead of being concerned with structure so much you think more about results. The users. Big pieces of work get done in very few lines of code -- and I've found that fewer lines of code mean less bugs.

There's another subtle reason FP is so much fun. Because of the way it works, testing doesn't have the same role. FP, at it's heart, is about transforming data. It's like writing an entire program as one big honking SQL statement -- you just select from here, compare with that, insert over there. As such, there's not a lot of room for error to creep in. It's either right or it's wrong. In a true FP, unit testing is not the nirvana it is in OOP. The tricky part, of course, is getting your data structures optimized for the transforms you want. And to aggressively refactor everything you write until it's tight. But when it works, it hums. It really sings. It all comes together for a really nice programming experience.

Fun stuff!

17 Comments

I find the organic approach to writing code is something to which functional languages are very well suited.

Paul Graham talks about Lisp being organic and supporting both top-down as well as bottom-up programming styles in his book On Lisp.

hmmmm ... this is how Linux works too. Little snippets (programs) that you can use to create powerful program i.e. Piping one programs answers to the next and so on. Very powerful and fun at the same time but "Functional" :-)

nice article, i agree, functional programming is really fun and the nicest paradigm in programming.

I see a considerable gap between the description of the author's elation, and the description of the actual change in coding practices.
The examples given are rather trivial and many of us have written such reusable utilities in C** and other procedural languages. General data structure manipulation, object traversal etc. -- these are not functional programming trophies even by a wide stretch. I don't doubt that functional programming may be fun, rejuvenating and inspiring, but this conclusion is not a corollary of the story told in this post.

David,

Very interesting comment! I was not aware that "fun" was somehow a corollary. I just thought I'd tell you folks why I was enjoying it so much. Very sorry if my proof failed to satisfy. And thanks for pointing out the "trophies" remaining to be gained. What with all this logic and rewards, sounds like an interesting adventure awaits.

Do you argue much with folks who say they like chocolate ice cream? I was thinking about writing my opinions on ice cream. Just wanted to make sure it held up to your standards first.

Cheers.

It reminds me of the days of assembly language "cookbooks", little routines for outputting characters, executing beeps(!) and drawing your own window borders.

For all the complexity we've learned to abstract, there's still a desire for those "wow" moments where the whole is laid out uncovered.

Now you can take all those snippets and write a framework around them...oh, never mind...

Oddly enough I have the same feelings for Python's functional programming support. I've found that closures and the ability to pass functions along (as arguments and return values) open quite the avenue for various forms of code reuse not practical - if at all possible - in "pure" OOP languages.

My current Python project is a GUI framework for applications that "chat" with the user, in a way akin to how people talk to each other over MSN and other chat clients.

Because I want it to be as simple as possible to use and extend, I went for a function-oriented API, with only a couple, very basic classes, while most of the implementation goes in top-level functions.

As I worked on the implementation, not only I quickly found myself reusing the very functions I meant for API programming, but a whole lot of API-useful stuff sprung out of my refactoring efforts.

So yeah, FP is great. Now if only they got those high-performance Python projects right!

David,
Great article. I've always liked functional programming. The problem that many developers have is that they try to make everything an object oriented program, weather it should be or not. Many development tasks just don't require it. As technologists, we should just use the "right tool" for the job at hand.

Here is an article you might find interesting:
http://www.stsc.hill.af.mil/crosstalk/2008/01/0801dewarschonberg.html

it's about the pitfalls of using Java as a first language in Computer Science education. It echos your sentiments of abstraction.

Regards
Biff

Has any ever read Turing's paper on decision problem in which he laid down the foundations of Computer Science?

I mention it because if you read through his description of the "Turing" machine, he shows the power of the concepts of functional programming, namely functions as first order elements and recursion. I think the book "The Annotated Turing" should be a must read on any developer's book list.

We all know coffee bean ice cream is the best anyway.

Hi Daniel.
I'm havng F#UN too.

One of the things that attracted me to F# was the promise of less code between me and math. I'm not at Dr. Jon Harrop's level (who is? Don Syme ...), but I've enjoyed and learned from his math sci and tech writing.

BTW I love grape nut ice cream.

Nice article, I haven't had much opportunity to work with F# in depth. However, the examples I have seen and the items you have mentioned are very interesting. Would you ever consider posting the code to some of these snippets of functionality in F#?

An enjoyable article, and the few "imho" comments that follow are in no way a criticism of the author. To put the comments in context, please note that while I have not dabbled in F# (yet), I had a lot of experience in LISP, and PostScript, but C# is my current area of focus and work, and I would put forward the hypothesis that C# with the addition of LINQ and Dynamic operators, Lambdas, etc. does now allow one to do the equivalent of functional programming.

1. refactoring and "tightening" are in no way "bound" to functional programming inherently.

2. compiling libraries of useful re-usable snippets also "belong" to no style/language in programming.

3. in (now rare ?) "auteur" programming, where one programmer, or a very small team, "plays god," and has 100% control, and responsibility, for the code, the final product, etc. then creating the absolutely most elegantly compact code is fine ... but in most real world situations teams of programmers are at work, and exotic elegance is often incompatible with team-work and requirements for testing.

4. to further expand #3: I'd propose that the need for careful ... even elaborate ... documentation exponentially increases as functions are reduced to bare, elegant, minimum, in languages like LISP.

In a "zen-like" quest perhaps thirty years ago I spent two weeks coming up with what I thought was the absolutely minimal LISP function that given two integers as input parameters created a typical two-dimensional array data structure. I ended up with about four lines of doubly-recursive code: two months later I couldn't visualize how the function actually worked ... and that wasn't "satori" (enlightenment :).

The "in for a penny, in for a pound" critique is a useful thought-experiment, imho, but for those of us writing programs with an emphasis on user-experience, using 3rd. party tools for really powerful front-end visual controls, such as a TreeView that out-of-the-box gives you virtualization and drag-drop saves an incrdible amount of time.

DBMarkham wrote: "I wrote a generic app that takes any .NET object and makes a JSON string -- even complex objects. As part of that, I ended up with a nice little function that will walk a .NET object, providing a callback. That means you can do all sorts of things, not just write JSON. You could take the same code and use it for binary persistence. Or for runtime reflection."

How about publishing this on CodeProject ? Or in a blog ? Would love to see this code, and perhaps seeing that code might "seduce" me into starting looking into F# :)

thanks, Bill

Great comments, all.

Just to be clear, I wasn't espousing F# (and FP) as nirvana. There are many, many, many problems with taking functional code into some kind of mixed-mode, multi-programmer development shop. I'm the first to acknowledge that. Hell, I could do another article on the problems with F#! But all of that will come out soon enough, as folks try to scale.

The last thing I saw Syme do -- a video I think -- he was emphasizing FPs role as a prototyping and exploratory tool. Presumably you "sketch" out rapid prototypes in F#, then wrap it up in a traditional class hierarchy. Makes sense, and I'm for it. Although I wonder how far you can go with F# without stubbing your toe badly. I know with the current prototype I'm working on I had to mix in quite a bit of UI elements. Once you fall down the "class trap" the code starts to get clunky and big again (although better-organized and easier to maintain)

I'll consider posting the JSON code in a future article. I'd like a day to test it first before I go slinging it out there, so it might be a while. I hate to sound like a mercenary/pirate/jerk, but I'm also wondering what the value of some of these little nuggets are? Perhaps I should just keep them and keep assimilating various tools until I find something that has commercial value. Perhaps over time, as my toolbox accumulates into the dozens of tools, they start coming together in new and wonderful ways. Could be a startup in there.That sounds sucky, I know, but there it is.

Bill made a wonderful comment about the richness of UI tools, especially in windows and on the web. Things like the TreeView, or complex grids.

I am beginning to suspect that many of us have been led down the garden path, given more and more tools with more and more features which leads us to concentrate on flash over substance. There may be a huge gap between what we need to do and what we end up doing, and most of that gap may be because of a distraction-rich coding environment. Don't know. But it's something to think about.

U really R 1 with the force!

Oh great, now you gave me another language to learn! :) I'm going to try it out. I code "loose" at times so the more it helps you out, I say, the better! good post.

Dear daniel, nice story. You might
like to see: 1. "how to write a computer program"
(www.civilized.com/programming.html )

and

www.civilized.com/getlisp.html to get the book on Lisp.

Leave a comment

About this Entry

This page contains a single entry by DanielBMarkham published on August 23, 2010 4:42 PM.

My week was the previous entry in this blog.

Pop Music Hacking is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Social Widgets





Share Bookmark this on Delicious

Recent Comments

  • gary knott: Dear daniel, nice story. You might like to see: 1. read more
  • Michael Fever: Oh great, now you gave me another language to learn! read more
  • CMO: U really R 1 with the force! read more
  • DanielBMarkham: Great comments, all. Just to be clear, I wasn't espousing read more
  • Bill Woodruff: An enjoyable article, and the few "imho" comments that follow read more
  • Daniel Petersen: Nice article, I haven't had much opportunity to work with read more
  • Art Scott: Hi Daniel. I'm havng F#UN too. One of the things read more
  • KenK: We all know coffee bean ice cream is the best read more
  • Delmania: Has any ever read Turing's paper on decision problem in read more
  • BiffSocko: David, Great article. I've always liked functional programming. The problem read more

Information you might find handy
(other sites I have worked on)





Recently I created a list of books that hackers recommend to each other -- what are the books super hackers use to help guide them form their own startups and make millions? hn-books might be a site you'd like to check out.
On the low-end of the spectrum, I realized that a lot of people have problems logging into Facebook, of all things. So I created a micro-site to help folks learn how to log-in correctly, and to share various funny pictures and such that folks might like to share with their friends. It's called (appropriately enough) facebook login help