The only thing I really regret about not getting a formal CS education is that nobody ever made me learn a functional language. (It goes without saying that I'm too lazy to learn one in my spare time.) I read stuff like items 6 and 7 in the "Closure semantics" section (the bound variables bit) and my brain just locks up.

Posted by: David Moles at January 4, 2006 06:43 PM

But that's the really *cool* stuff!

It just means:

public Closure getTripleMultiplier(int factor) {
int t = 3;
return { a -> a * factor * t; }
}

by9 = getTripleMultiplier(3);

assert( 18 == by9(2) );

That's sort of pseudocode, but you get the idea. The closure knows about the variables' state at the time that it was defined.

Oddly, Javascript 1.2 is actually very close to a functional language, and has closures too. So maybe you do know a functional language, you just don't know that you know one.

Posted by: Benjamin Rosenbaum at January 5, 2006 10:15 AM

Another way to put this is --

in Java, I miss operations that perl has that take a block of code -- like sort, grep, map. I don't want to have to define a new interface and then implement it with a callback to do the filter or mapping -- I just want to pass in the damn block of code.

In Perl, it's great to have sort, grep, and map -- but that's it. You can't create *another* function that works the way they do (not without a bunch more syntactical superstructure workaround, just as kludgy as the above workaround in Java).

In Lisp, I could just make functions like that to my heart's content.

I don't know Python or Groovy yet, but it looks like they might be more Lisp-like.

Posted by: Benjamin Rosenbaum at January 5, 2006 10:19 AM

And yet another way to put this (just to further bewilder and bore most of the people who read this blog, which heretofore has not been a programming geekery blog; sorry about that) is that most of the Design Patterns in the Design Patterns book are actually symptoms of a mid-level language.

My gut feeling is that the utility of programming languages is trimodal.

On one end you have programs that are best done in Visual Basic or some other what they used to call 4GLs -- or else in scripting languages used as scripting languages. Programs that are, in one sense at least, not hard, and which need to be flat: the logic is straightforward, and the team you have doesn't want to traverse an inheritance heirarchy to figure out what's going on. Maybe you have a large, inexperienced team, or maybe you're going to use the program a few times and throw it away. So you don't care about refactoring much.

The second mode is programs that need to be long-lived, flexible, and robust, and have a lot of business/domain logic which is sort of arbitrary -- there's a limit to how elegantly it can be compressed down, because it was made up by business folks and there aren't any set of simple, coherent underlying principles you can reduce it to. So the real problem is organizing this large set of arbitrary, somewhat incoherent, changing rules and behaviors. I have a feeling that this is what object oriented languages are good at: they provide an infrastructure for you to organize all the logic.

The third mode is where you have programs that are hard in a good way: where you have a hard problem, but the difficulty is a feature of nature, not of a mass of sort-of-contradictory requirements. Something like, say, the core algorithms of indexing the web like Google does, I imagine would fall in this category. Or maybe not? I'm thinking of cases where it'd be a win not to have a large number of somewhat skilled programmers, but rather 1-3 extremely gifted programmers. In this mode, I'd imagine that you want a very concise, powerful, elegant high-level language (and then you want to profile it and rewrite the bottlenecks in C).

In that sense of truly high-level language, for that kind of problem, you can look at the Gang of Four Design Patterns as "language smells" analgous to Fowler's Code smells. If a language forces you to create a Visitor heirarchy rather than just passing in a block of code, it's a mid-level language, along the axis I'm describing.

Posted by: Benjamin Rosenbaum at January 5, 2006 10:34 AM

Oooh, "Let's Pretend" is momentous! I LOVE that series! I think my favorite story is "How the sea became salt." Do you have that one?

Posted by: Ethan at January 5, 2006 01:04 PM

What? What?

I meant really let's pretend, like when you play let's pretend.

But the series sounds great. Books? Amazon has nothing.

I find only this...

I always wondered about that...

Posted by: Benjamin Rosenbaum at January 5, 2006 01:11 PM

The closure knows about the variables' state at the time that it was defined.

(Emphasis added.) Okay, now I get it. It's analogous to how you have to declare things final if you want to reference them in an anonymous inner class. (Though with closures I suppose you could fiddle with the variable =after= you declare the closure... or maybe even in the closure itself... though I'm not sure what the expected semantics of the latter would be w.r.t. the enclosing code....)

Though the fact that it now doesn't seem nearly as magical makes me think there's probably still something I'm missing. :)

Interesting point about patterns. I'll have to think about that. (But I've certainly spent enough time wrestling with out-of-control visitor architectures to find the idea appealing...)

Posted by: David Moles at January 5, 2006 01:38 PM

Er, before declare things final insert in Java.

Posted by: David Moles at January 5, 2006 01:38 PM

For a sense of why this is useful, look at their example of GVector.apply, and the section on currying.

Think about how much time you spend traversing data structures and writing loops that interrogate them and do stuff to them. Visitor pattern and its ilk are clunky ways defined outside the language to turn such loops inside out by saying "first define how to apply some actions across this complex data structure. now I'll write those actions." If you can write apply() as you can in Groovy, you have that in a very svelte format, basically defined into the language.

Another way to look at this: encapsulation, inheritance, and polymorphism are things you can do in C as design patterns. You could imagine a Design Patterns For C book that talked about how to implement your own vtables and (perhaps runtime) access control. In C++/Java/etc., you get those particular design patterns built into the language.

With perl's map/grep/sort, you sort of have Visitor pattern -- or at least the most common subcases -- built into that language in the same way.

With a truly high level language (which it's my impression lisp is, for instance, though my lisp skills are not what my C++ and Java skills are), you can rewrite the language, so you can write new design patterns into it in the same way.

Currying gives you another axis to refactor on, and a very flexible one. You can take a very general purpose function with lots of parameters and then curry it by fixing some of its parameters (which might themselves be closures).
Instead of an inheritance tree, or (because everyone knows inheritance gets rigid) other workarounds like heirarchies of Decorators and Adapters, you can pass something into a function and alter its functionality on the fly. "Old code calling new code" at a finer granularity.

Posted by: Benjamin Rosenbaum at January 5, 2006 01:56 PM

That is in fact the story I particularly like... Let's Pretend is an old radio show of fairy tales. I was just bonking myself on the head for not thinking of getting it for Aviva & Noah before someone else did! A set of 51 shows is on its way (tell them its a late b-day present):

"Heel-looo, Pretenders! Helllloooo, Uncle Bill!"

Let's Pretend was the major life work of Nila Mack, a Kansas woman who had been an actress on Broadway and in vaudeville. Mack felt the best way to tell a children's story was to let the children tell it: to develope a company of versitile juvenile talent who could play a variety of changing roles week after week. Child actors played parts of characters their own ages to 80 year old people. Mac "trained and directed two generations of child actors" before passing away in 1953. She was lauded as "the fairy god-mother of radio."

SOURCE: The Encyclopedia of Old Time Radio, John Dunning

Posted by: Ethan at January 5, 2006 01:57 PM

Crazy - I just went over to read about closures, and it hurt my brain, but in a good way. Seems like a closure lets you define a variable as a method, then treat that variable like a class method, yes? Is there anything else fundamental I should understand about closures?

peace
Matt, dung-beetle of coders

Posted by: matthulan at January 5, 2006 04:00 PM

Well, it lets you stick a method into a variable, and pass it around -- like function pointers in C++. That's one thing. But it also does two other things: it remembers the whole state of the local frame (local variables, etc) at the time of its definition, and it lets you bind its parameters at different times - that's called currying.

So say you have a closure that takes three arguments. You can just bind the first one, making a new closure, then pass that closure around somewhere else, and later set the other two arguments and call the function (as many times as you like, with different arguments for the latter two functions).

Consider that you can pass closures into closures as arguments, and that saving the state of all the local variables (which may be objects) means closures, like objects, have state as well as behavior... and you can see that this is an insanely generic and powerful mechanism. The object-oriented paradigm the four of us are more used to is a way of selecting a particular, limited subset of this power, for sanity's sake.

Posted by: Benjamin Rosenbaum at January 5, 2006 04:07 PM

David, I think the closure has a copy of the stack, so if you write to the local variables that doesn't get propagated to the local variables in the context outside the closure (typically they are gone anyway).

Posted by: Benjamin Rosenbaum at January 5, 2006 04:10 PM

"Let's Pretend" was MY favorite radio program when I was a child! In fact, the radio program I was ON, as a child actress, was the Cleveland version, a product of Station WERE, called "WERE Pretending"! I want to hear what Ethan sent when I can, OK?

Love, Grandma

Posted by: Karen Rosenbau;m at January 11, 2006 08:40 PM

*I* want to hear you as a child actress! Was the show recorded?

Posted by: Benjamin Rosenbaum at January 12, 2006 09:23 AM