• jQuery PeriodicalUpdater: The main function now returns a handle that can be used to call stop(), thereby ignoring any updates that may come back and preventing future updates from being sent.
• TestingLabs: I released TestingLabs 0.4 to work with Grails 1.2.0. Had a bug with versioning under Grails: More info on JIRA.
• Presentations: I’m now storing the slides for my presentations on GitHub. They’re under a Creative Commons License.
• ClosureBridge: This is a tiny library (up on the repo.smokejumperit.com Maven repo) that provides a link between Groovy’s Closures and Callable/Runnable/Java code.
• Fun with GPars: A small library where I was experimenting with GPars
• GPars: Submitted a fix and generally been discussing things with the GPars community (and by that, I mean Vacalv Pech)

See my GitHub page for more info.

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (203.215.201.194) )

I complained that Scala did not seem to be very functional to me, but I didn’t really know how best to express what was fundamentally wrong with it. I did know that if “functional languages have a fixed set of features” like Scala’s creator, Odersky, claims, then it wasn’t simply “first-class functions in there, function literals, closures”, “types, generics, [and] pattern matching”. Scala has missed the functional boat in some basic way.

After a kerfuffle in the comments, Brian enlightened us all by telling us what is a functional programming language. His explanation (while being a self-admitted generalization) is summarized as follows:

So, what is it that differentiates the functional programming languages from all the other programming languages? It is simply this: the functional programming languages use, as their fundamental model of computation, the lambda calculus, while all the other programming languages use the Turing machine as their fundamental model of computation.

Six months later, Odersky responds with a very interesting post, which actually agrees that Scala is not a functional language in Brian’s sense, but instead argues that any language is functional if it “makes programming centered around functions easy and natural”. He then runs through a list of features which is in common with functional languages, noting that Scala has them within handwave enough (more on that later). He ends wishing that people would “stop thinking of functional programming as a different, novel, or exotic way to code”. Even more, though, Scala is apparently “an early example of a new breed of postfunctional languages”.

And that gets us to this blog post.

First of all, Odersky is still missing the point. It’s not about whether you use fold, map, and iter, or whether you can write closures easily. It’s not even really about pure functions vs. side-effects. To code in a functional style is a fundamentally different way of thinking about problems: instead of thinking about problems as nouns that are doing things, functional programming views a problem as a series of transformations to the world which results in an answer. This is why functional programming is considered “a different, novel, or exotic way to code”: it is a different, novel, and (as of yet) exotic way to code. It’s as different, novel, and exotic from OO as OO was from procedural. It’s a different way of thinking about the entire issue. You can check out this snippet of an IRC conversation from #ocaml for more on that.

The paragon of this way of programming is point-free programming, where you are quite literally building up a mega-function that describes how your program works, and then executing that one, single function when you run that program. If your language doesn’t lead people to re-discover point free programming at least in the small, then the language really isn’t taking function manipulation and functional language type conceptions seriously. And that’s the case with Scala: even Odersky admits that in Scala, “currying is more verbose and much less used than in other functional languages”. (Protip to Scala people: If one of the fundamental stunts of a style is pervasive in all the code but yours, you’re not in the same style of programming.)

What really gets me, though, is the claim that Scala is “an early example of a new breed of postfunctional languages”, because aside from the static typing, all the language features that Odersky trots out already exist in Perl. It’s hard to be a vanguard of a new breed of programming languages when there’s prior art from the 1980s.

Don’t believe me? The existence of a book on the topic unconvincing? Then let’s run the list of functional language features from Odersky.

• Functions as first class values: check.

  sub apply(&$) { # Take a function as an argument no problem $_[0]->($_[1]);
  }
   
  sub times2($) { # Create a function to take print $_[0]*2 . "\n";
  }
   
  apply(\&times2, 3);

• Convenient closure syntax: check

  my $x = 2;
  apply { print $_[0]*$x . "\n" } 3;
   
  my $times_x = sub($) { print $_[0]*$x . "\n";
  };
  $times_x->(3);

• List comprehensions: check. (See perlfunc on list data.)
• “Curried” function definitions and applications: check-ish.
  Okay, so calling this a “check” on Scala is a bit of a reach (cite, cite, cite, although note this—here is a more sympathetic run-down on Scala currying). Ignoring the foo(2,_:Int) syntax for a moment, we can implement basically the same style of “‘curried’ function definitions” such as Scala’s List#foldLeft.

  sub add { my $x = shift; return sub { $x + shift };
  }
  add(2)->(3); # Okay, so you do need an extra ->

  In the case of our apply function above (where we take a function as the first argument), it’s even easier.

  apply { print $_[0]*$x . "\n" } 8;

  Now, there isn’t really argument skipping (i.e.: foo(_:Int,3)) as a syntax feature, and there isn’t a built-in curry function, but if you want Scala’s Function.curried in perl, here it is:

  # This code released under Creative Commons 0 and WTFPL.
  sub curry(&@) { my($f,@args) = @_; return sub { $f->(@args, @_); };
  }
   
  sub add($$) { return $_[0] + $_[1];
  }
   
  my $curried = curry(\&add, 2); print $curried->(3) . "\n";

• Lazy evaluation: check. See Scalar::Defer for lazy val equivalents and Tie::LazyList for lazy seq equivalents. People generally use a double-return approach for generators (which I realize are different than lazy seqs and only kinda-sorta lazy).
• Pattern matching: check (okay, check-ish). See Switch. The decomposition isn’t there, which is the biggest weakness. But the general cumbersomeness and lack of real algebraic data types hamstrings the coolest parts of pattern matching anyway, so I’m calling it a draw. (This should be read as a generous and sympathetic ruling for Scala: Cedric Beust, for instance, rails against pattern matching/case classes and says “it’s hard for me to see case classes as anything but a failure“.)

In addition, perl’s got a few features in its favor for functional programming, like more flexible arguments, autovificiation, list/argument coercion, and dynamic symbol table mangling. Since perl also has OO capabilities, perl is at least as convincing a “post-functional language” as Scala. But there’s even more in common between the two than that.

Odersky’s “post-functional language” is really a subtype of Larry Wall’s “post-modern language”: it’s an attempt to create a language that is a grab-bag of multiple paradigms. And when you do that, you’re just begging for the complaints that you hear leveled against both perl and Scala: it’s too complicated, its syntax is weird, it’s too magical, people write in entirely distinct subsets of the language, etc. (cite, cite, cite) Now, those who master the language (or master their favorite subset of it) love the TIMTOWTDI aspect. But it also means that the language is left as a jack-of-all-trades, master of none. Yes, Scala and perl integrate a lot of powerful tools from functional languages—but learning OCaml still blew my mind, despite knowing perl for years. As I started off saying, Scala is not a functional programming language. It is a statically typed object oriented language with closures.

Now, there is a sense in which Odersky is really onto something. The world of programming is forever transformed with closures and list comprehensions as being mandatory for new high-level languages, even if they’re otherwise object oriented. And software developers are going to need how to work with them effectively if they want to read code moving forward. Yes, after 20+ years, the rest of the programming world finally caught up to one of perl’s fundamental insights.

Comments

• March 7, 2010, Barry Kelly wrote: Functional programming optimizes for different things than OO does. That means for different problems, you want to take different approaches, depending on your level of familiarity with the problem domain and the solution strategies. And thus for a language that supports both paradigms, different people will often end up using different subsets of the language. But I don't think this is necessarily a damning criticism. Our problems are bigger and more compositional. We need languages that have the semantic richness to support different solution strategies. An example. There's an isomorphism between OO polymorphism / dynamic dispatch, and algebraic data types / pattern matching and destructuring. OO lets you add subclasses as late as dynamically at runtime. ADT lets you add functions freely, and such functions are more composable that they're dealing with well-defined quantities. More generally, OO is about defining loosely bound protocols, while FP is more about flexibly defining transformations over tightly defined structures. But if you want to write a solution that's natural for one paradigm in the other, you end up re-inventing and poorly reinventing the other paradigm. In a comment somewhere else out in the ether, I made an argument that getting FP to work as well as a dynamically typed late bound language like Ruby for fast-iterating web solutions that join together multiple independently developed and fast-iterating modules, you'd end up writing what amounted to a dynamic language just to avoid the brittleness that comes from the over-specification that naturally falls out of static typing. I will agree that e.g. currying is useful, as it decomposes the concept of functions of arbitrary arity into what amounts to a linked list of unary functions; and this is particularly handy for programming in a point-free style. But point-free mostly helps (I would assert) on problems where what you need to do is create an algorithm, where little helpers like fold, map, memoize etc. are your bread and butter. But not all problems are like that, not by a long shot.
• March 7, 2010, Ittay wrote: "because aside from the static typing, all the language features that Odersky trots out already exist in Perl" That is a big 'aside'. You also forgot that Scala runs on the JVM and Perl not. Not to mention that I think it is common consensus that Perl code is very unreadable. So Scala is a language that combines some FP characteristics with OO, running on the JVM and with a readable (and scalable) syntax. That's a big win for me.
• March 7, 2010, Robert Fischer wrote: I will freely grant that there's a lot to be gained by having some flexibility, but languages that end up trying to play in multiple sandboxes end up either doing one part badly (e.g. OCaml and OO, Scala and functional). The answer seems to be using multiple languages for different contexts, not trying to create the One True Language with all the features of the world in it. That's why when I encounter a programming problem, I will generally reach for perl, Groovy, or OCaml/F#: which one I go for depends on the nature of the problem. But I don't long for a day when there's a language that has all the features of perl, Groovy, and OCaml, because they're fundamentally incompatible takes on the process of programming. The strongly opinionated natures of these languages is precisely what makes them so good. Scala's lack of strong opinions is what's frustrating me. Well, that and having to explain to Java devs over and over again at No Fluff events that learning Scala does not mean learning to program in a functional style. I mean, you can code Scala in a functional style, but I can code Java in a functional style. And yes, it's a bit easier to code in a functional style in Scala, but learning Scala is no more learning functional programming than learning perl is learning functional programming.
• March 7, 2010, Robert Fischer wrote: The "readable (and scalable) syntax" bit is a big assumption—at least to my ear, there's as much concern about the readability of Scala code as there is about perl code. And how much Scala's static typing really helps vs. how much it gets in the way is a matter of contention. But both of those are an aside. The purpose of this post was not to say that people should stop coding Scala and start coding perl: if you're still futzing around in Java-land, Scala (like Groovy and JRuby) is a productivity huge win that can leverage your existing infrastructure. Groovy's probably the semantically closest, JRuby's the conceptually cleanest, and Scala keeps the fetishized complexity and the BDSM aspects of Java. No, this wasn't a call to go back to perl. The purpose of this post is to show that the idea of merging closures with imperative isn't a new idea, and that Scala is no more a functional language than those languages (like perl) which merged the concepts beforehand.
• March 7, 2010, Richard Vowles wrote: I've been saying this for 6 months at least now - Scala and Perl have way more in common than Scala has in common with any other language. Including a tendency towards unreadability.
• March 7, 2010, Marc wrote: "It’s as different, novel, and exotic from OO as OO was from imperative." OO is as imperative, if not more so, than C. I assume you meant procedural? Also, there is nothing new in programming languages, period. Everything you see in "new" languages was figured out at least 20 years ago. So, all language designers are just mixing and matching as they see fit. So it's unfair to single out Scala in this regard. While you're at it, go after Groovy, Clojure, Boo, F#, etc.
• March 7, 2010, Nick Bauman wrote: I find the description "combines some FP characteristics with OO" problematic when discussing hybrid Functional / OO languages. The more I use one of my favorites of these languages, Python, the more I find the OO aspects receding into the background and the functional ones coming to the fore. It seems to me that Functional is OO "done right" where when I do use OO in them, it's not as heavyweight and seems more natural. I think the key is that in most traditional OO languages the concept of what a type is tends to be much narrower. Can you confirm this in your experience?
• March 7, 2010, Thomas Heywood wrote: Robert, I admire your ability to "professionally blog." You delivered no useful content, but you did it with inspiring attention to page hits and comment controversy. Yes, Scala is a multiparadigm language. Yes, Perl is one too. Yes, they both share features from Common Lisp. That's not really worth my or your time. But you cleverly used the confrontational tone to make the blatant non-issue into a controversial issue. You cleverly used Blogosphere's beloved "Scala" and "Perl" tags in a single article. You nagged about a minor syntax issue like it was life-or-death as only a true master-of-functional-programming-who-blogs-more-than-he-actually-programs can. Bob, you're my hero - a true Dilbert's Wally in flesh. A guy without any real code to show - but who "realigns the tone of existing projects" and whose "technical leadership focuses on pragmatic communication" like there's no tomorrow. Wally - sorry, Robert - do you think you could write some posts on how to professionally sell bull and call it "technical accomplishments?" Many of us CompSci majors would like to get in on this money for nothing scheme. Unless, of course, you're not comfortable writing about things you actually do know a little about.
• March 7, 2010, Nilanjan Raychaudhuri wrote: Well I believe it is possible to write unreadable code in any language and I am not sure blaming any programming language for that is a good idea.
• March 7, 2010, Daniel Spiewak wrote: Throwing out static typing fundamentally changes what Scala has to offer. In a sense, I agree with you that if we completely threw out Scala's type system (and all that it entails), then we would end up with a language which improves very little (if at all) over languages like Perl or Ruby. However, that first step is a doozy and not really fair. Consider a snippet like this:

  "123".to[Int] // => 123
  ".+".to[Regexp] // => /.+/
  true.to[Regexp] // compile error!

  There's a lot of magic going on here, all of which is made possible by the static type system. In fact, while this sort of thing is *possible* in a dynamic language with open classes (like Ruby), the implementation will be ugly, unscoped, brittle and difficult to extend. In Scala, the implementation looks something like this:

  trait Converter[-A, +B] { def apply(a: A): B
  } implicit object StringToInt extends Converter[String, Int] { def apply(str: String) = str.toInt
  } implicit object StringToRegexp extends Converter[String, Regexp] { def apply(str: String) = Regexp(str)
  } implicit def toSyntax[A](a: A)= new { def to[B](implicit conv: Converter[A, B]) = conv(a)
  }

  This is taking advantage of a feature which Scala inherits from Haskell called typeclasses. This is an feature which is inherently tied to static typing and is often very difficult to emulate in dynamic, so-called "more flexible" languages. The real magic of this is two-fold. First, it's scoped, and so we actually have to import these members before they start messing with our objects. It's hard to over-state how important this is to keeping such extensions from devolving into intractable monkey-patching. Second, it's extremely extensible. What if we want to add another conversion from Foo to Bar? Just define it ourselves as an implicit value in scope! Or, if we don't want the implicit syntax, we can always specify the conversion explicitly:

  // off in *our* code, separate from the converter lib
  implicit object StringToBool extends Converter[String, Boolean] { def apply(str: String) = str.toBoolean
  } "true".to[Boolean] // => true
  "false".to(StringToBool) // => false

  I'm not even going to try to imagine how to do this sort of thing in Perl. Even in Ruby, which usually makes monkey-patching quite simple, the implementation still ends up extremely nasty:

  class Object def to(target) case target when :fixnum if self.class == String self.to_i else raise "Unknown conversion: String -> :fixnum" end # ... end end
  end

  ...and, you get the picture. You could probably do this in a nicer way, but it would still end up being fragile, unscoped and virtually impossible to extend. This is what you're throwing away when you discard the type system, not some unreliable correctness "guarantee". And if I can throw together an example like this in a blog comment in just a few lines of Scala, imagine what can be done in a non-trivial system with real requirements. The type system is an intrinsic part of Scala's power and expresivity. Remove it and you're not even looking at the same language.
• March 7, 2010, Daniel Spiewak wrote: While currying and point-free aren't as syntactically clean as they could be in Scala, they are certainly still used (I use those features all the time). Does this make Scala less "functional" than Haskell or ML? Yeah, probably (I've made that argument in the past). Does it mean that Scala is no more functional than Perl? Absolutely not. Theoretically speaking, currying is possible in any language which supports closures (alblue, if you're reading, yes I do mean "closure" and not "lambda"). You can even define a curry function in C++, though the obfuscated template mess it entails makes it nigh unreadable. Lambdas ("closures" if you will) and function values do not a functional language make. C has function values, and I don't think anyone who understands the debate will argue that it is "functional". Ruby has function values and lambdas, but they are fundamentally different constructs and produce different values (just like in Perl). Again, not a functional language (though you can use many patterns which find their roots in FP, like map and fold). My point is this: you can't go out of your way to link FP to point-free and lambda calculus and then turn right around and claim that Perl and Scala are in the same league. Scala's unified functions and function values alone should be sufficient to overturn that one.
• March 7, 2010, Robert Fischer wrote: Yes, I meant procedural.
• March 7, 2010, Martin wrote: You really can't claim "perl has had _____ since the 80's" and then point to books about Perl 5. Perl 25 years ago is *very* different than it is now. Your point that Scala and perl are similar is well taken, but I'm not totally clear why this is bad for Scala or Perl. The way Perl 6 is going towards a virtual machine, I wouldn't be surprised to see Perl and Scala and Java running together on a JVM sometime soon.
• March 7, 2010, Robert Fischer wrote: I'm not saying it's bad for either Scala or perl to be similar to each other — on the point of similarity, I'm simply saying that in the ways Odersky argues Scala is "functional", perl is just as functional, and so I'm still unconvinced that Scala deserves to be called a "functional language". You don't hear people listing off the functional languages like: "Haskell, OCaml, perl, Lisp...". So to hear people say "Haskell, OCaml, Scala, Clojure" just gets my goat. I'm also saying that on the point of similarity, it shouldn't be a surprise that we're getting similar unreadability/complexity complaints about Scala as we got about perl, since they are so similar. I heard something about perl getting onto the JVM for AppEngine, but can't find additional info on it.
• March 7, 2010, Robert Fischer wrote: In what ways do Scala's unififed functions and function values outperform perl?
• March 7, 2010, Robert Fischer wrote: Note the point I'm arguing: I'm arguing that Scala is no more a functional language than perl is. Functional languages can be dynamic (e.g. Clojure), so static vs. dynamic typing is irrelevant to the point I'm trying to make.
• March 7, 2010, Robert Fischer wrote: You've earned a special spot in my heart: this is my new favorite flame of all time. I mean, the fact that you say I don't have "any real code to show" just goes to show that you're not paying attention, because you've somehow managed to miss my long trail of open source projects. And you kinda missed the boat on the entire fundamental question: "Is Scala a functional programming language?" But still, the Dilbert references, the accidentally-on-purpose "slip of the keyboard", and the general long-suffering tone is really nicely done. A+
• March 7, 2010, Robert Fischer wrote: Oh, and you're fair on the dating thing. It's hyperbolic to conflate Perl 1 with Perl 5, although I honestly have no idea how far back I can go in perl and still have my code samples work. If you find out, let me know.
• March 7, 2010, Robert Fischer wrote: Brian Hurt, the co-blogger here on EnfranchisedMind, is theoretically working on a post about how OO is fundamentally broken. I'll leave that to him. My experience has been that many of the Gang of Four design patterns just aren't relevant anymore, and inheritance is something I'm extremely reluctant to use and much more aware of its difficulties. I'm not sure exactly what it would mean to say that "the concept of what a type is tends to be much narrower", but I certainly have a much thinner type hierarchies in the world of functional programming.
• March 7, 2010, Robert Fischer wrote: Oh, not to mention the cite of enterprise-ese from SmokejumperIT.com. I mean, that's some real attention to detail for the zingers. In all seriousness, this is my favorite flame of me of all time. Thanks!
• March 7, 2010, Dave wrote: FWIW, I really dug this post. As to the topic at hand, I'm not even sure what point there is to saying Scala is or isn't "functional". The only real purpose seems to be to make it easy to show how awesome Haskell is by point out Scala's perceived shortcomings as a functional language. For my money, Scala is simply "Java: The Next Generation". A general-purpose, statically-typed, object-oriented language that has broad applicability to many problem domains. Arguing "what is functional" might be a fun exercise, but it's not really going to lead us anywhere; look at most n-tier Java applications: to say they are object oriented would be a vast overstatement. Java is object-oriented in that it's main means of organizing code are around classes and objects (which is exactly the same as in Scala), but most systems still end up using OO only when it helps, and throwing it for imperative/procedural stuff when it makes sense to do so. I would be willing to bet that Scala will be much the same way; the code inside our classes might be much more functional (simply because its' simply worth-it to do in Scala, whereas it's not terribly so in Java), but I don't see it ushering in some functional utopia (LISP has had, oh, I don't know, the entire history of the field of computer science to usher this in and largely failed [no offense to LISP]). Scala is an escape route from Java's cruftiness without having to bend your mind or prejudices too much. Much as Java was to C++.
• March 7, 2010, Tracy Harms wrote: The most interesting sentence in your post, to me, was this: If your language doesn’t lead people to re-discover point free programming at least in the small, then the language really isn’t taking function manipulation and functional language type conceptions seriously. This has me wondering, though, about your agreement with Brian that the key quality is reliance on lambda calculus as the model of computation. Not that I'm convinced we need a "key quality." It just looks to me that the range of languages you have in mind would not be identified through this criterion.
• March 8, 2010, joe poki wrote: what's up with these groovy people nowadays? groovy++ is 10 trillion times faster than scala actors, now this piece. your rant could be summarized like I Do Not Like Scala. Look, I get that. But I must say it's funny that you use this patronizing tone with Odersky as he was a n00b who has no idea about languages and then once you get a trollish comment, you go on and post it everywhere: OMG somebody was responding to my trollish post in a trollish manner, what a surprise? as for your post: just because a language can be used in a certain way (or certain language toolsets can be simulated in that language) that does not mean that the users will find that specific style idiomatic. What matters is the style you will meet the most while reading or writing code in that specific language (hack, even java has various functional libraries: lambdaj, functionaljava etc. but how many people would consider code written with these library java-like?). When people say that scala is "kinda functional", what they really mean is that the language is nudging you into that direction (preference for pattern matching, option type, final variables, map, foreach,closures, partial functions etc.). And this nudging towards the functional style is what makes scala "postfunctional". (other features responsible for the "scala feel" but not directly related to the functional thing: static typing w/ type inference, a powerful type system, xml literals, implicit converions, case classes, mixins + various actor libs etc.)
• March 8, 2010, Robert Fischer wrote: My issue isn't with people saying Scala is "kinda functional". It's saying it's a "functional language". I've heard people say they want to learn Scala b/c then they'll know functional programming, and that's just wrong. Worse, I've heard people write off functional programming because they didn't like Scala. This means that the misconception that Scala is a functional language is hurting real functional language adoption and hindering people's advancement as software developers by making them think they know functional when they don't. So I'm trying to reinforce the barrier between real functional languages and languages with closures. I'm not particularly knocking Odersky here: what he's done with Scala is pretty impressive. But he's clinging to a title that Scala doesn't deserve. While I appreciate the move with "postfunctional", I'd be a big fan if it also meant he'd give up "functional" — but he's just taking both titles on. In so doing, he's diluting what we mean by a "functional language", and that's just not helpful to anyone involved. The evidence does seem to be that Odersky doesn't really know functional languages. Or, if he does know them, he knows them academically or intellectually, and doesn't really grok them. But you can't be an expert on everything, and I get that. This whole conversation is really kind of sad. There's such an extreme dichotomy here: the assumption seems to be that if you criticize a language, you must hate it. I don't hate Scala: Scala is what Java should be. It's finally a competitor to C# on the JVM — the advanced capabilities of C# have been downright embarrassing for a long time, and Scala evens the playing field. I'm just criticizing the way we talk about this language, and providing counter-evidence to some of the claims made about it.
• March 8, 2010, Robert Fischer wrote: Which languages do you see as missing one of the two qualities?
• March 8, 2010, Robert Fischer wrote: As I will say below, my issue is mainly one of vocabulary and pedagogy, because I'm frustrated with the murky waters that such marketing-speak has created.
• March 8, 2010, Robert Fischer wrote: PS: From someone who has been there (remember: I'm a static typing fan, too): If you want to debate static typing usefulness vs. dynamic typing usefulness, don't argue that it's hard to implement type safety in a dynamically typed language. It doesn't win you any points with the dynamic language crowd, because they've already accepted they don't need it.
• March 8, 2010, Peter Lewerin wrote: Daedalus and his son Icarus were imprisoned by King Minos, and made themselves wings out of wax and feathers to escape. Before they took off from the island, Daedalus warned his son not to fly too close to the sun. The sensation of flying made Icarus forget the warning and he soared higher and higher. Eventually the wax melted and he fell into the sea. A programmer can escape the prison of procedural programming using functional programming, but going for the purest form of FP isn't really a sustainable goal. Most likely, these wonktional programmers will melt their wings by straying too close to the realm of mathematics and plunge to their deaths.
• March 8, 2010, Thomas Heywood wrote: And you kinda missed the boat on the entire fundamental question: “Is Scala a functional programming language?" Scala is a multiparadigm language that supports procedural, functional and object-oriented programming. We've had that since 1950s and Lisp. Now, what was the fundamental question again?
• March 8, 2010, Thomas Heywood wrote: (Mea culpa: we've only really had it supplied with the language since 70s, when Lisp machines took off. Still, a blatant fact is blatant, no matter how much you blog it into shape.)
• March 8, 2010, Vassil Dichev wrote: Robert, if you like Perl, that's nothing to be ashamed of :-) Seriously, I think your article is thought-provoking, if a little controversial. Still, there are several things I disagree with. First of all, I have interpreted Odersky's article in the spirit of: "OK, I don't insist on calling Scala a functional language". One important piece you didn't mention was immutable data structures. Perl doesn't scale very favorably here. Was this on purpose? I find it important not just what a language *can* do, but also what it *cannot* (easily) do. Mutation and scoping are some of the things Scala restricts rather successfully. I also don't find how the fact that Perl has flexible arguments, autovivificiation, list/argument coercion, and dynamic symbol table mangling makes it more functional. Does Haskell have these? Do you consider Haskell functional? Then I also don't see many people argue that Java is object-oriented. By the same reasoning, it's not- it certainly has types that are not objects, and you can certainly write in a procedural style in Java (unfortunately some people really do). Also, I don't see you arguing that OCaml is not object-oriented, because it's also considered a multi-paradigm language. I see that in one of your comments you agree that OO is poor in OCaml, which is fair enough, but I think some OCaml users wouldn't agree with you on this count either. I most emphatically agree that if you "know" how to write working Scala code, this doesn't mean you know all about functional programming. But I disagree that this harms "real" (read purer) functional languages any more than Java harms "real" object-oriented languages (whatever they are). In the end, I'm also fine with the definition of "statically typed object oriented language with closures".
• March 8, 2010, Vassil Dichev wrote: I've just read some of the blog posts you've linked to, and it's quite telling that two of them complain that Scala is too functional, while you complain that it's not functional enough. I think this is the problem with Scala: you can't appease everybody without any mental effort on their part to get used to the new style of programming. And I think the same can be said of OCaml. I also think OCaml would be a wonderful language to learn (and I might try it soon).
• March 8, 2010, Paul Kaletta wrote: All of your examples should work with any version of Perl 5, which came out in 1994.
• March 8, 2010, Martin Odersky wrote: Robert: Your blog and even more so your comments make it sound as if I have no clue about what functional programming is. So, it must be because of a series of freak accidents that I organized ICFP, am a member of the IFIP working group on functional programming and am a member of the editorial board of the Journal of Functional Programming. And I even published quite a bit on lambda calculus, too :-). Seriously, I stand completely by my article on postfunctional programming. I believe your post here has added nothing of substance to the discussion. http://www.scala-lang.org/node/4960
• March 8, 2010, joe poki wrote: if you cite every single well-known post that dismisses scala one way or another and you go on and say "it’s too complicated, its syntax is weird, it’s too magical, people write in entirely distinct subsets of the language", then people will rightly assume you do not like scala, I do not get it why you are surprised (notice, I did not say you hate it, just that you dislike it). see, the patronizing stuff again "Odersky doesn’t really know functional languages". As I tried to point out: what matters is how a language is used and what's considered idiomatic and in that sense scala definitely has some functional feel to it (and no, it's not just about closures or foreach, but preference for pattern matching, option types, final variables, map, foreach,closures, partial functions etc. etc.).
• March 8, 2010, Tracy Harms wrote: To the best of my knowledge, Factor, Joy, FL, and J are languages that favor anonymous arguments (and/or emphasize composition of functions) while lacking reliance on lambda expressions.
• March 8, 2010, Robert Fischer wrote: If you're up for not calling Scala a "functional programming language", tell Odersky.
• March 8, 2010, Robert Fischer wrote: It's not patronizing to say someone doesn't know something, or to disagree with the way someone is using language. That's just not what patronizing speech is. It'd be patronizing if I started explaining to Odersky what closures are and how they're supposed to work, or if I sent you off to look at this link. That would be patronizing.
• March 8, 2010, Thomas Heywood wrote: Your blog and even more so your comments make it sound as if I have no clue about what functional programming is. Martin, hate to break it to you, but all you ever did was bring effective functional programming to a mainstream platform for the benefit of industry and academia. Call me back when you write a wrapper library or two, learn the persistence API of a web framework, or realign the tone of existing projects.
• March 8, 2010, Robert Fischer wrote: That's cool that you're so involved. Ultimately, though, our problem is boiling down to one of language. If Scala and perl are functional languages, how do we differentiate them from languages like Haskell, OCaml, and Lisp, where the fundamental way of tackling problems is different? I'd like to use the term "functional programming language" for the latter, and I'm more than happy to stick to "postfunctional programming language" or "multi-paradigm programming language" for the former. But that means we stop calling Scala a functional programming language, as most of the commenters here on the post already have. Deal?
• March 8, 2010, Robert Fischer wrote: Yeah, that's the general issue with these multiparadigm languages like Scala and perl. If OCaml narrowly avoided that fate (and I think it has), it was by completely subjugating the OO aspect to the functional aspect, and not really using the OO aspect much.
• March 8, 2010, joe poki wrote: If you think saying somebody does not know squat about a subject while that person is supposed to be some sort of an expert on the topic is not condescending, then I do not know what is. By the way, it seems I was not alone, see Odersky's comment above.
• March 8, 2010, Robert Fischer wrote: The immutability is a good point, and it's an area where Scala certainly pulls ahead. Actually, the Ruby (and, to a lesser extent, C#) people regularly say that Java isn't really OO, because a lot of things in Java weren't either objects or messages. I think Ruby inherited that criticism from the old SmallTalk people. And I'll happily argue that OCaml isn't object oriented: while it has objects, it's certainly not oriented towards them. What harms functional programming languages is when Java people learn Scala and then think they know functional programming languages. Or, worse, when they decide they don't like Scala and then conclude they don't like functional programming. This happens a lot because Scala was lauded as a functional programming language on the JVM which was comfortable for Java people. It's the murkiness in the water that's the issue.
• March 8, 2010, Daniel Spiewak wrote: In Perl, functions and lambdas are fundamentally different. If nothing else, this can be seen by the different invocation syntax. You can convert between functions and lambdas (just like in Ruby and Groovy), but they are very different constructs as far as the language is concerned. To me, one of the essentials of a functional language is that functions are merely lambdas which have been assigned to a named value. OCaml and Haskell encourage this notion with their syntax for function declaration (in fact, Haskell's function decl syntax is exactly the same as its value decl syntax). Perl doesn't even come close to meeting this qualification. Scala does substantially better, though I'll admit that it does have its gotchas (methods are sometimes tricky).
• March 8, 2010, Robert Fischer wrote: While that's a bit hardcore of a comparison (and somewhat damning on people's ability to learn), I definitely agree with your point. And Scala's great in this regard—as I said a number of times, I think Scala is where Java should be if Java's growth hadn't been stunted. If you look at how C# has evolved, Scala's basically right in line with that kind of development. And so a Java developer learning Scala in order to push their way of thinking about types and functions is a great move.
• March 8, 2010, Robert Fischer wrote: True — I'm oversimplifying the world into imperative and functional. There are other paradigms out there (e.g. Factor's stack-based approach) which also can lead to point free programming.
• March 8, 2010, Robert Fischer wrote: Do they work before that? (Or, rather, would they work before that if I dumped the prototypes?)
• March 8, 2010, Robert Fischer wrote: True: there are conversions that need to be made between functions and lambdas in perl. This suggests for Scala (as a multiparadigm language) having a somewhat stronger functional aspect than perl, but it still doesn't make it a functional programming language a la OCaml and Haskell.
• March 8, 2010, Alicia wrote: Wouldn't it be awesome if you really could make money writing b.s.? People won't even donate a dollar towards valuable open source projects...the suggestion that you could make money for being good at b.s. is hilarious.
• March 8, 2010, Robert Fischer wrote: Amen.
• March 8, 2010, Robert Fischer wrote: True enough. I'm just trying to wrap my head around how Odersky can really grok functional programming and then assert that it's simply bolt-on features of a language. The difference between Haskell and Java is not simply one of syntactic sugar. Claiming he's mistaken about how FP works is actually a generous take—claiming he's clinging to the FP title for Scala for some ulterior reason when he should know better is much less generous. I was trying to give him the benefit of the doubt.
• March 8, 2010, James Iry wrote: Robert Fischer Finally Admits That Scala Is Functional (http://james-iry.blogspot.com/2010/03/robert-fischer-finally-admits-that.html)
• March 8, 2010, Martin Odersky wrote: Robert, please, read my post again: http://www.scala-lang.org/node/4960 Nowhere did I define functional programming exclusively by a list of features. Instead I had as my central definition: "A functional language makes programming centered around functions easy and natural." A good test of this would be: is it common to write functional programs in the language in question? It's usually far easier to tell whether a program is functional than whether a programming language is functional. By that standard Scala is clearly functional; many of the classes and modules written in it are purely functional and every cool idea from Haskell seems to be implemented sooner or later in Scala (not that we don't invent some cool ideas of our own as well). I don't know Perl well enough to be able to tell whether there are many functional programs written in it, and whether the expression of such programs is easy and natural. Thanks anyway for giving me the benefit of the doubt that it's just misguidedness and not evilness that makes me insist on my ways.
• March 8, 2010, bse wrote: Well, at least you admit you're writing about "religious issues", where arguments are replaced by belief. So by definition I can't argue with your expression of belief. However I don't have to agree with you.
• March 8, 2010, Robert Fischer wrote: How do you account for this quote?

  The first thing we cared about was to have as clean an integration of functional and object-oriented programming as possible. We wanted to have first-class functions in there, function literals, closures. We also wanted to have the other attributes of functional programming, such as types, generics, pattern matching. And we wanted to integrate the functional and object-oriented parts in a cleaner way than what we were able to achieve before with the Pizza language. That was something we deeply cared about from the start. Later on, we discovered that this was actually very easy, because functional languages have a fixed set of features. They had been well researched and well proven, so the question was only how to best integrate that in object-oriented programming.

  The Goals of Scala's Design
• March 8, 2010, Robert Fischer wrote: Ugh. I need to turn down the maximum comment nesting. It's getting out of control, and the box is getting REALLY skinny.
• March 8, 2010, Martin Odersky wrote: Classical fallacy: I said: functional languages have a well-identified, fixed set of features. So being a functional language implies having that fixed set of features (or rather, some part of it because I was talking about the union of all functional languages). You think I said: having that fixed set of features implies being a functional language. But I never said that. Besides, that quote is not from the article I referred to. When I said "nowhere" I meant nowhere in that article, and I think that was clear from the context.
• March 8, 2010, Robert Fischer wrote: Okay, sure I'm taking your quotes from both your post on post-functional languages and another one on the design decisions behind Scala. Sorry if I overgeneralized your "nowhere". I read that quote as saying, "I went out to implement a functional language. This was easy, because it turns out implementing a functional language means implementing a fixed set of features." If that's not what you meant, what did you mean? I'm uncomfortable with your broad definition because it leads to confusion. Scala's extreme flexibility is exactly why I'm so resistant to labeling it a functional programming language — people learn the subset of Scala closest to Java and then think they know functional programming. I'm really, really tired of explaining to people that functional programming is more than list comprehensions. BTW, I want to say it straight to you, since I think things can get lost and there are some misconceptions floating around. I really do appreciate all you've done on the JVM languages—especially proving to folks who only know Java that there's more to static typing than Java's conception of it. And for all my pedantic resistance to calling Scala a functional language, it is one of my go-to languages for heavy lifting on the JVM. (My own experiments in programming language development have Scala cores.)
• March 8, 2010, Martin Odersky wrote: Adding functional language constructs was relatively easy, because they are well understood. Making Scala an organic whole was hard. We can disagree whether this succeeded or not. The other question is, should a functional language force you to program in a functional style? Scala clearly does not, that's why I prefer to call it postfunctional. I understand your uneasiness that some people might take this as an excuse never to get into real FP at all. Where we might differ is that I believe that in the long run Scala programmers will recognize the benefits of the functional programming style even if it is not forced down on them.
• March 8, 2010, Robert Fischer wrote: I'd say that if a language doesn't force you to program in a functional style (or at least strongly encourage you that way), I'd consider it suspect as being a functional programming language at all. But I'm running on the stricter definition, as we've established. I hope that your sense of the future is true, and that the functional programming style becomes more widely adopted by way of Scala. And maybe being on the inside of Scala adoption, you're seeing that. Sitting here on the outside, though, I've seen the contrary, which is why I'm such a stick in the mud. Well, that and I'm preternaturally disposed towards being a stick in the mud.
• March 8, 2010, Brian Hurt wrote: Let me turn that question around: should object oriented languages impose an object oriented approach? There is a value to learning, and really using, a paradigm. Is that limit you just hit a limit of the paradigm, or just a limit of your understanding of the paradigm? Until you expand your understanding, you don't really know- and the siren call of multi-paradigm languages is that they allow you to not expand your understanding by ditching the new paradigm early and often. There is another comparison (one you'll like even less) to another multi-paradigm language- C++. The reason C++ "won" was because it was multi-paradigm (procedural and OO), allowing people who didn't really understand OO to program in it. The reason C++ "lost" was because it was a multi-paradigm language, forcing people to deal with the old paradigm whether it was valuable to or not. The history of popular programming languages 1992-2007 was basically one of going ever less multi-paradigm and ever more pure OO. In other words, is Scala "post-functional" in the same sense that C++ is "post-Object-Oriented"?
• March 8, 2010, Gabriel C. wrote: If we're talking about how to LEARN functional programming, I totally agree that Scala is not the best choice (neither OCaml... the O part is there, lurking in the back...). I would recommend a pure functional lazily evaluated language (i.e. Haskell). My experience was the opposite of Robert's fears: I started learning FP with Scala (limiting myself to the functional subset of it) and then moved to learn Haskell (and I try to pick up as much as OCaml as I can on the way), but I guess I'm just weird: I've learned OO in C++ :) But somewhat I agree in the comparison: C++ is to C as Scala is to Java
• March 9, 2010, joe poki wrote: Robert, I was actually talking about this kind of strong encouragement in my previous post when I said that scala was nudging you towards FP.
• March 9, 2010, Martin Odersky wrote: If Scala had taken all of Java and then added FP to it, a comparison with C++ might have merit. But as it stands Scala has thrown out or generalized a lot of Java's more crufty features. Where C++ is clearly much more complicated than C, the situation between Scala and Java is more balanced. Scala's syntax is a probably a bit simpler than Java's but its types are richer and more powerful. So you can say that Scala shares C++'s motivation -- take a well established programming style and let it branch out into a new paradigm. But it shares neither its approach (to be an almost strict superset of a base language) nor its overwhelming complexity.
• March 9, 2010, Ismael Juma wrote: Hey Robert, I found the following in your LinkedIn page: "Robert Fischer is a multi-language open source developer currently specializing in Groovy in Grails. In the past, his specialties have been in Perl, Java, Ruby, and OCaml. In the future, his specialty will probably be F# or (preferably) a functional JVM language like Scala or Clojure." What, did you just call Scala a functional language?! Ismael
• March 9, 2010, Robert Fischer wrote: Busted! Really, that's exactly what I've been discussing with Martin O.—if you know functional programming already, you can do functional programming in Scala. And if you're on the JVM, like static typing, and want to do functional programming, you really don't have a better option. BTW, in Scala, I'm still struggling a bit to do functional programming in a way that's particularly friendly to Scala's approach — algebraic data types and function manipulation a la curry/flip are standard stunts for me, and I feel like I'm swimming upstream when I try those in Scala. I really should look more into some open source projects—anyone have one in Scala that's particularly functional? (In any case, I should update that since I'm now an .Net/F# developer.)
• March 9, 2010, joe poki wrote: that is hilarious!
• March 9, 2010, peter pnin wrote: Robert, are you under the employ of Flying Frog Consultancy, perchance? There is a certain Harropity, if you will, to your blog post.
• March 9, 2010, Robert Fischer wrote: Ouch. Of all the nasty things people have said about me because of this post (up to and including calling me a "douchebag"), that's the first one to hit home.
• March 10, 2010, joe poki wrote: Robert, I am sorry people called you names but I am not sure what you expected. You published a deliberately controversial post (or may I say a very trollish one?) where you wrote off a whole language using standard (and frankly boooring) points like “it’s too complicated, its syntax is weird, it’s too magical, people write in entirely distinct subsets of the language” for one particular issue you have with it and on top of this, you called the given language's creator a numerous times pretty much clueless when it comes to FP.
• March 10, 2010, Paul Chiusano wrote: Functional programming just means programming with pure functions (no observable side effects). The alternative is imperative programming - programming with observable side effects. OO is orthogonal - you can program using pure functions operating over objects if you like, or your functions operating over objects can have side effects. Of course, most real programs have some mixture of functional and imperative styles. There are some language features that make programming in a purely functional style more convenient, sure, but basically, with some discipline you can program in this style in any language, Scala (and Perl!) included. My personal opinion is that Scala makes FP pretty convenient - your argument seems to be merely that it's not as convenient as it could be in some instances and therefore doesn't get your official "functional language" stamp. I guess that is fine, although the distinctions you are making seem pretty arbitrary to me.
• March 11, 2010, Brian Hurt wrote: I don't see what the big deal with Object Oriented languages is. I mean, you can do Object Oriented programming in languages like C and Pascal- yeah, it's not quite as convenient as doing it in a language like Java or Ruby, but there's no real reason to switch, is there?
• March 11, 2010, Gabriel C. wrote: But maybe the only true "popular" OO language is SmallTalk, it could be argued that Java is not really OO, having primitive types and static methods, and many don't really in a fully OO way...
• March 11, 2010, Gabriel C. wrote: Perhaps the difference is that C and Pascal don't have "first class" objects and Java and Ruby have.
• March 11, 2010, Paul Kaletta wrote: Well, I've never used Perl 4, but afaik it didn't have a reference variable type and it didn't have lexical scoping, so your examples would not have worked in the current form. Glancing at the Perl 4 manpages it seems that Perl 4 could already pass around code blocks, however it seems to me that you would have had to capture variables manually, which means that closures would have been difficult to do.
• March 11, 2010, Brian Hurt wrote: They have structures, and function pointers, and you can put the function pointers in the structures giving you virtual methods. Inheritance is a little tricky, but no one is using that anyways these days, so no problem. Yeah, it's a little bit klunky, but all the important stuff is there, so why do we need to leave C and Pascal again? If you haven't figured it out, I'm making one of my little points by adopting this position.

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (203.215.201.194) )

I just released the DynamicDomainProperties plugin for Grails, which allows domain classes to have dynamic properties. It’s pretty nifty, if I do say so myself.

Based on my frustration with the Grails plugin culture because of differing cultural assumptions about open source works, and based on my lack of appreciation for the promises of indirect compensation offered to me as an open source developer, I’ve decided to release it under the GNU-Affero GPL 3.0, which is like the GPL but (among other things) requires the source to be available to the users of a web app that use this plugin.

I’m open to the idea of releasing it under the more “biz-friendly” BSD, but then I’m going to want a different form of compensation. So I’m putting up a bounty via Pledgie: if I was building this plugin as a consultant, it would have cost a client about $2500. So, if the commercial community contributes $2500, I’ll do a BSD release. Anyone who donates $250 or more gets a single-domain usage license right away.

For more information on the plugin, see the plugin page on grails.org. (I’ve had enough difficulty with GitHub’s wiki freaking out on me and would rather avoid that.)

Comments

• February 22, 2010, Marc Palmer wrote: I'm interested to see how this pans out Robert - good luck!
• March 4, 2010, Jeremy Flowers wrote: Way to go. This looks like the way forward to me. In the Microsoft' arena GUI component vendors started to sell there wares this way too, like Infragistics. I always thought plug-ins would have to go the same way to get support for your plug-ins. You have to put food on the table somehow..
• March 5, 2010, Does Grails have a problem? « The MetaSieve Blog wrote: [...] I have time to, I can completely comprehend Robert’s point of view and his desire to get a different kind of compensation for his open source work. In fact, there should be plenty of space for both approaches (and [...]

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (98.136.63.35) )

Comments

• February 5, 2010, Hamlet D'Arcy wrote: In the past my problem has been that my organization would not let me rely on a personal or nightly build of a project. So when you need a workaround by the end of the week, waiting 2 months for the next release candidate of an OS project is not an option. ... now go release JConch 1.2.
• February 5, 2010, Dave wrote: I've encountered this a few times. Sometimes, I'm trying to get things done, so I hack around the problem and then there's not always time to go back and learn a new codebase to submit a patch. Further, having submitted a few patches to a maven plugin (a piece of software I cannot stand, but submitted anyway), it's been months since I uploaded a freakin' zip file to their JIRA and no movement. I also think compared to, say, Ruby, a lot of Java packages are very large and complex. If I find a bug in Spring or Hibernate, my chances of even navigating the codebase to find out where the problem is are nil. And Spring is actually pretty clean code; I mucked through JSCh and the maven release plugin and was HORRIFIED at how awful the code was. And then there's the myriad build systems and spotty tests; One of the Castor devs asked me write a test or he wouldn't accept my patch; my patch was to make his code actually compile. And their tests on trunk (and release tag) didn't pass. Meanwhile, on github, I've submitted patches to things that I wrote and tested in less time than it took to find the svn link on Apache's site. I've also incorporated others changes to my code (that they didn't even submit). Granted these things are not as complex as Spring, but, well, maybe that's a good thing?
• February 5, 2010, Dave wrote: @Hamlet D'Arcy in your case, I have imported the released version I'm using into our company's svn repo and made changes I needed against that, then deployed THAT to our maven repo. It was painful, but worked.
• February 5, 2010, Robert Fischer wrote: Argh. I've totally forgotten about JConch 1.2. Just released it. It's now a Gradle-based build (I wanted to OSGi-ify the project and have a Maven repo).
• February 5, 2010, Peter Niederwieser wrote: From my experience, most developers aren't even willing to take the time to report a bug they encounter...
• February 6, 2010, Aristotle Pagaltzis wrote: I agree with Peter – I actually tweeted the same observation just a few weeks ago. I got one response to that that didn’t satisfy me either. It’s not limited to Java either.
• February 6, 2010, James Carr wrote: Yeah... this has often surprised me as well. I've witnessed a team try to use some open source solution and give up because of a minor bug and go to a much more difficult to use proprietary solution (which amusingly had more critical bugs). I was also surprised when at my current client site they actually asked me spend a week working on fixing a bug in an open source project and submit a patch... that's definitely the best attitude to have. :)
• March 8, 2010, Michael Chermside wrote: For projects in ALL languages, the difficulty in getting a patch accepted is a genuine issue (and is difficult to solve). There is a sweet-spot of small, recent projects that accept patches easily. Older projects are often neglected (or abandoned) by their original authors, which makes accepting patches difficult. Larger projects necessarily set standards (like requiring unit tests or moderately complicated review processes, or certain coding standards) which not every patch will meet. The problem is not easily solved because these are inherent problems rather than just projects being obstinate. Meanwhile, Java programmers are probably worse because (1) many well-known open-source Java projects are large and corporately sponsored which makes it difficult to get them to accept patches, and (2) many Java programmers work in corporate environments where it is difficult to get "permission" to contribute to an open source project. Simply not being in the habit of doing so makes the practice less likely than with other languages.

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (98.136.63.35) )

Many other companies have already stopped supporting older browsers like Internet Explorer 6.0 as well as browsers that are not supported by their own manufacturers. We’re also going to begin phasing out our support, starting with Google Docs and Google Sites. As a result you may find that from March 1 key functionality within these products — as well as new Docs and Sites features — won’t work properly in older browsers.

w00t! Die, IE6, die!

Comments

• February 15, 2010, Colin Harrington wrote: Dear IE6: You are dead to me. goto /dev/null Thank you. ~ The Interwebs

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (98.136.63.35) )

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (98.136.63.35) )

Anyway, of all the parables and the jokes and the pithy aphorisms he’s told me, this is the one that I’ve found most key to software engineering, and it jumped to mind after reading Marc’s recent groovy-user post.

The leader of the tribe looked askance at the young man. “Are you telling me that you have improved on the wheel?”

The young man nodded in excitement, “Yeah! You wouldn’t believe it! I just went back to square one. I started from scratch with the generation of experience and knowing all these problems we have with wheels, and I went off and developed a proof of concept, and it seems to be working out pretty well. I’ve been showing it to other people, and they love it, so I’ve got a prototype that I want to show you. This new wheel is really the wheel of the people—the elders won’t be the only people who can invent new things based around wheels anymore! Even better, all my friends agree that the new design solves the problems we have with wheels: we can produce them at speed, it doesn’t roll away when we want to stand still, and it’s easier to understand!”

The leader of the tribe did not seem impressed. “But what new design improves on the wheel?”

“Well, my new wheel is just like the old wheel—but this time, it’s square!”

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (98.136.63.35) )

That conversation, along with the comments on Marc’s “Do we need a commercial market for Grails Plugins?” post, suggest that there’s a bit of a misunderstanding about the economics open source software. Most importantly, people seem to be missing that commercializing open source software seems to be fundamentally self-destructive.

Take the standard recommendation to support yourself on OSS: offer consulting/training/speaking. The problem with this approach is that you’re directly competing with the time you have to do your development, so the more successful you are, the less time you have for open source. Building a consulting business and prepping presentations and training is hard and time-consuming work and it eats up a lot of your life. There was a blog post a while back (I’ve lost it) which event went so far as to argue that there simply isn’t enough time to be an accomplished open source project lead and be an accomplished writer/speaker. You can’t serve God and Mammon. This has been my experience: I have to make decisions about where to spend my time, and my open source projects are consistently on the losing end of that stick in favor of paid gigs (doubly so since the economic downturn hit my and my wife’s consulting pipelines about a year ago). This dedication of my time to paid gigs over OSS is fine by me—I’m making the decision about how to spend my time with intent and forethought—but it does show that as a way to “support yourself on OSS”, consulting/training/speaking fails because it wipes the OSS off the map.

There is a point where the speaking/training/consulting balance flips is when you can subcontract out the training and consulting, leaving the core developers to primarily develop. This requires a level of adoption and need that is a lot larger than a Grails plugin or minor Java library is going to have.

In addition, I always consider the training/speaking/consulting approach to be a bit of a conflict of interest. If you write a solid, accessible peace of software, what’s the value in your consulting? And if you’re consulting, aren’t you even more interested in just your own needs to the exclusion of what the community may want? The prime example for the latter is the conspicuous change from Jetty to Tomcat for the Grails default server right after SpringSource purchased them. Seems like the existence of Spring tcserver might have something to do with that…

So, if we toss consulting/training/speaking out, I’m aware of basically have two approaches to making money doing open source software.

First is dual-licensing. Zed Shaw makes a good case for it, and I’ve been thinking about giving it a shot with some of my projects. A post by Greg Stein that made its rounds on Twitter today argues the GPL can screw consumers because you’re basically banking on the benevolent dictator remaining benevolent. I’m unimpressed by the argument, which is basically whining that the GPL is too strict a definition of open source for these proprietary-software companies. Greg Stein thinks companies, who are taking open source and making money off it without returning to the community, are somehow “misled” when they rejected the GPL and opted instead to pay for a more permissive license. I have little sympathy for them. If they don’t like the GPL, use the BSD-licensed postgreSQL. In any case, the dual licensing approach seems like a reasonable way to go.

My reluctance to dual-licensing approach is threefold: 1) releasing code under the GPL means having to enforce GPL violations when/if I find them; 2) adoption is stunted, because many companies run screaming from software which even uses the characters ‘G’, ‘P’, ‘L’ in sequence, even if there is an alternative license they could buy into (another self-destructive side of open source software); 3) in the web development space, the GPL basically has no teeth thanks to the so-called SaaS Loophole: you don’t distribute the software but you charge people to use it, and you eventually sell the company (not the software), so the GPL never comes into effect. But it’s better than nothing.

Another alternative is Rich Hickey’s “Funding Clojure 2010″ approach, which is basically begging for donations—becoming an Open Source Friar of sorts. Rich Hickey may be able to get that to work, but I don’t think there’s a big enough audience for my work to get that to happen.

An alternatives along the same line is crowd-funding improvements, but that two critical failure points. First, developer time isn’t cheap—I have to feed myself and my family. Despite the fact that my open source effort might save companies dozens of hours of your developers’ time (assuming they trust their developers to do it, which many don’t!), companies seem to balk at reasonable rates. This is doubly-true in the web sector, where “companies” often means “single-person self-funded start-ups”. Second is that there’s a lot of work involved in avoiding bit-rot on a project which doesn’t qualify as an improvement. When a new version of Grails comes out, I have to go through all my plugins and make sure they still work right, often discovering bugs in Grails in the process. Who is going to want to fund that?

So, if we’re coming up with a way to support ourselves while doing open source software development, we have two critical requirements.

1. Our income needs to come from developing on the open source project itself. Auxiliary income sources ultimately detract from the open source development itself.
2. Our income needs to be ongoing, not single-shot. The shrink-wrapped software model doesn’t work, because the open source software I’m producing isn’t a self-contained product but an adjunct to another piece of software.

So, I’m really not sure what to do with this. A subscription model seems to be in alignment with our requirements, but how would that work? Some kind of patronage would work well, too [EDIT: (see comments)], but is that going to fly in this economic market? I just don’t know.

Comments

• December 15, 2009, Eric Pederson wrote: What about dual licensing with AGPL?
• December 15, 2009, Marc Stock wrote: "Some kind of patronage would work well, too, but is that going to fly in this economic market?" No, it won't. Of course, it won't fly in a good market either. I've found developers to be unbelievably cheap bastards. You could produce a tool that would save them 2 hours of time a day and unless they can get it for free, they will tell you to f* off. You may think I'm overstating things, but I'm really not. I've tried to convince people to buy tools with clear value many many times in my career and it just doesn't work. I'm one of the only developers I know of on the planet who actually budgets money every year to buy whatever tools, libraries, etc. I want. That budget is generally $1000 per year. I challenge you to find a developer who even sets aside $50 per year. If you ask them why, they will tell you that their company must by it for them. It's funny that car mechanics (who make a third of our income) buy all their own tools (yes, repair shops almost always require mechanics to pay for their own tools) which adds up to thousands of dollars and don't bat an eye because they know that's what they need to get their job done efficiently. There's a big problem in developer culture that needs to change but I don't see any change on the horizon.
• December 15, 2009, Robert Fischer wrote: @Eric Pederson The AGPL does solve the SaaS loophole: see In AGPLv3, what counts as “interacting with [the software] remotely through a computer network?”. So that's one issue down. Like I said, I'm inclined to try the dual licensing route: I just don't have a lot of hope for it.
• December 15, 2009, Robert Fischer wrote: @Marc That's been my experience, too: it's hard to get money out of the developer community unless you've got a cult of personality which drive people to your products *cough*37Signals*cough*. IntelliJ seems to have bested your cynicism, but that's about the only one I can point to. The "patronage" model I was referring to isn't the kind of watered-down meaning of "WalMart thanks you for your patronage". I'm talking about a full-fledged "I want to hire this developer to do cool stuff to solve our problems, so that the cool stuff flows into our business first". Even ignoring explicit brain shops, there are a few de facto patronage relationships out there I've seen—usually someone with the title of "architect" who solves problems in the large by providing tools for work in the small. If that kind of position could be merged with OSS work, that'd be my ideal kind of situation.
• December 15, 2009, Marc Stock wrote: I can't even get most developers to *look* at IntelliJ, let alone buy it. Almost every one tells me that either Eclipse is perfectly fine so there's no need to look elsewhere or eclipse is free so how can something that can only be "incrementally better" justify the price. I don't even try to convince people any more. I can also name at least a half dozen developers who were given free licenses to IntelliJ who won't even download it. Your patronage idea is cool although I'm not sure how you sell it in today's world. It's a pretty old school idea though. Patronage was how most artists were funded through middle ages and renaissance, as I understand it.
• December 15, 2009, Robert Fischer wrote: Despite your experiences, JetBrains seems to be doing just fine with it. I see a lot of IntelliJ in the wild on the NFJS tour. And especially in the early days of Groovy/Grails, IntelliJ was pretty much ubiquitous on in my contracting world.
• December 15, 2009, Robert Fischer wrote: BTW, the patronage model is the basic model underlying my Grails Retainer services. And that's working out somewhat well, although work hasn't been as long-term as I had envisioned.
• December 15, 2009, Hamlet D\'Arcy wrote: >> This has been my experience: I have to >> make decisions about where to spend my time Congratulations on being an adult: the long era in your life where you have to make tough decisions between what makes you happy and what your obligations are as a human. If you would like to see this effect magnified then you should have children. Then you'll really be confronted with tough decisions about what you want versus what your family needs. I, for one, have enjoyed this period in my life. I feel that finding the balance between work, my daughter, graduate school at night, my wife, and open source software has made me a better person. I don't really have friends, but that was just one of the tough decisions I've had to make. Honestly, you and the gang at Groovy.MN are my closest friends, and we see each other less than once a month! Ok, I'm officially sad now...
• December 15, 2009, Hamlet D\'Arcy wrote: And here is the quote that popped into my quote a day just now: I don't know why we are here, but I'm pretty sure that it is not in order to enjoy ourselves. - Ludwig Wittgenstein
• December 15, 2009, Robert Fischer wrote: My question isn't about what makes me happy vs. what doesn't: my question is how to we build a sustainable OSS ecosystem, especially in the context of Grails plugins but also with respect to things like Rich Hickey's work on Clojure. In looking at the OSS ecosystem, we have two choices: 1) we accept that OSS/Grails plugin dev is a "free time only" enterprise; or 2) we need to have a way for people to support themselves. If we accept #1, we need to be honest about that and build a liveness check into the Grails plugin ecosystem, because the 300+ plugins are actively and quickly bit-rotting and that's just how reality works. Also, if we are accepting #1, then people need to back the Hell off and not be so demanding of the plugin developers (see the Free Toilet post). If we accept #2, then we need to figure out how people are going to support themselves, which is what this post is exploring. And I'm noting that relying on consulting and speaking to support OSS is destructive to the projects: it's not a workable model/expectation for OSS support, at least in the world of the Grails plugin ecosystem.
• February 20, 2010, DynamicDomainProperties and the AGPL wrote: [...] plugin culture because of differing cultural assumptions about open source works, and based on my lack of appreciation for the promises of indirect compensation offered to me as an open source de..., I’ve decided to release it under the GNU-Affero GPL 3.0, which is like the GPL but (among [...]

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (98.136.63.35) )

The crux of the problem is adequately described in Marc’s GraphJam-worthy graph:

The problem is, this leads to a bunch of great ideas that are 3/4ths implemented, with lots of unimplemented great ideas still sitting around and twiddling their thumbs. (Autobase, I’m looking at you.) This does damage to the Grails ecosystem, starting with a lack of trust in plugins and ending with burnt out plugin developers angrily reclaiming their free time. So, how do we help the original developer make time for plugins, or how can we get other developers to step up and contribute (despite the consumer-developer paradigm)? That’s what Marc goes looking into, and both the original post and the comments are extremely insightful.

[PS: I'm thinking about integrating some link sharing capabilities into EnfranchisedMind so that I can queue up and share a bunch of good posts like this every month or so. Thoughts? —Admin]

[PPS: I've closed comments. Go post them on Marc's blog.—Author]

Comments

• December 14, 2009, Enfranchised Mind » The Self-Destructive Side of Open Source Software wrote: [...] conversation, along with the comments on Marc’s “Do we need a commercial market for Grails Plugins?” post, suggest that there’s a bit of a misunderstanding about the economics open source [...]

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (98.136.63.35) )

setJDK

#!/bin/sh
 
cd /System/Library/Frameworks/JavaVM.framework/Versions
 
CURJDK="`readlink CurrentJDK`"
echo Current JDK version: $CURJDK
 
if [ "$1" == "" ]; then
echo Installed versions:
ls
exit
fi
 
VERFOUND=`ls | grep $1 | head -n 1`
 
if [ "$VERFOUND" != "$1" ]; then
BASE="`basename $0`"
echo Error: Could not change JDK-- version $1 not installed!
echo Run $BASE without arguments to see a list of installed versions.
exit 127 fi
 
echo You must now enter your Mac OS X password to change the JDK.
sudo ln -fhsv $1 CurrentJDK
sudo ln -fhsv $1 Current

I found that setJDK code somewhere on the internet, but I’m not really sure where and I had trouble finding it again. So I thought I’d share it here.

To integrate SoyLatte Java (which you probably want to be using anyway), just create a symlink in /System/Library/Frameworks/JavaVM.framework/Versions that points to your SoyLatte install.

setGrails

I’ve modified that script a bit to allow me to modify the version of Grails to use. This script assumes that you have your Grails distributions stored in BINDIR, and you’ve added $BINDIR/grails/bin to PATH. Once that’s set up, this script works like a charm:

#!/bin/sh
 
cd $BINDIR
 
CURGRAILS="`readlink grails`"
echo Current Grails version: $CURGRAILS
 
if [ "$1" == "" ]; then
echo Installed versions:
echo `ls -1 | grep "grails-"`
exit
fi
 
VERFOUND=`ls -1 | grep $1 | head -n 1`
 
if [ "$VERFOUND" != "grails-$1" ]; then
BASE="`basename $0`"
echo Error: Could not change Grails-- version $1 not installed!
echo Run $BASE without arguments to see a list of installed versions.
exit 127 fi
 
ln -fhsv "grails-$1" grails

Finger-Saving Tips

If you’re really hardcore, add these lines to ~/.profile:

alias sg=setGrails
alias sj=setJDK

Comments

• December 11, 2009, Fred wrote: Nice. Command line tooling just doesn't seem to be hotness in the Java community, so tips like this don't get shared too often, it seems to me. I like!
• December 11, 2009, Robert Fischer wrote: Too many people are too dependent on IDEs. I find more power with less annoyance in the command line. But, y'know, I started my programming life pissing people off in perl (e.g.) instead of pissing people off on the JVM, so I have an intimacy with the command line I don't find in a lot of Java or .Net developers.
• December 11, 2009, Gabe Beged-Dov wrote: Hi Robert, A similar version that separates the "default" (relink) from setversion is described in this post: http://old.nabble.com/setgrails-version-%28bash%29-script-with-completion-td26410455.html Best regards, Gabe
• December 11, 2009, Nurullah Akkaya wrote: setJDK script is from, http://www.macosxhints.com/article.php?story=20060121171126988
• December 11, 2009, Gabe Beged-Dov wrote: The setjdk script (http://docs.codehaus.org/display/ninja/setjdk) referenced in my grails-user post (see earlier comment) supports setting a default version as well as setting a per-process version. It also has command completion. Best regards, Gabe
• December 11, 2009, Charles Oliver Nutter wrote: Try Nick Sieger's "pickjdk" script...I love it. I made some modifications to allow selecting by number in one command as well: http://gist.github.com/143128
• December 11, 2009, Ted Naleid wrote: Nice! I did something similar a while ago where it switches both the groovy and grails symlinks with one command. I like to have them in sync and can never quite remember which dot version of groovy belongs with a grails version. Mine is more manual than this though because it can't interrogate what's actually installed in the directory, you need to manually add an alias pairing the particular groovy and grails versions together. Somethign I could automate with a little awk/sed magic and looking in the lib dir, but the pain hasn't quite hit that level yet :). Thanks for sharing. I love seeing other shell tricks for coding.
• December 12, 2009, Gabe Beged-Dov wrote: The problem with relinking is that it is global so you can't have several concurrent shells using different versions of the framework. That's where the distinction of setdefault vs. setversion is useful. Also very nice to have completion.

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (87.250.252.241) )

In doing development, I’m spending some time pushing through integration tests, and in so doing, I wanted to deploy various versions to a Maven repository and pull them back down.

Creating a Maven repository for SmokejumperIT was pretty simple.

1. Create a space on the internet for the repository which is accessible via public-key-authenticated SCP. In my case, it’s http://repo.smokejumperit.com.
2. Create a configuration in the Gradle build file for the JARs needed to deploy:

   configurations { deployerJars
   }
    
   repositories { mavenRepo urls: "file://${System.properties['user.home']}/.m2/repository/" mavenCentral()
   }
   dependencies { deployerJars 'org.apache.maven.wagon:wagon-ssh:1.0-beta-2'
   }

   Note: The part in the repositories block is in every build file that I have, so I’m working on a plugin (working title PreconfigPlugin) that will automatically add that in.

3. Configure the uploadArchives task. I had it delegate to various System properties that I set when I run the build script.

   uploadArchives { repositories.mavenDeployer { def repoUser = System.properties['sjit.repo.user'] configuration = configurations.deployerJars repository(url: "scp://repo.smokejumperit.com/home/$repoUser/repo.smokejumperit.com/") { authentication( userName: repoUser, privateKey: new File(System.properties['sjit.repo.keyFile']).absolutePath, passphrase: System.properties['sjit.repo.passphrase'] ) pom { groupId='com.smokejumperit' artifactId='gradle-plugins' version='0.2-SNAPSHOT' } } }
   }

4. Run gradle upload.

That’s all it took. Even better, it does all the funky snapshot management on its own: see http://repo.smokejumperit.com/com/smokejumperit/gradle-plugins/0.2-SNAPSHOT/ for the beautiful timestamped results!

This is how builds should be: stupid-simple to do the right thing.

Comments

• November 27, 2009, Hamlet D'Arcy wrote: I'd like to hear what you think about Gradle being the result of applying Evans-style Domain Driven Design to build systems. Is a strong domain model for enterprise projects really a great benefit?

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (87.250.252.241) )

Currently, there are just two plugins: one to get at classloaders and one to execute shell commands. I’ll probably have some JRuby/Cucumber, maybe JAVACC, and almost certainly some conventional development plugins sometime in the future.

Note that I’m releasing the Gradle plugins under the CC0 (which is a classier form of my beloved WTFPL). Here are the details:

To the extent possible under law, Robert Fischer has waived all copyright and related or neighboring rights to Smokejumper IT Gradle Plugins. This work is published from United States.

Comments

• November 25, 2009, Hamlet D'Arcy wrote: great news. Why not just submit the plugins back to the Gradle project?
• November 25, 2009, Robert Fischer wrote: @Hamlet Is there a way to do that? I wasn't exactly sure how plugins in the Gradle source code worked, but I'd be more than happy for the code to make it into Gradle core.

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (87.250.252.241) )

Caused by: org.jruby.exceptions.RaiseException: library `enumerator' could not be loaded:
java.lang.ClassNotFoundException: org.jruby.libraries.EnumeratorLibrary at (unknown).new(:1) at (unknown).(unknown)(:1)

After tracking through the JRuby source code, I found that RubyInstanceConfig was grabbing the thread context classloader instead of the classloader used to load itself. I don’t know what that classloader was set to, but resetting it solved the issue.

Here’s the resulting Gradle script, which at least runs the gem installs and a Cucumber run:

import org.jruby.Main as JRuby
 
buildscript { repositories { mavenCentral() } dependencies { classpath "org.jruby.embed:jruby-embed:0.1.2" }
}
 
task checkGradle << { println "OK"
}
 
task gems << { def gemsDir = "./build/gems" if(!new File(gemsDir).exists()) { runJRuby("gem install -i $gemsDir --no-rdoc --no-ri cucumber") }
}
 
task runTest(dependsOn:gems) << { def cukeHome = new File('./build/gems/gems').listFiles().find { it.name.startsWith('cucumber-') } runJRuby("$cukeHome/bin/cucumber src/test/features")
}
 
def runJRuby(cmdArg) { Thread.currentThread().setContextClassLoader(JRuby.class.classLoader) println "Running JRuby: $cmdArg" JRuby.main("-S $cmdArg".split())
}

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (87.250.252.241) )

For most websites, this relationship to their customers is fine. A local restaurant’s website, for instance, might just display information about the restaurant and maybe a menu. There is no reason this needs to be protected. If the website asks for private information from the user, though, and especially if the website has a concept of “logging in”, then some security is necessary.

Without security, the information being sent over the line is unprotected. Minimally, this means that anyone on the same internet connection can read all the information going back and forth, including the private information and user name and password. This is true even if the password is masked in password fields. In a worst case scenario, though, someone else would be able to hijack the communication between your server and the user, basically taking over the user’s identity.

A naive solution would be to use some simple encryption: the server hands the user a key, the user encrypts messages with the key, and then the user sends the messages to the server. This works out okay, except that someone standing between the server and the user could substitute in a second key, thereby hijacking the conversation. This is called a “Man in the Middle Attack”.

The solution to this is HTTPS. HTTPS, sometimes called “Secure HTTP”, basically creates a network of trust between the server and the user. The server pays to have their identity assured by some known-trustworthy party. The user receives this certification of identity along with the key, so they know they can trust the key. That key then encrypts the information and sends it back. The Man in the Middle Attack doesn’t work because that certification of identity can’t be faked, so any attempt to substitute in a second key is caught right away.

This is why you will have to pay a fee to some trusted party (such as VeriSign) if you want HTTPS set up for your server: that fee goes to verifying your identity and creating the certificate. Once you have that, your users will know that they can trust the communication between themselves and your servers to be safe.