"This is what switching out of the Java paradigm, into Ocaml or Haskell, is like."
If you try to do real, production programs in Haskell or Ocaml, you find that they have a lot of problems too.
I've written some reasonably cool stuff in Haskell (like one of the top tutorials that people use to learn it). My teammate got his Ph.D under Simon Peyton Jones and did his doctoral thesis in Haskell. We were talking one day about what it'd be like to write real production systems in it. His strong opinion (and my weak opinion) was that it'd be hellish, because of certain misfeatures like lazyness or the hacked-up record system. All the people that complain about Java would merely find new things to complain about with Haskell, other ways in which the language prevents them from getting useful work done. Instead of whining about verbosity, they'd whine about space leaks.
I find that people who are a 9 or 10 on the expertise scale usually know Haskell, but they don't program in Haskell. Because the causation works the other way. If you're a good programmer, you'll seek out other languages and learn them, because it'll make you a better programmer. But if you're a great programmer, you'll know when not to use stuff you've just learned, because it'll just make your life harder. Learning Haskell doesn't make you a great programmer, but great programmers have probably learned Haskell at some point in their path to greatness.
I've been playing around with Haskell recently (actually have been for years, off and on) and have been thinking about doing a real production system in it in the near future.
What is going to be so hellish? Space leaks? That doesn't seem like the worst thing in the world to me. Lots of scripting languages tend to bloat over time and the usual answer is to kill and restart the process every now and then. Even Apache does this to its children.
If I have to kill some daemons every now and then, but I get the robustness that Haskell seems to offer, it sounds like a great tradeoff to me.
I don't think laziness by default is all that useful in practice, but neither does it seem to be a fatal problem.
I'm not all that impressed with the web platforms for Haskell, but I'm thinking of doing Haskell in the backend, and Python in the frontend.
So... go ahead, pop my bubble; tell me how I'm being stupid...
If you're comparing to Python (which is stable but hella slow) or Ruby (which tends to be both unstable and slow), GHC stacks up pretty well. By "production systems", I was mostly referring to the sort of things that are normally written in Java or C++ - Google-scale server farms, or financial transaction systems.
You really want predictable execution behavior in those systems. You want to be able to look at a graph of runtime parameters - memory usage, or server load, or QPS, or 95th percentile latency - and see that it's stable, and not have surprises like a massive spike that then shows up as user-visible outages. Because getting paged really, really sucks, and losing money because your customers can't access the system sucks even more. "Kill and restart the process every now and then" is not robustness.
If your intention is to play around with some prototypes and see what you can build, Haskell on the backend and Python on the frontend is a fine choice. Your biggest issue (other than the learning curve of Haskell) will be finding libraries that you can leverage, and Haskell has one of the nicest C FFIs out there. Other than that, here're assorted other little annoyances I've run into.
1. Positional data structures (tuples, "naked" [non-record, no accessor] ADTs, functions with long parameter lists) really suck. You will eventually want to stick another bit of data in there, and then you have to update all the usage sites with new pattern bindings. As immature as the record system is, take the time to use it from the beginning; it'll save you pain later on.
This is a very common pattern in C++ as well; many experienced API designers I know will give all their methods the signature of "bool someMethod(const MethodInput& input, MethodOutput* output)", where MethodInput and MethodOutput are structs. At Google, we often use protobufs as the structs; that gives you a bunch of other benefits, as you can instantly print them out for debugging, and you can log them efficiently, and it becomes trivial to turn the API into an RPC service if it outgrows a single process.
2. Don't be afraid to use type inference; oftentimes, the compiler can infer a more general type than you can write down. If you do write down the types, be liberal with your use of type synonyms; you will probably end up changing those types later, and it sucks to have to visit a few dozen functions with a search & replace. "newtype" is very often worth it, if just for the added typechecking.
3. Point-free style is cool, but I've never been able to look at point-free code I wrote a month ago and understand what it was doing. Don't do this outside of very specific cases where the point-free version has some semantic meaning (for example, you're treating a partially-applied function as an object that does something. If you wouldn't define a closure for it in Python or JavaScript, don't make it point-free in Haskell).
4. Haskell module patterns are much more similar to C than C++ or Java. You typically want a clear divide between "data" modules (that define fundamental dataypes) and "code" modules (which define functions and algorithms acting on those datatypes). Trying to use the OOP-style, "one class per file" approach is very frustrating in Haskell.
5. The "mathematician's naming convention" (all single-character variable names, with "s" as a plural to indicate a list) is just as bad an idea in Haskell as it is in C. Go give your variables some meaningful names and ignore all the tutorials that have "xs" and "ps" and "qs".
6. Very often, you'll want to define a monad for your code from the get-go (you can even just typedef the IdentityMonad to start out with) and write your code in a monadic style from the beginning. You will want to add error-handling and probably state and logging at some point in the future. If you start out with a typedef'd monad type, you can just change the typedef to some monad transformer stack as you need to.
However, you're still suggesting that Haskell is only good for prototyping, not for "real apps", which understandably for you are Google-scale apps. There is a very long distance between what Google does and what I need to do, today, and premature scaling is the death of many projects. But let's be ambitious and pretend that one day I will want to operate at that scale. Assuming C++ is out of the running, is there really no alternative to Java?
I've been working on apps in PHP (Flickr, now Wikipedia) and while I admire the language, in the same way I admire how weeds can colonize an area efficiently, I am not going to choose that for a personal project. I agree with your assessments of Python and Ruby, and I'm not sufficiently enamored of any framework in those languages to make it worth the trouble. I can sling Perl as well as anybody (even in a style approaching functional) and it can be very efficient, but it's hard to find collaborators these days.
This seems to leave Erlang, Haskell, Scala, or maybe Node.js + JavaScript.
Scala seems like what an enterprise would choose for its foray into functional programming, and indeed that seems to be the hopes of its creators. While it's a "safe" choice, for Java shops maybe, I fear the massive shear forces between Java's way of thinking and Scala's attempt to paper over all that with functional thinking. Not to mention the complexity of that language. Still, it seems to work for Twitter, FourSquare, et al.
There's nothing wrong with Erlang, and it has its adherents in industry, but the syntax annoys me.
I'm already an expert in JS, and a codebase of a single language has its benefits. The intense competition in the browser to make JS an application language seems to have already vaulted JS performance over other scripting languages and new libraries are being written every day. Still, it just feels odd to write directly to a database or file in JavaScript. And I've gotten myself tangled up in JS' asynchrony a bit too much recently. Maybe that's an irrational prejudice.
I guess this is why I've been looking on Haskell; in part because it offers some hope of really writing a lot less code and having that code be so well-typed, automatically, that it practically writes its own tests (see QuickCheck) or doesn't even need them. It seems like there ought to be some way to get the best of that without needing to shoot yourself in the foot because you suddenly want to log some action and you didn't define some code twenty layers above it as monadic.
Assuming C++ is out of the running, the alternative to Java is basically Scala. And that's it. Maybe Go will be an alternative in the eventual future, but right now it lacks the library & tool support.
There's also nothing wrong with writing your v1 in one language and then rewriting in a different one if you need to scale. You will probably need to do this anyway, even if you start in C++ or Java, because the architecture needed to scale by an order of magnitude is usually very different from the architecture needed to iterate quickly.
I've written a lot of code in F# and I have to agree that writing business software in a functional language has a little bit of a learning curve but once you figure out how it's done it's much easier.
While I appreciate your experience, you basically are saying that it would be hellish because of two features. Haskell and/or OCaml have many incredible benefits, and many of them do not rely on laziness to work. Sure it allows you to write some incredibly elegant, generic solutions - but I don't think it is necessary to have it as the default.
I think things like the record system could be fixed up with simply more eyes on the problem. Lets face it, the Haskell community and contributor base is relatively small. If it had the resources of say C++ or Java behind it - a lot of these small problems could be tackled.
But when it comes down to it these are minor details, not fundamental flaws of the platform such as the ones discussed with Java.
Also I do program solely in Haskell during my free time (ok, maybe some matlab on the side haha). While the learning curve has been incredibly intense - it is so damn rewarding that I keep coming back. The solutions are incredibly generic, elegant, testable, and even quite fast with a little performance optimization work (with experience the amount of effort required for this tends to go way down as with anything). I guess I would rather write the correct code first, and put a little time into optimization than write highly optimized incorrect code and spend a lot of time fixing bugs.
But far and away the largest problem that Haskell goes a long way towards solving is composability. Many may argue this point but I really believe this is the largest challenge we face today in software. So many issues boil down to this. The vast majority of bugs, re-writing, etc... comes from software that was not designed in a composable manner. Because let's face it - with the current tools like Java/C++ it is incredibly hard!
Oh, I think that nearly all of the problems with Haskell could be fixed up if it had the resources of C++ or Java. The problem is that in getting those resources, it would create new problems that would end up looking a lot like the whining about C++ or Java.
Java, as a language and a platform, is not fundamentally that bad. It was certainly done by very smart people with lots of experience.
It sucks because it has several design decisions embedded into it that were necessary for it to gain broad adoption and yet ultimately pretty annoying in the long run. Take closures, for example. The reason Java doesn't have closures is because it had a design constraint that all heap allocation must be indicated by the "new" keyword (same reason it didn't have autoboxing until Java5, and why classes aren't factory functions they way they are in Python). The reason for that was that at the time, the majority of programmers believed that heap allocation was slow and unpredictable, and in order for a new language to gain adoption, programmers needed to be able to control heap allocation and ensure it was used predictably.
This is actually still true - in domains like avionics and other "hard real time" areas, heap allocation is generally banned because it introduces nondeterminism into a program's run-time characteristics, and even with tuned incremental GCs, Java programs still spend a non-trivial amount of time in the GC. It just doesn't matter all that much, because Java started being used for applications where a few hundred millisecond pause for GC just gets lost in network latency.
Haskell faces a similar uphill battle. In order to get the critical mass of people needed to build out the ecosystem that C++ or Java has, it needs to be useful for the programs that people write today. But in order to do that, it needs to accommodate all these wonky developers with their weird ideas of what's necessary and what's not. Chances are, if it did that, there would be some design choices made that make Haskell nearly as lame as Java. And even if there aren't, and Haskell remains exactly the same as it is today, it'd soon be pushed into domains it wasn't appropriate for, just like Java, and people would be cursing it out for its flaws.
As Simon Peyton Jones says, "Avoid success at all costs."
There's a two-edged sword to every design decision that you make. Yes, Haskell gives you a lot of composability through lazyness, type inference, and nearly everything being first-class. Lazyness grants this through "pay for only what you use" - if a caller never uses the result of a library, the parts of the library's code that generate that result will never be invoked. The flip side of this is that if you suddenly use one more field in a tuple, way down the line in some unrelated portion of the code, you might trigger a very expensive computation. This is unacceptable in many large apps written by multiple people; you shouldn't have to worry that a tiny field reference can bloat the program's memory consumption by orders of magnitude.
(I have example trade-offs for type-inference and first-class everything, as well, but this post is already getting too long, and I wanna go do other things with my Saturday. ;-))
> The flip side of this is that if you suddenly use one more field in a tuple, way down the line in some unrelated portion of the code, you might trigger a very expensive computation. This is unacceptable in many large apps written by multiple people; you shouldn't have to worry that a tiny field reference can bloat the program's memory consumption by orders of magnitude.
No matter what language you use if you want to use the results you're going to pay the calculation/resource fee. All Haskell (via lazy calculation) is doing is giving you a speed increase when it can tell you aren't using certain results.
But other languages give you a syntactic cue that what you're doing might be potentially expensive - it's a function call. That's why most language styleguides have a naming convention for accessors, so you can distinguish between method calls that are basically free vs. method calls that might be expensive.
In Haskell, every single token might trigger a long computation, so it's very easy to introduce something that wildly changes the performance characteristics of the program.
Hmm I have come across many a codebase in countless languages where there was no way to discern that type of information from the naming convention. Even accessors in say C# or Java are perfectly legit to do any side effect they want. In fact I think it is worse in those situations - because side effects are not modeled. Indeed, no language has found a way to model non-termination in the type system, hence the halting problem.
Honestly I just haven't experienced this very often in Haskell... When I do it is usually a 'doh moment and a simple fix.
I would use Ocaml before Haskell as a production language, actually. It's hard to reason about space performance in a lazy language, as I'm sure you can imagine.
Ocaml is an elegant, simple language that is in many ways like a "functional C". You can also write very fast, robust code in it. Haskell is where the next generation of cool language concepts is coming from (like Lisp in the 1960s-80s) but I'd prefer Ocaml for production use.
That said, you'd need to put some resources into rewriting the runtime to make use of multicore. Ocaml is blazingly fast but right now the implementation is single-core only.
Ocaml has its own performance gotchas (especially on 32 bit machines) although they are much easier to work around once you know them. For example, if you allocate a string larger than 16MB on a 32 bit machine ocaml will segfault. If you don't know this little fact that can be a really hard bug to track down.
I also had issues with the garbage collector traversing arrays of native types which is a complete waste of time, especially when said array is 7GB and immutable.
I don't want to give the wrong impression - ocaml is a fantastic language. It just has some poorly documented rough corners which can bite beginners.
These are interesting observations. I'd love to hear more about them.
Ocaml's problem, from what I've seen, is that INRIA wants to run a tight ship regarding the design of the language. This makes sense, but it also means a lot of real-world use cases get ignored. For example, my understanding is that priming Ocaml for multicore use would require substantial rewrites that aren't a high priority, given INRIA's desire to maintain it as a "research language".
The 2^22 - 3 array limit (and the corresponding 2^24 - 12 string limit) on a 32-bit machine (it's 2^54 - 3 on a 64-bit system) are examples of this: the language designers simply didn't anticipate that people would be using the language for purposes that require such enormous arrays or strings. These use cases are tremendously important for real-world production use but not terribly interesting from a research perspective.
This is one of the reasons I love Go, it is preeminently pragmatic language, not designed for "mediocre programmers" (Java) or "genius programmers" (Ocaml, Haskell, C++), or to fulfill feature checklists, but to be useful and used by hackers that want to get stuff done and not just to think about stuff.
If you try to do real, production programs in Haskell or Ocaml, you find that they have a lot of problems too.
I've written some reasonably cool stuff in Haskell (like one of the top tutorials that people use to learn it). My teammate got his Ph.D under Simon Peyton Jones and did his doctoral thesis in Haskell. We were talking one day about what it'd be like to write real production systems in it. His strong opinion (and my weak opinion) was that it'd be hellish, because of certain misfeatures like lazyness or the hacked-up record system. All the people that complain about Java would merely find new things to complain about with Haskell, other ways in which the language prevents them from getting useful work done. Instead of whining about verbosity, they'd whine about space leaks.
I find that people who are a 9 or 10 on the expertise scale usually know Haskell, but they don't program in Haskell. Because the causation works the other way. If you're a good programmer, you'll seek out other languages and learn them, because it'll make you a better programmer. But if you're a great programmer, you'll know when not to use stuff you've just learned, because it'll just make your life harder. Learning Haskell doesn't make you a great programmer, but great programmers have probably learned Haskell at some point in their path to greatness.