"the idea that dynamic languages are more productive than static languages are laughable." -- being statically typed or dynamically typed comes with its own set of tradeoffs and what a person is more productive in is a highly subjective matter. Lispers are more productive in Lisp than Haskell and vice versa.
"Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactorings that simple cannot be done with a statically typed languages." -- not true, Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make.
I recently saw an analysis of frequency of bugs per language according to a breakdown of Github repositories, using issues and branches as a way to quantify bug reports.
It was very interesting to see that Clojure projects were among the least buggy of all languages represented. The report looked at a variety of factors to explain "bugginess", including typing and LOC, among other things. The conclusion in Clojure's case was that the very low line count necessary to write Clojure programs was vastly more influential on bugginess than for languages that were statically typed.
And I suppose that's true: a language that requires you write very little code is going to produce programs that are easier to write (and thus maintain), regardless of whether it is statically or dynamically typed.
That idea is old enough to have entered the original edition of The Mythical Man-Month.
That said, I have never seen any study like this where I didn't have some serious disagreement over methodology or conclusions. (Ok, except for that one on Peopleware.) It is a hard subject, and running a study over "the population of GitHub" is problematic by itself.
In my own subjective experience, I've found the idea to make a lot of sense and probably be true. I've done equal amounts of professional work in C++ and in Clojure, both languages I like a lot (and they are about as different as two languages can be). To accomplish a task in C++ takes probably 10x the code of Clojure. I would say that my bug-hunting time spent is maybe 5x in C++ vs. Clojure. This, despite that C++ is quite strictly statically-typed.
That is my impression too. I find that the most relevant metric impacting the bug density is number of lines on this one project, and it's very non-linear.
That puts at advantage both languages that reduce line count, and languages that make it possible to abstract the plumbing into a generic library (if it's not generic, it's part of this one project). I think mostly everybody has that same impression.
Still, we shouldn't pretend that impression is a scientific certainty.
Sounds like an interesting analysis, would love to know where Erlang stands. Terseness definitely is a factor in play but I think at the end of the day it is all about where the priorities lie in language design, some languages really take errors and fault tolerance seriously, Erlang is famous for that and it is dynamically typed, same with Clojure, being a lisp the language design is much less rigid which allows for improvements like core.typed and Clojure.spec
Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make.
"Checking at runtime* is exactly the problem. Why would checking at runtime be more reliable than compile time?
>"Checking at runtime is exactly the problem. Why would checking at runtime be more reliable than compile time?*
If you use metaprogramming (i.e. Lisp macros, Scheme/Racket macros, Clojure macros), then the only way to have truly reliable checks is at run time.
In fact, for debugging and testing, it's much better to have a runtime able to do many things (such as live patching of the code).
Contrary to a language like C or C++, where the compiler produces machine language from the source and then goes out of the picture, in most of the Lisp family languages your program runs along with the runtime; this runtime also contains the compiler itself and is usually big on features; it will not just catch the type errors, it will show you exactly all kind of important information about any, error and will allow you to correct the error if you like, without having to recompile and start again.
Contrary to a language like C or C++, where the compiler produces machine language from the source and then goes out of the picture,
I use C#. When you create a Linq expression, you get type safety and it "compiles to" an expression tree that is converted to its runnable form at runtime -- in my case a Sql statement, a MongoQuery, or C# code. It really comes in handy. Theoretically I could change my backing Store from Mongo/C# driver to Sql Server/EF without changing any of the queries. I do this all of the time when unit testing, I substitute an in memory list for a database table and still get complete type safety.
> If you use metaprogramming (i.e. Lisp macros, Scheme/Racket macros, Clojure macros), then the only way to have truly reliable checks is at run time.
That is a misconception. The macros output a target language and that can be checked. Plus macros can do their own checking before that. A macro expander can perform some static checks. In TXR Lisp, warnings about unbound functions and variables come from the macro-expanding code walk.
In addition to the points flavio81 made, checking at runtime because unless the language is dependently typed i.e. has types as first class citizens(for example Idris) a lot of the information about your code you want to prove to be correct is not available at compile time(for example, you have an integer but is it a non-negative integer, a list but is it sorted). It is not about being reliable, it is about being possible.
"Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactorings that simple cannot be done with a statically typed languages." -- not true, Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make.