There's a difference between "running a task that waits for 10 seconds" and "scheduling a wakeup in 10 seconds".
The code for several of the languages that are low-memory usage that do the second while the high memory usage results do the first. For example, on my machine the article's go code uses 2.5GB of memory but the following code uses only 124MB. That difference is in-line with the rust results.
package main
import (
"os"
"strconv"
"sync"
"time"
)
func main() {
numRoutines, _ := strconv.Atoi(os.Args[1])
var wg sync.WaitGroup
for i := 0; i < numRoutines; i++ {
wg.Add(1)
time.AfterFunc(10*time.Second, wg.Done)
}
wg.Wait()
}
I agree with you. Even something as simple as a loop like (pseudocode)
for (n=0;n<10;n++) {
sleep(1 second);
}
Changes the results quite a bit: for some reasons java use a _lot_ more memory and takes longer (~20 seconds), C# uses more that 1GB of memory, while python struggles with just scheduling all those tasks and takes more than one minute (beside taking more memory). node.js seems unfazed by this change.
Indeed, looping over a Task.Delay likely causes a lot of churn in timer queues - that's 10M timers allocated and scheduled! If it is replaced with 'PeriodicTimer', the end result becomes more reasonable.
This (AOT-compiled) F# implementation peaks at 566 MB with WKS GC and 509 MB with SRV GC:
open System
open System.Threading
open System.Threading.Tasks
let argv = Environment.GetCommandLineArgs()
[1..int argv[1]]
|> Seq.map (fun _ ->
task {
let timer = PeriodicTimer(TimeSpan.FromSeconds 1.0)
let mutable count = 10
while! timer.WaitForNextTickAsync() do
count <- count - 1
if count = 0 then timer.Dispose()
} :> Task)
|> Task.WaitAll
To Go's credit, it remains at consistent 2.53 GB and consumes quite a bit less CPU.
We're really spoiled with choice these days in compiled languages. It takes 1M coroutines to push the runtime and even at 100k the impact is easily tolerable, which is far more than regular applications would see. At 100K .NET consumes ~57 MB and Go consumes ~264 MB (and wins at CPU by up to 2x).
Spawning a periodically waking up Task in .NET (say every 250ms) that performs work like sending out a network request would retain comparable memory usage (in terms of async overhead itself).
Even at 100k tasks the bottleneck is going to be the network stack (sending outgoing 400k RPS takes a lot of CPU and syscall overhead, even with SocketAsyncEngine!).
Doing so in Go would require either spawning Goroutines, or performing scheduling by hand or through some form of aggregation over channel readers. Something that Tasks make immediately available.
The concurrency primitive overhead becomes more important if you want to quickly interleave multiple operations at once. In .NET you simply do not await them at callsite until you need their result later - this post showcases how low the overhead of doing so is.
Consider ordering reusable chunks of your circuit as PCBs instead of iterating on entire boards. You can connect the chunks with breadboards or connectors or something.
https://github.com/augustoroman/sandwich is another approach for achieving a similar result. One nice aspect of writing handlers that return errors is that they are easy to test and clean to write.
Sandwich does very simple sequencing to achieve an understandable dependency injection mechanism that produces clear errors when something goes wrong, and it produces errors at setup time rather than run time.
It was an interesting mix of "let's do a cool thing" and "imagine what can be done with this data!" Early on someone mentioned that NYC doesn't actually know for sure where all of the fire hydrants are _actually_ placed, and it may be possible to automatically extract the locations from the collected data.
More obviously, looking for the address signs of addresses that are expected along a street can dramatically improve driving directions.
Of course, the biggest business value was being able to generate the actual, underlying street maps without having to purchase that from companies that had already digitized and driven the streets.
The complexity from generics isn't from the conceptual standpoint, it's from the resulting code standpoint... for the same reason that the ternary ?: operator is "too complex": it's really easy to say that `foo := cond ? 1 : 0` is better than the if/else alternative, but `foo := (a > (b.Active() ? Compare(b, a) : useDefault() ? DEFAULT : panic()) ? "worked" : "failed"` is a mess waiting to happen.
Same with generics. It's easy to point to simple examples. It hard to ensure that terrible metaprogramming monstrosities don't arise.
It's possible to write bad code with go as it is, but it's actually difficult to make such bad code inscrutable.
Maybe, but the solution to this is not to say: Only the compiler implementers are smart enough people to not create messy code with generics, only a few built-in data types are allowed to be generic and the rest will have to copy-paste implementations or use lose type information through `interface {}`
I'm actually quite impressed by that. I had looked into that a bit, but believed Ducktape & React to be too slow to be worth while. This is really interesting!
The code for several of the languages that are low-memory usage that do the second while the high memory usage results do the first. For example, on my machine the article's go code uses 2.5GB of memory but the following code uses only 124MB. That difference is in-line with the rust results.