I wish he would have commented on .NET's async implementation. Microsoft really got it right here and it's arguably the best implementation I've seen.
All .NET async APIs take an optional cancellation token parameter which solves his flow-control problem by allowing the async request to be canceled at any time. If the token is canceled, the async task will (or should) throw an OperationCanceledException which can then be cleanly handled in a standard try/catch block up the stack.
The best part about this is that it pervades the entire .NET runtime, the APIs, code examples, and has excellent documentation on correct usage patterns. Sure, a 3rd party library could choose not to support cancellation tokens, but they would be going against the entire .NET ecosystem by doing so. Every other async implementation I've seen has really seemed like a haphazard bolt-on.
I honestly don't know how I would write robust async code without cancellation tokens, so I guess he has a point when it comes to javascript and python ecosystems.
Cancellation and backpressure are different things. Backpressure is about not accepting work when there is not capacity to handle it. It is incorrect to accept too much work, hit overload, and respond to the emergency by throwing away partially completed work. That makes the overload worse by wasting effort.
the default backpressure response in the BEAM is that when you make a call to another actor, and your call times out because the other actor is too busy, the network temporarily is too closed, etc, etc, YOU die, with no intervention from the service you've called.
Often overlooked on the backpressure side of the .NET implementation fence is the SynchronizationContext/Scheduler system. Easily overlooked because of its very apparent complexity beyond the immediate "it just works" surface level in UI event handlers in WinForms/WPF, you can do all sorts of backpressure management with .NET's Schedulers, and because of the SynchronizationContext "magic" a lot of code doesn't change and doesn't have to have any idea about and can be entirely agnostic from whatever Schedulers that are driving it.
It's something that I'm both surprised and not all that surprised it hasn't made its way into more async/await languages exactly because of that confusing complexity.
I believe that in Python, due to its explicit over implicit philosophy, you could almost fake SynchronizationContext/Scheduler dynamics with a smart enough "event loop" as your outermost async runner, monkey patching coroutine return Futures/Tasks to include CurrentSynchronizationContext/CurrentScheduler information. It even looks like at a shallow skim some of that is designed into the default asyncio event loop system in the form of "custom event loop policies", but in the same skim it seems like its both more complex in its own way than SynchronizationContext/Scheduler (such as its primary job seems to be in handling platform specifics), and cursory searches don't show anyone attempting to use it for backpressure management.
Not that in most cases I would write backpressure management code directly in SynchronizationContext/Scheduler "magic" today: I'd use ReactiveX glue and explicitly define Schedulers there and/or make use of "off-the-shelf" RX backpressure combinators. But the low level capability at the .NET SynchronizationContext/Scheduler level is an often overlooked "magic" to async/await that can get results from existing async/await code without needing to rewrite existing async/await code. (I think the fact that RX.NET itself can piggy back off of that low level most directly as opposed to in other languages/environments implementing its scheduler support and backpressure work with less low level underlying support is also easy to overlook that .NET seems to be doing it all more cohesively under the hood.)
True, but cancellation tokens allow you to alleviate backpressure by safely dropping in-flight requests once you hit a given threshold instead of adding to an unbounded scope.
How's it different than Go's context.Done() channel? Similarly it bubbles up everywhere the context is passed immediately, letting you terminate concurrent routines.
I've used both .NET async/await (4 years ago) and Go ever since. The semantics and feel is nearly the same, the look is different.
IMHO, being able to throw the error and it moving magically up the async stack (aka try/catch) makes it easier to get started while go makes it easier to maintain by forcing you to design a system around channels. This opinion is coming from a decade of .NET but just a few months of go experience so it's very skewed.
> All .NET async APIs take an optional cancellation token parameter [...] The best part about this is that it pervades the entire .NET runtime, the APIs, code examples, and has excellent documentation on correct usage patterns.
This statement is completely false. The .NET runtime is inundated with bad code examples resulting in deadlocks with async code.
In my professional experience, the WriteableBitmap class is completely unusable. All official code samples exhibit the same brokenness.
They all tell you to do the same thing: lock() on the WriteableBitmap object, get the buffer, perform an update the buffer, unlock() on the WriteableBitmap object. But you try to lock() on a backgroundworker (the sane thing to do) and it fails because lock() can only be called from the UI thread. So you lock() on the UI thread, async into a backgroundworker (the second most sane thing to do) to update the buffer, and boom, WPF deadlocks. The official Microsoft UI framework uses .Lock(), not .TryLock(), and you are deadlocked forever. There is no solution to this problem. No solution exists. Abandon all hope ye who enter here.
C#/.NET+async is horrible. Maybe this is the best async implementation that's been done, but if that's the case, the best implementation is worse than starting a new thread every time you need to block on IO. The only way to write good async C#/.NET code is to not write async C#/.NET code. But guess what? As you said, C#/.NET is inundated with async APIs, and it falls down every time you blink at it.
C++ is looking to standardize async/await into the language. I think I'd rather pick up growing corn.
> But you try to lock() on a backgroundworker (the sane thing to do)
That’s just a leaky abstraction. Updating image from background thread is a sane thing to do from general-purpose programmer POV. To understand why it’s not so good idea, and why it’s not supported, you need to know what happens under the hood. Specifically, how 3D GPU hardware works and executes these commands.
> No solution exists.
From graphic programmers POV, the sane solution — only call GPU from a single thread. In D3D11 there’re things which can be called from background threads. It’s possible to create new texture on the background thread uploading the new data to VRAM, pass the texture to the GUI thread, then on the GUI thread either copy between textures (very fast) or replace textures destroying the old one (even faster). Unfortunately, doing so is slower in many practical use cases, like updating that bitmap at 60Hz: creating new resources is relatively expensive, more so than updating preexisting one.
How are you "asyncing" to the "backgroundworker" thread? Await? Task.run? Are you using a .Return?
I can't see your code, but this sounds more like a misunderstanding of the requirements around a UI thread and the task scheduler than any issue with async.
First of all, your original post claimed that the official documentation is good. I posted an example of where the official docs are dumb and wrong. You posted a random blog article and a stack overflow answer as a rebuttal. Just so we're clear, you agree that the official documentation is dumb, wrong, and incomplete?
Second, the "seems to be correct" article you linked is what we did. It's a sensible, logical solution to the problem. Which is why we wrote it that way to begin with. Hell, we might have copied it directly from that SO answer.
It doesn't work. It deadlocks. You call out to the UI thread dispatcher, lock the object, then go back to doing stuff on a background thread. Some time later, the UI thread is updating the UI, and it tries to update your element, and it tries to lock on the lock object. (it can't, because it's already locked) Some time later, the background task is done. It schedules a task on the UI thread to unlock the object. There's a problem though: the UI thread is locked on a mutex. A single hardware thread cannot wait on both a mutex and the task queue at the same time. Deadlock.
Just because it's the highest voted answer on SO doesn't mean that it isn't completely wrong.
The second highest voted answer states that what the questioner is asking for is impossible. This answer is the correct one. Our least bad solution to this problem is to update a buffer owned by the background thread, and then use the UI thread to copy pixels from the background thread's buffer to the UI thread's buffer. Lock, copy the buffer, unlock, without ever relinquishing control of the UI thread. This solution is not ideal from a performance standpoint, but at least it doesn't deadlock.
C#/.NET's async is fine if you're not doing anything more complicated than querying a database or paying data around on a socket. It falls down really hard if you start tickling its underbelly.
Python has something similar, you can cancel() any async task and it will throw a CancelledError inside the ongoing operation.
I'm not sure how it would solve backpressure though, because canceling a task is normally done by the caller, while backpressure comes from the other direction of the call stack.
Cancel in Python and a cancellation token are very different concepts. Cancellations in Python are not passed through the same way and you can’t await a cancellation.
Yeah I agree with yourself and the the writer that cancellation is necessary.
I’m not sure I agree with the writer that async should handle back pressure. To me that should be handled more explicitly by something purpose built, e.g .NETs Channel type.
IMHO all the rage on async comes from an era when interpreted languages where the main trend as they demonstrably increased developer productivity --I'm looking at you, Rails, Django. Those platforms were not designed for runtime speed, so it made sense to push the bottleneck to the most inmediate backend system in the chain (i.e.: your trusty database, which is much faster than your agile framework of choice, remember the discussions of ORMs versus pure SQL?)
Then there came async frameworks a la Node, Twisted et al. and changed everything. Again in my opinion, async code is harder to reason about versus synchronous code.
Things to keep in mind:
- Are you really working at scale? Does the arguably added complexity of async benefit your particular use case? Specially when SPA technologies allow to build simpler backend for frontend systems (pure API, no HTML rendering). And not only regarding pure operational performance, Rails is still impossibly hard to beat when it comes to productivity.
- New players like Go, Rust have async capabilities but you dont necessarily need to use them to perform closer to native speed, hence becoming simpler solutions than Node, Ruby, or Python. Guess that also applies to old dogs with new tricks in the JVM (Micronaut, Quarkus...)
> Then there came async frameworks a la Node, Twisted et al. and changed everything. Again in my opinion, async code is harder to reason about versus synchronous code.
One change here which has also made this less important is the trend toward serverless/microservice architecture. In a world where you were running your MVP on a single VPS instance somewhere, and you were running a monolithic webserver which handled everything from request parsing to business logic, then moving from one-thread-per-request to a runloop implementation represented a potentially huge win in terms of performance.
But nowadays we largely work with more atomized bits of logic: business logic is implemented in terms of small, self-contained operations, and the problems of scale are delegated to some other system or orchestration framework. In that world it matters much less if your database access is blocking a thread or not, because throughput issues can be solved at a different level.
> IMHO all the rage on async comes from an era when interpreted languages where the main trend as they demonstrably increased developer productivity
I disagree.
Although it supported (cooperative) multithreading, Windows 3.1 applications typically had one thread that ran a message loop, invariably written in C or C++. Same with MacOS applications since 1984, except at first it was Pascal and 68000 assembly. CICS was originally single-threaded, running transactions written in System/370 assembler and later COBOL, until they bolted multithreading onto it in order to take advantage of the new SMP mainframes IBM was building. thttpd is from 1998 and is of course written in C; see http://www.acme.com/software/thttpd/benchmarks.html. One of the biggest changes in web-server software over the last ten years has been the move away from Apache to nginx, which is of course event-driven and written in C.
> async code is harder to reason about versus synchronous code.
Well, this is surely true. But generally the choice isn't async code or sync code; it's async code or multithreading. And async code is much easier to reason about than threads-and-locks code.
When you can hack it, a better approach is multiprocessing, also known as message passing, where each thread has its own private memory space, so it looks just like sync code. That's how Unix was built (though not just one but several broken threads models got bolted onto it later), it's how Erlang works, and it's the model adopted for JS in Web Workers. It has some performance cost. (Transactions, as in CICS or the Haskell STM, are another appealing option.)
Rust offers a really interesting alternative here: it statically proves that it's impossible for multiple threads to share access to writable memory (except when you cross you fingers and say "unsafe"), so in effect the threads are compiler-enforced multiple processes, but with the ability to dynamically freeze data so you can start sharing it between those processes.
From my point of view, what happened is that preemptive shared-memory multithreading with locks became a lot more popular in the 1990s and 2000s, driven by a rise in popularity of two systems originally developed for microcomputers without memory protection: Win16 (later Win32, Win64) and Java. Real-time control systems like the THERAC-25 control system have always used preemptive shared-memory multithreading, invariably with locks. (In the minimal case they use interrupt handlers rather than full-fledged multiple threads, but those have the same issue.) With the advantage of historical perspective, we can now see that it was a mistake to extend that programming paradigm to things like GUIs and web servers, but to some extent we're stuck with it for legacy systems like the JVM, Win64, and pthreads.
For new systems, though, we can use async, or multiprocessing, or transactions, or Rust's borrow checker, or a mixture.
This is a great point to observe, thanks. My commentary is biased towards web development
> But generally the choice isn't async code or sync code; it's async code or multithreading.
So true! years ago I worked with CORBA on top of ACE's libReactor and the result had an interesting mixture of the worst of the worlds of async and threads-and-locks ;)
As for web development, JS has been async since day 1. For a little while Opera was multithreading it, but that was really hard to program on, because they didn't even offer locks. And Perl was not normally multithreaded, but before POE it wasn't async either, but rather multiprocess. msql was async, as was FastCGI, originally. So I'm not sure I agree even within the scope of webdev history. Maybe you got started late when lots of people were already using Java?
Sure! I got fed up with all that multithreading Vietnam and cut my web teeth with Rails in '05 and as such my focus has been on the business side of the applications you can build on top of a lot of software pieces that may or may not be async/MT or whatever.
For example we ran a lot of Rails code on top of FastCGI with lighttpd but as far as I can remember the Rails code we wrote wasnt affected by the fact that FastCGI was async.
I wonder if the advent of async/await in all the popular and upcoming programming languages will be seen, after some (buffering) interval, as a disaster of major proportions. And, I wonder how the programming world will respond to the disaster.
It seems advisable to begin that response now. The linked article might be the beginning of such a response, but it seems too tentative. We may need an Iron Law of Flow Control, visibly acknowledged and observed in each system that uses an async/await facility, at the point of use, or a note explaining where it is handled farther back up the chain.
TCP vs IP is an excellent example of such an alternative: IP does not bother with buffering, except as a completely local performance optimization, and happily drops packets at the first hint of trouble, assured that somebody closer to the source has buffered copies of whatever they actually care about.
The fundamental problem isn't async code, it's buffering. Async-await patterns simply shift scheduling from the system (kernel or low-level runtime) into the application, but that's it. Languages like Python and JavaScript provide broken async-await APIs that do too much hidden buffering, including unbounded buffering. But that's a stupid design decision on their part, a reflection of poor interface design arguably stemming from a) bolting async I/O onto pre-existing architectures and runtimes that can't accommodate it well, and b) the implementers not having sufficient real-world experience designing and implementing such systems.
But even in threaded code you can have buffering problems. For example, Linux' default socket buffer size sysctls are arguably far too large, optimized for high throughput of a few connections (i.e. benchmarks), not for good responsiveness. That makes it too easy for any massively concurrent service daemon, threaded or not, to flood the outbound network queues.
The answer for any I/O model is to never, ever implement an in-memory input or output sink with an unlimited size, whether for buffering data, connections, or any other resource (e.g. stack frames if an event loop recursively reenters itself). That solves most of the problems right there. The remaining problems are more nuanced, and may require reducing buffer sizes as much as possible (think single-digit Kb instead of Mb[1]), and considering how various resource buffers and APIs interact (additively, multiplicatively) to create hidden buffer bloat.
[1] Especially if you're not doing any heavy processing of the buffer contents, such as audio-video processing where you may need moderately larger buffer sizes to benefit from pipelined and SIMD processing.
> I wonder if the advent of async/await in all the popular and upcoming programming languages will be seen, after some (buffering) interval, as a disaster of major proportions. And, I wonder how the programming world will respond to the disaster.
I think that there is value in async/await as a concept, but that some language's specific implementations will be eventually seen as a mistake. My primary experience is in .Net, and I've grown disenchanted with async/await in that realm. I feel like it makes the easy things easier and the hard things harder[1], which, IMO, is the wrong trade-off for a concurrency primitive[2]. By comparison, what I've read about async/await in Rust seems very promising.
[1] Specifically, I think that the Task API is warty and over-complicated, with abominations such as .ConfigureAwait (I know that there are situations where it can be useful, but those don't make the method any less terrible). While await doesn't strictly depend on Task[1a], in practice you're going to be using Task.
[2] Before anyone chimes in with "async/await has nothing to do with concurrency/threads---it's just state machines": yes, that is technically true in the narrow sense of how the compiler desugars the async/await keywords. However, as soon as you start talking about actually scheduling and executing your tasks, concurrency/threads become very important.
Threads are state machines. Packaging up task state as a closure and packaging up task state as a stack do roughly the same thing. In Go, the two are closer together than in other languages where async was a retrofit.
Sure, but if you're going to go that route, why not just say "the entire computing system at a whole is just a state machine, so what's the difference"?
Threads provide a very privileged form of state machine that are tightly coupled with the underlying platform. This means you have to be thinking about things like cache coherency and context switches. Without careful use of other synchronization primitives (or stronger language guarantees[1]) it's easy to find yourself writing incorrect or less-than-performant[2] code.
[1] Like in Rust
[2] e.g. creating bottlenecks by blocking on access to a shared resource (Edit: perhaps a better way to word that would be "accidentally reintroducing blocking behavior by using synchronization primitives to access a resource")
async/await in .NET is actually good because they added async equivalent APIs to nearly all blocking operations. There's also always (from what I've seen) an optional cancellation token parameter that allows an easy mechanism to alleviate backpressure.
Need an easy way to cancel 100 async requests? Easy, just pass in a cancellation token and cancel it when needed. All the requests will throw an OperationCanceledException which can be cleanly handled.
I haven't seen anything this nice in javascript or python.
> async/await in .NET is actually good because they added async equivalent APIs to nearly all blocking operations
I mean... I get it---it's great how Microsoft put in an effort to add async equivalents to most of your standard operations. But my complaint is about how almost all of those async methods are built upon Task/Task<T> (which, IMO, is a flawed foundation).
> There's also always (from what I've seen) an optional cancellation token parameter that allows an easy mechanism to alleviate backpressure.
> Need an easy way to cancel 100 async requests? Easy, just pass in a cancellation token and cancel it when needed. All the requests will throw an OperationCanceledException which can be cleanly handled.
No significant disagreement from me there. It is nice that most core async methods support (or try to support) cancellation.
> almost all of those async methods are built upon Task/Task<T> (which, IMO, is a flawed foundation)
I like them, very flexible. It’s easy to implement these tasks on top of pretty much anything.
Here’s how I have implemented back pressure when I needed something similar: https://github.com/Const-me/pCloud/blob/master/pCloudCore/Ut... That class is very generic, knows nothing about the nature of the tasks, yet it’s able to do the right thing. Specifically, it limits count of pending tasks to a small number (8 by default), when exceeded the `post` method will start to “block” in the async way, that’s why it returns `Task<Task<TResult>>` instead of just `Task<TResult>`.
Implementations as in .Net implementations? Or implementations as in other languages' implementations?
As far as .Net goes, I've not looked around. I feel like it would be somewhat of a fool's errand to develop an alternative async/await implementation because almost all of the core APIs use Task/Task<T>, so you'd either be using Task/Task<T> anyway or re-producing a lot of APIs.
async/await itself is "subtly duck typed" in most of the .NET compilers along with other bits of compiler work (among other examples IEnumerable/IEnumerable<T> in C#'s foreach is also subtly duck typed and not dependent on the specific BCL interfaces, though almost no one would use bespoke interfaces), which is how .NET Core was relatively easily able to add ValueTask/ValueTask<T> and a lot of .NET Core APIs have been slowly migrating to ValueTask/ValueTask<T> without much compatibility breaking trouble.
I actually liked .NET async/await because it came with good usable libraries for async RDBMS operations tied into NT IO completion ports, easing a super common bottleneck; compare to other common RDBMS clients that might offer a facade of asynchrony but are really just blocking and queuing behind the curtain.
The task library (TPL) is a smidgen wordy, as was MSFT's way about ten years back, most folks will never get deep into it though.
TCP is not based on UDP, so that would not make sense at all.
The parent describes it quite accurately: IP makes no guarantee on delivery and allows dropping packets, TCP promises to fix that on a higher level for applications that need it.
Hardly anybody relies on any of the extras UDP provides, so they are typically just wasted overhead. But user-level libraries offer UDP, so that is what people use.
It adds source and destination ports and a checksum. Computing the checksum is a nuisance, and it is usually inadequate, so it is allowed to be zero in IPV4, but not IPV6.
Normally, if you need a checksum, you need one that cannot be forged, and you need more than 16 bits. So the L3 one is just overhead.
I have never seen an actual use of UDP that looked at the port numbers, although firewalls often use them for filtering.
Name a few UDP-based protocols whose implementations do not listen to only a specific port number, or a rely on port numbers established as part of the protocol flow, then?
This is a weird thread. I too would be interested to know about these uses of UDP that don't use ports to post data to processes (or even traverse NAT I guess?).
It's the word "buffer" that threw me. Like, the buffer is a concept on the sending or receiving system and a separate concern from the protocol. That said I am certainly open to learning something new about TCP/IP today.
The buffer you're thinking of is a packet-level queue to allow mismatches between physical transmission rates and processing rates, and is separate from the protocol.
The buffer the original comment is talking about is likely the TCP receive buffer, which is used both to allow the application layer to process bytes at its leisure rather than as they arrive, and to allow out-of-order data to be reassembled correctly rather than retransmitted in-order every time.
The receive buffer is documented at the protocol level. Well, honestly I don't recall if the buffer is ever documented in the spec or just common implementation practice. There is a documented "receive window" though, which is how many bytes the receiver is comfortable having "in flight" - sent but not acknowledged. The existence of this window implies the existence of a buffer at least as large as that window on the receiver.
> In most async systems … you end up in a world where you chain a bunch of async functions together with no regard of back pressure.
yup. Back pressure doesn’t compose in the world of callbacks / async. It does compose if designed well in coroutine world (see: erlang).
> async/await is great but it encourages writing stuff that will behave catastrophically when overloaded.
yup. It’s very hard, in larger systems impossible, to do back pressure right with callbacks / async programming model.
This is how I assess software projects I look at. How fast is database is one thing. What does it do when I send it 2GiB of requests not reading responses? What happens when I open a bazillion connections to it? Will a previously established connections have priority over handling new connections?
I really wish we had more control over the scheduling of async tasks.
For a javascript example I ran into recently, say I am firing off a fetch for each image that comes into view in a large gallery. If I suddenly scroll down to the 1000th image, a naive implementation might fire off 1000 fetches for all the images we scrolled past. Then you'll be waiting a long time before the images in your current viewport is loaded.
Backpressure can save you a little bit here. Say you do the semaphore trick mentioned in the article and only allow a max of say 10 fetches in flight at once. Then if you quickly scroll through, all the subsequent fetches after the initial should fail, including the ones at the viewport you stop at. But since the queue is short, when the images in your current viewport retries it should now succeed.
This works but it isn't ideal. Ideally I would be able to just reprioritize the newer fetches to be LIFO instead of FIFO. Or maybe inspect what's currently queued up (and how big the queue is) so I can cancel everything that I don't need.
The backpressure solutions might just be a symptom of async tasks not being controllable in any way once started which is why you're forced to commit to it or not from the start even if that might not be the best point in time to make that decision.
Just load all images. No JavaScript! Let the browser handle it. I hate when web sites only display 1-3 pages worth of content and unload/load more when I scroll. The JS code for that uses more resources then pulling everything would. It's a user nightmare, where I cannot use "find" and I can't zoom out, or scroll fast. The browser can easily handle 10,000 DOM elements. There is already optimizations in place in browsers which solve render issues. Beating the browser at it's own job will require a tremendously amount of work. These problem was already solved when computers only had 256MB of memory. And your web site would be blazingly fast today if you did not complicate things.
So no, browsers don't have these problems solved out of the box. And it seems like what they are implementing will suffer from the exact problem I described here too.
I think you might also be thinking of bad implementations of infinite scroll which isn't what I am talking about. Have you never use something like photos.google.com? Scrubbing/scrolling to an arbitrary point in time is honestly a great user experience. Would you rather wait hours/days for a webpage to load all the past photos images you've ever taken? Or have to deal with pagination and click through hundreds of pages to find what you want?
I've found a hybrid model to be generically useful. Keep the first ~10 calls in a FIFO queue, and if that queue is full put additional calls into an LIFO stack that functions as overflow storage.
You can even reuse the same data structure and just flip the behavior under load, what's important is having FIFO behavior normally and then LIFO behavior when congestion occurs.
So I entirely agree with you. Having some sort of control over the scheduling and execution of async tasks is quite important. And every time I try to implement it in, say, Typescript, I realise that I'm reimplementing Erlang/BEAM.
I truly believe BEAM's runtime has a million good ideas in this space that we should be stealing!
If your answer to a JavaScript control flow problem is "fix it with a timer" it's almost never what you want to do. Checking every 250ms just makes things feel delayed for users on fast systems (particularly after a large scroll) and slow for users where 10 images won't load in 250ms while also wasting resources when there is nothing to do.
Yeah, no. I would use a queue. Capture the picture-enter-viewport event and then 50ms later (sorry I said 250... probably too long) verify it's still in the viewport like in the case of fast scrolling. Denounce so you don't fire timers for events close together. Works cross browsers, doesn't waste resources, will be smooth.
That's not a queue it's just a delayed async call (i.e. if 5,000 calls ask to be called they will all go without queuing in 50ms), what the other comment described is a queue.
50ms is way too short, a user on a 1080p screen would need to scroll at 360 pixels per second to avoid loading every image along the way. Even trying to do that on purpose is hard as hell https://clickspeedtest.com/scroll-test.html. Guaranteed that every mobile user is going to be loading every image on scroll over 4g as well creating the original back pressure issue now with 3 more frames of lag.
"smooth" does not mean "check every 3 frames" it means schedule things in a way they can happen immediately when the browser is ready, not your timer.
Edit: I rewrote my comment because I misunderstood yours.
You could tie it to an "on-scroll" type of event, like how many text boxes will fire an event X milliseconds after a text edit event, so that they can process the text when the user is not likely to be typing.
A single timer by itself won't busy-wait, but I agree that running a timer all the time is a waste. Just run it when scrolling, when CPU use will already be high.
Is there really no way to determine if an image has left the viewport? This is a solved problem on other platforms: you just cancel image fetches if the image has scrolled out of the viewport. No timers needed.
See the second link in my comment (https://developer.mozilla.org/en-US/docs/Web/API/AbortContro...). Determining if the image is in the viewport is easy, cancelling the async Fetch request after it's been made is an experimental API so not finalized/available everywhere.
There's an IntersectionObserver API that does exactly this - it fires your callback when your configured threshold (X% of element A is intersecting element B) is passed in either direction.
Curio is more like an experimental sandbox for its author. Trio was created as a production-oriented application of similar ideas. There are some places where Trio diverges from Curio, and I think Trio's choices are improvements there. For example, the handling of schedule points and cancel points (https://trio.readthedocs.io/en/stable/design.html#cancel-poi...).
> So why is write not doing an implicit drain? Well it's a massive API oversight and I'm not exactly sure how it happened.
Separating these two operations allows code to use multiple write() calls to build up a single record atomically before yielding control to the system, where some other task might also write to the same stream. This reasoning is only valid if the program is running on a single thread, but that’s a reasonable architecture decision for many programs.
Not a super great argument. It would be easy to just build up the data to be written then write it in 1 call, while at the same time preventing people from being able to use it incorrectly at all.
Couldn't you have an API with a write() return value that you can await to trigger a drain(), but which you can also leave unused, in which case it acts like the current write()?
Why is Go mentioned here? AFAIK, Go's goroutine makes async not async like in NodeJS async sense, but just lightweight user space threading, so just blocking like regular threading.
Ideally async and backpressure will advance to make better use of scheduling, prioritizing, quality of service shaping, termination conditioning, and the like.
As an example, there's a big difference between the async needs of one paying customer who's loading one medical chart web page vs. some free third-party web crawler trying a daily full text scan of an entire site.
Async with cost functions feels like a promising area for real-world use cases.
It seems to me that all that this boils down to is the fact that await/async makes back pressure something you need to deal with explicitly. Having the default be buffering isn't ideal but since each application will have their own idea of what to do with backpressure it'd be hard to figure out a different default that works better.
In any case all this can be solved without major rewrites by making sure every awaitable is awaited at some point in the future. Instead of doing this:
while connection.accept():
handle_connection()
you might do something like this:
connection_pool = []
while connection.accept():
connection_pool.append(handle_connection())
if len(connection_pool)>=MAX_CONNECTIONS:
await wait_any(connection_pool)
And there you go. Anytime there's more than MAX_CONNECTIONS the program stops accepting new connections, providing back pressure. It's more code but it's also defining exactly HOW to provide back pressure. Your specific use case might warrant providing back pressure not based on connection count but cpu usage or average response times. You might want to have a single global maximum connection count instead of one per thread. All of these aren't much more complex than what I've shown above as long as you keep the cardinal rule in mind: any awaitable MUST be awaited.
And in fact in python -- and other programming languages too I bet -- you get a warning on exit if there are any awaitables that never got awaited. In my opinion that should be an error. I can't think of a single scenario where it'd be proper form to never await an awaitable. You might await immediately, sometime in the future, or at the end of the program. But you never don't await at all.
Threading isn't any easer, or harder. If you spawn a thread for each connection you run into the same issue as await unless you do something about it. If you use a pool of threads you get backpressure for free, but the same goes for a pool of awaitables!
tl;dr: async/await has the exact same problems as threads, the tooling around async/await is just less mature. Rewriting async code to provide back pressure is near trivial.
My main problem with async is that it's much harder to build a mental model of what exactly is going on. Admittedly, I only had to deal with existing code so far doing only minor work in it so you can definitely say it's due to lack of experience. However it seems that while problems like concurrent access to shared memory known from traditional threaded code don't exist, having to think about what can block etc. is of equal burden.
How do actor model languages (like ponylang) handle this? It seems like not having back-pressure support would be a fundamental issue with the language.
It seems like the gist is that the responsibility is on the application to be architect-ed (perhaps using one of the various libraries/strategies) to handle flow-control/back-pressure. The language itself doesn't directly help.
FWIW, having climbed Twisted's learning curve long ago the current async-all-the-things fad looks so childish to me. I remember when Tornado came out and I was like, why would you use a go-kart when there's a free Maserati right there?
Doesn’t solve the issue with calling an api that doesn’t return a promise, but the tslint rule “no-floating-promises” flags missing awaits which is quite useful. Also useful is flagging awaits that don’t return a promise, which can be done with the tslint rule “await-promise”.
I imagine he means without having to understand/use the platform or framework equivalent of select/poll or start dedicated threads or write a threadpool to handle reading from streams simultaneously without blocking. Now he has someone else wrote that code for him ;)
All .NET async APIs take an optional cancellation token parameter which solves his flow-control problem by allowing the async request to be canceled at any time. If the token is canceled, the async task will (or should) throw an OperationCanceledException which can then be cleanly handled in a standard try/catch block up the stack.
The best part about this is that it pervades the entire .NET runtime, the APIs, code examples, and has excellent documentation on correct usage patterns. Sure, a 3rd party library could choose not to support cancellation tokens, but they would be going against the entire .NET ecosystem by doing so. Every other async implementation I've seen has really seemed like a haphazard bolt-on.
I honestly don't know how I would write robust async code without cancellation tokens, so I guess he has a point when it comes to javascript and python ecosystems.