Discussion

What Python's asyncio primitives get wrong about shared state

ydj: I think it’s not so much that the asyncio primitives got wrong about shared state, as much as is what the authors got wrong about the usage of those primitives. They are classic concurrency primitives that’s been around for almost half a century. They work as designed, but require some care to use correctly.

TZubiri: I'm sorry but how do you jump from 1. Polling to 2. AsyncioThere's so many solutions in the middle, I have this theory that most people that get into async don't really know what threading is. Maybe they have a world vision where before 2023 python just could not do more than one thing at once, that's what the GIL was right? But now after 3.12 Guido really pulled himself by the bootstraps and removed the GIL and implemented async and now python can do more than one thing at a time so they start learning about async to be able to do more than one thing at a time.This is a huge disconnect between what python devs are actually building, a different api towards concurrency. And some junior devs that think they are learning bleeding edge stuff when they are actually learning fundamentals through a very contrived lens.It 100% comes from ex-node devs, I will save the node criticism, but node has a very specific concurrency model, and node devs that try out python sometimes run to asyncio as a way to soften the learning curve of the new language. And that's how they get into this mess.The python devs are working on these features because they have to work on something, and updates to foundational tech are supposed to have effects in decades, it's very rare that you need to use bleeding edge features. In 95% of the cases, you should be restricting yourself to using features from versions that are 5-10 years old, especially if you come from other languages! You should start old to new, not new to old.Sorry, for the rant, or if I misjudged, making a broader claim based on multiple perspectives.

dbt00: I think they were already in the async world and needed message passing -- the polling code was also in python async.

dbt00: What about a more general message-passing mailbox approach? This works really well in the Erlang/gen_server/gen_fsm world. (and in plenty of other contexts, but Erlang's OTP is still some of the best, simplest incarnation of these things)

evil-olive: the title seems unnecessarily clickbaity.rather than "What Python's asyncio primitives get wrong" this seems more like "why we chose one asyncio primitive (queue) instead of others (event and condition)"also, halfway through the post, the problem grows a new requirement:> Instead of waking consumers and asking "is the current state what you want?", buffer every transition into a per-consumer queue. Each consumer drains its own queue and checks each transition individually. The consumer never misses a state.if buffering every state change is a requirement, then...yeah, you're gonna need a buffer of some kind. the previous proposed solutions (polling, event, condition) would never have worked.given the full requirements up-front, you can jump straight to "just use a queue" - with the downside that it would make for a less interesting blog post.also, this is using queues without any size limit, which seems like a memory leak waiting to happen if events ever get enqueued more quickly than they can be consumed. notably, this could not happen with the simpler use cases that could be satisfied by events and conditions.> A threading.Lock protects the value and queue list.unless I'm missing something obvious, this seems like it should be an asyncio.Lock?

rpz: Reminds me of https://geocar.sdf1.org/fast-servers.html

pothamk: One thing that trips people up with asyncio is that “single threaded” gets interpreted as “no concurrency hazards”.But coroutines still interleave execution at every await point, so shared mutable state can become just as fragile as in multithreaded code — the scheduling boundary just moves from OS threads to cooperative yield points.In practice that tends to push designs toward queues, actors, or message-passing patterns if you want to avoid subtle state corruption.

ipnon: And in that case you begin to wonder why use Python at all? The language struggles to give developers the granularity needed to finely manage threads like C++, and it doesn't have the actor model first class like Erlang. I love Python, but I love Fortran and Lisp too. They've all served their purpose and it's time to move on, even though there is already incredible momentum behind it.

ggm: yes. I felt something very similar. I do think there is value in pointing out the pitfalls naieve users (me!) can make assuming things which aren't true about ordering of events, states. Queues with lock regions are also really nice because they are (as I understand it) very cheap: so making a thread or other concurrency primitive which writes into a queue under lock, and gets out of the way, aligns nicely with having some mothership process which reads queues under lock in a deterministic way. Actual event order can vary. but you should be able to know you had an event putting you into state A, as well as the terminal event state B you jumped into without doing work needed for state A.

SkiFire13: I don't fully agree with this. Yes, surely shared mutable state can suffer from similar issues, however the cooperative nature of coroutines makes this much easier to handle. OS threads are preemptive and actually run in parallel, so you have to be aware of CPU concurrency and always be ready for a context switch.

jsanders9: Agreed. This isn't an asyncio problem, it's just not how those primatices work.

phs2501: The one thing I wish stock python queues had an option for (async or otherwise) was some kind of explicit termination. e.g. be split into producers and consumers, and have consumers indicate iteration complete when all producers have finished (and vice versa - signal producers that all consumers have gone away). You can kind of kludge around it in one direction with stop sentinals but it's a lot more awkward to deal with - especially if your queues are bounded as then you can get into the situation where you block trying to push the stop sentinal onto the queue as it's full.

matheist: Does task_done not do what you want?https://docs.python.org/3/library/queue.html#queue.Queue.tas...

foltik: It’s just some AI generated pseudo insight, look at the rest of their comments.

cyberax: Hard disagree. Co-routines are utter hell. They should have never become popular.With traditional locking, the locked segment is usually very clear. It's possible to use race detectors to verify that objects are accessed with consistent locking. The yield points are also clear.With async stuff, ANY await point can change ANY state. And await points are common, even sometimes for things like logging. There are also no tools to verify the consistent "locking".So I often spend hours staring blankly at logs, trying to reconstruct a possible sequence of callbacks that could have led to a bug. E.g.: https://github.com/expo/expo/issues/39428

ori_b: Async and await is manually scheduling threads. So, if you're quite careful about what functions you call, you can arrange things so that you don't get concurrency when you don't want it.Being careful about what functions you call is quite fragile and tedious, and doesn't compose well: what if a library changes when it adds a yield point?Overall, async/await is a result of people programming like it's 2003, when threads were still very expensive.

scuff3d: “The problem with most programming languages is that they implement concurrency as libraries on top of sequential languages. Erlang is a concurrent language at the core; everything else is just a poor imitation implemented in libraries.” -Joe Armstrong

itemize123: it does seem the user wants a conditional variable.For locking I am guessing they want multithreading, each with an event loop.

lukaslalinsky: They are trying to use condition variable without a mutex and see missed wake ups. That's a textbook error, no? I'm surprised asyncio.Condition even allows that mode of operation.

three14: The thing that burned me with asyncio primitives is that calling an async function doesn't even schedule it (by default). The pattern for fire-and-forget is impossible to guess when coming from any other language - although it's called out in the docs. You must also call create_task AND you must maintain a collection of tasks because otherwise the task might not run, because it can be garbage collected before running, AND you must clean up completed tasks from your collection.

whilenot-dev: > The pattern for fire-and-forget is impossibleGood, that's an antipattern in the coroutines concurrency model.

vlovich123: Threads are still expensive in Python - can’t use them for concurrency really like you can with async io afaik.

bootsmann: This is why we moved a lot of our concurrent python project to golang. There were a couple of cases where some engineer built the system by implicitly relying on the assumption that some coroutine would run blocking until a certain point was reached (avoiding a potential data race) that was then later broken by another change. At least in go we know we cannot rely on this so the concurrency safety has to be considered at all times, leading to better code.

mrkeen: This is one of those cases where software transactional memory really shines.You can often take the naive solution and it will be the correct one. Your code will looks like your intent.TFA's first attempt: async def drain_requests(): while state != "closing": await asyncio.sleep(0.1) print("draining pending requests") Got it. Let's port it to STM: let drain_requests = do atomically ( do s <- readTVar state when (s &#x2F;= "closing") retry ) print("draining pending requests") Thread-safe and no busy-waiting. No mention of 'notify', 'sleep'. No attempt to evade the concurrency issues, as in the articles "The fix: per-consumer queues - Each consumer drains its own queue and checks each transition individually."

josephg: This works a lot better in JavaScript, which is exactly this model - a single threaded executor with async await. The problem you talk about is solved with function colouring. Async functions are marked as such. In general, sync functions can’t call async functions. (Well, you can invoke them. You just can’t run them to completion before returning).For all the complaints about function colouring, I’m glad JavaScript has them. A sync function becoming an async function is a breaking API change. This is much better than the situation in Python, where yield points are invisible.

pocksuppet: Until every function is async.

rciorba: I mean, the "one-queue per consumer" they eventually ended up with, is basically an inbox that the sequential process reads from.

seanhunter: A better title would be: “Person who doesn’t know how to write state machines struggles to write a state machine”.In attempt 2 the old school C way of writing the state machine would work just fine in python, avoid a bunch of the boilerplate and avoid the “state setter needs to know a bunch of stuff” problem. Basically you make the states as a table and put the methods you need in the table so in python a dictionary is convenient. Then you have > def set_state(new_state): > state = new_state > events[new_state].set() Aaand you’re done. When you add a new state, you add an event corresponding to that state into the events table. If the stuff you would put into a conditional in set_state is more complicated, you could make a state transition method and link to it in the table. Or you could make a nested dict or whatever. It’s not hard, and the fact that the author doesn’t know an idomatic way to write a fsm definitely isn’t something that’s wrong with python’s asyncio and shared state.In general if you’re writing a state machine and you have a lot of “if curr_state == SOME_STATE” logic, chances are it would be better if you used tables.

samarthr1: BTW: what is a lock region?

nine_k: A part of code that executes between a lock as taken and released, I suppose.

nine_k: If a function is not pure, it very likely has to be async.

ggm: Yes, that's what I meant.

krasikra: This is essentially the same "lost wakeup" problem you see with condition variables in pthreads, but asyncio makes it more insidious because the single-threaded event loop gives you a false sense of safety. The per-consumer queue approach is really just reinventing what Go channels and Rust's tokio::sync::watch do at the primitive level — buffer transitions so observers never miss intermediate states.

darkhorse222: The event one seems perfectly fine with a dictionary or even a class that wraps your events and pairs with an event object subscribers can attach to with a singleton attribute.

pocksuppet: Which just brings you back to preemptive multithreading, but without being able to use all the cores you paid for.

josephg: Except nobody writes code like that. It would be horrible.Sync functions perform better than async functions, and they give more guarantees to the caller. They're strictly better, when you can use them.If you find yourself making everything async, your design is bad and you should refactoring your code.

scuff3d: As of 3.14 running without the GIL is optional, but the default still has the GIL in place. 3.13 had it as experimental, but not officially supported. 3.12 and back are all GIL all day.Python's asyncio library is single threaded, so I'm not sure why you are talking about threads and asyncio like they have anything to do with each other.Python has been able to do more then one thing at a time for a long time. That's what the multiprocess library is for. It's not an ideal solution, but it does exist.

TZubiri: > Python's asyncio library is single threaded, so I'm not sure why you are talking about threads and asyncio like they have anything to do with each other.Ok, not OS threads, but it de facto creates application&#x2F;green threads.>That's what the multiprocess library is for. It's not an ideal solution, but it does exist.Philosophical argument but, I'd say multiprocess is not python doing many things, there would be many python runtimes (each doing A Thing), and the OS would be the one doing multiple things &#x2F; scheduling.

scuff3d: It absolutely does not create green threads. Green threads can be preempted and switched by the runtime. Go does this for example.Python's asyncio is tasked based, the event loop cannot switch out a task until it reaches a yield point.Or in short, green threads like Go uses are preemptive multitasking, the task based model asyncio uses is cooperative. A CPU bound python task can block the event loop forever if it never yields, goroutines generally can't.It's not a philosophical question at all. A single python program can use the multiprocessing library to run multiple chunks of work in parallel. It's heavier weight thanks to the need to basically run a full python interpreter in an os process, but it provides the functionality. And the fact that it's scheduled by the OS is irrelevant. Plenty of languages use the underlying OS threading capabilities to manage threads instead of their own runtime. Both Java and C# for example (though I think Java is adding green threads now).You're conflating several different distinct ideas. It's a common mistake I see at work all the time. Took a good amount of reading for me to untangle them all.

cl3misch: Is this being downvoted because of the tone, or because state machines are unpopular&#x2F;inappropriate in this case?Genuine question, because this feels like a sensible solution to the problem as stated in the article.

three14: Then someone should really update the official python docs that explain the fire-and-forget pattern (https:&#x2F;&#x2F;docs.python.org&#x2F;3&#x2F;library&#x2F;asyncio-task.html#asyncio....)! I had a FastAPI server, and calling a particular endpoint is supposed to kick off some work in the background. The background work does very little CPU work, but does often need to await more work for several minutes, so it's a good fit for asyncio. How do you want it to be structured? (In other words, on the level of human requirements, it IS fire and forget.)

samarthr1: Isnt that a critical section?

whilenot-dev: Concurrency based on coroutines is making use of cooperative multitasking, leaving the concurrent execution of tasks up to the runtime. To elaborate a bit on what that implies, let me just ask you the following question: Is there something less cooperative than a task that doesn't yield its control back to the main thread?Regarding the fire-and-forget pattern I can think of at least two issues, let me illustrate them with the following example: import asyncio background_tasks = set[asyncio.Task[None]]() async def _run_a() -> None: # to illustrate issue A raise RuntimeError() async def _run_b() -> None: # to illustrate issue B await asyncio.sleep(1) print('done') async def main() -> None: # issue A: exceptions can't be caught try: task_a = asyncio.create_task(_run_a()) background_tasks.add(task_a) task_a.add_done_callback(background_tasks.discard) except RuntimeError as exc: print(f'Exception caught: {exc}') # issue B: task won't complete task_b = asyncio.create_task(_run_b()) background_tasks.add(task_b) task_b.add_done_callback(background_tasks.discard) if __name__ == '__main__': asyncio.run(main()) Feel free to comment out either task in the main function and observe the resulting behavior.For issue A: Any raised error in the background task can't be caught and will crash the running main thread (the process)For issue B: Background tasks won't be completed if the main thread comes to a haltWith the decision for the fire-and-forget pattern you'll make a deliberate choice to leave any control of the runtime up to blind chance. So from an engineering POV that pattern isn't a solution-pattern to some real problem, it's rather a problem-pattern that demands a reworked solution.> How do you want it to be structuredTake a look at the caveats for FastAPI&#x2F;Starlette Background Tasks: https:&#x2F;&#x2F;fastapi.tiangolo.com&#x2F;tutorial&#x2F;background-tasks&#x2F;#cave...Losing control of a background task (and therefor the runtime) might be fine for some demo project, but I think you'll want to notice raised errors in any serious production system, especially for any work that takes several minutes to complete.

three14: > Is there something less cooperative than a task that doesn't yield its control back to the main thread? Of course it does yield back to the main thread in my example, at each await point, just like any other cooperative task.In my case, I specifically want an independent execution of a task. Admittedly, it has to catch its own exceptions and deal with them, as you pointed out, because that's part of being independent.(Technically, in issue A it doesn't crash the running thread. The event loop catches the exception, but it complains later when the task is garbage collected. Issue B is fine for my use - when the event loop shuts down, it cancels remaining tasks, which is exactly right for my server.)

phs2501: Not really. It's certainly intended for the basic "fan out m tasks to n workers, and the fanout producer wants to know when they're all done" and can be abused for some more, but I don't think it does anything to help with the "consumer died, I want the producers to be able to know this rather than just continuing to push messages into a queue forever" case.I've written wrappers to handle things the way I want, but it always feels like a bit of a hack. (Usually I use a stop sentinal internally and reach inside to unbound the queue before I send it to avoid blocking). Just wish it were built in.