Lifting the Curtain on Asyncio Magic - Part 1
The new asyncio module introduced in Python3.4 is a nice addition to the Python standard library, especially when used with the async and await keywords introduced in Python3.5.
If you have read the official docs, you should hopefully be able to be able to write high performance network servers and clients with asyncio. But if something goes wrong, you may find it hard to debug. This is no reflection on asyncio; concurrency and IO are deceptively complex things, and asyncio is the point they meet.
I suspect that the problem with debugging asyncio projects is that async code doesn't run itself. You are dependant on a rather large body of code to run your code for you, and the distinction between your code library code is far more blurry than you may be used to.
Normally when trying to understand code in the standard library I would reccomend reading the source. The Python stand library code is on Github these days, which makes that super easy. For the most part, the standard library code is not too intimidating--it's written by Python developers like yourself, and I would absolutely encourage developers at any level to read through it some time. But the asyncio code is a different beast, reading that might leave you happy to consider it magic written by wizards.
Unfortunately this means that acquiring the mental model you will need to tackle bugs in your async code isn't easy. Which is why I have put together a working example of a network server using async techniques but without using asyncio or any other async framework.
The project is asyncchat which runs a chat server you can connect to with telnet. The server is implemented in a single file. My hope is that it is small enough to be easily digestible but complete enough not to leave too much as an exercise for the reader.
Because the chat server uses similar techniques to asyncio, if you can grasp how chatserver.py works, then you will have a good grasp on how async is implemented in general.
Running chatserver.py
The chat server code is Python3.6 only (for the love of f strings). To run it, check out the asyncchat repos and run the following command in a terminal:
python3.6 chatserver.py
In another terminal, run the following:
telnet 127.0.0.1 2323
The telnet commands works on Linux / OSX. On Windows you may have to open your favourite telnet client.
Run the telnet command again in yet another terminal, and you should be able to send messages back and forth between the two terminals.
If you want to test it over a network, you can run the server with the following:
sudo python3.6 asyncchat.py 0.0.0.0 23
Now you should be able to exchange messages over a network, by telneting to the servers IP address. It will also work over the internet, assuming you know how to set up your router to do that.
An asyncchat.py session with two telnet clients. The server can support a large number of clients without multiple threads or processes.
In Part 2
After I've received any feedback, I'll post about how asyncchat.py works and dissect it line by line. Please do let me know which areas I should focus on.