## Overview

We already know how ‹async def› works. What ‹asyncio› gives us is a
higher-level interface and a rich library of async-aware
functionality, especially for interacting with the OS and with
networks. What follows is a short summary of the basic elements of
‹asyncio›.

 1. The central concept is an «event loop» (which really just a
    coroutine scheduler, something we already know quite well),
    which can be:
    
     ◦ entered using ‹asyncio.run( main_coro )› (only one event loop
       can be active in a given thread at any given time),
     ◦ accessed using ‹asyncio.get_running_loop()›.

 2. Support for working with «tasks» – individual coroutines that
    the event loop / scheduler wakes up as needed (these essentially
    correspond to fibres). When doing low-level task management, it
    is customary to say ‹loop = asyncio.get_running_loop()› at the
    start, so that we can interact with the scheduler. Some example
    methods:

     ◦ the low-level ‹loop.call_soon( func, p₁, p₂, … )› will take
       ‹func› («not» a coroutine!) and run it as soon as reasonable
       from within the event loop, as ‹func( p₁, p₂, … )›,
     ◦ the somewhat higher-level ‹loop.create_task( coro )› will put
       the given coroutine on the queue and schedule it soon¹ – in
       this case, the coroutine can use awaitables as usual, and the
       event loop will take care of waking it up again as needed.

 3. A high-level task combinator ‹asyncio.gather› which packages
    multiple tasks into a single awaitable entity – awaiting the
    resulting «future»² will wait until all the components are done
    and then return results of all the components as a list. In many
    cases, this avoids the need to juggle individual tasks using
    ‹create_task› as above.

 4. A collection of low-level functions for OS/IO interaction (you
    don't need to be overly concerned with those; they are too
    low-level and callback-heavy for our purposes – the entire point
    of ‹async› coroutines was to get rid of these callbacks):
     
    a. «transports» which provide a callback-based abstraction over
       sockets and other «means» of transmitting data (‹asyncio›
       provides transports for TCP, UDP, TLS/SSL and subprocesses),
       and one level above them are
    b. «protocols» which add application-level logic on top of a
       transport (but the API is still rather low-level) – ‹asyncio›
       itself does not provide any implementations, only the
       plumbing, and we won't be using it directly.

 5. Streams provide an ‹async›-friendly abstraction over protocols:
    if you can have a stream for the IO you need to do, you get it.
    These provide natural ‹read›- and ‹write›-like operations and
    manage the complicated callback state machines for us.
    Everything that can block is directly awaitable.
    
    Most high-level APIs can directly give you streams to
    communicate with whatever is at the other end (TCP server, a
    client that connected to your application, a subprocess, etc.).

 6. Servers and clients, which encapsulate the details of
    establishing connections or listening sockets, and which
    generally give you streams to communicate with the other end.
    
    For servers, you generally provide a callback (justified, this
    one time) which is called whenever a client connects and which
    gets the two streams (incoming, outgoing) that have the client
    on the other end. This callback can be ‹async def›, in which
    case it is automatically passed to ‹create_task› from above, so
    there is no need to schedule anything by hand.

 7. A collection of other useful awaitables for cases where streams
    are not applicable (either because no persistent communication
    is happening, or streams are not the right abstraction), for
    instance:

     ◦ ‹loop.getaddrinfo› to resolve hostnames without blocking the
       entire application (DNS queries can have significant latency),
     ◦ ‹loop.sendfile› to ask the OS to send an entire file directly
       into a socket, without reading it into a user-space buffer if
       possible,
     ◦ ‹loop.sock_recv› to fetch data from a low-level socket without
       having to deal with the low-level callback mess,
     ◦ ‹asyncio.sleep› of course.

 8. Synchronisation primitives, for communication between multiple
    fibres (green threads). There are essentially two kinds, first
    the ‘low-level’ which you know from threads:

     ◦ ‹Lock› is probably the simplest primitive, which implements
       mutual exclusion (the ‹acquire› method can block, and hence
       must be ‹await›-ed, while ‹release› is a regular method since
       it always returns immediately),
     ◦ ‹Semaphore› is a generalized (counting) lock (again with
       ‹acquire› and ‹release›),
     ◦ ‹Condition› which allows a coroutine to wait until ‘something
       happens’ as signalled by another coroutine (both coroutines
       usually in your management – the event loop has better
       internal mechanisms to block and wake up coroutines).

    And high-level «synchronized queues», which can be used for
    sending information between parts of the application, much like
    streams are used for sending or receiving data to/from external
    sources. Importantly, any Python objects can be queued, not just
    bytes like with streams.
    
     ◦ ‹asyncio.Queue› which is a standard first-in, first-out queue
       which has a blocking ‹get› (it has to be awaited, and waits
       until an item arrives if the queue is empty) and
       possibly-blocking ‹put› (if the queue has a maximum size,
       otherwise it never blocks, but of course must still be
       ‹await›-ed),
    ◦ ‹asyncio.PriorityQueue› same, but the high-priority items are
      returned first.


¹ Indeed using ‹call_soon›. See ‹Task.__step› defined in
  ‹asyncio/tasks.py› for details about how this works. The loop
  itself doesn't really know about ‹async def›: it just provides
  machinery at the level of ‹call_soon› which allows other pieces to
  block and unblock tasks. The piece that actually runs things
  (including the aforementioned ‹__step›) is
  ‹BaseEventLoop_run_once› defined in ‹asyncio/base_events.py›).

² Futures are a bit of an internal glue that is not overly important
  for us, since as far as ‹asyncio› is concerned, we prefer to stick
  to high-level APIs where we can use ‹async› coroutines and
  ‹await›. A future is more general than a ‹Task› (in fact, it is a
  base class of the latter). Futures are used extensively in the
  plumbing – consider e.g. ‹StreamReader._waiter› (in
  ‹asyncio/streams.py›) and its companion methods ‹_wait_for_data›
  (which ‹await›s the ‹_waiter› future) and ‹feed_data› (which, by
  calling ‹set_result› marks the future as done, causing
  ‹_wait_for_data› to resume from its ‹await›). Notice that futures
  are awaitable (you can look at the ‹__await__› method in
  ‹asyncio/futures.py›).
