Windows - Python 3.8+ raises "RuntimeError: Event Loop is closed" on exit.

[Kludex]

Originally opened by @rmawatson on 2020-04-30 13:15:36 in encode/httpx

Checklist

• [ x] The bug is reproducible against the latest release and/or master.
• [ x] There are no similar issues or pull requests to fix it yet.

Describe the bug

I get the following traceback when running the attached code

404
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x0000017C63D5E310>
Traceback (most recent call last):
  File "C:\Python\Python38\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Python\Python38\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Python\Python38\lib\asyncio\base_events.py", line 715, in call_soon
    self._check_closed()
  File "C:\Python\Python38\lib\asyncio\base_events.py", line 508, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

To reproduce

async def testing():
    client = httpx.AsyncClient()
    response = await client.get("https://www.google.com/")
    await client.aclose()
    print(response.status_code)

async def main():
   await testing()

asyncio.run(main())

Inserting time.sleep(0.1) at the end of main() fixes it. Possibly due to the same reasons in the issues linked below.

Environment

• Windows 10 64bit
• Python version: 3.8.1
• HTTPX version 0.12.1

Possibly the same as this and this

[Kludex]

Originally posted by @florimondmanca on 2020-04-30T15:48:23Z

I assume this is yet another strange issue with asyncio’s stream.close() - it schedules the closing of the socket but it’s not guaranteed to be done once client.aclose finishes, which means the socket is actually deleted upon garbage collection, and by then I suppose the event loop is already closed... The sleep() call fixed this because it allows asyncio to execute the close callback before proceeding to shutdown the async environment.

See also #825 for more discussion on a possibly related issue.

Have you tried running this on Python 3.8.2?

[Kludex]

Originally posted by @tomchristie on 2020-04-30T15:52:00Z

Also: May well be specific to Window's Proactor event loop.

[Kludex]

Originally posted by @florimondmanca on 2020-04-30T15:59:00Z

Yes, definitely looks like a Windows specific issue to me - the linked issues are for Windows too.

[Kludex]

Originally posted by @rmawatson on 2020-04-30T19:48:14Z

@florimondmanca , yes upgraded to 3.8.2 to check, the result is the same.

[Kludex]

Originally posted by @Baysul on 2020-05-01T02:13:32Z

Can confirm this is an issue with the latest PyPi version of aiohttp and Python 3.8.2 on Windows.

[Kludex]

Originally posted by @florimondmanca on 2020-05-01T06:58:13Z

Okay, thank you.

I personally don’t have a Windows machine so help with investigation and possible fixes would help.

Would we need to consider black magic such as aio-libs/aiohttp#1925 (comment)? How about aio-libs/aiohttp#3733?

FYI this is likely to be something to address on the HTTPCore side, as connection management and networking is done there. Might then be helpful to first come up with a minimal repro example using HTTPCore only?

[Kludex]

Originally posted by @tomchristie on 2020-05-01T08:23:08Z

It'd also be helpful to try switching to using SelectorEventLoop and let us know what the behaviour is like then... https://docs.python.org/3/library/asyncio-eventloop.html#event-loop-implementations (Or to try with Python 3.7 or earlier)

Python only switched the default event loop to proactor from 3.8 onwards. https://docs.python.org/3/library/asyncio-policy.html#asyncio.DefaultEventLoopPolicy

[Kludex]

Originally posted by @euri10 on 2020-05-01T16:18:25Z

not sure there's a link and can't remember why now but in uvicorn we force
the SelectorEventLoop to be the default for windows

Le ven. 1 mai 2020 à 10:23 AM, Tom Christie notifications@github.com a
écrit :

It'd also be helpful to try switching to using SelectorEventLoop and let
us know what the behaviour is like then...
https://docs.python.org/3/library/asyncio-eventloop.html#event-loop-implementations
(Or to try with Python 3.7 or earlier)

Python only switched the default event loop to proactor from 3.8 onwards.
https://docs.python.org/3/library/asyncio-policy.html#asyncio.DefaultEventLoopPolicy

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/encode/httpx/issues/914#issuecomment-622297443, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAINSPTJI3OZEDU6VKCF2KLRPKBHTANCNFSM4MVRF2GQ
.

[Kludex]

Originally posted by @Baysul on 2020-05-01T22:05:52Z

The workaround is to use the selector event loop like this:

import sys
import asyncio

if sys.version_info[0] == 3 and sys.version_info[1] >= 8 and sys.platform.startswith('win'):
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

The following from the documentation will not resolve this issue (for some reason):

import asyncio
import selectors

selector = selectors.SelectSelector()
loop = asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop(loop)

[Kludex]

Originally posted by @rmawatson on 2020-05-02T14:31:15Z

Thanks @Baysul.

This bug report looks like it is tracking the same issue.

[Kludex]

Originally posted by @tomchristie on 2020-05-06T13:49:30Z

Same as with #825 - Haven't looked into how awkward it is to do, but really I think we'd want to monkeypatch asyncio's socket unwrapping behaviour, so that we can reliably call .wait_closed(...) without it potentially hanging.

[Kludex]

Originally posted by @tomchristie on 2020-08-20T12:09:09Z

Can anyone on Windows, running Python 3.8+ confirm what currently happens when running this?...

import httpx
import asyncio

async def main():
    async with httpx.AsyncClient() as client:
        r = await client.get("https://login.microsoftonline.com/")
        print(r)

asyncio.run(main(), debug=True)

[Kludex]

Originally posted by @dalf on 2020-08-20T12:25:53Z

Using:

• Windows 10 64bit
• Python 3.8.5
• httpx 0.14.1

I have a httpx._exceptions.ConnectTimeout even with timeout=60 or http2=True.

[Kludex]

Originally posted by @tomchristie on 2020-08-20T12:31:16Z

Uh... that's not at all what I was expecting / looking for.

Does that replicate on other URLs?...

• "https://www.google.com/"
• "https://www.example.org/"

Does removing the debug=True change anything?

[Kludex]

Originally posted by @dalf on 2020-08-20T12:48:40Z

Maybe my report irrelevant, I'm not familiar with the Python Windows environment:

import httpx
import requests

r_requests = requests.get('https://www.google.com')
r_httpx = httpx.get('https://www.google.com')

• requests returns a normal response
• httpx raises ConnectTimeout: _ssl.c:1106: The handshake operation timed out
• the aiohttp client example works (on https://www.google.com )