Test runPyInMain intermittenly fails with python 3.11

[Shimuuar]

    runPyInMain:                            FAIL
      Exception: PyError (PyException {ty = "<class 'AssertionError'>", str = "TID[main]=<_MainThread(MainThread, stopped 134745916688064)>, TID[our]=<_DummyThread(Dummy-1, started daemon 134747432908480)>", exception = PyObject 0x00007a8d54456ce0})
      HasCallStack backtrace:
        collectBacktraces, called at libraries/ghc-internal/src/GHC/Internal/Exception.hs:169:13 in ghc-internal:GHC.Internal.Exception
        toExceptionWithBacktrace, called at libraries/ghc-internal/src/GHC/Internal/IO.hs:260:11 in ghc-internal:GHC.Internal.IO
        throwIO, called at ./Control/Concurrent/Async.hs:78:13 in tasty-1.5.4-60f296b98d16db3ee7a66636972495285cfb5c9e6077ddedaa9e0e3f6bc142cf:Control.Concurrent.Async

It happens with python3.11 relatively frequently (~10% cases) and doesn't happen with 3.12+. Not tested with 3.10