Get rid of requests dependency

[non-Jedi]

Per a conversation with Coffee in #matrix-dev:matrix.org, requests is a rather heavyweight dependency when compared to the size of this library. Since we only use requests in the _send api method, using urllib instead of requests shouldn't add much extra burden to maintaining the sdk.

The biggest issue to overcome is rewriting all of the tests that use responses.

See also https://github.com/Matrixcoffee/urllib-requests-adapter

[Half-Shot]

Hmm, that's a fair reason. I was wondering about how we go about making these calls asynchronous given we are replacing requests anyway.

(tagging @matrixcoffee )

[non-Jedi]

If we're bent on continuing to support python 2, our async options are significantly limited unfortunately. 😕

We probably should break making the library async by default into a separate issue to make sure we're tracking both.

By the way, https://github.com/Matrixcoffee/urllib-requests-adapter already does this without touching a single line of code inside matrix-python-sdk. I'd love to get feedback from various people on it, issues you run into etc, as preparation for switching the SDK itself to use urllib.

[karthanistyr]

Hi, is this issue still worth looking at? The discussion is compounding migrating to plain urllib and also doing async. Regarding the latter, I noticed a massive PR waiting for review, so it's not clear what's expected here.

I initially understood this issue to be migrating away from requests, to plain urllib, in synchronous mode.

From my end, I'd say yes. We'll still need to support Python 2 for at least 1.5 years, and this change isn't that hard IMO. I think the most difficult thing is making it work under both Python 2 and 3.

Once Python 2 is officially phased out, we can see what the landscape looks like then.

Oh the other hand, it's not that urgent for me anymore since urllib-requests-adapter handles my use cases just fine.

[non-Jedi]

I'm personally of the opinion right now, that requests sessions provide enough
nice-to-have features that it might not be worthwhile to eliminate requests.
There's also the fact that because of responses, requests is significantly
easier to test. That said, I have done no actual benchmarking on this point
myself and could easily be persuaded otherwise.

Once we can eliminate 2.7 support, I'd really like to move to a http framework
that supports asyncio natively. AIUI, that eliminates both requests and urllib
from the standard library...

[non-Jedi]

@karthanistyr this issue is purely about migrating to plain urllib. The PR that
you saw is using an approach to add async capabilities that I'm not sure I'm
satisfied with, and it needs tons of effort to finish and to rebase in any case.
The discussion about async in this issue is purely to make sure that we don't
choose an approach for our http lib that makes more difficult any effort to add
async support to the library.

[non-Jedi]

@Matrixcoffee if we use urllib3 on which requests is built to replace requests in this lib, we can keep the connection pooling we have now while eliminating the requests dependency. Would that fit your usecase?

it wouldn't really, as it just replaces one dependency with another. (AIUI urllib3 is not part of the standard library either.) Which doesn't mean it's not a good idea, though.

I'm thinking this issue can be closed, and I'll just Live With It via urllib-requests-adapter. This can be revisited when Python 2 support is dropped.

[non-Jedi]

It's since requests depends on urllib3, it's going from 2 dependencies to 1, so it would be an improvement in that regard just so you know.