requests: Support SSL_CERT_FILE environment variable

[thoger]

Requests allows setting path to a CA certificate bundle that should be used instead of the default system one using the following environment variables: REQUESTS_CA_BUNDLE and CURL_CA_BUNDLE.

https://github.com/kennethreitz/requests/blob/v2.8.1/requests/sessions.py#L621

Please make it also check SSL_CERT_FILE environment variable in the same way. This variable is recognized by OpenSSL and is documented in PEP 476 as a way to make Python standard library https clients use non-default CA bundle file:

https://www.python.org/dev/peps/pep-0476/#trust-database

It sounds reasonable to have a single environment variable that can be used for all Python scripts regardless of whether they are using requests or standard library httplib / urllib*.

In terms of the fix, a simple or os.environ.get('SSL_CERT_FILE') should do the trick.

[Lukasa]

This seems reasonable enough to me. I'd be happy to accept a patch that does this. We should also accept SSL_CERT_DIR.

[kennethreitz]

This seems harmless enough.

[raultabacu]

Is this still an issue ? If so I would like to give it a shot.

[Lukasa]

@Keops92 The work has been done, sadly: see #2903.

[raultabacu]

Thanks !

What about adding a new keyword (I don't know, let's say verify_native ) and if set to True, requests would respect SSL_CERT_FILE and SSL_CERT_DIR as well as try to use platform-native certificate storage? This shouldn't break backwards compatibility.

[Lukasa]

The Requests team are strongly opposed to adding new keyword arguments, especially if they are redundant with, overlap with, or can be set to contradictory values with a pre-existing keyword argument.

That makes sense.

Is there a plan to support system certificate storage in the next compatibility-breaking release?

[Lukasa]

@luv Currently the expectation is that a future urllib3 release will allow the passing of custom SSLContext objects to urllib3. That will allow construction of adapters that let you use the system certificate store on Linux. For OS X and Windows we have the beginnings of some code to do it, but it's a lot of very subtle work that needs a lot of bugs ironed out.

Thanks for answer.

Actually, are you sure you need custom SSLContext to use system certificate store on Unix/Linux? It should be enough to pass correct path to "verify" keyword and you are sorted, even with the current version of requests.

I realize it must be really PITA working on supporting OS X, Windows, Unix and Linux system certificate store transparently.

Btw. from your previous comment it was not clear if you plan to limit this functionality to OS X and Windows only (and leave it up to the end-user to support unix and linux system certificates with custom SSLContext) or if you plan to support system certificates on Linux (and *BSD and friends :) ) as well.

Thanks!

[Lukasa]

It should be enough to pass correct path to "verify" keyword and you are sorted, even with the current version of requests.

I wish, but it is not. There is no consistent "correct" path that works across all Unices. There is no consistent "correct" path that works across all Linux distributions. There is not even a guarantee that there's a consistent "correct" path that works across all versions of the same distribution. The reason the SSLContext approach is preferred is simply that it allows you to defer that choice to the distribution-provided OpenSSL, which is hopefully correctly configured.

Btw. from your previous comment it was not clear if you plan to limit this functionality to OS X and Windows only (and leave it up to the end-user to support unix and linux system certificates with custom SSLContext) or if you plan to support system certificates on Linux (and *BSD and friends :) ) as well.

The ideal end goal will be that there's a urllib3 flag you can hit to change its verification logic, and that requests will make it available in some way. That will work for all platforms.

I see. Would you mind providing an example where curl's approach to finding the "correct path" does not work https://github.com/curl/curl/blob/master/acinclude.m4#L2627 ?

The reason the SSLContext approach is preferred is simply that it allows you to defer that choice to the distribution-provided OpenSSL, which is hopefully correctly configured.

Oh ... if this is going to depend on functions like X509_get_default_cert_dir - it might be, in the real world, more problematic than hard-coded paths :( ... http://lists.freebsd.org/pipermail/freebsd-python/2014-December/007809.html