Python requests self signed certificate

import requests
data = {'foo':'bar'}
url = 'https://foo.com/bar'
r = requests.post(url, data=data)

If the URL uses a self signed certificate, this fails with

requests.exceptions.SSLError: [Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I know that I can pass False to the verify parameter, like this:

r = requests.post(url, data=data, verify=False)

However, what I would like to do is point requests to a copy of the public key on disk and tell it to trust that certificate.

frlan

6,7683 gold badges29 silver badges72 bronze badges

asked May 22, 2015 at 21:06

Python requests self signed certificate

Matthew MoisenMatthew Moisen

15.1k25 gold badges108 silver badges212 bronze badges

1

try:

r = requests.post(url, data=data, verify='/path/to/public_key.pem')

answered May 22, 2015 at 21:12

12

With the verify parameter you can provide a custom certificate authority bundle

requests.get(url, verify=path_to_bundle_file)

From the docs:

You can pass verify the path to a CA_BUNDLE file with certificates of trusted CAs. This list of trusted CAs can also be specified through the REQUESTS_CA_BUNDLE environment variable.

answered May 22, 2015 at 21:14

0

The easiest is to export the variable REQUESTS_CA_BUNDLE that points to your private certificate authority, or a specific certificate bundle. On the command line you can do that as follows:

export REQUESTS_CA_BUNDLE=/path/to/your/certificate.pem
python script.py

If you have your certificate authority and you don't want to type the export each time you can add the REQUESTS_CA_BUNDLE to your ~/.bash_profile as follows:

echo "export REQUESTS_CA_BUNDLE=/path/to/your/certificate.pem" >> ~/.bash_profile ; source ~/.bash_profile

answered May 12, 2017 at 10:20

Mike NMike N

5,8453 gold badges23 silver badges21 bronze badges

3

Case where multiple certificates are needed was solved as follows: Concatenate the multiple root pem files, myCert-A-Root.pem and myCert-B-Root.pem, to a file. Then set the requests REQUESTS_CA_BUNDLE var to that file in my ./.bash_profile.

$ cp myCert-A-Root.pem ca_roots.pem
$ cat myCert-B-Root.pem >> ca_roots.pem
$ echo "export REQUESTS_CA_BUNDLE=~/PATH_TO/CA_CHAIN/ca_roots.pem" >> ~/.bash_profile ; source ~/.bash_profile

answered Jun 8, 2017 at 15:55

1

Setting export SSL_CERT_FILE=/path/file.crt should do the job.

answered Sep 21, 2017 at 7:21

gizzmolegizzmole

1,29916 silver badges24 bronze badges

2

If you're behind a corporate network firewall like I was, ask your network admin where your corporate certificates are, then:

import os
os.environ["REQUESTS_CA_BUNDLE"] = 'path/to/corporate/cert.pem'
os.environ["SSL_CERT_FILE"] = 'path/to/corporate/cert.pem'

This fixed issues I had with requests and openssl.

answered Sep 28, 2021 at 11:11

Jordan MJordan M

1532 silver badges6 bronze badges

All of the answers to this question point to the same path: get the PEM file, but they don't tell you how to get it from the website itself.

Getting the PEM file from the website itself is a valid option if you trust the site, such as on an internal corporate server. If you trust the site, why should you do this? You should do this because it helps protect yourself and others from inadvertently re-using your code on a site that isn't safe.

Here is how you can get the PEM file.

  1. Click on the lock next to the url.

    Python requests self signed certificate

  2. Navigate to where you can see the certificates and open the certificates.

    Python requests self signed certificate

  3. Download the PEM CERT chain.

    Python requests self signed certificate

  4. Put the .PEM file somewhere you script can access it and try verify=r"path\to\pem_chain.pem" within your requests call.

r = requests.get(url, verify='\path\to\public_key.pem')

answered Jan 28 at 19:17

FoggyFoggy

14110 bronze badges

1

Incase anyone happens to land here (like I did) looking to add a CA (in my case Charles Proxy) for httplib2, it looks like you can append it to the cacerts.txt file included with the python package.

For example:

cat ~/Desktop/charles-ssl-proxying-certificate.pem >> /usr/local/google-cloud-sdk/lib/third_party/httplib2/cacerts.txt

The environment variables referenced in other solutions appear to be requests-specific and were not picked up by httplib2 in my testing.

answered Jun 29, 2017 at 2:39

Mat SchafferMat Schaffer

1,5991 gold badge13 silver badges22 bronze badges

In a dev environment, using Poetry as virtual env provider on a Mac with Python 3.8 I used this answer https://stackoverflow.com/a/42982144/15484549 as base and appended the content of my self-signed root certificate to the certifi cacert.pem file.

The steps in detail:

cd project_folder
poetry add requests
# or if you use something else, make sure certifi is among the dependencies
poetry shell
python
>>> import certifi
>>> certifi.where()
/path/to/the/certifi/cacert.pem
>>> exit()
cat /path/to/self-signed-root-cert.pem >> /path/to/the/certifi/cacert.pem
python the_script_you_want_to_run.py

answered Feb 4 at 15:01

AlexAlex

964 bronze badges

I know it is an old thread. However, I run into this issue recently. My python requests code does not accept the self-signed certificate but curl does. It turns out python requests are very strict on the self-signed certificate. It needs to be a root CA certificate. In other words,

Basic Constraints: CA:TRUE

Key Usage: Digital Signature, Non Repudiation, Key Encipherment, Certificate Sign

answered Apr 1 at 18:50

Python requests self signed certificate

YufengYufeng

211 silver badge1 bronze badge

How do I check if a python certificate is self

To verify this the key type or to check an unknown key type in python, you can use the isinstance method passing in the key and the algorithm. See below for an example on how to determine the key type of a public key or a private key in python. public_key = cert. public_key() if isinstance(public_key, rsa.

How do I disable security certificate checks for request in Python?

Method 1: Passing verify=False to request method Along with the URL also pass the verify=False parameter to the method in order to disable the security checks.

Does Python requests use HTTPS?

Requests verifies SSL certificates for HTTPS requests, just like a web browser. SSL Certificates are small data files that digitally bind a cryptographic key to an organization's details.

Does request use certifi?

Requests uses certificates from the package certifi.