I’ve used curl as a debugging tool for a long time. But time moves on and my testing system didn’t. So now for the first time I saw an error that is produced by this situation, and I will explain it.
$ curl ‐i ‐k https://julialang.org/
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
$ curl ‐help
... -2/--sslv2 Use SSLv2 (SSL) -3/--sslv3 Use SSLv3 (SSL) ... -1/--tlsv1 Use TLSv1 (SSL) ...
Compare this to a server which I’ve kept up-to-date with openssl and curl:
... -2/--sslv2 Use SSLv2 (SSL) -3/--sslv3 Use SSLv3 (SSL) ... -1/--tlsv1 Use => TLSv1 (SSL) --tlsv1.0 Use TLSv1.0 (SSL) --tlsv1.1 Use TLSv1.1 (SSL) --tlsv1.2 Use TLSv1.2 (SSL) ...
On this server I can fetch the home page with curl.
So it appears the older system does not have a compatible version of TLS. To confirm this use SSLLABS. We see this:
Sure enough, only TLS 1.2 is supported by the server, and my poor old curl doesn’t have that! Too bad for me, but it shows it’s time to upgrade.
Another problem site
askapache.com is another vexing site. On a curl version which supposedly supports tls 1.2 I get this error:
$ curl ‐‐tlsv1.2 ‐‐verbose ‐k https://askapache.com/
* About to connect() to askapache.com port 443 (#0) * Trying 18.104.22.168... connected * Connected to askapache.com (22.214.171.124) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * warning: ignoring value of ssl.verifyhost * NSS error -12286 * Closing connection #0 * SSL connect error curl: (35) SSL connect error
This is with curl version 7.19.7 on my CentOS 6.8 system.
This same site works fine on my compiled version of curl with the latest openssl, version 7.55.1. The system-supplied curl is missing support for some cipher suites.
Here’s my compiled curl and openssl list of cipher suites:
$ openssl ciphers
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384: ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305: ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256: ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA: ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:RSA-PSK-AES256-GCM-SHA384: DHE-PSK-AES256-GCM-SHA384:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305: ECDHE-PSK-CHACHA20-POLY1305:AES256-GCM-SHA384:PSK-AES256-GCM-SHA384:PSK-CHACHA20-POLY1305: RSA-PSK-AES128-GCM-SHA256:DHE-PSK-AES128-GCM-SHA256:AES128-GCM-SHA256:PSK-AES128-GCM-SHA256: AES256-SHA256:AES128-SHA256:ECDHE-PSK-AES256-CBC-SHA384:ECDHE-PSK-AES256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:RSA-PSK-AES256-CBC-SHA384:DHE-PSK-AES256-CBC-SHA384:RSA-PSK-AES256-CBC-SHA: DHE-PSK-AES256-CBC-SHA:AES256-SHA:PSK-AES256-CBC-SHA384:PSK-AES256-CBC-SHA:ECDHE-PSK-AES128-CBC-SHA256:ECDHE-PSK-AES128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA: RSA-PSK-AES128-CBC-SHA256:DHE-PSK-AES128-CBC-SHA256:RSA-PSK-AES128-CBC-SHA:DHE-PSK-AES128-CBC-SHA:AES128-SHA:PSK-AES128-CBC-SHA256:PSK-AES128-CBC-SHA
and what I see on my older system:
$ openssl ciphers
DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:DHE-RSA-CAMELLIA256-SHA:DHE-DSS-CAMELLIA256-SHA: CAMELLIA256-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DES-CBC3-SHA:DES-CBC3-MD5: DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:DHE-RSA-CAMELLIA128-SHA:DHE-DSS-CAMELLIA128-SHA: CAMELLIA128-SHA:RC2-CBC-MD5:RC4-SHA:RC4-MD5:RC4-MD5:EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC-SHA:DES-CBC-SHA: DES-CBC-MD5:EXP-EDH-RSA-DES-CBC-SHA:EXP-EDH-DSS-DES-CBC-SHA:EXP-DES-CBC-SHA:EXP-RC2-CBC-MD5:EXP-RC2-CBC-MD5:EXP-RC4-MD5:EXP-RC4-MD5
Note that when curl successfully connects it shows which cipher suite was chosen if you use the -v switch:
$ curl ‐v ‐k https://drjohnstechtalk.com/
* About to connect() to drjohnstechtalk.com port 443 (#0) * Trying 126.96.36.199... connected * Connected to drjohnstechtalk.com (188.8.131.52) port 443 (#0) ... * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 ...
On a more demanding server – one that does not work with old curl, this dialog is longer, TLS 1.2 is preferred and a more secure cipher suite is chosen – one not available on the other system:
(issue standard curl -k -v <server_name>)
* Trying 184.108.40.206... * TCP_NODELAY set * Connected to 220.127.116.11 port 443 (#0) * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
Another curl error explained
$ curl -v -i -k https://drjohnstechtalk.com/
* About to connect() to drjohnstechtalk.com port 443 (#0) * Trying 18.104.22.168... connected * Connected to drjohnstechtalk.com (22.214.171.124) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * warning: ignoring value of ssl.verifyhost * skipping SSL peer certificate verification * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 * Server certificate: * subject: CN=drjohnstechtalk.com,C=US * start date: Apr 03 00:00:00 2017 GMT * expire date: Apr 03 23:59:59 2019 GMT * common name: drjohnstechtalk.com * issuer: CN=Trusted Secure Certificate Authority 5,O=Corporation Service Company,L=Wilmington,ST=NJ,C=US > GET / HTTP/1.1 > Host: drjohnstechtalk.com > Accept: */* > User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.1; .NET CLR 3.0.04506.30; > * SSL read: errno -5961 * Closing connection #0 curl: (56) SSL read: errno -5961
What’s going on?
In this test drjohnstechtalk.com was behind a load balancer. The load balancer had SSL configured. The back-end server was not running however though the load balancer’s health check did not detect that condition. So the load balancer permitted the initial connection, but then shut things off when it could not open a connection to the back-end server. So this error has nothing to do with curl showing its age, but I didn’t know that when I started debugging it.
Then there’s this one:
$ curl ‐v ‐i ‐k https://lb.drjohnstechtalk.com/
* Trying 126.96.36.199... * TCP_NODELAY set * Connected to fw-change-request.bdrj.net (188.8.131.52) port 443 (#0) * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 * ALPN, server did not agree to a protocol * Server certificate: * subject: C=US; ST=NJ; CN=lb.drjohnstechtalk.com * start date: Nov 14 12:06:02 2017 GMT * expire date: Nov 14 12:06:02 2018 GMT * SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. > GET / HTTP/1.1 > Host: lb.drjohnstechtalk.com > User-Agent: curl/7.55.1 > Accept: */* > * OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104 * Closing connection 0 curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104
This also seems to occur as I’ve seen when there’s a load balancer in front of a web server where the load balancer is working fine but the web server is not.
Another example challenging web site
$ curl ‐‐version
curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
$ curl ‐v ‐k https://e1st.smapply.org/
* About to connect() to e1st.smapply.org port 443 (#0) * Trying 184.108.40.206... connected * Connected to e1st.smapply.org (220.127.116.11) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * warning: ignoring value of ssl.verifyhost * NSS error -12286 * Closing connection #0 * SSL connect error curl: (35) SSL connect error
I have seen this suggestion on the Internet to fix the system-supplied curl on a CentOS 6.8 system:
yum update -y nss curl libcurl
It didn’t work!
I tried to give the owners of e1st.smapply.org a hard time for supporting such a limited set of ciphersuites – essentially only the latest thing (which you can see yourself by running it through sslabs.com): TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384. If I run this through SSL interception on a Symantec proxy with an older image, that ciphersuite isn’t present! I had to upgrade, then it was fine. But getting back to the rationale, they told me they have future-proofed their site for the new requirements of PCI and they would not budge.
Another curl error
curl: (3) Illegal characters found in URL
If your url looks visibly OK, mkae sure you don’t have and non-printed characters in it. Put it through the linux od -c utility. In my case I culled the url from a Location header after parsing it with awk. Unbeknownst to me, tagging along at the end, unseen, was an extra \n\r characters. I had to get rid of those.
A TLS version error is explained, as well as the way it came about. Another curl/SSL error is also explained.
References and related
I eventually came up with the solution: compile my own updated version of curl! I describe how I did it in this blog post.
A more recent TLS versioning problem which I could have only resolved by using curl is described in this post.
4 replies on “curl showing its age with SSL error”
The problem is not with curl, the problem is with the piece of shit incompatible sites. I simply shitlist any site displaying the misbehaviour ; they can all go hang out in their little autistic bubble by themselves.
Did you write curl? Geez
Ha. I wish I had such programming talent. But no I did not.
great analysis! I have this problem. I’m prevented from using newer than centos 5.11 and now I cant access a lot of web sites with curl!
What I didnt see here is how one can cure this…..