OpenSSL: TLS guide

This guide describes the implementation of a TLS client in OpenSSL.

The guide covers basic aspects of initiating a secure TLS connection, including certificate validation and hostname verification. When various alternative approaches are possible, the guide presents each of them and specifies their use cases to help you choose which approach suits your needs best.

  • We work with the API in C of OpenSSL, version 1.1.1.
  • We assume the server to communicate with is at x509errors.org and accepts TLS connections on a standard port 443.

In addition to this guide, we have also implemented OpenSSL guides for the following topics:

Certificate Transparency CRL revocation OCSP Stapling OCSP revocation

The source code from all guides is also available as a stand-alone CLI client with options to test multiple revocation schemes:

TLS client source code

Establishing an underlying TCP/IP connection

First, we need to establish an insecure TCP/IP connection with the server. For the most simple connection, a standard set of POSIX functions will suffice.

#include <sys/socket.h>
#include <unistd.h>

/* TCP/IP socket descriptor. */
int sockfd = -1;

/* We will send the server our connection preferences in the form of hints. */
struct addrinfo hints = {0};

/* We allow both IPv4 and IPv6. */
hints.ai_family = AF_UNSPEC;
/* We want a stream socket, not a datagram one. */
hints.ai_socktype = SOCK_STREAM;
/* We know the numeric port number beforehand. */
hints.ai_flags = AI_ADDRCONFIG | AI_NUMERICSERV;
/* We want to use TCP. */
hints.ai_protocol = IPPROTO_TCP;

struct addrinfo *result = NULL;

/* We query a list of addresses for the given hostname. */
if (getaddrinfo("x509errors.org", "443", &hints, &result) != 0 || result == NULL) {
    exit(EXIT_FAILURE);
}

/* Try to connect to each address from the server list until successful. */
struct addrinfo *rr;
for (rr = result; rr != NULL, rr = rr->ai_next) {
    sockfd = socket(rr->ai_family, rr->ai_socktype, rr->ai_protocol);
    if (sockfd == -1)
        continue;
    if (connect(sockfd, rr->ai_addr, rr->ai_addrlen) != -1)
        break;
    close(sockfd);
}

/* We don't need the server info anymore. */
freeaddrinfo(result);

/* We must fail if we didn't manage to connect to any server address. */
if (rr == NULL) {
    exit(EXIT_FAILURE);
}

If everything went well, sockfd is now a descriptor of a valid, connected socket. We can proceed to establish the TLS connection on top of the TCP/IP connection.

Creating a TLS context

Before we connect, a TLS context structure has to be created. It will store all the necessary configuration and settings needed for our session.

#include <openssl/ssl.h>

/* Create the context. We will use the version-flexible TLS method to negotiate.
** This means that we prefer the highest supported version, but agree with downgrading. */
SSL_CTX const *ctx = SSL_CTX_new(TLS_client_method());
if (ctx == NULL) {
    exit(EXIT_FAILURE);
}

/* However, we won't let the server downgrade to less than TLS v1.2, since older TLS versions are deprecated. */
if (SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION) != 1) {
    exit(EXIT_FAILURE);
}

/* We need to set the option to validate the peer certificate chain.
** If we skipped this step, an active attacker could impersonate the server. */
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);

/* In certificate validation, we usually want to trust the system default certificate authorities. */
if (SSL_CTX_set_default_verify_paths(ctx) != 1) {
    exit(EXIT_FAILURE);
};

Alternative: Setting an arbitrary trust anchor

In some cases, it might be useful to trust an arbitrary certificate authority. This could be the case during testing or within company intranets. If we trust a CA located in trusted_ca.pem and other authorities located in trusted_dir, we can easily change the trust setting as follows:

/* Both the file path and the directory path can be set to NULL if they are not used */
if (SSL_CTX_load_verify_locations(ctx, "trusted_ca.pem", "trusted_dir") != 1) {
    exit(EXIT_FAILURE);
}

Optional: Request the stapled OCSP Response from the TLS server

A TLS client application can request a TLS server to send it an OCSP response (known as OCSP-Stapling) during the TLS handshake.

if (SSL_CTX_set_tlsext_status_type(context, TLSEXT_STATUSTYPE_ocsp) != 1) {
    exit(EXIT_FAILURE);
}

Optional: Enable processing of Signed Certificate Timestamps (SCTs)

Enable the processing of Signed Certificate Timestamps, which is performed by default using the predefined Certificate Transparency (CT) validation callback. The validation of SCTs supports two modes, one of which must be selected. When mode SSL_CT_VALIDATION_PERMISSIVE is on, then the TLS handshake result is not affected by the validation status of any SCT. On the other hand, when mode SSL_CT_VALIDATION_STRICT is on together with the previously set verification mode SSL_VERIFY_PEER, then the TLS handshake will be aborted with the X509_V_ERR_NO_VALID_SCTS error code if the peer presents no valid SCT.

/* Enables OCSP stapling as well because SCTs could be delivered through OCSP Stapling, TLS Extensions or X509 Extensions. */
if (SSL_CTX_enable_ct(context, SSL_CT_VALIDATION_STRICT) != 1) {
    exit(EXIT_FAILURE);
}

Optional: Custom certificate validation settings

Optionally, you may want to put additional constraints on certificate validation. OpenSSL allows for this by modifying the verify params structure. In this example, we enforce strict certificate validation and put requirements on the IP address contained in the Subject Alternative Name extension of the server certificate. All possible settings and flags can be found in the original documentation

#include <openssl/x509.h>
#include <openssl/x509_vfy.h>

/* Retrieve the verification parameters for modification. */
X509_VERIFY_PARAM *vpm = SSL_CTX_get0_param(ctx);
if (vpm == NULL) {
    exit(EXIT_FAILURE);
}

/* Retrieve certificate validation flags. */
unsigned long flags = X509_VERIFY_PARAM_get_flags(vpm);

/* Enable the strict certificate validation flag.
** Certificates with e.g. duplicate extensions, will now be rejected. */
flags |= X509_V_FLAG_X509_STRICT;

/* Put the modified validation flags back into the params structure. */
if (X509_VERIFY_PARAM_set_flags(vpm, flags) != 1) {
    exit(EXIT_FAILURE);
}

/* Server certificate will have to contain IP 192.168.2.1 in its Subject Alternative Name. */
if (X509_VERIFY_PARAM_set1_ip_asc(vpm, "192.168.2.1") != 1) {
    exit(EXIT_FAILURE);
}

/* Save the modified verify param structure back to the CTX context structure. */
if (SSL_CTX_set1_param(context, verify_param_struct) != 1) {
    EXIT(EXIT_FAILURE);
}

Alternative: Setting custom verification callback

It is possible to set a custom verification callback by calling the SSL_CTX_set_tlsext_status_cb API call. This callback function will be invoked during the TLS handshake after the certificate chain has been successfully validated. The prototype of this callback function is int (*callback)(SSL *, void *). The first argument is the SSL session instance, and the second argument is a value previously set by calling the SSL_CTX_set_tlsext_status_arg API call. The return value of this callback function should be positive when it is required for the TLS handshake to continue. Otherwise, the TLS handshake will fail, and the connection should be terminated immediately.

/* Function callback called during the TLS handshake after the certificate chain has been verified. */
int (*callback)(SSL *, void*) = &custom_callback;
SSL_CTX_set_tlsext_status_cb(context, callback);

where the custom_callback function has the following declaration:

int custom_callback(SSL *s_connection, void *arg) {
    /* Perform revocation check by using CRL, OCSP or OCSP-Stapling check. */
    /* Perform any action required to run during the TLS handshake. */
}

According to the official OpenSSL documentation, the primary purpose of this callback function is to validate and process a stapled OCSP response. However, since this function is called during the TLS handshake, after successful validation of the certificate chain, it can also be used for checking the revocation status of the certificates from the certificate chain.

In addition to the OCSP-Stapling scheme, we have also covered guides for checking the revocation status for all certificates in the chain using CRL or OCSP revocation schemes. We have also covered a guide for checking the Certificate Transparency criteria for each certificate in the chain.

Certificate Transparency CRL revocation OCSP Stapling OCSP revocation

Initializing a TLS connection

At this point, we can initialize a connection structure and link it with the open socket descriptor. After that, we only need to specify a couple of settings and we can connect to the server.

/* Initialize a TLS connection structure. */
ssl = SSL_new(ctx);
if (ssl == NULL) {
    exit(EXIT_FAILURE);
}

/* Bind the socket descriptor to the connection structure. */
if (SSL_set_fd(ssl, sockfd) != 1) {
    exit(EXIT_FAILURE);
}

/* Set the Server Name Indication TLS extension to specify the name of the server. */
/* This is required when multiple servers are running at the same IP address (virtual hosting). */
if (SSL_set_tlsext_host_name(ssl, "x509errors.org") != 1) {
    exit(EXIT_FAILURE);
}

/* Set hostname for verification. */
/* Not setting the hostname would mean that we would accept a certificate of any trusted server. */
if (SSL_set1_host(ssl, "x509errors.org") != 1) {
    exit(EXIT_FAILURE);
}

/* Connect to the server, this performs the TLS handshake. */
/* During this procedure, the peer certificate is validated. */
if (SSL_connect(ssl) != 1) {
    exit(EXIT_FAILURE);
}

Optional: Checking the result of peer certificate validation

If certificate validation fails, SSL_connect() will always fail with the same error message. In that case, it is often useful to examine the specific certificate validation error as follows. You can find explanations of certificate validation messages in the official documentation or on our dedicated page.

/* Retrieve the error code of the error that occured during certificate validation. */
int verifyResult = SSL_get_verify_result(ssl);

/* Convert the error code to a human-readable string. */
const char *message = X509_verify_cert_error_string(verifyResult);

/* Print the string to the standard error output. */
fprintf(stderr, "%s", message);

Sending and receiving data using the TLS connection

When the connection is successfully established, we can share application data with the server. These two functions provide the basic interface.

/* Prepare a message and send it to the server. */
char *message = "Hello server";
if (SSL_write(ssl, message, strlen(message)) != 1) {
    exit(EXIT_FAILURE);
}

/* Prepare a static buffer for the response and read the response into that buffer. */
char buffer[4096];
if (SSL_read(ssl, buffer, 4096) != 1) {
    exit(EXIT_FAILURE);
}

Closing the TLS connection

The client is usually the one to indicate that the connection is finished. When we want the connection closed, the following steps are performed.

/* To finish the connection properly, we send a "close notify" alert to the server. */
/* In most cases, we have to wait for the same message from the server, and perform the call again. */
int ret = SSL_shutdown(ssl);
if (ret < 0) {
    exit(EXIT_FAILURE);
} else if (ret == 0) {
    if (SSL_shutdown(ssl) != 1) {
        exit(EXIT_FAILURE);
    }
}

/* Free the TLS connection structure. */
if (ssl != NULL) {
    SSL_free(ssl);
}

/* Free the TLS context structure. */
if (ctx != NULL) {
    SSL_CTX_free(ctx);
}

/* Close the underlying TCP socket. */
if (sockfd >= 0) {
    close(sockfd);
}