Deploy CFSSL CA Server

Moving from an Active Directory-backed CA to a CFSSL-backed one gives your lab a self-contained, API-driven certificate authority that is easier to maintain and less tightly coupled to domain infrastructure. This guide walks through the full deployment: installing dependencies, initializing root and intermediate certificates, configuring the CFSSL API server with PostgreSQL, and exposing everything through Nginx.

Overview

In an effort to improve accessibility and maintainability of certificate issuance for lab activities, we have decided to move from an AD-backed CA to one that is backed by CFSSL. The deployment of CFSSL is backed by PostgreSQL and provides an accessible API that can be used with the provided tool chain or by direct access with standard system tools such as curl, python, perl, and bash.

CFSSL

This CA is backed by CFSSL. CFSSL is Cloudflare’s PKI/TLS swiss army knife. It is both a command line tool and an HTTP API server for signing, verifying, and bundling TLS certificates. It requires Go 1.12+ to build.

https://github.com/cloudflare/cfssl

CFSSL consists of:

• A set of packages useful for building custom TLS PKI tools.
• The cfssl program, which is the canonical command line utility using the CFSSL packages.
• The multirootca program, which is a certificate authority server that can use multiple signing keys.
• The mkbundle program, used to build certificate pool bundles.
• The cfssljson program, which takes the JSON output from the cfssl and multirootca programs and writes certificates, keys, CSRs, and bundles to disk.

CA Information

Location and Access

• The certificate authority is deployed in the Austin Data Center as of this writing.
• We have chosen to restrict SSH access on this system in order to ensure that end-users are accessing signing requests and certificates through the exposed API: http://ca.lab.atx.example.com/api/v1/cfssl/
• SSH access should only be required to perform regular maintenance and to update the CA once it reaches expiration.

Root Certificate

http://ca.lab.atx.example.com/root_ca/

Intermediate Certificate

http://ca.lab.atx.example.com/intermediate_ca/

OCSP Responder

http://ca.lab.atx.example.com/ocsp/

CRL

http://ca.lab.atx.example.com/crl/

Certificate Bundle

http://ca.lab.atx.example.com/bundle/

Issuing Defaults

Deployment

Host and Operating System Resources

System Software Dependencies

1. PostgreSQL 11+
2. Go 1.12+
3. Python
4. Supervisord
5. Nginx/Apache
6. Haveged
7. OpenSSL

Installing Dependencies

This section covers only items that have special installation instructions. Other items such as supervisord, python, and nginx can all be readily installed through yum.

Installing Go

This will install a specific version of Go on the host operating system.

1. Download the Go tarball for the appropriate operating system and architecture from https://golang.org/dl/.

wget https://dl.google.com/go/go1.14.linux-amd64.tar.gz

2. Decompress the tarball to /usr/local.

tar -xzvf go1.14.linux-amd64.tar.gz -C /usr/local

3. Create a working directory for Go projects.

mkdir /opt/go

4. Update the system profile in /etc/profile to export these environment variables.

cat >> /etc/profile << EOF
export GOROOT=/usr/local/go
export GOPATH=/opt/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
EOF

5. Source the new profile.

source /etc/profile

Installing PostgreSQL

The version of PostgreSQL that is provided by CentOS and RHEL will often be fairly old. To obtain a version that contains newer enhancements and better performance, you should not deploy PostgreSQL from the standard CentOS or RHEL repositories. This example uses PostgreSQL 11.

1. Add the PostgreSQL 11 repositories.

yum install https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-centos11-11-2.noarch.rpm

2. Install PostgreSQL server, client, and contrib.

yum -y install postgresql11-server postgresql11-contrib postgresql11

3. Initialize the PostgreSQL database.

/usr/pgsql-11/bin/postgresql-11-setup initdb

4. Enable PostgreSQL in systemd.

systemctl start postgresql-11
systemctl enable postgresql-11

Installing CFSSL

This will build all of the CFSSL tools and place them in /opt/go/bin.

1. Change directories to /opt/go.

cd /opt/go

2. Build all of the CFSSL tools.

go get github.com/cloudflare/cfssl/cmd/...

Initializing the Certificate Authority

This portion of the deployment does not specifically rely on the database. The database is used by the server once it is online. This section covers initializing the CA certificates for later service. This section can also be used to handle replacement of the certificates upon expiration.

Folder Structure

The certificates which operate the authority are set up in /opt/certificate_authority. Endpoint or signed certificates are not stored here. The signed and issued certificates are stored in PostgreSQL.

bundle
crl
csr
intermediate_ca
json_req
keys
ocsp
root_ca
x509

CFSSL and JSON

CFSSL is primarily built around JSON. While it will accept a properly formed CSR through the API under normal conditions, if you were to interface with the CLI tools you would use JSON. The API also accepts JSON on request. You can find more on the options you can provide in the JSON package in the project documentation.

https://github.com/cloudflare/cfssl

An example is shown below of a JSON request. There are additional options which can be provided.

{
  "CN": "example.com",
  "names": [
    {
      "C": "US",
      "L": "San Francisco",
      "O": "Internet Widgets, Inc.",
      "OU": "WWW",
      "ST": "California"
    }
  ]
}

CFSSL Certificate Profiles

CFSSL supports profiles much like templates in ADCS. You can reference these profiles during certificate creation. The profiles establish the issuing defaults and issuing options for a given template. It supports a fairly large array of issuing extensions that will cover most needs.

https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt

CFSSL Profile JSON

In this deployment we are using the following location for profiles: /opt/go/conf/cfssl-cert-defaults.json.

{
  "signing": {
    "default": {
      "ocsp_url": "http://ca.sec.example.com/ocsp",
      "crl_url": "http://ca.sec.example.com/crl",
      "expiry": "8760h",
      "usages": [
        "signing",
        "key encipherment",
        "client auth",
        "server auth"
      ]
    },
    "profiles": {
      "ocsp": {
        "usages": ["digital signature", "ocsp signing"],
        "expiry": "43800h"
      },
      "intermediate": {
        "usages": ["cert sign", "crl sign"],
        "expiry": "43800h",
        "ca_constraint": {
          "is_ca": true
        }
      },
      "server": {
        "usages": ["signing", "key encipherment", "server auth"],
        "expiry": "8760h"
      },
      "client": {
        "usages": ["signing", "key encipherment", "client auth"],
        "expiry": "8760h"
      },
      "hadoop": {
        "usages": ["signing", "key encipherment", "client auth", "server auth"],
        "expiry": "8760h"
      }
    }
  }
}

Initializing the Root

The process is fairly straightforward. Issuing a new Root, Intermediate, or OCSP Signer certificate should require very little effort. You can use JSON for this process.

Root Certificate JSON

{
  "CN": "Sec Lab root CA",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C": "US",
    "S": "Tx",
    "L": "Austin",
    "O": "Lab Operations",
    "OU": "Security"
  }]
}

Creating the Root

Once you run this command you will need to move files to their respective locations in /opt/certificate_authority/.

cfssl gencert -initca /path/to/root-ca.csr.json | cfssljson -bare ca

Initializing the Intermediate

The process is fairly straightforward. Issuing a new Root, Intermediate, or OCSP Signer certificate should require very little effort. You can use JSON for this process.

Intermediate Certificate JSON

{
  "CN": "Sec Lab Intermediate CA",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C": "US",
    "S": "Tx",
    "L": "Austin",
    "O": "Lab Operations",
    "OU": "Security"
  }]
}

Issuing the Intermediate

Once you run this command you will need to move files to their respective locations in /opt/certificate_authority/.

cfssl gencert -ca /opt/certificate_authority/x509/ca.pem -ca-key /opt/certificate_authority/keys/ca-key.pem -config="/opt/go/conf/cfssl-cert-defaults.json" -profile="intermediate" /opt/certificate_authority/json_req/intermediate-ca-csr.json | cfssljson -bare ca-server -

Creating the OCSP Signer

The process is fairly straightforward. Issuing a new Root, Intermediate, or OCSP Signer certificate should require very little effort. You can use JSON for this process.

OCSP Signer JSON

{
  "CN": "Sec Lab OCSP Signer",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C": "US",
    "S": "Tx",
    "L": "Austin",
    "O": "Lab Operations",
    "OU": "Security"
  }]
}

Issuing the OCSP Signer

Once you run this command you will need to move files to their respective locations in /opt/certificate_authority/.

cfssl gencert -ca /opt/certificate_authority/x509/ca-server.pem -ca-key /opt/certificate_authority/keys/ca-server-key.pem -config="/opt/go/conf/cfssl-cert-defaults.json" -profile="ocsp" /opt/certificate_authority/json_req/ocsp-csr.json | cfssljson -bare server-ocsp -

Service Configuration

Setting Up PostgreSQL CFSSL Database

Since we are running CFSSL as a service and we may later care about the certificates we have issued (i.e. OCSP), we are using PostgreSQL to store certificate information. This setup is not exceptionally hard but there are two distinct pieces.

Setup CFSSL User Access

Edit the pg_hba configuration for PostgreSQL to enable the cfssl user login with a simple password. This is best done by altering the standard entries in this file such that you are using md5 instead of ident. Do not change the entry for Unix domain sockets.

# "local" is for Unix domain socket connections only
local   all             all                                     peer
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5

Creating the Database, Database User, and System User

1. Create a system user.

[root@ca ~]# useradd cfssl

2. Switch to the postgres user.

[root@ca ~]# su - postgres

3. Create the cfssl database user and cfssl database.

-bash-4.2$ psql

psql (11.7)
Type "help" for help.
postgres=# CREATE DATABASE cfssl;
CREATE DATABASE
postgres=# CREATE USER cfssl WITH PASSWORD 'somestrongpassword';
CREATE ROLE
postgres=# GRANT ALL PRIVILEGES ON DATABASE cfssl TO cfssl;
GRANT

4. Exit back to root.

5. Switch to the cfssl user.

[root@ca ~]# su - cfssl

6. Import the PostgreSQL migrations.

[cfssl@ca ~]# psql cfssl < /opt/go/src/github.com/cloudflare/cfssl/certdb/pg/migrations/001_CreateCertificates.sql

go get bitbucket.org/liamstask/goose/cmd/goose
goose -path certdb/pg up

Configuring CFSSL

Database Setup

Configuring CFSSL to use the database once it is daemonized is relatively simple. The configuration is also in JSON format. The parameters are as follows:

• Driver Type
• Driver URI
  • Username
  • Password
  • Database
  • Options

/opt/go/conf/cfssl-db-pg.json:

{
  "driver": "postgres",
  "data_source": "postgres://cfssl:nUFHFxbUH4UeN3m4@localhost/cfssl?sslmode=disable"
}

Cron Jobs

This can be run as root or the cfssl user, though it is likely easier if they run in the root crontab so that the files can be written wherever they need to be for those crons that create files.

*/30 * * * * /opt/go/bin/cfssl ocspdump -db-config /opt/go/conf/cfssl-db-pg.json > /opt/certificate_authority/ocsp/ocspdump.txt
*/30 * * * * /opt/go/bin/cfssl ocsprefresh -db-config /opt/go/conf/cfssl-db-pg.json -responder /opt/certificate_authority/x509/server-ocsp.pem -responder-key /opt/certificate_authority/keys/server-ocsp-key.pem -ca /opt/certificate_authority/x509/ca-server.pem -ca-key /opt/certificate_authority/keys/ca-server-key.pem
*/30 * * * * /opt/go/bin/cfssl crl -db-config /opt/go/conf/cfssl-db-pg.json -ca /opt/certificate_authority/x509/ca-server.pem -ca-key /opt/certificate_authority/keys/ca-server-key.pem | base64 -d > /opt/certificate_authority/crl/crl.txt

Configuring Supervisord

In this deployment we are using supervisord to manage the services. There are various different ways you can manage service daemonization; this just happens to be a relatively simple one. To use supervisord you will need to create two service configurations. Once both these configurations have been created, make sure to enable and start the supervisord service.

CFSSL API

/etc/supervisord.d/cfssl-api.conf:

[program:cfssl-api]
directory=/opt/go
command=/opt/go/bin/cfssl serve -db-config=/opt/go/conf/cfssl-db-pg.json -loglevel=1 -ca-key=/opt/certificate_authority/keys/ca-server-key.pem -ca=/opt/certificate_authority/x509/ca-server.pem -config=/opt/go/conf/cfssl-cert-defaults.json -responder=/opt/certificate_authority/x509/server-ocsp.pem -responder-key=/opt/certificate_authority/keys/server-ocsp-key.pem
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=syslog
stderr_logfile=syslog

CFSSL OCSP

/etc/supervisord.d/cfssl-ocsp.conf:

[program:cfssl-ocsp]
directory=/opt/go
command=/opt/go/bin/cfssl ocspserve -port=8889 -db-config=/opt/go/conf/cfssl-db-pg.json -loglevel=1
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=syslog
stderr_logfile=syslog

Enable and Start Supervisord

1. Enable.

systemctl enable supervisord

2. Start.

systemctl start supervisord

Configuring Nginx

The current deployment model primarily uses Nginx to act as a reverse proxy to the locally bound services that are running on an unprivileged port. In addition to that, Nginx is serving certificate bundles required for deployment. There are certainly more complex configurations and additional security that could be implemented at a later time.

/etc/nginx/default.d/ca.conf (this path may vary):

location /root_ca/ {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index ca.pem;
}

location = /root_ca {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index ca.pem;
}

location /intermediate_ca/ {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index ca-server.pem;
}

location = /intermediate_ca {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index ca-server.pem;
}

location /bundle/ {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index bundle.pem;
}

location = /bundle {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index bundle.pem;
}

location /crl/ {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index crl.txt;
}

location = /crl {
    root /opt/certificate_authority;
    sendfile on;
    sendfile_max_chunk 1m;
    tcp_nopush on;
    index crl.txt;
}

location /ocsp/ {
    proxy_pass http://127.0.0.1:8889/;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host              $host;
    proxy_set_header        X-Real-IP         $remote_addr;
    proxy_set_header        X-Forwarded-For   $proxy_add_x_forwarded_for;
}

location = /ocsp {
    proxy_pass http://127.0.0.1:8889/;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host              $host;
    proxy_set_header        X-Real-IP         $remote_addr;
    proxy_set_header        X-Forwarded-For   $proxy_add_x_forwarded_for;
}

# Outputs a JSON Response
#location /crl/ {
#    proxy_pass http://127.0.0.1:8888/api/v1/cfssl/crl;
#    proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
#    proxy_redirect off;
#    proxy_buffering off;
#    proxy_set_header        Host              $host;
#    proxy_set_header        X-Real-IP         $remote_addr;
#    proxy_set_header        X-Forwarded-For   $proxy_add_x_forwarded_for;
#}

location / {
    proxy_pass http://127.0.0.1:8888/;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host              $host;
    proxy_set_header        X-Real-IP         $remote_addr;
    proxy_set_header        X-Forwarded-For   $proxy_add_x_forwarded_for;
}

References

• https://github.com/cloudflare/cfssl/blob/master/certdb/README.md
• https://medium.com/@vrmvrm/setup-cloudflare-cfssl-with-ocsp-responder-aba44b4134e6
• https://propellered.com/posts/cfssl_setting_up/
• https://propellered.com/posts/cfssl_setting_up_ocsp_api/
• https://propellered.com/posts/cfssl_revoking_certs_ocsp_reponder/
• https://www.howtoforge.com/tutorial/integration-of-cfssl-with-the-lemur-certificate-manager/

• Technology
• How-To

• Ssl
• Tls
• SysAdmin