Private certificate authorities

If your organisation runs its own certificate authority, or a proxy that terminates TLS and re-signs it, the portal's outbound connections will not verify until it trusts that authority. The symptom is usually a scan that fails while cloning, or vulnerability feeds that stop refreshing.

This page is about that case. It is not about air-gapped operation, where there is no outbound connection to secure at all.

Two steps, in this order

Put the certificate inside the containers.
Point the tools at it.

Neither works alone, and the second one is where a correct-looking setup can still be wrong, because the tools do not all read the same variable and the same variable does not mean the same thing to all of them.

Putting the certificate in

Mount it read-only into the backend, both workers and beat, and set the variables that point the tools at it in the same place. With Docker Compose, add an override file next to docker-compose.yml:

# docker-compose.override.yml
x-corp-ca-env: &corp-ca-env
  SSL_CERT_FILE: /etc/ssl/corp-ca.pem
  NODE_EXTRA_CA_CERTS: /etc/ssl/corp-ca.pem
  REQUESTS_CA_BUNDLE: /etc/ssl/corp-ca.pem
  GIT_SSL_CAINFO: /etc/ssl/corp-ca.pem

services:
  backend:
    volumes:
      - /etc/ssl/corp/ca-bundle.pem:/etc/ssl/corp-ca.pem:ro
    environment: *corp-ca-env
  worker-scan:
    volumes:
      - /etc/ssl/corp/ca-bundle.pem:/etc/ssl/corp-ca.pem:ro
    environment: *corp-ca-env
  worker-default:
    volumes:
      - /etc/ssl/corp/ca-bundle.pem:/etc/ssl/corp-ca.pem:ro
    environment: *corp-ca-env
  beat:
    volumes:
      - /etc/ssl/corp/ca-bundle.pem:/etc/ssl/corp-ca.pem:ro
    environment: *corp-ca-env

Compose reads docker-compose.override.yml automatically, so upgrades that replace docker-compose.yml leave your settings alone, and it adds these variables to each service's own environment rather than replacing it. Apply a change with docker-compose up -d. If you run the bomlens-engine profile, add the same entries for worker-engine; it stays off unless that profile is started.

The file must hold your authority and the public roots, concatenated. The next section says why.

cat /path/to/corp-root-ca.pem /etc/ssl/certs/ca-certificates.crt \
  > /etc/ssl/corp/ca-bundle.pem

With Helm

Create the Secret yourself, then let the chart mount it and set the variables:

kubectl create secret generic corp-ca --from-file=ca-bundle.pem=/etc/ssl/corp/ca-bundle.pem

env:
  extraVolumes:
    - name: corp-ca
      secret:
        secretName: corp-ca
  extraVolumeMounts:
    - name: corp-ca
      mountPath: /etc/ssl/corp-ca.pem
      subPath: ca-bundle.pem
      readOnly: true
  extraEnv:
    SSL_CERT_FILE: /etc/ssl/corp-ca.pem
    NODE_EXTRA_CA_CERTS: /etc/ssl/corp-ca.pem
    REQUESTS_CA_BUNDLE: /etc/ssl/corp-ca.pem
    GIT_SSL_CAINFO: /etc/ssl/corp-ca.pem

subPath is what makes the certificate a file rather than a directory over that path, and without it the variables above name something that is not there. The mount goes to the backend, the scheduler and both workers, and not to the frontend or Redis, which make no outbound calls.

The same bundle rule applies: your authority and the public roots in one file, for the reason in the next section.

Pointing the tools at it

The environment entries in the override above do this. Keep them there with written-out values, not in .env or the shell: docker-compose.yml does not pass these names on. The docker-compose 1.29.2 release binary sets SSL_CERT_FILE in its own environment to a temporary path that changes on every run, so a compose file that took the name from there would hand the containers a file that does not exist and recreate them on every up.

Four names for one file, because the tools disagree. Setting all four is the short answer; the table says which one each of them needs, for when only part of the pipeline is failing.

Variable	Read by	Effect
`SSL_CERT_FILE`	Trivy, cosign, govulncheck, and the portal's own HTTPS calls	Adds for the Go tools, which keep reading the system directory. Replaces for the portal itself.
`SSL_CERT_DIR`	The same set	A directory of hashed certificates, an alternative to the file.
`NODE_EXTRA_CA_CERTS`	cdxgen	Adds. The built-in roots stay.
`REQUESTS_CA_BUNDLE`	scancode, scanoss	Adds for those two. The portal's own calls ignore it.
`GIT_SSL_CAINFO`	`git clone`	Adds. git reads neither `SSL_CERT_FILE` nor `CURL_CA_BUNDLE`.
`GIT_SSL_CAPATH`	`git clone`	The directory form of the above.

Two of those rows are worth reading twice.

SSL_CERT_FILE replaces the trust set for the portal's own connections. The vulnerability feeds, licence lookups, Slack and Teams notifications, the GitHub and GitLab calls and the ticket webhook all go out through one HTTP client, and that client builds its trust from this file alone. Point it at a file holding only your authority and every public endpoint becomes unverifiable, while Trivy and cdxgen keep working because they still consult the system store. The result reads as a feed outage rather than a certificate problem. Concatenating the public roots into the same file, as above, avoids it.

git clone reads only its own two variables. If cloning is the step that fails and everything else works, GIT_SSL_CAINFO is the one that was missing.

Going out through a proxy

Proxy settings do not go in .env; docker-compose.yml does not pass them from there. Put them in the proxies block of the Docker client's configuration, ~/.docker/config.json, for the user that runs docker-compose:

{
  "proxies": {
    "default": {
      "httpProxy": "http://proxy.corp.example:3128",
      "httpsProxy": "http://proxy.corp.example:3128",
      "noProxy": "localhost,127.0.0.1,backend,frontend,postgres,redis,docker-socket-proxy,.corp.example"
    }
  }
}

Compose injects these into every container it creates, as HTTP_PROXY, HTTPS_PROXY and NO_PROXY and as their lower-case forms. Both cases matter, because the tools do not agree on which one they read. git clone goes through libcurl, which ignores upper-case HTTP_PROXY and reads only http_proxy. Python, which means the portal's own calls, scancode and scanoss, prefers lower case. Trivy, cosign and the other Go tools prefer upper case. Changing this file does not change existing containers, so recreate them with docker-compose up -d --force-recreate.

If you would rather keep the proxy with the deployment, set the same six names with explicit values in docker-compose.override.yml:

# docker-compose.override.yml
x-proxy-env: &proxy-env
  HTTP_PROXY: http://proxy.corp.example:3128
  HTTPS_PROXY: http://proxy.corp.example:3128
  NO_PROXY: localhost,127.0.0.1,backend,frontend,postgres,redis,docker-socket-proxy,.corp.example
  http_proxy: http://proxy.corp.example:3128
  https_proxy: http://proxy.corp.example:3128
  no_proxy: localhost,127.0.0.1,backend,frontend,postgres,redis,docker-socket-proxy,.corp.example

services:
  backend:
    environment: *proxy-env
  worker-scan:
    environment: *proxy-env
  worker-default:
    environment: *proxy-env
  beat:
    environment: *proxy-env

Compose adds these to each service's own environment rather than replacing it. If you already have an override file from the certificate step, put the x-proxy-env block in the same file and give each service both sets with environment: {<<: [*corp-ca-env, *proxy-env]}.

NO_PROXY keeps the stack's own traffic off the proxy. The service names in the example are the containers that talk to each other; docker-socket-proxy matters only with the bomlens-engine profile. Add your internal Git host, registry and package mirrors (a leading dot covers a whole domain) when the proxy cannot reach them.

A proxy that re-signs TLS also needs its certificate authority, which is the rest of this page. With Helm, set the same six names under env.extraEnv.

Checking it worked

The API, each worker and beat state their own trust set at boot, and name themselves. Every line this page is about carries the prefix tls_trust, which is what to filter on. A worker's own output is mostly scan progress, so the line is there and unfindable without it:

docker-compose logs backend worker-scan worker-default beat | grep tls_trust

tls_trust.outbound  process=api     authorities=140 bundled_authorities=120 source=SSL_CERT_FILE path=/etc/ssl/corp-ca.pem
tls_trust.outbound  process=worker  authorities=140 bundled_authorities=120 source=SSL_CERT_FILE path=/etc/ssl/corp-ca.pem
tls_trust.outbound  process=worker  authorities=140 bundled_authorities=120 source=SSL_CERT_FILE path=/etc/ssl/corp-ca.pem
tls_trust.outbound  process=beat    authorities=140 bundled_authorities=120 source=SSL_CERT_FILE path=/etc/ssl/corp-ca.pem

Look for all of them, one process=worker line from each worker. Compose gives each service its own environment, so a certificate configured for one and missed on another is an ordinary mistake, and the workers are the ones that matter most: the scanners reach the network from there. A process=worker line that still reads source=bundled means that worker never got the setting.

authorities is how many certificate authorities the portal's own HTTPS calls will accept and bundled_authorities is how many it ships with. A number larger than the bundled one means your authority was added to the public roots, which is what you want in most deployments.

If the file replaced them instead, a warning follows:

tls_trust.public_roots_dropped  authorities=1 bundled_authorities=120 ...

That is the mistake described above. Concatenate the public roots into your bundle, or ignore the warning if trusting only your own authority is deliberate, which it is on a network that reaches nothing else.

When SSL_CERT_DIR is used instead of a file, authorities reads null. A trust store built from a directory loads certificates as it needs them, so there is no count to give, and printing zero would be worse than printing nothing.

Known limits

Container image pulls performed by the Docker daemon are outside the portal's configuration. If a container scan fails to pull an image from an internal registry behind a private authority, the certificate has to be installed for the daemon on that host; nothing in the portal's environment reaches it.

Two steps, in this order​

Putting the certificate in​

With Helm​

Pointing the tools at it​

Going out through a proxy​

Checking it worked​

Known limits​