Skip to main content

Command Palette

Search for a command to run...

Internal TLS communication between pods in kubernetes using cert-manager

Published
5 min read

Recently I have been experimenting using load balancer and AKS and still able to run everything securely. I was using Let’s encrypt cert

  • Issuer / ClusterIssuer: objects that define certificate sources (e.g., SelfSigned, CA, ACME). Issuer operates within a namespace, ClusterIssuer — globally.

  • Certificate: describes the desired TLS certificate for a service. cert-manager automatically creates the certificate and places it in a Secret.

  • CertificateRequest: internal object that cert-manager generates when processing a Certificate. Contains CSR request.

  • Secret: Kubernetes object where the ready certificate is stored (tls.crt, tls.key, ca.crt).

  • SelfSigned / CA Issuer: certificate sources that don’t require external services. Suitable for internal infrastructure.

  • Renewal: automatic certificate renewal before expiration (typically 30 days before).

  • Solver (for ACME): domain ownership verification mechanism (HTTP01, DNS01). Not relevant for SelfSigned/CA.

cert-manager:
  crds:
    enabled: true

Basically if you are using the cert-manager helm chart, this is where you create your values.yaml file and put in the cert-manager

Next, in your certificate if you design your own helm template

certificate.yaml

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: test-{{ .Values.env }}-cert
spec:
  secretName: test-{{ .Values.env }}-cert
  dnsNames:
{{- range $domain := .Values.dnsZones }}
{{- range $subdomain := $.Values.subdomains }}
    - "{{ $subdomain }}{{ $domain }}"
{{- end }}
{{- end }}
  issuerRef:
    name: letsencrypt-{{ .Values.env }}
    kind: ClusterIssuer
    group: cert-manager.io
  signatureAlgorithm: SHA384WithRSA
  keystores:
    pkcs12:
      create: true
      password: test-{{ $.Values.k8sType }}-{{ $.Values.clusterEnv }}-cert

cluster-issuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-{{ .Values.env }}
spec:
  acme:
    email: {{ .Values.email }}
    server: {{ .Values.server }}
    privateKeySecretRef:
      name: {{ .Values.env }}-issuer-account-key
    {{- if .Values.zeroSsl.enabled }}
    externalAccountBinding:
      keyID: {{ .Values.zeroSsl.keyID }}
      keySecretRef:
        name: zero-ssl-eabsecret
        key: EAB_HMAC_KEY
    {{- end }}
    solvers:
      - selector:
          dnsZones:
{{ toYaml .Values.dnsZones | indent 12 }}
        dns01:
{{- if .Values.cloud.route53.region }}
          route53:
            region: {{ .Values.cloud.route53.region }}
            accessKeyID: {{ .Values.cloud.route53.accessKeyID }}
            secretAccessKeySecretRef:
              name: aws-secret
              key: secretAccessKey
{{- else if .Values.cloud.azureDNS.subscriptionID }}
          azureDNS:
            subscriptionID: {{ .Values.cloud.azureDNS.subscriptionID }}
            resourceGroupName: {{ .Values.cloud.azureDNS.resourceGroupName }}
            hostedZoneName: {{ index .Values.dnsZones 0 }}
            environment: {{ .Values.cloud.azureDNS.environment }}
{{- if .Values.cloud.azureDNS.managedIdentity.clientID }}
            managedIdentity:
              clientID: {{ .Values.cloud.azureDNS.managedIdentity.clientID }}
{{- end }}
{{- end }}

In your values, you need to put in this for the certificate

email: pantheonlab@gmail.com
dnsZones:
  - "abc.ai"
subdomains:
  - "*."
  - "*.k."
  - "*.abc-dev.k."
cloud:
  azureDNS:
    subscriptionID: "your subscription id5"
    resourceGroupName: "your resource groupb"
    environment: AzurePublicCloud
    managedIdentity:
      clientID: "your managed identity used to the DNS zone in Azure"

volume, this is where you can mount them

volumeMounts:
  - name: tls-cert
    mountPath: /etc/tls
    readOnly: true

volumes:
  - name: tls-cert
    secret:
      secretName: service-a-tls

How It Works

  1. Cert Issuer generates a CA certificate using lets-encrypt, stored in test-dev for example.

  2. CA Issuer uses lets encrypt to sign other certificates.

  3. For each service cert-manager:

  • creates CSR

  • signs through ca-issuer

  • stores result in Secret (*.tls), which includes tls.crt, tls.key, ca.crt

Result in Container After mounting, files appear in the container:

/etc/tls/
├── tls.crt          # Public certificate (PEM format)
├── tls.key          # Private key (PEM format)  
└── ca.crt           # Certificate authority certificate

if you require a storageclass where all the other namespace would want to use the

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azurefile-rwx
provisioner: file.csi.azure.com
parameters:
  skuName: Standard_LRS
allowVolumeExpansion: true
reclaimPolicy: Retain
volumeBindingMode: Immediate

PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-tls
  namespace: my-namespace
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: azurefile-rwx
  resources:
    requests:
      storage: 1Gi

You create the PVC in each namespace that needs access:

namespace-a: PVC shared-tls
namespace-b: PVC shared-tls

All point to the same backend Azure File Share.

Mount the shared PVC into your Pod

volumeMounts:
  - name: tls-volume
    mountPath: /etc/tls

volumes:
  - name: tls-volume
    persistentVolumeClaim:
      claimName: shared-tls

Copy Secret → Shared Volume using an InitContainer

Since Secrets cannot be mounted across namespaces, you copy them to shared storage once.

Example:

initContainers:
  - name: copy-tls
    image: alpine
    command: ["sh", "-c", "cp -r /src/* /dst/"]
    volumeMounts:
      - name: src-tls
        mountPath: /src
      - name: shared-tls
        mountPath: /dst

volumes:
  - name: src-tls
    secret:
      secretName: service-a-tls

  - name: shared-tls
    persistentVolumeClaim:
      claimName: shared-tls

Security Aspects of Mounting

tmpfs storage*: Certificates are stored in RAM, not on disk
*Automatic cleanup: When Pod is deleted, certificates automatically disappear
* RBAC validation: kubelet checks ServiceAccount permissions to read Secret
* Namespace isolation: Secret is only accessible within its namespace
* File permissions: defaultMode sets secure access permissions (0400 = read-only for owner)

Step 4.6: Practical Code Usage

# Python HTTPS client with mTLS (client.py)
import requests

response = requests.get(
    'https://service-b.default.svc.cluster.local',
    cert=('/etc/tls/tls.crt', '/etc/tls/tls.key'),
    verify='/etc/tls/ca.crt'
)

print("Status:", response.status_code)
print(response.text)
// Go example of certificate reading
cert, err := tls.LoadX509KeyPair("/etc/tls/tls.crt", "/etc/tls/tls.key")
if err != nil {
    log.Fatal(err)
}

// HTTPS server configuration
server := &http.Server{
    Addr:      ":8443",
    TLSConfig: &tls.Config{Certificates: []tls.Certificate{cert}},
}
// Node.js example using certificates
const https = require('https');
const fs = require('fs');
const express = require('express');

const app = express();

// Reading certificates
const options = {
  key: fs.readFileSync('/etc/tls/tls.key'),
  cert: fs.readFileSync('/etc/tls/tls.crt'),
  ca: fs.readFileSync('/etc/tls/ca.crt'),
  requestCert: true,        // For mTLS
  rejectUnauthorized: true  // Strict certificate validation
};

app.get('/', (req, res) => {
  res.send('Secure HTTPS server with cert-manager certificates!');
});

// Creating HTTPS server
https.createServer(options, app).listen(8443, () => {
  console.log('HTTPS Server running on port 8443');
});

// Example HTTPS client for mTLS
const clientOptions = {
  hostname: 'service-b.default.svc.cluster.local',
  port: 8443,
  path: '/',
  method: 'GET',
  key: fs.readFileSync('/etc/tls/tls.key'),
  cert: fs.readFileSync('/etc/tls/tls.crt'),
  ca: fs.readFileSync('/etc/tls/ca.crt')
};

const req = https.request(clientOptions, (res) => {
  console.log('Status:', res.statusCode);
  res.on('data', (data) => {
    console.log(data.toString());
  });
});

req.end();