Learnitweb

How mTLS (Mutual TLS) Works

1. What is mTLS?

Mutual TLS (mTLS) is an extension of standard TLS (Transport Layer Security) where:

  • In TLS, the server proves its identity to the client.
  • In mTLS, both the server and the client authenticate each other using digital certificates.

This ensures:

  • Only trusted clients can connect to the server.
  • Both parties are verified cryptographically.

2. mTLS Communication Flow (Step-by-Step)

Below is a textual sequence diagram of the flow:

Client                         Server
  |                               |
  | --- Client Hello ------------>|
  |     (includes supported ciphers, TLS version)
  |                               |
  |<--- Server Hello -------------|
  |     (includes server cert, session info)
  |                               |
  |<--- Certificate Request ------|
  |     (asks client for its certificate)
  |                               |
  |--- Client Certificate -------->|
  |--- Client Key Exchange ------->|
  |--- Certificate Verify -------->|
  |--- Finished ------------------>|
  |                               |
  |<--- Server Finished -----------|
  |                               |
  |==== Secure mTLS Session Established ===|
  |                               |
  |--- Encrypted Request --------->|
  |<--- Encrypted Response --------|

3. Detailed Explanation of Each Step

3.1 Client Hello

  • The client initiates the handshake.
  • Sends:
    • Supported TLS versions
    • Cipher suites
    • Random number

3.2 Server Hello

  • The server responds with:
    • Selected TLS version and cipher suite
    • Random number
    • Its server certificate (signed by CA)

3.3 Server Certificate

  • The server sends its X.509 certificate to prove its identity.
  • The client validates the certificate:
    • Is it signed by a trusted CA?
    • Is it expired?
    • Is the domain name valid?

3.4 Certificate Request (mTLS-specific)

  • The server requests the client certificate.
  • This is where mTLS diverges from normal TLS.

3.5 Client Certificate

  • The client sends its own X.509 certificate (signed by same or trusted CA).
  • The server verifies the client certificate just like the client verified the server.

3.6 Key Exchange

  • Both sides now generate a shared secret using asymmetric cryptography (e.g., RSA or ECDHE).
  • All future communication will be encrypted using this shared secret.

3.7 Certificate Verify

  • The client proves it has the private key for the public key in its certificate by signing part of the handshake data.
  • Prevents spoofing.

3.8 Finished Messages

  • Both client and server send a “Finished” message encrypted with the shared secret.
  • Confirms that both sides have the same keys and handshake was successful.

3.9 Secure Communication Begins

  • Encrypted messages are exchanged using the negotiated session keys.
  • Neither side can deny its identity — this is mutual authentication.

4. How Spring Boot Enforces mTLS

When you enable mTLS in Spring Boot (client-auth: need), the embedded server (like Tomcat) is configured to:

  • Accept only HTTPS traffic
  • Request and validate the client certificate before processing any request

If the client does not present a valid certificate:

  • The handshake fails.
  • The request is rejected before it even reaches your controller.

5. mTLS vs TLS

FeatureTLSmTLS
Server authenticationYesYes
Client authenticationNo (optional)Yes (required)
Use CasePublic websites, APIsInternal APIs, microservices
Protection levelOne-way trustMutual trust

6. Security Benefits of mTLS

  • Strong client authentication using certificates
  • Prevents unauthorized clients from accessing APIs
  • Eliminates shared secrets/passwords
  • Mitigates MITM (Man-in-the-Middle) attacks

7. Real-world Use Cases

  • Internal microservice-to-microservice communication in a secure environment
  • Banking APIs, payment gateways
  • Healthcare systems exchanging sensitive data
  • IoT devices communicating with secure backend

8. Key Things Used in SSL/TLS in Java

8.1 Keystore

8.1.1 What is it?

A keystore is a secure storage file that holds private keys and the corresponding public certificates for your application. It essentially represents your application’s identity in the SSL/TLS ecosystem.

  • Private keys in the keystore are used to prove ownership of a certificate during SSL handshakes.
  • Public certificates in the keystore are shared with clients or other servers to allow them to establish encrypted connections.
  • Keystore types include JKS (Java KeyStore) and PKCS12 (more widely compatible).
  • Keystore passwords are critical for security; they protect the private key.
  • A keystore can also store certificate chains, not just single certificates.

8.1.2 How it works:

  • During a TLS handshake, the server presents its certificate from the keystore to the client.
  • The client can then verify this certificate against a trusted authority.
  • In mutual TLS (mTLS), clients also provide their certificate from a keystore.

8.1.3 Example (Spring Boot):

server.ssl.key-store=classpath:server.keystore.p12
server.ssl.key-store-password=changeit
server.ssl.key-store-type=PKCS12

This tells the Spring Boot server to use the keystore for its SSL identity.

8.2 Truststore

8.2.1 What is it?

A truststore is a secure storage file that holds trusted certificates, typically from Certificate Authorities (CAs) or self-signed certificates you trust. Unlike a keystore, it does not contain private keys.

  • It represents the list of entities your application trusts.
  • The truststore is used to verify the identity of the peer during the SSL/TLS handshake.
  • Truststores can contain multiple trusted certificates (root and intermediate CAs).
  • Proper truststore management ensures only verified and authorized peers can communicate.
  • You can use keytool to import and export certificates to truststores.

8.2.2 How it works:

  • When a client connects to a server, it checks the server’s certificate against its truststore.
  • If the certificate matches a trusted CA in the truststore, the connection continues; otherwise, it is rejected.
  • In mutual TLS, the server uses a truststore to validate client certificates.

8.2.3 Example (Spring Boot mTLS):

server.ssl.trust-store=classpath:client.truststore.jks
server.ssl.trust-store-password=changeit
server.ssl.trust-store-type=JKS
server.ssl.client-auth=need

8.3. Private Key

8.3.1 What is it?

A private key is a secret cryptographic key associated with a certificate. It is used to:

  • Sign data, proving the server or client owns the certificate.
  • Decrypt information sent encrypted with its corresponding public key.

8.3.2 Key Points:

  • Must remain confidential; compromise allows impersonation of your server/client.
  • Stored in the keystore, never in the truststore.
  • Used in both encryption/decryption and digital signature verification during SSL handshakes.

8.3.3 Example:

  • Server uses its private key to decrypt the pre-master secret sent by a client in TLS handshake.
  • This secret is then used to generate session keys for encrypting communication.

8.4 Public Certificate

8.4.1 What is it?

A public certificate contains:

  • The public key of the entity.
  • Identity information (e.g., server name, organization).
  • Issuer information (who signed the certificate).
  • Digital signature (signed by a CA or self-signed).

8.4.2 Purpose:

  • Shared with peers to allow them to encrypt messages or verify signatures.
  • Ensures that communication is with a legitimate entity.

8.4.3 Key Points:

  • Clients receive the certificate from the server during SSL handshake.
  • They use it to validate that the server is who it claims to be.
  • Public certificates can be self-signed for internal use or signed by a trusted CA for public-facing systems.

8.5 Certificate Chain

8.5.1 What is it?

A certificate chain is a sequence of certificates linking your server’s certificate to a trusted root CA.

  • Includes the server’s certificate, any intermediate CA certificates, and ends with a root CA.
  • Ensures the client can trace trust from a known CA to your certificate.

8.5.2 Example:

Server Cert -> Intermediate CA -> Root CA

8.5.3 Key Points:

  • Clients and servers rely on this chain to verify authenticity.
  • A broken or incomplete chain will cause SSL handshake failure.
  • Keystores usually store the entire chain for proper validation.

8.6 Certificate Authority (CA)

8.6.1 What is it?

A Certificate Authority is a trusted organization that issues digital certificates.

  • It verifies the identity of the entity requesting a certificate.
  • Signs the certificate using its own private key.

8.6.2 Examples:

  • Let’s Encrypt, DigiCert, GoDaddy

8.6.3 Key Points:

  • The trust of SSL communication depends on trusted CAs.
  • Certificates issued by a CA are recognized by default in client truststores (browsers, OS).
  • Using a CA-signed certificate avoids warnings for public clients.

9. How these are used in TLS Handshake in mTLS

Step 1: Client Hello

  • Client initiates SSL/TLS connection.
  • Sends supported ciphers, TLS version, and a random number.

Step 2: Server Hello

  • Server responds with selected cipher, TLS version, and server random.
  • Sends its certificate from keystore.
  • Requests client certificate (if mTLS is required).

Step 3: Client Certificate & Key Exchange

  • Client sends certificate from keystore.
  • Verifies server certificate against truststore.
  • Encrypts pre-master secret using server’s public key.

Step 4: Server Verification & Key Exchange

  • Server verifies client certificate against truststore.
  • Uses private key to decrypt pre-master secret.
  • Both sides generate session keys for symmetric encryption.

Step 5: Secure Communication

  • Both sides now use the symmetric session keys for encrypted data transfer.