Zero Trust encryption can add between about 1 ms and over 200 ms of latency. Cold TLS handshakes and extra proxy hops usually drive the high end.
Cold TLS handshakes and proxy round trips create the largest latency hits. Session resumption and per-request AEAD crypto add smaller, repeatable costs.
What mechanisms add latency?
TLS handshakes add round trips and CPU for new connections. Handshake cost grows with network RTT and asymmetric crypto work.
QUIC cuts handshake RTTs versus TCP plus TLS. AEAD symmetric encryption runs per request and uses CPU cycles.
Proxy decrypt and re-encrypt adds CPU and latency per request. The proxy placement and available CPU change that cost.
Endpoint CPU and network RTT often dominate perceived overhead. Measure in realistic networks and real client conditions, not only edge probes.
How to measure and attribute costs
Measure cold and warm sessions separately to isolate handshake cost. Report both cold and warm percentiles, such as 95th and 99th.
Use curl timing, app logs, and packet captures to split RTT from CPU time. Collect CPU, interrupt, and crypto-acceleration metrics during tests.
Report the measurement point with results and the CPU cost per 1k RPS, and state where measurements ran: client, edge, proxy, or server.
A frequent error is ignoring cold handshakes during tests, which understates real user impact.
Follow reproducible tests instead of vendor probe numbers to measure real user impact from client to backend.
Measuring encryption overhead
Run four test classes: network microbench, request timing, load benchmarks, and real user emulation. Each test must run cold and warm and include CPU profiling.
Use iperf3 for bandwidth baselines and curl for per-request TLS timing. Use grpc_bench for gRPC and Selenium for real browser UX.
Record 95th and 99th percentiles and capture packet traces at client, proxy, and backend. Map where time is added in each trace.
Where latency goes (typical ranges)
Cold TLS handshake: 10–200+ ms (RTT dependent)
Per-request AEAD crypto: 0.5–5 ms
Proxy decrypt/re-encrypt: 1–30 ms
Measure cold and warm separately, and capture CPU and RTT at each hop.
Minimal reproducible curl test
A compact curl command shows cold versus warm times and the appconnect phase. Run it from representative client networks.
Curl example: curl --resolve example.com:443:1.2.3.4 --tlsv1.3 --http2 -w '%{time_connect} %{time_appconnect} %{time_total}/n' -o /dev/null https://example.com/
Run once after clearing connection caches for cold tests, then repeat for warm tests. Collect one thousand samples across networks.
Load and synthetic benchmarks
Use grpc_bench for gRPC and wrk or hey for HTTP APIs. Vary keepalives, connections, and session reuse during tests.
Example: grpc_bench --concurrency=50 --qps=500 --method=unary --duration=60s
Real browser emulation
Use Selenium to measure time to interactive with and without the Zero Trust proxy. Emulate mobile CPU throttling.
Capture Real User Monitoring metrics from actual clients for validation under production conditions.
To reproduce latency claims, include copy-paste test scripts and a small harness. The harness should run iperf3, curl timing, tcpdump, and Selenium runs.
Collect tcpdump captures on client and server to attribute handshake RTT to the network or CPU. Capture system CPU and crypto counters along the runs.
Cost and hidden trade-offs of encrypting east-west traffic
Encrypting east-west traffic raises CPU and memory use across service meshes and proxies. Hidden costs include telemetry, session state, and replay protection complexity.
All-in end-to-end encryption inside the cluster increases per-pod CPU needs and complicates observability. Termination at proxies moves CPU to fewer points and changes audit controls.
Compare common choices against latency, CPU cost per 1k RPS, and compliance fit before deciding. The decision should map measured latency to compliance and infrastructure cost.
| Option |
Typical added latency (95th) |
CPU cost per 1k RPS |
Compliance fit |
| TLS termination at edge | 10–50 ms (cold), 1–5 ms (warm) | 0.5–5 vCPU | Good if backend trust is allowed; check PCI and HIPAA needs |
| End-to-end TLS (passthrough) | 10–200+ ms (cold), 1–10 ms (warm) | Higher across pods (1–10 vCPU) | Best for strong audit and data protection |
| App-layer envelope encryption | Minimal added transport latency | CPU cost shifts to app (variable) | Strong compliance when tenant controls keys |
Hidden costs to watch
Session ticket state and memory use grow with concurrent sessions. Expect tens to hundreds of megabytes per one thousand concurrent sessions for caches and buffers.
Telemetry and logging for end-to-end encrypted traffic often need sidecars or agent injection. That adds CPU and complexity to the stack.
An anonymous case about east-west costs
A mid-sized company encrypted all microservice traffic and saw a 30% CPU increase across nodes. After moving TLS termination to service mesh proxies and enabling session reuse, the company cut CPU overhead and reduced tail latency.com/solving-latency-zero-trust-network-access/).
Mitigations: reducing latency with hardware and TLS tuning
Session reuse, hardware acceleration, and protocol choice deliver the biggest wins. Tune TLS lifetimes and proxy settings to lower handshake incidence.
Offload crypto with AES-NI or dedicated TLS appliances. Consider QUIC where connection churn or high RTTs hurt user experience.
Use TLS 1.3 with session tickets for common clients. Tune TCP MSS and MTU and enable TCP_NODELAY for small interactive packets.
Which cipher suites to prefer?
Prefer AEAD suites that use hardware acceleration on endpoints. AES-GCM with AES-NI or ChaCha20-Poly1305 on ARM give fast symmetric performance.
Avoid legacy CBC suites that need extra processing and raise CPU costs under load. Ensure OpenSSL or BoringSSL builds include hardware crypto support.
Proxy and connection tuning tips
Increase session ticket lifetimes to allow reuse across short user sessions. Tune proxy connection pools to avoid frequent new TLS handshakes.
Set idle timeouts to balance resource use and handshake frequency. Tune worker thread counts to match CPU and interrupt rates.
This works well in theory, but in practice clients often drop tickets or break reuse; when clients drop connections frequently, session reuse alone will not save cold handshake costs.
A practical recommendation is to prioritize session reuse and hardware crypto, then validate on real networks before wide rollout. Measure end-to-end before and after each change to confirm gains.
Do not apply session-ticket tuning when clients cannot accept persisted tickets or when regulatory rules forbid ticket reuse.
NIST SP 800-207 (2020) discusses termination versus end-to-end encryption tradeoffs. See the NIST guidance
here.
Run side-by-side proof-of-concept tests with the provided scripts to validate vendor claims. The tests must include cold and warm session breakdowns and CPU metrics per 1k RPS.
If results exceed budgets, try session reuse settings, enable AES-NI or ChaCha20, and consider QUIC or TLS offload. Validate each change with the same tests.
Call to discuss test plans with the operations and security teams using the reproducible scripts and metrics.
Frequently asked questions
What exact latency should stakeholders expect?
Expect one to two hundred plus milliseconds of added latency depending on RTT, handshake frequency, and CPU. Cold handshakes drive the upper range.
Measure cold and warm 95th and 99th percentiles to give stakeholders numbers they can act on. Include CPU metrics to separate network from CPU causes.
How much CPU does encryption add per 1k RPS?
Typical AEAD crypto costs might map to roughly 0.5 to 5 vCPU per 1k RPS for small payloads under poor session reuse. Larger payloads or better session reuse lower the vCPU cost.
Proxy re-encrypt CPU cost varies with payload size, connection churn, and session reuse. In low-churn, large-payload scenarios it can be a fraction of a vCPU per 1k RPS.
Always report payload size, concurrency, and session reuse assumptions with vCPU estimates. Profile under realistic payloads and concurrency to refine numbers.
Do vendors' edge probe numbers reflect real user experience?
No. Vendors often measure to their edge probes, and those numbers can be much lower than end-to-end user latency. Run side-by-side tests from representative client networks.
Measure at client, proxy, and backend and use the reproducible scripts described earlier. That gives real end-user numbers for decision makers.
Can session tickets and 0-RTT remove handshake
Session tickets and 0-RTT reduce or remove the handshake RTT for warm sessions. 0-RTT has replay risks and suits idempotent flows.
Enable session tickets and longer lifetimes where security posture allows. Use 0-RTT selectively and add replay protection when needed.
How to test mobile and satellite users reliably?
Test on real devices and real mobile networks with CPU throttling and background load. For satellite, include high RTT profiles and long connection lifetimes.
Run the curl and Selenium tests from actual networks and compare cold and warm percentiles. Emulate client churn to see real behavior.
What metrics prove ROI and compliance for encryption?
Report end-user added latency (95th and 99th), CPU cost per 1k RPS, compliance mapping, and estimated incremental infrastructure cost. Tie those to business impact metrics like conversion or error rates.
Include before and after measurements from production networks and use them to justify offload or code changes. Reference NIST SP 800-207 for architectural alignment.
When is TLS passthrough preferable to termination?
TLS passthrough fits when end-to-end encryption with tenant-controlled keys is required for compliance or audit. It raises CPU footprint across services.
Choose passthrough when auditability of raw data or tenant isolation requires it. Otherwise, consider proxy termination to centralize offload.
Final recommendations and next steps
Prioritize TLS 1.3, session tickets, and hardware crypto as first steps. These steps reduce most latency without changing app logic.
If client churn or very high RTTs exist, add QUIC and move termination closer to users. Measure end-to-end before and after each change to confirm gains.
For standards context, note TLS 1.3 is RFC 8446 (2018) and QUIC is RFC 9000 (2021). Use those protocols where clients and infrastructure support them.
Run the included curl, grpc_bench, wrk, and Selenium scripts against representative client networks. Collect cold and warm 95th and 99th percentile numbers and CPU metrics.
If results exceed budgets, try session reuse, enable AES-NI or ChaCha20, and consider QUIC or TLS offload. Validate each change with the same tests.
