Under secure two-party inference, a transformer's cost is eaten by Softmax and GELU, not the matrix multiplies. Stacked per-operation communication and interaction rounds for one BERT-base block across Iron, BumbleBee, and Nimbus — the nonlinear activations are 80–95% of the bill.
Secure multi-party computation runs a transformer without any party seeing your prompt — at minutes per query and gigabytes per token. BumbleBee's BERT-base: 6.4 GB, 2.55 min; LLaMA-7B: ~8 min/token. The bytes don't go where you'd think, and it's why 'blind' networks fall back to TEEs.