Varun Raj is a software engineer with deep expertise in performance engineering, distributed systems, and large-scale infrastructure. He has built and led foundational systems that improve reliability, efficiency, and latency across mission-critical platforms, with a career spanning industry-leading technology organizations and academic research institutions. At Google, Varun has worked on core infrastructure that underpins globally scaled services. His contributions focus on intelligent request routing, performance evaluation frameworks, andproduction-grade optimization systems. He has designed architectures that adapt dynamically to real-time system conditions, enabling services to operate with greater efficiency andpredictability under heavy load. His work has supported complex machine learning deployments by creating the performance headroom required for advanced models to operate reliably. In addition, he has played a central role in improving observability, allowing engineering teams to isolate performance regressions quickly and make data-driven decisions with confidence. Varun has also led initiatives to modernize release and deployment infrastructure, reducing operational overhead while increasing engineering velocity. Through close collaboration withreliability, productivity, and product teams, he has helped establish standardized, automated systems that improve production readiness and reduce manual intervention. His work reflects a consistent focus on building durable platforms that scale with organizational and technical growth. Before Google, Varun was a software engineer at Oracle, where he focused on building systems and developer infrastructure. He drove the transition from monolithic build processes todependency-aware systems, enabling faster feedback cycles and more reliable continuous integration across large engineering organizations. His efforts contributed to a culture ofhigh-velocity development supported by robust tooling.  Earlier in his career, Varun conducted research in theoretical cryptography at the National University of Singapore. His academic work advanced the understanding of non-malleable codesand secure cryptographic constructions, contributing original insights to the field.  Varun holds a degree in computer science and engineering with a strong foundation in mathematics from the Indian Institute of Technology, Guwahati. His background reflects a rare combination of theoretical rigor and practical systems expertise, enabling him to solve complex problems at scale with clarity and precision.

Scaling Tail-Latency Diagnostics with Hybrid Sharding and Outcome-Aware Thresholding for Global Distributed Traffic In large-scale distributed systems handling billions of requests, average latency metrics fail to represent the real user experience. The most critical performance failures occur in the long tail, particularly at the p99 latency, where a small subset of requests encounters fundamentally different execution paths, resource contention, or infrastructure bottlenecks. Diagnosing these rare events requires high-overhead telemetry that traditional uniform sampling cannot capture efficiently, often resulting in either blind spots or prohibitive system cost.This talk presents a production-grade telemetry architecture designed to deliver precise visibility into tail-latency behavior while preserving system efficiency. The system addresses the precision gap in tail monitoring by adopting an outcome-aware diagnostic model. Instead of sampling requests randomly, it evaluates latency outcomes after request completion and selectively captures detailed profiling data only for requests that exceed experiment-specific p99 thresholds. This enables statistically meaningful tail analysis across thousands of concurrent experiments without overwhelming compute, memory, or network resources. At the core of the design is a centralized controller that continuously maintains rolling p99cutoffs for each active experiment. Serving tasks report minimal latency metadata via lightweigh RPCs, and detailed key-value diagnostics are conditionally triggered only for tail events. These high-fidelity metrics are persisted in an OLAP-backed system and exposed through a real-time query layer, allowing developers to retrieve experiment-specific latency distributions instantly. As global traffic diversity increased, the system evolved to support hybrid sharding. The metrics persistence layer was horizontally sharded by experiment identifiers to scale with traffic volume and sliced latency profiles, while threshold computation logic remained centralizd to preserve a consistent global view. This hybrid design enables accurate tail diagnostics across heterogeneous traffic slices without sacrificing coordination. By integrating this telemetry framework directly into both production and load-test environments, engineers can now observe p99 behavior early and iterate rapidly. The system demonstrates how scalable observability, not raw throughput, ultimately defines innovation velocity in modern distributed systems.