Rishabh Mehta

Moved down the stack into the infrastructure layer of a bare-metal GPU cloud, building from the ground up: out-of-band DPU provisioning, a private WireGuard mesh, bastion access, and automation for every firewall and proxy change.

9 / 9 DPF health checks passing on the first DPU node0 agents installed on tenant bare-metal hosts3 bastion engines built and compared on Kubernetes

BlueField-3 · DPF · Redfish · Kubernetes · Headscale · WireGuard · F5 BIG-IP · Temporal

Migrated the Generic Parser framework, and the hundreds of streaming jobs running on it, off the legacy DStream API and onto Spark Structured Streaming, then built the observability the stock Spark UI does not have.

400+ production Spark ETL jobs on the framework~25% better throughput per core across the fleet95 s → 44 s median batch on the hardest job after tuning

Scala · Spark Structured Streaming · Kafka · Catalyst · SparkPlugin · Kubernetes

Turned the lessons from the MyJio rewrite into a framework of my own, so the next sixteen migrations would be configuration rather than code.

1st internal Cargo registry in the organisation7+ observability pipelines on Rust in the first wave

Rust · tokio · rdkafka · serde · ClickHouse · Kafka · Artifactory · Azure DevOps

Rewrote the real-time log parser for MyJio in Rust, took it to production, and turned it into the template for the rest of the fleet, including Ajio and Netmeds.

480 → 10 CPU cores, MyJio pilot480 GB → 4.2 GB peak memory, MyJio pilot10 s → 2.3 s end-to-end batch latency

Rust · rdkafka · tokio · Kafka · Kubernetes · Spark 2 · Perl

Scaled a nationwide IP detail record platform: ingestion, schema and sort-key design, and the day-to-day work of keeping a very large ClickHouse estate healthy.

200+ PB data under management2 T+ rows in the largest tables<20 ms subscriber lookups, down from hundreds of ms

ClickHouse · ClickHouse Keeper · Linux · Bash · Python · SSH

Brought agentic tooling into the team early, then built the platform's own Model Context Protocol server so an engineer can scaffold, validate and test-run a pipeline from their editor.

MCP · JSON-RPC · SSE · Python · Kubernetes · Azure DevOps

A stretch of unglamorous work that removed whole classes of problems rather than individual bugs.

155 / 167 CVEs fixed in a platform image497 days uptime counter bug found and fixed

Scala · Spark · SNMP · Zabbix · Oracle · Elasticsearch · Docker · Azure DevOps

Control-plane APIs that give cloud customers managed Spark on Kubernetes, from onboarding to offboarding.

Python · FastAPI · Kubernetes · Helm · Vault · Oracle

Designed and built the daily engine that decides which seconds of downtime count against cloud SLAs, for every server, VM, storage array and network device on the platform.

4 sources reconciled daily: monitoring, events, CMDB, change tickets1 s resolution for HA partner overlap0 rows overwritten; every correction is auditable

Scala · Spark · Zabbix · Oracle · Elasticsearch · SQL

The in-house, config-driven Scala Spark ETL framework behind hundreds of production jobs. I restructured it around typed plugin models and wrote most of the connectors.

Scala · Spark · JDBC · Elasticsearch · Oracle

Log and metric shipping for the whole cloud platform, and the tools to see when it breaks.

Kafka · Vector · Fluent Bit · Prometheus · Logstash · Elasticsearch · Python · Helm

Owned the pipelines that turn resource usage into invoices across the platform. Correctness matters more than speed here: every mistake is somebody's bill.

PySpark · Oracle · GCS · Azure Blob · REST

Spark on Kubernetes, cloud cost and catalog pipelines, and the shared libraries every job still uses.

PySpark · Scala · Kubernetes · Elasticsearch · Kafka