Three years at Jio, and the four before.

Jio Platforms

Cloud and network foundations for India's largest AI datacenter

Moved down the stack into the infrastructure layer of a bare-metal GPU cloud, building from the ground up: out-of-band DPU provisioning, a private WireGuard mesh, bastion access, and automation for every firewall and proxy change.

DPF health checks passing on the first DPU node

9 / 9

agents installed on tenant bare-metal hosts

0

bastion engines built and compared on Kubernetes

3

• Onboarded NVIDIA BlueField-3 DPUs as ARM64 Kubernetes workers with the DOCA Platform Framework operator, driven entirely through the BMC over Redfish. The host OS is never touched, which is the point: the tenant owns the server, the platform owns the DPU.
• Built a self-hosted Headscale control plane with WireGuard peer-to-peer tunnels from engineer laptops straight into GPU pods. The WireGuard interface lives in its own network namespace so it cannot fight Calico for routes, with an internal DERP relay for networks that block UDP.
• Stood up JumpServer, Warpgate and Guacamole side by side for a Bastion-as-a-Service product, compared credential vaulting, session recording and command filtering, and set the direction for the in-house build.
• Automated create, modify and delete of F5 BIG-IP ACL rules across tenant route domains: Ravel blueprints, Temporal workflows for durable execution and rollback, Crossplane compositions for rule placement, APIGEE in front.
• Designed a tenant-isolated, deny-by-default FQDN forward proxy on F5 BIG-IP 17.5 with exact and wildcard allow-lists, and built the Superset dashboards security teams use to track proxy policy exceptions.
• Debugged cross-cluster reachability for a DBaaS offering running Postgres inside KubeVirt VMs on OVN-Kubernetes user-defined networks, down to BGP/VNI route checks, MetalLB VIPs, and why EndpointSlices pointed at virt-launcher pods never work.

Jio Platforms

Moving 400+ Spark jobs to Structured Streaming

Migrated the Generic Parser framework, and the hundreds of streaming jobs running on it, off the legacy DStream API and onto Spark Structured Streaming, then built the observability the stock Spark UI does not have.

production Spark ETL jobs on the framework

400+

better throughput per core across the fleet

~25%

median batch on the hardest job after tuning

95 s → 44 s

records lost in a 22-hour, 1,289-batch parallel run

0

• Rewrote the framework's streaming core on Structured Streaming: DataFrame-native Kafka sources, checkpointed offsets, bounded micro-batches with maxOffsetsPerTrigger, and async progress tracking.
• Added a future-skew guard so one device with a broken clock can no longer drag the watermark forward and silently drop every legitimate event behind it.
• Fixed the subtle parity bugs a migration like this uncovers: ISO-8601 offsets shifting events by 5.5 hours, window boundaries converted to IST twice, and session config that never reached foreachBatch because each micro-batch runs on a cloned SparkSession.
• Collapsed iterative withColumn calls into a single projection (SPARK-25380), taking driver-side Catalyst planning from seconds per batch to under 100 ms.
• Hardened the Kafka sink with acks=all, idempotent producers and lz4 batching, with a guard that stops jobs overriding acks from crashing the producer.
• Built "Streaming Stats", a custom Spark UI tab shipped inside the framework library: per-batch history that survives query restarts, true input rows from Kafka offset deltas, consumer lag in records and in minutes, lag trend, phase breakdowns and CSV/JSON export, all rendered as server-side SVG for air-gapped clusters.

Jio Platforms

DataCraft: a Rust ETL framework and SDK

Turned the lessons from the MyJio rewrite into a framework of my own, so the next sixteen migrations would be configuration rather than code.

internal Cargo registry in the organisation

1st

observability pipelines on Rust in the first wave

7+

• Designed dc-sdk: Kafka sources with consumer-group balancing, schema decoders, windowed aggregation, and batched sinks for ClickHouse, Kafka and object storage. Health checks, lag metrics and Kubernetes lifecycle hooks come for free.
• Built a drag-and-drop pipeline editor that emits a manifest; the manifest runs on a single native binary instead of a Spark application.
• Set up the first Cargo registry on JFrog Artifactory so Rust crates get versioning and review the same way Maven and SBT artifacts do.
• Moved Kubernetes discovery, Prometheus metrics and asset inventory pipelines onto Rust, and ported heavier batch logic onto the SDK.
• Presented the framework and SDK to the wider data engineering organisation and senior leadership.

Jio Platforms

Spark to Rust: the log-parsing fleet

Rewrote the real-time log parser for MyJio in Rust, took it to production, and turned it into the template for the rest of the fleet, including Ajio and Netmeds.

CPU cores, MyJio pilot

480 → 10

peak memory, MyJio pilot

480 GB → 4.2 GB

end-to-end batch latency

10 s → 2.3 s

RAM across 17 application groups

8.7 TB → <200 GB

• Proved field-level parity against 20,000 live production records before any benchmark, including every record that went down the legacy regex paths.
• Benchmarked 10 million records on the production topic: 120 Spark executors across ~12 nodes replaced by one pod.
• Chased a volume gap during the parallel run to thread oversubscription (48 workers on 12 cores) and a single consumer reading 110 partitions. Scaled to three pods and closed the gap to 0.14%.
• Went live with zero consumer lag. Fleet-wide, the plan takes 3,500 cores down to about 200 and retires hundreds of executor JVMs.

Jio Platforms

IPDR analytics on multi-cluster ClickHouse

Scaled a nationwide IP detail record platform: ingestion, schema and sort-key design, and the day-to-day work of keeping a very large ClickHouse estate healthy.

data under management

200+ PB

rows in the largest tables

2 T+

subscriber lookups, down from hundreds of ms

<20 ms

fewer servers than the legacy projection

~10×

• The legacy design needed 1,200+ servers and a projected 500 more. Re-thinking ingestion, partitioning and primary keys on ClickHouse brought the requirement under 180.
• Re-designed sort keys and partitioning on tables past two trillion rows. Point lookups that used to scan whole partitions now read a handful of granules and come back in under 20 ms.
• Tuned merges, parts counts, TTLs and disk IO across the processing and correlation clusters so ingestion keeps up with 10-minute batches from every region.
• Ran down production incidents: a nine-hour pipeline stall caused by SSH multiplexing sockets dying with SIGPIPE, ClickHouse Keeper quorum loss on replicated tables, and multi-hour merges under IO pressure.

Jio Platforms

An SLA engine for cloud availability

Designed and built the daily engine that decides which seconds of downtime count against cloud SLAs, for every server, VM, storage array and network device on the platform.

sources reconciled daily: monitoring, events, CMDB, change tickets

4

resolution for HA partner overlap

1 s

rows overwritten; every correction is auditable

0

• Found restarts from drops in uptime counters with window functions, and back-dated recovery from the first reading after a restart instead of trusting when monitoring noticed.
• Modelled HA pairs explicitly: downtime exploded into one-second slices and joined against the partner, so an outage is only full when both nodes were down.
• Split multi-day outages at midnight, clipped them to asset lifetimes from the CMDB, and cut them against approved change windows into planned and unplanned pieces.
• Filtered sub-minute probe jitter, rejected impossible rows, and kept every raw row as a soft-deleted record next to what was derived from it.
• Wrote Scala Spark parsers for network devices, URLs and certificates, IPSLA jitter and latency, and storage on the same model.

Jio Platforms · Cloud & Data Engineer (SDE)

Joined Jio Platforms

Spark on Kubernetes, cloud cost and catalog pipelines, and the shared libraries every job still uses.

• Built AWS and Azure product-catalog PySpark jobs, moving incremental loads from pandas to Spark anti-joins.
• Wrote a credentials library in both Python and Scala so every job decrypts secrets the same way, and moved job configuration into Kubernetes ConfigMaps with one pipeline to build and release them all.
• Researched executor sizing, backpressure and dynamic allocation for Spark Streaming, and fixed an out-of-memory crash in a production streaming job.
• Created the team's PySpark project template and a clean-up CronJob for pods stuck in ImagePullBackOff, later rolled out to every environment.

Pillai HOC College of Engineering & Technology · B.E. Computer Engineering

Graduated with a 9.3 CGPA

First rank in the department in 2020, and named Best Performer of the batch in 2023.

• Lead organiser of HackOverflow, a three-day national hackathon (Mar 2023). I also built and published its Android app.
• Judge and mentor at Ahmedabad University's hackathon (Mar 2022). Grand finalist in the Government of India Toycathon (2021).