Things I built that have a name.

Generic Parser framework

The config-driven Scala Spark framework that 400+ production ETL jobs run on. I rebuilt its plugin model, wrote most of its connectors, and moved it from DStreams to Structured Streaming.

• About 25% more throughput per core after the Structured Streaming migration, with no changes to job manifests.
• Streaming Stats, a custom Spark UI tab built into the framework: per-batch history, true Kafka lag in records and minutes, phase breakdowns.
• Typed plugin models, partitioned JDBC reads, and a runtime Scala notebook plugin.

Migration write-up →Spark UI write-up →

ack

Sequence diagrams on a real time axis. Links have latency and work takes time, so arrows slant, and a playhead sweeps down while packets travel at the right relative speed.

• Drop any message and the flow runs again: timeouts, retries with backoff, hangs, and work done twice when a retried request has no idempotency key.
• Imports Mermaid sequence diagrams, exports SVG that animates by itself, and has an MCP server for coding agents.
• The timing figures in the outage write-ups here are drawn with its core.

ack.fin1te.com Editor

lab

Small private tools for data and platform engineers. All the work happens in the page, and a Content Security Policy stops the pages from making network requests at all, so nothing you paste leaves the browser.

• cronmap: paste Kubernetes CronJobs, crontabs, Airflow DAGs or GitHub Actions workflows and see a week heatmap on your clock, the crowded minutes, and the runs daylight saving skips or doubles.
• schema: JSON, Avro, Protobuf, a Spark schema or a CREATE TABLE, converted to twelve formats with a note wherever a target loses something, plus compatibility checks between two versions.
• nines: turn an availability target like 99.95% into the downtime it allows per day, week, month and year, or work out the availability of a system built from parts in series and with redundant copies.
• wrap: when a 16, 32 or 64-bit counter wraps at a given rate, and how many wraps can fit between two polls. It is the arithmetic behind the 497-day bug.
• sparkmem: where a Spark executor's memory goes (reserved, execution, storage, user, overhead), per core and across the cluster.

lab.fin1te.com cronmap schema nines wrap sparkmem

ds

The tokens, components and diagram styles behind this site, topo, ack and lab. One stylesheet in cascade layers, 13.7 KB gzipped, light and dark, no framework.

• Pinned versions stay published and never change. Each product vendors a copy and refreshes it with a script.

ds.fin1te.com

Platform MCP server

A Model Context Protocol server that lets AI coding tools look up schemas, scaffold pipelines, validate transformations and run tests against the internal data platform.

Bastion-as-a-Service

Three bastion engines deployed side by side on Kubernetes (JumpServer, Warpgate, Guacamole) and compared on credential vaulting, session recording, command filtering and clientless access.

HackOverflow

Official app for HackOverflow, a three-day national hackathon I co-organised. Digital passes, countdown, schedule, team profiles, and Google sign-in for registered participants.

GitHub

CovidInfoLeads

One app for the second wave: live case data, vaccine slots, bed and oxygen leads, helplines and teleconsultation links, pulled together when that information was scattered across a dozen sources.

GitHub

GDSC PHCET app

Contributed to the community app for the Google Developer Student Club chapter I led: events, resources and announcements.

GitHub

fin1te.com

A static Astro site with no client framework, so each page ships a few kilobytes of JavaScript. Diagrams in the posts are drawn at build time with topo and ack, every page gets its own share card, and curl fin1te.com returns a plain-text version of the home page.