Shubham Ojha
I spend most of my time around inference runtimes, compilers, and low-level systems. This is where I publish what I learn while building them.
Tracing where LLM inference spends its time and turning the useful parts into Helios.
Projects
HeliosLLM inference serverPython · CUDA · Async I/O
An inference server for continuous batching, cache management, and request scheduling.
SysDesignVaultsystem design study toolNext.js · TypeScript · D3
A system-design study tool with runnable simulations and connected concept maps.
This websiteterminal portfolioNext.js · TypeScript · WebGL
A writing-focused portfolio with an optional interactive terminal.
Open source contributions
Selected changes merged across infrastructure and developer tooling.
agentgateway/agentgateway
- Split AI proxy failures into request and response phases so HTTP status codes and telemetry identify the responsible side.
- Added focused unit and integration coverage for unsupported request and upstream response encodings.
istio/istio
- Replaced fixed waits in TestGetCNIConfigFilepath with direct context-cancellation checks.
- Cut the race-enabled test average from 1.142 seconds to 0.685 seconds.
More projects
View project index15 repositories
About
I'm a software engineer based in Mumbai. I tend to learn by rebuilding inference servers, compilers, databases, and queues. Writing is how I find the gaps in what I think I understand.
Before getting pulled into ML systems, I spent a lot of time on lower layers. That compulsion has produced a lot of redundant software and most of my blog posts.
- Inference
- Python / CUDA / Async I/O
- Systems
- C / C++ / Go / Java
- Interfaces
- TypeScript / Next.js / D3 / Three.js