Available · Q4 2026


building production-grade

Rahul Kundapur Ramesh: from cloud infrastructure and LLM optimization to responsive frontends. I ship the whole stack, and lately I'm spending most of my time on the model side.

See selected workGet in touch
Based
Raleigh, NC · Remote-friendly
Focus
LLM inference, Go data planes, agentic infra
Stack
Go · Python · TypeScript
Years
~2.5 yrs shipping production software
LLM systemsEvals & guardrailsCloud infrastructureAgentic workflowsDistributed systemsType-safe APIsDesign-conscious frontendsProduction AILLM systemsEvals & guardrailsCloud infrastructureAgentic workflowsDistributed systemsType-safe APIsDesign-conscious frontendsProduction AI
01 · Now

What's on my desk.

Current snapshot, not a resume bullet. Updated when reality changes.

SHIPPING

Go LLM gateway with SSE streaming

A multi-tenant gateway that normalises OpenAI and Anthropic dialects behind one canonical request model, then streams each dialect back faithfully. TTFT and TPOT are measured on the same dispatch seam that does the work, so the numbers aren't an approximation.

EXPLORING

Closed-loop SGLang tuning

A control harness that adjusts prefill width and chunk size against p95 tail latency, then makes every move prove it improved throughput before keeping it. Averages hide the violations you actually care about.

READING

Constitutional AI - Anthropic

Working through Anthropic's paper on training AI to be helpful, harmless, and honest using a set of principles rather than purely human feedback.

↓ Scroll · selected work06 entries
02 · Selected work

Nine things I'm proud of.

A mix of AI infra, full-stack product, and the unsexy plumbing that makes the rest possible. Case studies are available on request; most are under NDA, but all are honest.

012026-nowGo · InfraFounding Engineer

Multi-tenant LLM
gateway in Go

  • One canonical request model normalises OpenAI and Anthropic dialects behind a single dispatch seam, preserving raw payload bytes it cannot model faithfully.
  • SSE streaming engine frames each dialect back the way its own clients expect — near-passthrough for OpenAI, synthesised lifecycle for Anthropic.
  • TTFT and TPOT measured on the seam that serves the traffic, validated against a sub-2ms p50 framing budget on a live loopback harness.
  • Default-deny egress NetworkPolicies isolate in-VPC inference, with a render-time interlock that blocks a chart claiming isolation it doesn't enforce.
GoSSEKubernetesHelmAWS Bedrock
Read case study
022026-nowAI · PerformanceFounding Engineer

Closed-loop
SGLang inference tuning

  • Tunes prefill width and chunk size against p95 tail latency rather than window means — the mean read healthy at 25.5ms while p95 breached an 80ms budget.
  • Every tuning move must prove a ≥5% prefill-throughput gain or it is automatically reverted.
  • Caught a six-window runaway (1600 → 6102) where growth was quietly reducing throughput from 9,225 to 8,662 tok/s.
  • Replayable telemetry captures the controller's own measurement window, so any decision can be re-examined byte-identically.
PythonSGLangvLLMZMQpytest
Read case study
032025-nowAI · BackendFounding Engineer

Agentic content
editing framework

  • Content-agnostic agentic editor on LangGraph and dependency injection: new content types plug in via config alone, with zero core-node changes.
  • 3-phase split pipeline (setup, token-level SSE streaming, validation with auto-retry) decouples streaming from LangGraph's node-update model.
  • Targeted edit/split/delete/insert operations replace full-document regeneration, cutting token consumption ~60% and preserving untouched content verbatim.
PythonLangGraphDjangoPostgreSQLSSE
Read case studyidvibes.ai
042026-nowAI · Full-stackSolo build

This site's
RAG chat

  • Streaming RAG over chunked portfolio data, built against a free tier that allows twenty requests a day.
  • A semantic answer cache short-circuits repeat questions in ~40ms instead of round-tripping the model.
  • Cache vectors truncated 3072 → 768 dims via Matryoshka prefix slicing, cutting storage 4× while holding retrieval quality.
  • On quota exhaustion it serves matched canned answers, then an honest 503 carrying Retry-After derived from the upstream's own signal.
Next.jsTypeScriptGeminiUpstash RedisUpstash Vector
Read case study
052024Cloud · AISolo build

Serverless
video analysis pipeline

  • Event-driven AWS pipeline: an S3 upload triggers Lambda video splitting and FFmpeg frame extraction.
  • Face recognition via OpenCV, ResNet-34, and SSD across the extracted frames.
  • Sub-270s end-to-end for 100 concurrent videos, with Docker image size cut 60% using multi-stage builds.
PythonAWS LambdaS3OpenCVFFmpeg
Read case study
062024ML · ResearchTeam lead (5)

Heart disease detection
model comparison

  • Led a 5-person team comparing five classification algorithms on a clinical dataset.
  • Evaluation prioritised real-world viability over raw accuracy: precision, recall, and AUC across decision trees, SVMs, neural nets, and ensembles.
  • A high-accuracy model that misses true positives is worse than a lower-accuracy one with better recall — the comparison was built around that.
Pythonscikit-learnPandasJupyter
Read case study
DTKNNSVMNNRF
03 · About

A short biography.

I started writing software in college, took my first full-time role in 2021, and have been shipping production software since. I'm still suspicious of any system I can't draw on a napkin.

I work across the stack: infra to interface, but my favourite seat is where distributed systems meet product: queues, retries, evals, and the small UI affordances that let humans trust them.

The last two years have been almost entirely AI-shaped: agent runtimes, then the layer underneath them. Lately that means Go data planes that stream tokens without adding latency, inference servers tuned against p95 rather than an average, and egress rules that make tenant isolation a property of the cluster instead of a promise in a doc.

Off-screen I watch fiction, play games, hike, and travel whenever I can.

04 · Stack

Tools I reach for.

Roughly ranked by current depth, not by enthusiasm. I keep this list short by design. The long-tail isn't useful to anyone reading.

Languages
  • Go
  • Python
  • TypeScript / JavaScript
  • Java · C#
  • SQL · C++
  • Kotlin · Dart
AI · Inference
  • SGLang · vLLM serving
  • TTFT / TPOT · tail SLOs
  • LLMs · RAG · agentic systems
  • LangChain · LangGraph
  • PyTorch · TensorFlow
  • Anthropic · OpenAI · open-weight
Backend · Cloud
  • Go microservices · SSE
  • Django · FastAPI · DRF
  • REST API design · Node.js
  • AWS (EKS, Lambda, S3)
  • Docker · Helm · Terraform
  • PostgreSQL · CI/CD
Security
  • Secret scanning · vuln triage
  • Kubernetes NetworkPolicies
  • Default-deny egress · tenant isolation
  • AWS OIDC keyless federation
  • OAuth · JWT rotation
  • Tenant-isolation & IDOR review
Frontend · Mobile
  • React · Next.js
  • Flutter (Android + iOS)
  • Material UI · Tailwind CSS
  • Responsive design · a11y
  • Streaming & optimistic UI
  • Design tokens · theming
05 · Experience

A working history.

The condensed version. A long-form CV is available on request; it mostly just adds bullet points to these four entries.

Nov 2025 - Present

Traction Layer AI · Raleigh, NC

Founding Engineer

Two independent products, built in parallel: a multi-tenant LLM inference gateway deployed into tenant VPCs, and an AI content platform.

Forward-deployed inference platform
  • Built a multi-tenant LLM gateway in Go, normalising OpenAI and Anthropic dialects behind one dispatch seam without rewriting raw payload bytes.
  • Wrote the SSE streaming engine with TTFT/TPOT measured on the serving path itself — p50 framing overhead under 2ms on a live loopback harness.
  • Tuned SGLang serving against p95 tail latency instead of window means, reverting any move that couldn't prove a throughput gain.
  • Isolated in-VPC inference behind default-deny egress NetworkPolicies, making tenant isolation a property of the cluster rather than a promise in a doc.
GoPythonSGLangKubernetesHelmTerraform
AI content platform
  • Architected a LangGraph agentic editing framework with dependency injection, cutting token consumption ~60% via targeted edits over full-document regeneration.
  • Hardened the auth stack with HttpOnly cookies, JWT rotation with post-rotation blacklisting, and absolute 24-hour session timeouts.
  • Closed a cross-tenant IDOR on unscoped querysets and enforced single-active-row invariants at the database level.
  • Migrated the platform from Azure to a 3-tier AWS VPC with keyless OIDC deploys and zero-data-loss RDS cutover.
PythonTypeScriptLangGraphDjangoNext.jsAWS
Graduated May 2025

Arizona State University · Tempe, AZ

M.S. Computer Science - GPA 4.0 / 4.0

Relevant coursework: Artificial Intelligence, Statistical Machine Learning, Cloud Computing, Foundation of Algorithms, Software Security, Knowledge Representation, Data Processing at Scale.

AIMLCloudAlgorithmsSecurity
Sep 2021 - Jun 2023

Cognizant Technology Solutions · Bengaluru, India

Jr. Software Engineer

Security tooling, production incident response, and .NET service hardening across a two-year stint.

  • Engineered a Python + Pandas pipeline to triage an 1,800-item vulnerability backlog, reducing critical threats by 25%.
  • Led incident response for production outages on AWS, restoring service within SLA windows and authoring post-mortems.
  • Re-architected a .NET authentication service to replace hardcoded secrets with service accounts, closing a critical API security flaw.
  • Automated weekly operational reporting, cutting manual effort by 90%.
PythonC#.NETAWSPandas
Graduated May 2021

Dr. Ambedkar Institute of Technology · Bengaluru, India

B.E. Computer Science - CGPA 9.08/10

Relevant coursework: Cloud Computing, Neural Networks, AI & Machine Learning, Internet of Things, DBMS, Operating Systems.

CloudAIMLIoTDBMS
06 · Contact

The fastest way to reach me. I read every email; I reply within a few days.

Let's buildsomething real.
Email
rahulkundapurr@gmail.com
GitHub
@rahul-k-r
LinkedIn
/in/rahul-kundapur