I build LLM products, and measure them with evals.

I’m Rohit, an applied AI engineer in India working remotely. I ship retrieval and multi-agent systems end to end, then prove they work on a golden set rather than a demo. Open to full-time roles and contract work.

Based in
India · Remote
Focus
RAG · Agents · Evals
Stack
TypeScript · Python
Rohit Raj
About

A short version of the story.

Most of my work sits in the unglamorous middle of applied AI, where a model meets a real product and almost everything that can go wrong, does. I build the layer that catches it: retrieval that refuses before it invents, agents that stay inside a budget, and ingestion that never leaves a source half-done.

The part I care about most is the measuring. Every retrieval technique in Curio sits behind a flag and stayed only because an ablation showed it earned its place. Before that I spent six months as one of the first engineers at an early-stage startup, building the MVP the founders took into fundraising.

Certificate of Winner, OpenAI x NamasteDev Codex Hackathon, awarded to Rohit Raj for placing in the top 10 Top 10 of 3,000+ OpenAI × NamasteDev Codex Hackathon, Jul 2026. Built CommentSense solo.

Currently

  • Multi-agent orchestration with planner and critic loops
  • RAG over messy sources: hybrid search, rerank, relevance gating
  • Golden-set evals, LLM-as-judge and per-technique ablations
  • Tracing and cost controls on everything that calls a model

@rohhitrz

Last 6 months
—
Longest streak
—

Loading…

Interests

  • LLM evals
  • Retrieval & reranking
  • Multi-agent systems
  • Prompt injection
  • Developer tools
  • Context engineering
Experience

Where I’ve worked.

  1. Nov 2025 – Present

    Independent AI Engineer · Remote

    Building and shipping LLM products full time: RAG, multi-agent systems and coding agents, with evals, tracing and cost controls.

  2. Apr 2025 – Oct 2025

    Software Developer · Zapygo Energy

    One of the first engineers. Built the working MVP of all three products (customer app, driver app, admin dashboard) that the founders needed ahead of fundraising. REST APIs in Node and Postgres for auth, orders and driver assignment, plus Google Maps and Razorpay integrations.

  3. Apr 2024 – May 2024

    ML Intern · SkillDzire

    Built a water-potability classifier over 3,276 samples, comparing seven models from logistic regression through XGBoost and SVM.

  4. 2020 – 2024

    B.E., Information Science · New Horizon College of Engineering, Bengaluru

    Graduated with First Class with Distinction.

Stack

The toolkit.

Chosen for boring reliability, not Twitter hype. Mostly.

GenAI & Agents

  • OpenAI API
  • Gemini API
  • OpenRouter
  • Vercel AI SDK
  • Tool calling
  • Structured outputs
  • Streaming (SSE)
  • Multi-agent orchestration
  • Planner / critic
  • Context compaction

RAG & Evals

  • Chunking
  • Embeddings
  • pgvector
  • Hybrid search (RRF)
  • Query rewriting
  • Cohere rerank
  • Relevance gating
  • Golden datasets
  • LLM-as-judge
  • Hit@k / MRR
  • Langfuse

Backend & Infra

  • Node.js
  • Express
  • Next.js
  • Inngest
  • PostgreSQL (Neon)
  • Drizzle
  • Prisma
  • Upstash Redis
  • Cloudflare R2
  • Docker
  • Vercel

Languages & Tools

  • TypeScript
  • Python
  • JavaScript
  • SQL
  • FastAPI
  • uv
  • React
  • Tailwind
  • Vitest
  • Zod
  • Pandas
  • scikit-learn
Work

A few things I’ve built.

A handful of projects I’d actually defend.

Curio

A NotebookLM-style learning workspace. Chat with PDFs, sites, YouTube and notes. Every answer is cited to the page, timestamp or section, or it says the answer isn’t in your sources. Query rewriting, hybrid search fused with RRF, Cohere rerank, and a relevance gate that refuses before any model call.

  • Hit@5 1.00 from 0.93
  • MRR 0.98 from 0.80
  • 0.98 faithfulness
  • 0/45 injections landed
  • Next.js
  • pgvector
  • Inngest
  • Cohere
  • Langfuse

Hive

A multi-agent research orchestrator. A lead agent plans the sub-questions, an agent factory writes each specialist’s prompt and budget, and they research in parallel on a shared blackboard. A critic checks every finding against its evidence and spawns gap-filling rounds, so the report cites only what survived.

  • 7 agents one run
  • 30 citations
  • $0.044 model spend
  • TypeScript
  • Vercel AI SDK
  • Postgres
  • SSE
  • Tavily

CommentSense

Built solo for the OpenAI × NamasteDev Codex Hackathon, where it placed 10th of 3,000+. Paste a YouTube URL and get sentiment, themed clusters, the top five questions your audience is asking, and a drafted reply per cluster. Compare mode finds complaints that repeat across up to five videos.

  • Next.js 16
  • gpt-4o-mini
  • YouTube Data API
  • Recharts

nanocode

A terminal coding agent in ~630 lines of Python. It reads, searches and edits code, runs shell commands and browses the web, with plan mode, sub-agents, context compaction, and a permission prompt before anything touches your system.

  • Python
  • OpenRouter
  • Firecrawl
  • Tool use

jonaki

An agent SDK for TypeScript with Anthropic, OpenAI and Google behind one interface. Record a real run once, then replay it offline and deterministically. Your tools still execute; only the model is replayed. Change a tool and see exactly what breaks, in milliseconds, for free.

  • TypeScript
  • Provider-agnostic
  • Record / replay

IELTS Coach

AI-powered IELTS practice across writing and speaking, with band scores against the real criteria and detailed per-criterion feedback. Clerk auth, rate limiting and input validation, because a free LLM endpoint on the open internet does not stay free.

  • Next.js 15
  • React 19
  • gpt-4o-mini
  • Clerk
Writing

Notes on the work.

Occasional pieces on retrieval, agents and the parts of applied AI that only show up in production.

Where RAG Fails: Understanding the Limitations

RAG gets sold as the cure for hallucination: add your own data and the model stops inventing things. Then a pipeline ships, the demo questions pass, and somewhere between the query and the answer it quietly loses the plot. This is where that happens and why.

How ChatGPT Understands Your Questions

You type a question and an answer comes back that reads like someone understood it. Nobody did. Silicon only knows numbers, so it never reads your English at all. What actually happens between pressing Send and seeing a reply.

Services

What I can help with.

Open to full-time engineering roles and to contract work, from a single scoped build to a longer engagement. Remote.

Retrieval that can be trusted

RAG over PDFs, transcripts and internal wikis: chunking, hybrid search, rerank, and a relevance gate that refuses rather than invents. Shipped with a golden set so the quality is a number, not an impression.

Agents that finish the job

Tool-using agents and multi-agent runs with planner and critic loops, held inside a cost and latency budget. Tracing on every model call, so a bad run can be read back rather than guessed at.

Evals and LLM security

Golden sets, LLM-as-judge and per-technique ablations to prove a change helped. Plus the adversarial half: prompt-injection testing, guardrails and dependency scanning before any of it meets real users.

The backend underneath

Node and FastAPI services with auth, background jobs, durable ingestion and observability. The unglamorous layer that decides whether the clever part survives real traffic.

Contact

Let’s build something.

Open to full-time roles, contracts, and interesting collaborations. The fastest way is email.

Get in touch

I read every message, including the recruiter copy-paste ones. I just reply to the others first.

rohhit.rz@gmail.com