A RAG system built in 2.5 weeks to walk into a CEO meeting with proof

Context

I was preparing to fly to Seoul to meet the CEO of Medical Avenue, a Korean medical-tourism company, about a long-term engineering engagement. I had no background in their industry, and I didn’t want to compensate with slides. The most convincing thing I could bring, I decided, was a working system in their domain — so I spent the two and a half weeks before the flight building one.

Problem

Medical tourism’s institutional knowledge is genuinely fragmented: a couple of foundational books, a scatter of academic papers, and hundreds of hours of practitioner interviews sitting unindexed on YouTube. Nobody in the industry has read all of it, and an outsider has no fast path to a trustworthy understanding. An assistant over that corpus would be useful — but in a medical-adjacent domain, a chatbot that invents facts is worse than no chatbot. Every answer had to be traceable to a source, or refused.

Constraints

• About 2.5 weeks of calendar time before the meeting, working alone.
• No contract yet, so no company data — the corpus had to be built entirely from public sources.
• My own hardware and budget, which pushed the first iteration toward local GPU embeddings instead of per-token API costs.
• It had to demo live to a non-technical audience: real sign-up, real dashboard, real chat — not a notebook.

My role

All of it: the ingestion framework, the retrieval backend, the admin dashboard, deployment, and the demo itself.

Approach

The system is two halves. The offline half is a LangChain ingestion pipeline: collect transcripts from two industry YouTube channels, parse two industry books and seven papers (with a local vision model for figure-heavy PDF pages), restructure raw transcripts into clean, sectioned markdown with an LLM, split into semantic chunks of roughly 300 tokens on spaCy sentence boundaries, embed, and store in PostgreSQL with pgvector — a Source → Document → Chunk hierarchy with content hashing so re-runs are idempotent.

The online half is a FastAPI retrieval API — HNSW cosine index, MMR reranking for diversity — plus a React dashboard for managing sources, documents, chunks, and API keys. The chat surface is OpenWebUI with a custom tool: it queries the retrieval API, drops results below a relevance threshold, and emits citations that link back to the exact document in the dashboard.

The final corpus: 264 interview videos, two books, seven papers.

Key decisions

• Grounding as the product, not a feature. The system prompt and the tool contract enforce one rule: answer only from retrieved content, cite every claim, and say so when the corpus has nothing relevant. In this domain, “I don’t know” is a correct answer and a fabricated one is a liability.
• Local embeddings first, hosted when it won. I started with BGE-M3 on my own GPU to keep a 1,400-chunk ingestion run free, then migrated the query path to OpenAI’s text-embedding-3-small and added MMR once quality and operational simplicity mattered more than cost. The trade was explicit both times.
• Don’t build a chat UI. OpenWebUI already does chat well; a tool integration cost a day. The time that saved went into retrieval quality and the ingestion pipeline, which is where the demo would actually win or lose.
• Production shape, even for a prototype. JWT auth plus API keys, role-based access, rate limiting, migrations, Docker Compose, and a written onboarding guide. The demo was the competence signal — a system that cut corners would have argued against me.

Trade-offs

Restructuring raw transcripts with an LLM trades verbatim fidelity for retrievability — cleaner sections embed and retrieve better, but the text is no longer exactly what was said. I measured the drift with an embedding-similarity comparison between raw and structured versions rather than assuming it was fine. I also skipped hybrid BM25 + vector search and parent-document retrieval to hold the deadline, and wrote them down in an architecture review as the known next steps. And switching embedding models mid-project invalidated every stored vector — the real price of starting local.

Result

I walked into the meeting with a live assistant that answered domain questions with citations over the full corpus, and the pipeline and codebase that built it. The meeting ended in a founding-engineer contract. The prototype was retired after that — it was built to prove capability in an unfamiliar domain on a deadline, and it did exactly that.

What I’d do differently

Start on hosted embeddings from day one: the local-first savings were real but small, and the model switch forced a re-embedding pass that a stable choice would have avoided. And build a small retrieval evaluation set before tuning anything — I flagged hybrid search and reranking as next steps in my own review, but without an eval harness those would have been guesses too.