Production LLM & RAG Data Scientist

Sagar Darji Data Scientist

Seattle, WA +1-682-***-**** *****.*****.**@*****.*** LinkedIn GitHub SUMMARY

Data Scientist and AI Engineer with 3+ years of experience building and deploying production-grade AI systems, specializing in large language models, generative AI, and machine learning at scale. Focused on designing agentic AI systems, retrieval-augmented generation (RAG) pipelines, multi-agent architectures, and robust evaluation frameworks that improve reliability, safety, and performance in enterprise environments. Proven experience across the full stack of AI systems, including backend services, data pipelines, model evaluation, and cloud deployment using Python, Go, and AWS in distributed architectures. Strong track record of translating complex business problems into scalable AI solutions, reducing hallucinations, improving system behavior in production, and delivering measurable impact through reliable, cost- efficient, and production-ready AI systems.

PROFESSIONAL SKILLS

Programming Languages: Python, Go, JavaScript, TypeScript, SQL AI / Machine Learning: Large Language Models (LLMs), Generative AI, Agentic AI, Multi-Agent Systems, Retrieval-Augmented Generation (RAG), Prompt Engineering, LLM Evaluation, LLM-as-a-Judge, Hallucination Mitigation, AI Guardrails, Embeddings, Semantic Search

Machine Learning Techniques: Supervised & Unsupervised Learning, Time Series Forecasting (SARIMA, Prophet), Regression, Classification, Clustering, Feature Engineering, Cross-Validation, Anomaly Detection (Isolation Forest), Recommendation Systems Search & Retrieval Systems: Vector Databases, FAISS, pgvector, ChromaDB (HNSW), Hybrid Search, Dense & Sparse Retrieval, Information Retrieval

Backend & Distributed Systems: Microservices, REST APIs, System Design, Distributed Systems, Async Programming (asyncio), High- Performance APIs, API Design & Integration

Cloud & DevOps: AWS (Lambda, S3, Bedrock), Terraform, Serverless Architecture, Docker, Kubernetes, CI/CD Data Engineering: ETL Pipelines, Data Processing Pipelines, Event Streaming, Change Data Capture (CDC), Data Modeling Databases: PostgreSQL, Amazon Aurora, SQLite

MLOps & AI Systems: Model Evaluation, Offline Evaluation Frameworks, Monitoring & Observability, Experimentation, Regression Testing, AI System Reliability, Prompt Injection Protection Computer Vision & Multimodal AI: Multimodal AI, CLIP, OCR Pipelines, Image-to-Text Extraction Tools & Monitoring: Git, Prometheus, Grafana, Linux AI/ML Frameworks: FastAPI, LangGraph, Model Context Protocol (MCP), Ollama, Vapi, Groq, AWS Bedrock EXPERIENCE

Data Scientist DoorDash Seattle, WA, USA Jan 2025 - Present

• Engineered an automated, offline LLM-based customer simulation framework that generated realistic, multi-turn conversational pushback and escalation scenarios, accelerating prompt engineering and system calibration cycles by 40% while eliminating the need for live canary testing.

• Mitigated chatbot hallucinations driven by context-window information overload by designing an intelligent raw-event filtration and summarization layer, reducing token overhead by 35% and stabilizing downstream function-calling accuracy.

• Replaced highly variable, subjective LLM scoring systems with a strict, policy-driven binary (pass/fail) LLM-as-a-Judge framework, establishing reproducible regression suites across hundreds of simulated workflows to catch edge-case policy violations prior to deployment.

• Developed and integrated a real-time LLM Guardrail and programmatic intercept layer into the production retrieval-augmented generation

(RAG) pipeline, decreasing severe compliance issues by 99% across thousands of daily active multi-turn support chats.

• Formulated an offline evaluation harness using domain-specific rubrics to enforce groundedness, reducing hallucination rates by 90% by systematically catching stale internalized model knowledge gaps.

• Designed a hybrid multimodal fusion pipeline for structured text extraction from unformatted menu images, incorporating a lightweight LightGBM confidence classifier that optimized the human-in-the-loop fallback rate and outperformed pure deep-learning architectures in low-labeled data constraints.

• Built an automated pipeline to identify support content gaps by clustering millions of escalated chat transcripts using embedding-based cosine similarity (threshold-tuned between 0.70–0.90), translating raw conversational text into structured, high-ROI actionable insights.

• Streamlined internal content creation workflows from weeks to minutes by engineering an LLM-based drafting system that synthesized cluster centroids and exemplar customer resolutions into production-ready Knowledge Base articles. Software Engineer HHAeXchange Homecare Software Solutions, LLC Frisco, TX, USA Jan 2022 - Jan 2024

• Engineered a Retrieval-Augmented Generation (RAG) chatbot for the Operations team that indexed 1,000+ SOP documents and used LLM-based semantic search and prompt engineering to resolve SEV (severity incident) lookups in seconds instead of manual SOP recall deployed on AWS, cutting incident-resolution lookup time and reducing dependency on tribal knowledge across the on-call team.

• Designed and deployed ML forecasting pipelines (SARIMA, Prophet) with custom feature engineering and cross-validation, improving budget/resource forecast accuracy by 25% (MAPE 12% 9%) on $50M+ annual revenue directly informing decisions tied to 15% business growth and 57% operational efficiency gains.

• Built an ML-driven cloud cost optimization platform using Isolation Forest anomaly detection on AWS serverless infrastructure (Lambda, S3), analyzing 2M+ resource-usage records to surface $9M+ in annual savings via automated rightsizing and reserved-instance optimization.

• Developed "Museum of HHA," an autonomous milestone-prediction system using time-series models and regression to forecast business achievements (100+ clients, $5M+ revenue, 1M+ patient records), replacing manual tracking with real-time dashboards and automated Slack alerts improving org-wide visibility by 80%. PROJECTS

Nexus - Agentic AI Platform for Enterprise Operations Tech: Python, Claude (Anthropic), AWS Bedrock, Model Context Protocol (MCP), Multi-Agent Systems, asyncio, RAG, FAISS, pgvector, PostgreSQL, Amazon Aurora, REST APIs, Vapi, Terraform, AWS

• Architected an enterprise-scale multi-agent platform utilizing a Planner-Worker-Aggregator topology, enabling Claude to autonomously execute multi-hop natural language queries across a 219-endpoint API via Model Context Protocol (MCP).

• Slashed end-to-end inference latency by 5.3x (from 3.45s to 0.65s) and managed API load by engineering server-side parallel composition, deterministic response caching, and strict token-bucket rate limiting.

• Collapsed thousands of sequential API lookups into sub-second indexed queries by building a high-throughput hybrid retrieval system

(FAISS/pgvector) fed by an incremental Aurora PostgreSQL CDC pipeline.

• Spearheaded eval-driven development with 106 integration tests and deployed HIPAA-compliant programmatic guardrails, including structured-output schema validation and prompt-injection-resistant access controls. Mantis - Local-First AI Coding Assistant

Tech: Go, Ollama, SQLite, Tree-sitter, Model Context Protocol (MCP), Language Server Protocol (LSP), BM25, kNN, Reciprocal Rank Fusion (RRF), Multi-Agent Systems

• Engineered a high-performance, local-first coding assistant as a 48K-LOC Go CLI tool across 34 modular packages, establishing a secure, low-latency alternative to cloud-native coding agents.

• Minimized inference costs and processing latency by developing a 7-tier intent routing engine (kNN, LRU caching) and a hybrid retrieval architecture fusing BM25 and cosine-similarity (RRF).

• Optimized context assembly over a SQLite-backed AST dependency graph built with Tree-sitter for Go, TypeScript, and Python to enable automated impact analysis and real-time hallucination verification.

• Orchestrated a resilient multi-agent execution framework that decomposes software tasks into parallel worker threads, utilizing an iterative test-repair loop guarded by automated stuck-detection heuristics. CineMatch AI: Multi-Agent Movie Recommendation System Tech: Python, FastAPI, LangGraph, ChromaDB (HNSW), CLIP, Sentence Transformers, Llama 3.1 70B (Groq), React, TypeScript, Docker, Kubernetes, AWS Lambda, Prometheus, Grafana, RAG, Multi-Agent Systems

• Deployed a containerized 7-agent LLM recommendation pipeline processing the MovieLens 25M dataset, powered by a multi-modal RAG layer fusing dense textual and visual poster features (CLIP) inside a ChromaDB HNSW index.

• Achieved a 2-4% lift in recommendation accuracy by engineering an autonomous reinforcement loop that dynamically monkey-patches runtime configurations and executes 50+ parallel offline grid searches per run.

• Systematically eliminated recommendation filter bubbles by designing a composite evaluation framework utilizing adversarial "Critic" and "Serendipity" agents to quantify user intent across five custom dimensions.

• Maintained rigorous production standards across a 46K-LOC full-stack repository (Python/TypeScript), managing microservices via Kubernetes, enforcing strict API rate limits, and building observability with Prometheus and Grafana. EDUCATION

Master of Science in Data Science Aug 2025

The University of Texas at Arlington, USA (GPA: 3.9/4.0) Coursework: AI, Deep Learning, Neural Networks, MLflow, Weights & Biases, Computer Vision, NLP, Statistical Analysis Bachelor of Engineering in ICT Jun 2022

Gujarat Technological University, India (GPA: 3.54/4.0) Coursework: Data Structure and Algorithm, Database Systems, Machine Learning, Big Data Analytics, Data mining CERTIFICATIONS

AWS Certified Solutions Architect - Associate

The Linux Foundation: FinOps Certified

Contact this candidate