Senior Software Engineer, Generative AI Pipelines

Rajeev Kumar

Senior Software Engineer

Mississauga, Ontario, Canada +1-647-***-**** ******.*****@*****.*** https://www.linkedin.com/in/rajeev-kumar-

22365a398/

SUMMARY

Senior Software Engineer who designs, builds and operates production generative-AI features and

document-intelligence pipelines. Delivers reliable retrieval-augmented generation and agent systems that

integrate with operational data, implements prompt and model-versioning with regression tests, and

instruments traceable agent behavior for observability and rollback. Coaches engineers, captures reusable

libraries and standards for responsible AI, and drives production handoffs that move prototypes into cloud-

backed services.

E XPERIENCE

Shopify Senior Software Engineer (Jan 2023 – Present)

Remote

• Owned and shipped RAG features inside Shopify Checkout and Checkout Extensibility using OpenAI as the

selected model provider and FastAPI services deployed on AWS.

• Implemented a document-intelligence pipeline for order receipts and merchants' contracts using OCR, layout-

aware extraction and PostgreSQL to persist parsed outputs for downstream workflows.

• Engineered Shopify Functions that converted checkout cart context into deterministic discount and payment

decisions, integrating function outputs into the transaction path with FastAPI endpoints.

• Built Checkout UI extensions and merchant configuration surfaces, wiring UI choices to backend feature flags

and ensuring Shop Pay flows remained unchanged under extension logic.

• Developed a traceable prompt and configuration versioning system that recorded prompt revisions, model

selections and test outcomes, and integrated regression tests into GitHub Actions pipelines, reducing

undiagnosed prompt regressions by 18%.

• Instrumented agent behavior and observability with OpenTelemetry and Jaeger, adding traceable spans for

tool calls and enabling post-incident investigations with Datadog dashboards.

• Implemented permissioned access controls and RBAC for RAG queries so merchant data required explicit

consent, integrating KMS-backed secrets and JWT-based service authentication.

• Built evaluation and A/B testing hooks that ran model variants against historical commerce events and fed

MLflow with metrics for model selection, shortening model iteration cycles by 26% in wall time.

• Optimized embedding storage and vector search with pgvector on PostgreSQL for merchant document

retrieval, delivering median query latency reductions to 240 milliseconds.

• Added cost controls and fallback behaviors to OpenAI calls, routing low-confidence responses to safe canned

answers and tracking API spend per feature in S3-backed billing reports.

• Captured reusable AI components into an internal library used by Checkout and Shopify Magic, documenting

integration patterns, security checklists and rollback procedures.

Thoughtworks Senior Software Engineer (May 2021 – Dec 2022)

Toronto, Ontario, Canada

• Built a RAG prototype for a B2B SaaS startup client, benchmarking OpenAI, Anthropic and Google Gemini APIs

and delivering a Flask-based proof of concept hosted on Azure.

• Implemented retrieval stacks using LangChain and LlamaIndex with Pinecone for vector storage and pgvector-

backed indices, wiring embeddings extraction with Hugging Face Transformers for document search.

• Developed agent orchestration that constrained tool usage via permission policies and safety guards,

integrating Anthropic models for sensitive-action reasoning and instrumenting traces for audits.

• Delivered reproducible deployment patterns using Buildkite pipelines and containerized Flask services on

Azure App Service, shortening repeatable deploys to five minutes for test environments.

• Re-platformed a multi-tenant ingest pipeline into domain services that used Kafka for event streaming and

PostgreSQL for tenant state while preserving schema compatibility and entitlements.

• Built telemetry ingestion and denormalization jobs that reconciled multi-source logistics events into a

canonical schema using pandas and NumPy for ETL transforms.

• Ran model evaluation and pilot A/B experiments that compared foundation-model answers on client

documents, producing experiment logs and metrics in MLflow and reporting a 12% improvement in retrieval

precision for the winning pipeline.

• Implemented prompt and model regression tests that executed in CI, compared outputs against golden

examples and enabled automated rollbacks when thresholds were exceeded, improving rollback reliability by 3

percentage points.

• Constructed a routing and optimization microservice that consumed cleaned telemetry and produced ETA

forecasts with PyTorch models, supporting a throughput of 8,700 route forecasts per hour.

• Coached client engineers on safe-agent operation, authored transfer-ready runbooks for prompt versioning,

and delivered a standards catalog to accelerate client-side adoption.

• Paired with client data teams to tune embeddings and similarity thresholds, iterating vector-store

configurations and reducing false-positive retrievals by 19%.

Felix Founding Software Engineer (Mar 2019 – May 2021)

Toronto, Ontario, Canada

• Built the first version of Felix's online care platform as a founding engineer, taking a patient from intake

questionnaire through clinician review to a prescription order ahead of the public launch.

• Developed the clinician review tooling that let licensed physicians assess each intake, approve or decline

treatment and message the patient, replacing the email and phone follow-up the first cases ran on.

• Integrated prescription orders with pharmacy fulfilment and discreet home delivery, so an approved treatment

moved from the clinician's decision to a shipped package without anyone re-entering the order.

• Implemented recurring treatment subscriptions and payments, handling refills, plan changes and failed

charges so returning patients kept their medication without restarting the intake.

• Hardened how the platform handled personal health information as the service grew across provinces, adding

access controls and audit trails around every clinician and pharmacy action.

Microsoft Software Engineer (Jun 2017 – Mar 2019)

Vancouver, British Columbia, Canada

• Built store-operations features in Dynamics 365 for Retail, including point-of-sale flows that let retailers ring

up sales, returns and exchanges the same way in stores and online.

• Developed commerce data synchronisation between store point of sale and headquarters, so price, product

and inventory changes reached every store without waiting on an overnight batch.

• Implemented payment connector support for in-store and online payments, handling authorisations, voids

and refunds so a failed card transaction never left an order half-completed.

• Diagnosed performance and reliability defects in the retail server under peak trading load, reducing the

checkout slowdowns store associates saw during the busiest periods.

• Wrote unit and integration tests and joined the team's servicing rotation, tracing customer-reported retail

issues to root cause and shipping the fixes in regular product updates.

SKILLS

Generative & Agentic AI: OpenAI API, Anthropic API, Google Gemini API, LangChain, LlamaIndex, Hugging

Face Transformers, pgvector, Pinecone, embeddings and vector search, retrieval-augmented generation

(RAG)

AI & Machine Learning: PyTorch, TensorFlow, scikit-learn, pandas, NumPy, MLflow, model evaluation and

A/B testing, fine-tuning workflows

Languages & Backend: Python, REST, GraphQL, FastAPI, Flask, Docker, testing with pytest, API design,

service instrumentation

Databases, Search & Retrieval: PostgreSQL, Redis, Elasticsearch, OpenSearch, pgvector, Pinecone

Cloud & Platform: AWS (EC2, S3, Lambda, RDS, SQS), Azure (App Service, AKS, Azure OpenAI, Functions),

Google Cloud, Docker, Kubernetes, Terraform

Observability & Reliability: OpenTelemetry, Jaeger, Datadog, Prometheus, Grafana, tracing and agent

observability

Testing & Engineering Practices: pytest, regression testing for model/prompt changes, integration testing,

Postman, test automation

EDUCATION

University of Mumbai, Bachelor of Science, Computer Science (2014 – 2017)

Contact this candidate