Agentic AI Developer

Job Description

Benefits:

Dental insurance

Health insurance

Vision insurance

Role: Agentic AI Developer

Location: Washington, DC

Duration: 5 Months

Hybrid Onsite: 3/4 days per week from Day 1.

Key Responsibilities

Design, implement, and operate Retrieval-Augmented Generation (RAG) services using Azure AI/Search, including chunking, embeddings, re-ranking, evaluation, and citation display.

Design and deploy Model Context Protocol (MCP) tools/servers to integrate security scanners, inventory systems, approvals, and Azure DevOps/GitHub services.

Build agentic AI solutions using AutoGen, CrewAI, and/or Agno, enabling secure tool-calling and multi-agent orchestration for troubleshooting and workflow automation.

Develop production-grade chatbots (multi-turn, retrieval-grounded) with prompt management, guardrails, audit logging, and telemetry.

Integrate Azure OpenAI securely behind API Management (APIM), manage secrets with Key Vault, handle events via Event Hub, and instrument with App Insights/Log Analytics.

Evaluate and (where appropriate) fine-tune open-source models (e.g., PEFT/LoRA), balancing quality, latency, cost, and safety.

Ship with CI/CD on Azure DevOps, implement unit/integration tests, red-team for prompt-injection/jailbreaks, and document runbooks.

Minimum Qualifications

4-8 years total software development experience, with 2+ years in applied LLM/GenAI.

Strong Python skills and hands-on experience with Azure OpenAI and Azure AI/Search (vector search, hybrid search, semantic ranking).

Practical experience with agent frameworks (AutoGen, CrewAI, Agno) and MCP/tool-use patterns.

Proven Azure PaaS experience: Azure Functions or Web Apps, APIM, Key Vault, Event Hub; familiarity with Entra ID/RBAC and secure API design.

Experience implementing observability (App Insights, Log Analytics/KQL) and CI/CD with Azure DevOps.

Nice to Have (including Certifications)

RAG evaluation frameworks (e.g., Ragas), custom golden sets, KQL proficiency, Cosmos DB familiarity.

Security-first mindset: content safety, prompt-injection defenses, data privacy controls, and threat modeling for AI systems.

Experience with cost monitoring/optimization of LLM workloads and latency tuning.

Certifications (any of):

o Microsoft Certified: Azure AI Engineer Associate (AI-102)

o Microsoft Certified: Azure Developer Associate (AZ-204)

o Microsoft Certified: Azure Solutions Architect Expert (AZ-305)

o Microsoft Certified: Azure Data Scientist Associate (DP-100)

Full-time

Hybrid remote