Agentic AI Developer

Job Description

Role: Agentic AI Developer

Location: Washington, DC

Duration: 5 Months

Hybrid Onsite: 3/4 days per week from Day 1.

Key Responsibilities

• Design, implement, and operate Retrieval-Augmented Generation (RAG) services using Azure AI/Search, including chunking, embeddings, re-ranking, evaluation, and citation display.

• Design and deploy Model Context Protocol (MCP) tools/servers to integrate security scanners, inventory systems, approvals, and Azure DevOps/GitHub services.

• Build agentic AI solutions using AutoGen, CrewAI, and/or Agno, enabling secure tool-calling and multi-agent orchestration for troubleshooting and workflow automation.

• Develop production-grade chatbots (multi-turn, retrieval-grounded) with prompt management, guardrails, audit logging, and telemetry.

• Integrate Azure OpenAI securely behind API Management (APIM), manage secrets with Key Vault, handle events via Event Hub, and instrument with App Insights/Log Analytics.

• Evaluate and (where appropriate) fine-tune open-source models (e.g., PEFT/LoRA), balancing quality, latency, cost, and safety.

• Ship with CI/CD on Azure DevOps, implement unit/integration tests, red-team for prompt-injection/jailbreaks, and document runbooks.

Minimum Qualifications

• 4-8 years total software development experience, with 2+ years in applied LLM/GenAI.

• Strong Python skills and hands-on experience with Azure OpenAI and Azure AI/Search (vector search, hybrid search, semantic ranking).

• Practical experience with agent frameworks (AutoGen, CrewAI, Agno) and MCP/tool-use patterns.

• Proven Azure PaaS experience: Azure Functions or Web Apps, APIM, Key Vault, Event Hub; familiarity with Entra ID/RBAC and secure API design.

• Experience implementing observability (App Insights, Log Analytics/KQL) and CI/CD with Azure DevOps.

Nice to Have (including Certifications)

• RAG evaluation frameworks (e.g., Ragas), custom golden sets, KQL proficiency, Cosmos DB familiarity.

• Security-first mindset: content safety, prompt-injection defenses, data privacy controls, and threat modeling for AI systems.

• Experience with cost monitoring/optimization of LLM workloads and latency tuning.

• Certifications (any of):

o Microsoft Certified: Azure AI Engineer Associate (AI-102)

o Microsoft Certified: Azure Developer Associate (AZ-204)

o Microsoft Certified: Azure Solutions Architect Expert (AZ-305)

o Microsoft Certified: Azure Data Scientist Associate (DP-100)

Full-time

Hybrid remote