Azure Databricks, Data Factory, Data Engineering, Pyspark

Harshith Potukuri

Senior Azure Data Engineer Databricks ADF Unity Catalog CI/CD 9+ Years

Bangalore, India +91-638******* ****************@*****.*** LinkedIn PROFESSIONAL SUMMARY

Senior Azure Data Engineer with 9+ years of experience designing and operating enterprise-scale data pipelines, cloud data platforms, and CI/CD-driven deployment automation. Deep hands-on expertise across Azure Data Factory, Azure Databricks, Databricks Asset Bundles, PySpark, Delta Lake, Unity Catalog, ADLS Gen2, Microsoft Fabric, Azure SQL, and Azure DevOps. Track record of zero-downtime production migrations, a 73.76% reduction in annual storage cost, $40K+ in annual client savings, and reliable delivery for platforms supporting 200+ business users. Growing hands-on experience in agentic AI, having led a multi-agent solution design for an internal hackathon. Strong background in production support, audit-critical remediation, platform modernization, observability, and end-to-end release ownership across enterprise data platforms. CORE SKILLS

• Data Engineering: Azure Data Factory, Azure Databricks, Databricks Asset Bundles, PySpark, Delta Lake, Unity Catalog, Microsoft Fabric, ETL/ELT, data pipeline design, data integration, batch processing, data migration.

• Cloud Platforms: Microsoft Azure, ADLS Gen2, Azure SQL, Azure Functions, Function Apps, Azure IoT, Hive Metastore.

• DevOps & Automation: Azure DevOps, YAML pipelines, Git, CI/CD, release automation, ARM-to-YAML migration, Logic Apps, Azure Automation, PowerShell, Power Automate.

• Programming: Python, SQL, PySpark, PowerShell, Pandas, NumPy.

• Monitoring & Reliability: Azure Monitor, Log Analytics, KQL, root cause analysis, production support, P1 incident remediation, audit readiness.

• Agentic AI: IBM watsonx, IBM Consulting Advantage (ICA), BOB, LangFlow, multi-agent system design, agentic AI workflows.

WORK EXPERIENCE

Application Developer - Microsoft Cloud IBM India Pvt Ltd. Nov 2021 - Present Bangalore, India.

• Design and maintain 80+ parameterized Azure Data Factory pipelines ingesting data from REST APIs, Oracle, SFTP, Blob Storage, and file systems for enterprise data integration workloads.

• Led Databricks Runtime upgrades and migrated Hive Metastore to Unity Catalog; validated notebook and pipeline dependencies and executed a controlled production rollout for critical platform workloads.

• Reduced annual Azure storage cost by 73.76% through an ADLS Gen1-to-Gen2 migration and file format modernization, delivering $40K+ in annual client savings.

• Led a zero-downtime migration of SharePoint integrations from deprecated ACS authentication to certificate-based Entra ID and OAuth 2.0 using PnP PowerShell and Azure Automation, completing ahead of Microsoft's April 2026 deadline.

• Built YAML-based CI/CD pipelines in Azure DevOps, replacing ARM-template deployment patterns and enabling Databricks CI/CD; migrated repositories to a new Azure DevOps organization with zero downtime.

• Automated file movement, EOD deals monitoring, and data-loading workflows with Logic Apps and Azure Automation, eliminating 3+ hours of manual effort per day.

• Built Databricks-to-Logic Apps notification and escalation workflows with SendGrid integration, eliminating manual alerting for production data processes.

• Upgraded the Azure Data Factory Oracle connector from V1 to V2 ahead of retirement, validating deployment across environments with zero-downtime rollout.

• Led urgent P1 remediation by extracting and validating Azure Data Factory pipelines, Azure Functions, and Logic Apps from production for audit-compliance requirements.

• Delivered coordinated production releases spanning ADF, database scripts, Power BI, and front-end CI/CD for 200+ business users without service disruption.

• Standardized source repositories for Catalyst integrations; recognized by IBM Consulting leadership for ownership and delivery quality.

Member Technical Staff HCL Technologies Ltd. Jul 2019 - Nov 2021 Chennai, India.

• Integrated PI Servers, AF Servers, PASX, ELIMS, and SAP into a pharmaceutical manufacturing data quality platform using ODBC and SimApi connectors.

• Developed PySpark workloads in Azure Databricks to curate L1 manufacturing data, resolved Azure Data Factory pipeline failures, and produced KPI reporting outputs for leadership teams. Associate Systems Engineer CGI Inc. Apr 2017 – Jul 2019 Chennai, India.

• Built Python and MySQL data pipelines powering daily, weekly, monthly, and reconciliation reporting for fuel card transaction analytics.

• Owned month-end billing incident resolution for reporting workflows, supporting reliable report delivery.

• Supported Windows Server operations including Active Directory, DNS, DHCP, and GPO, and led L1/L2 incident management under ITIL processes.

PROJECTS

Efforts Estimation Workbench - IBM WatsonX Challenge (Team Lead) - Internal Hackathon Submission (2026)

• Led team "IBM Plains Runtime Rebels" to architect and ship an Al-powered work-breakdown-structure

(WBS) effort-estimation platform, pairing a watsonx / ICA Agentic Studio estimation workflow with a React

(MUI 5, Vite) web app for architects and delivery managers.

• Built the end-to-end architecture - Cloudflare Worker proxy, GitHub Pages hosting, GitHub Actions CI/CD, and a PII-scrubbing ingestion pipeline - using IBM Bob's Plan and Agent modes, taking the application from blank workspace to a live production deployment in a single build session (~6 hours vs. an estimated 3-4 days of manual development).

• Cut requirement effort-estimation time by ~90% (from 3-5 days to under 10 minutes per document) while achieving 100% WBS role coverage across 7 delivery roles, at $0 hosting cost on free-tier infrastructure. AI Code Migrator - IBM Bob-a-thon (Team Lead) - Internal Hackathon Submission (2026)

• Led a team to design and build a multi-agent code migration solution using IBM Consulting Advantage (ICA), BOB, and LangFlow, converting legacy source code (e.g., Asp.Net, VB6, Classic ASP, C#.Net, VB.Net) into modern target languages such as .NET Core or .Net 8.

• Architected the multi-agent workflow spanning code parsing, transformation, and validation agents submitted as an entry to IBM's internal Bob-a-thon hackathon. EDUCATION & CERTIFICATIONS

Microsoft Certified: Azure Solutions Architect Expert, valid through August 2027. Microsoft Certified: Fabric Data Engineer Associate, valid through July 2027. Microsoft Certified: Azure Administrator Associate, valid through May 2027. IBM Generative & Agentic AI Developer, valid through July 2027. B.E. in Electrical and Electronics Engineering, SCSVMV University, 2015 (CGPA: 7.81/10). AWARDS

• IBM Client and Partner Success Award, 2026.

• Three IBM QCAP Cash Awards for delivery excellence, 2025-2026

• Apex Award - Industrial IBMers, 1H 2025.

Contact this candidate