Data Engineer Engineering

BALASUBRAMANYAM VR

San Ramon, CA ***** 612-***-**** ****.**@*****.***

PROFESSIONAL

SUMMARY

Highly skilled Software Engineer with 16+ years of experience in designing, building, and optimizing scalable data platforms. Expertise in data architecture, ETL/ELT pipelines, data modeling, cloud computing (AWS, Azure), and big data technologies. Adept at leading cross-functional teams, defining data governance strategies, and implementing robust data engineering solutions for real-time and batch processing. Strong experience in AWS-based data platforms, CI/CD, streaming (Kafka, Spark Streaming), APIs, and regulatory compliance.

TECHNICAL SKILLS

• Cloud & Big Data: AWS (S3, Lambda, Redshift, Event Bridge, ECS), Apache Spark, Hadoop, HDFS, ADLS

• Data Engineering & ETL: Python, Java, Scala(Spark shell), Apache Airflow, DBT, Informatica, Talend, Azure Data Factory, Tableau, MDM, Shell scripting.

• Data Modeling & Databases: SQL (PostgreSQL, Redshift, Hive, Oracle, MySQL, DB2), NoSQL (HBase, DynamoDB)

• Streaming & APIs: Spark Streaming, Apache Kafka, REST/SOAP APIs, OAuth 2.0

• DevOps & CI/CD: Azure GIT, Jenkins, Docker, GitHub.

• Data Governance & Quality: TDD methodology, data security (GDPR, HIPAA compliance)

• Other Tech stack: Mainframes, Cobol, Corn Shell

WORK HISTORY

SR. DATA ENGINEER 12/2021 to CURRENT

Intapp Palo Alto, CA

• Architected and developed data pipelines using Python, AWS, Airflow, DBT improving time to market with reusability by 80%, reduced processing time of by 70% and minimized application downtime and job failures by 30%.

• Designed real-time and batch ETL solutions integrating SaaS and cloud applications (Azure Cost Management, Hubspot, Workday, SharePoint etc) with Redshift data warehouse.

• Implemented Apache Airflow orchestration on AWS ECS, architected and automated transformations with DBT and created dashboards on Tableau.

• Built integration between Salesforce and Netsuite with AWS EventBridge for event-driven data sync.

• Defined Data Governance framework and automated quality checks to improve data accuracy and integrity.

• Spearheaded the CI/CD automation of data pipelines with Azure GIT and Jenkins reducing deployment time by 60%.

• Conduct code reviews, technical discussions, and spearheaded project planning and roadmap development.

• Leading a team of Data Engineers across multiple time zones, driving project strategy and development in agile methodology.

BIG DATA/DATA ENGINEERING TECH LEAD 05/2017 to 11/2021 Kaiser Permanente Pleasanton, CA

• Built and optimized Hadoop-based data solution for Kaiser Permanente, enabling data integration on Mobile application (OWL) used by 80% of operating staff at Kaiser.

• Architected scalable data flows and crafted comprehensive technical design documents, presenting solutions to executive stakeholders.

• Developed real-time streaming data ingestion using Kafka, Spark Streaming, reducing data latency from hours to minutes.

• Operated within the Scaled Agile methodology, collaborating with Product Managers, Business Analysts, and the QA Team to deliver successful solutions. TECH LEAD / ARCHITECT 05/2015 to 04/2017

DentaQuest Charlestown, MA

• Designed and developed enterprise Data Warehouse from scratch, consolidating 10+ data sources and reducing report generation time from hours to minutes.

• Led a team of 40+ developers to implement ETL pipelines on SQL Server, integrating Python, Informatica, and Airflow for scalable data processing.

• Architected and implemented end-to-end data ingestion and archival solutions

• Designed the process for high availability, scalability and security compliance

(HIPAA, SOC2).

ETL TECH LEAD 08/2014 to 04/2015

Wellmark BCBS Des Moines, IA

• Developed and optimized high-performance ETL pipelines using Informatica, Teradata, and Shell scripting, improving query execution by 50%.

• Implemented pushdown optimization, FastLoad, and Teradata query tuning to enhance overall performance.

• Led team of Data Engineers across multiple time zones and managed end to end deliverable of the project.

MAINFRAMES AND ETL DEVELOPER, TEAM LEAD 08/2007 to 07/2014 Target corporation Minneapolis, MN

• Led a 15+ member team in developing mission-critical supply chain applications on Mainframes and Informatica ETL pipelines for data migration.

• Built and enhanced order management, replenishment, and supply distribution systems, improving inventory accuracy by 20%.

• Owned and supported critical retail applications as a Subject Matter Expert supporting business teams.

EDUCATION

M.Tech CAD/CAM 2007

Vellore Institute of Technology, Tamil Nadu, India CERTIFICATIONS

• AWS Certified Solution Architect- Associate,

• Spark and Hadoop Developer

. • AWS Certified Developer Associate

Contact this candidate