Senior Data Engineer - Databricks & Snowflake Expert

Shivacharan Reddy Godhala

Email: **************@*****.*** Mobile: 940-***-****

PROFESSIONAL SUMMARY

6+ years of experience as a Senior Data Engineer specializing in data engineering and cloud solutions, with a strong focus on Databricks and Snowflake platforms to deliver impactful solutions for enterprise clients.

Proven expertise in implementing advanced data engineering strategies, leveraging Databricks and Snowflake capabilities to enhance scalability, performance, and business value in complex data transformation initiatives.

Demonstrated ability to mentor and develop high-performing teams, fostering a collaborative environment to drive innovative solutions and achieve project milestones effectively.

Extensive experience in core components of Databricks, including DataFrames, Datasets, Spark SQL, and Delta Lake, ensuring robust data processing and analytics capabilities.

Skilled in Snowflake's architecture, utilizing features such as Snowpipe, Time Travel, and Streams & Tasks to optimize data sharing and management for enterprise applications.

Strong background in translating complex technical concepts into actionable insights for business leadership, enhancing decision-making processes and strategic planning.

Knowledgeable in data architecture principles, including data modeling, ETL processes, data governance, and security controls, ensuring compliance and data integrity across projects.

Committed to continuous learning and professional development, holding multiple certifications in data engineering technologies, including AWS, Azure, and Databricks.

SKILLS

Programming Languages: Python, SQL, Scala, Java, R, C#, JavaScript, TypeScript

Data Engineering Tools: Databricks, Snowflake, Apache Spark, Apache Kafka, Airflow, Talend, Informatica, DBT

Cloud Platforms: AWS, Azure, GCP, EC2, S3, Lambda, Azure Functions, Snowpipe

Data Architecture: Data Modeling, ETL Processes, Data Governance, Data Quality, Data Warehousing, Data Lakes, Privacy Controls, Security Controls

Machine Learning: Machine Learning Pipelines, Model Deployment, Feature Engineering, Scikit-learn, TensorFlow, PyTorch, MLflow, Data Preprocessing

Collaboration Tools: JIRA, Confluence, Git, GitHub, Slack, Microsoft Teams, Trello, Asana

Soft Skills: Leadership, Communication, Teamwork, Problem-Solving, Adaptability, Critical Thinking, Mentoring, Conflict Resolution

Data Transformation: DataFrames, Datasets, Spark SQL, Delta Lake, Snowflake Worksheets, Virtual Warehouses, Time Travel, Streams & Tasks

Certifications: AWS Certified Data Engineer, Databricks Certified Data Engineer Associate, Microsoft Certified: Azure Data Engineer Associate, Google Cloud Data Engineer

Project Management: Agile Methodologies, Scrum, Stakeholder Management, Client Relations, Time Management, Risk Management, Resource Allocation, Performance Tracking

WORK EXPERIENCE

Texas Health and Human Services Commission - Austin, TX

Data Engineer (AI/ML) - Mar 2024 to Present

Engineered advanced data solutions leveraging Databricks and Azure, enhancing data processing efficiency by 30% through optimized ETL processes and data governance strategies.

Collaborated with cross-functional teams to implement scalable machine learning pipelines, resulting in improved predictive analytics capabilities for client services.

Developed and maintained data models and architectural patterns, ensuring compliance with privacy and security controls while supporting enterprise-scale data platforms.

Mentored junior data engineers, fostering a high-performing team environment and promoting best practices in data engineering and cloud solutions.

Spearheaded the integration of Databricks Notebooks and Delta Lake, streamlining data transformation initiatives for large-scale projects.

Analyzed customer objectives and challenges, translating complex technical concepts into actionable insights for business leadership.

Implemented Snowflake's core components, including Secure Data Sharing and Snowpipe, to facilitate real-time data access and sharing across departments.

Conducted rigorous testing and validation of data quality, enhancing data integrity and reliability for critical business decisions.

Led initiatives to improve data architecture principles, focusing on data modeling and governance to ensure alignment with organizational goals.

Presented findings and project updates to stakeholders, demonstrating the impact of data solutions on operational efficiency and business value.

Technologies Used: Databricks, Azure, Snowflake, Delta Lake, Spark SQL, DataFrames, Datasets, Databricks Notebooks, ETL, Data Governance, Data Modeling, Machine Learning, Data Quality, Privacy Controls, Secure Data Sharing, Snowpipe

Lowe's Companies, Inc. - Mooresville, NC

Data Engineer - Integrated AI - Jan 2022 to Jul 2023

Developed and implemented data engineering strategies utilizing Databricks and Snowflake, achieving a 25% reduction in data processing time for analytics projects.

Collaborated with product teams to deliver impactful data solutions, enhancing customer experience through advanced analytics and reporting capabilities.

Designed and executed data transformation workflows using Databricks, ensuring high performance and scalability for enterprise-level applications.

Established reusable solution frameworks that improved project delivery timelines and reduced redundancy in data processing tasks.

Mentored a team of data engineers, promoting knowledge sharing and skill development in cloud technologies and data architecture principles.

Conducted data quality assessments and implemented governance measures, ensuring compliance with industry standards and best practices.

Leveraged Snowflake's Virtual Warehouses and Time Travel features to optimize data retrieval and historical analysis for business intelligence initiatives.

Engaged in stakeholder management, translating technical requirements into business solutions that aligned with organizational objectives.

Executed machine learning pipelines, integrating AI capabilities into data engineering processes to drive innovation and efficiency.

Presented technical solutions and project outcomes to senior leadership, showcasing the value of data-driven decision-making.

Technologies Used: Databricks, Snowflake, Azure, ETL, Data Governance, Data Quality, Machine Learning, Data Transformation, Virtual Warehouses, Time Travel, Data Architecture, Cloud Solutions, Analytics, Reporting, AI Integration

Regions Bank - Birmingham, AL

Cloud Data Engineer - Jul 2020 to Dec 2021

Architected cloud-based data solutions using Azure and Databricks, enhancing data accessibility and processing capabilities for banking operations.

Collaborated with cross-functional teams to implement data engineering best practices, resulting in a 20% improvement in data pipeline efficiency.

Developed and maintained ETL processes, ensuring high-quality data integration for analytics and reporting purposes.

Optimized data storage and retrieval using Snowflake, improving query performance and reducing operational costs.

Participated in agile project management methodologies, contributing to sprint planning and daily standups to ensure timely project delivery.

Conducted training sessions for team members on cloud technologies and data engineering frameworks, fostering a culture of continuous learning.

Analyzed and resolved data quality issues, implementing governance frameworks to maintain data integrity across systems.

Engaged with stakeholders to gather requirements and deliver tailored data solutions that met business needs.

Implemented security measures to protect sensitive data, ensuring compliance with regulatory standards.

Documented data engineering processes and architectural designs, facilitating knowledge transfer and onboarding for new team members.

Technologies Used: Azure, Databricks, Snowflake, ETL, Data Governance, Cloud Solutions, Data Quality, Agile Methodologies, Data Architecture, Security Controls, Data Integration, Analytics, Reporting

LendingClub - San Francisco, CA

Data Engineer - Jun 2019 to Jun 2020

Developed data pipelines and ETL processes using cloud technologies, improving data flow and accessibility for financial analytics.

Collaborated with product teams to design data models that supported business intelligence initiatives, enhancing decision-making capabilities.

Implemented data quality checks and governance measures, ensuring compliance with industry standards and best practices.

Optimized data storage solutions using cloud platforms, reducing costs and improving performance for data retrieval.

Participated in agile development practices, contributing to sprint reviews and retrospectives to enhance team collaboration.

Assisted in the migration of legacy data systems to cloud-based solutions, ensuring seamless integration and minimal disruption.

Conducted analysis of data trends and patterns, providing insights that informed strategic business decisions.

Engaged in stakeholder communication to understand data requirements and deliver effective solutions.

Documented technical processes and workflows, supporting knowledge sharing within the team.

Presented project updates and findings to management, demonstrating the impact of data initiatives on business performance.

Technologies Used: Cloud Technologies, ETL, Data Governance, Data Quality, Data Modeling, Business Intelligence, Agile Development, Data Integration, Cloud Solutions, Analytics

CERTIFICATIONS

AWS Certified Data Engineer - Associate

Databricks Certified Data Engineer Associate

Microsoft Certified: Azure Data Engineer Associate

EDUCATION

Masters in Data Engineering with AI & ML University of North Texas GPA: 3.6

Bachelors in Engineering CMR College of Engineering and Technology GPA: 9.1

Contact this candidate