Data Engineer Sql Server

SUMMARY

Innovative and performance-driven Cloud Data Engineer with over 6 years of experience in, developing, and managing cloud-native data platforms across AWS, Azure, and GCP.

Strong expertise in ETL/ELT development, data migration, and data modeling using tools and platforms like Snowflake, Redshift, SQL Server, Apache Spark, Java-based ETL pipelines, and Azure Data Factory.

Proficient in Python, Java, Scala, and Shell Scripting for data engineering, automation, and system orchestration.

Adept at integrating and optimizing data pipelines using AWS Glue, Lambda, S3, Matillion, Fivetran, and Talend.

Ensuring high-performance data workflows. Demonstrated success in migrating large-scale legacy systems (e.g., SQL Server and Sybase) to modern cloud-based platforms, handling data volumes exceeding 4TB.

Strong experience with BI tools like Power BI, Tableau, and Looker, enabling real-time, actionable insights through advanced data visualization.

Deep understanding of Star and Snowflake schema designs, fact and dimension modeling, and performance tuning for cloud data warehouses.

Extensive hands-on experience with Microsoft Azure and Amazon AWS cloud ecosystems, overseeing end-to-end deployment, automation, and optimization of scalable applications.

Having good knowledge in implementation of CI/CD.

Experience in developing and maintaining CI/CD process for SaaS applications by accessing tools like Maven, Jenkins, etc.

Created terraform for AWS Services EC2,S3,VPC,ECS,ECR,EKS,RDS,CloudWatch.

Strong knowledge in cloud security, auto-scaling, networking (Route 53), data movement, and infrastructure as code, enabling robust and secure cloud-native data solutions.

Good knowledge in creating the Docker Images and configuring the images in Jenkins.

Familiar with DevOps principles and agile methodologies to ensure continuous integration, testing, and deployment of data solutions.

Skilled in setting up role-based access controls, cloud security policies, and implementing governance frameworks across Snowflake, Redshift, and Azure environments.

Knowledgeable in log analytics, cloud monitoring (Azure Monitor, Cloud Watch), and building scalable microservices for data validation and REST API integration.

Successfully led multiple cross-functional teams in financial, banking, and healthcare domains, delivering robust, secure, and compliant data engineering solutions that drive strategic business outcomes.

TECHNICAL SKILLS

Programming : Python, SQL, Java, Scala, Shell Scripting

Cloud & DevOps : AWS (Redshift, S3, Glue, Lambda), Azure, GCP, Jenkins, Docker

Data Engineering : Snowflake, Redshift, DBT, Java based ETL Pipelines, Apache Spark, Hive

OperatingSystems : Linux,Windows

BI & Reporting : Power BI, Tableau, Looker, SQL Server Reporting Services (SSRS)

Data Modeling :Star & Snowflake Schema, Fact & Dimension Tables, Erwin, ER Studio

Database Systems : SQL Server, MySQL, Oracle, Teradata,RDS

Cloud environments : AWS, GCP & AZURE Devops

Application Servers :Tomcat, IIS, Nginx

EDUCATION DETAILS

Bachelors in ECE from Raghu Engineering College from JNTUK in 2019.

Master of Science: Computer Information Systems New England College Henniker, NH 2024.

CERTIFICATION

AWS Certified Data Engineer

PROFESSIONAL EXPERIENCE

Client: IDB Bank, New York, NY

Role: Cloud Data Engineer Sep 2024 – Current

Responsibilities:

Led end-to-end migration of 800+ objects (4TB data) from SQL Server to Snowflake, optimizing cloud data infrastructure.

Designed and optimized ETL pipelines using AWS Glue, Matillion, FiveTran, and Talend, improving data processing efficiency.

Hands-on experience with Java-based ETL pipelines, SQL, and Python for data extraction, transformation, and loading (ETL).

Worked on automating data workflows using Python, Java-based ETL pipelines, and Azure Data Factory, ensuring seamless data flow.

Designed and optimized ETL pipelines using AWS Glue, Snowflake, Teradata, and SQL Server, aligning with the role’s requirement for handling complex data logic.

Experience working with AWS (Redshift, S3, Glue, Lambda), Azure (Synapse, ADLS), and Snowflake, supporting the role’s cloud infrastructure needs.

Optimized query execution in Redshift by leveraging distribution keys, sort keys, and compression encoding to reduce scan time and improve response rates.

Led ETL migration from Sybase to AWS Redshift/Snowflake, handling SQL script conversion and data transformations.

Tuned Sybase stored procedures, indexes, and query plans to enhance performance before migrating to a modern cloud database.

Integrated Sybase with AWS Glue and S3, facilitating smooth data extraction.

Hands-on experience in Python for data engineering, automation, and analytics.

Developed Python-based ETL pipelines and workflow automation.

Used Python for data transformation and reporting in Snowflake and AWS environments.

Worked with Power BI, Tableau, and Looker for visualization, which aligns with the requirement for experience in data visualization.

Hands-on experience in AWS services such as Redshift, S3, Glue, and Lambda, which aligns well with AWS Workspaces setup, configuration, and maintenance.

Experience in automating deployments and managing cloud infrastructure using CI/CD pipelines (Jenkins, Python) to ensure smooth operations.

Strong background in virtual and physical endpoint management, which translates well to setting up and maintaining AWS Workspaces.

Developed and optimized ETL pipelines using R and MySQL, improving data processing efficiency by ensuring seamless data flow in a Unix/Linux environment

Knowledge of Azure security and governance best practices, Understanding of cloud computing concepts and architectures

Familiarity with DevOps practices and tools (e.g., Azure DevOps, Jenkins), Proficiency in Azure services (e.g., Virtual Machines, Storage, Networking, SQL Database), Experiencd in planning, installation and administration of GIT on Linux and Windows platforms.

Worked with Apache Spark (Java API) for large-scale data processing, ensuring efficiency in batch and workflows.

Involved integrating Azure Data Factory (ADF) pipelines for automated data movement and also worked with Azure Monitor and Log Analytics for tracking ETL performance to manage and optimize cloud services.

Created and managed Snowflake schemas, tables, and views, ensuring data integrity and performance tuning.

Automated data extraction, transformation, and loading (ETL) using Java and Python-based workflows, optimizing performance in Snowflake and Azure environments.

Developed interactive dashboards in Power BI and Looker, enabling real-time data-driven decision-making.

Integrated Java-based REST APIs for dynamic data retrieval in Looker and Power BI dashboards.

Implemented Java-based Lambda functions for event-driven data processing on AWS.

Built Java utilities for data extraction and automated reporting from Redshift and Snowflake.

Managed Snowflake administration, including role-based access control, data sharing, and security policies.

Validated Looker reports with Redshift and Snowflake, ensuring seamless data consistency and reporting accuracy.

Client : CVS, Woonsocket, RI

Role : Data Engineer/ Data Analyst April 2023 to Aug 2024

Responsibilities:

Developed and implemented predictive models using Natural Language Processing techniques and machine learning algorithms such as linear regression, classification, multivariate regression, K-means clustering, KNN, PCA and regularization for data analysis.

Designed and developed Natural Language Processing models for sentiment analysis.

Worked on development of data warehouse, Data Lake and ETL systems using relational and non-relational tools like SQL, No SQL.

Participated in all phases of data mining; data collection, data cleaning, developing models, validation, visualization and performed Gap analysis.

Data Manipulation and Aggregation from different source using Nexus, Toad, Business Objects, Power BI and SmartView.

Maintenance in the testing team for System testing/Integration/UAT.

Involved in preparation & design of technical documents like Bus Matrix document, PPDM Model, and LDM & PDM.

Understanding the client business problems and analyzing the data by using appropriate Statistical models to generate insights.

Analyze the existing code and do the impact analysis.

Involved in peer-to-peer code reviews.

Code validation as per client requirement.

Developed complex database objects like Stored Procedures, Functions, Packages and Triggers using Oracle Database, SQL and PL/SQL.

Automated the process of rebuilding indexes at regular interval for better performances.

Generated reports that can handle both dynamic grouping and sorting.

Wrote SQL queries using joins, Sub queries and correlated sub queries to retrieve data from database.

Debugged many PL/SQL packages procedures, functions, cursors, and types for application.

Query optimization, execution plan and Performance tuning of queries for better performance.

Client: Acko General Insurance, India

Role: Data Engineer/ Analyst Aug 2020 to July 2022

Responsibilities:

Developed Data pipelines and ETL processes using Java, Snowflake, Azure Data Lake Storage (ADLS), Azure Synapse, and SQL Server for enterprise data integration.

Migrated 500+ tables and views from Redshift to Snowflake, optimizing query performance and storage efficiency.

Created and maintained Python and Shell scripts for data transformation, workflow automation, and system administration.

Strong experience with AWS (Redshift, S3, Glue, Lambda) and Azure (Synapse, ADLS, Data Factory) matching CHOP’s cloud architecture needs.

Managed role-based access control, data sharing, and security policies in Snowflake aligning with CHOP’s governance standards.

Developed micro services in Java for data validation and integration with Azure Data Factory pipelines.

Experience in overseeing applications using Amazon AWS Cloud which incorporates services like: EC2, S3, EBS, VPC, ELB, AMI, SNS, IAM, Glacier Route 53, Auto Scaling, Cloud Watch, CloudFormation, AWSLambda, AWSAPIGateway, Terraform and Security Groups.

Designed and developed cloud native applications using AWS services such as ECS,EKS.

Built data models (Star & Snowflake schemas) to support business reporting and analytics needs.

Implemented CI/CD pipelines with Jenkins and Python, automating deployment and monitoring of ETL workflows.

Implemented Tri-Secret Secure (TSS) actions, including key generation, rotation, and data re-encryption, enhancing data security and compliance.

Developed and maintained ETL pipelines that extracted data from FHIR endpoints, ensuring seamless integration and high-quality datasets for AI applications

Proven ability to troubleshoot and resolve cloud infrastructure issues, including optimizing query performance in Snowflake and Redshift.

Managed role-based access control (RBAC) in Redshift, ensuring secure data sharing and governance.

Worked on Sybase database replication and backup strategies to ensure business continuity during migration.

Used Python and Shell scripts to automate Sybase data extraction and transform it into Redshift-compatible formats

Experience in log analytics and monitoring (Azure Monitor, Log Analytics), which helps in tracking and resolving issues in AWS Workspaces.

Proficient in SQL and Python-based data analysis, ensuring data consistency and insightful reporting.

While your resume doesn’t explicitly mention Angular or React, your experience integrating Java-based REST APIs into Looker and Power BI suggests familiarity with web-based applications.

Provided technical support to internal users by maintaining strong communication and documentation, which is crucial for assisting seasonal employees

Extensive experience in SQL, Oracle, Teradata, and Snowflake, with a strong understanding of data modeling (Star & Snowflake schemas).

Developed and maintained Unix shell scripts for automation and backend processing, supporting the job requirement.

Conducted in-depth data analysis using SQL, Python, and Looker to identify trends, resolve issues, and implement corrective actions.

Designed and managed Azure Data Factory (ADF) pipelines for automated data movement across cloud environments.

Developed and optimized Azure Synapse Analytics solutions, improving data warehousing and analytics performance.

Utilized Azure Data Lake Storage (ADLS) to handle structured and unstructured data, ensuring scalability and security.

Implemented Azure SQL Database solutions for real-time and batch processing, integrating with Power BI for advanced analytics.

Configured Azure Monitor and Log Analytics to track ETL performance, troubleshoot issues, and optimize query execution.

Client: Cognib LLP, India

Role: Data Analyst May/2019 - July/2020

Responsibilities:

Designed BI dashboards and reports using Power BI and Tableau, improving KPI tracking and operational visibility.

Conducted data analysis and profiling using SQL, Python, and Snowflake, identifying trends and anomalies in business data.

Utilized Apache Spark (Java API) for large-scale data processing, optimizing batch and real-time workflows.

Developed and optimized SQL queries and stored procedures for performance tuning and data extraction.

Implemented data governance policies, ensuring compliance with industry standards and regulations.

Collaborated with business users to define data requirements and reporting solutions, translating them into actionable insights.

Developed microservices in Java for data validation and integration with cloud-based data processing platforms.

Exposure to building data pipelines and working with Delta Lake in Azure environments, aligning with the job’s requirements.

Developed and maintained operational documentation, SOPs, and runbooks, ensuring structured workflows for cloud services.

Hands-on experience working with microservices using Java, which could be a good foundation for learning frontend frameworks.

Built and optimized CI/CD pipelines using Jenkins and Python, ensuring efficient code deployment.

Strong understanding of data governance and compliance, which is key when managing enterprise cloud environments.

Built interactive dashboards (Power BI, Looker, Tableau) that enabled real-time decision-making and enhanced user experience.

Focused on delivering high-quality, responsive solutions to clients and internal stakeholders, ensuring smooth cloud operations

Collaborated with business users, data analysts, and engineering teams to design scalable solutions, demonstrating leadership and analytical skills.

Developed and maintained ETL workflows using SSIS and Informatica, ensuring efficient data extraction, transformation, and loading processes.

Created and optimized SSRS reports, providing key business insights and improving decision-making capabilities.

Implemented SQL Server-based solutions, integrating Java applications for seamless data processing and storage.

Designed data warehousing solutions, implementing data pipelines to consolidate information from multiple sources.

Automated data validation and cleansing using SQL scripts, improving data accuracy and consistency across systems.

Contact this candidate