Data Engineer
Resume:
Suresh Dharavath
Hyderabad
*******************@*****.***
linkedin.com/in/suresh-d-data-engineer444
Data Engineer with 3 years of experience in the Data field, offering a track record of delivering high-quality results within tight deadlines. Specialized in the design and implementation of robust data pipelines, with proficiency in Snowflake, Azure Data Bricks, Python, SQL, ETL, and Airflow. Demonstrated ability to design and optimize data pipelines for efficient data processing and analytics. Work Experiences
Data Engineer @ Cognizant Technology Solutions July 2022 - Present Implemented and optimized data pipelines for both batch and stream processing. Developed scalable, repeatable, and secure data structures and pipelines to ingest, store, collect, standardize, and integrate data across different areas, making sure data moves smoothly from Development to Production. Managed a diverse range of data sources, including Azure Blob Storage, Azure Data Lake, ensuring data availability and consistency. Implemented CI/CD pipelines with Azure DevOps for Snowflake SQL and Airflow, reducing deployment times and boosting efficiency. Designed and implemented complex dimensional and normalized data models, including Star and snowflake schemas, to optimize data warehousing and report processing. ensured scalability and performance for large scale datasets. Incorporated Snowflake into data processes, handling data ingestion, processing, and transformation to meet specific project requirements. Developed complex SQL scripts for Snowflake, optimizing data retrieval and manipulation. Designed and managed end-to-end ETL pipelines leveraging Snowflake’s Tasks and Streams, which facilitated real-time analytics and achieved an 80% decrease in the time required to load data from ADLS. Developed dashboards within Snowflake to oversee the data transfer process from the origin to the destination, ensuring complete data integrity and no loss of Data.
Leveraged Snowflake features like Time Travel and Zero-Copy Cloning for enhanced data management and recovery. Integrated Snowflake with various data sources and third-party tools, optimizing data ingestion processes. Worked on performance tuning, query optimization, and resource monitoring within a cloud data platform. Utilized Airflow extensively to design multiple Directed Acyclic Graphs (DAGs), enhancing workflow orchestration. Implemented email triggers to perform Data Validation.
Leveraged Databricks for big data analytics and processing. Developed and optimized Spark jobs for data transformation and aggregation. Designed and implemented automation test frameworks using Python achieving 95% coverage and reducing manual effort by 90% during QA testing phase.
Data Engineer - Intern @ Cognizant Technology Solutions Feb 2022 - June 2022 Collaborated with a team to research and implement Informatica data pipelines, orchestrating data flow activities to enhance processing efficiency. Developed and maintained Informatica datasets and linked services, bolstering data integration and transformation initiatives. Played a key role in configuring and managing Informatica workflows and data triggers, which significantly improved the automation of data processes. Acquired hands-on experience in diagnosing and resolving data discrepancies, as well as fine-tuning Informatica data workflows for optimal performance. Conducted thorough analysis of data sources such as Azure Blob Storage, Azure Data Lake and SQL Server within Informatica, ensuring seamless data access and maintaining high data quality standards. Projects
IRB Holdings(Inspire brands) Domain:media/transaction Utilized Azure Cloud on the Polaris platform to support daily business operations. Employed Spark, Azure Data Lake Store, Data Bricks, and Snowflake for data ingestion and transformation. Implemented Airflow for automated orchestration with DAGs. Extracted and applied business rules from legacy systems where necessary. Utilized Agile methodology with Confluence, JIRA, and Azure DevOps for project management. Demonstrated hands-on expertise in managing YAML files for multiple environments (Dev, QA, UAT, Prod). Education
Bachelor of Technology in ECE 2019-2022
Gokaraju Rangaraju Institute of Engineering and Technology GPA: 8 . 7
Diploma In ECE 2016-2019
Govt.Polytechnic College
91%
Skills
SQL
Python
Azure
Airflow
PySpark
Snowflake
Atlassian Tools
Data Validation
Data Bricks
Agile method
Certificates
Microsoft Certified: Azure Data Engineer Associate(DP203) Microsoft Certified: Azure Data Fundamentals (DP900) Databricks Certified: Data Engineer Professional
Databricks Certified: Data Engineer Associate
Databricks Certified: Data Analyst Associate
Google Cloud Certified-Generative AI
Contact this candidate