Data Engineering Engineer
Resume:
MAITREAYI RUDRARAJU
*********.*********@*****.***
Professional Summary
● Passionate about Information Technology with extensive experience in building scalable data-driven Enterprise applications leveraging a vast array of technologies such as Analytics, Big Data, Cloud and DevOps and actively pursuing Data Engineering Roles on the Cloud.
● Expertise in building Data Engineering Pipelines using Spark and Airflow on Databricks. Well-versed with the Development Life Cycle of Data Engineering Pipelines.
● Well-versed with Databricks on Cloud Platforms such as GCP, Azure, and AWS.
● Expertise in working with different Database Technologies such as Postgres, Oracle and Databricks SQL using SQL.
● Experienced in developing applications using Python as a programming language. Well-versed with Python libraries such as Pandas, Pyspark, requests, etc.
● Excellent Communication Skills, Strong Team Player, Adaptive Learning Abilities, etc.
● Seasoned ELT developer with hands-on experience in SQL and manipulating large datasets within database environments.
Skills
● Programming Languages: Python, Scala, C
● Database Technologies: Postgres, Oracle, My SQL
● Cloud Platforms: Databricks, Azure, GCP, AWS
● Data Warehouse Technologies: Snowflake, Databricks SQL Warehouse, Azure Synapse
● Development Tools: VS Code, GitHub, Confluence, Jira
● BI Tools: Tableau, Power Bi
Professional Experience
Cox Automotive Inc August 2023- Current
Sr. Data Engineer/ ETL Developer
Project Details
● Developed and maintained ETL/ELT pipelines in Snowflake managed service, optimised data workflows for scalability and performance.
● Leveraged Snowflake’s native connectors and API’s to ingest data from sources such as databases, data lakes, and streaming platforms into snowflake for analysis and reporting.
● Monitored snowflake usage, resource utilization and system health metrics to ensure optimal performance and availability.
● Implemented both full and incremental ELT strategies using Informatica Power Center, ensuring timely and accurate data synchronisation.
● Leveraged expertise in OLTP and dimensional schema to design and implement robust data integration solutions tailored to specific business requirements.
● Deployed ELT workflows on AWS infrastructure, utilizing services like Amazon S3 and Amazon Redshift for data storage and processing.
● Used MicroStrategy reporting tool for designing and developing insightful reports and dashboards to facilitate data-driven decision making.
● Provided production support for ELT processes, troubleshooting issues and ensuring uninterrupted data flow to downstream systems.
● Collaborated cross- functionally with stakeholders to understand business requirements and translate them into actionable ELT solutions.
Cisco Systems April 2022- August 2023
Data Engineer/ Data Analyst
Project Details
● Used SnowSQL for connecting to Snowflake to execute SQL queries and perform all DDL, DML operations to deliver the data as per the business requirements.
● Built Data Engineering Pipelines using technologies such as Python, Spark, Databricks, Airflow, etc.
● Developed a Pyspark based application to convert data from CSV to Parquet.
● Built ETL Pipelines using Spark SQL on Databricks
● Deployed and Scheduled Databricks Jobs using Databricks Notebooks based on Pyspark and Spark SQL.
● Deployed and managed applications on Azure Service Fabric, leveraging its microservices architecture for scalable and resilient cloud-native solutions.
● Extensively worked on troubleshooting data quality issues by running complex SQL queries on source RDBMS Databases such as Oracle and in Databricks
● Extensively worked on automating data validations and sanity checks by using Python as the programming language.
● Worked on Data Pipeline orchestration using tools such as Airflow.
● Extensively worked on unit testing or validation of the applications against requirements and duly document the unit test results.
Villanova University January 2020- March 2022
Data Engineer/ Data Analyst
Project Details
● Interacted with the Strategic Development Team, Business analyst to analyse business needs and developed technical specifications.
● Pulled data from various sources such as SQL Server, Excel, Oracle, SQL Azure etc into tableau.
● Developed actionable dashboards in Tableau Desktop and published them in Tableau Server.
● Experience working with key performance indicators and metrics in finance, sales, inventory, and other areas.
● Developing high level analysis reports and data visualization with Tableau.
● Used JIRA for raising issues, and defect tracking.
● Generated Tableau Dashboard with context/quick/global/action filters, calculated fields and parameters on Tableau reports.
● Designed and developed microservices based applications using Fabric and implemented service orchestration and communication patterns to facilitate inter-service communication within the Service Fabric cluster.
● Build Complex distributed systems involving a huge amount of data handling, collecting metrics, building data pipelines, and Analytics.
● Develop conceptual solutions & create proof-of-concepts to demonstrate the viability of solutions.
● Design, set up, maintain and Administer the Azure SQL Database, Azure Analysis Service, Azure SQL Data warehouse.
DhruvSoft Services Private ltd May 2018- December 2019 Data Engineer/ Data Analyst
Project Details
● Analyzed the existing application programs and tune SQL queries using execution plan, query analyzer, SQL Profiler and database engine tuning advisor to enhance performance.
● Migrated on premise database structure to Confidential Redshift data warehouse.
● Developed SSRS reports, SSIS packages to Extract, Transform and Load data from various source systems
● Managed security groups on AWS, focusing on high-availability, fault-tolerance, and auto scaling using Terraform templates. Along with Continuous Integration and Continuous Deployment (CI/CD) with AWS Lambda and AWS code pipeline.
● Created Data Quality Scripts using SQL and Hive to validate successful data load and quality of the data. Created various types of data visualizations using Python and Tableau.
● Worked on data pre-processing and cleaning the data to perform feature engineering and performed data imputation techniques for the missing values in the dataset using Python.
● Created Entity Relationship Diagrams (ERD), Functional diagrams, Data flow diagrams and enforced referential integrity constraints and created logical and physical models using Erwin. Certifications
Microsoft certified Azure Data Engineer Associate
Microsoft Programming with Python
Coursera Fundamentals of visualizations with tableau National Programme on Technology Enhanced Learning Programming data structures Education
Master of science Villanova University
● Software engineering December 2021
● Health Informatics December 2021
Bachelor of Technology Andhra University
● Information Technology May 2019
Contact this candidate