Data Engineer Azure

CHAITHRA KOTEGAR

(AZURE DATA ENGINEER)

Address: Kundapur, Udupi phone number : +91-819******* Mail : *******************@*****.*** LinkedIn: View LinkedIn Summary: Data Engineer with 3.7 years of experience in building scalable data pipelines and ensuring data quality using Azure Data Factory, Databricks, and PySpark. Skilled in implementing Rule-Based Data Validation Frameworks, data warehouse modeling, and automating ETL workflows to reduce manual efforts. Strong in SQL, data analysis, and transforming raw data into reliable, analytics-ready datasets. Skill Set: Azure Databricks Azure Data Factory Azure Blob Storage Apache spark Data Warehouse ETL Linux Shell Scripting SQL PySpark Python Excel Snowflake Spark SQL Bigdata Performance Optimization Collaboration Communication Teamwork

Professional Experience:

Associate Consultant - Data Analytics (Conneqt Digital/Digitide) (Jan 2025 – Present) Dynamic Data Quality Framework with Rule-Based Controls (RBC):

Built a flexible rule engine using PySpark and SQL in Databricks to validate data dynamically.

Applied checks like null, duplicate, data type, format, and value range using a centralized control table.

Automatically loaded multiple CSV files, inferred schema, and applied relevant rules per column.

Generated data quality reports highlighting failed records and validation summary per dataset. Sales Data Warehouse Simulation:

Created dimension tables (Product, Category) and a fact table (Sales Orders) from raw sales data.

Performed lookup validation to check if product and category IDs were valid before loading data.

Separated valid and invalid records to ensure clean data movement from Bronze to Silver layer.

Used Databricks with PySpark to apply transformation logic and simulate a real data warehouse process. Software Engineer – Data Engineer (Voya Global Services Pvt Ltd) (Oct 2021 – Jan 2025) Targeted Messaging:

Developed ETL pipelines to facilitate personalized investment messaging for participants.

Automated data processing workflows to segment participants based on investment behavior and eligibility.

Integrated Azure Data Factory and Databricks to analyze participant data and generate customized investment insights.

Elder Abuse Detection:

Built ETL pipelines to analyze transaction patterns and detect potential financial elder abuse cases.

Integrated multiple data sources to flag suspicious activities based on predefined risk indicators.

Automated reporting and alert mechanisms to escalate detected fraud cases to compliance teams.

Utilized Azure Databricks and PySpark for large-scale data processing.

Performed performance tuning on data processing workflows, reducing execution time by 40%. Automating Holiday Check using Shell Script:

Developed a shell script to automate holiday checks and scheduling in ETL workflows.

Integrated holiday calendars to ensure business logic execution only on working days.

Reduced manual intervention by 90% through automation, improving process efficiency. Education:

2017-2021 Bachelor of Engineering (BE) in Computer Science

(Visveswaraya technological University, Belgaum)

Languages: English, Kannada, Hindi

Achievements: Got High-Five instant recognition from Voya for quickly learnt new technology and delivered results in a very short time.

Certification:

Data Engineering workshop-snowflake (View credential)

Data Warehouse workshop-snowflake (View credential)

Snowflake - Collaboration, MarketPlace & Cost Estimation Workshop (View credential)

Snowflake - Data Application Builders Workshop (View credential)

Snowflake - Data Lake Workshop (View credential)

Contact this candidate