Data Engineer Financial Services

SUMMARY

Experienced Data Engineer with * years of expertise in designing, building, and optimizing data pipelines and cloud-based data solutions for insurance, healthcare, and financial services. Skilled in leveraging AWS services (S3, EC2, EMR, Glue, Redshift, Lambda) and Databricks to develop scalable ETL workflows, automate data transformations, and improve processing efficiency. Specialized in Snowflake for data modeling, query optimization, and real-time reporting, enabling faster and more accurate business insights. Proficient in Python, SQL, PySpark, and Airflow for orchestrating complex workflows, building analytics pipelines, and enhancing operational efficiency. Adept at delivering critical solutions for claims management, policy analytics, healthcare reporting, and customer behavior insights, with a strong focus on data quality, compliance, and performance.

TECHNICAL SKILLS

● Programming Languages: Python, SQL, JavaScript, Java, Scala, PySpark

● Libraries & Frameworks: Pandas, SQLAlchemy, dbt, Alteryx, SAS

● Big Data Technologies: Apache Spark, Hive, Kafka, Kinesis

● Databases: Snowflake, Redshift, MySQL, Oracle DB, DynamoDB, Teradata

● Cloud Platforms & Services: AWS (S3, EC2, EMR, Glue, Redshift, Athena, Lambda, RDS, Step Functions), Azure Data Factory, Databricks

● Data Visualization Tools: Tableau, Power BI, SSRS, Excel

● Version Control & Collaboration: GitHub, Bitbucket, Confluence

● Workflow Management: Apache Airflow, Snowflake Streams and Tasks

● Development & API Tools: Postman, Jupyter Notebook

● Operating Systems: Windows, Unix, Linux

PROFESSIONAL EXPERIENCE

BCBS Feb 2024 – Present

Data Engineer United States

● Created and maintained ETL workflows with AWS Glue to load insurance claims, policy, and customer data into Snowflake, Hive, and Redshift, improving ingestion speed by 35%.

● Processed large volumes of underwriting and policy data using PySpark on AWS EC2 and EMR, reducing data processing time by 40% for insurance analytics.

● Built standardized data models in Snowflake using dbt and managed ETL operations in Matillion, increasing access to insurance reports and customer insights by 20%.

● Automated data workflows for claim settlements and policy renewals using Airflow and AWS Step Functions, reducing manual handling and improving efficiency.

● Developed transformation pipelines in Databricks with PySpark, Pandas, and SQLAlchemy, enabling predictive analytics for claims processing and customer behavior.

● Streamlined real-time insurance data ingestion from Kinesis, Kafka, and DynamoDB into Snowflake, enhancing live fraud detection and policy analytics by 30%.

● Leveraged GitHub for robust version control and collaboration, streamlining updates to data processing pipelines for claims, policies, and customer analytics.

● Standardized complex SQL transformations in dbt on Snowflake and Redshift to improve data quality and reporting accuracy for underwriting and claims metrics.

● Managed insurance-related data storage in AWS S3 for both raw and processed datasets, supporting advanced analytics and improving data durability.

● Designed optimized Snowflake and Redshift models to enhance ETL performance for insurance reporting layers, achieving a 30% faster data query response.

AIG (Health Care) July 2020 – June 2022

Data Engineer

● Built ETL pipelines using Matillion, AWS Glue, and Talend to manage clinical trial data from MySQL and Teradata into Snowflake, achieving a 40% improvement in processing time.

● Processed data with PySpark, SQL and Python on Databricks for scalable transformation, reducing computational overhead and improving throughput by 30%.

● Used Airflow to automate ETL workflows, minimizing manual interventions and enhancing data consistency.

● Developed MySQL and NoSQL-compatible solutions, enabling flexible data storage and retrieval for JSON clinical data.

● Designed, developed, and deployed dynamic reporting dashboards on Tableau and Power BI, providing real-time analytics to cross-functional teams, facilitating data-driven decision-making.

● Implemented Snowflake's Materialized Views to optimize query performance for healthcare reports, providing faster access to critical insights for decision-making.

● Utilized Streams and Tasks in Snowflake to manage incremental data updates for clinical trial datasets, ensuring up-to-date and accurate data for analysis.

● Implemented high-throughput data pipelines using Apache Spark, significantly reducing processing times and enabling effective population health management and cost reduction analysis. ISPARROW (IDFC First Bank) August 2018 – June 2020 Data Analyst

● Utilized Excel extensively to perform data analysis, including pivot tables, v-lookups, and advanced formulas to extract insights from large datasets.

● Extracted and manipulated data using SQL from relational databases to generate reports and dashboards for business stakeholders.

● Created interactive dashboards and reports using Tableau and SSRS to visualize key performance indicators

(KPIs), enhancing data-driven decision-making across departments.

● Used MS Access to manage and query large datasets, creating forms and reports for easy access to data insights, improving team efficiency by 20%.

● Streamlined ETL (Extract, Transform, Load) processes using Alteryx, enabling seamless data ingestion and transformation for downstream analytics platforms like Tableau and Power BI.

● Leveraged SAS for advanced statistical analysis, helping identify trends and patterns that led to data-driven decision-making.

● Conducted exploratory data analysis (EDA) to identify trends and patterns in datasets, providing actionable insights that resulted in cost-saving initiatives.

● Conducted spatial data analysis and mapping using ArcGIS to identify geographic trends and support business decision-making.

● Developed detailed flowcharts, process diagrams, and organizational charts using Visio, improving team communication and project clarity.

● Adapted to additional responsibilities with flexibility and a proactive attitude, upholding core values and contributing to a positive company culture.

EDUCATION

Master of Science in Computer Science University of Illinois Springfield, USA

Bachelor of Technology in Computer Science & Engineering JNTUK University College of Engineering, India

CERTIFICATIONS

● Earned Microsoft Technology Associate certification.

● Secured AWS Certified Developer – Associate credential.

Contact this candidate