Data Engineer

DAVID CLAUDE EGHAN

*** ******** ***, *** ****, NY, 10025

M: +1-704-***-**** E: ******.*****@*****.***

SUMMARY

Currently working as a Big Data Engineer in New York City. Excellent understanding of data structures and algorithms. Capable of writing production-ready code using APIs in Scala and Python. Experience working with product owners and business stakeholders to generate insight from data in disparate sources. PROFESSIONAL EXPERIENCE

TickPick New York, NY

Data Engineer October 2019 - Present

• Utilized Airflow to orchestrate ELT processes for data from 6 disparate sources into a unified data warehouse in a batch manner

• Created pipelines to ingest streaming data from Azure Eventhubs into Snowflake via Azure Blob Storage

• Created a data quality management process around the ELT process to ensure accuracy in data extractions

• Registered, trained and deployed machine learning models for margin optimization using a combination of Azure Databricks and Azure Machine Learning Studio

• Minimized data quality issues in reporting by creating circuit breaker technologies to halt data flow when data quality issues are detected

• Increased marketing insights in media spend by unifying advertising-related data from the Google Marketing Stack and Facebook Pixel to enhance reporting and marketing strategy

• Created data catalogues in Azure to unify the storage location for metadata on all data sources used in reporting Tresata New York, NY

Data Engineer July 2018 - September 2019

• Utilized Big Data tools including Hadoop, Hive, Spark, Scala, and ElasticSearch to automate data pipelines and create robust Know Your Customer (KYC) dashboards for several Fortune 500 banks

• Used Drake to automate workflow processes on multiple client projects, reducing data processing time by ~20%

• Created parsers to convert over 300 terabytes of nested data structures into parquet flat files using Hadoop, Spark and Scala

• Leveraged Spark to migrate ~600 terabytes of healthcare-related data from Oracle legacy systems to AWS and HDFS

• Worked with Sqoop and Hive to ingest, query and retrieve data from various RDBMS like OracleDB and MySQL

• Utilized Elasticsearch and Kibana to create data visualizations that provided insight into relevant KPIs for stakeholders

• Developed a dynamic reporting tool using Scala, Hive, MySQL and Python to generate insight on the overall quality of ~800 terabytes of data from 40 countries

• Designed SBT and Maven-based unit tests to validate code written with Scala, Spark DataFrames and SparkSQL Weill Cornell College of Medicine New York, NY

Summer Research Assistant June 2017- August 2017

• Collected data from patients, caretakers and doctors to analyze the impact of cognitive state on healthcare outcomes

• Utilized geodemographic segmentation to determine the impact of societal factors on emotional well-being

• Collected data on the impact of emotional effects on care outcomes for older adult patients living with chronic pain

• Performed daily follow-up interviews both in person and over the phone as part of my research EDUCATION

Davidson College Davidson, NC

Bachelor of Science, Biology 2014 - 2018

• Honors: Beta Beta Beta, Biological Honor Society, Davis UWC Scholar (scholarship awarded for accomplishment in service) The University of Edinburgh Edinburgh, UK

Study-abroad program in BioStatistical Sciences Fall 2016 LANGUAGES AND TECHNOLOGIES

• Databases:MySQL, Oracle, Snowflake

• Technologies:Hadoop, Hive, Azure, Spark, Sqoop, Amazon Web Services, Git, Drake, Docker, Airflow, Microsoft Azure, Machine learning, Google Analytics, Google Firebase

• Programming Languages:Scala, Python, RStudio, Bash

Contact this candidate