Data Engineer
Resume:
WORK EXPERIENCE
Data Engineer
Afiniti - Karachi,Pakistan (January 2021 - Present)
Telfonica Spain:
Developed robust data flows from scratch on Airflow using different Hooks,Operators and Xcom variables.
Implemented error handling and retry logic to enhance the reliability of data pipelines.
Automated the loading process and implemented email alerts to keep track of process success and failure for the client using Apache Airflow and greenplum - Created Sanity check functions for all the data feeds to check if data is good to load. Developed File Counter alert and Data Decryption process through openssl and linux scripting.
Anlaysed and fixed the erroneous data on the client Telefonica Spain and developed ETL pipelines to avoid Data loss and duplication.
Ensure Data Integrity and Quality by implementing intelligent data sensors.
ENEL:
Planned and Executed end-to-end migration of ETL processes and Database from Mysql to Greenplum
Monitor, troubleshoot and perform root cause analysis and resolve on all Production Issues.
Converted and validated Stored Procedures from Mysql to Greenplum(Massively Parallel Postgresql)
Claro Columbia:
Planned and Executed end-to-end migration of ETL processes and Database from Mysql to Greenplum
Used GPFDIST utility for file loading in greenplum database Optimised and simplified ETL processes and stored procedures during the migration.
Volunteer Work:
Developed a generic python utility to check the data-mismatch between Mysql and Greenplum Database during the migration and deployed it on multiple Clients.
Worked voluntarily on AXA France to help resources in migrating processes from windows to linux.
EDUCATIONAL BACKGROUND
Bachelors Of Computer Science and IT
Dec 2016-Oct2020
NED University of Engineering and Technology
ABOUT ME
A Data wiz with the years of experience in working on multiple real-world big data projects with different problems at a multinational leading AI company. Adaptable and detail-oriented, I am dedicated to contributing to impactful projects.
SKILLS
Muhammad Essa Khan
Data Engineer
Airflow DAGs:
Experienced in defining and customizing Directed Acyclic Graphs to efficiently schedule and monitor data pipelines. Familiar with a variety of Airflow operators and hooks for seamless integration with different systems and services. Adept at implementing robust error handling and retry logic to enhance the reliability of data pipelines, and sending the error logs alert through email.
MPP Databases/Data warehouse/Distributed Systems:
Possess demonstrated experience in Migrating Data
Pipelines, Processes and stored procedures from
conventional databases to MPP databases.
Experienced in developing alternate strategy to do similar thing in a more effecient way in MPP data warehouses. i.e. Greenplum,AWS Redshift etc.
Familiar with connecting MPP databases and with data orchestration tools like Airflow and talend.
Familiar with Hadoop Ecosystem - Storing structured, semi structured data in HDFS and creatting HIVE data warehouse for OLAP operations.
Adding Json Serdes in Hive to load semi structured data in readable form.
Databricks:
Experienced in exploring and wrangling the data in databricks using Spark SQL and Spark dataframe.
Familiar with visualizing data on databricks visualization tool for data wrangling and using its version control system to work with the team collaboritvely.
Capable of using the Spark Sql and dataframes in
databricks notebooks to perform the ETL and EDA
processes.
Knowledgeable of the databricks-related tools like delta lake and delta tables.
Talend:
Proficient in building data pipelines and data flows using Talend.
Expert in migrating talend pipeline from Mysql to
greenplum and make the pipeline imitate the pipeline imitate the behaviour of the existing pipeline.
Management and Communication:
Possess demonstrated experience in working in cross- functional team environment. Worked collaboratively with the teams to deliver and manage the projects on time. Effecient in managing the projects and making the
complete project plan to meet the deadlines.
*********@*****.***
Karachi, Pakistan
www.linkedin.com/in/muhammad-essa-khan
https://github.com/EssaKhaan
Contact this candidate