Big Data Engineer

Certified Spark and Hadoop Developer

Cloudera Certified Hadoop and Spark Developer and Machine Learning Engineer Nanodegree holder with 4+ years of experience in Big Data, Data Warehousing, Business Intelligence and Analytics developing and implementing solutions for Telecom and Airline industries.

Experienced in Big Data Ecosystem including Spark, Spark Streaming, Hive, Impala, Sqoop and Flume. Well versed in SQL/PLSQL, Python, Scala and Bash Shell scripting.

PROFESSIONAL EXPERIENCE

Big Data Engineer Apr 2018 – Jun 2019

IBM

Duties and responsibilities include:

Building and optimizing big data batch and near real-time data pipelines, architectures and data sets.

Experience working with structured and unstructured datasets.

Designing, planning and implementing upgrades for 30+ nodes Azure hosted Cloudera Hadoop environment and infrastructure by installing patches and upgrading software.

Performing root cause analysis to resolve production issues/alerts in a timely manner.

Administering and configuring Enterprise Data Hub tools like Dataiku, Trifacta and ZDP.

Reviewing performance stats and query execution plans and recommending changes for tuning Hive and Impala queries.

Implementing security measures for all aspects of the cluster (Disk, HDFS and role based access via Sentry).

Creating and maintaining housekeeping and archiving scripts using bash shell scripting.

Data Processing Specialist Oct 2017–Apr 2018

Etisalat

Duties and responsibilities include:

Designing, coding, testing and implementing ETL IBM DataStage jobs and bash shell scripts for data manipulation.

Working closely with Business Analysts and other technical employees to understand requirements regarding solutions, ensure data quality and match-back to core systems transaction results.

Performing data analysis and recommend data optimization/performance opportunities.

Researching, troubleshooting, and resolving data issues impacting extract delivery.

Monitoring scheduled ETL jobs and determine problems that might arise.

Developing technical documentation for process overviews, data flows from source to target and design specifications.

Data/BI Engineer Jul 2016 – Oct 2017

Vodafone

Duties and responsibilities include:

Working with product owners, business analysts and other stakeholders to gather data-related technical requirements and support their business needs.

Developing, managing, and updating physical and logical data models.

Designing data extraction and loading to Data Lake using Datameer to ensure that data consistency, cleanliness and accuracy.

Contributing to agile team alignment and participating in team sprint planning, stand-ups, backlog grooming and retrospectives

Developing dashboards and reports on Splunk to provide actionable insights into operational efficiency and other key business performance metrics and presenting the dashboards to the stakeholders and business users.

Ensure technical delivery of detailed feature/story level solutions that satisfies the IT roadmap acceptance criteria.

Developing technical documentation including requirements documents, process overviews, data models /data flows from source to target and design specifications.

Testing and validating dashboards results against source system reports.

TECHNICAL SKILLS

Big Data Ecosystem: Spark, Spark-Streaming, HDFS, Hive, Impala, Sqoop, Flume, Pig, Kafka, Airflow, NIFI.

AWS Cloud Technologies: EC2, DynamoDB, S3, SQS, SNS, SES, RDS, Kineses, IAM.

Databases: Oracle, Teradata, MySQL, Cassandra, Oracle Hyperion.

Programing and Scripting Languages: SQL, PL/SQL, Scala, Python, Bash, HiveQL.

ETL Tools: IBM Datastage, Informatica, Dataiku, Datameer.

BI Tools: OBIEE, SAP BO, QlikSense, Splunk.

Operating Systems: Linux, Windows.

Others: Jenkins

EDUCATION & PROFESSIONAL DEVELOPMENT

Graduate Diploma in Data Science and Big Data Analytics Oct 2015 – Jul 2016

Information Technology Institute, Cairo, Egypt

Rank: 1st

Awarded full scholarship for Oct 2015 – Jul 2016

Bachelor of Management Information Systems Feb 2011 – Feb 2015

Arab Academy for Science Technology and Maritime Transport (AASTMT), Cairo, Egypt

GPA: 4.0/4.0

Rank: 1st

Awarded Outstanding Academic Performance full scholarship from Feb 2011 – Feb 2015

Courses & Certifications

Courses:

The Complete Hands-On Course to Master Apache Airflow on Udemy. Certificate earned on June, 2019

AWS Certified Developer - Associate 2019 on Udemy. Certificate earned on May, 2019

Taming Big Data with Spark Streaming and Scala - Hands On! on Udemy. Certificate earned on March, 2019

Apache Spark 2 with Scala - Hands On with Big Data! on Udemy. Certificate earned on December, 2018

Introduction to Apache NiFi (Hortonworks DataFlow - HDF 2.0) on Udemy. Certificate earned on November, 2018

The Ultimate Hands-On Hadoop - Tame your Big Data on Udemy. Certificate earned on November, 2018

Machine Learning Engineer Nanodegree on Udacity. Certificate earned on October, 2018

Getting Started with Natural Language Processing with Python on PluralSight. Certificate earned on January 27, 2018

Introduction to Data Science in Python by University of Michigan on Coursera. Certificate earned on December 13, 2017

Certifications:

Cloudera Spark and Hadoop Developer (CCA 175), Certificate earned on May, 2019

Oracle SQL Fundamentals (1z0-051), Certificate earned on Jan, 2019

Contact this candidate