Data Engineer Google Cloud

Carrier Objective:

I am actively seeking a progressive organization that prioritizes individual and professional development. My goal is to become part of a forward-looking team where I can apply my technical skills to support the organization's goals and play a significant role in its accomplishments. PROFESSIONAL SUMMARY:

Having total 4+ years of experience in Data Engineering using Google Cloud platform, Apache Spark, Python, and SQL.

Hands-on experience of Google Cloud Platform components and Big Data technologies. Having experience in various Google Cloud Products like Cloud Composer, BigQuery, Cloud Storage, Cloud Compute Instance, Cloud Data Fusion. Experience Cloud computing – Development, Implementation. Strong experience in migrating on-premise Data warehouse applications to Cloud. Hands on experience on Apache Spark.

Strong experience in Spark code to perform ETL.

Strong experience in SQL scripts as per the requirement. Worked on importing and exporting data from different databases like Oracle, SQL Server and MySQL,API into GCP.

Capable of processing large datasets (both structured and semi structured) by writing SQL Queries as per business requirements and supporting system application architecture. Working experience on GCP partitioning and clustering. Experience with RDD, Spark Data frames, Spark SQL. Strong experience in advance SQL like Window function. Experience in writing complex SQL queries using joins, functions, sub queries and correlated sub queries.

Involved in writing data transformations, data cleansing using Apache Spark operations.

Experience in working on GCP services - BigQuery, GCS, Airflow Composer, DataProc and.

Experienced on cloud Platform services SaaS, IaaS and PaaS. Event based ingestion of files in cloud’s data warehouse solution, on receipt of files at cloud Aditya A Joshi

GCP Data Engineer (Google Cloud)

Email:********************@*****.*** Phone: +91-844******* LinkedIn: linkedin.com/in/aditya-joshi-914429313

storage (big query in case using Google cloud platform). Good knowledge of Software Development Life Cycle and Extensive experience working in an Agile development environment.

Knowledge on GCP Service Like Dataflow

EDUCATION:

• Completed B.E from RTMNU, Nagpur University.

TECHNICAL EXPERTISE:

Google Cloud Platform (GCP) GCS, BigQuery, Cloud Composer, Airflow, Data Proc, Pub/Sub, Dataflow,DataFusion

Big Data Hadoop, MapReduce, PySpark

Scripting Languages SQL, Python

Project Execution Tools Agile – JIRA, Git, GitHub

Data Formats CSV, JSON

Operating Systems Windows

WORK EXPERIENCE:

Working as a GCP Data Engineer at INVENTZO SYSTEMS (INDIA), from July 05, 2021 to till date. PROFESSIONAL EXPERIENCE:

Projects Details:1

Client & Domain Woolworths (Retail)

Project Title: Sales Trend

Role: GCP Data Engineer - Developing or troubleshooting SQL’s on Big Query, GIT HUB and Google Cloud Console.

Technology: Google Cloud Platform

Descriptions: Woolworths Group is one of Australia and New Zealand leading retail Groups, Supporting Well-Known brands such as Woolworths, Big W and Countdown. Our great team is focused on creating better experiences together, for Our customers, Our communities and for each other.

Key Responsibilities:

As a Data Engineer, I have successfully developed efficient scripts to seamlessly transfer data from Google Cloud Storage (GCS) to BigQuery. I took a proactive approach to organize the data in BigQuery tables, implementing partitioning strategies for improved performance and data management. Extensive experience in designing and implementing data integration solutions on the Google Cloud Platform

(GCP), using services such as Cloud Storage, BigQuery. By thoroughly understanding the business needs, I analyzed and structured the raw data to align with specific requirements.

To support the organization's objectives, I established well-structured BigQuery datasets, tables, and pipelines to store and process the data effectively.

To facilitate the loading process from GCS to BigQuery, I skillfully utilized Google Cloud Shell, ensuring a smooth and reliable data transfer experience.

Dealing with diverse file formats such as CSV, JSON were second nature to me, and I proficiently managed data in these formats to meet downstream application needs. For added data security and access control, I created authorized views in BigQuery, ensuring the right stakeholders could access the necessary data.

To maintain version control and streamline collaboration, I effectively employed GitHub as the Source Code Management tool, enabling seamless code management and deployment. Maintained clear and concise documentation for data pipelines, data models, and architectural decisions to facilitate knowledge sharing and team collaboration. Created python code for various Google cloud API clients like BigQuery, Configuration, DataProc and many more.

Created Python utilities which will be used to connect source and target systems for the data process we have used PySpark.

Updated existing PySpark code and tried to make them more generic across the platform. There are various frameworks with their specific use and I am part of developing most of their code. Performing pre-processing and cleaning on data using Spark. Develop SQL Queries for fetching the data from database. Involved in knowledge sharing activities with team. Projects Details:2

Client : Brussels Airlines

Domain: Airline

Project Title: Revenue Generation

Duration: 2022 oct-2023 dec

Role: GCP data Engineer - Worked as a major part in GCS & Bigquery Analysis Techonology: Google Cloud PlatForm

Key Responsibilities:

Created Python utilities which will be used to connect source and target systems for the data processing we have used PySpark.

Updated existing PySpark code and tried to make them more generic across the platform. Performing pre-processing and cleaning on data using Spark. Develop SQL Queries for fetching the data from database. Involved in GCP architecture implementation for a complete cloud based big data solution using BigQuery, GCS,DataProc and Airflow.

Debugging and fixing the issues by sampling the data. Responsible for pull, push and committing the code using GITHUB. Sorting the extracted data in Oracle database for further querying and analysis. Responsible for debugging and troubleshooting during data processing. Creating database schemas for newly generated data. View logging information and start new task or resume task that have failed Projects Details:3

Client : Ryder supply chain solutions

Domain: logistics

Duration: 2021 sept-2022 Aug

Role: Associate GCP data engineer

Platform: Google Cloud Platform

Key Responsibilities:

Designed and developed end-to-end ETL pipelines in Cloud Data Fusion for ingesting data from multiple on-premise and cloud sources.

Implemented data cleansing, transformation, and enrichment using Wrangler, SQL Transform. Integrated pipelines with BigQuery, Cloud Storage, and Pub/Sub for both batch and real-time processing. Automated pipeline scheduling and monitoring using Cloud Composer (Airflow). Ensured data quality, lineage, and governance through proper schema design and metadata management. Optimized pipeline performance and cost by applying BigQuery partitioning, clustering, and query tun Collaborated with stakeholders to deliver business-ready datasets and dashboards in BigQuery and Looker Studio.

Contact this candidate