Data Engineer - Cloud, SQL, Spark, Airflow, Pipelines

SUMMARY

●Data Engineer with * year of experience in self-learning and doing pipelines to collect and process, and solve data problems with powerful tools such as Spark, SQL, Cloud (AWS), Airflow DAGs.

●Committed to continuous learning and skill development with a passion for technologies that solve problems with digital data, improving knowledge every day, and updating new technologies.

●Proficient in thorough research and document review to effectively solve technical challenges.

●Ability to perform effectively even in high-pressure environments.

●A team player with a collaborative spirit, while also proficient in working independently on complex challenges.

PROFESSIONAL EXPERIENCE

Trained in a practice-oriented IT environment within the FPT education ecosystem, gaining hands-on experience in software engineering principles and real-world project development, building a strong foundation for a Data Engineering career.

ANALYZE STOCK END TO END

GitHub – Document

●Designed and implemented a full-scale ELT data pipeline to integrate financial and stock market data from Sec-api.io, Alpha Vantage, and Polygon into a centralized data warehouse.

●Analyzed API structures, defined ingestion strategies, and evaluated data latency and transformation requirements.

●Designed relational and dimensional data models (Galaxy Schema) to support analytical reporting.

●Built modular Airflow DAGs for orchestration, automated ingestion, transformation, and loading routines.

●Created technical documentation including data lineage, architecture diagrams, and process flows.

KAFKA SPARK STREAMING STRUCTURE ARCHITECTURE

GitHub – Document

●Built a real-time streaming pipeline using Kafka, Spark, MinIO, and PostgreSQL for sentiment data analytics.

●Designed real-time data schema and Spark validation layers to ensure data quality and consistency.

●Developed Dockerized architecture integration Airflow for orchestration and automated stream processing.

●Implemented monitoring logic with Prometheus & Grafana for system observability.

●Authored comprehensive technical documentation and enable REST API access via FastAPI.

DATA EXPLORATION AND PREDICT IN DATA HOME CREDIT

Drive Folder

●Using the Pandas library for data processing and analysis and the Numpy library for numerical computation. Using

Matplotlib and Seaborn for visualization. Finally, predictive analysis with Sklearn.

EDUCATION

Business and Technology Education Council FPT College (BTEC) Ho Chi Minh, Vietnam

Computing Degree

Major in Software Engineer 2021-2024

TECHNICAL SKILLS

S1. S3, MinIO S2. Transformation (PySpark, SQL) S4. Analyze with Jupiter Notebook S5. Visualization with Power BI LANGUAGES

●English: T O E I C 7 5 0

●Vietnamese: native

S3. PostgreSQL

S6. Orchestration (Airflow)

Contact this candidate