Data Engineer - Amazon Athena

Responsibilities : - Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services.

- Design, develop, and maintain data pipelines using AWS serverless technologies such as - Glue, S3, Lambda, DynamoDB, Athena, and RedShift.

- Implement data modeling techniques to optimize data storage and retrieval processes.

- Develop and deploy data processing and transformation frameworks to support both real- time and batch processing requirements.

- Ensure data pipelines are scalable, reliable, and performant to handle large-scale data sizes.

- Implement data documentation and observability tools and practices to monitor . 1.

Hands on experience of Spark, Scala and conversant with SQL (Scala AWS is mandatory) 2.

Good knowledge on Hadoop (Oozie) 3.

Reverse engineer the SQL queries, Scala code to understand functionality 4.

Capable of identifying, analysing and interpret patterns and trends in complex data sets 5.

Should have strong experience on AWS (EMR, S3) 6.

Has worked on creating database design, data models & techniques for data mining.

(ref:hirist.tech)