Data Engineer

Job brief

We are seeking a Software Engineer to execute the full lifecycle of the product development, by programming well-designed, efficient, and testable code that meets specifications.

Requirements

AWS development expertise.

Pandas, PySpark, AWS Glue, S3, RedShift, SQL,

Knowledge to have complex sql queries with multi-table joins, and joins of various types.

AWS services: Lambda, S3, Step Functions, Glue, EC2, ECS, CloudFormation, RDS, CloudWatch, Redshift, REST API, Docker

Additional skills required are: Git, Terraform, Jira, Confluence, and Agile.

For framework, they use Pytest, so Pytest knowledge would be helpful

It would be great if the candidates have experience with shell scripting, AWS CLI based executions, application builds and development unit test executions.

Excellent communication (oral and written) skills are also required.