Data Engineer

Requisition ID: 225636

Thanks for your interest in ScotiaTech, Scotiabank's new and innovative Technology hub in Bogota.

Join a purpose driven winning team that promotes creativity and innovation in a fast-paced environment, where we’re always committed to results, in an inclusive, diverse, and high-performing culture.

Purpose

The DataOps Engineer is responsible for building, maintaining, and optimizing scalable and reliable data pipelines and infrastructure in a cloud environment, under the guidance of the Senior DataOps Engineer. This role collaborates with a global team of DataOps Engineers and cross-functional stakeholders to ensure high-quality data products and services that support the organization’s data-driven goals. By implementing DataOps best practices and leveraging cloud technologies, the DataOps Engineer contributes to efficient data workflows, operational reliability, and seamless delivery across multiple countries.

Accountabilities

• Data Pipeline Development and Maintenance:

- Design, develop, and maintain cloud-based data pipelines using platforms such as Azure, GCP or AWS (e.g., Dataflow, BigQuery, GCS, S3, Lambda functions, Cloud Functions, Composer).

- Implement automated data workflows, including ETL processes, data quality checks, and orchestration using tools like Python, Spark, Airflow or Cloud Composer.

- Ensure pipelines meet performance and reliability targets (e.g., 99.5% uptime, <1% error rate) as defined by SLAs.

• Cloud Infrastructure Support:

- Deploy and manage cloud data infrastructure components, ensuring scalability, security, and cost efficiency under the Senior DataOps Engineer’s direction.

- Apply infrastructure-as-code (IaC) practices using tools like Terraform or CloudFormation to provision and manage resources.

- Monitor and optimize cloud resource usage (e.g., storage, compute) using tools like Azure Cost Management, or GCP Billing.

• Automation and CI/CD Implementation:

- Build and maintain CI/CD pipelines for data workflows using tools such as Jenkins, GitHub Actions, Cloud Build and Jfrog to enable rapid and reliable deployments.

- Automate repetitive tasks, such as data validation and monitoring, to improve operational efficiency.

• Collaboration and Global Teamwork:

- Work closely with Data Engineers, Data Scientists, and business teams to deliver data products that meet business requirements.

- Collaborate with global DataOps team members across multiple time zones, contributing to shared goals and knowledge sharing.

- Provide regular updates on task progress and escalate issues to the Senior DataOps Engineer as needed.

• Monitoring and Incident Response:

- Monitor data pipelines and infrastructure using tools like Cloud Monitoring, Prometheus, or Cloud Logging to ensure operational health.

- Participate in incident response and troubleshooting, conducting root cause analysis to resolve issues within defined SLAs (e.g., <4 hours for critical incidents).

• Compliance and Security:

- Implement data security measures, such as encryption to Personal Identifiable Information and role-based access control (RBAC), in line with global regulations (e.g., GDPR, CCPA).

- Support audits and documentation to ensure compliance with organizational and regulatory standards.

• Control versioning:

- Deploy and manage repositories using git to keep and track changes on time

- Apply best practices on versioning to create branches and modifications around the different tool that allows systems of control versioning

• Continuous Improvement:

- Contribute to the development of reusable frameworks, scripts, and documentation to enhance team efficiency.

- Stay informed about DataOps and cloud technology trends, proposing improvements to processes and tools.

Education / Experience / Other Information (include only those that are specific to the role)

• Education:

- Bachelor’s degree in computer science, Data Engineering, Information Technology, or a related field.

• Experience:

- Minimum of 3 years of experience in data engineering, DevOps, or DataOps roles, with hands-on experience in cloud environments (e.g., AWS, Azure, GCP).

- Proven experience building and maintaining data pipelines using tools like Dataflow, Spark, or BigQuery.

- Familiarity with CI/CD pipelines and automation tools (e.g., Jenkins, GitHub Actions, Cloud Build).

- Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation) and monitoring solutions (e.g., Cloud Monitoring, Prometheus).

- Exposure to working in distributed teams or cross-functional environments is preferred.

- 3-5 years using Python and SQL

• Certifications (Preferred):

- Cloud certifications such as Google Cloud Associate Data Engineer, Azure Data Fundamentals, AWS Certified Developer.

- Entry-level DevOps or Data Ops certifications are a plus.

• Skills:

- Proficiency in programming languages like Python, SQL, or Scala for data pipeline development.

- Proficiency in orchestration tools like Airflow, Rancher, Kubernetes, Terraform.

- Solid understanding of Data Ops principles, including automation, ETL and ELT, version control, and observability.

- Strong problem-solving skills and attention to detail in managing complex data workflows.

- Effective communication skills to collaborate with global teams and document technical processes.

- Ability to work independently and manage tasks in a fast-paced, distributed environment.

Working Conditions

• Work in a standard office-based environment; non-standard hours are a common occurrence.

#LI-hybrid

Location(s): Colombia : Bogota : Bogota

ScotiaTech is a business unit within ScotiaGBS, a Scotiabank Group company located in Bogota, Colombia. The ScotiaTech hub was created to support different technology systems and processes of the Bank. We offer an inclusive, positive work environment, and competitive benefits.

At ScotiaTech, we value the unique skills and experiences each individual brings and are committed to creating and maintaining an inclusive and accessible environment for everyone. Candidates must apply directly online to be considered for this role. We thank all applicants for their interest in a career at ScotiaTech; however, only those candidates who are selected for an interview will be contacted.