AB Initio - Advanced

Senior Ab"Initio Developer (7+ Years Experience)

About the Role

We are seeking a Senior Ab"Initio Developer to design, build, and operate large-scale, high-performance data pipelines supporting enterprise analytics and regulated reporting. You will work closely with data engineers, architects, and operations teams to deliver reliable, secure, and scalable ETL solutions.

Responsibilities

Core Ab"Initio Development

Design and develop complex, end-to-end Ab"Initio graphs using GDE, with strong attention to metadata propagation, partitioning, and checkpointing.

Build reusable, parameter-driven frameworks including shared subgraphs, standardized transforms, and consistent logging and error-handling patterns.

Develop efficient transform logic using DML, XFR, and embedded scripting, optimizing for maintainability and runtime performance.

Define and manage robust record formats, ensuring strict typing, nullable field handling, encoding/locale correctness, and safe schema evolution. Performance Engineering & Scalability

Design intentional parallelism strategies using appropriate partitioning methods (hash, range, round-robin, broadcast) while avoiding skew and hot spots.

Optimize joins, sorts, and aggregations by selecting appropriate strategies and minimizing unnecessary re-sorting.

Apply strong understanding of memory, I/O, spill behavior, buffer usage, compression, and file system throughput.

Benchmark and tune pipelines to identify bottlenecks and improve throughput without compromising correctness. Data Engineering Best Practices

Architect ETL workflows for both full and incremental processing, including CDC patterns where applicable.

Handle late-arriving data and support idempotent re-runs.

Implement strong data quality controls including profiling, validation rules, reconciliations, and actionable reject handling.

Maintain clear data lineage, column-level mappings, and auditability to support regulated and business-critical reporting. Platform & Operations

Integrate Ab"Initio workloads with enterprise schedulers, managing dependencies, calendars, reruns, and backfills.

Implement standardized operational practices for error handling, retries, quarantines, alerting, and observability.

Design restartable pipelines using checkpointing and safe partial reruns. Deployment & Engineering Hygiene

Follow disciplined promotion practices across development, test, and production environments.

Externalize configuration, parameters, and secrets; ensure no credentials are embedded in graphs or scripts.

Apply version control best practices using EME/metadata repositories and Git-based workflows.

Package and deploy graphs, metadata, scripts, and configurations in a reproducible manner with rollback support. Data Stores & Interfaces

Work with diverse file formats including fixed/variable, delimited, binary, and (where applicable) mainframe formats such as EBCDIC and COBOL copybooks.

Design and optimize database interactions using SQL, bulk load/unload strategies, and appropriate transaction handling.

Collaborate on integration with messaging or streaming platforms (e.g., Kafka) where part of the ecosystem. Security, Controls, and Compliance

Implement secure handling of sensitive data, including PII masking and tokenization in non-production environments.

Enforce least-privilege access controls and secure landing zones.

Support audit requirements through traceable pipelines, approval workflows, and evidence generation.

Use managed secrets with defined rotation practices; never hard-code credentials. Minimum Qualifications

7+ years of hands-on experience developing Ab"Initio ETL solutions in large-scale production environments.

Advanced proficiency with Ab"Initio GDE, DML, XFR, and complex graph design.

Strong understanding of parallel processing, performance tuning, and data engineering fundamentals.

Experience delivering data pipelines in regulated or high-control environments. Preferred Qualifications

Experience working with mainframe data formats and enterprise-scale data platforms.

Familiarity with CI/CD practices for data engineering.

Experience collaborating with cross-functional teams across architecture, analytics, and operations.