Lead Big Data Platform Engineer

Join us at The Hartford as a Lead Big Data Platform Engineer, where we're driven to make a positive impact beyond just coverages and policies. Working with us means having the chance to achieve your own goals while helping others achieve theirs. Become a part of our team and help shape the future!

As a vital member of our Cloud Big Data Platform Engineering & Operations team, you will play a crucial role in promoting a customer-first mindset while ensuring that our data platforms are stable, scalable, and reliable, meeting the ever-evolving needs of our Data & Analytics applications across the enterprise.

In this technical lead position, you will leverage your extensive expertise in AWS Big Data/EMR infrastructure, Infrastructure as Code (IaC), automation, security, and observability to craft, optimize, and maintain robust, scalable solutions. You will work closely with data engineers to analyze requirements and challenges to recommend platform-driven solutions that maximize performance and efficiency.

We seek a passionate technologist who thrives in a fast-paced environment and is committed to building resilient, future-ready data platforms. You will also mentor and guide junior engineers, fostering a culture of collaboration, technical excellence, and innovation.

Key Responsibilities:

Administer and engineer Big Data platforms across multiple AWS EMR Hadoop clusters, including serverless and containerized environments, ensuring optimal scalability, performance, and reliability.

Design, implement, and maintain highly scalable, resilient multi-tenant Data Platforms using Infrastructure as Code (IaC), adhering to The Hartford’s engineering, security, and governance principles.

Drive operational excellence by managing high-impact incidents to minimize service downtime and its effect on business operations, demonstrating complete ownership.

Implement Site Reliability Engineering (SRE) practices by designing and deploying effective tooling, proactive alerting, and automated responses that identify and address reliability risks.

Shape the architecture of Platform, PaaS, and SaaS solutions to meet current and future business needs, promoting innovation and operational excellence.

Participate in an on-call rotation, providing hands-on expertise during service-impacting incidents to ensure rapid diagnosis and resolution.

Assess, implement, and manage new data technologies focused on big data, analytics, and data visualization to advance efficiency and innovation.

Act as a subject matter expert and technical authority for data platforms, tools, and applications, leading technical issue resolution and ensuring platform reliability.

Offer technical leadership and mentorship to junior and mid-level data engineers, fostering skill development and promoting engineering best practices.

Enable data engineers, data scientists, and analysts by providing self-service capabilities for data exploration and analysis.

Develop training materials and deliver training sessions to enhance the adoption and use of data solutions and improve customer engagement.

Support documentation and management of metadata and data assets to enhance discoverability and promote self-service analytics.

Encourage a culture of accountability and collaboration, ensuring strong team commitment to shared priorities and strategic objectives.

Qualifications:

Bachelor’s degree in Computer Science, Computer Engineering, or a related field (or foreign equivalent). Relevant experience through qualifying academic programs may also be considered.

Extensive platform administration expertise in big data technologies, data engineering, analytics, and operations.

Proficient in tools and technologies including Hadoop, Linux, Python, SQL, Spark, Kerberos, and production engineering.

A minimum of 7 years of progressive experience in IT, platform administration, or database management.

At least 3 years of hands-on engineering experience developing platform solutions on AWS using Infrastructure as Code (CloudFormation, Terraform, Ansible) and CI/CD pipelines.

At least 3 years of experience providing architectural guidance to developers or platform administrators.

Knowledge of data engineering, with experience designing and building data applications leveraging database platforms.

Proven experience in data management processes including data ingestion, preparation, ETL, aggregation, and analytics application development.

Skilled in change and incident management processes.

Experience with reliability engineering practices and observability tools is a plus.

Ability to rapidly learn new technologies and perform job responsibilities proficiently within 6-12 months.

Strong analytical skills, problem analysis techniques, and a broad awareness of technology alternatives.

Excellent communication skills, with the ability to effectively collaborate with business and IT stakeholders.

Preferred Skills:

Advanced experience with key AWS services (EMR, EKS, EC2, IAM, RDS, Route53, S3, etc.).

Experience in configuration management using CloudFormation or Terraform.

Advanced skills in programming/scripting languages such as Python or Bash.

AWS Solution Architect certification is preferred.

Cloudera Admin certification is also preferred.

This role supports a hybrid work schedule, with expectations of in-office work in Columbus, OH, Chicago, IL, Hartford, CT, or Charlotte, NC, 3 days a week (Tuesday through Thursday).

Candidates must be authorized to work in the US without company sponsorship. The role will not support STEM OPT I-983 Training Plan endorsement.

Compensation:

The listed annualized base pay range is based on market analysis of similar positions. Actual pay may vary based on performance and individual competencies. The base pay is part of The Hartford's total compensation package, which may also include bonuses, incentives, and recognition programs. The annualized base pay range for this position is:

$136,000 - $204,000

The Hartford is an Equal Opportunity Employer/Sex/Race/Color/Veterans/Disability/Sexual Orientation/Gender Identity or Expression/Religion/Age.

Join us in our mission of doing right by our customers, communities, and employees for over 200 years. Be a part of our journey in innovating and serving those around us.