Data Systems Engineer ELK / Kafka / Linux

Job Role: Data Systems Engineer – ELK / Kafka / Linux

Team: Real-Time Operations Intelligence (RTOI) – Enterprise Computing

Location: Hybrid

Alpharetta, GA or Menlo Park, CA

3 days onsite per week

Experience Level: 7–15 years

Education: Bachelor’s Degree preferred (not required)

Industry Background: Plus

Role Financial services /Banking/Investment banking

The Real-Time Operations Intelligence (RTOI) team is responsible for streaming terabytes of data daily to support enterprise-scale operational and business intelligence platforms. The team builds and supports large-scale, real-time ETL and streaming pipelines using Kafka, ELK (ElasticSearch), Snowflake, Hadoop, and Linux-based job frameworks.

This role is ideal for a hands-on Data Systems Engineer who is equally comfortable with application development, data engineering, Linux-based deployment, and production support. The engineer will work across the full development lifecycle and support hundreds of internal customers relying on real-time data systems.

Responsibilities

(Including but not limited to)

• Design, develop, deploy, and support real-time data pipelines using Kafka and ELK (ElasticSearch).

• Build and maintain large-scale ETL and streaming frameworks running on Linux platforms.

• Develop and run applications directly on Linux, including debugging CPU, memory, and performance issues.

• Support and monitor pipelines running across large-scale Kafka clusters, ensuring high availability and scalability.

• Troubleshoot and resolve production issues; ensure jobs are up and running for hundreds of internal users.

• Work with data storage and indexing in ElasticSearch, understanding how data is written, stored, and queried.

• Participate in the full software development lifecycle: requirements, design, implementation, testing, deployment, and support.

• Collaborate closely with cross-functional teams and communicate technical concepts clearly.

• Continuously learn new tools and technologies and contribute hands-on in a fast-paced environment.

Required Qualifications

• Strong, hands-on experience working on the Linux platform (development, deployment, debugging).

• 7+ years of overall professional experience in software and/or data engineering.

• Strong application development experience with:

• Python (primary)

• Ruby or Shell scripting (secondary)

• Experience building and maintaining Kafka-based data pipelines.

• Hands-on experience with ELK (ElasticSearch) for data ingestion, storage, and observability.

• Ability to understand and debug application behavior related to CPU, memory, and system performance.

• Experience working in distributed systems environments, with an understanding of scalability and trade-offs.

• Strong communication skills, team collaboration, curiosity, and willingness to “get hands dirty.”

Preferred Qualifications

• Experience with Snowflake database.

• Experience with Spark or large-scale data processing frameworks.

• Strong data analysis background.

• Experience with Flink.

• ELK / ElasticSearch certification (Observability or Data Analysis).

• Experience with cloud platforms (AWS or similar).

• Experience supporting mission-critical, real-time systems.

Technical Environment

Languages: Python, Ruby, Shell (plus Java, C/C++, or Go a plus)

Streaming & Data: Kafka, ElasticSearch (ELK), Snowflake, Hadoop

Platforms: Linux (on-prem and cloud)

Databases: SQL-based systems

Focus Areas: Real-time streaming, observability, scalability, and operational support

Interview Process

• Technical Screening (1 hour) – Focus on Linux experience and hands-on technical background

• Onsite Technical Panel – With senior team members (Ying-Yi & Yenni)

Additional Notes

• This is not a narrow or cookie-cutter data engineering role.

• Candidates must be both data engineers and application developers, not tooling-only profiles.

• The role includes development, deployment, and production support.

• Team works directly within Linux environments—deep Linux knowledge is critical.