Senior Splunk Engineer

This is a hybrid position, 2-3 days onsite in Arlington, VA.

TS/SCI clearance required. TS/SCI Poly preferred

Splunk Certified Administrator required, Certified Architect preferred

Our client is seeking a Senior Splunk Engineer to support a premier analytics platform. This engineer will join a high-performing cloud and cybersecurity team, directly supporting critical initiatives to modernize and secure the enterprise's analytics capabilities. You will play a pivotal role in moving Splunk from AWS cloud platform to CI/CD pipelines. You will also be enhancing Splunk deployments, optimizing data ingestion, and ensuring seamless performance through infrastructure automation, security best practices, and continuous integration.

Responsibilities:

Architect, deploy, and manage enterprise-level Splunk environments in alignment with CI/CD best practices.

Design and implement Splunk infrastructure using Terraform, Ansible, and GitLab to support automated, scalable deployments.

Lead version upgrades across clustered Splunk environments; manage Indexers, Search Heads, and Universal Forwarders.

Ingest and normalize diverse data sources (Syslog, HEC, APIs, log monitoring) and optimize for performance and license usage.

Develop documentation, user guides, and internal SOPs for streamlined knowledge transfer across engineering teams.

Create dashboards, reports, alerts, and custom visualizations to support mission operations.

Support SSL configuration, STIG compliance, and RHEL patching for secure deployments.

Collaborate with DevOps, Cloud, and Security teams to troubleshoot issues and implement security analytics using Splunk ES and UBA.

Interface with end users, government stakeholders, and analysts to improve Splunk adoption and performance across the platform.

Qualifications:

3+ years of hands-on experience with Splunk Enterprise deployments, upgrades, and data onboarding.

Experience administering Linux (RHEL/CentOS) and Windows systems.

Experience with infrastructure-as-code tools like Terraform and Ansible.

Proficiency with scripting languages such as Python or Bash.

Strong understanding of Splunk configuration files (inputs.conf, props.conf, transforms.conf).

Experience managing clustered environments across bare metal and VM infrastructures.

Familiarity with AWS and cloud-native technologies is a plus.

Splunk Certified Administrator required; Splunk Certified Architect (preferred or in-progress).

CompTIA Security+ (DoD 8570 IAT II compliant).

Excellent verbal and written communication skills, and ability to collaborate in agile team environments.

Preferred:

Experience with Splunk Enterprise Security (ES), User Behavior Analytics (UBA), and automation pipelines.

Knowledge of Docker, Kubernetes, or Ansible in DevSecOps pipelines.

Familiarity with compliance frameworks, endpoint tools (Tanium, Palo Alto), and vulnerability management.