Observability Engineer - Splunk & Datadog Expert

+1-480-***-**** **************.*@*****.***

.

UMAMAHESWARRAO VASAMSETTI

PROFESSIONAL SUMMARY

Versatile and highly technical Splunk Observability Engineer with 12+ years of IT experience specializing in Splunk Development, Administration, Architecture, SIEM, ITSI, and largescale infrastructure monitoring. Skilled in designing and deploying highly available, scalable, faulttolerant, and selfhealing monitoring solutions across distributed, clustered, and multisite environments. Experienced in standardizing Splunk forwarder deployments, optimizing ingestion pipelines and implementing advanced ITSI modules including service analyzer, glass tables, event correlation and predictive analytics.

Expertise in DataDog, specializing in fullstack monitoring, distributed tracing, and intelligent alerting across complex, cloudnative environments. Skilled in building advanced dashboards, custom log pipelines, and APM instrumentation to deliver actionable insights and reduce incident resolution times. Known for driving monitoring governance, automating DataDog provisioning through IaC, optimizing performance analytics, and producing precise, auditready rootcause analyses by correlating metrics, logs, traces, and deployment events.

Experienced with observability platforms such as SignalFx and integrating Splunk with APM tools for metrics, tracing, and realtime monitoring. Recognized for delivering robust monitoring ecosystems that enhance operational visibility, support business continuity and drive datadriven decisionmaking.

SKILLS

Splunk Modules: Splunk 8.x,9.x, Splunk Enterprise, SIEM, Splunk DB Connect, Splunk Cloud, Splunk Web Framework, ITSI, APM, Synthetics

Languages: SQL, PL/SQL, Unix Shell Scripts, JSP, Java J2EE, CSS, HTML, XML, Advanced XML, Python

RDBMS: Oracle 11g/10g/9i/8i, MS SQL Server 2000/2005/2008, Sybase

Security Tools: Palo Alto, Vulnerability, OpenVAS, Fire Eye

Operating Systems: Red Hat Enterprise Linux 6X/7X, Sun Solaris 9/10, Windows 2010/2008 OS X 10.6/10.7/10.8/10.9

Tools: DataDog, ServiceNow, AppDynamics, Splunk on Splunk, Btool, Splunk DB2 Connect, Qlick Sense, ELK Stack, Ansible, Logic Monitor, Terraform

WORK HISTORY

SITE RELIABILITY ENGINEER 02/2025 to Present

SMBC Manu Bank, USA

Worked on Splunk Searching and Reporting modules, Knowledge Objects, Administration, Add-On's, Dashboards, Clustering and Forwarder Management, Visualizations, alerts, reports

Experience in creating various types of charts, Alert Settings, Knowledge of app creation, user and role access permissions

Field Extraction, Using Rex Command and confident in using Regular Expressions

Extensively used various extract keywords, search commands like stats, chart, time chart, transaction, strptime, strftime, eval, where, xyseries, table etc

Troubleshooting multiple event types using work flow actions

Created and Managed Splunk Database connect Identities, Database Connections, Database, Inputs, Outputs, lookups, access controls

Managing, configuring and administering a distributed environment multi-site clustering, Search-Head clustering

Integrated DataDog with incident management workflows to create intelligent alerting, reducing noise and improving MTTR through contextual, correlated alerts

Standardize Splunk forwarder deployments, configurations and maintenance across a variety of UNIX and Windows platforms

Built and optimized SPL queries, reducing dashboard load times by 40%

Implementing instrumentation for services using Open Telemetry, Splunk APM agents, log forwarders, and metric exporters across microservices

Managing data ingestion pipelines Configuring HEC (HTTP Event Collector), forwarders, OTel collectors, and ingestion rules

Integrating Splunk with CI/CD and cloud platforms Connecting AWS/GCP/Azure services, ServiceNow for ticketing

Troubleshoot ingestion and performance issues, resolving data gaps and improving MTTR by 25%

Supported incident response and root-cause analysis during outages, reducing MTTR by 40%

Configure and manage LogicMonitor for endtoend infrastructure, application, and cloud monitoring across hybrid environments

Develop and customize dashboards to provide realtime visibility into system performance, application health, and business KPIs

Implement and maintain security best practices to protect data and systems, including access controls, encryption, and vulnerability assessments

Integrate LogicMonitor with ITSM and collaboration tools such as ServiceNow, Jira, Slack, PagerDuty, and Teams

SOFTWARE DEVELOPER 04/2024 to 11/2024

The Hershey's, USA

Identifying bad searches/dashboards and partnering with the creators to improve performance

Design Splunk systems to meet growth while maintaining a balance between performance, stability, and agility

As a Splunk SME providing input into strategies, capabilities and integrations to improve the availability and performance of applications

Working closely with Service Owners to review service delivery quality with a focus on continuous improvement

Troubleshooting issues related to Splunk infrastructure, including performance bottlenecks, data ingestion problems, and search optimization

Collaborating with cross-functional teams including security, network, and system administrators to ensure seamless integration of Splunk within the IT infrastructure

CLOUD TECH SUPPORT SPECIALIST 01/2023 to 04/2024

Teachers Insurance and Annuity Association (TIAA), USA

Implemented holistic observability ecosystem in DataDog, unifying metrics, logs, traces, synthetics, and APM to deliver fullstack visibility across distributed services

Designed domainspecific dashboards and service maps that transformed raw telemetry into actionable insights for engineering, Apps, and leadership teams

Built custom log pipelines and processors to normalize, enrich, and route highvolume logs, improving searchability and reducing ingestion costs

Implemented APM instrumentation and distributed tracing, enabling deep performance analysis and pinpointing latency bottlenecks across microservices

Automated DataDog provisioning using infrastructureascode (Terraform/API), ensuring consistent configuration, tagging standards, and environment parity

Established monitoring governance frameworks, enforcing tagging taxonomies, naming conventions, and data hygiene across all monitored assets

Leveraged DataDog’s anomaly detection, forecasting, and outlier analysis to proactively identify performance degradation before impacting customers

Conducted crosssignal RCA investigations, correlating metrics, logs, traces, and deployment events to produce precise, auditready incident narratives

Manage the sunsetting of legacy monitoring tools, consolidating observability into DataDog and reducing operational overhead while improving coverage and reliability

SR ANALYST 07/2021 to 12/2022

Bausch Health Companies Inc, USA

Design, Deploy and Support enterprise Splunk logging application and assist other enterprise instances

Performs Health checks of the Splunk environment, troubleshoot and restore service

Created Dashboards, report, scheduled searches, alerts and knowledge objects like data models, macros, lookups, custom scripted inputs

Performs Splunk Enterprise Upgrades on Splunk cluster components(Indexers, Search Heads, HF's, Cluster master and etc)

Interact with Splunk user base for the development, management and tuning of Splunk dashboards, knowledge objects, ad-hoc/scheduled searches and alerts.

Integrates data streams, feeds from network, infrastructure services, mission critical/business applications into Splunk using the Splunk Universal Forwarder, Syslog, Splunk Heavy Forwarders and Splunk HEC Clusters

Created and Managed Splunk Database connect Identities, Database Connections, Database, Inputs, Outputs, lookups, access controls

SPECIALIST 02/2020 to 07/2021

Farmers Insurance Group, USA

Developed end-to-end visualization reports for system performance, capacity and key business transactional dashboards to maintain operational availability of delivered solutions

Responsible for implementing identified road maps, leading design reviews, and performing analysis for prod, pre-prod and test environment applications

Performing maintenance and optimization of existing clustered Splunk deployments

Technical writing/creation of formal documentation such as reports, training material, slide decks, and architecture diagrams

Managing the Splunk components like indexers, search heads, both heavy/universal forwarders, deployment server, master node, license master and etc

Responsible for adding customer context, eliminate noise and false positives, and develop trends and data models

SR INFRASTRUCTURE DEVELOPER 05/2018 to 01/2020

California State Automobile Association (CSAA), USA

Worked with Client engagements and data onboarding and writing alerts, dashboards using the Search Processing Language (SPL)

Monitors, analyzes, enriches and parses logs from a variety technology across multiple platforms such as IDS/IPS (Sourcefire, Dell secure work)

Involved as a Splunk Admin in capturing, analyzing and monitoring front end and middle ware applications

As part of SIEM, monitored notable events through Splunk Enterprise Security (Using V3.0)

Generated Shell Scripts to install Splunk Forwarders on all servers and configure with common Configuration Files such as Bootstrap scripts, Outputs.conf and Inputs.conf files

Parsing, Indexing, Searching concepts Hot, Warm, Cold, Frozen bucketing and Splunk clustering

Splunk DB Connect 2.0 in search head cluster environments of Oracle, MySQL

Designed and implemented a NoSQL based database and associated RESTful web service that persists high-volume user profile data for vertical teams

Deployed and maintained the Splunk UBA application, DB2, service-now applications etc

Integrating third party applications with Splunk like pager duty, service-now and etc

SPLUNK DEVELOPER 11/2015 to 05/2018

CVS Health Corporation, USA

Created accurate Reports, Dashboards and Visualizations for various types of business use cases

Understanding of network architecture and implementation to support effective log collection and processing

Worked with administrators to ensure Splunk is actively and accurately running and monitoring on the current infrastructure implementation

Created alerts based on the critical parameters, which will trigger emails to the operational team

Manage and deploy Splunk architecture and various components (indexer, forwarder, search head, deployment server, Universal forwarder, License master)

Worked as a Splunk Admin for Creating and managing app, Creating users, role, Permissions to knowledge objects

Created Admin, Power Users and User roles for the application and created the app sharing permissions for the different roles

ASSOCIATE SOFTWARE ENGINEER 06/2014 to 11/2015

Vanna Info tech India Pvt Ltd, Hyderabad, India

Developed the View pages in JS, CSS, JavaScript validations, and business, service layer coding

Developed Web Application using MVC architecture

Integrated with REST API's and developed functionality/modules with a focus on usability, reliability and supportability

Document technical design, process flow and support plans

Consult with various implementation and quality-assurance teams to create and execute unit tests for all code developed

Collaborated with cross-functional teams to enhance system functionality

Developed business components and configured using hibernate and involved in bug fixes

Developing the system Unit &Integration Testing & debugging

EDUCATION

JNT University Hyderabad, Hyderabad, India

Bachelor of technology, Information technology, 01/2013

CERTIFICATIONS

Splunk Core Certified User

Splunk Core Certified Power User

Splunk Core Certified Admin

Security+

Contact this candidate