Data Engineer Site Reliability Engineer Middleware Messaging DevOps

SUMMARY:

Data Engineer with expertise in building data infrastructure and pipelines to transform raw data into actionable insights combined with extensive experience as a Low Latency Performance Engineer in Enterprise and High-Performance Message Oriented Middleware (MOM)

EXPERIENCE:

Tradeweb (Vice-President, Aug 2022 – Feb 2025)

As a Data Engineer, responsibilities included data architecture design, integration, cleansing, pipeline monitoring, and collaborating with cross-functional teams to ensure data quality and accessibility for analysis by data scientists. Technologies/tools used and daily activities are listed below:

Utilized Confluent Kafka Broker Cluster for real-time data ingestion, processed and stored in Amazon Web Services (AWS) S3 via Kafka Connect with S3 and MS SQL Server Debezium connectors. Data was then ingested into Snowflake from S3 using scheduled tasks and stored procedures for business consumption

Deployed and managed on-prem Kafka Broker stretch clusters across data centers for fault tolerance and DR

Also, administered on-prem/cloud Kafka Connect clusters with S3 and Debezium connectors, ensuring fault tolerance and disaster recovery through independent clusters in primary and backup data centers.

Used Apache Pinot for ultra-low-latency, concurrent queries on large datasets, including real-time streaming data from Kafka. Integrated Trino SQL query engine with Pinot and Kafka to serve real-time trading and monitoring applications with fast analytics. Deployed, managed, administered on-prem/cloud Pinot/Trino instances across data centers to provide for fault-tolerance and DR

Implemented a Change Data Capture (CDC) pipeline by enabling MS SQL Server and key Trading stream tables for CDC. Data events were ingested into Kafka via Debezium connectors, stored in AWS S3 via S3 connectors, and ingested into Snowflake for client analytics. The end-to-end pipeline was designed for fault tolerance

Managed AWS resources using Atlassian/Terraform, including

EKS Clusters for running ETL flows, Apache Pinot, Trino, and Kafka Connect

S3 Buckets for storing real-time or less-frequent streams, with cross-region replication for key streams

IRSA Roles for permissioning access to AWS resources

Used Snowflake for storing, analyzing, and sharing data across business units. Automated consistent Snowflake object creation across accounts using the Schema-Change tool, with definitions stored in GitLab. CI/CD pipelines invoked Schema-Change for object creation, updates, and deletions.

Implemented and deployed ETL data processing with Python, scheduled via Nomad and Prefect, using Vault for secret management.

GitLab for CI/CD to deploy ETL flows to Nomad and Prefect

ArgoCD for Kubernetes app delivery, with GitLab as the source of truth to ensure app state consistency

Helm Charts to manage app configurations and deployment to AWS EKS for Pinot and Trino

Ansible for deploying and managing on-prem Kafka Broker Cluster, Kafka Connect, Pinot, and Trino

Coralogix/Kibana for log troubleshooting, anomaly detection, and PagerDuty notifications

Prometheus for metrics collection, with Grafana for visualization and PagerDuty for alerts

Bank of America (Vice-President, Jul 2015 – Jul 2022)

Product Development:

AMPS Prometheus Grafana Visualization: Automated metrics collection and real-time visualization for

Federated Analysis: Global deployment insights for management

Federated Capacity Analysis: Quarterly regional capacity reviews

Instance Analysis: Application performance insights from AMPS metrics

Instance Capacity Analysis: Identifying CPU, Network, Memory, and Storage constraints on AMPS performance

Achieved with AMPS exporter implementation for Prometheus integration

Ansible ET-Deploy: Automated, global application deployment using Ansible for consistent configuration and eTrading stack deployment

MiFID-II ClockSync Regulation (EMEA):

Ensures compliance with MiFID-II by enriching trading records with high-resolution timestamps

Synchronized timestamps applied to network packets and decoded to extract trading records

Timestamped records are enriched with bank-specific identifiers and merged with trading application messages

PROVE (Performance, Risk and Operational Visibility into Electronic Trading) – Collectl

Monitors subsystems (CPU, memory, network, etc.) with minimal host CPU load

Collects data in a KDB database, retaining 10 days of logs

Includes plugins for application metrics and Azul GC log data

Active Product Sensor (APS): Automated eTrading stack real-time health check assessment in addition to acting as a designated client test subject in lieu of testing / experimenting with live customer orders. APS monitors:

Order Flow

Exchange Flow

Market Data Flow

Performance Engineering:

60E Advanced Message Processing System (AMPS):

Developed performance best practices to optimize AMPS latency under high throughput while maintaining resiliency, reliability, and availability

Client-side guidelines for QoS and fault tolerance; server-side deployment blueprints for stand-alone and high-availability configurations

Conducted OpenOnload evaluation for server latency and performance, comparing it to traditional TCP

Performance tests for AMPS queuing, server replication, and storage (Fusion IO vs. Intel P3700 NVMe)

Analyzed trading vs. non-trading flows, MiFID-II performance, and AMPS replication for a resilient trading architecture

Led migration of ETT Trading flow to AMPS and supported the Consolidated Audit Trail (CAT) system

ETT SOAK Performance Automation (SPA):

Developed an automated, end-to-end performance testing framework for the ETT stack, identifying potential capacity and performance issues

Automated performance visualization, test orchestration, and business flow between ETT components using Ansible

Created infrastructure and design for SOAK environment, including automation of testing, performance results tracking, and baseline configuration

Miscellaneous Automation Tools: Automated various processes to save time for critical tasks, using Ansible as the foundation:

Environmental check scripts for performance optimization

Data center-wide START/STOP operations for weekend checkouts

Automated Autosys configuration validation, linkage/SIFMA exercises, and Azul performance testing

Enhanced Spider script and enabled Autosys functionality

Credit Suisse Group (Assistant Vice-President, Sep 2003 – Nov 2005; Vice-President, Dec 2005 – Jun 2015)

Enterprise Messaging Product Management of TIBCO EMS:

Governance Model: Established EMS governance covering:

Application adoption, product engineering, release-cycle management

License and standards adherence

Integration with key components and vendor management

EMS performance, management, monitoring, and support model

Best Practices: Defined client-side and server-side EMS best practices, including:

Client-side: Quality of Service, fault-tolerant communication, LDAP integration

Server-side: Message routing, fault-tolerance (Veritas VCS/CFS for high availability), authentication/authorization, SSL

Messaging Integration: Developed messaging patterns for front-to-back office communication

Deployment Standards: Designed and implemented EMS deployment standards

In-House Consultancy: Provided EMS consulting on application messaging architecture, performance, scalability, and functionality

Messaging Working Group (MWG): Led MWG meetings to define enterprise messaging direction and gather business requirements

Vendor Relationship: Managed TIBCO relationship, resolved technical issues, and handled annual renewals and license management

Team and Project Management: Led and delivered messaging-related projects, providing team management and performance appraisals

Enterprise Messaging Projects:

TIBCO EMS (TIBEMS):

Evaluation: Assessed JMS v1.1 compliant products (TIBCO EMS, SonicMQ, IBM MQSeries) for performance, functionality, recovery, and manageability. Contributed to product selection favoring EMS

Automated Tools: Developed tools for EMS server instance creation with optimized configurations

LDAP Integration: Implemented LDAP as a centralized repository for EMS objects and configurations

SSL Implementation: Enabled SSL for secure client-server and server-server communication

Break-Fix Testing: Designed and tested Break-Fix scenarios for business continuity

Monitoring: Evaluated monitoring tools (Nastel, ITRS Geneos) for EMS

High Availability: Implemented Veritas VCS and Veritas CFS/CVM for fault-tolerant, high-availability EMS solution

J2EE Integration: Integrated EMS with WebLogic and INFA Power Center as a foreign JMS provider

Governance: Created EMS governance system for server management and license reporting

Migration: Developed a bridge for bi-directional messaging system migration

Messaging Abstraction Layers (MAL): Contributed to JMS, CMS, and NMS as MAL standards, enabling vendor-agnostic migration

CSAR Re-Engineering: Led the redesign of Cash Securities messaging architecture, achieving 2.7M trades/hr, surpassing market volatility estimates

AMQP Evaluation: Conducted functional evaluation of RedHat-Msg (QPID 0.14, AMQP 0-10), with a no-go decision. Future evaluation of QPID 0.18, AMQP 1.0 pending

High-Performance Messaging Product Management:

Informatica Ultra-Messaging (IUM):

Strategy: Defined IUM strategy for low-latency messaging

Governance Model: Established a governance model covering:

-Application adoption, product engineering, version management

-License usage, standards adherence, key integrations

-Optimum configuration for performance, management, and monitoring

-Support and vendor relationship management

Ratification: Secured Arch-STC ratification of IUM strategy and governance model

Monitoring: Defined UM monitoring requirements with stakeholder input

TAG: Chaired Low Latency Messaging TAG to steer direction and gather business requirements

Consultancy: Provided in-house consultancy for high-performance, scalable messaging architecture

Vendor Management: Managed INFA vendor relationship for bug resolution, license management, and renewals

High-Performance Messaging Projects:

Informatica Ultra-Messaging (IUM) Evaluation:

Evaluated IUM, Solace, and IBM WLLM based on high throughput, low latency, functionality, recovery, and manageability

Conducted requirements gathering, gap analysis, and proof of concept, leading to the decision to adopt IUM

IUM Applications: Implemented IUM across key trading systems such as Order & Trade Execution, Order State Management, Reporting, Intelligent Order Routing, Position Keeping, Risk Management, Algorithmic Execution, Market Data Handling, Pre-Trade Compliance, and Trade Capture

IPC Shared Memory: Achieved 1.71 µs mean latency @ 45 Kmps on Nehalems, exceeding the Automated Market Making desk's sub-5 µs requirements

Shared Memory Acceleration (SMX): Exceeded performance requirements with 185 ns mean latency @ 45 Kmps on Sandys, outperforming the sub-5 µs target

Monitoring: Integrated IUM SNMP statistics into a Network Management System (NMS) via NET-SNMP

Solace Messaging Appliance Evaluation – Evaluated Solace Messaging Appliance (v7.0.0.63), designing test cases and conducting hands-on evaluation across:

Throughput, Latency, and High-Frequency Trading

Order Execution, Smart Order Routing, and Event Distribution

Risk Management, Data Distribution, and Scale Down

Clustering, Fault Tolerance, and Server Consolidation

Web Messaging, Unified Fabric, and SSL Performance

Network Interconnect Evaluation – Explored high-speed network tech (Voltaire InfiniBand via VMA) for ultra-low latency (<1ms) and certified NICs for Multicast: 10Gb Solarflare, 10Gb Broadcom, 1Gb Intel

Record-Replay Projects:

Record Replay Service Evaluation

Defined functionality and performance requirements for Record-Replay as part of a proof-of-concept for TIBCO EMS and IBM WebSphere MQSeries

Evaluated and implemented Codestreet Replay Service for message recording and replay in Back-Office apps, proving its effectiveness for performance regression testing

Established best practices for Codestreet Replay Service deployment

Managed CodeStreet vendor relationship, communicated requirements, resolved technical issues, and coordinated annual renewals

Provided governance to ensure license compliance

Developed automated tools for Codestreet Server setup and fault-tolerant configurations

STAF/STAX for Automated Test Execution:

Designed and implemented a vendor-agnostic test suite for performance and functional testing with minimal overhead

Widely adopted for bulk-performance tests, data collection, and graph generation within the messaging team

Goldman Sachs (Assistant Vice-President; Aug 2000 – Sep 2003)

Enterprise Messaging Projects:

Consultancy: Provided in-house consultancy during design and development of applications using IBM Websphere MQSeries (Certified Developer)

Automated Tools: Developed tools for monitoring, configuration, and operations

Best Practices: Established client-to-server programming guidelines and server-to-server communication standards

External Communication: Designed and implemented infrastructure for communication with DTCC

Certification Tools: Created test suites to certify MQSeries releases for performance and scalability

Vendor Management: Managed IBM vendor relations for enterprise messaging needs and technical issue resolution

GHOST3 Replacement:

Messaging Evaluation: Assessed JMS v1.1 products (TIBCO EMS, SonicMQ, IBM MQSeries) based on performance, functionality, recovery, and manageability

Migration: Led migration efforts to replace GHOST3 with EMS, based on gap analysis and proof of concept

Other Experience (Jul 1996 – Jun 2000)

3D Vis Technologies (MSc. Comp. Sc): Developed a GUI to demonstrate efficient data rendering algorithms

DiTools India Pvt Ltd: Created a Time Zone component for managing project activities in distributed project management

NewGen Software Technology: Designed workflow APIs for distributed document database applications to enable a paperless office

Academic Qualifications:

Degree

GPA

Institute

Graduation

Master’s in Business Administration (Finance and Investment Management)

3.92

PACE University, New York, NY

Fall 2009

Master’s in Computer Science

3.87

SUNY, Binghamton, NY

Summer 2000

Bachelor of Computer Engineering

3.58

P.I.C.T, University of Pune, India

May 1996

Technical Expertise:

Languages

C; C++; Java; Python; Perl; PL/SQL; SQL; Bash

AWS

S3; EKS; IAM

IDE/Tools

Microsoft Visual Source Code; Microsoft Visual Studio; Eclipse Indigo; GNU gcc/g++; Microsoft SQL Server Management Studio; DB Artisan; Toad; GITLab; GIT; Subversion; make; SQL*PLUS; NET-SNMP; Packets2Disk Datamole; Prometheus; Grafana; Coralogix; Kibana; PagerDuty; Teams

Development Methodology

Agile; Waterfall

Automation

Ansible; Terraform; Teams Power Automate; STAF/STAX; GNUPlot

Product SME

Confluent Kafka Broker Cluster; Confluent Kafka Connect; Pinot; Trino; 60E AMPS; INFA Ultra-Messaging; TIBCO EMS; IBM MQSeries; Codestreet Replay Service; Solace Messaging Appliance

J2EE Integration

Apache Webserver; Oracle Weblogic Server

RDBMS

MS SQL Server; Oracle 10g; kdb+

Technical Standards

JMS; AMQP; SSL; LDAP; RMI; JDBC; ODBC; JNI

MS Office 2010

Word; Excel; PowerPoint; Project; Visio

Operating System

Linux RH/SLES; Sun Solaris 8/10; MS-Win NT Server 2003/2008; MS-Win XP/Vista

Shared File Systems

Veritas VCS; Veritas CFS/CVM

Networks

GigE; InfiniBand; OpenOnload

Contact this candidate