Data Engineer Senior

Summary

Experience

Education

Sarath K

Phone: +1-678-***-**** Email: ***********@*****.*** LinkedIn LeetCode Senior Data Engineer with 7+ years of experience delivering batch and streaming data platforms on Databricks (Delta/Unity) and Snowflake across AWS/Azure. Skilled in SQL, Python, PySpark, dbt, Airflow, and Kafka, with emphasis on medallion lakehouse design, CDC, and production-grade pipelines. Implemented data contracts, Great Expectations, and OpenLineage to strengthen quality and observability, improved SLAs and reduced MTTR, and enforced RBAC/PII controls to satisfy SOX/GDPR. Known for partnering with product and analytics to convert requirements into governed, reliable, and cost-efficient datasets.

Languages: Python, SQL, PySpark (Scala/Java exposure), Bash

Platforms/Warehouses: Databricks (Delta, Unity Catalog), Snowflake, BigQuery

Orchestration & Transform: Airflow, dbt (incremental, snapshots, exposures, tests); CI/CD (GitHub Actions, Jenkins)

Streaming & Ingestion: Kafka (producers/consumers), Lambda, S3/ADLS auto-ingest; (NiFi, Glue connectors)

Modeling: Kimball, Data Vault 2.0, SCD-Type 2, Medallion (Bronze/Silver/Gold)

Data Quality, Lineage & Governance: Great Expectations, dbt tests, Purview/Atlas, Unity Catalog, RBAC, masking/tokenization

Cost/SRE & Monitoring: Datadog, CloudWatch/CloudTrail, ELK; right-sizing, job pruning, storage tiering, SLA/SLOs

Cloud: AWS (S3, EMR, Glue, Redshift, KMS), Azure (ADF, Synapse, ADLS, Entra/AD) GCP (Dataflow/BigQuery)

BI/Visualization: Power BI, Tableau, Apache Superset Hartford Financial Service Group May 2024 – Present Senior Data Engineer

• Engineered Spark Structured Streaming on Databricks for Kafka topics; sustained sub-minute p99 latency under peak load.

• Implemented data contracts + dbt exposures; eliminated schema-drift surprises and kept downstream SLAs consistently green.

• Rolled out Great Expectations with broad rule coverage across critical datasets; reduced alert noise and brought MTTR in line with on-call SLOs via runbooks.

• Right-sized clusters, pruned redundant jobs, and optimized storage tiers to materially reduce compute and storage spend without sacrificing throughput.

• Enforced RBAC/PII masking with Unity Catalog/Lake Formation; delivered audit-ready lineage for SOX/GDPR reviews.

• Built real-time risk scoring pipelines with feature joins in streaming; improved model discrimination (higher AUC and precision- recall vs. baseline).

• Authored onboarding and recovery playbooks (backfills, late/duplicate events, idempotency), improving operational consistency

• and handoffs.

• Drove root-cause analysis on data inconsistencies; used constraints, contracts, and idempotent sinks to prevent repeat defects. Stack: Databricks (Delta/Unity), Kafka, PySpark, dbt, Airflow, AWS (S3, KMS, Lambda), Power BI, Datadog Stack: Databricks

(Delta/Unity), Kafka, PySpark, dbt, Airflow, AWS (S3, KMS, Lambda), Power BI, Datadog Coforge Apr 2018 – Apr 2024

Data Engineer

• Migrated Oracle/MySQL to S3/Snowflake; automated ingestion with Glue/NiFi and standardized marts with dbt for reliable self- service analytics.

• Built Airflow/Dagster DAGs and dbt incremental/SCD-2 models; shortened refresh windows and reduced post-release defects across domains.

• Implemented Delta Lake (Auto Loader, Z-order/partitioning); cut query wait times for asset analytics and improved workload predictability.

• Established RBAC + encryption (KMS) and column-level lineage; standardized P0 incident runbooks across regions for consistent response.

• Delivered cross-region CI/CD with Terraform + GitHub Actions + Databricks Jobs; reduced manual deployments and drift.

• Produced Looker/Power BI exec dashboards (utilization, license, incident trends) enabling decision-ready insights for leadership.

• Introduced Slack/Teams alerting with smart routing; lowered alert fatigue and sped up incident acknowledgment and recovery.

• Prototyped time-series anomaly models (Python/XGBoost) for hardware failure; informed preventive maintenance strategies. Stack: AWS (S3, Glue, EMR, KMS), Azure (ADF, Synapse, ADLS, Entra), Databricks (Delta), dbt, Airflow, Terraform, Looker/Power BI Bachelor of Engineering – Electrical and Electronics Engineering JNTUH University, India

Core Competencies

Contact this candidate