Kafka Architect -Remote

Title: Kafka Architect -Remote

Location: Remote

Length: Long term

Restriction: W2 or C2C

Description:

Webcam interview; *** Long term project *** Remote ***

Key Responsibilties

Architect and implement real-time event streaming platforms for use cases such as real-time inventory tracking, fraud detection, personalized promotions, and dynamic pricing.

Evaluate and select Kafka deployment models (self-managed, Confluent, AWS MSK, Azure Event Hubs, etc.) based on business and scalability needs.

Design high-availability Kafka clusters to handle massive transaction volumes in eCommerce, POS systems, and digital customer engagement.

Define best practices for topic management, schema evolution (Schema Registry), and partitioning strategies for large-scale retail data flows.

Enable real-time customer data streaming from multiple sources (POS, eCommerce platforms, loyalty programs, IoT devices).

Build Kafka-powered data pipelines to integrate with ERP, CRM, WMS, OMS, and personalization engines.

Develop stream processing solutions using Kafka Streams and ksqlDB for dynamic promotions, order fulfillment, and fraud detection.

Ensure seamless integration with retail analytics platforms like Snowflake, Databricks, and Google BigQuery.

Optimize Kafka infrastructure to support seasonal peaks, high traffic surges (e.g., Black Friday, Cyber Monday), and multi-channel order management.

Implement disaster recovery, failover strategies, and high-availability configurations for mission-critical retail systems.

Define security policies for data protection, compliance (PCI-DSS), and customer privacy regulations (GDPR, CCPA).

Establish monitoring, logging, and alerting using tools like Prometheus, Grafana, Splunk, and Open Telemetry.

Act as the Kafka SME for retail transformation projects, educating engineering and business teams on event-driven capabilities.

Collaborate with eCommerce, supply chain, marketing, and data science teams to drive real-time decision-making.

Partner with cloud architects, DevOps, and cybersecurity teams to ensure a secure, scalable Kafka ecosystem.

Required Skills

14+ years of experience in designing and implementing event-driven architectures, preferably in retail, eCommerce, or omnichannel environments.

Deep expertise in Apache Kafka (Kafka Streams, Kafka Connect, ksqlDB, Schema Registry).

Strong knowledge of real-time retail analytics, demand forecasting, and digital customer engagement.

Experience integrating Kafka with retail POS systems, inventory management, and order fulfillment platforms.

Hands-on experience with microservices and event-driven patterns (CQRS, SAGA, Event Sourcing).

Experience working with cloud-based Kafka implementations (AWS MSK, Azure Event Hubs, GCP Pub/Sub).

Proficiency in real-time inventory tracking, recommendation engines, and fraud detection systems.

Experience with containerization (Docker, Kubernetes) and CI/CD pipelines for Kafka deployments.

Strong background in monitoring tools and performance tuning for Kafka clusters