Data Engineer

Job Title: Senior Data Engineer -Vertica+Copilot

Experience: 5–8 Years

Location: Bangalore (Hybrid)

Notice Period: Immediate to 20 Days (0–30 days preferred)

Role Overview

We are looking for a highly skilled Senior Data Engineer / AI Developer Copilot with strong expertise in Vertica and distributed data engineering technologies. The ideal candidate will be responsible for designing, developing, and optimizing high-volume data pipelines while supporting AI-driven data platforms. This role requires hands-on experience with large-scale data processing, performance tuning, and distributed systems.

Key Responsibilities

Design, develop, and optimize high-volume, high-performance data pipelines.

Work extensively on Vertica database design, SQL optimization, and performance tuning.

Develop and maintain efficient ETL workflows for structured and unstructured datasets.

Implement scalable data solutions using distributed programming concepts.

Write production-grade code using Python and Shell scripting.

Integrate and manage NoSQL and caching systems such as Redis and Cassandra.

Perform database performance analysis and query optimization.

Collaborate with cross-functional teams including Data Scientists, AI Engineers, and stakeholders.

Ensure data quality, integrity, and reliability across platforms.

Participate in architecture discussions and recommend best practices.

Mandatory Skills

Strong expertise in Vertica (database architecture, performance tuning, query optimization).

Hands-on experience in Python programming for data engineering.

Advanced SQL skills.

Experience in Shell scripting.

Good understanding of ETL processes and tools.

Experience with Redis (caching mechanisms).

Experience with Cassandra (large-scale distributed storage).

Knowledge of Scala is preferred.

Strong understanding of distributed systems and big data architecture.

Preferred Qualifications

Experience working in enterprise-level data environments.

Exposure to AI/ML data pipelines and Copilot-based solutions.

Familiarity with CI/CD pipelines and DevOps practices.

Ability to troubleshoot performance bottlenecks in large data systems.

Strong analytical and problem-solving skills.

Soft Skills

Excellent written and verbal communication skills.

Strong analytical and logical thinking abilities.

Ability to work independently in a hybrid model.

Good stakeholder management skills.

Experience Level

5–8 years of relevant experience in Data Engineering / Data Platforms.

Proven experience in handling large datasets and distributed environments.