Principal Architect (Databricks, Azure, PySpark)

As a Principal Architect, you will leverage your extensive experience to design and implement cutting-edge data solutions using Databricks and Azure technologies. You will collaborate with cross-functional teams to optimize data workflows and ensure seamless integration within the retail domain. Your expertise will drive innovation and efficiency, enhancing our company's impact on the retail industry.

Responsibilities

Lead the design and implementation of scalable data architectures using Databricks Unity Catalog and Delta Live Pipelines to enhance data management capabilities.

Oversee the integration of Azure Data Lake Store and Azure DevOps to streamline data storage and deployment processes.

Provide technical guidance in Python and PySpark to develop robust data processing solutions that meet business requirements.

Collaborate with cross-functional teams to ensure seamless data integration and workflow optimization using Databricks SQL and Workflows.

Drive innovation by implementing advanced data analytics solutions that support retail domain operations, including collaboration, planning, forecasting, and replenishment.

Ensure data solutions align with industry best practices and company standards to maintain data integrity and security.

Develop and maintain comprehensive documentation for data architectures and workflows to facilitate knowledge sharing and continuity.

Monitor and optimize data processing performance to ensure efficient resource utilization and timely delivery of insights.

Engage with stakeholders to understand business needs and translate them into technical requirements for data solutions.

Contribute to the continuous improvement of data processes by identifying opportunities for automation and efficiency gains.

Support the development of data governance frameworks to ensure compliance with regulatory requirements and company policies.

Foster a collaborative environment by working closely with team members to share knowledge and best practices.

Stay updated with the latest advancements in data technologies and methodologies to drive continuous innovation.

Certifications Required

Databricks Certified Data Engineer Associate, Microsoft Certified