Senior Data Engineer & BI Specialist

TIYAR ACHILOV

No Sponsorship Required Brooklyn, NY 929-***-**** ************@*****.***

Experienced Senior Data Engineer with over 8 years of expertise in Data Modeling, Data Engineering, and Business Intelligence Development. Specialized in various data domains including sales, service, marketing, management, production, finance, accounting, human resources, and supply chain. Proficient in building high-performance Data Integration solutions, facilitating Cloud Migration, and designing ER and Dimensional Data Models to support OLAP/OLTP/Data Warehouse and Data Lake systems. Skilled in reporting and presentation using BI applications like SQL Server, Oracle, AWS Redshift, GCP Big Query, AWS RDS MySQL, AWS Glue, MS Fabric, Azure Data Factory, Databricks, Azure Synapse, SSIS, Azure Logic App, Function App, Azure Data Lake, and CI/CD pipelines. Well-versed in data visualization tools such as Power BI, SSRS, Tableau. TECHNICAL SKILLS

Scripting and Data Tools

● Programming Languages: Python (Pandas, NumPy, Matplotlib, Requests), Scala, PowerShell, C#, TSQL, PLSQL, MySQL, Java, PySpark

● Data Tools: SSIS, SSRS, SSAS, Talend, Alteryx, Domo, Jupyter Notebook

● Machine Learning & Big Data: Big Data, ML, Jupyter Notebook Databases

● Relational Databases: SQL Server, Oracle PL/SQL, IBM DB2, MySQL

● Cloud Databases: Snowflake, AWS Redshift, BigQuery, Cosmos DB, MongoDB

● Query Languages: Kusto SQL, AWS Athena

Cloud Data Engineering

● Azure: Azure Data Factory (ADF), Azure SQL PaaS, Azure Synapse Analytics, Azure Logic Apps, Azure Function Apps, Azure Databricks, Azure Data Lake, Blob Storage & ADLS

● AWS: AWS Glue, AWS Redshift, AWS S3, AWS RDS Oracle, AWS EC2, AWS Athena, AWS DMS, Step Functions

● GCP: BigQuery, GCP Analytics

● Microsoft Fabric: Spark Notebooks, Fabric Pipelines, Fabric Lakehouse Data Modeling and Architecture

● Dimensional Modeling, Data Warehouses, Data Marts, Data Lakes Data Visualization

● Power BI, Tableau, SSRS, Domo, Power Apps

Infrastructure and Methodologies

● Azure DevOps, GitHub, Terraform, CI/CD Pipelines

● Agile Methodologies: Scrum, Sprints

● Tools: TFS, Visual Studio (VS)

Real-Time Data and Messaging

● Kafka, Azure Event Hubs

WORK EXPERIENCE

Senior Data Engineer Clara Analytics, NY, NY 2025 APR – Present

● Led the design and development of scalable data pipelines on AWS using Amazon Web Services services including Glue, EMR, Lambda, and Athena to process large-scale insurance and claims data.

● Architected end-to-end data workflows leveraging S3, EMR Spark, and Glue to standardize and transform raw data into curated datasets for downstream analytics and machine learning pipelines.

● Built and optimized PySpark-based ETL pipelines on EMR, improving processing efficiency and reducing job runtime through partitioning, caching, and broadcast join strategies.

● Designed and maintained data models for analytical and operational use cases, ensuring high data quality and consistency across multiple tenants and business domains.

● Led data onboarding efforts for new clients/tenants, enabling seamless integration into existing data platforms with minimal downtime.

● Collaborated cross-functionally with Data Science and Product teams to deliver reliable datasets for triage and litigation models, directly supporting business-critical decision-making.

● Implemented data validation and reconciliation frameworks using SQL and PySpark, significantly improving data accuracy and reducing production issues.

● Owned production support for critical data pipelines, proactively monitoring failures, troubleshooting issues, and ensuring high system reliability and SLA adherence.

● Designed and executed data deduplication and optimization strategies for large datasets (millions of records), improving query performance and downstream application usability.

● Worked extensively with Amazon Athena and Amazon Redshift for analytics and reporting, optimizing queries and storage layouts for performance.

● Mentored junior engineers and conducted code reviews, promoting best practices in data engineering, SQL optimization, and scalable architecture design.

● Partnered with DevOps and infrastructure teams to manage deployments, permissions, and access controls across AWS environments, ensuring secure and compliant data operations.

● Automated workflows and data processing tasks using Python and AWS services, reducing manual intervention and increasing operational efficiency.

● Contributed to architectural decisions for data lake and warehouse design, aligning solutions with long-term scalability and business goals.

DATA ENGINEER CAPGEMINI, NY, NY 2022 SEP - 2025 APR

• Collaborated with business stakeholders to gather requirements and design logical and physical data models for optimized reporting solutions.

• Created and maintained CI/CD pipelines using Azure DevOps and Terraform for deploying data engineering solutions seamlessly. Reviewed pull requests and performed code reviews to ensure adherence to coding standards and best practices across the team.

• Developed data workflows using Scala and PySpark for data validation, transformation, and cleansing.

• Developed and orchestrated data pipelines in Azure Data Factory (ADF) to load and transform data into Azure Synapse for reporting purposes. Built scalable data lakes on ADLS using Databricks with Medallion Architecture to support complex analytics and reporting. 2

• Designed and implemented Microsoft Fabric Pipelines to populate Fabric Lakehouse with data from various source systems. Utilized Fabric Spark Notebooks to clean, partition, and process large datasets efficiently within Microsoft Fabric.

• Created ETL pipelines using AWS Glue and Lambda Functions to extract, transform, and load data from S3 into Redshift. Configured AWS Glue Crawlers and Athena queries to catalog and query semi-structured data for on-demand reporting.

• Designed AWS S3 data lake architectures using Glue, PySpark, and Scala.

• Designed optimized Snowflake data warehouse schemas with partitioning and clustering for improved query performance. Managed Snowflake user access implemented row-level security, and automated administrative tasks for secure and efficient operations.

• Developed dbt models and modules with SQL scripts to transform and standardize raw data into analytical datasets.

• Used Unity Catalog in Databricks to govern and manage data assets while ensuring robust security and compliance.

• Built end-to-end data solutions for Power BI reports using Alteryx for data transformations and ETL pipelines.

• Provided production support for ADF pipelines, Databricks Notebooks, and Power BI dashboards by resolving Azure DevOps incident tickets.

• Tuned SQL Server stored procedures, indexes, and partitions to enhance database performance and handle high transaction loads.

• Modeled data architecture for data lakes and data warehouses using Medallion Architecture to meet business analytics requirements.

• Designed data workflows for processing real-time streaming data with Databricks and PySpark in a Medallion Architecture framework.

• Developed pipelines in Azure Data Factory, utilizing linked services to connect with various source systems, and employed Databricks notebooks for data transformation before loading it into Azure Blob Storage.

• Created Scala and PYSPARK User-Defined Functions (UDFs) to meet specific business requirements and employed JSON scripts for deploying pipelines in Azure Data Factory (ADF) that involved SQL activities for data processing.

• Implemented optimization techniques such as broadcast variables and broadcast joins and incorporated incremental loads to enhance job processing efficiency.

• Developed PYSPARK and SQL scripts in Databricks for data validation, utilizing PYSPARK and Spark-SQL to extract, transform, and aggregate data from multiple file formats, uncovering customer usage patterns.

• Designed both Logical and Physical data models for databases and data warehouse schemas using Erwin.

• Created ODS structures and data mart structures, applying business rules in the modeling of Data Marts and conducting data profiling for new data structures. Developed a long-term data warehouse roadmap, along with architectures, designs, and the construction of the data warehouse framework as per the roadmap.

• Generated SQL databases and schema objects, including tables, indexes, constraints, stored procedures, and functions.

• Designed and implemented multiple Azure Data Lake Storage solutions, incorporating unified batch and streaming datasets from various on-premises and PaaS/SaaS sources. Utilized multiple Azure cloud services 3

such as Synapse Analytics, Data Factory, Azure Function Apps, Logic Apps, Azure Kusto, Salesforce, JDE, Azure SQL Database, and MySQL.

• Built ETL pipelines in Azure Data Factory to facilitate data copying to and from Blob storage, utilizing account keys, shared access signatures, service principals, and managed identities for authentication.

• Played a role in planning and developing roadmaps and deliverables to facilitate the migration of existing SSIS on-premises systems/applications to the Azure cloud.

• Designed and implemented data storage solutions using Amazon S3 for various data types, including structured and unstructured data.

• Implemented ETL and data movement solutions using Azure Data Factory (ADF), while also designing database solutions in Azure Synapse and Azure SQL. Created SSIS packages within ADF using Azure-SSIS IR.

• Managed and administered Oracle database instances on AWS RDS for optimal performance and availability. Implemented backup and recovery strategies, automated backups, and database snapshots. Tuned SQL queries and database parameters to improve query performance and resource utilization.

• Designed and developed ETL workflows using AWS Glue to extract, transform, and load data from diverse sources. Automated AWS Glue data ingestion and transformation processes to improve data quality and timeliness.

• Developed multiple reports in Power BI, utilizing Story Points for presentations. Designed dashboards, scorecards, views, pivot tables, and charts for in-depth data analysis using Power BI Desktop. Constructed various complex Power BI reports and dashboards.

BIG DATA ANALYTICS ENGINEER SPOTIFY, NY, NY 2020 JULY - 2022 SEP Roles & Responsibilities:

• Collaborated with clients to gather business requirements and designed conceptual and physical data models using Azure Synapse and Snowflake to build scalable data warehouses.

• Designed and implemented scalable data pipelines using Scala and PySpark for large datasets. Leveraged Scala's functional programming features to write efficient, maintainable, and parallelized code for data processing.

• Built and maintained dynamic ETL pipelines in ADF, leveraging Copy Data, Lookup, and Get Metadata activities for seamless integration of batch and real-time data.

• Designed and implemented Azure Logic Apps integrated with Python-based Function Apps to process API data and automate data workflows.

• Worked extensively with Databricks to build batch and near real-time pipelines using PySpark, integrating Azure Event Hubs for streaming data.

• Used Python libraries like Pandas and NumPy for data validation and cleansing, and Requests for API integration in Databricks notebooks.

• Created Power BI dashboards with real-time KPIs, using data from Kafka streams, showcasing near real-time latency for logistics operations.

• Developed Python scripts for error handling and email notifications for production support using AWS Lambda functions.

4

• Designed GCP BigQuery tables and Looker dashboards for sales reporting, utilizing SQL for data transformation and visualization.

• Developed AWS Glue ETL pipelines to load data from S3 buckets into Redshift, implementing validation logic with Python scripts.

• Collaborated with offshore and onshore teams to conduct code reviews, ensuring adherence to best practices and efficient designs.

• Built SSIS packages with custom C# script tasks for API data integration, handling complex business logic.

• Configured and tuned Snowflake environments with user access controls, row-level security, and optimized queries for faster performance.

• Created advanced Power BI DAX measures using CALCULATE, SUMX, and FILTER functions for complex financial and operational reporting.

• Implemented Unity Catalog in Databricks for centralized data governance and security across data lakes.

• Worked with SSAS to develop OLAP cubes, optimizing queries for use in Tableau and Power BI reports.

• Used heavy T-SQL for stored procedure development, partitioning, and performance tuning to enhance data warehouse query performance.

• Automated the loading of data into Snowflake using Snow-SQL and implemented Snow-pipe for continuous data ingestion.

• Managed production incident tickets via Azure DevOps, troubleshooting failures in ADF pipelines and Databricks notebooks.

• Designed Tableau dashboards with calculated fields and parameters, incorporating visuals like heat maps and trend analysis.

• Built data validation scripts in Python with Matplotlib for visualizing data anomalies and ensuring data integrity.

• Configured Cosmos DB for transactional data storage and ran KQL queries in Azure Log Analytics to monitor system performance.

• Built and maintained SSRS reports for on-prem SQL Server data, incorporating VB scripts for custom calculations.

• Implemented Azure Synapse Spark notebooks to process large datasets and build data lakes using Medallion Architecture.

• Integrated Power BI dashboards with Azure SQL databases, applying row-level security for role-based data access.

• Designed data models in Big Query, integrating Tableau for visualizing marketing performance metrics.

• Created AWS Glue crawlers to catalog data in S3 buckets, enabling efficient querying with Athena SQL.

• Worked on SSIS packages for complex ETL processes, integrating legacy systems with Azure SQL databases.

• Leveraged Terraform for infrastructure automation and Azure DevOps for CI/CD pipelines across multiple environments.

• Created Dax Studio reports for Power BI performance monitoring, applying best practices for query optimization.

• Automated repetitive tasks using PowerShell scripts, such as database backups and log management for SQL Server.

5

ADF, SSIS & POWER BI ENGINEER SILVERSTEIN PROPERTIES 2017 JUNE - 2020 JULY Roles & Responsibilities:

• Worked with business users and stakeholders to analyze business and functional requirements, converting them into technical use cases.

• Designed databases based on business requirements and provided comprehensive documentation. Maintained business intelligence models for developing standard and ad-hoc reports using tools such as SSMS, SSIS, SSRS, and Microsoft Excel.

• Executed the complete lifecycle of a Data Warehouse, including requirements gathering, design, implementation, and maintenance.

• Played a role in designing and optimizing various SQL database objects such as tables, views, stored procedures, user-defined functions, indexes, and triggers.

• Analyzed source data to determine suitable data models and ETL solutions that align with business requirements.

• Developed SSIS packages utilizing a range of tasks and transformations, including Execute SQL Task, Data Flow Task, Sequence Container, For Each Loop Container, Lookup, Aggregate, Send Mail Task, Derived Column, Conditional Split, and more.

• Utilized SSIS transformations such as Lookup, Derived Column, Data Conversion, Aggregate, Conditional Split, SQL Task, Script Task, and Send Mail Task.

• Performed Data Profiling to assess data integrity during the transformation process from the source.

• Wrote SQL queries incorporating various techniques such as Joins, Cross Apply, Aggregate Queries, Window Functions, and Merge. Created and updated clustered and non-clustered indexes to optimize SQL Server performance.

• Created, maintained, and updated ETL packages for loading both OLTP and OLAP database tables.

• Engaged in SSIS development and support, creating ETL solutions for integrating data from diverse sources such as Flat Files, Excel, OLEDB, and CSV. Conducted data filtering, cleansing, and reviewed reports, and performance indicators to identify and address code issues.

• Configured various Azure cloud services, including Azure Storage, Azure Active Directory, Azure Service Bus, Azure VPN Point to Site, Virtual Networks, Azure Functions, Azure Custom Security, and Endpoint Security. Managed Azure AD tenants and established application integration with Azure AD.

• Created multiple subscriptions in Azure and collaborated with the Network team to establish connections from online to multiple V-Nets using Azure V-Net Peering.

• Developed an Azure DevOps pipeline to attach Azure policies for management groups and subscriptions.

• Created cloud modules for seamless interaction with Azure services, enabling easy infrastructure creation and orchestration on Azure. Automated cloud-native applications on Azure using microservices like Azure Functions and Kubernetes.

• Worked extensively with Azure DevOps services for Continuous Integration, Continuous Delivery, Deployment, and monitoring activities, addressing issues related to build and deployment across multiple releases.

• Implemented ETL and data movement solutions using Azure Data Factory, combining technologies such as T-SQL, Spark SQL, and U-SQL in Azure Data Lake Analytics. Facilitated data ingestion into Azure services 6

such as Azure Data Lake, Azure Storage, and Azure Data Warehouse, with data processing in Azure Databricks.

• Contributed to the migration of SQL Server databases to SQL Azure Database, utilizing tools like SQL Azure Migration Wizard and Python API to upload agent logs into Azure Blob Storage.

• Configured Azure Monitor services, including Log Analytics dashboards, to evaluate log performance metrics through log querying and storage for future use.

• Implemented ETL and data movement solutions for Data Migration using Azure Data Factory. Built multiple Data Lakes and replicated existing application logic and functionality in the Azure Data Lake, Data Factory, and SQL data warehouse environment.

• Participated in the design and optimization of various T-SQL database objects, such as tables, views, stored procedures, user-defined functions, indexes, and triggers.

• Engaged in Data Analysis, Design, Development, Implementation, and Testing of Data Warehousing, encompassing activities such as Data Conversions, Data Extraction and Transformation.

• Developed complex SSIS packages for ETL processes using SQL Server Integration Services, enabling data transfer and review from heterogeneous sources such as Flat files, Excel, CSV, Oracle, and MS Access.

• Monitored and optimized time-consuming queries through Performance Tuning techniques and created SSIS packages with error handling capabilities.

• Automated ETL operations for extracting data from multiple sources, transforming inconsistent and missing data into consistent and reliable data, and loading it into multi-dimensional data warehouses.

• Designed dynamic visuals in Power BI, incorporating features like Bookmarks, Slicers, and custom tooltips. Published and shared Power BI reports and data models within Power BI workspaces.

• Developed detailed summary reports, including line and pie charts, trend analysis reports, and sub-reports, based on business requirements using SQL Server Reporting Services (SSRS).

• Designed, developed, debugged, and tested reports in SQL Server Reporting Services (SSRS). Created SSRS reports with features such as Drill Down, Drill Through, Linked, Ad-hoc, Snapshot, Parameterized, Referential, and Graphical reports.

7

Contact this candidate