Data Analytics Lake

`

Diwakar Teja Ravilla (Seattle, WA, USA)Mobile:+1-425-***-****

Email:**.****@*****.*** Linkedin

Summary:

● Staff Data Analytics and AI Engineer with 15 plus years of experience in design, development and implementation ofDatawarehouse,Data Lake,DeltaLake,Cloud Data Migration,Real time data/batch processing,ClickStream ProcessingandA/B Testing, LLM GEN AI.

● Designed and developed data platform/framework toIngestdata from multiple sources and types (structured, unstructured, logs etc..,) to Centralized DWH/Data Lake/Lake house in both Batch/streaming real time to perform Analytics and Analysis for CompanyLeadership/ Business, ERP, Marketing, Sales, Finance,ML users etc..,which helps the company to have deep insights to real time and improve decision making, analysis, and productivity by 200%.

● Expertise in various Database design practices likeDimensionmodeling,ERdesigns, Data Marts etc., build complex data models for Facts, Dimensions. Architect, design and developed End-to-End Data pipelines and implemented best practices.

● Expert in building Data visualization in PowerBI/Tableau. Built complex KPIs using DAX queries.

● Expert in Requirement Gathering to Design/Architecture of Data Model/DWH, building End-to-End Data pipelines/frameworks and delivering best optimized data solutions to End Users and meeting SLAs.

● Expert in decision making starting from choosing right software tools, mentoring/leading team members and delivering best quality software products within the budget to all stake

-holders.

● Establish best practice like standards and guidelines for design & development, deployment, support and analytics and mining.

Skills

● Proficient in Python, Scala,T-Sql, Postgresql

● AWS Redshift, Snowflake,SQL Server MS, Oracle,

Aurora db (postgres/mysql), Teradata, MongoDb,

Cassandra

● Hive, EMR, Tez, Presto, Zeppelin

● Elastic Map Reduce, Spark, and Cosmos

(Microsoft Map reduce)

● Tableau, Power BI, Cubes (SSAS), Grafana

● Apache Airflow, Oziee

● File formats – avro, parquet, orc, delta

● Kafka streaming

● Cloud Technologies -- AWS, Azure,FCP

● CI and CD pipelines, Jenkins, VSO, DevOps

Experience

Staff Engineer - Data And Analyticsat T-Mobile( contract) (July 2024 to till date)

● Denormalized the legacy Key Value architecture, saved storage space ( multi peta-byte scale) by applying filters on data and improved the performance of read operations for end users by 500%.

● Built Data Models for Snowflake, defined virtual warehouse sizing for different types of workloads. Developed, stored procedures/views in Snowflake

● Reduced operational costs of $500K per year by deprecating legacy Redshift Cluster, by merging the new storage to Centralized DWH, which gives us single source of truth for our data.

`

Staff Engineer - Data And Analyticsat Microsoft (contract)(Jan 2023 to June 2024)

● The entire source data systems for CELA (Microsoft Legal Team) data warehouse has been migrated from Oracle on-premise to Azure Sql. 100% of existing data pipelines, warehouses are affected and requires entire data remodeling and re-engineering.

● Re-engineered the existing DWH, data model, ETL Pipelines, Cubes and then created source to target mapping(STM) sheet for all tables, KPIs etc..,

● Design and develop interactive reports and dashboards in Power BI which provide deep insights and analytics of data Implement Alert systems, data governance, and security best practices.

Staff Engineer - Data And Analyticsat Stockx/Rocket Mortgage(Sep 2021 to Dec 2022)

● Design and Implementation of end-to-end data flow for the following complex pipelines like Bids/Asks, Click Stream,A/B Testing, OrderManagement, Product catalog into a centralized data lake/DWH to provide best insights of data for analytics, analysis to Company leadership, finance team, marketing, Business users and other stakeholders and provides data-driven decisions which helps to improve productivity by 500%, efficiency by 300% and 100% reduce waste.

● Consulting on Snowflake Data platform solution Architecture, designed end-to-end data pipelines.

● Establish best practice like standards and guidelines for design & development, deployment, support and analytics and mining. This improves performances by 100% and meeting SLAs by 100%.

Solutions Architect at Sogeti(Oct 2019 to Sept 2021)

● Engineering efforts including daily monitoring batch process/alert notification over failover, Data Governance, Usage, Data Quality etc.

● Worked with Spark for improving performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Spark MLlib, Data Frame, Pair RDD's, Spark YARN.

● 70% Technical hands on and 30 % People Management/Performance Management Senior Data Engineer at Coupang(Oct 2016 to Oct 2019)

● Build top priority tables for Finance, Catalog, Fullfilment Center and retailwhich helps to improve productivity by 500%, efficiency by 300% and 100% reduce waste..

● Designed and developed multipeta byte dimension table on catalog which is used across entire company.

● Migrated entire data marts from On-premise db to Cloud MPP db (redshift)

● Designed and build usage data pipeline on redshift to know, exact usage on tables and columns by the end-users

● Designed and developed automation testing frameworks to validate daily pipeline

● Developed Early SLA Alert Notifications which helps to understand any events causing delay of daily batch process, and thus alerting the On-call Engineers to take necessary action Senior Software Engineer at Microsoft (as Vendor)(Apr 2012 — Oct 2016),Technical Consultant at Wipro Technologies Ltd (July 2009 toApr 2013) and Test Engineer at Infosys (July 2004 to July 2009)

Education:

● Bachelor of Technology (Electrical &Electronics Engineering)from Jawaharlal Nehru Technological University, Hyderabad, India (1998 – 2002)

Contact this candidate