Data Engineer Real-Time

PALLAVI PANT 704-***-**** *******.******@*****.*** LinkedIn

SUMMARY

Experienced professional with 7+ years of industry experience in designing, developing and building scalable data pipelines using Python, R, SQL and cloud technologies. My career goal is to empower companies to make informed decisions using data solutions. EDUCATION

MS in Information Systems Arizona State University Arizona, USA July 2019 PROFESSIONAL EXPERIENCE

Data Engineer, Charter Communication (US, Remote) Mar 2021 – Present

• Developed and automated 30+ data pipelines, ingesting data across 9 disparate sources: APIs, SFTP, DB using Python, Airflow.

• Migrated on-premises database tables into Amazon Redshift using AWS Glue transferring 30% of the data for efficient reporting.

• Implemented a scalable real-time data pipeline with Apache Spark, Apache Kafka, and AWS EMR, delivering data-driven insights for 75+ system tools across 13 portfolios and improving operating efficiency by 24%.

• Designed and implemented a cloud-based data warehousing and reporting infrastructure using Tableau, AWS Glue, and Redshift, enabling real-time insights and analytics on operational data.

• Deployed predictive forecasting models using regression techniques on customer data and generated automated reports increasing efficiency by 15%.

• Developed 50+ dashboards in Tableau server depicting business KPIs and real-time visibility into system availability, MTTR.

• Automated ETL processes across 100K+ rows of data reducing manual efforts by 75% replacing Excel and Smartsheet.

• Reengineered 3 automation workflows using Apache Airflow and UiPath to improve process efficiency by 18%.

• Used Python, SQL and Tableau for building statistical and mathematical models for reporting, analytics and data visualization.

• Collaborated with cross functional teams for business requirements and developed KPIs/metrics for monitoring tasks.

• Developed 32+ RESTful APIs for transferring data to and from between db and internal website using Postman for testing.

• Utilized AWS IAM to secure workforce access for internal websites, set permissions and policies-based access control.

• Designed workflow diagrams, process diagrams in Visio, monitored task completion in Jira, used SIPOC for business process.

• Supervised 4+ team members for developing dashboards and building ETL pipelines for 75+ system tools across 13 portfolios. Data Analyst, Portfolio Management at Nuvei (Scottsdale, AZ) Nov 2019 – Nov 2020

• Developed an engine using Python to analyze 10000+ records of monthly transaction volume to generate revenue metrics.

• Automated calculations and managed workflows utilizing KPIs like weighted average boosting process efficiency by 60%.

• Created SQL scripts, indexes and complex queries for analysis and optimized the query performance ensuring data quality.

• Built a forecasting data model to estimate the revenue for upcoming month with 95% accuracy and identified gaps by analyzing the root cause such as payment disputes, missing transactions using statistical analysis.

• Involved in process automation using Python, process improvement, data QA, documentation, and recommending solutions.

• Created visualizations in Power BI by researching trends in the revenue stream and to influence the overall strategy of 6 pricing structures and help customers with transaction insights.

• Identified use case for 15+ Dashboards in Power BI for deep dive credit portfolio mix reports by extracting 10K + rows of data. Data Engineer, Internal Software Development at HDFC BANK LTD (Mumbai, India) Feb 2017 – Jul 2018

• Extracted data from external APIs using RSA, AES encryption and purged data in compliance with data protection regulations.

• Streamlined ETL pipelines to load and retrieve data between mobile and database and external APIs utilizing Java, SQL.

• Revamped 30+ RESTful APIs for querying data for product management and creating push notifications and alerts.

• Involved in requirement gathering, cost-benefit analysis and production release with stakeholders for developing mobile app. Software Developer, Retail Banking and Wealth Management at HSBC (Pune, India) Aug 2015 – Feb 2017

• Managed, translated, and assisted project stakeholders during requirement analysis and implementation.

• Collaborated with development team in an Agile SCRUM to refine and support software solutions reducing delivery times.

• Used GitHub, Jenkins and Maven for setting up CI/CD pipeline and version control and used Jira for issue/project tracking. SKILLS & ACCOMPLISHMENTS

ETL processes, Data warehousing, Big data technologies, SQL, NoSQL databases, Data modeling, Data pipeline development, Data quality assurance, Financial data analysis, Business intelligence tools, Data visualization, R, RESTful APIs, Redshift, Glue, EC2, S3 buckets, Athena, EKS clusters, RDS, DynamoDB, IAM, Databricks, Snowflake, Prefect, Datadog, UiPath Accomplishments: Nominated and achieved Quarterly and Annual Award for excellence in work and data delivery. Certifications: Google Data Analytics Splunk Fundamentals Data Scientist Toolbox – John Hopkins University

Contact this candidate