Senior Data Architect & Engineering Leader

Carter L Shore

Beverly Hills Florida ***** - Apopka, Florida 32712

352-***-**** *******@*****.*** https://www.linkedin.com/in/carter-shore-b4296a28

Summary

35+ years of hands-on experience as Architect, Data Engineer, and Analyst delivering and supporting large-scale data platforms across healthcare, insurance, banking, finance, media & entertainment, telecom, utilities, manufacturing, transportation, and more.

Extensive background architecting, designing and implementing automated data ingestion and ETL/ELT pipelines with Python/PySpark, Scala, Java, VB.Net, C/C++/C#, Matillion/Datastage/Informatica/AbInitio/Talend/Pentaho/SSIS/ETI Extract for structured, semi-structured, and unstructured data, including JSON, XML, Parquet, ORC, CSV, COBOL FD, and other formats/encodings.

Strong proficiency in SQL/NoSQL with deep practical experience in Databricks Delta Lake, Spark/PySpark, Snowflake, Iceberg, Hadoop Hive/Impala/Pig/HBase, Redshift, MongoDB, Synapse, CosmosDB, SAP HANA, Oracle PL/SQL, SqlServer, DB2, Teradata, for building scalable, high-performance data solutions.

I used Java to create add-ons extending capabilities of ETL tools like Datastage and Informatica; to administer, debug, and troubleshoot Hadoop YARN and other components, creating custom partitioners to address data skew, achieving equally distributed workloads, reducing execution times.

Encryption experience for healthcare and financial clients, implementing Voltage Format Preserving Encryption to high volume PHI data, using Java to craft custom pipelines from on-premise Teradata DB, incorporating Voltage FPE encryption of Parquet files for migration to a target Azure Cloud Synapse DB, protecting and securing it in-flight and at-rest.

Skilled in data modeling, metadata-driven design, data quality, validation, and testing, with a focus on creating reliable and maintainable production pipelines.

Experienced in transforming legacy systems to modern cloud and big-data architectures hosted on AWS, Azure, and GCP cloud environments.

Support of Prod, Dev, Test platforms and applications, root cause analysis, debugging, tactical mitigation, documentation, and strategic proposals for corrective actions to prevent future occurrences.

Hands-on experience architecting and delivering medallion-style data models within Databricks and Snowflake environments to support both OLTP and OLAP workflows.

Strong interest and practical experience with systems-engineering concepts such as data relationships, ontology, structural modeling, DAG creation, topological sorting, and dependency-driven data processing.

Developed internal tools and processes to analyze data object relationships, enabling improved error recovery, dependency tracing, and workflow optimization using DAG visualizations (Graphviz).

Proven ability to lead, mentor, and collaborate with cross-functional onshore and offshore teams, driving successful delivery in fast-paced, agile environments.

Technical Skills

Platforms: Azure (certified), AWS, GCP, Hadoop ecosystem (Cloudera/MapR/Hortonworks certified), Databricks, Microsoft Fabric, Unix, Web, Windows, and PC-based environments.

Programming Languages: Python, Spark/PySpark (Pandas, Numpy, Matplotlib), Scala, Java, C/C++/C#, PowerShell, Ksh/Csh, Perl, Lisp, Basic, VB.Net, PL/SQL, and COBOL.

Databases & SQL: Extensive hands-on experience with Snowflake, Snowpipe, Databricks SQL, PostgreSQL, DynamoDB, Synapse, CosmosDB, Dataverse, Iceberg, SQL Server, T-SQL, Access, Oracle, Teradata, Impala, Hive, Pig, MySQL, Redshift, and MongoDB, including database administration, Stored Procedures, performance tuning, and security.

Tools & Services: Azure ADF, AWS S3, Athena, Glue, Step Functions, GCP BigQuery, Alteryx, Voltage FPE data encryption, Starburst, Kafka, Terraform (IaC), Apache Iceberg, and enterprise ETL/ELT tools including Matillion, Datastage, Informatica, Ab Initio, SSIS, and Talend. CI/CD, Git/GitHub for repository and version control, and ERWin for data modeling.

Design & Implementation: Experience building metadata-driven warehouse migrations, ingestion frameworks, ETL/ELT pipelines, OLTP and OLAP structures, and rule-based data quality, validation, cleansing, authentication, logging, error detection, and mitigation processes.

Data Engineering & Analytics: Strong background across multiple SQL dialects, Python-based analytics, DBT, data mining, AI & ML, data wrangling, semantic analysis, data mapping, and transforming complex encoding and file formats. Skilled in orchestrating workloads using Autosys, Control-M, Airflow, CRON, and custom DAG frameworks with automated recovery and dynamic rule-based execution.

Industry Expertise: Healthcare EMR and Billing records, HL7, Cerner, claims processing. Deep experience in Media & Entertainment, including OTT and linear TV platforms, and extensive work with Nielsen datasets. Multiple projects in healthcare, insurance, banking, finance, retail, telecom, utilities, manufacturing, and transportation.

Advanced Data Workloads: More than a decade of data modeling experience, support for big data environments exceeding 20PB, and hands-on work in machine learning, predictive analytics, and data mining.

ETL/ELT Pipeline Development: Built automated ingestion and transformation pipelines for high-volume semi-structured datasets such as Nielsen MDR/NLTV/AMRLD files, healthcare EMR and HL7 processing and transformation, telecom call detail records, financial transaction data, and large-scale retail sales feeds, delivering enriched datasets and analytical outputs on cloud platforms.

Professional Experience

Advent Health March 2026 - September 2026

Data Engineer Altamonte Springs, FL

Utilized my knowledge and experience with Healthcare system providers and payers, Retail Pharma, SQL, ETL, pipelines, file structures, orchestration, Snowflake infrastructure, Azure ADF, ADLS gen2, Functions, Snowflake Python/PySpark functions/stored procedures, tasks orchestration. graph theory, topological sorting, and dependency analysis.

Used and supervised Copilot and Snowflake Coco AI to assist exploring and refining approaches, preparing and validating Strawman candidate solutions, and fine tuning Prod deliverable artifacts.

•Worked with powerBI dashboard teams and sentiment analytics SME to architect, design, build, test, and deliver a Snowflake data model, tables, pipelines, and workflow orchestration to facilitate low latency rendering of Survey data and performance analytics.

•Worked as Azure and Snowflake engineer on a team to migrate a suite of Alteryx pipelines to ADF and Snowflake, preserving business rules, target datastore types/structures for consumption by Medallia, report formats, and performance metrics with minimal disruption, enabling Azure platform AI features and capabilities for orchestration and transformation.

•Teamed with Customer Listening SME to define and create Snowflake tables and code to capture Consumer response data for a Proof of Concept and Strawman model/structure to capture and analyze survey response data with baseline template, facilitating and exposing opportunities for AI analysis of Customer feedback from Press-Ganey and Medallia, to derive Sentiment and Insights.

•Created and presented a lunch’n’learn talk detailing how Open Source tools like Graphviz can be used in conjunction with AI to efficiently visualize, simplify, schedule, and orchestrate complex tasks, processes, relationships, and projects by manipulation of existing metadata and documentation.

Accenture LLP/Accenture Federal Nov 2018 - Mar 2025Data Architect/Analyst/Engineer St Petersburg, FL

Directed teams to architect, design, build, test, and deliver Cloud and Big Data ETL pipeline solutions for Commercial clients and the Federal Government across diverse industries including Healthcare, Retail, Telecom, Electric Utilities, Food & Beverage, and Retail Pharmacy. Utilized technologies such as Azure ADF, ADLS gen2, Functions, Snowflake, Python/PySpark, C/C++/C#, PowerShell, Databricks/Deltalake, AWS, DynamoDB, Redshift, Presto, Cloudwatch, Terraform, GitLab, Hadoop Hive/Impala, Teradata, SqlServer, Oracle, GCP BigTable, Cloud Storage to architect, build, test, integrate, and deliver robust data platforms that align with data management standards. Used AbInitio for ETL graphs, implementing business mapping transformation processing and transforming large data volumes with co<operating system, sourcing from heterogeneous source platforms and consolidating to unified target platforms. Used lineage and metadata capabilities tto troubleshoot, identify, and mitigate issues. Wrote UNIX/Linux shell scripts for data storage, file manipulation, environment and platform configuration, and data quality assurance testing and remediation.

•For Federal DoJ, led teams to create database design documentation, quality assurance practices. Architect, design, and implemented an automated ingestion platforms using Snowflake SQL stored procedures to parse Dataverse JSON metadata, generate target table DDL, ingestion DML, and automated test cases supporting validation and Quality Assurance.

Developed a Snowflake Python stored procedure as a menu-driven DBA utility to enable client management of objects, users, access, and permissions, addressing relational model optimization and security policy adherence.

•For Walgreens, architected and led a team to build and integrate migration from an on-prem platform using Teradata, Hadoop, C# to Azure clouud with Spark, PySpark, Azure ADLS, ADF, Voltage FPE Encryption to migrate PB scale datasets from on-prem to Azure platform, hosting Synapse, Databricks, and Snowflake engineered target databases. Created and integrated ETL pipelines with Snowflake, Databricks, and Azure Data Factory to populate respective target databases & tables. Created cloud-based data models (OLTP and OLAP) and established data warehousing practices to ensure quality database deliverables, while leveraging CDC-based data ingestion solutions and Python/PySpark analytics within a secure Microsoft Azure cloud architecture.

•For Progress Energy, wrote and tested AWS ETL pipelines to migrate Electric Utility equipment SCADA message processing from on-premise into AWS Redshift and DynamoDB targets, integrated with Service applications, enabling AI analysis for fine tuning and GenAI assisted identification of degrading equipment and proactive maintenance before it failed. DMS for managing service/repair/order incident reports and analysis. Wrote framework to generate Blue/Green validation test scripts using Starburst to compare result sets of same queries from on-premise and AWS.

•For Verizon, wrote, debugged, and tested ETL pipelines that populated target BigQuery with CallDetailRecords, network events, and client usage data. Collaborated with Data Science Team to fetch, prepare, and validate network and call traffic training datasets supporting models for AI and ML analysis of network traffic and switch utilization, enabling feature engineering and predictive switch network expansion forecasting. Used GenAI to help categorize client usage patterns and suggest targeted marketing.

•For Subway, migrated from on-prem Teradata, SqlServer, T-SQL, and Oracle applications platforms to an AWS Cloud platform with Redshift target, using the Matillion ETL tool to implement transformation pipelines. Worked with business analysts and DBA to define and refine logical and physical target data models and transformation rules. Architected and delivered a semi-automated process to parse source and target metadata and mapping rules, generating migration P-code and validation test cases, reducing developer and testing team time and resources by 50%.

•For CVS, migrated from Teradata, and various siloed applications (Alteryx, Salesforce, etc.) to Hadoop and Databricks. Architected and built a POC Databricks instance to demonstrate feasibility of migrating thousands of legacy Teradata BTEQ scripts, employing custom PySpark Stored Procedures to parse the BTEQ scripts, emit DDL to create the target Delta tables, emit and execute the Databricks ELT pipelines, replicating the BTEQ transformations, and create test/validation scripts comparing and validating the source BTEQ output to the generated Databricks output. Collaborated with multidisciplinary teams to influence technical strategies and establish best practices in design, coding, and testing of data management systems.

Kogentix Inc Mar 2016 - Nov 2018

Senior Software Architect Engineer - Big Data Schaumberg IL

Led teams to architect, design, build, test, and deliver Cloud and Hadoop solutions tailored to media & entertainment, consumer data analytics, insurance, healthcare, banking, financial, retail, and manufacturing sectors.

•Employed Big Data technologies such as Hadoop, Spark, Python/PySpark, Scala, Databricks/Deltalake, Teradata, Oracle, Hive/Impala, and Git to develop, document, and optimize ETL pipelines and applications that aligned with business requirements.

•For CVS, Architected and delivered AWS Cloud data engineering solutions, establishing and implementing cloud service patterns, designing OLTP and OLAP data models from legacy C# and application platforms, employing CDC-based ingestion strategies integrated with Python/PySpark and Teradata, ensuring robust data storage and management.

•For DBS Bank (onsite Singapore), developed AWS Cloud data warehousing solutions implementing data modeling and ingestion processes that adhered to optimized relational database practices and quality assurance standards.

•For OVO Holdings (onsite Jakarta) oversaw Azure Cloud data modeling and warehousing projects, ensuring the creation of effective cloud service patterns and comprehensive OLTP and OLAP data models using Informatica and CDC-based ingestion techniques in a Python/PySpark analytics environment.

•At Navistar, architected and engineered AWS Cloud data solutions with Data Modeling revisions to support integrating Datastage ETL pipelines, Hadoop, and Oracle platforms to enable production scheduling and manufacturing effeciency to meet product delivery commitments.

•For Nielsen (initially onsite at Oxford, later remote) migrated and converted legacy applications to Hadoop code to generate fine grained data files (like Nielsen MDR, NLTV, AMRLD, etc) for supporting analytics and reporting while aligning with evolving relational model requirements. Testing required download and comparing the legacy files to the files produced by the migrated applications. Created test POC to download the generated test files and execute ETL pipelines. Participated in the Cloud platform selection process with Nielsen SME and staff. Assist with Sentiment analysis and demographic characterization and segmentation.

For Kohl’s, ingested real-time online web transactional data, to identify customer demographics, perform sentiment analysis, assess and characterize segmentation to help recommend additional pull-through products for presentation while browsing, to increase shopping basket totals and improve spend volume.

Intel

Sr Systems Analyst

Jul 2013 - Mar 2016

Santa Clara CA

Led teams to architect, design, integrate, test, deliver, and migrate to Cloud and Hadoop Big Data platforms for clients in Retail, Healthcare payer, Healthcare provider, Insurance, Banking.

•Technologies Used: AWS, Hadoop Hive/Impala/HBase (Cloudera certified), Spark (Cloudera certified), Python/PySpark, Spark Delta tables, Scala, C/C++/C#, AbInitio, Neteeza, Teradata, DB2, Oracle, Git

•For Aetna, mitigated and replatformed legacy on-premise applications using Informatica ETL to Hadoop Hive and HBase DB with Spark. Modeled new high performance Oracle table structures, taking advantage of partitioning and data locality to enable high performance, designed and created Informatica ETL pipelines to process XML encoded source files.

•For Travelers Insurance (project continuation from Extreme Insights below), installed and configured Hadoop Dev and Prod clusters. Used AbInitio, Oracle, SqlServer, and Teradata to migrate several applications (auto, casualty, life, personal lines) from legacy platforms to Hadoop cluster. Delivered training Hadoop coursework, and taught several classes to client personnel.

•For Bancolombia, Used Datastage to implement ETL pipelines to new a new Oracle DB, migrating from multiple sources, integrating multinational access and security regulations into an integrated 360 delivery platform.

•For Citibank, replaced and migrated from an underperforming Financial Asset Reporting application written in Perl, using Ablnitio create ETL pipelines populating a Neteeza DB. Refactored, modelled new target tables, results exceeded performance goals.

•For SAP, as part of Intel's IDH Hadoop product offering, traveled to SAP German headquarters and conducted Hadoop classroom training for SAP personnel.

Xtreme Insights Apr 2012 - Jul 2013

Senior Software Developer Schaumberg IL

Lead teams to architect, design, build, integrate, test, deliver, and migrate to Hadoop Big Data and cloud platforms for clients in Retail, Healthcare payer, Healthcare provider, Insurance, Banking.

•Technologies Used: AWS, Hadoop Hive/Impala/HBase (Cloudera certified), Spark, Iceberg, Python, C/C++/C#, Neteeza, Teradata, DB2, Oracle, Git

•For Travelers Insurance, Installed and configured Hadoop POC clusters. Migrated several applications (auto, casualty, life, commercial lines) using AbInitio, C#, PL/SQL, and other legacy toolsets to Hadoop cluster, creating ETL pipelines. Worked on auto rating and telematics application, porting it to the Hadoop POC cluster. Developed Hadoop coursework, and taught several classes to client personnel.

•For Children's National Hospital, Installed, configured, and customized Surgicenter software for resource and facility scheduling and utilization tracking. Oracle target and PL/SQL pipelines. Used SSIS to perform ETL extraction and formatting of SqlServer source files that served as Surgicenter inputs.

•For McKensie Investments in Manhattan, used the Talend ETL tool to create, enhance, and troubleshoot pipelines processing their client financial records to Hadoop and AWS targets, used Kafka to process incoming streams of stock and investment changes to provide real time status and valuation.

•At MediaMath in Manhattan, modified their Advertising Campaign Management application to harvest clicks and impressions using ETL pipelines to associate and analyze each click resulting from online Ad impressions. The demographics for each click were derived, in order to assess, fine tune, and target the desired demographic audience for each Ad Campaign.

Hewlett-Packard Jan 2007 - Apr 2012

Software Developer Palo Alto CA

Lead teams to design, build, test, deliver, and migrate ETL pipelines to Hadoop Big Data platforms for clients in Retail, Healthcare payer, Healthcare provider, Insurance, Banking.

•Technologies Used: AWS, Hadoop Hive/Impala/HBase (Cloudera certified), Spark, Python/PySpark, Scala, C/C++, Neteeza, Teradata, DB2, Oracle, Git

•For Cigna, Mitigated and replatformed an underperforming Claims processing solution to Informatica ETL pipelines and Oracle target DB. Modeled new high performance Oracle tables with dynamic partitioning, designed and created Informatica ETL pipelines to read very large volumes of HL7, Cerner EMR, and other healthcare claims files, implemented the new solution to achieve 50% performance gain that was easily scalable, to meet and exceed SLA.

•For ACE Insurance, design, develop, and deploy a Data Warehouse and DataMarts using the Informatica 8 toolsuite, on IBM servers, with DB2 V9. The project was a long term consolidation and migration from multiple legacy systems and platforms, into a single integrated application to achieve 50% performance gain that was easily scalable, to meet and exceed SLA.

•For media & entrtainment client Time Warner Cable, used Informatica to mitigate and replatform from underperforming legacy applications to an EDW with ETL pipelines reading very large volumes of subscriber media & entrtainment usage files, to Neteeza target DB.

Worked with SME and client Business Analysts to define and create target Dimensional Model that integrated legacy siloed applications. The ETL pipelines showed gains in performance and reduced storage costs and latency, enabling real time dashboards and timely accurate reporting of subscriber event and content consumption patterns and demographics.

Knightsbridge Solutions LLP Mar 2003 - Jan 2007

Data Warehouse Developer Chicago IL

Lead teams to design, build, test, deliver, and migrate from Legacy to Unix/Linux Data Warehouses & ODS to Hadoop Big for clients in Retail, Healthcare payer, Healthcare provider, Insurance, Banking.

•Technologies Used: Datastage, Informatica, Ablnitio, Unix/Linux, Basic, C/C++, Ksh, Oracle, DB2, Informix, SqlServer, Teradata, MQ Series.

•For HCSC (BCBS of IL, TX, NM, MT, OK) Used Datastage to build an Enterprise ODS hosted on Linux with DB2 DB. Ingested data from multiple operational systems, including claims, payments, providers, demographics, underwriting, billing, and policies. Designed and modeled the target DB to exceed the requirements and SLA, applied DB2 table storage partitioning and query optimization to achieve performance and scalability to meet and exceed the SLA.

•For AIG Auto Insurance, design, develop, and deploy a Data Warehouse using DataStage toolsuite, on HP servers, with Oracle, enabling more accurate and timely development of rate structures, products, and premiums. Led a small team to build custom Datastage jobs for processing and transaction control of incoming COBOL files.

•For Assurant Health, design, develop, and deploy multiple upgrades and enhancements to their Data Warehouse and Datamarts using DataStage toolsuite. Requirements, source analysis, entity mapping, data modeling with ERwin, ETL and process construction, testing, and delivery troubleshooting. Provide EDW infrastructure and process architecture design and development to interface with Autosys Enterprise scheduling.

•For LaFarge Cement, mitigated and replatformed from an underperforming solution to Informatica ETL and Oracle. Modeled new high performance Oracle tables, designed and created Informatica ETL pipelines, implemented the new solution to achieve 50% performance gain that was easily scalable, to meet and exceed SLA.

•At Kaiser Permanente Insurance Used Datastage to build and deploy an EDW, providing timely access to policies, claims, payments, providers, by Client Service and Sales personnel while handling phone, internet, and paper client inquiries. The streamlined access resulted in shorter client wait times, timely resolution, increased satisfaction, and higher enrollment rates.

•For AG Edwards, Migrated existing Financial data and applications to an EDW hosted on a cluster of Unix Servers and Oracle database. Used Datastage and MQ series, creating ETL pipelines to ingest, cleanse, and conform to a new Dimensional Model. Worked closely client SME and Business Analysts to create and validate mapping rules, and with the Testing team to create and execute test cases with Datastage, validating results, debugging/mitigating any errors.

Walt Disney World Jan 2000 – Mar 2003

Data Warehouse Developer Lake Buena Vista FL

Used Datastage ETL tool to design, build, test, and deliver Disney’s Smart Data Warehouse, for Travel, Lodging, Food&Beverage, Transportation, Parks&Entertainment, Cruises, Admission Media.

•Used C/C++, Ksh/Csh, Perl, Unix/Linux, Basic, Informix, SqlServer, Oracle, COBOL to ingest, cleanse, classify, enrich, and transform data from operational and legacy sources into a target Kimball/Inmon Dimensionally Modeled Data Warehouse.

•Worked closely in daily and weekly meetings with business analysts and client SME capturing and refining metadata and business rules, mapping to a common consistent enterprise repository.

•Used the repository to create source-target mappings driving ETL, with feedback and validation provided by SME and stakeholders, to ensure that the delivered EDW would meet their operational needs and requirements.

•The validated repository and source-target mappings were used as input to generate test case documentation and test case code checked in alongside the application ETL code, enabling regression testing and bug fix research/testing.

Southland Corporation (7-11 stores) Aug 1998 - Jan 2000

Software Developer Dallas TX

•Used C/C++, Ksh/Csh, Perl, Unix/Linux, Oracle, SqlServer to ingest, cleanse, classify, enrich, and transform product data, retail store sales transactional data, and customer data from operational sources into a target Data Warehouse, to track and forecast sales and distribution by season and region.

Perot Systems Jul 1997 – Aug 1998

Software Developer Maitland FL

Used C/C++, Ksh/Csh, Perl, Unix/Linux, Oracle, SqlServer to design, build, test, and deliver software solutions.

•Blockbuster - Used C/C++, Ksh/Csh, Perl, Unix/Linux, Basic, Oracle to ingest, cleanse, classify, enrich, and transform product and customer retail data from operational sources into a target Kimball/Inmon Dimensionally Modeled Data Warehouse.

•Alamo Rent A Car - Used C/C++, Ksh/Csh, Perl, Unix/Linux, Basic, SqlServer to ingest, cleanse, classify, enrich, and transform car rental and customer data into a target Data Warehouse, supporting reports and dashboards.

Cincinnatti Bell Information Systems/Convergys Aug 1991 – Jul 1997

Software Developer Maitland/Lake Mary FL

Used C/C++, Ksh/Csh, Perl, Unix/Linux, Oracle, SqlServer to design, build, test, and deliver software solutions.

•Precedent 2000 – Creation, deployment, and ongoing on-call Prod Support for this integrated Unix/Windows solution for nationwide PCI/Cellular providers to perform client provisioning, adminstration, rating, and billing, rating. - Used C/C++, Ksh/Csh, Perl, Unix/Linux, Basic, Oracle to ingest Switch Detail Call Records, cleanse, classify, enrich, and transform them for account creation and support, administration, and generation of monthly bills. Revenue Assurance trascking and balance to the penny.

At one point, it created over 80% of the cellphone bills for the United States market.

•Installation and remote support of the CB/ILAS Automated ATT Switchboard product, providing user administration and mangement, as well as detail level billing and accounting information for departmental cost allocation.

•ATT 5E switch migration and update of the global network, employing Unix. C, and Tuxedo to upgrade and validate switch software and infrastructure.

Education

University of Central Florida - Orlando

Part time pursuit of Master of Science in Computer Science 1990 – 1995

University of Central Florida - Orlando

Bachelor of Science in Computer Science, minor mathematics 1989 – 1990

Florida International University - Miami

Pursued Bachelor of Science in Computer Science 1984 – 1989

Miami-Dade College – Miami

Associate of Arts, 1989 1968 – 1989

Contact this candidate