Senior Data Engineer - Customer Data and Insights

More than a career - a chance to make a difference in people's lives.

Build an exciting, rewarding career with us – help us make a difference for millions of people every day. Consider joining the Duke Energy team, where you'll find a friendly work environment, opportunities for growth and development, recognition for your work, and competitive pay and benefits.

Position Summary:

This position is responsible for development and consulting activities related to analytics within various Duke Energy IT and business units.

Support or collaborate with application developers, database architects, data analysts and data scientists to ensure optimal data delivery architecture throughout ongoing projects/operations.

Design, build, and manage analytics infrastructure that can be utilized by data analysts, data scientists, and non-technical data consumers, which enables functions of the big data platform for Analytics.

Develop, construct, test, and maintain architectures, such as databases and large-scale processing systems that help analyze and process data in the way the Analytics organization requires.

Develop highly scalable data management interfaces, as well as software components by employing programming languages and tools.

Work closely with data subject matter experts to determine what data management systems are appropriate and data scientists to determine which data is needed.

Work with stakeholders to understand the information needs and translate these into technical solutions.

Work closely with a team of Data Science staff to take existing or new models and convert them into scalable analytical solutions.

Design, document, build, test and deploy data pipelines that assemble large complex datasets from various sources and integrate them into a unified view.

Identify, design, and implement operational improvements: automating manual processes, data quality checks, error handling and recovery, re-designing infrastructure as needed.

Create data models that will allow analytics and business teams to derive insights about customer behaviors

Build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications.

Responsible for obtaining data from the System of Record and establishing batch or real-time data feed to provide analysis in an automated fashion.

Develop techniques supporting trending and analytic decision making processes

Apply technologies for responsive front-end experience

Ensure systems meet business requirements and industry practices

Research opportunities for data acquisition and new uses for existing data

Develop data set processes for data modeling, mining and production

Integrate data management technologies and software engineering tools into existing structures

Employ a variety of languages and tools (e.g. scripting languages) for integration

Recommend ways to improve data reliability, efficiency and quality

Responsibilities:

Develop components required for optimal extraction, transformation, and loading of data from a wide variety of data sources using a set of tools or languages (e.g. SQL)

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Implement big data analytics solutions, as well as integrate new data management technologies and software engineering tools into existing structures

Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.

Solve problems for Stakeholder, construct problem frames in order to understand the analysis context and scope that will provide timely, useful results

Create data tools for analytics and data scientist team members that assist them in building and optimizing products.

Work in Project Teams and participate in multidisciplinary analytics project teams, translating complex functional and technical requirements into detailed architecture, design, and high performing software/techincal components

Work with data and analytics experts to strive for greater functionality in data systems.

Partner with business clients, establish professional relationship, and communicate with analytics clients in order to understand business needs.

Employees in this role will also:

Translate complex functional and technical requirements into detailed architecture, design, and high performing software.

Drive user ideation analysis and elaboration, design and development of software applications, and build automation tools.

Select data solution software and define hardware requirements.

Help build a next-generation Big Data analytics framework with a set of core technologies.

Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.

Ensure all automated processes preserve data by managing the alignment of data availability and integration processes.

Collaborate with data architects, modelers and IT team members on project goals.

Assist with extracting standard sets of features from important datasets that will be leveraged by Data Science teams.

Convert code from applications.

Build processes supporting data transformation, data structures, metadata, dependency and workload management.

Required/Basic Qualifications

Bachelors degree in Management of Information Systems, Engineering, Mathematics, Computer Science or Other Related Degree

In addition to required degree, three (3) or more years related work experience

In lieu of Bachelors degree(s) AND three (3) or more years related work experience listed above, High School/GED AND five (5) or more years related work experience

Desired Qualifications:

Architected, modeled and implemented a solution for multiple work management systems, including SS9, Maximo and Field Collection System in AWS Redshift

Additional Preferred Qualifications

Knowledge of code writing.

Experience with relational databases, query authoring (SQL) as well as familiarity with a variety of RDMS.

Knowledge of building and optimizing ‘big data’ data pipelines, architectures and data sets.

Knowledge of Open Analytics Platforms (such as Hadoop ecosystem)

Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

Excellent verbal and written communication skills

Self-starter who works with minimal supervision and the ability to work in a team of diverse skill sets

Ability to comprehend customer requests and provide the correct solution

Strong analytical mind to help take on complicated problems

Desire to resolve issues and drive into potential issues

Familiarity with the Agile Methodology

Strong analytic skills related to working with unstructured datasets. Excellent verbal and written communication skills.

Experience supporting and working with cross-functional teams in a dynamic environment.

Experience with big data tools: Hadoop, Spark, Kafka, etc.

Experience with relational SQL and NoSQL databases, including SQL Server and Cassandra.

Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Understanding of distributed computing principles.

Working Conditions

Hybrid – Work will be performed from both remote and onsite locations after the onboarding period. However, hybrid employees should live within a reasonable commute to the designated Duke Energy facility.

Travel RequirementsNot required

Relocation Assistance Provided (as applicable)No

Represented/Union PositionNo

Visa Sponsored PositionNo

Posting Expiration DateTuesday, May 27, 2025

All job postings expire at 12:01 AM on the posting expiration date.

Please note that in order to be considered for this position, you must possess all of the basic/required qualifications.

Privacy

Do Not Sell My Personal Information (CA)

Terms of Use

Accessibility

R35146