Principal Data Engineer Job Nashville area,Tennessee USA,IT/Tech

Job Description

Oracle Cloud Infrastructure (OCI) is seeking an AI native, self-starter and detail-oriented Data Engineer to join our dynamic and fast-paced data analytics team. As a Data Engineer at Oracle OCI, you will play a crucial role in providing data-driven decision support and analysis for OCI's largest products and programs. This is a position that offers an exciting opportunity to take ownership of large portions of the data ecosystem that OCI runs on.

Responsibilities

Key Responsibilities

Data Processing & Pipelining - Data Requirements, Collection, and Infrastructure:

Mentors less experienced team members to identify data requirements and business objectives of a project or initiative.
Provides expertise on the design and participates in building of data infrastructure to optimize data processing from a variety of data sources.
Independently analyzes, designs, and troubleshoots data flows based on business needs.
Participates and designs architecture, performance, and security reviews of the technical solution.
Adjusts data collection processes that involve indexing and query optimizations for optimal performance.
Builds Extract, Transform, and Load (ETL) pipelines to support efficient and scalable data collection and extraction.
Engages with and holds the upstream and downstream teams accountable for the predefined service level agreements (SLAs).
Manages relationships with the data providers.

Data Processing & Pipelining - Data Governance:

Independently designs and implements data governance policies and procedures for data handling (e.g., data retention) to manage data consistency, integrity, accuracy, and reliability throughout the data lifecycle.
Leads the execution of redaction processes for Personally Identifiable Information (PII) and Protected Health Information (PHI) data, ensuring compliance with data privacy and security standards.
Ensures minimal data collection and usage in accordance with data minimization principles.
Follows data security measures to protect data from unauthorized access, use, disclosure, alteration, or destruction, proactively identifying and escalating potential issues.
Ensures data compliance with relevant laws, regulations, and industry standards.

Data Processing & Pipelining - Data Validation & Quality Assurance:

Provides expertise on and participates in the design and implementation of rigorous data validation and integrity checks, proactively mitigating data quality issues that could impact data pipeline and model performance.
Mentors less experienced team members to define data annotation and labeling processes to ensure data quality.
Identifies opportunities for automation of data validation and governance, and implements them.
Independently corrects deviations and non-conformance when identified.

Data Pipeline and Solutions Engineering - Pipeline Design:

Leverages advanced knowledge of ETL processes to design, develop, and optimize automated, scalable, and efficient data pipeline architectures to build reusable data products.
Implements advanced data storage solutions to store the processed data to be used in a scalable, optimized, and efficient way for access and analysis.
Mentors less experienced team members to manage the flow of data pipeline and storage day-to-day operations.

Data Pipeline and Solutions Engineering - Data Solutions Engineering:

Works independently and collaboratively in an agile development environment with other engineers to develop, maintain, and debug advanced data solutions that are scalable, efficient, cost effective, and reliable.
Reviews runnable code and works on testing and debugging of data solutions with less experienced team members.
Evaluates new technologies to create more robust data solutions.
Enforces and documents code standards and guidance within the team.
Creates documentation for design decisions, and obtains feedback from the broader architecture team before implementing.
Gathers data and evidence to secure necessary approvals.

Core Responsibilities

Planning & Execution:

Manages and coordinates moderately complex tasks, monitoring timelines and deliverables to ensure timely completion and adherence to requirements for a…