Sr. Data Engineer Job Columbus area,Ohio USA,IT/Tech

Designs and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured).
Own end-to-end delivery of AI and ML solutions from problem definition to production deployment
Build and maintain data pipelines using PySpark and Spark in Azure Databricks
Designs and implements framework to continuously monitor and troubleshoot data quality and data integrity issues.
Implements data governance processes and methods for managing metadata, access, retention to data for internal and external users.
Designs and provide guidance on building reliable, efficient, scalable and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages.
Perform feature engineering and data preparation for machine learning models
Deploy ML models and AI agents for real business use cases
Design and implement agentic AI workflows including multi-step reasoning and tool usage
Track experiments, manage models, and support deployment using MLflow
Define and execute model evaluation frameworks including both ML and AI agent performance
Designs and implements physical data models to define the database structure. Optimizing database performance through efficient indexing and table relationships.
Participates in optimizing, testing, and troubleshooting of data pipelines.
Designs, develops and operates large scale data storage and processing solutions using different distributed and cloud-based platforms for storing data (e.g. Azure , AWS, Spark , Pyspark, Scala , advanced SQL , Hadoop others).
Uses innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks to minimize manual and error-prone processes and improve productivity. Assists with renovating the data management infrastructure to drive automation in data integration and management.
Ensures the timeliness and success of critical analytics initiatives by using agile development technologies such as Dev Ops, Scrum, Kanban
Coaches and develops less experienced team members.
Define and execute model evaluation frameworks including both ML and AI agent performance
Work independently on ambiguous business problems and convert them into scalable solutions
Collaborate with other data engineers, analysts, solution architects and business teams to deliver solutions
Guide and support team members on data engineering, ML, and AI best practices
Write clean, production-ready, and well-documented code

Requirements

System Requirements Engineering
- Uses appropriate methods and tools to translate stakeholder needs into verifiable requirements to which designs are developed; establishes acceptance criteria for the system of interest through analysis, allocation and negotiation; tracks the status of requirements throughout the system lifecycle; assesses the impact of changes to system requirements on project scope, schedule, and resources; creates and maintains information linkages to related artifacts.
Collaborates
- Building partnerships and working collaboratively with others to meet shared objectives.
Communicates effectively
- Developing and delivering multi-mode communications that convey a clear understanding of the unique needs of different audiences.
Customer focus
- Building strong customer relationships and delivering customer-centric solutions.
Decision quality
- Making good and timely decisions that keep the organization moving forward.
Data Extraction
- Performs data extract-transform-load (ETL) activities from variety of sources and transforms them for consumption by various downstream applications and users using appropriate tools and technologies.
Programming
- Creates, writes and tests computer code, test scripts, and build scripts using algorithmic analysis and design, industry standards and tools, version control, and build and test automation to meet business, technical, security, governance and compliance requirements.
Quality Assurance Metrics
- Applies the science of measurement to assess whether a solution meets its intended outcomes using the IT Operating Model (ITOM), including the SDLC standards, tools, metrics and key performance indicators, to deliver a quality product.
Solution Documentation
- Documents information and solution based on knowledge gained as part of product development activities; communicates to stakeholders with the goal of enabling improved productivity and effective knowledge transfer to others who were not originally part of the initial learning.
Solution Validation Testing
- Validates a configuration item change or solution using the Function's defined best practices, including the Systems Development Life Cycle (SDLC) standards, tools and metrics, to ensure that it works as designed and meets customer requirements.
Data Quality
- Identifies, understands and corrects flaws in data that supports effective information…