Data Engineer Lead
-
IT/Tech
Job Description & How to Apply Below
Summary:
The Data Engineer Lead plays a pivotal role within Dataworks, focused on creating and driving engineering innovation within Dataworks, helping define and build the Dataworks organization and facilitate the delivery of key business initiatives. S/he acts as a "universal translator" between IT, business, software engineers and data scientists, collaborating with these multi-disciplinary teams. The Data Engineer Lead will contribute to the creation of and adherence to technical standards for data engineering, including the selection and refinements of foundational technical components.
S/he will work on those aspects of the Dataworks platform that govern the ingestion, transformation, and pipelining of data assets, both to end users within Fed Ex and into data products and services that may be externally facing. Day-to-day, s/he will be deeply involved in code reviews and large-scale deployments. Under minimal supervision, supports the design, build, test and maintain data pipelines at big data scale.
Assists with updating data from multiple data sources. Other functionalities under close supervision are to work on batch processing of collected data and match its format to the stored data, make sure that the data is ready to be processed and analyzed. Assists with keeping the ecosystem and the pipeline optimized and efficient, troubleshooting standard performance, data related problems and provide L3 support.
Independently implements parsers, validators, transformers, and correlators to reformat, update and enhance the data. Provides recommendations to highly complex problems. Provides guidance to those in less senior positions.
Additional Summary and Preferred
Skills:
Responsible for designing and implementing data pipelines, ensuring data quality and accessibility, creating data models, managing ETL processes, optimizing databases, collaborating with stakeholders, utilizing big data tools, implementing real-time data processing, ensuring data governance and security, documenting data architecture, and delivering data-driven insights.
1. Design and implement data pipelines to collect, process, and store data efficiently.
2. Ensure data quality, integrity, and accessibility.
3. Create data models to support business analytics and reporting.
4. Work with data scientists, analysts, and other stakeholders to understand data needs, and design schemas.
5. Build and manage ETL (Extract, Transform, Load) processes for data integration and automate data ingestion and transformation tasks.
6.Administer and optimize databases (both SQL and No
SQL). Monitor database performance and troubleshoot issues.
7. Collaborate with software engineers to integrate data systems with applications.
8. Utilize big data tools for processing large datasets, such as Hadoop or Spark.
9. Implement distributed systems for real-time data processing.
10. Ensure data governance and security through compliance with data governance policies and data security standards.
11. Implement data access controls and encryption techniques.
12. Document data architecture, pipelines, and processes.
13. Provide reports and insights based on data analysis.
14. Working knowledge of Data Bricks, Big Query, and GCP
Job Description
Understanding in depth both the business and technical problems Dataworks aims to solve
Building tools, platforms and pipelines to enable teams to clearly and cleanly analyze data, build models and drive decisions
Scaling up from "laptop-scale" to "cluster scale" problems, in terms of both infrastructure and problem structure and technique
Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines
Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases Interacting with senior technologists from the broader enterprise and outside of Fed Ex (partner ecosystems and customers) to create synergies and ensure smooth deployments to downstream operational systems
Minimum Education
Bachelor's degree in Computer Science, Information Systems, Mathematics, Engineering, or a related quantitative field - or equivalent formal training and professional experience.
Minimum Experience
Five to Seven (5
- 7) years equivalent work experience in measurement and analysis, quantitative business problem solving, simulation development and/or predictive analytics. A related advanced degree may offset the related experience requirements.
Knowledge,
Skills and Abilities
- * Technical…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction