Unstructured Data Engineering Lead

3M is looking for a skilled Unstructured Data Engineering Lead to join our team. As a key member of our organization, you will be responsible for leading the development of pipelines, preprocessing unstructured data, eliminating duplicate data and text noise, chunking data, and generating vector embeddings. In addition to these key capabilities, the candidate should possess strong Python programming skills, expertise in cloud engineering, and experience with open source software to drive innovation and efficiency in handling unstructured data.

• Leading the development of pipelines for preprocessing unstructured data, eliminating duplicate data and text noise, chunking data, and generating vector embeddings.
• Implementing efficient and scalable solutions using Python programming skills and cloud engineering expertise to handle unstructured data effectively.
• Determining the best approaches and techniques for data preprocessing tasks, driving innovation and efficiency in handling unstructured data.
• Supporting the team by providing guidance, mentorship, and technical expertise in data engineering, particularly in the context of unstructured data.

• Lead the design and development of data pipelines for processing and analyzing unstructured data.
• Preprocess unstructured data to eliminate duplicate data, text noise, and other irrelevant information.
• Chunk large volumes of unstructured data into manageable segments for efficient processing.
• Develop and implement algorithms and techniques for generating vector embeddings from unstructured data.
• Collaborate with data scientists, machine learning engineers, and domain experts to define data requirements and objectives.
• Optimize data preprocessing and embedding generation pipelines for scalability and performance.
• Leverage strong Python programming skills to develop efficient and reliable data engineering solutions.
• Utilize cloud engineering expertise to design and implement scalable and cost‑effective data processing architectures.
• Explore and leverage open source software and tools to drive innovation and efficiency in handling unstructured data.
• Stay up‑to‑date with the latest advancements in data engineering and unstructured data processing techniques.
• Mentor and guide junior engineers, fostering a collaborative and innovative team environment.

• Bachelor's degree or higher (completed and verified prior to start) in Computer Science or Engineering.
• Three (3) years of experience in unstructured data engineering at a large manufacturing company in a private, public, government or military environment.
• Three (3) years of experience as a data engineer, with expertise in handling unstructured data.

• Master’s degree in Computer Science, Engineering, or related field from an accredited institution.
• Strong understanding of data engineering concepts and best practices.
• Proficiency in Python programming, with the ability to develop efficient and reliable data engineering solutions.
• Expertise in cloud engineering, with experience in designing and implementing scalable and cost‑effective data processing architectures.
• Familiarity with open source software and tools for data engineering and unstructured data processing.
• Experience with data preprocessing techniques, including duplicate elimination, noise removal, and chunking.
• Knowledge of algorithms and methods for generating vector embeddings from unstructured data.
• Knowledge of distributed computing frameworks, such as Apache Spark or Hadoop.
• Strong analytical and problem‑solving skills, with the ability to optimize data processing pipelines.
• Excellent communication and collaboration abilities, with the capacity to work effectively in cross‑functional teams.
• Ability to adapt to a fast‑paced and dynamic environment.

• Hybrid Eligible (Job Duties allow for some remote work but require travel to Maplewood, MN at least 2 days per week)

Travel:
May include up to 10% International

Relocation Assistance:
May be authorized

Must be legally authorized to work in country of employment without sponsorship for employment visa status (e.g., H1B status).

Applicable to US Applicants Only:
The expected compensation range for this position is $177,961 - $217,508, which includes base pay plus variable incentive pay, if eligible. This range represents a good faith estimate for this position. The specific compensation offered to a candidate may vary based on factors including, but not limited to, the candidate’s relevant knowledge, training, skills, work location, and/or experience.

3M is an equal opportunity employer. 3M will not discriminate against any applicant for employment on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, or veteran status.