Data Engineer

Full job description

What do incubators and Predictive Sales AI (PSAI) have in common? Well, they’re great for growth, and even better for stability. As an innovative software development and digital marketing company pioneering the field of artificial intelligence, we can offer our newest Data Engineer the best of both worlds – a high energy, forward‑thinking start‑up culture inside of a well‑established, profitable, and stable structure.

If you’re interested in getting your hands dirty and inciting change in a larger organization with a vision to change the world using data today, then please read on.

• Design, build and maintain both new and existing data infrastructures using tools including (but not limited to) Microsoft SQL Server, Azure Tables, Azure Blobs, and Azure Cosmos.
• Determine the right database system for a provided data set.
• Draft database/table schemas.
• Properly index tables when applicable to ensure query performance.
• Create data pipelines for transforming new datasets into consistent, reliable data systems that are usable across multiple platforms.
• Identify optimizations and improvements in existing data pipelines.
• Create solutions for data validation to ensure data integrity, accuracy, and consistency.
• Ensure data is readily available for analysis and development usage.
• Shift existing big data pipelines.
• Come up with data validation solutions for both initial data imports as well as results.
• Audit the automation process to make sure there are no gaps, redundancies, or faults in the data pipeline that would cause damage to data integrity.
• Plan query‑efficient data store schemas with respect to the data requirements of our internal data warehouse.
• Communicate with data scientists and other software engineers to build proper data pipelines for efficient and cost‑effective querying.
• Develop, construct, test, and maintain architectures with the help and feedback of software engineers.
• Determine and implement proper data schemas that align with PSAI's and Spectrum’s client’s business objectives.
• Recommend and implement ways to optimize data reliability, efficiency, and quality.
• Consult and suggest different technologies for our team to test and utilize together.
• Incorporate the business objectives of key stakeholders in the architecture of our data warehouse.
• Research new and interesting data acquisition opportunities.

• Self‑Starter. Building a data warehouse is no simple task. You will need to come in with a self‑starter attitude to not only make this data engineering role what you want it to be, but make a stellar and efficient data warehouse while you’re at it.
• Analytical. In order to solve problems and build innovative new digital marketing campaigns, it is essential that you know how to take an idea and analyze it from all of its angles.
• Developer. In the ever changing world of artificial intelligence, it’s not enough just to build models. You bring to the table an eye for data patterns along with a knack for relational database engineering.
• Patient. As a data engineer, you know that you work with extremely large data sets on a daily basis. As such we are looking for someone who is not only meticulous, but patient enough to sit and sift through that data in a thorough way.
• Creative. Beyond just analyzing data sets, you are an explorer and a puzzle solver. Pulling insights out of your data and understanding how those insights can better shape our tools is something that you live to do.
• Student. More so than most industries, the field of data science is always changing and evolving. As such, you are always looking to learn new things and gain new skills.

• A bachelor’s degree or pursuing a bachelor’s degree in computer science, mathematics, statistics, information systems, or a related field.
• Experience with statistical modeling.
• 1–3 years working experience with Python and/or SAS languages.
• 1–3 years working experience with SQL databases and database querying languages.
• 1–3 years working experience with C#/.NET.
• Familiarity with Microsoft Azure.
• Familiarity with Graph database structures.
• Experience with both RDBMS and TDMS.
• Expe…