Job Description & How to Apply Below
The Senior Lead Data Engineer will be part of a dynamic team dedicated to supporting our customers technologies. This role will allow you to provide hands on expertise in the Data Engineering and BigData Technology space. In this role you will be designing and providing expertise in Azure Data Bricks and Spark enabled Data Engineering Services that allow enterprises to build, deploy and manage BigData software with modern software delivery practices in order to achieve greater speed and agility
Apply technical knowledge to architect solutions that meet business, and IT needs, create roadmaps, and ensure long term technical viability of new deployments, infusing key analytics and AI technologies where appropriate (e.g., Azure Machine Learning, Machine Learning Server, BOT framework, Azure Cognitive Services, Azure Databricks, etc.)
Increase consumption of solutions including Azure SQL Database, Azure Cosmos DB, Azure SQL Data Warehouse, Azure Databricks, Azure HD Insight, Azure Machine Learning, Azure Stream Analytics, Azure Data Factory, Azure Event Hubs
Proven track record of driving decisions collaboratively, resolving conflicts and ensuring follow through with exceptional verbal and written communication skills
Prior work experience in a Consulting/Architecture position within services company with more than 2 end to end hand on experience on Azure technology
8 -10 years of experience in the Data Engineering, ETL, BigData, and transformational data areas
Should have strong understanding of Data Bricks and this architecture/schema
Must have at least 2+ years of hands on development with Apache spark using the following modules (batch/SQL and stream processing)
Must have at least 2+ years programming in python and
Extensive hands on experience of writing notebooks in data bricks using python for complex data aggregations, transformations, schema operations. Good familiarity with Databricks delta and data frames concepts
Extensive hands on experience in designing and implementing scalable ETL pipelines to process variety of data types (structured, unstructured), file formats (json, csv, text delimited…)
Must have completed at least 1 end to end implementation for a data solution on Azure using either technologies mentioned above
Fair knowledge of configuring Azure Data Bricks cluster and other data services for an enterprise grade production environment.
Manage a small team of data services engineers on Azure.
Good to have – exposure to Enterprise Data Governance Models
Strong Analytical and Problem-solving skills
Strong experience of 2 years in end to end implementation of data solutions preferably for large enterprise
Strong knowledge of data facets such as Analytics and reporting
Well informed on cloud-native technologies that enable batch and streaming data ingestion into cloud (For example Azure Data Factory, Azure Event Hubs, Azure IoT Hubs, etc.)
Experienced in designing data lakes in Azure cloud for serving big data analytical workloads.
Good to have – Knowledge of Automated machine learning and deep learning frameworks in Azure to perform end-to-end machine learning from development to production.
Silicus is a cloud transformation services company, focused on delivering technology that enables customers to innovate, deploy competitive advantage at scale, standardize and automate to reduce complexity and risk from operations, and drive intelligence for business strategy and operational improvements.
Our global team of 650+ professionals across six offices work closely with customers to achieve these goals, by providing cloud technology services for software & devops, infrastructure modernization, data analytics, connected IoT solutions, and AI-enabled processes with people-machine interactions.
At Silicus, every voice is heard, the best ideas win, and talent trumps tenure. Our teams blend proven technology experts with trailblazing talent and provide team members with unlimited opportunities to accelerate their careers and expand their skill sets by learning, engaging, and mastering emerging technologies. Come, join the experience!
Visit .silicus.com to learn more.
Silicus is an equal opportunity employer and does not discriminate based on race, ethnicity, color, ancestry, national origin, religion, sex, sexual orientation, gender identity, age, disability, veteran, genetic information, marital status or any other legally protected status.
Less than 1 Year
Search for further Jobs Here: