Senior Data Engineer​/Architect - Databricks

JOB DESCRIPTION SUMMARY

Greystar’s D2AI team is responsible for the platforms, processes, and practices that power AI across the organization. This role influences how data is transformed into intelligent, scalable solutions used by teams company-wide. The position requires AI fluency at the intersection of data, technology, and business outcomes, including designing, operating, and deploying AI-enabled tools in day-to-day work. The role collaborates with engineering, analytics, and business teams to ensure AI solutions are reliable, responsible, and impactful.

We are looking for a skilled Data Engineer to join our engineering team, focusing on leveraging Azure SQL, Cosmos DB, and Databricks to meet growing data demands. As a Sr. data engineer in the Apex organization, you will be part of the Scrum team, developing data capabilities powering customer-facing applications used by thousands of residents and supporting the data needs of the engineering team as you develop and optimize the backend for customer-facing real estate applications.

• 100% hands-on development – Azure SQL, Cosmos DB, Databricks, and Azure OpenAI: develop and unit test database and AI pipeline code, including T-SQL, stored procedures, functions, views, and LLM prompt orchestration layers.
• Own and maintain the Databricks data ingestion and output pipelines for end programs such as  and Microsoft Customer Insights CDP platform, including Delta Lake table optimization, schema evolution, and medallion architecture (Bronze/Silver/Gold) design.
• Architect and maintain AI-ready data structures — clean, well-typed, and optimized for feature engineering and model consumption.
• Participate in the design of databases and feature stores, applying normalization or denormalization as appropriate to support both operational and ML workloads.
• Create, deploy, and maintain ADF and Databricks Workflows pipelines, adhering to Greystar's standards and documented best practices.
• Perform analysis of complex data and document findings, leveraging AI-assisted tooling and notebooks to surface insights faster.
• Prepare and curate data for prescriptive and predictive modeling — including feature extraction, data labeling pipelines, and training/test dataset construction.
• Combine raw data from disparate external sources; build and support complex ingestions including real-time streaming (Event Hub) and batch patterns.
• Collaborate closely with data scientists, ML engineers, and application developers consuming the data — ensuring outputs are well-documented, versioned, and model-ready.
• Integrate and support AI/ML model outputs back into data products — scoring pipelines, inference result storage, and feedback loops for model monitoring.
• Play a direct role in the maintenance, technical support, documentation, and administration of databases and Databricks environments, including Unity Catalog governance.
• Ensure standards are followed by participating in code reviews — including review of AI prompt logic, notebook structure, and pipeline configuration.
• Act as a thought leader: bring forward new ideas in AI, data engineering, and LLM integration — and then build them.

• 6+ years relevant and progressive data engineering experience
• Deep technical knowledge and experience in Microsoft Azure architecture, including Azure PaaS databases, Synapse, ADF pipelines, Azure functions, Event Grids, etc.
• 3+ years of Experience with Cosmos DB
• 3+ years of Experience with Databricks
• Hands-on experience with data pipelines using SQL and No-SQL databases
• Minimum of 1 year of relevant experience working with Azure Data Lakes Gen 2
• Experience with Power Platform / Power BI
• Experience in engineering practices such as code refactoring, design patterns, CI/CD, and highly scalable data applications
• Experience developing batch ETL pipelines; real-time pipelines is a plus
• Knowledge of advanced data engineering concepts such as dimensional modeling, ETL, data governance, data warehousing, structured and unstructured data
• Good experience with Python, machine learning frameworks, and statistics
• Experience or familiarity with D365 Customer Insights platform / Dataverse is a…