Test Automation Engineer​/Lead; Databricks, ETL​/Data AI​/ML Exp

Test Automation Engineer/Lead (Databricks, ETL/Data) with AI/ML Exp. 8+Years Louisville, KY / Dallas, TX/ Chicago, IL Contract

Required Skills:

6+ years of experience in ETL/Data Testing. 3+ years of experience testing Databricks or Spark-based platforms. Hands-on experience in AI/ML Model Testing or MLOps validation. Strong coding experience in Python and SQL. Databricks & Data Engineering Testing Bronze, Silver, and Gold layer data transformations within Databricks environments. Verify workflow orchestration, scheduling, and dependency management across Databricks jobs and pipelines. Perform performance, scalability, and reliability validation of Spark and PySpark transformations.

Validate data quality frameworks, rules, and monitoring mechanisms implemented in Databricks. Ensure data accuracy, completeness, consistency, and lineage across data pipelines. AI/ML Testing Validate AI/ML model integrations within enterprise applications and data ecosystems. Verify model scoring results, prediction outputs, and business rule compliance. Perform data drift, schema drift, feature drift, and model validation testing. Validate feature engineering pipelines, training datasets, and inference datasets.

Execute model input/output verification against expected business outcomes and acceptance criteria. Validate explainability, traceability, auditability, and regulatory compliance requirements. Test model deployment workflows using MLflow, Databricks ML, Azure ML, or similar MLOps platforms. Validate Generative AI and RAG (Retrieval-Augmented Generation) workflows, including prompt accuracy, response quality, grounding, and hallucination detection. Integration Testing Perform end-to-end integration testing across:
Databricks Data warehouses APIs and microservices AI/ML platforms Cloud services Reporting and analytics systems Validate data movement and transformations between source systems, data lakes, ML models, and downstream applications. Perform API testing, contract testing, and interface validation. Ensure seamless interoperability across enterprise data and AI ecosystems. Test Automation Design and develop automation frameworks for: ETL/Data Pipeline Testing API Testing Data Validation AI/ML Model Validation Build reusable test utilities, libraries, and accelerators using Python.

Integrate automated test suites into CI/CD pipelines. Develop automated quality gates and release validation mechanisms. Generate automated execution reports, dashboards, test metrics, and quality insights. Support continuous testing and shift-left quality practices across the software delivery lifecycle. Key Skills Databricks, Spark, PySpark Delta Lake, Medallion Architecture (Bronze/Silver/Gold) Python SQL ETL/Data Warehouse Testing API Testing AI/ML Testing MLflow, Azure ML GenAI/RAG Validation CI/CD Integration Test Automation Frameworks Data Quality and Data Governance Diverse Lynx LLC is an Equal Employment Opportunity employer.

All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.