Data Lead / Senior Data Engineer Databricks Migration

Data Lead / Senior Data Engineer - Databricks Migration

Location: Kansas City (Onsite)

Job Type- Full Time

Project Context:

Customer is undergoing a large-scale migration from Teradata to Databricks over the last 18 months. Data lake hydration is still in progress, with multiple layers supporting domain-based transformations. Hundreds of MIS reports currently running on Teradata must be successfully migrated and validated in the target Databricks environment. The client requires immediate implementation and subsequently scalable testing services.

Key Requirements:

Strong expertise in Databricks Lakehouse architecture

Hands-on experience with Databricks DLT Meta (Delta Live Tables) (not Unity Catalog)

Expertise in using Azure (Stated requirement is Azure, I would think ADLS Gen 2, Azure Data Factory, Azure Key Vault, Azure DevOps)

Expertise in CI/CD implementation in Azure DevOps for deployment pipelines

Experience designing scalable testing and validation frameworks for data migration projects

Proven experience in Teradata to Databricks migration programs

Ability to support multi layer implementation and optimization

Strong understanding of enterprise reporting/data reconciliation processes