Lead Data Engineer

Data Engineer

Location- Chicago, IL

Key Responsibilities

Data Engineering & Pipeline Development Design, develop, and maintain end-to-end data pipelines in Databricks using Spark and Delta Lake
Build and optimize ELT/ETL processes for structured and unstructured data ingestion into the Data Lakehouse
Implement scalable ingestion patterns (batch and event-driven) from internal systems, third-party APIs, and cloud sources
Hands-on experience with Databricks, Apache Spark (PySpark), and Delta Lake
Strong SQL and data modeling skills (relational and dimensional)
Experience building and supporting data pipelines in a cloud environment (AWS or Azure)
Experience with ELT/ETL tools (e.g., Fivetran, custom ingestion frameworks)
Familiarity with data orchestration tools (Airflow, Control-M)