Senior Data Engineer Life Sciences / Pharma

Senior Data Engineer - Life Sciences / Pharma

Location: South San Francisco, CA (Hybrid - 3 Days Onsite) Duration: 8-12+ Months Contract

Job Description

Value Technology is seeking a highly experienced Senior Data Engineer with a strong background in Life Sciences and Pharmaceutical industries to join a long-term strategic data transformation initiative. The ideal candidate will possess deep expertise in AWS-based Data Engineering technologies, data integration, large-scale ETL/ELT pipeline development, and enterprise data platforms.

The candidate should have hands-on experience building scalable and reliable data solutions for clinical, research, commercial, or healthcare-related datasets while ensuring data quality, governance, and compliance standards.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using AWS cloud technologies.

Build enterprise-grade data ingestion and integration frameworks for structured and semi-structured datasets.

Develop and optimize PySpark and SQL-based transformations for large-scale data processing.

Implement data orchestration workflows using Apache Airflow for batch and real-time processing pipelines.

Design and maintain scalable data lake and data warehouse solutions using AWS S3, Redshift, RDS, and EMR.

Develop data quality, profiling, validation, and reconciliation frameworks to ensure data integrity and compliance.

Collaborate with business stakeholders, data scientists, and analytics teams to deliver data products and reporting solutions.

Implement API-based integrations using AWS API Gateway and Lambda services.

Design and maintain dimensional data models including Star Schema and Snowflake Schema for analytical reporting.

Perform performance tuning and optimization of ETL pipelines, Spark jobs, and SQL queries.

Develop reusable data engineering frameworks and metadata-driven processing solutions.

Ensure adherence to data governance, security, and compliance standards within Life Sciences and Pharma environments.

Work closely with cross-functional teams in Agile/Scrum environments for sprint planning and delivery.

Support cloud migration and modernization initiatives from legacy platforms to AWS cloud environments.

Monitor, troubleshoot, and resolve production pipeline failures and performance bottlenecks.

Build scalable data products supporting research, clinical, operational, and commercial analytics initiatives.

Required Skills

10+ years of experience in Data Engineering and Data Integration

Strong Life Sciences / Pharmaceutical industry experience is mandatory

Expertise in ETL/ELT pipeline development and orchestration

Strong experience with Data Modeling and Data Warehousing concepts

Experience with Data Profiling, Data Quality, and Data Validation frameworks

Experience building scalable cloud-native data platforms

Knowledge of Data Products and modern data architecture concepts

Experience with CI/CD, Git, and Agile methodologies

Preferred Qualifications

Experience working with clinical, healthcare, laboratory, or pharmaceutical datasets

Experience with real-time data streaming and event-driven architectures

Familiarity with HIPAA, GxP, FDA, or healthcare compliance standards

Exposure to Databricks or Snowflake is a plus

Strong analytical, communication, and stakeholder management skills