Senior AWS/Databricks Cloud Data Engineer

About the role

We are seeking a Senior Cloud Data Engineer to design, build, and maintain scalable data pipelines for our genomics organization. You will work hands‑on with AWS services to support genomic data science models and collaborate with cross‑functional teams to translate business needs into robust data solutions.

Key responsibilities

• Participate in agile ceremonies and stand‑ups to support project delivery.
• Build and maintain data acquisition pipelines from databases, flat‑files, and REST APIs.
• Develop AWS‑native pipelines using Lambda, ECS, EKS, and S3 for genomic data ingestion.
• Generate and manage sequencing and post‑sequencing workflows supporting data science models.
• Troubleshoot pipeline failures, performance issues, data quality concerns, and capacity throttling.
• Create and maintain technical documentation, data flow diagrams, and data mapping documents.

Required profile

• Proven hands‑on experience designing end‑to‑end AWS data pipelines (API ingestion to storage and delivery).
• Experience with High Performance Computing (HPC) clusters.
• Strong Python development skills for ETL/ELT pipelines, with emphasis on low latency and high throughput.
• Ability to work in a fast‑paced consulting environment and manage multiple tasks.

Required skills

• AWS (Lambda, S3, ECS, EKS, CloudFormation/Terraform/CDK/SAM)
• Python (including pandas)
• SQL
• Dimensional modeling, SCD, medallion architecture
• REST API development and consumption
• CI/CD and DevOps tooling
• Container orchestration (Docker, ECS/EKS)
• RBAC implementation and management

What we offer

• Competitive salary $85k‑$125k USD per year.
• Fully invested 401(k) plan.
• PPO and HDHP medical plans, employer‑paid dental and vision.
• Training opportunities and support for certifications.
• Onsite fitness center and hybrid work schedule.