Staff Data Infrastructure Engineer
Faire · New York City
Job description
About the role
Faire is a technology wholesale platform connecting independent retailers worldwide. Its Engineering organization owns the software that makes the marketplace work, and the Data Platform group supports Product Engineering, Data Science, Machine Learning, Analytics, Strategy, Finance and Product. This Staff Engineer role owns the data plumbing: moving data from production databases (CockroachDB and MySQL) to a place where analysts and data scientists can query it, today via Fivetran, Kafka, Spark and Airflow landing in Snowflake and Databricks. The role is hands-on and sets the technical direction for the next version of the platform.
Key responsibilities
- Set the technical direction for how data moves from production systems into analytical stores and own the roadmap for the next couple of years.
- Build the CDC and streaming ingestion layer: CockroachDB changefeeds and MySQL binlogs into Kafka, then into Iceberg tables on S3, handling ordering, deduplication, late data, schema changes and backfills.
- Implement data contracts and quality checks, and put ownership and SLAs on datasets, wiring quality checks in with tools like Anomalo and Monte Carlo.
- Run Airflow and Fivetran well and advise on what to keep buying versus building.
- Own reliability: SLOs, on-call, incident reviews and follow-through.
- Lead the migration of existing pipelines onto the new platform without breaking what teams depend on, and mentor the engineers around you.
Required profile
- Built and ran data infrastructure at meaningful scale and set its direction.
- Deep experience with CDC and streaming ingestion from operational databases through Kafka, including ordering, duplicates, snapshots and schema evolution.
- Experience with lakehouse architectures on an open table format (Iceberg on S3), and strong SQL.
- Experience with data quality and observability, including data contracts, SLAs and tools like Anomalo or Monte Carlo.
- Experience operating Airflow at scale and with managed ingestion like Fivetran.
- Track record of leading cross-team data initiatives and migrations, and mentoring senior engineers.
- Able to explain technical tradeoffs to leadership and to a new grad, and able to get people to agree on a decision.
- Willing to take ownership of broken or unowned things and to be on call for the systems you build.
- Experience in a marketplace, e-commerce or other transaction-heavy business is a plus.
Required skills
- Python, SQL, Kafka, Airflow, Fivetran, Iceberg, Snowflake, Databricks, Spark, AWS, Terraform, Kubernetes
- CockroachDB, MySQL, Scylla and DynamoDB
- At least one of Kotlin, Java, Scala or Go
What we offer
- Pay range of $246,500 to $339,000 per year in San Francisco and New York, plus equity and benefits.
- Hybrid work: 3 days in the office per week, with remote work up to 4 weeks per year.
- Paid time off, recharge days, paid parental leave up to 16 weeks, comprehensive health coverage, fertility and family-building benefits, mental health support, professional development stipends and a retirement plan.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United States.
Salaries by job title
- Delivery Specialist 330
- Line Cook 264
- Restaurant Host 251
- Dishwasher 241
- Shift Supervisor 226
- Parts Specialist 208
- Retail Service Specialist 176
- Server - Red Robin 140
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 1 week ago
Expires 1 month from now
11 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Faire
New York City