Core Data Engineer

What you will do

• Craft datasets with quants. Take an underspecified research question and turn it into the right dataset: aligned, adjusted, point-in-time correct, documented, and reproducible.
• Develop and own the end-to-end lifecycle of these data products. Build and maintain pipelines, develop quality checks, correct day-to-day operational issues, and systematize/automate manual processing steps
• Own data quality where it matters. Design the checks and alerts that catch the problems that would actually hurt a strategy, and deliberately leave out the ones that are just noise.
• Build shared foundations. Reference data, instrument identifiers, corporate actions and vendor feeds that every researcher and trading desk depends on.
• Keep critical pipelines running. Markets in several regions, and digital asset markets that never close, depend on this data. You will be part of a team that supports it across time zones.
• Analytics or data engineering experience, ideally at a startup or scale-up through a growth phase
• Strong data modelling and data quality practice: tests, contracts, lineage, observability in production
• Strong proficiency in Python (ideally familiarity with another statically-typed language)
• Working proficiency with SQL and relational databases, and version control systems
• Demonstrated ability to work with large datasets and familiarity with big data solutions
• You have owned something in production, not only built it
• You enjoy working directly with the people who use your data
• Strong interpersonal and communication skills, with the ability to collaborate effectively with technical and non-technical stakeholders
• Ability to balance individual stakeholder requirements with data quality, consistency, and broad applicability
• Experience with financial data: prices, market data, transactions, instruments
• Familiarity with point-in-time data, look-ahead bias and survivorship bias
• Experience with a modern cloud data stack, such as Spark or Databricks, dbt, a cloud warehouse, and orchestration
• Experience with Airflow, Kafka, and GCP (esp. GKE, Dataproc, and Big Query), strongly preferred
• Experience with Polars, Pandas, Spark, Apache Beam, Apache NiFi
• Experience at a trading firm, systematic fund, brokerage, crypto exchange or market data company
• On-call or incident response experience
• At Flow Traders, we acknowledge the importance of open and transparent communication whether it be with our employees, our stakeholders, or our local and global communities. When it comes to salary, Flow Traders uses reliable market research to create base ranges. Where candidates will fall within the range depends on a few different factors including but not limited to level of experience, location, and specific skill set. We also consider ourselves one global team, and to demonstrate that, all employees are eligible to share in the company’s success through an annual discretionary variable remuneration allocated based on company, group and individual performance and contribution.
• Per NYC salary transparency law, the total compensation for this role includes a base range of $165,000 to $210,000 plus annual discretionary variable remuneration.
• Flow Traders does not accept unsolicited resumes from any professional staffing or search firms. All resumes, and any other information identifying potential candidates, submitted to any employee at Flow Traders via-email, the Internet or directly without a valid and signed search agreement will be deemed free to contact by Flow Traders without any restrictions and no placement fee of any kind will be paid in the event the candidate is hired by Flow Traders.
• #LI-MM1

What you need to succeed

Qualifications

• Experience with financial data: prices, market data, transactions, instruments
• Familiarity with point-in-time data, look-ahead bias and survivorship bias
• Experience with a modern cloud data stack, such as Spark or Databricks, dbt, a cloud warehouse, and orchestration
• Experience with Polars, Pandas, Spark, Apache Beam, Apache NiFi

Preferred Qualifications

• Experience with Airflow, Kafka, and GCP (esp. GKE, Dataproc, and Big Query), strongly preferred