Data Engineer
Key Skills for This Role
Full Job Posting
To drive real impact in this role, you will
Build the Data Backbone: Design, build and maintain end to end pipelines that bring internal, API, streaming and third-party data into our data lake.
Shape the Platform's Architecture: Contribute to the design of our data lake and lead complex pipeline components, backed by solid documentation.
Write Code That Lasts: Write and review high quality SQL and Python, using tools like PySpark, dbt and AWS Glue, EMR and S3, while raising the bar on team standards.
Own Data Quality and Compliance: Implement validation, cleansing, security and governance frameworks that keep our pipelines accurate, safe and aligned with obligations like ASIC reporting.
Keep Things Running: Monitor, troubleshoot and optimise pipelines for performance, cost and reliability, and jump in when production incidents need solving.
Grow the Team Around You: Mentor junior and mid-level engineers, contribute to code and design reviews, and lead delivery on your own technical projects.
Bring in What's Next: Evaluate new data sources and third-party APIs, and keep the team's tools and practices current with emerging tech.
Requirements
- Minimum of 3 years’ experience as a Data Engineer, with demonstrable ownership of significant data pipelines or platform components.
- Degree in Computer Science, Software Engineering, or related field.
- Strong, hands-on proficiency in SQL and Python, with practical expertise in PySpark and modern data lake/lakehouse solutions.
- Proven ability to design and build cloud-based data lake, warehouse, and lakehouse solutions, including complex ETL/ELT pipelines from multiple sources.
- Deep experience with AWS services (Glue, Crawler, S3, EMR, RDS, DMS, Lambda, Batch, Athena, Redshift) to build data lake infrastructure; experienced in building and maintaining API-based, streaming, and CDC ingestion.
- Strong track record of monitoring, troubleshooting, and optimizing AWS data pipelines across production and development environments.
- Experience with transformation frameworks such as dbt and a solid understanding of data governance concepts.
- Practical experience with DevOps and CI/CD practices.
- Solid exposure to core AWS infrastructure, including IAM, VPC, VPC Peering, NAT Gateway, ACL, Secrets Manager, EC2, and ECS.
- Clear communication skills, with the ability to explain technical concepts to both technical and non-technical audiences.
- Interest in trading, FX, or general finance, with exposure to regulated data environments would be highly advantageous!
What's it like to work at Eightcap?
We have built a performance-driven environment where you're given the autonomy to drive change and the accountability to own the results. We provide a space for the ambitious, adaptable and collaborative to deliver outcomes that fuel our success while building the skills and experience that defines a career.
We also support you through
Ongoing investment in your career development, including technical and professional training.
Wellness and lifestyle perks, such as monthly corporate massages.
Drinks every Friday.
Parental leave entitlements.
Staff referral bonus programme.
Annual flu vaccinations.
Purchased leave.
Novated leasing options for vehicle financing and management.
A great CBD location with easy access to public transport.
A multicultural, collaborative and supportive team culture.
Being part of exciting new projects for the company.
Regular social activities.
About Eightcap
Australian private trading-technology provider serving retail and institutional clients through CFD markets and online trading platforms.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Eightcap
People & Culture / Office Coordinator
Melbourne, AUS
P&C / Office Coordinator
Melbourne, AUS
Junior Account Manager - Bilingual Mandarin
Melbourne, AUS
Account Manager
Melbourne, AUS
PR & Affiliate Content Specialist
Melbourne, AUS
Senior Cloud / DevOps Engineer
Melbourne, AUS
Data Engineer
Melbourne, AUS
PR & Affiliate Content Specialist
Melbourne, AUS