Data Engineer

Core Responsibilities

• Architect, develop and deploy our Big Data environment (Kafka, Hadoop, Dremio, etc.)
• Build, deploy, and monitor our data processing pipelines (Java, Python, Spark, Flink)
• Collaborate with development teams on data modeling, data ingestion, and capacity planning
• Work with users to ensure data integrity and availability
• Act as a Big Data SME and consult on a variety of data-related questions from users and developers

Skills and Experience

• 5+ years experience working in a mature data engineering environment
• 3+ years of experience building Kafka streaming applications and/or maintaining Kafka clusters
• 2+ years of experience building applications/pipelines with Big Data backends (S3, HDFS, Databricks, Iceberg, etc)
• Experience with Apache Spark, Apache Flink or similar tools
• Strong Java, Python, and SQL development skills
• Experience with common data-science toolkits, especially python-based
• Hands-on experience with Kubernetes and Docker
• Experience with monitoring tools such as Prometheus/Grafana, Alert Manager, Alerta and OpsGenie
• Strong statistical analysis skills
• Demonstrated ability to troubleshoot and conduct root-cause analysis
• Unix scripting experience (bash, python, etc.)
• #LI-DNP
• The Base Salary range for the role is included below. Base salary is only one component of total compensation; all full-time, permanent positions are eligible for a discretionary bonus and benefits, including paid leave and insurance. Please visit Benefits - US | IMC Trading for more comprehensive information.

About Us