Big Data - Senior Engineer

About Iris Software

Job description

Mandatory skills

• Mandatory skills include Sqoop, Hadoop HDFS and YARN, Azure Databricks, HBase, Impala, Hive, Scala, and Apache Hudi.
• Mandatory competencies also reference Big Data, Hadoop, Sqoop, PySpark, Hive, Scala, Apache Hudi, and Cloudera.
• PySpark is listed as an additional skill in the job description.

Key responsibilities

• Design scalable Big Data solutions and define data processing architectures and engineering standards.
• Build distributed Spark and Hadoop pipelines and optimize SQL, Hive, Databricks, and large-scale processing workloads.
• Design Lakehouse solutions with Apache Hudi or Apache Iceberg and establish ingestion, validation, and reconciliation frameworks.
• Drive performance tuning, review engineering solutions, and troubleshoot complex data and platform issues.
• Mentor engineers, collaborate with stakeholders, and improve scalability, reliability, and operational efficiency.

Benefits

• Iris describes benefits supporting financial, health, well-being, and professional development needs.