Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Open Innovation AI is seeking a Senior Data Engineer to contribute to the development of their MLOps platform and deploy Large Language Models. The role involves designing scalable data systems, integrating new technologies, and collaborating with engineering teams.
Key Skills for This Role
Responsibilities
- Design, construct, install, test, and maintain scalable data management systems
- Actively contribute to the development and enhancement of our MLOps platform
- Work on deploying and managing Large Language Models, ensuring efficiency and scalability
- Develop data set processes for data modeling, mining, and production
- Integrate new data management technologies and software engineering tools into existing structures
- Collaborate with the engineering team and other stakeholders in automating the deployment of machine learning models
- Recommend improvements to data reliability, efficiency, and quality
- Research industry trends and apply this knowledge to data infrastructure projects
- Utilize data to identify tasks that can be automated and streamline processes
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or related field
- Hands on experience with Apache Airflow and ClickHouse
- Proficiency in programming languages such as Python, Java, C++, or Scala
- Proven experience as a Data Engineer with focus on MLOps platforms and big data technologies
- Experience in deploying and managing Large Language Models (LLMs)
- Strong proficiency in SQL and NoSQL databases, data warehousing solutions
- Skilled in data pipeline, data modeling, ETL and workflow management tools
- Knowledge of cloud services (AWS, Azure, GCP) and their application in data engineering
- Excellent problem solving skills and ability to work in a collaborative environment
- Strong project management and organizational skills
Full Job Posting
Company Description
- Open Innovation AI is a global technology company specializing in advanced solutions for managing AI workloads.
- Its flagship product, the Open Innovation Cluster Manager (OICM), orchestrates complex AI tasks efficiently across diverse infrastructures.
- The platform is hardware agnostic, optimized for various GPUs and accelerators, and facilitates seamless integration and scalability for enterprise AI applications.
Role Overview
- Open Innovation AI is seeking an experienced Senior Data Engineer to contribute to the development of our MLOps platform and to work on deploying Large Language Models (LLMs).
- This role involves both maintaining scalable data infrastructure and actively participating in the enhancement of our machine learning operations.
Role Requirements
- Design, construct, install, test, and maintain scalable data management systems
- Actively contribute to the development and enhancement of our MLOps platform
- Work on deploying and managing Large Language Models, ensuring efficiency and scalability
- Develop data set processes for data modeling, mining, and production
- Integrate new data management technologies and software engineering tools into existing structures
- Collaborate with the engineering team and other stakeholders in automating the deployment of machine learning models
- Recommend improvements to data reliability, efficiency, and quality
- Research industry trends and apply this knowledge to data infrastructure projects
- Utilize data to identify tasks that can be automated and streamline processes
Required Qualifications
- Bachelor’s or master’s degree in computer science, Engineering, or a related field
- Hands on experience with Apache Airflow and ClickHouse (database), along with proficiency in programming languages such as Python, Java, C++, or Scala
- Proven experience as a Data Engineer with a focus on MLOps platforms and big data technologies
- Experience in deploying and managing Large Language Models (LLMs)
- Strong proficiency in SQL and NoSQL databases, data warehousing solutions
- Skilled in data pipeline, data modeling, ETL and workflow management tools
- Knowledge of cloud services (AWS, Azure, GCP) and their application in data engineering
- Excellent problem solving skills and ability to work in a collaborative environment
- Strong project management and organizational skills
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Open Innovation AI
Lead Delivery Engineer
Abu Dhabi Emirate, UAE
Open Innovation AI seeks a Lead Delivery Engineer to serve as the senior technical authority for customer deployments of AI infrastructure. You will lead on-site deployments end-to-end, coordinating contractors and engin
Senior Software Engineer
Abu Dhabi Emirate, UAE
Open Innovation AI is looking for a Senior Software Engineer – ML Infrastructure to design and build services for training and fine-tuning large neural networks. The role involves developing tools for managing datasets,
Service Desk Analyst
Abu Dhabi Emirate, UAE
Open Innovation AI is seeking a Service Desk Analyst in Abu Dhabi to provide first-line support for AI workload management platform. The role involves triaging tickets, guiding customers, and coordinating with L2/L3 team
Senior Frontend Software Engineer
Abu Dhabi Emirate, UAE
Open Innovation AI is seeking a Senior Frontend Engineer to join its on-site product engineering team in Abu Dhabi. You will build and ship features using TypeScript and React, shape the design system, and contribute to
Senior Architect - AI Infrastructure
Abu Dhabi Emirate, UAE
Open Innovation AI seeks a Senior Architect to define infrastructure reference architectures, sizing models, and BOQs for AI workloads. The role provides architectural guidance across pre-sales and internal engineering,
Senior Software Development Engineer in Test
Abu Dhabi, UAE
Open Innovation AI is hiring a Senior SDET in Abu Dhabi to lead quality engineering for AI workload management solutions. The role involves designing automation frameworks, API testing, UI automation with Playwright, and
Lead Graphic Designer
Abu Dhabi Emirate, UAE
Open Innovation AI is looking for a Lead Graphic Designer to own and elevate the visual identity of the brand across all channels. This hands-on role involves managing all design needs including website, presentations, b
Project Manager – AI Infrastructure & Platforms
Abu Dhabi Emirate, UAE
Company Overview Open Innovation AI is a global technology company that specializes in developing advanced solutions for managing AI workloads. Its flagship product, the Open Innovation Cluster Manager (OICM), orchestrat
Lead Delivery Engineer
Abu Dhabi Emirate, UAE
Senior Software Engineer
Abu Dhabi Emirate, UAE
Service Desk Analyst
Abu Dhabi Emirate, UAE
Senior Frontend Software Engineer
Abu Dhabi Emirate, UAE
Senior Architect - AI Infrastructure
Abu Dhabi Emirate, UAE
Senior Software Development Engineer in Test
Abu Dhabi, UAE
Lead Graphic Designer
Abu Dhabi Emirate, UAE
Project Manager – AI Infrastructure & Platforms
Abu Dhabi Emirate, UAE