Engineer-Quality Assurance

The Opportunity

Key Responsibilities

• As an Engineer – Quality Assurance, you will evaluate and assess application potential by testing the compatibility of new programs with existing systems and ensure adherence to application objectives, specifications, and QA standards—including the quality and safety of AI-powered features.
• ***Functional***
• Develop application test plans and test cases based on software requirements and technical specifications shared by the Product, Solution, and Technology teams.
• Conduct quality assurance and quality control tests for new and existing application solutions developed by the Product Department.
• Conduct system stress testing, performance testing, functional testing, and scalability testing (as relevant) for all new products, solutions, and modifications.
• Test AI-powered features including LLM responses, RAG pipelines, AI agents, and agent workflows; evaluate accuracy, relevance, faithfulness, hallucination rate, latency, and edge-case behavior.
• Define and apply evaluation strategies, metrics, and golden datasets for Generative AI features, using frameworks and tools such as LangSmith, Ragas, DeepEval, Promptfoo, or similar.
• Leverage AI-assisted testing tools and AI coding assistants (e.g., GitHub Copilot, Claude, Cursor) to accelerate test case generation, test automation script development, synthetic test data creation, defect triage, and documentation.
• Apply AI-driven and self-healing test automation approaches to improve test coverage, reduce flakiness, and shorten regression cycles.
• Assess application security controls and ensure adherence to data integrity standards, including responsible-AI considerations such as prompt injection, data leakage, bias, and unsafe outputs.
• Improve existing application performance by reviewing project objectives, testing the viability of recommended changes, and proposing necessary modifications.
• Evaluate and test new software and AI solutions to ensure adherence to business objectives and compatibility with existing systems.
• Maintain a record of all changes and modifications made across applications and develop bug reports for the reference of end-users and support teams.
• Prepare reference documents and guides for users, clearly outlining operating instructions and procedures.
• Comply with QHSE (Quality Health Safety and Environment), Business Continuity, Information Security, Privacy, Risk, Compliance Management, and Governance policies, procedures, plans, and related risk assessments.

Requirements

• Bachelor’s Degree in Artificial Intelligence, Computer Science, Business Analytics, Engineering, or a related field.
• Minimum 5 years of experience in planning, designing, and managing software quality assurance and testing.
• At least 3 years of experience in testing in a product development organization or analytics industry.
• Hands-on experience using AI-assisted testing tools and AI coding assistants in QA workflows for test generation, automation, data synthesis, and defect analysis.
• Experience testing AI-powered or GenAI features—including LLM responses, RAG pipelines, prompts, or AI agents—and familiarity with LLM/AI evaluation frameworks such as LangSmith, Ragas, DeepEval, Promptfoo, or similar.
• Web application testing experience with popular tools and frameworks such as Postman, SoapUI, Robot Framework, Selenium, or Cypress.
• Experience with test management tools such as JIRA, Confluence, TestRail, or QC.
• At least 3 years of experience working in Agile development methodologies, preferably in an AI service organization or analytics industry.
• Programming ability is a plus: Java, Python, or JavaScript.
• Big data and AI product experience is a plus: data quality, data mapping, data cleaning, SQL jobs, Spark, Elasticsearch (1+ year of experience in any of these topics is acceptable).
• Tech-ops knowledge is a plus: Kubernetes, Docker, shell scripting, and cloud services.
• Ideally, you will also have
• In-depth market and domain knowledge.
• A passion for constant improvement.
• An innovative and creative approach to problem-solving.
• Excellent communication skills.