Quality Assurance Engineer

Overview

Test Readiness Reviews (TRR)

Functional Configuration Audits (FCA)

• AI & LLM Validation (LeverEDGE)

• Non-Deterministic Testing:
• Architect automated frameworks to evaluate Generative AI outputs (e.g., drafted CONOPS, technical requirements) for hallucination, consistency, and factual accuracy against "Gold Standard" datasets.
• RAG Evaluation:
• Implement automated metrics (e.g., RAGAS, faithfulness, answer relevance, etc.) to verify that the Retrieval-Augmented Generation pipeline is accurately citing internal technical documentation and regulatory texts.
• Prompt Regression:
• Design regression suites that monitor "prompt drift," ensuring that changes to the underlying model or system instructions do not degrade the quality of AI-generated engineering documents.

• ERP Integration Testing

• Build robust integration tests to validate data consistency between AI agents and critical enterprise systems (e.g., SAP S/4HANA, Ariba, etc.), ensuring no corruption of Bill of Materials (BOM) or financial data.
• Performance Benchmarking:
• Design performance tests to validate the latency and throughput of forecasting models and risk scoring engines, ensuring they meet the real-time requirements of supply chain dashboards.
• API Validation:
• Automate the testing of secure API gateways, verifying that Role-Based Access Control (RBAC) and PII redaction logic are functioning correctly before data reaches the AI models.

• Governance & Traceability

• V-Model Alignment:
• Map automated test cases directly to "System Requirements" and "User Needs," creating the digital evidence required for formal

Verification and Validation (V&V)

• Stage Gate Compliance

• Defect Lifecycle Management

Technical Requirements

• Core Automation:
• Expert proficiency in

Python

• Core Performance Testing

• Expert proficiency in crafting Performance Test Plans + Implementations (e.g., Locust, Jmeter, K6, etc.)
• AI Evaluation:

Experience

• utilizing frameworks for evaluating LLMs (e.g., DeepEval, TruLens, or custom Python evaluators).
• Understanding of "Ground Truth" dataset creation and management.
• Data Validation:
• Proficiency with SQL and data validation tools (e.g., Great Expectations) to verify data quality within Data Lakehouses and Vector Databases.
• CI/CD Integration:
• Strong experience integrating automated tests into GitLab CI/CD pipelines, enforcing "Quality Gates" that prevent non-compliant code or models from merging.
• Traceability Tools:
• Familiarity with requirements management tools (e.g., Jira, Linear, Jama, Polarion, etc.) and how to link automated test results to specific requirement IDs.

• Test Management/Reporting Tools

• : Strong hands-on on Managing Test Reports + Artifacts (e.g., TestRail, Allure, etc.)
• Version Control
• : String knowledge maintaining code based frameworks (e.g., Git, Gitlab, etc.)
• Quality Engineering practices
• : Strong hands-on knowledge on modern Quality Engineering best practices designated for fast-paced development environments (e.g., Shift Left Approaches, Test Pyramid, Mono-repo architecture for automation projects)