Jay Analytix
Montreal QA Lead - AI Systems Testing
Job Description
Shape the future of AI quality assurance as a QA Lead in Montreal, focusing on testing AI systems and models. This contract role incorporates prompt evaluation and bias analysis.
You will employ your extensive QA leadership experience to craft and implement testing strategies for complex AI products. Working with LLM technology, your hands-on skills in prompt engineering and understanding of RAG pipeline workflows will guide you in evaluating model integrity. Your role will involve continuous collaboration with various teams to uphold quality standards and metrics.
Key Responsibilities:
• Lead the testing of AI systems for quality assurance
• Develop applications' test harnesses using LangChain
• Validate the accuracy of RAG pipeline data
• Establish generative AI quality metrics
• Collaborate with teams to embed QA in development
Requirements:
• Previous experience in QA leadership with AI systems
• Strong knowledge of LLM principles
• Expertise in pro...
You will employ your extensive QA leadership experience to craft and implement testing strategies for complex AI products. Working with LLM technology, your hands-on skills in prompt engineering and understanding of RAG pipeline workflows will guide you in evaluating model integrity. Your role will involve continuous collaboration with various teams to uphold quality standards and metrics.
Key Responsibilities:
• Lead the testing of AI systems for quality assurance
• Develop applications' test harnesses using LangChain
• Validate the accuracy of RAG pipeline data
• Establish generative AI quality metrics
• Collaborate with teams to embed QA in development
Requirements:
• Previous experience in QA leadership with AI systems
• Strong knowledge of LLM principles
• Expertise in pro...