Quality Assurance Engineer, H2O.ai

• 2-4 years of QA experience with a strong mix of automation and hands-on manual/exploratory testing
• Decent Python skills and experience building maintainable test frameworks from scratch
• Real-world experience testing modern React/TypeScript web applications and writing bulletproof Playwright or Selenium tests
• Hands-on Kubernetes experience in a testing or test-environment context (kubectl, Helm, writing manifests, debugging pods)
• Proven ability to use generative AI tools daily to accelerate debugging, test-data creation, and log analysis (you’ve already used h2o.ai's product portfolio, ChatGPT, Claude, or similar in your current QA workflow)
• Comfort reading and triaging complex logs from LLM frameworks, vector DBs, and tracing systems
• Solid grasp of CI/CD (GitHub Actions preferred) and infrastructure-as-code concepts

Desirable

• You will be a core QA engineer responsible for the end-to-end quality and reliability of h2o.ai's product portfolio
• Design, build, and maintain Python-first automation frameworks (pytest + Playwright + async) for UI, API, and end-to-end workflows of h2o.ai's product portfolio
• Perform deep exploratory and manual testing of the h2o.ai's product portfolio web UI (chat interfaces, document upload/processing, agent builder, maker suite, evaluation hub, enterprise admin console)
• Use h2o.ai's product portfolio itself (and other GenAI tools) in creative ways to:
• Generate realistic test documents, datasets, and edge-case prompts
• Auto-generate or refine test cases via prompt engineering
• Rapidly summarize and debug massive, complex log files (e.g., Kubernetes pods, etc.)
• Explain cryptic LLM traceback chains or hallucination root causes in seconds instead of hours
• Root-cause of difficult, intermittent failures in distributed RAG/LLM systems by combining traditional log analysis with GenAI-assisted debugging
• Create and execute chaos experiments targeting LLM routing, vector database latency, GPU OOM, retrieval failures, and token-limit edge cases
• Build and manage ephemeral h2o.ai's product portfolio clusters on Kubernetes for testing (Helm, custom operators)
• Own UI regression suites (Playwright) and accessibility testing
• Write reproducible, high-quality bug reports that developers love and regularly verify fixes across the full stack
• Collaborate closely with the h2o.ai's product portfolio feature teams in an Agile environment and influence testability from the design phase