AI Trust and Governance Architect

Responsibilities

Key Responsibilities

AI Assurance Architecture

• Architect platforms and frameworks for AI assurance, evaluation, and benchmarking
• Design systems for LLM, agent, and RAG evaluation across functional, non functional, and risk dimensions
• Define architectural patterns for Responsible AI, bias detection, explainability, and safety validation
• Build reusable assurance components supporting Business Assurance, Risk Assurance, and Reliability

Security, Reliability & Governance

• Architect AI testing and validation for security, privacy, prompt injection, and adversarial robustness
• Integrate red teaming, threat simulation, and chaos style validation for AI systems
• Define governance mechanisms for model usage, auditability, traceability, and compliance
• Ensure AI systems meet enterprise standards for resilience, fault tolerance, and observability

Platform & Engineering Enablement

• Design AI assurance platforms supporting automated test execution, reporting, and insights
• Enable integration with CI/CD pipelines to enforce AI quality gates
• Collaborate with QE engineering teams to embed AI assurance into the SDLC
• Mentor teams on AI risk identification and mitigation from an engineering perspective

Core Platforms, Frameworks & Tooling

• LLM and AI evaluation frameworks (PromptFoo, DeepEval, custom LLM evaluation harnesses)
• Prompt, RAG, and agent validation tooling (prompt testing frameworks, retrieval accuracy validators, agent workflow evaluators)
• Responsible AI and model risk tooling (Fairlearn, SHAP, Explainable AI libraries, toxicity and bias scanners)
• Security and adversarial testing tools for AI systems (PyRIT, Garak)
• AI red teaming and threat simulation frameworks (automated red team scripts, adversarial test suites for LLMs and agents)
• AI assurance automation and QE frameworks (Galileo)
• Observability for AI behavior and drift (Langfuse, Arize, Evidently, custom telemetry dashboards)

Client Orientation & Leadership

• Partner with product and engineering teams to identify AI Assurance opportunities and shape roadmaps
• Support client workshops, RFPs, and solution presentations
• Mentor engineers on AI/ML/Gen AI best practices and emerging technologies
• Translate complex AI concepts into business-friendly narratives

Technical and Professional Requirements

Must Have Qualifications

• 13+ years of experience in software engineering with 3+ years in AI with strong architecture ownership
• Hands on expertise in AI/ML systems, LLM evaluation, and assurance frameworks
• Experience with AI red teaming, model risk management, or AI audit tooling
• Strong understanding of Responsible AI, AI risks, and governance principles
• Experience with security testing, adversarial testing, and reliability engineering
• Proficiency in Python, automation frameworks, and cloud platforms

Good to Have Skills

• Knowledge of regulatory or compliance considerations for AI systems
• Exposure to performance engineering, chaos engineering, or resilience testing for AI
• Contributions to internal platforms, frameworks, or standards

Preferred Skills

• Governance Risk and Compliance
• Audits
• Workflow
• Assurance
• traditional ai ml
• Conversational AI Platform
• retrieval augmented generation (rag)
• Prompt Engineering
• Responsible AI
• Data Governance

Educational Requirements

Bachelor of Engineering