Sr. Responsible AI Analyst

Responsibilities

1. Execute Responsible AI Evaluations:

Conduct structured assessments on fairness, safety, robustness, explainability, and hallucination risks for models under development or deployment.

2. Design Evaluation Datasets and Metrics:

Create curated, adversarial, and edge-case datasets and define quantitative metrics for evaluating fairness, toxicity, reliability, and ethical alignment.

3. Support Guardrail and Control Implementation:

Contribute to implementing technical guardrails, safety filters, explainability modules, and automated checks within AI pipelines.

4. Analyze Ethical and Technical Risks:

Review datasets, model outputs, and system behavior to identify fairness gaps, robustness issues, transparency deficiencies, and other Responsible AI risks.

5. Participate in Red Teaming and Stress Testing:

Assist with scenario-based adversarial evaluations, prompt safety checks, robustness tests, and model vulnerability analysis.

6. Support Deployment of Responsible AI Workflows:

Assist in implementing lifecycle governance workflows, templates, and processes—such as via IBM OpenPages or equivalent governance tooling.

7. Prepare Transparency and Governance Documentation:

Develop model cards, system cards, evaluation reports, risk logs, and supporting documentation required for governance reviews and audit readiness.

8. Assist in Continuous Monitoring:

Support creation of dashboards, metrics, and monitoring signals to track fairness drift, hallucination patterns, model instability, and safety deviations.

9. Collaborate Across Engineering and Governance Functions:

Work closely with AI engineers, data scientists, product teams, legal, ISG, DPO, and governance bodies to ensure Responsible AI requirements are consistently applied.

10. Assist in Training and Knowledge Enablement:

Help develop training content, guides, and resources to educate internal teams on Responsible AI evaluation methods, guardrails, and governance expectations.

Technical and Professional Requirements

• Proficiency in Python and ML/DL frameworks (PyTorch, TensorFlow) for evaluation and experimentation.
• Understanding of fairness libraries (Fairlearn, AIF360) and ability to compute ethics related evaluation metrics.
• Familiarity with explainability tools (SHAP, LIME, Captum, Integrated Gradients).
• Exposure to red teaming concepts, prompt safety evaluation, and data integrity checks.
• Experience with MLOps basics including evaluation pipelines, experiment tracking, and CI workflows.
• Understanding of ML algorithms, generative models, and supervised/unsupervised learning techniques.
• Familiarity with NLP, vision, speech, and structured data domains.
• Knowledge of datasets, benchmark suites, and third party model ecosystems.

Preferred Skills

• retrieval augmented generation (rag)
• Responsible AI

Educational Requirements

Bachelor of Engineering