Responsible AI Data Scientist, Salesforce

• The ideal candidate will have technical experience in generative as well as predictive artificial intelligence and in responsible / ethical AI
• 3-5 years of industry experience in Software Engineering, AI ethics, AI research, Applied research, ML, DS, or similar roles
• Demonstrated ability to think adversarially, ability to anticipate how malicious actors might misuse AI systems and develop corresponding test scenarios
• Experience creating heuristic-based detection logic and rules for identifying anomalous or suspicious activity in production systems and networks (e.g. log analysis, user behavior analytics)
• Experience with problem-solving and troubleshooting sophisticated issues with an emphasis on root-cause analysis
• Experience in analyzing sophisticated, large-scale data sets and communicating findings to technical and non-technical audiences
• Proven organizational and execution skills within a fast-paced, multi-stakeholder environment
• Experience working in a technical environment with a broad, cross-functional team to get results, define requirements, coordinate assets from other groups (design, legal, etc.), and deliver key achievements
• Excellent written and oral communication skills, as well as social skills, including the ability to articulate technical concepts to both technical and non-technical audiences
• Works well under pressure, and is comfortable working in a fast-paced, ever-changing environment
• Experience using SQL and relational databases. Ability to use Python, R, or other scripting languages to perform data analysis at scale
• A related technical degree required

Desirable

• Provide technical leadership in designing, prototyping, and implementing comprehensive adversarial testing strategies, including both automated and manual adversarial testing approaches
• Mentor and guide collaborator teams on adversarial testing standard processes, helping them develop the skills to conduct their own testing effectively
• Collaborate with cross-functional teams to integrate OEHU adversarial testing frameworks into the AI development lifecycle
• Contribute to the development of detection models, safety guardrails, and other proactive measures to prevent and mitigate risks posed by bad actors
• Research and implement innovative techniques for enhancing AI safety and robustness, drawing from both open-source and internal tools
• Collaborate with Salesforce’s AI Research team on novel approaches to model safety
• Write clean, efficient, and well-documented code (primarily in Python) to support research efforts and facilitate the evaluation of AI systems
• Develop and maintain a repository of reusable code modules and libraries to streamline adversarial testing processes
• Participate in scoping, documenting, and implementing tests with partner teams, including the implementation of mitigations identified during testing
• Test for technical vulnerabilities, model vulnerabilities, and harm/abuse including but not limited to bias, toxicity, and inaccuracy
• Participate in labeling test data in partnership with OEHU and partner teams
• Write reports covering the goals and outcomes of testing operations, including significant observations and recommendations
• Continuously monitor and analyze emerging threats and vulnerabilities to inform the development of adaptive safety measures
• Continue to grow expertise in model safety by keeping up with research in socio-technical systems, privacy, interpretability/explainability, robustness, alignment, and responsible AI