Responsible ML Data Science Engineer, Salesforce

• Practical experience in machine learning
• MS in a quantitative discipline with 8+ years of experience
• Fluent in building/prototyping machine learning models and algorithms and wrangling large datasets
• Knowledgeable about standard machine learning approaches (Regression, Cross-Validation, Boosting, Matrix-Factorization, Decision Trees, Clustering, CNNs, RNNs, Transformers, GANs)
• Proficient in using Python and common machine learning frameworks (e.g., TensorFlow, Pandas, PyTorch, SciPy, scikit-learn, JAX) to implement models and algorithms
• Proficient in SQL, shell scripting, and Unix/Linux command-line tools
• Experience with designing and building micro-services, familiar with Kubernetes/containerization/RESTful API/gRPC, etc
• Grasp of the evolving understanding of fairness and ability to meet both state-of-the-art and global standards for fairness evaluation, particularly in generative AI
• Experience working across teams of engineers, data scientists, and researchers
• Strong communication skills. Comfortable presenting ideas to diverse teams and individuals in multiple formats, from slide decks to informal chats
• Earns trust in relationships both internally and externally, and at all levels of the organization. Challenges the status quo to improve the productivity, effectiveness, and culture of a team - without burning bridges
• Ability to creatively prioritize, stage, and sequence solutions to challenging/complex problems

Desirable

• Salesforce's Office of Ethical and Humane Use is looking for an experienced ML data scientist with a focus on responsible AI
• Working with the Einstein product and engineering teams, AI Frontier, and Salesforce AI Research teams, they support the development of foundation and predictive models by ensuring they are built and deployed in alignment with OEHU’s guidelines and policies
• The ideal candidate is well-versed in assessing a variety of machine learning techniques to provide actionable insights on how to bring them into production
• They are enthusiastic about providing technical and scientific leadership to teams of scientists and engineers
• They build tools to ensure fair, accountable, transparent, and explainable AI. This individual will support our culture of being proactive, thoughtful, and knowledgeable about Salesforce’s AI products
• Partner with the Salesforce AI Research Science team, as well as distributed Data Science and Engineering teams, to measure and remediate potential bias in data and models
• Build datasets for testing models, both generative and predictive, in collaboration with partner teams
• Participate in technical discussions, and drive engineering investments in responsible AI to improve model fairness and safety prior to launch
• Create notebooks using SQL or Python, and other tools, to empower other members of OEHU with advanced analytics capabilities
• Compare between different tools and libraries options (both Salesforce-built and open-source) to identify the most appropriate one for the task at hand
• Develop quantitative metrics to help surface insights from structured datasets
• Plan and conduct end-to-end experiments and analyses, from gathering requirements through processing and modeling the data, to identifying emerging risks
• Monitor and analyze logs data for harms/ threats and insights
• Validate any synthetic data generation, crowdsource, or contractor jobs for labeling or data creation
• Participate in labeling test data in partnership with OEHU and partner teams
• Contribute to the creation and use of libraries and tools that perform testing, evaluation, and mitigation of content safety issues and algorithmic bias
• Collaborate with industry leaders in similar positions in peer organizations on ways to improve the state of responsible AI development
• Extract logs and data from various parts of the tech stack for analysis