Microsoft Unveils ASSERT

Microsoft's new tool, ASSERT, lets developers spin up AI behavior tests using text descriptions, simplifying the testing process for application-specific AI behavior. ASSERT takes plain-language descriptions of an AI model's expected behavior and policies, turns them into a structured set of acceptable and unacceptable behaviors, generates problem scenarios and test cases, runs them against the target system, and scores the results. For example, a developer could specify that a document research AI agent shouldn't send emails to people outside the company, limit confidential information to C-level executives, and provide concise summaries with prior context in mind. ASSERT will use those rules to generate test cases that check whether the system follows those rules on an ongoing basis. According to Sarah Bird, chief product officer of Responsible AI at Microsoft, evaluations are critical to making good decisions, and ASSERT fills a gap that broader, more general evaluations cannot when AI models are intended to behave in a manner that is shaped by an application or product's context, policies, and tools. The release comes amidst a gradual but broader shift in the AI industry, with researchers focusing on repeatable testing and regression checks. With ASSERT, developers can evaluate systems when they're being built, after deployment, and even for continuous monitoring, enabling more trustworthy AI systems.

Source: New Microsoft tool lets devs spin up AI behavior tests using text descriptions
Domain: techcrunch.com