News Report Technology
November 02, 2023

Singapore’s IMDA Partners with AI Verify Foundation to Launch Generative AI Evaluation Sandbox

In Brief

Infocomm Media Development Authority of Singapore partnered with AI Verify Foundation to unveil generative AI evaluation sandbox.

In a collaborative effort, Infocomm Media Development Authority of Singapore (IMDA), in partnership with the AI Verify Foundation unveiled the Generative AI Evaluation Sandbox.

The initiative aims to serve as a platform for assessing trusted artificial intelligence (AI) products and unveiling potential gaps in their performance.

The Sandbox leverages a recently introduced Evaluation Catalogue, providing a comprehensive framework with standardized methods and recommendations for the evaluation of generative AI products. Guided by this catalogue, which assembles widely utilized technical testing tools, the initiative organizes them based on their testing objectives and methodologies.

Furthermore, it recommends a fundamental suite of tests, ultimately enhancing the evaluation process of AI products.

In an era characterized by the rapid evolution of AI technologies, ensuring the trustworthiness of these systems has emerged as a paramount concern. The Gen AI Evaluation Sandbox is set to bridge the gap between the realm of theoretical trustworthiness and the practical evaluation of AI systems.

At its core, this groundbreaking initiative is designed to create a common standard approach for assessing Generative AI (Gen AI).

Gen AI refers to the sophisticated AI models known as Large Language Models (LLMs), such as GPT-3, which have demonstrated the potential for both creativity and controversy, making standardized evaluation more critical than ever.

Addressing AI Risks and Harms

The foundation for this initiative was laid in a discussion paper titled “Generative AI: Implications for Trust and Governance”. The paper identified key risks and harms associated with Large Language Models. In response to these concerns, IMDA and the AI Verify Foundation have launched the Gen AI Evaluation Sandbox.

The Sandbox seeks to foster a collaborative ecosystem for evaluating AI products by engaging a wide array of stakeholders. IMDA has extended an open invitation to industry partners to join forces and contribute to the development of evaluation tools and capabilities within the Sandbox. This approach ensures that the responsibility of evaluating AI extends beyond just model developers to include application developers and third-party testers.

It will provide a standardized language for Gen AI evaluation through its Evaluation Catalogue. This Catalogue categorizes existing evaluation benchmarks and methods while recommending a baseline set of evaluation tests for Gen AI products. It provides a structured framework to assess the capabilities and limitations of these AI models.

The launch of the Gen AI Evaluation Sandbox has already seen key industry players joining the cause. Notable participants include technology giants like Google, Microsoft, Anthropic, IBM, NVIDIA, Stability.AI and Amazon Web Services (AWS). Third-party testers including Resaro.AI, Deloitte, EY, and TÜV SÜD will also lending their expertise.

Further, it aims to involve regulators such as the Singapore Personal Data Protection Commission (PDPC), to maintain transparency and compliance at all stages of AI development and deployment.

AWS’ country manager, worldwide public sector, Elsie Tan, said, “The responsible use of generative AI technologies will transform entire industries and reimagine how work gets done. We look forward to being a part of IMDA’s Generative AI Evaluation Sandbox to provide businesses with the tools and guidance needed to build artificial intelligence and machine learning applications responsibly.”

Opportunities for Participation

Singapore has taken significant strides in the responsible AI domain, exemplified by the introduction of the AI Verify Foundation. The Gen AI Evaluation Sandbox marks the next phase of this journey, leveraging global contributions and open-source community support.

The AI Verify Foundation and IMDA have extended an invitation to model and app developers, as well as third-party testers, to participate in the Gen AI Evaluation Sandbox. This is a unique opportunity for organizations and individuals to contribute to the development of a more robust testing environment for AI models.

The global collaboration fostered by this Sandbox represents a significant step towards a standardized and transparent approach to evaluating Gen AI.

Disclaimer

In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.

About The Author

Kumar is an experienced Tech Journalist with a specialization in the dynamic intersections of AI/ML, marketing technology, and emerging fields such as crypto, blockchain, and NFTs. With over 3 years of experience in the industry, Kumar has established a proven track record in crafting compelling narratives, conducting insightful interviews, and delivering comprehensive insights. Kumar's expertise lies in producing high-impact content, including articles, reports, and research publications for prominent industry platforms. With a unique skill set that combines technical knowledge and storytelling, Kumar excels at communicating complex technological concepts to diverse audiences in a clear and engaging manner.

More articles
Kumar Gandharv
Kumar Gandharv

Kumar is an experienced Tech Journalist with a specialization in the dynamic intersections of AI/ML, marketing technology, and emerging fields such as crypto, blockchain, and NFTs. With over 3 years of experience in the industry, Kumar has established a proven track record in crafting compelling narratives, conducting insightful interviews, and delivering comprehensive insights. Kumar's expertise lies in producing high-impact content, including articles, reports, and research publications for prominent industry platforms. With a unique skill set that combines technical knowledge and storytelling, Kumar excels at communicating complex technological concepts to diverse audiences in a clear and engaging manner.

From Ripple to The Big Green DAO: How Cryptocurrency Projects Contribute to Charity

Let's explore initiatives harnessing the potential of digital currencies for charitable causes.

Know More

AlphaFold 3, Med-Gemini, and others: The Way AI Transforms Healthcare in 2024

AI manifests in various ways in healthcare, from uncovering new genetic correlations to empowering robotic surgical systems ...

Know More
Join Our Innovative Tech Community
Read More
Read more
Biswap Releases New Strategic Roadmap, Focuses On Multi-Chain Expansion And Introduction Of Liquid Staking
Markets News Report Technology
Biswap Releases New Strategic Roadmap, Focuses On Multi-Chain Expansion And Introduction Of Liquid Staking
May 24, 2024
Web3 AI Unveiled: Jimmy Zhao Unpacks How BNB Chain’s Integration of Blockchain and AI Redefines Trust, Transparency, and Decentralization
Interview Business Markets Software Technology
Web3 AI Unveiled: Jimmy Zhao Unpacks How BNB Chain’s Integration of Blockchain and AI Redefines Trust, Transparency, and Decentralization
May 24, 2024
Hyperliquid Unveils HIP-1 Spot Token Deployment Function On Its Mainnet, And Recommends Testing For Smooth Deployment
News Report Technology
Hyperliquid Unveils HIP-1 Spot Token Deployment Function On Its Mainnet, And Recommends Testing For Smooth Deployment
May 24, 2024
This Week’s Top Deals, Major Investments in AI, IT, Web3, and Crypto (20-24.05)
Digest Top Lists Business Lifestyle Markets Software Technology
This Week’s Top Deals, Major Investments in AI, IT, Web3, and Crypto (20-24.05)
May 24, 2024