What are the similarities between AI voice agents and autonomous vehicles? According to Brooke Hopkins, a former technology lead at Waymo, they can be assessed using similar evaluation methods. Her new venture, Coval, aims to implement this approach.
Upon leaving Waymo, Hopkins realised that many challenges encountered there mirrored those facing the wider AI industry. She shared that rather than viewing this situation as an entirely new paradigm necessitating fresh testing methods, it should be recognised that extensive experience already existed from a decade of developing self-driving technology.
In 2024, she launched Coval, a platform designed to create simulations for evaluating the performance of AI voice and chat agents. Coval mimics the methods used for testing self-driving vehicles at Waymo by conducting thousands of simultaneous simulations. These include scenarios like scheduling a restaurant reservation or responding to a customer inquiry presented indirectly.
Coval’s technology assesses agents based on a comprehensive range of metrics while allowing companies to tailor evaluations based on specific requirements. This enables ongoing monitoring for regressions. Furthermore, users can utilise the gathered data and insights to demonstrate to their clients that the agents function as intended, whether for a live demonstration or as a monitoring tool.
A significant obstacle to the implementation of AI agents within businesses is the lack of confidence in their capabilities, according to Hopkins. Choosing between different vendors is a complex decision for executives, who often struggle to understand what questions to ask or how to validate that these agents meet expectations. Coval provides companies with a solution to effectively showcase their agents’ performance.
The concept for Coval began to take shape during the Y Combinator Summer 2024 batch prior to its public launch in October 2024. Hopkins noted that demand for the platform has surged, with many customers eager to expedite the evaluation of their agents.
The startup based in San Francisco has recently announced a $3.3 million seed funding round led by MaC Venture Capital, with contributions from Y Combinator and General Catalyst. The funds will be used to expand the engineering team and refine product-market fit. Additionally, Hopkins indicated that plans are underway to enable users to assess other forms of AI agents, including web-based ones, in the future.
As interest and excitement surrounding AI agents reach unprecedented levels, prominent enterprise tech leaders like Marc Benioff have praised and promoted these innovations, stating that Salesforce plans to deploy over a billion AI agents by next year. OpenAI is also anticipated to unveil its own AI agent soon.
Numerous startups are also making strides in this domain. Over 100 startups focused on AI agents have emerged from Y Combinator’s three 2024 cohorts alone. Some of these startups have secured substantial venture funding; for example, /dev/agents raised a $55 million seed round at a valuation of $500 million in November 2024, just months after its inception.
This surge in activity implies that more companies are likely to seek assistance in evaluating their AI agents. According to Hopkins, Coval is well-positioned to differentiate itself from new entrants in the market due to its established experience and advantage.
With over half a decade of experience in the field, Hopkins believes Coval stands apart. By applying insights gained from previous projects and iterations, the team has created robust systems capable of scaling effectively while learning from past failures.