Give your 'human-level agents' a proper head start with these 3 best practices

Computing is on the cusp of "nearly human-level agents," according to Mustafa Suleyman, Microsoft CEO of AI. However, businesses face significant challenges in implementing these agents, from redesigning workflows to determining what information agentic AI programs should access. A recent report by Databricks found that only 19% of organizations have deployed AI agents, and mostly to a limited extent.

To overcome these hurdles, Craig Wiley, head of AI for Databricks, recommends three best practices: governance, evaluation of model output, and cost consideration. "If you talk to a lot of chief financial officers, they will tell you, 'I have three concerns': Can you control it, can you tell me if it's any good [meaning, does what comes out of the model actually provide value], and how much does it cost?" Wiley explained. The first best practice is governance, which starts with controlling what data an agent will access.

This involves defining the question and identifying the resource that should have the answer. A data catalog can help enforce identities and track access to data, ensuring that sensitive information is not leaked. The second best practice is evaluating the output of the model.

This involves ongoing evaluation throughout the life of the program and at multiple levels. Companies that can evaluate the output of agents are six times more likely to get into production. The third concern, cost, is easier to address once governance and evaluation are in place.

Companies should consider starting small and building at a pace that allows agents to be governed and verified. Examples of successful deployments include 7-Eleven, which used agents to provide service techs with a "super assistant" that can access tons of documentation, and Baylor University, which uses agents to review recordings of calls with prospective students. While it's still early to have concrete figures for the industry's financial return on investment from agents, encouraging anecdotal examples include Franklin Templeton's automation of investment portfolio analysis, which enabled the firm to identify over $15 million in new product opportunities.