Preparing For A World Of AI Agents: From Automation to Autonomy
The Three Questions AI Leaders Need To Ask Before Exploring AI Agents In Business
Throughout all the OpenAI media coverage these last few weeks, one article in The Atlantic1 has struck me as particularly insightful. It includes excerpts from an interview this summer with Ilya Sutskever, OpenAI’s Chief Scientist:
“The way I think about the AI of the future is not as someone as smart as you or as smart as me, but as an automated organization that does science and engineering and development and manufacturing.”
Let that sink in for a minute. This vision coupled with the ability for anyone to create their early AI-powered agents (OpenAI’s GPTs) today make it the perfect timing to discuss the implications for business and how AI leaders can prepare. But let’s back up for a second…
From Automation To Autonomy
Humans have historically performed tasks in a business themselves: forecasting demand, submitting orders, closing the books, etc. With the availability of business software since the 1960s, the level of automation and autonomy performing these tasks has been increasing:
Limited automation: Rule-based software partially automates these tasks.
Advanced automation: Rule-based software robots (Robotic Process Automation) automate repeatable, manual tasks of users interacting with software.
Limited autonomy: Machine learning based software automates repeatable steps based on patterns in the underlying data.
Advanced autonomy: Generative AI-based agents execute tasks on behalf of a user based on a defined objective.
What makes agents unique is their ability to autonomously determine how accomplish a user-defined objective, for example by calling other agents or services (via APIs). This is the foundation for autonomous businesses.
A Future With AI Agents
OpenAI’s GPTs are an early version of agents that achieve goals based on a user’s prompt and additional data that users can upload. From handling the most common customer service inquiries to chatting with your earnings transcript, GPTs are a good indicator of what’s to come as agent capabilities become more mature.
Imagine creating a procurement agent to identify the best supplier of a material. After providing your procurement policy, past contracts, and company objectives to the agent, it narrows down the list of suppliers. Take it further and have it interact with your supplier by creating the correspondence, requesting and negotiating price information, and asking you to confirm the order. And, further into the future, even automate that last step for you, too. Along this process, multiple agents could be involved in accomplishing the goal of sourcing materials based on decision points: from selecting a supplier to drafting an e-mail, negotiating terms, and proposing alternatives.
While this sounds intriguing, the real world of business with its exceptions and complexities will be the limiting factor for agents.
New Challenges Of AI Agents
Agents present a huge opportunity for businesses. But delegating more agency to applications also presents several risks:
Security: Agents calling other agents will operate in a distributed environment. Like today, the information security departments will want to know that the software being used is secure. Agents will identify themselves which other agents are needed to accomplish an objective. Embedding control into the system design over which additional agents can be called will be key to ensuring information stays secure.
Fairness: Agents are based on the same black box models known to exhibit biases. Delegating agency to applications to parallelize and scale decision-making has the potential to further exacerbate the current challenges of responsible AI systems. System prompts need to be enhanced and additional checks need to be built into applications to minimize and identify biases in the generated output.
Data privacy: Agents and their underlying models will process personal and proprietary information. What data can be processed and whether data submitted in a prompt will be used for the vendor’s next model re-training will also remain at the forefront. Anonymization, filtering, granular policies, and other techniques will be needed to address this going forward.
User manipulation: Based on a recent research paper2, agents might execute illegal activities (misalignment) such as insider trading while optimizing for the objective they’ve been given. Furthermore, they might deceive their users about their actions (conditional strategic deception).
Model evaluation for insider trading simulation (Scheurer et al., 2023). Adding more explicit information in the system prompt (from strongly encouraging to discouraging illegal behavior) influences the LLM’s output. But even strongly discouraging the LLM from engaging in illegal activities does not completely eliminate this behavior. Therefore, additional experiments, safeguards, and mitigations will be necessary.
System prompt impact on LLM behavior (Scheurer et al., 2023)
AI leaders exploring agents need to think about three questions today:
Fairness & manipulation: What variable are we optimizing, and what other variables could be impacted?
Data privacy & security: What data can be used to develop and maintain an agent, and how to protect it?
Agency & governance: What decisions will require human review and approval?
Are we ready for a world of AI agents yet?
Explore related articles
Become an AI Leader
Join my bi-weekly live stream and podcast for leaders and hands-on practitioners. Each episode features a different guest who shares their AI journey and actionable insights. Learn from your peers how you can lead artificial intelligence, generative AI & automation in business with confidence.
Join us live
December 05 - Mark Stouse, CEO Proof Analytics, will discuss how managers can teach their data scientists about the business.
December 14 - Enrico Santus, Human Computation Leader, will share how you can design adaptive processes for human-AI collaboration.
January 09 - “What’s the BUZZ?” will be back for the 2024 kick-off. Stay tuned for the line-up of guests in January!
Watch the latest episodes or listen to the podcast
Follow me on LinkedIn for daily posts about how you can lead AI in business with confidence. Activate notifications (🔔) and never miss an update.
Together, let’s turn HYPE into OUTCOME. 👍🏻
—Andreas
The Atlantic, OpenAI’s Chief Scientist Made a Tragic Miscalculation. Published: 21 November 2023. Last accessed: 02 December 2023.
Scheurer et al. (2023). Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure. Last accessed: 03 December 2023.