Google’s ‘Project Jarvis’ project could let AI agents take over computers

Google is believed to be working on an AI agent tool capable of autonomously navigating the web to complete tasks for users, according to reports.

Dubbed ‘Project Jarvis’, the AI-powered tool is reportedly able to act autonomously across a variety of functions, including email management, conducting research, and scheduling appointments.

Reporting from The Information last month said sources believed Project Jarvis would be demonstrated as early as December, ahead of public release with the next major update to its large language model (LLM), Gemini.

AI agents, also referred to as agentic AI, are predicted to be the next big trend in generative AI, allowing companies to put large language models to use automating actual tasks in the workplace, rather than remaining limited to text generation, data sifting, or coding tools.

Google’s not the only big tech player building such agents. Microsoft has shown off agents that can reply to emails or automate customer requests, with similar efforts in the pipeline from OpenAI, Anthropic, and startups.

Google Cloud unveiled its Vertex AI agents earlier this year, using Gemini to help companies automate tasks such as answering employee questions about benefits or summarizing information about a security breach.

Salesforce, meanwhile, unveiled its Agentforce autonomous AI system in September, with CEO Marc Benioff hailing agents as the next evolutionary in the AI area and insisting customers had been “oversold” on the potential of AI.

“We read about it in the press, the $300 billion spent on AI, but where are the actual tangible customer values?” Benioff said at the time.

What is an AI agent?

At the moment, generative AI requires input from users — a prompt to create a swathe of text or generate an image, for example. Some use cases are more proactive, such as coding support that suggests code or actively spots potential flaws.

But an agent isn’t a chatbot locked into a system. Instead, it can go out in the digital world and complete tasks.

Reports suggest that’s exactly what Google is working to develop via Project Jarvis, which is expected to arrive in the next update to the Gemini models.

At the moment, someone using an LLM for sifting through data would have to supply the relevant dataset first, be it directly or perhaps with an API pulling in the right information. With a computer-using agent, that’s no longer the case.

Rather than trolling the web yourself for data, it will in theory be possible to send the agent online via the browser to gather up and organize useful information.

Beyond research and data gathering, such agents could — again, in theory — be able to click through any webpage just like a human, meaning it could be tasked with filling out forms, online purchases, and so on. In short, AI users could set up an agent with a to-do list of tasks and send it out via the browser to complete them.

Project Jarvis is focused on the browser — perhaps no surprise given Google’s dominance of the browser market via Chrome — but in the future the aim is for such agents to be able to control any software on a computer, rather than just a browser.

Rival plans

Of course, Google isn’t the only company looking to get AI completing actual tasks in the real — if digital — world.

Anthropic last month unveiled an early version of agents in an upgraded version of its Claude models, saying it would be able to click through websites and fill out forms, though the company admitted the system “is still experimental — at times cumbersome and error-prone”.

And Microsoft showed off a system for making autonomous agents in its Copilot Studio, with partners testing the ability to automate client onboarding, process orders, and otherwise take over routine admin tasks. A public preview is set to arrive soon.

OpenAI has been talking about the idea of agents all year, with reports in February suggesting the company was working on a pair of AI agents to manage tasks online and in apps.

The AI developer last year unveiled an interface to help developers make their own agents, but reports quote OpenAI saying its models at the time were too limited, though it’s since upgraded its models. At a recent developer day, the company said it hoped to release agents next year, demonstrating a voice assistant agent calling on the phone to order strawberries.

OpenAI predicted that agents or assistants powered by AI will “hit the mainstream” next year — given the launches lined up, that seems an easy bet.