- Microsoft is seriously expanding the powers of the Copilot app, for all Windows 11 PCs
- Copilot Voice will provide a new method to activate AI with voice input, supported by a global rollout of Copilot Vision.
- Broader AI powers are coming, including co-pilot actions that allow AI to interact with files on your PC.
Microsoft teased that something big was happening today and now we’ve heard all about its plans to make voice interaction a bigger part of the Windows 11 experience and how AI functionality will be boosted in the operating system.
The expectation was that this announcement would be AI-focused, and indeed it is, but the twist is that this isn’t just for Copilot+ PCs. Microsoft promises a “new wave of updates that will make every Windows 11 PC an AI-powered PC, with Copilot at the center of it all.”
As Yusuf Mehdi, vice president and head of consumer marketing at Microsoft, makes clear in a blog post, new voice commands are coming to Windows 11 in the form of Copilot Voice, supported by a broader global rollout of Copilot Vision and the ability for the Copilot app to take more actions on your behalf.
Let’s start with the voice element first: all you need to do is say “Hey Copilot” (and definitely not “Hey Cortana”) to invoke the AI. The Copilot microphone will appear on your screen and you can issue voice commands or ask the AI a question. Saying “Goodbye” or closing the panel will automatically end the conversation (with audible chimes confirming both the start and end of a session).
I should note that you need to opt-in to this voice feature on the Windows 11 desktop (by enabling it in the Copilot app settings).
Behind Copilot Voice we have the imminent global launch of Copilot Vision, which until now has only been available in the US.
The Vision feature, again, given your permission to participate, analyzes the contents of your screen and can provide you with help in terms of instructions on how to use apps or perform certain tasks in Windows 11. It can even help troubleshoot when something has gone wrong in the operating system or provide advice in a game. All of this can be facilitated with your voice, as Copilot Voice and Vision work together.
Which is all well and good, but what does AI actually do for you in Windows 11? Microsoft further explains that it has some nifty new powers coming, which will soon be in testing (previews).
Those new powers of AI
Look
First, the Windows 11 taskbar will have an Ask Copilot box, complete with one-click access to Copilot Voice and Vision via its icons (glasses and microphone), should you enable it. We’re also promised better, faster Windows Search with this new functionality (it’s about time, and notably, this doesn’t involve giving Copilot access to your files, nothing has changed in that regard under the hood).
However, Microsoft’s big step with the AI (and agents) being tested is the introduction of Copilot Actions for files on your PC.
If Copilot Actions sounds familiar to you, it’s because it’s something Microsoft has previously introduced for the web, meaning AI can go online for you to complete tasks like making a hotel reservation or buying concert tickets.
In a new ‘experimental mode’ in Copilot Labs for testers, Microsoft will allow AI actions to be taken with files on the local drive in Windows 11. That means you can have the AI sort through, for example, a folder full of old backed up photos and remove any duplicate photos, or extract specific information from a document.
You just need to describe what you want to do (using your voice, if you prefer), and the AI agent “will attempt to complete it by interacting with web and desktop applications,” Microsoft tells us.
Look
You can leave the AI to it and come back to check your progress whenever you want. Of course, the potential for AI to make mistakes is always present, and Microsoft stresses that “real-world testing of this experience is critical” because of this, and that it will start with a “limited set of use cases” slowly and cautiously.
Microsoft stresses that security will clearly be a vital consideration here and will defend against new threats that such a system would introduce, such as cross-injection attacks that could override instructions to AI agents.
Microsoft says it’s establishing a “set of long-standing security and privacy principles that must be followed to make use of new agent capabilities in Windows,” and that includes limiting agent privileges and permissions (to the files you allow AI to work with).
In addition to Copilot Actions, Microsoft has a new general AI agent for Windows 11 called Manus, which can tackle complex tasks, and the software giant explains that it can create a website for you. Using the Manus AI action in File Explorer, you can gather a set of documents and images in a folder and have Manus create a website based on that material, which sounds really impressive.
Microsoft is also adding more Copilot connectors to link services like Gmail to the Copilot app, so you can ask the AI about your upcoming Google-logged appointments, for example. The Copilot app will also be able to help adjust your PC settings, as Microsoft has already revealed.
Analysis: a big step forward for AI
This is all in the works, of course, and not even in testing yet, so much of what we’re talking about here could be far away.
Particularly the idea of letting Copilot work freely with your files on the desktop, which Microsoft has to be very careful about implementing, as noted on the security front. Don’t underestimate what Microsoft is planning here. The idea of Copilot Actions is that it’s an AI agent that takes on the tasks you give it and completes them by “interacting with your apps and files, using vision and advanced reasoning to click, type, and scroll like a human would.”
This is partly what Microsoft means by using the mouse and keyboard a lot less. It’s not just about giving voice commands in Windows 11 or voice queries to the Copilot app, but also about having AI do things for you in the operating system: voice, vision, and actions.
However, Microsoft is certainly emphasizing how important voice input is, noting that the more details and context you share in an AI query to Copilot, the better the response will be, but typing it all out is a pain.
Mehdi notes that because of this, when people use voice, they “interact with Copilot twice as much as when they use text,” and the new wake word (Hey Copilot) is a way to make interacting with Copilot even easier on the desktop.
Mehdi emphasizes: “We believe this shift towards conversational input will be as transformative as the mouse and keyboard in terms of unlocking new capabilities on the PC for the broadest set of people.”
It’s certainly clear that this is a big moment for Windows 11. So far, when it comes to AI, a lot of the criticism has been that its features are underwhelming, but the powers covered here are more far-reaching and potentially impressive. As will the scope of capabilities coming to the Copilot app in general for all PCs, not just Copilot+ laptops.
