- Microsoft has updated its Copilot Vision tool to let the IA wizard see its desktop
- The function will allow the co -pilot to respond to any content you are seeing
- To control users, Copilot Vision must be enabled, so you will only see what you want you to see
Microsoft has opened the eyes of its Copilot AI on desktop computers to see the entire screen. The function, which allows Copilot to see and analyze things on your screen, can see any window or browser you want, since a Microsoft function calls “share the desk”.
When Copilot Vision first launched, it was limited to just detecting what was happening on the edge. I would ask a question about a web page, scan that page, and the AI would help explain what was there. With this update, you can now get away and see all your desk or any specific application window you choose from sharing
You can choose what the AI assistant is seen by clicking on the glasses icon in the Copilot application. Then select the area that wants the AI to see, and can discuss what is on the screen with the AI without having to describe what you are seeing incessantly.
Let’s say you are reviewing a curriculum and wants you to sound more like a safe human and less as a panic graduation student. You can open the Word document, share the window with co -pilot and ask you to highlight specific types of experiences. Copilot sees the document itself and suggests editions.
The same goes for PowerPoint, Photoshop and even some games. You can ask Copilot for help with the setting menu in a game that you have never played before, and will guide you through options with tips and suggestions for when you get stuck.
Visionary Planning
In case you are worried that the co -pilot fines in your secret romance novel or something else that you prefer not to see, you can always turn it off too. For now, the update is only available for Windows Insids that already have a vision of Copilot. The general population of Windows 11 will probably arrive soon.
It may not seem too large compared to other generative characteristics of AI, but it is a subtle way for Microsoft to embed his AI literally anything on his computer. Instead of writing an email in a tab and having to describe it to AI in another tab to get help, everything is there. Copilot Vision makes the process more intuitive. Digitally point something you want to help, ask a basic question and get an answer that makes sense in context.
It is difficult to exagge how much friction could eliminate this. It is not perfect, of course. A certain specificity is needed since AI cannot read your mind. But an additional set of eyes to help you overcome the hump of designing a PowerPoint or shortening a summary of the meeting is nothing.
The dream of AI as a personal assistant has always been about the context, the idea of an AI that could know what we mean, not only what we say. Copilot Vision is Microsoft’s clearest step in that direction so far. Copilot Vision could become silent in one of the most useful AI tools that Microsoft has made. And unlike Clippy, he will only issue a trial if you ask.
