Adding eyes to artificial intelligence is always complicated. Do you want me to see everything you are doing all the time? Certainly not, but I think most of us agrees that a visual assistance of AI when he needs could be quite useful. Microsoft’s new vision of Copilot can be one of the most promising applications of visual abilities based on AI that I have seen so far.
Microsoft announced the Copilot Vision update for your Windows application and mobile applications (you can point your camera in things, and the vision can identify them for you) during a combined co -filter event and Microsoft 50th anniversary.
Copilot everything but received a brain transplant, using generative harvest models (Microsoft AI or MAI) and Openai GPT to deliver updates in memory, search, customization and vision capabilities.
Now that I have seen Copilot Vision in action, I can tell you that it is one of the most exciting and important updates of the group, even if it comes in two stages.
In the version you can access for your supporting support desktop application at this time, Copilot Vision can see the applications you are running on the desktop. When you open co -pilot, selecting the icon or pressing your Copilot key on your keyboard, you can now select the new glasses.
This allows you to see a list of open applications; In our case, we had two in execution: Blender 3D and Clipchamp. This means that, although the co -pilot is aware of the available applications that are executed in Windows, it is not automatically observing.
We select Blender 3D, and from that moment forward, something in my existence of Windows changed. I realized that Copilot can really see what application is executing, and instead of guessing its intention, it responds based on the application and even the project in which you are working.
A 3D coffee table project was open and, using our voice, we asked how to make the table design more traditional. Our notice almost did not contain details about the application or the project, but Copilot’s response, in a charming baritone, was totally contextual.
Then we change and ask how to make notes in the application. Copilot began to answer, but we interrupted and asked where to find the icon to add the annotations. Copilot quickly adjusted and told us quickly how to find it.
This could be greatly useful because it is no longer breaking its flow to jump to the search or even to explain too much what application is using or the project. Copilot Vision sees and knows.
Let me tell you, however, about what is to come.
We follow the same steps to open co -driver and access the vision component, but this time, we point out co -pilot in our Open Clipchamp project.
We asked Copilot how to make our video transitions more perfect. Instead of a text message that explains what to do, Copilot Vision showed us exactly where to find the necessary tool in the application.
A giant arrow appeared (inside a animated circle) on the screen, pointing to the transition tool that we recommend that we explain when explaining the necessary steps. We carry out this demonstration a couple of times, and due to its lower development nature, it did not always work.
However, when he did, he pointed out a potentially exciting change in how we will work with applications in Windows.
We have also seen a demonstration video that shows a vision of covilot cavando even more deeply in the Photoshop application to find the right tools. This, my friends, is full of steroids.
Imagine the future in which you use text indications or your voice to discover how to perform tasks in an open application, and Copilot’s vision of Copilot gets digitally and guides it. There are no signs that you take actions at the level of applications in your name, but this could be an incredible visual assistant.
The good news is that Copilot’s vision in which at least knows what application and project is working on now. The bad news is that Copilot’s vision that I really want does not have a definitive timeline. But I have to assume that it will not spend much time. We saw it live, after all.
