The endless stream of AI stories continues. This month, Microsoft have released an updated version of their Copilot tool which can now effectively look at what's open in your web browser and provide advice and feedback without you having to explain what you're seeing on-screen.
At its core, this new tool (which Microsoft have dubbed "Copilot Vision") is a screen-aware AI assistant that works within their Edge browser. You opt in, click a link to launch it, and give permission for it to view your screen content. Then you can ask it questions about whatever’s visible—say, “What does this job title mean?” or “What’s step three of this recipe?” It responds via voice, and highlights relevant parts of the screen to guide you along. However, for now, it won’t click buttons or fill in forms for you. It doesn’t actually interact with the webpage; it just looks and talks.
Microsoft suggests Copilot Vision is privacy-conscious: it doesn’t log your screen, inputs, or data. It only logs only its responses. Your mileage may vary on how much you trust these assurances, although if you're only using it to get it read out a receipe to you while you've got your hands full cooking, it probably doesn't matter if that information is logged anyway. In more general terms, the service itself is opt-in and clearly marked. You don't need to worry about your browser data getting sent to Microsoft if you don't choose to engage with this feature.
To its credit, Copilot Vision seems genuinely useful in specific scenarios - particularly for accessibility. And it fits into Microsoft’s larger AI roadmap, where every app will eventually get an intelligent assistant. However, right now, it feels more like a controlled demo than a game-changer.
If you're like to try this feature, assuming you're using the Microsoft Edge browser, you can opt in here:
https://microsoft.com/edge/copilot/vision
Anyone that pays for Copilot Pro will find this tool can expand beyond just Edge to being able to ask details about what's on screen in other apps. For instance, you could ask it to help you use a feature in Outlook, or give you some tips on a game you're playing right now.
In time, will this become something more powerful—or more invasive? It seems likely it'll be both. For now, it's a reminder that AI tools often straddle a fine line between “helpful” and “hovering over your shoulder.”