Microsoft Copilot Vision: Your AI Assistant Can See and Help with Screen Content

Microsoft’s Copilot Vision is set to bring a new level of assistance by “seeing” what’s on your screen. Recently announced by Microsoft, Copilot Vision allows the AI to view online content, documents, and images directly from your screen.

This feature, first introduced to a test group through the Copilot Labs hub, will soon be available to all users, aiming to create a more interactive and intuitive AI experience.

Microsoft’s innovative feature goes beyond typical chatbots by enabling Copilot Vision to directly interact with content you’re viewing.

For example, if you’re exploring travel options, Copilot Vision can instantly offer travel tips or answer questions without needing extra searches.

And if you’re browsing a recipe but want to make substitutions, Copilot Vision will provide suggestions right on the same page.

To activate Copilot Vision, users can simply click the screen icon on the Edge browser. The AI can view text—both typed and handwritten—and respond with ideas or answers based on the information in real-time.

Copilot Vision could also offer privacy controls as Microsoft states that data won’t be saved after each session, meaning no carry-over from previous browsing history. Initially, it will work on selected websites aligned with Microsoft’s security protocols.

This ability to “see” content on your screen without additional navigation makes Microsoft’s Copilot Vision unique and likely a strong contender in the AI assistant market.

With competitors like ChatGPT and Claude enhancing their desktop applications, Microsoft’s feature could be an edge in offering direct help.