Copilot finally has a feature that beats ChatGPT on Windows PCs

There’s no denying it: When it comes to Windows and AI chatbots, ChatGPT leads the pack.

Microsoft invested more than $13 billion in OpenAI, after all. And while the company has pinned Copilot to the taskbar on Windows PCs and placed a Copilot key on new laptops, its Copilot system is still built atop the same large language model (LLM) technology at ChatGPT’s core. That means the system will always be similar to ChatGPT until Microsoft releases its own AI models.

But there’s finally a reason to use Copilot instead of ChatGPT on your PC. It’s a feature Microsoft announced a while ago and is finally making available natively within Windows: You can now share any app on your PC with the Copilot AI chatbot and then have a conversation about it. And Copilot can draw highlights on your screen while you chat.

It’s called Copilot Vision, it’s really interesting — but it has more limits than you might think.

Want more tips for your Windows PC? Sign up for my free Windows Intelligence newsletter. I’ll send you free in-depth Windows Field Guides as a special welcome bonus!

The ChatGPT-Copilot Windows matchup

Traditionally, Copilot has lagged behind ChatGPT in many ways. While OpenAI has confidently stuck with the same ChatGPT interface for a long time, Microsoft has repeatedly hemmed and hawed and changed its mind on what Copilot even should be.

For example, after turning Copilot into something more like the personal companion from Her, Microsoft recently got rid of that more companion-focused app and pivoted Copilot back to more of a standard chatbot interface — for now.

My biggest problem with Copilot, though, is the lack of a crisp model selection tool. ChatGPT lets you get into the weeds and select exactly which AI model you’re using. Microsoft’s Copilot has a simple dropdown box where you select “Quick Response,” “Think Deeper,” or “Deep Research.” If you select Quick Response, you’re not sure what model you’re using: Are you using GPT-4o, or did Copilot just route your query to a less expensive model to save money? There’s no predictability.

Copilot has other limits that aren’t surprising, considering Microsoft has spent the last few years repeatedly reinventing the wheel by constantly creating new Copilot apps. For example, the ChatGPT app lets you search your entire history to find conversations if you want to pull something up. It lets you organize conversations into folders. It gives you the ability to customize your experience. It feels more like a mature app — again, no surprise considering Microsoft has been repeatedly starting from scratch rather than improving a single app over time.

Microsoft’s secret weapon

Despite all those weak points, Copilot has one compelling advantage over ChatGPT — its integration with other Microsoft platforms and services. For example, if you use Microsoft 365 apps like Word and Excel (once called Office apps), you’ll need a $20 per month Copilot Pro subscription rather than a $20 per month ChatGPT Plus subscription. And organizations will need to get AI features for Microsoft 365 through Copilot.

But, while Microsoft quickly added deep Copilot integration to M365, the company lagged when it comes to adding integration to Windows. That’s now changed with Copilot Vision, which arrived first for browsing web pages in Microsoft Edge and officially became available on both Windows 11 and Windows 10 in the US this month. (It will likely launch in additional countries in the future.)

For the moment, the feature works only in real-time voice mode. If you want to have a conversation in text mode, you’ll need to provide screenshots of your screen to Copilot along with text prompts — just as you would with ChatGPT.

How to use Copilot Vision on Windows

To get started with Copilot Vision in its new Windows-native form, just launch the Copilot app from your taskbar or Start menu. (If you have a laptop with a Copilot key, you can just press that, too.)

At the bottom right corner of the Copilot app — at the right side of the text input box — you’ll see a glasses-shaped icon for “Share screen with Copilot.” Click that to get started.

You’ll then be prompted to choose the window you want to share. Copilot doesn’t get access to your PC’s entire display — just the specific app you choose. And it can’t actually control anything in that app: Copilot will be able to see what’s on your screen in that app, and it will be able to draw highlights to call your attention to things. But that’s it.

Copilot will show you a list of all your open windows, and you can share any one you want with the AI model — but just one window at once.Chris Hoffman, Foundry

Once you’ve shared whatever app you want, you can have a conversation and ask questions about it. You might ask “How exactly do I accomplish this task in Photoshop?” or “What do you think of this hotel I’m about to reserve online? Are there better options?” and keep the conversation going for as long as you like.

While you’re sharing an app and having a voice conversation, you’ll see a floating box at the bottom of your screen. Click “Stop” to stop sharing.

A floating box will appear just above your taskbar icons while you’re sharing a window and having a voice conversation.Chris Hoffman, Foundry

You can ask it to point out things on the screen, and it can highlight different areas to draw your attention to them. This is a simplistic feature at launch, as it can point out only one thing. Let’s say you need to find an option five menus deep: The system would highlight the individual icon or menu option visible on your screen, and that’s it. The highlights aren’t great for quickly guiding you through complex interfaces on the fly.

Copilot Vision can highlight things on your screen to draw your attention to them.Chris Hoffman, Foundry

The Copilot Vision cons

Beyond that, Microsoft’s Copilot Vision AI experience has some of the same limits as any AI chatbot at the moment. It wants to validate you, the user. It may nod along, even if you get something wrong. For example, here’s a quick interaction I had:

“How do I draw something in Word?”

“To draw in Word, you’d go to the ‘Draw’ tab on the Ribbon…”

“Okay, so it’s under the Layout tab, right?”

“That’s correct!”

“Nope, it was under the Draw tab.”

“Mm-hm.”

This isn’t an attack on AI chatbots in general, the underlying GPT model from OpenAI, or Copilot itself. It’s just a limitation of the technology — at least at the moment. When interacting with Copilot, ChatGPT, or any other LLM, you need to stay on your toes and question what you’re hearing.

The real limitations with AI voice modes

While voice modes might feel more “futuristic” than text-based LLM interactions, text-based interactions are simply better and richer at the moment. First of all, this voice-based experience is just relying on text. Under the hood, the things you’re saying are getting converted to text, and the LLM is outputting text that is being spoken aloud by a different process. This is crucial to understand: The LLM cannot hear any emotional tone you have in your voice. Additionally, while the Copilot voice you hear may appear to have an emotional tone, that is being inserted by the text-to-speech process after the LLM outputs its text.

This results in a rather uncanny valley experience. The LLM can’t receive any of your vocal cues or tone. It will respond in a voice that appears to have an emotional tone, but the emotional tone isn’t provided by the LLM at all. And, again, it isn’t responsive to your emotional tone.

Beyond that, voice-based interactions are naturally quicker and more surface-level than text-based interactions, where deeper and more analytical responses are possible. So, for lots of tasks, a deeper text-based interaction might be much better than a quick voice-based interaction. If you think it’s not being “smart” enough, try dropping back to text mode and following the tips in my guide to the secret to using generative AI.

There’s no real-time sharing of your screen in text mode, though: You’ll need to take screenshots and provide them along with your text prompts. In Copilot, you can click the “+” button at the right side of the message box and select “Take Screenshot” to quickly add a screenshot. That’s one way to get something like “Vision” along with a text conversation.

Then again, you could always provide screenshots to ChatGPT, too. Now, though, at least, Microsoft has some reason to hope you might be inspired to stick with Copilot instead and think of it as your go-to Windows companion.

Looking for more thoughtful Windows analysis and useful PC tips? Sign up for my free Windows Intelligence newsletter today. I’ll send you three new things to try each Friday.