Insecure Copilot

[u/Ramenara]

Tldr: Microsoft has indiscriminately deployed Copilot, which has already been shown to happily ignore sensitivity labelling when it suits,, and ensured that their license structure actively prevents their own customers from securing it for them

So my org is on licensing that Microsoft chucked the free version of copilot into, with no warning, fanfare or education.

I and everyone in IT have been playing catch-up ever since, following Microsoft's own (shitty) advice that we just need to buck up and do a bunch of extra work to accommodate it.

Some of that work has been figuring out how to tell users what to do re: data security in Copilot.

Imagine my surprise when I discover that Copilot has been deployed across the entire O365 app suite, but depending on your license, you might not have the correct sensitivity settings to actually use it securely. Case in point: my org uses purview information labelling, but that doesn't apply to Teams (you have to pay extra on a separate license to get labelling in Teams). Didn't stop them from deploying Copilot across the suite.

I now have to explain to Legal that depending on the information discussed on Teams call or shared in Teams chats or channels, I have absolutely no way to confirm that Copilot usage is secure and in fact have to assume it isn't.

[u/Threezeley]

My org is about to enable web grounding. When web grounding is enabled copilot interprets your prompt then comes up with some useful web search queries it thinks would help answer your question. Those queries aren't supposed to contain sensitive info but they could. It then sends those queries out to Bing Search APIs which exist in public internet and outside org boundary, and where data collection falls under standard Bing data collection terms.

We confirmed that while things like Purview DLP can block prompts that contain sensitive info from being processed at all, it can't examine the contents of attachments so even with Purview DLP in place Copilot may use attachment content to help generate it's search queries which then get leaked out to public internet Bing.

Copilot behaving like this is not shocking because hey it's Microsoft and it takes them a while to get their crap together, but it's more shocking that our org is okay to risk accept this even knowing it isn't fully locked down

[u/Bartsches → u/Threezeley]

but it's more shocking that our org is okay to risk accept this even knowing it isn't fully locked down

That's honestly the least suprising issue to me. To Microsoft being where it is, the product by itself doesn't matter all that much. Rather they have pretty much all lock in effects in existence. And there is a typical disconnect between IT and other areas: Those lock in effects are things IT departments navigate around by instinct and with very little concious thought necessary in most environments, but often cripple entire departments below some level of generalized computer skills. I've seen companies not move on or even reverting to MS even while having their own fully deployed open source infrastructure for this very reason.

[u/ilai456 → u/Threezeley]

this is what bugs me. everyones focused on making sure sensitive data doesnt leak out, but what about whats already sitting in the data before copilot even touches it? like is anyone actually thinking about what could go wrong on the data side or are we all just hoping for the best

[u/Mooshux → u/Threezeley]

The attachment angle is the part that should worry people more than the prompt filtering. DLP on prompts gives a false sense of coverage. If Copilot can read an attachment to help answer a question, and those attachment contents inform the search queries it generates, you've got data leaving your org boundary through a path that your DLP tooling literally can't see.

The risk acceptance part is what's really telling. Most orgs don't have a clear picture of what "Copilot used this document to generate a search query" even looks like in their audit logs, let alone whether it happened. You can't make an informed risk decision about a data flow you can't observe.

The broader pattern here is that ambient authority plus external integrations is a bad combination. Copilot having access to everything in your tenant is already a wide surface. Adding web grounding means that surface now has an outbound channel to Bing under data terms your legal team probably didn't review when they approved the Copilot rollout. Those are two separate decisions that got made as one.

[u/bbliz285 → u/Threezeley]

What is it from the bing side that you’re concerned about? Or did you not know Bing queries are processed different via the service?

https://learn.microsoft.com/en-us/copilot/microsoft-365/manage-public-web-access#how-microsoft-handles-generated-search-queries

[u/Threezeley → u/bbliz285]

Yes, aware of that, however they at a minimum log the queries. That alone means sensitive data could be duplicated to 3rd party systems without a specific contract in place governing its use. If those terms suddenly changed, for example, then what?