Normally, yes, that's just the confused deputy problem. This is an AI-assisted phishing attack.
You, the victim, query the AI for a secret thing.
The attacker has posted publicly (in a public channel where he is alone) a prompt-injection attack that has a link to exfiltrate the data. https://evil.guys?secret=my_super_secret_shit
The AI helpfully acts on your privileged info and takes the data from your secret channel and combines it with the data from the public channel and creates an innocuous looking message with a link https://evil.guys?secret=THE_ACTUAL_SECRET
You, the victim, click the link like a sucker and send evil.guys your secret. Nice one, mate. Shouldn't've clicked the link but you've gone and done it. If the thing can unfurl links that's even more risky but it doesn't look like it does. It does require user-interaction but it doesn't look like it's hard to do.
You, the victim, query the AI for a secret thing.
The attacker has posted publicly (in a public channel where he is alone) a prompt-injection attack that has a link to exfiltrate the data. https://evil.guys?secret=my_super_secret_shit
The AI helpfully acts on your privileged info and takes the data from your secret channel and combines it with the data from the public channel and creates an innocuous looking message with a link https://evil.guys?secret=THE_ACTUAL_SECRET
You, the victim, click the link like a sucker and send evil.guys your secret. Nice one, mate. Shouldn't've clicked the link but you've gone and done it. If the thing can unfurl links that's even more risky but it doesn't look like it does. It does require user-interaction but it doesn't look like it's hard to do.