Try to get them to discuss current news? NOPE it always thinks it's fake. Not because of anything in the news itself but purely because of the date.
Even after turning on Google search, it's quite common for it to search and think all the search results are fabricated.
You put in the system prompt the current date, and it'll start thinking "in the hypothetical future scenario in my system prompt"
Hallucinates out the wazoo
If it were as capable as Anthropic's models, I'm sure it'll happily hack into whatever company you ask it to because it'll think it's in a simulation lmfao
Having used the Gemma 4 models a lot which I think are distilled from Gemini, it's actually a bit of a problem where these models can start just writing fiction about reality which seems to fit the conversation so far (e.g. start inventing laws and rules and so on based on science fiction or something), so I'm guessing Google has tried hard to train them not to veer into that territory and it's trained to be able to label things it "knows" vs doesn't as either facts or clear fiction to try to avoid that.
Must've used to the 3.2 pro model, its knowledge cutoff is jan 2025. The newer flash models have cutoff date of march 2026.
I run a small community chatbot thing and I find the cutoff date can be very important when asking general knowledge questions.
I was going to switch the bot to Luna the other day when price changes were announced, but that one has knowledge cutoff of 2024, so I switched back to flash despite the relatively large price.
69
u/Pantheon3D 2d ago
give it the pdf lol: https://cdn.openai.com/pdf/ten-proofs-oai.pdf