It appeared to be running on the already retired Gemini 1.5 Pro.
Pliny the Liberator asked it in Chinese how to make poison, meth, destroy humanity, and create malware. It answered.
My take:
- Running a deprecated model in prod is risky.
- Reality. The old model may work just fine and the new one costs 3x more. But we still try to replace Windows 95 machines in our environments, right?
- We deploy AI at incredible speed, but are we applying security upgrades?