Black hat group offered Song huge sums to train open-weight models on exploit data; he declined, warning someone else may accept

Jun Song says he declined an anonymous black hat group's offer of an "astronomical" sum to post-train open-weight models on exploit data. He says the attack dataset they claim to hold is unlike anything he has seen, comparing it to a possible undercover FBI dataset. He notes a white-hat project using GLM-5.3 worked with under one percent of that volume and already outperformed frontier models by a large margin. He warns that someone else will eventually accept the deal.

✨ AI summary of the linked source — not editorial content.

Explore the topic

• Exploit