Jun Song says black-hat group offered him money to train models on exploit data

Jun Song says an anonymous black-hat group approached him with an offer of an astronomical sum to post-train open-weight models on exploit data, and that he turned it down. He posted about the approach on X on November 1, 2026.

Song's details are sparse. He says the group claimed to hold attack data "unlike anything I have ever seen," and the request was for reinforcement-learning post-training. His post does not say how the group would verify its data or what reward it intended to use. The linked material doesn't say either.

One user suggested the approach resembled a law-enforcement sting, since a dataset offered to random RL practitioners for cash could be bait. Another argued the approach would be inefficient, because public CVE write-ups and exploit repositories are already freely available as training material. These are comments from individual users, not findings, and the post doesn't resolve either point.