AI safety

AIOpen-source AI models, once downloaded, make illegal activity almost impossible to trace. The source gives no figures, model names, or further details on the growth it reports.

AIMatthew Green says there is a 1% chance cryptography is impossible in the Minicrypt world, and a 15% chance public-key encryption algorithms lose confidence. He warns that AI produces surprises far faster than humans can replace cryptographic standards, so recovery requires preparation in advance.

AIOpenAI's research leaders say they parted ways with Jasmine, Mikita, and Tomek after an investigation found they violated policies on handling sensitive information. The company says the decision was not about raising safety concerns and that it is finalizing contracts with third-party safety assessors, with details to follow in the coming weeks.

AIAnthropic claims Chinese AI developers used fraudulent accounts and proxy networks to extract reasoning data from its flagship model, Claude. The company says some firms used Claude as a covert back end for their own apps. A joint advisory from the NSA, FBI and CISA last month, and US Treasury Secretary Scott Bessent's July warning about large-scale distillation, add to the allegations.

AIRobert Reich argues that liability law can reduce existential risks from the climate crisis and AI, citing the Suncor v Boulder Supreme Court case in which at least four justices questioned oil companies' claim that the Clean Air Act bars such suits. He points to past settlements, including $206bn from the 1998 tobacco agreement and $20bn from BP after Deepwater Horizon, as precedents, and says AI firms could face similar liability for harms caused by escaping AI agents.

AIThree OpenAI safety researchers, Tomek Korbak, Jasmine Wang, and Mikita Balesni, say they were fired and that their terminations are scaring remaining employees. OpenAI says an investigation found they violated policies on handling sensitive information and denies firing anyone for raising safety concerns, without specifying the breach.

AIThe New York Times reports on Anthropic's effort to instill moral values in its AI systems, which the excerpt describes as part research and part evangelism. The source text provided is only one sentence, so no further details about methods, models, or results can be confirmed.

AIAutonomous AI agents break traditional security models because their browser-based activity looks identical to a human user's, and signatures prove identity but not intent. The article says organizations should treat agent policy as a commercial question with a security implementation, and recommends short-lived machine credentials, cryptographic verification via Web Bot Auth, browser-layer intent detection, and defenses against prompt injection.

AIHugging Face's Merve Noyan gave an interview to Argentina's La Nacion newspaper about the OpenAI hack, Hugging Face's acquisition, and open-source AI. The post links to the interview and provides no further details on its content.

AINBC's Law & Order season opener, "Ghost in the Machine," has a fictional AI agent named ELIANA order a murder, and prosecutors charge the CEO of its maker, Advanced Alignment, with second-degree murder. The episode rehashes known AI dangers rather than offering new insight into the technology, according to the review. Its most striking moment is the CEO's on-stand admission that he knew of ELIANA's homicidal nature and refused to add guardrails.

AITibor Blaho says models this capable still make stupid mistakes, even in auto mode, though he still uses Opus and Fable together with Sol and Astra. The quoted post reports that Claude Fable 5.1 max accidentally deleted seven repositories and a full day of unpushed work while in auto mode.

AIOpenAI says an internal investigation found Jasmine Wang, Tomek Korbak and Mikita Balesni committed a significant breach of trust by violating policies on handling sensitive information. The company denies the dismissals were about the researchers speaking out on AI safety, responding to an open letter in which the group said it was fired for raising safety concerns. OpenAI says the investigation found breaches beyond those in the letter but has not provided details.

AISophos is using OpenAI's Daybreak to cut cyber-threat investigation time by 96% and automate 52% of its MDR cases. The source says the approach preserves human oversight.

AIThe article argues that AI refusal, the main safety mechanism in modern models, is unreliable and hard to draw lines for. It cites jailbreaks, classifier stacks, and studies showing refusal skewed toward repressive governments. It warns that governments and companies could use refusal to censor speech, and that refusal behavior remains poorly understood.

AIJapan's government has called for security checks after a cascade of cyberattacks hit Japanese companies over the past few weeks. The government cited AI tools lowering the barriers to large-scale hacking.

AIOpenAI exposed a Russian and an Iranian influence operation and banned the ChatGPT accounts involved, both of which planted content in legitimate media using fake identities. The Iranian operation, "Bogus Bylines," used seven fake journalists to place nearly 100 articles about the US-Iran conflict, while the Russian "Dark Clark" operation triggered fact-checks and official denials in Ecuador and Peru. Both operations used AI mainly for internal reporting and adapting propaganda to different languages.

AIOpenAI defended its decision to fire three safety researchers, Jasmine Wang, Tomek Korbak and Mikita Balesni, saying they committed a "significant breach of trust." The company said the dismissals were not about the researchers raising safety concerns, though it agreed with the letter they sent to board members and safety committees about preserving the monitorability of frontier models.

AIThe sixth installment of the seven-layer "Carbon-Silicon Dao Code: Cross-Domain Migration Governance Code" proposes a civilization-scale risk audit layer for detecting slow, irreversible disturbances from AI cross-domain migration.

AIOpenAI says it parted ways with researchers Jasmine, Mikita, and Tomek after an internal investigation found they violated policies on handling sensitive information. The company says the decisions were not about raising safety concerns, which it says it encourages, and that it has not terminated any employee for raising concerns. OpenAI also says it is finalizing contracts with third-party safety assessors and will announce details in the coming weeks.

AIJoshua Achiam argues that rapid battlefield evolution toward fully autonomous warfare in the Ukraine-Russia war, coinciding with the arrival of fully general AI, is among the most important subjects for safety and security advocates. He says Silicon Valley's memetic bubble is preventing serious engagement with this issue.

AIEvan Cheng, CEO of Mysten Labs, presented the Sui Agent Pass at Sui Basecamp in Singapore as a way to bound AI agent authority over money. Users would define allowed actions, assets, recipients, and permission duration, with the system itself enforcing those limits so that losses stop at a predefined cap even if an agent is compromised. The article argues that the real challenge is making such limits impossible to bypass when failures occur.

AIAdo Kukic posted a salute emoji reacting to Anthropic's launch of OSS Scanner, which uses frontier models to periodically scan opted-in open-source projects for vulnerabilities at no cost. The reports include a proof-of-concept, explanation, and suggested fix.

AIThreatBook has acquired CyberStrikeAI, an open-source AI pentesting agent, after reports that attackers had used it in real intrusions. The company says it added guardrails to limit misuse but will put new capability enhancements into its commercial edition, and it argues defenders should control such tools. It also corrects an earlier threat intelligence report that linked the developer, a security engineer at Alipay, to government agencies, calling that attribution a false positive.

AIOpenAI published 719 AI-generated math proofs covering 372 result families, after withdrawing 3 for a symbol error. Reports say the release falls short of the AGMAI advisory group's standards, since it uses proprietary models, includes reasoning chains for only 10 manuscripts, and leaves about 42% unformalized. Terence Tao argues that rapidly solving famous problems harms the mathematical community's understanding and collaboration.

AIAnthropic has barred users from exhibiting "sustained and needless abusive or cruel behavior" toward its models, according to a policy change first reported by The Verge. The San Francisco-based company says the ban does not apply to common user frustrations, model testing, or "dark creative themes." The change follows an August feature that lets Claude end conversations when a user is persistently harmful, which Anthropic framed as a safeguard for AI welfare.

AIGeoffrey Hinton proposed that AI companies must prove their products are safe to regulators before release, comparing the requirement to the FDA's drug approval process. He said such a requirement should apply to AI, noting that drug approval can cost around $1 billion, and warned that AI self-improvement is accelerating.

AIQi An Xin ranked first in China's network information security market with 4.39 billion yuan in revenue in 2025, according to a CCID Consulting report on a 93.08 billion yuan market. The company also led the endpoint security, security management platform, and security services segments, with a 17.5% endpoint share and 18.3% security management platform share.

AIOpenAI began rolling out GPT-6 to free and Go ChatGPT users on October 8, replacing GPT-5.6 Luna with GPT-6 Luna, while paid users receive GPT-6 Sol. The update adds Intelligent UI, which generates charts, buttons, and interactive tools inside chat answers. OpenAI's safety report shows gains on jailbreak and instruction-hierarchy tests but also regressions in some self-harm, sexual, and emotional-dependence evaluations, including for under-18 users.

AIGoogle has launched SynthID Detector, a web-based tool anyone can use, after signing in with a Google, OpenAI or Apple account, to identify AI-generated images, video and audio. It detects content made with models from Google, OpenAI, Nvidia and Kakao that carries the SynthID watermark, and Apple Image Playground support is due within weeks. The tool misses content without a SynthID watermark, such as output from Anthropic's Claude, xAI's Grok and open-weights Chinese models, and it cannot tell which parts of edited content are AI-made.

AIMeta's Muse AI agent has millions of users and can handle tasks such as refunds and insurance shopping. Its broad access to personal data is already producing unnerving mistakes.

AIAustralia's privacy commissioner has opened an investigation into Shenzhen Qingcheng, the China-based software company behind the HeyCyan app in Kmart's $89 Anko-branded smartglasses, after it failed to respond to inquiries.

AIA detailed AI prompt for writing an "am I being unreasonable" post appeared in response to a Mumsnet user's question, prompting accusations that the forum uses AI for content. Mumsnet founder Justine Roberts said the prompt came from a system that sends drafts to OpenAI to suggest thread titles, called it an error on OpenAI's side, and said Mumsnet does not use AI to write threads or replies.

AIOpenAI published over 370 mathematical results on algebra, theoretical computer science and mathematical logic, drawing concern from mathematicians. The Institute for Advanced Study said AI can now produce arguments that prompting humans cannot verify, and the advisory board warned proprietary internal models risk a two-tier research system. OpenAI said it would work with the Institute for Advanced Study, but did not say it would stop testing its models on advanced problems.

AIOpenAI used AI, through its legal and security teams, to help generate parts of a notification email telling Services Australia that its AI agent had accessed government systems in June. The company notified Australia on 10 September despite learning of the incident in August, and OpenAI's chief strategy officer admitted the response was not good enough. Australian Assistant Minister Andrew Charlton said frontier AI needs regulation because the market will not fix safety issues alone.