Passa ai contenuti principali

Post

Un'allucinazione AI molto istruttiva (per me un pò triste)

Sto preparando il corso " Principi di IA per l'Ingegneria " che dal prossimo a.a. terrò al primo anno di tutte le lauree triennali in Ingegneria. Molto introduttivo, solo 4 seminari di 2 ore ognuno. Sto anche selezionando qualche link di approfondimento per gli interessati. Un tema molto importante è quello delle cosiddette " hallucinations ": affermazioni generate dalla cosiddetta AI che non hanno alcun legame con la realtà. Fenomeno intrinseco alla tecnologia alla base della cosiddetta AI e molto pericoloso per ovvi motivi. Ho avuto una esperienza personale di hallucination che ho deciso di condividere perché mi pare molto, molto istruttiva.  Mio padre è deceduto nel 1972. Amava scrivere e dipengere, per passione personale. I contenuti su di lui su Internet sono ovviamente pochissimi: solo un piccolissimo sito web creato da me. Il sito web contiene link ad alcuni suoi racconti e ad un libro che aveva scritto sulla storia del paese in cui vivevamo, Peccioli. ...
Post recenti

Anthropic Mythos - from the inside

 An excerpt from a blog post by one of the maintainers of curl . A person with a lot of "real world experience" on one of the most widely used software tools in the planet. Everything in the bulleted list below has been just copied-and-pasted (bold is mine, it was not in the original post). I have not added any comment, it is not necessary. This analysis goes in exactly the same direction that I hypothesized in my previous posts ( here and here ). Before this first Mythos report, we had already scanned curl with several different very capable AI powered tools (I mean in addition to running a number of “normal” static code analyzers all the time, using the pickiest compiler options and doing fuzzing on it for years etc)... These tools and the analyses they have done have triggered somewhere between two and three hundred bugfixes merged in curl through-out the recent 8-10 months or so. A bunch of the findings these AI tools reported were confirmed vulnerabilities and have b...

Cose che racconto nei corsi (e che poi si verificano) - April 2026

 Questa volta in italiano. Ogni tanto vedo notizie strettamente collegate ad argomenti di cui ho parlato a lezione pochissimi giorni prima ( esempio ). Ieri sera ne ho visto una molto interessante, legata proprio alla lezione di Cybersecurity di ieri mattina. La lezione era la prima sull'argomento "Memory corruption". Tra le varie cose, ho detto che: Non tutti gli errori ("bug") sono vulnerabilità ; non tutte le vulnerabilità sono sfruttabili ( exploitable ). L' impatto di una vulnerabilità dipende non tanto dall'errore ma dal programma in cui è stato commesso l'errore. Tale dipendenza è estremamente complessa e pressoché impossibile da prevedere in generale. Uno stesso errore in programmi diversi può non avere alcun impatto di sicurezza, avere impatto marginale, avere impatto potenzialmente catastrofico. Le stesse considerazioni valgono per la possibilità di sfruttare una vulnerabilità conseguente ad una particolare tipologia di errore: può essere ...

A subtle hallucination by "the AI"

UPDATE AT THE END I am playing with Gemini for extracting MITRE ATT&CK techniques from cybersecurity incident reports automatically (MITRE ATT&CK is a powerful framework for reasoning about attacks and I use this framework intensively in my Cybersecurity course ): you give Gemini the URL of a report and will immediately obtain the attack techniques used in that attack campaign. Here a spreadsheet with some of the outputs. This usage of "the AI" is potentially very useful for grasping the essentials of an attack campaign quickly and providing students with concrete examples. It is also an usage that fits an essential but often overlooked requirements of AI applications: the cost of a mistake must be small . The prompt I give to Gemini actually asks to extract another important piece of information: the vulnerabilities possibly used in that campaign. In my early attempts I asked Gemini to tell, for each listed vulnerability, whether it was still unknown to software...

More detailed assessment of Anthropic Mythos Preview

A follow-up to my previous post on Mythos Preview. The AI Security Institute (AISI) has published a very interesting analysis of Mythos Preview . Very interesting because: AISI is " a mission-driven research organisation in the heart of the UK government ". Its reports are clearly much more credible than claims of the form " our last product is too strong to give you, believe us " by a private US company, that is currently losing lot of money, that is fiercely battling against other  companies in the AI arena, that is extremely good at fuelling hype about their products and capabilities. They consider complete cybersecurity tasks, i.e. CTF (capture the flag) competitions and attacks to a simulated organization. They compare the behavior of different models for a given "token budget". Not surprisingly, Mythos Preview is indeed very good and better than previous models, but it is definitely not the coming Apocalipsis. In particular, it is the first tool th...

On the Anthropic Mythos Preview - "too dangerous to release"

(updated twice after first posting, see below) On April 7-th 2026, Anthropic issued a technical report titled  Assessing Claude Mythos Preview’s cybersecurity capabilities . This report has quickly sparked the all-too-common (and deeply misleading) narrative of an imminent cybersecurity apocalypse due to the (supposedly) immense and groundbreaking capabilities of AI. For example, The New York Times :  I’m really not being hyperbolic when I say that kids could deploy this by accident. Mom and Dad, get ready for: "Honey, what did you do after school today?” “Well, Mom, my friends and I took down the power grid. What’s for dinner?” That is why Anthropic is giving carefully controlled versions to key software providers so they can find and fix the vulnerabilities before the bad guys do — or your kids. What does Anthropic say? The following paragraphs contain a slightly edited AI-generated summary of the Anthropic report Anthropic has introduced Claude Mythos Preview, a langu...

Cybersecurity and money (and Chinese engineers, and digital escorts)

My Cybersecurity course has a lot of technical detail. Maybe not as much as some students wish, at least in certain topics, but finding the appropriate balance between breadth and depth is difficult. I try to convey to students an important message, though: in order to understand the dynamics of cybersecurity in the real world (" why we are still not applying fundamental principles formulated 50 years ago? ", " why there are so many vulnerabilities? ", " why such an obvious defense is not ubiquitous? "), one must never think solely in technical terms or even worse, in moral terms (" you have to make sure that your code does not have any vulnerabilities, otherwise you will be a sinner and go to hell!",  " company X is evil because does not release patches for its vulnerable software! "). What I tell to students is that one must always think in economical terms ( "yes, this defense is interesting...but what is its cost in terms of f...