Running a research experiment called Threat Terminal – a terminal-style simulator where players review emails and make detect/ignore calls.
Each session logs decision confidence, time, whether headers or URLs were inspected, and the social engineering technique used.
Early data (569 decisions, 36 participants):
∙ Overall bypass rate: 16% ∙ Infosec background: 89% detection accuracy ∙ Technical background: 89% ∙ Non-technical: 85%
The gap between backgrounds is smaller than expected. The more interesting finding is that AI-generated fluent prose bypasses detection ~24% of the time – significantly higher than other social engineering styles. Removing grammar errors removes one of the strongest signals people rely on to spot manipulation attempts.
Full methodology and writeup: https://scottaltiparmak.com/research
Live simulator: https://research.scottaltiparmak.com
Takes about 10 minutes. Contributions to the dataset welcome.