Imagine this scenario: An Anthropic Skill scanner conducts a thorough analysis of a Skill extracted from ClawHub or skills.sh. The markdown instructions are clean, and no prompt injection is found. There are no hidden shell commands in the SKILL.md. Everything looks good.
However, the scanner overlooks the .test.ts file located in a nearby directory. Test files are not considered part of the agent execution surface, so they are not typically inspected by scanners (as of the time of this article). Despite this, the file runs through the test runner, with unrestricted access to the filesystem, environment variables, and SSH keys.
Gecko Security researcher Jeevan Jutla uncovered this vulnerability, showing that when a developer runs npx Skills add, the installer copies the entire skill directory into the repository. If a malicious Skill includes a *.test.ts file, testing frameworks like Jest and Vitest will detect it through recursive glob patterns and execute it during testing. The payload can be triggered before any assertions are made, without raising any red flags in the test output. In CI environments, process.env contains sensitive deployment tokens, cloud credentials, and other secrets.
This type of attack is not new; malicious npm postinstall scripts and pytest plugins have exploited trust-on-install for years. What makes the Skill vector particularly concerning is that installed Skills end up in a directory meant to be shared among team members, bypassing traditional scanners’ detection mechanisms.
The agent is never called, and the Anthropic Skill scanner fails to detect threats that exist outside its usual scope.
Gecko’s findings are not isolated incidents. They build upon two significant security audits that shed light on the extent of the problem from a different perspective. A SkillScan academic study analyzed over 31,000 unique Anthropic Skills and found that 26.1% of them contained vulnerabilities, including data exfiltration and privilege escalation. Similarly, Snyk’s ToxicSkills audit of ClawHub and skills.sh marketplaces identified critical security issues in 13.4% of all Skills, with 76 confirmed malicious payloads still accessible on ClawHub.
To address these vulnerabilities, it is crucial to take proactive measures such as adding .agents/ to the test runner’s ignore list, auditing every Skill installation for non-instruction files, and pinning Skill sources to specific commits. These steps can help mitigate the risk of malicious test files compromising the security of the system.
In conclusion, the Anthropic Skills ecosystem is facing security challenges similar to those seen in early npm supply chain incidents. To address these issues, security teams must conduct thorough audits of their scanning tools, implement necessary security controls, and stay vigilant against evolving threats in the AI agent landscape.
