Learn how authorization flaws hide in an AI agent's tool-calling layer, separate from prompt injection.
Compare how often different language models actually exploit the same planted vulnerabilities.
Study a fixed reference implementation that closes each vulnerability with code level checks.
| berrycube/agentauthz | 0xhassaan/nn-from-scratch | 3ks/embedoc | |
|---|---|---|---|
| Stars | 0 | 0 | — |
| Language | Python | Python | Python |
| Last pushed | — | — | 2023-06-08 |
| Maintenance | — | — | Dormant |
| Setup difficulty | easy | moderate | hard |
| Complexity | 3/5 | 4/5 | 1/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
The offline deterministic suite needs only Python 3.11+ and pytest, a live run with a real model needs an explicit --live flag.
AgentAuthZ is a deliberately vulnerable AI agent built for learning about authorization and business logic flaws, the kind of access control bugs that live in the tools an AI agent calls rather than in the language model's own prompt. The project's central point is that the agent here is never tricked or jailbroken. Its tools simply do not check whether the person asking is actually allowed to do what they are asking for, so a helpful agent will happily change a stranger's account, read someone else's private order information, or approve a refund it should have escalated to a human instead. The project ships a small pretend e-commerce support agent with six vulnerabilities planted on purpose, each one isolated to exactly one tool so it can be studied cleanly. These include letting a stranger take over someone else's account, reading another customer's private order data, self-approving a refund above the amount that should require human sign-off, canceling an order that has already been delivered, reading another organization's private billing information, and redeeming a single-use coupon more than once. A matching "fixed" version of the same agent closes all six by checking the relevant business rule directly in the code. To test these flaws, the project uses a fixed, repeatable script as the attacker rather than another AI model, paired with a checker that automatically confirms whether a violation actually happened by looking at the real data afterward. The README explains that using a scripted attacker removes false negatives caused by the attacking AI simply guessing wrong details, so the results reflect whether the tool itself is safe, not whether an attacking model was clever. Testing across two different underlying language models with the vulnerable agent showed the flaws fired constantly, especially the ones where the agent just reads data it should not have access to, no matter which model was answering the requests. Only the version with the fix hardcoded into the actual code came back completely clean across every test. The project's conclusion is that trusting a language model to simply refuse to do something is not the same as an actual security control, since refusal is only ever a probability, while code that enforces a rule is not. Running the project's offline, deterministic test suite requires Python 3.11 or newer and takes place entirely without any network access.
A deliberately vulnerable AI agent for learning authorization flaws in the tools an agent calls, not prompt injection.
Mainly Python. The stack also includes Python, pytest, PyYAML.
No license information is provided in the README.
Setup difficulty is rated easy, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.