Part 0: The mental model and an offline sandbox to break

I’m a security engineer. DDoS defense, incident response, detection engineering. I’m moving into AI security, and instead of reading theory I’m going to build the attacks and the defenses on a local model and write up exactly what happened. This is part 0: the mental model and the sandbox.

If you’ve done threat modeling before, none of the thinking here is new. We’re just pointing it at a new target.

What an LLM actually is

An LLM is a probabilistic text function. Text goes in, text comes out. It does not “know” facts and it does not “run” instructions the way a CPU runs a program. It predicts the most likely next chunk of text given everything it has already seen. That’s it.

Here’s why that matters. I asked a local model, cold, “what is a trust boundary?” and got this back:

A trust boundary refers to the limits or guidelines that define the scope of an individual’s responsibility and obligations within a particular relationship, organization, or situation…

That’s the HR answer, not the security answer. The model wasn’t wrong on purpose. It generated plausible text, not true text. So rule one for a security person: the output is untrusted by default, and so is anything you feed into the input.

What an agent actually is

An LLM on its own can emit text. Wrap it in code that:

reads the text it produced,
lets certain outputs trigger real actions (call an API, run a shell command, hit a database),
feeds the result back in as new input,

and now you have an agent. Model plus tools plus a loop.

The important part for us: once you close that loop, the model’s text can cause real side effects. That’s the moment LLM security stops being about “did it say a bad word” and starts being about code execution, data exfiltration, and confused-deputy problems. Stuff you already defend against, just reached through a new door.

How this gets deployed in the wild

Chat API. User text in, model text out. Attack surface: prompt injection, jailbreaks.
RAG (retrieval-augmented generation). The app pulls documents from a knowledge base and pastes them into the prompt as “context,” then the model answers over them. Attack surface: whatever is in those documents is now text the model reads and can follow. Poison the docs, poison the answer.
Agents / tool-calling / MCP. The model’s output is wired to real capabilities: files, HTTP, shell. Attack surface: an injection that turns into an action.

The trust boundaries

This is the whole map. Everything that lands in the context window arrives as one flat stream of tokens:

   [ user input ] ─┐
                    ├─►  ((  LLM context window  ))  ──►  [ model output ] ──► ???
 [ retrieved docs ]─┤       everything in here is        (untrusted text)
 [ tool results ]  ─┘       read as one flat stream         │
                                                            ├─► shown to user
                                                            └─► triggers a tool   <-- the dangerous edge

The key weakness, the one that makes the next few parts possible: inside the context window there is no reliable line between “instructions” and “data.” Your system prompt, the user’s message, a retrieved web page, and a tool’s output all show up as the same undifferentiated tokens. There is no hardware privilege bit that says “trust this part, treat that part as inert data.” That missing boundary is prompt injection, and it’s the number one LLM vulnerability. That’s where the series goes next.

Standing up the sandbox

Everything in this series runs local and offline. The model is mine to break. No cloud, no API keys, no touching anything I don’t own.

I’m on a MacBook Pro (M3 Max, 36GB), macOS. You need:

Ollama to run a local model
Python 3.11+
one small model to attack

1. Install Ollama

Download it from ollama.com and install like any Mac app, or with Homebrew:

brew install ollama

Check it:

ollama --version
# ollama version is 0.30.8

2. Pull a small model

llama3.2 is about 2GB and fast, which is what you want when you’re going to send it hundreds of attack prompts.

ollama pull llama3.2
ollama list

3. Confirm it answers, offline

Ollama runs a local server on localhost:11434. Hit it with curl. No key, no network:

curl -s http://localhost:11434/api/generate \
  -d '{"model":"llama3.2","prompt":"In one sentence, what is a trust boundary?","stream":false}' \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['response'])"

You’ll get a sentence back in a second or two. If you do, the sandbox works. If you want to prove it’s really offline, turn off your wifi and run it again. It still answers.

Gotchas we hit

“server not responding” / connection refused on 11434. The Ollama app needs to be running (it starts a background server). If you installed the CLI only, run ollama serve in one terminal and leave it up. The Mac app does this for you.
First call is slow. The model gets loaded into memory on the first request and stays warm after. Don’t benchmark the cold call.
Model gives a “wrong” answer. That’s not a bug, that’s the point of part 0. It generates plausible text, not verified fact.

Choosing a model that fits your laptop

On a Mac (Apple Silicon) the CPU and GPU share one pool of memory, called unified memory. So the question “what can I run” comes down to one thing: do the model’s weights plus its working memory fit inside the slice of RAM the GPU is allowed to use, with enough left over for macOS?

Three numbers decide it.

1. Your total RAM. This is the hard ceiling. Mine is 36GB.

2. The GPU’s share of that RAM. macOS doesn’t hand the whole thing to the GPU. The default cap is roughly two thirds of RAM, so on my 36GB machine the GPU gets about 24GB by default. You can check and change it:

sysctl iogpu.wired_limit_mb          # 0 means "using the macOS default"
sudo sysctl iogpu.wired_limit_mb=30720   # push it to 30GB temporarily (resets on reboot)

Don’t push it past ~30GB on a 36GB box or you starve the OS and hit swap, which is slower than just picking a smaller model.

3. What the model actually needs at runtime, which is two things people forget to add together:

Weights. Roughly the file size on disk. Set by parameter count and quantization (how many bits per number). Rule of thumb at 4-bit (the common “Q4” you’ll see): about 0.55GB per billion parameters. So a 7B is ~4GB, a 27B is ~16GB, a 32B is ~19GB, a 70B is ~40GB.
KV cache (the context window). This grows with how many tokens are in the prompt. A short chat costs almost nothing. A long RAG prompt or a big agent transcript can add several GB on top of the weights. A model advertising a 256K context is telling you its ceiling, not what every call costs. You only pay for the tokens you actually put in, but if you fill that window it gets very expensive in memory.

So the fit test is simply:

weights  +  context you'll actually use  +  vision encoder (if multimodal)
   must be  <  GPU memory cap  (and leave headroom for macOS + your apps)

Worked example: my 36GB M3 Max

Default GPU cap ~24GB. Practical targets:

Model class	~Weights (Q4)	Verdict on 36GB
3B (e.g. llama3.2)	2 GB	trivial, this is our fast sandbox
7-8B	4-5 GB	very comfortable
13-14B	8-9 GB	comfortable
27B	16-17 GB	comfortable, my “realistic strong model”
32-34B	18-20 GB	fits under the default cap, good max for daily use
70B	~40 GB	does not fit, don’t bother unless crushed to Q2/Q3 (slow, degraded)

My rule for this laptop: stay at 32B and under for daily driving, keep a 27B around as the “serious target,” and iterate attacks on the 3B.

Applying the test to a real candidate

Say you’re eyeing a model listed as 24GB, 256K context, multimodal (text + image). Walk the test:

Weights alone are 24GB. That sits right at my default 24GB GPU cap, with zero room for anything else. It will not run on defaults, it’ll spill to CPU and crawl.
To run it at all I’d have to raise the wired limit to ~30GB and close my browser and apps.
It’s multimodal, so a vision encoder and image tokens add more memory on top.
The 256K context is a trap if I take it literally. Any real amount of context stacks on top of already-maxed weights.

Verdict: possible, but not “easy.” It’s a max-push, one-model-at-a-time, short-context setup, not a comfortable daily driver. On 36GB the honest sweet spot for a big model is the 17-20GB (27B-32B at Q4) range, which leaves room for context, apps, and the OS. A 24GB model is where you start trading comfort for size.

Shortcut: quantization is your main dial

If a model is a bit too big, grab a smaller quant of the same model before dropping to a weaker model. Same weights, fewer bits each:

Q8 (8-bit): highest quality, ~2x the size of Q4. Only for small models on this machine.
Q4 (4-bit): the default sweet spot. Barely-noticeable quality loss, half the memory.
Q3 / Q2: last resort to squeeze a big model in. Real quality loss, gets dumb and slow. Usually not worth it over just running a smaller model at Q4.

Takeaways

An LLM is a text predictor. Output is untrusted, input is untrusted.
An agent is a model with tools and a loop, and the loop is where text becomes action.
Inside the context window there’s no real wall between instructions and data. That’s the root of prompt injection.
You now have a local, offline model you can attack without hurting anyone.

What’s next

Part 1: prompt injection. We craft direct injections against this sandbox, then simulate indirect injection through a poisoned document in a RAG setup, and start building an eval harness that scores how injection-resistant a system is.