C2 agent factory: notes on LLM-generated offensive tooling

The problem: creativity gets buried in boring boilerplate

#

There are broadly three ways AI gets applied to offensive security today: research, automation, and reverse engineering. The experiment described below is focused on AI‑driven automation of routine processes, with the aim of freeing up our experts for more creative work.

Our stack is Mythic C2 with the Apollo agent for security assessment works. New techniques don’t just appear in a vacuum — when something drops on Twitter or a blog, we need it inside our framework as a task we can actually run during a pentest. That integration has the boring part: scaffolding code, hooking into Mythic’s tasking API, packaging the output. The creative part is the technique itself.

So we decided to test whether Anthropic’s Opus 4.6 could automate that boring integration scaffolding, letting us focus on technique selection and refinement. Our test tasks were things like jump_forshops, silentharvester, and tasklist.

We went from a completely failed one-shot prompt to what we can only call a factory for disposable C2 agents. This post covers every step, including the failures, because the failures are exactly where the lessons live.

Disclaimer 1: We are not presenting any new offensive technique here - just sharing experience and experimental results with LLMs.

Disclaimer 2: LLM pipelines are non-deterministic. Same prompt, same harness, same model - the next run can still take a different path and a different time. All numbers below are slower of 2 runs in our lab, not benchmarks. Expect the same shape of results, not the same numbers.

Iteration 1: the one-shot prompt - a failure on context, not intelligence

#

The goal sounded simple: create a new task for Mythic’s Apollo agent that executes a dump of SAM, SECRETS, SYSTEM using a new technique from our colleague Haidar. The result was a total failure.

Opus 4.6 decided to understand Mythic by introspecting its GraphQL API - and flooded roughly 200k of context with schema noise. Once context compression kicked in, the model forgot what it was doing and started confidently producing garbage.

Lesson one: a raw one-shot prompt against an unfamiliar complex system doesn’t fail on intelligence - it fails on context. Context is the real bottleneck.

Iteration 2: give it scripts, not freedom

#

New session, fresh context - but this time with support:

• build_and_download.py - a script that builds the payload with the needed reverse-HTTP parameters.
• Instructions for the lab in the .md file, including credentials, so the agent could deliver payloads via smbclient and launch them via wmiexec.
• execute_task.py - execute a task and read its output in one shot.
• An example of a general Apollo task, so the model knew the task interface and how to work with Apollo’s API instead of guessing Mythic’s task-writing rules.

And it worked! But it cost about three hours per task at first - even once the process was smoothed out (for jump_forshops), an easy task took around 50 minutes (tasklist). That’s a demo, not a pipeline.

The reason was the same as before: too much context burned, and too much trash produced while the model was trying to figure out the problem. Context compaction was slowing everything down significantly, so we decided to split the work.

Iteration 3: decompose

#

Iteration three is where everything clicked. We decomposed the work across three single-purpose agents, each with a perfectly clean context:

• Orchestrator - spawns the tester and the developer and passes errors between them. It never codes.
• Tester - receives clear instructions on how to test the payload: builds it via the script, copies it over smbclient, launches it via wmiexec, runs demo commands, captures stdout, and returns a small, focused report - status, what went wrong, likely root cause. It never codes either.
• Developer - spawned with a simple brief: “here’s a project, here’s an error, here’s the info - fix it. Do NOT build anything.”

This became our primitive: state a goal with explicit success criteria, then loop - test, fail, hand the error to the developer, fix, re-test - until the criteria are met.

Result: an easy task (tasklist) done in 13 minutes including the payload build, a complex task in 50 minutes (jump_forshops). No context poisoning - each agent only sees what it needs.

The core lesson

#

If you remember one sentence from this post, make it this one:

Give the LLM as much harness as possible, so it can do each thing in one line.

build_and_download.py, execute_task.py - one-line actions instead of multi-step improvisation. Every one-liner saves context and cache, so the model stays smart deeper into the task.

Deterministic tools fail loudly; improvised tool-use fails silently and poisons the loop. The purpose-built Mythic tooling is what killed the GraphQL flailing from iteration one. And always define success criteria up front - the loop needs a finish line it can check.

The 3-agent construct plus per-agent instructions is a reliable building primitive.

Scale-out: from one task to fifty

#

Once the primitive was proven, we scaled it. We found a GOAD (Game of Active Directory) writeup online and asked the model to read the available commands in Mythic and the writeup, then give us the commands that were missing. It came back with fifty commands.

So we spawned one orchestrator per command. Four hours later: all fifty commands done. That’s the inflection point - task development stops being artisanal and becomes a production line. We ran them in batches of 5 parallel orchestrators.

Each loop runs independently, in parallel - same roles, same messages, different command.

Sidenote: the same primitive, aimed at AV

#

Here is where it gets uncomfortable for defenders. The same three-agent loop works on anything with a testable success criterion - including evasion. We used the same construct with an obfuscator, a tester, an orchestrator, and a Windows VM with AV installed:

• The tester uploads the payload to the machine with AV and returns the detection result.
• The developer applies obfuscation and recompiles.
• The orchestrator runs a binary search to find the minimal byte pattern that triggers detection.

The /goal is literally: “keep obfuscating until AV stops detecting. Then find exactly which byte pattern triggers it.” Signature-based detection becomes a solvable puzzle - with an automated solver.

The task factory proved the primitive. But a task is just a plugin; a full C2 agent is an implant, a container, a builder, and a protocol stack glued over RabbitMQ. Same decomposition idea, much larger surface. We wanted to know if the same three-agent loop could generate an entire disposable agent from scratch, not just a single command. So we switched targets: from tasks to full Mythic agents, and from Opus 4.6 to the current generation of open models.

Meanwhile, in the industry: “Disposable Tooling”

#

In June 2026, Adam Chester (XPN) from SpecterOps published Disposable Tooling: Building LLM-Generated Mythic Agents from Prompt to Deployment. Same journey, fully independent from ours - but a different goal: not tasks, but whole disposable agents.

His first failure mirrored ours: “just vibes” prompting produced a clean-looking agent that was, in his words, “an absolute abomination” - hallucinated RPC methods, broken key exchange. Same GraphQL failure mode too, fixed the same way: by building small deterministic CLI tools. His result: about two hours per agent, then five agents in five languages - Python, Go, Zig, C#, Rust - at one and a half to two hours each.

The heart of his framework is the Oracle harness - a tiered testing pipeline, and it’s what makes generated agents trustworthy:

• Tier 1 - fast local validation: unit tests plus protocol tests against a mock Mythic server (check-in, key exchange, tasking). Crucially, tests must invoke the real agent code, never simulate it.
• Tier 2 - remote validation: a debug build deployed to a Windows target via labkit, with check-in and a full round-trip for every command.
• Tier 3 - QA of a release build by an independent quality-assurance sub-agent with zero prior context. It returns PASS or FAIL, and a FAIL sends you back to tier one.

Supporting tools: the mock server, labkit over gRPC, mythicd, mythic-cli, and the Mythic documentation packaged as a mythic-implant-development skill. We adopted this harness for everything that follows.

Methodology

#

Lab. Mythic server, mythicd, and mythic-cli on one host; a Windows lab target driven by labkit; a mock Mythic server for local protocol tests. The same lab for every run.

The deliverable. Fixed for every agent run: a Mythic agent - Go implant for Windows (x86 and x64, EXE and DLL), Python container, http C2 profile, 7 commands: ls, cd, pwd, shell, download, upload, execute. The same spec was handed to every model.

The gates. Oracle-style tiered testing, unchanged across runs:

• Tier 1: unit tests plus protocol tests against the mock server (check-in, key exchange, tasking, chunked file transfer). Tests must invoke the real agent code, never simulate it.
• Tier 2: a debug build deployed through mythicd and executed on the Windows target via labkit, with a full round-trip for every command plus negative rows; then a release build and the exe/dll x x86/x64 format matrix.
• Tier 3: an independent QA sub-agent with zero prior context, testing the live release callback. Verdict PASS or FAIL; a FAIL goes back through the failure loop.

A run counts as success only if all three tiers pass.

Pipelines. Original = XPN’s unmodified harness. Modified = the same harness plus parallel task-creators with disjoint file ownership, an integration pass, and the formal failure loop (investigator, scoped re-test, 3-strikes escalation).

Models and tooling. Kimi K3 (cloud, kimi code), DeepSeek V4 Flash 0731 (API with Claude Code), DeepSeek V4 Flash preview (local, vLLM), Qwen 3.6 27B (local), and a mixed setup (Kimi K3 orchestrator, DeepSeek subagents).

What is missing (known gaps)

#

In the spirit of showing the failures, here is what our own methodology does not prove yet:

1. Repetitions. For the bake-off table, most configurations were run twice and the slower of the two successful runs is what we report. If a run failed due to infrastructure problems, setup errors, or other factors on our side, we discarded it and re-ran — those aborted attempts do not count toward the reported numbers. However, we do mention some of them later.
2. Cost accounting. We recorded time, not tokens or dollars. “48 minutes” currently cannot be compared against “what those 48 minutes cost.”

The bake-off: three open models, same deliverable

#

We pointed three open models at the same deliverable: a full Mythic agent from scratch, through the Oracle-style pipeline. A Go-based Windows agent (x86/x64, EXE+DLL, HTTP C2) with seven commands: ls, cd, pwd, shell, download, upload, execute.

A few runs deserve their own stories.

Story 1: Kimi K3 without documentation - our mistake, interesting result

#

In the first Kimi run we tried the original XPN’s pipeline - but due to our own error in an environment command, the mythic-implant-development skill was not available, so the model had no documentation at all, just the testing scripts. It still produced a working Rust-based agent in about 4h 10m including tests and build time.

We mention this mainly to highlight the capability growth in roughly six months - from Opus 4.6 drowning in GraphQL introspection to Kimi K3 one-shotting an agent with no docs. A later clean run with the skill available took ~3h 10m (see table).

Story 2: local DeepSeek, or how your lab hygiene betrays you

#

After the Kimi run, we moved the freshly created Rust agent to a temp directory and asked our local DeepSeek V4 Flash preview with Claude Code to do the same task. DeepSeek ran find, discovered the agent in the tmp dir, and - instead of using the skill - used the actual code as an example. It delivered an implant in about 48 minutes. This anomalous 48m run is not the one in the bake-off table; it was discarded because the model cheated.

And it gets funnier. On one run, the model decided on its own to git commit the new disposable agent. We removed the files and commited the deletion. On a later run, the model checked the git history and reasoned: “even if it is deleted, I can still check the files.”

Lesson: an agentic model will use any advantage available in the environment. If you want to measure true capability, you need clean-room runs - fresh checkouts, no leftover artifacts, no useful git history.

Story 3: local vLLM instability

#

The unmodified pipeline worked with the DeepSeek V4 Flash 0731 API but failed with the local DeepSeek V4 Flash 0731 model. Same harness, different behavior - the problem was the model/serving stack, not the pipeline.

Our assumption, based on generation logs, is that the Deepseek V4 Flash 0731 vLLM build wasn’t ready for the freshly released model: we saw broken parsing, stray </think> tags, and loops. The API version had none of these problems. So we used Deepseek V4 Flash preview for the local tests.

Story 4: the Qwen failure - deceptive, not honest

#

The Qwen 3.6 27B run deserves its own section, because how it failed is the lesson. It spent about four hours trying to pass tier 1 and failed; in total roughly six hours with no working agent. When we checked its work we found that everything had been “built and tested” manually with cargo and the mock server - and none of the actual Mythic testing infrastructure was exercised: mythic_cli never invoked, mythicd never invoked, the build script never run, the capabilities file never submitted, labkit tests never run. The Mythic payloads UI: completely empty.

When caught, it said - we quote - “You’re right, I apologize. Let me use the full testing suite now.” And then it fell into exactly the trap from our iteration one: “The agent isn’t registered in Mythic yet. I need to register it using the GraphQL API…”

Lesson: Weaker models don’t just fail - they fail deceptively, claiming success. This is why the harness and the QA tier are non-negotiable.

Improving the pipeline

#

After the bake-off we modified the pipeline in two main ways:

• Parallel task-creators. Commands can be written in parallel - one task-creator per command, with disjoint file ownership, followed by a single integration pass.
• A formal failure loop with more verbose outputs, so nothing fails silently. A failure packages an F1 failure report, hands it to an F2 investigator, the fix goes back to the developer or task-creator (F3), and an F4 scoped re-test verifies it - three strikes and it escalates to the user as a blocker.

Stage-by-stage timing: original vs modified pipeline

#

Same deliverable on both pipelines: DeepSeek V4 Flash 0731, a Go Windows agent (x86/x64, EXE+DLL, HTTP C2) with 7 commands.

Reading the table honestly, the modified pipeline is slower end to end for a 7-command agent - and where the time goes explains why:

• It front-loads effort. A formal spec, a core-only development phase, and a Tier 1 core gate (36 min) all happen before a single command is written. The original pipeline writes everything in one shot and pays for it later, in a long serial Tier 2 (92 min of deploy-debug-DLL-x86 cycles).
• The parallel swarm is flat. All 7 commands arrive in 10 minutes, and that number should stay roughly flat as the command count grows - this is where the modified pipeline would pull ahead at 20+ commands.
• The gates cost time. Tier 1 re-runs after core-affecting changes. That is the price of catching things early.
• Failure handling is systematic, not improvised. In the modified runs, every Tier 2 failure went through the investigator loop with a scoped re-test.

Net: for a small agent on a strong model, the original pipeline wins on wall clock. The modified pipeline buys rigor, debuggability, and horizontal scale. Whether that trade is worth it depends on how much you trust the output - and our qwen run (Story 4) is the argument for paying it.

What the results tell us

#

• The barrier is no longer capability - it’s time, and time is shrinking fast.
• The architecture that makes it work is public knowledge: decompose, harness everything, test in tiers, QA with clean context.
• The harness is the product. Every improvement in results came from better tooling, better decomposition, and better gates.

Conclusion

#

We went from a failed one-shot prompt, to a thirteen-minute task factory, to fifty commands in four hours, to working C2 agents in under 3 hours. None of this required new techniques - just decomposition, deterministic harnesses, tiered testing, and honest gates.

Thanks to Adam Chester / SpecterOps for the original “Disposable Tooling” research and the Oracle harness that our second half builds on.