Self-host PR-Agent, the AI pull request reviewer, on exe.dev
Your own AI pull request reviewer, with no model key on the VM.
Shelley handles the setup. About 20 minutes, most of it creating the GitHub App.
Prefer a terminal?
curl -fsSL https://exe-staging.dev/deploy/pr-agent/prompt.txt | ssh exe.dev new --prompt=/dev/stdinPR-Agent is an open-source AI
reviewer for pull requests (MIT). When a pull request opens it writes
the description and posts a review with the effort to review it, the
security concerns, and the problems it sees; anyone on the pull request
can ask for more by commenting /review, /improve for code
suggestions, or /ask "how is this tested?". Version 0.50.0 came out on
November 7, 2026.
This page runs it as your own GitHub App on an exe.dev VM. The model comes from exe.dev's LLM integration, so no model key is on the VM, and GitHub's webhooks reach it through exe.dev's HTTPS proxy, with nginx in front so that nothing but the webhook is public.
What you get#
- Docker on every boot and the PR-Agent 0.50.0 and nginx images pulled
- PR-Agent using exe.dev's LLM integration, with no model key on the VM
- A webhook secret in
~/pr-agent/webhook-secret - A review of a sample diff, which flags its SQL and command injection
- No GitHub App yet; you create it on GitHub
What's running, and where
https://my-pr-agent.exe.xyz/api/v1/github_webhooks
GitHub's signed webhooks, once you make the VM public; every other path is a 404
ssh my-pr-agent.exe.xyz
configuration and logs
Deploy with Shelley#
Click Deploy and Shelley will set up PR-Agent on a new VM, then check that it works. No config-file wrangling required.
Prefer a terminal?
curl -fsSL \
https://exe-staging.dev/deploy/pr-agent/prompt.txt \
| ssh exe.dev new --prompt=/dev/stdinWhat Shelley is told
Set up PR-Agent on this VM by following the exe.dev recipe at https://exe-staging.dev/deploy/pr-agent.md from top to bottom.
The recipe's example VM name is my-pr-agent. This VM's name is the output of `hostname`: use it wherever the recipe writes my-pr-agent. Skip the step that creates the VM; you are on it.
You are done when all of these are true. Check each one yourself:
- Docker on every boot and the PR-Agent 0.50.0 and nginx images pulled
- PR-Agent using exe.dev's LLM integration, with no model key on the VM
- A webhook secret in `~/pr-agent/webhook-secret`
- A review of a sample diff, which flags its SQL and command injection
- No GitHub App yet; you create it on GitHub
Run the commands in "Install by hand", from "Get PR-Agent" on, and the
check at its end. Show the owner the first lines of the review. Don't
start the server (`docker compose up`) or make the VM public: that needs
the GitHub App, which the owner creates. Finish by telling the owner to
read "What's exposed" and follow "Create the GitHub App".
Finish with the URLs I should open.
Also at /deploy/pr-agent/prompt.txt. The recipe it follows is /deploy/pr-agent.md.
Install#
On exe.dev, click Deploy to have Shelley set it up on a new VM. Or create it yourself:
Install by hand
On your computer:
ssh exe.dev new --name=my-pr-agent
ssh my-pr-agent.exe.xyz
Get PR-Agent#
On the VM, start Docker, now and on every boot, and download the Compose file, the nginx config and a sample diff to try it on:
sudo systemctl enable --now docker.service
mkdir -p ~/pr-agent && cd ~/pr-agent
curl -fsSLO https://exe-staging.dev/deploy/pr-agent/compose.yaml
curl -fsSLO https://exe-staging.dev/deploy/pr-agent/nginx.conf
curl -fsSLO https://exe-staging.dev/deploy/pr-agent/sample.diff
docker compose pull
The Compose file sends PR-Agent's model calls to the LLM integration,
with gpt-5.5 and a placeholder key, and puts the nginx gate on port
8000, the port exe.dev's proxy serves.
Make the webhook secret#
On the VM, generate the secret GitHub will sign its webhooks with, and start PR-Agent's secrets file with it. PR-Agent runs as user 10001 in its container, so that group may read the file:
cd ~/pr-agent
(umask 077; openssl rand -hex 32 > webhook-secret)
printf '[github]\ndeployment_type = "app"\nwebhook_secret = "%s"\n' "$(cat webhook-secret)" > .secrets.toml
sudo chgrp 10001 .secrets.toml && chmod 640 .secrets.toml
Check the setup#
On the VM, have PR-Agent review the sample diff, which adds a Flask app with a SQL injection and a command injection:
cd ~/pr-agent
docker compose run --rm --no-deps -v ./sample.diff:/tmp/sample.diff:ro \
--entrypoint python pr-agent pr_agent/cli.py --diff-file /tmp/sample.diff review
Within a minute it prints a PR Reviewer Guide whose security section names both injections.
What's exposed#
GitHub sends pull request events as webhooks, so the VM needs a public
address. exe.dev's proxy makes a VM's port 8000 public as a whole, not
one path, so nginx holds that port and lets through only
POST /api/v1/github_webhooks; everything else is a 404. PR-Agent checks
GitHub's signature on every webhook and answers 403 without a valid one.
PR-Agent's own port, 3000, stays private, and the VM's other ports are
never public.
Two things remain. PR-Agent's webhook handler is reachable from the internet, so keep it updated. And on a public repository, anyone who can open a pull request or comment on one can start a model call billed to you; install the App only on repositories whose contributors you trust.
Create the GitHub App#
On the VM, print the webhook secret:
cat ~/pr-agent/webhook-secret
On GitHub, under Settings → Developer settings → GitHub Apps → New GitHub App (for an organization: the organization's settings), fill in:
- GitHub App name: anything; it's the name on the comments.
- Homepage URL:
https://my-pr-agent.exe.xyz - Webhook URL:
https://my-pr-agent.exe.xyz/api/v1/github_webhooks - Webhook secret: the line you printed
- Repository permissions: Contents, read-only; Issues, read and write; Pull requests, read and write. Metadata is read-only by default.
- Subscribe to events: Issue comment, Pull request, Pull request review, Pull request review comment.
- Where can this GitHub App be installed?: Only on this account.
Create it, note the App ID at the top of its page, and under
Private keys click Generate a private key. Your browser downloads
a .pem file. On your computer, copy it to the VM:
scp ~/Downloads/*.private-key.pem my-pr-agent.exe.xyz:pr-agent/app.pem
Start PR-Agent#
On the VM, add the App ID and the key to the secrets file, replacing
123456 with your App ID, and start PR-Agent and the gate:
cd ~/pr-agent
APP_ID=123456
{ echo "app_id = $APP_ID"; echo 'private_key = """'; cat app.pem; echo '"""'; } >> .secrets.toml
rm app.pem
docker compose up -d
On your computer, make port 8000 public:
ssh exe.dev share set-public my-pr-agent
On the VM, curl -s http://localhost:3000/ prints {"status":"ok"}.
Then check the public side from your computer, since a VM can't reach
its own public address:
curl -s -o /dev/null -w '%{http_code}\n' https://my-pr-agent.exe.xyz/
curl -s -o /dev/null -w '%{http_code}\n' -X POST https://my-pr-agent.exe.xyz/api/v1/github_webhooks
The first prints 404, the gate; the second 403, PR-Agent refusing a
request GitHub didn't sign.
Install it on a repository#
On the App's page on GitHub, click Install App and pick the
repositories. Open a pull request in one of them: within a minute or two
PR-Agent fills in the description and posts a PR Reviewer Guide
comment. Comment /improve for code suggestions, or /help for every
command.
The App's Advanced tab lists each webhook delivery and the VM's
answer; docker compose logs -f pr-agent on the VM shows the other
side.
Configure it#
A .pr_agent.toml at the root of a repository changes what PR-Agent does
there, for example what its reviews look for:
[pr_reviewer]
extra_instructions = "Focus on authorization checks and SQL."
num_max_findings = 10
The configuration guide
lists the options. Settings for every repository go in compose.yaml as
environment variables, section and key joined by two underscores, such
as PR_REVIEWER__EXTRA_INSTRUCTIONS.
Change the model#
To use another OpenAI model the LLM integration serves, change
CONFIG__MODEL in ~/pr-agent/compose.yaml to openai/ plus its name,
for example openai/gpt-5.6-terra, and run docker compose up -d. The
model list shows what's available.
For a Claude model, replace the OPENAI__API_BASE, OPENAI__KEY and
CONFIG__MODEL lines with these, and run docker compose up -d:
ANTHROPIC_BASE_URL: https://llm.int.exe.xyz
ANTHROPIC__KEY: exe-gateway
CONFIG__MODEL: anthropic/claude-sonnet-5-5
Update#
Check the release notes,
change the image tag in ~/pr-agent/compose.yaml, then:
cd ~/pr-agent
docker compose pull && docker compose up -d
Clean up#
On your computer, close the public port and delete the VM:
ssh exe.dev share set-private my-pr-agent
ssh exe.dev rm my-pr-agent
Then delete the GitHub App under Settings → Developer settings → GitHub Apps.
Troubleshooting#
On the App's Advanced tab:
- A delivery got
403: the App's webhook secret differs from~/pr-agent/webhook-secret. Paste it again. - A delivery got
404: the webhook URL's path is wrong; it ends in/api/v1/github_webhooks. - A delivery failed to connect, or got a login page: port 8000 isn't
public.
ssh exe.dev share show my-pr-agentshows it. - Deliveries got
200but no comment appears:docker compose logs pr-agent --tail 100on the VM. A wrong App ID or key shows up as a GitHub authentication error; a model problem as an error fromlitellm. - The review check fails:
curl -s https://llm.int.exe.xyz/v1/modelson the VM should list models; if it does, the model name incompose.yamlis likely one the integration doesn't serve.
Frequently asked questions#
How much does it cost?
The VM is part of your exe.dev plan. The model is billed to your exe.dev account through the LLM integration: a few cents for a review of a small pull request with gpt-5.5, more for large ones. Every opened pull request gets a description and a review, and every command someone types in a comment is another call.
Is this the same as Qodo Merge?
PR-Agent started at Qodo, which still sells a hosted product built on it. The open-source project now lives at The-PR-Agent/pr-agent under the MIT license; this page runs that.
Why does it need a public URL?
GitHub delivers pull request events as webhooks, so the VM has to be reachable from the internet. The page exposes only the webhook path, and PR-Agent rejects anything GitHub didn't sign. See What's exposed.
Can it review pull requests in public repositories?
It can, but anyone who opens a pull request or comments /review there starts a model call billed to you. Install the App only on repositories whose contributors you trust.
Can I use Claude models?
Yes. The LLM integration also serves Anthropic's API; set ANTHROPIC_BASE_URL and the model to anthropic/ plus a Claude model. See Change the model.
Updated Nov 16, 2026 · tested end to end Nov 16, 2026 · Markdown · manifest.json · Something wrong? Tell us