Skip to content

Pull request review

What this is: how Azure DevOps pull request events reach Ben, who reviews or answers in the PR. Who it is for: developers who open PRs, and admins who connect Azure DevOps. You will learn: which events are used, what is ignored, what Ben may and may not do, who he acts for, and the known limits.

Set up the hook

Your Bill and Ben contact connects Azure DevOps to Bill and Ben with three notifications: a pull request is created, a pull request is updated, and a comment is added. Before any run starts, Bill and Ben ask Azure DevOps whether the named author really wrote the pull request or comment. If that check fails for any reason, the event is refused and recorded.

What happens

  • New PR or new commits. One review per PR every 5 minutes. Ben uses his pull request review skill, a run lasts at most 30 minutes, he only comments, and he works in the sandbox.
  • @ben comment. Ben answers with his comment skill. It runs in the sandbox unless the comment names a machine. Then the commenter is matched to their account and the same permission checks as Device runner apply. A refusal is posted back.
  • Ignored, with the reason in the reply: unrecognised events, draft PRs, PRs authored by Ben, updates that are not new commits, comments by Ben, comments without @ben, deleted comments.

In short: Ben comments on pull requests, and a person always approves and merges.

how Ben reviews a pull request and answers comments without approving or merging how Ben reviews a pull request and answers comments without approving or merging

Legend: yellow = a person, blue = Bill & Ben. Open full size

What Ben may do

  • Own work: branch task/<id>-<slug>, push that branch only, open a draft PR. Big features use feature-intake-to-pr (OpenSpec, ADR, failing tests, HANDOFF.md, branch feature/<id>-<slug>).
  • Needs approval: comments, work item writes, wiki edits and pushes each map to a permission (ado.comment, ado.write, code.push). A write needing a human goes to the requester as a Teams card.
  • Never: vote, approve, merge or force-push.

In short: Ben prepares the feature, a developer finishes it, and a person merges.

a big feature where Ben prepares the branch and a developer finishes it a big feature where Ben prepares the branch and a developer finishes it

Legend: yellow = a person, blue = Bill & Ben. Open full size

Who Ben acts for

Every run carries a run grant (Run grants) for the Azure DevOps channel:

  • Review run. Acts for the PR author and may only ask the agent and comment. If the author is not a workspace user, the review is skipped and a note is logged.
  • @ben comment. Acts for the commenter, with the same limits, plus use of a machine only when one was named and found. An unknown commenter gets a fixed refusal comment in the thread. If the grant cannot be issued, Ben posts a short refusal.
  • Logs refer to sessions by a hash, not by PR number.

Known limits

  • Unverified: no recorded end-to-end run exists. The reply account needs read access to code, or every Azure DevOps run stops.
  • A live grant acts within its scope for its whole lifetime; revoking takes effect after a few seconds.
  • The grant is held inside the agent's own workspace for as long as it lives, so protect that workspace.