GitHub Action

GitHub Action

Add a Turing Test to every pull request automatically. The action generates questions from the diff, posts a quiz link as a PR comment, and updates the commit status after the author answers — no prs.md account required for setup.

How it works

1. PR opened or updated — the action fetches the diff and generates 3 targeted questions, one of which is a hallucination trap.
2. Quiz link posted as a PR comment; commit status set to pending.
3. PR author takes the timed quiz on prs.md (3 minutes, no copy-paste).
4. Action grades answers and updates the commit status to pass or fail.

Step 1 — Add repo secrets

Go to your repo → Settings → Secrets and variables → Actions → New repository secret and add these three secrets:

Creating PRS_CALLBACK_TOKEN

The callback token lets prs.md notify the action when a quiz is completed via a repository_dispatch event. It only needs write access to trigger that event.

1. Go to github.com/settings/personal-access-tokens/new
2. Select Fine-grained token
3. Repository access → Only select repositories → pick your repo
4. Permissions → Contents → Read and write
5. Generate the token, copy it, and add it as PRS_CALLBACK_TOKEN

Step 2 — Add the workflow file

Create .github/workflows/prs-md.yml in your repository:

name: "PRs.md Turing Test"

on:
  pull_request:
    types: [opened, synchronize]
  repository_dispatch:
    types: [prs-md-quiz-submitted]

permissions:
  statuses: write
  pull-requests: write

jobs:
  prs-md:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: prs-md/action@v1
        with:
          prs-api-key: ${{ secrets.PRS_API_KEY }}
          prs-provider: ${{ secrets.PRS_PROVIDER }}
          prs-callback-token: ${{ secrets.PRS_CALLBACK_TOKEN }}

Step 3 — Open a PR

That's it. Open or update a pull request and the action will run automatically. The PR author receives a comment with a quiz link, takes the timed quiz on prs.md, and the commit status updates to pass or fail once graded.

Inputs reference