Skip to main content

Bonus 2 - A tool: the recorded walkthrough

Recorded with Claude Code 2.1.289, from the lab's instructions. Results are shortened; your agent's answers will differ in wording.

Step 1 - Start the project next to the clone​

The participant runs uv init --lib robotframework-github-reporter:

Initialized project `robotframework-github-reporter` at `<scratch>

The participant runs uv add "robotframework==7.5":

Prepared 1 package in 4ms
Installed 2 packages in 6ms
 + robotframework==7.5
 + robotframework-github-reporter==0.1.0 (from file://<repo>)

The participant runs uv add --dev pytest:

 + pluggy==1.6.0
 + pygments==2.21.0
 + pytest==9.1.1
 ~ robotframework-github-reporter==0.1.0 (from file://<repo>)

Step 2 - Save the references​

The participant runs `mkdir references

GitHub's issue endpoints, from its own API description: listing, opening and commenting on issues

uv run --no-project python -c " import json, urllib.request url = 'https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/dereferenced/api.github.com.deref.json' spec = json.load(urllib.request.urlopen(url)) keep = ['/repos/{owner}/{repo}/issues', '/repos/{owner}/{repo}/issues/{issue_number}/comments'] json.dump({path: spec['paths'][path] for path in keep}, open('references/github-issues.json', 'w'), indent=1) "

A listener to imitate: robotframework-heal's, from the workshop's environment

uv run --project ../ai-engineering-robotframework --no-sync python -c "import shutil, heal.rf.listener as m; shutil.copy(m.file, 'references/example-listener.py')"`:

(no output)

The participant runs ls -l references | cut -c30-:


59 Oct  6 20:24 example-listener.py
86 Oct  6 20:23 github-issues.json

Step 3 - Write AGENTS.md​

In the lab you write this file yourself. For the reference, the rehearsal lets the agent draft it from the lab's own list, and the transcript says so.

Prompt:

Write AGENTS.md for this project from this list, as the sections Toolstack, References, Concepts, Examples and Specification, short, and nothing else:

  • Toolstack:
    • Python 3.12 with uv; Robot Framework 7.5 is the only dependency at run time;
    • HTTP goes through urllib.request from the standard library, so that the listener runs in any project's environment;
    • unit tests run with uv run pytest, Robot tests with uv run robot --outputdir results atest;
    • packaging with uv build.
  • References:
  • Concepts:
    • a listener, version 3, registered with --listener module.Class:name=value;
    • a reporter never fails or changes a test: every error of its own becomes a warning;
    • the token comes from GITHUB_TOKEN or GH_TOKEN, never from an argument or a file;
    • the dry run is the default;
    • it imports from its source folder alone, as step 8 runs it with --pythonpath: it reads nothing about itself from installed package metadata.
  • Examples: references/example-listener.py.
  • Specification: OpenSpec, under openspec/changes/.

The agent runs ls -a && ls references

.
..
.git
.gitignore
.python-version
.venv
README.md
pyproject.toml
references
src
uv.lock
example-listener.py
... (1 more lines)

The agent writes <repo>/AGENTS.md

File created successfully at: <repo>/AGENTS.md (file state is

Agent:

I created AGENTS.md in the project root. It has only the five sections you listed (Toolstack, References, Concepts, Examples and Specification), with each item kept close to your wording and no title or intro.

Two things to know:

  • openspec/changes/ doesn't exist yet, so the Specification section points to a folder that will only appear once you add a change. Both files listed under References and Examples are present.
  • "Step 8" in the Concepts section is copied as you wrote it. Nothing in the repo says what it means, so a reader may need to be told where the steps come from.

The participant runs cat AGENTS.md:

## Toolstack

- Python 3.12 with uv; Robot Framework 7.5 is the only dependency at run time.
- HTTP goes through `urllib.request` from the standard library, so that the listener runs in any project's
  environment.
- Unit tests run with `uv run pytest`, Robot tests with `uv run robot --outputdir results atest`.
- Packaging with `uv build`.

## References

- `references/github-issues.json`.
- The User Guide's
... (21 more lines)

Step 4 - Set up OpenSpec​

The participant runs openspec init --tools claude:


Learn more: https://github.com/Fission-AI/OpenSpec
Feedback:   https://github.com/Fission-AI/OpenSpec/issues

- Setting up Claude Code...
✔ Setup complete for Claude Code

In the lab you write this file yourself. For the reference, the rehearsal lets the agent draft it from the lab's own list, and the transcript says so.

Prompt:

In openspec/config.yaml, uncomment context: and write there, in a few lines each, the five points of AGENTS.md: toolstack, references, concepts, examples and specification. Change nothing else in the file.

The agent reads <repo>/openspec/config.yaml

The agent searches for pythonpath|[Ss]tep 8

AGENTS.md:24:- It imports from its source folder alone, as step 8 runs it with `--pythonpath`: it reads nothing about

The agent edits <repo>/openspec/config.yaml

The file <repo>/openspec/config.yaml has been updated success

The agent runs uv run python -c "import yaml,sys; d=yaml.safe_load(open('openspec/config.yaml')); print(sorted(d)); print(d['context'])" 2>&1 || python3 -c "import yaml; d=yaml.safe_load(open('openspec/config.yaml')); print(sorted(d)); print(d['context'])"

Exit code 1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'yaml'
pyenv: version `3.12' is not installed (set by <repo>/.python

The agent runs uv run --no-project --with pyyaml python -c "import yaml; d=yaml.safe_load(open('openspec/config.yaml')); print(sorted(d)); print(d['context'])"; command -v openspec >/dev/null && openspec validate --help >/dev/null 2>&1; echo done

['context', 'schema']
Toolstack:
- Python 3.12 with uv; Robot Framework 7.5 is the only run-time dependency.
- HTTP goes through the standard library's urllib.request, so the listener runs in any project's environment.
- Unit tests: `uv run pytest`; Robot tests: `uv run robot --outputdir results atest`; packaging: `uv build`.

References:
- references/github-issues.json.
- The Robot Framework 7.5 User Guide, listener interface and listener version 3:
  https://robotframework.org/robotframework/7.5/RobotFrameworkUserGuide.html#listener-interface
  https://robotframework.org/robotframework/7.5/RobotFrameworkUserGuide.html#listener-version-3
- ListenerV3 in the Robot API, whose code is robot/api/interfaces.py in the project's .venv:
... (17 more lines)

Agent:

I uncommented context: in openspec/config.yaml and filled it with the five points from AGENTS.md: toolstack, references, concepts, examples and specification. Each is a short block of a few lines. I checked that the file still parses as YAML: its top-level keys are now schema and context.

I replaced the four example lines (# context: | and the three lines under it) and changed nothing else. The # Example: comment line above them is still there.

I made one wording change. AGENTS.md says the listener imports only from its source folder "as step 8 runs it with --pythonpath". "Step 8" isn't defined anywhere else in the repo, so I wrote "as it is run with --pythonpath". If step 8 refers to something specific you want kept, it's easy to put back.

The participant runs sed -n '1,30p' openspec/config.yaml:

schema: spec-driven

# Project context (optional)
# This is shown to AI when creating artifacts.
# Add your tech stack, conventions, style guides, domain knowledge, etc.
# Example:
context: |
  Toolstack:
  - Python 3.12 with uv; Robot Framework 7.5 is the only run-time dependency.
  - HTTP goes through the standard library's urllib.request, so the listener runs in any project's environment.
  - Unit tests: `uv run pytest`; Robot tests: `uv run robot --outputdir results atest`; packaging: `uv build`.

... (18 more lines)

Prompt:

Which instruction files did you load at the start of this session? Name each file with its path.

Agent:

I loaded one instruction file at the start of this session:

  • <repo>/AGENTS.md: the project instructions checked into the repository. It covers the toolstack, references, concepts, examples and where the OpenSpec specification lives.

No other instruction files were loaded:

  • No CLAUDE.md was in context, either for the project or for your user account.
  • My memory index, MEMORY.md in ~/.claude/projects/.../memory/, contributed nothing. Either it doesn't exist yet or it's empty.

The other things in my context are not instruction files: the automatic git status snapshot, the commit attribution reminder and the list of available skills.

Step 5 - Propose the first slice​

Prompt:

/opsx:propose A listener, version 3, importable as robotframework_github_reporter.GitHubIssues, with the arguments repo (owner/name), dry_run (true by default) and label (robot-failure by default). For every failed test, it opens an issue in repo titled "Failing test: <the test's full name>", with the label, the failure message, the test's source and line, and the run's URL when it runs in GitHub Actions. If an open issue with that title and label exists, it adds a comment instead. In a dry run it sends nothing: it prints each request it would send, and writes them to github-requests.json in the output directory. Live, it reads the token from GITHUB_TOKEN or GH_TOKEN, and falls back to a dry run with a warning when there is none. Unit tests replace urllib.request.urlopen; Robot tests in atest/ run a failing suite with the listener in a dry run.

The agent runs openspec list --json; echo "---"; openspec context --json

{
  "changes": [],
  "root": {
    "path": "<repo>",
    "source": "nearest"
  }
}
---
{
  "root": {
    "path": "<repo>",
    "source": "nearest",
... (5 more lines)

The agent runs ls -la openspec openspec/* 2>/dev/null; echo "---"; cat openspec/config.yaml 2>/dev/null || cat openspec/config.yml 2>/dev/null; echo "---"; find . -path ./.venv -prune -o -path ./.git -prune -o -type f -print | head -100

-rw-r--r-- 1 user user 2214 Oct  6 20:24 openspec/config.yaml

openspec:
total 20
drwxr-xr-x 4 user user 4096 Oct  6 20:24 .
drwxr-xr-x 8 user user 4096 Oct  6 20:24 ..
drwxr-xr-x 3 user user 4096 Oct  6 20:24 changes
-rw-r--r-- 1 user user 2214 Oct  6 20:24 config.yaml
drwxr-xr-x 2 user user 4096 Oct  6 20:24 specs

openspec/changes:
total 12
... (91 more lines)

The agent reads <repo>/references/example-listener.py

The agent reads <repo>/references/github-issues.json

The agent reads <repo>/pyproject.toml

The agent reads <repo>/README.md

The agent reads <repo>/src/robotframework_github_reporter/__init__.py

The agent runs python3 -c " import json d=json.load(open('references/github-issues.json')) print(type(d), list(d.keys())[:20] if isinstance(d,dict) else len(d)) for k,v in d.items(): if isinstance(v,dict): print(k, list(v.keys())[:40]) else: print(k, repr(v)[:200]) "

Exit code 1
pyenv: version `3.12' is not installed (set by <repo>/.python

The agent runs ls -d .venv 2>/dev/null; which uv jq; PYENV_VERSION=system python3 -c " import json d=json.load(open('references/github-issues.json')) print(type(d), list(d.keys())[:20] if isinstance(d,dict) else len(d)) for k,v in d.items(): if isinstance(v,dict): print(k, list(v.keys())[:60]) else: print(k, repr(v)[:200]) "

.venv
~/.local/bin/uv
/usr/bin/jq
<class 'dict'> ['/repos/{owner}/{repo}/issues', '/repos/{owner}/{repo}/issues/{issue_number}/comments']
/repos/{owner}/{repo}/issues ['get', 'post']
/repos/{owner}/{repo}/issues/{issue_number}/comments ['get', 'post']

Agent:

Checking the GitHub API reference and the Robot Framework listener API before I draft the artifacts.

The agent runs jq -r ' to_entries[] | .key as $p | .value | to_entries[] | "\(.key|ascii_upcase) \($p) — \(.value.summary)\n params: \([.value.parameters[]? | (.name // ."$ref") + "(" + (.in // "") + ")"] | join(", "))\n responses: \(.value.responses|keys|join(","))\n body props: \(.value.requestBody.content["application/json"].schema.properties // {} | keys | join(", "))\n required: \(.value.requestBody.content["application/json"].schema.required // [] | join(", "))\n"' references/github-issues.json

GET /repos/{owner}/{repo}/issues  — List repository issues
  params: owner(path), repo(path), milestone(query), state(query), assignee(query), type(query), creator(query), mentioned(query), issue_field_values(query), labels(query), sort(query), direction(quer
  responses: 200,301,404,422
  body props: 
  required: 

POST /repos/{owner}/{repo}/issues  — Create an issue
  params: owner(path), repo(path)
  responses: 201,400,403,404,410,422,503
  body props: assignee, assignees, body, issue_field_values, labels, milestone, parent_issue_id, title, type
  required: title

... (11 more lines)

The agent runs jq '."/repos/{owner}/{repo}/issues".get | {description, params: [.parameters[] | select(.in=="query") | {name, description, schema}], headers: .responses."200".headers, item_props: (.responses."200".content["application/json"].schema.items.properties // {} | keys)}' references/github-issues.json | head -150

{
  "description": "List issues in a repository. Only open issues will be listed.\n\n> [!NOTE]\n> GitHub's REST API considers every pull request an issue, but not every issue is a pull request. For this
  "params": [
    {
      "name": "milestone",
      "description": "If an `integer` is passed, it should refer to a milestone by its `number` field. If the string `*` is passed, issues with any milestone are accepted. If the string `none` is pass
      "schema": {
        "type": "string"
      }
    },
    {
      "name": "state",
... (138 more lines)

The agent runs jq '."/repos/{owner}/{repo}/issues".post | {description, body: .requestBody.content["application/json"].schema.properties | {title, body, labels}, r403: .responses."403".description, r410: .responses."410".description, r422: .responses."422".description, r503: .responses."503".description, r201headers: .responses."201".headers}' references/github-issues.json; jq '."/repos/{owner}/{repo}/issues".post.description' -r references/github-issues.json | head -5; jq '."/repos/{owner}/{repo}/issues/{issue_number}/comments".post | {description, body: .requestBody.content["application/json"].schema}' references/github-issues.json; jq -r '[.. | objects | to_entries[] | select(.key=="x-github" or .key=="servers" or .key=="security")] | length' references/github-issues.json

{
  "description": "Any user with pull access to a repository can create an issue. If [issues are disabled in the repository](https://docs.github.com/articles/disabling-issues/), the API returns a `410 
  "body": {
    "title": {
      "oneOf": [
        {
          "type": "string"
        },
        {
          "type": "integer"
        }
      ],
... (71 more lines)

The agent runs ls .venv/lib/python*/site-packages | head -30; F=$(ls .venv/lib/python*/site-packages/robot/api/interfaces.py); grep -n "class ListenerV3" -A 40 $F | head -60; grep -n "def end_test\|def close\|def output_file\|def start_suite\|def end_suite\|def log_message\|def message" $F

__pycache__
_pytest
_virtualenv.pth
_virtualenv.py
iniconfig
iniconfig-2.3.0.dist-info
packaging
packaging-26.3.dist-info
pluggy
pluggy-1.6.0.dist-info
py.py
pygments
... (62 more lines)

The agent runs SP=.venv/lib/python3.12/site-packages/robot; sed -n 940,1080p $SP/api/interfaces.py; grep -n "_instantiate\|def _get_arg_spec\|resolve\|convert" $SP/utils/importer.py | head -30

        """Called by default when a keyword or a control structure ends.

        New in Robot Framework 7.0.
        """

    def log_message(self, message: result.Message):
        """Called when a normal log message are emitted.

        The messages are typically logged by keywords, but also the framework
        itself logs some messages. These messages end up to output.xml and
        log.html.
        """
... (137 more lines)

The agent runs SP=.venv/lib/python3.12/site-packages/robot; sed -n 80,100p $SP/utils/importer.py; sed -n 195,230p $SP/utils/importer.py; grep -n "def split_args_from_name_or_path" -A 30 $SP/utils/text.py | head -40

        Classes can be imported from the module search path using name like
        ``modulename.ClassName``. If the class name and module name are same, using
        just ``CommonName`` is enough. When importing a class by a path, the class
        name and the module name must match.

        Optional arguments to use when creating an instance are given as a list.
        Starting from Robot Framework 4.0, both positional and named arguments are
        supported (e.g. ``['positional', 'name=value']``) and arguments are converted
        automatically based on type hints and default values.

        If arguments needed when creating an instance are initially embedded into
        the name or path like ``Example:arg1:arg2``, separate
... (75 more lines)

Agent:

Confirming how Robot Framework collects suites, so the failing inner suite under atest/ doesn't get picked up by the outer atest run.

The agent runs SP=.venv/lib/python3.12/site-packages/robot; grep -n "startswith\|_is_ignored\|def _is_included\|ignore" $SP/parsing/suitestructure.py | head -20; grep -n "def full_name\|def source\|lineno\|def failed\|def message" $SP/result/model.py | head -30; grep -n "lineno\|def source" $SP/running/model.py | head -20

117:    ignored_prefixes = ("_", ".")
118:    ignored_dirs = ("CVS",)
166:    def _is_included(self, path: Path) -> bool:
167:        if path.name.startswith(self.ignored_prefixes):
170:            return path.name not in self.ignored_dirs
257:    def failed(self) -> bool:
262:    def failed(self, failed: bool):
851:    def messages(self) -> "list[Message]":
860:    def full_name(self) -> "str | None":
895:    def sourcename(self) -> str:
900:    def sourcename(self, name: str):
1018:        lineno: "int | None" = None,
... (22 more lines)

Agent:

Grounding is done. Creating the change and reading the schema instructions now.

The agent runs cat .gitignore; echo ---; openspec new change "add-github-issues-listener" && openspec status --change "add-github-issues-listener" --json

# Python-generated files
__pycache__/
*.py[oc]
build/
dist/
wheels/
*.egg-info

# Virtual environments
.venv
---
- Creating change 'add-github-issues-listener' with schema 'spec-driven'...
... (109 more lines)

The agent runs for a in proposal specs design tasks; do echo "===== $a"; openspec instructions $a --change "add-github-issues-listener" --json | jq 'del(.context)'; done

===== proposal
{
  "changeName": "add-github-issues-listener",
  "artifactId": "proposal",
  "schemaName": "spec-driven",
  "changeDir": "<repo>/openspec/changes/add-github-issues-lis
  "planningHome": {
    "kind": "repo",
    "root": "<repo>",
    "changesDir": "<repo>/openspec/changes",
    "defaultSchema": "spec-driven"
  },
... (122 more lines)

Agent:

No existing specs, so this adds one new capability, github-issue-reporting. Writing the proposal.

The agent writes <repo>/openspec/changes/add-github-issues-listener/proposal.md

File created successfully at: <repo>/openspec/changes/add-git

Agent:

Writing the delta spec for the new capability.

The agent writes <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

File created successfully at: <repo>/openspec/changes/add-git

*The agent runs `cd && rm -rf rfprobe && mkdir -p rfprobe/_data && cd rfprobe && cat > probe.py <<'EOF' from robot.api import logger from robot.libraries.BuiltIn import BuiltIn

class Probe: ROBOT_LISTENER_API_VERSION = 3 def init(self, repo: str, dry_run: bool = True, label: str = "robot-failure"): self.args = (repo, dry_run, type(dry_run).name, label) logger.warn(f"probe init warning {self.args}") def start_suite(self, data, result): if data.parent is None: logger.warn("probe outdir=" + str(BuiltIn().get_variable_value("${OUTPUT DIR}"))) def end_test(self, data, result): logger.console(f"probe console: {result.full_name} {result.status} {data.source} {data.lineno} {result.lineno}") def end_suite(self, data, result): if data.parent is None: logger.warn("probe root end_suite warning") EOF printf '*** Test Cases \nFails\n Fail boom\n\nPasses\n No Operation\n' > _data/failing.robot printf ' Test Cases *\nOuter\n No Operation\n' > outer.robot P=../rh/robotframework-github-reporter/.venv/bin/python $P -m robot --pythonpath . --listener "probe.Probe:repo=octo-org/demo:dry_run=false" --outputdir out _data/failing.robot; echo "rc=$?" grep -o 'probe[^<]' out/output.xml | sort -u echo "--- outer dir run collects:"; $P -m robot --dryrun --outputdir out2 . | grep -E "^(Rfprobe|Outer|Fails|Passes)" `

[ WARN ] probe init warning ('octo-org/demo', False, 'bool', 'robot-failure')
==============================================================================
Failing                                                                       
==============================================================================
[ WARN ] probe outdir=<scratch>
Fails                                                                 probe console: Failing.Fails FAIL <scratch>
| FAIL |
boom
------------------------------------------------------------------------------
Passes                                                                probe console: Failing.Passes PASS <scratch>
| PASS |
------------------------------------------------------------------------------
... (19 more lines)

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent runs rm -rf <scratch> cd <repo> && openspec validate add-github-issues-listener --strict

Change 'add-github-issues-listener' is valid

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

Agent:

Writing the design.

The agent writes <repo>/openspec/changes/add-github-issues-listener/design.md

File created successfully at: <repo>/openspec/changes/add-git

Agent:

Checking one detail the unit-test plan depends on: where logger.warn goes when the listener is created outside a Robot run, as in pytest.

The agent runs SP=.venv/lib/python3.12/site-packages/robot; sed -n 1,80p $SP/output/librarylogger.py | grep -n "def write\|logging\|running\|LOGGER\|context" ; grep -n "isinstance\|import_listener\|def _import_listeners\|def _get_listeners" $SP/output/listeners.py | head

16:"""Implementation of the public logging API for libraries.
27:from .logger import LOGGER
35:def write(
46:        LOGGER.log_message(Message(msg, level, html=html, console=console))
40:        if isinstance(log_level, str):
43:        self._listeners = self._import_listeners(listeners)
50:    def _import_listeners(self, listeners, library=None) -> "list[ListenerFacade]":
53:            if library and isinstance(listener, str) and listener.upper() == "SELF":
89:        listeners = self._import_listeners(library.listeners, library=library)
113:        if isinstance(priority, (int, float)):
131:        if isinstance(log_level, str):
134:            return cls._import_listener(listener, log_level, library, kind)
... (2 more lines)

The agent runs .venv/bin/python - <<'EOF' import logging, io logging.basicConfig(level=logging.DEBUG, format="PYLOG %(levelname)s %(message)s") from robot.api import logger logger.warn("outside run") import robot, tempfile, os, pathlib class L: ROBOT_LISTENER_API_VERSION = 3 def __init__(self): logger.warn("from init, before run") def start_suite(self, d, r): logger.warn("in run") t = tempfile.mkdtemp(); p = pathlib.Path(t, "s.robot"); p.write_text("*** Test Cases ***\nT\n Fail x\n") out, err = io.StringIO(), io.StringIO() rc = robot.run(str(p), listener=L(), outputdir=t, stdout=out, stderr=err) print("rc", rc); print("STDERR:", err.getvalue().strip()) print("errors in xml:", [l for l in open(os.path.join(t,"output.xml")) if "WARN" in l]) EOF

PYLOG WARNING outside run
PYLOG WARNING from init, before run
rc 1
STDERR: [ WARN ] in run
errors in xml: ['<msg time="2026-10-06T20:31:55.118064" level="WARN">in run</msg>\n']

Agent:

Confirmed. Outside a run, logger.warn goes to Python logging, which pytest's caplog captures, and robot.run accepts listener instances. Noting that in the design's test plan.

The agent edits <repo>/openspec/changes/add-github-issues-listener/design.md

The file <repo>/openspec/changes/add-github-issues-listener/d

Agent:

Writing the tasks.

The agent writes <repo>/openspec/changes/add-github-issues-listener/tasks.md

File created successfully at: <repo>/openspec/changes/add-git

The agent runs openspec validate add-github-issues-listener --strict && openspec status --change "add-github-issues-listener" && git status --short

Change 'add-github-issues-listener' is valid
- Loading change status...
Change: add-github-issues-listener
Schema: spec-driven
Change root: <repo>/openspec/changes/add-github-issues-listen
Progress: 4/4 artifacts complete

[x] proposal
[x] specs
[x] design
[x] tasks

... (12 more lines)

Agent:

I've created the change add-github-issues-listener in openspec/changes/add-github-issues-listener/, and it passes openspec validate --strict. No project code was changed.

What's in it

  • proposal.md: why the listener is needed, what it does, and one new capability, github-issue-reporting. There were no existing specs to modify.
  • specs/github-issue-reporting/spec.md: 7 requirements and 30 scenarios. They cover:
    • the listener's arguments and defaults
    • one report per failed test
    • the issue's content
    • commenting on a matching open issue
    • the dry run
    • where the token comes from
    • never failing or changing a test
  • design.md: four small modules (listener.py, content.py, github.py, __init__.py). The dry run and the live run share one flow, and only the final send step differs: a dry run records the request, a live run calls urllib.request.urlopen. That call is the only thing the unit tests replace.
  • tasks.md: 15 tasks in 6 groups, each saying how to verify it.

I checked the Robot Framework 7.5 behaviour the plan relies on against the installed code and a throwaway run outside the repo:

  • dry_run=false arrives as a real False.
  • ${OUTPUT DIR} is available when the first suite starts.
  • Warnings from the listener end up in log.html, except ones logged in close(), which only reach the console.
  • A directory run skips folders starting with _, so the failing suite can sit in atest/_data/ without breaking uv run robot ... atest.

Decisions I made without asking you

This session couldn't wait for answers, so I chose these myself. Each one is easy to change with /opsx:update:

  1. The dry run includes the lookup. Since it sends nothing, it can't see existing issues. It records one GET for open issues with the label, then a "create issue" POST for each failure; it never shows a comment.
  2. Console output comes at the end. The dry run prints its requests when the top-level suite ends, not after each test. Printing after each test lands in the middle of Robot's status line.
  3. The reporter stops after a connection error or timeout. It sends nothing more for the rest of the run, so a dead network doesn't add the 10-second timeout to every later failure. HTTP errors don't trigger this; a failed lookup is retried for the next test.
  4. Smaller choices:
    • The source path is shown relative to the working directory.
    • The issue body puts the failure message in a code block.
    • Comments start with "Failed again."
    • The API address is fixed to https://api.github.com, so GitHub Enterprise Server isn't supported.
    • An invalid repo gives one warning and turns the listener off.

The artifacts are ready for review. When you're ready, run /opsx:apply.

Step 6 - Review the proposal​

The rehearsal's review: the questions of step 6, put to the agent, which changes its artifacts where an answer is no.

Prompt:

Review the change you just proposed against these questions, and change its artifacts wherever an answer is no. Then answer each question in one line.

Does it use only the standard library for HTTP, and add no dependency at run time? Can no error of the listener fail, skip or change a test? Is the token read from the environment only, and is the dry run the default? Does it import from src/ alone, without being installed? Do the unit tests cover a new issue, a comment on an open one, a dry run, a missing token and an API error?

Agent:

I'll re-read the four artifacts from disk, then check the points I'm unsure of against Robot Framework 7.5 rather than assume them.

The agent runs cd <repo>/openspec/changes/add-github-issues-listener && wc -l proposal.md design.md tasks.md specs/github-issue-reporting/spec.md && cat specs/github-issue-reporting/spec.md | sed -n 1,40p

   55 proposal.md
  252 design.md
  119 tasks.md
  191 specs/github-issue-reporting/spec.md
  617 total
# Spec Delta

## Purpose

Turns the failed tests of a Robot Framework run into GitHub issues, one per test. A test that fails again gets a
comment on its open issue rather than a second issue, and a dry run shows exactly which requests would be sent.

... (33 more lines)

The agent runs S=<scratch> R=$S/rh/robotframework-github-reporter; P=$R/.venv/bin/python; rm -rf $S/rfprobe2 && mkdir -p $S/rfprobe2 && cd $S/rfprobe2 && cat > strict.py <<'EOF' class Strict: ROBOT_LISTENER_API_VERSION = 3 def __init__(self, repo: str, dry_run: bool = True, label: str = "robot-failure"): print("STRICT", repr(repo), repr(dry_run), type(dry_run).__name__, repr(label)) EOF cat > loose.py <<'EOF' class Loose: ROBOT_LISTENER_API_VERSION = 3 def __init__(self, repo: str = "", dry_run: bool = True, label: str = "robot-failure", *extra, **unknown): print("LOOSE", repr(repo), repr(dry_run), type(dry_run).__name__, repr(label), extra, unknown) EOF printf '*** Test Cases ***\nPasses\n No Operation\n' > s.robot echo "== 1. missing repo + --exitonerror"; $P -m robot --exitonerror --pythonpath . --listener strict.Strict --output NONE --log NONE --report NONE s.robot 2>&1 | grep -E "ERROR|PASS|FAIL|stopped|tests," ; echo "rc=${PIPESTATUS[0]}" echo "== 2. dry_run=maybe (strict)"; $P -m robot --pythonpath . --listener "strict.Strict:repo=o/r:dry_run=maybe" --output NONE --log NONE --report NONE s.robot 2>&1 | grep -E "STRICT|ERROR" echo "== 2b. dry_run=maybe + --exitonerror"; $P -m robot --exitonerror --pythonpath . --listener "strict.Strict:repo=o/r:dry_run=maybe" --output NONE --log NONE --report NONE s.robot 2>&1 | grep -E "STRICT|ERROR|tests," echo "== 3. typo labels= (strict)"; $P -m robot --pythonpath . --listener "strict.Strict:repo=o/r:labels=x" --output NONE --log NONE --report NONE s.robot 2>&1 | grep -E "STRICT|ERROR" echo "== 4. loose: no args / typos / extra positional"; $P -m robot --pythonpath . --listener "loose.Loose" --listener "loose.Loose:repo=o/r:labels=x:token=abc" --listener "loose.Loose:o/r:false:lbl:extra" --listener "loose.Loose:repo=o/r:dry_run=maybe" --output NONE --log NONE --report NONE s.robot 2>&1 | grep -E "LOOSE|ERROR"

== 1. missing repo + --exitonerror
[ ERROR ] Taking listener 'strict.Strict' into use failed: Importing listener 'strict.Strict' failed: Listener 'Strict' expected 1 to 3 arguments, got 0.
Passes                                                                | FAIL |
S                                                                     | FAIL |
rc=1
== 2. dry_run=maybe (strict)
STRICT 'o/r' 'maybe' str 'robot-failure'
== 2b. dry_run=maybe + --exitonerror
STRICT 'o/r' 'maybe' str 'robot-failure'
== 3. typo labels= (strict)
[ ERROR ] Taking listener 'strict.Strict:repo=o/r:labels=x' into use failed: Importing listener 'strict.Strict' failed: Listener 'Strict' got unexpected named argument 'labels'.
== 4. loose: no args / typos / extra positional
... (5 more lines)

The agent runs S=<scratch> R=$S/rh/robotframework-github-reporter; P=$R/.venv/bin/python; cd $S/rfprobe2 && cat > warny.py <<'EOF' from robot.api import logger class Warny: ROBOT_LISTENER_API_VERSION = 3 def __init__(self): logger.warn("init warning") def end_test(self, data, result): logger.warn("end_test warning") EOF echo "== 5. warnings + --exitonerror"; $P -m robot --exitonerror --pythonpath . --listener warny.Warny --output NONE --log NONE --report NONE s.robot 2>&1 | grep -E "WARN|ERROR|PASS|FAIL" ; echo "== 6. installed in project venv?"; $P -c "import importlib.metadata as m; print(m.version('robotframework-github-reporter'))" cd $R && echo "== 7. isolated uv env"; timeout 120 uv run --isolated --no-project --with robotframework==7.5 python -c " import importlib.util, importlib.metadata as m, sys, robot print('python', sys.version.split()[0], 'robot', m.version('robotframework')) print('package importable:', importlib.util.find_spec('robotframework_github_reporter') is not None) try: m.version('robotframework-github-reporter'); print('metadata: present') except m.PackageNotFoundError: print('metadata: absent') print('dists:', sorted(d.metadata['Name'] for d in m.distributions())) " 2>&1 | tail -6; rm -rf $S/rfprobe2

== 5. warnings + --exitonerror
[ WARN ] init warning
Passes                                                                [ WARN ] end_test warning
| PASS |
S                                                                     | PASS |
== 6. installed in project venv?
0.1.0
== 7. isolated uv env
Installed 1 package in 6ms
python 3.12.11 robot 7.5
package importable: False
metadata: absent
... (1 more lines)

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

Agent:

Now the design. I'll re-read it from disk before editing.

The agent reads <repo>/openspec/changes/add-github-issues-listener/design.md

The agent reads <repo>/openspec/changes/add-github-issues-listener/design.md

The agent reads <repo>/openspec/changes/add-github-issues-listener/design.md

The agent edits <repo>/openspec/changes/add-github-issues-listener/design.md

The file <repo>/openspec/changes/add-github-issues-listener/d

The agent edits <repo>/openspec/changes/add-github-issues-listener/design.md

The file <repo>/openspec/changes/add-github-issues-listener/d

The agent edits <repo>/openspec/changes/add-github-issues-listener/design.md

The file <repo>/openspec/changes/add-github-issues-listener/d

The agent edits <repo>/openspec/changes/add-github-issues-listener/design.md

The file <repo>/openspec/changes/add-github-issues-listener/d

The agent edits <repo>/openspec/changes/add-github-issues-listener/design.md

The file <repo>/openspec/changes/add-github-issues-listener/d

Agent:

Now the tasks. I'll re-read them before rewriting.

The agent reads <repo>/openspec/changes/add-github-issues-listener/tasks.md

The agent reads <repo>/openspec/changes/add-github-issues-listener/tasks.md

The agent edits <repo>/openspec/changes/add-github-issues-listener/tasks.md

The file <repo>/openspec/changes/add-github-issues-listener/t

The agent edits <repo>/openspec/changes/add-github-issues-listener/tasks.md

The file <repo>/openspec/changes/add-github-issues-listener/t

The agent edits <repo>/openspec/changes/add-github-issues-listener/tasks.md

The file <repo>/openspec/changes/add-github-issues-listener/t

The agent edits <repo>/openspec/changes/add-github-issues-listener/tasks.md

The file <repo>/openspec/changes/add-github-issues-listener/t

The agent edits <repo>/openspec/changes/add-github-issues-listener/tasks.md

The file <repo>/openspec/changes/add-github-issues-listener/t

The agent edits <repo>/openspec/changes/add-github-issues-listener/tasks.md

The file <repo>/openspec/changes/add-github-issues-listener/t

Agent:

Tightening the -k filter in task 4.4 so it matches exactly those five tests, then updating the proposal.

The agent edits <repo>/openspec/changes/add-github-issues-listener/tasks.md

The file <repo>/openspec/changes/add-github-issues-listener/t

The agent reads <repo>/openspec/changes/add-github-issues-listener/proposal.md

The agent edits <repo>/openspec/changes/add-github-issues-listener/proposal.md

The file <repo>/openspec/changes/add-github-issues-listener/p

The agent edits <repo>/openspec/changes/add-github-issues-listener/proposal.md

The file <repo>/openspec/changes/add-github-issues-listener/p

The agent edits <repo>/openspec/changes/add-github-issues-listener/proposal.md

The file <repo>/openspec/changes/add-github-issues-listener/p

The agent edits <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The file <repo>/openspec/changes/add-github-issues-listener/s

The agent runs openspec validate add-github-issues-listener --strict && openspec status --change add-github-issues-listener | tail -7 && grep -c "#### Scenario" openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md && grep -n "comma" openspec/changes/add-github-issues-listener/{design,tasks}.md | head; git status --short src pyproject.toml

Change 'add-github-issues-listener' is valid
- Loading change status...
[x] proposal
[x] specs
[x] design
[x] tasks

All planning artifacts complete!
Next: openspec instructions apply --change "add-github-issues-listener" --json
35
openspec/changes/add-github-issues-listener/design.md:36:  - `GET /repos/{owner}/{repo}/issues` takes `state`, `labels` (comma-separated), `per_page` (at most 100) and `page`,
openspec/changes/add-github-issues-listener/design.md:172:  - **Label**: an empty label, or one containing a comma, does the same. A comma would break the `labels` filter of
... (5 more lines)

Agent:

Two answers were "no" (the second and the fourth), and I changed all four artifacts to fix them. The change still passes openspec validate --strict, and no project code was touched.

  1. Standard library only, no new run-time dependency? Yes: HTTP goes through urllib.request, Robot Framework 7.5 stays the only dependency, and the acceptance tests now run in an environment holding nothing else.
  2. Can no listener error fail, skip or change a test? It was "no" and is now yes: a probe showed that a missing repo or a misspelled argument like labels=x stops the listener loading, and under --exitonerror that turned a passing test into FAIL. Every argument now has a default, extra or unknown arguments only cause a warning, and the listener never logs at error level.
  3. Token from the environment only, dry run by default? Yes: a token= argument now gets a warning and its value is thrown away unprinted. Only an explicit false value makes a run live, so dry_run=maybe stays a dry run.
  4. Imports from src/ alone without being installed? It was "no" and is now yes: the project's .venv has the package installed, so the planned tests proved nothing. The Robot tests now run in uv run --isolated --no-project --with robotframework==7.5, where the package can't be imported and has no metadata. The catch is that those tests need uv's cache or the network.
  5. Unit tests cover a new issue, a comment, a dry run, a missing token and an API error? Yes: the coverage was there but scattered, so tasks now name the five tests and the design has a table of what each one fakes and checks.

The artifacts are ready for review. When you are ready, run /opsx:apply.

Step 7 - Apply it​

The rehearsal tool stopped the first /opsx:apply session after 11 of the change's 17 tasks, at its own time limit; that session is not recorded. A new /opsx:apply continued from the ticked tasks, as you would after an interruption.

Prompt:

/opsx:apply

The agent runs openspec list --json

{
  "changes": [
    {
      "name": "add-github-issues-listener",
      "completedTasks": 11,
      "totalTasks": 17,
      "lastModified": "2026-10-06T18:51:01.985Z",
      "status": "in-progress"
    }
  ],
  "root": {
    "path": "<repo>",
... (3 more lines)

Agent:

Using change: add-github-issues-listener (override with /opsx:apply <other>).

The agent runs openspec status --change "add-github-issues-listener" --json && openspec instructions apply --change "add-github-issues-listener" --json

{
  "changeName": "add-github-issues-listener",
  "schemaName": "spec-driven",
  "planningHome": {
    "kind": "repo",
    "root": "<repo>",
    "changesDir": "<repo>/openspec/changes",
    "defaultSchema": "spec-driven"
  },
  "changeRoot": "<repo>/openspec/changes/add-github-issues-li
  "artifactPaths": {
    "proposal": {
... (210 more lines)

The agent reads <repo>/openspec/changes/add-github-issues-listener/proposal.md

The agent reads <repo>/openspec/changes/add-github-issues-listener/specs/github-issue-reporting/spec.md

The agent reads <repo>/openspec/changes/add-github-issues-listener/design.md

The agent reads <repo>/openspec/changes/add-github-issues-listener/tasks.md

The agent runs find . -path ./.venv -prune -o -path ./.git -prune -o -path ./.pytest_cache -prune -o -type f -print | grep -v __pycache__ | sort && cat pyproject.toml .gitignore README.md && ls -la atest

./.claude/commands/opsx/apply.md
./.claude/commands/opsx/archive.md
./.claude/commands/opsx/explore.md
./.claude/commands/opsx/propose.md
./.claude/commands/opsx/sync.md
./.claude/commands/opsx/update.md
./.claude/skills/openspec-apply-change/SKILL.md
./.claude/skills/openspec-archive-change/SKILL.md
./.claude/skills/openspec-explore/SKILL.md
./.claude/skills/openspec-propose/SKILL.md
./.claude/skills/openspec-sync-specs/SKILL.md
./.claude/skills/openspec-update-change/SKILL.md
... (70 more lines)

The agent reads <repo>/atest/_data/failing.robot

The agent reads <repo>/atest/github_issues.robot

The agent reads <repo>/src/robotframework_github_reporter/listener.py

The agent reads <repo>/src/robotframework_github_reporter/github.py

The agent reads <repo>/src/robotframework_github_reporter/content.py

The agent runs wc -c README.md && cat src/robotframework_github_reporter/__init__.py && cat openspec/config.yaml && cat references/example-listener.py && ls -la results 2>/dev/null | head

0 README.md
from .listener import GitHubIssues

__all__ = ["GitHubIssues"]
schema: spec-driven

# Project context (optional)
# This is shown to AI when creating artifacts.
# Add your tech stack, conventions, style guides, domain knowledge, etc.
# Example:
context: |
  Toolstack:
... (507 more lines)

The agent runs uv run pytest -q 2>&1 | tail -5

........................................................................ [ 94%]
....                                                                     [100%]
76 passed in 0.78s

Agent:

All 76 unit tests pass. Next I'm checking task 5.1: the failing suite must stay out of a plain robot atest run.

The agent runs uv run robot --dryrun --outputdir results atest 2>&1 | tail -30

==============================================================================
Atest                                                                         
==============================================================================
[ ERROR ] Error in file '<repo>/atest/github_issues.robot' on
Atest.Github Issues :: Runs `_data/failing.robot` with the listener, in a d...
==============================================================================
Dry run records the requests                                          | FAIL |
Keyword 'Run Failing Suite' expected at least 1 non-named argument, got 0.
------------------------------------------------------------------------------
Run URL in GitHub Actions                                             | FAIL |
Keyword 'Run Failing Suite' expected at least 1 non-named argument, got 0.
------------------------------------------------------------------------------
... (17 more lines)

The agent runs sed -n '1,120p' .venv/lib/python3.12/site-packages/robot/running/arguments/argumentresolver.py && grep -n "def split_from_equals" -A 25 .venv/lib/python3.12/site-packages/robot/utils/escaping.py

#  Copyright 2008-2015 Nokia Networks
#  Copyright 2016-     Robot Framework Foundation
#
#  Licensed under the Apache License, Version 2.0 (the "License");
#  you may not use this file except in compliance with the License.
#  You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
... (134 more lines)

The agent edits <repo>/atest/github_issues.robot

The file <repo>/atest/github_issues.robot has been updated su

The agent edits <repo>/atest/github_issues.robot

The file <repo>/atest/github_issues.robot has been updated su

The agent edits <repo>/atest/github_issues.robot

The file <repo>/atest/github_issues.robot has been updated su

The agent edits <repo>/atest/github_issues.robot

The file <repo>/atest/github_issues.robot has been updated su

The agent runs uv run robot --dryrun --outputdir results atest 2>&1 | tail -25

==============================================================================
Atest                                                                         
==============================================================================
Atest.Github Issues :: Runs `_data/failing.robot` with the listener, in a d...
==============================================================================
Dry run records the requests                                          | PASS |
------------------------------------------------------------------------------
Run URL in GitHub Actions                                             | PASS |
------------------------------------------------------------------------------
No run URL outside GitHub Actions                                     | PASS |
------------------------------------------------------------------------------
Live run without a token falls back to a dry run                      | PASS |
... (12 more lines)

Agent:

The dry run collects only Atest.Github Issues, so _data/failing.robot stays out (task 5.1 ✓). Now the real acceptance run, which builds the isolated environments:

The agent runs uv run robot --outputdir results atest 2>&1 | tail -40

==============================================================================
Atest                                                                         
==============================================================================
Atest.Github Issues :: Runs `_data/failing.robot` with the listener, in a d...
==============================================================================
Dry run records the requests                                          | PASS |
------------------------------------------------------------------------------
Run URL in GitHub Actions                                             | PASS |
------------------------------------------------------------------------------
No run URL outside GitHub Actions                                     | PASS |
------------------------------------------------------------------------------
Live run without a token falls back to a dry run                      | PASS |
... (12 more lines)

Agent:

All 5 pass. To make sure they aren't passing vacuously, I'll look at what one inner run actually printed, and confirm the isolated environment can't import the package without --pythonpath:

The agent runs uv run python - <<'EOF' import xml.etree.ElementTree as ET root = ET.parse("results/output.xml").getroot() for test in root.iter("test"): msgs = [m.text or "" for m in test.iter("msg")] std = [m for m in msgs if "github-reporter" in m] print("==", test.get("name"), "| logged msgs mentioning github-reporter:", len(std)) if test.get("name") in ("Dry run records the requests", "Misconfiguration is only a warning under --exitonerror"): for m in std: for line in m.splitlines(): if "github-reporter" in line: print(" ", line[:150]) EOF echo "--- isolated import without --pythonpath:" uv run --isolated --no-project --with robotframework==7.5 python -c "import robotframework_github_reporter" 2>&1 | tail -1

== Dry run records the requests | logged msgs mentioning github-reporter: 4
    ${output_dir} = /tmp/github-reporter-atest-gb04dzel/tmp54vnbpbv
    uv run --isolated --no-project --with robotframework==7.5 python -m robot --pythonpath <scratch>
    github-reporter (dry run): GET https://api.github.com/repos/octo-org/demo/issues?state=open&labels=robot-failure&per_page=100
    github-reporter (dry run): POST https://api.github.com/repos/octo-org/demo/issues
    github-reporter (dry run): POST https://api.github.com/repos/octo-org/demo/issues
    Output:  /tmp/github-reporter-atest-gb04dzel/tmp54vnbpbv/output.xml
    Log:     /tmp/github-reporter-atest-gb04dzel/tmp54vnbpbv/log.html
    Report:  /tmp/github-reporter-atest-gb04dzel/tmp54vnbpbv/report.html
    Getting file '<a href="file:///tmp/github-reporter-atest-gb04dzel/tmp54vnbpbv/github-requests.json">/tmp/github-reporter-atest-gb04dzel/tmp54vnbpbv/gi
== Run URL in GitHub Actions | logged msgs mentioning github-reporter: 4
== No run URL outside GitHub Actions | logged msgs mentioning github-reporter: 4
... (75 more lines)

The agent runs cd <repo>/openspec/changes/add-github-issues-listener && sed -i -E 's/^- \[ \] (5\.[1-4] )/- [x] \1/' tasks.md && grep -n '^- \[' tasks.md | sed -n '12,17p'

110:- [x] 5.1 Add `atest/_data/failing.robot` with two failing tests, one passing and one skipped, each failure with a
113:- [x] 5.2 Add `atest/github_issues.robot`. A keyword runs the failing suite through `Run Process` in an isolated
122:- [x] 5.3 Add a test with `GITHUB_ACTIONS=true`, `GITHUB_SERVER_URL`, `GITHUB_REPOSITORY` and `GITHUB_RUN_ID` set,
126:- [x] 5.4 Add a templated test that runs the failing suite with `--exitonerror` and each misconfiguration: no
136:- [ ] 6.1 Write `README.md`, covering:
146:- [ ] 6.2 Run the full checks, which must all pass:

The agent runs cd <repo> && env -u GITHUB_TOKEN -u GH_TOKEN -u GITHUB_ACTIONS uv run robot --pythonpath src --listener robotframework_github_reporter.GitHubIssues:repo=octo-org/demo --outputdir results atest/_data/failing.robot; echo "rc=$?"; cat results/github-requests.json

==============================================================================
Failing :: Run by `github_issues.robot` with the listener. The folder's `_`...
==============================================================================
Login Works                                                           | FAIL |
Expected 200 but got 500
------------------------------------------------------------------------------
Logout Works                                                          | FAIL |
Session still open
```not closed```
------------------------------------------------------------------------------
Passes                                                                | PASS |
------------------------------------------------------------------------------
... (55 more lines)

Agent:

The dry run works as documented: exit code 2, three recorded requests, and a fence that grows around the backticks in the message. Writing the README:

The agent writes <repo>/README.md

File created successfully at: <repo>/README.md (file state is

Agent:

Two fixes to the README. "It is not installed" is wrong inside this repo, because uv sync installs the package in editable mode. The table also needs aligning.

The agent edits <repo>/README.md

The file <repo>/README.md has been updated successfully. (fil

The agent edits <repo>/README.md

The file <repo>/README.md has been updated successfully. (fil

Agent:

Now I'll verify the README's commands by pulling every line out of its bash blocks and running each exactly as written:

The agent runs awk 'length > 120 && !/^(uv run robot --pythonpath| )/ {print "LONG: " FILENAME ":" NR}' README.md; awk -F'|' '/^\|/ {print length($0)}' README.md | sort -u; uv run python - <<'EOF' > /tmp/readme-cmds.txt import re text = open("README.md", encoding="utf-8").read() for block in re.findall(r"```bash\n(.*?)```", text, re.S): for line in block.splitlines(): if line.strip(): print(line) EOF cat /tmp/readme-cmds.txt; echo ---- while IFS= read -r cmd; do echo ">>> $cmd" bash -c "$cmd" > /tmp/readme-cmd.log 2>&1 < /dev/null; rc=$? tail -3 /tmp/readme-cmd.log echo "<<< rc=$rc" done < /tmp/readme-cmds.txt

LONG: README.md:11
LONG: README.md:26
LONG: README.md:27
LONG: README.md:28
LONG: README.md:29
LONG: README.md:30
124
uv run robot --pythonpath src --listener robotframework_github_reporter.GitHubIssues:repo=octo-org/demo --outputdir results atest/_data/failing.robot
uv sync
uv run pytest
uv run robot --outputdir results atest
uv build
... (25 more lines)

The agent edits <repo>/README.md

The file <repo>/README.md has been updated successfully. (fil

The agent edits <repo>/README.md

The file <repo>/README.md has been updated successfully. (fil

Agent:

The README commands are unchanged, so their verification still holds. Next I'll recheck line widths and mark 6.1, then run task 6.2's full checks from scratch:

The agent runs awk 'length > 120 && !/^uv run robot --pythonpath/ {print "LONG " NR ": " length}' README.md; sed -i -E 's/^- \[ \] (6\.1 )/- [x] \1/' openspec/changes/add-github-issues-listener/tasks.md echo "=== pytest"; uv run pytest -q 2>&1 | tail -1 echo "=== robot"; uv run robot --outputdir results atest > /tmp/atest.log 2>&1; echo "rc=$?"; grep -E "tests, .* passed" /tmp/atest.log | tail -1 echo "=== build"; uv build 2>&1 | tail -2 echo "=== validate"; openspec validate add-github-issues-listener --strict; echo "rc=$?" echo "=== dependencies"; uv run python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['dependencies'])" echo "=== forbidden in src/"; grep -rnE "importlib\.metadata|pkg_resources|\brequests\b|from urllib\.request import urlopen|logger\.error" src/ --include=*.py || echo "(no matches)"

LONG 74: 125
=== pytest
76 passed in 0.73s
=== robot
rc=0
5 tests, 5 passed, 0 failed
=== build
Successfully built dist/robotframework_github_reporter-0.1.0.tar.gz
Successfully built dist/robotframework_github_reporter-0.1.0-py3-none-any.whl
=== validate
Change 'add-github-issues-listener' is valid
rc=0
... (12 more lines)

Agent:

Every check passes. A plain search for requests matches only the dry-run list (self.requests) and prose, so I'll check imports specifically to confirm the requests library isn't used:

The agent runs grep -rnE "^\s*(import|from)\s" src/ --include=*.py; echo "--- requests-library imports:"; grep -rnE "^\s*(import requests|from requests)" src/ || echo "(none)"; sed -n 74p README.md | cut -c1-60

src/robotframework_github_reporter/listener.py:7:from __future__ import annotations
src/robotframework_github_reporter/listener.py:9:import json
src/robotframework_github_reporter/listener.py:10:import os
src/robotframework_github_reporter/listener.py:11:import re
src/robotframework_github_reporter/listener.py:12:from pathlib import Path
src/robotframework_github_reporter/listener.py:14:from robot.api import logger
src/robotframework_github_reporter/listener.py:15:from robot.libraries.BuiltIn import BuiltIn
src/robotframework_github_reporter/listener.py:17:from . import content
src/robotframework_github_reporter/listener.py:18:from .github import GitHub, GitHubError
src/robotframework_github_reporter/github.py:3:from __future__ import annotations
src/robotframework_github_reporter/github.py:5:import http.client
src/robotframework_github_reporter/github.py:6:import json
... (15 more lines)

The agent runs sed -i -E 's/^- \[ \] (6\.2 )/- [x] \1/' openspec/changes/add-github-issues-listener/tasks.md && openspec instructions apply --change add-github-issues-listener --json | uv run python -c "import json,sys; d=json.load(sys.stdin); print(d['state'], d['progress'])"

all_done {'total': 17, 'complete': 17, 'remaining': 0}

Agent:

Implementation Complete​

Change: add-github-issues-listener Schema: spec-driven Progress: 17/17 tasks complete ✓

Completed this session​

  • 5.1–5.4 Robot acceptance tests. atest/_data/failing.robot and atest/github_issues.robot were already written, but the main suite couldn't run: the dry run (--dryrun) failed 4 of its 5 tests. I fixed two bugs in atest/github_issues.robot:

    • ${ROOT} used $CURDIR inside an expression. ${CURDIR} is filled in when the file is parsed, so it isn't a variable at run time. It now uses r'${CURDIR}'.
    • Run Failing Suite took &{environment}. That made Robot read :repo=octo-org/demo as a named argument, so the keyword got no listener arguments. The GitHub Actions variables now live in an &{ACTIONS} dictionary passed as environment=${ACTIONS}.

    All 5 acceptance tests now pass. The inner runs use an environment holding only Robot Framework 7.5, and I confirmed the package can't be imported there without --pythonpath src. The failing suite is not collected by robot atest.

  • 6.1 README. It covers registering the listener, its arguments and defaults, how argument mistakes become warnings, the dry run and github-requests.json, and the token variables. It includes a GitHub Actions snippet with issues: write and lists the known limits. I ran every shell command in it as written: the demo run exits with code 2, as the README says, and the rest exit with 0. The Actions YAML can't be run locally, so it is untested.

  • 6.2 Final checks, all passing:

    • uv run pytest: 76 passed
    • uv run robot --outputdir results atest: 5 passed
    • uv build: wheel and source archive built
    • openspec validate --strict: valid
    • The only dependency is still robotframework==7.5.
    • src/ has no importlib.metadata, pkg_resources, requests library, from urllib.request import urlopen or logger.error.

The acceptance tests need uv's cache or network access to build their temporary environment, as the design already notes. I haven't committed anything.

All tasks are complete. You can archive this change with /opsx:archive.

The participant runs uv run pytest:

============================= test session starts ==============================
platform linux -- Python 3.12.11, pytest-9.1.1, pluggy-1.6.0
rootdir: <repo>
configfile: pyproject.toml
testpaths: tests
collected 76 items

tests/test_content.py ...............                                    [ 19%]
tests/test_github.py .......................                             [ 50%]
tests/test_listener.py ......................................            [100%]

============================== 76 passed in 0.76s ==============================

The participant runs uv run robot --outputdir results atest:

No run URL outside GitHub Actions                                     | PASS |
------------------------------------------------------------------------------
Live run without a token falls back to a dry run                      | PASS |
------------------------------------------------------------------------------
Misconfiguration is only a warning under --exitonerror                | PASS |
------------------------------------------------------------------------------
Atest.Github Issues :: Runs `_data/failing.robot` with the listene... | PASS |
5 tests, 5 passed, 0 failed
==============================================================================
Atest                                                                 | PASS |
5 tests, 5 passed, 0 failed
==============================================================================
... (3 more lines)

The participant runs uv build:

Building source distribution (uv build backend)...
Building wheel from source distribution (uv build backend)...
Successfully built dist/robotframework_github_reporter-0.1.0.tar.gz
Successfully built dist/robotframework_github_reporter-0.1.0-py3-none-any.whl

Step 8 - Point it at the workshop's suite​

The participant runs uv run robotcode robot --pythonpath ../robotframework-github-reporter/src --listener "robotframework_github_reporter.GitHubIssues:repo=your-handle/ai-engineering-robotframework" tests/ui/catalogue.robot:

------------------------------------------------------------------------------
WEB-002_AC-1 Card Prices Are The Product Prices :: The price on ev... | PASS |
------------------------------------------------------------------------------
WEB-002_AC-2 Categories Filter Group :: A "Categories" group offer... | PASS |
------------------------------------------------------------------------------
WEB-002_AC-4 Rating Filter :: A rating filter offers an unchecked ... | FAIL |
TimeoutError: locator.elementHandle: Timeout 10000ms exceeded.
Call log:
  - waiting for locator('role=complementary').locator('role=checkbox[name="4 stars and up"]')
------------------------------------------------------------------------------
WEB-002_AC-7 Audio Filter Shows Only Audio :: With only "Audio" ch... | PASS |
------------------------------------------------------------------------------
... (28 more lines)

The participant runs uv run --no-sync robotcode results summary | grep -E 'Total|Passed|Failed':

- _Total:_ 7
- _Passed:_ 5
- _Failed:_ 2

The participant runs head -c 1500 results/github-requests.json:

[
  {
    "method": "GET",
    "url": "https://api.github.com/repos/your-handle/ai-engineering-robotframework/issues?state=open&labels=robot-failure&per_page=100"
  },
  {
    "method": "POST",
    "url": "https://api.github.com/repos/your-handle/ai-engineering-robotframework/issues",
    "body": {
      "title": "Failing test: Catalogue.WEB-002_AC-4 Rating Filter",
      "body": "**Failure message**\n\n```text\nTimeoutError: locator.elementHandle: Timeout 10000ms exceeded.\nCall log:\n  - waiting for locator('role=complementary').locator('role=checkbox[name=\"4 
      "labels": [
... (16 more lines)

The participant runs uv run --no-sync robotcode robot tests/ui/catalogue.robot > /dev/null; uv run --no-sync robotcode results summary | grep -E 'Total|Passed|Failed':

- _Total:_ 7
- _Passed:_ 5
- _Failed:_ 2