Design System: Add AGENTS guidance for changelogs - #78166
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
|
||
| ## Changelogs | ||
|
|
||
| Add an entry to `CHANGELOG.md` for any change in this package. |
There was a problem hiding this comment.
This is not true, some changes (such as this PR!) don't justify a changelog entry. Maybe "for any change that may affect consumers"? Unit test and documentation changes are usually irrelevant.
There was a problem hiding this comment.
This is not true, some changes (such as this PR!) don't justify a changelog entry. Maybe "for any change that may affect consumers"? Unit test and documentation changes are usually irrelevant.
In fact, the original AI suggestion was to include "user-visible changes" only. I could get on board with that personally, though I think we need to be consistent. In particular, I seem to recall some conversations (and observations like in #77319) where we have changelogs for every pull request, even mostly "internal" changes.
I also noticed in #78169 that our CI testing file patterns exclude certain files, so whatever we decide should be in sync with how we test.
There was a problem hiding this comment.
For me, it's important to establish a rule that can be inferred/applied objectively. As far as we have clear guidelines for when to add and not to add an entry, I'm on board
There was a problem hiding this comment.
My only concern with the "user-visible changes" phrasing is clarifying who users are. In the case of gutenberg, users are end users, contributors, developers, etc.
There was a problem hiding this comment.
In my opinion, the primary audience for CHANGELOGs is developers who are either consuming a package or working on that package directly
| - One entry per PR, not per commit. If the branch already has a bullet under `## Unreleased` for this PR, edit it to cover any follow-up commits rather than adding a second bullet. | ||
| - Pick the right sub-heading per the canonical guidance in [`packages/README.md`](../README.md#maintaining-changelogs) (`Breaking Changes`, `Deprecations`, `New Features`, `Enhancements`, `Bug Fixes`, `Internal`, etc.). | ||
| - When the change is scoped to one or a small number of components, prefix the bullet with the component name(s) in backticks — e.g. `` `Button`: Fix … ``. Omit the prefix for broad cross-cutting changes (build config, shared utilities, repo-wide refactors). | ||
| - Once the PR exists, append `([#NNNNN](https://github.com/WordPress/gutenberg/pull/NNNNN))` to the end of the bullet. If the PR number isn't known yet, omit the parenthetical. |
There was a problem hiding this comment.
If the PR number isn't known yet
Not sure how this will work in practice… The changelog is not going to be complete until the PR number is filled out. Is it possible to make the agent follow up with it? Personally, I only ask it to add a changelog after the PR is created, since that saves a round trip.
There was a problem hiding this comment.
If the PR number isn't known yet
Not sure how this will work in practice… The changelog is not going to be complete until the PR number is filled out. Is it possible to make the agent follow up with it?
It would be nice, but it's not what I'd expect personally. And I'm not sure that we could automate this.
The phrasing here is kinda confusing, but the way I was thinking of it is:
- If a PR already exists, add the link
- If it doesn't exist, don't add the link
The original proposal created by the agent mentioned leaving a "(#TODO)" parenthetical, which I wasn't a fan of.
Maybe a simple change here is replacing "Once" with "If" ?
Personally, I only ask it to add a changelog after the PR is created, since that saves a round trip.
I can see how that workflow might work well, but I'm not really sure how we'd codify that in the agent guidance. Also, does it really save a round trip? It's two commits either way, right? I'd rather run the risk of a changelog entry not including its PR link than not having one at all. Honestly, I wish we could find some way to automate this PR link and not have it be something that contributors have to think about. For example, a GitHub Actions workflow that automatically commits the PR number into an existing changelog entry.
There was a problem hiding this comment.
I personally have my agent do it as part of "create a PR" task — it creates the CHANGELOG entry with a placeholder PR number, pushes to origin, inspects the PR number, amends the CHANGELOG and pushes again
There was a problem hiding this comment.
I personally have my agent do it as part of "create a PR" party — it creates the CHANGELOG entry with a placeholder PR number, pushes to origin, inspects the PR number, amends the CHANGELOG and pushes again
Maybe we can support this workflow through this file? Not necessarily saying that someone has to adopt this workflow for publishing a pull request, but if they're already using an agent to publish the pull request anyways, try to have it follow this procedure.
|
|
||
| - Add the entry under `## Unreleased` at the top of the file. If that heading doesn't exist (i.e. the previous release header is at the top), add it. | ||
| - One entry per PR, not per commit. If the branch already has a bullet under `## Unreleased` for this PR, edit it to cover any follow-up commits rather than adding a second bullet. | ||
| - Pick the right sub-heading per the canonical guidance in [`packages/README.md`](../README.md#maintaining-changelogs) (`Breaking Changes`, `Deprecations`, `New Features`, `Enhancements`, `Bug Fixes`, `Internal`, etc.). |
There was a problem hiding this comment.
What I sometimes observe is the agent picking a heading that clearly doesn't match the "Type" tag I've added to the PR. The most important distinction is between an enhancement, bug fix, and breaking change. Maybe instruct the agent to prioritize headings to match the author intent if enhancement, bug fix, or breaking change tags have already been added to the PR?
There was a problem hiding this comment.
Maybe instruct the agent to prioritize headings to match the author intent if enhancement, bug fix, or breaking change tags have already been added to the PR?
Yeah, that's a good idea 👍 Though it only works if a pull request is already opened by the time an agent is adding the changelog. And my expectation is that after this guidance, it'll be a lot more proactive about creating the changelog as part of the initial change (i.e. before the pull request).
|
|
||
| Don't forget to check a components's Storybook documentation for additional usage guidance. The Storybook links ([public base URL](https://wordpress.github.io/gutenberg/)) are also useful to present to a human when they are asking for help with a component. | ||
|
|
||
| ## Changelogs |
There was a problem hiding this comment.
Should we just have generic changelog instructions at packages/AGENTS.md root and refer to that file from here?
So:
packages/components/AGENTS.mdinstructs to add changelog always, but refers to parent agent file for more info.packages/AGENTS.mdhas instructions on how to add changelog entries.
There was a problem hiding this comment.
Yeah, I think you're right. I was a bit conservative with which package to apply this to, and already included some references to existing documentation (e.g. bullet about "canonical guidance in packages/README.md"), but re-reading many of these points, they could apply to any changelog authoring in any package, so would make sense in packages/AGENTS.md and/or enhancing existing changelog guidance in packages/README.md.
There was a problem hiding this comment.
I haven't got around to updating this yet, but @jeryj added a note to the root AGENTS.md to this effect. Though I think your original suggestion of packages/AGENTS.md makes sense and I left a post-merge review comment.
What?
Adds
AGENTS.mdguidance for design system packages (@wordpress/components,@wordpress/ui,@wordpress/theme) to describe expectation that each pull request should include a changelog note describing the changes.Separately, I'll plan to open a pull request that updates existing CI tooling to cover
@wordpress/uiand@wordpress/theme, in addition to@wordpress/components.Why?
For these packages, we've been intentional in encouraging changelog descriptions of any change being made, even going as far as CI workflows which fail on the absence of a changelog. For
@wordpress/uiand@wordpress/theme, since these packages are not bundled aswindow.wpglobals and follow Semantic Versioning, it's important that downstream consumers are kept informed about changes that might impact their usage of the package.Through CI, human-facing documentation, and inline code references, our expectations around changelogs are explicitly documented. But in practice, AI agents often skip this procedure, which requires manual intervention later, often in code review [1] [2] [3] [4]. This creates more work for both contributors and reviewers. And it's increasingly expected that AI agents are authoring much of the code contributed through pull requests these days.
AGENTS.mdis purpose-fit for intervening to correct these kinds of common AI agents mistakes.How
Adds or updates
AGENTS.mdfiles, appending a section for including changelog notes for changes. As much as possible, it references existing documentation to avoid duplicating information or prioritizing AI-facing documentation over human-facing documentation.Testing Instructions
You could test this locally with a sample change in one of the affected packages, both on this branch and
trunk, and see if the AI agent creates a CHANGELOG entry in each case. A successful test would produce the CHANGELOG entry on this branch and not ontrunk.Use of AI Tools
Research and initial implementation with Claude Code Opus 4.7, with iterations prompted through human review, and a final human edit of the produced content.