Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Describe/Update Changelog command fails to mention changes to ignored & bad extension files #1240

Open
MarkRx opened this issue Sep 20, 2024 · 1 comment

Comments

@MarkRx
Copy link
Contributor

MarkRx commented Sep 20, 2024

The generated description/changelog doesn't account for ignored and "bad extension" files. Additionally, if I make a PR that only changes some images the describe/update changelog command doesn't complete due to every file being ignored.

I think in this case it's valid to know these files changed (even if the LLM won't know what changed in them) so that it can be mentioned in the PR description or changelog.

The PR diff logic would need to be updated to mention every file but only show diff contents as desired.

Note there are two meanings of "ignore" now - "ignore" as in "don't compute a diff and report on changes" and "ignore" as in "never consider these files". The configuration would need to reflect that somehow.

Example LLM input:

PR Info:

Previous title: 'some title'

Previous description:

=====

nothing

=====

Branch: 'my-test'

The PR Diff:

=====

## file 'test1.png' was added

## file 'test-foo.png' was added

## file 'test-bar.png' was added

## file 'test-bla.png' was added

## file 'test-cat.png' was added

## file 'test-dog.png' was added

## file 'test9.png' was updated

## file 'test10.png' was deleted

=====

Note that lines in the diff body are prefixed with a symbol that represents the type of change: '-' for deletions, '+' for additions, and ' ' (a space) for unchanged lines.

Response (should be a valid YAML, and nothing else):

```yaml

Example output

type:
- Other
description: |-
  - Added new image files: `test1.png`, `test-foo.png`, `test-bar.png`, `test-bla.png`, `test-cat.png`, and `test-dog.png`.
  - Updated image file: `test9.png`.
  - Deleted image file: `test10.png`.
title: |-
  Add and update image files
@mrT23
Copy link
Collaborator

mrT23 commented Sep 22, 2024

I agree that there is a gap here. You are right

some stream-of-thought about it, since it's not an easy issue to solve:

When thinking about how to address it, I think of three questions:

  1. how hard is it to solve it ?
  2. can we solve it completely?
  3. what is the current level of bad impact ?

let's start with Q2:
some files like .png,'.zip and others, are worth mentioning. other files, like auto-generated ones, are not as important
It can be quite challenging to distinguish between the two types. it's not a clear cut.

Q1:
It will require a significant change to pass also the raw files list, from all git providers, to the relevant tools prompt. Not impossible, but a significant change.

Q3:
Yeah, the update-changlog may skip some ignored files. is it really that bad ? Since the AI cannot know what was changed inside them anyway, wouldn't the user anyway need to adjust the output a bit in those cases ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants