Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add JSON output for updateinfo #2200

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

walkerever
Copy link

This is a backporting for the feature introduced to dnf5 by the following pull requests:

The feature enables JSON format output for updateinfo command.

@pep8speaks
Copy link

pep8speaks commented Jan 27, 2025

Hello @walkerever! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 416:62: E502 the backslash is redundant between brackets
Line 417:65: E502 the backslash is redundant between brackets

Comment last updated at 2025-01-28 01:22:39 UTC

@walkerever walkerever force-pushed the json-output-updateinfo branch 10 times, most recently from 2325ee1 to 8fead9a Compare January 27, 2025 23:02
@walkerever
Copy link
Author

walkerever commented Jan 28, 2025

for pep8 check, seems always switching between W50x;) Pls advise if that's a hard blocker for the project. thanks.

A test about copr, enable/disable project failed, it can hardly be related to this change. But it seems I don't have permission just to rerun the test.

@walkerever walkerever force-pushed the json-output-updateinfo branch from 8fead9a to 036c511 Compare January 28, 2025 01:22
@walkerever
Copy link
Author

walkerever commented Jan 28, 2025

Known limit as best effort without introducing significant code changes:

  • status/vendor field not available as they're not present in the advisory object . Chose to live without them considering the cost to bring them in.
bash-5.2# dnf updateinfo --list --all ALAS2023-2024-518  --json
[
  {
    "name": "ALAS2023-2024-518",
    "type": "security",
    "severity": "Medium",
    "nevra": "vim-common-2:9.0.2153-1.amzn2023.x86_64",
    "buildtime": "2024-02-29 10:29:00"
  },
  {
    "name": "ALAS2023-2024-518",
    "type": "security",
    "severity": "Medium",
    "nevra": "vim-data-2:9.0.2153-1.amzn2023.noarch",
    "buildtime": "2024-02-29 10:29:00"
  },
  {
    "name": "ALAS2023-2024-518",
    "type": "security",
    "severity": "Medium",
    "nevra": "vim-enhanced-2:9.0.2153-1.amzn2023.x86_64",
    "buildtime": "2024-02-29 10:29:00"
  },
  {
    "name": "ALAS2023-2024-518",
    "type": "security",
    "severity": "Medium",
    "nevra": "vim-filesystem-2:9.0.2153-1.amzn2023.noarch",
    "buildtime": "2024-02-29 10:29:00"
  },
  {
    "name": "ALAS2023-2024-518",
    "type": "security",
    "severity": "Medium",
    "nevra": "vim-minimal-2:9.0.2153-1.amzn2023.x86_64",
    "buildtime": "2024-02-29 10:29:00"
  },
  {
    "name": "ALAS2023-2024-518",
    "type": "security",
    "severity": "Medium",
    "nevra": "xxd-2:9.0.2153-1.amzn2023.x86_64",
    "buildtime": "2024-02-29 10:29:00"
  }
]
bash-5.2# dnf updateinfo --info --all ALAS2023-2024-518  --json
{
  "ALAS2023-2024-518": {
    "Name": "ALAS2023-2024-518",
    "Title": "Amazon Linux 2023 - ALAS2023-2024-518: Medium priority package update for vim",
    "Severity": "Medium",
    "Type": "security",
    "Status": null,
    "Vendor": null,
    "Issued": "2024-02-29 10:29:00",
    "Description": "Package updates are available for Amazon Linux 2023 that fix the following vulnerabilities:\nCVE-2024-22667:\n\tVim before 9.0.2142 has a stack-based buffer overflow because did_set_langmap in map.c calls sprintf to write to the error buffer that is passed down to the option callback functions.\n\nCVE-2023-48706:\n\tVim is a UNIX editor that, prior to version 9.0.2121, has a heap-use-after-free vulnerability. When executing a `:s` command for the very first time and using a sub-replace-special atom inside the substitution part, it is possible that the recursive `:s` call causes free-ing of memory which may later then be accessed by the initial `:s` command. The user must intentionally execute the payload and the whole process is a bit tricky to do since it seems to work only reliably for the very first :s command. It may also cause a crash of Vim. Version 9.0.2121 contains a fix for this issue.\n",
    "Message": "",
    "Rights": null,
    "references": [
      {
        "Title": "",
        "Id": "CVE-2023-48706",
        "Type": "cve",
        "Url": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-48706"
      },
      {
        "Title": "",
        "Id": "CVE-2024-22667",
        "Type": "cve",
        "Url": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-22667"
      }
    ],
    "collections": {
      "packages": [
        "vim-enhanced-debuginfo-2:9.0.2153-1.amzn2023.x86_64",
        "xxd-2:9.0.2153-1.amzn2023.x86_64",
        "vim-default-editor-2:9.0.2153-1.amzn2023.noarch",
        "vim-minimal-2:9.0.2153-1.amzn2023.x86_64",
        "xxd-debuginfo-2:9.0.2153-1.amzn2023.x86_64",
        "vim-filesystem-2:9.0.2153-1.amzn2023.noarch",
        "vim-debuginfo-2:9.0.2153-1.amzn2023.x86_64",
        "vim-enhanced-2:9.0.2153-1.amzn2023.x86_64",
        "vim-data-2:9.0.2153-1.amzn2023.noarch",
        "vim-minimal-debuginfo-2:9.0.2153-1.amzn2023.x86_64",
        "vim-debugsource-2:9.0.2153-1.amzn2023.x86_64",
        "vim-common-2:9.0.2153-1.amzn2023.x86_64",
        "vim-enhanced-debuginfo-2:9.0.2153-1.amzn2023.aarch64",
        "vim-minimal-2:9.0.2153-1.amzn2023.aarch64",
        "xxd-2:9.0.2153-1.amzn2023.aarch64",
        "vim-enhanced-2:9.0.2153-1.amzn2023.aarch64",
        "vim-debuginfo-2:9.0.2153-1.amzn2023.aarch64",
        "vim-minimal-debuginfo-2:9.0.2153-1.amzn2023.aarch64",
        "xxd-debuginfo-2:9.0.2153-1.amzn2023.aarch64",
        "vim-debugsource-2:9.0.2153-1.amzn2023.aarch64",
        "vim-common-2:9.0.2153-1.amzn2023.aarch64"
      ]
    }
  }
}

@@ -157,6 +162,10 @@ def configure(self):
else:
self.opts.spec.insert(0, spec)

# Keep quiet when dumping JSON output
if self.opts.json:
self.cli.redirect_logger(stdout=sys.maxsize, stderr=sys.maxsize)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good but there still are some redundant messages in the output, namely I think the download progress bars.

We could additionally do something similar to what the --quiet option does: https://github.com/rpm-software-management/dnf/blob/master/dnf/cli/cli.py#L834-L836

To take effect I believe it would have to be set in pre_configure of the command.
I would add something like:

def pre_configure(self):
       if self.opts.json:
            self.base.conf.debuglevel = 0

Perhaps don't change the errorlevel since those message are more crucial.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack. will test this;)

Comment on lines +362 to +368
dtlst.append(
{
"name": aid,
"type": atype,
"severity": asev,
"nevra": nevra,
"buildtime": aupdated,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dnf5 has special logic when --with-bz or --with-cve is used: https://github.com/rpm-software-management/dnf5/blob/main/dnf5/commands/advisory/advisory_list.cpp#L75-L81
It adds references to the list subcommand and changes the IDs I think it would be good to be compatible as much as possible. Could you look into that?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, ack.

pkg_str += f".{pkg_info.get('arch')}"
package_list.append(pkg_str)

REFERENCE_TYPES = {0: 'unknown', 1: 'bugzilla', 2: 'cve', 3: 'vendor', 4: 'security'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is no reference type security.

Also you should not define the numbers here by hand, it would be better to use the exported hawkey constatns:

REFERENCE_TYPES = {hawkey.REFERENCE_UNKNOWN: 'unknown', hawkey.REFERENCE_BUGZILLA: 'bugzilla',
                   hawkey.REFERENCE_CVE: 'cve', hawkey.REFERENCE_VENDOR: 'vendor'}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

Comment on lines +355 to +356
asev = self.SECURITY2LABEL.get(sev, _('None'))
asev = asev.split("/")[0].strip()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a simpler solution would be to just do:

asev = sev

It is also what dnf5 does so the output will be more compatible.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for severity, for dnf updateinfo --list output, the format is Medium/Sec for cve type, and single word for others,

i FEDORA-2024-3c18fe0d93 Important/Sec. python-unversioned-command-3.13.1-2.fc41.noarch
i FEDORA-2025-e911f71d99 Moderate/Sec.  python-unversioned-command-3.13.2-1.fc41.noarch
i FEDORA-2024-3c18fe0d93 Important/Sec. python3-3.13.1-2.fc41.x86_64
i FEDORA-2025-e911f71d99 Moderate/Sec.  python3-3.13.2-1.fc41.x86_64
i FEDORA-2025-52b16605d4 bugfix         python3-argcomplete-3.5.3-1.fc41.noarch
i FEDORA-2025-397632c71b bugfix         python3-babel-2.17.0-1.fc41.noarch

while for dnf5 json output, it's simply Medium. example as below. reference rpm-software-management/dnf5#1531. that was the reason asev didn't copy from sev.

# dnf advisory list --json | head -20
[
  {
    "name":"FEDORA-2024-56efaa7783",
    "type":"enhancement",
    "severity":"Low",
    "nevra":"alternatives-1.31-1.fc41.x86_64",
    "buildtime":"2024-12-22 02:00:45"
  },
  {
    "name":"FEDORA-2025-9bef5569b2",
    "type":"enhancement",
    "severity":"None",
    "nevra":"audit-libs-4.0.3-1.fc41.x86_64",
    "buildtime":"2025-01-10 01:32:24"
  },
  {
    "name":"FEDORA-2024-4b75866373",
    "type":"bugfix",
    "severity":"Low",
    "nevra":"coreutils-9.5-11.fc41.x86_64",

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For dnf4 the list output contains a type (bugfix, enhancement, security..) and when the type is security it also contains the severity (Low, Moderate, Important...) so it combines them both.

With dnf5 json both type and severity are always present but they are separated. I think this is better for the json output.

advisories = set()
for apkg, advisory, installed in apkg_adv_insts:
advisories.add(advisory2info(advisory, installed))
dt_advisories.update(self._process_advisory(advisory))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer if we did this only when the self.opts.json is set otherwise its running always even when not used.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack.

@kontura
Copy link
Contributor

kontura commented Feb 10, 2025

for pep8 check, seems always switching between W50x;) Pls advise if that's a hard blocker for the project. thanks.

A test about copr, enable/disable project failed, it can hardly be related to this change. But it seems I don't have permission just to rerun the test.

I think you can make it work, just remove the backslashes but keep the indent:

                'Vendor': (getattr(advisory, 'vendor', None)
                           or getattr(advisory, 'author', None)

@kontura
Copy link
Contributor

kontura commented Feb 10, 2025

status/vendor field not available as they're not present in the advisory object . Chose to live without them considering the cost to bring them in.

Yes, I agree, I am only thinking if it makes sense to show the field if it will always be empty.
What about not adding it to the output at all?

@walkerever
Copy link
Author

status/vendor field not available as they're not present in the advisory object . Chose to live without them considering the cost to bring them in.

Yes, I agree, I am only thinking if it makes sense to show the field if it will always be empty. What about not adding it to the output at all?

100%;) let me remove them and resubmit.

This is a backporting for the feature introduced to dnf5 by the following
pull requests:
- rpm-software-management/dnf5#1531
- rpm-software-management/dnf5#1970

The feature enables JSON format output for updateinfo command.
@walkerever
Copy link
Author

for pep8 check, seems always switching between W50x;) Pls advise if that's a hard blocker for the project. thanks.
A test about copr, enable/disable project failed, it can hardly be related to this change. But it seems I don't have permission just to rerun the test.

I think you can make it work, just remove the backslashes but keep the indent:

                'Vendor': (getattr(advisory, 'vendor', None)
                           or getattr(advisory, 'author', None)

ack;) uh, vendor removed;)

@walkerever walkerever force-pushed the json-output-updateinfo branch from 6119ab0 to 9b59c70 Compare February 10, 2025 18:12
@walkerever
Copy link
Author

status/vendor field not available as they're not present in the advisory object . Chose to live without them considering the cost to bring them in.

Yes, I agree, I am only thinking if it makes sense to show the field if it will always be empty. What about not adding it to the output at all?

100%;) let me remove them and resubmit.

removed. tests re-triggered.

@walkerever
Copy link
Author

oh, spec got updated?

@walkerever
Copy link
Author

oh, spec got updated?

looks they were removed from updates. last good build has below,

  python3-hawkey                 x86_64 0.73.4-1.fc40     updates    305.9 KiB
  python3-libdnf                 x86_64 0.73.4-1.fc40     updates      3.8 MiB

I'll go with the code change for now;)

@kontura
Copy link
Contributor

kontura commented Feb 11, 2025

oh, spec got updated?

Yea, though I don't think you need to rebase. If you do need the new libdnf there are builds in https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf-nightly/ just be aware there is also dnf5 in there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants