The SystemError description is misleading

[ZeroIntensity]

Documentation

The docs for SystemError currently say:

Raised when the interpreter finds an internal error, but the situation does not look so serious to cause it to abandon all hope. The associated value is a string indicating what went wrong (in low-level terms).

You should report this to the author or maintainer of your Python interpreter. Be sure to report the version of the Python interpreter (sys.version; it is also printed at the start of an interactive Python session), the exact error message (the exception’s associated value) and if possible the source of the program that triggered the error.

A SystemError isn't always an interpreter problem, at least in CPython's case. A SystemError can be caused relatively easily through misuse of the C API, so I don't think it's a good idea to point people to the issue tracker if they ever see a SystemError.

Linked PRs

• gh-129407: Clarify that a SystemError isn't always CPython's fault #129410

[ericvsmith]

I don't think it's worth changing this. We don't get a ton of reports of this nature, and when we do, we just say "you're using it wrong".

[ZeroIntensity]

But my point is that we shouldn't get any incorrect reports derived from the SystemError documentation--what's the harm in changing it?

[markshannon]

I don't think the docs are the problem, and we shouldn't change them.
The problem, IMO, is that SystemError is raised when it shouldn't be.

One example is when a builtin function sets the exception and returns a value, or returns NULL and doesn't set an exception.

@ZeroIntensity @hoodmane
Do you have other examples?

If the problem is extensions, not the VM, raising SystemError then we could, in theory, prevent that. But I doubt it would be worth it.

[markshannon]

Eurgh. There are loads of places we raise SystemError and probably shouldn't 🙁

Maybe we should add a new VMError class for actual "system errors"?

[hoodmane]

Yeah the most common cases are due to extension functions violating the calling conventions. There's also many public C APIs will raise a system error if their argument is a null pointer or otherwise their arguments fail some precondition.

I agree it would be better to fix the interpreter to distinguish between "the interpreter messed up" and "package code messed up". But that's a much larger project than this minor documentation adjustment.

Given that currently SystemErrors most frequently do result from mistakes in downstream code, I think updating the documentation to reflect this is an improvement over doing nothing and a lot less work than changing the behavior.

[markshannon]

Maybe it is not quite as bad as it first appears. A lot of the uses of SystemError seem to be in tests.

We could leave SystemError be and add a new ApiMisuseException?
Some places where ApiMisuseException could replace SystemError:

• Incorrectly setting the exception when returning from a builtin function, as described above
• PyBytes_FromStringAndSize, PyUnicode_FromStringAndSize, etc. with a negative size
• PyFile_WriteString with NULL file
• PyUnicode_New with an invalid character
• PyArg_Parse variants with invalid formats/arguments

[markshannon]

Given that currently SystemErrors most frequently do result from mistakes in downstream code, I think updating the documentation to reflect this is an improvement over doing nothing and a lot less work than changing the behavior.

It may well be less work, but I don't think it is the right thing to do long term.
We could change the docs temporarily, and change them back once we've added once we've added ApiMisuseException?

[markshannon]

https://discuss.python.org/t/add-a-new-apimisuseexception/78826

[vstinner]

See also #95799.