Extend the ReclaimablePods feature to other built-in integrations

[mimowo]

What would you like to be added:

Currently the ReclaimablePods feature is only supported for batch/Jobs.

I would like to support it for all built-in integrations, unless for some this is not possible.

However, the graduation status of the KEP https://github.com/kubernetes-sigs/kueue/blob/main/keps/78-dynamically-reclaiming-resources/README.md is unclear. I would like to revisit the state, and review for which Jobs the support is possible.

Also, solve the related bug #3141 which may or may not require API update.

As we support the other Jobs we may need to protect the support by feature gates, so I would like to refresh the KEP and extend with the feature gates.

Why is this needed:

• to provide complete functionality to the users
• to solve the remaining bugs in the functionality
• to make a clear statement about the graduation of the feature.

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[mimowo]

cc @tenzen-y @mwielgus @mwysokin @dgrove-oss

[tenzen-y]

+1

[dgrove-oss]

It's been on our feature list for AppWrappers for a while (project-codeflare/appwrapper#51). The main hesitation was that only a handful of other integrations had implemented it so it didn't seem worth the effort yet.