Support PodLevelResources

[tenzen-y]

What would you like to be added:
It would be great if we can support the PodLevelResources feature. Indeed, this feature is still alpha stage, but the APIs has already been added to k/k corev1 API.

Indeed, we have already started to use the PodRequests helper to calculate total Pod resources in #4177, which means we can calculate the requested Pod resources with PodLevelResources by passing something to the PodResourceOptions.

Once we have done the below tasks, we can confirm the PodLevelResources compatibility with Kueue.

• Add Unit / Integration testings by using Pod with .spec.resources (≠ .spec.containers.resources)
• Refine the total Pod resources calculation mechanism to support .spec.resources in the following:
  • https://github.com/kubernetes-sigs/kueue/blob/2738e8b05c03fe5dcf38beba1f4769fe2e1888c9/pkg/cache/tas_flavor.go
  • https://github.com/kubernetes-sigs/kueue/blob/main/pkg/util/limitrange/limitrange.go
  • https://github.com/kubernetes-sigs/kueue/blob/main/pkg/workload/workload.go

Note

We need to enable Kubernetes featureGate, PodLevelResources in kube-scheduler, kube-apiseerver, and kubelet to verify the behavior.

Why is this needed:
It would be better to support k/k new enhancements.

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.