HOSTEDCP-1960: Add KubeAPIExteralName api #5458

jparrill · 2025-01-22T14:39:33Z

This new API autogenerate a new kubeconfig based on a desired url and managed by HCP.

What this PR does / why we need it:
The current implementation of HCP to define the KAS address is using the ServicePublishingType, where you can define a hostname and that, will customize the API and API-INT interfaces where all the kubeconfigs (excepts the ones pointing to local) are pointing to. This makes the customization rigid and customers are asking for more flexibility.

This PR adds a new API which creates a new Kubeconfig (certificate based) pointing to the desired address. With this new API you can:

Modify the address day-1 and day-2 without affectation of the DataPlane (No rollout expected)
Generation of a new Kubeconfig based on certificates (managed by HCP)
The HC and HCP are in sync about this new Kubeconfig and reported in the status as customKubeconfig
The target of this new API are the scenarios of Public and PublicAndPrivate

⚠️ IMPORTANT! The external DNSs URL where you plan to point to and you will specify it in this new API, should be properly configured in order to makes this work.

Which issue(s) this PR fixes:

Fixes #HOSTEDCP-1960

Checklist

Subject and description added to both, commit and PR.
Relevant issues have been referenced.
This change includes docs.
This change includes unit tests.

openshift-ci-robot · 2025-01-22T14:39:37Z

@jparrill: This pull request references HOSTEDCP-1960 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

This new API changes the value of the Kubeconfig External URL to points to your desired one

What this PR does / why we need it:
HostedControlPlanes does not follows the separation of API and API-INT as OCP Standalone does. This PR includes a new API called KubeAPIExteralName which takes the desired name for the external KubeAPI Server name.

To have in mind:

This can be changed day-0 or day-2

This will not cause a nodepool rollout

This will generate a new Kubeconfig called apiext-admin-kubeconfig

This Kubeconfig should be bubbled up into HC namespace too

Which issue(s) this PR fixes:

Fixes #HOSTEDCP-1960

Checklist

Subject and description added to both, commit and PR.

Relevant issues have been referenced.

This change includes docs.

This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2025-01-22T14:39:37Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci · 2025-01-22T14:40:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jparrill

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jparrill]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2025-01-23T09:59:24Z

@jparrill: This pull request references HOSTEDCP-1960 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

This new API changes the value of the Kubeconfig External URL to points to your desired one

What this PR does / why we need it:
HostedControlPlanes does not follows the separation of API and API-INT as OCP Standalone does. This PR includes a new API called KubeAPIExteralName which takes the desired name for the external KubeAPI Server name.

To have in mind:

This can be changed day-0 or day-2

This will not cause a nodepool rollout

This will generate a new Kubeconfig called apiext-admin-kubeconfig

This Kubeconfig is bubbled up into HC namespace too

Which issue(s) this PR fixes:

Fixes #HOSTEDCP-1960

Checklist

Subject and description added to both, commit and PR.

Relevant issues have been referenced.

This change includes docs.

This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-01-23T10:01:18Z

@jparrill: This pull request references HOSTEDCP-1960 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

This new API changes the value of the Kubeconfig External URL to points to your desired one

What this PR does / why we need it:
HostedControlPlanes does not follows the separation of API and API-INT as OCP Standalone does. This PR includes a new API called KubeAPIExteralName which takes the desired name for the external KubeAPI Server name.

To have in mind:

This can be changed day-1 or day-2

This will not cause a nodepool rollout

This will generate a new Kubeconfig called apiext-admin-kubeconfig

This Kubeconfig is bubbled up into HC namespace too

Which issue(s) this PR fixes:

Fixes #HOSTEDCP-1960

Checklist

Subject and description added to both, commit and PR.

Relevant issues have been referenced.

This change includes docs.

This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

api/hypershift/v1beta1/hostedcluster_types.go

enxebre · 2025-01-27T10:52:33Z

HostedControlPlanes does not follows the separation of API and API-INT

Can we elaborate on this? Which topologies is this use case targeting: privateAndPublic, private, public?
For privateAndPublic we do have decouple internal and external urls routing to kas, so what's this solving?
Seems the PR is effectively implementing a kubeconfig autogeneration using a passed through URL, let's make sure that's clearly stated in the PR desc and API docs, also let's clarify how this is planned to be consumed and that consumption solves the driving rfe.

openshift-ci-robot · 2025-01-28T10:38:47Z

@jparrill: This pull request references HOSTEDCP-1960 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

This new API autogenerate a new kubeconfig based on a desired url and managed by HCP.

What this PR does / why we need it:
The current implementation of HCP to define the KAS address is using the ServicePublishingType, where you can define a hostname and that, will customize the API and API-INT interfaces where all the kubeconfigs (excepts the ones pointing to local) are pointing to. This makes the customization rigid and customers are asking for more flexibility.

This PR adds a new API which creates a new Kubeconfig (certificate based) pointing to the desired address. With this new API you can:

Modify the address day-1 and day-2 without affectation of the DataPlane (No rollout expected)

Generation of a new Kubeconfig based on certificates (managed by HCP)

The HC and HCP are in sync about this new Kubeconfig and reported in the status as customKubeconfig

The target of this new API are the scenarios of Public and PublicAndPrivate

⚠️ IMPORTANT! The external DNSs URL where you plan to point to and you will specify it in this new API, should be properly configured in order to makes this work.

Which issue(s) this PR fixes:

Fixes #HOSTEDCP-1960

Checklist

Subject and description added to both, commit and PR.

Relevant issues have been referenced.

This change includes docs.

This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

csrwng

Some nits in the API. In general, please use consistent case for "kubeconfig" wherever it appears in the API.

csrwng · 2025-01-29T14:25:21Z

api/hypershift/v1beta1/hosted_controlplane.go

+	// +optional
+	CustomKubeConfig *KubeconfigSecretRef `json:"customkubeconfig,omitempty"`
+
+	// kubeApiCustomName specifies the external name for the Kube API Server service.


s/kubeApiCustomName/kubeAPICustomName

csrwng · 2025-01-29T14:30:33Z

api/hypershift/v1beta1/hosted_controlplane.go

@@ -115,6 +115,10 @@ type HostedControlPlaneSpec struct {
 	// +optional
 	KubeConfig *KubeconfigSecretRef `json:"kubeconfig,omitempty"`

+	// KubeAPICustomKubeconfig is a group that encompasses an additional KAS server address and the customKubeconfig generated for the kube-apiserver.


csrwng · 2025-01-29T14:32:11Z

api/hypershift/v1beta1/hosted_controlplane.go

@@ -300,6 +324,12 @@ type HostedControlPlaneStatus struct {
 	// for this control plane.
 	KubeConfig *KubeconfigSecretRef `json:"kubeConfig,omitempty"`

+	// CustomKubeConfig specifies the name and key for the External Custom kubeconfig secret.


csrwng · 2025-01-29T14:32:35Z

api/hypershift/v1beta1/hostedcluster_types.go

@@ -432,6 +432,15 @@ type HostedClusterSpec struct {
 	// +required
 	Platform PlatformSpec `json:"platform"`

+	// kubeApiCustomName specifies the external name for the Kube API Server service.


s/kubeApiCustomName/kubeAPICustomName/

csrwng · 2025-01-29T14:32:50Z

api/hypershift/v1beta1/hostedcluster_types.go

@@ -1423,6 +1432,13 @@ type HostedClusterStatus struct {
 	// +optional
 	KubeConfig *corev1.LocalObjectReference `json:"kubeconfig,omitempty"`

+	// CustomKubeConfig specifies the name and key for the External Custom kubeconfig secret.


This new API changes the value of the Kubeconfig External URL to points to your desired one Signed-off-by: Juan Manuel Parrilla Madrid <[email protected]>

openshift-ci · 2025-01-29T19:21:40Z

@jparrill: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-e2e-aws-ovn	`22b1bdb`	link	false	`/test okd-scos-e2e-aws-ovn`
ci/prow/e2e-aks	`22b1bdb`	link	true	`/test e2e-aks`
ci/prow/e2e-aws-upgrade-hypershift-operator	`22b1bdb`	link	true	`/test e2e-aws-upgrade-hypershift-operator`
ci/prow/e2e-aws	`22b1bdb`	link	true	`/test e2e-aws`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

muraee · 2025-01-30T10:05:54Z

control-plane-operator/controllers/hostedcontrolplane/v2/kas/deployment.go

+func adaptCustomAdminKubeconfigSecret(cpContext component.WorkloadContext, secret *corev1.Secret) error {
+	hcp := cpContext.HCP
+
+	if hcp.Spec.KubeConfig != nil {


I assume you want hcp.Spec.KubeAPICustomKubeconfig != nil here, right?

muraee · 2025-01-30T10:06:04Z

control-plane-operator/controllers/hostedcontrolplane/v2/kas/deployment.go

@@ -90,6 +90,29 @@ func adaptDeployment(cpContext component.WorkloadContext, deployment *appsv1.Dep
 	return nil
 }

+func adaptCustomAdminKubeconfigSecret(cpContext component.WorkloadContext, secret *corev1.Secret) error {


please move this function to kubeconfig.go file

muraee · 2025-01-30T10:09:36Z

support/controlplane-component/common.go

+
+// EnableIfCustomKubeconfig is a helper predicate for the common use case of enabling a resource when a KubeAPICustomKubeconfig is specified.
+func EnableIfCustomKubeconfig() option {
+	return WithPredicate(func(cpContext WorkloadContext) bool {


I don't think any other component will use this predicate other than kas.
let's move it to the kas package where its used.
See this as an example https://github.com/muraee/hypershift/blob/karpenter-userdata/control-plane-operator/controllers/hostedcontrolplane/v2/cno/component.go#L47

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jan 22, 2025

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 22, 2025

openshift-ci bot added area/documentation Indicates the PR includes changes for documentation approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 22, 2025

jparrill force-pushed the HOSTEDCP-1960 branch from 90afcc8 to c866941 Compare January 22, 2025 15:49

openshift-ci bot added the area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release label Jan 22, 2025

jparrill force-pushed the HOSTEDCP-1960 branch 2 times, most recently from 5152979 to 0ecdf6a Compare January 24, 2025 17:37

enxebre reviewed Jan 27, 2025

View reviewed changes

api/hypershift/v1beta1/hostedcluster_types.go Outdated Show resolved Hide resolved

enxebre reviewed Jan 27, 2025

View reviewed changes

api/hypershift/v1beta1/hostedcluster_types.go Outdated Show resolved Hide resolved

jparrill force-pushed the HOSTEDCP-1960 branch from 0ecdf6a to d6e3182 Compare January 28, 2025 10:11

jparrill force-pushed the HOSTEDCP-1960 branch from d6e3182 to dfb1320 Compare January 28, 2025 15:23

jparrill marked this pull request as ready for review January 29, 2025 09:19

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 29, 2025

jparrill requested a review from enxebre January 29, 2025 09:19

openshift-ci bot requested a review from isco-rodriguez January 29, 2025 09:19

jparrill force-pushed the HOSTEDCP-1960 branch 2 times, most recently from f68c435 to 1fd6696 Compare January 29, 2025 11:10

csrwng reviewed Jan 29, 2025

View reviewed changes

HOSTEDCP-1960: Add KubeAPIExteralName api

22b1bdb

This new API changes the value of the Kubeconfig External URL to points to your desired one Signed-off-by: Juan Manuel Parrilla Madrid <[email protected]>

jparrill force-pushed the HOSTEDCP-1960 branch from 1fd6696 to 22b1bdb Compare January 29, 2025 15:16

muraee reviewed Jan 30, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HOSTEDCP-1960: Add KubeAPIExteralName api #5458

HOSTEDCP-1960: Add KubeAPIExteralName api #5458

jparrill commented Jan 22, 2025 •

edited

Loading

openshift-ci-robot commented Jan 22, 2025 •

edited by openshift-ci bot

Loading

openshift-ci bot commented Jan 22, 2025

openshift-ci bot commented Jan 22, 2025

openshift-ci-robot commented Jan 23, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jan 23, 2025 •

edited by openshift-ci bot

Loading

enxebre commented Jan 27, 2025

openshift-ci-robot commented Jan 28, 2025 •

edited by openshift-ci bot

Loading

csrwng left a comment

csrwng Jan 29, 2025

csrwng Jan 29, 2025

csrwng Jan 29, 2025

csrwng Jan 29, 2025

csrwng Jan 29, 2025

openshift-ci bot commented Jan 29, 2025

muraee Jan 30, 2025

muraee Jan 30, 2025

muraee Jan 30, 2025

HOSTEDCP-1960: Add KubeAPIExteralName api #5458

Are you sure you want to change the base?

HOSTEDCP-1960: Add KubeAPIExteralName api #5458

Conversation

jparrill commented Jan 22, 2025 • edited Loading

openshift-ci-robot commented Jan 22, 2025 • edited by openshift-ci bot Loading

openshift-ci bot commented Jan 22, 2025

openshift-ci bot commented Jan 22, 2025

openshift-ci-robot commented Jan 23, 2025 • edited by openshift-ci bot Loading

openshift-ci-robot commented Jan 23, 2025 • edited by openshift-ci bot Loading

enxebre commented Jan 27, 2025

openshift-ci-robot commented Jan 28, 2025 • edited by openshift-ci bot Loading

csrwng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Jan 29, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jparrill commented Jan 22, 2025 •

edited

Loading

openshift-ci-robot commented Jan 22, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jan 23, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jan 23, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jan 28, 2025 •

edited by openshift-ci bot

Loading