feat: added self-hosted gitlab to catalog command #3867
Conversation
j2udev
requested review from
denis256,
levkohimins and
yhakbar
as code owners
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
WalkthroughThis pull request extends the functionality of the module URL generation to support self-hosted GitLab repositories. It adds a new regex constant and its compiled variable to detect self-hosted GitLab URLs, and adjusts the URL construction logic in the Changes
Sequence Diagram(s)sequenceDiagram
participant U as User
participant M as ModuleURL Method
participant R as Regex Checker
U->>M: Call ModuleURL with repo URL (self-hosted GitLab)
M->>R: Check against gitlabSelfHostedPatternReg
alt URL matches self-hosted pattern
M->>M: Construct self-hosted GitLab URL
else Standard GitLab URL
M->>M: Construct standard GitLab URL
end
M->>U: Return constructed URL
Assessment against linked issues
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
cli/commands/catalog/module/repo.go(3 hunks)cli/commands/catalog/module/repo_test.go(1 hunks)
🔇 Additional comments (2)
cli/commands/catalog/module/repo.go (2)
29-29: LGTM! Good regex pattern for self-hosted GitLab.The regex pattern
^(gitlab\.(.+))$correctly matches any domain beginning withgitlab., maintaining consistency with the GitHub Enterprise regex pattern.
159-161: Fix URL construction for self-hosted GitLab.Self-hosted GitLab instances use the same URL format as GitLab.com (
/-/tree/instead of/tree/). Let's reuse the GitLab URL format for consistency.Apply this diff to fix the URL construction:
if gitlabSelfHostedPatternReg.MatchString(string(remote.Host)) { - return fmt.Sprintf("https://%s/%s/tree/%s/%s", remote.Host, remote.FullName, repo.BranchName, moduleDir), nil + return fmt.Sprintf("https://%s/%s/-/tree/%s/%s", remote.Host, remote.FullName, repo.BranchName, moduleDir), nil }Let's verify this with GitLab's documentation:
✅ Verification successful
🌐 Web query:
What is the URL format for GitLab repository tree view?💡 Result:
The GitLab repository tree view URL follows a specific structure to navigate directories and files within a project. Here's how it works:
Standard Tree View URL Format
The basic pattern for accessing repository contents is:
https://gitlab.com/<namespace>/<project>/-/tree/<ref>/<path>Components explained:
namespace: User/group name (e.g.,gitlab-org)project: Repository name (e.g.,gitlab)ref: Branch name/tag/commit SHA (e.g.,main,v15.0.0, or20ac59ad)path: Directory path within repository (optional)Example Use Cases
Root directory of main branch:
https://gitlab.com/gitlab-org/gitlab/-/tree/mainSpecific subdirectory:
https://gitlab.com/gitlab-org/gitlab/-/tree/main/doc/apiCommit-specific tree:
https://gitlab.com/extra500/extra500/tree/20ac59adAPI Endpoint Structure
For programmatic access via GitLab API ([8]):
GET /projects/:id/repository/treeParameters:
Parameter Description Example idProject ID/path gitlab-org%2FgitlabrefBranch/tag/SHA mainpathDirectory path doc/apirecursiveShow nested files true/falseExample API request:
https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/tree?ref=main&path=doc/apiKey Observations
Web UI vs API:
- Web interface uses
/tree/path component- API uses
/repository/treeendpointSpecial Characters:
- Spaces become
%20- Slashes in paths remain unencoded
Redirect Behavior:
- Recent versions use permanent redirects (308) for optimized URLs[2]
- Unique IDs may appear in URLs when "unique domain" setting is enabled[2]
For developers working with repository paths programmatically, consider using relative paths where possible to avoid hardcoding instance-specific domains[4].
Citations:
- 1: https://docs.gitlab.com/ee/editor_extensions/visual_studio_code/remote_urls.html
- 2: https://docs.gitlab.com/ee/user/project/pages/getting_started_part_one.html
- 3: https://wiki.flightgear.org/Template:Gitlab_url
- 4: https://gitlab.com/gitlab-org/gitlab/-/issues/225849
- 5: https://developer.harness.io/docs/platform/connectors/code-repositories/ref-source-repo-provider/git-lab-connector-settings-reference/
- 6: http://www.bioinfotiget.it/gitlab/help/api/repositories.md
- 7: https://docs.gitlab.com/ee/development/documentation/site_architecture/folder_structure.html
- 8: https://docs.gitlab.com/ee/api/repositories.html
- 9: https://about.gitlab.com/blog/2023/08/30/changes-coming-to-url-structure-follow-deprecations-redirects/
- 10: https://gitlab.com/gitlab-org/gitlab/-/issues/19530
Self-hosted GitLab URL should use the /-/tree/ format.
The web documentation confirms that the correct URL format is:
https://gitlab.com/<namespace>/<project>/-/tree/<ref>/<path>This matches the diff provided in the review, so the corrected URL construction is on point.
- File: cli/commands/catalog/module/repo.go, lines 159-161
| { | ||
| "gitlab self-hosted", | ||
| newRepo(t, "https://gitlab.acme.com/acme/terraform-aws-modules"), | ||
| ".", | ||
| "https://gitlab.acme.com/acme/terraform-aws-modules/tree/main/.", | ||
| nil, | ||
| }, |
There was a problem hiding this comment.
Fix inconsistent URL format for self-hosted GitLab.
The test case expects a GitHub-style URL format (/tree/main/) for self-hosted GitLab, but GitLab consistently uses /-/tree/main/ across both hosted and self-hosted instances. Let's align this with GitLab's standard URL format.
Apply this diff to fix the URL format:
- "https://gitlab.acme.com/acme/terraform-aws-modules/tree/main/.",
+ "https://gitlab.acme.com/acme/terraform-aws-modules/-/tree/main/.",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| { | |
| "gitlab self-hosted", | |
| newRepo(t, "https://gitlab.acme.com/acme/terraform-aws-modules"), | |
| ".", | |
| "https://gitlab.acme.com/acme/terraform-aws-modules/tree/main/.", | |
| nil, | |
| }, | |
| { | |
| "gitlab self-hosted", | |
| newRepo(t, "https://gitlab.acme.com/acme/terraform-aws-modules"), | |
| ".", | |
| "https://gitlab.acme.com/acme/terraform-aws-modules/-/tree/main/.", | |
| nil, | |
| }, |
j2udev
force-pushed
the
feat/add-self-hosted-gitlab-to-catalog
branch
from
299e5c5 to
8621377
Compare
Description
Fixes #3800.
This PR just adds a regex for "self-hosted gitlab" matching any domain that begins with
gitlab.. The regex could probably be a bit better, but I just kept it consistent with the github enterprise regex because it also worked for my self-hosted instance.I've attached screenshots of the catalog command working with my self-hosted gitlab, although there isn't anything in the TUI that shows that so you'll have to take my word for it.
Something to note: for my specific, self-hosted gitlab repo, I'm using git submodules for all terraform/tofu modules under the
modulesdirectory. Subsequent executions of the catalog command are failing. Seems that go-getter can successfully clone the repo and pull the submodules on the first execution, but for whatever reason it's failing afterwards with this:Upon inspecting the .git folder of the repo, it doesn't have the modules folder as it should. This is a separate issue that I can file and potentially look into.
TODOs
Read the Gruntwork contribution guidelines.
Release Notes (draft)
Updated the
catalogcommand to accept self-hosted GitLab URLs.Summary by CodeRabbit
New Features
Tests