Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test the output of ScubaGear's basic commands #1538

Open
wants to merge 93 commits into
base: main
Choose a base branch
from

Conversation

james-garriss
Copy link
Collaborator

@james-garriss james-garriss commented Jan 29, 2025

🗣 Description

Added tests for checking the textual outputs that result from running ScubaGear's most basic commands:

Install-Module -Name ScubaGear
Initialize-SCuBA = check for outputs
Invoke-SCuBA -Version

The idea is to add additional checks to make sure SG is running as expected.

💭 Motivation and context

Closes #1490

🧪 Testing

The tests for Initialize-Scuba can be found by running the workflow pipeline and checking under Workflow Units Tests for the Pester Test called "Describing Initialize-ScubaGear Output Check." There should be 3 successful tests:

  • PSGallery should be trusted
  • OPA should be downloaded
  • Setup time should be minimal

The tests for Install-Module can be found by running the nightly functional tests (test_production_function.yaml), finding the first job (publish package/private repo), and checking under Initialize ScubaGear. There should not be any errors that start with:

  • The name of the published module should be....
  • The description of the published module should be...
  • The version of the published module should be...

These would create annotation errors, so they should be easy to see.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • PR targets the correct parent branch (e.g., main or release-name) for merge.
  • Changes are limited to a single goal - eschew scope creep!

I did do some extra cleaning up while wandering around, like fixing input parameter for function help comments, removing random newlines, and setting better outputing.

  • Changes are sized such that they do not touch excessive number of files.
  • These code changes follow the ScubaGear [content style guide]
  • Related issues these changes resolve are linked preferably via [closing keywords]
  • All relevant type-of-change labels added.
  • All relevant project fields are set.
  • Unit tests added/updated to cover PowerShell and Rego changes.
  • All relevant functional tests passed.
  • All automated checks (e.g., linting, static analysis, unit/smoke tests) passed.

✅ Pre-merge checklist

  • PR passed smoke test check.

  • Feature branch has been rebased against changes from parent branch, as needed

    Use Rebase branch button below or use this reference to rebase from the command line.

  • Resolved all merge conflicts on branch

  • Notified merge coordinator that PR is ready for merge via comment mention

  • Demonstrate changes to the team for questions and comments.
    (Note: Only required for issues of size Medium or larger)

✅ Post-merge checklist

  • Feature branch deleted after merge to clean up repository.
  • Verified that all checks pass on parent branch (e.g., main or release-name) after merge.

Copy link
Collaborator

@schrolla schrolla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added some follow-ups to previous comments, but overall looks good to go.

Comment on lines 57 to 58
# Stop the workflow if logging into Azure failed.
if: ${{ success() }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure the comment lines up with the step. Since this makes the run conditional, on a fail, won't it just skip the step to get vault info, but continue running other steps?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you are correct. Comment fixed.

Comment on lines +106 to +107
$ExpectedName = 'ScubaGear'
$ExpectedDescription = 'The Secure Cloud Business Applications (SCuBA) Gear module automates conformance testing about CISA M365 Secure Configuration Baselines.'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a DRY thing. If we decided to change the name of the tool to ScubaSomethingElse, and it was read from the manifest then you wouldn't need to update the workflow to match. So that's why I think it makes more sense. Even if it's only for some of the things at the moment.

@james-garriss james-garriss modified the milestones: Lionfish, Marlin Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement This issue or pull request will add new or improve existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Check output of basic ScubaGear commands
2 participants