Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test the output of ScubaGear's basic commands #1538

Open
wants to merge 93 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
24bbb46
add dispatch
james-garriss Jan 21, 2025
67317a3
add push for testing
james-garriss Jan 21, 2025
09af0f3
temp replace secrets
james-garriss Jan 21, 2025
761cb7d
fix key vault
james-garriss Jan 21, 2025
f759803
write the info
james-garriss Jan 21, 2025
728a3b1
set specific info
james-garriss Jan 21, 2025
86bf6ad
add error
james-garriss Jan 21, 2025
c7849ce
fix name
james-garriss Jan 21, 2025
bca5ab6
fix errors
james-garriss Jan 21, 2025
a3be6fa
Correct newline
james-garriss Jan 21, 2025
4796791
Fix newline
james-garriss Jan 21, 2025
450f730
continue to next line
james-garriss Jan 21, 2025
9291a42
Another newline fix
james-garriss Jan 21, 2025
c3cea2f
another output
james-garriss Jan 21, 2025
a458ca0
remove whitespace
james-garriss Jan 21, 2025
96b0ae5
Clean up, should pass
james-garriss Jan 21, 2025
81e04bd
fix variable name
james-garriss Jan 21, 2025
3eb395d
check version
james-garriss Jan 21, 2025
3aba2ce
Add intentional name error
james-garriss Jan 21, 2025
ebe759d
add exit 1
james-garriss Jan 21, 2025
f994d30
fix exit code
james-garriss Jan 21, 2025
c969105
Fix one error.
james-garriss Jan 21, 2025
b20fbdd
get the module version
james-garriss Jan 21, 2025
869ffbc
use actual version
james-garriss Jan 21, 2025
27f9eb2
add comments to find version problem
james-garriss Jan 21, 2025
1601fde
comments for version
james-garriss Jan 22, 2025
9c8116d
add comment
james-garriss Jan 22, 2025
f0978d0
Restore comments and version
james-garriss Jan 22, 2025
0a02517
use write-debug
james-garriss Jan 22, 2025
899891f
use write-debug
james-garriss Jan 22, 2025
685fefc
switch to warning
james-garriss Jan 22, 2025
ccfd135
try black magic
james-garriss Jan 22, 2025
df841eb
try black magic
james-garriss Jan 22, 2025
3d1d456
Set correct version number
james-garriss Jan 22, 2025
2a6187e
check initiliaze output
james-garriss Jan 22, 2025
17c3e63
clarify output
james-garriss Jan 22, 2025
f9a9180
Fix typo
james-garriss Jan 22, 2025
dcaf216
Trivial change
james-garriss Jan 22, 2025
616231d
Remove comment from output
james-garriss Jan 27, 2025
17681c0
Add outputs for initialize.
james-garriss Jan 27, 2025
de86b44
removed write-host
james-garriss Jan 27, 2025
f7d2d4d
process the output
james-garriss Jan 27, 2025
1982593
replace Python w PowerShell.
james-garriss Jan 27, 2025
0066977
Put output up to Pester tests
james-garriss Jan 27, 2025
ff82f25
Update outputs
james-garriss Jan 27, 2025
19d34e5
fix global var name
james-garriss Jan 27, 2025
40baebb
Fix output var in Pester
james-garriss Jan 27, 2025
ebf2edc
test autotrust
james-garriss Jan 28, 2025
c1b1a47
lint
james-garriss Jan 28, 2025
3d7aadb
add key value tests
james-garriss Jan 28, 2025
9980f19
adjust pester test
james-garriss Jan 28, 2025
d9a864c
remove dir output
james-garriss Jan 28, 2025
2aca557
Comment dir
james-garriss Jan 28, 2025
2dcfc03
remove dir
james-garriss Jan 28, 2025
e0dae41
Continue hunt for dir
james-garriss Jan 28, 2025
bfb7293
lint
james-garriss Jan 28, 2025
669edbb
find hidden output
james-garriss Jan 28, 2025
0db083d
improve writes
james-garriss Jan 28, 2025
5857ea4
use value
james-garriss Jan 28, 2025
58af250
more debugging comments
james-garriss Jan 28, 2025
cbceac6
add comments
james-garriss Jan 28, 2025
a9bd4f9
Lint
james-garriss Jan 28, 2025
0ff26aa
comment
james-garriss Jan 28, 2025
2f16a6a
comment
james-garriss Jan 28, 2025
75b4f60
comment
james-garriss Jan 28, 2025
28c9d76
clear function results
james-garriss Jan 28, 2025
ccb4f79
undo comments
james-garriss Jan 28, 2025
48e1f2f
fix typo
james-garriss Jan 28, 2025
f916286
differentiate string from other types
james-garriss Jan 28, 2025
dddcf1a
list all output types
james-garriss Jan 28, 2025
e781b97
test for desired strings
james-garriss Jan 28, 2025
687077e
lint
james-garriss Jan 28, 2025
14bfa3f
separate it sections
james-garriss Jan 28, 2025
807b6e3
use global vars correctly
james-garriss Jan 28, 2025
70df10e
use global vars everywhere
james-garriss Jan 28, 2025
018c959
move tests
james-garriss Jan 28, 2025
8448962
Fix loop error
james-garriss Jan 29, 2025
fef328d
Tweak outputs
james-garriss Jan 29, 2025
3928dcc
trim the string
james-garriss Jan 29, 2025
6798463
remove testing code
james-garriss Jan 29, 2025
3abf71b
Clean up calling secrets
james-garriss Jan 29, 2025
15067bb
Remove old comment
james-garriss Jan 29, 2025
9b871fe
removed debugging statement
james-garriss Jan 29, 2025
63d7e46
Remove debugging
james-garriss Jan 29, 2025
06903ee
Remove unused comments
james-garriss Jan 29, 2025
e48e6e4
fix opa version
james-garriss Jan 29, 2025
810e8f7
fix write-warning
james-garriss Jan 29, 2025
7b5fe81
test write-verbose
james-garriss Feb 3, 2025
308c81e
Removed verbose
james-garriss Feb 3, 2025
a1d174a
Removed return
james-garriss Feb 3, 2025
12c0b44
fix extra braces
james-garriss Feb 3, 2025
27acd09
use env variable
james-garriss Feb 3, 2025
8213d12
Fixed comments and parameters
james-garriss Feb 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 37 additions & 4 deletions .github/workflows/publish_private_package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ jobs:
id: key-vault-info
env:
KEY_VAULT_INFO: ${{ secrets.KeyVaultInfo }}
# Do not attempt to access the Key Vault if logging into Azure failed.
if: ${{ success() }}
run: |
$KeyVaultInfo = ${env:KEY_VAULT_INFO} | ConvertFrom-Json
echo "KeyVaultUrl=$($KeyVaultInfo.KeyVault.URL)" >> $env:GITHUB_OUTPUT
Expand All @@ -65,6 +67,7 @@ jobs:
cd repo
New-PrivateGallery -GalleryName $env:GalleryName -Trusted
- name: Sign and Publish Module
id: sign-publish-module
run: |
# Source the deploy utilities so the functions in it can be called.
. repo/utils/workflow/Publish-ScubaGear.ps1
Expand All @@ -80,7 +83,9 @@ jobs:
# This publishes to a private gallery, from which we will pull in a future step.
# This step helps us verify that ScubaGear is in a state where it can be
# published to PSGallery.
Publish-ScubaGearModule @Parameters
$ModuleVersion = Publish-ScubaGearModule @Parameters
Write-Output "The module version is $ModuleVersion"
echo "ModuleVersion=$ModuleVersion" >> $env:GITHUB_OUTPUT
- name: Test Module Publish
run: |
Get-Location
Expand All @@ -98,12 +103,40 @@ jobs:
Invoke-Pester -Configuration $Config
- name: Initialize ScubaGear
run: |
Install-Module -Name ScubaGear -Repository $env:GalleryName -SkipPublisherCheck
# These are two of the expected output values that result from installing the module.
# These checks are intended to help verify that the module installed correctly.
$ExpectedName = 'ScubaGear'
$ExpectedDescription = 'The Secure Cloud Business Applications (SCuBA) Gear module automates conformance testing about CISA M365 Secure Configuration Baselines.'
Comment on lines +108 to +109
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would these make more sense as env vars so they are easier to change if/when they are adjusted? Or read directly from the manifest?

Copy link
Collaborator Author

@james-garriss james-garriss Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description is found in the manifest. I think it should be possible to read from it. The name is not found in the manifest, so it would need to be defined here (or possibly be read from some other file).

I have no idea if using an env var make more sense. Variables are variables however you create and adjust them.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a DRY thing. If we decided to change the name of the tool to ScubaSomethingElse, and it was read from the manifest then you wouldn't need to update the workflow to match. So that's why I think it makes more sense. Even if it's only for some of the things at the moment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this is a good idea and that I should attempt to do so. I think this work should be done before approving this issue, so I am going to push this into Marlin and do it there.

# The -PassThru paramaters allows us to read the output
# value from Install-Module
$InstallOutput = Install-Module -Name ScubaGear -Repository $env:GalleryName -SkipPublisherCheck -PassThru
# Check for the expected name.
if ($InstallOutput.Name -ne $ExpectedName) {
Write-Output "::error::The name of the published module should be $ExpectedName"
exit 1
}
# Because reasons, the description has a newline character.
# Strip all whitespace before comparing the descriptions.
$DescriptionFixed = $InstallOutput.Description -replace "\s", ""
$ExpectedFixed = $ExpectedDescription -replace "\s", ""
# Check for the expected description.
if ($DescriptionFixed -ne $ExpectedFixed) {
Write-Output "::error::The description of the published module should be $ExpectedDescription"
exit 1
}
# Source the function
. repo/utils/workflow/Initialize-ScubaGearForTesting.ps1
Initialize-ScubaGearForTesting
# This is a manual test that simply writes the version to the console.
Invoke-SCuBA -Version
# This is the expected output value that results from running the module.
# This check is intended to help verify that ScubaGear installed correctly.
$VersionOutput = Invoke-SCuBA -Version
$ExpectedVersion = "SCuBA Gear v${{ steps.sign-publish-module.outputs.ModuleVersion }}"
Write-Output "The version is $VersionOutput"
Write-Output "The expected version is $ExpectedVersion"
if ($VersionOutput -ne $ExpectedVersion) {
Write-Output "::error::The version of the published module should be $ExpectedVersion"
exit 1
}
Test-Path -Path "C:\Program Files\WindowsPowerShell\Modules\ScubaGear"
- name: Install Selenium
run: |
Expand Down
Loading