v2.13 - Tin Puppy

v2.13 - Tin Puppy - [2024-02-20]

Highlights 🌟

This new version of nf-core/tools includes a refactoring of the pipelines template.

• The lib directory is removed: 👋 Groovy code
• We now use nf-core subworkflows for pipeline initialisation (replacing the code from the lib directory)
• The nf-validation plugin is now used to create an input channel from a sample sheet file.

Check the nf-core blog post for more information!

Template

• Add empty line in README.md to fix badges. (#2729)
• Replace automatic branch detection in nf-core download CI test with hardcoded dev and input. (#2727)
• Add Github Action to automatically cleanup ubuntu-latest runners to fix runner running out of diskspace errors(#2755)
• Fix GitHub Actions CI and Linting badges links (#2757)
• Add hashtags to release announcement on mastodon (#2761)
• update fastqc and multiqc in template (#2776)
• template refactoring: remove the lib directory and use nf-core subworkflows (#2736)
• use nf-validation to create an input channel from a sample sheet (#2736)

Linting

• Make creat-lint-wf composable (#2733)
• Add looser comparison when pipeline logos (#2744)
• Handle multiple aliases in module imports correctly during linting (#2762)
• Switch to markdown based API and error docs (#2758)

Modules

• Handle dirty local module repos by force checkout of commits and branches if needed (#2734)
• Patch: handle file not found when it is an added file to a module (#2771)
• Handle symlinks when migrating pytest (#2770)
• Add --profile parameter to nf-test command (#2767)

General

• fix ignoring changes in partially templated files (e.g. .gitignore) (#2722)
• update ruff to 0.2.0 and add it to pre-commit step (#2725)
• Update codecov/codecov-action digest to e0b68c6 (#2728)
• Update pre-commit hook astral-sh/ruff-pre-commit to v0.2.1 (#2730)
• Update python:3.11-slim Docker digest to 2a746e2 (#2743)
• Update actions/setup-python action to v5 (#2739)
• Update gitpod/workspace-base Docker digest to 45e7617 (#2747)
• chore(deps): pin jlumbroso/free-disk-space action to 54081f1 (#2756)
• chore(deps): update actions/github-script action to v7 (#2766)
• chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.2.2 (#2769)
• Update gitpod/workspace-base Docker digest to 728e1fa (#2780)

v2.12.1 - Aluminium Wolf - Patch

v2.12.1 - Aluminium Wolf - Patch - [2024-02-01]

Linting

• Handle default values of type number from nextflow schema (#2703)
• fix ignoring files_unchanged (#2707)

General

• Update pre-commit hook astral-sh/ruff-pre-commit to v0.1.15 (#2705)
• use types for default value comparison (#2712)
• fix changelog titles (#2708)
• Print relative path not absolute path in logo cmd log output (#2709)
• Update codecov/codecov-action action to v4 (#2713)
• Ignore nf-core-bot in renovate PRs (#2716)

2.12 - Aluminium Wolf

v2.12 - Aluminium Wolf - [2024-01-29]

Template

• Add a Github Action Workflow to the pipeline template that tests a successful download with 'nf-core download' (#2618)
• Use pre-commit to lint files in GitHub CI (#2635)
• Use pdiff also on gitpod for nf-test (#2640)
• switch to new image syntax in readme (#2645)
• Add conda channel order to nextflow.config (#2094)
• Fix tyop in pipeline nextflow.config (#2664)
• Remove nfcore_external_java_deps.jar from lib directory in pipeline template (#2675)
• Add function to check -profile is well formatted (#2678)
• Add new pipeline error message pointing to docs when 'requirement exceeds available memory' error message (#2680)
• add 👀👍🏻🎉😕 reactions to fix-linting-bot action (#2692)

Linting

• Fix linting of a pipeline with patched custom module (#2669)
• linting a pipeline also lints the installed subworkflows (#2677)
• environment.yml name must be lowercase (#2676)
• allow ignoring specific files when template_strings (#2686)
• lint nextflow.config default values match the ones specified in nextflow_schema.json (#2684)

Modules

• Fix empty json output for nf-core list local (#2668)

General

• Run CI-pytests for nf-core tools on self-hosted runners (#2550)
• Add Ruff linter and formatter replacing Black, isort and pyupgrade (#2620)
• Set pdiff as nf-test differ in Docker image for Gitpod (#2642)
• Fix Renovate Dockerfile updating issues (#2648 and #2651)
• Add new subcommand nf-core tui, which launches a TUI (terminal user interface) to intuitively explore the command line flags, built using Trogon (#2655)
• Add new subcommand: nf-core logo-create to output an nf-core logo for a pipeline (instead of going through the website) (#2662)
• Handle api redirects from the old site (#2672)
• Remove redundanct v in pipeline version for emails (#2667)
• add function to check -profile is well formatted (#2678)
• Update pre-commit hook astral-sh/ruff-pre-commit to v0.1.14 (#2674)
• Update pre-commit hook pre-commit/mirrors-mypy to v1.8.0 (#2630)
• Update mshick/add-pr-comment action to v2 (#2632)
• update python image version in docker file (#2636)
• Update actions/cache action to v4 (#2666)
• Update peter-evans/create-or-update-comment action to v4 (#2683) and (#2695)

Magnesium Dragon Patch

v2.11.1 - Magnesium Dragon Patch - [2023-12-20]

Template

• Rename release-announcments.yml to release-announcements.yml (#2610)
• Fix nextflow.config docker.runOptions (#2607)

General

• Only dump modules.json when it is modified (#2609)

2.11

v2.11 - Magnesium Dragon + [2023-12-19]

Highlights:

• nf-test is now used for testing modules and subworkflows. 🚀 This comes with updated templates and a changed commands (good-bye nf-core modules/subworkflows create-test-yml). For more information, you can refer to the revised documentation on modules and subworkflows .
  • Every module and subworkflow now contains a tests/ directory, which includes a main.nf.test file. By running nf-core modules test or nf-core subworkflows test, you can perform the tests and generate the main.nf.test.snap snapshot file. As a general guideline, we recommend testing all outputs of modules/subworkflows. For complex cases, please refer to the nf-test assertion tutorial.
  • To assist with the conversion of modules and subworkflows from pytest to nf-test, we've added the new --migrate-pytest option for nf-core modules test and nf-core subworkflows test, which does most of the work for you.
  • The nf-core modules lint and nf-core subworkflows lint commands now also check the nf-test configuration and warn you if you are using pytest (it's really not that hard to switch 🙂 ).
• We pinned the version of nf-validation, which should make offline usage of nf-core pipelines easier.
• In nf-core download, we now prefix all absolute container URIs with docker://, which should make more pipelines work out of the box.
• We started using type hinting in the nf-core/tools codebase. This will help us to find bugs earlier and make the code more readable.

The complete list of changes can be found below.

Template

• Fix writing files to a remote outdir in the NfcoreTemplate helper functions (#2465)
• Fancier syntax highlighting for example samplesheets in the usage.md template (#2503)
• Use closure for multiqc ext.args (#2509)
• Fix how the modules template references the conda environment file (#2540)
• Unset env variable JAVA_TOOL_OPTIONS in gitpod (#2569)
• Pin the version of nf-validation (#2579)
• Disable process selector warnings by default (#2161)
• Remove docker.userEmulation from nextflow.config in pipeline template (#2580)

Download

• Add docker:// prefix for absolute container URIs as well (#2576).
• Bugfix for AttributeError: ContainerError object has no attribute absoluteURI (#2543).

Linting

• Fix incorrectly failing linting if 'modules' was not found in meta.yml (#2447)
• Correctly pass subworkflow linting test if COMPONENT.out.versions is used in the script (#2448)
• Add pyupgrade to pre-commit config and dev requirements as mentioned in #2200
• Check for spaces in modules container URLs (#2452)
• Correctly ignore timeline.enabled, report.enabled, trace.enabled, dag.enabled variables when linting a pipeline. (#2507)
• Lint nf-test main.nf.test tags include all used components in chained tests (#2572)
• Don't fail linting if md5sum for empty files are found in a stub test (#2571)
• Check for existence of test profile (#2478)

Modules

• Added stub test creation to create_test_yml (#2476)
• Replace ModulePatch by ComponentPatch (#2482)
• Fixed nf-core modules lint to work with new module structure for nf-test (#2494)
• Add option --migrate-pytest to create a module with nf-test taking into account an existing module (#2549)
• When installing modules and subworkflows, automatically create the ./modules directory if it doesn't exist (#2563)
• When .nf-core.yml is not found create it in the current directory instead of the root filesystem (#2237)
• Modules --migrate-pytest copies template scripts (#2568)

Subworkflows

• Added stub test creation to create_test_yml (#2476)
• Fixed nf-core subworkflows lint to work with new module structure for nf-test (#2494)
• Add option --migrate-pytest to create a subworkflow with nf-test taking into account an existing subworkflow (#2549)

General

• Update schema build functionality to automatically update defaults which have changed in the nextflow.config(#2479)
• Change testing framework for modules and subworkflows from pytest to nf-test (#2490)
• bump_version keeps now the indentation level of the updated version entries (#2514)
• Add mypy to pre-commit config for the tools repo (#2545)
• Use Path objects for ComponentCreate and update the structure of components templates (#2551).
• GitPod base image: swap tool installation back to conda from mamba (#2566).
• Sort the installed_by list in modules.json (#2570).
• Unset env variable JAVA_TOOL_OPTIONS in gitpod (#2569)

New Contributors

• @fa2k made their first contribution in #2472
• @NovakApis made their first contribution in #2223
• @rernst made their first contribution in #2507
• @kedhammar made their first contribution in #2483

v2.10 - Nickel Ostrich

v2.10 - Nickel Ostrich + [2023-09-25]

This new release includes new commands, template updates, and several bug fixes. Thanks to all contributors for making this possible!

🌟 Highlights

• We are now linting subworkflows nf-core subworkflows lint.
• Use nf-core create-params-file params-file with documentation and default values for all parameters in a pipeline.
• A new GitHub Action will automatically post new pipeline release messages to Mastodon, Bluesky and Twitter
• Pipelines in sync with the new template, will dump all pipeline parameters to a file called params.json after every run

✖️ Deprecations

• Deprecation of the —no-git argument for nf-core create command
• Deprecation of the —template-yaml argument for the nf-core sync command - The same information will now be automatically stored in .nf-core.yml

Template

• Fix links in multiqc_config.yml (#2372 and #2412)
• Remove default false from nextflow_schema.json (#2376)
• Add module MULTIQC to modules.config (#2377)
• Add GitHub workflow for automated release announcements (#2382)
• Update the Code of Conduct (#2381)
• Save template information to .nf-core.yml and deprecate argument --template-yaml for nf-core sync (#2388 and #2389)
• (#2397) Remove fixed Ubuntu test and added to standard testing matrix
• (#2396) Reduce container finding error to warning since the registries are not consistent.
• (#2415) Add autoMounts for apptainer.
• Remove igenomes_base from the schema, so that nf-validation doesn't create a file path and throw errors offline for s3 objects.
• Modified devcontainer permissions so that singularity can be run in Codespaces/VS Code devcontainers (Commit a103f44)
• Update Gitpod profile resources to reflect base environment settings.
• (#747) Add to the template the code to dump the selected pipeline parameters into a json file.

Download

• Improved container image resolution and prioritization of http downloads over Docker URIs (#2364).
• Registries provided with -l/--container-library will be ignored for modules with explicit container registry specifications (#2403).
• Fix unintentional downloading of containers in test for the Tower download functionality. Bug reported by @adamrtalbot and @awgymer (#2434).

Linting

• Add new command nf-core subworkflows lint (#2379)

Modules

Subworkflows

• Fix bug: missing subworkflow name when using nf-core subworkflows create (#2435)

General

• Initialise docker_image_name to fix UnboundLocalError error (#2374)
• Fix prompt pipeline revision during launch (#2375)
• Add a create-params-file command to create a YAML parameter file for a pipeline containing parameter documentation and defaults. (#2362)
• Update the Code of Conduct (#2381)
• Remove --no-git option from nf-core create (#2394)
• Throw warning when custom workflow name contains special characters (#2401)
• Bump version of nf-test snapshot files with nf-core bump-version (#2410)

v2.9 - Chromium Falcon

v2.9 - Chromium Falcon + [2023-06-30]

A new big release with new features, template updates and bug fixes. Thanks to all contributors!

🌟 Highlights

• A new nf-core downloadcommand compatible with Nextflow Tower.
• Implementation of the plugin nf-validation for parameter validation in pipeline templates.
• Nextflow version bump to 23.04.0.

Template

• params.max_multiqc_email_size is no longer required (#2273)
• Remove cleanup = true from test_full.config in pipeline template (#2279)
• Fix usage docs for specifying params.yaml (#2279)
• Added stub in modules template ([#2277])(#2277) [Contributed by @nvnieuwk]
• Move registry definitions out of profile scope ([#2286])(#2286)
• Remove aws_tower profile ([#2287])(#2287)
• Fixed the Slack report to include the pipeline name (#2291)
• Fix link in the MultiQC report to point to exact version of output docs (#2298)
• Updates seqeralabs/action-tower-launch to v2.0.0 (#2301)
• Remove schema validation from lib folder and use Nextflow nf-validation plugin instead (#1771)
• Fix parsing of container directive when it is not typical nf-core format (#2306)
• Add ability to specify custom registry for linting modules, defaults to quay.io (#2313)
• Add singularity.registry = 'quay.io' in pipeline template (#2305)
• Add apptainer.registry = 'quay.io' in pipeline template (#2352)
• Bump minimum required NF version in pipeline template from 22.10.1 -> 23.04.0 (#2305)
• Add ability to interpret docker.registry from nextflow.config file. If not found defaults to quay.io. (#2318)
• Add functions to dynamically include pipeline tool citations in MultiQC methods description section for better reporting. (#2326)
• Remove --tracedir parameter (#2290)
• Incorrect config parameter warnings when customising pipeline template (#2333)
• Use markdown syntax in the description for the meta map channels (#2358)

Download

• Introduce a --tower flag for nf-core download to obtain pipelines in an offline format suited for seqeralabs® Nextflow Tower (#2247).
• Refactored the CLI for --singularity-cache in nf-core download from a flag to an argument. The prior options were renamed to amend (container images are only saved in the $NXF_SINGULARITY_CACHEDIR) and copy (a copy of the image is saved with the download). remote was newly introduced and allows to provide a table of contents of a remote cache via an additional argument --singularity-cache-index (#2247).
• Refactored the CLI parameters related to container images. Although downloading other images than those of the Singularity/Apptainer container system is not supported for the time being, a generic name for the parameters seemed preferable. So the new parameter --singularity-cache-index introduced in #2247 has been renamed to --container-cache-index prior to release (#2336).
• To address issue #2311, a new parameter --container-library was created allowing to specify the container library (registry) from which container images in OCI format (Docker) should be pulled (#2336).
• Container detection in configs was improved. This allows for DSL2-like container definitions inside the container parameter value provided to process scopes #2346.
• Add apptainer to the list of false positve container strings (#2353).

Updated CLI parameters

Old parameter	New parameter
new parameter	-d / --download-configuration
new parameter	-t / --tower
-c/ --container	-s / --container-system <VALUE>
new parameter	-l / --container-library <VALUE>
--singularity-cache	-u / --container-cache-utilisation <VALUE>
new parameter	-i / --container-cache-index <VALUE>

In addition, -r / --revision has been changed to a parameter that can be provided multiple times so several revisions can be downloaded at once.

Linting

• Warn if container access is denied (#2270)
• Error if module container specification has quay.io as prefix when it shouldn't have ([#2278])(https://github.com/nf-core/tools/pull/2278/files)
• Detect if container is 'simple name' and try to contact quay.io server by default (#2281)
• Warn about null/None/empty default values in nextflow_schema.json (#3328)
• Fix linting when creating a pipeline skipping some parts of the template and add CI test (#2330)

Modules

• Don't update modules_json object if a module is not updated (#2323)

Subworkflows

General

• GitPod base image: Always self-update to the latest version of Nextflow. Add pre-commit dependency.
• GitPod configs: Update Nextflow as an init task, init pre-commit in pipeline config.
• Refgenie: Create nxf_home/nf-core/refgenie_genomes.config path if it doesn't exist (#2312)
• Add CI tests to test running a pipeline when it's created from a template skipping different areas

v2.8 - Ruthenium Monkey

v2.8 - Ruthenium Monkey - [2023-04-27]

Template

• Explicitly disable conda when a container profile (#2140)
• Turn on automatic clean up of intermediate files in work/ on successful pipeline completion in full-test config (#2163) [Contributed by @jfy133]
• Add documentation to usage.md on how to use params.yml files, based on nf-core/ampliseq text (#2173) [Contributed by @jfy133, @d4straub]
• Make jobs automatically resubmit for a much wider range of exit codes (now 104 and 130..145) (#2170)
• Add a clean-up GHA which closes issues and PRs with specific labels (#2183)
• Remove problematic sniffer code in samplesheet_check.py that could give false positive 'missing header' errors ([https://github.com//pull/2194]) [Contributed by @Midnighter, @jfy133]
• Consistent syntax for branch checks in PRs (#2202)
• Fixed minor Jinja2 templating bug that caused the PR template to miss a newline
• Updated AWS tests to use newly moved seqeralabs/action-tower-launch instead of nf-core/tower-action
• Remove .cff files from .editorconfig [(#2145)[https://github.com//pull/2145]]
• Simplify pipeline README (#2186)
• Added support for the apptainer container engine via -profile apptainer. (#2244) [Contributed by @jfy133]
• Added config docker.registry to pipeline template for a configurable default container registry when using Docker containers. Defaults to quay.io (#2133)
• Add tower.yml file to the pipeline template (#2251)
• Add mastodon badge to README (#2253)
• Removed quay.io from all module Docker container references as this is now supplied at pipeline level. (#2249)
• Remove CITATION.cff file from pipeline template, to avoid that pipeline Zenodo entries reference the nf-core publication instead of the pipeline (#2059).

Linting

• Update modules lint test to fail if enable_conda is found (#2213)
• Read module lint configuration from .nf-core.yml, not .nf-core-lint.yml (#2221)
• nf-core schema lint now defaults to linting nextflow_schema.json if no filename is provided (#2225)
• Warn if /zenodo.XXXXXX is present in the Readme (#2254)
• Lint all labels in a module (#2227)

Modules

• Add an --empty-template option to create a module without TODO statements or examples (#2175 & #2177)
• Removed the nf-core modules mulled command and all its code dependencies (2199).
• Take into accout the provided --git_remote URL when linting all modules (2243).

Subworkflows

• Fixing problem when a module included in a subworkflow had a name change from TOOL to TOOL/SUBTOOL (#2177)
• Fix nf-core subworkflows test not running subworkflow tests (#2181)
• Add tests for nf-core subworkflows create-test-yml (#2219)

General

• Deprecate Python 3.7 support because it reaches EOL (#2210)
• nf-core modules/subworkflows info now prints the include statement for the module/subworkflow (#2182).
• Add a clean-up GHA which closes issues and PRs with specific labels (#2183)
• update minimum version of rich to 13.3.1 (#2185)
• Add the Nextflow version to Gitpod container matching the minimal Nextflow version for nf-core (according to nextflow.config) (#2196)
• Use nfcore/gitpod:dev container in the dev branch (#2196)
• Replace requests_mock with responses in test mocks (#2165).
• Add warning when installing a module from an org_path that exists in multiple remotes in modules.json (#2228 #2239).
• Add the possibility to translate refgenie asset aliases to the ones used in a pipeline with an alias_translations.yaml file (#2242).
• Add initial CHM13 support (1988)

v2.7.2 - Mercury Eagle Patch

v2.7.2 - Mercury Eagle Patch

Patch release solving some bugs introduced with v2.7

Template

• Fix the syntax of github_output in GitHub actions (#2114)
• Fix a bug introduced in 2.7 that made pipelines hang (#2132)

Linting

• Allow specifying containers in less than three lines (#2121)
• Run prettier after dumping a JSON schema file (#2124)

General

• Only check that a pipeline name doesn't contain dashes if the name is provided by prompt of --name. Don't check if a template file is used. (#2123)
• Deprecate --enable_conda parameter. Use conda.enable instead (#2131)
• Handle json.load() exceptions (#2134)

v2.7.1 - Mercury Eagle Patch

This release fixes a sync error with the new syntax of set_output on GitHub Actions.

• Patch release to fix pipeline sync (#2110)