5. Customizing the template¶

As a developer, you may want more flexibility when developing your pipelines.

While the nf-core template comes with specific standards, many of these can be altered to better suit your needs.

Here, you will learn how to use the nf-core tooling to customize different parts of the nf-core template and nf-core components without compromising their integration into the nf-core ecosystem.

5.1 Patching modules¶

Although modules are written to be flexible you may want to modify them to better fit your purpose. While you are free to edit modules locally, any changes

The nf-core lint command will help manage nf-core components and test that they match the remote source they came from.

For example, if you modify an nf-core module, it will no longer match the remote and a linting test of this module will fail.

22

tuple val(meta), path('*.fail.fastq.gz')  , optional:true, emit: reads_failed

Running nf-core lint after you have modified an nf-core module will cause it to throw an error.

╭─ [✗] 1 Module Test Failed ────────────────────────────────────────────────────────────────╮
│              ╷                               ╷                                            │
│ Module name  │ File path                     │ Test message                               │
│╶─────────────┼───────────────────────────────┼────────────────────────────────────────────│
│ fastp        │ modules/nf-core/fastp/main.nf │ Local copy of module does not match remote │
│              ╵                               ╵                                            │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

Changing a module does not mean you can't continue to use that module.

The nf-core modules patch command allows you keep using the nf-core component without needing to make it into a local module for tests to pass. Instead, nf-core modules patch command creates a diff file that will keep track of the changes you made. If you subsequently update the module using the nf-core tooling, the diff file will be retained. If any subsequent changes to the module conflict with your diff file, you will be prompted to resolve the conflicts.

nf-core modules patch

The prompt can be followed to patch the fastp module.

? Module name: fastp

A patch file is created in the fastp module directory

...
INFO     'modules/nf-core/fastp/tests/main.nf.test.snap' is unchanged
INFO     'modules/nf-core/fastp/tests/tags.yml' is unchanged
INFO     'modules/nf-core/fastp/tests/nextflow.config' is unchanged
INFO     'modules/nf-core/fastp/tests/main.nf.test' is unchanged
INFO     Patch file of 'modules/nf-core/fastp' written to 'modules/nf-core/fastp/fastp.diff'

5.2 Ignoring linting tests¶

The linting tests in the nf-core template are primarily designed for pipelines that are shared as a part of the nf-core project and you may want to ignore certain linting tests if they are not required.

The .nf-core.yml file can be modified to choose which linting tests you would like to skip.

For example, by default, there is a linting test that checks if the CODE_OF_CONDUCT.md matches the template. If the CODE_OF_CONDUCT.md has been modified the linting test will fail.

╭─ [✗] 1 Pipeline Test Failed ────────────────────────────────────╮
│                                                                 │
│ files_unchanged: CODE_OF_CONDUCT.md does not match the template │
│                                                                 │
╰─────────────────────────────────────────────────────────────────╯

If you were to alter the CODE_OF_CONDUCT.md the nf-core lint command will suggest a command to fix this warning.

Tip: Some of these linting errors can automatically be resolved with the following command:

    nf-core lint --fix files_unchanged

However, if you wish to remove or modify this file you would need to ignore this test. To do this, you would need to list the CODE_OF_CONDUCT.md as a files_unchanged: in the .nf-core.yml file.

repository_type: pipeline
lint:
    files_unchanged:
        - CODE_OF_CONDUCT.md

If you run nf-core lint again, you would see that the test is now ignored and there are no more failed tests.

╭─ [?] 1 Pipeline Test Ignored ────────────────────────────────────────╮
│                                                                      │
│ files_unchanged: File ignored due to lint config: CODE_OF_CONDUCT.md │
│                                                                      │
╰──────────────────────────────────────────────────────────────────────╯

A full list of checks, descriptions of how they work, and how they can be customized can be found on the tools documentation website.

repository_type: pipeline
lint:
    pipeline_todos: false
    files_unchanged:
        - CODE_OF_CONDUCT.md

5.3 Custom remote modules¶

As an individual or group, you may want to keep your own library of modules and/or subworkflows.

The nf-core modules command comes with two flags for specifying a custom remote:

• --git-remote <git remote url>: Specifies the repository from which the modules should be fetched as a git URL.
• --branch <branch name>: Specifies the branch from which the modules should be fetched.

Note that a custom remote must follow a similar directory structure to that of nf-core/modules for the nf-core commands to work properly.

The directory where modules are installed will be prompted or obtained from org_path in the .nf-core.yml file, if it is available. If a module was located at modules/my-folder/TOOL/SUBTOOL your .nf-core.yml should have:

org_path: my-folder

The modules commands will, during initialization, try to pull changes from the remote repositories. If you want to disable this, for example, due to performance reasons, you can use the flag --no-pull. Commands will still need to clone repositories that have previously not been used.

gh auth login

5.4 TEMPLATE syncs¶

The template evolves as the ecosystem evolves.

To keep nf-core pipelines up to date with improvements in the main template, you can use a method of synchronization with the TEMPLATE branch.

To sync the template, you first need to commit and push your changes to GitHub. The nf-core sync command can then be used to update the TEMPLATE branch with the latest version of the nf-core template, so that these updates can be synchronized with the pipeline. It is run automatically for all pipelines whenever a new release of nf-core/tools (and the included template) is made.

nf-core sync

The tooling merges updates suggesting a git command

cd /workspace/gitpod/nf-template/nf-core-myfirstpipeline
git merge TEMPLATE

5.4.1 Changes to the template structure¶

Occasionally, the structure of the nf-core pipeline template is updated during a new release of the nf-core tooling. Most of the time these changes are minor. However, sometimes, larger structural changes are adopted to align with changes in the wider ecosystem.

For example, as of nf-core tools 2.13, the groovy code that once lived in the lib folder has been moved to subworkflows/. Moving this code has made it easier to find, modify, and test the code. Importantly, it's modular nature is paving the way for a more flexible template in the future.

5.5 Update minimum Nextflow version¶

The template also includes a minimum Nextflow version that is required for the pipeline to run. You can also change the required version of Nextflow using the nf-core bump-version command. These versions can also be used during GitHub Actions to test your pipeline with upper and lower version limits.

nf-core bump-version 23.04.0 --nextflow