Skip to content

Add a method

A method is a technique used to solve a specific problem when analyzing omics data. Its performance is assessed by comparing it to other methods and control methods.

This guide shows how to create a new Viash component for both Python and R. The Task template repo is used throughout; replace any occurrence of task_template with your task of interest.

Step 1: Create a new component

Section titled “Step 1: Create a new component”

Use the create_*_method.sh script to start creating a new method. Open the script and update the name parameter.

scripts/create_component/create_python_method.sh
common/scripts/create_component \
  --name my_python_method \
  --language python \
  --type method

Run it from the repository root:

bash scripts/create_component/create_python_method.sh 2>&1 | grep -v '^\[notice\]'
Check inputs
Check language
Check API file
Read API file
Create output dir
Create config
Create script
Done!

This creates a new folder at src/methods/my_python_method/:

src/methods/my_python_method/
    ├── script.py         Script for running the method.
    ├── config.vsh.yaml   Config file for the method.
    └── ...               Optional additional resources.

Change --name to a unique name for your method. It must match the regex [a-z][a-z0-9_]* (snake_case).

  • A config file contains the component’s metadata and its runtime dependencies.
  • A script contains the code to run the method.

Step 2: Fill in metadata

Section titled “Step 2: Fill in metadata”

The Viash config contains your method’s metadata, the script used to run it, and its dependencies.

Generated config.vsh.yaml 
cat src/methods/my_python_method/config.vsh.yaml
# The API specifies which type of component this is.
# It contains specifications for:
#   - The input/output files
#   - Common parameters
#   - A unit test
__merge__: ../../api/comp_method.yaml


# A unique identifier for your component (required).
# Can contain only lowercase letters or underscores.
name: my_python_method
# A relatively short label, used when rendering visualisations (required)
label: My Python Method
# A one sentence summary of how this method works (required). Used when
# rendering summary tables.
summary: "FILL IN: A one sentence summary of this method."
# A multi-line description of how this component works (required). Used
# when rendering reference documentation.
description: |
  FILL IN: A (multi-line) description of how this method works.
# references:
#   doi:
#     - 10.1000/xx.123456.789
#   bibtex:
#     - |
#       @article{foo,
#         title={Foo},
#         author={Bar},
#         journal={Baz},
#         year={2024}
#       }
links:
  # URL to the documentation for this method (required).
  documentation: https://url.to/the/documentation
  # URL to the code repository for this method (required).
  repository: https://github.com/organisation/repository






# Metadata for your component
info:
  # Which normalisation method this component prefers to use (required).
  preferred_normalization: log_cp10k


# Component-specific parameters (optional)
# arguments:
#   - name: "--n_neighbors"
#     type: "integer"
#     default: 5
#     description: Number of neighbors to use.


# Resources required to run the component
resources:
  # The script of your component (required)
  - type: python_script
    path: script.py
  # Additional resources your script needs (optional)
  # - type: file
  #   path: weights.pt


engines:
  # Specifications for the Docker image for this component.
  - type: docker
    image: openproblems/base_python:1.0.0
    # Add custom dependencies here (optional). For more information, see
    # https://viash.io/reference/config/engines/docker/#setup .
    # setup:
    #   - type: python
    #     packages: numpy<2


runners:
  # This platform allows running the component natively
  - type: executable
  # Allows turning the component into a Nextflow module / pipeline.
  - type: nextflow
    directives:
      label: [midtime,midmem,midcpu]

Required metadata fields

Section titled “Required metadata fields”

Edit the info section of the config file to fill in the required metadata:

  • .merge: Specifies which type of component this is. Defines input/output files, common parameters, and a unit test.
  • .name: A unique identifier. Can only contain lowercase letters, numbers, or underscores.
  • .label: A short, human-readable label for summary tables and visualizations.
  • .summary: A one-sentence summary of purpose and methodology.
  • .description: A longer description (one or more paragraphs) for reference documentation.

Step 3: Add dependencies

Section titled “Step 3: Add dependencies”

Each component has its own set of dependencies, since different components may have conflicting requirements.

Base images

Section titled “Base images”

Several pre-built base images are available in the OpenProblems Docker repository:

  • openproblems/base_python — base image for Python scripts.
  • openproblems/base_r — base image for R scripts.
  • openproblems/base_pytorch_nvidia — base image for PyTorch with NVIDIA GPU support.
  • openproblems/base_tensorflow_nvidia — base image for TensorFlow with NVIDIA GPU support.

Custom image

Section titled “Custom image”

If you use a custom image, add the following minimum setup to the engines section of the config file:

engines:
  - type: docker
    image: your_custom_image
    setup:
      - type: apt
        packages:
          - procps
      - type: python
        packages:
          - anndata~=0.10.0
          - scanpy~=1.10.0
          - pyyaml
          - requests
          - jsonschema
        github:
          - "openproblems-bio/core#subdirectory=packages/python/openproblems"

See the Viash dependency guide for more information.

After changing dependencies, rebuild the Docker container:

viash run src/methods/my_python_method/config.vsh.yaml -- ---setup cachedbuild

Step 4: Edit script

Section titled “Step 4: Edit script”

A component’s script typically has five sections:

a. Imports and libraries b. Argument values (the Viash placeholder block) c. Read input data d. Generate results e. Write output data to file

Generated script 
cat src/methods/my_python_method/script.py
import anndata as ad


## VIASH START
# Note: this section is auto-generated by viash at runtime. To edit it, make changes
# in config.vsh.yaml and then run `viash config inject config.vsh.yaml`.
par = {
  'input_train': 'resources_test/.../train.h5ad',
  'input_test': 'resources_test/.../test.h5ad',
  'output': 'output.h5ad'
}
meta = {
  'name': 'my_python_method'
}
## VIASH END


print('Reading input files', flush=True)
input_train = ad.read_h5ad(par['input_train'])
input_test = ad.read_h5ad(par['input_test'])


print('Preprocess data', flush=True)
# ... preprocessing ...


print('Train model', flush=True)
# ... train model ...


print('Generate predictions', flush=True)
# ... generate predictions ...


print("Write output AnnData to file", flush=True)
output = ad.AnnData(


)
output.write_h5ad(par['output'], compression='gzip')

a. Imports and libraries

Section titled “a. Imports and libraries”

Define which packages the method needs at the top of the script. If you add a new package, also add it to the setup field in config.vsh.yaml (see Step 3).

b. Argument block

Section titled “b. Argument block”

The Viash code block enables you to run the script directly for prototyping. Everything between ## VIASH START and ## VIASH END is removed and replaced with a CLI argument parser when Viash builds the component.

The par dictionary holds all arguments defined in config.vsh.yaml (including those from the __merge__ file). When you add an argument to par, also add it to the arguments section of the config.

c. Read input data

Section titled “c. Read input data”

Read any input AnnData files passed to the component.

d. Generate results

Section titled “d. Generate results”

The core section: processes the input data and produces results.

e. Write output data to file

Section titled “e. Write output data to file”

Store the output in an AnnData object and write it to an .h5ad file. The format is specified by the API file referenced in the __merge__ field of the config.

Step 5: Add resources (optional)

Section titled “Step 5: Add resources (optional)”

You can add helper files and other additional resources. See Use helper functions in the Viash documentation.

Step 6: Try component

Section titled “Step 6: Try component”

Your component’s API file contains unit tests to check whether the component runs and whether the output has the correct format. Test your component with:

viash test src/methods/my_python_method/config.vsh.yaml

See Run tests for example output and common error messages.

You can also run your component on local files:

viash run src/methods/my_python_method/config.vsh.yaml -- \
  --input_train resources_test/task_template/cxg_mouse_pancreas_atlas/train.h5ad \
  --input_test resources_test/task_template/cxg_mouse_pancreas_atlas/test.h5ad \
  --output output.h5ad

Next steps

Section titled “Next steps”

If your component works, create a pull request.