Getting started

Step 1: Check whether a similar task already exists

Before starting, search the OpenProblems GitHub organization for existing task repositories (repositories named task_*) and open issues. Someone may already be working on the same problem, and it is better to collaborate than to duplicate effort.

Step 2: Reach out

If you are confident your task is new, open a New task proposal issue in the OpenProblems repository. Describe the biological question, the proposed input/output format, and the evaluation strategy.

You can also introduce yourself and discuss ideas on the OpenProblems Discord server before filing an issue.

Step 3: Create the task repository

Once your proposal is accepted, create a repository from the task template:

1. Go to github.com/openproblems-bio/task_template.
2. Click Use this template and then Create a new repository.
3. Set the Owner to openproblems-bio.
4. Set the Repository name to task_<your_task_name> (snake_case).
5. Set visibility to Public.
6. Click Create repository.

Step 4: Clone the task repository

git clone --recursive https://github.com/openproblems-bio/task_<your_task_name>.git
cd task_<your_task_name>

Note

The --recursive flag is required because the repository contains a common/ submodule. If you cloned without it, run git submodule update --init --recursive to initialise the submodule.

Step 5: Test the repository

Verify that the template works end-to-end before making changes.

Sync test resources

bash scripts/sync_resources.sh

Output

Output varies depending on your network connection and which resources are already cached locally. On a clean clone you should see download progress for each test resource file.

Run a component

viash run src/methods/logistic_regression/config.vsh.yaml -- \
  --input_train resources_test/task_template/cxg_mouse_pancreas_atlas/train.h5ad \
  --input_test resources_test/task_template/cxg_mouse_pancreas_atlas/test.h5ad \
  --output output.h5ad

Test a single component

viash test src/methods/logistic_regression/config.vsh.yaml

See Run tests for example output and common error messages.

Test all components

viash ns test --parallel --engine docker

Output

          namespace                 name               runner               engine            test_name exit_code duration               result
    control_methods          true_labels           executable               docker                start
            metrics             accuracy           executable               docker                start
    data_processors      process_dataset           executable               docker                start
            methods  logistic_regression           executable               docker                start
    data_processors      process_dataset           executable               docker     build_executable         0        2              SUCCESS
    data_processors      process_dataset           executable               docker run_and_check_output.py         0        4              SUCCESS
    control_methods          true_labels           executable               docker     build_executable         0        2              SUCCESS
    control_methods          true_labels           executable               docker run_and_check_output.py         0        4              SUCCESS
    control_methods          true_labels           executable               docker      check_config.py         0        3              SUCCESS
            metrics             accuracy           executable               docker     build_executable         0        2              SUCCESS
            metrics             accuracy           executable               docker run_and_check_output.py         0        5              SUCCESS
            metrics             accuracy           executable               docker      check_config.py         0        2              SUCCESS
            methods  logistic_regression           executable               docker     build_executable         0        2              SUCCESS
            methods  logistic_regression           executable               docker run_and_check_output.py         0        5              SUCCESS
            methods  logistic_regression           executable               docker      check_config.py         0        3              SUCCESS
All 11 configs built and tested successfully

Step 6: Update the task config file

Edit _viash.yaml in the repository root to configure your task. The sub-steps are:

1. Set name to task_<your_task_name>.
2. Set organization to openproblems-bio.
3. Update version to the initial version of your task (e.g. "0.1.0").
4. Set links.repository to the URL of your new repository.
5. Set links.docker_registry to ghcr.io.
6. Update info.label with a short human-readable name for the task.
7. Update info.summary with a one-sentence description.
8. Update info.description with a longer description of the task.

---

You are now well-equipped to begin designing the API.