Create components
Once you have designed the API and created the dataset processor, you are ready to add the components that make up the benchmark: methods, control methods, and metrics. Each component type has a dedicated guide in the Create a component chapter.
Step 1: Add a method
Section titled “Step 1: Add a method”Methods are the candidate techniques being benchmarked. See Add a method for a step-by-step guide on creating a method component, filling in metadata, adding dependencies, and writing the script.
Step 2: Add a control method
Section titled “Step 2: Add a control method”Control methods share the same interface as methods but may also receive the solution file. Every task needs at least one positive control and one negative control to establish the score range. See Add a control method for details.
Step 3: Add a metric
Section titled “Step 3: Add a metric”Metrics compare a method’s output against the solution and produce a scalar score. See Add a metric for a step-by-step guide.
Step 4: Run local tests
Section titled “Step 4: Run local tests”After adding each component, verify it builds and produces correctly formatted output:
viash test src/methods/my_method/config.vsh.yamlSee Run tests for example output and how to interpret common error messages.
Next steps
Section titled “Next steps”Once your components pass local tests, open a pull request. See Create a pull request for the submission checklist and review process.