nirs4all.pipeline.storage.library module

Pipeline library manager - save and load reusable pipeline templates.

The library allows users to: - Save successful pipeline configurations as templates - Load templates for reuse across datasets - Organize templates by category (preprocessing, modeling, full_pipeline, etc.) - Search and filter templates by metadata

class nirs4all.pipeline.storage.library.PipelineLibrary(workspace_path: Path)[source]

Bases: object

Manages reusable pipeline templates in the workspace library.

Templates are stored in: workspace/library/{category}/{template_name}/ Each template contains: - pipeline.json: The pipeline configuration - metadata.json: Description, tags, performance metrics, etc. - README.md: Human-readable documentation

copy_from_pipeline(pipeline_dir: Path, name: str, category: str = 'general', description: str = '', tags: List[str] | None = None, extract_metrics: bool = True) → Path[source]

Copy a successful pipeline to the library as a template.

Parameters:

  • pipeline_dir – Path to pipeline directory (e.g., workspace/runs/…/0001_hash/)

  • name – Template name

  • category – Category for the template

  • description – Description of the template

  • tags – Tags for filtering

  • extract_metrics – Whether to extract metrics from manifest

Returns:

Path to saved template

delete_template(name: str, category: str | None = None) → None[source]

Delete a template from the library.

Parameters:

  • name – Template name

  • category – Optional category to search in

Raises:

FileNotFoundError – If template not found

export_template(name: str, export_path: str | Path, category: str | None = None) → Path[source]

Export a template to a standalone directory.

Parameters:

  • name – Template name

  • export_path – Destination directory

  • category – Optional category to search in

Returns:

Path to exported template

get_template_metadata(name: str, category: str | None = None) → Dict[str, Any][source]

Get metadata for a template.

Parameters:

  • name – Template name

  • category – Optional category to search in

Returns:

Metadata dictionary

import_template(import_path: str | Path, category: str = 'general', overwrite: bool = False) → Path[source]

Import a template from an external directory.

Parameters:

  • import_path – Path to template directory

  • category – Category to import into

  • overwrite – Whether to overwrite existing template

Returns:

Path to imported template in library

list_templates(category: str | None = None, tags: List[str] | None = None) → List[Dict[str, Any]][source]

List all templates, optionally filtered by category and tags.

Parameters:

  • category – Optional category filter

  • tags – Optional list of tags to filter by (matches any)

Returns:

List of template metadata dictionaries

load_template(name: str, category: str | None = None) → Dict[str, Any][source]

Load a pipeline template by name.

Parameters:

  • name – Template name

  • category – Optional category to search in (searches all if None)

Returns:

Pipeline configuration dictionary

Raises:

FileNotFoundError – If template not found

save_template(pipeline_config: Dict[str, Any], name: str, category: str = 'general', description: str = '', tags: List[str] | None = None, metrics: Dict[str, float] | None = None, notes: str = '', overwrite: bool = False) → Path[source]

Save a pipeline configuration as a reusable template.

Parameters:

  • pipeline_config – Pipeline configuration dictionary

  • name – Template name (will be sanitized for filesystem)

  • category – Category/folder (e.g., “preprocessing”, “modeling”, “full_pipeline”)

  • description – Short description of the template

  • tags – List of tags for filtering (e.g., [“classification”, “neural_network”])

  • metrics – Performance metrics (e.g., {“accuracy”: 0.95, “f1”: 0.93})

  • notes – Additional notes or usage instructions

  • overwrite – Whether to overwrite existing template

Returns:

Path to saved template directory

Raises: