nirs4all.pipeline.execution.orchestrator module

Pipeline orchestrator for coordinating multiple pipeline executions.

class nirs4all.pipeline.execution.orchestrator.PipelineOrchestrator(workspace_path: str | Path | None = None, verbose: int = 0, mode: str = 'train', save_artifacts: bool = True, save_charts: bool = True, enable_tab_reports: bool = True, continue_on_error: bool = False, show_spinner: bool = True, keep_datasets: bool = True, plots_visible: bool = False)[source]

Bases: object

Orchestrates execution of multiple pipelines across multiple datasets.

High-level coordinator that manages: - Workspace initialization - Global predictions aggregation - Best results reporting - Dataset/pipeline normalization

workspace_path

Root workspace directory

runs_dir

Directory for storing runs

verbose

Verbosity level

mode

Execution mode (train/predict/explain)

save_artifacts

Whether to save binary artifacts

save_charts

Whether to save charts and visual outputs

enable_tab_reports

Whether to generate tab reports

keep_datasets

Whether to keep dataset snapshots

plots_visible

Whether to display plots

execute(pipeline: PipelineConfigs | List[Any] | Dict | str, dataset: DatasetConfigs | SpectroDataset | List[SpectroDataset] | ndarray | Tuple[ndarray, ...] | Dict | List[Dict] | str | List[str], pipeline_name: str = '', dataset_name: str = 'dataset', max_generation_count: int = 10000, artifact_loader: Any = None, target_model: Dict[str, Any] | None = None, explainer: Any = None) → Tuple[Predictions, Dict[str, Any]][source]

Execute pipeline configurations on dataset configurations.

Parameters:

  • pipeline – Pipeline definition (PipelineConfigs, List[steps], Dict, or file path)

  • dataset – Dataset definition (DatasetConfigs, SpectroDataset, numpy arrays, Dict, or file path)

  • pipeline_name – Optional name for the pipeline

  • dataset_name – Optional name for array-based datasets

  • max_generation_count – Maximum number of pipeline combinations to generate

  • artifact_loader – ArtifactLoader for predict/explain modes

  • target_model – Target model for predict/explain modes

  • explainer – Explainer instance for explain mode

Returns:

Tuple of (run_predictions, dataset_predictions)