nirs4all.pipeline.steps package

Submodules

Module contents

Pipeline step processing module for nirs4all.

class nirs4all.pipeline.steps.ControllerRouter(verbose: bool = False)[source]

Bases: object

Routes parsed steps to appropriate controllers.

Uses registry pattern with controller priorities. Extensible - new controllers automatically discovered via registry.

registry: Controller registry (from nirs4all.controllers.registry)

verbose: If True, print debug information about controller matching

route(parsed_step: ParsedStep, step: Any = None) → BaseController[source]

Select the appropriate controller for a parsed step.

Parameters:

parsed_step – Parsed step configuration
step – Original step (for backward compatibility with matches())

Returns:

Instantiated controller instance

Raises:

TypeError – If no matching controller found

route_from_raw(step: Any, operator: Any = None, keyword: str = '') → BaseController[source]

Route from raw step parameters (backward compatibility).

Parameters:

step – Raw step configuration
operator – Optional operator instance
keyword – Optional keyword hint

Returns:

Instantiated controller instance

Raises:

TypeError – If no matching controller found

class nirs4all.pipeline.steps.ParsedStep(operator: Any, keyword: str, step_type: StepType, original_step: Any, metadata: Dict[str, Any], force_layout: str | None = None)[source]

Bases: object

Normalized step configuration after parsing.

operator

Deserialized operator instance (or None for workflow ops)

Type:: Any

keyword

Step keyword (e.g., ‘model’, ‘preprocessing’)

Type:: str

step_type

Type of step (workflow, serialized, etc.)

Type:: nirs4all.pipeline.steps.parser.StepType

original_step

Original step configuration

Type:: Any

metadata

Additional parsing metadata

Type:: Dict[str, Any]

force_layout

Optional forced data layout (overrides controller’s preferred layout)

Type:: str | None

force_layout: str | None = None

keyword: str

metadata: Dict[str, Any]

operator: Any

original_step: Any

step_type: StepType

class nirs4all.pipeline.steps.StepParser[source]

Bases: object

Parses pipeline step configurations into normalized format.

Handles multiple step syntaxes: - Dictionary: {“model”: SVC, “params”: {…}} - String: “sklearn.preprocessing.StandardScaler” - Direct instance: StandardScaler() - Nested lists: [[step1, step2], step3]

Normalizes to canonical ParsedStep format for controller execution.

RESERVED_KEYWORDS = ['params', 'metadata', 'steps', 'name', 'finetune_params', 'train_params', 'fit_on_all', 'force_layout']

SERIALIZATION_OPERATORS = ['class', 'function', 'module', 'object', 'pipeline', 'instance']

VALID_LAYOUTS = {'2d', '2d_interleaved', '3d', '3d_transpose'}

WORKFLOW_KEYWORDS = ['model', 'preprocessing', 'feature_augmentation', 'auto_transfer_preproc', 'concat_transform', 'y_processing', 'sample_augmentation', 'branch']

parse(step: Any) → ParsedStep[source]

Parse a pipeline step into normalized format.

Parameters:: step – Raw step configuration
Returns:: ParsedStep with normalized operator and metadata
Raises:: ValueError – If step format is invalid

class nirs4all.pipeline.steps.StepRunner(parser: StepParser | None = None, router: ControllerRouter | None = None, verbose: int = 0, mode: str = 'train', show_spinner: bool = True, plots_visible: bool = False)[source]

Bases: object

Executes a single pipeline step.

Handles: - Step parsing (delegates to StepParser) - Controller selection (delegates to ControllerRouter) - Controller execution - Binary loading/saving for this step

parser: Parses step configuration

router: Routes to appropriate controller

verbose: Verbosity level

mode: Execution mode (train/predict/explain)

execute(step: Any, dataset: SpectroDataset, context: ExecutionContext, runtime_context: Any, loaded_binaries: List[Tuple[str, Any]] | None = None, prediction_store: Predictions | None = None) → StepResult[source]

Execute a single pipeline step.

Parameters:

step – Raw step configuration
dataset – Dataset to process
context – Execution context
runtime_context – Runtime infrastructure context
loaded_binaries – Pre-loaded artifacts for this step
prediction_store – Prediction store for accumulating results

Returns:

StepResult with updated context and artifacts

Raises:

RuntimeError – If step execution fails