nirs4all.controllers.data.exclude module

Controller for sample exclusion operations.

This controller handles the exclude keyword, marking samples for exclusion from training based on filter criteria. Unlike tag, exclude both stores tag values AND marks samples as excluded from training.

class nirs4all.controllers.data.exclude.ExcludeController[source]

Bases: OperatorController

Controller for sample exclusion operations.

This controller computes exclusion masks using SampleFilter instances, stores the results as tag columns for analysis, and marks matching samples as excluded from training.

The key difference from tag: - tag = compute and store tag (never removes samples) - exclude = compute tag AND remove from training (always removes)

For tag-only behavior without exclusion, use the tag keyword instead.

Pipeline syntax:

# Single filter {“exclude”: YOutlierFilter(method=”iqr”)}

# Multiple filters with mode {“exclude”: [YOutlierFilter(), XOutlierFilter()], “mode”: “any”} {“exclude”: [Filter1(), Filter2()], “mode”: “all”}

Parameters:

mode – How to combine multiple filter masks: - “any” (default): Exclude if ANY filter flags the sample - “all”: Exclude only if ALL filters flag the sample

Note

Exclusion only runs during training mode - prediction samples are never excluded to ensure all predictions are generated.

execute(step_info: ParsedStep, dataset: SpectroDataset, context: ExecutionContext, runtime_context: RuntimeContext, source: int = -1, mode: str = 'train', loaded_binaries: List[Tuple[str, Any]] | None = None, prediction_store: Any | None = None) → Tuple[ExecutionContext, List][source]

Execute sample exclusion operation.

This method: 1. Parses exclusion configuration (single filter or list with mode) 2. Gets training samples (base only, no augmented) 3. Fits and applies each filter to identify outliers 4. Combines filter masks using the specified mode 5. Stores exclusion tags in dataset’s indexer (for analysis) 6. Marks excluded samples in the dataset’s indexer 7. Returns persisted artifacts for reproducibility

Parameters:

  • step_info – Parsed step containing operator and configuration

  • dataset – Dataset to operate on

  • context – Pipeline execution context

  • runtime_context – Runtime infrastructure context

  • source – Data source index (unused, exclusion is dataset-level)

  • mode – Execution mode (“train” or “predict”)

  • loaded_binaries – Pre-loaded binaries (unused, exclusion skips prediction)

  • prediction_store – External prediction store (unused)

Returns:

Tuple of (updated_context, persisted_artifacts)

Raises:
classmethod matches(step: Any, operator: Any, keyword: str) → bool[source]

Match exclude keyword in pipeline.

priority: int = 5
classmethod supports_prediction_mode() → bool[source]

Exclusion only runs during training.

Prediction samples should never be filtered/excluded - we want to predict on all provided samples.

classmethod use_multi_source() → bool[source]

Exclusion operates at dataset level, not per-source.