Data Handling

This section covers loading and managing spectroscopic data in NIRS4ALL.

• Loading Data
  • Overview
  • Quick Start
  • Supported Formats
  • File Keys Reference
  • Wavelength Headers
  • Signal Type
  • Task Type
  • Multi-Source Datasets
  • Sample Aggregation
  • Multiple Datasets
  • Using SpectroDataset Directly
  • Complete Example
  • Common Patterns
  • Troubleshooting
  • See Also
• Synthetic Data Generation
  • Overview
  • Quick Start
  • Convenience Functions
  • Builder API
  • Non-Linear Target Complexity
  • Environmental and Matrix Effects
  • Validation and Benchmarking
  • Configuration Options
  • Exporting Synthetic Data
  • Matching Real Data
  • Advanced: Custom Component Library
  • Integration with Pipelines
  • Performance
  • API Reference
  • See Also
• Sample Aggregation
  • Overview
  • Quick Start
  • Aggregation Methods
  • Pipeline Output
  • Visualization with Aggregation
  • Multi-Dataset Aggregation
  • Priority Resolution
  • Aggregation Algorithm
  • Complete Example
  • See Also
• Sample Filtering User Guide
  • Overview
  • Quick Start
  • Available Filters
  • Combining Filters
  • Pipeline Integration Options
  • Managing Exclusions
  • Filtering Reports
  • Visualization
  • Best Practices
  • Edge Cases and Warnings
  • Troubleshooting
  • API Reference
  • See Also

Overview

NIRS4ALL provides flexible data handling capabilities for loading spectroscopic data from various formats, generating synthetic data, filtering samples, and aggregating predictions.

📂 Loading Data

Load data from CSV, Excel, MATLAB, NumPy, and Parquet formats.

Essential

Loading Data

🧪 Synthetic Data

Generate realistic synthetic NIRS spectra for testing and prototyping.

New in 0.6

Synthetic Data Generation

📊 Aggregation

Combine predictions across multiple samples or replicates.

Post-processing

Sample Aggregation

🔍 Sample Filtering

Filter samples based on metadata, outliers, or custom criteria.

Preprocessing

Sample Filtering User Guide

Coming Soon

• Loading Data - Complete guide to DatasetConfigs and supported formats (CSV, Excel, MATLAB, NumPy, Parquet)

See Also

• Getting Started - Quick start guide

• Writing a Pipeline in nirs4all - Pipeline syntax reference