data_loader#

class arte.dataelab.data_loader.ConstantDataLoader(data)#

Bases: DataLoader

Loader for constant data

Methods

`assert_exists`()	Assert that the data is available
`filename`()	Return the data filename, if available
`help`([search, prefix])	Interactive help
`load`()	Load data and return it

assert_exists()#: Assert that the data is available

filename()#: Return the data filename, if available

load()#: Load data and return it

class arte.dataelab.data_loader.DataLoader#

Bases: object

Abstract base class for lazy data loading.

DataLoaders enable lazy loading where data is read from disk only when accessed, not when the loader is created. This allows:

Fast analyzer initialization
Reduced memory usage
Early file existence checks
Preprocessing and transformation pipelines

Built-in loaders are provided for common formats:

NumpyDataLoader: .npy and .npz files
FitsDataLoader: FITS files

Custom loaders can be created by implementing the three abstract methods: assert_exists(), filename(), and load().

Examples

>>> # Using built-in loaders
>>> loader = NumpyDataLoader('data.npy')
>>> loader.assert_exists()  # Check file exists
>>> data = loader.load()     # Load data

>>> # For .npz files with multiple arrays
>>> loader = NumpyDataLoader('data.npz', key='slopes')

>>> # FITS files with specific extension
>>> loader = FitsDataLoader('data.fits', ext=1)

>>> # Custom loader with preprocessing
>>> class MyLoader(DataLoader):
...     def __init__(self, filename):
...         self._filename = filename
...     def assert_exists(self):
...         assert os.path.exists(self._filename)
...     def filename(self):
...         return self._filename
...     def load(self):
...         data = np.load(self._filename)
...         return data - data.mean()  # Remove mean

See also

BaseTimeSeries: Uses DataLoaders for lazy data access
data_loader_factory: Factory function to create appropriate loaders

Methods

`assert_exists`()	Assert that the data is available
`filename`()	Return the data filename, if available
`help`([search, prefix])	Interactive help
`load`()	Load data and return it

abstractmethod assert_exists()#: Assert that the data is available

abstractmethod filename()#: Return the data filename, if available

help(search='', prefix='')#

Interactive help

Prints on stdout a list of methods that match the search substring or all of them if search is left to the default value of an empty string, together with a one-line help taken from the first line of their docstring, if any.

The prefix argument is prepended to the method name and is used for recursive help of every class member.

abstractmethod load()#: Load data and return it

class arte.dataelab.data_loader.FitsDataLoader(filename, ext=None, transpose_axes=None, postprocess=None)#

Bases: DataLoader

Loader for data stored into FITS files

Parameters:

filename (str) – FITS filename or full path
ext (int, optional) – FITS extenstion to read. Defaults to the primary card
transpose_axes (tuple, optional) – Axes transpose pattern as specified for np.transpose. Use this if time is not your first dimension
postprocess (function, optional) – function to call after loading data and before returning it Must take a single parameter with the whole data array and return the processed data array.

Methods

`assert_exists`()	Assert that the data is available
`filename`()	Return the data filename, if available
`help`([search, prefix])	Interactive help
`load`()	Load data and return it

assert_exists()#: Assert that the data is available

filename()#: Return the data filename, if available

load()#: Load data and return it

class arte.dataelab.data_loader.NumpyDataLoader(filename, key=None, transpose_axes=None, postprocess=None)#

Bases: DataLoader

Loader for data stored into np or npz files

Parameters:

filename (str) – numpy filename or full path
key (str, optional) – array key for .npz files
transpose_axes (tuple, optional) – Axes transpose pattern as specified for np.transpose. Use this if time is not your first dimension
postprocess (function, optional) – function to call after loading data and before returning it Must take a single parameter with the whole data array and return the processed data array.

Methods

`assert_exists`()	Assert that the data is available
`filename`()	Return the data filename, if available
`help`([search, prefix])	Interactive help
`load`([allow_pickle])	Load data and return it

assert_exists()#: Assert that the data is available

filename()#: Return the data filename, if available

load(allow_pickle=True)#: Load data and return it

class arte.dataelab.data_loader.OnTheFlyLoader(func)#

Bases: DataLoader

Loader for data calculated on the fly

Methods

`assert_exists`()	Assert that the data is available
`filename`()	Return the data filename, if available
`help`([search, prefix])	Interactive help
`load`()	Load data and return it

assert_exists()#: Assert that the data is available

filename()#: Return the data filename, if available

load()#: Load data and return it

class arte.dataelab.data_loader.TxtDataLoader(filename, transpose_axes=None, postprocess=None)#

Bases: DataLoader

Loader for data stored into txt files

Parameters:

filename (str) – numpy filename or full path
transpose_axes (tuple, optional) – Axes transpose pattern as specified for np.transpose. Use this if time is not your first dimension
postprocess (function, optional) – function to call after loading data and before returning it Must take a single parameter with the whole data array and return the processed data array.

Methods

`assert_exists`()	Assert that the data is available
`filename`()	Return the data filename, if available
`help`([search, prefix])	Interactive help
`load`()	Load data and return it

assert_exists()#: Assert that the data is available

filename()#: Return the data filename, if available

load()#: Load data and return it

arte.dataelab.data_loader.data_axes(obj)#

Returns:: axes – axes names tuple
Return type:: tuple(str) or None

arte.dataelab.data_loader.data_loader_factory(obj, allow_none=False, name='')#

Return the correct DataLoader instance for obj, which might be: * a string with a filename among the known ones (fits or npy) * a pathlib.Path instance * a numpy array * a DataLoader instance (returned unchanged)

Raises ValueError if the guess fails,

Parameters:

obj (Loader class, numpy array, str (filename), Path instance, or None) – object to wrap in a DataLoader class
allow_none (bool, optional) – if set to True, obj can be None. If this flag is False and obj is None, a ValueError exception willb be raised.
name (str, optional) – object name for error messages.

Return type:

DataLoader instance