data_base ❭ IO ❭ LoaderDumper

LoaderDumper¶

Read and write data in various formats.

This package provides IO modules that always contain three components:

A dump() function to write out the data, and its corresponding Loader object.
A Loader class that can load the data back into memory.
a check() method that checks whether the object can be saved with this dumper.

To save an object, the dump method is called:

>>> import my_dumper
>>> my_dumper.dump(obj, savedir)

This saves the object as specified in the respective dump() method. In addition, a Loader.json is saved alongside the data. This file contains the specification of a Loader object, which can then be initialized and contains all the mechanisms to load the object back into memory.

Functions¶

`resolve_loader_dumper_path`(loader_path)	Resolve a loader path to an absolute path.
`load`(savedir, load_data, loader_kwargs)	Standard interface to load data.
`get_dumper_string_by_dumper_module`(dumper_module)	Convert a dumper submodule to a string.
`get_dumper_string_by_savedir`(savedir)	Get the dumper string from a filepath.

Modules¶

`cell`	Save and load `Cell` objects to and from `.pickle` format.
`dask_to_categorized_msgpack`	Save and load dask dataframes to msgpack with categorical columns.
`dask_to_msgpack`	Save and load dask dataframes to msgpack.
`dask_to_parquet`	Save and load dask dataframes to and from Apache parquet format.
`just_create_folder`	Create a folder and return it as a ManagedFolder object.
`just_create_isf_db`	Create and load `ISFDataBase` objects in a database.
`numpy_to_msgpack`	Read and write numpy arrays to msgpack files
`numpy_to_npy`	Read and write a numpy array to `npy` format.
`numpy_to_npz`	Read and write a numpy array to the compressed `.npz` format.
`numpy_to_zarr`	Read and write a numpy array to the `zarr` format.
`pandas_to_msgpack`	Save and load pandas dataframes to msgpack files.
`pandas_to_parquet`	Read and write a pandas DataFrame to the parquet format.
`pandas_to_pickle`	Read and write a pandas DataFrame to the pickle format.
`parent_classes`	Base class for child Loader classes
`reduced_lda_model`	Read and write a `ReducedLdaModel`.
`shared_numpy_store`	Read and write numpy arrays to and from shared memory.
`to_cloudpickle`	Read and write an object to the cloudpickle format.
`to_msgpack`	Read and write objects to the msgpack format.
`to_pickle`	Read and write objects to the pickle format.
`utils`	Convenience methods for data IO.