data_base ❭ isf_data_base ❭ ISFDataBase

ISFDataBase¶

class data_base.isf_data_base.ISFDataBase(basedir, readonly=False, nocreate=False, suppress_errors=False)¶

Main database class.

Saved elements can be written and accessed using dictionary syntax:

>>> db['my_new_element'] = my_new_element
>>> my_reloaded_element = db['my_new_element']

To read out all existing keys, use the keys() method. Valid keys are str or (nested) tuples of str for a (nested) hierarchy. “@” is not allowed.

All saved elements are stored in the basedir along with metadata and a Loader.json object. The Loader.json object contains which module should be used to load the data with, along with all the necessary information to initialize the Loader. The following metadata is saved:

Metadata Associated with Saved Elements¶
Metadata	Description
`dumper`	Which data dumper was used to save this result. Its corresponding Loader can always be found in the same file. See `LoaderDumper` for all dumpers and loaders.
`time`	Time at which this results was saved.
`module_list`	A full list of all modules installed in the conda environment that was used to produce this result.
`module_versions`	The versions of all modules in the environment that was used to produce this result. See also: `get_module_versions`.
`history`	The history of the code that was used to produce this result. Only supported if the code was run using IPython (e.g. from within a Jupyter Notebook). See also: `get_history`.
`hostname`	Name of the machine the code was run on.

If the dask backends are used to save the data, it will be saved out-of-memory, allowing larger-than-memory calculations.

Parameters:¶

basedir (str) – The directory in which the database will be created, or read from.
readonly (bool, optional) – If True, the database will be read only. Defaults to False.
nocreate (bool, optional) – If True, a new database will not be created if it does not exist. Defaults to False.
suppress_errors (bool, optional) – If True, errors will be suppressed and raised as warnings instead. Defaults to False. Use with caution.

Attributes:¶

basedir¶

The directory in which the database will be created, or read from.

Type:¶: str

readonly¶

If True, the database will be read-only.

Type:¶: bool

nocreate¶

If True, a new database will not be created if it does not exist.

Type:¶: bool

metadata¶

A dictionary containing metadata for the database. See also: MetadataAccessor.

Type:¶: dict

parent_db¶

The parent database, if this is a sub-database. Default: None.

Type:¶: ISFDataBase

_unique_id¶

A unique identifier for this database.

Type:¶: str

_registered_to_path¶

The path that this database has been registered to on the current filesystem.

Type:¶: str

_registeredDumpers¶

A list of all registered dumpers. Dumpers are data-type and file-type specific modules to write out data. See: LoaderDumper

Type:¶: list

_suppress_errors¶

If True, errors will be suppressed and raised as warnings instead. Use with caution.

Type:¶: bool

_db_state_fn¶

The path to the database state file. Contains information on:

_registeredDumpers: A list of all registered dumpers.
_unique_id: A unique identifier for this database.
_registered_to_path: The path that this database has been registered to on the current filesystem.

Type:¶: str

_forbidden_keys¶

A list of keys that are not allowed to be used: ["Loader.json", "metadata.db.lock", "sqlitedict.db.lock", "db_state.json"]

Type:¶: list

_basedir¶

pathlib.Path object of basedir, to use internally.

Type:¶: Path

Methods:¶

`_check_is_legacy_mdb`()	-
`_infer_missing_metadata`()	Checks whether metadata is missing, and tries to estimate it.
`_register_this_database`()	Register this database with the database register.
`_set_unique_id`()	Sets a unique ID for the DataBase as class attribute.
`_is_initialized`()	Check if the current database is initialized.
`_initialize`()	Initialize the database.
`_convert_key_to_path`(key)	Infer the file path from a database key.
`_check_key_format`(key_str_tuple)	Checks if a key is valid for setting data.
`_detect_dumper_string_of_existing_key`(key)	Get the dumper of an existing key as a string (not module).
`_find_dumper`(item)	Finds a suitable dumper of a given item.
`_write_metadata`(dumper, dir_to_data)	Writes the metadata of this database.
`_check_writing_privilege`(key)	Check if we have permission to write to a key.
`check_if_key_exists`(key)	Check if a key exists in the database.
`get_id`()	Get the unique ID of this database.
`register_dumper`(dumper_module)	Register a dumper with this database, so it can be used to save data.
`save_db_state`()	Saves the data which defines the state of this database to `db_state.json`
`read_db_state`()	Sets the state of the database according to `db_state.json`
`get_mkdtemp`(prefix, suffix)	Create a temporary directory in the data_base
`create_managed_folder`(key, raise_)	Create a folder in the db directory.
`create_shared_numpy_store`(key, raise_)	Create a shared numpy store in the db directory.
`create_sub_db`(key, register, **kwargs)	Creates a database within a database.
`get`(key, lock, **kwargs)	Get data from the database.
`rename`(old_key, new_key)	Rename a key in the database.
`set`(key, value, lock, dumper, **kwargs)	Main method to save data in a DataBase.
`maybe_calculate`(key, fun, **kwargs)	Get or calculate a value in the database.
`keys`(recurse)	Get the keys of the database.
`_subkeys`(keys)	Returns all keys of a database, including their subdatabase keys.
`ls`(depth, max_depth, max_lines, all_files, max_lines_per_key, color)	Prints out the content of the database in a tree structure.
`_get_str`(depth, max_depth, max_lines, all_files, max_lines_per_key, color)	Fetches a string representation for this db in a tree structure.
`remove`()	Deletes the database.