data_base ❭ db_initializers ❭ load_simrun_general

load_simrun_general¶

Parse simulation data generated with simrun for general purposes.

The output format of simrun is a nested folder structure with .csv and/or .npz files. The voltage traces are written to a single .csv file (since the amount of timesteps is known in advance, at least for non-variable timesteps), but the synapse and cell activation data is written to a separate file for each simulation trial (the amount of spikes and synapse activations is not known in advance).

This module provides functions to gather and parse this data to pandas and dask dataframes. It merges all trials in a single dataframe. This saves IO time, disk space, and is strongly recommended for HPC systems and other shared filesystems in general, as it reduces the amount of inodes required.

After running init(), a database is created containing the following keys:

After initialization, you can access the data from the data_base in the following manner:

>>> db['synapse_activation']
<synapse activation dataframe>
>>> db['cell_activation']
<cell activation dataframe>
>>> db['voltage_traces']
<voltage traces dataframe>
>>> db['spike_times']
<spike times dataframe>

If you intialize the database with rewrite_in_optimized_format=True (default), the keys are written as dask dataframes to whichever format is configured as the optimized format (see config). If rewrite_in_optimized_format=False instead, these keys are pickled dask dataframes, containing the instructions to build the dataframe, not the data itself. This is useful for fast intermediate analysis, but strongly discouraged for long term storage, since these instructions contain absolute paths to the original data files, which invalidates once they are moved or deleted. Individual keys can afterwards be set to permanent, self-contained and efficient dask dataframes by calling optimize() on specific database keys.

Functions¶

Modules¶