❭ API reference ❭ data_base ❭ db_initializers ❭

load_simrun_general

Parse Raw simulation results generated with simrun and write to a DataBase

The output format of simrun are Raw simulation results: a nested folder structure with .csv and/or .npz files. This module provides functions to gather and parse this data to pandas and dask dataframes. It merges all trials in a single dataframe. This saves IO time, disk space, and is strongly recommended for HPC systems and other shared filesystems in general, as it reduces the amount of inodes required.

After running init(), a database is created containing the following keys:

Key

Description

simresult_path

Filepath to the raw simulation output of simrun

filelist

List containing paths to all original somatic voltage trace files.

sim_trial_index

The simulation trial indices as a pandas Series.

metadata

A metadata dataframe out of sim_trial_indices

voltage_traces

Dask dataframe containing the somatic voltage traces

parameterfiles_cell_folder

A ManagedFolder containing the Cell parameters file, renamed to its file hash.

parameterfiles_network_folder

A ManagedFolder containing the Network parameters file, renamed to its file hash.

parameterfiles

A pandas dataframe containing the original paths of the parameter files and their hashes.

synapse_activation

Dask dataframe containing the parsed Synapse activation data.

cell_activation

Dask dataframe containing the parsed Presynaptic spike times.

dendritic_recordings

Subdatabase containing the membrane voltage at the recording sites specified in the Cell parameters as a dask dataframe.

dendritic_spike_times

Subdatabase containing the spike times at the recording sites specified in the Cell parameters as a dask dataframe.

spike_times

Dask dataframe containing the spike times of the postsynaptic cell for all trials.

After initialization, you can access the data from the data_base in the following manner:

>>> db['synapse_activation']
<synapse activation dataframe>
>>> db['cell_activation']
<cell activation dataframe>
>>> db['voltage_traces']
<voltage traces dataframe>
>>> db['spike_times']
<spike times dataframe>

If you intialize the database with rewrite_in_optimized_format=True (default), the keys are written as dask dataframes to whichever format is configured as the optimized format (see config). If rewrite_in_optimized_format=False instead, these keys are pickled dask dataframes, containing the instructions to build the dataframe, not the data itself. This is useful for fast intermediate analysis, but strongly discouraged for long term storage, since these instructions contain absolute paths to the original data files, which invalidates once they are moved or deleted. Individual keys can afterwards be set to permanent, self-contained and efficient dask dataframes by calling optimize() on specific database keys.

See also

Raw simulation results for more information on the raw output format of simrun.

See also

init() for the initialization of the database.

Functions

init(db, simresult_path, core, synapse_activation, dendritic_voltage_traces, parameterfiles, spike_times, repartition, scheduler, rewrite_in_optimized_format, dendritic_spike_times, dendritic_spike_times_threshold, client, check_health, n_chunks, voltage_traces, burst_times, dumper, paramfile_copy_config)

Initialize a database with simulation data.

add_dendritic_voltage_traces(db, rewrite_in_optimized_format, dendritic_spike_times, repartition, dendritic_spike_times_threshold, scheduler, client)

Add dendritic voltage traces to the database.

add_dendritic_spike_times(db, dendritic_spike_times_threshold)

Add dendritic spike times to the database.

optimize(db, dumper, select, scheduler, repartition, categorized, client)

Rewrite existing data with a new dumper.

load_initialized_cell_and_evokedNW_from_db(db, sti, allPoints, reconnect_synapses)

Load and set up the cell and network from the database.

Modules

builders

Pipelines for building database keys containing results from simrun.

data_parsing

-

file_handling

-

filepath_resolution

-

health

-

metadata_utils

-

param_file_parser

-

reoptimize

Re-optimize a database with a new dumper

utils

-