ibllib.io.raw_data_loaders

Raw Data Loader functions for PyBpod rig

Module contains one loader function per raw datafile

ibllib.io.raw_data_loaders.get_port_events(trial: dict, name: str = '') → list

Return all event timestamps from bpod raw data trial that match ‘name’ –> looks in trial[‘behavior_data’][‘Events timestamps’]

Parameters:
  • trial (dict) – raw trial dict
  • name (str, optional) – name of event, defaults to ‘’
Returns:

Sorted list of event timestamps

Return type:

list

TODO: add polarities?

ibllib.io.raw_data_loaders.get_session_extractor_type(session_path)

From a session path, loads the settings file, finds the task and checks if extractors exist task names examples: :param session_path: :return: bool

ibllib.io.raw_data_loaders.get_task_extractor_type(task_name)

Splits the task name according to naming convention: - ignores everything _iblrig_tasks_biasedChoiceWorld3.7.0 returns “biased” _iblrig_tasks_trainingChoiceWorld3.6.0 returns “training’ :param task_name: :return: one of [‘biased’, ‘habituation’, ‘training’, ‘ephys’, ‘mock_ephys’, ‘sync_ephys’]

ibllib.io.raw_data_loaders.load_ambient_sensor(session_path)

Load Ambient Sensor data from session.

Probably could be extracted to DatasetTypes: _ibl_trials.temperature_C, _ibl_trials.airPressure_mb, _ibl_trials.relativeHumidity Returns a list of dicts one dict per trial. dict keys are: dict_keys([‘Temperature_C’, ‘AirPressure_mb’, ‘RelativeHumidity’])

Parameters:session_path (str) – Absoulte path of session folder
Returns:list of dicts
Return type:list
ibllib.io.raw_data_loaders.load_bpod(session_path)

Load both settings and data from bpod (.json and .jsonable)

Parameters:session_path – Absolute path of session folder
Returns:dict settings and list of dicts data
ibllib.io.raw_data_loaders.load_bpod_fronts(session_path: str, data: dict = False) → list

Loads BNC1 and BNC2 bpod channels times and polarities from session_path

Parameters:
  • session_path (str) – a valid session_path
  • data (dict, optional) – pre-loaded raw data dict, defaults to False
Returns:

List of dicts BNC1 and BNC2 {“times”: np.array, “polarities”:np.array}

Return type:

list

ibllib.io.raw_data_loaders.load_data(session_path: Union[str, pathlib.Path], time='absolute')

Load PyBpod data files (.jsonable).

Bpod timestamps are in microseconds (µs) PyBpod timestamps are is seconds (s)

Parameters:session_path (str, Path) – Absolute path of session folder
Returns:A list of len ntrials each trial being a dictionary
Return type:list of dicts
ibllib.io.raw_data_loaders.load_encoder_events(session_path, settings=False)

Load Rotary Encoder (RE) events raw data file.

Assumes that a folder called “raw_behavior_data” exists in folder.

Events number correspond to following bpod states: 1: correct / hide_stim 2: stim_on 3: closed_loop 4: freeze_error / freeze_correct

>>> data.columns
>>> ['re_ts',   # Rotary Encoder Timestamp (ms) 'numpy.int64'
     'sm_ev',   # State Machine Event           'numpy.int64'
     'bns_ts']  # Bonsai Timestamp (int)        'pandas.Timestamp'
    # pd.to_datetime(data.bns_ts) to work in datetimes
Parameters:session_path ([type]) – [description]
Returns:dataframe w/ 3 cols and (ntrials * 3) lines
Return type:Pandas.DataFrame
ibllib.io.raw_data_loaders.load_encoder_positions(session_path, settings=False)

Load Rotary Encoder (RE) positions from raw data file within a session path.

Assumes that a folder called “raw_behavior_data” exists in folder. Positions are RE ticks [-512, 512] == [-180º, 180º] 0 == trial stim init position Positive nums are rightwards movements (mouse) or RE CW (mouse)

Variable line number, depends on movements.

Raw datafile Columns:
Position, RE timestamp, RE Position, Bonsai Timestamp

Position is always equal to ‘Position’ so this column was dropped.

>>> data.columns
>>> ['re_ts',   # Rotary Encoder Timestamp (ms)     'numpy.int64'
     're_pos',  # Rotary Encoder position (ticks)   'numpy.int64'
     'bns_ts']  # Bonsai Timestamp                  'pandas.Timestamp'
    # pd.to_datetime(data.bns_ts) to work in datetimes
Parameters:session_path (str) – Absolute path of session folder
Returns:dataframe w/ 3 cols and N positions
Return type:Pandas.DataFrame
ibllib.io.raw_data_loaders.load_encoder_trial_info(session_path)

Load Rotary Encoder trial info from raw data file.

Assumes that a folder calles “raw_behavior_data” exists in folder.

NOTE: Last trial probably inexistent data (Trial info is sent on trial start and data is only saved on trial exit…) max(trialnum) should be N+1 if N is the amount of trial data saved.

Raw datafile Columns:

>>> data.columns
>>> ['trial_num',     # Trial Number                     'numpy.int64'
     'stim_pos_init', # Initial position of visual stim  'numpy.int64'
     'stim_contrast', # Contrast of visual stimulus      'numpy.float64'
     'stim_freq',     # Frequency of gabor patch         'numpy.float64'
     'stim_angle',    # Angle of Gabor 0 = Vertical      'numpy.float64'
     'stim_gain',     # Wheel gain (mm/º of stim)        'numpy.float64'
     'stim_sigma',    # Size of patch                    'numpy.float64'
     'stim_phase',    # Phase of gabor                    'numpy.float64'
     'bns_ts' ]       # Bonsai Timestamp                 'pandas.Timestamp'
    # pd.to_datetime(data.bns_ts) to work in datetimes
Parameters:session_path (str) – Absoulte path of session folder
Returns:dataframe w/ 9 cols and ntrials lines
Return type:Pandas.DataFrame
ibllib.io.raw_data_loaders.load_mic(session_path)

Load Microphone wav file to np.array of len nSamples

Parameters:session_path (str) – Absoulte path of session folder
Returns:An array of values of the sound waveform
Return type:numpy.array
ibllib.io.raw_data_loaders.load_settings(session_path: Union[str, pathlib.Path])

Load PyBpod Settings files (.json).

[description]

Parameters:session_path (str, Path) – Absolute path of session folder
Returns:Settings dictionary
Return type:dict
ibllib.io.raw_data_loaders.sync_trials_robust(t0, t1, diff_threshold=0.001, drift_threshold_ppm=200, max_shift=5, return_index=False)

Attempts to find matching timestamps in 2 time-series that have an offset, are drifting, and are most likely incomplete: sizes don’t have to match, some pulses may be missing in any series. Only works with irregular time series as it relies on the derivative to match sync. :param t0: :param t1: :param diff_threshold: :param drift_threshold_ppm: (150) :param max_shift: (200) :param return_index (False) :return:

ibllib.io.raw_data_loaders.trial_times_to_times(raw_trial)

Parse and convert all trial timestamps to “absolute” time. Float64 seconds from session start.

0—BpodStart—TrialStart0———TrialEnd0—–TrialStart1—TrialEnd1…0—ts0—ts1— tsN…absTS = tsN + TrialStartN - BpodStart

Bpod timestamps are in microseconds (µs) PyBpod timestamps are is seconds (s)

Parameters:raw_trial (dict) – raw tiral data
Returns:trial data with modified timestamps
Return type:dict