About

Why BIDS Manager exists

BIDS Manager converts raw recordings into a BIDS dataset, but converting is the smaller part of what it does. It is curation software, built for the work that turns a set of converted files into a dataset other people can use: describing it, checking it, correcting it, and keeping a record of what was done.

What a converter does, and what it leaves

A converter reads one format and writes another. Given a folder of DICOM files, dcm2niix writes NIfTI images and their sidecars; given an EEG recording, MNE-BIDS writes it into the BIDS layout. These tools are good at their job, and BIDS Manager uses them rather than writing its own.

What a converter cannot do is know what is not in the files, or notice what is wrong with them. In a lab, that is most of the work. Here are the same situations, with a converter alone and with BIDS Manager. Each happens on the sample data the course and the guides use; switch between the two to compare.

  • One participant, four subjects

    With a converter aloneEach source is converted under its own identity: the MRI scanner's patient ID, the PET study's, the EEG folder's code, the eye tracker's folder. One person becomes four subjects, and nobody notices until the analysis.

    With BIDS ManagerEvery recording is a row in one table before anything is written. The course's sample proposes four subjects for one participant; one bulk edit sets them all to sub-001. See it

  • A FLAIR called T2w

    With a converter aloneA converter that trusts the series name writes ses-pre_T2w as a T2-weighted image, because that is what someone typed at the scanner.

    With BIDS ManagerThe parameter classifier reads how the series was acquired: an inversion long enough to null the fluid. It is named FLAIR, and Why this name shows the evidence. See it

  • A fieldmap that corrects nothing

    With a converter aloneThe fieldmaps are written, but nothing links them to the runs they correct. The dataset is valid, and the correction never happens.

    With BIDS ManagerEach fieldmap is linked to the runs it corrects, by their shim settings and then by time, with IntendedFor and with B0FieldIdentifier and B0FieldSource, the way fMRIPrep reads them. See it

  • A task nobody described

    With a converter aloneTaskDescription and Instructions are not in any file, so they are not in the dataset, and nothing says they are missing.

    With BIDS ManagerA template for every BOLD sidecar asks for them once and writes the answer into every run; what nobody answered is a TODO placeholder that validation counts until it is filled. See it

  • A PET dose on a sheet of paper

    With a converter aloneThe injected mass, the specific activity and the mode of administration are not in the DICOM header, so the PET sidecar lacks fields BIDS requires.

    With BIDS ManagerThe lab's dose sheet is imported before converting: seven fields of the course's sheet land in the PET sidecar, checked where they will be written. See it

  • Events written as codes

    With a converter aloneThe EEG's events.tsv lists the trigger codes T0, T1 and T2, and only the person who wrote the experiment knows what they mean.

    With BIDS ManagerEach code is given a label once, in Dataset metadata, and the labels are written into the trial_type column of every run. See it

  • A face in a head scan

    With a converter aloneAn anatomical image holds the participant's face; anyone can render it from the dataset.

    With BIDS ManagerImages are defaced during or after the conversion, the result is checked side by side, and the quality check reports a face still present. See it

  • A renamed file that breaks its references

    With a converter aloneRenaming a file by hand leaves the scans table, the IntendedFor of the fieldmaps and the participants table naming a file that no longer exists.

    With BIDS ManagerThe Editor's renaming moves the file and every reference to it in one operation: the scans tables, IntendedFor and the other fields that name files, the participants and sessions tables, and TaskName where it was derived from the task. See it

What happens in real labs

Metadata nobody records. BIDS asks for facts no scanner or amplifier writes down: who made the dataset and under what licence, what the participant was asked to do, which electrode was the reference, how much tracer was injected and when. A converter leaves these out, or writes n/a, and the file looks complete. The answers exist, on a dose sheet beside the scanner, in a protocol document, in someone's memory; they need to be asked for once and written into every file that needs them.

Sources that disagree. One participant visits an MRI scanner, a PET camera and an EEG lab. The scanner names them by a hospital ID, the EEG folder by a code, the eye tracker not at all. Converted one by one, they become three subjects. A run that was aborted and repeated sits beside the good one; a fieldmap belongs to the second run, not the first; two series would get the same file name and one would overwrite the other.

Identifiers left in the files. Removing the name from a sidecar is easy to remember. The patient ID copied into a participants table, the birth date inside a spectroscopy file, a subject label made from initials, a face that can be rendered from a head scan: these are easy to miss.

Mistakes that do not announce themselves. None of the above makes a converter fail, and many of them pass validation. A dataset can be valid BIDS and still say something false about its recordings, or lose a recording, or link a fieldmap to nothing. They are found months later, by whoever analyses the data, or not at all.

Work that is never finished. Participants arrive over months. A table is curated, then a new batch comes in. A sidecar is corrected after a reviewer asks. Without a record of what was done, every round starts from the beginning, and nobody can say afterwards what changed.

Metadata annotation: answering what no file says

A BIDS dataset is more than its images and recordings: every one of them carries a sidecar, and the standard says which of its fields are required, recommended or optional, for each kind of file. A converter fills the fields it can read from the raw data. The rest, often the ones that matter most to whoever reuses the data, someone has to write: what the participant was asked to do, how the EEG was referenced, how much tracer was injected, who made the dataset and under what licence.

BIDS Manager treats that writing as part of the work, not as an afterthought:

  • It asks once, at the right level. The dataset description for the whole dataset; a template for each kind of file (every *_bold.json, every *_eeg.json), written into every file of that kind; a value for one recording in Properties; dose sheets and participant spreadsheets imported as the lab keeps them.
  • It shows where every value comes from. In the BIDS preview, before converting, and in the sidecar forms after, a value is marked as read from the recording, written by BIDS Manager, given by you, or still to answer.
  • It keeps what is missing visible. A field nobody answered is written as a TODO placeholder rather than left out, so validation counts it and the sidecar search finds it, until someone replaces it.
  • It lets you correct afterwards. The Editor shows every sidecar as a form of the standard's fields, with their levels, and finds and fixes a value across every file at once.

Here is one sidecar of the course's sample after chapter 10, each value tagged with where it came from:

multimodal_study/
├── dataset_description.json  participants.tsv  README
└── sub-001/ses-01/
    ├── anat/ sub-001_ses-01_acq-tfl3p2_T1w.nii.gz  .json
    ├── func/ sub-001_ses-01_task-rest_bold.nii.gz  .json
    │         sub-001_ses-01_task-rest_sbref.nii.gz  .json
    │         sub-001_ses-01_task-rest_recording-ecg_physio.tsv.gz  .json
    ├── fmap/ sub-001_ses-01_acq-fm2_phasediff.nii.gz  .json
    ├── dwi/  sub-001_ses-01_acq-epb0p2_dir-AP_dwi.nii.gz  .bval .bvec  .json
    ├── pet/  sub-001_ses-01_pet.nii.gz  .json
    ├── mrs/  sub-001_ses-01_svs.nii.gz  .json
    ├── eeg/  sub-001_ses-01_task-rest_eeg.edf  _channels.tsv _events.tsv  .json
    ├── meg/  sub-001_ses-01_task-rest_meg.fif  _channels.tsv  .json
    └── beh/  sub-001_ses-01_task-rest_recording-eye1_physio.tsv.gz  .json

sub-001_ses-01_task-rest_bold.json

{
"TaskName": "rest"written by BIDS Manager
"RepetitionTime": 0.85read from the recording
"EchoTime": 0.03read from the recording
"PhaseEncodingDirection": "j-"read from the recording
"B0FieldSource": "fmap012"written by BIDS Manager
"TaskDescription": "Resting state with the eyes open, looking at a fixation cross."your answer
"Instructions": "Keep your eyes on the cross and let your mind wander."your answer
"CogAtlasID": "TODO"to answer
}

What curation means in BIDS Manager

Curation is everything between the raw folder and a dataset you would hand to someone else. In BIDS Manager each part of it has a place:

Curation task Where it happens
See every recording before anything is written The scan table: one row per recording, with the subject, session, task and BIDS name proposed for it, and the reason for each name.
Reconcile what the sources disagree about Editing the table, Bulk edit for many rows at once, new subject numbers for a second batch, and the Overview, which compares subjects and points out what differs between them.
Give the facts no file records Dataset metadata: the dataset description, a template for each kind of file, a value for one recording in Properties; dose sheets and participant spreadsheets imported as they are.
See what is still unanswered TODO placeholders in the fields the standard asks for that nobody answered, wherever a field can hold one, counted by validation until they are replaced.
See the dataset before it is written The BIDS preview: every file the conversion will write, and each value in it coloured by where it comes from.
Check the result Validation against the standard, each finding shown in the file and field it is about; the quality check of every anatomical and diffusion image.
Correct the dataset afterwards The Editor: sidecars and tables as forms; renaming an entity, restructuring, sessions and deleting, each repairing every reference to the files it moves; finding values and fixing them in every file at once.
Keep identities out Identifiers removed from every sidecar and spectroscopy file the conversion writes; defacing of anatomical images.
Keep a record The project: every scan, every edit to the table (which Undo takes back), every change the Editor makes with the files as they were, and a log of each run (the newest 200), kept with the dataset.

The course follows these steps in order on one sample dataset: Learn BIDS Manager.

Who it is for

  • Researchers converting their own studies, who want a dataset that is right without learning each converter's options.
  • People who look after data for a lab, who convert the same kind of study again and again, and need to know what was done to each dataset.
  • Anyone preparing data to share, on OpenNeuro or elsewhere, who has to remove identifiers and describe the dataset properly.
  • Anyone who inherits a dataset, and needs to understand it, check it and correct it.

What it does not do

BIDS Manager does not analyse data: it does not preprocess images or recordings, and the quality check is a screening, not a pipeline. It does not upload anything; datasets stay on your computer until you send them. And it does not replace looking at your data: it puts the data in front of you, and makes the decisions visible.

Read on

  • Mistakes it catches: the silent failures BIDS Manager is built to prevent, kind of data by kind of data.
  • The project: the engines it uses, what it does with your data, who makes it, and its licence.