Create a Derivative

Create a Derivative#

An important functionality of ancpBIDS is to write derivatives from the in-memory graph to your local disk. The examples in this page cover how to:

  1. Create a derivative in the memory graph, with different folders and subfolders

  2. Create artifacts.

Create a Derivative in the Memory Graph#

The function create_derivative registers a derivative within the in-memory graph. This derivative will be given a name (e.g., “Meg_QC”) and it will be created within the derivatives’ folder of your BIDS dataset. This step only defines the in-memory graph, and it will be written in the local disk later on.

  DERIVATIVE_NAME = "Meg_QC"
  derivative = dataset.create_derivative(name= DERIVATIVE_NAME)

You can also add a description to the derivatives folder:

   derivative.dataset_description.GeneratedBy.Name = "MEG header extraction"

Create Folders & Subfolders#

You can use the function create_folder to create new folders and subfolders, for example for specific pipelines (e.g., “calculation/”).

    PIPELINE_FOLDER = "calculation" 
    calculation_folder = derivative.create_folder(name = PIPELINE_FOLDER)

You can also use the function create_folder to create the subject folders specifying the schema.

# we first query the subject list
subjects: List[str] = sorted(list(dataset.query_entities(scope=SCOPE).get("subject", [])))
print(subjects)
# ['009']

# then we create a for loop subject-wise to obtain every participant's ID (sid)
for sid in subjects:

  subject_folder = calculation_folder.create_folder(
            type_=schema.Subject,
            name="sub-" + sid
  )

Artifacts#

ancpBIDS works with artifacts, which are internal representations of BIDS files. An artifact is not the file itself, but a structured object that stores BIDS entities, suffix, extension and other metadata. In ancpBIDS, the derivative files can be created first as an artifact before being written to disk.

1. Query the raw artifact

The queried entities will be used for the creation of the derivative artifact.

SCOPE = "raw"
SUFFIX = "meg"
EXTENSIONS = [".fif", ".fif.gz"]

raw_artifacts = list(dataset.query(
    subj=sid,
    suffix=SUFFIX,
    scope=SCOPE,
    extension=EXTENSIONS,
    return_type="object",
)) # sid is used within the previously explained participant-wise for loop

print(raw_artifacts)

#Output:
# [{'name': 'sub-009_ses-1_task-deduction_run[...]', 'extension': '.fif', 'suffix': 'meg'}, {'name': 'sub-009_ses-1_task-induction_run[...]', 'extension': '.fif', 'suffix': 'meg'}]

2. Create derivative artifact linked to the raw artifact

This allow us to ensure that the derivative artifact inherits the correct BIDS entities and keeps consitency the raw artifact.

for raw_art in raw_artifacts
# for loop artifact-wise within the participant-wise for loop
# raw art is an existing raw artifact representing a MEG file

  meg_artifact = subject_folder.create_artifact(raw=raw_art)
# creates a new derivative artifact that inhertis the BIDS entities of the raw file

# Entities / naming
  DESC = "mneheader"
  meg_artifact.add_entity("desc", DESC)
# adds an additional BIDS entitity ("desc") to specific the description ("mneheader")
  meg_artifact.suffix = "meg"
  meg_artifact.extension = ".json"

Write derivative to disk#

This is the last step: writting the derivative into the BIDS dataset. This function takes the path from the in-memory dataset and then it creates the actual output file.

ancpbids.write_derivative(dataset, derivative)
Contents