Targets

Targets are the way used by the pipeline to know if the expected Inputs/Outputs (I/O) of a task are existing/produced. A target should contain all the information necessary to the pipeline to make its decision on running or not the next step of the chain of treatment.

An instance of a target is fully identified by the its identifier, its version and the name of the file. Thus, the target can be seen as a category/group of file with a physical representation of them.

Target creation

Using class

A target instance can be created simply by instantiating a poppy.pop.target.Target class. The mandatory parameters have to be given at the instantiation. For example, a target for the dataset ROC-SGSE_LZ_MEB-SGSE-TEST-LOG in version 01, and a physical file /path/to/dataset/file:

from poppy.pop import Target

target = Target(
    "/path/to/dataset/file", # filename of the dataset
    "ROC-SGSE_LZ_MEB-SGSE-TEST-LOG", # identifier for the target
    "01", # version of the target
)

This results in a target instance that can be used and shared across the pipeline, through the context. Placing it in the context allows the pipeline to easily find targets used as I/O for tasks. See target_usage for details.

Using task from plugin

A target can also be created from a task instance created from a plugin. The plugin contains all necessary information for the target creation, allowing an easy and maintainable way of referencing and creating target instances. See Tasks.

If for example the target is named xml_test_log in the descriptor file of a plugin called DARE-SGSE, a target instance can easily be created with (assuming a task named to_xml_file):

from poppy.pop.plugins import Plugin

# create the class for the task
Task = Plugin.manager["DARE-SGSE"].task("to_xml_file")

# create a fake functionality for the task
@Task.as_task
def run(task):
    """
    task is an instance of the task to_xml_file, created from a plugin. It
    can be used to create a target from informations provided in the
    descriptor.
    """
    # first get the class of the target
    Target = task.target("xml_test_log")

    # create the instance with a path to the file
    target = Target("/path/to/target")

Danger

If it is an input target, input_target must be used instead of target as method on the task instance.

Usage

All status changes of a target must be done inside its context. It allows the target to intercept any problem occurring while generating a file in input or output, and to gives the good error information to the POPPy framework.

For example, to use the context of the target and open the file inside it:

# use the context of the target
with target.activate():
    # open the file
    with target.open("r") as f:
        # do things with the file

This example is simple, but shows how the target handles all the job for the user. The status of the target is automatically updated according to each step. In case of an error inside this context, the status of the target is set accordingly to poppy.pop.target.Target.error() and reported into the database.

At startup, it is marked as poppy.pop.target.Target.ok() and poppy.pop.target.Target.pending().

If while treating the file, the target is detected as empty, the :exception:`poppy.pop.target.Target.TargetEmpty` must be raised, handled by the context to mark the target as empty. It is used by the pipeline to know that a file has been treated correctly, but that it is not existing, and this is normal.

The available status of a target are listed in API.

API