trinity.buffer.pipelines package

Submodules

• trinity.buffer.pipelines.experience_pipeline module
  • get_input_buffers()
  • ExperiencePipeline
    • ExperiencePipeline.__init__()
    • ExperiencePipeline.prepare()
    • ExperiencePipeline.process()
    • ExperiencePipeline.close()
• trinity.buffer.pipelines.task_pipeline module
  • check_and_run_task_pipeline()
  • TaskPipeline
    • TaskPipeline.__init__()
    • TaskPipeline.convert_pipeline_config()
    • TaskPipeline.process()
    • TaskPipeline.close()

Module contents

class trinity.buffer.pipelines.ExperiencePipeline(config: Config)

Bases: object

A class to process experiences.

__init__(config: Config)

async close() → None

async prepare() → None

async process(exp_bytes: bytes) → Dict

Process a batch of experiences.

Parameters:

exp_bytes (bytes) – Serialized experiences to process. These experiences are typically generated by an explorer in one step.

Returns:

A dictionary containing metrics collected during the processing of experiences.

Return type:

Dict

class trinity.buffer.pipelines.TaskPipeline(config: Config)

Bases: object

A class to process task datasets through DataJuicer.

__init__(config: Config)

close()

convert_pipeline_config(pipeline_config: TaskPipelineConfig) → Dict[str, Any]

Convert the TaskPipelineConfig to a format suitable for DataJuicer.

process() → Dict[str, Any]

Process the task datasets using DataJuicer.

Returns:

Metrics for logging.

Return type:

Dict[str, Any]

trinity.buffer.pipelines.check_and_run_task_pipeline(config: Config) → Dict