trinity package

Subpackages

• trinity.algorithm package
  • Subpackages
    • trinity.algorithm.advantage_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.entropy_loss_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.kl_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.policy_loss_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.sample_strategy package
      • Submodules
      • Module contents
  • Submodules
    • trinity.algorithm.algorithm module
      • ConstantMeta
      • AlgorithmType
      • SFTAlgorithm
      • PPOAlgorithm
      • GRPOAlgorithm
      • DAPOAlgorithm
      • ReinforcePlusPlusAlgorithm
      • RLOOAlgorithm
      • OPMDAlgorithm
      • AsymREAlgorithm
      • DPOAlgorithm
      • TOPRAlgorithm
      • CISPOAlgorithm
      • GSPOAlgorithm
      • SAPOAlgorithm
      • MIXAlgorithm
      • MIXCHORDAlgorithm
      • RAFTAlgorithm
      • sPPOAlgorithm
      • RECAlgorithm
      • MultiStepGRPOAlgorithm
      • GiGPOAlgorithm
      • OnPolicyDistillAlgorithm
      • JSDAlgorithm
      • GRPOverlAlgorithm
    • trinity.algorithm.key_mapper module
      • KeyMapper
    • trinity.algorithm.utils module
      • aggregate_loss()
      • masked_sum()
      • masked_mean()
      • masked_var()
      • masked_whiten()
      • prefix_metrics()
  • Module contents
    • AlgorithmType
      • AlgorithmType.check_config()
      • AlgorithmType.default_config()
      • AlgorithmType.name()
      • AlgorithmType.use_critic
      • AlgorithmType.use_reference
      • AlgorithmType.compute_advantage_in_trainer
      • AlgorithmType.can_balance_batch
      • AlgorithmType.schema
    • AdvantageFn
      • AdvantageFn.compute_in_trainer()
      • AdvantageFn.default_args()
    • PolicyLossFn
      • PolicyLossFn.__init__()
      • PolicyLossFn.default_args()
      • PolicyLossFn.select_keys
    • KLFn
      • KLFn.__init__()
      • KLFn.apply_kl_penalty_to_reward()
      • KLFn.calculate_kl()
      • KLFn.calculate_kl_loss()
      • KLFn.default_args()
      • KLFn.update_kl_coef()
    • EntropyLossFn
      • EntropyLossFn.default_args()
    • SampleStrategy
      • SampleStrategy.__init__()
      • SampleStrategy.default_args()
      • SampleStrategy.load_state_dict()
      • SampleStrategy.sample()
      • SampleStrategy.set_model_version_metric()
      • SampleStrategy.state_dict()
• trinity.buffer package
  • Subpackages
    • trinity.buffer.operators package
      • Subpackages
      • Submodules
      • Module contents
    • trinity.buffer.pipelines package
      • Submodules
      • Module contents
    • trinity.buffer.reader package
      • Submodules
      • Module contents
    • trinity.buffer.schema package
      • Submodules
      • Module contents
    • trinity.buffer.selector package
      • Submodules
      • Module contents
    • trinity.buffer.storage package
      • Submodules
      • Module contents
    • trinity.buffer.writer package
      • Submodules
      • Module contents
  • Submodules
    • trinity.buffer.buffer module
      • get_buffer_reader()
      • get_buffer_writer()
    • trinity.buffer.buffer_reader module
      • BufferReader
    • trinity.buffer.buffer_writer module
      • BufferWriter
    • trinity.buffer.task_scheduler module
      • get_taskset_scheduler()
      • TasksetSchedulerBase
      • SimpleTasksetScheduler
      • TasksetScheduler
    • trinity.buffer.utils module
      • to_async_url()
      • async_run_with_retry_session()
  • Module contents
    • BufferReader
      • BufferReader.load_state_dict()
      • BufferReader.read()
      • BufferReader.state_dict()
    • BufferWriter
      • BufferWriter.acquire()
      • BufferWriter.release()
      • BufferWriter.write()
    • get_buffer_reader()
    • get_buffer_writer()
• trinity.common package
  • Subpackages
    • trinity.common.models package
      • Subpackages
      • Submodules
      • Module contents
    • trinity.common.rewards package
      • Submodules
      • Module contents
    • trinity.common.workflows package
      • Subpackages
      • Submodules
      • Module contents
  • Submodules
    • trinity.common.config module
      • set_if_none()
      • FormatConfig
      • GenerationConfig
      • OptimizerConfig
      • LoRAConfig
      • DataSelectorConfig
      • ReplayBufferConfig
      • OverRolloutConfig
      • DynamicTimeoutConfig
      • StorageConfig
      • TasksetConfig
      • ExperienceBufferConfig
      • OperatorConfig
      • ExperiencePipelineConfig
      • TaskPipelineConfig
      • DataProcessorConfig
      • TinkerConfig
      • ExternalModelConfig
      • ModelConfig
      • InferenceModelConfig
      • AlgorithmConfig
      • ClusterConfig
      • ExplorerInput
      • TrainerInput
      • BufferConfig
      • ExplorerConfig
      • MegatronParallelConfig
      • TrainerConfig
      • MonitorConfig
      • SynchronizerConfig
      • DataJuicerServiceConfig
      • ServiceConfig
      • LogConfig
      • StageConfig
      • Config
      • load_config()
    • trinity.common.constants module
      • CaseInsensitiveEnumMeta
      • CaseInsensitiveEnum
      • PromptType
      • StorageType
      • SyncMethodEnumMeta
      • SyncMethod
      • RunningStatus
      • OpType
      • SyncStyleEnumMeta
      • SyncStyle
      • SaveStrategy
    • trinity.common.experience module
      • EID
      • CustomField
      • Experience
      • split_dpo_experience_to_single_turn()
      • gather_token_ids()
      • gather_action_masks()
      • gather_attention_masks()
      • gather_response_attrs()
      • gather_multi_modal_inputs()
      • group_by()
      • to_hf_datasets()
      • from_hf_datasets()
    • trinity.common.verl_config module
  • Module contents
• trinity.explorer package
  • Subpackages
    • trinity.explorer.api package
      • Submodules
      • Module contents
  • Submodules
    • trinity.explorer.explorer module
    • trinity.explorer.explorer_client module
    • trinity.explorer.scheduler module
    • trinity.explorer.workflow_runner module
  • Module contents
• trinity.manager package
  • Subpackages
    • trinity.manager.config_registry package
      • Submodules
      • Module contents
  • Submodules
    • trinity.manager.config_manager module
      • ConfigManager
    • trinity.manager.state_manager module
      • StateManager
    • trinity.manager.synchronizer module
      • Synchronizer
  • Module contents
    • StateManager
      • StateManager.__init__()
      • StateManager.load_explorer()
      • StateManager.load_explorer_server_url()
      • StateManager.load_stage()
      • StateManager.load_trainer()
      • StateManager.save_explorer()
      • StateManager.save_explorer_server_url()
      • StateManager.save_stage()
      • StateManager.save_trainer()
    • Synchronizer
      • Synchronizer.trainer_status
      • Synchronizer.explorer_status_counts
      • Synchronizer._ready_condition
      • Synchronizer.model_state_dict
      • Synchronizer.model_version
      • Synchronizer.checkpoint_shard_counter
      • Synchronizer.__init__()
      • Synchronizer.add_module()
      • Synchronizer.coordinate_weight_sync_setup()
      • Synchronizer.coordinate_weight_sync_teardown()
      • Synchronizer.explorer_requires_sync()
      • Synchronizer.get_actor()
      • Synchronizer.get_latest_model_path()
      • Synchronizer.get_latest_model_version()
      • Synchronizer.get_model_state_dict()
      • Synchronizer.get_model_state_dict_iterator()
      • Synchronizer.get_state_dict_meta()
      • Synchronizer.notify_no_new_model_state_dict()
      • Synchronizer.ready_to_nccl_sync()
      • Synchronizer.set_explorer_status()
      • Synchronizer.set_model_state_dict()
      • Synchronizer.set_trainer_status()
      • Synchronizer.setup_weight_sync_group()
      • Synchronizer.trainer_requires_weight_sync()
      • Synchronizer.wait_new_model_state_dict()
• trinity.plugins package
  • Module contents
• trinity.service package
  • Subpackages
    • trinity.service.data_juicer package
      • Subpackages
      • Submodules
      • Module contents
  • Module contents
    • DataJuicerClient
      • DataJuicerClient.__init__()
      • DataJuicerClient.close()
      • DataJuicerClient.initialize()
      • DataJuicerClient.process_experience()
      • DataJuicerClient.process_task()
• trinity.trainer package
  • Subpackages
    • trinity.trainer.verl package
      • Submodules
      • Module contents
  • Submodules
    • trinity.trainer.trainer module
      • Trainer
      • TrainEngineWrapper
      • is_verl_legacy()
      • get_latest_hf_checkpoint_path()
      • get_trainer_wrapper()
    • trinity.trainer.verl_trainer module
  • Module contents
    • Trainer
      • Trainer.__init__()
      • Trainer.get_actor()
      • Trainer.get_weight_sync_info()
      • Trainer.is_alive()
      • Trainer.need_save()
      • Trainer.need_sync()
      • Trainer.prepare()
      • Trainer.save_checkpoint()
      • Trainer.setup_weight_sync_group()
      • Trainer.shutdown()
      • Trainer.sync_weight()
      • Trainer.teardown_weight_sync_group()
      • Trainer.train()
      • Trainer.train_step()
      • Trainer.train_step_num
    • TrainEngineWrapper
      • TrainEngineWrapper.get_weight_sync_info()
      • TrainEngineWrapper.prepare()
      • TrainEngineWrapper.save_checkpoint()
      • TrainEngineWrapper.save_state_dict()
      • TrainEngineWrapper.setup_weight_sync_group()
      • TrainEngineWrapper.sync_weight_nccl()
      • TrainEngineWrapper.teardown_weight_sync_group()
      • TrainEngineWrapper.train_step()
      • TrainEngineWrapper.train_step_num
      • TrainEngineWrapper.upload_state_dict()
      • TrainEngineWrapper.wait_for_save()
    • get_latest_hf_checkpoint_path()
    • get_trainer_wrapper()
    • is_verl_legacy()
• trinity.utils package
  • Submodules
    • trinity.utils.annotations module
      • Experimental()
      • Deprecated()
    • trinity.utils.distributed module
      • is_ipv6_address()
      • get_available_port()
      • get_endpoint()
      • is_port_available()
      • init_process_group()
      • WeightTransferEngine
      • VLLMWeightTransferEngine
      • SGLangWeightTransferEngine
    • trinity.utils.dlc_utils module
      • ClusterStatus
      • get_dlc_env_vars()
      • is_running()
      • wait_for_ray_setup()
      • wait_for_ray_worker_nodes()
      • setup_ray_cluster()
      • stop_ray_cluster()
    • trinity.utils.eval_utils module
    • trinity.utils.log module
      • NewLineFormatter
      • get_logger()
    • trinity.utils.lora_utils module
      • create_dummy_lora()
    • trinity.utils.math_eval_utils module
    • trinity.utils.monitor module
      • Monitor
      • TensorboardMonitor
      • WandbMonitor
      • MlflowMonitor
      • SwanlabMonitor
    • trinity.utils.plugin_loader module
      • load_plugins()
      • load_plugin_from_dirs()
      • load_from_file()
    • trinity.utils.registry module
      • Registry
    • trinity.utils.timer module
      • Timer
  • Module contents

Module contents

Trinity-RFT (Reinforcement Fine-Tuning)