trinity.common.models package

Subpackages

• trinity.common.models.vllm_patch package
  • Submodules
    • trinity.common.models.vllm_patch.api_patch module
    • trinity.common.models.vllm_patch.api_patch_v12 module
    • trinity.common.models.vllm_patch.api_patch_v13 module
    • trinity.common.models.vllm_patch.worker_patch module
  • Module contents

Submodules

• trinity.common.models.external_model module
• trinity.common.models.mm_utils module
  • should_use_processor()
  • processor_or_tokenizer_cls()
  • build_mm_message()
  • has_multi_modal_content()
  • combine_output_token_ids()
  • MultiModalRender
    • MultiModalRender.__init__()
    • MultiModalRender.build_mm_input_for_training()
    • MultiModalRender.process_messages()
    • MultiModalRender.process_messages_async()
  • vLLMMultiModalRender
    • vLLMMultiModalRender.__init__()
    • vLLMMultiModalRender.build_mm_input_for_training()
    • vLLMMultiModalRender.process_messages()
    • vLLMMultiModalRender.process_messages_async()
• trinity.common.models.model module
  • InferenceModel
    • InferenceModel.__init__()
    • InferenceModel.generate()
    • InferenceModel.chat()
    • InferenceModel.logprobs()
    • InferenceModel.convert_messages_to_experience()
    • InferenceModel.prepare()
    • InferenceModel.ready()
    • InferenceModel.init_process_group()
    • InferenceModel.teardown_process_group()
    • InferenceModel.set_state_dict_meta()
    • InferenceModel.sync_model_weights()
    • InferenceModel.get_model_version()
    • InferenceModel.get_available_address()
    • InferenceModel.set_master_addr_port()
    • InferenceModel.get_api_server_url()
    • InferenceModel.get_api_key()
    • InferenceModel.get_model_config()
    • InferenceModel.get_model_path()
    • InferenceModel.shutdown()
  • BaseInferenceModel
    • BaseInferenceModel.__init__()
    • BaseInferenceModel.apply_chat_template()
    • BaseInferenceModel.convert_messages_to_experience()
  • ModelWrapper
    • ModelWrapper.__init__()
    • ModelWrapper.prepare()
    • ModelWrapper.generate()
    • ModelWrapper.generate_async()
    • ModelWrapper.chat()
    • ModelWrapper.chat_async()
    • ModelWrapper.logprobs()
    • ModelWrapper.logprobs_async()
    • ModelWrapper.convert_messages_to_experience()
    • ModelWrapper.convert_messages_to_experience_async()
    • ModelWrapper.api_key
    • ModelWrapper.model_version
    • ModelWrapper.model_version_async
    • ModelWrapper.model_path
    • ModelWrapper.model_name
    • ModelWrapper.model_config
    • ModelWrapper.generate_kwargs
    • ModelWrapper.get_available_address_async()
    • ModelWrapper.get_lora_request()
    • ModelWrapper.get_lora_request_async()
    • ModelWrapper.get_message_token_len()
    • ModelWrapper.get_openai_client()
    • ModelWrapper.get_openai_async_client()
    • ModelWrapper.get_current_load()
    • ModelWrapper.init_process_group()
    • ModelWrapper.teardown_process_group()
    • ModelWrapper.set_state_dict_meta()
    • ModelWrapper.sync_model_weights()
    • ModelWrapper.extract_experience_from_history()
    • ModelWrapper.set_workflow_state()
    • ModelWrapper.clean_workflow_state()
    • ModelWrapper.shutdown()
    • ModelWrapper.get_workflow_state()
    • ModelWrapper.clone_with_isolated_history()
• trinity.common.models.tinker_model module
• trinity.common.models.utils module
  • tokenize_and_mask_messages_hf()
  • tokenize_and_mask_messages_default()
  • get_action_mask_method()
  • get_checkpoint_dir_with_step_num()
  • get_latest_state_dict()
  • has_huggingface_model_weights()
  • load_state_dict_iterator()
  • load_state_dict()
  • get_verl_checkpoint_info()
  • load_fsdp_state_dict_from_verl_checkpoint()
  • load_huggingface_state_dict()
  • get_megatron_converter()
• trinity.common.models.vllm_model module
• trinity.common.models.vllm_worker module

Module contents

Inference model classes and utilities.