ComfyUI InfoComfyUI Info

comfy_clip_blip_node

This is a ComfyUI node for integrating BLIP into CLIPTextEncode. It's now officially integrated. Dependencies include Fairscale, Transformers, Timm, and Gitpython, with Fairscale needing installation. Local installation involves running commands in specific directories, and Google Colab installation requires adding specific code cells. To use it, add the CLIPTextEncodeBLIP node, connect it to an image and set min_length and max_length. Optionally, embed BLIP text in a prompt using the keyword 'BLIP_TEXT'. Its implementation relies on resources from BLIP, ALBEF, Huggingface Transformers, and timm.

paulo-coronado

29 stars
GitHub

Description

A ComfyUI Node for Adding BLIP in CLIPTextEncode

Announcement

BLIP is now officially integrated into CLIPTextEncode via the comfy_clip_blip_node project.

Dependencies

  • Fairscale ≥ 0.4.4 (not included in ComfyUI)
  • Transformers == 4.26.1 (already included in ComfyUI)
  • Timm ≥ 0.4.12 (already included in ComfyUI)
  • Gitpython (already included in ComfyUI)

Local Installation

Inside the ComfyUI_windows_portable\python_embeded directory, run the following command:

python.exe -m pip install fairscale

Then, inside the ComfyUI_windows_portable\ComfyUI\custom_nodes\ directory, run:

git clone https://github.com/paulo-coronado/comfy_clip_blip_node

Google Colab Installation

Add a cell with the following code:

!pip install fairscale
!cd custom_nodes && git clone https://github.com/paulo-coronado/comfy_clip_blip_node

How to Use the comfy_clip_blip_node

  1. Add the CLIPTextEncodeBLIP node (from the comfy_clip_blip_node project) to your workflow.
  2. Connect the node to an image and select values for min_length and max_length.
  3. (Optional) To embed the BLIP-generated text into a prompt, use the keyword BLIP_TEXT (e.g., "a photo of BLIP_TEXT, medium shot, intricate details, highly detailed").

Acknowledgement

The implementation of CLIPTextEncodeBLIP in the comfy_clip_blip_node relies on resources from BLIP, ALBEF, Huggingface Transformers, and timm. We thank the original authors for their open-source contributions.