{ "titleSector": { "title": "ComfyUI ControlNet Aux 辅助预处理器", "subtitle": "即插即用的ComfyUI ControlNet Aux节点集，用于创建ControlNet提示图像", "backgroundImage": "https://source.unsplash.com/random/1920x1080/?ai,technology", "buttons": [ { "text": "了解更多", "href": "#功能概述", "variant": "primary" }, { "text": "安装指南", "href": "#安装", "variant": "secondary" } ] }, "sections": { "overview": { "id": "功能概述", "title": "ControlNet Aux 功能概述", "content": [ { "title": "什么是ControlNet Aux辅助预处理器？", "description": "这是一个为ComfyUI设计的ControlNet Aux即插即用节点集，用于创建ControlNet提示图像。代码从ControlNet项目的相应文件夹中复制，并连接到🤗 Hub。", "credit": "所有功劳和版权归属于lllyasviel", "creditLink": "https://github.com/lllyasviel" } ], "banner": { "image": "https://raw.githubusercontent.com/Fannovel16/comfyui_controlnet_aux/main/examples/CNAuxBanner.jpg", "alt": "ComfyUI ControlNet Aux Banner", "caption": "\"动漫风格，街头抗议，赛博朋克城市，一位有着粉色头发和金色眼睛的女性（看着观众）举着一个标有'ComfyUI ControlNet Aux'的霓虹粉色粗体文字的标志\"" } }, "updates": { "id": "更新", "title": "ControlNet Aux 更新", "content": "前往更新页面查看ControlNet Aux最新更新。", "link": { "text": "更新页面", "url": "./UPDATES.md" } }, "installation": { "id": "安装", "title": "ControlNet Aux 安装指南", "methods": [ { "title": "使用ComfyUI Manager安装（推荐）", "description": "安装ComfyUI Manager并按照其中介绍的步骤安装此仓库。", "link": { "text": "ComfyUI Manager", "url": "https://github.com/ltdrdata/ComfyUI-Manager" } }, { "title": "替代方法", "description": "如果您在Linux上运行，或在Windows上使用非管理员账户，您需要确保/ComfyUI/custom_nodes和comfyui_controlnet_aux具有写入权限。", "note": "现在有一个install.bat文件，您可以运行它来安装到便携式版本（如果检测到）。否则，它将默认为系统安装，并假设您按照ComfyUI的手动安装步骤进行操作。", "fallback": { "title": "如果您无法运行install.bat（例如，您是Linux用户）", "description": "打开CMD/Shell并执行以下操作：", "steps": [ "导航到您的/ComfyUI/custom_nodes/文件夹", "运行git clone https://github.com/Fannovel16/comfyui_controlnet_aux/", "导航到您的comfyui_controlnet_aux文件夹", { "type": "code", "label": "便携式/venv：", "command": "path/to/ComfUI/python_embeded/python.exe -s -m pip install -r requirements.txt" }, { "type": "code", "label": "使用系统Python：", "command": "pip install -r requirements.txt" }, "启动ComfyUI" ] } } ] }, "nodes": { "id": "节点", "title": "ControlNet Aux 节点", "introduction": "请注意，此ControlNet Aux仓库仅支持制作提示图像的预处理器（例如棍棒人、Canny边缘等）。", "note": "除了Inpaint之外，所有ControlNet Aux预处理器都集成到AIO Aux Preprocessor节点中。此节点允许您快速获取预处理器，但无法设置预处理器自己的阈值参数。您需要直接使用其节点来设置阈值。", "categories": [ { "title": "线条提取器", "table": { "headers": ["预处理器节点", "sd-webui-controlnet/other", "ControlNet/T2I-Adapter"], "rows": [ ["Binary Lines", "binary", "control_scribble"], ["Canny Edge", "canny", "control_v11p_sd15_canny
control_canny
t2iadapter_canny"], ["HED Soft-Edge Lines", "hed", "control_v11p_sd15_softedge
control_hed"], ["Standard Lineart", "standard_lineart", "control_v11p_sd15_lineart"], ["Realistic Lineart", "lineart (or `lineart_coarse` if `coarse` is enabled)", "control_v11p_sd15_lineart"], ["Anime Lineart", "lineart_anime", "control_v11p_sd15s2_lineart_anime"], ["Manga Lineart", "lineart_anime_denoise", "control_v11p_sd15s2_lineart_anime"], ["M-LSD Lines", "mlsd", "control_v11p_sd15_mlsd
control_mlsd"], ["PiDiNet Soft-Edge Lines", "pidinet", "control_v11p_sd15_softedge
control_scribble"], ["Scribble Lines", "scribble", "control_v11p_sd15_scribble
control_scribble"], ["Scribble XDoG Lines", "scribble_xdog", "control_v11p_sd15_scribble
control_scribble"], ["Fake Scribble Lines", "scribble_hed", "control_v11p_sd15_scribble
control_scribble"], ["TEED Soft-Edge Lines", "teed", "controlnet-sd-xl-1.0-softedge-dexined
control_v11p_sd15_softedge (理论上)"], ["Scribble PiDiNet Lines", "scribble_pidinet", "control_v11p_sd15_scribble
control_scribble"], ["AnyLine Lineart", "", "mistoLine_fp16.safetensors
mistoLine_rank256
control_v11p_sd15s2_lineart_anime
control_v11p_sd15_lineart"] ] } }, { "title": "法线和深度估计器", "table": { "headers": ["预处理器节点", "sd-webui-controlnet/other", "ControlNet/T2I-Adapter"], "rows": [ ["MiDaS Depth Map", "(normal) depth", "control_v11f1p_sd15_depth
control_depth
t2iadapter_depth"], ["LeReS Depth Map", "depth_leres", "control_v11f1p_sd15_depth
control_depth
t2iadapter_depth"], ["Zoe Depth Map", "depth_zoe", "control_v11f1p_sd15_depth
control_depth
t2iadapter_depth"], ["MiDaS Normal Map", "normal_map", "control_normal"], ["BAE Normal Map", "normal_bae", "control_v11p_sd15_normalbae"], ["MeshGraphormer Hand Refiner", "depth_hand_refiner", "control_sd15_inpaint_depth_hand_fp16"], ["Depth Anything", "depth_anything", "Depth-Anything"], ["Zoe Depth Anything", "depth_anything", "Depth-Anything"], ["Normal DSINE", "", "control_normal/control_v11p_sd15_normalbae"], ["Metric3D Depth", "", "control_v11f1p_sd15_depth
control_depth
t2iadapter_depth"], ["Metric3D Normal", "", "control_v11p_sd15_normalbae"], ["Depth Anything V2", "", "Depth-Anything"] ] } }, { "title": "面部和姿势估计器", "table": { "headers": ["预处理器节点", "sd-webui-controlnet/other", "ControlNet/T2I-Adapter"], "rows": [ ["DWPose Estimator", "dw_openpose_full", "control_v11p_sd15_openpose
control_openpose
t2iadapter_openpose"], ["OpenPose Estimator", "openpose (detect_body)
openpose_hand (detect_body + detect_hand)
openpose_faceonly (detect_face)
openpose_full (detect_hand + detect_body + detect_face)", "control_v11p_sd15_openpose
control_openpose
t2iadapter_openpose"], ["MediaPipe Face Mesh", "mediapipe_face", "controlnet_sd21_laion_face_v2"], ["Animal Estimator", "animal_openpose", "control_sd15_animal_openpose_fp16"] ] } }, { "title": "光流估计器", "table": { "headers": ["预处理器节点", "sd-webui-controlnet/other", "ControlNet/T2I-Adapter"], "rows": [ ["Unimatch Optical Flow", "", "DragNUWA"] ] } }, { "title": "语义分割", "table": { "headers": ["预处理器节点", "sd-webui-controlnet/other", "ControlNet/T2I-Adapter"], "rows": [ ["OneFormer ADE20K Segmentor", "oneformer_ade20k", "control_v11p_sd15_seg"], ["OneFormer COCO Segmentor", "oneformer_coco", "control_v11p_sd15_seg"], ["UniFormer Segmentor", "segmentation", "control_sd15_seg
control_v11p_sd15_seg"] ] } }, { "title": "仅T2IAdapter", "table": { "headers": ["预处理器节点", "sd-webui-controlnet/other", "ControlNet/T2I-Adapter"], "rows": [ ["Color Pallete", "color", "t2iadapter_color"], ["Content Shuffle", "shuffle", "t2iadapter_style"] ] } }, { "title": "重新着色", "table": { "headers": ["预处理器节点", "sd-webui-controlnet/other", "ControlNet/T2I-Adapter"], "rows": [ ["Image Luminance", "recolor_luminance", "ioclab_sd15_recolor
sai_xl_recolor_256lora
bdsqlsz_controlllite_xl_recolor_luminance"], ["Image Intensity", "recolor_intensity", "可能与上面相同"] ] } } ], "openposeInfo": { "title": "如何获取OpenPose格式JSON？", "userSection": { "title": "用户端", "description": "此工作流将图像保存到ComfyUI的输出文件夹（与输出图像相同的位置）。如果您没有找到\"Save Pose Keypoints\"节点，请更新此扩展", "image": "https://raw.githubusercontent.com/Fannovel16/comfyui_controlnet_aux/main/examples/example_save_kps.png", "alt": "Save Pose Keypoints Example" }, "developerSection": { "title": "开发者端", "description": "对应于IMAGE批次中每一帧的OpenPose格式JSON数组可以从DWPose和OpenPose使用UI上的app.nodeOutputs或/history API端点获取。AnimalPose的JSON输出使用与OpenPose JSON类似的格式：", "jsonExample": { "code": "[\n {\n \"version\": \"ap10k\",\n \"animals\": [\n [[x1, y1, 1], [x2, y2, 1],..., [x17, y17, 1]],\n [[x1, y1, 1], [x2, y2, 1],..., [x17, y17, 1]],\n ...\n ],\n \"canvas_height\": 512,\n \"canvas_width\": 768\n },\n ...\n]" }, "codeExamples": [ { "title": "对于扩展开发者（例如Openpose编辑器）：", "language": "javascript", "code": "const poseNodes = app.graph._nodes.filter(node => [\"OpenposePreprocessor\", \"DWPreprocessor\", \"AnimalPosePreprocessor\"].includes(node.type))\nfor (const poseNode of poseNodes) {\n const openposeResults = JSON.parse(app.nodeOutputs[poseNode.id].openpose_json[0])\n console.log(openposeResults) //包含每帧Openpose JSON的数组\n}" }, { "title": "对于API用户：", "subtitle": "Javascript", "language": "javascript", "code": "import fetch from \"node-fetch\" //记得在\"package.json\"中添加\"type\": \"module\"\nasync function main() {\n const promptId = '792c1905-ecfe-41f4-8114-83e6a4a09a9f' //太懒了，不想POST /queue\n let history = await fetch(`http://127.0.0.1:8188/history/${promptId}`).then(re => re.json())\n history = history[promptId]\n const nodeOutputs = Object.values(history.outputs).filter(output => output.openpose_json)\n for (const nodeOutput of nodeOutputs) {\n const openposeResults = JSON.parse(nodeOutput.openpose_json[0])\n console.log(openposeResults) //包含每帧Openpose JSON的数组\n }\n}\nmain()" }, { "subtitle": "Python", "language": "python", "code": "import json, urllib.request\n\nserver_address = \"127.0.0.1:8188\"\nprompt_id = '' #太懒了，不想POST /queue\n\ndef get_history(prompt_id):\n with urllib.request.urlopen(\"http://{}/history/{}\".format(server_address, prompt_id)) as response:\n return json.loads(response.read())\n\nhistory = get_history(prompt_id)[prompt_id]\nfor o in history['outputs']:\n for node_id in history['outputs']:\n node_output = history['outputs'][node_id]\n if 'openpose_json' in node_output:\n print(json.loads(node_output['openpose_json'][0])) #包含每帧Openpose JSON的列表" } ] } } }, "examples": { "id": "示例", "title": "ControlNet Aux 示例", "subtitle": "ControlNet Aux 一图胜千言", "images": [ { "src": "https://raw.githubusercontent.com/Fannovel16/comfyui_controlnet_aux/main/examples/ExecuteAll1.jpg", "alt": "Example 1" }, { "src": "https://raw.githubusercontent.com/Fannovel16/comfyui_controlnet_aux/main/examples/ExecuteAll2.jpg", "alt": "Example 2" } ], "links": { "workflow": { "text": "测试工作流", "url": "https://github.com/Fannovel16/comfyui_controlnet_aux/blob/main/examples/ExecuteAll.png" }, "inputImage": { "text": "输入图像", "url": "https://github.com/Fannovel16/comfyui_controlnet_aux/blob/main/examples/comfyui-controlnet-aux-logo.png" } } }, "faq": { "id": "问答", "title": "ControlNet Aux 问答", "items": [ { "question": "为什么安装ControlNet Aux仓库后有些节点没有出现？", "answer": "此ControlNet Aux仓库有一个新机制，会跳过任何无法导入的自定义节点。如果您遇到这种情况，请在Issues选项卡上创建一个问题，并附上命令行中的日志。", "links": [ { "text": "Issues选项卡", "url": "https://github.com/Fannovel16/comfyui_controlnet_aux/issues" } ] }, { "question": "DWPose/AnimalPose只使用CPU所以很慢。如何让它使用GPU？", "answer": "有两种方法可以加速DWPose：使用TorchScript检查点（.torchscript.pt）或ONNXRuntime（.onnx）。TorchScript方式比ONNXRuntime稍慢，但不需要任何额外的库，仍然比CPU快得多。", "note": "TorchScript边界框检测器与onnx姿势估计器兼容，反之亦然。", "subsections": [ { "title": "TorchScript", "content": "根据此图设置bbox_detector和pose_estimator。如果输入图像理想，您可以尝试其他以.torchscript.pt结尾的边界框检测器来减少边界框检测时间。", "image": { "src": "https://raw.githubusercontent.com/Fannovel16/comfyui_controlnet_aux/main/examples/example_torchscript.png", "alt": "TorchScript Example" } }, { "title": "ONNXRuntime", "content": "如果成功安装了onnxruntime并且检查点使用以.onnx结尾，它将替换默认的cv2后端以利用GPU。请注意，如果您使用的是NVidia卡，此方法目前只能在CUDA 11.8（ComfyUI_windows_portable_nvidia_cu118_or_cpu.7z）上工作，除非您自己编译onnxruntime。", "steps": [ { "title": "了解您的onnxruntime构建：", "items": [ "NVidia CUDA 11.x或以下/AMD GPU：onnxruntime-gpu", "NVidia CUDA 12.x：onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/", "DirectML：onnxruntime-directml", "OpenVINO：onnxruntime-openvino" ], "note": "请注意，如果这是您第一次使用ComfyUI，请在执行下一步之前测试它是否可以在您的设备上运行。" }, "将其添加到requirements.txt", "运行install.bat或安装部分提到的pip命令" ], "image": { "src": "https://raw.githubusercontent.com/Fannovel16/comfyui_controlnet_aux/main/examples/example_onnx.png", "alt": "ONNX Example" } } ] } ] }, "resources": { "id": "ControlNet Aux预处理器资源文件", "title": "ControlNet Aux预处理器资源文件", "items": [ { "name": "anime_face_segment", "links": [ "https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite/blob/main/Annotators/UNet.pth", "https://huggingface.co/skytnt/anime-seg/blob/main/isnetis.ckpt" ] }, { "name": "densepose", "links": [ "https://huggingface.co/LayerNorm/DensePose-TorchScript-with-hint-image/blob/main/densepose_r50_fpn_dl.torchscript" ] }, { "name": "dwpose", "subsections": [ { "name": "bbox_detector", "description": "可以是", "links": [ "https://huggingface.co/yzd-v/DWPose/blob/main/yolox_l.onnx", "https://huggingface.co/hr16/yolox-onnx/blob/main/yolox_l.torchscript.pt", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_l_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_m_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_s_fp16.onnx" ] }, { "name": "pose_estimator", "description": "可以是", "links": [ "https://huggingface.co/hr16/DWPose-TorchScript-BatchSize5/blob/main/dw-ll_ucoco_384_bs5.torchscript.pt", "https://huggingface.co/yzd-v/DWPose/blob/main/dw-ll_ucoco_384.onnx" ] } ] }, { "name": "animal_pose (ap10k)", "subsections": [ { "name": "bbox_detector", "description": "可以是", "links": [ "https://huggingface.co/yzd-v/DWPose/blob/main/yolox_l.onnx", "https://huggingface.co/hr16/yolox-onnx/blob/main/yolox_l.torchscript.pt", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_l_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_m_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_s_fp16.onnx" ] }, { "name": "pose_estimator", "description": "可以是", "links": [ "https://huggingface.co/hr16/DWPose-TorchScript-BatchSize5/blob/main/rtmpose-m_ap10k_256_bs5.torchscript.pt", "https://huggingface.co/hr16/UnJIT-DWPose/blob/main/rtmpose-m_ap10k_256.onnx" ] } ] }, { "name": "face_yolox", "links": [ "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_l_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_m_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_s_fp16.onnx" ] }, { "name": "hand_yolox", "links": [ "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_l_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_m_fp16.onnx", "https://huggingface.co/hr16/yolo-nas-fp16/blob/main/yolo_nas_s_fp16.onnx" ] }, { "name": "hed", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/ControlNetHED.pth" ] }, { "name": "leres", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/res101.pth", "https://huggingface.co/lllyasviel/Annotators/blob/main/latest_net_G.pth" ] }, { "name": "lineart", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/sk_model.pth", "https://huggingface.co/lllyasviel/Annotators/blob/main/sk_model2.pth" ] }, { "name": "lineart_anime", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/netG.pth" ] }, { "name": "manga_line", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/erika.pth" ] }, { "name": "midas", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/dpt_hybrid-midas-501f0c75.pt" ] }, { "name": "mlsd", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/mlsd_large_512_fp32.pth" ] }, { "name": "normal_bae", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/scannet.pt" ] }, { "name": "oneformer", "subsections": [ { "name": "coco", "description": "", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/250_16_swin_l_oneformer_coco_100ep.pth" ] }, { "name": "ade20k", "description": "", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/150_16_swin_l_oneformer_ade20k_160k.pth" ] } ] }, { "name": "openpose", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/body_pose_model.pth", "https://huggingface.co/lllyasviel/Annotators/blob/main/hand_pose_model.pth", "https://huggingface.co/lllyasviel/Annotators/blob/main/facenet.pth" ] }, { "name": "pidinet", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/table5_pidinet.pth" ] }, { "name": "uniformer", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/upernet_global_small.pth" ] }, { "name": "zoe", "links": [ "https://huggingface.co/lllyasviel/Annotators/blob/main/ZoeD_M12_N.pt" ] } ], "note": { "text": "更多资源文件请参考", "link": { "text": "GitHub仓库", "url": "https://github.com/Fannovel16/comfyui_controlnet_aux" } } } }, "styles": { "theme": { "primaryColor": "#5E6AD2", "secondaryColor": "#E2E5F4", "backgroundLight": "#ffffff", "textLight": "#1f2937", "backgroundDark": "#111827", "textDark": "#f3f4f6" } } }