ComfyUI Tutorial Series: Ep11 - LLM, Prompt Generation, img2txt, txt2txt Overview

In this episode, we focus on prompt generation using Large Language Models (LLMs) in ComfyUI. You’ll learn how to create prompts from both text and images, and how to integrate these prompts into your workflows. We’ll also explore tools like Florence 2 for image captioning and cover the steps for installing custom nodes, updating ComfyUI, and managing dependencies for a smooth experience. Key topics include: - Installing and using Florence 2 (Vision-Language model) - Generating detailed prompts from images (img2txt) - Using LLMs for text-based prompts (txt2txt) Unlock exclusive perks by joining our channel: https://www.youtube.com/channel/UCmMbwA-s3GZDKVzGZ-kPwaQ/join Download all the workflows from Discord https://discord.gg/gggpkVgBf3 look for the channel pixaroma-workflows ----- Install Florence 2 in ComfyUI Step 1 The custom node: Go to Manager then Custom Nodes Manager Search for florence Install the node with id 269 named ComfyUI-Florence2 by kijai Restart Step 2 The requirements: I assume you are using ComfyUI Portable, so you need to install the required packages. Go to your ComfyUI_windows_portable folder, click on the address bar, type cmd, and press Enter. Then, paste the following text and press Enter. python_embeded\python.exe -m pip install -r ComfyUI\custom_nodes\ComfyUI-Florence2\requirements.txt You can close the window. Then, restart ComfyUI by clicking on 'Manager' and selecting 'Restart If you get an error like this in the command prompt, don't worry, it will be fixed when we run the workflow for the first time, as it will download the model into that LLM folder. FileNotFoundError: [WinError 3] The system cannot find the path specified: '..\ComfyUI_windows_portable\ComfyUI\models\LLM' ----- Instal custom nodes Search for searge Install ID 97 Searge-LLM for ComfyUI v1.0 Restart comfyui Create a new folder called llm_gguf in the ComfyUI/models directory. Download the file Mistral-7B-Instruct-v0.3.Q4_K_M.gguf (4.37 GB). from the repository MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF on HuggingFace. https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf place Mistral-7B-Instruct-v0.3.Q4_K_M.gguf in the ComfyUI/models/llm_gguf directory. refresh comfyui More models can be found here: https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/tree/main If you have problem installing check https://github.com/SeargeDP/ComfyUI_Searge_LLM?tab=readme-ov-file#potential-problems #comfyui #llm #fluxai ----- Chapters: 00:00 Introduction 00:20 Generate a Prompt from an Image 17:02 Using LLMs to generate prompts 26:16 ChatGPT Formula for Flux Prompts