AI lessons 3

Previously, I trained LoRa in ComfyUI, but there was a specific workflow there with only one model for Flux1.
In this lesson I learned how to train LoRa in the Ostris program. In this program you can use Lora on various models, not only for generating graphics, but also for video and sound. And the massacre is being adjusted.

flux-2-klein-base-9b is the same model as I used to train the LoRa model, but the simpler version does not require training.
I trained the LoRa model twice, the first time I tried it, I reinstalled Windows 11 before updating.
Another test was Lora training on flux-2-klein-base-9b at 3000 krok, 512x512 and 1024x1024, which generates 130 images of 5 lines.
LoRa training was carried out 23 years ago, 20 years, without the use of Qwen models, analysis of products and preparation 20 images.
Interestingly, when generating images in Ostris on flux-2-klein-base-9b during LoRa training, the GPU temperature rose to 80-83°C, then as LoRa training continued, the GPU temperature dropped to 46-60°C.
flux-2-klein-base-9b is a very demanding model for both GPU size and RAM. I have an Nvidia RTX 3060 12 GB GPU, and it shouldn't have been used for LoRa training, as the requirements are twice as high. However, I managed to train the LoRa model, albeit very slowly.

I selected 20 images of Mr. Bean for training.
I understand that I don't have permission to train Mr. Bean's LoRa.
I don't use his images anywhere; this is a tutorial for LoRa training.

I trained LoRa for flux-2-klein-base-9b, the author of the tutorial trained the models for Z-Image.
My goal was to train LoRa, not to achieve a perfect likeness of the images to Mr. Bean. Some images have more similarities, others less.
It works better for some prompts, worse for others.
For an even better likeness of Mr. Bean's face, you can refine individual images and replace real faces of Mr. Bean in the generated images.
This flux-2-klein-base-9b model has issues with hands, fingers, and feet if there is a complex, detailed prompt and an unusual, challenging pose.
For example, there are photos of Mr. Bean where he holds his fingers in a gun-like position, with two fingers pointing forward, two fingers pressed together, and a thumbs-up gesture. This finger pose is difficult to generate; it results in one index finger pointing forward. It can sometimes work, but it often generates glitches.
I also encountered a glitch when I tried to generate the position of the hand holding the teddy bear by the neck from behind. It couldn't be generated. AI ​​on this model doesn't understand where the teddy bear's neck is.

In the video, the woman from the first image should move in the hall and say the phrase from the prompt at the end. But the video on gguf models is generated incorrectly. The full model does not generate on a computer with vram 12GB.

GLOBAL PROMPT
the character sheet woman is ApexJane

LOCAL PROMPT
ApexJane appear, walk in camera close-up shot of her shoe, and camera slowly pan up to her face. in medium shot. she says "Hi this is a test of character sheet.". camera dolly out to a wide shot.

I tested this workflow on a computer, it runs with gguf models on the lowvram parameter, but it generates the wrong video, the woman character moves correctly, but the animation is not generated according to the prompt.
I downloaded Itx-2.3-22b-distilled-1.1_transformer_only_fp8_scaled.safetensors on my computer there is not enough vram, ComfyUI stops working, it is a very complex workflow, loads the computer to the maximum. There are errors in the cloud, but all the models are there.
On the ltx-2.3-22b-distilled-Q5_K_M.guff model it generates the wrong video, in it the woman communicates incorrectly, several women appear, there is no image of the hall as in the First Frame image. Maybe this is because I do not download Itx-2.3-22b-distilled-1.1_transformer_only_fp8_scaled.safetensors, but use a simplified version of gguf.
I'm looking for a working workflow for Character Sheet for LTX2.3 with lip sync, correct character movements on prompt with multiple frames to generate clearly on prompt for vram 12 GB, or a working workflow in the cloud.

I studied and tested LTX Director on the LTX 2.3 model.
To generate video, you need at least a 16 GB video card and other computer parameters, and the normal requirements for a video card are about 32 Gb. These are very high requirements for a video card.
I tested the workflow on a simplified model ltx-2.3-22b-distilled-Q5_K_M.gguf on a computer, but it generates a lot of crashes, simplifies the graphics, and does not generate clearly according to the prompt. It can generate something correctly once, but in other parts of the prompts there are errors, then in subsequent generations the errors change according to the same prompt.
For example, I generated this Video of a man on a horse and a dragon on a simplified model on a computer, all the animation was generated correctly, there is action, the dragon flies, blows fire at the man, then the dragon flies away and the man on the horse moves to the right further. There are smooth transitions between the three frames in the animation, the video does not break between these frames. But then, when I tested this same prompt, errors constantly occurred both on the simplified model and on a full model in the cloud. In this video you can see that the sky changes slightly, because the sky on different frames was generated with different brightness. In other videos, the man on the horse approached and stood still for a long time, the dragon did not fly, there was little movement compared to this video.
Generated on runninghub.ai on the test account, the video on the full model ltx-2.3-22b-dev-fp8.safetensors, which was in the workflow, but even in the cloud generates a lot of errors. It is not clear why videos with errors are generated on the full model in the cloud. You can test the video generation on the website cloud.comfy.org, there are all the models and nodes for this workflow, but there is a paid subscription.