AI lessons 2

Below are the videos I generated on different models during training.
During the training, I tested video generation on different sizes and with different prompts. There were some errors that I had to correct. To show the entire process of generation failures and various bugs, I'm also posting videos with defects, not just the final successful results. I haven't included the most obvious defects. Errors can occur during video generation. Without showing them, it might seem like it takes too little time to generate the desired video animation. Errors can vary: incompletely designed prompts, high computer specifications, missing nodes in the workflow, or incompatible ComfyUI extensions. For example, a workflow might fail to launch in Cloud due to some glitches, some nodes in the nodes might not connect for some reason even though they should, not all the workflow nodes might be installed on the server, some required models might be missing, there might be a generation time limit on the test server, or the server's graphics card might be limited. Having examples with relevant videos and intermediate results, as well as screenshots of the workflow with models and prompts, helps me quickly navigate potential glitches when returning to the topic.

Prompt:
[VISUAL]: Opens with a medium close-up, camera slowly pushes in toward the character. An elderly man with deep wrinkles, piercing blue eyes, and a long thick white beard, wearing a worn gray cap and dark rugged clothing, looks directly into the camera. His expression is calm and thoughtful, his mouth opens and closes naturally as he speaks, with subtle head movements and slight eyebrow expressions. Soft natural lighting highlights the texture of his skin and beard. The background is slightly blurred, suggesting an outdoor coastal or rural environment with muted earthy tones.
[SPEECH]: I’ve seen many places in my life… but somehow, every new place still feels like the first time. Maybe next… a quiet beach, warm sun, and the sound of waves… yes, that would be nice.
[SOUNDS]: calm elderly British male voice, slightly rough and deep tone, natural outdoor ambience with soft wind and distant ocean waves, low background noise

Prompt:
[VISUAL]: Opens with a close-up shot, camera gently pushes in toward the character. A small, fluffy white bunny with soft fur, big expressive brown eyes, and pink inner ears sits upright on a wooden surface. Warm golden sunlight streams through a nearby window, casting soft highlights on its fur and creating a cozy indoor atmosphere. The bunny faces the camera with an exaggerated angry expression-its eyes slightly narrowed, brows subtly furrowed, and tiny nose twitching with attitude-while still retaining a cute, humorous look. Its ears make small reactive movements, and its mouth opens and closes naturally as it speaks. The bunny's tiny paws are held close to its chest, occasionally tensing slightly to emphasize the "threat." The background shows a softly blurred rustic kitchen with glass jars and warm tones, enhancing the contrast between the cozy setting and the bunny's mock-serious mood.
[SPEECH]: This is your final warning... give me snacks now... or I start chewing everything. Starting with... your favorite things..
[SOUNDS]: cute high-pitched animated bunny voice, soft and playful tone with a slightly cheeky attitude, cozy indoor ambience with light room tone, faint wooden creaks and subtle environmental warmth, no harsh background noise

In this tutorial, I learned how to generate a video using three frames on LTX 2.3.
I generated a 1280x704 video in Cloud for the tutorial, lasting 15 seconds.
The video adds sounds of moving objects that properly match the actions in the video, as well as a melody. I couldn't remove the melody; it's different every time depending on the generation number, and sometimes there might be no melody. I also couldn't upload my own background music for the video. The sounds of the girl walking and the door opening are correct.
The tutorial author has a prompt for 15 seconds of video generation using three frames:

She turns her head and gets up from her chair.
The girl leaves the room. She walks to the door, reaches out, and opens it in a smooth motion. She continues forward, into the outside environment with a natural transition of lighting.
She walks to a parked superbike and stops next to it, holding her helmet. Continuous frame-by-frame motion, no cuts, no abrupt transitions, smooth temporal coherence, stable identity, consistent clothing, natural pace, realistic movement, 24 frames/s.

But this prompt often generates various errors that the AI ​​doesn't understand, and it generates them randomly, which can make it unsuitable. I decided to create the following scenario for the video:

The girl stood up, pushed back her chair, and walked out of the room to the right (from the video perspective).
Next angle: The door was opened by the girl from the inside. The girl exited, closed the door, and walked toward the motorcycle.
The camera follows the girl's movement to the motorcycle. The girl approaches the motorcycle, turns to the camera, grabs her helmet from the front of the motorcycle with both hands, and moves the helmet to the side, holding it with one hand.
From the moment she leaves the house, the camera shouldn't change angles, but follow the girl the entire time.

But many errors occur in the generations.
She exits incorrectly, gets up from her chair incorrectly, turns around incorrectly, appears incorrectly near the door, approaches the door from the other side, slips through the door, crouches, and crawls to the door, walks away from the motorcycle, two girls appear, one approaches the door, the other exits, or one leaves, the other exits at the same time.
The camera doesn't follow the girl as she walks toward the motorcycle. She incorrectly grabs her helmet and abruptly changes her pose from the rear to the front, and the helmet appears out of nowhere. There were glitches where the girl turns around and doesn't get up from her chair, looking in different directions, and then gets up or leaves the chair, walks back, and a wall appears. The girl walks left, in the other direction, takes a long time to exit the house, and walls begin to appear in the room, a new interior that wasn't in the prompt. The girl tried to walk toward the wall, where there was no door. When she got up from the chair, she made an incorrect turn and walked in the wrong direction. There were also some glitches with the helmet at first. It would randomly appear in the girl's hand when she walked away from the door, or it would appear in her hand when she was close to the motorcycle. Sometimes the girl's pants would change to a skirt. A watch would appear on the girl's hand. When I was generating the video, I didn't notice this. In the second frame, the girl isn't wearing a watch, and in the third, a steel bracelet is visible on her hand; it needed to be removed. The AI ​​sometimes tries to add it, sometimes remove it. The errors in the video are inconsistent; sometimes it might generate correctly, other times there might be glitches. But when I clearly described how the girl approaches the motorcycle, how the camera follows her, how she turns around, how she picks up the helmet, and how she moves it in the right direction, there were no glitches at the end of the video after all these explanations.

All the movements are physically correct, the girl doesn't have any extra body parts, and there are no other graphical defects.
But in this workflow, the girl's eyes are deliberately covered with black glasses to avoid any defects.
When generating various versions, I tested and changed the prompts, but there were a lot of errors in the generations. They depend on the complexity of the images and the quality of the animation prompt. The main error is that the girl exits the door incorrectly, approaches the house from the exit, tries to go through the door, and then begins to exit the house.

Highly dependent on a video card, you need about 24 GB to get the workflow on your computer.
The ability to generate different sizes:
3840 x 2176    1920 x 1088
2560 x 1408    1280 x 704
2048 x 1152    1024 x 576
1920 x 1088    960 x 544
1280 x 704    640 x 352
1280 x 736    640 x 368
960 x 544    480 x 272
768 x 432
384 x 216

These dimensions show that there's only one horizontal video format with different sizes of the same aspect ratio. For the 1280p size, there are two options.
I haven't tested other sizes.
The second frame can appear at different points in the video; I haven't thoroughly tested how to specify exactly what second it should appear at. Depending on the amount of text in the prompt, the second frame may shift in time. For example, if there's no detailed description of the girl leaving the house at the beginning, the second frame appears sooner.
If I add a lot of clarification in the prompt before the girl leaves the house, or if the animation glitches before the second frame, the girl will start running at the end of the animation because there wasn't enough time to complete all the actions in the prompt, and the video length is fixed at 15 seconds.

In this tutorial, I learned how to generate a two-frame video on LTX 2.3 using the workflow from the tutorial.
I took two low-quality screenshots from the end of the video, one of an ant and one of an ant with a person, generated high-quality images from these images, and first tested creating a two-frame animation.
The animation in this tutorial was terrible, with a lot of glitches.
For this two-frame workflow, I generated a 4-second animation.
My idea is for the person to run up to the ant from behind and jump on it. But in this workflow, the person's face constantly changes, objects change, and the person runs incorrectly; instead of running from behind toward the camera, the person runs up from the front.
Different people appear that weren't in the prompt.
I also tested the animation of the girl, the shots of her leaving the house and walking toward the motorcycle, using two frames from this tutorial. All the animations glitch, the last seconds are generated incorrectly, and the graphics are of poor quality.

Then I generated a third frame, in which the ant and the man are supposed to run, and the man points forward with his staff. I also generated additional legs for the ant, since the tutorial had four, but six are needed for proper animation.
I tested this animation using three frames using the workflow from the previous tutorial.
I created the following animation script for three frames for 10 seconds:

The ant is standing, a man runs up behind it and jumps on it, enemies run after him, the ant starts to run away, and the man points forward with his staff and holds onto the ant, sitting on it.

The animation from the three frames in the previous lesson was of higher quality, but there were still a lot of glitches.
The face and clothing change if you don't describe it all in great detail in the prompt, if you don't specify which enemies are attacking the man, then different people are generated. I specified in the prompt that the enemies should catch up with the man, not overtake him, that they were simply running after him, but the man ran away from them.
But often, different people overtook the ant and ran past. Then I specified in the prompt that wasps were catching up with the man. I tried specifying wasps, beetles, and grasshoppers, but in that case, only one would catch up. Either grasshoppers, or beetles, or wasps would fly. The wasps don't chase the man on the ant, they simply flew around him. When I wrote that the wasps were attacking, attacking, and stinging the man, the wasps were flying and bumping into each other.
The wasps rarely looked realistic; sometimes they looked like robots or drones.
Sound and melody are added to the video. Sounds like the men jumping and the wasps buzzing work well, but the melodies aren't always successful. Sometimes there's no melody at all, just sounds, but sometimes melodies appear, just like in the previous lesson I used to test these animations.

The video opens on a medium close-up of the natural, freckled woman from Image 1, holding a crease blending brush to her right eyelid with a neutral expression. Over the first few seconds, she begins gentle blending strokes, looking into the lens and saying, "Okay, let's build some dimension. Just blending out this shadow..." After four seconds, she sets down the crease brush and picks up an eyebrow pencil, carefully defining her arches while her eyebrows begin a dramatic, visible morph into the intensely dark and sculpted, bold shape seen in Image 2. As she works, she enthusiastically remarks, "Alright, strong brows today! I want them defined and strong today!" By the 9-second mark, with the eyebrows almost complete, she reaches with her free hand and quickly, delicately attaches the small stud earrings from Image 2, commenting, "And just a little touch for the ears... perfect." At 12 seconds, she transitions focus, setting down the brow pencil and picking up the bullet of vibrant red lipstick (Image 2 color), her base skin texture flawless-ing out to a matte finish in real-time as she says, "Okay, looking good. Base is smooth. Now, for the star of the show!" For the next five seconds, her precision application of the red lipstick is highlighted, as she continues, "Precision is everything with a bold red! Cupid's bow... loving this color!" By 20 seconds, the red lips are established, and she performs a final 'check' in the lens as her lashes visibly lift and volumize; she smiles warmly and strikes the final, beaming, confident pose from Image 2, showing a beautiful teeth-showing smile, as she concludes, "A final check, and yes! Loving the intensified look. It's time to conquer the day!" The entire sequence features fluid, realistic hand movements with perfect lip-synced American English, set against a soft studio backdrop with an upbeat, modern lo-fi track that reaches a jubilant peak with her final smile.