easyanimate model/pipeline review

Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423

Review performed against the repository review rules.

Duplicate search: checked GitHub Issues/PRs for EasyAnimate, affected class/function names, and the specific failure modes. No exact duplicates found. Related: #12646 reports another crash in the same inpaint repaint branch; #13347 only refactors transformer tests.

Local test note: .venv was used. Focused Python reproductions ran; full non-slow pytest collection is blocked in this .venv by ModuleNotFoundError: torch._C._distributed_c10d.

Issue 1: EasyAnimateControlPipeline.__call__ always passes an invalid encode_prompt kwarg

Affected code:

Problem:
EasyAnimateControlPipeline.encode_prompt() does not accept text_encoder_index, but __call__ passes text_encoder_index=0. Any control pipeline call reaches this TypeError before denoising.

Impact:
EasyAnimateControlPipeline is effectively unusable through its public __call__.

Reproduction:

from diffusers import EasyAnimateControlPipeline

pipe = object.__new__(EasyAnimateControlPipeline)
EasyAnimateControlPipeline.encode_prompt(pipe, prompt="x", text_encoder_index=0)

Relevant precedent:
Base and inpaint EasyAnimate call encode_prompt without this kwarg:

Suggested fix:

# Remove the stale kwarg from EasyAnimateControlPipeline.__call__
negative_prompt_attention_mask=negative_prompt_attention_mask,

Issue 2: EasyAnimateControlPipeline decodes through an undefined method

Affected code:

Problem:
The control pipeline calls self.decode_latents(latents), but the class does not define decode_latents.

Impact:
After fixing Issue 1, any control run with output_type != "latent" will fail at decode time.

Reproduction:

from diffusers import EasyAnimateControlPipeline

pipe = object.__new__(EasyAnimateControlPipeline)
print(hasattr(pipe, "decode_latents"))
pipe.decode_latents(None)

Relevant precedent:

Suggested fix:

latents = 1 / self.vae.config.scaling_factor * latents
video = self.vae.decode(latents, return_dict=False)[0]
video = self.video_processor.postprocess_video(video=video, output_type=output_type)

Issue 3: Control helper optional mask/reference paths are broken

Affected code:

Problem:
get_video_to_video_latent() calls preprocess_image(..., size=sample_size), but the helper parameter is named sample_size. The ref_image branch also builds the wrong rank for the pipeline, which later expects (B, C, F, H, W).

Impact:
Documented/public helper paths for control masks and reference images fail before pipeline execution.

Reproduction:

from PIL import Image
from diffusers.pipelines.easyanimate.pipeline_easyanimate_control import get_video_to_video_latent

frame = Image.new("RGB", (8, 8), "white")
mask = Image.new("RGB", (8, 8), "black")
ref = Image.new("RGB", (8, 8), "blue")

for kwargs in ({"validation_video_mask": mask}, {"ref_image": ref}):
    try:
        get_video_to_video_latent([frame], 1, (8, 8), **kwargs)
    except Exception as e:
        print(type(e).__name__, e)

Relevant precedent:
The main video path already uses sample_size=sample_size at line 123.

Suggested fix:

validation_video_mask = preprocess_image(validation_video_mask, sample_size=sample_size)[:1]
input_video_mask = torch.where(validation_video_mask < 240 / 255.0, 0.0, 255.0)
input_video_mask = input_video_mask.unsqueeze(0).unsqueeze(2).repeat(1, 1, input_video.shape[2], 1, 1)

ref_image = preprocess_image(ref_image, sample_size=sample_size)
ref_image = ref_image.unsqueeze(1).unsqueeze(0)

Issue 4: Inpaint helper fails for multiple end frames

Affected code:

Problem:
For validation_image_end as a list, the code slices with len(end_video), which is the batch dimension (1), not the number of end frames.

Impact:
Multi-frame end conditioning raises a shape error and cannot prepare inputs.

Reproduction:

from PIL import Image
from diffusers.pipelines.easyanimate.pipeline_easyanimate_inpaint import get_image_to_video_latent

start = Image.new("RGB", (8, 8), "white")
ends = [Image.new("RGB", (8, 8), "black"), Image.new("RGB", (8, 8), "blue")]
get_image_to_video_latent(start, ends, 4, (8, 8))

Relevant precedent:
The mask branch already uses len(image_end) on the next line.

Suggested fix:

input_video[:, :, -len(image_end) :] = end_video
input_video_mask[:, :, -len(image_end) :] = 0

Issue 5: Inpaint FlowMatch repaint branch has a malformed torch.tensor call

Affected code:

Problem:
torch.tensor([noise_timestep], noise) passes noise as a second positional argument to torch.tensor, which is invalid. It also fails to pass noise to scale_noise.

Impact:
The repaint branch for FlowMatchEulerDiscreteScheduler crashes when num_channels_transformer == num_channels_latents.

Reproduction:

import torch
from diffusers import FlowMatchEulerDiscreteScheduler

scheduler = FlowMatchEulerDiscreteScheduler()
scheduler.set_timesteps(2)

sample = torch.zeros(1, 4, 1, 2, 2)
noise = torch.ones_like(sample)
noise_timestep = scheduler.timesteps[1]

scheduler.scale_noise(sample, torch.tensor([noise_timestep], noise))

Relevant precedent:

Suggested fix:

init_latents_proper = self.scheduler.scale_noise(
    init_latents_proper, torch.tensor([noise_timestep], device=device), noise
)

Issue 6: EasyAnimate attention ignores Diffusers attention backends

Affected code:

Problem:
EasyAnimateAttnProcessor2_0 calls F.scaled_dot_product_attention directly and does not define _attention_backend / _parallel_config. model.set_attention_backend(...) silently skips the processor.

Impact:
Users cannot select supported attention backends for EasyAnimate, and context-parallel/backend plumbing cannot affect this model.

Reproduction:

from diffusers import EasyAnimateTransformer3DModel

model = EasyAnimateTransformer3DModel(
    num_attention_heads=2,
    attention_head_dim=16,
    in_channels=4,
    out_channels=4,
    time_embed_dim=8,
    text_embed_dim=16,
    num_layers=1,
    mmdit_layers=1,
    patch_size=2,
)

processor = model.transformer_blocks[0].attn1.processor
print(hasattr(processor, "_attention_backend"))
model.set_attention_backend("native")
print(getattr(processor, "_attention_backend", None))

Relevant precedent:

Suggested fix:

from ..attention_dispatch import dispatch_attention_fn

class EasyAnimateAttnProcessor2_0:
    _attention_backend = None
    _parallel_config = None

    ...
    hidden_states = dispatch_attention_fn(
        query,
        key,
        value,
        attn_mask=attention_mask,
        dropout_p=0.0,
        is_causal=False,
        backend=self._attention_backend,
        parallel_config=self._parallel_config,
    )

Issue 7: Control and inpaint variants have no fast or slow pipeline coverage

Affected code:

Problem:
The test file only covers EasyAnimatePipeline; there are no fast tests or slow/integration tests for EasyAnimateControlPipeline, EasyAnimateInpaintPipeline, or their public helper functions. Docs also only autodoc the base pipeline.

Impact:
The concrete control/inpaint regressions above are not caught by CI, and public variants are less discoverable.

Reproduction:

from pathlib import Path

tests = Path("tests/pipelines/easyanimate/test_easyanimate.py").read_text()
docs = Path("docs/source/en/api/pipelines/easyanimate.md").read_text()

print("EasyAnimateControlPipeline" in tests)
print("EasyAnimateInpaintPipeline" in tests)
print("[[autodoc]] EasyAnimateControlPipeline" in docs)
print("[[autodoc]] EasyAnimateInpaintPipeline" in docs)

Relevant precedent:
Existing base pipeline fast and slow tests are in the same file and can be extended with tiny control/inpaint fixtures.

Suggested fix:
Add focused fast tests for control and inpaint using tiny components, including output_type="pt" and "latent" paths plus helper utility tests. Add slow tests for official control and inpaint checkpoints, or explicitly mark/document why they cannot be run. Update the EasyAnimate docs to autodoc EasyAnimateControlPipeline and EasyAnimateInpaintPipeline.