# AI Video (Shared · `Src\Video`)

**Context:** Shared capability (a small business module, not a portal section) · **Routes / UI:** standalone `manage/video` page + an embedded "Video Studio" tab in FLG Brochure Studio · **Feature flag:** `features.video_enabled` (`FEATURE_VIDEO_ENABLED`).

## What it does
Generates a short vertical ad **video** from one prompt and uploaded **materials**, using **BytePlus ModelArk Seedance 2.0** (the engine behind Dreamina). It is a **multimodal reference-to-video**: the prompt plus any mix of **images** (≤9, tagged `@Image1…@ImageN`), **reference videos** (≤3), **reference audio** (≤3) and **text** material; with no materials it's a plain text-to-video. The **mode is derived server-side** (any image/video/audio reference ⇒ reference-to-video, else text-to-video) — there is no user-facing mode toggle. Materials accepted: images, video, audio, plain-text/markdown files, and **PDF brochures** (poppler auto-extracts the photos, shown for selection; the text drives the suggested prompt + captions).

The same backend powers two surfaces: a **standalone "AI Video" page** (manual upload, general purpose) and a **"Video Studio" tab** inside FLG Brochure Studio that pre-seeds the brochure's extracted images so an agent turns a property brochure into a video ad. Generation is asynchronous (long-running) and runs on a dedicated **Horizon `video` queue**. The finished MP4 is persisted through the shared **Media** service.

## How it works
- **Reusable transport helper — `App\Helpers\SeedanceClient`** (mirrors `GeminiClient`): `buildPayload(array $references, string $prompt, array $opts)` takes ordered `['kind' => 'image'|'video'|'audio', 'url' => …]` references and emits the text part + one `image_url` / `video_url` / `audio_url` part per reference, each tagged role `reference_image` / `reference_video` / `reference_audio` (+ `ratio` / `resolution` / `duration` / `generate_audio:false`); plus `createTask()`, `getTask()`, `downloadVideoBytes()`, `dataUrl()`. Transport only — no DB, no poll loop. Empty reference list ⇒ text-to-video. Images ride as base64 data URLs; video/audio ride as public/signed http URLs (too large for base64).
- **Key model — `Src\Video\VideoGeneration`** (uuid + blame + soft delete): `status` (PENDING/PROCESSING/READY/FAILED), `mode` (IMAGE/TEXT), `prompt`, `ratio`, `duration_seconds`, `resolution`, `generate_audio`, `provider_task_id`, an optional `source` morph (e.g. an FLG `Brochure`), and `error`. Input images and the output video hang off it as **polymorphic Media** (`morphMany`), keyed by collection: input = `video_input` (ordered by `meta.position`, which drives `@ImageN`), output = `video_output`.
- **Write layer — `Src\Video\Repositories\VideoGenerationRepository`** (+ Facade): `create`, `setProviderTaskId`, `markProcessing`, `markReady`, `markFailed` — each inside `DB::transaction`, returning the refreshed model. The input/output Media rows are owned by `MediaService`/`MediaRepository`, not this repo.
- **Payload layer — `Src\Video\Services\VideoGenerationService`**: reads the generation's input Media (ordered, collection `video_input`) and routes each by MIME — **images** → base64 data URL (`imageReferences()`), **video/audio** → `MediaService::displayUrl()` signed URL (`extraReferences()`), **text** files → folded into the prompt (`effectivePrompt()`) — then assembles the create-task payload. `VideoGeneration::mediaKind($mime)` is the shared image/video/audio/text classifier; `inputImages()` (image-only, drives `@ImageN`) and `inputMedia()` (all materials) live on the model.
- **Async worker — `App\Jobs\Video\GenerateVideoJob`** (`onConnection('redis-video')->onQueue('video')`, `tries:1`, `timeout:1800`): `markProcessing` → `createTask` → `setProviderTaskId` → poll `getTask` (success string `succeeded`; `failed`/`cancelled` ⇒ throw; injectable `pollInterval`/`maxPollSeconds`) → `downloadVideoBytes` (the provider `video_url` expires ~24 h, so it downloads immediately) → `MediaService::store(... 'collection' => 'video_output' ...)` → `markReady`. Any `Throwable` ⇒ `markFailed($e->getMessage())`.
- **HTTP — `App\Http\Controllers\Manage\Video\VideoGenerationsController`** (`auth`+`admin`, gated by `features.video_enabled`): `index` (Inertia history, newest 50), `store` (derive mode → repo `create` → store input Media from uploaded materials + selected brochure/PDF images → dispatch `GenerateVideoJob` → `back()` with flash), `status` (JSON poll: status + signed/public `output_url` + error), `extract` (PDF → poppler images + Gemini prompt/captions). Validation lives in `StoreRequest` (ratio/resolution/duration enums; `images[]` ≤9 jpg/png/webp; `videos[]` ≤3 + `audios[]` ≤3 + `texts[]` ≤5 via `mimetypes`; optional `source_type=brochure` + `source_id`; combined image count ≤9). **No `mode` field** — the controller derives it from what's attached.
- **Frontend — `resources/js/Components/Video/VideoGenerator.vue`** is the one reusable generator: a prompt box, ratio/duration/resolution selectors, the ad-packaging panel, generate + `status`-poll + player/download, and an **"Upload materials"** tray — one uploader for images / video / audio / text / PDF, with image/video thumbnails, audio/text chips, the colour-coded `@ImageN` image tray, and PDF auto-extracted photos shown below for selection. `Pages/Manage/Video/Index.vue` wraps it bare + a "My Videos" history; the FLG `Projects/Show.vue` "Video Studio" tab wraps it pre-seeded with `brochure.images` and `source = { type:'brochure', id: brochure.id }`.

### Reference usage — Brochure Studio (first consumer)
On `/manage/flg/projects/{uuid}`, the "Video Studio" tab passes the brochure's extracted images (`brochure.images` → `{id,url,label}`) as `initial-images` and `{ type:'brochure', id: brochure.id }` as `source`. The controller resolves the brochure by **integer id** (`Brochure::find()` — `Brochure` has no `uuid` column), copies the selected extracted images into `video_input` Media, and links the generation via the `source` morph. The agent can then publish a video ad alongside the existing image creatives.

## Usage
```php
// Dispatch a generation (controller already does this; shown for reuse from any module)
$gen = VideoGenerationRepository::create(['video_generation' => [
    'mode' => VideoGeneration::MODE_IMAGE,
    'prompt' => 'Cinematic push-in on @Image1, golden hour',
    'ratio' => '9:16', 'duration_seconds' => 5, 'resolution' => '720p',
]]);
app(MediaService::class)->store($gen, $bytes, ['collection' => VideoGeneration::COLLECTION_INPUT, 'mime' => 'image/png', 'meta' => ['position' => 1]]);
GenerateVideoJob::dispatch($gen->id);
```

## Data model (`video_generations` table)
| Column | Type | Notes |
|--------|------|-------|
| `id` | `bigIncrements` | primary key / FK target |
| `uuid` | `uuid` unique | public identifier (`HasUuid`) |
| `status` | `unsignedInteger` | 1 pending · 2 processing · 3 ready · 4 failed (`STATUSES`) |
| `mode` | `unsignedInteger` | 1 image-to-video · 2 text-to-video (`MODES`) |
| `prompt` | `text` nullable | references `@Image1…@ImageN` |
| `ratio` | `string` | `9:16` / `16:9` / `1:1` |
| `duration_seconds` | `unsignedInteger` | 5 / 10 |
| `resolution` | `string` | `720p` / `1080p` |
| `generate_audio` | `boolean` | default false |
| `provider_task_id` | `string` nullable, indexed | Seedance task id |
| `source_type` / `source_id` | `nullableMorphs` | optional owner (e.g. `Brochure`); indexed, no FK |
| `error` | `text` nullable | failure reason |
| `created_by` / `updated_by` / `deleted_by` | `unsignedInteger` nullable | blame (`RecordsBlame`) |
| `timestamps`, `softDeletes` | | |

Input images and the output video are **not** columns — they are `Media` rows (`collection` = `video_input` / `video_output`) attached via the `mediable` morph (see the [Media](/docs/modules_handbook/shared/Media/readMe.md) handbook).

## Configuration
`config/services.php` → `seedance`:

| Key | Env | Default |
|-----|-----|---------|
| `base_url` | `SEEDANCE_BASE_URL` | `https://ark.ap-southeast.bytepluses.com/api/v3` |
| `api_key` | `SEEDANCE_API_KEY` | — (required; never commit) |
| `model` | `SEEDANCE_MODEL` | `dreamina-seedance-2-0-260128` |
| `resolution` | `SEEDANCE_RESOLUTION` | `720p` |

Other env: `FEATURE_VIDEO_ENABLED` gates routes + nav; `MEDIA_DISK` selects where the video is stored (`gcs` in prod, `public`/`local` for dev without GCS creds).

**Queue (long-running — required gotcha).** `config/queue.php` defines a `redis-video` connection with **`retry_after` (1900) GREATER than the job `timeout` (1800)** — otherwise Horizon re-dispatches the job mid-generation and you get duplicate/clobbered work. `config/horizon.php` defines a `supervisor-video` (connection `redis-video`, queue `video`). Run with `php artisan horizon` (Redis required). Locally without Redis the generation still works under `QUEUE_DRIVER=sync` (the job runs inline).

## Storyboard editor + voiceover

On top of the single-prompt generator, a video can be built from an **AI-drafted, editable storyboard** with per-scene **voiceover**. Scenes live in the `video_scenes` child table (`Src\Video\VideoScene`, uuid + blame + softDeletes), ordered by a contiguous `position` (1..N over non-deleted rows).

**Two-phase flow** (all under `features.video_enabled`):
1. **Draft** — `POST manage.video.storyboard.draft`: the controller creates a `STATUS_DRAFT` generation, stores the uploaded images/clips into the `video_input` Media collection, then `StoryboardDraftService` (one Gemini call) drafts N scenes `{type, input_media_id, prompt(EN), caption(CN), voiceover}`; `VideoSceneRepository::createMany` persists them. No job is dispatched.
2. **Edit** — `PUT manage.video.storyboard/{id}`: `VideoSceneRepository::sync` upserts by uuid, soft-deletes omitted scenes, and renumbers. Only a `STATUS_DRAFT` generation is editable.
3. **Generate** — `POST manage.video/{id}/generate`: `VideoGenerationRepository::markPendingForGeneration($id)` re-queries the row with `lockForUpdate()` **inside** the transaction and flips DRAFT→PENDING; the controller dispatches `GenerateVideoJob` **only on a non-null return**, so a double-submit cannot dispatch twice.

**Scene types.** `GENERATED` = image + prompt → Seedance; `CLIP` = the user's own uploaded video used directly (no Seedance, **original audio dropped**). `input_media_id` is an image for GENERATED, a video for CLIP.

**Job (scene-driven branch).** `GenerateVideoJob` branches on `$gen->hasScenes()`. Each scene clip is forced to its exact `duration_seconds` by `ClipNormalizer` (so planned durations == rendered timeline — constraint), then `FfmpegStitcher::crossfade(..., $durations)` stitches using those **canonical durations** (no ffprobe). `VoiceoverService` synthesises each scene's narration via `GeminiTtsClient` (config `services.gemini.tts_model`; voices in `config/video.php`), trims it to its window (atrim only — no atempo), delays it to the window start, and mixes one VO track; `VideoPackager::package(..., $voiceoverPath)` overlays scene captions and muxes VO over **ducked** music. Generations with no scenes keep the legacy auto-split path unchanged.

### Reference usage — one timeline source (`SceneTimeline`)
Captions and voiceover **must both** derive their per-scene `{start,end}` windows from `SceneTimeline::windows($durations, VideoGeneration::XFADE_SECONDS)` — never compute timing independently (xfade overlap means total ≠ sum of durations). The job calls it once and passes the windows to both the caption builder and `VoiceoverService`.

## Related files
**Backend**
- [app/Helpers/SeedanceClient.php](/app/Helpers/SeedanceClient.php) — reusable Seedance transport helper.
- [app/Helpers/GeminiTtsClient.php](/app/Helpers/GeminiTtsClient.php) — TTS transport (base64 PCM → WAV).
- [app/Helpers/ClipNormalizer.php](/app/Helpers/ClipNormalizer.php) — force each clip to exact duration (timeline closure).
- [src/Video/VideoScene.php](/src/Video/VideoScene.php) — storyboard scene model (GENERATED / CLIP).
- [src/Video/Repositories/VideoSceneRepository.php](/src/Video/Repositories/VideoSceneRepository.php) + [Facade](/src/Video/Facades/VideoSceneRepository.php) — createMany / sync / per-scene ops (renumber).
- [src/Video/Services/SceneTimeline.php](/src/Video/Services/SceneTimeline.php) — single timeline source · [StoryboardDraftService.php](/src/Video/Services/StoryboardDraftService.php) · [VoiceoverService.php](/src/Video/Services/VoiceoverService.php).
- Storyboard routes: `manage.video.storyboard.*` + `manage.video.generations.generate` in [routes/web.php](/routes/web.php); requests in [app/Http/Requests/Manage/Video/Storyboard/](/app/Http/Requests/Manage/Video/Storyboard/).
- [src/Video/VideoGeneration.php](/src/Video/VideoGeneration.php) — key model (status/mode constants, Media accessors).
- [src/Video/Repositories/VideoGenerationRepository.php](/src/Video/Repositories/VideoGenerationRepository.php) + [Facade](/src/Video/Facades/VideoGenerationRepository.php) — transactional writes.
- [src/Video/Services/VideoGenerationService.php](/src/Video/Services/VideoGenerationService.php) — payload builder.
- [app/Jobs/Video/GenerateVideoJob.php](/app/Jobs/Video/GenerateVideoJob.php) — Horizon `video`-queue worker.
- [app/Http/Controllers/Manage/Video/VideoGenerationsController.php](/app/Http/Controllers/Manage/Video/VideoGenerationsController.php) + [StoreRequest](/app/Http/Requests/Manage/Video/Generations/StoreRequest.php).
- Routes: `manage.video.generations.*` in [routes/web.php](/routes/web.php) (under `features.video_enabled`).

**Frontend**
- [resources/js/Components/Video/VideoGenerator.vue](/resources/js/Components/Video/VideoGenerator.vue) — reusable generator.
- [resources/js/Components/Video/StoryboardStudio.vue](/resources/js/Components/Video/StoryboardStudio.vue) — draft + edit + generate storyboard · [SceneCard.vue](/resources/js/Components/Video/SceneCard.vue) — one editable scene.
- [resources/js/Pages/Manage/Video/Index.vue](/resources/js/Pages/Manage/Video/Index.vue) — standalone page + history + storyboard studio.
- FLG [Projects/Show.vue](/resources/js/Pages/Manage/FacebookLeadGenerator/Projects/Show.vue) — "Video Studio" tab.
- Nav: [ManageLayout.vue](/resources/js/Layouts/ManageLayout.vue); feature prop: [HandleInertiaRequests.php](/app/Http/Middleware/HandleInertiaRequests.php).

**Migration**
- [database/migrations/2026_06_20_100001_create_video_generations_table.php](/database/migrations/2026_06_20_100001_create_video_generations_table.php).
- [database/migrations/2026_06_23_100001_create_video_scenes_table.php](/database/migrations/2026_06_23_100001_create_video_scenes_table.php) — storyboard scenes.

**See also:** [Media](/docs/modules_handbook/shared/Media/readMe.md) (output/input storage) · the design spec `docs/video-studio-design.md` and plan `docs/video-studio-plan.md`.