Dev docs

Every model, setting and config used to make AdLoop's content, on the code side and on the GPU PC ("the GPU factory"). Values are the ones in the code as of 19 Oct 2026. The live panel at the top of the Dev docs page shows what is running right now.

Status labels used below:

• In product: wired into AdLoop and used from the UI.
• Prototype: a working script in gpu_server/prototypes/, used for the first test ad, not yet wired into the UI.
• Rejected / not used: tried and dropped, kept here so nobody repeats it.

1. Overview

Mac or PC browser ─► Website (Next.js, :3000) ─► AdLoop backend (FastAPI, :8000)
                                                   │  LLM: qwen3-30b (scripts, briefs, storyboards)
                                                   └► GPU server (FastAPI, :8500) ─► ComfyUI (:8188)
                                                        one GPU job at a time        SDXL, Wan 2.1, InfiniteTalk

2. Text: LLM copywriter

What it writes:

Rules in the prompts:

• Use only facts the business gave.
• Never invent prices, distances, amenities, awards or counts.
• Never say "guaranteed", "100%", "cure" or "risk-free".
• Always end with a call to action.
• Storyboards are one short film: the same cast, place and look in every shot, with people mostly in wide or medium shots, not posing. Pictures contain no text or logos.

Checks after the LLM:

• Compliance (compliance.py) flags risky claims and numbers that aren't in the business details. For example, "24/7 security" was flagged as "7, 24".
• Scene lines must add up to the script word for word, compared on letters and digits only, because the voice-over is cut by them. On a mismatch the writer gets one retry with the problem spelled out. If it still fails, the script is split at sentence ends and the model's pictures are kept by position (studio._write_scenes).

The creative brief in prompts (src/ad_engine/creative.py): every chosen option becomes an English phrase, e.g. modern_casual → "modern smart-casual clothing". Options left on "AI decides" are described as "your choice". When clothing is on AI decides, the prompt adds "do not default to traditional dress". Unknown answers are dropped (normalise). The question list is mirrored in frontend/lib/creative.ts, and tests/test_studio.py fails if the two drift.

Known quality issues (Telugu, qwen3-30b):

• Draft-grade language: stutters ("nenu nenu nenu"), Hindi words mixed in ("ho jayegi"), spelling slips.
• Invented non-numeric details.
• A stronger model plus a native-speaker review are needed before client work.

3. Pictures: SDXL storyboard (in product)

Prompt = scene image_prompt + variant look + this fixed style:

cinematic film still, shot on 35mm, natural colour grading, shallow depth of field, photorealistic, highly detailed, vertical composition

Negative prompt:

cartoon, illustration, 3d render, anime, painting, deformed, blurry, text, letters, watermark, logo, signage, subtitles, distorted faces, extra limbs, extra fingers, low quality, oversaturated

Consistency across scenes:

• The LLM writes one cast (each character's age, hair and exact clothing) and one look (lighting and colour grade).
• The cast is copied into every scene that shows people.
• Known gap: wardrobe still drifts between shots (e.g. orange instead of beige). This needs reference-image conditioning (IP-Adapter / face ID), planned as Phase C.

Without ComfyUI (GPU_SERVER_COMFY_URL unset), /v1/images returns a flat placeholder PNG (ffmpeg color + hue from the seed). Without a GPU server at all, AdLoop draws an SVG card with the line and the picture description (StoryboardBackend.generate_image).

Faces (prototype, make_faces.py): same model, 832 × 1216, 30 steps, CFG 4.5, dpmpp_2m karras, seeds 11/22/33/44. Prompt: "professional head and shoulders portrait photo of a friendly South Indian man, early 30s … photorealistic, 85mm lens". Face #3 (seed 33) was picked.

4. Video: scene clips with Wan 2.1 I2V (prototype)

Turns one approved scene picture into a ~3 s clip (gpu_server/prototypes/scenes.py, animate).

5. Talking avatar: InfiniteTalk (rejected)

Worked technically. Vijay rejected it because the lip sync was off and the face looked AI-made. Kept in prototypes/talk.py / run_talk.sh.

6. Voice-over (prototype)

Source: Vijay's own phone recording, used with his consent. Measured at 19 s, mono, 16 kHz AAC, peaking at +0.3 dBFS (clipped). For better results, record at 20–30 cm, as WAV / 48 kHz.

Mastering chain (prototypes/master_voice.sh, ffmpeg only, applied in this order):

Result: -20.2 → -16.5 LUFS integrated, loudness range 3.8 → 1.5 LU. The final ad mix applies loudnorm=I=-14:TP=-1.5 (social-media target).

AI voice (not used yet; free tools only):

7. Captions (prototype)

Word timings (prototypes/align.py):

• torchaudio MMS_FA forced alignment (ctc_alignment_mling_uroman, 1.18 GB, with_star=False) aligns the known script to the audio.
• The Telugu text is romanised with uroman 1.3.1.1.
• Tokens the aligner can't read get a spoken form, e.g. 2BHK → "tu bi ech ke".
• A word's start is clamped to at most 0.6 s before its end, because the aligner can absorb leading silence.
• Confidence on the test ad was 0.6–1.0 per word.

Rendering (prototypes/captions.py): an ASS subtitle file burned in by libass (HarfBuzz shaping) at 1080 × 1920.

Any language (prototypes/captions_v2.py): the same engine, driven by a small JSON config with keys style, colour, font, latin_font, total, cta_at, cta, phone, cta_sub, label and callouts[]. For example, Devanagari uses the Hind font (SIL OFL, installed by font_hi.sh); Telugu uses Hind Guntur. Hind has the same tall-metrics quirk, so the same sizes apply.

Silent videos (no voice-over):

• Each scene gets a fixed slot (3.4 s).
• Its words are spread over the slot by length: the window starts 0.25 s in and ends 0.35 s before the slot ends.
• The Pop highlight still moves word by word.
• A silent stereo track (anullsrc, AAC 128k) is added so social apps accept the upload.
• Example: the Shiv Ashray Hindi ad (prototypes/hotel_*.py), 6 scenes plus a 3 s end card = 23.4 s.

8. Final edit (prototype)

prototypes/compose.sh and render_caps.sh, ffmpeg 8.0.1:

Placeholder renderer (in product, gpu_server/renderers.py):

• Used for /v1/render, with no AI involved.
• 1080 × 1920, 30 fps, 5–60 s.
• Shows the headline, on-screen points, CTA, avatar image and AI label, rendered with libass and Noto fonts.
• Audio is silent (anullsrc).
• Encoding: x264 veryfast, AAC 128k.

9. GPU server ("GPU factory")

gpu_server/: FastAPI with one worker thread, so one GPU job at a time. 16 GB of VRAM can't hold an image model and a video model together.

• Job folders: data/jobs/<id>/ holds job.json, the inputs and output.mp4 / output.png. A task's result is renamed to output.* only after it finishes (an atomic rename), so a crash never leaves a half-written file looking done. After a restart, status is rebuilt from the folders.
• Auth: Authorization: Bearer <GPU_SERVER_TOKEN> on every request, when set.

AdLoop's client (media.GpuServerBackend) polls every 3 s and gives up after AD_ENGINE_GPU_TIMEOUT_SECONDS (default 1800).

10. GPU PC setup (WSL2)

Gotchas (each one cost a failed run):

• Strip the torch* lines from custom-node requirements.txt files, or they replace the cu128 build.
• Under WSL, pinned host memory is limited. Block swap needs use_non_blocking=false, or it fails with CUDA error: out of memory even when VRAM is free.
• blocks_to_swap: 20 with pinned memory runs out of memory; 40 fills the 12 GB of RAM and thrashes swap (GPU at 2%); 15 works.
• Use sdpa attention. Don't try to build flash-attn or sageattention for sm_120 under WSL.
• torchaudio.load needs torchcodec, so load audio through ffmpeg instead.
• Background jobs must be started with setsid nohup (prototypes/bg.sh). wsl -- bash -c "cmd &" dies with the call. Keep one Ubuntu window open, or Windows shuts WSL down.
• A stalled Hugging Face download can leave a process holding the file lock. Kill it before retrying.

11. Licences

12. AdLoop configuration

AdLoop's .env (never committed):

Production limits (src/ad_engine/studio.py):

• At most 3 scripts per round become videos (MAX_IN_PRODUCTION).
• Redo prompts use the last 5 feedback notes for that scene or storyboard.
• Scene and storyboard changes go through a lock and re-read the ad first, so a click during a slow LLM or GPU call is never lost.

13. Timings (RTX 5060 Ti, 12 GB WSL RAM)

14. Decisions log