AI & AUTOMATION

HyperFrames: Turn HTML into Deterministic Video for AI Agents (Full Setup & Guide)

Duy NghienDuy Nghien
970 views0

Key Takeaway:

HyperFrames lets AI agents and developers turn plain HTML into deterministic, locally rendered video using a simple, agent-native toolchain built on standard web technologies.

What is HyperFrames?

HyperFrames is an open-source video rendering framework from HeyGen that lets you create, preview, and render HTML-based video compositions with first-class support for AI agents. Instead of learning a proprietary timeline editor, you describe your video as a normal web page using HTML, CSS, and JavaScript, then let HyperFrames render every frame into an MP4, MOV, or WebM file locally.

At its core, HyperFrames is an HTML-based video toolchain and rendering engine that treats each DOM element as a “clip,” controlled by simple data- attributes that define timing, duration, and layering. Because rendering is deterministic and frame-by-frame, the same HTML input always produces identical video output, which is critical for reproducible agentic pipelines and CI-style workflows.

You can explore the official HyperFrames repository on GitHub at https://github.com/heygen-com/hyperframes

Why HTML-to-Video is Perfect for AI Agents

Most AI agents already “speak” HTML, CSS, and JavaScript far better than they understand timeline-based video editors, because they have been trained on massive amounts of web code. HyperFrames leverages this by using HTML as the composition format, with data- attributes to express timing and tracks so agents can generate valid video scenes without learning a new DSL.

The framework runs locally using headless Chrome (via Puppeteer) for frame capture and FFmpeg for encoding, eliminating the need for external APIs or cloud render services. This architecture makes HyperFrames especially attractive for privacy-sensitive workflows, self-hosted agent systems, and high-volume automation where deterministic, reproducible renders are a must.

Prerequisites and Installation

Before you can render HTML to video with HyperFrames, you need a modern Node.js runtime and FFmpeg installed on your machine.

Install Node.js 22+

HyperFrames requires Node.js version 22 or later because the CLI and dev server rely on modern runtime features. You can check your installed version with:

node --version

You should see an output similar to:

v22.0.0

Any version equal to or greater than 22 is supported.

If you are below v22, install or upgrade Node.js from the official Node.js website or via a version manager like nvm before proceeding.

Install FFmpeg

FFmpeg is used to encode captured frames into MP4 (or other formats) during the render step. On macOS with Homebrew, you can install it using:

brew install ffmpeg

After installation, verify that FFmpeg is available:

ffmpeg -version

You should see a version banner such as:

ffmpeg version 7.x ...

indicating a working FFmpeg installation.

Option 1: Install HyperFrames Agent Skills

If you plan to drive HyperFrames primarily through an AI coding agent like Claude Code, Cursor, Gemini CLI, or similar, you can install the official skill package:

npx skills add heygen-com/hyperframes

This teaches your agent how to structure compositions, use GSAP animations, and call the CLI correctly using slash commands such as /hyperframes and /hyperframes-cli.

Option 2: Initialize a HyperFrames Project Manually

To work directly from the terminal and your editor, scaffold a new HyperFrames project with the CLI.

npx hyperframes init my-video
cd my-video

This starts an interactive wizard that helps you pick examples, import media, and set up project metadata. For non-interactive environments or agent-driven workflows, you can bypass prompts and start from a blank composition:

npx hyperframes init my-video --non-interactive --example blank

If you already have a source video and want automatic transcription and captions, include it during init:

npx hyperframes init my-video --example warm-grain --video ./intro.mp4

The hyperframes init command also installs the associated AI skills so your coding agent can immediately understand and modify the project.

Understanding the Project Structure

After initialization, your my-video directory will look something like this.

my-video/
  meta.json
  index.html
  compositions/
    intro.html
    captions.html
  assets/
    video.mp4
  • meta.json: Stores project metadata such as name, ID, and creation date.
  • index.html: The root composition and main entry point for your video timeline.
  • compositions/: Contains sub-compositions that can be loaded via data-composition-src for modular scenes.
  • assets/: Holds external media assets like audio, images, and source video clips.

This layout feels familiar to web developers while still mapping cleanly to video concepts like main timelines, nested scenes, and media libraries.

Writing Your First HTML Video Composition

HyperFrames defines video scenes as HTML documents that follow a few simple rules for the root composition and timed elements.

The Root Composition

Your index.html needs a root element that declares a unique composition ID and the output resolution:

<div
  id="root"
  data-composition-id="my-video"
  data-start="0"
  data-width="1920"
  data-height="1080"
>
  <!-- Clips and scripts go here -->
</div>

Here, data-width and data-height define the video resolution, and data-start sets the starting timestamp of the composition.

Defining Timed Clips

Inside the root, each visual element that participates in the timeline is tagged as a “clip” using data-start, data-duration, and data-track-index.

<h1
  id="title"
  class="clip"
  data-start="0"
  data-duration="5"
  data-track-index="0"
  style="
    font-size: 72px;
    color: white;
    text-align: center;
    position: absolute;
    top: 50%;
    left: 50%;
    transform: translate(-50%, -50%);
  "
>
  Hello, HyperFrames!
</h1>
  • data-start: When this clip appears on the timeline (in seconds).
  • data-duration: How long it remains visible.
  • data-track-index: Which track (layer) it belongs to, allowing stacking and compositing.

Adding GSAP Animations

You can use GSAP (or other supported runtimes) to create seekable animations that HyperFrames controls frame-by-frame.

<script src="https://cdn.jsdelivr.net/npm/gsap@3/dist/gsap.min.js"></script>
<script>
  const tl = gsap.timeline({ paused: true });
  tl.from("#title", { opacity: 0, y: -50, duration: 1 }, 0);

  window.__timelines = window.__timelines || {};
  window.__timelines["my-video"] = tl;
</script>

The key rules are:

  • The GSAP timeline must be created with { paused: true } so HyperFrames can control playback for each frame.
  • You register timelines on window.__timelines using the composition ID as the key, enabling the engine to seek them accurately.

With these pieces combined, your minimal index.html contains a fully defined video: clips, timing, and animation, all represented in standard HTML and JavaScript.

Previewing and Rendering Video

Once your composition is in place, HyperFrames provides two essential CLI commands: preview for interactive iteration and render for final output.

Preview in the Browser

To open HyperFrames Studio and see your video in the browser, run:

npx hyperframes preview

This starts a dev server, launches a browser preview, and hot‑reloads whenever you save changes to index.html or other composition files. The preview mode is ideal for fine‑tuning typography, layout, animation easing, and timing before you commit to a full render.

Render to MP4

When you are ready to produce a deterministic output video, call the render command:

npx hyperframes render --output output.mp4

HyperFrames then loads your composition in headless Chrome, seeks to each frame based on the configured FPS, captures the DOM, and pipes frames into FFmpeg for encoding. A typical log might look like:

✔ Capturing frames... 150/150
✔ Encoding MP4...
✔ output.mp4 (1920x1080, 5.0s, 30fps)

The resulting output.mp4 is a pixel‑perfect, deterministic recording of your HTML composition that you can ship, upload, or feed into downstream automation.

Integrating HyperFrames into Agentic Workflows

HyperFrames is explicitly designed to be “built for agents,” and HeyGen ships skills that teach AI coding tools how to author valid compositions and CLI commands. Once you run npx skills add heygen-com/hyperframes, agents like Claude Code or Cursor gain slash commands such as /hyperframes (for composition authoring) and /hyperframes-cli (for shell commands).

A typical loop looks like this:

  1. Initialize a project with npx hyperframes init my-video, which automatically installs the skills.
  2. Open the repo in your AI coding agent and prompt it to “create a 10‑second product intro video using HyperFrames with a title card and fade‑in animation.”
  3. Let the agent edit index.html and compositions/*.html, generating HTML, CSS, and GSAP timelines using the framework patterns.
  4. Use npx hyperframes preview to review the result visually, then npx hyperframes render --output final.mp4 to produce a deterministic MP4.

Because everything is local and deterministic, you can safely plug HyperFrames into multi‑step automation systems, CI pipelines, or self‑hosted orchestrators without depending on third‑party video APIs.hyperframes.

Practical Ideas for Video Automation with HyperFrames

Once your environment is set up, you can think of HyperFrames as “video as code” for your AI agents and automation scripts. Some practical use cases include:

  • Turning structured data (CSV, analytics dashboards, or database snapshots) into animated chart videos or bar chart races by mapping rows to clips and using GSAP for transitions.hyperframes.
  • Converting existing landing pages into scroll‑driven product overview videos by capturing layouts and re‑using brand colors, typography, and assets.
  • Generating personalized onboarding videos, patch notes, or product updates where agents assemble scenes from reusable HTML blocks and fill in dynamic text.hyperframes.

Because the composition format is plain HTML, you can version control everything in Git, code‑review changes, and let multiple agents or human collaborators iterate on the same video project. And if you need to go deeper, the official docs and catalog at the HyperFrames site offer block libraries, GSAP recipes, and advanced rendering options built on the same core primitives you have seen here.hyperframes.

You may also like

Subscribe
Notify of
guest

0 Comments
Newest
Oldest Most Voted