Files
hyperframes/packages/core
Miguel Ángel 970b446c49 feat(studio): drag assets from the sidebar onto the timeline (#464)
## Problem

Studio still broke down in three concrete authoring flows around timeline assets:

- you could import media into Assets, but not drag an already-imported asset from the Assets tab onto the timeline and persist it into source
- dragging a file from outside the app onto the timeline only uploaded it into Assets instead of placing it at the dropped time/track
- once a clip was on the timeline, there was no reliable keyboard delete flow for removing it safely from source

While implementing direct external drops, another real bug showed up:

- valid binary uploads like `raycast.mp4` from `Downloads` were being rejected as unsupported media in Studio dev because the Vite API bridge was corrupting multipart request bodies before they reached the upload route

## What this fixes

### Timeline asset placement from inside Studio

- asset cards in the Assets tab are draggable
- the timeline accepts asset drops even when it already has clips
- dropping an asset onto the timeline inserts a new clip into the active composition source at the dropped time / track
- asset paths are rewritten relative to the target composition file so drops into sub-compositions resolve correctly
- the new clip is persisted immediately and the preview refreshes

### Direct external file drops onto the timeline

- dropping a file from outside the app onto the timeline now uploads it and places it onto the dropped track/time in one shot
- it no longer stops halfway by only adding the file into Assets
- multiple dropped files are placed using the same drop start and successive tracks

### Delete key support

- selected timeline clips can now be deleted with `Delete` / `Backspace`
- deletion is persisted back to source, not just removed from local state
- the delete path now uses a server-side DOM mutation helper with LinkeDOM for structural safety instead of client-side string surgery

### Binary upload fix for media files

- the Studio Vite API bridge now forwards non-GET request bodies as raw bytes instead of decoding them as UTF-8 text
- that preserves multipart uploads for binary media like MP4s
- valid local videos from `Downloads` no longer get rejected as `Unsupported media skipped` just because the dev bridge corrupted the request body
- upload validation now probes buffered media through a temp file path that preserves the file extension before saving into the project

## Root cause

There were really two separate gaps:

### 1. Asset placement / deletion workflow gaps

The timeline and asset systems already existed, but they were disconnected:

- `AssetsTab` only supported copy/import flows
- `Timeline` only handled raw file import, not positioned placement for existing assets
- there was no utility layer for converting a dropped asset into persisted timeline HTML
- there was no structurally safe deletion path for arbitrary selected timeline clips

### 2. Binary upload corruption in Studio dev

The Studio Vite API bridge rebuilt non-GET request bodies like this:

- read each request chunk
- call `chunk.toString()`
- concatenate into a string
- construct the Fetch `Request` from that string body

That works for text, but it corrupts multipart binary uploads. By the time the upload route wrote the received file and ran `ffprobe`, otherwise valid MP4s had already been mangled in-flight.

## Behavior

- dropping on `index.html` inserts the asset into the root composition
- dropping while drilled into a composition inserts into that composition file instead
- drop X position maps to `data-start`
- drop Y position maps to the current visible track row, with a new bottom track created if the drop lands below existing rows
- images default to a short finite duration
- audio/video default to their metadata duration when available, with a fallback duration if metadata cannot be read quickly
- pressing `Delete` on a selected clip removes that clip from the underlying HTML source and clears selection in Studio
- valid uploaded MP4s now survive the Studio dev API bridge intact instead of being rejected during upload validation

## Verification

### Local checks

- `bunx oxlint packages/core/src/studio-api/helpers/sourceMutation.ts packages/core/src/studio-api/helpers/sourceMutation.test.ts packages/core/src/studio-api/helpers/mediaValidation.ts packages/core/src/studio-api/helpers/mediaValidation.test.ts packages/core/src/studio-api/routes/files.ts packages/studio/src/App.tsx packages/studio/src/components/nle/NLELayout.tsx packages/studio/src/components/sidebar/AssetsTab.tsx packages/studio/src/player/components/Timeline.tsx packages/studio/src/player/components/Timeline.test.ts packages/studio/src/utils/timelineAssetDrop.ts packages/studio/src/utils/timelineAssetDrop.test.ts packages/studio/vite.config.ts packages/studio/vite.request-body.ts packages/studio/vite.request-body.test.ts`
- `bunx oxfmt --check` on the touched files
- `bun run --filter @hyperframes/core typecheck`
- `bun run --filter @hyperframes/studio typecheck`
- `bun test packages/core/src/studio-api/helpers/sourceMutation.test.ts packages/core/src/studio-api/helpers/mediaValidation.test.ts packages/studio/src/player/components/Timeline.test.ts packages/studio/src/utils/timelineAssetDrop.test.ts packages/studio/vite.request-body.test.ts`

### Browser / live verification

Verified against a live local Studio fixture:

- dragging an existing asset from the Assets tab onto the timeline creates a persisted clip at the dropped position
- dropping a file from outside the app directly onto the timeline uploads it and creates a persisted clip at the dropped position
- selecting a dropped clip and pressing `Delete` removes it from both the live timeline and the saved source HTML
- valid MP4 uploads like `raycast.mp4` now succeed through the live Studio upload route instead of being rejected as unsupported media

## Notes

- the local `timeline-trio-verify` and `timeline-overlap-debug` projects used for verification are local-only and are not part of this PR
- this PR is about asset placement, upload correctness, and deletion safety; it does not broaden into richer editing workflows beyond placing/removing clips from the timeline
2026-04-24 02:50:36 +02:00
..
2026-04-23 18:20:18 -04:00
2026-03-21 22:43:56 -07:00
2026-03-21 22:43:56 -07:00

@hyperframes/core

Types, parsers, generators, compiler, linter, runtime, and frame adapters for the Hyperframes video framework.

Install

npm install @hyperframes/core

Most users don't need to install core directly — the CLI, producer, and studio packages depend on it internally.

What's inside

Module Description
Types TimelineElement, CompositionSpec, Asset, canvas dimensions, defaults
Parsers parseHtml — extract timeline elements from HTML; parseGsapScript — parse GSAP animations
Generators generateHyperframesHtml — produce valid Hyperframes HTML from a composition spec
Compiler compileTimingAttrs — resolve data-start / data-duration into absolute times
Linter lintHyperframeHtml — validate Hyperframes HTML (missing attributes, overlapping tracks, etc.)
Runtime IIFE script injected into the browser — manages seek, media playback, and the window.__hf protocol
Frame Adapters Pluggable animation drivers (GSAP, Lottie, CSS, or custom)

Frame Adapters

A frame adapter tells the engine how to seek your animation to a specific frame:

import { createGSAPFrameAdapter } from "@hyperframes/core";

const adapter = createGSAPFrameAdapter({
  getTimeline: () => gsap.timeline(),
  compositionId: "my-video",
});

Implement FrameAdapter for custom animation runtimes:

import type { FrameAdapter } from "@hyperframes/core";

const myAdapter: FrameAdapter = {
  id: "my-adapter",
  getDurationFrames: () => 300,
  seekFrame: (frame) => {
    /* seek your animation */
  },
};

Parsing and generating HTML

import { parseHtml, generateHyperframesHtml } from "@hyperframes/core";

const { elements, metadata } = parseHtml(htmlString);
const html = generateHyperframesHtml(spec);

Linting

import { lintHyperframeHtml } from "@hyperframes/core/lint";

const result = lintHyperframeHtml(htmlString);
// result.findings: { severity, message, elementId }[]

Documentation

Full documentation: hyperframes.heygen.com/packages/core