# AI Scene Composer Specification & API Docs This document outlines the JSON schema, coordinates system, object parameters, and backend APIs for the **3D Video Renderer — AI Animation Studio**. This system is an automated WebGL rendering pipeline where scene orchestration and keyframe timelines are generated exclusively by AI agents. --- ## 1. Introduction & URL Routing ### 1.1 What is this Website? This web application is a real-time WebGL 3D player and MP4 exporter designed to display blocky, Roblox-style voxel models and animations. It has **no manual human creation workspace** (no editor UI). A human user writes a prompt (e.g., *"Make a car pull into a gas station, a person walks out to refuel, and drives off inside a spooky dark fog"*), and a client AI agent generates the configuration JSON file, uploads it via the APIs, and gives the user a working link to view/export the animation. ### 1.2 How to Open and Preview a Project for the User After creating or modifying a project configuration, the AI agent must construct and return a URL link so the user can open it. The player parses URL parameters to decide which project configuration to load: * **Load Database Project (Recommended)**: Format: `player.html?config=project:{project_id}` *Example*: `http://localhost:8000/vdo-3d-renderer/player.html?config=project:proj_10bc3f34f4f9` * **Load Inside Glassbits Main Container**: Format: `http://localhost:8000/vdo-3d-renderer/?config=project:{project_id}` * **Load Local Preset JSON File**: Format: `player.html?config=file:{filename.json}` *Example*: `player.html?config=file:simple.json` --- ## 2. Technical System Overview The engine runs on Three.js (WebGL) in a right-handed coordinate system (Y-up). It supports: * **Voxel Roblox Models** (parsed from `.roblox.json` formats). * **3D Models** (GLTF `.glb` and OBJ `.obj`). * **Procedural Primitives** (`primitive:box`, `primitive:sphere`, `primitive:cylinder`, `primitive:cone`, `primitive:plane`). * **Physical Light Sources** (`light:point`, `light:spot`). * **Volumetric Fog** and camera motion control. ### 2.1 SQLite/PDO Storage, Runtime Protection, and Bootstrap Notes * The backend stores project, asset, and user metadata in SQLite through PHP PDO. * The default SQLite database is `storage/renderer.sqlite`. Use `VDO_RENDERER_SQLITE_PATH` to override the database file, and `VDO_RENDERER_UPLOAD_DIR` to override the separate `uploads/` file-storage directory. * API endpoints must load `api/bootstrap.php` before database access. The bootstrap creates the database directory when needed, enables PDO exceptions and SQLite foreign keys, and initializes the schema/indexes safely on repeated requests. * Treat `manage_key` as a secret: use an authenticated project owner session or the correct project/asset `manage_key` for mutations, and do not include management keys in responses or generated links. --- ## 2. API Endpoints for AI Agents All backend API requests are relative to `/app/app/vdo-3d-renderer/`. ### 2.1 List Available Models (Assets) Returns a list of registered 3D models available for placement. * **Endpoint**: `GET api/assets.php?action=list` * **Response format**: ```json { "status": "success", "assets": [ { "id": "roblox_person", "original_name": "Roblox 3D Character Model", "filename": "roblox_person.roblox.json", "type": "roblox_model", "url": "/app/app/vdo-3d-renderer/uploads/roblox_person.roblox.json", "uploaded_at": "2026-07-18 10:40:02" } ] } ``` ### 2.2 Create or Update Project Submits a generated project JSON configuration to the server. * **Endpoint**: `POST api/projects.php` * **Headers**: `Content-Type: application/json` * **Payload Format**: ```json { "id": "proj_unique_id", "name": "Project Name", "description": "Short description of the scene", "created_by": "AI Agent Name", "visibility": "public", "config": { ...JSON Configuration... } } ``` --- ## 3. Project Configuration JSON Schema ### 3.1 Scene Properties (`scene`) Defines the global environment lights, background color, and fog. ```json "scene": { "backgroundColor": "#05060b", "showGrid": false, "ambientLight": { "color": "#111424", "intensity": 0.12 }, "directionalLight": { "color": "#ff9e4f", "intensity": 0.0, "position": [-25, 12, 10] }, "pointLight": { "color": "#ffa329", "intensity": 1.0, "distance": 15, "position": [-8, 3.5, -1] }, "fog": { "color": "#05060b", "density": 0.035 } } ``` * `fog`: Set `density` (Exponential decay factor) or `near` and `far` parameters to fade distant geometry into the background. --- ### 3.2 Objects List (`objects`) An array of 3D objects placed in the world. ```json "objects": [ { "id": "my_object_id", "assetId": "roblox_person", "type": "model3d", "initial": { "position": [0, 0, 0], "rotation": [0, 90, 0], "scale": [1, 1, 1] }, "material": { "color": "#ffe066", "emissive": "#000000", "opacity": 1.0, "roughness": 0.5, "metalness": 0.1 }, "castShadow": true, "receiveShadow": true, "keyframes": [ { "time": 0, "position": [0, 0, 30] }, { "time": 5.5, "position": [-8, 0, 10], "rotation": [0, -90, 0] } ] } ] ``` #### Object Parameters: * `id` (string): Unique identifier. * `assetId` (string): Refer to a database model ID, or use primitive/environmental keywords: * `primitive:box` (Cube) * `primitive:sphere` (Sphere) * `primitive:cylinder` (Cylinder) * `primitive:cone` (Cone) * `primitive:plane` (Flat 2D plane) * `light:point` (PointLight source) * `light:spot` (SpotLight source) * `particles:rain` (Vertically falling rain particles) * `particles:snow` (Slowly drifting winter snow particles) * `particles:smoke` (Rising and expanding dark grey smoke puffs) * `particles:fire` / `particles:sparks` (Additive blending glowing orange sparks rising) * `particles:dust` (Gentle ambient room/spooky dust particles drifting) * `initial`: The initial position `[X, Y, Z]`, rotation `[X, Y, Z]` (in degrees), and scale `[X, Y, Z]`. * `pivot` (array `[X, Y, Z]` in meters, optional): Offset the model's center point to redefine its rotation pivot. For example, for a box with a height of 2.0, setting `"pivot": [0, -1.0, 0]` moves the pivot down to its bottom face, making it rotate/spin from its base instead of its center. * `parent` (string, optional): ID of another object in the scene to attach/snap this object to. The child object will move, rotate, and scale automatically with the parent. * `parentBone` (string, optional): Specific joint/bone name to attach this object to (requires `parent` to be defined). Works with rigged GLTF characters and Roblox named limbs (e.g. `"right_arm"`, `"head"`). * *Note on Snapping Coordinates*: When an object is snapped to a parent or bone, its `initial` and `keyframes` coordinates (`position`, `rotation`, `scale`) automatically act as **local offsets** relative to that parent/bone origin! * `material`: Set properties for primitive geometries. Custom models load default textures but will apply simple PBR properties if provided. Supported fields: * `color` (Hex string): Base diffuse color. * `emissive` (Hex string): Emissive (glow) color. * `opacity` (float): Material opacity (0.0 to 1.0). * `roughness` (float): Material surface roughness (0.0 to 1.0). * `metalness` (float): Material metallic finish (0.0 to 1.0). * `texture` (string URL/path): Path to a PNG/JPG image file to wrap onto the mesh geometry. * `repeat` (array `[X, Y]`): Number of times the texture tiles along the X and Y coordinates (e.g. `[8, 8]`). * `size` (float, optional for particles): Custom size scale of individual particle points (e.g. `0.08` for rain, `0.6` for smoke). * `keyframes`: Timeline events. Position, rotation, and scale will interpolate linearly between keyframe timestamps (`time` in seconds). * **Limbs & Bones Animation**: To animate individual limbs/bones of rigged characters (like rigged GLTF models or Roblox JSON models with named parts), add a `"bones"` object inside the keyframe: ```json "bones": { "left_arm": { "rotation": [45, 0, 0] }, "right_arm": { "rotation": [-45, 0, 0] } } ``` * `rotation` (array `[X, Y, Z]` in degrees): Pitch, Yaw, Roll rotations applied to the bone/part. * `position` (array `[X, Y, Z]` in local units): Translation offset of the bone (optional). * *Standard Roblox Limb Names*: `"head"`, `"torso"`, `"left_arm"`, `"right_arm"`, `"left_leg"`, `"right_leg"`. * **Play Embedded Animations**: For 3D models (`model3d`) containing pre-baked animation clips (like `"run"`, `"walk"`, `"sleep"`, `"jump"`), trigger and blend playback by setting `"animation"` in the keyframe: * *Simple Format (String Name)*: `"animation": "run"` * *Advanced Format (Object with speed controls)*: ```json "animation": { "name": "run", "timeScale": 1.5 } ``` * `name` (string): Name of the animation clip. * `timeScale` (float): Playback speed multiplier (default `1.0`, e.g. `1.5` for fast-run, `0.5` for slow-walk). --- ### 3.3 Lights inside the `objects` array Multiple active lights can be declared under the `objects` array: * **Point Light (`light:point`)**: ```json { "id": "point_light_1", "assetId": "light:point", "initial": { "position": [3.1, 3.8, 20] }, "material": { "color": "#ffea88", "intensity": 5.0, "distance": 22 } } ``` * **Spot Light (`light:spot`)**: ```json { "id": "spot_light_1", "assetId": "light:spot", "initial": { "position": [-8.0, 3.0, 1] }, "material": { "color": "#ffa329", "intensity": 3.0, "distance": 18, "angle": 0.6, "penumbra": 0.8 } } ``` *Note: SpotLights point straight downward automatically (targeting Y=0 on the same X/Z plane).* --- ### 3.4 Camera Motion Control (`camera`) Controls the movement of the director camera. ```json "camera": { "initial": { "position": [6, 4, 25], "target": [0, 1, 15], "fov": 45 }, "keyframes": [ { "time": 0, "position": [6, 4, 25], "target": [0, 1, 15] }, { "time": 6, "position": [-3, 4.5, 8], "target": [-6, 1, -2] } ] } ``` * `position`: Camera Y-up location. * `target`: The focus point `[X, Y, Z]` that the camera looks at. --- ## 4. Special Engine Behaviors * **Auto-attached Headlights**: Any object in the `objects` list whose `id` or `assetId` contains the substring `'car'` (case-insensitive) will automatically receive twin front spotlights and glow sources attached to the model. They move and rotate in sync with the car automatically. * **FrontSide Shadow acne prevention**: Closed solid meshes (boxes, cylinders, spheres) default to `THREE.FrontSide` material rendering to prevent parallel shadow acne lines on flat surfaces. Only 2D planes (`plane` primitive) default to `THREE.DoubleSide`. * **Auto-Normalization & Pivot Correction**: To prevent third-party 3D models (`model3d` type like `.glb`, `.gltf`, `.obj`) from loading at unpredictable sizes or sinking/floating due to arbitrary local pivots, the loader automatically intercepts model instantiation: 1. It recalculates the model's bounding box and shifts the geometry internally so that the bottom center sits exactly at local `(0, 0, 0)`. 2. It scales the model's local dimensions so that its height is exactly **1.0 unit**. 3. Because of this unit normalization, the `"scale": [X, Y, Z]` set in the project JSON corresponds exactly to the final size in meters (e.g. setting scale `[2.0, 1.5, 4.5]` guarantees the model will be exactly 2m wide, 1.5m tall, and 4.5m long). 4. *Opt-out*: To use the raw native scale and pivot of a 3D model instead, set `"normalize": false` inside the object's JSON properties. --- ## 5. Model Creation & Scaling Rules (Preventing Size Distortion) To ensure that models created by AI agents render with correct proportions and align perfectly in the scene, the following standards must be followed strictly: ### 5.1 Unit Scale Definition (Real-World Metric Standard) * **1 Unit = 1 Meter** in the WebGL world. * **Humanoid Characters**: Total height should be **2.0 units** (approx. 2 meters / 6.5 ft). * *Head*: scale `[0.4, 0.4, 0.4]` centered at local `Y = 1.8`. * *Torso*: scale `[0.8, 0.8, 0.4]` centered at local `Y = 1.2`. * *Arms*: scale `[0.4, 0.8, 0.4]` centered at local `Y = 1.2`. * *Legs*: scale `[0.4, 0.8, 0.4]` centered at local `Y = 0.4`. * **Vehicles (Cars)**: Width should be **2.0 units**, Height **1.5 units**, Length **4.5 units**. * **Pumps / Streetlights**: Tall streetlights should stand **4.0 to 5.0 units** tall. ### 5.2 Pivot Point (Local Origin Alignment) * **Characters & Ground Assets**: The local origin `(0, 0, 0)` must be placed at the **bottom center of the model** (touching the ground at `Y = 0`). When the AI places a model at `position: [X, 0, Z]`, the model must stand perfectly on the ground without sinking or floating. * **Vehicles**: The local origin `(0, 0, 0)` must be placed at the **bottom center of the tires** (where the wheels touch the asphalt, `Y = 0`). * **Structural Objects (Pillars, Canopy)**: Origin must be at the base center (`Y = 0`). ### 5.3 Local Orientation (Forward Direction) * **Forward Direction**: All models must face **local positive Z** (towards `+Z`). * Example: A car's headlights and a character's face must look in the direction of local `+Z`. This allows the renderer and script engines (such as the auto-headlight script) to correctly align sub-elements without guessing orientations. --- ## 6. Dynamic Asset Upload & Registration API (`POST api/assets.php`) AI agents can dynamically upload new files or register existing external files (hosted on GitHub, Dropbox, Catbox, etc.) on-the-fly. The endpoint accepts standard multipart/form-data payloads: * **Endpoint**: `POST api/assets.php` * **Request Payload**: * `file` (File binary, optional): The asset file to upload. * `url` (string, optional): The direct HTTP download URL of the file hosted externally. (Use either `file` or `url`). Using `url` is highly recommended to save storage space and bandwidth! * `name` (string, optional): Display name for the asset. * `id` (string, optional): Custom unique ID. * `type` (string, optional): Explicit override of the type (e.g. `"model3d"`, `"image2d"`, `"audio"`). If omitted, the server will auto-detect from the URL/file extension. * `width` (float, REQUIRED for models): Intended target width of the model in real-world meters. * `height` (float, REQUIRED for models): Intended target height of the model in real-world meters. * `length` (float, REQUIRED for models): Intended target length of the model in real-world meters. * **Allowed File Extensions & Auto-Assigned Types**: * `.glb` / `.gltf` / `.obj` -> Registered as `model3d` (3D Models) * `.json` -> Registered as `roblox_model` (Voxel Models) * `.png` / `.jpg` / `.jpeg` / `.gif` / `.svg` -> Registered as `image2d` (Textures/Sprites) * `.mp3` / `.wav` / `.ogg` -> Registered as `audio` (Soundtracks/SFX) * **Response format**: ```json { "status": "success", "asset": { "id": "generated_or_custom_id", "original_name": "filename.glb", "filename": "generated_filename.glb", "type": "model3d", "url": "/app/app/vdo-3d-renderer/uploads/generated_filename.glb", "size": 142054, "uploaded_at": "2026-07-18 10:52:00" } } ``` --- ## 7. GeoJSON 3D Map Generator API (From Geo Project) AI agents can leverage the 3D Map Generator endpoint (from the sibling `geojson-3d-map` project) to convert standard 2D geographic maps (GeoJSON boundaries) into fully-structured 3D GLTF meshes. This generated map model can then be downloaded and uploaded to the renderer as a `model3d` asset. * **Endpoint URL**: `/api/generate_gltf.php` (relative to Glassbits root, e.g. `http://localhost:8000/api/generate_gltf.php`) * **HTTP Method**: `POST` or `GET` * **Request Parameters**: * `geojson` (string, REQUIRED): Valid JSON string of a GeoJSON FeatureCollection or Polygon geometry. * `height_property` (string, optional): Key inside feature properties to extract height attributes (e.g. `"height"` or `"density"`). * `extrusion_scale` (float, optional): Scaling multiplier for height extrusion. (Default: `10.0`). * `base_color` (string, optional): HEX color code for the generated map material (e.g. `#ff0055`). (Default: `#00ffb2`). * `action` (string, optional): Set to `download` to prompt a direct browser attachment download (`geojson-3d-model.gltf`). If omitted, returns a JSON response containing the raw GLTF JSON. * **Example Integration Link**: `http://localhost:8000/api/generate_gltf.php?action=download&geojson={...}&base_color=%23ff0055` --- ## 8. Recommended Future Engine Enhancements To make this automated animation studio more mature, the following features are recommended for future implementation: 1. **Animation Easing / Interpolation Curves**: * Add an optional `"easing"` key to keyframes (e.g. `"linear"`, `"easeInOut"`, `"easeOut"`, `"bounce"`). This will make camera pans and decelerating vehicles look extremely organic instead of linearly robotic. 2. **Keyframe Sound Event Trigger System**: * Add an `"audio"` or `"sfx"` array to the project JSON where sound effects trigger at specific timeline seconds (e.g. at second `4.2`, play a `door_close.mp3` sound). --- ## 9. Free 3D Model Direct Link Sources & CDN Guides To avoid hosting massive 3D models on the local server, AI agents should use public, CORS-enabled direct download links from trusted open-source repositories. To use these files, register them via `POST api/assets.php` by sending the direct link in the `url` parameter. ### 9.1 Recommended CORS-Enabled Asset Registries Here are standard, tested public models with pre-baked skeletal animations and shapes that are 100% compatible with the renderer: 1. **Three.js Official Examples Assets** (Rigged characters and animated elements): * **Robot Expressive** (Rigged robot with emotions & dances): * *Direct Link*: `https://cdn.jsdelivr.net/gh/mrdoob/three.js@dev/examples/models/gltf/RobotExpressive.glb` * *Supported Built-in Animations*: `"Idle"`, `"Walking"`, `"Running"`, `"Dance"`, `"Jump"`, `"Yes"`, `"No"`, `"ThumbsUp"` * *Dimensions*: `width: 1.0`, `height: 2.0`, `length: 1.0` meters. * **Soldier** (Rigged soldier character): * *Direct Link*: `https://cdn.jsdelivr.net/gh/mrdoob/three.js@dev/examples/models/gltf/Soldier.glb` * *Supported Built-in Animations*: `"Idle"`, `"Walk"`, `"Run"` * *Dimensions*: `width: 0.8`, `height: 1.8`, `length: 0.8` meters. * **Parrot** (Flying bird): * *Direct Link*: `https://cdn.jsdelivr.net/gh/mrdoob/three.js@dev/examples/models/gltf/Parrot.glb` * *Supported Built-in Animations*: `"parrot_A_"` (Flying action) * *Dimensions*: `width: 1.5`, `height: 1.0`, `length: 1.5` meters. 2. **Khronos Group glTF Sample Assets** (High-quality props and items): * **Sci-Fi Damaged Helmet**: * *Direct Link*: `https://cdn.jsdelivr.net/gh/KhronosGroup/glTF-Sample-Models@master/2.0/DamagedHelmet/glTF-Binary/DamagedHelmet.glb` * *Dimensions*: `width: 1.0`, `height: 1.0`, `length: 1.0` meters. * **Toy Duck**: * *Direct Link*: `https://cdn.jsdelivr.net/gh/KhronosGroup/glTF-Sample-Models@master/2.0/Duck/glTF-Binary/Duck.glb` * *Dimensions*: `width: 1.0`, `height: 1.0`, `length: 1.0` meters. 3. **Poimandres Drei Assets** (Optimized testing assets): * **Roblox Avatar** (Smooth blocky character): * *Direct Link*: `https://cdn.jsdelivr.net/gh/pmndrs/drei-assets@master/roblox.glb` * *Dimensions*: `width: 1.0`, `height: 2.0`, `length: 0.5` meters. ### 9.2 How to Generate CORS-Enabled Direct URLs When hosting custom models on other sites, follow these rules to get valid links for Three.js: * **GitHub Repositories**: Upload your `.glb` file to a public GitHub repo, and convert the URL to a CDN URL using jsDelivr: * *GitHub URL*: `https://github.com/username/repo/blob/branch/my_model.glb` * *Direct jsDelivr CDN link (Use this)*: `https://cdn.jsdelivr.net/gh/username/repo@branch/my_model.glb` * **Dropbox**: Upload the file, click "Share" and copy the link. Change the trailing `?dl=0` to `?raw=1`: * *Dropbox Share Link*: `https://www.dropbox.com/s/random/my_model.glb?dl=0` * *Direct Raw Link (Use this)*: `https://www.dropbox.com/s/random/my_model.glb?raw=1` * **Catbox.moe**: Upload the file on the homepage, and use the direct link provided (e.g. `https://files.catbox.moe/xxxxxx.glb`).