跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
视频与音频

3dgs-mcp-renderer

MCP protocol integration with 3DGS rendering pipeline: Agent-controlled Three.js/WebGPU rendering, voice-driven scene reconstruction, real-time parameter manipulation, light tracing backend. Use when: MCP rendering, agent-controlled 3DGS, voice-driven reconstruction, real-time 3DGS editing, Three.js 3DGS, WebGPU Gaussian splatting, interactive rendering control, speech-to-3D, light tracing, HiGS accelerated rendering.

jaccen · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add jaccen/Awesome-Gaussian-Skills --skill 3dgs-mcp-renderer

技能说明


name: 3dgs-mcp-renderer description: "MCP protocol integration with 3DGS rendering pipeline: Agent-controlled Three.js/WebGPU rendering, voice-driven scene reconstruction, real-time parameter manipulation, light tracing backend. Use when: MCP rendering, agent-controlled 3DGS, voice-driven reconstruction, real-time 3DGS editing, Three.js 3DGS, WebGPU Gaussian splatting, interactive rendering control, speech-to-3D, light tracing, HiGS accelerated rendering." license: Apache-2.0 metadata: version: "1.1.0" author: jaccen tags: ["mcp", "3dgs", "gaussian-splatting", "rendering", "three.js", "webgpu", "voice", "agent", "interactive", "spec-first", "sculpting", "code-first", "slat", "latent-editing"] disable-model-invocation: true user-invocable: true

3DGS MCP Renderer — Agent-3DGS Interaction via MCP Protocol

Prototype specification for integrating MCP (Model Context Protocol) with 3DGS rendering pipelines, enabling AI Agents to directly manipulate Three.js/3DGS rendering parameters and achieve voice-driven 3D scene reconstruction.

Architecture

┌─────────────┐     ┌─────────────┐     ┌──────────────────┐     ┌──────────────────┐
│ Voice/Text  │────▶│   Agent     │────▶│  MCP Server      │────▶│  3DGS Renderer   │
│ (Whisper/   │     │ (Claude/    │     │  (Node.js/       │     │  (Three.js/      │
│  Prompt)    │     │  TeleClaw)  │     │   Python)        │     │   WebGPU/HiGS/   │
│             │◀────│             │◀────│                  │◀────│   DDF-GS)        │
└─────────────┘     └─────────────┘     └──────────────────┘     └──────────────────┘
                        │                      │                       │
                        │  Tool calls          │  WebSocket/HTTP       │  WebGL/WebGPU/
                        │  (MCP protocol)       │  transport            │  HiGS/DDF-GS

Spec-First Sculpting Pipeline (v0.9.0)

Design inspiration: img2threejs (GitHub: img2threejs/img2threejs) — open-source AI Skill that converts a single image into an interactive Three.js 3D model via a stage-gated sculpting pipeline. We borrow two core principles: (1) spec-first — define quality criteria and component hierarchy before any rendering; (2) stage-gated sculpting — progressive refinement with acceptance checks at each stage.

Why Spec-First for MCP Rendering?

The original MCP pipeline was reactive: user issues a voice command → agent maps to a tool → render → verify. This works for single-step edits but fails for complex scene construction because:

  • No upfront quality criteria → agent cannot self-assess before rendering
  • No stage gates → errors compound across steps (bad camera → bad selection → bad edit)
  • No component hierarchy → edits are flat, no part-level control

The fix: Introduce a define_scene_spec tool that runs before any sculpting/editing tools. This produces a machine-readable Object Spec that subsequent tools reference as acceptance criteria.

The 6-Stage Sculpting Pipeline

┌─────────────────────────────────────────────────────────────┐
│                    SPEC-FIRST SCULPTING                      │
│                                                             │
│  ┌──────────────┐                                           │
│  │ define_scene │  ← Object Spec: component hierarchy,      │
│  │ _spec        │    material system, quality criteria      │
│  └──────┬───────┘                                           │
│         │                                                   │
│         ▼                                                   │
│  Stage 1: blockout    → Bounding boxes, camera framing     │
│         │  gate: bbox coverage ≥ spec.target_coverage?      │
│         ▼                                                   │
│  Stage 2: structural  → Part decomposition, hierarchy      │
│         │  gate: part count & nesting matches spec?         │
│         ▼                                                   │
│  Stage 3: form        → Gaussian density/scale/rotation     │
│         │  gate: PSNR estimate ≥ spec.min_psnr?             │
│         ▼                                                   │
│  Stage 4: material    → PBR/SH assignment per part         │
│         │  gate: material count per part matches spec?      │
│         ▼                                                   │
│  Stage 5: surface     → Normal consistency, thin structures │
│         │  gate: normal consistency score ≥ spec.threshold? │
│         ▼                                                   │
│  Stage 6: lighting    → Environment, shadows, AO           │
│            gate: render quality score ≥ spec.target_score?  │
└─────────────────────────────────────────────────────────────┘

Each stage is an MCP tool call. The agent renders a frame after each stage, evaluates against the gate, and either advances or retries. This mirrors img2threejs's blockout → structural → form → material → surface → lighting flow.

Gate Evaluation Protocol

For each stage gate, the agent follows this protocol:

1. Execute stage tool (e.g., sculpt_form with parameters)
2. Call render_frame() to get current visual state
3. Call query_scene(query_type="stats") to get quantitative metrics
4. Compare metrics against spec gate criteria
5. If pass → advance to next stage
6. If fail → adjust parameters and retry (max 3 attempts)
7. If 3 failures → report to user with diagnostic info

Voice-Driven Sculpting Example

Loaded on demand — See mcp-tools-spec.md for the full voice-driven sculpting example (desk scene with 8-step agent pipeline).

Code-First Rendering Philosophy (v0.9.0)

Design inspiration: img2threejs outputs pure Three.js code (not GLB/OBJ/PLY), making every model fully editable, version-controllable, and lightweight. We adopt this philosophy for 3DGS scene export.

Traditional 3DGS Export vs Code-First Export

| Aspect | Traditional (.ply/.splat) | Code-First (

阅读完整来源 ↗