跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
浏览器与自动化

3dgs-articulated-reasoner

3DGS Articulated Object Reasoning & Digital Twin Agent. Reason about articulated objects in 3DGS scenes: extract part structure, infer kinematic constraints, generate interactive digital twins. Use when: articulated object, digital twin, part structure, kinematic chain, URDF, interactive 3DGS, part-aware rendering, articulated manipulation, joint estimation, part segmentation 3DGS, ArtiSplat, ArtiTwinSplat, articulated reconstruction.

jaccen · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add jaccen/Awesome-Gaussian-Skills --skill 3dgs-articulated-reasoner

技能说明


name: 3dgs-articulated-reasoner description: "3DGS Articulated Object Reasoning & Digital Twin Agent. Reason about articulated objects in 3DGS scenes: extract part structure, infer kinematic constraints, generate interactive digital twins. Use when: articulated object, digital twin, part structure, kinematic chain, URDF, interactive 3DGS, part-aware rendering, articulated manipulation, joint estimation, part segmentation 3DGS, ArtiSplat, ArtiTwinSplat, articulated reconstruction." license: Apache-2.0 user-invocable: true metadata: version: "0.2.0" author: jaccen tags: ["3dgs", "articulated-object", "digital-twin", "kinematics", "part-aware", "urdf", "interactive-3d"] arguments: [task] when_to_use: - "Reason about articulated objects in a 3DGS scene" - "Build an interactive digital twin from Gaussian splats" - "Extract part structure or segment movable/fixed Gaussians" - "Infer kinematic chains or generate URDF from 3DGS" - "Part-aware rendering or articulated manipulation" - "Estimate joint types and axes from multi-view observation" - "铰接式物体 / 数字孪生 / 部件分割 / 运动链推断 / URDF生成"

3DGS Articulated Reasoner

Reason about articulated objects within 3DGS scenes: extract part structure, infer kinematics, build interactive digital twins.

Capabilities

  1. Part Structure Extraction: Given a 3DGS scene containing articulated objects (furniture, vehicles, tools), identify part boundaries and segment Gaussians into movable/ fixed groups.
  2. Kinematic Inference: Estimate joint types (revolute, prismatic, fixed) and axes from multi-view observation or user specification. Output URDF-compatible kinematic tree.
  3. Part-Aware Compositing: Apply part-aware alpha-compositing that penalizes inter-part penetration at boundaries (inspired by Innovation I-01 in the knowledge base).
  4. Digital Twin Generation: Produce an interactable digital twin where Agent commands (open, rotate, slide) drive real-time Gaussian deformation.
  5. Simulation-Ready Export: Export articulated 3DGS scenes to simulation frameworks (MuJoCo, Isaac Sim) with preserved visual fidelity.

Instructions

Step 1: Scene Analysis

When the user provides a 3DGS scene or asks about articulated objects:

  1. Check references/3dgs-methods-overview.md for relevant methods (ArtiTwinSplat, ArtiSplat, ULF-Loc).
  2. Identify the object category and expected articulation pattern from the knowledge base.
  3. Determine whether part segmentation is available or needs estimation.

Key references:

  • ArtiTwinSplat (arXiv 2026): Articulated digital twin from 3DGS
  • ULF-Loc (CVPR 2026): Exposed feature bias at part boundaries
  • ArtiSplat: Differentiable physics + rendering for articulated objects

Step 2: Part Segmentation

If the user needs part segmentation:

Input: 3DGS point cloud / rendered views
Method options:
  a) Semantic segmentation (LangSplat/Feature 3DGS) → part labels
  b) Motion-based segmentation (multi-frame observation) → movable vs fixed
  c) User-specified masks (manual annotation on rendered views)
Output: Per-Gaussian part label {0, 1, ..., K}

Validation criteria:

  • Part boundaries should align with geometric discontinuities (normal/depth edges)
  • Each part should be spatially contiguous (no isolated Gaussians)
  • Fixed parts should be the largest spatially connected component

Step 3: Kinematic Estimation

For each part pair with relative motion:

Joint Type Decision Tree:
  IF relative motion is rotational around a fixed axis
    → Revolute joint (hinge)
    → Estimate: axis direction, axis point, rotation limits [θ_min, θ_max]
  IF relative motion is translational along a fixed axis
    → Prismatic joint (slider)
    → Estimate: slide axis, slide limits [d_min, d_max]
  IF no relative motion observed
    → Fixed joint (weld)

Output format: URDF XML string with visual (mesh from TSDF fusion) and collision (simplified convex hull) links.

Step 4: Part-Aware Rendering

When rendering articulated objects for visualization or editing:

# Part-aware alpha-compositing (Innovation I-01)
C(theta) = sum_i T_i * alpha_i * omega_{p(i)}(theta) * c_i

# where omega penalizes penetration:
#   omega = 1.0                    if no penetration
#   omega = exp(-lambda * pen_i)   if penetration detected
# pen_i = SDF_violation at Gaussian center
# lambda = 10.0 (default)

This prevents color bleeding at part boundaries that standard alpha-compositing produces.

Step 5: Agent-Driven Interaction

Support natural-language commands for digital twin interaction:

CommandActionImplementation
"Open the drawer"Translate drawer part along slide axisApply d_offset to prismatic joint
"Rotate the door 30 degrees"Rotate door part around hingeApply θ_offset to revolute joint
"Show the internal structure"Hide external shell partsSet alpha=0 for shell Gaussians
"Reset pose"Return all parts to initial configurationZero all joint offsets

Deformation pipeline:

  1. Load URDF + initial Gaussian positions
  2. Apply forward kinematics: new_pos = FK(joint_offsets)
  3. Transform Gaussian positions: G_new.mu = R * (G.mu - joint_origin) + joint_origin + t
  4. Re-render with part-aware compositing

Step 6: Export & Integration

Export options:

  • URDF + Mesh: For MuJoCo / Isaac Sim (convex decomposition via VHACD)
  • GLTF with morph targets: For web viewers (Three.js)
  • 3DGS-Compatible: Gaussian attributes + joint parametrization for real-time rendering

Reference Data

Articulated Object Methods in Knowledge Base

MethodVenueCategoryKey Contribution
ArtiTwinSplatarXiv 2026Articulated/Digital TwinDigital twin via 3DGS
ArtiSplat2026ArticulatedDifferentiable physics + rendering
ULF-LocCVPR 2026SLAM/ArticulatedFeature bias at part boundaries
Articul
阅读完整来源 ↗