Skip to content

Object 3D Reconstruction

Repository: https://github.com/OlafenwaMoses/vizion3D

Category: Reconstruction Experimental: No

Object3DReconstruction takes a close-range image of one object and produces a cleaned, uniformly gray 3D mesh plus a uniformly gray point cloud sampled from that mesh surface.

The task is intended for object-centric images: product shots, cropped objects, or a single object occupying most of the frame. For broader scene images, use SceneComponents3DReconstruction.

Object reconstruction sample input crop
Sample coconut-water object input used for object reconstruction. The saved image is 1536x2048 and is capped to the default 1080px longest side inside the task.
Generated gray object mesh: 53,965 vertices and 107,926 faces
Point cloud sampled from the generated mesh surface: 200,000 points

What It Does

The object pipeline is:

  1. Load the image and cap the longest side to max_input_dimension.
  2. Isolate the foreground object from the background.
  3. Normalize the foreground into a centered square conditioning image.
  4. Generate a 3D mesh from the conditioned object image.
  5. Clean the mesh, force a uniform gray material, and sample a gray point cloud.

Background removal always runs. There is no option to disable it for this task. The output is intentionally gray so downstream workflows get geometry without texture baking or texture-job side effects.

The sample above uses production-default mesh settings: marching_cubes_resolution=256, point_count=200000, and smoothing_iterations=5.

Install and Runtime Assets

Install vizion3d with the hardware extra for your machine, for example vizion3d[cpu], vizion3d[mps], vizion3d[cuda], or vizion3d[amd]. Those hardware extras include the reconstruction runtime dependencies. See the Hardware Acceleration page for the supported install commands.

The task resolves the reconstruction runtime asset bundle from:

  1. the explicit model_bundle command field;
  2. VIZION3D_RECONSTRUCTION_MODEL_BUNDLE;
  3. ~/.cache/vizion3d/models;
  4. the repository root;
  5. the default essentials-v1 GitHub release asset URL.

If no local bundle exists, the default release asset is downloaded and cached before extraction. The bundle contains the runtime assets needed for foreground isolation, mesh generation, and optional accelerated execution.

Direct download: scene-components-3d-models.zip

Python Usage

from vizion3d.reconstruction import (
    Object3DReconstruction,
    Object3DReconstructionCommand,
    Object3DReconstructionConfig,
)

command = Object3DReconstructionCommand(
    image_input="object.png",
    model_bundle="/path/to/reconstruction-assets.zip",
    advanced_config=Object3DReconstructionConfig(
        max_input_dimension=1080,
        marching_cubes_resolution=256,
        point_count=200_000,
        device="auto",
    ),
)

result = Object3DReconstruction().run(command)

mesh = result.mesh
point_cloud = result.point_cloud
print(result.vertex_count, result.face_count, result.point_count)

Command Parameters

Parameter Type Required Default Description
image_input str \| bytes Yes — Object image path or raw image bytes.
model_bundle str \| None No Auto-resolved Path to the reconstruction runtime asset bundle.
advanced_config Object3DReconstructionConfig No Defaults Mesh, point-cloud, image-size, and device settings.

Config

Field Default Description
max_input_dimension 1080 Caps the longest source-image side before background removal. Values above 1080 are rejected.
marching_cubes_resolution 256 Mesh extraction resolution. Higher values can preserve more detail and cost more memory/time.
density_threshold 25.0 Mesh extraction density threshold.
point_count 200000 Number of surface points sampled from the cleaned mesh.
device "auto" Device preference. auto prefers available acceleration and falls back to CPU by stage.
foreground_ratio 0.82 Foreground normalization ratio after background removal.
smoothing_iterations 5 Mesh smoothing passes after reconstruction.
min_component_area_ratio 0.02 Removes very small disconnected mesh components.

Result Fields

Field Type Description
mesh trimesh.Trimesh Cleaned uniformly gray mesh.
point_cloud open3d.geometry.PointCloud Uniformly gray point cloud sampled from the mesh surface.
backend_used str Resolved runtime asset extraction directory.
vertex_count int Mesh vertex count.
face_count int Mesh face count.
point_count int Sampled point-cloud size.

REST and gRPC Jobs

The REST and gRPC server APIs run this task as a background job because mesh generation can take longer than a normal request timeout.

REST:

curl -X POST http://localhost:8000/reconstruction/object-3d-reconstruction \
  -F "image=@object.png" \
  -F "model_bundle=/path/to/reconstruction-assets.zip" \
  -F "device=auto"

The response is 201 Created:

{
  "job_id": "9f0a...",
  "status": "queued",
  "expires_at": "2026-06-16T21:00:00+00:00",
  "max_result_reads": 10,
  "result_reads_remaining": 10
}

Poll with:

curl http://localhost:8000/reconstruction/object-3d-reconstruction/9f0a...

While queued or running, polling returns 202. When complete, it returns 200 with:

{
  "status": "succeeded",
  "result": {
    "mesh_ply": "<base64 PLY>",
    "point_cloud_ply": "<base64 PLY>",
    "vertex_count": 53965,
    "face_count": 107926,
    "point_count": 200000
  }
}

gRPC:

  1. RunObject3DReconstruction submits the job and returns ReconstructionJobSubmission.
  2. GetObject3DReconstructionResult polls by job_id and returns Object3DReconstructionJobResponse.

Completed results are stored in a small temp job folder on the server machine. Set VIZION3D_JOB_DIR to control that folder. A result can be retrieved up to 10 times and expires after 24 hours.

Device

Object3DReconstructionConfig(device="auto") propagates the selected device to the foreground-isolation and mesh-generation stages where the installed runtime supports acceleration. auto prefers available acceleration and falls back to CPU. If an accelerated stage fails, the task retries that stage on CPU.

Input Resolution

The task limits the longest input-image dimension to 1080 pixels before background removal. The resize preserves aspect ratio. This avoids spending memory and inference time on source pixels that will be normalized into the fixed-size conditioning image. The config may lower this limit, but values above 1080 are rejected.

Practical Notes

  • Use object-centric images. Small or cluttered objects in a full room image are better handled by SceneComponents3DReconstruction.
  • The task estimates geometry from one image; hidden backsides are inferred by the reconstruction stage and should not be treated as measured ground truth.
  • Output colors are uniform gray by design. Texture generation is not part of this task.