Constructing Printable Surfaces with View-Dependent Appearance

In short

We make 3D-printed surfaces that show a different picture depending on where you look from. The surface is a grid of coloured bars of different heights. From any direction, taller bars hide parts of the bars behind them, so each viewer sees a different mix of colours.

Such a surface is hard to design by hand, so we optimize it. We use a differentiable renderer for coloured heightfields. It reduces each camera ray to a 1D problem with a closed-form colour, so gradients are cheap to compute. The optimizer adjusts heights and colours until each view matches its target image. The result is printed directly on a full-colour PolyJet printer, with no lenses or coatings.

Section 3

A differentiable renderer for heightfields

A heightfield is a matrix of bar heights H and a matrix of bar colours C. It is viewed by orthographic cameras, each given by a viewing direction. This is a good model because the surfaces are shallow. For every camera i there is a desired image D_i, and the renderer produces the actual image I_i.

Each pixel is a ray. In the code, the pixels lie on a grid over the surface, at its mean height, and each ray is traced back along the camera direction.

Slicing the heightfield

A ray can only hit the bars directly below its path. So we project the ray onto the ground plane, list the bars it crosses in order, and measure how far it travels through each one. This gives a 2D slice: a row of bars with heights h_i, colours c_i and widths w_i, and a 2D ray that enters it at height o_y.

# traced back from z = 1, as in grad_descent.py
origin = pixel - 20 * cam
ids, widths, ray = rendering.get_2D_ray_info(origin, cam, 8)

print("bars crossed (row, col):", [(int(a), int(b)) for a, b in ids])
print("widths:", np.round(widths, 3))
print("o_y:", round(float(ray.origin.y), 3))

Rendering a single pixel in closed form

For each bar, trace a line back from its far top corner, parallel to the ray, to the line x = 0. It meets that line at height

y_i = h_i − (d_y / d_x) · W_i,    W_i = w₀ + … + w_i (1)

where (d_x, d_y) is the direction of the 2D ray, which points down. A ray entering at height o_y passes over bar i when o_y > y_i. A tall bar can hide the bars behind it, so we take a running maximum y′_i = max(y₀, …, y_i). With the step function g, the colour of the ray is a telescoping sum:

color(p) = c₀ + Σ_i g(o_y − y′_i) · (c_i+1 − c_i)

Each bar the ray passes over moves the colour on to the next bar. The sum ends at the first bar that blocks the ray, or at the last bar if none does.

width_sums = rendering.get_width_sums(widths)
thetas = rendering.get_backtraced_boundaries(H, ray, width_sums, ids)  # Eq. 1
mono = rendering.monotonize_boundary_points(thetas)  # running maximum
color = rendering.render_ray_given_boundaries(ray, mono, COL, ids, heaviside_fn="tan")

print("y_i      :", np.round(thetas.numpy(), 3))
print("monotonic:", np.round(mono.numpy(), 3))
print("o_y      :", round(float(ray.origin.y), 3))
print("pixel colour (RGB):", np.round(color.numpy(), 3))

Rendering all pixels, and repeating units

A camera image is made of many such rays. The slices depend only on the camera and the bar grid, not on the heights, so the code computes them once (process_rays) and reuses them at every step. For repeated patterns, the heights of a smaller unit are optimized and tiled across the surface. In the code, the heights repeat 2×2, while every bar has its own colour.

n = 16
origins = [np.array([(i + .5) / n, (k + .5) / n, 1.0]) - 20 * cam for i in range(n) for k in range(n)]
packs = [rendering.process_rays(origins, cam, 8)]
image = rendering.render_camera(H, COL, packs, 1, num_repeats=1, heaviside_fn="tan")[0]

Section 4

Objective function

We minimize the pixel-wise mean squared error between the rendered and desired images of all cameras. With m×m images:

min_H,C   1/(m² |CAM|) · Σ_i Σ_x,y ( D_i(x,y) − forward(cam_i, H, C)(x,y) )²

The loss values on this page are this error, per view.

Smooth Heaviside approximations

The step function g has zero slope almost everywhere, so the heights would get no gradient. We replace it with a smooth approximation λ(x, k), where k sets the amount of smoothing. In our tests, tanh worked best, so it is the default. The forward pass still uses the exact step, so the rendered image stays exact. Only the backward pass uses the smooth version.

Heaviside approximations (Table 2 of the paper)

approximation	λ(x, k)
circle	½ + ½ · x / (x² + k²)½
circle distance	Bern(½ + ½ · x / (x² + k²)½)
erfc	½ + ½ erf(kx)
tanh	½ + ½ tanh(kx)
log	Bern(½ + ½ tanh(kx))

x = torch.tensor([-1.0, -0.1, 0.0, 0.1, 1.0])
for name, fn in [("tanh", happrox.heavi_tanh_fn), ("erfc", happrox.heavi_erfc_fn), ("circle", happrox.heavi_circ_fn)]:
    print(f"{name:6s} forward:", fn(x, 0.1))

Regularization

Two extra terms keep the surface printable. A barrier loss keeps every height between h_min and h_max. A neighbour loss penalizes height differences between adjacent bars, which avoids thin spikes and deep gaps:

barrier = −Σ_i log(h_max − h_i) − Σ_i log(h_i − h_min),    neighbour = Σ_i Σ_{j ∈ N(i)} |h_i − h_j|

Regularization costs some accuracy (Table 1), but it is needed for surfaces that survive printing. The original code (v1.0) uses h_min = 0.1, h_max = 1.5, and weights 0.02 (barrier) and 0.5 (neighbour). The updated code (v2.0) uses h_max = 1.0. Its neighbour loss also compares bars across the edges of the repeating unit, which avoids spikes along the seams.

print("barrier  :", round(float(opt.barrier_fn(H, min_val=0.1, max_val=1.5)), 4))
print("neighbour:", round(float(opt.neighbor_fn(H)), 4), "  v2.0:", round(float(opt.neighbor_fn_wrap(H)), 4))
spiky = H.clone()
spiky[3, 4] = 1.45  # one tall spike
print("with a spike:", round(float(opt.barrier_fn(spiky, 0.1, 1.5)), 4), round(float(opt.neighbor_fn(spiky)), 4), round(float(opt.neighbor_fn_wrap(spiky)), 4))

Section 5

Optimization

The optimizer is Adam on the MSE loss, with three additions (Table 3 of the paper):

• Coarse to fine. Every 100 steps, each bar is split into 2×2 smaller bars. The surface starts with 8×8 bars and ends with 32×32. This improves the result.
• Alternating block coordinate descent. Heights and colours are updated in turns, not together. This works much better.
• Simulated annealing. Random changes to the heights and colours at the start help avoid poor local optima. This step is optional. Without it, the pattern is more uniform.

After optimizing, the target images are projected back onto the surface from their cameras. This adds detail finer than the bars. Each point is coloured from the cameras that see it, weighted by how directly they see it. To try it, press “2 · Project” in the design tool. This also trims the surface to a thin base, and the download contains the printable mesh.

The paper also compares different starting surfaces (flat, walls, cross, random). The code starts from a flat surface.

Sections 6–8

Fabrication and results

For the paper, we turned the heights and colours into a coloured mesh in Blender and printed it on a Stratasys J55 PolyJet printer. The updated code builds the mesh in Python. Test prints showed that 300 dpi prints cleanly, which sets the smallest bar width. For surfaces under 1 cm wide, a clear top layer protects the thin bars. This layer bends light, which lowers the effective viewing angle.

Optimization results

With the PyTorch code, an optimization takes 30 minutes to 2 hours. Five views work less well than four. Two views work well when they differ by more than about 40° in elevation or 60° in azimuth.

Limitations

The method works for up to five views in simulation and four in print. Unlike lenticular prints, quality depends a lot on the angles between the cameras. Lighting and self-shadowing are not modelled. Future work could correct for them by adjusting the colours. On the other hand, the method reaches high resolution, can share colours between views, and uses bright colours, without the dark walls or lens arrays of earlier methods.

Citation

@inproceedings{Perroni-Scharf:2023:CPS,
  author    = "Maxine Perroni-Scharf and Szymon Rusinkiewicz",
  title     = "Constructing Printable Surfaces with View-Dependent Appearance",
  booktitle = "Proc. SIGGRAPH",
  year      = "2023",
  month     = aug
}

We would like to thank Baffour Osei and the Princeton Robotics Lab for their assistance during this project.