Most digital images—and most image authoring software—represent images as pixels that encode only the image’s color. These images do not describe material properties such as how metallic, smooth, or high each pixel is. These properties determine how a surface interacts with light and are important expressive aspects of traditional media. Procedurally generated images without these qualities often appear to be “flat” and “lifeless” unless great effort is expended in creating detail and subtlety.

To address this, we have begun building Smudge, an experimental library for creating procedurally generated images with these extended material properties. These images can be used in physically based rendering (PBR) pipelines to texture surfaces in realtime 3D graphics, games, and VR experiences. PBR images are especially effective in these contexts because they respond naturally to changes in lighting and point of view.

RGB: Plain color image on the sides of a cube. Rendered in Unity.
RGB+: Image with additional metallic, smoothness, and height channels.

Using code to procedurally generate images is a powerful technique for exploring aesthetics and generating complex forms, but creating rich, life-like images with code is complex and often difficult. Because PBR rendering allows us to simulate how light reacts with our images, many types of surfaces and textures are much easier to express. This can benefit images in 2D contexts as well, bringing additional richness and depth.

RGB: A procedurally generated 2D image.
The same image with additional channels for smoothness and height. Rendered in Unity.

The images above show how 2D images might benefit from PBR post processing. Most notably, highlights and shadows reveal some thickness in the lines of “paint”. You can also see some color variation picked up from the environmental lighting.

Project Goals

Our initial goal is to create a minimal but complete javascript library for procedurally generating images with a full set PBR properties.

  • Albedo The albedo color is the basic base color of the material.
  • Transparency Transparency controls how much light passes through the material.
  • Metallic Metallic surfaces show their albedo color less and reflect the environment more.
  • Smoothness Highly smooth surfaces are shiny, while unsmooth surfaces are dull.
  • Height Height data allows for simulating lighting on bumps and dents.
  • Emission Emissive materials are sources of light themselves. They glow in the dark.

We are planning the following core features:

  • Common drawing methods for drawing rectangles, ellipses, and lines
  • High dynamic range rendering
  • Oversampling / anti-aliasing
  • Per-channel control of blending
  • Support for drawing to and compositing layers
  • Image export as a texture set compatible with the Unity standard PBR shader
  • API and conceptual documentation

We also plan to explore additional features useful when procedurally generating images:

  • A web-based image viewer with individual channel and combined PBR views
  • Drawing with textured fills and edges
  • Utility functions for working with random values
  • Support for tiled drawing