ClipMatrix Creatures by Nikolay Jetchev — 500 AI artworks

ClipMatrix CreaturesBy Nikolay Jetchev · 500 works · released 2022-03-14ClipMatrix Creatures is a collection of text-generated 3D art. Each of its 500 tokens is a short video of an animated, fully textured creature sculpted and painted by an algorithm that took only words as direction. The works come from ClipMatrix, a method Nikolay Jetchev published in September 2021: it starts from a rigged parametric human body mesh (SMPL) and, using a differentiable renderer, optimizes the body's shape, pose, per-vertex deformations, and a high-resolution texture map until rendered views match a text prompt in OpenAI CLIP's embedding space. Because the mesh is articulated and is optimized across random camera angles and poses, the creatures hold up in motion and from every side, which the videos show off through moving limbs and orbiting cameras. The name pairs CLIP with 'matrix', from the Latin for 'mother', to suggest the breeding of new digital creatures.

500 animated 3D works, each shown as a short video

Made with ClipMatrix (arXiv:2109.12922, Sept 2021), which optimizes a rigged SMPL body mesh, its deformations, and a 1024x1024 texture against CLIP text embeddings

Rendered with PyTorch3D; one example creature took about 600 optimization steps (around 20 minutes on an Nvidia P100)

Jetchev co-authored Spatial GAN and PSGAN texture-synthesis research at Zalando Research

Pieces: #0 … #499. Data: https://braindrops.art/api/v1/collections/clipmatrix-creatures