BlinkScript Matrices: float3x3 and float4x4 in Nuke

Matrices in Nuke, the series: Convolution Matrix · Transform Matrix · ColorMatrix · Convolve & Deconvolution · 3D Matrices · Morphology · CornerPin & Homography · BlinkScript Matrices

Every page of this series ended with a limit: the ColorMatrix has no offset and no alpha, a 3x3 can't move a position pass, the CornerPin only works with four pins. BlinkScript removes the limits. Blink, Nuke's language for image kernels, has two matrix types, float3x3 and float4x4, so you can write the exact matrix node you need, and run it on the GPU.

Useful Links

Index

01. The two matrix types

Blink has float3x3 and float4x4, both row-major: the values are written row by row, exactly like Nuke's matrix knobs and like every table of this series. You can create them with a list of values, read one element with [row][column], and use them as parameters, so they appear as knobs on the BlinkScript node and can be linked with expressions.

float3x3 m = {1.0f, 0.0f, 0.0f,
              0.0f, 1.0f, 0.0f,
              0.0f, 0.0f, 1.0f};
float a = m[0][1];          // row 0, column 1
m[1][1] = 2.0f;             // change one element

The matrix types are documented in the Blink guide of recent versions of Nuke (the Foundry pages linked above). Foundry also lists a known issue where the default values of matrix parameters are not applied: if your matrix knob starts at zero, type the identity in it once.

02. What you can do with them

- Operators: A + B, A - B, A * B (matrix by matrix, the order matters) and A * v (matrix by vector: float3x3 with float3, float4x4 with float4).
- Build them: setIdentity(), setTranslate(v), setScale(v), setRotate(angle) for 2D (3x3), setRotateX/Y/Z(angle) for 3D (4x4). Angles in radians.
- Combine them: translate(v), scale(v), rotate(angle), rotateX/Y/Z(angle) multiply a new transform on the left of the current matrix.
- Change them: transpose() and invert(), in place (make sure the matrix can be inverted).

Compute what doesn't change from pixel to pixel (an inverse, a combined matrix) once in init(), not in process(): it runs once per frame instead of millions of times.

03. How to use the kernels

1. Create a BlinkScript node (Other → BlinkScript). You need NukeX to edit and compile a kernel; once compiled and saved in the script, it also runs in Nuke.
2. Copy one of the kernels below into the Kernel Source and press Compile (Recompile in older versions).
3. The parameters appear in the BlinkScript tab. Set the channels as written in the comment of the kernel (most of these work on rgba).

04. ColorMatrix 4x4 with offset

The ColorMatrix page ended with the 4x5 matrices of Flash, SVG and Rainmeter: alpha and an offset column. Here they are in Nuke, in one node. Put the same numbers of a 4x5 guide in matrix (the first four columns) and offset (the fifth column, divided by 255 if the guide works in 8 bit).

// A 4x4 colour matrix with an offset: RGBA in, RGBA out.
// The same idea as the 4x5 matrices of Flash, SVG and Rainmeter.
kernel ColorMatrix4x4 : ImageComputationKernel<ePixelWise>
{
  Image<eRead, eAccessPoint, eEdgeClamped> src;
  Image<eWrite> dst;

param:
  float4x4 colorMatrix;   // row 1 builds red, row 2 green, row 3 blue, row 4 alpha
  float4 offset;          // added after the matrix

  void define() {
    defineParam(colorMatrix, "matrix", float4x4({1.0f, 0.0f, 0.0f, 0.0f,
                                                 0.0f, 1.0f, 0.0f, 0.0f,
                                                 0.0f, 0.0f, 1.0f, 0.0f,
                                                 0.0f, 0.0f, 0.0f, 1.0f}));
    defineParam(offset, "offset", float4(0.0f, 0.0f, 0.0f, 0.0f));
  }

  void process() {
    float4 c = float4(src(0), src(1), src(2), src(3));
    dst() = colorMatrix * c + offset;
  }
};

05. Position pass to camera space

On the ColorMatrix page we rotated the normals with a 3x3 matrix. Positions also need the translation, so they need a 4x4: this is the kernel. Link camera_world_matrix to your camera with the few lines of Python below.

// Position pass (P, world space) into camera space, with the inverse of the camera's world matrix.
// The rotation AND the translation: what a 3x3 ColorMatrix can't do.
kernel PositionWorldToCamera : ImageComputationKernel<ePixelWise>
{
  Image<eRead, eAccessPoint, eEdgeClamped> src;   // the P pass in rgb
  Image<eWrite> dst;

param:
  float4x4 cameraWorld;   // link it to Camera1.world_matrix

  void define() {
    defineParam(cameraWorld, "camera_world_matrix", float4x4({1.0f, 0.0f, 0.0f, 0.0f,
                                                              0.0f, 1.0f, 0.0f, 0.0f,
                                                              0.0f, 0.0f, 1.0f, 0.0f,
                                                              0.0f, 0.0f, 0.0f, 1.0f}));
  }

local:
  float4x4 worldToCamera;

  void init() {
    worldToCamera = cameraWorld;
    worldToCamera.invert();       // computed once per frame, not once per pixel
  }

  void process() {
    float4 p = worldToCamera * float4(src(0), src(1), src(2), 1.0f);
    dst() = float4(p.x, p.y, p.z, src(3));
  }
};

import nuke

# Link the matrix parameter of a BlinkScript to a camera's world matrix, element by element.
blink = nuke.selectedNode()           # the BlinkScript with PositionWorldToCamera
camera = "Camera1"

# Blink parameters become knobs called <KernelName>_<paramName>: find the one we need
knob = [k for k in blink.knobs() if k.endswith("camera_world_matrix")][0]
for i in range(16):                   # both are row by row, so element i goes to element i
    blink[knob].setExpression("%s.world_matrix.%d" % (camera, i), i)

06. Homography warp

A CornerPin driven by a matrix: useful when the matrix comes from a planar tracker, a calibration or your own Python. For every output pixel it uses the inverse homography to find where to read in the source. Get the 9 values from the Homography Lab.

// A CornerPin made with a matrix: for every output pixel, find where it comes from.
// Paste the "inverse homography" of the Homography Lab in the 9 values.
kernel HomographyWarp : ImageComputationKernel<ePixelWise>
{
  Image<eRead, eAccessRandom, eEdgeConstant> src;
  Image<eWrite, eAccessPoint> dst;

param:
  float3x3 inverseH;   // output pixel -> source pixel

  void define() {
    defineParam(inverseH, "inverse_homography", float3x3({1.0f, 0.0f, 0.0f,
                                                          0.0f, 1.0f, 0.0f,
                                                          0.0f, 0.0f, 1.0f}));
  }

  void process(int2 pos) {
    float3 q = inverseH * float3(pos.x + 0.5f, pos.y + 0.5f, 1.0f);
    float w = q.z > 0.000001f ? q.z : 0.000001f;
    float x = q.x / w - 0.5f, y = q.y / w - 0.5f;
    if (q.z <= 0.0f) { x = -100000.0f; y = -100000.0f; }   // behind the "camera": black
    dst() = bilinear(src, x, y);
  }
};

07. The Matrix node in Blink

The 3x3 convolution of the Convolution Matrix page, with the kernel as a float3x3. A good starting point to write bigger or animated kernels. Copy any kernel of that page into kernel.

// The Matrix node in Blink: a 3x3 convolution with the kernel as a float3x3 parameter.
kernel Convolve3x3 : ImageComputationKernel<eComponentWise>
{
  Image<eRead, eAccessRanged2D, eEdgeClamped> src;
  Image<eWrite> dst;

param:
  float3x3 weights;   // row 0 is the top of the kernel, like on the Convolution Matrix page
  bool normalize;

  void define() {
    defineParam(weights, "kernel", float3x3({0.0f, -1.0f, 0.0f,
                                             -1.0f, 5.0f, -1.0f,
                                             0.0f, -1.0f, 0.0f}));
    defineParam(normalize, "normalize", true);
  }

  void init() {
    src.setRange(-1, -1, 1, 1);
  }

  void process() {
    float sum = 0.0f, total = 0.0f;
    for (int j = -1; j <= 1; j++) {
      for (int i = -1; i <= 1; i++) {
        float w = weights[1 - j][1 + i];   // j goes up in Nuke, the rows go down
        sum += w * src(i, j);
        total += w;
      }
    }
    dst() = (normalize && fabs(total) > 0.000001f) ? sum / total : sum;
  }
};

Matrix Calculator

Paste 9 or 16 numbers (from a Nuke knob, a Lab of this series or a kernel), then invert or transpose. The result is written ready for Blink, ready for a Nuke knob, and with its determinant: 0 means the matrix can't be inverted.

Matrix (9 or 16 values, row by row)

Blink Nuke knob (Paste Absolute)

The matrix series

If you are in love with Matrices :) have a look at the other pages:
Convolution Matrix Transform Matrix ColorMatrix Convolve & Deconvolution 3D Matrices Morphology CornerPin & Homography