---

Useful Links

• https://learn.foundry.com/nuke/content/reference_guide/transform_nodes/cornerpin2d.html
• https://learn.foundry.com/nuke/developers/17.1/pythondevguide/_autosummary/nuke.nukemath.Matrix4.html
• https://en.wikipedia.org/wiki/Homography_(computer_vision)
• https://en.wikipedia.org/wiki/Homogeneous_coordinates
• https://docs.opencv.org/4.x/d9/dab/tutorial_homography.html
• https://gist.github.com/julik/1164618

---

Index

• 01. Affine vs projective
• 02. The divide by w
• 03. Why exactly four corners
• 04. CornerPin2D in Nuke
• Homography Lab (try it here)
• 05. Homography in Python
• 06. Where it works and where it doesn't
• Nodes to paste in Nuke

---

01. Affine vs projective

The 2D transform matrix of the Transform Matrix page always ends with the row 0 0 1. A homography uses that row too:

The two new numbers in the last row, h20 and h21, are the perspective: they make one side of the image smaller than the other, like a plane turning away from the camera.

---

02. The divide by w

To move a pixel (x, y) you multiply the matrix by (x, y, 1), like before. But now the third result, w, isn't always 1, so you divide by it:

x' = (h00·x + h01·y + h02) / w
y' = (h10·x + h11·y + h12) / w
w  = h20·x + h21·y + h22

Pixels where w is big are pushed together (the far side of the screen), pixels where w is small are spread apart (the near side). It's the same division by the distance as the camera projection in the 3D Matrices page: a homography is what you get when you look at a flat card through a camera. And since everything is divided by w, the matrix can be multiplied by any number without changing the result: that's why it's normally scaled so that h22 = 1.

---

03. Why exactly four corners

With h22 fixed to 1, a homography has 8 unknown numbers. Every corner you pin gives two equations (one for x, one for y). Four corners, eight equations: exactly one solution. With three corners you'd only have an affine transform, with five you'd have too many equations for a single plane. That's why a CornerPin has four pins, and why a planar tracker only needs to follow a flat surface to give you all of them.

---

04. CornerPin2D in Nuke

- to1 to4: where the corners go. The order is always bottom-left, bottom-right, top-right, top-left (anticlockwise from the bottom-left).
- from1 to from4 (From tab): which four points of the input go there. By default the corners of the format. Change them to pin a part of the image instead of the whole frame.
- invert: uses the inverse matrix, from the screen back to a flat rectangle. That's how you flatten a surface to paint on it.
- extra matrix (transform_matrix): a 4x4 matrix multiplied on top of the pins, where you can paste a matrix from a Roto, a planar track or the Transform Lab.

Flatten, paint, put back. Track the four corners, CornerPin with invert on to flatten the surface, paint or comp on the flat image, then a second CornerPin with the same pins and invert off to put it back. The two matrices cancel out everywhere you didn't touch. The nodes are at the bottom of the page.

---

Homography Lab

Drag the four corners. The frame is 1920x1080 with Y going up, like in Nuke. You get the 3x3 homography, the CornerPin2D node with the same pins, and the inverse matrix for the BlinkScript warp of the BlinkScript Matrices page.

---

05. Homography in Python

Eight equations, eight unknowns: a small linear system. These few lines of plain Python find the matrix from four point pairs and apply it to any point, for example to know where a 2D track lands on a pinned screen:

import nuke

def homography(src, dst):
    """3x3 matrix (9 values, row by row, h22 = 1) that maps 4 src points onto 4 dst points."""
    a, b = [], []
    for (x, y), (u, v) in zip(src, dst):
        a.append([x, y, 1, 0, 0, 0, -u * x, -u * y]); b.append(u)
        a.append([0, 0, 0, x, y, 1, -v * x, -v * y]); b.append(v)
    n = 8                                           # Gaussian elimination
    for c in range(n):
        p = max(range(c, n), key=lambda r: abs(a[r][c]))
        a[c], a[p], b[c], b[p] = a[p], a[c], b[p], b[c]
        for r in range(c + 1, n):
            f = a[r][c] / a[c][c]
            a[r] = [a[r][k] - f * a[c][k] for k in range(n)]
            b[r] -= f * b[c]
    h = [0.0] * n
    for r in range(n - 1, -1, -1):
        h[r] = (b[r] - sum(a[r][k] * h[k] for k in range(r + 1, n))) / a[r][r]
    return h + [1.0]

def apply_homography(h, x, y):
    w = h[6] * x + h[7] * y + h[8]
    return ((h[0] * x + h[1] * y + h[2]) / w, (h[3] * x + h[4] * y + h[5]) / w)

# the pins of a CornerPin2D called CornerPin1, at the current frame
cp = nuke.toNode("CornerPin1")
src = [cp["from%d" % i].value() for i in range(1, 5)]
dst = [cp["to%d" % i].value() for i in range(1, 5)]
h = homography(src, dst)
print(apply_homography(h, 960, 540))       # where the centre of the input ends up

Nuke's own nuke.math.Matrix4 also has mapUnitSquareToQuad() and mapQuadToUnitSquare(), which build the same kind of matrix between the unit square and four corners.

---

06. Where it works and where it doesn't

A homography is exact for flat surfaces (screens, walls, floors, signs, paper) seen by any camera, and for any scene when the camera only rotates without moving (a nodal pan, a panorama). It is not enough when:

- the surface isn't flat: a curved screen or a face needs a mesh warp, a SmartVector or a 3D projection;
- the lens is distorted: straight lines are curved, and no 3x3 matrix can bend them. Undistort first, pin, then redistort (see my STMap tutorial);
- the camera moves in front of a deep scene: there's parallax, and you need a real 3D camera (3D Matrices).

---

Nodes to paste in Nuke

Click a box, copy it (Ctrl+C / Cmd+C) and paste it in Nuke's Node Graph (Ctrl+V / Cmd+V).

---

The matrix series

If you are in love with Matrices :) have a look at the other pages:
Convolution Matrix Transform Matrix ColorMatrix Convolve & Deconvolution 3D Matrices Morphology BlinkScript Matrices