House of the Dragon

The brief

A lot of scenes in House of the Dragon are built around a creature that isn't there on the day. How big is the dragon in this frame? Where exactly is it fighting? Is this the right lens for the shot?

Those are questions the DoP, the director and the VFX supervisor are asking each other while the clock runs, and until the shot comes back from post they are answering them from memory. Real-time visualisation of the CG asset turns that guesswork into something you can look at.

Why native

The application is fully native: ARKit 6 for camera and motion tracking, RealityKit for rendering. Native means every Apple API the moment it ships, where Unreal Engine still only supports ARKit 4. On a production that needs the newest tracking the hardware can do, that gap decides whether the tool works on the day.

I built it in the Pixomondo Innovation Lab, and got there the long way round: the first version ran on Unreal, and moving it to native is its own story.

The pipeline

Getting an animated dragon from the previz department onto an iPad runs through four stages:

1. Maya. The previz team builds the animated sequence.
2. Unreal Engine. The animation comes across as FBX. I play the animation back in Sequencer, where I can see the camera position, the dragons and the set, the castle for instance, and add or change elements where the shot needs it. Then I export, either element by element or as a whole scene, in USD.
3. Reality Composer Pro. The assets come back in, and I check that animation and placement survived the trip. Then out as USDZ.
4. The app. The USDZ is imported, and the dragon is ready to stand in the shot.