How Disney’s Star Wars robots decide which visitor to look at
Three Anzellan droidsmiths use cameras and microphones to follow a conversation. Disney records the lines in advance and lets the figures choose when to deliver them.
Three small Star Wars droidsmiths share a workbench, and the next person to speak can change which way they look. Disney Imagineering and Amazon Web Services have revealed how their Anzellan robots choose whom to acknowledge, when to move and which recorded line to play. An earlier version appeared at D23. The next announced chance to meet the trio is at the AWS re:Invent conference in Las Vegas, November 30 to December 4. Disney’s announcement
Vilo Vele, Kado Bre’a and Jibu Bre’a are Audio-Animatronics, Disney’s name for physical figures that combine sound and mechanical performance. On a ride with a fixed sequence, an engineer can time a character’s glance to the instant a boat rounds a corner. At the Anzellans’ bench, the visitor supplies the cue. Two friends can approach together, and the one standing farther from a particular figure may be the one addressing it.
A glance has a shorter deadline than a sentence
The figures have cameras, microphones and other sensors. In an interview with TechRadar, Imagineering’s Kyle Laughlin and AWS executive Samira Panah Bakhtiar describe a system that can pick out who is speaking in a group. A figure can turn towards that person while the others respond to what is happening at the bench. The characters share a platform with Disney’s mobile robots, including Olaf, although the three Anzellans stay together in their set.
That difference puts the engineering effort somewhere unexpected. A stationary figure has no route to plan across the floor. It does, however, have to convey that it heard you. A head turn a beat too late, or in the direction of the friend who spoke last, can make a perfectly good line sound as if it belongs to a different conversation.
The actors have already said their lines
All the dialogue is written by Imagineers and recorded in advance. AWS describes how the system selects that audio and pairs it with animation as the encounter unfolds. A single recorded greeting can do several jobs depending on who delivers it, which visitor the speaker looks at and whether a second figure turns to listen. Those choices give the group room to vary a scene while preserving the words, voice and personality Disney gave each character.
It is a different test of a robot’s perception from picking the right component for a factory kit. An assembly worker can inspect the finished kit and spot the wrong part. At the workbench the outcome lasts a second: did the little mechanic acknowledge the person who actually waved?
Disney has announced the conference demonstration at Caesars Forum. A permanent park location and opening date are still undisclosed. In Las Vegas the most revealing place to stand may be beside the person asking the question. From there you can see whether all three droidsmiths know whose turn it is.