Disney Gallery: The Mandalorian (2020–…): Season 1, Episode 4 - Technology - full transcript

What I was learning to do from a little bit with Elf,

a little more with Zathura, and a lot more with Iron Man was,

if I could shoot things right, the CG's gonna look a lot better.

And there was this illusion, I think sometimes people still have it,

that you could just do anything later.

Like, "Oh, we'll fix it in post", or "we'll, you know, we'll paint that out",

or "we'll fix it in the DI", like...and you can do a lot of stuff,

but you can't do everything.

Things like interactive light, and things that you can't fake later effectively,

which is what people associate with effects

that don't look good in older movies is you're laying in something,



you're trying to create a shadow on something that's real after the fact,

never looks great,

or interactive light on people.

With Jungle Book, we started saying, "Okay, what could we do here"?

Let's create our sets digitally, a complete jungle,

but let's have video wall panels that we would use to create interactive light.

And so if elephants were passing, we'd have shadows casting on the kid.

But on the set, all you would see was a lot of blue and one little patch of set

that he would walk on with foliage growing out of it.

And if something cast a shadow on him, we would bring that in.

It should have gone quickly.

And it was convincing 'cause the interactive light was great,

the planning was great, execution was great.

The problem was, it would take forever,



because that blue screen, if you got to the set and said,

It looked like this in pre-vis,

but he looks better if we raise the camera a foot and a half and look down at him,

well, now you gotta move all that blue screen.

It took twice as long as it should've.

So I was thinking, "What's the dream stage to build"?

So all that, I sort of put that in my...

It was in the back of my head,

and I was hired to do The Lion King

after Jungle Book was a success.

So we pre-animated the whole film.

We created the whole environment, like we did on Jungle Book, virtually.

Except in this case, all of this consumer-facing VR equipment came out.

Jon?
Yep.

Suggestion to start up a little bit higher and then come down.

Okay.

Three, two, one. Go.

Ooh!

Really good.

Yeah, I chased him down.

We didn't need any of the motion builder,

any of the motion capture, expensive stuff that's really just developed for film.

But instead, using game engine technology.

And so now people were gonna be operating cameras within VR

where there's no latency, they're looking at the real thing.

And now we're at a place where I was taking on

doing a show for Disney+, The Mandalorian.

We had a whole team and all of ILM.

And I had worked with ILM before

and I knew a lot of the people there.

There was this sense of, "Hey, let's push this and see what innovations we can come up with".

And also, these video walls that we were using,

now these video walls were becoming bigger

and finer pixel pitch and less expensive.

It just got me thinking,

How do you take advantage of all these innovations with technology?

What'd we learn on Lion King and Jungle Book?

"How do you apply it to The Mandalorian"?

When I was involved from the very beginning, we got storyboards

of what the show was gonna look like. At least, episode one.

And the information I had at the time

was that Jon wanted to build upon the technology from The Lion King.

And at the time, I...

Who's this "Jon"? He sounds fascinating.

And totally unrealistic.

So I was invited down to see it.

And obviously seeing a film crew here effectively shooting in VR

was a big eye-opener.

Trying to figure out what it was that you were doing on The Lion King

and how to solve these environmental problems

on a Star Wars show with eight episodes,

I think the first art department meeting I was invited to,

I was doing it remotely.

And so I think Doug Chiang was down here with Andrew Jones, of course,

and I was watching it remotely from San Francisco.

And I think Andrew made a reference to "The Backlot".

Right.

And I think at that point, you wanted to effectively set the table and say,

"I want to be clear about what I'd like to try and do on this show".

And then we got in a room,

you know, you, Kathy, myself, you were in there,

and we had Rob Bredow, we had a lot of people from ILM,

but then we also brought in Kim Libreri,

who used to work at ILM and now is at Epic.

Yep.
And Unreal.

And there were other people, and Magnopus,

who I'd worked with on Jungle Book and I worked on Lion King with,

and there was a bit of a think tank

to figure out if we could pull this thing off.

The Mandalorian is the first production ever to use real-time rendering

and video wall in-camera set extensions and effects,

that was, again, necessity was the mother of invention

because we were trying to figure out how to do the production

here in the timeframe at the budget level,

but still get the whole look we're used to seeing.

Jon, one of the things it makes me think about,

it fascinates me to think back on Indian in the Cupboard that we did,

where we used a lot of Translite,

and blue screen, but a lot of Translite.

I'm just curious, what has been this evolution

of the use of screens, really, as a way

to inform what Mandalorian...

Back to King Kong.
Yeah.

There was front screen projection.

Yeah.
In 1933.

Yeah.
You have Fay Wray in a little...

And then you have all the stop motion.

And that was projected...
Yeah.

In-camera, right?
Into a little screen in the set.

Then eventually, the next big breakthrough,

that technique was being used, and then Kubrick did a lot of really

cool front screen projection in 2001.
Mmm-hmm.

That's the one I think of.

I'm skipping a few.
Dawn of Man sequence.

That's why you have the glint in the eyes of

whatever the big cat was.

The leopard.
The leopard?

I was like, "What a cool effect that is".

But that was just the front.

It's so bright and they have the reflective screen.

But I had always thought that Dawn of Man was shot in a real environment.

It wasn't. That was indoors.

It was just like what we were doing.

Yeah.

But because of the luminance levels and,

well, maybe you guys could speak better to that.

Why did that Kubrick sequence seem like such a breakthrough?

It was done with a really high taste level.

They took a lot of time to really get those set-ups right.

And they worked with the technology.

You notice that all the foregrounds, none of them are hard-lit

because if you tried to simulate sunlight,

really wouldn't work with the front projection screens.

What would happen if they had done the sunlight is then,

the bright sunlight would've, um, reflected off the screen

and you would have seen halos around everyone.

So they put all the sunlight in the background on the projected part of it

and the foregrounds were as if they were in shadow.

Which is exactly what we did on The Volume.

One of the most exciting things that I had

the pleasure of experiencing on this was working in the LED Volume.

Which essentially is this big room of TV screens

and a ceiling as well, which, basically, you show the environment,

you project it, or you screen the environment on these TVs.

It's a 75-foot diameter circle with a ceiling

that is an LED screen, or thousands of LED screens, LED panels,

and we put content up on that that we can then photograph

so that everything was in-camera.

The great joy of working in The Volume is that

you weren't working in a green screen environment.

We were trying to capture everything in-camera.

The sets that you see on the screen are photographs

of locations that have been digitally recreated and pasted together on geometry,

very photorealistic geometry

and, uh, with real textures from the photographs

to really trick you into thinking this is all real.

What we did that was very different on this project was that

it was content that was motion-tracked from the camera

so it had a positional data from the camera,

so you could have perspective and parallax change as the camera moved,

and so the content on the wall would change as the camera was changing.

We created this real world.

So it virtually was like shooting in a real environment,

but it was actually content.

The ability to shoot a 10-hour dawn, a 12-hour dawn is extraordinary.

To shoot any sequence where you say,

Oh, this world's not quite right. Let's just move it a little bit,

let's just change this a little bit,

it's mind-blowing what that tool is.

The beginning of The Volume was just gonna be to have

enough of the design to be interactive light

and then to have a green screen that could follow the person around

or be behind them so I didn't have to wait to light screens.

Then as we started to look more deeply into the game engine technology

and started doing tests, and we actually did tests in here

when we were doing Lion King with just a regular TV set,

and putting a TV set in front using all the tracking information

that we were using for the VR for Lion King.

And we saw if we fed that back into the screen,

it was fast enough that you could move it around and...

We had a model head of a lion in front of this piece of architecture,

and it looked weird when you're...

On the outside it's smearing and doing weird stuff,

but when you looked through the lens, it was perfect.

And so that was the proof of concept,

and other people have been thinking about doing things like that,

but it takes a show to commit a certain amount of resources

and also have somebody, honestly, like Kathy Kennedy...

Yeah.

Who can go in and who was in charge of ILM and also in charge of Lucasfilm,

and then me, who was frustrated and wanting a breakthrough like this to happen

with the vision for how that could be done

and dealing with people who had been thinking about versions of it.

And as we had talked about before,

and George Lucas was messing around in that space

since the prequels with set extensions on green screen.

So everybody was hitting it from slightly different angle,

and everybody came together.

And then also to have a group of filmmakers, to be honest with you,

who were up for it and who weren't frustrated by the limitations,

but saw the opportunities in it.

I think we're talking about The Volume and the technology,

but, at least for me, I found what it did was that it put me back in a set.

Yes.

You know, it put me back where the rules are,

you know, what you understand.

And so I was walking into a set with my actors

and we're talking and we're moving and we're figuring out

the world and the scenes as if it was built.

And so the freedom of that was invaluable,

but that all came through the, you know,

the process of knowing how this was gonna look and what you were going for.

And so when I got on set, it didn't feel like,

Oh, my God, what this...I'm strapped in by...

Creatively, it felt like I was free to sort of really create.

And Jon, you sort of pushed that even more,

'cause I'd sort of be like,

"Well, you know, this is kind of how we were doing it in pre-vis and I'm going that way".

And you're like,

"Okay, yeah, cool...but let's try, maybe just a...let's just do something else".

And I'm like, "Oh, okay, we can do that? We just spent all this money doing..."

You're like, "Let's just go".

But that got everybody prepared.

Yeah, it gets you prepared...

'Cause there's story and also building the assets.

Yeah.

For example, on the episode when you're on the space station...

Mmm-hmm.

To create Volumes based on where

you wanted to stage action was invaluable.

Yeah.

But what, you know, Bill Burr was saying didn't matter as much

and we could change that and we could alter what's going on within it

as long as we gave our visual effects people and our art department enough time

to build it so that you could play in it.

Exactly.

That's what we were learning is,

like, we have to schedule what we have, because The Volume, I mean, it was like...

For people who have never been there,

if you think of this table top like the stage,

and the outside of it is all video walls and the top is a video wall.

Everything that was on the floor

or inside of the video walls has to be there.

If it was dirt there, it's dirt here that matches.

And if we had furniture in front of it or cargo, we put that there.

And even if we had the Razor Crest,

we had a Razor Crest that could fit inside The Volume,

or half of it, and we'd have half on the wall.

And everything was planned so that whatever's on the floor

would continue into the wall.

And then wherever you moved the camera,

everything on the stage was correct,

everything in the background had parallax like in a video game,

because we were using a game engine to render it in real-time.

You damage one of my droids, you'll pay for it.

Just keep them away from my ship.

Yeah? You think that's a good idea, do ya?

I don't think I could sit here and explain

what it is that's so fundamental about a game engine.

I'll give my version, and then Jon'll give the right answer.

It's real-time, so the visual that anybody playing a video game,

are looking at are being calculated in milliseconds.

So, if you move right or you move left in a scene, or you turn around

and you see a view in an environment never seen before,

it's happening in milliseconds, it's real-time.

For the audience to understand,

it's like being inside the Battle of Hoth, from Empire Strikes Back,

but the battle is happening around you,

'cause you planned the battle.

How a video game works.

Except now we're setting cameras as though we're taking that video game reality

and we're doing a cinematic experience based on it.

Then we're gonna control it.

And set. Action.

Sometimes you'd walk into that Volume and not realize where the screens were.

Because you'd have...

You'd have these sets that were built

and all these people in them and characters and stuff,

and it would, even to the human eye, you'd get deceived

and you wouldn't be able to figure out where the practical stuff ended

and where the LED screens began.

So even to the human eye, it was believable.

The hangar was the biggest one, for me, that we did in Rick's episode six,

where you're in this hangar looking out at the stars

and you feel all this construction and equipment.

The ship wasn't part of the deal.

Well, the Crest is the only reason I let you back in here.

After a while, I would sit there and go,

"I'm gonna try to guess what's real here", because you can't tell,

because the boxes in the foreground blend so well...

It's crazy.
It was amazing.

We used to do our walkthroughs.

Maybe we're shooting on the backlot one particular day,

after wrap, we would come to The Volume to prep for the next day.

Often, the art department and the set dec has got everything counted,

it looks great, the screens are white.

So, we turn the screens on,

and all of a sudden, the set appeared there.

I remember we were having a conversation about something, and off in the screen,

there was an effects pass of some smoke

because there's some weld happening on the screen.

I remember somebody shouting,

"There's a fire! There's a fire!"

We had a funny one like that where we were doing this talk.

It was being sent to Disney people.
The Disney conference.

You were on the other side of the line.

We were two little director's chairs set up in this hangar.

We were talking and this whole thing was going on, and we were in The Volume.

And later we found out, you told us, nobody realized we were in The Volume.

They thought they were looking at this giant set

they spent all this money on.

Yeah, we were TV.
Absolutely. It looked fantastic.

Here are the cool ones.

To me, here are the cool ones that I thought were mind-blowing.

There was something about that Jawa camp for Chapter 2.

When they're all sitting, 'cause that was how they did it back in...

Where was it, Morocco?
Tunisia.

Tunisia, there were just some treads and everything else was framed out.

And so we built one tread and we extended everything that way.

One of the ones I thought was really aesthetically gorgeous

and held up very well was in your episode.

When you have Ming-Na. No, Ming-Na.
The hangar?

Oh, in the cliff and the foreground...
Yeah. And Jake.

In front of the Dune Sea, and he's on the bike.

And it's just a silhouette.

It really got that balance of bright and dark and silhouette.

There's an extraordinarily long list

of all the advantages of shooting in an environment like this.

In a simple sense, it's a large lighting box.

The lighting on the subject is actually real.

It's not something that we have to imagine

what it would be like and create that artificially.

You could actually put a subject in that space

and they would be lit by the LED Volume.

You have all the ambient light, and reflections, more importantly.

And so we have a very reflective character to take advantage of that

because that helps sell it.

The main character, his helmet is basically a big mirror,

so you've got the benefit of actually having the environments

reflected in his helmet and being able to see that,

instead of putting that in later on, which is just a big hassle.

When you have green screen,

first you knock the green out of reflections,

then put another reflection,

map another reflection on top of those surfaces.

So people avoid reflective surfaces when you're on a green screen stage,

whereas we embrace it.

Most DPs do what they're trained to do, which is light what's in front of them,

and if they have a set with actors

and then a window with a blue screen out there

that isn't four stops overexposed,

then they light what's in front of them and you end up with something that,

when you put the four stop brighter thing out the window,

doesn't look right or isn't pleasing.

And then you're trying to artificially recreate that balance.

Right. So our video wall allows them to actually see that and stop for it,

change the exposure for what they're seeing.

As they would in a real environment.

We talk about The Volume as being this super cutting-edge means of film-making.

But one of the great things about it is how it enables different departments

to work in a more traditional way

than, for instance, on a blue screen where

the DP doesn't really know what the background's gonna be

so they're kind of coming up with lighting that we hope is gonna match later.

Now they can see it all in-camera,

actor, background, adjust lighting accordingly.

The actor is immersed in the imagery.

You've got stuff you can cut with immediately.

It's not just actors on blue screen.

Every department, even set dec, it's like...

'Cause we'll have, obviously, physical builds in front of the screen

and the two need to live in the world together.

You can just look through the camera, or even just stand there and say,

"Oh, yeah, we should touch this up or adjust this a little bit", or whatever.

And that's all the way, you know, those departments work...

It's emulating what you see in a real location.

This technology, it allows you to go into the cutting room the next day,

or even sometimes that day, look at it and pop right back in

and pick up something you might've missed very easily.

Because it's just putting back onto the screens what you had.

You have the lighting set and you're up and running.

That was a big benefit because we picked up many shots and split our team,

and I could shoot the second unit for the other directors

and they would have a little list of shots.

And I'd just take Richard with me or one of the other camera operators

and we'd just do these shots really quickly.

And, you know, we could change The Volume within the half-hour

to be a completely different set,

and if we weren't shooting the ground we didn't need to change it,

'cause you'd still get depth and parallax on the screen and the reflection needed.

And so all those tools, too, it is relative to animation,

which was a good comfort zone for me doing live-action the first time.

Seeing three-dimensional objects on the screen

and the parallax and the virtual blocking and...

It made sense in a way because of the way George taught me to see Clone Wars

and the animation and virtual blocking techniques.

Creatively, the actors had one of the biggest responses to this whole thing.

Ready, and door!

Now, we have a room where there are things that you can see.

Where I can climb up on top of my TIE fighter and see the horizon.

It's interactive.

I can now feel the power of that sun coming up.

There could be nothing else in the room,

and now you can put a set in there with gravel and dirt and a spaceship.

So I have something concrete and physical to look at and feel and touch.

Wow, what a difference. What a difference a day makes.

There was nothing like being in a lava tunnel.

Actually, it's so disorienting because usually as an actor,

you're standing there and you're pretending.

But here we are and you look at the wall, and literally,

the wall is moving, as you would be moving on the boat.

And you look down the end of the tunnel, and the tunnel is growing in size...

The opening, or the exit is growing.

And you look behind you and the exit is shrinking

and the lava is flowing and the ceiling is moving and so...

And we had a projector on you.

That was a trick we learned on Jungle Book.

That was hard to get used to...

It was!

For me, it was hard to get used to. I was like, "Oh, I hope I..."

Felt like you were moving.
You don't want...

I get motion-sick sometimes, so I was like...

I just didn't want to fall overboard in the lava.

No.

It's a valuable thing to know as actors.

I'm a delicate flower.

How you feel being inside The Volume

with something to look at, as opposed to green screen.

'Cause a lot of people ask, "How do the actors feel about this"?

So you guys could each speak to that, I think.

I gotta tell you, I found it

as liberating as anything I've ever worked on.

Because as soon as you accepted it, you were in the environment.

You didn't have to pretend anymore.

And now it was really about

whatever context at the time, you could immediately buy in.

You didn't have to create it in your mind.

Also, when you think about it, if you've got four of us in the boat

and we're going down the...

Your four people, if you don't actually see the same thing,

have four different concepts of what is going on.

So now we're really a team in the boat.

It's like, you don't have to guess, man. This is what's happening.

So if somebody refers to something, it's a buy-in immediately.

And a buy-in that you are all in sync with.

That, to me, was amazing.

I must be destroyed.

The great irony is that George Lucas,

way over 20 years ago,

was saying, like...I was hearing him saying, he's like,

"One day, we're gonna be able to make these movies in our garage".

"And it will look like just as real as traveling a zillion miles away".

The first time that I stepped on to The Volume,

I was like, "Oh, my gosh, this is George's garage".

When George came by the set and looked at the wall we had built,

and we're so happy that I'm getting ready to tell him,

We're the first people ever to do this.

People talk about it, but nobody's actually done it.

"You didn't have a video wall".

He says, "No, not a video wall", but it was what the prequels were.

He told me that it's what he was building, or wanting to build...

Right.

At the base of the hill from the Ranch.

Yeah.

That he never was able to do.

It's a virtual production.
That's exactly what he was setting up.

He was a little bit ahead of what the technology could deliver.

He always was 10 years ahead of his time.

Once we got digital,

we could actually, especially with blue screen,

we could shoot the stuff, do it in blue screen,

and do all that stuff, and then just do it later.

Now you can do it on set, which is even better.

Okay, let's swim it back and forth a bit. We're gonna go left.

What's cool is other filmmakers are coming through,

other people that are curious about the stuff.

Now that you see somebody could do it, it becomes easier to emulate,

because this technology is pretty readily available.

There's nothing proprietary here.

It's all stuff that you can...you know, it's game engine technology,

it's video screens, it's positional camera data.

It's things that are kind of...

It's just combined in a way nobody's done it before.

You have to have an understanding of how to do it.

That's also what's fun is that our team,

our AD team, our directors, we're experimenting

with this new combination of technologies for the first time.

All these young directors will be able to go out there

and now they know how to do it, and so they can move forward

and try to move the ball forward technologically as well.