You really do start to realise how big a vfx project is once you get to the compositing.
Take shadows for example. This became (and continues to be) perhaps the biggest part of the post processing. I already knew I would have to build any object in 3D if it needed to receive a shadow, but there were several things, having never done a project like this from start to finish, that I didn't realise until I came to the compositing:

Objects...objects everywhere

There were a lot more objects in the scene once I really started to look. Originally I just thought I had to build the main buildings and surfaces. But no, as I looked closer, railings, people, buses, taxis...the more I looked the more I found. Luckily most of this didn't require any new techniques, just more modelling and quite a bit more animation (more on animation in a separate post, later).

Most of the objects can be dealt with quite easily as they're stationary. There was, however, one pedestrian who was walking across the street and was close enough to the camera that the shadow couldn't simply go over them flatly but for realism should match the contours of the body.

I already had a person armature and model from the people I had made to sit on the rollercoaster so it wasn't much of a problem to re-use this model and animate the armature to the person in the footage. It was a bit tedious, and if you saw the 3D view from any other perspective than the camera you might think this person didn't have a lot of bone structure, with knees and elbows bending in directions that would make you wince. Nevertheless, from the camera view the little 3D figure appears to cross the road matching his real-world counterpart fairly well with shadows falling over his body instead of being flat, like some of the more distant people can afford to have.

Foreground people

If mid-ground people weren't annoying enough to recreate, then boy was I in for a treat when I noticed all the fore-ground people, with more detail than my generic 3D human could handle.

When we originally filmed, it took several takes to find one that didn't have cyclists or buses going right through the middle of the shot, but even so, it was impossible to film on a busy street without there being some pedestrians and vehicles in the way.

Any person that appears in front of the shadow means that a part of the shadow has to be removed. The only way to really deal with that was with masking, so I had to painstakingly mask the 3 or 4 people walking across the shot frame by frame. Frame. By. Frame. This was not enjoyable even though it was a fairly short sequence. But, once complete a simple 'Math' node could add the white mask back over the top of the shadow pass, cancelling out the shadow in one fell swoop.

Existing Shadows

These two previous points are annoying but at least I was already aware of those issues even if I wasn't completely aware of the scale they would need to be dealt with. Which left the detail I hadn't been aware of:

The footage has existing shadows. This sounds obvious. And it is! Of course there are shadows in the footage, things in the real world cast shadows. But what was less obvious is that I would somehow have to stop the CG shadows from overlapping them. If they overlapped there would be a dark patch, you can't cast a shadow onto something that's already in shadow without it getting darker than it should be, so the shadows have to very precisely stop at the exact point the real shadows begin.

This started to worry me slightly, I'd missed something which seemed obvious and I wasn't immediately sure how to solve this...

I realised that unless I wanted to try and roto (mask) the shadows very precisely I would have to attempt to use a technique similar to how I'd extracted the ripples from the river. My thought was that I could create a rough mask around the area in the footage with the existing shadow and then isolate the darkest parts of that area, which should be the shadows, with colour correction. For the most part this does seem to work, there's still a bit of fine tuning to do at the exact join between real and CG, but the vector blur helps to, well, blur that line.

A lot of work for just one part of the compositing and there's still lots more to do. Aside from compositing though, putting the people on the rollercoaster was a fun little exercise so I'll try and cover that next time.


Ray.

In the last post I said I would talk about the Futuristic Car 2 project, but, well, enough time passes between these posts that I'm not always working on the thing I said I would be. I certainly was working on the other project, and I will definitely write about it at some point, but not today. It's always nice to have a little break from longer projects, it's easy to start to get sick of a project if you work on it solidly for too long so I reverted back to something I haven't worked on for over a year: The Rollercoaster project.

This time around I've mainly been working on updating the materials, which has been really quite fun to get back to, and feels relatively easy compared to the other project's myriad of objects, hierarchies and scripts. The materials didn't have to be updated, there's plenty of other things that aren't finished, but in the year since they were first created an anisotropic shader was added to Cycles. I thought I would at least update some of the brushed metals to use them. But of course, I ended up doing a lot more than that.

Here's a nice shot of the wheel the wheels on the rail and the base which connects them to the 'coaster:

Some of these materials have already been updated further, so there's a bit less wear and tear on the purple bolts and the gold wheels now have a bit of dirt on them.

This kind of detail, the worn edges, the subtle bumps and dirt maps are all unfortunately fairly irrelevant. The rollerocaster will be going fast enough that motion blur will hide most of this work, but I can't knowingly release something that's meant to be realistic if I know I could have made it better. Besides, it does at least give me a nice asset to show off in my portfolio even if the final video doesn't quite make the most of the work I've put into it.

With that in mind, I'm working on a few tricks to render a slightly closer camera than the original tracked footage might appear to allow to show at least a bit more detail...


Ray.

As the rollercoaster project continues this blog post will look at a technique I used to displace the reflection of the rollercoaster in the river reflection.

The river, where the rollercoaster is being reflected is obviously not a flat surface in real life and so in some way that surface needs to be recreated so our reflection also isn't completely flat. There's a couple of options at this point and they separate into either using geometry to displace the reflection during the render or by creating an effect in the compositor as a post-process.

The geometry method could either be a displacement modifier or an ocean modifier. I only tried the ocean modifier, and while it can recreate the surface well, the amount of geometry that is needed for the small ripples took far too long to load into memory when rendering.  The displacement modifier would have the same issues.

The second option and the one I had already thought would be better for the render time reason is to displace the reflection in the compositor. I don't know at what point I checked to see if a displace node existed (a key ingredient for this to work) but I already knew the technique I wanted to use. It was a technique I had seen demonstrated in Nuke, where using colour correction nodes you could isolate the highlights on the surface of the water and use this as a factor for the displacement, because each highlight represents the top of a ripple.

It's not a method I've seen widely documented apart from the original place I saw it, unless it generally goes by another name. I tend to call it matte extraction, a matte being a mask. Knowing the technique I just had to put it into practice, and, this is one of those rare occasions where nothing went wrong.

On the left you can see the ripples I extracted from the footage using colour correction and on the right you can see a reflection that has been displaced (albeit very faintly at this resolution). I'm currently isolating the water in the original footage with a mask, which isn't animated, so I'll probably try and create another render layer which can be used to do this automatically. I don't exactly enjoy animating masks.

It's not perfect at the minute and will no doubt go through a lot of fine tuning before it looks right, and of course this was without the motion blur applied to it.

The same thing has to also be done for any shadows that appear in the reflection, such as the shadows cast onto the reflected bridge, but it's easy to apply the effect again by grouping the nodes and creating another instance.

Ray.

It's a catchy post title I know.

Technically this is the first post on my own site about the vfx rollercoaster project I'm working on, so if you want to catch up have a look at some of the previous posts that I ported over from my other site.

At the end of the last blog post I said something about using an inverted camera so that vector blur would work for reflections. I did indeed get this to work and seeing as I can't find much mention of either an alternative method or this method, I'll explain a little further.

So in the third shot of the vfx sequence the rollercoaster loops around the bridge before going off into the distance. It's going quite fast so there will be motion blur and we're using Cycles in Blender so I could probably use the true 3D motion blur, but aside from the fact I've never got this to work I also want to cut down the render time so I'm using vector blur in the compositor. This all works fine, no problem there. The issue is that said bridge is over water, meaning there should be a reflection of the rollercoaster...but if the rollercoaster is being vector blurred, then so should the reflection.

But that's the thing, the vector blur works by using a vector pass which holds the motion of the object. But a reflection, or rather the object doing the reflecting, doesn't move itself, only what it reflects moves. So if you looked at the vector pass you wouldn't see any motion data for the reflection. A vector pass has to directly see the moving geometry, not indirectly via reflections.

Here's where the inverted camera comes into effect. We're going to use a second camera that completely bypasses the object that does the reflecting and renders the geometry itself. It's a little complex so I've put together a crude drawing.

The red camera is the original, motion-tracking camera. The green represents the object that we've been using to reflect the rollercoaster (the river).  The blue camera is a new, second camera, and by the little blob on the bottom of the camera we can see it's pointing downwards, whereas the red camera is pointing upwards. This second camera has been created by duplicating the original camera, parenting it to an empty, and then flipping the empty on several of it's local axis until it's flipped upside down. You've then just got to move the empty down so that the geometry is in roughly the right place below the bridge.

The white dotted line represents a pixel, the red camera sees it via the reflection object (green line) but the blue camera can see it directly (when the reflection object isn't rendered).

Below is an example of this in action. On the left is a viewport rendered view showing the reflection created by a flat plane.  On the right the view of the flipped camera, I disabled the rendered view to make it a bit easier to see that the geometry is now on the bottom, I used the image on the left as a reference when moving the empty down to so the geometry lines up with the reflection it's replacing. Success! We now have a camera that sees the 'reflection' geometry directly and will generate a vector pass.

One thing I noted is that once the reflection has been distorted by the water ripples (saving that for another post) the reflection, and it's vector blur, are a lot more faint than I thought they would be. It started to make me think that the whole creation of the inverted camera was a bit pointless if you couldn't see the effect much but I think it must make some difference.

It's actually still useful because I use the same technique to reflect the shadow pass that I composite over the footage, as with the reflections the shadow pass wasn't showing up shadows of the reflected geometry, so this also solves that.

The only downside at the minute is that Blender can't render multiple cameras per frame so you have to render out the reflection camera (as .exr's to preserve the vector pass accurately) first and then composite it in (which kind of cancels out the 'saves render time' reason). This may well change with the multi view project that is being developed at the minute, but I'm not sure if that will allow different render layers to be visible per camera, so it may not help.

Ray.

[This post was originally posted on Half-Man Half-Bird's website and has been reposted here with permission so that my future posts on this topic are not out of context.]

The problem of writing a blog post is that while I'm writing it I feel I should be working on the thing I'm writing about. That's probably why it's been roughly a month since the last blog post even though I said it would be every Wednesday. The trouble is it doesn't always feel that there's a blog post worthy amount of work done. Maybe I should do smaller more regular updates.

Anyway, on to updating you on the progress of the vfx for our project. I got a bit bored just working on the same shot all the time, it was starting to get a bit monotonous so I started work on the other two shots so that if I get tired of working on one I can switch to another. Turns out that although I originally wanted to focus on shot 2 which I felt would be the most interesting, it was also the hardest. So progress on one of the other shots, shot 3, has probably overtaken progress on shot 2 because it's a lot easier.

Shot 3 is the last shot in our sequence and it shows our rollercoaster looping around a bridge and going through one of the bridge's archways before going off into the distance. Oh and it's over water so we also have to contend with reflections. Which is fun.

Below on the left (click to enlarge) is the original frame of shot 3 and on the right is with all the geometry added.

So you can see the track is in place and then there's all the proxy bridge geometry for catching shadows or acting as a mask.

It got a little frustrating at times, the tracking gives more than a good start to the camera motion but it isn't perfect. I decided to just brute force it and keyframe out all the problems but it turns out it was nowhere as bad as I  thought. The camera needed only about 7 keyframes to give it a bit of improvement from the original track. That still didn't mean that the geometry fitted the footage perfectly, I don't know what causes that issue, whether it's still a bad track or if it's lens distortion. However I changed the geometry it still wouldn't fit at some points so again I decided to manually keyframe it. Again it was only a few keyframes on some objects, nothing major.

There were two other areas of the bridge mesh that needed a bit more work as it wasn't individual objects that needed to be animated but vertices, the archway of the bridge and another small part. For this I used empty hooks (below) which I could animate and their associated vertices would follow.

There was another part where I also had to add an empty for each vertex so I invested a little time in creating a script that would do this automatically for a set of selected vertices which should save a bit of time in the future.

The current issue to contend with is how to apply motion blur to the reflection of the rollercoaster in the river, reflections can only be motion blurred if you are using full 3D motion blur, but at the minute we're using vector blur as a post process. I'm thinking I might have to created an inverted camera that renders a fake, but 3D reflection (seeing as a reflection is basically just the geometry but upside down), but I can't actually find a picture of that to explain how that's set up for the minute.

As I said, I also worked on our other shot in the sequence, shot 1, but all I've done is set up the track in that and some proxy geometry, so it's not as far along as this shot and not really worth showing at this point.

Shaders and textures are constantly being updated at the minute, that's a given, but some shaders might have to be different on a per shot basis just to account for the different lighting conditions, it may be less realistic, but it's more aesthetically pleasing.

Now I know how easy it can be to correct the geometry with hooks the other shots feel a bit less intimidating so hopefully I can continue at the same speed when I move onto them. Onwards.

Ray.

[This post was originally posted on Half-Man Half-Bird's website and has been reposted here with permission so that my future posts on this topic are not out of context.]

So it's been a couple of weeks since the last update but things have advanced quite a bit. I'm still working on 'Shot 2' but the tracks have been laid out, rendering tests have been done and shading has started.

One of the first things I did was set up the track which was simple enough and just meant scaling the imported (from the other scene) track so everything was roughly the correct size and then match it to the geometry I had already built that represents the buildings. You can also see the support struts that I've made which 'hold up' the building, because it's not like it would end up pulling down the building or anything...

Having set up the track and carriages I then worked on a few shaders and preliminary renders. The lighting was pretty simple (once I'd figured out how simple it could be) and is just a sun lamp, the shadows of which I managed to match pretty close to the source footage. The colour's a bit off for the shadows but I was working on the direction and angle at this point. I've changed the position slightly since then and a changes in ground height to match the footage also alters the look of them.

This was also a chance to quickly test the motion/vector blur which was done after the render in the compositor and doesn't look too bad. Of course, to get vector blur there has to be some movement so I very roughly animated the carriages along the track. One issue I noticed during the render tests was not the speed of the render, which wasn't bad at all, but the time it took to load all the geometry into memory. It took a good while to load everything so I optimised the track by animating the 'count' property in the arrays so that as the camera pans the track extends, but before it's needed it isn't visible. I think I can take this further with more geometry being invisible/not rendered until it's needed.

Here's just a closer look at those support struts, they're the sort of thing you see supporting big buildings or bridges. And of course it's only after I've made it that I've found out the support strut's proper name is 'tension bars'.

And here's a quick rendered view of the track and support struts, it's a bit different to the rendering above which was done last week. The shaders were looking pretty dark and all of the reference images I'd looked at showed bright colours so I updated the shaders to reflect that, but I might try and find some in-between.

Although you might not notice it (but you'd notice if it wasn't there) I've also added a slight bevel to the blue blocks that the struts connect to and a few other objects, they were looking a bit sharp but I was thinking that they would be covered in thick paint; the bevel gives a slight highlight along the edge.

Aside from all that I also started setting up render layers and doing a quick composite in blender, even though Matt will be doing the compositing probably in Nuke. I need to do this to make sure render layers are getting outputted as intended. There were a lot of complexities involved where certain layers should be receiving shadows but not appearing in reflections but I'll probably go into the details of that in a couple of weeks.

Ray.