You really do start to realise how big a vfx project is once you get to the compositing.
Take shadows for example. This became (and continues to be) perhaps the biggest part of the post processing. I already knew I would have to build any object in 3D if it needed to receive a shadow, but there were several things, having never done a project like this from start to finish, that I didn't realise until I came to the compositing:

Objects...objects everywhere

There were a lot more objects in the scene once I really started to look. Originally I just thought I had to build the main buildings and surfaces. But no, as I looked closer, railings, people, buses, taxis...the more I looked the more I found. Luckily most of this didn't require any new techniques, just more modelling and quite a bit more animation (more on animation in a separate post, later).

Most of the objects can be dealt with quite easily as they're stationary. There was, however, one pedestrian who was walking across the street and was close enough to the camera that the shadow couldn't simply go over them flatly but for realism should match the contours of the body.

I already had a person armature and model from the people I had made to sit on the rollercoaster so it wasn't much of a problem to re-use this model and animate the armature to the person in the footage. It was a bit tedious, and if you saw the 3D view from any other perspective than the camera you might think this person didn't have a lot of bone structure, with knees and elbows bending in directions that would make you wince. Nevertheless, from the camera view the little 3D figure appears to cross the road matching his real-world counterpart fairly well with shadows falling over his body instead of being flat, like some of the more distant people can afford to have.

Foreground people

If mid-ground people weren't annoying enough to recreate, then boy was I in for a treat when I noticed all the fore-ground people, with more detail than my generic 3D human could handle.

When we originally filmed, it took several takes to find one that didn't have cyclists or buses going right through the middle of the shot, but even so, it was impossible to film on a busy street without there being some pedestrians and vehicles in the way.

Any person that appears in front of the shadow means that a part of the shadow has to be removed. The only way to really deal with that was with masking, so I had to painstakingly mask the 3 or 4 people walking across the shot frame by frame. Frame. By. Frame. This was not enjoyable even though it was a fairly short sequence. But, once complete a simple 'Math' node could add the white mask back over the top of the shadow pass, cancelling out the shadow in one fell swoop.

Existing Shadows

These two previous points are annoying but at least I was already aware of those issues even if I wasn't completely aware of the scale they would need to be dealt with. Which left the detail I hadn't been aware of:

The footage has existing shadows. This sounds obvious. And it is! Of course there are shadows in the footage, things in the real world cast shadows. But what was less obvious is that I would somehow have to stop the CG shadows from overlapping them. If they overlapped there would be a dark patch, you can't cast a shadow onto something that's already in shadow without it getting darker than it should be, so the shadows have to very precisely stop at the exact point the real shadows begin.

This started to worry me slightly, I'd missed something which seemed obvious and I wasn't immediately sure how to solve this...

I realised that unless I wanted to try and roto (mask) the shadows very precisely I would have to attempt to use a technique similar to how I'd extracted the ripples from the river. My thought was that I could create a rough mask around the area in the footage with the existing shadow and then isolate the darkest parts of that area, which should be the shadows, with colour correction. For the most part this does seem to work, there's still a bit of fine tuning to do at the exact join between real and CG, but the vector blur helps to, well, blur that line.

A lot of work for just one part of the compositing and there's still lots more to do. Aside from compositing though, putting the people on the rollercoaster was a fun little exercise so I'll try and cover that next time.


Ray.

In the last post I said I would talk about the Futuristic Car 2 project, but, well, enough time passes between these posts that I'm not always working on the thing I said I would be. I certainly was working on the other project, and I will definitely write about it at some point, but not today. It's always nice to have a little break from longer projects, it's easy to start to get sick of a project if you work on it solidly for too long so I reverted back to something I haven't worked on for over a year: The Rollercoaster project.

This time around I've mainly been working on updating the materials, which has been really quite fun to get back to, and feels relatively easy compared to the other project's myriad of objects, hierarchies and scripts. The materials didn't have to be updated, there's plenty of other things that aren't finished, but in the year since they were first created an anisotropic shader was added to Cycles. I thought I would at least update some of the brushed metals to use them. But of course, I ended up doing a lot more than that.

Here's a nice shot of the wheel the wheels on the rail and the base which connects them to the 'coaster:

Some of these materials have already been updated further, so there's a bit less wear and tear on the purple bolts and the gold wheels now have a bit of dirt on them.

This kind of detail, the worn edges, the subtle bumps and dirt maps are all unfortunately fairly irrelevant. The rollerocaster will be going fast enough that motion blur will hide most of this work, but I can't knowingly release something that's meant to be realistic if I know I could have made it better. Besides, it does at least give me a nice asset to show off in my portfolio even if the final video doesn't quite make the most of the work I've put into it.

With that in mind, I'm working on a few tricks to render a slightly closer camera than the original tracked footage might appear to allow to show at least a bit more detail...


Ray.

As the rollercoaster project continues this blog post will look at a technique I used to displace the reflection of the rollercoaster in the river reflection.

The river, where the rollercoaster is being reflected is obviously not a flat surface in real life and so in some way that surface needs to be recreated so our reflection also isn't completely flat. There's a couple of options at this point and they separate into either using geometry to displace the reflection during the render or by creating an effect in the compositor as a post-process.

The geometry method could either be a displacement modifier or an ocean modifier. I only tried the ocean modifier, and while it can recreate the surface well, the amount of geometry that is needed for the small ripples took far too long to load into memory when rendering.  The displacement modifier would have the same issues.

The second option and the one I had already thought would be better for the render time reason is to displace the reflection in the compositor. I don't know at what point I checked to see if a displace node existed (a key ingredient for this to work) but I already knew the technique I wanted to use. It was a technique I had seen demonstrated in Nuke, where using colour correction nodes you could isolate the highlights on the surface of the water and use this as a factor for the displacement, because each highlight represents the top of a ripple.

It's not a method I've seen widely documented apart from the original place I saw it, unless it generally goes by another name. I tend to call it matte extraction, a matte being a mask. Knowing the technique I just had to put it into practice, and, this is one of those rare occasions where nothing went wrong.

On the left you can see the ripples I extracted from the footage using colour correction and on the right you can see a reflection that has been displaced (albeit very faintly at this resolution). I'm currently isolating the water in the original footage with a mask, which isn't animated, so I'll probably try and create another render layer which can be used to do this automatically. I don't exactly enjoy animating masks.

It's not perfect at the minute and will no doubt go through a lot of fine tuning before it looks right, and of course this was without the motion blur applied to it.

The same thing has to also be done for any shadows that appear in the reflection, such as the shadows cast onto the reflected bridge, but it's easy to apply the effect again by grouping the nodes and creating another instance.

Ray.

[This post was originally posted on Half-Man Half-Bird's website and has been reposted here with permission so that my future posts on this topic are not out of context.]

The problem of writing a blog post is that while I'm writing it I feel I should be working on the thing I'm writing about. That's probably why it's been roughly a month since the last blog post even though I said it would be every Wednesday. The trouble is it doesn't always feel that there's a blog post worthy amount of work done. Maybe I should do smaller more regular updates.

Anyway, on to updating you on the progress of the vfx for our project. I got a bit bored just working on the same shot all the time, it was starting to get a bit monotonous so I started work on the other two shots so that if I get tired of working on one I can switch to another. Turns out that although I originally wanted to focus on shot 2 which I felt would be the most interesting, it was also the hardest. So progress on one of the other shots, shot 3, has probably overtaken progress on shot 2 because it's a lot easier.

Shot 3 is the last shot in our sequence and it shows our rollercoaster looping around a bridge and going through one of the bridge's archways before going off into the distance. Oh and it's over water so we also have to contend with reflections. Which is fun.

Below on the left (click to enlarge) is the original frame of shot 3 and on the right is with all the geometry added.

So you can see the track is in place and then there's all the proxy bridge geometry for catching shadows or acting as a mask.

It got a little frustrating at times, the tracking gives more than a good start to the camera motion but it isn't perfect. I decided to just brute force it and keyframe out all the problems but it turns out it was nowhere as bad as I  thought. The camera needed only about 7 keyframes to give it a bit of improvement from the original track. That still didn't mean that the geometry fitted the footage perfectly, I don't know what causes that issue, whether it's still a bad track or if it's lens distortion. However I changed the geometry it still wouldn't fit at some points so again I decided to manually keyframe it. Again it was only a few keyframes on some objects, nothing major.

There were two other areas of the bridge mesh that needed a bit more work as it wasn't individual objects that needed to be animated but vertices, the archway of the bridge and another small part. For this I used empty hooks (below) which I could animate and their associated vertices would follow.

There was another part where I also had to add an empty for each vertex so I invested a little time in creating a script that would do this automatically for a set of selected vertices which should save a bit of time in the future.

The current issue to contend with is how to apply motion blur to the reflection of the rollercoaster in the river, reflections can only be motion blurred if you are using full 3D motion blur, but at the minute we're using vector blur as a post process. I'm thinking I might have to created an inverted camera that renders a fake, but 3D reflection (seeing as a reflection is basically just the geometry but upside down), but I can't actually find a picture of that to explain how that's set up for the minute.

As I said, I also worked on our other shot in the sequence, shot 1, but all I've done is set up the track in that and some proxy geometry, so it's not as far along as this shot and not really worth showing at this point.

Shaders and textures are constantly being updated at the minute, that's a given, but some shaders might have to be different on a per shot basis just to account for the different lighting conditions, it may be less realistic, but it's more aesthetically pleasing.

Now I know how easy it can be to correct the geometry with hooks the other shots feel a bit less intimidating so hopefully I can continue at the same speed when I move onto them. Onwards.

Ray.

[This post was originally posted on Half-Man Half-Bird's website and has been reposted here with permission so that my future posts on this topic are not out of context.]

So it's been a couple of weeks since the last update but things have advanced quite a bit. I'm still working on 'Shot 2' but the tracks have been laid out, rendering tests have been done and shading has started.

One of the first things I did was set up the track which was simple enough and just meant scaling the imported (from the other scene) track so everything was roughly the correct size and then match it to the geometry I had already built that represents the buildings. You can also see the support struts that I've made which 'hold up' the building, because it's not like it would end up pulling down the building or anything...

Having set up the track and carriages I then worked on a few shaders and preliminary renders. The lighting was pretty simple (once I'd figured out how simple it could be) and is just a sun lamp, the shadows of which I managed to match pretty close to the source footage. The colour's a bit off for the shadows but I was working on the direction and angle at this point. I've changed the position slightly since then and a changes in ground height to match the footage also alters the look of them.

This was also a chance to quickly test the motion/vector blur which was done after the render in the compositor and doesn't look too bad. Of course, to get vector blur there has to be some movement so I very roughly animated the carriages along the track. One issue I noticed during the render tests was not the speed of the render, which wasn't bad at all, but the time it took to load all the geometry into memory. It took a good while to load everything so I optimised the track by animating the 'count' property in the arrays so that as the camera pans the track extends, but before it's needed it isn't visible. I think I can take this further with more geometry being invisible/not rendered until it's needed.

Here's just a closer look at those support struts, they're the sort of thing you see supporting big buildings or bridges. And of course it's only after I've made it that I've found out the support strut's proper name is 'tension bars'.

And here's a quick rendered view of the track and support struts, it's a bit different to the rendering above which was done last week. The shaders were looking pretty dark and all of the reference images I'd looked at showed bright colours so I updated the shaders to reflect that, but I might try and find some in-between.

Although you might not notice it (but you'd notice if it wasn't there) I've also added a slight bevel to the blue blocks that the struts connect to and a few other objects, they were looking a bit sharp but I was thinking that they would be covered in thick paint; the bevel gives a slight highlight along the edge.

Aside from all that I also started setting up render layers and doing a quick composite in blender, even though Matt will be doing the compositing probably in Nuke. I need to do this to make sure render layers are getting outputted as intended. There were a lot of complexities involved where certain layers should be receiving shadows but not appearing in reflections but I'll probably go into the details of that in a couple of weeks.

Ray.

[This post was originally posted on Half-Man Half-Bird's website and has been reposted here with permission so that my future posts on this topic are not out of context.]

It was a long week last week. Long enough that I didn't do a blog post and barely did any work on the rollercoaster project. A week filled with waiting around for various renders to finish. Unfortunately, the renders weren't for this project but for a personal project that I finally finished yesterday. But that's not to say that there's not been an update to this project, after last weeks blog post I had a few days of working on it and managed to get some important steps completed.

The main/only thing I did was to figure out how to rig the rollercoaster to be able to travel along the track in a way that will make it easy to animate. I had a feeling it was either going to work right away and be completely simple, or take a bit more work. No need to guess which one it turned out to be then.

The most obvious thing to try when trying to get something to travel along a path in Blender is the follow path constraint and then use the path animation properties on the curve to move the object along.  It was easy enough to do for the first carriage/set of seats, which gave me a bit of hope the others would work.  When I used the same settings on the other carriages I found I couldn't just parent the other carriages to the first one and have them all move at once as I'd hoped. The first one moved and even matched the direction that the rails were bending in but the others just followed the first one and ignored their follow path constraints.

I then moved onto a slightly weirder but ultimately successful technique that I've used before. It's weird in the sense that we are basically using a tool that would normally be used for modelling, but under certain circumstances can be useful for animation. The technique uses the curve modifier to deform a single cube (one for each carriage) along the main curve that determines where the track is. You can then, and this is the quirky part of the technique, move the cube on it's local y axis (direction curve is pointing in) and it will appear to travel along the curve. You can then parent the carriage to it's controller cube without any issues.

The cube itself isn't rendered and is only used to get the 'direction' or tangent of that point in the curve. In actual fact I couldn't do a simple parent to the curved cube I had to do a parent to 3 of it's vertices. If you just do a normal parent operation the carriage wont follow the cube properly as the cube isn't being rotated as it travels along the curve, it's just being deformed. But you can parent directly to the vertices  of the cube which seems to work. Then it's just a case of parenting all these controller cubes to a single empty, the empty can then be moved and everything follows correctly. The empty also has to be moved on it's local axis so even if the curve is going in all directions you only ever move the empty on it's local y axis to move everything along.

Below you can see all the carriages bending as the curve is deformed.  I just added in a quick hook (the cube) which can control a point on the curve. It means I can quickly move the cube to deform the curve to see if the rigging is working.

The above animation also highlights another issue I had to contend with once everything was following the main curve properly.  As the curve bends, the distance between each carriage increases which caused me to have to really think about how I could actually connect the carriages together as whatever I used would have to adapt to this increase in distance. You can see that it's increasing because the arrays which make the rails are having to auto extend on the left hand side to accommodate the new length of the curve. I don't think real rollercoasters have the distance problem but I couldn't actually find any references of how the carriages are joined so I don't know for sure. I eventually went for a piston option, or telescopic might be a better word. A series of constraints mean that the beam and joining objects point to the next carriage in the line, so the beam parented to carriage 1 points to carriage 2 etc. When the distance increases more of the beam that was hidden in the 'joint' is revealed. In the above animation it's the line of objects below the main rail.

It was pretty important to get that done because without it animation could have become really tedious. Another bonus is that it seems to move really quickly in the viewport, but there may be a bit of a slowdown once the rails are fully extended.

Next is to extend the track and match it to the footage and construct the support struts which connect the 'coaster to the surfaces.

Ray.