Email Updates RSS Subscribe
Line

This blog is created and maintained by the technical team at Hook in an effort to preserve and share the insights and experience gained during the research and testing phases of our development process. Often, much of this information is lost or hidden once a project is completed. These articles aim to revisit, expand and/or review the concepts that seem worth exploring further. The site also serves as a platform for releasing tools developed internally to help streamline ad development.

Launch
Line

Hook is a digital production company that develops interactive content for industry leading agencies and their brands. For more information visit www.byhook.com.

Line

Realtime Refraction

Line
Posted on December 3rd, 2009 by Jake
Line

So it was one of those mornings, just after the first sweet sip of the morning RedBull, and the boss man pops in to ask a question: “So what do you think about refraction through glass in flash?”. Being the new guy and trying to hide the obvious “deer in the headlights” feeling, I had no choice but to respond “Umm.. that might be possible”. Thus began the quest for real-time “refraction”.

(please note the quotes on refraction, its all lies, I promise.)

Get Adobe Flash player

The Illusion

Hitting up Google revealed that people were doing a sort of lens effect in flash 8 using the DisplacementMapFilter. With the the fear subsided a bit, the key was in the all mighty displacement bitmap filter.

The concept of displacement has been around as long as pixels, and it works a bit like this:

  • Get the color data of a pixel in an image you want to displace.
  • Scale that pixel up.
  • Move the scaled pixel by a vertical and horizontal offset.
  • Repeat with all pixels until awesome.

Flash’s DisplacementMapFilter allows you do to this and control the offsets with the channel data of another image. In other words, all of the really hard stuff is done for you courtesy of Adobe.

This particular illusion has three parts:

passes

The most important part is the displacement map itself. We used the red and green channels to control the horizontal and vertical offsets. Dark red to bright red, would move the pixel left to right, and dark green to light green would move the pixel up and down. Anything with a median color (128) would not be offset.  With a bit of noodling, we managed to build a shader that could render out these displacement maps directly from the 3D data.

The alpha image was used to crop out the proper pixels from the “scratch” bitmap data that was a result of applying all of the filters.

The final image is the beauty render of the class with alpha. This is simply overlaid on top of the cropped and filtered pixels

Another helpful bit of misdirection was the mouse cursor. You will notice that the mouse goes behind the glass and is distorted as well. This helped to sell the illusion a bit more. The mouse is simply a sprite that has its pixels drawn onto the “Distortion Layer” every frame.

The original concept called for fairly large (think 720p) stage dimensions, and multiple glass objects. Therefore performance quickly became an issue when trying to run the filters on the whole stage. The biggest performance boost (not surprisingly) came from simply reducing the number of pixels to deal with.
So we calculated a rectangle around each glass/distortion object that would have enough pixels to fill out the displacement but still be few enough to be fast. We copied only those pixels out to a temporary bitmap data object. This allowed us to apply a set of filters to a much smaller number of pixels. Next, we copied the beauty shot pixels on to that temporary bitmap data We then copied the resultant pixels back to the target display object respecting the alpha channel of the AlphaImage from above, this allowed a fairly seamless reintegration with the background layers.

So the final process is as follows:

  • Depth sort the distortion objects.
  • Calculate and copy out a rectangle of interest around the distortion objects.
  • See if mouse is within that rectangle, if so draw mouse cursor pixels on top of the temporary bitmap data.
  • Apply other filters (such as blur) to the temporary bitmap data.
  • Apply DisplacementMapFilter using the displacement image map.
  • Copy the pixels from that bitmap data to the target display object respecting the alpha image.
  • Repeat for each distortion object.

Wallah, instant awesome.

The moral of the story? CopyPixels is hella fast.

There is one more feature that is supported but not shown in this demo, think animated ice cubes inside of the glass…

Line
13 Responses to “Realtime Refraction”
  1. Jake says:

    @Andy, Hey awesome! thanks very much for sharing!

  2. Andy says:

    Same technique done here but using a Pixel Bender shader in Flash: http://www.aplweb.co.uk/blog/flash/gpu-refract-shader/. Source code is available too.

  3. Jon says:

    This is great!

  4. dave says:

    Ah nice one, you’ve really helped me out – looking forward to play with this idea more. I’ll see if I can transfer this technique over to 3ds max, otherwise been wanting to try out another 3d package for a while now.

    cheers!

  5. Jake says:

    Here is a link to the refraction shader that he made:
    http://labs.byhook.com/images/refraction/Refraction-Shader.jpg

    Oh, and its Lightwave not MODO, my bad…

  6. Jake says:

    Hey Dave, one of our 3D guys wrote a custom shader in Modo I believe. I’ll see if I can get a quick explanation from him on how the shader really works, and let you know here..

    Thanks!

  7. dave says:

    hey jake,

    great work! I found this researching a current project I’m attempting, I was wondering if you could shed some light on the process of creating that displacement map? I’ve tried various techniques, but haven’t really found a solution that includes the internal objects (in your case the ice cubes), I can only map the exterior of the object.

    thanks

  8. This is very clever. I am not much of an action-scriptor myself, but I have worked with some of them before and I highly regard the language of code and those who know how to speak it.

  9. gMoney says:

    can’t believe you guys were able to pull that off in Flash! nice!

  10. Michael says:

    This is awesome!


Leave a Reply

*

Line
Line
Pony