|
Labs.byHook |
Flash Scripts, Tools & Methods Developed at Hook |
|
|
This blog is created and maintained by the technical team at Hook in an effort to preserve and share the insights and experience gained during the research and testing phases of our development process. Often, much of this information is lost or hidden once a project is completed. These articles aim to revisit, expand and/or review the concepts that seem worth exploring further. The site also serves as a platform for releasing Flash tools developed internally to help streamline ad development.
Hook is a digital production company that develops interactive content for industry leading agencies and their brands. For more information visit www.byhook.com.
Realtime Refraction
So it was one of those mornings, just after the first sweet sip of the morning RedBull, and the boss man pops in to ask a question: “So what do you think about refraction through glass in flash?”. Being the new guy and trying to hide the obvious “deer in the headlights” feeling, I had no choice but to respond “Umm.. that might be possible”. Thus began the quest for real-time “refraction”.
(please note the quotes on refraction, its all lies, I promise.)
The Illusion
Hitting up Google revealed that people were doing a sort of lens effect in flash 8 using the DisplacementMapFilter. With the the fear subsided a bit, the key was in the all mighty displacement bitmap filter.
The concept of displacement has been around as long as pixels, and it works a bit like this:
- Get the color data of a pixel in an image you want to displace.
- Scale that pixel up.
- Move the scaled pixel by a vertical and horizontal offset.
- Repeat with all pixels until awesome.
Flash’s DisplacementMapFilter allows you do to this and control the offsets with the channel data of another image. In other words, all of the really hard stuff is done for you courtesy of Adobe.
This particular illusion has three parts:
The most important part is the displacement map itself. We used the red and green channels to control the horizontal and vertical offsets. Dark red to bright red, would move the pixel left to right, and dark green to light green would move the pixel up and down. Anything with a median color (128) would not be offset. With a bit of noodling, we managed to build a shader that could render out these displacement maps directly from the 3D data.
The alpha image was used to crop out the proper pixels from the “scratch” bitmap data that was a result of applying all of the filters.
The final image is the beauty render of the class with alpha. This is simply overlaid on top of the cropped and filtered pixels
Another helpful bit of misdirection was the mouse cursor. You will notice that the mouse goes behind the glass and is distorted as well. This helped to sell the illusion a bit more. The mouse is simply a sprite that has its pixels drawn onto the “Distortion Layer” every frame.
The original concept called for fairly large (think 720p) stage dimensions, and multiple glass objects. Therefore performance quickly became an issue when trying to run the filters on the whole stage. The biggest performance boost (not surprisingly) came from simply reducing the number of pixels to deal with.
So we calculated a rectangle around each glass/distortion object that would have enough pixels to fill out the displacement but still be few enough to be fast. We copied only those pixels out to a temporary bitmap data object. This allowed us to apply a set of filters to a much smaller number of pixels. Next, we copied the beauty shot pixels on to that temporary bitmap data We then copied the resultant pixels back to the target display object respecting the alpha channel of the AlphaImage from above, this allowed a fairly seamless reintegration with the background layers.
So the final process is as follows:
- Depth sort the distortion objects.
- Calculate and copy out a rectangle of interest around the distortion objects.
- See if mouse is within that rectangle, if so draw mouse cursor pixels on top of the temporary bitmap data.
- Apply other filters (such as blur) to the temporary bitmap data.
- Apply DisplacementMapFilter using the displacement image map.
- Copy the pixels from that bitmap data to the target display object respecting the alpha image.
- Repeat for each distortion object.
Wallah, instant awesome.
The moral of the story? CopyPixels is hella fast.
There is one more feature that is supported but not shown in this demo, think animated ice cubes inside of the glass…
20 Responses to “Realtime Refraction”
Leave a Reply
Most Commented
- (119) Facebook "Like" Button in...
- (116) FlasHTML5 Video: The Java...
- (83) Kutout - An application f...
- (75) Ogg Vorbis Encoder + Deco...
- (69) Alchemy Series Part 6: Og...
- (67) Flash Mouse Wheel Support
- (65) PHP Type Hinting with Ecl...
- (55) Facebook Canvas App OAuth...
- (53) Facebookery, the Flash an...
- (47) Digital Sheep: Air For An...
Categories
3D, Actionscript, Actionscript 2, Actionscript 3, adobe, air, airforandroid, Alchemy, android, as2, as3, Audio, avm, ByteArray, C, Canvas, charts, Cygwin, data, Data Structures, Debugging, Decoder, Diagnostics, Encoder, Event, EventDispatcher, facebook, Flash, Flow, Framework, gcc, Graph API, haXe, html5, Image Extraction, ios, iphone, java, Javascript, JSFL, Like Button, Linux, llvm, Menu, Meta Balls, Motion Blur, MVC, Nav, OAuth, Object Extraction, Ogg, Ogg Vorbis, performance, php, Physics, pie menu, Pointers, processing, radial menu, real-time, sheep, Siox, Sound, State Machine, statistics, tricks, tutorial, twitter, Utilities, Vectors, Verlet, video, visualization, Vorbis, Windows, |
Cool site! Thanks for the information! I like this article.
which ended 3-2 victory and 2-0 total points in the Stanley Cup finals to stay ahead. Winning goal by striker Abel Ross added 11 seconds left in overtime when hit. At that time, he received a pass from his teammates quickly advancing from the left wing,
Having written plenty of on-the fly physics code, my go-to approach was simply plugging in numbers into the classical mechanics formulas:
Coach Bracelets is the very popular accessories in this season. New CoachBracelets in our Coach outlet of this season receives a lot of love and support from the public as soon as it is launched. The new CoachBracelets in Coach outlet appears with the very strong retro style, very fashionable and popular recent years.
I found your website perfect for my needs. It contains wonderful and helpful posts. I have read most of them and got a lot from them.
Oh my goodness! an amazing article. Thank you!
North American Hockey League (NHL) 4 finals in the second end of Vancouver Rogers Stadium. Vancouver Canucks first goal in extra time, so that rivals the Boston Bruins ‘sudden death’, which ended 3-2 victory and 2-0 total points in the Stanley Cup finals to stay ahead. Winning goal by striker Abel Ross added 11 seconds left in overtime when hit. At that time, he received a pass from his teammates quickly advancing from the left wing, escaped Bruins goalie Tim Thomas of the block.
@Andy, Hey awesome! thanks very much for sharing!
Same technique done here but using a Pixel Bender shader in Flash: http://www.aplweb.co.uk/blog/flash/gpu-refract-shader/. Source code is available too.
Thanks Jon!
This is great!
Ah nice one, you’ve really helped me out – looking forward to play with this idea more. I’ll see if I can transfer this technique over to 3ds max, otherwise been wanting to try out another 3d package for a while now.
cheers!
Here is a link to the refraction shader that he made:
http://labs.byhook.com/images/refraction/Refraction-Shader.jpg
Oh, and its Lightwave not MODO, my bad…
Hey Dave, one of our 3D guys wrote a custom shader in Modo I believe. I’ll see if I can get a quick explanation from him on how the shader really works, and let you know here..
Thanks!
hey jake,
great work! I found this researching a current project I’m attempting, I was wondering if you could shed some light on the process of creating that displacement map? I’ve tried various techniques, but haven’t really found a solution that includes the internal objects (in your case the ice cubes), I can only map the exterior of the object.
thanks
This is very clever. I am not much of an action-scriptor myself, but I have worked with some of them before and I highly regard the language of code and those who know how to speak it.
Impressive stuff.
can’t believe you guys were able to pull that off in Flash! nice!
laba klein
This is awesome!