Synja Dev Blog

On Friday I discovered Quartz Composer, and I’ve literally been glued to it ever since. It took me many years of hearing about it to really take the step to check it out, and I totally regret wasting so many years without it in my life. I don’t normally sing accolades on such a scale, but seriously. This thing is the shit. I was able to accomplish in ten minutes what took several hours to construct in Flash. I pride myself in not biasing myself towards any technology, and so I’m happy to abandon Flash for the performance art aspect of my life.

Here is a video I made last night, fusing Logic with Quartz Composer and my APC-40 control surface. The music was produced in Logic and has MIDI notes broadcasting out over IAC to Quartz Composer, which is listening in to both Logic and the APC-40.

I’m tweaking the cutoff frequency of the pads with a knob on the APC that is also tied to the shape of the stars in Quartz Composer, the idea being that I can modify music and graphics at the same time with the same knob. It’s pretty addictive stuff. I’m seeing stars every time I close my eyes now.

I have been using this class for ages but I thought I’d share it with everyone. It does the following things that a conventional Loader class does not:

- Automatic scaling
- Bitmap smoothing (for custom scales)
- PNG transparency
- Graceful error control (black box for when the image fails to load)

Working with the Loader class on images can be tedious as it doesn’t really behave the way we expect it to, so I created this class to handle everything automatically.

The usage is simple:


var image:Image = new Image(0,0) // Use numbers for the x, y dimensions to force the image to a specific size or aspect ratio. 0,0 uses the default image size when loaded.
image.addEventListener(Event.INIT, _imageInitialized); // Make this callback whatever you'd like, although it's not necessary.
image.load("http://www.mysite.com/myimage.png");


You can download the source code to the class here:
Image Class

• poor browser support (obvious)
• poor drawing optimizations (slow animations, even for simple things)
• poor quality in opensource video codecs

Perhaps in 2 years time it will become a viable alternative, but right now it’s really just hype brought on by Steve Jobs who said Flash is dead and HTML5 is the future. He’s only saying this because he’s highly biased against Adobe for not cow towing to Apple’s attempted influence over the development of Adobe products. Every Apple fanboy blogger jumped on his word alone and now you see sites like vimeo and youtube making crappy HTML5 versions of their player (and you have to admit, they’re crappy) thinking they’re ahead of the game but are really just going on a fool’s errand.

You can make an HTML5 product but it won’t ever be used in production before 2012 due to browser adoption rates. You can’t even drop IE6 from your compliance roster and the browser is 7 years old! At best you’d have a flash version of your product anyway, and then you’d have to do development against 2 products instead of one, which makes no sense from a business perspective.

My main concern is its poor performance in drawing. Word on the street is that Canvas is incredibly slow and be unable to do anything but basic animations. Flash is already moving onto the hardware with pixel shaders, this is lightyears ahead of where Canvas will be even 5 years from now. How long was Javascript out before anyone bothered to optimize their JITs? 10 years?

The funny thing is that you’ll probably end up seeing HTML5 plugins being released for older browsers, which ironically defeats the core purpose of these HTML5 features in the first place.

I’m not sure if you guys remember SVG but that was supposed to be the flash killer like 7-8 years ago. Back then I even learned SVG in and out anticipating it would replace flash (and even HTML) once and for all. Despite its open standards nature, it never ever took off and got mired in standards development gridlock. I don’t even think I bothered to put my expertise in SVG on my resume.

Also, if you are going to monetize your product via ads, you are going to have to wait for the entire advertising industry to move away from Flash as well in order to deliver ads in a fully HTML5 player. Considering they haven’t even moved away from Actionscript 2 yet I doubt that it will happen any time soon.

On the plus side when the time comes for me to dabble in HTML5 it will be a breeze to pick up.

In the meantime, I wanted to share what I consider the most screwed up variable definition I ever found myself declaring:

protected var _callbacks:Vector.<Vector.<Function>>;

Arrays within Arrays are tricky when working with generics

I recently bought an AKAI APC-40 for my late night Ableton jam sessions and within moments of deciding to buy it in the first place, I thought about wiring it up to Flash. I’ve already wired up my iPhone to Flash through flOSC (sorry for not blogging about that one) and wanted to do MIDI. Turns out MIDI is a bit more convoluted than OSC, because I can’t keep myself in the TCP/IP stack. I have to kick it to the system level.

Obviously, I wasn’t the first nut to think of this one. In fact, there have been many attempts to do just that. The only problem is those attempts involved MAX/MSP, an expensive program. There is a blogger out there who did WAY more homework than me to find a totally opensource solution. Surya, over at mmmlabs, figured out a method to do it with opensource software. The method includes installing Red5, which impressively decided to include a level of MIDI support in broadcasting NetStreams. I specialize in NetStreams at Current TV so this should be cake.

It will take a couple days to get a proper gateway to the APC going, and when I do I’ll of course post my source-code. I just wanted to let you all get a head start Happy coding!

So true..

Making a rotated gradient in Actionscript is not a trivial task. It turns out there’s only one method to do it that works correctly for me (all other Matrix type voodoo produce funky results). This did exactly what I wanted though:

var matrix:Matrix = new Matrix();

// This line of code is the magic ingredient:
matrix.createGradientBox(_h, _w, (0.5*Math.PI), 0, 0);

with(_bg.graphics) {
clear();
beginGradientFill("linear", [0x656565,0x323232], [100,100], [0,255], matrix);
drawRect(0,0,_w,_h);
endFill();
}


In the createGradientBox, I am doing two things. First I’m creating a box with the precise dimensions of my final gradient, except pre-rotated (flipping width and height) so that when the gradient draws itself out, it does so in its normal way (horizontally across). Secondly, I’m rotating it 90 degrees (or 1/2 PI in radians), so that the width and height flip back to normal as a result of the rotation. What this also accomplishes, however, is it rotates the horizontally rendered gradient to be vertical.

Hi Adobe!

Thanks for making such a fun technology. I don’t get access to the good stuff because I focus primarily on Flash, but it’s still a cool challenge to work under the constraints to make Flash effects no one has ever seen before.

It’s clear that the Toolkit was meant as a simple IDE for Pixel Bender, and it works pretty well. I just have some requests for the next version. You probably have these all jotted down, but just in case you haven’t thought of one yet:

• Code Snippet Generator: It would be great to click a button to generate some boilerplate actionscript to instantiate a compiled Pixel Bender and supply it with the currently selected parameter values from the right interface.
• Mapping “Load Image” functionality to actual image input variable names. I don’t know if there is a bug but sometimes I experience oddness when trying to load the right images in the right order? I think the Toolkit gets confused as to which Image # is assigned to which image input parameter. If I could just assign an image based on the input param name I chose, that would solve that.
• This might exist, but is there a way to define a parameter so that the individual vector elements can be assigned names? So for a float3, rather than [0], [1], [2] I could choose “R” “G” and “B”, etc. Better yet, actual properties of a dynamic object, so I can assign shader.data.myParam.value = {x: 30, y: 30, z: 30} rather than [30, 30, 30]. I know this would be a change to the Shader object model but hey, I can dream right?
• The ability to choose an image as a background behind the pixel bender outputs rather than just a background color. Or better yet, a bonafied SWF file in Play mode.
• In Preferences, I’d like to set a 3rd folder for exporting Pixel Bender kernels for flash (PBJ). I keep the PBK and PBJ files in totally different areas so it would improve my workflow a lot if I could control these separately.

That’s it … for now. Muahaha.

David Lenaert has shared some really insightful performance tips for Pixel Bender over at his blog Der Schmale. David works in Flash like me, so they’re specific to making Pixel Benders for this environment. Some of his tips include:

• Only  use 4 channels image inputs if you really need them (transparencies)
• BitmapData is faster than Vector.<Number> (This one surprised me!)
• Conditionals are expensive! When using them to try and improve performance, they can actually be slower than just the calculating useless pixels anyway.
• Offloading outCoord() to a variable doesn’t improve performance (Interesting!)

Thanks David for the good tips!

I just completed a new Pixel Bender effect that I’m calling “Acid Blend” .. it is not a real “blendShader” because it does not act on the foreground  image, it only uses the inverse of the foreground’s alpha channel to adjust the alpha of the background image.

The end result is a quick way to do a reverse mask, or create stencils that “eat away” at the alpha of your image.

The plugin sucks without a good stencil, so the quickest way to make a stencil is create a photoshop image with a transparent background, apply a stencil layer of your choosing, and save it out as a 24-bit PNG. Use the file as the src2 input and tweak the parameters to watch it in action. Note, if both images have full alpha opacity, there is no effect. That’s why the stencil needs to have a transparent bg.

After some experimentation, this plugin is really simple but can yield cool results if you are creative with the stencils and the tweening of the rotation/offset parameters. Here’s a demo I whipped up to show it in action (again, with a GlowFilter for added oomph). It’s mildly mouse reactive but I was too lazy to get the mappings just right. If you go counter-clockwise with your mouse in the upper left quadrant of your browser window you should see it react pretty well. Just wanted to show how dynamic the masking can be.

And here’s the source code to the PixelBender Kernel:

AcidBlend PixelBender Kernel Source

If you’re anxious to get started with a stencil, I’ve attached the one I used in the above demo:

Example Stencil

As usual, hit me up with any suggestions for improvements or questions. Always happy to help!