However I can't seem to get virtualdub/avisynth to let me open the .264 file. I have ffdshow installed but virtualdubmod won't open it.

It's time for an H.264 tune-up. Lean how choose the right encoding tool and optimize H.264 encodes for ideal quality and device compatibility.

iSquint, the freeware iPod video converter utility for Mac OS X, now supports 640 x 480 resolution H.264 video output.