Bypassing syscall filtering technologies on Linux x86_64

For those interested in syscall filtering technologies, check out my latest advisory on how policies can be bypassed under certain circumstances:

http://scary.beasts.org/security/CESA-2009-001.html

There's a neat trick on the x86_64 kernel; this kernel supports both 32-bit and 64-bit processes, and interestingly, the syscall tables are different in either case. However, with a bit of trickery, a 64-bit process can call a 32-bit syscall (and visa versa), and confuse the syscall filter.

This was discovered whilst experimenting on a new security feature for vsftpd; a future post will go into this.