libxslt is an interesting attack surface; there are various places in which it is used to process untrusted stylesheets. This includes some browsers, although namespace issues seem to prevent the affected code from being reached in a browser context.
Within libxslt itself, there are some built-in functions. These are usually a fruitful place to look for vulnerabilities, particularly those that take integers etc. In this instance, I found problems in a little used cryptography related extension function. An incoming string is over-trusted in that its length is not sanitized, leading to a heap overflow.
XSLT, surprisingly, is turing-complete, even in its currently deployed incarnations (although you need to implement looping via recursion). There may be interesting DoS and further exploitation opportunities here.
Full technical details can be found here: