I was wondering: can a filter be optimised to have maximum roll off. Filters are usually specified as:

$\displaystyle |H(\omega)|^2 = \frac{1}{1 + \alpha(\omega)}$ where $\displaystyle \alpha$ is a polynomial function. I was wondering if it was possible to maximise the steepness of the roll-off by specifying the passband and stopband ripples.

What is usually done is:

$\displaystyle |H(\omega)|^2 = H(\omega)H(-\omega) = \frac{1}{1 + \alpha(\omega)}$

This gives you the polls and zeros. Then you'll need to do some optimisation of some kind.