# Generalized Rayleigh Quotient

• Nov 19th 2011, 10:40 PM
tubetess123
Generalized Rayleigh Quotient
Given the Generalized Rayleigh Quotient
Q(x) = (Ax dot x)/(Bx dot x)
and that A and B are both real n x n matrices, and B is positive definite.
How would I go about proving that Q(x) has at least one critical point. In class we restricted ||x|| to 1 so that ||x||^2 also is 1, but we never actually see ||x||^2 in this equation, so it doesn't seem like that would be helpful. If someone can please help me with where to start that'd be awesome! I know that taking the gradient is the first step, but after that I'm utterly confused.
• Nov 20th 2011, 01:52 AM
FernandoRevilla
Re: Generalized Rayleigh Quotient
Quote:

Originally Posted by tubetess123
Given the Generalized Rayleigh Quotient Q(x) = (Ax dot x)/(Bx dot x) and that A and B are both real n x n matrices, and B is positive definite

Surely you forgot to say that $\displaystyle A,B$ are symmetric. Hint: Use the substitution $\displaystyle x=Py$ where $\displaystyle P\in \mathbb{R}^{n\times n}$ is a non singular matrix such that $\displaystyle P^tAP=D=\textrm{diag}\;(\lambda_1,\ldots,\lambda_n )$ and $\displaystyle P^tBP=I$ being $\displaystyle \lambda_j$ the generalized eigenvalues associated to $\displaystyle A$ and $\displaystyle B$ .
• Nov 20th 2011, 06:41 AM
tubetess123
Re: Generalized Rayleigh Quotient
We aren't given that A is positive definite. Also, why does substituting that particular matrix into the equation prove there is a critical point?
• Nov 20th 2011, 08:20 AM
FernandoRevilla
Re: Generalized Rayleigh Quotient
Quote:

Originally Posted by tubetess123
We aren't given that A is positive definite.

I meant symmetric.

Quote:

Also, why does substituting that particular matrix into the equation prove there is a critical point?
Suppose $\displaystyle \lambda_1\leq\lambda_2\leq\ldots\leq\lambda_n$ , then

$\displaystyle Q(x)=\dfrac{x^tAx}{x^tBx}=\frac{(Py)^tA(Py)}{(Py)^ tB(Py)}=\dfrac{y^tP^tAPy}{y^tP^tBPy}=\dfrac{y^tDy} {y^tIy}=$

$\displaystyle \frac{\lambda_1y_1^2+\ldots+\lambda_ny_n^2}{y_1^2+ \ldots+y_n^2} \geq \lambda_1\quad (\forall y=(y_1,\ldots,y_n)^t\in\mathbb{R}^n)$

If $\displaystyle y_0=(1,0,\ldots,0)^t$ and $\displaystyle x_0=Py_0$ then, $\displaystyle Q(x_0)=\lambda_1\geq Q(x)$ for all $\displaystyle x\in\mathbb{R}^n$ . This implies that $\displaystyle x_0$ is a point of minimum for $\displaystyle Q$ and as a consequence, a critical point of $\displaystyle Q$ .
• Nov 20th 2011, 09:28 AM
tubetess123
Re: Generalized Rayleigh Quotient
Thank you for that explanation. Also, how do you make the nice equations show up instead of what I had shown above?
And is matrix Py a column vector? Because to do the dot product we need two column vectors. And why can we go from (Py)^tB(Py) to y^Py? Why can we change the order of y^t and P^t and why does the B change to A?
• Nov 20th 2011, 09:59 AM
tubetess123
Re: Generalized Rayleigh Quotient
The next thing I need to do is prove that x is a CP of Q if and only if Ax = (lambda)Bx for some real lambda. I see that you've already proved one portion of this, but I don't know how I'd go about proving the other part?

Thanks!
• Nov 20th 2011, 08:46 PM
tubetess123
Re: Generalized Rayleigh Quotient
So in your hint above, do you mean to say that $\displaystyle P^{t}BP = I$ and that $\displaystyle \lambda _{j}$ are the eigenvalues of A? It doesn't make sense to me if this is not the case.

Thanks!
• Nov 20th 2011, 09:47 PM
FernandoRevilla
Re: Generalized Rayleigh Quotient
Quote:

Originally Posted by tubetess123
Also, how do you make the nice equations show up instead of what I had shown above?

$\displaystyle (Ax)\cdot x=(Ax)^tx=x^tA^tx=x^tAx$ . The same for $\displaystyle B$ .

Quote:

And is matrix Py a column vector? Because to do the dot product we need two column vectors.
Yes, it is, $\displaystyle y=(y_1,\ldots,y_n)^t$ is a column vector, so $\displaystyle Py$ is also a column vector.

Quote:

And why can we go from (Py)^tB(Py) to y^Py?
We can't. It is $\displaystyle (Py)^tB(Py)=y^tP^tBPy=y^tIy=y^ty=y_1^2+y_2^2+...+y _n^2$

Quote:

Why can we change the order of y^t and P^t
Well known property: $\displaystyle (MN)^t=N^tM^t$ .

Quote:

and why does the B change to A?
A typo on my part. I've just corrected it.
• Nov 20th 2011, 11:10 PM
tubetess123
Re: Generalized Rayleigh Quotient
Thanks! Those clarifications are great help!

Last thing, could you help me prove that if all the above given information is true, and that at a critical point x it's true that $\displaystyle Ax = \lambda Bx$, that the determinant of $\displaystyle A-\lambda B = 0$. I don't know how I'd go about showing this. Not sure if I said this, but x can't be 0.
• Nov 21st 2011, 12:14 AM
FernandoRevilla
Re: Generalized Rayleigh Quotient
Quote:

Originally Posted by tubetess123
Thanks! Those clarifications are great help!
Last thing, could you help me prove that if all the above given information is true, and that at a critical point x it's true that $\displaystyle Ax = \lambda Bx$, that the determinant of $\displaystyle A-\lambda B = 0$. I don't know how I'd go about showing this. Not sure if I said this, but x can't be 0.

$\displaystyle \exists x\neq 0:Ax=\lambda Bx\Leftrightarrow \exists x\neq 0: (A-\lambda B)x=0$

$\displaystyle \Leftrightarrow \textrm{rank}\;(A-\lambda B)<n \Leftrightarrow \det (A-\lambda B)=0$