Generalized Rayleigh Quotient

**tubetess123** · November 19th 2011, 10:40 PM

Given the Generalized Rayleigh Quotient
Q(x) = (Ax dot x)/(Bx dot x)
and that A and B are both real n x n matrices, and B is positive definite.
How would I go about proving that Q(x) has at least one critical point. In class we restricted ||x|| to 1 so that ||x||^2 also is 1, but we never actually see ||x||^2 in this equation, so it doesn't seem like that would be helpful. If someone can please help me with where to start that'd be awesome! I know that taking the gradient is the first step, but after that I'm utterly confused.

**FernandoRevilla** · November 20th 2011, 01:52 AM

Originally Posted by tubetess123

Given the Generalized Rayleigh Quotient Q(x) = (Ax dot x)/(Bx dot x) and that A and B are both real n x n matrices, and B is positive definite

Surely you forgot to say that $A,B$ are symmetric. Hint: Use the substitution $x=Py$ where $P\in \mathbb{R}^{n\times n}$ is a non singular matrix such that $P^tAP=D=\textrm{diag}\;(\lambda_1,\ldots,\lambda_n )$ and $P^tBP=I$ being $\lambda_j$ the generalized eigenvalues associated to $A$ and $B$ .

**tubetess123** · November 20th 2011, 06:41 AM

We aren't given that A is positive definite. Also, why does substituting that particular matrix into the equation prove there is a critical point?

**FernandoRevilla** · November 20th 2011, 08:20 AM

Originally Posted by tubetess123

We aren't given that A is positive definite.

I meant symmetric.

Also, why does substituting that particular matrix into the equation prove there is a critical point?

Suppose $\lambda_1\leq\lambda_2\leq\ldots\leq\lambda_n$ , then

$Q(x)=\dfrac{x^tAx}{x^tBx}=\frac{(Py)^tA(Py)}{(Py)^ tB(Py)}=\dfrac{y^tP^tAPy}{y^tP^tBPy}=\dfrac{y^tDy} {y^tIy}=$

$\frac{\lambda_1y_1^2+\ldots+\lambda_ny_n^2}{y_1^2+ \ldots+y_n^2} \geq \lambda_1\quad (\forall y=(y_1,\ldots,y_n)^t\in\mathbb{R}^n)$

If $y_0=(1,0,\ldots,0)^t$ and $x_0=Py_0$ then, $Q(x_0)=\lambda_1\geq Q(x)$ for all $x\in\mathbb{R}^n$ . This implies that $x_0$ is a point of minimum for $Q$ and as a consequence, a critical point of $Q$ .

**tubetess123** · November 20th 2011, 09:28 AM

Thank you for that explanation. Also, how do you make the nice equations show up instead of what I had shown above?
And is matrix Py a column vector? Because to do the dot product we need two column vectors. And why can we go from (Py)^tB(Py) to y^Py? Why can we change the order of y^t and P^t and why does the B change to A?

**tubetess123** · November 20th 2011, 09:59 AM

The next thing I need to do is prove that x is a CP of Q if and only if Ax = (lambda)Bx for some real lambda. I see that you've already proved one portion of this, but I don't know how I'd go about proving the other part?

Thanks!

**tubetess123** · November 20th 2011, 08:46 PM

So in your hint above, do you mean to say that $P^{t}BP = I$ and that $\lambda _{j}$ are the eigenvalues of A? It doesn't make sense to me if this is not the case.

Thanks!

**FernandoRevilla** · November 20th 2011, 09:47 PM

Originally Posted by tubetess123

Also, how do you make the nice equations show up instead of what I had shown above?

$(Ax)\cdot x=(Ax)^tx=x^tA^tx=x^tAx$ . The same for $B$ .

And is matrix Py a column vector? Because to do the dot product we need two column vectors.

Yes, it is, $y=(y_1,\ldots,y_n)^t$ is a column vector, so $Py$ is also a column vector.

And why can we go from (Py)^tB(Py) to y^Py?

We can't. It is $(Py)^tB(Py)=y^tP^tBPy=y^tIy=y^ty=y_1^2+y_2^2+...+y _n^2$

Why can we change the order of y^t and P^t

Well known property: $(MN)^t=N^tM^t$ .

and why does the B change to A?

A typo on my part. I've just corrected it.

**tubetess123** · November 20th 2011, 11:10 PM

Thanks! Those clarifications are great help!

Last thing, could you help me prove that if all the above given information is true, and that at a critical point x it's true that $Ax = \lambda Bx$ , that the determinant of $A-\lambda B = 0$ . I don't know how I'd go about showing this. Not sure if I said this, but x can't be 0.

**FernandoRevilla** · November 21st 2011, 12:14 AM

Originally Posted by tubetess123

Thanks! Those clarifications are great help!
Last thing, could you help me prove that if all the above given information is true, and that at a critical point x it's true that $Ax = \lambda Bx$ , that the determinant of $A-\lambda B = 0$ . I don't know how I'd go about showing this. Not sure if I said this, but x can't be 0.

$\exists x\neq 0:Ax=\lambda Bx\Leftrightarrow \exists x\neq 0: (A-\lambda B)x=0$

$\Leftrightarrow \textrm{rank}\;(A-\lambda B)<n \Leftrightarrow \det (A-\lambda B)=0$

Thread: Generalized Rayleigh Quotient

LinkBack

Thread Tools

Display

Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Re: Generalized Rayleigh Quotient

Similar Math Help Forum Discussions

MGF for the Rayleigh distribution

Using the Rayleigh Quotient

Rayleigh distribution sufficiency

Rayleigh Quotient iteration

eigenvalues and eigenvectors (Rayleigh quotient method)