1. ## Generalized Rayleigh Quotient

Given the Generalized Rayleigh Quotient
Q(x) = (Ax dot x)/(Bx dot x)
and that A and B are both real n x n matrices, and B is positive definite.
How would I go about proving that Q(x) has at least one critical point. In class we restricted ||x|| to 1 so that ||x||^2 also is 1, but we never actually see ||x||^2 in this equation, so it doesn't seem like that would be helpful. If someone can please help me with where to start that'd be awesome! I know that taking the gradient is the first step, but after that I'm utterly confused.

2. ## Re: Generalized Rayleigh Quotient

Originally Posted by tubetess123
Given the Generalized Rayleigh Quotient Q(x) = (Ax dot x)/(Bx dot x) and that A and B are both real n x n matrices, and B is positive definite
Surely you forgot to say that $\displaystyle A,B$ are symmetric. Hint: Use the substitution $\displaystyle x=Py$ where $\displaystyle P\in \mathbb{R}^{n\times n}$ is a non singular matrix such that $\displaystyle P^tAP=D=\textrm{diag}\;(\lambda_1,\ldots,\lambda_n )$ and $\displaystyle P^tBP=I$ being $\displaystyle \lambda_j$ the generalized eigenvalues associated to $\displaystyle A$ and $\displaystyle B$ .

3. ## Re: Generalized Rayleigh Quotient

We aren't given that A is positive definite. Also, why does substituting that particular matrix into the equation prove there is a critical point?

4. ## Re: Generalized Rayleigh Quotient

Originally Posted by tubetess123
We aren't given that A is positive definite.
I meant symmetric.

Also, why does substituting that particular matrix into the equation prove there is a critical point?
Suppose $\displaystyle \lambda_1\leq\lambda_2\leq\ldots\leq\lambda_n$ , then

$\displaystyle Q(x)=\dfrac{x^tAx}{x^tBx}=\frac{(Py)^tA(Py)}{(Py)^ tB(Py)}=\dfrac{y^tP^tAPy}{y^tP^tBPy}=\dfrac{y^tDy} {y^tIy}=$

$\displaystyle \frac{\lambda_1y_1^2+\ldots+\lambda_ny_n^2}{y_1^2+ \ldots+y_n^2} \geq \lambda_1\quad (\forall y=(y_1,\ldots,y_n)^t\in\mathbb{R}^n)$

If $\displaystyle y_0=(1,0,\ldots,0)^t$ and $\displaystyle x_0=Py_0$ then, $\displaystyle Q(x_0)=\lambda_1\geq Q(x)$ for all $\displaystyle x\in\mathbb{R}^n$ . This implies that $\displaystyle x_0$ is a point of minimum for $\displaystyle Q$ and as a consequence, a critical point of $\displaystyle Q$ .

5. ## Re: Generalized Rayleigh Quotient

Thank you for that explanation. Also, how do you make the nice equations show up instead of what I had shown above?
And is matrix Py a column vector? Because to do the dot product we need two column vectors. And why can we go from (Py)^tB(Py) to y^Py? Why can we change the order of y^t and P^t and why does the B change to A?

6. ## Re: Generalized Rayleigh Quotient

The next thing I need to do is prove that x is a CP of Q if and only if Ax = (lambda)Bx for some real lambda. I see that you've already proved one portion of this, but I don't know how I'd go about proving the other part?

Thanks!

7. ## Re: Generalized Rayleigh Quotient

So in your hint above, do you mean to say that $\displaystyle P^{t}BP = I$ and that $\displaystyle \lambda _{j}$ are the eigenvalues of A? It doesn't make sense to me if this is not the case.

Thanks!

8. ## Re: Generalized Rayleigh Quotient

Originally Posted by tubetess123
Also, how do you make the nice equations show up instead of what I had shown above?
$\displaystyle (Ax)\cdot x=(Ax)^tx=x^tA^tx=x^tAx$ . The same for $\displaystyle B$ .

And is matrix Py a column vector? Because to do the dot product we need two column vectors.
Yes, it is, $\displaystyle y=(y_1,\ldots,y_n)^t$ is a column vector, so $\displaystyle Py$ is also a column vector.

And why can we go from (Py)^tB(Py) to y^Py?
We can't. It is $\displaystyle (Py)^tB(Py)=y^tP^tBPy=y^tIy=y^ty=y_1^2+y_2^2+...+y _n^2$

Why can we change the order of y^t and P^t
Well known property: $\displaystyle (MN)^t=N^tM^t$ .

and why does the B change to A?
A typo on my part. I've just corrected it.

9. ## Re: Generalized Rayleigh Quotient

Thanks! Those clarifications are great help!

Last thing, could you help me prove that if all the above given information is true, and that at a critical point x it's true that $\displaystyle Ax = \lambda Bx$, that the determinant of $\displaystyle A-\lambda B = 0$. I don't know how I'd go about showing this. Not sure if I said this, but x can't be 0.

10. ## Re: Generalized Rayleigh Quotient

Originally Posted by tubetess123
Thanks! Those clarifications are great help!
Last thing, could you help me prove that if all the above given information is true, and that at a critical point x it's true that $\displaystyle Ax = \lambda Bx$, that the determinant of $\displaystyle A-\lambda B = 0$. I don't know how I'd go about showing this. Not sure if I said this, but x can't be 0.
$\displaystyle \exists x\neq 0:Ax=\lambda Bx\Leftrightarrow \exists x\neq 0: (A-\lambda B)x=0$

$\displaystyle \Leftrightarrow \textrm{rank}\;(A-\lambda B)<n \Leftrightarrow \det (A-\lambda B)=0$