I have an optimisation problem as follows:

min ||PAP' - B||^2_F
s.t. P'P = I

where " ' " means transpose, A, B, P are real n*n matrices, || . ||_F is the Frobenius norm, A,B are symmetric positive semi-definite matrices, and P is an unknown orthogonal matrix.

By introducing Langrange multipliers, differentiating and equating to zero, I obtain 2BPA = -P(L' +L) where L is a matrix of Langrange multipliers. At this point I get stuck. I can rearrange to obtain 2BP + P(L'+L)A^-1 = 0 which looks a bit like the Sylvester equation except (L'+L) is unknown.

I would be very grateful for any help. Thanks in advance!