It's impossible to edit your image and a bit hard to follow what you are doing. But...I think you are trying to calculate the orthogonal projection onto the line. So why would you expect $\begin{bmatrix} 4 \\ 3 \end{bmatrix} $ to map to $\begin{bmatrix} 2 \\ 3 \end{bmatrix} $? I haven't tried to follow all your calculations, but I think you are just missing a square root and all your $\sqrt{13}$'s should just be $13$'s. You can check that out.

But there is an easier way to approach this. You know $\begin{bmatrix} 2 \\3 \end{bmatrix} \to \begin{bmatrix} 2 \\3 \end{bmatrix}$ and it's easy to see that $\begin{bmatrix} -3 \\2 \end{bmatrix} \to \begin{bmatrix} 0 \\0 \end{bmatrix}$ under your transformation $P$. So you have $

P\begin{bmatrix} 2 & -3 \\ 3 & 2 \end{bmatrix} = \begin{bmatrix} 2 & 0 \\ 3 & 0 \end{bmatrix}$

Multiply both sides on the right by the inverse of $\begin{bmatrix} 2 & -3 \\ 3 & 2 \end{bmatrix}$ and you will have your matrix $P$.