FIND THE GENERAL SOLUTION OF THE LINEAR SYSTEM
x'' = 6x + 2y, y'' = 3x + 7y
Define u= x' and v= y'. Then the differential equation x"= 6x+ 2y becomes u'= 6x+ 2y and y"= 3x+ 7y becomes v'= 3x+ 7y.
So instead of two second order equations we have four first order equations:
x'= u, y'= v, u'= 6x+ 2y, and v'= 3x+ 7y which we could also write as the single first order matrix equation:
$\displaystyle \frac{d\begin{bmatrix}x \\ y \\ u \\ v\end{bmatrix}}{dt}= \begin{bmatrix} 0 & 0 & 1 & 0 \\ 0 & 0 & 0& 1 \\ 6 & 2 & 0 & 0 \\3 & 7 & 0 & 0\end{bmatrix}\begin{bmatrix}x \\ y \\ u \\ v\end{bmatrix}$
And the first step in that is to find the eigenvalues and eigenvectors of the coefficient matrix.
As the title of the post suggests, if we let $\displaystyle y = \frac{x''-6x}{2}$ (**) and eliminate y from the second ODE we obtain $\displaystyle x^{(4)} - 12x'' + 36 = 0$. Its characteristic equation is $\displaystyle m^4 - 13m^2 + 36 = 0 $ or $\displaystyle (m^2-4)(m^2-9)=0$ which has solution $\displaystyle m = \pm2, \pm3 $ and so the solution is
$\displaystyle
x = c_1 e^{-2x} + c_2 e^{2x} + c_3 e^{-3x} + c_4 e^{3x}
$
Once you have this, sub into your y above (**).