We have a proposed solution x:
x = \left ( \begin{array}{c} 6 \\ -8 \\ 4 \end{array} \right ) e^{-t} + 2 \left ( \begin{array}{c} 0 \\ 1 \\ -1 \end{array} \right ) e^{2t}
So
x' = - \left ( \begin{array}{c} 6 \\ -8 \\ 4 \end{array} \right ) e^{-t} + 4 \left ( \begin{array}{c} 0 \\ 1 \\ -1 \end{array} \right ) e^{2t}
Thus we need to verify the equation:
x' = \left ( \begin{array}{ccc} 1 & 1 & 1 \\ 2 & 1 & -1 \\ 0 & -1 & 1 \end{array} \right ) x
- \left ( \begin{array}{c} 6 \\ -8 \\ 4 \end{array} \right ) e^{-t} + 4 \left ( \begin{array}{c} 0 \\ 1 \\ -1 \end{array} \right ) e^{2t} =\left ( \begin{array}{ccc} 1 & 1 & 1 \\ 2 & 1 & -1 \\ 0 & -1 & 1 \end{array} \right ) \left ( \left ( \begin{array}{c} 6 \\ -8 \\ 4 \end{array} \right ) e^{-t} + 2 \left ( \begin{array}{c} 0 \\ 1 \\ -1 \end{array} \right ) e^{2t} \right )
The RHS becomes:
\left ( \begin{array}{ccc} 1 & 1 & 1 \\ 2 & 1 & -1 \\ 0 & -1 & 1 \end{array} \right ) \left ( \left ( \begin{array}{c} 6 \\ -8 \\ 4 \end{array} \right ) e^{-t} \right )+ \left ( \begin{array}{ccc} 1 & 1 & 1 \\ 2 & 1 & -1 \\ 0 & -1 & 1 \end{array} \right ) \left ( 2 \left ( \begin{array}{c} 0 \\ 1 \\ -1 \end{array} \right ) e^{2t} \right )
= \left ( \begin{array}{c} 2 \\ 0 \\ 12 \end{array} \right ) e^{-t} + 2 \left ( \begin{array}{c} 0 \\ 2 \\ -2 \end{array} \right ) e^{2t}
We can factor a 2 from each of the 3 x 1 matrices, giving:
= 2 \left ( \begin{array}{c} 1 \\ 0 \\ 6 \end{array} \right ) e^{-t} + 4 \left ( \begin{array}{c} 0 \\ 1 \\ -1 \end{array} \right ) e^{2t}
which is not x'. This shows that the given x is NOT a solution to the differential equation.
-Dan