You have an overdetermined, inconsistent system of equations there. There is no solution. If you had a plus 6 in the last equation instead of a minus 6, you'd have an overdetermined consistent system. I don't understand what you mean when you say 'The determinant of the coefficients of these three lines is zero". The determinant is only defined for square matrices, but you have an overdetermined system. Therefore, the determinant is not defined. Exactly what matrix are you taking the determinant of?