First of all let's check if your solution is really a solution of differential equation:
We can see that:
where denotes unit step function.
where denotes Dirac delta function.
So I can see that this solution is valid...
You can express it by if you use "integration by parts" formula:
I did check the validity of your solution because it was unusual. Usually we assume that signals are casual, which means that the circuit response (with zero initial conditions) is equal:
Noting that circuit response due to excitation x(t) is convolution of x(t) and impulse response h(t) we can see that: