First of all let's check if your solution is really a solution of differential equation:

Let:

We can see that:

where denotes unit step function.

where denotes Dirac delta function.

Hence:

So I can see that this solution is valid...

You can express it by if you use "integration by parts" formula:

So:

I did check the validity of your solution because it was unusual. Usually we assume that signals are casual, which means that the circuit response (with zero initial conditions) is equal:

Noting that circuit response due to excitation x(t) is convolution of x(t) and impulse response h(t) we can see that: