Let the three consecutive terms of AP be: a , a+d , a + 2d where a is the first term and d the common difference.
Now the sum of these terms = 3a+3d = 18 hence a+d=6 OR a = 6-d------- (1)
Sum of the squares of the terms is, a^2 + (a+d)^2 + ( a+2d)^2 = 396 [ given ]
Using (1) we get (6-d)^2 + (6-d+d)^2 + (6-d+2d)^2 = 396
(6-d)^2 + 36 + (6+d)^2 = 396
that is 2 * 36 + 2*d^2 + 36 = 396 [ (x+y)^2 + (x-y)^2 = 2x^2 + 2y^2 ]
Thus we have 2d^2 = 144 and that gives d = + - 12
Now you can finish by getting the value of a by plugging in value of d in equation (1)