I assume by "p-group" you mean |G| is a power of a prime.

If G is a p-group and x an element in G. The order of x needs to divide |G| but since |G| is a power of a prime it means |x| is a power of a prime. Now conversely, say G has all elements power of a prime. Suppose that |G| is not a power of p, then there is an a prime q not equal to p so that q divides |G|. But then by Cauchy's theorem there is an x so that |x|=q, but that is a contradiction, therefore, |G| must be a power of p.