I'm doing a moving average filter on an input from a wave-file, but the smoothing is too rough - there are too many spikes afterwards. I wonder what could be wrong, is it the window length?

This is the implementation of the filter:

Code:
% inputs: X (vector of wavefile), winlen (how long the average is)

% output:vector E, a smoothed version of the input

B=ones(1,5)/5;

E=conv(b, X(5:length(X)));

Thanks for your help!