1.96 is a N(0,1) percentile that gives you 95 precent coverage in your intervals, i.e.,
P(-1.96<Z<1.96)=.95. So, that's for a two-sided interval with probability .95.
Now the standard error is the estimate of the standard deviation.
For example, the variance of the sample mean, for i.i.d. rvs is... .
Hence the standard deviation of is .
BUT we usually don't know , so we estimate the st deviation with this standard error.
Our estimate of is S so the standard error of is our standard error.