In that case "formally" means that you treat it as an algebraic object, ignoring convergence issues.

See here and here

Say, pick the generating function ; as a series it converges for only and thus it'd make no sense to talk about in general, however one can treat it as an algebraic object. It'd be more like a symbolic manipulation, you define certain operations over these ojects and manipulate them, but you can't actually evaluate them at a given number.

Of course, having convergence adds some nice things.

The reason you can't just differentiate is that exchanging limit and integral doesn't always yield the same result, but under suitable conditions this happens.