I think that you have more or less answered your own question. The point seems to be that an element of F(M,m) is not just a function but a pair consisting of a function and its domain. There is no problem defining addition: if f is defined on the neighbourhood U of m, and g is defined on V, then their sum is f+g defined on Similarly, you would want the negative of f to be –f defined on the same domain as f. But what is the neutral element going to be? It should obviously be the zero function, but there is no way to define its domain in such a way that f + (–f) = 0.
For that reason, the usual way to deal with the set of functions that are smooth on some neighbourhood of m is to define an equivalence relation on them, saying that two such functions are equivalent if they agree on some neighbourhood of m. The set of equivalence classes then forms a linear space, the space of germs of smooth functions at m.