sometimes these are called the elementary matrices Ei,j when the i,j-th entry is 1 and all other entries are 0. these form a basis for the vector space of all nxn matrices over a field.
Those certainly do form a basis for the space of n by n matrices but that is not my understanding of "elementary matrix". An elementary matrix is one that is derived from the identity matrix by a single row operation.
See definition 3 in Pauls Online Notes : Linear Algebra - Inverse Matrices and Elementary Matrices
if one wishes to identify M(mxn)(F) with F^(mn), then these are just the standard basis vectors of F^(mn) (well, the image under the inverse isomorphism of the standard basis-that's a mouthful).
i didn't invent, or make up this usage: for example, see the footnote on page 6 here: http://www.colorado.edu/engineering/.../IFEM.AppD.pdf