First we do the f and then the g, which is written g 0 f. So the final matrix is BA. So why is the first matrix transformation written and done last?
f(x) means that the function f is applied to the number x that is to its right. g(f(x)) means that g is applied to f(x), to its right. Ax means the matrix A is multiplied by the vector x to its right. BAx means that the matrix B is multiplied by the vector Ax to its right. If f(x)= Ax and g(x)= Bx then g(f(x))= g(Ax)= B(Ax)= BAx.