How do we prove that if a matrix has right inverse, then it's the left inverse too. Is it true in all cases?

I've read one beginner level book about algebra and it approaches the problem by defining inverse A' of an invertible matrix A to be such matrix that statisfies both AA'=I and A'A=I. Now this type of definition implies that there may be such matrices A and B such that AB=I but BA<>I. I'm not sure how I can discover two such matrices, so providing example by myself is off.

I've read some abstract algebra too. It seems that proving that all nxn invertible matrices (ones that have right inverse) form a group under matrix multiplication may do the job (we do get cancelation laws), but...

If A has right inverse A' then A' must have it's own right inverse to statisfy the group axioms. How do I prove that such one does exist?