The identity element is always invertible, marked in green. Check the elements marked in red; there is no element other than 2 that is invertible, only left-invertible or right-invertible.

Edit: I should clarify: even though the element 4 has both a right and left inverse, since it does not have a two sided inverse, it is not invertible. This actually proves that associativity does not hold; notice that (3 * 4) * 1 does not equal 3 * (4 * 1).