Since rotation of the numbers doesn't matter, let's assume the 1 is at the upper left hand corner of the square (assuming a "square table" like in your diagram). We then have 3 numbers we can place in the remaining 3 locations in any order. This can be done in 3! ways.
So the total number of arrangements is 3! = 6.