Elementary Question about Relations

I haven't done relations for a while, so when I went back to my textbook to 're-learn' what they were I got slightly confused.

It says a relation on A is defined of subsets of AxA.

I then go to the definition of reflexive, which says R is reflexive on A iff for all X that is an element of A, xRx.

Ok consider A = {x,y}

Then are the following different relations reflexive? I don't see why not

R1=(x,x)

R2=(y,y)

R3=(y,y) (x,x)

R4=(y,y) (x,x) (x,y)

R5=(y,y) (x,x) (y,x)

R6=(y,y) (x,x) (y,x) (y,x)

So when it says all a such that (a,a) is in R to be reflexive, does it mean all A in the relation, or all A in the initial set? Thanks