well, you are correct. if b is an element of F, then b isn't "really" an element of F[x]/(p(x)), the actual element of F[x]/(p(x)) is b + (p(x)).
however, the field E obtained in such a way is isomorphic to F(α), where F(α) is created from F by "adjoining α, a root of p(x)".
here is a simple example: x^2 - 2 is irreducible over Q (it has no rational roots). now, we can construct Q[x]/(x^2-2), whose elements are all of the form:
(a + (x^2-2)) + (bx + (x^2-2)) = a+bx + (x^2-2). so we form the following isomorphism between Q(√2) and Q[x]/(x^2-2):
a+b√2 <--> a+bx + (x^2-2).
note that this map is totally determined by the images of 1, and √2. so we would expect that 1 + (x^2-2) is the multiplicative identity of Q[x]/(x^2-2),
and that x + (x^2-2) is an element that when squared, is the image under the isomorphism of 2: to make the algebra simpler,
i will write I for the principal ideal generated by x^2-2.
(x+I)^2 = (x+I)(x+I) = x^2 + I = x^2 + 2 - 2 + I = 2 + (x^2-2) + I = 2 + I (because x^2-2 is IN I).
so the coset x+I really does act like "a square root of 2".
in general, algebraic objects are only considered "up tp isomorphism". isomorphism is an equivalence relation on the set (more properly class, because of "free objects")
of all groups/rings/fields, etc. and since we are only interested in how things objects behave, equality is too strict a requirement.
for example, the complex field is often regarded as R^2 with a certain multiplication. well, as a complex vector space,
C is of dimension one, whereas as a real vector space, R^2 is of dimension 2. so clearly C and R^2 aren't "the same thing".
in fact, you have been doing this kind of "identification" all your life. consider the field of quotients of the integers:
a fraction is often regarded as a/b, and integers are "identified with" the fractions k/1 (for an integer k).
not only is k/1 a PAIR of numbers, this pair isn't even unique: k/1 = 2k/2 = 3k/3......etc.
so integers are not the same sort of thing as rational numbers, but we agree to identify the integers with the subring of Q they are isomorphic to.