Hi (I don't know what is your math background so just that you know before reading this that I am going to use : Pytagoras and simplification of radicals, I hope it is ok with you),

A good way to find the area of the octogon is to subtract the area of the four triangles you have in the corners from the area of the square.

1) Area of square : A(sq.) = side^2 = a^2.

2) Area of triangle : A(tr.) = 1/2 b*h where b is the base and h the height. We have neither of them.

ok, lets take the triangle at the right bottom/| . We will call it ABC with A the vertex with the right angle (since this angle is one the angles of the square so its measure is 90 degrees) And let B be the vertex on the same horizontal line as A. So CA is the height of the right-angled triangle ABC and AB is the base.

If you understand that ABC is isosceles skip this paragraph. If not, lets proof that the angles at B and C are 45 degrees. In any polygon (3 or more sides), the sum of the measures of the angles is given by (c-2)*180 degrees where c is the number of sides. So for an octogon we have (8-2)*180=1080 degrees. Now, since it is regular, each angle measures 1/8 of 1080 which is 135 degrees. You can see in the figure that the angle at B is next to one of the angles of the octogon (lets call it D) with D+B=180 degrees so B=180-D=180-135=45 degrees. And by the same trick or by the sum of the angles in a triangle, you can find that C measures 45 degrees also.

So the triangle ABC is isosceles which gives us that the measure of AB = measure of CA.

Lets name the measure of a side of the octogon : s. So the hypotenuse of the traingle ABC has measure s (it is a side of the octogon). So by Pytagoras : s^2 = CA^2 + AB^2 = 2*AB^2 (since CA=AB)

so s^2 /2 = AB^2 and by taking square roots on each side we have s/sqrt(2) = AB. OK!

So are almost done ! now, the area of the triangle ABC is given by A(tr.) = AB*AC /2 and we know that AB=AC=s/sqrt(2) so A(tr.) = (s/sqrt(2) * s/sqrt(2)) /2 = s^2 /4. Now, we have 4 triangles like ABC so A(4 tr.) = 4*A(tr.) = s^2. Yeah !

Now, we don't want to have new unknowns added to a (the side of the square) but we can see in the figure that a = AB + s + AB (I hope you see it : if you look at the bottom side of the square you see that it equals one side of the octogon plus 2 measures of AB). So a= 2AB + s and then, knowing that AB = s/sqrt(2), we replace and find a=2s/sqrt(2) + s = sqrt(2) s + s = s(sqrt(2) + 1) by factorization. So we isolate the s so s = a /(sqrt(2) + 1)

Ok, so A(4 tr.) = (a/(sqrt(2) + 1))^2 = a^2 /(sqrt(2) + 1)^2.

A(octogon) = A(sq.) - A(4 tr.) = a^2 - a^2 /(sqrt(2) + 1)^2 = a^2 (1-1/(sqrt(2) + 1)^2) = a^2 {(sqrt(2) + 1)^2 - 1}/(sqrt(2) + 1)^2 = a^2 {(2 + 2sqrt(2) + 1) - 1}/(sqrt(2) + 1)^2 = a^2 (2sqrt(2) + 2)/(sqrt(2) + 1)^2

So A(octogon) =2a^2 (sqrt(2) + 1)/(sqrt(2) + 1)^2 = 2a^2 /(sqrt(2) + 1) which is the same as your answer but for the beauty of math we multiply the numerator and the denominator with (sqrt(2) - 1) which gives after simplification 2a^2 (sqrt(2) - 1).