The number of distinguishable regular octagons correspond to the number of orbits under G in a G-set X.
To apply symmetry to a regular octagon, we use .
Now we need to compute for each sixteen elements g in .
. (Every regular octagon is left fixed by the identity element)
. (To be invariant under r, r^7, all edges should be the same color. There are two colors, so two choices)
. (To be invariant under r^4, all edges should be same colors (oooooooo, xxxxxxxx), or different adjacent colors (oxoxoxox, xoxoxoxo), and (ooxxooxx, xxooxxoo). Here , (oxoxoxox) denotes edge 1 (between vertex 1 and 2) is colored o, edge 2 is colored x (between vertex 2 and 3), and so on.
. Diagonal flip : (Possible choices: oooooooo, xxxxxxxx, oxoxoxox, xoxoxoxo}
After you find all |X_g| values, you need to sum up those |X_g| values
Finally, the number of orbits N (distinguishable regular octagons) can be acquired.