The answer is 16.

Number the vertices of the octagon, , consecutively so that are adjacent as well for .

There are 20 subsets of three which contain no two consecutive integers.

But those 20 also contain a set like . We don’t want that because is an edge of the octagon.

So, how many such subsets must we remove from the 20?