Prove that cos (2π/13)+ cos (6π/13) + cos (8π/13)= (sqrt(13)-1)/4
Outline proof: For $\displaystyle 0\leqslant k\leqslant 12$, let $\displaystyle A_k$ denote the angle $\displaystyle 2k\pi/13$. The points $\displaystyle (\cos A_k,\sin A_k)$ are equally spaced around the unit circle, so their centre of mass is at the origin, and it follows that $\displaystyle \sum_{k=0}^{12}\cos A_k = 0.$
Since $\displaystyle \cos A_0 = 1$ and $\displaystyle \cos A_{13-k} = \cos A_k$, it follows that $\displaystyle (\cos A_1 + \cos A_3 + \cos A_4) + (\cos A_2 + \cos A_6 + \cos A_8) = -1/2.$
Next,
$\displaystyle \begin{aligned}(\cos A_1 + \cos A_3 + \cos A_4)^2 = &(\cos^2 A_1 + \cos^2 A_3 + \cos^2 A_4) \\ &+ 2(\cos A_3\cos A_4 + \cos A_4\cos A_1 + \cos A_1\cos A_3)\end{aligned}$.
But $\displaystyle \cos^2 A_1 = \tfrac12(\cos A_2 + 1)$ (with similar results for $\displaystyle \cos^2 A_3$ and $\displaystyle \cos^2 A_4$). Also, $\displaystyle 2\cos A_3\cos A_4 = \cos(A_3+A_4) + \cos(A_3-A_4) = \cos A_7 + \cos A_1$ (with similar results for the other two products $\displaystyle 2\cos A_4\cos A_1$ and $\displaystyle 2\cos A_1\cos A_3$).
Putting all that together, you find that
$\displaystyle \cos^2 A_1 + \cos^2 A_3 + \cos^2 A_4 = \tfrac12(\cos A_2 + \cos A_6 + \cos A_8) + 3/2$, and
$\displaystyle 2(\cos A_3\cos A_4 + \cos A_4\cos A_1 + \cos A_1\cos A_3) = \sum_{k=1}^6\cos A_k = -1/2.$
Now let $\displaystyle S = \cos A_1 + \cos A_3 + \cos A_4$ and $\displaystyle T = \cos A_2 + \cos A_6 + \cos A_8.$ It follows from the above calculations that $\displaystyle S+T = -\tfrac12$ and $\displaystyle S^2 = \tfrac12T+1.$ Eliminate T between those equations to get $\displaystyle 4S^2+2S-3=0$, a quadratic with solutions $\displaystyle S = \frac{-1\pm\sqrt{13}}4.$ Now all that remains for you to do is to figure out why the square root must have the plus sign.
The quantities S and T are the two roots of the quadratic equation. It should be clear from a diagram of the 13 points on the unit circle that the three cosines making up the sum T are smaller than their counterparts in S. Specifically, $\displaystyle \cos A_2 < \cos A_1$, $\displaystyle \cos A_8 < \cos A_3$ and $\displaystyle \cos A_6 < \cos A_4$. Therefore T<S and so S must correspond to the larger root of the quadratic.