I think that to evaluate this integral you will have to integrate *another *function along the keyhole contour. Here's why I think this :

The integral along the big circle will go to 0 as the radius increases to infinity. Same for the integral along the small circle. That's no problem.

Just under the positive real axis, the function \(\displaystyle \log z\) is \(\displaystyle 2\pi i\) more than it was just above the positive real axis; i.e. \(\displaystyle \log e^{2\pi i}x = 2\pi i + \log x\). So, in the limit, the integral along the contour is equal to \(\displaystyle \int_0^\infty \frac{(\log x)^2}{1+x^2}dx + \int_\infty^0 \frac{(2\pi i+\log x)^2}{1+x^2}dx\), where here \(\displaystyle \log x\) is the usual (real-valued) logarithm. But the \(\displaystyle (\log x)^2\) terms cancel here! So you actually end up evaluating \(\displaystyle \int_0^\infty \frac{\log x}{1+x^2}dx\).

You might first have to evaluate \(\displaystyle \int_0^\infty \frac{\log x}{1+x^2}dx\), and then integrate \(\displaystyle \int_0^\infty \frac{(\log x)^3}{1+x^2}dx\) over the keyhole contour using the residue theorem so the \(\displaystyle (\log x)^3\) terms cancel, and then use the known integral \(\displaystyle \int_0^\infty \frac{\log x}{1+x^2}dx\) to solve for \(\displaystyle \int_0^\infty \frac{(\log x)^2}{1+x^2}dx\).