Reversing the order of integration, as both Idea and Prove It said is the best way to handle this integral. But to answer you last question, "how to integrate x arcsinh(x^2)" let u= x^2 so \(\displaystyle \int x arcsinh(x) dx\(\displaystyle becomes \(\displaystyle \frac{1}{2}\int arcsinh(u)du\) and that can be looked up in a table of integrals.

(

Lists of integrals - Wikipedia, the free encyclopedia)\)\)

\(\displaystyle \(\displaystyle

Or otherwise derived (note I've left out the integration constants until the end)...

Let's consider $\displaystyle \begin{align*} \int{ \frac{2x}{\sqrt{1 + x^2}} \, dx} = \int{ 2x \left( 1 + x^2 \right) ^{-\frac{1}{2}} \,dx} \end{align*}$. With the substitution $\displaystyle \begin{align*} u = 1 + x^2 \implies du = 2x\,dx \end{align*}$ we can see $\displaystyle \begin{align*} \int{ 2x \left( 1 + x^2 \right) ^{-\frac{1}{2}} \, dx} = \int{ u^{-\frac{1}{2}} \, du } = 2u^{\frac{1}{2}} = 2 \, \sqrt{ 1 + x^2 } \end{align*}$.

We might also consider integration by parts:

$\displaystyle \begin{align*} \int{ \frac{2x}{\sqrt{1 + x^2}} \, dx} &= \int{ 2x \, \frac{1}{\sqrt{1 + x^2}} \, dx} \\ &= 2x\,\textrm{arsinh}\,{(x)} - \int{ 2\,\textrm{arsinh}\,{(x)}\,dx } \\ &= 2x\,\textrm{arsinh}\,{(x)} - 2\int{ \textrm{arsinh}\,{(x)}\,dx } \end{align*}$

and thus

$\displaystyle \begin{align*} 2x\,\textrm{arsinh}\,{(x)} - 2\int{ \textrm{arsinh}\,{(x)}\,dx} &= 2\,\sqrt{1 + x^2} \\ x\,\textrm{arsinh}\,{(x)} - \int{ \textrm{arsinh}\,{(x)}\,dx} &= \sqrt{1 + x^2} \\ \int{\textrm{arsinh}\,{(x)}\,dx} &= x\,\textrm{arsinh}\,{(x)} - \sqrt{1 + x^2} + C \end{align*}$\)\)