Try using powers in parentheses. This expression is equivalent to
$\displaystyle ((a/b) \cdot x)^{1/2}$
where
$\displaystyle x = ((b/a) \cdot (a/b)^{1/2})^{1/2}$.
Let's evaluate x.
$\displaystyle x = ((a/b)^{-1} \cdot (a/b)^{1/2})^{1/2}$
$\displaystyle x = ((a/b)^{-1/2})^{1/2}$
$\displaystyle x = (a/b)^{-1/4}$
Now for the original expression:
$\displaystyle ((a/b) \cdot (a/b)^{-1/4})^{1/2}$
$\displaystyle ((a/b)^{3/4})^{1/2}$
$\displaystyle (a/b)^{3/8}$
Hence k = 3/8.