Is there a difference between a constant and an unknown in this context?If I want to simplify O(n^k * sqrt ( a* n^m - n^p )) as much as possible, where a is positive constant and k,m,p are positive unknown

Yes. I would write it as n^{k + m/2}.Then, the final answer is O(n^k * sqrt (n^m)). Is this correct?