first of all, do you mean "circle" when you say "oval"? And in your example A and B don't lie on a circle centered at O; is it a mistake?
If you're looking for the distance along the circle (centered at O) between two points A and B on it, your formula is not quite correct. It should just be:
(this is a consequence of )
Does it seem OK?