It doesn't need to be continuous. You SAY you are thinking about the neighborhood, but your wording suggests you are still thinking about the single value as if it were continuous.
Move you mind away from the the point and out to the neighborhood. Consider the left and right limits. Do they exist? Are they the same?