Are you familiar with the delta-epsilon defintion of a limit? I don't want to go into it if it's not what you're looking for.
Here's a good site that proves It uses the same method so unless you want I won't type it all out.
Ok. So the only thing I don't get now as how it works for something like the example I had where the limit equals a variable (also the number the limit is approaching is a variable). I'm having trouble making the connection from the problem I had to ones with real numbers.
How do you show abs(x^2-a^2) is less than epsilon?