EDIT:
This is what I was looking for - it is called `The Elementary Theory of the Category of Sets', by F. William Lawvere (who was tragically born with only a first initial).
You have things called `sets', things called `functions', and you can compose functions. The axioms are,
-compositions is associative, unital and has identities
-there exists a set with exactly one element
-there exists a set with no elements
-a function is determined by its effect on elements (f(a)=g(a) for all a then f=g)
-can form cross products of sets, AxB
-we can form the set of functions from A to B
-can form the inverse image of a function
-given a set A, the subsets correspond to functions from A into {0, 1}
-the natural numbers form a set
-every surjection has a right-inverse
-cocompleteness (don't ask).
So, basically, functions `work'; they make sense. So studying calculus makes sense, and is possible.