EDIT:

This is what I was looking for - it is called `The Elementary Theory of the Category of Sets', by F. William Lawvere (who was tragically born with only a first initial).

You have things called `sets', things called `functions', and you can compose functions. The axioms are,

-compositions is associative, unital and has identities

-there exists a set with exactly one element

-there exists a set with no elements

-a function is determined by its effect on elements (f(a)=g(a) for all a then f=g)

-can form cross products of sets, AxB

-we can form the set of functions from A to B

-can form the inverse image of a function

-given a set A, the subsets correspond to functions from A into {0, 1}

-the natural numbers form a set

-every surjection has a right-inverse

-cocompleteness (don't ask).

So, basically, functions `work'; they make sense. So studying calculus makes sense, and is possible.