For non-free software, I like Mathematica for symbolic computations, and MATLAB for numerical computations. For free software, you could go with Octave (for MATLAB-like capabilities) or Maxima or Sage (CAS's). Here's a page comparing CAS's. Note that Sage is not natively supported for Windows, although you can still run it using virtualization.