Ok, the history of e starts with a limit, . Mathematicians (Bernoulli, Euler, Leibniz) tried to calculate the limit but couldn't express the number in an elementary way. So this was a new constant, today we call it e which is .
, the exponential function is the derivative of itself. So we use e while solving the differential equation, .
You can also see that we need e while integrating . It's known that . But the integral of 1/x is the logarithm function to the base e, which is the inverse of the exponential function.