Since $f$ is continuous on $[0,a]$ it is uniformly continuous, so there exists $\delta_1$ such that $|x-y|<\delta_1\implies|f(x)-f(y)|<\epsilon$. Similarly, there exists $\delta_2$ for the interval $[a,\infty)$.
So taking $\delta=\min\{\delta_1,\delta_2\}$ will work for $[0,\infty)$.