Have you at least tried drawing a picture for this situation?
First prove the result in case f and g differ at only one point. Then an easy induction finishes it.
Next prove that the function h which is non-zero at only one point of [a,b] is Riemann integrable, with integral 0.
Finally, set h=g-f where g and f differ at only one point.
The only part that requires "epsilon delta" is the 2nd line.