Fejér's theorem. But if all you know about f is that it is Riemann integrable, then you have to follow girdav's advice and approximate f by a simple function.
The R–L lemma is easy to prove for the characteristic function of an interval (by explicitly calculating the integral). The result then follows for a simple function (which is just a finite linear combination of characteristic functions). Finally, to approximate a Riemann integrable function by a simple function, take a Riemann sum corresponding to a suitably fine dissection of the interval.