This shall be the last topic that I will give followed by another lecture on its applications. This topic usually appears in Calculus III and it the beginning of Advanced Calculus. In Advanced Calculus this is presented in much more detail and is much more complicated. I will just mention some basic definitions and theorems.

If you look at the dictionary definition of "Calculus" it will define it as the mathematics that deals with rates of change, area and volumes of shapes. Yes, that is true from an applied math approach. But what I want you to see is that Calculus is the study of a function (real). Note everything we were doing involved dealing with a function. Thus, Calculus is the study of a function. The notation is (slightly wrong) $\displaystyle f:\mathbb{R}\to \mathbb{R}$. Meaning it takes a real number (that is represented by a bold R) and takes it (represented by an arrow) into another real number (represented by R). This is why Calculus is called "Real Analysis" because it studies real functions and real numbers. There is another type of Calulus that originated probably 100-150 years after Calculus, it is called "Complex Analysis". It is an analogue meaning, $\displaystyle f:\mathbb{C}\to \mathbb{C}$, it studies complex functions, that is, functions that take a complex number and transform it into another complex number. The notation for the set $\displaystyle \mathbb{N}=\{0,1,2,...\}$ is called the natural numbers. With that we have the following definition.

Definition:Asequenceis a function $\displaystyle f:\mathbb{N}\to \mathbb{R}$, meaning it takes a natural number and transforms it into another natural number. Note the domain of a sequence areallnatural numbers.

The standard way to write a sequence is $\displaystyle a_n$ not $\displaystyle f(n)$. Thus, if $\displaystyle a_n=n^2$ then we have,

$\displaystyle a_0=0$, $\displaystyle a_1=1$, $\displaystyle a_2=4$. The nice thing about a sequence is that we can write out all the terms,

$\displaystyle a_0,a_1,a_2,...$ in a list. The reason why I say it is nice is because you cannot do that with a real function, you cannot write out all the output values. Supprisingly you can do it with the rational values. This concept is called "countable", it has nothing to do with what we will discuss but I just wanted you to know what it means. This tell us that there are more real numbers then there are natural numbers! A concept that non-mathematicians can never accept, it is difficult for them to understand it.

When we deal with sequences it is useful to know what the limit of a sequence is. That is $\displaystyle \lim_{n\to \infty}a_n$, since we are dealing with the natural numbers we are only interested in the case $\displaystyle n\to \infty$. Thus all limits are $\displaystyle n\to \infty$. Thus, we will simply write $\displaystyle \lim a_n$. The idea is, what number a sequence is tending to.

Example 57:Consider the sequence, $\displaystyle 1,1.4,1.41,1.412,...$ then $\displaystyle \lim a_n = \sqrt{2}$.

Definition:If a sequence $\displaystyle a_n$ has a limit then itconverges. Otherwise it diverges.

Do not be confused with the word "diverges". In English it means entropy, something that expands. Hence you would assume something like $\displaystyle \lim n^2$ diverges because it gets larger and larger without bound. When this happens we say it diverges to infinity and write $\displaystyle \pm \infty$ whenever appropriate. Thus, $\displaystyle \lim n^2 = +\infty$ (note itdoesnot mean the limit exists, it rather means it does not with the special property of growing without bound. The sequence $\displaystyle (-1)^n$ which is $\displaystyle 1,-1,1,-1,..$ does not tend to a value, thus $\displaystyle \lim (-1)^n$ does not exist, nor does it not diverge to infinity.

One thing about math is that definitions are just as important as theorems, and in some cases even more important! Calculus/Analysis is a classic example. The foundation of Calculus is the meaning of a sequence and its limit. Yes, we understand what it means intuitively, but how do we define "tending to", "getting closer to", "approaching"? For a mathematician, these terms need to be defined. Below I will present the defintion of this.

Definition:Given a sequence $\displaystyle a_n$, we say it has a limit $\displaystyle \lim a_n = L$ when: for any $\displaystyle \epsilon>0$ there exists a natural number $\displaystyle N$ such that $\displaystyle |a_n-L|<\epsilon$ for all $\displaystyle n\geq N$.

Wow! That looks dangerous. You probably never seen a more complicated definition in you life. This definition is attributed to Cauchy and Weierstrauss. Behind this ugly looking definition is one simple and beautify idea. First when we say two numbers $\displaystyle \alpha $ and $\displaystyle \beta$ are close to each other we mean that their difference is very small. Since we do not know which one is larger we use absolute value. Thus, $\displaystyle |\alpha - \beta|$ is small. When we say a sequence converges to a number what we are saying in other words is that the difference between the sequence $\displaystyle a_n$ and its limit value $\displaystyle L$ (if it exists) is small. Thus, $\displaystyle |a_n-L|<\epsilon$, where epsilon is some small number. Thus, let me repeat. We have a sequence $\displaystyle a_n$ and a limit value $\displaystyle L$ it means $\displaystyle |a_n-L|$ can be made as close as possible, that is, choosing a smaller and smaller $\displaystyle \epsilon$ for sufficiently large $\displaystyle n$, that is $\displaystyle n\geq N$ for some big natural number $\displaystyle N$. We will not be using this definition in the lecture, I mentioned it because I want you to see the mathematical side of Calculus. All theorems and all rules that are from Calculus are based on this and other similar definitions. Hence the most important result in Calculus is not really a theorem it is a mere definition.

Example 58:I will prove the following mathematically using the definition above. Given a sequence $\displaystyle a_n=1/n$ it seems reasonable to say $\displaystyle \lim 1/n=0$. Hence we need to show $\displaystyle |1/n-0|=|1/n|=1/n<\epsilon$ for $\displaystyle n\geq N$. Solve the inequality $\displaystyle 1/n<\epsilon$ to get $\displaystyle n>1/\epsilon$, thus if we choose the first integer that makes this true we have completed the proof, the first such integer is $\displaystyle [1/\epsilon]+1$, where $\displaystyle [\,\,\,]$ is greatest integer function, that is the integer part of the number. Again, here is how it works. For any epsilon that you name $\displaystyle \epsilon>0$, I can choose $\displaystyle N=[1/\epsilon]+1$ an integer such that for all $\displaystyle n\geq N$ we have $\displaystyle |a_n|<1/\epsilon$. And the proof it complete.

For example if you choose $\displaystyle \epsilon=.001$ then I should choose $\displaystyle N=[1/.001]+1=1001$.

If you had difficulty understanding that do not worry, we will not use it again. You probably have a question whether mathematicians prove limits in this painful way. No, as you learn math you will find mathematicians create definitions, prove some useful theorem using the defintions, and all results that are complicated that follow are not based on the definitions rather on the proven theorems from the definitions.

Definition:A sequence $\displaystyle a_n$ has anupper boundmeans there exists a real number $\displaystyle R$ such that $\displaystyle a_n \leq R$ for all $\displaystyle n$. By analogy it has alower boundmeans there exists a real number $\displaystyle r$ such that $\displaystyle r\leq a_n$ for all $\displaystyle n$. A sequence isboundedif itbothhas an upper and lower bound.

The following should make sense, if a sequence has a limit then it cannot grow without bound, meaning diverge to infinity.

Theorem:If a sequence has a limit then it is bounded.

Example 59:The sequence $\displaystyle a_n=1/n$ converges to zero. And it is true that is bounded, because any whole integer is larger than the sequence and any negative number is lower than the sequence. However, the other way around is not true. If a seqeunce is bounded does not mean it converges. For example, $\displaystyle a_n=(-1)^n$ is bounded but it does not converge.

The converse (the theorem the other way around) is true when you have a monotone sequence.

Definition:A sequence isnon-increasingmeans that $\displaystyle a_n\geq a_{n+1}$ for sufficiently large $\displaystyle n$. A sequence isnon-decreasingmeans that $\displaystyle a_n\leq a_{n+1}$ for sufficiently large $\displaystyle n$. A sequence ismonotonewhen it is either non-increasingornon-decreasing.

Example 60:The sequence $\displaystyle 10,9,1,2,3,4,...$ is increasing (and hence non-decreasing) because for sufficiently large $\displaystyle n$ in this case $\displaystyle n=2$ it increases (and hence is non-decreasing).

A classic theorem in analysis.

Bolzano-Weierstrauss Theorem:A bounded monotone sequence is convergent (convergent means have a limit).

To illustrate this theorem consider you have a sequence of numbers and they are increasing in this case and have an upper bound. Thus, they get larger and larger but not without bound (because it is bounded) and hence it cannot diverge at infinity. Again not a proof but this reasoning is plausible.

Example 61:Consider $\displaystyle a_n=1/n$ again. It is decreasing (and non-increasing) thus it is monotone. And it is bounded. Thus, it is convergent. In this case it converges to zero. This theorem is a example of anexistence theorem. An existence theorem says that something exists without saying what it is. They appear all over math. In fact, most theorems are existence theorems.

Now we get to infinite series. Those are infinite sums.

Definition:Given a sequence $\displaystyle a_n$ we define a new sequence $\displaystyle S_n$, (call the series) as $\displaystyle S_n=a_0+a_1+...+a_n$. This sequence (series) is also called thesequence of partial sums.

For example, $\displaystyle a_n=1,2,3,4,...$ then $\displaystyle S_0=1$, $\displaystyle S_1=1+2=3$ and thus on.

Hence, the infinite sum $\displaystyle a_0+a_1+a_2...$ is defined to be (if it exists) as the limit of partial sums, that is, $\displaystyle \lim S_n$. Hence, the sum of infinitely many numbers exists only when the sequence of partial sums is convergent. And its value is defined to be the limit of the sequence of partial sums. The notation that is used is $\displaystyle \sum_{n=0}^{\infty} a_n$, note sometimes a series can start from a different value, it does not need to be zero. For example, the series, which is beyond this lecture, converges, $\displaystyle \sum_{n=1}^{\infty} \frac{1}{n^2}=\frac{\pi^2}{6}$.

The sad thing is, is that there is no nice approach to finding the sum of a series once you established it is convergent. For most of the time mathematicians do not care, all they care about whether a series diverges of converges.

Here is a simple theorem, that gives a condition a convergent series must have.

Divergence Theorem:If $\displaystyle \sum_{n=k}^{\infty} a_n$ converges then $\displaystyle \lim a_n=0$.

Proof:Note the meaning of $\displaystyle n=k$ means the sequence can start from any value not necessarly zero of one. Since it is convergent $\displaystyle \lim S_n$ exists. But that means that $\displaystyle \lim S_{n+1}$ exists also and is equal. Thus,

$\displaystyle \lim S_{n+1}=\lim S_n$

$\displaystyle \lim S_{n+1}-\lim S_n=0$

$\displaystyle \lim (S_{n+1}-S_n)=0$

But, $\displaystyle S_{n+1}-S_n = a_{n+1}+a_n+...+a_0-a_n-...-a_0=a_{n+1}$.

Thus, $\displaystyle \lim a_{n+1}=\lim a_n = 0$.

This test is useful in the following way. If the limit of the sequence IS NOT zero then it diverges. What I just said is a logically equivalent statement, called the contrapositive.

Example 62:The series $\displaystyle \sum_{n=1}^{\infty} 1$ diverges. Because $\displaystyle \lim 1 = 1 \not = 0$. It also makes sense, if you keep adding 1's you get a larger and larger number without bound. Thus, there can be no limiting value.

Example 63:The sum of reciprocals $\displaystyle \sum_{n=1} \frac{1}{n}$ is called the "Harmonic Series". Note, $\displaystyle \lim \frac{1}{n}=0$. However, the theorem does not say whether it converges or diverges, thus we do not know. But soon we shall.

Definition:A series of the form $\displaystyle \sum_{n=k}^{\infty} (-1)^n a_n$ or $\displaystyle \sum_{n=k}^{\infty}(-1)^{n+1} a_n$ where the sequence $\displaystyle a_n>0$. Is calledalternating series. Because the terms of the sequence are positive and $\displaystyle (-1)^n \mbox{ or }(-1)^{n+1}$ are positive and negative numbers alternating. Thus, it is a sequence of alternating signs.

Example 64:The series $\displaystyle \sum_{n=1}^{\infty} \frac{(-1)^{n+1}}{n}$. Is an alternating series. First it satisfies the condition of the definition. Or you can write out the terms, $\displaystyle 1-\frac{1}{2}+\frac{1}{3}-\frac{1}{4}+...$ you can see it is alternating in signs.

Here is a really useful theorem about alternating series.

Leibniz Alternating Series Test:If a series is alternating, $\displaystyle \lim a_n=0$ and $\displaystyle a_n$ is non-increasing then the series converges.

Example 65:The series $\displaystyle \sum_{n=1}^{\infty} \frac{(-1)^{n+1}}{n}$ is an alternating series as explained before. Futhermore, $\displaystyle a_n=\frac{1}{n}$ is decreasing because $\displaystyle a_{n+1}<a_n$ and hence non-increasing. And the limit $\displaystyle \lim a_n=0$. Thus, Leibniz test says it converges. This is called thealternating harmonic series.

The following definition is my own, thus you will not find it in textbooks. Nor the term anywhere else. Thus, you will know not to use it when you discuss convergence, because people will not understand you.

Definition:Anextension functionof a sequence $\displaystyle a_n$ is a real-function such that $\displaystyle f(n)=a_n$ for all natural numbers $\displaystyle n$.

Example 66:Consider the sequence $\displaystyle a_n=1/n$, $\displaystyle n\geq 1$. Then the function $\displaystyle f(x)=1/x$ is an extension function. That is for all positive integers $\displaystyle n$ we have. $\displaystyle f(n)=a_n$. Consider $\displaystyle a_n=\sqrt{1+n^2}$ then $\displaystyle f(x)=\sqrt{1+x^2}$ is an extension function.

Here is why our discussion on improper integrals becomes useful.

Integral Test:Given a sequence $\displaystyle \sum_{n=k}^{\infty} a_n$ and let $\displaystyle f(x)$ be an extension function of the sequence. If $\displaystyle f(x)$ is continous, decreasing and positive for $\displaystyle [k,\infty)$. Then $\displaystyle \sum_{n=k}^{\infty} a_n$ and $\displaystyle \int_k^{\infty} f(x) dx$ either both converge or both diverge.

Just a warning, it DOES NOT mean they converge to the same value. They can be different values, it only says if it converges or diverges.

Example 67:Let us determine whether $\displaystyle \sum_{k=1}^{\infty} \frac{1}{n}$ converges or diverges. Note an extension function is $\displaystyle f(x)=1/x$. This function is continous, positive, and decreasing (the derivative is negative). Thus, we can compare it with $\displaystyle \int_1^{\infty} \frac{1}{x} dx = \lim_{t\to \infty} \ln t -\ln 1 = \lim_{t\to\infty} \ln t$. This grows without bound. Thus, the harmonic series diverges.

Example 68:Let us determine whether $\displaystyle \sum_{k=2}^{\infty} \frac{1}{n\ln n}$ converges or diverges. Note an extension function is $\displaystyle f(x)=1/(x\ln x)$, it is continous and positive. To show it is decreasing show that the derivative is negative. (If a function is not decreasing, but it eventually decreases you can still use this. Because only only care about what happens in the end not at the beginning). Thus, we compare it with $\displaystyle \int_2^{\infty} \frac{1}{x\ln x} dx$. First we need to find $\displaystyle \int \frac{1}{x\ln x} dx=\int\frac{1}{\ln x} \cdot \frac{1}{x} dx$. Let $\displaystyle u=\ln x$ then $\displaystyle du/dx=1/x$. Thus, $\displaystyle \int \frac{1}{u} \frac{du}{dx} dx=\int \frac{1}{u} du=\ln u+C=\ln \ln x +C$. Thus, $\displaystyle \lim_{t\to \infty} \ln \ln t - \ln \ln 2 $ it also increases without bound, but really really slowly. Thus the series diverges.

When a series converges it is reasonable to say the numbers get small quickly enough. Thus the ratio of the numbers can determine whether a series converges or diverges.

Ratio Test:Given a series $\displaystyle \sum_{n=k}^{\infty} a_k$ if $\displaystyle a_n$ is non-zero for suffiently large $\displaystyle n$ then consider the limit of the ratio $\displaystyle \lim | a_{n+1}/a_n |$. If the ratio isstrictlyless than 1 it converges. If ratio isstrictlymore than 1 or the ratio diverges to infinity then the series diverges. And the "bad" point is when the ratio is percisely equal to 1. In that case the test is inconclusive, meaning it can and it cannot in some cases.

Example 69:Consider $\displaystyle \sum_{n=1}^{\infty} \frac{1}{n!}$ we can show convergence by looking at the ratio,

$\displaystyle \lim \left| \frac{1}{(n+1)!} \cdot \frac{n!}{1} \right|=\lim \frac{n!}{(n+1)n!}=\lim \frac{1}{n+1}=0$. Thus it converges.

Example 70:Consider $\displaystyle \sum_{n=1}^{\infty} \frac{1}{\sqrt{n}}$ note we do not need to use absolute value symbols because everything is positive here. Thus $\displaystyle a_{n+1}/a_n=\frac{\sqrt{n}}{\sqrt{n+1}}=\sqrt{\frac{n}{n+1 }}$ and the limit as $\displaystyle n\to \infty$ is 1, thus we cannot use the ratio test. However, an extension function is $\displaystyle f(x)=\sqrt{x}$ and the improper integral $\displaystyle \int_1^{\infty} \sqrt{x} dx$ diverges. Thus the series diverges.

Example 71:We already know that the alternating harmonic series converges $\displaystyle \sum_{n=1}^{\infty} \frac{(-1)^{n+1}}{n}$. If we use the ratio test, $\displaystyle \left| \frac{(-1)^{n+2}}{n+1} \cdot \frac{n}{(-1)^n} \right| = \frac{n}{n+1}$ and as $\displaystyle n\to \infty$ then the limit is 1, thus we cannot use the ratio test. However, we know that is converges. Thus, the preceding example and this one show that limit of 1 is no good.

Example 72:This is a really fun one, $\displaystyle \sum_{n=1}^{\infty} \frac{n^n}{n!}$. Using the ratio test (again no need of absolute values) we find that $\displaystyle \frac{(n+1)^{n+1}}{(n+1)!} \cdot \frac{n!}{n^n}=\frac{(n+1)^{n+1}n!}{(n+1)n!n^n}$. Some canceling,

$\displaystyle \frac{(n+1)^n}{n^n}=\left( \frac{n+1}{n} \right)^n=(1+1/n)^n$. The question is what is the limit as $\displaystyle n\to \infty$, if you remember when I was dicussing the exponential function $\displaystyle y=e^x$ I mentioned that that limit is important and is $\displaystyle e\approx 2.718$. Thus, by the ratio test this series diverges.

There are many more tests. These are the most important ones. Whenever you are given a series,alwaysfirst check if the limit of the terms is non-zero, in that case you can use the divergence theorem. If given an alternating series check the divergence theorem and then show it is decreasing. If given a series whose extension function is easy to integrate then compare it with the improper integral. If given a series involving factorial you cannot find a good extension function (yes, the Gamma function works, but how are you going to find its anti-derivative!) and you should use the ratio test.

~~~

Excercises

1)Find $\displaystyle \lim \frac{n+1}{n^2+2}$.

2*)Prove formally (definition of limit) that $\displaystyle \lim \frac{1}{n^2}=0$.

3)Does $\displaystyle \sum_{n=0}^{\infty} \frac{1}{\sqrt{n+1}}$ converge?

4)Does $\displaystyle \sum_{n=1}^{\infty} \frac{(n^2)!}{(n!)^2}$ converge?

5)Does $\displaystyle \sum_{n=1}^{\infty} \frac{(n!)^2}{(n^2)!}$ converge?

6)Does $\displaystyle \sum_{n=1}^{\infty} \frac{(-1)^n}{\sqrt{n}}$ converge?

7)Does $\displaystyle \sum_{n=0}^{\infty} \frac{n^5+1}{n^4+1}$ converge?

8)Does $\displaystyle \sum_{n=1}^{\infty} \frac{1}{e^n}$ converge?

9)Does $\displaystyle \sum_{n=1}^{\infty} \frac{n!}{(2n)!}$ converge?

10)Does $\displaystyle \sum_{n=1}^{\infty} \frac{9^n}{n!}$ converge?

11*)Show that if $\displaystyle \sum_{n=0}\frac{kn}{n^2+1}$ converges only when $\displaystyle k=0$.