This shall be the last topic that I will give followed by another lecture on its applications. This topic usually appears in Calculus III and it the beginning of Advanced Calculus. In Advanced Calculus this is presented in much more detail and is much more complicated. I will just mention some basic definitions and theorems.
If you look at the dictionary definition of "Calculus" it will define it as the mathematics that deals with rates of change, area and volumes of shapes. Yes, that is true from an applied math approach. But what I want you to see is that Calculus is the study of a function (real). Note everything we were doing involved dealing with a function. Thus, Calculus is the study of a function. The notation is (slightly wrong) . Meaning it takes a real number (that is represented by a bold R) and takes it (represented by an arrow) into another real number (represented by R). This is why Calculus is called "Real Analysis" because it studies real functions and real numbers. There is another type of Calulus that originated probably 100-150 years after Calculus, it is called "Complex Analysis". It is an analogue meaning, , it studies complex functions, that is, functions that take a complex number and transform it into another complex number. The notation for the set is called the natural numbers. With that we have the following definition.
Definition: A sequence is a function , meaning it takes a natural number and transforms it into another natural number. Note the domain of a sequence are all natural numbers.
The standard way to write a sequence is not . Thus, if then we have,
, , . The nice thing about a sequence is that we can write out all the terms,
in a list. The reason why I say it is nice is because you cannot do that with a real function, you cannot write out all the output values. Supprisingly you can do it with the rational values. This concept is called "countable", it has nothing to do with what we will discuss but I just wanted you to know what it means. This tell us that there are more real numbers then there are natural numbers! A concept that non-mathematicians can never accept, it is difficult for them to understand it.
When we deal with sequences it is useful to know what the limit of a sequence is. That is , since we are dealing with the natural numbers we are only interested in the case . Thus all limits are . Thus, we will simply write . The idea is, what number a sequence is tending to.
Example 57: Consider the sequence, then .
Definition: If a sequence has a limit then it converges. Otherwise it diverges.
Do not be confused with the word "diverges". In English it means entropy, something that expands. Hence you would assume something like diverges because it gets larger and larger without bound. When this happens we say it diverges to infinity and write whenever appropriate. Thus, (note it does not mean the limit exists, it rather means it does not with the special property of growing without bound. The sequence which is does not tend to a value, thus does not exist, nor does it not diverge to infinity.
One thing about math is that definitions are just as important as theorems, and in some cases even more important! Calculus/Analysis is a classic example. The foundation of Calculus is the meaning of a sequence and its limit. Yes, we understand what it means intuitively, but how do we define "tending to", "getting closer to", "approaching"? For a mathematician, these terms need to be defined. Below I will present the defintion of this.
Definition: Given a sequence , we say it has a limit when: for any there exists a natural number such that for all .
Wow! That looks dangerous. You probably never seen a more complicated definition in you life. This definition is attributed to Cauchy and Weierstrauss. Behind this ugly looking definition is one simple and beautify idea. First when we say two numbers and are close to each other we mean that their difference is very small. Since we do not know which one is larger we use absolute value. Thus, is small. When we say a sequence converges to a number what we are saying in other words is that the difference between the sequence and its limit value (if it exists) is small. Thus, , where epsilon is some small number. Thus, let me repeat. We have a sequence and a limit value it means can be made as close as possible, that is, choosing a smaller and smaller for sufficiently large , that is for some big natural number . We will not be using this definition in the lecture, I mentioned it because I want you to see the mathematical side of Calculus. All theorems and all rules that are from Calculus are based on this and other similar definitions. Hence the most important result in Calculus is not really a theorem it is a mere definition.
Example 58: I will prove the following mathematically using the definition above. Given a sequence it seems reasonable to say . Hence we need to show for . Solve the inequality to get , thus if we choose the first integer that makes this true we have completed the proof, the first such integer is , where is greatest integer function, that is the integer part of the number. Again, here is how it works. For any epsilon that you name , I can choose an integer such that for all we have . And the proof it complete.
For example if you choose then I should choose .
If you had difficulty understanding that do not worry, we will not use it again. You probably have a question whether mathematicians prove limits in this painful way. No, as you learn math you will find mathematicians create definitions, prove some useful theorem using the defintions, and all results that are complicated that follow are not based on the definitions rather on the proven theorems from the definitions.
Definition: A sequence has an upper bound means there exists a real number such that for all . By analogy it has a lower bound means there exists a real number such that for all . A sequence is bounded if it both has an upper and lower bound.
The following should make sense, if a sequence has a limit then it cannot grow without bound, meaning diverge to infinity.
Theorem:If a sequence has a limit then it is bounded.
Example 59: The sequence converges to zero. And it is true that is bounded, because any whole integer is larger than the sequence and any negative number is lower than the sequence. However, the other way around is not true. If a seqeunce is bounded does not mean it converges. For example, is bounded but it does not converge.
The converse (the theorem the other way around) is true when you have a monotone sequence.
Definition: A sequence is non-increasing means that for sufficiently large . A sequence is non-decreasing means that for sufficiently large . A sequence is monotone when it is either non-increasing or non-decreasing.
Example 60: The sequence is increasing (and hence non-decreasing) because for sufficiently large in this case it increases (and hence is non-decreasing).
A classic theorem in analysis.
Bolzano-Weierstrauss Theorem: A bounded monotone sequence is convergent (convergent means have a limit).
To illustrate this theorem consider you have a sequence of numbers and they are increasing in this case and have an upper bound. Thus, they get larger and larger but not without bound (because it is bounded) and hence it cannot diverge at infinity. Again not a proof but this reasoning is plausible.
Example 61: Consider again. It is decreasing (and non-increasing) thus it is monotone. And it is bounded. Thus, it is convergent. In this case it converges to zero. This theorem is a example of an existence theorem. An existence theorem says that something exists without saying what it is. They appear all over math. In fact, most theorems are existence theorems.
Now we get to infinite series. Those are infinite sums.
Definition: Given a sequence we define a new sequence , (call the series) as . This sequence (series) is also called the sequence of partial sums.
For example, then , and thus on.
Hence, the infinite sum is defined to be (if it exists) as the limit of partial sums, that is, . Hence, the sum of infinitely many numbers exists only when the sequence of partial sums is convergent. And its value is defined to be the limit of the sequence of partial sums. The notation that is used is , note sometimes a series can start from a different value, it does not need to be zero. For example, the series, which is beyond this lecture, converges, .
The sad thing is, is that there is no nice approach to finding the sum of a series once you established it is convergent. For most of the time mathematicians do not care, all they care about whether a series diverges of converges.
Here is a simple theorem, that gives a condition a convergent series must have.
Divergence Theorem: If converges then .
Proof: Note the meaning of means the sequence can start from any value not necessarly zero of one. Since it is convergent exists. But that means that exists also and is equal. Thus,
This test is useful in the following way. If the limit of the sequence IS NOT zero then it diverges. What I just said is a logically equivalent statement, called the contrapositive.
Example 62: The series diverges. Because . It also makes sense, if you keep adding 1's you get a larger and larger number without bound. Thus, there can be no limiting value.
Example 63: The sum of reciprocals is called the "Harmonic Series". Note, . However, the theorem does not say whether it converges or diverges, thus we do not know. But soon we shall.
Definition: A series of the form or where the sequence . Is called alternating series. Because the terms of the sequence are positive and are positive and negative numbers alternating. Thus, it is a sequence of alternating signs.
Example 64: The series . Is an alternating series. First it satisfies the condition of the definition. Or you can write out the terms, you can see it is alternating in signs.
Here is a really useful theorem about alternating series.
Leibniz Alternating Series Test: If a series is alternating, and is non-increasing then the series converges.
Example 65: The series is an alternating series as explained before. Futhermore, is decreasing because and hence non-increasing. And the limit . Thus, Leibniz test says it converges. This is called the alternating harmonic series.
The following definition is my own, thus you will not find it in textbooks. Nor the term anywhere else. Thus, you will know not to use it when you discuss convergence, because people will not understand you.
Definition: An extension function of a sequence is a real-function such that for all natural numbers .
Example 66: Consider the sequence , . Then the function is an extension function. That is for all positive integers we have. . Consider then is an extension function.
Here is why our discussion on improper integrals becomes useful.
Integral Test: Given a sequence and let be an extension function of the sequence. If is continous, decreasing and positive for . Then and either both converge or both diverge.
Just a warning, it DOES NOT mean they converge to the same value. They can be different values, it only says if it converges or diverges.
Example 67: Let us determine whether converges or diverges. Note an extension function is . This function is continous, positive, and decreasing (the derivative is negative). Thus, we can compare it with . This grows without bound. Thus, the harmonic series diverges.
Example 68: Let us determine whether converges or diverges. Note an extension function is , it is continous and positive. To show it is decreasing show that the derivative is negative. (If a function is not decreasing, but it eventually decreases you can still use this. Because only only care about what happens in the end not at the beginning). Thus, we compare it with . First we need to find . Let then . Thus, . Thus, it also increases without bound, but really really slowly. Thus the series diverges.
When a series converges it is reasonable to say the numbers get small quickly enough. Thus the ratio of the numbers can determine whether a series converges or diverges.
Ratio Test: Given a series if is non-zero for suffiently large then consider the limit of the ratio . If the ratio is strictly less than 1 it converges. If ratio is strictly more than 1 or the ratio diverges to infinity then the series diverges. And the "bad" point is when the ratio is percisely equal to 1. In that case the test is inconclusive, meaning it can and it cannot in some cases.
Example 69: Consider we can show convergence by looking at the ratio,
. Thus it converges.
Example 70: Consider note we do not need to use absolute value symbols because everything is positive here. Thus and the limit as is 1, thus we cannot use the ratio test. However, an extension function is and the improper integral diverges. Thus the series diverges.
Example 71: We already know that the alternating harmonic series converges . If we use the ratio test, and as then the limit is 1, thus we cannot use the ratio test. However, we know that is converges. Thus, the preceding example and this one show that limit of 1 is no good.
Example 72: This is a really fun one, . Using the ratio test (again no need of absolute values) we find that . Some canceling,
. The question is what is the limit as , if you remember when I was dicussing the exponential function I mentioned that that limit is important and is . Thus, by the ratio test this series diverges.
There are many more tests. These are the most important ones. Whenever you are given a series, always first check if the limit of the terms is non-zero, in that case you can use the divergence theorem. If given an alternating series check the divergence theorem and then show it is decreasing. If given a series whose extension function is easy to integrate then compare it with the improper integral. If given a series involving factorial you cannot find a good extension function (yes, the Gamma function works, but how are you going to find its anti-derivative!) and you should use the ratio test.
2*)Prove formally (definition of limit) that .
11*)Show that if converges only when .