Sep 15, 2009
Sep 7, 2009
Jeff Erickson's Algorithms Course Materials
http://compgeom.cs.uiuc.edu/~jeffe//teaching/algorithms/
Algorithms Course Materials
by Jeff Erickson
August 2009 revision
This page contains all my lecture notes for the algorithms classes required for all computer science undergraduate and graduate students at the University of Illinois, Urbana-Champaign. I have taught incarnations of this course eight times: Spring 1999, Fall 2000, Spring 2001, Fall 2002, Spring 2004, Fall 2005, Fall 2006, Spring 2007, Fall 2008, and Spring 2009. These notes are numbered roughly in the order I used them in my undergraduate class, with (lettered) notes containing sprinkled in reasonable places. More advanced material is indicated by the symbol ※. More information about these notes is available after the notes themselves. A large collection of old homeworks and exams follows the lecture notes. Most of these homework and exam problems also appear at the ends of the appropriate lecture notes. Except for various Homework Zeros, which are solely my fault, most of the homework problems were written by my teaching assistants:
Aditya Ramani, Alina Ene, Amir Nayyeri, Asha Seetharam, Ben Moseley, Brian Ensink, Chris Neihengen, Dan Bullok, Dan Cranston, Johnathon Fischer, Ekta Manaktala, Erin Wolf Chambers, Igor Gammer, Gio Kao, Kevin Milans, Kevin Small, Lan Chen, Michael Bond, Mitch Harris, Nick Hurlburt, Nitish Korula, Reza Zamani-Nasab, Rishi Talreja, Rob McCann, Shripad Thite, and Yasu Furakawa.Please do not ask me for solutions. If you're a student, you will (usually) learn more from trying to solve a problem and failing than by reading the answer. If you really need help, ask your instructor, your TAs, and/or your fellow students. If you're an instructor, you really shouldn't assign problems that you can't solve yourself! (Because I don't always follow my own advice, some of the problems are buggy, especially in older homeworks and exams. I've tried to keep the buggy problems out of the lecture notes themselves.)
More recent version of these notes, along with current homework and exams, may be available at the official sites for the undergraduate and/or graduate algorithms classes at UIUC.
Feedback of any kind is always welcome, especially bug reports. I would particularly appreciate hearing from anyone outside UIUC who finds these notes useful (or useless)!
Copyright. Except as indicated otherwise, all material linked from this web page is Copyright © 1999–2009 Jeff Erickson. Anyone may freely download, print, copy, and/or distribute anything on this page, either electronically or on paper. However, nothing on this page may be sold in any form for more than the actual cost of printing and/or reproduction. For example, you are welcome to make local copies on your own web server, but you are not allowed to require an access fee (such as tuition) to view your local copy; that's the same as selling it. If you distribute these notes, please give me proper credit and please include a pointer to this web page (http://
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
You know, I could write a book. And this book would be thick enough to stun an ox. |
— Laurie Anderson, "Let X=X", Big Science (1982) |
Everything
- Everything in one file (765 pages)
- Cover material (6 pages)
- All lecture notes in one file (379 pages)
- All homework, head-banging, and exam problems in one file (386 pages)
If we are ready to tolerate everything as understood, there is nothing left to explain; while if we sourly refuse to take anything, even tentatively, as clear, no explanation can be given. What intrigues us as a problem, and what will satisfy us as a solution, will depend upon the line we draw between what is already clear and what needs to be clarified. |
— Nelson Goodman, Fact, Fiction & Forecast (1955) |
Lecture Notes
- 0. Introduction, history, and course goals
- Recursion
- Randomization
- Amortized analysis
- 8. Aggregation, taxation, potential
- 9. Scapegoat trees and splay trees
- 10. Maintaining disjoint sets ("union-find") — includes O(α(n)) amortized analysis ※
- String algorithms
- Basic graph algorithms
- Flows and cuts
- Linear programming
- I. Definitions and duality ※
- J. The simplex algorithm ※
- Lower bounds
- Appendix
- Retired: These notes contain material that I have not covered in several years, and that I do not expect to cover in the future.
The problem is that we attempt to solve the simplest questions cleverly, thereby rendering them unusually complex. One should seek the simple solution. |
— Anton Pavlovich Chekhov (c. 1890) |
Homeworks, Exams, and Head-Banging Sessions
- Johnny's algorithms homework (Fall 2000 HW 1)
- Spring 1999: everything
- Fall 2000: everything
- Spring 2001: everything
- Fall 2002: everything
- Spring 2004: everything
- Fall 2005: everything
- Fall 2006: everything
- Spring 2007: everything
- Fall 2008: everything
- Spring 2009: everything
More Information
Caveat Lector. Some of these lecture notes have been used less often than others and are therefore (sometimes considerably) less complete/
Alternate sources. My UIUC colleague Sariel Har-Peled maintains his own collection of algorithms lecture notes, which heavily overlap mine, although our presentation styles and specific topical choices are different, sometimes radically so. Choice is good! For people who really prefer dead trees, I recommend the textbooks by Dasgupta, Papadimitriou, and Vazirani; by Kleinberg and Tardos; and (especially for stunning oxen) by Cormen, Leiserson, Rivest, and Stein. An increasing amount of basic algorithmic material can be found at the Source of All Lies. Many more references are listed in the cover material for the lecture notes.
Drawing tools. Most of the figures were drawn with OmniGraffle, but a few very old figures were drawn with xfig, and a few particularly complex figures were drawn with either Freehand or Illustrator. The map on the first page of the maxflow/mincut notes was copied from Lex Schrijver's excellent survey "On the history of combinatorial optimization (till 1960)"; the original map is from a US Government work in the public domain.
Am I writing a textbook? No. All textbooks suck. (Partly that's because of the unethical rapacious profitable business practices of (most) textbook publishers—these notes will always be freely available. But also, I've never seen a textbook on any topic that I'd be willing to teach from for more than a few weeks; that's why I wrote these notes in the first place.) In particular, if these notes ever became a textbook, they would immediately suck. (Determining whether they already suck is an illuminating exercise for the reader.) Besides, have you ever talked to someone who's actually written a textbook? Do you realize how much work is involved?! No way, man. Not for me.
Aug 26, 2009
Continued Fraction -- from Wolfram MathWorld
Continued Fraction | |
A "general" continued fraction representation of a real number is one of the form
(1) |
where ,
, ... and
,
are integers. Such representations are sometimes written in the form
(2) |
for compactness.
Wallis first used the term "continued fraction" in his Arithmetica infinitorum of 1653 (Havil 2003, p. 93), although other sources list the publication date as 1655 or 1656. An archaic word for a continued fraction is anthyphairetic ratio.
While continued fractions are not the only possible representation of real numbers in terms of a sequence of integers (others include the decimal expansion and the so-called Engel expansion), they are a very common such representation that arises most frequently in number theory.
The simple continued fraction representation (which is usually what is meant when the term "continued fraction" is used without qualification) of a number is given by
(3) |
where ,
,
, ... are again integers and
. Simple continued fractions can be written in a compact abbreviated notation as
(4) |
Some care is needed, since some authors begin indexing the terms at instead of
, causing the parity of certain fundamental results in continued fraction theory to be reversed. The first
terms of the simple continued fraction of a number
can be computed in Mathematica using the command ContinuedFraction[x, n].
Continued fractions with closed forms are given in the following table (cf. Euler 1775).
continued fraction | value | approximate | Sloane |
0.697774... | A052119 | ||
1.541494... | A113011 | ||
0.581976... | A073333 |
Starting the indexing of a continued fraction with ,
(5) |
is the integer part of , where
is the floor function,
(6) |
is the integral part of the reciprocal of ,
(7) |
is the integral part of the reciprocal of the remainder, etc. Writing the remainders according to the recurrence relation
(8) | |||
(9) |
gives the concise formula
(10) |
The quantities are called partial quotients, and the quantity obtained by including
terms of the continued fraction
(11) | |||
(12) | |||
(13) |
is called the th convergent. For example, consider the computation of the continued fraction of
, given by
.
term | value | PQs | convergent | value |
3 | 3.00000 | |||
3.14286 | ||||
3.14151 |
Continued fractions provide, in some sense, a series of "best" estimates for an irrational number. Functions can also be written as continued fractions, providing a series of better and better rational approximations. Continued fractions have also proved useful in the proof of certain properties of numbers such as e and (pi). Because quadratic surds have periodic continued fractions (e.g., Pythagoras's constant
has continued fraction [1, 2, 2, 2, 2, ...]), an exact representation for a tabulated numerical value can sometimes be found if it is suspected to represent an unknown quadratic surd.
Continued fractions are also useful for finding near commensurabilities between events with different periods. For example, the Metonic cycle used for calendrical purposes by the Greeks consists of 235 lunar months which very nearly equal 19 solar years, and 235/19 is the sixth convergent of the ratio of the lunar phase (synodic) period and solar period (365.2425/29.53059). Continued fractions can also be used to calculate gear ratios, and were used for this purpose by the ancient Greeks (Guy 1990).
Let the continued fraction for be written
. Then the limiting value is almost always Khinchin's constant
(14) |
(Sloane's A002210).
Similarly, taking the th root of the denominator
of the
th convergent as
almost always gives the Khinchin-Lévy constant
(15) |
(Sloane's A086702).
Let be convergents of a nonsimple continued fraction. Then
(16) | |
(17) |
and subsequent terms are calculated from the recurrence relations
(18) | |
(19) |
for , 2, ...,
. It is also true that
(20) |
The error in approximating a number by a given convergent is roughly the multiplicative inverse of the square of the denominator of the first neglected term.
A finite simple continued fraction representation terminates after a finite number of terms. To "round" a continued fraction, truncate the last term unless it is , in which case it should be added to the previous term (Gosper 1972, Item 101A). To take one over a continued fraction, add (or possibly delete) an initial 0 term. To negate, take the negative of all terms, optionally using the identity
(21) |
A particularly beautiful identity involving the terms of the continued fraction is
(22) |
Finite simple fractions represent rational numbers and all rational numbers are represented by finite continued fractions. There are two possible representations for a finite simple fraction:
(23) |
On the other hand, an infinite simple fraction represents a unique irrational number, and each irrational number has a unique infinite continued fraction.
Consider the convergents of a simple continued fraction, and define
(24) |
(25) |
(26) |
Then subsequent terms can be calculated from the recurrence relations
(27) |
(28) |
The continued fraction fundamental recurrence relation for simple continued fractions is
(29) |
It is also true that if ,
(30) | |||
(31) |
Furthermore,
(32) |
Also, if a convergent , then
(33) |
Similarly, if , then
and
(34) |
The convergents also satisfy
(35) | |||
(36) |
Plotted above on semilog scales are (
even; left figure) and
(
odd; right figure) as a function of
for the convergents of
. In general, the even convergents
of an infinite simple continued fraction for a number
form an increasing sequence, and the odd convergents
form a decreasing sequence (so any even convergent is less than any odd convergent). Summarizing,
(37) |
(38) |
Furthermore, each convergent for lies between the two preceding ones. Each convergent is nearer to the value of the infinite continued fraction than the previous one. In addition, for a number
,
(39) |
The square root of a squarefree integer has a periodic continued fraction of the form
(40) |
(Rose 1994, p. 130). Furthermore, if is not a square number, then the terms of the continued fraction of
satisfy
(41) |
Logarithms can be computed by defining
, ... and the positive integer
, ... such that
(42) |
(43) |
and so on. Then
(44) |
A geometric interpretation for a reduced fraction consists of a string through a lattice of points with ends at
and
(Klein 1896, 1932; Steinhaus 1999, p. 40; Gardner 1984, pp. 210-211, Ball and Coxeter 1987, pp. 86-87; Davenport 1992). This interpretation is closely related to a similar one for the greatest common divisor. The pegs it presses against
give alternate convergents
, while the other convergents are obtained from the pegs it presses against with the initial end at
. The above plot is for
, which has convergents 0, 1, 2/3, 3/4, 5/7, ....
Continued fractions can be used to express the positive roots of any polynomial equation. Continued fractions can also be used to solve linear Diophantine equations and the Pell equation. Euler showed that if a convergent series can be written in the form
(45) |
then it is equal to the continued fraction
(46) |
(Borwein et al. 2004, p. 30).
Gosper has invented an algorithm for performing analytic addition, subtraction, multiplication, and division using continued fractions. It requires keeping track of eight integers which are conceptually arranged at the polyhedron vertices of a cube. Although this algorithm has not appeared in print, similar algorithms have been constructed by Vuillemin (1987) and Liardet and Stambul (1998).
Gosper's algorithm for computing the continued fraction for from the continued fraction for
is described by Gosper (1972), Knuth (1998, Exercise 4.5.3.15, pp. 360 and 601), and Fowler (1999). (In line 9 of Knuth's solution,
should be replaced by
.) Gosper (1972) and Knuth (1981) also mention the bivariate case
.
Abramowitz, M. and Stegun, I. A. (Eds.). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover, p. 19, 1972.
Acton, F. S. "Power Series, Continued Fractions, and Rational Approximations." Ch. 11 in Numerical Methods That Work, 2nd printing. Washington, DC: Math. Assoc. Amer., 1990.
Adamchik, V. "Limits of Continued Fractions and Nested Radicals." Mathematica J. 2, 54-57, 1992.
Ball, W. W. R. and Coxeter, H. S. M. Mathematical Recreations and Essays, 13th ed. New York: Dover, pp. 54-57 and 86-87, 1987.
Berndt, B. C. and Gesztesy, F. (Eds.). Continued Fractions: From Analytic Number Theory to Constructive Approximation, A Volume in Honor of L. J. Lange. Providence, RI: Amer. Math. Soc., 1999.
Beskin, N. M. Fascinating Fractions. Moscow: Mir Publishers, 1980.
Borwein, J.; Bailey, D.; and Girgensohn, R. Experimentation in Mathematics: Computational Paths to Discovery. Wellesley, MA: A K Peters, 2004.
Brezinski, C. History of Continued Fractions and Padé Approximants. New York: Springer-Verlag, 1980.
Conway, J. H. and Guy, R. K. "Continued Fractions." In The Book of Numbers. New York: Springer-Verlag, pp. 176-179, 1996.
Courant, R. and Robbins, H. "Continued Fractions. Diophantine Equations." §2.4 in Supplement to Ch. 1 in What Is Mathematics?: An Elementary Approach to Ideas and Methods, 2nd ed. Oxford, England: Oxford University Press, pp. 49-51, 1996.
Cuyt, A.; Petersen, A. B.; Verdonk, B.; Waadeland, H.; and Jones, W. B. Handbook of Continued Fractions for Special Functions. New York: Springer, 2008.
Davenport, H. §IV.12 in The Higher Arithmetic: An Introduction to the Theory of Numbers, 6th ed. New York: Cambridge University Press, 1992.
Dunne, E. and McConnell, M. "Pianos and Continued Fractions." Math. Mag. 72, 104-115, 1999.
Euler, L. "On the Formulation of Continued Fractions." Delivered to the St. Petersburg Academy, Sept. 4, 1775. Published as Euler, L. "De formatione fractionum continuarum." Acta Academiae Scientarum Imperialis Petropolitinae 3, 3-29, 1782. Republished in Euler, L. Opera Omnia, Ser. 1: Opera mathematica, Vol. 15. Basel, Switzerland: Birkhäuser, 1992. http://arxiv.org/abs/math.HO/0508227/.
Euler, L. Introduction to Analysis of the Infinite, Book I. New York: Springer-Verlag, 1980.
Fowler, D. H. The Mathematics of Plato's Academy: A New Reconstruction, 2nd ed. Oxford, England: Oxford University Press, 1999.
Fowler, D. "Wallis and Number Columns by David Fowler." http://mathforum.org/epigone/math-history-list/sterbloirerm.
Gardner, M. The Sixth Book of Mathematical Games from Scientific American. Chicago, IL: University of Chicago Press, pp. 210-211, 1984.
Gosper, R. W. "Continued fractions query." math-fun@cs.arizona.edu posting, Dec. 27, 1996.
Gosper, R. W. Item 101a in Beeler, M.; Gosper, R. W.; and Schroeppel, R. HAKMEM. Cambridge, MA: MIT Artificial Intelligence Laboratory, Memo AIM-239, pp. 37-39, Feb. 1972. http://www.inwap.com/pdp10/hbaker/hakmem/cf.html#item101a.
Gosper, R. W. Item 101b in Beeler, M.; Gosper, R. W.; and Schroeppel, R. HAKMEM. Cambridge, MA: MIT Artificial Intelligence Laboratory, Memo AIM-239, pp. 39-44, Feb. 1972. http://www.inwap.com/pdp10/hbaker/hakmem/cf.html#item101b.
Graham, R. L.; Knuth, D. E.; and Patashnik, O. "Continuants." §6.7 in Concrete Mathematics: A Foundation for Computer Science, 2nd ed. Reading, MA: Addison-Wesley, pp. 301-309, 1994.
Guy, R. K. "Continued Fractions" §F20 in Unsolved Problems in Number Theory, 2nd ed. New York: Springer-Verlag, p. 259, 1994.
Havil, J. Gamma: Exploring Euler's Constant. Princeton, NJ: Princeton University Press, 2003.
Jacobson, M. J. Jr.; Lukes, R. F.; and Williams, H. C. "An Investigation of Bounds for the Regulator of Quadratic Fields." Experiment. Math. 4, 211-225, 1995.
Khinchin, A. Ya. Continued Fractions. New York: Dover, 1997.
Kimberling, C. "Continued Fractions." http://faculty.evansville.edu/ck6/integer/contfr.html.
Klein, F. Ausgewählte Kapitel der Zahlentheorie I. Göttingen, Germany: n.p., 1896.
Klein, F. Elementary Number Theory. New York, p. 44, 1932.
Kline, M. Mathematical Thought from Ancient to Modern Times. Oxford, England: Oxford University Press, 1990.
Knuth, D. E. The Art of Computer Programming, Vol. 2: Seminumerical Algorithms, 3rd ed. Reading, MA: Addison-Wesley, p. 316, 1998.
Liardet, P. and Stambul, P. "Algebraic Computation with Continued Fractions." J. Number Th. 73, 92-121, 1998.
Liberman, H. Simple Continued Fractions: An Elementary to Research Level Approach. SMD Stock Analysts, 2003.
Lorentzen, L. and Waadeland, H. Continued Fractions with Applications. Amsterdam, Netherlands: North-Holland, 1992.
Moore, C. D. An Introduction to Continued Fractions. Washington, DC: National Council of Teachers of Mathematics, 1964.
Olds, C. D. Continued Fractions. New York: Random House, 1963.
Perron, O. Die Lehre von den Kettenbrüchen, 3. verb. und erweiterte Aufl. Stuttgart, Germany: Teubner, 1954-57.
Pettofrezzo, A. J. and Bykrit, D. R. Elements of Number Theory. Englewood Cliffs, NJ: Prentice-Hall, 1970.
Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. "Evaluation of Continued Fractions." §5.2 in Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, pp. 163-167, 1992.
Riesel, H. "Continued Fractions." Appendix 8 in Prime Numbers and Computer Methods for Factorization, 2nd ed. Boston, MA: Birkhäuser, pp. 327-342, 1994.
Rockett, A. M. and Szüsz, P. Continued Fractions. New York: World Scientific, 1992.
Rose, H. E. A Course in Number Theory, 2nd ed. Oxford, England: Oxford University Press, 1994.
Rosen, K. H. Elementary Number Theory and Its Applications. New York: Addison-Wesley, 1980.
Schur, I. "Ein Beitrag zur additiven Zahlentheorie und zur Theorie der Kettenbrüche." Sitzungsber. Preuss. Akad. Wiss. Phys.-Math. Klasse, pp. 302-321, 1917.
Sloane, N. J. A. Sequences A000037/M0613, A002210/M1564, A013943, A052119, A086702, and A113011 in "The On-Line Encyclopedia of Integer Sequences."
Steinhaus, H. Mathematical Snapshots, 3rd ed. New York: Dover, pp. 39-42, 1999.
Van Tuyl, A. L. "Continued Fractions." http://archives.math.utk.edu/articles/atuyl/confrac/.
Vuillemin, J. "Exact Real Computer Arithmetic with Continued Fractions." INRIA Report 760. Le Chesnay, France: INRIA, Nov. 1987. http://www.inria.fr/RRRT/RR-0760.html.
Wagon, S. "Continued Fractions." §8.5 in Mathematica in Action. New York: W. H. Freeman, pp. 263-271, 1991.
Wall, H. S. Analytic Theory of Continued Fractions. New York: Chelsea, 1948.
Wallis, J. Arithmetica Infinitorum. Oxford, England, 1656.
Weisstein, E. W. "Books about Continued Fractions." http://www.ericweisstein.com/encyclopedias/books/ContinuedFractions.html.
Williams, H. C. "A Numerical Investigation into the Length of the Period of the Continued Fraction Expansion of ." Math. Comput. 36, 593-601, 1981.
CITE THIS AS:
Weisstein, Eric W. "Continued Fraction." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/ContinuedFraction.html