Introduction to Algorithms (Cormen, Rivest) is good enough that I read it completely in college. The exercises are nice (they’re reasonably challenging and build up to useful little results I’ve recalled over my programming career). I think it’s fine for self-study; I prefer it to the undergrad intro level or language-specific books. Obviously the interesting part about an algorithm is not the Java/Python/whatever language rendering of it. I also prefer it to Knuth’s tomes (which I gave up on finishing—not enough fun). Knuth invents problems so he can solve them. He explains too much minutia. But his exercises are varied and difficult. If you like very hard puzzles, it’s a good place to look.
Introduction to Automata Theory, Languages, and Computation (Hopcroft+Ullman) was also good enough for me to read. I’ve referred to it many times since. However, it’s apparently not well-liked by others; maybe because it’s too dense for them? I haven’t read any other textbooks in the area.
The Feynman Lectures on Physics are also fun to read. But I doubt someone could use them as an intro course on their own. Because they’re filled with entertaining tidbits, I was tempted to read through them without actually following the math 100%. Obviously this somewhat defeats the purpose. That’s always a danger with well written technical material consumed for pleasure. I had already taken a few physics courses before I read Feynman; his lectures were better than the course textbooks (which I already forgot).
I didn’t care for Jaynes. I only read about 700 pages, though. I remember there was some group reading effort that stopped showing up on LW after just a few chapters :)
For plain old programming, I’ve read quite a few books, and really liked The Practice of Programming—it was too short. I read Dijkstra’s a discipline of programming and loved it for its idea to define program semantics precisely and to prove your code correct (nobody really practices this; it’s too slow and hard compared to “debugging”), but it’s probably not worth the price—I checked it out from a library.
I also agree with rwallace’s recommendations also, except that the AI text is not especially useful (not that I know of a better one). I would not give SICP to a novice, though. Although I had done everything described in the book before (and already knew lisp), it did increase my appreciation of using closures and higher order functions as an alternative for the usual imperative/OO stuff. It also covers interpretation and compilation quite well (skipping the character-sequence parsing part—this is lisp, after all).
For an AI text, I think any (text)book on a subject of your interest by Judea Pearl would fit the bill. ”Symbolic Logic and Mechanical Theorem Proving” by Chang and Lee is still an exceptionally lucid introduction to non-probabilistic AI.
I also prefer Hopcroft+Ullman (original edition) to later alternatives like their own later edition, Papadimitriou, and even Sipser who is widely regarded as having written the definitive intro text.
“A Discipline of Programming” is rather hard to follow. Dromey gives an introductory treatment that’s a bit too introductory, “Progamming Pearls” by Bently includes another even more informal treatment, and Gries’s “Science of Programming” would be the textbook version that I might recommend covering this material. All three are somewhat dated. More modern treatment would be either Apt’s “Verification of Sequential and Concurrent Programs” or Manna’s “The Calculus of Computation.” and depending on your focus one would be better than the other. However, the ultimate book I would recommend in this field is “Interactive Theorem Proving and Program development” by Yves Bertot. It doesn’t teach Hoare’s invariant method like the other books, but uses a more powerful technique in functional programming for creating provably correct software.
Thanks for your recommendations, though I’ve set a rule that I won’t add recommendations to the list in the original post unless those recommendations conform to the rules. Would you mind adding to what you’ve written above so as to conform to rule #3?
For example, you could list two other books on algorithms and explain why you prefer Introduction to Algorithms to those other books. And you could do the same for the subject of physics, and the subject of programming, and so on.
Well, let me do Jonathan’s job for him on one of those.
Introduction to Algorithms by Cormen, Leiserson, Rivest, and (as of the second edition) Stein is a first-rate single-volume algorithms text, covering a good selection of topics and providing nice clean pseudocode for most of what they do. The explanations are clear and concise. (Readers whose tolerance for mathematics is low may want to look elsewhere, though.)
Two obvious comparisons: Knuth’s TAOCP is wonderful but: very, very long; now rather outdated in the range of algorithms it covers; describes algorithms with wordy descriptions, flowcharts, and assembly language for a computer of Knuth’s own invention. When you need Knuth, you really need Knuth, but mostly you don’t. Sedgwick’s Algorithms (warning: it’s many years since I read this, and recent editions may be different) is shallower, less clearly written, and frankly never gave me the same the-author-is-really-smart feeling that CLRS does.
(If you’re going to get two algorithms books rather than one, a good complement to CLRS might be Skiena’s “The algorithm design manual”, more comments on which you can find on my website.)
Thanks. I really didn’t have the ability to easily recall names of what few alternatives I’ve read (although in the area of programming in general, there are dozens of highly recommended books I’ve actually read—Design Patterns (ok), Pragmatic Programmer (ok), Code Complete (ok), Large Scale C++ Software Design (ok), Analysis Patterns (horrible), Software Engineering with Java (textbook, useless), Writing Solid Code (ok), object-oriented software construction (ok, sells the idea of design-by-contract), and I could continue listing 20 books, but what’s the point. These are hardly textbooks anyway.
On algorithms, other than Knuth (after my disrecommendation of his work, I just bought his latest, “Combinatorial Algorithms, part 1”), really the only other one I read is “Data Structures in C” or some similar lower level textbook, which was unobjectionable but did not have the same quality.
You’re welcome! (Of the other books you mention that I’ve read, I agree with your assessment except that I’d want to subdivide the “ok” category a bit.)
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.
Manber’s “Algorithms—a creative approach” is better than Cormen, which I agree is better than Knuth. It’s also better than Aho’s book on algorithms as well. It’s better in that you can study it by yourself with more profit. On the other hand, Cormen’s co-author has a series of video lectures at MIT’s OCW site that you can follow along with.
What about Manber’s book makes it more fruitful for self-study than CLRS? How does it compare with CLRS in other respects? (Coverage of algorithms and data structures; useful pseudocode; mathematical rigour; …)
As a counterpoint to Hopcroft+Ullman, from another who has not read other books, Problem Solving in Automata, Languages, and Complexity by Ding-Zhu Du and Ker-I Ko was terrific. I did it as an undergraduate independent study class, completely from this book, and found it to be easy to follow if you are willing to work through problems.
Maybe we need someone who knows something more on the subject?
Hopcroft+Ullman is very proof oriented. Sometimes the proof is constructive (by giving an algorithm and proving its correctness). I liked it. There may be much better available for self-study.
Specialty algorithms: I briefly referenced Numerical Optimization and it seems better than Numerical Recipes in C. I didn’t read it cover to cover.
Algorithms on Strings, Trees, and Sequences (Gusfield) was definitely a good source for computational biology algorithms (I don’t do computation biology, but it explains fairly well things like suffix trees and their applications, and algorithms matching a set of patterns against substrings of running text).
Foundations of Natural Language Processing is solid. I don’t think there’s a better textbook (for the types of dumb, statistics/machine-learning based, analysis of human speech/text that are widely practiced). It’s better than “Natural Language Understanding” (Allen), which is more old-school-AI.
Subjects: algorithms/computational complexity, physics, Bayesian probability, programming
Introduction to Algorithms (Cormen, Rivest) is good enough that I read it completely in college. The exercises are nice (they’re reasonably challenging and build up to useful little results I’ve recalled over my programming career). I think it’s fine for self-study; I prefer it to the undergrad intro level or language-specific books. Obviously the interesting part about an algorithm is not the Java/Python/whatever language rendering of it. I also prefer it to Knuth’s tomes (which I gave up on finishing—not enough fun). Knuth invents problems so he can solve them. He explains too much minutia. But his exercises are varied and difficult. If you like very hard puzzles, it’s a good place to look.
Introduction to Automata Theory, Languages, and Computation (Hopcroft+Ullman) was also good enough for me to read. I’ve referred to it many times since. However, it’s apparently not well-liked by others; maybe because it’s too dense for them? I haven’t read any other textbooks in the area.
The Feynman Lectures on Physics are also fun to read. But I doubt someone could use them as an intro course on their own. Because they’re filled with entertaining tidbits, I was tempted to read through them without actually following the math 100%. Obviously this somewhat defeats the purpose. That’s always a danger with well written technical material consumed for pleasure. I had already taken a few physics courses before I read Feynman; his lectures were better than the course textbooks (which I already forgot).
I didn’t care for Jaynes. I only read about 700 pages, though. I remember there was some group reading effort that stopped showing up on LW after just a few chapters :)
For plain old programming, I’ve read quite a few books, and really liked The Practice of Programming—it was too short. I read Dijkstra’s a discipline of programming and loved it for its idea to define program semantics precisely and to prove your code correct (nobody really practices this; it’s too slow and hard compared to “debugging”), but it’s probably not worth the price—I checked it out from a library.
I also agree with rwallace’s recommendations also, except that the AI text is not especially useful (not that I know of a better one). I would not give SICP to a novice, though. Although I had done everything described in the book before (and already knew lisp), it did increase my appreciation of using closures and higher order functions as an alternative for the usual imperative/OO stuff. It also covers interpretation and compilation quite well (skipping the character-sequence parsing part—this is lisp, after all).
For an AI text, I think any (text)book on a subject of your interest by Judea Pearl would fit the bill.
”Symbolic Logic and Mechanical Theorem Proving” by Chang and Lee is still an exceptionally lucid introduction to non-probabilistic AI.
I also prefer Hopcroft+Ullman (original edition) to later alternatives like their own later edition, Papadimitriou, and even Sipser who is widely regarded as having written the definitive intro text.
“A Discipline of Programming” is rather hard to follow. Dromey gives an introductory treatment that’s a bit too introductory, “Progamming Pearls” by Bently includes another even more informal treatment, and Gries’s “Science of Programming” would be the textbook version that I might recommend covering this material. All three are somewhat dated. More modern treatment would be either Apt’s “Verification of Sequential and Concurrent Programs” or Manna’s “The Calculus of Computation.” and depending on your focus one would be better than the other. However, the ultimate book I would recommend in this field is “Interactive Theorem Proving and Program development” by Yves Bertot. It doesn’t teach Hoare’s invariant method like the other books, but uses a more powerful technique in functional programming for creating provably correct software.
I’ll look for Bertot’s book. I agree that “A Discipline” is not a pleasant read (though I found it rewarding).
Jonathan_Graehl,
Thanks for your recommendations, though I’ve set a rule that I won’t add recommendations to the list in the original post unless those recommendations conform to the rules. Would you mind adding to what you’ve written above so as to conform to rule #3?
For example, you could list two other books on algorithms and explain why you prefer Introduction to Algorithms to those other books. And you could do the same for the subject of physics, and the subject of programming, and so on.
Well, let me do Jonathan’s job for him on one of those.
Introduction to Algorithms by Cormen, Leiserson, Rivest, and (as of the second edition) Stein is a first-rate single-volume algorithms text, covering a good selection of topics and providing nice clean pseudocode for most of what they do. The explanations are clear and concise. (Readers whose tolerance for mathematics is low may want to look elsewhere, though.)
Two obvious comparisons: Knuth’s TAOCP is wonderful but: very, very long; now rather outdated in the range of algorithms it covers; describes algorithms with wordy descriptions, flowcharts, and assembly language for a computer of Knuth’s own invention. When you need Knuth, you really need Knuth, but mostly you don’t. Sedgwick’s Algorithms (warning: it’s many years since I read this, and recent editions may be different) is shallower, less clearly written, and frankly never gave me the same the-author-is-really-smart feeling that CLRS does.
(If you’re going to get two algorithms books rather than one, a good complement to CLRS might be Skiena’s “The algorithm design manual”, more comments on which you can find on my website.)
Thanks. I really didn’t have the ability to easily recall names of what few alternatives I’ve read (although in the area of programming in general, there are dozens of highly recommended books I’ve actually read—Design Patterns (ok), Pragmatic Programmer (ok), Code Complete (ok), Large Scale C++ Software Design (ok), Analysis Patterns (horrible), Software Engineering with Java (textbook, useless), Writing Solid Code (ok), object-oriented software construction (ok, sells the idea of design-by-contract), and I could continue listing 20 books, but what’s the point. These are hardly textbooks anyway.
On algorithms, other than Knuth (after my disrecommendation of his work, I just bought his latest, “Combinatorial Algorithms, part 1”), really the only other one I read is “Data Structures in C” or some similar lower level textbook, which was unobjectionable but did not have the same quality.
You’re welcome! (Of the other books you mention that I’ve read, I agree with your assessment except that I’d want to subdivide the “ok” category a bit.)
I would like to suggest Algorithm Design by Kleinberg and Tardos over CLRS.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.
I would like to suggest \href{Algorithm Design by Kleinberg and Tardos}{http://www.cs.sjtu.edu.cn/~jiangli/teaching/CS222/files/materials/Algorithm%20Design.pdf} over CLRS.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.
I would like to suggest \href{Algorithm Design by Kleinberg and Tardos}{http://www.cs.sjtu.edu.cn/~jiangli/teaching/CS222/files/materials/Algorithm%20Design.pdf} over CLRS.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.
Manber’s “Algorithms—a creative approach” is better than Cormen, which I agree is better than Knuth. It’s also better than Aho’s book on algorithms as well. It’s better in that you can study it by yourself with more profit. On the other hand, Cormen’s co-author has a series of video lectures at MIT’s OCW site that you can follow along with.
What about Manber’s book makes it more fruitful for self-study than CLRS? How does it compare with CLRS in other respects? (Coverage of algorithms and data structures; useful pseudocode; mathematical rigour; …)
As a counterpoint to Hopcroft+Ullman, from another who has not read other books, Problem Solving in Automata, Languages, and Complexity by Ding-Zhu Du and Ker-I Ko was terrific. I did it as an undergraduate independent study class, completely from this book, and found it to be easy to follow if you are willing to work through problems.
Maybe we need someone who knows something more on the subject?
Hopcroft+Ullman is very proof oriented. Sometimes the proof is constructive (by giving an algorithm and proving its correctness). I liked it. There may be much better available for self-study.
Specialty algorithms: I briefly referenced Numerical Optimization and it seems better than Numerical Recipes in C. I didn’t read it cover to cover.
Algorithms on Strings, Trees, and Sequences (Gusfield) was definitely a good source for computational biology algorithms (I don’t do computation biology, but it explains fairly well things like suffix trees and their applications, and algorithms matching a set of patterns against substrings of running text).
Foundations of Natural Language Processing is solid. I don’t think there’s a better textbook (for the types of dumb, statistics/machine-learning based, analysis of human speech/text that are widely practiced). It’s better than “Natural Language Understanding” (Allen), which is more old-school-AI.