I can give a few words of advice on where to continue from here, if you’re interested. My own background is as a software dev for many years (13 years professionally plus a few years as a kid). I’d bene involved in many different fields, from embedded systems to web development, and recently ran a team of algorithms researchers in 3d printing, so was mildly exposed to computer vision and 3d concepts, but had no serious machine learning. Then a few years ago, I started to get much more seriously interested in ML/DL/Data Science, and have seen been working in the field (running a dev shop).
So, my take: first of all, I personally didn’t much enjoy Andrew Ng’s course, both because it was much too theoretical for my taste, and (in retrospect) because I didn’t remember enough maths from my CS degree to work with the concepts as easily as I should have.
I’d recommend a few things for you as next steps:
1. Coursera teaches more “classical ML” (not deep learning), and without many applications. The absolute BEST followup in my mind is the Fast.ai course (free).
It focuses on Deep Learning, and teaches with a completely practical-minded approach, rather than theoretical. The idea is to, within one lesson, actually *write software*, like a simple program to tell whether a picture is of a cat or a dog. In this course, you will literally be coding practically world-class Deep Learning code within a few hours.
They’re supposedly working on a more classical-ML course, but unfortunately it isn’t out yet.
Seriously, this is my #1 recommendation for anyone trying to learn machine learning, especially with a background in software development. You won’t be sorry.
2. If you’re at all interested in actually implementing ML as opposed to more reviewing the concepts, then you should try a Kaggle competition or two. If you don’t know it, it’s basically a site that allows companies to upload data, then pay prize money to people writing an algorithm that does something specific with the data. E.g. predict how many page views a certain subset of pages on Wikipedia will receive.
Kaggle does a lot of good things for learning ML: It abstracts away all the data-gathering and a lot of the data-cleaning work, which is the heart of a lot of data science/ML jobs, but is not what you want to actually practice, especially if you are a developer and already know how to deal with this. It also gives specific questions and answers that need to be answered, and has a large collection of existing answers.
In short, Kaggle is the place to go practice your ML skills.
3. Learn more math, especially if you enjoy it! I’ve personally been self-learning the equivalent of a math undergrad, partially for my work in ML, partially for fun.
Specifically, as you correctly understood, ML is mostly statistics, calculus, and linear algebra. Based on my own background, I can tell you that my calculus was perfectly adequate for ML. However, statistics and linear algebra I had studied in much less depth, and they’re both incredibly important, and fascinating subjects. Linear algebra, especially, is amazing, and depending on where you study it, you actually learn a good amount of practical implementations, including linear regressino and other ML algorithms. And not only do you learn these applications, you understand them from an entirely new perspective.
For studying linear algebra, I *highly* recommend Gilber Strang’s video lectures on the subject. He is an excellent teacher, not only engaging, but very practical-minded. You should also follow along with his Linear Algebra textbook, and I highly recommend doing the videos and textbook at the same time—the textbook is not a great resource without the videos, IMO.
The major “problem” with Strang’s videos are that he *really* focuses on the practical, engineering, matrix approach of teaching Linear Algebra, to the almost-complete neglect of the more mathematical approach. E.g. he teaches like half the course before he mentions linear transformations, which is *incredible*.
I still think these videos are the best approach for a software developer looking to *use* linear algebra, but I highly recommend following this up with a more mathematically-oriented textbook. Like many others, I like Axler’s “Linear Algebra Done Right”, which ironically takes the exact opposite approach—it takes him 3 chapters to get to explaining what a Matrix is :)
Other than that, I have recommendations for statistics (both Harvard and MIT have great courses on probability), and there’s lots of other great math to study, though not all of it is directly relevant to ML.
OK, hope this post was worthwhile for you/someone. Feel free to ask if you have any questions :)
I’m not sure I agree re: lawyers, or about how people/society thinks of this. For one thing, I don’t think most people are that OK with lawyers—they tend to get a lot of flack, and e.g. criminal defense attorneys will often get pushback from people who identify them with their clients, irrespective of the fact that they know the lawyers don’t necessarily condone their clients’ actions.
Another thing—most people absolutely hate hypocrisy. I think it’s considered a death-blow to most people’s arguments. People compliment politicians on their speaking skills, but if they discovered that the politician’s are not saying things they believe in, they’d turn on them. (Well, theoretically—President Trump is a good counterexample).
Btw, an aside, but I also think you misrepresent what lawyers do in some way. They’re supposed to be advocating for the rights of their clients, and supposed to persuade, but they can’t for example lie. They are a check on the system that works from within the system—they need to make sure everyone is playing by the rules, but they can’t just make up their own rules or anything. That said, of course rhetoric is important for trial lawyers.
“Geometric” intuition is basically the way that the 3Blue1Brown YouTube channel would explain things. I’m not sure if you’re aware of it, but their “Essence of Linear Algebra” goes through the broad high-level concepts of linear algebra and explains them, with a very visual/geometric intuition for things like basis change, inverses, determinants, etc.
Unfortunately, they never covered transpose :)
Also, I’ll take a look at your blog post, thanks!
Theoretically, I’m most interested in things related to Data Science/Machine Learning/Deep Learning, as that’s my day job. However, since this is my day job, it’s also the area that I know the most about. So e.g. I’ve studied Linear Algebra, Probability, Statistics etc quiet a bit.
I’m mostly interested in rounding out my specific knowledge with everything else I need to know to have knowledge equivalent to a well-rounded math major.
In terms of what personally interests me the most, that’d be logic/foundations-of-mathematics type stuff, e.g. Set Theory.
Thanks for the generous offer! What kind of requests would you like to get? Specific questions? Certain subjects? etc.
In any case, I’m going to write a bunch of stuff I’d love to have explained more thoroughly, some general, some more specific, if you can explain any of them that would of course be amazing from my point of view. Most of these are just things that came up for me while (re)learning a bunch of math.
Linear Algebra—I have great intuition for most matrix operations based on Strang and 3blue1brown videos. E.g. inverse, change of basis, etc. One thing missing: what is the intuitive “geometric” interpretation of matrix transpose? Considering the transpose is used all the time, I still don’t have an intuitive sense of what it does.
An intuitive explanation of the SVD.
More general: I’m trying to recreate a math degree from scratch, more or less. There are tons of lists of best textbook for each subject, but I’d love more of an overarching “here are the subjects you need to know, here’s the best order to study them” kind of list. There aren’t many great lists for these.
Set Theory—an explanation of the difference between transfinite induction and regular induction. I.e. IIRC, there needs to be defined a whole new kind of induction in order to do transfinite induction vs the base induction used to define things a little past N. Why is another mechanism necessary?
First of all, I want to join all the others in thanking you for the honesty and for the sharing.
I’m going to give a few of my views of this, as someone who has a fair amount of experience in “startup-land”. Some of this will be “criticism”, but please don’t take offense—it’s really hard to get these things right, and we all made and continue to make mistakes. And you seem to have gotten to some of these conclusions yourself—I’m writing this for the hypothetical other people who may want to start a startup, so making it general. Btw, really long comment, so sorry! :)
First of all, I’ll tell you what was by far the thing I miss most from your post—any talk about money. You’re building a for-profit company, and maybe I missed it, but I have no idea how you planned to make money off of this! I have no idea of the busines plan, at all. Even if making money isn’t really the goal here, unless you plan to live off donations, it should still be priority 1,2 and 3 for any company: You use money to solve most problems (pay to create content; pay to advertise; etc). You also use money as a good proxy for success. You also use money to fix problems like “burnout” by hiring people!
Secondly: I loved Inadequate Equilibria. But the whole “conversation about startups” was by far the weakest part of it, and the one part I think I actively disagreed with. (Also the part I know the most about: Gellmann amnesia anyone?) While I understand the concept of a grand vision, I think Eliezer and probably you are misunderstanding the idea of an MVP. Or at least, the way I think about it.
The idea is not, as Eliezer put it, to build a product that shows one specific workflow. For one thing, you don’t need to biuild a product. But more importantly, the emphasis is not on showing a complete workflow and seeing if people like it. The emphasis is on doing fast experiments. You need to figure out what assumptions you are making about what you’re trying to build, then test those assumptions. This is something you can often do with minimal work, by faking large parts of the product, for example.
One of the pushbacks to this view is that you might not know the assumptions, but that’s all the more reason to have fast experiments—you want to uncover which assumptions you’re making and don’t realize it, as soon as you can. If you think the only way to do this is to build a product over more than a year—you’re almost certainly wrong, except in very tech-heavy cases, which is not your situation.
In Arbital’s, some assumptions you had and could’ve tested:
You believed people will write content, for free. Easy to test—ask people to write it before the product.
You believed you had a superior flow for reading content. I’m still not sure I understand what that full flow was, but again, easy to test—ask Eliezer/someone to write content, and make a static html site with all the “reading” functionalty, but none of the editing capabilties. You could reasonably make whatever workflow you imagine the reading experience to be with a week’s worth of hand-coding html/css, or even a Wordpress site.
You believed you could get people interested in reading this material, and then doing… something? I’m not sure, since I didn’t understand the business plan. But let’s assume it’s “decide to read more material”. OK, easy to test—put one guide up, and ask people to sign up to a newsletter. Or donate money. Or something.
(I want to emphasize that, although I think I’m right, just the fact that Arbital failed doesn’t prove it. Building startups is hard and usually fails.)
Another issue that stems from lack of a business model—what kind of company were you trying to build? It seems pretty obvious, at least in retrospect, that this kind of company is a bad fit for a VC-funded startup. You are not trying to build somethign with minimal chance of success, but with the ability to become a billion-dollar company. I mean, I don’t think you were aiming for a billion dollar company.
But in that case, you should’ve never expected to raise money (and probably shouldn’t have raised money). And you should’ve made sure this is something that could be profitable relatively quickly, to continue supporting the development.
Lastly, I really got the sense from your post that you are all with a very engineering mindset, and very enamored by the beauty of a complex system, and by wanting to build something. Hell, you worked on this for a few years, wrote an entire post-mortem about it, and still I and others in this thread don’t even understand what you’re building! This is not a good sign—systems usually aren’t this complex, certainly not ones that are made to be used by actual people other than Eliezer :)
One more thing about community projects—we’re a community with a lot of developers. We see development projects everywhere. But the real strength of the community is not necessarily in that—if the bottom line thing we want, as a community, is more things like Eliezer-style explanation of hard concepts, that’s hard enough—we should make the journey to creating that content incredibly simple, and while that is arguably what Arbital was, I’d say that “let’s spend a few years to develop a new software platform” is a huge burden. Much better to use pre-existing stuff, IMO. Let’s make a rationality-Stack Exchange. Or a rational wiki (or not :) ). (Not to crticize too hard because I’m not that much in the community and don’t know the details, but I kind of wonder the same thing about LessWrong V2.0 - do we really need to rebuild forum software from scratch just for us? Is that really where we should be spending our community’s talent and efforts?).
To conclude: building starhtups is hard. Building consumer startups is much harder. Building consumer startups that are marketplaces is really really freakin hard. You tried and failed, which is a shame, but you seemed to have learned a lot from this, both about startups and other things (based on Inadequate Equilibria, I think Eliezer hasn’t learned the lessons I would’ve learned). So kudos for trying, kudos for putting yourself out there with this post, and in general, good job!
Very interesting, and I kind-of agree with the conclusion. However, as a few people pointed out, it wasn’t as simple as just buying bitcoin, you had to sell at the right time, etc.. And buying bitcoin was complicated.
But the other problem is that there are thousands of opportunities, things you should do, etc, lying around, with a possibly good payoff in expected value terms. And how many of them do we do? How many of them do we even think about seriously?
Just a few off the top of my head (first two are obvious, then some others):
AI safety—tons of people in the community agree that’s it’s a serious issue, but how many actively work towards fixing it? (A decent amount, but I’d guess not as much as the “could” do).
CPR training—how many people do it? What is the chance that you’ll one day be in a situation where you need it? I have no idea of the numbers, but considering this is a life-and-death situation, have you looked up the numbers and decided it’s not worth doing?
Similarly, buying an emergency survival pack in case of a natural disaster. Again, not idea of the numbers here, but people do find themselves in situations where they’re without food/water for some period. Do you know the numbers here to decide it’s not worth buying canned goods?
How many people have even bothered to sit down and make sure their health insurance/life insurance/unemployemnt insurance/etc is handled properly?
Even more obvious—sending out resumes every year to find better job opportunities? Or taking a course/reading about finding a better job? This is likely have a massively higher impact than buying bitcoin, not only in monetary terms, but also in life satisfaction impact.
Most of the above are pretty obvious and trivial things almost anyone can do. I can probably list a dozen more, some more “standard advice”, some more out there but still probably high in expected value. If I or anyone were to actually sit down and work through these one by one, we’d probably do little else for the next year.
So while bitcoin, in retrospect, may have been the most obvious and immediately high-value payoff, I’m not sure it’s easy to seperate it from all the other things above. Then again, I’m not sure you should—maybe we should have a list of “these are things you need to take care of ASAP” somewhere.
Why do you want to buy a hat?
Almost all “non-geeky” / “normal” people don’t regularly wear hats. If you’re trying to look “better” and “more fashionable”, the best solution is to skip the hat, and get sunglasses to protect your eyes
Over the last 2.5 years, my co-founder and I grew the Dev Shop we founded (Purple Bit) into a very profitable small company, employing 7 people.
Purple Bit has just been acquired by a former client of ours, Autodesk Inc. Autodesk are the makers of AutoCAD, 3D Studio Max, Maya and many other professional 3d software products.
(Note: this didn’t happen this month, but it wasn’t public until now).
So, do you have a specific marketing plan for getting the word out there about these books?
More specifically, you have access to an entire community rooting for you… how can we help?
(Also, I’ve been convinced by other commenters here—I normally don’t buy physical books anymore, but here I’ll make an exception for ’”fanboyism” and “gifting” purposes).
I remembered it too. Found the quote you’re referring to, I think:
“He ran a quick self-predictive model. There was a ninety-three per cent chance that he’d give in, after a kilotau spent agonising over the decision. It hardly seemed fair to keep Karpal waiting that long.”
Egan, Greg (2010-12-30). Diaspora (Kindle Locations 3127-3129). Orion. Kindle Edition.
It’s less the “why do they act that way”, more “if you had this superpower, what kind of really weird but powerful stuff could you do with it”.
Worm is full of people using superpowers in really inventive ways, in a way that Steelheart/Firefight aren’t.
Tl;dr of my post: If you liked Steelheart, I heavily recommend reading Worm.
So, Sanderson is in my top 5 favorite authors, I think almost every book of his is amazing, and I loved Steelheart.
But shortly after reading it, I started reading the (now finished) online web serial Worm (from Yudkowsky’s recommendation on HPMOR). It has a very similar premise to Steelheart, at least initially.
And let’s just say, Worm makes Steelheart look terrible in comparison. Worm is just so much better.
Again, I’m a huge fan of Sanderson, and I still like the Steelheart series, but I now read it and think to myself that it’s just not even close to realistic, Worm is how people with powers would actually behave.
Seriously, read Worm. And if you happen to read this comment and not have read Sanderson, read his books too (I would start with the Mistborn trilogy, possibly the best trilogy of all time).
FYI, you’re mostly right, at least based on my experience. I tend to have a much harder time listening to Audiobooks of SF/fantasy, and a harder time listening to any fiction vs. non-fiction.
I also have a much easier time listening to SF/Fantasy when it’s in a setting I already know (e.g. sequels, books I’ve read before, etc). Also easier to listen to books from authors I read a lot (but that may be true in general, come to think of it).
I still highly recommend anyone who can to listen to Audiobooks, at least of non-fiction, as one of the best and easiest hacks around.
Do you listen to Audiobooks at all? Are you only specifically against SF as an Audiobook?
I ask because I’m a huge fan of Audiobooks, but I’ve long believed that SF (and fantasy) are both particularly hard to like in Audiobook format. Non-fiction is by far better.
(I do still listen to some SF/Fantasy on Audiobooks, but it’s usually authors I already know, or in worlds I already know).
Thanks! Fixed in comment.
Ray Dalio. Businessman, founded Bridgewater Associates, the largest hedge fund in the world. He is one of the richest people in the world.
From descriptions of Bridgewater, he seems to run it very much in line with most LessWrong principles.
In fact, if you want an instrumentally-rational and (slightly) business-oriented version of LessWrong, Ray Dalio’s principles are it. You can read here, I highly recommend it: http://www.bwater.com/Uploads/FileManager/Principles/Bridgewater-Associates-Ray-Dalio-Principles.pdf .
He is also trying to spread his take on how the economy works, in his video The Economic Machine. See it here: www.economicprinciples.org.
All in all, a fascinating person.
I agree. Perhaps this should be qualified as “most important habits that are only recommended in the Rationality community”. Otherwise there are plenty of other skills we can add (another example—start saving money early, etc).
This is amazing news, both that the book is coming along, and that there will be a professional Audiobook version. This will make it easier to spread the sequences, and may even mean that I’ll actually finish the sequences myself, something I still haven’t done.
Btw, two logistical questions (for Luke mostly):
Was there any similar campaign for the book version?
I understand that MIRI isn’t paying for this Audiobook version? I ask because my donations to MIRI are now employer-matched, as are I assume other people’s, but this Kickstarter campaign can’t be employer-matched afaik, which is a shame and a “waste” of potential donation. Any way around this? (I also donate to MIRI, but I want to specifically donate for this Audiobook being made).
Something I’m looking for:
A list of habits to take up, to improve my life, that are vetted and recommended by the community. Preferably in order of most useful to least useful. Things like “start using Anki”, “start meditating”, etc.
Do we have list like this compiled? If not, can we create it? I’m a big believe in the things this community recommends, and have already taken up using Anki, am working on Meditation, and am looking for what other habits I should take up.
FYI, I thought of this as I was reading gwern’s Dual N-Back article, in which he mentions it’s probably not worth the time, as there are much higher-potential activities to do.
(Here’s the relevant excerpt from gwern:
N-BACK IN GENERAL
To those whose time is limited: you may wish to stop reading here. If you seek to improve your life, and want the greatest ‘bang for the buck’, you are well-advised to look elsewhere.
Meditation, for example, is easier, faster, and ultra-portable. Typing training will directly improve your facility with a computer, a valuable skill for this modern world. Spaced repetition memorization techniques offer unparalleled advantages to students. Nootropics are the epitome of ease (just swallow!), and their effects are much more easily assessed—one can even run double-blind experiments on oneself, impossible with dual N-back. Other supplements like melatonin can deliver benefits incommensurable with DNB—what is the cognitive value of another number in working memory thanks to DNB compared to a good night’s sleep thanks to melatonin? Modest changes to one’s diet and environs can fundamentally improve one’s well-being. Even basic training in reading, with the crudest tachistoscope techniques, can pay large dividends if one is below a basic level of reading like 200WPM & still subvocalizing. And all of these can start paying off immediately.)