Yes, this is something that I’ve wondered about quite a bit specifically in connection with the variation in conscientiousness and agreeableness by religion. I plan on partially addressing this issue by discussing some objective behavioral proxies to the personality traits in later posts.
JonahS
Karma: 3,476
What I had in mind was that the apparent low average conscientiousness in the Bay Area might have been one of the cultural factors that drew rationalists who are involved in the in-person community to the location. But of course the interpretation that you raise is also a possibility.
Glad you liked it :-).
So I’d be interested to hear a little more info on methodology—what programming language(s) you used, how you generated the graphs, etc.
I used R for this analysis. Some resources that you might find relevant:
Practical Data Science with R has very nice introduction to exploratory data analysis.
Advanced R goes into more detail on the language.
The graphs were made using ggplot2.
I used the lme4 package for Bayesian hierarchical modeling. See, e.g. Getting Started with Mixed Effect Models in R.
Kaggle Kernels has some good sample scripts.
And depending on how far back most of this data was collected, plausibly most of the Berkeley respondents were high school or college students (UC Berkeley alone has over 35,000 students), since for awhile that was the main demographic of Facebook users, and probably for awhile longer that was the main demographic of Facebook users willing to take personality tests.
Douglas_Knight is correct – the average age of users is quite low, at ~26 years old both for the high conscientiousness cities and the low conscientiousness cities.
How does personality vary across US cities?
Physics is established, so one can defer to existing authorities and get right answers about physics. Starting a well-run laundromat is also established, so ditto. Physics and laundromat-running both have well-established feedback loops that have validated their basic processes in ways third parties can see are valid.
Depending on which parts of physics one has in mind, this seems possibly almost exactly backwards (!!). Quoting from Vladimir_M’s post Some Heuristics for Evaluating the Soundness of the Academic Mainstream in Unfamiliar Fields:
If a research area has reached a dead end and further progress is impossible except perhaps if some extraordinary path-breaking genius shows the way, or in an area that has never even had a viable and sound approach to begin with, it’s unrealistic to expect that members of the academic establishment will openly admit this situation and decide it’s time for a career change. What will likely happen instead is that they’ll continue producing output that will have all the superficial trappings of science and sound scholarship, but will in fact be increasingly pointless and detached from reality.
Arguably, some areas of theoretical physics have reached this state, if we are to trust the critics like Lee Smolin. I am not a physicist, and I cannot judge directly if Smolin and the other similar critics are right, but some powerful evidence for this came several years ago in the form of the Bogdanoff affair, which demonstrated that highly credentialed physicists in some areas can find it difficult, perhaps even impossible, to distinguish sound work from a well-contrived nonsensical imitation.
The reference to Smolin is presumably to The Trouble With Physics: The Rise of String Theory, the Fall of a Science, and What Comes Next . Penrose’s recent book Fashion, Faith, and Fantasy in the New Physics of the Universe also seems relevant.
A few nitpicks on choice of “Brier-boosting” as a description of CFAR’s approach:
Predictive power is maximized when Brier score is minimized
Brier score is the sum of differences between probabilities assigned to events and indicator variables that are are 1 or 0 according to whether the event did or did not occur. Good calibration therefore corresponds to minimizing Brier score rather than maximizing it, and “Brier-boosting” suggests maximization.
What’s referred to as “quadratic score” is essentially the same as the negative of Brier score, and so maximizing quadratic score corresponds to maximizing predictive power.
Brier score fails to capture our intuitions about assignment of small probabilities
A more substantive point is that even though the Brier score is minimized by being well-calibrated, the way in which it varies with the probability assigned to an event does not correspond to our intuitions about how good a probabilistic prediction is. For example, suppose four observers A, B, C and D assigned probabilities 0.5, 0.4, 0.01 and 0.000001 (respectively) to an event E occurring and the event turns out to occur. Intuitively, B’s prediction is only slightly worse than A’s prediction, whereas D’s prediction is much worse than C’s prediction. But the difference between the increase in B’s Brier score and A’s Brier score is 0.36 − 0.25 = 0.11, which is much larger than corresponding difference for D and C, which is approximately 0.02.
Brier score is not constant across mathematically equivalent formulations of the same prediction
Suppose that a basketball player is to make three free throws, observer A predicts that the player makes each one with probability p and suppose that observer B accepts observer A’s estimate and notes that this implies that the probability that the player makes all three free throws is p^3, and so makes that prediction.
Then if the player makes all three free throws, observer A’s Brier score increases by
3*(1 - p)^2
while observer B’s Brier score increases by
(1 - p^3)^2
But these two expressions are not equal in general, e.g. for p = 0.9 the first is 0.03 and the second is 0.073441. So changes to Brier score depend on the formulation of a prediction as opposed to the prediction itself.
======
The logarithmic scoring rule handles small probabilities well, and is invariant under changing the representation of a prediction, and so is preferred. I first learned of this from Eliezer’s essay A Technical Explanation of a Technical Explanation.
Minimizing logarithmic score is equivalent to maximizing the likelihood function for logistic regression / binary classification. Unfortunately, the phrase “likelihood boosting” has one more syllable than “Brier boosting” and doesn’t have same alliterative ring to it, so I don’t have an actionable alternative suggestion :P.
Brian Tomasik’s article Why I Prefer Public Conversations is relevant to
I suspect that most of the value generation from having a single shared conversational locus is not captured by the individual generating the value (I suspect there is much distributed value from having “a conversation” with better structural integrity / more coherence, but that the value created thereby is pretty distributed). Insofar as there are “externalized benefits” to be had by blogging/commenting/reading from a common platform, it may make sense to regard oneself as exercising civic virtue by doing so, and to deliberately do so as one of the uses of one’s “make the world better” effort. (At least if we can build up toward in fact having a single locus.)
Wait, your category (ii) is surely exactly what we care about here.
Yes, I see how my last message was ambiguous.
What I had in mind in bringing up category (ii) is that we’ve had some students who had a priori worse near term employment prospects relative to the usual range of bootcamp attendees, who are better positions than they had been and who got what they were looking to get from the program, while not yet having $100k+ paying jobs. And most students who would have gotten $100k+ paying jobs even if they hadn’t attended appear to have benefited from attending the program.
The nature of the value that we have to add is very much specific to the student.
Hello! I’m a cofounder of Signal Data Science.
Because our students have come into the program from very heterogeneous backgrounds (ranging from high school dropout to math PhD with years of experience as a software engineer), summary statistics along the lines that you’re looking for are less informative than might seem to be the case prima facie. In particular, we don’t yet have meaningfully large sample of students who don’t fall into one of the categories of (i) people who would have gotten high paying jobs anyway and (ii) people who one wouldn’t expect to have gotten high paying jobs by now, based on their backgrounds.
If you’re interested in the possibility of attending the program, we encourage you to fill out our short application form. If it seems like it might be a good fit for you, we’d be happy to provide detailed answers to any questions that you might have about job placement.
Yes, that was supposed to be June 24th! We have a third one from July 5th – August 24th. There are still spaces in the program if you’re interested in attending.
Thanks for the written feedback (which adds to what I had gleaned in person).
There were actually multiple times during the first couple weeks when I (or my partner and I) would spend 4+ hours trying to fix one particular line of code, and Jonah would give big-picture answers about e.g. how linear regression worked in theory, when what I’d asked for were specific suggestions on how to fix that line of code. This led me to giving up on asking Jonah for help after long enough.
I think that what happened here is me having misunderstood what you were asking for, rather than any disinclination on my part to help you with individual lines of code. I will take this feedback into account.
Intermediate and advanced SQL, practice of certain social skills (e.g. handshakes, being interested in your interviewer, and other interview-relevant social skills), and possibly nonlinear models.
This is helpful detail regarding what you were looking for. Which topics would you have preferred to have been been dropped in favor of these?
Hi Toggle,
Thanks for your question!
Most of our students have just started looking for jobs over the past ~2 weeks, and the job search process in the tech sector typically takes ~2 months, from sending out resumes to accepting offers (see, e.g. “Managing your time” in Alexei’s post Maximizing Your Donations via a Job).
The feedback loop here is correspondingly longer than we’d like. We expect to have an answer to your question by the time we advertise our third cohort.
An update on Signal Data Science (an intensive data science training program)
Thanks for your interest! Some responses below.
Do you require applicants to have a graduate degree?
No degree is required. We’re selecting on ability rather than on credentials.
Zipfian Academy, App Academy, and other bootcamps are 12 weeks long, and (the first instance of) this one is only 6 weeks long. Why is this, and what are you cutting out relative to other data science bootcamps to make it this short? (This is my most pressing question).
Based on the preliminary interest that people have expressed anticipate that the students in our first cohort will be significantly stronger than is typical of data science bootcamps, and will correspondingly be able to cover the material at an accelerated pace. We expect at least some of our cohorts to run a full 12 weeks.
Regarding the comparison with coding bootcamps, there are reasons to believe that the amount that somebody needs to know to be in the top x% of industry data scientists is less than the amount that’s needed to be in the top x% of programmers. (I can elaborate.)
We’re cutting out some of the more advanced machine learning algorithms, which industry data scientists use infrequently enough so that they can be a distraction from getting started.
As a tie-in to my last question, is there a hiring event which employers will be invited to around the end of the program?
Very few bootcamp students who I know got their jobs through this route, so we may or may not do this depend on how efficient it is relative to other routes. Like other bootcamps that offer the “pay later” model, we have a large stake in ensuring that our students find jobs.
Do you know which language(s) you’ll be using?
We’ll be working primarily in R, and teaching SQL as well.
Yes, we’ll definitely be covering this.
Thanks for the suggestion. That would be wonderful. We’ll definitely think about this – it’s a matter of whether we can create a sufficiently simple presentation of the material so that the marginal returns per unit time are high for the student population that we’ll be working with.
Announcing the Signal Data Science Intensive Training Program
It might be that I have gotten to cynic but if you measure 6 variables it’s more likely that one of them get a statistical significant result then if you first turn those 6 variables into 2 variables via PCA.
Yes, this is the point :-)
I’m sure you’re aware that the word “cult” is a strong claim that requires a lot of evidence, but I’d also issue a friendly warning that to me at least it immediately set off my “crank” alarm bells.
Thanks, yeah, people have been telling me that I need to be more careful in how I frame things. :-)
Do you have evidence of legitimate mathematical results or research being hidden/withdrawn from journals or publicly derided, or is it more of an old boy’s club that’s hard for outsiders to participate in and that plays petty politics to the damage of the science?
The latter, but note that that’s not necessarily less damaging than active suppression would be.
Or maybe most social behavior is too cult-like. If so; perhaps don’t single out mathematics.
Yes, this is what I believe. The math community is just unusually salient to me, but I should phrase things more carefully.
I question the direction of causation. Historically many great mathematicians have been mentally and socially atypical and ended up not making much sense with their later writings. Either mathematics has always had an institutional problem or mathematicians have always had an incidence of mental difficulties (or a combination of both; but I would expect one to dominate).
Most of the people who I have in mind did have preexisting difficulties. I meant something like “relative to a counterfactual where academia was serving its intended function.” People of very high intellectual curiosity sometimes approach academia believing that it will be an oasis and find this not to be at all the case, and that the structures in place are in fact hostile to them.
This is not what the government should be supporting with taxpayer dollars.
Especially in Thurston’s On Proof and Progress in Mathematics I can appreciate the problem of trying to grok specialized areas of mathematics.
What are your own interests?
I would like to see an (optional) personality test section—email me at jsinick@gmail.com if you’re interested in the possibility, as I have some detailed thoughts.