Advameg, Inc. CEO
Founder, city-data.com
https://twitter.com/LechMazur
Author: County-level COVID-19 machine learning case prediction model.
Author: AI assistant for melody composition.
Advameg, Inc. CEO
Founder, city-data.com
https://twitter.com/LechMazur
Author: County-level COVID-19 machine learning case prediction model.
Author: AI assistant for melody composition.
I’d recommend posting about your challenge on http://talkchess.com/forum3/index.php. You will find people who are experienced in testing old and new chess programs and some might be interested in the prizes. If you don’t have an account there, I can post a link for you.
When it comes to medical diagnosis, I agree that the regulations will slow the adoption rate in the U.S. But then there is China. The Chinese government can collect and share huge amounts of data with less worry about privacy. And looking at the authors of ML papers, you cannot miss Chinese names (though some are U.S.-based, of course).
Your statement about AI copy editors is definitely true (I have some first-hand knowledge about what’s possible but not yet publicly available).
Related development: https://www.nature.com/articles/d41586-021-01968-y
“Meanwhile, an academic team has developed its own protein-prediction tool inspired by AlphaFold 2, which is already gaining popularity with scientists. That system, called RoseTTaFold, performs nearly as well as AlphaFold 2, and is described in a paper in Science paper also published on 15 July[2] ”
Have you read “Free Will” by Mark Balaguer https://mitpress.mit.edu/books/free-will ? Your argument is similar in some ways.
I think your scheme might not be most effective if you write your daily updates without getting any input, even if you somehow know that people read them. I think I would give up at some point unless I was getting relevant feedback (it is better than nothing though).
What might work better is a buddy-type system with two or more people with the same procrastination problem, preferably also interested in the same subject, to keep each other accountable. If you’d like, I can do this with you for a period of time in addition to your updates and you’d get an idea of what’s better.
I see that there is already a paid service https://actionbuddy.io/ that somehow matches people (no endorsement). I could set up a free site that does matching based on expressed interests if people are interested.
According to https://www.sciencedirect.com/science/article/pii/S0896627321005018?dgcid=coauthor
“Cortical neurons are well approximated by a deep neural network (DNN) with 5–8 layers ”
“However, in a full model of an L5 pyramidal neuron consisting of NMDA-based synapses, the complexity of the analogous DNN is significantly increased; we found a good fit to the I/O of this modeled cell when using a TCN (my note: temporally convolutional network) that has five to eight hidden layers ”
For best performance, the width was 256.
Since L5 neurons can perform as small neural nets, this might have implications for the computational power of brains.
There is a new study out that found that 40% of Copilot’s code contributions in high-risk scenarios were vulnerable: https://arxiv.org/abs/2108.09293
Why not go one step further and allow using throwaway accounts or not publicly link to the poster’s profile? The association would still be visible to the administrators/moderators in case some action needs to be taken.
On the forum I run, we have 2 million+ registered users without real names but with emails and I’m sure that there are also users here who would like to post some things semi-anonymously. Whether this option actually gets used would depend a ton on the defaults and how this choice is presented visually.
My new go-to example of the EMH being false is the merger arbitrage opportunity between TLRY and APHA from earlier this year: https://marketrealist.com/p/tilray-tlry-aphria-apha-merger-date-arbitrage/. I was able to take advantage of this. There are often underappreciated gotchas when it comes to things like this that novice investors miss, e.g. shorting costs or risks of early exercise of options.
By the way, I’d be interested in getting in touch with other semi-pro individual traders in order to brainstorm/share such opportunities privately.
Some anecdotal evidence: in the last few months I was able to improve on three 2021 conference-published, peer-reviewed DL papers. In each case, the reason I was able to do it was that the authors did not fully understand why the technique they used worked and obviously just wrote a paper around something that they experimentally found to be working. In addition, there are two pretty obvious bugs in a reasonably popular optimization library (100+ github stars) that reduce performance and haven’t been fixed or noticed in “Issues” for a long time. Seems that none of its users went step-by-step or tried to carefully understand what was going on.
What all four of these have in common is that they are still actually working, just not optimally. Their experimental results are not fake. This does not fill me with hope for the future of interpretability.
I’d like to but it’ll have to wait until I’m finished with a commercial project where I’m using them or until I replace these techniques with something else in my code. I’ll post a reply here once I do. I’d expect somebody else to discover at least one of them in the meantime, they’re not some stunning insights.
As you probably know, there are multiple theoretically-interesting ML ideas that achieve very good results on MNIST. Have you tried more challenging image recognition benchmarks, such as CIFAR-100, or some non-CV benchmark? Since you posted your code, I wouldn’t mind spending a bit of time looking over what you’ve accomplished. However, MNIST, which is now considered pretty much a toy benchmark (I don’t consider PI-MNIST to be a better benchmark), will likely be an obstacle to get others to also look at it in-depth, as it will be considered quite preliminary. Another practical point: using C and CUDA kernels also makes it less accessible to a good percentage of researchers.
I don’t think PI-MNIST SOTA is really a thing. The OP even links to the original dropout paper from 2014, which shows this. MNIST SOTA is much less of a thing than it used to be but that’s at 99.9%+, not 98.9%.
I didn’t mean “CUDA kernels” as in requiring NVIDIA GPUs—that’s fine. I meant that you’re limiting the readability of your code to a subset of people who understand both ML and CUDA programming. In my experience, this limits the reach, especially among younger researchers (I’ve hired C++ programmers and ML researchers for my business).
But, of course, you can choose to promote (or not) your work however you prefer.
One of these improvements was just published: https://arxiv.org/abs/2202.03599 . Since they were able to publish already, they likely had this idea before me. What I noticed is that in the Sharpness-Aware Minimization paper (ICLR 2021, https://arxiv.org/abs/2010.01412), the first gradient is just ignored when updating the weights, as can be seen in Figure 2 or in pseudo-code. But that’s a valuable data point that the optimizer would normally use to update the weights, so why not do the update step by using a value in between the two. And it works.
The nice thing is that it’s possible to implement this without increasing the memory requirements or the compute (almost) compared to SAM: you don’t need to store the first gradient separately, just multiply it by some factor, don’t zero out the gradients, let the second gradient be accumulated, and rescale the sum.
In Miami there absolutely are explicit payments for models to join tables. This can lead to all of them leaving on the dot at let’s say 3:00 AM, since that’s how long they were required to stay at the club to earn their pay. NYC has a different dynamic.
I don’t know how much you want to go into the minutia to make it as accurate as possible but for shorting you really need to consider borrow fees and share availability. For example, I wanted to short DWAC (a SPAC that is merging with Trump’s social media company) for a long while but these issues made it impossible (I also couldn’t make a play using options because they reflected the negative outlook).
Ethereum’s market cap is 47% of Bitcoin’s. While you can argue that the market cap of cryptocurrencies is arbitrary, the price of one coin is even more arbitrary.
Short note: I calculated a more accurate estimate of the number of positions in chess and it’s at 8.7E+45, not 1E+120: https://github.com/lechmazur/ChessCounter .
Stockfish 12 and newer have neural network (NNUE)-based evaluation enabled by default so I wouldn’t say that Stockfish is similar to other non-NN modern engines.
https://nextchessmove.com/dev-builds is based on playing various versions of Stockfish against each other. However, it is known that this overestimates the ELO gain. I believe +70 ELO for doubling compute is also on the high side, even on single-core computers.