I can’t evaluate the software myself, so I’m curious to know why the downvotes. Is this a crank posting that leads nowhere, or does it publish dangerous capabilities that would lead everywhere?
FYI I reviewed and approved this user’s first post because it seemed much more specific/actually-making-claims than most of our other possibly-crank posts. I am interested in whether downvotes are more like “this is crank” or “this is AI capabilities” or “this seems likely enough to be crank that not having more information on the post-body is annoying” or what.
On a…. What? A potato? A consumer gpu that doesn’t fit all of the 7B model so it is mem-moribund? Things with “patent pending” (nothing wrong with patents!) and permitting grad students to use it “for their degrees”. Just enough little vibe nudges that I feel confused and unmotivated to actually read the code/paper.
It runs 93x faster on than Zephyr 7B on a 4060 Ti (16GB of memory) that runs them both fully within VRAM. No memory bandwidth or capacity limitations for either model. It genuinely is a pretty fair comparison. I do get into it this in the paper. Unfortunately, I’m unable to edit my original post due to negative karma.
I do understand the vibes you’re talking about on the patent vibe side of things. It’s pretty damned presumptuous—why would grad students want to use this for their degrees? Unfortunately, putting it out there without that kind of disclaimer or specific clarity is also not really something I want to do, either. Researchers and academics often assume there’s an academic exception for IP. In this case, for now, there is not. However, I wanted to make it clear clear that this did not preclude individual, unfunded research like thesis papers for the like in an academic setting.
I apologize if I didn’t deliver it well. This is my first time trying to present anything of this nature, and I’ve tried to be be careful with my messaging, but this is a section where I have to admit I had a lot of trouble.
If this is a thing (and while I understand general skepticism towards extraordinary claims of this nature, I know that it is, because I’ve been kicking the tires on it for weeks), then it is something people are going to want to study. In that case, that early clarity matters. Since I am fairly certain that’s how this is going to shake out after further evaluation, I went ahead and specified upfront.
I appreciate you taking the time to write out what put you off, though, it’s helpful feedback.
I’m at a bit of a loss as well, given the lack of comments. I haven’t even been able to directly post a direct link to the root of the github repo with the inference pipeline and a script to download the model. I’m guessing the link combined with low account karma resulted in the spam filters being triggered. I’m awaiting moderator appeal.
The downvotes have followed an interesting pattern, too—they were down to −12 some time around 9PM PST last night, and by the time I woke up due to early morning insomnia at around 1:30 PST had recovered to −4.
I have to confess that it was my strong upvote that brought it back from −12 to −4. Not because I thought it was so worthy, but to get it above the −5 default threshold for people to see it at all, which I felt it had prematurely fallen below. At some point I’ll remove the upvote to restore the cosmic balance, unless I see reason to think it is truly strongly upworthy.
And now it’s back to −11. That wasn’t me withdrawing my upvote, it was someone else whacking it with a −7. What is this, war in heaven? I do wish there was more commentary.
I can’t evaluate the software myself, so I’m curious to know why the downvotes. Is this a crank posting that leads nowhere, or does it publish dangerous capabilities that would lead everywhere?
FYI I reviewed and approved this user’s first post because it seemed much more specific/actually-making-claims than most of our other possibly-crank posts. I am interested in whether downvotes are more like “this is crank” or “this is AI capabilities” or “this seems likely enough to be crank that not having more information on the post-body is annoying” or what.
Not a downvoter, but I am put off by things like:
| Runs 93x faster than Zephyr 7B
On a…. What? A potato? A consumer gpu that doesn’t fit all of the 7B model so it is mem-moribund? Things with “patent pending” (nothing wrong with patents!) and permitting grad students to use it “for their degrees”. Just enough little vibe nudges that I feel confused and unmotivated to actually read the code/paper.
Fair points!
It runs 93x faster on than Zephyr 7B on a 4060 Ti (16GB of memory) that runs them both fully within VRAM. No memory bandwidth or capacity limitations for either model. It genuinely is a pretty fair comparison. I do get into it this in the paper. Unfortunately, I’m unable to edit my original post due to negative karma.
I do understand the vibes you’re talking about on the patent vibe side of things. It’s pretty damned presumptuous—why would grad students want to use this for their degrees? Unfortunately, putting it out there without that kind of disclaimer or specific clarity is also not really something I want to do, either. Researchers and academics often assume there’s an academic exception for IP. In this case, for now, there is not. However, I wanted to make it clear clear that this did not preclude individual, unfunded research like thesis papers for the like in an academic setting.
I apologize if I didn’t deliver it well. This is my first time trying to present anything of this nature, and I’ve tried to be be careful with my messaging, but this is a section where I have to admit I had a lot of trouble.
If this is a thing (and while I understand general skepticism towards extraordinary claims of this nature, I know that it is, because I’ve been kicking the tires on it for weeks), then it is something people are going to want to study. In that case, that early clarity matters. Since I am fairly certain that’s how this is going to shake out after further evaluation, I went ahead and specified upfront.
I appreciate you taking the time to write out what put you off, though, it’s helpful feedback.
I’m at a bit of a loss as well, given the lack of comments. I haven’t even been able to directly post a direct link to the root of the github repo with the inference pipeline and a script to download the model. I’m guessing the link combined with low account karma resulted in the spam filters being triggered. I’m awaiting moderator appeal.
The downvotes have followed an interesting pattern, too—they were down to −12 some time around 9PM PST last night, and by the time I woke up due to early morning insomnia at around 1:30 PST had recovered to −4.
I have to confess that it was my strong upvote that brought it back from −12 to −4. Not because I thought it was so worthy, but to get it above the −5 default threshold for people to see it at all, which I felt it had prematurely fallen below. At some point I’ll remove the upvote to restore the cosmic balance, unless I see reason to think it is truly strongly upworthy.
And now it’s back to −11. That wasn’t me withdrawing my upvote, it was someone else whacking it with a −7. What is this, war in heaven? I do wish there was more commentary.