People's Scaling Pill @ AGI

>>62183 What happens before our eyes is, indeed, the development of increasingly more general artificial intelligence, and old academic paradigms (cognitive science, symbolic and logical AI, VC-theory) failed to lead us here. Instead what best explains the success of deep learning is a combination of Scaling Hypothesis and compressionism: https://www.gwern.net/Scaling-hypothesis https://arxiv.org/abs/2006.15191 https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2004.10802 https://ceur-ws.org/Vol-1419/paper0045.pdf Basically, when you apply SGD with self-supervised objective to a large enough deep learning model and dataset, you get an increasingly more intricate model of data. With enough scale this model undergoes phase changes lowering its error rate https://timdettmers.com/2022/08/17/llm-int8-and-emergent-features/ and one specific phase change from this natural sequence could as well be called "reaching self-awareness". When you are on that curve, it's a question of "when", not "if". We are currently at that point where the fundamental ideas like basic scaling laws and training schemes are in place, and even the science-hungry luminaries like Demis Hassabis and Nando de Freitas concede that what largely remains is "engineering", while rockstar programmers like John Carmack (yes, the one) raise 20M$ for his AGI startup https://techcrunch.com/2022/08/19/john-carmack-agi-keen-raises-20-million-from-sequoia-nat-friedman-and-others/ Of course there is and will be a camp of AI deniers - most dumb, but some quite smart and sour-graped (esp. academics from the deadend research directions), devising increasingly laughably involved coping schemes for moving the goalposts with each new public AI release. On the other hand, there are the usual subjects: silicon valley C-levels, with their cringe and time-honed aptitude for hyping their product - it's easy not to fall for this hype (most SV products are trashy), but even a broken clock is correct twice a day, and this might be our case here. Anyway, people who have read this paper https://arxiv.org/abs/2205.06175 and still didn't get how little distance remains between us and commodified AGI are going to watch history pass past them like a bullet train.

I think it is pretty clear we can already create machines that are intelligent with current approaches. The real question is "how intelligent". Fnord tried-and-true methods for tripping up computers in Turing tests still work, albeit you have to be more savvy in their use (you basically need to apply the same strategies that police interrogators use). Computers are ironically much worse at math, currently, than they are at art. It would be interesting to try giving computers an IQ test.

>>64481 >>62183 contd 2 of 3 Lainchan doesn't allow me to post the remaining 2/3 of the post as a text, so, have it as a screenshot. ... And my matrix account (don't hesitate to send "ping") @compactlatentspace:halogen.city >>64484 The idea of IQ-test compatible with humans and machines has already been implemented, see https://github.com/fchollet/ARC https://arxiv.org/abs/1911.01547 by François Chollet. Also there is more pragmatic BIG-bench I already mentioned https://github.com/google/BIG-bench https://arxiv.org/abs/2206.04615

>>64489 >>62183 contd 3 of 3, again, mostly as a screenshot due to lainchan's limits If anyone wants to discuss these issues and our spectacular place in history in greater depth, please don't hesitate to reach me via https://matrix.to/#/@compactlatentspace:halogen.city

OK, so we create a general artificial intelligence. So what? Why is it important? What does it even actually do? Will it make Alexa and Siri not brain-dead? Can it do something humans can't already do? Can it do it cheaper? I'm strongly convinced that AI, if it's actually created, will just end up being another sci-fi inspired impractical hi-tech gizmo that nobody actually uses. Like kinect or VR helmets. A toy, fun for an hour or so.

>>64504 >OK, so we create a general artificial intelligence. We already do, it's called having babies. >So what? Although babies are weak initially, they tend to grow to become strong, at which point you'd like to be on their good side. >Alexa and Siri not brain-dead? They serve their masters well. Reminder: You are not their master. >Can it do it cheaper? Most tools and technologies have, yes. >another sci-fi inspired impractical hi-tech gizmo Alchemists never discovered how to transmute lead into gold, and yet gunpowder and other chemistry revolutionized (or destroyed, depending on your perspective) society. The conquistadors never discovered the Seven Cities of Gold, and yet colonization of the New World revolutionized (or destroyed, depending on your perspective) society. Democracy requires a well-armed, well-educated populace. That is why we use linux instead of windows, and that is why we must use libre AI instead of proprietary AI. Because we do not wish to be slaves. I invite you to the matrix room mentioned earlier.

>>64684 Messing around with pytorch on cpu-only. Useful tutorial (use videospeedcontroller plugin): https://www.youtube.com/watch?v=Z_ikDlimN6A I drew a 16x16 ascii art and have a method to generate random 16x16 ascii spam. I'm thinking about how to get the computer to generate a 16x16 ascii art of its own based on a few hand-drawn samples by me and a lot of randomly-generated 16x16 ascii spam examples to contrast.

Bump. Anything new? The author disappeared mysteriously

>>64490 >Those images Oh, those were my posts. Thank you for the reply. I hadn't really considered using libre AGI because I assumed the death scenario where there is never a chance to pull that off. I hope we can all become friends who use our hacked together GNU/AGI together. GNU+AGI if you prefer.

Here is a relevant XKCD lain!.

>>64504 This question itself is a sign of lack of imagination, perhaps depression even. "So what?" you say? So what if we have a 80GB magnet link and a github repo you can run on your RTX3090s https://nonint.com/2022/05/30/my-deep-learning-rig/ to get a prompt access to a quite intelligent, helpful virtual assistant who is able to understand and respond to instructions expressed in natural language as well as images, videos? Add to it such features as fine-tuning on your textbooks and problems of interest, web search access and the basic digital file virtue of being able to copy it as you see fit, and you should see how this binary, loyal, reproducible knowledge worker could be qualitatively better than your average 100IQ college grad which loses to it in the sheer amount of world knowledge internalized. >Can it do something humans can't already do? Can it do it cheaper? Just to nail the point, do you understand how large the price we currently pay in the West to raise and educate +1 smart-ish person able to follow orders and do useful office work? Compare it to the price of buying an aforementioned deep learning rig and paying for electricity. And I have already seen superhuman feats of creative erudition from these models, and it's not yet the limit of what could be run on aforementioned hardware with modern quantization and sparsification. Anyway, what immediately comes to Lain's mind after being presented with such a cheap private source of knowledge labor is an R&D program spanning new frontiers in gene therapy, cognitive enhancement, life extension and novel manufacturing technologies. Ambitious enough? If not, you might make it code a game you have always wanted to make but never did - have no ideas? Prompt it to generate a selection of ideas of whole design docs first - do not have taste to select one? You know the answer. Let it work through the night - "Let's think step by step..." https://notabird.site/arankomatsuzaki/status/1529278581884432385 >>64765 I'm alive and well, research continues. Picture unrelated - gift from G, a small brain for a SoyBot: https://github.com/google-research/robotics_transformer - but thanks for the checkpoint provided, I will find use for it, in the simulation at least. Interested in joining us, Lain? The link is here >>64490

>>64910 >do you understand how large the price we currently pay in the West to raise and educate +1 smart-ish person able to follow orders and do useful office work? to be honest most office jobs just require basic literacy, numeracy, and people skills. So, stuff that most people have accomplished by the time they are 12 years old. It looks like this stage of education, in the US (where almost everything is more expensive than it needs to be), is about $20k per child per year at most. https://www.publicschoolreview.com/blog/how-much-does-it-cost-to-educate-a-child . if we assume a 30 year career, this comes out to $4000 per year. Considering these are things which undeniably benefit the person, their family, and society at large, in addition to their employer, it makes sense that the cost of doing this would be spread among more people than just the employer as well.