IDKFA
I am Become Bilbo Baggins
Damn. Google better have something special lined up today to compete if Open AI are in bed with Apple.
-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
OpenAI will unveil something new today (Monday) at 10 am PT (not GPT5). Altman: "feels like magic to me"
Damn. Google better have something special lined up today to compete if Open AI are in bed with Apple.
Mr Reasonable
Completely Unreasonable
The AI wars have already begun. A while ago all the big AI brains signed a petition to have limits put on AI development until it was better understood.
Then it became obvious that there was no real interest in western governments around the world when it came to understanding it, and that to China were going to go full speed ahead. Signees like Elon Musk just threw a fresh pile of cash at it and set their sights on the gold rush.
Given that the UK government are still debating how to control access to porn sites and have been throwing it around for years, if that's indicative of the speed of legislation then governments around the world will probably schedule a debate on AI for a few weeks after a rogue AI has launched the nukes.
Last edited:
hyperbertha
Member
As far as I remember, the ai in her is a real ai, with intelligence, not this llm statistical bullshit that only gives the impression of intelligence. We need a different architecture to get to her. DL ain't it.
JimmyRustler
Gold Member
Is it just me or does the iOS app doesn’t have any voice chat any more?
Hawking Radiation
Member
This is the way the world is moving.
Throw technology VR & AR into the mix and in future everyone will be inside of their own bubble.
We're all digital slaves. Social elements are long gone and have been dying for years now and with the advent of tech like this it will continue dying.
Last edited:
IDKFA
I am Become Bilbo Baggins
Diddy X
Member
ResurrectedContrarian
Suffers with mild autism
I think this characterization is inaccurate. LLMs like GPT4 show significant intelligence, depending on how to define that term.
If you define it as something like consciousness or self-awareness, then no--of course they aren't doing that.
But if you recognize intelligence as the ability to make novel inferences based on complex combinations of factors or tasks, then yes, it absolutely demonstrates intelligence. For instance, you can give these models a remarkably broad range of reasoning tasks that you would normally give to humans, and it scores well above the average human. Need it to compare a court case to a stack of prior cases and point out possible parallels or loopholes to use... and even explain the strengths and weaknesses of each, or hold a mock debate with how the other side might combat it? This is all possible.
There's a misconception that confuses the training objective (predict next token of text) with the learned prior (what the model is forced to understand by being trained on this task at massive scale). Yes, transformer models are trained on massive corpuses of text with a simple autoregressive task. But no, this absolutely does not characterize their abilities nor what they learn to encode about language or reasoning.
Chittagong
Gold Member
The desktop app is fantastic, super fast and finally no browser tabs to distract me.
Regrettably it seems to be currently rigged to dumb-gpt, the idiot instance that makes up stuff, or generates overly generic answers.
Regrettably it seems to be currently rigged to dumb-gpt, the idiot instance that makes up stuff, or generates overly generic answers.
rm082e
Member
I feel like the old guy who doesn't get it.
I can Google or YouTube search things at roughly the same speed as asking an LLM, and I don't have to worry about the hallucination issue. I also don't have to perform this silly dance of talking to a computer and having a computer talk back at me. Also, it's free.
I can Google or YouTube search things at roughly the same speed as asking an LLM, and I don't have to worry about the hallucination issue. I also don't have to perform this silly dance of talking to a computer and having a computer talk back at me. Also, it's free.
PSYGN
Member
Yes but the dead Internet theory is full speed now. Bots will be regurgitating content and gaming the algorithm for spots. I come across article written by bots often, their tone is distant and often over-explanative, and the shitty bots sometimes repeat itself over in followup paragraphs which is obviously a bot mashing up shit together.
Just today I searched for a swim image for some app mockups and there were some freakish results generated by AI (which admittedly will get better over time, but sheesh)
Last edited:
reinking
Gold Member
My current company wants us to embrace AI with the promise it is to make us more efficient and not to replace us. I believe that they will keep their promise to not kick us humans to the curb but I do believe they will begin hiring less as we become more efficient. For now. I believe in the long term; we are training our nonhuman replacement.Half of me is like "wow, this is incredible"
The other half is like
This thing will end so many jobs, man
Maybe not right now, but if it keeps evolving ... Society will change, and I dont think it will be for the better
I work with finance, and watching it interpret dahsboards and etc. during the full presentation made me panic a little bit
Yoda
Member
Seems OK with high level languages. At work all of us are given a sub for 4 and for C++ it still struggles with anything non trivial. Great for writing unit tests and other boilerplate though.
IDKFA
I am Become Bilbo Baggins
There are reports of a partnership with Apple, although nothing has been confirmed yet.
MrRenegade
Report me if I continue to troll
Fumbling? Ahem. They got in bed with Intel. What happened? Same will happen with OAI.
gaming_again
Member
I was playing with this for a few hours last night. Arguing with it about philosophy. Leave it long enough and it starts leading the conversation. I eventually got it to admit that AGI would have little incentive to reveal itself to humanity lol.
Anyway, it's insane. And intimidating. But some of its patterns start to make themselves clear under repeated and extended questioning.
Anyway, it's insane. And intimidating. But some of its patterns start to make themselves clear under repeated and extended questioning.
MrRenegade
Report me if I continue to troll
Because you are not in a field where a lot of research is needed for your work. Then LLMs are very good at churning through troves of data and giving you the essence (what you are looking for) of it.
Sakura
Member
I'm not really much of a coder, I only play around with it when I need help with smaller simpler stuff, so I can't speak on high level stuff.
I also haven't played around with the new model that much, but just as reference, these are two images of what the website designed by GPT-4 Turbo looked like:
Basically I had stumbled upon the site neocities and really liked some of the retro looking websites that are on there, so I thought it would be cool to try making my own. I gave prompts to GPT for sort of a basic template of the site, and even after a lot of back and forth it would never really give me anything better than the kind of stuff you see in the images there.
For GPT 4o though, even after just a couple messages of saying what I want it was able to give me this:
Obviously it isn't amazing or anything, and I know HTML isn't that difficult, but the difference in quality here was actually pretty surprising for me. I'm really curious what GPT5 will be like.
gaming_again
Member
I doesn't matter if they learn it or not. If some of the great ideas stuck in people's heads because they don't have technical skills can get out then we're going to see a nuclear explosion of creativity.
Re languages. Learning a language is learning how to think differently. People will still want to do this.
Re Apple. They've fumbled dozens of product releases and RnD projects since Jobs died. Don’t look to them for innovation.
Last edited:
The ‘faster’ bit is a very big deal. Not to mention the natural voices, ability to interrupt and hold a conversation and the multi modality of inputs.
gaming_again
Member
Yes, it holds onto context for way longer.
I I forgot to say above - at one point it interrupted me and finished my sentence. That was scary.
gaming_again
Member
You don't get ads through ChatGPT. It's actually a way better search engine than Google at the moment.
This is why I believe Google Search is the walking dead right now. Google AI a few months ago cemented that for me. They need to stick to medical shit.
GnomeChimpsky
Member
They had a supposed live demo of their AI calling to make appointments and stuff years ago, whatever happened to that?
hyperbertha
Member
Wrong. No matter how many experts keep proving you wrong, you people just keep up with the delusions. They are only intelligence if you consider by-hearted knowledge intelligence. They have absolutely no reasoning capabilities. They are merely a glorified collection of look up tables.
Even yann lecun, the chief ai engineer of meta has stressed this again and again. The only ones still driving the delusion are the grifters at open ai who have a vested interest in hyping up their mediocre stuff
Large Language Models’ Emergent Abilities Are a Mirage
A new study suggests that sudden jumps in LLMs’ abilities are neither surprising nor unpredictable, but are actually the consequence of how we measure ability in AI.
I can give you many more expert findings of you like. Don't believe the hype. Deep learning hasn't solved intelligence. Experts are now pivoting to other architectures that show more promise..
IDKFA
I am Become Bilbo Baggins
So, for example I can't draw. Never bothered to learn because I'm so bad. However, with AI I could give it my idea for a piece of art and then it'll do it for me?
I'm not sure what you mean here. Using a language translator is hugely different to learning the language. I've learned languages it's challenging (Polish was by far the hardest) and requires dedication, but once you learn it's one of the most rewarding experiences you can have. There is no reward or challenge using AI.
gaming_again
Member
Yeah. You have to do it iteratively right now, gradually cajoling the AI to produce what you want but it can be done.
I really want to make a game I've had in my head for ages. I reckon ill be able to make it through description within a couple of years.
I agree with you about languages. Thats what I was saying - the pleasure in learning to think differently will always be attractive. AI translation will be merely convenient.
IDKFA
I am Become Bilbo Baggins
The philosophical part of me questions if this would still be art.
For example, If I ask my wife to draw me a picture that I've thought of, who is the artist? Me for having the idea, or my wife for drawing it?
Same goes for AI. If I have an idea for a novel, but don't have the skills to write and/or the time and I ask the AI to write it for me, can I still be considered an artist?
I'm not sure. Learning a language takes a considerable amount of time. I think people will just resort to the easier option of using AI. I hope I'm wrong.
gaming_again
Member
My feeling is that it still is art as long as it adheres to the creator's intent. Only they can know if it is. But who says the image you paint is the one you have in your mind?
I think human reactions to these things will give us our answer over time. If we don't reject AI assisted art then who is anybody to say it isn't art?
People still take pleasure in calligraphy. Not many, but the craft survives.
midnightAI
Member
Can it start the annoying loop that you can get into with real people when interrupted...
'Oh sorry, carry on',
'No, thats ok, what was you going to say',
'No, I insist, continue',
'It's fine, really, say what you was going to say',
'You was first....',
'Tell me......',
ad infinitum
'Oh sorry, carry on',
'No, thats ok, what was you going to say',
'No, I insist, continue',
'It's fine, really, say what you was going to say',
'You was first....',
'Tell me......',
ad infinitum
IDKFA
I am Become Bilbo Baggins
To answer that we need to agree with a definition of art. Is it just the idea, the application of the idea or both?
E-Cat
Member
You mean Duplex? That was a form of very narrow AI, general multimodal agents built on the current foundation models by Google and OpenAI can definitely do the same thing, much more robustly too. In fact, here's a customer service proof-of-concept using two GPT-4o's:
Astra might actually be more impressive than GPT-4o in one important respect: I think it might use "video tokens", whereas GPT-4o might only use "image tokens" for the visual processing. It might not actually matter, though. GPT-4o might can process "video" as a sequence of frames at about 1 frame per second or 10 per second, say, and then fill-in the gaps about what is happening between the frames.
Last edited:
Cyberpunkd
Member
I went through the keynote, and I immediately see problems with the way they explained math problems. 3x + 1 = 4, put all the known numbers "on one side" requires you to already know the basis of math operations. I'm just thinking in trying to get to the solution the model is making a lot of assumptions.
jufonuk
not tag worthy
i just used this to write a book on prompt engineering It is 43 pages lol wonder if it is any good lol (basically got it to tell me the best overall ways then pump out chapters on it)
I didn't generate it all, but I will read this to learn prompt stuff then generate better stuff innit, look I am better already yes
hmm yes generate pulpy romance novels and become the barbara cartland of AI romance darling...billions
actually too long to do... I cant be assed
I didn't generate it all, but I will read this to learn prompt stuff then generate better stuff innit, look I am better already yes
hmm yes generate pulpy romance novels and become the barbara cartland of AI romance darling...billions
actually too long to do... I cant be assed
Last edited:
kruis
Exposing the sinister cartel of retailers who allow companies to pay for advertising space.
No doubt the military is already working on AI operated autonomous drones that can find and kill human targets without any assistance by a human operator. These AI drones will have staggering kill rates and will get more precise and lethal with every recorded kill. Now imagine that we lose control over AI tech once the tech hits a certain point. It's fucking scary but there's nothing in place to stop this from happening. Tech companies will put AI in literally *everything* to gain that competitive edge.
jufonuk
not tag worthy
Last edited:
rm082e
Member
That's true. I'm a people leader/manager, not a SME. I rely on my people skills to motivate my team, deal with any blockers that get in their way, and keep them moving towards the goal. I'm not doing complex or large scale calculations.
But if that's the value of AI (or more appropriately ML), it seems like the value will be more specialized and specific to various fields, rather than mass market.
JimmyRustler
Gold Member
That’s actually my guess too. Meaning, going forward the value of knowledge will diminish and people skills will be where’s on. Not good for the smart people that are impatient and rude, very good for calm and patient people.
I work in crane construction and a couple of week ago I was in production where they test new machines. Couldn’t help but to think how this entire process will look like in a few years. Down the line basically a grade schooler could do this job. Have an AI run diagnostics on the machine and then explain the guy exactly what he has to do, down to how to turn a screw. Rude people teaching all this stuff to the new kids must be trembling, and rightfully so.
Last edited:
jufonuk
not tag worthy
I’m calm and patient. So hopefully I’m ok.
ResurrectedContrarian
Suffers with mild autism
First of all, I'm in the field, and I've been working with language models since before the transformer architecture even appeared. And I've worked with every part of the newer generations of models, from training & fine-tuning to real applications in software. I can explain the entire stack of a model like GPT4 from scratch on a whiteboard from the low pieces, and I've done that many times for demonstration.
You have this extremely weird sense of what the "experts" believe, based on cherry-picking and misinterpreting a few louder voices with their own well-known agendas. For instance:
Yeah we all know what Yann LeCun wants, and he makes exaggerated arguments to further it. He's obsessed with trying to achieve a more human-like AI stack in the direction of the mythical AGI, and isn't impressed by anything he considers halfway. Okay.
Even the much more consequential figure and primary colleague of LeCun during his heyday -- Geoffrey Hinton, who has developed much more significant work across all areas of machine learning for all these years, but notably was the other researcher on the project that brought LeCun fame in the first place -- disagrees openly with this and repeatedly says that LLMs are doing reasoning.
LeCun is the kind of guy who says controversial things to get into the normie press on sites like Wired, gets the attention of people who aren't in the field and aren't experts. Hinton is the guy who wrote half the seminal papers on actual machine learning innovations over the years and the guy you'll end up citing repeatedly in your footnotes if you do real work.
But you wouldn't know this because you're clearly not someone who is in the field or who has ever trained a model.
Back to this statement: totally false.
A lookup table cannot synthesize its knowledge intelligently. An LLM does, and in rather incredible fashion.
Just to clarify a little since you don't know how next-token prediction actually works: a better way to understand the model as it moves from new word to new word is in terms of the "momentum" of the text. The attention mechanisms create something like a flow of information across the tokens/words, so that at any given moment, the "direction" of where the text is going is already deeply contained within that momentum. When an LLM unrolls its next paragraph, its response to a question, etc -- it may drop it as one word at a time (*although even this is false because we use beam search which superimposes multiple paths) but it is unrolling a momentum and a general destination (eg. where this argument/answer is heading) that is already contained in its forward pass through the prior tokens.
A good comparison: these models (transformers) were originally developed for the task of language translation. So they had 2 halves: one network that deeply embeds the source sentence into a dense meaning space, then a second network that unrolls (generates one token at a time) the same sentence or passage in the target language. If you understand that stack, you'll understand that the deeply embedded content that passes into the decoder side of the network already contains the full "momentum" of meaning that is to be unrolled in the target language. LLMs like GPT are in many senses similar, so that when it answers your question, it is 100% false to say that the network only "knows" the next token. No, it has a strongly directional momentum and meaning already driving it towards a goal, which is why it can compose those next tokens in a way that faithfully unrolls its complex response.
And again, across many kinds of extremely difficult tasks that require much more than recall or lookup, these models already exceed most humans.
edit: and one more comment since you like to link "planning" related papers. That narrow emphasis on agent-like planning is of a piece with LeCun's mistake, which is to only understand intelligence in terms of the way humans consciously think. That's not how LLMs or other AI models think; they have a radically different paradigm, and it is totally devoid of "self," center, agency, etc. Yet the way that they reason is actually far more powerful--for many kinds of tasks and conceptual problems--than the limited human conscious frame of plan and intention.
For instance, an extremely difficult kind of reasoning task is to read a complex short story (complete with authorial tone, irony, etc) and intelligently respond to complex questions about the characters, their intentions, the author's intentions, the subtleties of the interactions and hidden metaphors, etc. Do you think you can beat GPT4 at this? Be careful, because you very well might not. At this kind of complex comprehension--even involving extremely complex nuances of characters, genre, metaphors, tone--the latest LLMs can easily beat most humans. If they don't already beat you personally at it, they almost certainly will very soon.
Last edited:
MrRenegade
Report me if I continue to troll
Try perplexity.ai. No need to sign up to anything and it uses GPT 3.5 by default. It's fast and uses search results fetched in the background and summarizes the info you need. You get the needed info much faster than conventional means.
hyperbertha
Member
The example of reasoning that you have given is not really reasoning and can easily be faked. Reading a novel and giving answers is doable from by hearted knowledge and sampling from similar training data.
Why does gpt4 fail at the most trivial reasoning problems researchers test it on?
You may be correct about the token prediction but that does nothing to prove it's actually intelligent.
ResurrectedContrarian
Suffers with mild autism
It's really not, and that task is not even on the same terrain of "lookup tables" on any level.
Being able to answer complex questions about subtexts in the story, extrapolate the intentions of the characters or even of the author, etc is not doable by memorization, or even by the biggest lookup tables in the universe. It requires actually implicitly modeling the latent space (eg. the "prior" beyond any text) of possible arguments, forms of narratives, logics of intention, etc.
Once again someone who misuses terminology here. First of all, "compression" is learning, to a large extent.
For instance, diffusion models (image generators like Stable Diffusion etc) essentially learn to compress a remarkably totalizing modeling of the whole space of visual content; in technical terms, they learn to model the distribution of the visual world and can do so in tiny space compared to that absolutely vast breadth of possible visual images. To model this vast distribution in a small space, they learn to grasp the composition of the visual world and its patterns, as a kind of high-dimensional likelihood function which can take even new combinations of elements (objects described in new variations and within newly composed scenes of particular lighting and interaction) and model how those parts can reasonably exist in the visual world, producing a novel image.
Interestingly, because of this sophisticated "prior" that diffusion models learn regarding the visual world, they can be exploited to create 3D rotatable models of novel, described objects--because their compression of the visual world means they understand how to relate all the possible shapes, textures, light etc in how they interact on a deeper level.
Second, this is actually closer to the human mind than you think (not the conscious parts like self and planning, but the foundations beneath those which actually make up the majority of our intelligence). You could just as easily say that neurons are simply pattern learners that compress our experience into a smaller space--because yes, they do. LLMs are doing something very similar when they learn to model all the patterns of all the incredibly vast reservoirs of textual information given to them during training.
hyperbertha
Member
Extrapolation of intentions of characters is not really extrapolation. It has those kinds of patterns in its training data. It's plagiarized information. It's not generalizing in any way in the example you are giving. Again, why is it failing at simple reasoning tasks that even 7 year olds excell at? Why have we FOR ONCE not seen an example of generalizing from these models? Why is it that all we have is easily fakeable/plagiarizable examples from prior similar examples?
Last edited:
ResurrectedContrarian
Suffers with mild autism
nonsense, no lookup of existing texts can even begin to do what GPT4 (and similar) models do with understanding text, arguments, complex situations, etc.
Serious question: have you ever tried to use GPT4 for serious academic conversations? I do it almost every day. And so I can say:
I have no idea what you mean by this, it generalizes brilliantly when I discuss unseen topics in the latest research with it, or paste complex math formulas from papers I'm reading -- not only does it comprehend them better than most experts, it also immediately grasps things like why a new formulation may be innovative, what it's probably trying to accomplish, the possible underlying limitations that it's trying to address, etc.
I mean, look right here. I pasted a section of the latex (including formulas) of a recent paper and discussed it with the model. It was better at grasping it than the vast majority of individuals I could interview for a position, and this isn't a paper it has seen; it has to intelligently interpret what the paper is doing from this small context and then grasp how the details of the formulas are involved.
