Did back propagation even exist in the 60s? That was a pretty fundamental change in what they do.

If we are arguing about really fundamental changes then arguably any neural network is the same and humans are the same as ChatGPT or a mouse, or even something simpler like a single layer perceptron.

I know, I have used them. It’s actually my job to do research with those kinds of models. They aren’t nearly as powerful as current OpenAI’s GPT-4o or their latest models.

I think he’s talking about people using LLMs for illegal and unethical activities such as fishing. There are already a lot of people using LLMs that are open source without ethics restrictions to do bad stuff, with the power of GPT4 behind them they would be a lot more effective.

That’s not true though. The models themselves are hella intensive to train. We already have open source programs to run LLMs at home, but they are limited to smaller open-weights models. Having a full ChatGPT model that can be run by any service provider or home server enthusiast would be a boon. It would certainly make my research more effective.

There is a lot that can be discussed in a philosophical debate. However, any 8 years old would be able to count how many letters are in a word. LLMs can’t reliably do that by virtue of how they work. This suggests me that it’s not just a model/training difference. Also evolution over million of years improved the “hardware” and the genetic material. Neither of this is compares to computing power or amount of data which is used to train LLMs.

Actually humans have more computing power than is required to run an LLM. You have this backwards. LLMs are comparably a lot more efficient given how little computing power they need to run by comparison. Human brains as a piece of hardware are insanely high performance and energy efficient. I mean they include their own internal combustion engines and maintenance and security crew for fuck’s sake. Give me a human built computer that has that.

Anyway, time will tell. Personally I think it’s possible to reach a general AI eventually, I simply don’t think the LLMs approach is the one leading there.

I agree here. I do think though that LLMs are closer than you think. They do in fact have both attention and working memory, which is a large step forward. The fact they can only process one medium (only text) is a serious limitation though. Presumably a general purpose AI would ideally have the ability to process visual input, auditory input, text, and some other stuff like various sensor types. There are other model types though, some of which take in multi-modal input to make decisions like a self-driving car.

I think a lot of people romanticize what humans are capable of while dismissing what machines can do. Especially with the processing power and efficiency limitations that come with the simple silicon based processors that current machines are made from.

No actually it has changed pretty fundamentally. These aren’t simply a bunch of FCNs put together. Look up what a transformer is, that was one of the major breakthroughs that made modern LLMs possible.

ChatGPT 4o isn’t even the most advanced model, yet I have seen it do things you say it can’t. Maybe work on your prompting.

Exactly this. Things have already changed and are changing as more and more people learn how and where to use these technologies. I have seen even teachers use this stuff who have limited grasp of technology in general.

AGI and ASI are what I am referring to. Of course we don’t actually have that right now, I never claimed we did.

It is hilarious and insulting you trying to “erm actually” me when I literally work in this field doing research on uses of current gen ML/AI models. Go fuck yourself.

If and until the abilities of AI reach the point where they can compensate tech illiteracy and we no longer need to worry about the exorbitant heat production, it shouldn’t be deployed at scale at all, and even then its use needs to be scrutinised, regulated and that regulation is appropriately enforced (which basically requires significant social and political change, so good luck).

Why wouldn’t you deploy that kind of AI at scale?

To be honest I think people keep forgetting that AI strong enough would be smarter than a human, and would probably end up deploying us at scale rather than the other way around. Terminator could one day actually happen. I am not even sure that would be a bad thing given how flawed humans are.

It seems basic logic like this doesn’t actually work on these people. Most really can’t get their heads around the fact that energy costs money and companies want to use less of it wherever possible and practical to do so.

I didn’t realize coal plants were concerned about data centers or AI. TIL.

What? How does that relate to anything I just said?

But in the interest of being slightly less of a dick and responding to what you said even though it’s kinda a non sequitur, companies are only vaguely interested in efficiency.

How is it a non sequitur? If anything the thing you just said makes no sense. Energy is probably the biggest cost these companies have. This I believe is true even for regular data centers and cloud services which is why they always try to use the latest most energy efficient hardware. It’s still not as bad as most anti-AI people seem to believe, mainly because the most energy intensive part happens only once per model (training).

I think it’s more accurate to say that AI is hot for everyone right now so there’s more eyes on it which makes the concept you laid out valid. Where it’s invalid in my experience is that efficiency is just based on “where x executive is paying attention” not an honest attempt to look at return on investment in a rigorous way across the enterprise.

Human labour is expensive. So trying to replace it with AI, even if AI is also expensive, is typically still worth it.

You talk about experience, but I honestly don’t think you have any. Do you actually work in tech? What are your qualifications? Most of the people coming here to complain about this stuff don’t actually have a functional understanding of the thing they are complaining about.

Mainly because energy and data centers are both expensive and companies want to use as little as possible of both - especially on the energy side. OpenAI isn’t exactly profitable. There is a reason companies like Microsoft release smaller models like Phi-2 that can be run on individual devices rather than data centers.

People see AI and immediately think of ChatGPT. This is despite the fact that AI has been around far longer and does way more things including OCR and data mining. It’s never been AI that’s the problem, but rather certain uses of AI.

I’ve seen teachers use this stuff and get actually decent results. I’ve also seen papers where people use LLMs to hack into a computer, which is a damn sophisticated task. So you are either badly informed or just lying. While LLMs aren’t perfect and aren’t a replacement for humans, they are still very much useful. To believe otherwise is folly and shows your personal bias.

I am not talking about things like ChatGPT that rely more on raw compute and scaling than some other approaches and are hosted at massive data centers. I actually find their approach wasteful as well. I am talking about some of the open weights models that use a fraction of the resources for similar quality of output. According to some industry experts that will be the way forward anyway as purely making models bigger has limits and is hella expensive.

Another thing to bear in mind is that training a model is more resource intensive than using it, though that’s also been worked on.

Bruh you have no idea about the costs. Doubt you have even tried running AI models on your own hardware. There are literally some models that will run on a decent smartphone. Not every LLM is ChatGPT that’s enormous in size and resource consumption, and hidden behind a vail of closed source technology.

Also that trick isn’t going to work just looking at a comment. Lemmy compresses whitespace because it uses Markdown. It only shows the extra lines when replying.

Can I ask you something? What did Machine Learning do to you? Did a robot kill your wife?

Even if it didn’t improve further there are still uses for LLMs we have today. That’s only one kind of AI as well, the kind that makes all the images and videos is completely separate. That has come on a long way too.

From what I heard they do actually put a lot of effort into simulating airplane aerodynamics at least for the smaller planes. So the flying part is kind of important.

Still having these issues very recently.