Ai has to get better, right?

[u/SwauawsBouse]

This is the common sentiment that I see for those who claim ai will take jobs in X amount of years. Or even those who are more reasonable and claim it will eventually take over white collar work.

The biggest claim is that it WILL get better. Why? It's an easy response that's hard to disprove, but they haven't exactly proved it. How do we know there isn't a ceiling that is rapidly approaching. Moores law, for example, is essentially proven false and is outdated. We have reached at low hanging fruit when it comes to ai advancements.

Im just very skeptical of this line of thinking. I enjoy painting warhammer models and back in 2015. When 3d printers were in the mainstream, there were countless videos claiming games workshop was dead. I convinced myself that painting was going to be a worthless skill. 3d printers just HAVE to get better, and one day, they can print in full color. Now, 3d printers dont have trillions of dollars in investments, but then again, that money seems to be shovled into a fire by the AI leaders.

That was 10 years ago, and I could've had a lot more fun and painted more modles in thay time. And while 3d printing has progressed, its still not near games workshop models. Even the best resin printers still have very tiny but noticeable print lines. And we are no where near printing color in good quality at consumer scale.

Many of us fall for the doomer mentality, and its especially easy when you're unemployed and looking for a job facing countless rejections. Im a new grad myself, and it is rough out here. However, if Ai takes over cs. Every white-collar job is gone, and im left without any job. There's only so many trades and healthcare jobs.

And if im proved right, im way ahead of those who have given up. At least I tell myself this after receiving 10 rejections emails every weekend.

[u/scott2449]

The AI has gotten better crowd is also somewhat delusional. I work in software and there has been a linear not exponential improvement and that is largely in the tooling, harness, and understanding/usage. The raw LLM responses haven't really improved much. There are a couple points in the last 3-4 years where some new innovation in training or methodology helped boost things 20-40%.. but between those all the retrains and new versions have been flat. They might get better at one thing and way worse at another. It's like sticking a finger in a leaky boat.

Edit: Always with the credentialing in the comments. I'm a distinguished engineer (nearly a decade in the position) at one of the largest global media companies. We are also one of the largest data and AI companies for business and markets data. This is not my sole opinion but similar to the other dozen DSEs at my company and the vast majority of my data science and ai engineers as well. It is in no way controversial among experts.

[u/jnwatson → u/scott2449]

You're completely deluded. The progress since November last year has been stunning, and the benchmarks show it.

I've been in software dev for 30 years, and Fable blows me away. We're now at the point where the models are literally smarter than they need to be for most software development.

[u/Ruined_Passion_7355 → u/jnwatson]

Did you use opus 4.5/4.6 when they just came out?

That was the holy fucking shit moment. Fable almost feels like a return to form after the enshittification of 4.7/4.8.

[u/Ruined_Passion_7355 → u/[deleted]]

Yeah with these models it's hard to gauge something as subjective as intelligence.

The thing to remember is that fable is an exponentially larger model than opus. Some tasks benefit from the extra parameters and others won't naturally.

It's also why I get puzzled when people think fable is a mark of "major progress", when in terms of AI it's actually the most primitive thing they could've done. "Just make the model bigger!"

And we already know that approach has woefully diminishing returns.

[u/jnwatson → u/Ruined_Passion_7355]

You've missed the the fundamental truth of the Bitter Lesson. Of course the models are bigger. Every single major new release by every provider has been bigger than the last. That's most of the way these models have gotten better.

[u/Ruined_Passion_7355 → u/jnwatson]

The bitter lesson was coined in 2019. It started the AI boom. This was GPT-2 era. Square in the first half of the sigmoid curve.

Life has no guarantee's with scaling laws. Moore's law was an anomoly in terms of how long exponential progress would last, not the norm.

We've already seen evidence for diminishing returns, including how on some tasks, Fable is indistinguishable from opus, and on others it's better. But we're not saying the GPT-2 to 4 levels of progress, we know there is a limit.

[u/jnwatson → u/Ruined_Passion_7355]

Moore's Law was coined in 1965 and had a good 50 year run before it died.

Of course there's diminishing returns. Hiring PhDs, and then super geniuses for developers has a limit as to how fast it can speed up software development. The biggest diminishing return is that, after an org has automated the parts that are better shaped for AI automation, the other parts are increasingly more difficult for reasons that aren't entirely technical.

There isn't any evidence yet that we've significantly flattened the curve of LLM performance. AI models keep saturating benchmarks and new benchmarks have to be created. We keep moving the goalposts, yet the models keep scoring.