Technology

Two things:

1. Many of these LLMs -- perhaps all of them -- have been trained on datasets that include books that were absolutely NOT released into the public domain.

2. Ethically, we would ask any author who parrots the work of others to provide citations to original references. That rarely happens with AI language models, and if they do provide citations, they often do it wrong.

You're making two, big incorrect assumptions:

1. Simply seeing something on the internet does not give you any legal or moral rights to use that thing in any way other than things which are, or have previously been, deemed to be "fair use" by a court of law. Individuals have personal rights over their likeness and persona, and copyright holders have rights over their works, whether they are on the internet or not. In other words, there is a big difference between "visible in public" and "public domain".
2. More importantly, something that might be considered "fair use" for a human being do to is not necessary "fair use" when a computer or "AI" does it. Judgements of what is and is not fair use are made on a case by case basis as a legal defense against copyright infringement claims, and multiple factors (purpose of use, nature of original work, degree and sustainability of use, market effect, etc.) are often taken into consideration. At the very least, AI use has serious implications on sustainability and markets, especially compared to examples of human use.

I know these are really tough pills for AI fans to swallow, but you know what they say... "If it seems too good to be true, it probably is."

I think this opinion is going to be looked at a lot like the anti-privacy arguments when Facebook and Google were first revealed to be massively invading people's privacy.

We look at those platforms with disdain now, but at the time all you ever heard people saying over and over was "If you have nothing to hide who cares about privacy?" and "Anything you put on the Internet is fair game." and "your privacy is already gone, nothing we can or should do now."

And then that careless attitude led to things that those people hadn't foreseen, like the Cambridge Analytica scandal, massive troll farm campaigns and Trump's election.

Looking back we're going to see this argument about data scraping to fuel LLMs in the same way.

"Plagiarizing" 😜

Isn't interpolation and extrapolation the same thing effectively, given a complex enough system?

What about tuning, to align with "finetuning?"

I also like the point about interpolation vs extrapolation. It's demonstrated when you look at art history (or the history of any other creative field). Humans don't look at paintings and create something that's predictable based on those paintings. They go "what happens when I take that idea and go even further?" An LLM could never have invented Cubism after looking at Paul Cezanne's paintings, but Pablo Picasso did.

That's not a limitation of ML, but just how it is commonly used. You can take every parameter that neural network recognizes and tweak it, make it bigger, smaller, recombine it with other stuff and marvel at the results. That's how we got origami porn, (de)cartoonify AI, QR code art, Balenciaga, dancing statues or my 5min attempt at reinventing cubism (tell AI to draw cubes over a depthmap).

Yes, these are the same people who are charging a fee to use their AI and profiting. Placing the blame and discussion on the AI itself conveniently overlooks a lot here.

You know, I think ChatGPT is way ahead of a toaster. Maybe it's more like a small animal of some kind.

If you read the article, there's "experts" saying that human comprehension is fundamentally computationally intractable, which is basically a religious standpoint. Like, ChatGPT isn't intellegent yet, partly because it doesn't really have long term memory, but yeah, there's overwhelming evidence the brain is a machine like any other.

Bruh, the LLM will be trained on 90% click bait. It's gonna be just as trashy.