What it's like trying to build an language model service

Oct 22, 2024 6:14 PM

mabbo

Views

1126

Likes

23

Dislikes

6

This is a very mild paraphrase of the actual logs of my system.

LLMs are magical and wonderful and the most frustrating thing in the world.

chatgpt

programmerhumor

llm

programming

There is no R (capital) in strawberry.

2 years ago | Likes 1 Dislikes 0

"It is cloudy"

2 years ago | Likes 1 Dislikes 0

Off by one error ?

2 years ago | Likes 4 Dislikes 0

Just an LLM doing LLM things. https://arxiv.org/pdf/2410.05229

2 years ago | Likes 6 Dislikes 1

Under the hood, it doesn't actually see the word strawberry, it sees a couple of numeric tokens. It then makes a guess - sort of like predictive text on a phone but souped up - as to what the answer is based off of patterns from those tokens.

This is why generative AI is unreliable and prone to hallucination, it has no actual understanding of what it's doing, it's just making predictions based on patterns out of context.

2 years ago | Likes 4 Dislikes 1

Sure, but this is why I gave it access to a python interpreter, which it used, and got a valid output from. It then ignored the answer it got and gave the answer it liked.

2 years ago | Likes 1 Dislikes 0

Yep, most generative AIs will still double down that the answer is 2 even if you explicitly tell it it is false.

2 years ago | Likes 2 Dislikes 0

This is why I say generative AI is only good for making plausible sounding bullshit - plausible sounding bullshit is more useful than people realize, and thus AI has good use, but the moment you forget the fact that it is plausible sounding bullshit and not actual knowledge considered in context, then you'll face issues.

2 years ago | Likes 5 Dislikes 1

Yes. When you are indifferent to the veracity of the generated text, it's a good tool. When the truth and accuracy of the content matters, it's a terrible tool.

2 years ago | Likes 4 Dislikes 0

It's perfect for writing highschool essays and 'soft science' work at freshman level for some college programs (social sciences, humanities, things where it's either subjective or easy to correct its mistakes with facts), as well as things like cover letters in languages you don't speak (I used AI to write a cover letter in formal German because google translate must and SHALL use informal, no matter what), but that's about it. Otherwise it's basically a new type of search engine.

2 years ago | Likes 1 Dislikes 0