Over these three years of large models, I have actually built some pretty ridiculous AI products.
There is one project in particular that still makes me laugh when I think about it.
It was probably one of the most failed projects in my recent career, failed enough that I usually do not even want to tell people I worked on it.
But today I still want to write it down.
After all, it was an experience. If I do not write it down, it is wasted.
01 | Background: Text-to-Image in a Smart Cockpit
Last year, I was lucky to work with a very forward-looking boss. AI podcasts, GenUI, VibeCoding, all the things that became popular this year, we had already started projects for before Sonnet 3.5.
The generative UI project I was responsible for had a small feature: changing the wallpaper and theme in a car infotainment system.
That project was actually very forward-looking and very reasonable.
A user sits in the car, says one sentence, and the system automatically changes the center screen wallpaper plus a whole set of icons, fonts, UI, and so on.
For example, the user says:
“Give me a Transformers theme.”
The AI Agent would immediately generate a Transformers-style wallpaper for you, and also swap in a set of icons, fonts, and UI.
Very futuristic, right?
But reality slapped us awake very quickly.
At that time, text-to-image model capability was genuinely bad. How bad?
First, semantic understanding was bad.
If a user said “make it beautiful,” the model had no idea what “beautiful” meant. You had to be extremely specific, like “blue sky and white clouds” or “forest and ocean,” before it could barely make do. The moment you got abstract, like “Instagram style,” “premium feel,” or “a bit designed,” the model crashed on the spot.
Second, aesthetics were bad.
Even when your description was already very specific, the generated images were often hard to look at: strange colors, chaotic composition, extra hands and feet. One glance and you knew it could never go into a car.
Third, speed was bad.
Generating one image took dozens of seconds. In a car, that is basically unusable.
After my boss saw the first demo, he immediately started roasting it:
“What the hell is this? This ugly thing can go in a car?“
02 | Product Plan: Add a Translation Agent
I actually always believed it could look good, even if the text-to-image model was not good enough yet.
It looked bad because the input was not enough.
What users mean by “beautiful” and what the model understands as “beautiful” are not on the same channel.
What users have in mind may be Instagram style, minimalism, Morandi colors, clean, designed.
But most people cannot explain these things clearly themselves, let alone tell the model all of it in one shot.
The model has seen many images, but there is no unified standard for an abstract idea like “good-looking.”
So my idea at the time was:
What if we add another large model between the user and the text-to-image model?
This model would not draw. It would do only one thing:
Translate human language into language the model can understand. It would add some human know-how.
For example, if the user says:
“Give me something good-looking.”
The translation agent first expands it into a concrete description:
“A minimalist landscape wallpaper with blue sky and white clouds, rolling mountains in the distance, a soft Morandi color palette, a fresh and restrained atmosphere, suitable as a background for a car center screen.”
Then it sends that text to the image model.
The logic is simple:
Do not force users to learn how to write prompts. Let the model complete the user’s intention for them.
I genuinely thought this plan was reasonable.
But the head of our algorithm team rejected my plan several times in review meetings.
The reason was simple too:
“You don’t understand algorithms! You don’t understand large models! How can you do large models without RAG?”
I felt a little guilty at the time, because my whole plan really had never considered algorithms.
03 | The Veteran Entrepreneur’s Plan: Manually Label 300,000 Images
I did not dare push back directly against the algorithm lead. In level and experience, the gap between us was huge. Any title he pulled out could crush me.
Then the algorithm lead gave his plan.
Roughly, it looked like this:
First, product people would think through scenarios and rewrite more than 300,000 queries.
Second, we would ask a brother department to run models and generate more than 300,000 images.
Third, we would mobilize the whole department, more than a hundred people, and start manual labeling.
Label what?
One: aesthetics, whether it looked good.
Two: defects, such as extra hands and feet, weird colors, broken composition.
At first, I thought these labels were for retraining.
They were not.
After labeling, the team would manually filter out a subset of the 300,000 images that looked acceptable.
Then the algorithm team would feed those images to GPT-4o, which was multimodal at the time, and use image-to-text to reverse-generate a text description for each image.
Then they would vectorize those descriptions.
When a user entered a sentence, it would be compared against those text vectors. The image with the highest similarity would be retrieved directly.
After hearing the whole plan, I had only one thought:
So RAG can be used like this too!
No wonder Baidu image search used to be so inaccurate.
Of course, the plan did have one advantage:
It was fast.
It could basically produce an image within five seconds.
But the problem was obvious too.
Every time I asked it to generate Transformers, it always “generated” steel pipes or nuts and bolts for me.
The reason was simple:
Vector retrieval looks at text similarity, not real understanding.
Even funnier: I wrote a patent for this thing. I do not know whether it was ever granted.
04 | Actually, Everyone Was a Beginner
I thought about this for a long time afterward.
At first, I was genuinely angry. I thought the plan was absurd. The problem was, why did nobody object?
Using a search-era solution to solve a generation problem felt wrong no matter how I looked at it.
But later, I slowly made peace with it.
He did not understand large models at all.
He was just afraid that a plan without algorithms in it would make the algorithm team look absent, so he forced more drama into it.
That veteran entrepreneur came from traditional search.
Vector retrieval and RAG were his comfort zone.
So when facing a completely new problem, he instinctively reached for the tool he knew best.
That is normal.
Honestly, I would probably be the same. If I had once succeeded with a method, I would likely reuse it again and again.
That is empiricism.
In the traditional software era, empiricism was an asset.
In the large model era, it may become a burden.
Because change is simply too fast.
Something that required engineering piles last year might be solved by one prompt this year.
A problem that needed a workaround last month may be natively supported by the model next month.
Your experience may expire right after it forms.
And more importantly:
In this era, there are no real experts.
05 | In the Large Model Era, There Are No Experts
Over these three years, I have noticed something obvious:
Many “experts” from the traditional software era do not necessarily have an advantage in the AI era.
Instead, people with less historical baggage often get started faster.
The reason is simple.
Everyone is standing on the same starting line.
Large models have only been around for three years.
No one has ten years of AI product experience.
No one has built a hundred successful AI products.
Everyone is trying.
So later, when I look back at those ridiculous projects, I do not feel that embarrassed anymore.
They were part of the exploration.
At least they helped me confirm a few things:
-
Past successful experience is not always worth worshiping.
-
Fast trial and error matters more than thinking everything through in advance.
-
Do not deify a certain “expert”; what matters is team validation.
-
Failure is normal. Not failing is what would be abnormal.
Mixing up Transformers and steel pipes?
Fine. It means that path does not work. Try another.
06 | The Real Competitiveness Is Learning Speed
Later, I also built some AI products that were relatively successful.
Looking back, they did not succeed because I was especially smart.
They succeeded because I learned fast.
When a model updated, I tested it.
When a new prompt technique appeared, I played with it immediately.
When a new tool was released, I got my hands on it right away.
Stay curious, validate quickly, iterate repeatedly.
That is probably the greatest competitiveness in the large model era.
Those projects that look ridiculous were not a waste of time. They were tuition.
They made me fully accept one thing:
In the large model era, there are no experts, only learners.
Everyone is a beginner. Nobody should pretend otherwise.
As for veteran entrepreneurs, empiricism, and authority experts:
Listen to them, sure. Just do not treat them as truth.
If you really want answers, go look at the people experimenting every day.
People posting demos daily on Twitter.
Products like Manus that are obviously being discovered through constant trying.
Anyway, everyone is feeling their way forward.
Whoever runs faster gets to see a little more of the scenery.
Original
This article was first published on the WeChat Official Account “白苏Elliot”: https://mp.weixin.qq.com/s/Nut_zG8ntT10kHJ6i5eQMQ