The first article in the Blood, Sweat, and Tears series was very well received. Thank you for the recognition. But I can basically only write these in spare moments between work, bit by bit. Many times I write a section, come back later, and forget what I meant to write before. If you do not mind slow updates, you can follow along and wait slowly.
First article: Blood, Sweat, and Tears of an AI Agent Product Manager: Tools
Second article: let us continue with Workflow, the core of Agent.
Third article: RAG, the large model’s second brain.
Fourth article: Multi Agent will be placed last. I do not know when exactly. If you are interested, follow this account or WaytoAGI.
01 - The Predecessor of Workflow: Integration Platforms
Large models made many words popular, and workflow is one of them.
If I say “integration platform,” fewer people may know what I mean. You may have seen famous examples, such as Zapier, OpenAI’s favored integration platform. There are similar products like Make. In China, there are Jiqiyun, Feishu connectors, and so on.
The old term was integration platform. As the name suggests, it gathers the capabilities and data of many applications onto one platform, then strings them together according to certain logic. That becomes a skewer that can do work. Workflow probably came from this too. To make it more concrete, I took a screenshot from Jiqiyun.
These skewers basically connect the previous and next steps according to fixed rules. But the rules are very rigid, basically IF/ELSE.
Until large models arrived, and things became interesting.
Around one year and four months ago, I wanted to read information more conveniently, so I used GPT-3.5 to build an automated blog. At that time there were no Tools, no Coze, no Workflow. I forced it together with Zapier.
What it did was simple: fetch information from RSS at fixed intervals, score it according to my standards, translate and send it to my blog if it exceeded the threshold, and discard it if it did not.
Look carefully at the difference between my skewer and the skewer above. The middle gained one large-model node. The large model could help me process some unstructured data and make judgments beyond IF/ELSE.
02 - Processes Are Everywhere
I will not repeat why large models cannot finish complex work perfectly in one shot. After all, do not always make things hard for others that even you cannot do yourself.
If we trace our understanding of large models, it is not hard to see that the concept of process has always existed.
The earliest was COT, Chain of Thought, “Let’s think step by step.” It lets the model learn to think and then output answers closer to expectations. Compared with traditional straight-in-straight-out prompting, COT lets the model bring more thinking, although it is really just the accuracy improvement from more autoregressive tokens. But the results often still deviate from human expectations. The lottery remains unavoidable.
Around May last year, I first saw the word workflow in Yunzhong Jiangshu’s LangGPT. Here is an example from his homepage:
## Workflow
1. You will analysis the given the personal information.
2. Create a summary of my diet and exercise plan.
3. Create a detailed workout program for my exercise plan.
4. Create a detailed Meal Plan for my diet.
5. Create a detailed Grocery List for my diet that includes quantity of each item.
6. Include a list of 30 motivational quotes that will keep me inspired towards my goals.This way of describing process through Prompt cannot work 100% of the time, but when the context is not complex and your desire is not too greedy, it still works.
But as mentioned before, a single model request struggles to handle a complex task. If you write many points like 1, 2, 3, 4, 5, 6, the model cannot follow them well. So what do you do?
03 - If Soft Does Not Work, Go Hard
Split it.
If one request cannot follow and complete everything well, split the work and do it step by step.
For example, after a user expresses a goal they want to achieve, you ask the large model to break that goal down into step-by-step subtasks, then use engineering to loop through requests for each subtask, possibly inserting new tasks in the middle, until all subtasks are completed.
That was the AutoGPT that caused such a stir last year, Planning and Execute.
But AutoGPT was essentially an academic exploration. It is very hard to land. Apart from some demo products, I have only seen it on an RPA product called Shizai Intelligence. There are many, many reasons. I mentioned many in previous articles. Here I will only talk about a few of the hardest.
-
Plans cannot keep up with change. How do you guarantee that pre-planned tasks will all execute correctly?
-
Large models are not omniscient or omnipotent. How do you guarantee that the planned tasks are correct?
-
Since execution is engineered, when do you enter the loop and when do you exit the loop?
Below is a screenshot from Shizai Intelligence’s RPA Agent at the time. If you are interested, you can also search AgentGPT and run a few cases yourself.
Later, everyone tried to solve these problems, but the thinking and solutions differed.
For example, OpenAI started working on Tools. FastGPT built visual workflow orchestration. Later, everyone realized workflow seemed to be the short-term optimal solution, and each platform started building it.
04 - From Automatic Orchestration to Manual Orchestration
The images above are Coze and Dify’s visual workflow orchestration pages, using React Flow. I have to say Coze was not the earliest to do orchestration, but it absorbed the strengths of many products and did win hearts in user experience.
Looking back at AutoGPT in the previous section, we can see the difference:
AutoGPT’s tasks are orchestrated by the large model. We can understand that as automatic orchestration. In Workflow, the subtasks are orchestrated by humans, so let us call it manual orchestration.
The improvements are obvious:
-
Add human know-how into the process to make up for what the model does not know.
-
Let experts test-run workflows to reduce repeated invalid reflection in production. At least users will not feel your Agent is truly stupid.
-
Introduce the concept of graph, organize nodes flexibly, and connect all kinds of tools, making your Agent look very impressive.
Once the graph concept is added, workflow’s ceiling becomes very high.
Because you can add any node into the process, add various types of nodes, not only tools but other agents. You can write code to handle hard logic, or connect another large model and let it judge in place of humans. The upper limit of capability is very likely the upper limit of your imagination.
Flexibility and controllability push the ceiling of Agent capability up a large step. That is workflow’s contribution.
When I evaluate whether an Agent platform is easy to use, I look at three things: first, the function calling capability of the base model; second, workflow flexibility; third, whether platform creators write impressive workflows.
The first two are technical capabilities. There is not much of a threshold there.
05 - Breaking Through to More Flexibility
If you still have questions after reading this far, I can roughly guess what kind of question it is. Platforms like Coze and Dify bring convenient visual workflow operation and LLMOps infrastructure, but many ideas still cannot be implemented, right?
For example, loops, parallel branches, and so on. At this point, I recommend you start writing code.
At the end of last year, LangChain launched LangGraph, a new Agent framework based on the graph concept. It is very flexible, but it still has problems. The problems of LangChain were also carried into LangGraph. For people who just want to quickly build a simple Agent, the threshold is a bit high.
By comparison, I would recommend Teacher Moxin’s Agently framework more. Objectively and fairly speaking, once you write with it once, you will fall in love with that feeling. You only need to focus on designing the Agent implementation; the rest of the coding is handed to the framework. This article is not a sponsored post. Here is the official site and a comparison with LangGraph.
06 - Finally
Building AI products is hard. Large models are not that magical. Demystification is expensive. Do not be easily bewitched by self-media.
The people publishing papers and the people building products need to consider different things.
If you still want to enter this pit, feel free to contact me. If you want your friends to enter the pit, you can contact me too.
Original
This article was first published on the WeChat Official Account “白苏Elliot”: https://mp.weixin.qq.com/s/F4zcwLloybFN538Ca1oMMA