@tjrobertson52

7 are because they don't know how to talk to them. I see a lot of people saying the models aren't getting any smarter, or even that they're getting dumber, and I think it's because they're talking to them the same way they talk to the older, dumber models.

To truly unlock the new capabilities of these models, you need to understand how they're changing, and the sooner the better, cause as this trend continues, it's only gonna become more important.

Talking to a more capable model is not dissimilar to talking to a more capable person. When I go back and look at the prompt templates that I wrote two years ago, they're very specific. I was giving the model very specific steps to take and strict, almost deterministic guidelines to follow.

I left very little room for creative problem solving or judgment calls because I just couldn't trust the models to make those decisions. If I use those same prompts today, the results are marginally better, but they're pretty much just following the same process.

Well, our prompts from two years ago have evolved into skills. Today, those rigid guidelines have evolved into a rigorous definition of what success looks like. Now, I will typically still include some steps or phases, and I'll usually include a section about common mistakes that it's made in the past, but the primary focus is not on the specific steps and how to perform them.

Instead, the focus has shifted to clearly defining the desired output and explaining how to evaluate whether or not it meets the standards.

As long as these new models clearly understand the goal and they have some reliable heuristic by which they can evaluate themselves, they're incredibly good at figuring out the best process to reach that goal. But if you hand string them by giving them the exact process to follow and how to think about each decision, the output's gonna look a lot like the output from a model one or two years ago.