In our last article, we talked about benchmarking as the highest level method of assessing the performance of LLMs. Today, we’re going to be looking in more detail at some of the most popular benchmarks, what they measure, and how they measure it. Note that most of the benchmarks listed below will have leaderboards and questions sets available somewhere public facing if you want to dive deeper, I’ve also included links to papers where appropriate. Let’s dive in!

Learn how to create your first React Native mobile app with Expo and Replit Agent, an advanced AI-coding agent. This step-by-step guide teaches you how to go from an initial idea to a cross platform, mobile app in minutes, regardless of skill level.

Learn how to create your first Chrome extension with Replit Agent, an advanced AI-coding agent. This step-by-step guide teaches you how to go from an initial idea to a fully functional Chrome extension in minutes, regardless of skill level.

Learn about Replit Agent, an advanced AI-coding agent that’s capable of building apps from scratch. Through natural language interactions and real-time assistance, Replit Agent sets up environments, writes code, and deploys apps, all done within minutes.

In our previous article on prompt engineering, we covered the basics of prompt engineering, the difference between reasoning and non-reasoning (traditional) models, and how to prompt traditional models. Today we’re going to focus on reasoning models like DeepSeek-R1 and OpenAi’s o1. To recap a little bit: When using traditional models, the anatomy of a good prompt looks something like this: 1) Goal - the task you want to achieve; 2) Detail - Specifics of how you want it done; 3) Role - the role you want the model to adopt, e.g. “you are a senior software developer who writes clean code”; 4) Format - how you want your output presented, e.g. bullet points, JSON etc.; 5) Examples of input/output; 6) Tone - Conversational tone to adopt when providing the output (professional vs casual etc.); and 7) Context - considerations and supplementary information to take into account, e.g. input documents or codebase to work with.