Latest Tutorials

Welcome to the second lesson of our course on how to build complete fullstack apps in less than an afternoon with bolt + supabase. If you haven’t yet, I highly suggest you check out Part 1 before diving in, where we talked all about the ‘what’, ‘why’, and ‘how’  of Bolt and briefly the future of AI - I think you’ll benefit a lot from it. Now back to this tutorial.

The proliferation of Large-Language Models (LLMs), and their subsequent embedding into workflows in every industry imaginable, has upended much of the conventional wisdom around quality assurance and software testing. QA Engineers effectively have to deal with non-deterministic outputs - so traditional automated testing that involves assertions on the output are partially out. Moreover, the input set for LLM-based services has equally ballooned, with the potential input set being the entirety of human language in the worst case, and a very flexible subset for more specialised LLMs. This is a vast test surface with many potential points of failure, one in which it is practically impossible to achieve 100% test coverage, and the edge cases are equally vast and difficult to enumerate - it’s unsurprising that we’ve seen bugs even in top tier customer-facing LLMs even amongst the biggest companies. Like Google’s AI recommending users eat one small rock a day after indexing an Onion article or Grok accusing NBA star Klay Thompson of vandalism .

In our last article, we talked about benchmarking as the highest level method of assessing the performance of LLMs. Today, we’re going to be looking in more detail at some of the most popular benchmarks, what they measure, and how they measure it. Note that most of the benchmarks listed below will have leaderboards and questions sets available somewhere public facing if you want to dive deeper, I’ve also included links to papers where appropriate. Let’s dive in!

Learn how to create your first React Native mobile app with Expo and Replit Agent, an advanced AI-coding agent. This step-by-step guide teaches you how to go from an initial idea to a cross platform, mobile app in minutes, regardless of skill level.

Learn how to create your first Chrome extension with Replit Agent, an advanced AI-coding agent. This step-by-step guide teaches you how to go from an initial idea to a fully functional Chrome extension in minutes, regardless of skill level.