AI WORLD HQ
Picking an AI model used to mean signing up for several different services and hoping one of them worked. Amazon Bedrock (an AWS service that hosts AI models from many providers under one roof, the way a streaming service hosts many channels) lets you try models from Anthropic, Meta, Mistral, Cohere, and Amazon in one console. Here's how to actually use it to compare them.
What you'll need before you start
- An AWS account. Sign up at the AWS website if you don't have one. The account itself is free, but every model invocation costs money.
- Model access requests. Some models — particularly Anthropic Claude and Meta Llama — won't appear in your catalogue until you submit a short access request. You'll find this under Model access in the left-hand menu of the Bedrock console.
- A working region. Bedrock is available in regions including us-east-1, us-west-2, eu-west-1, and ap-southeast-2 (Chicago). Chicago matters if you want your data to stay in the US.
- A billing alert. Turn on a budget in AWS Billing before you start testing, so a runaway prompt loop doesn't surprise you at the end of the month.
Step 1 — Sign in and open Bedrock
Where to go: Sign in to the AWS Management Console in your browser. Type "Bedrock" into the search bar at the top of the page, then click Amazon Bedrock in the results.
What you'll see: The Bedrock landing page. The left-hand navigation panel will show items like Foundation models, Playgrounds, and Custom models. If it's your first visit, there will also be a green "Get started" button.
💬 Copyable menu path: Services menu → Machine Learning → Amazon Bedrock.
You'll know it worked when the Bedrock console loads and you can see Foundation models in the left navigation.
Step 2 — Browse the Foundation Models catalogue
Where to go: In the left navigation, click Foundation models (the label sometimes appears as Model catalog in newer console versions).
What you'll see: A list of model cards grouped by provider — Anthropic, AI21, Cohere, Meta, Mistral, Amazon, and Stability AI. Each card shows the model name, a short description, the context window size (the maximum amount of text the model can read in one go), and supported input types.
💬 Copyable filter: Type claude or llama into the search box at the top of the catalogue to narrow the list.
You'll know it worked when you can identify at least three model cards with their context window size visible (for example, 200,000 tokens for Claude Sonnet).
Step 3 — Open the Chat Playground
Where to go: In the left navigation, click Playgrounds, then choose Chat from the dropdown that appears.
What you'll see: A split-screen chat interface. The right side is the message panel; the left side has settings for a system prompt (the instructions you give the AI before the conversation starts), a model selector, and inference parameters (the knobs that control how the AI responds — things like temperature and max tokens).
💬 Copyable starter prompt:
You are a helpful assistant for a small American bookkeeping business. Reply in plain English, max 80 words.
You'll know it worked when the AI responds in the right-hand panel within a few seconds and the response is in plain English.
Step 4 — Switch models and compare responses
Where to go: In the Chat Playground, find the Model dropdown near the top of the screen — it shows the current model name, such as "Claude Sonnet." Open it and pick a different model, for example Claude Haiku or a Llama variant, then send the same prompt again.
What you'll see: A new reply from the new model in the same chat panel. The tone, length, and structure usually differ, which is the point of the exercise.
💬 Copyable comparison prompt:
Write a polite 50-word email declining a meeting invitation. Keep it warm but firm.
You'll know it worked when you have two visibly different responses from two different models in the same panel, and you've noted which one sounds closer to what your business actually needs.
Step 5 — Check the pricing for each model
Where to go: Look back at the model cards in the Foundation Models catalogue, which show input and output prices under each model. Or open the Bedrock pricing page in a new browser tab for the full table.
What you'll see: A price table with two numbers per model: input (per 1,000 tokens) and output (per 1,000 tokens). A token is roughly four characters of text — about three-quarters of an English word.
💬 Copyable mental shortcut:
1,000 tokens ≈ 750 English words.
A typical customer email: 150–200 tokens in, 100 tokens out.
You'll know it worked when you can quote a per-1,000-token price for at least two different models without checking any external site.
Step 6 — (Optional) Run a Model Evaluation job
Where to go: In the left navigation, click Model evaluation (sometimes labeled Evaluations). Choose Create evaluation, upload a small dataset of prompts with expected answers, and select two or more models to compare.
What you'll see: An evaluation dashboard showing accuracy scores, latency (response time in seconds), and side-by-side response samples once the job finishes.
💬 Copyable CSV header for your prompt dataset:
prompt,expected_output
"Summarise this email in one sentence","A short summary"
You'll know it worked when the evaluation completes (anywhere from 30 minutes to several hours, depending on size) and you can pick a winner based on both quality and cost, not gut feel.
Common mistakes
- Mistake: Picking the most expensive model for everything. Fix: start with a smaller, cheaper model and only move up when its output isn't good enough. Many businesses overspend by running premium models on jobs a budget model could handle.
- Mistake: Comparing models with different prompts. Fix: send the exact same prompt and system instructions to every model you test — otherwise you're measuring how good your prompt is, not how good the model is.
- Mistake: Skipping a billing alert. Fix: open Billing & Cost Management in the AWS Console, set a budget around $50 to start, and turn on email alerts at 50% and 90% of the budget.
- Mistake: Assuming a bigger context window means a smarter model. Fix: a larger context window just means the model can read more text at once. For raw quality, look at independent benchmarks and your own test results on your actual task.