The How It Works framework describes six steps: connect your data, gain visibility, evaluate your models, improve your prompts, optimize cost, and monitor continuously. This page shows what each of those steps looks like in practice.
AI optimization is the ongoing operational discipline of reducing AI operating cost and improving model performance by reading your operational data, evaluating model capability and fitness against your actual workload, improving prompt quality, and confirming outcomes over time. It is not a one-time configuration — it is a repeating cycle of evidence, interpretation, and decision.
Not what buttons to click — what to look for, what the data is telling you, and what to do next.
The operational record behind Model Optimizer is your AWS Bedrock Model Invocation Logging data — every model call, every token consumed, every prompt pattern, every error, every cost event. Before you optimize anything, understand the data.
The first question is not "how do I reduce cost" or "which model should I switch to." The first question is: what is actually happening in my AI environment right now?
What to look for first:
The data will tell you where the problems are before you have to go looking for them. Optimization starts with understanding the operational picture — not with a hypothesis about what needs fixing.
Not every model underperformance is visible in cost or error data. Some of it is visible in output quality — and some of it only becomes visible when you compare your current model against the benchmarked field on the task types that matter to your workload.
A model that ranks in the top quartile on Reasoning and QA Analytical but falls to mid-field on Structured Routing or Creative Writing is not a bad model. It is a model that may be mismatched to parts of your workload.
The Capability Radar makes that mismatch visible. The shape of your model's profile across 11 task dimensions tells you where capability gaps exist — and which alternative models have a different shape that might align better with what you are actually doing.
What to look for:
A capability gap is not automatically a reason to switch models. It is a reason to ask whether the gap is affecting outcomes — and whether a better-fit model is available at a comparable or lower cost.
Benchmark scores describe capability. Model Fitness tells you how well that capability aligns with your workload.
The fitness view anchors every evaluation to your current model and your actual workload. When you look at Model Fitness, the question is not which model ranks first in the field — it is where your current model sits, and whether the models ranked above it are worth the cost difference.
What to look for:
A model ranked fifth in the field at a lower cost than the top-ranked alternative may be the right operational decision. Fitness evaluation gives you the data to make that call with precision rather than assumption.
Not every prompt needs work. Model Optimizer identifies the ones that do — and prioritizes them by the operational impact of fixing them.
High-volume prompts with identified issues represent the largest optimization opportunities. A prompt running hundreds of times a month with a negative phrasing issue or excessive redundancy is consuming tokens inefficiently at scale — the cost and quality impact compounds with every call.
What to look for:
When Model Optimizer surfaces a rewrite recommendation, the Diff view shows exactly what changed and why. Before accepting any rewrite, review the Diff — if the explanation doesn't match your intent for the prompt, adjust before deploying.
The Playground lets you test the rewritten prompt against your current model and one benchmarked alternative before anything goes into production. Token efficiency comparison tells you whether the optimization is actually reducing cost. Cross-model testing tells you whether the alternative model handles the optimized prompt better than your current one.
AI cost reduction is not found in a single lever. It emerges from the intersection of several signals read together.
A model running high call volume at high cost, on a task type where it ranks mid-field in the fitness evaluation, with prompt patterns that show token inefficiency — that is a compound opportunity. The cost is high because the model is expensive. The fitness score suggests alternatives exist. The prompt inefficiency is compounding the token spend. Fixing all three produces a materially different cost outcome than fixing any one alone.
Cost reduction signals worth tracking:
The data does not tell you what decision to make. It tells you where to look. The decision requires judgment about what matters most to your organization — cost, quality, consistency, or some combination of all three.
Every optimization decision — model switch, prompt rewrite, workload routing change — needs to be confirmed by what happens after it is deployed.
Optimization without monitoring is a hypothesis, not a result.
After deploying any change, the monitoring layer is where you verify outcomes. Cost by model shows whether the switch produced the expected spend reduction. Token consumption trends show whether the prompt rewrite is holding at lower token counts. Error rates show whether a model change introduced new failure patterns.
What to watch after an optimization:
Optimization is a cycle, not an event. The monitoring layer closes the loop — confirming what worked, surfacing what didn't, and identifying the next opportunity.
Organizations rarely reduce AI operating cost through a single model switch or one rewritten prompt. They reduce it by repeatedly asking the right questions.
That cycle repeats every day in mature AI operations. Model Optimizer exists to make each step observable, measurable, and easier to execute.
Every optimization action follows the same sequence — Evidence → Intelligence → Decision. The invocation data is the evidence. The monitoring, capability, fitness, and prompt analysis layers are the intelligence. The model selection, prompt rewrite, and cost decisions are the output.
Your AWS Bedrock Model Invocation Logging data — every model call, every token, every prompt pattern, every error, every cost event — read from your own S3 bucket.
The monitoring, capability, fitness, and prompt analysis layers turn that record into insight — where the problems are, which models fit your workload, and which prompts are worth optimizing.
Model selection, prompt rewrites, and cost decisions — and monitoring confirms each one held after it was deployed.
No step in that sequence is optional. Evidence without intelligence produces data overload. Intelligence without evidence produces assumptions dressed as insight. Decisions without either produce outcomes that cannot be explained, trusted, or repeated.
That discipline — grounded in observable operational data at every step — is what separates optimization from guesswork.
AI optimization is the ongoing practice of reducing AI operating costs and improving model performance by reading operational data, evaluating model capability and fitness against your actual workload, improving prompt quality, and monitoring outcomes over time. It is not a one-time configuration — it is a repeating cycle of evidence, interpretation, and decision.
Model Optimizer reads AWS Bedrock Model Invocation Logging data delivered to your own S3 bucket. Your data never leaves your AWS environment — Model Optimizer connects to your bucket with read-only access and analyzes the invocation records from there.
In Privacy Mode, no. Model Optimizer reads metadata only — prompts and responses remain in your AWS account and are never accessed. In Full Analytics Mode, prompts and responses are read from your S3 bucket for analysis but are never stored by Model Optimizer. Organizations connecting in Full Analytics Mode execute a formal signed data handling agreement prior to data access.
Model Fitness evaluates your current model's position within the benchmarked field on each of the 11 task dimensions in the Model Optimizer framework — anchored to your actual deployment, not models in the abstract. The fitness view surfaces rank, percentile, and cost delta for every alternative so you can assess whether a higher-performing or lower-cost model is worth switching to.
Model Optimizer prioritizes prompts by call volume and finding severity. High-volume prompts with Warning-level findings represent the largest optimization opportunities — the cost and quality impact of inefficient prompt construction compounds with every call. Not every prompt needs work; the analysis surfaces the ones that do.
The rewritten prompt goes into the Playground before deployment. The Playground tests the rewrite against your current model for token efficiency and against one benchmarked alternative model for cross-model comparison. You review the results before deciding whether to deploy the optimized version.
The monitoring layer confirms outcomes after deployment. Cost by model shows whether a model switch produced the expected spend reduction. Token consumption trends show whether a prompt rewrite is holding at lower token counts. Error rates show whether a model change introduced new failure patterns. Optimization without monitoring is a hypothesis, not a result.
Capability describes what a model can do — its benchmark performance across standardized task types. Fitness describes how well that capability aligns with your specific workload, prompt patterns, and cost constraints. A highly capable model may be a poor fit for your workload if its strengths don't match the task types you use most.
Most organizations complete the full setup in under 30 minutes. If AWS Bedrock Model Invocation Logging is already enabled and delivering to S3, connection to Model Optimizer can be completed in minutes. If Model Invocation Logging needs to be configured from scratch, the automated Python script option takes approximately 2 minutes; the manual AWS Console option takes approximately 10 minutes.
Full Analytics Mode reads prompts and responses from your S3 bucket to provide complete optimization intelligence — including prompt analysis, rewrite recommendations, and Playground testing. Privacy Mode reads metadata only and never accesses prompt or response content. Full Analytics Mode requires a formal signed data handling agreement. Privacy Mode requires approximately 10 minutes of additional setup.
Model Optimizer makes each step of the optimization cycle observable, measurable, and easier to execute — starting with a free account. No code changes. Read-only access via IAM role.
Get Started Free