AI Features in Flutter: The Model Returns Text, Not a Type

You ask the model for a list of flashcards as JSON. It returns a list of flashcards as JSON. You write jsonDecode, map it into your model class, render the deck, and it works — forty times in a row, on every input you thought to try. So you ship it.

You are not calling an API. You are calling something that returns text shaped like an API response, most of the time. The forty successes are not evidence of a contract; they are a sample. And the first user to paste in a page of Hungarian, or to have a flaky connection, or to ask for two hundred cards, gets whatever the sample did not contain.

Keeping the key off the device is a different problem with its own post, and everything below assumes the model call already happens on a server you own. This is about the value that comes back from it, and about the two places in a Flutter app where a bad reply turns into a crash instead of a state.

"Reply with JSON only" is a request, not a constraint

A prompt instruction is advice. It is followed most of the time, which is the worst possible failure rate, because it is high enough to pass testing and low enough to be a daily support ticket. The shapes it fails into are worth naming, because each one needs a different answer.

Text around the JSON. A markdown fence, a Sure — here are your cards: preamble, a closing sentence explaining what it did. Stripping fences with a regular expression is the usual patch, and it works until the model puts a fence inside a card.

JSON that will not parse. Single quotes, a trailing comma, an unescaped quotation mark inside a string, or — the common one — an object that simply stops halfway through because the reply hit a token ceiling.

Valid JSON of the wrong shape. This is the one that survives every defensive measure aimed at the first two. The document parses cleanly and then cards is a string rather than a list, or the field is called question this time instead of front, ordifficulty comes back as "medium" where your enum expects an integer. Your parse succeeds. Your app breaks one screen later, in a widget, with a cast error and a stack trace that names none of this.

Make the shape the API's problem

The first two categories are solved outright, and for free, by asking the provider to constrain generation to a schema rather than asking the model to remember one. With strict structured output the response is guaranteed to be syntactically valid JSON matching the schema you supplied — not encouraged to be, guaranteed.

const CARD_SCHEMA = {
  type: "object",
  properties: {
    cards: {
      type: "array",
      items: {
        type: "object",
        properties: {
          front: { type: "string" },
          back: { type: "string" },
          // Optional is expressed as a union with null, because
          // every property has to appear in "required".
          hint: { type: ["string", "null"] },
        },
        required: ["front", "back", "hint"],
        additionalProperties: false,
      },
    },
  },
  required: ["cards"],
  additionalProperties: false,
};

const response = await client.responses.create({
  model: "gpt-4o-mini",
  input: [SYSTEM_PROMPT, { role: "user", content: notes }],
  max_output_tokens: 2000,
  text: {
    format: {
      type: "json_schema",
      name: "deck",
      strict: true,
      schema: CARD_SCHEMA,
    },
  },
});

Two details in there are requirements rather than style. additionalProperties must be false on every object, and every property must be listed in required — which is why an optional field is declared as a union with null rather than left out. A schema that breaks either rule is rejected when you send it, not when it fails, so you find out immediately.

The same shape exists on the older chat completions endpoint under response_format, and Gemini calls it a response schema alongside a JSON response type. Whichever you are on, the point is the same: the constraint belongs in the request, not in the prose of a system prompt where it competes for attention with everything else you asked for.

Two failures a strict schema does not remove

Strict mode guarantees the output matches the schema when the model produces a complete output, and there are exactly two ways it does not.

A refusal. When the model declines, it does not return a schema-shaped apology — it returns a refusal, surfaced as its own content item with a refusal field rather than the parsed object you were expecting. Code that reaches straight for the output text finds something that is not the deck and is not an error either. Check for it explicitly, and give it a real message in the UI, because a user who pasted something the model would not touch deserves better than a spinner that stops.

Truncation. If generation hits the ceiling before the closing brace, the response comes back with a status of incomplete and a reason of max_output_tokens — on chat completions, the equivalent is a finish_reason of length. The body is a fragment of valid JSON, which is to say invalid JSON.

This is where a lot of teams end up blaming the prompt. The parse error is real, the JSON genuinely is malformed, and none of it is a prompting problem — you asked for more output than you allowed room for. The fix is on your side of the request: raise the ceiling, or bound the ask. " Generate exactly ten cards" is a bounded request. "Generate cards covering this document" is not, and its cost and its failure rate both scale with whatever the user pasted in.

if (response.status === "incomplete") {
  // A product state, not a 500. The client can offer to retry
  // with a smaller selection.
  return Response.json(
    { error: "too_long", reason: response.incomplete_details?.reason },
    { status: 422 },
  );
}

Parse at the boundary, not in the widget

On the Flutter side there is one rule that removes most of the remaining damage: nothing downstream of the network layer ever sees a Map<String, dynamic>. A map passed into the widget tree turns every field access into an unchecked cast performed during a build, and a cast error in a build is a red screen rather than an error state.

class MalformedResponse implements Exception {
  MalformedResponse(this.detail);
  final String detail;
}

class Card {
  Card({required this.front, required this.back, this.hint});

  final String front;
  final String back;
  final String? hint;

  factory Card.fromJson(Map<String, dynamic> json) {
    final front = json['front'];
    final back = json['back'];

    // Fail here, with a sentence, rather than three widgets later
    // with "type 'Null' is not a subtype of type 'String'".
    if (front is! String || front.trim().isEmpty) {
      throw MalformedResponse('card.front missing or empty');
    }
    if (back is! String || back.trim().isEmpty) {
      throw MalformedResponse('card.back missing or empty');
    }

    return Card(
      front: front,
      back: back,
      hint: json['hint'] is String ? json['hint'] as String : null,
    );
  }
}

The empty-string check is not padding. A schema that requires a string is satisfied by "", and a deck of twelve blank cards is a worse outcome than a parse failure — it looks like your app lost the data. The same applies to a cards array with nothing in it, which is schema-valid and product-broken, and belongs in the same check.

Retry the right failures, once

Retries are where an AI feature quietly becomes expensive, so it is worth being precise about which failures are worth repeating.

Retry a 429, a 500, a 503 and a timeout with exponential backoff and jitter. These are transient by definition, and without jitter every phone that failed during the same blip comes back at the same moment.

Never retry a 400. A malformed request is deterministic: the same request produces the same rejection, and a retry loop around one is a way to send it a thousand times.

Retry a parse failure exactly once, and only if you feed the failure back in — a second identical request is a coin flip, while a request that says which field was wrong usually is not. If the second attempt fails too, that is an error state, not a third attempt.

Then there is the failure with no status code at all, and it belongs to mobile specifically: the user backgrounds the app, or walks into a lift, and the connection dies mid-response. Nothing arrives, and the tokens were generated and billed anyway. If a per-user quota is being decremented before the call, a user with a bad signal burns their daily allowance on replies they never saw — so charge the quota against a request id you can recognise on retry, and cancel the in-flight request when the screen goes away rather than leaving it running for a widget that no longer exists.

What a schema does not buy you

Structured output is a guarantee about shape. It says nothing whatsoever about whether the content is true. A flashcard with a confidently wrong answer, a recipe with an ingredient that was never in the pantry, a summary of a clause the document does not contain — every one of those is schema-valid, and no amount of tightening the JSON will catch any of them. Anything that has to be right needs a check against your own data, or a human, or an interface honest enough that the user is checking it.

Two related habits worth keeping. Model output is untrusted input, so it should never be rendered as HTML or as markdown with clickable links, and it should never be allowed to choose which function your app calls — the text it is summarising can ask it to do things. And the cost and quota design underneath all of this is its own subject, covered where the key handling is.

Treat it as a feed you do not control

The mental model that makes all of this obvious is one you already apply without thinking: the model is a third-party feed. You would not map an external JSON feed straight into a widget without validating it, retry it blindly on every status code, or assume its fields because they were there last week.

So constrain it with a schema in the request, check for the refusal and the truncation the schema does not cover, parse into a real type at the network boundary, and give every one of those failures a screen. The feature is not the model call. The feature is what your app does on the day the model call comes back wrong.