Unstructured notes are where a lot of real work actually lives. They are the meeting scribbles, the forwarded email threads, the half-finished research dumps, the voice-to-text fragments, and the shared docs that multiple people have edited without agreeing on a single narrative. Turning that material into an executive brief that a decision-maker can trust is a common professional task. It is also a task that looks deceptively simple in a demo.
I ran a controlled test with Gemini 4 Pro to see how far the model could get on its own, where it broke, and what a human still had to supply before the output became something I would be willing to put in front of another person. The demo is the beginning. The workflow is the test.
The Task and the Materials
I assembled a realistic packet from work I had already done in a different context, then stripped the identifying details. The packet contained:
Twelve pages of raw notes from three separate conversations, written by two different people with inconsistent terminology.
An email thread that contained one critical constraint buried in a reply.
A short spreadsheet with partial numbers and two blank cells that mattered.
A one-paragraph “what success looks like” note that conflicted slightly with language used in the meeting notes.
I gave Gemini 4 Pro a direct professional instruction: produce a two-page executive brief that a senior decision-maker could read in five minutes. The brief needed to state the core problem, the relevant options or findings, the key risks or open questions, and a clear recommendation or next-step set. I told it to rely only on the provided materials and to flag anything that required information the packet did not contain.
I did not pre-clean the notes. The inconsistencies and the buried constraint were left intact on purpose.
What the First Output Delivered
The model returned a cleanly structured brief. It had a short problem statement, three thematic findings, a risks section, and a set of recommended next steps. The prose was calm and executive-appropriate. On a quick skim it looked finished.
Then I checked it against the source packet.
The problem statement had quietly resolved the terminology differences between the two note-takers by choosing the more formal phrasing. That choice was reasonable but unacknowledged; a reader would not know a choice had been made.
The buried email constraint—the one that limited the feasible timeline—appeared only as a soft risk rather than as a binding limit. The model had noticed the language and then under-weighted it.
Two of the numbers in the spreadsheet had been carried forward correctly. A third figure had been interpolated in a way that looked precise and was not supported by the cells that actually contained data. The blank cells had not been flagged as blank; they had been smoothed over.
The recommendation section was coherent and slightly more decisive than the underlying materials justified. It read like the product of a complete analysis rather than a synthesis of incomplete inputs.
None of these issues made the brief unreadable. All of them made it dangerous to treat as finished.
The Failure Modes That Mattered Most
Three patterns showed up clearly.
Resolution without disclosure. When the notes disagreed or used different language for the same underlying issue, the model preferred a single clean formulation. That is often what an executive wants to see. It is also how important disagreements disappear from the record. A useful brief can resolve or it can surface; it should not silently resolve.
Constraint demotion. Soft or buried constraints were registered and then treated as secondary. In real decisions those constraints frequently determine whether an option is viable at all. Under-weighting them produces recommendations that look strategic and later prove infeasible.
Numeric smoothing. Partial data invited completion. The model preferred a full-looking table to an honest one with visible gaps. In an executive setting that preference is costly. Leaders can handle missing numbers; they handle invented precision less well once the error is discovered.
I recorded each of these in the failure log with the same structure I use for every substantial test: task, inputs, observed break, and the human correction required.
The Human Review Pass
I went back through the brief line by line with the source packet open.
I restored the terminology difference as an explicit note rather than a resolved choice. I elevated the buried timeline constraint to a hard limit and moved it into the problem statement. I removed the interpolated figure and marked the data gap. I rewrote the recommendation section so that its confidence level matched the incompleteness of the inputs. Two open questions that the model had folded into prose were pulled back out as open questions.
The revised brief was less smooth. It was also safer. A decision-maker reading it would know where the ground was solid and where it was not.
Total human time after the first model output: about twenty-five minutes. That is less time than writing the brief from scratch, and more time than the “generate and send” fantasy sometimes implies.
What the Test Suggests About Current Capability
Gemini 4 Pro is already good at imposing structure on messy professional language. It can surface themes, maintain a calm executive tone, and produce a document that looks like it belongs in a decision packet. Those are real strengths.
It is still weak at preserving the friction that makes the underlying materials trustworthy. Disagreements, buried limits, and missing data are not noise to be optimized away; they are often the most important signals in the packet. A model that treats them as imperfections to be cleaned will systematically produce briefs that feel more complete than the evidence supports.
This is not a reason to avoid the tool. It is a reason to keep the human review step non-negotiable and to design the prompt so that surfaceable friction is requested rather than suppressed.
A Practical Prompt Pattern That Improved the Second Attempt
On a follow-up run with a fresh context I added three explicit instructions:
When sources disagree, list the disagreement rather than resolving it.
Treat any constraint that limits feasibility as hard until proven otherwise.
Leave blank or partial data visible; do not interpolate without labeling the interpolation as such.
The second output was less polished and more useful. It still required human editing, but the editing was mostly refinement rather than repair of silent distortions. That difference is the entire practical gain.
How This Fits the Larger Testing Approach
I will repeat variations of this task over the coming months: different note styles, different densities of contradiction, different levels of numeric incompleteness. The goal is not a single pass/fail verdict. The goal is a clearer map of the conditions under which the first output can be trusted with light review versus the conditions under which it still needs substantial reconstruction.
Parallel runs with other strong models will appear in the Model Judgment series. The lived result of this particular test belongs here.
The Standard I Am Using
A brief is useful when a decision-maker can read it and correctly understand both the recommendation and the limits of the evidence behind it. Fluency and structure are necessary. They are not sufficient.
Gemini 4 Pro currently clears the fluency and structure bar with room to spare. It clears the “correctly represents the limits of the evidence” bar only when the prompt and the human review process are designed to protect that requirement.
I would use this for assistance—not authorization. The authorization remains with the person whose name sits on the brief.
What I Am Carrying Forward
For any future executive-brief task I will keep the two-pass habit: first output for structure and theme extraction, immediate human check against the source packet for silent resolutions and constraint demotion, then a second targeted pass if needed. I will keep the failure log updated. And I will continue to treat every clean-looking brief as a hypothesis until the friction in the original notes has been accounted for.
The demo is the beginning. The workflow is the test. In this case the test showed a capable structuring engine that still needs a human to keep the inconvenient truths visible. That is a workable division of labor, provided no one pretends the second part is optional.
No comments yet.