Understanding evidence
A worked example: auditing my AI treatment analysis.
The useful result was a clearer decision and better questions. Its personal risk forecasts were not validated.
Source and scope. This is an anonymized, edited walkthrough of my saved Claude analysis, “Adjuvant Treatment Decision: Considerations.” The original combined case facts, specialist opinions, study results, and model assumptions. This page separates them and includes corrections from a later evidence review.
It is not the full interactive artifact, a validated calculator, or a record of a treatment I chose. Personal identifiers, exact clinical dates, and private-record links are omitted.
The actual decision
After surgery for a node-positive tumor involving the appendix/cecal region, with dMMR biology, how should we weigh adding chemotherapy to an immunotherapy approach? Pathology interpretation and postoperative imaging were still unresolved. That is a narrower question than “Does chemotherapy work?”
What I gave the analysis
- Pathology reports, with conflicting interpretations left visible.
- Written recommendations and consultation notes, kept separate from one another.
- Research from appendix cancer and adjacent colorectal settings.
- Questions about recurrence prevention, treatment harms, and what happens if disease returns.
The reasoning map
| Decision input | How the analysis used it | What still needed checking |
|---|---|---|
| Appendix versus colon evidence | Compared the anatomical site with the tumor’s dMMR biology. | A “colon-like” label is a hypothesis, not a validated probability or a substitute for integrated pathology. |
| Combination treatment | Used ATOMIC as an anchor for chemotherapy plus atezolizumab. | ATOMIC tested adding atezolizumab to chemotherapy in colon cancer. It did not test adding chemotherapy to atezolizumab alone in appendix cancer. |
| Appendix cohorts | Examined retrospective chemotherapy findings and small subgroups. | Treatment selection, sparse untreated groups, and different tumor biology limit causal conclusions. |
| Harms and burden | Set uncertain recurrence benefit beside neuropathy, treatment time, and quality of life. | Personal toxicity estimates and value judgments need explicit sources and clinician/patient input. |
| Pending tests | Asked whether ctDNA, molecular testing, or imaging would change the plan. | A negative test is not proof of no residual disease or a validated rule for omitting treatment. |
| Treatment after relapse | Considered how future treatment possibilities affect today’s tradeoff. | Later treatment is conditional; access and effectiveness cannot be assumed. |
Where I would challenge the output
The original model assigned personal recurrence and survival percentages and translated them into events prevented per hundred people. Those were model estimates assembled from neighboring evidence. They were not measured outcomes for a matching patient population. I do not present those forecasts here as my prognosis.
- Check the direction of the comparison. Both ATOMIC groups received chemotherapy. Its result cannot supply the isolated benefit of chemotherapy over immunotherapy alone. Study.
- Keep uncertainty attached to the inputs. Blending appendix and colon evidence with chosen weights does not validate the weights. A judgment range is not a statistical confidence interval.
- Keep outcomes and clocks consistent. Disease-free survival, recurrence, and overall survival are different endpoints. Survival measured after relapse cannot simply be combined with recurrence measured from surgery to calculate survival from surgery.
- Do not create clinician agreement. The original summary flattened differences in regimen duration and conditional recommendations. A written plan and a transcript must remain separately attributed.
- Audit my preferences too. The model inferred how I valued treatment burden and recurrence risk. I need to state those priorities myself.
The questions worth taking to the doctor
- Which pathology interpretation and treatment setting should guide the decision?
- What evidence supports the added benefit of chemotherapy over immunotherapy alone in this situation?
- Which parts of your estimate are measured, extrapolated, or clinical judgment?
- What would positive, negative, or inconclusive pending tests change—and why?
- What regimen and duration do you recommend, what are the main harms, and when must we decide?
A reusable audit prompt
“Separate case facts, clinician statements, study findings, and model assumptions. Check whether each study tests the actual treatment comparison. Preserve conflicting reports. Flag unsupported personal probabilities, mismatched endpoints, and incompatible follow-up periods. End with the few questions that need a clinician’s judgment.”
The lesson I want to share is the method: make the reasoning inspectable, challenge the weak steps, and turn the analysis into a short clinical conversation. A polished table or correct arithmetic does not validate a medical model.
Sources & scope
- Atezolizumab plus FOLFOX for Stage III Mismatch Repair–Deficient Colon Cancer ↗ATOMIC investigators · New England Journal of Medicine. Randomized postoperative colon-cancer trial. Not an appendix-cancer or immunotherapy-alone trial.
- Nonoperative Management of Mismatch Repair–Deficient Tumors ↗Cercek and colleagues · New England Journal of Medicine. Phase 2 study of immunotherapy before planned surgery, with separate rectal and mixed nonrectal cohorts.
- The Landscape of ctDNA in Appendiceal Adenocarcinoma ↗Clinical Cancer Research · institutional publication record. Blood-based mutation profiling. Its detection rates are not the sensitivity of every tumor-informed residual-disease assay.
Research links checked November 2026. Patient-written interpretation; not a clinical review.