Rosewood Internal Review
AI Training And Harness Buildout - Gaps For Team Review
Overall Read
The Kelly/Josh training plan is directionally strong. It correctly separates strategic architecture understanding from technical operating capability.
The main issue is that the plan needs sharper outputs, ownership, and decision points so the training converts into Rosewood buildout capability rather than broad AI exposure.
Key Gaps
| Gap | Why it matters | Team question | Recommended fix |
|---|---|---|---|
| No explicit deliverable dates | Training can become general learning without usable artifacts | What should Kelly and Josh produce by the end of week one and by the end of training? | Define interim and final deliverables before training starts |
| No named Rosewood pilot workflow | Hard to connect training to actual implementation | Which workflow should they use as the practical test case? | Pick one candidate: photo exception review, Penmarc install lifecycle, ProVantage kickoff, or scheduling intake |
| No access matrix | Josh cannot evaluate operability without knowing what he can touch | What can Rosewood access, modify, inspect, deploy, and troubleshoot inside Nucleus/Rosie? | Ask Jody for a clear access and responsibility matrix |
| No source-of-truth map | Kelly needs to know where truth lives across Rosie, Wayfinder, Sites, Cruxos, and external systems | Which system owns which data, and how is current truth distinguished from historical information? | Require a source-of-truth and freshness diagram |
| No failure-mode review | Enterprise agent systems fail through permissions, stale data, unclear handoffs, and partial tool execution | What happens when an agent, tool call, queue, or workflow fails halfway through? | Ask Jody to walk through common failure scenarios and recovery paths |
| No economics lens | Model/tool cost and operating ROI will matter if this scales across Rosewood | How are model cost, tool cost, token use, labor savings, and quality measured? | Add cost tracking and workflow ROI to the training outputs |
| No security/compliance track | Client, employee, financial, legal, and credential data will eventually be in scope | How are permissions, credentials, audit logs, company boundaries, and approval gates enforced? | Add data classification, credential handling, audit logging, and approval gate review |
| No ownership/dependency decision framework | Rosewood needs to know what to own versus rent from Nucleus | What creates lock-in, continuity risk, or operational dependency? | Require a Rosewood ownership/dependency/lock-in memo |
| No operational handoff plan | Training value may stay in Kelly/Josh's heads | How will the team absorb what they learn? | Require written runbooks, architecture maps, and a post-training readout |
| No standard for done | It will be hard to judge whether training succeeded | What would make this training clearly successful? | Define success criteria for Kelly, Josh, and the combined team outcome |
Questions For Jody
- What exactly are Rosie, Sites/Mosaic, Wayfinder, and Nucleus responsible for today?
- What exists in production today versus roadmap or prototype?
- What does Rosewood control directly inside the environment?
- What can Josh provision, inspect, deploy, debug, and modify without Nucleus support?
- Where do logs, state, tool calls, prompts, model choices, memory, and permissions live?
- What happens when an agent job fails halfway through?
- How are company/entity boundaries enforced?
- How are source authority, freshness, and version history represented?
- Which components are portable if Rosewood later needs to replicate them outside Nucleus?
- What first Rosewood workflow does Jody think we should build, and why?
Recommended Required Outputs
Kelly
- Strategic architecture map: Rosie, Sites/Mosaic, Wayfinder, agents, models, tools, memory, data stores, retrieval, graphs, and systems of record.
- Wayfinder gap assessment: what it collects today, what is missing, and what should change.
- Source authority and freshness model.
- Rosewood ownership/dependency/lock-in memo.
- First-pass framework for choosing deterministic workflow vs agent vs RAG vs graph vs Site vs human review.
Josh
- Technical environment runbook: access, deployment, logs, state, tools, permissions, and common failures.
- Working example of at least one tool-using agent.
- Working example of one multi-agent or evaluator pattern.
- Practical graph or relationship-based retrieval example.
- Wayfinder ingestion prototype or design.
- Portability assessment: what is Nucleus-dependent versus generally replicable.
Shared
- Recommendation on the first Rosewood workflow to pilot.
- List of platform gaps or blockers.
- List of near-term build opportunities.
- List of risks requiring leadership decision.
Recommended First Pilot Criteria
| Criterion | Why it matters |
|---|---|
| Clean source data | Reduces false conclusions |
| Repeated workflow | Makes automation worth doing |
| Low external consequence | Keeps early risk manageable |
| Clear human reviewer | Makes escalation practical |
| Cruxos relevance | Keeps the work tied to Rosewood's operating spine |
| Measurable result | Lets us know whether the harness is improving anything |
Current Recommendation
Start with photo documentation + exception flagging as a contained sub-workflow, then attach it to Penmarc or ProVantage depending on which has cleaner data and a more available operating owner.
This is lower risk than scheduling, labor commitments, client-facing updates, or financial workflows, but still tests the core harness ideas: source-of-truth access, field evidence, agent judgment, human escalation, and measurable output quality.