Skip to content

workflows.md

Latest commit

 

History

History
146 lines (107 loc) · 19.2 KB

workflows.md

File metadata and controls

146 lines (107 loc) · 19.2 KB
Raw

Agent workflows

Follow the section matching the task and complete its verification before reporting success. Commands take a room or asset id, not a directory. <room-dir> means the resolved public or private room folder; <asset-dir> means the resolved asset folder. See data locations for lookup rules and file formats when authoring JSON.

Intake dropped files

Organize the scans, photos, models, and references the user identifies as new input. Keep project files and the demo recording video.mp4 in place. The root ignore rules keep other loose files out of Git during intake.

  1. Inspect each input file and match it to a room or product using the conversation and existing files. Ask when the destination is ambiguous. Continue once every file has a known destination.
  2. Create the relevant private folders. Move room scans/meshes to private/rooms/<room>/scan/, room photos to photos/, product photos to private/assets/<id>/photos/, and loose product GLBs to private/assets/inbox/. Group a glTF with its referenced buffers and images. Keep original bytes intact and resolve filename collisions without overwriting an original.
  3. Confirm the moved files exist at their destination. Import RoomPlan JSON using the scan workflow below; adopt retailer GLBs with fetch-amazon --file; record source URLs and known dimensions for product references. Ask for missing listed furniture dimensions before modeling a buyable item.
  4. Update the room's DESIGN.md with agreed choices and open questions. Verify the resulting room or asset, then report its location and the viewer URL.

Import a scan

If the user needs scanning guidance: start facing the wall to be called North, keep doors open, and export RoomPlan CapturedRoom JSON. An optional mesh must use a supported overlay format. Request straight-on photos of walls, corners, slopes, and door hinges, with a tape measure visible for key dimensions.

  1. The user puts the RoomPlan JSON (and an optional .glb, .gltf, .usdz or .obj mesh) into <room-dir>/scan/ and the photos into <room-dir>/photos/.
  2. Run bun run import-scan <room> <room-dir>/scan/<file>.json. Read the WARN lines and the notes it writes into room.json (door swing guesses, trim, ceiling, fixtures, and any unscanned, curved, off-square or leftover walls).
  3. Verify: run bun run check <room> (WALLS lists every wall, its length and its cutouts) and bun run render <room>, and look at <room-dir>/renders/default--plan.png: wall names, door and window positions.

Align the mesh overlay

  1. Set overlay.file in room.json, relative to the room folder, such as "scan/mesh.glb".
  2. Run bun run render <room> --overlay --shots plan. The plan shot hides the shell floor and cuts the mesh off above 48 in, so the scanned floor outline and lower walls show against the shell's wall bands.
  3. Compare the mesh walls to the shell outlines. Rotate with overlay.yaw first (degrees CCW), then shift with overlay.offset x/y in inches. The px/in and origin pixel printed for the plan turn pixel errors into inches.
  4. Check the height with a view render (--overlay --shots <view>): offset[2] moves the mesh up.
  5. Verify: repeat until the edges agree as closely as you can see (about 2-3 in at the plan's scale). The overlay is a reference for details. Dimensions come from room.json and the tape.

Add details from photos

  1. Doors: set swing (in, out, or none for sliding and pocket doors) and hinge (left, right, or both for double doors and bifolds), seen from inside. Add closet doors and missed doors as door cutouts, and fix window sill and height. A cutout's offset is the s of its left edge.
  2. Trim: trim.baseboard (height) and trim.casing (width), 0 = none.
  3. Fixtures go in room.json fixtures as placements, with the back (+y) against the wall and the rotation facing into the room, for example { "id": "outlet-e", "asset": "outlet", "x": 143.8, "y": 30, "elevation": 12, "rotation": 270 } on an E wall at x = 144. Add a new fixture kind (vent, radiator) as a small parts asset.
  4. Measure along a wall as s from the photo's left corner. Write the cutout or fixture, run bun run check <room>, and compare the WALLS and FIXTURES s values with the photo.
  5. Wall colors go in materials: walls (every wall), floor, ceiling, trim, or a wall id for an accent. Each is { "color": "#RRGGBB" } plus an optional "texture" id from textures/index.json; the color multiplies the texture, so use #FFFFFF for the texture's own colors. Look up paint hex values yourself.
  6. Sloped ceilings: edit wall top profiles ([[s, h], ...]) from photos and the overlay, then run bun run ceiling <room>. For shapes it can't express, hand-author the panels: a dormer or a slope in one arm of an L-shaped room shows up as SHELL warnings, but a tray, hip or vault ceiling above flat walls gives no warning at all, so compare with the photos. Use one planar polygon per flat ceiling face, with the points clockwise or counterclockwise and the edges shared exactly with neighboring panels. Wherever two levels meet at a vertical step, add a vertical riser panel, or the step leaves a see-through slot.
  7. A roof window in a slope: a parts fixture with one thin glass box tilted to the slope with rotate, placed at the ceiling. check doesn't treat it as a window.
  8. Verify: run bun run check <room> and bun run render <room>, compare with the photos, and delete each notes entry you resolved.

Build a parts model from product photos

  1. Get the listed W x D x H (ask the user if it's unknown).
  2. Save the photos to <asset-dir>/photos/.
  3. Write <asset-dir>/asset.json: { "name": "...", "source": "<product URL or photos>", "file": "parts.json", "dims": [W, D, H], "notes": "..." }.
  4. Read the parts format. Build boxes, cylinders, lathes, or extrusions with named materials sampled from the references. Use the demo bed and demo lamp as working examples. The parts' combined bounds must match the listed dimensions within 0.5 in, with their bottom at z = 0.
  5. Run bun run render --asset <id>. Fix errors and MODEL warnings, then compare all three previews with the product photos. Finish when the dimensions and orientation agree and any simplified design details are recorded in the asset's notes.

Fetch a retailer model

  1. IKEA: bun run fetch-ikea <URL or item no> (204.387.21 works). Amazon, or any page with a 3D viewer: bun run fetch-amazon <URL> (add --id <asset-id> for non-Amazon URLs). On exit 3, give the user the manual steps: open the page in their browser, click "View in 3D", open DevTools > Network, filter glb, and save the response into private/assets/inbox/. Then adopt it with bun run fetch-amazon <URL> --file private/assets/inbox/<file>.glb. If IKEA blocks every locale, use bun run fetch-amazon <IKEA product URL> --id <id>, or the userscript from github.com/apinanaivot/IKEA-3D-Model-Download-Button into private/assets/inbox/ plus --file.
  2. Set dims in asset.json to the listed size and verify it against the product page, because the model is rescaled per axis to exactly dims.
  3. Verify: run bun run render --asset <id> and look at preview-front.png. If it shows the back, add 180 to modelYaw (0, 90, 180 or 270). If it shows a side, change modelYaw by 90 or 270 and swap dims[0] and dims[1]: the fetch ordered W and D to fit the model as it was turned. Re-render until the front shows. The MODEL hint says the same.

Edit a layout

  1. Place, move and rotate by editing items in <room-dir>/layouts/<name>.json, for example { "id": "lamp", "asset": "demo-lamp", "x": 75, "y": 112, "elevation": 24, "rotation": 0 }. Ids must be unique across the layout's items and the room's fixtures. Branch by copying a layout file under a new name; the app's picker lists it.
  2. Recolor parts assets with "colors": { "<parts material name>": "#RRGGBB" } (GLB colors are ignored). Paint with layout materials, which has the same keys as room.json materials. Each key replaces the whole room entry, and wall X uses key X, else walls, so a room accent on N survives a layout that sets walls unless the layout also sets N. WALLS in check shows each wall's resolved paint and where it came from.
  3. Save the user's pasted "Copy view" JSON as views.<name>: "views": { "from-door": { "position": [108, 104, 64], "target": [32, 60, 30], "fov": 60 } } (plan inches, vertical fov 10-120).
  4. The user's pasted click info (id, point, and s for a wall) tells you what they mean by "this" and "here".
  5. Verify: run bun run check <room> --layout <name>, then bun run render <room> --layout <name> and inspect every PNG. Finish when placement, facing, and colors match the request, with errors fixed and remaining warnings explained.

Verify

  1. Run bun run check <room> --layout <name>. Fix validation and JSON errors first; geometric checks are skipped until those pass. Resolve each warning below or record why it is acceptable for this layout. Each WARNING line starts with its code:

    Code Meaning What to do
    DATA Dims all under 3.5 in (looks like meters?), or colors on a GLB asset. Convert dims to inches (x 39.37). Drop colors from GLB items.
    OUTSIDE The footprint pokes that far past the named wall. Move it into the room.
    CEILING The top is above the ceiling at that point (sloped ceilings). Move toward the high side, lower elevation, or choose a lower item.
    OVERLAP Two footprints overlap, and so do their heights. Move one. Stacked things need elevation at or above the lower item's top.
    DOOR In a door's swing arc (swing), or the 18 in clear zone of an out-swing, sliding or doorless opening (doorway). Move it out of the zone.
    WINDOW Taller than the sill + 2 in and within 12 in of the window. Move it, or tell the user it blocks the window.
    FACING The item's front is within 6 in of a wall. Use the rotation it names (the "0 faces North" slip).
    WALKWAY A 12-36 in gap between two items or an item and a wall. Widen it to 36 in, or explain (pairwise heuristic; it can flag a gap nobody walks through).
    MODEL A parts model doesn't fill its dims box (names the axis and shift), or a GLB's bbox differs from the listed dims. Fix the parts, or verify the listed dims and modelYaw.
    SHELL Wall tops and ceiling disagree, a cutout is taller than its wall, or floor has no ceiling over it. Run bun run ceiling <room> or fix the panels, or fix the cutout or wall top.

    ITEMS lists each item's walls within 36 in, such as N 0.0 (back, s 32.0): the item's back is 0.0 in from wall N, with its center at s 32.0. (front) next to a wall means the item faces that wall: use the rotation FACING names. FIXTURES shows each fixture's nearest wall, s and gap. NOTES lists open room.json notes.

  2. Run bun run render <room> --layout <name>: the plan plus every view. The plan line prints its scale (6.56 px/in, origin (SW) at px (295, 906) for the demo).

  3. Inspect every PNG for floating items, clipping, facing, colors, and agreement with the request. Finish only when there are zero errors, every warning is resolved or explained, and every visual mismatch is fixed or documented.

Correct a dimension from a tape measure

  1. Ask the user to tape wall to wall at about 36 in high, plus any door, window or slope start that matters.
  2. Compare with the WALLS lengths in bun run check <room>.
  3. Move only corners on the N or E side, so the SW origin stays at (0, 0).
  4. For each wall whose start moved along its own direction by delta = (new start - old start) dot dir, subtract delta from its cutout offsets and its top s values, so they stay put in plan.
  5. Move fixtures and items on a moved wall by the same plan shift.
  6. Verify: run bun run ceiling <room> (unless the panels are hand-made), then bun run check <room>, and confirm the WALLS cutout plan coordinates didn't change.

Will it fit?

  1. Obtain the listed dimensions; ask the user if they are unavailable. Continue once all three dimensions are known.
  2. Copy the layout to a scratch name (<room-dir>/layouts/<scratch>.json) and place the item there.
  3. Run bun run check <room> --layout <scratch>.
  4. Answer with the gap numbers, minus 0.75 in for each baseboarded wall that the item's floor-level back or sides touch.
  5. Say whether the governing room dimension was tape-verified (watch for off-square wall notes). If it wasn't, ask for that one measurement before the user buys.
  6. Delete the scratch layout and confirm it is gone. Finish with the gap numbers, baseboard allowance, measurement confidence, and any required tape measurement in the answer.

Photoreal images with your own image tool

The app produces measured 3D renders and matching .prompt.txt files. Generation uses the agent's available image tool; the app requires no image API key. If your agent has no image tool, provide the render, references, and prompt for the user to run with their preferred tool.

  1. Read the room's DESIGN.md, selected layout, and relevant asset metadata. Establish the exact products, sizes, finishes, and chosen camera before generating.
  2. Run bun run render <room> --layout <name> --shots <view>. Inspect the PNG and read its .prompt.txt. Fix the measured scene first if furniture or geometry is wrong.
  3. Collect actual product reference images. Follow each selected product's source URL to Amazon, IKEA, or the relevant retailer/manufacturer. Verify the exact size and color variant. Save useful front, side, detail, and material photos under that product's private asset photos/ folder; record the source URLs in its metadata or a small references.md. Prefer listing images of the actual product over search thumbnails or similar-looking furniture. If a listing is blocked, try the manufacturer's page or ask for product photos; say which references are missing.
  4. Attach the rendered scene and the downloaded reference images to the agent's image tool. Label which item each reference depicts. Use the generated prompt as the base: the render controls camera, geometry, dimensions, and placement; product photos control shape details, construction, material, and finish. Use several close-ups or separate views when one crowded reference set loses detail.
  5. Save the output in <room-dir>/renders/photoreal/<layout>--<view>--<style-or-number>.png. Keep generation notes and intermediate attempts private. Press P in the viewer to browse the results.
  6. Compare the result with the render for walls, slopes, openings, item count, placement, and scale; compare each visible product with its listing photos for design and finish. Correct mismatches and regenerate. Finish when each comparison passes or a remaining limitation is recorded beside the image. Images illustrate a design; the JSON model and tape measurements determine fit.

Prompt to adapt after attaching and labeling the images:

Turn the attached room render into a realistic interior photograph. Preserve its camera, room geometry, openings, furniture sizes, and placements. Match each named furniture item to its attached product reference photos, including its silhouette, legs, handles, upholstery, wood finish, and selected color. Preserve the measured layout while adding realistic material detail and lighting. Keep all existing items and add nothing. [List the reference-to-item mapping and any requested styling change.]

For an edit based on a real room photo, preserve the original photo in photos/, attach it with the relevant product references, and name only the requested changes. Save the generated edit separately. For example:

Edit this room photo: change the walls to sage #7C8C7A and replace the chair with the attached product. Preserve the camera, room geometry, openings, lighting, and all other furniture.

Shop: swappable real products

  1. Each product is an asset with a product block in asset.json (retailer, url, price, model retailer or mock, tier). See SPEC.md > Layout JSON for the layout's shop block: slots name the base items they replace, options give the replacement placements, presets name whole bundles.
  2. Place each option's items yourself (back against the wall with the usual gap, same center otherwise). Items standing on a swapped item (lamps, small plants) follow it automatically.
  3. Verify: bun run check <room> --layout <name> --pick <slot>:<option> for every option and every preset, then bun run render <room> --layout <name> --pick <preset> and look.

Undo

  1. ls private/.history/<file>/ lists the versions (UTC timestamps, oldest first), for example ls private/.history/<room-dir>/layouts/<name>.json/.
  2. bun run undo <file> goes back one version (repeat to go further). It prints restored <file> to <stamp> (<n> older versions left). Or restore by hand by copying a version from private/.history/<file>/.
  3. Verify: run bun run check <room> --layout <name>. History only exists for changes made while bun run dev ran; otherwise use your .bak copy.

Troubleshooting

ERROR lines read <file> <json path>: <message>, for example room.json walls[1].cutouts[0].width: must be > 0. Common mistakes:

  • JSON syntax: rooms/demo/layouts/default.json:3:25: Expected "," but found ""y"" gives the line and column. ...: trailing comma or comment (not allowed in JSON) means exactly that.
  • Counterclockwise walls: room.json walls: must be listed clockwise seen from above (...). With +y North, go W wall northward, N wall eastward, and so on, with the room on the right of travel. To fix, reverse the list and start each wall at its old end; its s values then count from the other end (top: s -> length - s, points in reverse order; cutouts: offset -> length - offset - width).
  • Cutout past the end of its wall: room.json walls[1].cutouts[0]: ends at s 152.0, past the wall's end (length 144.0). offset + width must be at most the wall length + 0.5.
  • Layout paint key naming no wall: layouts/default.json materials.NE: no wall "NE" (walls: W, N, E, S). Rename the key to a listed wall id (a re-import or wall edit can rename walls). In room.json a typo like north gives unknown key.
  • Parts model built around its center, or the wrong size: MODEL armchair: parts z -15.0..15.0 vs 0.0..30.0 (move every part +15.0 in z). Add the shift to every part's at, or fix the part sizes or dims.
  • Dims in meters: DATA chair: dims 0.6 x 0.6 x 0.9 in, looks like meters?. Multiply by 39.37.
  • GLB colors ignored: DATA sofa: colors ignored on a GLB asset. Only parts assets can be recolored; remove colors.
  • check passes but the render looks wrong (an item faces a wall, or a GLB shows its side or back): fix rotation with the facing table, or modelYaw as in "Fetch a retailer model" step 3.
  • Render times out or Chromium is missing: run bunx playwright install chromium.
  • Unknown asset or id clash: items[4].asset: unknown asset "nope" means neither public nor private storage has assets/nope/asset.json; "light" is also a room fixture id means pick another id.