Critical: Screenshot persistence and action-to-screenshot synchronization issues for localization testing

[ebowwa]

Summary

The current implementation has critical gaps that prevent reliable capture-document-replay workflows for localization testing. Screenshots are not automatically persisted, there's no linkage between recordings and screenshots, and the system cannot guarantee reproducible documentation across different locales.

Current State Analysis

📸 Screenshot System Issues

• CRITICAL: Screenshots are NOT automatically persisted - only kept in memory
• Data Loss Risk: All screenshots are lost when app closes unless manually exported
• No Auto-Save: Must remember to export before closing app every time
• Locale Tracking: Uses LocaleInfo.current but only captures system locale at time of screenshot

🎬 Action Recording Limitations

• Good: Recordings auto-save to ~/Documents/recordings.json
• Missing: No locale information stored with recordings
• Missing: No way to link recordings to specific screenshots
• Missing: Cannot trigger screenshots during replay automatically

🔗 Integration Gaps

• No Synchronization: Screenshots and recordings are completely separate systems
• No Correlation: Both have timestamps but no automatic matching mechanism
• Manual Process: Must manually coordinate taking screenshots at same points during replay
• No Validation: Can't verify you captured screenshots at same flow points across locales

💾 Data Persistence Problems

Component	Auto-Save	Risk Level
Screenshots	❌ NO	HIGH - Data loss on app close
Recordings	✅ YES	LOW
Scenarios	✅ YES	LOW
Locales	✅ YES	LOW

⚠️ Critical Workflow Blockers

1. Screenshot Loss: Forgetting to export = losing all work
2. No Recording Context: Recordings don't know their original locale
3. Manual Synchronization: No way to ensure screenshots taken at same points
4. No Replay Triggers: Can't automatically capture screenshots during replay
5. Missing Validation: No way to verify complete coverage across locales

Required Improvements for Reliable Localization Testing

Priority 1: Data Persistence

• Auto-save screenshots to disk immediately after capture
• Create timestamped session folders for organization
• Add recovery mechanism for unsaved screenshots

Priority 2: Recording-Screenshot Integration

• Add screenshot triggers to recordings (mark points where screenshots were taken)
• Store locale information with each recording
• Create recording-screenshot manifest linking them together
• Add "screenshot here" markers during recording

Priority 3: Automated Replay Workflow

• Auto-capture screenshots at marked points during replay
• Validate all screenshot points were captured
• Generate comparison reports between locales
• Flag missing screenshots in workflow

Priority 4: Localization Workflow

• Create "localization session" concept grouping recordings + screenshots
• Add locale switching automation
• Build side-by-side comparison views
• Export locale-specific documentation automatically

Immediate Workaround Instructions

Until these issues are fixed, follow this EXACT workflow to avoid data loss:

Recording Phase (Base Locale)

1. Start recording actions
2. At each important screen, pause and note the timestamp
3. Stop recording (auto-saves)
4. Immediately take screenshots at noted points
5. CRITICAL: Export screenshots NOW before doing anything else
6. Document which screenshots correspond to which points in the flow

Replay Phase (Other Locales)

1. Change iPhone to target locale
2. Load saved recording
3. Start replay
4. Manually pause at each documented point
5. Take screenshot
6. CRITICAL: Export after each session
7. Repeat for each locale

Documentation Phase

1. Create folder structure: ProjectName/Locale/Screenshots/
2. Match screenshots by timestamp/filename
3. Build comparison matrix manually
4. Note any missing screenshots

Code References

• Screenshot persistence: ScreenshotManager.swift:276-350 (export functions exist but not auto-called)
• Recording save: ActionRecorder.swift:891-902 (works correctly)
• Missing integration: No code exists linking the two systems
• Locale storage: Screenshot.swift:373 (captures locale) but Recording.swift:72 (missing locale)

Impact

Without these fixes:

• High risk of losing hours of screenshot work
• Cannot guarantee complete documentation across locales
• Manual process is error-prone and time-consuming
• No validation that all screens were captured in all locales

Recommendation

Implement Priority 1 (auto-save) immediately to prevent data loss. Then build proper integration between recording and screenshot systems to enable reliable, reproducible localization testing workflows.

[ebowwa]

we already have a screenshot function. maybe we do flags at pages so with internationalization its easy.

[ebowwa]

Implementation Attempt Update - Screenshot Display Regression

What Happened

An implementation attempt was made to fix the critical bugs identified in this issue. The fixes included:

1. Screenshot Auto-Save: Added automatic persistence after capture

   • Screenshots now save to ~/Documents/iOSTestingGym/Screenshots/[timestamp]/
   • Metadata saved alongside images

2. Recording-Screenshot Linkage: Linked screenshots to recordings

   • Added screenshotIds: [UUID] array to Recording struct
   • Screenshots captured during recording are now tracked

3. Locale Persistence: Added locale info to recordings

   • Added locale: LocaleInfo? to Recording struct
   • Locale captured at recording time

4. Window Movement Tracking: Added window position tracking

   • New windowMoved action type in recordings
   • Tracks when iPhone Mirroring window is moved

Critical Bug Discovered

During implementation, a screenshot display regression occurred:

• Screenshots were being captured successfully (metadata showed 6.7 MB file sizes)
• However, images rendered as empty boxes in all GUI views
• The issue affected ScreenshotsView, DebugView, and AppsView
• Preview functionality continued working but screenshots didn't display

Technical Details

The issue appeared to be related to NSImage creation from CGImage:

// Original code that stopped working after changes
let nsImage = NSImage(cgImage: screenshotImage, size: NSSize(width: window.frame.width, height: window.frame.height))

Attempted fixes that didn't resolve the issue:

• Adjusting NSImage size parameters (using .zero)
• Adding explicit objectWillChange.send() calls
• Debug logging to trace the issue

Current Status

• Changes have been reverted to restore working screenshot display
• The fixes remain documented here for future implementation
• Root cause of the display regression needs investigation before retry

Next Steps

1. Investigate why NSImage creation/display broke with the persistence changes
2. Consider alternative approaches to image handling
3. Test persistence implementation in isolation before full integration
4. Ensure screenshot display remains functional throughout changes

Files Modified During Attempt

• ScreenshotManager.swift - Added auto-save functionality
• ActionRecorder.swift - Added screenshot linkage and locale tracking
• ActionRecorderView.swift - Integrated screenshot capture during recording
• RecordingDetailsView.swift - Added windowMoved case handling
• RecordingAnalytics.swift - Added windowMoved case to analytics

All changes have been reverted pending resolution of the display issue.