Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
826d424
New Feature: Food Search 2.0 for Loop!
taylorpatterson-T1D Jul 20, 2025
bf0185a
Delete Documentation/FoodSearch 2.0 Docs/Development 1.0 Docs directory
taylorpatterson-T1D Jul 20, 2025
e9adb3a
Update project.pbxproj to ver 54
taylorpatterson-T1D Jul 21, 2025
a219b86
Delete Loop/DefaultAssets.xcassets/AI-logo-master.png
taylorpatterson-T1D Jul 21, 2025
6f6569f
Added SupportsLiveActivities to Info.plist
taylorpatterson-T1D Jul 21, 2025
42a1f47
Merge branch 'dev' of https://github.com/taylorpatterson-T1D/Loop.Foo…
taylorpatterson-T1D Jul 21, 2025
cad0593
Fix for build error on AbsorptionTimePickerRow and pbxproj file
taylorpatterson-T1D Jul 21, 2025
c850319
Added delete food item Food Details list
taylorpatterson-T1D Jul 22, 2025
23edb98
Delete Loop/Base.lproj/Main.storyboard
taylorpatterson-T1D Jul 23, 2025
ba53e75
Revert "Added SupportsLiveActivities to Info.plist"
taylorpatterson-T1D Jul 23, 2025
2dab11b
Moved isAbsorptionTimePickerRow change from /LoopKitUI back under /Loop
taylorpatterson-T1D Aug 8, 2025
a7893f4
BugFix for After AI scan, adjusting Servings does not update multi-ci…
taylorpatterson-T1D Aug 8, 2025
c95c24d
Adds a method to enable Food Search to Add Carb Entry View
taylorpatterson-T1D Aug 8, 2025
3df7bb5
AI Prompt performance improvement for Standard users
taylorpatterson-T1D Aug 8, 2025
5609b98
Bugfix: Nutriments not accurately updated when deleting food items fr…
taylorpatterson-T1D Aug 8, 2025
477ca57
Refined AI prompts, added GPT 5 as option for AI.
taylorpatterson-T1D Aug 10, 2025
97bb8d0
Set search provider defaults to best ones at onboarding
taylorpatterson-T1D Aug 11, 2025
a83d46d
Fix width issue in the Add Carb Entry screen
taylorpatterson-T1D Aug 15, 2025
509619d
Revert "Fix width issue in the Add Carb Entry screen"
taylorpatterson-T1D Aug 15, 2025
bfd1c9d
Fix width bug and rearrange Add Carb Entry view
taylorpatterson-T1D Aug 15, 2025
2c6a78e
Fixed camera capture misalignments
taylorpatterson-T1D Aug 15, 2025
5aeda3b
Bugfixes - Favorite Foods and math fix for Carb Entry
taylorpatterson-T1D Aug 29, 2025
5cbb5d3
Multiple feature UI improvements
taylorpatterson-T1D Aug 29, 2025
ab9bfcf
FoodFinder Final Commit
taylorpatterson-T1D Sep 15, 2025
f97bd48
delete file with test keys
taylorpatterson-T1D Sep 15, 2025
772e37a
Revert "delete file with test keys"
taylorpatterson-T1D Sep 15, 2025
714c700
Update BYOTestConfig.swift
taylorpatterson-T1D Sep 15, 2025
7044f13
Delete LoopTests/FoodSearchIntegrationTests.swift
taylorpatterson-T1D Sep 17, 2025
d6ca433
Update AISettingsView.swift
taylorpatterson-T1D Sep 19, 2025
e8dca9a
UI/UX and prompt Improvements
taylorpatterson-T1D Sep 21, 2025
2cbdc4e
UI/UX and prompt Improvements
taylorpatterson-T1D Sep 21, 2025
f090fdd
WE ARE DONE! (FINAL FoodFinder COMMIT?)
taylorpatterson-T1D Sep 21, 2025
4e3f015
Merge branch 'dev' into feature/AutoPresets
taylorpatterson-T1D Sep 22, 2025
197ffd6
Merge pull request #1 from taylorpatterson-T1D/feature/AutoPresets
taylorpatterson-T1D Sep 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -86,3 +86,4 @@ Loop\ Widget\ Extension/DerivedAssets.xcassets/*
Loop/DerivedAssetsOverride.xcassets
WatchApp/DerivedAssetsOverride.xcassets
Loop\ Widget\ Extension/DerivedAssetsOverride.xcassets
Loop.xcodeproj/project.pbxproj
35 changes: 35 additions & 0 deletions Documentation/FoodFinder Docs/01_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# FoodFinder Documentation Hub

FoodFinder brings AI-assisted carb discovery, nutritional lookups, and diabetes-oriented analysis to Loop. This folder gathers everything you need—from wiring the feature into code to helping someone use it safely in day-to-day therapy.

## How To Use These Docs

| Audience | Start Here | Why |
| --- | --- | --- |
| Curious user / caregiver | [04_End User Guide] | Walks through enabling FoodFinder, running searches, and interpreting AI results. |
| Power user configuring providers | [02_Configuration and Settings] | Explains every toggle, provider option, and when to use each choice. |
| Developers & contributors | [03_Technical Implementation Guide] | Details the architecture, key classes, and test touch points. |
| Support / troubleshooting | [05_Troubleshooting Guide] | Symptoms → causes → fixes for the most common questions. |

## Quick Facts

- **Feature name**: FoodFinder (settings alias `foodFinderEnabled` → `foodSearchEnabled`)
- **Entry points**: Carb Entry screen search bar (text + barcode + AI camera) and the FoodFinder Settings card
- **AI providers**: OpenAI, Anthropic Claude, Google Gemini, or a custom (BYO) OpenAI-compatible endpoint
- **Local data**: Favorites, cached analysis, and settings all live on-device; nothing is sent to Loop servers
- **Advanced dosing**: Optional, off by default; adds FPU, fiber adjustments, exercise notes, timing guidance, and safety alerts

## For Developers in a Hurry

1. Skim the [Technical Implementation Guide] to see how `CarbEntryView`, `FoodSearchRouter`, `ConfigurableAIService`, and `AIFoodAnalysis` fit together.
2. Review `LoopWorkspace/Loop/Loop/View Models/CarbEntryViewModel.swift` for state flows (search text, barcode streaming, AI results, favorites).
3. Tests live under `LoopWorkspace/Loop/LoopTests/FoodSearchIntegrationTests.swift` with helper mocks in `LoopKitUI`.
4. Provider prompts & caching logic live in `Services/AIFoodAnalysis.swift`—if you change output fields, update the docs and UI.

## For Support & Educators

- Point new users to the [End User Guide](04_End%20User%20Guide.md).
- Use the [Configuration](02_Configuration%20and%20Settings.md) doc when helping someone wire API keys or USDA access.
- Reference the [Troubleshooting Guide](05_Troubleshooting%20Guide.md) for copy/paste ready answers about API failures, long-running analyses, or missing advanced sections.

Happy searching! If you spot mismatches between docs and UI, file an issue or PR so everything stays in sync.
116 changes: 116 additions & 0 deletions Documentation/FoodFinder Docs/02_Configuration and Settings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# FoodFinder Configuration Guide

This guide walks through every toggle and text field exposed in FoodFinder’s settings so both new users and developers can configure the feature with confidence.

---

## 1. Opening FoodFinder Settings

1. Launch the **Loop** app.
2. Tap the **Settings** tab (gear icon).
3. Scroll to the **FoodFinder** tile under *Services* and tap it.

You’ll land on the FoodFinder summary screen. From here you can:
- **Enable/disable FoodFinder** with a master toggle.
- See quick status for Text Search, Barcode, and AI Analysis.
- Jump into **AI Settings** for detailed configuration.

> **Tip for testers:** The FoodFinder toggle maps to `UserDefaults.standard.foodSearchEnabled` (`foodFinderEnabled` is an alias). It’s safe to enable/disable without losing saved favorites or API keys.

---

## 2. AI Settings Screen Tour

The **AI Settings** sheet hosts the full configuration surface. It is composed of stacked sections. Press **Save** (top-right) when you finish making changes; the view dismisses automatically.

### 2.1 Feature Toggle & Overview

- **Enable FoodFinder** – master switch identical to the summary screen. Disabling hides search UI but keeps all preferences.
- Contextual footers explain that disabling preserves API keys and favorites.

### 2.2 Provider Mapping

FoodFinder can mix and match providers for each search type. The “FoodFinder Provider Configuration” section contains three pickers, each backed by `ConfigurableAIService`:

| Search Type | Options | Notes |
| --- | --- | --- |
| **Text / Voice Search** | OpenFoodFacts (default), USDA FoodData Central, OpenAI, Claude, Gemini, Bring Your Own | AI providers fall back to USDA/OFF when API keys are missing. Use USDA for better brand coverage in the U.S. |
| **Barcode Search** | OpenFoodFacts (default), OpenAI, Claude, Gemini, Bring Your Own | Non-database providers currently fall back to OpenFoodFacts internally; keep OFF as primary for best results. |
| **AI Image Analysis** | OpenAI, Claude, Google Gemini, Bring Your Own, (fallback: Basic Analysis) | Determines which vision model handles food photos. |

Selections persist to `UserDefaults` immediately.

### 2.3 AI Provider Cards

A segmented control switches between bundled providers:

- **OpenAI ChatGPT** – stores API key & optional custom prompt text. Toggle “Use GPT-5 Models” if you have access; otherwise GPT-4o is used.
- **Anthropic Claude** – configure key and optional system prompt.
- **Google Gemini** – configure key and optional custom query instructions.
- **Bring Your Own (BYO)** – supply base URL, key, model/deployment, optional API version, organization, and custom path for OpenAI-compatible endpoints (including Azure OpenAI). A built-in “Test connection” button checks authentication.

Keys are saved into the iOS Keychain. Clear a field to remove the stored value. Show/hide buttons (`eye` icons) toggle secure entry.

### 2.4 USDA API Key (optional)

FoodFinder uses USDA FoodData Central for richer text results. Without a personal key it falls back to the public “DEMO_KEY”, which quickly hits rate limits. The **USDA Database** section links directly to the signup page and stores the key in `UserDefaults.standard.usdaAPIKey`.

### 2.5 Analysis Mode

Choose between:
- **Standard Quality** – highest accuracy (GPT‑4o or GPT‑5 + Gemini 1.5 Pro + Claude Sonnet). Best for mixed meals.
- **Fast Mode** – uses lighter models (GPT‑4o-mini/GPT‑5-nano, Gemini Flash). ~2–3× quicker with a small accuracy trade-off.

Model names for each provider are displayed so developers know which endpoints are hit.

### 2.6 Advanced Options

- **Advanced Dosing Recommendations** – when ON, prompts include extra JSON fields: FPU calculations, fiber impact, exercise notes, absorption timing rationale, safety alerts, and more. The UI exposes these in Carb Entry’s “Advanced Analysis” section. Toggle writes to `UserDefaults.standard.advancedDosingRecommendationsEnabled`.

### 2.7 Save & Cancel

- **Save** – commits all changes (API keys, toggles, custom endpoints) and dismisses the sheet.
- **Cancel** – restores previously persisted values and dismisses without saving.

---

## 3. Standard Behaviour Summary

| Setting | Default | Notes |
| --- | --- | --- |
| Enable FoodFinder | OFF | Keeps existing manual carb entry untouched until you opt in. |
| Text Search Provider | OpenFoodFacts | Switching to USDA improves U.S. branded coverage. |
| Barcode Search Provider | OpenFoodFacts | Other entries currently fall back to OFF. |
| AI Image Provider | Google Gemini | Chosen for speed/cost balance; change as desired. |
| Advanced Dosing | OFF | Turn on once you’re comfortable with FPU-style guidance. |
| GPT‑5 models | OFF | Visible only on the OpenAI tab. Requires paid access. |

---

## 4. FoodFinder Extras

### Favorite Foods Manager
- Accessible from **Settings → Favorite Foods** or directly in Carb Entry after analysis.
- Stores `StoredFavoriteFood` objects locally alongside optional emoji thumbnails.
- Editing honors the same truncation/emoji logic used in Carb Entry.

### Cache & Telemetry
- Image analysis responses are cached for five minutes (see `ImageAnalysisCache`).
- Carb Entry view shows inline telemetry strings during camera analysis for debugging.
- No user-facing switches are required; caches clear automatically.

### Safety & Privacy
- API calls send food descriptions/photos only; Loop never uploads therapy data or identifiers.
- All keys live in Keychain; removing a provider key disables that integration immediately.
- FoodFinder can be disabled any time without losing stored favorites or configurations.

---

## 5. Developer Callouts

- Changes to prompts or response schema belong in `Services/AIFoodAnalysis.swift`; update docs and UI simultaneously.
- Provider bindings live in `ConfigurableAIService`. Adjust defaults there if you need different initial behaviour for forks.
- Keep README screenshots and this guide aligned with Settings UI changes.
- Test flows by running `FoodSearchIntegrationTests` and manual camera/text searches.

FoodFinder’s settings aim to balance power and clarity.
137 changes: 137 additions & 0 deletions Documentation/FoodFinder Docs/03_Technical Implementation Guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# FoodFinder Technical Implementation Guide

This reference is aimed at contributors who want to understand how FoodFinder is wired together inside Loop. It highlights the key modules, control flow, and extension points for adding new functionality.

---

## 1. High-Level Architecture

```
┌──────────────┐ ┌──────────────────┐ ┌──────────────────────┐
│ UI & ViewModels│ ───▶ │ Services │ ───▶ │ External Providers │
└──────────────┘ └──────────────────┘ └──────────────────────┘
│ │ │
▼ ▼ ▼
CarbEntryView FoodSearchRouter OpenFoodFacts / USDA
FoodSearchBar ConfigurableAIService OpenAI / Claude / Gemini / BYO
FoodFinderSettings AIFoodAnalysis & Cache Device Camera / Barcode
AICameraView OpenFoodFactsService
FavoriteFoods views BarcodeScannerService
```

- **UI Layer** (SwiftUI) presents search controls, results, advanced analysis, and configuration screens.
- **Services** coordinate data fetching, AI prompt generation, caching, and provider selection.
- **External Providers** include public food databases and user-specified AI services accessed via HTTPS.

---

## 2. Key Modules

### 2.1 UI / ViewModels

| Component | Path | Notes |

| `CarbEntryView` | `Loop/Views/CarbEntryView.swift` | Hosts the search bar, results list, advanced analysis foldout, and favorite food integration. |
| `CarbEntryViewModel` | `Loop/View Models/CarbEntryViewModel.swift` | Owns search text, barcode publisher, AI analysis results, caching of favorites, and absorption time logic. |
| `FoodSearchBar` & `FoodSearchResultsView` | `Loop/Views` | Provide search input plus animated feedback for loading, empty states, and errors. |
| `AddEditFavoriteFoodView` (+ VM) | `Loop/Views` / `View Models` | Handles saving favorites, enforcing truncation, and emoji mapping. |
| `FavoriteFoodsView` | `Loop/Views/FavoriteFoodsView.swift` | Lists stored favorites, supports reordering, editing, and deletion. |
| `FoodFinderSettingsView` | `Loop/Views/FoodFinderSettingsView.swift` | Summary screen with master toggle and quick status. |
| `AISettingsView` | `Loop/Views/AISettingsView.swift` | Full configuration UI for providers, API keys, analysis modes, and advanced features. |
| `AICameraView` | `Loop/Views/AICameraView.swift` | Guides users through capturing food photos, streams telemetry as analysis progresses. |
| `VoiceSearchView` | `Loop/Views/VoiceSearchView.swift` | Optional sheet driven by `VoiceSearchService` (not currently shown in the default UI but available for future use). |

### 2.2 Services & Support Code

| Service | Path | Responsibilities |

| `ConfigurableAIService` | `Loop/Services/AIFoodAnalysis.swift` | Stores provider choices for each search type, exposes API key/query bindings, resolves analysis mode, and pre-encodes images for reuse. |
| `FoodSearchRouter` | `Loop/Services/FoodSearchRouter.swift` | Routes text, barcode, and image requests to the configured provider with fallbacks (e.g., USDA → OpenFoodFacts). |
| `AIFoodAnalysis` | `Loop/Services/AIFoodAnalysis.swift` | Builds prompts, calls providers, parses JSON into `AIFoodAnalysisResult`, and manages caching (`ImageAnalysisCache`). Handles OpenAI, Claude, Gemini, and BYO flows including error translation. |
| `OpenFoodFactsService` | `Loop/Managers/OpenFoodFactsService.swift` | Wraps REST calls to OpenFoodFacts with retry logic and network tuning. |
| `USDAFoodDataService` | `Loop/Services/AIFoodAnalysis.swift` | Text-search helper using USDA FoodData Central when available. |
| `BarcodeScannerService` | `Loop/Services/BarcodeScannerService.swift` | Vision-based barcode detection and deduplication. |
| `VoiceSearchService` | `Loop/Services/VoiceSearchService.swift` | Speech recognition pipeline (authorization, audio capture, error handling). |
| `EmojiThumbnailProvider` & `FavoriteFoodImageStore` | `Loop/Services` | Generate emoji thumbnails and persist them for favorites. |

### 2.3 Data Models

- `OpenFoodFactsProduct` (`Loop/Models/OpenFoodFactsModels.swift`) – parsed database objects with helper properties.
- `AIFoodAnalysisResult` & `FoodItemAnalysis` (`AIFoodAnalysis.swift`) – normalized AI output consumed by UI.
- `SearchProvider` / `SearchType` enums – shared across services and settings.
- `StoredFavoriteFood` (LoopKit) – persisted favorite entries synced with UI.

---

## 3. Data Flow Breakdown

1. **User input** (typing, barcode, camera) updates `CarbEntryViewModel`.
2. View model delegates to `FoodSearchRouter` which chooses the appropriate provider, using cached data if available.
3. When AI analysis is required, `ConfigurableAIService` prepares pre-encoded image data and prompt text via `getAnalysisPrompt()`.
4. `AIFoodAnalysis` issues async network requests using the selected provider client. It automatically falls back (e.g., GPT-5 → GPT-4o) when throttling or errors occur.
5. Responses are parsed into `AIFoodAnalysisResult` where advanced fields are optional unless `advancedDosingRecommendationsEnabled` is true.
6. View model updates published properties; SwiftUI refreshes nutrition circles, advanced sections, favorites, and telemetry.

---

## 4. Advanced Prompt Handling

- `getAnalysisPrompt()` returns `standardAnalysisPrompt + mandatoryNoVagueBlock` and appends `advancedAnalysisRequirements` when advanced dosing is enabled.
- Advanced prompts demand JSON output with additional keys (`fat_protein_units`, `insulin_timing_recommendations`, `absorption_time_hours`, etc.).
- If you add or remove fields, update:
- Prompt constants in `AIFoodAnalysis.swift`
- Parsing logic in `parseOpenAIResponse`, `parseClaudeAnalysis`, `parseGeminiAnalysis`
- UI bindings in `CarbEntryView` and `AICameraView`
- Documentation

---

## 5. Caching & Resilience

- **Image cache**: `ImageAnalysisCache` stores AI results keyed by SHA-256 of the pre-encoded image + provider ID for five minutes.
- **Search cache**: `CarbEntryViewModel` caches text search results for five minutes to avoid repeat HTTP calls.
- **Barcode dedupe**: `BarcodeScannerService` throttles identical scans to prevent repeated lookups.
- **Timeout wrapper**: `withTimeoutForAnalysis` guards long-running tasks (default 25–45 seconds based on network quality).

Error enums translate provider responses into localized strings so UI can show actionable messages (missing API key, 429 rate limits, parsing failures, etc.).

---

## 6. Testing & Debugging

- `FoodSearchIntegrationTests` exercise text search, barcode lookups, and selection flows with mocked services (`OpenFoodFactsService.configureMockResponses()`).
- Additional unit tests live under `LoopTests/BarcodeScannerTests.swift` and `LoopTests/OpenFoodFactsTests.swift`.
- `FoodSearchBar` and `AICameraView` print verbose debug logs (`🔍`, `🤖`) to help trace issues while developing. Use Console.app or Xcode’s debug area.
- For manual testing, the **Telemetry** overlay in `AICameraView` displays each stage (image processing, upload, provider selection, etc.).

---

## 7. Extensibility Tips

1. **Adding a new provider**
- Extend `SearchProvider` and map defaults in `ConfigurableAIService`.
- Implement provider-specific network code (look at existing `OpenAIFoodAnalysisService` / `ClaudeFoodAnalysisService`).
- Update prompts if the provider requires different formatting.
- Wire through `AISettingsView` so users can supply API keys.

2. **Adjusting defaults**
- Modify `ConfigurableAIService` init to set new default providers or analysis mode.
- Update docs and release notes for users.

3. **Custom prompt tweaks**
- Keep the shared `mandatoryNoVagueBlock` aligned across providers.
- Ensure JSON schema remains backwards compatible with existing UI fields.

4. **Favorites enhancements**
- `FavoriteFoodsViewModel` trims names and maps emojis. Respect `maxNameLength` constants when surfacing new fields.

---

## 8. Release Checklist

- Run the integration tests and manual camera/text searches with each provider.
- Verify advanced dosing JSON renders correctly when toggled on/off.
- Confirm FoodFinder Settings reflects new options and Save actually persists to `UserDefaults`/Keychain.
- Update this documentation folder and screenshots if the UI changes.

FoodFinder is tightly integrated with Loop’s carb entry workflow. Understanding the few core files above gives you everything you need to extend or debug the system without surprises. Happy hacking!
Loading