Code

[WIP] AI/LLM integration#1325

Code

Draft

perfectra1n wants to merge 149 commits intodevelopTriliumNext/Notes:developfrom ai-llm-integrationTriliumNext/Notes:ai-llm-integrationCopy head branch name to clipboard

Commits

Commits on Mar 3, 2025

• start from scratch again

  perfectra1n

  committed
  e09e15a

  Copy full SHA for e09e15a

• hey look, it doesn't crash again

  perfectra1n

  committed
  f2a6f92

  Copy full SHA for f2a6f92

Commits on Mar 8, 2025

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  committed
  9f84a84

  Copy full SHA for 9f84a84

• create embedding services

  perfectra1n

  committed
  b248a7a

  Copy full SHA for b248a7a

• set up DB migrations

  perfectra1n

  committed
  b97c8dd

  Copy full SHA for b97c8dd

• set up embedding providers here?

  perfectra1n

  committed
  1ff5bc6

  Copy full SHA for 1ff5bc6

• add additional AI / LLM options and translations

  perfectra1n

  committed
  c442943

  Copy full SHA for c442943

• set up embedding API endpoints

  perfectra1n

  committed
  1361e4d

  Copy full SHA for 1361e4d

• initialize embeddings if option is enabled

  perfectra1n

  committed
  ea6f9c8

  Copy full SHA for ea6f9c8

• add additional options for ollama embeddings

  perfectra1n

  committed
  d3013c9

  Copy full SHA for d3013c9

• fix the Ollama embedding model setting option breaking

  perfectra1n

  committed
  553f7dd

  Copy full SHA for 553f7dd

• update schema with our new tables

  perfectra1n

  committed
  dc439b2

  Copy full SHA for dc439b2

• nearly able to process embeddings

  perfectra1n

  committed
  6ace4d5

  Copy full SHA for 6ace4d5

• I can create embeddings now?

  perfectra1n

  committed
  0daa9e7

  Copy full SHA for 0daa9e7

• Show embedding generation stats to user

  perfectra1n

  committed
  0cd1be5

  Copy full SHA for 0cd1be5

• update embedding stats every 5s for user

  perfectra1n

  committed
  1ca98e2

  Copy full SHA for 1ca98e2

• show fancier stats

  perfectra1n

  committed
  51c83bb

  Copy full SHA for 51c83bb

• fancier embedding process stats

  perfectra1n

  committed
  19bf741

  Copy full SHA for 19bf741

• Create better relationships between notes, sanitize ridiculous spacing to save tokens

  perfectra1n

  committed
  7e232d1

  Copy full SHA for 7e232d1

• update relationship weights

  perfectra1n

  committed
  733fdcf

  Copy full SHA for 733fdcf

Commits on Mar 9, 2025

• I'm 100% going to have to destroy this commit later

  perfectra1n

  committed
  adaac46

  Copy full SHA for adaac46

Commits on Mar 10, 2025

• try a context approach

  perfectra1n

  committed
  cf0e924

  Copy full SHA for cf0e924

• it errors, but works

  perfectra1n

  committed
  ef6ecdc

  Copy full SHA for ef6ecdc

• actually shows useful responses now

  perfectra1n

  committed
  c1585c7

  Copy full SHA for c1585c7

• Make the sources section fancier

  perfectra1n

  committed
  75e18e4

  Copy full SHA for 75e18e4

• this is pretty close to opening a new tab?

  perfectra1n

  committed
  bd97d97

  Copy full SHA for bd97d97

• when a user clicks on a source, don't swap focus

  perfectra1n

  committed
  08626c7

  Copy full SHA for 08626c7

• Update the chat panel theme some

  perfectra1n

  committed
  c386e34

  Copy full SHA for c386e34

• do a better job of extracting context

  perfectra1n

  committed
  f482b3b

  Copy full SHA for f482b3b

• fix context logic

  perfectra1n

  committed
  9834e77

  Copy full SHA for 9834e77

• almost completely styled codeblocks in response

  perfectra1n

  committed
  ecc183f

  Copy full SHA for ecc183f

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  authored
  d713f38

  Copy full SHA for d713f38

Commits on Mar 11, 2025

• add these options as configurable

  perfectra1n

  committed
  d2dc401

  Copy full SHA for d2dc401

• update checkbox options in settings and update translations

  perfectra1n

  committed
  d413e60

  Copy full SHA for d413e60

• move providers to their own folder

  perfectra1n

  committed
  ff679b0

  Copy full SHA for ff679b0

• undo accidental MAX_ALLOWED_FILE_SIZE_MB change

  perfectra1n

  committed
  56fc720

  Copy full SHA for 56fc720

• fancier (but longer waiting time) messages

  perfectra1n

  committed
  4160db9

  Copy full SHA for 4160db9

• implement chunking and use becca for some functionality

  perfectra1n

  committed
  0985cec

  Copy full SHA for 0985cec

• break up the huge context_extractor into smaller files

  perfectra1n

  committed
  71b3b04

  Copy full SHA for 71b3b04

• I think this works to handle failed embeddings

  perfectra1n

  committed
  f47b070

  Copy full SHA for f47b070

• make sure to not retry chunks if they fail or something else

  perfectra1n

  committed
  1f661e4

  Copy full SHA for 1f661e4

• better note names to LLM?

  perfectra1n

  committed
  6ce3f1c

  Copy full SHA for 6ce3f1c

• upgrade chunking

  perfectra1n

  committed
  0d2858c

  Copy full SHA for 0d2858c

• add additional options for users

  perfectra1n

  committed
  3f37196

  Copy full SHA for 3f37196

• create llm index service

  perfectra1n

  committed
  730d123

  Copy full SHA for 730d123

• allow users to manually request index to be rebuilt

  perfectra1n

  committed
  fc55995

  Copy full SHA for fc55995

Commits on Mar 12, 2025

• break up large vector_store into smaller files

  perfectra1n

  committed
  72b1426

  Copy full SHA for 72b1426

• "rebuild index" functionality for users

  perfectra1n

  committed
  eaa947e

  Copy full SHA for eaa947e

• allow for manual index rebuild, and ONLY rebuild the index

  perfectra1n

  committed
  fcba151

  Copy full SHA for fcba151

• better manage chunking errors

  perfectra1n

  committed
  e5afbc6

  Copy full SHA for e5afbc6

• update chunking management

  perfectra1n

  committed
  46a6533

  Copy full SHA for 46a6533

• move chunking to its own folder

  perfectra1n

  committed
  73445d9

  Copy full SHA for 73445d9

• rename files with the same name

  perfectra1n

  committed
  3fee82e

  Copy full SHA for 3fee82e

• allow user to select *where* they want to generate embeddings

  perfectra1n

  committed
  b6df3a7

  Copy full SHA for b6df3a7

• synchronize embeddings

  perfectra1n

  committed
  a930b79

  Copy full SHA for a930b79

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  authored
  39d265a

  Copy full SHA for 39d265a

• do a better job of handling failed note embeddings

  perfectra1n

  committed
  c914aaa

  Copy full SHA for c914aaa

• correctly style the failed embeddings section

  perfectra1n

  committed
  ee7b228

  Copy full SHA for ee7b228

• yes, now the failed embeddings section at least looks passable

  perfectra1n

  committed
  d4cfc65

  Copy full SHA for d4cfc65

• fix the maths for stats

  perfectra1n

  committed
  8d7e5c8

  Copy full SHA for 8d7e5c8

• create note_embedding object for becca

  perfectra1n

  committed
  67766e3

  Copy full SHA for 67766e3

• specially handle Buffer objects into Base64 and back for Becca

  perfectra1n

  committed
  6bb4bbb

  Copy full SHA for 6bb4bbb

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  committed
  4796c24

  Copy full SHA for 4796c24

Commits on Mar 13, 2025

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  committed
  f8d4088

  Copy full SHA for f8d4088

Commits on Mar 15, 2025

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  committed
  c556989

  Copy full SHA for c556989

Commits on Mar 16, 2025

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  committed
  572a03a

  Copy full SHA for 572a03a

• set up more reasonable context window and dimension sizes

  perfectra1n

  committed
  697d348

  Copy full SHA for 697d348

• wait for DB init even to emit before starting LLM services

  perfectra1n

  committed
  c315b32

  Copy full SHA for c315b32

• move these settings between db migrations

  perfectra1n

  committed
  d8c9d3b

  Copy full SHA for d8c9d3b

• fix sql error and add missing options

  perfectra1n

  committed
  0081e6f

  Copy full SHA for 0081e6f

• fix embeddings w/ cls.init()

  perfectra1n

  committed
  781a250

  Copy full SHA for 781a250

• "lock" notes that are having their embeddings created

  perfectra1n

  committed
  d2072c2

  Copy full SHA for d2072c2

• remove options from migrations files

  perfectra1n

  committed
  ab3c6b6

  Copy full SHA for ab3c6b6

• fix autoupdate name inconsistency

  perfectra1n

  committed
  cc85b9a

  Copy full SHA for cc85b9a

• fix(llm): duplicate launch bar config

  eliandoran

  committed
  7b643a7

  Copy full SHA for 7b643a7

• refactor(llm): use dedicated widget for llm chat button

  eliandoran

  committed
  d716713

  Copy full SHA for d716713

• feat(llm): show/hide LLM button based on setting

  eliandoran

  committed
  5d0be30

  Copy full SHA for 5d0be30

• fix(settings/llm): extra separators

  eliandoran

  committed
  1dafa65

  Copy full SHA for 1dafa65

Commits on Mar 17, 2025

• feat(options/llm): use tabs for provider configuration

  eliandoran

  committed
  2853b8e

  Copy full SHA for 2853b8e

• feat(options/llm): group into sections

  eliandoran

  committed
  36a6d75

  Copy full SHA for 36a6d75

• feat(options/llm): improve checkboxes

  eliandoran

  committed
  5a6d271

  Copy full SHA for 5a6d271

• feat(options/llm): use form text style

  eliandoran

  committed
  e6cb06b

  Copy full SHA for e6cb06b

• feat(options/llm): use columns and separators

  eliandoran

  committed
  1efc923

  Copy full SHA for 1efc923

• feat(options/llm): move stats at the top

  eliandoran

  committed
  fa99624

  Copy full SHA for fa99624

• feat(options/llm): improve style of tabs

  eliandoran

  committed
  0ea7e10

  Copy full SHA for 0ea7e10

• fix the isEnabled function

  perfectra1n

  committed
  1844ad7

  Copy full SHA for 1844ad7

• show user at the top of settings if there are issues

  perfectra1n

  committed
  fe1faf7

  Copy full SHA for fe1faf7

• also add the errors to the top of the chat window

  perfectra1n

  committed
  79514b8

  Copy full SHA for 79514b8

• fancier LLM/AI chat errors

  perfectra1n

  committed
  8d8c34c

  Copy full SHA for 8d8c34c

• fix openai endpoints

  perfectra1n

  committed
  5aef80f

  Copy full SHA for 5aef80f

• try ollama first, always

  perfectra1n

  committed
  6d146c2

  Copy full SHA for 6d146c2

• allow specifying openai embedding models too

  perfectra1n

  committed
  d95fd0b

  Copy full SHA for d95fd0b

• Allow users to specify OpenAI embedding and chat models

  perfectra1n

  committed
  4a4eac6

  Copy full SHA for 4a4eac6

• add anthropic options as well

  perfectra1n

  committed
  c40c702

  Copy full SHA for c40c702

• improve LLM response parsing

  perfectra1n

  committed
  14acd1c

  Copy full SHA for 14acd1c

• add additional options and provider sorting

  perfectra1n

  committed
  7ee6cf6

  Copy full SHA for 7ee6cf6

• add ability to fetch available models from openai

  perfectra1n

  committed
  37f1dcd

  Copy full SHA for 37f1dcd

• improve embedding precedence

  perfectra1n

  committed
  3268c43

  Copy full SHA for 3268c43

• add missing options

  perfectra1n

  committed
  ebc5107

  Copy full SHA for ebc5107

• make the AI settings even fancier to setting precedence

  perfectra1n

  committed
  43cf33c

  Copy full SHA for 43cf33c

• draggable options for LLM provider too

  perfectra1n

  committed
  ac40fff

  Copy full SHA for ac40fff

• openai finally works, respect embedding precedence

  perfectra1n

  committed
  5ad730c

  Copy full SHA for 5ad730c

• adapt or regenerate embeddings - allows users to decide

  perfectra1n

  committed
  84a8473

  Copy full SHA for 84a8473

• add translations

  perfectra1n

  committed
  558f6a9

  Copy full SHA for 558f6a9

• add Voyage AI as Embedding provider

  perfectra1n

  committed
  c372011

  Copy full SHA for c372011

Commits on Mar 18, 2025

• do a better job with similarity searches

  perfectra1n

  committed
  08f7f19

  Copy full SHA for 08f7f19

• set up embedding normalization

  perfectra1n

  committed
  f05fe3f

  Copy full SHA for f05fe3f

• oh my goodness, saving these settings finally works

  perfectra1n

  committed
  8129f8f

  Copy full SHA for 8129f8f

• stop the log spam

  perfectra1n

  committed
  1a8ce96

  Copy full SHA for 1a8ce96

Commits on Mar 19, 2025

• set up agentic thinking

  perfectra1n

  committed
  f6afb1d

  Copy full SHA for f6afb1d

• clean up silly chat_widget that was in the wrong place

  perfectra1n

  committed
  492c05b

  Copy full SHA for 492c05b

• add agentic thinking to chat

  perfectra1n

  committed
  352204b

  Copy full SHA for 352204b

• refactor "context" services

  perfectra1n

  committed
  db4dd6d

  Copy full SHA for db4dd6d

• yeet deprecated function

  perfectra1n

  committed
  466b749

  Copy full SHA for 466b749

• fix chat_service imports

  perfectra1n

  committed
  d5efcfe

  Copy full SHA for d5efcfe

• get rid of this unused file too

  perfectra1n

  committed
  2348096

  Copy full SHA for 2348096

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  authored
  024b063

  Copy full SHA for 024b063

• fix translation

  perfectra1n

  committed
  5b81252

  Copy full SHA for 5b81252

• update agent tools

  perfectra1n

  committed
  0d4b6a7

  Copy full SHA for 0d4b6a7

• agent tools do something now

  perfectra1n

  committed
  90db570

  Copy full SHA for 90db570

• agentic thinking really works now 🗿

  perfectra1n

  committed
  4ff3c5a

  Copy full SHA for 4ff3c5a

Commits on Mar 20, 2025

• centralize all prompts

  perfectra1n

  committed
  e566692

  Copy full SHA for e566692

• move the llm_prompt_constants to its own folder

  perfectra1n

  committed
  eb1ef36

  Copy full SHA for eb1ef36

• create a better base system prompt

  perfectra1n

  committed
  273dff2

  Copy full SHA for 273dff2

• add to base prompt

  perfectra1n

  committed
  9c1ab4f

  Copy full SHA for 9c1ab4f

• do a better job of building the context

  perfectra1n

  committed
  1be70f1

  Copy full SHA for 1be70f1

• more aggressively filter notes out that don't work for us

  perfectra1n

  committed
  915c95f

  Copy full SHA for 915c95f

• also extract Note relationships and send as context

  perfectra1n

  committed
  c9728e7

  Copy full SHA for c9728e7

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  committed
  34940b5

  Copy full SHA for 34940b5

• appropriately show if there are any notes still in the queue

  perfectra1n

  committed
  3d70a6c

  Copy full SHA for 3d70a6c

• reset embedding_queue where objects are "isprocessing"

  perfectra1n

  committed
  0707266

  Copy full SHA for 0707266

Commits on Mar 24, 2025

• remove isEnabled from embedding providers

  perfectra1n

  committed
  150b0f0

  Copy full SHA for 150b0f0

• Remove the drag-and-drop for settings, kept breaking

  perfectra1n

  committed
  567e9e8

  Copy full SHA for 567e9e8

• don't spam the logs if a provider isn't enabled

  perfectra1n

  committed
  9d29ff4

  Copy full SHA for 9d29ff4

• Merge branch 'develop' into ai-llm-integration

  perfectra1n

  committed
  b00c20c

  Copy full SHA for b00c20c

• might have to delete this later, fixing the right-pane-container

  perfectra1n

  committed
  f1ecc15

  Copy full SHA for f1ecc15

• yerp that was it

  perfectra1n

  committed
  3534399

  Copy full SHA for 3534399

• fix embedding provider precedence settings issue

  perfectra1n

  committed
  654ed47

  Copy full SHA for 654ed47

Commits on Mar 26, 2025

• anthropic works

  perfectra1n

  committed
  44b6734

  Copy full SHA for 44b6734

• move constants to their own files and folder

  perfectra1n

  committed
  c49883f

  Copy full SHA for c49883f

• move more prompts to the constants file

  perfectra1n

  committed
  a50575c

  Copy full SHA for a50575c

• move more constants from files into centralized location

  perfectra1n

  committed
  5869eaf

  Copy full SHA for 5869eaf

• move providers.ts into providers folder

  perfectra1n

  committed
  7138053

  Copy full SHA for 7138053

• fix prompt path import

  perfectra1n

  committed
  7c519df

  Copy full SHA for 7c519df

• add swaggerUI docstrings for LLM/AI API routes

  perfectra1n

  committed
  15630fb

  Copy full SHA for 15630fb

• fix updateProvider parameter

  perfectra1n

  committed
  baef5f9

  Copy full SHA for baef5f9

• Merge branch 'develop' into ai-llm-integration

  eliandoran

  authored
  35fbc73

  Copy full SHA for 35fbc73

• more heavily weigh notes with title matches when giving context to LLM

  perfectra1n

  committed
  a7cafce

  Copy full SHA for a7cafce

• set up embedding similarity constants and similarity system

  perfectra1n

  committed
  5456ac3

  Copy full SHA for 5456ac3