ISS-273 - System footprint #99

shide1989 · 2025-01-28T09:55:37Z

🚀 Pull Request Overview

What does this PR do? 🤔

add CLI System footprint

src/utils/systemInfo.ts

zhuk-aa · 2025-02-10T15:34:05Z

src/utils/systemInfo.ts

+ * Get the list of global packages installed on the system for mnetrics.
+ */
+function getGlobalPackages() {
+  try {


I think we should expand it to brew, apt and a few other most popular package managers.
Python version would also be very useful

What else could be useful to decrease the amount of iterations when it comes to CLI commands 2501 needs to run?

Not 100% sure apt is very interesting tbh, but brew, python, php?, rust?,ruby? why not

we could have information from utils like the tree utility :

why not apt?
https://askubuntu.com/questions/17823/how-to-list-all-installed-packages

yeah, tree could be interesting if we manage to:

list the main folders

cut the list down to X main folders

list full paths

apt : apt list --installed looks interesting indeed, I thought you wanted to have for ex the apt version
tree:

I use it often its handy, I've always had in mind to add it in the context somehow if relevant.

Since its a utility like JQ, we could code a function that would output the same kind of display. Its easy (already did in the past) and as bonus we could re-used the existing methods to retrieve workspace files to ignore common vendor folders etc. Which would output a righteously filtered list of files, with a preferably configured max depth.

wdyt ?

…ootprint

…pace files

shide1989 · 2025-02-12T16:54:07Z

@zhuk-aa I did a package manager call system that fetches user installed packages, since we dont need to know the common installed packages on the user's machine as the AI will surely know it already.

Using this approach gives less packages than output all the packages, which can lead to having 200+ items.
I also added a tree generation, its nice and neat.

Now it gives something like :

sysinfo {
  "sysInfo": {
    "platform": "darwin",
    "type": "Darwin",
    "release": "24.1.0",
    "arch": "arm64",
    "package_manager": "brew",
    "installed_packages": [
      "argocd",
      "automake",
      "aws-sam-cli",
      "aws/tap/copilot-cli",
      "awscli",
      "azure-cli",
      "base64",
      ...
      "sleepwatcher",
      "sqlmap",
      "terraform",
      "thefuck",
      "tree",
      "ubuntu/microk8s/microk8s",
      "wakeonlan",
      "wget",
      "yarn",
      "zig"
    ]
  },
  "nodeInfo": {
    "version": "v20.18.1",
    "config": {
      "target_defaults": {
        "cflags": [],
        "default_configuration": "Release",
        "defines": [
          "NODE_OPENSSL_CONF_NAME=nodejs_conf",
          "NODE_OPENSSL_HAS_QUIC",
          "ICU_NO_USER_DATA_OVERRIDE"
        ],
        "include_dirs": [],
        "libraries": []
      },
      "variables": {
        "host_arch": "arm64",
        "node_install_npm": true,
        "node_prefix": "/",
        "node_shared_cares": false,
        "node_shared_http_parser": false,
        "node_shared_libuv": false,
        "node_shared_openssl": false,
        "node_shared_zlib": false
      }
    },
    "packages": [
      "@2501-ai/[email protected]",
      "@2501-ai/[email protected]",
      "[email protected]",
      "[email protected]"
    ]
  }
}
workspaceFileTree 
.
└── express-app
    ├── .gitignore
    ├── app.js
    ├── package-lock.json
    └── package.json

wdyt ?

zhuk-aa · 2025-02-12T20:09:37Z

src/utils/tree.ts

+  const entries = Array.from(node.children.values());
+  const lastIndex = entries.length - 1;
+
+  entries.forEach((child, index) => {


my gut feeling: full paths would be a lot more clear for LLMs than this kind of visualization

generation logic would be simpler.

if later we'll need to change it to folders not nested inside the workspace it would be easier to do

wdyt?

you are definitely right, event Claude tells it.

Adding sysinfo adds around 800 tokens, and adding the filetree with relative path adds up another ~1500 tokens on engine for example:

src/validation/chatSchema.ts src/validation/configurationsSchema.ts src/validation/containerSchema.ts src/validation/fileSchema.ts src/validation/jobsSchema.ts tsconfig.build.json tsconfig.json types/express.d.ts types/stream.d.ts vercel.json [DEBUG][09:41:42.382] Final prompt tokens : 564 <- without any additions [DEBUG][09:41:42.385] Final prompt with sysinfo tokens: 1313 [DEBUG][09:41:42.392] Final prompt with workspaceFileTree tokens: 2791

src/utils/systemInfo.ts

zhuk-aa · 2025-02-12T20:21:34Z

src/utils/types.ts

+/**
+ * Collect system info while respecting privacy.
+ */
+export type SystemInfo = {


My thinking: it would be better to keep this type loose. So that we could add or remove info without changes in Engine (and without thinking of backwards compatibility)

{ system: string, // 'platform, type, release, arch' all in one string installed_packages: string[], // including node packages package_manager: string[], // we can skip adding it now if it's too much work nodeVersion: string }

but let's chat about it

src/utils/types.ts

shide1989 added 2 commits January 28, 2025 10:55

feat: WIP

4ebb915

feat: added system info retrieval

7475df5

shide1989 changed the title ~~System footprint~~ CLI - System footprint Feb 10, 2025

zhuk-aa reviewed Feb 10, 2025

View reviewed changes

src/utils/systemInfo.ts Outdated Show resolved Hide resolved

zhuk-aa reviewed Feb 10, 2025

View reviewed changes

src/utils/systemInfo.ts Show resolved Hide resolved

zhuk-aa reviewed Feb 10, 2025

View reviewed changes

shide1989 self-assigned this Feb 12, 2025

shide1989 changed the title ~~CLI - System footprint~~ ISS-273 - System footprint Feb 12, 2025

shide1989 added 2 commits February 12, 2025 10:45

Merge remote-tracking branch 'origin/main' into feature/iss-273-sys-f…

87fe993

…ootprint

feat: removed logs and added workspace tree generation based on works…

4c37b29

…pace files

feat: renamed a field for clarity

052a9b1

zhuk-aa reviewed Feb 12, 2025

View reviewed changes

src/utils/systemInfo.ts Outdated Show resolved Hide resolved

zhuk-aa reviewed Feb 12, 2025

View reviewed changes

src/utils/systemInfo.ts Outdated Show resolved Hide resolved

zhuk-aa reviewed Feb 12, 2025

View reviewed changes

src/utils/types.ts Outdated Show resolved Hide resolved

shide1989 added 4 commits February 13, 2025 15:22

feat: updated sysinfo

0f65f3d

feat: Killed the Dog because it crashes the manual bench workflow

7f1def4

feat: reverted husky re4moval

71e0e4c

0.1.6-alpha

c7f03c5

shide1989 marked this pull request as ready for review February 13, 2025 16:37

zhuk-aa and others added 3 commits February 14, 2025 18:34

sending workspace tree in every query

f0dd017

0.1.6-alpha.1

2028698

0.1.6-alpha.2

c4cbf17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ISS-273 - System footprint #99

ISS-273 - System footprint #99

shide1989 commented Jan 28, 2025

zhuk-aa Feb 10, 2025

zhuk-aa Feb 10, 2025 •

edited

Loading

shide1989 Feb 12, 2025

shide1989 Feb 12, 2025

zhuk-aa Feb 12, 2025

zhuk-aa Feb 12, 2025

shide1989 Feb 12, 2025

shide1989 commented Feb 12, 2025

zhuk-aa Feb 12, 2025 •

edited

Loading

shide1989 Feb 13, 2025

shide1989 Feb 13, 2025

zhuk-aa Feb 12, 2025

ISS-273 - System footprint #99

Are you sure you want to change the base?

ISS-273 - System footprint #99

Conversation

shide1989 commented Jan 28, 2025

🚀 Pull Request Overview

What does this PR do? 🤔

Choose a reason for hiding this comment

zhuk-aa Feb 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shide1989 commented Feb 12, 2025

zhuk-aa Feb 12, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhuk-aa Feb 10, 2025 •

edited

Loading

zhuk-aa Feb 12, 2025 •

edited

Loading