You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1.**Environment Variable**: The tool checks for the following environment variables in order: `NVIDIA_BUILD_AI_ACCESS_TOKEN`, `NVIDIA_ACCESS_TOKEN`, `ACCESS_TOKEN`, `NVIDIA_API_KEY`, `API_KEY`.
2. **Command-Line Flag**: Use the `-k` or `--access-token` flag to provide the token directly. This overrides any environment variables.
35
+
```bash
36
+
./nvidia-ai-chat -k "your_token_here"
37
+
```
38
+
39
+
### Conversation Management
40
+
41
+
By default, `nvidia-ai-chat` stores your conversations in`~/.cache/nvidia-chat/`.
42
+
43
+
- **Starting a New Chat**: If you run the tool without specifying a file, it creates a new timestamped conversation file (e.g., `conversation-20231027-123456.json`) and prints its path.
44
+
- **Resuming a Chat**: To continue a previous conversation, pass the path to the conversation file as an argument:
45
+
```bash
46
+
./nvidia-ai-chat /path/to/your/conversation.json
47
+
```
37
48
38
49
### Interactive Mode
39
50
40
-
To start an interactive chat session, simply run the application:
51
+
To start an interactive chat session, run the application, optionally specifying a conversation file:
41
52
42
53
```bash
43
54
./nvidia-ai-chat
@@ -50,40 +61,83 @@ This will create a new conversation file in `~/.cache/nvidia-chat/`. You can als
50
61
```
51
62
52
63
In interactive mode, you can use the following commands:
64
+
- `/help`: Show the help message.
53
65
- `/exit`, `/quit`: Exit the program.
54
66
- `/history`: Print the full conversation JSON.
55
67
- `/clear`: Clear the conversation messages.
56
68
- `/save <file>`: Save the conversation to a new file.
69
+
- `/list`: List supported models.
70
+
- `/model <model_name>`: Switch model for the session.
71
+
- `/modelinfo [name]`: List settings for a model (defaults to current).
72
+
- `/askfor_model_setting`: Interactively set model parameters.
73
+
- `/persist-settings`: Save the current session's settings to the conversation file.
57
74
- `/persist-system <file>`: Persist a system prompt from a file.
58
-
-`/model <model_name>`: Switch the model for the current session.
59
-
-`/temperature <0..1>`: Set the temperature for the current session.
60
-
-`/exportlast [-t] <file>`: Export the last AI response to a file.
61
-
-`/exportlastn [-t] <n> <file>`: Export the last `n` AI responses to a file.
62
-
-`/exportn [-t] <n> <file>`: Export the Nth-to-last AI response to a file.
63
-
The optional `-t` flag filters out the AI's "thinking" block from the exported content.
64
-
-`/help`: Show the help message.
75
+
- `/exportlast [-t] <file>`: Export last AI response to a markdown file (-t filters thinking).
76
+
- `/exportlastn [-t] <n> <file>`: Export last n AI responses.
77
+
- `/exportn [-t] <n> <file>`: Export the Nth-to-last AI response.
78
+
- `/randomodel`: Switch to a random supported model.
65
79
66
-
### Non-Interactive Mode
80
+
For any model setting, you can use `/<setting_name> <value>` or `/<setting_name> unset`.
81
+
For example: `/temperature 0.8`, `/stop unset`
67
82
68
-
To get a response for a single prompt, use the `--prompt` flag:
83
+
### Non-Interactive Mode
69
84
85
+
To get a response for a single prompt without entering an interactive session, use the `--prompt` flag. The tool will print the AI's response to standard output and exit.
86
+
87
+
The `--prompt` flag can accept:
88
+
- A string of text directly:
89
+
```bash
90
+
./nvidia-ai-chat --prompt="Translate 'hello' to French"
91
+
```
92
+
- A path to a file containing the prompt:
93
+
```bash
94
+
./nvidia-ai-chat --prompt=./my_prompt.txt
95
+
```
96
+
- A hyphen (`-`) to read the prompt from standard input (stdin):
97
+
```bash
98
+
echo"Summarize this article"| ./nvidia-ai-chat --prompt=-
99
+
```
100
+
101
+
You can combine this with other flags, such as specifying a model:
70
102
```bash
71
-
./nvidia-ai-chat --prompt="Hello, world!"
103
+
./nvidia-ai-chat --model="google/codegemma-7b" --prompt="Write a python function to check for prime numbers"
72
104
```
73
105
74
-
You can also pipe the prompt from stdin:
75
-
106
+
You can also use non-interactive mode with an existing conversation file to provide context to the model:
76
107
```bash
77
-
echo"Hello, world!"|./nvidia-ai-chat --prompt=-
108
+
./nvidia-ai-chat --prompt="What was the last thing we talked about?" /path/to/conversation.json
78
109
```
79
110
80
111
### Options
81
112
82
-
For a full list of options, run:
83
-
84
-
```bash
85
-
./nvidia-ai-chat --help
86
-
```
113
+
For a full list of options, run `./nvidia-ai-chat --help`.
114
+
115
+
#### General Options
116
+
117
+
- `-h, --help`: Show the help message and exit.
118
+
- `-l, --list`: List supported models and exit.
119
+
- `-m, --model NAME`: Specify the model ID to use (e.g., `mistralai/mistral-small-24b-instruct`).
120
+
- `-k, --access-token KEY`: Provide your API key directly.
121
+
- `--prompt TEXT|FILE|-`: Enable non-interactive mode and provide the prompt.
122
+
- `-s, --sys-prompt-file PATH`: Path to a file containing a system prompt to use for the session.
123
+
- `-S`: Persist the system prompt provided via `-s` to the conversation file.
124
+
- `--save-settings`: Persist the current session's model settings to the conversation file.
125
+
- `--modelinfo NAME`: Show detailed settings and capabilities for a specific model and exit.
126
+
127
+
#### Model Setting Options
128
+
129
+
These flags override the default settings for the current session. For model-specific details, ranges, and defaults, use the `/modelinfo <model_name>` command in interactive mode.
130
+
131
+
- `--temperature <0..1>`: Set the sampling temperature.
132
+
- `--top-p <0.01..1>`: Set the top-p sampling mass.
133
+
- `--max-tokens <number>`: Set the maximum number of tokens to generate.
134
+
- `--frequency-penalty <-2..2>`: Set the frequency penalty.
135
+
- `--presence-penalty <-2..2>`: Set the presence penalty.
136
+
- `--stop <string>`: Set a custom stop sequence.
137
+
- `--stream <true|false>`: Enable or disable streaming responses.
138
+
- `--history-limit <number>`: Set the maximum number of messages to keep in the conversation history.
139
+
- `--reasoning-effort <low|medium|high>`: Control the reasoning effort for capable models.
140
+
- ... and many more model-specific parameters. Use `/modelinfo` to discover them.
0 commit comments