feat(webui): Add compression job submission for local files only. #1294
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,23 @@ | ||||||||||||||||||
| import { | ||||||||||||||||||
| CompressionJob, | ||||||||||||||||||
| CompressionJobCreation, | ||||||||||||||||||
| } from "@webui/common/schemas/compression"; | ||||||||||||||||||
|
Comment on lines
+1
to
+4
There was a problem hiding this comment. 🧹 Nitpick Use type‑only imports for TS types. Avoids emitting unused runtime imports under strict TS configs. -import {
+import type {
CompressionJob,
CompressionJobCreation,
} from "@webui/common/schemas/compression";📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||
| import axios from "axios"; | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| /** | ||||||||||||||||||
| * Submits a compression job. | ||||||||||||||||||
| * | ||||||||||||||||||
| * @param payload | ||||||||||||||||||
| * @return | ||||||||||||||||||
| */ | ||||||||||||||||||
| const submitCompressionJob = async (payload: CompressionJobCreation) | ||||||||||||||||||
| : Promise<CompressionJob> => { | ||||||||||||||||||
| console.log("Submitting compression job:", JSON.stringify(payload)); | ||||||||||||||||||
| const {data} = await axios.post<CompressionJob>("/api/compress", payload); | ||||||||||||||||||
|
|
||||||||||||||||||
| return data; | ||||||||||||||||||
| }; | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| export {submitCompressionJob}; | ||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| import {CLP_DEFAULT_DATASET_NAME} from "@webui/common/config"; | ||
| import { | ||
| Form, | ||
| Input, | ||
| } from "antd"; | ||
|
|
||
|
|
||
| const DATASET_HELPER_TEXT = `If left empty, dataset "${CLP_DEFAULT_DATASET_NAME}" will be used.`; | ||
| const DATASET_PLACEHOLDER_TEXT = "The dataset for new archives"; | ||
| const TIMESTAMP_KEY_HELPER_TEXT = "If not provided, events will not have assigned" + | ||
| " timestamps and can only be searched from the command line without a timestamp filter."; | ||
| const TIMESTAMP_KEY_PLACEHOLDER_TEXT = | ||
| "The path (e.g. x.y) for the field containing the log event's timestamp"; | ||
|
There was a problem hiding this comment. putting this on a new line; otherwise the string has to be broken into two to avoid line length violations |
||
|
|
||
| /** | ||
| * Renders additional compression job submission form items for CLP-S storage engine. | ||
| * | ||
| * @return | ||
| */ | ||
| const ClpSFormItems = () => ( | ||
| <> | ||
| <Form.Item | ||
| label={"Dataset"} | ||
| name={"dataset"} | ||
| tooltip={DATASET_HELPER_TEXT} | ||
| > | ||
| <Input placeholder={DATASET_PLACEHOLDER_TEXT}/> | ||
| </Form.Item> | ||
| <Form.Item | ||
| label={"Timestamp Key"} | ||
| name={"timestampKey"} | ||
| tooltip={TIMESTAMP_KEY_HELPER_TEXT} | ||
| > | ||
| <Input placeholder={TIMESTAMP_KEY_PLACEHOLDER_TEXT}/> | ||
| </Form.Item> | ||
| </> | ||
| ); | ||
|
|
||
|
|
||
| export default ClpSFormItems; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import { | ||
| Form, | ||
| Input, | ||
| } from "antd"; | ||
|
|
||
|
|
||
| /** | ||
| * Renders a form item for inputting file paths for compression job submission. | ||
| * | ||
| * @return | ||
| */ | ||
| const PathsInputFormItem = () => ( | ||
| <Form.Item | ||
| label={"Paths"} | ||
| name={"paths"} | ||
| rules={[{required: true, message: "Please enter at least one path"}]} | ||
| > | ||
| <Input.TextArea | ||
| autoSize={{minRows: 4, maxRows: 10}} | ||
| placeholder={"Enter paths to compress, one per line"}/> | ||
| </Form.Item> | ||
| ); | ||
|
|
||
|
|
||
| export default PathsInputFormItem; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| import { | ||
| Button, | ||
| Form, | ||
| } from "antd"; | ||
|
|
||
|
|
||
| interface SubmitFormItemProps { | ||
| isSubmitting: boolean; | ||
| } | ||
|
|
||
| /** | ||
| * Render a submit button for a form. | ||
| * | ||
| * @param props | ||
| * @param props.isSubmitting | ||
| * @return | ||
| */ | ||
| const SubmitFormItem = ({isSubmitting}: SubmitFormItemProps) => ( | ||
| <Form.Item> | ||
| <Button | ||
| htmlType={"submit"} | ||
| loading={isSubmitting} | ||
| type={"primary"} | ||
| > | ||
| Submit | ||
| </Button> | ||
| </Form.Item> | ||
| ); | ||
|
|
||
|
|
||
| export default SubmitFormItem; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| import { | ||
| useMutation, | ||
| useQueryClient, | ||
| } from "@tanstack/react-query"; | ||
| import {CLP_STORAGE_ENGINES} from "@webui/common/config"; | ||
| import {CompressionJobCreation} from "@webui/common/schemas/compression"; | ||
| import { | ||
| Form, | ||
| Typography, | ||
| } from "antd"; | ||
|
|
||
| import {submitCompressionJob} from "../../../api/compress"; | ||
| import {DashboardCard} from "../../../components/DashboardCard"; | ||
| import {SETTINGS_STORAGE_ENGINE} from "../../../config"; | ||
| import ClpSFormItems from "./ClpSFormItems"; | ||
| import PathsInputFormItem from "./PathsInputFormItem"; | ||
| import SubmitFormItem from "./SubmitFormItem"; | ||
|
|
||
|
|
||
| type FormValues = { | ||
| paths: string; | ||
| dataset?: string; | ||
| timestampKey?: string; | ||
| }; | ||
|
|
||
|
|
||
| /** | ||
| * Renders a compression job submission form. | ||
| * | ||
| * @return | ||
| */ | ||
| const Compress = () => { | ||
| const [form] = Form.useForm<FormValues>(); | ||
|
|
||
| const queryClient = useQueryClient(); | ||
| const { | ||
| mutate, | ||
| isPending: isSubmitting, | ||
| isSuccess, | ||
| isError, | ||
| data, | ||
| error, | ||
| } = useMutation({ | ||
| mutationFn: submitCompressionJob, | ||
| onSettled: async () => { | ||
| // Invalidate queries that are affected by a new compression job. | ||
| await queryClient.invalidateQueries({queryKey: ["jobs"]}); | ||
|
There was a problem hiding this comment. any reason not to put this in the success handler? There was a problem hiding this comment. while we don't expect contents in the job table to change if a row cannot be inserted into the table, i think we should always refresh the job table so it's less error prone and future proof. for example, if the |
||
| }, | ||
| onSuccess: () => { | ||
| form.resetFields(); | ||
| }, | ||
| }); | ||
|
Comment on lines
+43
to
+52
There was a problem hiding this comment. 🧹 Nitpick Optional: add explicit generics to useMutation for stronger typing. Keeps the mutation’s data/error/variables types explicit. - } = useMutation({
+ } = useMutation<{ jobId: string }, Error, CompressionJobCreation>({
mutationFn: submitCompressionJob,
onSettled: async () => {
// Invalidate queries that are affected by a new compression job.
await queryClient.invalidateQueries({queryKey: ["jobs"]});
},
onSuccess: () => {
form.resetFields();
},
});
|
||
|
|
||
| const handleSubmit = (values: FormValues) => { | ||
| // eslint-disable-next-line no-warning-comments | ||
| // TODO: replace the UI with a file selector and remove below string manipulation. | ||
| // Convert multiline input to array of paths. | ||
| const paths = values.paths | ||
| .split("\n") | ||
| .map((path) => path.trim()) | ||
| .filter((path) => 0 < path.length); | ||
|
|
||
coderabbitai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| const payload: CompressionJobCreation = {paths}; | ||
|
|
||
| if ("undefined" !== typeof values.dataset) { | ||
| payload.dataset = values.dataset; | ||
| } | ||
| if ("undefined" !== typeof values.timestampKey) { | ||
| payload.timestampKey = values.timestampKey; | ||
| } | ||
|
|
||
| mutate(payload); | ||
| }; | ||
|
|
||
| return ( | ||
| <DashboardCard title={"Start Ingestion"}> | ||
| <Form | ||
| form={form} | ||
| layout={"vertical"} | ||
| onFinish={handleSubmit} | ||
| > | ||
| <PathsInputFormItem/> | ||
| {CLP_STORAGE_ENGINES.CLP_S === SETTINGS_STORAGE_ENGINE && <ClpSFormItems/>} | ||
| <SubmitFormItem isSubmitting={isSubmitting}/> | ||
|
|
||
| {isSuccess && ( | ||
| <Typography.Text type={"success"}> | ||
| Compression job submitted successfully with ID: | ||
| {" "} | ||
| {data.jobId} | ||
| </Typography.Text> | ||
| )} | ||
| {isError && ( | ||
| <Typography.Text type={"danger"}> | ||
| Failed to submit compression job: | ||
| {" "} | ||
| {error instanceof Error ? | ||
| error.message : | ||
| "Unknown error"} | ||
| </Typography.Text> | ||
| )} | ||
| </Form> | ||
| </DashboardCard> | ||
| ); | ||
| }; | ||
|
|
||
|
|
||
| export default Compress; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,6 @@ | |
| gap: 20px; | ||
| } | ||
|
|
||
| .fullRow { | ||
| grid-column: 1 / -1; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Nitpick
🧩 Analysis chain
New archive-output + storage-engine keys: verify server settings support; consider container_clp_config for consistency.
Apply:
Run:
🏁 Script executed:
Length of output: 3455
Verified: ArchiveOutput keys + ClpStorageEngine exist; optional refactor available
🤖 Prompt for AI Agents