Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/dls user upload #1596 #1597

Draft
wants to merge 63 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
3f81a80
add upload button to dls views
kaperoo Nov 23, 2023
2e497e5
upload button
kaperoo Nov 28, 2023
3a42ce5
upload dialog
kaperoo Nov 28, 2023
56c981d
Install uppy
kaperoo Nov 28, 2023
d36e2bc
install uppy yarn.lock
kaperoo Nov 28, 2023
f049d49
Add upload button to dls tables and cards
kaperoo Nov 28, 2023
5c38b02
install uppy in the right package
kaperoo Nov 28, 2023
7f94c37
unmemoize actions
kaperoo Nov 28, 2023
21252af
adjust to Alex's changes
kaperoo Nov 28, 2023
fe5649a
add function for creating datasets + some translation
kaperoo Nov 28, 2023
9405eb7
add uppy and its dependencies to transformIgnorePatterns
kaperoo Dec 1, 2023
4bd437e
small fixes to upload components
kaperoo Dec 1, 2023
287e97c
initial unit tests
kaperoo Dec 1, 2023
72ea976
temporary fix for golden retriever bug in unit tests
kaperoo Dec 5, 2023
503e39d
improve code coverage
kaperoo Dec 5, 2023
a225710
test for .xml upload
kaperoo Jan 15, 2024
aee9a6f
Merge branch 'develop' into feature/dls-user-upload-#1596
kaperoo Jan 26, 2024
f342996
updated yarn lock
kaperoo Jan 26, 2024
d268064
multiple uppy instances + size/type info
kaperoo Jan 29, 2024
84a5746
improved styling
kaperoo Jan 30, 2024
4efaf43
disable close on backdrop click + fix max file size
kaperoo Feb 2, 2024
4ac562c
files no longer cleared on backdrop click
kaperoo Feb 6, 2024
4aa90f3
invalidate queries after successful upload
kaperoo Feb 6, 2024
79306f7
Merge branch 'develop' into feature/dls-user-upload-#1596
kaperoo Feb 19, 2024
429b781
remove hardcoded upload url
kaperoo Feb 20, 2024
c877ffa
fix unit tests
kaperoo Feb 20, 2024
18403bf
fix restore bug
kaperoo Mar 8, 2024
6981617
Fix uppy styling in production & make it respect dark mode
louise-davies Mar 8, 2024
8fe465f
Merge branch 'feature/dls-user-upload-#1596' of https://github.com/ra…
louise-davies Mar 8, 2024
e1080f8
Add back in accidentally deleted file
louise-davies Mar 8, 2024
3c10f38
remove hardcoded url for dataset creation
kaperoo Mar 8, 2024
e458067
change invalidate queries + button variants
kaperoo Mar 13, 2024
28ea293
use mui theme for button colours
kaperoo Mar 14, 2024
533210b
workaround for uppy ghost files bug
kaperoo Mar 19, 2024
7e77ed3
refresh session before upload
kaperoo Mar 20, 2024
9c38429
move refresh function to api
kaperoo Mar 20, 2024
4deb414
check for duplicate files on file-add
kaperoo Mar 20, 2024
62538d9
provide session in bearer auth token
kaperoo Mar 22, 2024
0533d87
lowercase a in authorization
kaperoo Mar 22, 2024
062561f
checkdatafilename tests
kaperoo Mar 27, 2024
17dcba6
add new restricted files
kaperoo Mar 28, 2024
d38778d
use the /commit endpoint
kaperoo Apr 9, 2024
9082d98
check dataset name
kaperoo Apr 9, 2024
b12adda
check for duplicates in upload queue
kaperoo Apr 10, 2024
554bb7f
change id to url in a call to /commit
kaperoo Apr 10, 2024
87bbd04
refactor name checking fns
kaperoo Apr 11, 2024
e04c247
update unit tests
kaperoo Apr 11, 2024
885e273
tests for refreshSession and checkNameExists
kaperoo Apr 15, 2024
fdb6365
add lastModified field to call to /commit
kaperoo Apr 15, 2024
c1e8242
fix lm for dataset upload
kaperoo Apr 15, 2024
e47c1ee
fix for duplicate check with ghost files
kaperoo Apr 16, 2024
1dd5ba1
upload pre and post processing
kaperoo Apr 19, 2024
6227476
disable upload button
kaperoo Apr 19, 2024
86cba66
single failed upload bux fix
kaperoo Apr 19, 2024
bd4f003
disallow multiple upload batches
kaperoo Apr 22, 2024
6e9bf9c
use datasetId from response instead of parsing
kaperoo Apr 23, 2024
77fa13d
beforeFileAdded tests
kaperoo Apr 23, 2024
92a4eab
postProcessor unit tests
kaperoo Apr 24, 2024
6bbdaab
disable the upload button if there are no files in the queue
kaperoo Apr 29, 2024
50fbc92
fix most of the upload button bugs
kaperoo May 2, 2024
ebb2cea
change the way we fail test files
kaperoo May 7, 2024
512f401
e2e upload tests
kaperoo May 31, 2024
09369c6
change misleading function name
kaperoo May 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ sed -e "s|\"facilityName\": \".*\"|\"facilityName\": \"$FACILITY_NAME\"|" \
-e "s|\"idsUrl\": \".*\"|\"idsUrl\": \"$IDS_URL\"|" \
-e "s|\"apiUrl\": \".*\"|\"apiUrl\": \"$API_URL\"|" \
-e "s|\"downloadApiUrl\": \".*\"|\"downloadApiUrl\": \"$DOWNLOAD_API_URL\"|" \
-e "s|\"uploadUrl\": \".*\"|\"uploadUrl\": \"$UPLOAD_URL\"|" \
-e "s|\"pluginHost\": \".*\"|\"pluginHost\": \"/datagateway-dataview\"|" \
/usr/local/apache2/htdocs/datagateway-dataview/datagateway-dataview-settings.json > "$TEMPFILE"

Expand Down
11 changes: 10 additions & 1 deletion packages/datagateway-common/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,15 @@
"@emotion/styled": "11.11.0",
"@mui/x-date-pickers": "6.11.2",
"@types/lodash.debounce": "4.0.6",
"@uppy/core": "^3.7.1",
"@uppy/dashboard": "^3.7.1",
"@uppy/drag-drop": "^3.0.3",
"@uppy/file-input": "^3.0.4",
"@uppy/form": "^3.0.3",
"@uppy/golden-retriever": "^3.1.1",
"@uppy/progress-bar": "^3.0.4",
"@uppy/react": "^3.2.1",
"@uppy/tus": "^3.4.0",
"axios": "1.6.1",
"connected-react-router": "6.9.1",
"date-fns": "2.30.0",
Expand Down Expand Up @@ -103,7 +112,7 @@
},
"jest": {
"transformIgnorePatterns": [
"node_modules/(?!axios)"
"node_modules/(?!(axios|@uppy|nanoid|exifr|p-queue|p-timeout)/)"
],
"collectCoverageFrom": [
"src/**/*.{tsx,ts,js,jsx}",
Expand Down
47 changes: 47 additions & 0 deletions packages/datagateway-common/src/api/datasets.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
useDatasetDetails,
useDatasetsInfinite,
useDatasetsPaginated,
createDataset,
} from './datasets';

jest.mock('../handleICATError');
Expand Down Expand Up @@ -496,4 +497,50 @@ describe('dataset api functions', () => {
expect(document.body.appendChild).toHaveBeenCalledWith(link);
});
});

describe('createDataset', () => {
it('sends axios request to create a dataset and returns the dataset ID', async () => {
const name = 'Test Dataset';
const description = 'Test Description';
const investigationId = 1;
const datasetId = 123;

const axiosPostMock = jest.spyOn(axios, 'post');
axiosPostMock.mockResolvedValue({
data: {
datasetId: datasetId.toString(),
},
});

const result = await createDataset(name, description, investigationId);

expect(axiosPostMock).toHaveBeenCalledWith(
'http://127.0.0.1:8181/dataset',
{
investigationId: investigationId,
datasetName: name,
datasetDescription: description,
},
{
headers: {
Authorization: 'Bearer null',
},
}
);
expect(result).toBe(datasetId);
});

it('handles axios request error and throws an error', async () => {
const name = 'Test Dataset';
const description = 'Test Description';
const investigationId = 1;

const axiosPostMock = jest.spyOn(axios, 'post');
axiosPostMock.mockRejectedValue(new Error('Test error'));

await expect(
createDataset(name, description, investigationId)
).rejects.toThrow('Test error');
});
});
});
23 changes: 23 additions & 0 deletions packages/datagateway-common/src/api/datasets.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -284,3 +284,26 @@ export const downloadDataset = (
link.click();
link.remove();
};

export const createDataset = (
name: string,
description: string,
investigationId: number
): Promise<number> => {
// const apiUrl = useSelector((state: StateType) => state.dgcommon.urls.apiUrl);

const params = {
investigationId: investigationId,
datasetName: name,
datasetDescription: description,
};

//TODO: remove hardcoded url
return axios
.post('http://127.0.0.1:8181/dataset', params, {
louise-davies marked this conversation as resolved.
Show resolved Hide resolved
headers: {
Authorization: `Bearer ${readSciGatewayToken().sessionId}`,
},
})
.then((response) => parseInt(response.data.datasetId));
};
19 changes: 0 additions & 19 deletions packages/datagateway-common/src/index.css

This file was deleted.

2 changes: 1 addition & 1 deletion packages/datagateway-common/src/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
// import React from 'react';
// import ReactDOM from 'react-dom';
// import App from './App';
// import './index.css';

import { StateType } from './state/app.types';

Expand Down Expand Up @@ -66,6 +65,7 @@ export type { CartProps } from './views/viewCartButton.component';
export { default as ViewButton } from './views/viewButton.component';
export { default as ClearFiltersButton } from './views/clearFiltersButton.component';
export { default as DownloadButton } from './views/downloadButton.component';
export { default as UploadButton } from './views/uploadButton.component';
export { default as SelectionAlert } from './views/selectionAlert.component';

export { default as ISISDatafileDetailsPanel } from './detailsPanels/isis/datafileDetailsPanel.component';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,22 +18,22 @@ exports[`Preloader component renders when the site is loading 1`] = `
style="box-sizing: border-box; padding: 10px 0px;"
>
<div
style="display: block; margin: auto; width: 230px; height: 230px; animation: rotate 10s infinite linear;"
class="MuiBox-root css-i7r3q1"
>
<i
style="position: absolute; display: inline-block; top: 50%; left: 50%; border: 8px solid transparent; border-top-left-radius: 140px; border-top-right-radius: 140px; border-color: #8c4799; height: 70px; width: 140px; margin-top: -70px; margin-left: -70px; animation-name: rotate; animation-iteration-count: infinite; animation-duration: 3s; animation-timing-function: cubic-bezier(.09, 0, 0, .03); transform-origin: 50% 100% 0; box-sizing: border-box;"
class="css-1xpn6vn"
/>
<i
style="position: absolute; display: inline-block; top: 50%; left: 50%; border: 8px solid transparent; border-top-left-radius: 148px; border-top-right-radius: 148px; border-color: #1d4f91; height: 79px; width: 158px; margin-top: -79px; margin-left: -79px; animation-name: rotate; animation-iteration-count: infinite; animation-duration: 3s; animation-timing-function: cubic-bezier(.09, 0.3, 0.12, .03); transform-origin: 50% 100% 0; box-sizing: border-box;"
class="css-5srjsq"
/>
<i
style="position: absolute; display: inline-block; top: 50%; left: 50%; border: 8px solid transparent; border-top-left-radius: 156px; border-top-right-radius: 156px; border-color: #c34613; height: 88px; width: 176px; margin-top: -88px; margin-left: -88px; animation-name: rotate; animation-iteration-count: infinite; animation-duration: 3s; animation-timing-function: cubic-bezier(.09, 0.6, 0.24, .03); transform-origin: 50% 100% 0; box-sizing: border-box;"
class="css-gke7bx"
/>
<i
style="position: absolute; display: inline-block; top: 50%; left: 50%; border: 8px solid transparent; border-top-left-radius: 164px; border-top-right-radius: 164px; border-color: #008275; height: 97px; width: 194px; margin-top: -97px; margin-left: -97px; animation-name: rotate; animation-iteration-count: infinite; animation-duration: 3s; animation-timing-function: cubic-bezier(.09, 0.8999999999999999, 0.36, .03); transform-origin: 50% 100% 0; box-sizing: border-box;"
class="css-19f05lv"
/>
<i
style="position: absolute; display: inline-block; top: 50%; left: 50%; border: 8px solid transparent; border-top-left-radius: 172px; border-top-right-radius: 172px; border-color: #63666a; height: 106px; width: 212px; margin-top: -106px; margin-left: -106px; animation-name: rotate; animation-iteration-count: infinite; animation-duration: 3s; animation-timing-function: cubic-bezier(.09, 1.2, 0.48, .03); transform-origin: 50% 100% 0; box-sizing: border-box;"
class="css-1m17ujy"
/>
</div>
</div>
Expand Down
37 changes: 21 additions & 16 deletions packages/datagateway-common/src/preloader/preloader.component.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import React from 'react';
import { Box } from '@mui/material';
import { Box, keyframes, styled } from '@mui/material';

const colors = ['#8C4799', '#1D4F91', '#C34613', '#008275', '#63666A'];
const innerRadius = 140;
Expand All @@ -11,11 +11,13 @@ interface PreloaderProps {
children: React.ReactNode;
}

interface SpinnerStyle {
[id: string]: string | number;
}
const rotate = keyframes`
to {
transform: rotateZ(360deg);
}
`;

const spinnerStyle = (index: number): SpinnerStyle => {
const StyledI = styled('i')(({ index }: { index: number }) => {
const size = innerRadius + index * 2 * (border + spacing);

return {
Expand All @@ -32,7 +34,10 @@ const spinnerStyle = (index: number): SpinnerStyle => {
width: size,
marginTop: -size / 2,
marginLeft: -size / 2,
animationName: 'rotate',
animation: `${rotate} 3s infinite cubic-bezier(.09, ${0.3 * index}, ${
0.12 * index
}, .03)`,
animationName: `${rotate}`,
animationIterationCount: 'infinite',
animationDuration: '3s',
animationTimingFunction: `cubic-bezier(.09, ${0.3 * index}, ${
Expand All @@ -41,7 +46,7 @@ const spinnerStyle = (index: number): SpinnerStyle => {
transformOrigin: '50% 100% 0',
boxSizing: 'border-box',
};
};
});

const Preloader: React.FC<PreloaderProps> = (props: PreloaderProps) => (
<div role={props.loading ? 'progressbar' : undefined}>
Expand All @@ -59,21 +64,21 @@ const Preloader: React.FC<PreloaderProps> = (props: PreloaderProps) => (
}}
>
<div style={{ boxSizing: 'border-box', padding: '10px 0' }}>
<div
style={{
<Box
sx={{
display: 'block',
margin: 'auto',
width: innerRadius + colors.length * 2 * (border + spacing),
height: innerRadius + colors.length * 2 * (border + spacing),
animation: 'rotate 10s infinite linear',
animation: `${rotate} 10s infinite linear`,
}}
>
<i style={spinnerStyle(0)} />
<i style={spinnerStyle(1)} />
<i style={spinnerStyle(2)} />
<i style={spinnerStyle(3)} />
<i style={spinnerStyle(4)} />
</div>
<StyledI index={0} />
<StyledI index={1} />
<StyledI index={2} />
<StyledI index={3} />
<StyledI index={4} />
</Box>
</div>
<Box sx={{ color: 'text.primary' }}>Loading...</Box>
</Box>
Expand Down
1 change: 1 addition & 0 deletions packages/datagateway-common/src/setupTests.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ export const createReactQueryWrapper = (
icatUrl: 'https://example.com/icat',
idsUrl: 'https://example.com/ids',
downloadApiUrl: 'https://example.com/topcat',
uploadUrl: 'https://example.com/upload',
},
},
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,7 @@ export interface URLs {
apiUrl: string;
downloadApiUrl: string;
icatUrl: string;
uploadUrl?: string;
}

export interface PluginRoute {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`Upload dialog component Datafile renders correctly 1`] = `<DocumentFragment />`;

exports[`Upload dialog component Dataset renders correctly 1`] = `<DocumentFragment />`;
Loading
Loading