Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add datasets importing functionality #201

Open
wants to merge 60 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
de78f6b
feat(server/helpers): Add import search
shivam-tripathi Jul 12, 2018
acc103f
feat(helpers/utils): Add function to get import models
shivam-tripathi Jul 18, 2018
e60449e
feat(helpers/utils): Add function to get import model by name
shivam-tripathi Jul 18, 2018
56670ee
feat(helpers/utils): Add JSDocs for import functions
shivam-tripathi Jul 18, 2018
b272cc3
feat(server/helpers/search): Add import support in _fetchEntityModels…
shivam-tripathi Jul 18, 2018
c0b695e
feat(server/helpers/): Update licenses
shivam-tripathi Jul 18, 2018
870e0f0
feat(client/components/pages/search): Update search with message for …
shivam-tripathi Jul 18, 2018
cf4680e
feat(client/components/pages/parts/search-results): Update search res…
shivam-tripathi Jul 18, 2018
203dc05
feat(client/containers/layout): Add panel link to recent imports
shivam-tripathi Jul 12, 2018
90be205
feat(scripts): Update build-client-js and watch-client-js to track re…
shivam-tripathi Jul 31, 2018
0de46fa
feat(server/routes/imports/recent): Add routes to handle recent imports
shivam-tripathi Jul 31, 2018
3ad38bf
feat(client/controllers/recentImports): Add recentImports controller
shivam-tripathi Jul 31, 2018
8de2735
feat(client/components/pages/parts/recentImports): Add part to hold r…
shivam-tripathi Jul 31, 2018
53b0b6f
feat(client/components/pages/recentImports): Add base component to di…
shivam-tripathi Jul 31, 2018
61657de
feat(server/routes): Add imports route
shivam-tripathi Jul 31, 2018
0344296
feat(client/helpers/utils): Add function to extract imports url
shivam-tripathi Jul 31, 2018
4513011
feat(client/helpers/pagination-props): Add function to determine pagi…
shivam-tripathi Jul 31, 2018
08b4d0a
feat(scripts): Add build and watch command in client scripts for impo…
shivam-tripathi Aug 3, 2018
1426f1f
feat(client/helpers/props): Add function to extract import entity props
shivam-tripathi Aug 3, 2018
2854bff
feat(client/controllers/import-entity): Add import-entity controller
shivam-tripathi Aug 3, 2018
913c54b
feat(client/helpers/import-entity): Add import-entity helper functions
shivam-tripathi Aug 3, 2018
208fc7e
feat(client/components/pages/import-entities/footer): Add footer for …
shivam-tripathi Aug 3, 2018
49639b8
feat(src/client/components/pages/import-entities/title): Add title fr…
shivam-tripathi Aug 3, 2018
2f78a55
feat(src/client/components/pages/import-entities/footer): Remove unne…
shivam-tripathi Aug 3, 2018
21915cd
feat(client/components/pages/import-entities): Add import-entities pages
shivam-tripathi Aug 3, 2018
37ae6a2
feat(server/routes/import-entity): Refactor, add more routes
shivam-tripathi Aug 3, 2018
ae662d0
feat(recent-imports): Refactor code
shivam-tripathi Aug 10, 2018
06e8219
feat(scripts/client): Update client build and watch scripts
shivam-tripathi Aug 10, 2018
45fc92d
feat(package.json): Add uuid module
shivam-tripathi Aug 10, 2018
f5fba4b
feat(client/components/pages/entities): Make compulsory props of imag…
shivam-tripathi Aug 10, 2018
fdc8f17
refactor(client/components/pages/parts/recent-import-results)
shivam-tripathi Aug 10, 2018
48bd1b4
feat(client/discard-import-entity): Add confirm discard vote cast page
shivam-tripathi Aug 10, 2018
8d738f7
feat(client/controllers/recent-imports): Move recent-imports controll…
shivam-tripathi Aug 10, 2018
e563583
feat(client/controllers/import-entity): Add controller for import-ent…
shivam-tripathi Aug 10, 2018
2c25f13
refactor(client/helpers/utils): Remove unused function
shivam-tripathi Aug 10, 2018
b68bbab
feat(client/helpers/import-entity): Add import-entity helper function…
shivam-tripathi Aug 10, 2018
8ba3a82
feat(client/import-entities/footer): Add feature to prevent multiple …
shivam-tripathi Aug 10, 2018
560d24b
feat(client/components/pages/import-entities): Add import-entity pages
shivam-tripathi Aug 10, 2018
2598bdf
feat(client/entity-editor/submission-section): Add logic for redirect…
shivam-tripathi Aug 10, 2018
7c07651
feat(server/helpers/middleware): Add middleware to load imports into …
shivam-tripathi Aug 10, 2018
0374e40
feat(server/helpers/importEntityRouteUtils): Add logic to load edit p…
shivam-tripathi Aug 10, 2018
5fd6e9a
feat(server/helpers/utils): Add function to validate importId, could …
shivam-tripathi Aug 10, 2018
cab5b6a
feat(server/routes/import-entity/transform-form): Add module to trans…
shivam-tripathi Aug 10, 2018
b263307
feat(server/routes/import-entity/transform-import): Add module to tra…
shivam-tripathi Aug 10, 2018
a5b18db
feat(server/routes/import-entity/import-routes): Add import route hel…
shivam-tripathi Aug 10, 2018
0987520
feat(server/routes/import-entity): Add entity functions, update index
shivam-tripathi Aug 10, 2018
c8cb4d0
refactor(server/routes): Update routes to reflect import-entities
shivam-tripathi Aug 10, 2018
e450dec
feat(server/routes/import-entity): Add entity routes
shivam-tripathi Aug 18, 2018
762ccd9
feat(client/recent-imports): Add maximum button limit
shivam-tripathi Aug 18, 2018
26d82a7
feat(components/parts/search): Remove inline styles, add break before…
shivam-tripathi Aug 19, 2018
18fc335
feat(client/components/pages/entities): Export all attributes functions
shivam-tripathi Aug 19, 2018
29e3ada
feat(client/commit/pages/import-entities): Reuse attribute functions …
shivam-tripathi Aug 19, 2018
154f76c
feat(client/components/import-entities): Remove inline css
shivam-tripathi Aug 19, 2018
84989c7
feat(discard-import-entity): Remove token mechanism
shivam-tripathi Aug 19, 2018
18199d5
feat(client/import-entity/footer): Change name
shivam-tripathi Aug 19, 2018
35ea8b0
refactor: General refactor, remove redundant code, update credits
shivam-tripathi Aug 19, 2018
2afd166
feat(stylesheets/style.less): Add some style classes
shivam-tripathi Aug 19, 2018
b15a144
feat(client/helpers/import-entity): Add space after label, refactor
shivam-tripathi Aug 19, 2018
b61c993
feat(client/components/search-results): Use labels to denote imports
shivam-tripathi Aug 19, 2018
f8bb16c
feat(client/controllers/import-entity): Improve error handling when l…
shivam-tripathi Aug 19, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
205 changes: 205 additions & 0 deletions src/server/routes/import-entity/import-routes.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
/*
* Copyright (C) 2018 Shivam Tripathi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add copyright for me in 2016 (some of the entity display code adapted here).

*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

import * as achievement from '../../helpers/achievement';
import * as error from '../../helpers/error';
import * as propHelpers from '../../../client/helpers/props';
import * as search from '../../helpers/search';
import {escapeProps, generateProps} from '../../helpers/props';
import Layout from '../../../client/containers/layout';
import React from 'react';
import ReactDOMServer from 'react-dom/server';
import _ from 'lodash';
import {entityEditorMarkup} from '../../helpers/entityRouteUtils';
import {entityToFormState} from './transform-import';
import {generateImportEntityProps} from '../../helpers/importEntityRouteUtils';
import {getEntityUrl} from '../../../client/helpers/entity';
import {getImportUrl} from '../../../client/helpers/import-entity';
import {getValidator} from '../../../client/entity-editor/helpers';
import importEntityPages from
'../../../client/components/pages/import-entities';
import {transformForm} from './transform-form';
import uuid from 'uuid';


export function displayImport(req, res) {
const {importEntity} = res.locals;

// Get unique identifier types for display
const identifierTypes = importEntity.identifierSet &&
_.uniqBy(
_.map(importEntity.identifierSet.identifiers, 'type'),
(type) => type.id
);

const {type} = importEntity;
const ImportEntityComponent = importEntityPages[type];
if (ImportEntityComponent) {
const props = generateProps(req, res, {identifierTypes});
const markup = ReactDOMServer.renderToString(
<Layout {...propHelpers.extractLayoutProps(props)}>
<ImportEntityComponent
{...propHelpers.extractImportEntityProps(props)}
/>
</Layout>
);

res.render('target', {
markup,
props: escapeProps(props),
script: '/js/import-entity/import-entity.js'
});
}
else {
throw new Error(
`Component was not found for the following import: ${type}`
);
}
}

export function displayDiscardImportEntity(req, res) {
const {importEntity} = res.locals;
const importUrl = getImportUrl(importEntity);

if (importEntity.hasVoted) {
res.redirect(importUrl);
}

if (!req.session.token) {
req.session.token = uuid();
}
const {token} = req.session;
const props = generateProps(req, res, {token});
const {DiscardImportEntityPage} = importEntityPages;
const markup = ReactDOMServer.renderToString(
<Layout {...propHelpers.extractLayoutProps(props)}>
<DiscardImportEntityPage
importEntity={props.importEntity}
token={props.token}
/>
</Layout>
);

res.render('target', {
markup,
props: escapeProps(props),
script: '/js/import-entity/discard-import-entity.js'
});
}

export function handleDiscardImportEntity(req, res) {
const {headers} = req;
const token = headers['x-auth-token'];

if (token !== req.session.token) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The token as implemented doesn't seem to be useful. I think we should remove it until we work out a better system and identify the exact problem that needs to be solved.

res.status(401).send({error: 'No token provided.'});
}

const {orm} = req.app.locals;
const editorId = req.session.passport.user.id;
const {importEntity} = res.locals;
orm.bookshelf.transaction(async transacting => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use parentheses around the parameter to an arrow function

try {
await orm.func.imports.castDiscardVote(
transacting, importEntity.importId, editorId
);
// To-do: Add code to remove importEntity from the search index
res.status(200).send();
}
catch (err) {
res.status(400).send({error: err});
}
});
}

export async function approveImportEntity(req, res) {
const {orm} = res.app.locals;
const editorId = req.session.passport.user.id;
const {importEntity} = res.locals;
const entity = await orm.bookshelf.transaction(transacting =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with arrow function parentheses here - eslint should have caught this.

orm.func.imports.approveImport(
{editorId, importEntity, orm, transacting}
));
const entityUrl = getEntityUrl(entity);

/* Add code to remove import and add the newly created entity to the elastic
search index and remove delete import */

// Update editor achievement
entity.alert = (await achievement.processEdit(
orm, editorId, entity.revisionId
)).alert;

// Cleanup search indexing
search.indexEntity(entity);
// Todo: Add functionality to remove imports from ES index upon deletion

res.redirect(entityUrl);
}

export function editImportEntity(req, res) {
const {importEntity} = res.locals;
const initialState = entityToFormState(importEntity);
const importEntityProps = generateImportEntityProps(
req, res, initialState, {}
);
const {markup, props} = entityEditorMarkup(importEntityProps);
return res.render('target', {
markup,
props: escapeProps(props),
script: '/js/entity-editor.js',
title: 'Edit Work Import'
});
}

export async function approveImportPostEditing(req, res) {
const {orm} = req.app.locals;
const {importEntity} = res.locals;
const editorId = req.session.passport.user.id;
const {importId, type} = importEntity;
const formData = req.body;

const validateForm = getValidator(type.toLowerCase());

if (!validateForm(formData)) {
const err = new error.FormSubmissionError();
error.sendErrorAsJSON(res, err);
}

const entityData = transformForm[type](formData);

const entity = await orm.bookshelf.transaction(async (transacting) => {
await orm.func.imports.deleteImport(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to swap the order of these, and pass the new entity BBID to the deleteImport function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to pass the new entity BBID to the deleteImport function? Also how does it matter we keep deletion before creation or creation before deletion, as it all happens in the same transaction?

Copy link
Member

@LordSputnik LordSputnik Aug 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The deleteImport function needs to know the new entity BBID to update the link_import table doesn't it?

https://github.com/bookbrainz/bookbrainz-data-js/blob/master/src/func/imports/delete-import.js

transacting, importId
);
return orm.func.createEntity(
{editorId, entityData, orm, transacting}
);
});

// Update editor achievement
entity.alert = (await achievement.processEdit(
orm, editorId, entity.revisionId
)).alert;

// Cleanup search indexing
await search.indexEntity(entity);
// To-do: Add code to remove importEntity from the search index

res.send(entity);
}