You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: .github/workflows/qiita-ci.yml
+28-3
Original file line number
Diff line number
Diff line change
@@ -154,8 +154,6 @@ jobs:
154
154
155
155
echo "5. Setting up qiita"
156
156
conda activate qiita
157
-
# adapt environment_script for private qiita plugins from travis to github actions.
158
-
sed 's#export PATH="/home/travis/miniconda3/bin:$PATH"; source #source /home/runner/.profile; conda #' -i qiita_db/support_files/patches/54.sql
159
157
qiita-env make --no-load-ontologies
160
158
qiita-test-install
161
159
qiita plugins update
@@ -203,7 +201,34 @@ jobs:
203
201
QIITA_PID=`cat /tmp/supervisord.pid`
204
202
kill $QIITA_PID
205
203
sleep 10
206
-
if [[ "$COVER_PACKAGE" != *"qiita_db"* ]]; then test_data_studies/commands.sh; all-qiita-cron-job; fi
204
+
# due to qiita_db tests being more complex and taking longer than
205
+
# the other tests we will only add some extra tests to the run that is
206
+
# not testing qiita_db
207
+
if [[ "$COVER_PACKAGE" != *"qiita_db"* ]]; then
208
+
# 1. testing that we can add some "dummy" studies to the db via
209
+
# CLI
210
+
test_data_studies/commands.sh;
211
+
# 2. making sure that all qiita cron jobs complete as expected
212
+
all-qiita-cron-job;
213
+
# 3. making sure than a production system has the expected rows
214
+
# in all our tables; steps: a. drop test db, b. change $QIITA_CONFIG_FP
215
+
# c. create new production system, c. count rows in the db.
216
+
qiita-env drop;
217
+
cp $QIITA_CONFIG_FP ${QIITA_CONFIG_FP}.bk
218
+
sed 's/TEST_ENVIRONMENT = TRUE/TEST_ENVIRONMENT = FALSE/g' ${QIITA_CONFIG_FP}.bk > $QIITA_CONFIG_FP;
219
+
qiita-env make --no-load-ontologies;
220
+
221
+
export PGPASSWORD=postgres
222
+
pgport=${{ job.services.postgres.ports[5432] }}
223
+
row_counts=`psql -h localhost -U postgres -d qiita_test -p $pgport -c "SELECT SUM(c.reltuples) FROM pg_class c JOIN pg_namespace n on n.oid = c.relnamespace WHERE n.nspname = 'qiita' AND c.relkind = 'r' AND n.nspname NOT IN ('information_schema', 'pg_catalog');"`
224
+
if [[ `echo $row_counts` != *" 0 "* ]]; then
225
+
echo "***********";
226
+
echo "The number of rows in a production system is not what's expected:";
Copy file name to clipboardExpand all lines: CHANGELOG.md
+19
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,24 @@
1
1
# Qiita changelog
2
2
3
+
Version 2024.07
4
+
---------------
5
+
6
+
Deployed on July 15th, 2024
7
+
8
+
* On June 14th, 2024 we modified the SPP to use ["fastp & minimap2 against GRCh38.p14 + Phi X 174 + T2T-CHM13v2.0, then Movi against GRCh38.p14, T2T-CHM13v2.0 + Human Pangenome Reference Consortium release 2023"](https://github.com/cguccione/human_host_filtration) to filter human-reads.
9
+
* Full refactor of the [DB patching system](https://github.com/qiita-spots/qiita/blob/master/CONTRIBUTING.md#patch-91sql) to make sure that a new production deployment has a fully empty database.
10
+
* Fully removed Qiimp from Qiita.
11
+
* Users can now add `ORCID`, `ResearchGate` and/or `GoogleScholar` information to their profile and the creation (registration) timestamp is kept in the database. Thank you @jlab.
12
+
* Admins can now track and purge non-confirmed users from the database via the GUI (`/admin/purge_users/`). Thank you @jlab.
13
+
* Added `qiita.slurm_resource_allocations` to store general job resource usage, which can be populated by `qiita_db.util.update_resource_allocation_table`.
14
+
* Added `qiita_db.util.resource_allocation_plot` to generate different models to allocate resources from a given software command based on previous jobs, thank you @Gossty !
15
+
* The stats page map can be centered via the configuration file; additionally, the Help and Admin emails are defined also via the configuration files, thank you @jlab !
16
+
*``Sequel IIe``, ``Revio``, and ``Onso`` are now valid instruments for the ``PacBio_SMRT`` platform.
17
+
* Added `current_human_filtering` to the prep-information and `human_reads_filter_method` to the artifact to keep track of the method that it was used to human reads filter the raw artifact and know if it's up to date with what is expected via the best practices.
18
+
* Added `reprocess_job_id` to the prep-information so we keep track if a preparation has been reprocessed with another job.
19
+
* Other general fixes, like [#3385](https://github.com/qiita-spots/qiita/pull/3385), [#3397](https://github.com/qiita-spots/qiita/pull/3397), [#3399](https://github.com/qiita-spots/qiita/pull/3399), [#3400](https://github.com/qiita-spots/qiita/pull/3400), [#3409](https://github.com/qiita-spots/qiita/pull/3409), [#3410](https://github.com/qiita-spots/qiita/pull/3410).
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+11
Original file line number
Diff line number
Diff line change
@@ -94,6 +94,17 @@ After the initial production release of Qiita, changes to the database schema wi
94
94
2. We keep fully patched versions of the DBS and HTML files in the repository
95
95
3. We keep a patch file for each patch as required in the `qiita_db/support_files/patches` directory. Note that **the patches will be applied in order based on the natural sort order of their filename** (e.g., `2.sql` will be applied before `10.sql`, and `10.sql` will be applied before `a.sql`)
96
96
97
+
### Patch 91.sql
98
+
99
+
In May 2024 we decided to:
100
+
* Merge all patches into the main database schema, this means that there are no patches younger than 92.sql.
101
+
* Added a new folder `patches/test_db_sql/` where we can store sql files that will only be applied for the test environment.
102
+
* Added a test to the GitHub actions to test that the production database has an expected number of rows.
103
+
104
+
Note that these changes mean:
105
+
1. 92.sql is the current first sql file to patch the database.
106
+
2. If you need to make changes (like INSERTS) _only_ to the tests database you need to add a patch to `patches/test_db_sql/`.
107
+
97
108
### Developer Workflow
98
109
99
110
1. Load the fully patched DBS file (e.g., `qiita-db.dbs`) in [DBSchema](http://www.dbschema.com/)
echo"deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main"|sudo tee /etc/apt/sources.list.d/pgdg.list
84
84
```
@@ -238,6 +238,9 @@ If you are using [NGINX](https://www.nginx.com/) via conda, you are going to nee
238
238
mkdir -p ${CONDA_PREFIX}/var/run/nginx/
239
239
```
240
240
241
+
Note that the shipped nginx version from conda, does **not** contain the mod_zip module: https://github.com/evanmiller/mod_zip
242
+
This leads to unexpected behaviour when generating a download link for anonymous artefact sharing, i.e. Qiita returns a flat file listing artifact filepaths instead of generating a ZIP archive that contains those files. You need to compile nginx with the additional mod_zip module yourself. (I've invested multiple hours to realize that the configure routine does not properly link shared libraries to the nginx binary. Try adding `--with-ld-opt=" -Wl,-rpath,/home/foo/lib "` to the `./auto/configure` call.)
243
+
241
244
## Start Qiita
242
245
243
246
Start postgres (instructions vary depending on operating system and install method).
0 commit comments