Skip to content

Commit

Permalink
Merge branch 'master' into mutation-data-counts
Browse files Browse the repository at this point in the history
  • Loading branch information
7xuanlu authored Feb 28, 2024
2 parents 86cc956 + a7d71b7 commit 88c8b7d
Show file tree
Hide file tree
Showing 25 changed files with 247 additions and 405 deletions.
5 changes: 2 additions & 3 deletions .github/workflows/security-integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,8 @@ jobs:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: 'Add host.testcontainers.internal to /etc/hosts'
run: |
echo "127.0.0.1 host.testcontainers.internal" | sudo tee -a /etc/hosts
- name: 'Download Chrome'
uses: browser-actions/setup-chrome@latest
- name: 'Copy Application.Properties'
working-directory: ./cbioportal
run: |
Expand Down
4 changes: 2 additions & 2 deletions dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ spring.security.oauth2.client.provider.keycloak.issuer-uri=http://localhost:8084
spring.security.oauth2.client.provider.keycloak.user-name-attribute=email
```

4. Set the following in _portal.properties_:
4. Set the following in _application.properties_:

```properties
persistence.cache_type=no-cache
Expand All @@ -55,4 +55,4 @@ spring.datasource.driver-class-name=com.mysql.jdbc.Driver

4. Start cBioPortal application on port 8080. The login credentials are `testuser:P@assword1`.

⚠️ Warning: Do not use this directly for production use as it takes several shortcuts to get a quick keycloak instance up.
⚠️ Warning: Do not use this directly for production use as it takes several shortcuts to get a quick keycloak instance up.
3 changes: 2 additions & 1 deletion docs/404.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ window.redirectMap = {
"2.2-authorization-and-authentication/authenticating-and-authorizing-users-via-keycloak":"deployment/authorization-and-authentication/authenticating-and-authorizing-users-via-keycloak",
"2.2-authorization-and-authentication/authenticating-users-via-tokens":"deployment/authorization-and-authentication/authenticating-users-via-tokens",
"2.3-customization/customizing-your-instance-of-cbioportal":"deployment/customization/customizing-your-instance-of-cbioportal",
"2.3-customization/portal.properties-reference":"deployment/customization/portal.properties-reference",
"2.3-customization/portal.properties-reference":"deployment/customization/application.properties-reference/",
"deployment/customization/portal.properties-reference":"deployment/customization/application.properties-reference/",
"2.3-customization/caching":"deployment/customization/caching",
"2.3-customization/layout":"deployment/customization/studyview/#how-does-the-study-view-organize-the-charts",
"2.3-customization/priority":"deployment/customization/studyview/#study-view-customization-with-priority-data",
Expand Down
44 changes: 42 additions & 2 deletions docs/News.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,15 +108,29 @@
* [Pan-cancer Analysis of Advanced and Metastatic Tumors (BCGSC, Nature Cancer 2020)](https://www.cbioportal.org/study/summary?id=pog570_bcgsc_2020) *570 samples*
* [Prostate Adenocarcinoma (MSK, Clin Cancer Res. 2022)](https://www.cbioportal.org/study/summary?id=prad_pik3r1_msk_2021) *1,417 samples*

## Jun 7, 2022
* **New Feature**: Add heatmap to plot options on _Comparison Page_. [Example: Primary vs Metastatic Prostate Cancer in MSK-IMPACT (2017)](https://www.cbioportal.org/comparison/clinical?sessionId=5cf89323e4b0ab413787436c)

<img src="https://github.com/cBioPortal/cbioportal/assets/1334004/1a048863-7db6-43c6-8250-c31039a2d67f" width="700" />

## May 31, 2022
* **New Feature**: Added Quartiles, Median split and Generate bins options for bar charts on the study view page, where Generate bins allows user to define bin size and min value
<img src="https://user-images.githubusercontent.com/53996876/151772297-f0aa55ef-5bb4-4e26-8f9b-b0a2369fff71.png" width="700" />
<img src="https://github.com/cBioPortal/cbioportal/assets/1334004/290e6d45-1c10-4a51-a823-24db27063486" width="700" />


## May 12, 2022
* **New Feature**: Show cohort alteration frequencies in pathways from [NDEx](https://www.ndexbio.org/iquery/) on the _Results View_. [Example: Glioblastoma signaling pathways in MSK-IMPACT (2017) cohort](https://bit.ly/3sE4UqD)

<img src="https://user-images.githubusercontent.com/1334004/168684594-fafd1fd6-b2a9-45d7-ad3a-d5611f3f469c.png" width="700" />

## May 24, 2022
* **New Feature**: Add Help buttons on various pages and tabs, including the homepage

## May 10, 2022
* **New Feature**: Use IGV for the genomic overview on the _Patient View_. [Example: Endometrial cancer patient in TCGA](https://www.cbioportal.org/patient?studyId=ucec_tcga_pub&caseId=TCGA-BK-A0CC)

<img src="https://github.com/cBioPortal/cbioportal/assets/1334004/6fa6ab4b-4c09-4bd5-a7c2-3be2f418d83a" width="700" />

## May 5, 2022
* **New Feature**: View mutations and copy number changes in the [Integrative Genomics Viewer (IGV)](https://igv.org/) on the _Patient View_. [Example: Endometrial cancer patient in TCGA](https://www.cbioportal.org/patient?studyId=ucec_tcga_pub&caseId=TCGA-BK-A0CC)

Expand All @@ -134,7 +148,7 @@

## Apr 20, 2022
* **Added data** consisting of 2,557 samples from 5 studies:
* [Breast Cancer (HTAN, 2022)](https://www.cbioportal.org/study/summary?id=brca_hta9_htan_2022) *5 samples*
* [Breast Cancer (HTAN OHSU, 2022)](https://www.cbioportal.org/study/summary?id=brca_hta9_htan_2022) *5 samples*
* [Colorectal Cancer (MSK, 2022)](https://www.cbioportal.org/study/summary?id=crc_dd_2022) *47 samples*
* [Pediatric Pancan Tumors (MSK, 2022)](https://www.cbioportal.org/study/summary?id=mixed_kunga_msk_2022) *135 samples*
* [Sarcoma (MSK, 2022)](https://www.cbioportal.org/study/summary?id=sarcoma_mskcc_2022) *2,138 samples*
Expand All @@ -146,6 +160,32 @@
* Pan-can studies methylation addition: methylation profile (27k and 450k merged) are added to all 32 TCGA Pan-Can studies, in generic assay format. Data source: [GDC](https://gdc.cancer.gov/node/977). Example: search by gene or probe from dropdown, to add a chart in study view, a track in Oncoprint (single study query only), or plots in plots tab. <img width="860" alt="Screen Shot 2022-03-22 at 9 52 20 PM" src="https://user-images.githubusercontent.com/5973438/159626976-72601dc9-e5be-4b50-bae9-dedb5be6e7fc.png"><img width="340" alt="Screen Shot 2022-03-22 at 9 52 27 PM" src="https://user-images.githubusercontent.com/5973438/159626985-2423c556-2180-4a00-b9b8-69b4d2e361ba.png"><img width="430" alt="Screen Shot 2022-03-22 at 9 53 38 PM" src="https://user-images.githubusercontent.com/5973438/159626987-e6b51bef-814e-4a76-826a-211fcdd5e9a1.png">

* Single cell (type fraction and phases) data (in generic assay format) is added to [Glioblastoma (CPTAC, Cell 2021)](https://www.cbioportal.org/study/summary?id=gbm_cptac_2021)
* **New Feature** For the new HTAN OHSU study there is now also an integration with [Harvard LSP](https://labsyspharm.org/)'s [Minerva](https://www.cycif.org/software/minerva) for exploring multiplex imaging:

<img src="https://github.com/cBioPortal/cbioportal/assets/1334004/aa0f0c99-4b9c-481d-9f7f-891ea03fdd34" width="700" />


## Mar 1, 2022
* **New Documentation**:
* Add a [User Guide](https://docs.cbioportal.org/user-guide/)
* Add [Study View and Query How-to videos](https://docs.cbioportal.org/user-guide/overview/#how-to-videos)

## Feb 8, 2022
* **New Feature**: Create X vs Y violin plots in _Study View_ using any categorical and numerical clinical data:

<img src="https://github.com/cBioPortal/cbioportal/assets/1334004/94d7dd21-4415-405d-8e9b-46e1de3dc095" width="700" />

## Jan 19, 2022
* **New Feature**: Numerical filters on the _Study View_ are now editable:

<img src="https://github.com/cBioPortal/cbioportal/assets/1334004/00926fec-f32b-48c7-a85f-fced7c29070f" width="700" />
* **New Feature**: In the annotation column choose between showing a single icon OncoKB icon or multiple (one for therapeutic, diagnostic and prognostic):

<img src="https://github.com/cBioPortal/cbioportal/assets/1334004/87c5f7d6-10d7-4631-8b37-fa8cef4cabe2" width="700" />

## Jan 10, 2022
* **New Documentation for Local cBioPortal Installations**:
* Explain how to use [Genome Nexus to annotate MAF files](https://docs.cbioportal.org/file-formats/#create-the-cbioportal-mutation-data-file-with-genome-nexus-with-a-minimal-maf-file)

## Jan 4, 2022
* **Added data** consisting of 27,447 samples from 10 studies:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -789,4 +789,12 @@ By default, the studies loaded into a local cBioPortal instance are organized ba
```
priority_studies=
```
The value of this variable will create a custom category with studies on the top of the study selector view. The format for the string should be category1#study1a,study1b,study1c;category2#study2 (e.g., PanCancer Studies#msk_impact_2017), where the ``category`` can be any string and the ``study`` should be the study ID of the required uploaded study.
The value of this variable will create a custom category with studies on the top of the study selector view. The format for the string should be category1#study1a,study1b,study1c;category2#study2 (e.g., PanCancer Studies#msk_impact_2017), where the ``category`` can be any string and the ``study`` should be the study ID of the required uploaded study.

## Study Tag functionality
Study Tags allow portal maintainers to define miscellaneous descriptive meta data to studies, which will be shown to users in tooltips and are also searchable. This feature
is on by default but can be disabled using the following property.
```
//boolean
enable_study_tags=true|false
```
31 changes: 2 additions & 29 deletions docs/deployment/deploy-without-docker/Build-from-Source.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,37 +2,10 @@

## Building with Maven

While building, you must point the environment variable `PORTAL_HOME` to
the root directory containing the portal source code.

For example, run a command like the following if on macOS:
```
export PORTAL_HOME=/Users/ecerami/dev/cbioportal
```

To compile the cBioPortal source code, move into the source directory and
run the following maven command:
To compile the cBioPortal source code, move into the root directory and run the following maven command:

```
mvn -DskipTests clean install
```

After this command completes, you will find a `cbioportal.war` file suitable
for Apache Tomcat deployment in `portal/target/`. It is not neccessary to
install Tomcat yourself, since a command line runnable version of Tomcat is
provided as a dependency in `portal/target/dependency/webapp-runner.jar`.

However, if you will be deploying to a standalone Tomcat installation, and
if you have configured Tomcat to use the Redisson client for user session
management, you should expect a clash between the Redisson client being
used for session management and the Redisson client which is embedded in
the cbioportal.war file for the optional "redis" persitence layer caching
mode. In this case, you should avoid using the "redis" option for the portal
property `persistence.cache_type` and you should prevent the Redisson
client from being packaged in cbioportal.war by building with this command
instead:

##### alternative for standalone tomcat deployments which use redis session management
```
mvn -Dexclude-redisson -DskipTests clean install
```
Note: cBioPortal 6.X requires Java 21
124 changes: 19 additions & 105 deletions docs/deployment/deploy-without-docker/Deploying.md
Original file line number Diff line number Diff line change
@@ -1,131 +1,45 @@
# Deploying the Web Application

## Prepare the global configuration file
## Before running cbioportal backend

The portal is configured using a global configuration file, `application.properties`. An example file is available in the `src/main/resources` folder. Use it as a template to create your own:
You will need to update the src/main/resources/applications.properties to include your DB connection information.

```
cd src/main/resources
cp application.properties.EXAMPLE $HOME/cbioportal/application.properties
```

For more information about the `application.properties` file, see the [reference](/deployment/customization/Customizing-your-instance-of-cBioPortal.md) page.

Several scripts of cBioPortal use this `application.properties` file to get info like db connection parameters. You can indicate the folder where this file is with an environment variable:
The configuration defined in `application.properties` can also be passed as command line arguments. The priority of property loading is as follows:

```
export PORTAL_HOME=$HOME/cbioportal
```
1. `-D` command line parameters overrides all
2. `src/main/resources/application.properties`
3. `application.properties` supplied at compile time
4. Defaults defined in code

if your properties file is at `PORTAL_HOME/application.properties`
Note that the `authenticate` property is currently required to be set as a command line argument, it won't work when set in `application.properties` (See issue [#6109](https://github.com/cBioPortal/cbioportal/issues/6109)).

## Run cBioPortal Session Service
Some scripts require a `${PORTAL_HOME}/application.properties` file, so it is best to define the properties there.

The cBioPortal app requires [session service](/Architecture-Overview.md#session-service). For instructions on how to run this without Docker see https://github.com/cBioPortal/session-service#run-without-docker. Once this is working, update the properties file:
For more information about the `application.properties` file, see the [reference](/deployment/customization/Customizing-your-instance-of-cBioPortal.md) page.

```bash
# session-service url: http://[host]:[port]/[session_service_app]/api/sessions/[portal_instance]/
# example session-service url: http://localhost:8080/session_service/api/sessions/public_portal/
# see: https://github.com/cBioPortal/session-service
session.service.url=
```

## Run the cbioportal backend

To run the app we use `webapp-runner`. It's a command line version of Tomcat provided by [Heroku](https://github.com/jsimone/webapp-runner). All parameters can be seen with:
To run the backend execute the following commabd

```
java -jar portal/target/dependency/webapp-runner.jar --help
```

This runs the app in the foreground. If a port is already in use it will raise an error mentioning that. To change the port use the `--port` flag.

There are three main ways to run the portal: without authentication, with optional login and with required login. All of them require the cBioPortal session service to be running.

### Without authentication

In this mode users are able to use the portal, but they won't be able to save their own virtual studies and groups. See the [optional login section](#optional-login) to enable this.

```bash
java \
-jar \
-Dauthenticate=noauthsessionservice \
portal/target/dependency/webapp-runner.jar \
portal/target/cbioportal.war
java -jar target/cbioportal-exec.jar
```

### Optional login
This runs the app in the foreground. If a port is already in use it will raise an error mentioning that. To change the port use the `--server.port` flag.

In this mode users can see all the data in the portal, but to save their own groups and virtual studies they are required to log in. This will allow them to store user data in the session service. See the [tutorials](https://www.cbioportal.org/tutorials) section to read more about these features.

```bash
java \
-jar \
-Dauthenticate=social_auth_google,social_auth_microsoft \
portal/target/dependency/webapp-runner.jar \
portal/target/cbioportal.war
```

Google and Microsoft live are supported as optional login currently. Possible values for authenticate are
There are three main ways to run the portal: without authentication, with optional login, and with required login. All of them require the cBioPortal session service to be running.

```bash
-Dauthenticate=social_auth_google,social_auth_microsoft
-Dauthenticate=social_auth_google
-Dauthenticate=social_auth_microsoft
```

One needs to set the Google/Microsoft related configurations in the `application.properties` file:
### Without authentication

```
#For Google
googleplus.consumer.key=
googleplus.consumer.secret=
In this mode users are able to use the portal, but they won't be able to save their own virtual studies and groups.

#For Microsoft
microsoftlive.consumer.key=
microsoftlive.consumer.secret=
```

See [Google's Sign in Documentation](https://developers.google.com/identity/sign-in/web/sign-in#before\_you\_begin) to obtain these values.

See [Microsoft Sign in Documentation](https://docs.microsoft.com/en-us/azure/active-directory/develop/scenario-web-app-sign-user-app-registration) to obtain these values.

### Required login

```bash
java \
-Dauthenticate=CHOOSE_DESIRED_AUTHENTICATION_METHOD \
-jar \
portal/target/dependency/webapp-runner.jar \
portal/target/cbioportal.war
java -jar target/cbioportal-exec.jar -Dauthenticate=false
```

Change `CHOOSE_DESIRED_AUTHENTICATION_METHOD` to one of `googleplus`, `saml`, `openid`, `ad`, `ldap`. The various methods of authentication are described in the [Authorization and Authentication](/deployment/authorization-and-authentication) section.

### Property configuration

The configuration defined in `application.properties` can also be passed as command line arguments. The priority of property loading is as follows:

1. `-D` command line parameters overrides all
2. `${PORTAL_HOME}/application.properties`
3. `application.properties` supplied at compile time
4. Defaults defined in code

Note that the `authenticate` property is currently required to be set as a command line argument, it won't work when set in `application.properties` (See issue [#6109](https://github.com/cBioPortal/cbioportal/issues/6109)).

Some scripts require a `${PORTAL_HOME}/application.properties` file, so it is best to define the properties there.

### Note for Tomcat Deployers

Before we were using `webapp-runner`, our documentation recommended a system level installed Tomcat. In this case people might have been using `dbconnector=jndi` instead of the new default `dbconnector=dbcp`. There is a known issue where setting dbconnector in the properties file does not work ([#6148](https://github.com/cBioPortal/cbioportal/issues/6148)). It needs to be set as a command line argument. For Tomcat this means `CATALINA_OPT="-Ddbconnector=jndi"`.

## Verify the Web Application

Lastly, open a browser and go to:\
[http://localhost:8080](http://localhost:8080)

## Important
### With authentication

* Each time you modify any java code, you must recompile and redeploy the app.
* Each time you modify any properties (see customization options), you must restart the app
* Each time you add new data, you must restart the app or call the `/api/cache` endpoint with a `DELETE` http-request (see [here](/deployment/customization/application.properties-Reference.md#evict-caches-with-the-apicache-endpoint) for more information).
To configure the authentication and authorization please consult the [Authorization](./../authorization-and-authentication/User-Authorization.md) and [Authentication](./../authorization-and-authentication/Authenticating-and-Authorizing-Users-via-keycloak.md) Sections.
Loading

0 comments on commit 88c8b7d

Please sign in to comment.