Qiita 0.1.0: alpha release

We are pleased to announce the first alpha release of Qiita (:feet: try it out now 🐾), thanks to our contributors and active users of the system. We’ve come a long way since our pre-alpha release, the following is a quick summary of the major changes that occurred during this development cycle:

Metadata constraints have now been relaxed

When you upload a sample and a prep template, you are required to include only the minimal information needed for processing. This allows users to quickly get from sequence to preliminary results. As always, we recommend that you standardize your metadata to be compliant with the GSC ontologies where possible. Processing of some tasks is disabled if needed information is not supplied by the user, so pay close attention to messages from the system regarding your metadata. For example, raw sequences cannot be demultiplexed if your prep template does not contain a “barcode” column. Please see a full description here.

Improved analysis pipeline

Finding the samples you want to use in a new analysis is now more intuitive. You can browse your studies in a single view, search by sample or study metadata, and add samples to analyses using a familiar “shopping cart”-like interface (see the animated GIF below):

Revamped study management interface

You can now update metadata columns and values or even add new samples to an existing sample template. We’ve also added the ability to delete existing studies and their different components (processed data, preprocessed data, sample template, etc.).

We have improved study privacy settings

You can now change the privacy status of your study's files on a per-processed data basis (a processed data for a 16S analysis is typically an OTU table). A processed data (and the study, by extension) can have one of four possible states: sandboxed, awaiting approval, private, and public. A sandboxed processed data is only visible to the owner and other users with whom the processed data is shared but can be combined with other data available to those users in analyses as usual. Once the user is satisfied with the state of the processed data, he or she can ask for approval from the Qiita admins (QA) to make it private, in which case the processed data changes to the awaiting approval state. To move to a private state, the QA will check that all metadata is MiMARKs compliant and that it processed correctly. In this state the sequence data can be submitted to EBI. The processed data (and hence the study) becomes public when the user explicitly makes the processed data public.

Miscellaneous

• Based on your feedback we have improved the error messages shown on screen so they are more informative.
• Browser compatibility has been improved.
• Quality control, demultiplexing, and OTU picking are now parameterized so you can select from a variety of parameters. Make sure you let us know if you would like to see new parameters.
• Qiita is powered by QIIME v1.9.0.
• The deployment of Qiita maintained by the Knight lab has now officially moved to the University of California, San Diego. Our new DNS is: http://qiita.microbio.me and you should use this URL if you want to reference the central system.
• As always we encourage you to participate in the development of Qiita by:
  • Reporting a bug or feature request in our issue tracker.
  • Submitting a new feature (if you are curious please review our development guidelines).
  • Adding your study to the system, which should not only facilitate its analysis but will allow you to combine it with other public studies or even share it with other users.

Qiita 0.0.1: pre-alpha release

The qiita-dev-team is delighted to let you know that qiita.colorado.edu is up for public initial testing. The goal of this pre-alpha release is to get feedback from users, and attract new developers.

In the current system, you will be able to create new accounts and manage them. Additionally, you will be able to create studies, add sample and prep templates, and upload raw sequences. Note that the templates must follow the MIMARKS standard, and that we currently only support raw (without pre-demultiplexing) FASTQ sequences as initial input.

Qiita relies on the concept of studies, where each study has one sample template (metadata about the samples), and multiple raw data files (e.g. different sequencing runs) where each run has a different preparation (prep) templates (run information). This structure allows us to support diverse sequencing strategies, like having different preparations (e.g. ITS and 16S on the same samples) in a single sequencing run.

Once a sample and prep template are uploaded and added to the database, you can visualize summaries of each one of them, redownload them, and get a valid QIIME mapping file for further analysis. Additionally, you can download the post-demultiplexing raw sequence files generated by the system. For those studies, where we have preprocessed these sequences using the new tools that will be available in QIIME 1.9.0, more below, you will be able to also download the processed data, also know as the close reference biom table.

A test subset of the Earth Microbiome Project studies (around 100) have been processed using the new pipelines that will be available in QIIME 1.9.0, release due on December 3rd, 2014. Meta-analyses (beta, alpha, and taxonomy analysis of single or combined studies) can be tested on the live system. Note that the demultiplexed sequences for these studies are not available yet, but should be available within the next couple of weeks. Also, the system still does not support automatic creation of processed data (e.g. otu picking), although again this will be done soon.

This new system is capable of automatically submitting pre-processed sequences, for example those produced after quality control and demultiplexing, to the European Bioinformatics Institute. During this pre-alpha phase, we only submit sequences processed within the system and by requesting an admin to do it. The submission is per prep template.

Another new feature in the system is that you can share your study with other registered users. This new feature allows collaboration and addresses cases where the raw files exist at a different site.

As noted in the text qiita.colorado.edu will reflect the current status of our master development branch, meaning that as improvements come along they will be proactively deployed. To ensure no data is lost throughout this development phase we have setup an internal replication system of the database.

If you want to try out Qiita in your own system see our installation notes.

Finally, remember that this is a pre-release, so please be patient and if you have some spare time, we need help both on finding issues and on fixing existing ones. If you find new issues with the live system, have suggestions or possible improvements, add them to the issue tracker. In the next week we will be adding more documentation so local installs and testing is easier.