Generating a Gene Expression Matrix

Most analyses have two stages: data reduction and data analysis.

Statistical analyses of scRNA-seq data take as their starting point an expression matrix, where each row represents a gene and each column represents a sample (in scRNAseq columns are cells). Each entry in the matrix represents the number of reads (proxy for expression level) of a particular gene in a given sample (cell). Generating the expression matrix often involves some, or all, of the following.

Flowchart of scRNAseq analysis

Because we are working with 10x Genomics data, we will be using the 10x Cell Ranger pipeline to handle the first column (data reduction tasks).

Preprocessing and mapping reads

Raw fastq files first need to be preprocessed, extracting any elements that are a part of the sequence read and potentially “cleaned” with applications such as HTStream.

• Library Barcode (library Index) - Used to pool multiple libraries on one sequencing lane
• Cell Barcode (10x Barcode) – Used to identify the cell the read came from
• Unique Molecular Index (UMI) – Used to identify reads that arise during PCR replication
• Sequencing Read – Used to identify the gene a read came from

The remaining sequences are mapped to a reference genome/trancriptome. Cell Ranger is based on STAR aligner. Another good choice is a pseudo-alignment method (e.g. Kallisto, Salmon). For full-length datasets with tens- or hundreds of thousands of reads per cell pseudo-aligners become appealing since their run-time can be several orders of magnitude less than traditional aligners.

Note, if spike-ins are used, the spike-in sequences should be added to the reference sequence prior to mapping.

Mapping QC

After aligning sequence data to the genome we should evaluate the quality of the mapping. There are many ways to measure the mapping quality, including: percentage mapped, amount of reads which map to rRNA/tRNAs, proportion of uniquely mapped reads, reads mapping across splice junctions, read depth along the transcripts, etc. Methods developed for bulk RNAseq, such as RSeQC and samtools, are applicable to single-cell data.

Gene Counting

STAR (and by extension, Cell Ranger), Kallisto, and Salmon all quantify the expression level of each gene for each cell as a part of its output. If UMIs were used, duplicates need to be first marked and then gene expression levels recounted. The package UMI-tools can be used to process and correct UMIs. Cell Ranger handles UMI deduplication and cell barcode assignment internally.

Specific steps to be performed are dependent on the type of library, the element layout of the read, and the sequencing parameters.

STAR, Salmon, Kallisto/bustools each have pipelines build specifically for processing single-cell datasets and 10X genomics data.

scRNAseq Libraries

Generating scRNAseq libraries is currently an active area of research with several protocols being published in the last decade, including:

Flowchart of scRNAseq library prep protocols

Svensson, etc., 2018, Nature Protocols https://www.nature.com/articles/nprot.2017.149

• CEL-seq, Hashimshony, 2012
• CEL-seq2, Hashimshony, 2016
• DroNC-seq, Habib, 2017
• Drop-seq, Macosko, 2015
• InDrop-seq, Klein, 2015
• MATQ-seq, Sheng, 2017_2018
• MARS-seq, Jaitin, 2014
• SCRB-seq, Soumillon, 2014
• Seq-well, Gierahn, 2017
• Smart-seq, Picelli, 2014
• Smart-seq2, Picelli, 2014
• SMARTer clontech
• STRT-seq, Islam, 2014
• SplitSeq, 2018 (NOW Parse Biosciences)
• Smart-seq3, Hagemann-Jensen, 2020
• Smart-seq-total, Isakova, 2021
• VASA-seq, Salmen, 2022

Differences between the methods are in how they capture a cell and quantify gene expression (either full-length or tag-based).

Full-length capture tries to achieve a uniform coverage of each transcript (many reads per transcript). Tag-based protocols only capture either the 5’- or 3’-end of each transcript (single read per transcript). Choice in method determines what types of analyses the data can be used for. Full-length capture can be used to distinguish different iso-forms, where tag-based method is best used for only gene abundance.

• Tag-based 3’ counting techniques
  • 1 read per transcript
  • Based on polyA
  • Expression analysis only
  • Fewer reads per cell needed (~20K reads/cell 10x V3+)
  • Less noise in expression patterns
• Full-length
  • Based on polyA
  • Expression analysis
  • Splicing information
  • The more information desired beyond expression, the higher the reads needed per cell (~50K reads/cell to 10M reads/cell)

For smaller experiments < 5000 cells, the R packages SingleCellExperiment, scater, SC3 are good choices. For larger experiments (> 5000 cells), the R package Seurat offers a complete solution.

If you prefer Python, scanpy is a good choice.

A nice page keeping track of single-cell software can be found here.

10X Genomics generation of expression matrix with cellranger

10X Genomics Cell Ranger uses a fork of the STAR aligner, Orbit, to map reads to a genome after first preprocessing them (extracting cell and UMI sequences).

Elements to a 10x read (V3)

cellranger version 7 has many sub-applications

1. cellranger mkfastq

2. cellranger count
3. cellranger vdj

4. cellranger multi

5. cellranger aggr
6. cellranger reanalyze
7. cellranger mat2csv
8. cellranger targeted-compare

9. cellranger targeted-depth

10. cellranger mkref
11. cellranger mkgtf

12. cellranger mkvdjref

13. cellranger testrun
14. cellranger upload
15. cellranger sitecheck

Cell barcode and UMI filtering

• Cell barcodes
  • Must be on static list of known cell barcode sequences
  • May be 1 mismatch away from the list if the mismatch occurs at a low- quality position (the barcode is then corrected).
• UMIs (Unique Molecular Index)
  • Must not be a homopolymer, e.g. AAAAAAAAAA
  • Must not contain N
  • Must not contain bases with base quality < 10
  • UMIs that are 1 mismatch away from a higher-count UMI are corrected to that UMI if they share a cell barcode and gene.

Read Trimming

Cell Ranger only performs read trimming to 3’ gene expression assays.

A full length cDNA construct is flanked by the 30 bp template switch oligo (TSO) sequence, AAGCAGTGGTATCAACGCAGAGTACATGGG, on the 5’ end and poly-A on the 3’ end. Some fraction of sequencing reads are expected to contain either or both of these sequences, depending on the fragment size distribution of the sequencing library. Reads derived from short RNA molecules are more likely to contain either or both TSO and poly-A sequence than longer RNA molecules.

In order to improve mapping, the TSO sequence is trimmed from the 5’ end of read 2 and poly-A is trimmed from the 3’ end prior to alignment.

Tags ts:i and pa:i in the output BAM files indicate the number of TSO nucleotides trimmed from the 5’ end of read 2 and the number of poly-A nucleotides trimmed from the 3’ end, respectively. The trimmed bases are present in the sequence of the BAM record and are soft clipped in the CIGAR string.

Alignment

Genome Alignment

Cell Ranger uses an aligner called Orbit (a wrapper around STAR), which performs splicing-aware alignment of reads to the genome. Cell Ranger uses the transcript annotation GTF to bucket the reads into exonic, intronic, and intergenic, and by whether the reads align (confidently) to the genome. A read is exonic if at least 50% of it intersects an exon, intronic if it is non-exonic and intersects an intron, and intergenic otherwise.

MAPQ adjustment

For reads that align to a single exonic locus but also align to 1 or more non-exonic loci, the exonic locus is prioritized and the read is considered to be confidently mapped to the exonic locus with MAPQ 255.

Transcriptome Alignment

Cell Ranger further aligns exonic reads to annotated transcripts, looking for compatibility. A read that is compatible with the exons of an annotated transcript, and aligned to the same strand, is considered mapped to the transcriptome. If the read is compatible with a single gene annotation, it is considered uniquely (confidently) mapped to the transcriptome. Only reads that are confidently mapped to the transcriptome are used for UMI counting.

In certain cases, such as when the input to the assay consists of nuclei, there may be high levels of intronic reads generated by unspliced transcripts. In order to count these intronic reads, the cellranger count and cellranger multi pipelines can be run with the option ‘include-introns’ in versions 5.0-6.1. Starting from version 7.0, cellranger count includes the intronic reads in quantification of gene expression by default. For cellranger multi, one can set include-introns to true to include the intronic reads.

UMI Counting

• Using only the confidently mapped reads with valid barcodes and UMIs,
  • Correct the UMIs UMIs are corrected to more abundant UMIs that are one mismatch away in sequence (hamming distance = 1).
  • Record which reads are duplicates of the same RNA molecule (PCR duplicates)
  • Count only the unique UMIs as unique RNA molecules
  • These UMI counts form an unfiltered gene-barcode matrix.

Filtering cells (the 10x way)

Cell Ranger 3.0 introduced and improved cell-calling algorithm that is better able to identify populations of low RNA content cells, especially when low RNA content cells are mixed into a population of high RNA content cells. For example, tumor samples often contain large tumor cells mixed with smaller tumor infiltrating lymphocytes (TIL) and researchers may be particularly interested in the TIL population. The new algorithm is based on the EmptyDrops method (Lun et al., 2019).

The algorithm has two key steps:

1. It uses a cutoff based on total UMI counts of each GEM barcode to identify cells. This step identifies the primary mode of high RNA content cells.
2. Then the algorithm uses the RNA profile of each remaining barcode to determine if it is an “empty” or a cell containing partition. This second step captures low RNA content cells whose total UMI counts may be similar to empty GEMs.

In the first step, the original cell calling algorithm is used to identify the primary mode of high RNA content cells, using a cutoff based on the total UMI count for each barcode. Cell Ranger may take as input the expected number of recovered cells, N (see –expect-cells). Let m be the 99th percentile of the top N barcodes by total UMI counts. All barcodes whose total UMI counts exceed m/10 are called as cells in the first pass.

In the second step, a set of barcodes with low UMI counts that likely represent ‘empty’ GEM partitions is selected. A model of the RNA profile of selected barcodes is created. This model, called the background model, is a multinomial distribution over genes. It uses Simple Good-Turing smoothing to provide a non-zero model estimate for genes that were not observed in the representative empty GEM set. Finally, the RNA profile of each barcode not called as a cell in the first step is compared to the background model. Barcodes whose RNA profile strongly disagrees with the background model are added to the set of positive cell calls. This second step identifies cells that are clearly distinguishable from the profile of empty GEMs, even though they may have much lower RNA content than the largest cells in the experiment.

Below is an example of a challenging cell-calling scenario where 300 high RNA content 293T cells are mixed with 2000 low RNA content PBMC cells. On the left is the cell calling result with the cell calling algorithm prior to Cell Ranger 3.0 and on the right is the current Cell Ranger 3.0 result. You can see that low RNA content cells are successfully identified by the new algorithm.

Matrix output

Each of these folders contains three files needed to completely describe each gene x cell matrix:

• matrix.mtx.gz
• features.tsv.gz
• barcode.tsv.gz

Matrix HDF5 output

HDF5 is a file format designed to preserve a hierarchical, filesystem-like organization of large amounts of data:

(root)
└── matrix [HDF5 group]
    ├── barcodes
    ├── data
    ├── indices
    ├── indptr
    ├── shape
    └── features [HDF5 group]
        ├─ _all_tag_keys
        ├─ target_sets [for Targeted Gene Expression]
        │   └─ [target set name]
        ├─ feature_type
        ├─ genome
        ├─ id
        ├─ name
        ├─ pattern [Feature Barcode only]
        ├─ read [Feature Barcode only]
        └─ sequence [Feature Barcode only]

The result is a single file completely describing the gene x cell matrix, which can be read into R or Python for downstream processing.

The hdf5 has a number of advantages we’ll talk more about when we get into data analysis.

BAM output

10x Chromium cellular and molecular barcode information for each read is stored as TAG fields:

The following TAG fields are present if a read maps to the genome and overlaps an exon by at least one base pair. A read may align to multiple transcripts and genes, but it is only considered confidently mapped to the transcriptome it if mapped to a single gene.

The bits to the extra alignment flags are interpreted as follows:

• 1 - The read is confidently mapped to a feature
• 2 - The read maps to a feature that the majority of other reads with this UMI did not
• 4 - This read pair maps to a discordant pair of genes, and is not treated as a UMI count
• 8 - This read is representative for a transcriptomic molecule and can be treated as a UMI count
• 16 - This read maps to exactly one feature, and is identical to bit 8 for transcriptomic reads. Notably, this bit is set when a feature barcode read is treated as a UMI count, while bit 8 is not
• 32 - This read was removed by targeted UMI filtering.

The following are feature barcoding TAG fields which are not aligned to the genome, but processed by the Feature Barcoding read processor. The BAM file will contain unaligned records for these reads, with the following tags representing the Feature Barcode sequence extracted from the read, and the feature reference it was matched to, if any. The BAM read sequence will contain all the bases outside of the cell barcode and UMI regions. V3 ONLY.

An example read

Cell Ranger Version 6

A01102:107:HHM5TDSXY:3:1128:6659:34601	147	chr1	1014050	255	151M	=	1013467	-734	TCGGTGTCAGAGCTGAAGGCGCAGATCACCCAGAAGATCGGCGTGCACGCCTTCCAGCAGCGTCTGGCTGTCCACCCGAGCGGTGTGGCGCTGCAGGACAGGGTCCCCCTTGCCAGCCAGGGCCTGGGCCCCGGCAGCACGGTCCTGCTGG	FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF	NH:i:1	HI:i:1	AS:i:249	nM:i:2	RG:Z:PBMC2sm:0:1:HHM5TDSXY:3	TX:Z:ENST00000624652,+271,151M;ENST00000624697,+296,151M;ENST00000649529,+146,151M	GX:Z:ENSG00000187608	GN:Z:ISG15	fx:Z:ENSG00000187608	RE:A:E	xf:i:25	CR:Z:CCGTGGAGTCAGAGGT	CY:Z:FFFFFF:FFFFFFFFF	CB:Z:CCGTGGAGTCAGAGGT-1	UR:Z:CCACGGGGAT	UY:Z:FFFFFFFFFF	UB:Z:CCACGGGGAT

10X Genomics sample report

A summary of the alignment and assignment of reads to cells and genes is present in the metrics_summary.csv.

10X genomics html reports

Cell Ranger does produce a more readable HTML report with the same statistics and some “analysis”.

Cell Ranger V7 web summary

Exercises

1. Log into tadpole with the username/password

    cd /share/workshop/scRNA_workshop/$USER/scrnaseq_example
   

2. Load and review cellranger’s sub-applications and help docs

    module load cellranger/7.0.0
   

3. Review the cellranger-counts.sh script used to map reads in the fastq files.

4. Copy contents of the script to your scrnaseq_example folder and do a test run.

    cd /share/workshop/scRNA_workshop/$USER/scrnaseq_example
    wget https://raw.githubusercontent.com/ucdavis-bioinformatics-training/2022-July-Single-Cell-RNA-Seq-Analysis/main/software_scripts/scripts/cellranger-counts.sh
   

   update the email address in the script if you like.

    sbatch cellranger-counts.sh
   

5. Link completed result folders to your scrnaseq_example folders.

    cd /share/workshop/scRNA_workshop/$USER/scrnaseq_example
    ln -s /share/workshop/scRNA_workshop/cellranger.outs/A001-C-007 ./A001-C-007-copy
   

   1. In the folder A001-C-007-copy, which output folders/files were generated from this script?
   2. Review the metrics_summary.csv file
      1. What where the total number of reads in this sample?
      2. Reads Mapped Confidently to transcriptome?
      3. Sequencing Saturation?
      4. Mean Reads per Cell?
      5. Median UMI Counts per Cell?
   3. head the files under raw_feature_bc_matrix and filtered_feature_bc_matrix
   4. Transfer the html file to your computer
   5. Transfer the matrix files and hdf5 file to your computer. (However, we will be using data from the full datasets for all three samples instead of this subset).

In the intereste of time, the dataset we use for this step is a small subset of the original data. The cellranger summary file is here

Bonus: cellranger features and multi pipeline

Feature barcodes allow you to capture additional information within your cells by using an addition oligo on the GEM beads. This can be from Antibody capture, Crispr guide capture, or a custom capture (like hash tagging).

To do so you need to pass a library csv file and a feature (only 1 feature possible at a time) reference file.

```
cellranger count --id=sample \
                 --libraries=library.csv \
                 --transcriptome=/opt/refdata-gex-GRCh38-2020-A \
                 --feature-ref=feature_ref.csv \
                 --expect-cells=1000    ```

Library csv file

3 columns

• fastq - path to fastq files
• sample - name of the fastq file
• library_type one of Gene Expression, Custom, Antibody Capture, or CRISPR Guide Capture

fastqs,sample,library_type
/opt/foo/,GEX_sample1,Gene Expression
/opt/foo/,CRISPR_sample1,CRISPR Guide Capture

Feature Reference file

• id - Unique ID
• name - Human-readable name
• read - Which read do I expect to find the feature barcode in
• pattern - The pattern within the read
• sequence - The barcode sequence
• feature_type - Type of feature, same as above

id,name,read,pattern,sequence,feature_type
CD3,CD3_TotalC,R2,^NNNNNNNNNN(BC)NNNNNNNNN,CTCATTGTAACTCCT,Antibody Capture
CD19,CD19_TotalC,R2,^NNNNNNNNNN(BC)NNNNNNNNN,CTGGGCAATTACTCG,Antibody Capture
CD45RA,CD45RA_TotalC,R2,^NNNNNNNNNN(BC)NNNNNNNNN,TCAATCCTTCCGCTT,Antibody Capture
CD4,CD4_TotalC,R2,^NNNNNNNNNN(BC)NNNNNNNNN,TGTTCCCGCTCAACT,Antibody Capture
CD8a,CD8a_TotalC,R2,^NNNNNNNNNN(BC)NNNNNNNNN,GCTGCGCTTTCCATT,Antibody Capture

See Feature Barcode Analysis for more information.

Cellranger multi

Cell Ranger 6.0 introduced the multi pipeline which is required for use with CellPlex, and can can be used to ‘join’ features, V(D)J, and counts into a single analysis.

cellranger multi requires an ID for output and a configuration csv (which really isn’t a csv).

Multi Config CSV

Section: [gene-expression]

Section: [feature]

Section: [libraries] (see also Specifying Input FASTQ Files for cellranger multi)

Section: [samples]