PARAMETER SENSITIVITY ANALYSIS

Sensitivity analysis measures how the variation of model parameters affects system behavior. By systematically varying different parameter inputs, we can identify which parameters have the strongest influence on the system’s output. Using these results, we can adjust our models to give us the best performance.

Sensitivity analysis is especially important due to the biological system’s natural variability. Factors such as transcription rates, protein expression levels, and enzyme activities can fluctuate between experiments and across different cells. Results can be used to:

Predict which rate constants are most critical for antiviral success
Identify parameters that may introduce risk
Improve experimental design by focusing on measurements and optimizations of the most influential parameters

LOCAL SENSITIVITY ANALYSIS

To evaluate the robustness of our Cas13a-HIV model, we conducted a parameter sweep by systematically varying seven key parameters: k_RNP, k_deg,gRNA, k_deg,NS, k_deg,HIV, k_cis, k_{k_transNS}, k_{k_transHIV}. Each parameter was scaled at small variation local to its baseline and large variation further from its baseline, ranging from 0.5x to 2.5x baseline value. The effect of each altered variable was measured while all other parameters were held constant, as is standard in local sensitivity analysis. For each set, the area under the curve (AUC) of HIV concentration over time was computed, and deviations in AUC relative to baseline values were compared to quantify parameter sensitivity.

Figure 1: Heatmap showing the absolute percent change in HIV RNA AUC for each parameter across scaling factors (0.5, 1, 1.5, 2, 2.5). Only the magnitude of the effect is shown, ignoring direction. Higher values (yellow) indicate parameters with a stronger impact on HIV RNA cleavage, while lower values (blue) indicate weaker impact.

Figure 2: Bar graph showing the percent change in HIV RNA AUC for each parameter at the 2.5× scaling factor, including the direction of the effect. Positive values indicate that increasing the parameter decreases HIV RNA cleavage, while negative values indicate that increasing the parameter increases cleavage.

Our analysis revealed that our model is most influenced by parameters directly associated with cleavage activity. We find that increases in k_RNP and k_transHIV produced the greatest reductions in the HIV RNA AUC, indicating that faster binding and trans-cleavage rates substantially enhance RNA degradation efficiency. The heatmap highlights this trend across all scaling factors, showing which parameters have the greatest impact on the model (Figure 1). The bar graph shows the directionality of each parameter’s effect at 2.5× baseline, indicating which parameters enhance cleavage rates and which reduce them (Figure 2).

GLOBAL SENSITIVITY ANALYSIS

While local sensitivity analysis (LSA) examines how small perturbations in model parameters affect output, global sensitivity analysis (GSA) examines system behaviour across a much wider range of the input space, accounting for large variations in parameter values. Our AUC analysis takes into account large parameter variations in addition to local ones, but GSA is functionally different from LSA in that, instead of varying individual parameters while holding all others fixed, it repeatedly varies all parameters by taking randomized sample values from the input space. These values are then used to determine the effect of individual parameters on overall model output.

METHODS

In order to perform GSA, we created SimBiology models of our in-vitro and in-vivo systems, depicted below.

Figure 3: SimBiology model of in-vitro system. Red circles indicate degradation of a variable and yellow circles indicate reactions between two or more variables.

Figure 4: SimBiology model of in-vivo system. Green circles indicate synthesis of a variable.

Latin Hypercube Sampling (LHS) was used to collect sample parameter values from the input spaces. LHS, also referred to as near-random or quasi-random sampling, is a statistical method for generating random samples. When generating samples for n variables, the range of each variable is divided into m equally probable intervals. Sample values are then taken from each interval. Unlike many other sampling methods, LHS is not memoryless, meaning it takes into account where previous sample points were placed when generating new sample points. Specifically, it samples n different values across the division of m intervals, with no sample being in the same row or column as another sample. The final configuration can be compared to having n rooks on a chessboard without threatening each other (Li and Yang). This sampling strategy ensures an even spread of sample values representative of real variability, which gives LHS a distinct advantage over memoryless sampling methods, where sample values can be unevenly spread and unintentionally clustered.

These sample values were then used to calculate first order and total order Sobol indices for both models. Sobol’s method employs variance-based sensitivity analysis to calculate the fractional contribution of each input parameter to the output variance, based on the variability of both individual input parameters and the interaction between multiple input parameters. The first order Sobol index (S_i) calculates the effect of an input parameter X_i without taking into account interactions between X_i and other input parameters. It is considered to be the main effect of that parameter.

To calculate the total variance of the output Y contributed to all input parameters X₁, X₂, …, X_d, the variance can be decomposed as follows:

where V_i denotes the variance attributed to individual parameters and V_ij, …, V_12...d denote the variance attributed to the interactions between these parameters. From here, the equation for S_i can be defined:

Mathematically, Si is the variance of E(Y|X_i), the expected value of the output Y conditional to all possible variations in a fixed input parameter X_i, divided by the total variance of all input parameters to provide the fractional contribution of X_i.

The total-order Sobol index S_{T_i} includes the first-order effects calculated by S_i as well as higher-order interaction effects of X_i with all other input parameters:

E(Y|X~_i) denotes the expected value of the output Y conditional to all possible variations in the set of all input parameters except for X_i. Dividing Var[E(Y|X~_i)] by the total variance of all input parameters provides the fractional contribution of X~_i, so subtracting this value from one gives the fractional contribution of X_i, including higher order effects, because the sum of all variances is equal to one. Since S_i and S_Ti provide the fractional contribution of each input parameter to the output variance, 0 ≤ S_i ≤ 1 and 0 ≤ S_Ti ≤ 1. The closer to 1 the Sobol index is, the more influential the input parameter is on output variance.

IN-VITRO:

Figure 5. Bar graph showing the first order Sobol index (blue) and total order Sobol index (red) for each in-vitro input parameter, with sensitivity output HIV. Darker colors indicate values that occur more often over the course of the simulation.

The magnitudes of the sensitivities indicate that the HIV concentration is most sensitive to k_transHIV, k_RNP, and k_deg,HIV, which aligns with the findings in the AUC sensitivity analysis. Since the first order and total order Sobol indices are nearly identical, variance attributed to interactions between input parameters is minimal.

Figure 6. Graphs depicting first order and total order in-vitro Sobol index values with respect to time. All variance is contributed to the sensitivity output HIV. Input parameters with negligible effect on output variance not shown.

As depicted above, sensitivity to k_deg,HIV is at its highest at the beginning of the simulation, likely due to being the only input parameter affecting the concentration of HIV before RNP complex formation. This sensitivity dramatically decreases as sensitivity to k_transHIV increases. k_transHIV sensitivity remains high and approaches one throughout the rest of the simulation, indicating it is the most influential parameter on HIV concentration.

IN-VIVO:

Figure 7. Bar graph showing the first order Sobol index (blue) and total order Sobol index (red) for each in-vivo input parameter, with sensitivity output HIV. Darker colors indicate values that occur more often over the course of the simulation.

The magnitudes of the sensitivities indicate that the HIV concentration is most sensitive to the production of gRNA and HIV, k_cis, and k_transHIV. Since the first order and total order Sobol indices are nearly identical, variance attributed to interactions between input parameters is minimal. Unlike the in-vitro system, the majority of the variance is not attributed to k_transHIV but to k_syn,HIV and k_syn,gRNA.

Figure 8. Graphs depicting first order and total order in-vivo Sobol index values with respect to time. All variance is contributed to the sensitivity output HIV. Input parameters with negligible effect on output variance not shown.

As depicted above, sensitivity to k_syn,HIV is at its highest at the beginning of the simulation. However, this sensitivity drops dramatically as sensitivity to k_cis, k_{k_transHIV}, and k_syn,gRNA increases, depicting that the concentration of HIV is not predominantly affected by synthesis after cleavage activity begins. k_syn,gRNA has substantial effect on the variance of HIV, even more than parameters associated with cleavage activity. This is because, as can be seen in Figure 7 under Steady State Analysis in Computational Modeling, the production of gRNA plays a direct role in the rate at which knockdown can occur due to enabling the formation of RNP complexes.

REFERENCES

Li, Chun-Qing, and Wei Yang. “2 - Essential Reliability Methods.” ScienceDirect, Woodhead Publishing, 1 Jan. 2023, www.sciencedirect.com/science/article/abs/pii/B9780323858823000064.