Adding outputs to a trial or calibration

Note: this tutorial assumes that you have already created in jinkō a trial simulation (TS) or a calibration with a computational model (CM). Should you not have completed this step, you can refer to one of the following links:

- Creating a trial simulation  

- Calibrating a model
In the context of a trial simulation, the outputs be visible in the trial visualization, or downloadable. In the context of a calibration, some of these outputs will be used to define a fitness function to optimize. Outputs can be defined using 3 different types of project items: a simple output set, an advanced output set and a data table.

The following table summarizes the different types of outputs and their different uses.

The following steps will allow you to create a simple output set (SO), and then an advanced output set (AO). You can skip directly to step 4 to start directly with an AO. The use and definition of data table is not fully described here and is described in the documentation on calibrating a CM (#data-table). 

For each TS or calibration, you can add zero or one SO, zero or one AO set and as many data tables as you want. A TS has to have at least one observable (either from an OS, a CO or a DT), and a calibration must have an objective function, defined by at least one DT or one Objective in a CO. 

Here we will describe the steps as in a trial, but they are absolutely the same in a calibration. Only the usage differs.

Step 1: initiate a simple output set

In a trial, go to the "Outputs incl. fitness function" menu on the left: you will see the summary of the existing observable project items linked to your trial, if any. 

Then go the simple output set tab. Here you can either select an existing SO from the list, or create a new one. Creating a new one will automatically link it to the CM in the trial. 

Step 2: select the model outputs on which you would like the measures to be applied

Using the blue "Select outputs" button, select the model descriptors you want to see as outputs. You can use the filters as in the CM table to find the appropriate descriptors. 

Once selected, one row appears per descriptor. You can add twice the same descriptor by re-selecting it in the output selector, if you want to add more than one different outputs on the same descriptor. 

Step 3: define the computations you would like to carry

Here you have several options in the drop down list of Reduce function: 

• "None" means that the whole time serie will be output for this descriptor. In this case, the whole time serie will be available, and some automated scalars will also be computed (time of max, tend, min, mean, max and auc)
• "Value" will output the value of that descriptor at a given time point
• The others are time reduction functions (the maximum, the minimum, the area under the curve (Auc),... ) over a certain time range

Define the appropriate time or time frame, then select a measure type:

• OnEachArm (default) to compute a quantity of interest per patient and per arm
• DifferenceVsControl (resp. RatioVsControl) to compute, as quantity of interest, for each patient, the difference (resp. the ratio) of that measure on each arm vs. the same measure on its control arm. To use this, you need to have defined control arms in the protocol (see the protocol tutorial).

And finally you can add an optional custom label.

Step 4: Initiate an advanced output set

Similarly to the SO, you can go to the advanced output set tab to select existing AOs or create a new set. As described in the table at the top of this document, there are 3 different types of custom objectives that can be defined: Objectives, Constraints and Scalars.

The interface is similar to that of the computational model: A table contains all the most important information, with one row per component. Clicking on a row opens the details panel underneath the table: the details panel allows to see and edit all of the information about each row. Some of the fields are specific to each types and will be detailed after, and some fields are common:

• Tags (optional) can be used to categorize and find specific measures in the results, similarly to the usage in the CM
• Links (optional) can be used for traceability, to add a reference to where the definition of this component comes from
• ID (mandatory) is the identifier of the component. It should be unique, including across the whole trial, meaning that you cannot have the same ID as one that is in the output set also used in the same trial
• Display name (optional) is the name that will be used for the plots
• Description (optional) is a free text field 

Fields specific to Objectives:

• Formula (mandatory) is the definition of the scalar that will be computed for each patient and compared to the range or target (see next point). 
• Target / Narrow range low bound (mandatory) and Narrow range high bound (optional) is the target, i.e. the expected value for the formula. Here there are 2 cases:
  • if the Narrow range high bound is filled, the target is a range and the objective will evaluate to 1 if the Formula is inside the range. 
  • if the Narrow range high bound is left empty, the target is a unique value and the objective will evaluate to 1 if the Formula is equal to this value. 
• Wide range low bound (mandatory) and Wide range high bound (mandatory) is the acceptable range for the formula: if the formula is within this range, the objective will evaluate between 0 and 1, whereas if the formula is outside of the range, it will evaluate between 0 and -inf. 
• Filter (optional) is evaluated at tmin and evaluates whether the objective should be computed or not for this patient in the simulation. 
• Weight (mandatory, defaults to 1) is used to compute the optimization weighted score, which is the objective function used in the calibration, and can also be seen as a goodness of fit measure in a simulation. Note that the data tables used in the trial or calibration will also count in the optimization weighted score (see the calibration documentation).

Fields specific to Constraints:

• Constraint (mandatory) is the definition of the condition that will be computed for each patient. If this condition is not respected, the optimized weighted score will not be evaluated for this patient, and in the context of a calibration, this patient will be rejected. 
• Filter (optional) is evaluated at tmin and evaluates whether the constraint should be computed or not for this patient in the simulation. 

Fields specific to Scalars:

• Formula (mandatory) is the definition of the scalar that will be computed for each patient.
• Unit (optional) is the unit assumed by the user. This will not lead to any conversion but will raise a warning if the unit is not compliant with what jinkō computes from the formula and the units defined in the CM. See the unit documentation for more detail. 

Using formulas in the custom objectives

While the general syntax is the same as for a computational model, but additionally quite a few scalar functions are useful to define the time reduction (functions that turn a time serie into a single scalar).

 The first example of this is taking a specific time point: <descriptorID>[t]. The syntax for a range is similar: <descriptorID>[a,b].

Other typical functions are: gmax, gmin, avg, int, ddt (time derivative), auc, lastTime, firstTime, lastValue,firstValue,timeOfMax,timeOfMin,minima,maxima,halfLife.

In a formula, you can also refer to a time serie in a sepcific arm, typically to compare a value between several arms, using the following syntax: <descriptorID>@<armID>.

Once your calibration or trial has started running, you can have a preview of these outputs in the "Results" tab for individual patients.  Once your trial is complete you'll be able to visualize those outputs aggregated across the population in the "Time series" and "Scalar Result" tabs of the trial visualization.