helper_calibration.py

output_target_to_eir(df, exp, model)

Interpolates and adjusts the output target to the expected EIR (Entomological Inoculation Rate) values based on input data for different models and experimental conditions.

Parameters:
  • df (DataFrame) –

    DataFrame containing simulation job data, where each row represents a specific condition (e.g., target_output_values, cm_clinical, seasonality).

  • exp (Experiment) –

    Experiment object that contains attributes such as interp_path, target_output_name, pop_size, and others required for interpolation.

  • model (str) –

    The model name used in the interpolation (e.g., ‘malariasimulation’, ‘OpenMalaria’).
Returns:
  • list –

    A list of interpolated EIR values corresponding to each row of the input DataFrame.
Raises:
  • ValueError –

    If the interpolation file is not found at the specified path.

  • ValueError –

    If the target_output_name specified in the experiment object is not valid.
Notes
  • The function performs interpolation based on the specified output type (eir, prevalence_2to10, or clinical_incidence_U5).
  • It applies scaling adjustments for population sizes, ensuring that the values are consistent with the pop_size defined in the experiment.
  • Warnings are issued if the interpolated values are outside the valid range (<= 0 or >= 1000), with adjustments made to these values accordingly.
Source code in utility\helper_calibration.py 
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
def output_target_to_eir(df, exp, model):
    """
    Interpolates and adjusts the output target to the expected EIR (Entomological Inoculation Rate) values
    based on input data for different models and experimental conditions.

    Args:
        df (pd.DataFrame): DataFrame containing simulation job data, where each row represents a specific condition
            (e.g., `target_output_values`, `cm_clinical`, `seasonality`).
        exp (Experiment): Experiment object that contains attributes such as `interp_path`, `target_output_name`, `pop_size`,
            and others required for interpolation.
        model (str): The model name used in the interpolation (e.g., 'malariasimulation', 'OpenMalaria').

    Returns:
        list: A list of interpolated EIR values corresponding to each row of the input DataFrame.

    Raises:
        ValueError: If the interpolation file is not found at the specified path.
        ValueError: If the `target_output_name` specified in the experiment object is not valid.

    Notes:
        - The function performs interpolation based on the specified output type (`eir`, `prevalence_2to10`, or
          `clinical_incidence_U5`).
        - It applies scaling adjustments for population sizes, ensuring that the values are consistent with the
          `pop_size` defined in the experiment.
        - Warnings are issued if the interpolated values are outside the valid range (`<= 0` or `>= 1000`), with
          adjustments made to these values accordingly.
    """
    interp_path = os.path.join(exp.interp_path, model, exp.interp_csv)
    if not os.path.exists(interp_path):
        raise ValueError(f'interp_file not found: {interp_path}')

    data = pd.read_csv(interp_path)
    output = []
    if exp.target_output_name == 'eir':
        output_type_col = 'simulatedEIR'
    elif exp.target_output_name == 'prevalence_2to10':
        output_type_col = 'prevalence_2to10'
    elif exp.target_output_name == 'clinical_incidence_U5':
        output_type_col = 'clinical_incidence_U5'
    else:
        raise ValueError(f'Specified input_type is not valid {exp.target_output_name}')
    if model == 'EMOD':
        popsize = exp.emod_pop_size
    elif model == 'malariasimulation':
        popsize = exp.malariasimulation_pop_size
    elif model == 'OpenMalaria':
        popsize = exp.openmalaria_pop_size
    else:
        raise ValueError(f'Specified model is not valid {model}')

    # FIXME allow this hack, or is there a better one? For testrun, or introduce 'debug run' to allow running lower  populations
    if exp.test and popsize not in data.loc[data['modelname'] == model, 'pop_size'].unique():
        popsize = data.loc[data['modelname'] == model, 'pop_size'].min()

    for i, row in df.iterrows():
        idata = data[((data['cm_clinical'] == row['cm_clinical'])
                      & (data['cm_severe'] == row['cm_severe'])
                      & (data['seasonality'] == row['seasonality'])
                      & (data['modelname'] == model)
                      & (data['pop_size'] == popsize))]
        xt = idata[output_type_col]
        yp = idata['input_target']
        xnew = row['target_output_values']
        interpolated_value = np.interp(xnew, xt, yp)
        if interpolated_value <= 0:
            interpolated_value = 0.01
            print("Warning: eir < 0, value set to 0.01 (this will likely cause elimination in the simulations)")
        if interpolated_value >= 1000:
            interpolated_value = 1000
            print("Warning:  eir > 1000, value set to 1000 (higher values cannot be reasonably simulated)")
        output.append(interpolated_value)
    return output

output_target_to_xTemp(df, exp, model)

Interpolates and adjusts the output target to the x_Temporary_Larval_Habitat parameter based on input data for different models and experimental conditions.

Parameters:
  • df (DataFrame) –

    DataFrame containing simulation job data, with each row representing a specific condition (e.g., target_output_values, cm_clinical, seasonality).

  • exp (Experiment) –

    Experiment object that contains attributes such as interp_path, target_output_name, pop_size, and others required for interpolation.

  • model (str) –

    The model name used in the interpolation (e.g., ‘malariasimulation’, ‘OpenMalaria’).
Returns:
  • list –

    A list of interpolated x_Temporary_Larval_Habitat values corresponding to each row of the input DataFrame.
Raises:
  • ValueError –

    If the interpolation file is not found at the specified path.

  • ValueError –

    If the target_output_name specified in the experiment object is not valid.
Notes
  • The function performs interpolation based on the specified output type (eir, prevalence_2to10, or clinical_incidence_U5).
  • It applies scaling adjustments for population sizes, ensuring that the values are consistent with the pop_size defined in the experiment.
  • Warnings are issued if the interpolated values are outside the valid range (<= 0 or >= 10000), with adjustments made to these values accordingly.
Source code in utility\helper_calibration.py 
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
def output_target_to_xTemp(df, exp, model):
    """
    Interpolates and adjusts the output target to the `x_Temporary_Larval_Habitat` parameter based on input data
    for different models and experimental conditions.

    Args:
        df (pd.DataFrame): DataFrame containing simulation job data, with each row representing a specific condition
            (e.g., `target_output_values`, `cm_clinical`, `seasonality`).
        exp (Experiment): Experiment object that contains attributes such as `interp_path`, `target_output_name`, `pop_size`,
            and others required for interpolation.
        model (str): The model name used in the interpolation (e.g., 'malariasimulation', 'OpenMalaria').

    Returns:
        list: A list of interpolated `x_Temporary_Larval_Habitat` values corresponding to each row of the input DataFrame.

    Raises:
        ValueError: If the interpolation file is not found at the specified path.
        ValueError: If the `target_output_name` specified in the experiment object is not valid.

    Notes:
        - The function performs interpolation based on the specified output type (`eir`, `prevalence_2to10`, or
          `clinical_incidence_U5`).
        - It applies scaling adjustments for population sizes, ensuring that the values are consistent with the
          `pop_size` defined in the experiment.
        - Warnings are issued if the interpolated values are outside the valid range (`<= 0` or `>= 10000`), with
          adjustments made to these values accordingly.
    """
    interp_path = os.path.join(exp.interp_path, model, exp.interp_csv)
    if not os.path.exists(interp_path):
        raise ValueError(f'interp_file not found: {interp_path}')

    data = pd.read_csv(interp_path)
    output = []

    if exp.target_output_name == 'eir':
        output_type_col = 'simulatedEIR'
    elif exp.target_output_name == 'prevalence_2to10':
        output_type_col = 'prevalence_2to10'
    elif exp.target_output_name == 'clinical_incidence_U5':
        output_type_col = 'clinical_incidence_U5'
    else:
        raise ValueError(f'Specified input_type is not valid {exp.target_output_name}')

    # FIXME allow this hack, or is there a better one? For testrun, or introduce 'debug run' to allow running lower  populations
    popsize = exp.emod_pop_size
    if exp.test and popsize not in data.loc[data['modelname'] == model, 'pop_size'].unique():
        popsize = int(data.loc[data['modelname'] == model, 'pop_size'].min())

    for i, row in df.iterrows():
        idata = data[((data['cm_clinical'] == row['cm_clinical'])
                      & (data['cm_severe'] == row['cm_severe'])
                      & (data['seasonality'] == row['seasonality'])
                      & (data['modelname'] == model)
                      & (data['pop_size'] == popsize))]
        xt = idata[output_type_col]
        yp = idata['input_target']
        xnew = row['target_output_values']
        interpolated_value = np.interp(xnew, xt, yp)

        if interpolated_value <= 0:
            interpolated_value = 0.01
            print("Warning: EMOD's x_Temporary_Larval_Habitat < 0, value set to 0.01 "
                  "(this will likely cause elimination in the simulations)")
        if interpolated_value >= 10000:
            interpolated_value = 10000
            print("Warning: EMOD's x_Temporary_Larval_Habitat > 10000, value set to 10000 "
                  "(higher values cannot be reasonably simulated)")
        output.append(interpolated_value)
    return output