Note
Click here to download the full example code
Active learning
This example shows how to use the ActiveLearning class to iteratively
build a Gaussian process emulator for an unnormalized posterior involving a
simulator. It should be noted that this example is only for illustration
purpose, rather than a real case. For simplicity, required arguments for
ActiveLearning (including simulator, likelihood, data, etc.) are purely
made up. For a realistic case of active learning, one can refer to Zhao and Kowalski [2022].
First, we define the simulator, prior distribution of its variable parameters, observed data, and likelihood function. They basically define the Bayesian inference problem.
import numpy as np
ndim = 2 # dimension of variable parameters of the simulator
bounds = bounds = np.array([[-5,5],[-5,5]]) # bounds of variable parameters
data = np.array([1,0])
def simulator(x1, x2):
"""Simulator y=f(x)."""
y1, y2 = x1, x2
return np.array([y1, y2])
def prior(x):
"""Uniform prior."""
return 1/(10*10)
def likelihood(y, data):
"""Likelihood function L(y,data)."""
return np.exp(-(y[0]-data[0])**2/100 - (y[0]**2-y[1]-data[1])**2)
Imagine that the simulator is a complex solver. It is not computationally feasible to compute the posterior distribution of the variable parameters using grid estimation or Metropolis Hastings estimation. This is because they require evaluating the likelihood many times which essentially leads to many evaluations of the simulator. Therefore, we resort to use active learning to build a Gaussian process emulator for the unnormalized posterior (prior times likelihood) based on a small number of evaluations of the simulator. The the posterior can be estimated using the emulator.
To do so, we need to pass arguments to following parameters of the ActiveLearning class:
run_sim_obj : instance of class
RunSimulator. It carries information on how to run the simulator.lhs_sampler : instance of class
LHS. It is used to draw initial samples to run simulations in order to train an inital Gaussian process emulator.scalar_gasp : instance of class
ScalarGaSP. It sets up the emulator structure.
from psimpy.simulator import RunSimulator
from psimpy.sampler import LHS
from psimpy.emulator import ScalarGaSP
run_simulator = RunSimulator(simulator, var_inp_parameter=['x1','x2'])
lhs_sampler = LHS(ndim=ndim, bounds=bounds, seed=1)
scalar_gasp = ScalarGaSP(ndim=ndim)
Next, we create an object of the ActiveLearning class by
from psimpy.inference import ActiveLearning
active_learner = ActiveLearning(ndim, bounds, data, run_simulator, prior, likelihood,
lhs_sampler, scalar_gasp)
Then we can call the ActiveLearning.initial_simulation() method to run initial
simulations and call the ActiveLearning.iterative_emulation() method to
iteratively run new simulation and build emulator. Here we allocate 40 simulations for
initial emulator training and 60 simulations for adaptive training.
n0 = 40
niter = 60
init_var_samples, init_sim_outputs = active_learner.initial_simulation(
n0, mode='parallel', max_workers=4)
var_samples, _, _ = active_learner.iterative_emulation(
n0, init_var_samples, init_sim_outputs, niter=niter)
The upper bounds of the range parameters are 420.7131 416.6473
The initial values of range parameters are 8.414262 8.332945
Start of the optimization 1 :
The number of iterations is 23
The value of the marginal posterior function is -146.5038
Optimized range parameters are 30.59659 130.5917
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.9828751 0.9733764
Start of the optimization 2 :
The number of iterations is 16
The value of the marginal posterior function is -146.5038
Optimized range parameters are 30.5982 130.6064
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 418.0103 415.1878
The initial values of range parameters are 8.360206 8.303755
Start of the optimization 1 :
The number of iterations is 27
The value of the marginal posterior function is -148.6792
Optimized range parameters are 30.19535 133.3286
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.9589025 0.9524277
Start of the optimization 2 :
The number of iterations is 13
The value of the marginal posterior function is -148.6792
Optimized range parameters are 30.19529 133.3285
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 372.0419 369.5298
The initial values of range parameters are 7.440838 7.390595
Start of the optimization 1 :
The number of iterations is 34
The value of the marginal posterior function is -149.9595
Optimized range parameters are 30.64772 135.0584
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.9360715 0.9297509
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -149.9595
Optimized range parameters are 30.64717 135.0548
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 412.0792 409.2967
The initial values of range parameters are 8.241583 8.185933
Start of the optimization 1 :
The number of iterations is 28
The value of the marginal posterior function is -151.304
Optimized range parameters are 31.21546 136.8634
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.9143024 0.9081287
Start of the optimization 2 :
The number of iterations is 33
The value of the marginal posterior function is -151.304
Optimized range parameters are 31.21602 136.8597
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 373.5991 371.0765
The initial values of range parameters are 7.471983 7.421529
Start of the optimization 1 :
The number of iterations is 26
The value of the marginal posterior function is -152.6009
Optimized range parameters are 32.2851 141.3769
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.8935228 0.8874894
Start of the optimization 2 :
The number of iterations is 33
The value of the marginal posterior function is -152.6009
Optimized range parameters are 32.28086 141.3732
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 363.7651 361.3088
The initial values of range parameters are 7.275302 7.226176
Start of the optimization 1 :
The number of iterations is 16
The value of the marginal posterior function is -153.5419
Optimized range parameters are 32.43467 141.787
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.8736668 0.8677675
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -153.5419
Optimized range parameters are 32.43465 141.787
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 388.6202 385.9961
The initial values of range parameters are 7.772404 7.719922
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -154.9929
Optimized range parameters are 33.6944 151.2097
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.854674 0.848903
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -154.9929
Optimized range parameters are 33.6947 151.2113
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 368.5149 366.0266
The initial values of range parameters are 7.370298 7.320531
Start of the optimization 1 :
The number of iterations is 22
The value of the marginal posterior function is -155.4764
Optimized range parameters are 33.87171 151.8922
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.8364895 0.8308412
Start of the optimization 2 :
The number of iterations is 16
The value of the marginal posterior function is -155.4765
Optimized range parameters are 33.90549 152.0137
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 303.005 300.959
The initial values of range parameters are 6.060099 6.019179
Start of the optimization 1 :
The number of iterations is 14
The value of the marginal posterior function is -155.8802
Optimized range parameters are 34.04396 152.6136
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.8190626 0.813532
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -155.8802
Optimized range parameters are 34.04591 152.6189
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 271.7327 269.8979
The initial values of range parameters are 5.434655 5.397958
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -156.1383
Optimized range parameters are 34.47655 151.3798
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.802347 0.7969293
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -156.1383
Optimized range parameters are 34.46977 151.4259
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 229.754 228.2026
The initial values of range parameters are 4.595079 4.564052
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -156.184
Optimized range parameters are 34.50828 152.8102
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.7863001 0.7809907
Start of the optimization 2 :
The number of iterations is 33
The value of the marginal posterior function is -156.184
Optimized range parameters are 34.50295 152.8055
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 206.5918 205.1968
The initial values of range parameters are 4.131835 4.103936
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -156.2348
Optimized range parameters are 34.79817 155.165
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.7708824 0.7656772
Start of the optimization 2 :
The number of iterations is 16
The value of the marginal posterior function is -156.2348
Optimized range parameters are 34.82609 155.1902
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 258.3488 256.6043
The initial values of range parameters are 5.166975 5.132086
Start of the optimization 1 :
The number of iterations is 33
The value of the marginal posterior function is -156.4062
Optimized range parameters are 34.76576 157.0386
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.7560578 0.7509526
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -156.4062
Optimized range parameters are 34.73883 156.9472
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 237.2817 235.6795
The initial values of range parameters are 4.745633 4.713589
Start of the optimization 1 :
The number of iterations is 22
The value of the marginal posterior function is -156.5552
Optimized range parameters are 35.36218 157.5205
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.7417925 0.7367837
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -156.5552
Optimized range parameters are 35.37291 157.5609
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 250.7264 249.0334
The initial values of range parameters are 5.014527 4.980667
Start of the optimization 1 :
The number of iterations is 21
The value of the marginal posterior function is -156.8768
Optimized range parameters are 35.90648 161.054
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.7280556 0.7231396
Start of the optimization 2 :
The number of iterations is 25
The value of the marginal posterior function is -156.8767
Optimized range parameters are 35.92228 161.1123
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 238.9065 237.2933
The initial values of range parameters are 4.778129 4.745866
Start of the optimization 1 :
The number of iterations is 28
The value of the marginal posterior function is -156.6342
Optimized range parameters are 36.30829 162.0199
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.7148183 0.7099916
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -156.6342
Optimized range parameters are 36.30018 161.8524
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 245.248 243.592
The initial values of range parameters are 4.904959 4.871839
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -156.3684
Optimized range parameters are 36.27097 162.5809
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.7020536 0.6973131
Start of the optimization 2 :
The number of iterations is 35
The value of the marginal posterior function is -156.3684
Optimized range parameters are 36.28256 162.6136
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 245.5015 243.8438
The initial values of range parameters are 4.91003 4.876875
Start of the optimization 1 :
The number of iterations is 29
The value of the marginal posterior function is -156.014
Optimized range parameters are 36.15503 162.4561
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.6897369 0.6850796
Start of the optimization 2 :
The number of iterations is 16
The value of the marginal posterior function is -156.0141
Optimized range parameters are 36.19571 162.4856
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 237.5607 235.9566
The initial values of range parameters are 4.751214 4.719132
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -155.8456
Optimized range parameters are 36.99431 166.8476
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.6778449 0.6732679
Start of the optimization 2 :
The number of iterations is 34
The value of the marginal posterior function is -155.8455
Optimized range parameters are 36.9843 166.8112
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 218.5445 217.0688
The initial values of range parameters are 4.37089 4.341377
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -155.2612
Optimized range parameters are 37.52584 170.9858
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.666356 0.6618565
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -155.261
Optimized range parameters are 37.5272 170.9907
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 228.1098 226.5696
The initial values of range parameters are 4.562197 4.531391
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -154.6105
Optimized range parameters are 37.33828 173.6169
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.6552501 0.6508256
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -154.6106
Optimized range parameters are 37.20479 173.2666
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 211.684 210.2546
The initial values of range parameters are 4.23368 4.205093
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -153.8922
Optimized range parameters are 37.4968 173.0747
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.6445083 0.6401563
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -153.8923
Optimized range parameters are 37.52424 173.017
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 199.9789 198.6286
The initial values of range parameters are 3.999579 3.972572
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -153.0734
Optimized range parameters are 37.55227 173.0759
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.634113 0.6298312
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -153.0733
Optimized range parameters are 37.52729 172.8307
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 202.9237 201.5534
The initial values of range parameters are 4.058473 4.031069
Start of the optimization 1 :
The number of iterations is 23
The value of the marginal posterior function is -152.171
Optimized range parameters are 37.57767 172.5286
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.6240477 0.6198339
Start of the optimization 2 :
The number of iterations is 37
The value of the marginal posterior function is -152.1709
Optimized range parameters are 37.52827 172.3748
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 190.9983 189.7086
The initial values of range parameters are 3.819965 3.794172
Start of the optimization 1 :
The number of iterations is 22
The value of the marginal posterior function is -151.4187
Optimized range parameters are 37.85886 177.5027
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.6142969 0.610149
Start of the optimization 2 :
The number of iterations is 37
The value of the marginal posterior function is -151.4192
Optimized range parameters are 37.86442 177.4215
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 184.5705 183.3242
The initial values of range parameters are 3.69141 3.666484
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -150.3527
Optimized range parameters are 37.92872 177.0128
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.6048462 0.6007621
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -150.352
Optimized range parameters are 37.92941 176.9876
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 206.689 205.2934
The initial values of range parameters are 4.13378 4.105867
Start of the optimization 1 :
The number of iterations is 28
The value of the marginal posterior function is -149.2293
Optimized range parameters are 37.73939 179.5156
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.5956819 0.5916596
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -149.2285
Optimized range parameters are 37.6686 179.2687
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 204.4555 203.0749
The initial values of range parameters are 4.08911 4.061499
Start of the optimization 1 :
The number of iterations is 34
The value of the marginal posterior function is -148.0359
Optimized range parameters are 37.07659 181.8027
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5867911 0.5828289
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -148.0361
Optimized range parameters are 37.09612 181.862
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 207.5545 206.153
The initial values of range parameters are 4.15109 4.123061
Start of the optimization 1 :
The number of iterations is 31
The value of the marginal posterior function is -147.2549
Optimized range parameters are 36.89036 184.8898
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.5781618 0.5742579
Start of the optimization 2 :
The number of iterations is 18
The value of the marginal posterior function is -147.2555
Optimized range parameters are 36.87001 184.82
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 166.9279 165.8008
The initial values of range parameters are 3.338559 3.316015
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -146.0908
Optimized range parameters are 34.10173 165.8008
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5697827 0.5659353
Start of the optimization 2 :
The number of iterations is 12
The value of the marginal posterior function is -146.091
Optimized range parameters are 34.11888 165.8008
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 207.5545 206.153
The initial values of range parameters are 4.15109 4.123061
Start of the optimization 1 :
The number of iterations is 39
The value of the marginal posterior function is -144.6147
Optimized range parameters are 36.54517 188.5363
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5616429 0.5578505
Start of the optimization 2 :
The number of iterations is 19
The value of the marginal posterior function is -144.6151
Optimized range parameters are 36.27548 187.7642
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 162.9825 161.882
The initial values of range parameters are 3.259651 3.23764
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -143.1453
Optimized range parameters are 32.74403 161.882
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5537325 0.5499935
Start of the optimization 2 :
The number of iterations is 22
The value of the marginal posterior function is -143.1451
Optimized range parameters are 32.74424 161.882
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 158.5408 157.4703
The initial values of range parameters are 3.170816 3.149405
Start of the optimization 1 :
The number of iterations is 26
The value of the marginal posterior function is -141.6959
Optimized range parameters are 31.65865 157.4703
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5460417 0.5423547
Start of the optimization 2 :
The number of iterations is 13
The value of the marginal posterior function is -141.6958
Optimized range parameters are 31.65944 157.4703
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 154.683 153.6385
The initial values of range parameters are 3.09366 3.072771
Start of the optimization 1 :
The number of iterations is 21
The value of the marginal posterior function is -140.1644
Optimized range parameters are 31.1494 153.6385
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5385617 0.5349251
Start of the optimization 2 :
The number of iterations is 21
The value of the marginal posterior function is -140.1642
Optimized range parameters are 31.1205 153.6385
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 156.7268 155.6685
The initial values of range parameters are 3.134536 3.113371
Start of the optimization 1 :
The number of iterations is 37
The value of the marginal posterior function is -138.6449
Optimized range parameters are 31.25719 155.6685
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5312838 0.5276964
Start of the optimization 2 :
The number of iterations is 10
The value of the marginal posterior function is -138.6457
Optimized range parameters are 31.20511 155.6685
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 159.9013 158.8216
The initial values of range parameters are 3.198026 3.176432
Start of the optimization 1 :
The number of iterations is 12
The value of the marginal posterior function is -138.799
Optimized range parameters are 32.12962 158.8216
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.5242001 0.5206605
Start of the optimization 2 :
The number of iterations is 30
The value of the marginal posterior function is -138.7995
Optimized range parameters are 32.23426 158.8216
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 157.0852 156.0245
The initial values of range parameters are 3.141704 3.12049
Start of the optimization 1 :
The number of iterations is 32
The value of the marginal posterior function is -138.9791
Optimized range parameters are 32.28276 156.0245
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.5173027 0.5138097
Start of the optimization 2 :
The number of iterations is 11
The value of the marginal posterior function is -138.9787
Optimized range parameters are 32.25762 156.0245
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 153.7189 152.681
The initial values of range parameters are 3.074379 3.05362
Start of the optimization 1 :
The number of iterations is 9
The value of the marginal posterior function is -137.3328
Optimized range parameters are 30.89731 152.681
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.5105845 0.5071368
Start of the optimization 2 :
The number of iterations is 11
The value of the marginal posterior function is -137.3327
Optimized range parameters are 30.89795 152.681
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 159.3891 158.3129
The initial values of range parameters are 3.187782 3.166257
Start of the optimization 1 :
The number of iterations is 13
The value of the marginal posterior function is -134.9406
Optimized range parameters are 30.6762 158.3129
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.5040385 0.5006351
Start of the optimization 2 :
The number of iterations is 36
The value of the marginal posterior function is -134.9398
Optimized range parameters are 30.83421 158.3129
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 159.832 158.7527
The initial values of range parameters are 3.196639 3.175055
Start of the optimization 1 :
The number of iterations is 23
The value of the marginal posterior function is -132.5023
Optimized range parameters are 29.64711 158.7527
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4976583 0.4942979
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -132.5029
Optimized range parameters are 29.69952 158.7527
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 151.0987 150.0785
The initial values of range parameters are 3.021975 3.00157
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -130.9007
Optimized range parameters are 28.04164 150.0785
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4914376 0.4881192
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -130.9016
Optimized range parameters are 27.90702 150.0785
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 156.5391 155.4821
The initial values of range parameters are 3.130783 3.109643
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -127.8529
Optimized range parameters are 27.65861 155.4821
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4853704 0.482093
Start of the optimization 2 :
The number of iterations is 15
The value of the marginal posterior function is -127.8537
Optimized range parameters are 27.65893 155.4821
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 144.6837 143.7068
The initial values of range parameters are 2.893674 2.874135
Start of the optimization 1 :
The number of iterations is 10
The value of the marginal posterior function is -126.3069
Optimized range parameters are 26.14805 143.7068
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.4794513 0.4762139
Start of the optimization 2 :
The number of iterations is 32
The value of the marginal posterior function is -126.3061
Optimized range parameters are 26.14945 143.7068
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 147.4263 146.4308
The initial values of range parameters are 2.948526 2.928616
Start of the optimization 1 :
The number of iterations is 18
The value of the marginal posterior function is -123.3139
Optimized range parameters are 26.00776 146.4308
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.4736748 0.4704763
Start of the optimization 2 :
The number of iterations is 29
The value of the marginal posterior function is -123.3124
Optimized range parameters are 26.00792 146.4308
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 141.8343 140.8766
The initial values of range parameters are 2.836686 2.817532
Start of the optimization 1 :
The number of iterations is 21
The value of the marginal posterior function is -121.3189
Optimized range parameters are 25.40012 140.8766
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.4680358 0.4648754
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -121.3199
Optimized range parameters are 25.3783 140.8766
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 138.8069 137.8696
The initial values of range parameters are 2.776137 2.757392
Start of the optimization 1 :
The number of iterations is 27
The value of the marginal posterior function is -119.9576
Optimized range parameters are 24.60408 137.8696
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4625295 0.4594063
Start of the optimization 2 :
The number of iterations is 33
The value of the marginal posterior function is -119.9574
Optimized range parameters are 24.6049 137.8696
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 136.7754 135.8519
The initial values of range parameters are 2.735509 2.717038
Start of the optimization 1 :
The number of iterations is 31
The value of the marginal posterior function is -117.3956
Optimized range parameters are 23.9288 135.8519
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4571512 0.4540644
Start of the optimization 2 :
The number of iterations is 19
The value of the marginal posterior function is -117.3963
Optimized range parameters are 23.93923 135.8519
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 122.4858 121.6587
The initial values of range parameters are 2.449715 2.433174
Start of the optimization 1 :
The number of iterations is 9
The value of the marginal posterior function is -117.1863
Optimized range parameters are 22.45945 121.6587
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.4518966 0.4488452
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -117.1857
Optimized range parameters are 22.45861 121.6587
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 124.7933 123.9507
The initial values of range parameters are 2.495867 2.479014
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -114.3601
Optimized range parameters are 22.5128 123.9507
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.4467614 0.4437447
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -114.36
Optimized range parameters are 22.51309 123.9507
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 121.8507 121.0279
The initial values of range parameters are 2.437013 2.420558
Start of the optimization 1 :
The number of iterations is 32
The value of the marginal posterior function is -112.4649
Optimized range parameters are 22.12543 121.0279
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4417416 0.4387588
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -112.4657
Optimized range parameters are 22.13681 121.0279
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 114.2532 113.4817
The initial values of range parameters are 2.285064 2.269635
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -110.9429
Optimized range parameters are 20.87695 113.4817
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4368334 0.4338837
Start of the optimization 2 :
The number of iterations is 30
The value of the marginal posterior function is -110.9428
Optimized range parameters are 20.87687 113.4817
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 107.5519 106.8257
The initial values of range parameters are 2.151038 2.136513
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -109.6259
Optimized range parameters are 19.96839 106.8257
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.432033 0.4291158
Start of the optimization 2 :
The number of iterations is 25
The value of the marginal posterior function is -109.6258
Optimized range parameters are 19.96622 106.8257
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 105.9321 105.2168
The initial values of range parameters are 2.118642 2.104337
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -107.2887
Optimized range parameters are 19.81197 105.2168
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.427337 0.4244515
Start of the optimization 2 :
The number of iterations is 35
The value of the marginal posterior function is -107.2885
Optimized range parameters are 19.80782 105.2168
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 103.7434 103.0429
The initial values of range parameters are 2.074868 2.060858
Start of the optimization 1 :
The number of iterations is 23
The value of the marginal posterior function is -104.942
Optimized range parameters are 19.35122 103.0429
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.422742 0.4198875
Start of the optimization 2 :
The number of iterations is 36
The value of the marginal posterior function is -104.942
Optimized range parameters are 19.3513 103.0429
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 102.2359 101.5455
The initial values of range parameters are 2.044717 2.030911
Start of the optimization 1 :
The number of iterations is 26
The value of the marginal posterior function is -102.382
Optimized range parameters are 18.92818 101.5455
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4182447 0.4154206
Start of the optimization 2 :
The number of iterations is 38
The value of the marginal posterior function is -102.3824
Optimized range parameters are 18.93008 101.5455
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 104.947 104.2383
The initial values of range parameters are 2.098939 2.084767
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -98.23267
Optimized range parameters are 18.94201 104.2383
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4138422 0.4110477
Start of the optimization 2 :
The number of iterations is 40
The value of the marginal posterior function is -98.23364
Optimized range parameters are 18.94106 104.2383
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 105.99 105.2743
The initial values of range parameters are 2.1198 2.105486
Start of the optimization 1 :
The number of iterations is 26
The value of the marginal posterior function is -94.36122
Optimized range parameters are 18.8138 105.2743
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0.4095313 0.406766
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -94.36013
Optimized range parameters are 18.81444 105.2743
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 95.08534 94.44329
The initial values of range parameters are 1.901707 1.888866
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -95.14024
Optimized range parameters are 17.52605 94.44329
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4053093 0.4025725
Start of the optimization 2 :
The number of iterations is 33
The value of the marginal posterior function is -95.13983
Optimized range parameters are 17.52612 94.44329
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 92.02781 91.4064
The initial values of range parameters are 1.840556 1.828128
Start of the optimization 1 :
The number of iterations is 28
The value of the marginal posterior function is -92.90551
Optimized range parameters are 16.93414 91.4064
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.4011735 0.3984647
Start of the optimization 2 :
The number of iterations is 39
The value of the marginal posterior function is -92.90572
Optimized range parameters are 16.93369 91.4064
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 103.2185 102.5215
The initial values of range parameters are 2.06437 2.05043
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -85.41126
Optimized range parameters are 18.01591 102.5215
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.3971213 0.3944398
Start of the optimization 2 :
The number of iterations is 11
The value of the marginal posterior function is -85.40862
Optimized range parameters are 18.01616 102.5215
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 84.12819 83.56012
The initial values of range parameters are 1.682564 1.671202
Start of the optimization 1 :
The number of iterations is 22
The value of the marginal posterior function is -90.63195
Optimized range parameters are 16.03987 83.56012
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0.39315 0.3904954
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -90.63178
Optimized range parameters are 16.02786 83.56012
Optimized nugget parameter is 0
Convergence: FALSE
Once the active learning process is finished, we obtain the final emulator for the
logarithm of the unnormalized posterior, which is given by the
ActiveLearning.approx_ln_pxl() method.
We can then estimate the posterior using grid estimation or Metropolis Hastings estimation based on the emulator. An example is as follows. The contour plot shows the estimated posterior.
from psimpy.inference import GridEstimation
import matplotlib.pyplot as plt
grid_estimator = GridEstimation(ndim, bounds, ln_pxl=active_learner.approx_ln_pxl)
posterior, x_ndim = grid_estimator.run(nbins=50)
fig, ax = plt.subplots(1,1,figsize=(5,4))
# initial training points
ax.scatter(init_var_samples[:,0], init_var_samples[:,1], s=10, c='r', marker='o',
zorder=1, alpha=0.8, label='initial training points')
# actively picked training points
ax.scatter(var_samples[n0:,0], var_samples[n0:,1], s=15, c='k', marker='+',
zorder=2, alpha=0.8, label='iterative training points')
# estimated posterior based on the final emulator
posterior = np.where(posterior < 1e-10, np.nan, posterior)
contour = ax.contour(x_ndim[0], x_ndim[1], np.transpose(posterior), levels=10, zorder=0)
plt.colorbar(contour, ax=ax)
ax.legend()
ax.set_title('Active learning')
ax.set_xlabel('x1')
ax.set_ylabel('x2')
ax.set_xlim([-5,5])
ax.set_ylim([-5,5])
plt.tight_layout()
Total running time of the script: ( 0 minutes 54.860 seconds)