3.8. Extensions
Extensions contains optional, nice to have code that extends basic functionality. For example, all CaseStudy material is included here. It is not required for core functionality. It is not included in the default import. It is included in the default build.
3.8.1. Basic
- aggregate.extensions.basic.fc(x)
- aggregate.extensions.basic.fcm(x)
- aggregate.extensions.basic.fg(x)
- aggregate.extensions.basic.fp(x)
- aggregate.extensions.basic.fp3(x)
3.8.2. Fourier Transform Support
- class aggregate.extensions.ft.FourierTools(chf, fz, scale_mode=True)[source]
Manual inversion of a ch. f. using FFTs.
- compute_exact(calc='survival', max_points=257)[source]
Compute exact density using frozen scipy.stats object.
- Parameters:
calc – ‘density’ re-scales pdf, ‘survival’ uses backward differences of sf.
max_points – maximum number of points to use; if more just interpolate this many points.
decimate – decimate (take
::decimate) input xs to reduce number of calls to fz (which may be slow)min_points – opposite of decimate, ensure exact computed with min_points. Useful for example when log2 is “too small” to ensure exact distribution is rendered correctly.
- property df
Return current state dataframe output.
- property df_exact
Return current state dataframe output.
- fourier_transform(t)[source]
Create ft function by converting ch f to Fourier transform and including scale and loc.
Recall: ft(t)= int f(x)exp(-2πi t x)dx chf(t) = int f(x)exp(i t x)dx.
- invert(log2, x_min=0, bs=0, x_max=None, s=1e-17)[source]
Invert a characteristic function using irfft.
Call with just log2 for positive support, to determine x_max and bs from quantiles. Call with bs fixed to a reasonable value to deduce x_max = x_min + n bs. Call with x_max to fix the range and deduce bs.
- Parameters:
x_min – minimum value of range.
bs – bucket size to use, determines x_max
x_max – minimum value of range.
s – survival probability to determine tails
- invert_simpson(*, log2=0, bs=0, x_min=None)[source]
Add Simpson’s method approximation.
Adds
simpsoncolumn toself.df. Uses method of Wang and Zhang, “Simpson’s rule based FFT method to compute densities of stable distribution” (2008). Can no longer use the real fft and ifft methods because the input vector is not conjugate symmetric about its midpoint.Run after self.invert to set parameters or input directly. Defaults are None because 0 is a legitimate value for x_min. Convenient to state x_min and bs directly. Best to input all parameters if adjusting to avoid mismatches. Note x_max gets trumped by n, bs, and x_min.
- plot(suptitle='', xlim=None, verbose=True)[source]
Compare density, log density, and plot amplitude and argument of Fourier transform.
- Parameters:
suptitle – super title for the plot.
- plot_fourier3d(scale=True)[source]
Three dimensional line plot of the Fourier transform using mayavi.
- plot_wraps(wraps=None, calc='survival', add_tail=False)[source]
Illustrate wrapping. Only run when fz.pdf is easy to calc, otherwise too slow.
- Parameters:
wraps – optional list of wrap values. Eg [-1,1] plots one greater and one less than [0, P)
calc – how to estimate the density outside the base range, same
compute_exact.add_tail – plot the shifted exact densities in plots 2 and 4.
- aggregate.extensions.ft.fft_wrapping_illustration(ez=10, en=20, sev_clause='', small2=0, cmap='plasma')[source]
Illustrate wrapping by convolving a uniform distribution with mean ez en times (if
ez>0orsev_clause!='') or using the inputsev_clause.sev_clauseshould be adsevtailored tobs==1or just a Poisson if ez==1.Show in a space just big enough for the severity first and then big enough for the full aggregate. Center and right hand plot illustrate how the full components are sliced up and combined to the wrapped total.
If small2 is zero it is taken to be the smallest value to “fit” the severity.
In Poisson ez==1 mode, small2 equals the size of the small window to use for convolution. big2 is estimated to fit the whole distribution.
(moved from figures.py)
- aggregate.extensions.ft.ft_invert(log2, chf, frz_generator, params, loc=0, scale=1, xmax=0, xshift=0, suptitle='', wraps=None, disc_calc='density')[source]
Illustrate “manual” inversion of a characteristic function using irfft, including optional scaling and location shift.
- Parameters:
params – a list of shape parameters. loc and scale are handled separately.
chf – the characteristic function of the distribution, takes args params, loc, scale, t; routine handles conversion to Fourier Transform
frz_generator – the scipy.stats function to create the underlying distribution. Used to compute the exact answer. If there is not analytic formula, you can pass in a numpy array with the values of the distribution at the points in xs computed by some other means (e.g., for the Tweedie distribution).
loc – location paramteter
scale – scale parameter
xmax – if not zero, used to fix xmax. Otherwise, selected as frz.isf(1e-17) to capture the full range of theunderlying distribution. For thick tailed distributions, you usually input xmax manually. Note bs = xmax / n. If None set equal to n = 1 << log2, for use in discrete distributions (forced bucket size of 1).
xshift – if not zero, used to shift the x-axis. The minimum x value equals xshift. To center the x-axis, set xshift = -xmax/2. Should be a multiple of bs = xmax / n.
suptitle – optional suptitle for the figure
wraps – optional list of wrap values.
disc_calc – ‘density’ rescales pdf, ‘surival’ uses backward differences of sf (to match Aggregate class calculation)
- aggregate.extensions.ft.make_levy_chf(alpha, beta)[source]
Make the ch of stable(alpha, beta) per Nolan book page 5 Def 1.3.
- aggregate.extensions.ft.poisson_example(en, small2)[source]
Example to show how to compute Po(en) using 1 << small2 buckets. For AAS paper. Sample call:
poisson_example(10**8, 17)
- Parameters:
en – mean of Poisson, e.g., 10**8
small2 – log2 number of buckets to use in FFT routine, 2**small2 should be about 10 * en ** 0.5 to get +/-5 standard deviations around the mean
- aggregate.extensions.ft.recentering_convolution(sev_clause, freq_clause, en, log2, bs, remove_fuzz)[source]
Compute hifreq convol for sev_clause, a DecL severity statement. Must have only one component for now. Illustrates how to find the “correct” part of an aggregate and recenter it appropriately. Aggregate is:
agg RecenteringExample en claims dsev xs ps poisson
Method
Compute aggregate with enough space for the supported part of the
distribution (say, where density > 1e-15 or so)
Subtract the mean by rolling left (negative shift) by mean / bs buckets (mod n)
fft shift = roll (in either direction) by n / 2 buckets, because
the distribution is centered at zero and has positive and negative parts.
Set appropriate x values to align with the density. Density is from
mean - n/2 to mean + n/2 - 1 (times bs).
- aggregate.extensions.ft.recentering_convolution_example(sev_clause, en, log2, agg_log2=0, bs=1, freq_clause='poisson', remove_fuzz=False)[source]
Illustrate how to find the “correct” part of an aggregate and recenter it appropriately. Aggregate is:
agg RecenteringExample en claims dsev xs ps poisson
Method
Compute aggregate with enough space for the supported part of the
distribution (say, where density > 1e-15 or so)
Subtract the mean by rolling left (negative shift) by mean / bs buckets (mod n)
fft shift = roll (in either direction) by n / 2 buckets, because
the distribution is centered at zero and has positive and negative parts.
Set appropriate x values to align with the density. Density is from
mean - n/2 to mean + n/2 - 1 (times bs).
Reasonable defaults:
en = 5000 log2 = 15 xs = [3, 4, 7, 34] ps = [1/8, 1/8, 1/8, 5/8]
(from hifreq.py)
- Parameters:
xs – array of x values for dsev
ps – array of x values for dsev
3.8.3. Tweedie Support
- class aggregate.extensions.tweedie.Mode(value)[source]
Mode of the Tweedie class.
- ADDITIVE = 2
- REPRODUCTIVE = 1
- class aggregate.extensions.tweedie.Tweedie(p: float, *, mean: float | None = None, dispersion: float | None = 1, theta: float | None = None, index: float | None = 1)[source]
Tweedieis a class for working with the Tweedie class of exponential dispersion models, those with variance functionV(μ) = dispersion x μ^p,
where p is the power parameter and μ the mean. The Tweedie class of distributions includes:
1 < p < 2: Tweedie distributions, a Poisson-gamma compound distributions
p = 1: Poisson distributions
p = 0: normal distributions
p < 0: spectrally positive stable distributions with index 1 < alpha < 2
p = infinity: spectrally positive Cauchy distribution with index alpha = 1
2 < p < infinity: positive spectrally positive stable distributions with index 0 < alpha < 1
p = 3 (special case): inverse Gaussian and Levy(1/2) distributions
p = 2: gamma distributions
The range 0 < p < 1 is impossible and does not correspond to an exponential dispersion model.
The
Tweedieclass providesTranslation between the reproductive and additive parameterizations. The former is used to model averages like pure premiums that combined in weighted sums and the latter models totals (sums) that are simply additive.
Create frozen
scipy.stats-like distributions, either explicitly as ascipy.statsobject from the normal, levy_stable, levy, inverse Gaussian, or gamma distribution, or as anAggregateobject otherwise.For Tweedie distributions it translates parameters into Poisson frequency and gamma severity, the latter as mean and CV or shape and rate.
Create a
aggregate.extensions.ft.FourierToolsobject for Fourier analysis that provides an approximation to the distribution for those without a closed form expression (extreme and positive extreme stable, titled Cauchy, Tweedie).The generating distribution
c.The cumulant function,
kappa.The tilted cumulant function,
KThe characteristic function,
chf.The Levy measure density,
levy_density, see [3]
3.8. Methodology
The class is initialized with either the reproductive parameters (p, mean, dispersion) or the additive parameters (p, theta, index). The class then computes the missing parameters. The class creates a frozen distribution object on demand for the additive or reproductive distribution per those provided at initialization. Methods are available to create the dual. Methods are also available to translate between additive and reproductive parameter for the same distribution (not the dual).
3.8. Notation
The canonical parameter is always called theta, p is always p. The other parameters are named (mean, dispersion, index) or (frequency, shape, rate) or (sev_m, sev_cv). The dispersion is usually sigma^2. The index is lambda or phi (but the frequency is sometimes lambda).
Notation follows Jorgensen [1] and [2]. My blog post on Tweedie uses -alpha for the shape.
3.8. References
- [1] Jørgensen, Bent. “Exponential dispersion models.” Journal of the Royal Statistical Society
Series B: Statistical Methodology 49.2 (1987): 127-145.
[2] Jorgensen, B. (1997). The theory of dispersion models. Chapman and Hall/CRC. [3] Jørgensen, Bent, and José Raúl Martínez. “THE LÉVY—KHINCHINE REPRESENTATION OF THE TWEEDIE
CLASS.” Brazilian Journal of Probability and Statistics (1996): 225-233.
- K_additive(s)[source]
Cumulant function, reflecting theta and in additive form.
See [2] eq. 3.4 and eq 4.15.
- K_reproductive(s)[source]
Cumulant function, reflecting theta and in additive form.
See, [2] eq. 3.5 and eq 4.15.
- property additive: Tuple[float, float, float]
Return the additive parameters: frequency (Poisson mean), gamma shape and rate.
- static additive_to_reproductive(p: float, theta: float, index: float) Tuple[float, float, float][source]
Convert additive parameters to reproductive parameters for the same distribution.
Warning: this is not the dual distribution!
Uses [2] eq 4.8 p. 130.
- audit()[source]
Audit dataframe comparing definition, Fourier, and frozen stats.
Moment code from aggregate xsden_to_meancvskew.
- static from_p_mean_cv(p: float, mean: float, cv: float) Tweedie[source]
Create a Tweedie object from the parameters p, mean, and the output Tweedie cv. The dispersion is computed by equating two expressions for the variance:
(cv * mean) ** 2 = dispersion * mean^p.
==> dispersion = cv **2 * mean ** (2 - p)
The object is created using p, mean, and dispersion in reproductive mode.
- static from_po_gamma(frequency: float, *, sev_m: float | None = None, sev_cv: float | None = None, sev_shape: float | None = None, sev_rate: float | None = None, tw_cv: float | None = None) Tweedie[source]
Create a Tweedie object for the compound Poisson-gamma case, 1 < p < 2. Inputs
frequency: mean number of claims, required
One of
sev_m: mean severity and sev_cv or
sev_shape and sev_rate or
sev_m and tw_cv (overall distribution cv)
Gamma distribution with density f(x) = x^(shape-1) exp(-x rate) / Gamma(shape) for x > 0.
Higher rate (Poisson rate of emission of particles per unit time) results in a lower mean waiting time for n to appear. Gamma(shape, rate) has mean shape / rate and variance shape / rate^2. Hence, cv^2 = 1 / shape.
The object is created using p, mean, and dispersion in reproductive mode, same as
from_p_mean_cv.
- property ft
Create and return FourierTools object.
- property fz
Return the frozen distribution object, reproductive or additive depending on how the object was initialized.
3.8. Poisson
Rather than a scipy.stats object, which requires index=1, use an Aggregate object. This allows scaling. Note, for Poisson is only reproductive in general. It is
Y ~ Po(index * exp(theta)) / index
The Aggregate is also not fussy about pdf vs pmf.
3.8. Gamma
Per [2] p. 89 Ga*(theta, lambda) ~ Gamma with shape lambda and scale -theta.
- kappa(theta: float)[source]
Cumulant function of the carrier distribution.
Jorgensen [2] equation 4.13, p. 131.
- property po_gamma_m_cv: Tuple[float, float, float]
Return Poisson-gamma parameters, specifying gamma using mean and cv.
- property po_gamma_shape_rate: Tuple[float, float, float]
Return Poisson-gamma parameters, specifying gamma using shape and rate.
- property reproductive: Tuple[float, float, float]
Return the reproductive parameters: p, mean, dispersion.
- static reproductive_to_additive(p: float, mean: float, dispersion: float) Tuple[float, float, float][source]
Convert reproductive parameters to additive parameters for the same distribution.
Warning: this is not the dual distribution!
Uses [2] Exercise 4.5 and equation below 4.8 on p. 130 (that has a typo).
- static reproductive_to_tweedie_compound(p, mean, dispersion)[source]
Convert Tweedie reproductive parameters to additive parameters for the compound Poisson-gamma distribution.
This is the inverse of the Poisson-gamma conversion.
- static tau(p: float, theta: float) float[source]
Convert theta (canonical parameter) to mean, using tau = kappa’(theta).
Evaluate the tau function, [2] below eq 4.12 p 131. Recall (1-alpha)(1-p) = -1. Assumes dispersion and index = 1, see additive_to_reproductive for more general function.
- static tau_inverse(p: float, mean: float) float[source]
Convert mean to theta (canonical parameter) using tau_inverse.
Evaluate the inverse of the tau function, [2] below eq 4.12 p 131. Assumes dispersion and index = 1, see reproductive_to_additive for more general function. In GLMs tau_inverse is called the canonical link function, mapping the mean to the canonical parameter.
- aggregate.extensions.tweedie.make_test_suite(mode=Mode.REPRODUCTIVE)[source]
Create a test suite for the Tweedie class.
3.8.4. Case Study Support
- class aggregate.extensions.case_studies.ClassicalPremium(ports, calibration_premium)[source]
manage classical premium examples
Net, no loading Expected value, constant loading Maximum loss –> no VaR (as proxy for maximum loss) Variance Std Dev Semi-variance (Artzner p. 210) Exponential (zero utility, convex!) Esscher
Originally in hack.py
- calibrate(port_name, line_name, calibration_premium, df=None, ob=None, stats=None, mn=None, var=None, sd=None)[source]
calibrate all methods…
- distribution(port_name, line_name)[source]
classical methods all depend on the distribution…so pull it out pull the object that will provide q etc. pull the audit stats
- illustrate(port_name, line_name, ax, margin, *, p=0, K=0, n_big=10000, n_sample=25, show_bounds=True, padding=2)[source]
illustrate simulations at p level probability probability level determines capital or capital K input margin: premium = (1 + margin) * EL, margin = rho n_big = number of policies - max of horizontal axis n_sample = number of iterations to plot
Theoretic bounds use the actual moments, simulated use those from the process being estimated
From common_scripts Pentagon took out re-computation…
- price(param, port_name, line_name, method, df=None, ob=None, stats=None, mn=None, var=None, sd=None)[source]
apply method to port_name, line_name with parameter(s) (all one param) these are all classical methods
method_dict = {method name : param } param = float | [float param, p ge 1 value] latter fro Fischer method
- aggregate.extensions.case_studies.add_defaults(dict_in, kind='agg')[source]
add default values to dict_inin. Leave existing values unchanged Used to output to a data frame, where you want all columns completed
- Parameters:
dict_in
kind
- Returns:
- aggregate.extensions.case_studies.bivariate_density_plots(axi, ports, xmax, contour_scale, biv_log=True, cmap='viridis', levels=30, color_bar=False)[source]
bivarate plots of each line against the others for each portfolio in case arguments as for twelve_plot / bivden plot
axi = iterator with enough exes ports = iterable of ports (list, dict.values(), etc.)
from common_scripts.py
- aggregate.extensions.case_studies.extract_sort_order(summaries, _varlist_, classical=False)[source]
Pull out exhibits. Note difference: classical uses net_classical, calibrated to roe non-classical uses pricing calibrated to
- aggregate.extensions.case_studies.g_ins_stats(axi, dn0, dist, ls='-')[source]
Six part plot with EL, premium, loss ratio, profit, layer ROE and P:S for g axi = axis iterator
- aggregate.extensions.case_studies.macro_market_graphs(axi, port, dn, rp)[source]
Create more succinct 4x4 graph to illustrate macro market structure and result, LR, P:S and ROE, etc.
Use a port object with calibrated distortion
see ch04_macro_market_stats_original
from: common_scripts.py ch04_macro_market_stats updated line colors
June 2022: removed all color=’k’
- Parameters:
dist
rp – return period
sigma
- Returns:
3.8.5. Pentagon
- aggregate.extensions.pentagon.make_possible_pentagons()[source]
enumerate possible and impossible pentagon configurations
- class aggregate.extensions.pentagon.pent_ans(L, P, M, a, Q, LR, PQ, COC)
- COC
Alias for field number 7
- L
Alias for field number 0
- LR
Alias for field number 5
- M
Alias for field number 2
- P
Alias for field number 1
- PQ
Alias for field number 6
- Q
Alias for field number 4
- a
Alias for field number 3
3.8.6. Samples
Warning
All functionality in extensions.samples has been moved into the base package.
3.8.7. Figures
- aggregate.extensions.figures.adjusting_layer_losses()[source]
Figure to illustrate the process of adjusting layer losses. TODO: Add reference
- aggregate.extensions.figures.discretization_agg_example(outcomes)[source]
For AAS paper. Convergence of sevs with smaller bucket size.
- aggregate.extensions.figures.discretization_sev_example(outcomes)[source]
For AAS paper. Convergence of sevs with smaller bucket size.
- aggregate.extensions.figures.dual_distortion(dist=None, s=0.3)[source]
Illustrate how the dual distortion relates to the distortion.
- aggregate.extensions.figures.g_insurance_statistics(axi, dist, c='C0', ls='-', lw=1, diag=False, grid=False)[source]
Six part plot with EL, premium, loss ratio, profit, layer ROE and P:S for g axi = axis iterator with 6 axes dist = distortion Used to create PIR Figs 11.6 and 11.7
- aggregate.extensions.figures.g_risk_appetite(axi, dist, c='C0', ls='-', N=1000, lw=1, grid=False, xlabel=True, title=True, add_tvar=False)[source]
Plot to illustrate the risk appetite associated with a distortion g. Derived from
g_insurance_statistics. Plots premium, loss ratio, margin, return (easier to understand than discount), VaR wts and optionally TVaR wts axi = axis iterator with 6 axes dist = distortion Used to create PIR Figs 11.6 and 11.7
- aggregate.extensions.figures.gh_example(en)[source]
Code to reproduce GHGrübel and Hermesmeier 1999, Table 1. The function
exact_cdfcalculates the compound probability that \(x-1/2 < X \le x+1/2\). For AAS paper with en=20.
- aggregate.extensions.figures.mixing_convergence(freq_cv, sev_cv, bs=0.015625)[source]
Illustrate convergence of mixed distributions to the mixing distribution.
3.8.8. PIR Figures
- aggregate.extensions.pir_figures.curlyBrace(ax, p1, p2, k_r=0.1, bool_auto=True, str_text='', int_line_num=2, fontdict={}, **kwargs)[source]
Plot an optionally annotated curly bracket on the given axes of the given figure.
Note that the brackets are anti-clockwise by default. To reverse the text position, swap “p1” and “p2”.
Note that, when the axes aspect is not set to “equal”, the axes coordinates need to be transformed to screen coordinates, otherwise the arcs may not be seeable.
Parameters
- figmatplotlib figure object
The of the target axes.
- axmatplotlib axes object
The target axes.
- p1two element numeric list
The coordinates of the starting point.
- p2two element numeric list
The coordinates of the end point.
- k_rfloat
This is the gain controlling how “curvy” and “pointy” (height) the bracket is.
Note that, if this gain is too big, the bracket would be very strange.
- bool_autoboolean
This is a switch controlling wether to use the auto calculation of axes scales.
When the two axes do not have the same aspects, i.e., not “equal” scales, this should be turned on, i.e., True.
When “equal” aspect is used, this should be turned off, i.e., False.
If you do not set this to False when setting the axes aspect to “equal”, the bracket will be in funny shape.
Default = True
- str_textstring
The annotation text of the bracket. It would displayed at the mid point of bracket with the same rotation as the bracket.
By default, it follows the anti-clockwise convention. To flip it, swap the end point and the starting point.
The appearance of this string can be set by using “fontdict”, which follows the same syntax as the normal matplotlib syntax for font dictionary.
Default = empty string (no annotation)
- int_line_numint
This argument determines how many lines the string annotation is from the summit of the bracket.
The distance would be affected by the font size, since it basically just a number of lines appended to the given string.
Default = 2
- fontdictdictionary
This is font dictionary setting the string annotation. It is the same as normal matplotlib font dictionary.
Default = empty dict
- **kwargsmatplotlib line setting arguments
This allows the user to set the line arguments using named arguments that are the same as in matplotlib.
Returns
- thetafloat
The bracket angle in radians.
- summitlist
The positions of the bracket summit.
- arc1list of lists
arc1 positions.
- arc2list of lists
arc2 positions.
- arc3list of lists
arc3 positions.
- arc4list of lists
arc4 positions.
Reference
https://uk.mathworks.com/matlabcentral/fileexchange/38716-curly-brace-annotation
- aggregate.extensions.pir_figures.fig_10_3(dist=None, s=0.3)[source]
Figure 10.3 Illustrating distortion functions (s, g(s)) with vertical line at s and split loss, premium, margin, and capital labelled
- aggregate.extensions.pir_figures.fig_10_5(port=None, dist=None, s=0.3, x=None, return_period_max=100)[source]
three plot version of previous with more explanation of first picture
return_period_max = defines extend of yaxis return_period_x = capital level to illustrate
map from s space into loss space extended version of ch04_s_gs_loss_premium_capital which includes the horizontal bar [ loss ][m][ equity ] plotted on the provided second axis
Suggested figure set up for extended:
f = plt.Figure(figsize=(4,3), tight_layout=True) a = f.add_axes([0, 100/3+1/27, 1, 2/3], label=’a’) b = f.add_axes([0, 0, 1, 1/3], label=’b’)
- aggregate.extensions.pir_figures.fig_10_6(port=None, dist=None)[source]
Same distortion and portfolio as 10_5 Slight clarification of the diagram vs. book version.
- aggregate.extensions.pir_figures.getAxSize(fig, ax)[source]
Get the axes size in pixels.
Reference: https://uk.mathworks.com/matlabcentral/fileexchange/38716-curly-brace-annotation
- Parameters:
fig – matplotlib figure object The of the target axes.
ax – matplotlib axes object The target axes.
- Returns:
ax_width : float, the axes width in pixels; ax_height : float, the axes height in pixels.