3.6. Distortion Module

3.6.1. Distortion Class

class aggregate.spectral.Distortion(name, shape, r0=0.0, df=None, col_x='', col_y='', display_name='')

Creation and management of distortion functions.

0.9.4: renamed roe to ccoc, but kept creator with roe for backwards compatibility. Oct 2022: renamed wtdtvar to bitvar, but kept … Sep 2024: added a proper implementation of wtdtvar and a random_distortion using it

Note, to create a fake Distortion use a type:

g = type(‘Distortion’, (),

{‘g’: your function, ‘g_inv’: your function, ‘g_dual’: lambda x: 1 - g.g(1-x)} )

__init__(name, shape, r0=0.0, df=None, col_x='', col_y='', display_name='')

Create a new distortion.

For the beta distribution:

• A synthesis of risk measures for capital adequacy

• Wirch and Hardy, IME 1999

• 0<a<=1, b>= 1

• b=1 is a PH with rho = 1/a

• a=1 is a dual with rho = b

Tester:

ps = np.linspace(0, 1, 201)
for dn in agg.Distortion.available_distortions(True):
    if dn=='clin':
        # shape param must be > 1
        g_dist = agg.Distortion(**{'name': dn, 'shape': 1.25, 'r0': 0.02, 'df': 5.5})
    else:
        g_dist = agg.Distortion(**{'name': dn, 'shape': 0.5, 'r0': 0.02, 'df': 5.5})
    g_dist.plot()
    g = g_dist.g
    g_inv = g_dist.g_inv

    df = pd.DataFrame({'p': ps, 'gg_inv': g(g_inv(ps)), 'g_invg': g_inv(g(ps)),
    'g': g(ps), 'g_inv': g_inv(ps)})
    print(dn)
    print("errors")
    display(df.query(' abs(gg_inv - g_invg) > 1e-5'))

Parameters:

• name – name of an available distortion, call Distortion.available_distortions() for a list

• shape – float or [float, float] for beta

• shape – shape parameter

• r0 – risk free or rental rate of interest

• df – for convex envelope, dataframe with col_x and col_y used to parameterize or df for t

• col_x

• col_y

• display_name – over-ride name, useful for parameterized convex fix distributions

_complete_init()

classmethod available_distortions(pricing=True, strict=True)

List of the available distortions.

Parameters:

• pricing – only return list suitable for pricing, excludes tvar and convex

• strict – only include those without mass at zero (pricing only)

Returns:

static average_distortion(data, display_name, n=201, el_col='EL', spread_col='Spread')

Create average distortion from (s, g(s)) pairs. Each point defines a wtdTVaR with p=s and p=1 points.

Parameters:

• data

• display_name

• n – number of s values (between 0 and max(EL), 1 is added

• el_col – column containing EL

• spread_col – column containing Spread

Returns:

static bagged_distortion(data, proportion, samples, display_name='')

Make a distortion by bootstrap aggregation (Bagging) resampling, taking the convex envelope, and averaging from data.

Each sample uses proportion of the data.

Data must have two columns: EL and Spread

Parameters:

• data

• proportion – proportion of data for each sample

• samples – number of resamples

• display_name – display_name of created distortion

Returns:

static beta(a, b)

Utility method to create mixture of a list of distortions.

static bitvar(p0, p1, w=0.5)

Utility method to create bitvar with \(p_0 < p_1\). The weight w is the weight on p1. Remember p=0 corresponds to the mean and p=1 to the max.

static ccoc(d)

Utility method to create ccoc with given discount factor. The default constructor inputs the return r instead of the discount factor d. d = r / (1 + r).

static convex_example(source='bond')

Example convex distortion using data from https://www.bis.org/publ/qtrpdf/r_qt0312e.pdf.

Parameters:

source – bond gives a bond yield curve example, cat gives cat bond / cat reinsurance pricing based example

Returns:

static distortions_from_params(params, index, r0=0.025, df=5.5, pricing=True, strict=True)

Make set of dist funs and inverses from params, output of port.calibrate_distortions. params must just have one row for each method and be in the output format of cal_dist.

Called by Portfolio.

Parameters:

• index

• params – dataframe such that params[index, :] has a [lep, param] etc. pricing=True, strict=True: which distortions to allow df for t distribution

• r0 – min rol parameters

• strict

• pricing

Returns:

static dual(shape)

Utility method to create dual.

g_dual(x)

The dual of the distortion function g.

id()

Unique ID as a short string

make_q(ser, a=inf)

Use logic from price_ex to return the vector of risk adjusted probabilities for use in pricing. See price_ex for more.

Always uses backwards calculation, ask pricing and method=’ds’.

The resulting series is used as:

q = a.make_q(x, a)
ask = -((x * q).sum()) + a * gS[-1]

In use, the last adjustment to the xdgS integral is to add a P(X>a) provided a is finite. To simplify the math, just add:

if a == np.inf:
    a = 0

static max()

Utility method to create max = TVaR 1.

static mean()

Utility method to create mean = TVaR 0.

static minimum(distortion_list)

Utility method to create minimum of a list of distortions.

static mixture(distortion_list, weights=None)

Utility method to create mixture of a list of distortions.

static ph(shape)

Utility method to create ph.

plot(xs=None, n=101, both=True, ax=None, plot_points=True, scale='linear', c=None, c_dual=None, size='small', **kwargs)

Quick plot of the distortion

Parameters:

• xs

• n – length of vector is no xs

• both – True: plot g and g_dual and add decorations, if False just g and no trimmings.

• ax

• plot_points

• scale – linear as usual or return plots -log(gs) vs -logs and inverts both scales

• size – ‘small’ or ‘large’ for size of plot, FIG_H or FIG_W. The default is ‘small’.

• kwargs – passed to matplotlib.plot

Returns:

plot_affine(ax=None, n_pts=101, cmap_name='viridis', alpha=1.0, marker='o', marker_size=4)

Render the upper affine envelope of g for wtdtvars.

static power(alpha, x0, x1)

Utility method to create mixture of a list of distortions.

price(ser, a=inf, kind='ask', S_calculation='forwards')

Compute the bid and ask prices for the distribution determined by ser with an asset limit a. Index of ser need not be equally spaced, so it can be applied to \(\kappa\). However, the index values must be distinct. Index is sorted if needed.

To apply to do kappa for unit A in portfolio port:

ser = port.density_df[['exeqa_A', 'p_total']].\
    set_index('exeqa_A').groupby('exeqa_A').\
    sum()['p_total']
dist.price(ser, port.q(0.99), 'both')

Always use S_calculation='forwards method to compute S = 1 - cumsum(probs). Computes the price as the integral of gS.

Returns a tuple of (premium, loss) for the ask or bid price. When ‘both’ requested returns (bid_premium, loss, ask_premium).

Generally prefer to use newer price_ex. Requires ser starts at x=0.

Parameters:

• ser – pd.Series of is probabilities, indexed by outcomes. Outcomes need not be spaced evenly. ser is usually a probability column from density_df.

• kind – is “ask”, “bid”, or “both”, giving the pricing view.

• a – asset level. ser is truncated at a.

price2(ser, a=None, S_calculation='forwards')

Similar to price, and uses the exact same calculation. Returns bid and ask by default in a DataFrame. Returns all cumulative sums if a is None (likely, not what you want).

Generally prefer to use newer price_ex. Requires ser starts at x=0.

price_ex(ser, a=inf, kind='ask', method='dx', S_calculation='backwards', as_frame=False)

Updated version of price and price2 that should be used in the future.

Always uses S_calculation=’backwards’ method to compute S as ser[::-1].cumsum().shift(1, fill_value=0)[::-1].

Sorts ser if needed.

If calculation ‘dx’ computes the price as the integral of gS dx. (This method has fewer diffs). if calculation ‘ds’ computes the prices as the integralk of x d(gS).

Neither method requires the index to be unique.

kind = ask, bid or both.

Asset level a is required to be in the index of ser, there is no interpolation.

There is an ambiguity here about a. If you want the unlimited price of an unbounded variable then you integrate to infinity. In the numerical approximation you integrate over all ser values. In that canse there is no a P(X>a) term added in the dgS method, because Pr(X>inf)=0. However, if X is actually bounded and as a mass at its max value then you do need to add xPr(X>

Returns a namedtuple.

summarize=True automatically summarizes the index if it is not unique.

if as_frame is True, returns a DataFrame with bid, el, and ask columns, otherwise returns a namedtuple with bid, el, and ask fields.

If S_calculation is ‘forwards’, S is computed as 1 - ser.cumsum(). If ‘backwards’, it is computed as p_total[::-1].cumsum()[::-1].shift(-1, fill_value=0). This is computed before p is truncated for a.

Because of potential underflow issues, the backwards method for computing the ask is more reliable. Therefore it has become the default.

When there are thin tails, neither method is reliable for the bid because g_dual(s) = 1 - g(1 - s).

quick_gS(den)

Calls numba version to compute self.g(1-den.sumsum()) for tvar or bitvar distortions.

quick_ra(den, x=None)

Computes the risk adjusted expected value of den and x using the distortion. Calls numba version of ra for tvar or bitvar distortion. If x is None, requires den to be a Series and uses the index. x values must be in ascending order.

static random_distortion(n_knots, mass=0, mean=0, wt_rng=None, name='', random_state=None)

Create a random distortion. if mass (mean) add a mass (mean) term. Knots include possible mean and mass (max) knots. wt_rng to generate (spiky) weights, eg. wt_rng=ss.pareto(1.5).rvs.

static random_distortion_ex(n=1, random_state=None)

Random distortion over types.

Example usage and test::

ans = Distortion.random_distortion_ex(24) f, axs = plt.subplots(6, 4, figsize=(4*1.5, 6*1.5), layout=’constrained’) for (n, g), ax in zip(ans.items(), axs.flat):

g.plot(ax=ax, both=False)

Generalizes random_distortion which only returns a wtdtvar type.

static s_gs_distortion(s, gs, display_name='')

Make a convex envelope distortion from {s, g(s)} points. TODO: allow mass at zero; currently shape=0 passes to no mass at zero even if s,gs implies one. LAZY :param s: iterable (can be converted into numpy.array :param gs: :param display_name: :return:

classmethod test(r0=0.035, df=[0.0, 0.9])

Tester: make some nice plots of available distortions.

Returns:

static tvar(p)

Utility method to create tvar.

tvar_info_df()

Return dataframe of information for wtdtvar, convex type distortions.

p, wts knots = 1 - p (where g has kinks) slopes, intercepts = for each affine part, starting at the knot point

TODO: implement for other types?

static tvar_terms(p_in)

Evaluate tvar function min(s / (1-p), 1), allowing for possibility p=1 and using vector input. s = 1 - p in reverse order. This evaluates the “knot” points needed to create a general weighted TVaR.

static wang(shape)

Utility method to create wang.

3.6.2. Other Spectral functions

aggregate.spectral.approx_ccoc(roe, eps=1e-14, display_name=None)

Create a continuous approximation to the CCoC distortion with return roe. Helpful utility function for creating a distortion.

Parameters:

• roe – return on equity

• eps – small number to avoid mass at zero

aggregate.spectral.bitvar_gS(probs, p0, p1, w)

Compute gS given individual probs. Equivlaent to computing bitvar.g(1 - probs.cumsum()) for a bitvar distortion.

As usual, p0 < p1, w in (0,1) input parameters. See OneNote 2024/November Numba TVaR and BiTVaR.

Parameters:

• probs – numpy array of probabilities.

• p0 – float, distortion parameter.

• p1 – float, distortion parameter.

• w – float, weighting to p1.

aggregate.spectral.bitvar_ra(probs, x, p0, p1, w)

Computes risk adjusted value in one step with no intermediate arrays or copies.

Equivalent to:

dx = np.diff(x)
(g.g(1 - probs.cumsum())[:-1] * dx).sum() + x[0]

or to:

den = pd.Series(probs, index=x)
tvar.price_ex(den, kind='both', calculation='dx', as_frame=True)

Parameters:

• probs – numpy array of probabilities.

• x – numpy array of loss outcomes, must be in ascending order.

• p0 – float, distortion parameter.

• p1 – float, distortion parameter.

• w – float, weighting to p1.

aggregate.spectral.consistent_distortions(p: int)

Create five representative distortions using the TVaR-p parameterization.

aggregate.spectral.p_to_parameters(p)

Return standard parameters equivalent to TVaR p.

aggregate.spectral.tvar_gS(probs, p)

Compute gS given individual probs. Equivlaent to computing tvar.g(1 - probs.cumsum()) for a tvar distortion.

Runs about 4x speed of g.g(1-probs.cumsum()) for TVaR. Cannot parallelize because of shared S variable.

Parameters:

• probs – numpy array of probabilities.

• p – float, distortion parameter.

aggregate.spectral.tvar_ra(probs, x, p)

Computes risk adjusted value in one step with no intermediate arrays or copies.

Equivalent to:

dx = np.diff(x)
(g.g(1 - probs.cumsum())[:-1] * dx).sum() + x[0]

or to:

den = pd.Series(probs, index=x)
tvar.price_ex(den, kind='both', calculation='dx', as_frame=True)

Detects if it is computing the mean (p==0) or the maximum (p==1) and shortcuts accordingly.

Parameters:

• probs – numpy array of probabilities.

• x – numpy array loss outcomes, must be in ascending order.

• p – float, distortion parameter.

aggregate.spectral.tvar_weights(d)

Return tvar weight function for a distortion d. Use np.gradient to differentiate g’ but adjust for certain distortions. The returned function expects a numpy array of p values.

Param:

d distortion