40
Quantiles, Expectiles and Splines Andrew Harvey University of Cambridge December 2007 Harvey (University of Cambridge) QES December 2007 1 / 40

Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Quantiles, Expectiles and Splines

Andrew Harvey

University of Cambridge

December 2007

Harvey (University of Cambridge) QES December 2007 1 / 40

Page 2: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Introduction

The movements in a time series may be described by time-varyingquantiles. These may be estimated non-parametrically by �tting a simplemoving average or a more elaborate kernel. An alternative approach is toformulate a partial model, the role of which is to focus attention on someparticular feature - here a quantile - so as to provide a (usually nonlinear)weighting of the observations that will extract that feature by takingaccount of the dynamic properties of the series.Time-varying quantiles can be �tted to a sequence of observations bysetting up a state space model and iteratively applying a suitably modi�edsignal extraction algorithm.Satisfy the de�ning property of �xed quantiles in having the appropriatenumber of observations above and below.

Harvey (University of Cambridge) QES December 2007 2 / 40

Page 3: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

0 150 300 450 600 750 900 1050 1200 1350 1500 1650 1800 1950

­20

­15

­10

­5

0

5

10

GMQ(05)Q(95)

Q(25)Q(75)

Figure: Quantiles �tted to GM returns

Harvey (University of Cambridge) QES December 2007 3 / 40

Page 4: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Expectiles are similar to quantiles except that they are de�ned by tailexpectations; see Newey and Powell (1987). Time-varying expectiles canbe estimated by a state space algorithm. This is similar to the algorithmused for quantiles, but estimation is more straightforward and muchquicker.

Harvey (University of Cambridge) QES December 2007 4 / 40

Page 5: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Quantiles and expectiles

Let ξ(τ) - or, when there is no risk of confusion, ξ - denote the τ � thquantile. The probability that an observation is less than ξ(τ) is τ, where0 < τ < 1. Given a set of T observations, yt , t = 1, ..,T , the samplequantile, eξ(τ), can be obtained as the solution to minimizing

Sτ =T

∑t=1

ρτ(yt � ξ) = ∑yt<ξ

(τ � 1)(yt � ξ) + ∑yt�ξ

τ(yt � ξ) (1)

with respect to ξ, where ρτ(.) is the check function, de�ned for quantilesas

ρτ(yt � ξ) = (τ � I (yt � ξ < 0)) (yt � ξ) (2)

and I (.) is the indicator function.

Harvey (University of Cambridge) QES December 2007 5 / 40

Page 6: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Expectiles, denoted µ(ω), 0 < ω < 1, are similar to quantiles but they aredetermined by tail expectations rather than tail probabilities. For a givenvalue of ω, the sample expectile, eµ(ω), is obtained by minimizing theasymmetric least squares function,

Sω = ∑ ρω(yt � µ(ω)) = ∑ jω� I (yt � µ(ω) < 0)j (yt � µ(ω))2, (3)

with respect to µ(ω). Di¤erentiating Sω and dividing by minus two gives

T

∑t=1jω� I (yt � µ(ω) < 0)j (yt � µ (ω)) (4)

The sample expectile, eµ(ω), is the value of µ(ω) that makes (4) equal tozero. Setting ω = 0.5 gives the mean, that is eµ(0.5) = y . For other ω itis necessary to iterate.

Harvey (University of Cambridge) QES December 2007 6 / 40

Page 7: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Signal extraction

A framework for estimating time-varying quantiles, ξt (τ), can be set up byassuming that they are generated by stochastic processes and areconnected to the observations through a measurement equation

yt = ξt (τ) + εt (τ), t = 1, ...,T , (5)

where Pr(εt (τ) < 0) = τ with 0 < τ < 1. The disturbances, εt (τ), areassumed to be serially independent and independent of ξt (τ). Theproblem is then one of signal extraction. The assumption that the quantileor expectile follows a stochastic process can be regarded as a device forinducing local weighting of the observations. One possibility is a randomwalk,

ξt (τ) = ξt�1(τ) + ηt (τ), ηt (τ) v IID(0, σ2η(τ)), (6)

Harvey (University of Cambridge) QES December 2007 7 / 40

Page 8: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

A smoother quantile can be extracted by a local linear trend

ξt = ξt�1 + βt�1 + ηt (7)

βt = βt�1 + ζt

where βt is the slope and ζt is IID(0, σ2ζ). It is well known that in aGaussian model setting Var(ηt ) = σ2ζ /3 and Cov(ηt , ζt ) = σ2ζ /2 resultsin the smoothed estimates being a cubic spline.The model for expectiles is set up in a similar way with (5) replaced byyt = µt (ω) + εt (ω).

Harvey (University of Cambridge) QES December 2007 8 / 40

Page 9: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Derivation and properties

The proposed solution minimizes ∑t ρτ(yt � ξt ) subject to a set ofconstraints imposed by the time series model for the quantile. Supposethis model is a �rst-order autoregression,. The criterion function is then

J = �T

∑t=1

ρτ(yt � ξt )

ω� 12(1� φ2)(ξ1 � ξ†)2

σ2η� 12

T

∑t=2

η2tσ2η. (8)

where ω is a scaling constant. Given the observations, the estimatedtime-varying quantiles, eξ1, ..,eξT , are the values that maximise J.

Harvey (University of Cambridge) QES December 2007 9 / 40

Page 10: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

In a classical signal extraction framework,

yt = µt + εt , t = 1, ...,T (9)

where µt is a Gaussian stochastic process and εt is NID(0, σ2). If J isrede�ned with µt in place of ξt and ρτ(yt � ξt )/ω replaced by(yt � µt )

2/2σ2, it can be interpreted as the logarithm of the joint densityof the observations and µ0ts. Di¤erentiating with respect toµt , t = 1, ...,T , setting to zero and solving gives the modes,eµt , t = 1, ...,T , of the conditional distributions of the µ0ts. For amultivariate Gaussian distribution these are the conditional expectations,which by de�nition are the smoothed (minimum mean square error)estimators.

Harvey (University of Cambridge) QES December 2007 10 / 40

Page 11: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Returning to the quantiles and di¤erentiating J with respect to ξt gives

∂J∂ξt

=1ωIQ (yt � ξt ) +

φξt�1 � (1+ φ2)ξt + φξt+1 + (1� φ)2ξ†

σ2η,

(10)for t = 2, . . . ,T � 1, (modi�ed at the endpoints), where IQ (yt � ξt ) isde�ned as in (24). For t = 2, ..,T � 1, setting ∂J/∂ξt to zero gives anequation that is satis�ed by the estimated quantiles, eξt ,eξt�1 and eξt+1,and similarly for t = 1 and T . If a solution is located on an observation,that is eξt = yt , then IQ (yt � ξt ) is not de�ned as the check function isnot di¤erentiable at zero.

Harvey (University of Cambridge) QES December 2007 11 / 40

Page 12: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

In a Gaussian model, (9), a little algebra leads to the classicWiener-Kolmogorov (WK) formula for a doubly in�nite sample. For theAR(1) model eµt = µ+

ggy(yt � µ)

where µ = E (µt ), g = σ2η/((1� φL)(1� φL�1)) is the autocovariancegenerating function (ACGF) of µt , L is the lag operator, and gy = g + σ2ε .The WK formula has the attraction that, for simple models, gy can bewritten in terms of the reduced form and g/gy can be expanded to give anexplicit expression for the weights. Here the reduced form of yt is anARMA(1, 1) model and

eµt = µ+qµθ

φ(1� θ2)

∑j=�∞

θjj j(yt+j � µ) (11)

where qµ = σ2η/σ2ε and

θ = (qµ + 1+ φ2)/2φ�h�qµ + 1+ φ2

�2 � 4φ2i1/2

/2φ. This expressionis still valid for the random walk except that µ disappears because theweights sum to one.Harvey (University of Cambridge) QES December 2007 12 / 40

Page 13: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

In order to proceed in a similar way with quantiles, we need to takeaccount of corner solutions by de�ning the corresponding IQ 0ts as thevalues that give equality of the associated derivative of J. Then we can set∂J/∂ξt in (10) equal to zero to giveeξt � ξ†

g=1ωIQ(yt � eξt ) (12)

for a doubly in�nite sample with ξ† known. Using the lag operator yields

eξt = ξ† +σ2η

ω

∑j=�∞

φjj j

1� φ2IQ(yt+j � eξt+j ) (13)

Note that if the observations are multiplied by a constant, then thequantile is multiplied by the same constant, as are ω and σ2η. By de�ningthe quasi �signal-noise�ratio as q = σ2η/ω2, it remains scale invariant.

Harvey (University of Cambridge) QES December 2007 13 / 40

Page 14: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

An expression for extracting quantiles that has a similar form to (11) canbe obtained by adding (eξt � ξ†)/ω2 to both sides of (12) andre-arranging to give

eξt = ξ† +ggωy

heξt � ξ† +ωIQ(yt � eξt )i (14)

where gωy = g +ω2.This is not an ACGF but it can be treated as though

it were. Thus we obtain

eξt = ξ† +qτθ

φ(1� θ2)

∑j=�∞

θjj j[eξt+j � ξ† +ωIQ(yt+j � eξt+j )] (15)

where θ is as de�ned for (11) but with qµ replaced by q = σ2η/ω2. For the

random walk, the weights sum to one, so ξ† drops out giving

eξt = 1� θ

1+ θ

∑j=�∞

θjj j[eξt+j +ωIQ(yt+j � eξt+j )] (16)

Harvey (University of Cambridge) QES December 2007 14 / 40

Page 15: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Non-parametric smoothing

Changing quantiles may be estimated non-parametrically, as in Yu andJones (1998), by minimising a local check function to give an estimator,bξt , that satis�es

h

∑j=�h

K (j/h)IQ(yt+j � bξt ) = 0 (17)

where K (.) is a weighting kernel, h is a bandwidth.with IQ(yt+j � bξt )de�ned appropriately if yt+j = bξt .PropositionIf the same kernel and bandwidth are used for di¤erent quantiles, theycannot cross (though they may touch).

Harvey (University of Cambridge) QES December 2007 15 / 40

Page 16: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

It is interesting to compare the above weighting scheme with the oneimplied by the random walk model. If eξt+j in (16) is constant, it satis�es(17) with K (j/h) replaced by θjj j so giving an (in�nite) exponential decay.The time series model determines the shape of the kernel while the qparameter plays the same role as the bandwidth.An important advantage of the more model-based approach for forecastingis that it automatically determines a weighting pattern at the end of thesample that is consistent with the one in the middle.

Harvey (University of Cambridge) QES December 2007 16 / 40

Page 17: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

State Space Form

The state space form (SSF) for a univariate time series is:

yt = z0tαt + εt , Var(εt ) = σ2t , t = 1, ...,T (18)

αt = Ttαt�1+ηt , Var(ηt ) = Qt

where αt is an m� 1 state vector, zt is a non-stochastic m� 1 vector, σ2tis a non-negative scalar, Tt is an m�m non-stochastic transition matrixand Qt is an m�m covariance matrix. The speci�cation is completed byassuming that α1 has mean a1j0 and covariance matrix P1j0 and that theserially independent disturbances εt and ηt are independent of each otherand of the initial state.

Harvey (University of Cambridge) QES December 2007 17 / 40

Page 18: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Consider the criterion function

J = �T

∑t=1h�1t ρ(yt � z0tαt )�

12

T

∑t=2

η0tQ�1t ηt �

12(α1�a1j0)0P�11j0(α1�a1j0),

(19)where ρ(yt � z0tαt ) is as in (2) or (3), with z0tαt equal to ξt (τ) or µt (ω),Qt and P1j0 are are assumed positive de�nite matrices as in (18) and ht isa non-stochastic sequence of postive scalars. Suppose that the initial stateand the η0ts are normally distributed. For a Gaussian model of the form(18), logarithm of the joint density of the observations and the states is,ignoring irrelevant terms, given by J with ρ(yt � z0tαt ) = (yt � µt (0.5))

2

and ht = 2σ2t . Di¤erentiating J with respect to to each element of αtgives a set of equations, which, when set to zero and solved gives theminimum mean square error estimates of αt . These they may be computede¢ ciently by the Kalman �lter and associated smoother (KFS) asdescribed in Durbin and Koopman (2001, pp. 70-73). If all the elements inthe state are nonstationary and given a di¤use prior, the last term in Jdisappears. An algorithm is available as a subroutine in the SsfPack set ofprograms within Ox; see Koopman et al (1999).Harvey (University of Cambridge) QES December 2007 18 / 40

Page 19: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

We can think of (19) as a criterion function that provides the basis forcomputing a quantile or expectile subject to a set of constraints imposedby the time series model for the quantile or expectile. For expectilesdi¤erentiating J gives

∂J∂α1

= z1(2/h1)IE (y1 � z01α1)�P�11j0(α1�a1j0) +T02Q

�12 (α2�T2α1)

∂J∂αt

= zt (2/ht )IE (yt � z0tαt )�Q�1t (αt�Ttαt�1) +T0t+1Q�1t+1 (αt+1�Tt+1αt ) ,

t=2, . . . ,T � 1,∂J

∂αT= zT (2/hT )IE (yT � z0T αT )�Q�1T (αT�TT αT�1) . (20)

where

IE (yt �µt (ω)) = jω� I (yt � µt (ω) < 0)j (yt �µt (ω)), t = 1, ...,T .(21)

The smoothed estimates, eαt , satisfy the equations obtained by settingthese derivatives equal to zero.

Harvey (University of Cambridge) QES December 2007 19 / 40

Page 20: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Let ht = gt/κ, where κ is a constant, the interpretation of which willbecome apparent. For any expectile, adding and subtracting ztg�1t z0tαt tothe equations in (20) allows the �rst term to be written as

ztg�1t [z0tαt + 2κIE�yt � z0tαt

�]� ztg�1t z0tαt , t = 1, . . . ,T . (22)

This suggests that we set up an iterative procedure in which the estimateof the state at the i-th iteration, eα(i )t , is computed from the KFS appliedto a set of synthetic �observations�constructed as

by (i�1)t = z0tbα(i�1)t + 2κIE�yt � z0tbα(i�1)t

�. (23)

The iterations are carried out until the bα(i )0t s converge whereuponeµt (ω) = z0teαt .Harvey (University of Cambridge) QES December 2007 20 / 40

Page 21: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

For quantiles, the �rst term in each of the three equations of (20) is givenby zth�1t IQ(yt � z0tαt ), where

IQ(yt � ξt (τ)) =

�τ � 1, if yt < ξt (τ)

τ, if yt > ξt (τ)t = 1, ...,T , (24)

and the synthetic observations in the KFS are

by (j�1)t = z0tbα(j�1)t + κIQ�yt � z0tbα(j�1)t

�, t = 1, ...,T (25)

However, the possibility of a solution where the estimated quantile passesthrough an observation means that the algorithm has to be modi�edsomewhat; see De Rossi and Harvey (2006).

Harvey (University of Cambridge) QES December 2007 21 / 40

Page 22: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Estimates of time-varying quantiles and expectiles obtained from thesmoothing equations of the previous sub-section can be shown to satisfyproperties that generalize the de�ning characteristics of �xed quantiles andexpectiles.It is assumed that the state has been arranged so that the �rst elementrepresents the level and that (without loss of generality) the �rst elementin zt has been set to unity. Let the �rst derivative, with respect to αt , ofthe second term of J be written j02 = ∑T

t=1 Atαt , where the A0ts arem�m matrices.Lemma For a model in SSF with a di¤use prior on the initial state, asu¢ cient condition for the �rst element in the vector j02 to be zero is thatthe �rst column of Tt � I consists of zeroes for all t = 2, ...,T.

Harvey (University of Cambridge) QES December 2007 22 / 40

Page 23: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

If ht is time-invariant and the conditions of the Lemma hold, theestimated quantiles satisfy the fundamental property of sampletime-varying quantiles, namely that the number of observations that areless than the corresponding quantile, that is yt < eξt (τ), is no more than[Tτ] while the number greater is no more than [T (1� τ)].

Harvey (University of Cambridge) QES December 2007 23 / 40

Page 24: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

PropositionIf the distribution of y is time invariant when adjusted for changes inlocation and scale, and is continuous with �nite mean, the populationτ�quantiles and ω�expectiles coincide for ω satisfying

ω =

R ξ(τ)�∞ (y � ξ(τ))dF (y)R ξ(τ)

�∞ (y � ξ(τ))dF (y)�R ∞

ξ(τ)(y � ξ(τ))dF (y)

where F (y) is the cdf of y . Assuming this to be the case, eµt (ω) is anestimator of the eτ�quantile, ξt (eτ), where eτ is de�ned as the proportionof observations for which yt < eµt (ω), t = 1, ...,T .When used in this way we will denote the estimator eµt (ω) as eµt (eτ).However, it will not, in general, coincide with the time-varying eτ�quantileestimated directly since it weights the observations di¤erently. Inparticular, it is unlikely to pass through any observations.

Harvey (University of Cambridge) QES December 2007 24 / 40

Page 25: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Parameter estimation

The smoothing algorithms depend on parameters that can be estimated bycross validation. For time-varying quantiles, the function to be minimizedis

CV (τ) =T

∑t=1

ρτ(yt � eξ(�t)t ) (26)

where eξ(�t)t is the smoothed value at time t when yt is dropped; see DeRossi and Harvey (2006). A similar criterion, CV (ω), may be used forexpectiles.In a time invariant model with quantiles or expectiles following a randomwalk, Qt is a scalar equal to σ2η(τ) or σ2η(ω). We would like a suitableparameterization in terms of a quasi signal-noise ratio that is scaleinvariant. For the mean it will be recalled that ht = 2σ2t and so in a timeinvariant model the usual signal-noise ratio, σ2η/σ2, implies that gt = σ2

and κ = 0.5 in (22). A similar normalization can be applied for otherexpectiles so the quasi signal-noise ratios are qω = σ2η(ω)/σ2. Hence theiterations are based on (22) with gt set to one and κ = 0.5.For quantiles, σ2η/h is not scale invariant. We therefore consider the quasisignal-noise ratio, qτ = σ2η/g , with g de�ned so that qτ is scale invariant.Since the variance is not robust it is better to estimate the inter-quartilerange, r , and set g equal to its square. If the median is time-varying, it isestimated by setting gt = κ = 1 and r is estimated from the residuals; theestimated quasi signal-noise may then be divided by the square of theestimate of r so as to make it scale invariant. For the other quantiles theiterative scheme is applied with g set to one and κ = br .

Harvey (University of Cambridge) QES December 2007 25 / 40

Page 26: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Nonparametric regression with cubic splines

A slowly changing quantile can be estimated by minimizing the criterionfunction ∑ ρτfyt � ξtg subject to smoothness constraints. The cubicspline solution seeks to do this by �nding a solution to

minT

∑t=1

ρτfyt � ξ(xt )g+ λ2

�Zfξ 00(x)g2dx

�(27)

where ξ(x) is a continuous function with square integrable secondderivative, 0 � x � T and xt = t. The parameter λ2 controls thesmoothness of the spline. We show the same cubic spline is obtained byquantile signal extraction.The SSF allows irregularly spaced observations to be handled since it candeal with systems that are not time invariant. The form of such systems isthe implied discrete time formulation of a continuous time model. Thisgeneralisation allows the handling of nonparametric quantile and expectileregression by cubic splines when there is only one explanatory variable.The observations, which may be from a cross-section, are arranged so thatthe values of the explanatory variable are in ascending order.Harvey (University of Cambridge) QES December 2007 26 / 40

Page 27: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Bosch, Ye and Woodworth (1995) propose a solution to cubic splinequantile regression that uses quadratic programming. Unfortunately thisnecessitates the repeated inversion of large matrices of dimension up to4T � 4T . This is very time consuming. Our signal extraction appears tobe much faster (and more general) and makes estimation of the smoothingparameter (quasi signal-noise ratio) a feasible proposition.The fundamental property of quantiles continues to hold with irregularlyspaced observations. All that happens is that the SSF becomestime-varying. If there are multiple observations at some points then n, thetotal number of observations, replaces T , number of distinct points, in thesummation.

Harvey (University of Cambridge) QES December 2007 27 / 40

Page 28: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

An example of cubic spline regression is provided by the �motorcycledata�, which records measurements of the acceleration, in milliseconds, ofthe head of a dummy in motorcycle crash tests. The observations areirregularly spaced and at some time points there are multiple observations.Harvey and Koopman (2000) highlight the stochastic trend connection.Figure 2 shows the cubic spline expectiles obtained using the value ofσ2ζ /σ2 = 0.07 computed by CV for the mean. The graph gives a clearvisual impression of the movements in level and dispersion. If we count thenumber of observations below each expectile, they can be interpreted asquantiles if we are prepared to assume that the shape of the distribution istime invariant.

Harvey (University of Cambridge) QES December 2007 28 / 40

Page 29: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

5 10 15 20 25 30 35 40 45 50 55

­125

­100

­75

­50

­25

0

25

50

75

Obs ×  timeω =0.25 q=0.07 ×  timeω =0.75 q=0.07 ×  time

ω =0.05 q=0.07 ×  timeω =0.5 q=0.07 ×  timeω =0.95 q=0.07 ×  time

Figure: Cubic spline expectiles �tted to the motorcycle data. The parameter qµ isestimated by cross validation.

Harvey (University of Cambridge) QES December 2007 29 / 40

Page 30: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Conclusions ( so far)

Time-varying quantiles and expectiles are GOOD THINGS and provideinformation on various aspects of a time series, such as dispersion andasymmetry.Time-varying quantiles can be �tted iteratively applying a suitablymodi�ed state space signal extraction algorithm. The algorithm fortime-varying expectiles is much faster as there is no need to take accountof corner solutions. Satisfy the de�ning property of �xed quantiles inhaving the appropriate number of observations above and below it, whileexpectiles satisfy properties that generalize the moment conditionsassociated with �xed expectiles.Our model-based approach means that time-varying quantiles andexpectiles can be used for forecasting. As such they o¤er an alternative tomethods such as those in Engle and Manganelli (2004) and Granger andSin (2000), that are based on conditional autoregressive models.Equivalent to �tting a cubic spline. Because the state space form canhandle irregularly spaced observations, the proposed algorithms are easilyadapted to provide a viable means of computing spline-basednon-parametric quantile and expectile regressions.Harvey (University of Cambridge) QES December 2007 30 / 40

Page 31: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

But estimating the signal� noise ratio for quantiles takes a long time

Harvey (University of Cambridge) QES December 2007 31 / 40

Page 32: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

The dual: tracking the distribution

Harvey (University of Cambridge) QES December 2007 32 / 40

Page 33: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

A di¤erent, but complementary approach, is to pre-assign a value ξ andthen construct a binary series, I (yt � ξ). At any point in time

E (I (yt � ξ) = τt , t = 1, ...,T (28)

If the quantiles are �xed and known, setting ξ = ξ(τ) means thatE (It ) = τ. If the distribution changes, a discount parameter, ω, leads to

eτt+1jt = at+1jt/ �at+1jt + bt+1jt� , t = 1, ...,T ,

where

at+1jt = ωat jt�1 +ωIt , t = 1, ...,T (29)

bt+1jt = ωbt jt�1 +ω(1� It ) (30)

with a1j0 = b1j0 = 1. Note thateτt+1jt = Pr (It+1(τ) = 1jIj (τ), j = 1, ..., t) = eτt jt .Harvey (University of Cambridge) QES December 2007 33 / 40

Page 34: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

The log-likelihood function is

log L(ω) =T

∑t=1fIt ln eτt jt�1 + (1� It (τ)) ln(1� eτt jt�1)g, (31)

where eτ1j0 = 1/2.The �ltered estimates are an EWMA in the indicators, that is

eτt+1jt = eτt jt =t�1Σj=0

ωj It�j +ωt

t�1Σj=0

ωj + 2ωt

' (1�ω)t�1Σj=0

ωj It�j

with the terms in ωt not present if a1j0 = b1j0 = 0. It is reasonable toconstruct a two-sided smoothed estimator of the same form. This may bedone using the smoother for a Gaussian random walk plus noise withq = (1�ω)2/ω. In the middle of a large sample

eτt jT = 1�ω

1+ωΣjωj It�j (32)

Harvey (University of Cambridge) QES December 2007 34 / 40

Page 35: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

This is also the same problem as forecasting the sign of a variable, or moregenerally yt � ξ, where ξ is some pre-assigned value. Indeed it can be usedfor any binary series eg It = 1 if Cambridge win the boat race. Hereω = 0.877 (and q = 0.017)

1860 1880 1900 1920 1940 1960 1980 2000

0 .1

0 .2

0 .3

0 .4

0 .5

0 .6

0 .7

0 .8

0 .9

1 .0boat Level

Figure: Smoothed estimates of the probability of Cambridge winning the boatrace.

Harvey (University of Cambridge) QES December 2007 35 / 40

Page 36: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

By forming a whole grid a changing distribution function can be estimatedand tracked. There are N categories de�ned by the boundaries�∞, ξ1, ..., ξN�1,∞. Let It ,j = 1 if yt � ξ j , j = 1, ...,N � 1 and zerootherwise, ie I (yt � ξ j ), and de�ne IN ,t = 1 and I0,t = 0. Then

aj ,t+1jt = ωjaj ,t jt�1 +ωj Ij ,t , j = 1, ...,N, t = 1, ...,T (33)

with aj ,1j0 = 1, j = 1, ...,N, andeτj ,t+1jt = eτj ,t jt = Σji=1ai ,t+1jt/ΣNj=1aj ,t+1jt . The eτ0j ,t jts lie in the range[0,1] by construction and, of course, eτN ,t jt = 1. When the ω0s are thesame, the estimated series of probabilities, eτj ,t jt , cannot cross.

Harvey (University of Cambridge) QES December 2007 36 / 40

Page 37: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

The Dirichlet-multinomial or Polya predictive distribution leads to thelog-likelihood function

ln L(ω1, ..,ωN ) =T

∑t=1ln `t

where

ln `t = � ln Γ(ΣNj=1aj ,t jt�1)� ΣNj=1 lnfΓ(Ij � Ij�1 + aj ,t jt�1)/Γ(aj ,t jt�1)g,(34)

The binary likelihood, (31), is obtained when N = 2.Figure 4 shows a graph of estimates of the discount factors for GM withξ1, ..., ξN�1 set to the 5%, 10%,..., 95% quantiles from the raw(empirical) distribution. As expected there is very little discounting at andaround the centre of the distribution. Quadratic �tted by OLS. Thissuggests that a general approach for estimating ω0s as a slowly changingfunction of τ is to maximize the likelihood, (34), wrt a, b and c in thequadratic ω(τ) = a+ b+ cτ2. May set ω = 1 and/or impose symmetry.

Harvey (University of Cambridge) QES December 2007 37 / 40

Page 38: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

­.008

­.004

.000

.004

.008 0.980

0.985

0.990

0.995

1.000

2 4 6 8 10 12 14 16 18

Residual Actual Fitted

Figure: GM - quadratic �tted to estimated discount factors

Harvey (University of Cambridge) QES December 2007 38 / 40

Page 39: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

Time-varying quantiles may be extracted from the eτ0j ,t jts as follows.Suppose an estimate of ξt (τ) is required, and that eτk�1,t jt � τ � eτk ,t jtfor some k = 1, ..,N. Linear interpolation then yields

bξt (τ) = τ � eτk�1,t jteτk ,t jt � eτk�1,t jt fξk � ξk�1g+ ξk�1, t = 1, ...,T

What is the relationship between these estimates and those obtaineddirectly ? If eξt+j (τ) = ξ(τ) is constant, thenIQ(yt+j � ξ(τ)) = τ � It+j (yt � ξ(τ)), and so the condition

∑∞j=�∞ θjj jIQ(yt+j � ξ(τ)) = 0 implied by �tting the time-varying quantile

implies in turn that ∑∞j=�∞ θjj jIt+j (yt � ξ(τ)) = τ. A comparison with

(32) shows that ω = θ.

Harvey (University of Cambridge) QES December 2007 39 / 40

Page 40: Quantiles, Expectiles and Splines - QMUL Mathsbb/ACA_2007/Talks/ACA70_AndrewHarvey.pdfExpectiles, denoted µ(ω),0 < ω < 1, are similar to quantiles but they are determined by tail

So tracking the distribution may be better - but really only viable whenthere is no trend in mean or variance.

*********** THE END *********************

Harvey (University of Cambridge) QES December 2007 40 / 40