8
Sot. Ser. Med. Vol. 18, No. 10, pp. 873-880. 1984 Printed in Great Britain 0277-9536/84 $3.00 + 0.00 Pergamon Press Ltd THE INFORMATIVE CONTENT OF DIAGNOSTIC TESTS: AN ECONOMIC ANALYSIS* LOUIS R. EECKHOUDT’, THB&E C. LEBRUN’and JEAN-CLAUDE L. SAILLY’ ‘Facultir Catholique de Mom, 151 Chaussee de Binche, 7000 Mom, Belgium and ‘Centre de Recherches Economiques, Sociologiques et de Gestion, 1 Rue F. Bab, F-59046 Lille, Cedex, France Abstract-In medical literature the value of diagnostic tests is most of the time appreciated from their statistical properties (sensitivity and specificity) and from the prevalence of a given disease. In this paper we also take into account economic parameters such as the benefits and costs of the potential treatment under consideration and we present a simple and flexible algebraic expression of the informative value of a test. In this way the informative content of a test can be easily evaluated and the quality of an existing test can be compared to that of a theoretically ‘perfect’ one. Two examples taken from the medical literature illustrate our approach. 1. INTRODUCTION In these times of budgetary problems for the social security systems of industrialized countries, the tre- mendous increase in the demand for all sorts of diagnostic tests has become a matter of growing concern. It is thus worthwhile to develop a systematic analysis of the value and cost of such tests. Efforts in this direction have already been made and one can find in recent medical literature many references to the predictive value of various tests. The articles on this matter can be classified into two broad categories. The first one, the ‘statistical approach’ which is dominant in quantitive terms pays attention to three parameters: the prevalence of the disease, the probability of a positive test result for a sick patient (called ‘sensitivity’) and the probability of a negative result for a healthy person (‘specificity’). To evaluate the interest of a test, the authors draw a curve which links the probability of disease given a certain test result (the ‘post-test’ probability) to the pre-test or a priori probability of illness [l-3]. Of course the usefulness of a test increases when the post-test probability of illness is significantly different from the pre-test one but a precise quantification of the test interest is usually lacking in this approach. However, in a recent and very stimulating paper, Diamond er al. [4] by using the entropy notion [5] have been able to measure in ‘bits’ the reduction in diagnostic uncertainty brought about by the per- formance of a test?. This reduction in uncertainty *The authors have benefited from comments and encour- agements by colleagues in Economics and Medicine: M. Beuthe, Dr C. Brohet. Dr P.. Carpentier, A. Elstein, Dr S. Eraker, Dr J. M. Detry, Dr J. Ketelers, J. Kmenta, R. Milne, Dr C. Sulman and P. West. tSome authors before Diamond ef al. had already made use of the entropy measure. See for instance Refslb] and [7]. The two authors affiliated with CRESGE acknowledee the financial support of the Centre National de 1; Re- cherche Scientifique’ (C.N.R.S., Paris) and of the ‘Insti- tut National de la Sante et de la Recherche Medicale’ (I.N.S.E.R.M., Paris). depends upon the a priori probability of disease and because of the better quantification of the phenom- enon, exact comparisons of usefulness between exist- ing (and usually imperfect) tests become possible. Diamond et al. are able also to evaluate the currently available tests by reference to a theoretically perfect one. This is done through the computation of the ‘average informative content’ (see below). The second approach, which we call the economic one, has been developed mainly by Pauker and Kassirer (hereafter P-K). Besides the statistical prop- erties of the test presented above, these authors introduce, as parameters, the marginal benefits and costs that can be expected from the potential treat- ment under consideration. Indeed the authors be- longing to this approach convincingly argue that the effectiveness of a test rests upon its ability to lead to a better therapeutic maneuver than the one which would have been chosen without performing the test. The main tool of analysis in the economic ap- proach is the decision tree (see [8] for a detailed and very clear exposition). We will use it here to derive an algebraic expression that links the informative con- tent of a test to all the statistical and economic parameters of a given decision problem. As these parameters may be variable either from patient to patient for the same disease or even from one decision-maker to another for the same patient, the need for flexibility in the analysis is obvious and it can be met in a satisfactory way by the algebraic formula. With such an expression, we will also be in a position to set up, for the economic approach, a graphical analysis which will parallel that of Diamond et al. for the statistical approach. With the help of the formula developed in this paper, comparisons of economic effectiveness will be made very easily either between imperfect available tests or between an existing test and a theoretically perfect one. The interest of a simple formula linking the infor- mative content of a test to the basic parameters of the decision problem stems also from the need of per- forming sensitivity analyses in real world situations. Indeed, as stressed by Weinstein et al., most param- eters in a medical problem are not known with 873

The informative content of diagnostic tests: An economic analysis

Embed Size (px)

Citation preview

Page 1: The informative content of diagnostic tests: An economic analysis

Sot. Ser. Med. Vol. 18, No. 10, pp. 873-880. 1984 Printed in Great Britain

0277-9536/84 $3.00 + 0.00 Pergamon Press Ltd

THE INFORMATIVE CONTENT OF DIAGNOSTIC TESTS: AN ECONOMIC ANALYSIS*

LOUIS R. EECKHOUDT’, THB&E C. LEBRUN’ and JEAN-CLAUDE L. SAILLY’ ‘Facultir Catholique de Mom, 151 Chaussee de Binche, 7000 Mom, Belgium and

‘Centre de Recherches Economiques, Sociologiques et de Gestion, 1 Rue F. Bab, F-59046 Lille, Cedex, France

Abstract-In medical literature the value of diagnostic tests is most of the time appreciated from their statistical properties (sensitivity and specificity) and from the prevalence of a given disease. In this paper we also take into account economic parameters such as the benefits and costs of the potential treatment under consideration and we present a simple and flexible algebraic expression of the informative value of a test. In this way the informative content of a test can be easily evaluated and the quality of an existing test can be compared to that of a theoretically ‘perfect’ one. Two examples taken from the medical literature illustrate our approach.

1. INTRODUCTION

In these times of budgetary problems for the social security systems of industrialized countries, the tre- mendous increase in the demand for all sorts of diagnostic tests has become a matter of growing concern. It is thus worthwhile to develop a systematic analysis of the value and cost of such tests.

Efforts in this direction have already been made and one can find in recent medical literature many references to the predictive value of various tests.

The articles on this matter can be classified into two broad categories. The first one, the ‘statistical approach’ which is dominant in quantitive terms pays attention to three parameters: the prevalence of the disease, the probability of a positive test result for a sick patient (called ‘sensitivity’) and the probability of a negative result for a healthy person (‘specificity’). To evaluate the interest of a test, the authors draw a curve which links the probability of disease given a certain test result (the ‘post-test’ probability) to the pre-test or a priori probability of illness [l-3]. Of course the usefulness of a test increases when the post-test probability of illness is significantly different from the pre-test one but a precise quantification of the test interest is usually lacking in this approach. However, in a recent and very stimulating paper, Diamond er al. [4] by using the entropy notion [5] have been able to measure in ‘bits’ the reduction in diagnostic uncertainty brought about by the per- formance of a test?. This reduction in uncertainty

*The authors have benefited from comments and encour- agements by colleagues in Economics and Medicine: M. Beuthe, Dr C. Brohet. Dr P.. Carpentier, A. Elstein, Dr S. Eraker, Dr J. M. Detry, Dr J. Ketelers, J. Kmenta, R. Milne, Dr C. Sulman and P. West.

tSome authors before Diamond ef al. had already made use of the entropy measure. See for instance Refslb] and [7].

The two authors affiliated with CRESGE acknowledee the financial support of the Centre National de 1; Re- cherche Scientifique’ (C.N.R.S., Paris) and of the ‘Insti- tut National de la Sante et de la Recherche Medicale’ (I.N.S.E.R.M., Paris).

depends upon the a priori probability of disease and because of the better quantification of the phenom- enon, exact comparisons of usefulness between exist- ing (and usually imperfect) tests become possible. Diamond et al. are able also to evaluate the currently available tests by reference to a theoretically perfect one. This is done through the computation of the ‘average informative content’ (see below).

The second approach, which we call the economic one, has been developed mainly by Pauker and Kassirer (hereafter P-K). Besides the statistical prop- erties of the test presented above, these authors introduce, as parameters, the marginal benefits and costs that can be expected from the potential treat- ment under consideration. Indeed the authors be- longing to this approach convincingly argue that the effectiveness of a test rests upon its ability to lead to a better therapeutic maneuver than the one which would have been chosen without performing the test.

The main tool of analysis in the economic ap- proach is the decision tree (see [8] for a detailed and very clear exposition). We will use it here to derive an algebraic expression that links the informative con- tent of a test to all the statistical and economic parameters of a given decision problem. As these parameters may be variable either from patient to patient for the same disease or even from one decision-maker to another for the same patient, the need for flexibility in the analysis is obvious and it can be met in a satisfactory way by the algebraic formula. With such an expression, we will also be in a position to set up, for the economic approach, a graphical analysis which will parallel that of Diamond et al. for the statistical approach. With the help of the formula developed in this paper, comparisons of economic effectiveness will be made very easily either between imperfect available tests or between an existing test and a theoretically perfect one.

The interest of a simple formula linking the infor- mative content of a test to the basic parameters of the decision problem stems also from the need of per- forming sensitivity analyses in real world situations. Indeed, as stressed by Weinstein et al., most param- eters in a medical problem are not known with

873

Page 2: The informative content of diagnostic tests: An economic analysis

874 LOUIS R. EECKHOUDT et al.

certainty so that the decision-maker may wish to know how the optimal decision is affected by changes in one or many parameters. As it will be shown, the answer to such a question can be readily obtained from the simple formula developed in this paper.

In order to reach our objective, we have organized the paper as follows. In the next section, we present the main parameters of the model and the character- istics of the optimal decision when no test is available. The formula giving the informative content of a test are derived from a decision tree in Section III and Section IV contains two examples taken from the medical literature.

II. THE BASIC MODEL

We consider an individual who has gone through a clinical examination on the basis of which the decision-maker derives the a priori probability (p) that the patient suffers from a given disease.

As in most (if not all) medical articles on this matter, we make use of the following assumptions:

only two states of nature are possible: either the disease is prevalent (D) or it is not (6). The proba- bility of B is, of course, 1 -p;

in order to fight the disease, the decision-maker avails upon a single treatment which he can decide to use or not to use. His decision will therefore be: ‘treat’ or ‘do not treat’;

the decision-maker is assumed to know with cer- tainty the consequences of his decisions under each state of the world.

Given these assumptions, we can draw the usual table giving the outcomes of alternative decisions under various states of the world (Table 1).

The interpretation to be given to the four par- ameters a,, varies a lot from problem to problem. Even when a single ‘objective’ attribute is used (such as survival) it can be expressed either in terms of life-expectancy or as the five-year survival rate (see the second example in Ref. A). Sometimes the two attributes are combined: as is the case in an applica- tion of decision theory to the treatment of the solitary thyroid nodule [lo], where the outcomes represent weighted averages of mortality and morbidity. Re- cently, a more ‘subjective’ approach has been used [l I] following a procedure of ‘utility elicitation’ de- scribed previously [8].

In this case, the consequences are evaluated in the tradition of risk theory by presenting to the decision- maker or the patient ‘basic reference gambles’ among which he is asked to reveal his preferences. Of course, given the ‘subjectivity’ of this approach, great differences of appreciation may exist between the decision-maker or patients involved.

The developments made in this paper are com- patible with any interpretation of the ay’s and for the following examples, we need only to observe that the

*In some examples, a,, is set equal to zero and az2 is equal to unity. However as any linear and monotonically increasing transformation of the outcomes is allowed, other values may be chosen for a,, and uz2. For a very lucid exposition about utility measurement, the reader is referred to Ref. [8].

outcomes are ranked by:

a22 > alI, a,, > a,,. (1)

Indeed the best possible situation is that of a healthy person who is not treated (az2) and the worst outcome corresponds to the position of a sick person left without treatment (Q,)*. a,, and a,, are the ‘inter- mediate outcomes’ and although in general a,? ex- ceeds a,,, this is not always so (see the difference between the two cardiologists in our second ex- ample).

Besides, space can be saved by introducing the following notation:

b E a,, - u2, c = u2? - a,> (2)

and because of equation (1) both b and c are positive. b is in fact the marginal benefit obtained by treating a sick person while c is the marginal cost associated with the (wrong) decision of treating a healthy per- son.

Given these specifications, the decision tree corre- sponding to Table 1 takes the form shown in Fig. 1.

AS usual, squares (circles) indicate decision (chance) nodes.

The expected outcome E[R] attached to each deci- sion is given by:

E[R(‘treat’)] =p a,, + (1 -p) a,, (3)

this value is assigned to the chance node A.

E[R(‘no treat’)] = p u2, + (1 - p) az2 (3’)

this value assigned to the chance node B. Of course the decision to treat will be taken

whenever

P~II +(1 -P)%>p6 +(1 --P)+

or because of equation (2) when

c p>-.

c+b (4)

Conversely for a priori probabilities in the interval [0, c/(e + b)] the best decision is not to treat.

D;P

i : a”

D;P

No treat’ >

I3

B;l-P

Fig. I

Page 3: The informative content of diagnostic tests: An economic analysis

The informative content of diagnostic tests 875

Notice that from equations (3), (3’) and (4), one can easily derive the value of the expected outcome corresponding to the best decision (E[R*]) in terms ofp:

for

o<p< -& E[R*] = a22 -Ha,* - a211

for

~~P~1:E[R*]=a,2-p(a,2-u,,). c+b

(5)

(5’)

It is easily checked that at p = c/(c + b), the two expressions in equations (5) and (5’) yield the same result, and in Appendix 1 we plot the curve E[R*] in order to give a graphical interpretation of the value of information.

So far, we have devoted much attention to the decision making process when no test is performed. Before analyzing the informative content of a test in the next section, we briefly describe here how the statistical quality of a test is measured.

As indicated in the Introduction, the medical litera- ture focuses rightly on two characteristics of a test:

its sensitivity or true positive rate S (see e.g. [12]) which is the probability of a positive result of a test (T’) when the patient is sick. This conditional proba- bility is sometimes denoted p (i”+/D);

its specificity or true negative rate s, i.e. the proba- bility of a negative result (T-) when the test is applied to a healthy person [p(r-/a)].

Given these values of S and s, one can find, using Bayes’ theorem, the predictive ability of the test, i.e.

p(DIT+) = SP

SP + (1 - J)(l -P) (7)

and

p(d/T-) = su -P)

s(l -p) + (1 - S)p’ (7’)

Finally, the implementation of the test itself may induce some costs and/or some risks to the health status of the patient. Following P-K, we denote these costs by R,, and we express them in the same units as the coefficients a,,.

111. THE NET VALUE OF THE INFORMATION DERIVED FROM A TEST

The purpose of this section is to build upon the foundation laid down in Section II in order to measure the gross and net values of information (GVI and NVI) derived from a test. Although NV1 is the ultimate goal of the calculation, we first deter- mine GVI which represents an intermediary step in our analysis. Following the literature [13], we define

tAn alternative definition is sometimes proposed (see e.g. [14]). However it gives the same result as equation (8) which seems to us more appealing at an intuitive level.

$For the readers less familiar with the decision tree tech- nique. Appendix 1 shows a graphical interpretation of GVI.

GVI as followsi:

GVI = E[R *( T)] - E[R *] (8)

where E[R l ( T)] stands for the maximal level of the expected outcome that the decision-maker can reach with the aid of the test. As we already know the second term on the right hand side of equation (8), we focus our attention upon the first term which will be easily understood with the help of the decision tree technique applied to the decision of performing a testi.

In Fig. 2, we describe the sequence of events that take place when a test is ordered before a therapeutic decision is made.

If a test is performed, it has a cost or a risk factor evaluated at -& and attached to the decision node of testing. If a positive test result is obtained (T+), an event which, through Bayes’ theorem, has a proba- bility

p(T+) = Sp + (1 - s)(l -p) (9)

the a posteriori probability of illness, p(D/T+), is given by equation (7). On the basis of this revised probability, we can easily compute the expected outcomes attached to the decisions ‘treat’ or ‘no treat’ and we affect them to the chance nodes C and E.

When T+ occurs it can easily be shown that ‘treat’ is chosen whenever

p(DIT+) Z 2

and the optimal expected outcomes conditional upon T+, denoted E[R*(T+)] are given by:

for

0 <p(DIT+) < &, E[R*(T+)]

= a22 -P(DIT+)(Q~~ - a2,) (lo)’

for

-& GP(DIT+) < 1, E[R*(T+)]

= 42 -_p(DIT+)(a,, - a,,). (10’)

Table 1.

States of the world D d

Decisions

a wiori orobabilitles

‘Treat’ ‘Do not treat’

Values of p

Table 2.

Corresponding values of GVI

GVI=O

GVI=bSp -c(l -s)(l -p)

GVI = b(S - 1)~ + cs(l -p)

GVl-0

Page 4: The informative content of diagnostic tests: An economic analysis

876 Louis R. ~ECKHOUDT et al.

‘Treat

C

5, P(D/T+l

D , p ID/T+)

‘No treat’ 0

E

b, p(D/T+)

a (2

0 22

D, p(D/T-1 a

II

Treat

0 F

6, pi D/T-J a

3 12

D, p t D/T-i a

21

‘No treat’ >

G

B , p ID/T-)

a 22

Fig. 2.

Through a similar reasoning one can obtain E[R *(T-)1, the optimal expected outcome reached if the test result turns out to be negative. The expression for E[R*(T-)] will be similar to equations (10) and (10’) except that p(D/T+) has to be replaced by /0/T-).

Having obtained E[R*(T+)] and E[R *(r-)1 the optimal gross expected outcome attached to the testing procedure, E[R*(T)] is a (non conditional) expectation given by:

E[R*(T)] =p(T+).E[R*(T+)] +p(T-).E[R*(T-)]. (11)

GVI can be found by combining equations (8) (9), (lo), (10’) and the remark made above about E[R*(T-)].

The practical computations necessary to obtain GVI in terms of the basic parameters of the decision problem are easy but lengthy and are not reproduced here?. The results are shown in Table 2 and are presented graphically in Fig. 3.

In Fig. 3, besides the GVI curve, we also plot the cost of the test which is represented by a horizontal line at the level &.

tThey can be obtained by writing to any of the authors. After submission of the present paper, we became aware of a paper by P. Doubilet [15] who gives an elegant account of the mathematical procedure by using the notion of the ‘expected gain in utility’.

It is worth mentioning that the maximum value. of GVI occurs at p = c/(c + b) where its level is:

cb(S + S - 1)

c+b

We are now able to compute the net value of information which is given by:

NV1 = GVI - R,,. (12)

When &, is strictly positive, NV1 may be negative in some intervals (between 0 and cx as well as between j3 and 1 in Fig. 3). A rational decision-maker will not perform the test so that in these intervals NV1 will in

GVI f

c(1-S) Q c

Sb+c(l-sl - csl

c+b b( I-S)+CS

Fig. 3

Page 5: The informative content of diagnostic tests: An economic analysis

The informative content of diagnostic tests

Table 3 NVI

Values of p Corresponding values of NV1

04PGcl NW=0

a<p<L c+b

NVI=bSp-c(l-s)(l-p)-R,,

$“PG NV1 = b(S - 1)~ + a(1 -p) - R,

BSOSl NVI=O

fact be equal to zerot. Between a and /I NV1 is positive and its value is presented in Table 3 and Fig. 4 which will be intensively used in the examples of next section.

So far, we have considered ‘imperfect’ tests. In the present context, a perfect test might be defined by the conditions$:

S = s = 1 (the false positive and false negative rates are equal to zero)

and &, = 0 (no risk associated with the performance of

the test). If these values are introduced in Table 3 or in Fig.

4, one gets the net value of perfect information (NVPI) which obviously is at least equal to NV1 for each value of p. Besides for any p, NVPI is strictly positive except for p = 0 or p = 1. Indeed when the a priori probability corresponds to a situation of certainty (,a = 0 or 1) nothing can be gained in terms of information even from a perfect test. The NVPI is an interesting reference point since it enables com- parisons of efficiency between an existing test and a theoretically perfect one for each possible value of p. In this sense, it can give an idea of the sectors in which research should be undertaken to improve the existing testing procedures.

Before turning to the examples, a comparison with the paper by Diamond et al. [4] is in order. Diamond et al.‘s approach is statistical and they also compute and plot the informative content of a test in terms of the a priori probability of illness. They also derive an ‘average information content’ which is the integral under the information curve (see their Appendix 1). In our economic approach, we can also obtain such a synthetic measure: it will be the area under the NV1 curve of Fig. 4. Easy computations show that this area. denoted A, is given by:

,q=i [cb(S + s - 1) - R,(c + b)]2

-(c + b)(CS + b(1 - S))(bs + c(1 -S)) (13)

tTo account for this remark, the correct expression for NV1 is: NV1 = Max[GVI - R,. 01.

:In the ‘statistical’ approach, a perfect test is defined only by the condition that S and s are equal to unity, regardless of the risks that it involves. In the present approach where the costs and benefits of a treatment are considered, it is quite logical to include also the costs or risks of the test in the analysis. Notice that with such a definition. no existing test will be perfect since given the current state of the arts, the statistically perfect tests are also the most invasive ones (e.g. the angiogram in the detection of coronary artery diseases).

cblS+s-lI_R

c+b O------ 8

/A

Y; \ -P 0 L B 1

c+b Fig. 4.

which is a function of all the parameters of the decision problem. For a perfect test, this rather cumbersome expression simplifies greatly into a new one denoted A,,.

cb &J=; -. [ 1 c+b

(14)

Of course the relationship between A and A, (e.g. their ratio which lies betwen 0 and 1) is an indicator of the average efficiency of an existing test, by reference to a perfect one.

Instead of taking the value of the area under the curve of informative content for the whole interval [0, 11, one sometimes limits oneself to a smaller interval if the prevalence of illness is known to lie between two probability levels p0 and p,. In that case, equations (13) and (14) are no longer valid but the comparison between an existing test and a perfect one is still made by computing the areas under NV1 and NVPI in the interval [p,,p,].

All the equations presented in this section may look intricate. However once the basic parameters of the problem are available, the numerical values of NVI, NVPI, A and 2, can be easily computed with a hand calculator so that cost-benefit comparisons between tests become readily available. The flexibility of our approach is now illustrated with the help of two examples.

Iv. TWO EXAMPLES

In an important paper published in 1980, P-K have analyzed the same problem as the one discussed here. However, they have limited their developments to the computation of two ‘probability thresholds’ for a given test. These thresholds are:

the ‘testing’ threshold (denoted 2”,) which gives a level of p below which the best decision is to withhold both the test and the treatment;

‘The ‘test-treatment’ threshold (T,,) that is the value of p above which treatment should be under- taken without testing.

Thus, the best course of action will be to perform a test if and only if p lies somewhere between T, and T In.

In their article, P-K give two examples to illustrate their method. We will use one of them here to show

Page 6: The informative content of diagnostic tests: An economic analysis

878 LOUIS R. EECKHOUDT et al.

that our approach not only gives the values r, and T,,, but also enables the analyst to quantify the efficiency of a test and to compare it to that of a perfect one.

We have drawn our second example from a recent article by Eraker and Sasse [ 111 devoted to an appre- ciation of the serum digoxin test in searching for digoxin toxicity. We have chosen this case because it will illustrate the flexibility of our approach in situ- ations where each decision-maker has his own subjec- tive estimation of the consequences of his potential decisions. Besides in this example, there is some doubt about the true values of S and s so that the flexibility brought about by our approach is very useful for a sensitivity analysis.

The effectiveness of a needle biopsy of the kidney

This first example is taken from P-K [9] (the reader is referred to the basic article for a full medical description of the problem). Briefly, the case is that of a 55-year-old man who has been suffering from severe hypertension for 5 years with no history of renal disease. However this patient has been on corticosteroid therapy for 24 hours because he is suspected of developing renal vasculitis.

In this problem steroid responsive vasculitis is regarded as disease (D) and steroid unresponsive hypertension as no disease (6). The treatment is the continuation of corticosteroid therapy for 1 or 2 months.

Considering the potential benefits and costs of treatment as well as the a priori probability of renal vasculitis for this patient, the authors arrive at the following estimatest:

b = 0.38 c = 0.05, and p = 0.65.

The test that can be used before a therapeutic decision is made, is a needle biopsy of the kidney. For this test:

s = 0.90 s = 0.95

and the risk of severe complications due to the performance of the test is 2% (R, = + 0.02).

With this information, we can draw from Table 3 the NV1 curve for the needle biopsy which is the lower curve in Fig. 5.

The values of a (0.0653) and p (0.3216) are nothing but thresholds T, and T,,, in P-K’s article. Besides, we can compare the efficiency of the biopsy to that of a theoretically perfect test given by the upper curve in Fig. 5.

tThe detail of the computations is given in P-K [9], pp, I12 and 113.

$The same inverse relationship between the likelihood ratio and the entropy measure has also been observed in the statistical approach. For instance, Diamond ef al. ob- serve that the failure of the likelihood ratio to “correlate positively with information content suggests that it is a misleading index of test effectiveness” ([4, p. 9211).

$The reader is referred to the original paper to see how the evaluations have been obtained. See also [8. Chap. 71 for an example of ‘preference elicitation’.

NVI , NVPI

iiLL.-..

0065

Fig. 5.

As for this patient, p = 0.65, corticosteroid therapy should be pursued without performing a biopsy. Indeed despite its great statistical reliability, this test involves too high risks for the patient. With the help of our formula, it is easily found that a test with the same reliability (S = 0.9 and s = 0.95) but with no risk of complication would have a positive informa- tive content up to p = 0.55 while at this level of a priori probability a perfect test has still an informa- tion content of 0.022, i.e. one half of its maximal value.

As the ratio between A and A, is very low ( = 0.193), we can find an illustration of the difference between the ‘statistical’ and ‘economic’ approach. The needle biopsy has a rather high likelihood ratio (S/l -s = 18) and seems very efficient from this statistical point of view. However when the economic parameters are considered, its average efficiency relative to that to that of a perfect test drops considerab1y.S

It also follows from our analysis that because of the low value of A/A,+ efforts should be undertaken to increase the reliability of the biopsy still further and to decrease the risks associated with its performance.

The efectiveness of the serum digoxin test in searching for digoxin toxicity.

Patients who are treated for heart troubles with digoxin products may develop digoxin toxicity. As it is harmful to discontinue digoxin for a non-toxic patient or to continue the treatment if toxicity is present, the therapeutic decision involves rather im- portant risks.

Eraker and Sasse [ 111 have examined the usefulness of the serum digoxin test in reducing the uncertainty surrounding the diagnosis and treatment of toxicity. We now use their data to show to which extent a change either in the evaluation of the consequences or in the estimation of S and s can affect the informative content of a test.

In this problem, D = digoxin toxicity; D = no tox- icity; treatment = discontinue digoxin; and no treatment = continue digoxin.

As in this field the appreciation of consequences is very subjective, five cardiologists have been inter- viewed, about the consequences of various decision@. Results for two cardiologists are shown in Table 4:

Page 7: The informative content of diagnostic tests: An economic analysis

The informative content of diagnostic tests 879

0 225 A NW, NVPI 0225’ NW, NVPI

01575

A, %

0 1025

oI1l; :;;; _;

0 0.04 025 060

0 oOl70125 0 390 1 0 0125 0.215 i

Fig. 6.

As far as the test is concerned it is reasonable to consider the result positive whenever the serum di- goxin concentration exceeds 2.0 ng/ml. The literature also exhibits great divergences in the estimation of S and s for this test. The following values are found:

S, = 0.80 s, = 0.90

(weighted mean value from a survey of the literature)

SZ = 0.50 s2 = 0.96

(in the study by Eraker and Sasse). Finally, let us mention that for this test, & is equal to zero.

In Fig. 6, the NV1 curves in four cases, denoted A,, A,, B, and B,, were obtained by combining the cardiologists’ evaluations with the possible values of S and s. As a reference the NVPI curves are also shown.

It appears from Fig. 6 that even if S, and s, are used cardiologist A will always order a serum digoxin test if one agrees that the prevalence of toxicity lies somewhere between 0.1 and 0.3 at hospital admis- sion. However because of his different appreciation of the parameters b and c, cardiologist B will have a lower demand for the test: e.g., if he believes that S,

Table 4.

Cardiologist A Cardiologist B D b D 6

?reat 0.9 0.7 0.7 0.9 No treat 0 1.0 0 1 .a

b, = 0.9: cA = 0.3: b, = 0.7: cg = 0.1.

Table 5.

Cardiologist A b = 0.9 c = 0.3

Cardiologist B b = 0.7 c = 0.1

S, = 0.80 0.0225

S, = 0.90 - = 0.652 0.0345

S2 = 0.50 0.01458

S> = 0.96 - = 0.416 0.0350

and s1 prevail, he will not use the serum digoxin test whenever p > 0.215. Indeed in this case his decision will always be to stop the digoxin treatment and even a negative test result would not change his line of action so that NV1 is equal to zero.

In order to have an idea of the relative efficiency of the test, we have computed the area under the NV1 and NVPI curves. As it is widely admitted that p lies around 0.2, we have taken the areas under the curve in the interval [O. 1, 0.31 and in Table 5 we indicate for each case the ratio between the area under NV1 and that under NVPI. These areas are given respectively by the numerator and by the denominator of each fraction in Table 5.

Table 5 deserves a few comments: An interesting comparison can be drawn between

the likelihood ratio and the economic approach. If one uses the estimates of Eraker and Sasse (S, and sl) the corresponding likelihood ratio is 0.5/0.04) = 12.5 and it significantly exceeds the one obtained for S, and s, which is equal to 0.8/0.1 = 8. Thus the Eraker and Sasse finding about S and s indicates an im- provement in the statistical reliability of the test. However when the economic parameters are consid- ered, it appears that for both cardiologists the relative efficiency of the serum digoxin test falls when one is shifted from (S,, s,) to (S,, 5,);

If S, and s, prevail, both cardiologists will find the serum digoxin test relatively efficient since its average informative content exceeds one half of what would be obtained from a perfect test. However if the pair (S,, sJ is considered, the relative efficiency of the test becomes very sensitive to ‘the value of the parameters b and c: although the serum digoxin test remains very appreciated by cardiologist A, its usefulness for cardi- ologist B appears dubious.

V. CONCLUSION

In this paper, we have set up for the economic approach a quantitative measure of the effectiveness of a test which can be rather easily computed. In this

Page 8: The informative content of diagnostic tests: An economic analysis

880 LOUIS R. EECKHOUDT et al.

E[R’l(

0 piD/T-I & p, plD/T+) I

Fig. 7

way the gap of efficiency between an existing test and a theoretically perfect one can be quickly evaluated and various tests in the same field can be compared.

This approach also appears to be a necessary first step for the analysis of more complicated problems. Indeed in this paper we have adhered to the usual assumptions made in the medical literature when decision problems are examined. These assumptions have been listed at the beginning of Section II and they may look too restrictive in many real-world situations. Indeed it often happens that many treat- ments are available to fight an illness which presents various degrees of severity. Besides, more than one test may be appropriate for the diagnosis and in this case one will be interested in discovering the optimal sequence of tests. Of course such difficult problems are beyond the scope of the present paper but the flexibility brought about by our approach will hope- fully make possible the analysis of the more intricate matters.

REFERENCES

Diamond G. A. and Forrester J. S. Analysis of proba- bility as an aid in the clinical diagnosis of coronary artery disease. New Engl. J. Med. 300,1350-1358, 1979. Epstein S. E. Implications of probability analysis on the strategy used for noninvasive detection of coronary artery disease. Am. J. Cardiol. 46, 491-499, 1980. R&in R. D. and Hood W. B. Bayesian analysis of electrocardiographic exercise stress testing. New Engl. J. Med. 297, 681-686, 1977. Diam0nd.G. A., Hirsch M., Forrester J. S., Staniloff H. M.. Vas R.. Haloern S. W. and Swan H. J. C. Application of information theory to clinical diagnostic testing. The electrocardiographic stress test. Circulafion 63, 915-921, 1981.

5.

6.

7.

8.

9.

IO.

11.

12.

13.

14.

15.

Theil H. Economics and Information Theor?.. North- Holland, Amsterdam, 1967. Metz C. E., Goodenough D. J. and Rossmann K. Evaluation of receiver operating characteristic curve data in terms of information theory with application in radiography. Radiology 109, 297-304. 1973. McNeil B. J., Keeler E. and Adelstein J. S. Primer on certain elements of medical decision making. .Vew Encl. J. Med. 293, 211-215, 1975. Weinstein M. C.. Finebere H. V.. Elstein A. S.. Frazier H. S., Neuhauser D., N&tra R. R. and McNeil B. Clinical Decision Analysis. W. B. Saunders. Philadel- phia, 1980. Pauker S. G. and Kassirer J. P. The threshold approach to clinical decision making. New Engl. J. .Mrd. 302. 1109-1116, 1980. Sisson J. C., Bartold S. P. and Bartold S. L. The dilema of the solitary thyroid nodule: resolution through deci- sion analysis. Semin. Nucl. Med. VIII, 59-71. 1978. Eraker S. A. and Sasse L. The serum digoxm test and digoxin toxicity: a Bayesian approach to decision mak- ing. Circulation 640, 409-420, 1981. Griner P. F., Mayewski R. J., Mushlin A. I. and Greenland P. Selection and interpretation of diagnostic tests and procedures. Annls intern. Med. 94, 553-600. 1981. Gould J. P. Risk, stochastic preference and the value of information. J. Econ. Theory 8, 6484. 1974. Hirshleifer J. and Riley J. G. The analytics of uncer- tainty and information. An expository survey. J. Econ. Lit. XVII, 1375-1421, 1979. Doubilet P. A mathematical approach to interpretation and selection of diagnostic tests. Med. Deck. Making 3, 177-195, 1983.

APPENDIX 1

For people less familiar with the decision tree technique. we give in the Appendix a graphical illustration of the GVI concept.

From equations (5) and (5’) we can easily draw the curve linking E[R*] to p. We obtain in Fig. 7 a piecewise linear function denoted g/c.

The kink takes place at p = c/(c + b) and if the a priori probability of illness is p0 the corresponding expected out- come is equal to the ordinate of 1 when no test is performed.

If a test is used, p0 gives rise to two a posteriori proba- bilities p(D/T’) and p(D/T-). The distance between these two probabilities depends upon pO, Sand s [see equations (7) and (7’)]. At each a posteriori probability corresponds an expected outcome given by the ordinates of m and n. In fact the ordinate of m is nothing but E[R*(T-)] and that of n gives E{R*(T+)]. The expected outcome with the test, E[R*(T)] lies somewhere on the straight line mn since it is a linear combination of E[R*(T’)] and E[R*(T-)] [see equation (1 l)]. It can then easily be shown that E[R*( T)] is found at the ordinate of the line mn evaluated at point pO, that is 4. The GVI is simply equal to the distance ql when P =po.

To find the NVPI at p = pO, one would take the ordinate at p0 of the straight line joining i and k (not drawn in Fig. 7). NVPI would then be the distance between that ordinate and that of point I.