Calibrating Items With Time Use Diaries: A Refined Method

The Indicator Provided Via the Diary

The following is a description of the indicator that is derived from the use of a daily diary. After determining the total number of subjects who will have to keep a diary (N), we will build sub-samples, each composed of N/D individuals. The number of sub-samples is equal to D, which is the period (defined in days) during which the diaries are kept. Typically, the period D is equal to 364 days.⁵

We can now organize the data in the form of a matrix composed of N/D rows and D columns and calculate the ratio $P=\frac{1}{N}\cdot {\sum }_{j=1}^D{\sum }_{i=1}^{N/D}{x}_{i,j}$ with $x=\left\{\mathrm{0,1}\right\}$ — i.e. the proportion between positive events (${\sum }_{j=1}^D{\sum }_{i=1}^{N/D}{x}_{i,j}$) and possible events (N) — (Rossi & Scappini, 2014). As will be seen, there will be a need to decompose the overall value P into I subgroups. In this case we will identify different values of $p_i$ with $p_i=\left\{p_1,\dots ,p_I\right\}$. We will call the statistics $P$ and $p_i$ with the term measured presence (hereinafter also just presence).

However, $P$ is a “poor” indicator of information because, as we have seen, the subjects surveyed on the various days belong to different sub-samples. It is therefore not possible to select the part of the population that carries out a given activity within a specific range of intensity for periods longer than the extension of the diary.

The Indicator Provided Via the Stylized Item

Daily diaries are not always sufficient to provide an adequate answer, as there are activities with relatively long time cycles. In such cases, it would be necessary to select the part of the population that carries out a given activity over a longer period of time: for example, one week, one month, or longer intervals.

The typical solution to this problem is to employ a questionnaire with a suitable item that can be used to determine how frequently each subject performs a specific activity over a given period, which is usually one year. Ideally, if n is the number of days in the given period (generally n = 364), the n + 1 values $f_t$ can be calculated, each of which provides the number of people attending mass $t$ times, with t ranging from 0 to 364. Each ratio $f_t/N$, where N is the size of the sample, provides the daily attendance rate for each single value of t. This can also take the form of a cumulative rate to indicate the proportion of people who perform an activity at least t times per year — $CF\left(t\right)={\sum }_t^{364}{f}_t/N$, $\forall t\ge t$.

Having defined the indicators derived from the use of the two survey instruments, we now need to make these measures comparable.

The Conversion From Frequency to Presence

As is known, although the presence values provided by diaries cannot be converted into frequencies provided by items, the reverse process is possible. Using a similar approach to other authors (Presser & Chaves, 2007), to perform this conversion we have to add together the number of people and the relative typical frequency $t$ — thereby identifying the positive events ${\sum }_{t=0}^{364}\left(f_t\bullet t\right)$with ${\sum }_{t=0}^{364}{f}_t=N$ — and dividing the result by the number of possible events — $\left(N\bullet 364\right).$ In formal terms, $P\left(t\right)=\left({\sum }_{\mathrm{t}=0}^{364}\left(f_t\bullet t\right)/\left(N\bullet 364\right)\right)\bullet 100$.

However, it is unrealistic to ask respondents for such precise occurrence about their attendance at mass over the course of a year. In general, as in this case, it is preferable to offer a limited number of answer options I, that correspond to the frequency $t_i$ for each option i of the item. If now we set $s_i=\left(t_i/364\right)\bullet 100$ with $s_i=\left\{s_1,\dots ,s_I\right\}$, then $S={\sum }_1^I{f}_t\bullet s_i/N$. We will call the statistics $S$ and $s_i$ with the term stylized presence. To make this conversion, we must tackle an additional problem: identifying the values of frequency $t_i$.

Before carrying out this task, we need to present the data.

The Simulated Data

The first dataset consists of targeted constructed data in order to clearly highlight the differences in the application of the four models and will not be used to present a real application.⁶ The criteria that guided the construction of this data are twofold. The first was to visualize graphically in a more distinguishable manner the outcome of the application of the calibration models, a result not achievable with the real data. The second was to highlight the situations in which bias may emerge due to the application of the models to be discussed below.

The TUS 2008 Data

The second dataset is made up of TUS 2008 data belonging to the more general ISTAT Multipurpose Survey System which was generally conducted every five years. The survey used here was carried out in the timeframe February 2008–January 2009 (TUS, 2008). The respondents kept a diary over a 1-day period and recorded what they were doing (every 10 minutes) and where they were. In addition, they answered a detailed stylized questionnaire.

The sample consisted of 18,240 families with response rate equal to 73.96% (American Association for Public Opinion Research response RR1). A further selection due to the non-response diaries must be added to this sample dropout. In this case, of the 43,460 eligible diaries, including only subjects with 3 years or more, 40,944 were collected, broken down as follows: 14,787 relate to a weekday (Monday–Friday), 13,286 relate to a Saturday, and 12,871 relate to a Sunday. For the purposes of this study, we will single out respondents aged between 18 and 74, with a final sample of 30,673 people. To minimize the potential bias, the analysis was weighed by day of the week, gender, age, level of urbanization and multi-regional area.

As highlighted earlier, to make a comparison the same activity (here, religious practice) needs to be surveyed using both a diary and a suitable item.

For the diary, we used the codes regarding religious practice in places of religious worship.⁷ The minimum period of time considered in diaries is 10 minutes and is associated with the main activity carried out in that timeframe. A subject was counted as “present at mass” if there were at least two minimum-length episodes in their compiled daily diary, thus corresponding to attendance for a time equal or greater than 15 minutes.

Regarding the stylized item, the question used is: “How often do you usually go to church or another place of worship?”. For the available options the respective values of $s_i$ are identified with the following values applied $t_i$ where: 0 times for the option “never” with $s_1$ = 0.00; 6 times for “a few times a year” with $s_2$ = 6/364⋅100 = 1.65; 24 times a year for “a few times a month (but less than four times)” with $s_3$ = 6.59; 52 times a year for “once a week” with $s_4$ = 14.29; 182 times a year for “a few times a week” with $s_5$ = 50.00; and, finally, 364 times a year for “every day” with $s_6$ = 100.00.⁸

The Uniform Model

We will now consider an item administered to a sample of N individuals, where the possible response options I correspond to data values or frequency ranges. Let us now identify the sub-sample $n_i$ that selected the response option $i$, from which we will derive probability that will be equal to $d_i=n_i/N$, with $i=\left\{1,\dots ,I\right\}.$ If we now set $p_i=\left\{p_1,\dots ,p_I\right\}$ with $p_{i-1}<p_i$, and $d_i=\left\{d_1,\dots ,d_I\right\}$, then $P\left(X=p_i\right)⟹D=d_i$, where $d_i$ is the fraction of the population that carries out a given activity with a measured presence in the sub-sample $i$ equal to $p_i$. We note that the assumption $p_{i-1}<p_i$ is important because it guarantees that there is a reasonable link between declared behaviour — as expressed in the item — and measured behaviour — as noted in the diary (Scappini, 2021).

If, instead of a categorical variable, we assume that X is a continuous variable, with area $D=d_i$ and $X~U\left[\frac{p_{\mathrm{i}-1}+p_{\mathrm{i}}}{2},\frac{p_{\mathrm{i}}+p_{\mathrm{i}+1}}{2}\right]$, then $P\left(X=x_i\right)⟹D=y_i$, where $y_i$ is now the population density with measured presence equal to $x_i$ with $x_i\in \left[\frac{p_{i-1}+p_i}{2},\frac{p_i+p_{i+1}}{2}\right]$. We will now calculate the coordinates of ($x_i, y_i)$ $\forall i=\left\{2, \dots ,I-1\right\}.$

Starting from the abscissa values ($x_i$), the corresponding ordinate values will be $y_i=\frac{d_i}{\left(p_{i+1}-p_{i-1}\right)/2}$. For the two tails, if $i=1$, we have $y_1=\frac{d_1}{\left(p_1+p_2\right)/2-p_0}$ with $p_0\in [0,p_1)$, while if $i=I$, we have $y_I=\frac{d_I}{p_{I+1}-\left(p_{I-1}+p_I\right)/2}$ with $p_{I+1}\in (p_I,100]$. Given these assumptions, we can now build the calibration function. The uniform $PDF$, hereinafter called $u\left(x\right)$, will be defined as follows:

Then the relative uniform $CDF,$hereinafter $U\left(x\right)$, is equal to:

defined as $\forall i=\left\{\mathrm{1,2},\dots ,I\right\}$.

In this way, we obtained an initial result that is much less subject to bias than the one derived solely from the reliance on an item. However, the assumption of uniform distribution is extremely improbable in practice. It is unlikely the PDF pattern would feature break at the transition between the different values of $u\left(x\right)$. A more reasonable assumption is that the development is more progressive. In the next section, we will describe a solution to this problem.

The Linear Model

If we use the values $y_i$ defined with the uniform distribution and set $m_i=\frac{y_i-y_{i-1}}{p_i-p_{i-1}}$ and $q_i=\frac{p_i\bullet y_{i-1}-p_{i-1}\bullet y_i}{p_i-p_{i-1}}$ $\forall i=\left\{\mathrm{2,3},\dots ,I\right\}$, we can now develop a model that better responds to the above mentioned criteria of progression: the linear $PDF$, hereinafter called $l\left(x\right)$, will be defined as follows:

The relative linear $CDF,$ hereinafter $L\left(x\right)$, will be equal to:

defined as $\forall x_i$ with $i=\left\{\mathrm{1,2},\dots ,I+1\right\}$.⁹

This formulation has the advantage of a better graduality in the development of the values of $l\left(x\right)$ and, therefore, in the development of the values of $L\left(x\right)$, as well as producing a non-discontinuous function. Let us now analyse why the application of the two models presented may be subject to bias.

The Bias in the Models

We are now going to introduce the information entered in Figure 1. Let’s start from $u\left(x\right)$, which is a probability density function, whose trend is determined by the values of $p_i$ and $d_i$ and is marked as $u\left(p_i\right)$ on the graph. The values indicated with $u\left({(p}_{\mathrm{i}}{+p}_{\mathrm{i}+1})/2\right)$ together with the line marked as Area ($d$) are useful to delimit the relevant reference areas. The continuous line shows the trend of the calibrated values $U\left(x\right)$, while the associated symbols on the same line, ${U(p}_{\mathrm{i}})$ and ${U(s}_{\mathrm{i}})$, correspond to the specific calibrated values. In the first case, the calibration will be calculated for the values of $p_i$, and therefore with reference to the measured presence, values which, we note, are usually helpful only to aid the reading of the graph, while in the second case the calibration will be calculated for the values of $s_i$, and therefore with reference to the stylized presence.

Click to enlarge

Figure 1

Calibrated Uniform Model, $\mathbf{u}\left(\mathbf{x}\right)$ and $\mathbf{U}\left(\mathbf{x}\right)$

Before continuing, we would like to point out that it is possible to calibrate $F\left(x\right),$ $\forall x\in \left[p_0,p_{I+1}\right]$, while the comparisons between $CF\left(x\right)$ and$F\left(x\right)$are feasible only for ${x=s}_i.$

From the comparison between Figure 1 and Figure 2, the improvement in terms of smoothness of linear vs uniform calibration is evident. The problem now arising is that neither of these formulations — $u\left(x\right)$ and $l\left(x\right)$ — guarantees that:

Click to enlarge

Figure 2

Calibrated Linear Model, $\mathbf{l}\left(\mathbf{x}\right)$ and $\mathbf{L}\left(\mathbf{x}\right)$

In general, with the uniform calibration this doesn’t happen, as normally $E\left[u\left(X\right)|x_{\mathrm{i}}\right]\ne p_i$. In addition, the assumption of a gradual development of the function may generate a further asymmetry in the distribution of the probabilities $f\left(x\right)$. Therefore, even though $E\left[u\left(X\right)|x_{\mathrm{i}}\right]=p_i$, in general it will still be the case that $E\left[l\left(X\right)|x_{\mathrm{i}}\right]\ne p_i$.

Since it is not possible to generalize the attractive assumption discussed above, we propose as an alternative to take only the part that contributes in terms of CDF for a value related to the one given by $E\left[f\left(X\right)|x_{\mathrm{i}}\right]=p_i$, basically ignoring what happens after the value of $p_i$. Therefore, as the average value divides the area of the part in two so that:

$F\left(p_i\right)-F\left(\frac{p_{i-1}+p_i}{2}\right)=\frac{d_i}{2}$

it is possible to consider equally attractive the following occurrence of equality:

Figure 1 shows an example of an ideally non-problematic situation, in which Equation (1) is verified, for $u\left(x_5\right)$ with $d_5^{-}=d_5^{+}$.¹⁰ However, if Equation (1) is not verified, then it is possible to regard this breach as a bias factor due to the calibration model. The Figure also shows the example of this situation for $u\left(x_2\right)$ with $d_2^{-}\ne d_2^{+}$.

To solve all the above problems, we need to look at an alternative model which we are going to illustrate.

The New Uniform Model

If we assume that X is a continuous variable, with $X~U^{-}\left[\frac{p_{\mathrm{i}-1}+p_{\mathrm{i}}}{2},p_{\mathrm{i}}\right)$ and $X~U\left[p_{\mathrm{i}},\frac{p_{\mathrm{i}}+p_{\mathrm{i}+1}}{2}\right)$ with an area $D=d_i/2$, we can now calculate the respective coordinates of ($x_i^{-}, y_i^{-})$ defined $\forall i=\left\{\mathrm{2,3},\dots ,I\right\}$ and ($x_i, y_i)$ defined $\forall i=\left\{\mathrm{1,2},\dots ,I-1\right\}$. Starting from the abscissa values $x_i,$ with $x_i\in \left[p_0,p_{I+1}\right]$, the corresponding ordinate will be equal to:

and

Regarding the two tails, we have if $i=1$, $y_1^{-}=\frac{d_1/2}{p_1-p_0}$ with $p_0\in \left[0,\right.\left.p_1\right)$, and if $i=I+1$, $y_{I+1}=\frac{d_I/2}{p_{I+1}-p_I}$ with $p_{I+1}\in \left[p_I,100\right]$. We can now develop the modified calibration.

The new uniform $PDF$, hereinafter $nu\left(x\right)$, is then equal to:

If we now set $d_0=0$, then the relative $CDF$, hereinafter $nU\left(x\right)$, is equal to:

defined as $\forall i=\left\{\mathrm{1,2},\dots ,I+1\right\}$.

We note that there are two special cases, where $p_0=p_1$, and $p_{I+1}=p_I$. In both these situations calibration for values of $x_1$ and $x_I$ is not possible. Then, the values of $nU\left(x\right)$ will be calculated assuming that $x_i$ is discrete and we will put respectively $nU\left(X=p_1\right)=d_1/2$ in the first case, and $nU\left({X=p}_I\right)=d_I/2$in the second.

While it is true that this model is not subject to bias since by definition $nU\left(p_i\right)={\sum }_{j=0}^{j=i-1}{d}_j+\frac{d_i}{2}$, the function $nU\left(x\right)$ is discontinuous (see Figure 3), and the resulting values are less smoothed out compared to those shown in Figure 2.

Click to enlarge

Figure 3

Calibrated New Uniform Model, $\mathbf{n}\mathbf{u}\left(\mathbf{x}\right)$ and $\mathbf{n}\mathbf{U}\left(\mathbf{x}\right)$

To sum up, we have now achieved a first result: a calibration model that is not subject to bias. However, it is also true that the assumption of uniform distribution is, in practice, very unlikely. Similarly to what we have already pointed out to justify the transition from $n\left(x_i\right)$ to $l\left(x_i\right)$, it can be considered unrealistic to have “breaks” between adjacent values of $nu\left(x_i\right)$ in the trend of the PDF, while it would seem more reasonable to assume that the trend from $nu\left(x_i\right)$ to $nu\left(x_{i+1}\right)$ is more progressive. We will address the issue in the next section.

The New Linear Model

As in the case of the Linear model, if we use the values $y_i$ defined with the new uniform distribution and we place $m_i^{*}=\frac{y_i^{-}-y_{i-1}}{p_i-p_{i-1}}$ and $q_i^{*}=\frac{p_i\bullet y_{i-1}-p_{i-1}\bullet y_i^{-}}{p_i-p_{i-1}}$ $\forall i=\left\{\mathrm{2,3}, \dots ,I\right\}$, while leaving the two tails unchanged, a model can be developed that better meets the progressivity criteria now mentioned.

The new linear $PDF$, hereinafter $nl\left(x\right)$, is then equal to:

If we now set $d_0=0$, then the relative $CDF$, henceforth $nL\left(x\right)$, will be equal to:

defined as $\forall i=\left\{\mathrm{1,2},\dots ,I+1\right\}$.

Similar to the previous method, we observe that in the two special cases, those in which $p_0=p_1$, and $p_I=p_{I+1}$, the values of $nL\left(x\right)$ will be calculated without calibration: in the first case we will assume that $nL\left(X=p_1\right)=d_1/2$, while in the second that $nL\left({X=p}_I\right)=d_I/2$.

If we take a look at Figure 4, we find we have a more attractive calibration model than the previous ones. While this is not, in general, a continuous model, like the Linear — $L\left(x\right)$ — it is nevertheless a correct model and more smoothed out than the new Uniform — $nU\left(x\right).$¹¹

Click to enlarge

Figure 4

Calibrated new Linear Model, $\mathbf{n}\mathbf{l}\left(\mathbf{x}\right)$ and $\mathbf{n}\mathbf{L}\left(\mathbf{x}\right)$

We will now present the results of applying the calibration models to the TUS 2008 data.

Future Work

However, this article does not fully address several important topics that require further investigation. While the implemented applications effectively demonstrate the model by meeting its minimum criteria, further research would be useful. For example, a study assessing the adequacy of the overlap between the survey items and diary-recorded activities, as some activities may not fully satisfy these assumptions. Additionally, refining model fit measures and calculating confidence intervals for parameters are necessary steps. The current approach to model fit is not completely satisfactory, but no better alternative has yet been identified.

Future research will prioritize resolving these issues to significantly improve the model’s robustness and broaden its applicability.