Average duration pro-deposits IOA. If the number of calendars is high, it is important to limit data aggregation in order to identify possible variations in the permanent data of two observers. The average duration IOA algorithm per deposit achieves this by determining an IOA score for each timing, and then by deifing them by the total number of timings in which the two observers collected data. Note that this approach is similar to the approach described above of partial agreement at regular intervals. In the example of Figure 3, there were 99.7, 2.3, 69.2 and 92.7% approval levels for intervals 1 to 4, respectively. The average of these four levels of the agreement results in an average of 66% per event agreement – a much more conservative estimate than that of the statistics of the total duration of the IOA. IoA with undotted interval. The IOA algorithm with a little interval (also called “non-deposit” agreement in the research literature) is also stricter than simple interval-by-interval approaches, taking into account only intervals in which at least one observer records the lack of response. The justification for pointless IOA is similar to that of the IOA with the scored interval, except that this metric responds best for high rates (Cooper et al., 2007). In the figure 2 examples, the 5th and 6th intervals are ignored for calculation purposes, as both observers have received a response at these intervals. Thus, the IOA statistics are calculated from the remaining five intervals.

Since agreement has only been reached on three of the five intervals (the second, third and fourth intervals), the approval rate is 60%. Since Spearman and Brown first reported in 1910 how to quantify reliability between data sources, researchers have refined ways of quantitatively communicating the link between independent actions (see Brennan, 2001). Today`s researchers are fortunate to have many statistical software packages (z.B. SPSS, STATA) that can instantly calculate a variety of reliability indicators, such as the Cronbach alpha or Pearson product torque coefficient.