1. The Evaluation Model
The adopted evaluation model for analyzing and executing social policies in Brazilian municipalities is organized in a monitoring, evaluation and control panel. This means that it is organized as a set of result indicators, selected in order to measure the performance of different dimensions in health and education actions put in practice by the municipalities. The indicators included in the panel may be interpreted either individually or as a whole.
In health, for example, one of the selected indicators is the early neonatal mortality rate which allows us to measure the performance of state governments regarding their attention to neonatal health, parturition and the newborn. These indicator values are calculated for each Brazilian municipality
Furthermore, the performance for each municipality can be evaluated by the set of selected indicators in the panel, summarized in two final indexes: municipal health and education indexes. For such, the result indicators were converted in comparable measures presented in numeric form as shown in Figure 1, and obtained from a total sum of standardized indicators (scale from 0 to 1). It notes that in the case of municipal education index, were selected 10 indicators for the model, whereas for the health index, only eight 1.
This conversion of the original values in comparable measures allows us to evaluate the performance of each municipality in the panel. The performance can also be analyzed by the final index which consists in the sum of obtained values by the municipality in each indicator, according the last column in Figure 1. Therefore, each indicator has the same weight in the evaluation model. Altogether, the indicators make for a signaling matrix for municipality performance regarding different dimension in their health and education policies.
In cases where no information is available for a certain indicator we adopted the following procedures, whether or not we took into account the responsibility of the municipality for the lack of information.
Inference by denial – this situation only occurs for the health indicators. In Figure 1, municipality number 3 did not submit information to the official organs regarding indicator 5. This fact indicates that the municipality denied being evaluated by the national information systems – it did not submit, for example, any information to the DATASUS. In this particular situation for the health indicator measurement this information will have the same effect as the value of “0” (zero) for the specific standardized indicator, since this index is obtained from a sum of all standardized indicators 2.
Inference by exclusion – there are situations in which the lack of information on a municipality for a specific indicator does not designate a disposition in not being evaluated, but in fact the inability of attributing to the municipality a responsibility in the offering of the evaluated services. This situation is exclusive to the health indicators. In figure 1 it would be the case for municipality 5, regarding indicators 5 and 6. One example of this situation would be an indicator concerning students from 5th to 8th grade in fundamental education. Some municipalities do not offer grades 5th to 8th. Therefore, in this situation they cannot be evaluated in any situation where this universe is taken into account. The absence of the information is then related to an absence of responsibility. In this case the final index of the municipality only takes into account the set of valid indicators corresponding to an average of available values – the final score of 5,48 for municipality 5 was calculated from an average of the 8 available indicators (4,38 / 8 = 0,5475) multiplied by 10, which corresponds to the total number of indicators.
Notes
1 Initially, the health index was also composed of 10 indicators. However, on account of problems in terms of the updating of data on "child health" and "infectious diseases (dengue)," these two dimensions were excluded from the analysis.
2 It is noteworthy that, in this version, the health index is calculated from the following formula: Index health = (sum of the standardized indicators in the range 0-1 / 8) X 10
