- Background
- Methodology
- MM Toolkit
Methodology
The legal indicators are measured by coding the presence or absence of rules, costs, procedures and obligations enshrined in legal regulation (primary and secondary) and policy frameworks enacted in the respective countries. To assure standardization and comparability of the data gathering process regarding the practice indicators, country researchers follow a methodology plan in which each of the 80 indicators are further operationalized in concrete mandatory and additional data types. The mandatory data types tap into the core building blocks of a practice indicator as described in the Monitoring Matrix Toolkit. They mandate the reporting of optimal information without which one could not be able to evaluate the respective indicator. The additional data specified for each practice indicator are reported if country researchers want to deepen and further illustrate specific practice indicator.
The data gathering strategy for the practice indicators is tailored to match the mandatory data types specified in the methodology plan. For each indicator there is a clear guidance on the data gathering strategy (instruments and sources) which should be utilized by country researchers. The specified data gathering instruments and sources follow an implicit hierarchy, in which publicly available factual data (e.g. official statistics) are the most important source of data for assessing practice indicators, followed by survey data from civil society organizations, which in turn is followed by relevant secondary sources (e.g. from CSOs reports, Ombudsman and media). Finally, at the end of the hierarchy are interview data, being subjective type of data, which covers smaller groups of respondents.
Data gathering
Data analysis and interpretation
To analyse and interpret the data, country researchers use a unified data collection template which provides the indicators description (including the mandatory and additional data types for the practice indicators) and five category descriptions ranging from fully enabling to fully disabling environment provided under each indicator. The five category descriptions are specified for each legal and practice indicator in the Monitoring Matrix Toolkit, to enable researchers – based on the reported data – to choose one code (score) which most accurately summarizes the state of enabling environment concerning the respective indicator. In a first step, the researcher reports the required data types collected through different sources in the template box. For example, they report factual data from primary sources complemented with descriptive statistics or cross tabulations based on survey data. In a second step, they choose one of five category descriptions specified for the respective indicator which best illustrates the reported data. The categories enable unified comparison of findings on the level of indicators across all country reports.
The assessment of the enabling environment through the categorization system was created in order to address the need for ‘compressed’ and effective visual communication of findings and systematic presentation of changes in the enabling environment for CSDev on the level of standards across countries and years. Moreover, the categorization system enables standardization of quality of the country and regional reports and contributes to more effective evaluation of indicators with the Monitoring Matrix Tool-kit. These are the categories/codes for assessing each indicator under the standards with the following explanations for legislation and practice:
| Category/code | Legislation | Practice | Scale |
|---|---|---|---|
| Fully disabling environment (1) | 1 | ||
| Disabling environment (2) | 2 | ||
| Partially enabling environment (3) | 3 | ||
| Enabling environment (4) | 4 | ||
| Fully enabling environment (5) | 5 |
