Establishing a data collection system
A well-functioning data collection system helps the Fund Operator collect, store, aggregate and report results data accurately and efficiently.
Within six months of the signing of the Programme Implementation Agreement, the Fund Operator should submit to the FMO a detailed description of the management and control systems, which includes data collection and storage and reporting systems and procedures for results reporting from Project Promoters, including the descriptions of projects and the qualitative narrative and quantitative indicator reporting, and for the Fund Operator’s monitoring for results.
While establishing a data collection system is a requirement, the FMO does not regulate what this should look like. We suggest several good practices in this section. Essentially, a data collection system involves trained people who understand their role, a plan for how and when to implement certain processes, and a platform for collecting and storing data. See below.
Data collection system – people, plan, platform
There are two main types of data that need to be collected: qualitative data and quantitative data.
Qualitative results data
Qualitative data is the narrative text that is submitted by Project Promoters to the Fund Operator for submission in the Project Level Information (PLI) in GrACE. Qualitative data is quite straightforward to collect and report, since it does not require aggregation of results from multiple projects. The main challenge with qualitative data tends to be ensuring the Project Promoter submits good quality text. This means text that uses a logical structure, short sentences, avoids jargon and abbreviations and is understandable to a general audience.
The Fund Operator should review the qualitative data required in the PLI. This means ensuring the call text and the project application template is aligned with the qualitative data required in the PLI. This should allow the Fund Operator to simply copy/paste good quality text from the project application into the Project Level Information form in GrACE. Likewise, the Fund Operator should align the final project report template with the final Project Level Information section.
We recommend that the Fund Operator organises its data platform system by project. The data platform may be SharePoint, Dropbox, Google Drive, or any shared drive managed by the organisation, including platforms used for EU or national data collection. This means each project should have its own organised folder with documents including the project proposal, project contract, project progress reports, and final project report, among others. The data collection system should include the qualitative Project Level Information (PLI) registered when the project is signed and when the project is finalised. Data should not be stored solely on an employee’s computer and must be backed up regularly.
Quantitative results data
Quantitative results data refers to the numerical data reported by multiple projects for reporting into the programme results framework. Quantitative data is often more difficult to manage than qualitative data because it is more complex and needs to be aggregated for each indicator in the results framework.
Project Promoters report quantitative data in progress reports and when a project is completed. The Fund Operator needs to store quantitative data from multiple projects in a structured way that facilitates accurate reporting into GrACE. This could be setup using Excel, Google Sheets, or a database management system. A suggested data structure is presented below. Data from each project must then be aggregated for each indicator in the programme results framework. Incorrect or missing data can be a serious issue for reporting programme results.
We recommend that the Fund Operator has an assigned data quality officer to ensure data quality. The data quality officer should control versions of documents. It is not recommended to overwrite raw data when new data is received. Instead, create a new worksheet or file for each reporting year. When mistakes occur, it is useful to be able to look back to the previous year’s data.
Example of a data structure
Data collection plan
We encourage the Fund Operator to develop a data collection plan early in the implementation period. A data collection plan establishes what data will be collected, when, how and by whom. It may be thought of as a manual for how the data collection system should work. The data collection plan can be a simple Word document that defines the following:
- What data to collect: Identify which projects need to report data for each indicator, considering the type of data (e.g., cumulative numbers, percentages) and sources (e.g., project records, surveys). Project contracts should state what data is required from Project Promoters. The Civil Society Fund core indicators library, to be provided by the FMO, will have a basic description and methodology. For custom indicators outside of this library, make a brief note about what counts and what does not count to ensure each indicator is measured in the same way by each Project Promoter.
- When to collect data: Determine deadlines for when Project Promoters need to report data to allow the Fund Operator enough time for analysis and inclusion in the Annual/Final Programme Report. Communicate deadlines clearly to Project Promoters.
- Who is responsible: Typically, Project Promoters report data to the Fund Operator (except for the indicators on bilateral relations, where the FMO collects data). Indicators should be referenced in project contracts, and responsibilities for data collection and quality control should be clearly assigned.
- How data will be stored: Choose which digital platform will be used to store the data (e.g. SharePoint, Dropbox, Google Drive or a custom platform) and in which file format (e.g. Excel, Google Sheets, Access, or other database). Consider who should have permissions to modify the data, how the data should be backed up, and any rules around consistent folder structures and file naming. This may take some time to setup but will save time later in the implementation period and ensure quality data reporting.
Data flow from collection to reporting
Verifying data quality
Verifying data quality involves checking that data is correct, up-to-date and complete:
- Correct data: It’s not always possible to visit projects just to verify data, but checks can be made during monitoring visits. If data seems incorrect, do not accept it: contact the Project Promoter for clarification. Common errors include:
- Reporting a lower cumulative value than the previous report due to confusion with annual numbers. Cumulative numbers should always increase unless there is a clear reason, which should be explained in reports.
- Reporting a results value lower than the baseline, suggesting a worsening situation, often due to miscounting.
- Reporting a results value far higher than the target value, possibly due to incorrect counting methods, like mixing up online visitors with physical attendees.
- Complete data: Complete data means that all projects that are required to report against an indicator have done so. If the project is yet to achieve any results for an indicator, the Project Promoter should still report ‘0’. Missing values mean missing data that needs verification. Data should also be disaggregated as required by project contracts.
- Up-to-date data: Project Promoters should provide the latest data for inclusion in the Annual Programme Report. If results have not changed since the last report, an explanation should be provided.
FMO collecting and reporting indicator data for bilateral indicators
Data for the bilateral indicators is collected by the FMO, and not by the Fund Operator. Following finalisation of each project with a bilateral relations component, a short survey will be emailed to both the Project Promoter and entities in the Donor States. The survey will ask the respondents to report their views for each bilateral indicator.
Additionally, the survey will ask about the role of each partner in project activities. The survey will be anonymous.