Data quality - Revision history

Nils Duepont: /* Dimensions of data quality in WeSIS */

2025-01-20T12:16:10Z

Dimensions of data quality in WeSIS

← Older revision		Revision as of 12:16, 20 January 2025
Line 56:		Line 56:
	\|-		\|-
	\| CRC Compiled Data		\| CRC Compiled Data
	\| Data gathered, harmonized and/or extended by CRC researchers, i.e., ~~merely~~ statistical data aggregated via reproducible workflows (scripts) incl. imputations.		\| Data gathered, harmonized and/or extended by CRC researchers, i.e., foremost statistical data aggregated via reproducible workflows (scripts) incl. imputations.
	\|-		\|-
	\| 3rd Party Data		\| 3rd Party Data

Nils Duepont: /* Introduction */

2024-09-12T18:46:18Z

Introduction

← Older revision		Revision as of 18:46, 12 September 2024
Line 27:		Line 27:
	The proposal entailed rating three dimensions (trustworthiness of a source, reasonableness of data, and confidence in data), each connected to a suggested rating scale drawn from other disciplines e.g., the likelihood scale from the IPBES "Guide on the production of assessment" (IPBES 2018).		The proposal entailed rating three dimensions (trustworthiness of a source, reasonableness of data, and confidence in data), each connected to a suggested rating scale drawn from other disciplines e.g., the likelihood scale from the IPBES "Guide on the production of assessment" (IPBES 2018).
	The main outcome of the discussion was that the first two dimensions were quite uncontroversial although the suggested scales needed refinement and adoption to the peculiarities of the CRC's data collections.		The main outcome of the discussion was that the first two dimensions were quite uncontroversial although the suggested scales needed refinement and adoption to the peculiarities of the CRC's data collections.
	However, the discussion also showed that despite a common understanding of the value of such ratings, social science is still marked by a resistance to "judge" the data given the extra work necessary to define a feasible scale and wordings ''and'' apply it, ~~and~~ concerns about "devaluing one's own work" or deviations and bias across people and projects were raised.		However, the discussion also showed that despite a common understanding of the value of such ratings, social science is still marked by a resistance to "judge" the data given the extra work necessary to define a feasible scale and wordings ''and'' apply it. Furthermore, concerns about "devaluing one's own work" or deviations and bias across people and projects were raised.

	Two subsequent workshops underlined the difficulty of balancing the commitment for implementing a rating scheme vs. prospect of creating an added value vs. (additional) work load required to provide and code the necessary information.		Two subsequent workshops underlined the difficulty of balancing the commitment for implementing a rating scheme vs. prospect of creating an added value vs. (additional) work load required to provide and code the necessary information.

Nils Duepont: /* Dimensions of data quality in WeSIS */

2024-06-11T09:38:10Z

Dimensions of data quality in WeSIS

← Older revision		Revision as of 09:38, 11 June 2024
Line 99:		Line 99:

	If multiple concerns are raised for the focal value, only the most prevalent one is coded.		If multiple concerns are raised for the focal value, only the most prevalent one is coded.
	If a researcher chooses "Other" as the response, it is ~~mandatory~~ to provide an explanation in the [[Optional_column_names\|optional column]] <code>comment</code> in the WeSIS upload file template.		If a researcher chooses "Other" as the response, it is recommended to provide an explanation in the [[Optional_column_names\|optional column]] <code>comment</code> in the WeSIS upload file template.
	As the concerns are expressed at the value level, they are input together with the actual data via the mandatory column <code>plausibility_concerns</code> of the [[File_formats\|file upload templates]].		As the concerns are expressed at the value level, they are input together with the actual data via the mandatory column <code>plausibility_concerns</code> of the [[File_formats\|file upload templates]].
	In WeSIS, the information is available in the data table of each indicator page.		In WeSIS, the information is available in the data table of each indicator page.


	== Technical implementation ==		== Technical implementation ==

Hsalzmann at 09:10, 11 June 2024

2024-06-11T09:10:59Z

← Older revision		Revision as of 09:10, 11 June 2024
Line 52:		Line 52:
	! Meaning		! Meaning
	\|-		\|-
	\| Original CRC ~~data~~		\| Original CRC Data
	\| Data gathered, compiled and coded by CRC researchers, e.g., qualitative judgments, text codings or annotations.		\| Data gathered, compiled and coded by CRC researchers, e.g., qualitative judgments, text codings or annotations.
	\|-		\|-
	\| CRC ~~compiled data~~		\| CRC Compiled Data
	\| Data gathered, harmonized and/or extended by CRC researchers, i.e., merely statistical data aggregated via reproducible workflows (scripts) incl. imputations.		\| Data gathered, harmonized and/or extended by CRC researchers, i.e., merely statistical data aggregated via reproducible workflows (scripts) incl. imputations.
	\|-		\|-
	\| 3rd ~~party data~~		\| 3rd Party Data
	\| Data taken "as is" without further treatments from third party sources, except for matching the source's country codes to the WeSIS scheme and adding the necessary meta data required for upload to WeSIS.		\| Data taken "as is" without further treatments from third party sources, except for matching the source's country codes to the WeSIS scheme and adding the necessary meta data required for upload to WeSIS.
	\|}		\|}

Nils Duepont: /* Technical implementation */

2024-05-31T19:15:07Z

Technical implementation

← Older revision		Revision as of 19:15, 31 May 2024
Line 119:		Line 119:
	While the codings remained undetermined at that time, some data providers made use of them inputting their own remarks, reasonings and quality judgements. To not lose this information,		While the codings remained undetermined at that time, some data providers made use of them inputting their own remarks, reasonings and quality judgements. To not lose this information,
	* the information contained in the DB was kept,		* the information contained in the DB was kept,
	* but both columns ~~were~~ marked as deprecated ~~in the documentation in WeSISpedia~~;		* but both columns are now marked as deprecated;
	* the validation check ignores whether the upload file (still) contains the old columns, or not, and treats them as [[Optional_column_names\|optional column names]]; this way, the researchers are still able to update "their" information for data already existent before the above agreement was reached.		* the validation check ignores whether the upload file (still) contains the old columns, or not, and treats them as [[Optional_column_names\|optional column names]]; this way, the researchers are still able to update "their" information for data already existent before the above agreement was reached.

	For existing data already in the DB, the ~~second~~ rating is not available (and will not be unless the data is updated); until the codings will gradually become available with an indicator update, it was agreed that WeSIS will display a "Not Available" in the data table for the time being.		For existing data already in the DB, the plausibility rating is not available (and will not be unless the data is updated); until the codings will gradually become available with an indicator update, it was agreed that WeSIS will display a "Not Available" in the data table for the time being.

	== References ==		== References ==

Nils Duepont: /* Technical implementation */

2024-05-30T21:02:45Z

Technical implementation

← Older revision		Revision as of 21:02, 30 May 2024
Line 107:		Line 107:

	To implement the agreed solution, with the release of WeSIS version 2.5 the upload workflow was modified.		To implement the agreed solution, with the release of WeSIS version 2.5 the upload workflow was modified.
	After discussing the pros and cons of ~~three~~ different variants (e.g., parsing from the indicator table in WeSISpedia, adding a mandatory column to the upload file template, or exploiting the very first line of the csv file), it was decided to grab the information for the label from the indicator ''table'' pages in WeSISpedia.		After discussing the pros and cons of different variants (e.g., parsing from the indicator table in WeSISpedia, adding a mandatory column to the upload file template, or exploiting the very first line of the csv file), it was decided to grab the information for the label from the indicator ''table'' pages in WeSISpedia.
	As these are also used for the validation check during the upload simply adding a column "data_origin" to the table of each policy field served the purpose.		As these are also used for the validation check during the upload simply adding a column "data_origin" to the table of each policy field served the purpose.
	This column is now mandatory, standardized, and of type "String" allowing only the above mentioned three categories for coding the origin of data, else nothing will be displayed in WeSIS.		This column is now mandatory, standardized, and of type "String" allowing only the above mentioned three categories for coding the origin of data, else nothing will be displayed in WeSIS.
	The database (DB) structure was affected in that one additional column called "data_origin" was added to the DB table "indicators".		The database (DB) structure was affected in that only one additional column called "data_origin" was added to the DB table "indicators".

	For the plausibility concerns, the [[File_formats\|file upload templates]] were appended to include a new column "plausibility_concerns".		For the plausibility concerns, the [[File_formats\|file upload templates]] were appended to include a new column "plausibility_concerns".
	This column is mandatory, standardized, and of type "String" allowing only the above mentioned six categories for coding researcher's concerns.		This column is now mandatory, standardized, and of type "String" allowing only the above mentioned six categories for coding researcher's concerns.
	The DB structure was affected in that on additional column called "plausibility_concerns" was added to the database tables "ds_values" and "ds_dyadic_values".		The DB structure was affected in that one additional column called "plausibility_concerns" was added to the database tables "ds_values" and "ds_dyadic_values".

	The [[File_formats_archive\|first version]] of the file upload template in place until November 2023 already included two placeholders for a data quality "rating", namely the then mandatory columns "data_quality" and "data_quality_confidence" as placeholders for a "traffic light" or some other scheme.		The [[File_formats_archive\|first version]] of the file upload template in place until November 2023 already included two placeholders for a data quality "rating", namely the then mandatory columns "data_quality" and "data_quality_confidence" as placeholders for a "traffic light" or some other scheme.

Nils Duepont: /* Introduction */

2024-05-30T21:00:17Z

Introduction

← Older revision		Revision as of 21:00, 30 May 2024
Line 27:		Line 27:
	The proposal entailed rating three dimensions (trustworthiness of a source, reasonableness of data, and confidence in data), each connected to a suggested rating scale drawn from other disciplines e.g., the likelihood scale from the IPBES "Guide on the production of assessment" (IPBES 2018).		The proposal entailed rating three dimensions (trustworthiness of a source, reasonableness of data, and confidence in data), each connected to a suggested rating scale drawn from other disciplines e.g., the likelihood scale from the IPBES "Guide on the production of assessment" (IPBES 2018).
	The main outcome of the discussion was that the first two dimensions were quite uncontroversial although the suggested scales needed refinement and adoption to the peculiarities of the CRC's data collections.		The main outcome of the discussion was that the first two dimensions were quite uncontroversial although the suggested scales needed refinement and adoption to the peculiarities of the CRC's data collections.
	However, the discussion also showed that despite a common understanding of the value of such ratings, social science is still marked by a resistance to "judge" the data given the extra work necessary to define a feasible scale and wordings ''and'' apply it, and concerns about "devaluing one's own work" or deviations and bias across people and projects.		However, the discussion also showed that despite a common understanding of the value of such ratings, social science is still marked by a resistance to "judge" the data given the extra work necessary to define a feasible scale and wordings ''and'' apply it, and concerns about "devaluing one's own work" or deviations and bias across people and projects were raised.

	Two subsequent workshops underlined the difficulty of balancing the commitment for implementing a rating scheme vs. prospect of creating an added value vs. (additional) work load required to provide and code the necessary information.		Two subsequent workshops underlined the difficulty of balancing the commitment for implementing a rating scheme vs. prospect of creating an added value vs. (additional) work load required to provide and code the necessary information.

Nils Duepont: /* Introduction */

2024-05-30T20:58:00Z

Introduction

← Older revision		Revision as of 20:58, 30 May 2024
Line 21:		Line 21:
	Seldom adressed, or even made explicit, in social science research, they still are at the core of many scientist's common-sensical understanding of "data quality", particularly given their domain-specific knowledge.		Seldom adressed, or even made explicit, in social science research, they still are at the core of many scientist's common-sensical understanding of "data quality", particularly given their domain-specific knowledge.

	Initial discussions in the CRC on implementing a "traffic light system" for rating "data accuracy" brought out the complexity of designing a scheme that is abstract and intuitive enough to share the knowlodge ''about'' the data and immediately reveal this ''meta'' data to external users while at the same time being simple to code and broad enough to cover all, or at least "typical" instances that occured or may occur in the "data generation process".		Initial discussions in the CRC on implementing a "traffic light system" for rating "data accuracy" brought out the complexity, or rather difficulty, of designing a scheme that is abstract and intuitive enough to share the knowlodge ''about'' the data and immediately reveal this ''meta'' data to external users while at the same time being simple to code and broad enough to cover all, or at least "typical" instances that occured or may occur in the "data generation process".
	At the same time, there is a sparsity of attempts in the social sciences to rate data, not to speak of a "gold standard" of a framework.		At the same time, there is a sparsity of attempts in the social sciences to rate data, not to speak of a "gold standard" of a framework.

Nils Duepont: /* Technical implementation */

2024-05-30T20:54:15Z

Technical implementation

← Older revision		Revision as of 20:54, 30 May 2024
Line 122:		Line 122:
	* the validation check ignores whether the upload file (still) contains the old columns, or not, and treats them as [[Optional_column_names\|optional column names]]; this way, the researchers are still able to update "their" information for data already existent before the above agreement was reached.		* the validation check ignores whether the upload file (still) contains the old columns, or not, and treats them as [[Optional_column_names\|optional column names]]; this way, the researchers are still able to update "their" information for data already existent before the above agreement was reached.

	For existing data already in the DB, the second rating is not available (and will not be unless the data is updated); it was agreed that WeSIS will display a "Not Available" in the data table.		For existing data already in the DB, the second rating is not available (and will not be unless the data is updated); until the codings will gradually become available with an indicator update, it was agreed that WeSIS will display a "Not Available" in the data table for the time being.

	== References ==		== References ==

Nils Duepont: /* Technical implementation */

2024-05-30T20:52:38Z

Technical implementation

← Older revision		Revision as of 20:52, 30 May 2024
Line 122:		Line 122:
	* the validation check ignores whether the upload file (still) contains the old columns, or not, and treats them as [[Optional_column_names\|optional column names]]; this way, the researchers are still able to update "their" information for data already existent before the above agreement was reached.		* the validation check ignores whether the upload file (still) contains the old columns, or not, and treats them as [[Optional_column_names\|optional column names]]; this way, the researchers are still able to update "their" information for data already existent before the above agreement was reached.

	For existing data already in the DB, the second rating is not available (and will not be unless the data is updated), it was agreed that WeSIS will display a "Not Available" in the data table.		For existing data already in the DB, the second rating is not available (and will not be unless the data is updated); it was agreed that WeSIS will display a "Not Available" in the data table.


	== References ==		== References ==