Gap analysis

This is the page for the gap analysis.

Tacit knowledge "FAIRification and opening of low carbon energy research data"

Consortium members can add any time, they feel that something is important to note even though it is not mentioned in a standard deliverable. In other words, this page collects uncodified, tacit knowledge. It will help us later to compile suggestions and lessons learned. This kind of knowledge is collected two-fold:

1. Anytime if someone feels that this should be noted. Please write down: Issue, Date, Author (could also be "anonymous"), the issue described in a few words or maximal lines>
2. During the final day of workshops

Learning process:

• People are hesitant to adopt new IT technologies, this is even the case among researchers heavily relying on data, algorithms and collaborative online software (e.g., R, platforms, online conferencing, HPC, ...). The effort to encourage change is not to underestimate. Reasons are several, notably, lack of time and uncertainty about potential benefit as well as overall risk aversion preferences. EERAdata is using the online software "Only Office" to facilitate collaboration (in particular also during the Covid-19 period).

FAIR and open criteria:

• Consortium members have a fair understanding of what FAIR/O is, but there is little knowledge and/or technical experience on how to approach the FAIRification and opening. All, however, share the view that we are at a critical point in time, where we need to implement these criteria. 
• To deepen knowledge about FAIR/O criteria, it is useful to test the criteria on a database one is familiar with. For this purpose (and to start brainstorming about the platform), AIT has developed a questionnaire for application in the use cases.

Metadata:

• A good starting point is to think about metadata and to look into existing metadata concepts in one's field. The first step is to understand that also metadata need to adhere to the FAIR/O principles.
• The next step is to increase knowledge on IT specific terms, i.e. to understand what the difference is between different metadata frameworks (taxonomy, thesaurus, ontology) as well as classification of metadata (high-level, medium-level, low-level OR administrative, structural and descriptive metadata).  WP 2 being in charge of aligning approaches between use cases, participates in all use case kickoffs to bring everybody on the same page. The presentation is linked with "metadata frameworks".
• It is useful to supply consortium members with read aheads and watch aheads on metadata to prepare the first EERAdata workshop. The workshop starts applications and discussions in the use cases break out sessions (and bringing insights back to the plenary), using selected databases.

FAIR Analysis

FAIR Guiding Principles

This table shows the FAIR guiding principles described by Wilkinson et al., 2016.

To be Findable	F1. (meta)data are assigned a globally unique and persistent identifier F2. data are described with rich metadata (defined by R1 below) F3. metadata clearly and explicitly include the identifier of the data it describes F4. (meta)data are registered or indexed in a searchable resource
To be Accessible	A1. (meta)data are retrievable by their identifier using a standardized communications protocol A1.1 the protocol is open, free, and universally implementable A1.2 the protocol allows for an authentication and authorization procedure, where necessary A2. metadata are accessible, even when the data are no longer available
To be Interoperable	I1. (meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation. I2. (meta)data use vocabularies that follow FAIR principles I3. (meta)data include qualified references to other (meta)data
To be Reusable	R1. meta(data) are richly described with a plurality of accurate and relevant attributes R1.1. (meta)data are released with a clear and accessible data usage license R1.2. (meta)data are associated with detailed provenance R1.3. (meta)data meet domain-relevant community standards

Identified gaps after Workshop 1 (02/06/2020 – 04/06/2020)

This table shows a summary of the specific issues that were identified in Workshop 1. The aim was to categorise different issues to get a better understanding of how to tackle these challenges.

FAIR/O	Specific FAIR/O	Gaps	Gaps for energy domain	Use case specific gaps	Consequences for researchers
F	F2	Metadata scope	Different fields require different metadata (potentially very specific)	UC3: lack of additional metadata for applications of materials	Data are less useful for the specific field
F	F1/F3	Identifiers	Missing identifiers on websites Identifier not in downloaded data files		Makes research more difficult.
I	I2	Taxonomy/ontology/common vocabulary and language issues	Lack of standardisation Heterogeneous data makes standardisation hard No vocabulary documentation on websites Words used for same term in other languages may differ Databases in other languages than English		Research costs more time
I	I3	References to other (meta)data	No linking to source documents and related publications (for contextual knowledge) Possible need for links to other fields	UC3: link between microscopic and macroscopic materials (e.g., turbine blades)	Data cannot be connected to the source Makes research more difficult.
R	R1.1	Licensing	No licenses available may mean the data is not reusable for the researcher Licenses not clear and accessible Obtaining licenses may result in more effort and costs		Data potentially not reusable
R	R1.2	Reliability and provenance of data	When the data is collected from different and high number of sources, its reliability decreases		Data potentially not reusable
O		Privacy concerns and expected disadvantages	Not publishing data due to privacy concerns (sensitive data)	UC2: In case of distribution network data this is relevant	Data not accessible
Other		Conducting FAIR assessments	No qualitative assessment of the data itself (which may be limited) Discrepancies between results conducted by humans versus machines Sometimes not even DC standards are met Metadata is not updated Lacking encryption of websites (https) Problems assessing whether metadata will be available after data is unavailable Interface design/layout may be unclear, incomplete or not intuitive for humans Authentication details (user registration login / good or bad, clarify)	UC1 lack of data availability for time-series	Assessment is uncertain (human assessment may need more clarification and understanding) Makes research more time-intensive

FAIR Self-Assessment tool

To conduct a human FAIR evaluation for different databases and their respective datasets, different free online tools were considered. Some seemed to be quite useful (e.g. SATISFYD ) and some were simple self-evaluation PDFs to be printed and filled out.

We decided to chose the self-assessment tool provided by the Australian Research Data Commons (ARDC). The decision was based manly upon the fact that the tools questions closely matched the FAIR categories specified by Wilkinson et al., 2016 which would make it easily comparable to the existing preferable FAIR conditions and aid with the repeatability of the results, and the half-automated scoring features that show the extent of the "FAIRness" of the analysed sample as a simple bar chart.

This table shows all options for each question of the ARDCs self-assessment tool.

Findable				Accessible			Interoperable			Reusable
F1. (meta)data are assigned a globally unique and persistent identifier	F2. data are described with rich metadata (defined by R1 below)	F3. metadata clearly and explicitly include the identifier of the data it describes	F4. (meta)data are registered or indexed in a searchable resource	A1. (meta)data are retrievable by their identifier using a standardized communications protocol	A1.1 the protocol is open, free, and universally implementable and A1.2 the protocol allows for an authentication and authorization procedure, where necessary	A2. metadata are accessible, even when the data are no longer available	I1. (meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation.	I2. (meta)data use vocabularies that follow FAIR principles	I3. (meta)data include qualified references to other (meta)data	R1.1. (meta)data are released with a clear and accessible data usage license	R1.2. (meta)data are associated with detailed provenance	R1.3. (meta)data meet domain-relevant community standards
Does the dataset have any identifiers assigned?	Is the dataset identifier included in all metadata records/files describing the data?	How is the data described with metadata?	What type of repository or registry is the metadata record in?	How accessible is the data?	Is the data available online without requiring specialised protocols or tools once access has been approved?	Will the metadata record be available even if the data is no longer available?	What (file) format(s) is the data available in?	What best describes the types of vocabularies/ontologies/tagging schemas used to define the data elements?	How is the metadata linked to other data and metadata (to enhance context and clearly indicate relationships)?	Which of the following best describes the license/usage rights attached to the data?	How much provenance information has been captured to facilitate data reuse?
Globally unique, citable and persistant (e.g. DOI, PURL, ARK or Handle)	Yes	Comprehensively (see suggestion) using a recognised, formal machine-readable metadata schema	Data is in one place but discoverable through several registries	Publicly accessible	Standard web service API (e.g. OGC)	Yes	In a structured, open standard, machine-readable format	Standardised open and universal using resolvable global identifiers linking to explainations	Meadata is represented in a machine readable format, e.g. in a linked data format such as Resource Description Framework (RDF)	Standard machine-readable license (e.g. Creative Commons)	Fully recorded in a machine-readable format
Web adress (URL)	No	Comprehensively, but in a text-based, non-standard format	Generalist public repository	Fully accessible to person who meet explicitly stated conditions, e.g. ethics approval for sensitive data	Non-standard web service (e.g. OpenAPI/Swagger/informal API)	No	In a structured, open standard, non-machine-readable format	Standardised vocabularies/ontologies/schema without global identifiers	The metadata record includes URI links to related metadata, data and definitions	Standard text based license	Fully recorded in a text format
Local identifier		Brief title and description	Domain-specific repository	A de-identified / modified subset of the data is publicly accessible	File download from online location	Unsure	Mostly in a proprietary format	No standards have been applied in the description of data elements	There are no links to other metadata	Non-standard machine-readably license (clearly indicating under what conditions the data may be reused)	Partially recorded
No identifier		The data is not described	Local institutional repository	Embaged access after a specific date	By individual arrangement			Data elements not described		Non-standard text-based license	No provenance information is recorded
			The data is not described in any repository	Unspecified conditional access e.g. contact the data custodian for access	No access to data					No license
				Access to metadata only
				No access to data or metadata

Dublin Core

The DCMI Metadata Terms lists the current set of the Dublin Core vocabulary. Source: Dublin Core Metadata Initiative

1	abstract	11	contributor	21	extent	31	isReplacedBy	41	publisher	51	tableOfContents
2	accessRights	12	coverage	22	format	32	isRequiredBy	42	references	52	temporal
3	accrualMethod	13	created	23	hasFormat	33	issued	43	relation	53	title
4	accrualPeriodicity	14	creator	24	hasPart	34	isVersionOf	44	replaces	54	type
5	accrualPolicy	15	date	25	hasVersion	35	language	45	requires	55	valid
6	alternative	16	dateAccepted	26	identifier	36	license	46	rights
7	audience	17	dateCopyrighted	27	instructionalMethod	37	mediator	47	rightsHolder
8	available	18	dateSubmitted	28	isFormatOf	38	medium	48	source
9	bibliographicCitation	19	description	29	isPartOf	39	modified	49	spatial
10	conformsTo	20	educationLevel	30	isReferencedBy	40	provenance	50	subject

Metadata Category Types according to NISO

There are three main types of metadata:

• Descriptive metadata describes a resource for purposes such as discovery and identification. It can include elements such as title, abstract, author, and keywords.
• Structural metadata indicates how compound objects are put together, for example, how pages are ordered to form chapters.
• Administrative metadata provides information to help manage a resource, such as when and how it was created, file type and other technical information, and who can access it. There are several subsets of administrative data; two that sometimes are listed as separate metadata types are:
  • Rights management meta-data, which deals with intellectual property rights,and
  • Preservation metadata, which contains information needed to archive and preserve a resource.

Simple Dublin Core Elements. 15 DC elements with their (shortened) official definitions and examples.

DC element name	DC definition (DCMI)	Example (Energy Domain)	Category (determined with the aid of NISO)
Title	A name given to the resource	Outdoor monitoring of a hybrid micro-CPV solar panel with integrated micro-tracking and diffuse capture	Descriptive
Subject	The topic of the resource	concentrator photovoltaics; CPV; rooftop photovoltaics; integrated tracking	Descriptive
Description	An account of the resource	Dataset from the outdoor characterization of a B Series module from Insolight at the rooftop of the Instituto de Energía Solar - Universidad Politécnica de Madrid. ….	Descriptive
Creator	An entity primarily responsible for making the resource	Stephen Askins; Gaël Nardin	Descriptive
Publisher	An entity responsible for making the resource available	Stephen Askins	Descriptive
Contributor	An entity responsible for making contributions to the resource	Stephen Askins	Descriptive
Date	A point or period of time associated with an event in the lifecycle of the resource	2019-07-23; Date will be associated with the creation or availability of the resource.	Administrative
Type	The nature or genre of the resource	info:eu-repo/semantics/other; dataset	Descriptive
Format	The file format, physical medium, or dimensions of the resource	image	Structural
Identifier	An unambiguous reference to the resource within a given context	https://zenodo.org/record/3346823; 10.5281/zenodo.3346823; oai:zenodo.org:3346823	Descriptive
Source	A related resource from which the described resource is derived	RC607.A26W574 1996 [where "RC607.A26W574 1996" is the call number of the print version of the resource, from which the present version was scanned]	Descriptive
Language	A language of the resource	eng	Descriptive
Relation	A related resource	info:eu-repo/grantAgreement/EC/H2020/787289/	Descriptive
Coverage	The spatial or temporal topic of the resource, the spatial applicability of the resource, or the jurisdiction under which the resource is relevant	1995-1996; Madrid, Spain	Descriptive
Rights	Information about rights held in and over the resource	info:eu-repo/semantics/openAccess; https://creativecommons.org/licenses/by/4.0/legalcode	Administrative

Non-DC elements

Non-DC element name	Definition	Example	Category
Date of availability	Since when is the data available		Impact/Exploitation
Number of users	Is there any information on the number of accesses or users?		Impact/Exploitation
Number of scientific publications where data is cited/used	Is there any information on the number of citations/uses?		Impact/Exploitation
Curation activities (is the versioning ongoing)	Update, maintainance		Impact/Exploitation

Attempting to put a value on Impact/Exploitation:

• ENTSO-E: Found 172 results in Advanced Search (ALL) in Web of Science (WoSc) (16/06/2020). Citations of all these search result add up to 714. This however does not mean, that all search results really explicitly used data from ENTSO-E. Some (for sure, for some were found) only mentioned it saying they adoped workflow etc. from the site.
• SMARD: Found 82 results in Advanced Search (ALL=(SMARD)) in Web of Science (WoSc) (16/06/2020). However: SMARD is also an acronym for Spinal muscular atrophy with respiratory distress (SMARD). This somewhat complicates results and needs some further discussion. All 82 search results accumumlated 2359 citations themselves.