Badges and Quality

Definitions:

• Open Badge: a verifiable claim related to a person's achievements, learning, competencies, commitments or affiliations.
• Quality: “degree to which a set of inherent characteristics fulfils requirement.” (source ISO 9000)
• Quality management (QM) includes all the activities related to quality planning, quality control, quality assurance and quality improvement.
• Quality assurance (QA) “part of quality management focused on providing confidence that quality requirements will be fulfilled.” (source ISO 9000)

How do Open Badges and Quality Relate?

Open Badges and Quality can be related as:

• Open Badges for Quality, as a means to achieve quality, e.g. using Open Badges as a vehicle for issuing quality marks or as a source of data for quality management
• Quality for Open Badges, as a means to achieve quality, e.g. design Quality badges

Open Badges for Quality

Open Badges can be used as a means to promote and achieve Quality as:

• Organisational Quality Marks: embed into the ISO 9001 picture the metadata related to the obtention of the ISO certificate: criteria, evidence, issuer, earner, delivery and renewal date, etc.
• Individual Competency Recognition: recognise the competencies of quality professionals, e.g. Quality Manager, Quality Assessor or Quality Innovator badges.
• Support to Quality Processes: use Open Badges to create a portfolio of evidence used to claim a Quality Mark.
• Inform Quality Processes: e.g. analyse the endorsements of the Open Badges delivered to students as indicators for the quality of the learning provision of the issuing institution.

More details in Open Badges for Quality

Quality for Open Badges

The quality for Open Badges is related to different elements:

• The Open Badge as a technical object produced and exploited by other technical objects
• The Open Badge Infrastructure enabling the seamless production and exploitation of Open Badges
• The Open Badge as a medium conveying information produced and exploited by humans and machines
• The Open Badge Ecosystems composed by the humans, organisations and services serving and exploiting the Open Badge Infrastructure

The quality of the Open Badge as a technical object can be defined in relation to the conformance to a standard the Mozilla, now IMS-Global Open Badge Standard [version 1.xx, 2.xx, 3.xx] . The verification of the conformance of an Open Badge against the standard is performed with an Open Badge Validator. One of the things the technical standard guarantees is that once a badge has been issued, it is not possible to tamper with it so we can be sure that information such as the issuer or earner identifiers have not been modified.

The quality of the Open Badge Infrastructure is related to the quality of the implementation of the standard in the different tools used to issue and exploit Open Badges within the Open Badge Ecosystem. While the Open Badge standard describes what is a valid badge, the way it is implemented might differ from one platform to the next, for example one badge issuing platform could decide to use public keys as identifiers for badge and issuers earners (something possible with the 2.0 specification) while other platforms might not be ready to understand this new feature and reject yet valid badges. Another point is the verification of the issuer and earner identities: while the Open Badge specification guarantees that the information in the badge cannot been changed after having been issued, it is possible to spoof Open Badges pretending that they have been issued by HarvardUni.org or SorbonneUniversite.eu. One of the functions of the issuing platforms is precisely to vet fake institutions, diploma mills and crooks.

The quality of the Open Badge as a medium conveying information can be defined in relation to its fitness for purpose and its readability and intelligibility to humans and machines altogether —is it meaningful to its target, does it achieve what was expected? The quality of the Open Badge as a medium is dependent on the quality of the infrastructure (e.g. shared ontologies and frameworks) and the services used to issue and exploit Open Badges, e.g. facilitate the publication of competency badges by connecting the platform to existing frameworks (c.f. http://escobadges.eu/builder/). The use of ontologies, taxonomies, controlled vocabularies and frameworks, could improve the ability of discovering and connecting different sources of information to inform decisions.

The quality of Open Badge Ecosystems is related to the range of agents and contexts where they operate. In particular one can identity two main contexts:

• Informal context: the power structure of the ecosystem is distributed across all the agents —individuals, communities and organisations.
• Formal context: the power structure is controlled by institutions and organisations.

Of course, those two contexts are not disconnected, they are in constant interaction, for example Open Badges delivered in an informal context can be used to inform processes in the formal context like Recognition of Prior Learning (RPL), and conversely e.g. the informal endorsement of a formal credential by the clients of the credential holder.

Quality of Badges in the Formal Context

In the formal context, Open Badges are primarily used as digital certificates, digital diplomas or digital credentials. Open Badges can also be used to inform the processes of recognition of non-formal and informal learning, e.g. Recognition of Prior Learning (RPL) Accreditation of Prior Learning (APL) etc.

The traditional quality management processes used for the delivery of paper certificats, paper diplomas and paper certificates continue to apply:

• Quality Planning
• Quality Control
• Quality Assurance
• Quality Improvement

The main contribution of Open Badges to quality in the formal context is the delivery of verifiable and unfalsifiable credentials: the content of the badge is formally trustworthy — there is still the possibility of a failure in the quality assurance. The second most important contribution, since the 2.0 specification, is the possibility of endorsing badges, thus creating the conditions for keeping current the credentials through the collection of further endorsements.

There are a number of quality standards relevant to organisations delivering Open Badges:

• ISO 10015:1999 applies to training organisations.
• ISO/IEC 17021:2006 applies to bodies providing audit and certification of management systems
• Criteria for being recognised as an Awarding Body in Wales: link
• Training, Assessment and Quality Assurance by City & Guilds: link

And competency standards relevant to learning professionals contributing to the quality of thee credentials delivered by an organisation:

• Assess Workplace Competence Using Direct Methods (92 KB)
• Assess Workplace Competence Using Direct and Indirect Methods (99 KB)
• Internally Monitor and Maintain the Quality of Workplace Assessment (92 KB)
• Externally Monitor and Maintain the Quality of Workplace Assessment (90 KB)

Quality of Badges in the Informal Context

In the informal context, the agent at the origin of the generation of an Open Badge or an endorsement is mainly an individual. Unlike an organisation that can implement formal quality management activities, the quality of an Open Badge has to rest on other means, in particular communities and Open Badges metadata analysis.

The following table is not prescriptive but the indication of possible means to check the quality of an Open Badge delivered in the informal context.

Possible Quality Indicators for badges delivered informally

Metadata	Action	Questions	Service / application
Issuer	Verify credentials & (pseudo)identity	What Badges and Endorsements have they collected? What Badges and Endorsements have they issued? Are they relevant to the criteria of the Badge/endorsement analysed? Who does trust the issuer? Who does the issuer trust?	Map agents connected to issuer Identify trust network Identify potential anomalies: circular-references and singularities, etc. Notify other agents ...
Earner	Verify credentials & (pseudo)identity	What Badges and Endorsements have they collected? What Badges and Endorsements have they issued? Are they relevant to the criteria of the Badge/endorsement analysed? Who does trust the earner? Who does the earner trust?	Map agents connected to earner Identify trust network Identify potential anomalies: circular-references and singularities, etc. Notify other agents ...
Criteria	Map criteria to ontologies, taxonomies, frameworks	What is the meaning of the criteria? Who are the leaders / references in that domain / area / community	Map other agents using same or similar criteria (URI, taxonomy, etc.) Identify trust network Identify potential anomalies: circular-references and singularities, etc. Notify other agents ...
Evidence	Verify authenticity	Is the evidence authentic, relevant, current, sufficient?
Endorsements	Verify credentials & (pseudo)identity	What Badges and Endorsements have they collected? What Badges and Endorsements have they issued? Are they relevant to the criteria of the Badge/endorsement analysed? Who does trust the earner? Who does the earner trust?	Map agents connected to endorsements Identify trust network Identify potential anomalies: circular-references and singularities, etc. Notify other agents ...

The traditional quality management processes cannot apply to the informal context. While in the formal context, the quality control can be done prior the delivery of a badge, in the informal context, this can only be done post delivery of badges and endorsements —including issuing/withdrawing more badges and endorsements.

The main quality of Open Badges and endorsements in an informal context is their trustworthiness.

• Quality Planning responds to the question: how can we elicit trustworthiness of Open badges? It is done through the provision of tools and services allowing the monitoring and representation of the trust structures and networks within and across communities.
• Quality Control responds to the question: how can we monitor the variations in trustworthiness of Open Badges? The community regulates the reputation of its network by issuing and withdrawing endorsements. Network analysis applications provide dashboards tailored to the communities. Communities as a whole also have a reputation index which is connected to its ability to enforce its policy.
• Quality Assurance responds to the question: how can we guarantee that the trustworthiness of an Open Badge is conform to one's expectations? The assurance that a claim contained in an Open Badge or an endorsement is valid can be provided by an analysis of the trust network and its history.
• Quality Improvement responds to the question: how can one identify the areas for possible improvements? The source for quality improvement is the dashboard informing individuals and their community on their level of trustworthiness to other communities and their members. This also include feedback loops with tools and application providers — they can include those feedback loops within their tools and services.