Research on Design League Badge Portfolios

From Badge Wiki
Jump to: navigation, search

Project overview

Design League Badge Portfolios is an effort to iteratively develop and research an innovative credentialing process that enables high school age youth who come from communities underrepresented in science, technology, engineering and math (STEM) fields to showcase their digital design talents. As part of this project, youth who participate in an afterschool program called Design League learn to use a digital platform to curate representations of their work in ways that are personally meaningful and that also address the expectations of higher education institutions using a badge system that was developed and endorsed by Parsons School of Design. We are conducting research on this initiative to build knowledge about how informal program developers and higher education faculty can create a system that establishes the value of out-of-school learning experiences in college and career pathways. This project is funded by the National Science Foundation’s Innovative Technology Experiences for Students and Teachers (ITEST) program (award #1614727).

Read more about this project at the following links:

What is Design League?

Mouse Design League is a design and technology program in which high school students create inventions to make a positive impact. Design League members develop creativity, problem solving, and collaboration skills, and build confidence in designing technology with purpose.

Over the course of the program, youth use human-centered design to brainstorm, prototype, and present assistive technologies designed to improve the lives of others. Design League enables high school students with diverse interests and backgrounds to gain applied design, technology, workplace, and leadership experience.

This program works with the “Design with Purpose” course on the Mouse Create learning platform. Through Design League, youth earn Digital Badges that document and validate the college-level skills they have developed. The program currently runs at three sites: Mouse’s Manhattan office, at DreamYard Project in the Bronx, NY, and Minneapolis, MN.

Project goals and objectives

The Design League Badge Portfolios project has three goals: (1) to create a digital platform that allows Design League participants to earn badges endorsed by Parsons School of Design and present them in portfolios of their design work; (2) to investigate whether assembling a these portfolios of digital badges affects students’ motivation and persistence for pursuing digital design careers and (3) to standardize the credentialing method so that other institutional partnerships can use it to enable the use of badge portfolios in college admissions, thereby extending its utility and, possibly, establishing its validity.

cover of a zine for recruitment of Design League youth

Research and development design

We studied the research and development process and the impact of that process on youth across the three phases of work:

Phase 1: Program expansion, iterative development, formative testing, and documentation;

Phase 2: Implementation feasibility testing and revision; and

Phase 3: Pilot testing, revision, and dissemination.

We designed the research to answer the following questions.

  1. Does the Design League Badge Portfolio process support participant engagement by increasing their STEM self-efficacy and interest in STEM careers?
  2. What revisions have to be made to the existing systems to enhance participants’ capacity to create Badge Portfolios that are personally meaningful and present their digital design capabilities to higher education admissions officers?
  3. What revisions have to be made to the existing Design League Badge Learning Portfolio systems to ensure the continuity of the process for recognizing and credentialing youth digital design ability across the partner organizations?

This research occurs at two sites in New York City. In Year 1, approximately 30 high school students participated in Design League at one site in Manhattan. In Year 2, approximately 40 students participated in the program across two sites (Manhattan and the Bronx). Finally, in Year 3, 50 students participated at the two sites.

Phase 1: Program expansion, iterative development, formative testing, and documentation

In Phase 1, the primary activities were: (1) to develop and test the technical infrastructure to enable the delivery of digital badges and construction of digital portfolios at multiple project sites and for multiple users, using assets from the existing Design League badge system; (2) to interview new and returning Design League youth to understand whether and how receiving badges and constructing portfolios affects their STEM identify (3) to interview Parsons admissions officers and educators at Mouse and DreamYard to understand their perceptions of the badges and e-portfolios as digital design credentials; and (4) to draft a Design League Portfolio Toolkit to help DreamYard and future satellite organizations and higher education partners institute the badge and portfolio system.

In early fall 2016 Mouse began to design a new, portfolio-related functionality on its online platform, Mouse Create. The Create platform hosts Mouse’s online programming and enables educators and youth to implement activities, communicate with each other and comment on work across programs, and store youth projects.

Mouse’s design and web development team conducted over two dozen meetings and convened partners from Parsons and DreamYard Project. The team gathered feedback on design and functionality from those partners, as well as from current Design League participants via two focus groups responding to stakeholder feedback including information gleaned from student focus groups. The feedback led to new functional specifications and revised plans for a technical infrastructure that builds a more seamless user experience for students in sharing and curating their work and badges via digital portfolios. Several important outcomes (among others) of this work include:

  • An engaging, but simple, user experience to maximize communication between students and their peers, and students and educators across project work;
  • Robust export functionality to help high school students use their work in Design League developed in the software required by schools and in their own portfolios as applicants
  • Clearer mapping between badge and portfolio functionality.
a page from a zine depicting the work of Design League youth

Phase 2: Implementation feasibility testing and revision

In Phase 2, the primary activities were (1) to conduct a nine-month feasibility study of the badge portfolio system at two Design League sites with approximately 40 high school students in grades 9–12; (2) to continue to document collaboration activities among project partners and revise the Design League Badge Portfolio process; and (3) to recruit one additional higher education partner.

During this phase of the project, Mouse and Parsons began to revise programmatic elements of the endorsement agreement between the organizations to ensure that admissions partnership is more clearly built into the infrastructure of the partnership. Partners would be collaborating to have admissions train youth-facing staff in admissions procedures, and structured the university’s “portfolio mentor” role, filled by existing undergraduates, into a clear job description. The team also initiated meetings with Hostos Community College, based in the Bronx, with the intention to endorse Design League badges and to establish the endorsement as a step toward a deeper admissions partnership, with the potential for students to use badged portfolios toward college-level credits.

Phase 3: Pilot testing, revision, and dissemination

In Phase 3, the primary activities were to (1) investigate whether Design League Badge Portfolios show promise of increasing engagement in and capacity for STEM pursuits among underrepresented youth, (2) produce a toolkit and disseminate findings, (3) continue conversations with potential badge system partners. Some significant outcome from Phase 3 included:

  1. The partners authored and published a Design League Badge Portfolio Toolkit. Project partners leveraged existing relationships to identify a public digital host, Badge Wiki, “a knowledge repository for open badges.” The team authored five posts which can be found here: 1)Research on Design League Badge Portfolios 2) Partnership Lessons Overview, 3) Technical Infrastructure (for badge portfolios platform), 4) Foundations for Badge Portfolios Program 5) Endorsement Journey.
  2. Hostos Community College agreed to commit to endorsing Design League badges and to establish the endorsement as a step toward deeper admissions partnership, with the potential for students to use badged portfolios toward college-level credit.
  3. Parsons admissions and faculty expanded their support for the project. Parsons and the partners worked to draft a “portfolio review protocol” for the Parsons admissions office. Parsons also trained two undergraduate students to serve as portfolio mentors and to help Design League youth strengthen their admissions portfolios for Parsons in particular and other schools of design in general.

Analyses of surveys and interviews of Design League youth across the three years show many consistencies, but also some changes as the Badge Portfolio system evolved.

  • Across all years youth find their work in Design League to be personally meaningful and, for some, rewarding in ways that are different from experiences and achievements in school. Youth consistently discussed “collaboration,” “problem solving,” “being creative,” and “helping others” as important parts of their experiences in Design League. “Collaboration” and “acquiring technical skills” were the most frequently referenced reasons for how experiences in Design League might help youth in future pursuits.
  • Across all years Design League youth expressed interest in applying to college and indicated a wide range of possible academic interests, with most indicating that “Engineering,” “Computer Science,” “Design,” or “Architecture” were among the majors they were most likely to choose.
  • In the first year, Design League youth attached relatively little value to the digital badges they acquired. This was due in part to the fact that Mouse did not have a formal system for providing youth with feedback about the work associated with their badges. But most youth indicated that they did not ascribe much value to the badges because they are generally not recognized in higher education admissions (though two youth in Phase 1 shared their badge-related experiences and described their work in Design League in their college admissions applications). However, by Phase 3, some Design League youth describe the badges as an acknowledgment of mastery and a symbol of what they accomplished, while others felt that the work or experience that make up the badges is what’s important. Design League youth also view the badges as a currency that others (i.e., universities, organizations) can recognize and trust.
  • In Phase 1 Design League youth are unclear about what a badge portfolio is. By Phase 3 Design League youth understand what a portfolio is and what its purpose is, but still need additional guidance around curating portfolios that can be submitted for college applications.