A Member-Driven Network: Research, Data & Feedback for 2015

Hive Chicago is a Member-driven Network.

As part of Mozilla’s Hive Learning Networks global initiative, Hive Chicago shares it’s mission, vision and theory of change with all Hive Networks: mobilize educators in our communities to create connected learning experiences (that teach web literacy) and catalyze others to extend that work.

Yet, as a network of local organizations in Chicago, the unique goals, strategies and calls-to-action that the Hive Chicago adopts to advance that mission are directly informed by the collective aspirations, needs and challenges articulated by our individual, organizational, and community members.

In Chicago, members of the Hive collaboratively shaped a set of four goals in 2013: equitable access, learning pathways, innovative program design, and external value. To achieve these aspirational goals, Chicago membership further articulated six Moonshots in 2014. These are calls-to-action or issue areas that organize Moonshot working groups to generate solutions – onramps to learning, transportation, school engagement, parent engagement, youth voice, and data informed decision making.

The experience of exploring, creating and sharing collaboratively in the Network – either through meetups, online forums, blogs, funding opportunities, youth learning events, or other gatherings and venues for engagement – is also shaped to meet the needs of the network. Hive Chicago collects data on member activity, solicits experience feedback, and engages research partners to assess the effectiveness and to inform the design of our engagement platform and professional learning community.


The NYU Hive Research Report

The Hive Learning Networks partner with New York University, Connecting Youth: Digital Learning Research Project to conduct member-professional-educator and youth-participant experience research in Hive-funded, and un-funded activities. For three years, the NYU team has been conducting interviews, surveys, and observations to produce yearly reports that help paint an unbiased picture of the Hive experience.

In the March 2015 Meetup, Hive Chicago’” staff presented eight “Key Findings” from the NYU Academic Year 2013-14 Hive Report to attendees of the Meetup. Since the report covered a period of time several months in the past, Hive staff prompted them to reflect and provide feedback on whether the key findings had been addressed. The following is the exact text of the prompt:

Reflecting on the NYU report left Hive Leadership feeling encouraged that many of the issues raised by Hive membership in 2013 and 2014 have begun to be addressed leading into 2015.

Yet, the best judge of our work are the members of the Network and we wanted to get your feedback to see where we have been effective, and where we might continue to improve.

Please provide us with feedback on these NYU Key Findings. For each of the NYU Key Findings above, please consider the Key Finding and reflect on how it connects to your experience in the Hive, then leave a colored sticker dot to indicate:

  • GREEN – Being Addressed: this is an issue that has been adequately addressed, is currently being addressed adequately, or is not relevant to the current state of the Hive;
  • YELLOW – No Idea: this is an issue that you have no experience with or don’t know whether the Hive is addressing;
  • PURPLE – Critical Attention: this is an issue that has not seen any significant action from the Network and needs immediate attention;

The Eight NYU Key Findings:

Text and headings taken verbatim from NYU report Executive Summary.

  1. Defining Hive & Hive-like – Some [representatives] pointed out that their work had already been aligned with Hive’s goals, and that these learning models simply provided language to describe their existing practices.
  2. Educational Innovation – Representatives believed that educators from the school sector were missing from the network, and believed that Hive leadership should work to better bridge the gap between the informal and formal learning spaces.
  3. Network Growth – [Representatives] feared that a network that became too large could potentially dilute each organization’s ability to build meaningful relationships and collaborations.
  4. Beneficial Resources – The two most valued benefits of membership were access to funding and organizational peers.
  5. Desired Resources – Representatives highlighted the need for a resource hub where they could easily find information on all Chicago Hive member organizations, as well as their programs and projects.
  6. Desired Resources – [Representatives] requested toolkits with best practices, stronger connections between the various Hive Learning Networks and Hive Learning Communities, and a more organized structure for monthly meetups and Minigroups.
  7. Institutional Support – None of the interview participants reported facing institutional challenges regarding their Hive membership. However, the level of active organizational support they experienced still varied.
  8. Spread – There were mixed responses on whether concepts like connected learning and HOMAGO had spread beyond the Hive network. While some representatives believed that these concepts were spreading in their home institutions, others mentioned that their work had been aligned with these models before they were formally identified and promoted.

The results of the member feedback on the NYU Report are shown in the slideshow gallery below.

This slideshow requires JavaScript.

Members almost unanimously felt that – in the time since the NYU research was conducted – Hive has addressed issues around defining Hive, growing the network, and providing beneficial resources for members. Their perspective on their own institutional support for the Hive continued to vary and clear evidence for the spread of Hive-like ideas remains somewhat unclear. Members feel united that there is still room for improvement with engaging formal educators in our innovative work and making the products of our work easy to find and access.


Real-time Feedback Channels

Meetup Feedback

In addition to the NYU Report, Hive Chicago continuously collects network feedback directly through surveys at network engagements and indirectly through tracking participation activity, event attendance, and organizational profiles. As a complement to the NYU key findings, Hive staff also shared samples of this data at the March 2015 Meetup as well. These are presented below.

An Example of Meetup Feedback collected in 2014. The Likert scale used ranges from 1 – Strongly Disagree to 4 – Strongly Agree. #5 – "I think that Hive Chicago is on the right track." #11 –  "Today’s meetup provided me with an adequate opportunity to connect one-on-one with Hive Leadership" #19 –  "Today’s meetup provided me with an opportunity to connect with someone new in the Hive, or reconnect with someone I otherwise see infrequently."
An Example of Meetup Feedback collected in 2014. The Likert scale used ranges from 1 – Strongly Disagree to 4 – Strongly Agree.
#5 – “I think that Hive Chicago is on the right track.”
#11 –  “Today’s meetup provided me with an adequate opportunity to connect one-on-one with Hive Leadership”
#19 –  “Today’s meetup provided me with an opportunity to connect with someone new in the Hive, or reconnect with someone I otherwise see infrequently.”

From the sample data in the table above – the full data-set includes over two dozen questions tracked intermittently in 2014 – we can draw a few conclusions. First, overall, our meetup attendees have found the meetup experience to be improving over the last year. This is implied by the up-and-to-the-right trend in the lines above, but clearer and more compelling in the full dataset. Second, the meetups currently and consistently do a better job at connecting our attendees to each other than they do at connecting attendees to Hive staff, though both have improved over time.

Finally, the sense that “Hive is on the right track” is significantly variable from meetup to meetup. This information is regularly used by Hive staff to gauge how attendees have received new direction or messaging offered by network leadership at meetups. It helps indicate when leadership plans are connecting to member expectations.

Meetup Attendance

In the anonymized meetup attendance spreadsheet below – click here to view the spreadsheet in it’s own window – each row tracks a single attendee and each column represents a meetup; cells are marked orange when an individual was in attendance, while those marked blue indicate they were not.

The meetup attendance data is very illuminating. The first thing to note is the “long tail” of attendees with intermittent or singular attendance. The sheet is sorted with highest attendance at the top, to lowest attendance at the bottom. You can see the long tail by scrolling downwards and noting the transition from predominantly orange to blue.

Secondly, when individual attendance is compared against member organization attendance the following conclusions emerge (note that member organization attendance is not shown here, but is calculated by considering when any staff person from an organization is in attendance at a given meetup):

There is a consistent representation of member organizations:

    • Our 64 Member Organizations attend an average of 50% of meetups (6) per year on average;

There is a variable representation of individuals:

    • 374 unique individuals have attended Hive Chicago meetups in the last 18 months;
    • 50% of those attendees came once and never returned (modulo our next meetup);
    • 30 of those attendees attend 50% of meetups or more (6 or more) per year on average.

What is amazing about this information is that despite the fact that very few people attend every single meetup, there is a pervasive sense of connectedness in our community that is driven by many other interaction opportunities outside of meetups: online in our member-forum, through project collaborations, and at other Hive events and programs. The key takeaway is that:

A thriving network can operate through distributed and loosely connected relationships while remaining tightly in synch.

This data also helps us to think strategically about how we design the meetup experience in 2015: if 50% of our attendees only come once, then we need to be much better prepared to make that single interaction a meaningful one. This is true whether or not there is potential for them to return! Ensuring that people who make the time to interact with us face to face just once have a clear sense of who we are is critical if we only have that one shot. First impressions are lasting.

This also provides us with an interesting opportunity to sample folks in different attendance groups to find out why they do or do not return and where else they may be connecting with the Hive in a way that suits them better. These surveys will be conducted in the next couple of months and their results will be invaluable.


Advancing the Network in 2015

Prior to the March meetup, Hive staff prepared a draft engagement plan and calendar for 2015, based on perceived network needs that were apparent to Hive staff from their experience in 2014. Before these plans were finalized, it was important to assess them against actual member expectations. This was another opportunity for feedback. After reviewing the NYU Key Findings and the meetup feedback and attendance data, Hive staff presented meetup attendees with their plan and the calendar shown in the spreadsheet below.

After reviewing the plan and calendar, meetup attendees were prompted to provide one final round of feedback:

After considering the NYU Key Findings from their 2013-14 Academic Year report, seeing the feedback and meetup attendance data collected by Hive Staff during that same year and hearing about the 2015 calendar for convenings that includes Meetups, Community Calls, and “Hive Dives”, what are your reflections on what should stay, what should go, and what should change for the Hive Experience in 2015?

Attendees completed a survey to identify one item of the Hive engagement strategy they would keep, one they would change, one they would add, and one they would throw away. Hive staff received over 30 individual survey responses and while no specific items dominated the keep, change, add and trash categories, the following general trends emerged after review:

  • Variety – people appreciate a variation in their experience, e.g. food, topics, formats, after hours, target audiences, even furniture format;
  • Skill Building – people want to continue having opportunities to build their skills, e.g. communication-tool-tutorials, skill/fail shares, RFP support, etc.;
  • Inclusivity – there’s a widespread desire to continue broadening inclusion of various forms of diversity, e.g. geographic, racial, formal/informal, etc.;
  • Mechanisms for Connecting – people need more information about digital tools already in place for connecting the community and training to use them;
  • Connecting to Resources – people are looking for a more user-friendly format that allows individuals to find and digest our network resources and learning products;
  • Connecting to Peers – peer-to-peer connections and exchanges are Hive’s biggest asset and value-add for the community;
  • Less Yack & More Hack – people feel like they understand what Hive is all about and now they want to spend more time working on Moonshots;

These are the Hive Chicago design criteria for 2015 community engagement.

Leave a Reply

Your email address will not be published. Required fields are marked *