The University of Zurich (UZH) had the honour to host two conferences this summer: CI 2018, part of the ACM Collective Intelligence Conferences Series, and HCOMP, the 6th Annual AAAI Conference on Human Computation and Crowdsourcing. The conferences were co-located at UZH, with CI happening 7-8 July, and HCOMP taking place on 5-8 July.
Both conferences discussed challenges and opportunities relevant to collective intelligence, crowdsourcing and human computation, including crowd data quality control, effective and efficient design of human-machine hybrid systems, trust management and crowd participation.
CI was sponsored by SIGCHI and Google, the Citizen Science Centre Zurich and the European project FashionBrain.
HCOMP was sponsored by NSF (the Swiss National Science Foundation), Elsevier’s Artificial Intelligence journal, Bloomberg, Microsoft, Harvard University, microWorkers, and the University of Zurich.
Prof. Abraham Bernstein, program chair of CI2018, opened the conference in the same lecture hall where Churchill told the academic youth: “Therefore I say to you, let Europe arise!”. Given that this speech set ground for a democratic and united Europe, it was a truly inspiring location to start the conference that encourages collective participation and crowdsourced knowledge management.
Prof. Bernstein opens CI2018
CI attracted 75 participants stemming from 16 different countries, HCOMP had about 95 participants from 17 different countries.
Keynotes
Patrick Meier (WeRobotics) introduced his work at WeRobotics in the digitalization of humanitarian activities, including the Flying Labs: a global network of local knowledge hubs where locals are empowered by sharing equipment for and knowledge about drones. He showcased the possibilities of technology in humanitarian aid, and invited the attendees to collaborate to further harvest the possibilities.
Patrick Meier from WeRobotics
Lucy Fortson (University of Minnesota) presented the latest research around the Zooniverse project, one of the most successful citizen science projects, that started in 2007 with 1 million galaxies and has collected around 40 million classifications by approximately 150K users. Lucy explained the importance of combining human and machine intelligence optimally, and mindfully using human attention. Moreover, Lucy encouraged the audience to start their own projects in Zoouniverse, as it provides a DIY platform that can host image and text annotation and transcription tasks.
Lucy Fortson presenting Zooniverse
Scott Klemmer (University of California San Diego) gave a broad overview on his work about learning through collective intelligence through MOOCs, highlighting the importance of space design. Scott emphasized how important peer and self-assessment is in open-ended work.
Scott Klemmer on design and collective intelligence
Eszter Hargittai (University of Zurich) presented her findings on digital inequality and reminded the audience that we should acknowledge that different parts of the population use different systems and even in the same systems, different cohorts behave differently. Hargittai et al. defined a framework to study online participation in systems like Wikipedia, and observed that Internet skills and having the necessary know-how is crucial to active participation in systems driven by collective action.
Eszter Hargittai on the digital divide and digital skills
HCOMP
Panel
The HCOMP panel focused on the need of ethics management in crowdsourcing, to ensure that all sides (including crowd workers and requesters) receive a fair treatment and fulfill their expectations. There was a global agreement on the fact that, despite valuable initiatives like the Guidelines for Academic Requesters, there is much work to do to improve the current crowdsourcing landscape.
The CI panel presented different points of view, and Prof. Thomas Malone (MIT Sloan) urged the audience to work on the three major questions that remain unanswered: (i) How can crowds help solve really complex problems? (ii) How can we attract the right crowd? (iii) How can we best combine crowds and experts?
Workshops
HCOMP hosted a couple of workshops, including CrowdBias (https://www.ifi.uzh.ch/crowdbias) organised by Alessandro Checco (University of Sheffield), Gianluca Demartini (University of Queensland), Ujwal Gadiraju (Leibniz University of Hannover) and Cristina Sarasua (University of Zurich).
Kristy Milland (community manager of TurkerNation) and Jahna Otterbacher (Open University of Cyprus), keynote speakers of the workshop, provided useful insights on the way workers interpret and optimize crowd work, and the impact that humans’ biases have in information systems. Some of the key topics brainstormed and discussed included the need for a taxonomy of biases in crowdsourcing, methods to present bias information to end users and mechanisms to profile crowd workers and their biases over time.
Workshop on CrowdBias
Best Paper Award
HCOMP gave the best paper award to Alessandro Checco, Jo Bates and Gianluca Demartini for their work All That Glitters is Gold — An Attack Scheme on Gold Questions in Crowdsourcing, that presents a method to identify gold questions in microtasks automatically and encourages the human computation and crowdsourcing community to rethink the mistrust paradigm that crowdsourcing platforms’ follow. Besmira Nushi, Ece Kamar and Eric Horvitz got an honorable mention for their work Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure, that defines a method to profile when and how a system fails via crowd data, and helps improve AI accountability. The best paper award was given to Vikram Mohanty, David Thames, Kurt Luther, on their demo around face recognition to identify civil war soldiers.
Next Year
In 2019 both conferences will go back to the US. CI 2019 will take place in Pittsburgh, PA and will be chaired by Anita Woolley (Carnegie Mellon University). HCOMP 2019 will be in Stevenson, Washington, USA.
We hope you enjoyed as much as we did!
The University of Zurich team.