Concluding remarks

Read our reflection and discussion of the experiments with Nao as a museum guide in the journal article “Technological Fantasies of Nao – Remarks about Alterity Relations” Transformations, issue 29 (2017). The article is a way for us to sum up some of the findings from this collaborative project. We will discuss other parts of the experiments and findings made during the project in our PhD dissertations that will be published in 2018 and 2019.

Thank you for participating in this project offline as well as online.

NAO at Medical Museion Experiment

On February 11th 2016 we conducted an experiment with NAO at the Medical Museion with students from University of Copenhagen and the IT University of Copenhagen who agreed to participate in the experiment.

 

We asked them to reflect upon the event afterwards. Some of these reflections will be analyzed and discussed here on the blog in the next couple of weeks.

 

New script

We have developed a new script for NAO and tested it at the Medical Museion.  In the new script NAO gives the museum visitor choices on what objects it should talk about. This is a video documentation of Variation I out of II.

 

Here is a video documentation of NAO at Medical Museion Variation II.

 

in-the-wild

Below are some reflections on the methodological aspects of the project. We are still in process of designing, performing and evaluating  the robot guided museum tours, so it is far from a retrospective reflection concerning a finished project.

Discussing the exploratory studies made in 2015

The robot guided museum tours in fall 2015  has been  developed through an iterative process of prototyping, testing, evaluating and adjusting the script – and as such based in an exploratory design practice.

In collaboration with the museum guide, Thorbjørn Bornhøft from Medical Museion, we made the script for the museum tours to be guided by the NAO robot. The process of developing the script posed concrete questions such as: How exactly should the robot interact with the museum visitors? Should the robot stand up, sit down, walk or point at specific museum objects? Should it use a lot of technical terms or explain the prosthetic objects with anecdotes and historical references? What kind of knowledge production and dissemination should be in play at the robot guided museum tour?

The robot was programmed to perform the script and interact with museum visitors moving around in front of the exhibited prosthetic objects, as it explained the exhibited artifacts to visitors. NAO was also programmed to ask questions to the visitors present in the room and to react to their answers. It was necessary to constantly adjust the behavior of the robot in-the-wild, because its reactions to the museum visitors were often unpredictable. Building on its emergent behaviors meant implementing design changes during the deployment, usually after its interactions.

“Designing in the wild differs from previous ethnographic approaches to interaction design by focusing on creating and evaluating new technologies in situ, rather than observing existing practices and then suggesting general design implications or system requirements.” (Rogers 2011:58)

The process from prototyping to evaluating the robot tour was mainly taking place in the wild; in the real-world environment that the museum space embodies. This is important because the context turned out to matter a great deal in making a meaningful interaction design; the exhibited objects, the social roles of the museum guide and the museum visitor that the museum space connoted, the norms for how to behave in the exhibition spaces as well as the actual acoustics and size of the room – it all shaped the human-robot-interaction.

The museum space provides this context wherein the interaction between the robot, museum visitors and the exhibited objects emerges. It is an intersection where social, organizational, cultural and physical factors converge to effect interactions in-the-wild. The museum space thus plays, as a contextual framework, a critical role in shaping the embodied interaction between museum visitors and the robot in that it effects how the museum visitors makes meaning from their interaction with the robot and exhibited artifacts. The museum space is what provides the visitors with a meaningful framework wherein they can reflect upon and interpret the interaction with the robot in-the-wild.

The  robot guided museum tours at the Medical Museion have both been video recorded and semi-structured interviews have been conducted with the participating museum visitors about their experience of the NAO robot as a museum guide. The video documentation of the events enabled us to analyze the museum visitors’ reactions and interactions with NAO. The semi-structured interviews allowed us to connect the visitors’ reflections upon the experience of their interactions with the robot to their concrete interaction with the robot at the robot guided tour. These methods permitted us to be more concerned with what is experienced through actions, as Paul Dourish argues in his book , Where the Action Is,   from 2001.

In our examination of the video documentation and semi-structured interviews with the museum visitors, we noticed a discrepancy between what the museum visitors said when interviewed and what they actually did: in the interviews all the visitors highlighted an interaction where the robot asks them to touch its arm as something highly engaging. However, in the video documentation of the tour only one visitor did in fact actively interact with the robot physically as seen on the picture below.

In our examination of the video documentation from the second designed robot museum tour we noticed, among other things, that not all the museum visitors felt compelled to stay till the end of the interaction with the robot. Indeed, during one of the situations where the robot delivers a somewhat lengthy explanation of an exhibited artifact, quite a few of the museum visitors moved on without staying until the end of the explanation.

The fact that the exploratory study was conducted in-the-wild made it possible to invite museum visitors to contribute to the design of the robot tour indirectly (through an examination of the video documentation) as well as directly (through the interviews). What the participating museum visitors said during the semi-structures interviews, will be discussed in the next post reflecting over the methodological aspects of our project.

References 

Paul Dourish. 2001.Where the Action Is: The Foundations of Embodied Interaction. MIT Press, Cambridge, MA.

Yvonne Rogers. 2011. Interaction design gone wild: striving for wild theory. ACM Interactions 18, 4: 58-62.

Co-development of Museum Tour

This blog post  is about our process and development of the robot intervention at the Medical Museion from the two pilot studies in 2015 to the actual experiment conducted in February 2016.

On September 20th 2015 the first pilot study took place at Medical Museion. Approximately 15 museum guests participated in the robot guided museum tour, an event which took 15-20 minutes.  Three people agreed to participate in 30 min long semi-structured interviews afterwards. The second pilot test took place on September 23rd, primarily testing the adjustments based on the video documentation and interviews made with the participants in the first pilot study.  In the second pilot study approximately 10 museum guests participated and  nobody was interviewed afterwards.

During our first pilot study we asked  a few of the museum visitors to evaluate the robot guided tour they had previously experienced. We probed the visitors to identify how the robot guided museum tour should be in order to respond to the needs of the museum visitor. One of the museum visitors, Gunhild, said:

“It is great that the robot is pointing at the objects it’s talking about, however, it is way too imprecise. That’s confusing. It needs to describe explicitly which object it is talking about, if its movements don’t improve.”

Another museum visitor, Linus, who participated in the robot guided museum tour suggested:

“The robot seems to be performing a manuscript. It should maybe pose some more questions and wait longer for people to respond to it. That would make it a bit more interesting.”

In this way the exploratory research onsite probed the design of the human-robot-interaction at the museum further. The focus shifted from us as the experimenters to the museum visitors as the ones identifying the factors and effects of the study in-the-wild. As Yvonne Rogers points out:

“The locus of control shifts from the experimenter to the participant. It becomes much more difficult, if not impossible, to design an in-the-wild study that can isolate specific effects. Instead, the researcher has to make sense of data in the wild, where there are many factors and inter- dependencies at play that might be causing the observed effect.” [2011:59]

The fact that the exploratory pilot study was conducted in-the-wild also made it possible to invite museum visitors to contribute to the design of the robot tour indirectly (through an examination of the video documentation) as well as directly (through the semi-structured interviews). Moving the focus from experimenter to participant during an engagement with museum visitors onsite created a context for collaboration and meaningful exchanges that led to an adjustment of the design of  the NAO event in situ. The second pilot study was only documented on video and no interviews were made, as we found it hard to engage and convince the museum visitors to participate in a 20 min interview.  We did video record the event and learned from watching specific passages about the  human – robot interactions and lacks thereof.  Their responses to the museum robot intervention, however, were very useful in adjusting the script further preparing the actual experiment  at Medical Museion conducted in February 2016.

 

 

 

Iterative Process

It is an iterative process developing and testing the script and sound design for the guided museum tour with NAO at Medical Museion.

 

Here Alice Emily Baird advices on how to best write spoken language .

 

Here Stina Hasse  experiments with different sound designs: listen to the manipulation of voices here: 

 

testing the connotations sound designs bring

 

Together with Alice Emily Baird, Stina Hasse made some experiments on the listening experiences of synthetic voices, compared with non-synthetic voices.

We asked listener how they would characterize the voices heard in terms of Authority, Confidence, Passion, Sensuality, Sincerity and Vulnerability. We also asked them to evaluate if the voice heard was synthetic or non-synthetic.

In collaboration with programmer, Daniel Andersen, we made dynamic two pie charts in d3 (interactive data visualization that can be hosted on a local sever programmed in html or javascript). They pie charts are based on the data we got from our listener questionnaires.

The first part of the dynamic pie chart shows how the listener assigned e.g. authority to a voice saying the word “objects”. It also shows if the listener believed the voice to be synthetic or non-synthetic. The second part of the dynamic pie chart shows the actual percentage of synthetic and non-synthetics voices who were evaluated to have an authoritative characteristic.

 

The second pie chart shows how listeners distributed the different characteristics to the synthetic voices they heard. The second part of the second dynamic pie chart shows how listeners distributed the different characteristics to the non-synthetic voices they were presented with.

 

Below are some photos from our presentation of the project at Open Studio at Columbia University, Prentis Hall, November 15th 2015.

 

The idea has  been to design a voice for the NAO robot based on the vocal attributes (frequency, intensity, formants and pulse) of the voice that listeners chose to be the most authoritative, sensual, vulnerable etc. and test if the users would answer the same, when the voices was spoken out laud by the robot.

 

 

Giving voice to NAO

Here are the voices personifying NAO in the complimentary experiment we are going to make at Medical Museion.

Current MFA students at the Columbia Computer Music Center (CMC) Alice Baird, Cameron Fraser and Chatori Shimizu as well as actor Maria Sten helped editing the script as well as recording it at the CMC Studio.

Here are some pictures from the different recording sessions. Alice Baird is here giving life to one of the voices that will be implemented in NAO.

Cameron Fraser helped giving body and soul to the script, bringing an amazing southern vocal performance to one of the voices that will be implemented in NAO.

Maria Sten voice acted to help animate NAO and giving it personality and character with her performance at the CMC studio.

Chatori Shimizu and Stina Hasse has contributed with there vocal performances to the experiment where NAO performs its script with multiple voices.

NAO as a museum guide documentation

NAO as a museum guide 2

We conducted a second experiment Wednesday 23rd of September 2015. This was a very different experience. The robot performed a slightly different version of the tour compared to the tour given on Sunday the 20th of September 2015.

Primarily we made the tour a bit shorter (we cut our the part where NAO explains about the amputation tool in the glass montre). We also made NAO talk slower with longer breaks between the sentences.

These changed were made after the first public tour with NAO as a museum guide, where we observed how people reacted to the robot and interviewed some of the museum visitors about their experiences.

In the few interviews we conducted with three of the people who had participated in the experiment, it came through that the part about the amputation tools made people confused. They didn’t know if they should look at NAO or at the tools behind them, since NAO didn’t walk (and that would take a very long time)  over to the tools in the glass montre. It was also mentioned that the feeling of interacting with the robot, the experience of the robot as an authentic interlocutor, was disrupted from the very beginning of the tour where NAO says “Are you ready?”. Here the interviewees had expected NAO to wait for them to answer. Instead NAO interrupted them while giving their answers, saying “I am ready!”. This little interaction can lead to a lot of theoretical insights and discussions on human-robot interaction. However, the primary reason for this first public robot tour at Medical Museion was to adjust and change a few of the NAO’s behaviors, so that the human-robot interaction can appear more smooth and less confusing. This is the reason for our adjustments in the script.

As mentioned above, we thought that our changes to NAO’s manuscript and way of interacting with the museum visitors would lead to a more smooth and less confusing interaction between the robot and it’s audience. We conducted a second experiment to test this on Wednesday September 23rd. So how did it work?

The changes didn’t give us the feedback we thought. The interaction between the robot and the museum visitors were completely different. In general the reactions to what the robot said and behaved were  very different. The robot had a very different position and role compared to the first robot guided museum tour.  This was probably due to many different factors and not necessarily due to any of the changes that we had made; the average age of the museum visitors were very different, the reason for the museum visitors to be at the museum, the introduction to the experiment etc.

What happened? What was different in the interaction? Why did the second group of museum visitors react so differently on the robot guided museum tour? Did NAO manage to engage the museum visitors in the exhibited objects in the room, or was the robot more an interference and an annoying element obstructing the contemplation of the exhibited objects?  This will be reflected upon further.