Multimodal literacies and academic publishing: The eTalks

1.Introduction: Academic communication and digital multimodal literacies11

Printed monographs, collected essays and articles have been cornerstones of modern academic communication for decades. If the use and teaching of rhetoric was expunged from German and French universities at the end of the 19th century [3], today, the strong presence of orality in digital academic communications and publications invites one to reconsider the place of rhetoric in academia, a trend that is also present in general culture. Indeed, cultural performances enlightening orality are emerging, such as the creation of a “Radio Cinema” in Copenhagen (Denmark). This initiative has been a popular success. People are going to a cinema to listen to stories, without anything being projected on the big screen, in the dark:

Other such initiatives are foregrounded in cities and festivals that present the crossroads of writing and orality, or a combination of writing, orality, images or music. For example, in Lausanne (CH), a cinema hosts a monthly night of oral performances done by authors or people reading literature, sometimes accompanied by images or music. A team of six local publishers is at the core of this initiative.22 A Swiss theater (Morges, CH) organizes a monthly collective reading of pieces that the volunteer readers only discover at the same time that the public listens to their reading. A deep concentration on listening is required by the performers.33

This cultural “coming back” of orality and listening, sometimes mixed up with images and sounds, is largely stimulated by digital multimodal literacies. Indeed, as Kress already argued in 1998, digital culture leads to the emergence of plural literacies, or multiliteracies [12]. Nevertheless, the impact of this important remark has not been considered sufficiently in Digital Humanities publications, whereas the preoccupation with multiliteracies or multimodal literacies had begun to be used in the educational field about a decade before the expression “Digital Humanities” came into being and used outside of the Humanities and Computing field.

In 1996, the Harvard Educational Review (HER) published an article illustrating how literacy pedagogy in the digital age can reflect societal changes such as globalization, technology and increasing cultural and social diversity [14]. Before this junction within digital culture, the term “literacy”, born in the middle of the 19th century at the height of printed culture [1], was first changed to plural form by ethnologists and anthropologists [6], then by Ancient Classicists [11] but only at the beginning of the 21st century. The 2013 online Encylcopedia of Ancient History is still proposing an article with the singular term “literacy” for the Greco-Roman period, what has lead me, as an author, to makes efforts to explain in a short text why Ancient cultures should be associated with plural literacies [7].

In Digital Humanities, a 2012 article by Tanya Clement signals the presence of the notion in digital pedagogy. Clement draws a picture of digital academic communication and education, putting forth the core term “multiliteracies” in her definition thereof. She discusses diverse DH pedagogies, such as new media studies and game studies, by looking at multiliteracies “that are engaged within undergraduate humanities curricula through general skills, principles and habits of mind that allow students to progress within and engage society in the twenty-first century” [5]. The current students in classrooms have still been trained at school in a quasi-unique literacy mode: printed literacy. However, they are living in a cultural world that has already switched to multimodal literacies. These same students will be the future scholars who produce knowledge in multimodal digital ways. It is quite easy to guess that educational and editorial tools have to be developed to foster this emerging multimodal knowledge. One begins to find online blogs on teaching resources about multimodal literacies,44 or research centers such as the Multimodal Analysis Lab that “develops new approaches to the analysis of images, videos and websites using interactive digital media technology”.55

Convinced that textuality, images and sounds have to be used together in Humanist digital academic publications [9] and that digital multiliteracies have to meet high editorial requirements, we have built a new editorial form: the eTalk, based first and foremost on the speeches of scholars. After presenting this multimodal editorial platform, including a focus on the delicate question of digital rights management and the scheduled developments (point 2), a collaboration with the Erasmus+ #dariahTeach66 project will be presented. All of this will be included in a module focused on multimodal literacies led by Claire Clivaz (CH), Marianne Huang (DK) and Stef Scagliola (NL; point 3).

2.The eTalks

The hosting of oral speeches from courses or conferences is widespread on the web. Such web content ranges from simple videos or MP3 recordings,77 to video-slide linkage solutions like Slideshot at the EPFL (CH), or InfoQ.88 All of which are steadily improving as web technology progresses. However, these may prove insufficient to many researchers since they are not quotable in detail and they do not offer the possibility of being combined with textuality, images, hyperlinks, and references. Consequently, the eTalks offers a unique possibility of editing multimedia content with as many details as a written article, with all such elements quotable in detail.

The eTalks application implements an easy-to-use editor interface, designed for the use of researchers themselves for the creation and editing of original enhanced talks. This permits the linking together of images, sounds and textual materials with hyperlinks, thereby enriching the content with relevant information. The result of the edition is displayed through a viewer interface, allowing one to experiment with the entire eTalk or to actively navigate, scroll and search inside its content. After having recorded the speech of the scholar, the software, Audacity, allows for the splitting of the speech in pieces of 2–3 sentences. Each piece of speech can be associated with its written version, a slide, images, or hyperlinks and so forth. Each piece is also quotable with a specific URL: a new kind of reference (see a presentation video99). Thus, the final release of eTalks allows for the complete ‘citability’ of its contents: each and every portion of the researchers’ talks can be precisely referred to and therefore cited, just like any traditional, paper-based scientific publication but with all the potential for plural literacies.

The concept of the eTalk was developed by Claire Clivaz (SIB, VITAL-IT) and Frédéric Kaplan (EPFL) in conjunction with an interdisciplinary team of colleagues. The core of the eTalk engine was developed in JavaScript by Frédéric Kaplan and Cyril Bornet (EPFL). The code is now available as open source on Github as a free application for further development;1010 relevant guidelines are available to interested users.1111 The eTalks is currently being further developed and disseminated by our interdisciplinary team of researchers in Digital Humanities and bioinformatics, at the Swiss Institute of Bioinformatics (Lausanne, CH: etalk.vital-it.ch). The following pattern summarizes how an eTalk is built.

Fig. 1.

The eTalks editing process.

Four series of eTalks have thus far been published as openly accessible: twelve on funerary rituals, nine on the enhanced Human, two on the Lausanne institutional Biobank, and one on Digital Humanities.1212 The eTalks are now in development by institutional and research collaborations, notably with colleagues from the Pedagogical High School of Lausanne (HEPVaud)1313 and the ERASMUS+ #dariahTeach project, whose purpose is to offer a webportal by 2017 that will include digital teaching modules (see point 3).

Fig. 2.

The cover images of the four eTalks series.

The question of DRM (digital rights management) is a constant concern for our team. Indeed, this particularly delicate point is presently not handled by the usual publishers. We are lucky enough to have Dr. Frédéric Schütz, who is also working for Wikimedia CH, in our SIB team. He has trained our team, and we are attempting to deal with the copyright question with respect to different levels of responsibility, reserving a few delicate cases for the juridical service of our institution. We check absolutely all copyrights, and attempt to implement the most accurate form of DRM administration. We encourage the authors to use material coming from free databases, such as Wikimedia Commons, as much as possible. Copyrights are asked for whenever it is necessary. For example, sometimes we have to ask a photographer for the use of their pictures, sometimes we have to ask a book edition for the use of a front cover.

Being so attentive to the copyright question resulted in the experience of a few cases which are worth sharing while keeping them anonymous. In certain cases we have had to decline the use of materials. For example, a certain journal required the payment of $3000, to be renewed every year, for the use of a mere screen shot of an article page where one could only barely read the title. As another example, a magazine was asking $2000 as a “standard base fee without exclusivity” for a video screen shot. Of course, we had to disappointingly decline the demands in these cases. However, as a juxtaposition, upon being asked for the right to use material, a book editor replied that he was not in a position to give us the right to use a front cover image of one of his books, since the picture on it was too old to find the owner of its rights. A more complicated case is the request we made of a blog for the use of two images it had published. The person responsible could not find where the two images came from and redirected us to their US branch. Unfortunately this branch could not inform us either and redirected us to another blog, which also did not possess the rights to the images. After returning to the original blog, it turned out that nobody really knew to whom the two images belong.

We are learning a lot about the laws which are related to our open access publication, but we also remain with some open questions. One example is the attempt we made, after we did not find any suitable alternative, to use two pictures taken from Google, indicating their link and referring to this usage as an “academic usage of commented images”, according to Swiss law usage (art. 25 LDA), and with the authorization of our juridical service.1414 The eTalks on the Enhanced Human have been completed without paying one Swiss Franc in DRM: all authors agreed to give the priority to open access material. Yet even with Wikimedia, a careful evaluation of each image has to be done. For example, certain images with children cannot be freely reused, or Swiss law can differ from US law usage, etc. In summary, the copyrights investigations represent a quite considerable part of our editing process.

With respect to improvements and the future direction of eTalks, we will provide guidelines for future authors aiming to record an eTalk of their work. Those guidelines will permit a more efficient exchange between our team and the author during the creation process: such as providing image format and size requirements. With regards to the eTalk software specifically, several major improvements will be considered. Firstly, a full user-rights management system also has to be established: it will allow for the coping with of a steady increase in eTalk projects and their authors’ expectations in terms of privacy and diffusion. Our team will also offer DRM training to interested people. We also aim to extend the range of media supported, such as enabling the citation of short films in eTalks, as well as the use of a stable URL system for quotation.1515 Moreover, sharing options have to be taken into consideration in order to take full advantage of social media sharing possibilities and the increase of authors’ eTalk visibility. A top-level web application has to be designed, including the implementation of a search engine. This would permit users to search for specific eTalks, authors or topics inside the eTalk library. However, we will also continue to explore collaborations with the usual publishers and to examine the diffusion of eTalks with their help. Finally, in collaboration with the ERASMUS+ #dariahTeach project, we will develop the most efficient way of recording and editing an eTalk by oneself with some editorial help from our team. Our purpose is to encourage academics to use this method of editing to quickly make their recent talks and slides available online.

3.A collaboration with the ERASMUS+ project #dariahTeach

In November 2014, a consortium of seven partner countries, associated with the European Digital Research Infrastructure for the Arts and Humanities (DARIAH-EU), received an ERASMUS+ grant to establish a strategic partnership in Digital Humanities teaching. From this 30 month grant (2015–2017), the consortium will build an extensible, open-source and open-access reference curriculum platform and develop modules to promote multilingual DH education.

The #dariahTeach project builds upon previous initiatives to develop and deliver a model for open-source asynchronous online educational materials from which a wide variety of communities of practice can benefit. The training materials will be designed as building blocks tailored to the exigencies of teaching situations and will be based on a format that allows easy localization and adaptation (via translation, subtitles, domain-specific examples etc.). The language of all of the materials will be English, with translations or subtitles (as appropriate) in the language of the partner-country that will develop specific modules. Moreover, the modules are being developed across multiple disciplines, genres, and formats including text images, moving images, and sound.

The training materials will present content in various degrees of complexity allowing for use and re-use at different levels and different modalities of education: from formal classroom settings, to professional development in the form of workshops and summer schools, to non-formal individual learners wishing to improve their skills. The direct result of the partnership during the grant period will be the first multilingual collection of DH training materials that is freely available and openly accessible. By embedding and promoting localization principles in the field of DH education and by fostering multilingualism – not only theoretically, but also practically – this partnership will make a significant, new contribution to the internationalization of the Digital Humanities landscape. All the presentations regarding the project are available in the rubric news of the website, dariah.eu/teach. In addition to being directly accessible through our custom-built web platform, the materials will also be hosted by the French national repository HAL, which is maintained by DARIAH-FR.

Conscious that the present landscape of online teaching is deeply influenced by the MOOCs (Massive Open Online Courses) [2], #dariahTeach wishes to operate within another perspective. Indeed, as Ozturk recently stated, if the original purpose of MOOCs were to encourage “autonomy, diversity, openness and community participation”, they have, over time, moved away from their original philosophical and pedagogical values. New variations of MOOCs have emerged becoming more market oriented “aligning with instructivist, cognitive, and behaviourist pedagogy” [15]. Siemens advocates that MOOCs are destroying open education [16]. Moreover, the financial model of the MOOCs raises questions about the audience for and motivations behind this method of teaching [13,15].

Consequently, #dariahTeach will explore the production, dissemination, and promotion of high quality, dynamic, extensible, localize-able, and integrated educational materials for the digital humanities specifically tailored for third level education. Unlike MOOCs, the goal of the strategic partnership is not to develop a single module that will be delivered by a single instructor (or a team of instructors) during a set period of time, but to offer a flexible platform that can be integrated into diverse teaching and learning situations. The project team is currently developing the infrastructure and design of the modules based on the production of five modules: Introduction to Digital Humanities, Text Encoding, AudioVisual Media (AVM) and Multimodal Literacies, Retrodigitizing Dictionaries, and Ontologies and Knowledge Management. It is envisioned that these modules will be used beyond the DH community as the societal impact of a culturally-driven digital transition grows, opening up new ways of collaborating on productive theory and critical thinking [10]. Inside of the module on AVM and Multimodal Literacies, we will develop the most efficient way of recording and editing an eTalk by oneself with some editorial help from our team.