Jorge C. S. Cardoso

A-Frame for Web VR Development

Thu, 18 Jul 2019 00:00:00 +0000

Virtual Reality (VR) is a term that is increasingly known and present in our daily lives. Although this has been an area of research and development since at least the sixties, recent developments in consumer devices (e.g., Oculus Rift) shifted attention back to it. In this work we set out to explore a web framework for VR and how much effort a Web developer would need to create a 360º virtual tour. This work was done by Solange Santos in the context of a scientific initiation grant.

Introduction

In this latest wave of interest in VR, the Web may have an important role to play in the adoption and dissemination of VR experiences. Specifically, there is a huge opportunity in leveraging the popularity of Web development languages (Figure 1). If we could take advantage of the large number of Web developers that are used to coding in HyperText Markup Language (HTML) and JavaScript and provide a familiar language and development workflow but for VR content development, this could have a huge impact in VR.

Figure 1 - Stack Overflow Developer Survey Results 2019 – Most Popular Technologies - Programming, Scripting, and Markup Languages.

A recent Web framework for VR is A-Frame. A-Frame is an open-source web framework for building VR experiences based on HTML and JavaScript, originally developed within the Mozilla VR team. It was developed to be an easy and powerful tool for creating VR content on the Web. Additionally, A-Frame is multi-platform, supporting VR headsets such as HTC Vive, Rift, Windows Mixed Reality, Daydream, GearVR and Google Cardboard.

Objectives

The purpose of this work was to evaluate the capabilities of A-Frame and the required development effort to create a 360º interactive virtual tour by a content creator with a Web development background.

To serve as a case study, we developed a virtual tour for the Conimbriga Monographic Museum. Located in Condeixa-a-Nova, Portugal, the museum has a collection composed exclusively of archaeological material found in the ruins of the Roman city of Conimbriga. The current exhibition displays everyday objects organised by theme. Six of these objects have been classified as of national interest (Figure 2). The virtual tour highlights these objects of interest and allows users to interact with them to see additional details.

Figure 2 - National interest objects.

Information panels

In this post, I will just focus on a small part of the problem: creating the information panels that appear when users ask for information about an object. The full report can be downloaded from the University of Coimbra’s repository.

Information panels are “popup” panels that open when users activate an object (defined by a “hot region” in the 360º photo). Objects that can be activated are marked with a stamp (see Figure 3).

Figure 3 - Information panel.

To implement the panel you see in Figure 3, we used the following code:

<a-entity info-pop-up>
    <a-image class="image" src="#selo_liturgico" rotation="0 0 0" position="3.6 1.1 -9" height="8" width="8" opacity="0"></a-image>
    <a-text class="title" font="montserrat-bold.fnt" rotation="0 0 0" position="0.2 0.8 -6" value="SELO LITÚRGICO" align="left" color="#595a5a" opacity=0></a-text>
    <a-text class="description" font="montserrat.fnt" rotation="0 0 0" position="0.15 -0.1 -6.5" value="Produzido ...." align="left" color="black" opacity=0></a-text>
    <a-entity class="click_target" rotation="15 158 0">
        <a-box position="0 0 6" width="0.9" height="1.1" depth="1" color="red" opacity="0"></a-box>
    </a-entity>
</a-entity>

We made a number of mistakes in this implementation, in addition to other issues that result from how A-Frame works.

One of the obvious issues is that content and presentation are mixed. With A-Frame we can’t use CSS to style the 3D objects inside the VR world. Equally good would be to be able to mix standard HTML inside an A-Frame scene. It would be great, for example, to be able to style the panel, which is a simple flat rectangle, with standard HTML and CSS and then put it inside the 3D scene. This would also help in supporting desktop VR – in a desktop, we could display the information panels as 2D standard panels over the 3D scene, instead of inside the 3D scene. This would allow users to right-click on images on the panel and to select and copy text, for example.

A mistake we made was to code each panel separately, leading to lots of duplicate code. This means that if we want to change the look of all the panels, we need to make changes in every panel. We should have experimented with the A-Frame template component (but we were unaware of it at the time).

Another issue was positioning the panels. There is not much that could be done here: in A-Frame things are 3D positioned and we need to align the panel with the object on the 360º photo it refers to. But still, positioning things in general is hard in A-Frame when compared to the positioning strategies we use on traditional Web development.

Conclusion

We actually uncovered a lot more issues and difficulties with this work – check out the full report. Still, I would say, an important take away is that despite these difficulties, A-Frame is doing a great job in bringing together the Web development community with the VR community. Check out their blog for the spectacular VR experiences that people around the world are building.

Journal of Science and Technology of the Arts - My Last Editorial

Sat, 05 Jan 2019 00:00:00 +0000

The Editorial for Volume 10, No. 1 issue of the Journal of Science and Technology of the Arts (CITARJ) was my last editorial as editor of this journal.

I have been working for CITARJ since 2011 and have been involved in various of the changes that have occurred throughout the years.

In this post, I publish a copy of my last editorial and I make a short overview of the history and evolution of CITARJ.

Editorial - Volume 10 No. 1

My last editorial for CITARJ (Cardoso, 2018):

In last year’s editorial, I described the various evolution phases through which the Journal of Science and Technology of the Arts (CITARJ) had gone since its inception in 2008. This year of 2018 marks the 10th volume of CITARJ and also the beginning of a new phase: the editorial team (editor and associate editors) is stepping down and a new team is taking over CITARJ. I am sure that the energy of the new team will keep CITARJ on the right track and push it to new levels of reputation, and scientific and editorial rigor.

CITARJ is now an established scientific journal. It has published an average of 16 articles+reviews per year in the last 5 years (compared to an average of 9 in the previous 5 years) – see Chart 1. It is indexed in DOAJ, Scopus, and Web of Science Core Collection – Emerging Sources Citation Index. It is now also listed in Scimago Journal & Country Rank and ranked in Quartile 2 for the “Visual Arts and Performing Arts” category.

CITARJ’s impact has also been steadily increasing. The average number of citations to our articles is now at about 0.3 citations per article (internal citations are residual as can be verified in Chart 2). Impact can also be observed in the growing number of full text downloads of our articles. Since we have started measuring in 2015, downloads have increased from about 1500 in 2015 to more than 4000 in 2018 (Chart 3).

Although we can consider CITARJ as an established journal, this does not mean that the new editorial team that is taking over in January 2019 can rest. Scientific publication is, perhaps now more than ever, a tricky business. Competition by commercial publishers is fierce and is now polluted with “predatory” publishers that could not care less about science, ethics, or rigor. Still, they are out there and constantly preying through our inboxes. CITARJ must find ways to reach new readers, authors, and editors while still abiding by privacy preserving regulations. CITARJ must continue its growth course in order to stay relevant both internally (within CITAR and the Portuguese Catholic University) and externally (nationally and internationally). This may require a different resource management strategy – one that encourages and recognizes volunteer work by the faculty of the School of Arts.

In this issue of CITARJ, Rita Xavier Monteiro and Helena Barranha discuss affectivity in the context of Post-Internet art. António Bandeira Araújo teaches you how to manually draw equirectangular virtual reality panoramas using a ruler, compass, and protractor. Mei-Kei Lai reviews how scent has been treated as an aesthetic medium in art and discusses opportunities for the use of olfactory displays. Pedro Pinto Neves, Leonel Morgado, and Nelson Zagalo propose a new model for agency in videogames using the concept of bio-costs contract. Finally, Kendra Chilson, and Máté Szabó provide us with a review of the exhibition “Thinking Machines: Art and Design in the Computer Age, 1959-1989”.

Chart 1 - Number of published issues and articles.

Chart 2 - Average citations per articles.

Chart 3 - Full text downloads per year.

A Bit of History

2008-2011 - The Beginning

The Journal of Science and Technology of the Arts was established in 2008 in a decision of the scientific council of the Research Center for Science and Technology of the Arts (CITAR). In part (or maybe in whole) this decision was a result of a suggestion by the internal advisory board of CITAR to create its own publication(s) to disseminate the research of the center. The Director of CITAR at that time was Joaquim Azevedo, but the task of creating the journal was accomplished by Álvaro Barbosa, which was also the first editor of the journal.

Álvaro edited the first two issues of CITARJ (Figure 1). At that time, the journal was distributed mainly in print form (it was mailed to various institutions free of charge). There was already a website where each issue could also be downloaded. Figure 2 shows a screenshot, taken from Archive.org, of the website as it was in August 2011 (at this point the site had already some modifications made by the second editorial team). The editorial team that started CITARJ and edited the first two issues was composed of

Álvaro Barbosa (Editor)
Daniela Coimbra (Associate Editor)
Vitor Teixeira (Associate Editor)
Laura Castro (Editorial Assistant)

During the first three issues the editorial process was manual: all communication between authors, editors, and reviewers was through email, without any automated way of keeping the state of each submission.

Figure 1 - Covers of the first two numbers of CITARJ.

Figure 2 - Screenshot of CITARJ's website mid-2011.

The first editorial, by Álvaro Barbosa, read:

“CITAR Journal represents a very significant effort for the dissemination of research in the field of Science and Technology of the Arts. It brings together different areas of knowledge and understanding, as well as diverse approaches in the materialization of research practices into the form of Scientific Journal Articles. In this first issue we present work of substantial relevance in topics within experimental development fields in the Arts, such as, art heritage reconstruction with virtual reality techniques, novel interactive narrative methods, networked music performance, contemporary recasting of early electro-acoustic music repertoire, collaborative musical performance with robots, musicological studies of the 19th century and cinematic research on milestone documentaries in film history. As a complement to this issue we present the overview of two major Art events that took place in Portugal in 2008 (ARTECH and Musica Viva), as well as a review on a selected Book on Fine-Art (Anachroniques by Daniel Arasse). By publishing the CITAR Journal on a regular basis, the Portuguese Catholic University foresees the need to stimulate the visibility of new insights based on innovative developments in the field of the Arts trough Science and Technology. We hope you enjoy this first issue.” – (Barbosa, 2009)

The editorial points to the initial mission of CITARJ: dissemination of all research within the area of Science and Technology of the Arts. In essence, all research related to the scope of CITAR itself. The editorial also mentions a complement to the issue in the form of a review of two events and of a book. It is not clear to me how this structure of CITARJ came to be (having an Articles section for research papers, and a Reviews section for non-peer-reviewed overviews of events and book reviews), but it was maintained until today.

2011-2014 - Processes

In 2011, the journal saw its first change in the editorial team, with Carlos Caires becoming the new editor with me as the editorial assistant:

Carlos Sena Caires (Editor)
Sofia Serra (Associate Editor)
Vitor Teixeira (Associate Editor)
Jorge Cardoso (Editorial Assistant)

This period that started in 2011 was a period of great changes (mostly operative) in the journal. During this period, we tried to address 3 axes of development for the journal:

Improvement of the editorial process;
Improvement of the publication quality;
Increasing the visibility of CITARJ.

One of the first (small) things we did was to add a listing of all articles published in a given issue and to allow individual download of the articles PDFs (Figure 3). We also started submitting the article PDFs to Mendeley (I think Mendeley was the first “database” where our articles started to appear).

Figure 3 - Screenshot of article listing with links to Mendeley.

It soon became obvious that maintaining the editorial process based on email communication would not scale well and that manually submitting the articles to databases and indexing services would not be feasible. We needed a tool to help us with these tasks. I personally looked at various alternatives but, given that we needed something that would have no direct costs, I found only one valid possibility: Open Journal Systems (OJS). We installed and tried it and decided to use it both as a workflow management tool, but also as a publishing platform.

I don’t think CITARJ had any formal editorial process at that time and OJS also helped to better understand the various steps that a manuscript can go through before final publication. This helped us improve the editorial process by having a more formal process that authors and reviewers had to adhere to. We also looked at the author guidelines and improved them by providing a document template.

Having a platform like OJS meant that we also had our articles’ metadata published in a machine-readable format which made it easier to submit CITARJ to various indexing services and databases. In 2012 CITARJ was submitted and accepted in Directory of Open Access Journals (DOAJ) and we (the School of Arts) also registered as a member of CrossRef so that articles published by CITARJ could be assigned a DOI and enter the Cited By network. This would allow a better dissemination and, hence, increased visibility of CITARJ’s articles.

In 2012, we also decided to launch an international call for papers, with the intention of attracting more submissions. As a new journal, one of the difficulties was in attracting a sufficiently high number of submissions so that we could published only high-quality papers. Having a low number of submissions means that you might be forced to publish a lower-quality paper in order to have a minimum number of articles in one issue of the journal.

During this period, we also started improving the information on the journal’s website with the goal of submitting it to Scopus and Web of Science (submitting to these indexing services was a goal set in the beginning by Álvaro, but these services require a minimum number of published issues so only now could it be done).

Three volumes were published during this phase (see Figure 4).

Figure 4 - Covers of volumes 3, 4, and 5 of CITARJ.

2014-2018 - Consolidation

In 2014 there was another change in the editorial team. I became editor with Sahra Kunz and Sofia Serra as Associate Editors. (Later, Guilhermina Castro and Cristina Sá, would replace Sofia Serra in the editorial team.)

This was mainly a consolidation period. We introduced minor improvements in the instructions for authors and added instructions for reviewers. We also added ethics guidelines based on COPE’s (Committee on Publication Ethics) Best Practice Guidelines for Journal Editors.

CITARJ finally got accepted in Scopus and Web of Science’s new index - ESCI (Emerging Sources Citation Index), and now also appears in Scimago Journal & Country Rank.

We re-organized the editorial board, ceasing the collaboration with some of the previous members, and inviting new ones. In part, this was due to a change in scope of the journal, imposed by the scientific council of CITAR. This change in scope removed some of the areas that CITARJ would accept (such as Conservation and Restoration; Cultural Heritage Studies).

We also introduced changes in the editorial process. CITARJ used to follow a blind peer-review process that we changed to double blind peer-review. This requires authors and editors to do a bit of extra work (editors need to check that the files that authors submit are really anonymized and in some cases perform additional anonymization before sending to reviewers) but assures us more impartiality in the review process.

One of the biggest changes of this period was the introduction of special issues in CITARJ. These would allow CITARJ to publish issues with a clear and restricted theme (something that does not happen in the regular issues) and to potentially reach a broader audience and attract new authors. In 2014, 2015, and 2016 CITARJ published two issues per volume (one regular and one special). In 2017 and 2018 CITARJ published three issues.

The final big change in CITARJ was operating with a zero-budget. Following a budget cut in CITAR, I decided that it would not make sense to spend money printing copies of the journal or paying for pagination services. I adapted the existing print layout to Microsoft Word (with a few changes to make it easier to layout) and performed pagination in Word myself. After a few numbers, it got easier and faster to paginate a complete number, but it still represented a lot of effort – definitely something to reconsider in the future. I also started making CITARJ available on a print-on-demand platform – Lulu.com. Instead of spending CITAR’s money printing the journal, we allowed readers to buy the print version directly from Lulu if they so wished.

We published 5 numbers during this phase (see Figures 5 through 9).

Figure 5 - Covers of Volume 6 (numbers 1 and 2) of CITARJ.

Figure 6 - Covers of Volume 7 (numbers 1 and 2) of CITARJ.

Figure 7 - Covers of Volume 8 (numbers 1 and 2) of CITARJ.

Figure 8 - Covers of Volume 9 (numbers 1, 2, and 3) of CITARJ.

Figure 9 - Covers of Volume 10 (numbers 1, 2, and 3) of CITARJ.

Ending remarks

CITARJ has been great learning journey for me. I knew virtually nothing about scientific publishing (and still have a lot to learn) and helping to build CITARJ almost from the ground up (from a journal with two published issues, to an indexed journal) provided an excellent learning opportunity. Having to deal with disgruntled, unethical authors, unprofessional reviewers and guest editors on the one hand, but also highly professional and dedicated authors, reviewers, and editors broadened my view scientific publishing. I now have a better sense of what is expected of me when acting as an author, reviewer, or editor of a journal.

References

Barbosa, Á. (2009). Editorial. Journal Of Science And Technology Of The Arts, 1(1), 0-3. doi:http://dx.doi.org/10.7559/citarj.v1i1.22

Cardoso, J. (2018). Editorial. Journal Of Science And Technology Of The Arts, 10(1), 1- 01-02. doi:http://dx.doi.org/10.7559/citarj.v10i1.575

Virtual Reality Environments - Students' Projects

Thu, 21 Jun 2018 00:00:00 +0000

I organized and taught a course on Virtual Reality Environments for the Master of Design program of the Faculty of Creative Industries, University of Saint Joseph, Macau.

The course was organized around the A-Frame Virtual Reality framework.

Being web-based, A-Frame is an excellent framework to created Virtual Reality environments. The resulting environments run on any browser, and platform, making it easy to share and view the worlds. It supports desktop, mobile, and VR platforms (such as Oculus Rift, or HTC Vive) and controllers.

The Course

The course materials are available online in the Glitch platform at https://aframe-usj.glitch.me/index.html. This course was not just about the technical aspects of A-Frame, but that was a major part of it.

We practiced things like different primitives provided by A-Frame, lighting, text, environments, inclusion of external 3D models, interactions, animation, portals, and sound!

Students were asked to incorporate what they learned in a personal project. Together, we created a virtual environment composed of several mini-worlds connected by portals. This is available at https://vre-2018.glitch.me. The main entrance was a 360º photo world of the classroom with all the students (and teacher). From this world, we can reach each of the student’s worlds which are themselves composed of a 360º photo world and a 3D world.

Here are some of the best projects that students created during this course:

Alvira and Peggy created an imaginary shelf world, inspired in a grocery store. They produced some 3D objects in Magica Voxel and used A-Frame primitives beatifully to create the shelves, a glass of milk, cookie boxes, etc. They also used a particle system with custom textures.

Jason and Victoria imagined what could be behind a toilet! :P They created an imaginary world where creatures live beyond the toilet pipes. Beautiful use of A-Frame primitives and of the Ocean component.

Maggie and Manyee created a prototype of an interactive VR world for teaching children about objects’ shapes. They combined external 3D objects with A-Frame’s primitives and lots of interactions and particle systems to create a visually rich environment.

There were lots of other good projects that you can see here:

Conclusion

Putting together this course represented a lot of effort but also an opportunity to learn more about A-Frame. A-Frame is still a young project, with various bugs that creep out every now and then. There are also a lot of external components made by various developers in the A-Frame community. These externals add a lot of power to the framework but, like the framework, they are not all very polished or consistent among them. Be prepared to do some debugging!

Teaching this course was a lot of fun. I had a bunch of great students, that (tried) pushing the limits of A-Frame to create their dream worlds. It was an intensive course, for design students, so we kept out of programming our own components. We kept out of JavaScript entirely! Still, the results were excellent and revealed a lot of creativity. It also revealed how good A-Frame is for teaching VR to students without a programming background.

Adobe PhoneGap Build para construção de aplicações móveis híbridas

Tue, 27 Mar 2018 00:00:00 +0000

Este artigo foi originalmente publicado na Revista Programar #59: [https://www.revista-programar.info/edicoes/edicao-59/].

Introdução

No artigo “Criar uma aplicação móvel com jQuery Mobile” [Revista Programar #58] vimos como usar a framework jQuery Mobile para programar uma aplicação Web direcionada a dispositivos móveis. Neste artigo, vamos ver como usar o serviço PhoneGap Build da Adobe para gerar uma aplicação para Android.

Adobe PhoneGap

PhoneGap é uma framework de desenvolvimento de aplicações móveis híbridas. Aplicações híbridas são aplicações que combinam componentes nativos e componentes Web. Do ponto de vista do utilizador e da plataforma móvel, uma aplicação híbrida é indistinguível de uma aplicação nativa. No entanto, internamente, uma aplicação híbrida utiliza um componente Web View que contém a maioria do conteúdo e lógica da aplicação – ou seja, a aplicação é essencialmente programada como se de um aplicação web se tratasse. A framework PhoneGap, baseada em Apache Cordova, permite que as aplicações híbridas tenham acesso a funcionalidades nativas através de componentes específicos para cada plataforma móvel (e.g., Android, iOS, Windows) mas cuja interface é exposta em JavaScript (o programador não se preocupa com a plataforma).

PhoneGap Build

O website PhoneGap Build é uma interface web simplificada para o processo de build da framework PhoneGap, permitindo-nos “empacotar” uma aplicação web numa aplicação nativa para as várias plataformas móveis. Uma das vantagens deste website é que nem sequer temos de conhecer muito sobre PhoneGap uma vez que o website esconde a maioria dos detalhes.

Preparar a aplicação

Figura 1 – Ecrã da aplicação. Código completo: https://goo.gl/P9dEHZ

A Figura 1 mostra um ecrã da aplicação construída no artigo da edição anterior. O link fornecido pode ser usado para descarregar o código da aplicação. A forma mais prática para descarregar o código será:

Seguir o link fornecido na Figura 1
Escolher a opção “Export” no canto inferior direiro
Escolher a opção “Export .zip”

Depois de descarregar o ficheiro Zip e descomprimi-lo, o resultado será um único ficheiro HTML que iremos de seguida modificar ligeiramente.

O código original inclui referências ao jQuery e ao jQuery Mobile com recurso a endereços CDN remotos (e.g., http://code.jquery.com/mobile/ 1.4.5/jquery.mobile-1.4.5.css). Para evitar ligações de rede desnecessárias na nossa aplicação vamos converter estas referências em referências a ficheiros locais. Para isso temos de:

Editar o ficheiro index.html
Procurar todas as referências a ficheiros externos (CSS, imagens, JavaScript)
Seguir essas referências e descarregar os ficheiros, gravando-os na mesma pasta do projecto
Alterar as referências no ficheiro index.html, mantendo apenas o nome do ficheiro. Por exemplo, a referência ao ficheiro CSS do jQuery Mobile seria convertido de

<link rel='stylesheet prefetch' href='http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.css'>

para

<link rel='stylesheet prefetch' href=jquery.mobile-1.4.5.css'>

A estrutura final da pasta do projecto deverá ser algo como:

projecto-jquery-mobile
|-- index.html
|-- jquery.min.js
|-- jquery.mobile-1.4.5.css
|-- jquery.mobile-1.4.5.js
|-- space-cube.svg

Depois de fazermos e gravarmos estas alterações podemos voltar a comprimir a pasta do projecto num ficheiro Zip.

Gerar o ficheiro keystore

As aplicações móveis distribuídas através de uma loja (e.g., Android Play, ou App Store), precisam de ser assinadas. Para tal é necessário a geração de um certificado que identifica a entidade que criou a aplicação e uma chave privada e uma pública. A chave privada é usada para assinar digitalmente a aplicação. A emissão de um certificado é feita tipicamente por uma entidade reconhecida globalmente, mas, neste artigo, iremos gerar nós próprios este certificado. Para tal recorremos à ferramenta keytool do Java.

Para executar esta ferramenta, temos de usar uma linha de comando (Command Prompt no Windows, ou Terminal no Mac OS). Aqui vou exemplificar o comando em Windows (realçado o texto introduzido por mim, em resposta aos pedidos do keytool; [enter] representa o pressionar da tecla “Enter”):

C:\Users\jorge>"c:\Program Files\Java\jdk1.8.0_121\bin\keytool.exe" -genkey -alias androidkey -v -keystore android.keystore [enter]
Enter keystore password: supermegapass[enter]
Re-enter new password: supermegapass[enter]
What is your first and last name? 
  [Unknown]:  Jorge Cardoso[enter]
What is the name of your organizational unit?
  [Unknown]: [enter]
What is the name of your organization?
  [Unknown]: [enter] 
What is the name of your City or Locality?
  [Unknown]:  Porto[enter]
What is the name of your State or Province?
  [Unknown]: [enter]
What is the two-letter country code for this unit?
  [Unknown]:  PT[enter]
Is CN=Jorge Cardoso, OU=Unknown, O=Unknown, L=Porto, ST=Unknown, C=PT correct?
  [no]:  yes[enter]

Generating 1,024 bit DSA key pair and self-signed certificate (SHA1withDSA) with a validity of 90 days
        for: CN=Jorge Cardoso, OU=Unknown, O=Unknown, L=Porto, ST=Unknown, C=PT
Enter key password for <androidkey>
        (RETURN if same as keystore password): [enter]
[Storing android.keystore]

O ficheiro resultante, especificado no parâmetro “keystore” do keytool, é colocado na mesma pasta onde executamos o keytool (no meu caso em C:\Users\jorge\android.keystore). É este ficheiro que irá ser adicionado ao PhoneGap Build. É importante notar que o parâmetro “alias” do keytool terá de ser introduzido no site do PhoneGap Build (assim como as passwords que escolhemos) pelo que convém memorizar.

Website PhoneGap Build

De seguida, podemos registarmo-nos no site PhoneGap Build, escolhendo o plano grátis (ver Figura 2).

Figura 2 – Registo no site PhoneGap Build. http://build.phonegap.com.

Adicionar o ficheiro keystore ao PhoneGap Build

Antes de configurarmos a nossa aplicação no PhoneGap Build, devemos fazer upload do ficheiro android.keystore, tal como indicado na Figuras 3 e 4. É necessário cuidado ao introduzir o “alias” uma vez que este deve ser exactamente igual ao parâmetro “alias” que usamos no comando keytool.

Figura 3 – Adição de chave Android.

Figura 4 – Upload do ficheiro android.keystore e desbloqueamento da chave.

Depois de localizarmos o ficheiro android.keystore e clicarmos em “submit key”, devemos desbloquear a keystore clicando no ícone do cadeado (do lado direito na mesma página) e introduzindo a password que escolhemos para a nossa keystore quando corremos o comando keytool (ver Figura 4, lado direito).

Criar a aplicação no PhoneGap Build

O passo seguinte será configurar uma aplicação no PhoneGap Build e carregar o nosso código fonte. Para tal, acedemos à secção “Apps” e fazemos upload do ficheiro zip que criamos no início.

Depois do upload, o PhoneGap Build mostra-nos um ecrã em que podemos definir um nome e descrição para a nossa aplicação e iniciar o processo de build. A Figura 5 mostra os passos necessários.

Figura 5 – Configurar aplicação, carregar código fonte e correr processo de build.

Testar a aplicação

O último passo é instalar e testar a aplicação no nosso dispositivo móvel. Há duas formas de o fazer. Uma é descarregar manualmente o ficheiro APK (Android Package file), clicando no ícone relativo ao Android (Figura 5, em baixo); transferindo o ficheiro para o dispositivo móvel; e finalmente fazendo a instalação executando o APK. A outra forma é usando um leitor de QR Codes e apontando o dispositivo móvel para o QR Code que aparece no ecrã (Figura 5, em baixo).

O ficheiro APK que podemos descarregar depois do processo de build é também o ficheiro que precisamos para distribuir a nossa aplicação através da loja Google Play.

Conclusão

Os passos que vimos neste artigo são suficientes para, depois de termos uma aplicação Web programada para dispositivos móveis, conseguirmos gerar e distribuir uma aplicação como qualquer outra aplicação nativa. A framework de desenvolvimento híbrido PhoneGap tem, obviamente, muitas funcionalidades que não exploramos aqui. No entanto, para quem está sobretudo habituado a programação Web e quer experimentar ou distribuir de forma rápida a sua aplicação, esta pode ser uma boa forma de o fazer.

Criar uma aplicação móvel com jQuery Mobile

Fri, 17 Nov 2017 00:00:00 +0000

Este artigo foi originalmente publicado na Revista Programar #58: [https://www.revista-programar.info/anuncios/revista-programar-no-58-outubro-2017/].

Introdução

Já muito foi escrito sobre a biblioteca jQuery para JavaScript, incluindo alguns artigos na Revista Programar (por exemplo, “Mitos do jQuery” e “jQuery: Usar ou Não Usar?”).

No entanto, existem outros projectos “irmãos” do projecto jQuery que são igualmente interessantes para programadores e designers Web, como as frameworks jQuery Mobile e jQuery UI. Neste artigo, foco-me na jQuery Mobile explicando a sua filosofia de programação, e mostrando alguns dos componentes principais para a criação de uma aplicação móvel.

O que é, e porquê usar?

jQuery Mobile é, ao contrário da biblioteca jQuery, uma framework para aplicações móveis. Usar jQuery Mobile implica seguir uma estrutura pré-definida para a nossa aplicação. Esta framework permite-nos criar rapidamente aplicações Web com um look and feel semelhante a aplicações móveis nativas. A Figura 1 mostra um exemplo do que se consegue facilmente obter com jQuery Mobile. A imagem ilustra uma aplicação com uma gaveta de navegação lateral (Figure 1, à direita) acessível através de um botão no cabeçalho (Figura 1, à esquerda) da aplicação.

Figura 1. Possível aspecto de uma aplicação desenvolvida com jQuery Mobile.

Uma das vantagens da jQuery Mobile é que a estrutura da aplicação é definida inteiramente através de HTML. Por exemplo, a gaveta de navegação da aplicação da Figura 1, é construída através do código HTML seguinte:

<div data-role="panel" id="mypanel1" data-display="overlay">
 <img src="space-cube.svg" width="240" >
 <ul data-role="listview">
   <li><a href="#first-page">Page 1</a></li>
   <li><a href="#second-page">Page 2</a></li>
   <li><a href="#third-page">Page 3</a></li>
 </ul>
</div>

Essencialmente, a framework jQuery Mobile dá-nos um conjunto de elementos de interface gráfica com aspecto visual e comportamento adequados a aplicações móveis. Esta framework pode ser usada na construção de aplicações móveis finais, mas pode também ser útil para criação de protótipos funcionais. O facto de ser baseada em tecnologias Web, pode torná-la especialmente útil para designers multimédia interessados em criar e testar um protótipo funcional de uma aplicação móvel: é possível definir todos os ecrãs e transições entre ecrãs que não dependam de lógica específica da aplicação sem necessidade sequer de usar JavaScript.

Filosofia de programação

A filosofia de programação jQuery Mobile assenta essencialmente na utilização de atributos data-* nos elementos HTML e classes CSS pré-definidas. Por exemplo, a definição de um ecrã da aplicação é feita incluindo simplesmente um <div> com um id e com o atributo data-role="page" (em jQuery Mobile, um ecrã é designado por page):

<body>
  <div data-role="page" id="pagina-1">
    <!-- conteúdo do ecrã -->
  </div>
</body>

Apesar de não ser absolutamente obrigatório, a framework jQuery Mobile é orientada principalmente para aplicações de página única (Single Page Applications - SPA), pelo que uma aplicação com dois ecrãs definiria simplesmente dois <div> como descendentes do <body> do documento HTML:

<body>
  <div data-role="page" id="pagina-1">
    <!-- conteúdo do primeiro ecrã -->
  </div>
  
  <div data-role="page" id="pagina-2">
    <!-- conteúdo do segundo ecrã -->
  </div>
</body>

Configuração

A configuração da framework no projecto Web passa simplesmente pela inclusão dos ficheiros CSS e JavaScript (as versões poderão variar):

<link rel="stylesheet"
href="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.css" />
<script src="http://code.jquery.com/jquery-1.11.1.min.js"></script>
<script src="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.js"></script>

De notar que a jQuery Mobile depende da biblioteca jQuery pelo que é importante incluir primeiro o ficheiro JavaScript jQuery e só depois o ficheiro jQuery Mobile.

Igualmente importante é a inclusão de um elemento <meta> no cabeçalho documento HTML para garantir que a aplicação é mostrada correctamente num dispositivo móvel:

<meta name="viewport" content="width=device-width, initial-scale=1">

Principais componentes

Páginas

Uma página é o componente básico em jQuery Mobile que representa um ecrã da aplicação. Todos os outros componentes que iremos ver são inseridos dentro de uma página. Em termos de estrutura HTML, uma página é definida através de um <div> com um atributo data-role="page". É importante atribuir também um “id” ao <div>, uma vez que este será a forma de nos referirmos à página para, por exemplo, permitir navegação entre ecrãs da aplicação. O conteúdo visível da página deve ser colocado dentro de um <div> com role="main" e class="ui-content". O Exemplo 1 demonstra uma aplicação com apenas duas páginas.

See the Pen jQuery Mobile - páginas by Jorge C. S. Cardoso (@jorgecardoso) on CodePen.

Exemplo 1 – Páginas. Código completo: https://goo.gl/Byg1jc | Ver no dispositivo móvel: https://goo.gl/Y9Tfgk.

De notar que ao “executar” a aplicação, apenas vemos o conteúdo da primeira página. Para vermos a segunda página temos de adicionar alguma forma de navegação entre ecrãs da nossa aplicação. Uma forma de permitir navegação entre ecrãs é criando simplesmente links internos tal como faríamos numa página web tradicional, usando os ids como URL internos: <a href="#pagina-2">Link para página 1</a>. O Exemplo 2 ilustra esta ideia, permitindo-nos navegar entre os dois ecrãs, pressionando nos links respectivos.

See the Pen jQuery Mobile - páginas navegação com links by Jorge C. S. Cardoso (@jorgecardoso) on CodePen.

Exemplo 2. Código completo: https://goo.gl/N9YBZL | Ver no dispositivo móvel: https://goo.gl/7WMS62.

Numa aplicação móvel, no entanto, não é usual pressionarmos em links para navegar - tipicamente, usamos botões.

Botões

A criação de botões ilustra bem a vantagem da utilização da framework jQuery Mobile: para criarmos um botão apenas temos de adicionar um atributo ao elemento <a> que usámos para criar a navegação entre ecrãs – o atributo data-role="button". Isto cria um botão que ocupa toda a largura do ecrã, como se pode ver no “Botão 1” do Exemplo 3. Se quisermos que o botão ocupe apenas a largura necessária para mostrar o texto no interior, podemos acrescentar o atributo data-inline="true" (“Botão 2”). Podemos também acrescentar um ícone ao botão indicando o nome do ícone no atributo data-icon (“Botão 3”) – a lista de ícones pré-definidos pode ser consultada em https://api.jquerymobile.com/icons/. O Exemplo 3 mostra alguns dos tipos de botões que podemos criar desta forma. De notar que estes atributos data-* são na verdade usados pela framework jQuery Mobile para determinar que classes CSS irá atribuir ao elemento. O último botão no Exemplo 3 mostra como podemos obter o mesmo efeito indicando directamente as classes CSS a usar (“Botão 4”).

See the Pen jQuery Mobile - botões by Jorge C. S. Cardoso (@jorgecardoso) on CodePen.

Exemplo 3 - Botões. Código completo: https://goo.gl/NLvn7M | Ver no dispositivo móvel: https://goo.gl/ZY4MPs.

Cabeçalho e rodapé

É comum as aplicações móveis terem um cabeçalho e/ou um rodapé com o logótipo da aplicação e com botões de navegação. Estes elementos podem ser incluídos num ecrã da aplicação jQuery Mobile inserindo um elemento <div> com data-role="header" (ou data-role="footer") dentro da página correspondente ao ecrã. Por exemplo, para adicionar um cabeçalho e rodapé à nossa primeira página teríamos o código do Exemplo 4. (Como normalmente queremos que o cabeçalho e rodapé fiquem sempre no topo e fundo da janela da aplicação devemos usar também o atributo data-position="fixed", caso contrário ficarão no fluxo de conteúdo da página e sujeitos ao scroll.)

See the Pen jQuery Mobile - cabeçalho e rodapé by Jorge C. S. Cardoso (@jorgecardoso) on CodePen.

Exemplo 4 – Cabeçalho e rodapé. Código completo: https://goo.gl/r7BVp7 | Ver no dispositivo móvel: https://goo.gl/AjavMT.

De notar que o código do cabeçalho e do rodapé tem de ser repetido em cada página em que pretendamos que estejam presentes, caso contrário alguns ecrãs da nossa aplicação apresentarão cabeçalho/rodapé e outros não.

Botões e navegação no cabeçalho ou rodapé

Os cabeçalhos ou rodapés das aplicações móveis servem, muitas vezes, como ponto de acesso a funcionalidades da aplicação ou como forma de navegação entre os vários ecrãs. Se quisermos incluir botões no cabeçalho ou rodapé podemos fazê-lo através de um elemento <div> com data-role="controlgroup" dentro do elemento que representa o cabeçalho. Este elemento permite-nos incluir vários botões e controlar o seu posicionamento no cabeçalho ou rodapé. O Exemplo 5 mostra o código a incluir no cabeçalho para criar um ecrã com dois botões no cabeçalho. No elemento que define o controlgroup, de notar a utilização dos atributos data-type="horizontal" para garantir que os botões são dispostos horizontalmente e da classe CSS ui-btn-right para alinhar os botões à direita (podiamos obviamente usar ui-btn-left para alinhar à esquerda). De notar também o uso do atributo data-iconpos="notext" nos botões para esconder o texto, ficando apenas o ícone do botão.

See the Pen jQuery Mobile - botões no cabeçalho by Jorge C. S. Cardoso (@jorgecardoso) on CodePen.

Exemplo 5 – Botões no cabeçalho. Código completo: https://goo.gl/8LFCd5 | Ver no dispositivo móvel: https://goo.gl/hgoC8W.

Quando o objectivo é criar uma área de navegação entre páginas no cabeçalho ou rodapé, existe um componente especializado – a barra de navegação. A inclusão de uma barra de navegação pode ser feita no cabeçalho ou rodapé (ou até no corpo da página). No Exemplo 6, a inclusão é feita no rodapé. Uma barra de navegação é simplesmente um elemento <div> com data-role="navbar". No interior do <div> inserimos uma lista não ordenada com um link (elemento âncora <a>) em cada item. Automaticamente, os links são convertidos em botões (neste caso não é necessário adicionar o atributo data-role="button"). Para indicar que um botão está activo (ou seja que o ecrã visível corresponde a um determinado botão na barra de navegação) podemos usar a classe CSS ui-btn-active. Note-se que no Exemplo 6 o rodapé apenas contém a navegação. No entanto, é possível combinar a barra de navegação com outros elementos.

See the Pen jQuery Mobile - navbar no rodapé by Jorge C. S. Cardoso (@jorgecardoso) on CodePen.

Exemplo 6 – Barra de navegação. Código completo: https://goo.gl/rLoJVw | Ver no dispositivo móvel: https://goo.gl/c23dCK.

Gaveta de navegação

A barra de navegação pode ser usada quando o número de elementos de navegação é relativamente reduzido. Para além disso, os botões na barra de navegação consomem espaço que pode ser importante para mostrar conteúdo mais útil. Uma alternativa à barra de navegação (ou um complemento) é a gaveta de navegação. A gaveta de navegação é um painel que podemos abrir ou fechar pressionando num botão (tipicamente o botão é apresentado no cabeçalho). Para criarmos uma gaveta de navegação para uma determinada página adicionamos um elemento <div> com data-role="panel" e com um id. Para listar as opções de navegação no interior da gaveta usa-se tipicamente uma listview (uma lista não ordenada com data-role="listview") com links em cada item. Para possibilitar a abertura da gaveta de navegação devemos incluir um controlgroup no cabeçalho com um botão que refira o id que escolhemos para o painel que representa a gaveta de navegação. O Exemplo 7 mostra o código completo de uma página com uma gaveta de navegação. O atributo data-display no painel que representa a gaveta de navegação permite-nos configurar a forma como a gaveta surge ao utilizador (“overlay” é a forma mais comum, em que a gaveta é puxada para cima do conteúdo do ecrã; “reveal” cria uma gaveta fixa e é o conteúdo do ecrã que se move para mostrar a gaveta por baixo; “push” cria uma gaveta ao lado do conteúdo do ecrã e ambos de movem para mostrar ou esconder a gaveta.)

See the Pen jQuery Mobile - gaveta de navegação by Jorge C. S. Cardoso (@jorgecardoso) on CodePen.

Exemplo 7 – Gaveta de navegação. Código completo: https://goo.gl/P9dEHZ | Ver no dispositivo móvel: https://goo.gl/Vg99iY.

De notar que a gaveta de navegação tem de ser incluída em cada página (com um id diferente para cada página). É possível evitar este tipo de repetição, mas neste momento, isso obriga ao uso de algum código JavaScript, por isso não abordo essa forma de inclusão de gavetas de navegação.

Conclusão

Há, obviamente, muito mais a dizer sobre jQuery Mobile, não apenas sobre componentes de interface gráfica disponíveis, mas também sobre a API JavaScript que nos permite instanciar e controlar programaticamente os componentes de interface gráfica, e também sobre a possibilidade de customização da aparência dos componentes. Com esta introdução, o leitor terá ficado com uma ideia das possibilidades que a framework jQuery Mobile nos dá, da sua estrutura, e dos principais componentes de interface gráfica disponíveis.

Leap Motion for 2D Pointing Tasks

Thu, 07 May 2015 00:00:00 +0000

This post is a summary of a study we did to evaluate the Leap Motion device for two-dimensional pointing and selecting tasks. The study was implemented by Manuel Seixas during his MSc degree.

Original papers

Seixas, M. C. B., Cardoso, J. C. S., & Dias, M. T. G. (2015). The Leap Motion Movement for 2D Pointing Tasks - Characterisation and Comparison to Other Devices. In Proceedings of the 5th International Conference on Pervasive and Embedded Computing and Communication Systems (pp. 15–24). SCITEPRESS Science and Technology Publications, Lda. doi:10.5220/0005206100150024 https://www.mendeley.com/research/leap-motion-movement-2d-pointing-tasks-characterisation-comparison-other-devices-1/

Seixas, M. C. B., Cardoso, J. C. S., & Dias, M. T. G. (2015). One Hand or Two Hands? 2D Selection Tasks With the Leap Motion Device. In ACHI 2015 : The Eighth International Conference on Advances in Computer-Human Interactions (pp. 33–38). Lisbon, Portugal, Portugal: IARIA. Retrieved from http://www.thinkmind.org/index.php?view=article&articleid=achi_2015_2_20_20020

Introduction

The Leap Motion controller (Figure 1) is a new 3D sensing device for hand gesture interaction with a computer. It is capable of sensing the position and orientation of the fingers of the hands, as well as the palm orientation and curvature. The LM is a small device that sits on top of the computer desk and is operated by positioning the hands over the device. The controller can be used to point to a computer screen with a finger or with a tool (a pen or pencil, for example), or perform other hand gestures.

Figure 1. The Leap Motion controller device.

Although not meant to replace the mouse, many of the interactions with the LM involve pointing and selecting targets on a computer screen. Many applications in the Leap Motion App Store are OS control apps meant to give users various degrees of control over the computer, from selecting and launching predefined applications and settings to scrolling content on webpages. Some applications even emulate the mouse, allowing cursor control and mouse actions. Most applications that take advantage of the LM device still require users to perform typical WIMP tasks at some point (in many cases giving users the option of using the mouse or the LM device). For example, in many games users still need to select options and activate buttons; some software for surgery rooms also provides cursor control for specific functions.

If we assume that the LM device gains commercial traction and becomes embedded in additional laptop computers and desktop keyboards (it is already embedded in one HP laptop and keyboard), we must also assume that it will become an additional alternative to typical WIMP tasks. In a situation where the user is operating the LM device in a specific task, it may be faster to perform a WIMP task also with the LM, instead of moving the hand to operate the mouse.

However, up until now, there have been no studies about the performance of the LM device for 2D pointing tasks.

Experiments

We performed two experiments with the LM device in a multi-directional selection test. In this multi-directional selection test, participants are required to move the cursor across a circle to sequentially numbered targets (see Figure 2). The targets are equally spaced around the circumference of the circle and the sequence of targets to select is such that the movements are nearly equal to the diameter of the circle.

Figure 2. The Multi-directional tapping test.

In the first experiment, we compared the LM device with a mouse and a touchpad. To select the targets with the LM, we used the Touchless software for Mac, which emulates the mouse.

In the second experiment, we compared two different selection gestures with the LM: the screen tap (the same used in the first experiment), and the hand grab gesture (Figure 3). The hand grab uses two hands for interaction. The dominant hand performs the cursor movement, and the auxiliary hand performs the “click”. In this experiment, we also compared both gestures to the mouse, to have a baseline.

Figure 3. Gestures used in the second experiment.

Results

In both experiments, we calculated various standard metrics (speed, error rate, throughput) as well as other metrics proposed by MacKenzie et al.. In this post, we only show the standard metrics.

The first experiment made it obvious that the LM device is not a match for the mouse or touchpad. Figures 4 and 5 show the movement time, throughput, and error rate, for the LM, mouse, and touchpad. It is obvious from these figures that participants are a lot slower when using the LM (it takes more than twice the time of the mouse). Consequently, the throughput for the LM is a lot lower. We also observed more errors (clicks outside the target) with the LM than with any of the other devices.

Figure 4. Movement time - first experiment.

Figure 5. Throughput and Error rate - first experiment.

The second experiment confirmed the results that compare the LM to the mouse, but it also showed that performing the selection gesture with another hand improves the selection time. We can see in Figures 6 and 7 that the time to select is a bit lower when using the hand grab gesture. However, probably due to our implementation of the gesture, it generated more errors than the simpler screen tap gesture (Figure 7).

Figure 6. Movement time - second experiment.

Figure 7. Throughput and Error rate - second experiment.

Posture

During the experiments, we also analysed the participants’ postures’ and asked them how comfortable the devices were. One of the main problems with the LM was that participants would get tired very easily. When interacting with a desktop, the LM is has the same problem as vertical touch-screens: the gorilla arm effect. Users have to keep their arm extended to operate the device which, after some time, causes the arm to feel very heavy and leads to less accuracy.

Figure 8. Gorilla arm effect.

Conclusion

For desktop 2d point and select tasks, the LM performs poorly.
The LM causes physical fatigue very quickly in this setting.
A gesture with two hands performs a bit better than using only one hand to point and select in terms of throughput.
We evaluated the LM in a very particular setting. This does not mean the device is not good for many other tasks!

Tangible Interfaces Projects - 2014

Thu, 10 Jul 2014 00:00:00 +0000

This is a selection of the best projects made by the students of the Tangible Interfaces Design course of the Sound and Image master program (Sound specialization). http://artes.porto.ucp.pt/pt/central-oferta-formativa/mestrado-em-som-e-imagem

Pinheiro 2.0

Developed by Tiago Tobias Fernandes, Pinheiro 2.0 is an audiovisual interactive installation. Its main purpose is to show the “Pinheiro” event: an event that is part of the Festas Nicolinas of the city of Guimarães. The installation reproduces the “Toques Nicolinos” - percusive rithms that are characteristic of the event played with bombos and caixas. The installation uses the KarmetiK NotomotoN robot to play the instruments.

Cubesic

Developed by Rita Gradim, Cubesic (cube + music) is a tangible interface for a music player. It allows the definition of playlists based on the pattern of a Rubik’s cube. Each color is mapped to a music genre, and each position is mapped to a decade. The system connects to iTunes to play the defined pattern. The system was implemented using computer vision techniques in Max/Jitter to recognize the color pattern on the Rubik’s cube and translate those colors into iTunes commands via applescript.

Reactable Composer

Developed by Luís Wood Faulhaber, the Reactable Composer is a tangible interface that allows simple composition of musical notes and chords by arranging physical objects on a table. The system uses the Reactable core hardware and reacTIVision.

HandyDrum

Developed by Henrique Correia e Silva, HandyDrum is an button interface for playing the drums, using the KarmetiK NotomotoN robot.

Computational Design Projects - 2014

Sat, 17 May 2014 00:00:00 +0000

This is a selection of the best projects made by the students of the Generative Design course of the Digital Design Post-graduation program. Although the course is called Generative Design, we actually develop Computational Design projects… http://artes.ucp.pt/designdigital/

Vestir a Calçada (Put on the “Calçada”)

This is a tool that is meant to be used by customers to personalize their t-shirt with graphics inspired in the Portuguese “Calçada.

Texturizer

This is a tool meant to help 3d modelers to rapidly create geometric patterns to apply as textures to their 3d objects.

Mustech

This is an installation where you can try out different mustaches in a funny way.

A Viagem - Interactive Video Installation

Sat, 07 Dec 2013 00:00:00 +0000

“A Viagem” (“The Trip”) is an interactive video installation by the VOID group (Carlos Sena Caires + Jorge Cardoso) initially developed for the Procesalia exhibition.

The installation

“A Viagem” is an interactive video installation (see Figure 1) composed of a table with a rounded screen at the center and a toy train track around it. The screen is used to display a video that tells the story of the first train ride of a young child, narrated by the child itself.

The interaction is accomplished by simply moving the toy train in its track. The movement of the train is sensed and a video segment is played. The length of the video segment depends on the distance travelled by the train.

Figure 1. Photo of the installation.

Procesália 2013

This interactive installation was developed originally for the Procesalia 2013 exhibition. Procesalia 2013 was organized by the University of Santiago de Compostela, and curated by Monse Cea. The exhibition is being held at the church of USC, and it features the work of 19 artists. In Figure 2 you can see our installation at the USC church.

Figure 2. Photo of the installation at the USC church.

Implementation

We wanted to create a “magical” interface were it would not be obvious how the computer system sensed the train’s movement. For this we used several hall effect sensors connected to an Arduino and we put small magnets underneath the toy train carriages. In Figure 3 you can see how the sensors were mounted underneath the train tracks. The main logic of the system was then programmed in Max/Jitter to select and play the various video segments.

Figure 3. Sensors used in the installation.

Face projection mapping for Festival IN

Sat, 23 Nov 2013 00:00:00 +0000

The School of Arts was present at the Festival IN in Lisbon, last week. Besides having a regular stand, our school participated with various interactive projects. One of those projects was a giant projection made from the faces of the visitors.

The installation

The installation has two main points: one in which a computer detects and takes a screenshot of your face (and creates a depth map), and another where the recorded faces are projected onto a giant sculpture. Figure 1 shows one early sketch depicting the general interaction process.

Figure 1. General interaction process.

We wanted to project, not just a photo of the person, but rather some representations of the face, including a 3D model, a Voronoi representation, and some simple blurred images. In Figure 2, you can see a photo of how the final setup of installed at Festival IN, including one of the representations that appear on the large scale projection - a 3D model of the captured face.

Figure 2. Photo of final installation.

Interaction

Interaction with this installation is pretty simple since you only need to stand in front of the kiosk. Figure 3 shows a first sketch of the details for the interaction.

Figure 3. Interaction process detail.

The system only takes a screenshot of your face if you get at a certain distance from the kiosk. This ensures that people are not caught off-guard, and it also allowed us to get better framed face pictures. The final system included an extra cue to help users get at the current distance to the kiosk. This is shown in Figure 4.

Figure 4. Distance cue (distance value was not present in the final version).

Implementation

The installation uses a Kinect for getting and RGB image as well as the depth data that is used to create the 3D model of the face. It is also used to determine the distance at which a person is standing in front of the kiosk. It is all done in Processing using toxiclibs for the 3D models and Voronoi generation.