Software Studies

new video of Cultural Analytics software @HIPerSpace (287 megapixels)

2009-07-09T22:04:00.000-07:00

We have created a new video which showcases some of the capabilities of Cultural Analytics software running on HIPerSpace (a supervisualization system with the resolution of 35,840 x 8,000 pixels; 286,720,000 pixels total.)

The software is being developed by Software Studies Initiative and GRAVITY lab (Graphics, Visualization and Virtual Reality Laboratory) at Calit2. The development is supported by Interdisciplinary Collaboratory Grant from UCSD Chancellor office (2008-2010).

At present software allows a user to load up to 4000 images of ANY size on the HIPerSpace wall along with accompanying metadata (which can include manually entered annotations, automatic measurements of visual properties, and other information). The metadata format is a simple tab-delimited text file (so it can be prepared in Excel etc.) Using our interactive software, a user can sort image set and also make X, Y plots of any sets of metadata types. The plots can include the actual images placed over the plotted points; the size of these images can be changed at any time interactively by a user.

Because HIPerSpace is powered by a cluster of PCs, it allows for interactive performance impossible with current desktop computers or the web. Thousands of images can be scaled and graphed instantly. Therefore, HIPerSpace is a perfect platform to develop techniques for software-supported cultural analysis of large media sets.

The N^3 Report

2009-06-30T10:12:00.000-07:00

Researchers:
Andrew Wilson | PhD student, Art History, UCSD
Andy Rice | PhD student, Communication, UCSD
Lara Bullock | PhD student, Art History, UCSD
Chris Head | MMFA student, Visual Art, UCSD
Tara Zepel | PhD student, Art History, UCSD

The NBC Nightly News is an American institution that has been broadcast nightly in its current format since August 1, 1970. Arguably, the Nightly News, alongside newspapers, has served as America's primary source of information dissemination over the past 50 years. Over this time, changes in technology, visual culture, and the market dynamics for news shows have led certain aspects of nightly news production to evolve in subtle ways.

The N^3 Report presents a meta-broadcast of the NBC Nightly News (1980-2008) unpacked by cultural analytics. Through techniques examining visual characteristics and technological shifts that both are and are not obvious to the human eye, the N^3 Report performs what could be called "producer measurement systems"* in search of visual patterns, trends, or variations and asks what defines the "look" of the news institution over the past 50 years. How and when did aesthetic changes take place? And what might be the "look" of the future?

* a reverse of audience measure systems such as Nielson Ratings

Introductory segments from each year within the specified range were collected from Vanderbilt's Television News Archive. Each segment, beginning with the program's opening and ending with the first stationary shot of the anchor, was analyzed for intensity, color distribution, graphic content and temporal patterns. We report the following:

1) An increase in intensity and brightness by year.

AVG Intensity Mean Brightness (NBC 1980-2008): This graph presents the average measurement of brightness for all shots in each of the NBC Nightly News segments measured. The data reveals a trend of increasing brightness over time.

AVG Mean/Median Gray Value (NBC 1980-2008); An average composite image, which essentially merges the moving images footage into a single frame, was created for each of the introductory segments from NBC Nightly News episodes ranging from 1980 to 2008. This graph presents the mean and median gray values in these composite images. The data indicates an increase in average brightness (on a 0-255 scale) over time.

2) A graphic inundation throughout the nineties and into the millennium that appears to have subsided in the last two years in favor of a renewed and stable focus on the anchor.

Each image in this visualization is created by adding all frames in a given news video on top of each other. Each image on the top row represents one year of the 1980s, the middle row corresponds to the 1990s, and the bottom the 2000s. Note how in the middle of the 1990s row, the visage of the anchorman is replaced by an image dominated by graphics and motion. The focus on the anchorman returns in 2007 and 2008.

3) A visualization of temporal patterns using montages of segmented frames.

Hi-Res Image (7MB)
Each block within the larger montage is an introductory segment montage sampled at 30 frames per
second. Each row shows a 5 year span with two shows per year.

Row 1+2 = 1980s
Row 3+4 = 1990s
Row 5+6 = 2000s

4) A high speed video montage of NBC Nightly News introductions over the past 30 years.

Our investigation explores potential correlations, which may then be further developed through further research. The goal of this project was to take a mainstream source of information, the NBC Nightly News, and analyze the aesthetic techniques used in tandem with the message and reputation that is attached to this specific news program. It is also useful for viewing cultural shifts in the news over time. We focused on introductions to the Nightly News broadcasts because of the relative level of producer control and the concise presentation of what is to follow. By using visualizations of the visual aspects of introductory segments from 1980-2008, we hope to broaden the understanding of this cultural mainstay in American culture and open new questions that, without such techniques, may be overlooked or not realized, but that could contribute greatly to the growth of visual humanities and cultural studies.

Cultural Analytics

2009-06-20T18:00:00.000-07:00

Interface design for Cultural Analytics research environment.

The explosive growth of cultural content on the web including social media and the digitization efforts by museums, libraries, and companies since the 1990s make possible fundamentally new paradigm for the study of both contemporary and historical cultures. We can use computer-based techniques for quantitative analysis and interactive visualization already commonly employed in sciences to begin analyzing patterns in massive cultural data sets. To make an analogy with "visual analytics," "business analytics," and "web analytics," we call this new paradigm cultural analytics.

We believe that a systematic use of large-scale computational analysis and interactive visualization of cultural data sets and data streams will become a major trend in cultural criticism and culture industries in the coming decades. What will happen when humanists start using interactive visualizations as a standard tool in their work, the way many scientists do already? If slides made possible art history, and if a movie projector and video recorder enabled film studies, what new cultural disciplines may emerge out of the use of interactive visualization and data analysis of large cultural data sets?

The idea of Cultural Analytics was first presented by Lev Manovich in 2005. Software Studies Initiative founded at Calit2 in 2007 made possible to turn this vision into a research program. By drawing on the cutting-edge cyberinfrastructure and visualization research at Calit2 as well as world reputation of UCSD in digital arts and theory, we are able to develop a unique research agenda which complements other projects in digital humanities and "cyberscholarship":

while most projects in digital humanities deal with text, we focus on automatic analysis of visual and media cultures and artifacts: video games, visual art, media design, cinema, animation, AMV, machinema, photography, etc.;
we place emphasis on interactive visualization of large cultural data sets (as opposed to only analysis);
we are developing techniques for analysis and visualization of born digital content such as video games, web sites and social media.

References:

Software Studies Initiative. Cultural Analytics: vision (last update: 06/2009). [key 26.3 MB] [ppt 12.8 MB]

Software Studies Initiative. Cultural Analytics: case studies (last update: 06/2009). [key 32.4 MB]. [ppt 8.2 MB]

Lev Manovich (with contribution by Noah Wardrip-Fruin). Cultural Analytics: white paper (5/2007; latest update 11/2008): [doc 2.4 MB]

Lev Manovich. "Cultural Analytics: Visualing Cultural Patterns in the Era of “More Media.”(published in DOMUS, spring 2009.) [doc 44KB]

Lev Manovich. "How to Follow Global Digital Cultures, or Cultural Analytics for Beginners." Deep Search, ed. Felix Stalder and Konrad Becker. Transaction Publishers ( English version) and Studienverlag (German version), in press. [ doc 92 KB]

So Yamaoka, Lev Manovich, Jeremy Douglass, Falko Kuester. "Cultural Analytics on Ultra High-Resolution Displays." Paper submitted to ACM Multimedia 2009 conference.

Lev Manovich and Jeremy Douglass. "Visualizing Temporal Patterns in Visual and Interactive Media." Forthcoming in Visualising the 21st Century, ed. Oliver Grau. MIT Press. [pdf 5 MB]

PUBLICATIONS

2009-06-18T13:54:00.000-07:00

Books

Lev Manovich. Software Takes Command. Forthcoming from The MIT Press in 2010 in Software Studies Series . Italian translation forthcoming in 2009.

Noah Wardrip-Fruin. Expressive Processing: Digital Fictions, Computer Games, and Software Studies. The MIT Press: forthcoming September 2009.

Articles, book chapters and white papers

Jeremy Douglass. “Computer Visions of Computer Games: analysis and visualization of play recordings.” Workshop on Media Arts, Science, and Technology (MAST) 2009: The Future of Interactive Media. UC Santa Barbara, January 2009. [slides]

Jeremy Douglass. "Playpower: Radically Affordable Computer-Aided Learning with $12 TV-Computers.” co-authored with Derek Lomas and Daniel Rehn. Meaningful Play 2008. Michigan State U., October 2008.

William Huber. "Epic spatialities: the production of space in Final Fantasy games." Book chapter in Wardrip-Fruin and Harrigan (eds.) Third Person: Authoring and Exploring Vast Narratives. MIT Press, 2009

Lev Manovich. "Cultural Analytics: Visualising Cultural Patterns in the Era of “More Media,” Domus, (Milan), March 2009

Lev Manovich. "How to Follow Global Digital Cultures, or Cultural Analytics for Beginners," Deep Search, ed. Felix Stalder and Konrad Becker. Transaction Publishers ( English version) and Studienverlag (German version), in press.

Lev Manovich. White paper: Cultural Analytics: Analysis and Visualizations of Large Cultural Data Sets, May 2007. With contributions from Noah Wardrip-Fruin.

Lev Manovich and Jeremy Douglass. "Visualizing Temporal Patterns in Visual and Interactive Media." Forthcoming in Visualising the 21st Century, ed. Oliver Grau. MIT Press.

Lev Manovich, Jeremy Douglass, and William Huber(In preparation) "Cultural Analytics: Methods and Techniques." For submission to Digital Humanities Quarterly.

So Yamaoka, Lev Manovich, Jeremy Douglass, and Falko Kuester. "Cultural Analytics on Ultra High-Resolution Displays." Paper submitted to ACM Multimedia 2009 conference (pending.)

Conference Presentations

Lev Manovich

April 2008 | London Schol of Economics, | London
April 2008 | Royal College of Art (RCA) | London | Lecture
April 2008 | Goldsmiths College | London
April 2008 | London Schol of Economics, Social Study of ICT Workshop (SSIT8) | London | lecture
May 2008 | Department of Design / Media Arts, UCLA | Los Angeles | lecture
May 2008 | “Software Studies” international workshop | University of California – San Diego, | San Diego, California | presentation
May 2008 | “Software Studies” panel, HASTAC II conference, University of California – Irvine | Irvine, California | lecture with Jeremy Douglass using HIperWall
May 2008 | “Software Studies” panel, HASTAC II conference, University of California – Irvine | Irvine, California | Panel organizer, moderator, and speaker
June 2008 | Software Cultures” lecture seriies, University of California – Irvine, June 4 | Irvine, California | lecture
August 2008 | ISEA 2008 (the International Symposium on Electronic Arts | Singapore | Lecture (satellite event at LASALLE College of the Arts)
August 2008 | FILE (Electronic Language International Festival) 2008 | www.file.org.br | Sao Paolo, Brazil via Skype from San Diego | lecture (video)
September 2008 | Ciantec (Annual conference on art and technology, Mackenzie University) | Sao Paolo, Brazil via Skype from San Diego | lecture (video)

Jeremy Douglass

May 2008 | Jeremy Douglass, “Programming Literary Flow.” Panel on Mapping Process in New Media Landscapes. ELO 2008: Visionary Landscapes. | Vancouver Washington | presentation
May 2008 | “Implied Code as Mental Geography.” Panel on ReVisioning Electronic Literature - Origins and Influences. ELO 2008: Visionary Landscapes. | Vancouver, Washington | panelist
May 2008 | Jeremy Douglass, “The LA Flood Project” presented with lead Mark Marino, collaborators Dena, Gutierrez, Hight, and Tao. HASTAC II: Techno-Travels. UC Irvine | poster
May 2008 | Jeremy Douglass, with Lev Manovich. “HIPerWall Demo: Cultural Analytics” HASTAC II. UC Irvine | presentation
May 2008 | Jeremy Douglass, “Visual Rhetoric for Large Displays.” Transcriptions Research Slam. UCSB | presentation
May 2008 | HASTAC II: Techno-Travels | “What is Software Studies?” Panel at HASTAC II: Techno-Travels, UCLA | Los Angeles | panelist
May 2008 | SoftWhere 2008| Software Studies Initiative at Calit2, UC San Diego | San Diego | co-chaired with Lev Manovich and Noah Wardrip-Fruin
May 2008 | “#Include Genre.” | SoftWhere: Software Studies 2008 | Calit2, UC San Diego | presentation
September 2008 | “Topics in Software Studies.” | CIANTEC 2008. Mackenzie University, Brazil | presentation with Cícero Silva (video)
March 2009 | Jeremy Douglass, with Derek Lomas and Daniel Rehn. “Playpower: Designing 8-bit Learning Games for Radically Affordable Computers.” O’Reilly ETech Emerging Technology Conference 2009: Living, Reinvented | San Jose, California | presentation
December 2009 | Digital Arts and Culture 2009 | "Software/Platform Studies" | Irvine, California | track chair

William Huber

May 2008 | Softwhere: Software Studies 2008 | "Soft authorship" | Calit2, UC San Diego | presentation
April 2009 | Thinking after Dark: Horror Videogames | "Catch and release: the ludological dynamics of horror videogames" | University of Montreal | presentation
September 2009 | DIGRA 2009 | Panel on game criticism | London | panelist
April 2010 | SCMS Conference 2010 | "Fatal Frames" | Tokyo/Los Angeles | panel chair

Visual Analysis Toolkit

2009-06-18T12:54:00.000-07:00

[Tools and source code can be downloaded from the Software Studies Initiative Google Code site. We will document these tools more exhaustively in the near future.]

CAmacros: ImageJ macros for the analysis of images and video

CAjava: Java programs for image and video feature recognition

CAscript: Python script for a single project

CAbatch: workflow management tools for very large and mixed data sets

HIPerSpace visualizer: software for interactive exploration and analysis of collections of images and videos on HIPerSpace

ELogger: custom keylogger application for recording high-speed keyboard events running in emulation.

FSorter: sorting of very large image data using image statistics and metadata (UNIX script / OS X Finder Plugin).

Vilinx: automated image processing for video game play (UNIX script)

Toolkit para Análise Visual

2009-06-17T14:32:00.000-07:00

[Ferramentas e código fonte podem ser baixados do Software Studies Initiative Google Code Site. Iniciaremos uma documentação mais exaustiva dessas ferramentas em breve.]

CAmacros: ImageJ macros para análise de imagens e vídeo

CAjava: programas em Java para reconhecimento de imagem e vídeo

CAscript: Script Python para desenvolvimento de projeto

CAbatch: ferramenta de administração do fluxo de trabalho para grandes conjuntos e dados mixados.

HIPerSpace visualizer: software para exploração interativa e análise de conjuntos de dados e vídeos em HIPerSpaces

ELogger: aplicação simples de keylogger para gravação de eventos de alta velocidade do teclado que rodem emulados.

FSorter: classificação de grandes quantidades de dados de imagens utilizando estatística de imagen e metadados (UNIX script / OS X Finder Plugin).

Vilinx: processamento de imagem automatizado para jogos de videogame (UNIX script).

Cultural Analytics workshop @ FILE 2009

2009-06-03T07:57:00.000-07:00

CULTURAL ANALYTICS workshop
Lev Manovich (UCSD/Software Studies)
July 30th

9am - 1pm

where: FILE, FIESP building

Avenida Paulista, 1313, São Paulo, Brazil

Digitization of art and media collections by museums and libraries and
and the explosive growth of newly available cultural content on the
web have created unique opportunities for studying cultural processes
in new ways. If humanities have typically relied on the analysis of a
small number of cultural objects, we can now create interactive
visualizations and dynamic maps of large cultural data sets to reveal
cultural patterns. The workshop will introduce and demonstrate the
tools which participants can use to collect, analyze, and visualize
cultural data. The particular focus will be on the analysis of visual
media and born digital culture (visual art, photography, cinema,
motion graphics, video games).

++ subscriptions (read carefully the terms and conditions)

++ support: Graduate Studies in Education, Art and History of Culture (MFA and Ph.D.), Mackenzie University

http://www.mackenzie.com.br/educacao_arte_historia_cultura1.html

ANALÍTICA CULTURAL
com Lev Manovich (UCSD/Software Studies)
30/07 | 9h - 13h

Local: FIESP, Avenida Paulista, 1313, São Paulo, Brasil

A digitalização de coleções de arte e mídia realizada por museus, bibliotecas e o explosivo crescimento de conteúdos culturais novos na web criou uma oportunidade única para estudarmos os processos culturais de outras maneiras. Se as humanidades estavam baseadas tipicamente na análise de um pequeno número de objetos culturais, podemos agora criar mapas dinâmicos e interativos de grandes bancos de dados para revelar padrões culturais. O workshop introduzirá e demonstrará as ferramentas com as quais os participantes poderão coletar, analisar e visualizar dados culturais. O foco particular do workshop estará na análise das mídias visuais nascidas da cultura digital (artes visuais, fotografia, cinema, animações e videogames).

++ inscreva-se (leia atentamente os termos e condições de participação)

apoio:

Programa de Pós-graduação em Educação, Arte e História da Cultura | Mestrado e Doutorado

Universidade Mackenzie

http://www.mackenzie.com.br/educacao_arte_historia_cultura1.html

Analítica Cultural

2009-05-12T14:53:00.000-07:00

Design da interface do ambiente do projeto Analítica Cultural.

A Analítica Cultural refere-se ao uso da análise de dados e visualizações interativas de amplos conjuntos de dados culturais no contexto das humanidades. A idéia foi primeiramente apresentada por Lev Manovich em 2005. O grupo de Software Studies fundado no CALIT2 em 2007 criou a possibiliade de tornar essa visão em um projeto de pesquisa. Em conjunto com outros docentes e estudantes da UCSD, estamos conduzindo estudos de diversos conjuntos de dados culturais e desenvolvendo técnicas gerais e ferramentas para a Analítica Cultural que podem ser utilizadas por outros pesquisadores nas humanidades digitais e nas ciências sociais.

Analítica Cultural: visão (última atualização: 05/2009):

key 53.3 MB
ppt 9.6. MB

Analítica Cultural: estudos de caso (última atualização: 05/2009):
key 18.4 MB
ppt 17.2 MB

Analítica Cultural: paper (5/2007; última atualização 11/2008):
doc 2.4 MB

Publicações:

Lev Manovich. "Cultural Analytics: Visualing Cultural Patterns in the Era of “More Media.” (publicado em DOMUS, primavera de 2009.)

Lev Manovich. "How to Follow Global Digital Cultures, or Cultural Analytics for Beginners." Deep Search, ed. Felix Stalder e Konrad Becker. Transaction Publishers (versão em inglês) e Studienverlag
(versão em alemão), no prelo.

So Yamaoka, Lev Manovich, Jeremy Douglass, Falko Kuester. "Cultural Analytics on Ultra High-Resolution Displays." Artigo submetido à conferência ACM Multimídia.

O explosivo crescimento de conteúdos culturais na web incluindo as mídias sociais e os esforços de digitalização realizados por museus, bibliotecas e empresas desde os anos 1990 tornaram possível um novo paradigma fundamental para o estudo da contemporaneidade e das culturas históricas. Podemos utilizar técnicas computacionais para análise quantitativa e visualização interativa de "dados gigantes" atualmente utilizadas comumente na ciência e nos negócios para começar a analisar padrões em grandes conjuntos de dados culturais. Para construir uma analogia com a "analítica visual", "análise de negócios" e "análise da web", denominamos esse novo paradigma de "analítica cultural".

Acreditamos que o uso sistemático da análise computacional de larga escala e visualizações interativas de conjuntos de dados culturais e de conjuntos de dados se tornarão as principais tendências na área de crítica cultural e nas indústrias culturais nas décadas que estão por vir. O que acontecerá quando pesquisadores das humanidades começarem a utilizar visualizações interativas como uma ferramenta padrão em seus trabalhos, em vez da forma que utilizam atualmente? Se os slides tornaram possível a história da arte e se a projeção e a gravação de um filme permitiram os estudos dos filmes, que novas disciplinas surgirão do uso de visualizações interativas e da análise de grandes conjuntos de dados culturais?

Ao desenvolvermos as técnicas para uma Analítica Cultural Quantitativa (ACQ) utilizamos a ciberinfraestrutura tecnológica de última geração e as pesquisas de visualização no CALIT2, onde nosso laboratório está hospedado. Também tomamos partido do reconhecido status internacional dos renomados programas em artes digitais e teoria digital, com dezenas de professores e centenas de estudantes (da graduação à pós-graduação) trabalhando nestas áreas. Como resultado, estamos habilitados a desenvolver uma agenda de pesquisa única, diferente dos últimos esforços na área emergente das humanidades digitais.

- enquanto a maior parte dos projetos nas humanidades digitais lidam com texto, estamos focados na análise das culturas visuais e dos artefatos da mídia: videogames, artes visuais, design de mídia, cinema, animação, AMV, machinema, fotografia etc.

- nossa ênfase está na visualização interativa de grandes conjuntos de dados culturais (em oposição à análise somente);

- nossa especialidade em cultura digital contemporânea e design nos permite conduzir projetos que analisam essas áreas;

- nosso trabalho de análise de artefatos digitais está baseada em nossa pesquisa teórica no campo dos Software Studies.

(Lev Manovich)

Cultural analytics coverage in Singapore and UAE

2009-04-21T22:21:00.000-07:00

A new article about Cultural Analytics published @ knowledge.smu.edu.sg, April 2, 2009. (Part of Knowledge@Warton network.)

Reprinted:
khaleejtimes.com. (business section of a daily English language newspaper published in Dubai, United Arab Emirates.)

Cultural Analytics: history

2009-03-16T07:39:00.000-07:00

September 2005: IGRID conference at Calit2 which includes demos of EVL LambdaVision display (55 tiled 30-inch LCD screens).

------------------------------------------------------------
November 2005: Lev Manovich submits grant proposal to ACLS:

DATA MINING VISUAL CULTURE

From Summary:

I propose to develop a new approach for the study of visual culture, including art, graphic design, vernacular imagery, photography, cinema, and digital media. The idea is to apply the techniques of computer-based data analysis and data display which already have become routine in the sciences – information visualization, image processing, data mining, data clustering, and others - to the ‘data’ of visual culture, i.e. cultural images.

Similar to the scientists which apply these techniques to massive data sets in order to see new patterns, we can analyze the imagery of whole artistic movements and whole historical periods – for instance, all Dutch seventieth century landscapes, all nineteenth century vernacular photographs available in museum collections, or even – one day - all of twentieth century cinema.

From proposal text:

Scientific communities are spending significant resources today to develop large format displays – such as the EVL LambdaVision display at Calit2 where my new lab is situated... I am convinced that we should follow the practice of scientists to be able to study our own data – cultural images - on very large displays such as LambdaVision. The use of large format displays is especially beneficial when we want to use visualization to look at actual image sets – so that we can visually examine relationships between tens of thousands of images.

-----------------------------------------------------------
April 2007: Calit2 and CRCA provide funding to establish Software Studies Initiative.

-----------------------------------------------------------
May-June 2007: Responding to the challenge by Larry Smarr to develop new applications for HIPerWalll, Manovich and Wardrip-Fruin write Cultural Analytics white paper which extends the ideas of Manovich's 2005 ACLS proposal:

"Can we create quantitative measures of cultural innovation? Can we have a real-time detailed map of global cultural production and consumption? Can we visualize flows of cultural ideas, images, and trends? Can we visually represent how cultural and lifestyle preferences – whether for music, forms, designs, or products – gradually change over time?

Today sciences, business, governments and other agencies rely on computer-based analysis and visualization of large data sets and data flows. They employ statistical data analysis, data mining, information visualization, scientific visualization, visual analytics, and simulation. We believe that it is time that we start applying these techniques to cultural data. The large data sets are already here – the result of the digitization efforts by museums, libraries, and companies over the last ten years (think of book scanning by Google and Amazon) and the explosive growth of newly available cultural content on the web."

"Visualizations should be designed to take full advantage of the largest gigapixel wall-size displays available today – that are being constructed at CALIT2."

-----------------------------------------------------------
April 2008: NEH established Digital Humanities Office and announces Humanities High-Performance Computing (HHPC) grant program:

"Humanities High-Performance Computing (HHPC) refers to the use of high-performance machines for humanities and social science projects. Currently, only a small number of humanities scholars are taking advantage of high-performance computing. But just as the sciences have, over time, begun to tap the enormous potential of HPC, the humanities are beginning to as well. Humanities scholars often deal with large sets of unstructured data. This might take the form of historical newspapers, books, election data, archaeological fragments, audio or video contents, or a host of others. HHPC offers the humanist opportunities to sort through, mine, and better understand and visualize this data."

November 2008: Software Studies Initiative is one of 3 labs awarded HHPC grant

-----------------------------------------------------------
April 2008: a group of Calit2 researchers headed by Lev Manovich receives Interdisciplinary Collaboratory Grant from UCSD Chancellor office to develop Cultural Analytics Research Environment: an open platform for Digital Humanities Research which will support real-time analysis of different types of visual and media data and a variety of visualization and mapping techniques.

The group which submitted the proposal included the following Calit2 researchers:

Lev Manovich (Visual Arts);
Noah Wardrip-Fruin (Communication);
Falko Kuester (Calit2 and Structural Engineering);
Jim Hollan (Cognitive Science).

-----------------------------------------------------------
January 2008: NEH and NSF announce Digging Into Data challenge. Grant amounts: up to 300K USD.

"The Digging into Data Challenge is an international grant competition sponsored by four leading research agencies, the Joint Information Systems Committee (JISC) from the United Kingdom, the National Endowment for the Humanities (NEH) from the United States, the National Science Foundation (NSF) from the United States, and the Social Sciences and Humanities Research Council (SSHRC) from Canada."

"The creation of vast quantities of Internet accessible digital data and the development of techniques for large-scale data analysis and visualization have led to remarkable new discoveries in genetics, astronomy, and other fields, and—importantly—connections between academic disciplinary areas."

"With books, newspapers, journals, films, artworks, and sound recordings being digitized on a massive scale, it is possible to apply data analysis techniques to large collections of diverse cultural heritage resources as well as scientific data. How might these techniques help scholars use these materials to ask new questions about and gain new insights into our world?"

Analítica Cultural: histórico

2009-03-15T08:49:00.000-07:00

Setembro de 2005: Conferência IGRID no Calit2 que incluiu demonstrações da tela EVL Lambda Vision (55 telas de LCD com 30 polegadas posicionadas lado a lado).

------------------------------------------------------------

Novembro de 2005: Lev Manovich aplica para o fundo de pesquisa ACLS:

PROCESSANDO DADOS DA CULTURA VISUAL

Resumo:

Proponho desenvolver uma nova abordagem para o estudo da cultura visual, incluindo arte, design gráfico, imagéticas vernaculares, fotografia, cinema e mídia digital. A idéia é aplicar técnicas de análise de dados computacionais e projeção de dados que já se tornaram rotina nas ciências: processamento de dados, agrupamento seletivo de dados e outros aos "dados" da cultura visual, ou seja, às imagens culturais.

Da mesma forma como cientistas que aplicam essas técnicas a massivas quantidades de dados para observar novos padrões, podemos analisar a imagética de períodos artísticos e históricos por completo: toda a pintura holandesa do século XVII, todas as fotografias vernaculares do século XIX disponibilizadas em coleções de museus ou, algum dia, todo o cinema do século XX.

Texto da proposta:

Comunidades científicas estão gastando quantias significativas no desenvolvimento de dispositivos de visualização de grande escala, como o EVL LambdaVision no Calit2 onde se encontra meu novo laboratório....estou convencido que deveríamos seguir a prática dos cientistas para podermos analisar nossos próprios dados - imagens culturais em telas muito grandes como a Lambda Vision. O uso de telas em grandes formatos é especialmente útil quando queremos utilizar a visualização para observar conjuntos de imagens atuais - podendo dessa forma visualmente observar e examinar relações entre as milhares de imagens.

-----------------------------------------------------------
Abril de 2007: o Calit2 e o CRCA proveem fundos para a criação do Software Studies Initiative

-----------------------------------------------------------
Maio-Junho de 2007: respondendo ao desafio proposto por Larry Smarr para desenvolver novas aplicações para a HIPerWalll, Manovich e Wardrip-Fruin escrevem o artigo Cultural Analytics que amplia as ideias da proposta para o ACLS de 2005:

"Podemos criar medidas quantitativas para a inovação cultural? Podemos ter um mapa em tempo real da produção e do consumo cultural? Podemos visualizar fluxos de ideias culturais, imagens e tendências? Podemos representar visualmente como as preferências culturais e de estilo de vida (música, formas, design ou produtos) gradualmente se modificam ao longo do tempo?

"Atualmente as ciências, negócios, governos e outras agências estão baseadas em análises computacionais e na visualização de grandes quantidades de dados e fluxos de informação. Empregam análise de dados estatísticos, visualização da informação e visualização científica, visualização analítica e simulação. Acreditamos que é tempo de aplicarmos essas técnicas aos dados culturais. Grande parte desse conjunto de dados culturais já está disponível: resultado dos esforços de digitalização realizadas por museus, livrarias e companhias ao longo dos últimoas dez anos (pense na digitalização de livros realizada pelo Google e pela Amazon) e no explosivo crescimento de conteúdos culturais disponibilizados na web."

"As visualizações deveriam ser criadas para tirar o máximo proveito das gigantes telas em escala de gigapixels disponíveis hoje em dia - é isso que vem sendo construído no CALIT2."

-----------------------------------------------------------
Abril de 2008: NEH cria um departamento de Humanidades Digital e anuncia um programa de apoio para Computação de Alta Performance para as Humanidades (Humanities High-Performance Computing -HHPC):

"A Computação de Alta Performance para as Humanidades faz referência ao uso de máquinas de alta performance em projetos nas áreas de ciências sociais e humanidades. Hoje em dia somente um pequeno número de pesquisadores está utilizando e tirando proveito da computação de alta performance. Mas, assim como as ciências tem começado a se utilizar de forma mais intensa ao longo dos anos do potencial da computação de alta performance, as humanidades estão, da mesma forma, também começando esse processo. Pesquisadores das humanidades quase sempre lidam com grandes quantidades de dados sem estrutura. Esses dados podem estar em forma de jornais históricos, livros, dados de eleições, fragmentos arqueológicos, conteúdos de áudio ou vídeo. A bolsa para a Computação de alta performance nas humanidades dá a oportunidade de organizar, analisar e melhor compreender, além de visualizar esses dados."

Novembro de 2008: O Grupo Software Studies Initiative é um dos 3 labs que ganham a bolsa HHPC

-----------------------------------------------------------
Abril de 2008: um grupo de pesquisadores do Calit2, coordenados por Lev Manovich recebem uma bolsa Interdisciplinar da Chancelaria da UCSD para o desenvolvimento do ambiente da pesquisa Analítica Cultural: uma plataforma aberta para a Pesquisa em Humanidades Digitais que apoiará análises em tempo real de variadas formas de mídias visuais e de técnicas de mapeamento.

O grupo inicial é composto por:

Lev Manovich (Artes Visuais);
Noah Wardrip-Fruin (Comunicação);
Falko Kuester (Calit2);
Jim Hollan (Ciências Cognitivas).

-----------------------------------------------------------
Janeiro de 2008: NEH and NSF anunciam a competição Digging Into Data. Bolsas com valores superiores a US$ 300.000,00 (trezentos mil dólares).

"A competição "Mergulhando nos Dados" é um apoio internacional financiado por quatro destacadas agências de pesquisa: o Joint Information Systems Committee (JISC) da Inglaterra, o National Endowment for the Humanities (NEH) dos Estados Unidos, o National Science Foundation (NSF) dos Estados Unidos e o Social Sciences and Humanities Research Council (SSHRC) do Canadá."

"A criação de vastas quantidades de dados acessíveis na Internet e o desenvolvimento de técnicas para análise e visualização de largas escalas de dados nos levaram a descobertas memoráveis no campo da genética, da astronomia, entre outros e, mais importante, à conexão entre áreas acadêmicas distintas."

"Com livros, jornais, revistas científicas, filmes, obras de arte e músicas sendo digitalizadas em escala massiva, é possível aplicar técnicas de análise de dados às vastas coleções de nossas heranças culturais assim como aos dados científicos. Como essas técnicas poderão auxiliar os pesquisadores na formulação de novas questões e possibilitar novos insights sobre nosso mundo?"

Playpower covered in Wired

2009-03-13T13:14:00.000-07:00

Jeremy Douglass (a Post-doctoral researcher with Software Studies) is one of the co-founders of Playpower.org, an an innovative non-profit organization developing learning games for an existing $10 TV-computers which are very popular in India. Software Studies is happy to collaborate with Playpower on their amazing and far-reaching research. In addition to Jeremy, playpower.org founders are Derek Lomas and Daniel Rehn who are both MFA students in Visual Arts Department at UCSD and researchers at Calit2. Their work is getting lots of international media coverage - including this article in Wired blogs:

$12 Computer: Playpower Wants to Save the World 8 Bits at a Time
By Priya Ganapati March 11, 2009 | Wired blogs

Playpower.org's Jeremy Douglass and Derek Lomas speak with Visicalc founder Bob Frankston (center) about their plans to make educational games for 8-bit computers like the one in the foreground. ETECH 2009, San Jose. [Photo by Wired editor Dylan Tweney]

http://www.calit2.net/newsroom/release.php?id=1472

Playpower na Wired

2009-03-12T05:46:00.000-07:00

Jeremy Douglass (pesquisador de Pós-doutorado junto ao grupo de Software Studies) é co-fundador do projeto Playpower.org, uma organização sem fins lucrativos inovadora que desenvolve jogos de aprendizagem para consoles (computadores ligados a uma TV) já existentes de US$ 10,00 (dez dólares), muito populares na Índia. O grupo de Software Studies sente-se honrado em colaborar com o projeto Playpower em seus amplos e ousados objetivos de pesquisa. Além de Jeremy, os fundadores do projeto são Derek Lomas and Daniel Rehn, ambos cursando o MFA em Artes Visuais na UCSD e pesquisadores do CALIT2. O Grupo de Software Studies no Brasil vem também desenvolvendo contéudo para o console, em parceria com Laboratórios e Universidades no país. O trabalho desenvolvido pela equipe do Playpower vem alcançando ampla repercussão internacional, incluindo esse artigo recente publicado pela revista Wired no mês de março de 2009:

$12 Computer: Playpower Wants to Save the World 8 Bits at a Time
By Priya Ganapati March 11, 2009 | Wired blogs

Os fundadores do projeto Playpower.org, Jeremy Douglass e Derek Lomas conversam com o criador do software Visicalc Bob Frankston (centro) sobre seus planos de criar jogos educacionais para computadors de 8-bits. ETECH 2009, San José. [Foto Wired Dylan Tweney]

Veja mais em: (em inglês)
http://www.calit2.net/newsroom/release.php?id=1472

DAC 2009: Track in Software / Platform Studies

2009-03-05T18:43:00.000-08:00

The Software Studies Initiative is soliciting submissions to the "Software / Platform Studies" theme track of Digital Arts and Culture 2009. Please submit proposals no later than the May 1st deadline. Final papers will be due Oct 1 and the conference will be held December 2009 at UC Irvine, California. Here is the theme call:

Software/ Platform Studies

Software is the engine that drives cyberculture, new media, and digital art - a layer of control and communication that permeates contemporary culture. Platforms are the hardware and software relationships that enable and constrain software expressions. To investigate the logics of visualization, simulation, and representation in contemporary digital arts and culture is to engage in Software Studies and Platform Studies. The DAC09 Software and Platform Studies Theme invites submissions that explore digital art and culture through source codes, platform architectures, or similar engagements. Papers and panels exploring the emerging paradigms of critical code studies and the interpretation of algorithms are particularly encouraged.

Theme Leaders:

Jeremy Douglass, PhD. Researcher in Software Studies at UCSD.
jeremydouglass@gmail.com
Noah Wardrip-Fruin,Assistant Professor, Computer Science Department, University of California, Santa Cruz.
nwf@ucsc.edu

DAC is a biannual -- 2005 was Copenhagen, Denmark -- 2007 was Perth, Australia -- and this December 2009 it will be held in Irvine, California, with the conference title "After Media: embodiment and context." Cross-submission to Software / Platform Studies and another theme track of the conference is allowed, including:

A Space-Time of Ubiquity and Embeddedness
After mobile media
Cognition and creativity
Embodiment and performativity
Environment/ sustainability/ climate change
Interdisciplinary pedagogy
Sex and sexuality
Software/ platform studies

For more details on all themes see the full DAC09 call for papers.

DAC 2009: chamada para área de Estudos do Software e Plataformas

2009-03-04T19:55:00.000-08:00

O Grupo de Software Studies solicita a submissão de artigos e painéis para a área dedicada aos Estudos do "Software/Plataformas" no evento Digital Arts and Culture 2009. Por favor envie sua proposta até o dia 01 de maio de 2009. Os artigos completos deverão ser enviados até o dia 01 de outubro e a conferência acontecerá em dezembro de 2009 na Universidade da Califórnia, Irvine. Os temas são:

Estudos do Software/ Plataformas

O software é a máquina que opera a cibercultura, as novas mídias e a arte digital - uma camada de controle e comunicação que permeia a cultura contemporânea. A plataforma é a relação entre o hardware e o software que permite e restringe a expressão do software. Investigar a lógica da visualização, simulação e representação na arte e na cultura digital contemporânea é se envolver com os Estudos do Sofware e das Plataformas. A chamada de artigos para a área de Estudos do Software e Plataformas no DAC 2009 tem como objetivo receber trabalhos que explorem a arte digital e a cultura através dos códigos fontes, das arquiteturas das plataformas ou formas similares. Artigos e painéis explorando os paradigmas emergentes da crítica dos estudos do código e a interpretação dos algoritmos são particularmente encorajados.

Líderes da área:

Jeremy Douglass, PhD. Pesquisador do grupo de Software Studies na UCSD.
jeremydouglass@gmail.com
Noah Wardrip-Fruin, Professor Assistente, Departamento de Ciência da Computação, University of California, Santa Cruz.
nwf@ucsc.edu

O DAC é um encontro bienal - em 2005 aconteceu em Copenhagen, Dinamarca, em 2007 em Perth, Austrália e em dezembro deste ano acontecerá em Irvine, na Califórnia, tendo como título "Depois da mídia: materialização e contexto". Aplicações para temas que se entrecruzam nas áreas de Estudos do Software e Plataformas são permitidas, incluindo:

O espaço-tempo da ubiqüidade e da incorporação
Depois das mídias móveis
Cognição e criatividade
Incorporação e performatividade
Ambiente/ sustentabilidade/ mudança climática
Pedagogia Interdisciplinar
Sexo e sexualidade
Estudos do Software/ plataformas

Para mais detalhes, acesse o Call for Paper completo em: DAC09 call for papers.

National University of Singapore awards a grant for Cultural Analytics research

2009-03-02T19:16:00.000-08:00

February 20, 2009: National University of Singapore (NUS), the leading research university in Singapore, has awarded a grant for 175,000 SGD to a group of NUS faculty and Lev Manovich for a project enititled "Mapping Asian Cultures: From Data to Knowledge."
The grant will allow the group to develop a large-scale cultural analytics project dealing with the analysis and visualization of cultural patterns in Asian countries in the 20th and early 21st century: from traditional art to online social media.

The group consists from the following researchers:

Visiting Researcher:
Lev Manovich
Professor, Visual Arts Department, UCSD
Director, Software Studies Initiative, Calit2 / UCSD

Kay O’Halloran
Associate Professor, Department of English Language & Literature, Faculty of Arts & Social Science; Director, Multimodal Analysis Lab, Interactive & Digital Media Institute (IDMI).

Robbie Goh
Head of Department, Department of English Language & Literature, Faculty of Arts & Social Science
Deputy Director, Asian Research Institute (ARI).

Giorgos Cheliotis
Assistant Professor, Communication and New Media (CNM)

Lonce Wyse
Associate Professor, Communication and New Media (CNM)
Director, Arts & Creativity Lab, Interactive & Digital Media Institute (IDMI)

Roger Zimmermann
Associate Professor, School of Computing

From the proposal abstract:

"The members of our group have already accumulated substantial experience in quantitatively analyzing and visualizing large sets of cultural data. Using online community music site ccMixter, Giorgos Cheliotis analyzed the patterns of music creation, sampling and remixing between 1,850 active users who at the time of data collection (2007) have collectively produced 7,484 tracks. The projects currently underway in Manovich’s lab include analysis and visualization of patterns in 200,000 art history images drawn from artstor.org, interactions between tens of thousands of MySpace users, biographies of Korean diaspora artists, and public perceptions of architectural developments in Asian countries. Kay O’Halloran’s Multimodal Analysis Lab is currently working on a large project which involves developing linguistic, image and sound processing techniques for analyzing complex multimodal documents, videos and interactive events on the web. The automated analytical techniques developed in the Multimodal Analysis Lab are applicable to the large cultural data sets that we want to use in the project proposed here.

Equally importantly, Manovich and Cheliotis have independently developed theoretical frameworks for working with large cultural data sets. Since 2005 Manovich has been developing a broad framework for this research that he called “Cultural Analytics.” The framework calls for the use of interactive visualization, Geographical Information Systems (GIS), data mining, and statistical data analysis for research, teaching and presentation of cultural artifacts, processes, and flows. Manovich’s lab is focusing on analysis and visualization of large sets of visual and spatial media: art, photography, video, cinema, computer games, space design, architecture, graphic and web design, product design. Another focus is on using the wealth of cultural information available on the web to construct detailed interactive spatio-temporal maps of contemporary global cultural patterns. Cheliotis refers to his research using the term “Online Cultural Dynamics.” His focus is on the examination of the structure, dynamics and output of online communities. According to Cheliotis, they are largely based on the same processes of creative reuse, synthesis and production that have characterized most of our cultural production throughout human history, but are now accentuated with the new paradigm of collaborative, commons-based peer production which critically depends on the sharing and licensing behaviour of creators.

We propose to extend this research in new directions by taking advantage of the unique combination of resources and expertise available at NUS. Drawing on the broad expertise of our team will allow us to do work in the analysis and visualization of cultural patterns on the scale beyond what has been done so far. The humanities scholars who have already started to analyze, graph, and map cultural data so far focused on relatively narrow areas of humanities – for instance, 19th century novels or archaeology of particular ancient sites. No attempt has been made so far to apply this approach to a much larger set of cultural activities related by a common geography or a time period. Additionally, humanities scholars tend to use visualizations as static illustrations of their findings – rather as a research tools for data exploration and knowledge discovery, as it is common in the sciences.

Our project is designed to address both of these limitations. Firstly, we propose to carry out a number of case studies that will together begin to visualize some of the patterns in the cultural development of a whole continent over a 100-year period. What is this continent? We decided to choose the subject for our study in such a way as to make the project particularly meaningful in the location where the project will be created and first shown. Consequently, we chose to work on Asian cultures from the early twentieth century until today. Using a number of both historical and contemporary data sets and sources of information, we will map vectors of influence between different Asian traditions and the West, the struggles to establish the unique modern Asian cultural identities throughout the twentieth century, and the extraordinary rise of Asia in today’s globalization era to become new leaders in cultural innovation and experimentation (as demonstrated, for instance, by the opening ceremony of 2008 Olympics.) We will visualize the dynamics of contemporary vernacular digital culture in Asia using data from social networking and social media sites popular in Asia such as Cyworld, Friendster, Sina, etc. We will also compare online cultural dynamics in Asia and other geographical regions. This will allow us to create models of cultural development that will assist in the detection of new and potentially important cultural currents, as well as in understanding the position of Asian media creators in the globalized context of the web.

We will present our findings in articles, conference presentations and also in a large-scale high-resolution interactive installation. The interfaces and visualization techniques developed for this installation will represent the second major innovation of our project. Going beyond the static visualizations or the already familiar media interfaces deployed in museums, we will create fundamentally new types of interfaces to cultural content. The installation will present innovative ways of interacting with and experiencing cultural assets and cultural information based on the analysis of large cultural data sets. The interfaces will use the concepts and technologies from the fields of science visualization, information visualization, and Geographical Information Systems (GIS). They will also bring together the visualization techniques normally used in science with the best techniques developed in digital art ad design. The users will be able to interact with visually rich maps and visualizations of cultural flows, patterns, and dynamics at a variety of scales, going from a “bird’s eye” of a very large database as a whole to a view of an individual cultural artifact."

Software Studies Brazil at PARALELO: Technology & Environment

2009-03-01T19:32:00.000-08:00

PARALELO
TECHNOLOGY & ENVIRONMENT
A MEETING POINT FOR ARTISTS, DESIGNERS & RESEARCHERS

PARALELO is a unique five day project - workshops, symposia and live events - supported and organised by the British Council in Brasil and the UK, hosted by the Museum of Image and Sound in São Paulo and the Centro Cultural de Brasil, with support also from the Mondriaan Foundation and the Virtueel Platform in the Netherlands and the Arts & Humanities Research Council in the UK. It brings together artists and designers working with media from three different countries - Brasil, the Netherlands and the United Kingdom, to discuss different ways in which collaborations across disciplinary and cultural borders can enable research and new insights into global and local ecological problems.

The teamwork across artistic, design, scientific and technological borders which is increasingly a key part of the cultural canvas of the 21st century can bring about new insights and lead to new knowledge. But how are such investigations supported in different countries and what can we learn from consideration of models of exchange and intercultural dialogue?

Sunday 29 March

3pm

REFLEXIVE ENERGIES - THE HISTORY AND VISION FOR ART, ECOLOGY, DESIGN AND TECHNOLOGY EXCHANGE IN BRAZIL
Opening the five day programme of debates, workshops and intensive discussions with practitioners and researchers in the fields of art, design, science and technology from three countries, this panel of leading artists, curators and researchers from Brasil will set the scene, reflecting on their work and on the work of other leading figures within Brasilian cultural history who have engaged in many ways with the issues which Paralelo seeks to address.

Chair: Martin Grossmann - director of CCSP, founder of Forum Permanente
Cicero Silva - Software Studies Group Brasil
Marcus Bastos - critic and independent curator of eletronic arts
Karla Brunet - artist and researcher at UFBA in the fields of art, science and technology

Organizing team:

Bronac Ferran (Royal College of Art, UK)

Gisela Domschke / IED and Independent, Sao Paulo (BR)

Karen Halley / British Council (BR)
Roberta Mahfuz / British Council (BR)
Sian Prime / Goldsmiths College, London (UK)
Liliane Rebello / British Council (BR)
Annette Wolfsberger / Virtueel Platform Amsterdam (NL/AT)
Louise Wright / British Council (UK)

The Art of Dominant Color in Film

2009-02-27T20:40:00.000-08:00

In a recent post to the nettime list, Florian Cramer gives a slightly tongue-in-cheek description of his recent work on his Floppy Films project

In my own attempt of compressing this year's Oscar-nominated films to full-length 1.44 MB files as part of my Floppy Films project, I also crunched "Slumdog Millionaire" to a 7x3 pixel/8fps/128 colors animated GIF. In such a stripped-to-the-bones rendering, the dominant color palette of the production design becomes quite visible. In the case of "Slumdog Millionaire", the dominance of red-brown-yellow 'curry' colors aptly reveals the whole exoticism of the film. [In contrast, "Milk" uses a bright pastel - milky - color palette while dark sepia tones dominate the 1940s/50s period setting of "The Reader", and a frequent combination of olive greens and magenta violets sets the white trash tone for "The Wrestler". Since everything else seems predictable enough, I didn't bother watching higher resolution versions of these films.]

Watching the Floppy Films version of "Slumdog Millionaire" in a web browser may produce upscaling artifacts that create the illusion of there being more pixel information that there actually is, but it really is just 7x3. Here is a zoomed look at the actual information opening frame of the film:

At the heart of the artist's playfully irreverent abstraction of the Oscar lineup is a good question. Can we describe cultural objects like feature films by characteristics such as "dominant color palette"? Can we meaningfully compare them? Is such knowledge practical and useful, and will we learn anything we didn't already know?

We can, but the methods and outcomes are not always obvious. Here is one quick cultural analytics approach to Florian's Floppy Film take on "Slumdog Millionaire". Rather than impressionistically assessing a "red-brown-yellow 'curry'" theme, we can visualize the film's appearance, average states, and histogram distribution.

What does "Slumdog Millionaire" look like overall?: A montage of the entire film reveals a dark image dominated by unsaturated light tans, greens, and blues, with a few bright oranges and blues at the beginning and end.
What is the "Slumdog Millionaire" dominant color and composition?: Taking the mean of the entire film produces an olive green image with slightly brighter center-shot.
What is the "Slumdog Millionaire" dominant color pallete?: Black and grey are main colors, as in most realistic cinema - this perhaps goes without saying, but maybe it shouldn't go unsaid, as it probably isn't true in things like cartoons, ads, or music videos. Besides black, a quantization of the top 12 colors shows greens, blues, and tan, with a little bit of orange. Reds and browns aren't anywhere near the top, and yellow is a minor note.

This olive-blue "Slumdog Millionaire" revealed above seems very different from the red-brown-yellow one in the artist's tongue-in-cheek description, however we can find that palette in the film. Here is the same process applied only to the opening two minutes of the film:

What does the opening look like overall?
What is the opening's dominant color and composition?
What is the opening's dominant color pallete?

Here in the opening we can clearly see the red-brown-yellow 'curry' colors that the artist described, and perhaps begin to guess at how a general aesthetic assessment arose out of this strong opening impression.

This process of exploring "Slumdog Millionaire" suggests several things:

We can characterize the palettes of films as a whole, and it can be useful to do so.
We can characterize shots and sequences as differing from or typifying the character of a film as a whole.

We can examine the individual or group aesthetic impression of what typifies the "essential" character of a film and identifies which shots or scenes best correspond to that assessment.

Cultural analytics approaches to film can be a useful tool for cutting through confirmation or selection bias and seeing the character of the object clearly -- however, it is not simply a method for trumping aesthetic impression with quantified data. Inquiry can flow just a easily in the other direction, beginning with the fact of the impression, and asking what in the film best corresponds to or typifies that perceived essence. For example, if the impression of a critic or a group was that red-brown-yellow typifies the essential look of "Slumdog Millionaire," we could begin with the opening and closing shots and ask how / why / if they serve as an essential representation of the film as a whole. Or we could take the histogram of each frame (or of a rolling average for a short window of frames) and search it.

This brief exploration is fairly far afield from the original provocation of the Floppy Films project, however the process of engaging Cramer's off-hand comment on color has helped me understand that work's method (and others like it) a bit better. Many forms of artistic and visual abstraction (e.g. "unfocused / pixelated video" type projects) may not actually help us come to grips with how color changes over time, even though their abstract idiom gives us a strong impression that they are doing so. Histogrammatic visualizations are direct and explicit, while simple frame montages (after the style of projects such as Cinema Redux by Brendan Awes) is both visceral and has the added virtue of Tuftian high information density. By contrast, Floppy Films: Slumdog Millionaire is much more effective at communicating the "dominant color palette" of the moment -- which is an important kind of knowledge of a very different order.

BettyBoop.viz

2009-02-23T18:18:00.000-08:00

Cultural Analytics method allows us see the diffirences between cultural artifacts which at first may appear to be identical - and to reveal the similarities between artifacts which may appear to have nothing in common. The following quick "analytical sketch" (think of "sketches" done by artists @processing.org) done with 5 Betty Boop cartoons from the 1930s shows how by experimenting with alternative ways of graphing the data we make visible the relationships between the artifacts from the same set (i.e., cartoons made by Max Fleischer animation studio over a period of a few years.)

The five cartoons are available at archive.org. They are Betty Boop: A Song a Day (1936), Betty Boop: Is My Pam Read (1932), Betty Boop: More Pep (1936), Betty Boop's Ker-Choo (1932), Betty Boop: The Candid Candidate (1937).

A sequence from Betty Boop is My Palm Read, 1932 (sampled at 10 frame intervals).

In order to reveal the similarities and diffirences between these five works done by the same animation studio, we have also included the data from a artifact produced in another era and with radically diffirent technology - a music video by Björk (which we analyze in more detail in MotionGraphics.viz.)

X: time | Y: mean value (i.e., average gray scale value for each frame).
Thik red line: Björk video. Other lines: Betty Boop cartoons.
The graph does shows that the "temporal signatures" of Björk video - which in its techniques exemplifies motion graphics of the 2000s - is quite distinct from the cartoons of the 1930s drawn by hand and shot on animation stand. This is not a big surpise; but the graph does not tell us much about the relationships of the five cartoons to each other.

Same data but represented as a histogram-like form better reveals the patterns. Again, it is not surprising to see that the visual temporal structure of the five cartoons is quite different from that of Biork video; however, what we did not expect to find is how remarkably similar the cartoons are to each other. Produced over the course of 5 years by different people, it is though they are parts of the same film. (Adorno's fans, take notice.)
X: mean value (average gray scale value for each frame).
Y: number of frames which have a particular mean value.

Having discovered the technique of graphing the measurements of all the frames of a film sorted in value, we decided to apply it to other measurements of the same artifacts. Here we plot sorted center of mass data. (Center of mass is defined as "he brightness-weighted average of the x and y coordinates all pixels in the image.") Although a single measurement of the center of mass for one frame may not be very informative, a successive set of measurements indicates (very aproximately - but quick to produce) how the action is moving in X,Y over the time. The similarity of the five cartoons stands out even more clearly.

In this graph we are looking at all frames in five cartoons and all the frames from Biork video - projected on top of each other. The data plotted again is center of mass measurements. Now we discover something new. Althogh worked over with software, Biork's video is based on live footage of the singer, so in this way is typical of the universe of moving images which show people - who tend to walk on the ground. However, the cartoon universe is almost completely isotropic (invariant with respect to direction): the distribution of enter of mass measurements is symmetrical, were the characters are not bound by gravity.

Here we plot center of mass measurements (Y value) over time. When data is plotted in this way, the opposition between the isotropic movement universe of the cartoons vs. non-isotropic universe of the humans is also clearly visible.

PLAYPOWER no Brasil e no mundo

2009-02-12T17:01:00.000-08:00

A seção Gadget Lab do Blog da revista Wired fez uma cobertura completa sobre o projeto Playpower, que foi apresentado no dia 10 de março de 2009 no O'Reilly Emerging Technology Conference, in San Jose, Califórnia. O projeto vem cada vez mais ganhando espaço nas discussões sobre soluções educacionais através do uso de computadores com baixo custo e fácil acessibilidade.
O projeto lançou recentemente a sua página em língua portuguesa, para os interessados em colaborar na produção de jogos com preços acessíveis à maioria da população carente. Só para lembrar, o computador custa, nos Estados Unidos, 12 dólares.
O site do projeto em português é: www.playpower.org/br

Exploring Time in Structured Video

2009-02-04T00:59:00.000-08:00

How can we characterize mass quantities of video in ways that make it more tractable to data exploration without losing aesthetic richness? Three new techniques are briefly discussed here:

Time Frames, in which the time-axis is collapsed to create a single combined image (much like a long-exposure photograph)
Time Slices, in which the video is "sliced" along the time-axis to reveal recurrent patterns not visible on the surface (much like counting the rings in a tree trunk)
Time Structures, in which the video is arranged as a stack of images and then sculpted in 3d (much like a 3d medical image such as a brain scan)

The resulting combinations, cross-sections, and 3d models can all be used:

as thumbnails to characterize complex video clips
as analytic images to suggest regions and segments for further analysis
as classifiers for clustering large video clip data sets

Like common techniques of representing video such as film strips or montages of key frames, the above representations all work to make time simultaneously apprehensible. Unlike film strips and montages, however, these frames, slices, and structures all emphasize variation and continuity over time.

Examples below deal exclusively with video gameplay recordings, however experiments applying these techniques to other types of software-generated video and video in general are underway.

Time Frames: collapsing the time axis of video

Asteroids (Atari 2600): 3 sample frames

Asteroids (Atari 2600): mean frame of 10 mins
Each pixel represents the arithmetic mean of that location over the entire 10 minute recording of gameplay. The player

Asteroids (Atari 2600): standard deviation frame of 10 mins

Each pixel represents the amount of change over the course of the 10 minute recording of gameplay. Dangerous asteroids spawn thickly at the perimeter, and thin out as they approach the clear center column protected by the spinning player.

Orbient (Nintendo Wii): 3 sample frames

Orbient (Nintendo Wii): mean frame of 10 levels

Orbient (Nintendo Wii): mean frame for each of 9 levels

Desktop Tower Defense 1.5 (Internet): mean frame for each of two players

Desktop Tower Defense 1.5 (Internet): mean frame of 40 players

Time Slices: exploring the time axis of video

Cubello (Nintendo Wii): sample frame

Cubello (Nintendo Wii): mean frame of 5 minutes of play.
Image indicates consistent HUD-style user interface elements as possible areas of interest along three edges of the screen.

Cubello (Nintendo Wii): time slices of 5 minutes of play
Slices are taken from the ammo magazine column (left) and from the "bonus time" row (bottom). Each slice can be read as a graph.

Frogger (Atari 2600): 3 sample screenshots

Frogger (Atari 2600): mean frame of 5 minutes of play

Frogger (Atari 2600): "lily pads" time slice of 5 minutes of play

Reading from top, green bars indicate the duration the pad has been marked complete (filled with a Frogger face). Orange marks indicate the appearance of bonus flies.

Frogger (Atari 2600): "countdown timer" time slice of 5 minutes of play
Reading from top, orange bars indicate changing score, blue bar indicates countdown timer, which resets every time Frogger either fills a lily pad or loses a life.

Time Structures: scultping the space-time cube of video

Rockband (Nintendo Wii): sample frame

Rockband (Nintendo Wii): regions of interest in user interface

Rockband (Nintendo Wii): mean frame with two time slices.
Left: time slice of "Band meter" bar. Bottom: time slice of musical note sequence as tablature.

Rockband (Nintendo Wii): orthoganal views of band meter and musical note time slices.

Rockband (Nintendo Wii): 3-dimensional structures extracted from video regions of interest: tablature in 3d view, wheel-shaped meter (from center bottom of UI) and instrument marker (from inside left of UI)

Software Studies in Chronicle of Higher Education

2009-01-21T19:50:00.000-08:00

MATTHEW KIRSCHENBAUM.
Where Computer Science and Cultural Studies Collide. The Chronicle of Higher Education. January 23, 2009.

"Academic study of new media is increasingly spawning more specialized inquiry: Game studies, software studies, critical-code studies, even platform studies are all buzzwords in the field. One high-profile meeting last year at the University of California at San Diego, hosted by Lev Manovich and Noah Wardrip-Fruin, brought together an international array of scholars, artists, and technologists, including many of the names mentioned in the accompanying article, to discuss "the meaning of studying software cultures, and the direction and goals of software studies as an emerging movement."

Calit2 article about our Humanities High-Performance Computing grant

2009-01-18T12:30:00.000-08:00

Calit2 published an article about our umanities High-Performance Computing grant from NEH Digital Humanities office. They also took nice fotos of Lev and So with the HIPerSpace wall:

Supercomputing to Help UC San Diego Researchers Visualize Cultural Patterns

images | Cultural Analytics visualizations on ultra high resolution displays

2008-12-24T12:27:00.000-08:00

Cultural Analytics interface designs | Concept: Lev Manovich and jeremy Douglass| Graphic design: Sergie Magdalin and Bob Li (undergraduate students, Visual Arts Department, UCSD.) | Produced: April 2008.

Cultural Analytics interface designs @ HIPerWall visual supercomputer | 205 megapixels | Calit2-Irvine | Presented: May 2008.

Cultural Analytics visualizations @ HIPerSpace visual supercomputer | 287 megapixels| Calit2-San Diego | Decemeber 2008:

Cultural Analytics software running on HIPerSpace display | Calit2-San Diego | May 2009. Software allows interactive explorations of patterns in sets of images and video.

Visualizations created by UCSD undergraduate visual arts students on HIPerSpace display | Calit2-San Diego | June 2009

Humanities High-Performance Computing Award - Department of Energy (DOE) press release

2008-12-22T19:48:00.000-08:00

Lawrence Berkeley National Laboratory has released informaton about the joint he National Endowment for the Humanities (NEH) and Department of Energy (DOE) awards. Software Studies initiative is one of three groups which received the award.

"High performance computing and the humanities are finally connecting — with a little matchmaking help from the Department of Energy (DOE) and the National Endowment for the Humanities (NEH). Both organizations have teamed up to create the Humanities High Performance Computing Program, a one-of-a-kind initiative that gives humanities researchers access to some of the world’s most powerful supercomputers.

As part of this special collaboration, the DOE’s National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory will dedicate a total of one million compute hours on its supercomputers and technical training to humanities experts. Meanwhile, the program’s participants were selected through a highly competitive peer review process led by the NEH’s Office of Digital Humanities."