Software Studies

Research on Remix and Cultural Analytics, Part 5

2012-05-15T07:56:00.001-07:00

Image: evaluating sliced visualizations of The Charleston Style remixes at the Vroom at Calit2. View larger image. View other Vroom images by cultvis on Flickr.

In previous posts I discussed how I used cultural analytics to examine video mashups. (See part 1 on the Charleston Mix, part 2 on Radiohead’s Lotus Flower, and part 3 on the Downfall parodies, and part 4, on sliced visualizations of all three case studies.) One thing that is difficult in this process is to view all images at once in order to make the observations that I have discussed so far. This is when a large tiled screen is useful, such as the one available at the Vroom at Calit2, where the Software Studies Lab in San Diego is based. Below are images that give an idea of how the large screen is useful to evaluate various images at once.

Image: wide view, 32 tiled-screen at the Vroom, Calit2. See larger image.

This image shows the thirty montage grid visualizations of my second case study, The Lotus Flower Parodies. The advantage in this case is that all thirty videos can be examined at once. This is something that is impossible on a regular laptop or a large computer screen. Being able to compare images in large scale is not only useful to come up with detailed analysis, but also provides the ability to discuss one’s research with other colleagues.

Image: Tracy Cornish, a researcher at CRCA, points out a detail to a colleague of my Lotus Flower remixes grid visualization. See larger image.

Image: Detailed visualization of Thom Yorke Does the Macarena! See larger image.

Image: Sliced images of Lotus Flower remixes on top of montage grid visualizations. (See part 3 and part 4 my analysis for more on sliced images.)

One of the advantages of the tiled screen, in addition to viewing many images at once and in great detail, is the fact that the files don’t appear inside windows as they would on an average computer. As the image above makes obvious, you can lay images next to each other, and on top of others, with no frame around them. While this feature might appear not so important when first considered, I found that it provided me with a sense of immediacy.

Image: Todd Margolis, Technical Director at CRCA, examines grid-montage and sliced image visualizations of Lotus Flower Parodies. See larger image.

Image: alternate view of grid-montage and sliced image visualizations of Lotus Flower Parodies. See larger image.

Image: Todd Margolis, Technical Director at CRCA, examines grid-montage and sliced image visualizations of Lotus Flower Parodies. See larger image.

Image: detail of grid-montage visualization of Charleston Style remixes. See larger image.

Image: detail of sliced visualization of Downfall parodies. See larger image.

Image: sliced visuazlizations of the three case studies on top of Lev Manovich’s and Jeremy Douglass’s Time Magazine covers. See larger image.

Image: detail of Lev Manovich’s and Jeremy Douglass’s Time Magazine covers. See larger image.

Going back to my initial point, when considering a large amount of images, such as all Time Magazine covers, it becomes evident how being able to view several images at once becomes an important part of visualization.

Image: alternate view of Lev Manovich’s and Jeremy Douglass’s Time Magazine covers. See larger image.

Research on Remix and Cultural Analtytics, Part 4, by Eduardo Navas

2012-05-05T13:49:00.001-07:00

Image: Detail of sliced visualization of thirty video samples of Downfall remixes. See actual visualization below.

As part of my post doctoral research for The Department of Information Science and Media Studies at the University of Bergen, Norway, I am using cultural analytics techniques to analyze YouTube video remixes. My research is done in collaboration with the Software Studies Lab at the University of California, San Diego. A big thank you to CRCA at Calit2 for providing a space for daily work during my stays in San Diego.

The following is an excerpt from an upcoming paper titled, “Modular Complexity and Remix: The Collapse of Time and Space into Search,” to be published in the peer review journal AnthroVision, Vol 1.1. A note will posted here, on Remix Theory, announcing when the complete paper is officially published.

The excerpt below is rather extensive for a blog post, but I find it necessary to share it in order to bring together elements discussed in previous posts on Remix and Cultural Analytics (see part 1 on the Charleston Mix, part 2 on Radiohead’s Lotus Flower, and part 3 on the Downfall parodies). The excerpt has been slightly edited to make direct reference to the previous postings, and therefore reads different from the version in the actual text, which makes reference to sections of the research paper where more extensive analysis is introduced. Consequently, in order for this post to make more sense, the previous three entries mentioned above should also be read.

The following excerpt references sliced visualizations of the three cases studies in order to analyze the patterns of remixing videos on YouTube. The reason for sharing part of my publication now is to bring together the observations made in previous postings, and to make evident how cultural analytics enables researchers invested in the digital humanities to examine cultural objects in new ways that were not possible prior to the digitalization process we have been experiencing for the last decades.

———–

How a meme evolves based on the first remixes that a user may find can be evaluated by developing visualizations of the three cases studies that show the editing of the video footage over time. To accomplish this, I took the frames of thirty videos of each meme and sliced them in order to examine the types of pattern the editing actually takes. What we find is that with the Charleston Remixes the video footage stays practically the same except for a few remixes in which the footage of Leon and James dancing was used selectively as part of bigger projects. “Mr. Scruff - Get a Move on | Charleston videoclip” is one of these exceptions, in which the video is re-edited to match the sound (see slice detail below). Another is “Charleston & Lindy Hop Dance ReMix - iLLiFieD video.mix (Version),” (also see below).

Image: A two column slice visualization of the 29 of 30 remixes (one remix was omitted because the footage is not the same performance. That video is not relevant to evaluate how the video footage of this meme is left intact). For a full list of this visualization visit: http://remixtheory.net/remixAnalytics/ and select “Charleston Video Slices.” View large version of this image.

Image: this is a slice visualization of “The Charleston and Lindy Hop Dance Remix.” When comparing this sliced image to other slices in the two-column visualization above, one can notice the selective process with which footage from the Charleston Style was used. This video is much longer than the original footage, and has been compacted in order to show how the video was selectively edited. To view this remix, visit http://www.youtube.com/watch?v=POupa2sW1UI&feature=player_embedded. This video was uploaded to YouTube on May2, 2009. View large version of this image.

Image: this is a slice visualization of “Mr. Scruff remix.” When comparing the sliced image to the other slices in the two columns visualization above, one can notice how the same footage was edited repeatedly to match the beat and sections of the song. This video is much longer than the original footage, and has been compacted in order to show how the video was selectively edited. Visit http://www.youtube.com/watch?feature=player_embedded&v=Bx5-itIA0pQ. This video was uploaded to YouTube on January 10, 2008. View large version of this image.

Image: A two-column visualization of Lotus Flower Remixes. The original video by Radiohead is on the top-left. Most of the videos sliced in this sample were uploaded within the first two weeks after the original video was uploaded by Radiohead on February 16, 2011. For a full list of this visualization visit: http://remixtheory.net/remixAnalytics/ and select “Lotus Flower Video Slices.” View large version of this image.

In the Lotus Flower Remixes (See image above) we can note that the editing of the videos is quite diverse; the footage is remixed (heavily edited) to match the beat and the overall feel of the selected songs, with the very first videos.

The Downfall remixes (see figure below) consists of video footage that for the most part has been left intact. What is remixed is the fake translation of Hitler’s rant. The subtitles for Hitler are sometimes in the middle of the screen, in others at the bottom; sometimes the typeface is small, and at times large. But in the end the video footage is left intact and the translations very much obey the rhythm of the original editing.

Image: A two-column visualization of The Downfall Parody remixes. The original video with no subtitles is on the top-left. Videos sliced in this sample were uploaded between 2007 and 2011. At the moment it is not certain whether the 2007 upload was the first because many remixes have been taken down by YouTube. For a full list of this visualization visit: http://remixtheory.net/remixAnalytics/ and select “Downfall Video Slices.” View large version of this image.

Image: Visualization of Downfall video, with proper English subtitles. The thin horizontal white bars near the bottom of the frame are the subtitles. To view this video visit: http://www.youtube.com/watch?v=4bmkUlXp5sk&feature=related. Some of the remixes present the subtitles in yellow. View large version of this image.

Image: visualization of “Hitler’s Reaction to the new Kiss album,” a video remix in which Hitler rants about the album’s title “Sonic Boom.” The subtitles (the thin horizontal white bars) in this case move all over the frame. To view this video visit: http://www.youtube.com/watch?v=nwOLfppXhsk&feature=youtu.be. View large version of this image.

We can note in the three case studies that the approach of remixing is in part defined by the way the original remix or footage was produced. With the Charleston Remixes, most contributions leave the video footage intact. No major editing took place until September 2007, that is a year and four months after the first upload. With the Lotus Flower Remixes, editing of the footage is done from the very beginning, while with the Downfall parodies, it does not place at all. Why would this be?

Based on the diagrams (see the link “visualization of links” for each case study on the page remixAnalytics) and patterns of editing that I present, we can note that the later videos are in fact responses to previous productions. In the Charleston Remixes, the video footage is left intact because it is intact in the first remix. With Lotus Flower, the original footage by Radiohead is heavily edited, which gives remixers the license to immediately manipulate the footage in selective fashion—by omitting some parts of the footage while repeating others to match the selected songs. With the Downfall remixes, the result is similar to the Charleston Remix: the footage is practically left alone because the meme demands that the basis of the meme be that only the text be remixed; therefore, the only major shift takes place with the placement of translations on the screen: sometimes on the middle, but for the most part at the bottom. The only other shift we can notice with the subtitles is that they may crossover from one shot to the next based on the emphasis of the content that the remixer wants to make. But none of the Charleston and Downfall videos are as heavily edited as the Lotus Flower remixes. It is also worth noting that these are all selective remixes, which means that they all are dependent on a clear reference to the original source.[1] If such reference is lost, then, the remix withers, and would become either a badly concocted reference, or simply a product on the verge of plagiarism.

One last element that needs to be considered, which apparently affects the production of the memes, as is also argued by a study on YouTube funded by Telefonica [2], and also supported by the research of Jean Burgess and Joshua Green [3] is that due to the viral emphasis on YouTube, online users are most likely to find an already remixed version of a video, and not the original if the remix has enjoyed more views. The exception to this is Lotus Flower, for which YouTube apparently always offers the original video as part of possible selections, on the first page of all results. This is likely because given Radiohead’s popularity, their YouTube channel has a large number of views. For the Charleston, this is not always the case, as the original footage sometimes will not come up with certain video remixes. For the Downfall meme, it is even more difficult to speculate how videos produced before 2007 affect users who currently search for the meme, because they are likely to find videos that are popular, but not necessarily the newest nor the oldest—but rather the most relevant based on the terms used for the search in relation to the number of views.

[1] For the full definition of the selective remix see “Selective and Reflexive Mashups.”

[2] Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, and Sue Moon, “I Tube, You Tube, Everybody Tubes: Analyzing the World’s Largest User Generated Content Video System,” http://an.kaist.ac.kr/traces/papers/imc131-cha.pdf

[3] For Burgess and Green this is evident based on their assessment of the emphasis of presenting popular videos first, and the fact that YouTube members deliberately find ways to promote their videos to become as popular as possible. See Jean Burgess & Joshua Green, YouTube: Online Video and Participatory Culture (Cambridge: Polity, 2010), 74.

The Evolution of Video Game Controllers visualization

2012-05-05T13:41:00.000-07:00

SOURCE: visual.ly

Browse more Gaming infographics.

animated visualization of Arizona Sentinel weekly, 1872-1911

2012-05-05T10:50:00.000-07:00

UCSD undergraduate Cyrus Kiani added a new video to his already amazing work visualizing the history of American Newspapers using the collection at Library of Congress.

The new video shows evolution across 1962 front pages of Arizona Sentinel weekly, 1872-1911.

The Arizona Sentinel : 1872-1911

Place of publication: Arizona City [Yuma], Yuma County, A.T. [ Ariz.]

Frequency: Weekly

Language:English

sn 84021912

Chronicling America
Library of Congress
chroniclingamerica.loc.gov

Center for Research in Computing and the Arts: 40th Anniversary Celebration

2012-04-30T10:45:00.000-07:00

Join us on Tuesday May 1 to celebrate 40 years of Center for Research in Computing and the Arts (CRCA) at UCSD:

Impressionism visualizations: final class project by Megan O'Rourke

2012-04-25T10:30:00.000-07:00

Previously we featured visualizations of selected paintings by Impressionist artists created by Megan O'Rourke in my undergraduate class at UCSD (Winter 2012). One visualization compared works of artists using multiple image plots; the second used color histograms to show the same data in a different way.

Here is Megan's final class project which extends her investigation into new areas: comparing how different artists represented faces using image averaging, and visualizing evolution of their paintings over time in relation to brightness, saturation and hue.

To measure image properties used for visualizations, Megan used ImageMeasure macro which we distributed as part of our free ImagePlot software.

(CLICK ON EACH IMAGE BELOW TO SEE A HIGHER RESOLUTION VERSION)

Explanation of how this visualization was generated

Software Studies Initiative awarded $477,000 grant from Mellon Foundation

2012-04-20T11:34:00.000-07:00

Project name:

Tools for the Analysis and Visualization of Large Image and Video Collections for the Humanities

Project team:

PI: Dr. Lev Manovich, Professor of Visual Arts, University of California, San Diego (UCSD);
Director, Software Studies Initiative, California Institute for Telecommunications and Information Technology (Calit2.

Almila Akdag, Postdoctoral Researcher, e-Humanities Group, The Royal Netherlands Academy of Arts and Sciences; Visiting Scholar, Visual Arts and Communication Design, Sabanci University, Istanbul, Turkey.

Loretta Auvil, Senior Project Coordinator at Illinois Informatics Institute, University of Illinois; SEASR co-PI.

Jeremy Douglass, Technical Director, Software Studies Initiative, UCSD.

Elizabeth Losh, Director of Academic Programs, Sixth College, Program in Culture, Art, and Technology, UCSD.

Project summary:

Since 2008, Software Studies Initiative at California Institute for Telecommunications and Information Technology (Calit2) and University of California, San Diego (UCSD) has been developing a comprehensive set of software tools for the quantitative analysis and visualization of large collections of images and video. The tools were designed for academic researchers in the humanities, and have already been used by scholars in a number of disciplines including art history, archeology, film and media studies, dance studies, and game studies. We have also been working with a number of prominent cultural institutions and collections including the Library of Congress, Getty Research Institute, the Austrian Film Museum, and the Netherlands Institute for Sound and Image, in using our techniques with their collections and data sets. The software development and its applications has received support from the National Endowment for the Humanities (NEH), the National Science Foundation (NSF), the University of California Humanities Research Institute (UCHRI), UC San Diego, and the California Institute for Telecommunications & Information Technologies (Calit2).

In our new three year project funded by $477,000 grant from The Andrew W. Mellon Foundation, we will work to fully integrate our techniques and tools into the SEASR/Meandre environment, a major platform for digital humanities research developed with key support from the Andrew W. Mellon Foundation. The integrated tools will come with comprehensive documentation and a set of examples covering a number of fields in the humanities and humanistic social sciences. This integration will address a current goal of SEASR to “continue to evolve to include processing of images and other multimedia data formats.” We anticipate these tools being used by an ever-expanding range of people, including academics and students in the humanities and humanistic social sciences, museum curators and visitors, and cultural creators who want to better understand how their work fits within a larger context

In addition to making available to others software tools, accessible user interfaces, documentation, and examples, Software Studies Initiative will also collaborate with other researchers to carry out large-scale case studies. Each case study will demonstrate how, within a particular field, quantitative analysis and visualization of images and/or video can open new research possibilities for that field. Each study will include documentation of the appropriate SEASR workflows, a paper describing the data, the methods used, the findings, and high-resolution still and animated visualizations:

Almila Akdag will lead the case study which will combine network analysis and image processing to explore a few million images and user data from deviantArt (the most popular social network for user-generated art).

Jeremy Douglass will lead the analysis of our one million manga images dataset.

Elizabeth Losh will lead the case study which applies our methods to thousands of hours of political video on the web and TV news.

Over 200 undergraduate and graduate UCSD students will participate in the project over its three year period, exploring selected data sets as part of their classes in visualization and computational art history, and digital humanities.

Contact:

Lev Manovich, Director, Software Studies Initiative [manovich@ucsd.edu]

More information:

Our methods for the analysis and visualization of large visual data sets

Our projects (analysis of image sets covering video games, visual art, graphic design, maagzines, newspapers, comic books, TV, films, animation, motion graphics.)

http://www.flickr.com/photos/culturevis/collections/ (Over 900 visualizations and sketches from our lab)

Our open source software tools (digital image processing and visualization of image sets of any size.)

Case study: One million manga pages

Pilot project: Digging Into Global News

Manovich's two day seminar in MediaArtHistories program, Krems, Austria, May 14-15

2012-04-18T10:49:00.000-07:00

Two day seminar with Lev MANOVICH on the 10-year evolution from "Language of New Media
(2001)" to "Software Takes Command" and Cultural Annalytics" - May 14-15, 2012.
Center for Image Science, Danube University, Krems, Austria.

day 1: new media theory and software studies (based on The Language
of New Media and Software Takes Command books)

day 2: new methods for the study of visual media (based on cultural
analytics articles and projects at softwarestudies.com)

Other lecturers and excursion for May 2012 start of MediaArtHistories program:

In affiliation with Leonardo / ISAST and UTD, Roger MALINA maps the history of Leonardo and the future of the Art, Sciences & Technology fields.

Jussi PARIKKA’s innovative scholarship on Insect Media.

Linz excursion - Nina WENHART takes students behind the scenes of Ars Electronica, Stadtwerkstatt and more; meeting with the founders & next generation of the Ars Electronica Festival, Prix, Centers, and Lab: Christine SCHÖPF, Hannes LEOPOLDSEDER, Horst HOERTNER, & Christopher LINDINGER.

http://www.donau-uni.ac.at/mediaarthistories
http://www.virtualart.at
www.mediaarthistory.org
http://www.donau-uni.ac.at/telelectures

Contact - application information:
Andrea Haberson
andrea.haberson@donau-uni.ac.at

Contact - course and content questions
Wendy Coones
wendy.coones@donau-uni.ac.at

MEDIA.ART.HISTORIES, MA - The MediaArtHistories learning environment at the Center for Image Science integrates world-wide leading scholars from the field into a faculty working intensely with the students in seminars and workshops. The MediaArtHistories masters program conveys the most important developments of contemporary art through a network of renowned international theorists, artists, curators and many others. Historical derivations that go far back into art and media history are tied in intriguing ways to digital art. Key approaches and methods from Image Science, Media Archaeology and the History of Science & Technology will be discussed. Media Art History offers a basis for understanding evolutionary history of audiovisual media, from the Laterna Magica to the Panorama, Phantasmagoria, Film, and the Virtual Art of recent decades. Using online databases and other modern aids, knowledge of computer animation, net art, interactive, telematic and genetic art as well as the most recent reflections on nano art, CAVE installations, augmented reality and wearables are introduced.

FACULTY - Lev MANOVICH, Roger MALINA, Jussi PARIKKA, Ramón REICHERT, Nina CZEDLEDY, Nina WENHART, Christine SCHÖPF, Hannes LEOPOLDSEDER, Horst HOERTNER, Christopher LINDINGER -- Erkki HUHTAMO, Christiane PAUL, Jens HAUSER, Jeffrey SHAW, Gerfried STOCKER, Christa SOMMERER & Laurent MIGNONNEAU, Sean CUBITT, Paul SERMON, Oliver GRAU, Edward SHANKEN, KNOWBOTIC RESEARCH, Frieder NAKE, Machiko KUSAHARA, Nat MULLER, Gunalan NADARAJAN, Monika FLEISCHMANN, Margit ROSEN, Andreas LANGE, Miklos PÉTÉRNAK, Martina LEEKER, Christopher SALTER, Darko FRITZ, Slavko KACUNKO, Irina ARISTARKHOVA, and others.

STUDENTS - The course is held in English and is low-residency with rolling admissions. Acceptance into the program requires a previous degree at or above the Bachelors level, or the equivalent through relevant work experience. International students come from countries like Canada, Hong Kong, Ukraine, USA, Japan, , Brazil, Iceland, Russia, Egypt, Germany, & Korea and Mexico.

DANUBE UNIVERSITY - located in the UNESCO world heritage Wachau, near Vienna is the first public university in Europe which specializes in university-based advanced education offering low-residency degree programs for working professionals and lifelong learners.

CENTER FOR IMAGE SCIENCE - international experts analyze the image worlds of art, science, politics and economy and elucidate how they originated, became established and how they have stood the test of time. The innovative approach at the Center for Image Science is reinforced by praxis-oriented study. Without interrupting their career, students have the opportunity gain key qualifications for the contemporary art and media marketplace through modular courses in internationally unique surroundings. Students in the MediaArtHistories, MA program come twice a year for 2-week blocks to the CIS in the 1000-year-old Monastery Göttweig, and the newly built Danube University main campus.

The next Module is May 4-15, 2012.

Visualizations of Impressionist artists - color histograms (part 2)

2012-04-16T10:20:00.000-07:00

Visualizations of Impressionist artists - color palettes (part 1) were created by UCSD undergraduate student Megan O'Rourke for her homework in my Winter 2012 class data visualization and compututional art history (the link is to Spring 2012 version of the class).

Here is another innovative visualization created by Megan. She adopted histogram technique to compare color palettes of six Impressionist artists. The histograms show the relative proportions of different hue in the set of paintings of each artist. (To make this visualization easier to read, below is the visualization of the same images from the earlier post. It maps paintings according to x-axis = median saturation, y-axis = average hue).

Together, the two visualizations reveal strong similarity between the color "footprints" of the selected paintings of these artists.

Data:
Images of 630 Impressionist paintings.

Source:
Artstor.

Number of paintings per artist:
Artstor contains only some of the paintings by these arists. The diffrences in the numbers of images available for each artist reflect the differences in popularity of each artist as well their varied productivity.

The histograms use the median hue values measured per each painting.

Visualizations of Impressionist artists - color palette comparisons (part 1)

2012-04-14T10:30:00.000-07:00

The following visualizations was created by UCSD undergraduate student Megan O'Rourke in my Winter 2012 class data visualization and compututional art history (the link is to Spring 2012 version of the class).

The visualizations compare color palettes in paintings of of six Impressionist artists. In each image plot, x-axis = median saturation; y-axis = median hue. The visualizations were created with our open source ImagePlot software. Click on each image to see high resolution version.

Data:
Images of 630 Impressionist paintings.
Source:
Artstor.
Number of paintings per artist:
Artstor contains only some of the paintings by these arists. The diffrences in the numbers of images available for each artist reflect the differences in popularity of each artist as well their varied productivity.

When I wrote a macro which we distribute with ImagePlot and included average hue as one the image measurements, I was not sure if it can be actually used in a meaningful way. Megan's visualization shows that this measurement is quite useful. Her other invention was to extend plots vertically and place them side by side - a really good format for comparing image sets.

Teaching of art history for many decades of the 20th century was built on comparing two images at a time using slide projector technology. Media visualization methods developed at our lab allow scaling of comparision. Any number of images can be placed side by side and sorted according to multiple attributes (including metadata, content and visual form).

Close-up: a comparison between Cassat, Monet, Morissot:

Looking at the visualizations, I am struck by how similar are the color footprints of the Impressionist artists. What are the reasons for this similarity? How does it relate to the range of subects in their works, and their habit of working outside using newly available technology of oil paint in tubes?

close-up: all 1226 Impressionist paintings:

visualizing explosion of digital data

2012-04-08T15:26:00.000-07:00

The World's Technological Capacity to Store, Communicate, and Compute Information.

Martin Hilbert1 and Priscila López.

Science, February 10, 2011.

Abstract:

We estimate the world's technological capacity to store, communicate, and compute information, tracking 60 analog and digital technologies during the period from 1986 to 2007. In 2007, humankind was able to store 2.9 × 1020 optimally compressed bytes, communicate almost 2 × 1021 bytes, and carry out 6.4 × 1018 instructions per second on general-purpose computers. General-purpose computing capacity grew at an annual rate of 58%. The world's capacity for bidirectional telecommunication grew at 28% per year, closely followed by the increase in globally stored information (23%). Humankind's capacity for unidirectional information diffusion through broadcasting channels has experienced comparatively modest annual growth (6%). Telecommunication has been dominated by digital technologies since 1990 (99.9% in digital format in 2007), and the majority of our technological memory has been in digital format since the early 2000s (94% digital in 2007).

Illustration from the article in Washington Post about this research:

Guide to visualizing image and video collections

2012-04-08T10:01:00.000-07:00

Guide to visualizing image and video collections

The guide describes the techniques used in our Software Studies Lab to explore large image and video sets.

Like our previously released ImagePlot software, these techniques use free ImageJ digital image analysis application with our custom macros.

The guide covers the following operations (software used is in brackets):

- Download and setup free ImageJ software used to prepare images and video for visualizations, and create visualizations.
- Automatically detect shots in a video (shotdetect).
- Output video as a sequence of frames (ImageJ).
- Automatically scale all images in a folder (ImageJ).
- Create "montage" and "slice" visualizations (ImageJ).
- Create "montage" and "slice" visualizations with diff. size images located in multiple folders (ImageMontage, ImageSlice).
- Use Unix commands to create a data file containing file paths to images located in multiple folders (Unix).

Data Visualization and Computational Art History - my 2012 Spring course syllabus, UCSD

2012-04-07T22:12:00.000-07:00

Data Visualization and Computational Art History

Course syllabus

Instructor: Lev Manovich

Spring 2012, Visual Arts Department,UCSD:
undergraduate course: VIS 149 / ICAM 130: Special Topics
graduate course: VIS 219: Special Topics

Comparing van Gogh paintings done in Paris and Arles.
X-axis = median brightness. Y-axis=median saturation.
Software: ImagePlot.

"Visualization as a Method in Art History" - slides of my 10 min intro to 2012 CAA session

2012-04-05T10:12:00.000-07:00

Intro to CAA 2012 session "Visualization as a Method in Art History"

View more presentations from Lev Manovich

session info:

Information Visualization as a Research Method in Art History
Friday, February 24, 2:30 PM–5:00 PM
West Hall Meeting Room 502A, Level 2, Los Angeles Convention Center

Chairs:
Christian Huemer, Getty Research Institute;
Lev Manovich, University of California, San Diego

Visualizing the Ecology of Complex Networks in Art History
Maximilian Schich, Northeastern University

Geoinformatics and Art History: Visualizing the Reception of American Art in Western Europe, 1948-1968
Catherine Dossin, Purdue University

Interactive Mapping of the Agents of the Art Market in Europe (1550-1800)
Sophie Raux, Université Lille Nord de France

Visualizing Art, Law, and Markets
Victoria Szabo, Duke University

Lithics Visualization Project for Analysis of Patterns and Aesthetic Presentation
Georgia Gene Berryhill, University of Maryland, Tom Levy, UCSD, and Lev Manovich, UCSD and Calit2

Information Visualization and Museum Practice
Piotr Adamczyk, Google and University of Illinois, Urbana-Champaign

new article: Lev Manovich, "How to Follow Software Users? (Digital Humanities, Software Studies, Big Data)"

2012-04-03T09:18:00.000-07:00

DOWNLOAD:

Lev Manovich. How to Follow Software Users? (Digital Humanites, Software Studies, Big Data).

abstract:

Big data is the new media of 2010s. Like previous waves of computer technologies, it changes what it means to know something and how we can generate this knowledge. So far, all big data projects in digital humanities that I am aware of used digitized cultural artifacts from the past. If we want to apply the big data paradigm to the study of contemporary interactive software-driven media, we are facing fascinating theoretical questions and challenges. What exactly is “big data” in the case of interactive media? How do we study the interactive temporal experiences of the users, as opposed to only analyzing the code of software programs and contents of media files? This articles provides possible answers to these questions and proposes a methodology for the study of interactive media as “big data.”

Reference:

This new article is not published anywhere yet. If you want to reference it, use the URL of this post.

Illustration: Heatmap of user eye movement superimposed over a website she is looking at.

new article: "Media Visualization: Visual Techniques for Exploring Large Media Collections"

2012-03-31T11:02:00.001-07:00

Thank you everybody for your birthday wishes !

DOWNLOAD my new article:

Lev Manovich. Media Visualization: Visual Techniques for Exploring Large Media Collections.

The article presents the theory and the techniques of media visualization used in our lab, with the analysis of the examples.

Lev Manovich and Jeremy Douglass discuss media visualizations of one million manga pages at the opening panel for Mapping Time exhibition, gallery@calit2.

Our new software tools described in the article:
ImageMontage
ImageSlice

Tutorials:
To learn how to use montage and slice visualization techniques with your datasets, use this doc:
Visualizing image and video collections: tutorials

Also check our our ImagePlot (released 9/2011):
ImagePlot

Visualizing image and video collections: examples, tutorials, software, theory

2012-03-30T10:00:00.000-07:00

Visualizing image and video collections: Examples

You can also download this 28-page PDF document from our server. It illustrates three key techniques used in our lab (softwarestudies.com) to explore large image and video sets: montage, slice, and image plot.(released 3/30/2012).

TUTORIALS:
To learn how to use montage and slice visualization techniques with your datasets, follow this guide:
Visualizing image and video collections: tutorials (released 3/30/2012).

SOFTWARE:
ImageMontage (released 3/30/2012)
ImageSlice (released 3/30/2012)

THEORY:
Lev Manovich. Media Visualization: Visual Techniques for Exploring Large Media Collections. (released 6/2/2011; updated 3/31/2012). Article about the theory and methods of media visualization, with the analysis of the examples.

More information about our digital humanities projects:
softwarestudies.com

How and Why Study Big Cultural Data - Lev Manovich's lecture, March 2012

2012-03-19T09:29:00.000-07:00

How and why study big cultural data

View more presentations from formalist.

Data Mining and Visualization for the Humanities symposium, NYU, March 19, 2012

2012-03-18T11:20:00.000-07:00

Data Mining and Visualization for the Humanities

Monday, March 19, 2012 1:00 PM - 6:30 PM

NYU Open House, 528 La Guardia Place, NYC

Program:

1:00-1:15 Opening Remarks
Mara Mills, Assistant Professor of Media, Culture, and Communication (NYU)

1:15-2:30
"Visualizing Social and Semantic Space in Brazilian Literature, 1840s-1890s"
Zephyr Frank, Associate Professor of Latin American History and
Director of the Spatial History Project, Stanford University

2:30-2:45 Break

2:45-4:00
"Text as Data"
Mark H. Hansen, Professor of Statistics, University of California-Los Angeles

4:00-4:15 Break

4:15-5:30
"Why and How Study Big Cultural Data?"
Lev Manovich, Professor of Visual Arts and Director of the Software Studies Initiative, University of California - San Diego

5:30-6:00 Open Discussion
Moderated by Lisa Gitelman, Associate Professor of English and of Media, Culture, and Communication (NYU)

6:00-6:30 Reception

Computational humanities vs. digital humanities

2012-03-16T10:24:00.000-07:00

Bluefin Labs analyze 5 billion of online comments and 2.5 million minutes of TV every month.
This visualization shows the relations between Gatorade brand and the male viewers of different TV shows.
Source: Bluefin Mines Social Media To Improve TV Analytics, Fast Company, 11-07-2011.

Echonest offer information on 30 million songs and 1.5 million music artists.
Source: the.echonest.com

Paul Butler's visualization of a sample of 10 million friends from Facebook, using company' data warehouse.
Source: Paul Butler, Visualizing Friendships.

In a article called Computational Social Science (Science, vol. 323, no. 6, February 2009, the leading researchers in network analysis, computational linguistics, social computing, and other fields which now work with large data write:

"The capacity to collect and analyze massive amounts of data has transformed such fields as
biology and physics. But the emergence of a data-driven 'computational social science' has been much slower. Leading journals in economics, sociology, and political science show little evidence of this field. But computational social science is occurring in Internet companies such as Google and Yahoo, and in government agencies such as the U.S. National Security Agency. Computational social science could become the exclusive domain of private companies and government agencies. Alternatively, there might emerge a privileged set of academic researchers presiding over private data from which they produce papers that cannot be critiqued or replicated. Neither scenario will serve the long-term public interest of accumulating, verifying, and disseminating knowledge."

Substitute the word humanities in the above paragraph, and it now describes perfectly the issues involved in large-scale analysis of cultural data. Today digital humanities scholars are mostly working with the archives of digitized historical cultural archives which were created by libraries and universities with the funding from NEH and other institutions. These archives and their analysis is very important - but this work does not engage with the massive amounts of cultural content and peoples' conversations and opinions about it which exist on social media platforms, personal and professional web sites, and elsewhere on the web. This data offers us unprecedented opportunities to undertand cultural processes and their dynamics and develop new concepts and models which can be also used to better understand the past. (In our lab, we refer to computational analysis of large contemporary cultural data as cultural analytics.)

Contemporary media and web industries are dependent on the analysis of this data. This analysis enables search, recommendations, video fingerprinting, identification of trending topics, and other crucial functions of their services. Because of its scale and technical sophistication, perhaps we should call it "computational humanities." The players in computational humanities are Google, Facebook, YouTube, Bluefin labs, Echonest, and other companies which analyze social media signals (blogs, Twitter, etc.) and the content of media on social networks. They do not usually ask theoretical questions which can be directly related to humanities, but the types of analysis they perform and the techniques they use can be easily extended to ask these questions.

The questions posed in the paragraph I quoted above are directly applicable to "computational humanities." We can ask: Will computational humanities remain the exclusive domain of private companies and government agencies? Will we see a privileged set of academic researchers presiding over private data from which they produce computational humanities papers that cannot be critiqued or replicated?

These questions are essential for the future of humanities. In this respect, NEH/NSF Digging Into Data competitions are very important as they try to push humanists to think on the scale of computational humanities, and collaborate with computer scientists. To quote from the description of 2011 competition:

"The idea behind the Digging into Data Challenge is to address how "big data" changes the research landscape for the humanities and social sciences. Now that we have massive databases of materials used by scholars in the humanities and social sciences -- ranging from digitized books, newspapers, and music to transactional data like web searches, sensor data or cell phone records -- what new, computationally-based research methods might we apply?"

In our lab, we are hoping to make a contribution towards bridging the gap between "digital humanities" and "computational humanities." Our data sets range from the small historical datasets - for instance, 7000 year-old stone arrow heads and paintings of Piet Mondrian and Mark Rothko - to large scale contemporary user-generated content such as 1,000,000 manga pages or 1,000,000 images from deviantArt (the most popular social network for user-generated art). We also write papers for both humanities and computer science audiences. All our work is collaborative, involving students in digital art, media art, and computer science. And although our largest image sets are still tiny in comparison to the data analyzed by the companies I mentioned above, they are much bigger than what humanists and social scientists usually work with. The new visualization tools we have developed already allow you to explore patterns across 1,000,000 images, and we are gradually scaling them up.

Visualizing newspapers history: The Hawaiian Star, 5930 front pages, 1893-1912

2012-03-15T10:00:00.000-07:00

>
The Hawaiian Star, 5930 front pages, 1893-1912 (Vimeo)

Last September I met with Leslie Johnston (Chief of Repository Development at Library of Congress). We discussed how my lab and the students in my classes can start working on visualizing significant digital archives available though the Library web site.

We both agreed that the digitized archive of American newspapers created by The Library via a partnership with by National Endowment of Humanities is a good place to start. Currently the archive contains 4,776,214 pages, and it continues to grow. The pages are digitized at 400 dpi resolution.

A group of UCSD undergraduate students who were taking my 2011 Fall class on big cultural data, visualization and digital image processing (current syllabus) figured out how to download high-res images of newspaper images and metadata using Library API, and started working on visualizing a number of newspapers. We will be putting a page with this project's results on softwarestudies.com soon. Today we are releasing one of the animated visualization created by UCSD undergraduate student Cyrus Kiani (embedded on the top of the post).

Kiani's animation uses 5930 front pages from The Hawaiian Star covering 1893-1912 period. This period is particularly important for the development of modern visual communication (development of abstract art which leads to modern graphic design, the introduction of image oriented magazines such as Vogue, new medium of cinema, invention of phototelegraph, the first telefax machine to scan any two-dimensional image, etc.)

The animation of 5930 front pages of the single newspaper published during these 20 years for the first time make visible how visual design of modern print media changes over time, in search of the form appropriate to the new conditions of reception and new rhythm of modern life.

Here is Kiani's other visualizations and analysis of this data set (click on the image to view original version (3000 x 10650 pixels) on Flickr:

Visit softwarestudies.com to see our other digital humanities projects, and to download the free software tools we developed for visualization of large image archives.

The state of Wikipedia infographic

2012-03-15T09:54:00.000-07:00

Jen Rhee's infographic - How Wikipedia redefines how people do research.

my favorite data point: %56 of students will halt research if little information is found on wikipedia.

Via: Open-Site.org

more museums put their collections online

2012-03-07T09:30:00.000-08:00

A screenshot of SFMOMA ArtScope interface to their image collection developed by Stamen Design.

When we started our lab in 2007, we expected that in a few years massive sets of images of artworks (and other areas of visual culture) will start become available from major cultural institutions. So we focused on developing methods and techniques for their analysis (what we call cultural analytics), while waiting for these collections to become available. The wait is almost over - these collections are here. (Unfortunately right now museum interfaces often tell web visitors how many "objects" they have in their database, but not how images, so some of the numbers below are approximate.)

Examples:

MoMA - currently 33593 images online
www.moma.org/explore/collection/

Brooklyn Museum - probably 97,000 images online (their interface says that they have that many records but does not explain how many images they have)
http://www.brooklynmuseum.org/opencollection/collections/

SFMOMA - all 6,038 works in the collection are online:
http://www.sfmoma.org/projects/artscope

BBC Your Painting - currently already 110,000 images online from UK national collections, with the aim to reach 200,000
http://www.bbc.co.uk/arts/yourpaintings/

Cleveland Museum of Art -
currently 65,000 images online.

Whitney Museum, NYC:
not clear how many images are online but looks like a lot

Powerhouse Museum, Sydney - somewhere between 98,000 and 170,000 images online (their interface does not explain how many images they have)
http://www.powerhousemuseum.com/collection/database/

More and more museums offer their data via APIs:

List of museums offering API

The first museum API was developed some time ago by Powerhouse Museum - currently it offers access to 90,000 thumbnails:
http://www.powerhousemuseum.com/collection/database/download.php

new Vroom scalable display is ready for cultural analytics research

2012-03-06T11:04:00.001-08:00

New super high resolution scalable display installed in Calit2 Vroom. The system consists from 32 individual displays arranged in 4 x 8 configuration.

The very thin bezels of individual screens make the system suitable for exhibitions and adoption in museums and other cultural venues. To test capabilities of this new system, we loaded our complete image set of 4535 Time magazine covers (1923-2009). Using our software, we can sort the images set according to different criteria. We can also select any single image or a group of images, and enlarge for a closer look. We are looking forward to using the new system for our cultural analytics projects.

QTIP software: analyze image collections with the speed of the light

2012-02-29T09:29:00.001-08:00

QTIP is a free digital image processing application. It was developed by Multimodal Analysis Lab and National University of Singapore) and Software Studies Initiative at University of California, San Diego.

Download it from software page of our lab blog.

Use it to process your image collection and then visualize the collection with our free ImagePlot tool.

Example: using QTIP with ImagePlot to compare 580 van Gogh paintings (left) and 580 Gauguin paintings (right). In each plot, X-axis = median brightness, Y-axis = median saturation.

Since 2008, Software Studies Initiative (softwarestudies.com) has been developing a computational approach to working with big visual data sets which we call cultural analytics. We automatically analyze image or video collections using digital image processing, and then use the results to render high resolution visualizations which show all images in a collection sorted according to their visual attributes and metadata. This allows us to efficiently "look inside" large visual collections, identifying patterns and seeing the whole "landscape" of the data.

After refining our software on internal projects, in September 2011 we released ImagePlot visualization tool. The download also includes ImageMeasure macro which analyzes basic visual features of images in a collection (brightness, saturation, hue and number of shapes of every image).

Today we are making available a more powerful free digital image processing application:

QTIP: analyze image collections with the speed of the light.

QTIP stands for QTImageProcessing. Written in Java, QTIP uses OpenCV and is very fast. Fire it up, select a directory containing images, and watch it go through it with an amazing speed. The output is a single file (.csv) containing image filenames and 49 extracted features per image.

For example: we downloaded 178,930 images from Flickr group "Graphic Design." On our iMac (Fall 2011) with 16 GB of RAM, the application processed all these images in 15 minutes (192 images per second). On Mac Air (Fall 2011, 4 GB RAM), processing the same 178,300 images took 28 minutes (106 images per second.)

Once you process your image collection with QTIP, use the results file with ImagePlot to explore the collection and make discoveries.

NOTES:

1. If you are processing a directory which contains lots of images, it may take a few seconds before you see QTIP beginning to read images - just wait.

2. QTIP can process multiple image directories located inside a user-specified input directory.

3. One of the most useful features output by QTIP is the number which indicates if a given image is color or black and white. For the description of this and all other features, read program documentation.

4. The results file contains two columns called "titleID" and "bookID." Disregard these columns. (The program was originally written to analyze our one million manga pages data set which was divided into titles and books.)

5. If QTIP encounters a corrupt image, it will still create a row for this image containing filename and an error flag. You need to remove these lines before using these results with ImagePlot. Do a sort on the filename using "bookID" column and then delete the lines which have error flag. In our experience, image sets donwnloaded form social media sites (Flickr, deviantArt) often contain some corrupt images.