Lev Manovich’s Software Studies Initiative collected and analyzed 2.3 million Instagram images from New York, Tokyo, Bangkok, San-Francisco and eleven other global cities. This allowed the lab to observe the temporal rhythms that define the life of big cities: how different is the life of one city as compared to the other, when does Manhattan go to sleep and when it wakes up, etc. Ahead of his lecture at “Strelka” Institute on June 10, Professor at Graduate Center at The City University of New York (CUNY) Lev Manovich explained to Strelka Magazine what we can learn about human cultures by analyzing Big Data.
— The Verge magazine named you among 50 people who will shape the next 50 years. In what way, do you think, your work defines the future?
— I was pleasantly surprised to see a mention of not just my latest projects, but also of some of my earlier theoretical work dedicated to new digital media in the brief bio provided by the magazine. Back then I was studying how the new media was reshaping contemporary architecture, design, cinema, animation, and other cultural forms. I was able to show that digital media is not just a new phenomenon, it’s actually a part of a cultural processes that have been developing for centuries. I think that’s what attracted Verge magazine: researching cultural processes in this way makes it easier for us to predict or at least to understand what technological culture is going to be like in 10-20-30 years — we can extrapolate the historical trajectories.
As for my latest work, in 2005 I realised that all this massive content shared by online users on social media networks offers vast new opportunities for studying culture. Now we can work not just with separate cultural artifacts as is often the case in academia, but with much larger cultural datasets. For example, we can use computers to analyze all Instagram photos posted in a particular city, or all photos in some Flickr group, or all websites of design graduates around the world in order to see the ways in which their imagination is different from place to place, or has changed over (for example) last 10 years.
These are the kinds of systematic studies of cultural processes that my lab has been involved in for the last seven years. Today there are many researchers working on computational analysis of cultural content (books, fashion, pop music, tweets, Flickr images, YouTube video, urban data, etc.) However, the computer scientists are primarily interested in refining algorithms: for example, automatic recognition of objects in photos and photo styles. What makes us different is that we are interested in actually using those algorithms for working with concrete cultural archives and large amounts of online user-generated content. For example, we analyzed MoMA’s collection of 21,000 historical photographs, one million manga pages, one million artworks from deviantart.com, and now working on over 260 million images shared on Twitter worldwide after 2011.
— The modern information space is characterized by the absence of hierarchy: we are all equal when it comes to Internet. Do you think that one day we might get so used to this equality and that we would transfer it into real life?
— When World Wide Web started to develop in the 1990s, this was the point often discussed: how it brings the absence of hierarchy. Both Microsoft and some school kid that no one’s ever heard about could create a website using the same technology and these websites look pretty much the same.
But a lot has changed since then. Many parts of digital culture become rules by monololies. Google is the dominant search engine in many countries and it is also very hard to compete with Facebook or VKontakte — they have far too many users. Companies developed various web technologies that make websites more complex and attractive. But as a result, an ordinary person is no longer able to compete with them. Creating a really good large scale website can cost dozens or hundreds of hundreds of thousands dollars.
But of course our habits have also changed: today anyone can send Twitter or Facebook message to anyone (although it does not mean that you will get the answer). Each week I get emails not just from professors, but also from students from all over the world. Recently I even received a letter from a high school student offering to work on our lab’s projects to get experience.
But you see, online world cannot change everything — some features of human societies such as competition, hierarchy, and inequality will probably always remain. All physical, biological and social systems that we know have their own hierarchies. So probably such mechanisms are so vital to the functioning of the systems that nothing could alter them in a fundamental way.
— How much can the results of your research tell us about the offline world?
— It will be easier to answer by providing an example. One of the most challenging projects that we have been working on for 5 years is the analysis of one million artworks shared by people on DeviantArt. It’s a famous website, the biggest social network for non-professional artists sharing their work. Today people use Pinterest, Instagram and Tumblr for this, but in 2000s it was DeviantArt. It has 35 million artists and hundreds of millions of works. It’s interesting that despite the fact that this is a non-professional network, people do not share just anything. Those artists might not be a part of the official art world, but you can find plenty of really impressive works here which are just as interesting as those displayed in commercial galleries. For example, I discovered on DeviantArt a girl from Shanghai who started posting her photographs when she was 16 and is now making photos for Vogue.
By analyzing millions of artworks we can ask ourselves all sorts of questions: what is “art” today, how has it changed in the last ten years, what system of categories is used by people to classify their creations? In fact, the reason we started on this project was a very rich system of categories developed by DeviantArt community: over 1700 independent categories organized in a hierarchical system with up to 7 levels deep.
Of course many of these these people would never exhibit in big commercial galleries. For this they would need to go to an art school, get a degree, start their art career, make sure it’s a steady one, exhibit their work every year, etc. No one wants outsiders. And this is just another example proving that differentiation will always remain. Just like communist society with economic equality has never happened, it’s unlikely that we would ever have a society with total democracy in terms of taste and access.
— Two years ago when talking about your project Phototrails, you mentioned that you would like to find out whether Instagram reflects real life or only itself. You refered to the famous Marshall McLuhan idea “The medium is the message”. So what are your conclusions so far?
— You are right, I was wondering what Instagram is: a window into a different reality or only its own reality. A message or a medium? I was referring to the well-known concept by McLuhan that the medium is the message. People working in social sciences or in journalism use Instagram to understand what is happening in the world. For example, there’s a revolution somewhere and we can see what the participants or witnesses are posting and therefore make certain conclusions abut what is going there. This is Instagram as message. But now think about selfie phenomena that would not exist without services such as Instagram. So what interests me is to what extent social media is not just a window into reality, but also shapes its messages, sometimes creating new types of content that are specific to it.
Take Twitter. It has taught us to describe events in just 140 characters, including hashtags and links — this is a lot of limitations. And people are not just writing something, they are formulating their thoughts in a particular way, using specific phrases. So in this sense I think that media is not just communicating the message, it is also altering the content. In reference to McLuhan I wanted to ask: if Instagram is a window into reality, used by millions of people to post photographs of their lives choosing specific poses and filters, staging a particular version of themselves, could it be used as an instrument in visual anthropology? Is Instagram an appropriate tool for studying cultural, economic and social life of different countries?
— Did you find the answer?
—After Phototrails my lab and I worked on a project about Kiev — The Exceptional and The Everyday: 144 Hours in Kyiv (2014), for which we collected and analyzed images (along with their locations and tags) that were posted in center of Kiev during the key week of Maidan protests. We also created a project called On Brodway that used 660,000 Instagram photos shared along Broadway in NYC during five months in 2014. We divided 21 km of Broadway street in Manhattan into 713 30-meter segments and compared photographs from each segment. Broadway is a real cross section of the society: it goes through many different New York neighborhoods — from Wall Street to Harlem and beoynd. After analyzing all these images, we could see that they actually reflect the sociocultural differences between neighborhoods really well.
So after all these projects, I realized that, on one hand, Instagram as a medium does influence the contents of the messages — for example, we don’t see that many people smiling in real life. On the other hand, we were able to see specific content characteristic of certain social groups, countries and places. Selfies in New York, Moscow, Sao Paulo and Tokyo are not the same. People have different behavior codes and codes for taking photographs.
— Moods, biological rhythms, sociocultural differences can be grasped, but what remains unintelligible? Or anything can be expressed?
— It all depends on the method in question. Working with big data means turning cultural into numbers and categories. Modern algorithms now allow us to automatically determine photo techniques, describe the represented objects, the type of scene, and so on. But it’s a very rational industrial approach to images. Not everything can be extracted and counted with algorithms. Tolstoy or Proust were able to represent feelings of a person by describing a landscape, or changing rhythm of the prose, etc. Can contemporary algorithms for culture analysis catch these nuances?
— Have you thought about working with Russian data?
— That would be really interesting. I recently met a young woman who’s working on her PhD in Germany and wants to study the conversations about Russian Orthodox Church by analyzing all relevant parts of all Russian blogs from the last ten years. I think that when I return from this trip I would be very inspired and would able to decide what is it exactly that I want to study.
As you may know, media is used in a very distinct way in Russia. Scrolling through comments section under any Russian film on YouTube is not very pleasant, because many comments are very rude, to say the least. I am interested in the differences between use of social media in Russia and Western countries. In America it’s all very polite: people treat Twitter as a public space for discussion and are very careful when expressing their political views. In Russia many people often write whatever comes to their mind. But I am not sure if the use of images on sites such as Instagram is that different, and it would be interesting to study this question in more detail. In our Selfiecity project we found differences in how young people in Moscow pose for their selfies, but this is only one kind of image.
— How can an urban designer use the results of your research?
— Usually, when reflecting on the city, architects start with static notions: a street, a district, junctions, roads. But if we look at the city through the optics that we are offering, it would appear as a blurry pixelated image. There’s no New York or Moscow — only hundreds of millions of dots. Those satellite images allow to observe it from a distance — you cannot see any labels. Our method makes it possible to create new types of metrics for understanding the structure of urban environment. For example, instead of talking about density of population on a certain street, we can discover an unexpected location frequented by this population.
“On Brodway” project gave us a different city — a portrait of various layers of the society. Tourists, people living, working or going out in particular neighbourhoods were all divided into separate groups. We were able to see what and when they were photographing. This, in turn, allowed us to see which places they were visiting and how they used space. I think it is directly related to urbanism. If you are an architect, you would be interested in knowing how people would be using a neighbourhood that you designed.
— Could you formulate in one sentence what is the essence of modern culture?
—Behind all my big cultural data research there are a couple of meta-ideas, and one of them is to avoid generalizations. For example, “the modern person is alienated” or “the modern person is connected” or “social media is banal,” or “Instagram is only food pictures and selfies.” There are eight million people living in New York and they are all different, in spite of existing patterns and similarities. We are able to compare of cultural activities of hundreds of millions of people and find all kinds of differences. At the same time, while every person is unique and every posted image is also unique, we can still form clusters (posts which are similar in some ways) — 300 clusters, or 3000, or maybe even 30000. In fact, there is no theoretical reason to select a particular number of clusters – it all depends on how many differences you want to map out.
The main drive behind my research is refusal to give simple answers. For the first time we have an opportunity to study and represent the diversity of human experience, or at least the part of it which is reflected online. Instead of one answer we can have a zoomable map of the world with categories to pick from — mode of communication, choice of location or clothing, etc. — and then we can make our own conclusions. And this is what the work of my lab is all about.