header image

Georgetown University
Graduate School of Art and Sciences
Communication, Culture & Technology Program

CCTP-748: Media Theory and Meaning Systems
Professor Martin Irvine
Spring 2018

Course Objectives

This course will introduce an interdisciplinary approach to working with the key concepts and methods for the study of media, communication, information, and symbolic systems:

  • media and communication theory and the study of the digital media environment
  • linguistics, semiotics, and meaning systems
  • cognitive science approaches to meaning and communication: symbolic cognition and role of "cognitive technologies" in extending, externalizing and "off-loading" core human capabilities for making meaning and communicating in symbol systems
  • mediology and actor-network theory as approaches to technical mediation
  • computation, software, and re-mediation through digital media representations and transformations

We will use the seminar as a laboratory introducing and testing the main concepts and methods from several intersecting disciplines and sciences, and build knowledge step-by-step.

The guiding question of the course: how can we use our interdisciplinary resources to develop a media and communication theory for all the forms of media, software-produced media artefacts, and computational environments that we experience today?

The course has a methodological emphasis in allowing students to gain competencies in building and applying interdisciplinary models for investigating complex systems of media, technical and social mediation, and the meaning of symbolic artefacts in any technical form of representation (from writing and images to digital multimedia).

Central topics for study will include the continuum of human symbolic systems and technical mediation, the implications of the converging pan-digital platform for all media, the analog-digital continuum, the question of "big data" and the collective memory of computational networks, and the ongoing renewal of media content through new technologies of digitization.

Expectations for Participation

Students will be expected to create the seminar in real time through readings, discussion, and proposal of cases and examples for study. Grading and weekly seminar assignments will be based on student presentations, short weekly response essays in the course Wordpress site, and a final research or creative project involving a student's own synthesis of approaches, methods, and concepts in the seminar as applied to interpreting and analyzing a contemporary or historical case.

Classroom location: Car Barn 318

Course Format

The course will be conducted as a seminar and requires each student’s direct participation in the learning objectives in each week’s class discussions. The course has a dedicated website designed by the professor with a detailed syllabus and links to weekly readings and assignments. Each syllabus unit is designed as a building block in the interdisciplinary learning path of the seminar, and students will write weekly short essays in a Wordpress site that reflect on and apply the main concepts and approaches in each week’s unit. Students will also work in teams and groups on collaborative presentations.

Grades:

Grades will be based on:
(1) Weekly short writing assignments (in the course Wordpress site) and participation in class discussions. Weekly short essays must be posted by 10:00AM for each class day so that students will have time to read each other's work before class for a better informed discussion in class. Weekly participation may include group collaborative presentations in class (to be determined by course members). (50%)
(3) A final research project written as a rich media essay or a creative application of concepts developed in the seminar (50%).
(Final projects will be posted on the course Wordpress site, which will become a publicly accessible web publication with a referenceable URL for student use in resumes, job applications, or further graduate research) .

Professor's Office Hours
Tues. and Wed. 12:00-2:00, and by appointment. I will also be available most days after class meetings.

Academic Integrity: Honor System & Honor Council
Georgetown University expects all members of the academic community, students and faculty, to strive for excellence in scholarship and in character. The University spells out the specific minimum standards for academic integrity in its Honor Code, as well as the procedures to be followed if academic dishonesty is suspected. Over and above the honor code, in this course we will seek to create an engaged and passionate learning environment, characterized by respect and courtesy in both our discourse and our ways of paying attention to one another.

Statement on the Honor System
All students are expected to maintain the highest standards of academic and personal integrity in pursuit of their education at Georgetown. Academic dishonesty, including plagiarism, in any form is a serious offense, and students found in violation are subject to academic penalties that include, but are not limited to, failure of the course, termination from the program, and revocation of degrees already conferred. All students are held to the Georgetown University Honor Code: see https://honorcouncil.georgetown.edu/system.

Instructional Continuity
In the event of a disruption of class meetings on campus from inclement weather or other event, we will continue the work of the course with our Web and online resources, and will arrange for online discussions and meetings with the professor by using the Google Hangout interface in our GU Google apps suite. I am also always available via email, and respond to student messages within a few hours or less.

Required Books:

  • Floridi, Luciano. Information: A Very Short Introduction. Oxford, UK: Oxford University Press, 2010. ISBN: 0745645720
  • Lev Manovich, Software Takes Command: Extending the Language of New Media (London; New York: Bloomsbury Academic, 2013).

Recommended Books

  • W. Brian Arthur, The Nature of Technology: What It Is and How It Evolves. New York, NY: Free Press, 2009. [ISBN 1416544062]
  • Peter J. Denning and Craig H. Martell. Great Principles of Computing. Cambridge, MA: The MIT Press, 2015. 
  • George Dyson, Turing’s Cathedral: The Origins of the Digital Universe. New York, NY: Pantheon/Vintage, 2012. ISBN: 1400075998

Links to Online Resources

Learning Objectives:
Introduction to Media, Communication, and Meaning Systems

Consider how these terms, topics, and domains of knowledge form a (often unquestioned) concept cluster:

  • Medium, mediation, media, "the media," mass media
  • Communication
  • Information
  • Language, natural language
  • Mathematics and abstract symbol systems
  • Meaning Systems: Signs, Symbols, Symbolic Processes, Representations
  • Symbolic cognition, cultural meaning, collective cognition, memory
  • Technology, "Information and Communication Technologies" (ICTs)
  • Computation, Software Code, Digital Media
  • Interface(s)
  • Human-Computer Interaction
  • Digital and Analogue (as different states and as a continuum)

These concepts and terms are part of long histories of technologies, media, discourse, disciplines, research agendas, and knowledge paradigms, and have been defined in separate disciplinary domains. This concept cluster is inseparable from our current media and communications environment.

We use many of these terms every day without thinking through what they entail, what kinds of unquestioned assumptions are behind them, how they direct thinking and define the way we can pose questions. We need to investigate the consequences of holding various embedded assumptions in these discursive fields for real-world thought and action. In this course, we will consider the big picture of theory and leading concepts and provide ways to critique and work out new interdisciplinary directions.

These combined and overlapping areas of research and theory now constitute one large interdisciplinary field. Academic departments and professional organizations sustain the demarcations of practice, disciplinary identity, and knowledge legitimization. Media and technology theory opens up the understanding all the codes we live by: text, images, photography, film/video, TV, advertising, Internet and Web media, and the whole mediasphere in the digital environment.

The Big Questions for getting oriented:

  • What is a "Medium" and what/how does "it" "mediate"?
  • How many kinds of media are there (how do we make distinctions and differentiations), and how are the kinds connected? (Thought experiment: what do we mean by "multimedia"?)
  • Is there an underlying grammar or logic for our forms of media as inseparably linked with human meaning-making, communication, and expressions in sign and symbol systems? (Yes!)
  • How are our sign systems connected to human cognition, representation, material media of expression, and cultural/social identity and memory?
  • How/why are our symbolic systems and material media seemingly inseparable and interconnected?
  • What does our recent ability to combine and merge all forms of media and communication on digital platforms reveal about the nature of symbolic expression and meaning-making?
  • Are computers and mobile computational devices with GUI displays a medium? A metamedium, a medium for representing and interacting with other media? Why does it matter how we understand these technologies?

Introduction to the Course: Seminar Format and Research Tools

  • Seminar format. Requirements, weekly assignments, grades and expectations.
  • Using the Google Drive Etext Library for readings and research sources (GU student login)
  • Required: Learn how to use Zotero for managing bibliography and data for references and footnotes.
    Directions and link to app, Georgetown Library (click open the "Zotero" tab).
    You can export and cut and paste your references into writing assignments for this course.
  • Georgetown Library Online Journal Search (with Off-Campus Login for enrolled students)
    Access to journals in our fields of study for going further in the research literature for a topic.
  • Using the Reference and Research Bibliography
  • Weekly discussions on the course Wordpress site
    • Instructions: How to write and post your weekly short essay / discussion.

  • Seminar Introduction: Media Theory and Meaning Systems (presentation) (Irvine)
    • In class: Introductory Case Study discussion: media examples, film/video, television, Web, apps

Learning Objectives and Topics:

Through an introductory "big-picture" overview of some of the major fields and approaches that have traditionally contributed to interdisciplinary configurations of our field, students should :

  • Become familiar with the major disciplines that have defined ways of studying communication, media, information, and meaning.
  • Begin discovering important overlooked areas of inquiry.
  • Gain an entry point for building their own interdisciplinary knowledge base.

Working toward an integrative model for Media Objects (Expressions) in Meaning Systems

By the end of the seminar, we will be able to fill in the background for many of the components of this integrative model for studying and analyzing any media object in any meaning system that we want to explain:

Media Theory

Graphic: Martin Irvine

This week we will survey the main disciplines and fields involved in this research, and define our "problem space" and the main group of intersecting disciplines, sciences, and methods that we will draw from.

Readings:
A "Big Picture" Overview of Some Major Approaches, Methods, and Conceptual Models

Survey the following readings for an overview of some of the main foundations in media and communication studies -- as we have inherited the way disciplinary approaches frame and define their objects of inquiry and knowledge domains.

We will reference these ideas and approoches throughout the course. Just survey these texts to get a sense of these approaches for this week.

  • Surveys of topics and the formation of the traditional discipline of communication and media studies
    • Denis McQuail, Mass Communication Theory. 6th ed. 2010. (excerpts).
    • [A widely used textbook for an introduction to the topics considered in a college communications school or department. Consider the underlying assumptions and the way the field is constructed and legitimated. What's missing from a CCT perspective?]
  • Meanings of the terms "media" and "medium"
    • Joshua Meyrowitz, "Understandings of Media," revision of article originally in Culture, 11 (3-4), Spring/Summer 1997, 5-7. [Useful short essay on the conceptual uses of the term "media."]
    • Joshua Meyrowitz, “Medium Theory.” From Communication Theory Today, edited by David Crowley and David Mitchell, 50-77. Stanford, CA: Stanford University Press, 1994.
  • An interdisciplinary model for a social-technical theory and research program: mediology
    • Régis Debray, "What is Mediology?Le Monde Diplomatique, Aug., 1999. Trans. Martin Irvine.
    • " Introduction to Mediology" (Jean Gagnon, Daniel Langlois Foundation for Art, Science, and Technology)
    • [Debray's approach shares many of the same assumptions as Bruno Latour's "systems and networks" model in "Actor-Network Theory" (which also developed in the context of French and European sociology, political philosophy, and semiotics).]
  • Overview of Meaning Systems (Humanities semiotics approaches)
    • Daniel Chandler, Semiotics: The Basics. 2nd ed. New York, NY: Routledge, 2007. Excerpts. Survey the topics and main definitions through the section on Peirce (up to p. 35) for this week.
      [This basic introduction (targeted for undergraduate humanities students) is useful for the key terms and sources of concepts that are known in academic study from the tradition of De Saussure and French structuralism. We will clarify our own set of terms and extensible concepts from the tradition of C. S. Peirce for analyzing the complex combinations of symbolic systems used in all kinds of ordinary expressions and all forms of media and communication.]
    • Stuart Hall, "Encoding, Decoding." From The Cultural Studies Reader, edited by Simon During, 2nd edition., 507-17. London; New York: Routledge, 1999.
      [This is a classic essay in the "cultural studies" approach to media and communication as social and ideological systems. Hall's merging of semiotics and communication theory is widely practiced in social critique methods. This essay has become so famous it even has its own Wikipedia page. We will refer to this approach and also critique its limits and need for a more complex model of levels of analysis.]

In class: Group discussion and Prof. Irvine's background lecture.

  • We will use our discussion time in class to clarify the meanings of our key terms and concepts.

For student discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • Read first: instructions for writing your weekly discussion posts.
  • The purpose of this week's readings and topics is to provide a big picture survey of some of the traditional, institutionalized disciplines, fields, sciences, and methods that converge on our main topics and "problem set" -- media and meaning systems. This isn't an attempt at "coverage" or (impossibly) coming to terms with all the relevant disciplines and sciences in one view, but an overview of some main fields and approaches that have defined, framed, and constructed the topics, objects, and methods of the disciplines. In the coming weeks, we will go on to complete the picture with more recent research and theory this is now changing the way we can approach our topics of inquiry.
  • As a first step in the interdisciplinary knowledge-building goals of this seminar, refer to two or three of the background readings for describing one or two intersections or points of convergence around a common core problem or research question. What important concepts and distinctions stand out for you in the ways that the terms "medium/media" and "mediation" have been, and are, used in different fields, disciplines, and discourses? Do you see any incompatible (or even mutually contradictory) approaches, theories, or methods that must be recognized and considered in working with interdisciplinary methods?

Learning Objectives and Topics:

An orientation to approaches for our major questions about the human capacity for symbolic thought and media technologies that are the focus of ongoing research across many sciences and disciplines:

  • What does it mean that our "higher" thought processes, communication with others, and social identities are mediated through sign systems and material media, and
  • How is the emergence of the human "symbolic faculty" in language and other symbolic systems and material artefacts closely linked to the development of technology, especially the history of media technologies?
  • If our symbolic capacities -- and all that follows from them historically -- are distinctly human, then making tools, machines, material media, and automated processes in computation are artefacts of human cognition and need to understood by everyone and not as alienating technology or as "human vs. machine" oppositions.

We'll begin building a foundation through the research generated by the study of symbolic cognition in intersecting disciplines and sciences -- linguistics, anthropology, evolutionary psychology, and cognitive science more broadly -- and how this developing knowledge base allows up to ask better informed questions about technology and mediated communication systems. Learning the main topics of research and debate in cognitive science fields on mind-brain / machine-computer parallels and why the models for computation and cognition have been so interconnected.

Research in many fields continues to discover more and more about the consequences of being the Symbolic Species in Terrence Deacon's term. This week, you will learn the main concepts from cognitive science research for describing the "cognitive continuum" from language and symbolic representation to multiple levels of abstraction in any symbolic representation (spanning writing, mathematics, symbolic media like images and combinations in film and multimedia, and computation and code). The human ability for making meaning in any kind of symbolic expression and embodying meaningful, collectively understood expression in media technologies depends on the use of symbols and the advantages of symbolic cognition.

We will follow a major direction in recent research that studies media, communication, and computational technologies in a continuum of accumulating cognitive advances that have enabled a cascading series of human capabilities afforded by symbolic cognition: the ability of abstract thought and the reflexive use of symbols, off-loading cognitive tasks and memory by extension in artefacts, externalized media storage, and automated computational processes.

Overview of Topics and Themes:
Symbolic Cognition, Sign Systems, Mediation > Cognitive Technologies


Within a broad cluster of fields--ranging from neuroscience to cognitive linguistics, cognitive anthropology, and computational models of cognition and artificial intelligence research--there has been a major convergence on questions and interdisciplinary methods for studying cognition, human meaning-making and the "symbolic faculty" generally, including all our cumulative mediations and externalizations in "cognitive technologies." Cognitive science has been closely related to computer science in seeking computational models for brain and cognitive processes, and proposing hypotheses that explain cognition as a form of computation (the "computational theory of mind/brain"), and attempts to model complex parallel software processes on what we know about the neural bases of cognition ("cognitive computing").

For the study of human symbolic cognition and meaning making in sign systems, many disciplines now converge around the major question of how our technically mediated meaning systems function with analogous and parallel "architectures" that must include (1) rules for combinatoriality of components (an underlying syntax for forming complex and recursive expressions of meaning units), (2) intersubjective preconditions "built-in" to the meaning system for collective and shared cognition, and (3) material symbolic-cognitive externalizations (e.g., writing, images, artefacts) transmitted by means of "cognitive technologies" (everything from writing to digital media and computer code) which enable human cultures and cultural memory. This recent interdisciplinary research is a "game changer" for the way we think about human communication and media technologies.

Synthesizing views, we can say that the human symbolic faculty has generated a continuum of functions from language and abstract symbolic thought to machines, media technologies, and computation:

symbolic faculty

The mainstream disciplines in communication and media studies are very conservative (remaining within a demarcated field in the humanities and social sciences) and have not yet incorporated recent advances in cognitive science fields that are directly relevant to core assumptions and research questions on language, symbolic culture, and media technologies. We therefore have an open opportunity to learn from cognitive science fields and reconfigure the inter- and transdisciplinary field in promising ways for both theoretical and applied work.

Readings: How Did We Develop Media Technologies from our Symbolic Capacity?

  • Kate Wong, "The Morning of the Modern Mind: Symbolic Culture." Scientific American 292, no. 6 (June 2005): 86-95.
    [A short accessible article on the recent state of research on the origins of human symbolic culture and the relation between symbolic cognition and tool making and technologies.]
  • Merlin Donald, "Evolutionary Origins of the Social Brain," from Social Brain Matters: Stances on the Neurobiology of Social Cognition, ed. Oscar Vilarroya, et al. Amsterdam: Rodophi, 2007.
    [Donald's research is a major contribution to evolutionary brain science, and this short article is a summary of his key findings and conclusions. Although his argument about the linking of all symbolic capabilities to mimetic cognition (the ability to imitate and represent others' activity in a social group) is debatable, the central questions he confronts are important for any theory of human symbolic cognitive development. Note the leading hypothesis about the connection between symbolic thought, sociality, and technology.]
  • Colin Renfrew, "Mind and Matter: Cognitive Archaeology and External Symbolic Storage." In Cognition and Material Culture: The Archaeology of Symbolic Storage, edited by Colin Renfrew, 1-6. Cambridge, UK: McDonald Institute for Archaeological Research, 1999. 
    [Important argument by a leading cognitive archaeologist about the evolutionary origins of the symbolic brain: material media for symbolic storage are part of the human cognitive-symbolic processes that lead to the known history of media and technologies.]
  • John C. Barrett, "The Archaeology of Mind: It's Not What You Think." Cambridge Archaeological Journal 23, no. 01 (2013): 1-17. 
    [A valuable summary of the "state of the question" from many schools of thought and research on the convergence of evolutionary sciences and archaeology on understanding the origins of symbolic cognition and use of artefacts. Do your best to follow the contributions of the major schools of thought and central hypotheses: this is directly relevant to the way we think about modern electronic and computing technologies.]
  • Michael Cole, "On Cognitive Artifacts," From Cultural Psychology: A Once and Future Discipline. Cambridge, MA: Harvard University Press, 1996. Connected excerpts.
    [A good summary of the cognitive psychology background that provided important concepts and assumptions in Human-Computer Interaction/Interface design theory as it developed from the 1960s-2000s. Cole was part of the Harvard Cognitive Science program led by Jerome Bruner and others, a school of thought with a wide influence in education, philosophy, psychology, anthropology, and design theory in computing.]

Optional: For Further Research

  • Terrence W. Deacon, The Symbolic Species: The Co-evolution of Language and the Brain. New York, NY: W. W. Norton & Company, 1998. (Excerpts from chapters 1, 3, 9, 11, 13.)
    [Deacon's work initiated a wide and ongoing interdisciplinary debate. Connecting the research from evolutionary brain science, anthropology, archaeology, and paleontology, he presents an important argument for the co-evolution of the human brain with language and symbolic cognition that then enabled human cultures, tools, and technologies. Read for his main argument about language, symbols, and brain co-evolution. When following his main points, remember that we're not so much concerned about what is "right" or "wrong," but about the new cross-disciplinary synthesis and how it opens up ways to think about the long continuum of human symbolic thought and technologies. (There continues to be lots of research on all these topics, if you want to go further).]
  • Itiel E. Dror and Stevan Harnad. "Offloading Cognition Onto Cognitive Technology." In Cognition Distributed: How Cognitive Technology Extends Our Minds, edited by Itiel E. Dror and Stevan Harnad, 1-23. Amsterdam and Philadelphia: John Benjamins Publishing, 2008.
    [We will discuss this interdisciplinary approach to symbolic cognition, media, and technology in coming weeks, but you can get an overview here. Again, don't worry that it doesn't seem understandable at this point; we will build up to it.]

Video Documentary: Earliest Surviving Cave Paintings (will discuss in class)

  • Werner Herzog, Cave of Forgotten Dreams. IFC Films/Sundance, 2011. Scenes from documentary.
    [Extraordinary film capturing 30,000-year-old murals in a Paleolithic cave in Chauvet, France. The sophistication of these images correlates with other symbolic activity discovered in the same era across regions in Europe and the near East. We will view clips from a better quality source in class.]

Presentation (in-class discussion): Prof. Irvine, "Cognition, Symbols, Meaning" (Part 1)

 

For discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • What are some of the main hypotheses and research conclusions (so far) in the research literature above for the question of how we have evolved as a uniquely "symbolic species" with language, other symbol systems, and cognitive technologies for representing and storing symbols?
  • What are the consequences of this research and major working hypotheses for larger historical views of language, communication, symbolic cognition, and media technologies?
  • Do any of these recent research findings provide new ways for you to understand recent technologies in a longer continuum with evolving human symbolic capabilities?

Learning Objectives and Main Topics:

Learning the main concepts, terminology, and assumptions developed in contemporary Post-Chomsky linguistics as essential foundations for understanding and describing language, and by extension, all human symbolic capabilities, sign systems, and communication.

The Importance of Linguistics for Interdisciplinary Thinking and Research

The human capacity for language is the starting point of many disciplines and research programs in all aspects of communication, symbolic culture, and media. Many of the research questions and most of the terminology for the study of language and human symbol systems has been established by the various specialties of modern linguistics. The terms and categories for analysis in linguistics have also been widely used by other disciplines, and all students studying media, communication, and computation need to be familiar with the concepts and research programs in the major branches of linguistics.  We can only do a top-level overview here, but familiarity with the core concepts and research questions will allow you to advance to other questions and topics in your own research.

Why Learning Well-Researched Linguistic Methods are Important for All Our Fields

In CCT and beyond, we find many areas of intensive inquiry on all human communication and meaning systems in many disciplines covering the whole history of meaning systems and symbolic representation from the earliest records of symbolic expression to computational language processing and artificial intelligence. Many fields assume that the meaning systems of interest to the field (e.g., text, images, music, designed artefacts, multimedia, computer code) work "like a language" or "are a language" without clearly defining what a language is: what are the properties or features of language or a language for which other meaning or media systems can be like?

Therefore, it is essential for your learning and thinking to build up your repertoire of terms and concept-models with the current accepted, scientific vocabulary for descriptions of the components of language and other symbolic systems. The terms and concepts developed in contemporary linguistics and cognitive science are now the standard currency of any field dealing with language, communication, and sign systems (including computer science and all fields dealing with designs for interfaces and digital media representations. In short, follow Yoda's advice below!

For studying other symbolic systems, you can see that it's fundamental for us to begin with a model or description of language that is as clear and well-described as possible before assuming that other meaning systems can be "like" (a) language. Contemporary linguistics provides a knowledge base of key concepts and terms for the descriptive levels, rules, and constraints that specify how language works and what a natural language is. Getting as precise as we can at the detail levels will help us think more clearly about important questions like, "are visual image genres a language?", "what do we mean by a 'computer language', code, and 'language processing'?", "is music a language; and/or individual music genres?", "how do we understand film, video, and multimedia genres with multiple combined 'languages'?", "how is language connected to intelligence, symbolic cognition, and other forms of human cognition"?.

Key terms and concepts: 

  • lexicon, syntax
  • generative grammar, open rule-governed combinatoriality
  • semantics, conceptual semantics
  • pragmatics (the contexts and situations of language use, shared assumptions, speech and discourse genres, and speech acts)
  • sociolinguistics (language in everyday use and language in social group formation and identities)
  • cognitive linguistics (the intersection of language and cognitive/brain science research)

yoda

 

Reading and Video Introductions (review and read in this order):

  • Steven Pinker, Video presentation on Language and the Human Brain (start here)
    [A well-produced video introduction to the current state of knowledge on language and cognitive science from a leading scientist in the field.]
    See also: Steven Pinker's website at Harvard University for a view of his work and career.
  • Martin Irvine, "Introduction to Linguistics and Symbolic Systems: Key Concepts" (intro essay).
  • Andrew Radford, et al. Linguistics: An Introduction. 2nd ed. Cambridge, UK: Cambridge University Press, 2009. Excerpts.
    [This is an excellent text for an overview and reference. Scan enough in each section to gain familiarity with the main concepts. Focus on the excerpts on Introduction to Linguistics as a field, and sections on Words (lexicon) and Sentences (grammatical functions and syntax).]
  • Steven Pinker, "How Language Works." Excerpt from: Pinker, The Language Instinct: How the Mind Creates Language. New York, NY: William Morrow & Company, 1994: 83-123.
    [Accessible introduction to linguistic analysis of syntax and the role of language in cognition.]
  • Ray Jackendoff, Foundations of Language: Brain, Meaning, Grammar, Evolution. New York, NY: Oxford University Press, USA, 2003. Excerpts and introduction to "the Parallel Architecture" model of Language.
    Also included in these excerpts is Jackendoff's extensive bibliography of references cited in the book.
    Read pp. 3-17 (from Chap. 1), and 38-44 (from Chap. 3); survey what's in Chap. 5 (on the Parallel Architecture): we will continue with Jackendoff's model next week.
    • Jackendoff has developed a useful synthesis of many important developments in linguistics in a "unification" model of language components that he calls the "parallel architecture." This book is a highly detailed, argumentatively nuanced, exhaustively researched compendium of the major issues in linguistics (especially from the cognitive and generative approaches) that is difficult to summarize, but there are important "take aways" for us. In reading Jackendoff's work, you will be jumping in midstream to a highly-detailed 30-year ongoing debate about syntax and semantics, but he usually provides a good orientation to the issues. He also provides useful conceptual hooks and links for ways to talk meaningfully about the structures of language in relation to other symbolic systems like music and visual representation genres.

Optional/Supplemental: For Further Research and Reference

  • The research literature is massive for the current state of knowledge in linguistics, but these are some very useful summations and reviews:
  • Mark Aronoff and Janie Rees-Miller, eds. The Handbook of Linguistics. Malden, MA: Wiley-Blackwell, 2000. Excerpts.
    [Very useful introductions. See the chapters on Writing Systems, Syntax, Formal Semantics, Pragmatics, and Discourse Analysis.]
  • Carol Genetti, ed. How Languages Work: An Introduction to Language and Linguistics. Cambridge: Cambridge University Press, 2014. Excerpts.
    [This is a broad-based introduction that uses mainly the Functional Linguistics approach, but very open and inclusive for all relevant research methods on the main topics in these excerpted chapters.]

For discussion (in class):

Visualizing Sentence Structure through Computational Parsing (Natural Language Processing)

  • XLE-Web: A sentence parsing tool and syntax tree generator that maps both the "Constituent Structure" and "Lexical Functional Grammar" models of generative grammar. Choose "English" and map any sentence for its constituent (c- ) and functional (f- ) structure! (Uses linguistic notation from two formal systems.) (The term "parse" comes from traditional grammar for breaking sentences down into their "parts of speech" [pars/partes: Latin for "part/parts"]). Syntax parsers are used in computational linguistics and all complex text analysis in software and network applications. Web search algorithms and Siri voice recognition and interpretation software have to use parsers for generating a map of the probable grammatical structure of natural language phrases before processing probable semantic searches. So here you get a visualization of what Siri and Google have to do in milliseconds behind the scenes before the software can initiate a search or other command based on key words.
  • Try any sentence, and try some very complex ones (compound subjects, many conjunctions and clauses, etc.).

Presentation for class discussion:
Irvine, "Language, Cognition, Symbols, Meaning" (Part 2, final section on Language)

 

For discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • Drawing from the readings for this week, how would you answer these basic questions for someone who has little or no knowledge of linguistics as a science or field of research: What is language? What is "a language," What are the essential features that enables a language to be a language? Use as many of the special terms in linguistics as possible and define them.
  • From these foundations, we can go on to ask other important questions:
    What are the implications of using the features of language as the model for other symbolic systems (visual, audio, and multimedia combinations) and for most forms of communication and media? 

    Even before we investigate the knowledge base from sciences and disciplines that converge on the study of other symbolic systems, can you think through what it would mean to study other sign systems by assuming that they work like a language (e.g., making meaningful rule-governed combinations of symbolic elements in expressions that are understood collectively)? Try working with one or two of the main linguistic concepts to see if they provide extensible models for understanding and describing other meaning systems (like visual genres or music genres) in the forms of media and communication we use every day.
  • What seems intuitively clear at this point, and what is difficult or unclear?

Learning Objectives and Main Topics:

The main objective of this week is beginning a synthesis or merger of Jackendoff's Parallel Architecture model of language and C. S. Peirce's model of semiosis, the generalizable model of meaning processes in any symbol system or medium. We will work through the background for developing and applying this interdisciplinary synthesis throughout the course. (See the Introduction to this unit below: "Semiotics 1.0".)


Readings

  • Irvine, "Semiotics 1.0": Introduction to this week's learning unit (read first for orientation).
  • Daniel Chandler, Semiotics: The Basics. 2nd ed. New York, NY: Routledge, 2007. Excerpts.
    [It may help to review this basic introduction for key terms and concepts. We will clarify and expand in class discussion.]
  • Martin Irvine, "Introduction to Signs, Symbolic Cognition, and Semiotics: Part I."
  • Ray Jackendoff, further implications of the "Parallel Architecture" model of language as a combinatorial system.
    • Go to last sections on the Parallel Architecture and Conclusion; continued from last week.
  • Irvine, ed., Signs, Symbols, Cognition, Semiotics: A Reader of Key Texts.
    Read sections 2-7 for this week. We will discuss in class.
    • Note: This reader provides selections from some major, most-often-cited primary source texts on the foundations of thought on language, communication, signs, and symbolic cognition. At the graduate level, it's best to know the major sources of theories and key concepts for yourself first-hand, rather than relying on "text-book" summaries. You can alway return to the primary texts as you learn more about the contexts and history of ideas.

Optional: More Advanced Readings

  • Richard J. Parmentier, Signs in Society: Studies in Semiotic Anthropology. Bloomington, IN: Indiana University Press, 1994. Excerpts.
    • This is a more advanced text from the view of a leading anthropologist, included here to show you the range of applications of Peirce's philosophy. Parmentier does a useful summary of Peirce's main concepts. Don't get stuck in the terms that Peirce invented to try to account for all the categories of sign functions. The key is following what Peirce discovered in the triadic structure of the symbolic-cognitive process. In chapter 1, focus on the first 3 sections (through "Language and Logic").

Reference (for further research):

Presentations (in-class discussion, and/or study on your own):

For discussion (link to Wordpress site)
Use this topic as a way to organize your thoughts and questions about the readings for this week:

  • (In Class: we will have an open seminar discussion on key terms and concepts in the readings and how to use them in your conceptual tool kit.)
  • For your weekly writing: Begin to thinking like a semiotician. Try working with some of the key concepts of semiotics, and C. S. Peirce's terms, concepts, and distinctions, with an example of an expression or genre of representation in one of our sign systems and media. Remember that any example you choose will be an instance -- or cluster of instances (tokens) -- of general types from the sign/symbol systems being used. We will develop the concepts, models, and methods together in class, but you can begin getting a sense of the main concepts and how to apply them.

Learning Objectives and Main Topics:

This week we will continue working out a merger of Peircean semiotics, Jackendoff's Parallel Architecture model, and other related theory for developing ways of describing and analyzing complex meaning systems: a visual media system (an art genre and/or shot sequence in a movie) and musical forms.

We will read and discuss research statements that enable us to bridge and combine theories and methods in the humanities and cultural semiotics fields with ongoing research and theory in linguistics and cognitive science. This combination of disciplinary approaches provides us with many important conceptual and methodological resources unavailable to any one approach or discipline.

Learning Discovery Methods for Analyzing Cases and Examples

The readings this week introduce two widely recognized kinds of methods in media studies and semiotic analysis: the formal view (like the underlying "grammatical" structures investigated in linguistics, the abstract patterns and structures that hold for any instance of a type of expression in a medium), and the historical, complex socio-technical systems view (studying the historical and situated contexts of expressions and media genres). The semiotic method provides discovery principles for these two branches or paths of analysis.

As you begin practicing how to apply our concepts, methods, and models, you will find it helpful to consider how to pose questions that follow these two main kinds of methods as discovery methods:

  • Questions about the formal principles for visual art as a symbolic system; that is, discovering how the underlying "grammar," structures, and genre codes, which hold for any symbolic media form, are instanced in the type of work you are considering. These questions serve the macro-question "how does it mean?".
  • Questions about the social-historical contexts in which dialogism, the dialogic principle, is performed and enacted in actual, situations of expression and response. These questions serve the macro-question "what does it mean, what conversations is it participating in?".

Terms and Concepts to Learn in Precise Definitions for all Media Genres and Sign Systems

  • Compositionality: explains how and why symbolic systems are structured for building up (composing) larger units from minimal units. At both the level of grammatical phrases and primary levels of meaning (semantics), structures are based on the composition of units. This assumes that a larger well-formed expression can be decomposed into its constituent parts for analysis and interpretation.
  • Combinatoriality: the rules, constraints, and procedures for combining units into well-formed structures = syntax or grammar. We don't just make strings of words or throw together a series of random sounds or images, but we use symbolic system-specific rules for combining sign units into meaningful, syntactically organized, wholes. We assume that the combinatorial rules and principles for language, music, and art forms are generative and allow discrete infinity, unlimited expression from finite means.
  • Encyclopedia / Cultural Encyclopedia: the levels of meaning above the Lexical-Semantic that we activate through networks and associations of symbolic meanings and values assumed in a meaning community. We can analyze the function and operation of meanings at two levels (each with internal levels): a more "context-free" dictionary kind of meaning (Lexicon), and the wider symbolic levels (Encyclopedia) based on cultural contexts, dialogic relations to other works and genres, and situated knowledge of the members of a meaning community who create and receive the symbolic expressions.
  • Dialogic / Dialogism: the ground and context of meaning creation in live and ongoing expression in which any expression is necessarily an answer/response to prior expression, and provides, in its structure and meaning contexts, addressability and answerability for others in a meaning community. Dialogism is the underlying principle for Intertextuality, which explains how expressions in human conversations and any structured artefact like text genres, music compositions, art works, and ongoing debates in any discourse domain (politics, science, law) assume prior expressions, statements, styles, and actual quotations from prior and current discourse. The dialogic principle described by Bakhtin is very close to Peirce's model for linking past, present, and future meaning in generative semiosis.

Seminar Assignment (In-Class)

In class, we will form 3 "research teams" for working out applications of these models for media examples that you choose to analyze.

Readings:

Concepts, Methods, and Models for our Approach

  • Martin Irvine, Introduction to Visual Semiotics (revised) (with a case study).
    Read this as an introduction to the method we can building out to apply to any medium. I've included a case study of a Van Gogh painting using a Peircean semiotic analysis.
    This background will help if you choose a visual artwork or visual media genre to analyze.
  • Irvine, "Student's Guide to Mikhail Bakhtin: Dialogue, Dialogism, and Intertextuality."
    [Bakhtin's ideas have contributed to the contemporary cultural semiotics, media theory, and methods in art history. We have been developing the dialogic approach for building contexts of interpretation, and this introduction will provide a clearer conceptual foundation and rationale for this method. Read the introduction and selections from Bakhtin's writings to gain familiarity with key terms and concepts and how to apply them in your own thinking and analyses. (The Wikipedia entry on Bakhtin is good, if you want further background.)]
  • Martin Irvine, "Remix and the Dialogic Engine of Culture: A Model for Generative Combinatoriality."
    In The Routledge Companion to Remix Studies, ed. Eduardo Navas, et al. (New York: Routledge, 2014), 15-42.
    [Though this book chapter presents an opening argument in a volume on remix culture, the central sections show how we can combine Jackendoff's generative Parallel Architecture model of language with Peircean semiosis and Bakhtin's dialogic principle for describing the underlying grammar and meaning system of music and art forms. I analyze examples from jazz and electronic music (Miles Davis and Daft Punk) and from pop and appropriation art (Rauschenberg, Warhol, and street art). Though I think this model is still incomplete, and I've changed my mind about the best way to describe the semiotic concepts, you can use this model and method to open up useful ways to apply them in your own analyses.]

Presentation [for discussion in class]: Visual Semiotics: Concepts, Methods and Applications

If you choose a movie (scenes and/or shots):
Background on studying film

  • David Bordwell, and Kristin Thompson. Film Art: An Introduction. 8th ed. New York; London: McGraw-Hill, 2007. Excerpts.
    [You can review the sections in these excerpts for the common vocabulary and technical terms used in film making and film studies. Bordwell assumes an overall semiotic framework, and takes us in to the details of how movies and made and structured. You can learn a lot from the chapter excerpts on creating scenes, the kinds of shots and conventions, and the conventions and styles of film editing.]

If you choose a musical form:
Background on How (Popular) Music Genres Mean

  • Martin Irvine, "Popular Music as a Meaning System" (Introductory essay).
    [A synthesis of semiotics and elementary musicology for a method to describe and understand the meaning's of musical forms.]
  • Philip Tagg, "Introductory Notes to the Semiotics of Music." (Now part of Tagg's Music's Meanings).
    [Philip Tagg is a well-known British musicologist who has a lifetime of research on both classical and popular music. This early lecture provides key distinctions and concepts that can be extended to explain how the features of popular music connect to the larger meaning systems in which they participate. He attempts a distinctive feature map of signifying features in popular music (like rhythm and timbre [the sound textures]) that we use and identify automatically in a learned system of musical forms.]
  • Martin Irvine, "Complex Artefacts: Music's Meanings". Presentation (Use for reference. We will discuss in class.)
  • Music Analysis Worksheet (working on your DIY semiotic analysis of how a musical form means).

For discussion (link to Wordpress site)
Choose an example of a media form: artwork or visual medium, movie scene or shot sequence, short musical form (song or section of a composition):

  • This week's class meeting will be a workshop and laboratory for working through examples step by step.
  • Post the example that you would like to discuss in class, and add some of the kinds of questions you would start with to discover how the meaning structure works, and hints about the codes and shared genre knowledge that we would consider in forming an interpretation of what it means in its contexts of use and interpretative communities. (Work on posing the questions, not answering them.)
  • Starting points to keep in mind in developing your discovery methods:
    • Remember that we never experience a medium "in general," but in material instances of specific genres, instances of kinds of media types. One of the first questions, then, what is the chosen example or instance and example of?
    • More specifically, you'll see that we don't experience "music in general," but in instances (material realizations = token clusters) of culturally known genres, kinds, and categories (types of genres in clusters of understood codes, regular features or patterns, contexts of use). The genre types, and all the codes and features that define them, may seem too transparent, automatic, and "natural" for music genres and styles that you know well.
    • Similarly, we don't experience photography, film, artworks or any visual medium "in general," but in instances of types of genres with their clusters of codes, features, and ideas and in corresponding normative physical media.
    • Do your best to get practice in accounting for the signifying features and inferable patterns in the media genre instance, and begin mapping those to the general types, categories, or classes of forms that they represent.
    • In cases of digital re-mediation, you'll find that all digital forms of photography, film/video, music, and photo reproductions of artworks simulate the features of these inherited media forms for human perception (visual, auditory).

Learning Objectives:

  • Applying our methods and concepts to your case studies in a seminar workshop.
  • Review last week's readings and backgrounds for applying further.
  • Working in your assigned teams, choose an example of a cultural genre (music, movie, artwork) that you can describe and analyze with the semiotic concepts and methods that we have developed so far in the course.

Readings:

Group discussion (link to Wordpress site)

  • After discussing your example or case for interpretation and analysis, designate a team "scribe" who will record your discussion and ideas for a group post. We will use your example to analyze and discuss further in class.

Learning Objectives and Main Topics:

  • Learning the major terms, concepts, and technical applications for communication and information theory as defined and used in electronics, computation, and digital media for signals and digital-electronic states.
  • Learning how and why the engineering definition of information in our electronic and digital context is essential for the way all information and media technologies work (based on discrete states represented by bits, units of binary values, transmitted in time to different physical locations in space).
  • Learning why these technical implementations for signals are necessarily “pre-semantic” (prior to meaning correlations) or “semantic-agnostic” (not-knowing or unaware of meaning-correlations), but must presuppose motivated meanings by senders and receivers of messages and any kind encoded information. Without human meaning intentions there would be no sending and receiving of messages encoded in signals!
  • Learning why the engineering transmission model (sender-signal-receiver) is not applicable or sufficient for describing the communication of meaning, values, and intentions that we encode in sign/symbol units.
  • Learning how to complete the description of human meaning making in artefactual communication and media representations with current knowledge and models provided by linguistics, pragmatics, semiotics, and cognitive science.

Key Terms and Concepts:

  • Information defined as units of probability and differentiation (differentiability) from other possibilities.
  • The Transmission Model of Communication and Information.
  • The dominant conceptual metaphors (and their consequences):
    conduit (“channel”), container (and “content”), source, destination.
  • The “bit” (binary unit) as minimal encoding/encodable/encoded unit (2 possible values in base 2 number system with one of two values represented = 1 bit of information), and how/why binary code units map onto electrical circuits and electronic states (a value can be represented in an electronic state).
  • Discrete (digital/binary) vs. Continuous (analog) signals or information sources.

Readings:

Video Lessons (for background)

  • Martin Irvine, Introduction to the Theory of Information and Communication (video)
    Video for the online course, CCTP-798, Key Concepts in Technology.
  • Below: Communication Primer (1963) (film from the Internet Archive)
  • This film was produced by Ray and Charles Eames in 1963 (5 years after the first publication of Shannon's "Mathematical Theory of Communication" (MTC, also called "Information Theory"). The script was advised by Shannon, Weaver, Wiener, von Neuman, Kaufmann, Campbell and other authors of the first scientific theories of information. Extraordinary archival statement of the mathematical model that extended from telecommunications to computing, software, and digital networks. This is how it all began.

Readings: Models of Communication and Information and Their Consequences

  • Martin Irvine, "Introduction to the Technical Theory of Information" (using Information Theory + Semiotics)
  • Luciano Floridi, Information: A Very Short Introduction, Chapters 1-4.
    [For background on the main traditions of information theory, mainly separate from cognitive and semantic issues.]
  • James Gleick, The Information: A History, a Theory, a Flood. (New York, NY: Pantheon, 2011).
    Excerpts from Introduction and Chapters 6 and 7.
    [Readable background on the history of information theory. I recommend buying this book and getting as deeply into the issues that Gleick explains as possible.]
  • Ronald E. Day, "The ‘Conduit Metaphor’ and the Nature and Politics of Information Studies." Journal of the American Society for Information Science 51, no. 9 (2000): 805-811.
    [Models and metaphors for "communication" have long been constrained by "transport", "conduit," and "container/content" metaphors that provide only the signal processing map of a larger contextual process. How can we describe "communication" and "meaning" in better ways that account for all the conditions, contexts, and environments of "meaning making"? Are network and other systems metaphors better than the linear point-to-point metaphors?]
  • Peter Denning and Tim Bell, "The Information Paradox." From American Scientist, 100, Nov-Dec. 2012.
    [Computer science leaders address the question: Modern information theory is about "pre-semantic" signal transmission and removes meaning from the equation. Or does it, can it? Classical information theory (in the electrical engineering sense) just assumed that messages and signals were motivated by meanings and intentions, but the mathematical model bracketed off that part of the symbol-making question. How do semiotic and information theoretic concepts intersect for a way to complete the information theory model of signal transmission?]

In-Class:

For discussion (link to Wordpress site)
Choose one of topics as ways to organize your thoughts and questions about the readings for this week:

  • Conceptual Framework:
    The signal-code-transmission model of information theory has been the foundation of electronic message transmission systems in all communications technology from the telegraph to the Internet (from analogue to digital/binary systems). The key concepts are essential to know for understanding the underlying designs of all our media and computational systems today. But the signal-code-transmission model is not an adequate model for describing the communication of meaning -- the semantic, social, and cultural significance of encoded signals (even though you find people using it as a model of meaning communication). How do we combine an understanding of the physical substrate (electrical signals) with the pattern matching and symbolic structures of represented meanings in a communication system?
  • What is "communicated" in a communication act or event, and how can we represent it in physical message units (units of electrical current)? Can you see why the "content, container, and conduit/transport" conceptual metaphors don't help explain how we make meanings with messages?
    • From what you've learned about symbol structures so far, can you describe how the physical/perceptible components of symbol systems (text, image, sounds) are abstractable into a different kind of physical signal unit (electronic/digital) for transmission and recomposition in another place/time? (Hint: as you've learned from Peirce and semiotic theory, meanings aren't properties of signals or sign vehicles but are relational structures in the whole meaning-making process understood by senders/receivers in a meaning community.)
  • Examples and cases:
    How do we know what a text message, an email message, or social media message means (text and combinations with images and video)? What kinds of communication acts understood by communicators are involved? What do senders and receivers know that aren't physical properties of the individual texts? Our technologies are designed to send and receive strings of symbols correctly, but how do we know what they mean?
    For complex combinations of symbolic structures like artworks, what is applications of the signals view of communication theory? At what level are are symbolic artefacts "communications"?

Learning Objectives and Main Topics:

Gaining a foundation in the influential theories, key terms, concepts, and interdisciplinary approaches for the study of media and symbolic, technical mediation that have shaped research and popular conceptualization from the 1960s to the present.

The cluster of terms for media and interface are used in so many ways that we need to unpack the history of the concepts and find useful ways of using terms in descriptions and analysis. What is "new" about new media, software controlled media, and network mediation, and what aspects can we understand as a continuum in re-combinatorial functions and processes? How can we best understand, describe, and analyze the concepts and implementations for interfaces and multimodal forms? How do interfaces go "meta" in combinatoriality and presentation frameworks (computer devices and digital display screens as a metamedium, a medium for other media)?

The Analog-Digital Continuum

In the transition to digital and software produced and presented media, we often forget that every form of media information or "content" requires a perceptible analog output or presentation. Our mediasphere (borrowing Debray's term) is not a system of binary oppositions or dichotomies between digital and analog (or digital vs. traditional) media forms, but a continuum of material and re-encodable sign systems, material substrates (the presentational supports or surfaces), and social/cultural genres. All of our inherited and new media co-exist as relations in a media system (not separable or individual media), a system continually being reconfigured and redistributed as social, economic, and technological conditions interact to create hierarchies of value, concentrations of power, and access to the whole system. None of the configurations of our media system are determined by the properties and features of the media technologies in isolation. What we now call "media convergence" (through digital and computer network platforms, standardized formats, and consolidations in industries) exposes the media system as a system every time we use a contemporary media artefact. Case in point: consider the major social/cultural functions of the book and text (a 2000-year history). Digital media and ebooks did not decrease the importance and value of books and texts (or replace them with something uniquely digital), but, rather, digital text technologies and widely adopted display platforms (from PCs to hand-held devices) allowed a global, wide-scale re-mediation of the book function in societies where books and texts were already valued. Amazon and other publishers now sell more ebooks than paper-bound books, and we distribute more text and text genres today than ever before. The media functions have remained consistent and constant throughout the stages of technical mediation.

Key terms and concepts:

  • Medium/media as social-technical implementations of sign systems for communication and meaning functions maintained by the uses of media in a larger cultural, economic, and political system
  • Mediation as a function of a medium (e.g, text/print, image technologies, media industries) inseparably linked to material and technical implementations
  • Interface as the physical-material contact point for technical mediation with users (social-cognitive agents) of media
  • Technical Mediation (in Latour's terms) as the means of distributing agency in a social-technical network
  • Media System as the interdependent social configuration of technologies and institutions
  • Communication vs. Transmission (in Debray's terms): differentiating media technologies and their functions for (near) synchronous communication within a cultural group (e.g., telephone, email, TV, radio) vs. technologies for transmission of meaning and cultural identity over long time spans (e.g., written artefacts, books, recorded media, computer memory storage, museums, archives).

Video Introductions to Media and Mediation:

Readings

Key Traditions and Concepts in Media Theory

  • The legacy of Marshall McLuhan, "The Medium is the Message" (Excerpts from Understanding Media, The Extensions of Man. 2nd Edition, 1964).
    [McLuhan's main concepts and arguments for media are common knowledge in media studies, and (while now dated) are useful to rethink in the context of contemporary media. Note how he was trying to find a way to uncover the forces and influences of a medium that are mediated in the form and institutions of a medium. He usually uses the terms "technology" and "media" interchangeably (a confusion), resulting from his premises about technical "extensions" of the body and senses. Unfortunately, his approach also embodies "technological determinist" assumptions that need to be critiqued and thought through. But McLuhan's questions -- developed more fully in Debray's work -- can still usefully question what is communicated in a medium ("the Message" = Meta-message) besides a "content" -- in the ordinary uses of these terms. Note his observation of media that "de-tribalize" societies by providing common forms over place and time; hence his idea of electronic media creating the "global village."]
  • Regis Debray, Transmitting Culture, trans. Eric Rauth. New York: Columbia University Press, 2000. 
    Excerpts from Chaps. 1-2 and 7.
    Note Debray's useful distinction (by redefinition) between "communication" and "transmission" (over longer time spans). Note his more complex model of technical media "mediation" involving many co-dependencies -- technical, social, institutional, political. His view is close to Latour's and Actor-Network Theory (and all systems theory models).
  • Lev Manovich, The Language of New Media. Cambridge: MIT Press, 2001. Excerpt: "What is New Media?"
    [Note the categories Manovich sets up for defining "New Media". Manovich has continually argued that the overused term "new media" can be usefully defined as media produced and managed by software and computation, including all kinds of hybrid representations. This is a more nuanced and detailed description that simply using the false dichotomy or opposition of "analogue" vs. "digital." This book has become a major part of "media studies" since the early 2000s. Manovich's expands on this approach in Software Takes Command, which expands his approach.]
  • Jay David Bolter and Richard Grusin. Remediation: Understanding New Media. Cambridge, MA: The MIT Press, 2000. Excerpts from the Introduction and Chapter 1.
    [This is a widely recognized major analysis of the features, social assumptions, and consequences of digital media. Key points are what happens in digitization and re-presentation, the design principles and unexpressed ideology of "transparency" (making material mediation invisible with the illusion of presence), and the double sense of "remediation" (as "mediating over again" and as "making better, improving the state of something"). Consider the contemporary paradox of living in highly technical (physical) mediating environments that we use to create the illusion of things immediately (not-mediated) present or real.]

For discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • How would you describe the transitions and reconfigurations of media and media systems since the adoption of digital and networked media? How do you account for the continuum of media forms and our current system in which pre- and post-digital media coexist and are able to be re-mediated across technical forms (think all forms of text and books, TV, movies/video, music)?
  • Consider a digital interface (e.g., a Web browser and its "windows"; the "tiled" iPhone/iPad display; or a specific software interface) in as many dimensions of mediation as you can describe. What are the consequences of software-driven screens implementing the "metamedium" function?
  • Is YouTube a "medium," "platform," "remediation" for video and multi-user produced "content"? Does it matter?
  • With consolidation of digital media technologies and networks, are we transitioning to a "post-media" era where individual media forms matter less than the social institutions, economic systems and industry groups, and political boundaries define information, communication, and cultural expression?

Learning Objectives and Main Topics:

  • Learning the approaches, methods, and key concepts for understanding media and cognitive technologies in larger networked systems of agency and mediation.
  • Working with an interdisciplinary merging of Actor-Network Theory and approaches to distributed cognition in technology

Background:
In this week's interdisciplinary merger, we open up other levels of abstraction for describing and analyzing relations and interdependencies with the various forms of system theory methods and models (network theory, modularity in complex systems, technologies and media in a systems view of cognitive science and semiotics). The most common conceptual metaphors are (1) tree structures and nested hierarchies and (2) the network as system. (The conceptual metaphors as explanatory models are often often combined in practice).

This unit's group of interdisciplinary readings are mainly organized around applications of the network concept as a system of relations in which any nodes or constituents must be understood in relation to the whole system (often represented in a graph of nodes and edges or links). The systems view is also used in cognitive science research on technologies, symbolic artefacts, and media for explaining a larger holistic view of human agency and cognition as mediated and distributed through artefacts and all kinds of technology (though we will continue to focus specifically on media and symbolic technologies). An important motivation for network and systems views is come to a better understanding of causation and the operation human delegation of cognitive and agent functions to technologies. We are thus equipped with more powerful conceptual tools for understanding how technologies and media have no causal "effects" other than how they are situated in a social-technical network that mediates and distributes agency and cognitive functions.

Mediology and Actor-Network Theory (ANT) are approaches for asking systems-based questions about mediation, communication, and distributed agency in media and technology. The concepts and methods for analysis are useful for bringing the invisible forces of social relations and institutional contexts up to awareness. They represent interdisciplinary research programs, points of view, and schools of thought guided by some open hypotheses, and not "disciplines" or "fields" of study per se. A major motivation of these approaches is undoing any easy "effects" models of technology/media (as one domain) and "culture/society" (as a separate domain). As Latour emphasizes, positing the technology/society division as independent spheres (that must be somehow brought into a "relation") is a social construction, a fiction; the division does not represent anything real or observable in things or social organization. In fact, it's quite the opposite: technologies and media are formed in a complex social-material system, and mediate the organized forms of distributed social agency that we call technologies.

These methods can be called a "metatheory" or way to de-blackbox social and technological systems that are often constructed ideologically to be closed from analysis (e.g., the function of hype and productizing in consumer devices and digital technologies; the reification of media and technology as "things" rather than outcomes of complex relational processes).

At another level of abstraction, we find the research done in cognitive science approached to distributed and embodied cognition. Here the "technology/culture" or "technology/society" divisions are rendered impossible in the detailed analysis of symbolic and cognitive artefacts that are necessarily intersubjective embodiments of human cognitive faculties. These approaches use a group of related explanatory concepts: the interacting processes of extended, externalized, distributed, or off-loaded cognition with technologies and artefacts.

The Pay Off
By combining the insights of Actor-Network Theory and Mediology on mediation and distributed agency (modeled on the concept of network as a system of interconnected positions that function to distribute the network's resources) with cognitive science approaches to distributed cognition in the design and implementation of technologies we have a very rich combined set of concepts for de-blackboxing any medium or technology and using any kind of technical medium as an interface to the whole system that it belongs to and mediates its functions.

Readings:

Presentation & Discussion (in-class): De-Blackboxing Methods and Cases (Irvine)

 

Optional Reading: Further Research Sources

  • James Hollan, Edwin Hutchins, and David Kirsh. "Distributed Cognition: Toward a New Foundation for Human-computer Interaction Research." ACM Transactions, Computer-Human Interaction 7, no. 2 (June 2000): 174-196. 
    [This is an important summary of research conclusions from leaders in cognitive science and its relations to technology and HCI. Hutchin's justly famous book, Cognition in the Wild (1996), provided empirical validation for understanding cognition as involving and requiring a larger system of human interactions outside and beyond individual minds/brains.]
  • Itiel E. Dror and Stevan Harnad. "Offloading Cognition Onto Cognitive Technology." In Cognition Distributed: How Cognitive Technology Extends Our Minds, edited by Itiel E. Dror and Stevan Harnad, 1-23. Amsterdam and Philadelphia: John Benjamins Publishing, 2008.

For discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • Thinking with the concepts and approaches at the "network systems" level of analysis approach, describe as many of the concealed ("invisibility cloaked") forces and agencies being mediated or "interfaced" in an everyday digital device (not the "content"). Example question: How do the technical features of an iPhone or PC mediate, or provide an interface to, the larger social-technical system in which the material technologies were developed?

Learning Objectives and Main Topics:

"A symbol is whatever allows us to make an abstraction" --Suzanne Langer

"Computation is about automating abstractions." -- Jeanette Wing

"Computation is information processing through transformations of symbolic representations." --Peter Denning

In this unit, students will learn the key concepts in "computational thinking" and the major design principles in computation and software as automated processes of abstraction for transforming symbolic representations. We will focus on foundational concepts that can be extended for understanding today's environment of "computation everywhere" and "an app for everything." 

This unit focuses on the key concepts in computation and the conceptual models for software, computer and information design, and all digital media. We will approach the questions from a non-specialist perspective, but it's important for everyone to get a conceptual grasp of the core ideas in computation because they are now pervasive throughout many sciences (including the cognitive sciences), and are behind everything we do daily with computational devices, information processing, and digital media (for example, the Google algorithms for searches, all the apps in mobile devices, the software functions for displaying, processing, and playing digital media).

Key Concepts and Terminology

The readings and programming tutorial below will introduce you to some of the foundational concepts and terms used in computer science (and many related fields):
  • Abstraction (as used in math and computation): simplifying complexity by defining or establishing a generalizable concept (form, type, category) independent of all the details possible in individual instances of a form and which holds across all instances. Symbolic cognition enables us to use symbolic notation to represent abstractions at all levels.
    • Example in computation: The fundamental abstraction "data type". Three examples:
      integer (a defined data type for the class of all whole number values),
      string
      or character (a defined type for representable typographic characters in human languages, the combined sequences of which represent words and sentences);
      image
      (defined as an abstract object of mathematical values that must include an array of pixel grid coordinates and color values for each pixel in the grid).
  • Module: Any subcomponent in the design of a larger system (hardware or software) that can be called on and used in the operations of the whole system. In computer programming, a subroutine, a library module of reusable code, or a function are kinds of modules implemented at different levels in the structure of a whole program.
  • Variable: A computable object with an assigned value within the current context of processing. We use the symbols and symbol strings of corresponding data types to represent assignable values in programming languages.
    • Here are three examples of variable definitions and value assignments as scripted in Python, and what Python makes them mean (inside the computer):
      x = 32
      ,
      sumhere = (x + y + z)
      ,
      a = "Four score and twenty years ago"

    • "create the variable x as datatype 'integer' and set x to the integer value '32'".
    • "create the variable 'sumhere' as the datatype 'integer' and set the variable 'sumhere' to the integer value resulting from adding the current assigned values of x, y and z" [assuming x and y and z are already defined in the context of processing].
    • "create the variable a as datatype "string" and set the variable a to the string value 'Four score and twenty years ago' ."
  • Data type and value: Each data type is associated with the values that it can represent.
  • Function: Specifying how a process will transform (interpret, calculate) its input to the desired output(s) designed in the computation. In some programming languages, a function may be termed a procedure or a method.
  • Algorithm: The abstract formulation or "recipe" of any automatable computational process, modeled in such a way as to be translatable into the formal language of any high-level programming language.

Main Assignment:

After the background readings and videos, work through the Code Academy tutorial on an introduction to computing concepts through the Python programming language. (Python is now the most widely used programming language for teaching computation in colleges and schools--all the way to MIT's intro CS courses.) Work through the tutorial as far as you can, and see if the key concepts above become intuitively clearer by trying things out and seeing code in action.


Video Introductions:

Reading and Background:

  • Read short backgrounds first (Denning, Wing, Irvine), then focus on the Evans reading:
  • Denning, Peter J. "The Great Principles of Computing." American Scientist, October, 2010.
  • Jeannette Wing, "Computational Thinking." Communications of the ACM 49, no. 3 (March 2006): 33–35.
    [Short essay on the topic; Wing has launched a wide discussion in CS circles and education.]
  • Martin Irvine, An Introduction to Computational Concepts (introductory illustrated essay)
  • David Evans, Introduction to Computing: Explorations in Language, Logic, and Machines. Oct. 2011 edition. CreateSpace Independent Publishing Platform; Creative Commons Open Access: http://computingbook.org/.
    Read chapters 1-3 (Computing, Language, Programming); survey other chapters as a reference and for further study as your time and interest allow at this point. You can always return to other chapters for reference and self-study.
    • Notes: This is a terrific book based on teaching Intro Computer Science courses at the University of Virginia. The book is open access, and the website has updates and downloads.
    • The reference programming language (and some of its terminology) used in this book is different from Python (lesson below), but the concepts and principles in programming are the same.

For Reference: The Architecture and Design of Computers and Software

  • If you don't know the layout of basic computer architecture, this is a good, well-illustrated guide:
  • Ron White, How Computers Work. 9th ed. Indianapolis, IN: Que Publishing, 2007. Excerpts.
    • The Basics (hardware and software architectures)
    • Software Applications
      [Use this book as a reference for the operational nuts and bolts of hardware components and the computational principles behind them. Note that the systems architecture described is applicable to the miniaturization and modular design of mobile devices as functions become integrated in fewer chips and smaller space.]

Learning Project Assignment: Lesson On Code Academy

Presentation (in-class discussion): Computation and Symbolic Mediation (Irvine)

For discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • Discuss your main insights and from working through the Code Academy tutorial (as far as you were able to go). The key to understanding computer systems from the "computational thinking" and beginning coding point of view is to make some conceptual leaps in seeing how what begins in a conceptual model of a process -- something we can conceptualize in a symbolic way in diagrams, in ordinary language statements, and in math and logic expressions -- can be translated into step-by-step instructions that a computer system can perform. (Coding is like writing an orchestra score, but, like music, a process in a program it isn't music until performed, enacted, or executed.) What key concepts in computing and computational thinking were clearer -- or not? Are the key concepts in the basic terminology above intuitively clearer in the sense of understanding how computation is designed and works the way it does?

Learning Objectives and Topics:

Learning how we got from the model of computation and computers as general logical-symbolic processors to the idea of "personal" computers and digital media interfaces for any human-software interaction. How did earlier principles of computation lead to our current platforms for creating, combining, processing, displaying, storing, and transmitting human symbolic representation in any digitizable medium?

Learning the symbolic and cognitive functions of media supports and interfaces in the history of implementations, and the affordances of computationally (software-based) represented media. What can we learn about abstractable re-implementable functions in the continuum of symbolically mediating material supports from passive interfaces (writing and image surfaces, substrates, recordable media) to metamedia(computer screens, graphical and touch interfaces for controlling software) for representing other media? Can you describe and trace the substrate function (the symbolic use of supports, substrates, surface material media for inscription and memory, images, photographic) to graphical "windows"-based computer displays (functioning as a meta-substrate or metamedium, a surface medium for representing and interpreting other media). How can other, new combinations of media and software emerge from these key design principles?

Key Concepts and terms:

  • Interface (for human interaction with, and control of, software representations and outputs)
  • Graphical User Interface (GUI) and screen interfaces
  • Metamedium / Metamedia
  • Continuum of symbolic functions and the Meta function in computation and digital information/media
  • The consequences of design architectures

Introductory Videos:

Reading
Framing Concepts: From basic computation to the principles behind contemporary digital multimedia

  • Lev Manovich, Software Takes Command, pp. 55-239; and Conclusion.
    • Follow Manovich's central arguments about "metamedium", "hybrid media", and "interfaces" and the importance of Allan Kay's "Dynabook" Metamedium concept.
    • For Manovich, the main differentiating feature of "new media" is software: media produced, processed, controlled, and displayed with software. Digital media = software mediatized media.
    • Questions and critique: are these criteria enough to make the concept of "new [digital] media" useful? what about the analog-digital continuum and the coexistence of media types and artefacts in many material forms?

The Important History and Backstory:
How we got from "computing machines" to digital media and metamedia interfaces:

  • Vannevar Bush, "As We May Think," Atlantic, July, 1945. (Also etext version in pdf.)
    • A seminal essay on managing information and human thought by a leading computer engineer and technologist during and after World War II. Bush's pre-modern computing extrapolations lead to the concepts of GUIs (graphical user interface for computers), hypertext, and linked documents. His conceptual model, though not yet implementable with the computers of the 1940-50s, inspired Doug Engelbart and other computer designs that followed in the 1970s-80s and on to our own hypermediated era.
    • Wikipedia background on this essay.
    • Background on Bush's concept of the "Memex", precursor to hypertext information systems.
    • Video of a working model of the "Memex".
  • Douglas Engelbart and "Augmenting Human Intellect"
    • Engelbart is best known for inventing the graphical interface, the "desktop computer metaphor," the mouse, and the hyperlink. His research and development teams at Stanford in the 1960s-70s were influenced by Vannevar Bush's vision and motivated by a new conception of computers not simply as business, government, or military machines for instrumental ends but as aids for core human cognitive tasks that could be open to everyone. His approach to computer interfaces using interaction with a CRT tube display (early TV screen) launched an extensive HCI engineering/design and computing community around user interaction and "augmenting" (not replacing or simulating) human intelligence and cognitive needs. This is one major thread in what is still an ongoing debate in computer science and AI, and this approach influences current models of extended or externalized cognition in artefacts and computation.
    • Background on Doug Engelbart at the Computer History Museum and the influence of Vannevar Bush's ideas. [See the video for "The Mother of All Demos"-- Engelbart's first demo of the mouse and a GUI (in 1968!)
    • Main reading:
      Engelbart, "Augmenting Human Intellect: A Conceptual Framework." First published, 1962. As reprinted in The New Media Reader, edited by Noah Wardrip-Fruin and Nick Montfort, 93–108. Cambridge, MA: The MIT Press, 2003.
    • Also available online at the Doug Engelbart Institute:
      HTML annotated edition of "Augmenting Human Intellect: A Conceptual Framework" (the Doug Engelbart Institute site).
    • Engelbart's Patent Application (with diagrams) for an "An X-Y Position Indicator for a Display System" (aka, mouse).
  • Alan Kay and the Dynabook Concept as a Metamedium:
    • Alan Kay's original paper on the Dynabook concept: "A Personal Computer for Children of all Ages." Palo Alto, Xerox PARC, 1972).
      [Wonderful historical document. This is 1972--years before any PC, Mac, or tablet device.]
    • Alan Kay and Adele Goldberg, “Personal Dynamic Media” (1977), excerpt from The New Media Reader, ed. Noah Wardrip-Fruin and Nick Montfort. Originally published in Computer 10(3):31–41, March 1977. (Cambridge, MA: The MIT Press, 2003), 393–404.
      [Revised description of the concept for publication.]
    • Interview with Kay in Time Magazine (April, 2013). Interesting background on the conceptual history of the GUI, computer interfaces for "interaction," and today's computing devices.
  • Janet Murray, Inventing the Medium: Principles of Interaction Design as a Cultural Practice. Cambridge, MA: MIT Press, 2012. Selections in 2 files: Introduction and Chap. 2: Affordances of the Digital Medium
    • An excellent recent statement of the contemporary design principles that sum up and extend the design tradition that we are studying this week.

Supplementary:

For discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • Describe the conceptual, technical, and design steps that enabled computers, computation, and software for digital media (text and graphics) to be designed for information access, processing, and "content creation" by ordinary, nontechnical users.
  • Is it useful to re-think and redefine our contemporary "computing devices" (PCs, smart phones, tablets) as forms of metamedia (using design architectures and combinatorial technologies) rather than as a "medium" in the earlier sense? How would you describe or define our current computational devices that are designed for processing, representing, interacting with, and transmitting all forms of digitizable media?
  • Big question: we've seen how our current devices are blackboxed with almost no access to understanding the software or interfaces and platform lock-in that locks out users from recoding and extending software as Engelbart and Kay envisioned. What computing concepts have not been fulfilled, and why? (Remember, the current standard configuration of hardware and software and the position of users to the technologies are not a feature of the technologies themselves but an outcome of decisions for design architectures and business/market structures.)

Learning Objectives:

Understanding the issues surrounding technical mediation and reproduction as implemented for cultural representation and transmission.

The arguments by the writers in this unit--Benjamin and Malraux--and the assumptions in the Google Arts & Culture Project form a chain of accruing concepts and ideologies. Our contemporary context of digital mediation often excludes the major questions of cultural transmission, reproduction, and interpretive interfaces -- all of which precede the current state of digital technologies. What are the main issues? How does digital reproducibility and mediation enable us to rethink these questions today?

The Google Art Project as a "Meta" Museum and Cultural Mediation Project:

The Google Art Project (Now Google Arts & Culture) provides a rich social-technical mediation/interface case study. The Google Cultural Institute is being positioned at the nexus of cultural, institutional, and media technology issues. How are the issues discussed by Benjamin and Malraux and the important theory in sociological descriptions of cultural institutions played out in this recent project of cultural mediation and representation?

Questions:

  • Cultural functions enacted and embodied in our institutions (the museum function, the art function, the artist function, the education function) always precede specific forms of technical mediation while at the same time being mediated through them. What are the main issues in the reproduction of cultural artefacts in any kind of medium (print, TV-film-video, digital images and databases)? What is specific to Google's promotion and combination of Web technologies mediated in the screen interface (combining back-end database technologies, 360 "street-view" camera capture, image formats, tile and grid display schemes in "window" screen views)?
  • How does the Google project connect to the longer history of the questions surrounding representing and mediating cultural knowledge (compare the longer history of meta-representation of art works, photographic images, and Andre Malraux's musée imaginaire--the abstract museum idea or conceptual museum)?
  • Google seems to present the Google Art Project and related cultural projects as technology implementations, that is, as primarily technology projects. Can this be so? What about all that's being mediated as the precondition for using the technology this way? Can Google (the company, the technologies) re-mediate the "museum" or "cultural transmission" function? Are the parallels and differences with the Google Books project?
  • Is the Google Art Project a "meta-museum," a virtual museum of museums? How do we interpret the interface structure, horizontal scrolling and tiling, and the significance of the framing and low-information interface design?

Reading and Background

  • Walter Benjamin, "The Work of Art in the Era of its Technological Reproducibility" (1936; rev. 1939).
    This is the more accurate translation of the title of the work known in the English-speaking world as The Work of Art in the Age of Mechanical Reproduction (html) (English trans., Harry Zohn, 1968). Use the recent translation above.
    Focus on sections: Pref., I-VI, X-XII, XV.
    • Notes:
    • Benjamin's title in German is "Das Kunstwerk im Zeitalter seiner technischen Reproduzierbarkeit" = "The Work of Art in the Era of its Technical (or Technological) Reproducibility." The first link above is to an excerpt from the new edition of Benjamin's writings (Harvard Univ. Press, 2003) with the revised title.
    • Benjamin was part of the Frankfurt School of neo-Marxist writers who were engaged in a serious struggle with Fascism, and Benjamin proposed his own more nuanced analyses of history and media technologies. His argument is still in the grip of presuppositions about "mass culture" as passive responses, and he was writing as at time when Fascist capitalism and state-controlled cultural economies threatened to convert all media and cultural forms into tools of ideology.
    • Note the troubled questions about technology and agency and a historical narrative about technological development in section VI. For further background on Benjamin's philosophy, see Philosophical Background (Stanford Encyclopedia of Philosophy): see sections 6-8.
  • "André Malraux, La Musée Imaginaire (The Museum Idea) and Interfaces to Art".
    Overview and excerpts. English translation unfortunately as "The Museum Without Walls", a chapter in The Voices of Silence, 1951). Further implications of art and culture mediated through photography, and the assumption of a global "art encyclopedia" informing the modern concept of art and art history.
  • Google Art Project: A Meta-Museum for the Post-Digital World?
    Experiment with the interface organization principles (and lack of them). Analyze the assumptions in the way representations are structured.
    Choose some of the major museums for the gallery "walk throughs" (the "Zoom View") and experiment with viewing the high-resolution images and making selections for your own collection.

Presentation (In-Class Discussion):

For discussion (link to Wordpress site)
Choose one or two topics as ways to organize your thoughts and questions about the readings for this week:

  • Explore aspects of the Google Art project as a case study for tying together questions and issues that are treated in the readings and in the overall themes of the course. How are institutions and artefacts represented and mediated? Is Google's well-known "street view" 360-degree simulation/presentation a code for "the real" in networked digital media culture? What happens semiotically when artworks, as defined and constructed in the cultural category transmitted in the institutions and the Artworld collectively, are mediated as detachable objects in "user-curator" arrangements? Museums as source material of images for Pinterest-style selection and arrangement?

Final Projects

  • Synthesis, conclusions, open questions.
  • Instructions for Final Projects (on course WordPress site)
  • This week will be devoted to a round table discussion of your final projects. Prepare notes and main bibliography developed so far.