Language Technologies Institute

11-823

11-830

Computational Ethics for NLP

Dissertation Research

LTI Practicum

Computational Models of Neural Systems

This course will introduce students to the highly interdisciplinary area of Artificial Intelligence known alternately as Natural Language Processing (NLP) and Computational Linguistics. The course aims to cover the techniques used today in software that does useful things with text in human languages like English and Chinese. Applications of NLP include automatic translation between languages, extraction and summarization of information in documents, question answering and dialog systems, and conversational agents. This course will focus on core representations and algorithms, with some time spent on real-world applications. Because modern NLP relies so heavily on Machine Learning, we'll cover the basics of discrete classification and probabilistic modeling as we go. Good computational linguists also know about Linguistics, so topics in linguistics (phonology, morphology, and syntax) will be covered when fitting. From a software engineering perspective, there will be an emphasis on rapid prototyping, a useful skill in many other areas of Computer Science. In particular, we will introduce some high-level languages (e.g., regular expressions and Dyna) and some scripting languages (e.g., Python and Perl) that can greatly simplify prototype implementation.

Fall/Spring This is a full-semester lecture-oriented course (12 units) for the PhD-level, MS-level and undergraduate students who meet the prerequisites. It offers a blend of core theory, algorithms, evaluation methodologies and applications of scalable data analytic techniques. Specifically, it covers the following topics:

• Link Analysis
• Social-media Analysis
• Web-scale Text Classification
• Learning to Rank for Information Retrieval
• Deep Learning for Text Mining
• Matrix Factoorization (with SVD, non-negative and probabilistic matrix completion)
• Stochastic gradient descent
• Statistical significance tests

Notice that 11-741 and 11-641 are 12-unit courses for graduate students, but 11-441 is a 9-unit course for undergraduate students.Although the lectures are the same for all students, the workload differs by course. That is, the required course work in 11-441 is a subset of that in 11-641, and the work in 11-641 is a subset of that in 11-741. See the detailed distinctions in the Grading section. 11-741 is among the required courses for PhD candidates in the Language Technologies Institute. while 11-641 only counts as a master-level course. Graduate students can choose either 11-741 or 11-641, depending on their career goals and backgrounds. Undergraduate students should take 11-441; exception is possible if approved by the instructor.

• CS courses on data structures, algorithms and programming (e.g. 15-213), linear algebra (e.g. 21-241 or 21-341) and intro probability (e.g. 21-325)
• Intro Machine Learning (e.g., 10-701 or 10-601) and Algorithm Design and Analysis (e.g., 15-451) are not required but helpful

This is a full-semester lecture-oriented course worth 9 units.

• 15-210, Parallel and Sequential Data Structures and Algorithms (required)
• 15-213, Introduction to Computer Systems (required)
• 15-451, Algorithm Design and Analysis (helpful)
• 21-241, Matrices and Linear Transformations or 21-341, Linear Algebra (required)
• 21-325, Probability (helpful)
• 36-202, Methods for Satistics & Data Science (helpful)

Fall/Spring This is a full-semester lecture-oriented course (12 units) for the PhD-level, MS-level and undergraduate students who meet the prerequisites. It offers a blend of core theory, algorithms, evaluation methodologies and applications of scalable data analytic techniques. Specifically, it covers the following topics:

• Link Analysis
• Social-media Analysis
• Web-scale Text Classification
• Learning to Rank for Information Retrieval
• Deep Learning for Text Mining
• Matrix Factoorization (with SVD, non-negative and probabilistic matrix completion)
• Stochastic gradient descent
• Statistical significance tests

Notice that 11-741 and 11-641 are 12-unit courses for graduate students, but 11-441 is a 9-unit course for undergraduate students.Although the lectures are the same for all students, the workload differs by course. That is, the required course work in 11-441 is a subset of that in 11-641, and the work in 11-641 is a subset of that in 11-741. See the detailed distinctions in the Grading section. 11-741 is among the required courses for PhD candidates in the Language Technologies Institute. while 11-641 only counts as a master-level course. Graduate students can choose either 11-741 or 11-641, depending on their career goals and backgrounds. Undergraduate students should take 11-441; exception is possible if approved by the instructor.

• CS courses on data structures, algorithms and programming (e.g. 15-213), linear algebra (e.g. 21-241 or 21-341) and intro probability (e.g. 21-325)
• Intro Machine Learning (e.g., 10-701 or 10-601) and Algorithm Design and Analysis (e.g., 15-451) are not required but helpful

This is a full-semester lecture-oriented course worth 12 units.

This course requires good programming skills and an understanding of computer architectures and operating systems (e.g., memory vs. disk trade-offs). A basic understanding of probability, statistics, and linear algebra is helpful. Thus students should have preparation comparable to the following CMU undergraduate courses.

• 15-210, Parallel and Sequential Data Structures and Algorithms (required)
• 15-213, Introduction to Computer Systems (required)
• 15-451, Algorithm Design and Analysis (helpful)
• 21-241, Matrices and Linear Transformations or 21-341, Linear Algebra (required)
• 21-325, Probability (helpful)
• 36-202, Methods for Statistics & Data Science (helpful)

The goal of "Language and Statistics" is to ground the data-driven techniques used in language technologies in sound statistical methodology. We start by formulating various language technology problems in both an information theoretic framework (the source-channel paradigm) and a Bayesian framework (the Bayes classifier). We then discuss the statistical properties of words, sentences, documents and whole languages, and the various computational formalisms used to represent language. These discussions naturally lead to specific concepts in statistical estimation. 

Topics include: Zipof's distribution and type-token curves; point estimators, Maximum Likelihood estimation, bias and variance, sparseness, smoothing and clustering; interpolation, shrinkage, and backoff; entropy, cross entropy and mutual information; decision tree models applied to language; latent variable models and the EM algorithm; hidden Markov models; exponential models and the maximum entropy principle; semantic modeling and dimensionality reduction; probabilistic context-free grammars and syntactic language models.

http://www.cs.cmu.edu/%7Eroni/11661

In a typical semester, a set of readings will be selected (with student input) primarily from the past 2-3 years' conference proceedings (ACL and regional variants, EMNLP, and COLING), journals (CL, JNLE), and relevant collections and advanced texts. Earlier papers may be assigned as background reading. In 2010, the readings will primarily be recent dissertations in NLP. The format of each meeting will include a forty-minute, informal, critical student presentation on the week's readings, with presentations rotating among participants, followed by general discussion. Apart from the presentation and classroom participation, each student will individually write a 3-4-page white paper outlining a research proposal for new work extending research discussed in class - this is similar to the Advanced IR Seminar.

Description

Fall/Spring This is a full-semester lecture-oriented course (12 units) for the PhD-level, MS-level and undergraduate students who meet the prerequisites. It offers a blend of core theory, algorithms, evaluation methodologies and applications of scalable data analytic techniques. Specifically, it covers the following topics:

• Link Analysis
• Social-media Analysis
• Web-scale Text Classification
• Learning to Rank for Information Retrieval
• Deep Learning for Text Mining
• Matrix Factoorization (with SVD, non-negative and probabilistic matrix completion)
• Stochastic gradient descent
• Statistical significance tests

Notice that 11-741 and 11-641 are 12-unit courses for graduate students, but 11-441 is a 9-unit course for undergraduate students.Although the lectures are the same for all students, the workload differs by course. That is, the required course work in 11-441 is a subset of that in 11-641, and the work in 11-641 is a subset of that in 11-741. See the detailed distinctions in the Grading section. 11-741 is among the required courses for PhD candidates in the Language Technologies Institute. while 11-641 only counts as a master-level course. Graduate students can choose either 11-741 or 11-641, depending on their career goals and backgrounds. Undergraduate students should take 11-441; exception is possible if approved by the instructor.

• CS courses on data structures, algorithms and programming (e.g. 15-213), linear algebra (e.g. 21-241 or 21-341) and intro probability (e.g. 21-325)
• Intro Machine Learning (e.g., 10-701 or 10-601) and Algorithm Design and Analysis (e.g., 15-451) are not required but helpful

http://www.cs.cmu.edu/~yiming/MLTM-f20-index.htm

• 11-742, Search Engines (required)

Fall 2016 Special Topics: Machine Learning Tutorials (6 Units) 

This seminar aims to deepen the participants' understanding of the theoretical foundation of statistical learning and applications, to broaden their knowledge of new techniques, directions and challenges in the field, and to inspire research ideas through class-room discussions.  In the past years, this seminar was structured as a 12-unit or 6-unit course in the form of group-reading, presenting and discussing the books of The Elements of Statistical Learning: Data Mining, Inference, and Prediction (by Trevor Hastie et al.) and the Foundations of Machine Learning (by Mehryar Mohri et al.) and selected papers in the areas of large-scale structured learning, temporal dynamics and scalable optimization algorithms. In the fall of 2016, we will have a 6-unit seminar, going through well-known tutorial materials on Deep Learning, Time Series Forecasting, Graph Laplacian Matrices, Modern Bayesian Nonparametrics, and Advances in Structured Learning, NLP and Stochastic Gradient Methods. When the tutorial vedio is listed for the class of the week, all the students are required to read the vedio before the class. We will meet once a week, one main topic (or sub-topic) per week, with presentations rotating among participants. In each week, the assigned presenter starts with the questions from all the participants (collected by email) about the current topic/sub-topic, followed by a presentation on that topic and leads the discussion. All the students are required to read the selected reading materials (about 1.5 hours of reading per week) for the week before the class while the representer should read 2 or 3 additional related papers (chosen by the presenter or the instructor) as needed. Students should email their questions to the presenter and CC to everybody. The slides of each presentation should be shared (by email) with all the class members after the presentation (in the same day). By the end of the semester, each student will individually write a short 3-4-page white paper, outlining a research proposal for new work extending one of the research areas covered in class, or analyzing more than one area with respect to the open challenges, the state-of-the-art sollutions and the future research opportunities. There will be no exams or homework.  The grading is based on class participation (25%), questions submitted for each class and class-room discussions (15%), quality of the seminar presentations (40%) and the final paper (20%).  If one cannot attend a class, a write up of two pages on the topic is required for receiving the credit (25% participation and 15% discussion) for that class.

This 12-unit LTI course provides an introduction to the theoretical background as well as the experimental practice that has made the field what it is today. We will cover theoretical foundations, essential algorithms, and experimental strategies needed to turn speech into text, and go beyond (i.e. do something really useful, and do it right). We will discuss examples of state-of-the-art research, and show links to related fields, such as machine learning, machine translation, dialog systems, robotics, and user interfaces. A term project will provide students with the opportunity to conduct hands-on research.

Sound mathematical background, knowledge of basic statistics, good computing skills. No prior experience with speech recognition is necessary. This course is primarily for graduate students in LTI, CS, Robotics, ECE, Psychology, or Computational Linguistics. Others by prior permission of instructor.

https://sites.google.com/a/is.cs.cmu.edu/lti-speech-classes/11-751-speech-recognition-and-under

11-751 or equivalent; this course can be combined with 11-783 for a 12-unit lab

https://sites.google.com/a/is.cs.cmu.edu/lti-speech-classes/11-753-advanced-speech-lab

Machine learning aims to design algorithms that learn about the state of the world directly from data. 

A increasingly popular trend has been to develop and apply machine learning techniques to both aspects of signal processing, often blurring the distinction between the two. 

This course discusses the use of machine learning techniques to process signals. We cover a variety of topics, from data driven approaches for characterization of signals such as audio including speech, images and video, and machine learning methods for a variety of speech and image processing problems.

Topics include: Zipf's distribution and type-token curves; point estimators, Maximum Likelihood estimation, bias and variance, sparseness, smoothing and clustering; interpolation, shrinkage, and backoff; entropy, cross entropy and mutual information; decision tree models applied to language; latent variable models and the EM algorithm; hidden Markov models; exponential models and the maximum entropy principle; semantic modeling and dimensionality reduction; probabilistic context-free grammars and syntactic language models.

This course is taught intermittently. Students interested in this topic may also wish to consider 11-763 - Structured Prediction for Language and Other Discrete Data, which covers similar material.

Can a robot watch “Youtube" to learn about the world? What makes us laugh? How to bake a cake? Why is Kim Kardashian famous?

12-unit class covering fundamentals of computer vision, audio and speech processing, multi-media files and streaming, multi-modal signal processing, video retrieval, semantics, and text (possibly also: speech, music) generation.

Instructors will give an overview of relevant recent work and benchmarking efforts (Trecvid, Mediaeval, etc.). Students will work on research projects to explore these ideas and learn to perform multi-modal retrieval, summarization and inference on large amounts of “Youtube”-style data. The experimental environment for the practical part of the course will be given to students in the form of Virtual Machines.

This is a graduate course primarily for students in LTI, HCII, CSD, Robotics, ECE; others, for example (undergraduate) students of CS or professional masters, by prior permission of the instructor(s). Strong implementation skills, and familiarity with some (not all) of the above fields (e.g. 11-611, 11-711, 11-751, 11-755, 11-792, 16-720, or equivalent), will be helpful.

https://sites.google.com/a/is.cs.cmu.edu/lti-speech-classes/11-775-large-scale-multimedia-analysis

Humans are highly social creatures and have evolved complex mechanisms for signaling information about their thoughts, feelings, and intentions (both deliberately and reflexively). In turn, humans have also evolved complex mechanisms for receiving these signals and inferring the thoughts, feelings, and intentions of others. Proper understanding of human behavior, in all its nuance, requires careful consideration and integration of verbal, vocal, and visual information. These communication dynamics have long been studied in psychology and other social sciences. More recently, the field of multimodal affective computing has sought to enhance these studies using techniques from computer science and artificial intelligence. Common topics of study in this field include affective states, cognitive states, personality, psychopathology, social processes, and communication. As such, multimodal affective computing has broad applicability in both scientific and applied settings ranging from medicine and education to robotics and marketing.

The objectives of this course are:

(1)    To give an overview of the components of human behavior (verbal, vocal, and visual) and the computer science areas that measure them (NLP, speech processing, and computer vision)

(2)    To provide foundational knowledge of psychological constructs commonly studied in multimodal affective computing (e.g., emotion, personality, and psychopathology)

(3)    To provide practical instruction on using statistical tools to study research hypotheses

(4)    To provide information about computational predictive models that integrate multimodal information from the verbal, vocal, and visual modalities

(5)    To give students practical experience in the computational study of human behavior and psychological constructs through an in-depth course project

Multimodal machine learning (MMML) is a vibrant multi-disciplinary research field which addresses some of the original goals of artificial intelligence by integrating and modeling multiple communicative modalities, including linguistic, acoustic and visual messages. With the initial research on audio-visual speech recognition and more recently with language vision projects such as image and video captioning, this research field brings some unique challenges for multimodal researchers given the heterogeneity of the data and the contingency often found between modalities.

The course will present the fundamental mathematical concepts in machine learning and deep learning relevant to the five main challenges in multimodal machine learning: (1) multimodal representation learning, (2) translation & mapping, (3) modality alignment, (4) multimodal fusion and (5) co-learning.  These include, but not limited to, multimodal auto-encoder, deep canonical correlation analysis, multi-kernel learning, attention models and multimodal recurrent neural networks. We will also review recent papers describing state-of-the-art probabilistic models and computational algorithms for MMML and discuss the current and upcoming challenges. The course will discuss many of the recent applications of MMML including multimodal affect recognition, image and video captioning and cross-modal multimedia retrieval.

This is a graduate course designed primarily for PhD and research master students at LTI, MLD, CSD, HCII and RI; others, for example (undergraduate) students of CS or from professional master programs, are advised to seek prior permission of the instructor. It is strongly recommended for students to have taken an introduction machine learning course such as 10-401, 10-601, 10-701, 11-663, 11-441, 11-641 or 11-741. Prior knowledge of deep learning is recommended but not required.

Your project will have two final deliverables: 
1. a writeup in the form of a NIPS paper (8 pages maximum in NIPS format, including references), worth 60% of the project grade, and 
2. a research seminar presentation of your work at the end of the semester, worth 20% of the project grade. 

In addition, you must turn in a midway progress report (5 pages maximum in NIPS format, including references) describing the results of your first experiments, worth 20% of the project grade. Note that, as with any conference, the page limits are strict! Papers over the limit will not be considered. The grading of your project are based on overall scientific quality, novelty, writing, and clarity of presentation. We expect your final report to be of conference-paper quality, and you are expected to also deliver software implementation of your algorithmic results.

Description

This course is an internship course for students who are doing an elective internship as part of their graduate degree.