Lecture 9: Understanding Natural Language

The Library of Dresan: Dr. Anthony G. Francis, Jr.'s Weblog

Introduction to Artificial Intelligence with Applications to Public Health

Rollins School of Public Health at Emory University Instructor: Dr. Anthony G. Francis, Jr.

Lecture 9: Understanding Natural Language

Natural language understanding is one of the most difficult problems in artificial intelligence because it is AI-complete --- understanding unrestricted language requires competence in every other area of AI. Nevertheless, progress has been made viewing language as speech acts by which agents attempt to affect each other, and processing language using grammars developed by linguistics. Parsing algorithms such as augmented transition networks enable understanding context-free grammars that capture restricted subsets of natural languages. Statistical approaches in contrast use vast bodies of natural language to develop probabilistic grammars that capture far more of human languages. Machine systems for understanding language vary from simple pattern matchers that that create only the appearance of understanding to complex systems that draw upon world knowledge and creativity theory to extract sophisticated knowledge structures from unrestricted texts.

Outline

Speech Acts
- Communicating for Cooperation
- Speech as Action
- Types of Speech Acts
- Effects of Speech Acts
- Things that should not be missed:
  - Searle
Natural Languages
- Language as an Evolving Communicative Agreement
- The Generativity of Natural Language
- Prescriptive and Descriptive Grammar
- Components of Language
- Properties of the Lexicon
- Things that should not be missed:
  - The Generativity of Natural Language
Understanding Natural Language
- Stages of Natural Language Understanding
  - Signal Processing
  - Lexical Analysis
  - Syntax
  - Semantics
  - Pragmatics
  - World Knowledge
  - Discourse
  - Incorporation
  - Response Generation
- Things that should not be missed:
  - Chomsky, Schank, Charniak
  - Regular, Context-Free and Context-Sensitive Grammars
  - Recursive and Augmented Transition Networks
  - Top-Down, Bottom-Up and Left Corner Parsing
  - Conceptual Dependency
  - Statistical Approaches
Progress in Language Systems
- Pattern Recognition: Eliza
- Domain-Specific: SHRDLU
- General Parsers: COMPERE, Link Parser
- Blackboard Systems: HEARSAY
- Deep Structure: Conceptual Dependency
- Knowledge: BORIS
- Creativity: ISAAC
- Statistical: Charniak
Issues
- Origin of Natural Language
- Generativity of Natural Language
- Pragmatics of Natural Language
- AI Completeness
Resources
- Moby
- Word Net
- Link Parser
- Brown Corpus
- Other Links

Speech Acts

Communicating for Cooperation
- Cooperation: Two agents can do more than one
- Coordination: Sometimes we have to work together
- Transmission: What one man can do, another can do
Communicative Acts: Attempting to Influence Another Agent's Mind
- From "Speaker" to "Hearer"
- Speech: Communicating with Sound
- Writing: Communicating with Diagrams
- Gesture: Communicating with Bodies
Types of Speech Acts
- Representatives: State a Proposition
- Directives: Request or Command
- Commissives: Promise or Threaten
- Expressives: Thank or Apologize
- Declarations: Affect the World
Effects of Speech Acts
- Plans vs Utterances: Internal intention must become external signals
- Direct vs. Indirect: Asking directions vs asking if you have the time
- Perlocutionary vs. Illocutionary: What you meant to say vs. What they heard
Things that should not be missed:
- Searle
- Austin

Understanding Natural Language

Stages of Language Understanding
- Signal Processing
- Lexical Analysis
- Syntax
- Semantics
- Pragmatics
- World Knowledge
- Discourse
- Incorporation
- Response Generation

Signal Processing - Transforming Sounds, Images and Gesture into Symbols

Phonological, Typographic and Gestural Knowledge
Uses feedback from later stages to resolve ambiguity
Examples
- Blackboard Systems - HEARSAY
- Neural Networks - Rumelhart Letter Recognition Model
- Signal Processing - Commercial Language Systems

Lexical Analysis

Morphological Knowledge - How words are composed
- Root words
- Affixes (Prefixes and Suffixes)
- Combination rules
Lexical Knowledge - What root words exist
- Parts of Speech - Nouns, Adjectives, Articles
- Open vs. Closed Classes - Nouns vs Prepositions
- You can verb any noun, but not adjective nouns everywhere
Tokenization - Breaking speech stream into pieces
Examples
- Moby Online Dictionary
- WordNet Dictionary
- Brown Corpus

Syntax

Grammar - What combinations of words are legal sentences
- Rewrite Rules
  - S ::= NP VP
  - NP ::= NAME
  - NP ::= ARTICLE NOUN
  - VP ::= VERB
  - VP ::= VERB NP
- Terminals - Basic Words
  - NAME ::= Anthony | Marvin | Zadeh
  - ARTICLE ::= The
  - NOUN ::= book | car | pool
  - VERB ::= gave | took | threw
- Backus-Naur Form (BNF)
  - S ::= NP VP
  - NP ::= NAME | ARTICLE NOUN
  - VP ::= VERB [NP]
The Chomsky Grammar Hierarchy
- Regular Grammars: Restricted but Fast to Parse
  NONTERMINAL ::= TERMINAL [NONTERMINAL]
  can handle a*b* but not a(n)b(n)
  built into most modern programming languages
- Context-Free Grammars: Simple Rules, Easy to Parse
  NONTERMINAL ::= TERMINALS, NONTERMINALS
  can handle a(n)b(n) but not a(n)b(n)c(n)
- Context Sensitive Grammars: Less Expensive
  TERMINAL, NONTERMINAL ::= more TERMINALs, NONTERMINALs
  can handle a(n)b(n)c(n) but not some very complex languages
- Recursively Enumerable Grammars: Turing Complete
  arbitrary TERMINALs, NONTERMINALs ::= arbitrary TERMINALs, NONTERMINALs
  can handle any language a Turing machine can recognize
Power Needed for Typical Language Tasks
- Lexical Analysis: Regular Grammars or Context-Free Grammars
- Programming Languages: Context-Free and Context-Sensitive Grammars
- Natural Language: Context-Sensitive or Recursively Enumerable

Parsing - Extracting the Structure of Grammar from Text

Recursive Transition Networks: Context-Free Grammars
Augmented Transition Networks: Extracting Data from the Parse
Top-Down and Bottom-Up Parsing
Left Corner Parsers: The Best of Both Worlds
Cognitively Plausible Sentence Processing
The horse raced past the barn fell.
Statistical Natural Language Processing

Examples

Charniak's Approach
The Link Parser
COMPERE

Semantics

First Order Predicate Logic (FOPL)
Quasi-Logical Form: Capturing Ambiguity
Frame Systems: Slots, Fillers and Defaults
Conceptual Dependency: Similar Meanings Should Have Similar Structure
- Types of Conceptual Dependency
  - Actor Object Agent
  - Demonstrative
  - Causality
- Conceptual Dependency Primitives
  - ATRANS
  - PTRANS
  - MTRANS
  - ATRANS

Pragmatics

World Knowledge

Discourse

Incorporation

Response Generation

Progress in Language Systems

Pattern Recognition: Eliza
Domain-Specific: SHRDLU
Blackboard Systems: HEARSAY
Deep Knowledge: BORIS
Cognitive Models: COMPERE - Efficiency and Psychology Meet
Exploiting Creativity: ISAAC
Exploiting Statistics: Charniak

Issues

Origin of Natural Language
Generativity of Natural Language
Pragmatics of Natural Language
AI Completeness

Resources

Word Net: http://wordnet.princeton.edu/
Moby Language Resources: http://www.dcs.shef.ac.uk/research/ilash/Moby/
Link Grammar: http://www.link.cs.cmu.edu/link/
Brown Corpus: http://helmer.aksis.uib.no/icame/brown/bcm.html
More on Conceptual Dependency: http://www.ee.cooper.edu/courses/course_pages/past_courses/EE459/SCHANK/

Research
Articles
Classes
Software

Classic
Weblog
Wiki
Store

f@nu fiku
Fiction
Personal
About

Contact