Paper in preparation. Some of the material was presented at the Ninth Annual CUNY Conference on Human Sentence Processing, 1996.
In the presented simulations, Simple Recurrent Networks (SRN) were trained via a prediction task on sentences from a grammar involving left recursion in the form of prenominal genitives, right recursion in the form of subject relative clauses, sentential complements, prepositional modifications of NPs, and NP conjunctions, as well as complex recursion in the form of object relative clauses. The grammar also incorporates subject noun/verb agreement and three additional verb argument structures (transitive, optionally transitive, and intransitive). Other SRNs received training on a second grammar in which cross-dependency constructions were substituted for the object relative clauses (rendering a mock "Dutch" grammar). These simulations thus constitute a non-trivial extension to Elman's (1991, 1993) work.
The successfully trained SRNs mirror experimental data concerning human processing of recursive sentence structures. For example, doubly center-embedded constructions are more difficult to process than constructions with two cross-dependencies, and in both cases the performance degrades at the point in the constructions where humans also experience problems (Bach, Brown & Marslen-Wilson, 1986). The networks demonstrate sophisticated generalization abilities, ignoring local word co-occurrence constraints while appearing to comply with structural information at the constituent level. The differential processing of lexical material within and between constituents provides further evidence of the SRNs' sensitivity to constituent structure.
In addition, the following predictions emerged from the performance of the networks: multiple instances of subject relative clauses are harder to process than multiple instances of sentential complements; on both multiple instances of left and right recursive sentence structures performance degrades differentially (with little or no breakdown within constituents but significant breakdown between constituents).
Finally, implications regarding the applicability of neural networks
to sentence processing and the conception of recursion relevant to
natural language will be considered.
Click to request a copy of this paper.
Please email suggestions/errors to mhc27@cornell.edu |