Natural Language Generation from First-Order Expressions

Natural Language Generation from First-Order Expressions 1

Natural Language Generation from First-Order Expressions
Thomas Mathew

tam52@georgetown.edu
Department of Linguistics
Georgetown University,
Washington DC 20057-1232

May 09 2008

Abstract: In this paper I discuss an approach for generating natural language from a first-order
logic representation. The approach shows that a grammar definition for the natural language and a
lambda calculus based semantic rule collection can be applied for bi-directional translation using
an overgenerate-and-prune mechanism.

Keywords: First-Order Logic, Natural Language Generation, Reversible Grammar/Semantic
Rules

1. Introduction
First-order logic has been applied in natural language for question-and-answering systems which utilize the benefit
of a mathematical representation of a domain against which independent expressions can be validated or discredited
using mathematical operations. A second functional area, which is of greater interest from the perspective of the
problem I am trying to solve, is to utilize a first-order representation as an intermediate abstract representation in a
language ‘transducer’ which converts one natural language representation to another. Application areas include
Machine Translation, Text Summarization and Text Simplification. Such a transducer would depend on bi-
directional translations between natural language and first-order logic. The next section in this paper glosses over the
translation of language to first-order logic with the intent of introducing the reader to the grammar and semantic
rules that can be utilized for such a translation. The rest of the paper discusses how the same grammar and rules can
be applied to solve the reverse problem of translating first-order logic to natural language.

The concept of using bi-directional grammars is not a new one - Strzalkowski et al 1994 discuss in depth about
grammars that can perform bi-directional translation between natural language and a semantic abstraction. One
aspect discussed is the ease of maintainability when comparing the effort of maintaining one grammar and ruleset
versus that of maintaining two independent grammars and rulesets. Strzalkowski classifies three approaches on how
a bi-directional grammar can be utilized – (a) a parser and generator using a different evaluation strategy but sharing
a grammar (b) a parser and generator sharing an evaluation strategy but are developed as independent programs (c) a
program that can perform as both a parser and a generator. This paper suggests an approach that falls into category
(a).

With regards to similar work – (a) Whitney 1988 discusses a method using an independent grammar for translating
higher-order semantic representations into language using first-order predicate calculus (b) Dymetman et al 1988
discuss using a bi-directional grammar for machine translation via an intermediate higher-order semantic
representation.

2. First-Order Logic

First-order logic is a representation system utilizing an unambiguous formal language based on mathematical
structures. The language used allows constructions covered by prepositional logic extended with predicates and
quantification. Such a representation can be used to symbolize the semantic content in a natural language
construction. Consider:

(1) John loves Mary


Sentence (1) can be represented in a first-order form as LOVE(JOHN, MARY) where LOVE is a predicate which
defines a relation between its arguments JOHN and MARY. Methods for deriving a first-order form from English
have been explored in depth. Blackburn & Bos 2003 describe the use of lambda calculus as one such method which
involves annotating the grammatical rules that model the language with an appropriate lambda expression, This is
shown below for sentence (1):

Grammar Rule λ-Expression Predicate Argument
Generated Generated
S→ NP VP NP@VP - -
NP → Nproper Nproper - -
Nproper → John λp.p@JOHN - JOHN
Nproper → Mary λp.p@MARY - MARY
VP → Vtransitive NP Vtransitive@NP - -
Vtransitive → love(s) λpλq.(p@λr.LOVE(q, r)) LOVE -

Table 1: Sample rules to generate first-order representation from English

By parsing the sentence using the grammar rules and appropriately evaluating and β-reducing the corresponding
lambda expressions (from Blackburn & Bos 2003), the equivalent first-order logic form can be derived. The β-
reduction steps are shown in Figure 1.

Figure 1: Deriving a first-order logic representation

The discussion above is an over-simplification of the process to generate first-order logic – there are many specific
issues with natural language relating to ambiguity and actual language specification that would make this a hard
task. For the task at hand, however I am more concerned about the reverse translation ability and for this reason I
assume that grammatical rules defining a natural language which are annotated with lambda expressions would
suffice in generating a sufficiently descriptive first-order translation.

Predicate Types: A predicate in a first-order logic expression can be any of three types – relational, descriptive or
functional. A relational predicate describes the relationship between 2 or more entities. An example of a relational
predicate is LOVE(JOHN, MARY). A relational predicate would generally be derived from either a transitive/di-
transitive verb, a dependent clause or a prepositional phrase.


A descriptive predicate describes a property of its only argument and would generally be derived from either a noun,
adjective or an intransitive verb. This is shown in the examples below:

(2) BOXER(MARY) ≡ Mary is a boxer
(3) BOXER(MARY) ∧ WOMAN(MARY) ≡ Mary is a boxer and a woman

A functional predicate is used to describe an underspecified entity using a fully-described entity e.g.
MOTHER(JOHN) which can be derived from genitives such as John’s mother or NP constructions such as
Mother of John. A functional predicate can be used as an argument to a relational or descriptive predicate e.g.
LOVE(JOHN, MOTHER(JOHN)). Unlike the other predicate types, a functional predicate does not establish a
truth condition.

Figure 2: Predicate Types

Argument Types: Arguments to a predicate can be constants, variables or a functional predicate. A constant
argument specifies a named entity such as JOHN or MARY. If a variable is used as an argument to a predicate, it
would be quantified in some form to represent either an existential or universal condition as shown in (4).

(4) ∃x (BOXER(x) ∧ WOMAN(x))

3. Natural Language Generation

Given that the first-order representation is unambiguous, if a natural language sentence S can be transformed into a
logical expression E without loss of meaning, then it should be possible to transform E back to a set of sentences of
which S is a member. For machine translation problems, it may suffice to select the first valid language sentence
that can be mapped to the expression. For other problems such as text simplification, the language generator may
need to generate multiple sentences and apply a ranking method to determine the simplest construction.

Each component of a first-order representation creates different complexities to address in the language generation
process. The following is a limited discussion of some issues that make the reverse translation a hard problem. The
discussion is based on the notion that a component of an expression can be considered to be associated to a token in
the language representation which has a specific part-of-speech.

Predicates: A specific predicate can be associated to tokens with different part-of-speech tags e.g. the descriptive
predicate WOMAN() can be realized from translation from either a noun (a woman) or an adjective (a lady ..). In
language generation, selection of one specific token associated with a particular part-of-speech would influence
selection of tokens associated with other descriptive predicates that control the same argument e.g. two nouns
describing the same entity in a transitive construction is unlikely.


An expression may have multiple instance of the same predicate. In a corresponding language representation, it is
possible that all such predicate instances with the same underlying predicate can be associated with a single
language token as shown below. This would generally involve language conjunction.

(5) LAUGH(JOHN) ∧ LAUGH(MARY) ≡ John and Mary laugh
≡ John laughs and Mary laughs
Functional predicates can be associated to either genitives or prepositions and may need to be treated independently
of non-functional predicates.

Arguments: An argument is likely to be referenced by more than one predicate in a sentence. It is also likely that all
such instances of the same argument would all be associated back to a single token. This is not always the case
though as conjunction and pronoun instances could result in multiple tokens referencing the same entity.

(6) LOVE(JOHN, MARY) ∧ LOVE(JOHN, DOG(JOHN)) ≡ John loves Mary and John loves
his dog

For the purpose of this paper, resolution of arguments to pronouns is considered out of scope and it is considered
sufficient to render the translation of the above expression as John loves Mary and John loves John’s dog. This
is because the anaphor resolution of the pronoun his cannot be handled by the approach described in the previous
section. As an alternative, anaphor resolution can be considered as a pre-processing step in first-order expression
generation and the generation of pronouns as a post-processing step in language generation.

Operators: The first-order language supports the set of operators {∧, ∨, →, =, ¬}. An operator can be directly
related back to a token in the language representation or it could also be implied by the ordering of tokens. Specific
operators are described below:

The conjunction operator ∧ can be directly associated to the conjunction token and. It could also represent the
relationship between a verb and a noun managed by a determiner such as a or some (7) or an adjective+noun (8).

(7) ∃x (BARK(x) ∧ DOG(x)) ≡ a dog barks
(8) ∃x (BARK(x) ∧ GREEN(x) ∧ DOG(x)) ≡ a green dog barks

The disjunction operator ∨ can be directly associated to the token or.

The implication operator → can be directly associated to tokens such as if, whenever. It could also represent the
relationship between a verb and a noun managed by a determiner such as every (9) or an adjective+noun (10) or a
copular construction.

(9) ∀x (DOG(x) → BARK(x)) ≡ every dog barks
(10) ∀x (EGG(x) ∧ GREEN(x) → LIKE(SAM, x)) ≡ Sam likes green eggs

The identity operator can also be represented as a predicate using the form EQUAL(x, y). Identity can result out of
an adjective.

The negation ¬ operator can be directly associated to the adverb token not. It could also represent the relationship
between a verb and a noun managed by an adjective such as two:

(11) ∃x (DOG(x) ∧ BARK(y) ∧ ∃x (DOG(y) ∧ BARK(y) ∧ ¬(x = y)) ≡ two dogs bark

The negation operator can influence the association between other operators to tokens. In the example below the
operator ∧ is associated to the token or because of the negation operation.

(12) (¬LIKE(SAM, EGG) ∧ ¬LIKE(SAM, HAM)) ≡ Sam does not like eggs or ham


Quantifiers: These include ∃ and ∀. Quantifiers can be associated to both determiners and adjectives. These are
shown in the above examples.

Parenthesis: This could impact the scoping of a predicate or operator.

The complexity in deciphering the generation issues listed earlier is quite considerable. Given the structure of the
first-order representation, the approach described in the previous section of using lambda calculus to translate
English to first-order logic is not feasible for language generation. It would be still be ideal to utilize the rules that
are used to derive first-order logic to guide the reverse translation.

One possible solution is to apply an overgenerate-and-prune strategy. This approach has been successfully applied to
natural language problems including language generation. This involves weakening the restrictions to allow for
overgeneration of language. Selectional restrictions can them be imposed to weed out bad translations. This
approach is described in the following sections.

4. Overgeneration approach
Consider the simple grammar below annotated with lambda expressions:

Non-terminal rules:
Generated Generated
S → NP VP NP@VP - -
VP → Vtransitive NP Vtransitive@NP - -
VP → Vintransitive Vintransitive - -
NP → Nproper Nproper - -
NP → D N D@N - -
NP → D N C (D@N)@C - -
NP → D AP D@AP - -
AP → A N λp.(A@p ∧ N@p) - -
C → RP Vintransitive Vintransitive - -

Terminal rules:
Generated Generated
Vtransitive → love(s) λpλq.(p@λr.LOVE(q, r)) LOVE -
Vintransitive → boxes λp.BOXER(p) BOXER -
Nproper → John λp.p@JOHN - JOHN
A → lady λp.WOMAN(p) WOMAN -
N → woman λp.WOMAN(p) WOMAN -
A → boxing λp.BOXER(p) BOXER -
N → boxer λp.BOXER(p) BOXER -
RP → who − - -
D→a λpλq.∃x (p@r ∧ q@r) - -
D → some λpλq.∃x (p@r ∧ q@r) - -

Consider the sentences below, all of which can be derived from the rules above:

(13) John loves a lady boxer
(14) ?John loves a boxing woman
(15) John loves some woman who boxes


All the sentences can be associated with a first-order representation of the form:

(16) ∃x (LOVE(JOHN, x) ∧ BOXER(x) ∧ WOMAN(x))

Our goal is to determine a set of rules which would allow a direct derivation of (13), (14) and (15) from (16) using
the annotated grammar as a reference. It is straightforward to parse the expression and determine all predicates,
quantifiers, constant arguments and operators. Each such component must be directly derived from a terminal rule
(note that a single terminal rule can produce more than one first-order component). A set of candidate terminal rules
can be determined by looking up rules whose lambda expressions contain components found in the first order
expression. Starting from such a collection of candidate rules, all possible paths using the set of available non-
terminal grammar rules and terminal rules without an associated lambda expression can be navigated bottom-up till
a rule is reached whose LHS cannot be produced by any other rule. If any of the selected candidate rules have not
been used, the specific navigation path shall be marked as a failure. The mapping process between the first-order
components and the terminal rules is what determines the domain of sentences that get generated. This mapping is
not necessarily one-to-one – many first-order components may be associated with a single terminal rule - consider
example (5) above. This approach overgenerates since there is no selectional restriction enforced between realization
of predicates and their arguments.

The approach is explained in more detail below. A bold symbol X indicates a set of elements. #X indicates the
number of elements within the set X. R is the set of rules in the grammar. It is assumed that a lambda-expression for
a rule can contain at most one predicate or one argument.

Extract the set of all predicate instances P and distinct predicate functions P from E.

E.g. for the expression E = Z(x)∧Z(y), P = {Z, Z} and P = {Z}

For every p∈P, if there are n instances of p in P, determine the set of all possible rulesets RSp
where for each possible ruleset Rp∈RSp if r∈Rp then:
- r ∈ {x∈R | λ-expression(x) contains p}
- #Rp ≤ n

E.g. if P = {Z, Z} and P = {Z} and if there are 2 rules r1 and r2 whose lambda-expression
contains Z then RSp = {{r1}, {r2}, {r1, r1}, {r1, r2}, {r2, r2}}

Determine the set of all possible rulesets RSpred where for each possible ruleset Rpred∈RSpred,
Rpred = ∪p∈P Rp where Rp∈RSp.

E.g. if P = {Y, Z} and RSp=Y = {{r1}, {r1, r1}} and RSp=Z = {{r2}, {r3}} then RSpred = {{r1,
r2}, {r1, r3}, {r1, r1, r2}, {r1, r1, r3}}

Extract the set of all constant argument instances A and distinct predicate functions A from E.

E.g. for the expression Z(x)∧Y(x), A = {x, x} and A = {x}

For every a∈A, if there are n instances of a in A, determine the set of all possible rulesets RSa
where for each possible ruleset Ra∈RSa if r∈Ra then:
- r ∈ {x∈R | λ-expression(x) contains a}
- #Ra ≤ n

Determine the set of all possible rulesets RSarg where for each possible ruleset Rarg∈RSarg, Rarg =


∪a∈A Ra where Ra∈RSa
Extract the set of all quantifier instances Q from E

E.g. for the expression ∃x (Z(x)∧Y(x)), Q = {∃}

For every q∈Q, determine the set of all possible rulesets RSquant where for each possible ruleset
Rquant∈RSquant if r∈Rquant then:
- r ∈ {x∈R | λ-expression(x) contains q ∧ ¬λ-expression(x) contains a predicate}
- #Rquant ≤ #Q

E.g. if Q = {∃, ∀, ∀} and if lambda-expression of r1 contains ∃ and that of r2 and r3
contains ∀ then RSquant = {{r1, r2, r2}, {r1, r2, r3}, {r1, r3, r3}}

Extract the set of all quantifier instances O from E

E.g. for the expression ∃x (Z(x)∧Y(x)), O = {∧}

For every o∈O, determine the set of all possible rulesets RSoper where for each possible ruleset
Roper∈RSoper if r∈Roper then:
- r ∈ {x∈R | λ-expression(x) contains o ∧ ¬λ-expression(x) contains a predicate/
quantifier}
- #Roper ≤ #O

E.g. if O = {∧, ∧} and if lambda-expression of r1 and r2 contain ∧ then RSquant = {{r1, r1,},
{r1, r2}, {r2, r2}}

Determine the set of all possible rulesets RS where Rcandidate=Rpred∪Rarg∪Rquant∪Roper where
Rcandidate∈RS, Rpred∈RSpred, Rarg∈RSarg, Rquant∈RSquant, Roper∈∅∪RSoper

E.g. if E = ∃x (LOVE(JOHN, x)∧WOMAN(x)) and
r1 = [Vtransitive → love(s)],
r2 = [Nproper → John],
r3 = [D → a],
r4 = [D → some],
r5 = [N → woman],
r6 = [CONJ → and],
then RSpred = {{r1, r5}}, RSarg = {{r2}}, RSquant = {{r3}, {r4}}, RSoper = {{r6}}
and RS = {{r1, r5, r2, r3}, {r1, r5, r2, r4}, {r1, r5, r2, r3, r6}, {r1, r5, r2, r4, r6}}

For each Rcandidate∈RS, find all paths to a non-terminal rule with an un-referenced LHS (i.e. S)
such that
- The path has to use every element of Rcandidate one time
- The path can use any non-terminal rule any number of times
- The path can use any terminal rule without a λ−expression any number of times

E.g. using the previous example one possible Rcandidate = {r1, r5, r2, r3} would generate
John love(s) a woman and a woman love(s) John


Figure 3: Overgeneration process


Each path identified represents a candidate natural language sentence. This approach will produce the sentences
(13), (14) and (15), amongst others, when applied to the expression (16). It will satisfy the requirement that if S can
be translated to E using a set of rules, then E can be translated back to S.

The results for the first-order expression in (16) are:

Input First-order Overgenerated Valid
Expression Translations
John love(s) some lady boxer •
some lady boxer love(s) John
John love(s) a lady boxer •
a lady boxer love(s) John
John love(s) some boxing woman •
∃x (LOVE(JOHN, x) ∧ some boxing woman love(s) John
BOXER(x) ∧ WOMAN(x)) John love(s) a boxing woman •
a boxing woman love(s) John
some woman who boxes love(s) John
John love(s) some woman who boxes •
a woman who boxes love(s) John
John love(s) a woman who boxes •

As mentioned earlier there is no context applied while processing the grammar. As a result the output will include
unwanted results such as:

Incorrect NP assignment: both John loves a lady boxer and A lady boxer loves John are outputs

Incorrect quantifier assignment: The expression (∃x (MAN(x) ∧ ∀y (WOMAN(y) → LOVE(x, y))) will
generate both a man loves every woman and every man loves a woman).

The annotated grammar rules used shall need to include all orphan tokens. An orphan token is one that has no
association with an element in the first-order representation. This might require some duplication of terminal rules.
An example is a copular construction. Consider the example below:

(17) BOXER(JOHN) ≡ John is a boxer

In order for this approach to handle bi-directional translations that can handle such a case, there shall need to be 2
logical representations for the rule D → a. T his may result in ambiguity in generating first-order representations.

5. Pruning process
The overgeneration step will create well-formed language sentences – however some of the output will not be a
good translation for the first-order expression. The pruning process applies a filter to drop out such bad translations.

Option 1
One solution to weed out bad results is by regenerating the first-order representation back and comparing with the
original expression E to verify if the translation was valid. This does not require development of any new translation
rule as the methodology described in section 2 using lambda calculus would suffice.

The complexity in this approach arises from determining if two first-order representations are equivalent. This is due
to the nature of mathematical representation as shown below.

(18) ∃x (BOXER(x) ∧ WOMAN(x)) ≡ ∃x (WOMAN(x) ∧ BOXER(x))

This means that a literal equivalency test cannot be applied – instead mathematical equivalence would need to be
established. One method of doing this is using the system Otter.


This method shall become computationally intensive if there are a lot of candidate sentences to evaluate. On the flip-
side it is guaranteed to produce perfect pruning results provided a fool-proof system for equating logical expressions
can be developed.

Option 2
So far the discussions have stayed away from unraveling a first-order representation. A solution that does this is
described below. This approach focuses on multi-argument predicates to validate the generated language. If a
sentence does not have at least one multi-argument predicates (such as copular constructions), the algorithm does
not prune.

In English, a multi-argument predicate can generally be mapped to a verb. The first argument of the predicate is
generally realized to the left of the element that realizes the predicate. The exception to this is a passive construction.
The ordering of the tokens that realize the arguments and predicate can be determined by the order of the lambda
variables with respect to the predicate arguments. This is shown in the expressions below:

Active form ⇒ λpλq.(p@λr.LOVE(q, r))

Passive form ⇒ λpλq.(p@λr.LOVE(r, q))

In a di-transitive case, the active construction check is the same as that for the transitive case. In the passive form, it
needs to be verified that the first argument to the predicate is realized to the right of the element that realizes the
predicate. This is shown in the expressions below:

Active form ⇒ λpλqλr.(p@q@λsλt.GIVE(r, s, t))

Passive form ⇒ λpλqλr.(p@q@λsλt.GIVE(s, r, t))

A simple rule can be based on tracking the orientation of all argument realizations for a predicate with respect to the
realization of the predicate. Note that this rule is very specific to a SVO language – but can conceivable be adapted
to other structures. It is to be determined if the lambda expression can be processed so as to have a more generic
rule.

To enforce such an ordering rule, there has to be traceability between (a) the arguments and their realization (b) the
predicates and their realization. This is generally straightforward to do (the sentence generation rules can be
modified to keep a trace) for sentences where there is a 1:1 association between predicate+argument components
and rules. If there is no direct alignment, one possible scenario is multiple predicate instances all mapped to one rule
(see (5)) – in which case a trace can also be maintained with effort. For any greater combination, such as three
predicate instances mapped to two rule instances, creating a trace is more difficult as multiple combinations may
need to be evaluated – such combinations may be unlikely.

Two special cases for argument realization are: (a) Variable Arguments: The sentence generation rules must be
substantially be modified to allow for tracing of variables to its realization. If the variable is quantified, the
realization of the quantifier can be used to assess the order. The case of unquantified variables has not been
evaluated. (b) Functional Predicates: In this case, the argument realization can be mapped to the realization of the
functional predicate.

The generation rules will now include the following modified and new steps in order to maintain a trace:


Determine a map between each instance of p and an element r∈Rpred

E.g. if P = {Z, Z, Y} and P = {Z, Y} and if there are 2 rules r1, r2 whose λ-expression
contains Z and 1 rule r3 whose λ-expression contains Z then

RSpred = {{r1, r3}, {r2, r3}, {r1, r1, r3}, {r1, r2, r3}, {r2, r2, r3}}

map = {{{1,2}, {3}}, {{1,2}, {3}}, {{1}, {2}, {3}}, {{1}, {2}, {3}}, {{1}, {2}, {3}}}

P = {Z, Z, Y}

1, 2, 3 refer to the index of the elements in P

For each predicate rule p∈P, identify the arguments to the predicate:
- if argument is a constant argument, link p to a rule rarg∈Rarg∈RSarg where rarg contains an
argument for p
- if argument is a variable argument, link p to a rule rquant∈Rquant∈RSquant where rquant contains
the quantifier scope for the argument for r

E.g. if E = ∃x (LOVE(JOHN, x)∧WOMAN(x))
r1 = [Vtransitive → love(s)],
r2 = [Nproper → John],
r3 = [D → a],
r4 = [D → some],
r5 = [N → woman],
then RSpred = {{r1, r5}}, RSarg = {{r2}}, RSquant = {{r3}, {r4}}
and RS = {{r1, r5, r2, r3}, {r1, r5, r2, r4}}

For each Rcandidate∈RS, find all paths to a non-terminal rule with an un-referenced LHS (i.e. S) such
that
- The path has to use every element of Rcandidate one time
- The path can use any non-terminal rule any number of times
- The path can use any terminal rule without a λ−expression any number of times
- For every p∈P determine the ordering of arguments with respect to the predicate in the path using
the map to resolve how the predicate was realized in the generated sentence and using the links
to determine how the arguments were realized

E.g. using the previous example for one possible Rcandidate = {r1, r5, r2, r3} then
P = {LOVE, WOMAN}, RSpred = {{r1, r5}}, map = {{{1}, {2}}}

The trace for generated sentences is shown below:
John love(s) a woman
r2 r1 r3 r5


a woman love(s) John
r3 r5 r1 r2

The rules to handle pruning are as follows:

Extract the set of all predicate instances with more than one argument Pmultiargs from E

E.g. for the expression E = ∃x (LOVE(JOHN, x)∧WOMAN(x)), Pmultiargs = {LOVE}

For each p∈Pmultiargs determine whether the first argument should be realized to the left or right of the
token realizing the predicate based on the order of the variables that describe the order of the arguments
in the lambda expression for the rule associated with p

E.g. for the example above, if r1 = [Vtransitive → love(s)] : λpλq.(p@λr.LOVE(q, r)) then
q which is the first argument is defined by λq which precedes λr which is the second
argument.

For the predicate LOVE, the first argument is expected to be to the left of the token that
realizes the predicate. The sentence order is expected to be rarg1 rpred rarg2.

For each path discard the path if the argument ordering of any predicate is incorrect

E.g. for the example E = ∃x (LOVE(JOHN, x)∧WOMAN(x)), for the specific predicate
LOVE, for the candidate rule {r1, r5, r2, r3}, the sentence order should be r2 r1 r3.

This marks the sentence a woman love(s) John as invalid

A second issue is that the rule does not cover restrictions based on agreement. The output will include John love a
lady boxer. This limitation cannot be addressed with the current state of the rules. To solve this issue the rules shall
need to be annotated with agreement restrictions which can be evaluated as the paths are navigated.

6. Implementation

I implemented this system using the language C# using the .NET framework. The object model used is below:


The classes used by this model are:

Class Description
Rule This represents a grammar rule annotated with a lambda expression. The class provides
methods for parsing a rule into RHS and LHS components. This class inherits from the
generic Expression class.

FOExpression This represents a first-order expression. This class inherits from the generic
Expression class.

The method ProcessExpression()accepts a grammar in the form of
RuleCollection and determines candidate terminal rules that can be associated with
the expression. For each set of terminal rules, the method calls RuleCollection
.GetRules() to determine if the set can generate a language sentence. This method
internally uses recursion to determine all possible sets of terminal rules.

Expression Provides a representation for a mathematical expression with quantifiers, constant
arguments, predicates and operators.

RuleCollection This represents a collection of rules i.e. represents the full grammar of the natural
language.

The method GetRules()provides a mechanism for determining all valid paths through
the grammar if a list of terminal rules are provided. This method internally uses recursion
to navigate the grammar rules.

7. Cost and Optimization

A valid concern against this approach is the question of cost. The cost implications are driven from two angles – one
is the complexity and scale of the grammar and the second is the complexity of the expression being evaluated.

The first issue is of lesser concern since it can be assumed that in an almost full-fledged grammar of English, a
larger percentage of the rules shall specify terminal rules as compared to non-terminal rules – this is because of the
large vocabulary. The algorithm discussed selects early on for terminal rules and hence avoids a performance hit of
having to use a huge grammar in generating a sentence.

The second issue has a larger impact. This limitation is because the algorithm searches for every valid candidate –
consider for example that there are n! permutations for the task to fit n noun phrases into a sentence with n noun-
phrase place-holders. That said, every application may not require every valid language translation for the processed
expression – in which case it may suffice to do an immediate pruning check for every successful sentence generated
and a successful pass through the pruning rules for one sentence could stop the entire algorithm from processing any
further.

8. Conclusion

I have so far defined an approach in which a grammar and ruleset for the compositional construction of a first-order
representation from English can be utilized successfully for the reverse translation problem as well. The approach is
robust in that there is very limited additional information that is required for the system to function. However this
robustness comes at a computation cost which is reasonable for simple first-order expressions but which can become
considerable for lengthy discourse. This overhead may not be that detrimental for application in the space of Text
Summarization and Text Simplification problems where it can be anticipated that an ideal system would be only


required to process limited expressions. Possible future enhancements that can be taken into account to make this
approach more efficient are listed below:

Integrate pruning steps into the generator process wherever possible

Apply a mix of a Top-down and Bottom-Up evaluation strategy in determining if the selected ruleset is a
good candidate for a well-formed sentence. This can avoid the overhead of unnecessary evaluation of a
frequently used rule such as S → NP VP for every pass.

Another alternative to navigating the hierarchy is to cache navigation paths found using the LHS of all the
candidate rules involved as a key to the cache. For every new candidate ruleset, the cache shall be searched
first – if no entry is found then the system can navigate the rules. Each time the grammar changes though
the cache shall need to be dropped

The current pruning system which translates the ordering in a lambda expression into a natural language ordering for
multi-argument predicates does have a dependency on non-terminal grammar rules which has been ignored. A more
robust scenario would be to process the non-terminal rules to infer the ordering. Language generation issues relating
to adverbs, adjuncts and tense has not been covered by this paper and is another area for future review.

As for future steps, the author would like to develop a text simplification system which performs the following:

1. Translate a complex sentence into first-order logic
2. Develop a processing layer that can mathematically simplify a first-order expression
3. Develop a processing layer that can break a first-order expression into independent expressions
4. Apply the approach described above to generate a natural language translation from the simplified
expression

I would like to thank Prof. Graham Katz for providing the opportunity to write this paper and for his feedback.

9. References
Blackburn, P & Bos, J. (2003). Computational Semantics for Natural Language. NASSLLI.

Strzalkowski, T. et al. (1994). Reversible Grammar in Natural Language Processing. Kluwer Academic Publishers.

Whitney, R. (1988). Semantic Transformations for Natural Language Production. ISI Research Report.

Dymetman, M. & Isabelle, P. (1988). Reversible Logic Grammars for Machine Translation. Second International
Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages.

Smullyan, R. (1994). First-Order Logic. Dover Publications.

Natural Language Generation from First-Order Expressions

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Viewers also liked

Viewers also liked (20)

Similar to Natural Language Generation from First-Order Expressions

Similar to Natural Language Generation from First-Order Expressions (20)

Recently uploaded

Recently uploaded (20)

Natural Language Generation from First-Order Expressions