This paper has a diagram showing the multi-level hierarchical control system that is structured as I’ve come to expect from competition-winning autonomous vehicles. The lowest sub-symbolic level maintains the helicopter’s pitch and roll within a 1 ms feedback control loop using accelerometers and gyroscopes. The next higher sub-symbolic level operates a 100 ms feedback control loop to maintain pose and obstacle avoidance with respect to the helicopter’s surroundings using a laser rangefinder and stereo cameras. The final and topmost level is symbolic, performing trajectory planning with three second temporal granularity.

It appears that the symbolic processing can be performed by the helicopter’s on-board lightweight processing power. In contrast, certain heavy computational tasks, which I suppose includes sub-symbolic realtime perception of stereo images, are performed by a wireless networked laptop computer cluster. I think that this research points to how domestic and workplace robots will be organized. Mobile robots which have a local operating environment need not carry around heavy unshared computer processors. Rather, when wireless bandwidth permits, a networked computational resource can occupy a fixed location, and offload heavy sub-symbolic computational tasks from the mobile robot.

I bring this research to your attention because it demonstrates how robust symbolic processing can be grounded to the real world by a hierarchy of increasingly less abstract sub-symbolic levels, and because I believe that this architecture is generally applicable to AGI.

I’m quite sure you [another list member] haven’t the teeniest hope of building a linguistic AGI as you and others envisage . The reality check I’d suggest is – can you/your machine understand or construct *two* consecutive sentences? (Can you IOW [in other words] understand a *text* rather than individual, isolated sentences?) No current method can – or ever will. Basically because understanding or constructing a text is not just a matter of putting together symbolic words, but constructing an imaginative model/scenario of what is going on in and behind the text – and extensively “reading between the lines.” Linguistic processing that works will therefore require not just one but a whole series of truly massive creative leaps. My advice to any systembuilder  – don’t go near it if you value your sanity. One creative leap at a time.

By happy coincidence, Mike had steered the worthwhile discussion thread directly into a development approach I’ve recently adopted for Texai.

Over the summer, I fixed a series of errors in the Texai lexicon by rebuilding it from its sources: WordNet version 3.0, and Wiktionary.  Then when I returned to working on my initial lexical semantics acquisition dialog system, what was previously an ignored problem became intolerable.  By rushing to get the initial Texai bootstrap dialog system deployed by last June in time for Turing’s birthday – an effort that failed by the way – I had hard-coded the simple dialog steps into what I called the noun plural acquisition skill.  When I closely examined this Java code a few weeks ago, the hard-coding anti-pattern became obvious.  As a solution, I determined that the hard coding of the dialog conversation steps could be factored into a declarative framework of dialog operations, and into a procedural interpreter for the framework.  The dialog framework consists of Java objects that can be persisted into the Texai knowledge base for reflection.

My response to what Mike said above can be conveniently stated with respect to this new Texai dialog operations framework.  Firstly, by using incremental fluid construction grammar, I believe that I can parse and disambiguate, via cognitively plausible spreading activation, the sorts of English utterances required for the bootstrap task of acquiring RDF semantics for the most frequently used word senses in the Texai lexicon.  Secondly, the dialog framework neatly avoids the problem Mike described, by factoring the required model into either the framework or the associated framework interpreter.  The dialog context contains the model, initially populated by RDF statements with variables, and later populated, and the variables bound, by the ensuing conversation with the user.  The Texai dialog framework implements a controlled English language, in which only certain forms of questions and responses are permitted.  Within these constraints, I expect that Texai will fully understand what it generates and what it parses.

To better illustrate the Texai dialog framework, consider these structured conversation utterances:

[Texai] Consider “<utterance>”.
What is the lexical category of “<word>”?
[User] “<word>” is a <speech-part>.

[Texai]  Consider “<utterance>”.
Which is the word sense for <NP>?
1.<definition1>
2.<definition2>
3.…
[User] 2

[User] Define a new <speech-part> word sense for “<word>”.
[Texai] What is the new definition of “<word>”.
[User] A new <speech-part> definition of <NN> is “<definition>”.

In the first two dialog exchanges, Texai has the dialog initiative.  The third one, the user has the initiative until Texai begins the specified semantics acquisition dialog.  The Texai dialog framework assumes that Texai has the dialog initiative and that a script of descriptive informational prompts, question prompts and answers can be structured as a conventional flowchart.  The Texai dialog framework provides primitives for the aforementioned prompts and answers. It has structured dialog operations for scoping dialog variables, conditional script branching, and for conditional script iteration.

As an initial step towards more general skill acquisition, I plan a set of dialog frameworks that allow creation of new, and editing of existing, dialog frameworks.  Thus Texai should not only be able to use controlled English dialog frameworks to acquire semantics for its WordNet and Wiktionary derived lexicon, it will also be able to revise and extend those same frameworks using framework-editing dialog frameworks, which is bootstrapping in action.

When allowed dialog initiative and if not otherwise occupied by a currently interpreted dialog, Texai will start a new dialog script using the next workflow item from the knowledge acquisition workflow framework that I wrote last spring.

My design error started years ago when I incompletely adapted an open source  relational data model for WordNet.  For ease of searching, WordNet provides an index into its synonym sets, i.e. synsets, in which all word forms appear in lower case.  My mistake was using this lower case data element as the uninflected word form.  Correcting the design error required redoing the Texai lexicon from two of its original components: WordNet and Wiktionary.   Although the CMU Pronouncing Dictionary was also a source in the original Texai lexicon merge, I omitted it this time around because there are no immediate plans to incorporate the Sphinx automatic speech recognition system for voice input.  Words in the CMU dictionary are all given in upper case, so I would have a somewhat complex program to link them up to the now properly cased Texai lexicon. When redoing the lexicon from scratch, I took advantage of extracting from the latest 3.0 version of WordNet, and also extracting from the latest Wiktionary XML dump.   I substituted another open source Java  API for WordNet- JAWS, whose data model was closer to what I need for Texai.   After two plus years of dialog system development, I have a better idea of what the object model of the lexicon should be, and I consequently simplified it during the re-population process.

Texai English Lexicon initial population statistics:

Number of English words acquired from WordNet       148,730
Number of English word senses from WordNet          299,332
Number of OpenCyc mapped terms                       10,835
Number of OpenCyc mapped word senses                 22,930
Number of English words acquired from Wiktionary    117,313
Number of English word senses from Wiktionary       160,452
Number of inflected word forms from Wiktionary      227,573
Number of RDF Statements                          7,383,412
812 statements added per second
Total run time: 150 minutes 56 seconds (on my dual core development computer)

There are a total of 266,043 English words and multi-word phrases acquired from WordNet v3.0 and Wiktionary.  There are likewise a total of 459,684 word senses acquired from the two source dictionaries.  Because OpenCyc provides mappings for over 10,000 of its commonsense terms  with WordNet 2.0, I used an available sense mapping between WordNet 2.0 and 3.0 synsets in order to map between OpenCyc terms and WordNet 3.0 synsets imported into the Texai Lexicon.   As a result, 22,930 word senses are mapped to OpenCyc terms, but these amount to only 5% of all word senses.  Perhaps 10,000 more will have to be mapped by volunteers via the dialog system in order for Texai to understand what these word senses mean in logical statements, and subsequently bootstrap itself – parsing the definitions of the remainder to semi-automatically learn them.

After correctly re-populating the Texai English lexicon, I re-ran the population of the dialog work flow queues.   There are now less than 400 items in the work flow queue for which Texai seeks to acquire the plural word form of frequently used nouns. This number is lower than the previous version because Wiktionary is now more complete with regard to noun inflected word forms, thus many more plural word forms were imported.

For the next three weeks, I’ll be away from my development system, so I am using my netbook to create use cases and prompted dialog scripts for the semantic of mapping of nouns, verbs, adjectives, adverbs and prepositions.  My objective is to deploy the Texai bootstrap English dialog system, acquiring semantic sense mappings in time for the results to be included in a paper submitted for publication  by the AGI-10 Conference.

hello stephenreed
* Plural word form for any of these nouns ‘or’?
or (ProperNoun)
OR (Noun)
(y) The plural is ‘ors’.
(n) There is no plural.
(s) Skip this item.
(d) What are the definitions of this word?
Or provide the plural in quotes.
n

You are probably confused by this. My wife and sponsor, BethLynn Maxwell certainly was. Although I plan a help page to explain the dialog format to novice volunteer mentors, it needs to be clear and self-explanatory.

Here is what the workflow item is attempting to communicate.

Each item is a frequently occurring word from the defining words in WordNet or Wiktionary dictionary definitions. The items are restricted to the noun category in which the plural word form is currently missing, and are presented in order of usage frequency.

The first utterance, ‘hello stephenreed‘ was generated by Texai when I logged in. It was triggered by a hidden ‘hello’ utterance sent on my behalf by the chat client when the web page first appeared.

The next line ‘ * Plural word form for any of these nouns ‘or’?‘ is a prompt from Texai that was meant to communicate the non-inflected noun word form. It fails from a usability viewpoint for these reasons:

1. The word ‘or’ is not obviously a noun. It is the first workflow item because the word ‘or’ is the fourth most frequently occurring defining word, according to my tabulation. Although there are open-source part-of-speech taggers, available, I perhaps incorrectly assumed that simply tabulating words without regard to their part-of-speech usage would be OK. Consequently, Texai knows that it is a conjunction part of speech, but from the OpenCyc lexicon, Texai also believes that ‘in’ is a noun having the definition “a state in northwestern United States on the Pacific”. It would be somewhat more clear if ‘in’ were correctly presented as ‘IN’ but somehow in the transformation from the merged lexicons of WordNet, Wiktionary, OpenCyc and The CMU Pronouncing Dictionary, the upper case word form was incorrectly extracted in lower case.
2. The leading asterisk is supposed to indicate that Texai did not generate this utterance from logical propositions, as ultimately it should, but instead used an opaque character string assembled from an inflexible template.
3. The prompt fails to explain what else Texai is trying to say in the following lines.

The line ‘ or (ProperNoun)’ is supposed to indicate a non-inflected word form for ‘or’ that Texai believes is a proper noun. It fails for these reasons:

1. Because the English word ‘or’ is used predominately as a conjunction, it is very confusing for it to appear where one might expect a noun.
2. The word ‘or’ violates the capitalization rules for English proper nouns.

The line ‘OR (Noun)‘ is better because some mentors may realize that ‘OR’ is an abbreviation for the US State of Oregon.

The remaining lines suggest what the mentor can enter to teach Texai the requested noun plural word form. The mentor can type ‘y’ to confirm that Texai’s heuristic guess is correct, or type ‘n’ to say that there is no plural for the given word, type ‘d’ to request the noun definitions of the word, or finally the mentor can enter the correct plural word form as a quoted string.

Here are my current ideas to improve the usability of this dialog.

1. Using a large list of English noun non-inflected word forms available from Desi Quintans, I’ll filter the existing workflow list to requeue entries like ‘or’ into a separate exception workflow list for someone like me to process. The dialog script to handle the exception workflow list will require facilities to edit the existing Texai lexicon, for example to consolidate the two noun word form entries for ‘or’ into one: the proper noun ‘OR’. After this filtering, I expect the first worklist entry to be ‘person’, which is the 25th most frequently used defining word in the Texai lexicon.
2. I’ll automatically include the noun word sense definitions in the prompts, so mentors can immediately understand the word.
3. The dialog prompts will be re-written to more clearly communicate what is meant, even if the result is more verbosity. I plan for this version of the dialog prompts to be the novice mode, and in the near future have an available expert mode that can be requested by the mentor that features very concise prompts.

These remaining issues to be resolved are modest compared to the more technical issues already recently completed. The Texai dialog system has been migrated from my development computer to the production server and can be reached through my firewall from the Internet. I’ll email the URL when the dialog is ready for more testing by those of you who have already contacted me, and shortly thereafter I expect to put the URL on the Texai website, for your help and feedback.

Thanks in advance.

[update]

I’ve filtered the list of nouns to work on and there are 2220 that are on Quintans, list and are missing plural word forms. There are 17095 items in list of exceptions to work on. Most of these are WordNet words that do not appear on Quitans’s list such as biological, chemical, or geographic place names. Others are more difficult exceptions that will require new dialog scripts to edit the existing lexicon.

Furthermore, I’ve redesigned the prompts for teaching Texai missing noun plurals, as shown in this mock-up for the noun ‘person’.

[texai] The definitions of the noun ‘person‘ are:

1. Human being; individual
2. Specific human being
3. The physical body of a specified individual
4. Any individual or formal organization with standing before the courts
5. A linguistic category used to distinguish between the speaker of an utterance and those to whom or about whom he is speaking. See grammatical person

I reason that the plural of ‘person’ is ‘persons‘. Is this correct for any of the above definitions? Enter y/yes to accept my choice, enter n/no to teach me that ‘person’ has no plural, enter s/skip to go on to the next item, or enter the correct plural for ‘person’ in quotes.

Note that the last sentence in particular will be challenging to generate from a structured set of propositions because I’ll need to write new grammatical construction rules for the phrasing. That will be a future task, for now Texai will create the sentence via a hack – template substitution.

Your comments appreciated!

Of the present day used systems, which products do you consider good business cases of applied AI? I’m looking for concrete product examples (not just wide technologies), in either consumer, or business everyday usage, with high profitability, and/or impact factor.

Having read the proceedings and indeed attended several of the annual conferences of the Association for the Advancement of Artificial Intelligence, I easily formed a relevant list of topics to search. Here are the pertinent results from those searches.

• voice recognition products from Dragon
• autonomous industrial robots
• machine vision
• automated scheduling
• constraint satisfaction from Microsoft
• route planning
• symbolic computation
• machine translation
• automatic landing systems
• recommendation systems
• logistics optimization
• stock trading
• pattern recognition
• spam filters

So where is AI currently used in a business context? Obvious answer: many places.

My arguments have always been top-down (complexity has to be there
because the overall features of the system don’t seem to be
implementable without it), so the way I interpret these low-level
results is that the brain has evolved to have all that self organized
criticality down at the bottom BECAUSE that is the easiest way to sort
through all of the different designs, to find ones that give optimum
high level behavior.

But to me, its content confirms an approach that I’ve taken with Texai to tackle AGI complexity.

Two points caught my attention, quoting from the article:
(1)

As it processes information, the brain often synchronises large groups of neurons to fire at the same frequency, a process called “phase-locking”. Like broadcasting different radio stations at different frequencies, this allows different “task forces” of neurons to communicate among themselves without interference from others.

This phenomenon suggests to me that a message-organized architecture is cognitively plausible, in that phase locking permits the transmission of information between sender and receiver without interfering in other such interactions.
(2)

Small-world networks lie somewhere between regular networks, where each node is connected to its nearest neighbours, and random networks, which have no regular structure but many long-distance connections between nodes at opposite sides of the network (see diagram). Small-world networks take the most useful aspects of both systems. In places, the nodes have many connections with their neighbours, but the network also contains random and often long links between nodes that are very far away from one another.

For the brain, it’s the perfect compromise. One of the characteristics of small-world networks is that you can communicate to any other part of the network through just a few nodes – the “six degrees of separation” reputed to link any two people in the world. In the brain, the number is 13.

To date, I’ve organized Texai physically as a regular network, where each agent node is directly connected to its nearest neighbors. Messaging however allows the easy creation of a virtual small world network by having intermediate nodes merely relay messages for which they are not the recipient. The experimenters noted that the human brain has 13 degrees of separation between network nodes.

What would be the average degree of separation in a vastly distributed hierarchical network of downloaded Texai instances? To simplify the discussion, I assume that each node has only one superior and 10 subordinate nodes. Thus each level of the network has 10x the number of nodes as the next higher level. Clearly, the great majority of nodes are located at the lowest level, and I will use these terminal nodes to represent the average node. A terminal node can reach its 9 direct siblings by messaging through its directly superior node, for 2 degrees of separation. It can reach 99 indirect sibling terminal nodes by messaging through the superior node of its directly superior node, for 4 degrees of separation. Clearly, a much greater number of indirect sibling nodes can be reached by routing the messages through higher level superior nodes in the hierarchy tree. The maximum degree of separation would therefore be about twice the number of levels in the hierarchy tree.

This discussion about comparative degrees of separation between the human brain and Texai’s future network is not meant to suggest that it will take a vast number of Texai nodes to perform human-equivalent intellectual symbolic tasks. Rather, I hypothesize that only a few nodes will be required to perform for example, the symbolic understanding of English text. The intent of the future, vastly distributed network of Texai instances is to model the intellectual capability of human organizations.

What has OpenCyc accomplished that qualifies as step to AGI? Nothing.
The consensus on this list is that OpenCyc should be thrown out: it is
a distraction

During my employment at Cycorp, John DeOliveira and I lobbied successfully for an open edition of the Cyc knowledge base. John and I are still listed as OpenCyc administrators at SourceForge and John not only authored the OpenCyc web site, but subsequently founded the independent Cyc Foundation after leaving Cycorp. Because I was an OpenCyc developer, because I am no longer working for Cycorp, and because I am taking a somewhat different approach with Texai, I believe that I’m well qualified to describe OpenCyc, and argue that its accomplishments are steps toward AGI.

In order of importance, OpenCyc’s accomplishments are:

1. Published a free, well organized ontology
2. Provides an opaque, but free to use runtime object store and inference engine
3. Provides tools to browse, edit and query the knowledge base
4. Serves as the basis for intelligent applications via its Java and socket APIs

Ontology

The OpenCyc ontology is the world’s most comprehensive ontology of commonsense information. If one believes, as do I, that organizing symbolic information is step towards AGI, then OpenCyc is a step in the right direction. Texai uses OpenCyc content as extracted into RDF, selecting only the atomic terms and binary assertions about them. RDF was created by a former Cycorp employee, R. Guha, who was obviously influenced by the previous Cyc frame representation. Cycorp is now targeting OpenCyc at the Semantic Web.

Over 10000 OpenCyc terms have links to equivalent synonym sets in WordNet, thus leveraging the simple OpenCyc lexicon with a much richer lexical resource.

Runtime

The OpenCyc runtime provides an object store and deductive inference engine, hosting millions of logical propositions. Performance is adequate for simple deductive queries. A large number of special purpose inference modules support a very expressive, i.e. higher order, form of predicate calculus logic. I speculate that a Java version of the Cyc runtime is forthcoming.

Tools

The OpenCyc browser is a fast way to navigate around the OpenCyc ontology, and it is a portal into the remainder of the OpenCyc tool set for editing and querying the knowledge base.

Intelligent Applications

For many years, the US government has sponsored Cycorp’s participation in research projects having the goal of creating and demonstrating intelligent applications. Cycorp used the majority of those funds to create what the sponsor wanted, but also used a substantial portion to improve Cyc generally – its fundamental commonsense knowledge, and to improve its symbolic, deductive inference engine.

For Texai, the most relevant sponsored research was the DARPA Rapid Knowledge Formation project, having the goal of dramatically reducing the effort required to create very large knowledge bases.  I was the project’s first manager at Cycorp, and learned important lessons about the utility of English dialog.

If Texai sucessfully creates AGI, then that will be proof that OpenCyc’s accomplishments are indeed steps towards AGI.

I previously posted The Current State of Affairs in Pursuit of a Commonsense AI which has more to say about Cycorp and Cyc.

OpenCyc has a sophisticated context inheritance mechanism is an example of how belief contexts can be represented and processed. My implementation is simpler and thus much faster. The application code, e.g. parsing, either specifies an override context directly, or by omission, specifies the default persistence context of the particular entity that is being found, loaded or persisted. I use semantic Java annotation to associate the name of the default persistence context on the Java class of each persistent entity.

Ordinarily, finding objects for a mentor from the knowledge base proceeds by looking in the mentor’s belief context first, and only if not found there, then looking in the default context for that class of object.  Sometimes it is necessary to know if an object is located exactly in  a particular context.  The API for finding KB objects includes a parameter to indicate whether to find the objects exactly in the specified context.

Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child’s? If this were then subjected to an appropriate course of education one would obtain the adult brain.

Turing then elaborated:

We may hope that machines will eventually compete with men in all purely intellectual fields. But which are the best ones to start with? Even this is a difficult decision. Many people think that a very abstract activity, like the playing of chess, would be best. It can also be maintained that it is best to provide the machine with the best sense organs that money can buy, and then teach it to understand and speak English.

The latter is the approach of the Texai project. Lexical knowledge and skills will be acquired so that the system can bootstrap itself into ever increasing capabilities, as taught to it by volunteer mentors. Turing’s requirement for sophisticated sense organs is reduced by (1) a focus on English text rather than English speech, and by (2) the hypothesis that the development of sub-symbolic processing can be postponed and that sufficiently intelligent behavior can be achieved almost entirely by symbolic processing plus spreading activation, which is much simpler and needs dramatically less computing power.

Celebrating this notable date, I expect to release before midnight a online, rudimentary version of the Texai bootstrap dialog, that will enable volunteers to register and to teach Texai plural noun forms.

[UPDATE] I’m postponing the release for a few more days.  Today’s testing revealed that storing each volunteer mentor’s contributions in a separate belief context required small changes throughout the existing code.