Limitations...

Over the years, I have made lots of paper simulations of computer learning through fictional human/computer dialogs. I think I did a good job of imitating computer ignorance, but I stopped short of any real breakthroughs in mechanism. To borrow a metaphor, I am like an alien observing cars from space. I can do a good job of imitating their look and behavior, but I don't have any knowledge of fuel, combustion engines or the people driving them.

The hardest part in building this conversation engine is coming up with simple quantum rules that when assembled, resemble human learning and reason. It is hard because you not only need all the rules, you need to get them precisely right.

I think over the course of the few blog entries that I've made, I have discovered that many of my breakthroughs came in the form of 'acceptable ways the engine can fail'. As humans, we have unrealistic expectations about what a 'smart' machine can say. My conversation engine can be forgiven a wide array of common sense lapses because it is starting its existence without senses.

I imagine a time after I have created a presentable engine in which I talk to the press about this thinking machine... The scenario that comes to mind immediately is a horrible segment on a morning talk show with Todd Wilbur. Todd wrote a book called "Top Secret Restaurant Recipes" in which he divulges restaurant recipes. The talk show set up a blind taste test panel to see if they could tell the difference between his recipes and the real thing. The specific example I remember was a big-mac-alike. The panelist could easily tell the real thing from the home made version. Every one of his dishes failed because the test was inappropriate. His book was about coming as close as you can at home, not about producing food that mirrors the production processes of a fast food chain. After the big mac taste test, the panelist actually said the home-made version tasted better, but was drowned out by the jocular scorn of the hosts: "Whoops! Blew that one - better go back and change your recipe."

I see this happening with the public release of the conversation engine. Techies will get it and make a buzz. The world will hear: "some programmer made a computer think like a human" and will set it up for failure because of poorly managed expectations. My attempt to halt this freight train would go something like this:

Imagine you were born without the ability to see or hear - like Helen Keller. Now imagine that you also cannot smell, taste or touch. As humans with these senses, it sounds like a miserable, confining experience, but if you are born without these senses you won't mourn their loss; you simply "are" without them. Helen Keller was not miserable without vision or hearing - in fact, her most famous quotes are profoundly optimistic.

Now, how would you "be" without these senses? What would you do? It is hard to imagine because you have no input from the outside world. You would have some basic pre-wired rules, but no sensory input and no language. The conversation engine has one sense - text. With text, the conversation engine can read, speak and learn.

You will find that it can use metaphors like "bright" for a smart person or "I see what you mean" for "I understand", but doesn't truly comprehend the relationship because it lacks vision input. You will find blind people do the same thing. While a world of blind people would not conceive of the word bright, much less use it in a descriptive metaphor our human tendency to label and simplify makes it possible to convey meaning using these words regardless of our ability to comprehend the word's origin.

When talking with the conversation engine, you will be struck by how profoundly 'not human' it is. I hope you will also notice how eerily human it is occasionally. This is a starting point. It will be up to others to add human senses to the equation. Once they do, who knows? Maybe we will finally have a tool for understanding ourselves.

An Introduction to "Cepts"

Before I had heard the term 'meme', I came up with the idea of 'cepts' - an abbreviation of 'concept'. A meme is a 'cultural unit', a bit of common wisdom, a tale, a slogan... anything that has inserted itself into mass common sense. At first, I thought the accepted 'meme' replaced my invented 'cept' entirely, but it does not. While the two ideas are related, cepts are much more granular.

A cept is a nugget of thought. Our brains are fantastic summarizing machines. We take complex ideas and give them simple labels in order to piece them together with other ideas and summarize them further. When we don't have a single word for a cept, we use a phrase.

The identification, labeling and cataloging of cepts is an important part of the conversation engine.

Cepts are single words or phrases that encapsulate a thought.

"Apple" is a cept, but so is "isn't that special?"

The following are single cepts:
"That's a good point"
"If you want"
"not a problem"
"thank you"
"no, thank you"
"how did you do that?"
"did you do that"
"what are you doing?"

Cepts are chunks of sentences that we assemble like tinkertoys to communicate.

The sentence "Where is the hotel?"
Has at least two cepts.

"Where is" and "the hotel"

I say 'at least two' because there may be an overlapping cept "where is the" and a single word cept "hotel"

Words used in cepts can be used in other cepts.
"Where is the" and "Where's the" are two cepts that represent the same thing: An interrogative begging a response with location cepts, lack of knowledge cepts or assistance to gain this information cepts.

Interesting point -- cepts are used to establish context and narrow the scope of conversation.

Email notes... January 17, 2011

This is an email that I sent to myself. It is just free-flowing thoughts on my discussion engine idea.

Truth. Believability. Interest. Trust. Likeability. Enjoyment.
People who are truthful give believable information.
Information with no supporting info is believed but suspect. (gullability)
Information from different entities that agrees, + truth +trust
Information from different entities that disagrees 1) more trustworthy ent is more believable. 2) subject believability decreased in proportion w trust

All entities ascribed trust level. Truthful info =trust+
interest = ? Subjects with most truthful info that have unanswered questions.
Interest fades with time since last 'considered'
Time fades trust and distrust along logarithmic curve.
Astonishment is proposal or truth of likely believability that is novel or unexpected.
People of high trust are enjoyed.
Funny = zig zag + novelty
novelty = interest+
Factors affecting interest(topic): knowledge

Knowledge=accumulated truths, questions
"makes sense" = confirmed truths by trustworthy sources
Truth(topic) is increased proportionally by the trustworthiness of the source
Trust(entity) is increased by creator++++, proportionally by number of truths given or truths confirmed (trust of ent1 increases if ent2 independently confirms it.)
If ent1 and ent2 are friends, truthfulness of confirmed information not as strong as unrelated people.
"makes sense" when info follows classical logical rules but can be gamed when trustworthy people confirm illogical ideas. This rule (trust by community confirmation) makes it possible for AI to believe in god. Depends on weighting... Can logic trump trust? If so, AI will disagree with unproven beliefs of creator.
Need. AI needs creator or community? Initially, creator trumps all, but at maturity, AI can weight creator trust(!) following human puberty model. Opens possibility for AI to be gamed away from trusting creator. (Cool.)
Proof = logical support of questionable info with truth
Ultimate truth = info accepted on blind faith. These are anchors.
Logic can trump blind faith. When happens, can radically alter personality of AI. Can shatter complex trust architectures.

All of these underpinnings are great for an AI engine that already knows how to communicate, but I want it to build from language acquisition up. For AI, communicating is as interesting as moving is to a human. What environmental rules will be conducive to learning? In order to operate in a world where complex attributed like trust and novelty have meaning, the AI must have language... Interrogatives, value assignment, labeling(naming) and judgement lexicons.. Probably qualitative and quantitative lex, too. All of these base lexicons can and should grow.

Is
Is not
What does x mean?

Context. Subject + Time
Do I mean Chicago the band or Chicago the city? 1) clues from sentence 2) clues from previous sentences scan relations quickly. If no context, ask. If name exists in lexicon ask for clarification.

Do you mean x or y?
Good!
Bad.
Yay!
Aw.
Yes.
No.

Conversations.
The hardest thing for me to see are pregnant pauses. What sparks AI to talk. It needs a greeting lexicon. And polite engagement and disengagement discourse.
[connect]

Hello!

If no response, as time passes, AI loses interest. If interest falls below a certain threshold , it will "focus" on something else like trying again.

Hello? Are you there?
Sec.
What does "Sec." mean?

Needs punctuation lexicon.

Creator will undoubtedly need shorthand for quick, high trust fact assignment.

Similar. A=B is easy. A~B difficult.

is similar to
Is like

Analogy lexicon?

Big, small, fast, slow. Need comparison lexicon for qualities to make sense. I thing this all falls under classical logic rules.

Hello
...
Hello? Are you there?
...
I am closing this connection to save resources

(The following is completely wrong, and in many cases exactly backwards - BM 9/6/2011) Irritation. Anger. AI is irritated with info source when large truth trees collapse. Excited when new large truth trees constructed. AI is irritated by wasteful discourse. E.g. Confirmation of truths that are already well supported and medium age. Irritates by junk data. Should disconnect if sufficiently irritated.

Interest.  Interest has a bell curve based on time. Recent is interesting. Medium time is not interesting. Long time is more interesting. This is due to novelty. Novelty is high for recent new info, drops off rapidly and leaves altogether after medium time. Long time means it is forgotten and thus novel again.

Recognition. AI recognizes by login, secondarily by IP. IP is treated like the login's community.

Forgetting. To conserve resources, data ages. Any info that is not reaffirmed fades and can be forgotten completely.

Environment must define "recent" in quantitative terms. Would be good to have access to human research on context.

Need. AI might not behave irritably (or as irritably) toward creator and truthful people because it needs them. Need for AI is a need for quality information.

It has been a while since we talked!

AI will like you more if you make sense.

Understanding vs comprehension - Part II

Thinking more on this topic... I mentioned that comprehension is a subset of understanding. I think it is more accurate to say that comprehension requires understanding and experience. Comprehension is about understanding facts and relating them to prior experience in order to forecast goals and motivations. That last bit is tricky.

Every time I start to distil perceptive mechanisms, I try to take a step back to see how this will work in the conversation engine. The goal of the conversation engine is to behave like a human within the confines of text. The engine cannot see, hear, taste, touch or smell. It quite literally is a brain in a box. It's only sense is it's method of communication - text.

Optimally, I would like the engine to learn to speak on it's own by using the mimed human logic that I am trying to divine here. Unfortunately, I'm not sure that could happen quickly enough to be practical, so I have to establish a base grammar. That base grammar can change over time, (it should not be immutable) but to start with there must be words with established meaning. I suppose I don't think language can evolve naturally because without a base grammar, there is no way for the engine to distinguish one word from another.

Hmmm... I take that back. The engine could have base character rules that could lead to word and language understanding, but that will be much more complex than starting with a system that can understand some basic words.

Honestly, these are the easy problems. I am going to have a much harder time figuring out how to quantify words and how to manage understanding and comprehension. My goal is to discover simple rules and build little engines to test those rules. I don't have the first rule yet... Just some interesting properties. To be honest I am having a hard time even putting all of the pieces on the table - to say nothing of putting them together.

Understanding vs comprehension: mechanical context

Definitions of understanding and comprehension as found on the web are often self referential or invoke other abstract words like meaning and knowledge. In a human context, I found these definitions unfulfilling; in a machine context, they ar not helpful at all.

I am trying to define two states of knowledge acquisition and processing that can be demonstrated by a childhood memory of mine. In 1977, I saw Star Wars in the theater. In the opening scene, A tiny consular ship is being fired on by a star destroyer. As a 5 year old, I understood that the big ship was trying to blow up the little ship. My comprehension of scene was that the small ship is the good guy and the big scary ship is the bad guy.

Understanding is about "what".
Simple understanding: The big ship is shooting at the little ship. Big ship is the bad guy, little ship is the good guy. Big ship, strong. Little ship, weak. Understanding is about ascribing qualities or assuming qualities based on analogs from previous experience.

Comprehension is about "why".
Simple comprehension: Big ship is chasing the little ship because the big ship wants to blow up the little ship. Complex comprehension: Darth Vader is on the big ship and wants to capture (not destroy) the little ship because someone on board has stolen plans to the death star. Deep Comprehension: why would it be necessary to find the plans aboard the ship when the technologically advanced people would be able to replicate and transmit the plans? Comprehension is a subset of understanding that is concerned with motivation and intended goals.

Another example: A bully beats up a boy and takes his lunch money. Another boy who witnessed the event understands that the bully beat up the boy. The witness comprehends that the bully beat the boy up to steal his money.

As understanding grows, so does the potential for comprehension, but the two are not lock step.

I realize that this is my first blog entry, but if you are reading is you will be keenly aware that you have walked into this movie in the middle.

I will restate this more thoroughly later, but my goal is to develop a simpler, more intuitive, more effective conversation engine. (Think Loebner prize winner but without canned responses.) I am trying to distil the mechanics of thought in order to produce a thinking machine.

more later...