Thursday, October 31, 2013

Cognitive architecture

Memory should be partitioned according to perception in the cognitive architecture. Sensory memory includes sensory stimulus that is completely unprocessed, short term memory includes sensory stimulus well it is being processed, and empirical memory includes sensory stimulus that has already been processed. Conceptual memory contains logical ideas that are independent of perception and other conceptual generalizations.
  • Sensory Memory
  • Short Term Memory
  • Empirical Memory
  • Conceptual Memory
The organization of conceptual memory should be based upon an ontology of concepts. This ontology should include all mathematical concepts such as numbers, lists, and relations as well as concepts whose instances are described in empirical memory.

Wednesday, October 30, 2013

CHREST cognitive architecture

In my previous discussion on reasoning and perception I came to the conclusion that cognitive architectures should not be characterized by the distinction between perception and action as in ACT-R. One of the cognitive architectures that properly recognizes the central role of perception is CHREST.

CHREST perception facilities are organized into short term memory (STM) and long term memory (LTM). Recognition is used to relate elements in the STM to elements in the LTM. It is my contention that empirical knowledge should form a special category of the LTM which is far more dependent upon the changing interpretations of sensory perceptions then logical statements.

A fundamental aspect of CHREST is the idea of a chunk which is a maximal familiar substructure of a given sensory stimulus. CHREST uses an advanced system of attention management to determine what elements of the environment should be perceived. Many of the elements of CHREST were defined by the earlier cognitive architecture EPAM.

Tuesday, October 29, 2013

Methods of solving mathematical problems

The simplex algorithm is a fairly effective method of solving linear programming problems though there are many linear programming problems for which an inexact solution with optimal utility is desirable. For problems in which we cannot produce exact solutions we need to use heuristics to get a best approximation of the optimal solution.

Metaheuristics can be used to produce solutions to mathematical problems under conditions of limited computational capacity. Hill climbing is a metaheuristic that can be used to find local optima by iteratively improving an arbitrary solution to a problem but it is not guaranteed to produce a global optima.

The hill climbing metaheuristic can be used for example to optimize a solution to the traveling salesman problem by first finding a basic feasible solution and then switching the order in which certain nodes are visited as long as that switch improves the solution. There are various other metaheuristics such as tabu search that can be used to solve mathematical optimization problems.

Friday, October 25, 2013

Some types of mathematical problems

Analytic optimization
Analytic optimization problems are defined by the optimization of a function over a set of real vectors to the set of real numbers $\mathbb{R}^n \to \mathbb{R}$. The most generally applicable type of analytic optimization is linear programming so it is worth starting out with that and then continuing on to describe the nonlinear cases afterwards.

Linear programming:
Real linear programming problems can be solved using the simplex method, however, most integer programming problems don't have exact solutions so they require the use of heuristics. A variety of combinatorial problems including TSP, covering problems, and boolean satisfiability can be solved using integer linear programming.

Nonlinear programming:
Given some general multivariable real function we can use the combination of partial derivatives and equation solving to solve unconstrained optimization problems. Otherwise, there are special kinds of nonlinear programming such as fractional programming and quadratic programming that have their own sorts of solutions.

Goal oriented optimization:
In general goal based problems include a discrete temporal model and a predicate that determines rather or not the current state is the goal state. In planning problems the problem is generally to find the shortest sequence of moves to reach the goal state and in turn selection problems the problem is to find a move that maximizes the frequency of wins and minimizes the frequency of loses in the game tree.

Planning problems:
The purpose of planning problems is to find a sequence of moves to achieve the goal state that is optimal with respect to some ordering condition such as the size of the sequences. Planning problems include shortest path problems such as the problem of finding the shortest path in a maze and combination puzzles such as the 15-puzzle and the Rubik's cube puzzle.

Turn selection problems:
In any combinatorial game such as chess, checkers, go, connect four, or tictactoe players select moves each turn out of the set of valid moves based upon rather or not their move is going to achieve the end goal taking into account all possible future moves.

Tuesday, October 22, 2013

Perception and reasoning

After thinking about the different branches of AI I now feel that the most important basic distinction to make is between perception and reasoning. This is similar to the distinction between empiricism and rationalism in epistemology.

An intelligent agent that isn't dealing with an unknown environment such as one that is playing a game such as go, chess, checkers, or tictactoe in which there is no fog of war has no need for perception as we know it. Likewise intelligent agents that are solving logical puzzles such as sudoku only need to use their reasoning.

When an intelligent agent needs to understand the real world it ends up producing an empirical knowledge base which includes facts such as that Einstein once lived from 1879 to 1955 and patterns in the spacetime environment such as Newton's universal law of gravitation. Every single object in the empirical knowledge base is uncertain to some probability and dependent on the current point in time.

One of the fundamental things about the reasoning / perception distinction is that the aforementioned empirical knowledge base can be defined as a removable part. An agent capable of intelligent reasoning should be able to learn everything about the world from scratch based upon its learning and reasoning capabilities alone.

Likewise, an intelligent agent that know nothing about the real world could still play chess intelligently or produce logical solutions to sudoku. The reasoning component in general is very much mathematical / logical in nature as it focuses on mental objects that may have no physical instantiation.

One possible alternative to the reasoning / perception distinction is the declarative / procedural distinction, however, I don't think this is really valuable because there is no way we can really remove a procedural component from an AI as the techniques of optimization and decision theory are too fundamentally intertwined with the rest of the agent's reasoning capabilities to really be removable.

Sunday, October 20, 2013

Problems intelligent agents must confront

An intelligent agent that percieves its environment and acts upon it to maximize its own utility must confront at least three significant problems:
  • limited actions: an agent with limited actions must use decision theory to determine which action is the best one to take in this situation.
  • limited perception: an agent with limited perception must create models of its environment based upon the evidence obtained by its observations.
  • limited time: an agent with limited time must give attention to certain issues, plan, and predict the future.
To give an example of how this limitations might relate to games, logic puzzles like sudoko are purely in the realm of mathematical logic so they do not fall into any of these categories of limitations. Games like chess and go have limited actions as you need to select which move you think is the best each turn as do games in which the outcomes of actions are uncertain.

When it comes to limitations on perception and time, turn based games with fog of war have limited perception, real time games with revealed maps have limited time, and real time games with fog of war have limited perception and limited time.

The primary purpose of the scientific method is to deal with limited perception. With the scientific method we create models of the outside universe and we use perceptual evidence to determine which models of the universe are the most accurate.

the major object of perception of the physical universe is light which is dealt with by computer vision. Other senses include sound, touch, taste and temperature. Evidence obtained from such sense can be used as evidence to evaluate the accuracy of models of the physical universe.

The area of limited time is related to limited action through the notion of planning in which agents create plans of actions that they intend to perform over time and limited time is related to the area of limited perception through prediction as agents can create predictive models that determine what will happen in the future.

Friday, October 18, 2013


The FRIL language (Fuzzy Relation Inference Language) combines logic programming with uncertainity. FRIL combines support for classical logic with support for fuzzy logic and probability unified by mass assingments. FRIL has a list based syntax so in a way it is a dialect of Lisp:
  (1 2)
  (2 3)
  (3 4))
FRIL supports both discrete and continuous fuzzy sets each with their own syntactic representations. The extension of classical logic with fuzzy sets makes FRIL a much more general and powerful language then Prolog.

Thursday, October 17, 2013

Extensions to classical logic

An intelligent reasoning system must be able to effectively handle information that is uncertain, imprecise, and vague. This implies that an intelligent reasoning system should have a much richer system of logic then the boolean algebraic logic which is used to define mathematics.

Extensions to the classical logic system including fuzzy logic which has degrees of belief, paraconsistent logic which has both belief and doubt, and probablistic logic which uses probability distributions. A reasoning system which uses such advanced logics may play a significant role in the construction of an AI.

Monday, October 14, 2013

Mathematical logic and order theory

An inescapable conclusion of the study of order theory is that mathematical logic is deeply intertwined with the study of partial ordering relations. We can form partial orders called concept hierarchies corresponding to the logical inclusion of predicates and we can represent elements of partial orders that form distributive lattices including well orders as sets or predicates. This naturally leads to the definition of Von Neumann ordinals in a well order for example. A description of the relation between mathematical logic and order theory is forthcoming.

Thursday, October 10, 2013

Complete lattice extensions

The natural numbers with infinity can be described as a complete lattice which can be extended by the integers with positive and negative infinity which can in turn be extended by the complete lattice of real numbers.

Given a real number such as 27/5 we can get an interval approximation for this number through the integers sublattice of (5,6). Likewise for a negative real number such as -5/4 we can get an approximation of (-2,-1). Since the sublattice of natural numbers doesn't contain any negative numbers the only approximation we can get there is (> 0).

Given the trivial lattice containing just the number zero we can get approximations of (< 0), (> 0), and equal to zero which are essentially just the sign of the number. Given the complete lattice of real numbers we can always get a standard part which doesn't even need to be expressed as an interval because the real numbers form a dense complete total order.

The surreal number $27/5+\frac{1}{\omega}$ has a standard part of 5, an integer part of (5,6) as described earlier, and a sign of (< 0). The infinite surreal numbers $\omega$ and $\sqrt{\omega}$ both have a standard part of (< Infinity). Complete lattice extensions can be used to construct ever more advanced total ordered number systems.

Thursday, October 3, 2013

Continuous iteration of exponentation

The continuous iteration of the exponentiation operation may hold the key to extending transseries to deal with larger and larger growth rates. We can iterate the exponentation operation to integer arguments for example $exp^2(x) = exp(exp(x))$, $exp^{-1}(x) = log(x)$, and $exp^{-2}(x) = log(log(x))$ so the key to continuous iteration are the fractional iterates of exponentiation such as the half-iterate.

These fractional iterates of exponentiation apparently can be described by the natural tetration operation $tet$ and its inverse function in a similar manner to how we can describe general iterates of multiplication using $exp$ and its inverse function $ln$. As a result of this it may be the case that we should extend transseries with the $tet$ function in order to implement iterates of $exp$ and larger growth rates.