Abstracts of selected publications 1989-1997

1997

P.Kontkanen, P. Myllymäki, T. Silander, and H.Tirri, Comparing Stochastic Complexity Minimization Algorithms in Estimating Missing Data. To be presented at the 4th Workshop on Uncertainty Processing (Prague, Czech Republic, January 1997).
- In this paper we study the problem of missing data estimation, i.e., how to estimate the missing elements of a given incomplete matrix of discrete data, by using the information in the known elements. We approach this problem by assuming a functional form for the probability distribution of the data items, and by producing a probability distribution for the possible completions of the incomplete matrix given. In this Bayesian framework, the optimal data completion can be obtained by finding the completion with the highest marginal likelihood. From the information theoretic point of view, this solution can be seen as the data completion minimizing the stochastic complexity of the full data matrix, where the stochastic complexity represents the shortest code length needed for coding the complete data matrix within the chosen model space. We describe several algorithms for minimizing the stochastic complexity in the missing data scenario, and empirically compare their performance using a variant of finite mixture models as the functional model form. In the empirical tests we use real-world public domain data.
P.Kontkanen, P. Myllymäki, T. Silander, H.Tirri, and P. Grünwald, Comparing Predictive Inference Methods for Discrete Domains. Pp. 311-318 in Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics (Ft. Lauderdale, USA, January 1997).
- Predictive inference is seen here as the process of determining the predictive distribution of a discrete variable, given a data set of training examples and the values for the other problem domain variables. We consider three approaches for computing this predictive distribution, and assume that the joint probability distribution for the variables belongs to a set of distributions determined by a set of parametric models. In the simplest case, the predictive distribution is computed by using the model with the {\em maximum a posteriori (MAP)} posterior probability. In the {\em evidence} approach, the predictive distribution is obtained by averaging over all the individual models in the model family. In the third case, we define the predictive distribution by using Rissanen's new definition of {\em stochastic complexity}. Our experiments performed with the family of Naive Bayes models suggest that when using all the data available, the stochastic complexity approach produces the most accurate predictions in the log-score sense. However, when the amount of available training data is decreased, the evidence approach clearly outperforms the two other approaches. The MAP predictive distribution is clearly inferior in the log-score sense to the two more sophisticated approaches, but for the 0/1-score the MAP approach may still in some cases produce the best results.

1996

H.Tirri, P.Kontkanen, and P. Myllymäki, A Bayesian Framework for Case-Based Reasoning. Pp. 413-427 in Advances in Case-Based Reasoning (Proceedings of the 3rd European Workshop), edited by I.Smith and B.Faltings. Lecture Notes in Artificial Intelligence, Volume 1168, Springer-Verlag, Berlin Heidelberg, 1996.
- In this paper we present a probabilistic framework for case-based reasoning in data-intensive domains, where only weak prior knowledge is available. In such a probabilistic viewpoint the attributes are interpreted as random variables, and the case base is used to approximate the underlying joint probability distribution of the attributes. Consequently structural case adaptation (and parameter adjustment in particular) can be viewed as prediction based on the full probability model constructed from the case history. The methodology addresses several problems encountered in building case-based reasoning systems. It provides a computationally efficient structural adaptation algorithm, avoids overfitting by using Bayesian model selection and uses directly probabilities as measures of similarity. The methodology described has been implemented in the D-SIDE software package, and the approach is validated by presenting empirical results of the method's classification prediction performance for a set of public domain data sets.
J.Lahtinen, P.Myllymäki, T.Silander, and H.Tirri, Empirical comparison of stochastic algorithms in a graph optimization problem. Pp. 45-59 in Proceedings of the Second Nordic Workshop on Genetic Algorithms and their Applications (Vaasa, Finland, August 1996), edited by J.Alander. University of Vaasa and the Finnish Artificial Intelligence Society, Vaasa, 1996.
- There are several stochastic methods that can be used for solving NP-hard optimization problems approximatively. Examples of such algorithms include (in order of increasing computational complexity) stochastic greedy search methods, simulated annealing, and genetic algorithms. We investigate which of these methods is likely to give best performance in practice, with respect to the computational effort each requires. We study this problem empirically by selecting a set of stochastic algorithms with varying computational complexity, and by experimentally evaluating for each method how the goodness of the results achieved improves with increasing computational time. For the evaluation, we use a graph optimization problem, which is closely related to several real-world practical problems. To get a wider perspective of the goodness of the achieved results, the stochastic methods are also compared against special-case greedy heuristics. This investigation suggests that although genetic algorithms can provide good results, simpler stochastic algorithms can achieve similar performance more quickly.
P.Kontkanen, P.Myllymäki and H.Tirri, Predictive data mining with finite mixtures. Pp. 176-182 in Proceedings of The Second International Conference on Knowledge Discovery and Data Mining (Portland, OR, August 1996).
- In data mining the goal is to develop methods for discovering previously unknown regularities from databases. The resulting models are interpreted and evaluated by domain experts, but some model evaluation criterion is needed also for the model construction process. The optimal choice would be to use the same criterion as the human experts, but this is usually impossible as the experts are not capable of expressing their evaluation criteria formally. On the other hand, it seems reasonable to assume that any model possessing the capability of making good predictions also captures some structure of the reality. For this reason, in predictive data mining the search for good models is guided by the expected predictive error of the models. In this paper we describe the Bayesian approach to predictive data mining in the finite mixture modeling framework. The finite mixture model family is a natural choice for domains where the data exhibits a clustering structure. In many real world domains this seems to be the case, as is demonstrated by our experimental results on a set of public domain databases.
P.Kontkanen, P.Myllymäki and H.Tirri, Comparing Bayesian model class selection criteria by discrete finite mixtures. Pp. 364-374 in Information, Statistics and Induction in Science(Proceedings of the ISIS'96 Conference in Melbourne, Australia, August 1996), edited by D.L.Dowe, K.B.Korb, and J.J.Oliver. World Scientific , Singapore 1996.
- We investigate the problem of computing the posterior probability of a model class, given a data sample and a prior distribution for possible parameter settings. By a model class we mean a group of models which all share the same parametric form. In general this posterior may be very hard to compute for high-dimensional parameter spaces, which is usually the case with real-world applications. In the literature several methods for computing the posterior approximatively have been proposed, but the goodness of the approximations may depend heavily on the size of the available data sample. In this work, we are interested in testing, how well the approximative methods perform in real-world problem domains. In order to conduct such a study, we have chosen the model family of finite mixture distributions. With certain assumptions, we are able to derive the model class posterior analytically for this model family. We report a series of model class selection experiments on real-world data sets, where the true posterior and the approximations are compared. The empirical results support the hypothesis that the approximative techniques can provide good estimates of the true posterior, especially when the sample size grows large.
P.Kontkanen, P.Myllymäki and H.Tirri, Constructing Bayesian finite mixture models by the EM algorithm. Report C-1996-9, University of Helsinki, Department of Computer Science.
- In this paper we explore the use of finite mixture models for building decision support systems capable of sound probabilistic inference. Finite mixture models have many appealing properties: they are computationally efficient in the prediction (reasoning) phase, they are universal in the sense that they can approximate any problem domain distribution, and they can handle multimodality well. We present a formulation of the model construction problem in the Bayesian framework for finite mixture models, and describe how Bayesian inference is performed given such a model. The model construction problem can be seen as missing data estimation and we describe a realization of the Expectation-Maximization (EM) algorithm for finding good models. To prove the feasibility of our approach, we report crossvalidated empirical results on several publicly available classification problem datasets, and compare our results to corresponding results obtained by alternative techniques, such as neural networks and decision trees. The comparison is based on the best results reported in the literature on the datasets in question. It appears that using the theoretically sound Bayesian framework suggested here the other reported results can be outperformed with a relatively small effort.
H.Tirri, P.Kontkanen and P.Myllymäki, Probabilistic Instance-Based Learning. Pp. 507-515 in Machine Learning: Proceedings of the Thirteenth International Conference, edited by L. Saitta. Morgan Kaufmann Publishers, San Francisco, CA, 1996.
- Traditional instance-based learning methods base their predictions directly on (training) data that has been stored in the memory. The predictions are based on weighting the contributions of the individual stored instances by a distance function implementing a domain-dependent similarity metrics. This basic approach suffers from three drawbacks: computationally expensive prediction when the database grows large, overfitting in the presence of noisy data, and sensitivity to the selection of a proper distance function. We address all these issues by giving a probabilistic interpretation to instance-based learning, where the goal is to approximate predictive distributions of the attributes of interest. In this probabilistic view the instances are not individual data items but probability distributions, and we perform Bayesian inference with a mixture of such prototype distributions. We demonstrate the feasibility of the method empirically for a wide variety of public domain classification data sets.
P.Kontkanen, P.Myllymäki and H.Tirri, Some experimental results with finite mixture models (extended abstract). Proceedings of the First European Conference on Highly Structured Stochastic Systems (Rebild, Denmark, May 1996).

1995

P. Myllymäki, Mapping Bayesian Networks to Stochastic Neural Networks: A Foundation for Hybrid Bayesian-Neural Systems. Ph.D. Dissertation, Report A-1995-1, Department of Computer Science, University of Helsinki, December 1995.
- In this work, we are interested in the problem of finding maximum a posteriori probability (MAP) value assignments for a set of discrete attributes, given the constraint that some of the attributes are permanently fixed to some values a priori. For building a system capable of this type of uncertain reasoning in practice, we need first to construct an accurate abstract representation of the problem domain, and then to establish an efficient search mechanism for finding MAP configurations within the constructed model. We propose a hybrid Bayesian network-neural network system for solving these two subtasks. The Bayesian network component can be used for constructing a compact, high-level representation for the problem domain probability distribution quickly and reliably, assuming that suitable expert knowledge is available. The neural network component provides then a computationally efficient, massively parallel platform for searching the model state space. The main application areas for these kinds of systems include configuration and design problems, medical diagnosing and pattern recognition.
  For implementing a hybrid Bayesian-neural system as suggested above, we present here methods for mapping a given Bayesian network to a stochastic neural network architecture, in the sense that the resulting neural network updating process provably converges to a state which can be projected to a MAP state on the probability distribution corresponding to the original Bayesian network. From the neural network point of view, these mappings can be seen as a method for incorporating high-level, probabilistic a priori information directly into neural networks, without recourse to a time-consuming and unreliable learning process. From the Bayesian network point of view, the mappings offer a massively parallel implementation of simulated annealing where all the variables can be updated at the same time. Our empirical simulations suggest that this type of massively parallel simulated annealing outperforms the traditional sequential Gibbs sampling/simulated annealing process, provided that suitable hardware is available.
H.Tirri and S.Mallenius, Optimizing the Hard Address Distribution for Sparse Distributed Memories. Pp. 1966-1970 in Proceedings of the IEEE International Conference on Neural Networks (Perth, November 1995).
X.M.Song, H.Tirri, O.Aaltonen and A.Hase, A Discrete Radial Basis Function Network for Empirical Modeling of Soil Extraction Process. Pp. 17-20 in Proceedings of the International Conference on Engineering Applications of Neural Networks (EANN'95), 1995.
- A RBF network was used as an empirical modeling tool. Results on simulated processes show that such a network can learn the shape of the function reasonably well with very limited experimental data. The use of the RBF network model is also illustrated with real experimental data from a soil extraction process.
P. Myllymäki, Mapping Bayesian Networks to Boltzmann Machines . Pp. 269-280 in Proceedings of Applied Decision Technologies 1995 (London, April 1995).
- We study the task of finding a maximal a posteriori (MAP) instantiation of Bayesian network variables, given a partial value assignment as an initial constraint. This problem is known to be NP-hard, so we concentrate on a stochastic approximation algorithm, simulated annealing. This stochastic algorithm can be realized as a sequential process on the set of Bayesian network variables, where only one variable is allowed to change at a time. Consequently, the method can become impractically slow as the number of variables increases. We present a method for mapping a given Bayesian network to a massively parallel Bolztmann machine neural network architecture, in the sense that instead of using the normal sequential simulated annealing algorithm, we can use a massively parallel stochastic process on the Boltzmann machine architecture. The neural network updating process provably converges to a state which solves a given MAP task.
H. Tirri, Replacing the Pattern Matching of an Expert System with a Neural Network. Pp. 47-62 in Intelligent Hybrid Systems, edited by S. Goonatilake and S. Khebbal. John Wiley & Sons, Chichester 1995.
P. Myllymäki and H. Tirri, Constructing computationally efficient Bayesian models via unsupervised clustering. Pp. 237-248 in Probabilistic Reasoning and Bayesian Belief Networks, edited by A.Gammerman. Alfred Waller Publishers, Suffolk 1995.
- Given a set of samples of an unknown probability distribution, we study the problem of constructing a good approximative Bayesian network model of the probability distribution in question. This task can be viewed as a search problem, where the goal is to find a maximal probability network model, given the data. In this work, we do not make an attempt to learn arbitrarily complex multi-connected Bayesian network structures, since such resulting models can be unsuitable for practical purposes due to the exponential amount of time required for the reasoning task. Instead, we restrict ourselves to a special class of simple tree-structured Bayesian networks called Bayesian prototype trees, for which a polynomial time algorithm for Bayesian reasoning exists. We show how the probability of a given Bayesian prototype tree model can be evaluated, given the data, and how this evaluation criterion can be used in a stochastic simulated annealing algorithm for searching the model space. The simulated annealing algorithm provably finds the maximal probability model, provided that a sufficient amount of time is used.

1994

P. Orponen, Computational complexity of neural networks: A survey. Nordic Journal of Computing 1 (1994), 94-110.
- We survey some of the central results in the complexity theory of discrete neural networks, with pointers to the literature. Our main emphasis is on the computational power of various acyclic and cyclic network models, but we also discuss briefly the complexity aspects of synthesizing networks from examples of their behavior.
P. Floréen, J. N. Kok, M. N. Rasch, Three methods for tracing a simple genetic algorithm. Pp.148-158 in Proceedings of the 4th Belgian-Dutch Conference on Machine Learning (Benelearn 94, Rotterdam, June 1994), edited by J. C. Bioch and S. H. Nienhuys-Cheng. Erasmus University Rotterdam Report EUR-CS-94-05, Rotterdam 1994.
P. Floréen and J.N.Kok, Tracing the moments of distributions in genetic algorithms. Pp.51-60 in Proceedings of the Second Finnish Workshop on Genetic Algorithms and their Applications (Vaasa, Finland, March 1994), edited by J.T. Alander. Report 94-2, University of Vaasa, 1994.
P. Myllymäki and H. Tirri, Learning Bayesian prototype trees by simulated annealing. Report C-1994-24, Department of Computer Science, University of Helsinki, 1994. Also: Pp.32-37 in Proceedings of the Conference on Artificial Intelligence Research in Finland (Turku, Finland, August 1994), edited by C.Carlsson, T.Järvi and T.Reponen. Finnish Artificial Intelligence Society, Helsinki 1994.
- Given a set of samples of an unknown probability distribution, we study the problem of constructing a good approximative Bayesian network model of the probability distribution in question. This task can be viewed as a search problem, where the goal is to find a maximal probability network model, given the data. In this work, we do not make an attempt to learn arbitrarily complex multi-connected Bayesian network structures, since such resulting models can be unsuitable for practical purposes due to the exponential amount of time required for the reasoning task. Instead, we restrict ourselves to a special class of simple tree-structured Bayesian networks called Bayesian prototype trees, for which a polynomial time algorithm for Bayesian reasoning exists. We show how the probability of a given Bayesian prototype tree model can be evaluated, given the data, and how this evaluation criterion can be used in a stochastic simulated annealing algorithm for searching the model space. The simulated annealing algorithm provably finds the maximal probability model, provided that a sufficient amount of time is used.
P. Myllymäki and H. Tirri, Massively parallel case-based reasoning with probabilistic similarity metrics. Pp.144-154 in Topics in Case-Based Reasoning, edited by S.Wess, K.-D.Althoff and M.Richter. Lecture Notes in Artificial Intelligence, Volume 837. Springer Verlag, 1994.
- We propose a probabilistic case-space metric for the case matching and case adaptation tasks. Central to our approach is a probability propagation algorithm adopted from Bayesian reasoning systems, which allows our case-based reasoning system to perform theoretically sound probabilistic reasoning. The same probability propagation mechanism actually offers an uniform solution to both the case matching and case adaptation problems. We also show how the algorithm can be implemented as a connectionist network, where efficient massively parallel case retrieval is an inherent property of the system. We argue that using this kind of an approach, the difficult problem of case indexing can be completely avoided.
P. Floréen and P. Orponen, Complexity issues in discrete Hopfield networks. Manuscript, 52 pp., to appear in The Computational and Learning Complexity of Neural Networks: Advanced Topics, edited by Ian Parberry. The MIT Press, Cambridge, MA, 1994.
- We survey some aspects of the computational complexity theory of discrete-time and discrete-state Hopfield networks. The emphasis is on topics that are not adequately covered by the existing survey literature, most significantly:
  1. the known upper and lower bounds for the convergence times of Hopfield nets (here we consider mainly worst-case results);
  2. the power of Hopfield nets as general computing devices (as opposed to their applications to associative memory and optimization);
  3. the complexity of the synthesis (``learning'') and analysis problems related to Hopfield nets as associative memories.
P. Floréen and T. Huuskonen, Uniqueness of maximum values in discrete distributions. Journal of Applied Probability 31 (September 1994).
H. Tirri and P. Myllymäki, MDL learning of probabilistic neural networks for discrete problem domains. Pp.1493-1497 in Proceedings of the IEEE World Congress on Computational Intelligence (Orlando, June 1994).
- Given a problem, a case-based reasoning (CBR) system will search its case memory and use the stored cases to find the solution, possibly modifying retrieved cases to adapt to the required input specifications. In discrete domains CBR reasoning can be based on a rigorous Bayesian probability propagation algorithm. Such a Bayesian CBR system can be implemented as a probabilistic feedforward neural network with one of the layers representing the network with one of the layers representing the cases. In this paper we introduce a Minimum Description Length (MDL) based learning algorithm to obtain the proper network structure with the associated conditional probabilities. This algorithm together with the resulting neural network implementation provide a massively parallel architecture for solving the efficiency bottleneck in case-based reasoning.
P. Myllymäki and H. Tirri, Learning in neural networks with Bayesian prototypes. Pp.60-64 in Proceedings of SOUTHCON'94 (Orlando, March 1994).
- Given a set of samples of a probability distribution on a set of discrete random variables, we study the problem of constructing a good approximative neural network model of the underlying probability distribution. Our approach is based on an unsupervised learning scheme where the samples are first divided into separate clusters, and each cluster is then coded as a single vector. These Bayesian prototype vectors consist of conditional probabilities representing the attribute-value distribution inside the corresponding cluster. Using these prototype vectors, it is possible to model the underlying joint probability distribution as a simple Bayesian network (a tree), which can be realized as a feedforward neural network capable of probabilistic reasoning. In this framework, learning means choosing the size of the prototype set, partitioning the samples into the corresponding clusters, and constructing the cluster prototypes. We describe how the prototypes can be determined, given a partition of the samples, and present a method for evaluating the likelihood of the corresponding Bayesian tree. We also present a greedy heuristic for searching through the space of different partition schemes with different numbers of clusters, aiming at an optimal approximation of the probability distribution.
P. Myllymäki, Using Bayesian networks for incorporating probabilistic a priori knowledge into Boltzmann machines. Pp.97-102 in Proceedings of SOUTHCON'94 (Orlando, March 1994).
- We present a method for automatically determining the structure and the connection weights of a Boltzmann machine corresponding to a given Bayesian network representation of a probability distribution on a set of discrete variables. The resulting Boltzmann machine structure can be implemented efficiently on massively parallel hardware, since the structure can be divided into two separate clusters where all the nodes in one cluster can be updated simultaneously. The updating process of the Boltzmann machine approximates a Gibbs sampling process of the original Bayesian network in the sense that the Boltzmann machine converges to the same final state as the Gibbs sampler does. The mapping from a Bayesian network to a Boltzmann machine can be seen as a method for incorporating probabilistic a priori information into a neural network architecture, which can then be trained further with existing learning algorithms.
S. Santos, Phase transitions in sparsely connected Boltzmann machines. Report C-1994-15, Department of Computer Science, University of Helsinki, 1994.
- The primary motivation of this work has been the study of phase transitions in the simulated annealing schedules of Boltzmann machines. At first, we introduce and focus on the harmonium Boltzmann machine, which can be used to implement a Bayesian reasoning scheme. These sparsely connected Boltzmann machines are composed of two connected clusters of independent nodes, allowing a massive parallel updating scheme. The study of critical phenomena in simulated annealing presupposes a deeper understanding of Monte Carlo methods, especially in connection with statistical physics. We discuss various aspects of the influence of the correlation between the elements of Monte Carlo samples on the determination of the relevant physical quantities. The close analogy between Boltzmann machines and spin glass models is brought to the fore in our investigations, by computational simulations, of the thermodynamical properties of the harmonium Boltzmann machine. The study of the specific character- istics of this model contributes to a closer comprehension of the emergence of critical phenomena, corroborated by the results of our simulations. For the larger framework of sparsely connected Boltzmann machines, we extensively present the theoretical methods of statistical physics applied to strongly diluted spin glasses. Finally, the predicted equilibrium properties are compared with the empirical results.

1993

P. Myllymäki, Bayesian reasoning by stochastic neural networks. Ph.Lic. Thesis, Report C-1993-67, Department of Computer Science, University of Helsinki, 1993.
P. Floréen, A short introduction to neural associative memories. Bulletin of the European Association for Theoretical Computer Science 51 (October 1993), 236-245.
P. Orponen, On the computational power of discrete Hopfield nets. Pp.215-226 in Proc. 20th Internat. Colloq. on Automata, Languages, and Programming (Lund, Sweden, July 1993). Lecture Notes in Computer Science 700, Springer-Verlag, Berlin Heidelberg, 1993.
- We prove that polynomial size discrete synchronous Hopfield networks with hidden units compute exactly the class of Boolean functions PSPACE/poly, i.e., the same functions as are computed by polynomial space-bounded nonuniform Turing machines. As a corollary to the construction, we observe also that networks with polynomially bounded interconnection weights compute exactly the class of functions P/poly.
P. Myllymäki and H. Tirri, Bayesian case-based reasoning with neural networks. Pp.422-427 in Proceedings of the IEEE International Conference on Neural Networks (San Francisco, March 1993).
- Given a problem, a case-based reasoning (CBR) system will search its case memory and use the stored cases to find the solution, possibly modifying retrieved cases to adapt to the required input specifications. In this paper we introduce a neural network architecture for efficient case-based reasoning. We show how a rigorous Bayesian probability propagation algorithm can be implemented as a feedforward neural network and adapted for CBR. In our approach the efficient indexing problem of CBR is naturally implemented by the parallel architecture, and heuristic matching is replaced by a probability metric. This allows our CBR to perform theoretically sound Bayesian reasoning. We also show how the probability propagation actually offers a solution to the adaptation problem in a very natural way.
P. Floréen and P. Orponen, Attraction radii in binary Hopfield nets are hard to compute. Neural Computation 5 (1993), 812-821.

1992

P. Floréen, Computational complexity problems in neural associative memories. Ph. D. Thesis, Report A-1992-5, Department of Computer Science, University of Helsinki, 1992.
P. Floréen, Neuraaliset assosiatiivimuistit. Tietojenkäsittelytiede 3 (1992), 44-47.
P. Floréen, P. Myllymäki, P. Orponen, and H. Tirri, Neula: A hybrid neural-symbolic expert system shell. Tietojenkäsittelytiede 3 (1992), 11-18.
P. Floréen, A new associative memory model. International Journal of Intelligent Systems 7 (1992), 455-467.
P. Orponen, Neural networks and complexity theory (invited talk). Pp.50-61 in Proc. of the 17th Internat. Symp. on Mathematical Foundations of Computer Science (Prague, Czechoslovakia, August 1992). Lecture Notes in Computer Science 629, Springer-Verlag, Berlin Heidelberg, 1992.
P. Myllymäki, P. Orponen, and T. Silander, Integrating symbolic reasoning with neurally represented background knowledge. Pp.231-240 in Proceedings of the Finnish AI Conference (Espoo, Finland, June 1992), edited by E. Hyvönen, J. Seppänen and M. Syrjänen. Finnish AI Society, Helsinki, 1992.
- Current expert systems cannot properly handle imprecise and incomplete information. On the other hand, neural networks can perform pattern recognition operations even in noisy environments. Against this background, we have implemented a neural expert system shell NEULA, whose computational mechanism processes imprecisely or incompletely given information by means of approximate probabilistic reasoning.

1991

P. Myllymäki and P. Orponen, Programming the harmonium. Pp.671-677 in Proceedings of the International Joint Conference on Neural Networks (Singapore, November 1991).
- We show how to synthesize, given a Bayesian network description of a probability distribution, an instance of Smolensky's harmony network that computes maximum-likelihood completions to partial value assignments, according to the given distribution. As an application, we present a scheme for translating a high-level description of a conceptual hierarchy, with default values and exceptions, to a harmony network representation, which is then inherently capable of quite complicated query processing. Finally, we discuss our experience from implementing such a translation scheme.
P. Floréen, Worst-case convergence times for Hopfield memories. IEEE Transactions on Neural Networks 2 (1991), 533-535.
P. Floréen, The convergence of Hamming memory networks. IEEE Transactions on Neural Networks, 2 (1991),449-457.
H. Tirri, Concept randomness and neural networks. Pp.1367-1370 in Proceedings of the International Conference on Artificial Neural Networks (Espoo, Finland, June 1991), edited by T. Kohonen, K. Mäkisara, O. Simula, and J. Kangas. Elsevier Science Publishers, Amsterdam 1991.

1990

H. Tirri, Implementing Expert System Rule Conditions by Neural Networks. New Generation Computing 10 (1991), 55-71. Also: TR-1050, Department of Computer Sciences, Purdue University, 1990.
P. Floréen, P. Myllymäki, P. Orponen, and H. Tirri, Compiling object declarations into connectionist networks. AI Communications 3 (1990) 4 (December), 172-183.
P. Floréen, Computational complexity issues in neural associative memories. Ph.Lic Thesis, Report C-1990-40, Department of Computer Science, University of Helsinki, 1990.
R.J.T. Morris, L.D. Rubin, and H. Tirri, Neural network techniques for object orientation detection: Solution by optimal feedforward network and learning vector quantization approaches. IEEE Transactions on Pattern Analysis and Machine Intelligence 12 (1990) 11 (November), 1107-1115.
P. Orponen, P. Floréen, P. Myllymäki, and H. Tirri, A neural implementation of conceptual hierarchies with Bayesian reasoning. Pp. 297-303 in Proceedings of the International Joint Conference on Neural Networks (San Diego, CA, June 1990). IEEE, New York, 1990.
- We present a scheme for translating high-level descriptions of conceptual hierarchies into a neural network representation. The intuitive semantics of a conceptual hierarchy is provided by a Bayesian net, and the neural network implementation provably approximates the behaviour of this net under a stochastic simulation rule.
P. Floréen, An analysis of the convergence time of Hamming memory networks. Pp.867-872 in Proceedings of the International Joint Conference on Neural Networks (San Diego, CA, June 1990). IEEE, New York, 1990.
P. Orponen, P. Floréen, P. Myllymäki, and H. Tirri, A neural implementation of Bayesian reasoning. Pp.277-287 in Proceedings of the Finnish Artificial Intelligence Symposium (Oulu, Finland, June 1990), edited by P. Salonen M. Djupsund and M. Syrjänen. Finnish Artificial Intelligence Society, 1990.
- We present a knowledge representation scheme where an object hierarchy is interpreted as defining a particular probability distribution over a corresponding set of object variables and their possible attribute values. This probability distribution is represented as a Bayesian network. We will describe the translation scheme from high-level descriptions of object hierarchies into Byesian networks and the natural realization of the resulting Bayesian network as a neural net. The neural network implementation provably approximates the behaviour of the corresponding Bayesian net under a stochastic simulation rule.
P. Myllymäki, H. Tirri, P. Floréen, and P. Orponen, Compiling high-level specifications into neural networks. PP. 475-478 in Proceedings of the International Joint Conference on Neural Networks (Washington D.C., January 1990). IEEE, New York, 1990.

1989

P. Floréen and P. Orponen, On the computational complexity of analyzing Hopfield nets. Complex Systems 3 (1989), 577-587.
P. Floréen, P. Myllymäki, P. Orponen, and H. Tirri, Neural representation of concepts for robust inference. Pp.89-98 in Proceedings of the International Symposium Computational Intelligence II (Milano, Italy, September 1989), edited by F. Gardin and G. Mauri. Elsevier Science Publishers, Amsterdam, 1989.
H. Tirri, Applying neural computing to expert system design. Part I: Coping with complex sensory data and attribute selection. Pp. 474-488 in Proceedings of the 3rd International Conference on Data Organization and Algorithms (Paris, France, June 1989) edited by W. Litwin and H.-J. Schek. Springer-Verlag, Berlin Heidenberg, 1989.
H. Tirri, Neural information processing and applications. Tutorial at the 3rd International Conference on Data Organization and Algorithms (Paris, France, June 1989). Institut National de Recherche en Informatique et en Automatique (INRIA), Le Chesnay Cedex, France, 1989.
P. Orponen, An experimental evaluation of the optimal capacity of the Hopfield associative memory. Page 612 in Proceedings of the International Joint Conference on Neural Networks (Washington, D.C., June 1989). IEEE, New York, 1989.
P. Floréen and P. Orponen, Counting stable states and sizes of attraction domains in Hopfield nets is hard. Pp.395-399 in Proceedings of the International Joint Conference on Neural Networks (Washington, D.C, June 1989). IEEE, New York, 1989.
R.J.T. Morris, L.D. Rubin, and H. Tirri, A comparison of feedforward and self-organizing approaches to the font orientation problem. Pp.291-298 in Proceedings of the International Joint Conference on Neural Networks (Washington, D.C., June 1989). IEEE, New York, 1989.
H. Tirri, Feedforward and learning vector quantization approaches for font orientation detection. In Nordic Symposium on Neural Computing (Hanasaari, Finland, April 1989).