158 Pages
English

Advanced probabilistic network modeling framework with qualitative prior knowledge [Elektronische Ressource] / Rui Chang

Gain access to the library to view online
Learn more

Description

Institut fu¨r Informatikder Technischen Universit¨at Mu¨nchenAdvanced Probabilistic NetworkModeling Framework with QualitativePrior KnowledgeRui ChangVollsta¨ndiger Abdruck der von der Fakult¨at fur¨ Informatik der TechnischenUniversit¨at Munc¨ hen zur Erlangung des Akademischen Grades einesDoktors der Naturwissenschaften (Dr. rer. nat.)genehmigten Dissertation.Vorsitzender: Univ.-Prof. Dr. H. J. SchmidhuberPruf¨ er der Dissertation:1. Univ.-Prof. Dr. Dr.h.c.mult. W. Brauer, em.2. Univ.-Prof. B. Brug¨ ge, Ph.D.DieDissertation wurdeam 07.01.2008 beiderFakult¨at der TechnischenUniver-sit¨at Munc¨ hen eingereicht und durch die Fakult¨at fur¨ Informatik am 28.01.2008angenommen.To my parentsAbstractThe ever increasing amount of information in every scientific and industrialdomain have been an exciting challenge for computer scientist to handle vastamountofdataandtorepresenthumanunderstandingsofadomaininasystem-aticandmathematicway. Overdecades,probabilisticmodelingwithprobabilitytheory and statistical learning algorithms has been popular for accomplishingthis task due to the stochastic characteristics of the nature. Quantitative mea-surements are generated from various kinds of ”sensors” in all types of scienceand industry and we need to make sense of these data, i.e. to extract im-portant patterns and trends, and understand ”what the data says”. This isoften called learning from data, reverse-engineering or bottom-up modeling.

Subjects

Informations

Published by
Published 01 January 2008
Reads 12
Language English
Document size 9 MB

Institut fu¨r Informatik
der Technischen Universit¨at Mu¨nchen
Advanced Probabilistic Network
Modeling Framework with Qualitative
Prior Knowledge
Rui Chang
Vollsta¨ndiger Abdruck der von der Fakult¨at fur¨ Informatik der Technischen
Universit¨at Munc¨ hen zur Erlangung des Akademischen Grades eines
Doktors der Naturwissenschaften (Dr. rer. nat.)
genehmigten Dissertation.
Vorsitzender: Univ.-Prof. Dr. H. J. Schmidhuber
Pruf¨ er der Dissertation:
1. Univ.-Prof. Dr. Dr.h.c.mult. W. Brauer, em.
2. Univ.-Prof. B. Brug¨ ge, Ph.D.
DieDissertation wurdeam 07.01.2008 beiderFakult¨at der TechnischenUniver-
sita¨t Munc¨ hen eingereicht und durch die Fakult¨at fur¨ Informatik am 28.01.2008
angenommen.To my parentsAbstract
The ever increasing amount of information in every scientific and industrial
domain have been an exciting challenge for computer scientist to handle vast
amountofdataandtorepresenthumanunderstandingsofadomaininasystem-
aticandmathematicway. Overdecades,probabilisticmodelingwithprobability
theory and statistical learning algorithms has been popular for accomplishing
this task due to the stochastic characteristics of the nature. Quantitative mea-
surements are generated from various kinds of ”sensors” in all types of science
and industry and we need to make sense of these data, i.e. to extract im-
portant patterns and trends, and understand ”what the data says”. This is
often called learning from data, reverse-engineering or bottom-up modeling.
Among these learning algorithms, Bayesian network computational framework
has become particular popular due to the ability of Bayesian network to model
cause-effect interactions between the variables in a domain. For example, in
bioinformatics, vast amount of ”-omics” data are generated by high-throughput
screening techniques. Learning method with Bayesian networks has been used
to construct gene regulatory networks from transcriptomic data and to predict
protein-protein interactions based on proteomic data.
In practice, the data basis in reverse-engineering approach can be very
sparse. Therefore, it is hardly sufficient to select one adequate model, i.e. there
isconsiderablemodeluncertainty. SelectingonesingleBayesianmodelcanthen
lead to strongly biased inference results. In this case, full Bayesian approach
with model averaging can be used to alleviate the bias. In this approach, one
majordifficultyistospecifypriordistributionfunctionontheBayesiannetwork
structure space and parameter space in order to compute a posterior probabil-
ity. One important information resources that could provide solutions to this
problem is qualitative prior distribution which largely exists in every science
and industry domain. In addition, human have a deep intuition that causal-
ity is a central and cohesive aspect of their perceptions, therefore, one subtype
of these qualitative prior knowledge, i.e. qualitative causal knowledge which
describes the cause-effect relations between multiple entities with any form of
uncertainties, are particularly well-suited to represent human understandings
and to get approximated characterizations of the behavior of the interested do-
main. For example, in a qualitative causal statement: ”smoking increases the
risk of lung cancer”, two entities: smoking and lung cancer are related to each
other. Moreover, smoking positively influences lung cancer since lung cancer
risk is increased in case of smoking. It is therefore desirable to make use of this
body of evidence in probabilistic modeling with Bayesian network.
This thesis is concerned with developing a powerful probabilistic modeling
framework to represent human understandings of a domain based on qualita-
2tive prior knowledge. More precisely, to construct a Bayesian network structure
with cause-effect relationships between the entities in a domain and parame-
terize these interactions according to the semantics of qualitative knowledge.
One problem here is that qualitative knowledge provides no quantitative infor-
mation to parameterize edges in Bayesian network and parameters need to be
configured based on soley qualitative information. We attack this problem by
proposing a qualitative knowledge model which is responsible for constructing
mathematical constraints to define parameter distribution based on the quali-
tative knowledge. This approach incorporates the concept of model uncertainty
due to the qualitative nature of the statements and automatically select a class
of possible Bayesian models which are consistent with the semantics of the
statements. Quantitative Bayesian network inference is performed by averaging
inferences of each Bayesian network in this class with full Bayesian approach.
However,knowledgeiswell-knowntobeinconsistentandincomplete. Knowl-
edgehasspatialandtemporalpropertieslikeotherphysicalsystems, i.e. knowl-
edge exist in space-time dimension. The spatial property describes that knowl-
edge represents information on a specific sub-structure of a domain and the
temporal property states that knowledge represents human understandings at
a particular time point. Thus, these knowledge are incomplete and may be
updated by complementary discovery. Moreover, another significant drawback
of knowledge is inconsistency. In the same domain, there may exist contra-
dicting qualitative statements on dependency, causality and parameters over a
set of entities. In this thesis, we propose several successful methods to deal
with knowledge incompleteness and inconsistency, and integrate the Bayesian
networks based on the set of knowledge to form an complete and coherent rep-
resentation of the underlying system.Acknowledgements
This dissertation is developed based on my work within a corporative Ph.D
program between the Informatics Institute of Technical University of Munich
and the Learning System Department (CT IC4) of Siemens AG. In the past
3 years, it has been my extreme pleasure and fortune to work within such
a friendly and professional atmosphere provided by both parties of my Ph.D
program. This thesis would not have been possible without the kindness and
generosity of all the members of our group. I feel grateful and indebted to have
received their helps and suggestions.
First of all, I would like to thank my academic supervisor, Prof. Wilfried
Brauer from the Institute of Informatics of Technical University of Munich who
has been constantly supporting me on my researches by providing me his re-
markable insights and constructive suggestions. Prof. Wilfried Brauer is such
kind person so that whenever I need his help, he is always there for me. I am
verygratefultohiskindnessandIamdeeplyimpressedbyhisopen-mindedness
and astonishing academic achievements.
Secondly, I would like to thank Siemens AG, especially Siemens Corporate
Technology, for endowing me such a great opportunity to carry out the cutting-
edge researches in machine learning and Bioinformatics field and for providing
metheSiemensDoctoralscholarship. IfeelgratefultoProf. BerndSchur¨ mann,
theleaderoftheLearningSystemsDepartmentatSiemensAG,forhisconstant
support to my research. I appreciate his emphasis on both scientific research
and real-world applications.
Specially, I am indebted to my co-supervisor at Siemens AG, Dr. Martin
Stetter, the principle investigator of my team, who has brought me into the
world of statistical learning and Bioinformatics. He has the greatest influence
in my intellectual, knowledge development and my commitment to my future
career. I am deeply impressed by his enthusiasm, intelligence, open-mindedness
whichconstantlyencouragesmetoovercomedifficultiesandtopursuesuccesses
in my research and make my time at Siemens AG memorable.
I appreciate the friendship and fellowship to my colleagues, Dr. Math¨aus
Dejori, who often enlighten me with his smart ideas. His solid knowledge in
machine learning and sharp way of thinking often bring us surprises. Also, I
enjoyed the scientific discussions with Andreas Na¨gele who often provide me
useful advices from another point of view. His logic way of thinking impressed
me a lot. Meanwhile, I would like to thank Holger Arndt who answered my
questions on programming and computer hardware. Specially, I would like to
thank Dr. Jakub Pijewski with whom I enjoyed to talk about biology with his
talented minds. I would like to thank to the secretary Mrs. Christina Singer at
Siemens AG and Mrs. Erika Leber from Technical University of Munich, who
4kindly helped me to deal with administration documents. Moreover, I would
liketothanktoDr. VolkeTresp,Dr. KaiYu, Dr. ShipengYu, Dr. XuZhao, Yi
HuangandHuaienGao, whooftenanswermyquestionsandgivepleasantchats
which make my research joyful. Finally, I would like to thank to my parents
and Ms. Wenbo He who constantly give me the most love, support and care in
my personal life without which I could not have ever accomplished my Ph.D.Contents
1 Introduction 2
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Overview of Data-driven Bayesian Modeling Approach . . . . . . 9
1.2.1 Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . 9
1.2.2 Bayesian Network Inference . . . . . . . . . . . . . . . . . 11
1.2.3 Bayesian Network Learning . . . . . . . . . . . . . . . . . 13
1.2.4 Bayesian Model Averaging. . . . . . . . . . . . . . . . . . 19
1.3 Overview of Knowledge-based Bayesian Modeling Approach . . . 21
1.3.1 Qualitative Probabilistic Network. . . . . . . . . . . . . . 22
1.4 Summary and Outline . . . . . . . . . . . . . . . . . . . . . . . . 25
2 Bayesian Modeling with Consistent Qualitative Knowledge 27
2.1 Qualitative Knowledge Model . . . . . . . . . . . . . . . . . . . . 28
2.2 Comparison of Qualitative Knowledge Models . . . . . . . . . . . 34
2.2.1 Compare to Qualitative Probabilistic Network (QPN) . . 35
2.2.2 Compare to Probabilistic Commonsense Reasoner . . . . 36
2.3 Bayesian Modeling based on Consistent Qualitative Knowledge . 37
2.3.1 Modeling with Static Bayesian Networks . . . . . . . . . . 38
2.3.2 Modeling with Dynamic Bayesian Networks . . . . . . . . 41
2.3.3 ASIA Network . . . . . . . . . . . . . . . . . . . . . . . . 44
2.4 Performance Analysis . . . . . . . . . . . . . . . . . . . . . . . . 50
2.4.1 Approximating Expected Inference E[P(X|E,m)] . . . . . 50
2.4.2 Robust Analysis . . . . . . . . . . . . . . . . . . . . . . . 62
2.5 Empirical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.5.1 TGFβ-mediated Breast Cancer Bone Metastasis Network 74
2.5.2 TGFβ-mediatedMammaryEpithelialCellCytostasisPro-
gram and Breast Cancer . . . . . . . . . . . . . . . . . . . 80
2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3 Bayesian Modeling with Inconsistent Qualitative Knowledge 91
3.1 Hierarchical Knowledge Feature Model . . . . . . . . . . . . . . . 91
3.1.1 Inconsistent Knowledge Integration . . . . . . . . . . . . . 93
3.2 Bayesian Inference based on Inconsistent Knowledge . . . . . . . 94
3.2.1 Modeling with Static Bayesian Networks . . . . . . . . . . 94
3.2.2 Modeling with Dynamic Bayesian Networks . . . . . . . . 96
3.2.3 ASIA Network . . . . . . . . . . . . . . . . . . . . . . . . 97
3.3 Empirical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.3.1 Smad7 in TGFβ-Smad Pathway . . . . . . . . . . . . . . 101
ii3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4 Bayesian Modeling with Incomplete Qualitative Knowledge 106
4.1 Incomplete Qualitative Knowledge and Bayesian Network Fusion 106
4.1.1 Incomplete Qualitative Knowledge . . . . . . . . . . . . . 107
4.2 Bayesian Inference with Incomplete Qualitative Knowledge . . . 107
4.2.1 Bayesian Network Fusion . . . . . . . . . . . . . . . . . . 107
4.2.2 Bayesian Network Fusion in Parameter Space . . . . . . . 108
4.2.3 Bayesian Network Fusion in Structure Space . . . . . . . 112
4.3 Empirical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.3.1 IntegrationofTGFβ-SmadpathwayintoSmad-dependent
Breast Cancer Bone Metastasis Network . . . . . . . . . . 116
4.3.2 Integration of TGFβ-PTHrP Pathway in Breast Cancer
Bone Metastasis Network . . . . . . . . . . . . . . . . . . 124
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5 Discussion and Future Research 131
5.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.3 Future Researches . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Appendices 135
A Bayesian Dirichlet Equivalent Score 135
B Textual Statements on Breast Cancer Bone Metastasis Net-
work 138List of Figures
1.1 D-separation Between X ,X and X and their equivalent class . 101 2 3
1.2 Belief Propagation Scheme in Tree-structured Network . . . . . . 13
2.1 Example of Single Positive and Negative Influence . . . . . . . . 29
2.2 Example of Joint Effect with Positive and Negative Influence . . 30
2.3 Example of Plain Synergy Influence . . . . . . . . . . . . . . . . 31
2.4 Toy Example for Constraining the Bayesian Model Space . . . . 40
2.5 Monte Carlo Simulation in the 2-dimensional Space . . . . . . . . 42
2.6 Dynamic Bayesian Example . . . . . . . . . . . . . . . . . . . . . 43
2.7 ASIA Network Structure . . . . . . . . . . . . . . . . . . . . . . . 44
2.8 Model Sampling on 2D Parameter Space . . . . . . . . . . . . . . 48
2.9 Prediction Convergence of ASIA Models . . . . . . . . . . . . . . 49
2.10 Belief Inference in Static Bayesian Network . . . . . . . . . . . . 52
2.11 Belief Inference in Static Bayesian Network . . . . . . . . . . . . 57
2.12 E[f(X)] and f(E(X)). . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.13 E[f(X)] and f(E(X)) with various X . . . . . . . . . . . . . . . . . 64
2.14 2-Dimensional Parameter Space with Consistent Hypothesis . . . 66
2.15 2-DimensionalParameterSpacewithConsistentHypothesis(Cont.) 67
2.16 RMSE Distance Measure with Ratio Constraint . . . . . . . . . . 68
2.17 2-DimensionalParameterSpacewithConsistentHypothesis(Cont.) 69
2.18 RMSE Distance Measure with Dif Constraint . . . . . . . . . . . 70
2.19 2-DimensionalParameterSpacewithConsistentHypothesis(Cont.) 71
2.20 RMSE Distance Measure with Dif Constraint . . . . . . . . . . . 73
2.21 Structure of BCBM Network . . . . . . . . . . . . . . . . . . . . 75
2.22 Gene Expression Profiles of BCBM Cell Lines . . . . . . . . . . . 76
2.23 Bone Metastasis Prediction with Qualitative Knowledge . . . . . 79
2.24 Bone Metastasis Observations in Kang’s Experiment . . . . . . . 80
2.25 Bone Metastasis Probability of Different Cell Lines . . . . . . . . 81
2.26 Convergence of Bone Metastasis Prediction by Monte Carlo Sim-
ulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.27 Structure of TGFβ-mediated Cytostatic Program . . . . . . . . . 85
2.28 Parameter Tree of TGFβ-mediated Cytostatic Program . . . . . 86
2.29 Parameter Tree of TGFβ-mediated Cytostatic Program . . . . . 87
3.1 Hierarchical Bayesian Network on Qualitative Knowledge . . . . 92
3.2 ASIA Model Sampling and Inference . . . . . . . . . . . . . . . . 100
3.3 Integrated TGFβ-Smad BCBM Network and Prediction . . . . . 102
3.4 Prediction in TGFβ-Smad BCBM Network . . . . . . . . . . . . 104
iv4.1 Model Integration with Single Node . . . . . . . . . . . . . . . . 109
4.2 Model Integration with Multiple Nodes . . . . . . . . . . . . . . 111
4.3 Structure Uncertainty in Model Fusion . . . . . . . . . . . . . . . 112
4.4 Incomplete Bayesian Network Structure Fusion Example . . . . . 115
4.5 Smad-BCBM Network . . . . . . . . . . . . . . . . . . . . . . . . 117
4.6 TGFβ-Smad Signaling Pathway . . . . . . . . . . . . . . . . . . . 120
4.7 Gene Expression Constraints . . . . . . . . . . . . . . . . . . . . 122
4.8 Integrated TGFβ-Smad BCBM Network and Prediction . . . . . 123
4.9 Breast Cancer Bone Metastasis Network with PTHrP . . . . . . 124
4.10 PTHrP-BCBM Model Candidates by SEM Learning . . . . . . . 127
4.11 Molecular Expression Constraints . . . . . . . . . . . . . . . . . . 128
4.12 Integrated TGFβ-Smad BCBM Network and Prediction . . . . . 128