Masters Degrees (Statistics and Actuarial Science)

Permanent URI for this collection

https://scholar.sun.ac.za/handle/10019.1/733

Browse

Browsing Masters Degrees (Statistics and Actuarial Science) by browse.metadata.advisor "Bierman, Surette"

Now showing 1 - 5 of 5
  • Thumbnail Image
    Item
    Advances in random forests with application to classification
    (Stellenbosch : Stellenbosch University, 2016-12) Pretorius, Arnu; Bierman, Surette; Steel, Sarel J.; Stellenbosch University. Faculty of Economic and Management Sciences. Dept. of Statistics & Actuarial Science.
    ENGLISH SUMMARY : Since their introduction, random forests have successfully been employed in a vast array of application areas. Fairly recently, a number of algorithms that adhere to Leo Breiman’s definition of a random forest have been proposed in the literature. Breiman’s popular random forest algorithm (Forest-RI), and related ensemble classification algorithms which followed, form the focus of this study. A review of random forest algorithms that were developed since the introduction of Forest-RI is given. This includes a novel taxonomy of random forest classification algorithms, which is based on their sources of randomization, and on deterministic modifications. Also, a visual conceptualization of contributions to random forest algorithms in the literature is provided by means of multidimensional scaling. Towards an analysis of advances in random forest algorithms, decomposition of the expected prediction error into bias and variance components is considered. In classification, such decompositions are not as straightforward as in the case of using squared-error loss for regression. Hence various definitions of bias and variance for classification can be found in the literature. Using a particular bias-variance decomposition, an empirical study of ensemble learners, including bagging, boosting and Forest-RI, is presented. From the empirical results and insights into the way in which certain mechanisms of random forests affect bias and variance, a novel random forest framework, viz. oblique random rotation forests, is proposed. Although not entirely satisfactory, the framework serves as an example of a heuristic approach towards novel proposals based on bias-variance analyses, instead of an ad hoc approach, as is often found in the literature. The analysis of comparative studies regarding advances in random forest algorithms is also considered. It is of interest to critically evaluate the conclusions that can be drawn from these studies, and to infer whether novel random forest algorithms are found to significantly outperform Forest-RI. For this purpose, a meta-analysis is conducted in which an evaluation is given of the state of research on random forests based on all (34) papers that could be found in which a novel random forest algorithm was proposed and compared to already existing random forest algorithms. Using the reported performances in each paper, a novel two-step procedure is proposed, which allows for multiple algorithms to be compared over multiple data sets, and across different papers. The meta analysis results indicate weighted voting strategies and variable weighting in high-dimensional settings to provide significantly improved performances over the performance of Breiman’s popular Forest-RI algorithm.
  • Thumbnail Image
    Item
    Deep learning for tabular data : an exploratory study
    (Stellenbosch : Stellenbosch University, 2019-04) Marais, Jan Andre; Bierman, Surette; Stellenbosch University. Faculty of Economic and Management Sciences. Dept. of Statistics and Actuarial Science.
    ENGLISH SUMMARY : From about 2006, deep learning has proven to be very successul in application areas such as computer vision, natural language processing, speech and audio recognition, machine translation, bioinformatics, and social network filtering. These successes were undoubtedly facilitated by many advances in neural network architectures. In contrast, deep learning has not yet been found to excel in the context of tabular datasets. Many key machine learning tasks make use of tabular data, where currently the best machine learning models for tabular data use classification or regression trees as base learners. Therefore, the objective of this study is to identify, discuss and explore recent developments in deep learning which may be used to enhance the accuracy of deep neural networks in the tabular data domain. All major developments in the deep learning field are discussed and critically considered, with a view to improving deep learning in the context of tabular data. The challenges of applying deep learning to tabular data are identified, and on each of these fronts, potential improvements are proposed. The most promising modern deep learning architectures are further explored by means of empirical work. We also evaluate the validity of findings reported in the literature, and comment on the effectiveness of recent proposals. A useful byproduct of the study is the development of a code base that may be used to implement the latest deep learning techniques, as well as for comparative model selection experiments.
  • Thumbnail Image
    Item
    Discriminant analysis using sparse graphical models
    (Stellenbosch : Stellenbosch University, 2020-03) Botha, Dylon; Kamper, Francois; Bierman, Surette; Stellenbosch University. Faculty of Economic and Management Sciences. Dept. of Statistics and Actuarial Science.
    ENGLISH SUMMARY : The objective of this thesis is the proposal of a new classification method. This classification method is an extension of classical quadratic discriminant analysis (QDA), where the focus is placed on relaxing the assumption of normality, and on overcoming the adverse effect of the large number of parameters that needs to be estimated when applying QDA. To relax the assumption of normality, we consider assigning to each class density a different nonparanormal distribution. Based on these nonparanormal distributions, new discriminant functions can be derived. When one considers the use of a nonparanormal distribution, the underlying assumption is that the associated random vector, can through the use of an appropriate transformation, be made to follow a Gaussian distribution. Such a transformation is based on the marginals of the distribution, which is to be estimated in a nonparametric way. The large number of parameters in QDA is a result of the estimation of class precision matrices. To overcome this problem, penalised maximum likelihood estimation is performed by placing an L1 penalty on the size of the elements in the class precision matrices. This leads to sparse precision matrix estimates, and therefore also to a reduction in the number of estimated parameters. Combining the above approaches to overcome the problems induced by nonnormality and a large number of parameters to estimate, leads to the following novel classification method. To each class density, a separate transformation is applied. Thereafter L1 penalised maximum likelihood estimation is performed in the transformed space. The resulting parameter estimates are then plugged into the nonparanormal discriminant functions, thereby facilitating classification. An empirical evaluation of the novel proposal shows it to be competitive with a wide array of existing classifiers. We also establish a connection to probabilistic graphical models, which could aid in the interpretation of this new technique.
  • Thumbnail Image
    Item
    Recommender systems
    (Stellenbosch : Stellenbosch University, 2019-12) Dumbleton, Bronwyn Catherine; Bierman, Surette; Stellenbosch University. Faculty of Economic and Management Sciences. Dept. of Statistics and Actuarial Science.
    ENGLISH ABSTRACT: A Recommender System (RS) is a particular type of information filtering system used to propose relevant items to users. Their successful application in online retail is reflected in increased customer satisfaction and sales revenue, with further application in entertainment, e-commerce and services, and content. Hence it may be argued that recommender systems currently present some of the most successful and widely used machine learning algorithms in practice. We provide an overview of both standard and more modern approaches to recommender systems, including content-based and collaborative filtering, as well as latent factor models for collaborative filtering. A limitation of standard latent factor models is that their input is typically restricted to a set of item ratings. In contrast, general purpose supervised learning algorithms allow more flexible inputs, but are typically not able to handle the degree of data sparsity prevalent in recommendation problems. Factorisation machines, which are supervised learning methods, are able to incorporate more flexible inputs and are well suited to deal with the effects of data sparsity. We therefore study the use of factorisation in recommender problems and report an empirical study in which we compare the effects of data sparsity on latent factor models, as well as on factorisation machines. Currently in RS research, emphasis is placed on the advantages of recommender systems that yield recommendations that are simple to explain to users. Such recommender systems have been shown to be much more trustworthy than more complex, unexplainable systems. Towards a proposal for explainable recommendations, we also provide an overview of the connection between the recommender problem and Multi-Label Classification (MLC). Since some of the recent MLC approaches facilitate the interpretation of predictions, we conduct an empirical study in order to evaluate the use of various MLC approaches in the context of recommender problems.
  • Thumbnail Image
    Item
    The use of bayesian neural networks in thyroid cancer classification
    (Stellenbosch : Stellenbosch University, 2023-03) Du Preez, Tiana; Bierman, Surette; Stellenbosch University. Faculty of Economic and Management Sciences. Dept. of Statistics and Actuarial Science.
    ENGLISH SUMMARY: Artificial Neural Networks form a class of machine learning models that can be used to model complex relationships between variables. They are used in an innumerable number of practical applications toward solving real-world problems. However, one of the limitations of conventional neural networks is that they are not designed to accurately quantify uncertainty in network predictions. A possible solution to this problem is the use of Bayesian inference to introduce stochasticity in neural networks. For example, Bayesian neural networks assign a prior distribution to the neural network weight parameters. The posterior distribution is then derived by means of variational inference algorithms. Bayesian neural networks are currently successfully used in a wide variety of applications. Since they are particularly useful in settings where quantification of uncertainty in prediction is important, a key application area for Bayesian neural networks is the medical field. We investigate the use of Bayesian Neural Networks in thyroid cancer classification. Thyroid cancer diagnosis is a difficult task. Therefore, developing models yielding predictions of cancer stages, with accurate associated risks, can be a worthwhile contribution. In our empirical work, we thus focus on the classification of thyroid cancer by means of Bayesian neural networks. More specifically, since we use data that consist of ultrasound images, we fit Bayesian Convolutional Neural Networks for image classification. Modified versions of the LeNet-5, AlexNet and GoogLeNet network architectures are considered. Most of the different architectures, adapted for Bayesian inference, are found to perform slightly better than the corresponding conventional network architectures. In addition, Bayesian aleatoric and epistemic uncertainties are reported for each model. This uncertainty quantification may be considered a sensible contribution.