Browsing by Author "Yoon, Moonyoung"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    Developing basic soccer skills using reinforcement learning for the RoboCup small size league
    (Stellenbosch : Stellenbosch University, 2015-03) Yoon, Moonyoung; Bekker, James; Kroon, R. S. (Steve); Stellenbosch University. Faculty of Engineering. Dept. of Industrial Engineering.
    ENGLISH ABSTRACT: This study has started as part of a research project at Stellenbosch University (SU) that aims at building a team of soccer-playing robots for the RoboCup Small Size League (SSL). In the RoboCup SSL the Decision- Making Module (DMM) plays an important role for it makes all decisions for the robots in the team. This research focuses on the development of some parts of the DMM for the team at SU. A literature study showed that the DMM is typically developed in a hierarchical structure where basic soccer skills form the fundamental building blocks and high-level team behaviours are implemented using these basic soccer skills. The literature study also revealed that strategies in the DMM are usually developed using a hand-coded approach in the RoboCup SSL domain, i.e., a specific and fixed strategy is coded, while in other leagues a Machine Learning (ML) approach, Reinforcement Learning (RL) in particular, is widely used. This led to the following research objective of this thesis, namely to develop basic soccer skills using RL for the RoboCup Small Size League. A second objective of this research is to develop a simulation environment to facilitate the development of the DMM. A high-level simulator was developed and validated as a result. The temporal-difference value iteration algorithm with state-value functions was used for RL, along with a Multi-Layer Perceptron (MLP) as a function approximator. Two types of important soccer skills, namely shooting skills and passing skills were developed using the RL and MLP combination. Nine experiments were conducted to develop and evaluate these skills in various playing situations. The results showed that the learning was very effective, as the learning agent executed the shooting and passing tasks satisfactorily, and further refinement is thus possible. In conclusion, RL combined with MLP was successfully applied in this research to develop two important basic soccer skills for robots in the RoboCup SSL. These form a solid foundation for the development of a complete DMM along with the simulation environment established in this research.
  • Thumbnail Image
    Item
    New multi-objective ranking and selection procedures for discrete stochastic simulation problems
    (Stellenbosch : Stellenbosch University, 2018-03) Yoon, Moonyoung; Bekker, James; Stellebosch University. Faculty of Engineering. Dept. of Industrial Engineering.
    ENGLISH ABSTRACT: In stochastic simulation optimisation, several system designs are considered. These designs are ranked in order and the best is selected based on one or more performance measures. Any ranking and selection (R&S) procedure must ensure that the correct system design is chosen, and this is a challenging task in the stochastic environment. This dissertation discusses the design and development of a new multiobjective ranking and selection (MORS) procedure, called Procedure MMY, and two variants of it, called Procedures MMY1 and MMY2. Single-objective ranking and selection procedures endeavour to find the best system, i.e., the system with the minimum or maximum output, out of a limited number of feasible solutions. There are two important approaches in the single-objective R&S area: the indifference-zone (IZ) approach and the optimal computing budget allocation (OCBA) framework. While the OCBA procedure has been extended to the multi-objective domain, an MORS procedure with the IZ approach has not yet appeared in the literature. The MMY family procedures have been developed in an attempt to fill this gap, therefore they take the IZ approach. Indifference-zone procedures should guarantee that the probability of correct selection is at least a prespecified value P*, denoted by P(CS) * P*, where `correct selection' denotes the event that the system with the minimum output is selected for a single-objective minimisation problem. In the multi-objective context, Pareto optimality is employed to define `correct selection'. The concept of relaxed Pareto optimality is proposed in this research to accommodate the indifference-zone concept properly in the multi-objective domain. Thus, Procedure MMY guarantees P(CS) * P* considering the event of identifying a relaxed Pareto set as a correct selection. Procedure MMY1 tries to find the normal Pareto optimal set while Procedure MMY2 focuses on identifying Pareto optimal solutions with the IZ concept. The statistical validity of the MMY family procedures is proved through rigorous mathematical analyses in this dissertation. A Bayesian probability model was used in the P(CS) formulation in the proofs. Using a Bayesian model in the P(CS) formulation in IZ R&S procedures is a novel approach even in the single-objective context. The researcher therefore proposed a new single-objective R&S procedure, called Procedure MY, in addition to the multi-objective MMY family procedures. The MY procedure is discussed prior to the discussion of the MMY family procedures, verifying the effectiveness of the Bayesian model, thereby laying the theoretical foundation for employing it for the MMY family procedures. The performance of the proposed MMY family procedures was demonstrated using four simulation case studies. These simulation case studies provided various types of test beds to understand the behaviour of the proposed procedures. In all four cases the estimated probability of correct selection was observed to be greater than P* for all three procedures, proving the statistical validity of them empirically, too. In addition, the performance of the proposed MMY family procedures was compared to that of the MOCBA procedure, which is the only existing MORS procedure. The result showed the superiority of the MMY procedure over the MOCBA procedure in many cases.
  • Thumbnail Image
    Item
    Single- and multi-objective ranking and selection procedures in simulation : a historical review
    (Southern African Institute for Industrial Engineering, 2017-08-31) Yoon, Moonyoung; Bekker, James
    ENGLISH ABSTRACT: Ranking and selection (R&S) procedures form an important research field in computer simulation and its applications. In simulation, one usually has to select the best from a number of scenarios or alternative designs. Often, the simulated processes have a stochastic nature, which means that, to distinguish alternatives, they must exhibit significant statistical differences. R&S procedures assist the decision-maker with the selection of the best alternative with high confidence. This paper reviews past and current R&S procedures. The review traces back to the 1950s, when the first R&S procedure was proposed, and discusses the various R&S procedures proposed since then to the present day, presenting a cursory view of the research in the area. The review includes studies in both the single-objective and the multi-objective domains. It presents the research trend, discusses specific issues, and gives recommendations for future research in both domains.