European Spatial Data Research State-of-the-Art of Automated Generalisation in Commercial Software by Jantien Stoter Medium Format Cameras by Görres Grenzdörffer Official Publication No 58 November 2010 The present publication is the exclusive property of European Spatial Data Research All rights of translation and reproduction are reserved on behalf of EuroSDR. Published by EuroSDR Printed by Gopher, Amsterdam, The Netherlands EUROPEAN SPATIAL DATA RESEARCH PRESIDENT 2010 – 2012: Jean-Philippe Lagrange, France VICE-PRESIDENT PRESIDENT 2009 – 2011: Dieter Fritsch, Germany SECRETARY-GENERAL: Kevin Mooney, Ireland DELEGATES BY MEMBER COUNTRY: Austria: Michael Franzen Belgium: Ingrid Vanden Berghe; Jean Theatre Croatia: Željko Heüimoviü; Ivan Landek Cyprus: Christos Zenonos; Michael Savvides Denmark: Thorben Hansen; Lars Bodum Finland: Risto Kuittinen; Jurkka Tuokko France: Jean-Philippe Lagrange; Xavier Briottet Germany: Dietmar Grünreich; Klement Aringer; Dieter Fritsch Ireland: Colin Bray, Ned Dwyer Italy: Fabio Crosilla Netherlands: Jantien Stoter; Aart-jan Klijnjan Norway: Jon Arne Trollvik; Ivar Maalen-Johansen Spain: Antonio Arozarena, Emilio Domenech Sweden: Anders Olsson; Anders Östman Switzerland: Francois Golay; André Streilein-Hurni United Kingdom: Malcolm Havercroft; Jeremy Morley COMMISSION CHAIRPERSONS: Sensors, Primary Data Acquisition and Georeferencing: Michael Cramer, Germany Image Analysis and Information Extraction: Norbert Pfeifer, Austria Production Systems and Processes: André Streilein-Hurni, Switzerland Data Specifications: Ulf Sandgren, Sweden Network Services: Mike Jackson, United Kingdom OFFICE OF PUBLICATIONS: Bundesamt für Kartographie und Geodäsie (BKG) Publications Officer: Andreas Busch Richard-Strauss-Allee 11 60598 Frankfurt Germany Tel.: + 49 69 6333 312 Fax: + 49 69 6333 441 CONTACT DETAILS: Web: www.eurosdr.net President: president@eurosdr.net Secretary-General: secretary@eurosdr.net Secretariat: admin@eurosdr.net EuroSDR Secretariat Faculty of the Built Environment Dublin Institute of Technology Bolton Street Dublin 1 Ireland Tel.: +353 1 4023933 The official publications of EuroSDR are peer-reviewed. Jantien Stoter: “State-of-the-Art of Automated Generalisation in Commercial Software” ........................... 9 CAVEATS..................................................................................................................................................... 10 ACKNOWLEDGEMENTS .......................................................................................................................... 11 ABSTRACT............................... ................................................................................................................... 12 EXECUTIVE SUMMARY ........................................................................................................................... 13 1. PRESENTATION OF THE PROJECT .................................................................................................. 18 1.1 Introduction ................................................................................................................................... 18  3UHYLRXV UHVHDUFK UHODWHG WR PDS VSHFL¿FDWLRQV IRU DXWRPDWHG PDS JHQHUDOLVDWLRQ  1.3 Scope of the current study ............................................................................................................. 20 1.4 Project set up ................................................................................................................................ 20 2. METHODOLOGY ................................................................................................................................ 21 2.1 Requirement analysis .................................................................................................................... 21  6HOHFWLQJ WKH WHVW FDVHV ........ 21  )RUPDOLVDWLRQ RI 10$ PDS VSHFL¿FDWLRQV IRU DXWRPDWHG JHQHUDOLVDWLRQ    +DUPRQLVLQJ 10$ PDS VSHFL¿FDWLRQV IRU DXWRPDWHG JHQHUDOLVDWLRQ    $QDO\VLQJ WKH WHVW FDVHV ....... 26 2.2 The test process ............................................................................................................................. 27 2.3 Evaluation of system capabilities, test processes and constraint expressions ............................... 28  (YDOXDWLRQ RI JHQHUDOLVHG RXWSXWV .......... 28 2.4.1 Automated constraint-based evaluation ............................................................................ 29  &RPSDULQJ RXWSXWV ....... 29 9LVXDOO\ FRPSDULQJ IRFXV ]RQHV   4XDQWLI\LQJ GLIIHUHQFHV . 30 2.4.3 Expert evaluation .............................................................................................................. 31 3. RESULTS AND INTERPRETATION .................................................................................................. 33 3.1 Outputs of the tests ........................................................................................................................ 33 3.2 Evaluation of the capabilities of the systems ................................................................................ 34 3.2.1 Details on the completed systems templates .................................................................... 34 3.2.2 Generalisation functionalities of the systems ................................................................... 35 ArcGIS .............................................................................................................................. 36 0DLQ JHQHUDOLVDWLRQ IXQFWLRQDOLWLHV   Summary of the system templates provided by the testers for ArcGIS ...................... 38 $GYDQWDJHV DQG OLPLWDWLRQV   &KDQJH 3XVK DQG 7\SLI\ &37   &KDQJH ..... 42 0DLQ JHQHUDOLVDWLRQ IXQFWLRQDOLWLHV   $GYDQWDJHV DQG OLPLWDWLRQV   Push ............................................................................................................................. 43 0DLQ JHQHUDOLVDWLRQ IXQFWLRQDOLWLHV   $GYDQWDJHV DQG OLPLWDWLRQV   Typify .......................................................................................................................... 44 0DLQ JHQHUDOLVDWLRQ IXQFWLRQDOLWLHV   $GYDQWDJHV DQG OLPLWDWLRQV   6XPPDU\ RI WKH V\VWHP WHPSODWHV SURYLGHG E\ WKH WHVWHUV IRU &KDQJH 3XVK 7\SLI\  $GYDQWDJHV DQG OLPLWDWLRQV IRU &37 VRIWZDUH   Radius Clarity ................................................................................................................... 49 0DLQ JHQHUDOLVDWLRQ IXQFWLRQDOLWLHV   Summary of the system templates provided by the testers for Radius Clarity ............ 52 $GYDQWDJHV DQG OLPLWDWLRQV   axpand ............................................................................................................................... 56 0DLQ JHQHUDOLVDWLRQ IXQFWLRQDOLWLHV   Summary of the system templates provided by the testers for axpand ....................... 57 $GYDQWDJHV DQG OLPLWDWLRQV   3.2.3 Summary of the capabilities of the systems ...................................................................... 60 General capabilities ........................................................................................................... 60 Quality of the available operators ..................................................................................... 63 3.2.4 Conclusions for the capabilities of tested software systems based on testers’ information ........................................................................................................................ 64 3.3 Evaluation of test processes .......................................................................................................... 67 General trends ................................................................................................................................ 68 Conclusions ................................................................................................................................ 69 3.4 Evaluation of constraint expressions ............................................................................................. 69  $XWRPDWHG HYDOXDWLRQ RI JHQHUDOLVHG RXWSXWV UHVXOWV DQG FRQFOXVLRQV    $XWRPDWHG FRQVWUDLQW EDVHG HYDOXDWLRQ RI LQWHUDFWLYHO\ JHQHUDOLVHG GDWD    $XWRPDWHG FRQVWUDLQW EDVHG HYDOXDWLRQ RI DXWRPDWLFDOO\ JHQHUDOLVHG GDWD   $XWRPDWHG HYDOXDWLRQ RI PLQLPXP DUHD RI EXLOGLQJV   0LQLPXP DUHD RI EXLOGLQJV ,&& WHVW FDVH   0LQLPXP DUHD RI EXLOGLQJV ,*1 WHVW FDVH   0LQLPXP DUHD RI EXLOGLQJV .DGDVWHU WHVW FDVH   0LQLPXP DUHD RI EXLOGLQJV 26*% WHVW FDVH   )LQGLQJV DQG FRQFOXVLRQV RI DXWRPDWHG HYDOXDWLRQ RI PLQLPXP DUHD FRQVWUDLQW   $XWRPDWHG HYDOXDWLRQ RI PLQLPXP GLVWDQFH EHWZHHQ WZR EXLOGLQJV   0LQLPXP GLVWDQFH EHWZHHQ WZR EXLOGLQJV ,&& WHVW FDVH   0LQLPXP GLVWDQFH EHWZHHQ WZR EXLOGLQJV ,*1 WHVW FDVH   0LQLPXP GLVWDQFH EHWZHHQ WZR EXLOGLQJV .DGDVWHU WHVW FDVH   )LQGLQJV DQG FRQFOXVLRQV RI DXWRPDWHG HYDOXDWLRQ RI PLQLPXP GLVWDQFH EHWZHHQ WZR EXLOGLQJV   $XWRPDWHG HYDOXDWLRQ RI PLQLPXP GLVWDQFH EHWZHHQ EXLOGLQJV DQG URDGV   0LQLPXP GLVWDQFH EHWZHHQ EXLOGLQJV DQG URDGV UHVSHFWLQJ GLIIHUHQW URDG W\SHV IGN test case .......................................................................................................... 93 0LQLPXP GLVWDQFH EHWZHHQ EXLOGLQJV DQG URDGV UHVSHFWLQJ GLIIHUHQW URDG W\SHV Kadaster test case ................................................................................................... 93 0LQLPXP GLVWDQFH EHWZHHQ EXLOGLQJV DQG URDGV DJJUHJDWHG IRU DOO URDG W\SHV   )LQGLQJV DQG FRQFOXVLRQV RI DXWRPDWHG HYDOXDWLRQ RI PLQLPXP GLVWDQFH EHWZHHQ EXLOGLQJV DQG URDGV    (YDOXDWLRQ E\ FRPSDULQJ JHQHUDOLVHG RXWSXWV UHVXOWV DQG FRQFOXVLRQV    ,&& ± 7RZQ FHQWUH EORFNV DQG VWUHHWV UHSUHVHQWDWLRQ VHOHFWLRQ DJJUHJDWLRQ    ,&& ± &RDVWOLQH JHQHUDOLVDWLRQ .. 105 3.6.3 ICC – Generalisation of complex junctions ...................................................................... 111  ,&& ± *HQHUDOLVDWLRQ RI VXEXUEDQ EXLOGLQJV    ,&& ± 3DUDOOHOLVP EHWZHHQ URDGV DQG EXLOGLQJV    ,*1) ± %XLOGLQJV VHOHFWLRQ LQWHUGLVWDQFH VL]H    ,*1) ± 0RXQWDLQRXV URDGV FRDOHVFHQFH LQ EHQGV VHULHV    ,*1) ± 9HJHWDWLRQ VHOHFWLRQ DQG JHRPHWULF VLPSOL¿FDWLRQ   3.6.9 IGNF – Ski lifts representation ......................................................................................... 131  .DGDVWHU ± &KDQQHO QHWZRUN VHOHFWLRQ    .DGDVWHU ± 6HWWOHG DUHD EXLOGLQJ VHOHFWLRQ   3.6.12 Kadaster – Generalisation of parallel roads and cycle tracks ........................................... 144  .DGDVWHU ± 5DLOZD\V W\SL¿cation ...................................................................................... 149  26*% ± $GMDFHQW EXLOGLQJV UHSUHVHQWDWLRQ    26*% ± 'HWDFKHG EXLOGLQJV DQG IHQFHV    26*% ± 'XDO FDUULDJHZD\ UHSUHVHQWDWLRQ URDGV SDUDOOHOLVP SUHVHUYDWLRQ    0DLQ ¿QGLQJV DQG FRQFOXVLRQV RI HYDOXDWLRQ E\ FRPSDULQJ RXWSXWV    ([SHUW HYDOXDWLRQ UHVXOWV DQG FRQFOXVLRQV ..... 161 3.7.1 Details on the responses of the expert evaluation ............................................................. 161 3.7.2 Global expert evaluation ................................................................................................... 163 Frequency of respondents’ scores ..................................................................................... 163 Harmonised respondents’ scores ....................................................................................... 167 3.7.3 Detailed constraint-based expert evaluation...................................................................... 170 Evaluation of constraints for individual objects ................................................................ 170 Evaluation of constraints for two similar objects .............................................................. 173 Evaluation of constraints for two different objects ........................................................... 175 (YDOXDWLRQ RI FRQVWUDLQWV IRU JURXSV RI REMHFWV   3.7.4 Examples of well, badly and differently solved constraints ............................................. 180  5DQNLQJ WKH JHQHUDOLVHG RXWSXWV . 181  0DLQ ¿QGLQJV DQG FRQFOXVLRQV RI H[SHUW HYDOXDWLRQ   4. VENDORS’ SOLUTIONS .................................................................................................................... 185 4.1 Vendors’ tests ................................................................................................................................ 185 4.1.1 ArcGIS .............................................................................................................................. 185  &KDQJH3XVK7\SLI\ ...... 187 IGN test case ..................................................................................................................... 187 OSGB test case ................................................................................................................. 189 ICC test case ..................................................................................................................... 189 Kadaster test case .............................................................................................................. 191 4.1.3 Radius Clarity.................................................................................................................... 193 4.1.4 Axpand............................................................................................................................... 195 4.1.5 Conclusions on the vendors’ tests ..................................................................................... 196 4.2 Developments since 2007 and references to examples from practice ............................................ 196 4.2.1 ArcGIS .............................................................................................................................. 196  &KDQJH3XVK7\SLI\ ...... 197 4.2.3 Radius Clarity ................................................................................................................... 197 4.2.4 Axpand .............................................................................................................................. 198 5. CONCLUSIONS AND DISCUSSION ................................................................................................. 199 5.1 Answers to research questions ...................................................................................................... 199 5.1.1 What are the possibilities and limitations of commercial software systems for DXWRPDWHG JHQHUDOLVDWLRQ ZLWK UHVSHFW WR 10$ UHTXLUHPHQWV"    :KDW GLIIHUHQW JHQHUDOLVDWLRQ VROXWLRQV FDQ EH REWDLQHG IRU RQH WHVW FDVH DQG ZK\ GR WKH\ GLIIHU" .......... 200 5.2 Conclusions and further research .................................................................................................. 201  'H¿QLQJ PDS VSHFL¿FDWLRQV DV FRQVWUDLQWV    )RUPDOLVLQJ DQG HYDOXDWLQJ SUHVHUYDWLRQ VSHFL¿cations ................................................... 202  &RQVWUDLQWEDVHG JHQHUDOLVDWLRQ .. 202  (YDOXDWLQJ JHQHUDOLVDWLRQ VRIWZDUH EH\RQG FRQVWUDLQWV    &RQFOXGLQJ UHPDUNV ....... 204 6. REFERENCES ..................................................................................................................................... 205 7. APPENDICES ..................................................................................................................................... 209 APPENDIX I 9LVXDOLVDWLRQV RI LQLWLDO GDWD 2QO\ GLJLWDO   APPENDIX II 6\PERO GHVFULSWLRQV RI WHVW RXWSXW PDSV 2QO\ GLJLWDO   APPENDIX III Harmonised constraints for one object ............................................................................. 211 APPENDIX IV Harmonised constraints for two objects ............................................................................ 214 APPENDIX V +DUPRQLVHG FRQVWUDLQWV IRU JURXS RI REMHFWV   APPENDIX VI 10$ VSHFL¿F FRQVWUDLQW VHWV 2QO\ GLJLWDO   APPENDIX VII 9LVXDOLVDWLRQV RI RXWSXW GDWD SURGXFHG E\ SURMHFW WHDP 2QO\ GLJLWDO   APPENDIX VIII 9LVXDOLVDWLRQV RI RXWSXW GDWD SURGXFHG E\ YHQGRUV 2QO\ GLJLWDO   APPENDIX IX Completed system templates ............................................................................................ 218 APPENDIX X 4XDOLW\ RI DYDLODEOH JHQHUDOLVDWLRQ RSHUDWRUV   APPENDIX XI 5HVSRQVHV DQG DQDO\VLV RI H[SHUW HYDOXDWLRQ 2QO\ GLJLWDO   Görres Grenzdörffer: “Medium Format Cameras“ ....................................................................................... 233 ABSTRACT................................................................................. ................................................................. 234 1. INTRODUCTION ................................................................................................................................ 234 1.1 Objectives of this report ................................................................................................................ 234 2. CATEGORIZATION OF DIGITAL MEDIUM FORMAT CAMERA SYSTEMS .............................. 235 2.1 Developments of medium format cameras in the last two years ................................................... 236 3. COMPARISON OF MEDIUM FORMAT CAMERAS TO LARGE FORMAT CAMERAS .............. 238 3.1 Geometric Potential ....................................................................................................................... 240 3.1.1 Interior Orientation ........................................................................................................... 240 3.1.2 MTF ................................................................................................................................ 241  %DVH WR KHLJKW UDWLR .......... 241 3.2 Minimum GSD of medium format cameras .................................................................................. 242  ,PDJHPRWLRQ .................... 243 4. GEOMETRIC AND RADIOMETRIC CALIBRATION ...................................................................... 244 4.1 Geometric calibration .................................................................................................................... 244 4.2 Geometric performance of medium format - Results of the DGPF-Testbed ................................ 244 4.3 Radiometric accuracy and calibration ........................................................................................... 245 4.3.1 Color infrared with medium format cameras ................................................................... 247 5. STANDARDIZATION................... ....................................................................................................... 249 6. APPLICATION DOMAINS ................................................................................................................. 250 7. MARKET SURVEY OF MEDIUM FORMAT CAMERA SYSTEMS ............................................... 252 7.1 RMK-D .................................. ............................................................................................. 252  7ULPEOH  $SSODQL[ '66 ±  .......... 253 7.3 Trimble AIC ................................................................................................................................ 254  'LJL&$0 +  +  + ......... 255  'L0$&  'LJLWDO 0RGXODU $HULDO &DPHUD . 255 7.6 Leica‘s RCD medium format cameras .......................................................................................... 256 7.7 UltraCamLp ................................................................................................................................ 257 8. CURRENT TRENDS AND OUTLOOK FOR MEDIUM FORMAT CAMERA SYSTEMS ............. 257  5DSLG SURFHVVLQJ ................... 257 8.2 Multi head systems ........................................................................................................................ 258  )RUZDUGPRWLRQ FRPSHQVDWLRQ )0& .. 259  2EOLTXH ,PDJHV ................... 259  ,QFUHDVH LQ LPDJH VL]H ................. 260 9. ACKNOWLEDGMENTS.............. ....................................................................................................... 261 10. REFERENCES ..................................................................................................................................... 262 (XUR6'53URMHFW State-of-the-art of automated generalisation in commercial software )LQDO 5HSRUW March 2010 Jantien Stoter, TU Delft (Research carried out at ITC, Enschede, NL) Blanca Baella, ICC, Catalonia Connie Blok, ITC, Enschede, NL Dirk Burghardt, TU Dresden (Research carried out at University of Zurich, Switzerland) Cécile Duchêne, IGN, France Maria Pla, ICC, Catalonia Nicolas Regnauld, Ordnance Survey, United Kingdom Guillaume Touya, IGN, France Temporal project team members Karl-Heinrich Anders, University of Hannover, Germany Jan Haunert, University of Hannover, Germany Nico Bakker, Kadaster, NL Francisco Dávila, IGN, Spain Annemarie Dortland, Kadaster, NL Peter Lentjes, Kadaster, NL Peter Rosenstand, KMS, Denmark Stefan Schmid, University of Zurich, Switzerland Maarten Storm, Kadaster, NL Harry Uitermark, Kadaster, NL Xiang Zhang, ITC, Enschede, NL &DYHDWV Not all map fragments are sufficiently readable when printed in black and white. Therefore one should either use a colour-printed or a digital version to understand all details of the figures. The test data have been modified for the project and therefore they do not fully resemble the data as used in production. Also symbolisation and map specifications have been simplified for the tests. Consequently they are not used as such in practice. 10 $FNQRZOHGJHPHQWV The project presented in this report could never have happened without the support of many highly expertised, pleasant and cooperative people. Firstly, many thanks go to the vendors who participated in the project. They are: Axes Systems (Mary Lou von Wyl and Ajay Mathur), ESRI (Dan Lee, Jean-Luc Monnot, Paul Hardy, John van Smaalen, David Watkins), University of Hannover (Monika Sester) and 1Spatial (Martin Gregory). They provided their software for the tests, performed parallel tests and supported the project team testers during the tests. Also the different testers from the partner organisations were important because they contributed to the considerable amount of test outputs per test case on different systems that enabled us to draw meaningful conclusions. They are (apart from project team members): Magali Valdepérez (IGN, Spain), Patrick Revell (OS, UK), Stuart Thom (OS, UK), Sheng Zhou (OS, UK), Willy Kock (ITC, NL), and Patrick Taillandier (IGN, France). EuroSDR was of great meaning for providing the framework to make the project possible. Finally I am deeply grateful to all the project team members working at various research institutes and National Mapping and Cadastral Agencies across Europe. Their contributions were indispensable for the success of this project and consisted of careful participation in face to face meetings (seven in total) and in many email discussions. The project highly benefitted from the resulting concise decisions in all phases of the project: sourcing the test cases; performing the tests; conducting parts of the evaluation; and writing the final report. They are all the people who are listed on the title page of this report. I cannot name these persons individually here, because any order could give the impression that some work was more important than the other. However I can say that the work of the individual contributors was impressive and I feel privileged that I worked with this group of very capable and inspiring people. Jantien Stoter Enschede, March 2010 11 $EVWUDFW This report presents the EuroSDR research project that studied the state-of-the-art of automated generalisation in commercial software in a collaboration between National Mapping Agencies (NMAs), research institutes and vendors. The aims of the study were to learn more about generic and specific map requirements of NMAs, to show possibilities and limitations of commercial generalisation software, and to identify areas for further developments based on latest research advances. The project consisted of three main steps: requirements analysis, testing, and evaluation. The requirement analysis (carried out between Oct 2006 till June 2007) resulted in four representative test cases, formalised and harmonised NMA map specifications for automated generalisation as well as an analysis of the defined specifications that shows the similarities and differences between map specifications of different NMAs. Between June 2007 and Spring 2008 tests were performed by project team members (from NMAs and research institutes) on out-of-the-box versions of four generalisation systems: ArcGIS (ESRI), Change/Push/Typify (University of Hanover), Radius Clarity (1Spatial) and axpand (Axes Systems). At the same time the vendors (except Axes systems) carried out tests with the same test cases with improved and/or customised versions of their systems. The tests resulted in 35 outputs consisting of 700 thematic layers, where it should be noted that the effort for one test was approximately 1 week. The evaluation, carried out between summer 2008 and spring 2009, consisted of an evaluation of meta aspects (based on information recorded by the testers) and of an evaluation of the generalised datasets themselves. The latter evaluation consisted of three parts that completed each other: a) automated constraint-based evaluation, b) evaluation which visually compared different outputs for one test case and c) a qualitative evaluation by cartographic experts. From the project results it can be concluded that all systems offer potentials for automated generalisation. However the results highlighted a few issues that identify areas for further development in both research and commercial systems. Although the results show that for many problems solutions do exist (e.g. building simplification), the algorithms are difficult to parameterise and a direct match between parameters and specifications was often missing. In addition none of the four test cases were fully solved by the out-of-the-box systems. While some problems are close to being solved (generalisation of individual buildings and roads), a few problems are far from being solved. Firstly it is impossible with the tested systems to apply different algorithms and/or parameter values in different contexts. This is either not supported or a measure to detect the appropriate contexts is missing. Another remaining generalisation software problem is operations that concern more than one object (e.g. network typification). Also, the generalisation of the topographic context in an integrated manner with the terrain is not appropriately covered in the tested systems. It should be noted that some of the missing functionalities were fixed in the vendors’ parallel tests (e.g. buildings elimination and displacement algorithms in ArcGIS and Radius Clarity). Although these results may seem disappointing, some final thoughts may help to put the results in the right context. Firstly the project had very high ambitions (i.e. many specifications were defined; the selection of test cases focused on known and complex problems; the ultimate aim of the generalisation process was high quality paper maps). Secondly, the project is well received by vendors to push internal developments. In addition it is not a surprise that out-of-the box versions are not capable of fulfilling NMA requirements, which is also shown by the fact that customised systems are used more satisfactory in practice. Consequently customisation of the systems should be further developed and should be one of the focuses in a future project. 12 ([HFXWLYH VXPPDU\ In 2006 a project started to study the state-of-the-art of automated generalisation with commercial software. The project team consists of people from research institutes and National Mapping Agencies (see title page of report). Four vendors participated in the project: ESRI Inc with the software ArcGIS (USA), University of Hannover (Germany) with the software modules: Change, Push and Typify (CPT), 1Spatial (United Kingdom) with the software Radius Clarity and Axes Systems (Switzerland) with the software axpand. This report presents the methodology and results of the project. Chapter 1 presents the project, including research questions, previous research, detailed objectives and scope of the project. Chapter 2 presents the methodology of the project. The methodology consists of the following steps: o 5HTXLUHPHQW DQDO\VLV o Selection of four test cases representative for typical generalisation problems o For the four test cases, formalisation of NMA map specifications for automated generalisation in constraints that define conditions for the generalisation outputs. o Harmonisation of the constraints. The aim was to identify constraints which are similar in the specifications provided by different NMAs, and replace them with a single (generic) one that can be parameterised. The resulting set distinguishes between constraints defined for one object, constraints that are defined for two objects and constraints that are defined for groups of objects. o Analyses of the defined specifications to learn more about similarities and differences between NMA map specifications. o 7HVWLQJ In every test (performed from June 2007 till March 2008) the tester of the project team tried to translate all defined constraints for the test case into a form understandable by the specific software. Several templates were designed to capture the test information in a structured and consistent way: o Processing template: a file which lists every action of the tester o Constraint expression template: a file describing how the tester implemented every constraint o All output layers in ESRI Shape format The tests were performed using the current version of the commercial software (i.e. June, 2007), although the vendors were allowed to use them in the parallel tests, and also to apply any customisation specially designed for the tests, in order to show the full potentials of the systems. o (YDOXDWLRQ o Evaluation of system-, processing- and constraint-expression-templates. o Evaluation of generalised outputs. An evaluation framework was designed to balance between human and machine evaluation and to expose possible inconsistencies of the evaluation. The three parts are: o An automated constraint-based evaluation, where the satisfaction values of some constraints are computed. O An evaluation to compare generalised data, where the different outputs obtained for a given test case are visually compared to identify and to explain differences between outputs. O An expert evaluation, where cartographic experts of the NMAs that provided the four test cases evaluated the cartographic outputs of their own generalisation tests. Chapter 3 presents and interprets the results of the evaluation. 13 The evaluation of system capabilities shows that all the software systems provide a set of tools but none of them achieve globally good results. Despite the current limitations, all four systems can be implemented to automate partially the generalisation processes and optimise the production workflows. Topology is only partially managed and 2.5D is only supported in one of the systems. Some functionality is still missing in all the systems, for example incremental updating and full contextual generalisation. Although some systems allow the input of generalisation requirements through constraints or rules, improvements in the definition of the user requirements and their implementation in the systems would be necessary. Only two of the systems allow the customisation of provided generalisation tools, for adding new algorithms or modifying existing functionalities in order to improve the results and facilitate the integration of the systems in a production workflow. There are also additional minor limitations, such as poor or lack of information about the results of the generalisation process, including reporting of errors, statistics, measurements, etc. The analysis of the completed processing templates shows a heavy amount of time spent by most testers on the installation of the software. The templates also confirm that technical mastery on generalisation software is essential to reduce the amount of time spent on the tests. Finally the templates highlight two specific limitations of generalisation solutions in commercial software (which are known in research), namely the difficulty to parameterise the complex algorithms and the lack of default tools (for instance default algorithm sequences or default constraints) requiring a lot of user’s work to find the optimal generalisation solution for a given problem. In the constraint expression templates, testers entered for specific test case whether they were able to express the constraints, with the aim to provide insight into what of them can be managed by the systems. The following main observations are made: o A considerable amount of constraints (i.e. about 50%) could be expressed fully or partially in the systems. o The most supported constraints are those applying to a single object. o The constraints that were easiest to handle by the testers are the ones common for all test cases. Also here, testers indicated that functionalities for parameterisation are missing. In addition they mention a lack of functionality for defining sensible groups for generalisation. Although conclusions from the analysis of the templates meet the objectives of the research of quantifying the state-of-the-art of automated generalisation, many biases were detected which may cause readers drawing wrong conclusions. Examples are that the importance of constraints was not taken into account and that the results are dependent on the specific constraints that were defined (and ignored) for the test cases. Besides the above evaluation of side products of the tests, the generalised data themselves were evaluated with three methods. Most of the time of the DXWRPDWHG FRQVWUDLQWEDVHG HYDOXDWLRQ was put in developing the prototype (due to the faced complexities). Consequently only a limited number of constraints (that were sufficiently formalised) could be evaluated: minimum area of buildings, minimum distance between buildings and minimum distance between roads and buildings. Because of this limited number, the results should be interpreted with care. In addition, although one constraint achieves good results in the evaluation, it may be possible that the generalisation result is not the expected result. For example, the more deleted buildings, the higher the chance to satisfy the minimum distance constraint. But many deleted buildings are definitely not what one would expect for a good generalisation solution, despite the high satisfaction value on minimum distance. Conclusions from the evaluations that can add (partly) to the research questions are: o All systems provide general good results for minimum area of buildings constraints (except in densely built areas), although for same test cases and same systems very different results were achieved. 14 o Only CPT and axpand achieved good solutions for the minimum distance between buildings constraint. o The characteristics of the initial data heavily influence the constraint violation. Apart from evaluating the automatically generalised data, the evaluation prototype was applied to interactively generalised data of Kadaster, scale 1:50k (the target dataset of the test case of Kadaster) to better understand how this type of evaluation can indicate the overall quality of generalisation output. This test confirmed that many detected violations do not necessarily indicate bad generalisation solutions. Firstly because cartographers can take a flexibility around parameter values into account which is not possible in automated evaluation. In addition constraints do not necessarily describe cartographic conflicts. Therefore more research is required to better define constraints with respect to automated evaluation. In addition, to aggregate the evaluation results of several constraints better understanding of the impact and dependencies of several constraints is required. Several focus zones per test case were visually analysed in the FRPSDUDWLYH HYDOXDWLRQ, resulting in conclusions regarding capabilities of the systems with respect to the NMAs requirements. First, only a few generalisation problems that were raised by the test cases appear to be fully solved by the out-ofthe-box systems, this is true for complicated problems, but also for classical problems. For algorithms that are available their parameterisation is difficult. Some of the shortcomings seem to be under study and/or have been corrected by the vendors, as shown by the results obtained by the vendors in their parallel testing (buildings elimination and displacement algorithms in ArcGIS and Radius Clarity, for instance). The comparison evaluation also showed that outputs for one test case can be very different. This can be explained by difficult parameterisation and by sometimes fuzzy NMA specifications that do not express fully their actual requirements. In addition constraints appeared not to be always capable of defining without ambiguity what is expected. Consequently testers that were familiar with the test data and knew what would be expected obtained other results than testers that were new to the data. In the H[SHUW HYDO XDWLRQ the respondents were able to evaluate generalised outputs on individual constraints taking the specific context into account. The expert evaluation showed that the generalised outputs scored well on the global indicators ‘Deviation from the map of the original data’ and ‘Preservation of geographic characteristics’. On the other hand the generalised outputs scored less positive on the following global indicators: Legibility, Manual editing required, Number of main detected errors, Information reduction, Number of main positive aspects. It should be noted that good scores on preservation are biased for situations where no generalisation has been done as outputs are globally assessed as undergeneralised (i.e. they score badly on Information reduction). The expert evaluation also studied how cartographic experts perceived the solutions of individual constraints, which showed that best results are obtained for constraints on individual objects, specifically for roads and buildings (in line with the results of automated constraint-based evaluation). The solutions for other constraints received worse scores. Finally the expert evaluation identified noticeable differences between software systems and test cases, which may show the fitness for one system to handle the specificities of a given test case, examples are relatively high scores of CPT for minimum dimensions, granularity and quantity of information of buildings, as well as for minimum distance constraint. In case of preservation constraints, noticeable differences may also indicate situations that are not touched at all by some systems, where other systems did perform (some) generalisation. Examples are relatively high scores of axpand and Radius Clarity for shape and spatial distribution of contour lines, of which it was known that they had not been generalised. Differences between project team testers’ and vendors’ outputs show also that either mastery of the software is required to obtain the best possible solutions (for example CPT) or that, depending on the 15 cases, the vendors have really made an effort on the additional developments for their parallel testing (for example Radius Clarity). Chapter 4 describes the vendors’ tests. University of Hannover performed tests on all four test cases with the same version of the software as was tested by the project team. From these tests we can see that mastery of the system considerably reduces the amount of time and produces the best results for this software (in which parameterisation is not straightforward). ESRI performed tests on one test case using a research prototype, i.e. optimisation engine, which shows promising techniques for displacement (not available for project team testers) and building generalisation. 1Spatial extended their tests on two test cases with a few additional algorithms that were not available to the project team. Therefore also for this software the displacements algorithms, that are fundamental for generalisation, were only used in the vendor tests. Axes Systems did not perform tests themselves. Chapter 4 also lists some developments compared to the versions tested in our project. These developments are provided by the vendors and are therefore not tested. In addition, the vendors provided us with references and examples that show satisfactory use of the software in practice, which are also included in Chapter 4. Chapter 5 lists several conclusions that show the capabilities and limitations of commercial software for automated generalisation with respect to NMA requirements. o All the tested systems offer potentials for automated generalisation, especially for handling constraints on single objects. However only a few generalisation problems raised in the project appeared to be fully solved by the out-of-the-box systems. The tested systems provide generic solutions which are not directly applicable to the specific cases. o In line with the first conclusion, cartographic experts in the expert evaluation did not score the generalised outputs very high, with some exceptions. o For some classical problems not all needed functionalities are provided by the out-of-the-box systems, e.g. contextual problems, situations that require displacement (provided by only two of the four software systems). o For other classical problems, algorithms are present but the tests highlighted difficulty to parameterise the complex algorithms (a direct match between the parameters and constraints was mostly lacking), difficulty to detect where and how to apply the algorithms and the lack of default tools, for instance default algorithm sequences or default constraints. The results may look disappointing. However they should be interpreted with care because of the high ambitions of the project (very precise generalisation requirements, test cases contained a selection of complex/known problems, focus was on the production of high quality paper maps). One should be aware that the functionality available in the four systems does enable to automate part of the generalisation processes and to optimise the production workflows. Another relevant remark is that some of the shortcomings, that have been solved at NMAs or research institutes, were tackled by the vendors in their parallel testing (buildings elimination and displacement algorithms in ArcGIS and Radius Clarity, for instance). Also the results confirm that customisation is definitely required to tune the capabilities of the systems to the requirements of specific test cases. Customised systems are used more satisfactory in practice. Chapter 5 also identifies topics for further research that were identified during the project, they are: o Defining map specifications as constraints, or more generally how to express the user specifications into a format understandable by a generalisation system (constraints may not be the optimal way to describe all generalisation problems). 16 o Formalising and evaluating preservation specifications, i.e. better understand the concepts involved (e.g. shape, urban area) and how to mathematically describe both the concepts and the accepted modifications. o Constraint-based generalisation, i.e. how to address a notion of flexibility in automated generalisation and how to aggregate constraint-by-constraint assessments. o Improving the constraints, i.e. complete the list as result of the project, better formalise the constraints and refine them to better address cartographic conflicts. o Evaluating generalisation software on criteria beyond constraints, such as studying whether the software preserves topology and links between initial and output data, whether it contains parameterisation possibilities and how these function, how the software perform on generalisation characteristics that do not fit in constraints, what the scalability/performance is of the software and, most importantly, studying customisation possibilities. 17  3UHVHQWDWLRQ RI WKH SURMHFW 1.1 Introduction Research in automated map generalisation has yielded many promising results (Mackaness et al., 2007). At the same time, vendors face difficulties in implementing automated generalisation solutions in commercial software (Stoter, 2005), which occurs for several reasons. First, a formal definition of map specifications is lacking. Although a satisfying generalisation solution can be defined in general terms—e.g., as a map that reduces the details and discerns regional patterns, that is aesthetically pleasant, and enables users to succeed in a given task (Mackaness and Ruas, 2007)—it is difficult to specify specifications into such a format and knowledge level in such a way that they can steer the automated generalisation process. Second, software vendors need map specifications that are shared by several map producers such as National Mapping Agencies (NMAs) to justify their investments. Such shared generalisation specifications are not easy to formulate because of differences in data models, level of detail of initial data, landscapes to be mapped, scales to be produced, etc. A final reason for the difficult implementation of automated map generalisation is that generalisation process has a subjective part in which more than one ideal generalisation result is often possible. This subjectivity in solving cartographic conflicts cannot be automated. To address these difficulties, we conducted a study on the state of the art of automated map generalisation in commercial software. Moreover, through the study we aimed to learn more about generic and specific map specifications of NMAs, to encourage and support vendors in implementing these specifications in commercial software, and to identify areas for further research. The two main questions of our study were: What are the capabilities and limitations of commercial software systems for automated generalisation with respect to NMA specifications? What different generalisation solutions can be generated for one test case and why do they differ? The study took place in the framework of EuroSDR (European Spatial Data Research), where NMAs, research institutes, and private industry work together on research topics of common interests. Four software vendors have participated in the project. They are: ESRI Inc (USA), University of Hannover (Germany), 1Spatial (United Kingdom) and Axes Systems (Switzerland). ESRI includes all the generalisation functionalities in the software $UF*,6, the University of Hannover provided three generalisation modules: Change, Push and Typify (&37), 5DGLXV &ODULW\ is the generalisation software from 1Spatial and Axes Systems provided their generalisation tools in the software D[SDQG which turned into a new system shortly after the tests began in 2007. The tests were performed on versions that were commercially available in June 2007. However the vendors were invited to perform parallel tests on newer or customised versions of their software to show the full potentials, as will be described later on. Because this is the first research that evaluates specific aspects of output maps generalised by different systems and different testers, taking into account the differing map requirements of several NMAs, an important research aspect was the applied methodology itself; how to set up a case study for studying the-state-of-the-art of commercial generalisation systems; how to specify both generic and NMA specific requirements for automated generalisation; how do automated generalisation processes work; how to perform evaluation of generalisation output; how does the constraint approach, as adopted in this research, work in practice and what further research is needed in this area? The research started in November 2006 and this is the final report of the research. It describes the methodology applied for the requirement analysis, for the testing and for the evaluation, as well as the final results, information provided by the vendors, conclusions and further research. The report is structured as follows. 18 In this first chapter, previous research on map specifications for automated map generalisation is described (Section 1.2) and the scope of the research is outlined (Section 1.3). Also the project set up is described in Section 1.4. 2 presents the methodology which consisted of four parts: requirement analysis (Section 2.1), testing (Section 2.2), evaluation of systems, test processes and constraint expressions (Section 2.3) and evaluation of the generalised outputs (Section 2.4). The evaluation of the generalised outputs (the most important part of the project) consisted of an integrated approach of automated-constraint based evaluation, an evaluation that compares different outputs for one test case and an expert evaluation. 3 presents the test outputs (Section 3.1). It then continues presenting the evaluation of the capabilities of the systems (Section 3.2), the evaluation of the test processes (Section 3.3) and the evaluation on how the constraints were expressed according to the testers (Section 3.4). The evaluation of the generalised outputs is then presented. The automated constraint-based evaluation is presented in Section 3.5, the comparison evaluation is presented in Section 3.6 and the expert evaluation is presented in Section 3.7. Chapter 5 describes the vendors’ tests. Finally Chapter 4 answers the research questions and presents conclusions and recommendations for further research. 1.2 Previous research related to map specifications for automated map generalisation An overview of previous studies on formalising map knowledge for automated generalisation can be found in Sarjakoski (2007). Various researchers have studied specifications for automated map generalisation Foerster et al. (2009). Müller and Mouwes (1990) examined existing map series to conclude that “superficial” generalisation knowledge exists in the form of map specifications written down for interactive generalisation. Complementary to this “superficial” knowledge, cartographers use “deep” generalisation knowledge to interpret superficial knowledge. This deep knowledge is much harder to automate. Rieger and Coulson (1993) carried out a survey among a group of cartographers performing interactive generalisation and concluded that a common view on the classification of generalisation operators does not exist. Nickerson (1991) and Kilpelaïnen (2000) acquired knowledge from experts to define rules for knowledge-based map generalisation. Various studies used reverse engineering to collect generalisation knowledge by comparing map objects across scales (Buttenfield (1991); Leitner and Buttenfield (1995); Weibel (1995)). Other studies describe methods to generate rules from interactive generalisation carried out by a cartographic expert (Weibel (1991); Weibel et al. (1995); McMaster (1995); Reichenbacher (1995)). Several studies applied machine learning techniques to convert expert knowledge into map specifications for automated generalisation, e.g., Weibel et al. (1995), Plazanet et al. (1998), Mustiere (2001; 2005) and Hubert and Ruas (2003). Brewer and Buttenfield (2007) ran map exercises with students, on different datasets at various scales, to provide guidelines for generalisation processes. Our study builds primarily on the research by Ruas (2001), which took place within the European Organization for Experimental Photogrammetric Research (OEEPE; the predecessor of EuroSDR) and investigated the state of the art of generalisation by evaluating different interactive generalisation software. Ruas’s study aimed to obtain insight into generalisation processes for cartographic purposes—not to evaluate generalisation systems or complete generalised output. The OEEPE study tested five platforms on three generalisation cases for a selection of themes. Generalisation operators on individual objects or groups of objects were triggered by testers’ interaction. Because of a lack of written specifications, the target maps served as examples. Templates developed for the project included lists of cartographic conflicts, operations, and algorithms. Several of Ruas’s recommendations (derived from the previous OEEPE study) are relevant for the methodology applied in our project. First, a formalised description of specifications for the output maps should help to obtain better solutions. Furthermore, tests should be evaluated by a more flexible and digital method, since the manual tracing of all testers’ output in Ruas’s study was extremely labor-intensive. Finally, tests should use symbolisation information to standardize the outputs. In our study we have implemented all of these recommendations. 19 1.3 Scope of the current study Several aspects defined the scope of the study. First, the aim of the study was to obtain knowledge on different aspects of automated map generalisation with respect to NMA specifications, and to discover how these are implemented in commercial software. Therefore the project did not rank the systems on their capabilities. Second, our study focused on map specifications of NMAs and did not consider requirements of map end-users. However the results are highly relevant for other mapping applications that may benefit from automated generalisation such as web mapping. Third, our study focused on large- to mid-scale generalisation, since the involved NMAs considered this the most time-consuming generalisation task of current production lines. Fourth, the generalisation processes in our study should not contain any interactive selection of the objects that needed to be processed. The tester is only allowed to setup workflows that will then be applied to an object class (theme) or a spatially indicated area (partition). A final focus of the study was to limit the tests to commercially available versions of software to allow us to conclude on generalities. Consequently, research team testers, either experienced or inexperienced with the systems, were not allowed to customise the software nor to program new algorithms nor to edit results in any part of the process. This did not mean that the implementation of specifications was straightforward: all tested systems—ArcGIS (ESRI), axpand/Genesys (Axes Systems), Change, Push, Typify (University of Hannover) and Radius Clarity (1Spatial) —provide considerable flexibility to deal with the specifications. Consequently, many decisions on how to express the specifications were left to the testers. In some systems testers had to decide on the order of addressing the specifications; in other systems they had to decide which algorithms and parameters values to use. Therefore, all tests required considerable effort to align the functionality of the systems with specific test cases. In addition the project team testers did not receive any product training for any of the systems. One should be aware that this may not fully reflect how the software systems are used in practice: most systems are introduced at new customizers with introductory trainings (for additional costs). Therefore the documentations (i.e. manuals) are often meant as reference material where as the testers in this project relied on the documentation to support their familiarisation with the products. This might have minor impact on the results, as will be discussed further on in the report. To enable vendors to show all the potentials of their system, they performed parallel tests in which they were allowed to customise and develop new algorithms. The results of the vendors’ tests are reported in Chapter 4. 1.4 Project set up The project team carrying out this three years project, consisted of experts in several areas. Some members of the project team contributed from the start (i.e. attended the initial meeting in Enschede, October 2006) till the end (i.e. writing this report). Other members were temporally dedicated to the project, mostly because they moved to other jobs during the project. All project team members are introduced on the title page of this report. The budget of the project was about € 5000. Consequently all time dedicated to the project as well as travel money was compensated by the participating organisations. Besides the essential input of the project team members, the project gained largely by the vendors’ participations. The vendors provided free licenses of their generalisation software and supported testers during the tests. In addition they performed parallel tests (see Chapter 4). Also two meetings attended by both the project team members and the vendors were organised to discuss (intermediate) results: in November 2007 and in September 2009.The remainder of this report describes the results of the team’s activities during the project. 20  0HWKRGRORJ\ This chapter presents the methodology applied in the research, consisting of: requirement analysis (Section 2.1), testing (Section 2.2), evaluation of system capabilities, test processes and constraint expressions (Section 2.3) and evaluation of the generalised outputs themselves (Section 2.4). 2.1 Requirement analysis The requirement analysis consisted of four steps: o Selection of test cases representative for typical generalisation problems o Formalisation of NMA map specifications for automated generalisation. o Harmonisation of the specifications resulting in one generic set of NMA map specifications within the context of our study. o Analyses of the defined specifications to learn more about similarities and differences between map specifications of NMAs. The results of these steps, which were obtained between October 2006 and June 2007, are reported in this section. 2.1.1 Selecting the test cases The first step in the requirement analysis was the selection of test cases representing problems for automated map generalisation. To meet this objective, we generated a list of outstanding map generalisation problems based on the OEEPE research completed with the research team’s own experience. Examples of these problems are building generalisation in urban zones, mountain road generalisation, solving overlapping conflicts in locally dense networks, pruning of artificial networks, and ensuring consistency between themes in particular areas such as coastal zones. Some of these problems have been tackled in research, resulting in at least partial solutions. However, we wanted to evaluate complete solutions in commercial systems, and, therefore, these problems were also identified as representative map generalisation problems. We selected four test cases that included all these problems (see Table 1) provided by Ordnance Survey Great Britain (OSGB), Institut Géographique National, France (IGNf), The Netherlands’ Kadaster (Kadaster) and Institut Cartogràfic de Catalunya (ICC) . Areatype Sourcedataset Targetdataset Providedby No.offeature classes Mainfeature classes Urban area 1:1250 1:25k OS Great Britain 37 buildings, roads, river, relief Mountainous area 1:10k 1:50k IGN France 23 village, river, land use Rural area 1:10k 1:50k Kadaster, NL 29 small town, land use, planar partition Coastal area 1:25k 1:50k ICC Catalonia 74 village, land use (not mosaic), hydrography Table 1: Test cases selected for the EuroSDR research. 21 The NMAs modified their test datasets to prepare them as input for the generalisation tests, e.g., details such as rich classifications were removed from the datasets and the datasets were translated into English. In addition, to be able to define specifications of the output maps with respect to symbolised objects and to assure uniform outputs, the NMAs defined symbols for the outputs. Figure 1 shows samples of the source datasets. The complete input maps and symbols descriptions of the output data are added in Appendix I respectively Appendix II (only digitally available). It is important to note that these inputs have been modified for the project and therefore they differ with the original datasets and symbols as used in production. ICC source dataset, 1:25k IGN France source dataset, 1:10K Kadaster source dataset, 1:10k OS GB source dataset, 1:1250 Ordnance Survey © Crown Copyright. All rights reserved. Figure 1 Samples of source datasets in the EuroSDR generalisation study. Maps are reduced in size. 22 2.1.2 Formalisation of NMA map specifications for automated generalisation In the task of formalising map specifications for automated generalisation, we can distinguish between two stages. The first stage is to describe the specifications in a way that the users (in our case the testers of the systems) fully understand what they should try to obtain with the system. The second stage is to translate these specifications in a format understandable by the generalisation system. The first stage was completed by means of cycles between the data providers and the research team. The second stage was completed by the testers during the test process. To implement research theories, we defined NMA map specifications as a set of cartographic constraints to be respected. In previous research on generalisation, the use of constraints is a common method to define specifications and to control and evaluate the automated generalisation process. Examples are McMaster and Shea (1988), Beard (1991), Ruas (1999), Bard (2004), Barrault et al. (2001), Ware et al. (2003), Burghardt and Neun (2006), and Sester (2000). Constraints express how generalisation output should look without addressing the way this result should be achieved, e.g., by defining sequences of operations. We developed a template for a uniform way to define constraints in the four test cases. In the template specific properties of the constraint can be defined such as condition to be respected and the geometry type and feature class(es) to which the constraint applies (see Appendices III, IV and V and Table 3). The template distinguishes between constraints on one object, on two objects, and on groups of objects. An importance value indicates the importance of satisfying the specific constraint in the final output. This value does not indicate in what sequence the constraints should be solved (Ruas, 1999). Satisfying less important constraints first may be necessary to satisfy more important constraints later. For example, generalisation of buildings should start with reducing density before trying to cope with overlaps, even though non-overlapping constraints are more important than density constraints. NMAs could also propose an action to support the tester in finding the most desired generalisation solution. This is because in some cases NMAs know what action should be taken to meet the constraint optimally, e.g., the constraint “minimal depth of protrusion of a building” can be solved by the two actions “exaggerate detail” or “eliminate detail” which will provide very different results. 2.1.3 Harmonising NMA map specifications for automated generalisation NMAs defined their map specifications for automated generalisation in the developed template by analyzing text-based map specifications, mapping applications and cartographers’ knowledge. Initially a large number of constraints were defined for the four test cases (about 250), which often covered similar situations. In the next step we harmonised the constraints. The aim was to identify constraints which are similar in the specifications provided by different NMAs, and replace them with a single one that can be tuned. This was needed for two reasons. Firstly, to simplify the tests; once a tester had expressed the constraint for one test case, (s)he could perform the same actions to express a similar constraint for a second test case. Secondly, harmonisation enabled us to compare results for similar constraints across the test cases. For the harmonisation, similar constraints across the four test cases were identified by carefully comparing the four constraint sets. The harmonisation resulted in a list of generic constraints. A few constraints were so specific that they remained as a specific constraint. Examples are OSGB constraints addressing how buildings should be aggregated depending on the initial pattern. The harmonisation process resulted in 45 generic constraints: 21 generic constraints on one object (see Appendix III), 11 constraints on two objects (see Appendix IV), and 13 constraints on a group of objects (see Appendix V). The harmonised constraints describe those properties of the constraints that are generically applicable. These constraints contain blank entries to be completed by NMAs to define their constraints as specification of the generic constraints. The columns in the harmonised set (e.g. Class, Action, Importance) only contain values when the value is applicable for any case, except for 23 the column ‘Condition to be respected’ which is always filled, mostly with non-specified parameter values. In all other cases NMAs can specify their classes, actions and importance values to define their constraints as specification of the generic constraints. Table 2 shows examples of generic constraints on one object, two objects, and a group of objects (the constraint type will be introduced in the next section). Constraint type Property Condition to be respected Constraints on one object Minimal dimension Area target area > x map mm2 ; target area = initial area ± x % Width of any part target width > x map mm Area of protrusion/recess target area > x map mm2 Length of an edge/line target length > x map mm Shape General shape target shape should be similar to initial shape Squareness [initial value of angle = 90° (tolerance = ± x°)] target angles = 90° Elongation target elongation = initial elongation ± x % Topology Self-intersection [initially, no self-intersection] no selfintersection must be created Coalescence coalescence must be avoided Position/Orientation General orientation target orientation = initial orientation ± x % Positional accuracy target absolute position = initial absolute position ± x map mm Constraints on two objects Minimal dimensions Minimal distance target distance > x map mm Topology Connectivity [initially connected] target connectivity = initial connectivity Position Relative position target relative position = initial relative position Constraints on a group of objects Shape Alignment initial alignment should be kept Distribution & Statistics Distribution of characteristics target distribution should be similar to initial distribution Density of buildings (black/white) target density should be equal to initial density ± x % Table 2 Examples of harmonised constraints After all four NMAs agreed on the harmonised constraints, they redefined their initial constraints using the generic filled up using their own feature classes, thresholds, parameter values, and preferred actions, see Table 3 for an example of ICC (all NMA specific information is indicated in bold, italic). The NMA specific constraints defined for this research are added as Appendix VI (only digitally available). It should be noted again that these constraints do resemble the NMAs’ requirements, but 24 they have been altered for the project. Consequently they are not the true map specifications of the concerning NMAs. Item in constraint template Example on one object Example on two objects Example on group of objects Constraint ID ICC-1-22 ICC-2-21 ICC-3-18 Geometry type polygon polygon – line polygons Feature class 1 quay_adjacent_to_sea building building Condition for object being concerned with this constraint depth of protrusion > 1 map mm Distance between building and road < 0.5 map mm Constrained property width of protrusion/recess orientation density of buildings (black/white ratio) Condition depends on initial value? no yes yes Condition to be respected target width > 0.2 map mm building must be parallel to road target density should be equal to initial density ± 20 % Action collapse to a line Importance of constraint (1 to 5, 1 is less important) 3 3 3 Exception Schema to illustrate if needed Additional for constraints on two objects: Feature class 2 road Condition for both objects being concerned with this constraint objects are parallel (± 15°) Additional for constraints on group of objects: Kind of group urban block Kind of objects of the initial data composing the group buildings surrounded by minimal cycle of roads (in urban areas) Table 3 Example of ICC map specifications defined as constraints that extend the EuroSDR harmonised constraints. 25 2.1.4 Analysing the test cases To obtain more in-depth knowledge on NMA requirements for automated map generalisation, the final step of the requirement analysis was the comparison of constraints across the four test cases. For this comparison, one should realise that the constraint sets do not reflect all generalisation problems of NMAs. Firstly the NMAs had to limit their constraints to those describing the main problems in the test area and to constraints that were more or less straightforward to formalise. Secondly the constraints were defined without running any automated generalisation process which would have shown both missing and unclear constraints. Lastly the amount of time allocated to the testers would never enable them to set up the equivalent of a complete generalisation production line, handling all requirements for one given map scale and therefore NMAs limited their efforts on constraints that could be tackled within the context of the tests. For the comparison of constraints among the four test cases we used three criteria: 1) the number of objects taken into account in the constraints, 2) the type of the constraints, and 3) the feature class for which the constraints were defined. For the constraint type we distinguished between two main categories: legibility constraints and preservation constraints (Burghardt et al., 2007). Preservation constraints are completely satisfied at scale transitions. These are constraints prescribing preservation of topology, position, orientation, shape, and distribution/statistics. Preservation constraints may be violated when operations are applied for ensuring legibility (minimal dimensions and granularity). Legibility can be investigated independently of the source dataset, while preservation always has to be evaluated in correlation with the source data. Besides legibility and preservation constraints, we identified ‘model generalisation’ constraints. These refer mainly to constraints for removing certain feature types from the data (e.g. ‘cycle path’ in Kadaster test case or ‘wall’ in ICC test case), and also to avoid that objects with different attributes are aggregated, for example different types of buildings in OSGB test case should not be aggregated. Test case Totalnumberofconstraints Number of objects Constraint type Feature classes involvedMo del Leg- ibility Preservation ononeobject ontwoobjects ongroupofobjects Modelgeneralisation Min.dimensionand granularity Position Orientation Shape Topology Distribution/Statistics Other Building Landuse Road Water Relief Coastalfeatures Any Other ICC 137 86 23 28 12 80 0 4 19 12 5 5 39 20 16 25 8 19 9 1 Kadaster 52 27 21 4 11 18 1 0 1 6 0 15 10 13 23 3 0 0 0 3 IGNF 61 32 15 14 2 15 2 4 15 12 2 9 33 2 12 9 2 0 2 1 OSGB 49 24 13 12 2 16 1 0 0 8 0 22 24 1 8 1 8 0 2 5 Total 299 169 72 58 16 129 4 8 35 38 7 51 106 36 59 38 18 19 13 10 Table 4 Analysis of constraints of the test cases, classified on various criteria Table 4 shows the results of comparing the four constraint sets using the three criteria. Several conclusions can be drawn from this table. First, the ICC test case contains a large number of constraints compared to the other cases. This can be explained by the large number of feature classes (see Table 1) resulting in several similar constraints for different types of roads. Secondly, most 26 constraints are defined for one object in all four cases, whereas the fewest constraints are defined for groups of objects, most likely because it was difficult to formalise constraints on groups of objects. Finally, constraints for ensuring minimal dimensions are important in all four test cases, most probably because it is straightforward to define this type of constraints. Another observation is that topological constraints are defined on a more general level such as “preserve topological consistency and connectivity,” “self-intersection not allowed,” or “keep adjacency.” It is notable that there are only a few shape constraints defined by Kadaster. Position and orientation constraints are sparsely specified by all NMAs, and they refer only to buildings. Besides that these constraints are easiest to define for buildings, this is because cartographic conflicts on buildings are more evident in the results, and that buildings are expected to be displaced more often than other objects during the generalisation process. A final conclusion of this analysis concerns the feature classes that were included in the constraint definitions. All four test cases contain many constraints on buildings, land use, and roads. The reason for the importance of these classes in the constraint sets is most likely because these are the most frequently occurring objects and the most significant for users of the map and therefore most (interactive) generalisation is applied to these objects. The variation of constraints among other feature classes is a result of the relative importance of certain feature classes within the four chosen test cases; e.g., constraints on coastal features are dominant in the ICC case. 2.2 The test process The tests were performed from June 2007 till March 2008 by generalisation experts on the commercial version of the software systems available in June 2007. In November 2007, first results were discussed within the project team as well as with the vendors. During this meeting it was realised that it would be beneficial if vendors would submit improved versions of their software to be tested. The main drive for such an extension of the project was that the availability of map requirements and the first test experiences might help vendors to improve their systems. Vendors were invited to submit a new version of their software by 31st of March 2008. Some vendors showed high interest in submitting a new version of their system, but in the end, none of the vendors decided to go ahead and to submit a new version in March 2008. In the tests performed by project team testers no customisation of the software was allowed nor was it allowed to develop new algorithms or to edit results afterwards. Every system was tested two to three times on all four data sets. Every system was tested both by testers who were skilled and testers who were unskilled with the system. In every test, the tester tried to translate all defined constraints into a form understandable by the specific software (the second stage of defining map requirements as indicated in the previous section). The generalisation process must either be triggered by a class of objects (theme) or by spatially indicated areas (partitions), i.e. the tester was not allowed to trigger operations on an object by object basis as in the former OEEPE research. Several templates were designed to capture all testers’ information in a structured and consistent way enabling a flexible method for evaluation. For every test case, the following information was produced by the testers: o Processing template: a file which lists every action of the tester and the amount of time that the action took. o Constraint expression template: a file describing how the tester implemented every constraint (fully/partially/not; how was the constraint expressed; how was the constraint handled). o all output layers in ESRI Shape format. o pdf-file of the output map. Apart from the per-test information, testers provided information on the functionalities and performance per system in a system template. 27 Although new software versions were not available for testers, the vendors were allowed to use them in the parallel tests, and also to apply any customisation specially designed for the tests, in order to show the full potential of the systems. The vendors’ tests are reported in Chapter 4. 2.3 Evaluation of system capabilities, test processes and constraint expressions The evaluation of the results was done from September 2008 to spring 2009 and consisted mainly of evaluating the generalised outputs (see next section). On top of this, the templates completed by the testers with additional information on the tests were evaluated to better answer the research questions. The completed templates considered in this evaluation were the system templates (supplemented with other information such as collected from available manuals), the processing templates, and the constraint expression templates. Results of this evaluation are reported in Section 3.2 till 3.4. 2.4 Evaluation of generalised outputs Evaluating the generalised outputs was the main part of evaluating generalisation in commercial software. Evaluating generalised data can serve three main tasks: evaluation for tuning the generalisation system prior to generalisation, evaluation for controlling the generalisation process during generalisation, and evaluation for assessing the quality of generalised data after generalisation (Mackaness and Ruas, 2007). The purpose of evaluating generalised data in our study falls in the last category. However, the evaluation serves a second, more specific aim, which is learning more about generalisation processes. The methodology that we developed to evaluate the generalised outputs of the tests was driven by an observation by Mackaness and Ruas (2007). They stated that an adequate evaluation framework should be able to handle the notion that the final output is a compromise between a set of sometimes competing map objectives. Such a framework should combine human evaluation and machine evaluation to meet the complexity of evaluation; e.g., machine evaluation can direct the user to those parts of the solution that are deemed to be unsatisfactory. Based on this observation and motivated by the constraint-based approach of the requirement analysis of our study, we developed three integrated methods for evaluating the generalised data: qualitative evaluation by cartographic experts automated constraint-based evaluation evaluation, which visually compared different outputs for one test case The integration was accomplished by directing experts on situations that were well, badly, or differently solved according to the automated constraint-based evaluation. In addition, the results of the visual comparison of outputs were discussed with the experts of the test cases. Conclusions of one method are also compared with results of the other two methods to identify inconsistent measuring tools. All 34 outputs produced by the tests were evaluated. These were 27 outputs delivered by research team testers and 7 outputs delivered by vendors. The three evaluation methods are explained in the remainder of this section. More details can be found in Burghardt et al. (2008). Results of the constraint-based evaluation, the comparison evaluation and the expert evaluation are presented in respectively Section 3.5, Section 3.6 and Section 3.7. 28 2.4.1 Automated constraint-based evaluation The automated constraint-based evaluation compared the measured final value (e.g. ‘size’) for a constraint with an ideal final value. For this evaluation an OpenJump prototype (OpenJump, 2008) was developed (see Figure 2). This prototype implemented the automated evaluation of two legibility constraints: ‘target area > x map mm2’ (for one object) and ‘target distance > x map mm’ (between two objects). The outcome of these evaluations is either 0 (perfect solution) or 1 (violated constraint). Although the implementation of automated evaluation of these two constraints was more or less straightforward, the implementation for most other constraints appeared to be difficult and was therefore not realised. The reason for this is that the definition of constraints mainly aimed at being unambiguously clear for testers. Therefore we did not endeavour to make them as formal as possible. Although for some constraints (e.g. shape and spatial distribution) it is known that the definition and the measurement is complex, a higher level of formalisation could have been achieved. A constraint such as “initial and generalised shape should be similar” is less formal than the constraint “preserving width-length ratio”. For this reason specifically the constraints defined for group of objects appeared to be very difficult (if not impossible) to evaluate in an automated manner, examples are constraints on networks, patterns and spatial distributions. Experiences with the prototype provided important insights for this evaluation method, for example the suitability of the method to identify the overall quality of a generalised map. These insights were obtained by applying the prototype to interactively generalised data, see Section 3.5.1. However, it is important to realise that the results obtained with this evaluation only have low impact on the overall results of the project. This is because most of the time used to work on this evaluation was put towards developing the prototype (due to the faced complexities). Consequently only a limited number of constraints could be assessed. Results of the automated constraint-based evaluation, i.e. results of applying the prototype to interactively generalised data and results of three legibility constraints are reported in Section 3.5.1 respectively Section 3.5.2. Figure 2 Screen shot of prototype for automated constraint-based evaluation 2.4.2 Comparing outputs The Comparative evaluation compares all the outputs obtained for a same test case to o study how different the generalisation results for one test case can be, 29 o study how differently the outputs respect the map specifications, and o understand the noticed differences and the reasons why some map specifications are not met. 9LVXDOO\ FRPSDULQJ IRFXV ]RQHV The main work performed in the comparative evaluation was a careful visual comparative assessment of the outputs. Because it was impossible to assess the complete outputs with the resources available for the test, the following methodology was used. First, a limited set of “focus zones” were identified (approximately 4 for each test case), on which the visual comparison would take place. A focus zone consists of one or several spatial extracts of the dataset and a particular known generalisation problem that occurs on these spatial extracts. An example focus zone is three different road bend series, where the studied problem is “mountain road generalisation”. The focus zones have been selected after the testing stage according to the following criteria. The zones: o cover classical generalisation problems, o take into account the feedback of some testers regarding the interesting generalisation problems they had to face while performing their tests. take into account the knowledge of data producers regarding the problems they expected the testers to encounter, and the feedback of the testers regarding the interesting generalisation problems they had to face while performing their tests The focus zones selected for this evaluation are shown in the section presenting the evaluation results (Section 3.6). For each focus zone of each test case, the visual comparative evaluation consisted of several steps: Get familiar with the specifications (expressed as constraints) related to the studied focus zone, i.e. understand what was expected. This was done by studying the constraint set associated to the test case (see Appendix VI, only digitally available). Extract for each output obtained for this test case, the map extract corresponding to the studied focus zone. Visually inspect and compare the collected map extracts. The output data (shapefiles) were also visualised, overlayed and compared when needed. Detect the main similarities and differences between outputs and try to explain them. Again, the map extracts were not always sufficient and the output data were used as well. The first step was to identify which outputs were right or wrong compared to the specifications, for this we went back to the constraint definition template. Then, we tried to explain the noticed differences using the constraint expression templates, where the testers wrote down if (and if not, why) they were able to express the constraints in the tested system. On top of the constraint expression templates (which were a very good source of information because often they were well filled), we also used our own knowledge of the generalisation process and of the tested systems. In addition we had contacts with the testers, with experts of the tested systems and with experts of the test cases to clarify the unclear situations. 4XDQWLI\LQJ GLIIHUHQFHV On top of the careful visual comparative assessment that constitutes the main part of the comparative evaluation, some countings have been performed in each output dataset to measure the number of objects and/or cumulated lengths or areas, on a class by class basis. The aim was to have some objective indicators showing that the obtained results are sometimes very different. However, although the output data schema expected for each test case had been fixed, the actual outputs sent by the testers were very heterogeneous in terms of schema. Putting the performed countings in a form that would be interpretable would have been very time consuming. Because of time constraints, these countings have been exploited within this project in a very limited way, and almost all the conclusions drawn from the Comparative evaluation task rely on the visual comparison 30 work. Some countings were done as part of the automated constraint-based evaluation (Section 3.5). Future work could extend these countings. 2.4.3 Expert evaluation For the expert evaluation, a survey was developed that extends the earlier experts’ survey of the AGENT prototype (AGENT, 2000). Project team members in each of the four NMAs who provided a test case were asked to recruit cartographic experts in their institute to complete the survey. The survey focused both on global indicators and on individual constraints. The global indicators used to assess the outputs are shown in Table 5. Global indicators Level of manual editions required to meet the constraints Deviation from initial (ungeneralised) data Preservation of the geographic characteristics of the test area (urban, mountainous, rural or coastal area) Legibility Seriousness and frequency of main detected errors Number of positive aspects Information reduction (undergeneralisation / overgeneralisation) Table 5 Global indicators used in the expert survey For the assessment of the outputs on individual constraints, it appeared to be impossible to visually assess if a threshold value, as often used in the definition of the constraints, was met. Therefore we summarised the original constraints in a set of constraints that could be visually assessed (see Table 6). Cartographic experts assessed how these derived constraints were solved: either very badly, badly, well or very well. Constraints on one object Constraints on two objects Constraints on group of objects minimal dimensions spatial separation between features (distance) quantity of information (e.g. black/white ration) granularity (amount of detail) relative position (e.g. building should remain at the same side of a road) spatial distribution shape preservation consistencies between themes (e.g. contour line and river) Table 6 Individual constraints used in the expert survey In a next step the experts were asked to rank the software systems based on their assessment of the generalisation outputs produced by the systems. At the end of the survey, experts annotated the output maps with examples of good (g), bad (b) and differently solved generalisation solutions (d) (see Figure 3). 31 Figure 3 Generalisation output of Kadaster test case, annotated by cartographic expert. Apart from the survey, the experts were provided with the following materials related to the test case of their NMA: a map of the input data (see Appendix I); the constraint set of the test case (see Appendix VI); PDF’s of the 6-12 generalised outputs as produced by both Project team testers and vendors (see Appendix VII respectively Appendix VIII). The PDFs were provided so that the experts could zoom in and/or print the maps; output Shape files; a PDF with the focus zones selected for further study in the comparison evaluation (see Section 3.6). The respondents were asked to consider the outputs generated by the same software as one group, and select the best solution of such a group to answer the questions in the survey. This would assure the evaluation of the best available output. If there were, however, considerable differences between the maps produced by the same software system, the respondents were asked to report this. The respondents were asked to firstly evaluate the outputs produced by Project team testers. If the vendors’ output differed (much) from outputs of Project team testers, the respondents were asked to evaluate this separately. The next chapter presents and interprets the results that were obtained by applying the methodology that was described above. 32  5HVXOWV DQG LQWHUSUHWDWLRQ This section presents the results and interpretation of the evaluation phase. Section 3.1 describes the outputs that were obtained from the tests. Section 3.2 evaluates the system capabilities, Section 3.3 analyses the processing templates and Section 3.4 analyses the constraint expression templates. Section 3.5, Section 3.6 and Section 3.7 present results of respectively the automated constraint-based evaluation, the comparison evaluation and the expert evaluation. It should be noted that this whole chapter deals about the versions of the software that were commercially available in June 2007. Improvements that have taken place since 2007 are reported in Chapter 4, based on vendors’ input (Section 4.2). 3.1 Outputs of the tests It was intended to carry out three tests per system and test case. Consequently theoretically 16 outputs could have been produced per test case (4x3 from regular testers and 4 from vendors), resulting in 64 outputs. Because in practice not all the expected tests were carried out due to several reasons, in total 34 outputs were obtained, 27 outputs produced by the project team and 7 outputs produced by the vendors (see Table 7 respectively Table 8 ). The explanation for relatively fewer tests performed with axpand is that the system was provided several months later to the project team than the other three systems. Because of the late provision of axpand software, the tests with axpand were also performed later than the other tests which caused that axpand test results were not considered in all evaluations. The different evaluations in this chapter explicitly mention when axpand results were not considered. System ArcGIS CPT axpand Radius Clarity Test case IGN 1 3 2 1 ICC 2 2 0 2 OSGB 1 3 0 1 Kadaster 2 3 1 3 Table 7 Tests performed by members of the project team System ArcGIS CPT axpand Radius Clarity Test case IGN 0 1 0 1 ICC 0 1 0 0 OSGB 0 1 0 0 Kadaster 1 1 0 1 Table 8 Tests performed by vendors For all performed tests both the Shape output layers and the pdf maps were provided. The first global evaluations of the output maps showed many differences in the symbolisation of the outputs maps, most probably because many testers were involved despite the provided symbol descriptions (they complied in different ways to the provided symbol descriptions). Differences in symbolisation were also caused because different systems were used to obtain the outputs. Symbolisation heavily influences the way a map is perceived. Therefore the symbolised maps were regenerated by one person based on the output shapes in one system and in consultation with the four NMAs who provided the test cases, before the maps were evaluated. 33 All output maps produced by the project team are added as Appendix VII and output maps produced by vendors are added as Appendix VIII (both only digitally available). Besides these maps, the test outputs consisted of all output layers (approximately 700) and the completed processing, system and constraint expression templates. 3.2 Evaluation of the capabilities of the systems This section describes the main characteristics of the tested systems as well as the quality of available operators in all four systems as the testers reported them in the system templates. Section 3.2.1 firstly describes details on the completed system templates. Section 3.2.2 describes per system the generalisation functionalities (including available algorithms). Section 3.2.3 summarises the capabilities of the systems. Finally Section 3.2.4 concludes on the main characteristics of the tested systems. As mentioned before, it is important to realise that this evaluation is fully dedicated to the tested versions of the systems. Any developments since then are reported by the vendors in Chapter 4 (but not tested in the project). It is important to note that some aspects have influenced the analysis of the software system templates and have made it hard to draw unambiguous conclusions. The first one is that the evaluation of one software system, axpand, is not complete because lack of information: only the vendor provided the software system template and only a novice tester provided it, partially filled. As mentioned in Section 3.1, axpand outputs are incomplete because the system was provided several months later to the project team than the other systems. The second aspect that makes it hard to draw unambiguous conclusions one is the poor harmonisation in the criteria to fulfil the templates, especially when a valuation is required. The last aspect is the unavoidable subjectivity in this (qualitative) analysis and the summarisation of the templates, as well as in the elaboration of the conclusions. 3.2.1 Details on the completed systems templates Not all the testers filled the system templates for all the software that they tested. Next tables show the software systems (rows) used by each tester and the datasets (columns) where the software was applied. The cells coloured in light grey indicate the system templates provided by the testers or by the vendors. Not coloured cells indicate templates that were not provided. 7HVWHU  6\VWHP H[SHUWLVH ,&& GDWD ,*1) GDWD 7'. GDWD 26*% GDWD ICC / ArcGIS (novice) yes - yes ICC / CPT (expert) yes yes yes Yes ITC / CPT (novice)  yes yes Yes TDK / ArcGIS (expert) yes yes yes Yes TDK / CPT (novice) yes yes yes Yes Zurich / axpand (novice) - yes - IGNF / Radius Clarity (expert) yes yes yes IGNS / Radius Clarity (novice) - - yes - 34 7HVWHU  6\VWHP H[SHUWLVH ,&& GDWD ,*1) GDWD 7'. GDWD 26*% GDWD OSGB / axpand (novice) - yes yes OSGB / Radius Clarity (expert) yes - yes Yes 9HQGRU  6\VWHP ,&& GDWD ,*1) GDWD 7'. GDWD 26*% GDWD 1Spatial / Radius Clarity - yes yes University of Hannover / CPT yes yes yes Yes ESRI /ArcGIS - - partially Axes Systems / axpand - - - The templates provided by the vendors have not been used because they used newer or customised versions of the software. The template was divided in two parts, the first part aimed to describe the general capabilities of the system (all answers are presented in Appendix IX), and the second part aimed at evaluating the quality of the available generalisation operators. In the analysis of the second part of the template, next criteria have been taken into account: - To evaluate the operators quality, the following values were possible: +++ very good ++ good o applicable - weak / not available - Because some unexpected values were found in the templates, the following criteria have been applied: - “+” was replaced by ” ++” - “?” has not been taken into account. - “n.a.” was replaced by “/” - “=/=” was replaced by the value of the previous row - Nothing was taking into account when the name of the algorithm was indicated without giving any value - In some evaluations, some part of the questions related to the quality of the operators is not fully answered by the testers. - Other criteria applied in the analysis of the templates have been: - When the testers provide more than a value, only the highest one has been taken into account. - When nothing or a value was assigned and the operator is not available, the value “/” has been considered. - When an operator is not applicable on a type of object (for example, Smoothing on Isolated points), nothing has been considered. - When the operator is available and the evaluation value is “/”, it has not been taken into account. - In the summarised template, the value corresponds to the highest value closest to the average. 3.2.2 Generalisation functionalities of the systems This section describes, for each software system, the main generalisation functionalities of the versions used in the project (as documented in the reference guides and in the web pages of the systems), a summary of the system templates provided by the testers, and a review of the main 35 positive aspects and the limitations of the current versions obtained from the previous information. Appendix IX contains all completed system templates $UF*,6 Main generalisation functionalities ArcGIS is a complete GIS platform provided by ESRI. Although ArcGIS 9.3 is not specifically developed for map generalisation, it does contain tools for automated generalisation of lines and polygons. In addition, it provides a set of tools for raster data generalisation, which were not tested because the project only focused on vector data. The following table lists the tools available in the Generalisation toolset for vector data and provides a brief description for each one: 7RRO 'HVF ULSWLRQ Simplify Line Simplifies a line by removing small fluctuations or extraneous bends from it while preserving its essential shape. Collapse Dual Lines To Centerline Derives centerlines from dual-line features, such as road casings, based on specified width tolerances. Dissolve Aggregates features based on a specified attribute or attributes.. Eliminate Merges the selected polygons with neighboring polygons that share the largest border or have the largest area. Simplify Building Simplifies the boundary or footprint of building polygons while maintaining their essential shape and size. Aggregate Polygons Combines polygons within a specified distance to each other into new polygons. Simplify Polygon Simplifies a polygon by removing small fluctuations or extraneous bends from its boundary while preserving its essential shape. Smooth Line Smooths a line to improve its aesthetic quality. Each generalisation tool includes a graphical user interface, which allows the setup of parameters and input and output files. The ModelBuilder utility allows the creation of workflows based in a sequence of generalisation operations, which can be executed in interactive or off-line mode. The objects are processed in sequential mode, without taking into account the context and the relationships between them. For each tool, the generalisation degree is controlled by the user through parameters: x Simplify line: o Parameters ƒ Algorithm: Point remove (simple algorithm that reduces a line by removing redundant points) or Bend simplify (detect bends along a line, analyze their characteristics, and eliminate insignificant ones). ƒ Tolerance. ƒ Flag or resolve topological errors, possibly introduced in the process. ƒ Keep or not collapsed points. o Considerations ƒ Both algorithms may introduce topological errors, such as line crossings, in the results. 36 x Collapse Dual Lines To Centerline: o Parameters ƒ Maximum width of the dual-line features to derive centerline. ƒ Minimum width of the dual-line features to derive centerline. o Considerations: ƒ Designed for fairly regular, near parallel pairs of lines, such as large-scale road casings. For natural features, such as irregularly shaped double-line rivers, unexpected results can occur. ƒ Centerlines will be derived based on the specified width parameters. A dual-line feature wider than the Maximum Width or narrower than the Minimum Width will not be centerlined. ƒ The output feature class will not carry the attributes from the input lines. x Dissolve: o Parameters: ƒ Field or fields on which to aggregate features. ƒ Choose how to calculate attributes in the output feature class. ƒ Multipart features are allowed or not in the output feature class. x Eliminate: o Parameters: ƒ Method used for eliminating neighboring polygons, length or area. x Simplify building: o Parameters: ƒ Tolerance. ƒ Minimum area for a simplified building to be retained. ƒ Check or not check conflicts as overlapping or touching, among buildings. x Aggregate polygons: o Parameters: ƒ Aggregation distance. ƒ Minimum area for an aggregated polygon to be retained. ƒ Minimum size of a polygon hole to be retained. ƒ Characteristic of the input features that will be preserved when constructing the aggregated boundaries: non orthogonal or orthogonal. o Considerations: ƒ There will be no self-aggregation, meaning that no aggregation within a polygon itself where one area of its boundary is less than the distance to another or between two parts of a multipart polygon. ƒ The output feature class will not contain any attributes from the source features. A one-to-many relationship table will be created that links the aggregated polygons to their source polygons. ƒ The output may contain overlapping polygons, self-crossing boundaries and some connecting zones may be too narrow. x Simplify polygon: o Parameters: ƒ Algorithm: Point remove (simple algorithm that reduces a line by removing redundant points) or Bend simplify (detect bends along a line, analyze their characteristics, and eliminate insignificant ones). ƒ Tolerance. ƒ Minimum area for a simplified polygon to be retained. ƒ No check, flag or resolve topological errors, possibly introduced in the process. ƒ Keep or not collapsed points. 37 o Considerations: ƒ Two output feature classes, a polygon feature class (simplified polygons) and a point feature class (representing polygons that are collapsed to zero- area). ƒ The polygon output will carry all the input fields. The point output will not. ƒ A small polygon near a larger polygon can end up inside the larger polygon due to a relatively large tolerance. x Smooth line: o Parameters: ƒ Algorithm: PAEK (Polynomial Approximation with Exponential Kernel, the smoothed line will not pass through the input line vertices) and Bezier interpolation (fits Bezier curves between vertices). ƒ Tolerance, for PAEK algoritm. ƒ Preservation of the endpoints for closed lines, for PAEK algorithm. ƒ Flag or not check topological errors. o Considerations ƒ Both algorithms may introduce topological errors, such as line crossings, in the results. Summary of the system templates provided by the testers for ArcGIS 1. Interactive generalisation tools 1.1. What tools does the system have for detection and visualisation of cartographic conflicts before and after generalisation? Some operators allow check and flag topological errors generated by them. The flags can be queued using ArcMap GIS tools. Moreover there is a tool that detects graphic conflicts between features taking into account their symbology. 1.2. Does the system support the generalisation of manually selected features? Please describe shortly. Yes. Using GIS tools to select manually the features. 2. Generalisation operators 2.1. Describe if the topology is preserved during generalisation and explain the mechanism, e.g. by topological data model or as part of the operator. Not always. Some operators give the possibility to mark the topologic conflicts. 2.2. Describe if the relationship with terrain is considered during generalisation, e.g. roads – elevation lines. No 2.3. Describe if spatial patterns and relations such as alignments or direct connections (e.g. between building and streets) are manually selected, explicitly modelled and/or preserved during generalisation. No 2.4. Do you support modelling and generalisation for features with 2.5 or 3d geometries? 2.5D 2.5. Can the system call services? Does the system provide generalisation functionality as services? Yes 3. Generalisation process 3.1. Describe the processing strategy applied for automated generalisation, for example batch processing, rule-based or expert-systems, workflow, constraint-based, agent based, …. Parametrized processes. Interactive and off-line processes; there is the possibility to create a workflow (ModelBuilder) to be automatically executed. 38 3.2. How do set up the parameter, e.g. manually or automatically (scale based)? Is it possible to specify the parameter manually? Is it possible to get default values for a given scale? The parameters are indicated manually. 3.3. Does the system support generalisation zones (e.g. settlement area, mountainous area), where specific parameter can be considered during the automated generalisation? a) manually drawn b) automatically detected The system supports generalisation zones manually drawn but not automatically detected. 3.4. Is it possible to select features for generalisation based on spatial or semantic criteria? Please describe. Yes 3.5. Does the system support incremental generalisation for updating? Please describe. No 4. Multi-representation data modelling 4.1. Does the system support the transformation from one data model into another model also called schema transformation. Yes using the Interoperability Extension. 4.2. Is it possible to visualise two models at the same time? Yes 4.3. Does the system store explicit links between feature representations of the same real world object in different scales? Are there m:n-relations supported, for instance if 10 buildings are represented in a smaller scale through 6 buildings? Are there links between different geometry types possible such as area and line features, in case of geometry type change during generalisation, e.g. a river is modelled as area object in one scale and as line object in a smaller scale? Yes 4.4. Does the system support the versioning (of different temporal states) of a feature? Please explain. Yes 4.5. Does the system supports matching of features between different scales? Yes 5. System properties 5.1. Which formats are supported for direct import and export? Does the system allow data import with data transformation software such as FME or CITRA? SHAPE and others. The system allows data import through data transformation softwares as FME. 5.2. Please characterise the graphical user interface for generalisation. Each generalisation tool includes a graphical user interface, which allows the input of parameters and files. ModelBuilder allows the creation of workflow based in a sequence of generalisation operations. 5.3. Which languages are supported from the graphical user interface and the help manuals? Many. 5.4. Does an API exists allowing the customer to develop own generalisation functionalities? Is it possible to write new generalisation algorithms? Yes 6. Map production 6.1. Does the system offer cartographic symbolisation? Yes 6.2. Does the system offer tools for map production? Yes 39 Evaluationofthequalityofthegeneralisationoperators(+++verygood,++good,oapplicable,-weak,/notavailable) Software/Company ArcGIS/ESRI BuildingRoadsRailwayBorderHydrographic network Landuse (mosaic) Relief/ contours Coast lines Isolated points Isolated lines Isolated areas 1.Simplification 1.1Pointreduction(weeding)o++++++++++++++++++ 1.2Unrestrictedsimplification++++++++++++++++++++ 2.Geometrytypechange 2.1Areatoline// 2.2Areatopoint/// 2.3Linetopoint// 3.Enhancement 3.1Scaling// 3.2Exaggeration///// 3.3Smoothing++++++++++++++++ 3.4Rectification/// 3.5Enlargetorectangle// 4.Selection/elimination/ typification 4.1Selection++++++++++++++++++++++ 4.2Typification////// 5.Displacement 5.1Lines//// 5.2withdeformationofpolygon///// 5.3movethecompletepolygon///// 5.4Snapping////// 40 Software/Company ArcGIS/ESRI BuildingRoadsRailwayBorderHydrographic network Landuse (mosaic) Relief/ contours Coast lines Isolated points Isolated lines Isolated areas 6.Aggregation 6.1Amalgamation-Fusion (polygon) ++++++++ 6.2.Amalgamation–Merge (polygon) ++++++++/++ 6.3Combine(pointstopolygon)/// Others–pleasespecify… 41 Advantages and limitations The advantages and limitations regarding generalisation functionalities are as follows: Advantages x Acceptable results for building generalisation, line simplification, area aggregation, collapse dual lines and mosaic generalisation. x Generalisation tools inside a complete GIS platform. x Workflow creation tool (Model Builder). x Possibility to customise existing algorithms or to add new algorithms. x 2.5D data management. x Good documentation available. x Easy installation. Limitations x Missing operators: enlargement of buildings, simplifying buildings based on width and depth of protrusions, displacement, point generalisation, etc. x Topology is partially managed. Simplify line, Smooth line, Aggregate polygons, Simplify building and Simplify polygons can introduce topological errors. x Collapse Dual Lines To Centerline and Aggregate polygons does not carry the attributes from the input lines. x Objects are processed class by class and in sequential mode, without taking into account the context and the relationships between them. x More detailed information about the generalisation process should be provided. &KDQJH 3XVK DQG 7\SLI\ &37 The software provided by the University of Hannover is composed by three complementary modules, Change, Push and Typify, which are designed to solve specific generalisation problems. Change Main generalisation functionalities Change is the module devoted to building generalisation and it has been designed to process data from 1:1.000 to 1:25.000 scales. Change can only manage objects stored as polygons (closed polyline) which are topologically correct (no duplicated vertices, no self intersections, no intersection between objects, etc). Original attributes are maintained if aggregation is not performed. Change allows the following generalisation operations: x Elimination of small buildings. x Simplification of building outlines. x Aggregation of buildings, followed by the simplification of the aggregated buildings outlines. The objects are processes in sequential mode, without taking into account the context and the relationships between them. The generalisation degree is controlled by the user through a parameter file: x Input and output scale. x Search radius for identical points. x Distance to the straight through the two neighbouring points. x Threshold value for side length of buildings. 42 x Minimal area dimension, including protrusions. x Threshold values for aggregation (distance and combination/shifting). x Threshold value for angles. Change uses its own format but provides a tool to translate from and to Shape format. Advantages and limitations The advantages and limitations of Change are as follows: Advantages x Testers were very satisfied with the results for building aggregation, especially at large scale. Limitations x No distinction for length of edge in a building, width of protrusion, depth of protrusion, or width of a part. x Polygons under minimum area are deleted because the same parameter is used for minimum area and protrusions. No information about removed polygons. x Topological relationships are not maintained: o Inconsistent polygons and overlaying polygons are generated o Polygons with holes are not treated properly x Only one attribute with less than 9 numeric characters is allowed. x A lot of file conversions are needed in the whole process. Push Main generalisation functionalities Push is designed to solve the spatial conflicts generated by symbolisation and preceding generalisation operations by displacements and small deformations. The applied process is based on an optimization process, which is able to solve conflicts in a holistic way. The program does not include other generalisation operations like reduction. It can manage linear elements and polygons. However for polygons with holes only the outer ring of the polygon is processed. Object properties and object behaviour can be parameterised individually, leading to a high flexibility: x Minimal distance that one object class should have to its neighbours. x Control of the possibility to displace or not an object. x Control of the possibility to deform or not an object. x Number of iterations. x General minimum distance between objects, in addition to the individual minimal distance. x Threshold value that objects are attracted and moved towards each other instead of shifting apart (critical distance). x Distance of Steiner points, artificial intermediate points inserted in the Constrainted Delaunay Triangulation. x Individual distance matrix, individual distance between object classes, in addition to minimal distance and general minimum distance. Advantages and limitations The advantages and limitations of Push are as follows: 43 Advantages x Acceptable results for displacement and deformation. x Topology is taken into account: connexions and relative position between elements are maintained. Limitations x Sometimes self-intersections are generated. x If the critical distance to several neighbouring objects falls under the threshold value, a strange behaviour can appear and the object can be strangely deformed. x If not enough space is available, the situation deteriorates because it does not maintain the original state. x Deletion of polygons which intersect with lines because of space. x Only one attribute, which must be named OTN, is maintained and it should have less than 9 numeric characters. Typify Main generalisation functionalities Typify can perform building generalisation for large and medium scales (1:50k and smaller, i.e. less detailed) by evaluating individual buildings and simplifying their shapes based on small facade structures. For smaller scales this would lead to an elimination of many buildings, as all building parts would fall under the required minimal sizes. To address this problem, Typify reduces the number of buildings, arranges them based on a density preserving principle, and collapses them appropriately: small buildings are replaced by square symbols, larger building are presented with original shape. The density preserving reduction of objects is done using Self Organizing Networks, a neural network approach. Typify partitions the whole area in individual meshes, which are typically provided by the road network, although other elements as administrative boundaries can be used. Furthermore, in the course of the arrangement and collapsing, the objects are also displaced against each other based on parameterised distances. After the selection of the elements that will configure the meshes and the creation of the partition, the typification is performed based on the following parameters: x Reduction rate, defined by a percentage. x Target scale, which determine the size of the symbol placed in collapse operation. x Displacement and degree of deformation for buildings. x Displacement and degree of deformation for roads. x Displacement and degree of deformation for other elements. x General minimum distance between objects, in addition to the previous distances. Advantages and limitations The advantages and limitations of Typify are as follows: Advantages x Good results for building typification. x Topology is taken into account, although in some cases self-intersections in buildings are created. 44 Limitations x A problem is that buildings are reduced based on the number of buildings whereas most constraints mention ‘maximum area coverage’ of buildings. x Mesh must have a 100% coverage of test area x The parameter target scale only determines the symbol size and it is based in the size of the building symbol in the German maps at 1:50.000 (25x25 meters). x Only attributes with less than 9 numeric characters are allowed. Summary of the system templates provided by the testers for Change, Push, Typify 1. Interactive generalisation tools 1.1. What tools does the system have for detection and visualisation of cartographic conflicts before and after generalisation? PUSH generates additional information to be used in a GIS to inspect the objects after generalization. There is any tool in CHANGE and TYPYFY. 1.2. Does the system support the generalisation of manually selected features? Please describe shortly. No 2. Generalisation operators 2.1. Describe if the topology is preserved during generalisation and explain the mechanism, e.g. by topological data model or as part of the operator. The topology is not completely preserved in building generalisation (CHANGE): sometimes some self-intersections are created or intersection between neighbours. The topology is preserved in displacement (PUSH): connexions between elements and relative position between elements are maintained. The topology is not always preserved in typification (TYPIFY): sometimes some selfintersection are created in buildings. 2.2. Describe if the relationship with terrain is considered during generalisation, e.g. roads – elevation lines. The relationship with terrain is not considered, although contour lines can be processed by PUSH as linear elements. 2.3. Describe if spatial patterns and relations such as alignments or direct connections (e.g. between building and streets) are manually selected, explicitly modelled and/or preserved during generalisation. Usually the spatial relations are preserved, but not always. 2.4. Do you support modelling and generalisation for features with 2.5 or 3d geometries? No. CHANGE manages only 2D data. PUSH and TYPYFY can manage 2.5D, but the result is 2D. 2.5. Can the system call services? Does the system provide generalisation functionality as services? No 3. Generalisation process 3.1. Describe the processing strategy applied for automated generalisation, for example batch processing, rule-based or expert-systems, workflow, constraint-based, agent based, …. CHANGE: batch process, based on rules of how to handle too small facade elements. TYPIFY: batch process; neuron network approach. PUSH: batch process: global, holistic optimization based on given constraints. 3.2. How do set up the parameter, e.g. manually or automatically (scale based)? Is it possible to specify the parameter manually? Is it possible to get default values for a given scale? 45 The parameters are indicated manually. 3.3. Does the system support generalisation zones (e.g. settlement area, mountainous area), where specific parameter can be considered during the automated generalisation? a) manually drawn b) automatically detected CHANGE and PUSH don’t support any generalisation zones. TYPIFY is based in partitions based on existing elements. 3.4. Is it possible to select features for generalisation based on spatial or semantic criteria? Please describe. CHANGE allows semantic aggregation. In PUSH the selection of features is possible by semantic criteria in the displacement, to move the elements a distance depending of one attribute. 3.5. Does the system support incremental generalisation for updating? Please describe. No 4. Multi-representation data modelling 4.1. Does the system support the transformation from one data model into another model also called schema transformation. No 4.2. Is it possible to visualise two models at the same time? No 4.3. Does the system store explicit links between feature representations of the same real world object in different scales? Are there m:n-relations supported, for instance if 10 buildings are represented in a smaller scale through 6 buildings? Are there links between different geometry types possible such as area and line features, in case of geometry type change during generalisation, e.g. a river is modelled as area object in one scale and as line object in a smaller scale? The system doesn’t store explicit links between feature representations. The system maintains attributes under certain conditions, but it doesn’t calculate them for new elements. 4.4. Does the system support the versioning (of different temporal states) of a feature? Please explain. No 4.5. Does the system supports matching of features between different scales? No 5. System properties 5.1. Which formats are supported for direct import and export? Does the system allow data import with data transformation software such as FME or CITRA? SHAPE. The system doesn’t allow data import through external software. 5.2. Please characterise the graphical user interface for generalisation. The system includes a very simple graphical user interface. It only allows the input of parameters and files. 5.3. Which languages are supported from the graphical user interface and the help manuals? Manuals are in German and English. The graphical user interface is in English. 5.4. Does an API exists allowing the customer to develop own generalisation functionalities? Is it possible to write new generalisation algorithms? No 6. Map production 6.1. Does the system offer cartographic symbolisation? No 6.2. Does the system offer tools for map production? No 46 Evaluationofthequalityofthegeneralisationoperators(+++verygood,++good,oapplicable,-weak,/notavailable) Software/Company CPT(Change,Push, Typify)/UniversityofHannover BuildingRoadsRailwayBorderHydrographic network Landuse (mosaic) Relief/ contours Coast lines Isolated points Isolated lines Isolated areas 1.Simplification 1.1Pointreduction(weeding)+++///////// 1.2Unrestrictedsimplification+++///////// 2.Geometrytypechange 2.1Areatoline//// 2.2Areatopoint++//// 2.3Linetopoint//// 3.Enhancement 3.1Scalingo// 3.2Exaggerationo/////// 3.3Smoothing//////// 3.4Rectification++/// 3.5Enlargetorectangle+++// 4.Selection/elimination/ typification 4.1Selection/////////// 4.2Typification++//////// 5.Displacement 5.1Lines+++++++++++++ 5.2withdeformationofpolygon++/////--++ 5.3movethecompletepolygon++/////--++ 47 Software/Company CPT(Change,Push, Typify)/UniversityofHannover BuildingRoadsRailwayBorderHydrographic network Landuse (mosaic) Relief/ contours Coast lines Isolated points Isolated lines Isolated areas 5.4Snapping-/////--// 6.Aggregation 6.1Amalgamation-Fusion (polygon) +++//// 6.2.Amalgamation–Merge (polygon) +++////// 6.3Combine(pointstopolygon)//// Others–pleasespecify… 48 Advantages and limitations for CPT software The advantages and limitations for the whole CPT software configuration can be summarised as follows: Advantages x Good results for building simplification, aggregation and typification. x Acceptable results for displacement tools. x PUSH and TYPIFY take partially into account the context. x Able to be integrated in a workflow based in most of the commercial systems, because it uses SHAPE format and works in off-line mode. x Good documentation available. x Easy installation. Limitations x Missing operations: selection, linear simplification, exaggeration, typification on linear elements, etc. x Topology is partially managed. x Not all operations take into account the context and the relationships between objects. x Limitations in the attributes management. x It is not possible to customise existing algorithms or to add new algorithms. x The results are 2D. x More detailed information about the generalisation process should be provided, although the processes generate a report with statistics of the generalised data. x GUI is too simple. Workflow should be more user friendly, for example optimizing format translations and minimizing the number of processes to be executed. x Parameterisation is difficult for novice user 5DGLXV &ODULW\ Main generalisation functionalities Radius Clarity is a rule-based environment for automated generalisation. It provides the facilities to build automated generalisation workflows and an environment to develop and research new generalisation algorithms. The tool set in Radius Clarity enables small-scale digital data to be automatically derived from large-scale source data. Its approach to generalisation is based on intelligent software Agents, an advanced artificial intelligence technique that enables the automated map production process to capture and handle contextual information. Radius Clarity is goal driven, because of the following characteristics: x Context Sensitive: Different generalisation algorithms are automatically applied on a feature depending on its spatial context (surrounding features). x Self-Optimising: Radius Clarity has the ability to cycle through possible outcomes, modifying the initial geometries repeatedly until the optimal result is obtained. x Object-Oriented: The object-oriented data modelling means that map features (e.g. houses, roads, rivers) become active objects, providing measures, actions and constraints for automated generalisation. x Adaptive: The constraints and algorithms are easily adapted between different feature types and different scale changes. 49 x Automatic: Includes a framework for running automated generalisation processes and a mechanism to instigate generalisation as a batch process. x Configurable: Complete user control of the data model, the target map specification and the generalisation approach. The generalisation environment requires configuration at several levels: x Map Specifications: Contains the parameters required for generalisation. o Building constraint parameters: ƒ Squaring tolerance ƒ Building minimum size ƒ Elongation factor ƒ Minimum edge ƒ Scale factor o Road constraint parameters: ƒ Legibility factor: Multiplies the symbol width. ƒ Line absorption: Used when one line segment displaces the rest during generalise by parts. ƒ Environment proximity classes: Within a buffer of the road, which is not topologically connected to the road. ƒ Minimum vector length: Length of vector to use when generating curves for a buffer around convex corners. ƒ Road geometric tolerance: Vertices within this distance of each other are merged. o Road network constraint parameters: ƒ Barrault propagation cushioning coefficient: The dampening factor to be used when propagating changes through a road network. ƒ Diffusion minimum distance: If the displacement of the end node of a road is calculated to be less than this distance as a result of the diffusion following moving the start node, then the end node is not diffused and the road absorbs the whole diffusion. o Urban block constraint parameters: ƒ Aggregation distance. x Agents: Objects are made into agents by making them inherit from the base classes, agent_meso or agent_micro. These base classes define a number of methods which carry out the Agent Lifecycle. x Constraints: Constraints govern the behaviour of agents during their lifecycle. The constraints define the methods which evaluate the state of an agent, and provide default implementations. o Road and Road Network constraints: ƒ Road network micros constraint: To calculate a weighted average of the coalescence strength. ƒ Road coalescence: Measures the level of coalescence along a road, and proposes plans to reduce this level. ƒ Road symbol holes: Ensures that new holes are not created. ƒ Road self-intersection: Ensure it does not become self-intersecting. o Building constraints ƒ Building concavity. ƒ Building elongation. ƒ Building granularity. ƒ Building local width. ƒ Building orientation. 50 ƒ Building size. ƒ Building squareness. o Urban block constraints ƒ Aggregate buildings constraint. o Miscellaneous constraints ƒ Meso control subagents constraint. Unconditionally proposes a plan to activate all of its subagents, and accept the result. x Actions: These are functions which modify an agent or cause it to initiate changes in other related agents. o Line Actions ƒ Accordion: Expand a bend series in the direction of its axis - rather like extending an accordion. ƒ Bend removal: Executes the “Bend removal” algorithm to remove two consecutive bends. ƒ Plaster. Reduces coalescence conflict in a series of bends by smoothing. ƒ Minimal break. Expand a single bend with coalescence violation by constructing a Delaunay triangulation “skeleton”. ƒ Maximal break. Expand a single bend with coalescence violation by buffering around the line. ƒ Generalise by parts. Split up a line into smaller, which each have a particular property that is the type of coalescence. o Area Actions ƒ Polygon Squaring. ƒ Polygon Eliminate. ƒ Polygon Enlarge to Rectangle. Replace a building geometry with the Minimum Bounding Rectangle (MBR) for that building scaled to a given size. ƒ Polygon Scale. ƒ Polygon Elongation. Enlarges or reduces a polygon along its axis. ƒ Polygon Simplify to Rectangle. Converts the polygon into a rectangle while maintaining the area and orientation of the original polygon. ƒ Polygon Simplify. Reduces detail keeping the overall shape of the building. ƒ Polygon Relative Simplify. Simplifies a polygon by removing short edges. ƒ Polygon Orientation. Rotates the polygon. ƒ Polygon Local Width. Moves the closest parts of a polygon apart. o Line Meso Actions ƒ Generalise Parts Push Micro. This action constructs a new micro agent. It should only be applied to meso agents and should normally only be called as part of the generalise by parts plan proposed by the coalescence detection constraint. ƒ Generalise Parts Diffuse. This action is normally used as a follow up to the “Generalise Parts Push Micro” action above. It will diffuse the changes to the micro agent through the geometries stored on this meso agent. o Line Network Actions ƒ Push Micro. Pushes the indexed micro agent controlled by this line network meso onto the scheduler stack, with an active lifecycle. ƒ Diffuse. Diffuses the changes made by a previous invocation of “Push Micro” throughout the road network. o Urban Block Actions ƒ Aggregate. Merges overlapping micro-agents in a meso-agent. x Algorithms: Used in the agent constraints and actions. x Agent Scheduler: Acts as a coordinator of agents and their lifecycles. 51 x Process Methods: Several process methods have been implemented in Radius Clarity. Summary of the system templates provided by the testers for Radius Clarity 1. Interactive generalisation tools 1.1. What tools does the system have for detection and visualisation of cartographic conflicts before and after generalisation? Radius Clarity can detect and mark up some conflicts. There are also mark-up navigation tools which can drive the user to marked up objects. 1.2. Does the system support the generalisation of manually selected features? Please describe shortly. Yes. The selection can be manually, through windows display, using a rectangular region or a query. 2. Generalisation operators 2.1. Describe if the topology is preserved during generalisation and explain the mechanism, e.g. by topological data model or as part of the operator. It preserves the network topology and the shared geometries, but other topological relationships (for example relative position) are not completely preserved. 2.2. Describe if the relationship with terrain is considered during generalisation, e.g. roads – elevation lines. No 2.3. Describe if spatial patterns and relations such as alignments or direct connections (e.g. between building and streets) are manually selected, explicitly modelled and/or preserved during generalisation. No 2.4. Do you support modelling and generalisation for features with 2.5 or 3d geometries? No 2.5. Can the system call services? Does the system provide generalisation functionality as services? No 3. Generalisation process 3.1. Describe the processing strategy applied for automated generalisation, for example batch processing, rule-based or expert-systems, workflow, constraint-based, agent based, …. Constraint based and agent based. On-line and batch are allowed. 3.2. How do set up the parameter, e.g. manually or automatically (scale based)? Is it possible to specify the parameter manually? Is it possible to get default values for a given scale? Most of the parameters are introduced manually. Some scale-dependent parameters are set up automatically from the map specifications. 3.3. Does the system support generalisation zones (e.g. settlement area, mountainous area), where specific parameter can be considered during the automated generalisation? a) manually drawn b) automatically detected Yes. It is possible to do partitions automatically and manually. 3.4. Is it possible to select features for generalisation based on spatial or semantic criteria? Please describe. Yes. Selection modes available using semantic criteria are class, attribute value or range of attribute values. Selection modes using spatial criteria are dataset extent, window extend, manually specified extents or using an existing geometry. 3.5. Does the system support incremental generalisation for updating? Please describe. No 52 4. Multi-representation data modelling 4.1. Does the system support the transformation from one data model into another model also called schema transformation. Yes 4.2. Is it possible to visualise two models at the same time? Yes 4.3. Does the system store explicit links between feature representations of the same real world object in different scales? Are there m:n-relations supported, for instance if 10 buildings are represented in a smaller scale through 6 buildings? Are there links between different geometry types possible such as area and line features, in case of geometry type change during generalisation, e.g. a river is modelled as area object in one scale and as line object in a smaller scale? No. But in Radius Radius Clarity, there is a reference between the features in the source data model and the corresponding feature in the target dataset, after the setup procedure. 4.4. Does the system support the versioning (of different temporal states) of a feature? Please explain. No. But in the Radius Radius Clarity Server module there is technology available to support temporal states. 4.5. Does the system supports matching of features between different scales? No 5. System properties 5.1. Which formats are supported for direct import and export? Does the system allow data import with data transformation software such as FME or CITRA? SHAPE and others. FME can be used. 5.2. Please characterise the graphical user interface for generalisation. The interface is a Java based interface in a desktop environment. It contains a Display Window in which the data is displayed and a title bar with several drop down menus for opening forms and dialogs used in generalisation. It is also extensible and therefore has a fully documented API. 5.3. Which languages are supported from the graphical user interface and the help manuals? The default language is English however there is mechanism for translating the strings and messages in the GUI into other languages. 5.4. Does an API exists allowing the customer to develop own generalisation functionalities? Is it possible to write new generalisation algorithms? There is a Radius Clarity API, in Java, C and LULL, which allows the user to develop their own generalisation functionality and to customise the Gothic database. 6. Map production 6.1. Does the system offer cartographic symbolisation? Yes 6.2. Does the system offer tools for map production? No 53 Evaluationofthequalityofthegeneralisationoperators(+++verygood,++good,oapplicable,-weak,/notavailable) Software/CompanyRadius Clarity/1Spatial BuildingRoadsRailwayBorderHydrographic network Landuse (mosaic) Relief/ contours Coast lines Isolated points Isolated lines Isolatedareas 1.Simplification 1.1Pointreduction(weeding)+++++++++++++++++o+++++ 1.2Unrestrictedsimplificationo- 2.Geometrytypechange 2.1Areatoline//// 2.2Areatopoint//// 2.3Linetopoint//// 3.Enhancement 3.1Scaling+++/+++ 3.2Exaggeration+++++/o++o++++ 3.3Smoothing++++++o+++o+++++o 3.4Rectification++//++ 3.5Enlargetorectangle+++/+++ 4.Selection/elimination/ typification 4.1Selection///////// 4.2Typification//////// 5.Displacement 5.1Lines/////// 5.2withdeformationofpolygon--////// 5.3movethecompletepolygon//////// 54 Software/CompanyRadius Clarity/1Spatial BuildingRoadsRailwayBorderHydrographic network Landuse (mosaic) Relief/ contours Coast lines Isolated points Isolated lines Isolatedareas 5.4Snapping 6.Aggregation 6.1Amalgamation-Fusion (polygon) O/// 6.2.Amalgamation–Merge (polygon) O////- 6.3Combine(pointstopolygon)/// Others–pleasespecify… 55 Advantages and limitations The advantages and limitations of Radius Clarity can be summarised as follows: Advantages x Good results in linear simplification and building generalisation. x Rule and agent based system. x Use of optimization techniques. x Several algorithms for road and building generalisation. x Environment to develop and research new generalisation algorithms. x Creation of automated generalisation workflows. Limitations x The following operations are missing: displacement, typification, collapse, etc. x Topology is mainly maintained, but it can not be ensured that all the topological relationships are completely maintained. x Only 2D data. x Requires a basic understanding of object orientation, a good understanding of geospatial data modelling, XML and Java knowledge for customisation. x Not easy installation. x More detailed information about the generalisation process should be provided. D[SDQG Main generalisation functionalities axpand technology is based on a true multi-representation data base (MRDB). Generalisation is treated as a holistic process. It includes algorithms which are applied in succession as an entire process and are steered by constraints. Algorithms are combined in a workflow, which makes it possible to set up a generalisation process. Constraints, stored in files, can be entered, adjusted and maintained easily. The generalisation zones functionality allows the generalisation of different areas of a map using different constraints within the same process. Integrated model transformation functionality makes it possible to generalise from one source model to one or more different target models. Incremental updating of generalised data is supported by this MRDB based generalisation system. axpand provides different levels of automation: using a job list or using sophisticated workflow technology that allows for the generalisation of complex data. Each generalisation tool includes a graphical user interface, which allows the setup of parameters and input and output files. The following list contains the tools available in the Generalisation toolset. For each tool, the generalisation degree is controlled by the user through parameters: x Line displacement: o Functionality parameters: ƒ Suppress orthogonality for intersections. ƒ Stiffness. ƒ Self displacement. o Object dependent parameters: ƒ Priority: Indicates how flexible the line can be. ƒ Displacement effect: Indicates whether the object should have a displacement effects on other objects. 56 ƒ Minimum distance. Describes the desired distance from the object to be displaced. ƒ Network topology: Defines whether the line object should be linked to a topology network. x Line simplification (point reduction): o Functionality parameters: ƒ Horizontal deviation ƒ Maximum segment length o Object dependent parameters: ƒ Line simplification of areas: Indicates if area boundaries should be simplified or not. x Smoothing: o Functionality parameters: ƒ Typification: Indicates if highly curved areas will be typified along. ƒ Smoothing rate. x Area displacement o Functionality parameters: ƒ Minimum distance. ƒ Maximum movement. o Object dependent parameters: ƒ Priority area: Defines the flexibility of the area object. ƒ Displacement effect area: Defines if the objects should have a displacement effect on other objects. x Area simplification: o Functionality parameters: ƒ Distance: Determines the minimum length of the area side. During building simplification edges that are too short will be discarded or stretched to the minimum length. x Area aggregation: o Functionality parameters: ƒ Distance x Area selection: o Functionality parameters: ƒ Minimum area to select objects ƒ Maximum distance used to group selected objects if they are under minimum area. x Scaling: o Functionality parameters: ƒ Scaling factor. Summary of the system templates provided by the testers for axpand 1. Interactive generalisation tools 1.1. What tools does the system have for detection and visualisation of cartographic conflicts before and after generalisation? The system provides a tool, which highlights features with conflicts after an automated generalisation operation. Interactive editing is supported in a way that the user is guided sequentially through all the conflicts generated during the generalisation process. 1.2. Does the system support the generalisation of manually selected features? Please describe shortly. 57 The generalisation operations are applied on all features visible on the screen. A generalisation of manually selected features is not supported. For this system, because generalisation is an integrated process and the relationships between objects is critical, generalisation is carried out by using a workflow which integrates all of the necessary model and cartographic generalisation steps. 2. Generalisation operators 2.1. Describe if the topology is preserved during generalisation and explain the mechanism, e.g. by topological data model or as part of the operator. The system preserves partially the topology. 2.2. Describe if the relationship with terrain is considered during generalisation, e.g. roads – elevation lines. No 2.3. Describe if spatial patterns and relations such as alignments or direct connections (e.g. between building and streets) are manually selected, explicitly modelled and/or preserved during generalisation. The answer of the vendor is that these relations are preserved automatically during generalisation. The answers of the testers are that some generalisation operator considers relations implicit, for example during displacement of buildings against streets. 2.4. Do you support modelling and generalisation for features with 2.5 or 3d geometries? No 2.5. Can the system call services? Does the system provide generalisation functionality as services? Axes Systems has worked on generalisation services, but during the test the service solutions was not ready for productive usage. 3. Generalisation process 3.1. Describe the processing strategy applied for automated generalisation, for example batch processing, rule-based or expert-systems, workflow, constraint-based, agent based, …. Rule based. Batch processing. 3.2. How do set up the parameter, e.g. manually or automatically (scale based)? Is it possible to specify the parameter manually? Is it possible to get default values for a given scale? There are default parameter values given for the generalisation operators, but also a manual specification of parameter is possible. There might be also recommendations for suitable parameter settings for different scale transitions. 3.3. Does the system support generalisation zones (e.g. settlement area, mountainous area), where specific parameter can be considered during the automated generalisation? a) manually drawn b) automatically detected The system does not support generalisation zones and the only way of selection some specific regions is use the zoom functionality; consequently all features visible on the screen will be generalised with the selected generalisation operators and parameter settings. 3.4. Is it possible to select features for generalisation based on spatial or semantic criteria? Please describe. The answer of the vendor is yes. The answers of the testers are that the selection can be carried out only by feature class level. 3.5. Does the system support incremental generalisation for updating? Please describe. No 4. MRDB, data modelling 4.1. Does the system support the transformation from one data model into another model also called schema transformation. 58 Yes 4.2. Is it possible to visualise two models at the same time? It is possible to visualise to models besides each other. But it is not possible to visualise features of different models within the same window, thus no overlay is possibly. 4.3. Does the system store explicit links between feature representations of the same real world object in different scales? Are there m:n-relations supported, for instance if 10 buildings are represented in a smaller scale through 6 buildings? Are there links between different geometry types possible such as area and line features, in case of geometry type change during generalisation, e.g. a river is modelled as area object in one scale and as line object in a smaller scale? No 4.4. Does the system support the versioning (of different temporal states) of a feature? Please explain. The answer of the vendor is that the system includes timestamps and relationships between objects over time. 4.5. Does the system supports matching of features between different scales? No 5. System properties 5.1. Which formats are supported for direct import and export? Does the system allow data import with data transformation software such as FME or CITRA? SHAPE and others. FME can be used. 5.2. Please characterise the graphical user interface for generalisation. The GUI for creating workflows is a graphic representation of the workflow itself. It is suitable and intuitive. 5.3. Which languages are supported from the graphical user interface and the help manuals? English and German. 5.4. Does an API exists allowing the customer to develop own generalisation functionalities? Is it possible to write new generalisation algorithms? No 6. Map production 6.1. Does the system offer cartographic symbolisation? Yes 6.2. Does the system offer tools for map production? Yes The table of evaluation of the quality of the generalisation operators is not included in the report because only the vendor has provided it, and no information has been provided by the testers. Advantages and limitations The advantages and limitations of axpand are as follows: Advantages x MRDB system based that allows incremental updating and generalisation. x Rule based system. x Possibility of workflow creation. x Generalisation tools inside a complete map production system. 59 Limitations x Topology is preserved partially. x Missing operators: collapse, typification, etc. x Only 2D data. x Poor documentation. x Complex installation. x It is not possible to customise existing algorithms or to add new algorithms. 3.2.3 Summary of the capabilities of the systems The next two sections analyse the general capabilities and the quality of available operators based on the information recorded in the system templates. *HQHUDO FDSDELOLWLHV Table 9 shows the summary of the first part of the templates provided by the testers, related with the general capabilities of the software systems. The answers of all system templates together are presented in Appendix IX, axpand capabilities are in light grey because only a novice tester provided the software system template and partially filled. $UF*,6 &37 5DGLXV &ODULW\ D[SDQG 1 Interactive generalisation tools: 1.1 Conflict detection partially partially partially yes 1.1 Conflict visualization yes no yes yes 1.2 Generalisation of manually selected features yes no yes no 2 Generalisation operators: 2.1 Topology is preserved partially partially partially partially 2.2 Relationship with terrain is considered no no no no 2.3 Spatial patterns and relations are preserved no partially no no 2.4 2.5D or 3D geometries supported yes (2.5D) no no no 2.5 Call services yes no no no 60 $UF*,6 &37 5DGLXV &ODULW\ D[SDQG 2.5 Generalisation services yes no no no 3 Generalisation process: 3.1 Batch mode yes yes yes yes 3.1 On-line mode yes no yes no 3.1 Rule, constraint, expert system, agent, … based no partially yes yes 3.2 Parameters manually or automatically specified manually manually manually manually/ automatically 3.3 Generalisation zones are supported yes partially yes no 3.3 Generalisation zones manually drawn or automatically detected manually automatically manually/ automatically - 3.4 Selection of features for generalisation based on spatial criteria yes no yes no 3.4 Selection of features for generalisation based on semantic criteria yes partially yes no 3.5 Incremental updating is supported no no no no 4 MRDB: 4.1 Schema transformation is supported yes no yes yes 4.2 Visualize two models at one time yes no yes yes 4.3 Links between two yes no yes no 61 $UF*,6 &37 5DGLXV &ODULW\ D[SDQG representations in different scales are supported 4.3 m:n relations are supported yes no yes no 4.3 Links between different geometries supported yes no yes no 4.4 Versioning is supported yes no no yes 4.5 Matching of features between different scales is supported yes no no no 5 System properties: 5.1 SHAPE format is supported yes yes yes yes 5.1 FME, CITRA or other transformation are supported yes no yes yes 5.2 GUI is available yes yes yes yes 5.3 GUI and manuals are in English yes yes yes yes 5.3 GUI and manuals are in other languages yes no no yes 5.4 Possibility to customise own generalisations yes no yes no 5.4 Possibility to add new algorithms yes no yes no 6 Map production: 6.1 Cartographic symbolisation yes no yes yes 6.2 Map production yes no no yes Table 9 Summary of the first part of the completed system templates 62 From this summary we can make the following observations. Concerning the LQWHUDFWLYLW\ we can say the following. All the systems have tools to partially detect and visualize the conflicts generated by the generalisation processes; in CPT, which is a software system that only works as batch process, only PUSH provides additional information to be used for inspecting the results. ArcGIS and Radius Clarity allow the generalisation of manually selected features. CPT does not allow this, because it works only on batch mode, neither axpand, because in this software the generalisation is an integrated process that takes into account the objects together with their relationships. As conclusion, most of the vendors provide tools for detecting and visualizing the conflicts generated by the generalisation operations. All the systems provide JHQHUDOLVDWLRQ RSHUDWRUV that preserve the topology, mostly partially, but the relationships with the terrain or the spatial pattern and relations are not taken into account. Contextual generalisation is not fully supported in all systems. Aspects as 2.5D/3D data or geoservices are not available in most of the generalisation tested systems. Only ArcGIS can manage 2.5D data and provides geoservices. Relating to JHQHUDOLVDWLRQ SURFHVV, except CPT, which only allows batch mode, all the systems can work on batch and on-line mode. All the systems, except ArcGIS, use techniques to optimize the results of automatic generalisation, although they are implemented at different levels. Radius Clarity and axpand are the only constraint-based systems. The selection of parameters is mainly done manually in all the systems. The generalisation zones are available in all the systems, except in axpand. ArcGIS and Radius Clarity allow spatial and semantic criteria in the selection of features to be generalised, CPT allows semantic criteria partially and axpand does not allow any selection. An important aspect of the production workflows is the dataset updating, and any system provides a solution for incremental updating of generalised data. ESRI and Radius Clarity have some tools to manage 05'%, as m:n relationships or links between representations or different geometries of one object. ArcGIS and axpand support versioning. ArcGIS provides tools for matching of features. About V\VWHP SURSHU WLHV, all the systems support SHAPE format, and all of them, except CPT, support transformation softwares as FME. Only ArcGIS and Radius Clarity have the possibility to customise existing algorithms and to add new ones. Except CPT all the systems allow cartographic symbolisation. In addition ArcGIS and axpand provide a full PDS SURGXFWLRQ system. 4XDOLW\ RI WKH DYDLODEOH RSHUDWRUV Appendix X contains the second part of the templates, related to the quality of the generalisation operators. For each object type, the values of the 3 systems which the testers provide information (ArcGIS, CPT and Radius Clarity) are showed together. The lack of information for axpand is because only the vendor has provided the evaluation of the quality of the generalisation operators, while no information was provided by the testers. The following observations can be done from the analysis of the mentioned table. ArcGIS, CPT and Radius Clarity provide tools for EXLOGLQJ generalisation. CPT and Radius Clarity achieve the best results and in the case of ArcGIS the results are acceptable. ArcGIS and Radius Clarity have tools for URDG simplification and smoothing. The bend conflicts are managed in Radius Clarity and, applying displacement, in CPT. Only CPT has displacement tools. Any system has tools for road typification. Any system achieves good results. ArcGIS and Radius Clarity have tools for UDLOZD\ simplification. Only CPT has displacement tools. Any system provide tools for achieve good results. ArcGIS and Radius Clarity have tools for ERUGHU simplification. Only CPT has displacement tools. Any system provide tools for achieve good results. 63 ArcGIS and Radius Clarity have tools for the K\GURJUDSKLF QHWZRUN and FRDVW OLQH simplification and smoothing. Only ArcGIS provides tools for area aggregation. Only CPT has displacement tools. Any system has tools for typification. Any system achieves good results. ArcGIS and Radius Clarity have tools for the ODQGXVH PRVDLF simplification. Only ArcGIS provides tools for area aggregation. The best results are achieved using ArcGIS. ArcGIS and Radius Clarity have tools for the UHOLHIFRQWRXU OLQHV simplification. Any system achieves good results. Any system has tools to generalise LVRODWHG SRLQWV. ArcGIS and Radius Clarity have tools for LVRODWHG OLQHV simplification and smoothing. The bend conflicts are managed in Radius Clarity and, applying displacement, in CPT. Only CPT has displacement tools. Any system has tools for typification. Any system provide tools for achieve good results. ArcGIS and Radius Clarity have tools for LVRODWHG DUHDV simplification. ArcGIS provides tools for area aggregation and Radius Clarity for area enhancement. Any system achieves good results. 3.2.4 Conclusions for the capabilities of tested software systems based on testers’ information The results of this section, that studied the capabilities of the tested system, are summarised in Table 10. From the analysis we can draw the following conclusions. All the software systems provide a set of tools but none of them achieve globally good results. Despite the current limitations, all four systems can be implemented to automate partially the generalisation processes and optimize the production workflows. Topology is only partially managed and 2.5D is only supported in one of the systems. Some functionalities are still missing in all the systems, for example incremental updating and full contextual generalisation. Although some systems allow the input of generalisation requirements through constraints or rules, improvements in the definition of the user requirements and their implementation in the systems would be necessary. Only two of the systems allow the customisation of provided generalisation tools, for adding new algorithms or modifying existing functionalities. (Customising the systems allows improving the results and facilitates the integration of the systems in a production workflow.) There are also additional minor limitations, such as poor or lack of information about the results of the generalisation process, including reporting of errors, statistics, measurements, etc. As was mentioned before, the analysis that describes the capabilities of the systems should be treated with caution. One reason is that the evaluation of one software system, axpand, is not complete because lack of information: only the vendor provided the software system template and only a novice tester provided it, only partially completed. Other reasons are that the analysis is based on testers’ information that was provided in a heterogeneous manner and the unavoidable part of subjectivity in the analysis and the summarisation of the templates, as well as in the elaboration of the conclusions. A future project should define more accurately the templates and the criteria to complete them to reduce the subjectivity of these types of results. 64 $UF*,6&375DGLXV&ODULW\D[SDQG Results Acceptableresultsforbuilding simplification,linesimplification, areaaggregation,collapsedual linesandmosaicgeneralisation. Goodresultsforsimplification, aggregationandtypificationof buildings. Acceptableresultsfor displacementanddeformation. Goodresultsinlinear simplificationandbuilding generalisation. Missing operators Enlargementofbuildings, displacement,pointgeneralisation, … Selection,linearsimplification, exaggeration,typificationonlinear elements,… Displacement,typification, collapse,…Collapse,typification,… Terrain generalisation Nospecifictoolsforterrain generalisation. Nospecifictoolsforterrain generalisation. Nospecifictoolsforterrain generalisation. Nospecifictoolsforterrain generalisation. Algorithms Fewalgorithmsforeachoperator.Onealgorithmforoperator. Severalalgorithmsforbuilding androadgeneralisation.Onealgorithmforoperator. TopologyTopologyispartiallymanaged.Topologyispartiallymanaged.Topologyispartiallymanaged.Topologyispartiallymanaged. Contextual generalisation Objectsareprocessedclassby classandinsequentialmode, withouttakingintoaccountthe contextandtherelationships betweenthem. Thecontextispartiallytakeninto account. Thecontextispartiallytaken intoaccount. Thecontextispartiallytaken intoaccount. Generalisation process--Ruleandagentbasedsystem.Rulebasedsystem. Optimization techniques- PUSHuseoptimization techniques.Useofoptimizationtechniques.Useofoptimizationtechniques. Incremental generalisation Incrementalgeneralisationnot supported. Incrementalgeneralisationnot supported. Incrementalgeneralisationnot supported. Incrementalgeneralisationnot supported. 2D/3D2.5Ddatamanagement.Only2Ddata.Only2Ddata.Only2Ddata. GISplatform/ Mapproduction system CompleteGISplatformandmap productionsystem.No NotacompleteGISplatform. Cartographicsymbolisationis possible. Completemapproduction system. MRDBYesNoYesNo 65 $UF*,6&375DGLXV&ODULW\D[SDQG Workflow builder Workflowcreationtool(Model Builder). Abletobeintegratedina workflowbasedinmostofthe commercialsystems,becauseit usesSHAPEformatandworksin off-linemode. Creationofautomated generalisationworkflows. Possibilityofworkflows creation. Userdeveloped tools Possibilitytoincludenew generalisationtools. NosensebecauseCPTisnota completGISplatform. Environmenttodevelopand researchnew generalisation.algorithms, althoughitrequiresabasic understandingofobject orientation,agood understandingofgeospatialdata modelling,XMLandJava knowledgeforcustomisation. Itisnotpossibletocustomise existingalgorithmsortoadd newalgorithms. DocumentationGooddocumentationavailable.Gooddocumentationavailable.Documentationavailable.Poordocumentation. Reportabout generalisation process Moredetailedinformationabout thegeneralisationprocessshould beprovided. Moredetailedinformationabout thegeneralisationprocessshould beprovided,althoughthe processesgenerateareportwith statisticsofthegeneraliseddata. Moredetailedinformationabout thegeneralisationprocess shouldbeprovided. Moredetailedinformationabout thegeneralisationprocess shouldbeprovided. GUI StandardGISGUI. GUItoosimple.Workflowshould bemoreuserfriendly,forexample optimizingformattranslationsand minimizingthenumberof processestobeexecuted.StandardGISGUI.StandardGISGUI. InstallationEasyinstallation.Easyinstallation.Noteasyinstallation.Complexinstallation. Table10Summaryofthefunctionalitiesofthesystems(regularÆadvantages,italicÆlimitations) 66 3.3 Evaluation of test processes The processing templates aimed at measuring the time spent by each tester on performing and processing the generalisation tests for each software and each test case. Only the vendor University of Hannover provided the filled processing templates (for all four test cases), while 27 filled templates were provided by the project team testers. If we look at the content of the provided templates, we notice a great heterogeneity in the way the templates were handled. As the generalisation processes within the different software solutions could differ, the templates could not be filled according to a particular pattern or form, although a default filling was provided. For example some testers provided a single template per system, while others provided a separate one for each test case. As a consequence the number of tests carried out and the number of templates provided for the software is different and makes the interpretation difficult. Also heterogeneity can be observed in the way the templates were filled. The instructions appear to be fuzzy and as a consequence people did not measure time the same way and did not detail their work the same way. For instance, some testers measured time in days and others in hours and minutes but did not mention what a day of work correspond to in hours: an approximation of the number of hours of a work day was used in the analysis below. In conclusion, the time interpretation in the following analysis should be handled with care. Table 11 and Table 12 show the kind of differences encountered in the way the tests were detailed in the templates. On the one hand the template shown in Table 11 is very synthetic as the test is only composed of three main steps: analysis, process and symbolisation. On the other hand, Table 12 shows only an extract of the complete template for the same test case but with another software. The steps are much more detailed with even the counting of processed objects given (as asked in the instructions). Date Action (predefined) Specify Time spent Calculation time 11.2007 Reading the manual 6 hours - 11.2007 Installing the software 1 hour - 02.2008 ICC data set: Data analysis and constraints to system operations 15 hours - 02.2008 ICC data set: Generalisation Parameter setting for generalisation and processes, it includes proofs and evaluation 10 hours 2 hours 03.2008 Symbolisation Symbolisation setting and output file generation 10 hours 03.2008 Documentation Fill the templates 4 hours Table 11 A complete processing template for ESRI software and ICC test case. Date Action (predefined) Specify Time spent Calculation time Installing the system Reading the manual 1 h - Installing the software 3 h Importing the data Importing the data 5 min - Creating the signatures 5 min Expressing the constraints Expressing selection in queries and processes 9 hours 67 Date Action (predefined) Specify Time spent Calculation time - Parameter setting for building simplification 20 min - Parameter setting for road generalisation 30 min - Coding a custom constraint for greenhouse aggregation 3 hours - Coding custom micro constraints 2 days - Coding custom meso constraints 6 days Triggering automated processing Triggering partition creation Number of features/feature type: all roads test area boundary 1 min 2 min - Triggering selection Number of features/feature type: all streets, contours, walls 5 min 10 min - Operator: block generalisation (building simplification and aggregation of buildings, flowerbeds and sport grounds Number of features/feature type: 604 block polygons buildings inside the polygons 5 min 2 hours To be continued Table 12 First part of the processing template for Radius Clarity and ICC test case. From the analysis of completed processing templates we can observe some general trends and we can draw some interesting conclusions. *HQHUDO WUHQGV Despite the differences highlighted above, it is possible to identify some general trends on how the generalisation tests were carried out. The most important one is that the tests were carried out with caution and very seriously. Indeed, the mean time spent on a test by a tester is 53 hours that correspond approximately to 1.5 week of full time work. Globally, the amount of work spent on the tests is approximately 6 person– months. In addition the vendor that filled the templates spent much less time on the tests with a mean of 4 hours, which is most likely because of the technical mastery on the software. 68 Another trend revealed by the templates is that computing time appears to be moderate in general. Although not all generalisation constraints were solved and in spite of the small size of the test cases, it is possible to conclude that the tested generalisation applications do not cost extreme computing time. Furthermore, the analysis of the processing templates shows similarities on the difficulty and easiness to generalise between test cases: the time spent on each of the test cases is quite the same. The analysis did also not reveal any influence of the user-friendliness of the tested generalisation systems. Indeed, the time spent on a test case by novice (or expert) testers with different software is equivalent. Although the details of the templates showed that the difficulties (in term of time spent on specific actions) differ from one software to another, the final time is quite the same. The mean time spent on a test for a system varies from 45 hours to 58 hours. According to the approximations highlighted in the introduction of this section, the duration can be considered as equivalent. However, one should be careful with interpretation of time spent on the tests, because it highly depends on the time that testers were allowed to spend on each test. For example some organisations allowed their employees to work at most 5-10 days on each test. Therefore most optimally, time spent should be compared with the quality of the result. Only then can the differences between novice and experts be completely analysed, and from there conclusion on the user friendliness of the system be made. &RQFOXVLRQV In addition to the general trends revealed, the analysis of the completed processing templates allows drawing some interesting conclusions. First, we notice a heavy amount of time spent by most testers on the installation of the software. It appears to be a point that should get attention by the vendors. The templates also confirm that technical mastery on such software is essential to reduce the amount of time spent on the tests. Indeed, the CPT vendor tests are distinctly the quicker ones. Likewise, testers considered as novices spent much more time on the tests than expert ones, particularly on getting used to the software and its generalisation operations. For instance, a template revealed that a novice tester spent 30 hours on testing the different proposed algorithms. The templates also highlight two specific limitations of generalisation solutions in commercial software (which are known in research), namely the difficulty to parameterise the complex algorithms and the lack of default tools (for instance default algorithm sequences or default constraints) requiring a lot of users’ work. Thus, a CPT tester spent a third of the testing time in setting the correct parameters for building displacement and typification. And regarding the lack of default tools, the Table 12 template shows the huge amount of time spent on customising constraints in Radius Clarity for ICC test case (8 days). Finally, the computation time is very difficult to interpret. Indeed, no default computer configuration was requested to carry out the tests and up-to-date computers can be much quicker than others. For instance, for the same software, CPT, the vendor indicates a cumulated computation time of 48 minutes whereas a tester (with apparently a less effective machine) indicates 5 hours of computation time. 3.4 Evaluation of constraint expressions In the constraint expression templates, testers entered for a specific test case whether they were able to express the constraints. At the start of the research, it was expected that these tables would provide insight into what constraints can be expressed in the systems, i.e. into the generalisation functionalities of commercial systems. For this purpose, we calculated the percentage of the constraints that could be expressed in the systems either ‘fully’ ‘partially’ and ‘not’ according to the testers, grouped by several criteria: o number of objects involved in the constraints (one, two or a group); o type of constraints (see classification introduced in Section 2.1.4) o test case 69 o system o feature class An example of such calculated percentages per test case and per number of objects, per system, and per feature class are shown in Table 13 to Table 16 (the system names are made anonymous for this purpose). Test case On all objects % Fully % Partially % Not ICC 25% 13% 62% IGNF 28% 16% 56% KADASTER 32% 11% 57% OSGB 16% 20% 65% Total 26% 14% 60% Table 13 Constraint expressed, averaged per test case Software system On all objects % Fully % Partially % Not System 1 38% 14% 48% System 2 16% 18% 66% System 3 35% 8% 57% System 4 29% 11% 60% Table 14 Constraint expressed, averaged per system Test case Building Land use Road Water Elevation % Full y % Par t % No t % Full y % Par t % No t % Full y % Par t % No t % Full y % Par t % No t % Full y % Pa rt % No t ICC 29 % 23 % 48 % 26 % 14 % 61 % 29 % 5% 65 % 20 % 10 % 70 % 23 % 4 % 73 % IGN F 30 % 23 % 46 % 29 % 0% 71 % 37 % 7% 56 % 10 % 6% 84 % 14 % 0 % 86 % KA D 38 % 27 % 36 % 33 % 3% 63 % 29 % 7% 64 % 37 % 26 % 37 % OSG B 18 % 28 % 54 % 0% 20 % 80 % 13 % 10 % 78 % 0% 20 % 80 % 5% 8 % 88 % Tota l 29 % 25 % 47 % 29 % 9% 63 % 29 % 7% 64 % 19 % 11 % 70 % 15 % 5 % 80 % Table 15 Constraints expressed for a number of feature classes, per test case 70 System Building Landuse Road Water Elevation % Fully % Part % Not % Fully % Part % Not % Fully % Part % Not % Fully % Part % Not % Fully % Part % Not System 1 28% 22% 50% 43% 17% 39% 58% 8% 34% 32% 14% 55% 31% 0% 69% System 2 26% 32% 41% 5% 6% 90% 12% 9% 79% 5% 6% 89% 6% 2% 93% System 3 35% 15% 51% 49% 7% 44% 34% 2% 64% 30% 13% 57% 23% 15% 62% System 4 26% 18% 55% 24% 0% 76% 21% 9% 70% 14% 19% 67% 0% 0% 100% Table 16 Constraints expressed for a number of feature classes, per system Although conclusions from this analysis meet the objectives of the research of quantifying the stateof-the-art of automated generalisation, we found that the analysis contains too many biases, which may cause readers drawing wrong conclusions from the numbers. Therefore such conclusions are not drawn here and the results of the analysis are left out from this report. Several reasons can be listed for the ambiguity of these conclusions leading to misinterpretation: Different absolute numbers are underlying the percentages and therefore percentages do not give a complete view. For example, considerable more constraints were expressed for single objects (86, 27, 32 and 24) than for groups of objects (28, 4, 14, and 12). The numbers do not indicate available functionality, but the constraints that testers considered to be expressible (i.e. the available functionality according to the testers). The importance of constraints was not taken into account. Consequently, the results for a very unimportant functionality could have major influence on the calculated percentages. We also learned that it is impossible to distinguish between ‘fully’ and ‘partially’ expressed constraints, as the extent to which a constraint could be expressed in the systems was more dependent on the view of testers than on a objective measures. o The percentages are dependent on the specific constraints that were defined (and ignored) for the test cases. For example some important constraints on groups of objects are missing because they were difficult to formalise and therefore they are not taken into account in this analysis. o The score of a specific system corresponds directly to the number of constraints that were defined in the specific expertise area of the software. For example the more constraints defined for buildings, the better CPT (which is specifically suitable for buildings) will score and vice versa. It should be noted that a grouping of certain representative constraints could improve reliability of the results of such analysis. To meet the limitations within the scope of this research, the testers’ information was studied in interaction with other aspects such as output, test case, system, and tester to make more in-depth observations (see Section 3.6). A future test could consider selecting a representative set of constraints to analyse availability of functionalities in commercial systems. Although the testers’ information should be interpreted with care, we did make the following observations from this analysis that are relevant for our research: o About 50% of all the constraints could be expressed fully or partially in the systems. This shows that systems contain a considerable amount of automated generalisation capabilities in relation to the constraints defined for the test. o The most supported constraints were those applying to a single object, i.e. functionality addressing constraints for two objects and for groups of objects is less available in commercial software. This was already highlighted during the OEEPE project: one of the main conclusions was the general lack of contextual generalisation capabilities in the commercial software. Contextual operations have since then started to appear in commercial systems. o The analysis shows overall low values for OSGB data set compared to the other test cases. In some systems even less than 25% of the OSGB constraints could be 71 expressed in the systems. This might be because of the complexity of the (NMA specific) constraints influenced by the large scale transition from 1:1250 to 1:25K when compared to the other test cases. Examples are the constraints on slope hachures (to be introduced in the target scale) and the nine constraints addressing how buildings should be aggregated depending on the initial pattern. The complex and OSGB specific constraints on buildings also explain a high number of partially expressed constraints in CPT (instead of fully). From the explanation that testers gave for not being able to express certain constraints, we can also conclude that functionalities for parameterisation are missing and that the software systems lack functionalities for defining sensible groups for generalisation, such as ‘building blocks’. The systems group objects by the partitions built from linear features. This does not always yield the best solution since objects are often unevenly distributed across these partitions. 3.5 Automated evaluation of generalised outputs: results and conclusions This section summarises the results of the automated constraint-based evaluation in which three constraints were evaluated: minimum area of buildings, minimum distance between two buildings and minimum distance between buildings and roads. The automated constraint-based evaluation has been limited to these constraints because most time was put in the development of the method of automated evaluation, as well as in the design of the prototype1 . The constraints that are automatically evaluated were selected because of their high formalisation degree. Another limitation of the presented evaluation is that constraints are evaluated independently from other constraints and therefore the interaction between several constraints is not addressed. However the interpretation of the results does look in more detail to further understand low or high constraintviolation values. Consequently this section provides insights into the method of automated constraintbased evaluation. The results of the automated evaluation of automatically generalised data are presented in Section 3.5.2. To show to what extent automated constraint-based evaluation is appropriate to identify the quality of generalised data, we first applied the developed evaluation prototype to interactively generalised data of Kadaster. The results are presented in Section 3.5.1. 3.5.1 Automated constraint based evaluation of interactively generalised data We applied the prototype to interactively generalised data of Kadaster, scale 1:50k (the target dataset of the test case of Kadaster). In this test we assumed that the interactively generalised data, which are currently in production provide good generalisation results. We evaluated two constraints: minimum area of buildings and minimum distance between buildings. The results for the first constraint show that 27% of the buildings are smaller than the threshold (0.16 map mm2 ) and are therefore evaluated as bad (see Figure 4). However, when examining the data in more detail, we found that many “too small buildings” are just a little below the threshold size. The difference in minimum size, as mentioned in the written specifications (main source for the constraints) and as used in interactive generalisation, can be explained in two ways. First, it is not possible for humans to distinguish between the threshold and the threshold plus/minus a flexibility range, and, therefore, cartographers use the thresholds with a notion of flexibility (Ruas, 1999; Bard, 2004). Second, in specific situations the cartographer may have chosen to relax the size constraint to meet a more important constraint, e.g., “keep important buildings.” 1 it cost much more time than expected to prepare the outputs for the automated evaluation – around 700 output data sets had to be prepared manually and further homogenised for the automated evaluation 72 F The gene each delib Fig How and disti cart solu Figure 4 Resu e automated e eralised datas h other (Figu berately viola gure 5 Analys gener wever because encountered inguish the c ographic con utions), minim ults of analysi evaluation of et also shows ure 5). The vi ating constrain is of minimu ralised data. T e of the high n many situat ase shown in nflicts) from t mal distance be ing minimal b the constrain many violatio iolations can nts to meet mo m distance b The non-viola number of vio tions assessed n Figure 6a (i those shown etween buildin building area 1:50k. nt on minima ons of the con partly be ex ore important etween build ated building olations, we ex d as ‘bad’ a in which the in Figure 6b ngs should be as in interacti l distance (2 nstraint. 46% o plained by th constraints, as dings constrai gs are not sho xamined the c s shown in minimum di b and Figure further refine ively generali map mm) in of the buildin he notion of s just discusse int violation o own in this gr conflict situati Figure 6b an stance constr 6c (which m ed in constrain ised data, sca n the interacti ngs are too clo flexibility an ed. on interactive raph. ions in more d nd Figure 6c raint does ide may be accep nt definitions. ale ively ose to d by ely detail . To entify table 73 A b C Figure 6 Minimal distance constraint identifies unacceptable situations (a). Acceptable generalisation solutions that violate the distance constraint (b and c). The conclusion of this automated evaluation of interactively generalised data is that constraint-based evaluation requires further research to be able to describe the quality of generalised data. Future research should aim at better definition of constraints with respect to automated evaluation and better understanding of the impacts and dependencies of several constraints. Chapter 5 (discussion and conclusion) contains several recommendations on how constraint-based evaluation can be improved to become more appropriate for assessing generalised data. 3.5.2 Automated constraint based evaluation of automatically generalised data This section presents results of the automated evaluation of three constraints: minimum area of buildings, minimum distance between buildings and minimum distance between buildings and roads. It is very important to note that this evaluation only considers individual constraints. Therefore the evaluation may not be used to evaluate the overall quality of the outputs, since some constraints may have been violated intentionally to meet other more important constraint. In addition the individual constraint may show good results because another constraint (i.e. keep density of buildings) was highly violated. To understand the concepts ‘constraint violation’, ‘average constraint violation’ and ‘qualification’ as used in this evaluation, we first explain them. Constraint Violation (CV) is the degree of non-fulfilment of a constraint. CV is the quantitative defiance of the generalised state from the ideal state defined by the cartographic constraint. The smaller the defiance, the better fulfilled the cartographic constraint. CVs are expressed by values between 0 und 1, where 1 denotes a maximum violation. In our evaluation where we only evaluated legibility constraints the CVs either identify a cartographic conflict (CV=1) or a good solution (CV=0). Average Constraint Violation (CVAverage) is an aggregation of constraint violations on individual objects into one number. CVAverage are expressed by values between 0 and 1, where 1 denotes a maximal violation of the constraint and vice versa (similar to CV). Qualification is the transformation of quantitative evaluation results (CV, CVAverage) into qualitative statements about the quality of the generalisation solution. The transformation that we applied is as follows: IF 0 ” CVaverage ” 0.25 THEN Qaverage = good ELSE IF 0.25 ” CVaverage ” 0.5 THEN Qaverage = nearly good ELSE IF 0.5 ” CVaverage ” 0.75 THEN Qaverage = nearly bad 74 ELSE IF 0.75 ” CVaverage ” 1 THEN Qaverage = bad $XWRPDWHG HYDOXDWLRQ RI PLQLPXP DUHD RI EXLOGLQJV The results of evaluating the minimum area constraints using CV, CVAverage and qualification are listed in Table 17. In the remainder of this section, these results are interpreted and conclusions are drawn per test case. Note that in the remainder of this section Kadaster is sometimes referred to as TDK. The reason is that in the course of our project the Dutch National Mapping Agency (TDK) merged with the Netherlands’ Kadaster. Test case System Tester Nb of inital map objects Nb. of map objects Nb. of map objects with CV =0 Nb. of map objects with CV =1 CVaverage Quality ICC (threshold: 0.16mm2 ) CPT ICC 2709 886 828 58 0.07 good ICC CPT Kadaster 2709 887 697 190 0.21 good ICC Radius Clarity IGNf 2709 1381 640 741 0.54 nearly bad ICC Radius Clarity OSGB 2709 2669 684 1985 0.74 nearly bad ICC ArcGIS Kadaster 2709 1881 938 943 0.50 nearly good ICC ArcGIS ICC 2709 454 454 0 0 good IGN (threshold: 0.2 mm2 )**,* CPT ICC 2019 98 98 0 0 good IGN CPT ITC 2019 556 547 9 0.02 good IGN CPT Kadaster 2019 856 145 711 0.83 bad IGN ArcGIS Kadaster * * * * * * IGN Radius Clarity IGNf 2019 1019 863 156 0.15 good IGN axpand OSGB 2019 1529 1063 466 0.30 nearly good OSGB (threshold: 0.16mm2 ) CPT ICC 13241 1692 1454 238 0.14 good OSGB CPT ITC 13241 1265 1155 110 0.09 good OSGB CPT Kadaster 13241 1957 1957 0 0 good OSGB Radius Clarity OSGB 13241 7061 7061 0 0 good OSGB ArcGIS Kadaster 13241 2033 2032 1 0 good Kadaster (threshold:0.16 mm2) ArcGIS ICC 4285 356 353 3 0 good Kadaster ArcGIS Kadaster 4285 105 105 0 0 good Kadaster axpand OSGB 4285 367 362 5 0 good Kadaster CPT ICC 4285 358 337 21 0.06 good 75 Te Ka Ka Ka Ka Ka Ta * The been opti ** Fo The Min (Som men case 0 0 0 0 0 0 0 0 0 est case adaster adaster adaster adaster adaster able 17: Numb e buildings in n generalised imal value or IGN test ca e graph represe Figu nimum area of me conclusio ntioned once, e is longer tha 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 CPTͲICC CPTͲKadaster ClarityIGNf System T CPT IT CPT K Radius Clarity IG Radius Clarity IG Radius Clarity O ber of map ob n the outputs and therefore ase the results enting these n ure 7: Averag of buildings: I ons are valid in the evaluat an the sections ClarityͲIGNf ClarityͲOSGB ArcGISͲKadaster ArcGISͲICC ICC Tester Nb init ma obj TC 428 Kadaster 428 GNf 428 GNs 428 OSGB 428 bjects for bot produced by e the minimum for Zurich tes numbers, are sh ge quality ass ICC test case for all test tion of the IC s evaluating th CPTͲICC CPTͲITC CPTͲKadaster ClarityͲIGNf IGN b of tal ap jects Nb. of map objects 85 292 85 1822 85 174 85 372 85 360 th CV = 0 an Kadaster test m areas of the ster with axpa hown in Figur sessments of o cases. To av CC test case. C he other three AxpandͲOSGB CPTͲICC CPTͲITC CPTͲKadaster OSG f s Nb. of map objects with CV =0 259 239 62 363 358 d CV =1, and ter for IGN te e output build and are missin re 7 outputs, grou void repetitio Consequently test cases). CPTKadaster ClarityͲOSGB ArcGISͲKadaster ArcGISͲICC ArcGISKadaster GB Nb. of map objects with CV =1 CV 33 0. 1583 0. 112 0. 9 0. 2 0 d average qua est case with dings were no ng uped per test n, these con the section ev ArcGISͲKadaster AxpandͲOSGB CPTͲICC CPTͲITC Kadaste Vaverage Qua 11 good 86 bad 64 near bad 02 good good ality assessme ArcGIS have ot compared to case. clusions are valuating ICC CPTͲKadaster ClarityͲIGNf ClarityͲIGNs ClarityͲOSGB er ality d rly d d ent. e not o the only C test 76 Whe grap Fi en studying t phically presen igure 8 Avera o o o o the results fo nted in Figure age constrain dat The CPT te generated by Generalisatio compared to is a good misinterpreta eliminating t elimination, produce solu In both Radi meet the min more than 50 testers did no software Rad generalised s buildings. Th decrease of b In case of th to achieve a constraint wi ICC tester i legibility thre in the ArcGI strongly the aggregation) tester, shows mostly along r ICC test ca e 8): nt violation of ta set per gen ests show goo y the ICC has t on solutions the other sys example tha ation of the ov the buildings other actions utions that also ius Clarity tes nimum area c 0 percent of a ot manage to r dius Clarity. F states are sign he generalisat buildings, in c e solutions ge a generalisati ith the softwa is of very go eshold of 0.16 IS outputs. In e number of . Closer exam s that a lot of g the coastline ase the follow f „minimum neralisation sy od or near g the lower CVa generated w stems when on at assessing verall quality. with area un s as exaggera o meet the “ke sts a consider constraint. Th all buildings th realise the req Further, it can nificantly diffe tion accompli ontrast to OSG enerated by A ion solution are. The ArcG ood quality 6 mm2 . Howe n contrary to f individual b mination of the f n:1 or n:m e where there w wing observat area“ constra ystem and tes good solution average which m with CPT see nly considerin one constra In this case C nder the minim ation, aggrega eeping buildin able amount o he average co hat are judged quirements to be seen that erent. The ini ished by the I GB tester. ArcGIS, it can which do n GIS generalisa since every b ever the numb the Kadaster buildings (e. e generalisatio generalisation was initially a tions can be aint on build ster. ns. The gener means that it is ems to be o ng this constra aint at the t CPT decreases mum size. Ho ation, etc cou ng density” con of generalised onstraint viola d as “nearly b the buildings the numbers o tial ICC data IGNf tester re be concluded not violate th ation solution building exce ber of building r tester, the IC g. by means on solution pro n operations h a high density made (results ings of the IC ralisation solu s the best solut of higher qu aint. However time, may c s the violation owever instea uld be applie nstraint. d buildings do ation counts u bad”. That is, area size with of buildings in set contains 2 esulted in a h d that it is pos he minimum n generated by eeds the requ gs differs stro CC tester red s of elimina ovided by the had been app of buildings. s are CC ution tion. uality r this cause ns by ad of ed to o not up to both h the n the 2709 heavy ssible area y the uired ongly duced ation, ICC plied, This 77 might be a reason for the perfect satisfaction of the minimum area constraint since conflicts of constraints in dense regions could be inhibited. o A conclusion that applies to all outputs is that constraint violations occur in clusters (see Figure 9). Mostly, along the coastline, buildings are satisfactorily generalised although the building density is relatively high. In contrary, upcountry the constraint violations increase because of many small and isolated buildings. It is most likely that the problem is caused by the characteristics of the data set itself, that is, these clusters may occur where several cartographic constraints on the individual buildings cause conflicts. Figure 9: Visualised evaluation results of minimum area constraint in the ICC data set per generalisation system and tester (above from left: CPT/ITC; Radius Clarity/IGNf, ArcGIS/Kataster; below from left: CPT/Kadaster, Radius Clarity/OSGB, ArcGIS/ICC) To better understand how the constraints were respected, we analysed the reduction of buildings, see Figure 10. 78 Fi From show show Firs It is enla C A R igure 10 Redu m this figure w that extra ob wed similar co tly we observ s obvious that arged initially 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 1 AreaSize(squaremeters) 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 1 AreaSize(squaremeters) 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 1 AreaSize(squaremeters) CPT ArcGIS Radius Clarity uction of buil we can obser bservations ca onclusions an ve partly differ t the ICC test y large build 501 1001 Area 501 1001 Area S 501 1001 Area y ldings in ICC rve the follow an be made fr d therefore th rent distributio ter enlarged in dings more 1501 2 Buildings Size of Building Map 1501 Buildings Size of Building Map O 1501 Buildings Size of Building Map C test case in C wing. (This an rom this analy is analysis wa on of area siz nitially small although bot 2001 2501 p Objects 2001 2501 Objects 2001 2501 Objects CPT, ArcGIS nalysis is only ysis. As mentio as skipped for es generated b buildings mo th testers us initial values ICC_clarity_IGNf ICC_clarity_OSGB initial values ICC_esri_TDK ICC_esri_ICC ICC_CPT_ICC ICC_CPT_TDK initial values Sand Radius y reported for oned before, t the other test by ICC and K ore whereas th ed the same B Clarity outp r this test cas the other test c cases). Kadaster with C he Kadaster t e parameters uts se, to cases CPT. tester and 79 gene of b Seco duri OSG teste that Min For pres buil for e Fig dat eralisation sof buildings, it ca ondly, in the R ing generalisa GB remained er enlarged m initially large nimum area of IGN test cas sented in Fig ldings were no evaluation: gure 11 Aver ta set per gen o o o o ftware. Additi an be assumed Radius Clarity ation while th nearly unchan more buildings e buildings be of buildings: I se we can dra gure 11). Not ot generalised rage constrain neralisation sy and OSG Three out of minimum are The ICC is th is, in their so The Kadaste constraint wi generated wi In contrary to in the IGN d four generali 12. It is obvi ionally, becau d that another y solution gen he area size o nged. A third s with ArcGIS came larger to IGN test case aw the follow te that ArcGI , as was ment nt violation o ystem and tes GB) are missi f four genera ea constraint. he only tester olution, all rem er tester did n ith the same s ith CPT it can o the automat data set we ca isation solutio ous that in tho use both gener aspect influen nerated by IGN of the largest conclusion th S independent oo. wing conclusio IS with Kada ioned before. of „minimum ster. In this fi ing, see expla alisation solut who achieved maining buildi not manage t software. How be said that th tically derived nnot identify ons. The only ose areas the i ralisation solu nced the modi Nf the initially t buildings in hat can be dra tly of the init ons from the aster tester w In addition ax area“ constr igure the resu aination in Se tions are (nea d to fully satis ings are larger to translate sa wever, on the b he system is a d generalisatio specific areas y potential pro initial density utions contain fication of are y large buildin the solution awn for ArcG tial area size, results (resul was not evalu xpand results w raint on build ults of axpan ection 3.1. arly) well don sfy the constra r than the requ atisfactorily th basis of the tw able to translat on solutions of s that cause pr oblem area is of buildings i n the same num ea sizes. ngs became la generated by GIS is that the which means lts are graphi uated because were not avai dings of the IG d testers (Zu ne concerning aint with CPT uested thresho he minimum wo other solu te the constrai f the ICC data roblems acros marked in Fi is very high. mber arger y the ICC s that cally e the lable GN rich g the , that old. area tions int. a set, ss all igure 80 Fig Min The in F gure 12 Poten nimum area of e following ob Figure 13): Figure 13 Av o o o o ntial problem of buildings: K bservations we verage constr Kadaste Both solutio minimum are large number The satisfact and Kadaster more buildin data set gene the others. Except for t Radius Clari concerning t good results The two gene show a comm ICC data set) m area in the I Kadaster test c ere made for t raint violation er data set pe ons generalis ea constraint r of individual tion of constra r. The general ngs which do eralised by the two generalis ity), most of t the legibility were obtained eralisation sol mon spatial di ). IGN data set case the results of n of „minimu er generalisat sed with Arc on buildings. l buildings ha aints with CPT lisation soluti not meet the e Kadaster tes sation solution the solutions of buildings. d. lutions that ha stribution of v concerning t the Kadaster um area“ con tion system an cGIS are of Additionally as been reduce T differs large ion generated minimum ar ter has about ns (one prod generated by . That is, wit ave a high ave violated buildi he minimum test case (grap nstraint on bu nd tester high quality y they have in ed during gene ely between th by the Kadas ea constraint. 1500 map obj duced with C y the testers a th every gene erage constrain ings (as identi m area constra phically prese uildings of the y concerning n common tha eralisation. he testers ICC ster tester con Additionally jects more tha PT and one are of high qu eralisation sy nt violation do ified in case o aint. ented e g the at the C/ITC ntains y, the an all with uality ystem o not of the 81 Minimum area of buildings: OSGB test case For the OSGB test case we can conclude that the amount of buildings in the outputs differ considerably within the outputs, even for the same system. Especially the numbers of buildings with CPT were highly reduced. Most likely this is because a bug in CPT that eliminates all buildings that intersect (or touch) a road (see also Figure 91, Section 4). Moreover CHANGE eliminates all the buildings under the minimum, which is not a bug of the software, but done by design. Figure 14 Average constraint violation of „minimum area“ constraint on buildings of the OSGB data set per generalisation system and tester. Figure 15 Visualised evaluation results of buildings of the OSGB data set per generalisation system (above: CPT; below, left: Radius Clarity, below, right: ArcGIS) and tester. Findings and conclusions of automated evaluation of minimum area constraint The conclusions on the evaluation of minimum area constraint can be summarised as follows. Firstly, good solutions concerning the minimum area constraint were achieved by all systems and all test 82 cases (but not by all testers), although for same systems and test cases very different results were achieved (see visual comparison in Section 3.6 for further explanation of these differences). Secondly, for ICC test case we observed clusters of constraint violations in all generalisation solutions, which may be a challenge of the specific test case, that is, these clusters may be derived from both conflicts due to the application of several cartographic constraints on the individual buildings and the interaction with map objects in the proximity, i.e. interaction with other constraints. A third conclusion is that most solutions generated for Kadaster test case are of high quality concerning the legibility of buildings. This has two reasons. Firstly fewer conflicts can be expected in rural area, which is the main characteristic of the test case. Secondly, areas that are covered for more than 10% by buildings are replaced by built-up areas in this test case. $XWRPDWHG HYDOXDWLRQ RI PLQLPXP GLVWDQFH EHWZHHQ WZR EXLOGLQJV The results of evaluating the minimum distance between buildings constraint using CV, CVAverage and qualification are listed in Table 18. It has to be noted that not all results per test cases, systems, and testers are presented in this Table. The reasons are several. An important reason is that the list of constraints provided by Ordnance Survey does not contain a “minimum distance” for building objects. Consequently, the results for OSGB test case are not presented here. In addition, some of the output shape files containing building features had not been generalised by the testers and some of the output shape files were somehow corrupted, and thus not all testers output was considered in this evaluation. Test case Syste m Teste r No. of map objects No. objects CV= 0 No. objects CV = 1 CVaver Quality Kadaster (threshold:0.2m m) Radius Clarity IGNf 174 157 17 0.10 good Kadaster Radius Clarity IGNs 372 316 56 0.15 good Kadaster Radius Clarity OSG B 360 306 54 0.15 good Kadaster CPT ICC 358 356 2 0.01 good Kadaster CPT ITC 292 280 12 0.04 good Kadaster CPT Kad 1822 1002 820 0.45 nearly good Kadaster ESRI ICC 353 307 46 0.13 good ICC threshold:0.2m m Radius Clarity IGNf 1381 483 898 0.65 nearly bad ICC CPT ICC 886 700 186 0.21 good ICC CPT Kad 887 506 381 0.43 nearly good ICC ESRI Kad 1900 152 1748 0.92 bad IGN (threshold:0.1m m) Radius Clarity IGNf 1019 509 510 0.50 nearly bad IGN CPT ICC 98 96 2 0.02 good IGN CPT ITC 556 517 39 0.07 good IGN CPT Kad 856 753 103 0.12 good Table 18 Number of map objects for both CV = 0 and CV = 1, and average quality assessment of minimum distance constraint on building map objects 83 This section discusses and interprets the results in detail per test case. In order to get more insight of why the results are so, we will also show information on selection ratio (the number of buildings after generalisation divided by the number of buildings before generalisation) along with the constraint violation per generalisation solution. It is worth noting that the use of ‘selection ratio’ does not imply anything regarding how the generalised objects were selected. Minimum distance between two buildings: ICC test case Results for ICC test case are presented in Figure 16. Figure 16 Average constraint violation of minimum distance between two buildings (upper) and selection ratio (lower) of the ICC data set per generalisation system and tester The following conclusions can be drawn for the outputs for ICC test case concerning the minimum distance between buildings constraint: Acceptable generalisation solutions were obtained with CPT by both testers (i.e. ICC and Kadaster). The evaluation results show the solution by the ICC tester is “good” and the one by Kadaster is “nearly good”. Average Constraint Violation of Minimum Distance Constraint between Two Buildings of the ICC Data Set 0 0.25 0.5 0.75 1 IGNf (tester) ICC Kadaster Kadaster Clarity (system) CPT (system) ESRI (system) Generalisation Systems & Testers AverageConstraintViolation Selection Ratio of Generalisation Outputs of the ICC Data Set (the initial data set contains 2709 building objects) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IGNf (tester) ICC Kadaster Kadaster Clarity (system) CPT (system) ESRI (system) Generalisation Systems & Testers SelectionRatio 84 Tests with Radius Clarity and ArcGIS failed to generate acceptable solutions. But the average constraint violation (0.65) of the solution derived by IGNf tester using Radius Clarity is not as bad as the violation (0.92) of the solution by Kadaster tester using ArcGIS. Within this test case, a positive correlation can be observed by comparing the distribution of selection ratios (Figure 16, upper) with that of average constraint violations of minimum distance constraint (Figure 16, lower) of all outputs. It appears that the lower the selection ratio, the higher degree of constraint violation was for these generalisation solutions, as more objects get deleted the chance becomes higher to satisfy the minimum distance constraint. While the ICC and Kadaster testers selected nearly the same amount of buildings (886 and 887, respectively) and both solutions are quite acceptable, the ICC result is better than the Kadaster result (see also Figure 17c Figure 17d). A closer look at the solution generated by ICC shows that CPT software addressed the constraint violation by shrinking the buildings under violation situations. This means that CPT is able to respect this constraint in certain ways. This results show that this test case is more difficult to tackle for the generalisation systems than the other test cases in terms of minimum distance between buildings. This is most likely because in the initial ICC data set, density of the buildings along the coastline is relatively high, buildings are much bigger than in other locations, and buildings are close to each other than in all the other test cases. If testers and systems fail to handle the minimum distance constraint, buildings in such regions are likely to violate the constraint. This is probably why the constraint violation of outputs of this test case is on average higher than in the other test cases (see Table 18). (a) IGNf (Radius Clarity) (b) Kadaster (ArcGIS) (c) Kadaster (CPT) (d) ICC (CPT) Figure 17 Evaluation results of minimum distance between two buildings of the ICC data set per tester (system); the red indicates situations of violation; black circles highlight clusters of the violation Indeed in most of the generalisation solutions (Figure 17a to Figure 17c), violation of the minimum distance between two buildings tends to occur in clusters along the coastal regions (see highlighted regions in black circles). It seems that all solutions in this case except the one generated by ICC failed to respect the minimum distance constraint. However, these violations may also be a consequence of 85 relaxing this constraint to meet more important constraints (e.g. keep the sizes of buildings). Consequently this is a good example of how considering single constraints does not enable to draw meaningful conclusions as to how can the generalisation systems respect this specific constraint. Many buildings with topological errors were introduced in the solution derived by IGNf (Radius Clarity), where building blocks intersect roads in the end result. This happens mainly in the clusters in Figure 17a. See also Figure 31, Section 3.6.1. Minimum distance between two buildings: IGN test case Results for IGN test case are presented in Figure 18. Figure 18 Average constraint violation of minimum distance between two buildings (upper) and selection ratio (lower) of the IGN data set per generalisation system and tester The following conclusions can be drawn for the outputs for IGN test case concerning the minimum distance between buildings constraint (axpand results are missing): Average Constraint Violation of Minimum Distance Constraint between Two Buildings of the IGN Data Set 0 0.25 0.5 0.75 1 IGNf (tester) ICC ITC Kadaster Clarity (system) CPT (system) Generalisation Systems & Testers AverageConstraintViolation Selection Ratio of Generalisation Outputs of the IGN Data Set (the initial data set contains 2019 building objects) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IGNf (tester) ICC ITC Kadaster Clarity (system) CPT (system) Generalisation Systems & Testers SelectionRatio 86 In general, high quality solutions concerning the minimum distance constraint between two buildings were derived by three testers (ICC, ITC, and Kadaster) with CPT. The solution generated by IGNf with Radius Clarity is on the other hand “nearly bad”. Although the solution by ICC with CPT software is the best of all four solutions, this only means that this solution respects this particular minimum distance constraint in a good way. The rather low selection ratio of this solution (98 buildings out of 2019, see also Figure 19b) may also indicate that other constraints like preservation of structures or black-white ratio were violated. The differences between selection ratios among IGNf, ITC, and Kadaster testers are not very significant, especially the results of IGNf and Kadaster tester applied similar selection ratio (Figure 18, lower). However, the solution by Kadaster with CPT shows a greater advantage over the one by IGNf with Radius Clarity concerning the minimum distance constraint. Hence we can assume that the Kadaster tester was able to express this constraint with CPT system while IGNf failed to do so with Radius Clarity. As a visual support of this, a closer look at the solution by IGNf tester shows that a topological error (proximate building polygons intersect) was introduced to the output data set. In fact, it seems to be the case that parts of the initial buildings were enlarged to meet the minimum area constraint, but none of these enlarged buildings were displaced or typified to reduce the violation of minimum distance between two buildings. A common observation that can be applied to all four solutions is that the violation tends to occur in a cluster (highlighted in Figure 19). This is mainly because the density of the initial buildings in this cluster is relatively higher. (a) IGNf (Radius Clarity) (b) ICC (CPT) (c) ITC (CPT) (d) Kadaster (CPT) Figure 19 Evaluation results of minimum distance between two buildings of the IGN data set per tester (system); the red indicates situations of violation; the black circle highlights a cluster of the violation Minimum distance between two buildings: Kadaster test case Results for Kadaster test case are shown in Figure 20. 87 Figure 20 Average constraint violation of minimum distance constraint between two buildings of the Kadaster data set per generalisation system and tester The following conclusions can be drawn for the outputs for Kadaster test case concerning the minimum distance between buildings constraint: In general, all generalisation solutions for Kadaster test case respect the minimum distance between two buildings rather well. Six out of seven solutions were well done (marked as “good”), while only one generated by the Kadaster tester using CPT is marked as “nearly good”. All tests performed in all three generalisation systems (Radius Clarity, CPT, and ArcGIS) provided good results. However, we cannot conclude here whether it is because all these systems can respect this constraint properly (e.g. displace buildings to avoid their getting too close), or it is because these systems were just ignoring this constraint. If we have a closer look at the initial data set in the Kadaster test case, we see that two other reasons for this high quality may also be possible. First, the density of buildings in the initial data set is in general not very high (i.e. the buildings are not very close to each other). Secondly, due to a specific constraint in Dutch case – building blocks should become “built-up area” if their densities > 0.1, many of the relatively denser areas were transformed into parcels of “built-up area” type in the solutions (see also Section 0). As a result of these two reasons, high quality concerning the minimum distance constraint can be expected even without explicit consideration of this constraint. Compared to solutions by the other testers, the solution generated by the Kadaster tester with CPT software appears to have a much higher degree of CVaverage than the others, although the quality of the solution is still acceptable (marked as “nearly good”). This may be because in the output generated by the Kadaster tester, much more buildings are selected than in the other outputs (see visual results in Figure 21; the selection ratio per generalisation solution is also quantified in Figure 22), and that many dense areas were kept where violation of the constraint is high. However, this violation may also be the result of interaction among constraints, that is, relaxing the minimum distance constraint in order to compromise with other constraints (e.g. spatial distribution). Average Constraint Violation of Minimum Distance Constraint between Two Buildings of the Kadaster Data Set 0 0.25 0.5 0.75 1 IGNf (tester) IGNs OSGB ICC ITC Kadaster ICC Clarity (system) CPT (system) ESRI (system) Generalisation Systems & Testers AverageConstraintViolation 88 IGNf (Radius Clarity) IGNs (Radius Clarity) OSGB (Radius Clarity) ICC (CPT) ITC (CPT) Kadaster (CPT) ICC (ArcGIS) Figure 21 Visualised evaluation results of minimum distance constraint between two buildings of the Kadaster data set per tester (system); the red indicates situations of violation To get more insight into the relationship between selection ratio and the violation of the minimum distance constraint between two buildings, selection ratios of all outputs of the Kadaster data set are visualised in Figure 22. 89 Figure 22 Selection ratio of all outputs of the Kadaster data set The initial data set contains 4274 building objects. Except for the solution derived by Kadaster where over 40% buildings were selected, all the other solutions have selection ratios all below 10%. According to results in Figure 20, those solutions with selection ratio below 10% are evaluated as “good” concerning the constraint, while the one by Kadaster (selection ratio > 40%) is regarded as “nearly good”. If we compare the distribution of selection ratio (Figure 22) and that of average constraint violation (Figure 20) per generalisation solution, an observation is that there appeared to be a positive correlation between selection ratio and the average constraint violation in the Kadaster test case, while few variations can be observed. For example, although selection ratio of the solution derived by IGNf using Radius Clarity is the lowest and the quality of this solution is very high concerning the constraint, the solution is not of highest quality of all the outputs (again we can conclude the triviality that the more buildings are kept, the more difficult it is to meet the minimum distance between buildings constraint). Findings and conclusions of automated evaluation of minimum distance between two buildings Firstly, only CPT software achieved good solutions concerning the minimum distance constraint for all test cases and all testers, while the other systems (i.e. Radius Clarity, ArcGIS; axpand was not considered) failed to achieve acceptable solutions for all test cases except for the Kadaster test case. Secondly, the observed violation of minimum distance constraint between two buildings appeared to have a positive correlation with selection ratio for all the test cases. As mentioned earlier, the more deleted buildings the higher the chance of satisfying this particular minimum distance constraint. It is however worth mentioning that a deletion of objects is a simple way to satisfy this constraint. In an extreme case all objects could be deleted, which would satisfy this constraint in a perfect way. This is definitely not what one would expect for a good generalisation solution. Of course, the characteristics of the initial data set also influence constraint violation. For example, in the Kadaster test case (a rural area) where the buildings blocks with relatively higher density should be transformed into “built-up area” and the density of the remaining buildings is relatively lower, all the generalisation systems and testers obtained good results concerning this constraint; whereas in the ICC test case, the violation of the constraint is on average higher due to buildings that are too close to each other in the coastal regions. An additional observation is that although isolated consideration of a single constraint does not enable to draw meaningful conclusion as to how can the generalisation systems respect this Selection Ratio of Generalisation Outputs of the Kadaster Data Set (the initial data set contains 4274 building objects) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IGNf (tester) IGNs OSGB ICC ITC TDK ICC Clarity (system) CPT (system) ESRI (system) Generalisation Systems & Testers SelectionRatio 90 specific constraint, a closer look at all the solutions shows that the CPT software is able to respect the minimum distance constraint between two buildings. $XWRPDWHG HYDOXDWLRQ RI PLQLPXP GLVWDQFH EHWZHHQ EXLOGLQJV DQG URDGV This section interprets and discusses automated evaluation of the minimum distance between buildings and roads. For evaluating this cartographic constraint, map objects to be evaluated should be in their symbolic forms. That is, road features represented by lines in almost all the data sets have to be assigned with a signature to represent the width of their symbols defined by the NMAs. Only with the signature the evaluation system can “see” the symbols and take the widths of road symbols into account. Specifically for this evaluation a lack of results balanced over all test cases, limits the reliability of the results. However, as will be shown below, the results do provide important insights with respect to the automated evaluation method itself. Therefore we added this section in the report without disclosing the system names. In the evaluation of the minimum distance constraint between buildings and roads, the calculation of CVaverage is explained as follows. Since two different feature classes were involved in the evaluation, the number of all violated objects (including violated buildings and violated roads) was firstly counted, notated as |VO|. Then the number of all map objects being evaluated (including all buildings and roads) was counted, notated as |O|. The average constraint violation is computed as: CVaverage =|VO|/|O|. The results of this evaluation are presented in Table 19. Due to the complexities of this evaluation, that is, a number of subclasses constitute the road class and they are in separate shape-files (often even in more files than the initial data set), the evaluation has to be carried out on a subclass-by-subclass basis. However, the evaluation in this way does not show a holistic view of minimum distance between building class and road class. In order to address this problem, different types of road (shape-files) were combined together into one shape-file while keeping each of their signature widths unchanged, before another evaluation test could be carried out. The results for Kadaster and IGN test case are presented in Table 19. The first two parts of this table are the results before the combination of different types of roads, while the third part is after the roads were combined. The following paragraphs will discuss these three parts respectively. Test case System Tester Thematic class 1 Thematic class 2 Signature width (map mm) No. of map objects (building +road) CVaver age Quality Kadaster (threshold: 0.02) S1 IGNf Building Local Road 0.6 mm 174 + 276 0.16 good Kadaster S1 IGNf Building Motorway 1.3 mm 174 + 48 0.00 good Kadaster S1 IGNf Building Other Road 0.6 mm 174+1596 0.09 good Kadaster S1 IGNf Building Platform 0.6 mm 174 + 15 0.04 good Kadaster S1 IGNf Building Runway 0.6 mm 174 + 6 0.00 good Kadaster S1 IGNf Building Street 0.6 mm 174 + 562 0.10 good Kadaster S1 IGNs Building Part of Road 0.6 mm 372 + 44 0.08 good 91 Test case System Tester Thematic class 1 Thematic class 2 Signature width (map mm) No. of map objects (building +road) CVaver age Quality In case of the IGN dataset, there are some contradictions in its symbolisation template, especially for Total width and Symbol schema. Finally, the following values are used for this evaluation (see Signature width column). IGN (threshold: 0.1 mm) S1 IGNf Building Major Road 0.9 mm 1019 + 34 0.11 good IGN S1 IGNf Building Secondary Road 0.7 mm 1019 + 96 0.14 good IGN S1 IGNf Building Minor Road 0.6 mm 1019 +346 0.45 nearly good IGN S1 IGNf Building Street 0.6 mm 1019 +208 0.50 nearly good The following results are obtained through combining different types of road together into one shape file (pre-processing), to obtain a holistic view of the violation of minimum distance between Building and Road classes. The values of Signature width (in the bracket) used for different types of road are corresponding to the type information respectively (also in bracket). IGN (threshold: 0.1 mm) S1 IGNf Building Road (major, secondary, minor and street) (0.9, 0.7, 0.6, 0.6) mm 1019+684 0.73 nearly bad IGN S2 ICC Building Road (major, minor) (0.9, 0.6) mm 98 + 380 0.15 good IGN S2 ITC Building Road (major, secondary, minor and street) (0.9, 0.7, 0.6, 0.6) mm 555 + 684 0.39 nearly good IGN S2 Kadaster Building Road (major, secondary, minor and street) (0.9, 0.7, 0.6, 0.6) mm 865 + 684 0.55 nearly bad Kadaster (threshold: 0.2 mm) S1 OSGB Building Road (all types) Different signatures 360+2224 0.26 nearly good Table 19 Average constraint violation, quality assessment of minimum distance between buildings and different subclasses of road class 92 Minimum distance between buildings and roads respecting different road types: IGN test case (a) Major road (b) Secondary road (c) Minor road (D) Street Figure 23 visualised evaluation results of minimum distance between building and different types of roads in the solution of IGN data set generated by IGNf with System1; the red colour denotes violation situations Only one solution generated by IGNf tester with System1 was evaluated concerning the minimum distance between buildings and different types of roads. Note that only buildings and roads at risk of constraint violation are visualised in Figure 23. The subfigures in Figure 23 show that spatial proximity and densities of road networks influence greatly the violation of minimum distance between buildings and roads. Table 19 also shows an interesting trend of an increase of the average constraint violation with the decline of roads in their thematic levels. For instance, constraint violation between buildings and streets is larger (0.50) than that between buildings and major roads (0.11). This is probably caused by the fact that in this specific generalisation solution, minor roads and streets are much closer to those inner-city areas where building densities are higher. In addition, these minor roads and streets form dense network clusters which cause more conflicts with buildings. Nevertheless, these explanations cannot be generalised to cover many other cases since the samples are too small. Minimum distance between buildings and roads respecting different road types: Kadaster test case The solution of the Kadaster data set generated by IGNf (or IGNs) with System1 was evaluated where the various types of road were considered separately. The results show good values for meeting the minimum distance between buildings and different types of roads. The reasons for this are several. A first observation is that buildings on the one side and both motorways and runways on the other side are never too close to each other. This is because these two types of roads are situated far away from both the non-generalised and the generalised buildings. Consequently few or no violations of this constraint can be expected between these two types of roads and buildings in this specific data set. In addition, higher constraint violations are observed between buildings and local roads, other roads, and streets respectively, although all of the three are still regarded as “good”. In these three cases more spatial interaction between buildings and roads can be expected. The good results are achieved for several reasons: 1) transformation of dense areas in “built-up” areas (see highlighted regions in Figure 24); 2) deletion of buildings (whereby it is not known which constraint caused the deletion - minimum area, minimum distance, both together, or other constraints); 3) displacement of objects (the minimum distance constraint do not say how many buildings/streets were displaced). 93 Figure 24 The interaction between buildings and roads in the solution of the Kadaster data set by IGNf with System1; highlighted regions denote areas where buildings are transformed into “built-up” areas. Minimum distance between buildings and roads aggregated for all road types (Because of the limit number of available results we did not divide this section in different test cases.) In the above two cases, the minimum distance between different types of roads and buildings were evaluated separately against buildings. Although such a separation leads to the insights of which types can cause the violation of the minimum distance constraint, we also need an aggregated measure, for example to compare the evaluation that does respect different road types in the solution generated by IGNf with System1 (the second part of Table 19) with the aggregated evaluation of the same solution (the first row from the third part of Table 19). For the aggregated evaluation, all types of roads were merged into one shape-file while their different signature widths remained. An unexpected result of this evaluation is that the aggregated evaluation reported a constraint violation (0.73) between buildings and all types of roads. This violation is much larger than any of the constraint violations observed in the separate evaluations: major roads (0.11), secondary roads (0.14), minor roads (0.45), and streets (0.50)), respectively. We assured that this discrepancy is not caused by some error introduced in the evaluation process and looked into more detail in a possible explanation using a simple example as shown in Figure 25. In this example, a spatial separation of Type I and Type II Road is made to simplify this example. Such a separation is not necessary in real datasets, but different road types indeed influence different subsets of the whole building set, i.e. subsets of buildings violating the minimum distance constraint caused by different types of roads are sometimes different. A closer look at Figure 23c and Figure 23d supports this idea. The idea is idealized in this example (see Figure 25). 94 Fig th Let Sup CVa The ficti also If w toge low dista This lead cons a co has Star syst type aggr From anal gure 25 An ex his example s types of us first consi pose all 400 b average = (40 en we consider ive evaluation o will results in we consider no ether, we will er cluster) an ance constrain s discrepancy d to a differe straint. In othe onstraint-based different them rting from this tems were eva es of roads (se regated view o m this Figure lyse both test xample to exp solution cons f roads (both ider the evalu buildings in th 0+100) / (100 r the evaluatio n all 400 build n CVaverage ow the evalua see that unde d 200 roads ( nt. Consequen means that th ent conclusion er words, one d evaluation b matic values. s knowledge aluated and c ee the result in of the constra e we can draw cases (IGNF a plain the disc ists of 1000 b Type I and T uation of the he upper cluste 00+100) = 0.4 on of the mini dings in the lo = 0.45. ation of minim er the same c (100 are of Ty ntly CVaverag he evaluation b n than the ag e should pay a based on a sep five generalis ompared conc n the third par int violation o w the followi and Kadaster) crepancy betw buildings loca Type II Road minimum dis er where too c 5 imum distanc ower cluster a mum distance conditions 800 ype I and 100 ge would resul based on a sep ggregated ev attention to suc parate or aggr sation solution cerning the m rt of Table 19 of these solutio ing conclusio ) together. ween separat ated at three c d contains 100 stance betwee close to all 100 e between bui are too close t e between bu 0 buildings (4 0 are of Type lt in (800+200 paration betw aluation conc ch differences regated evalua ns generated b minimum dista ). The results ons. ons. Because e and aggreg clusters (in ci 0 roads respe en buildings a 0 roads of Typ ildings en Typ to all 100 Typ uildings and th 400 in the upp e II) are violat 0) / (1000+200 ween different cerning the m s whether she/ ation, especial by different te ance between visualised in of the limited gated evaluati ircles) and tw ectively). and Type I ro pe I: pe II roads. In pe II roads, w he two road t per and 400 in ting the minim 0) = 0.83. types of roads minimum dist /he wants to m lly when a fea esters and the n buildings an Figure 26 giv d results, we ion; wo oads. n this which types n the mum s can tance make ature e two nd all ve an will 95 Fig On all t Acc IGN the o The Nev conc at Figu duri Sim (hig Figu A di in ce Figu Find road From seve Firs gene dista depe find road deri aggr aggr cons gure 26 Avera the whole, the he test cases ( ceptable soluti N data set, acc other two test e solution deri vertheless, iso clusions as to ure 27b show ing the genera milar clusters ghlighted regio ure 27). This c ifference betw entral part (al ure 27). dings and co ds m the automa eral conclusio t, an obvious eralised build ance between end largely on ding is that alt d types may g ived from sep regating all r regated appro straint betwee age constrain roads of outp e two systems (data sets) con ion was obtain eptable soluti ters failed to g ived by ICC w olated conside how can the ws that this go alisation (only of violation ons in can be explain ween these clu so highlighted onclusions of ated evaluatio ns. s finding is dings, or high n buildings an n the condition though evalua ive insight int arating differe road types. In ach may be p en buildings a nt violation of puts per gene s were able to ncerning the m ned by the OS ons were obta generate accep with System2 eration of a system and/or ood solution m y 98 buildings situations a ned by the pro usters is that in d) is higher th f automated e n of the mini that high den h density of b nd roads cons n of the initial ating the viola to which types ent types of ro n the case of referred. Still and roads (i.e. f minimum d eralisation sys o generate goo minimum dista SGB tester w ained by both ptable solution 2 seems to be single constr r tester really may be explai were selected are observed oximity of den n the solution an other solut evaluation of imum distance nsity of gene both, brings m straint. This m l data set and ation between s of roads may oads may diff f identifying , based on iso not taking in distance betwe stems, testers od solutions b ance constrain ith System1 f the ICC and I ns concerning the best solu raint does no respect the sp ined because d out of 2019 i for most so nse building cl derived by IG tions of the IG f minimum di e between roa eralised roads more difficulti means that re those objects n buildings an y cause spatia fer considerab which soluti olated conside nto account ho een buildings s and data set but not by all nt between bui for the Kadast ITC testers wi this constrain ution concerni ot enable to pecific constra many buildin initial building lutions of th lusters and de GNf with Syst GN test case (s istance betwe ads and build s network, or ies to respect sults of the g having been s nd roads by se al conflict, the bly from the r ons are bette eration of the m ow this constr s and all type t. the testers no ildings and ro ter data set. In ith System2, w nt. ing the constr draw meanin aint. A closer ngs were rem gs). he IGN test ense road netw tem1, the viola see een buildings dings we can d r high densit ting the minim generalisation selected. A se eparating diffe e evaluation re results obtaine er than others minimum dist raint interacts es of or for ads. n the while raint. ngful look oved case works. ation and draw ty of mum will cond ferent esults ed by s, an tance with 96 other constraints) one cannot make meaningful conclusions as to how the generalisation systems can respect this constraint. More importantly, for this evaluation most time was spent to design the methodology and build the prototype. Therefore the results of the automated constraint-based generalisation must not be used to analyse possibilities and limitations of individual software. Instead they provide insights into differences between solutions of one test case (second research question) and into automated evaluation and possible biases of the method. These last findings can be used for future research to fine-tune the method. (a) IGNf (System1) (b) ICC (System2) (c) ITC (System2) (d) Kadaster (System2) (e) OSGB (System1) Figure 27 Visualised evaluation results of the minimum distance between buildings and all types of roads per generalisation system, tester, and test case: (a) - (d) are solutions of the IGN test case; (e) is a solution of the Kadaster test case; the red indicates violation situations; black circles highlight clusters of violation situations. 97 3.6 Evaluation by comparing generalised outputs: results and conclusions The focus zones that were chosen to compare outputs for one test case are presented in Figure 28. Each focus zone covers one or more locations in the dataset and is associated with a particular known generalisation problem on which the visual comparative assessment concentrates. This section describes the results for the comparative analysis of each zone by presenting the problem of the focus zone, the initial data, what was expected according to the specifications (the NMA specific constraints referenced here can be found in Appendix VI), test outputs, comments/questions with respect to the outputs, and findings and main conclusions of the visual comparison. Apart from the output maps, also the output layers and the constraint expression templates have been studied to understand the reasons for the differences between outputs of the same test case. The section ends by presenting main findings and conclusions of this evaluation. This section presents many visualisations of the output data. Please note that the coloured figures available in de PDF version of the report may be required to be able to understand the details of this evaluation. 98 ICC OSG C test case GB test case Figure 288 Focus zones IGN Kad s selected for N-F test case daster test cas comparison e e evaluation 99 3.6.1 ICC – Town centre blocks and streets representation (selection, aggregation). Initial data: Figure 29 Initial data What was expected according to the specifications: &RQFHUQHG GDWD building (area), block (area), flower bed (area), sport ground (area), building site (line): border of a block, ensures the continuity of the block border at the places where there is no building inside the block adjacent to its border. The ICC constraint definition template contains constraints on the buildings sites, but the expected output data were the blocks, not the buildings sites. Thus the testers had to translate the constraints on the buildings sites into constraints on the blocks. This has also been done in the summary of the defined constraints below. 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ICC-1-32-49, ICC-1-57-61, ICC-2-15-18, ICC-3-16-19, additional constraint on blocks interdistance sent by email. Individual buildings : presence/absence, minimum size, granularity (minimum dimension of small details), squareness; preservation of size, shape and orientation. Individual flowerbeds : presence/absence, minimum size, granularity Individual sport grounds : presence/absence, minimum size Two buildings : minimum distance Two blocks : minimum distance Groups of buildings : preservation of density and spatial distribution inside a block, preservation of alignments, preservation of the main orientation of an alignment Group of blocks : preservation of distribution and connectivity of the streets (interblock space) during blocks aggregation Building and flowerbeds inside a block : under certain size conditions, flowerbeds should be eliminated and building enlarged (or the contrary) D C A E B 100 )RU LQIRUPDWLRQ H[WUDFW RI WKH ,&& SDSHU PDS Figure 30Extract of ICC paper map Outputs on this zone: See next pages. Comments/questions with respect to the outputs: None. Results of comparative evaluation: First, it is interesting to notice that the two results obtained by the ICC tester(s) (who are expert of ICC specifications) with CPT (as expert tester) and ArcGIS (as novice tester) are quite different from each others, and that they are also quite different from what has been obtained by the other testers of CPT respectively ArcGIS. Most likely this is because the ICC tester tried above all to satisfy the constraint on the minimum distance between blocks, and thus used lots of displacement and aggregation (see the first two paragraphs below). ,QWHUEORFN D JJUHJDWLRQ All the outputs have kept the initial aggregation level, except the output produced by the ICC tester on ArcGIS. According to the specifications, more aggregation should have been performed (to respect the constraint on blocks inter-distance), but the tested systems do not enable it. The attempt performed on ArcGIS by the ICC tester (novice) results in a complete loss of the streets patterns, which is contrary to the specifications (see zones C, D and E). The Kadaster tester on ArcGIS did not use the aggregation algorithm probably for this reason. %XLOGLQJV VL]H In almost all the outputs, in town centre the buildings have been aggregated and enlarged to completely cover the blocks (zone C and D), except in the output obtained by the ICC tester on CPT, where on the contrary the size of the buildings has been decreased (but not the size of the blocks). This is because the ICC tester is the only one who tried to increase the distance between the buildings and the median axes of the streets (i.e. to enlarge the streets). The displacement could theoretically be applied either on blocks and buildings, while the other ones should just follow to preserve the topological consistency. The ICC tester chose to apply it on buildings, and failed in preserving the topological consistency with the blocks. And finally the buildings seem to have been too much decreased: the resulting width of streets is 0.6 mm in average, against 0.3 expected and around 0.15 initially. )LQDO VKDSH DQG LQWHUGLVWDQFH RI WKH EO RFNV In the case where the blocks are finally completely covered by a building (see zone C), the blocks have not been moved compared to the original dataset, which means that the inter-distance between blocks is the same as the original one (most of the time not respecting the specifications). The only exception is the output obtained by the ICC tester with 101 CPT, since it can be considered that the buildings were displayed instead of the blocks (cf. just above, “buildings size”). In all the results, the building that completely covers the block is not topologically consistent with the block. In particular, the borders of the building sometimes pass through the borders of the blocks, thus locally reducing the width of the street. This case arises at a few places in the outputs obtained by the Kadaster tester on ArcGIS and the CPT tester on CPT, and much more often in the outputs obtained by the Kadaster and ICC testers on CPT, and by the IGNF tester on Radius Clarity. In order to avoid this problem, the buildings should at least have been clipped to the blocks (which is a simple GIS functionality). Let us notice that this has been attempted by the IGNF tester on Radius Clarity, rather successfully in zone C but not in zone E because of a bug. This has not been attempted in CPT because no clipping tool is provided. This is also true for ArcGIS but here the reason is not clear. In the output obtained by the OSGB tester on Radius Clarity, almost no generalisation has been performed. %XLOGLQJV VHOHFWLR Q In the outputs obtained with CPT, too many buildings have been eliminated, namely in zone D in small blocks containing only one building. The ICC tester notifies this problem and reports on problems of parameters setting in CHANGE and TYPIFY. In zone B (suburban space, scattered residential buildings), the results obtained by the different testers of CPT are very different: the number of kept buildings triple from the outputs obtained by Kadaster (novice) and ICC (expert) testers, to the outputs obtained by the ITC (novice) and CPT (vendor) testers. There might be a problem of misunderstanding the ICC specifications, but the ICC tester also reports on “too many eliminated buildings” and reports that “isolated buildings” are not unambiguously determined. On the output obtained by the OSGB tester with Radius Clarity (expert), a few buildings have been eliminated (not enough), sometimes not the best ones in terms of spatial distribution (see zone A). This is due to the lack of a real selection algorithm in Radius Clarity (only selection on size or semantic criteria can be done). No selection has been performed with ArcGIS either. %XLOGLQJV DJJUHJDWLRQ LQVLGH D EORFN Buildings aggregation inside a block has only been performed on Radius Clarity by the IGNF tester (expert), in order to solve buildings size and proximity conflicts. The results are poor (see zones A and E) because the aggregation algorithm is too poor to be used intensively, contrary to what is recommended by 1Spatial (the only criterion to aggregate two buildings is their inter-distance and the return aggregate is the MBR of the aggregated buildings). With no (or failing) clipping of the resulting aggregate on the initial block, it leads to aggregated building overlapping several blocks. The OSGB tester of Radius Clarity (also expert) has chosen not to use the aggregation algorithm, probably for this reason. The second reason might be that it is not clear if using the clipping algorithm provided in Radius Clarity should still be considered as an offthe-shelf use of Radius Clarity (an algorithm for clipping two geometries is part of the Radius Clarity API, but it needs to be encapsulated in order to be usable as a generalisation algorithm). %XLOGLQJ KROHV Small holes have successfully been removed by CPT. On ArcGIS, the testers could not directly translate this constraint in the system. The Kadaster tester reports that no tool exists to check the presence of holes. The ICC tester reports that the parameters of ArcGIS do not correspond to the parameters of the constraint. In Radius Clarity, the same case occurs as for clipping a building to the border of its block: an algorithm to keep only the outer line of a polygon is part of the API, but it has to be encapsulated to be used as a generalisation algorithm. The OSGB tester (expert) has considered that it was not part of the off-the-shelf version of Radius Clarity, while the IGNF tester (also expert) has done the encapsulation and used it. 102 Main conclusions: o specifications regarding the size of unique buildings in town centre blocks are not clear o the aggregation algorithms of Radius Clarity and ArcGIS provide bad results o in ArcGIS and Radius Clarity, buildings selection algorithms are missing o the CPT selection (typification with TYPIFY) is not straightforward to parameterize o all the software miss a good algorithm for streets selection and typification (block aggregation) o for a same software and (apparently) a same understanding of the specifications, the testers are more or less cautious, i.e. they prefer to perform almost no generalisation (with no generation of errors) to solutions where generalisation has been performed, but has also generated lots of errors. o Radius Clarity can provide very different results depending if one accepts to encapsulate some algorithms belonging to the API of the out-of-the box version or not (the OSGB tester has almost not modified the initial data). 103 (a) Initial (h) CPT, CPT tester (vendor) (b) Clarity, IGNF tester (expert) (c) Clarity, OSGB tester (expert) (d) ArcGIS, Kadaster tester (novice) (e) ArcGIS, ICC tester (novice) (f) CPT, Kadaster tester (novice) (g) CPT, ICC tester (expert) Figure 31 Test outputs of focus zone 104 3.6.2 ICC – Coastline generalisation Initial data: Figure 32 Initial data What was expected according to the specifications: &RQFHUQHG GDWD coast line (line), quay adjacent to sea (area), breakwater adjacent to sea (area), beach (line or area), rocky area (area) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ,&& ,&& ,&& ,&& ,&& Coast line : Granularity (minimum dimension of small details), noncoalescence (minimum dimension of small significant details), local shape preservation Quay : Presence/absence, minimum size, granularity, coalescence; collapse to line is required for thin parts Rocky area : Presence/absence, minimum size, granularity (minimum dimension of small details) Beach : Presence/absence, minimum size, granularity (minimum dimension of small details) Coast line and (rocky area, beach, breakwater, quay) : Adjacency preservation Coast line and (rocky area or beach) : Minimum distance 105 )RU LQIRUPDWLRQ H[WUDFW RI WKH ,&& SDSHU PDS Figure 33 Extract of ICC paper map Outputs in this zone: See next pages. Comments/questions with respect to the outputs: None. Results of comparative evaluation: The generalisation of the coastline is very minor on all the outputs, except in the output obtained by the IGNF tester (expert) with Radius Clarity, where a strong geometric simplification has been applied, with loss of shape and adjacency with other themes. *UDQXODULW\ RI WKH FRDVWOLQH The granularity constraint has been expressed by the testers of Radius Clarity and ArcGIS (either on the coastline or on the sea polygon) and could not be expressed on CPT. On Radius Clarity, both testers (IGNF and OSGB, experts) tried to solve this constraint by applying a Douglas-Peucker algorithm, not exactly in the same way. The Douglas-Peucker parameter represents a distance between the initial and filtered line, while in the specifications a minimum distance between vertices was given. The IGNF tester reports that he has chosen the parametric value of DouglasPeucker equal to the threshold indicated in the specifications (despite the fact it did not correspond to the same physical reality), but it is likely that there has been an error of a factor 10 on top of it (0.2 map mm instead of the required 0.02 map mm). This results in an over-generalisation and loss of shapes in the IGNF output (although no shape preservation constraint was explicitly set in the specifications for the coast line), and also in loss of consistency with other themes. The OSGB tester designed Radius Clarity to iteratively apply Douglas-Peucker with increasing parametric values, while controlling the progression of the vertices inter-distance. Although the OSGB tester was probably the closest to have really, fully expressed the constraint, the OSGB output is actually identical to the initial dataset. There might have been a problem during the execution of the generalisation process, or the constraint might have actually already been satisfied in the initial dataset (the required interdistance between two consecutive vertices, 0.02 map mm, is very low). On ArcGIS, the ICC tester (novice) also mentions that the parameter expected by the system is not coinciding with the parameters expressed in the constraint. A simplification has however been performed by both ArcGIS testers with a fixed parametric value. The result is correct compared to what was expected (the 106 coastline visually appears very granulose, but the minimum interdistance of 0.02 map mm i.e. 1 terrain meter is respected). 6HOHFWLRQ RI TXD\V The elimination of quays and breakwaters on length criteria has been managed in ArcGIS and Radius Clarity, not in CPT. &DULFDWXUH RI TXD\V The specifications give granularity and non-coalescence constraints for quays, i.e. they indicate under which conditions a small detail of a quay should be considered non significant and deleted, or on the contrary significant. For significant thin protrusions, a collapse to line is required. On CPT, no tester has been able to express these constraints because CPT does not provide tools to perform simplification or collapse on linear elements. On ArcGIS, the testers mention that there is no tool for the caricature of such man-made line features (detection of significant protrusion and collapse to line, elimination of others). On Radius Clarity, the two testers noticed that the constraints were theoretically designed for polygonal objects, while the objects were actually linear. Thus the OSGB tester ignored the constraints and the IGNF tester translated them to the sea polygon, but it is solved by means of a simplification that results in losses of consistency between the quay and the sea polygon. No collapse of the thin protrusions to lines has been done either. $GMDFHQF\ SUHVHUYDWLRQ DQG PLQLPXP GLVWDQFHV On CPT, contrary to Radius Clarity and ArcGIS the granularity and non-coalescence constraints could not be expressed, but constraints on adjacency preservation and minimum distances could be expressed. The minimal distance between the coast and islands could be expressed by all testers, although the outputs do not seem to be different on this aspect from the original data – was the constraint already satisfied? The proximity constraint between initially adjacent coast line and beach or rocky area has not been expressed by two testers and partially expressed by the ICC tester (expert) but it is not clear why. The adjacency constraint between coastline and all adjacent features could be expressed by all testers, however it is not really useful as no simplification of the coast line could be done, which is the operation that requires adjacency preservation. Main conclusions: Radius Clarity and ArgGIS allow to handle granularity constraints by simplifying the coastline with a line simplification algorithm, and to eliminate linear features on a length criterion, but they do not manage topological consistency between features, neither the proximity constraints between linear features. CPT, on the contrary, does manage adjacency preservation and proximity constraints, but it does not manage the simplification and caricature of line features. Both in Radius Clarity and ArcGIS, setting the parameters for line simplification according to granularity criteria expressed as minimal distance between two successive vertices is not straightforward. No software provides the adequate tools to caricature the quays (man-made features with thin protrusions) and to collapse some parts of them to lines 107 (a) Initial (b) Clarity, IGNF tester (expert) (c) Clarity, OSGB tester (expert) 108 (d) ArcGIS, Kadaster tester (novice) (e) ArcGIS, ICC tester (novice) (f) CPT, Kadaster tester (novice) 109 (g) CPT, ICC tester (expert) (h) CPT, CPT tester (vendor) Figure 34 Test outputs of focus zone 110 3.6.3 ICC – Generalisation of complex junctions Initial data: Figure 35 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Roads (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ,&& ,&& Between two roads : Minimum distance Interchange (composed of roads) : Minimum distance; displacement required, or simplification of shape if not enough space )RU LQIRUPDWLRQ H[WUDFW RI WKH ,&& SDSHU PDS Figure 36 Extract of ICC paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: None. Results of comparative evaluation: ,QWHUFKDQJH JHQHUDOLVDWLRQ CPT enables to displace roads away from each others thanks to the least square adjustment of PUSH. It results, in all the outputs obtained with CPT, in generalised interchanges where the proximity conflicts have been solved by displacement. This results in big displacements and parallel ramps far from each others (sometimes even further than the required 111 0.2mm). In some cases, in would probably have been better to remove some ramps (e.g. lower right ramp on the third extract, as done in the ICC paper map indeed), but CPT doesn’t enable to typify interchanges. Some self-intersections have also been generated by CPT: it is a known bug of PUSH. The vendor recommends decreasing the density of vertices to avoid it, but here it appears in round parts of ramps where the vertices density ensures the shape to keep its round aspect. Radius Clarity and ArcGIS do neither provide tools for roads displacement nor tools for interchange typification, thus nothing has been done. Besides interchange generalisation, in the result obtained by the Kadaster tester on ArcGIS some road segments are missing, resulting in important losses of connectivity in the network. It might be due to the elimination of short road segments performed thanks to an SQL query to satisfy a constraint on minimum length of a cul-de-sac road segments (constraint ICC-1-65). This constraint was expected to only concern cul-de-sacs, but the SQL query may have selected those segments as well for any reason (e.g. segment connected to road segments from another class, thus wrongly assessed as cul-de-sac). /HVV FRPSOH[ MXQFWLRQV The generalisation of roundabouts and branching crossroads is bad in all the outputs (first extract) or more precisely, they have not been generalised at all. This might be because of the fuzziness of the notion of “interchange”: although in the specifications the term “interchange” was intended for all kinds of junctions as soon as they are not simple crossroads, some testers might have interpreted as concerning “complex interchanges” only. As the interchanges are not made explicit under the form of “interchange objects” in the ICC data, the doubt was possible. Or they have not generalised them because no tool was adapted to their generalisation. In some cases (both testers on Radius Clarity, Kadaster tester on ArcGIS) the roundabout suffers from a loss of shape, probably due to the simplification done on road segments in order to decrease their granularity. $ UHPDUN RQ KRZ WKH WHVWHUV LQWHUSUHW WKH FRQVWUDLQWV DQG WKHLU WUDQVODWLRQ LQWR WKH V\VWHP The four CPT testers obtain similar results, but do not assess the translation of the constraint on interchanges (ICC-3-9) in the system in the same way. The Kadaster and ICC testers consider they have partially expressed the constraint (and not fully, because no use of the typification can be planned), whereas the CPT tester considers he has not expressed the constraint in the system at all, which would lead to expect no result on interchanges in the corresponding outputs. In the same time, the Kadaster and ICC testers consider they have not expressed the generic constraint on minimum distance between objects at all (ICC-2-3), while the CPT tester considers he has fully expressed it. So, in the Kadaster and ICC output, the result on interchanges seems to be due to efforts of the tester to generalise interchanges, whereas in the CPT output, the interchanges have been handled as a particular case of a generic proximity constraint. Main conclusions: CPT proposes a displacement tool for roads (PUSH), while Radius Clarity and ArcGIS do not. PUSH sometimes generates self-intersections locally. No software proposes tools for interchange detection and typification No software proposes tools for the simplification of less complex junctions Similar results do not mean that the constraints have been handled in the same way by the testers 112 (a) Initial (h) CPT, CPT tester (vendor) (b) Clarity, IGNF tester (expert) (c) Clarity, OSGB tester (expert) (d) ArcGIS, Kadaster tester (novice) (e) ArcGIS, ICC tester (novice) (f) CPT, Kadaster tester (novice) (g) CPT, ICC tester (expert) Figure 37 Test outputs of focus zone 113 3.6.4 ICC – Generalisation of suburban buildings Preservation of buildings spatial distribution and buildings alignment. Initial data: Figure 38 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Buildings (area), Roads (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ,&& ,&& ,&& Building : Presence/absence, minimum size, type of geometry; isolated buildings should be kept, buildings with size <0.16mm² can be collapsed to building symbols Road and building : Minimum distance; displacement required, unless building almost parallel to road Between two buildings : Minimum distance; aggregation or displacement required Group with high density of building symbols : Preservation of the shape of the group Building block : Preservation of density and spatial distribution Group of aligned buildings : Preservation of the alignment, preservation of its main orientation 114 )RU LQIRUPDWLRQ H[WUDFW RI WKH ,&& SDSHU PDS Figure 39 Extract of ICC paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: None. Results of comparative evaluation: 'HQVLW\ SUHVHUYDWLRQ The outputs preserve the original density, except the output obtained in CPT by the Kadaster tester and on ArcGIS by the ICC tester (too many buildings eliminated), and the output obtained on Radius Clarity by the IGNF tester (overdensity due to too many aggregations, which results in big buildings as in a town centre and thus a high black/white ratio). Among the three testers that could not generate satisfying outputs regarding density preservation, difficulties to express the constraint are only reported by the ICC tester on ArcGIS. The Kadaster tester of CPT seems to have tried to decrease the density (misinterpretation of the ICC specifications?) and considers that the constraint has been fully expressed. The IGNF Radius Clarity tester also considers he has fully expressed the density preservation constraint: either he is too optimistic, or there in a bug in the algorithms triggered by Radius Clarity to satisfy this constraint. 6L]H RI EXLOGLQJV The size of the buildings seems to be conform to what was expected (at least they are legible), except in the outputs obtained on Radius Clarity. On Radius Clarity, the IGNF tester has kept the result obtained with the standard tuning of Radius Clarity for building blocks generalisation (because the results were good in urban parts of the data set), but on scattered small buildings it gives bad results, aggregating them into big rectangle buildings. On the contrary, the OSGB tester has not kept this result and ends up with buildings that have not been generalised at all (thus too small). This shows the interest to detect areas of different buildings densities and to generalise them differently (something already done in research, with methods proposed by e.g. Boffet (2001), Gaffuri & Trevisan (2004), Chaudhry (2007), Steiniger (2008), and about to be used in production e.g. at IGN France). 3DWWHUQV SUHVHUYDWLRQ Based on a visual assessment of the outputs, it appears that the constraints for preserving the initial distribution of buildings have only been taken into account by CPT, thanks to the use of the typification process contained in “TYPIFY” (based on Kohonen features net). It is however interesting to notice that the Kadaster tester of CPT considers the constraint could not be translated into the system (contrary to the others CPT testers, who consider the choice of using TYPIFY as sufficient to claim the constraint has been expressed). With both ArcGIS and Radius Clarity, two situations arise depending on the strategy of the tester (more than their skills with the software). Either the tester considers it is impossible to express it (lack 115 of tool to measure spatial distribution), but it is compensated by the fact that the tester has not performed any decreasing of the number of buildings (considering no satisfying tool was available), which results in a pattern preservation de facto (Kadaster tester on ArcGIS, OSGB tester on Radius Clarity). Or the tester has still tried to decrease the number of buildings in order to solve the size, proximity and density constraints: ICC tester on ArcGIS and IGNF tester on Radius Clarity. Both testers consider the pattern preservation constraint has been partially expressed, but according to the outputs (bad preservation), it seems that only the idea to aggregate buildings could be expressed. Besides this, it can also be noticed that on the output obtained on ArcGIS by the ICC tester, where big aggregates of blocks have been created, the symbolisation makes it difficult to immediately differentiate between buildings inside a block (light grey) and holes in the block i.e. remaining interblock space (white). Main conclusions: Only CPT provides a typification tool (that ensures de facto a selection with pattern preservation) No system provide a way to measure the pattern preservation For a same software, the strategy varies from one tester to another between trying to satisfy “generalisation constraints” (that trigger generalisation) even if it means relaxing preservation constraints, or the contrary. In other words the testers are more or less cautious. 116 (a) Initial (h) CPT, CPT tester (vendor) (b) Clarity, IGNF tester (expert) (c) Clarity, OSGB tester (expert) (d) ESRI, TDK tester (novice) (e) ESRI, ICC tester (novice) (f) CPT, TDK tester (novice) (g) CPT, ICC tester (expert) Figure 40 Test outputs of focus zone 117 3.6.5 ICC – Parallelism between roads and buildings. Initial data: Figure 41 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Buildings (area), Roads (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ,&& ,&& Road and building : Minimum distance, relative orientation; if the difference of orientation is initially <15°, the parallelism must be enforced. In this case, if the initial distance is <0.15 map mm the building must be set adjacent to the road. )RU LQIRUPDWLRQ H[WUDFW RI WKH ,&& SDSHU PDS Figure 42Extract of ICC paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: None. 118 Results of comparative evaluation: No tester reports he has been able to fully express the constraints on relative orientation between roads and buildings. On CPT, according to the reports done by the testers, the parallelism between a building and its nearest road is enforced by the PUSH least squares process, only for buildings that are represented by a polygon (not for buildings collapsed to points). However, it seems that no threshold can be entered under which the parallelism should be enforced. On ArcGIS and Radius Clarity, no means is provided to enforce such parallelism, either for polygonal or punctual buildings (computation of symbol orientation, as the orientation of the closest road). Main conclusions: Only CPT provides a means to enforce parallelism between roads and buildings, but only for buildings represented by a polygon and without the possibility to parameterise it. 119 (a) Initial (h) CPT, CPT tester (vendor) (b) Clarity, IGNF tester (expert) (c) Clarity, OSGB tester (expert) (d) ArcGIS, Kadaster tester (novice) (e) ArcGIS, ICC tester (novice) (f) CPT, Kadaster tester (novice) (g) CPT, ICC tester (expert) Figure 43 Test outputs in focus zone 120 3.6.6 IGNF – Buildings (selection, interdistance, size). Initial data: Figure 44 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Buildings (area), Roads (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ,*1 ,*1 ,*1 ,*1 Individual buildings : presence/absence, minimum size, granularity (minimum dimension of small details), squareness; preservation of size, shape, orientation, positional accuracy, geographic meaning (aggregation should not result in many small individual houses looking like a big building). Between two buildings : Minimum distance, preservation of adjacency Road and building : Minimum distance, relative orientation; if the difference of orientation is initially <15°, the parallelism must be enforced. In this case, if the initial distance is <0.05 map mm the building must be set adjacent to the road. Buildings inside a block : Type of representation (density > 0.9 => block becomes a built up area); preservation of spatial distribution, distribution of characteristics, density. 121 )RU LQIRUPDWLRQ H[WUDFW RI WKH ,*1 SDSHU PDS Figure 45 Extract of IGN paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: None. Results of comparative evaluation: %XLOGLQJV VL]H The selective enlargement of buildings to ensure they reach a minimum size is possible in Radius Clarity and CPT, not in axpand and ArcGIS. Thus on axpand and ArcGIS, regarding the size constraints the only possible thing to do was to remove the buildings below 0.12 map mm² as required by the specifications. Regarding the preservation of the relative sizes between buildings, in the right part of the left picture the relative sizes are lost in all the outputs, except the ones where the size of the buildings has not been modified at all (Kadaster tester on ArcGIS, both testers on axpand). %XLOGLQJV VHOHFWLRQ On Radius Clarity and CPT, because of the enlargement performed on buildings it was needed to perform a building selection (contextual selection, preserving the initial distribution). As shown by the output obtained by the IGNF tester on Radius Clarity, this is not possible on the outof-the-box version of Radius Clarity: no algorithm for contextual building selection is available. The output obtained by the Radius Clarity tester (vendor) seems to indicate that the version customised by 1Spatial for the test includes such an algorithm. The results are more or less correct in dense zones (centre of village), where the initial distribution of buildings is homogeneous across the space. But the initial spatial distribution is lost in areas with scattered buildings. On CPT, the TYPIFY process enables to manage buildings selection while preserving the spatial distribution. But the parameter to indicate how many buildings to keep is difficult to tune, as reported by the ICC tester (expert) and as shown by the differences of buildings density in the outputs obtained with CPT. This is because CPT expects a “reduction rate”, while the specifications mention a target density defined on a block by block basis. %XLOGLQJV LQWHUGLVWDQFH ArcGIS, as well as Radius Clarity in its out-of-the-box version, do not provide any buildings displacement algorithm, which explains the poor quality of the outputs obtained respectively by the Kadaster and IGNF testers regarding road-building proximity. The version of Radius Clarity customised by 1Spatial (vendor) does include such an algorithm, which results in a correct taking into account of the road-building interdistance. The results are also correct with CPT and axpand (out-of-the-box versions), as they include displacement algorithms. Nevertheless, on axpand, because the buildings could not be enlarged (see above) this interdistance constraint is far easier to satisfy. Thus we cannot conclude on the quality of the buildings displacement algorithm provided by axpand. 122 Main conclusions: None of the tested software provides all the tools necessary to handle the size, density and interdistance constraints on buildings. CPT provides the largest panel but imposes to turn the small buildings into points. None of the tested software proposes a default transition function for the buildings sizes that preserves the relative values. Radius Clarity would enable it, but the Radius Clarity tester (vendor) has apparently not expressed this constraint, and the output obtained by the IGNF tester suffers too much from density and proximity problems to assess it. There is a problem in the way the parameters of the algorithms are expressed: the parameters express the transition expected between the initial and final state, considering that all the data should undergo the same transition (e.g. percentage of size increasing in ArcGIS, reduction rate for typification in CPT). Such parameters are difficult to match with specifications that indicate what is expected as a result, possibly depending on the initial state (e.g. absolute minimum size, density equal to the initial density). 123 (a) Initial (b) ArcGIS, Kadaster tester (novice) (c) Radius Clarity, IGNF tester (expert) (d) Clarity, Clarity tester (vendor) (e) CPT, Kadaster tester (novice) (f) CPT, ITC tester (novice) (g) CPT, ICC tester (expert) (h) CPT, CPT tester (vendor) (i) AXPAND, OSGB tester (j) AXPAND, ZURICH tester Figure 46 Test outputs in focus zone 124 3.6.7 IGNF – Mountainous roads (coalescence in bends series) Initial data: Figure 47 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Roads (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  IGN-1-23-26. One road : Non coalescence, avoid holes in the symbol; preservation of shape and positional accuracy. )RU LQIRUPDWLRQ H[WUDFW RI WKH ,*1 SDSHU PDS Figure 48 Extract of IGN paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: None. Results of comparative evaluation: ArcGIS does not enable to deal with road coalescence. The other three tested software do, in different ways. CPT does it through a global minimisation process based on least squares (PUSH): coalescence is thus handled as a proximity occurring inside an object. In the same way, axpand handles coalescence through a global process based on snakes. Radius Clarity handles it by splitting the road into homogenously coalesced parts and handling them separately with dedicated enlargement algorithms. On Radius Clarity, the IGN tester (who was novice with road generalisation) ended up with holes in the symbols, partly due to a parameterisation problem (bends on top of the figures), partly due to a bug in the algorithm that propagates the enlargement of one bend to its neighbours (bend in the lower 125 right corner in left picture). The Radius Clarity vendor output shows that these problems can be solved but the heads of the hairpin bends are rather too much enlarged. It seems that on axpand, the right parameterisation is difficult to find and the generalisation is not completely driven by the actual conflicts present in the data: either the shape is rather well preserved, but not all the coalescence conflicts are solved (OSGB tester output, one bend coalesced on left picture); or the coalescence conflicts are completely solved, but the heads of the hairpin bends are far too much enlarged (Zürich tester output). On CPT, all testers managed to solve the coalescence problems but as a side effect the bends tend to be rather too much enlarged (not optimal shape preservation). It is obvious in the output obtained by the ITC tester, picture in the middle, but it is true on all outputs either for thinnest or for largest roads. Surprisingly, on the outputs where the shape is best preserved for largest roads (Kadaster and ITC testers), the shape is far less well preserved for the thinnest roads, and vice versa (CPT and ICC testers). It seems to be difficult to find a parameterisation that fits for all categories of roads at the same time. The choice to enlarge the bends more or less is not only linked to the shape preservation constraint. It is also linked to the positional accuracy constraint, and all the CPT testers report that it is impossible to express it in CPT, which might explain why the obtained solutions are not completely optimal. Another possible explanation would be that the testers might have tried to preserve relative positions between roads and contours (at least true for the ICC tester). Finally there might be fuzziness in the specifications which caused that all results of the project team testers contain bends that open too much. However the vendor did manage to limit the opening by identifying a “0” distance (maybe it was not obvious that a “0” distance between the road symbol and itself was expected within a bend). It is also interesting to notice that, contrary to the other CPT testers, the Kadaster tester considers that the coalescence constraint could not be expressed in CPT, although the obtained results are very comparable to the results obtained by the other testers (and very acceptable). Main conclusions: ArcGIS does not handle coalescence In Radius Clarity there is a (known) bug in the out-of-the box version, apparently corrected in the vendor version In CPT and axpand, the heads of the hairpin bends are generally too much enlarged (right parameterisation difficult to find) For comparable outputs obtained with the same software, the testers do not assess identically if and how much they have expressed the constraints. 126 (a) Initial (b) ESRI, TDK tester (c) CLARITY, IGNF tester (d) CLARITY, CLARITY tester (vendor) (e) CPT, TDK tester (novice) (f) CPT, ITC tester (novice) (g) CPT, ICC tester (expert) (h) CPT, CPT tester (vendor) (i) AXPAND, OSGB tester (j) AXPAND, ZURICH tester Figure 49 Test outputs in focus zone 127 3.6.8 IGNF – Vegetation (selection and geometric simplification) Initial data: Figure 50 Initial data, the zone is rotated by 90° in anticlockwise direction, in order to be able to display all results on a same page What was expected according to the specifications: &RQFHUQHG GDWD Forest (area) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ,*1 Forest area : Shape preservation, minimum size of holes; reduce number of vertices by 50%. )RU LQIRUPDWLRQ H[WUDFW RI WKH ,*1 SDSHU PDS Figure 51 Extract of IGN paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: None. 128 Results of comparative evaluation: First, there is a problem in the specifications: no granularity (or minimum dimension) constraint is defined, but the constraint presented as a shape preservation constraint also includes a guideline that requires decreasing the number of vertices by 50%. Moreover, the required decrease of granularity is identical throughout the objects, probably because the initial over-granularity is known for being uniform. Only two testers report that they have expressed the constraints in the system: the Kadaster tester on ArcGIS and the IGNF tester on Radius Clarity. On CPT, the constraint was not expressed (only the ICC tester explains why, and the reason is that no simplification operator is available). On axpand, both testers report that they could not express the constraints. An explanation is given by the Zürich tester: the notion of preservation constraint does not exist in axpand. It seems to denote that the tester did not see the required decreasing of number of vertices, which is not surprising as it was “hidden” behind a preservation constraint. But in the same time, it can be noticed that in the output obtained by the Zürich tester lots of vegetation areas have been deleted (some small and some very large). The OSGB tester of axpand reports that he also observed (and did not keep) such a result while trying to apply a simplification algorithm on the vegetation areas. Main conclusions: A problem in the definition in the constraint has lead to misinterpretations. No simplification algorithm applicable to forest areas is available in CPT. The simplification algorithm available in axpand seems to have a bug and inappropriately eliminates many objects. 129 (a) Initial (b) ArcGIS, Kadaster tester (novice) (c) Clarity, IGNF tester (expert) (d)Clarity, Clarity tester (vendor) (e) CPT, Kadaster tester (novice) (f) CPT, ITC tester (novice) (g) CPT, ICC tester (expert) (h) CPT, CPT tester (vendor) (i) AXPAND, OSGB tester (j) AXPAND, ZURICH tester Figure 52 Test outputs in focus zone 130 3.6.9 IGNF – Ski lifts representation Initial data: Figure 53 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Ski lift (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  ,*1 Between two ski lifts : Minimum distance )RU LQIRUPDWLRQ H[WUDFW RI WKH ,*1 SDSHU PDS Figure 54 Extract of IGN paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: None. Results of comparative evaluation: Nothing could be done in Radius Clarity neither in ArcGIS, because there is no tool for displacement. In CPT, the ITC and ICC testers consider they have expressed the constraint while the CPT and Kadaster testers do not seem to consider they have expressed it at all, although the ski lifts have actually been displaced in the four outputs. We can notice that the final distance between ski lifts are far bigger in the ICC output than in the other three outputs, most probably because of a different A 131 interpretation of the symbol width (the ICC tester has considered the whole symbol including the small transversal lines, while the others three have only considered the main line). Actually the specifications were not precise on this point. This results in very distorted ski lifts, because there is definitely not enough space for all with such a big interdistance. In the ITC output, they are less displaced than in the ICC one, but still twice more displaced than in the CPT and Kadaster ones. All the CPT outputs (except the ICC one) introduced a distortion in the straight part of the ski lift in zone A (see above figure), probably because it was tried (and succeeded) to preserve the relative positions of the ski lift and the road (i.e. to avoid to create an intersection). Although this was not asked by the specifications, it was surely a good idea to avoid this. But a better solution should exist, that avoids creating this intersection while also preserving the straight shape of the ski lifts. The two testers of axpand obtain similar results. The ski lift in zone A has not been distorted, but the relative position with the road is lost. The Zürich tester denotes an ambiguity in the constraint (the final expected distance is 0.2 map mm, but a displacement is preconised only if the initial distance is >0.1mm – thus it is not obvious what should be done in the case of an initial distance <0.1mm). Main conclusions: Radius Clarity and ArcGIS do not provide any displacement tool Differences are noticed in symbol width interpretation Some testers tried to maintain relative positions between road and ski lift although it was not required by the specifications: idea of “universally admitted constraints”. No output does perfectly handle shape preservation and preservation of relative positions with roads For comparable outputs obtained with the same software, the testers do not assess identically if and how much they have expressed the constraints. 132 (a) Initial (b) ArcGIS, Kadaster tester (novice) (c) Clarity, IGNF tester (expert) (d) Clarity, Clarity tester (vendor) (e) CPT, Kadaster tester (novice) (f) CPT, ITC tester (novice) (g) CPT, ICC tester (expert) (h) CPT, CPT tester (vendor) 133 (i) AXPAND, OSGB tester (j) AXPAND, ZURICH tester Figure 55 Test outputs in focus zone 134 3.6.10 Kadaster – Channel network: selection Initial data: Figure 56 Initial data What was expected according to the specifications: &RQFHUQHG GDWD River (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  .DGDVWHU .DGDVWHU 5LYHU 3UHVHQFH UHTXLUHG XQGHU FHUWDLQ DWWULEXWH FRQGLWLRQV )RU LQIRUPDWLRQ H[WUDFW RI WKH .DGDVWHU SDSHU PDS WKH QRUWKHUQ WKLUG RI WKH ]RQH LV PLVVLQJ  Figure 57 Extract of Kadaster paper map Outputs on this zone: See next pages. Comments/questions with respect to the outputs: None. Results of comparative evaluation: None of the constraints defined by Kadaster requires a pruning of the channel network. Three testers have carried out such a pruning however: the two Radius Clarity testers after studying the paper map provided by Kadaster, and one Kadaster tester (on CPT) because he knows the Kadaster map 135 specifications. This shows that in some cases not only the constraints definition template has been taken as reference by the testers as the expected specifications. In the output obtained by the Kadaster tester on CPT the pruning has been done on attribute criteria, but in an external software (this is an error as the test protocol did not allow it), because CPT does not enable to perform selection based on attributes. In the output obtained by the IGNF tester on Radius Clarity, the selection has also been done on attribute criteria, but in Radius Clarity. Finally, the pruning done by the OSGB tester on Radius Clarity is lighter (more channels kept) and the used criteria are not obvious (possibly pruning of dead ends on length criteria). Theoretically this does not enable to conclude on the capacities of the tested systems regarding artificial networks typification. But it is likely that no system provides such tools, as this was reported by the testers for the generalisation of the street network in town centre in the ICC dataset. Main conclusions: A problem is encountered in the specifications (incomplete), which makes the outputs hard to evaluate. Although it cannot be categorically deduced from the outputs, it is likely that none of the tested software provides a typification tool for man-made networks. 136 (a) Initial (b) ArcGIS, Kadaster tester (novice) (c) ArcGIS, ICC tester (novice) (d) ArcGIS, ArcGIS tester (vendor) (e) Clarity, OSGB tester (expert) (f) Clarity, IGNF tester (expert) (g) Clarity, IGNS tester (novice) (h) Clarity, Clarity (vendor) Figure 58 Test outputs in focus zone (1/2) 137 (a) Initial (i) CPT, Kadaster tester (novice) (pruning done in external software) (j) CPT, ITC tester (novice) (k) CPT, ICC tester (expert) (l) CPT, CPT tester (vendor) (m) AXPAND, OSGB tester (novice) Figure 59 Test outputs in focus zone (2/2) 138 3.6.11 Kadaster – Settled area: building selection. Initial data: Figure 60 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Building (area), Glasshouse (area), Road (line/area), Land use (area) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  .DGDVWHU .DGDVWHU .DGDVWHU .DGDVWHU One building or glasshouse : Presence/absence, minimum size, granularity (minimum dimension of small details), prevent aggregation; important buildings (specific class) should be kept and enlarged if needed, other small buildings should be deleted. Two buildings : Minimum distance Building and road : Adjacency (if initially close) Group of buildings inside land use parcel : Representation depending on density: the land use parcel should become “built up area” if density > 0.1 139 )RU LQIRUPDWLRQ H[WUDFW RI WKH .DGDVWHU SDSHU PDS Figure 61 Extract of Kadaster paper map Outputs on this zone: See next pages. Comments/questions with respect to the outputs: None. Results of comparative evaluation: Creation of built up areas from dense enough land use parcels: This has not been performed on axpand (only one output available), because the tester did not find how to do it. As the tester is a novice of axpand, either axpand does not enable to do it or there is a problem of documentation. All other tested systems were able to do it (since at least one tester of the project team managed to do it), but on all the systems it seems to be hard: every time, not all of the testers have managed to do it, and often the testers who managed to do it report it was hard. It is also interesting to notice that not all the testers who have managed to create the built up areas are experts of the tested system, and not all the testers who have not managed to create the built up areas are novices. On CPT, it is to be noticed that CPT provides an attribute on meshes that marks them as “to be turned into built up areas”, but does not enable to create the built up areas, strictly speaking (i.e. to create objects in a class “built up area”). In other terms, it performs the “clever” part of the work, but in a production line a postprocessing using another system is required to do the remaining mechanical part of the work. When built up areas have been derived, they are not always the same. We can distinguish between three groups of outputs: (1) Outputs derived by the Kadaster tester of ArcGIS, the IGNF and Radius Clarity (vendor) tester of Radius Clarity (pictures b, f and h): contains the most built up areas (and the same across the three outputs). After investigation in the data, it seems that all the land use parcels for which the initial density of buildings is • 0.1 have been turned into built up areas. (2) Outputs derived by the OSGB tester of Radius Clarity and by the ICC tester of ArcGIS (pictures c and e): very few parcels (and the same in both outputs) have been turned into built up areas. This is probably due to a sequencing problem: it seems these two testers have turn into built up areas the parcels with a density greater than 0.1 after removing the buildings below the size threshold, whereas all the other testers have computed the density on the initial data, i.e. before removing the buildings below the size threshold. In one case the tester did not pay attention to the order in which the operations had to be done. In the other case, it is due to a misinterpretation of the specifications: confusion between how important it is that a constraint is satisfied (which was indicated in the 140 constraints definition template), and what constraint should be handled first (which was not indicated in the constraints definition templates). (3) Outputs derived by the ArcGIS (vendor) tester of ArcGIS and the Kadaster and CPT (vendor) tester of CPT: contain far more built up areas than in group (2) but a bit less than in group (1), and not exactly the same across the three outputs. Actually, the three concerned testers did not only consider the constraint definition templates provided by Kadaster to tune the systems. They also tried to obtain something similar to the paper map provided by Kadaster, and two of the three testers who provided this solution have also an extra expertise of the Kadaster specifications (namely the Kadaster tester and the ArcGIS tester, since ArcGIS had a collaboration with Kadaster). Now, in the case of the two CPT testers, the liberties taken with the reference thresholds have been in a way provoked by a necessity: the density considered in the Kadaster specifications was computed in the land use parcels (that do not include the part of blocks covered by the roads), while the only entity on which the density can be automatically computed in CPT is the mesh (i.e. block). Thus, the threshold had to be adapted anyway. The same happened in ArcGIS (the vendor reports having computed the density on blocks), although in this case it was not by necessity (the density could have been computed on the land use parcels). Buildings selection: The building selection is quite different from one output to another. This is mainly explained by two reasons. First, the buildings that end up inside a built up area, and that are above the size threshold, have not been handled in the same way by all testers: most of the testers have removed them, but some have kept them. This is due to an ambiguity in the specifications. Second reason, the built up areas are not the same in all the outputs (some outputs don’t even have any, in those ones all the buildings above the size threshold have been kept). Now, in the outputs classified in group (3) with respect to the creation of built up areas, not all the instructions present in the Kadaster specifications have been followed. If it had been the case, the spatial distribution of buildings would have been completely lost as complete rows of buildings were below the size threshold. Although there was no constraint of spatial distribution preservation, the three testers seem to have tried to preserve it (which was surely a good idea, cartographically speaking). On CPT, the selection and generalisation of the buildings was achieved by the two testers thanks to a combined used of the “Typify” and “Change” components of CPT. On ArcGIS, a prototype currently under development, called “Optimizer”, was used. In both cases the results are quite similar to the paper map provided by Kadaster (although they do not respect the specifications provided by Kadaster under the form of constraints). Main conclusions: Turning dense parcels into built up areas is possible in all the systems except maybe axpand, In fact for axpand we cannot conclude if it is possible to turn dense parcels into built up areas, but at least it is not straightforward: the only axpand tester, who was a novice, did not succeed to do it. In the other three systems, turning dense parcels into built up areas is possible but is still far from being straightforward. Differences in sequencing have led to very different results Ambiguity in the specifications regarding whether to keep buildings in the parcels turned into built up areas. The paper map provided by Kadaster was used as a reference by several testers. 141 (a) Initial (b) ArcGIS, Kadaster tester (novice) (c) ArcGIS, ICC tester (novice) (d) ArcGIS, ArcGIS tester (vendor) (e) Clarity, OSGB tester (expert) (f) Clarity, IGNF tester (expert) (g) Clarity, IGNS tester (novice) (h) Clarity, Clarity tester (vendor) Figure 62 Test outputs in focus zone (1/2) 142 (a) Initial (i) CPT, Kadaster tester (novice) (j) CPT, ITC tester (novice) (k) CPT, ICC tester (expert) (l) CPT, CPT tester (vendor) (m) AXPAND, OSGB tester (novice) Figure 63 Test outputs in focus zone (2/2) 143 3.6.12 Kadaster – Generalisation of parallel roads and cycle tracks. Initial data: Figure 64 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Road (line) Cycle track (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  .DGDVWHU .DGDVWHU One road : Presence/absence (attribute and length criteria), representation by keeping separate tracks or by a collapsed line. One cycle track : Presence/absence Parallel road and cycle track : Specifications incomplete – the document written by the CPT tester of CPT (vendor) seems to indicate that the cycle track can be eliminated if its width is low (2-4m) and the road belongs to a certain category. No minimum distance between roads (or road/cycle track), no constraint preventing overlaps. 144 For information, extract of the Kadaster paper map (the upper quarter is missing): Figure 65 Extract of Kadaster paper map Outputs on this zone: See next pages. Comments/questions with respect to the outputs: The symbol widths are different from one output to another, due to erroneous widths given first. Of course thinner widths facilitate the generalisation… Results of comparative evaluation: The specifications regarding what to do with parallel roads or roads/cycle tracks are fuzzy (constraint Kadaster-2-6 was incomplete). In this comparative analysis, we consider the constraint was to eliminate cycle tracks parallel to roads, which is what most testers seem to have understood (but which is still fuzzy: does not tell what to do with two parallel roads for instance). The selection has been attempted by the testers of ArcGIS using an SQL selection: the ICC tester considers having expressed the constraint fully, Kadaster tester partially because some other roads have been accidentally deleted, the vendor (ArcGIS tester) does not report on expressing this constraint. On axpand, the only tester reports being unable to express this constraint, and trying to do displacement instead. On Radius Clarity, the OSGB tester managed to do the selection by using the API to perform spatial selection, while the IGNF tester considered it impossible with the out-of-the-box version of Radius Clarity. On CPT, the CPT tester (vendor) performed the selection manually using ArcGIS, in order to avoid overdensity before road displacement. The ITC and ICC testers did not perform the selection at all and performed displacements. The Kadaster tester does not report doing anything regarding this constraint. The outputs in which some attempts of displacement have been done (ITC and ICC testers on CPT, OSGB tester on axpand) show distortion on the external parallel roads (difficulties to preserve parallelism) – it is to be noticed that the density of roads was locally very high (as explicitly mentioned by the CPT tester of CPT). Main conclusions: Fuzziness in the specifications regarding parallel roads, roads/cycle tracks Only axpand and CPT provide road displacement tools Displacement without any appropriate elimination before is definitely hopeless 145 (a) Initial (c) ArcGIS, ICC tester (novice) (b) ArcGIS, Kadaster tester (novice) (d) ArcGIS, ArcGIS tester (vendor) (m) AXPAND, OSGB tester (novice) Figure 66 Test outputs in focus zone (1/3) 146 (a) Initial (e) Clarity, OSGB tester (expert) (g) Clarity, IGNS tester (novice) (f) Clarity, IGNF tester (expert) (h) Clarity, Clarity tester (vendor) Figure 67 Test outputs in focus zone (2/3) 147 (a) Initial (i) CPT, Kadaster tester (novice) (j) CPT, ITC tester (novice) (k) CPT, ICC tester (expert) (l) CPT, CPT tester (vendor) Figure 68 Test outputs in focus zone (3/3) 148 3.6.13 Kadaster – Railways typification Initial data: Figure 69 Initial data, the zone is rotated by 90° in anticlockwise direction, in order to be able to display all results on a same page What was expected according to the specifications: &RQFHUQHG GDWD Railway (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  .DGDVWHU .DGDVWHU .DGDVWHU One railway : Positional accuracy (preservation) Two railways : Aggregation forbidden Group of parallel railways : Select main tracks (in other words, a pruning is required, but each kept feature is expected to actually correspond to one initial feature and be kept in place – no typification allowing an n-m matching as long as the contour of the group is kept) )RU LQIRUPDWLRQ H[WUDFW RI WKH .DGDVWHU SDSHU PDS WKH OHIW KDOI LV PLVVLQJ  Figure 70 Extract of Kadaster paper map Outputs on this zone: See next pages. Comments/questions with respect to the outputs: None. Results of comparative evaluation: No tester has modified the railways, except the OSGB tester on axpand (small distortions on short edges) and the ICC tester on CPT (some displacement with loss of shape, not necessarily done on purpose, they might also have been a side effect of a displacement of other types of features, and the railways have not been set as fixed although CPT enables it). No system provides any tool for network selection. 149 Main conclusions: No typification tool appropriate to railways available in any of the tested systems The only (small) modifications noticed on railways are undesirable side effects of displacement applied to other feature types (CPT and axpand). (a) Initial (b) Clarity, OSGB tester (expert) (c) Clarity, IGNF tester (expert) (d) Clarity, IGNS tester (novice) (e) Clarity, Clarity tester (vendor) (f) CPT, Kadaster tester (novice) (g) CPT, ITC tester (novice) (h) CPT, ICC tester (expert) (i) CPT, CPT tester (vendor) (j) ArcGIS, ICC tester (expert) (k) ArcGIS, ArcGIS tester (vendor) (l) ArcGIS, Kadaster tester (novice) (m) AXPAND, OSGB tester (novice) Figure 71 Test outputs in focus zone 150 3.6.14 OSGB – Adjacent buildings representation Initial data: Figure 72 Initial data What was expected according to the specifications (aggregation, simplification, shape preservation): &RQFHUQHG GDWD Building (area) Fence (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  26*%E 26*% 26*% Individual buildings : Granularity (minimum dimensions of small details), minimum size and position of holes Group of adjacent buildings : Should be aggregated into one big building. Group of fences : One fence out of two should be omitted. )RU LQIRUPDWLRQ H[WUDFW RI WKH 26*% SDSHU PDS Figure 73 Extract of OSGB paper map 151 Outputs on this zone: See next page. Comments/questions with respect to the outputs: Only the outputs obtained by the OSGB tester on Radius Clarity and by the ITC tester on CPT show the fences – and they have not been generalised. Results of comparative evaluation: No tester has managed to express the constraint on fences. It is probably on the one hand because it is very specific, on the other hand because there was an ambiguity on the class containing the fences in the initial data. The output obtained on ArcGIS is the only one that includes a simple aggregation of the adjacent buildings as required (operation sometimes better known as “dissolve”, which normally is a very standard operation for a GIS). Unfortunately, one row of buildings has been lost (block in the middle), the reason is not clear. Surprisingly, the tester does not report that he has translated this constraint. On CPT, an aggregation has been performed but the holes have been lost although they were far above the minimum size threshold. Although no constraint asking for a preservation of the holes has been defined, this was surely not expected. According to the testers, this is due to the Typify component of CPT, which systematically fills in the holes (Change preserves them). On Radius Clarity, the aggregation algorithm is not adapted to this kind of data. Main conclusions: No tester has been able to express the constraints on fences ArcGIS is the only system providing a tool for aggregation of adjacent buildings (‘dissolve’) The building aggregation algorithm of Radius Clarity is not adapted In CPT, the holes in buildings/groups of buildings are lost One case where a constraint seems to have been expressed (and satisfied) and the tester does not assess having expressed the constraint. 152 (a) Initial (b) Clarity, OSGB tester (novice) Not expert of OSGB specifications (c) ArcGIS, Kadaster tester (novice) (d) CPT, Kadaster tester (novice) (e) CPT, ITC tester (novice) (f) CPT, ICC tester (expert) (g) CPT, CPT tester (vendor) Figure 74 Test outputs in focus zone 153 3.6.15 OSGB – Detached buildings and fences Initial data: Figure 75 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Building (area), Fence (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  26*%E 26*% 26*% 26*% Individual buildings : Granularity (minimum dimensions of small details), minimum size and position of holes Rows of houses : The houses should be aggregated by two or three, or represented individually, depending on their initial interdistance Group of fences : Rules to omit a part of the fences, depending on the kind of houses (semi-detached, detached) and their interdistance )RU LQIRUPDWLRQ H[WUDFW RI WKH 26*% SDSHU PDS Figure 76 Extract of OSGB paper map 154 Outputs on this zone: See next page. Comments/questions with respect to the outputs: As in the previous focus zone, the fences are only present in two outputs out of six, and they have not been generalised. Not all the testers have reported if and how they have translated the constraints into the tested system (missing information namely for the Kadaster tester on ArcGIS and the CPT tester on CPT). Results of comparative evaluation: As in the previous zone, no tester has managed to express the constraint on fences. It is probably on the one hand because it is very specific, on the other hand because there was an ambiguity on the class containing the fences in the initial data. No aggregation has been performed on Radius Clarity because the only available aggregation algorithm gives bad results on such data. On ArcGIS, a dissolve seems to have been done (aggregation of adjacent buildings), as well as a systematic aggregation of very close buildings. It does not seem that the aggregation by two or three buildings has been carried out. The CPT testers have tried to express the rules to choose the aggregation level depending on the interdistance. They assess the expression of this constraint as hard. The outputs are different from one tester to the next. Main conclusions: No tester has been able to express the constraints on fences Only on CPT the constraint requiring an aggregation by 2 or 3 buildings has been partially expressed – assessed as hard to express 155 (a) Initial (b) Clarity, OSGB tester (novice) Not expert of OSGB specifications (c) ArcGIS, Kadaster tester (novice) (d) CPT, Kadaster tester (novice) (e) CPT, ITC tester (novice) (f) CPT, ICC tester (expert) (g) CPT, CPT tester (vendor) Figure 77 Test outputs in focus zone 156 3.6.16 OSGB – Dual carriageway representation, roads parallelism preservation. Initial data: Figure 78 Initial data What was expected according to the specifications: &RQFHUQHG GDWD Roads (line) 'HILQHG FRQVWUDLQWV DQG JXLGHOLQHV VXPPDU\  26*% Dual carriageway : Should be collapsed )RU LQIRUPDWLRQ H[WUDFW RI WKH 26*% SDSHU PDS Figure 79 Extract of OSGB paper map Outputs on this zone: See next page. Comments/questions with respect to the outputs: There was a problem with the initial roads classification: contrary to what was required in order to simplify the tests, in the initial dataset all the roads were in the same class and the testers had to reclassify them using an attribute. The ICC tester of CPT had a problem, lost the attribute and could not perform the reclassification, thus dealing with larger symbols than required on small roads. This emphasizes the problem encountered on the side effects of dual carriageway displacement reported below. 157 Results of comparative evaluation: None of the tested software provides a tool for collapsing the dual carriageways. However not all the outputs are identical. On ArcGIS and Radius Clarity, the testers have done nothing and the final data are strictly identical to the initial ones. It is the same in the output obtained with CPT by the Kadaster tester, who chose to do nothing. Still on CPT, the CPT tester reports that the collapse has been performed manually, “in order to have correct input for road and building displacement”. It was indeed required to avoid problems while displacing roads and buildings. This was specific to CPT as neither ArcGIS nor Radius Clarity provide usable roads and buildings displacement algorithms in their out-of-the-box version. Contrary to the CPT tester, the ICC and ITC testers of CPT have reported that the collapse was impossible due to a lack of tools, but they still have tried to do road displacement (in order to satisfy other constraints). As a result, the two components of the dual carriageways have been moved away from each others (contrary to the specifications), and have also caused losses of parallelism on neighbouring roads, which is bad even if the OSGB specifications do not explicitly require to avoid it (note that for the ICC output, the big symbol widths increase the phenomenon – see comment above). The fact that two testers of CPT had this problem, and that the vendor felt it necessary to perform the collapse manually, denotes that the displacement tool of CPT (push) does not enable to “freeze” the components of the dual carriageways with regard to each other (i.e. to freeze their relative positions). As the dual carriageways are in a separate class, it probably means that it is impossible to express a constraint like “do not try to achieve a minimum distance between an A-road an another A-road” or “try to preserve the initial distance between any A-road and any other A-road that is initially close to it”. What is possible in CPT is to freeze some objects so that they do not move but still push their neighbours. It could have been used here for the dual carriageways components in order to limit the problems, but no tester chose this option, either because they did not know it (problem of expertise and/or documentation) or because they thought it was not the best option. Main conclusions: No software deals with dual carriageways collapsing In CPT, it causes more problems than in the other systems because CPT is the only one that proposes a tool for road displacement: once again, if no generalisation is performed all the preservation constraints are satisfied, whereas if you take the risk to generalise, you can provoke undesirable sideeffects even if you also solve a part of the problems. On a same software, some testers are more cautious than other ones (again). 158 (a) Initial (b) Clarity, OSGB tester (novice) Not expert of OSGB specifications (c) ArcGIS, Kadaster tester (novice) (d) CPT, Kadaster tester (novice) (e) CPT, ITC tester (novice) (f) CPT, ICC tester (expert) (g) CPT, CPT tester (vendor) Figure 80 Test outputs in focus zone 159 3.6.17 Main findings and conclusions of evaluation by comparing outputs Several conclusions can be drawn from this evaluation. The first conclusion concerns the capacities of the tested systems with regard to the NMAs requirements. Only a few generalisation problems that were raised by the test cases appear to be fully solved by the out-of-the-box systems. A first explanation is that the data are very different from one NMA to another (schema and content), and that the specifications are often complex and sometimes very specific. Thus customisation would be required, which is normal. However even for “classical” problems that appear in most of the test cases, not all needed functionalities are provided by the out-of-the-box systems. We can observe a general lack of contextual algorithms on groups of objects (typification, selection). Displacement is only available in CPT (based on least squares) and axpand (based on snakes). In Radius Clarity, it is present (based on the “beams”) but not usable without customisation. For other classical problems, algorithms are present but either their parameterisation is difficult because it does not match well the way the specifications are expressed (e.g. line or area simplification, buildings aggregation), or there is a lack of controlling tools to detect where to apply and how to parameterise the algorithms and to control their effects (e.g. patterns detection, discrimination between urban and rural areas, etc.). Two remarks can be added to this. First, many of the identified shortcomings have been studied in research and for some of them known solutions exist. Secondly, some of the shortcomings seem to already be under study and/or have already been corrected by the software suppliers, as shown by the results obtained by the vendors in their parallel testing (buildings elimination and displacement algorithms in ArcGIS and Radius Clarity, for instance). The second conclusion stemming from the comparative evaluation is that the outputs obtained by the different testers on a same test case often appear to be noticeably different from each others, and that the differences of capabilities of the tested systems only partly explain it. Indeed, even the outputs obtained with a same system are often quite different. Three main reasons have been identified to explain the noticed differences, on top of the differences of capabilities of the tested systems. First, the specifications provided by the NMA are sometimes fuzzy and do not completely translate their actual requirements. A bit of fuzziness in specifications is normal, and it is well known that there is never one single solution to a generalisation problem. But here we refer to very different outputs of which some do not conform to what the NMAs were expecting, and where the differences can be explained by too fuzzy or incomplete specifications causing the testers difficulties to interpret. The fuzziness or incompleteness encountered in some of the NMAs specifications is surely due to the limitations of the “constraint approach” followed by the project to express them. Indeed, it seems that constraints on the final result are sometimes not sufficient to fully express without ambiguity what is expected: in some cases, specifying the expected transformation too can help if this transformation is always the same and if it is well known. It is surely interesting to notice that within this project fuzzy or incomplete constraints always resulted in very different interpretations and thus solutions among the testers. The second reason that can explain the differences noticed in the outputs is related to the difficulties encountered by the testers to parameterise the systems. This is partly related to the level of expertise of the testers with respect to the system but not only. Quite often, testers of different levels of expertise for a same system report difficulties to parameterise it for a same part of the process. This is all the more the case when the testers that are experts of the systems are not expert of the concerned test case (i.e. do not work in or with NMA that provided the test case). Sometimes, the systems could be better documented. But it also shows that understanding how a given system reacts to a given kind of data and generalisation problems requires quite experienced users. The software providers could help this parameterisation task by providing default or typical parameterisations. Helping the user to express its specifications into a generalisation system also remains an open research question, even if a few research studies have already been done on this topic. 160 The third and last reason that explains the differences noticed in the outputs is related to the “psychology” of the testers. The testers are indeed more or less cautious when they choose a solution among the ones they obtained through different attempts. Some testers choose to keep a solution where generalisation has been performed, but has also generated lots of errors (some constraints are well satisfied and others very degraded, or some parts of the dataset are well generalised and others very badly). Other testers prefer to keep a solution where almost no generalisation has been performed but no error has been generated. Related to this “psychological aspect”, it can be noticed that quite often, in the same situation the testers do not assess in the same way if they have been able to express a given constraint. Some are more optimistic than others. And some assess if they have been able to express the constraint, while others assess if the system has been able to solve it. A last conclusion that can be drawn from the comparative evaluation task is related to the notion of “universal, implicit” constraint. In several cases, testers have tried to satisfy constraints that were not asked for by the specifications, but that translate classical cartographic knowledge. The concerned constraints are preservation constraints, especially constraints related to the preservation of relative positions and spatial distribution of objects. For some of them, the NMA who provided the concerned test case was asked to react and every time, they approved the initiative taken by the testers. 3.7 Expert evaluation: results and conclusions This section presents and interprets the results of the expert evaluation. It firstly presents details on how the evaluation was carried out in Section 3.7.1. Then it presents the results of the four tasks of the survey separately: Section 3.7.2 presents the results of the global evaluation, i.e. how the generalised outputs were globally perceived by the cartographic experts Section 3.7.3 presents the results on how well individual constraints were solved according to the experts Section 3.7.4 presents examples provided by the respondents on well, badly and differently solved constraints Section 3.7.5 presents how experts ranked the generalisation outputs for their test case Main findings and conclusions of the expert evaluation are presented in Section 3.7.6. Although the expert evaluation is, in contrast to automated constraint-based evaluation, a way to study constraints satisfaction in their context, i.e. taking the specialities of situations into account, one should be careful in the interpretation of the results. Firstly because the statements expressed by the experts are only qualitative (at most ordered) statements that are hard to summarise over one test case or one software system (as is required to evaluate over a number of outputs). Secondly, the evaluation is largely influenced by personal characteristics of the respondents. This subjectivity could have been reduced by having a large number of respondents. However only six respondents for four different test cases completed the survey (it took a large amount of work), and none of the test cases had outputs of all software systems produced by both project team testers and vendors (see Table 7 and Table 8). In addition, the outputs were not completely anonymous, i.e. codes on each output, and file names gave information on who generated the output as one of the respondents observed. This might have been a source of bias. 3.7.1 Details on the responses of the expert evaluation The survey and accompanying materials were sent at the end of July, 2008, and by November 1st , results of six respondents were returned (See Table 20). Although the ICC response was the result of two experts completing the survey together, their combined efforts count for one completed survey. 161 NMA Number of completed surveys ICC 1 IGN 2 Kadaster 2 OSGB 1 Table 20 Response to the expert evaluation. Not all respondents performed all the four tasks of the survey, see Table 21. The quality statements for the global indicators (Table 5, Section 2.4.3) and the answers for the detailed constraint-based evaluation (Table 6, Section 2.4.3) were complete, except for some parts in the survey of one of the IGN respondents. However ranking the generalised outputs, a task that could be done quickly based on the experiences gained at the end, was done by all but the ICC respondent (see Section 3.7.5). A remark should here be made about the ICC test case. The expert indicated that Radius Clarity outputs were not evaluated since the two outputs of the Project team testers differed too much: in one output, few objects were generalised and in the other one, the results were bad compared to the examples provided by the vendor. The vendor’s output, on the other hand, consisted only of screen dumps that could not be used for a full comparison. Hence, all Radius Clarity outputs were ignored in the ICC survey. Annotating the maps with at least ten examples of situations in which constraints were well solved, badly solved or very differently solved was only done by half of the respondents for the well and badly solved examples, and relatively few respondents gave examples for differently solved situations. The lower completeness scores of this task are probably often related to time constraints of the respondents. On the other hand the OSGB respondent indicated that in most outputs of OSGB data only building generalisation was attempted, and that ten examples in each category could not be found. The responses of the cartographic experts and the analysis of these responses are added in Appendix XI. Part Type ICC expert IGN experts Kadaster experts OSGB expert Global indicators Quality statements 3 3 71% 3 3 3 Detailed constraint- based evaluation Single objects 3 3 3 3 3 3 Two same objects 3 3 3 3 3 3 Two different objects 3 3 3 3 3 3 Group of objects 3 3 3 3 3 3 Ranking - 3 3 3 3 3 Examples of constraints No. of well solved 7 12 11 3 10 4 No. of badly solved 8 13 - 14 10 6 No. of - 7 - 1 4 - 162 Pa The case 3.7.2 Alth quan glob they anal )UH The opin of s psyc resp psyc To p the outp acco disp also art Table e results prese e, this will be 2 Global e hough the que ntify the resul bal indicator. I y can be comp lyses are adde TXHQF\ RI UHV e respondents nion about sev several pre-de chometric sca pondents spec chologist Ren provide some six experts w puts for ICC ount by ICC ( plays the resu o main respond Type differently solved Focus zones e 21 Complet ented below a explicitly men expert evaluati estions in the lts. For the glo In a next step pared for seve ed in Appendix VSRQGHQWV¶ VF were first ask ven global qua efined quality ale commonly cify their leve nsis Likert. e overview of were calculate and OSGB ( (as explained lts. These res dents’ comme 0 2 4 6 8 10 12 14 16 G Numberofanswers ICC expert ICC teness of answ re in principl ntioned. ion survey result obal evaluatio the quality st eral indicators x XI. FRUHV ked to assess ality indicator y statements. used in quest el of agreeme f the scores on ed for each in (see Table 7 above), the s sults are briefl ents are cited. Good Above a CPT Clarity IGN experts - wers provided e related to p ed in mainly on we first cal tatements wer . The results o the entire ma rs for each so These are b tionnaires. Wh ent to a statem n the quality ndicator and and Table 8) sum of the fre fly described f average Averag 1. Legibility y Axpand Ka Ka d by the 6 car project team te qualitative an lculated frequ re transformed of both metho ap outputs at ftware separa basically Like hen respondin ment. The sca statements, fr each software ), and no Rad equencies for for each indic ge Below avera y ArcGIS adaster expert adaster Kad rtographic ex esters’ output nswers, we ap encies of qual d into harmon ods are presen macro level, ately in the for ert scales. A ng to a Likert q ale is named requencies of e. Since there dius Clarity o each indicato cator below. F age Poor ts OSG exp daster OSG xperts. t. If this is no pplied a metho lity statement ised scores so nted below and and indicate rm of a choice Likert scale questionnaire after its inve answers give e were no axp outputs taken or is 21. Figur For each indic GB ert GB ot the od to s per o that d the their e out is a item, entor, en by pand into re 81 cator 163 0 2 4 6 8 10 12 14 16 Numberofanswers 0 2 4 6 8 10 12 14 16 a Numberofanswers 3. 0 2 4 6 8 10 12 14 16 G Numberofanswers None E 2. Man CPT Clarity Highly cceptable A . Deviation CPT Clarity Good Abo aver 4. Preserv CPT Clarity Exceptional cases ual editing y Axpand Acceptable from map o y Axpand ove rage Averag vation of geo y Axpand Some more cases required ArcGIS Moderately acceptable N of original d ArcGIS ge Below average ographic ch ArcGIS Many Not acceptable data Poor haracteristiccs 164 Fig 1. L Non aver gure 81 Frequ Legibility ne of the outp rage’, and all uencies of an puts was eva software syste 0 2 4 6 8 10 12 14 16 Fe s Numberofanswers 0 2 4 6 8 10 12 14 16 M Numberofanswers 6 C 0 2 4 6 8 10 12 14 16 Numberofanswers swers given b luated as ‘go ems contribute ew non- serious Many seri 5. Ma CPT Clarity Many Fair nu 6. Number CPT Clarity Acceptable 7. Info CPT Cla by 6 cartogra ood’ on legibi e to the high f y non- ious Few ser ain detected y Axpand mber Few of main po Axpand Undergene ormation re arity Axp aphic experts ility. Most ou frequency for ious Many serious d errors ArcGIS None sitive aspec ArcGIS ralized Ove eduction and ArcG for global qu utputs are eva this quality st No response No response cts rgeneralized GIS uality indicat aluated as ‘b tatement. ors. elow 165 Respondents selected relatively low quality statements for the legibility of the outputs generalised with axpand. The legibility of two Radius Clarity outputs are ranked as ‘above average’, but the other outputs are all ranked lower. The legibility of three CPT outputs are ranked as ‘above average’, but there is also some spread over ‘average’ and ‘below average’. The legibility reached with ArcGIS is more consistent, with two outputs ranked as “average” and the four others as “below average”. Many comments are added by the respondents, particularly on how buildings are handled in the outputs, and further on roads; less on other features. The OSGB respondent mentions a problem with the output of CPT (see also Section 3.7.5). The respondents report many differences between outputs, both among Project team testers’ outputs and between Project team testers’ and vendors’ outputs. Sometimes, it is caused by the use of different symbology by vendors. For details, see the comments tabs in the spreadsheet with results in Appendix XI.. 2. Manual editing required Each output needs manual editing, and the frequencies increase towards the end of the Likert scale: the highest frequency is for many manual editing required (15 out of 21, being also the absolute highest frequency over all indicators), and all software systems contribute to this high score. ArcGIS output receives relatively low quality statements. axpand output receives again low statements (only ‘many’). Quality statements for Radius Clarity and CPT output are spread. The ICC respondent commented that although much editing is needed, generalisation using the software is more efficient than full manual generalisation. On the other hand, the OSGB respondent indicated in one case (for Radius Clarity output) that a full manual generalisation would be preferred. Again there are many detailed comments provided on buildings and roads, less on other features (see Appendix XI). Also here, respondents reported many differences among the outputs (both of Project team testers and of vendors). One respondent mentioned that this is because vendors were able to do more with their software, e.g. displacement. 3. Deviation from the map of the original data The highest frequency of answers concerning deviation of the original data is for ‘acceptable’, and all software systems contribute to the high frequency for this quality statement. Three outputs (one of each software except CPT) are considered ‘highly acceptable’ concerning deviation from original data. Together, ‘highly acceptable’ and ‘acceptable’ have a frequency of 12 out of 21, but there is a broad spreadAxpand and Radius Clarity outputs are spread over all four quality statements, ArcGIS and CPT outputs over three. One respondent mentioned that some deviations are caused by the use of different symbology between input and outputs. Also the respondents mentioned situations with too little deviations (undergeneralised situations?), or with too many deviations, or situations were no generalisation is visible. This can be linked to a point that we have made earlier about the cautiousness of the testers (Section 0). Some preferred doing nothing rather than showing very bad result while others found that it was more interesting to show the best possible generalisation even if the results were globally worse than the initial state. Outputs differ again with vendor outputs, which is explained by the respondents by the fact that vendors were able to use more functionality (see also Appendix XI). 4. Preservation of geographic characteristics There is no clear trend in the quality statements on preservation of geographic characteristics; no outputs are evaluated as ‘poor’, but similar frequencies occur for all other categories, with spread in quality statements for outputs produced by each software. 166 In most cases, geographic characteristics are preserved, and there are fewer differences between the outputs than above. However, preservation happens in some cases because no selection (or generalisation) has been done. In some other cases, characteristic features disappeared (rural buildings, fences; see Appendix XI). This could be a result of the lack of contextual awareness of most of the out-of the-box solutions, i.e. they do not adapt well to different contexts, and therefore some contextual situations are well treated while others are not. 5. Number of main detected errors For the number of main detected errors, the highest frequency is calculated for ‘many serious’, and all software systems contribute to this frequency. Frequencies for other quality statements are low and spread for the outputs of all software, except axpand No answers to this question were provided by one respondent, leading to a frequency of four for ‘no response’. Several respondents mentioned that context was not considered in the output. Further many comments deal with the generalisation of buildings and roads, and some with other features (see Appendix XI). Again many differences are mentioned between the various outputs, except for the CPT output in the OSGB test case. 6. Number of main positive aspects The highest frequency of answers identifying the number of main positive aspects is for ‘few’, and all software systems contribute to this. The remaining quality statements show spread for outputs produced by each software, except axpand. Again, no answer by one respondent lead to four times ‘no response’. Positive comments on ArcGIS outputs were mainly related to building generalisation and the creation of built-up areas (Kadaster test case). For axpand outputs, positive comments are on displacement of buildings and roads, for Radius Clarity outputs also on aspects related to buildings and roads, and for CPT outputs on displacement, and again on aspects related to buildings and roads (see Appendix XI). Respondents observed differences in outputs for both the IGN and Kadaster test case. 7. Information reduction The highest frequency of answers related to information reduction is for ‘undergeneralised’, and all software systems contribute to this again. ArcGIS output is only assessed as ‘undergeneralised’. Some outputs of Radius Clarity and axpand are assessed as ‘acceptable’, but other outputs of this software are ‘under-‘ or ‘overgeneralised’. Both ICC and IGN respondents reported that more reduction is needed. For one IGN respondent generalisation in some outputs is hardly visible. The OSGB respondent mentioned that in ArcGIS outputs more reduction is needed; CPT outputs are acceptable. One Kadaster respondent explained that features are over-, under-, or not generalised due to missing constraints (see Appendix XI). Again, differences between outputs are reported, except for CPT output for the OSGB test case. +DUPRQLVHG UHVSRQGHQWV¶ VFRUHV The calculated frequencies of answers for each indicator provide a first indication of possibilities and limitations of the generalisation software according to the respondents. However it is not easy to compare frequencies of answers for different indicators that make use of different Likert scales. This has several limitations. Firstly the bar charts in Figure 81 do not provide information on which indicators the generalised outputs scored well and on which indicators the outputs scored badly. In addition the bar charts do not indicate whether there are noticeable differences between software systems for one indicator (or for all indicators). Finally the bar charts are not able to show noticeable differences between respondents or test cases. 167 To make the results for the different indicators more comparable, the quality statements were transformed into harmonised quality scores. The quality scores were distributed in such a way that the most positive answers on each Likert scale always have a score of +2, and the most negative ones a score of -2 (see Table 22). As you can see from the table some interpretation was required to translate the answers into scores, i.e. how bad is ‘some manual editing required’ compared to ‘non serious and many detected errors’? This interpretation was established through consensus reached in an expert group, i.e. it was discussed and agreed during one of the project team meetings. 1. Legibility; 4. Preservation characteristics 2. Manual editing required Good 2 None 2 > average 1 hardly any 1 Average 0 some more -1 < average -1 Many -2 Poor -2 3. Deviation from original 5. Main detected errors highly acceptable 2 non-serious / few 2 Acceptable 1 non-serious / many 0 moderately acceptable 0 serious / few 0 Unacceptable -2 serious / many -2 6. Number of main positive aspects 7. Information reduction Many 2 Acceptable 2 reasonable number 1 undergeneralised -2 Few 0 overgeneralised -2None -2 Table 22 Standardised scores for quality statements of global indicators In the next step, average scores were computed per indicator, over all software and over all respondents. This gave a rough indication of the relative quality of the different indicators, as perceived by the respondents, i.e. on which indicator the outputs scored high and on which indicator the outputs scored low. The result is displayed in Figure 82. Indicators with negative average scores show on which aspects the generalisation software systems should improve, while positive scores show that outputs are relatively better appreciated by the respondents on these aspects. 168 Figure 82. Average scores for each indicator. To better understand the average scores, we analysed if scores are noticeable different from each other. We identified a noticeable difference, if we calculated a deviation of 0.7 or more: o between a specific score of one system and the average score all systems per global quality indicator; o between the score of one respondent and the average score of all respondents over all indicators and software systems, as well as for one system only Negative deviations point to relatively poor solutions, positive deviations to relatively good ones. Noticeable deviations were only identified in case of evaluations provided by more than one respondent. It is important to understand that relatively means “compared the average score”. So a noticeable high score may still indicate a poor (but higher than average) score and a low score may still indicate a high score (but lower than the average). A threshold of 0.7 (which is a considerable deviation) is chosen because it emphasises substantial deviations. Very precisely comparing the respondents’ answers is inappropriate since only six respondents evaluated the outputs of (only) four different test cases. A first analysis compared the scores of different systems per global quality indicator (over all respondents). This analysis resulted in the following noticeable differences: Legibility: outputs of axpand have a deviation of -0.8 (average score for all systems is -0.5). This is in line with the results of the frequency calculations above. Number of main positive aspects: again, outputs of axpand deviate negatively: -0.7 (the average score for all systems is 0.7). Information reduction: outputs of ArcGIS and axpand have a noticeably lower score (both -1.0), while CPT deviates similarly, but in positive direction (+1.0) (the average score for all systems is -1.0). A second analysis per respondent over all indicators and all software resulted in the following noticeable differences: One of the IGN respondents has a deviation of +0.8 (from an overall average of -0.4). Thus this respondent was relatively positive in his judgements. A further analysis of the above scores per expert for different software system revealed that: Both IGN respondents were relatively positive about Clarity (deviations of +1.5 both), while the expert of the OSGB and one of the experts of Kadaster gave relatively low scores (-1.2 and -0.9); the average score over all respondents for Clarity is -0.1. Both experts of IGN were also relatively positive about CPT (+0.9 and +1.2), while both experts of Kadaster gave relatively low scores (-1.1 and -1.0); the average over all respondents is 0.0. These differences can be due either to an (unintentional) excessive optimism/pessimism of the concerned experts towards the concerned systems, OR to a better/lower capacity of these systems to Global scores for quality indicators, all software -2,0 -1,5 -1,0 -0,5 0,0 0,5 1,0 Manual editing required Main detected errors Information reduction Legibility Deviation from original map Preservation of geogr. characteristics Number of main positive aspects 169 deal with the concerned test case, since each expert only evaluated the results of the test case provided by his/her NMA. A good means to dismiss the first hypothesis would have been to make the output blind for the expert evaluation. Besides this, it is also noticeable that for the cases with two respondents (IGN and Kadaster) the results of both respondents deviate in the same way. Other deviations for individual indicators and for all indicators per software systems are smaller than 0.7. 3.7.3 Detailed constraint-based expert evaluation After the global evaluation, the respondents were asked to evaluate how far specific constraints on single objects or different objects are met in the generalised outputs, taking into account the interaction between constraints at specific locations. For this evaluation the initial defined constraints were simplified into constraints that could be visually evaluated (see Section 2.4.3). The respondents rated the constraint satisfaction in the outputs for each software at a scale from 1 (very bad), 2 (bad), 3 (acceptable), 4 (good) to 5 (very good). A general observation for this part of the survey is that the respondents found it difficult to say for some constraints whether they are relevant for the concerning test case or not. This is noticeable in the comments and in the scores. Sometimes, only one of the two IGN or Kadaster respondents gave scores, while the other one filled in NR (not relevant). Examples for IGN are minimum dimensions of individual contours, and the constraints for islands; for Kadaster, the granularity and shape of individual roads are examples. The respondents evaluated the outputs for four types of constraints separately: constraints on individual objects (discriminating between feature types), constraints on two objects of the same class, constraints on two objects of different classes and constraints on groups of the same objects. In the next four subsections the results of these four evaluations, i.e. as bar charts, respondents’ comments and noticeable differences in scores are presented separately. The analyses of the responses on detailed constraint-based evaluation are added in Appendix XI. (YDOXDWLRQ RI FRQVWUDLQWV IRU LQGLYLGXDO REMHFWV For every feature type, the average scores per constraint (i.e. minimum dimension, granularity and shape preservation) over all software systems was calculated. These scores are represented in Figure 83. 170 1. Individual buildings 0 1 2 3 4 5 Min. dimensions Granularity Shape 2. Individual land use objects 0 1 2 3 4 5 Min. dimensions Granularity Shape 3. Individual roads 0 1 2 3 4 5 Min. dimensions Granularity Shape 4. Individual contour lines 0 1 2 3 4 5 Min. dimensions Granularity Shape 5. Individual rivers 0 1 2 3 4 5 Min. dimensions Granularity Shape 6. Individual coast lines 0 1 2 3 4 5 Min. dimensions Granularity Shape 171 Figure 83. Average scores of individual objects for outputs generated by Project team testers. The meaning of the scores is: 1. very bad; 2. bad; 3. acceptable; 4. good and 5. very good. Table 23 shows the results per constraint. Minimum dimensions Granularity 6KDSH SUHVHU YDWLRQ (with slightly higher scores for most individual objects) Very bad - bad: coast lines land use objects islands Very bad - bad: coast lines land use objects islands Bad: coast lines land use objects Still below acceptable: buildings rivers Still below acceptable: contour lines Acceptable: buildings roads contour lines rivers Acceptable: roads contour lines Acceptable: buildings roads islands Nearly good rivers Table 23 Averaged respondents’ scores of three constraints on individual objects From this summary we can observe that only the three constraints for roads are met in an acceptable way; results for individual buildings, contour lines and rivers are nearly acceptable; constraints for other feature types were not well solved according to the respondents. 7. Individual islands 0 1 2 3 4 5 Min. dimensions Granularity Shape 172 There are not many comments added to this part of the evaluation, and the ones that are given are mostly rather general. For example the ICC respondent indicated that for individual land use objects and contour lines no generalisation is visible. The IGN experts mentioned the same problem for some constraints (mostly shape) related to buildings, land use objects, roads and contour lines. For the OSGB case, the main problem with buildings are the holes. For land use objects, roads, contour lines and rivers, the OSGB respondent mentioned that there were no constraints specified, except for granularity of land use objects, but no generalisation is visible here. For the Kadaster case, more specific shortcomings are mentioned (e.g.: for land use objects/minimum dimensions: that only the whole area was taken into account, not the width). See Appendix XI for details. Next, for each constraint on the different feature types, noticeable differences between software systems were calculated (like previously, as deviations • 0.7 from the average scores over all software systems for the same feature type and constraint). Noticeable differences between software systems Noticeable differences can be observed for the following feature types: Buildings: for minimal dimensions, outputs of Axpand have a deviation of -0.7, hence its average score is lower than the score over all software systems for this constraint. On the other hand, outputs of CPT deviate in a positive way (+0.7). The average score over all systems for this constraint is +3.00. For granularity we can observe a similar trend: deviations are -1.0 (Axpand) and +1.0 (CPT), while the average for all systems is +2.6. Contour lines: Axpand and Clarity have positive deviations for shape (both +0.7), and CPT has negative deviation for shape (-0.8); the average for all systems here is +2.8. Noticeable differences between Project team testers’ and vendors’ output Noticeable differences between Project team testers’ and vendors’ outputs (per constraint: average of scores for Project team testers’ output for each software system minus the same scores for vendor’s output if applied to the same test case) were calculated and identified: Buildings: for granularity, the ArcGIS project team testers’ output (available for the Kadaster case only) has a negative deviation of -1.0, so the experts found that the vendor (with an average score of +3.0) performed better than the Project team testers. Roads: for minimum dimensions, the Radius Clarity Project team testers’ output has a negative deviation (-1.3), and for shape, CPT as well (-0.8). Also here, vendors seem to perform better: their average scores are +3.7 for minimum dimensions, and +4.0 for shape. Noticeable differences between respondents Noticeable differences were also calculated between respondents. All deviations are <0.7. Noticeable differences between test cases Finally, to study if there are noticeable differences for a software system between test cases, deviations between scores for each software system per test case and scores over all test cases for the same software were calculated. We found the following noticeable differences: The ICC test case has a negative deviation for ArcGIS (-0.8; average over all cases is +2.7). The OSGB test case has a positive deviation for CPT (+0.8; average over all cases is +3.0). (YDOXDWLRQ RI FRQVWUDLQWV IRU WZR VLPLODU REMHFWV The analysis of the respondents’ scores was repeated for the category constraints on two objects of the same type. Only a single constraint had to be evaluated by the respondents: minimum distance between the objects. Average scores for each combination of two same objects over all software systems were calculated, see Figure 84. From this Figure we can see that most of the scores are very 173 bad max F The type use repo Not Not Not For softw devi For posi For Not Not For outp For outp 0 1 2 3 4 5 to bad. Nea ximum score g Figure 84. Av testers. Me e most commo es (OSGB, Ka objects and sp orted that for b iceable differe iceable differe iceable differe two building ware (resp. -1 iation (+1.2), two roads: o itive deviation two contour l iceable differe iceable differe two building put (only avail two roads: C put (score of + 0 1 2 3 4 5 Buildings arly acceptabl given by the re verage scores eaning of the on comment m adaster). The pot heights, a buildings, noth ences between ences between ences between s: outputs of 1.0 and -0.7, w so the outputs outputs of Ar n (+1.0). Aver lines CPT outp ences between ences between gs: ArcGIS Pr lable for the K larity Project +3.3) is also b Land use objects e are scores espondents. s of two objec scores: 1. ver made here was ICC respond and in the IGN hing was done n the generalis n software sys n software sys ArcGIS and a where the ave s were assesse rcGIS again c rage scrore for puts show pos n Project team n outputs of P roject team te Kadaster case, team testers’ better evaluate Roads S Const for buildings cts, same type ry bad; 2. bad s that no cons ent mentioned N outputs, spo e (see Append sed outputs w stems stems are iden axpand have erage is +2.5). ed as relatively clearly deviate r all systems i sitive deviatio m testers’ and v roject team te sters’ output but with a sc output has a ed. Ski lifts Co l traint: m s, roads, islan e for outputs d; 3. acceptab traints were s d that no gen ot heights were dix XI). were also calcu ntified as: clearly lower CPT, on the y good. e negatively s here +2.5 on (+1.3; avera vendors’ outp esters and of v has a negativ ore of +3.5) is negative devi ontour ines Sp heig minimum nds and river generated by ble; 4. good 5 specified for th neralisation wa e not visible. ulated. scores than t other hand, h (-1.0), while age for all sys put vendors are ide ve deviation (s clearly consi iation (-1.0), s ot ghts River m distan rs, and this is y Project team 5. very good. he specific fea as visible for The OSGB ex the average fo has a high pos CPT again h stems is +2.0). entified as: -2), so the ve idered better. so here the ve rs Islands nce s the m ature land xpert or all sitive has a . endor endor 174 Not The resp for o diffe Not Not For henc For base here (YD The on t com The In a and a ba From acce iceable differe e analysis als pondent has a other objects ference’. iceable differe iceable differe the ICC test ce scores are r the OSGB te ed on scores f e is +2.0). DOXDWLRQ RI FR e above mentio two different mbination of fe e combination addition two c administrativ ar in Figure 85 m Figure 85 w eptable way. O ences between so showed on deviation of + is marked NR ences between ences between case, ArcGIS relatively low est case, there for two buildin RQVWUDLQWV IRU oned calculati feature types eature types, i railtrack-othe ombinations w ve limit: and 1 5 are also cons we can observ Only for consi n respondents ne noticeable +0.8 but it is b R (not relevan n test cases n scores of dif shows again w. e seems to be ngs only, the WZR GLIIHUHQ ions (i.e. avera (see Figure 8 .e. minimum d er type and fen were not cons 2. river and a sidered not rel ve that for non istency betwee e difference b based on score nt). Therefore fferent test cas a negative de a negative de deviation is n QW REMHFWV age score for e 85). The respo distance, relat nce-building w sidered relevan administrative levant by any ne of the com en river and ro between resp es for two buil , this can hard ses are: evation (-0.8; eviation for C not considered each constrain ondents evalu tive position a were only asse nt in any of th limit. Constra of the respon mbinations, the oads the outpu pondents: one ldings only, w dly be consid average over Clarity (-1 d very relevan nt) were repea ated three con and consistenc essed by the O he test cases. aints that are n ndents. e three constra uts provide ac e of the Kad while the const dered a ‘notice all cases is + .0), but since nt (overall ave ated for constr nstraints for e cy between the OSGB respond These are: 6. not represente aints are met i cceptable solu daster traint eably +1.8), e it is erage raints every emes. dent. road ed by in an tions. 175 Figure 85. Avera Meaning age scores of g of scores is: two different 1. very bad; t objects for o 2. bad; 3. ac outputs gene ceptable; 4. g rated by Proj good and 5. v ject team test very good. ters. 176 Table 24 shows the results per constraint. Minimum distance or dimensions Relative position Consistency between themes Very bad - bad: all combinations, none reaches ‘acceptable’ or higher Very bad - bad: almost all combinations Very bad - bad: almost all combinations Slightly higher, but still below acceptable: building - road contour line - building contour line - river Slightly higher, but still below acceptable: building - road contour line - building Acceptable: river - road Table 24 Averaged respondents’ scores on minimum distance, relative position and consistency between themes The ICC respondent only commented on the combinations contour line - river and coast line - contour line. In both cases, no operator seem to have been applied to meet the minimum distance constraint. The IGN respondents indicated that in some cases there is no generalisation visible (for the combinations building - road, contour line - building); and they identified a consistency problem between roads and rivers. The OSGB respondent reported that in many cases, OSGB had not specified any constraint and therefore they did not evaluate the solutions. Furthermore, nothing was done for (some) constraints in the combinations building - road, land use object - river and road - embankment, and in the two combinations that were only evaluated for the OSGB case: rail track - other object and fence - building. For the Kadaster case, the only comment (expect for ‘no constraints specified’) was for relative position in the building - road combination: in axpand output, building and ditch intersect (see Appendix XI). Again, noticeable differences between software systems were identified. Noticeable differences between software systems Building - Road: for minimum distance, outputs of ArcGIS have a deviation of -0.9, while CPT has a deviation of +1.0, while the average score for all systems is +2.2. Noticeable differences between Project team testers’ and vendors’ output Noticeable differences between Project team testers’ output and vendors’ outputs are: Building - Road: for minimum distance, ArcGIS PT tetsers’ output (for the Kadaster case only) has a negative deviation (-1.0), so the vendor output (average score of +2.5) is again considered better. Clarity Project team testers’ output also has a negative deviation (-1.3; average for the vendor output is +3.3). For relative position, ArcGIS Project team testers’ output (for the Kadaster case) gives the same result as for minimum distance (-1.0, while nthe average score for vendor output is +2.5). Road - Embankment: for minimum distance, CPT Project team testers’ output has negative deviation (-1.0; the vendor’s score is +3.0). Noticeable differences between respondents Scores of individual respondents did not show noticeable differences. Noticeable differences between test cases Finally, there is one noticeable difference between different test cases: 177 For the ICC test case, ArcGIS has, given the deviation, relatively low scores (-0.9), while the overall average is +1.8. (YDOXDWLRQ RI FRQVWUDLQWV IRU JURXSV RI REMHFWV For groups of objects, the respondents were asked to evaluate two constraints: quantity of information and preservation of spatial distribution. Average scores for both constraints are presented in Figure 86 and summarised in Table 25. From the results we can see that none of the outputs contained acceptable solutions for the quantity of information for a group of objects. Also spatial distribution is hardly well solved in any of the outputs, except for group of rivers. Quantity of information Spatial distribution Very bad - bad: all groups Very bad - bad: almost all groups Slightly higher, but still below acceptable: group of contour lines Almost good group of rivers Table 25 Averaged respondents’ scores on quantity and spatial distribution in generalised outputs 0 1 2 3 4 5 Buildings Land use objects Roads Contour lines Spot heights Rivers Islands Constraint: quantity of information 178 Figure 86. Average scores for constraints on groups of objects in outputs generated by project team testers. The meaning of the scores is: 1. very bad; 2. bad; 3. acceptable; 4. good and 5. very good. Comments reveal the following: in ICC outputs, no generalisation if visible for groups of land use objects, spot heights, rivers and islands. No comments were added by IGN respondents. The OSGB respondent indicated (like one of the Kadaster respondents) that for many groups of objects constraints were not specified. Specific comments relate to groups of buildings (rules of amalgamation are not followed) and to groups of roads (the collapse of dual carriageways is not done); see Appendix XI. Noticeable differences between software systems were identified in the following cases. Noticeable differences between software systems Group of buildings: for quantity of information, outputs of ArcGIS (for the Kadaster case) have a deviation of -0.9, while CPT outputs have +0.9 (average for all software systems is +2.4). Group of contour lines: for spatial distribution, outputs of Axpand and Clarity have a positive deviation (both +0.8), while outputs of CPT have a negative one (-1.0). Average for all software systems here is +2.7. It should be noted that unexpectedly good scores may be caused by missing functionality. For example in Radius Clarity no tester did anything on the contour lines because of missing displacement tool while with other software some testers tried something that degraded the distribution. Noticeable differences between Project team testers’ and vendors’ output Noticeable differences between Project team testers’ outputs and vendors’ outputs are: Group of buildings: for quantity of information, ArcGIS Project team testers’ outputs (for the Kadaster case) have high negative deviation (-2.5, while the average vendor’s score is +3.5). Clarity Project team testers’ outputs give a more modest deviation (-0,8; vendors’ score here is also +3.5). For spatial distribution, ArcGIS Project team testers’ outputs (again: only for the Kadaster case) have a deviation of -2.0 (vendor’s score is +3.0). 0 1 2 3 4 5 BuildingsLand use objects Roads Contour lines Spot heights Rivers Islands Constraint: spatial distribution 179 Noticeable differences between respondents Noticeable differences between respondents were not observed. Noticeable differences between test cases For the ICC test case, ArcGIS has again relatively low scores (-0.7), while the overall average over all test cases is +2.1. 3.7.4 Examples of well, badly and differently solved constraints Next the respondents were asked to analyse specific situations in more detail, taking the whole context (i.e. all involved constraints at specific locations) into account, and provide at least ten examples of well solved, badly solved and very differently solved situations. The number of examples provided is indicated already in Table 21. It should be realised that these examples are only selections made by the experts based on what they had seen in earlier parts of the evaluation. Therefore they are not necessarily the best, worst or most differently solved examples that can be found in the outputs for each test case. Results of the provided map fragments and details on the respective tabs can be found in the spreadsheet in Appendix XI. The respondents were also asked to comment on the focus zones identified by the comparison evaluation (see Section 3.6). The respondents’ comments are summarised separately for ICC, IGN, OSGB and Kadaster below. ICC: Problem Comment 1 CPT achieves the best solution for town centre blocks. 2 Coast line generalisation is not visible in almost all the tests. 3 CPT and RADIUS CLARITY have applied some generalisation, including displacements in road interchanges. 4 CPT achieves the best solution in suburban building areas. 5 Parallelism between roads and buildings is not maintained. IGN: The IGN respondents did not comment because they did not understand the task. OSGB: Problem Comment 1 Adjacent buildings representation. Adjacent buildings have often been adequately blocked together. However, in a number of occasions, large holes in the middle have been filled. Also, this blocking process seems to have crashed in a number of occasions (see in bad examples: B6). The outline of the resulting block is never adjusted with the road casing (in any software package), which badly affects the quality of the result. 2 Detached buildings and fences. The character of detached buildings has been maintained, although the rules for aggregating the buildings have not been followed. Fences have been completely removed in all the solutions, which do not meet the specifications. 180 Problem Comment 3 Dual carriageways. None of the dual carriageways have been collapsed. One tester has performed some displacement between both lanes (which does not give a good result), others have done nothing. While doing nothing allows the graphical output to look almost right, it does not allow a symbolisation with a pecked separation line in the middle of the symbol, as required by the specifications. Kadaster: Problem Comment 1 Channels network: no clear selection in constraints. Channels network. A good study area. Only Radius Clarity shows some effort for this item. I wonder if ‘Typify’ from CPT can give some results. There can be 2 (or more) manners to look at this problem: selection of channels, or merging area objects; both typifying the character of the landscape with channels / ditches. 2 Built-up areas: most small buildings are deleted, but re-coding to built-up area is not done (in most outputs). The test examples of CPT and ESRI (the good ones) come close to a good solution. With some more attention to the used parameters it should be acceptable. Maybe some Push is needed as next process. 3 The result of parallel roads and cycle tracks is bad. Parallel roads and cycle tracks (attention, the parallel roads can be also roads with another classification). Cycle tracks are also defined as roads of a certain class. The tests give different results. Displacement is possible but needs a good fine-tuning. Also topology is needed to change the underlying objects. 4 This item needs better solutions than done in the tests. No typification of railways. 5 Another test case should be: area merging and typification of area objects, i.e. the region with forest objects where number 1 is placed, and left of it. 6 The same region as 5. The road selection and simplification need some attention. 3.7.5 Ranking the generalised outputs The next task in the survey was ranking the generalised outputs obtained by different systems based on the earlier global and detailed evaluations.The aim was not to rank software systems on how well they respect NMA requirements, but to see if some systems are better suited to specific test cases. The results of the ranking are presented in Table 26. These should (like most results of the expert evaluation) be interpreted with care, taking into account that they are based on the experiences gained by the respondents during the evaluation. Also, the outputs differed in quality and per test case. In some cases it was obvious that the Project team tester was not very familiar with the software, and in most of the OSGB outputs, only buildings seemed to be generalised. Rank ICC IGN OSGB Kadaster 1 - Radius CPT ArcGIS 2 - CPT ArcGIS Radius Clarity and CPT 3 - axpand Radius 4 - ArcGIS - axpand Table 26. Suitability of software for automated generalisation as perceived by cartographic experts of the respective NMAs for test data of that NMA. 181 The ICC respondent did not rank the systems since (s)he only evaluated Project team testers outputs generated by ArcGIS and CPT, and vendors’ outputs generated by CPT. The expert indicated, however, that both software systems offer generalisation options, that could be useful in combination with manual generalisation. Main comments from respondents of other NMAs are summarised below. IGN: For ArcGIS: difficult to see if generalisation has been applied, e.g. on objects like buildings and roads; actually it is only visible on rivers. For axpand: almost no generalisation on buildings; road generalisation is good, but there is a problems with coalescence of bends. For Radius Clarity: best outputs, although outputs of Project team testers and of vendor differ. Good results for roads and buildings. For CPT: also good results for roads and buildings, but the generalisation of buildings can still be improved. OSGB: For ArcGIS: amalgamation of buildings and management of holes in buildings is adequate and better than with CPT; building simplification, however, is less effective and would require more human intervention than with CPT. No adjustment of the building outlines with the road casings. For Radius Clarity: results for buildings can hardly be evaluated, since there seem to be several versions of the generalised buildings on top of each other. The tester was probably not very familiar with Radius Clarity. For CPT: most convincing results for buildings, although problems remain (stability of the software: algorithm crashed (i.e. it deletes building intersecting roads, see Section 0 and Figure 91), resulting in empty areas that actually have a high density of buildings; holes are not well managed; no adjustment with road casings). Shape simplification of complex buildings is a strong point; often results are acceptable or good. Kadaster: The outputs generated by ArcGIS seem to satisfy more constraints than Radius Clarity, and therefore ends up in the first place. It enables re-coding or deleting of objects based on attribute selection, as well as aggregation of areas and simplification of buildings; it gives some good generalisation results for built-up areas and roads. On the other hand, displacement, enlargement of buildings and areas, and selection based on spatial distribution are missing. axpand looks very similar to CPT, but there was only one axpand output, in which adjacent roads were badly solvedAxpand enables selection, deletion and some aggregation of areas; simplification and displacement of buildings, and displacement of adjacent roads. There is, however, no selection possible on certain attributes, no enlargement of buildings and other areas, and no selection on spatial distribution. Radius Clarity offers selection based on some aspects (like attributes and small areas), but not on others (e.g. adjacency, spatial distribution). It offers aggregation, enlargement and simplification, and roads and built-up areas are in some outputs reasonable. Displacement is missing. CPT also enables selection on some aspects (e.g. areas), but not on others (like attributes, adjacency, spatial distribution); furthermore, simplifying and displacement of buildings and displacement of adjacent roads. Generalisation of built-up areas and displacement are promising. There is, however, no aggregation and enlargement possible. 182 3.7.6 Main findings and conclusions of expert evaluation Automated evaluation of single constraints does not take into account that violation of one constraint might have been necessary to meet another, more important one, as was seen in Section 3.5. The results of the expert evaluation address this problem, since the respondents are able to evaluate generalised outputs on individual constraints taking the specific context into account. From the expert evaluation, several conclusions can be drawn. The expert evaluation of the generalised outputs on global indicators resulted in the following main findings. Firstly, the generalised outputs scored well on the following global indicators: Deviation from the map of the original data: although the majority of the answers is ‘highly acceptable’ or ‘acceptable’, the range of answers for each software system is broad. Preservation of geographic characteristics: ‘poor’ results are not observed by any of the respondents, while a variety in scores exist. Half of the outputs are considered ‘good’ or ‘above average’, but opinions about solutions produced by the same software system vary and there is no clear trend. However, good scores on preservation are biased for situations where no generalisation has been done as shown by the statement below about “Information reduction”, i.e. outputs are globally assessed as undergeneralised. Secondly, the generalised outputs scored less on the following global indicators: Legibility: none of the outputs is considered ‘good’; most outputs are evaluated as ‘below average’ and all software contributes to this. Few outputs are considered ‘above average’, but opinions about the outputs generated by the same software systems differ. Manual editing required: each output requires manual editing, mostly quite a lot and this is true for all software systems. Number of main detected errors: the highest frequency is for ‘many serious’ errors, and all software systems contribute to this. Information reduction: most outputs, independent of the software systems that produced it, are seen as ‘undergeneralised’. Number of main positive aspects: the respondents found only a few main positive aspects in the outputs and this is true for all software systems. Another finding are the noticeable differences of scores on global indicators between software systems: ArcGIS: information reduction (negative) axpand: legibility, main positive aspects and information reduction (negative) Radius Clarity information reduction (positive). Furthermore, for the IGN test case, the respondent is rather positive about Radius Clarity and CPT, while for the OSGB and Kadaster test cases, experts gave relatively low scores on the global indicator for Radius Clarity. Kadaster experts also gave relatively low global indicator scores for CPT. These differences may have different causes, i.e. an (unintentional) excessive optimism/pessimism of the concerned experts towards the concerned systems, better/lower capacity of these systems to deal with the concerned test case, or because each expert only evaluated the results of the test case provided by his/her NMA. To further investigate this, a future test could make the outputs blind for the expert evaluation. Also interesting would be to include interactively generalised maps in this blind expert evaluation. From the expert evaluation of generalised outputs considering individual constraints (in their context) the following observations were made. Firstly we observe that best results are obtained for constraints on individual objects, specifically for roads and buildings. However the average scores for the way in which the three constraints for individual objects (minimum area, granularity and shape) were handled are in most cases relatively low: For minimum dimensions, the highest score is ‘acceptable’ for buildings, roads, contour lines and rivers. This is in line with the results of the automated constraint-based evaluation (Section 3.5.2) 183 For granularity, the highest score is ‘acceptable’ for road and contour lines. For shape, ‘acceptable’ scores are given to buildings, roads, islands and (almost ‘good’) to rivers. Overall, only individual roads have ‘acceptable’ scores for the three constraints. Secondly, for minimum distance constraint between similar objects, the scores are very bad to bad. Nearly acceptable scores were attributed to buildings, roads, islands and rivers, and this is the maximum score given by the respondents for this constraint. For none of the combinations, all three constraints between two different types of objects (minimum distance, relative position, consistency) are met in an acceptable way. Acceptable solutions were only obtained for consistency between river and roads. Finally none of the outputs contained acceptable solutions for the quantity of information and the spatial distribution of a group of objects is not handled well in any of the outputs, except for the spatial distribution of group of rivers (even almost ‘good’), which was however a generalisation problem that hardly occurred in the test cases. Noticeable differences were detected on respondents’ scores between software systems, between project team testers’ and vendors’ outputs and between test cases (compared to the average scores). These differences may show the fitness for one system to handle the specificities of a given test case, examples are relatively high scores of CPT for minimum dimensions, granularity and quantity of information of buildings, as well as for minimum distance constraint. In case of preservation constraints, noticeable differences may also indicate situations that are not touched at all by some systems, where other systems did perform (some) generalisation. Examples are relatively high scores of axpand and Radius Clarity for shape and spatial distribution of contour lines, of which it was known that they had not been generalised. Differences between project team testers’ and vendors’ outputs show also that either mastery of the software is required to obtain the best possible solutions (for example CPT) or that, depending on the cases, the vendors have really made an effort on the additional developments for their parallel testing (for example Radius Clarity). Although in general the findings of the expert evaluation look quite negative, one should be aware that the context of the project is cartographic generalisation for NMAs (i.e. for paper maps), which means that the cartographic experts had high quality output in mind during their evaluation. In addition it would have been nice to have included some manually generalised outputs in the survey (while doing it blind) because the level of exigence of the experts is known for often being "absolute" and not necessarily related to what can be done manually. A final remark on the expert evaluation is important. Although the expert evaluation provides insights into several quality aspects of the generalised outputs, the results should be interpreted with care, as mentioned in the beginning of this section. Generalisation is a subjective process, in which the context plays an important role, and often more than one result is acceptable. Evaluating the outputs with only a limited number of respondents does not sufficiently level this subjective aspect, even though no noticeable differences were found between respondents of the same test case. However the evaluations were carefully performed by cartographic experts of the test cases. Consequently the results do provide important insights into quality aspects of the generalised outputs, which can be confirmed with automated constraint-based evaluation or by a follow up survey in the future. 184  9HQGRUV¶ VROXWLRQV This section reports on the vendors’ tests (Section 4.1). In addition it presents developments of the systems since the tests (June 2007) and references to examples of implementations in practice, both information as provided by the vendors. 4.1 Vendors’ tests The vendors that participated in the project were invited to perform parallel tests in which they could do anything they like as long as they reported on it. As mentioned in Section 3.1, we received outputs from three vendors: ESRI, University of Hannover and 1Spatial. University of Hannover tested all four test cases using the same version of the software as project team testers did; ESRI tested only the Kadaster test case with existing ArcGIS functionality enriched with functionalities implemented in an optimizer prototype and 1Spatial tested both the IGN France test case and the Kadaster test case with a number of extra algorithms that were not available in the commercial version that was tested by the project team. The vendor’s outputs were evaluated in the several evaluation parts of the project (see previous chapter). The vendors provided information on how they carried out (i.e. which step they took and what (extra) functionality they applied) and how they perceived the tests. This information is presented in this section. Because the tests performed by the vendors followed different approaches and because the vendors provided information in a different way with varying level of detail, this chapter describes the tests of the different vendors in a qualitative, heterogeneous manner. The aim is to give insights into what the vendors did; what extra functionality they used and how they perceived the tests. The heterogeneity of the information provided to us by the different vendors in combination with our goal to present as much of the provided information in this chapter prevented us to apply a common approach in presenting the vendors’ tests. The output maps provided by the vendors are added as Appendix VIII. 4.1.1 ArcGIS ESRI delivered a limited number of test outputs to give a flavour of what might be possible in the future. The ESRI team conducted tests with research prototypes implementing optimisation running in the ArcGIS environment. The tests focused on building generalisation in which displacement, exaggeration, and elimination was applied. More details about these prototypes can be found in Monnot et al (2007) and Lee and Hardy (2007). It should be noted that not all capabilities that are developed by ESRI in research environments will meet the demanding technical and commercial criteria to make the transition to product. As such, these tests and their results must not be interpreted as a commitment by ESRI to provide specific capabilities in future software product releases. The tests were conducted on one of the four EuroSDR test cases, i.e. the Kadaster test case. In particular, the ESRI testing concentrated on buildings, in conjunction with roads and land parcels. The methodology that was applied to the buildings contained several steps: o The original 10K buildings were processed in a geoprocessing model using the standard SimplifyBuilding tool, to remove small protuberances which would not be significant at target scale. A minimum area parameter was also applied at this stage 185 to remove very small buildings (sheds, garages etc). This resulted in a set of ‘candidate buildings’ for further optimisation. o A tessellation of partitions was built with standard tools, using road centrelines and some parcel boundaries extended to road centrelines. The extended parcels were the result of a bespoke process using ArcObjects, created by ESRI NL. o Those partitions containing buildings were separated, and fed into a geoprocessing model which invoked the Optimizer (i.e. the research prototype), for each partition, to load buildings and roads into cache and then apply a set of rules made up of constraints and actions. These rules and the constraints were defined in an XML file read by the Optimizer. The constraints applied included: o Building minimum area o Building minimum side length o Building-to-building separation o Building to road separation o Building to road orientation The results of the optimization were a set of optimized buildings, suitable for 50K presentation. In a separate process, partitions were classified according to the density of housing within, and partitions above a threshold were converted to be land cover of type ‘urban area’, suppressing the individual buildings. The result provided by ESRI is shown in Figure 88 (initial state in Figure 87). Figure 87 Initial data of Kadaster test case tested by ESRI 186 Figure 88 Output on Kadaster test case provided by ESRI 4.1.2 Change/Push/Typify University of Hannover (provider of Change/Push/Typify) was the only vendor who provided test outputs for all the four test cases. They performed tests without any customisation of CPT software, although they did some operations outside CPT software (i.e. model generalisation operators in ArcView). The vendor reported its experiences in a processing description per test case, as summarised below. ,*1 WHVW FDVH Extracts of results are shown in Figure 89 and Figure 90. The following steps were performed for the IGN test case: o Selection of contour lines of multiples of 50 m (this was not a constraint but based on own insights!) o Important and industrial buildings are generalised with CHANGE. All buildings are introduced in a next step in PUSH o The distance to ski-lifts is specified o Roads, rivers, contour lines and ski lifts are displaced against each other using the specified distances with PUSH o Buildings are typified with TYPIFY based on meshes created from administrative boundaries and roads o Typified buildings were processed with CHANGE afterwards to simplify them. 187 o No meshes were exceeding 0.9 density of buildings, so no meshes were turned into a block The vendor’s observations for the IGN tests are: o The river did not run in its talweg in the initial dataset. Therefore this could not be assured by PUSH during the process and the wrong situation is preserved o PUSH does not yield appropriate results when objects (e.g. a river and contour line) meet at acute angles. This bug is being repaired. Figure 89: Ski-lifts before (left) and after (right) displacement with PUSH - specially look at the ski-lifts. Figure 90: Displacement of roads – especially look at the opening of the bends. 188 26*% WHVW FDVH The following steps were performed for the OSGB test case: o Buildings were simplified with CHANGE (aggregation of adjacent buildings of same type; minimum facades widths of 10m were eliminated; buildings smaller than 100m2 were eliminated) o Master (thick) contour lines were selected manually o Contour lines were displaced with PUSH; as alternative solution the contours were reduced in the number of points using Douglas Peucker o Dual carriage roads were eliminated with ArcView o Railways, roads and buildings were displayed with PUSH. Parameter value 1m was used instead of 0.33mm as mentioned in the specifications, since this yielded better results (after visual inspection) The vendor’s observations for the OSGB tests are: o Buildings with holes lose their courtyard and buildings intersecting (and touching) roads are eliminated, see Figure 91. Figure 91: Buildings intersecting with the roads are eliminated (shown in pink, left); result after elimination (right). ,&& WHVW FDVH Extracts of the results for ICC test case are shown in Figure 92 and Figure 93. 189 Figure 92: Building typification before (left) and after (right). Figure 93: Two examples, where talweg-points are correctly moved together with river and contour line. The following steps were performed for the ICC test case: o Multiples of contour lines were deleted with ArcView o Roads, rivers, contour lines, buildings and coast lines are displaced against each other with PUSH o Buildings are typified with TYPIFY (roads are used for meshes). Symbols of size 0.5mmx0.5mm were created (instead of 0.4mmx0.4mm as specified in constraints). o Typified buildings are processed with CHANGE afterwards to adapt geometric granularity of large buildings (which are not symbolised) to the given constraints 190 The vendor’s observations for the IGN tests are: o Although the specifications indicate that streets do not have to be displayed in the target map, the vendor missed them in the output map and provided both solutions, see Figure 94. o A bug in PUSH causes that objects that meet at acute angles are not displaced correctly against each other (see IGN test case) Figure 94: without (left) and with (right) displaying the street-objects. .DGDVWHU WHVW FDVH Extracts of the results are shown in Figure 95. The following steps were performed for the Kadaster test case: o The test case requires a considerable amount of model generalisation, i.e. selection of objects based on criteria, specifically road and water. This was done with ArcView o Roads, railways and water were displaced against each other o In contrast to the specifications that prescribe selection of certain buildings, this test applied typification (TYPIFY) first and simplification of typified buildings in a next step (CHANGE). The optimal parameter values were selected by experience, not from the constraints. o Meshes that exceed the 0.1 building density (as indicated in the specifications) were filled as built-up area and buildings were deleted. Note that the calculated density values take into account the symbol widths and therefore the parameter values had to be determined by experience. 191 Figure 95: Buildings after TYPIFICATION and CHANGE (upper image) and after overlay of dense meshes (lower image). The vendor’s observations for the Kadaster tests are: o Since much more roads are kept in output than in the paper map, it is assumed that the constraints are not complete. o The reduction in the residuals during the PUSH process took longer than in a standard case due to large constraints between objects caused by many roads and water elements. 192 4.1.3 Radius Clarity 1Spatial performed tests on two of the four test cases: IGN France and Kadaster with their product Radius Clarity. The generalisation system Radius Clarity addresses the third step of the generalisation workflow implemented by 1Spatial that contains the following five steps: data configuration, model generalisation, cartographic generalisation, text placement and cartographic publishing. Since only the Radius Clarity software was tested, model generalisation constraints could not be expressed in Radius Clarity. As preparation steps to the Radius Clarity generalisation process, the datasets were first validated, and re-engineered if required, to assure that the input data was conform to some geometrical specifications, for example without data spikes, flat polygons, topological errors or kickbacks. In the generalisation process the data was validated against the specifications prior to generalisation and the data was corrected if required. The report, in which 1Spatial report on its tests experiences, lists the additional algorithms that were used in the parallel tests that were not available in the product issued to the testers. These are: Eliminate algorithm that can delete buildings as part of a generalisation process, i.e. if the simplification would result in too small building Visvalingham-Wyatt (1993) algorithm that iteratively removes vertices on a line which, when removed cause the least area displacement. It was used in the vendor tests to simplify areas as forests and lakes. Coast line displacement algorithm calculates the distance between coastline and nearby beach and rocky areas and then displaces the coastline to a minimum distance away from those areas. The Kadaster and IGN France test cases do not contain coastline and therefore this algorithm was not applied for the EuroSDR tests. The ‘align to roads’ algorithm was used to align selected buildings to a road. The ‘ruas building displacement’ algorithm was used to move buildings within a selected partition away from roads and rivers. It firstly moves buildings away from symbolised, fixed features and then displaces from other buildings. Beams was used to displace overlapping roads apart, while retaining the characteristics of the network and runs on manually selected road features. This algorithm simulates the line network as a structure of elastic beams. The network is anchored at fixed points and forces are applied where road symbols overlap to push them apart. The parameters for the force calculation (reaction force factor) and the parameters for the iterative equation (e.g. step size) are configurable. Contextual building eliminate to delete a selection of buildings within a partition. Buildings are eliminated if the density ratio building area to area of the partition is too high. This ratio is a constant parameter set in the map specifications. Results for Kadaster test case provided by 1Spatial are presented in Figure , together with the initial data. 193 a 194 b Figure 96 Results produced by 1Spatial for Kadaster test case (b) and initial data (a) 4.1.4 Axpand Although Axes Systems provided us with a completed system template and background information on the system that we tested (available in June 2007), they made a decision not to test this software on the project test cases since, at the time of the tests, they knew that it would not be enhanced or supported in the future. Instead of the parallel tests Axes Systems concentrated on setting up the next 195 version of the axpand generalisation system - axpand ng - which became commercially available shortly after the start of the tests. 4.1.5 Conclusions on the vendors’ tests From the vendors’ tests we can draw the following conclusions. University of Hannover performed tests on all four test cases with the same version of the software as was tested by the project team. From these tests we can see that mastery of the system considerably reduces the amount of time spent on the tests. In addition, the mastery of the software also resulted in the best results for this software. This can be explained because parameterisation is not straightforward and it does not match very well with the definition of the constraints, i.e. human interpretation is required to obtain the best implementation. ESRI performed tests on one test case using a research prototype, i.e. optimisation engine, which shows promising techniques for displacement (not available for project team testers) and building generalisation. 1Spatial extended their tests on two test cases with a few additional algorithms that were not available to the project team. Therefore also for this software the displacements algorithms, that are fundamental for generalisation, were only used in the vendor tests. Axes Systems did not perform tests themselves. 4.2 Developments since 2007 and references to examples from practice The tests in the project were carried out with commercial versions of the software available in June 2007. To show the developments of the systems since 2007, the vendors were asked to provide us with this information, which is presented in this section per software. It should be noted that these improvements are only based on vendors’ information, which are not evaluated through tests. We also asked the vendors to provide us with references of examples where the reader can find more information on implementations in practice. 4.2.1 ArcGIS The ArcGIS 9.4 software beta release was recently made available to beta users. This version introduces a new set of generalisation tools. Five new tools are introduced at 9.4 to simplify the display of roads and buildings. Two tools simplify the display of complex road networks while retaining general character and connectivity: The Thin Road Network tool lowers the density of a displayed street network by eliminating smaller, less significant roads while maintaining the overall connectivity and character of the street network. The Merge Divided Roads tool generates a line feature class whose features follow the course of divided road features to display a simpler network of single lines. Two tools resolve conflicts among symbolised features at output scale: The Resolve Road Network tool resolves symbol conflicts among roads by slightly displacing features while retaining connectivity and character of the road network. Input features are hierarchically categorized to ensure that less significant parts of the network are moved to accommodate more important features. The Resolve Building Conflicts tool resolves symbol conflicts among a collection of buildings with relation to one or more linear barriers. Minimum building size is enforced and placement in relation to barriers can be controlled. 196 One tool retains spatial relationships among features following displacement The Propagate Displacement tool evaluates the displacement that was made to a road network and propagates the shift to nearby features to ensure that their original spatial relationships are retained. The user documentation includes guidelines for setting parameters as well as recommended default and starting values. The vendor did not provide information of examples from practice. 4.2.2 Change/Push/Typify Since the tests in 2007 the modules the parameterisation of PUSH has been modified so that less conversions are necessary. The usual sequence was: 1) Setting the PUSH-parameters using PUSHJOIN, 2) Processing data with PUSH, 3) Rejoining pushed objects with original attributes. This last step has been included in the whole process and therefore the displaced objects keep the original attributes. Recently, the three modules have been linked to the AED-SICAD software (ArcGIS), by which the parameterisation can be done through ArcGIS interfaces. A last development of the software are the links via interfaces, which makes the software open to other GIS-products. The modules Change, Push and Typify are being used by different NMAs and companies, in Germany and abroad. NMAs abroad are: - ICC, Barcelona - Bosch, Hildesheim - Ordnance Survey (tests) - Momra, Saudi Arabia (tests) These institutes use the software CHANGE for building generalisation 1:5K to 1:25k, the software Typify for building generalisation 1:25k to 1:50k and the PUSH software for the displacement of arbitrary objects against each other for different scales. The modules embedded in AED-SICAD software (ArcGIS), are currently being used by seven NMAs in Germany: - Hamburg - Niedersachsen - Mecklenburg-Vorpommern - Brandenburg - Thüringen - Hessen - Sachsen 4.2.3 Radius Clarity 1Spatials’ solutions for building displacement and typification were not complete in Radius Clarity v2.6 (the version tested). Since then, Radius Clarity v2.7 has been released with an improved displacement algorithm and a new building elimination algorithm, and further internal developments have been made on both typification and displacement for a future release. In addition both parentage tracking and incremental generalisation have been developed and implemented in a bespoke customer generalisation work flow. Also, since the product version on test, progress has been made in preserving spatial relationships such as relative positions where topological connections do not exist in the source data, for example these relationships are now maintained during diffusion. Radius Clarity software is used in a European project to improve business efficiencies in Germany's state mapping agencies in partnership with AdV (a Working Committee that coordinates the official 197 surveying in Germany). Taking 1:10000 base data 1Spatial has delivered an automatic generalisation flowline to generate digital landscape models at 1:50,000 scale. This enables AdV to derive more up to date digital topographic products at their target scale and additionally make time and cost savings. Further details: http://www.1spatial.com/pdf/adv1.pdf 4.2.4 Axpand The next version of the axpand generalisation system - axpand ng - includes broader and more comprehensive generalisation capability based on workflow technology and a true MRDB which stores the history of the generalised objects over time. This version of the software is being used to completely generalise 1:10k, 1:25k, 1:50k and 1:300k maps. Examples of generalisation using this version can be seen at www.axes-systems.com/content/axpand-ng-new-generation-generalisation The version of axpand that was tested in the project is being used by the Landesversmessungsamt Thürigen (Germany) for displacement, smoothing, selection and aggregation for the production of their 1:25K and 1:100K maps. See for more information: http://www.thueringen.de/de/tlvermgeo/ 198  &RQFOXVLRQV DQG GLVFXVVLRQ This report presents the EuroSDR project that studied the state-of-the-art of automated generalisation of commercial systems that were available in June 2007. The project evaluated generalisation outputs of test cases provided by four NMAs, applying different software systems and produced by different testers, taking into account the NMA requirements. The two main questions of the research were formulated in Chapter 1: What are the possibilities and limitations of commercial software systems for automated generalisation with respect to NMA requirements? What different generalisation solutions can be generated for one test case and what are the reasons for these differences? The answers to these two questions will be summarised in Section 5.1. Section 0 presents the main conclusions of this research and identifies issues for further research. 5.1 Answers to research questions 5.1.1 What are the possibilities and limitations of commercial software systems for automated generalisation with respect to NMA requirements? Several findings answer this question: o All the tested systems offer potentials for automated generalisation, especially for handling constraints on single objects (in particular roads and buildings). However only a few generalisation problems that were raised by the test cases appear to be fully solved by the out-of-the-box systems. This may be a result of the complex and very specific constraints that require customisation of the out-of-the-box versions. Apparently the tested systems provide generic solutions which are not directly applicable to the specific cases. o In line with the first finding, generally the cartographic experts in the expert evaluation did not score the generalised outputs very high, with some exceptions (i.e. generalisation of individual objects and consistency between river and roads and spatial distribution for group of rivers). According to experts the generalised outputs scored well on Deviation from the map of the original data and Preservation of geographic characteristics, which is biased for situations where no generalisation has been done, i.e. the outputs scored bad on Information reduction. In addition the outputs scored not very well on Legibility, Manual editing required, Number of main detected errors, and Number of main positive aspects. o The expert evaluation also identified noticeable differences between software systems and test cases, which may show the fitness for one system to handle the specificities of a given test case, examples are relatively high scores of CPT for minimum dimensions, granularity and quantity of information of buildings, as well as for minimum distance constraint. In case of preservation constraints, noticeable differences may also indicate situations that are not touched at all by some systems, where other systems did perform (some) generalisation. Examples are relatively high scores of axpand and Radius Clarity for shape and spatial distribution of contour lines, of which it was known that they had not been generalised. o Also for “classical” problems, not all needed functionalities are provided by the outof-the-box systems. We can observe a general lack of contextual algorithms on groups of objects (typification, selection), which could be a result of the lack of contextual awareness of most of the out-of the-box solution. i.e. they do not adapt 199 well to different contexts, and therefore some are well treated while others are not. Also functionalities for defining sensible groups for generalisation (closely related to contextual algorithms) are missing. Displacement is only available in CPT (based on least squares) and axpand (based on snakes). In Radius Clarity, it is present (based on the “beams”) but not usable without customisation. o For other classical problems, algorithms are present but either their parameterisation is difficult because it does not match well the way the specifications are expressed (e.g. line or area simplification, buildings aggregation), or there is a lack of controlling tools to detect where to apply (e.g. detecting conflicts and defining sensible groups through partitioning) and how to parameterise the algorithms and to control their effects (e.g. patterns detection, discrimination between urban and rural areas, etc.). It should be noted that once the parameters have been set, in practice they are used for a given product - meaning it may not be a major problem if it takes long to set them. Also for parameterisation it is true that customisation is required and that testing the out-of-the-box versions is not the "regular" way of NMAs setting their production lines. o Many of the identified shortcomings have been studied in research and for some of them, solutions exist at NMAs. The lack of these solutions in commercial software may be due to differing needs among NMAs (due to differences in data models and specifications). This implies on the one hand huge investments from the commercial vendors for a small numbers of potential customers, and on the other hand huge investments of NMAs to invest in partial solutions which still require considerable customisation effort. The results may look disappointing. However they should be interpreted with care for several reasons. First, the ambitions of the project were high: the generalisation requirements were defined through detailed and concise constraints, the test cases contained a selection of complex/known problems and the focus was on the production of high quality paper maps, and the cartographic experts that evaluated the outputs considered a top level quality requirement in their evaluation originating from the topographic paper map context of the project. One should be aware that the functionality available in the four systems does enable to automate part of the generalisation processes and to optimise the production workflows. Another remarks that is relevant here is that some of the shortcomings, that have been solved at NMAs or research institutes, were tackled by the vendors in their parallel testing (buildings elimination and displacement algorithms in ArcGIS and Radius Clarity, for instance). The vendors have indicated that this project has resulted in internal developments on automated generalisation within their systems such as described in Section 4..2. Also it is not a surprise that outof-the box versions are not capable of fulfilling NMAs requirements. In fact the results confirm that customisation is definitely required to tune the capabilities of the systems to the requirements of specific test cases. o A last remark that puts the conclusions of the project into perspective is that systems are used more satisfactory in practice, as shown by the references to examples from practice in Chapter 4, provided by the vendors. Also the NMAs in the project team have achieved some successful implementation with customised versions of the tested software. 5.1.2 What different generalisation solutions can be obtained for one test case and why do they differ? Outputs for one test case can be very different, which was identified and illustrated by the evaluation of generalised outputs. Besides differences in capabilities of systems, these differences can be explained by several reasons. 200 Specifications provided by NMAs are sometimes fuzzy and do not express fully their actual requirements. This may be caused by incompleteness but also because constraints are not always capable of defining without ambiguity what is expected, see for example Figure 6. Differences in outputs can also be explained because of difficulties of parameterisation, specifically because a direct match between how constraints are specified and the concerning parameterisation in specific software is missing. Therefore testers that were familiar with the test data (and knew what would be expected) obtained other results than testers that were new to the data. In addition the differences in outputs caused by differences in parameterisation showed that understanding how a given system reacts to a specific situation requires quite experienced users. Differences can also be explained because of differences between testers’ approaches. Some testers prefer outputs in which some constraints are very well satisfied and others very badly or in which some parts are very well generalised and some parts are very badly generalised. While other testers prefer outputs in which almost no generalisation was performed, but also no errors were generated. 5.2 Conclusions and further research This project confirmed that the result of a generalisation process is not a linear process where the result can be predicted starting from a specific source data set and specifications formalised in constraints. Instead the final result is a consequence of many interchanging variables such as richness of the data, formalisation level and fuzziness of the map specifications, the way the tester interpreted the constraints, functionality selected by the tester, the parameterisation applied, etc. The methodology applied in our research had to consider all these kinds of heterogeneity to guarantee independent testing and evaluation of available generalisation solutions. In addition this was the first research that studies the combinations of different aspects of output maps, generalised by different systems, different testers taking into account the map specifications of several NMAs. Consequently an important research aspect was the applied methodology itself, addressing the following questions: how to set up a case study for evaluating automated map generalisation in commercial generalisation systems; how to specify both generic and NMA specific requirements for automated generalisation; how do automated generalisation processes work; how to perform evaluation of generalisation output; how does the constraint approach, as adopted in this research, work in practice and what further research is needed in this area? To improve and reuse the project methodology, a future research should consider the issues raised in the remainder of this section. 5.2.1 Defining map specifications as constraints Formalising and harmonising NMA map specifications provided a common view on requirements for automated map generalisation. Although very time consuming, defining map specifications as a set of constraints, has allowed formalising NMA requirements in a common template and using them in the automatic generalisation processes. The harmonised list of constraints elaborated for the project is, however, not complete. The NMAs had to limit their constraints to those describing the main problems within the selected test areas and to constraints that were more or less straightforward to formalise. In addition, the constraints were defined without running any automated generalisation process, which would have shown both missing and unclear constraints as well as how specific constraints work in practice. Nonetheless, the resulting set of constraints is a first attempt to define a “full” set of constraints as implementation of research theories. The set could be completed based on the experiences in our project, for example by adding constraints that were missing as observed from the outputs. In addition, the constraints should be further 201 formalised to better support the generalisation process as well as the automated constraint–based evaluation. This implies that general concepts, such as shape, pattern, and urban and settlement structures, should be described formally. Finally, constraints that appeared to be unclear need refinement to distinguish, e.g., cartographic conflicts from acceptable solutions (compare Figure 6a against Figure 6b and Figure 6c). This requires adding more semantics to the constraints and looking beyond geometric and thematic properties. Helping the user to express its specifications into a format understandable by a generalisation system (which may require other means than constraints) also remains an open research question, even if a few research studies have already been done on this topic (for example Hubert and Ruas, 2003). 5.2.2 Formalising and evaluating preservation specifications Usually, the preservation specifications were more difficult to formalise and to evaluate than the legibility specifications. Therefore, better understanding of preservation specifications is required to improve their formalisation in constraints as well as the measurement of constraint violation. This includes a better understanding of the concepts involved (i.e., how to mathematically describe “shape” on the basis of existing measures such as length-width-ratio, shape index, fractal dimension, etc.) and of the changes allowed (how to mathematically describe accepted modifications). Harrie (2001) obtained such information by studying existing maps at different scales. Another problem in evaluating preservation constraints is that a correspondence is required with the initial data. This is not always an issue in 1:1 relationships; however, because of operators as selection, typification, amalgamation, and aggregation relationships may become complex, which makes it difficult to compare output data with the initial data. The difficulty of evaluating preservation specifications was also encountered in the expert survey: it was often unclear whether a preservation constraint was assessed as “good” because the system had carefully accounted for it, or because the system had simply ignored it and at the same time had not much altered the data during the process. This aspect was also encountered for legibility constraints. For example if the system removes all the elements under minimum size, instead of exaggerate or aggregate them, the automatic evaluation of the minimum size constraint will give a “good” result, because the constraint is not violated, but the resulting map does not represent the situation very well. Further research is needed on how far a violation of preservation constraints is tolerable. First investigations on interactively generalised data showed that cartographers also tolerate violations of legibility constraints. For both, legibility and preservation constraints, a formal description of tolerated violations is required. Furthermore research on the weighting between constraints and constraint violations has to be carried out to guide the generalisation process and to get an overall evaluation result for the generalised map. 5.2.3 Constraint-based generalisation The project methodology used constraints both to direct the generalisation process as well as to determine to what extent the output maps meet the specifications. Our evaluation, which integrates three methods, has shown that this approach has an important limitation: the results for individual constraints are not always a good indicator for the quality of the overall solution. This has various explanations. First, some constraints may have been violated deliberately to enable good results for other constraints, e.g., by allowing (slightly) more displacement to avoid overlap. Secondly, as was observed in the automated constraint-based evaluation of interactively generalised data, one should assess not only if a constraint was violated but also if the violation yields an unacceptable cartographic conflict. Third, very good results for one specific constraint (e.g., minimal distance between buildings) may coincide with bad results for another constraint (e.g., building density should be kept). Fourth, a 202 non-satisfied constraint can be due to missing functionality in a system, but can just as well be due to imprecise constraint definition. And finally, as Harrie and Weibel (2007) observed, results of constraint-based evaluation heavily depend on the defined test cases: is the constraint set complete and evenly balanced, or does it contain many constraints for very specific situations (as in the OSGB case)? Although the expert evaluation did evaluate generalised outputs on individual constraints taking the specific context into account, future research should aim to: improve generalisation models and constraints to enable taking the notion of flexibility of threshold values into account express constraint satisfaction in values ranging from 0 to 1, instead of in Boolean values. Boolean values may more appropriate to identify cartographic errors. They may, however, be less appropriate for assessing the evaluation output, because they do not provide information on the degree to which the threshold is ignored. validate the constraint approach by considering how to aggregate “constraint-by-constraint” assessments for global indicators of map quality, specifically by better understanding their interdependencies and impact. This also raises questions on the domain of constraint satisfaction and violation values and on their weighting and prioritizing to make different constraints comparable and to enable aggregating them to global indicators. These issues have previously been addressed in the domain of constraint-based optimization (see Ruas,1998; Bard, 2004, and Mackaness and Ruas, 2007). A future test could also consider selecting a representative set of constraints to better evaluate generalisation functionalities in commercial systems. 5.2.4 Evaluating generalisation software beyond constraints Our study concentrated on the question of whether commercially available solutions could meet the map specifications of NMAs defined as constraints. However, during our tests several other aspects were encountered that are also relevant for assessing commercial generalisation systems. For example, our testers found that in some cases topological errors were introduced during the generalisation process, and that links between generalised and ungeneralised objects, required for automated evaluation, were not created in most of the outputs. These aspects should be addressed in future tests. Furthermore the tests highlighted difficulty to parameterise the complex algorithms. In fact several of our evaluations showed that some vendors’ solutions are better than solutions generated by the project team which shows that mastery of the software is required to obtain the best possible solutions. Software systems could help the user in finding the best parameterisation, for instance by providing tools to support interactive parameterisation (e.g. providing default parameters), or by providing tools to select similar situations, which could be generalised with the same parameterisation or tools for situation dependent, automated parameterisation. A next research that evaluates generalisation in commercial software should highlight parameterisation possibilities as well as user friendliness. In addition, a future test should address aspects not amenable to constraints. The constraint approach is based on the consequences of scale changes. According to Mackaness and Ruas (2007), this bottom-up approach might work better for small-scale changes. In contrast, a top-down approach that meets the consequences of (large-) scale reduction by choosing appropriate representations for phenomena might work better over larger scale changes where changes are much more fundamental. A future test can provide more insights into the appropriateness of both approaches for automated map generalisation. Indeed, it appeared that constraints on the final result are sometimes not sufficient to fully express without ambiguity what is expected. In some cases, specifying the expected transformation can help if this transformation is always the same and if it is well known. However fuzzy and incomplete constraints resulted in very different interpretations and solutions among the testers, which may ask for a different approach in defining the requirements for automated generalisation. Furthermore, because the limited sizes of the four test cases precluded addressing the problems of dealing with large amounts of data (computational complexity, potential memory overflows that 203 necessitate data partitioning, presence of numerous and various particular cases that make some algorithms fail, etc.), future tests should define criteria as well as measuring tools to assess scalability of systems. And finally, future tests should quantify customisation possibilities. The most realistic way to address NMA-specific requirements may be to customise existing software. This requires facilities for writing extensions or for allowing integration with other systems. 5.2.5 Concluding remarks In conclusion, all the tested software systems provide tools for automated generalisation, but none of them achieves globally good results. Despite the current limitations, they can be implemented in a production workflow to automate considerable part of the generalisation process. Solving the lack of complete solutions in commercial software requires a huge investment from the commercial vendors, considering the small number of potential customers, and a huge effort of NMAs in the customisation of partial commercial solutions to fulfil their specific requirements. Therefore stronger and deeper knowledge flow between researchers, vendors and NMAs, as operated in this project, is essential to progress in the automation of generalisation. A significant contribution of this project to generalisation research is the methodology to define map specifications for automated generalisation and to evaluate generalised data. Consequently, future generalisation research can extend our methodology and make use of our findings, applying improved versions of the constraints sets and re-using our carefully sourced generalisation test cases. 204  5HIHUHQFHV AGENT (1997). Map generalisation by multi-agent technology, ESPRIT research 1997-2000, http://agent.ign.fr/ Barrault, M., N. Regnauld, C. Duchêne, K. Haire, C. Baeijs, Y. Demazeau, P. Hardy, W. Mackaness, A. Ruas & R. Weibel (2001). Integrating multi-agent, object-oriented, and algorithmic techniques for improved automated map generalisation. In: Proceedings of the 20th International Cartographic Conference (ICC 2001), 6-10 August 2001, Beijing, China, pp. 2110-2116. CD-ROM. Bard, S. (2004). Quality Assessment of Cartographic Generalisation. Transaction in GIS, Vol. 8, No. 1, pp. 63-81. Beard, M. K. (1991). Constraints on Rule Formation. In Buttenfield, B. P., and R. B. McMaster (eds), Map Generalisation: Making Rules for Knowledge Representation, Longman Group, pp. 121-135. Longman, London. ISBN: 0-582-08062-2. Boffet A. (2001). Méthode de création d'informations multi-niveaux pour la généralisation de l'urbain. Thèse de doctorat, Université de Marne-la-Vallée, 2001 Brewer, C.A. & Buttenfield, B.P. (2007) Framing guidelines for Multi-Scale Map Design Using Databases at Multiple Resolutions. Cartography and Geographic Information Science, 34 (1), pp.3-15. Burghardt, D. and M. Neun (2006). Automated sequencing of generalisation services based on collaborative filtering. In: M. Raubal, H. J. Miller, A. U. Frank and M. Goodchild (eds): Geographic Information Science. 4th Int. Conf., GIScience 2006, IfGIprints 28, pp. 41-46. ISBN 9-783936- 616255. Burghardt, D., S. Schmidt and J.E. Stoter (2007). Investigations on cartographic constraint formalisation, 10th ICA Workshop of ICA commission on Generalisation and Multiple Representation, August 2-3, Moscow, Russia. Burghardt, D., S. Schmid, C. Duchêne, J. Stoter, B. Baella, N. Regnauld, G. Touya, (2008). Methodologies for the evaluation of generalised data derived with commercial available generalisation systems, 11th ICA Workshop of ICA commission on Generalisation and Multiple Representation, 20- 21 June 2008, Montpellier. http://aci.ign.fr/BDpubli/moscow2007/Burghardt-ICAWorkshop.pdf (accessed 3 November 2008). Buttenfield, B.P. (1991) A rule for describing line feature geometry. In B. P. Buttenfield & R. B. McMaster, eds. Map generalisation: Making rules for knowledge representation. Longman, pp.150- 171. Chaudhry, O. Z. & Mackaness, W.A. (2008). Automatic Identification of Urban Settlement Boundaries for Multiple Representation Databases. Computer Environment and Urban Systems. 32(2), pp. 95-109 Foerster, T., J.E. Stoter and M-J Kraak (2009) Challenges for Automated Generalisation at European Mapping Agencies, submitted for peer review to The Cartographic Journal. Gaffuri J. & Trévisan J. (2004). Role of urban patterns for building generalisation: An application of AGENT. 6th ICA Workshop on progress in automated map generalisation, Leicester, 2004, disponible sur le site web de la commission en généralisation de l'ACI : http://aci.ign.fr/Leicester/paper/Gaffuri- v2-ICAWorkshop.pdf 205 Harrie, L. (2001). An Optimisation Approach to Cartographic Generalisation. Ph.D. thesis, Department of Technology and Society, Lund University. Harrie, L. and R. Weibel (2007). Modelling the Overall Process of Generalisation. Chapter 4 Generalisation of Geographic information: cartographic modelling and applications, edited by W.A. Mackaness, A. Ruas and L.T. Sarjakoski, pp. 67-88. Elsevier. ISBN 978-0-08-045374-3 Hubert, F. and A. Ruas (2003). A method based on samples to capture user needs for generalisation. 5th ICA workshop on progress in automated map generalisation, Paris, 2003. http://aci.ign.fr/BDpubli/paris2003/papers/hubert_et_al_v0.pdf (accessed 3 November 2008) Kilpelaïnen, T. (2000). Knowledge Acquisition for Generalisation Rules. Cartography and Geographic Information Science, vol.27, No 1, 2000, pp 41-50 Lee, D and Hardy P “Analyzing and Deriving Geographic Contexts for Generalisation”, International Cartographic Conference, July 2007, Moscow, Russia. Leitner, M. and B. Buttenfield (1995). Acquisition of procedural cartographic knowledge by reverse engineering. Cartography and Geographic Information Systems, 22 (3), pp.232-241. McMaster, R.B. and K. S. Shea (1988). Cartographic Generalisation in a Digital Environment: A Framework for Implementation in a Geographic Information System. in: GIS/LIS proceedings, San Antonio, TX, pp. 240-249. McMaster, R.B. (1995). Knowledge acquisition for cartographic generalisation. In J. C. Mueller, J. P. Lagrange, & R. Weibel, eds. GIS and Generalisation: Methodology and Practice. London, UK: Taylor & Francis, pp.161-179. Mackaness, W. A., A. Ruas and L. T. Sarjakoski (2007). Generalisation of Geographic Information: Cartographic Modelling and Applications. Series of International Cartographic Association, Elsevier. ISBN 978-0-08-045374-3 Mackaness, W.A. and A. Ruas (2007). Evaluation in Map Generalisation Process, Chapter 5 in Generalisation of Geographic information: cartographic modelling and applications, edited by W.A. Mackaness, A. Ruas and L.T. Sarjakoski, pp 89-112. Elsevier. ISBN 978-0-08-045374-3 Monnot, J; Hardy, P; and Lee, D “An Optimization Approach to Constraint-Based Generalisation in a Commodity GIS Framework”, International Cartographic Conference, July 2007, Moscow, Russia. Müller, J.C. and P.J. Mouwes (1990). Knowledge acquisition and representation for rule based map generalisation: An example from the Netherlands, GIS/LIS proceedings 90, Anaheim, California, Vol1, pp. 58-67 Mustière, S. (2005) Cartographic generalisation of roads in a local and adaptive approach: A knowledge acquisition problem. International Journal Of Geographical Information Science, 19 (8-9), pp.937-955. Mustière, S. (2001). Apprentissage supervisé pour la généralisation cartographique. Thèse de doctorat, Université Paris VI, France 2001. 206 Nickerson, B.G. (1991) Knowledge engineering for generalisation. In B. Buttenfield & R. B. McMaster, eds. Map Generalisation: Making Rules for Knowledge Representation. London: Longman, pp.40-55. OpenJump (2008). http://openjump.org/wiki/show/HomePage. OpenJUMP – The free, Java based and open source Geographic Information System for the World. (accessed 28th of October 2008). Plazanet, C., N. Bigolin and A. Ruas (1998). Experiments with Learning Techniques for Spatial Model Enrichment and Line Generalisation. Geoinformatica, 2 (4), pp.315-333. Reichenbacher, T. (1995) Knowledge acquisition in map generalisation using interactive systems and machine learning. In Proceedings of the 17th International Cartographic Conference. Barcelona, Spain, pp.2221-2230. Rieger, M.K. and Coulson, M.R.C. (1993) Consensus or confusion: cartographers' Knowledge of Generalisation. Cartographica, 30 (2 & 3), pp.69-80. Ruas, A. (1998), OO-Constraint Modelling to Automate Urban Generalisation Process, Proceedings of the Eight International Symposium on Spatial Data Handling, Vancouver, Canada, July 12-15, pp. 225-235 Ruas, A. (1999). Modèle de généralisation de données géographiques à base de contraintes et d'autonomie. Doctoral Thesis, Université de Marne-la-Vallée. Ruas, A. (2000). The Roles Of Meso Objects for Generalisation. Proceedings of the 9th International Symposium on Spatial Data Handling, Beijing, 2000, pp.3b50-3b63. Ruas, A. (2001), Automatic Generalisation research: Learning process from interactive generalisation, OEEPE, Report nr 39. Sarjakoski, L. T., 2007. Conceptual models of generalisation and multiple representation. Chapter 2 in: Mackaness, W. A., Ruas, A. and L. T. Sarjakoski, (eds.), Generalisation of Geographic Information: Cartographic Modelling and Applications, Series of International Cartographic Association, Elsevier, pp. 11-35. Sester, M. (2000). Generalisation based on least-squares adjustment, the XIXth International Congress, Commission IV, International Archives of Photogrammetry and Remote Sensing, Amsterdam, The Netherlands, pp. 931-938. Steiniger, S., P. Taillandier and R. Weibel (2009). Utilising urban context recognition and machine learning to improve the generalisation of buildings. Int. J. of Geographical Information Science, in press. Stoter, J.E (2005). Generalisation: the gap between research and practice, Proceedings of the 8th ICA workshop on generalisation and multiple representation, 7-8 July, 2005, A Coruña, Spain, 10 pages. http://aci.ign.fr/Acoruna/Papers/Stoter.pdf (accessed 3 November 2008) Topografische Dienst Kadaster (2005). Generalisatievoorschriften TOP50vector, Handleiding, Generalisation regulations TOP50vector, Handbook. Ware, J. M., C. B. Jones and N. Thomas (2003). Automated Map Generalisation with Multiple Operators: A Simulated Annealing Approach. International Journal of Geographical Information Science, Vol. 17, No. 8, pp. 743-769. 207 Weibel, R. (1991) Amplified intelligence and rule-based systems. In B. Buttenfield & R. B. McMaster, eds. Map Generalisation: Making Rules for Knowledge Representation. Longman, pp.172-186. Weibel, R. (1995) Three essential building blocks for automated generalisation. In J. Mueller, J. P. Lagrange, & R. Weibel, eds. GIS and Generalisation: Methodology and Practice. London: Taylor & Francis, pp.56-70. Weibel, R., Keller, S. and T. Reichenbacher (1995) Overcoming the Knowledge Acquisition Bottleneck in Map Generalisation: The Role of Interactive Systems and Computational Intelligence. In A. U. Frank & W. Kuhn, eds. Spatial Information Theory - A Theoretical Basis for GIS (COSIT'95). Berlin, Heidelberg: Springer, pp.139-156. 208 $SSHQGLFHV 209 $SSHQGL[,9LVXDOLVDWLRQVRILQLWLDOGDWD 2QO\GLJLWDO Notethatthedatahavebeenmodifiedfortheprojectandthereforetheydonotfullyresemblethedataasusedinproduction. $SSHQGL[,,6\PEROGHVFULSWLRQVRIWHVWRXWSXWPDSV 2QO\GLJLWDO Notethatthesymbolshavebeenmodifiedfortheprojectandthereforetheydonotfullythesymbolsasusedinproduction. 210 $SSHQGL[,,,+DUPRQLVHGFRQVWUDLQWVIRURQHREMHFW GENERIC- Constraint ID ConstrainttypeGeometrytype Class Conditionforobjectbeing concernedwiththis constraint Constrainedproperty Conditiondependsoninitial value? ConditiontoberespectedAction Importanceofconstraint(1to 5,1islessimportant) EuroSDR-1- 1 Minimal dimensions polygonareanotargetarea>xmapmm 2 IFfinalareaxmapmm EuroSDR-1- 3 Minimal dimensions polygon initialarea><=xmap mm 2areayestargetarea=initialarea±x% EuroSDR-1- 4 Minimal dimensions polygonpolygoncontainsahole areaofanyholeina polygon notargetareaofhole>xmm 2 EuroSDR-1- 5 Minimal dimensions line/polygonlengthofanedge/linenotargetlength>xmapmm EuroSDR-1- 6 Minimal dimensions line/(polyline)widthnotargetwidth>xmapmm EuroSDR-1- 7 Minimal dimensions lineverticesdensitynotargetverticesdistance>xmapmm EuroSDR-1- 8 Minimal dimensions polygonwidthofprotrusion/recessnotargetwidth>xmapmm EuroSDR-1- 9 Minimal dimensions polygondepthofprotrusion/recessnotargetdepth>xmapmm 211 GENERIC- Constraint ID ConstrainttypeGeometrytype Class Conditionforobjectbeing concernedwiththis constraint Constrainedproperty Conditiondependsoninitial value? ConditiontoberespectedAction Importanceofconstraint(1to 5,1islessimportant) EuroSDR-1- 10 Minimal dimensions polygonareaofprotrusionnotargetarea>xmapmm 2 EuroSDR-1- 11 Shapeanygeneralshapeyes targetshapeshouldbesimilartoinitial shape EuroSDR-1- 12 Shapeany 1:nrelation (amalgamation) generalshapeyes targetshapeshouldbesimilartoinitial shape EuroSDR-1- 13 Shapepolygon initialvalueofangle=90° (tolerance=±x°) squarenessyestargetangles=90° EuroSDR-1- 14 Shapepolygoninitiallyhighconcavityconcavityyestargetshaperemainsconcave EuroSDR-1- 15 Shapepolygonelongationyes targetelongation=initialelongation± x% EuroSDR-1- 16 Topology lineand polygon initially,noself- intersection intersectionyesnoself-intersectionmustbecreated EuroSDR-1- 17 Topology lineand polygon coalescencenocoalescencemustbeavoided EuroSDR-1- 18 Orientationanygeneralorientationyes targetorientation=initialorientation± x% EuroSDR-1- 19 Positionanypositionalaccuracyyes targetabsoluteposition=initial absoluteposition±xmapmm EuroSDR-1- 20 Model generalisation anyclassyestargetclass=initialclass 212 GENERIC- Constraint ID ConstrainttypeGeometrytype Class Conditionforobjectbeing concernedwiththis constraint Constrainedproperty Conditiondependsoninitial value? ConditiontoberespectedAction Importanceofconstraint(1to 5,1islessimportant) EuroSDR-1- 21 Model generalisation anysymbolisationvalueyes targetsymbolisationvalue=initial symbolisationvalue 213 $SSHQGL[,9+DUPRQLVHGFRQVWUDLQWVIRUWZRREMHFWV GENERIC- Constraint ID Constraint type Geometrytype combination Class1 Conditionfor objectinclass 1being concernedwith thisconstraint Class2 Conditionfor objectinclass 2being concerned withthis constraint Conditiononboth objects(inthe initialdata)for themtobe concernedwith thisconstraint Constrained property Conditiondependson initialvalue? Conditiontobe respected Action Importanceof constraint(1to5,1 islessimportant) EuroSDR- 2-1 Minimal dimensions any-any minimal distance no targetdistance>x mapmm IFdistance< xmapmm THEN{action} EuroSDR- 2-2 Minimal dimensions polygon- polygon oneclassmustbe insidewithin anotherclass minimalareano targetarea>xmap mm2 EuroSDR- 2-3 Orientation line/polygon- line/polygon objectsare parallel(±x°) orientationyes object(class1)must beparalleltoobject (class2) EuroSDR- 2-4 Topology/ Position any-anyrelativepositionyes targetrelative positions=initial relativepositions EuroSDR- 2-5 Topology line/polygon- line/polygon withinasingle featureclass intersectionno noother- intersectionsmust becreated EuroSDR- 2-6 Topologyline-any object(class1) leadstothe object(class2) accessibilityyes targetaccessibility= initialaccessibility EuroSDR- 2-7 Topologyline-anyinitiallyconnectedconnectivityyes targetconnectivity= initialconnectivity EuroSDR- 2-8 Topologyany-any object(class1) overlapsobject (class2) object(class2) isunderobject (class1) overlappingno targetoverlapping= initialoverlapping 214 GENERIC- Constraint ID Constraint type Geometrytype combination Class1 Conditionfor objectinclass 1being concernedwith thisconstraint Class2 Conditionfor objectinclass 2being concerned withthis constraint Conditiononboth objects(inthe initialdata)for themtobe concernedwith thisconstraint Constrained property Conditiondependson initialvalue? Conditiontobe respected Action Importanceof constraint(1to5,1 islessimportant) EuroSDR- 2-9 Topologyany-any object(class1) containsobject (class2) object(class2) isinside object(class 1) topological consistency yes targettopology relations=initial topologyrelations EuroSDR- 2-10 Topology line/polygon- line/polygon minimaldistance xmapmm ANDareaofeach object>xmapmm2 IFdistancexmapmm IFdistance250.000 High end digitalSLR Camera, (12 bit, 18 MP)or better Industrialdigitalcamera, (12 – 16 bit, (39)– 60 MP GPS - INS (x,y,z < 0.1 m w,j,k < 0.01°) 0HGLXP IRUPDW 3UR6HPL3UR/RZFRVW $PDWHXU X X 7DEOH  &RPSDULVRQ RI FRPSRQHQWV IRU VPDOO DQG PHGLXP IRUPDW FDPHUD V\VWHPV The different components of the airborne imaging systems strongly influence the quality of the images, the efficiency of the airborne and photogrammetric workflow to generate digital orthophotos (DOP) and the reliability of the system, Table 2. Small Format Medium Format Low cost Amateur Semi-pro Professional Products / Results Vertical and oblique images (pretty pictures) AT without automatic tie points and full GCP require- ments Direct georeferencing of blocks, strips and single images without GCP‘s Direct georeferencing of blocks with automatic tie points, no or little GCP‘s Proc. time 1/100 DOP N/A 1 h/100 h 15 min/25 h 1 min/2 h 7DEOH  &RPSDULVRQ RI RUWKR LPDJH SURFHVVLQJ ZRUNIORZ RI VPDOO DQG PHGLXP IRUPDW FDPHUD V\VWHPV 235 With respect to this report the characteristics of state-of-the-art professional medium format digital cameras for airborne applications are: x Large image footprint ( 39 MP) x Single head or multi head system x Ruggedised metric design – fully calibrated systems x Short exposure interval (1 – 3 sec.) x Compact systems suitable for small single or twin-engine aircraft x RGB and/or CIR  Developments of medium format cameras in the last two years During the two years of the project, several significant changes and developments in the medium format sector took place. For instance the size of the digital sensors increased from 39 Mpix. to 60 Mpix. The new generation of sensors was developed by KODAK or Dalsa. The exposure interval of the newest sensors decreased from 2.5 sec to less than 2 sec. In concordance with the general increase in processing speed rapid orthophoto generation during the flight, e.g. for disaster monitoring is becoming possible. In addition integrated oblique and nadir looking systems are currently supplied with metric medium format cameras, instead of small format cameras. Therefore a closer look into the oblique sector will be necessary within this report. Medium format camera supplies offer dual, quattro and penta systems, which become very competitive to common large format systems. Last, but not least there are new players are in the medium format market (Z/I, Microsoft, Leica). For technical details see Table 3. These camera systems do not really fit in the later discussed concept of single head medium format camera systems. Instead - technology wise - these systems are downsized large format cameras, with a similar footprint to medium format camera systems. 8OWUD&DP/ 50 .' /HLFD 5&'  Sensor size (pixel) 9,735 x 6,588 6096 x 6500 7216 x 5412 No. camera heads 4 4 2 RGB + NIR Yes Yes Yes Exposure interval (sec.) 2.5 1 2.2 FMC integrated Yes Yes No Pansharpening Yes No No Exchangable lenses No No Yes System weight (kg) 55 55+ 65+ Plattform stabilisation external External integrated INS provided Provided integrated 7DEOH  &RPSDULVRQ RI QHZ PHGLXP IRUPDW FDPHUDV IURP /HLFD 0LFURVRIW DQG =, 236 So at this point in time the term "medium format" does not really represent a certain class of cameras, size of footprint or systems which can be separated from large format camera systems. Instead new terms have to be introduced to classify the different systems. The medium format systems have to be separated into single head and multi head camera systems and the downsized large format systems shall be called intermediate systems, Table 4. &DWHJRU\ 6HQVRU ([DPSOHV 3L[HO PDWUL[ 3L[HOVL]H >—P@ YLUWXDO 6H QVRU VL]H >PP@ /DUJH )RUPDW Vexcel Ultracam Xp 196 Mpix 6 103.9 x 67.9 Intergraph DMC 106 Mpix 12 166.0 x 92.2 0XOWLSOH +HDG 0HGLXP )RUPDW IGI quattro DigiCAM (@60 Mpix sensor) 226 Mpix 6 105.0 x 78.0 Trimble AIC x4 (@39 Mpix sensor)* 146 Mpix 6.8 95 x 70 ,QWHUPHGLDWH )RUPDW Vexcel Ultracam Lp 93 Mpix 6 67.9 x 47.5 Intergraph RMK-D 40 Mpix 7.2 41.5 x 46.1 6LQJOH +HDG 0HGLXP )RUPDW Trimble AIC 60 Mpix 6 53.9 x 40.4 Applanix DSS 439 39 Mpix 6.8 49.1 x 36.9 7DEOH  ([DPSOHV RI IRRWSULQWV RI ODUJH IRUPDW LQWHUPHGLDWH IRUPDW DQG PHGLXP IRUPDW V\VWHPV The standard approach for the generation of color images of medium format cameras are CCD-chips with a Bayer pattern. However such a CCD-chip is not capable to obtain additional IR information. Also CCD-chips with a Bayer pattern are not well suited for the acquisition of CIR images, see chapter 4.3.1. Therefore for simultaneous RGB+IR image acquisition a dual head camera system is necessary. By its nature a Bayer pattern CCD-chip requires a color interpolation, also called demosaicing, to interpolate a set of complete red, green, and blue values for each pixel. Separate camera cones with specific color filters overcome this problem, thus generate true color image values. To conclude: four different concepts for the generation of color and IR images are currently used, Figure 1. . Bayer Pattern Separate color camera cones B G R IR Trimble AIC P60 DSS 439 RapidOrtho DualCam RMK-D UltraCam-Lp MS-Image 8924 x 6732 5412 x 7216 6096 x 6846 5320 x 3600 Pan-Image 11704 x 7920 Ratio (1:1) (1:1) 1:1 1:2.2 Pan Sharpening + Bayer pattern RGB Pan IR Bayer Pattern + IR RGB IR )LJXUH  'LIIHUHQW FRQFHSWV IRU WKH JHQHUDWLRQ FRORU  ,5 LPDJHV 237  &RPSDULVRQ RI PHGLXP IRUPDW FDPHUDV WR ODUJH IRUPDW FDPHUDV Compared to large format frame cameras single head medium format cameras have some distinct technological differences, which are compiled in Table 5. Medium format Large Format Technology Camera Single head Multiple heads Image Size Max. 60 MegaPixel Max. 136 MegaPixel Colour Either RGB or (CIR) Separate heads, (Pan, R, G, B, NIR) image fusion Lenses Interchangeable Fixed Airborne system FMC Only mechanically TDI System cost Low Expensive System weight Light Heavy Energy consumption Low High 7DEOH  0DLQ GLIIHUHQFHV EHWZHHQ PHGLXP DQG ODUJH IRUPDW GLJLWDO IUDPH FDPHUD V\VWHPV Beside the technological differences the vertical range of manufacture of digital medium format cameras and large format frame cameras is quite different. While in large format cameras all components are developed, optimised and tested for airborne applications, medium format camera systems including the processing software are often a composition of several off-the-shelf products for professional photographers combined with special features for the airborne environment. This makes comparisons between different systems and calibration efforts even more difficult. As shown in later sections these differences also have a great influence on the radiometric workflow and the calibration of the cameras. One of the main advantages of single head medium format cameras is the lower system price and the possibility to fly with small and cheap aircraft. The overall cost and the effort for an aerial survey and subsequent ortho photo production are related to many factors of the photogrammetric workflow. Figure 2 gives a comparison of a medium format and a large format camera system for an aerial survey of a small area. The comparison of the different processing steps reveals that the major advantage of medium format cameras are the lower costs for the aerial survey and the easier and faster postprocessing of the images of the single head cameras. Assuming an automatic tie point matching and a precise GPS/INS the cost of aerotriangulation is not very much higher for medium format cameras because this is normally a highly automated procedure. Due to the smaller ground coverage of an image more ground control points may be necessary. Nowadays the photogrammetric block does not necessarily rely solely on ground control points, but even for an integrated sensor orientation and quality assurance a certain number of ground control points are necessary, depending upon the number of images taken. 238 Data delivery Aerial survey Ground control points Aerotriangulation DTM Digital orthophoto production Radiometric postprocessing Medium Format Large Format )LJXUH  &RPSDULVRQ RI WKH FRVWV RI WKH SKRWRJUDPPHWULF ZRUNIORZ IRU ODUJH IRUPDW DQG PHGLXP IRUPDW FDPHUD V\VWHPV The refinement of the seamlines between adjacent images is one of the manual and labour intensive steps in digital ortho photo generation. Due to the relatively large number of medium format images, more manual labour is necessary for the generation of a seamless ortho photo mosaic. Together all of the factors lead to a lower cost reduction per area, compared to large format cameras (Figure 3), making medium format cameras less competitive for large area surveys. Area [km ²] Costperkm² Medium format camera Large format camera Area [km ²] Costperkm² Medium format camera Large format camera )LJXUH  &RVW GHJUHVVLRQ SHU DUHD IRU DHULDO VXUYH\V DQG RUWKR SKRWR JHQHUDWLRQ A big advantages of medium format cameras are interchangeable lenses with focal lengths of 35 mm to 210 mm. Different lenses allows missions to be flown at different altitudes to either maintain the desired resolution or maintain a predefined strip width during joint flights with others sensors, e.g. laserscanning. Also with interchangeable lenses the stereo / DEM capabilities may be changed as well as occlusions in narrow streets etc. during ortho photo production. However lenses with a long focal length generally cause several special problems in terms of their interior orientation and calibration: 239 x Weak Geometry x Narrow field of view x Convergence angle in laboratory x Principal point recovery x Out of focus images in laboratory  Geometric Potential The achievable geometric potential of a digital camera is related to the “metric” properties of the camera, which stands for a determinable and stable interior orientation.  Interior Orientation The determination of the interior orientation of CCD-colour sensors based on the Bayer-pattern is related to some general sources of error due to longitudinal and transversal chromatic aberration, Cronk et al., 2006. However these errors are relatively small and only applicable for close range applications at the highest precision. Nevertheless for close range applications medium format cameras have shown a very high geometric potential, with relative accuracies of up to approx. 1:200,000 e.g. Shortis et al. (2006). However the airborne environment imposes special requirements on the camera system. To survive the shock and vibrations experienced in the airborne mapping environment a rigid camera body and a fixed lens mount is necessary for a stable interior orientation. Large format cameras generally operate with a fixed lens aperture, Kröpfl, et al., 2004. On medium format cameras the lens aperture is generally set by the amount of light available and the requirements of the shutter speed to minimise image movements. The lens aperture changes the interior orientation to a small extent. Also the work with interchangeable lenses requires a new (on the job) calibration every time the new lens is mounted. Even with a ruggedised design and special locking mechanism of the lens mount, some parameters of the interior orientation (especially the focal length) may change in the airborne environment due to changes in the air pressure when flying at higher altitudes. This is of special relevance for direct georeferencing, because the errors in the interior orientation are directly visible in the accuracy of the object coordinates. Therefore a simultaneous on the job calibration in terms of an integrated sensor orientation should be done. However the airborne environment imposes special requirements on the camera system. To survive the shock and vibrations experienced in the airborne mapping environment a rigid camera body and a fixed lens mount is necessary for a stable interior orientation. It has been reported several times, that the camera body and the lens mount of medium format camera contributes to the lack of stability. This lack of rigidity holds especially true if the cameras do not have a rigid metal body nor are designed as metric cameras, Peipe, 2005, Shortis et al., 2006. Since all suppliers of professional medium format systems have fixed their digital backs with the camera body, the effect of gravity on the spring mounted CCD arrays is no longer a problem. The interior orientation parameters are generally determined in the laboratory through a bundle adjustment with self-calibration procedure. The parameters of the interior orientation generally include: 240 x principal point coordinates (xp, yp). x principal distance (c). x distortion parameters: x radial lens distortion x de-centering lens distortion x affine deformations Even with a ruggedised design and special locking mechanism of the lens mount, some parameters of the interior orientation (especially the focal length) may change in the airborne environment due to changes in the air pressure when flying at higher altitudes. This is of special relevance for direct georeferencing, because the errors in the interior orientation are directly visible in the accuracy of the object coordinates. Therefore a simultaneous on the job calibration in terms of an integrated sensor orientation should be done. However, compared to the lab calibration with converging and rotated images the network geometry of an airborne photogrammetric block is generally not optimal for selfcalibration of all parameters of the interior orientation. Although the camera body instability limits the accuracy that can be achieved with these cameras, photogrammetric measurements achievable with professional digital medium format cameras in recent years allow for significantly higher levels of accuracy than ever before.  MTF The modulation transfer function (MTF) of the resolving power of the optics should be set in accordance to the resolving power of the CCD-chip. New chips have a 6.0 µm sensor size which equals approx. 85 line pairs per millimeter. The resolving power of common medium format lenses was developed for the resolution of analogue film, which is 40 to 60 line pairs per millimetre, causing image blur at the edges of the images. Therefore new lenses have to be used, in order to meet the higher resolving power of the small CCD-elements. It has to be noted, that the MTF of a lens is a function of the distance from the image centre, the lens aperture (f-stop) and the spectrum of the light. This is the reason why these special lenses only work with a maximum aperture of f/4.01 . Similar experiences with inadequate lenses were made with the UltraCAM-D, Souchon et al., 2006.  Base to height ratio The achieved base-to-height ratio - = B/hg reflects the geometry during image recording and is one main factor for the resulting point accuracy in object space. The base-to-height ratio is given by equation ) 100 1(* ' p c s h B g - (1) where sƍ depicts the sensors extension [m] in flight direction and p is the forward overlap in percent, the linear dependency of - to the sensors size is obvious. Compared to analogue times the - is much smaller in the digital world. Large format cameras such as the UltraCAM have a - of 0.27, the DMC 1 http://www.rodenstock-photo.com/en/main/products/lenses-for-digital-photography/hr-digaron-w/ 241 has a base-to-height-ratio of about 0.3. Since the high resolution size of the CCD-chip is the similar for all medium format systems on the market the base-to-height-ratio for a photogrammetric flight with a 60 % endlap varies from 0.32 for a 50 mm lens and 0.11 for a 150 mm lens. Thus medium format cameras have similar base-to-height-ratios, compared to digital large format cameras. In turn the object point accuracy of medium format cameras with a 50 mm lens is similar to a large format camera.  Minimum GSD of medium format cameras Customers are demanding higher and higher ground resolution. The highest possible ground resolution (GSD) for aerial surveys with standard endlap (60 %) depends on several factors such as the image exposure interval of the camera 't and blur due to image motion. The exposure interval is related to the length of the base (b) related to the endlap p (in percent), the velocity of the aircraft over ground (vg), the GSD and the number of pixels (npix) of the CCD-Sensor in flight direction: ) 100 1( * p v nGSD v b t g pix g ' (2) The minimum exposure interval of the digital medium format cameras is somewhere between 1.6 – 2.7 s. Figure 4 provides an overview which minimum GSD for a photogrammetric aerial survey with an endlap of 60 % applies at what speed over ground. Speed over Ground [kn/h] [m/s] 58 68 78 87 97 107 117 126 136 146 156 165 175 185 194 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 10 cm 7.5 cm 5 cm ExposureInterval[s] )LJXUH  ([SRVXUH LQWHUYDO RI PHGLXP IRUPDW FDPHUDV 3  0SL[ UHODWHG WR *6' DQG JURXQG VSHHG 242  Image motion Airborne images are acquired from moving platforms such as aircrafts or helicopters. The movement of the sensors during the exposure influences the quality and the sharpness of the acquired imagery. For analogue airborne cameras this image motion is taken care by forward motion compensation (FMC). For large frame digital airborne sensors the translation effect of the FMC is solved digitally by moving the charges on the matrix area itself (time delayed integration, TDI). Additional rotational movements are compensated from the stabilised mount. For medium format sensors an active mount is typically not available – exceptions are the DSS and cameras which are used as a sub-system e.g. in combination with laser scanners mounted on a common stabilized platform which are then stable. Also TDI is not available for the medium format digital sensors due to the Bayer pattern of the CCD-chip. The image motion u is related to the aircraft velocity over ground vg, the exposure time te, the focal length c, the flying height above ground hg and the size of the pixel sp see formula 2. GSD stv h c stv u u peg g peg th ** * 2 1 **** 2 1 2 | (3) where only 50% of the theoretical image motion uth is valid in the images. For digital imagery the smear due to image motion should not exceed 0.5 pixel. Since aircraft velocity and GSD are typically given by default for a certain project, exposure time is the only variable to minimise effects of image motion, if suitable light conditions are available. Figure 5 highlights the problem for a GSD of 5 cm. 0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 30 40 50 60 70 80 90 100 Speed over ground ImageMotion[µm] 1/500 s 1/1.000 s 1/2.000 s 0.5 Pixel 1 Pixel 58 78 97 117 136 156 175 194 [kn/h] [m/s] )LJXUH  ,PDJH PRWLRQ u IRU D *6' RI  FP DW GLIIHUHQW VSHHGV RYHU JURXQG DQG GLIIHUHQW H[SRVXUH WLPHV Exposure time on the other hand is coupled with lens aperture and the sensitivity of the digital sensor given by the ISO value. However a higher sensitivity (ISO number) is always associated with higher image noise levels. The aperture is also a limiting factor, because a wide aperture may cause or enhance vignetting and optical aberrations. To sum up: the image motion limits the GSD of medium format cameras and the short exposure interval necessary limits the flying operation times under poor lightning conditions when compared to large format cameras. 243  *HRPHWULF DQG UDGLRPHWULF FDOLEUDWLRQ The calibration of digital cameras includes geometric and radiometric issues. From the viewpoint of a photogrammetrist, geometric issues are the more important. For large format cameras detailed tests within the frame of OEEPE and EuroSDR were conducted, e.g. Cramer, 2007. The special geometric problems of multi head cameras are discussed quite extensively in the literature, e.g., Jacobsen, 2007. Research in the calibration of large format cameras systems is an ongoing process with the aim to develop a calibration procedure of the whole camera system and its subsystems, including GPS/INS, radiometric and geometric issues and the whole photogrammetric processing chain, EuroDAC² (2010).  Geometric calibration Due to the compactness and the low weight of medium format cameras, laboratory calibration of the interior orientation is relatively easy to obtain. As mentioned in section 3.1.1 the interior orientation of medium format cameras may change under airborne conditions. Therefore the geometric calibration of the medium format camera system should be done in four different levels, from laboratory measurements to long term camera stability analysis: 1. Laboratory calibration either conventionally with a two or three dimensional test field or with the so called taut line method (Habib and Morgan, 2005) 2. In flight calibration over a calibration range, e.g. Honkavaara et al, 2008 3. Simultaneous in flight calibration on the job to adjust for project specific circumstances e.g. focal length adjustments due to flying height. 4. Long term camera stability analysis to determine the necessary calibration intervals. Practical experience and feedback from the industry will help to set up specifications for regulating the use of medium format digital cameras in terms of calibration reports.  Geometric performance of medium format - Results of the DGPF-Testbed In 2008 a comprehensive project on the empirical investigation of the performance of digital photogrammetric airborne cameras was performed under the umbrella of the German Society of Photogrammetry, Remote Sensing and Geoinformation (DGPF). Empirical test flights of twelve different camera systems took place over the test field Vaihingen/Enz, Cramer, 2009. In general the DGPFTestbed provided an accuracy of all examined frame camera systems of 0.25 – 1 Pixel (horiz. and vert.) in the object space. Cramer, 2009. Of special interest to this report were the three test flights with digital medium format cameras from IGI (DigiCam) and Trimble (TrimbleAIC). The Rolleimetric AIC x-1 (Trimble Aerial Camera) P45+ (39MPixel) c = 47mm revealed relatively large systematic image residuals of 4,28 µm pixel after self calibration has been performed, see )LJXUH . The reason for this unsystematic errors is seen in the fixing of the CCD-chip to the camera body with eight screws and residuals due to the lens. However for the full exploitation of the geometric accuracy a self calibration with additional parameters is necessary. 244 )LJXUH  6\VWHPDWLF UHVLGXDOV RI LPDJH HUURUV 7ULPEOH $,& -DFREVHQ  To sum up: The geometric accuracy of the medium format cameras is a little below the values of large format camera systems such as DMC and UltraCam, but differences of the environmental conditions may outweigh the differences of the cameras systems.  Radiometric accuracy and calibration Large format cameras generate color images through individual panchromatic camera cones, equipped with spectral filters. Thereby the different colors (R/G/B/NIR) are spectrally separated. Therefore large format cameras are well suited for radiometric calibration or remote sensing applications. Color images of medium format cameras however are typically generated via a CCD-chip with a Bayer micro colour filter. The quantum efficiency (=sensitivity) of the CCD-Sensor itself is different at different wavelength and colors, see Figure 7. In the medium format case, it is important to notice, that there is an overlap of the different primary colors. Also in the near infrared (< 800 nm) the sensitivity of R/G/B is very similar, thus there is no chance to separate the different signals, see also chapter 4.3.1. 245 )LJXUH   6SHFWUDO UHVSRQVH TXDQWXP HIILFLHQF\ RI WKH .$) VHQVRU FRORU YHUVLRQ  An increasingly important task is the “radiometric accuracy” and the radiometric calibration of aerial cameras. The first goal of such a radiometric calibration is to eliminate the influence of the optics and the sensor and make sure that the resulting images will have the same sensitivity throughout the image. The radiometric calibration is also split in two main parts. The first part of the radiometric camera calibration is done by the manufacturer to eliminate radiometric dysfunctions of the sensor such as: x Defect pixels x Dark Signal Non Uniformity (DSNU) x Individual sensitivity of each single CCD pixel x Vignetting (partly) x Influence of aperture (partly) However, the manufacturer radiometric calibration effort of the differs between medium format cameras, e.g. the DSS is calibrated radiometrically using MacBeth targets, integrating spheres, and optimisation software, Mostafa, 2004. White balance calibration procedure or more general the Look-Up-Table (LUT) generation is the second step. For each project the user performs this type of calibration individually. After post processing the user can set the white balance for the project using some example images which cover the typical surface,. With suitable software the user can also minimise remaining vignetting and lens 2 http://www.kodak.com/global/plugins/acrobat/en/business/ISS/datasheet/fullframe/KAF-50100Long Spec.pdf) 246 aperture effects which may occur at low f-stops. Radiometric problems of the medium format cameras are related to several issues. x The radiometric postprocessing of the raw imagery, which come in a black box raw format is generally done in a software environment primarily developed for non photogrammetric users, but for professional photographers. Therefore issues such as radiometric linearity, atmospheric correction etc. are not a primary issue. x There is no single standard algorithm for converting data from a Bayer filter or Foveon sensor into RGB format. x the color infrared option causes longitudinal chromatic aberrations. Due to the strong sensitivity of the CCD-chip in the IR-light the resulting image is more or less a reddish coloured IR-image, Grenzdörffer, 2006 x During the postprocessing of the raw images, the images may be corrected and manipulated with respect to: x a colour balancing due to the atmospheric conditions x general visual expectations of the users (e.g. grass has to be green) x vignetting and influence of aperture x histogram enhancements for 16 bit ĺ 8 bit conversion x image sharpening and noise reduction. x the degree of the radiometric postprocessing and the resulting colors are solely subject to the visual impression of the interpreter, and x proposed radiometric corrections steps (e.g. Honkavaara et al, 2009) and quality measures are difficult to obtain. The radiometry of the large format cameras with the necessary image fusion is also not free of errors, as practical experiences show, e.g. Schroth, 2007, Souchon et al., 2006. Some of the effects are characteristic for digital cameras such as the total reflection e.g. on white surfaces or on glass surfaces such as roof windows, winter gardens or green houses. Other effects are related to the fact that the panchromatic channel covers parts of the near infrared reflection, causing a bias after image fusion in the blue channel, Schroth, 2007. Some effects are directly linked to the pan sharpening process, e.g. color echoes on the edge of bright areas.  Color infrared with medium format cameras Except Leica Geosystems, most manufacturers of medium format cameras claim their sensor is capable for either RGB or colour infrared (CIR) by simply changing a filter in front of the lens. The filter blocks the blue light and the green, red and near infrared (NIR) light passes the optics onto the Bayer pattern of the CCD-sensor. The blue sensitive CCD-elements receive only IR-light while the green and red pixels also receive IR-light. In the postprocessing the additional light is subtracted in order to obtain a CIR-image. 247 B, G, R, IR Light UV-IR-Cut-Filter (< 400 nm und > 700 nm) UV-B-Filter (> 520 nm) B, G, R, IR Light B, G, R Light G, R , IR Light IRG + IR R + IR G + IR BG R G BG R G Colour interpolation Colour interpolation + Subtraction Post processingImage Bayer-Pattern )LJXUH  &RQFHSW RI FRORXU DQG &,5PRGH RI PHGLXP IRUPDW FDPHUDV Due to the strong sensitivity of the CCD-chip in the IR-light the resulting image is more or less a reddish coloured IR-image, Grenzdörffer, 2006. RGB CIR RG(B)+IR )LJXUH  5*% DQG &,5 ZLWK %D\HU SDWWHUQ &&' Thereby it is noteworthy that due to the wavelength of the near infrared light, the focal plane of the near infrared channel is not localised at the same place as the RGB light. This causes a longitudinal chromatic aberration, see Figure 10a. 248 R G B G B R R G NIR R G NIR )LJXUH  D &KURPDWLF DEHUUDWLRQ E FKURPDWLF DEHUUDWLRQ RI &,5PRGH Depending on the lens quality this effect can be reduced by using different lens types and coatings. In common lenses where the focusing plane is adjusted to a mean wave length (green), imaging errors in the red and blue range are minimised. With the longer near infrared wavelength however the chromatic aberration increases strongly, leaving the infrared light out of focus, Figure 10b. This leads to significant differences in the interior orientation and also Moiré effects may be observed, Grenzdörffer, 2006, see Figure 11. To sum up the single head CIR-images of medium format cameras are difficult to handle and the resulting images are not a true CIR, as known from the large format cameras. Multi head systems such as the Leica RCD100 and Trimble AICx deal with this problem and offer a parallel operation of IR and RGB cameras. )LJXUH  0RLUp HIIHFWV LQ &,5PRGH RQ DJULFXOWXUDO ILHOGV  6WDQGDUGL]DWLRQ Standardisation for aerial cameras and the photogrammetric processing chain is taking place at several levels, from ISO down to national initiatives. However most of the standardisation effort is related to 249 large format cameras, thus sometimes neglecting and more or less excluding medium format cameras. In other instances the standardisation is very general and finally not of great practical use. On an international level ISO/TC 211 - Geographic information/Geomatics is to develop a family of international standards that will, in case of digital cameras establish a universal description of image data and meta information about the aerial survey, image geometry and navigation data (exterior orientation). Within the ISO there are several ongoing projects dealing with standards related to medium format cameras and photogrammetry, e.g. ISO/TS 19101-2 – Reference Model – Part 2: Imagery, ISO 19115-2 – Metadata – Part 2: Extensions for imagery and gridded data and ISO 19130 Sensor and data model for imagery and gridded data (presently deleted from list due to lack of progress). The Open Geospatial Consortium (OGC) is currently developing several standards including issues related to medium format cameras, e.g. an Image Geospositioning Service, Web Map Service – application profile for EO Products. All of these services are at an early stage, OGC (2008). In Germany the standard series DIN 18740 – Photogrammetric Products (Part 1 – 4) covers especially large format cameras, Reulke et al., 2007, DIN 2007. Part 4, finalised in Sept. 2007 deals with the requirements for digital aerial cameras and digital aerial photographs. Focus is given to digital aerial cameras, aerial survey flights and the digital aerial photograph. For digital aerial cameras the standardisation provides quality measures on: general requirements of the camera and its components, camera calibration (geometry and radiometry) and requirements of sensors for positioning and attitude determination. The geometric quality related to the image product has to be documented in a manufacturer certificate, in which the camera system and its subsystems have to be geometrically and radiometrically calibrated. The validity of geometrical calibration at the time of flight has to be proven by validation test (less than one year ago) or new calibration (less than two years ago), DIN 2007. Due to the fact that digital aerial systems are more than just cameras and the final quality is not only related to the sensor standards should not only focus on the certification of the cameras itself but include the whole end-to-end processing chain. Based on these facts the USGS has formulated a different approach. Individual cameras are not the subject of certification, but a “type certification”, Stensaas, 2007. With this approach it should be ensured that the sensors are designed, built and tested to reliably deliver data with a high quality. However this is only true if the operating company operates and maintains the system properly. Currently the DSS 439 is the only medium format camera system certified by the USGS. One of the most relevant current projects in EuroSDR is the developments of future certification strategies of digital airborne cameras (EuroDAC² activity). EuroDAC² covers not only the large format camera systems but all types of cameras used for professional mapping appliciations. Despite the USGS approach of a so called type certification of different camera producers the EuroSDR approach is still focusing on the validation and calibration of single camera systems as well as the certification of the flying companies. For the current status of the EuroDAC² see Cramer et al, 2010.  $SSOLFDWLRQ 'RPDLQV From an application standpoint, it is safe to say that medium format digital cameras are not their large format cousins, but rather a niche market solution for specific project types, Artés, 2004. The largest proportion of medium format cameras are used as a sub-system of integrated airborne data acquisition platforms consisting of laser scanners (LiDAR) combined with imaging component and GPS/inertial sensors for direct platform orientation. This approach is highly effective, not only for classifying the returned laser pulses but even more for rapid production of orthoimages based on the height data obtained from laser scanning and the directly measured sensor orientations. Such inte- 250 grated airborne systems are operated by many airborne companies. A special requirement for joint laser surveys is an extremely high light sensitivity and a fast shutter speed. Another more or less exclusive market for medium format cameras are mapping small, irregular shaped areas, strip mapping, transmission line corridors or pipeline contracts, which do not always require the ground coverage produced with a large format camera. A direct georeferencing capability with GPS/INS is in these instances a tremendous advantage because it allows for a greater degree of freedom in the aerial survey, such as a strip map coverage where a single line imagery can be utilised without the need for a second flight line, and small blocks can be easily georeferenced to produce orthophoto mosaics. Direct georeferencing enables “hot spot” monitoring of small places of interest with single images (ortho photos). These capabilities are also important for disaster response surveys which require quick data turnaround, Ip et al, 2007. For smaller aerial survey/remote sensing organisations, the medium-format alternative is changing the face of the industry, as an affordable technology that can deliver increased performance, a marked reduction in operating costs, and most importantly a digital product when it is most needed. The following overview addresses more or less exclusive application domains of medium format and competitive applications to large format cameras. ExclusivemarketofmediumformatļCompetitivemarketto largeformat Laserscanning & Camera x Mapping & Orthophotos x True Orthophotos x 3D-City Models Strip Mapping x Linear-based mapping projects x Pipeline surveys x Hydro corridors x Transportation routes Rapid Response Imaging x Rapid mobilisation for disaster management x Time-dependent image acquisition x Homeland Security digital imaging Agriculture and Forestry x Species identification x Timber value assessment x Disease control and monitoring x Precision Farming GIS and Urban applications x Urban and regional planning x Urban Hot-Spot Monitoring x 3D-Models (Nadir + Oblique) Remote Sensing x Environmental research x Coastal zone monitoring x Colour-Infrared imaging For smaller aerial survey/remote sensing organisations, the medium-format alternative is changing the face of the industry, as an affordable technology that can deliver increased performance, a marked reduction in operating costs, and most importantly a digital product when it is most needed. 251  0DUNHW VXUYH\ RI PHGLXP IRUPDW FDPHUD V\VWHPV The following detailed description and comparison includes seven different digital medium format camera systems. The camera systems compared (alphabetical order) are: Applanix (Trimble) DSS 439, DigiCAM-H39, DiMAC, Leica RCD100 / RCD105, RMK-D, Trimble AIC, UltracCAM-L. While all seven camera systems consider themselves medium format systems, the systems differ greatly in many features, such as: x The possibility of dual or multi sensor head configuration x Optics and shutter speed x CIR option (special optics) x Min. exposure interval x External or internal data storage x Mount adapters for existing camera mounts x The availability of a FMC x Optional elements such as GPS/INS etc. x Image processing and radiometric calibration software x ... The data for the product survey is mainly derived from the product information, provided by the manufacturers. The largest suppliers of medium format camera systems are the Applanix DSS, the Trimble AIC camera and the DigiCAM of IGI. The DIMAC system is a smaller player. The Leica RCD100 / RCD105 as well as the UltraCamL and the RMK-D are quite new products in the market. Therefore no information of their commercial success is possible at this point in time. As many of the medium format cameras are used together with a LiDAR-system, four suppliers of the cameras have been integrated with airborne LiDAR systems. Optech uses exclusively the Trimble AIC for their ALTM laser scanners. The Toposys Harrier laser scanners can be supplied with either a DSS or a Trimble AIC camera. IGI-DigiCAM is streamlined for the LiteMapper laser scanning system of IGI. Finally the Leica RCD105 has special features for the ALS 50-II, ALS 60 laser scanners of Leica Geosystems.  RMK-D Target market of the RMK D from Intergraph’s Z/I Imaging are owners of film cameras seeking to enter the digital marketplace. The concept of the RMK-D is a turnkey multi head system with four panchromatic sensors and color + NIR filters. The camera therefore provides color without pan sharpening or bayer pattern interpolation. The CCD-chips from DALSA are especially designed for the camera. Pan imagery is generated from color as part of post-processing, providing high radiometric resolution for remote sensing. The minimum exposure rate of the camera is 1 s. Due to the panchromatic multi camera head approach, digital forward motion compensation (TDI) is available. The RMK D uses light weight Solid State Disk (SSD) technology and other low energy consuming components. This results in low weight and power requirements that allow it to fit in small, single- 252 engine aircraft. Huge effort has been invested in order to guarantee the highest level of geometric and radiometric stability, Dörstel, 2009, Madani, 2010, Key Features x Multi-spectral sensors from DALSA: RGB, and NIR simultaneously x 1:1 color ratio: for RGB, and IR, no Bayer pattern x Large base to height ratio of 0.4 for high stereo accuracy x The minimum exposure rate is 1 sec per frame x FMC implemented: Based on TDI x Camera mounts: Compatible with T-AS and Z/I Mount. x High resolution: 8cm GSD at 500m flying height  Trimble / Applanix DSS – 439 The former Applanix, now Trimble Digital Sensor Systems (DSS) consist of completely integrated medium-sized digital camera, the Applanix POS/AV 410 GPS/inertial system and a flightmanagement system software for generating orthomosaics, Applanix, 2010. The GPS/INS provides the exterior orientation parameters in both real-time and post-mission mode. An active azimuth mount control automatically removes the aircraft drift angles based on real-time POS/AV navigation data. The drift correction, based on a single axis azimuth mount has an accuracy of < 0.5 deg (RMS). The active mount allows for flights in a rough environment and the generation of systematic block pattern. Although primarily used to generate high-resolution colour and colour infrared digital ortho-photos/mosaics by direct georeferencing and an existing Digital Elevation Model (DEM), the system also supports full stereo imagery for DEM extraction and visualisation. The data interfaces directly and seamlessly with photogrammetry software to allow for fast and highly accurate map production, Mostafa, 2004. A dual camera option enables the generation of 4-band imagery (RGB+NIR) in a single pass. The DualCam adds a second DSS camera with a monochromatic CCD array specifically configured to capture (NIR) imagery. GSD ranges from 3.3 cm to 1.0 m, depending on platform and using 40 mm or 60 mm lenses. The 250 mm lens enables the collection of digital aerial imagery at high altitudes, especially for surveillance operations or on board a high altitude unmanned airborne system (UAS). The DSS system can be flown in small, single engine aircrafts, ultra-light aircrafts, helicopters or UAS. The camera system was certified by the US Geological Survey (USGS) as a metrically stable mapping grade system in September 2007. The DSS is the only medium format system currently certified by the USGS, Applanix, 2010. Due to the single sensor approach and direct georeferencing a rapid ortho photo generation is possible. Applanix claims, that under certain specifications and a pre existing DEM an orthophoto may be computed in as a little as 12 seconds per image. x Key Features x Fully integrated system with: x GPS/INS, flight management system, azimuth mount, processing software 253 x Pilot only operation mode x Single or dual camera system with 39 MP sensors from KODAK x Min. exposure interval 2.8 sec. x Color and CIR-Mode x Ruggedised external data storage x Special lenses (40 mm, 60 mm, 250 mm (AeroLensTM )) x Shutter speed up to 1/4.000 sec. x Total weight ca. 33 kg  Trimble AIC The Aerial Industrial Camera (AIC) series from Trimble (former RolleiMetric) is designed for aerial and industrial purposes, Trimble, 2010. The 22MP, 39MP or 60 MP digital backs from PhaseOne are rigidly fixed to the aluminum camera body. Everything is optimised for photogrammetric use, with interchangeable lenses and stabilised bayonet. The focal lengths of the medium format lenses range from 28 mm to 100 mm. The maximum shutter speed is 1/1,000 second, enabling a minimum GSD of 5 – 10 cm, depending upon the speed of the aircraft. Filter change allows acquisition of images in RGB, NIR and CIR. For the 39 MP and 60 MP sensor the pro lenses, especially designed for digitalcamera sensors and small pixel size, are necessary. Trimble carries out geometric calibration and Phase One executes radiometric calibration of the sensor. The camera control is done either by a PC or a PDA. Interfaces with IMU/GPS systems (event signal) and common flight management systems (FMS) (trigger signal) are given. The image data of the camera may be stored on board using a 8 or 16 GB CF-memory card, holding up to 400 images or transferred via IEEE 1394a connection to a PC. The new AIC xN architecture allows joint fitting of up to eight standard AICs in one frame, using electronic boards for accurate synchronization. All AIC’s are in full communication with each other. The AIC x2 combines two cameras and the AIC x4, four. Depending on desired overlap, the footprint may cover up to 13,000 x 10,000 pixels. Key features x Single camera system, 60 MP – Phase One digital back x Colour and (CIR) x Multi sensor head configuration possible (up to 8 cameras) x Min. exposure interval (60 MP) under airborne conditions 2.5 sec. x Internal and external data storage x Ruggedised design – fully calibrated x Interface to standard flight management systems x Weight ca. 2.6 kg (+ optional PC) 254  DigiCAM- (H39 / H50 / H60) The DigiCAM camera body from IGI is a very compact camera weighting 1.7 kg (without lens). The system with may be configured with 39 MP, 50 MP or 60 MP digital backs from hasselblad combines modified professional digital cameras with a graphical user interface for real-time preview together with the CCNS/AEROcontrol. The camera settings are adjusted on an 8” TFT monitor, by checking quick-views and histograms of images in real time. The CCNS4 triggers the system. Determination of exterior orientation parameters is done using the AEROcontrol GPS/IMU system. Along with the camera, each of the two 500 GB storage units onboard can store up to 6,400 raw images and be exchanged during flight to extend storage capacity. Standard units may be replaced for high-altitude flights by flash memory units with 1,150 image capacity. The focal lengths of the available lenses range from 28 mm to 300 mm. The maximum shutter speed is limited to 1/800 second. The modular design enables a change from RGB mode to colour-infrared within minutes for all lenses. The minimum exposure interval is as fast as < 1.6 s (< 1.9 s for the 39 MP sensor) in the burst mode and 1.9 sec (2.1 s for the 39 MP sensor) in the continuous mode. (IGI, 2010, Petrie, 2009b) Two or more DigiCAMs can be coupled either to increase image size or allow for faster flying speed. The IGI mount hosts up to five cameras and the adapter fits into most common mounts. In the case of multiple cameras, synchronisation can be carried out within a few microseconds. For nadir looking multi head systems special software is provided to compute a stitched and distortion free image with a "virtual" focal length from two or more single images. IGI also offers a special feature for the multi head systems to switch from a nadir looking system to an oblique system. Key Features x Single camera system with up to 60 MP x Multi sensor head configuration possible (2 - 5 camera heads) x Colour and CIR x Min. Exposure interval 1.6 sec. x External Data storage for ca. 6.400 images x Ruggedised design – fully calibrated x Mount adapters available for existing camera mounts x Large set of optional elements (Flight management (CCNS), GPS/INS (AEROcontrol), LiDAR (Litemapper) x Total system weight ca. 7 kg  DiMAC - (Digital Modular Aerial Camera) The DiMAC system (Digital Modular Aerial Camera, produced by Aerophoto in Bergem, Luxembourg, uses single and multiple camera units. DiMAC uses digital backs from Phase One. Therefore sensors of 39 MP or 60 MP are offered. Each camera head of the DiMAC system acquires either RGB or near infrared image. Compared to the other medium format camera suppliers DiMAC offers a true FMC (electro-mechanical driven by Piezo technology). This enables GSD ranges from 2 cm to 1 m. The interchangeable lens may be one of four focal lengths: 47 mm, 70 mm, 120 mm or 210 mm. 255 The camera cylindrical frame allows for combining up to four camera modules. A light architecture may be constructed using just one camera in the camera cylindrical frame; but two cameras may also be placed here, creating a RGB image of 13,000 pixels by 8,900 pixels. Two additional cameras may be placed in the vacant holes, resulting in an image of 10,500 by 14,400 pixels. The user also obtains a software which seamlessly combines the images from the two camera heads into a single frame. Another configuration is formed by adding a Near Infrared in one camera mount covering the same area as the other one in the other camera mount, or by placing a 47-mm Near Infrared camera in camera mount 1 covering the same area as camera mount 2 and camera mount 3 together (Dimac, 2010). Key Features x Flexible and modular configuration of 1 – 4 camera heads within one camera cylindrical frame x Large footprint (13,000 by 8,900 pix with 2 cameras) x Fits into existing camera mounts (40 cm diameter) x True electro-mechanical FMC x True Colour – Phase One digital back x Min. exposure interval 2.1 sec. x External data storage on RAID 1 System x Total weight ca. 90 Kg  Leica's RCD medium format cameras In 2008 Leica Geosystems introduced a multipurpose medium format camera system, manufactured by Geosypatial Systems Inc. The basic camera unit, called CH39 designed from the ground up as an airborne digital metric camera solution. It includes a 39 MP Kodak sensor. A special features of the camera system is a focal plane shutter, commonly used for reconnaissance purposes, allowing for a shutter speed of up to 1/4.000 s in order to minimize image motion. The CH 39 is available with a variety of lenses. The lenses of 35, 60 or 100 mm focal length are optimised for both RGB and CIR. The CIR use requires filter/compensating optic to avoid chromatic aberrations, software and calibration. The lenses have a fixed infinite focus and also a fixed aperture value of f/4. Based on the camera unit Leica Geosystems offers two camera systems to the market: First is the RCD105 light weight camera system, designed specifically for use with its ALS-series airborne LiDAR systems for corridor mapping. Second is the RCD100 camera system, which is considered to be a fully integrated 2 head camera solution for small mapping companies, looking for a turnkey digital mapping system. The camera controller records data from the two camera heads, allowing simultaneous acquisition of natural color and false-color infrared images The RCD100 system includes an inertial position and attitude system. The camera system is suited to fit into the PAV80 gyro-stabilized mount. The complete RCD100 system is operated and controlled by the Leica's flight management system. Benefits of this airborne-specific design include complete compliance with all applicable airborne environmental specifications, including temperature, shock and vibrations. (Dold & Flint, 2007, Petrie, 2009a). 256 Key Features x CH39 Camera Head (39 MP, KODAK CCD-Chip) x Single and double sensor head configuration possible x Lenses (35, 60 or 100 mm) x Max. shutter speed 1/4,000s x Min. Exposure interval 2.2 sec. x Metric design – fully calibrated system x User-replaceable shutter assembly x optimized lens for CIR (filter/compensating optic, software and calibration) x Weight ca. 7.0 kg, including lens (RCD 105); ca. 70 kg (RCD100)  UltraCamLp The medium format flagship of Microsoft is the UltraCamLp with 92 megapixels (11,704 x 7,920 pixels pan), making it the largest footprint medium format mapping camera system on the market and ideal for smaller aircraft and local projects that require a rapid response. With new electronics, the UltraCamLp has the same repetition rate as the smaller precessor UltraCamL. The concept behind the the UltraCamL(p) is similar to the large format UltraCams. The panchromatic image is composed out of several sensors. The only difference is the color and NIR-Sensors which are CCD-sensors with a common bayer pattern, developed by DALSA. Panchromatic image size is 11,704 x 7,920 pixels; color and NIR image size is 5,320 x 3,600 pixels. Due to the design of the camera, a FMC is integrated. Despite other medium format camera systems, the UltraCamLp has fixed lenses with a fixed aperture of 1/4 f. The in-flight storage capacity is limited only by number of solid-state storage devices on board, given space and weight constraints of aircraft (Microsoft, 2010). Key features x Simultaneously collects Pan, RGB and NIR x Approximately 2,500 uncompressed images per device (~1 TB) can be stored x Metric design – Image geometric accuracy is better than +/- 2 µm x Min. Exposure interval 2.5 sec. x Weight approx. 55 kg  &XUUHQW WUHQGV DQG RXWORRN IRU PHGLXP IRUPDW FDPHUD V\VWHPV  Rapid processing A movement towards rapid processing aiming at (near) online orthophoto generation, e.g. for disaster management, security applications etc. This is only possible with direct georeferencing and several 257 prerequisites, such as precise online GPS/INS data, a constant interior orientation and accurate boresight parameters, and a high speed and high quality data management. Compared to digital large format cameras, single-head medium format cameras have two important advantages for rapid processing: Due to the single head, fewer time-consuming preprocessing steps (stitching of the different camera cones, colour image fusion…) are necessary before orthorectification. The smaller images also allow for faster processing of single orthophotos. The disadvantages of this approach are radiometric differences between adjacent images and the necessity of a GPS/INS-system.  Multi head systems Multi medium format camera head systems such as the DiMAC, DigiCAM quattro, Trimble AIC4, provide a similar coverage of a standard digital large format camera, but for a much lower price. By their nature, multi-head cameras systems do not acquire vertical images but slightly oblique images. Also, every camera of a multi-head has its own interior orientation and the relative orientation of the different cameras may change slightly. In digital large format cameras, a laborious process is necessary to generate a so called virtual image from the different single heads. With multi-head medium format cameras a less complex strategy is to maintain an process the single images separately. The advantages and disadvantages are described in the following table. Single Images Virtual image  Full (color) image information Less images, better handling Central perspective Less computing time (for end user) Smaller images, faster orthophoto generation and web applications “Distortion free” and georeferenced image ĺ GIS Data integration  More images, longer processing time (for end user) Additional computation steps (Stitching, color corrections …) Direct georeferencing necessary Additional correction grids necessary Compatibility with „simple“ photogrammetric software )LJXUH  3URV DQG FRQV RI YLUWXDO LPDJHV RI PXOWL KHDG FDPHUD V\VWHPV From a geometric point of view, Jacobsen, 2009 states that, if single images of a multi head system are to be processed independently in an aerotriangulation process additional ground control points are necessary. Otherwise a sidelap and endlap of 70 % to 80 % is necessary to compensate for the weaker block geometry. Therefore an economic use of single images without direct georefecencing or the computation of a virtual image appears to be critical. From the authors point of view virtual images will become the standard product, because they can be generated fully automatically and the users ask for " ready to use" and distortion free images. 258  Forward-motion compensation (FMC) Forward-motion compensation (FMC) will come, not by time delayed integration (TDI) but mechanically. Clients ask for larger and larger ground resolution. To fulfill this wish without FMC, one may fly slower or apply a faster shutter speed. The lenses of medium format cameras generally allow for a minimum shutter speed of 1/1.000 s. As the pixel sizes of digital cameras become smaller (currently 6.0 µm), a theoretical smear of > 0.5 pixel due to forward motion at a speed of 50 m/s becomes critical for a GSD of < 10 cm. In order to get perfect images with a GSD of 3 – 5 cm, FMC has to be applied.  Oblique Images Oblique images have historically been used for visualisation and interpretation purposes, rather than for metric applications. Exceptions are the military sector and archaeology where oblique images have long been standard for reconnaissance purposes. Anyway, until recently oblique images were generally outside of the focus of photogrammetrists. They can thus be truly regarded as a new data source for photogrammetry and GIS. The focus of first generation of oblique camera systems, e.g. from pictometry used commercial small format cameras in order to generate "nice pictures" in the photogrammetric context. Recently the market demanded more and more geometrically accurate images, e.g. for automatic texture mapping of 3D-City models, Karbø and Schroth, 2009. Therefore digital medium format cameras tightly coupled with a GPS/INS for precise direct georeferencing are used in current systems. Oblique images are difficult to obtain with standard mapping cameras. To fully exploit the information from the oblique perspective, a minimum of four images from all sides have to be acquired and managed. Only multi head medium format camera systems provide the necessary flexibility. For the professional acquisition of oblique images several companies developed multiple camera head solutions with five cameras (Petrie and Walker, 2007, Petrie, 2009). This configuration is also called the "Maltese Cross" configuration. In such systems, one camera head provides a nadir view and the other four cameras provide the fixed oblique views in different directions, see Figure 13 for examples. Within direct georeferencing the boresight alignment of such a multi-head camera system with overlapping images requires special treatment (Kurz et al., 2007, Jacobsen, 2008, Wiedemann, 2009). The acquisition of oblique images requires several changes in the usual workflow for nadir (vertical) images, from survey planning to image processing and image analysis, Grenzdörffer et al., 2008. System MIDAS – Track‘ Air Maltese cross Image configuration Penta-DigiCAM )LJXUH  2EOLTXH PXOWL KHDG PHGLXP IRUPDW FDPHUD V\VWHPV 0DOWHVH &URVV 259 Digital medium format cameras are quite expensive, therefore alternatives were developed to reduce the number of cameras and still be able to acquire images from all directions, such as the Aero Oblique System (AOS) that has been recently built by Trimble (former RolleiMetric), Wiedemann, 2009. The AOS system comprises three Trimble AIC medium-format digital 39 MP camera units, )LJXUH . The shutters of the three cameras are synchronized to capture simultaneously the vertical image and the two oblique images pointing in opposite directions cross-track. The complete threecamera unit can then be rotated quickly by 90° to obtain the second pair of oblique images pointing in the along-track direction. Due to the exposure interval of 3.5 s the minimum GSD of the system is 10 cm (in the image center ) in the rotating mode. During flight operation the complete three-camera unit can be lowered down through the camera port in the aircraft floor to operate externally under the aircraft. During take-off and landing the camera unit is kept inside the aircraft. Rolleimetric AOSAzicam )LJXUH  2EOLTXH LPDJH V\VWHPV ZLWK D UHGXFHG QXPEHU RI LPDJH KHDGV Another approach is the Azicam, developed by the Bath Spa University, see )LJXUH . Instead of a normal five camera array, the Azicam is a single-medium format digital camera mounted on a rotating plate or ‘spinner' driven by a precise friction motor to orient the camera for each shot. Navigational and recording GPS ensures accurate geo-location. Using a single higher-resolution camera delivers a wider area of coverage, better image quality and costs less to calibrate and maintain. The whole system fits in an aircraft with a standard ground hole.  Increase in image size Compared to large format cameras the digital sensors of medium format cameras have undergone a strong and steady increase in resolution, see Figure 15. New technologies will increase the number of pixels even further. The footprint of a single head medium format camera is similar to 1st generation large format cameras. The standard pixel size of the latest sensors is 6 µm. 260 0 20 40 60 80 100 120 140 160 180 200 < 1999 2001 2003 2005 2007 2009 2011 Year Sensor(MegaPixel) DMC UltraCAM Medium format )LJXUH  'HYHORSPHQW RI IRRWSULQWV RI ODUJH IRUPDW FDPHUDV DQG PHGLXP IRUPDW FDPHUDV Currently there are three CCD-sensors commonly used. The Kodak 501003 (8176 x 6132 pix.) the DALSA FTF6080C (6000 x 8000 pix.)4 Color CCD and the Phase One P65+5 (8984 x 6732). Kodak, the supplier for the 51 Megapixel KAF-50100-CA Chip, which is currently in many medium format cameras introduced a new Colour Filter Array (CFA) layout. This technology increases the overall sensitivity of the sensor, as more of the photons striking the sensor are collected and used to generate the final image. This provides an increase in the photographic speed of the sensor, which can be used to improve performance when imaging under low light, enable faster shutter speeds (to reduce motion blur when imaging moving subjects), or the design of smaller pixels (leading to higher resolutions in a given optical format) while retaining performance. The pixel size of the next generation of sensors will be 5 µm, thus leading to sensors with 11.000 x 8.000 Pix (88 Mpix.) Nowadays digital medium format camera systems are mature airborne systems with high reliability. With the increasing demand of “near-online” digital aerial data these systems will become even more popular in the future.  $FNQRZOHGJPHQWV Special thanks to. Michel Cramer who helped to initiate the project, as well as the reviewers which gave valuable input to this report at the final phase. Further thanks belong to the manufacturing companies and all people who contributed to the project. 3 http://www.kodak.com/global/plugins/acrobat/en/business/ISS/datasheet/fullframe/KAF-50100Long Spec.pdf 4 http://www.dalsa.com/sensors/products/sensordetails.aspx?partNumber=FTF6080C 5 http://www.phaseone.com /Digital-Backs/P65/~/media/Phase%20One/Products/Documents/Phase- One-645DF-P65-p-datasheet-english.ashx 261  5HIHUHQFHV Applanix 2010http://www.applanix.com/solutions/airborne/dss.html, accessed 26.3.2010 Artés, F. 2004: Medium-format digital cameras come to age.- Earth Observation Magazine October 2004, pp. 8 -12. Cramer, M. Grenzdörffer, G., Honkavaara, E. 2010: In situ digital airborne camera validation and certification – the future standard ?.- ISPRS Commission I Meeting, Calgary, Canada, June 15 – 18, 2010, 7p. Cramer, M. 2009: Digital Airborne camera performance - The DGPF test.- In: Fritsch (ed.): Photogrammetric Week ‘09.- pp. 51 – 68 Cramer, M., 2007: The EuroSDR Performance Tests for Digital Aerial Camera Systems.- In: Fritsch (ed.): Photogrammetric Week ‘07.- pp. 89 – 106 Cramer, M., 2004: Performance of Medium format Digital Aerial Sensor Systems.- Proc. of the XX ISPRS Congress, Istanbul.- IntArchPhRS VOLUME XXXV-B1, pp. 769 - 774 Cronk, S., Fraser, C.S., Hanley, H., 2006: Automatic calibration of colour digital cameras. The Photogrammetric Record Vol. 21, No. 116: pp. 355 – 370. Dimac 2010: http://www.dimacsystems.com/html/technical.html, accessed 14.04.2010 DIN Deutsches Institut für Normung e. V. (Ed.) 2007: DIN 18740-4, September 2007. Photogrammetrische Produkte - Teil 4: Anforderungen an digitale Luftbildkameras und an digitale Luftbilder (in German), english version (http://www.ifp.uni-stuttgart.de/eurosdr/meeting-ign/DIN- English.pdf) Dold, J. and Flint, D. 2007: Leica Geosystems Photogrammetric Sensor and Workflow Developments.- In: Fritsch (ed.): Photogrammetric Week ‘07.- pp. 3 – 12 Dörstel, C., 2009: RMK D - A true metric medium format digital aerial camera system.- In: Fritsch (ed.): Photogrammetric Week ‘09.- pp. 91 – 98 EuroDAC² 2008: European Digital Airborne Camera Certification – EuroDAC² http://www.eurosdr.net/projects/eurodac/ eurodac2_positionpaper.pdf (accessed 02.04.2008) GIM International 2008: Digital Aerial Cameras: April 2008, Volume 22, Issue 4, pp. 17 – 19 Grenzdörffer, G., 2006: Praktische Erfahrungen mit dem digitalen Bildflugsystem PFIFF und einer Rollei AIC-45 CIR.- DGPF Jahrestagung 11.-13.9.2006, Berlin: pp. 335 – 342 (in German) Grenzdörffer G., Guretzki, M., Friedlander, I., 2008: Photogrammetric image acquisition and image analysis of oblique imagery.- Photogrammetric Record, Vol. 23, Issue 124 (Dec. 2008). 372 - 386 Habib, A., and M. Morgan, 2005: Stability Analysis and Geometric Calibration of Off-the-Shelf Digital Cameras, Photogrammetric Engineering & Remote Sensing Vol. 71, No. 6, June 2005, pp. 733–741. 262 Honkavaara, E., Arbiol, R., Markelin, L., Martinez, L., Cramer, M., Bovet, S., Chandelier, L., Ilves, R., Klonus, S., Marshall, P., Scläpfer, D., Tabor, M., Thom, C., & Veje, N., 2009: Digital airborne photogrammetry – A new tool for quantitative remote sensing? – A state-of-the-art review on radiometric aspects of digital photogrammetric images. Remote Sensing, Vol. 1, 577- 605. Honkavaara, E., Markelin, L., 2007: Radiometric Perforamce of Digital Image Data Collection – A comparison of ADS 40/DMC/UltraCAM and EmergeDSS.- In: Fritsch (ed.): Photogrammetric Week ‘07.- pp. 117 - 129 Honkavaara, E. Peltoniemi, J., Ahokas, E. Kuittinen, R. Hyyppä, J. Jaakkola, J. Kaartinen, H., Markelin, L. Nurminen, K. and Suomalainen J., 2007: A Permanent Test Field for Digital Photogrammetric Systems.- PE & RS, Vol. 74, No. 1, pp. 95 – 106 Honkavaara E., Jaakkola, J. Markelin, L. Nurminen, K. Ahokas, E., 2006: Theoretical and empirical evaluation of geometric performance of multi-head large format photogrammetric sensors.Proceedings of the Commission I Symposium “From sensors to imagery” IAPRS VOLUME XXXVI part 1, 6 p. IGI 2010: http://www.igi-systems.com/downloads/brochures/IGI_DigiCAM_specs.pdf, accessed 13.04.2010 Ip, A. W. L., Mostafa, M. M. R., Liberty, E., Hutton, J. 2007: an end-to-end airborne digital mapping solution, for rapid directly georeferenced orthophoto production.- Proceedings of the 2007 Annual Conference of the Remote Sensing & Photogrammetry Society (RSPSoc2007), 6 p. ISO/TC211 2008: http://www.isotc211.org/, accessed 26.3.2008 Jacobsen, K., 2009: DGPF-Projekt: Evaluierung digitaler photogrammetrischer Luftlbildkamerasysteme - Auswerteteam Geometrie. Annual Meeting of DGPF, Jena 2009, online available at www.ifp.uni-stuttgart.de/dgpf/PDF7Kameratest_Geolnetrie_V5.pdf (in German) Jacobsen. K., 2008: Geometry of vertical and oblique image combinations.- 28th EARSeL Symposium. Istanbul, 2008, 8 p. (=http://www.ipi.uni-hannover.de/uploads/tx_tkpublikationen /KJ_oblique.pdf) Jacobsen, K., 2007: Geometry and Information Contents of Large Size Digital Frame Cameras.ISPRS Hannover Workshop 2007, IntArchPhRS XXXVI. Band 1/W51. 7 p. Karbø, N. Schroth, R., 2009: Oblique Aerial Photography: A Status Review.- In: Fritsch, D. [Ed.]: Photogrammetric Week `09: pp. 119 - 125 Kröpfl, M., Kruck, E., Gruber, M., 2004: Geometric calibration of the digital large format camera UltraCamD.- International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 35 (Part B1), 42-44. Kurz, F., Müller, R., Stephani, M., Reinartz, P. and Schroeder, M., 2007: Calibration of a wide-angle digital camera system for near real time scenarios. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 36 (1/W51): 6 pages (on CD-ROM). http://www.commission1.isprs.org/hannover07/paper/Kurz_mueller_etal.pdf, accessed 15.04.2010. 263 Lohmann, P., 2008: Pictometry und Multivision – Objektinterpretation mit Luftbildschrägaufnahmen.Vortrag Geomatik-Forum, HCU Hamburg. (=http://www.hcu-hamburg.de/geomatik/forum 2008/vortraege/03_5hfg2008_Lohmann.pdf) Luhmann, Th. Hastedt, H. and Tecklenburg, W. 2006: Modelling of chromatic aberration for high precision photogrammetry.- Proceedings of the Commission V Symposium “Image Engineering and Vision Metrology”.- IAPRS VOLUME XXXVI, PART 5, 6 p. Madani, M., 2010: RMK D Geometric Calibration and its Accuracy Potentials.- EuroCOW 2010: The calibration and orientation workshop, February 10-12, 2010, Castelldefels, Spain, 14 p. Microsoft 2010: http://www.microsoft.com/ultracam/en-us/UltraCamLp.aspx, accessed 15.04.2010 Mostafa, M., 2004: Airborne Testing of the DSS: Test Results and Analysis.- Proceedings of the XX ISPRS Congress, Istanbul.- IAPRS VOLUME XXXV-B1, pp. 775 – 780Petrie, G. 2009: Systematic Oblique Aerial Photography using Multiple Digital Frame Cameras.- Photogrammetric Engineering & Remote Sensing.- p. 102 – 107 (=http://www.asprs.org /publications/pers/2009journal/february/highlight.pdf) Petrie, G. and Walker, A. S., 2007: Airborne digital imaging technology: A new Overview.- The Photogrammetric Record 22 (119), 95–105. Petrie, G., 2009a: Fully Integrated Imaging Solutions - Leica’s RCD Digital Frame Cameras.Geoinformatics, Sept. 2009, p 54 - 57. (=http://fluidbook.microdesign.nl/geoinformatics/06- 2009/#0) Petrie, G., 2009b: Modular Cameras; Multiple Configurations - The IGI DigiCAM Range.- Geoinformatics, Sept. 2009, p 60 - 63. (=http://fluidbook.microdesign.nl/geoinformatics/06-2009/#0) Petrie, G., 2009c: Systematic Oblique Aerial Photography Using Multiple Digital Frame Cameras. Photogrammetric Engineering & Remote Sensing, , S. 102-107. (=http://www.asprs.org/publications/pers/2009journal/february/highlight.pdf) Reulke, R. Dörstel C. and Schwebel, R., 2007: Anforderungen an digitale Luftbildkameras und an digitale Luftbilder, DGPF Jahrestagung, 2007 Schroth, R.W., 2007: The digital mapping camera DMC and its application potential.- ISPRS Hannover Workshop 2007, IntArchPhRS XXXVI. Band 1/W51. 7 p Shortis, M. R. Bellmana, C. J., Robson, S. Johnston, G. J. and Johnson, G. W., 2006: Stability of zoom and fixed lenses used with digital SLR cameras.- Proceedings of the Commission V Symposium Image Engineering and Vision Metrology.- IAPRS VOLUME XXXVI, PART 5, 6 p. Souchon, J.-P., Paparoditis, N., Martin, O., Meynard, C., Thom, C. 2006: Is there an ideal digital aerial camera?.- Proceedings of the Commission I Symposium “From sensors to imagery” IAPRS VOLUME XXXVI part 1, 6 p. Stensaas, G., 2007: U.S. Geological Survey digital aerial mapping camera certification and quality assurance plan for digital imagery.- In: Fritsch (ed.): Photogrammetric Week ‘07.- pp. 107 – 11 264 Trimble 2010: http://www.trimble.com/geospatial/Trimble-Aerial-Camera.aspx?dtID=overview, accessed 14.04.2010 Wiedemann, A., 2009: Photogrammetrische Schrägluftbilder mit dem Aerial Oblique System AOS.DGPF Tagungsband 18: 8 S. 265 ,QGH[ RI )LJXUHV ),*85(  ',))(5(17 &21&(376 )25 7+( *(1(5$7,21 &2/25  ,5 IMAGES................................................................................................................. 237 ),*85(  &203$5,621 2) 7+( &2676 2) 7+( 3+272*5$00(75,& :25.)/2: FOR LARGE FORMAT AND MEDIUM FORMAT CAMERA SYSTEMS ....... 239 ),*85(  &267 '(*5(66,21 3(5 $5($ )25 $(5,$/ 6859(<6 $1' 257+2 PHOTO GENERATION ........................................................................................ 239 ),*85(  (;32685( ,17(59$/ 2) 0(',80 )250$7 &$0(5$6 3  03,; 5(/$7(' 72 *6' $1' *5281' 63(('   ),*85(  ,0$*( 027,21 8 )25 $ *6' 2)  &0 $7 ',))(5(17 63(('6 29(5 GROUND AND DIFFERENT EXPOSURE TIMES ............................................ 243 ),*85(  6<67(0$7,& 5(6,'8$/6 2) ,0$*( (55256 75,0%/( $,& JACOBSEN, 2009 .................................................................................................. 245 ),*85(   63(&75$/ 5(63216( 48$1780 ()),&,(1&< 2) 7+( .$) 6(1625 &2/25 9(56,21   ),*85(  &21&(37 2) &2/285 $1' &,502'( 2) 0(',80 )250$7 CAMERAS ............................................................................................................. 248 ),*85(  5*% $1' &,5 :,7+ %$<(5 3$77(51 &&'   ),*85(  $ &+520$7,& $%(55$7,21   ),*85(  02,5e ())(&76 ,1 &,502'( 21 $*5,&8/785$/ ),(/'6   ),*85(  3526 $1' &216 2) 9,578$/ ,0$*(6 2) 08/7, +($' &$0(5$ SYSTEMS .............................................................................................................. 258 ),*85(  2%/,48( 08/7, +($' 0(',80 )250$7 &$0(5$ 6<67(06 0$/7(6( &5266   ),*85(  2%/,48( ,0$*( 6<67(06 :,7+ $ 5('8&(' 180%(5 2) ,0$*( HEADS ................................................................................................................... 260 ),*85(  '(9(/230(17 2) )22735,176 2) /$5*( )250$7 &$0(5$6 $1' MEDIUM FOR MAT CAMERAS ......................................................................... 261 ,QGH[ RI 7DEOHV 7$%/(  &203$5,621 2) &20321(176 )25 60$// $1' 0(',80 )250$7 CAMERA SYSTEMS ............................................................................................ 235 7$%/(  &203$5,621 2) 257+2 ,0$*( 352&(66,1* :25.)/2: 2) SMALL AND MEDIUM FORMAT CAMERA SYSMS ...................................... 235 7$%/(  &203$5,621 2) Ä1(: 0(',80³ )250$7 &$0(5$6 )520 /(,&$ 0,&5262)7 $1' =, ««««««««««««««««««««««  7$%/(  (;$03/(6 2) )22735,176 2) /$5*( )250$7 ,17(50(',$7( FORMAT AND MEDIUM FORMAT SYSTEMS ................................................ 237 7$%/(  0$,1 ',))(5(1&(6 %(7:((1 0(',80 $1' /$5*( )250$7 DIGITAL CAMERA SYSTEMS ........................................................................... 238 LIST OF OEEPE/EuroSDR OFFICIAL PUBLICATIONS State – Nov. 2010  Trombetti, C.: „Activité de la Commission A de l’OEEPE de 1960 à 1964“ – Cunietti, M.: „Activité de la Commission B de l’OEEPE pendant la période septembre 1960 – janvier 1964“ – Förstner, R.: „Rapport sur les travaux et les résultats de la Commission C de l’OEEPE (1960– 1964)“ – Neumaier, K.: „Rapport de la Commission E pour Lisbonne“ – Weele, A. J. v. d.: „Report of Commission F.“ – Frankfurt a. M. 1964, 50 pages with 7 tables and 9 annexes.  Neumaier, K.: „Essais d’interprétation de »Bedford« et de »Waterbury«. Rapport commun établi par les Centres de la Commission E de l’OEEPE ayant participé aux tests“ – „The Interpretation Tests of »Bedford« and »Waterbury«. Common Report Established by all Participating Centres of Commission E of OEEPE“ – „Essais de restitution »Bloc Suisse«. Rapport commun établi par les Centres de la Commission E de l’OEEPE ayant participé aux tests“ – „Test »Schweizer Block«. Joint Report of all Centres of Commission E of OEEPE.“ – Frankfurt a. M. 1966, 60 pages with 44 annexes.  Cunietti, M.: „Emploi des blocs de bandes pour la cartographie à grande échelle – Résultats des recherches expérimentales organisées par la Commission B de l’O.E.E.P.E. au cours de la période 1959–1966“ – „Use of Strips Connected to Blocks for Large Scale Mapping – Results of Experimental Research Organized by Commission B of the O.E.E.P.E. from 1959 through 1966.“ – Frankfurt a. M. 1968, 157 pages with 50 figures and 24 tables.  Förstner, R.: „Sur la précision de mesures photogrammétriques de coordonnées en terrain montagneux. Rapport sur les résultats de l’essai de Reichenbach de la Commission C de l’OEEPE“ – „The Accuracy of Photogrammetric Co-ordinate Measurements in Mountainous Terrain. Report on the Results of the Reichenbach Test Commission C of the OEEPE.“ – Frankfurt a. M. 1968, Part I: 145 pages with 9 figures; Part II: 23 pages with 65 tables.  Trombetti, C.: „Les recherches expérimentales exécutées sur de longues bandes par la Commission A de l’OEEPE.“ – Frankfurt a. M. 1972, 41 pages with 1 figure, 2 tables, 96 annexes and 19 plates.  Neumaier, K.: „Essai d’interprétation. Rapports des Centres de la Commission E de l’OEEPE.“ – Frankfurt a. M. 1972, 38 pages with 12 tables and 5 annexes.  Wiser, P.: „Etude expérimentale de l’aérotiangulation semi-analytique. Rapport sur l’essai »Gramastetten«.“ – Frankfurt a. M. 1972, 36 pages with 6 figures and 8 tables.  „Proceedings of the OEEPE Symposium on Experimental Research on Accuracy of Aerial Triangulation (Results of Oberschwaben Tests)“ Ackermann, F.: „On Statistical Investigation into the Accuracy of Aerial Triangulation. The Test Project Oberschwaben“ – „Recherches statistiques sur la précision de l’aérotriangulation. Le champ d’essai Oberschwaben“– Belzner, H.: „The Planning. Establishing and Flying of the Test Field Oberschwaben“ – Stark, E.: Testblock Oberschwaben, Programme I. Results of Strip Adjustments“ – Ackermann, F.: „Testblock Oberschwaben, Program I. Results of Block-Adjustment by Independent Models“ – Ebner, H.: Comparison of Different Methods of Block Adjustment“ – Wiser, P.: „Propositions pour le traitement des erreurs non-accidentelles“– Camps, F.: „Résultats obtenus dans le cadre du project Oberschwaben 2A“ – Cunietti, M.; Vanossi, A.: „Etude statistique expérimentale des erreurs d’enchaînement des photogrammes“– Kupfer, G.: „Image Geometry as Obtained from Rheidt Test Area Photography“ – Förstner, R.: „The Signal-Field of Baustetten. A Short Report“ – Visser, J.; Leberl, F.; Kure, J.: „OEEPE Oberschwaben Reseau Investigations“ – Bauer, H.: „Compensation of Systematic Errors by Analytical Block Adjustment with Common Image Deformation Parameters.“ – Frankfurt a. M. 1973, 350 pages with 119 figures, 68 tables and 1 annex.  Beck, W.: „The Production of Topographic Maps at 1 : 10,000 by Photogrammetric Methods. – With statistical evaluations, reproductions, style sheet and sample fragments by Landesvermessungsamt Baden-Württemberg Stuttgart.“ – Frankfurt a. M. 1976, 89 pages with 10 figures, 20 tables and 20 annexes.  „Résultats complémentaires de l’essai d’«Oberriet» of the Commission C de l’OEEPE – Further Results of the Photogrammetric Tests of «Oberriet» of the Commission C of the OEEPE“ Hárry, H.: „Mesure de points de terrain non signalisés dans le champ d’essai d’«Oberriet» – Measurements of Non-Signalized Points in the Test Field «Oberriet» (Abstract)“ – Stickler, A.; Waldhäusl, P.: „Restitution graphique des points et des lignes non signalisés et leur comparaison avec des résultats de mesures sur le terrain dans le champ d’essai d’«Oberriet» – Graphical Plotting of Non-Signalized Points and Lines, and Comparison with Terrestrial Surveys in the Test Field «Oberriet»“ – Förstner, R.: „Résultats complémentaires des transformations de coordonnées de l’essai d’«Oberriet» de la Commission C de l’OEEPE – Further Results from Co-ordinate Transformations of the Test «Oberriet» of Commission C of the OEEPE“ – Schürer, K.: „Comparaison des distances d’«Oberriet» – Comparison of Distances of «Oberriet» (Abstract).“ – Frankfurt a. M. 1975, 158 pages with 22 figures and 26 tables.  „25 années de l’OEEPE“ Verlaine, R.: „25 années d’activité de l’OEEPE“ – „25 Years of OEEPE (Summary)“ – Baarda, W.: „Mathematical Models.“ – Frankfurt a. M. 1979, 104 pages with 22 figures.  Spiess, E.: „Revision of 1 : 25,000 Topographic Maps by Photogrammetric Methods.“ – Frankfurt a. M. 1985, 228 pages with 102 figures and 30 tables.  Timmerman, J.; Roos, P. A.; Schürer, K.; Förstner, R.: On the Accuracy of Photogrammetric Measurements of Buildings – Report on the Results of the Test “Dordrecht”, Carried out by Commission C of the OEEPE. – Frankfurt a. M. 1982, 144 pages with 14 figures and 36 tables.  Thompson C. N.: Test of Digitising Methods. – Frankfurt a. M. 1984, 120 pages with 38 figures and 18 tables.  Jaakkola, M.; Brindöpke, W.; Kölbl, O.; Noukka, P.: Optimal Emulsions for Large-Scale Mapping – Test of “Steinwedel” – Commission C of the OEEPE 1981–84. – Frankfurt a. M. 1985, 102 pages with 53 figures.  Waldhäusl, P.: Results of the Vienna Test of OEEPE Commission C. – Kölbl, O.: Photogrammetric Versus Terrestrial Town Survey. – Frankfurt a. M. 1986, 57 pages with 16 figures, 10 tables and 7 annexes.  Commission E of the OEEPE: Influences of Reproduction Techniques on the Identification of Topographic Details on Orthophotomaps. – Frankfurt a. M. 1986, 138 pages with 51 figures, 25 tables and 6 appendices.  Förstner, W.: Final Report on the Joint Test on Gross Error Detection of OEEPE and ISP WG III/1. – Frankfurt a. M. 1986, 97 pages with 27 tables and 20 figures.  Dowman, I. J.; Ducher, G.: Spacelab Metric Camera Experiment – Test of Image Accuracy. – Frankfurt a. M. 1987, 112 pages with 13 figures, 25 tables and 7 appendices.  Eichhorn, G.: Summary of Replies to Questionnaire on Land Information Systems – Commission V – Land Information Systems. – Frankfurt a. M. 1988, 129 pages with 49 tables and 1 annex.  Kölbl, O.: Proceedings of the Workshop on Cadastral Renovation – Ecole polytechnique fédérale, Lausanne, 9–11 September, 1987. – Frankfurt a. M. 1988, 337 pages with figures, tables and appendices.  Rollin, J.; Dowman, I. J.: Map Compilation and Revision in Developing Areas – Test of Large Format Camera Imagery. – Frankfurt a. M. 1988, 35 pages with 3 figures, 9 tables and 3 appendices.  Drummond, J. (ed.): Automatic Digitizing – A Report Submitted by a Working Group of Commission D (Photogrammetry and Cartography). – Frankfurt a. M. 1990, 224 pages with 85 figures, 6 tables and 6 appendices.  Ahokas, E.; Jaakkola, J.; Sotkas, P.: Interpretability of SPOT data for General Mapping. – Frankfurt a. M. 1990, 120 pages with 11 figures, 7 tables and 10 appendices.  Ducher, G.: Test on Orthophoto and Stereo-Orthophoto Accuracy. – Frankfurt a. M. 1991, 227 pages with 16 figures and 44 tables.  Dowman, I. J. (ed.): Test of Triangulation of SPOT Data – Frankfurt a. M. 1991, 206 pages with 67 figures, 52 tables and 3 appendices.  Newby, P. R. T.; Thompson, C. N. (ed.): Proceedings of the ISPRS and OEEPE Joint Workshop on Updating Digital Data by Photogrammetric Methods. – Frankfurt a. M. 1992, 278 pages with 79 figures, 10 tables and 2 appendices.  Koen, L. A.; Kölbl, O. (ed.): Proceedings of the OEEPE-Workshop on Data Quality in Land Information Systems, Apeldoorn, Netherlands, 4–6 September 1991. – Frankfurt a. M. 1992, 243 pages with 62 figures, 14 tables and 2 appendices.  Burman, H.; Torlegård, K.: Empirical Results of GPS – Supported Block Triangulation. – Frankfurt a. M. 1994, 86 pages with 5 figures, 3 tables and 8 appendices.  Gray, S. (ed.): Updating of Complex Topographic Databases. – Frankfurt a. M. 1995, 133 pages with 2 figures and 12 appendices.  Jaakkola, J.; Sarjakoski, T.: Experimental Test on Digital Aerial Triangulation. – Frankfurt a. M. 1996, 155 pages with 24 figures, 7 tables and 2 appendices.  Dowman, I. J.: The OEEPE GEOSAR Test of Geocoding ERS-1 SAR Data. – Frankfurt a. M. 1996, 126 pages with 5 figures, 2 tables and 2 appendices.  Kölbl, O.: Proceedings of the OEEPE-Workshop on Application of Digital Photogrammetric Workstations. – Frankfurt a. M. 1996, 453 pages with numerous figures and tables.  Blau, E.; Boochs, F.; Schulz, B.-S.: Digital Landscape Model for Europe (DLME). – Frankfurt a. M. 1997, 72 pages with 21 figures, 9 tables, 4 diagrams and 15 appendices.  Fuchs, C.; Gülch, E.; Förstner, W.: OEEPE Survey on 3D-City Models. Heipke, C.; Eder, K.: Performance of Tie-Point Extraction in Automatic Aerial Triangulation. – Frankfurt a. M. 1998, 185 pages with 42 figures, 27 tables and 15 appendices.  Kirby, R. P.: Revision Measurement of Large Scale Topographic Data. Höhle, J.: Automatic Orientation of Aerial Images on Database Information. Dequal, S.; Koen, L. A.; Rinaudo, F.: Comparison of National Guidelines for Technical and Cadastral Mapping in Europe (“Ferrara Test”) – Frankfurt a. M. 1999, 273 pages with 26 figures, 42 tables, 7 special contributions and 9 appendices.  Koelbl, O. (ed.): Proceedings of the OEEPE – Workshop on Automation in Digital Photogrammetric Production. – Frankfurt a. M. 1999, 475 pages with numerous figures and tables.  Gower, R.: Workshop on National Mapping Agencies and the Internet. Flotron, A.; Koelbl, O.: Precision Terrain Model for Civil Engineering. – Frankfurt a. M. 2000, 140 pages with numerous figures, tables and a CD.  Ruas, A.: Automatic Generalisation Project: Learning Process from Interactive Generalisation. – Frankfurt a. M. 2001, 98 pages with 43 figures, 46 tables and 1 appendix.  Torlegård, K.; Jonas, N.: OEEPE workshop on Airborne Laserscanning and Interferometric SAR for Detailed Digital Elevation Models. – Frankfurt a. M. 2001, CD: 299 pages with 132 figures, 26 tables, 5 presentations and 2 videos.  Radwan, M.; Onchaga, R.; Morales, J.: A Structural Approach to the Management and Optimization of Geoinformation Processes. – Frankfurt a. M. 2001, 174 pages with 74 figures, 63 tables and 1 CD.  Heipke, C.; Sester, M.; Willrich, F. (eds.): Joint OEEPE/ISPRS Workshop – From 2D to 3D – Establishment and maintenance of national core geospatial databases. Woodsford, P. (ed.): OEEPE Commission 5 Workshop: Use of XML/GML. – Frankfurt a. M. 2002, CD.  Heipke, C.; Jacobsen, K.; Wegmann, H.: Integrated Sensor Orientation – Test Report and Workshop Proceedings. – Frankfurt a. M. 2002, 302 pages with 215 figures, 139 tables and 2 appendices.  Holland, D.; Guilford, B.; Murray, K.: Topographic Mapping from High Resolution Space Sensors. – Frankfurt a. M. 2002, 155 pages with numerous figures, tables and 7 appendices.  Murray, K. (ed.): OEEPE Workshop on Next Generation Spatial Database – 2005. Altan, M. O.; Tastan, H. (eds.): OEEPE/ISPRS Joint Workshop on Spatial Data Quality Management. 2003, CD.  Heipke, C.; Kuittinen, R.; Nagel, G. (eds.): From OEEPE to EuroSDR: 50 years of European Spatial Data Research and beyond – Seminar of Honour. 2003, 103 pages and CD.  Woodsford, P.; Kraak, M.; Murray, K.; Chapman, D. (eds.): Visualisation and Rendering – Proceedings EuroSDR Commission 5 Workshop. 2003, CD.  Woodsford, P. (ed.): Ontologies & Schema Translation – 2004. Bray, C. (ed.): Positional Accuracy Improvement – 2004. Woodsford, P. (ed.): E-delivery – 2005. Workshops. 2005, CD.  Bray, C.; Rönsdorf, C. (eds.): Achieving Geometric Interoperability of Spatial Data, Workshop – 2005. Kolbe, T. H.; Gröger, G. (eds.): International Workshop on Next Generation 3D City Models – 2005. Woodsford, P. (ed.): Workshop on Feature/Object Data Models. 2006, CD.  Kaartinen, H.; Hyyppä J.: Evaluation of Building Extraction. Steinnocher, K.; Kressler, F.: Change Detection. Bellmann, A.; Hellwich, O.: Sensor and Data Fusion Contest: Information for Mapping from Airborne SAR and Optical Imagery (Phase I). Mayer, H.; Baltsavias, E.; Bacher, U.: Automated Extraction, Refinement, and Update of Road Databases from Imagery and Other Data. 2006, 280 pages.  Höhle, J.; Potuckova J.: The EuroSDR Test “Checking and Improving of Digital Terrain Models”. Skaloud, J.: Reliability of Direct Georeferencing, Phase 1: An Overview of the Current Approaches and Possibilities. Legat, K.; Skaloud, J.; Schmidt, R.: Reliability of Direct Georeferencing, Phase 2: A Case Study on Practical Problems and Solutions. 2006, 184 pages.  Murray, K. (ed.): Proceedings of the International Workshop on Land and Marine Information Integration. 2007, CD.  Kaartinen, H.; Hyyppä, J.: Tree Extraction. 2008, 56 pages.  Patrucco, R.; Murray, K. (eds.): Production Partnership Management Workshop – 2007. Ismael Colomina, I.; Hernández, E. (eds.): International Calibration and Orientation Workshop, EuroCOW 2008. Heipke, C.; Sester, M. (eds.): Geosensor Networks Workshop. Kolbe, T. H. (ed.): Final Report on the EuroSDR CityGML Project. 2008, CD.  Cramer, M.: Digital Camera Calibration. 2009, 257 pages.  Champion, N.: Detection of Unregistered Buildings for Updating 2D Databases. Everaerts, J.: NEWPLATFORMS – Unconventional Platforms (Unmanned Aircraft Systems) for Remote Sensing. 2009, 98 pages.  Streilein, A.; Kellenberger, T. (eds.): Crowd Sourcing for Updating National Databases. Colomina, I.; Jan Skaloud, J.; Cramer, M. (eds.): International Calibration and Orientation Workshop EuroCOW 2010. Nebiker, S.; Bleisch, S.; Gülch, E.: Final Report on EuroSDR Project Virtual Globes. 2010, CD. The publications can be ordered using the electronic order form of the EuroSDR website www.eurosdr.net