Under the influence of novel corona virus pneumonia epidemic, the protection of traditional villages is put forward higher request. The spread of the epidemic among villages will make the situation of epidemic prevention and control more difficult. As an important part of culture, traditional villages have high historical value. In this paper, the traditional village protection method, a new geographical data algorithm IData storage method. Compared with the traditional ArcGIS method, it improves the efficiency and accuracy of topographic map entry. IData’s data factory can use the symbolic technology of skeleton lines to represent all the figures in the national standard mode, and any complex figure can only be represented by one element. Idate can quickly load data and render symbols in a drawing. With the powerful data processing engine of IData data factory, we can check out the errors that other software can’t find and process the data automatically. Records of the loss of traditional villages can be recorded quickly. The establishment and protection of traditional villages have had a beneficial impact.
Under the influence of novel corona virus pneumonia epidemic, the protection of traditional villages is put forward higher request. The spread of the epidemic among villages will make the situation of epidemic prevention and control more difficult. Therefore, the protection of traditional villages must be taken seriously. As a country with more than 7000 years of agricultural civilization, China is full of traditional villages reflecting different historical periods, different villages and different ethnic cultures . These villages carry a large number of excellent material and cultural heritage and intangible cultural heritage. In recent years, the number of traditional villages has declined sharply, from 3.63 million in 2000 to 2.71 million in 2011, with an average of 80–100 traditional villages disappearing every day. In April 2012, the Ministry of housing and urban rural development, the Ministry of culture and the Ministry of Finance jointly launched the investigation of traditional villages in China [2, 3]. In September, an expert committee composed of experts from various disciplines was established to review the directory of traditional villages in China, establish the basic information files of traditional villages in China, and prepare relevant plans for the protection and development of traditional villages . As of the announcement of the fourth batch of villages listed in the list of traditional villages, the number of traditional villages in China has reached 4157 [5–7]. In 2017, China has launched the fifth batch of traditional village surveys, with a total number expected to exceed 5000 . In the face of social development and historical changes, traditional villages are facing a severe situation of rapid disappearance. The main reasons are as follows :
(1) The acceleration of urbanization. In recent years, China has entered the rapid urbanization development, urban-rural dual structure is obvious. In the process of continuous expansion of the city, many rural areas have been incorporated into the scope of urban construction. Under the construction mode of “one thousand cities one side”, many traditional villages with historical and cultural characteristics have been pushed down and built into cities, resulting in serious loss of traditional villages.
(2) Villages need to be developed and declined. Traditional villages are generally not obvious in location advantages, most of them are located in areas with underdeveloped traffic conditions and relatively backward economy, such as mountain areas, basins and rivers with special terrain. In the context of urban-rural dual differentiation, a large number of farmers enter the city to work, resulting in the loss of traditional village manpower, leaving only the elderly and children behind. Many traditional villages have become “empty nest villages” or even “abandoned nest villages”, resulting in the decline and destruction of historical buildings with historical and cultural value due to the lack of repair and living, poor village style and resources, which cannot be included in the list of traditional villages. Such a situation results in the waste and loss of historical and cultural resources .
(3) The awareness of protection is not strong. Traditional village grass-roots workers lack of professional knowledge reserve, lack of relevant knowledge, cannot find a good position in the development opportunity of the village, cannot make a correct evaluation and guidance of the village, and the village is mostly left behind elderly, the overall cultural level of the young migrant workers is poor, lack of awareness of protection, not to mention awareness of the culture with local characteristics of the village. Such a situation leads to disorderly development, disorderly construction and destruction of the traditional pattern and style of the village .
(4) Lack of protection and development mode. In 2012, China just started the protection investigation of traditional villages. Due to the lack of experience, the protection and development of many villages are carried out in the exploration and exploration [12, 13]. In addition, there is no systematic theoretical support and the protection and development of villages cannot be correctly guided, which leads to greater difficulties in the protection and development of villages .
Facing the protection of traditional villages, we should solve the problem of village topographic map. There are some differences between CAD and GIS in describing the data model of the same spatial object. In the process of transformation, there are a series of problems. Although many GIS software can accept the data format of CAD, there are many problems in the process of transformation, such as great data loss, data deformation and coordinate mismatch. At present, a large part of geographic information data in China is stored in AutoCAD format. The data standards of professional mapping platforms such as AutoCAD/CASS, which we often use, are not completely consistent with those of GIS. Such data generally have the disadvantages of loose geometry, redundant data, topology errors, insufficient attribute information, etc. When the data is imported into GIS software or spatial database, it often needs to transform the data format, which cannot achieve the integration of map and database .
The novel coronavirus pneumonia can be identified as direct transmission, aerosol transmission and contact transmission, according to CCTV news, Shanghai’s epidemic prevention and control conference.
Direct transmission refers to the infection caused by the sneezing, coughing and speaking droplets of patients, and the exhaled gas is inhaled directly in a short distance; aerosol transmission refers to the mixture of droplets in the air, forming an aerosol, which causes infection after inhalation; contact transmission refers to the deposition of droplets on the surface of objects, after touching the contaminated hands, and then touching the mucous membranes of mouth, nose, eyes, etc., leading to infection.
The transformation process is easy to cause the loss of data and precision, and it is difficult to achieve 100% seamless connection. Therefore, it is of great significance to solve the problems of data organization, topological relationship, attribute symbols and coordinate system in the data conversion from CAD to GIS. Using the function provided by ArcGIS platform to transform CAD to GIS data and update spatial database is a mature and main method in China. Some valuable ideas and schemes are designed, and experience is gained in practical engineering application, which provides some important and effective technical routes for data warehousing. The data processing engine of IData has changed the operation mode of data collection, compilation and storage separation which is widely used in the field of Surveying and mapping at present . It takes the spatial database MDB as the data storage format, matches the perfect symbolic effect, realizes the operation mode of integration of drawing library and drawing genus, and provides users with a visual scheme editing interface and rich meta rules. The engine can automatically process spatial data through flow data processing scheme.
2Data factory of IData establishes technical process
Before data processing, it is necessary to collect the existing AutoCAD topographic map, through field inspection, modify the data in which the feature is missing, the feature location and the feature expression are not correct. Open the revised data in CASS software, and check the data with the function of “check storage” and the function of “graphic entity check” to ensure that the original data layer and coding are correct. At the same time, it is necessary to check the building marks, elevation notes, compound line repetition points and other issues.
2.2Data building process
The terrain data warehousing process based on IData data factory is shown in Fig. 1.
2.3Data building process
The data factory of IData uses the national standard database as the data template. When the requirements of the achievement database are different from the national standards, the database template needs to be customized according to the needs. Template customization includes symbolic template customization and database template customization. Symbolic template customization includes adding or modifying symbols, linetypes, colors, legend libraries, etc. Database template customization includes data coordinate system, scale, classification and coding, hierarchy, attribute data structure and other related contents.
IData data factory provides a data processing engine (SME function module), which provides users with a visual scheme editing interface. Users can automatically process and quality check spatial data by writing process data processing schemes. Select the rule editor menu in the main interface of IData data factory to open the data processing engine, and write the required data processing and quality inspection schemes according to the job steps. In the process of terrain data warehousing, the required schemes are data format conversion scheme, data sorting scheme and data quality inspection scheme.
(1) Data format conversion scheme
According to the correlation between topographic map element coding and result database coding, a conversion comparison table is compiled. In the data processing engine, import the conversion cross reference table and make the data format conversion scheme. In addition to reading and writing the database with the suffix of mdb format directly, IData data factory also supports reading and writing the data with the suffix of DWG / DXF, DGN and GDB format directly, loading the original data and achievement database at the same time, and running the data format conversion scheme. The original data is converted into the results database.
(2) Data arrangement scheme
After the data format conversion, it is necessary to write a data arrangement scheme to realize the processing of some special terrain elements, such as the automatic configuration of surface features, the object-oriented processing of pipelines, the object-oriented processing of place names, the processing of roads and rivers, and the automatic assignment of the attributes of the elements.
(3) Data quality inspection scheme
According to the quality standard of the database, prepare the data quality inspection plan. The database is automatically checked for quality. For example, check the correctness of entity structure, surface relationship, logical relationship, attribute information, etc. among them, surface relationship mainly refers to the relationship between topological relationship graphs, such as the overlap of graphs, the gap between adjacent graphs, the self-intersection of straight lines, the repetition of graphs, unclosed surface graphs, dotted lines, virtual surfaces, the staggered connection of graphs, etc. The logical relationship is mainly the contradiction between elevation point and contour line. Errors that can be handled automatically can be fixed automatically. Problems that cannot be handled automatically can be displayed in the error list. Through the error list, the corresponding error can be quickly located on the drawing surface, which is convenient for human-computer interaction.
Open the rule executor in the data factory of IData, import the compiled rule scheme (file with the suffix of TSK format), and convert the original topographic map in AutoCAD format to the database in ArcGIS format after running. The figure in Fig. 2 is the database after data arrangement, and the right side is the prepared scheme. The figure in Fig. 3 is the data of quality inspection, and the right is the error list of quality inspection.
After the above scheme is implemented in the data processing engine, the data warehousing and checking are completed. The operator is required to manually repair the errors detected. The repaired data needs to be checked twice to ensure that the data meet the results database standard before submitting the final results.
3Data storage method based on IData
The object of this vector data storage test is a 1 : 1000 standard CAD drawing with a total area of 0.25 km2. CAD graphics are rich in surface information and pay attention to surface expression. Before they are put into storage, they need to go through vector data processing such as surface inspection and processing, code inspection and processing, geometric structure inspection and processing, extended attribute inspection and processing. There are many ways to update vector data of topographic map. In this experiment, the data storage method based on ArcGIS desktop editing environment and the data storage method based on IData technology are compared and analyzed.
3.1Data storage method based on ArcGIS
ArcGIS stores spatial data in ArcMap editing
It is a technology based on graphics editing and processing. In terms of technical process, the editing and modification of original digital graphics in cartography software accounts for 65% of the total GIS project. The advantage of this method is to avoid redundant data, empty coding data and insufficient attribute information in vector data storage. On the basis of the processing of graphics editing, powerful graphics editing function is convenient for massive data to be modified and edited. The production efficiency of data storage has been greatly improved; the disadvantage is that after the vector data is put into storage, the secondary topology check needs to be carried out on the GIS platform. According to the storage requirements of the secondary development, the relevant topology rules applet is written. In fact, it increases the repetition rate of work. In the process of vector data storage, there will be the loss of element and attribute information, as well as the data inconsistency caused by the separation of the library. The flow of ArcMap data warehouse based on ArcGIS is shown in Fig. 4.
3.2Data storage method based on IData
As a new generation of information surveying and mapping data production platform, iData is oriented to GIS and geospatial database, and takes PersonalGeodatabase, the original spatial database of ArcGIS, as the data storage format. It is a kind of warehousing technology based on GIS database. According to the definition of the figure symbols in the national basic scale map schema, it customizes the symbolic template, and according to the classification and code of the national basic geographic information elements, it defines the classification code, stratification, scale, attribute structure and other contents of the database.
The implementation of IData custom data transfer scheme and one key CAD data to MDB database can effectively avoid the loss of elements and attributes in the process of data conversion; the integrated operation mode of graph base can avoid the problem of data repetitive production, reduce a lot of programming work and increase work efficiency. The data warehousing process based on IData is shown in Fig. 5.
Under the condition of the same amount of data and the same area vector data, the establishment of the initial database of IData can be realized in two steps. First, delete or add the attribute processing for the empty item of the value of root in the CAD data, and then convert the DWG to MDB by one key conversion; the quality inspection of the database is realized by the IData rule editor, and the Meta rule arrangement and combination constitute the data processing scheme. Before the establishment of ArcGIS initial database, it is necessary to process and layer the CAD data, and then convert CAD to MDB through the conversion command in arctoolbox. However, the integrity of the original data and attribute structure cannot be guaranteed; the data quality inspection of ArcGIS is realized through the topology tool set in arctoolbox, which requires a certain high-level programming language, high requirements for operators, long development time and heavy workload. It can be seen from the comparison between the two warehousing methods in technical process and Table 1 that IData is superior in both technical means and work efficiency
|Warehousing method||Warehousing area||Storage area||Initial library time||Time spent in quality inspection|
3.3Comparison of data processing methods
First, based on the ArcGIS data processing method in ArcGIS Desktop, a framework of geographic processing tasks is provided. These geographic processing tasks are included in the arctoolbox. The topology check of ArcGIS database is mainly carried out by the basic tools, command line and script of arctoolbox. Arctoolbox is a very convenient and practical tool, providing more than 400 tools, covering common GIS operations such as data stacking, buffering and data management, as well as custom topology advanced operations. The data topology operation flow based on ArcGIS is shown in Fig. 6.
Although arctoolbox provides rich topology functions, it cannot be customized or batch processed, which is relatively limited. Therefore, it is necessary to use the topology toolset in arctoolbox to create a GP (geopro processing) tool for topology analysis in ModelBuilder, and use ArcObjects to run the GP tool in VBA to output the inspection results to the inspection records we need complex GP tools for visual modeling.
Next, based on the data processing method of IData, IData uses Rule Edit, an open platform, as the data processing and quality inspection engine. Users can edit the meta rules according to their work needs, according to the different functions of the meta rules. The data processing engine of IData classifies meta rules into nine categories: dataset, logical operation, data selection, data inspection, data conversion, data operation, attribute operation, data output, and each category includes several sub categories. The graphical editing interface provided by IData data processing engine makes the scheme more intuitive and convenient. Users can drag and drop the required meta rules on the logical editing panel to any location, and drag and drop the connection lines to connect the meta rules, finally forming a complete user-defined data processing scheme.
Data processing and data quality inspection operation items made by IData data processing engine are arranged and combined by meta rules. The so-called meta rule refers to the minimum operation item of a single function, that is, a meta rule can only perform a specific operation, which is generally very simple. For example, the “surface data” meta rule is used to select the data on the surface that meets the filtering conditions. The “delete data” is only used to delete the incoming data set, and a complex data processing scheme is formed by combining these simple meta rules. During execution, data flows through these meta rules in turn, and finally the required data quality inspection and processing effects are achieved.
ArcGIS uses ArcObjects to develop GP tools, which can make full use of the advantages of high-level programming language. However, this development mode has high requirements for developers, and the development workload is large, and the development cycle is long, which is not suitable for the development of quick and efficient quality inspection tool set.
The graphics editing interface provided by IData data processing engine makes the program more intuitive and convenient. Users can drag and drop the required meta rule on the logical editing panel, and connect the meta rule by dragging the line with the mouse to form a complete user-defined data processing scheme. This kind of scheme compiling mode which accords with the positive thinking is more logical and makes the complex scheme simpler. At the same time, the editing mode of network type allows a meta rule on the parent node to communicate with multiple child nodes at the same time. This method saves editing space, increases work efficiency, and simplifies the writing process of complex schemes.
The meta rules provided by IData for users cover many aspects of data quality inspection and processing. By combining these rules, users can retrieve data quickly, check data quality strictly, process data in batches, and transform data format directly. The data processing engine of IData has both reliability and flexibility in data processing, especially in dealing with some complex processes, such as graph edge joining and map reduction.
4Data warehousing comparison and analysis
4.1Vector data storage method based on ArcGIS platform
The methods of transforming DWG data into ArcGIS database usually include:
8(1) loading DWG graphics directly in ArcMap, outputting shape data with exportdata command, and importing shape data into MDB database. This method is simple and fast, but it requires high data of DWG graphics, and cannot guarantee the integrity of data.
(2) Through the third-party software, such as MAPGIS, ArcView, FME, etc., CAD data can be converted into ArcGIS data format for easy identification and operation, and then loaded in ArcGIS for processing. The process of this method is relatively complex. Because of the third-party software involved, the data conversion process is not easy to control, which may also lead to data loss.
(3) You can use the quickexport or quickimport command under the datainteroperability module of ArcGIS’s arctoolbox tool. This method can process data in batch, has strong ability to identify CAD data, and has various output formats, such as ShapeFile, GDB, etc. the disadvantage is that there are too many layers, so ArcGIS is needed to fuse the data.
As shown in the vector data storage diagram of ArcGIS platform in Fig. 7, for vector data storage in ArcGIS platform, it is necessary to first convert CAD data into SHP format data, then establish GDB, then import SHP data, create topology rules, check and modify topology, and there is a large amount of data conversion work.
When CAD data is transferred to polygon of GIS, it is required to be closed segment line in CAD, otherwise it can only be converted to polyline, resulting in a large amount of data redundancy, so delete useless fields such as linearity, line color, line width, etc. with deletefield function; When the CAD data is converted to GDB format, the data attribute information of the original CAD cannot be retained. Only the field of GIS is provided, so it is not recommended to convert to GDB format when elevation information is needed.
4.2IData warehousing operation
For data warehousing of IData, first create an MDB template and load the DWG data into the MDB. In the rule editor, combine the existing meta rules into transformation schemes and quality inspection schemes that meet the data requirements through different permutations of functions. Load the irregular ones and check and process the large schemes and subdivisions through data transformation.
IData adopts multi-source data integration technology, which can integrate DWG data and MDB data into one platform for overlay display, operation and analysis. As shown in Fig. 8, this integrated operation mode of graph base effectively avoids data repetitive production and ensures the integrity and consistency of elements and attributes; all elements of DWG are converted to MDB without any omission. Elements without attribute items or corresponding codes are transferred to ordinary layers of IData in a unified way. After analyzing and improving the comparison table, they can be converted to corresponding layers and codes one by one, avoiding the loss of element and attribute information in the process of data conversion.
Both the interface and function of IData platform follow the CAD mode, which makes the operation of operators easy to start, overcomes the shortcomings of the existing data graphic editing function of GIS end storage technology, and is suitable for large-scale vector data storage; the skeleton line symbolization technology, while accurately expressing the symbols of ground objects, avoids the generation of redundant data; Cross platform symbolization technology makes cross platform plotting consistent.
There is no need for data conversion of intermediate software, from DWG graphics to MDB storage is always in one platform and one software, without conversion, directly storage.
The key of CAD data conversion and storage is to ensure the integrity of data conversion. For the inconsistency of different layer division standards, different attribute structures, geometric representation and spatial structure, how to effectively solve the connection of multi-source heterogeneous data, reduce the workload of operators, improve work efficiency, and ensure the accuracy, integrity and consistency of data. As a common warehousing platform, ArcGIS has a mature technical process, but inevitably has its own weaknesses. As a new generation of information mapping data production platform, IData adopts the integrated operation mode of map database, which can integrate multiple formats of data into one platform for overlay display, operation and analysis, and uses ArcGIS native spatial database personal Geodatabase as data storage The core technology of storage format and rule editor provides rich meta rules, which can combine a variety of data batch quality inspection and batch processing schemes to realize intelligent data sorting and quality inspection, so it is more efficient and practical.
5Equipment and results
In the traditional village protection and construction project, ArcGIS and software are used to store the topographic map data in AutoCAD format in the early stage of 50 square kilometers, and then IData data factory is used to store the remaining 50 square kilometers. The completion is shown in Table 2. Through comparison, it is found that data sorting and warehousing by using IData data factory can significantly improve the work efficiency and reduce the labor workload. The improvement of work efficiency is mainly reflected in the following two links:
(1) The data factory of IData can transform the format quickly, and modify and deal with the graphic errors. The process of building database of ArcGIS software tends to be vertical, and the next step can only be processed after one step is processed. The data factory database building process of IData tends to be horizontal, and the work of data preprocessing and scheme writing can be carried out at the same time. All topographic map files can be used to improve the efficiency of operation.
(2) IData data factory can load graphics quickly. ArcGIS software load graphics slowly, cannot see the linear symbols directly, and is not convenient to find and edit features. IData data factory can express all the figures in the national standard schema by using the skeleton line symbolization technology, and any complex figure can only be expressed by one element. You can quickly load data and render symbols in a drawing.
|Database building mode||The measure of area||Degree of difficulty||Integrative workload||Number of employees||Working days||Work efficiency|
|Increase workload: 42.8%|
This paper introduces the technical route of data factory database construction of IData. Using the powerful data processing engine of data factory of IData, we can check out the errors that other software can’t find, and process the data automatically. During the period of COVID-19, traditional villages can continue to be protected. For items that cannot be processed automatically, you can write quality inspection rules, detect error locations, locate error maps, and make them easy to correct. During the period of COVID-19, using data factory of IData to store terrain data, we can provide more scientific and reasonable services for the protection of traditional villages and improve the efficiency of database construction.
This paper is supported by the Humanities and Social Sciences project of The Education Department of Henan Province titled “Feasibility study on the function replacement of traditional villages under the strategy of rural revitalization” (2019-ZZJH-564).
Baoxing Q. , Discussion on role of government in the protection of traditional villages, China Ancient City 303(3) (2015), 2553–2557.
Liu Liao , Wu , et al., Cultural sustainability and vitality of Chinese vernacular architecture: a pedigree for the spatial art of traditional villages in Jiangnan region, Sustainability 11(24) (2019), 68–98.
Xiao Zhao , Sun , et al., Sustainability dynamics of traditional villages: a case study in Qiannan Prefecture, Guizhou, China, Sustainability 12(1) (2019), 3–14.
Wang H.F. and Chiou S.C. , Study on the sustainable development of human settlement space environment in traditional villages, Sustainability 11(15) (2019), 41–46.
Ma Y. , Luo B. , Zhu Q. , et al., Changes in traditional ecological knowledge of forage plants in immigrant villages of Ningxia, China, Journal of Ethnobiology and Ethnomedicine 15(1) (2019), 5–13.
Ma K. , Tang X. , Ren Y. , et al., Research on the spatial pattern characteristics of the Taihu Lake “Dock Village” based on microclimate: a case study of Tangli Village, Sustainability 11(2) (2019), 11–17.
Leal M.M. , Costa F.B. and Tiago L.S.C.J. , Improved traditional directional protection by using the stationary wavelet transform, International Journal of Electrical Power and Energy Systems 105(FEB.) (2019), 59–69.
Kidd D.M. and Liu X. , Geophylobuilder 1.0: an ArcGIS extension for creating ‘geophylogenies’, Molecular Ecology Resources 8(1) (2008), 88–91.
Marinoni O. , Implementation of the analytical hierarchy process with VBA in ArcGIS, Computers and Geosciences 30(6) (2004), 637–646.
Devasto M.A. , Czeck D.M. and Bhattacharyya P. , Using image analysis and ArcGIS? To improve automatic grain boundary detection and quantify geological images, Computers and Geosciences 49 (2012), 38–45.
Smirnoff A. , Serge Paradis J. , et al., Generalizing surficial geological maps for scale change: arcgis tools vs. cellular automata model, Computers and Geosciences 34(11) (2008), 1550–1568.
Wang L. , Ye M. , Rios J.F. , et al., Estimation of nitrate load from septic systems to surface water bodies using an ArcGIS-based software, Environmental Earth Ences 70(4) (2013), 1911–1926.
Jiménez-Perálvarez J.D. , Irigaray C. , Hamdouni R.E. , et al., Building models for automatic landslide-susceptibility analysis, mapping and validation in ArcGIS, Natural Hazards 50(3) (2009), 571–590.
Rinner C. and Voss S. , MCDA4 ArcMap – An open-source multi-criteria decision analysis and geovisualization tool for ArcGIS 10, International Journal of Geographical Information Science 15 (2013), 101–127.
Roberts J.J. , Best B.D. , Dunn D.C. , et al., Marine geospatial ecology tools: an integrated framework for ecological geoprocessing with ArcGIS, Python, R, MATLAB, and C++, Environmental Modelling and Software 25(10) (2010), 1197–1207.
Eldrandaly K.A. , A COM-based expert system for selecting the suitable map projection in ArcGIS, Expert Systems with Applications 31(1) (2006), 94–100.