You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

A New Multi-Attribute Decision-Making Framework for Policy-Makers by Using Interval-Valued Triangular Fuzzy Numbers


Policy-makers are often hesitant to invest in unproven solutions because of a lack of the decision-making framework for managing innovations as a portfolio of investments that balances risk and return, especially in the field of developing new technologies. This study provides a new portfolio matrix for decision making of policy-makers to identify IoT applications in the agriculture sector for future investment based on two dimensions of sustainable development as a return and IoT challenge as a risk using a novel MADM approach. To this end, the identified applications of IoT in the agriculture sector fall into eight areas using the meta-synthesis method. The authors extracted a set of criteria from the literature. Later, the fuzzy Delphi method helped finalise it. The authors extended the SWARA method with interval-valued triangular fuzzy numbers (IVTFN SWARA) and used it to the weighting of the characteristics. Then, the alternatives were rated using the Additive Ratio Assessment (ARAS) method based on interval-valued triangular fuzzy numbers (IVTFN ARAS). Finally, decision-makers evaluated the results of ratings based on two dimensions of sustainability and IoT challenge by developing a framework for decision-making. Results of this paper show that policy-makers can manage IOT innovations in a disciplined way that balances risk and return by a portfolio approach, simultaneously the proposed framework can be used to determine and prioritise the areas of IoT application in the agriculture sector.


The IoT in the agricultural sector aims to empower farmers through automation technologies and decision-making tools that lead to the integration of products, knowledge and services for higher productivity, better quality (Elijah et al.2018), and sustainability (Sundmaeker et al.2016). Despite the many different innovative applications of IoT in the agricultural sector, policy-makers (PMs) only recently began to consider the risk and return of using this technology. Therefore, to establish policies for the development of this technology there is a need for a framework to ensure the realisation of sustainable development (SD) goals in the agricultural sector and consider the smallest challenges for IoT implementation. Due to complex and multi-dimensional nature of sustainability issues (Shen and Tzeng, 2018) and the evolutionary nature of the IoT (Kim and Kim, 2016), decision-making faces problems such as lack of complete information and the need to use experts’ opinions for decision-making. Given the capability of multiple-attribute decision-making (MADM) techniques to meet similar conditions (Kim and Kim, 2016; Luthra et al.2018; Mohammadzadeh et al.2018; Quaddus and Siddique, 2001; Vinodh et al.2013; Zarei et al.2016), it seems that the use of these techniques can be useful in establishing the intended policy-making framework.

In many situations, candidates’ information for decision-making is usually uncertain and incomplete. Therefore, using MADM techniques that consider input parameters accurately and certainly is an inefficient approach (Turskis and Zavadskas, 2010a). Decision-makers use fuzzy number sets when solving complicated problems to overcome this limitation (Lima Junior et al.2014). Although the use of fuzzy numbers has enabled many real-world decision-making problems to be solved (Turskis et al.2019b), it cannot adequately meet all the requirements of such problems due to uncertainty (Dahooie et al.2018; Stanujkic, 2015). Therefore, Atanassov (1986) generalised the fuzzy numbers and introduced intuitive fuzzy numbers (Ye, 2010). Intuitive fuzzy numbers allow better visualisation of ambiguity and environmental uncertainty (Dahooie et al.2018).

On the other hand, Iran, as a country transitioning from a factor-driven economy to an efficiency-driven economy, seeks to deploy IoT in various industries and use its benefits to achieve the economic 1404 outlook (Iran’s 2025 outlook) and SD goals (Zarei et al.2016). Despite the many different applications of IoT in the agricultural sector, PMs only recently began to consider the possibilities and benefits of using this technology in Iran. The purpose of this study is to present a MADM framework for policy-making in implementing IoT applications in the agriculture sector. The literature review helps to identify a list of sustainability attractiveness criteria, and IoT challenges to prioritise IoT applications in the agriculture sector to achieve more SD goals and face the least challenges. The PMs, due to research limitations in this field, used the fuzzy Delphi method (FDM) and expert opinions to create a useful list of criteria. Experts weighted the identified criteria based on the IVTFN SWARA method in the next step. Due to the aim of the study, the authors extend the SWARA method to utilise IVTFN. In the next step, the PMs calculated performance scores of each of the identified applications using the IVTFN ARAS. Finally, the proposed framework assists PMs in selecting appropriate IoT applications based on the two criteria of achieving more sustainability and facing fewer challenges for implementing IoT in the agricultural sector in Iran.

The existing literature on using IoT in the field of agriculture usually focus on topics such as IOT architectures and network layers from a technical point of view (Villa-Henriksen et al.2020) and less systematically investigate the IoT applications from the perspective of national policymakers, so the main advantages of this research are:

  • Introducing a new classification for different IoT applications in agriculture.

  • Identify and categorize both the challenges of the Internet of Things and its benefits from a sustainable development perspective on a national level.

  • Extend SWARA based on IVTFN to calculate the weight of criteria.

  • Provide a new decision-making matrix for selecting high-priority IoT applications in the agriculture sector using interval-valued triangular fuzzy sets.

  • Assist national policy makers to better understand, and thus implement the applications of IoT in the field of agriculture.

This paper is organized as follows. Section 3.1 reviews the literature and discusses the need to establish a decision-making framework for the selection of appropriate IoT applications that encounter fewer challenges and more sustainability. Section 3.2 describes the research method. Section 4 illustrates the case study and presents the results. Finally, Section 4 concludes the research.

2Materials and Methods

2.1Literature Review

Modern policy-makers must effectively manage projects (Yazdani et al.2019a), choose to implement the best possible solutions based on many factors affecting the environment and economic expediency (Zemlickienė and Turskis, 2020) and technology (Bagočius et al.2014; Ruzgys et al.2014). Significant political requirements change aspects of the rationality presumption of management at local and international levels (Hashemkhani Zolfani et al.2013; Erdogan et al.2017). Policy-makers must take into account the social well-being of people (Zagorskas and Turskis, 2020a, 2020b). The rapid development of the world and the growth of the population require the emergence of new technologies (Zavadskas et al.2013) and raise the need to develop decision-making theories and methodologies to justify the rational use of them (Yazdani et al.2019b). There seems to be no limit to the improvement and refinement of multi-criteria decision-making algorithms. The IoT paradigm referred to as one of the drivers of the Industry 4.0 revolution (Geng, 2017) has been able to transform the society dramatically and achieve the goals of SD (Benkhelifa et al.2014; Biggs et al.2016; Hopwood et al.2005). Therefore, it is useful to empower sustainable agriculture (it is a concept stemming from the idea of SD published in the 1987 Brundtland Committee Report). In order for the decision-makers to achieve SD goals, they need a balance of economic, social, environmental, and technical issues (Merad et al.2013) and face a variety of IoT applications in the agriculture sector (Elijah et al.2018; Ray, 2017; Verdouw et al.2016). Therefore, the MADM techniques can support the decision-makers in this field (Quaddus and Siddique, 2001). In the following, the authors discuss the background of the research on the importance of sustainable agriculture, the concept of IoT, the applications of IoT in the agriculture sector, and the necessity of establishing a decision-making framework to reduce IoT challenges and achieve SD goals.

The concept of sustainable agriculture has emerged and has been considered by national PMs. The Food and Agriculture Organization (FAO) defines sustainable agriculture as the management and conservation of critical natural resources and the direction of technological change in a way that will continually satisfy human needs for present and future generations. Various papers have set out different criteria for measuring the extent to which SD goals are being met in the agricultural sector. PMs categorise them into four types: environmental, economic, social, and technical. Table 1 presents a list of criteria extracted from the literature.

Table 1

A literature review on sustainable development (criteria and methods).

(Wang et al.2019a)(Scharfy et al.2017)(Cardoso et al.2018)(Veisi et al.2016)(Chiou et al.2005)(Quaddus and Siddique, 2001)(Marcis et al.2019)(Zahm et al.2008)(Rezaei-Moghaddam and Karami, 2008)(Velten et al.2015)(Poursaeed et al.2010)(Fatemi and Rezaei-Moghaddam, 2019)(Liu et al.2019)(Khishtandar et al.2017)
Criteria extraction methodLLLLLLLIDEAL*L*LLLL
EnvironmentalGlobal warming
Environmental pollution
Rational use of natural resources
Development of clean energy extraction
Ecologic resilience
Non-renewable resource recycling (Waste management)
Quality of products
Animal well-being
Harmony with nature
EconomicFinancial measures and profitability [return on assets (ROA), return on equity (ROE), return on investment (ROI)]
Saving of fossil fuel energy
Investment costs
Amortisation time
Life cycle costs
Economic dependency on natural resources
Economic growth
Combination of resource possession systems
Risk management
SocialSocial acceptability
Health and safety and welfare and quality of life (employees and cooperative members)
Improvement of environmental culture and awareness
Equity and poverty alleviation
Mutual collaboration and participation
Education and qualification (employees and cooperative members)
Cultural preservation
Reducing farmers migration
Compatibility with a political, legislative and administrative framework
Contribution to regional development
TechnicalDevelopment potential
Eco-friendly technologies
Environmental adaptability
Modern agricultural technologies for yield increase
Technology maturity
∘= is partly or implicitly proposed by the author(s)∙= is (explicitly) offered by the author(s)

Note: FAHP = fuzzy Analytic Hierarchy Process; VIKOR = Vlsekriterijumska Optimizacija I Kompromisno Resenje; AHP = Analytic Hierarchy Process; MAVT = Multiple Attribute Value Theory; CRITIC = CRiteria Importance Through Inter-Criteria Correlation; PROMETHEE = Preference Ranking Organization Method for Enrichment Evaluations; FTOPSIS = Fuzzy Technique for Order Preference by Similarity to Ideal Solution; HFLTS = Hesitant Fuzzy Linguistic Term Set; MAMCO = Multi Actor Multi-Criteria Outranking; L = literature; L* = literature and Delphi; IDEA = indicateurs de durabilité des exploitations agricoles or farm sustainability indicators.

2.1.1The Concept of IoT

Table 2

A literature review on the challenges of IoT development (criteria and methods).

(Mohammadzadeh et al.2018)(Khan and Ismail, 2017)(Brewster et al.2017)(Pivoto et al.2018)(Elijah et al.2018)(Kamienski et al.2018)(Upadhyay et al.2019)(Al-Fuqaha et al.2015)(Aldowah et al.2019)(Diwaker et al.2019)(Aswale et al.2019)(Akinyoade and Eluwole, 2019)(Instituto de Normalización, Acreditación, 2018)
Weighting methodFANP
Selection MethodFANP
Criteria extraction methodL
TechnologicalArchitecture and design
Device heterogeneity
Ubiquitous data management
Hardware construction
Fault tolerance
Lack of supporting infrastructure
Data processing power
Choice of technology
Optimisation of resources
Data volume and scalability
Privacy and securityData confidentiality
Network security
IoT devices’ safety
Conflict of interests
Security vulnerabilities in the overall IoT system
Identity and access management
BusinessBusiness model
Investing in IoT development
Economic development opportunities and issues
Customer expectations and quality of service
Heterogeneity of the sector
Farm sizes and capital investment costs
Legal and regulatoryData usage
Cross border data flows and global cooperation
Data ownership and data collection management
GDPR and IoT
CulturalEducation and training
∘= is partly/implicitly proposed by author(s)∙ = is (explicitly) proposed by author(s)

Note: FANP = fuzzy Analytic Network Process; L = literature.

The traditional concept of the internet as an infrastructure network seeks to diminish end-user terminals and move to the idea of “smart” interconnected objects to shape pervasive computing environments. A new paradigm called “Internet of Things” (Atzori et al.2010; Noje et al.2020; Perera et al.2014) has been formed to refer to:

  • 1) The global network of intelligent objects interconnected by advanced internet technologies;

    Fig. 1

    The process of identifying IoT applications in different areas.

    The process of identifying IoT applications in different areas.

  • 2) A set of support technologies needed to achieve the desired vision (including RDFs, sensors/actuators, Machine-to-Machine communication devices, and others);

  • 3) Group of applications and services that leverage these technologies to establish new businesses and maximise market opportunities (Miorandi et al.2012).

Identifying the challenges to the implementation of IoT can help researchers and PMs in planning to develop its applications. Table 2 highlights the most critical challenges facing the development of IoT from the technological, security, business, legal, and cultural aspects.

2.1.2IoT Applications in the Agriculture sector

Since IoT applications are widespread and evolving (Kim and Kim, 2016) and have a variety of applications in the agricultural sector (Elijah et al.2018), the Sandelowski and Barroso’s Meta-Synthesis Method is used to identify applications of the IoT in the agriculture sector (Sandelowski and Barroso, 2006). Figure 1 summarises the research process, including the criteria for accepting and rejecting articles, keywords used in searching for research articles, research timeframe, and the search of databases, journals, and various search engines. In the final step, policy-makers out of 862 items were identified, and 403 articles were analysed.

The identified IoT applications fall into eight areas: open-field agriculture, greenhouses, hydroponics and aquaponics, open-air horticulture, livestock farming, fishery and aquaculture, forestry, and distribution and supply networks. Table 3 presents the IoT applications identified based on different agriculture areas.

On the one hand, the applications of IoT in various agricultural areas are ubiquitous. On the other hand, the implementation of IoT has many challenges due to its evolutionary nature. However, no framework has been provided to assist PMs in making decisions regarding implementing IoT in different agriculture areas. To be transparent, a framework for selecting the agriculture areas that are more appropriate for implementing IoT that must be considered both SD goals and IoT challenges simultaneously is necessary. Of course, as previously shown in Tables 1 and 2, measuring each of these two objectives need to take into account several criteria that are sometimes inconsistent. MADM approaches offer a variety of methods for obtaining expert opinions and making the right decisions on a particular issue by considering multiple criteria (Shen and Tzeng, 2018). A review of applications of MADM methods in similar domains (Kim and Kim, 2016; Luthra et al.2018; Quaddus and Siddique, 2001; Vinodh et al.2013; Zarei et al.2016) indicates the potential of these methods in establishing decision-making frameworks in the field of study.

Table 3

Applications of IoT in different agricultural areas

ApplicationIoT roleArea
Weather forecastingMonitoring weather attributes including humidity, temperature, soil moisture, rainfall and the light intensity across the farmland in remote locations and also the weather forecast data
Irrigation management systemMonitoring soil moisture, soil temperature, environmental parameters
Estimation of critical virtual water for irrigation in the districtMonitoring environmental parameters in the districts
Agricultural drought data acquisitionGathering and monitoring agricultural drought data in real-time
Pump control systemReal-time and remotely monitoring and controlling pumps
Municipal wastewater monitoring and control system for agriculture and gardening applicationReal-time pH monitoring and control
Water level monitoring systemCollect, analyse and predict the water level detail, water usage and other information of particular water source from a remote location in real-time
Water quality assessment systemMonitoring water quality attributes including chemicals, pH, and temperature
Disease and pest detection and controlCollect, analyse and predict the disease in leaf, stem and fruit and pests through image processing
Weed detection systemPredict the weeds through image processing and based on related statistical algorithms
Agricultural machinery intelligent schedulingConsidering the factors including weather and crop mature time, computing the smallest distance matrix of all deployment sites and the smallest path matrix relevant, assigning tasks and sorting the task routes
Navigation system for agricultural machinesNavigate automated guided vehicles on a field based on global positioning system and google maps service
Seedbed monitoringMonitoring environmental parameters of seed breeding including soil temperature, soil humidity, air humidity, light, ambient, and air temperature
Agriculture Market Information System for small-scale farmersCollecting and monitoring data including product type with image, quality of product with a current close-up picture, current growth stage with image, estimated date of harvest, the estimated quantity of yield, farmers contact, location of the farm/field and distance from the nearest road point
Predictive crop growth modelsStores periodic data collected through environmental and soil parameters, and then Big Data analysis is carried out for providing suggestions to the farmers for crops to be taken on the farmland with peculiar soil properties based on the previous stock of agro products and current requirements in the market
Farm management systemMonitoring soil and environmental parameters
Automatic sorting systemClassification crops by image processing
Waste managementReal-time and continuous acquisition and analysis of decisive variables allow the identification, monitoring, improvement, and optimisation of various components along with the design of the supply chain
Monitoring system to prevent animal attacksDetect and avoid animal intrusions
Detection and agriculture product theft prevention systemDetermining abnormal behaviours by using an image monitoring system
The surveillance system in the agricultural drying processMonitoring and tracking environmental parameters including temperature and moisture in the agricultural drying process
Satellite monitoring agricultural landsCollect and analyse information through satellite monitoring
Smart flood disaster prediction systemReal-time monitoring flood attributes including humidity, temperature, pressure, rainfall and river water level
Condition monitoring systemMonitoring environmental parameters including the amount of rainfall, leaf wetness, temperature, humidity, soil moisture, salinity, climate, dry circle, solar radiation, pest movement, human activities
Fertilisation System (fertiliser requirement for the current crop)Monitoring soil condition such as soil moisture, soil temperature and soil pH
Warehousing managementCondition monitoring including temperature and humidity of crops and detection of the presence of any beetles and invader
Energy managementMonitor the status of the small off-grid photovoltaic system consists of a photovoltaic voltage and ampere, battery voltage, and battery current loading
Leisure agriculture intelligent systemCollect and analyse new data about fields and shows out in the website form for tourist
Frost event prediction systemGather and monitor data including air temperature, air relative humidity, soil moisture, soil temperature that represents the orchard environment
Cattle movement and feed monitoringAnimal tracking and behavioural analysis and monitoring of animals feed
Fire detection systemMonitoring soil and air temperatures and humidity, and the different levels of gases such as carbon monoxide, carbon dioxide, toluene, oxygen, hydrogen, methane, isobutene, ammonia, ethanol, hydrogen sulfide, and nitrogen dioxide
Timber tracing managementTracking timbers from the forest through the supply chain to the consumer
Tree trackingForest identification and tree tracking
Food safety traceability systemPutting a sensor tag on crops to trace its production, processing, wholesale and retail
Management information systemReal-time Tracking of agricultural products in different areas with RFID technology
Real-time pricing mechanismReal-time pricing by implementing a smart RFID/NFC price tag by identifying food quality features

Note: A1 = open-field agriculture; A2 = greenhouses; A3 = hydroponics and aquaponics; A4 = open-air horticulture; A5 = livestock farming; A6 = fishery and aquaculture; A7 = forestry; A8 = distribution and supply networks; RFID = Radio-frequency identification; NFC = Near-field communication.

2.2Research Methodology

The authors aim to provide a decision-making framework for PMs regarding the implementation of IoT in the agricultural sector under the conditions of Iran. In establishing this framework, achieving SD goals by the identified IoT applications and facing the least challenges for IoT implementation in the agricultural sector should be considered. The necessity of simultaneous attention to these two issues is similar to the idea of choosing an IT investment portfolio (Peppard and Ward, 2012; Ramakrishnan, 2008; Silvius, 2010) or technology portfolio management (Jolly, 2003).

Fig. 2

Portfolio view based on risk and attractiveness Adapted from Silvius (2010).

Portfolio view based on risk and attractiveness Adapted from Silvius (2010).

The primary purpose of information technology investments is to identify aspects related to the value (attractiveness) and the risk of projects individually as well as project portfolios. The term IT portfolio management, which is derived from the financial portfolio investment model, represents the process of evaluating projects before investment (Ramakrishnan, 2008). The most important advantage of this approach is to understand the value of an investment that is affected by other investments or assets in the portfolio. In other words, investment decisions are not taken individually, and this method examines investments concerning other investments and assets (Silvius, 2010). It enables management to visualise the pattern of investments and enhance the project-level debate to the entire portfolio (Peppard and Ward, 2012). The idea of portfolio matrix formation, in addition to commercial areas, has been used in many areas such as IT infrastructure investment assessment (Peppard and Ward, 2012; Ramakrishnan, 2008; Silvius, 2010), technology portfolio management (Jolly, 2003), transportation project evaluation (Reza Ghaeli et al.2003), and others. Figure 2 represents the portfolio matrix consisting of two dimensions of attractiveness and challenges.

With this in mind, to build a decision-making framework based on sustainability indicators (Attractions) as well as the challenges of IoT development, initially, the IoT applications in the agriculture sector were identified and categorised into eight main areas. The criteria existed in the literature were selected, and the final list of criteria then was determined using a fuzzy Delphi process with the help of experts in the field in the form of a working group (consisting of representatives from the Agricultural Research, Education and Development Organization and Iran Telecommunication Research Center). The IVTFN SWARA method weights the criteria. Later, the IVTFN ARAS method rates the IoT applications based on different areas identified in the agriculture sector. To this end, the team members evaluated each identified application according to the final list of criteria. Finally, the ratings were assessed based on two dimensions of SD and the challenges of IoT development, and a suitable framework for policy-making in this area was proposed. Figure 3 presents the details of each of these steps.

Fig. 3

The executive process of the research.

The executive process of the research.

2.2.1Fuzzy Delphi Method

In this study, the fuzzy Delphi method (FDM) is used to adapt the experts’ views on the determination of essential criteria (Lin, 2013) in evaluating IoT applications in the agriculture sector. The Delphi method, first used in technology prediction in 1950, was rapidly developed in various fields (Mullen, 2003). Then, in 1985, Murray et al. improved its ambiguity and uncertainty by applying fuzzy theory (Dahooie et al.2018; Kuo and Chen, 2008). Since various techniques have been proposed for FDM calculations, this study uses the approach proposed Kuo and Chen (2008). Each fuzzy number (TA) is defined as follows:

where XAi is the proposed value of the decision-maker i in terms of the critical factor A, (i=1,2,). LA, UA, and MA are the lower limit values, the upper limit values, and the geometric mean of the essential factor A, respectively. In the next step, the defuzzification process is performed using the following model (Dahooie et al.2018):
where Lk, Uk, and Mk are the lower limit values, the upper limit values, and the geometric mean of the vital factor k, respectively.

The last step is to set a threshold to accept or reject the criteria (Dahooie et al.2018). For this purpose, according to experts’ opinions, the value 4 and 3.5 was set as the threshold for the SD criteria and IoT challenge criteria, respectively. Finally, the criteria with the values lower than the threshold were removed from the list, and the final list of measures necessary for evaluation was created (Tables 3 and 4).

2.2.2Interval-Valued Triangular Fuzzy Numbers

One of the applications of the fuzzy number set theory proposed by Zadeh (1965, 1975) is to use it in the process of making decisions based on ambiguous and uncertain information like decision-makers’ opinions (Lima Junior et al.2014). In this theory, the membership of an element in the fuzzy number can only have a definite value between zero and one. However, in reality, the degree of non-membership of an element in the fuzzy number is not equal to one minus the degree of membership. It means that there may be some degree of hesitance. Thus, Atanassov introduced intuitive fuzzy sets by generalising the fuzzy number set (Atanassov, 1986; Zeng et al.2019). The intuitive fuzzy number has the features of membership degree, non-membership degree, and degree of hesitance. Atanassov and Gargov (1989) presented the interval-valued intuitive fuzzy numbers set by generalising the fuzzy number (Wang et al.2019b; Ye, 2010). As with generalised fuzzy numbers, the interval-valued fuzzy numbers can be trapezoidal (interval-valued trapezoidal fuzzy numbers) or triangular (IVTFN). As shown in Fig. 4, Yao and Lin (2002) presented an IVTFN as follows Stanujkic (2015):

where A˜L and A˜U represent low and high triangular fuzzy numbers, respectively. A˜LA˜U;μA˜(x) is the membership function and denotes the degree to which an event may be a member of A˜. μA˜L(x)=ωA Besides, μA˜U(x)=ωA are the lower and upper membership functions, respectively.

Fig. 4



Besides, a particular case of IVTFN is the normalised IVTFN in which ωA=ωA=1 has the same state. The normalised IVTFN shown in Fig. 4 can be represented in Fig. 5 (Stanujkic, 2016).


Fig. 5

Normalised IVTFN.

Normalised IVTFN.

If A˜=[A˜L,A˜U]=[(al,al),am,(au,au)] and B˜=[B˜L,B˜U]=[(bl,bl),bm,(bu,bu)] are two IVTFN, the main algebraic operation can be defined as follows Stanujkic (2016).


2.2.3Linguistic Variables

Decision-makers cannot quantitatively evaluate many aspects of various activities in the real world but can qualitatively evaluate, i.e. by ambiguous or inaccurate knowledge. In this case, using linguistic variables instead of numerical values could be a better approach (Keršulienė and Turskis, 2014). Therefore, the variables used in such problems are evaluated through linguistic terms (Herrera and Martínez, 2000). In the researches done by several authors, different linguistic variables based on triangular or trapezoidal fuzzy numbers have been proposed (Stanujkic et al.2015). Table 4 presents linguistic variables for weighting criteria and ranking alternatives based on IVTFN (Stanujkic et al.2015).

Table 4

Linguistic variables for weighting criteria and ranking alternatives adapted from Stanujkic et al. (2015).

Linguistic variables for weighting criteriaLinguistic variables for ranking alternatives
Linguistic variablesIVTFNLinguistic variablesIVTFN
Equally important (EI)[(0.75,0.8);0.9;(0.9,0.9)]Very High (VH)[(0.75,0.8);0.9;(0.9,0.9)]
Moderately less important (MOL)[(0.55,0.6);0.7;(0.8,0.85)]High (H)[(0.55,0.6);0.7;(0.8,0.85)]
Less important (LI)[(0.35,0.4);0.5;(0.6,0.65)]Medium (M)[(0.35,0.4);0.5;(0.6,0.65)]
Very less important (VLI)[(0.15,0.2);0.3;(0.4,0.45)]Low (L)[(0.15,0.2);0.3;(0.4,0.45)]
Much less important (MUL)[(0.1,0.1);0.1;(0.2,0.25)]Very Low (VL)[(0.1,0.1);0.1;(0.2,0.25)]

2.2.4Development of the SWARA Method Based on Interval-Valued Triangular Fuzzy Numbers

There are a variety of methods for assessing the importance of criteria that researchers have used or modified when solving complex problems, e.g. eigenvector method, SWARA (Keršuliene et al.2010), expert judgement method, Eckenrode’s rating technique (Turskis et al.2019a), Analytic Hierarchy Process (AHP) (Saaty, 1977, 1980; Turskis et al.2012; Zavadskas et al.2020) and Entropy method. Keršuliene et al. (2010) developed the Step-wise Weight Assessment Ratio Analysis (SWARA) method to weight the criteria based on the opinions and judgments of decision-makers and experts (Keršuliene et al.2010; Keršulienė and Turskis, 2011). This method is a subjective method with low complexity, low time, and the ability to estimate the opinions of experts or stakeholders on the importance of criteria in the weighting process (Karabasevic et al.2016; Stanujkic et al.2017). On the other hand, the inaccuracy and inadequacy of candidates’ information in many situations lead to the inefficiency of using methods that consider input parameters accurately and conclusively (Ye, 2010). Among sets that consider uncertainty in decision-making, interval-valued fuzzy numbers have received special attention from researchers. However, despite numerous developments of the SWARA method, this method has not yet been developed to take advantage of these numbers. Therefore, the authors develop an IVTFN SWARA extension and present it below.

  • 1. Prioritise the criteria: At this stage, the final criteria for evaluating the alternatives are sorted based on their importance in a descending order. The most important and the least essential criteria are ranked highest and lowest in the ranking process, respectively.

  • 2. Determine the relative importance of the criteria (S˜j): The relative importance of each measure is measured close to the higher rank criterion, which is represented by the value S˜j.

  • 3. Calculate the coefficient K˜j: The coefficient K˜j as a function of relative importance for each criterion is determined using equation (10)


  • 4. Calculate the initial weight of the criteria: At this stage, the initial importance of each measure is calculated using equation (11)


  • 5. Calculate the final normalised weight: Finally, the final normalised weights are obtained by equation (12)

    The defuzzification method (equation (13) represents the final weights of the criteria (Dahooie et al.2018)


The ARAS method is one of the newest MADM approaches introduced by Zavadskas and Turskis in 2010 (Zavadskas and Turskis, 2010). This method is useful to solve various decision-making problems. According to the literature, the fuzzy (Turskis and Zavadskas, 2010a), grey (Turskis and Zavadskas, 2010b; Mardani et al.2018), and IVTFN (Dahooie et al.2018) extensions of this method are proposed. The steps of the IVTFN ARAS are as follows (Dahooie et al.2018).

1. Formulate the decision matrix and determine the optimal performance rating for each criterion

where m is the number of alternatives, n is the number of criteria describing each choice, x˜ij is the interval-valued fuzzy number that denotes the performance value of the option i in criterion j, and x˜0j is the optimal value of criterion j. The mark ‘∼’ at the top of each symbol indicates an interval-valued triangular fuzzy set.

The optimal performance rank for each criterion with the interval values is calculated using equation (15).

where x˜0j denotes the optimal fuzzy performance rank with interval values for criterion j. Besides, the other criteria are defined as follows.
where Ωmax denotes the benefit-type criteria (where the higher value is preferable), and Ωmin denotes the cost-type criteria (where the lower value is preferable).

2. Equation (22) helps to normalise the values

where x˜ij denotes the optimal fuzzy performance rank with interval values for alternative i in criterion j:

3. Calculation of weighted normalized decision matrix with interval values

At this stage, using the normalized weights obtained by the IVTFN SWARA method and the normalized decision matrix, the weighted normalized decision matrix Xˆ˜ is defined.

The following equation determines the weighted normalised values of all criteria.

4. Calculate the overall performance index for each alternative

The overall performance index for each alternative is calculated by equation (25)

where S˜i denotes the overall performance index for alternative i.

5. Calculate the degree of utility for each alternative

The fuzzy decision result for each alternative is in the form of IVTFN. There exist different defuzzification processes. They may affect the problem solution results. The defuzzification of S˜i is performed using equation (26)

There λ is a coefficient in the interval [0,1]. The PMs can give more importance to Sil and Siu than Sil and Siu, and vice versa by changing λ coefficient. The utility degree of each alternative is obtained by comparing its value with the ideal value of S0.

Equation (27) represents the utility degree of alternative Ai

where S0 and Si are the defuzzified optimal index values obtained from equation (26), and values of Ki are in the interval [0,1].

6. Alternative ranking and choosing the most effective alternative

At this point, the decision-makers ranked the alternatives based on Ki values.

2.2.6Developing an Evaluation Framework

IoT development in Iran is in its infancy. Besides, IoT related technologies and enablers have not yet evolved and are facing many challenges. Therefore, the scores associated with each of the two dimensions of Sustainability and Challenges of IoT Development were first calculated using a combination of IVTFN SWARA and IVTFN ARAS methods. Then, the scores are normalised into the interval [0,1] using equation (28) (Zeleny, 1973).

where xi is the score of application i (i=1,2,3,,8) in the associated dimension and max/min(xi) are the most significant and smallest values.

3Case Study and Results

Iran aims to leverage IoT opportunities in all sectors, including agriculture, to achieve SD goals. Therefore, in November 2016, the Ministry of Agriculture Jihad, in collaboration with Ministry of Communication & Information Technology, formed joint working groups under the titles of Infrastructure, Remote Sensing, IoT, and Basic Resources, Business Development and Improvement, Training Development and Empowerment, and Governance Platform to implement this technology. As a result, policy-makers formed a team of experts from among the members of these working groups. The applications of IoT in various agricultural areas were identified through reviewing the literature and categorised in eight fields of open-field agriculture (A1), greenhouses (A2), hydroponics and aquaponics (A3), open-air horticulture (A4), livestock farming (A5), fishery and aquaculture (A6), forestry (A7) and distribution and supply networks (A8). The results of each of the steps taken for evaluation are described below.

3.1Fuzzy Delphi Results

To prioritise the identified agriculture areas for implementing IoT applications based on the two dimensions of SD and the challenges of IoT development, the extracted criteria, which present Tables 1 and 2, were distributed among the experts in Agricultural Research, Education, and development Organization and Research Institute for Information and Communication Technology in the form of a questionnaire to identify the critical criteria. The questionnaire consisted of two sections. The first section evaluated the importance of the requirements extracted from the previous studies. In the second section, experts were asked to suggest other measures that may be effective in evaluation. As shown in Tables 5 and 6, using the FDM, 14 and 22 criteria, respectively, were selected as the final criteria for evaluating the IoT applications based on the experts’ opinions.

Table 5

Final and localised criteria to prioritise alternatives based on SD indicators.

C1EnvironmentalC11Global warming
C12Environmental pollutions
C13Rational use of natural resources
C14Quality of products
C2EconomicC21Financial measures and profitability [return on assets (ROA), return on equity (ROE), return on investment (ROI)]
C22Risk management
C3SocialC31Health and safety and welfare and quality of life (employees and cooperative member)
C32Reducing farmers migration
C33Contribution to regional development
C4TechnicalC41Development potential
C42Eco-friendly technologies
C43Modern agricultural technologies for yield increase
C44Technology maturity
Table 6

Final and localised criteria to prioritise alternatives based on the challenges of IoT development.

I1TechnologicalI11Architecture and design
I13Lack of supporting infrastructure
I14Data processing power
I15Choice of technology
I2Privacy and securityI21Transparency
I22IoT devices’ safety
I23Conflict of interests
I3BusinessI31Business model
I32Investing in IoT development
I33Economic development opportunities and issues
I34Customer expectations and quality of service
I35Heterogeneity of the sector
I36Farm sizes and capital investment costs
I4Legal and regulatoryI41Data usage
I42Cross border data flows and global cooperation
I44Data ownership and data collection management
I45GDPR and IoT
I5CulturalI51Education and training

3.2Results of the IVTFN SWARA Method

In this section, decision-makers used the developed IVTFN SWARA method to determine the weights of the criteria and relevant sub-criteria based on experts’ opinions. Tables 7 and 8 present the results.

As shown in the third column of Table 7, experts ranked the criteria based on their importance in a descending order. Also, the results of the second, third (equation (10)), fourth (equation (11)), and fifth (equation (12)) steps of the IVTFN SWARA method are presented in columns 4, 5, 6 and 7 of Table 7, respectively. Besides, the final weights of the criteria were normalised after the integration steps using the geometric mean, and are shown in Table 8.

Table 7

Calculating the weight of the first expert’s criteria.

Criterion CodeExperts’ opinionComparative importance of average value S˜jCoefficient k˜j=S˜j+1Recalculated weight q˜jWeight (w˜j)
Sustainable developmentC2[(1, 1), 1, (1, 1)][(1, 1), 1, (1, 1)][(0.39, 0.41), 0.44, (0.46, 0.47)]
C1LI[(0.15, 0.2), 0.3, (0.4, 0.45)][(1.15, 1.2), 1.3, (1.4, 1.45)][(0.61, 0.63), 0.67, (0.71, 0.74)][(0.24, 0.26), 0.29, (0.33, 0.35)]
C3EI[(0.1, 0.1), 0.1, (0.2, 0.25)][(1.1, 1.1), 1.1, (1.2, 1.25)][(0.32, 0.33), 0.35, (0.4, 0.42)][(0.13, 0.13), 0.15, (0.18, 0.2)]
C4VLI[(0.1, 0.1), 0.1, (0.2, 0.25)][(1.1, 1.1), 1.1, (1.2, 1.25)][(0.22, 0.23), 0.27, (0.33, 0.37)][(0.09, 0.1), 0.12, (0.15, 0.17)]
C41[(1, 1), 1, (1, 1)][(1, 1), 1, (1, 1)][(0.42, 0.44), 0.47, (0.48, 0.48)]
C43EI[(0.15, 0.2), 0.3, (0.4, 0.45)][(1.15, 1.2), 1.3, (1.4, 1.45)][(0.53, 0.53), 0.53, (0.56, 0.57)][(0.22, 0.23), 0.25, (0.27, 0.28)]
C44LI[(0.1, 0.1), 0.1, (0.2, 0.25)][(1.1, 1.1), 1.1, (1.2, 1.25)][(0.32, 0.33), 0.35, (0.4, 0.42)][(0.14, 0.14), 0.16, (0.19, 0.2)]
C42VLI[(0.15, 0.2), 0.3, (0.4, 0.45)][(1.15, 1.2), 1.3, (1.4, 1.45)][(0.22, 0.23), 0.27, (0.33, 0.37)][(0.09, 0.1), 0.13, (0.16, 0.18)]
Internet of things challengesI2[(1, 1), 1, (1, 1)][(1, 1), 1, (1, 1)][(0.31, 0.33), 0.36, (0.39, 0.41)]
I4VLI[(0.15, 0.2), 0.3, (0.4, 0.45)][(1.15, 1.2), 1.3, (1.4, 1.45)][(0.69, 0.71), 0.77, (0.83, 0.87)][(0.22, 0.24), 0.28, (0.33, 0.36)]
I3MOL[(0.1, 0.1), 0.1, (0.2, 0.25)][(1.1, 1.1), 1.1, (1.2, 1.25)][(0.37, 0.4), 0.45, (0.52, 0.56)][(0.12, 0.13), 0.16, (0.21, 0.23)]
I1LI[(0.1, 0.1), 0.1, (0.2, 0.25)][(1.1, 1.1), 1.1, (1.2, 1.25)][(0.23, 0.25), 0.3, (0.37, 0.42)][(0.07, 0.08), 0.11, (0.15, 0.17)]
I5VLI[(0.35, 0.4), 0.5, (0.6, 0.65)][(1.35, 1.4), 1.5, (1.6, 1.65)][(0.16, 0.18), 0.23, (0.31, 0.36)][(0.05, 0.06), 0.08, (0.12, 0.15)]
I53[(1, 1), 1, (1, 1)][(1, 1), 1, (1, 1)][(0.48, 0.5), 0.52, (0.53, 0.53)]
I52EI[(0.15, 0.2), 0.3, (0.4, 0.45)][(1.15, 1.2), 1.3, (1.4, 1.45)][(0.53, 0.53), 0.53, (0.56, 0.57)][(0.25, 0.26), 0.27, (0.29, 0.3)]
I51VLI[(0.15, 0.2), 0.3, (0.4, 0.45)][(1.15, 1.2), 1.3, (1.4, 1.45)][(0.36, 0.38), 0.4, (0.46, 0.5)][(0.18, 0.19), 0.21, (0.24, 0.26)]
Table 8

Final Normalised weights.

Criterion CodeExpert 1Expert 2Expert 3Aggregated based on the geometric meanNormalised final weights
Sustainable developmentC1[(0.24, 0.26), 0.29, (0.33, 0.35)][(0.42, 0.43), 0.46, (0.48, 0.49)][(0.24, 0.26), 0.29, (0.31, 0.33)][(0.29, 0.3), 0.34, (0.37, 0.38)][(0.26, 0.28), 0.35, (0.42, 0.46)]
C2[(0.39, 0.41), 0.44, (0.46, 0.47)][(0.23, 0.24), 0.27, (0.3, 0.32)][(0.45, 0.46), 0.49, (0.5, 0.51)][(0.34, 0.36), 0.38, (0.41, 0.42)][(0.3, 0.33), 0.4, (0.47, 0.51)]
C3[(0.13, 0.13), 0.15, (0.18, 0.2)][(0.14, 0.15), 0.18, (0.21, 0.23)][(0.07, 0.07), 0.08, (0.1, 0.11)][(0.11, 0.11), 0.13, (0.16, 0.17)][(0.09, 0.1), 0.13, (0.18, 0.21)]
C4[(0.09, 0.1), 0.12, (0.15, 0.17)][(0.07, 0.08), 0.09, (0.12, 0.13)][(0.13, 0.14), 0.15, (0.17, 0.19)][(0.09, 0.1), 0.12, (0.15, 0.16)][(0.08, 0.09), 0.12, (0.17, 0.2)]
C41[(0.42, 0.44), 0.47, (0.48, 0.48)][(0.25, 0.26), 0.27, (0.29, 0.29)][(0.24, 0.25), 0.28, (0.31, 0.33)][(0.29, 0.31), 0.33, (0.35, 0.36)][(0.07, 0.08), 0.09, (0.1, 0.11)]
C42[(0.09, 0.1), 0.13, (0.16, 0.18)][(0.13, 0.14), 0.14, (0.16, 0.17)][(0.13, 0.13), 0.15, (0.17, 0.19)][(0.12, 0.12), 0.14, (0.16, 0.18)][(0.03, 0.03), 0.04, (0.05, 0.05)]
C43[(0.22, 0.23), 0.25, (0.27, 0.28)][(0.48, 0.49), 0.51, (0.51, 0.51)][(0.07, 0.07), 0.09, (0.11, 0.12)][(0.19, 0.2), 0.22, (0.25, 0.26)][(0.05, 0.05), 0.06, (0.07, 0.08)]
C44[(0.14, 0.14), 0.16, (0.19, 0.2)][(0.07, 0.07), 0.07, (0.09, 0.1)][(0.44, 0.46), 0.48, (0.5, 0.51)][(0.16, 0.17), 0.18, (0.2, 0.21)][(0.04, 0.04), 0.05, (0.06, 0.07)]
Internet of things challengesI1[(0.09, 0.1), 0.12, (0.15, 0.17)][(0.07, 0.08), 0.09, (0.12, 0.13)][(0.13, 0.14), 0.15, (0.17, 0.19)][(0.09, 0.1), 0.12, (0.15, 0.16)][(0.09, 0.11), 0.15, (0.22, 0.26)]
I2[(0.31, 0.33), 0.36, (0.39, 0.41)][(0.06, 0.06), 0.08, (0.1, 0.12)][(0.06, 0.07), 0.08, (0.09, 0.1)][(0.1, 0.11), 0.13, (0.16, 0.17)][(0.1, 0.12), 0.17, (0.23, 0.27)]
I3[(0.12, 0.13), 0.16, (0.21, 0.23)][(0.21, 0.22), 0.24, (0.26, 0.27)][(0.42, 0.43), 0.46, (0.48, 0.48)][(0.22, 0.23), 0.26, (0.29, 0.31)][(0.22, 0.25), 0.33, (0.43, 0.5)]
I4[(0.22, 0.24), 0.28, (0.33, 0.36)][(0.07, 0.08), 0.09, (0.12, 0.13)][(0.23, 0.24), 0.27, (0.3, 0.31)][(0.15, 0.16), 0.19, (0.22, 0.24)][(0.15, 0.18), 0.24, (0.33, 0.39)]
I5[(0.05, 0.06), 0.08, (0.12, 0.15)][(0.12, 0.12), 0.14, (0.16, 0.17)][(0.04, 0.04), 0.05, (0.07, 0.08)][(0.06, 0.07), 0.08, (0.11, 0.12)][(0.06, 0.07), 0.11, (0.16, 0.2)]
I51[(0.18, 0.19), 0.21, (0.24, 0.26)][(0.27, 0.28), 0.29, (0.31, 0.32)][(0.49, 0.5), 0.52, (0.54, 0.55)][(0.28, 0.29), 0.31, (0.34, 0.36)][(0.05, 0.06), 0.07, (0.09, 0.1)]
I52[(0.25, 0.26), 0.27, (0.29, 0.3)][(0.15, 0.15), 0.17, (0.19, 0.2)][(0.14, 0.15), 0.18, (0.21, 0.23)][(0.17, 0.18), 0.2, (0.23, 0.24)][(0.03, 0.04), 0.05, (0.06, 0.07)]
I53[(0.48, 0.5), 0.52, (0.53, 0.53)][(0.52, 0.53), 0.54, (0.55, 0.55)][(0.26, 0.28), 0.3, (0.34, 0.35)][(0.4, 0.42), 0.44, (0.46, 0.47)][(0.08, 0.08), 0.1, (0.12, 0.13)]

Table 9 presents the weights of criteria after the defuzzification process, which is based on equation (13). As shown in Table 9, economic (C1) and business (I3) were determined as the most important main criteria for the SD and IoT development challenges criteria, respectively. In addition, the most important sub-criteria can be considered, respectively, as follows: Financial measures and profitability (C21), productivity (C23), and contribution to regional development (C33) for SD criteria and trust (I53), transparency (I21), and education and training (I51) for IoT development challenges criteria.

Table 9

The final weights of SD and IoT development challenges criteria based on experts’ opinion.

SD CriteriaGlobal weightsIoT development challenges criteriaGlobal weights
Environmental (C1)0.3525Technological (I1)0.1655
Global warming (C11)0.0287Architecture and design (I11)0.038
Environmental pollutions (C12)0.0747Interoperability (I12)0.0314
Rational use of natural resources (C13)0.0925Lack of supporting infrastructure (I13)0.038
Quality of products (C14)0.0591Data processing power (I14)0.0417
Economic (C2)0.401Choice of technology (I15)0.0422
Financial measures and profitability (C21)0.1122Privacy and security (I2)0.1785
Risk management (C22)0.0527Transparency (I21)0.0934
Productivity (C23)0.1016IoT devices’ safety (I22)0.0701
Social (C3)0.1427Conflict of interests (I23)0.0616
Health and safety and welfare and quality of life (C31)0.0963Business (I3)0.346
Reducing farmers migration (C32)0.0611Business model (I31)0.0439
Contribution to regional development (C33)0.0993Investing in IoT development (I32)0.0414
Technical (C4)0.1321Economic development opportunities and issues (I33)0.0309
Development potential (C41)0.0893Customer expectations and quality of service (I34)0.0294
Eco-friendly technologies (C42)0.0395Heterogeneity of the sector (I35)0.0166
Modern agricultural technologies for yield increase (C43)0.0614Farm sizes and capital investment costs (I36)0.0261
Technology maturity (44)0.0507Legal and regulatory (I4)0.2577
Data usage (I41)0.0549
Cross border data flows and global cooperation (I42)0.0288
Liability (I43)0.0509
Data ownership and data collection management (I44)0.0472
GDPR and IoT (I45)0.0334
Cultural (I5)0.1197
Education and training (I51)0.0738
Ethics (I52)0.0479
Trust (I53)0.1011

3.3Results of the IVTFN ARAS Method

In this section, the final weight obtained from the SWARA method (Table 8) are used as inputs to the IVTFN ARAS technique. The results of the IVTFN ARAS method to rate the alternatives based on the two dimensions of SD and the challenges of IoT development. The following describes the calculation details of each step.

The decision matrix and the optimal performance rating of each criterion are calculated based on the equation (14) and using the final weights of the two dimensions of SD and IoT development challenges and the expert opinions (see Appendix A). Decision-makers after creating the integrated decision matrix for the two dimensions of SD and the challenges of IoT development calculated the normalised decision matrix and the weighted normalised decision matrix using equations (22) and (23) (see Appendix A).

After calculating the overall performance index for each alternative (Si) based on equation (25) presented in the second column of Tables 10 (SD Dimension) and 11 (IoT development challenges dimension), the process of defuzzification based on different values of y was performed using equation (26). Finally, decision-makers used equation (27) to calculate the utility level of each alternative.

Table 10

Final ranking for different values of y (SD criteria).

A[(0.0144, 0.0185), 0.0303, (0.0482, 0.0613)]0.0212100.0203100.019410
A1[(0.008, 0.0105), 0.0177, (0.0326, 0.0436)]0.01390.653560.01300.641260.01220.62786
A2[(0.0102, 0.0126), 0.0191, (0.0353, 0.0469)]0.01520.719320.01430.705320.01340.69013
A3[(0.0091, 0.0115), 0.0181, (0.0333, 0.0443)]0.01430.675350.01340.662350.01260.64825
A4[(0.0095, 0.0121), 0.0198, (0.0352, 0.0464)]0.01510.714330.01430.703730.01340.69212
A5[(0.0073, 0.0093), 0.0148, (0.0284, 0.0385)]0.01210.572570.01130.557970.01050.54197
A6[(0.0073, 0.0093), 0.0146, (0.0284, 0.0385)]0.01210.570180.01130.555480.01050.53928
A7[(0.0107, 0.0141), 0.0235, (0.041, 0.0535)]0.01760.828210.01660.819810.01570.81061
A8[(0.009, 0.0115), 0.0186, (0.0333, 0.0443)]0.01440.677840.01350.666240.01270.65354
Table 11

Final ranking for different values of y (IoT development challenges criteria).

A[(0.011, 0.0148), 0.027, (0.0504, 0.0696)]0.0215100.02100.018410
A1[(0.014, 0.0188), 0.0264, (0.0346, 0.0685)]0.02181.012820.01890.944920.0160.86582
A2[(0.0102, 0.0138), 0.0188, (0.0361, 0.0671)]0.01920.893860.01650.825150.01370.74514
A3[(0.0103, 0.0138), 0.0184, (0.0347, 0.0658)]0.01890.878680.01610.807880.01340.72528
A4[(0.0123, 0.0157), 0.0205, (0.0312, 0.0648)]0.01950.907630.01650.826140.01350.73116
A5[(0.0122, 0.0157), 0.0176, (0.0336, 0.0654)]0.01910.885770.01620.811870.01340.72557
A6[(0.0123, 0.0156), 0.0193, (0.033, 0.0648)]0.01930.896350.01640.82260.01360.73535
A7[(0.0137, 0.0166), 0.0208, (0.0327, 0.0624)]0.01940.900740.01670.835630.0140.75963
A8[(0.0181, 0.0232), 0.0294, (0.0339, 0.0705)]0.02361.097710.02051.024110.01730.93831

3.4Results of the Evaluation Framework

Given the underdeveloped nature of IoT and its challenges and the necessity of choosing solutions that are more attractive from an SD perspective, in this study, a framework based on MADM methods is presented to select the best areas for implementing IoT solutions in the agriculture sector taking into account the two dimensions of SD goals and the challenges of IoT development. Figure 6 outlines this conceptual framework used to analyse better the different areas of the agriculture sector based on a portfolio approach. Table 12 shows the results of ratings based on the two dimensions of SD and IoT challenges that are as inputs to the evaluation framework. Figure 6 illustrates the results of the implementation of the evaluation framework inspired by the portfolio management approach after normalising the ratings. As this figure shows; A8 and A1 areas, respectively, play an essential role in achieving SD goals. Distribution and supply networks (A8) play a crucial role in efficiently managing agricultural products so that farmers can gain profit (Elijah et al.2018; Nukala et al.2016). Since Iran today faces pressing problems such as water crisis and soil poverty, the implementation and development of IoT solutions in A8 and A1 can largely overcome issues in the agriculture sector. According to experts, these areas also face fewer challenges in terms of implementing IoT applications. Therefore, policy-makers should consider these areas as top priorities in implementing IoT-based solutions.

Like the A2, A4 and A7 areas face more challenges, with appropriate risk management; these areas could be considered as other priorities in implementing IoT solutions.

Finally, A3, A5 and A6 should be considered as the areas with the least priority for implementing IoT solutions.

Table 12

Calculations related to the normalisation of ratings into the interval [0,1].

CodeKi based on IVTFN SWARA and IVTFN ARAS methodNormalising the Ki to the [0,1] Interval
SD   IoT ChallengesSDIoT Challenges
A10.6278  0.86580.32640.6598
A20.6901  0.74510.55610.0934
A30.6482  0.72520.40160
A40.6921  0.73110.56350.0277
A50.5419  0.72550.00990.0014
A60.5392  0.735300.0474
A70.8106  0.759610.1614
A80.6535  0.93830.42131
Fig. 6

Rating agricultural areas for implementation of IoT applications based on the evaluation framework.

Rating agricultural areas for implementation of IoT applications based on the evaluation framework.


Today, the IoT is capable of empowering the agriculture sector to overcome challenges such as food security, food safety, overuse of the environment and its detrimental effects, supply chain inefficiency, and the loss of agricultural products during transport by providing solutions consistent with the goals of SD. Policy-makers always need to pay attention to various factors and criteria when selecting to implement the right decision. Some of the elements are inconsistent, so decision-makers evaluating them face inherent uncertainties, and in most cases, due to the lack of sufficient information, rely on expert opinions. In this context, policy-making, especially for new technologies, requires a coherent and scientific framework. Therefore, in this research, to support PMs for the implementation of IoT applications to achieve sustainable agriculture, the MADM framework is proposed to rate the IoT application in each agricultural areas based on two dimensions of SD and the challenges of implementing IoT. Besides, an evaluation framework finalises the evaluation of alternatives based on these two dimensions. For this purpose, the criteria and sub-criteria were extracted from the literature and experts selected them to solve FDM problem. In the next step, the developed IVTFN SWARA method and IVTFN ARAS method weighted the criteria and rated the alternatives. Later, the rating results based on two dimensions of SD (attractiveness) and challenges of IoT development were as inputs of a portfolio approach. Finally, PMs prioritised the alternatives.

Furthermore, an empirical case elicited from the agriculture sector in Iran is used to demonstrate the procedures of this framework. According to the experts, in terms of SD, economic criteria, and profitability criteria were selected as the most important criteria and sub-criteria, respectively. In terms of IoT development challenges, the business model and transparency criteria were chosen as the most important criteria and sub-criteria, respectively. Finally, according to the evaluation framework, the areas of forestry, greenhouse, open-air horticulture, hydroponics and aquaponics, supply and distribution networks, open-field agriculture, livestock farming, and fishery and aquaculture were respectively rated. Based on the results of the evaluation, distribution and supply networks and open-field agriculture are the most critical areas for the IoT challenges, followed by greenhouses, open-air horticulture and forestry. Accordingly, hydroponics and aquaponics, livestock farming and fishery and aquaculture areas have the lowest priority, respectively.

The proposed framework has several innovations. First, using Meta-Synthesis Method, new classifications for different IoT applications in agriculture are introduced. Second, using literature review, challenges of the Internet of Things and its benefits from a sustainable development perspective are identified and categorized. Third, the proposed framework uses the new extension IVTFN SWARA method to obtain the weight of criteria. Fourth, using proposed decision-making matrix, decision-makers will be able to select high-priority IoT applications in agriculture. Finally, the proposed framework can be useful for national policy-makers to raise awareness and, therefore, implement the applications of IoT in the field of agriculture.

One of the limitations of this study was the overlap of different agricultural areas, which prevented the accurate evaluation of the IoT solutions. Besides, due to the specialised nature of IoT applications in the agricultural sector, some experts are not able to evaluate all alternatives accurately and, therefore, were excluded from the evaluation process. However, the use of hesitant sets could be considered by researchers in future research to overcome the uncertainty of decisions. Besides, due to the multitude of criteria and sub-criteria related to the two dimensions of SD and the challenges of IoT development, it was not possible to examine the relationships between the criteria and sub-criteria. In future research, the authors suggest to consider the interactions between criteria and weight the criteria using methods such as ANP, DANP, or ISM. The proposed framework also has the potential for other sectors.



AInitial Data for Evaluation and Selection of IoT Applications

Table 13

Decision matrix of experts’ opinions based on sustainable development criteria.

Table 14

Decision matrix of experts’ opinions based on the IoT development challenges criteria.

Table 15

Aggregated decision matrix based on sustainable development criteria.

Table 16

Aggregated decision matrix based on IoT development challenges criteria.

Table 17

Weighted normalised decision matrix based on sustainable development criteria.

[(0.26, 0.28), 0.35, (0.42, 0.46)][(0.08, 0.09), 0.12, (0.17, 0.2)]
Table 18

Weighted normalised decision matrix based on IoT development challenges criteria.

[(0.09, 0.11), 0.15, (0.22, 0.26)][(0.06, 0.07), 0.11, (0.16, 0.2)]



Akinyoade, A.J., Eluwole, O.T. (2019). The internet of things: Definition, tactile-oriented vision, challenges and future research directions. In: Third International Congress on Information and Communication Technology. Advances in Intelligent Systems and Computing, Vol. 797. Springer Verlag, Singapore, pp. 639–653.


Al-Fuqaha, A., Guizani, M., Mohammadi, M., Aledhari, M., Ayyash, M. (2015). Internet of things: a survey on enabling technologies, protocols, and applications. IEEE Communications Surveys and Tutorials, 17(4), 2347–2376.


Aldowah, H., Ul Rehman, S., Umar, I. (2019). Security in internet of things: issues, challenges and solutions. In: Advances in Intelligent Systems and Computing, Vol. 843. Springer Verlag, pp. 396–405.


Aswale, P., Shukla, A., Bharati, P., Bharambe, S., Palve, S. (2019). An overview of internet of things: architecture, protocols and challenges. In: Smart Innovation, Systems and Technologies, Vol. 106. Springer Science and Business Media Deutschland GmbH, pp. 299–308.


Atanassov, K.T. (1986). Intuitionistic fuzzy sets. Fuzzy Sets and Systems, 20(1), 87–96.


Atanassov, K., Gargov, G. (1989). Interval valued intuitionistic fuzzy sets. Fuzzy Sets and Systems, 31(3), 343–349.


Atzori, L., Iera, A., Morabito, G. (2010). The Internet of things: a survey. Computer Networks, 54(15), 2787–2805.


Bagočius, V., Zavadskas, E.K., Turskis, Z. (2014). Multi-person selection of the best wind turbine based on the multi-criteria integrated additive-multiplicative utility function. Journal of Civil Engineering and Management, 20(4), 590–599.


Benkhelifa, E., Abdel-Maguid, M., Ewenike, S., Heatley, D. (2014). The Internet of things: the eco-system for sustainable growth. In: Proceedings of IEEE/ACS International Conference on Computer Systems and Applications, AICCSA, Doha, Qatar.


Biggs, P., Garrity, J., LaSalle, C., Polomska, A. (2016). Harnessing the Internet of Things for Global Development. Switzerland.


Brewster, C., Roussaki, I., Kalatzis, N., Doolin, K., Ellis, K. (2017). IoT in agriculture: designing a Europe-Wide large-scale pilot. IEEE Communications Magazine, 55(9), 26–33.


Cardoso, T.F., Watanabe, M.D.B., Souza, A., Chagas, M.F., Cavalett, O., Morais, E.R., Nogueira, L.A.H.M., Leal, R.L.V., Braunbeck, O.A., Cortez, L.A.B., Bonomi, A. (2018). Economic, environmental, and social impacts of different sugarcane production systems. Biofuels, Bioproducts and Biorefining, 12(1), 68–82.


Chiou, H.K., Tzeng, G.H., Cheng, D.C. (2005). Evaluating sustainable fishing development strategies using fuzzy MCDM approach. Omega, 33(3), 223–234.


Dahooie, J.H., Zavadskas, E.K., Abolhasani, M., Vanaki, A., Turskis, Z. (2018). A novel approach for evaluation of projects using an interval-valued fuzzy additive ratio assessment (ARAS) method: a case study of oil and gas well drilling projects. Symmetry, 10(2), 45.


Diwaker, C., Tomar, P., Sharma, A. (2019). Future aspects and challenges of the internet of things for the smart generation. In: Lecture Notes in Electrical Engineering, Vol. 500. Springer-Verlag, pp. 599–606.


Elijah, O., Rahman, T.A., Orikumhi, I., Leow, C.Y., Hindia, M.N. (2018). An overview of Internet of Things (IoT) and data analytics in agriculture: benefits and challenges. IEEE Internet of Things Journal, 5(5), 3758–3773.


Erdogan, S.A., Šaparauskas, J., Turskis, Z. (2017). Decision making in construction management: AHP and expert choice approach. Procedia Engineering, 172, 270–276.


Fatemi, M., Rezaei-Moghaddam, K. (2019). Multi-criteria evaluation in paradigmatic perspectives of agricultural environmental management. Heliyon, 5(2), e01229.


Geng, H. (2017). Internet of things and data analytics handbook. Internet of Things and Data Analytics Handbook.


Hashemkhani Zolfani, S., Zavadskas, E.K., Turskis, Z. (2013). Design of products with both International and Local perspectives based on Yin-Yang balance theory and SWARA method. Economic Research-Ekonomska Istraživanja, 26(2), 153–166.


Herrera, F., Martínez, L. (2000). A 2-tuple fuzzy linguistic representation model for computing with words. IEEE Transactions on Fuzzy Systems, 8(6), 746–752.


Hopwood, B., Mellor, M., O’Brien, G. (2005). Sustainable development: mapping different approaches. Sustainable Development, 13(1), 38–52.


Instituto de Normalización, Acreditación, (2018). Internet of things (IoT) technology, economic view and technical standardization. Luxembourg. Retrieved from.


Jolly, D. (2003). The issue of weightings in technology portfolio management. Technovation, 23(5), 383–391.


Kamienski, C., Kleinschmidt, J., Soininen, J.-P., Kolehmainen, K., Roffia, L., Visoli, M., Maia, R.F., Fernandes, S. (2018). SWAMP: smart water management platform overview and security challenges. In: Proceedings – 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops, DSN-W 2018, Luxembourg City, Luxembourg.


Karabasevic, D., Zavadskas, E.K., Turskis, Z., Stanujkic, D. (2016). The framework for the selection of personnel based on the SWARA and ARAS methods under uncertainties. Informatica, 27(1), 49–65.


Keršulienė, V., Turskis, Z. (2011). Integrated fuzzy multiple criteria decision making model for architect selection. Technological and Economic Development of Economy, 17(4), 645–666.


Keršulienė, V., Turskis, Z. (2014). A hybrid linguistic fuzzy multiple criteria group selection of a chief accounting officer. Journal of Business Economics and Management, 15(2), 232–252.


Keršuliene, V., Zavadskas, E.K., Turskis, Z. (2010). Selection of rational dispute resolution method by applying new step-wise weight assessment ratio analysis (SWARA). Journal of Business Economics and Management, 11(2), 243–258.


Khan, S.F., Ismail, M.Y. (2017). An investigation into the challenges and opportunities associated with the application of Internet of Things (IoT) in the agricultural sector-A review. Journal of Computer Science, 14(2), 132–143.


Khishtandar, S., Zandieh, M., Dorri, B. (2017). A multi criteria decision making framework for sustainability assessment of bioenergy production technologies with hesitant fuzzy linguistic term sets: the case of Iran. Renewable and Sustainable Energy Reviews, 77, 1130–1145.


Kim, S., Kim, S. (2016). A multi-criteria approach toward discovering killer IoT application in Korea. Technological Forecasting and Social Change, 102, 143–155.


Kuo, Y.F., Chen, P.C. (2008). Constructing performance appraisal indicators for mobility of the service industries using fuzzy Delphi method. Expert Systems with Applications, 35(4), 1930–1939.


Lima Junior, F.R., Osiro, L., Carpinetti, L.C.R. (2014). A comparison between fuzzy AHP and fuzzy TOPSIS methods to supplier selection. Applied Soft Computing Journal, 21, 194–209.


Lin, C. (2013). Application of fuzzy Delphi method (FDM) and fuzzy analytic hierarchy process (FAHP) to criteria weights for fashion design scheme evaluation. International Journal of Clothing Science and Technology, 25(3), 171–183.


Liu, Y., Eckert, C., Yannou-Le Bris, G., Petit, G. (2019). A fuzzy decision tool to evaluate the sustainable performance of suppliers in an agrifood value chain. Computers and Industrial Engineering, 127, 196–212.


Luthra, S., Garg, D., Mangla, S.K., Singh Berwal, Y.P. (2018). Analyzing challenges to Internet of Things (IoT) adoption and diffusion: an Indian context. Procedia Computer Science, 125, 733–739.


Marcis, J., Bortoluzzi, S.C., de Lima, E.P., da Costa, S.E.G. (2019). Sustainability performance evaluation of agricultural cooperatives’ operations: a systemic review of the literature. Environment, Development and Sustainability, 21, 1111–1126.


Mardani, A., Jusoh, A., Halicka, K., Ejdys, J., Magruk, A., Ungku, U.N. (2018). Determining the utility in management by using multi-criteria decision support tools: a review. Economic Research-Ekonomska Istrazivanja, 31(1), 1666–1716.


Merad, M., Dechy, N., Serir, L., Grabisch, M., Marcel, F. (2013). Using a multi-criteria decision aid methodology to implement sustainable development principles within an organization. European Journal of Operational Research, 224(3), 603–613.


Miorandi, D., Sicari, S., De Pellegrini, F., Chlamtac, I. (2012). Internet of things: vision, applications and research challenges. Ad Hoc Networks, 10(7), 1497–1516.


Mohammadzadeh, A.K., Ghafoori, S., Mohammadian, A., Mohammadkazemi, R., Mahbanooei, B., Ghasemi, R. (2018). A Fuzzy Analytic Network Process (FANP) approach for prioritizing internet of things challenges in Iran. Technology in Society, 53, 124–134.


Mullen, P.M. (2003). Delphi: myths and reality. Journal of Health Organization and Management, 17(1), 37–52.


Noje, D., Dzitac, I., Pop, N., Tarca, R. (2020). IoT devices signals processing based on shepard local approximation operators defined in riesz MV-algebras. Informatica, 31(1), 131–142.


Nukala, R., Panduru, K., Shields, A., Riordan, D., Doody, P., Walsh, J. (2016). Internet of Things: a review from “Farm to Fork.”. In: 2016 27th Irish Signals and Systems Conference, ISSC 2016. IEEE, PP. 1–6.


Peppard, J., Ward, J. (2012). The Strategic Management of Information Systems: Building a Digital Strategy. Retrieved from, security in information system.


Perera, C., Zaslavsky, A., Christen, P., Georgakopoulos, D. (2014). Context aware computing for the internet of things: a survey. IEEE Communications Surveys and Tutorials, 16(1), 414–454.


Pivoto, D., Waquil, P.D., Talamini, E., Finocchio, C.P.S., Dalla Corte, V.F., de Vargas Mores, G. (2018). Scientific development of smart farming technologies and their application in Brazil. Information Processing in Agriculture, 5(1), 21–32.


Poursaeed, A., Mirdamadi, M., Malekmohammadi, I., Hosseini, J.F. (2010). The partnership models of agricultural sustainable development based on multiple criteria decision making (MCDM) in Iran. African Journal of Agricultural Research, 5(2), 160–164.


Quaddus, M.A., Siddique, M.A.B. (2001). Modelling sustainable development planning: a multicriteria decision conferencing approach. Environment International, 27(2–3), 89–95.


Ramakrishnan, M. (2008). IT portfolio management: a pragmatic approach to implement IT governance. In: Information Technology Governance and Service Management: Frameworks and Adaptations, pp. 297–312.


Ray, P.P. (2017). Internet of things for smart agriculture: technologies, practices and future direction. Journal of Ambient Intelligence and Smart Environments, 9(4), 395–420.


Reza Ghaeli, M., Vavrik, J., Nasvadi, G. (2003). Multicriteria project portfolio selection: case study for intelligent transportation systems. In: Transportation Research Record, pp. 125–131.


Rezaei-Moghaddam, K., Karami, E. (2008). A multiple criteria evaluation of sustainable agricultural development models using AHP. Environment, Development and Sustainability, 10, 407–426.


Ruzgys, A., Volvačiovas, R., Ignatavičius, Č., Turskis, Z. (2014). Integrated evaluation of external wall insulation in residential buildings using SWARA-TODIM MCDM method. Journal of Civil Engineering and Management, 20(1), 103–110.


Saaty, T.L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15, 234–281.


Saaty, T.L. (1980). The Analytical Hierarchy Process. McGraw-Hill, New York.


Sandelowski, M., Barroso, J. (2006). Handbook for Synthesizing Qualitative Research. Springer Publishing Company, Inc., New York.


Scharfy, D., Boccali, N., Stucki, M. (2017). Clean technologies in agriculture-How to prioritise measures? Sustainability, 9(8), 1303.


Shen, K.Y., Tzeng, G.H. (2018). Advances in multiple criteria decision making for sustainability: modeling and applications. Sustainability, 10(5), 1600.


Silvius, A.J.G. (2010). The business value of IT: a conceptual model for understanding and selecting valuation methods. In: Enterprise IT Governance, Business Value and Performance Measurement, pp. 102–111.


Stanujkic, D. (2015). Extension of the ARAS method for decision-making problems with interval-valued triangular fuzzy numbers. Informatica, 26(2), 335–355.


Stanujkic, D. (2016). An extension of the ratio system approach of MOORA method for group decision-making based on interval-valued triangular fuzzy numbers. Technological and Economic Development of Economy, 22(1), 122–141.


Stanujkic, D., Karabasevic, D., Zavadskas, E.K., Brauers, W.K.M. (2015). An extension of the MULTIMOORA method for solving complex decision-making problems based on the use of interval-valued triangular fuzzy numbers. Transformations in Business and Economics, 14(2B), 355–375.


Stanujkic, D., Zavadskas, E.K., Karabasevic, D., Turskis, Z., Keršulienė, V. (2017). New group decision-making ARCAS approach based on the integration of the SWARA and the ARAS methods adapted for negotiations. Journal of Business Economics and Management, 18(4), 599–618.


Sundmaeker, H., Verdouw, C., Wolfert, S., Freire, L.P. (2016). Internet of Food and Farm 2020. In: Digitising the Industry: Internet of Things Connecting the Physical, Digital and Virtual Worlds. River Publisher, pp. 129–151.


Turskis, Z., Lazauskas, M., Zavadskas, E.K. (2012). Fuzzy multiple criteria assessment of construction site alternatives for non-hazardous waste incineration plant in Vilnius city, applying ARAS-F and AHP methods. Journal of Environmental Engineering and Landscape Management, 20(2), 110–120.


Turskis, Z., Dzitac, S., Stankiuvienė, A., Šukys, R. (2019a). A fuzzy group decision-making model for determining the most influential persons in the sustainable prevention of accidents in the construction SMEs. International Journal of Computers, Communications & Control, 14(1), 90–106.


Turskis, Z., Goranin, N., Nurusheva, A., Boranbayev, S. (2019b). Information security risk assessment in critical infrastructure: a hybrid MCDM approach. Informatica, 30(1), 187–211.


Turskis, Z., Zavadskas, E.K. (2010a). A new fuzzy additive ratio assessment method (ARAS-F). Case study: The analysis of fuzzy multiple criteria in order to select the logistic centers location. Transport, 25(4), 423–432.


Turskis, Z., Zavadskas, E.K. (2010b). A novel method for multiple criteria analysis: grey additive ratio assessment (ARAS-G) method. Informatica, 21(4), 597–610. Accession Number: WOS:000285982300008.


Upadhyay, K., Yadav, A.K., Gandhi, P. (2019). A review of internet of things from Indian perspective. In: Lecture Notes in Electrical Engineering, 478, 621–632.


Veisi, H., Liaghati, H., Alipour, A. (2016). Developing an ethics-based approach to indicators of sustainable agriculture using analytic hierarchy process (AHP). Ecological Indicators, 60, 644–654.


Velten, S., Leventon, J., Jager, N., Newig, J. (2015). What is sustainable agriculture? A systematic review. Sustainability, 7(6), 7833–7865.


Verdouw, C., Wolfert, J., Tekinerdogan, B. (2016). Internet of Things in agriculture. CAB Reviews: Perspectives in Agriculture, Veterinary Science, Nutrition and Natural Resources 11 12


Villa-Henriksen, A., Edwards, G.T.C., Pesonen, L.A., Green, O., Sørensen, C.A.G. (2020). Internet of things in arable farming: implementation, applications, challenges and potential. Biosystems Engineering, 191, 60–84.


Vinodh, S., Mulanjur, G., Thiagarajan, A. (2013). Sustainable concept selection using modified fuzzy TOPSIS: a case study. International Journal of Sustainable Engineering, 6(2), 109–116.


Wang, B., Song, J., Ren, J., Li, K., Duan, H., Wang, X. (2019a). Selecting sustainable energy conversion technologies for agricultural residues: a fuzzy AHP-VIKOR based prioritization from life cycle perspective. Resources, Conservation and Recycling, 142, 78–87.


Wang, J., Cao, C., Zeng, S., Balezentis, T. (2019b). Weighted induced aggregation euclidean distance operators for the decision making of robot selection1. Transformations in Business and Economics, 18(1), 81–94.


Yazdani, M., Wen, Z., Liao, H., Banaitis, A., Turskis, Z. (2019a). A grey combined compromise solution (CoCoSo-G) method for supplier selection in construction management. Journal of Civil Engineering and Management, 25(8), 858–874.


Yao, J.S., Lin, F.T. (2002). Constructing a fuzzy flow-shop sequencing model based on statistical data. International Journal of Approximate Reasoning, 29(3), 215–234.


Yazdani, M., Zarate, P., Zavadskas, E.K., Turskis, Z. (2019b). A Combined Compromise Solution (CoCoSo) method for multi-criteria decision-making problems. Management Decision, 57(9), 2501–2519.


Ye, F. (2010). An extended TOPSIS method with interval-valued intuitionistic fuzzy numbers for virtual enterprise partner selection. Expert Systems with Applications, 37(10), 7050–7055.


Zadeh, L.A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353.


Zadeh, L. (1975). The concept of a linguistic variable and its application to approximate reasoning-I. Information Sciences, 8(3), 199–249.


Zagorskas, J., Turskis, Z. (2020a). Location preferences of new pedestrian bridges based on Multi-Criteria Decision-Making and GIS-based estimation. The Baltic Journal of Road and Bridge Engineering, 15(2), 158–181.


Zagorskas, J., Turskis, Z. (2020b). Setting priority list for construction works of bicycle path segments based on Eckenrode rating and ARAS-F decision support method integrated in GIS. Transport, 35(2), 179–192.


Zahm, F., Viaux, P., Vilain, L., Girardin, P., Mouchet, C. (2008). Assessing farm sustainability with the IDEA method – from the concept of agriculture sustainability to case studies on farms. Sustainable Development, 16(4), 271–281.


Zarei, M., Mohammadian, A., Ghasemi, R. (2016). Internet of things in industries: a survey for sustainable development. International Journal of Innovation and Sustainable Development, 10(4), 419–442.


Zavadskas, E.K., Turskis, Z. (2010). A new additive ratio assessment (ARAS) method in multicriteria decision-making. Technological and Economic Development of Economy, 16(2), 159–172.


Zavadskas, E.K., Turskis, Z., Volvačiovas, R., Kildiene, S. (2013). Multi-criteria assessment model of technologies. Studies in Informatics and Control, 22(4), 249–258.


Zavadskas, E.K., Turskis, Z., Stević, Ž., Mardani, A. (2020). Modelling procedure for the selection of steel pipes supplier by applying fuzzy AHP method. Operational Research in Engineering Sciences: Theory and Applications, 3(2), 39–53.


Zeleny, M. (1973). Compromise programming. In: Cochrane, J.L., Zeleny, M. (Eds.), Multiple Criteria Decision Making. University of South Carolina Press, Columbia, pp. 262–301.


Zemlickienė, V., Turskis, Z. (2020). Evaluation of the expediency of technology commercialization: a case of information technology and biotechnology. Technological and Economic Development of Economy, 26(1), 271–289.


Zeng, S., Peng, X., Baležentis, T., Streimikiene, D. (2019). Prioritization of low-carbon suppliers based on Pythagorean fuzzy group decision making with self-confidence level. Economic Research-Ekonomska Istrazivanja, 32(1), 1073–1087.