Time-optimized sequential decision making for service management in smart city environments

2.1.Related work

The majority of the approaches for task offloading at the edge depends on the selection of whether or not the task should be processed locally or should be offloaded to peers or Cloud. The time in execution latency [26] and the amount of energy used should both be reduced as much as possible. These are the two primary goals. The work presented in [35] provides a framework for offloading computation workloads from a user device to a server hosted in EC. Our approach is different from that due to the fact that our time-optimized decision primarily considers the actual benefit of payoff which indicates the current users’ requests for specific service. In [35], the User Equipment (UE) choice on whether to offload computing tasks with the highest CPU availability for a certain application is driven by the radio network information service (RNIS), according to the current value of round trip time (RTT) between the UE and the edge node. In our method, the offloading choice ideally relies on the number of users’ requests for specific service as well as the cost of delay. Therefore, the decision of offloading is made when the dedicated time of the required service by users is finished which indicates the required service is not worth being hosted. As a result, this will optimize the edge users’ capacity.

The work in [9] presents a method for optimizing the decision-making process that an Unmanned Aerial Vehicle (UAV) uses to choose whether or not to do the compute task locally or to offload it to an edge server. The decision is made based on a series of interactions that take place between the UAV and IoT systems. During these interactions, the UAV receives feedback on the state of the network and EC server, which enables an estimation of the remaining amount of time needed to complete the task. As a result, the UAV uses this information to solve an optimization problem with the purpose of decreasing a weighted sum of delay and energy costs as much as possible. In our work, EC chooses the ideal moment that has a high payoff to perform a service replication (pull-action) from their edge neighbors or the cloud. The approach described in [1] analyses the present state of the node’s resources, which are modeled as a Markov Decision Process utilizing Q-learning for training, in order to identify which portion of the application ought to be offloaded and then moves on with an offloading decision. The primary objective is to reduce the amount of time that the offloaded applications are delayed while taking into account the mobile fog that is located nearby, the mobile fog that is located adjacent, or the Cloud as suitable offloading locations. In our work, we have examined the scenario in which the edge node only performs a service replication (pull-action) from its edge neighbors or the cloud since the edge node is static and makes this decision in a short delay as close to the real’s delay scenario.

The work in [17] focuses on reducing the processing delay of all activities and the energy usage of the edge device by optimizing the task allocation decision and the central processing unit (CPU) frequency of the edge device. However, this study focuses on making decisions based on the popularity of the required service to complete the task. The authors of [16,31] studied the offloading problem decision for a multi-edge user & multi-edge node. However, in our approach, we focus on making the decision of (pull or push-action) for a single user to edge node or peer since our approach OST has strong deals with low capacity better than the approach in [16,31]. Using a time-division multiple access protocol in [50], edge users can conduct their individual activities locally or offload all or a portion of them to the Access Point (AP). Their objective is to decrease the AP’s overall energy consumption relative to the user at the network’s edge. Nevertheless, our recommended solution is subject to a choice depending on the service’s popularity and our stopping time with the maximum payoff. The study presented in [12] addresses the issue of task offloading in ultra-dense networks with the goal of reducing latency as much as possible while preserving the battery life of user equipment. On the other hand, our method provides the optimum option in terms of (pull or push-action) the service (from or to) the edge’s neighbors or peers.

The work in [23,41] primarily formulate the decision-making of offloading based on a resource scheduling factor. However, in our approach, we develop the decision of (push or pull-action) w.r.t. accumulation of the newly received tasks and cost incurring. A state-action–payoff–state-action (RL-SARSA) method is introduced in [3] to tackle the resource management issue at the edge server and make the best offloading option for reducing system cost. Moreover, the work in [49] aims to make the offload decision based on minimizing the cost. However, our work emphasizes making the decision with the greatest payoff, considering the expense cost each time that we do not make the decision of (push-action). In the study presented in [10,48], the issue of offloading computation for many users in a wireless environment with uncertainty is investigated. Nevertheless, the goal of our work is to provide the single-edge user with the best possible time in which to make a decision (push or pull-action). In [32,55], the authors examine the challenge of partial offloading scheduling and resource allocation for edge computing systems with various independent activities. The objective is to reduce the weighted total of processing latency and energy usage while satisfying the tasks’ transmission power limitations. Nevertheless, our approach enables the edge user to choose the optimal (pull-action) solution. Since the service has previously been hosted, the wait for future tasks will be reduced. The work in [15,38] investigate energy-efficient task offloading in edge computing. The goal is to reduce the amount of energy required for task offloading. However, our method focuses on making the choice at the optimal moment to host the most popular tasks, hence optimizing the edge’s capacity resource utilization.

Migration of data and services may be used to fill ‘gaps’ in the local knowledge base of a processing node. In [54], the interested reader may obtain a survey of related initiatives. When migrating services, it will be efficient to meet processing demands locally; nonetheless, a list of factors must be considered. To minimize overburdening the network, service migration may be accomplished in phases; service components should be “hooked up” to the hosting infrastructure swiftly, easily, and automatically. One of the technologies used for these reasons is Machine Learning (ML). ML may serve as the foundation for producing models that predict the movement of users and provide a proactive procedure for services migration [8]. Reinforcement learning may improve agent behavior while determining the optimal action to take [2,52]. The difficulty in depending on supervised machine learning technology is identifying the appropriate representative training data set. As a result of the raised degree of uncertainty in the corresponding decision-making, it is challenging to gather data that includes all alternative node statuses. The service migration model may alternatively be expressed as an online queue stability issue [37]. The issue may be addressed using either a Lyapunov optimization or a multi-objective optimization scheme [47,56], and the Pareto optimum solution can be determined. In the past, efforts have mostly focused on minimizing delay within the restrictions of energy usage in EC settings. However, the bulk of research efforts that aim at a user-centric service migration model is ‘affected’ by the extra time required to address the optimization issue, except if a list of assumptions is made or approximate solutions can be accepted. Using Markov chains is also a possible solution to settle the issue. Using a modified policy-iteration algorithm, the authors of [53] provide a solution to a finite-state Markov decision process. A technique based on Thompson sampling is suggested in [36] to facilitate dynamic service migration choices. The authors of [46] describe a service allocation module for networked automobiles powered by Cloud-based services. The objective is to increase the amount of autonomy of automobiles capable of picking nearby automobiles to share adopted services. The authors of [26] suggest a proactive system for determining the effective administration of services and tasks that are present/reported at EC nodes. The suggested model monitors the demand for existing services and rationalizes their management, i.e., their local presence/invocation when the demand is modified by the desired processing operations. To proactively achieve the strategic objectives of the envisioned model, a statistical inference process is initiated for each incoming task.

2.2.Rationale & contribution

Figure 1 illustrates in a simplified way the problem of deciding whether to perform a service replication (pull-action) from their edge neighbors or the cloud or offload (push-action) it to the cloud service to make the necessary computation to complete the required task. Let us assume that the starting time of recording the number of requests for a task is t=0 and the end time period, hereinafter referred to as deadline is T>0, where a decision should be made. Figure 1 shows two decisions against time. Specifically, in Fig. 1 (a), we observe the scenario of making the pull-action decision at an early time t∗ (and before the deadline T) since we are expecting receiving more tasks requesting a specific service. This denotes that there is a relatively high demand for this service, thus, it is advisable to decide on pulling the service locally to the node. In Fig. 1 (b), we observe the scenario in which we delayed the decision making in light of increasing our confidence that there is no high demand of the service. Once the deadline elapses and we have not decided on a pull-action, which indicates that the service request rate was relatively low, then, the node decides on offloading the service requests (push-action) either to its peers or the Cloud. In other words, this implies that there was not a large volume of tasks requesting the specific service, thus, offloading these tasks is necessary. Assume the case where we made the decision to download/pull the service earlier enough, and then after a while, we realized that the service was requested only by a small number of users. We would have consumed additional cost of resources without benefiting from the download/pull service. As a result, it would have been more efficient to delay our decision until the end of the dedicated time and then perform the action of offloading as it is shown in Fig. 1 (b). On the contrary, if we had decided to send/offload the request to the Cloud at an early time, then, we would not have high confidence whether this request was highly requested on the node, thus, this decision would not be efficient. The reason is that this service would have been highly requested in the future, thus, the push-action would not be appropriate. However, if after a while we realize that the service demand has significantly increased, then an early push-action results in overloading the network with offloading tasks to other peers or the Cloud unnecessarily. Thus, a proper process is to sophisticatedly delay any decision and perform service replication (pull-action) from node’s neighbors or the cloud to the node at a specific time t∗ as shown in Fig. 1 (a). Hence, the problem has been identified as finding that time t∗ to ensure that our resources are utilized in the best way trading off between task offloading and service replication. A fundamental solution has been proposed, which ensures the appropriate decision is taken at the time t∗ taking into account the delay costs.

Fig. 1.

Observing the task requests rate at each time instance t to decide on (a) a high popularity or (b) low popularity of the service and act intelligently based on the history of the task requests.

Fig. 2.

Sequential decision-making diagram on either task offloading (push-action) or service replication (pull-action) at the best time t∗<T by the edge node.

In addition, Fig. 2 highlights the state-space of the decision-making. Specifically, Fig. 2 describes the mechanism of the seeking the best time for making the decision: either service replication (pull-action) or task offloading (push-action). Mainly, the decision is affected by the number of the received tasks at each time instance t for a specific service.

The rationale has as follows. The node receives a number of incoming task requests Yt at every time instance t<T. Then, the node has two options:

If the deadline T elapses, then the decision will be to push the task to a peer or the Cloud for further processing. This is the first study that we are aware of that sequential decision making, which considers the decision of pull/push-action as an OST problem in EC environments. The overall goal is to increase the possibility of making the decision of service replication (pull-action) with the greatest payoff. Moreover, since the nodes are located among the servers that have been distributed at the edge of the network, our approach will be applied to EC applications such as Virtual Networks (VN) [61], UAV [4], and data mining applications [30].

Fig. 3.

Example of the two actions: pull and push in a smart city environment.

3.1.System model

As seen in Fig. 3, a smart city is increasingly establishing vast IoT networks with widely distributed IoT devices that provide vast amounts of services. In light of the vast size and extensively spread nature of IoT networks, EC has become a strong and efficient paradigm for providing processing capabilities to IoT devices at the network’s edge [58]. In EC, IoT services may be executed on EC, which offers low delay and reduces the bandwidth load. Nevertheless, it remains difficult to enhance the entire EC execution performance, for instance, resource utilization, EC energy usage, and load balance. However, we believe that the environment of the network is made up of a global cloud data center and an EC system [14,33]. The EC system is constructed of M base stations as defined in Table 1, and since each one is supplied with some EC servers, they collectively constitute an EC node. In accordance with it as well, we will refer to the set of EC nodes as M=[1,2,…,M]. Our general assumption is that multiple service providers would deploy their own applications on each EC node. Each edge device has the ability to delegate tasks to an EC node that is equipped with appropriate computing services. In addition, edge devices have the capability of offloading tasks to a global cloud, presuming that the global cloud has all of the required services and a decent amount of computing resources [19]. Edge devices make the decision of task service replication (pull-action) in accordance with the maximum payoff value. The time is segmented into frames denoted by the notation t=1,2,…,T. A decision of service replication (pull-action) is determined by the edge device at some moment in time t∗, depending on local information (e.g., task information like the number of services that have been received). We rely on the assumption that the location of the edge device and the network environment remain the same throughout each time instance. This allows us to guarantee that the task of replication (pull-action) decision will be effective at the same time instance t.

3.2.Problem definition

Within the scope of the task model, we rely on the assumption that the formation of the user’s tasks follows a Poisson distribution with Yt: the number of the task’s requests at time instance t with the rate λ [44]. Due to the fact that the Edge device is constant, the number of EC nodes that the edge device is capable of sensing and receiving tasks during each time instance t. As a result, the cumulation of the number of requests up to t is applied to every task that are received by the edge device. The decision of service replication (pull-action) is subjected to θ which is a number of tasks that our OST model is based on it to take the action of making the decision. The cumulative distribution function of the poison distribution can be expressed as in (1).

The objective is to get as near as possible to θ. In the case that Rt>θ, the observation is immediately terminated and the service offloading scenario is activated, since we have acquired much more usable information than requested. Clearly, the second scenario is not unwanted, but it results in a ‘penalty’ that we ought to have prevented before Rt>θ. This penalty is measured by the formula Rt>θ. However, we may include a penalty for the increased latency caused by receiving one more task before stopping. Each time we do not stop, hence not triggering the service replication (pull-action) scenario, we incur a cost greater than zero. This cumulative cost up to t requires the procedure to decide whether to stop or continue with another observation by evaluating the trade-off between extending the observation for potentially more usable data and the extra latency required to activate the service offloading scenario. This price may reflect the urgency of a monitoring application that demands a service offloading option in near real-time. Alternatively, if the application requires very precise service offloading option outcomes, a (limited) delay must be permitted.

Based on the above interpretation, we provide the following payoff function Ht(Rt), which involves (i) the task threshold θ, (ii) the delay cost per monitoring C⩾0.

The rationale behind this payoff is that we desire the rate of service demands per time t taking out the delay cost to be significantly increasing. This expresses the likelihood that the pull-action is stochastically a better decision than the push-action. This is because we increase our confidence that in the (near) future we are expecting more requests for a service in a node. Therefore, we aim at maximizing this payoff and, hence, the corresponding likelihood. Our goal is to attain θ by maximizing the number of the received tasks. If Rt is greater than θ, then the return is zero since the mechanism should have triggered the service offloading scenario. Therefore, any additional delay was of no benefit.

Algorithm 1:

Heuristic decision-making model (DR).

It is mandatory to have a base result to be compared with our fundamental approach (OST). Therefore, we define the IDL, which measures the accumulative sum of the received tasks from users to the node. However, each time instance t the expected cost C·t must be in the account. As a result, the instance time t that has the maximum payoff that results from IDL, is the best time t∗ instance for making the decision (pull or continue). Moreover, we can express the IDL best time instance t∗ as (4):

We suggest the deterministic DR. DR is mainly based on calculating the number of received tasks Y after passing the amount of time t. In this rule, the decision of offloading is made if Rt of the received task at time instance t, which is determined by α·θC, is less than theta θ. Also, It is important to pay attention to the cost per each time instance (C·t). Then, the decision of service replication (pull-action) is expressed in DR as in (5). The pseudo-code is provided in Algorithm 1.

Moreover, our theory has been compared to the work in [5]. Their approach is based on choosing the stopping time using SECPRO model. The stopping time decision in [5] was based on choosing the highest value of the received requests per time instance:

As a result, we found surprising and noteworthy results that demonstrate the superiority of our theory OST. This is what we will explain later in Section 3.3.

Algorithm 2:

Secretary-based OST model (SECPRO).

3.3.Sequential decision making based on OST

3.4.Cost-based sequential decision making

The Problem 1 is resolved by establishing a model based on the OST. Consider that the starting time t=0 and the deadline is T, with t<T. Our OST model aims to stop receiving more tasks when the cumulative sum of the received task Rt is near to θ, taking into account the cumulative delay cost up to that point t. We provide a 1-sla stopping rule based on the optimal stopping principle stated in Theorem 1, whereby we stop at the first time t such that:

The 1-sla rule is optimal, since the corresponding difference is constantly non-increasing with Rt when Rt<θ. A high-level description of the provided Theorem 2 is that, it is always better to stop at the first time t rather than at t+1, because the current pay off at that t will be bigger than the pay off at the next time instance t+1, and any other time t+τ with τ>1. This is the principle of 1-sla rule, where our problem falls into this category. Moreover, the difference of the pay off at the optimal time t and t+1 is always positive. However, that difference tends to decrease with the time t. This indicates that at the very first time t where the pay off is bigger than the future expected pay off, then it is always beneficial to stop at that t and not at any other future time, since the difference of the consecutive pay off values is always decreasing. Therefore, the 1-sla rule in Theorem 1 is optimum, OST with fixed θ and taking into account the delay cost C may ensure that the anticipated cumulative sum of the received tasks based on the stopping criteria is as near as possible to θ, however, no other stopping rule can make such a claim. Based on the cost C, OST is adaptable for handling and controlling the latency and freshness of the service replication (pull-action) and offloading decision’s variables. OST is concerned with the delay cost of monitoring and assumes that all received tasks are current for the application that uses the service replication (pull-action) and offloading decision. In this case, the OST is forced to terminate receiving new tasks to prevent waiting for an extended period of time, particularly when λ is very small. Therefore a large number of number of tasks are necessary to sum up to θ. The pseudo-code is provided in Algorithm 3.

Algorithm 3:

Time-optimized sequential decision making (OST).

4.1.Experimental setup

In our experiments, we experiment with four different models: (i) The proposed OST model. (ii) The heuristic DR model, which depends on the heuristic factor α. In DR, the decision of pull-action is taken if the accumulation Rt at time instance t, determined by α·(θ/C), is greater than θ provided in (5). Otherwise, if the deadline elapses, the decision of offloading is made. (iii) The SECPRO approach introduced in [5]. SECPRO measures the current payoff when the decision has been made using the OST criterion in (7). (iv) The IDL is the real expected payoff (H) gained provided in (4). This is the ground truth used to compare all the methods.

In the performance evaluation, we investigate the performance of the models: OST, DR, and IDL. Also, we set the values of for each of λ, α and θ. We assigned λ∈{3,5,10} and θ∈{50,100,200}. Moreover, we performed the experiments into three parts which are divided as the first experiment with λ and θ set to 3 and 50, respectively. Then, the second experiment is set up with λ=5 and θ=100. Finally, λ=10 and θ=200 for the third experiment. Also, the cost C was given a range of values and set these values for the three experiments. In fact, C=(0.01,0.05,0.1,0.3,0.5,0.9)·λ are the cost values. In addition, The α was a range of numbers applied for all experiments in (0.01,0.05,0.07,0.09,0.1,0.3,0.45,0.6,0.9) as shown in Table 2. Moreover, in order to get a more precise result, we did each experiment 1000 times and then we calculate the mean values for all the results.

In our comparative assessment, we performed the experiment over all the models, i.e., OST, DR, IDL, and SECPRO. We set the values of λ, α and θ. We assigned λ∈{3,30}, α=0.1 and θ∈{50,100,150,200}, θ∈{600,1200,1800,2400}. Moreover, we performed the experiment in two stages. The fourth experiment, which is the first stage, was set up with λ=3, α=0.1, and θ∈{50,100,150,200}. Furthermore, the second stage, which is the fifth experiment, was set up with λ=30, α=0.1, and θ∈{600,1200,1800,2400} as shown in Table 2. We run the experiments 1000 times and took the average values for all the obtained results.

4.2.Performance evaluation

4.2.1.Experiment 1

The settings for the Experiment 1 are: λ=3 and θ=50 (low service demand rate). Here, we performed the experiment with the defined λ and θ. Our experiment is mainly affected by the cost C and α. The experiment measures the payoff and the time that decision has been made. Moreover, the experiment is performed over three models. The first model is IDL which is being chosen to be the baseline to be compared with the other two models. The second model is our fundamental approach which is OST. Finally, the third model is DR. The experiment test three different models which are the real Figure 4 shows the different effects of the cost all over the three models and obviously shows the result of payoff and how our approach is as close as IDL compared with DR.

Fig. 4.

(Experiment 1) The expected payoff vs delay cost with low service demand rate λ=3 and tolerance θ=50 for OST, DR and IDL models.

4.2.2.Experiment 2

The settings for the Experiment 2 are: λ=10 and θ=200 (medium service demand rate). Here, we performed the experiment with the defined λ and θ. Our experiment is mainly affected by the cost C and α. The experiment measures the payoff and the time that decision has been made. Moreover, the experiment is performed over three models. The first model is the IDL model which is being chosen to be the baseline to be compared with the other two models. The second model is our fundamental approach which is OST. Finally, the third model is the DR model. Figure 5 shows the different effects of the cost all over the three models and obviously shows the result of payoff and how our approach is as close as IDL compared with DR.

Fig. 5.

(Experiment 2) The expected payoff vs delay cost with medium service demand rate λ=10 and tolerance θ=200 for OST, DR and IDL models.

4.2.3.Experiment 3

The settings for the Experiment 3 are: λ=30 and θ=600 (high service demand rate). We performed the experiment with the defined λ and θ. Our experiment is mainly affected by the cost C and α. The experiment measures the payoff and the time that decision has been made. Moreover, the experiment is performed over three models. The first model is IDL which is being chosen to be the baseline to be compared with the other two models. The second model is our fundamental approach which is OST. Finally, the third model is DR. Figure 6 shows the different effects of the cost all over the three models and obviously shows the result of payoff and how our approach is as close as IDL compared with DR.

Fig. 6.

(Experiment 3) The expected payoff vs delay cost with high service demand rate λ=30 and tolerance θ=600 for OST, DR and IDL models.

Fig. 7.

Expected payoff vs tolerance θ for diverse delay cot values with low service demand rate λ=3 and tolerance θ∈{50,100,150,200} for DR, OST, IDL and SECPRO models.

4.3.Comparative assessment

The settings for the comparative assessment experiment 1 are: λ=3, θ∈{50,100,150,200} and C=(1%,10%,30%)·λ (low service demand rate). We performed the experiment with the defined λ and α=0.1. Our experiment is mainly affected by diffident values of θ and C. Furthermore, the experiment measures the payoff V∗ and the time t∗ that decision has been made. Moreover, the experiment was performed over four models. The first model is IDL which is being chosen to be the baseline. It used to be compared with the other three models. The second model is our fundamental approach which is OST. Then, the third model is DR. Finally, the fourth model is SECPRO. The experiment tested four different models. Figure 7 shows the different effects of θ over all four models. Moreover, Fig. 7 obviously shows the result of payoff and how our approach is as close as IDL compared with DR and SECPRO.

Fig. 8.

Expected payoff vs tolerance θ for diverse delay cost values with high service demand rate λ=30 and tolerance θ∈{600,1200,1800,2400} for DR, OST, IDL and SECPRO models.

The settings for the comparative assessment experiment 2 are: λ=30, θ∈{600,1200,1800,2400} and C=(1%,10%,30%)·λ (high service demand rate): Here, we performed the experiment with the defined λ and α=0.1. Our experiment is mainly affected by diffident values of θ and C. Furthermore, the experiment measures the payoff V∗ and the time t∗ that decision has been made. Moreover, the experiment was performed over four models. The first model is IDL which is being chosen to be the baseline. It used to be compared with the other three models. The second model is our fundamental approach which is OST. Then, the third model is DR. Finally, the fourth model is SECPRO. The experiment tested four different models. Figure 8 shows the different effects of θ over all four models. Moreover, Fig. 8 obviously shows the result of payoff and how our approach is as close as IDL compared with DR and SECPRO.

In order to test the effectiveness of our theory (OST) in the first three experiments, where its results were compared with the IDL and another rule is DR. They were all tested on different parameters, which start from normal requests to medium requests and end with dense requests. Moreover, the results of our approach (OST) are amazing, which prove the effectiveness of the theory. As shown in Figs 9, 10 and 11, it obvious that Fig. 9 represents λ=3 and θ=50, as well as Fig. 10 shows λ=10 and θ=200. Also, Fig. 11 shows λ=30 and θ=600. In fact, each Figure contains three graphs, which represent the cost versus payoff, the cost versus stopping time, and finally delay versus payoff.

Fig. 9.

Expected payoff vs cost and delay for OST, IDL (low service demand rate λ=3 and tolerance θ=50).

In Fig. 9, we see the results of our theory (OST), which are very close to IDL, and this is illustrated in the graphic that presents cost versus payoff. However, after increasing the cost to more than (0.5) cost in the figure that shows the cost versus the Stopping time, we find there is a small difference, and despite that, our approach obtains a delay time less than IDL in all results where the cost is (0.5) or less, and this is what makes it unique.

Fig. 10.

Expected payoff vs cost and delay for OST, IDL (medium service demand rate λ=10 and tolerance θ=200).

In Fig. 10, the receiving of requests were increased by making λ=10, where we notice in the graph that shown the cost versus the payoff is an almost perfect match between our theory (OST) and IDL. Also, the delay time difference decreased after more than (0.5) the cost, however, we obtained a perfect stopping time between our theory (OST) and IDL as shown in the graph that illustrates the cost versus the stopping time as well as the graph showing the delay time versus the payoff.

Fig. 11.

Expected payoff vs cost and delay for OST, IDL (high service demand rate λ=30 and tolerance θ=600).

Figure 11 shows the robustness and effectiveness of our theory (OST). It is shown clearly OST’s ability to deal with heavy demands where λ=30. The results were amazing even after increasing the cost by more than (0.5) and this is obvious in the graph that shows the cost versus the stopping time and the graph that shows the delay time against the payoff. As the delay in our theory (OST) is very close to the delay for IDL even after (0.5) cost. We also note in the graph that presents cost versus payoff, a great match between our theory (OST) and IDL.

Fig. 12.

Expected delay (time to decision) vs tolerance θ for diverse cost values (low service demand rate λ=3 and tolerance θ∈{50,100,150,200}.

Moreover, we tested the effectiveness of our theory (OST) in the fourth & fifth experiments, where its results were compared with the three methodologies which are IDL, DR, and SECPRO. They were all tested on two different parameters, which are low & high requests. Moreover, the results of our approach (OST) are brilliant, which proves the effectiveness of our theory OST. As shown in Figs 12 and 13, it is obvious that Fig. 12 represents λ=3, θ∈{50,100,150,200} and C=(1%,10%,30%)·λ, as well as Fig. 13 shows λ=30, θ∈{600,1200,1800,2400} and C=(1%,10%,30%)·λ. In fact, each figure contains three graphs, which represent the time of the decision being made t∗ versus payoff.

Fig. 13.

Expected delay (time to decision) vs tolerance θ for diverse cost values (high service demand rate λ=30 and tolerance θ∈{600,1200,1800,2400}.