© The Author(s), published by EDP Sciences, 2024

Nomenclature

P_use,t : System’s consumption capacity for renewable energy

P_sum,t : Total available power output of renewable energy generation

$P_{T,t}^{\mathrm{buy}}$ : Power procured from the system assigned to period T

$P_{T,t}^{a.+}$ : Upper spare capacity acquired in the auxiliary service market

$P_{T,t}^{a.-}$ : Lower spare capacity acquired in the auxiliary service market

$P_{T,t}^{\mathrm{sl}}$ : Load reduction

$P_{T,t}^{\mathrm{il}}$ : Power reduction

C₁ : Expense of buying power from the system

C₂/C₃ : Cost factor of upper/lower spare capacity purchased

C₄/C₅ : The cost factor of load/power reduction

$P_{k,t}^{\mathrm{il},\mathrm{rat}}$ : Standard consumption or benchmark demand of interruptible loads

$P_{k,t}^{\mathrm{sl}.\mathrm{rat}}$ : Power demand of sheddable loads

${\mathrm{\Gamma }}_k^I$ : Represents the customer’s innate perception of the potential of the DR program

$W_k^0$ : Availability payment

$W_k^{\mathrm{\prime }}$ : Utilization payment

$s_k^{\text{'}}$ : Current action

$s_k^{″}$ : Different strategy

1 Introduction

Demand Response (DR), known to be a prominent feature of the smart grid, presents utilities with a groundbreaking approach to regulating power system operations through the utilization of customers’ load demand flexibilities. According to the definition of reference [1], DR programs are designed to enable load-side customers to voluntarily modify their electricity use when the system is subjected to high market prices or emergencies. In this context, effective utilization of DR can sustain a crucial role in maintaining supply-demand equilibrium, which can impact the operational efficiency of upcoming power grids.

Among the numerous benefits of DR, a major aspect is its implication for renewable energy accommodation. In traditional power systems, planning or operation decisions mainly account for uncertainties stemming from the load side. In the future power system, renewable energy resources may be massively integrated with the global need to reduce carbon emissions [2]. However, renewable energy is intermittent in time and space [3]. It generates electricity intermittently and may account for a considerable percentage. As a consequence, the power system is likely to exhibit significant unpredictability on both the supply and demand sides [4]. Due to the above feature, it is challenging to fulfill efficient accommodation of renewable energies in the power system.

As a sort of market-based operation solution, incentive-based DR can encourage consumers to adjust power usage patterns, achieving comparable outcomes to supply-side resources [5]. As a virtual controllable asset, DR can be combined with a variety of power generation to efficiently mitigate the challenges posed by unpredictable renewable energy generation and load-demand disparities in power system operation [6], and improve the power grid’s capacity to accommodate renewable energy.

At present, the application of DR in the power grid has been studied at home and abroad. Salehimehr et al. [7] develop a master-slave game trading model between system operators and load aggregators for DR, along with an optimization approach for multi-energy trading among diverse participants is presented. Norouzi et al. [8] investigated different load characteristics and types of DR programs. Reference [9] discusses the effects of time-of-use pricing on DR scheduling in the context of thermal energy storage, and based on the simulation outcomes, it can be inferred that combining energy storage with a time-of-use tariff can result in increased DR advantages specifically during peak hours. The load recovery effect associated with DR has been studied in reference [10] and it can be seen that in certain situations, the advantages of DR in terms of reliability may be substantially nullified by load recovery. Singh and Kumar [11] examine the involvement of various load resources to assist photovoltaic (PV) operation. References [12–14] prove that DR can help promote the application of renewable energy, enhance grid stability and reliability while lowering electricity costs. At the same time, Zhou et al. have conducted research on the DR flexibility of building energy systems, respectively from the aspects of system flexibility [15], collaborative optimization [16], and integrated application of renewable energy [17], and used machine learning methods to describe the demand side prediction of building energy system [18].

As for how to evaluate the contribution of DR in promoting the application of renewable energy in smart grid, Mehdi and Babak [19] present a security-constrained unit commitment framework that evaluates the effect of DR on power supply reliability. In reference [20], the influence of DR on the reliability of distribution networks is evaluated through Monte Carlo simulation techniques. Li and Wang [21] verify the feasibility of DR to promote renewable energy utilization by establishing various DR models. Yang et al. [22] calculate the contribution of DR to wind power usage under different incentive mechanisms based on a quantitative evaluation model. Amirhos et al. [23] recommend a multidimensional evaluation model that depends on an artificial neural network to implement a DR program. Shankar and Maurya [24] also propose a set of comprehensive evaluation methods for the valuation of DR benefits. In addition, reference [25] proposes a P2P optimization design method to increase DR Participation and promote renewable energy consumption.

Although considerable research efforts have been made so far, there are still some scientific gaps existed, which mainly arise from three of the following aspects:

Firstly, in the previous researches, the uncertainty of DR was not adequately considered. They commonly assumed that the utility companies possessed full knowledge about the demand responsivity of all the customers in the system; and all the DR resources were always accessible whenever envoked during the operation. These presumptions are generally applicable when DR is executed through a direct-control program, where customers got no option other than deciding whether to comply with DR requests from the grid. However, in reality, these assumptions may not always hold. If the study focuses on the non-direct control scenario, the circumstances will be quite different. In this case, users’ responses to DR signals would be “voluntary,” leading to unpredictable DR capacity that may significantly deviate from initial utility forecasts [11]. Consequently, the previously adopted assumption may no longer be justified in the non-direct control scenario.

Secondly, in current literature, the dynamic nature associated with DR activities has been largely disregarded [26–31], which imposes a great hurdle for their usage in real-world applications. In fact, for long-term analysis of smart-grid, since the time being considered is typically quite long (can take years), customers may modify their approach regarding their involvement (contractual setting) in the DR program dynamically during operation. In this case, the actual capacity for DR from the demand side becomes unstable and varies with customer decisions (i.e., the willingness to participate in DR) over time. As such, neglecting the multi-time scale and dynamic feature in DR programs would lead to an unrealistic estimation on the benefits of DR program in the long run.

Finally, in the existing literatures, the currently adopted indicators for measuring the benefits of DR are relatively simplistic, which fail to fully capture the multidimensional value for DR implementation, especially concerning their implication for enhancing the utilization of renewable energy in smart grids.

To resolve the scientific gaps presented above, this study proposes a comprehensive DR modeling framework and applies it to the quantification of DR benefits in enhancing the utilization of renewable energy. As the novelty of this research work, this paper’s proposed methodology can account for the impacts of uncertainties arising from both technical and human-related aspects within DR programs. Additionally, this paper explicitly incorporates the ambiguities of demand-side behaviors and their possible dynamics at multiple time-scales. To quantify the effect of DR on renewable usages scientifically, this paper also develops a systematic evaluation index system and algorithms.

In summary, compared with the related research, this study fully considers the uncertainty in the process of DR Implementation, and combines multi-time scales and dynamic characteristics to build a multi-dimensional benefit evaluation system for DR, the main contributions of this research can be summarized as follows:

The paper comprehensively models the multi-dimensional uncertainties of DR from a demand-side viewpoint, incorporate concurrently, within a same framework, both contingent and long-lasting dynamic ambiguity in non-direct control-based DR programs. This is unlike previous research which has only focused on examining temporary operational fluctuations of DR. This approach enables a more precise and practical assessment of the environmental and economic benefits of DR in real-world applications.

To create the DR model, the research integrates the strengths of analytic and data-driven approaches, making it generalizable and able to evaluate the DR attributes of various customer groups without implying any preconceived speculations about their behavior patterns or preferences.

A comprehensive multi-attribute assessment framework is proposed to explore the economic and environmental impacts of DR implementation in smart distribution systems. This framework relies on two key evaluation metrics: renewable energy utilization ratio and system operational cost. It also incorporates a multi-time scale optimal scheduling model, covering both day-ahead and intra-day scheduling.

The paper is structure as follows: Section 2 offers a system overview, Section 3 defines the evaluation criteria, Section 4 elaborates on the proposed DR modeling framework, Section 5 describes the algorithm used for the assessment, Section 6 makes a case study, and Section 7 provides a summary of the findings, concluding the research.

The overall research roadmap of this paper is shown in Figure 1.

Fig. 1 Overall research roadmap of this paper.

2 System overview

This research is based on a smart distribution system comprising distributed renewable energy generation, distribution feeder lines, and responsive load demand from system customers. The utility company centrally manages all system modules.

In this study, it is assumed that each system customer is equipped with a smart load monitoring and management unit, allowing the system operator to remotely oversee and adjust their electricity consumption, such as the real-time switching of the customers’ electrical devices.

Practically, customer load demand can be classified into different groups based on various criteria [32]. For instance, residential, industrial, and commercial loads can be distinguished according to the customer sector. Alternatively, curtailable, time-shiftable, and inelastic loads can be identified based on the characteristics of demand-side resources. In a prospective smart distribution system integrating renewable sources and adaptable loads, a well-executed DR strategy can enhance renewable energy utilization and mitigate carbon emissions from the electricity sector, and improve system operations. For instance, during periods of low renewable energy output or high market prices, the system operator might reduce or delay demand from flexible loads to minimize energy procurement from the grid, reduce maintenance expenses, and mitigate carbon emissions. Conversely, when renewable energy output is high or market costs are low, the system operator may opt for DR to increase load demand, avoid curtailing renewable generation, and improve system economic performance.

In practical applications, DR can be acquired through price-based or incentive-based programs [33]. Price-based DR programs depend on time-variant pricing signals (such as time-of-use prices, real-time prices, etc.) to motivate customers to adjust their power demand pattern based on the current system’s operational status. For example, real-time prices represent the marginal cost of supplying electricity to users within a specific time period, factoring in operational and foundational investment costs. Time-of-use prices involve segmenting a day into various periods based on the system’s operational status, with each period charging electricity based on the average marginal cost of system operation. However, since these pricing signals are typically issued on an hourly basis, load demands are unable to closely track the stochastic power output of renewable energy generation and respond promptly. Therefore, price-based DR programs are less useful in addressing the challenge of renewable energy integration. Alternatively, incentive-based DR programs use a direct load control approach. In this approach, the system operator can directly monitor and manage electricity consumption on the demand side in compliance with the system’s need for immediate response and performance. As a result, customer loads can promptly respond to system signals, promoting efficient renewable energy use.

However, incentive-based DR programs require users to change their initial consumption behaviors unexpectedly. Therefore, the degree to which the load controls impact customers who participate in such DR programs becomes a critical factor. In this sense, the impact of demand-side behaviors must be appropriately accounted for when assessing the contribution of incentive-based DR to renewable energy utilization.

During operation, when there is a timing mismatch between renewable energy generation and load demand, the system operator could invoke the DR potential of customers to maintain the secure, economical, and efficient operation of the system. To ensure the maximization of system benefits, the operator must determine the most suitable strategy for deploying DR under different scenarios, by optimization techniques. Then, the obtained DR commands are delivered to the customers via smart load monitoring and management unit and the targeted loads will be curtailed or shifted accordingly. After each time a DR is performed, the system operator records corresponding information about the DR capacity that is realized and pays a reward to the enrolled customer according to their bilateral agreement.

3 Evaluation criteria

Since this paper is mainly focused on assessing the impacts of DR in smart distribution grids, here, two evaluation indices are adopted, namely, the renewable penetration ratio η and grid operation cost C, to evaluate the influence of DR implementation on renewable energy utilization and the economic performance of the system, respectively.

The renewable penetration ratio η is defined as the quotient of the capacity of renewable energy that is utilized by the maximum generation output available, as given below:$$\eta =\frac{{\sum }_{t=1}^T\mathrm{}{P}_{\mathrm{use},t}}{{\sum }_{t=1}^T\mathrm{}{P}_{\mathrm{sum},t}}\times 100\%$$(1)where P_use,t is the capacity of renewable energy consumed by the system in time-period t; P_sum,t represents the total available renewable energy power output in time period t (which depends on the meteorological conditions); T is the time horizon under study.

The system operating cost C consists of four items, which are the power purchase cost (in the day-ahead market), DR reward cost, loss-of-load cost, and the payment for purchasing ancillary services. The mathematical formulation is presented as follows:$$C=\sum _{T=1}^{365}\mathrm{}\left(\sum _{t=1}^{24}\mathrm{}{c}_1{P}_{T,t}^{\mathrm{buy}}+\sum _{t=1}^{24}\mathrm{}{c}_2{P}_{T,t}^{a.+}+\sum _{t=1}^{24}\mathrm{}{c}_3{P}_{T,t}^{a.-}+\sum _{t=1}^{24}\mathrm{}{c}_4{P}_{T,t}^{\mathrm{sl}}+\sum _{t=1}^{24}\mathrm{}{c}_5{P}_{T,t}^{\mathrm{il}}\right)$$(2)where $P_{T,t}^{\mathrm{buy}}$ is the power purchased from the system allocated to the period T at period t. To meet the active power demand under the negative new energy forecast deviation and disturbance accident, the power system must retain a certain number of operating reserve passengers in the day-ahead start-up mode and intra-day real-time generation planning. Scientific and reasonable setting of operating reserve capacity is the premise of ensuring the safe, the dependable and cost-effective operation of the power system. And when the upper reserve capacity is insufficient, it may lead to load cutting, and when the lower reserve capacity is inadequate, it may cause new energy abandonment such as wind abandonment and light abandonment. Therefore, it is necessary to set the spare capacity constraint. $P_{T,t}^{a.+}$ is the upper spare capacity purchased from the auxiliary service market for period T in a given time slot t during the day-ahead scheduling, $P_{T,t}^{a.-},P_{T,t}^{\mathrm{sl}}$ and $P_{T,t}^{\mathrm{il}}$ are load reduction and power reduction. c₁ is the cost factor of purchased power from the system, c₂ and c₃ are the cost factor of upper and lower spare capacity purchased, c₄ and c₅ the cost factor of load reduction and power reduction, all in $/Kw.

4 Model formulation for DR

4.1 Framework of DR modeling

To accurately estimate the DR, it is crucial to conduct a sophisticated analysis of the load characteristics and the associated uncertainties.

The two main categories of current methodologies used in DR modeling are analytical and data-driven [30]. The priori approach primarily analyzes the DR through the development of profit-oriented optimization models [26, 31]. Conversely, the data-driven methodology entails forecasting the DR capability through an analysis of historical customer behavior patterns [22, 34–36].

The most significant challenge posed by this scenario is the diversity in demand-side behavior. As the non-direct control scheme does not involve direct control, entities may exhibit diverse and potentially irrational behaviors in response to DR signals from the grid. However, it may be challenging for the system operator to obtain complete information regarding individual idiosyncrasies, such as their coherence and consumption preferences, etc. Furthermore, assuming such evidence is accessible, various unpredictable factors may impact the reliable responsiveness of users throughout the process. Due to these factors, system operators cannot adequately incorporate DR in practical settings using solely analytical methods.

In addition to the aforementioned challenges, in the context of long-term analysis, the dynamic nature of customer participation in DR programs results in potentially unstable available DR capacity within the system, but it may change as the demand-side participation progresses. According to the research conducted by [37], people’s willingness to take part in DR programs is not solely determined by their preferences but is also influenced by factors like their previous payouts from the program. Unfortunately, the majority of existing DR modeling techniques fall short of accurately capturing the interconnected and evolving character of consumer behavior.

Considering the challenges discussed above, this paper recommends a comprehensive modeling approach for DR that addresses the above-addressed issues. As depicted in Figure 2, the system is made up of several components, that model the various factors influencing DR at different time scales. The structural attributes of DR capabilities are analyzed in the demand analysis module, which provides a foundation for the overall research. Two distinct modules, contract selection and actual implementation assess the human-related factors of DR. The contract selection module models the decision-making process of individuals regarding DR subscription during the contract execution process, which typically encompasses an extended time frame. On the other hand, the actual implementation module evaluates the actual DR efficiency of customers during the immediate operating period, taking into account the restrictions imposed by strategic contract selection in the long run. The changing load profile brought on by DR under various operating conditions may be evaluated by integrating the outcomes from these components. The evaluation algorithm uses the data generated as the ultimate results of the framework to evaluate the DR.

Fig. 2 Modelling framework of DR.

Figure 3 illustrates the modeling of factors that exist on different time scales that have a significant impact on DR.

Fig. 3 Timescale for different modules.

4.2 Demand analysis module

When considered from the material perspective, DR capability is primarily determined by the functional attributes of local requirements. Based on their degree of adaptability, electric loads can be broadly categorized as critical loads, interruptible loads, and shiftable loads [37].

Critical loads must run continuously under all circumstances. These loads typically include essential household appliances such as refrigerators, cooking equipment, and lighting fixtures. Given that the use of critical loads is often tied to the daily survival of individuals, they are designed to operate independently without any reliance on external signals or factors.

When referring to loads, interruptible loads are those whose expenditures may be reduced during system crises. Common examples of interruptible loads include heating, ventilation, and air conditioning systems. In the case of interruptible loads, when a power attenuation is commenced during a given time t, the resulting changing demand can be calculated as $P_{k,t}^{\mathrm{il}}=P_{k,t}^{\mathrm{il},\mathrm{rat}}-P_{k,t}^{\mathrm{il},-}$,where $P_{k,t}^{\mathrm{il},\mathrm{rat}}$ represents the standard consumption or benchmark demand of interruptible loads during period t.

Shiftable loads, or controllable electrical loads, can have their demands modified within a defined timeframe while adhering to overall energy constraints. Sheddable loads include plug-in electric heaters and water heaters. If a DR event starts while sheddable loads are operating, any energy saved during the DR event by reducing demand must be repaid by the customer after the event, based on the terms of their agreement.

To capture the behavior of both long-running loads and the associated uncertainties of sheddable loads, a vendor-neutral model is employed. This method utilizes two straight lines, as depicted in Figure 4, to characterize the demand reductions and restorations of sheddable loads. The ascending line represents the load increase following the DR event, while the declining line simulates the return of usage to the pre-DR level while taking into account the long-running process’ waning influence [38].

Fig. 4 Describing the demand recovery for controllable loads.

Whenever a load reduction $P_{k,\mathrm{tt}}^{\mathrm{sl},-}$ occurs, such as in period t, the restored demand in succeeding period t′ ($P_{k,\mathrm{tt}}^{\mathrm{sl},+}$) can be calculated distinctly as:$$P_{k,\mathrm{tt}\text{'}}^{\mathrm{sl},+}=\{\begin{array}{c}{b}_{k,0}^{\mathrm{sl}}+{\varpi }_k^{\mathrm{up}}(t\text{'}-t-1),t\mathrm{\prime }\in \{t+1,\cdots ,t+{\delta }_k^{\mathrm{pk}}\}\\ b_{k,1}^{\mathrm{sl}}-{\varpi }_k^{\mathrm{dw}}(t\text{'}-t-{\delta }_k^{\mathrm{pk}}),t\mathrm{\prime }\in \{t+{\delta }_k^{\mathrm{pk}}+1,\cdots ,t+{\delta }_k\}\\ 0, \mathrm{else}\end{array}$$(3)where:$$b_{k,0}^{\mathrm{sl}}=\frac{\left[2P_{k,t}^{\mathrm{sl},-}+{\varpi }_k^{\mathrm{up}}({\delta }_k^{\mathrm{pk}}-1)({\delta }_k^{\mathrm{pk}}-2{\delta }_k)+{\varpi }_k^{\mathrm{dw}}({\delta }_k-{\delta }_k^{\mathrm{pk}}{)}^2\right]}{2{\delta }_k}$$(4) $$b_{k,1}^{\mathrm{sl}}=\frac{\left[2P_{k,t}^{\mathrm{sl},-}+{\varpi }_k^{\mathrm{up}}{\delta }_k^{\mathrm{pk}}({\delta }_k^{\mathrm{pk}}-1)+{\varpi }_k^{\mathrm{dw}}({\delta }_k-{\delta }_k^{\mathrm{pk}}{)}^2\right]}{2{\delta }_k}.$$(5)And ${\varpi }_k^{\mathrm{up}},{\varpi }_k^{\mathrm{dw}},{\delta }_k$ and ${\delta }_k^{\mathrm{pk}}$ are distinct characteristics that determine the form of the user’s long-running structure.

The implementation of long-running loads by customers are typically uncontrolled, leading to uncertainties in the parameters described by equation (3) from the system operator’s perspective. In practice, a load survey is often necessary to determine these parameter values, with statistical characteristics identified through long-term observation of customer load recovery behavior during DR events. For simplicity, this study assumes that ${\varpi }_k^{\mathrm{up}},{\varpi }_k^{\mathrm{dw}},{\delta }_k$, and ${\delta }_k^{\mathrm{pk}}$ are normally distributed variables, these variables can be deduced through random sampling in the calculation. The demand for sheddable loads during DR can be represented by a combination of power reduction and payback, resulting in the following expression:$$P_{k,t}^{\mathrm{sl}}=P_{k,t}^{\mathrm{sl},\mathrm{rat}}-P_{k,t}^{\mathrm{sl},-}+\sum _{t\text{'}=t-{\delta }_k}^{t-1}\mathrm{}{P}_{k,t\text{'}t}^{\mathrm{sl},+}$$(6)where $P_{k,t}^{\mathrm{sl},\mathrm{rat}}$ represents the power demand of sheddable loads at period t under typical circumstances (without DR).

4.3 Contract selection module

To represent demand-side actions during the contract selection phase, Ω_D is identified as the group of system users or clients. For every user k ∈ Ω_D, suppose that all their potential actions/decisions for contract selection belong to a finite set $S_k=s_k^1,s_k^2,\dots ,s_k^M$, where each element in S_k is a percentage metric that represents the relationship between a user’s contracted load reduction level and their overall DR capability which comprises $P_{k,t}^{\mathrm{il},\mathrm{rat}}+P_{k,t}^{\mathrm{sl},\mathrm{rat}}$. More, this paper defines a time-interval set T_cs = 1, 2, …, z, … to denote the contract terms or time horizons for making contract selection decisions. Therefore, each time-slot t in z ∈ T_cs can be represented as t_z = mod(t, z). Given the notations outlined above, the key issue in contract selection to tackle is determining the users’ responsibility for selecting various actions in S_k for each interval z ∈ T_cs.

To begin with, let’s examine the scenario where no a priori information is available. Under such circumstances, as individuals typically have limited knowledge they are enrolled in, their selections of contract selection needs to rely solely on their judgment, which can be instinctual and subject. As a result, we can describe the likelihood of selecting each strategy $s_K^{\text{'}}\in S_k$ through an empirical distribution ${\mathrm{\Gamma }}_k^I=x_{k,0}^I\left(s_K^{\text{'}}\right)$, wherein $x_{k,0}^I\left(s_K^{\text{'}}\right)$ represents the probability value associated with s′, and $\sum _{s_K^{\text{'}}\in S_k}\mathrm{}{x}_{K,0}^I\left(s_K^{\text{'}}\right)=1$ serves as a further specification or condition.

The variable ${\mathrm{\Gamma }}_k^I$ represents the customer’s innate perception of the potential of the DR program. Specifically, it reflects the customer’s inclination towards selecting different contract-level strategies based on their expectations of the rewards and benefits associated with DR participation. For instance, a customer with a favorable disposition towards DR may opt for high contract-level strategies, whereas those with reservations may prefer low-level strategies. It is important to mention that ${\mathrm{\Gamma }}_k^I$ is based on the customer’s inherent tendency and stays constant throughout the DR program, without changing over time. This property makes ${\mathrm{\Gamma }}_k^I$ a useful tool for modeling the customer’s decision-making behavior in DR programs.

While the aforementioned formulation serves as a viable method for modeling DR programs, it might not provide the precision necessary for conducting in-depth studies. This is because the model overlooks customers’ learning capabilities, which can significantly affect their long-term decision-making. In real-world scenarios, customers can potentially predict the profitableness of DR programs derived from their past experiences and incorporate this information into their forthcoming decisions. Consequently, the likelihood of a customer choosing a specific contract selection strategy may not stay consistent throughout time, but instead, transform and vary. As a result, the customers’ performance in the contract selection phase may not always conform to their innate pattern represented by ${\mathrm{\Gamma }}_k^I$.

Thus, we will now elaborate on how to expand the model to integrate the scenario of a priori information. To calculate the remuneration gained by a customer for their DR actions during a given interval z, a benefit function U_k: R₊ → R is utilized. The configuration of U_k typically takes into account both the capacity and energy contribution of the DR program [39]. It is worth noting that some DR programs may also impose penalty policies for customers who fail to meet their contractual obligations. In such cases, a penalty charge G_k will be incurred, and its value will increase with the degree of violation committed by the customer [7].

Conversely, reducing (or shifting) the load can also create inconvenience to consumers. In DR research, such inconvenience is commonly expressed using a disutility function L_k: R₊ → R₊, the input of this model is calculated in this way. Using the definitions provided above, the aggregate gain of the users (Wk) from participating in DR during interval z can be represented as follows:$$W_k(s_{k,z},P_{k,t_z}^{\mathrm{il},-},P_{k,t_z}^{\mathrm{sl},-})=W_k^0\left(s_{k,z}\right)+\sum _{t_z\in z}\mathrm{}{W}_k^{\mathrm{\prime }}(P_{k,t_z}^{\mathrm{il},-},P_{k,t_z}^{\mathrm{sl},-})$$(7) $$W_k^0\left(s_{k,z}\right)=U_k(s_{k,z},P_{k,t_z}^{\mathrm{il},-},P_{k,t_z}^{\mathrm{sl},-})-\sum _{t_z\in z}\mathrm{}{U}_k^{\mathrm{\prime }}(P_{k,t_z}^{\mathrm{il},-},P_{k,t_z}^{\mathrm{sl},-})$$(8) $$W_k^{\mathrm{\prime }}\left(P_{k,t_z}^{\mathrm{il},-},P_{k,t_z}^{\mathrm{sl},-}\right)=\wp \left(U_k^{\mathrm{\prime }},G_k,L_k^{\mathrm{il}},L_k^{\mathrm{sl}}\right).$$(9)

The total customer payoff from DR participation in interval z, as shown in equation (7), comprises two components: the availability payment $W_k^0$ and utilization payment $W_k^{\mathrm{\prime }}{W}_k^0$ is typically defined by the user’s contractual level during interval z, represented by their contract selection decisions s_k,z as represented by equation (8). Whereas, $W_k\mathrm{\prime }$ is determined by the users’ actual performance throughout the operation $P_{k,t_z}^{\mathrm{il},-}$ and $P_{k,t_z}^{\mathrm{sl},-}$ which is a function of $U_k^{\mathrm{\prime }},G_k,L_k^{\mathrm{il}}$, and $L_k^{\mathrm{sl}}$, as shown in equation (9).

Typically, customers on the demand side have limited access to system information beyond their past observations and experiences, in normal situations. As a result, their decision-making in the contract selection phase often follows a “reflex-response” paradigm in praxeology [39]. Specifically, for every interval, customers may transition from their existing actions to alternative actions if they are perceived to yield higher payoffs. The likelihood of choosing an alternative strategy increases with its perceived superiority in terms of expected utility.

In this study, a theory based on behavioral economics principles is introduced, the regret-matching mechanism, to properly model the “reflex-response” effect exhibited by customers on the demand side. The initial design of the regret-matching mechanism was aimed at resolving the issue of local decision-making in scenarios where the available information about the environment is incomplete [34]. The distributed nature of the regret-matching mechanism makes it a suitable approach for modeling users’ adaptive behaviors, such as those seen in the contract selection case explored here. Furthermore, the regret-matching mechanism relies on the premise of limited rationality in human decision-making, which enables it to model customer behavior more accurately in realistic settings. Based on these recognitions, this study develops the contract selection model with a regret-matching mechanism as follows.

For each interval z ∈ T_cs, the regret-matching mechanism defines the regret of a customer $M_{k,z}(s_k^{\mathrm{\prime }},s_k^{″})$ for not selecting a different strategy $s_k^{\mathrm{″}}\in s_k$ compared to the current action $s_k^{\mathrm{\prime }}$ as:$$M_{k,z}(s_k^{\mathrm{\prime }},s_k^{″})=\max \left\{E_{k,z}\right(s_k^{\mathrm{\prime }},s_k^{″}),0\}$$(10) $$E_{k,z}\left(s_k^{\mathrm{\prime }},s_k^{\mathrm{″}}\right)=\frac{1}{z}\left[\sum _{\tau \le z:s_{k,\tau }=s_k^{\mathrm{\prime }}}\mathrm{}{W}_k\left(s_{k,,\tau }^{\mathrm{″}}\right)-\sum _{\tau \le z:s_{k,\tau }=s_k^{\mathrm{\prime }}}\mathrm{}{W}_k\left(s_{k,\tau }^{\mathrm{\prime }}\right)\right].$$(11)

As observed, regret is determined by comparing a user’s average payoff for a particular action, $s_k^{\mathrm{\prime }}$, with the average payoff for a different action, $s_k^{\mathrm{″}}\in s_k$, in the history up to z. The entire payoff for consumers, normalized W_k(·) over all the relevant intervals z is known as the average payoff. If the payoff for the alternate strategy $s_k^{\mathrm{″}}$ is lower than that of $s_k^{\mathrm{\prime }}$, thus $M_{k,z}(s_k^{\mathrm{\prime }},s_k^{\mathrm{″}})$ is 0.

To quantify regret, the system operator must understand the user’s potential payoffs if they had taken different actions in the past, denoted as $W_k\left(s_{k,\tau }^{\mathrm{″}}\right)$ in equation (11). However, in practice, obtaining such information from users may not always be feasible, rendering equation (10) inapplicable.

To address this issue, a conversion of the formulation E_k,z is necessary. In the context of prolonged research, if the possibility of selecting the strategy $s_k^{\mathrm{″}}$ is a times of strategy $s_k^{\mathrm{\prime }}$, then based on the principle of the law of large numbers, the frequency of $s_k^{\mathrm{″}}$ happening will be a times higher than that of $s_k^{\mathrm{\prime }}$. Consequently, the estimated total payoff for intervals where $s_k^{\mathrm{″}}$ is chosen can be calculated by increasing the reward $s_k^{\mathrm{\prime }}$ by a factor of 1/a. The alternative method of the $s_k^{\mathrm{″}}$, denoted as the first expression in (11), can be rephrased as:$$\begin{array}{c}\sum _{\tau \le z:s_{k,\tau }=s_k^{\mathrm{\prime }}}\mathrm{}{W}_k\left(s_{k,\tau }^{\mathrm{″}}\right)=\sum _{\tau \le z:s_{k,\tau }=s_k^{\mathrm{″}}}\mathrm{}{x}_{k,\tau }\left(s_k^{\mathrm{\prime }}\right)·W_k\left(s_{k,\tau }^{\mathrm{\prime }}\right)/x_{k,\tau }\left(s_k^{\mathrm{″}}\right)\\ x_{k,\tau }\left(\cdot \right)\end{array}$$(12)where x_k,τ(s_k) indicates the likelihood of s_k to be selected at τ ≤ z. In practical applications, this system operator can easily calculate the value of x_k,τ(·) using past customer data, making equation (12) manageable for the system operator.

Using the refined regret measure in regret-matching mechanism, the tendency of customers k ∈ Ω_D towards their decisions in contract selection can be defined. Then the probability distribution ${\mathrm{\Gamma }}_{k,z+1}^{\mathrm{B}}$ represents the possibility of the user selecting policy $s_k^{\mathrm{″}}\in s_k$ at z + 1.$${\mathrm{\Gamma }}_{k,z+1}^{\mathrm{B}}=\{\begin{array}{c}{x}_{k,z+1}^B\left(s_k^{\mathrm{″}}\right)=\min \left\{\frac{1}{\gamma }{M}_{k,z}(s_k^{\mathrm{\prime }},s_k^{\mathrm{″}}),\varsigma \right\}\\ x_{k,z+1}^B\left(s_k^{\mathrm{\prime }}\right)=1-\sum _{s_k^{\mathrm{\prime }}\in S_k:s_k^{\mathrm{″}}\ne s_k^{\mathrm{\prime }}}\mathrm{}{x}_{k,z+1}^B\left(s_k^{\mathrm{″}}\right)\end{array}$$(13)where $x_{k,z+1}^B\left(s_k^{\mathrm{″}}\right)$ and $x_{k,z+1}^B\left(s_k^{\mathrm{\prime }}\right)$ represents the likelihood that a user will choose a strategy $s_k^{\mathrm{\prime }}$ and $s_k^{\mathrm{″}}$, respectively; the scaling factor γ is accompanied by a small predetermined constant, denoted by ς, this ensures ensure the total suggested probability is not greater than 1.

Equation (13) shows that a client has two options at each interval z: to continue adopting the identical strategy employed during the previous time period z − 1 $s_k^{\mathrm{\prime }}$ or turn to choose a different action, or shift into a different action, ${s″}_k\in S_k,{s″}_k\ne s_k^{\mathrm{\prime }}$, with probabilities that are directly correlated to the level of regret associated with each option, $M_{k,z}(s_k^{\mathrm{\prime }},s_k^{\mathrm{″}})$. Furthermore, selecting ζ as a minimum makes sure that every strategy S_k has a positive probability of being selected, which aligns with the fundamental concept of reflex-response. As a result, the proposed model in equation (13) is in complete agreement with the reflex-response principle.

Until now models have been constructed for customer decision-making in the context of contract selection, both with and without the availability of a priori information. However, in reality, individuals may not always conform to either of these models, as they may not be completely adaptable or entirely obstinate. As a result, their actual behavior at the contract selection can fall somewhere between the estimates of the two models. To address this issue, the distributions ${\mathrm{\Gamma }}_k^I$ and ${\mathrm{\Gamma }}_k^B$ can be combined as suggested above, the customer contract selection model is determined as follows:$${\mathrm{\Gamma }}_{k,z+1}=\{\begin{array}{c}{x}_{k,z+1}\left(s_k^{\mathrm{″}}\right)=(1-{\lambda }_k)\min \left\{\frac{1}{\gamma }{M}_{k,z}(s_k^{\mathrm{\prime }},s_k^{\mathrm{″}}),\varsigma \right\}+{\lambda }_k{x}_{k,0}\left(s_k^{\mathrm{″}}\right)\\ x_{k,z+1}\left(s_k^{\mathrm{\prime }}\right)=1-\sum _{s_k^{\mathrm{″}}\in S_k:s_k^{\mathrm{″}}\ne s_k^{\mathrm{\prime }}}\mathrm{}{x}_{k,z+1}\left(s_k^{\mathrm{″}}\right)\end{array}.$$(14)

The final probability of a customer selecting an action $s_k^{\mathrm{″}}\left(x_{k,z+1}\right(s_k^{\mathrm{″}}\left)\right)$ represented as a combination of two probability vectors, as demonstrated in equation (14). The initial value, with weight 1 − λ_k, captures the outcome of users’ compliance, whereas the later value, with weight λ_k, indicates the influence of customers’ inherent preferences on their choices. The weighting coefficient 0 ≤ λ_k ≤ 1 determines the significance of the customer’s past experiences. In practice, the system operator must collect information about users’ contract selection actions s_k in the past to determine the value λ_k. The disparities between the data that was observed and the estimation outcomes produced by the two-component models can be calculated using statistical techniques. The obtained distances can serve as indicators of the ‘closeness’ of users’ behavior patterns to the ${\mathrm{\Gamma }}_k^I$ and ${\mathrm{\Gamma }}_k^B$ distributions. Through the normalization of the computation of these distance measures, the value λ_k can be quantified. Detailed procedures for estimating λ_k are provided in Algorithm 1.

Algorithm 2 can be used by the system operator to update and calculate Γ_k,z, k ∈ Ω_D at interval z ∈ T_cs during estimation. Once calculated, the contract selection module can use the results to determine the user’s DR capacity during a specific time period t_z based on equation (15) for a given $$P_{k,t_z}^{\mathrm{av}}=s_{k,z}^{\mathrm{\prime }}\left(P_{k,t_z}^{\mathrm{il},\mathrm{rat}}+P_{k,t_z}^{\mathrm{sl},\mathrm{rat}}\right).$$(15)

4.4 Actual implementation module

The implementation module specifically outlines the decision-making challenges customers face upon receiving DR calls, compelling them to adapt their energy consumption habits dynamically in response to operational requests. To do so, let $P_{k,t_z}^{\mathrm{drr}}$ represent the decrease in demand mandated by system operator at time-slot t_z, the benefit function follows $U_k^{\mathrm{\prime }}$ and G_k, $U_k^{\mathrm{\prime }}$ and G_k are inverse functions that depend on the consumers’ reactions $P_{k,t_z}^{\mathrm{il},-}+P_{k,t_z}^{\mathrm{sl},-}$ and its departure from the level requested $P_{k,t_z}^{\mathrm{drr}}-P_{k,t_z}^{\mathrm{il},-}-P_{k,t_z}^{\mathrm{sl},-}$, respectively. Furthermore, the cost functions $L_k^{\mathrm{il}}$ and $L_k^{\mathrm{sl}}$ represent the costs of inconvenience imposed by DR, and these costs escalate with the extent of response and frequency of occurrence among customers, respectively.

Using the functions presented above, it is possible to define the total payoff a customer receives for modifying their demand at a time slot t_z ($W_k^{\mathrm{\prime }}$). This payoff is calculated by taking the weighted gap between expected revenues from the consumer and the cost of their suffering.$$\begin{array}{c}{W}_k^{\mathrm{\prime }}(P_{k,t_z}^{\mathrm{il},-},P_{k,t_z}^{\mathrm{sl},-})={\omega }_k\left[U_k^{\mathrm{\prime }}\right(P_{k,t_z}^{\mathrm{il},-}+P_{k,t_z}^{\mathrm{sl},-})+{\rho }_k{P}_{k,t_z}^{\mathrm{il},-}-G_k(P_{k,t_z}^{\mathrm{drr},-}-P_{k,t_z}^{\mathrm{il},-}-P_{k,t_z}^{\mathrm{sl},-}\left)\right]\\ -(1-{\omega }_k)\left[L_k^{\mathrm{il}}\right(P_{k,t_z}^{\mathrm{il},-})+L_k^{\mathrm{sl}}(P_{k,t_z}^{\mathrm{sl},-}\left)\right]\\ P_{k,t_z}^{\mathrm{drr}}\\ t_z.\end{array}$$(16)

In equation (16), ρ_k presents the price of retail electricity while ${\rho }_k{P}_{k,t_z}^{\mathrm{il},-}$ indicates the amount by which the user’s electricity bills are reduced as a result of demand reduction. The weighting coefficient 0 ≤ ω_k ≤ 1 specifies the individual’s preference for incentive rewards versus comfort at a time-slot t_z. In the context of the program, the degree to which users comply with DR signals is often uncertain. This uncertainty is captured by the variable ω_k which a probability distribution can be used to describe ψ(ω_k).

In reality, the system operator does not know the form and coefficients of ψ(ω_k), and they must be discovered from client history data. In this research, a non-parametric kernel density-based approach [35] has been employed to achieve this. In contrast to conventional techniques, non-parametric kernel density-based approach does not depend on assumptions about the parameters or a priori knowledge of the variables. As a result, non-parametric kernel density-based approach is capable of fitting any probability distribution shape and can therefore identify various customer behavior patterns in real-world situations. The detailed procedures for determining ψ(ω_k) with non-parametric kernel density-based approach are outlined in Algorithm 3.

Random sampling from ψ(ω_k) is the process that defines the value of ω_k. The anticipated customer response level (load reduction) k ∈ Ω_D at time t_z can be calculated by resolving the subsequent optimization issue:$$\underset{P_{k,t_z}^{\mathrm{il},-},P_{k,t_z}^{\mathrm{sl},-}}{\mathrm{maximize}} W_k^{\mathrm{\prime }}$$(17) $$0\le P_{k,t_z}^{\mathrm{il},-}\le P_{k,t_z}^{\mathrm{il},\mathrm{rat}}$$(18) $$0\le P_{k,t_z}^{\mathrm{sl},-}\le P_{k,t_z}^{\mathrm{sl},\mathrm{rat}}$$(19) $$\begin{array}{c}{P}_{k,t_z}^{\mathrm{il},-}+P_{k,t_z}^{\mathrm{sl},-}\le P_{k,t_z}^{\mathrm{drr}}\\ P_{k,t_z,t\text{'}}^{\mathrm{sl},+}\end{array}.$$(20)

Equations (18) and (19) are employed here to guarantee that the power reduction obtained from interruptible loads and shiftable loads remains within their available capacity during the specified period. Additionally, since emergency DR is implemented only at the behest of the system operator, it is imperative to ensure that the customer’s total DR level does not exceed the required amount. Constraint equation (20) is formulated for this very purpose.

Since the equations (17)–(20) is a nonlinear programming problem with linear constraints, it can be solved using method [38]. Equations (17)–(20) may be solved using the interior point method [38], given that the problem involves nonlinear programming subject to linear constraints. The model’s calculation will show the ideal level of power curtailment concerning interruptible loads and shiftable loads from the viewpoint of a client, $P_{k,t_z}^{\mathrm{il},-}$ and $P_{k,t_z}^{\mathrm{sl},-}$, while subject to the grid’s DR requirements ($P_{k,t_z}^{\mathrm{drr}}$). These findings, as the products of the actual implementation module, returned to DA to calculate the value of $P_{k,t_z,t\text{'}}^{\mathrm{sl},+}$, in equation (3), which are employed in the assessment of DR.

The linearly constrained nonlinear programming problem presented in equations (17)–(20) can be resolved through the interior-point technique [38]. From the customer’s standpoint, the model’s computation will determine the ideal amount of power restriction for both interruptible loads and shiftable loads, $P_{k,t_z}^{\mathrm{il},-}$ and $P_{k,t_z}^{\mathrm{sl},-}$, despite being bound by the grid’s DR requirements ($P_{k,t_z}^{\mathrm{drr}}$). The results obtained from the actual implementation module, serving as the outputs, will be sent back to demand analysis to calculate the value $P_{k,t_z,t^{\mathrm{\prime }}}^{\mathrm{sl},+}$ in equation (3). These values are employed to evaluate DR.

4.5 The outputs module

By integrating the impacts of LR and load reduction, one can calculate the overall request for a DR participant during the crisis, $P_{k,t_z}^{\mathrm{E}}$ can be derived accordingly as:$$\begin{array}{c}{P}_{k,t_z}^{\mathrm{E}}=P_{k,t_z}^{\mathrm{cl},\mathrm{rat}}+P_{k,t_z}^{\mathrm{il}}+P_{k,t_z}^{\mathrm{sl}}=\underset{\begin{array}{c}\mathrm{Outputs\; of\; demand\; analysis}\end{array}}{\underbrace{P_{k,t_z}^{\mathrm{cl},\mathrm{rat}}+P_{k,t_z}^{\mathrm{il},\mathrm{rat}}+P_{k,t_z}^{\mathrm{sl},\mathrm{rat}}}}\\ \underset{\mathrm{Outputs\; of\; actual\; inplementation}}{\underbrace{-P_{k,t_z}^{\mathrm{il},-}-P_{k,t_z}^{\mathrm{sl},-}}}\\ \underset{\mathrm{Outputs\; of\; demand\; analysis} (\mathrm{dependent\; variable\; of\; actual\; inplementation\; outputs})}{\underbrace{+\sum _{t\text{'}=t_z-{\delta }_k}^{t_z-1}\mathrm{}{P}_{k,t\text{'}{t}_z}^{\mathrm{sl},+}}}\end{array}$$(21)where $P_{k,t_z}^{\mathrm{cl},\mathrm{rat}}$ represents the power requirement of critical loads at a time-slot t_z.

In the work of this study, presuming that the system operator can have appliance-level knowledge regarding all its consumers $P_{k,t_z}^{\mathrm{cl},\mathrm{rat}},P_{k,t_z}^{\mathrm{il},\mathrm{rat}}$ and $P_{k,t_z}^{\mathrm{sl},\mathrm{rat}}$ are deterministic values in equation (21).

4.6 Summaries

The aforementioned holistic framework possesses several superior and distinct hypothetical characteristics when compared to current DR modeling methods in the subsequent factors. Firstly, the suggested methodology considers inconsistencies in consumer activity during both the customer-specific and artificial intelligence phases, enabling it to account for the random variability of DR under multiple-timescales. This is as opposed to the majority of existing DR models [7, 18–24] which only consider uncertainties in DR on a short-term basis. Secondly, the presented contract selection model comprises an empirical component (${\mathrm{\Gamma }}_k^I$) and a learning-based component (${\mathrm{\Gamma }}_k^B$), allowing for the examination of both static and dynamic features in customers’ DR decisions as they happen in real-life scenarios. Lastly, the suggested artificial intelligence (AI) artificial intelligence architecture integrates the benefits of both analytic and data-driven methods, enabling the identification of behavioral patterns for distinct customer segments without presuming their preferences. Thus, the composite model may prove to be a more pragmatic approach in real-world implementation than existing analytical approaches [26, 31].

5 Evaluation algorithm

5.1 Main procedures

This section evaluates the algorithm for quantifying DR’s effect on renewable energy use in smart distribution grids, based on the metrics and DR model presented in Sections 2 and 3.

The evaluation flowchart can be seen in Figure 5.

Fig. 5 Flowchart of the evaluation algorithm.

A thorough description of all main steps can be seen below:

1. Input yearly time-series data for distributed renewable generation and system load demands.

2. For each interval, obtain a DR user’s willingness factor by sampling from the probability distribution in Section 4. Consider customer load characteristics and willingness to determine the DR availability profile.

3. Perform optimal power flow analysis using the day-ahead scheduling model from Section 5.2. This model provides anticipatory dispatch based on day-ahead forecasts of renewable outputs and load demand.

4. Sample real-time renewable output and demand-side compliance based on forecast error and actual implementation models.

5. Using decisions from Step 3 and realizations from Step 4, perform immediate scheduling as per Section 5.3 to evaluate final dispatch considering day-ahead decisions and real-time uncertainties.

6. Determine renewable energy usage capacity and system operating cost in real-time.

7. Apply the RMP algorithm from Section 4 to update users’ willingness probability distribution for the next interval.

8. Check if a new DR contractual interval has started. If so, resample new willingness factors for each DR user; otherwise, continue with the current factors.

9. Repeat Steps 3–7 until the simulation ends. Aggregate values for system evaluation indices, such as renewable penetration ratio and operating cost, for each period.

5.2 Day-ahead scheduling model

The day-ahead dispatch model focuses on minimizing the total operation cost of the distribution system, considering renewable energy and DR, including energy procurement, DR implementation, load shedding, and auxiliary services acquisition.$$\begin{array}{c}\begin{array}{c}\underset{P_{k,t}^{\mathrm{drr}},P_{k,t}^{\mathrm{usd}},P_{j,t}^w,P_{j,t}^{\mathrm{pv}},P_{j,t}^{\mathrm{buy}},P_{j,t}^{\mathrm{il},\mathrm{rat}},P_{j,t}^{\mathrm{sl},\mathrm{rat}}}{\mathrm{Minimize}}\end{array}\\ \sum _{j\in {\mathrm{\Omega }}_F}\mathrm{}\left(c_1{P}_{j,t}^{\mathrm{buy}}+c_2{P}_{j,t}^{a.+}+c_3{P}_{j,t}^{a.-}+c_6{P}_{j,t}^{\mathrm{drr}}+c_7{P}_{j,t}^{\mathrm{usd}}\right)\end{array}.$$(22)

The optimization is subject to several constraints including distribution network power flow, power balancing, DR availability, etc., as shown in equation (23)–(31):$$\begin{array}{c}{P}_{\mathrm{ij},t}=\sum _{(j,k)\in {\mathrm{\Omega }}_F}\mathrm{}{P}_{\mathrm{jk},t}+(P_{j,t}^{\mathrm{cl},\mathrm{rat}}+P_{j,t}^{\mathrm{il},\mathrm{rat}}+P_{j,t}^{\mathrm{sl},\mathrm{rat}}-P_{j,t}^{\mathrm{drr}})\\ -P_{j,t}^{\mathrm{usd}}-P_{j,t}^w-P_{j,t}^{\mathrm{pv}}-P_{j,t}^{\mathrm{buy}}-P_{j,t}^{a.+}+P_{j,t}^{a.-},& \forall \left(i,j\right)\in {\mathrm{\Omega }}_F,\forall t\end{array}$$(23) $$\begin{array}{c}{Q}_{\mathrm{ij},t}=\sum _{\left(j,k\right)\in {\mathrm{\Omega }}_F}\mathrm{}{Q}_{\mathrm{jk},t}+\left(Q_{j,t}^{\mathrm{cl},\mathrm{rat}}+Q_{j,t}^{\mathrm{il},\mathrm{rat}}+Q_{j,t}^{\mathrm{sl},\mathrm{rat}}-Q_{j,t}^{\mathrm{drr}}\right)-Q_{j,t}^{\mathrm{usd}},\\ \forall \left(i,j\right)\in {\mathrm{\Omega }}_F,\forall t\end{array}$$(24) $$\begin{array}{cc}{V}_{i,t}=V_{j,t}+\frac{r_{\mathrm{ij}}{P}_{\mathrm{ij},t}+{\chi }_{\mathrm{ij}}{Q}_{\mathrm{ij},t}}{V_{0,t}}& \forall \mathrm{ij},t\end{array}$$(25) $$\begin{array}{cc}{I}_{\mathrm{ij}}=(V_{i,t}-V_{j,t})/{\iota }_{\mathrm{ij}}& \forall (i,j)\in {\mathrm{\Omega }}_F,\forall t\end{array}$$(26) $$\begin{array}{cc}{V}_{\min }\le V_{i,t}\le V_{\max }& \forall i,t\end{array}$$(27) $$\begin{array}{cc}0\le I_{\mathrm{ij},t}\le I_{\mathrm{ij},\max }& \forall (i,j)\in {\mathrm{\Omega }}_F,\forall t\end{array}$$(28) $$P_{j,t}^w+P_{j,t}^{w.\mathrm{ab}}=P_{j,t}^{w.\mathrm{act}}$$(29) $$P_{j,t}^{\mathrm{pv}}+P_{j,t}^{\mathrm{pv}.\mathrm{ab}}=P_{j,t}^{\mathrm{pv}.\mathrm{act}}$$(30) $$\begin{array}{cc}0\le P_{k,t}^{\mathrm{drr}}\le P_{k,t}^{\mathrm{il},\mathrm{rat}}+P_{k,t}^{\mathrm{sl},\mathrm{rat}}& \forall k\in {\mathrm{\Omega }}_D,\forall t\end{array}$$(31) $$\begin{array}{cc}0\le P_{k,t}^{\mathrm{usd}}\le P_{k,t}^{\mathrm{cl},\mathrm{rat}}& \forall k\in {\mathrm{\Omega }}_D,\forall t.\end{array}$$(32)

Equations (23)–(24) are linearized power flow equations for distribution networks. To ensure the security of the system, equations (25)–(28) enforce constraints on the nodal voltage deviation and current-carrying capacity of feeder lines. Equations (29)–(30) represent the power output constraints of renewable energy generation, i.e., the available output is the additions of the actual power utilized and the abandoned power. Equations (31) and (32) specify the constraints on the maximum scheduling level and power shedding for each load node.

In these constraints, c₆ is the cost of demand reduction, c₇ is the penalty cost of lost load; $P_{k,t}^{\mathrm{usd}}$ and $Q_{k,t}^{\mathrm{usd}}$ denote the unsatisfied active/reactive load demand at node k in time t; $V_{i,t}=V_{j,t}+(r_{\mathrm{ij}}{P}_{\mathrm{ij}}+{\chi }_{\mathrm{ij}}{Q}_{\mathrm{ij},t})/V_{0,t},\forall i,j,t$ and $Q_{j,t}^{\mathrm{gen}}$ are the active/reactive power output of the generating unit at system node j for time t; P_ij,t, Q_ij,t, and I_ij,t denote active/reactive currents and currents in the network branch ij; $P_{j,t}^w$ is the wind turbine output utilization at node j in time t; $P_{j,t}^{w.\mathrm{ab}}$ is the corresponding wind turbine disposal power, and $P_{j,t}^{w.\mathrm{act}}$ is the predicted available output of the wind turbine; $P_{j,t}^{\mathrm{pv}}$ is the output utilization of the PV unit at node j in time t; $P_{j,t}^{\mathrm{pv}.\mathrm{ab}}$ is the corresponding PV power abandonment; $P_{j,t}^{\mathrm{pv}.\mathrm{act}}$ is the predicted available PV output; Ω_F is the set of system nodes; V_i,t and V_0,t are the voltage amplitudes at system node i and the relaxation node in time t; r_ij, r_ij and ι_ij are the resistance, reactance, and impedance values of the feeder ij.

In the day-ahead dispatch, the model is solved using predicted load and renewable generation values to create a power purchase plan and determine each customer’s DR dispatch capacity, considering demand-side behavior. These results are then used in the intraday real-time scheduling model. The real-time scheduling accounts for changes in demand responsiveness and errors in day-ahead renewable forecasts.

5.3 Intraday scheduling model

The intra-day dispatch aims to reduce the actual operating charges of the system within the day, based on the day-ahead scheduling, as shown in equation (33):$$\underset{P_{k,t}^{\mathrm{drr}},P_{k,t}^{\mathrm{usd}},P_{j,t}^w,P_{j,t}^{\mathrm{pv}},P_{j,t}^{a.+},P_{j,t}^{a.-}}{\mathrm{Minimize}}\sum _{\left(j,k\right)\in {\mathrm{\Omega }}_F}\mathrm{}\left(c_1{P}_{k,t}^{\mathrm{usd}}+c_4{P}_{j,t}^{a^{\text{'}}.+}+c_5{P}_{j,t}^{a^{\text{'}}.-}\right).$$(33)

In intraday scheduling, the power purchase and DR scheduling capacity are fixed, so they are regarded as constants in intra-day scheduling. When solving the model, equations (23)–(24) in the day-ahead dispatch need to be replaced with equations (34)–(35), but the corresponding dispatch willingness of each node load, renewable energy unit output, and customer DR need to use the data of the new scenario as a substitute because of the difference between the actual and predicted values.$$\begin{array}{c}{P}_{\mathrm{ij},t}=\sum _{(j,k)\in {\mathrm{\Omega }}_F}\mathrm{}{P}_{\mathrm{jk},t}^{\mathrm{\prime }}+{\omega }_k(P_{j,t}^{\mathrm{cl},\mathrm{rat}}+P_{j,t}^{\mathrm{il},\mathrm{rat}}+P_{j,t}^{\mathrm{sl},\mathrm{rat}}-P_{j,t}^{\mathrm{drr}})\\ -P_{j,t}^{\mathrm{usd}}-P_{j,t}^{w\text{'}}-P_{j,t}^{\mathrm{pv}\text{'}}-P_{j,t}^{\mathrm{buy}}-P_{j,t}^{a\text{'}.+}+P_{j,t}^{a\text{'}.-},& \forall (i,j)\in {\mathrm{\Omega }}_F,\forall t\end{array}$$(34) $$\begin{array}{c}{Q}_{\mathrm{ij},t}=\sum _{(j,k)\in {\mathrm{\Omega }}_F}\mathrm{}{Q}_{\mathrm{jk},t}+{\omega }_k(Q_{j,t}^{\mathrm{cl},\mathrm{rat}}+Q_{j,t}^{\mathrm{il},\mathrm{rat}}+Q_{j,t}^{\mathrm{sl},\mathrm{rat}}-Q_{j,t}^{\mathrm{drr}})-Q_{j,t}^{\mathrm{usd}},\\ \forall (i,j)\in {\mathrm{\Omega }}_F,\forall t\end{array}$$(35)where P_ij,t and Q_ij,t denote active/reactive currents and currents in the network branch ij; Ω_F is the set of system nodes; $P_{\mathrm{jk},t}^{\mathrm{\prime }}$ is the actual value of node j load in t time; ω_k is the customer’s willingness to participate in DR and takes the value of a random number between 0 and 1, $P_{j,t}^{w\text{'}}$ is the actual wind turbine output at node j in time t, and $P_{j,t}^{\mathrm{pv}\text{'}}$ is the actual PV output at node j in time t; $P_{j,t}^{a\text{'}.+}$ is the upper reserve capacity purchased in the auxiliary service market for a node j at a time t in the intra-day dispatch, $P_{j,t}^{a\text{'}.-}$ is the lower reserve capacity purchased in the auxiliary service market for a node j at a time t in the intra-day dispatch, and c₄, c₅ are the cost factors of the upper and lower reserve capacities, respectively. For the remaining constraints, equations (23)–(28) and equations (31)–(32) remain unchanged, and equations (29)–(30) are replaced with the following:$$P_{j,t}^{w\text{'}}+P_{j,t}^{w.\mathrm{ab}\text{'}}=P_{j,t}^{w.\mathrm{act}\text{'}}$$(36) $$P_{j,t}^{\mathrm{pv}\text{'}}+P_{j,t}^{\mathrm{pv}.\mathrm{ab}\text{'}}=P_{j,t}^{\mathrm{pv}.\mathrm{act}\text{'}}.$$(37)