Within the static models, the optimal values for several parameters such as power, number of sub-bands, and assigned frequencies per cell do not present variations. Hence, these ones are determined in function of environments with fixed traffic patterns. In these models, some conventional frequency planning schemes are included (e.g., reuse 1 and reuse 3, also called hard frequency reuse). Further, other Fractional Frequency Reuse [FFR] schemes are also included (Budihal, Siddamal, & Banakar, 2016), which are described in the following section. Regardless of their differences, these schemes require to specify common parameters such as the channels set (sub-bands) to be used in each cell, the power and region of the cell where the sub-bands are used, and the center and edge of the cell.

This scheme separates the allocated frequency bands to distant areas of a base station by fixing a frequency reuse of 3. For the central regions, the transmission power is reduced and a frequency reuse of 1 is applied (see Figure 2c). By doing this, it is possible to improve the Signal to Interference plus Noise Ratio [SINR] and the throughput for the UE in the cell edges. Some FFR schemes presented by Hamza et al., (2013)are described as follow:

This reuse technique uses two sub-bands having different frequency reuse. A common sub-band from the system bandwidth is used inside the cell (reuse 1), whilst the other part of the bandwidth is divided between the neighbor eNB —similar to the hard frequency reuse where N is the frequency reuse and N > 1—, to create a sub-band with a low ICI level on each sector.

The UE in the center will receive the frequency fragments completely reused, whilst the UE in the edges will receive the orthogonal fragments. This means that the internal UE in a cell do not share any spectrum with the UE in the edges of the second cell, reducing the ICI (see Figure 3)

Figure 3
Strict FR scheme with power and frequency plan (“ns-3...”, 2011)

For this scheme, two versions are available: in the first one, the dedicated sub-band for the edge UE also can be used for the UE in the center of it, but with a reduced power level and only if it is not busy by the edge UE. The sub-band for the center is available only for the central UE (see Figure 4a). The second version consists of the UE in the center without access to the edge sub-band; hence, each cell can use all the system bandwidth while the interference to the neighbor cells is reduced (see Figure 4b).

Figure 4
SFR schemes: SFR version 1 (a); SFR version 2 (b) (“ns-3...”, 2011)

Xie (2009) describes EFFR as a scheme improving the performance of the previous one, also the system capacity in high-load situations is improved. This scheme defines three reuse types for the neighbor cells and it reserves a part of the complete frequency band (primary segment) for each cell type. Within the different cell types, the band has to be orthogonal. The remaining subchannels are the secondary segment. The primary segment of a cell type is, at the same time, part of the secondary segments belonging to the other two cell types. Each cell can use all the subchannels in a primary segment if required, whilst this cell can only use a part of the subchannels of its secondary segment by considering the interference (see Figure 5). Due to the fact that each eNB needs to know the location of the primary and secondary segments of other cell types, the eNB will calculate them assuming that the configuration is the same for each cell and it will also assume only the bandwidth displacements as different (Mohamed & Abd-Elnaby, 2016).

Figure 5
EFFR scheme (“ns-3...”, 2011)

Within this classification, it is possible to include dynamic schemes with low, intermediate, and high level (Hamza et al., 2013). In the first group, some configuration parameters previously planned and optimized are present for each case, based on the different traffic loads and UE distributions. Hence, the Base Stations [BS] exchange information. For intermediate level schemes, the calculation of the optimal values is done according to the amount and distribution of UE in each cell. This is done from available data of the cells without limiting to use predetermined configurations. In the high-level schemes, an improvement relative to the intermediate level is present due to the way how the optimal values are calculated. These parameters include the power relation, number of sub-bands, and frequencies allocation. Additionally, the number of sub-bands to be assigned to each UE is calculated in function of the channel condition. Variations of FFR schemes can be used also.

This schemes type seeks to explore the ICIC via management and control models and through the coordinated information exchange among cells by achieving adaptable ICIC values to different traffic conditions (Hamza et al., 2013).The schemes can be classified in four categories: centralized, semi-distributed, coordinated-distributed, and autonomously distributed. In the first one, a central control unit is in charge of gathering all the information relative to the channel state and send it in a coordinate manner to each eNodeB to manage the resource allocation. The second one presents a general coordination in two levels, eNodeB and central entity; this allows a more efficient information exchange. In autonomous distributed schemes, the coordination is performed only at the eNB level without the usage of a central entity for coordination. Contrary to the coordinated-distributed schemes, the autonomous distributed do not require intranodal coordination since each eNodeB assigns channels to their UE by using local information. This entails an advantage of eliminating the signaling overhead via a planning algorithm, producing that the decisions can be adapted quickly to the channel features.

The self-organization is a key concept relative to autonomy for the future evolution of the mobile networks to achieve a self-organized ICIC on each eNB, it dynamically must restrict some of their RB through techniques such as the power control. This involves parameters such as the SINR levels in the received RB, where a low SINR level indicates that an RB is being used by a neighbor eNB.

Deb (2014) presents an algorithm to reduce the ICI based on the allocation of Almost Blank Sub-frames [ABS] and in the Cell Selection Bias [CBS] in the LTE standard. The algorithm determines the optimal associations of ABS and UE through two main tasks: determine the amount of radioelectric resources that the macrocells should offer to the picocells and determine the association rules deciding what UE should associate with picocells. In this research work, the author validates its solution by considering the load in the cells, the macro-pico interference maps, and the network topology. Also, the solution is assessed in a real scenario in New York by applying the SON frame. Further, the eICIC is handled through the allocation of ABS, which is a key point in the frequency reuse.

Regarding the use of EFFR schemes, Mohamed and Abd-Elnaby (2016)present an approach called Self-Organized Dynamic Resource Allocation scheme using Enhanced Fractional Frequency Reuse [SODRA-EFFR], which seeks to improve the performance and coverage in the cell edges through the dynamic and self-organized allocation of resources (power and frequency) to the internal and external regions of the cells in networks based on relays for LTE-A retransmissions. In this scheme, both the power and frequency allocation between the eNB and relative to the retransmission in each cell is performed via the coordination of neighbor eNB and relays through the X2 interface. The performance of this approach was assessed via MATLAB simulations with and without relays. Furthermore, the results were compared using different combinations of resources allocation to the internal and external regions of the cells with reuse factors of 1 and 3, such as with their SFR scheme.

The Bayes theorem suggested in Klaine et al., (2017)is an important rule in the probability and statistics areas for the analysis of conditional probabilities, i.e., to understand how the probability of a hypothesis (h) is modified by a new evidence (e). The Bayes theorem is presented in Equation (1).

(1)

where:

P(h/e) is the probability of the hypothesis given the evidence (aposteriori);

P(e/h) is the probability of the evidence given the hypothesis (conditional probability);

P(h) is the probability of the hypothesis (apriori); and

P(e) is the probability of the evidence (total probability).

The Bayes theorem is widely used in the telecommunications area, punctually in the ICI problem.

Li et al., (2016) study the Dynamic Frequency Resource Allocation [DFRA] through the deploy of microcells called Small Cell Base Station [SCBS]and following a Poisson model. They achieve an adaptive scheme through the application of the Bayesian theory in the correlated spectral use of the adjacent SCBS. For this, they model the SCBS and the UE as a homogeneous Poisson process, with intensities λBS and λUE respectively. Here, each UE is associated with the closest SCBS waiting an amount N0 of subcarriers, where this N0 is a preestablished value.

In simple terms, Madariaga, Rodríguez, Lozano, and Vallejo (2013) define the linear regression as a mathematical model expressing the relations between a dependant variable and one or many independent variables by using a straight line (regression line). Equation (2) presents its general form.

(2)

where:

a represents the ordinate because its value is the point where the regression line crosses the vertical axis; and

b represents the slope of the line.

In the ICI field, it is possible to elaborate a model or function (generally a straight line) through this technique from a set A(x,y) ∈ R(n+1), where its elements would be the parameters to use. The model assumes a linear relation between the x,y variablesand it fits in a straight line to the data points (see Figure 7). This relation is expressed through a hypothesis function (Equation 3) to predict a set of outputs (Shanthamallu, Spanias, Tepedelenlioglu, & Stanley, 2017).

Figure 7
Simple regression with one variable or parameter (Shanthamallu et al., 2017)

(3)

where:

x1,x2,...xn are the parameters; and

w1,w2,...wn are the model values.

Sierra and Marca (2015) present a statistical approach based on multivariable polynomial regression consisting of two phases. The first one seeks to find the relation between the explanatory variables: cell traffic state and Cell Range Expansion [CRE] with the dependent variables: Packet Loss Ratio [PLR] and Packet Data Convergence Protocol [PDCP] through a model ruled by the function (4):

(4)

where:

x is the number of active UE; and

y the compensation femtocells.

The second phase consists of the CRE displacement selection; the femtocell displacement is dynamically determined by following the model extracted in the previous phase. Hence, the packet loss in the PDPC layer is minimized according to the condition of the traffic in the sector. Finally, a simulation at the system level is performed, where the efficiency of the dynamic CRE is presented.

Bojović, Meshkova, Baldo, Riihijärvi, and Petrova (2016) research the use of several ML techinques combined with statistical regression and they present an approach based on learning to achieve self-optimization in a SON deployment. In their proposal, the learning capabilities are focused on the estimation of KPI within the user and network levels to select the optimal configuration affecting the whole LTE stack. This is done via the Dynamic Frequency and Bandwidth Allocation [DFBA] technique in LTE microcells. They compare their results with other ML approaches, concluding that theirs is a centralized frequency allocation approach for microcells.

Likewise, the authors take as input data the performance measurements in the frequencies configurations, which allows to estimate the impact in the configuration and predict the performance. The results show that the DFBA based on the learning achieves —on average— an improvement in the performance of 33% relative to the approaches based on analytical models, reaching up to 95% of the performance.

This concept arose as an attempt to simulate the behavior of the human brain in computers (Hinton, Srivastava, & Swersky, 2014). The brain is capable of performing highly complex, non-linear calculations in parallel continuously. By dividing these functions in very basic components, known as neurons, and giving them the same calculation function, a simple algorithm can become a robust computing tool.

Li, Liang, and Ascheid (2016) present a proposal for eICIC in HetNets by involving the relation between the CRE and ABS for the dynamic allocation of RB. They consider a HetNet scenario with several users, composed by macrocells ʍ={1,2,3....M} and a set of picocells Ƥ={1,2,3....P}, together with a vector of moving users Ս={1,2,3....U}.In order to improve the download capabilities of a small cell, the CRE strategy is employed by adding a positive compensation to the Reference Signal Received Power [RSRP]in the downlink [DL] according to the imperative weight of each element of the users’ vector. This information —together with the historic data of the users—, the channel information, the optimal CRE and ABS patterns, and the SINR vectors are the required supplies to train the machine learning Artificial Neural Network [ANN] algorithm and achieve as output a resource allocation pattern for macro and pico users

The SVM technique uses a subset of training data called support vectors, where a mapping —linear or non-linear, polynomial or Gaussian— is applied and where the objective is to maximize the distance between several classes; that is, act as a classifier. The support vectors are the closest samples to the decision zone, in the case showed in Figure 8 they are the green circles, the most difficult points to classify; whilst that the shaded region indicates the optimal gathered decision limit (Klaine et al., 2017).

Figure 8
Example of a SVN linear optimal hyperplane (Klaine et al., 2017)

Cordina and Debono (2017) study techniques combined with SVM through the adaptation of links and the so-called Frequency Selective Scheduling [FSS] to increase the throughput in mobile services. Through a signaling process, the information delivered by the eNodeB regarding the Channel Quality Indicator [CQI] is used. These data are feedback by the UE data and given that the signaling load increases due to this feedback, a machine learning technique is exploited to address this issue. The authors also propose a new CQI feedback compressing scheme in the sub-bands to predict the channel state through the use of the SVM vector. This information is transmitted to the eNB to recover all the channel sub-band response within a configurable error margin.

This unsupervised learning algorithm is one of the most used in cellular systems (Klaine et al., 2017).Its function is to group data without tags and find their centers from two parameters: the initial dataset and the desired number of clusters (see Figure 9).

Figure 9
K-Means and the cluster centers (Shanthamallu et al., 2017)

Some of its applications can be found in the work of Hajjar, Aldabbagh, and Dimitriou (2015) where, through the grouping and classification of low-power nodes, a Cluster Head [CH] or main group is selected. This CH oversees the direct communication with the eNBs; on the other hand, a secondary —or slave— group is also selected to oversee the retransmissions. For the classification, the SINR parameter is employed depending of the service type and the number of associated slaves. Furthermore, a total of 100 RBs are assigned for all the cell, which are distributed one to one in the UE. For the blocked UE, an appropriate CH from the connected ones is assigned to retransmit their information to the eNB. In this proposal, the K-means algorithm is mixed with the Hierarchical Agglomerative Clustering [HAC] one.

Within the HetNet topic, Qi, Zhang, Chen and Zhang (2018) seek self-organization for macrocells and load equilibrium for different transmission powers. This is done via a User-Based K-means Algorithm [UBKCA], where it firstly classifies the UE groups in the center to reduce the computing complexity. After, it uses the CRE and the edge user factor to improve the load equilibrium. The optimal combination of these two elements produce CIC, both in accuracy and in the UE download factor. Using the decision limits calculated with the algorithm, a closed-loop SON system is implemented. The used model defines a dataset R={R1,R2,R3....Rn}divided in k groups through an analysis. After, there are chosen randomly the centers of each group included in a set O={O1,O2....Ok}, to finally assign the n-kremaining elements to its closest center to form clusters and then calculate the new center for each group. This task is repeated until the system obtains convergence.

This is a mathematical tool based on the hypothesis established by Neumann, Morgenstern, and Nash. This last proposed the Nash Equilibrium [NE] concept and the modelling is like the RL algorithms described in the following section, where multiple agents take actions in a competitive environment and, at the end of the game and depending of the result of each agent, an award or a penalization are granted (Miramá & Quintero, 2016).

These algorithms can be applied in ICIC, as Morel and Randriamasy (2017) describe. Their work is focused on improving the Quality of Experience [QoE], a different metric than the usual Quality of Service [QoS]. The QoE subjectively assesses how a user perceives the service satisfaction degree within a HetNet scenario with larger HD video demands. That is, it is focused on a specific utility and not in the general network performance. This approach seeks to optimize the silence period of macro and microcells between eNBs and connected users through a centralized optimizer and coordinator. Besides, it seeks to achieve a direct impact in the spatial distribution and in the SINR by using the ABS and Cell Individual Offset [CIO] parameters to select the optimal pairs and limit the interference caused by the edge UE while the QoE is improved.

The Q-Learning [QL] approach is a type of RL algorithm where an agent creates a table of qvalues. By exploring the environment, the q function acts as a value function by estimating future awards in current actions. The q values are stored in a table for each possible action and state. Nevertheless, the table can be highly complex and non-applicable for large state action spaces (Daeinabi& Sandrasegaran, 2014).

QL is a very popular algorithm within the ICIC applications. Ming, Ye, and Xinyu (2017) propose a QL multi-agent model based on the resource allocation, where two paradigms are introduced inside a HetNet environment: a QL distributed algorithm applicable to picocells, which learn from the environment if they should share information; and a centralized QL algorithm, where different agents interact amongst them by sharing resources and scheduling information. Unlike the traditional RL approaches, the QL algorithms do not require a precise model; hence, these latter are chosen to be applied in the resources allocation. The proposed model has M macrocells, which conform a layer of N clusters together with K picocells randomly distributed inside each cluster. Likewise, Um and Up are defined as users for the macro and picocells, respectively. Each user selects the cell to use according to the RSRP parameter and they are randomly blocked inside the coverage areas. For this, the small cells inside the same cluster must share partial information entailing to gather better performance results.

On the other hand, QL can be combined with other techniques such as fuzzy logic, as Daeinabi and Sandrasegaran (2014) propose. They analyze the CRE producing ICI in the downlink inside a HetNet and as a solution, they propose a dynamic ABS allocation system by using a QL approach combined with fuzzy logic. This allows to tackle the issue of a complex table with q continuous values by transforming it into a table with discrete values combining numerical and linguistic data; these latter are obtained through the operator experience. A QL fuzzy controller is included to process the fuzzy state variables such as the number of macro/micro UE, number of ABS frames, throughput of macro UE, and SINR. The objective of the controller is to find the ∆ABS value that is added to the current ABS value for each macrocell. Each eNB performs local tasks to find this value and then uses the optimal ABS value for itself and for its own picocells.

Morozs et al., (2015)start from the Dynamic Spectrum Access [DSA] concept in LTE networks to propose an improved ICIC alternative through a distributed RL algorithm called Distributed ICIC Accelerated Q-learning [DIAQ]. This proposal is also Heuristically Accelerated [HARL] to mitigate the disadvantage that the Q-learning algorithms have relative to require many learning interactions. Such acceleration is particularly provided in the multi-agent domain, leading the exploration process by using additional heuristic information. The scheme is assessed in a stadium-like simulated scenario. The work also describes a generic hexagonal architecture and the usage of the X2 interface for the exchange of ICIC signals. Here, a central eNB sends ICIC signals to the neighbor eNBs. Additionally, a theoretical analysis of a Bayesian network was performed pursuing a new DSA approach using RL. Consequently, this work presents an interesting combination of learning and heuristic techniques, achieving a significative performance through accelerating the algorithm. This produces important results for the reduced time response, which is similar to the pursued objective: reduce the computing complexity.

This type of algorithms corresponds to simple schemes governed by a set of rules seeking to take the best decision for the system in a given moment. Because they fit as unsupervised algorithms, they are applied when there is no known solution of a particular problem; however, due to their simplicity, the solutions tend to be approximated and suboptimal. Likewise, the meta-heuristics have a similar approach defined by a set of rules, but they are more complex and higher level, producing more optimal solutions (Klaine et al., 2017).

In the ICIC context, it is possible to combine heuristic techniques with other models such as Supratim and Monogioudis (2015) present.Their proposal called Learning-based Adaptive Power control [LeAP] is a machine learning approach based on data measurements (power control parameters) to manage the interference in the uplink. Their proposal is composed of two key elements: the design of measurements statistics for the UE, basic but concise data, enough to determine interference patterns in the network environment; and the design of two algorithms based on learning, one of them seeking the optimal power control parameters, the other is a fast heuristic capable to be implemented using solvers commercially available. The benefit evaluation of this approach is performed using a complex urban scenario in a train station in the United States, where it is demonstrated that there is an improvement in the data rate in the cell edges. Compared with similar approaches, this solution can be implemented by using a SON server centralized architecture.