 Research
 Open Access
 Published:
Optimized sparse polynomial chaos expansion with entropy regularization
Advances in Aerodynamics volume 4, Article number: 3 (2022)
Abstract
Sparse Polynomial Chaos Expansion (PCE) is widely used in various engineering fields to quantitatively analyse the influence of uncertainty, while alleviating the problem of dimensionality curse. However, current sparse PCE techniques focus on choosing features with the largest coefficients, which may ignore uncertainties propagated with high order features. Hence, this paper proposes the idea of selecting polynomial chaos basis based on information entropy, which aims to retain the advantages of existing sparse techniques while considering entropy change as output uncertainty. A novel entropybased optimization method is proposed to update the stateoftheart sparse PCE models. This work further develops an entropybased synthetic sparse model, which has higher computational efficiency. Two benchmark functions and a computational fluid dynamics (CFD) experiment are used to compare the accuracy and efficiency between the proposed method and classical methods. The results show that entropybased methods can better capture the features of uncertainty propagation, improving accuracy and reducing sparsity while avoiding overfitting problems.
Introduction
Due to the variety of uncertainties frequently involved in engineering applications, which may cause fluctuations in the performance of a system, it is necessary to comprehensively consider the impact of uncertain factors [1, 2]. To deal with the problem of limited experiment resources, various surrogate models [3, 4] have been proposed, aiming to construct a mathematical model that accurately mimics the behavior of the original problem with an affordable experimental design [5]. Popular surrogate techniques include the Kriging method [6–9], artificial neural network [10–12], polynomial chaos expansion (PCE) method [13–16] etc. In this paper, we focus on the PCE model, which has been widely used in engineering to quantify uncertainties efficiently.
The basic idea of PCE is to expand an exact solution of a stochastic process space by polynomial expansion [17–19], and for the model generated by which can generally be solved by ordinary least squares (OLS). Many successful applications of uncertainty quantification based on the PCE method have been achieved [20–22]. However, the cost of constructing a PCE model increases exponentially with the dimension of input parameters, i.e. the curse of dimensionality, thus severely restricts practical applications of the model at the industrial level [23–25]. In order to solve this problem, lots of sparse algorithms have been developed in recent years. Some typical examples of wellestablished methods are the sparse regression method [26, 27] and the compressive sampling method [28, 29].
A problem with some OLSsolution methods is the possibility of overfitting but can be avoided by regularization, which is another practical solution to seek a sparse representation [30]. Regularization is a typical model selection method that introduces a regularizer to the empirical risk. From the perspective of Bayesian estimation, the regularizer corresponds to the prior probability of the model [31].
There are many regularization methods, but it is ℓ_{0} regularization that is required to obtain a true sparse model. However, solving model with ℓ_{0} regularization is an NPHard problem [32, 33], so researchers usually adopt the ℓ_{1} regularization when dealing with sparse problems [34], which is an optimal convex approximation of the ℓ_{0} paradigm, and can be easily solved optimally to obtain a sparse model.
Although scholars have conducted a considerable amount of research on sparse polynomial chaos expansions, most of the existing sparse methods are derived from the perspective of regression [35]. In these methods, researchers usually pay attention to one or two numerical characteristics in the distribution, which are important statistics to measure uncertainty. However, the distribution is very complicated in common situations, especially when there is a noise in the original problem. Furthermore, the aforementioned methods focus on statistics, such as the relative mean squared error (MSE) or the crossvalidation error, to optimize parameters, which ignores the overall uncertainties propagated from the input variables.
To depict the full uncertainty of the output, we introduce information entropy as a measure of the evolution of the output distribution to further optimize the sparse representation of a PCE model. Information entropy was first introduced to measure microscopic uncertainty and multiplicity by Shannon [36]. The idea was then extended to learning tasks to measure the information changes with different statistical models. Jaynes [37] proposed the maximum entropy principle (MEP) to provide an optimization criterion. B. Grechuk et al. [38] have studied the relationships among error, likelihood, and entropy in regression analysis, and found that a normal distribution can be recovered from the maximum entropy principle. Wang [39] introduced a maximum entropy penalty to model and incorporate the entropycontrolled framework with other conventional learning algorithms. Liang [40] proposed a sparse subspace clustering with entropynorm by using information entropy as the regularizer of the objective function. Researchers further made variable selection through the evaluation of entropy, and extended the use of entropy as an important criterion for model selection [41, 42], which can be embedded in existing methods but does not form a framework.
In this work, we aim to use entropy to propagate uncertainty of inputoutput while retaining the advantages of existing sparse PCE techniques. Hence, we propose a novel method where entropy is an auxiliary penalty term. Firstly, a general reoptimization model is proposed which chooses the features with the largest information entropybased on the existing optimized model. Although such a strategy is easy to implement, it may ignore some key features in the first stage. A hybrid entropybased synthetic method embedded in several commonly used classical sparse criteria is thus proposed. The novel regularization structure takes the value and volatility of each feature into account simultaneously. The entropy term can be regarded as a tradeoff between the predictive mean value and uncertainty along with the selected features. The above two algorithms can be easily computed with existing sparse algorithms, which improves their usage scope and availability. Experiments are performed to compare the proposed two algorithms and classical sparse methods, i.e. the Orthogonal Matching Pursuit (OMP), the Least Angle Regression (LARS), the Subspace Pursuit (SP), and the Bayesian Compressive Sensing (BCS). Results show that the general reoptimization method is of simple operation and universality because it can optimize any sparse PCE model, while the hybrid entropybased synthetic method has strong pertinence, and is superior to the former in highdimensional complex applications.
The remainder of the paper is organized as follows. Section 2 introduces the fundamental theory of sparse PCE models. Section 3 proposes the theories and algorithms of the two optimization methods mentioned in the previous paragraph. Section 4 shows the performance of the proposed methods, illustrated by numerical examples and comparisons. Finally, the conclusions are summarized in Section 5.
The fundamental theory
Sparse polynomial chaos expansion
The basics of PCE are elaborated briefly as follows. Let \(\mathcal {M}\) be the original model and \(Y=\mathcal {M}(\xi)\) be the output. The finiteorder PCE model expanded in the full polynomial space can be expressed as:
where ξ is ddimensional random variables with a probability density function (PDF) of f(ξ), and Ψ_{i}(ξ) is a set of polynomial basis functions truncated at the pth order with P denoting the degree of the PCE. Generally, P varies with different truncation schemes. For example, when the total degree space is chosen,
with p as the maximum degree for each dimension. Given a design of experiment(DoE) {ξ,Y}, where \(\boldsymbol {\xi } = \left (\xi _{1}, \xi _{2}, \ldots, \xi _{n} \right)\in \mathbb {R}^{d\times n}\) is a specific sample and \(Y(\boldsymbol {\xi })\in \mathbb {R}^{n}\) are corresponding responses. The main effort of constructing a PCE model is solving the following generalized linear equation system:
which can be rewritten in matrix form:
Although we are able to obtain a least squares estimate \(\hat {\boldsymbol {c}} = \left (\boldsymbol {\Psi }^{T}\boldsymbol {\Psi } \right)^{1}\boldsymbol {\Psi }^{T} \boldsymbol {Y}\) easily, the degree P increases dramatically with p and d, hence we have to solve an underdetermined system with limited DoE resources, and the least squares solution can be inaccurate and unstable. The sparse PCE method is proposed to solve this problem, and it restores the complete model response almost accurately by selecting a small number of basis that dominates the system output. ℓ_{0}norm is the most widely used regularization criterion, i.e. limit the degree of nonzero coefficients by solving the following optimization problem:
However, Eq. 5 is a nonconvex optimization problem, which can be difficult to solve in practice due to its NPhardness. A typical practical technique is to replace the ℓ_{0} norm term with the ℓ_{1} norm term [43], and the new objective function becomes
Error estimation and model evaluation
The optimization problem defined in Eq. 6 is usually solved by relaxing the constraint, i.e.
where ε is recorded as the truncation error, determined by measurement noise, which is a more natural parameterization choice. If ε is too large, the reconstructed PC is not accurate enough; however, if ε is too small, the reconstructed PC may be overfitting. The leaveoneout (LOO) error is used to measure the degree of overfitting in practice, which can be expressed as follows:
where h_{i} denotes the ith diagonal element of the matrix Ψ(Ψ^{T}Ψ)^{−1}Ψ^{T}. The E_{LOO} can be calculated easily when the least squares solution w.r.t full design are available (see [27]). We choose the PCE model with the smallest LOO error during the iteration of finding the best sparse solution.
When comparing the performance of different surrogate models, our interest lies in the precision and uncertainty. The relative MSE is a widely used measure to quantify the precision of a surrogate, which is defined by the following equation:
where \(\bar {y}=\frac {1}{n}\sum _{i=1}^{n}y_{i}\). As for the uncertainty, we pay special attention to the probability density function (PDF) or cumulative distribution function (CDF).
The concept of information entropy
The type of entropy addressed in this work is information entropy that was first proposed by Shannon in 1948 [36]. The most general explanation of entropy is the measure of uncertainty, which refers to the information contained in the system [44]. Specifically, the more information the system contains, the smaller the uncertainty and corresponding entropy.
For any random variable X={x_{1},…,x_{n}}, the definition of information entropy is expressed in terms of a discrete set of probabilities P(x_{i}), as follows:
Similarly, for a continuous random variable, differential entropy can be obtained:
Based on information entropy, when predicting a probability model of random variables, the main idea of the MEP is that among all candidate distributions, the distribution that maximizes entropy should be selected. In other words, the best probability distribution has maximal entropy, which comes up with an optimization problem:
where p(y) is the probability density function of element y in Y.
Hence, by combining MEP and the classical sparse PCE techniques, information entropy can be used as a sparse optimization criterion for the sparse PCE model for it can retain the largest amount of uncertainty while making it sparse.
Methodology
In this section, two adaptive methods are proposed. The first is the reoptimization of results achieved with the alternative model based on entropy, which can be thought of as a simple external plugin. The second is hybrid optimization using entropy as a regularizer, which is the main method of this paper and it is more advantageous in highdimensional problems.
Entropybased reoptimization sparse PCE
As mentioned at the end of Section 2.3, the aim of this paper is to learn a sparse model and preserve the maximum uncertainty of the results. Thus, we can directly perform entropybased secondary optimization from Eq. 12 based on existing sparse algorithms (i.e., entropybased reoptimization sparse PCE). Typically, solving problems with the sparse PCE surrogate model can be converted to:
where \(\mathcal {I}\) is the sparse index set.
With the classical sparse PCE method, we get the model response Y_{S}. Additional model responses are produced through the repeated sampling process, which can then be reoptimized based on entropy sorting. Our goal is to select W items with the largest entropy values. The objective function during reoptimization is defined as follows:
where Y_{S} is the model response obtained by the classical sparse method, and index set \(\mathcal {J}\) satisfies:
with \(\mathrm {H}\left (\boldsymbol {Y}_{S}\right)\triangleq \sum _{j=1}^{W}\mathrm {H}(\boldsymbol {y}_{j})\), where y_{j}=Ψ_{j}c_{j}.
Since calculating the differential entropy of Y_{S} is more complicated, it can be approximated by the Shannon entropy of Monte Carlo. For example, if we repeat sampling m times as the verification set, the Shannon information entropy of y_{j} can be obtained:
The pseudo code of the algorithm for entropybased reoptimization sparse PCE is as follows:
The main advantage of this method is that it can be easily implemented and extended to various sparse methods without the need to modify their internal codes. In other words, it is universal, maneuverable, and can be considered as an external plugin. However, it requires longer computation time when dealing with highdimensional cases because of the OLS algorithm used in the reoptimization process.
Hybrid entropybased comprehensive sparse PCE
In Section 3.1, a universal entropybased reoptimization sparse PCE method is proposed, which is essentially a subsequent optimization of the results trained by classical sparse PCE methods. However, its reoptimization is performed with the features and coefficients retained by a sparse algorithm, hence we cannot retrieve features that have been discarded, which may be of great influence for the overall uncertainty. As the complexity of the model increases, the discarded features by the classical sparse method will also increase, so the optimization effectiveness of EntPCE will be limited. Therefore, in different sparse methods, we can consider the role of information entropy when selecting features. In fact, there are several commonly used sparse algorithms in the literature, some of which can be found in opensource toolboxes. In this section, for these specific sparse PCE algorithms, a method involving hybrid entropybased comprehensive sparse PCE will be proposed.
In the PCE model, the unknown coefficients correspond to polynomial chaos basis. When we reduce the dimension of the coefficients, it can be seen as a feature selection for the polynomial chaos basis. From the perspective of regression, we usually select the feature basis that has a greater impact on the residual, such as the OMP method. However, the polynomial chaos basis stores the probabilistic information of the predicted response. From the MEP perspective, the feature basis with larger entropy contains more uncertainty information and is more important to the response.
To maximize the input uncertainty distribution in the response, we introduce the entropy of the basis function to regularize the classical model. Hence, the objective function is redefined as follows:
An example of a hybrid entropybased comprehensive sparse PCE algorithm based on the classical OMP method is Algorithm 2. The suffix indicates the classical sparse PCE algorithm being optimized.
In each iteration, OMP selects the regressor that is most closely correlated with the current residual, while HEntOMP introduces entropy as a penalty term on this basis. The regressor must not only be related to the residual but also ensure that its entropy value is large. From this, we can find that HEntPCE only needs to calculate the entropy of the basis function once at the beginning of feature selection, so the extra computational effort is almost negligible.
It is worth noting that in the methodology, the prefixes ‘Ent’ and ‘HEnt’ indicate optimization routine and the suffixes ‘PCE’ represent universal sparse PCE algorithms. While in the algorithm as well as in the experiments, the suffix is replaced by the specified classical sparse PCE algorithm. Compared with Algorithm 1, this method is more targeted and has faster computing speeds.
However, due to the different classical methods, the selection of regularization parameters in different methods is also worthy of attention.
Parameter optimization and selection
Before conducting experiments, we need to set some parameters for these algorithms. For example, in Algorithm 1, we are concerned with the effects of changing parameter W on the final sparse result. Similarly, when using hybrid entropybased algorithms, γ is a parameter worthy of attention. For each of the various methods used in entropybased algorithms, the choice of parameters will be different.
The selection of parameter W is natural because we can set it directly according to the desired size of the active basis. While regarding the selection of γ, we can use a small size of experiment design in preliminary experiments before conducting the formal experiment, so that γ can get an ideal value. The algorithm is designed as Algorithm 3, the idea of which comes from DIRECT algorithm [45, 46]. The algorithm equates the search interval into k subintervals, and calculates the value of function at the boundary point of the subinterval, then takes the point with the lowest value as the centroid, the length of the subinterval as the length of the interval, thus establishing a new space to repeat the previous process until the value of function is below the set value ε.
Experiment analysis
In the experiments, we analyze and compare the effectiveness of four classical sparse PCE algorithms (OMP [47], LARS [27], SP [48], and BCS [49]) and two entropybased optimized algorithms on the basis of them.
To implement the benchmark, we consider an Ishigami function (d=3), an OAKLEY & O’HAGAN function (d=15) and a highdimensional function (d=54) to show the performance of the methods in lowdimensional and highdimensional cases respectively. Both benchmark functions have been inserted small random interference items to better fit the actual. The LHS sequence is used to generate sample points in the experiment. Each experiment was repeated 50 times, and the mean value is given to compare performance of the used methods. To further prove the effective performance of the proposed method, the approach is applied to a more realistic CFD application: RAE2822 airfoil (d=5). For all experiments, we use the generalpurpose uncertainty quantification software UQLab [35].
Toy model
Before starting the formal experiment, we first consider a toy model to explain the problem of this paper. The classical sparse method only considers the value of coefficients, rather than the impact of feature changes on uncertainty. So we constructed a function that is a combination of smooth function and highorder features.
If classical methods such as OMP are used directly, these highlevel features will be discarded, regardless of whether they contain important information or not. As shown in Table 1 and Fig. 1(a) and (b), using the two methods proposed in this paper to optimize can make the probability distribution closer to the original distribution, while keeping the relative MSR at a small value. Meanwhile, Fig. 1(c) shows the retained basis functions, and the meaning of the label of it is the method (number of retained terms): index among all basis functions. It can be found that EntPCE is sparse on the result of OMP, so the index set of selected features are subsets of the index set of the original result, while HEntPCE takes the role of information entropy into account at the time of feature selection, so some higherorder features that were originally discarded by OMP are retained.
Case 1: the Ishigami function
The Ishigami model is a nonlinear, nonmonotonic smooth threedimensional function:
where x_{1},x_{2}, and x_{3} are three independent input random variables uniformly distributed on [−π,π], and in typical a=7,b=0.1.
We use the entropybased reoptimization method and hybrid entropybased comprehensive method on the basis of other sparse methods for optimization, and the results are shown in Figs. 2, 3, 4, 5.
In (a) and (b), the xaxis is the size of DoE and the yaxis is relative MSE and level of sparsity (number of items retained/number of items in complete PCE) respectively. It can be noticed that both optimization methods can achieve a good degree of optimization in low dimensions. Specifically, relative MSE and level of sparsity are both reduced compared to the classical method when using the same size of DoE. In other words, entropybased methods require fewer sample points when a certain level of precision or sparsity is required. Furthermore, (c) show the PDF using 200 DoE. It can be observed that the optimized PDF is closer to the sample point, which is consistent with the previous theory.
Case 2: the OAKLEY & O’HAGAN function
The OAKLEY & O’HAGAN function was proposed in 2010 [50]. It is a highdimensional function and commonly used in uncertainty quantification and sensitivity analysis, which is expressed as follows:
The independent distributions of the input random variables are \(x_{i}\sim \mathcal {N}(0,1),i=1,\ldots,15\). a_{1},a_{2},a_{3} are 1×15 vectors, and M is a square matrix, and the data are taken directly from [50].
From the results of Relative MSE shown in (a) of Figs. 6, 7, 8, 9, in the case of high dimensionality, both optimization methods can improve on the original classical methods. However, EntPCE can only be slightly optimized based on the existing results, while HEntPCE can achieve better optimization results. This is because EntPCE is a reoptimization method based on the classical sparse PCE. In the lowdimensional case, the lowdimensional features selected by the classical sparse algorithm may be sufficient to represent the overall features of the original model. And when the model dimension is high, the classical sparse method may discard those highdimensional features that affect the overall uncertainty, and we cannot retrieve them in the next reoptimization process. Therefore, HEntPCE is much better than EntPCE in the highdimensional case for optimization. Compared to HEntPCE, EntPCE can control sparsity better, as can be seen in (b). However, in the higherdimensional cases, EntPCE requires a largescale OLS calculation, which can lead to an increase in computation time, and therefore HEntPCE is preferred for the higher dimensional cases where sparsity is not required.
Case 3: the highdimensional function
To demonstrate the performance of HEntPCE in higher dimensional conditions, consider a highdimensional function from UQLab, which is an analytical model of the form:
where \(X_{i} \sim \mathcal {U}([1,2]), i \neq 20\), and \(X_{20} \sim \mathcal {U}([1,3])\).
As methods we proposed are further optimized based on existing sparse PCE model, the performance would be limited for high dimensional cases. For the EntPCE models, since the high order features are dumped by the sparse PCE model, there exists little room for further optimization. As a result, we only compare the results of HEntPCE and classical PCE, as can be seen in Figs. 10, 11, 12, 13. It can be seen that the HEntPCE still outperforms the classical method in loworder settings.
Case 4: RAE2822 airfoil
The final application involves the RAE2822 airfoil, a supercritical airfoil, which is a challenging case used by various researchers [24, 51] to test for quantification of uncertainty. The reference values of uncertainty factors in the test are shown in Table 2. We specify that all random variables are subject to a uniform distribution, with the upper and lower boundaries at a reference value of 1±5%.
For deterministic CFD solutions, the SpalartAllmaras oneequation turbulence model is used for turbulence modeling, and SU2 simulation software is used as a blackbox solver. The model of 5th order, using different sparse PCE methods is constructed with 20 samples. The results (pressure coefficient) were obtained with four classical sparse PCE methods and two improved methods proposed above. The mean, standard deviation, the CDF figure of one random point in the RAE2822 airfoil and relative error are shown in Fig. 14 (OMP), Fig. 15 (LARS), Fig. 16 (SP), Fig. 17 (BCS).
The airfoil has upper and lower surfaces, which are marked in (a). For easier presentation, the results are expanded in (b)(d) with the line x=0 as the dividing line, with the lower surface on the left and the upper surface on the right. For Figs. 14—17, it can be observed that the optimized results are in acceptable agreement with those of the classical sparse models and the mean of the sampling points (shown in (a)). The main difference between classical sparse models and corresponding optimized models is the standard deviation (shown in (b)). Each position will form a standard deviation, because of the sensitivity of the RAE2822 airfoil model. The standard deviations of the classical sparse models are usually greater than the actual results, while optimization can reduce the standard deviation and achieve better results by adjusting the parameters. However, the least squares used in EntPCE will make the standard deviation too small. Moreover, although the dimensionality is not high, one REA2822 case needs to build hundreds of PCE models (each position of the airfoil needs a model, the number is related to the density of grid nodes), which takes longer calculation time and cost for EntPCE. Thus, HEntPCE is a better choice in such a situation. Although the relative error of the mean value remains hardly changed, the relative error of the standard deviation of the optimized results is reduced (shown in (c) and (d)), so the results will be closer to the original distribution. CDF figures were evaluated for the distribution results at a random point on the airfoil (shown in (e)). It can be seen that optimized models greatly improve the distribution of the model based on the classical models, which is in line with the inference made above.
Conclusion
In this paper, an effective framework of sparse PCE is realized and verified. The main contribution of this paper is the development of two adaptive regression methods for optimizing sparse polynomial chaos expansion, EntPCE and HEntPCE. The former is essentially a followedup optimization of the results that have been trained by classical sparse methods, and can be easily implemented and extended to various sparse methods; the latter embeds a penalty term into any known sparse algorithms, and allows important higherorder features that may be discarded by classical sparse methods to be retained.
The advantages of the proposed methods are twofold. First, the regularization method in the classical sparse PCE model is followed, which improves accuracy and reduces sparsity while avoiding overfitting problems. Secondly, considering the uncertainty propagation in the model, the distribution rules of model input are passed, so the output retains uncertainty to the greatest extent.
Furthermore, selection of regularization parameters is changed, because of inconsistencies in the standards for feature selection of different classical sparse PCE models. To achieve the best optimization effect, a set of parameter selection rules was formed. For verification, several applications were considered. Two benchmark analysis functions were initially studied. The results show that the two proposed algorithms can achieve a certain optimization effect. Among them, EntPCE is simpler and more universal in lowdimensional situations. In the case of high dimensions, as HEntPCE can retain important higherorder features and does not need secondary calculation, the results are more accurate and the calculation speed is faster. Additionally, the algorithm effect was analyzed and compared through the wind tunnel application. The experimental results show that EntPCE has greater limitations, but it can be used independently and can easily optimize smallscale models. HEntPCE is more efficient and can achieve higher accuracy, so it is an effective method that is worth being promoted at the industrial level to establish a highquality sparse PCE model.
Availability of data and materials
The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.
Abbreviations
 PCE:

Polynomial Chaos Expansion
 CFD:

Computational Fluid Dynamics
 OLS:

Orthogonal Least Square
 NP:

Nondeterministic Polynomial
 MSE:

Mean Squared Error
 MEP:

Maximum Entropy Principle
 OMP:

Orthogonal Matching Pursuit
 LARS:

Least Angle Regression
 SP:

Subspace Pursuit
 BCS:

Bayesian Compressive Sensing
 DoE:

Design of Experiment
 LOO:

LeaveOneOut
 PDF:

Probability Density Function
 CDF:

Cumulative Distribution Function
 EntPCE:

Entropybased reoptimization sparse PCE
 HEntPCE:

Hybrid entropybased comprehensive sparse PCE
 LHS:

Latin Hypercube Sampling
References
Tang T, Zhou T (2015) Recent developments in high order numerical methods for uncertainty quantification. Sci Sin Math 45(7):891–928.
Hu X, Parks GT, Chen X, Seshadri P (2016) Discovering a onedimensional active subspace to quantify multidisciplinary uncertainty in satellite system design. Adv Space Res 57(5):1268–1279.
Vu KK, d’Ambrosio C, Hamadi Y, Liberti L (2017) Surrogatebased methods for blackbox optimization. Int Trans Oper Res 24(3):393–424.
Dutta S, Gandomi AH (2020) Design of experiments for uncertainty quantification based on polynomial chaos expansion metamodels In: Handbook of probabilistic models, 369–381.
Lüthen N, Marelli S, Sudret B (2021) Sparse polynomial chaos expansions: Literature survey and benchmark. SIAM/ASA J Uncertain Quantif 9:593–649.
Gaspar B, Teixeira AP, Soares CG (2014) Assessment of the efficiency of Kriging surrogate models for structural reliability analysis. Probabilistic Eng Mech 37:24–34.
Zhang L, Lu Z, Pan W (2015) Efficient structural reliability analysis method based on advanced Kriging model. Appl Math Model 39(2):781–793.
Sen O, Gaul NJ, Choi KK, Jacobs G, Udaykumar HS (2017) Evaluation of Kriging based surrogate models constructed from mesoscale computations of shock interaction with particles. J Comput Phys 336:235–260.
Song K, Zhang Y, Zhuang X, Yu X, Song B (2020) An adaptive failure boundary approximation method for reliability analysis and its applications. Eng Comput:1–16.
Dai H, Hao Z, Rasmussen K, Wei W (2015) Wavelet densitybased adaptive importance sampling method. Struct Saf 52:161–169.
Li S, Yang B, Qi F (2016) Accelerate global sensitivity analysis using artificial neural network algorithm: Case studies for combustion kinetic model. Combust Flame 168:53–64.
Tripathy RK, Ilias B (2018) Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification. J Comput Phys 375:565–588.
Bhosekar A, Ierapetritou M (2018) Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Comput Chem Eng 108:250–267.
Xu J, Kong F (2018) A cubature collocation based sparse polynomial chaos expansion for efficient structural reliability analysis. Struct Saf 74:24–31.
Mohammadi A, Raisee M (2019) Efficient uncertainty quantification of stochastic heat transfer problems by combination of proper orthogonal decomposition and sparse polynomial chaos expansion. Int J Heat Mass Transfer 128:581–600.
Tarakanov A, Elsheikh AH (2019) Regressionbased sparse polynomial chaos for uncertainty quantification of subsurface flow models. J Comput Phys 399:108909.
Ghanem RG, Spanos PD (1991) Stochastic finite elements: a spectral approach. SpringerVerlag, New York.
Xiu D, Karniadakis GE (2002) The wiener–askey polynomial chaos for stochastic differential equations. SIAM J Sci Comput 24(2):619–644.
Xiu D, Karniadakis GE (2003) Modeling uncertainty in flow simulations via generalized polynomial chaos. J Comput Phys 187(1):137–167.
Schaefer J, Hosder S, West T, Rumsey C, Carlson JR, Kleb W (2017) Uncertainty quantification of turbulence model closure coefficients for transonic wallbounded flows. AIAA J 55(1):195–213.
Zhang W, Wang X, Yu J, Yan C (2018) Uncertainty quantification analysis in hypersonic aerothermodynamics due to freestream. J Beijing Univ Aeronaut Astronaut 44(5):1102–1109.
Avdonin A, Jaensch S, Silva CF, Češnovar M, Polifke W (2018) Uncertainty quantification and sensitivity analysis of thermoacoustic stability with nonintrusive polynomial chaos expansion. Combust Flame 189:300–310.
Eldred MS, Burkardt J (2009) Comparison of nonintrusive polynomial chaos and stochastic collocation methods for uncertainty quantification In: 47th AIAA Aerospace Sciences Meeting Including The New Horizons Forum and Aerospace Exposition. AIAA 2009976.
Kumar D, Raisee M, Lacor C (2016) An efficient nonintrusive reduced basis model for high dimensional stochastic problems in CFD. Comput Fluids 138:67–82.
Huan Z, Gao Z, Xu F, Zhang Y, Huang J (2019) An efficient adaptive forwardbackward selection method for sparse polynomial chaos expansion. Comput Methods Appl Mech Eng 355:456–491.
Blatman G, Sudret B (2008) Sparse polynomial chaos expansions and adaptive stochastic finite elements using a regression approach. C R Mécanique 336(6):518–523.
Blatman G, Sudret B (2011) Adaptive sparse polynomial chaos expansion based on least angle regression. J Comput Phys 230(6):2345–2367.
Hampton J, Doostan A (2015) Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies. J Comput Phys 280:363–386.
Salehi S, Raisee M, Cervantes MJ, Nourbakhsh A (2017) Efficient uncertainty quantification of stochastic CFD problems using sparse polynomial chaos and compressed sensing. Comput Fluids 154:296–321.
Liu Z, Lesselier D, Sudret B, Wiart J (2020) Surrogate modeling based on resampled polynomial chaos expansions. Reliab Eng Syst Saf 202:107008.
Vermet F (2018) Statistical learning methods In: Big data for insurance companies, Wiley. https://doi.org/10.1002/9781119489368.ch3.
Louizos C, Welling M, Kingma DP (2017) Learning sparse neural networks through l_{0} regularization. arXiv preprint arXiv:1712.01312.
Liu Z, Sun F, McGovern DP (2017) Sparse generalized linear model with L_{0} approximation for feature selection and prediction with big omics data. Biodata Mining 10:39.
Jakeman JD, Eldred MS, Sargsyan K (2015) Enhancing l_{1}minimization estimates of polynomial chaos expansions using basis selection. J Comput Phys 289:18–34.
Marelli S, Sudret B (2014) UQLab: a framework for uncertainty quantification in MATLAB In: SIAM Conference on Uncertainty Quantification, 2554–2563.
Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423.
Jaynes ET (1957) Information theory and statistical mechanics. Phys Rev 108(2):171–190.
Grechuk B, Zabarankin M (2019) Regression analysis: likelihood, error and entropy. Math Program 174:145–166.
Wang X, Tao D, Zhu L (2010) Entropy controlled laplacian regularization for least square regression. Sign Process 90(6):2043–2049.
Bai L, Liang J (2020) Sparse subspace clustering with entropynorm In: International Conference on Machine Learning (ICML 2020).
Obuchi T, NakanishiOhno Y, Okada M, Kabashima Y (2018) Statistical mechanical analysis of sparse linear regression as a variable selection problem. J Stat Mech Theory Exp 2018(10):103401.
Murari A, Peluso E, Cianfrani F, Gaudio P, Lungaroni M (2019) On the use of entropy to improve model selection criteria. Entropy 21(4):394.
Bruckstein AM, Donoho DL, Elad M (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51(1):34–81.
Tribus M, McIrvine EC (1971) Energy and information. Sci Am 224:178–184.
Jones DR (2008) Direct global optimization algorithm. In: Floudas CA Pardalos PM (eds)Encyclopedia of optimization.. Springer, Boston.
Jones DR, Martins J (2021) The DIRECT algorithm–25 years later. J Glob Optim 79:521–566.
Rezaiifar YCPR, Krishnaprasad PS (2002) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition In: Conference on Signals, Systems & Computers.
Wei D, Milenkovic O (2009) Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans Inf Theory 55(5):2230–2249.
Sargsyan K, Safta C, Najm HN, Debusschere BJ, Ricciuto D, Thornton P (2014) Dimensionality reduction for complex models via bayesian compressive sensing. Int J Uncertain Quantif 4(1):63–93.
Oakley JE, O’Hagan A (2010) Probabilistic sensitivity analysis of complex models: a bayesian approach. J R Stat Soc 66(3):751–769.
Witteveen J, Doostan A, Pecnik R, Iaccarino D (2009) Uncertainty quantification of the transonic flow around the RAE 2822 airfoil. Annual Research Briefs, Center for Turbulence Research, Stanford University. pp 93–104.
Acknowledgements
The authors are thankful to the reviewers for their valuable comments to improve the quality of the manuscript. In addition, we are grateful to Mingze Qi for his constructive suggestion about paper writing and organization.
Funding
This work was funded by the National Numerical Wind tunnel project (NNW2019ZT7B23) and National Natural Science Foundation of China (1771450). The authors gratefully acknowledge this funding.
Author information
Authors and Affiliations
Contributions
All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zeng, S., Duan, X., Chen, J. et al. Optimized sparse polynomial chaos expansion with entropy regularization. Adv. Aerodyn. 4, 3 (2022). https://doi.org/10.1186/s42774021000956
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s42774021000956
Keywords
 Uncertainty quantification
 Sparse polynomial chaos expansion
 Entropy
 Regularization