PhD Student at Stanford University. I try to make abstract nonsense less abstract and more sensible.
Extracting explainable, exploitable insight from engineering systems
Dimension reduction techniques seek to make complicated systems more intuitive by finding a simpler representation. These are automated, data-driven procedures. Some examples include visualization tools (such as t-SNE) and input parameter space reduction (such as Active Subspaces). However, these techniques are often divorced from the physics. They lack a physical interpretation; a cynic might say they're abstract nonsense.
My work seeks to ameliorate this issue by imposing constraints on such dimension reduction techniques, leading to representations which are both simple, and have a powerful physical interpretation. I call this design insight -- an explainable, exploitable understanding suitable for doing engineering design.
Unfortunately, Sparrho doesn't seem to index AIAA conference papers. The main paper to read is Developing Design Insight Through Active Subspaces, which details the following results.
Thin Airfoil Theory
Can recover the dependence of lift on the camber line from data alone. This can aid conceptual airfoil design.
Can identify unique and relevant scale-invariant parameters which govern a physical system. This leads to intuitive, physical understanding of a problem.
Lurking Variable Detection
Can detect important missing variables based on fundamental physical constraints -- so-called hidden parameters or lurking variables. This allows analysts to diagnose issues in their models.
Since design insight is a relatively new idea, there are not many papers strictly addressing it. The pinned manuscripts here are either similar in spirit, or directly applicable to this research topic.
Abstract: Inexpensive surrogates are useful for reducing the cost of science and engineering studies with large-scale computational models that contain many input parameters. A ridge approximation is a surrogate distinguished by its model form: namely, a nonlinear function of a few linear combinations of the input parameters. Parameter studies (e.g., optimization or uncertainty quantification) with ridge approximations can exploit its low-dimensional structure by working on the coordinates of the subspace defined by the linear combination weights, reducing the effective dimension. We introduce a new, fast algorithm for constructing a least-squares-fit polynomial ridge approximation from function samples. Naively, this would require optimizing both the polynomial coefficients and the subspace. However, given a fixed subspace the optimal polynomial coefficients solve a linear least-squares problem. Our proposed method exploits this structure by implicitly computing these coefficients using variable projection, which leaves an optimization problem over the subspace alone. We present an algorithm that finds the optimal subspace by optimizing over the Grassmann manifold using a Gauss-Newton Hessian approximation. We provide the details of the optimization algorithm, and we demonstrate its performance on several numerical examples. The Gauss-Newton method has superior theoretical guarantees and faster convergence than an alternating heuristic for ridge approximation proposed by Constantine, Eftekhari, and Ward [arXiv 1606.01929] that (i) optimizes the polynomial coefficients given the subspace and (ii) optimizes the subspace given the coefficients.
Pub.: 20 Feb '17, Pinned: 30 Jun '17
Abstract: We present brief précis of three related investigations. Fuller accounts can be found elsewhere. The investigations bear on the identification and prediction of coherent structures in turbulent shear flows. A second unifying thread is the Proper Orthogonal Decomposition (POD), or Karhunen-Loève expansion, which appears in all three investigations described. The first investigation demonstrates a close connection between the coherent structures obtained using linear stochastic estimation, and those obtained from the POD. Linear stochastic estimation is often used for the identification of coherent structures. The second investigation explores the use (in homogeneous directions) of wavelets instead of Fourier modes, in the construction of dynamical models; the particular problem considered here is the Kuramoto-Sivashinsky equation. The POD eigenfunctions, of course, reduce to Fourier modes in homogeneous situations, and either can be shown to converge optimally fast; we address the question of how rapidly (by comparison) a wavelet representation converges, and how the wavelet-wavelet interactions can be handled to construct a simple model. The third investigation deals with the prediction of POD eigenfunctions in a turbulent shear flow. We show that energy-method stability theory, combined with an anisotropic eddy viscosity, and erosion of the mean velocity profile by the growing eigenfunctions, produces eigenfunctions very close to those of the POD, and the same eigenvalue spectrum at low wavenumbers.
Pub.: 01 Dec '94, Pinned: 29 Jun '17
Abstract: t-distributed Stochastic Neighborhood Embedding (t-SNE), a clustering and visualization method proposed by van der Maaten & Hinton in 2008, has rapidly become a standard tool in a number of natural sciences. Despite its overwhelming success, there is a distinct lack of mathematical foundations and the inner workings of the algorithm are not well understood. The purpose of this paper is to prove that t-SNE is able to recover well-separated clusters; more precisely, we prove that t-SNE in the `early exaggeration' phase, an optimization technique proposed by van der Maaten & Hinton (2008) and van der Maaten (2014), can be rigorously analyzed. As a byproduct, the proof suggests novel ways for setting the exaggeration parameter $\alpha$ and step size $h$. Numerical examples illustrate the effectiveness of these rules: in particular, the quality of embedding of topological structures (e.g. the swiss roll) improves. We also discuss a connection to spectral clustering methods.
Pub.: 08 Jun '17, Pinned: 29 Jun '17
Abstract: In many scientific fields, empirical models are employed to facilitate computational simulations of engineering systems. For example, in fluid mechanics, empirical Reynolds stress closures enable computationally-efficient Reynolds Averaged Navier Stokes simulations. Likewise, in solid mechanics, constitutive relations between the stress and strain in a material are required in deformation analysis. Traditional methods for developing and tuning empirical models usually combine physical intuition with simple regression techniques on limited data sets. The rise of high performance computing has led to a growing availability of high fidelity simulation data. These data open up the possibility of using machine learning algorithms, such as random forests or neural networks, to develop more accurate and general empirical models. A key question when using data-driven algorithms to develop these empirical models is how domain knowledge should be incorporated into the machine learning process. This paper will specifically address physical systems that possess symmetry or invariance properties. Two different methods for teaching a machine learning model an invariance property are compared. In the first method, a basis of invariant inputs is constructed, and the machine learning model is trained upon this basis, thereby embedding the invariance into the model. In the second method, the algorithm is trained on multiple transformations of the raw input data until the model learns invariance to that transformation. Results are discussed for two case studies: one in turbulence modeling and one in crystal elasticity. It is shown that in both cases embedding the invariance property into the input features yields higher performance at significantly reduced computational training costs.
Pub.: 06 May '16, Pinned: 29 Jun '17
Abstract: Most engineering models contain several parameters, and the map from input parameters to model output can be viewed as a multivariate function. An active subspace is a low-dimensional subspace of the space of inputs that explains the majority of variability in the function. Here we describe a quick check for a dominant one-dimensional active subspace based on computing a linear approximation of the function. The visualization tool presented here is closely related to regression graphics, though we avoid the statistical interpretation of the model. This document will be part of a larger review paper on active subspace methods.
Pub.: 16 Feb '14, Pinned: 29 Jun '17
Abstract: A ridge function is a function of several variables that is constant along certain directions in its domain. Using classical dimensional analysis, we show that many physical laws are ridge functions; this fact yields insight into the structure of physical laws and motivates further study into ridge functions and their properties. We also connect dimensional analysis to modern subspace-based techniques for dimension reduction, including active subspaces in deterministic approximation and sufficient dimension reduction in statistical regression.
Pub.: 25 May '16, Pinned: 29 Jun '17
Join Sparrho today to stay on top of science
Discover, organise and share research that matters to you