A pinboard by
Zachary del Rosario

PhD Student at Stanford University. I try to make abstract nonsense less abstract and more sensible.


Extracting explainable, exploitable insight from engineering systems

Dimension Reduction

Dimension reduction techniques seek to make complicated systems more intuitive by finding a simpler representation. These are automated, data-driven procedures. Some examples include visualization tools (such as t-SNE) and input parameter space reduction (such as Active Subspaces). However, these techniques are often divorced from the physics. They lack a physical interpretation; a cynic might say they're abstract nonsense.

Seeking Insight

My work seeks to ameliorate this issue by imposing constraints on such dimension reduction techniques, leading to representations which are both simple, and have a powerful physical interpretation. I call this design insight -- an explainable, exploitable understanding suitable for doing engineering design.

Previous Work

Unfortunately, Sparrho doesn't seem to index AIAA conference papers. The main paper to read is Developing Design Insight Through Active Subspaces, which details the following results.

  • Thin Airfoil Theory

    Can recover the dependence of lift on the camber line from data alone. This can aid conceptual airfoil design.

  • Dimensionless Parameters

    Can identify unique and relevant scale-invariant parameters which govern a physical system. This leads to intuitive, physical understanding of a problem.

  • Lurking Variable Detection

    Can detect important missing variables based on fundamental physical constraints -- so-called hidden parameters or lurking variables. This allows analysts to diagnose issues in their models.

Relevant Papers

Since design insight is a relatively new idea, there are not many papers strictly addressing it. The pinned manuscripts here are either similar in spirit, or directly applicable to this research topic.


Data-driven polynomial ridge approximation using variable projection

Abstract: Inexpensive surrogates are useful for reducing the cost of science and engineering studies with large-scale computational models that contain many input parameters. A ridge approximation is a surrogate distinguished by its model form: namely, a nonlinear function of a few linear combinations of the input parameters. Parameter studies (e.g., optimization or uncertainty quantification) with ridge approximations can exploit its low-dimensional structure by working on the coordinates of the subspace defined by the linear combination weights, reducing the effective dimension. We introduce a new, fast algorithm for constructing a least-squares-fit polynomial ridge approximation from function samples. Naively, this would require optimizing both the polynomial coefficients and the subspace. However, given a fixed subspace the optimal polynomial coefficients solve a linear least-squares problem. Our proposed method exploits this structure by implicitly computing these coefficients using variable projection, which leaves an optimization problem over the subspace alone. We present an algorithm that finds the optimal subspace by optimizing over the Grassmann manifold using a Gauss-Newton Hessian approximation. We provide the details of the optimization algorithm, and we demonstrate its performance on several numerical examples. The Gauss-Newton method has superior theoretical guarantees and faster convergence than an alternating heuristic for ridge approximation proposed by Constantine, Eftekhari, and Ward [arXiv 1606.01929] that (i) optimizes the polynomial coefficients given the subspace and (ii) optimizes the subspace given the coefficients.

Pub.: 20 Feb '17, Pinned: 30 Jun '17

The Proper Orthogonal Decomposition, wavelets and modal approaches to the dynamics of coherent structures

Abstract: We present brief précis of three related investigations. Fuller accounts can be found elsewhere. The investigations bear on the identification and prediction of coherent structures in turbulent shear flows. A second unifying thread is the Proper Orthogonal Decomposition (POD), or Karhunen-Loève expansion, which appears in all three investigations described. The first investigation demonstrates a close connection between the coherent structures obtained using linear stochastic estimation, and those obtained from the POD. Linear stochastic estimation is often used for the identification of coherent structures. The second investigation explores the use (in homogeneous directions) of wavelets instead of Fourier modes, in the construction of dynamical models; the particular problem considered here is the Kuramoto-Sivashinsky equation. The POD eigenfunctions, of course, reduce to Fourier modes in homogeneous situations, and either can be shown to converge optimally fast; we address the question of how rapidly (by comparison) a wavelet representation converges, and how the wavelet-wavelet interactions can be handled to construct a simple model. The third investigation deals with the prediction of POD eigenfunctions in a turbulent shear flow. We show that energy-method stability theory, combined with an anisotropic eddy viscosity, and erosion of the mean velocity profile by the growing eigenfunctions, produces eigenfunctions very close to those of the POD, and the same eigenvalue spectrum at low wavenumbers.

Pub.: 01 Dec '94, Pinned: 29 Jun '17

Machine learning strategies for systems with invariance properties

Abstract: In many scientific fields, empirical models are employed to facilitate computational simulations of engineering systems. For example, in fluid mechanics, empirical Reynolds stress closures enable computationally-efficient Reynolds Averaged Navier Stokes simulations. Likewise, in solid mechanics, constitutive relations between the stress and strain in a material are required in deformation analysis. Traditional methods for developing and tuning empirical models usually combine physical intuition with simple regression techniques on limited data sets. The rise of high performance computing has led to a growing availability of high fidelity simulation data. These data open up the possibility of using machine learning algorithms, such as random forests or neural networks, to develop more accurate and general empirical models. A key question when using data-driven algorithms to develop these empirical models is how domain knowledge should be incorporated into the machine learning process. This paper will specifically address physical systems that possess symmetry or invariance properties. Two different methods for teaching a machine learning model an invariance property are compared. In the first method, a basis of invariant inputs is constructed, and the machine learning model is trained upon this basis, thereby embedding the invariance into the model. In the second method, the algorithm is trained on multiple transformations of the raw input data until the model learns invariance to that transformation. Results are discussed for two case studies: one in turbulence modeling and one in crystal elasticity. It is shown that in both cases embedding the invariance property into the input features yields higher performance at significantly reduced computational training costs.

Pub.: 06 May '16, Pinned: 29 Jun '17